Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News
| Author | Nakov, Preslav |
| Author | Da San Martino, Giovanni |
| Author | Elsayed, Tamer |
| Author | Barrón-Cedeño, Alberto |
| Author | Míguez, Rubén |
| Author | Shaar, Shaden |
| Author | Alam, Firoj |
| Author | Haouari, Fatima |
| Author | Hasanain, Maram |
| Author | Mansour, Watheq |
| Author | Hamdan, Bayan |
| Author | Ali, Zien Sheikh |
| Author | Babulkov, Nikolay |
| Author | Nikolov, Alex |
| Author | Shahi, Gautam Kishore |
| Author | Struß, Julia Maria |
| Author | Mandl, Thomas |
| Author | Kutlu, Mucahid |
| Author | Kartal, Yavuz Selim |
| Available date | 2024-02-20T11:38:01Z |
| Publication Date | 2021-09-14 |
| Publication Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
| Identifier | http://dx.doi.org/10.1007/978-3-030-85251-1_19 |
| Citation | Nakov, P., Da San Martino, G., Elsayed, T., Barrón-Cedeño, A., Míguez, R., Shaar, S., ... & Kartal, Y. S. (2021). Overview of the CLEF–2021 CheckThat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21–24, 2021, Proceedings 12 (pp. 264-291). Springer International Publishing. |
| ISBN | 978-3-030-85250-4 |
| ISSN | 0302-9743 |
| Abstract | We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively. |
| Sponsor | This work is made possible by: - NPRP grant #NPRP-11S-1204-170060 from the Qatar National Research Fund (a member of Qatar Foundation). - GSRA grant #GSRA6-1-0611-19074 from the Qatar National Research Fund. |
| Language | en |
| Publisher | Springer Science and Business Media Deutschland GmbH |
| Subject | Check-worthiness estimation COVID-19 Disinformation Fact-checking Fake news detection Misinformation Verified claim retrieval |
| Type | Conference Paper |
| Pagination | 264-291 |
| Volume Number | 12880 LNCS |
| EISBN | 978-3-030-85251-1 |
Files in this item
| Files | Size | Format | View |
|---|---|---|---|
|
There are no files associated with this item. |
|||
This item appears in the following Collection(s)
-
Computer Science & Engineering [2202 items ]
Related items
Showing items related by title, author, creator and subject.
-
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
( Springer Science and Business Media Deutschland GmbH , 2020 , Conference Paper)We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification ... -
Overview of CheckThat! 2020 English: Automatic Identification and Verification of Claims in Social Media
( CEUR-WS , 2020 , Conference Paper)We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in Arabic and English, and here we focus on the three English tasks. Task 1 challenged the participants to predict ... -
Overview of the CLEF-2021 CheckThat! Lab Task 1 on check-worthiness estimation in tweets and political debates
( CEUR-WS , 2021 , Conference Paper)We present an overview of Task 1 of the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The task asks to predict which posts in a Twitter stream are worth ...