Dataset Input #Inputs Evidence Verdict Sources Lang
CrimeVeri (Bachenko et al., 2008) Statement 275 7 2 Classes Crime En
Politifact (Vlachos and Riedel, 2014) Statement 106 Text/Meta 5 Classes Fact Check En
StatsProperties (Vlachos and Riedel, 2015) Statement 7,092 KG Numeric Internet En
Emergent (Ferreira and Vlachos, 2016) Statement 300 Text 3 Classes Emergent En
CreditAssess(Popat et al., 2016) Statement 5,013 Text 2 Classes Fact Check/Wiki En
PunditFact (Rashkin et al., 2017) Statement 4,361 7 2/6 Classes Fact Check En
Liar (Wang, 2017) Statement 12,836 Meta 6 Classes Fact Check En
Verify (Baly et al., 2018) Statement 422 Text 2 Classes Fact Check Ar/En
CheckThat18-T2 (Barrón-Cedeño et al., 2018) Statement 150 7 3 Classes Transcript En
Snopes (Hanselowski et al., 2019) Statement 6,422 Text 3 Classes Fact Check En
MultiFC (Augenstein et al., 2019) Statement 36,534 Text/Meta 2-27 Classes Fact Check En
Climate-FEVER (Diggelmann et al., 2020) Statement 1,535 Text 4 Classes Climate En
SciFact (Wadden et al., 2020) Statement 1,409 Text 3 Classes Science En
PUBHEALTH (Kotonya and Toni, 2020b) Statement 11,832 Text 4 Classes Fact Check En
X-Fact (Gupta and Srikumar, 2021) Statement 31,189 Text 7 Classes Fact Check Many
cQA (Mihaylova et al., 2018) Answer 422 Meta 2 Classes Forum En
AnswerFact (Zhang et al., 2020) Answer 60,864 Text 5 Classes Amazon En
NELA (Horne et al., 2018) Article 136,000 7 2 Classes News En
BuzzfeedNews (Potthast et al., 2018) Article 1,627 Meta 4 Classes Facebook En
BuzzFace (Santia and Williams, 2018) Article 2,263 Meta 4 Classes Facebook En
FA-KES (Salem et al., 2019) Article 804 7 2 Classes VDC En
FakeNewsNet (Shu et al., 2020) Article 23,196 Meta 2 Classes Fact Check En
FakeCovid (Shahi and Nandini, 2020) Article 5,182 7 2 Classes Fact Check Many
Table 2: Summary of factual verification datasets with natural inputs. KG denotes knowledge graphs. ChectThat18
has been extended later (Hasanain et al., 2019; Barrón-Cedeño et al., 2020; Nakov et al., 2021). NELA has been
updated by adding more data from more diverse sources (Nørregaard et al., 2019; Gruppi et al., 2020, 2021)
Next, we discuss the inputs to factual verifica-
tion. The most popular type of input to verifi-
cation is textual claims, which is expected given
they are often the output of claim detection. These
tend to be sentence-level statements, which is
a practice common among fact-checkers in or-
der to include only the context relevant to the
claim (Mena, 2019). Many existing efforts (Vla-
chos and Riedel, 2014; Wang, 2017; Hanselowski
et al., 2019; Augenstein et al., 2019) constructed
datasets by crawling real-world claims from ded-
icated websites (e.g. Politifact) due to their avail-
ability (see Table 2). Unlike previous work that
focus on English, Gupta and Srikumar (2021) col-
lected non-English claims from 25 languages.
Others extract claims from specific domains,
such as science (Wadden et al., 2020), cli-
mate (Diggelmann et al., 2020), and public
health (Kotonya and Toni, 2020b). Alternative
forms of sentence-level inputs, such as answers
from question answering forums, have also been
considered (Mihaylova et al., 2018; Zhang et al.,
2020). There have been approaches that consider
a passage (Mihalcea and Strapparava, 2009; Pérez-
Rosas et al., 2018) or an entire article (Horne et al.,
2018; Santia and Williams, 2018; Shu et al., 2020)
as input. However, the implicit assumption that
every claim in it is either factually correct or in-
correct is problematic, and thus rarely practised by
human fact-checkers (Uscinski and Butler, 2013).
In order to better control the complexity of
the task, efforts listed in Table 3 created claims
artificially. Thorne et al. (2018a) had annota-
tors mutate sentences from Wikipedia articles to
create claims. Following the same approach,
Khouja (2020) and Nørregaard and Derczynski
(2021) constructed Arabic and Danish datasets
respectively. Another frequently considered op-
tion is subject-predicate-object triples, e.g. (Lon-
don, city_in, UK). The popularity of triples as in-
put stems from the fact that they facilitate fact-
checking against knowledge bases (Ciampaglia
et al., 2015; Shi and Weninger, 2016; Shiralkar
et al., 2017; Kim and Choi, 2020) such as DB-
pedia (Auer et al., 2007), SemMedDB (Kilicoglu
et al., 2012), and KBox (Nam et al., 2018). How-
ever, such approaches implicitly assume the non-
trivial conversion of text into triples.
3.2 Evidence
A popular type of evidence often considered is
metadata, such as publication date, sources, user
profiles, etc. Metadata provides insights that are
useful for claim detection, for example, domain-
specific metadata such as likes, or numbers of re-
posts. It offers information complementary to tex-
tual sources or structural knowledge, especially
when the latter are unavailable (Wang, 2017; Pot-