Checking PolitiFact's Fact-Checks

Abstract

In this article we examine PolitiFact's fact-checking process. We collect a random sample of 858 fact-checks and evaluate them in the light of criteria based on or inspired by fact-checking literature and the International Fact-checking Network's code of principles. Our analysis reveals the following: in general, PolitiFact fared well. However, from the point of view of the criteria, its practices leave room for improvement. The biggest issue is complex propositions. These are statements containing multiple claims, i.e., more than one proposition. In 279 cases (33% of our sample), PolitiFact checks a complex proposition and assigns one truth rating to it. This is problematic as the reader might misinterpret the truthfulness of an individual claim. PolitiFact also checks claims that we considered uncheckable. These are statements whose truthfulness cannot be defined in practice, e.g., claims about the future and vague claims. In 92 cases (11% of our sample), PolitiFact checked a claim like this. The article ends with a discussion about the limitations of the criteria used here.