By Sheldon Greaves

Note: This is the first in a set of articles on identifying, vetting, and using accurate news sources. Stay tuned.

News is a vital nutrient of the body politic. Without accurate information, democratic societies cannot survive, let alone produce the “alert and knowledgeable citizenry” necessary for their survival. It is no exaggeration to claim that misinformation and disinformation are an existential threat to the nation; they are in fact actively employed by our enemies to undermine and weaken democratic institutions around the globe. Because of this pandemic of “fake news”, citizens must assume the responsibility of vetting and carefully screening their daily diet of information if they wish to be fully informed on the issues of the day (Granted, not everyone wants to be accurately informed, but that’s a different discussion).

The burden of the intelligence analyst, which is a constant awareness that someone is trying to deceive them, has become the burden of all.

See also: “The Spook’s Home Companion: A Guide to Practical DIY Intelligence Analysis”

Bias vs. Accuracy

A valid question, one with implications far beyond this post, is the degree to which the political bias of a news source predicts the quality of information is publishes. For example, voices on the political right have claimed a “liberal bias” in the media, even though recent studies show is not the case. Liberal critics of news outlets such as Fox News tend to focus more on the accuracy of news than its political slant. But the question remains: does the political bias of a news outlet offer a clue to accuracy of that outlet?

The web site Media Bias Fact Check (mediabiasfactcheck.com) is an excellent and highly-respected tool for tracking the accuracy and political bias of hundreds of news sources, most of them on the Internet. Political bias categories are: Left Bias, Left-Center Bias, Least Bias, Right-Center Bias, and Right Bias. Factual categories are: Low, Mixed, Mostly Factual, High, and Very High. Each entry usually includes details explaining why a particular site is rated as it is. They show their work.

MBFC has an additional category, Questionable Sources, which includes ratings such as Low, Very Low, Pseudoscience, Satire, etc. It also includes bias categories such as Extreme Left and Extreme Right. However, their search tool does not allow one to search for search for Questionable Sources based on extreme bias categories. Studying these sources required some improvisation. More on this later.

See also: “Dodging Deception: Tips From the Pros”

The site is easy to use, frequently updated, and free. It should be the first stop for anyone who encounters a news source and wants to see how it compares with other sources. MBFC not only rates sources by their political bias (if any), but has a separate scale for factual reporting. This is an important distinction, too often ignored (see above).  Over the past week, I have used the search tools at MBFC to collect data in order to group different news sources based on their political bias versus factual accuracy.

Essential Findings:

• Political bias of a news site is a useful indicator or overall accuracy and factual reporting.
• Sources rated Least Bias to Left are significantly more factual than Right-Center to Right.
• Among “Questionable Sources” (see below) with an indicated political bias, the overwhelming majority have Extreme Right bias.

The Process

I started by using the search tool to generate lists all the sites of a given political leaning, with a separate list for each level of accuracy, and counted how many sites were on each list. This gave me the following table:

Right	0	175	21	18	0
Center-Right	0	11	18	122	0
Least Bias	0	2	8	244	36
Center-Left	0	21	10	313	6
Left	1	79	34	120	0
	Low	Mixed	Mostly Factual	High	Very High

Using Excel, I took the table and created the following graph:

Note that on Mixed sources, Right Bias is roughly double that of Left Bias. As one moves in the direction of greater accuracy, Lease Bias and left-leaning sources overwhelmingly outnumber those on the right, with the exception of Right-Center which is about the same as Left for High accuracy.

Where Did “Low” Go?

However, note also that almost no sources at all show up on Low accuracy. This requires some explanation. As mentioned earlier, the MBFC database includes some categories for bias (Extreme Left and Extreme Right) and accuracy that aren’t available in the search tool menus. There is a larger category, “Questionable Sources”, that covers Low, Very Low, Pseudoscience, etc. However, it doesn’t let you cross-search Questionable Sources by bias. Any QS search always brings up the same list regardless of what bias you select as a secondary search term. That list of results also does not show political bias in the results list; you have to click through each entry to find that.

When I searched for Questionable Sources, MBFC returned a list of 557 sources. Not relishing looking at all of them one by one, I used a random number generator to get a list of 60 numbers from 1 to 557, and used this to select a random sample of 60 sites. I then clicked through each one and noted the political bias, if any. Of all the sites sampled, six did not list a political bias and were excluded from the sample, leaving 54 sources.

Political Leanings of Questionable Sources

The breakdown strongly suggests that radical conservative sites make up nearly all of the questionable sources online. Sites tagged as “Extreme Bias” made up 79.6% of Questionable Sources in the sample. Total right-leaning sources made up 90.7%. By contrast, left-leaning sites made up less than 9.3% of Questionable Sources.

Comments and Conclusions

For many, this will not come as a surprise. There are plenty of right-wing sources such as InfoWars, or pundits such as Rush Limbaugh, who are so systemically wrong that they sometimes achieve self-parody. What did surprise me, however, was how completely the craziest regions of the infosphere are dominated by right wing sources, and extreme ones at that.

But for the rest of the spectrum of media accuracy, the ratings show that as one moves further to the left, the accuracy of news sources improves dramatically. Granted the best sources are those marked Least Bias, but they have plenty of company from left-leaning sites.

In retrospect, this makes sense, given something I mentioned earlier. Media critics tend to evaluate media sources based on political leanings (e.g., the dreaded “liberal media” that doesn’t actually exist), while those on the left pay more attention to accuracy. It turns out that those are different objectives; right wing news sources such as Fox News are openly about propaganda, to name just one example. It also reflects the trend of “ideology/party over country” that has come to dominate the politics of the American right. In that sense, perhaps these results are not too surprising.

It is encouraging that there are a fair few Center-Right sources that rate as High for accuracy; we need them and any discriminating news consumer’s source deck should include some.

Obviously, this is a rough experiment, little more than a back-of-the-envelope exercise. I would welcome seeing others try a similar exercise with other fact checking sites to see how those results compare with mine.

Sadly, however, it is difficult to escape the sense that conservative media has become so decoupled from reality that it is actually easier to dismiss it out of hand than take its assertions seriously. This is not a good thing. It not only reinforces the alternate reality in which too many Americans live, it tempts to intellectual laziness outside its sphere of influence. Any vibrant discussion needs different voices. But those voices must bring something meaningful to the conversation that will advance it in a useful, productive direction.