In this age of “fake news” and “alternative facts,” how do we know that so-called fact-checking sources

(snopes.com, politifact) are accurate and honest arbiters of what is true and what is false?

Dr. Julian Bashir: You know, I still have a lot of questions to ask you about your past.

Elim Garak: I have given you all the answers I'm capable of.

Bashir: You've given me answers, all right; but they were all different. What I want to know is, out of all
the stories you told me, which ones were true and which ones weren't?

Garak: (slightly agahst) My dear Doctor, they're all true!

Bashir: Even the lies?

Garak: Especially the lies!

- Star Trek: Deep Space Nine, “The Wire”

Back when I used to teach high school language arts, I did a unit with my sophomores on critical thinking
and source evaluation that focused on essentially this very question.

Not specifically Snopes or Politifact, but all sources.

How do you know that what you’re reading or viewing is credible and accurate?

I started the unit by asking how many of my students believed in the Grand Canyon. All of them raised
their hands. I asked them how many had been to the Grand Canyon. Usually a few would keep their
hands up, no more than three or so. I’d ask how much of it they had viewed. Whether they measured its
depth or length personally.
By the time we were done, no student could ever definitively say for absolute sure that the Grand
Canyon, precisely as described in scientific literature, existed for sure.

Then I’d ask the students to find some event that they had all witnessed personally, usually something
recent. I’d have them write down as many details as possible personally, and then go start comparing on
the board.

They’d start to notice inconsistencies between themselves. “Why?” I would ask, while hiding a smile.
They were all there, right? Why is it that they remember it different? What’s the truth?

This answer will be long. There will not be a TL;DR version.

Pretty much everything in source evaluation comes down to credibility. You have to decide whether or
not you trust that source. You have to decide whether the scientists who measured the length, depth,
and width of the Grand Canyon were honest about it. You have to trust that any student had an
accurate perception of an event.

There’s just one problem with that trust: human beings really suck at intuiting who to trust.

We all routinely tend to overestimate our abilities to determine whether or not someone is telling the
truth. We are very prone to disbelieving facts that do not correlate with our inbuilt implicit biases, or
which challenge our core beliefs. And likewise, we are prone to automatically accept information that
verifies our core beliefs without challenging it.

Some people are more prone to this than others.

This is why eyewitnesses to a crime are actually surprisingly untrustworthy. Our very memories are not
necessarily as reliable as we’d like to believe.
So, who should you trust? And what factors tell you that you can you trust them?

There are some important things to think about whenever evaluating sources.

(1) Do I want to believe it?

Ask yourself, do I want to believe this? Do I not want to believe this? Where are you predisposed?

Is this another story that confirms your fears about Killary having those with evidence of her wrongdoing
in Benghazi knocked off? Is this a story about some new horrible thing that Trump tweeted at 2:30 AM?
Do you immediately dismiss it as nonsense because it says something unflattering about something you
believe?

How do you feel about the story, right from the title?

The stronger you feel about the story, the more critically you should vet it, either way.

This is because of confirmation bias.

I’ll talk more about biases in general later, but I want to address this specifically as a gateway question
in source evaluation.

Confirmation bias is the tendency to accept information that corroborates our worldviews and reject
information that is dissonant to our worldviews. The Oatmeal has an excellent strip , which is how
confirmation bias leads people to entrench themselves more deeply in a belief when presented with
contrary facts.
Confirmation bias is the single biggest reason that fake news goes viral. People read something, it agrees
with their already-held views, and without even questioning it, they pass it along. Everyone that also
already agrees with those views sees that, nods their head, and without questioning it, passes it along.

People don’t do that when it violates their worldviews.

That’s why the first check for evaluating any source ought to be “do I want this to be true or false.”

Being aware of your own predisposition is critical in actively evaluating a source, because it can lead you
to overlook or place undue emphasis on other information that disagrees with or corroborates the
source.

Intelligence professionals have this hammered into them constantly. It is very tempting to get a piece of
intel and then only look to confirm it or dispute it - even when, especially when they don’t know it’s
what they’re doing. It’s a great way to get people killed.

This is how the United States got into the Second Gulf War in Iraq: when the intelligence community
didn’t tell the administration what it wanted to hear, it created a specialized task force within the
military to circumvent the rest of the intelligence community and find enough evidence to support what
the administration wanted.

Attorneys like me also have this hammered into them. It’s very tempting to just look for case law or
facts that support the client’s position. It affects the very way we construct searches for those pieces of
information, and it can cause us to overlook contrary law or facts, and then get ambushed with it. Good
attorneys learn to think antagonistically to their own case and look for ways to destroy their own
positions, so they can defend against them. That requires being very aware of our own confirmation
biases. (One mentor attorney I know routinely plays chess against himself to train his brain to attack his
own preferred positions.)

Before moving on to any other steps of source evaluation, this is the first and most critical one. Don’t let
the wish be the father of the thought.
(2) Levels of sources: primary, secondary, tertiary.

The next most important places to start with source evaluation is checking how close to the original fact
reporting the source is.

Primary sources are sources that have direct, first-hand knowledge.

Some examples of primary sources would be eyewitnesses, autobiographies, diaries, or photographs

that can be authenticated by the person who took them. Audio or video recordings are primary sources.
Original legal documents are primary sources.

Secondary sources are interpretations of those primary sources. Scholarly research would be a
secondary source, while the data scholarly research relies on would be the primary source. Editorial
commentary on a news story is a secondary source.

Secondary sources fall along a spectrum from original fact journalism that gathers up and reports
primary source material to analysis of primary facts to outright opinion on the facts.

Tertiary sources are unrelated to the facts themselves and are tools to help interpret primary and
secondary sources. Dictionaries, thesauruses, almanacs, fact books, encyclopedias, that kind of stuff are
tertiary sources.

Now, the line between primary and secondary sources can sometime be a little fuzzy.

For example, reporting itself can be a secondary source. Good, ethical journalism tries to refrain from
editorializing as much as possible, and focuses on just reporting the primary source material without
interpretation or commentary.
However, good journalists also understand that just reciting facts without putting those facts into
context is also problematic. Trying to add that context can sometimes end up interjecting analysis and
opinion into the mix.

A critical reader understands how to sort out the primary source material from the secondary source
material.

This is where Fox News gets itself into a lot of trouble, where Politifact sometimes straddles the line, and
where Snopes is generally quite good.

Snopes, Politifact, , and other fact checking sources will cite the primary sources and follow up on those
sources. They will link to online sources and vet those sources for verification.

Snopes is excellent at just checking whether or not there is primary source material, and whether that
primary source material supports the conclusion. Snopes does a very good job at providing links and
citations to the primary source material so that a responsible critical reader can follow up on that source
material and verify it for themselves.

Politifact is also very good at linking to its primary sources.

Fox News, Occupy Democrats, Breitbart, Huffington Post, InfoWars, the Daily Kos, and others are not
very good at this.

These sources tend not to link to primary source reporting or primary sources at all. If you follow those
links, the original source might be buried five layers deep. Fox News cites to World News Daily which
cites to Breitbart which cites to The Daily Mail tabloid which simply pulled it out of its ass to start with.

Or worse, these links are circular. Fox News cites to WND which cites to Breitbart which cites to
InfoWars which cited to a Fox News article in the first place. In that game of telephone, the information
got distorted enough that the new article becomes an amped up pile of steaming bullshit.
This kind of source burying and game of telephone is exactly how an Obama administration “scandal”
got started that never actually existed. President Obama went on a trade summit mission to India in
2010. These kinds of visits abroad require a lot of staff and routinely cost several million dollars a day.

On November 2nd, the website for New Delhi TV reported that this trip was going to cost the United
States two hundred million dollars a day. It cited an anonymous Indian official. There was no other
verification.

The Drudge Report picked up the story and posted a link to it, either believing the story or simply not
caring whether it was or wasn’t. Rush Limbaugh, popular conservative radio host, picked up the Drudge
Report story, and runs with it. Fox News picks up the Limbaugh broadcast, where then-Fox
commentator Glenn Beck adds 34 warships or approximately fifteen percent of the U.S. Navy, and ups
the staff count of folks coming along to three thousand people.

This prompts the administration to put out a statement that these figures “have no basis in reality.”

Then-Congresswoman Michelle Bachmann ignored the statement, and restated to Anderson Cooper
these entirely unverified numbers and now adding that President Obama’s administration had booked
over 800 rooms at the Taj Mahal Hotel - which has only 550 rooms total. When Cooper asked her where
they came from, she simply said, “These are the numbers coming out of the press.”

It took less than 48 hours for this story, which was made up whole cloth, to make it up to a
Congresswoman to be recited as gospel truth, because nobody in that chain looked for the primary
source to verify its accuracy or credibility. It passed through at least six levels of secondary sources. And
in that game of telephone, an inflated number of staff, nearly one-sixth of the United States Navy, and
double-booking at least half of the Taj Mahal Hotel were all added.

(3) What are the actual, verified facts the source contains?

There’s an old expression: trust, but verify. Look for facts.

Importantly: look for actual facts and not just opinions masquerading as facts. This is becoming an all-
too-common problem today. Someone’s unsupported conclusion is taken as fact.

Facts are verifiable. They happened, or they did not. They exist, or they do not.

Opinions carry around a conclusion or a preference with them. A person who says that the color green is
their favorite is stating what looks like a fact about themselves, but it’s a preferential opinion: green is
the best color. This can’t be verified. This is merely a conclusion.

Facts, on the other hand, are simply neutral. That grass is green. How do you know that? We can verify
it. Five people can look at the grass, and come to a consensus about whether it is green. We can settle
the matter by using a spectrometer and determining whether the grass reflects electromagnetic waves
at 560–520 nm (the accepted consensus definition of green.)

This is one of the reasons that Snopes and Politifact are generally regarded as safe: they verify the facts
that various statements rely on.

Some time back, a friend of mine posted an article she ran across in something called the Freedom Daily
about Islamberg, New York being raided by a massive counter-terrorism task force who caught more
than two dozen Muslim terrorists with various explosives and weapons.

The problem is… it never happened.

The only thing that can be verified is that one Muslim person from a town nearly fifty miles away with
no verified ties to Islamberg was arrested for stealing several boxes of ammunition, and in the
subsequent search related to that investigation, it was discovered that he had several illegal firearms.

There is absolutely no verification of a Federal raid on Islamberg.

Primary sources such as the county sheriff, someone that would have to be included in something like
that, noted that it was familiar with the article and that there was “absolutely no truth” to it.

There was literally no verification of the story.

But what if there’s proof, like photos, right? Surely they can’t be made up!

Even pictures can lie.

Graphic pictures are another area where untrustworthy sources can try to grab you and suck you in. For
example, there was a photograph that was circulated in around several evangelical Christian blogs
depicting a bloodied woman with her eyes and mouth sewn shut, and touted as a Saudi woman who had
been tortured for her Christian faith.

A reverse Google image search quickly reveals that this woman is actually Japanese, and is into “extreme
body modification.”

Stock images or composite images are often used in many fake news sites. While some composites are
difficult for the average person to detect, others are usually obvious - shadows or lighting that doesn’t
match, that sort of stuff. If a photograph is reported in the news source, if you’re not sure about it. And
if it’s graphic or seems extremely damning, definitely you shouldn’t be sure of it.

Even reputable news sources can sometimes re-use old photographs or video, particularly of various
natural disasters or extreme weather. A photograph or video of people walking through a blizzard could
be years old.

Snopes, Politifact, , etc., are excellent secondary sources when it comes to pulling out the verifiable,
relevant facts from the primary sources. They follow up on the sources from other information outlets,
internet stories, or statements of public figures, and find out what the verifiable facts are. The only
opinion they make is whether the facts support the conclusions reached by these information outlets,
and they’re up front about what their opinion is and how they reached it. They check whether photos
are real or not. They do the reverse image searches. They follow the links. They make sure to vet the
material before presenting it.

(4) What inferences can reasonably be drawn from the facts, and do you have all of the facts to draw
accurate inferences?

Inferences are conclusions that people make after looking at the facts.

This is where people mostly get into trouble. This is also where the spin doctors make their money. NPR,
MSNBC, and FOX can all report the exact same piece of news, and all three of these sources could result
in three entirely different and competing narratives. Why? Because of the why. Each will present the
story slanted to let the reader come up with their own context about what those facts mean, why the
story happened that way. The sources, the context, all of it is designed to lead you to a certain
conclusion.

Facts on their own are just trivia. People naturally try to put facts into a narrative. It’s just how we think,
how we operate as human beings.

That is why people will make viral a Natural News article that shows that a study “conclusively proved”
that there is usually 1 part per billion of glyphosate in wine.

Why does that matter?

Because glyphosate is a boogeyman to a large group of people. It could (very disputably) be a

carcinogen. Never mind that your typical wine contains approximately 130 million parts per billion of a
known carcinogen () because that’s not the story. The narrative is something sinister is amok. The article
invites you to draw a conclusion from the facts they present, namely that Big Wine is in the pocket of
Monsanto who doesn’t want you to know they’re spraying grapes with Roundup because Monsanto
wants you to have cancer. Why? Because they’re eeee-ville!
What logically follows from the facts? Not what you want them to mean. Not what you wish them to
mean. This is where you have to walk back over to step 1 and ask yourself what do I want to hear?

Logical inferences are created by applying facts to rules.

For example, what happened here?

Tragedy in the bathroom

It probably seems pretty obvious to most people that this guy’s wife snuck up behind him, smashed him
with a bottle while brushing his teeth, and then tried to make it look like he slipped on some soap
coming out of the shower, right?

There is a whole set of rules about the evidence we see in this picture getting applied. What does a
bathroom typically look like? How do shoe prints get there, and what rules do we have about the shape
of shoe prints? What do we know about monogrammed towels and their typical usage? There are a
whole host of rules that get subconsciously applied here: only women wear high heels. A man doesn’t
use the “hers” towel - he uses the “his” towel.

These rules are the critical link between the facts and the conclusions we can draw from those basic
facts. The facts here are not in dispute. We can all look at those facts and agree what the facts are.

But the rules we impose on those facts to draw inferences might not be accurate.

When assessing the credibility of a source, it’s essential to be self-reflective and personally critical of the
rules that you are applying to reach a conclusion. This goes back to that confirmation bias that I referred
to earlier.

Even certain pieces of evidence might be inferred, or conclusions stacked on top of each other to make
larger inferences.
For example, if there’s a missing person, a pool of blood, and drag marks, and you don’t have a body,
would you not typically infer there was a dead person somewhere? So, if it’s somewhere, where is it?
And what happened?

It seems obvious that someone was murdered, right? But that’s merely an inference, which filled in
certain missing pieces of evidence.

This is also why it’s important to have all of the facts before making an inferential conclusion.

Are you using all the facts, or ignoring ones that go against the narrative you want to hear?

During the 2016 U.S. election cycle, a black church was burned, and the words Vote Trump spray
painted on the side.

Factually, this happened. Depending on how crazy you want to get, you could either trust the photos, or
have bought a plane ticket and driven down to Mississippi to go see it with your own eyes. It’s verifiable,
authentic.

What came after that is the problem. Assumptions and inferences ran amok.

Not without good reason, mind you.

It was a historically black church in Mississippi. It’s not hard to add a little into the fact pattern. It’s not a
stretch to find a context. A lot of people wanted it to be Trump supporters. They wanted them to be as
vile and deplorable as they were led to believe Trump supporters were. A few facts were enough data
points combined with some historical precedence to reach a plausible conclusion. A lot of liberal people
wanted that story to be true.

Then, it turned out later that the church was burned down and tagged by a black man, who was a
congregation member.
Immediately, conservatives went crazy. This was vindication, based on that one little fact alone. They
tweeted and posted and blogged immediately, crowing their victory. They wanted a different story to be
true, one in which they weren’t the bad guys. Black guy trying to set us all up, we knew it all along!
See?! See?! Y’all called us deplorables, and look how mistaken you were!

And so they ignored (or didn’t bother to look for) any other facts. They went straight from a little bit of
fact to an inference about what those facts showed, because it supported their narrative.

I admit, when the story first broke originally, I wanted it to be true. Black church, Mississippi, plenty of
white supremacists at Trump rallies, sure. It wasn’t a stretch. It wasn’t implausible. And it said
something about Trump supporters that I personally liked, that some of them were terrible enough
people to firebomb a church and tag the ruins with self-supporting graffiti. I wanted that to be true.

Still, I cautioned myself and others at the time after a day of thinking about it to wait for the
investigation to come back. Let’s wait for the facts. All of the facts.

Turns out, after we got those facts, everyone was wrong. Everyone drew inferences from scant facts,
and every one of those inferences was wrong.

It turned out that the man arrested wasn’t doing it for racially motivated reasons at all. He was taking
advantage of the charged, polarized environment around the election to cover up a burglary he
committed at the church. It was a convenient cover for him. He would have gotten away with it, as well,
if it weren’t for good police work that went beyond the initial theory of the case, and looked for all the
facts. (And those meddling kids!) Not just facts that supported one version of events, but all the facts.

If all the police did was look at the early facts and didn’t investigate more, didn’t follow up on
everything, they might very well have reached the same conclusions the internet did. They might have
reached the same conclusions that many of their predecessors in the same town might have reached a
hundred years ago, and those people a hundred years ago typically ended up lynching someone over it.

Remember: there are two parts to inferences, the facts and the rules.
In my profession as an attorney, we talk about “garbage in, garbage out.” If there is a breakdown in the
facts, either inaccurate or incomplete facts, or if there is a breakdown in the rules, either inaccurate or
incomplete rules, then the inference drawn from those facts and rules will also be inaccurate.

When evaluating a source, evaluate the inferences it makes, and the inferences you draw from the
source, and ask whether it is logically supported, whether it is based on incomplete or inaccurate data,
or whether it is based on incomplete or inaccurate logical rules.

(5) What is the authorial bias?

I imagine most of you thought we ought to start here. If the source is biased, we can just dismiss it,
right?

Absolutely not.

Every source is biased. And that’s okay.

Bias is not inherently a discrediting problem.

There are a couple of common forms of authorial bias.

Confirmation bias. I discussed this earlier. It’s the tendency to accept things we already like and reject
things that we don’t.

Partisan or “statement” bias. This is what we most often think of as bias, or “spin,” where the author
engages in active advocacy. This kind of bias is where the author goes beyond simple primary fact
reporting and interjects a preferential or partisan slant on the material. They present the facts in such a
way that it clearly favors a perspective.
Selection or “gatekeeping” bias. Some news sources go out of their way to try to be balanced in their
presentment of a news story, such as NPR, but have a different kind of bias: what stories they present at
all. This is selection bias. By cherry-picking only certain events or facts, even if the story is presented in a
balanced manner, the overall source is biased.

A subset of this is “ventriloquism,” where a source edits experts or witnesses out of context so that their
quotes agree with the author’s bias, or only show the parts of the expert or witness that agree with the
author’s bias.

Coverage or “visibility” bias. Some stories are juicier or more popular than others, and so get more
coverage. Hundreds of children are killed by their parents every year in shaken baby incidents or abuse.
But the Casey Anthony trial ended up becoming incredibly high profile, dominating the national news for
weeks and captivating the country, because it had a special twisted kind of appeal to it.

Concision bias. In the TL;DR culture today, sources are having a harder and harder time with complex,
nuanced reporting and analysis. Investigative journalism can take months if not years of research; see
the Boston Globe’s Spotlight team’s investigation of the clergy sexual abuse in the Catholic church.
There is a lot of information that needs to be put into context.

And people are less patient to read that. They want the quarter-page summary. The 140 character
tweetable explanation.

The McDonald’s hot coffee case is a perfect example. The original story was long and complex. The jury
found that McDonald’s was not only negligent, but actively reckless in how it stored and served coffee,
having broken regulations on safe temperatures. The woman involved was not driving; her son was.
They had pulled over into a parking stall; they were not in motion when she spilled the coffee. She had
received third degree burns to much of her body. The jury award was not only for her medical bills, but
punitive damages because McDonald’s had acted so egregiously.

Sensationalism. This is bias for the extraordinary over the ordinary Sometimes also called “yellow
journalism,” it is the tendency of media outlets to hype up an otherwise less exciting story to get better
ratings or circulation, thereby giving the impression that relatively rare events (such as an illegal
immigrant committing a violent crime,) are more common than ordinary events (American citizens
committing violent crimes.)
Modern mass media’s newest form of sensationalism is what we call “clickbait.” If you see articles about
“X Politician Blasts Obama!” or “21 Unconstitutional Things Obama Did - You’ll Never Believe #17!” it’s
just sensationalized clickbait.

False Balance or “fairness bias.” Sometimes sources try to present facts in a balanced manner to avoid
the appearance of bias, but this can lead to giving the appearance of fairness at the expense of
artificially making it seem like a side of the story is more based in reality than it is.

There’s a good little exchange in the television show The Newsroom that illustrates this:

Maggie: How can you be biased towards fairness?

MacKenzie: There aren't two sides to every story. Some stories have five sides; some only have one.

Tess: I still don't underst...

Will: Bias towards fairness means that if the entire congressional Republican caucus were to walk into
the House and propose a resolution stating that the Earth was flat, the Times would lead with
"Democrats and Republicans Can't Agree on Shape of Earth."

False Timeliness. This can come in two main flavors:

Presenting an old story as new. This is more common to hoax sites or actual fake news, recycling old
scaremongering and just updating the release dates. This happens a lot with crimes; something from five
or ten years ago is brought out as having freshly happened.

Bringing up an old story as suddenly relevant because a similar event has now occurred, or without
specific context and thus implying that the old story is new or timely.

The key to evaluating a source for bias is understanding what bias it might have, and then accounting for
it when verifying the source material or judging the credibility of the source.

I don’t ignore Fox News, but I know very well that if I see a Fox News article, I’m going to have to do a
lot of extra digging to get the rest of the facts that they inevitably decided weren’t relevant (and usually
are,) and spend a lot of time sorting out the spin from the primary source material and isolating the rest
of the biases.

Now, most of the time I’m just too lazy to do that much work on it, so I don’t bother with Fox News in
most circumstances.
There’s an excellent chart out there called the Media Bias Chart.

This is a good chart, but it really only covers partisan bias and where along the secondary source
spectrum a source tends to exist on average.

This chart doesn’t tell you much about selection bias, coverage bias, or other authorial biases.

Some of the sources at the top of that chart are highly reliable sources in terms of their partisanship and
original primary source reporting. The AP and Reuters are some of the gold standard in journalism.

But the AP and Reuters can also still be subject to selection or concision biases without ever being
partisan. The AP doesn’t often do extremely in-depth investigative journalism; it tends to report shorter
stories of current events.

ProPublica, The Economist, USA Today, or even Time Magazine are all reasonably centrist in partisan
bias, but these tend to interject a great deal of selection bias simply due to format of presentation. Most
of these are periodicals for their long-format pieces. They don’t do much short reporting. They do more
analysis. It might be non-partisan analysis, but because they only have so much space and time, they
have to pick and choose what to write about. That inherently results in some selection or coverage bias.

That’s not necessarily anything bad. It just means you might not get other important, relevant facts or
stories that could shed relevant context on the original story.

Snopes, Politifact, and other fact checkers are also susceptible to these kinds of biases, particularly
selection bias.

Politifact is criticized by conservatives as liberal-biased because of this. Politifact tends to fact-check

more conservative claims than liberal claims. As a result, a broad survey of the site makes it seem like it
is biased towards liberalism.
And conservatives sometimes gripe about the site simply not bothering to check on certain liberal
claims. That can be a valid criticism.

Now, there’s also something to be said that conservatives might just also make more and more
incendiary claims, and make more erroneous claims.

For example, conservatives argue that the bulk of the media outlets must be liberally biased simply
because they tend to report much more negative press about President Donald Trump than positive
stories.

But, that doesn’t inherently indicate partisan, selection, or any other bias. It could simply be that
President Trump keeps consistently lying out every corner of his mouth and constantly picking fights
with the press over policies that are generally criticized by a majority of the public figures out there.

If Republicans do a dozen crazy things and Democrats do one, it isn’t unbiased to make up eleven crazy
stories about Democrats to balance it out. That would just be introducing a fairness bias, instead.

As I said, every source has some degree of bias. That’s fine. Apply the appropriate corrective lenses, and
evaluate the facts and inferences on their merits.

(6) What degree of institutional integrity does this source have?

Good journalistic outlets know that the only resource they have is credibility. If a journalist is not
credible, she is finished in the industry. Lying, just once, , is a career death sentence.

Sources that are reputable and credible take this very seriously. If they have to issue a retraction, they
have probably fired the journalist who reported the story and that guy is likely blackballed from
everything with more integrity than Buzzfeed.
Look at how the source handles corrections and retractions when it turns out they’re wrong about
something. That will tell you a lot about the credibility of the source right there.

There are some red flags you should immediately recognize for any source, article, or media outlet that
should alert you to credibility issues. Some can be “immediately discount this source and do not engage
further with it,” while others can be “proceed with caution.”

You are barraged with advertising and pop-ups the minute you get to the site. Even reputable sources
make money off of advertising, but if your screen starts to look like the house from Chevy Chase’s
Christmas Vacation, especially if it’s advertising to lots of clickbaity things like “This one weird trick in
your town can lower your insurance by 75%!” it’s less likely to be a reputable, credible source.

Run away and don’t come back to these sources.

Listicles and articles that require you to click through 15 pages to read all of it. If you’re in a source that
requires you to wade through 30 ads a page and click on “next” a dozen times to read the whole thing,
it’s a red flag that this is not a reputable, credible source, and is just a steaming pile of clickbait trying to
extract advertising dollars from you.

Again, run, don’t walk, and never come back.

Headlines that don’t match the articles. You’d be amazed at how often shitty sources will try to grab
you with a sensationalized headline that has little or nothing to do with the article. “Steve King Actually
Registered Member of KKK!” turns out to be a story about some idiot named Steve King that lives in
Mississippi and not the Congressman from Iowa.

One time here or there? Flag it and be skeptical of anything it publishes. If a source does this a lot, stop
using it.

Graphic or bizarre pictures with the headlines. This is clickbait. The source is trying to lure you in with
the promise of a novelty. That picture may not even be in the article.

This is why sources like Buzzfeed, Twenty-Two Words, etc. are generally crap. Don’t use these for news
or credible sources.

Vague attacks or generalized references. If you see something about “Washington” or “The White
House,” or “Trump supporters” or “Bernie fans,” you can safely discount it by at least 50%.
Anonymous sources. Be careful with these. They can be reputable. Mark Felt was the whistleblower who
brought down Nixon, and nearly until his death in 2008 was an anonymous source known only to the
public as “Deep Throat.”

But anonymous sources can also be extremely disreputable. Look to see if any of the facts can be
verified.

Future speculation. If the source is speculating on what might happen, be wary of it. Unless military
scientists have something with time travel not generally available to the public, it’s pretty difficult to get
accurate fact reporting from the future. Be wary of sources talking about what’s going to happen as if it
were fact.

“Lawmaker says [insert shitty foot-in-mouth statement here]” or “Lawmaker proposes bill to [insert
extremely stupid or divisive issue here.]” Turns out that “lawmaker” is a pretty generic term that can
apply right on down to a city alderman somewhere that has about as much political clout as the
secretary of the local PTA. Sources use “lawmaker” because it’s not a Congressman or Senator and it
makes the person sound more important.

That “lawmaker” is probably a low-level junior freshman politician that has absolutely zero chance of
passing his “ban breastfeeding for public morality” bill. But a disreputable source might try to make it
sound like he’s the governor of New York.

There are over seven thousand state-level elected legislators, some of whom won their seats while
getting fewer than 1500 people to vote for them. New Hampshire alone has over 400 elected legislators.

The same is true of “advisors” or “officials.” These are basically no better than anonymous sources. The
source is trying to make them sound more important than they probably are. Take it with a salt lick.

The same is also true of [insert celebrity figure with zero expert qualifications here.]

Let me get this off my chest:

Ted Fucking Nugent is not a reliable source regarding literally any goddamned piece of information,
including music. Ted Nugent is not politician. Ted Nugent is not a policy expert. Ted Nugent doesn’t have
any expertise in any area except how to sing “Cat Scratch Fever” and avoid getting drafted by the
military. Ted Nugent does not hold a single degree above a high school diploma nor have any relevant
professional experience that would qualify him to be a credible source of anything except for how to
look like human-rat hybrid experiment that went horrifically wrong.

Stop giving this moron any sort of credibility. He is not a credible source of information.

The same is true of all sorts of other celebrities who have decided that for some reason or another, their
opinion matters and should be taken serious. Just because you have a million followers on Instagram
does not make you a credible source. Just because your parent was a senator doesn’t qualify you as an
expert on foreign policy or politics.

It’s on the blog portion of an otherwise reputable source’s website. Forbes and Reuters are respected,
usually highly credible news sources. However, both Forbes and Reuters have third-party-blog portions
of their websites that are not vetted and edited by Forbes. I could go get a blog on Forbes. Reuters has a
section of its site dedicated to unvetted, unedited press releases that literally anyone can publish.
Contributors are responsible for the content. These sources do no fact checking. Make sure its from the
journalism side of those sources before trusting it, not Joe’s Totally Unbiased Blog hosted on Forbes.

Avoid these sources, or very carefully review them.

Look for weasel words. “Many experts agree” or “a recent study shows” is often a way to weasel out
and report something that isn’t really credible without actually saying it is. “Many experts” could turn
out to be four guys in a bar in Djibouti. The source ought to tell you who those experts are and where
they work.

Follow up and see if they’re actually credible sources and experts in the field they’re talking about.

Watch out for “journals” that aren’t actually peer-reviewed, credible sources, or think tanks,
foundations, and institutes that just spout made up bullshit.

There is an ever-increasing number of sources that really look and sound like they would be scientific,
prestigious journals, and are anything but.

For example, the American Journal of Engineering Research sounds like a pretty reputable source from
the title, but it’s nothing more than a predatory journal that does not publish anything even
approaching peer-reviewed quality scientific research.
There are also various organizations, think tanks, and “institutes” that sound like credible sources, but
publish conspiracy theories, pseudoscience, hoaxes, and other entirely made up nonsense. It’s really
easy to fall down the rabbit hole these days, especially on Facebook, YouTube, Twitter, etc.

Avoid these like the plague on humanity they are.

Public Survey Polls. Unless it’s conducted by a reputable pollster who specializes in this kind of statistical
work like Mac Tan, polls are not a very credible source of information.

Discount them as credible sources of anything except how many people use Facebook or Twitter.

Check the URL. There are hundreds of fake websites that look almost exactly like actual news sites and
are off by just a letter or two. ABC News is an actual source, and generally highly respected. There’s a
website at the URL abcnews dot com dot co, and it looks very similar, but is loaded with actual fake
news instead.

Conventional source evaluation used to hold that .edu or .gov sources were likely to be reputable. Many
educational institutions today host web content that is not reviewed or fact checked, much like Forbes
and Reuters above. Government sources are still likely to be mostly credible, but the current
administration has a tenuous grasp at best of what constitutes credible, reliable fact-based information
and those running various agencies are little better.

Your mileage may vary here.

Nobody else is talking about it. If your source’s headline or article isn’t reported literally anywhere else
except a handful of blogs or some threads on Reddit, that’s a good clue that it’s not credible. There is no
validity to the idea that there is a massive conspiracy to hide Teh Truth!™ from all of us by the
mainstream media, only to be thwarted by a plucky band of YouTube commentors and Redditors.

Look for at least local news articles. Most small towns have at least a local radio station or weekly
newspaper that publishes local news, even “Cow Crosses Road; Traffic Halted for Hour.”
If it’s something that seems nationally important and even the local herald doesn’t see it that way, odds
are it never happened.

View foreign government-run sources with a healthy skepticism. Some of my conservative friends have
started sharing a lot of things from RT. RT is literally Russia Today, a Russian state-run media
corporation. For the love of God, don’t. This is literally a propaganda arm of the Russian government.

Now, not all foreign sources are bad. The Times of India, British Broadcasting Corporation (BBC), Al
Jazeera, and Der Speigel are excellent journalism and well-respected sources.

But don’t rely on foreign state-run media.

Snopes, Politifact, and other fact checkers are usually very good at avoiding these kinds of red flags.

Snopes, Politifact, and other fact checkers are generally accurate and honest arbiters of truth because
they don’t engage in shady journalistic practices, verify and disclose their sources so anyone can decide
that if they don’t want to take the fact-checker’s word for it that they can follow up and do the math
themselves, and have reputations for correcting mistakes.

They have good journalistic integrity and that is why they are regarded as high quality, reputable,
credible sources.

Mostly Standard Addendum and Disclaimer: read this before you comment.

I welcome rational, reasoned debate on the merits with reliable, credible sources.

But coming on here and calling me names, pissing and moaning about how biased I am, et cetera and so
forth, will result in a swift one-way frogmarch out the airlock. Doing the same to others will result in the
same treatment.

Essentially, act like an adult and don’t be a dick about it.

Getting cute with me about my commenting rules and how my answer doesn’t follow my rules and blah,
blah, whine, blah is getting old. Again, ornery enough today to not put up with it. Stay on topic or you’ll
get to watch the debate from the outside.

If you want to argue and you’re not sure how to not be a dick about it, just post a picture of a cute baby
animal instead, all right? Your displeasure and disagreement will be duly noted. Pinkie swear.

I’m done with warnings. If you have to consider whether or not you’re over the line, the answer is most
likely yes. I’ll just delete your comment and probably block you, and frankly, I won’t lose a minute of
sleep over it.

Debate responsibly.

Footnotes

11.9k Views · ·