Fact checking: How to think like a journalist
Today, getting information is as easy as opening a browser on your computer or phone, typing in a question and waiting a nanosecond for a long list of links to load. These websites will promise to answer your question. But actually, they may not be all that trustworthy. What do you do?
Figuring out which site is believable and which is bunk takes work. Vetting that information is not impossible, though. In fact, it’s what every good journalist does daily. And students and other non-journalists can learn from the methods reporters use to determine the truth of what they read and hear.
There are plenty of people who call themselves a “journalist.” What separates a good journalist from a bad or lazy one is often where they get their information. Good reporters are always on the lookout for a “scoop” — new information or new interpretations of existing data. But for them, a scoop is worthless if it’s based on bogus or misinterpreted data.
To verify a new claim, a reporter contacts experts on the topic. Often many. In journalism, such experts are known as “sources.” This may be a scientist who discovered something new in the lab. Or it could be the witness to a crime. The more strange or controversial that some claim is, the more sources a reporter may need to contact.
In the decades I’ve been reporting on science, I’ve written more than 1,000 stories, with anywhere from 2 to 50 sources for each. As the editor of this magazine, I’ve read, tweaked — and even totally rewritten — some 2,000 more stories authored by other journalists. I have a lot of experience hearing claims of new and amazing things. Some have proven too good to be true. Others eventually were validated by research.
Just because I read something — or hear a scientist say something — doesn’t mean I believe it. Reporters are taught to ask questions, then verify every factoid they can. Indeed, sooner or later every cub reporter hears: “If your mother says she loves you, don’t believe it unless you can get confirmation from at least two independent sources.” That line is meant to make us question everything! Journalists are skeptics. And you should be too.
How certain can you be?
Some things are relatively easy to source. If you want to see who won a ballgame, consult the scoreboard. Or you could talk to people who watched the game, those who played in it or league officials.
But few things are as black and white as who won a ballgame.
If there’s a forest fire, how do you know where it started, when and how? You can ask government officials, but they may not know. You can ask a fire investigator or talk to people who were near the fire shortly after it started. But unless someone spotted the fire’s ignition as it happened, people may never know for sure. They may simply offer speculation. Saying something “might have” or even “probably” started that fire is not fact. So you would have to qualify any statement about the fire’s origins.
I describe such qualifiers as “weasel words.” These words and phrases include “might,” “could have,” “was likely to have been” or was “suggested” to have happened this way. Whenever something isn’t known for sure (meaning the sources of that information don’t either), journalists will highlight their uncertainty by using weasel words. They are a sign of a good journalist. They are being honest with the reader about the uncertainty rather than overblowing claims.
How do I know if such weasel words are necessary? When I consider a claim from a scientist, I ask a lot of questions. I consider whether I can trust that person. Did she collect the data personally? Did she publish the data in a respected peer-reviewed journal (which means that other experts had to accept the data as looking reliable)? How strong are the statistics behind her findings?
If I can get an original paper in which the findings were published, I might check for the answers. But I am not an expert, so I often turn to outside experts for help. These are people who are not directly related to the new research. I’ll ask them to look at the paper and give me their assessments.
Reporters who don’t have much time or motivation may just quote the first scientist, accepting anything she said as fact. If the reporter didn’t get an outside comment, the reader will not know how well to trust the original source. Keep that in mind when you read news reports.
Figuring out who to trust
Good reporting is time-consuming. It also can be costly. If the news you are reading was free to the world on the web, consider what that may mean. A good reporter should be paid for all of the work that he does, as are all of the people at this magazine (and at its sister publication, Science News). Some blogs and even news sites are authored by unpaid writers. Many of these writers are untrained in reporting. Their reports may not provide the depth and care that a professional journalist would have employed to check the facts.
Also, some “news” stories are written by people working for organizations that are more interested in promoting a certain point of view than in accuracy. Or the entire site may be biased, from the owners down to the journalists. This can cause them to willfully mislead their readers. You might search for the writer online and figure out who pays them. If the money comes from a vaping company, for instance, what is written about e-cigarettes might not be entirely the truth.
Likewise, check whether the site you’ve obtained information from appears to have done a good job in vetting what they’re reporting. Particularly where the news seems “unbelievable,” look for signs that the reporter was appropriately skeptical of such claims.
And don’t stop there.
Is the person who wrote a story or interviewed a source named? Was this writer a journalist — or just someone who heard about some new claims? Did the reporter cite his sources by name? Did he describe what qualified the cited sources as experts for this story? Did he give you enough information to fact check the data he mentioned, such as citing a published research paper that you can look up?
Are the scientists who are quoted in a news story expert in the field at issue? It may seem obvious, but details of a story on climate science are best interpreted by other climate scientists. When a physician, politician, economist or theologian comments on new data in a story on climate or another field outside their own, keep in mind that these people may be “experts” — but not necessarily in a field that enables them to fully understand the new findings.
One place you can turn for solid news is this publication. We are professional science journalists with degrees in science or journalism — and often both. We speak with and quote experts. We ask for outside comment. We cite our sources. But first and foremost: We are skeptical.
You should also be skeptical — of everything you read, even here. That’s acting like a journalist.
bias The tendency to hold a particular perspective or preference that favors some thing, some group or some choice. Scientists often “blind” subjects to the details of a test (don’t tell them what it is) so that their biases will not affect the results.
blog Short for web log, these Internet posts can take the form of news reports, topical discussions, opinionated rants, diaries or photo galleries.
browser (in computing) The software program or application that someone uses to find and retrieve information from Web pages on the internet.
data Facts and/or statistics collected together for analysis but not necessarily organized in a way that gives them meaning. For digital information (the type stored by computers), those data typically are numbers stored in a binary code, portrayed as strings of zeros and ones.
e-cigarette (short for electronic cigarette) Battery-powered device that disperses nicotine and other chemicals as tiny airborne particles that users can inhale. They were originally developed as a safer alternative to cigarettes that users could use as they tried to slowly break their addiction to the nicotine in tobacco products. These devices heat up a flavored liquid until it evaporates, producing vapors. People use these devices are known as vapers.
field An area of study, as in: Her field of research was biology. Also a term to describe a real-world environment in which some research is conducted, such as at sea, in a forest, on a mountaintop or on a city street. It is the opposite of an artificial setting, such as a research laboratory.
ignition (in chemistry) The act of igniting a fuel in combustion, such as natural gas or gasoline.
journal (in science) A publication in which scientists share their research findings with experts (and sometimes even the public). Some journals publish papers from all fields of science, technology, engineering and math, while others are specific to a single subject. The best journals are peer-reviewed: They send all submitted articles to outside experts to be read and critiqued. The goal, here, is to prevent the publication of mistakes, fraud or sloppy work.
online (n.) On the internet. (adj.) A term for what can be found or accessed on the internet.
peer (noun) Someone who is an equal, based on age, education, status, training or some other features. (verb) To look into something, searching for details.
skeptical Not easily convinced; having doubts or reservations.
statistics The practice or science of collecting and analyzing numerical data in large quantities and interpreting their meaning. Much of this work involves reducing errors that might be attributable to random variation. A professional who works in this field is called a statistician.
vaping (v. to vape) A slang term for the use of e-cigarettes because these devices emit vapor, not smoke. People who do this are referred to as vapers.
verify (n. verification) To demonstrate or confirm in some way that a particular claim or suspicion is true.
vet (n. vetting) A verb meaning to check the truthfulness or accuracy of something.
web (in computing) An abbreviation of World Wide Web, it is a slang term for the internet.
Survey: “News and America’s kids: How young people perceive and are impacted by the news.” Common Sense Media. March 8, 2017.
Report: S. Wineberg et al. “Evaluating information: The cornerstone of civic online reasoning.” Stanford History Education Group. November 22, 2016.
Book: C. Seife. Virtual unreality: The new era of digital deception. Penguin Books, August 4, 2015, 256 pp.