Monthly Archives: October 2011

Rupar’s research on evidence in newspapers: What to do?

As I said in my last post, my research into some of this year’s Pulitzer Prize-winning stories suggested that the most common problem in the stories, from a critical thinking perspective, was that they lacked evidence. I was surprised to find a lack of evidence, but maybe I shouldn’t have been.

In a study of 510 stories in New Zealand newspapers published in 2006, Veronica Rupar found that 62% of them did not “clearly describe[] the input of sources behind their stories.” These articles “only stated an opinion or gave quotes without indicating how the news was assembled … (example: ‘The Prime Minister thinks…’)” (130).

Why does Rupar’s finding interest me? Because it encourages me to think more about how to decide to accept something in the absence of evidence.

People citing how they knows what they claim is how a they get me to believe them. Without a citation of some sort, I have to take what they claim on trust.

Trust in what? Perhaps that he or she was at the scene and correctly understood what was said. Perhaps that the person spoken with was a reputable source about, or in a position to know, the thing in question.

Alternatively, if I won’t accept what the person says on trust, I could accept it based on past performance (was this person correct before? Did he try to be correct?) The claim might also go down more smoothly with the assumption that the speaker, journalist or otherwise, is generally well-intentioned.

In many ways, these questions are simplistic wrappers around research on vetting and accepting arguments from authority that have been discussed for millenia. Douglas Walton’s book on the subject is a great starting point; Thomas Haskell’s book also provided very interesting historical background for a newbie like I.

Rupar’s finding also interested me because of her demonstration that a news story could go from not citing sources to citing them with relative ease (133).

Admittedly, her example involves adding 16 words to what is only a 54-word story to start with. But that simple move would be all, I think, that’s necessary to ease many of my fears.

Kovach and Rosenstiel’s book Blur, and critical thinking as a mindset

This week I finished looking through the latest book by Bill Kovach and Tom Rosenstiel, Blur: How to Know What’s True in the Age of Information Overload. I rely on Kovach and Rosenstiel’s Elements of Journalism in my thesis a fair amount, so I am embarrassed to have only recently found Blur. Had I found it earlier I would have referenced it much more in my thesis than I will now.

I recommend the book to readers interested in learning more about testing the reasons and evidence in a story before accepting the story’s conclusions. Most importantly, I think the book provides a good set of critical questions to use when assessing the weight of evidence given in a story.

I’ve grown more convinced in the last few months that, when consuming journalism, having a store of critical questions for assessing evidence is at least as important as having one for fallacy-hunting.

When I began reading the sample of Pulitzer Prize-winning stories I’m studying in my thesis, I expected to spend most of my time thinking about fallacies. But I was surprised to find that insufficient evidence was more often keeping me from accepting reporters’ conclusions. As I wrap up my thesis in the next few weeks, I will begin posting both chapters from the thesis and separate analyses of each story in my sample to better show what I mean.

I was disappointed, though, that Kovach and Rosenstiel did not draw more from research in informal logic and critical thinking. Of course, that’s a painless criticism for me to offer, because I’ve spent almost a year now looking at such research. But I think Kovach and Rosenstiel would have found that that research had already done much of their work for them. Some of the critical questions they propose, such as “Who or what are the sources and why should I believe them?” and “What might be an alternative explanation or understanding?”, have been a part of critical thinking textbooks for a while.

I think they would have also found that research on critical thinking challenged some of their ideas. They claim that:

until recently we haven’t had to perform so much of the … basic sorting out of the facts of events[] for ourselves. We relied on mediating authorities — the press — to do much of that for us. How well they did it is beside the point. Now, with so many competing news conduits and so many partial accounts, we must adopt some of these diagnostic skills for ourselves, so we can at least identify good journalism from bad (p. 31).

From critical thinking might come the response that the mindset advocated by Kovach and Rosenstiel did not become relevant to journalism just because of the changes in journalism during the last 20 years. Critical thinking is “a kind of moral commitment” that was useful in deciding whether to accept journalists’ claims even before “information overload.”

Kovach and Rosenstiel seem to offer critical thinking as a sort of floatation device to use now that we can no longer “rely on mediating authorities.” But it isn’t clear to me why we shouldn’t have been asking “why should I believe those sources?” or “are there alternative explanations?” from the beginning.