Category Archives: Critical thinking

Error screen

Opportunities for critical thinking about data journalism

The 1988 compilation of essays Selected Issues in Logic and Communication included one by Ralph Johnson called “Poll-ution: Coping with Surveys and polls.”

Johnson wrote about critical thinking questions that help you decide whether to accept the conclusions in news coverage of polling data:

Polls are often reported and are increasingly significant in political life. We need to know how to assess reports of polls. Crucial information such as the nature of the sample and the precise question asked is often not reported. No matter how accurate sampling techniques are, a poll cannot provide valuable information if its question is misleading or loaded.*

I liked Jacob Harris’s recent essay in Source because it provided a similar collection of entry points for critical thinking questions about data journalism:

Data journalism does not fall perfect from the sky. It’s painstakingly built. … in my own experience it generally involves the following steps:

  • Acquisition
  • Cleaning
  • Loading
  • Verification
  • Analysis
  • Cherry-picking
  • Presentation
  • Maintenance

The fun of data journalism is that each of these steps can introduce errors that can affect the final story.

Read “The Times Regrets the Programmer Error” at Source

* When I added Johnson’s essay to my bibliography manager, I included this quote in the “Abstract” field. But I did not mark a page number and no longer have access to the book. My search for this essay in Google Books returns the beginning of the quote, so I feel pretty safe in assuming it is Johnson’s, but there is a chance I am wrong.

Photo by Anthony Catalano: http://flic.kr/p/6jVTaX

Advertisements

Recommended: ‘Investigative Storytelling Gone Awry’

The NPR ombudsman, Edward Schumacher-Matos, produced an extraordinary investigation-of-an-investigation earlier this year:

My finding is that the series was deeply flawed and should not have been aired as it was.

The series committed five sins that violate NPR’s code of standards and ethics. They were:

  1. No proof for its main allegations of wrongdoing;
  2. Unfair tone in communicating these unproven allegations;
  3. Factual errors, shaky anecdotes and misleading use of data by quietly switching what was being measured;
  4. Incomplete reporting and lack of critical context;
  5. No response from the state on many key points.

I was convinced after the first couple of chapters that Schumacher-Matos was worth reading for those who are invested in the issue covered by the original report. I found his writing to be fair, cautious, and reflective.

To me, his report is a model of critical thinking applied to journalism, and I was disappointed that NPR responded with only a defensive memo.

Journalism enhanced by the principle of charity

This post is a (somewhat modified) excerpt from a paper I presented a few weeks ago (PDF). For reasons I mention below, I think it’s also relevant to the vibrant recent discussions of about “post-truth” journalism.

The principle of charity

The principle of charity, broadly stated, is that when one responds to another person’s argument, one has a responsibility to respond to the strongest possible version of that argument. That can mean doing many things on the other person’s behalf, including supplying unstated assumptions or using more precise language. Essentially, the principle of charity gives the other person the benefit of the doubt. On this point, see pages 16 and 17 of Edward Damer’s “Attacking Faulty Reasoning.”

There are some practical reasons for employing the principle of charity to an argument, but primarily it is a move of fairness.

I think its purpose as an act of fairness gives the principle of charity potential to be part of a replacement for traditional objectivity in reporting. I think it can help guide journalists past the “view from nowhere” and towards a “view from somewhere” as an ethical framework for their reporting. See page 17 or 183 of Damer.

The meaning of “objectivity” is slippery, but it can be invoked to defend journalism that attempts to “play the story down the middle,” or “give both sides of an issue,” while keeping journalists’ “opinions” out of the news.

My position about objectivity, as I already betrayed, has been shaped by Jay Rosen’s writing regarding the “View from Nowhere” in journalism. A “view from somewhere” seems to me more preferable than a view from nowhere, and my thoughts here regarding the principle of charity obviously reflect that assumption.

Advancing beyond objectivity

Traditional objectivity is seen as a practice that assists citizens in a democracy. In the words of journalist Ted Koppel, “objectivity … is presented to the public at large so that you out there have enough information that you can make intelligent decisions of your own.”

But there are signs that the grip of objectivity on mainstream journalism is loosening. An example on the larger scale is from earlier this year, when NPR released an Ethics Handbook that downplayed the need to create “the appearance of balance,” and instead prioritized “being fair to the truth.” NPR declared itself more willing to say when “the balance of evidence in a matter of controversy weighs heavily on one side.” You can read the text of the handbook at ethics.npr.org.

(Had I written the paper today, I would have mentioned the wave of thinking during this presidential cycle about fact-checking in political journalism and the “post-truth” era of politics. I added some relevant links to the bottom of this post.)

Journalists who would abandon objectivity as an ethical framework must also face the challenge of proposing an alternative framework to objectivity. This alternative framework must defeat fears, perhaps held by journalists themselves, that journalism without objectivity would descend into partisan yapping — that journalism without objectivity would cease to inform citizens about the issues of the day. For example, here is Ted Koppel (again) expressing such a fear.

I argue that the principle of charity can assist in the effort of developing an alternate framework for journalistic ethics. I think the principle helps advance past traditional objectivity, but while preserving objectivity’s interest in presenting in a fair manner the arguments that citizens should understand.

Journalists whose work is informed by the principle of charity can more usefully reconstruct the argument of a politician or pundit before critiquing it. They can reconstruct those arguments in a way that respects their readers’ intelligence and the efforts of those engaged in the public square.

The principle would warn journalists against, “taking cheap shots” or “nit-picking” an opponent’s argument, as they might do with, perhaps, the use of short or incomplete quotes, or by setting up a “straw man” argument. Michael Scriven wrote more about what the principle of charity requires in his textbook “Reasoning.”

The principle of charity obliges that we acknowledge the existence of challenges to our position. But it does not require that we show respect for thoset challenges by “balancing” them with our own position, as in traditional objectivity. It requires that we reconstruct those challenges as strongly and fairly as can be done, but not more.

The principle of charity, then, enables a journalist with a view from somewhere to say: “There are challenges to my position, but I’ve taken the time to study them, and I’ve tried to respect them. Even if I disagree with you, I understand you.”

How it looks to others

Journalists who embrace the principle of charity might also have to publicly confront some questions about it that scholars have investigated. For instance, when reconstructing an argument, how generous must a critic be? Should arguments be interpreted as accurately as possible as they were delivered — that is, warts and all — or should needed but missing premises be inserted on the arguer’s behalf to create a stronger position? (David Hitchcock wrote about these questions in 1996.)

Considering these questions would have some obvious intellectual benefits. But the benefits would also be public-facing. Demonstrating an interest in details of the principle would go far towards supporting the position that there is opportunity for humility and curiosity, even among journalists who abandon objectivity. The need not be cast into partisan quarreling.

‘Post-truth’ politics

I would never finish this post if I tried to update it with each of the recent posts about journalism, fact-checking, and “post-truth” politics. Suffice to say, here are some interesting and relevant links, some of which engage with something much like the principle of charity as I understand it.

The original pitch was for “the five biggest lies in Paul Ryan’s speech.” I said no. It’s not that the speech didn’t include some lies. It’s that I wanted us to bend over backward to be fair, to see it from Ryan’s perspective, to highlight its best arguments as well as its worst.

Thoughts about Wahl-Jorgensen’s ‘Strategic ritual of emotionality’

A few weeks ago, I sketched a post in response to Karin Wahl-Jorgensen’s article “The strategic ritual of emotionality”. But it seems I got delete-happy and erased the file.

I wanted to post about the article anyway because I found it stimulating. Here are some lightly edited notes and questions from my initial reading of the paper.

Replacing objectivity’s legal safeguard

I am less familiar with Gaye Tuchman’s work than I should be, but Wahl-Jorgensen’s article interested me in Tuchman’s work even more.

Wahl-Jorgensen summarizes Tuchman as arguing that reporters embraced traditional objectivity, with its standardized process, in part because it helped protect them from errors that could lead to expensive libel suits.

One position I recently took is that reporters who want to abandon traditional objectivity can use the principle of charity as part of a replacement ethical framework. The principle of charity, broadly stated, is that when we critize an argument we have an obligation to represent the argument in its strongest form.

If Tuchman is correct that reporters have embraced objectivity in part for its legal protections, then it seems to me fair and practical for reporters to ask whether the principle of charity would provide them similar shelter. I couldn’t answer that question today. But I have no interest in seeing a chilling effect follow from use of the principle. I’d like to research the legal literature to see whether it has addressed objectivity in the context of libel laws.

Where is the evidence?

Regina Lawrence and Matthew Schafer recently found that journalists who labeled Sarah Palin’s “death panels” claim false did so without attribution surprisingly frequently.

Similarly, Wahl-Jorgensen found evidence that journalists justfied claims, emotional or otherwise, without often rushing to use quotations as evidence, which, she says, the “objective” style might lead you to expect. Instead, reporters tended to rely on their epistemic authority: Their saying it was enough to justify it.

The two samples are not necessarily comparable. Wahl-Jorgensen studied Pulitzer-winning stories, and the other story examined general-purpose stories. Still, what is going on here?

Make critical thinking cool, the journalism way

Train your readers to expect a certain experience not just from your website, but from every source of news. Make sure that experience is either expensive or impossible for alternative sources to replicate. Newspapers need to make their readers expect proof of everything. People should feel uncomfortable trusting information without explicit, functional credibility.

Dan Schultz says newspapers must advance past a form of argument that results in “being a bunch of words with prose-based evidence.”

I think the point can be summarized as: If news organizations don’t make critical thinking cool, then someone else will. If someone else does, then news organizations will suffer.

Read Dan’s post: “The Value of a Super Villain.”