Science journalism in context: What would the arguments look like?

A study published recently in PLoS ONE concluded that science journalism fails to contextualize, over time, initial findings in medical studies. “Initial observations are often refuted or attenuated by subsequent studies,” the researchers write. But the subsequent studies receive less coverage than the initial findings. Whatever coverage subsequent studies do receive rarely references their relationship to the initial findings.

The relationship to argument in journalism here is of an “ought” question: “What ought journalists argue when they cover medical research?”

The researchers here contend that journalists ought to include a claim like, “this study [supports / refutes / questions / etc.] previous studies that…” Of course, links to previous coverage would be appropriate here, too.

Further down the line, we can judge the quality of the argument, in the form of the contextualization offered as well. What support is offered for the claim that the study supports or refutes or questions research X?

h/t: The Economist

Advertisements

Clay Shirky on online journalism ethics

Clay Shirky recently published an essay at Poynter about the challenge of fostering trust when ideas in the public sphere are no longer scarce.

A relevant excerpt:

The philosophy of news ethics — tell the truth to the degree that you can, fess up when you get it wrong — doesn’t change in the switch from analog to digital. What does change, enormously, is the individual and organizational adaptations required to tell the truth without relying on scarcity, and hewing to ethical norms without the ability to use force.

Incidentally, the idea expressed in that excerpt is similar to the conclusion drawn by Jane Singer in “Norms and the Network: Journalistic Ethics in a Shared Media Space.”

Fact-checking: Channeling Arendt

Instead of naming opinions lies, we are better served by good investigative reporting and opinion journalism that makes sound arguments and clarifies the stakes. A well-reasoned article that seeks to argue pro or contra can offer a depth of opinion and insight that far surpasses the gotcha journalism of fact checking. What is needed is not a demand for simple factual reporting, but a willingness to read and talk with people with whom one disagrees.

From “Fact Checking the Fact Checkers,” by “RB” at the Hannah Arendt Center. The argument that too strong a focus on “fact checking” leads to the belief that the world is simpler and cleaner than it is seems close to prima facie true. But why believing in a simpler world is attractive, and what kind of utility the belief brings, look like they’re deep in Arendt theory, particularly regarding a person’s need for some sort of coherence and stability. Where would one go to start learning more about Arendt’s thinking?

Journalism enhanced by the principle of charity

This post is a (somewhat modified) excerpt from a paper I presented a few weeks ago (PDF). For reasons I mention below, I think it’s also relevant to the vibrant recent discussions of about “post-truth” journalism.

The principle of charity

The principle of charity, broadly stated, is that when one responds to another person’s argument, one has a responsibility to respond to the strongest possible version of that argument. That can mean doing many things on the other person’s behalf, including supplying unstated assumptions or using more precise language. Essentially, the principle of charity gives the other person the benefit of the doubt. On this point, see pages 16 and 17 of Edward Damer’s “Attacking Faulty Reasoning.”

There are some practical reasons for employing the principle of charity to an argument, but primarily it is a move of fairness.

I think its purpose as an act of fairness gives the principle of charity potential to be part of a replacement for traditional objectivity in reporting. I think it can help guide journalists past the “view from nowhere” and towards a “view from somewhere” as an ethical framework for their reporting. See page 17 or 183 of Damer.

The meaning of “objectivity” is slippery, but it can be invoked to defend journalism that attempts to “play the story down the middle,” or “give both sides of an issue,” while keeping journalists’ “opinions” out of the news.

My position about objectivity, as I already betrayed, has been shaped by Jay Rosen’s writing regarding the “View from Nowhere” in journalism. A “view from somewhere” seems to me more preferable than a view from nowhere, and my thoughts here regarding the principle of charity obviously reflect that assumption.

Advancing beyond objectivity

Traditional objectivity is seen as a practice that assists citizens in a democracy. In the words of journalist Ted Koppel, “objectivity … is presented to the public at large so that you out there have enough information that you can make intelligent decisions of your own.”

But there are signs that the grip of objectivity on mainstream journalism is loosening. An example on the larger scale is from earlier this year, when NPR released an Ethics Handbook that downplayed the need to create “the appearance of balance,” and instead prioritized “being fair to the truth.” NPR declared itself more willing to say when “the balance of evidence in a matter of controversy weighs heavily on one side.” You can read the text of the handbook at ethics.npr.org.

(Had I written the paper today, I would have mentioned the wave of thinking during this presidential cycle about fact-checking in political journalism and the “post-truth” era of politics. I added some relevant links to the bottom of this post.)

Journalists who would abandon objectivity as an ethical framework must also face the challenge of proposing an alternative framework to objectivity. This alternative framework must defeat fears, perhaps held by journalists themselves, that journalism without objectivity would descend into partisan yapping — that journalism without objectivity would cease to inform citizens about the issues of the day. For example, here is Ted Koppel (again) expressing such a fear.

I argue that the principle of charity can assist in the effort of developing an alternate framework for journalistic ethics. I think the principle helps advance past traditional objectivity, but while preserving objectivity’s interest in presenting in a fair manner the arguments that citizens should understand.

Journalists whose work is informed by the principle of charity can more usefully reconstruct the argument of a politician or pundit before critiquing it. They can reconstruct those arguments in a way that respects their readers’ intelligence and the efforts of those engaged in the public square.

The principle would warn journalists against, “taking cheap shots” or “nit-picking” an opponent’s argument, as they might do with, perhaps, the use of short or incomplete quotes, or by setting up a “straw man” argument. Michael Scriven wrote more about what the principle of charity requires in his textbook “Reasoning.”

The principle of charity obliges that we acknowledge the existence of challenges to our position. But it does not require that we show respect for thoset challenges by “balancing” them with our own position, as in traditional objectivity. It requires that we reconstruct those challenges as strongly and fairly as can be done, but not more.

The principle of charity, then, enables a journalist with a view from somewhere to say: “There are challenges to my position, but I’ve taken the time to study them, and I’ve tried to respect them. Even if I disagree with you, I understand you.”

How it looks to others

Journalists who embrace the principle of charity might also have to publicly confront some questions about it that scholars have investigated. For instance, when reconstructing an argument, how generous must a critic be? Should arguments be interpreted as accurately as possible as they were delivered — that is, warts and all — or should needed but missing premises be inserted on the arguer’s behalf to create a stronger position? (David Hitchcock wrote about these questions in 1996.)

Considering these questions would have some obvious intellectual benefits. But the benefits would also be public-facing. Demonstrating an interest in details of the principle would go far towards supporting the position that there is opportunity for humility and curiosity, even among journalists who abandon objectivity. The need not be cast into partisan quarreling.

‘Post-truth’ politics

I would never finish this post if I tried to update it with each of the recent posts about journalism, fact-checking, and “post-truth” politics. Suffice to say, here are some interesting and relevant links, some of which engage with something much like the principle of charity as I understand it.

The original pitch was for “the five biggest lies in Paul Ryan’s speech.” I said no. It’s not that the speech didn’t include some lies. It’s that I wanted us to bend over backward to be fair, to see it from Ryan’s perspective, to highlight its best arguments as well as its worst.

Arguments from authority: ‘Be Careful Who You Quote’

Arguments from authority in journalism can be dangerous these days.

On the surface, these people look like legitimate experts. And I think many journalists don’t necessarily look at the motivations of their sources. It’s obvious when you’re talking to a political campaign, but other than that I don’t think reporters look closely enough. You have to check the background. What can you find that legitimates their expertise?

From “Be Careful Who You Quote,” an interview with Melanie Sloan in Nieman Reports.

The Niall Ferguson affair: Beyond fact-checkers to argument-checkers

The reaction to Niall Ferguson’s Newsweek attack against Barack Obama interested me as I tried to fill in the blank: It’s important that Newsweek published a poorly argued article because ______________________.

Some writers, taking the journalistic angle, focused on the fact that Newsweek employs no fact-checkers:

In the case of the links listed above, the blank is filled: “it illustrates Newsweek’s disrespect for facts and shows how the magazine has fallen into disrepute.” The Ferguson article is so shameful because Newsweek ignored basic journalistic standards.

I read some of the web’s debunking of Ferguson:

What I read in those articles was calling out of mostly Ferguson’s conflations, irrelevant claims, and misleading transitions, but not as often factual inaccuracies. More often it was rebuttal along the lines of this, from O’Brien:

“The most recent estimate for the difference between the net present value of federal government liabilities and the net present value of future federal revenues–what economist Larry Kotlikoff calls the true ”fiscal gap“–is $222 trillion.”

That’s a lot of trillions! But if our fiscal gap is “really” this many trillions, why can we borrow for 30 years for a real rate of 0.64 percent? It’s because this number is meaningless. First of all, it seems to project many decades of growth figures and budget decisions that we simply don’t know will happen. It assumes the Bush tax cuts never ever expire and that the healthcare cost curve never ever bends. This is like projecting, in 1942, that the Empire of Japan will rule the entire Asian continent for 70 years based on a few years of battle outcomes. It’s an interesting prediction, but it’s not an empirical vision of the future.

or this, from Fallows:

He presents an ominous chart showing that, if Obama is reelected, China’s economy might become bigger than America’s around the time he leaves office … [the chart] … What this chart demonstrates is not “a nation losing ground” but the reality that China has four times as many people as America does. When its overall economy exceeds ours, its per capita output will be only one-quarter as great. A historian would presumably know that the conscious strategy of every president from Richard Nixon through Barack Obama has been to encourage rather than thwart China’s continued development, on the reasoning that a poor and festering China would be more dangerous to the United States than one that is becoming richer.

O’Brien and Fallows present rebuttals to poorly justified claims. But they do not write that Ferguson’s “$222 trillion” or “China’s economy” data are factually wrong. They write that the data are irrelevant, misleading, unconvincing, etc. That requires a certain amount of training in not just economics, but argument.

Would a “fact-checker” have noticed the same thing, or would they have been content with Ferguson’s provision of a source for the figure and the chart?

Ta-Nehisi Coates writes that "a culture of fact-checking, of honesty, is as important as the actual fact-checking.

In my experience, seeing your name on the cover of a magazine will take you far in the journey toward believing your own bullshit. It is human to do so, and fact-checkers serve as a valuable check to prevent writers from lapsing into the kind of arrogant laziness which breeds plagiarism and the manufacture of facts.

But if there weren’t many “facts” out of sorts — as opposed to out-of-sorts arguments — then I don’t know how much the culture of fact-checking would have helped. What would have been more important is a culture that encourages you to have your reasons and evidence straight before opening your mouth, and I don’t know whether that’s the same thing. Maybe it is.

Then again, would a high-flying “celebrity” academic have been affected by such a culture of honesty? Might he “believe [at least some of] his own bullshit” when, as Stephen Marche noted, he takes $50,000 to $75,000 per speaking appearance?

Perhaps Ferguson’s view of himself wouldn’t matter. Suppose Newsweek had fact-checkers. Fallows supposes that they might have approached the work with a different lens than they would have Coates or some other journalist:

Scholars are supposed to be different from mere pamphleteers and journalists. We give the judgments of academics — like those of doctors, scientists, renowned jurists, etc. — extra weight because we assume that they have considered evidence, precedent, and probabilities more carefully before offering conclusions. Think: E.O. Wilson on ants and ecological patterns more broadly.

He almost — but only almost — makes the case that even fact-checkers wouldn’t have helped. Even if they had read the story, might they have made the same assumption that Fallows suggests and tread more softly around Ferguson’s work?

Finally, on the subject of intellectuals as valuable “brands,” and on the subject of a “higher standard” for scholars and academics, see Daniel W. Drezner’s post “Intellectual power and responsibility in an age of superstars”.

Thoughts about Wahl-Jorgensen’s ‘Strategic ritual of emotionality’

A few weeks ago, I sketched a post in response to Karin Wahl-Jorgensen’s article “The strategic ritual of emotionality”. But it seems I got delete-happy and erased the file.

I wanted to post about the article anyway because I found it stimulating. Here are some lightly edited notes and questions from my initial reading of the paper.

Replacing objectivity’s legal safeguard

I am less familiar with Gaye Tuchman’s work than I should be, but Wahl-Jorgensen’s article interested me in Tuchman’s work even more.

Wahl-Jorgensen summarizes Tuchman as arguing that reporters embraced traditional objectivity, with its standardized process, in part because it helped protect them from errors that could lead to expensive libel suits.

One position I recently took is that reporters who want to abandon traditional objectivity can use the principle of charity as part of a replacement ethical framework. The principle of charity, broadly stated, is that when we critize an argument we have an obligation to represent the argument in its strongest form.

If Tuchman is correct that reporters have embraced objectivity in part for its legal protections, then it seems to me fair and practical for reporters to ask whether the principle of charity would provide them similar shelter. I couldn’t answer that question today. But I have no interest in seeing a chilling effect follow from use of the principle. I’d like to research the legal literature to see whether it has addressed objectivity in the context of libel laws.

Where is the evidence?

Regina Lawrence and Matthew Schafer recently found that journalists who labeled Sarah Palin’s “death panels” claim false did so without attribution surprisingly frequently.

Similarly, Wahl-Jorgensen found evidence that journalists justfied claims, emotional or otherwise, without often rushing to use quotations as evidence, which, she says, the “objective” style might lead you to expect. Instead, reporters tended to rely on their epistemic authority: Their saying it was enough to justify it.

The two samples are not necessarily comparable. Wahl-Jorgensen studied Pulitzer-winning stories, and the other story examined general-purpose stories. Still, what is going on here?