Category Archives: Evidence

Confirmation in a ‘threshold’ sense and in a ‘relevance’ sense

Professor Brandon Fitelson, appearing on the Elucidations podcast in January, introduced me to a useful distinction in forms of “confirmation,” as in some evidence and reasoning X “confirms” Y.

Suppose I offer you some evidence X that I say confirms claim Y. A response attuned to the distinction Fitelson described would ask: “Do you mean ‘confirmed’ in a threshold sense or in a relevance sense?”

Below I’ll try to describe the distinction as I understand it, first in a more abstract way and then with an example.

The distinction

The “threshold” sense of confirmation holds that X provides support for Y if X increases the probability of Y past some level. If we say Y is confirmed when it’s 90% likely (meaning, our threshold is 90%), and X pushes the likelihood of Y past 90%, then we can say X confirms Y.

The “relevance” sense of confirmation holds that X provides support for Y if X increases the probability of Y beyond what it was before we knew about X. In this case, X is “a difference-maker,” Fitelson says. It’s relevant to Y.

Fitelson offers the example of a deck of cards. We pick a card and want to demonstrate that it’s the ace of spades.

We learn that the card we picked is a black card. What does that tell us?

Well, it tells us that the probability of having picked the ace of spades just doubled: from 1/52 to 1/26.

From the “relevance” sense, “the card is black” provides support to the claim “the card is the ace of spades.” It’s now twice as likely that the card is the ace of spades — knowing that we picked a black card is a real difference-maker!

But under the “threshold” sense, knowing “the card is black” might tell us almost nothing. What if our threshold for confirming that “the card is the ace of spades” is even as low as 80 percent? We are closer, but “the card is black” cannot confirm “the card is the ace of spades.”

A mistake in my thesis

Fitelson discusses the threshold-relevance distinction in recounting some philosophical work by Rudolph Carnap and a subsequent critique by Karl Popper. Carnap confused the two senses of confirmation and Popper called Carnap out on it.

It’s a near-certainty (heh) that I made the same error in my results chapter in my thesis. Whether that is a fatal flaw, I don’t know, but would it not be something for reporters to think about clarifying in their work?


Medium and routines: Should they influence whether we accept arguments?

I read two academic articles earlier this week that got me thinking about whether, and how, our acceptance of an argument should be influenced by the conditions that the person producing the argument was in.

The first was Citizen Video Journalists and Authority in Narrative: Reviving the Role of the Witness by Mary Angela Bock. The article looked whether “citizen” and “professional” journalists differed in how they attempted to assert their authority in their work, given that “citizen” journalists lack the institutional authority most “professional” journalists enjoy.

The second was Epistemologies of TV journalism: A theoretical framework by Mats Ekström. Ekström’s proposed framework for studying TV journalism’s epistemology included questions about how TV journalists decide that a claim is one they want to put on the air, how they can gather and test facts, and what has to be true for the audience to accept the “knowledge” these processes produce.

Ignoring for a minute that these studies focus on only visual journalism, as I read I thought about myself, a journalism consumer (or user), reading or hearing the material that emerges from the places Bock and Ekström studied. Knowing what those scholars tell me about the images I see is very interesting. I want to know the questions that are in play about why the news I read is the way it is.

But what does this knowledge about the conditions of production have to do with whether I accept the arguments that journalists are offering me?

Say some journalists come out with some story with some conclusion, but initially I find their evidence unconvincing. Should I think differently knowing that the story was shaped by the conditions and institutions of journalism that Bock or Ekström describe (or, for that matter, those described by Gans, Tuchman, etc.)?

My first reaction is, “of course not” — no more than I would excuse the poor argument of, say, an economist just because the argument happened to be persuasive to people in the field. It strikes me as an explanation, but not an excuse.

But is my reaction too harsh? None of the journalists involved decided how the routines of journalism should be; in the case of TV journalism, none of them decided what sort of behavior and content the medium should encourage and what it discourages. Yet my reaction leads me to ignore them all the same.

Rupar’s research on evidence in newspapers: What to do?

As I said in my last post, my research into some of this year’s Pulitzer Prize-winning stories suggested that the most common problem in the stories, from a critical thinking perspective, was that they lacked evidence. I was surprised to find a lack of evidence, but maybe I shouldn’t have been.

In a study of 510 stories in New Zealand newspapers published in 2006, Veronica Rupar found that 62% of them did not “clearly describe[] the input of sources behind their stories.” These articles “only stated an opinion or gave quotes without indicating how the news was assembled … (example: ‘The Prime Minister thinks…’)” (130).

Why does Rupar’s finding interest me? Because it encourages me to think more about how to decide to accept something in the absence of evidence.

People citing how they knows what they claim is how a they get me to believe them. Without a citation of some sort, I have to take what they claim on trust.

Trust in what? Perhaps that he or she was at the scene and correctly understood what was said. Perhaps that the person spoken with was a reputable source about, or in a position to know, the thing in question.

Alternatively, if I won’t accept what the person says on trust, I could accept it based on past performance (was this person correct before? Did he try to be correct?) The claim might also go down more smoothly with the assumption that the speaker, journalist or otherwise, is generally well-intentioned.

In many ways, these questions are simplistic wrappers around research on vetting and accepting arguments from authority that have been discussed for millenia. Douglas Walton’s book on the subject is a great starting point; Thomas Haskell’s book also provided very interesting historical background for a newbie like I.

Rupar’s finding also interested me because of her demonstration that a news story could go from not citing sources to citing them with relative ease (133).

Admittedly, her example involves adding 16 words to what is only a 54-word story to start with. But that simple move would be all, I think, that’s necessary to ease many of my fears.