Professor Brandon Fitelson, appearing on the Elucidations podcast in January, introduced me to a useful distinction in forms of “confirmation,” as in some evidence and reasoning X “confirms” Y.
Suppose I offer you some evidence X that I say confirms claim Y. A response attuned to the distinction Fitelson described would ask: “Do you mean ‘confirmed’ in a threshold sense or in a relevance sense?”
Below I’ll try to describe the distinction as I understand it, first in a more abstract way and then with an example.
The “threshold” sense of confirmation holds that X provides support for Y if X increases the probability of Y past some level. If we say Y is confirmed when it’s 90% likely (meaning, our threshold is 90%), and X pushes the likelihood of Y past 90%, then we can say X confirms Y.
The “relevance” sense of confirmation holds that X provides support for Y if X increases the probability of Y beyond what it was before we knew about X. In this case, X is “a difference-maker,” Fitelson says. It’s relevant to Y.
Fitelson offers the example of a deck of cards. We pick a card and want to demonstrate that it’s the ace of spades.
We learn that the card we picked is a black card. What does that tell us?
Well, it tells us that the probability of having picked the ace of spades just doubled: from 1/52 to 1/26.
From the “relevance” sense, “the card is black” provides support to the claim “the card is the ace of spades.” It’s now twice as likely that the card is the ace of spades — knowing that we picked a black card is a real difference-maker!
But under the “threshold” sense, knowing “the card is black” might tell us almost nothing. What if our threshold for confirming that “the card is the ace of spades” is even as low as 80 percent? We are closer, but “the card is black” cannot confirm “the card is the ace of spades.”
A mistake in my thesis
Fitelson discusses the threshold-relevance distinction in recounting some philosophical work by Rudolph Carnap and a subsequent critique by Karl Popper. Carnap confused the two senses of confirmation and Popper called Carnap out on it.
It’s a near-certainty (heh) that I made the same error in my results chapter in my thesis. Whether that is a fatal flaw, I don’t know, but would it not be something for reporters to think about clarifying in their work?