A talk I recently gave prompted Mike Titelbaum to develop a purported counterexample to the Law of Likelihood. In this post I present the counterexample with minimal comments for the sake of eliciting reactions to it that are not influenced by my thoughts. I plan to follow up with my comments next week.

The Law of Likelihood says that evidence $E$ favors hypothesis $H_1$ over hypothesis $H_2$ if and only if $\Pr(E|H_1)>\Pr(E|H_2)$.

### The Counterexample

Here is Titelbaum’s counterexample, as he presented it to me in personal correspondence (shared with permission).

We’re playing Hearts (with a standard deck). At the beginning of the game Branden [Fitelson, who was also involved in the discussion,] passes me one card face down. I hate scoring any points in Hearts. It turns out that the Two of Hearts hardly ever yields points for its bearer, so if I receive the Two of Hearts from Branden I’m mildly annoyed. However if I receive a different heart, or the Queen of Spades, I’m really pissed off.

Now suppose you catch a glimpse of the card Branden passes me, and see only that it’s a heart. That’s your evidence; the two (mutually exclusive) hypotheses are that I’m mildly annoyed or that I’m really pissed off.

Mike claims that, intuitively, the fact ($E$) that the card is a heart favors the hypothesis ($H_1$) that he is really pissed off over the hypothesis ($H_2$) that he is mildly annoyed. However, **the Law of Likelihood says the opposite:** it says that $E$ favors $H_2$ over $H_1$ because $\Pr(E|H_2)=1>\Pr(E|H_1)=12/13$.

#### Please provide your reaction to Titelbaum’s example in the poll below. I would love it if you would also explain your response by leaving a comment.

[socialpoll id=”2203201″]

To share your thoughts about this post, comment below or send me an email.

Comments support Markdown formatting and $\LaTeX$ mathematical expressions (surround expressions with single dollar signs for in-line math or double dollar signs for display math).

Michael Lew says

Surely the evidence relates to the card, not to the consequent level of pissed offness. If the law of likelihood fails to provide a proper answer in this case it will be entirely because of the abstraction from the evidence space of cards which take many different values to the dichotomous space of mild annoyance versus really pissed off.

The evidence supports the hypothesis that the card is a heart, but supports all values of heart cards equally well. It does not support the hypothesis that the card is the queen of spades (in my family we call her Rickety Kate). Imagine a likelihood function that extends along four mutually exclusive axes, one for each suit and a level on each axis for each of the card face values. After the evidence the function is flat along the hearts axis and zero along the other axes.

The law of likelihood works well when applied to likelihood functions, but is susceptible to alleged counter-examples in cases like this where the evidence is removed from its natural space. If you stop thinking of hypotheses as philosophical objects and treat them as the parameter values that are subject to the likelihood principle then these misleading cases disappear.

Greg Gandenberger says

I’d say that it relates to both, but I’ll concede the claim for the sake of argument. Let $H_1$ be the hypothesis that the card is either a heart other than the two of hearts or the queen of spades, and let $H_2$ be the hypothesis that the card is the two of spades. I take it that Mike would still be inclined to say that $E$ favors $H_1$ over $H_2$. You’ll need some kind of restriction on the Law of Likelihood to keep it from saying that $E$ favors $H_2$ over $H_1$. What kind of restriction do you have in mind, and how will it do the work you need it to do? You seem to have in mind a restriction to “natural” hypothesis spaces, but that hypothesis space considered here seems entirely natural given the context.

Michael Lew says

To make the Law of Likelihood immune to counter-examples of this type all that is needed is to recognise that it applies only to points on a single likelihood function (as is implied, I think, by the connection between the Law of Likelihood and the Likelihood Principle).

You reworking of the example makes it fairly clear that you are trying to use the LL to compare a single point on a likelihood function (the function that I describe in my first comment) with two separate points on the function, one of which has the value of 1 and the other 0.

I recognise that you will think that I am proposing a ‘restriction’ but I think that I am doing nothing more than stating the the actual scope of the LL.

Greg Gandenberger says

I take it that you want to exclude $H_1$ because it is a disjunction of hypotheses in the “natural” hypothesis space given by the individual cards. But so what? Every hypothesis is equivalent to lots of disjunctions, and the notion of “naturalness” is hopelessly vague.

Michael Lew says

All I can say is that you do not seem to have imagined the four-dimensional function that I talked of in my original comment.

Greg Gandenberger says

What lesson do you draw from the shape of that function?

Michael Lew says

If you map the pissed-off points onto that function you see that most of the function relates to undefined states of pissed-offness (i.e. hearts cards that are not 2 of hearts). Thus the likelihood function has only one defined point that relates to the hypothesis space. As the evidential meaning of the function is expressed as the ratio of likelihoods. There is only one non-zero likelihood and so the evidence offers nothing quantifiable for your question.

If your hypotheses of interest were in the space of cards then the likelihood function is helpful.

Greg Gandenberger says

I’m missing something. The 2 of Hearts yields minor annoyance. Other hearts (and the queen of spades) yield pissed-offness. All other cards yield neither. Why do say that “most of the function relates to undefined states of pissed-offness?”

Michael Lew says

Sorry, I made the mistake of mapping the problem onto a real interpretation of cards where the degree of pissed-offness scales somehow with the face value of the card.

The likelihood function in question has only two points: really pissed of and mildly annoyed. The probability of the card being a heart if the result is that Mike is really pissed off is 12/13 and the probability of the card being a heart if Mike is mildly annoyed is 1/39. Thus the evidence supports the hypothesis really pissed off to the degree 39*12/13 or 36 to one. Thus observing a heart yields fairly strong evidence that Mike is really pissed off.

Matches my intuition.

Not a counter-example.

Greg Gandenberger says

Mike is only mildly annoyed if he gets a Two of Hearts. If he gets anything other than a heart or the Queen of Spades, then he is neither mildly annoyed nor really pissed off. Thus, the probability of the card being a heart if Mike is mildly annoyed is 1.

(Perhaps Mike’s emotions are not entirely apt given the rules of Hearts, which might make the example needlessly confusing for those who play it.)

Lewis Powell says

I am almost certainly wading in over my head here, but I am wondering about the following:

It seems like the Law of Likelihood, as stated, has the following implication:

Suppose I come across a puddle of water on the ground:

E = There is a puddle of water on the ground at location l1 at t1.

H1 = It rained at location l1 shortly before t1.

H2 = The Puddle-of-Water-at-t1 Demon (whose presence in a location at any time prior to t1 causes a puddle of water to appear there at t1) was at l1 prior to t1.

It seems like LoL says that E favors H2 over H1, since having rained shortly beforehand makes a puddle probable, but Puddly, the water-at-t1 demon, makes the puddle certain.

Is there some reason to think that the replies to the PUDDLY case won’t carry over to Titelbaum’s?

Again, this is not my area, so I might be missing a bunch of important features of the case, or completely misunderstanding the principle.

Greg Gandenberger says

I suspect that any plausible account of favoring that appeals only to features of the joint probability distribution over E, H1, and H2 will give you this result. If you want to avoid it then you will need to opt for a different kind of account, such as an account like Deborah Mayo’s that appeals to the probative capacity of a test procedure.

For what it’s worth, I’m inclined to say that E DOES favor H2 over H1. It doesn’t follow, of course, that we ought to believe H2 over H1.

Lewis Powell says

My thinking was that if that sort of result can be swallowed for Puddly cases (or, if we are to understand the “favors/supports” relation in such a way that the puddle lends more support to Puddly than to mild rainstorm,, then it is hard to see why Titelbaum’s case presents a special challenge.

Greg Gandenberger says

Ah, I think I see what you have in mind: Titelbaum’s intuition seems to be driven by the fact that the posterior probability of $H_1$ is much higher than that of $H_2$, but if we accept the idea that the puddle favors the demon hypothesis over the rain hypothesis then we’re committed to the possibility that a highly improbable hypothesis can be favored over a highly probable hypothesis. Is that the idea?

It might help to point of that there are plausible accounts of favoring that recover Mike’s intuition while still allowing for the possibility that a highly improbable hypothesis can be favored over a highly probable hypothesis. For instance, we could say that $E$ favors $H_1$ over $H_2$ if and only if $\Pr(E|H_1)/\Pr(E|\sim H_1)>\Pr(E|H_2)/\Pr(E|\sim H_2)$.

Jake Chandler says

FWIW, my own intuitions strongly favour the verdict delivered by the LL.

It’s not all that easy to introspect here, but I think that this feeling is rooted in the fact that the observation that the card is red rules out a possibility consonant with the hypothesis that Mike is really pissed off, but does not rule out any possibility consonant with the hypothesis that he is just mildly peeved. (Of course, this does not in itself seem sufficient to support the intuition: the fact that the relative probabilities of the remaining possibilities are not affected seems to be another pertinent condition at play.)

In my recent Synthese piece on contrastive support (http://link.springer.com/article/10.1007%2Fs11229-010-9845-9), I note that the relevant likelihood inequality requirement for mutually exclusive hypotheses is equivalent to the requirement that the evidence raise the probability of the favoured hypothesis conditional on the disjunction of the two hypotheses (I owe the observation to Branden Fitelson). This, I think, is perhaps the most intuitive route to the LL: favouring is probability-raising when one has restricted one’s attention to the two hypotheses under consideration.

Shivaram Lingamneni says

I don’t think this is a counterexample to the Law of Likelihood. Also, as a matter of general principle, I don’t think we should be able to produce a counterexample using the kind of scenario described here, where all the probabilities have straightforward frequency interpretations. If there is a plausible counterexample I would think that it would have to rest on intuitions about single-case primitive conditional probabilities, or some similar setting where clashing interpretations of probability could give rise to paradox.

My reading of the scenario is that the intuition that $E$ favors $H_1$ over $H_2$ is simply wrong, and it arises because the posterior probability $P(H_1 \mid E) = \frac{12}{13}$ is still so much higher than the posterior $P(H_2 \mid E) = \frac{1}{13}$. Our intuition conflates facts about the evidence with facts about the posteriors, because the likelihood ratio is small, the priors are so different, and the posterior still substantially favors $H_1$ — so the scenario points to something interesting about cognitive bias. (Perhaps it’s an instance of the “availability heuristic” or “representativeness heuristic” — maybe we imagine the twelve angry hearts more clearly than the lone queen of spades?) But the evidence really does (slightly) favor $H_2$. How could it be otherwise?

Branden Fitelson says

Thanks for the neat post, Greg — and to all for the interesting comments. I think it’s important to try to first clarify the explicandum we have in mind. I try to do this in some of my work on inductive logic. If we think of “argument strength” as a quantitative thing, which is sensitive to probabilistic relevance (and also respects entailments as “strongest arguments”), then it’s pretty clear that measures of “degree of argument strength” which violate (LL) — when they are used to define “favoring” — are better explicata than ones which satisfy (LL). I think Mike’s example is a nice illustration of this kind of “comparative argument strength” relation, which is definable in terms of underlying confirmation measures which violate (LL). This may or may not also be a good illustration of “evidential favoring”, depending on what that explicandum is supposed to be.

Jonathan Livengood says

Interesting post, Greg. I tried to make a comment over on Choice & Inference, where I first saw your puzzle, but the comment seems to have been eaten or something. Anyway, I’ll try again here, ’cause I’m interested to know what you think about my reactions.

So, we’re asked whether some evidence favors one hypothesis over another. I take it that the goal is to give an account of “favoring,” though I’m not sure what exactly that means or what exactly the constraints are on theorizing about “favoring.” Is the project to give a conceptual analysis or to use intuition to latch onto an abstract object or to produce something instrumentally valuable for some purpose or what? Bracketing the goal of the project for now, it seems like you want to (at minimum) offer some translations and then see whether the law of likelihood agrees with what the translation says. In that spirit, here are three readings of sentences like, “Some evidence E favors hypothesis H1 over hypothesis H2.”

Reading #1. Read, “Some evidence E favors hypothesis H1 over hypothesis H2,” as saying, “The posterior probability of H1 given E is greater than the posterior probability of H2 given E.” On the assumption that all of the cards in the deck are equally likely a priori, this first reading disagrees with the Law of Likelihood.

Reading #2. Read, “Some evidence E favors hypothesis H1 over hypothesis H2,” as saying, “Evidence E confirms H1 to a greater degree than it confirms H2.” Whether such a reading agrees with the Law of Likelihood depends on your choice of confirmation relation, but for at least some choices, like the naive c(H, E) = Pr(H | E) – Pr(H | ~E), this second reading also disagrees with the Law of Likelihood. (Again, assuming some natural priors.)

Reading #3. Read, “Some evidence E favors hypothesis H1 over hypothesis H2,” as saying, “Evidence E would confirm H1 to a greater degree than it would confirm H2 if the two hypotheses were a priori equally likely.” I take this reading to mean that we should imagine that Pr(H1) = Pr(H2) = 1/2 and then calculate Pr(H1 | E) and Pr(H2 | E). This third reading then agrees with the Law of Likelihood, though it also has the odd consequence that the prior probability of the evidence — the observation of a heart — is 25/26, rather than 1/4.

I don’t know what the best way to go is … I think I lean toward the second reading as the right general approach, but that of course leaves open the question of which confirmation relation is correct. And it might very well be that I would prefer a different approach if I had a better sense of the big-picture project.

Greg Gandenberger says

Thanks for the comments so far! I’ll give my responses in next Monday’s post. (In the meantime, you all should talk to each other! I’m just the host of this party.)