This post is part of a series (introduced here) in which I present objections to possible responses to the claim that likelihoodism is not a viable alternative to Bayesian and frequentist methodologies because it does not address questions about what to believe or do. I am currently considering the response that likelihoodism is a viable alternative methodology because characterizing one’s data as evidence is valuable in itself.
One argument for this claim is that the notion of rational belief is closely tied to the notion of evidential support. As Hume put it (with the sexism removed), “A wise [person] proportions his [or her] belief to the evidence.” The notion of rational belief is important for epistemology and methodology if anything is, so a methodology that correctly characterizes data as evidence is obviously valuable.
I addressed this argument here. In short, the problem with it is that likelihoodism explicates aspects of an incremental notion of evidence as good grounds for a shift in one’s beliefs, rather than an absolute notion of evidence as good grounds for belief itself. Thus, likelihoodism does not characterize data as evidence in a sense to which Hume’s dictum applies. The relevant dictum would instead be, “A wise person proportions shifts in his or her beliefs to the evidence.” But likelihoodist methods do not implement this dictum; Bayesian methods do. Likelihoodist methods characterize how beliefs should shift without attending to the beliefs themselves. They are for that reason potentially useful for the purpose of summarizing one’s data in a way that allows one’s audience members each to update his or her own subjective prior probability distribution, but using likelihoods in that way is an essentially Bayesian procedure with a Bayesian rationale that does not vindicate likelihoodism as a viable and genuinely distinctive methodological program.
A likelihood ratio is a ratio of posterior odds to prior odds under Bayesian updating. Likelihoodists agree with frequentists and disagree with Bayesians in that they claim that legitimate prior probabilities often do not exist. Unlike frequentists, they maintain that a likelihood ratio “means the same” regardless of whether prior probabilities are present or absent. Of course, they cannot mean that a likelihood ratio is a ratio of posterior odds to prior odds under Bayesian updating regardless of whether prior probabilities are present or not. What they say is that it retains its meaning as a measure of evidential favoring regardless of whether prior probabilities are present or not. But what is a measure of evidential favoring in the likelihoodist’s sense, if not a measure of the shift in one’s beliefs that the data warrant?
No likelihoodist to my knowledge has attempted to answer this question. Edwards (1972) and Royall (1997) each address it by arguing in effect that it does not need an answer. I address Royall’s argument (12-13 ) in this post.
The context for Royall’s argument is that it comes just after he presents what he calls a “canonical experiment” that he intends to be used to provide points of reference for understanding the significance of any likelihood ratio that might arise (11-12). The experiment is to draw balls with replacement from an urn containing either all white balls or equal numbers of black and white balls. A likelihood ratio of eight, for instance, can be understood as favoring the relevant hypothesis over the relevant competitor to the same degree that observing three white balls in a row favors the hypothesis that the urn contains all white balls over the hypothesis that it contains half white balls and half black.
Royall’s argument is a response to the objection to the use of this canonical experiment that it is not clear that the same likelihood ratio “represents the same strength of evidence in all contexts.” His initial comment on this objection is as follows (12):
These doubts come from failure to distinguish between the strength of the evidence, which is constant, and its implications, which vary according to the context of each application (prior beliefs, available actions, etc.).
He then points out that Bayes’s theorem guarantees that a likelihood ratio of k corresponds to a k-fold increase in probability ratio, “whether the prior probabilities are known or not.” Royall then acknowledges that some regard talk of prior probabilities for hypotheses that are not concerned with the outcome of a chance process as meaningless. According to such individuals, the relevant prior probabilities are not unknown; rather, they do not exist. At best, they represent idiosyncratic opinions that have no place in science.
The crucial part of Royall’s argument for present purposes is his response to that objection:
The numerical value of the likelihood ratio, which is given a precise interpretation in [cases in which known prior probabilities have a frequency interpretation] (via Bayes’s theorem), retains that meaning more generally…. The situation is analogous to that in physics where a unit of thermal energy, the BTU, is given concrete meaning in terms of water—one BTU is that amount of energy required to raise the temperature of one pound of water at 39.20F by 10F. But it is meaningful to measure thermal energy in BTUs in rating air conditioners and in other situations where there is no water at 39.20F to be heated. Likewise the likelihood ratio, given a concrete meaning in terms of prior probabilities, retains that meaning in their absence.
This analogy is faulty. Compare the heat equation (1) to the log-odds form of Bayes’s theorem (2):
where Q is the amount of heat added in BTUs; c is the specific heat of the substance in question in BTUs per pound times degrees Fahrenheit; m is its mass in pounds; ΔT is the temperature change in degrees Fahrenheit; B is the “bannage,” i.e., the logarithm of the likelihood ratio of the evidence for the pair of hypotheses in question; and ΔO is the difference between the posterior and prior log odds for those hypotheses. In words, (1) says that heat equals specific gravity times mass times temperature change, and (2) says that bannage equals change in log odds.
The problem with Royall’s analogy is that even though the BTU is defined with reference to one pound of water at 39.20F, Equation (1) applies to any substance at any temperature (within certain limits). The BTU “retains its meaning” in the absence of water at 39.20F because the heat equation tells us what the significance of a BTU is in other settings. The reference to one pound of water at 39.20F is used only as a convention for picking out one of the countably many permissible scales for measuring heat. By contrast, Equation (2) does not apply to hypotheses that lack prior probabilities. A reference to prior probabilities in an account of the likelihoodist notion of evidential favoring is essential in a way that a reference to water at 39.20F in an account of heat is not.
There is more to worries about whether the same likelihood ratio represents the same strength of evidence in all contexts than a failure to distinguish between strength of evidence and its implications. Equation (2) indicates a sense in which the same likelihood ratio does represent the same strength of evidence in all contexts, provided that prior probabilities are available. But when prior probabilities are available, likelihoodism is idle. To show that likelihoodism is a viable alternative to Bayesianism, one needs to show that likelihood ratios mean something when prior probabilities are absent. Royall’s analogy between evidential favor and heat fails to address this challenge.
Edwards, Anthony WF. Likelihood. Cambridge UP, 1984.
Royall, Richard M. Statistical Evidence: A Likelihood Paradigm. Chapman & Hall/CRC, 1997.
Want to keep up with new posts without having to check for them manually? Use the sidebar on the left to sign up for updates via email or RSS feed!