I have been arguing against the view that a likelihoodist methodology is a viable alternative to Bayesian and frequentist methodologies. A plausible response to this claim is that likelihoodist methods are useful because characterizing data as evidence is valuable in itself. To justify this claim a likelihoodist needs to explain what evidential favoring as measured by likelihood ratios is, if not the degree to which the datum in question warrants a Bayesian shift in one’s beliefs. If there is nothing more to the likelihoodist’s notion of evidential favoring than this, then likelihoodism is merely a footnote to Bayesianism: all of its proper uses have an underlying Bayesian rationale and justification.
Edwards (1972) and Royall (1997) each argue in effect that there is no need to articulate the notion of evidential favoring in the sense in which likelihood ratios measure it. I criticized Royall’s argument in a previous post. I criticize Edwards’s argument here.
Edwards’s Likelihood (1972) was the first sustained attempt to argue for using (logarithms of) likelihood ratios as a measure of evidential favoring, or support as he called it. Immediately after presenting this proposal and illustrating it with an example, Edwards raises the objection that “the measure of support does not have any ‘meaning’” (33).
Edwards offers two quick and clearly inadequate responses to this claim before offering a third response that requires more careful consideration. He begins by claiming that what those who complain that his measure of support lacks a meaning have in mind is that it lacks a probability interpretation. He does not regard this fact as damaging to his approach because his approach is meant to appeal to those who are suspicious of probability statements that cannot be interpreted in terms of objective frequencies. This response is inadequate because I have not assumed that the interpretation of the likelihoodist’s measure of evidential support must be probabilistic. I just ask what that interpretation is.
Edwards then offers the following as an “operational interpretation” of a likelihood ratio: “the ratio of the frequencies with which, in the long run, the two hypotheses [in question] will deliver the observed data.” This interpretation is correct but not operational in any significant sense. The hypotheses in question are incompatible, so the ratio of frequencies to which Edwards refers is a ratio of a frequency in one possible world to a frequency in a different possible world. The question of why reporting such a ratio is a useful practice–other than for purposes of Bayesian updating–remains unanswered.
After presenting these two clearly inadequate responses, Edwards presents the following rich passage:
…provided that support enables us to operate a logically-sound system of inference of undoubted relevance to the assessment of rival hypotheses, it will acquire a meaning as experience of its use accumulates. For many years temperature, as measured by Fahrenheit, had no ‘meaning’ other than as an arbitrary scale conforming to an ordered sequence. Boiling water is not to be regarded as 6.6 times as hot as freezing water. But the measurement of temperature was nevertheless very important to the advancement of physics, and led ultimately, through the concepts of absolute zero and molecular movement, to a much deeper understanding of heat. The numerical assessment of rival hypotheses may be expected to be of equal benefit.
Edwards’s claim that his measure of support “will acquire a meaning as experience of its use accumulates” has many possible interpretations. Edwards’s mentions of absolute zero and of the ratio of the temperature of boiling water to that of freezing water suggest the view that future discoveries will make the scale on which we measure evidential support scale richer.
Edwards provides no argument for this claim. I find it implausible and doubt its significance. There is already a loose analog to absolute zero on any scale for evidential support that the Law of Likelihood permits (that is, any monotone function of the likelihood ratio), namely the point of evidential neutrality (1 on the likelihood ratio scale, 0 on the log-likelihood ratio scale). This point does not mark one end of an evidential favoring scale in the way that absolute zero marks one end of a temperature scale because there is no lowest or highest possible degree of evidential support. It is merely one natural reference point on the scale. One could provide additional reference points, for instance by using something like Royall’s “canonical experiment”, but the use of such reference points presupposes the very point at issue, namely whether or not a given likelihood ratio has the same meaning across contexts that makes no explicit or implicit reference to prior probabilities.
There is also a analog to one temperature being 6.6 times as hot as another. Consider a sequence $X_1,X_2,\ldots,X_n$ of independent and identically distributed random variables. For any possible value $x_0$ of these variables and any pair of hypotheses $H_1$ and $H_2$ about their common distribution, we can say that $X_i=X_j=x_0$ for $i\neq j$ favors $H_1$ over $H_2$ (or vice versa) twice as strongly as $X_i=x_0$. Accordingly, the likelihood ratio for $H_1$ and $H_2$ of $X_i=X_j=x_0$ would be the square of that for $X_i=x_0$. More conveniently, the log of the likelihood ratio for $H_1$ and $H_2$ of $X_i=X_j=x_0$ would be twice that for $X_i=x_0$. In general, ratios of logarithms of likelihood ratios can be regarded as ratios of degrees of evidential favoring.
Edwards seems to suggest that the scale on which we measure evidential favoring will become richer over time, and that this process will make the notion of evidential support more “meaningful.” But the log-lkelihood ratio scale for evidential support already has a privileged zero and meaningful ratios, and neither of those facts seems to help with the basic problem of articulating what evidential favoring in the likelihoodist sense could be if not the degree to which the data warrant a shift in one’s beliefs in accordance with Bayes’s theorem.
Edwards also claims that measuring likelihood ratios will lead to progress in methodology in the same way that measuring temperature made possible a great deal of progress in physics. Physicists began measuring temperature long before they understood what it was. In the same way, methodologists might begin measuring evidential favoring before they fully understand what it is. They can expect to learn a great deal in the process, just as physicists learned a great deal from measuring temperature.
Edwards’s claim that measuring likelihood ratios will lead to great progress in methodology is an unsubstantiated bit of speculation that receives little support from the analogy with temperature. Temperature is a property of matter on a macroscopic scale. Understanding it required developing theories about matter on a microscopic scale. Those theories have been enormously successful and influential. Why think that investigating evidential favoring will be similarly fruitful? There is no plausible candidate for an analog to the theories of matter on a microscopic scale that were necessary for understanding temperature.
Edwards continues the analogy between temperature and support as follows:
…just as our feeling of warmth does not depend on the air temperature alone, so our assessment of hypotheses will not depend on the support alone. In the former case our impression will be affected by the wind, the humidity, the sun, our clothing, and a host of other factors; but the temperature will inform us about one particular factor. In the latter case, though by the likelihood axiom the support will inform us fully of the contribution to our judgement that the data can make, we shall also be influenced by the simplicity of the hypotheses, by their relevance to other situations, and by a multitude of subtle considerations that defy explicit statement.
This passage suggests the view that as one becomes accustomed to using likelihood ratios to measure evidential support, one will “develop a feel for them.” An American traveling abroad eventually develops a feel for the centigrade scale which allows him or her to decide what to wear on the basis of a local weather report without converting degrees Celsius to degrees Fahrenheit. Similarly, Edwards seems to be claiming, someone who begins to use (log-)likelihood ratios as a measure of support will eventually develop a feel which allows him or her to decide (I suppose) what to believe and do on the basis of those ratios.
This view has many problems. It does not address the worry that the only meaning support has is as a measure of the degree to which the data warrant shifts in one’s beliefs. And it fails to acknowledge the fact that air temperature is far more informative about what it feels like outside than support is about what degrees of belief are warranted. Edwards should not be comparing support to temperature, but to heat. Support warrants changes in belief in much the same way (see this post) that heat brings about changes in temperature. Deciding what to believe or do based on evidential support is more like deciding what to wear on the basis of information about recent flows of heat into one’s vicinity than it is like deciding what to wear on the basis of the temperature. To use information about heat flows in deciding what to wear, one would want to know the prior temperature. In the same way, using information about support to decide what to believe or do makes little sense with prior probabilities, as likelihoodists themselves insist when trying to defend their theory against counterexamples. The obvious way to account for the “multitude of subtle considerations” besides support that influence one’s judgement is to adopt prior probabilities that reflect them. As numerous examples (such as those discussed here) indicate, using support as a basis for inference or decision-making requires more than “developing a feel” for it. It seems to require adopting a fully Bayesian approach.
Log-likelihood ratios have a meaning in the presence of prior probabilities as the difference between the posterior log-odds and the prior log-odds of the pair of hypotheses in question. Edwards’s analogy with temperature fails to vindicate the crucial likelihoodist claim that they have some definite meaning that does not appeal even implicitly to prior probabilities which makes them useful in their absence.
Edwards, Anthony WF. Likelihood. Cambridge UP, 1984.
Want to keep up with new posts without having to check for them manually? Use the sidebar on the left to sign up for updates via email or RSS feed!