Likelihoodists admit that their methods are not useful for guiding belief and action directly. One could maintain that characterizing data as evidence is valuable in itself, apart from any possible use in guiding belief or action, but this view is remarkably indifferent to practical considerations. I do not know how else to argue against it, but I do know how to respond to various likelihoodist attempts to make it seem plausible (as I have done in posts here, here, and here).

Royall (2000) seems to provide some reason to believe that likelihoodist methods can in fact reasonably be used to guide belief and action when prior probabilities are not available. He claims that “a paradigm [for statistics] based on [the Law of Likelihood] can generate a frequentist methodology that avoids the logical inconsistencies pervading current methods while maintaining the essential properties that have made those methods into important scientific tools” (31). In other words, likelihoodist methods are warranted by their long-run operating characteristics in the same way that frequentists take their methods to be, without being subject to the many objections that frequentist methods face (such as that they violate the Likelihood Principle).

In fact, likelihoodist methods are not warranted by their operating characteristics in the same way that frequentist methods purport to be. The strongest kind of frequentist justification a method can have is that it is the best among some class of methods on some (typically worst-case) performance criterion. For instance, a uniformly most powerful level $\alpha$ hypothesis test is justified by the fact that it would reject the null hypothesis at least as often in repeated applications in the long run as any other test under any scenario within the model in which that hypothesis is false among all tests that would reject it no more than $100\alpha\%$ of the time if it were true. (In this case, the class of methods is those with Type I error rate no greater than $\alpha$, and the performance criterion is that of being a most powerful test under each alternative hypothesis. This criterion is not merely worst-case, but it includes the worst case.) The key feature of this kind of justification for present purposes is that it is *comparative*. It answers questions of the form, “Why use method $\mu$ rather than some other method?” by pointing out some respect in which $\mu$ is better than every other method.

There are many objections to appeals to frequentist performance, including objections to the performance criteria used and worries about the relevance of those criteria to the evaluation of a single case. Those objections are not relevant to my present claim, which is that such justifications do not apply to likelihoodist methods even if they are legitimate.

The problem with likelihoodist performance justifications from a frequentist perspective is that they are not comparative. They merely show that some method performs well in some sense in repeated applications in the long run. They do not show that some method is better than its competitors in any way. Thus, they are insufficient to answer questions of the form, “Why use method $\mu$ rather than some other method?”

The most often cited likelihoodist performance justification is the *universal bound*: $\mbox{Pr}(X=x:\mbox{Pr}(x;H_2)/\mbox{Pr}(x;H_1)\geq k;H_1)\leq 1/k$. That is, for fixed $H_1$ and $H_2$, the probability that an experiment yields a likelihood ratio of at least $k$ for $H_2$ against $H_1$ when $ H_1$ is true is at most $1/k$. Thus, the Law of Likelihood has the nice property that for fixed $H_1$ and $H_2$, one of which is true, an arbitrary experiment is highly unlikely to produce a result that is according to the Law of Likelihood strong evidence for the one that is false.

This result is indeed nice, but it does not suffice to warrant using the Law of Likelihood rather than some other method because it does not compare the Law of Likelihood to any other method. There could be (and often are) many other methods in addition to the Law of Likelihood that also achieve the universal bound. Why should we use the Law of Likelihood rather than one of those other methods? Likelihoodists do not have a performance-based answer to that question.

Tighter bounds on the probability of misleading evidence than the universal bound are available in some cases (Royall 2000), but those bounds are also non-comparative. In order to appeal to performance characteristics to justify their methods, likelihoodists need to show that they achieve better performance than other possible methods in some respect.

One might object that methods with frequentist justifications are often based on likelihood ratios. For instance, the Karlin-Rubin theorem states that when one’s hypothesis space has a monotone likelihood ratio, a test that rejects a point null against a one-sided composite alternative if and only if the likelihood ratio for the null against a pre-specified element of the alternative falls below a given cutoff value is a uniformly most powerful level $\alpha$ test, where $\alpha$ is its Type I error rate. It might seem that this result vindicates likelihoodism. However, it applies only when the null hypothesis and test procedure are predesignated. Thus, it does not vindicate the likelihoodist practice of interpreting any likelihood ratio on likes as a measure of evidential favoring.

Royall regards the universal bound and other likelihoodist performance criteria as relevant for pre-trial planning rather than for post-trial evaluation and does not give them great emphasis in his efforts to promote a likelihoodist approach to statistics. Nonetheless, he sometimes suggests in his writings that such results justify likelihoodist methods in the same way that frequentist performance characteristics purport to justify frequentist methods. It is this claim that I am rejecting. Likelihoodism is not a frequentist methodology.

Want to keep up with new posts without having to check for them manually? Use the sidebar on the left to sign up for updates via email or RSS feed!

## Leave a Reply