The Sense in Which Frequentist Methods Violate the Likelihood Principle
It is widely accepted that frequentist methods violate the Likelihood Principle. After all, there can be two pieces of data A and B such that the Likelihood Principle implies that A and B are evidentially equivalent with respect to the set of hypotheses H, yet frequentist methods will yield different conclusions about H depending on whether A or B is fed into them.
But What About Using Frequentist Considerations to Choose Among Priors?
There is another sense in which many frequentist methods do not violate the Likelihood Principle. A frequentist method is often equivalent (in a sense) to a Bayesian method with a particular prior probability distribution. From a Bayesian perspective, such frequentist methods involve updating a prior probability distribution in a way that does conform to the Likelihood Principle. They violate the Likelihood Principle only by using “implied priors” that vary with the sampling distribution of the experiment to be performed.
Choosing a prior probability distribution according to the sampling distribution of the experiment to be performed looks odd from an orthodox subjective Bayesian perspective. From that perspective, a prior probability distribution should represent one’s degrees of belief prior to the experiment being performed. Those degrees of belief should not depend on the design of the experiment in the usual case in which the design is not informative.
However, suppose that one’s degrees of belief are appropriately represented not with a single probability distribution, but with a set of probability distributions. And suppose that updating that set in the usual way for some experiment does not yield a definite result for some inference or decision problem of interest. Then it seems fairly reasonable to appeal to frequentist considerations to decide which of those distributions to use for that problem, and thus to use the implied frequentist prior for whichever experiment is actually being performed, provided that it is in the set of distributions that represent one’s doxastic state. This approach violates the Likelihood Principle in the sense that it makes one’s conclusion depend on the experimental design. However, it violates the Likelihood Principle only because it uses the performance characteristics of that distribution under repeated sampling as a way to “break the tie” among the prior distributions over which one is indifferent, when forced to do so in order to make a decision or draw an inference.
A Similar Argument Can Be Made in Defense of Objective Bayesian Methods
The same kind of argument can be used to defend objective Bayesian methods against the charge that they are inappropriate because they too violate the Likelihood Principle by selecting (in this case explicit) priors according to a procedure that varies with the sampling distribution.
Question: Is it really a violation of the Likelihood Principle to use a frequentist method in this way? If so, should we prohibit it?
To share your thoughts about this post, comment below or send me an email.
Comments support Markdown formatting and $\LaTeX$ mathematical expressions (surround expressions with single dollar signs for in-line math or double dollar signs for display math).