I’m trying out the idea of reframing my dissertation project as an investigation of conflicts between two desiderata for a theory of inductive inference that can be characterized roughly as follows:
Evidence-Tracking: If two pieces of evidence are evidentially equivalent with respect to a partition of the space of hypotheses under consideration, then a method should give the same outputs over that partition given either of those pieces of evidence, all else being equal.Objective Reliability: A method should have good operating characteristics regardless of which of the hypotheses under consideration is true.
In the first part of the project, I’ll flesh out a more precise notion of evidence-tracking by arguing that strong intuitions about evidential equivalence lead to the Likelihood Principle. In the second part, I’ll investigate various kinds of objective reliability that conflict with or have been said to conflict with that precisified notion of evidence-tracking. In each case, I’ll aim to articulate the relevant sense of objective reliability as clearly as possible and to say as much as I can about how compelling a desideratum it is.
I’m happy to acknowledge that one might arrive at a notion evidence-tracking that conflicts with the Likelihood Principle by choosing different starting points, e.g. by starting with the idea that some kind of objective reliability should act as a constraint on our account of evidence, even if that means revising intuitions such as those that lead to the Likelihood Principle. I’m not interested in arguing about how best to use the word “evidence.” I am interested in investigating the theory of evidence one gets out of the axioms that lead to the Likelihood Principle and thinking about how that theory of evidence should or should not shape statistical practice.
I’m anticipating objections to the phrase “objective reliability.” A more accurate phrase would be “objective reliability relative to a set of assumptions.” The kinds of assumptions I have in mind (e.g., that the data come from a distribution with a particular parametric form) are objective in the weak sense that (unlike a prior probability distribution) they are propositional and typically open to empirical investigation. I’m not opposed to the use of prior probability distributions in general even when they cannot be given a frequency interpretation, but I am interested in how far we can get without them.
Want to keep up with new posts without having to check for them manually? Use the sidebar on the left to sign up for updates via email or RSS feed!