Week 11

Deborah G. Mayo - Professor Emerita of Philosophy, Virginia Tech

Statistical Inference as Severe Testing: Beyond Probabilism and Performance

I develop a statistical philosophy in which error probabilities of methods may be used to evaluate and control the stringency or severity of tests.  A claim is severely tested to the extent it has been subjected to and passes a test that probably would have found flaws, were they present. The severe-testing requirement leads to reformulating statistical significance tests to avoid familiar criticisms and abuses. While high-profile failures of replication in the social and biological sciences stem from biasing selection effects—data dredging, multiple testing, optional stopping—some reforms and proposed alternatives to statistical significance tests conflict with the error control that is required to satisfy severity. I discuss recent arguments to redefine, abandon, or replace statistical significance.