Redefining the Critical Value of Significance Level (0.005 instead of 0.05): The Bayes Trace

Abstract

In 2017, a group of the leading mathematical statisticians published a paper-manifesto having an extremely simple sense: the common critical level of p-values should be decreased by an order of magnitude (0.005 instead of 0.05) (Benjamin, et al., 2017). In this review, the arguments of proponents and opponents of this proposal are discussed. Moreover, the problems related to the “reproducibility crisis” of the scientific results are considered. The corresponding argumentation cannot be understood without consideration of the fundamentals of the theory of statistical derivation. In this connection, the precise sense of some concepts, such as p-value, the Bayes factor, and the minimum a posteriori probability of the zero hypothesis are discussed in the review. This is made mainly with the examples related to the comparison of frequencies. It was shown that, when using p-values, particular attention should be paid to the comparison of low frequencies on the highly abundant samples. Some practical recommendations on application of the Bayes analysis are given.