The Bonferonni and Šidák Corrections for Multiple Comparisons
The more tests we perform on a set of data, the more likely we are to reject the null hypothesis when it is true (i.e., a “Type I” error). This is a consequence of the logic of hypothesis testing: We reject the null hypothesis if we witness a rare event. But the larger the number of tests, the easier it is to find rare events and therefore the easier it is to make the mistake of thinking that there is an ef- fect when there is none. This problem is called the inflation of the alpha level. In order to be protected from it, one strategy is to cor- rect the alpha level when performing multiple tests. Making the alpha level more stringent (i.e., smaller) will create less errors, but it may also make it harder to detect real effects.
2 The different meanings of alpha
Maybe it is because computers make it easier to run statistical analy- ses that researchers perform more and more statistical tests on a
1In: Neil Salkind (Ed.) (2007). Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage. Address correspondence to: Hervé Abdi Program in Cognition and Neurosciences, MS: Gr.4.1, The University of Texas at Dallas, Richardson, TX 75083–0688, USA E-mail: firstname.lastname@example.org http://www.utd.edu/∼herve