X hits on this document





2 / 9

Hervé Abdi:

The Bonferonni and Šidák Corrections

same set of data. For example, brain imaging researchers will rou- tinely run millions of tests to analyze an experiment. Running so many tests increases the risk of false alarms. To illustrate, imagine the following “pseudo-experiment":

I toss 20 coins, and I try to force the coins to fall on the heads. I know that, from the “binomial test," the null hypothesis is rejected at the α = .05 level if the number of heads is greater than 14. I repeat this experiment 10 times.

Suppose that one trial gives the “significant" result of 16 heads versus 4 tails. Did I influence the coins on that occasion? Of course not, because the larger the number of experiments, the greater the probability of detecting a low-probability event (like 16 versus 4). In fact, waiting long enough is a sure way of detecting rare events!

2.1 Probability in the family

A family of tests is the technical term for a series of tests performed on a set of data. In this section we show how to compute the prob- ability of rejecting the null hypothesis at least once in a family of tests when the null hypothesis is true.

For convenience, suppose that we set the significance level at α=.05. For each test (i.e., one trial in the example of the coins) the probability of making a Type I error is equal to α = .05. The events “making a Type I error" and “not making a Type I error" are com- plementary events (they cannot occur simultaneously). Therefore the probability of not making a Type I error on one trial is equal to

1 α = 1 .05 = .95 .

Recall that when two events are independent, the probability of observing these two events together is the product of their proba- bilities. Thus, if the tests are independent, the probability of not making a Type I error on the first and the second tests is

.95 × .95 = (1 .05)2 = (1 α)2 .


Document info
Document views119
Page views125
Page last viewedFri Jan 20 14:09:51 UTC 2017