Finding the Power of a Hypothesis Test

When you make a decision in a hypothesis test, there’s never a 100 percent guarantee you’re right. You must be cautious of Type I errors (rejecting a true claim) and Type II errors (failing to reject a false claim). Instead, you hope that your procedures and data are good enough to properly reject a false claim.

The probability of correctly rejecting H₀ when it is false is known as the power of the test. The larger it is, the better.

Suppose you want to calculate the power of a hypothesis test on a population mean when the standard deviation is known. Before calculating the power of a test, you need the following:

• The previously claimed value of

  

  in the null hypothesis,

  

• The one-sided inequality of the alternative hypothesis (either < or >), for example,

  

• The mean of the observed values

  

• The population standard deviation

  

• The sample size (denoted n)

• The level of significance

  

To calculate power, you basically work two problems back-to-back. First, find a percentile assuming that H₀ is true. Then, turn it around and find the probability that you’d get that value assuming H₀ is false (and instead H_a is true).

1. Assume that H₀ is true, and

   

2. Find the percentile value corresponding to

   

   sitting in the tail(s) corresponding to H_a. That is, if

   

   then find b where

   

   If

   

   then find b where

   

3. Assume that H₀ is false, and instead H_a is true. Since

   

   under this assumption, then let

   

   in the next step.

4. Find the power by calculating the probability of getting a value more extreme than b from Step 2 in the direction of H_a. This process is similar to finding the p-value in a test of a single population mean, but instead of using

   

   you use

   

Suppose a child psychologist says that the average time that working mothers spend talking to their children is 11 minutes per day. You want to test

versus

You conduct a random sample of 100 working mothers and find they spend an average of 11.5 minutes per day talking with their children. Assume prior research suggests the population standard deviation is 2.3 minutes.

When conducting this hypothesis test for a population mean, you find that the p-value = 0.015, and with a level of significance of

you reject the null hypothesis. But there are a lot of different values of

(not just 11.5) that would lead you to reject H₀. So how strong is this specific test? Find the power.

1. Assume that H₀ is true, and

   

2. Find the percentile value corresponding to

   

   sitting in the upper tail. If p(Z > z_b) = 0.05, then z_b = 1.645. Further,

   

3. Assume that H₀ is false, and instead

   

4. Find the power by calculating the probability of getting a value more extreme than b from Step 2 in the direction of H_a. Here, you need to find p(Z > z) where

   

   Using the Z-table, you find that

   

Hopefully, you were already feeling good about your decision to reject the null hypothesis since the p-value of 0.015 was significant at an

of 0.05. Further, you found that Power = 0.6985, meaning that there was nearly a 70 percent chance of correctly rejecting a false null hypothesis.

This is just one power calculation based on a single sample generating a mean of 11.5. Statisticians often calculate a “power curve” based on many likely alternative values. Also, there are some unique considerations to take into account if

but this gives you the gist of things.