When conducting a statistical test, too often people jump to the conclusion that a finding “is statistically significant” or “is not statistically significant.” Although that is literally true, it doesn't imply that only two conclusions can be drawn about a finding.
What if in the real world no relationship exists between the variables, but the test found that there was a significant relationship? In this case, you would be making a false positive error because you falsely concluded a positive result (you thought it does occur when in fact it does not).
On the other hand, what if in the real world a relationship does exist between the variables, but the test found that there was no significant relationship? In this case, you would be making a false negative error, because you falsely concluded a negative result (you thought it does not occur when in fact it does).
| In the Real World | Statistical Test Results | |
|---|---|---|
| Not Significant (p > 0.5) | Significant (p < 0.5) | |
| The two groups are not different | The null hypothesis appears true, so you conclude the groups are not significantly different. | False positive. |
| The two groups are different | False negative. | The null hypothesis appears false, so you conclude that the groups are significantly different. |
