Although this on-line resource is only text-based (and not interactive),
it's packed with interesting information about planned and post
hoc tests. For example, it contains a chart that shows clearly
why an inflated Type I error occurs, it presents lots of examples
from applied studies, and it offers tips for interpreting the
p-levels you'll see in research reports.
What to Do:
Click on the colored title of this on-line resource: "Multiple
Read the material on the screen that pops up.
Carefully consider the chart that shows how the probability
of making a Type I error increases dramatically (over the
nominal alpha level) as the number of tests being conducted
Sky Huck's Puzzle Question:
Which researcher (A or B) would be more likely to commit a Type
I error? Researcher A uses the hypothesis testing procedure just
once to test a null hypothesis that's true (although she doesn't
know that), and she sets alpha equal to .20 when making this test.
Researcher B uses the hypothesis test procedure 6 times to evaluate
a half dozen independent null hypotheses that are all true (although
he doesn't know that), and he sets alpha equal to .05 when making
each of these 6 tests.