Sample Size and Power (A)
As we pointed out earlier in this chapter, the sample
size used by a researcher is one of the factors that influences whether
or not a false null hypothesis will be rejected. With large samples, it
is possible that a false Ho will be rejectedeven
if there is little or no practical significance associated with the findings.
Examples of that possibility were presented in Excerpts 8.2 and 8.4. In
each case, the sample size was so large as to give the researcher a high
probability of rejecting Ho even if the null was "off" by a
small margin. The sample size, if too large, will make the statistical
test too "powerful" in the sense that null hypotheses that are
false by a trivial amount will be declared statistically significant.
Such a finding has statistical significance but no practical significance.
As we have seen, a small strengthofassociation index or an observed
effect size provides a red flag that serves to alert the researcher and
you that an unimportant finding has been declared statistically
significant because of a large sample size.
The sample size, on the other hand, can be too small
and, as a consequence, cause the findings to be misleading. Due to the
fact that there is a direct relationship between the sample size and the
probability of rejecting a false Ho a statistical
test based upon an insufficient amount of data will likely lead to a failtoreject
decisioneven if the discrepancy between the arbitrary null hypothesis,
on the one hand, and the reality of the population(s), on the other hand,
is so large as to deserve the label important or noteworthy.
If a researcher doesn't reach a reject decision when Ho
is "off target" by a wide margin, then a major Type II error
is committed.
(From Chapter 8, p. 208)
