Practical versus Statistical Significance
In Excerpts 16.29 and 16.30 [not shown here], we see two cases in which researchers attended to the important distinction between useful and trivial findings. The researchers associated with the first excerpt set a good example by conducting an a priori power analysis in the planning stage of their investigation. The researchers associated with the second of these excerpts deserve high praise for realizing (and warning their readers) that inferential tests can yield results that are statistically significant without being important in a practical manner.
In many research reports, researchers make a big deal about a finding that seems small and of little importance. Perhaps such researchers are unaware of the important distinction between practical and statistical significance, or it may be that they know about this distinction but prefer not to mention it due to a realization that their statistically significant results do not matter very much. Either way, it is important that you keep this distinction in mind whenever you are on the receiving end of a research report. Remember, you have the right to evaluate a statistical finding as having little or no meaningfulness after you examine the research report's summary statistics, and you can draw such a conclusion even if your opinion is at odds with those of the the researchers.
(From Chapter 16 in the 6th edition, pp. 400-401)
Copyright © 2012
Schuyler W. Huck