Don't Be Misled By "p-Statements" (A)

If a correlation coefficient is reported to be statistically significant, look at the size of the r and ask yourself what kind of relationship (weak, moderate, or strong) was revealed by the researcher's data. Better yet, square the r and then convert the resulting coefficient of determination into a percentage; then make your own judgment as to whether a small or large amount of variability in one variable is being explained by variability in the other variable. If the study focuses on means rather than correlations, look carefully at the computed means. Ask yourself whether the observed difference between two means represents a finding that has practical significance. We cannot overemphasize our warning that you can be (and will be) misled by many research claims if you look only at p-statements when trying to assess whether results are important. Most researchers use the simple six-step version of hypothesis testing, and the only thing revealed by this procedure is a "yes" or "no" answer to the question,"Do the sample data deviate from Ho more than we would expect by chance?" Even if a result is statistically significant with p < .0001, it may be the case that the finding is completely devoid of any practical significance!

(From Chapter 8, p. 229)

Copyright © 2003 Schuyler W. Huck
All rights reserved.