One of the most important points made in Chapter 9 concerns the interpretation of statistically significant correlations. This point is so important that I'd like to reiterate it here.
On page 217, I've cautioned you that "many researchers get carried away with the p-levels associated with their correlation coefficients" and that "discovering that a correlation coefficient is significant may not really be very important--even if the results indicate p<.01 or p<.001." Such a result, I argued, "may be significant in a statistical sense . . . but it may be quite insignificant in a practical sense."
Though you probably recall those statements (because Chapter 9 is still "fresh" in your mind), do you remember what was said at the end of Chapter 7? The final two paragraphs of that early chapter are worth considering once again.
Since you may not have our textbook handy, I'll type here the final 10 sentences of Chapter 7. I highly recommend that you carefully consider once again this important caution.
It is possible for a study to yield statistically significant results even though there is a tiny difference between the data and the null hypothesis. For example, in a recent study reported in the Journal of Applied Psychology, the researcher tested Ho: r = 0 within the context of a study dealing with correlation. After collecting and analyzing the sample data, this null hypothesis was rejected, with the report indicating that the result was "significant at the .001 level." The sample value that produced this finding was -.03!
Even if the issue being investigated is crucial, I cannot consider a correlation of -.03 to be very different in any meaningful way from the null value of 0. (With r = -.03, the proportion of explained variance is equal to .0009!) As you will soon learn, a large sample can sometimes cause a trivial difference to end up being statistically significant--and that is precisely what happened in the correlational study I am referring. In that investigation, there were 21,644 individuals in the sample. Because of the gigantic sample, a tiny correlation turned out to be statistically significant. Although significant in a statistical sense, the r of -.03 was clearly insignificant in terms of its importance.
Copyright © 2012
Schuyler W. Huck