You Can't Make a Silk Purse Out of a Sow's Ear

We cannot overemphasize how important it is to critically assess the worth of the hypotheses being tested within any study based upon a two-way ANOVA. No matter how good the study may be from a statistical perspective and no matter how clear the research report is, the study cannot possibly make a contribution unless the questions being dealt with are interesting. In other words, the research questions that prompt the investigator to select the factors and levels of the two-way ANOVA must be worth answering and must have no clear answer before the study is conducted. If these things do not hold true, then the study has a fatal flaw in its foundation that cannot be overcome by large sample sizes, rigorous alpha levels, impressive reliability and validity estimates, impressive F-ratios that are statistically significant, elaborate post hoc analyses, tests of assumptions, and power analyses. The old saying that "you can't make a silk purse from a sow's ear" is as relevant here as anywhere else.

(From Chapter 14, p. 424)

Copyright © 2003 Schuyler W. Huck
All rights reserved.