Warning: ANCOVA Used With Intact Groups

Besides ANCOVA's statistical inability to generate unbiased adjusted means when intact groups are compared, there is a second, logical reason why you should be on guard whenever you come across a research report in which ANCOVA was used in an effort to "equate" intact groups. Simply stated, the covariate variable(s) used by the researcher may not address one or more important differences between the intact groups. In Excerpt 18.24 [not shown here], for example, the researchers realized that their intact groups might have been different to begin with. Consequently, they covaried on GPA "to control for academic differences." However, it is debatable whether academic differences among people are reflected totally in their GPAs, and it is even more debatable whether academic differences alone influence recall. In this and many other studies, people's motivation plays no small part in how well they perform!

In summary, be extremely cautious when confronted with research claims based upon the use of ANCOVA with intact groups. If an important covariate variable was overlooked by those who conducted the study, pay no attention whatsoever to the conclusions based upon the data analysis. Even in the case where data on all important covariate variables were collected and used you still should be tentative in your inclination to "buy into" the claims made by the researchers.

(From Chapter 18, pp. 556-557)

Copyright © 2003 Schuyler W. Huck
All rights reserved.