True Experimental Design

Schuyler W. Huck


   R O1 X O2
   R O1 -- O2

There once was a research design
That look, on first glance, oh so fine
      Yet it is stupidity
      To view its validity
As chaste due to Rs on each line

For one thing, mere randomization
That's done to gain equalization
      Of Es versus Cs
      Gives no guarantees
That you'll have good gen'ralization

Results may, for instance, show all
That Es topped the Cs (with p small)
      But if that full troop
      Ain't your "target" group
Wide claims will lead to your downfall

Or X may work well for just some
While others to it are quite numb
      The treatment's true fate
      Depends on the trait
One needs if gains from X shall come

The one who rates subjects may know
Which Ss got treatment -- and so
      Effect Rosenthal
      May ruin it all
And truth from your study won't flow

If treatment is novel, each E
May first say "It's super for me!"
      But as time goes by
      E may think and sigh . . .
"No value in X do I see."

Or just the reverse can occur
With X thought at first to deter
      But once acclimated
      Es may be elated
And to X high marks they'll confer

Should hist'ry and X interact
'Tis true there is no artifact
      But if replicated
      You'll be decimated
By findings that prove inexact

The IV or DV may be
Reported not sufficiently
      If later one tries to
      Your study to redo
New findings may be what you'll see

If two or more studies are done
With Ss not used in just one
      A treatment . . . its "grade"
      May strengthen or fade
If only one study is done

A pretest or posttest may make
The E-group to X wide awake
      O leads to creation
      Of sensitization
And sans O the treatment won't take

The time from the X till O2
Is worthy of careful review
      For if it's revised
      When new plans devised
Fresh findings may seem quite askew

The subjects may be told or know
Observers are watching . . . and so
      They'll work with great skill
      But only until
The watchers stop watching and go

Or what if the subjects perceive
What YOU hope and want and believe
      Unconscious or not
      Like robots they'll trot
And data you want you'll receive

I hate now to sound so paternal
But we must move past threats "external"
      More pitfalls there are
      To rip, maim, and scar
And keep your work out of a journal

Should either X or O be thought
To deal with a construct that's "caught"
      When such is not true
      With "fit" that's askew
One's main claims won't be what they ought

Moreover, one should not assume
That stat work can't add to the gloom
      Assumptions may be
      False prima facie
And that makes the truth not illume

Or what if there's meager precision
Strong Xs you'll miss for poor "vision"
      The Error Type II
      Will hide truth from view
And you'll make an inapt decision

But what of those "internal" threats?
Are all tamed, like nicely trained pets?
      If that rings as true
      You've got learnin' to do
For Rs catch just some in their nets

For instance, take treatment diffusion
Which can make for lots of confusion
      Should E give to C
      All X that there be
You'll come to a faulty conclusion

Or what if the folks in the C-group
Think that they were put in a "B" group
      They're demoralized
      And you victimized
By "findings" as murky as pea soup

The converse may also take place
Cs saying, "We will win the race!"
      Each tries like a kid
      (John Henry once did!)
Yet this can one's findings debase

And what if there's subject attrition
Related to the X condition
      It may just appear
      That X was "in gear"
When that's just a sad apparition

Of course, there is further the chance
That someone may try to enhance
      The plight of those in
      The C-group wherein
Such help is a foul circumstance

And lastly consider the stew
You're in if on post Es construe
      On DV their standing
      As not so commanding
'Cause X brought respect for O2

By "lastly" I sure don't intend
To mean that one need not attend
      To problems that have now
      No label, yet somehow
They could wreck one's work in the end

The moral, I hope, is quite clear
The word "true" doth make it appear
      That problems of import
      Are all on the backcourt
When really they may lurk quite near!!!

Note: In the order presented, the possible problems (i.e., threats to validity) alluded to in this extended limerick are referred to via these terms or labels: experimentally accessible population vs. the target population, interaction between the treatment and subject characteristics, Rosenthal effect, novelty, disruption, interaction between the treatment and history, descripotion of the independent and/or dependent variable, multiple treatment interference, sensitization (from the pretest or posttest), interaction between the treatment and time of measurement), Hawthorne effect, demand characteristics, construct validity, nonrobustness, inadequate power, treatment diffusion, resentful demoralization, compensatory rivalry, mortality, compensatory equalization, and response shift.)

Copyright © 2012

Schuyler W. Huck
All rights reserved.

| Book Info | Author Info |

Site URL:

Top | Site Map
Site Design: John W. Taylor V