Type III Errors
I'm going to try to explain what a Type III error is (and also review what it means to have a Type I or a Type II error). If you already understand what these three errors are, stop reading, delete this message, and go find something better to do. If, however, you'd like to get a better handle on these three terms, please read on.
Suppose we want to compare two over-the-counter headache medicines: Advil and Tylenol. To do so, let's imagine that we locate 100 people who say they have daily headaches. Let's also imagine that we randomly assign these folks to our two treatment conditions (which we'll call "A" for Advil and "T" for Tylenol). Fifty folks go into each treatment condition. Our instructions to each of our 100 subjects tell him/her to take only the medicine that we supply if and when he/she experiences a headache. To make our study a bit more scientific, imagine that we do it with Advil and Tylenol pills that are made to look alike. That way, our subjects will be "blind" to the type of medicine we provide.
The "independent variable" in our little study is "type of headache remedy." In other words, type of headache remedy is the difference between our two comparison groups. Prior to collecting any data, this is the way our two groups differ. And since WE get to determine whether a person gets Advil or Tylenol, we could say that we are "manipulating" the independent variable, thus creating an experiment.
The "dependent variable" will be "subjective rating of headache relief." This will be the data of our study. To keep things simple, let's imagine that we simply ask each subject to rate, from 0-to-5, his/her opinion of how well his/her medicine worked to relieve any headaches. Let's also imagine that data on the dependent variable are collected 30 days after the study begins. Suppose we set up a null hypothesis that says (in words) that Advil and Tylenol are equally effective in relieving headaches. This would translate into an "Ho" statement that mA = mT. In other words, the null hypothesis would say that the mean rating of Advil, in the Advil population that we're thinking of, is identical to the mean rating of Tylenol, in the Tylenol population that fits our study.
Now, let's review what a Type I error or a Type II error would be. If the null hypothesis is true but we, based on our sample data, reject it, then that's a Type I error. In other words, a Type I error would occur if our sample data prompt us to claim that Advil is better than Tylenol (or vice versa) when the two medicines are equally good. In contrast, a Type II error would occur if the null hypothesis is false but we, based on our sample data, do NOT reject it. In other words, a Type II error would occur if our two sample means turn out to be so similar that we can't reject the null hypothesis . . . when in fact the null hypothesis is false (either because Advil is superior to Tylenol OR because Tylenol is superior to Advil). In summary, a Type I error takes place when a true null hypothesis is rejected whereas a Type II error takes place when a false null hypothesis is not rejected.
Now, how might a Type III occur in our study? Suppose a medical friend of ours somehow knows that Advil is better than Tylenol at relieving headaches. However, that friend is away on a long trip, we didn't have access to his/her expert opinion, and we therefore conduct our little study while this expert is out of town. But let's imagine that this friend of ours KNOWS FOR CERTAIN that Advil is superior to Tylenol . . . meaning that mA is larger than mT.
Even though higher ratings, on the average, are associated with the "A" population than with the "T" population, it's possible (due to sampling error) that the "T" sample mean might turn out to be larger than the "A" sample mean. Moreover, it's possible that such a difference between the two sample means could be so large that our study's null hypothesis gets rejected. Let's review this situation and then think about what has happened:
Now, did we make a Type I error? The answer to this question is "NO" because the null hypothesis is false. (See #1 above.) By definition, a Type I error takes place when a true null hypothesis is rejected. But Ho is false . . . so we for sure did NOT make a Type I error. Did we make a Type II error? The answer here is "NO" because we rejected the null hypothesis. (See #3 above.) By definition, a Type II error takes place when a false null hypothesis is not rejected. But we rejected Ho . . . so we for sure did NOT make a Type II error.
Since we didn't make a Type I error and since we didn't make a Type II error, did we do the right thing? In other words, did we make a correct inference about the two populations involved in our study? The answer here is "NO" because we claimed, on the basis of our sample data, that mT is larger than mA when in fact it's precisely the other way around. It's NOT the case that Tylenol works better than Advil (as indicated by our research results); in reality, Advil works better than Tylenol.
In our little study, we correctly rejected a false null hypothesis. However, the "direction" of our inference is "backwards" from the real truth of the situation. (Advil is truly better than Tylenol but we claimed that Tylenol is better than Advil.) The term Type III error is used to designate this kind of inferential error.
Copyright © 2012
Schuyler W. Huck