Quiz (Chapter 4)

The Meaning of Reliability and the Reliability Coefficient

1. What is really good one-word synonym for "reliable"?
2. Reliability coefficients can extend anywhere between ___ and ___ .

3. Whereas the number produced by test-retest reliability is called the "coefficient of ______," the number produced by parallel-forms reliability is called the "coefficient of ______ ."
4. What two other terms are sometimes used in place of the term "parallel-forms reliability"?
5. Name three reliability procedures that assess a test's "internal consistency."
6. (T/F) In estimating a test's split-half reliability, half of the examinees respond to the odd-numbered items while the other half respond to the even-numbered items.
7. When the K-R #20 reliability method is used, examinees will be tested ___ time(s).
8. Who invented the reliability procedure that's often called "alpha" or "coefficient alpha"?
9. Which internal consistency reliability procedure--K-R #20 OR Cronbach's alpha--fits a situation where data are collected on a 5-point scale that goes from "Strongly Agree" to "Strongly Disagree"?
10. Which of the reliability procedures that focus on internal consistency makes use of the Spearman-Brown formula?
11. (T/F) If applied to the same right/wrong (i.e., 0 or 1) data from a test, the split-half reliability coefficient will always turn out the same as the K-R#20 reliability coefficient.

12. The coefficient of concordance is symbolized by the letter ___ .
13. Interrater reliability will turn out equal to ____ if all raters are in full agreement.
14. What name is associated with the interrater reliability technique that produces a coefficient of concordance? This technique uses what kind of data?
15. What name is associated with the interrater reliability technique that produces a kappa coefficient? This technique uses what kind of data?
16. What do the letters ICC stand for?
17. (T/F) Although many techniques can be used to assess interrater reliability, Pearson's r is not one of them.

18. The size of the SEM is ____ (directly/indirectly) related to the amount of reliability present in the data.
19. If a student's score on a test is 82 and if the SEM = 4, that student's 68% confidence band would extend from __ to __. Now, change 68% to 95% and reanswer this question.
20. Which assessment of consistency--reliability OR SEM--is expressed "in" the same units as the scores around which confidence bands are built?

21. (T/F) High test-retest reliability implies high internal consistency reliability; conversely, low test-retest reliability implies low internal consistency reliability.
22. (T/F) Reliability "resides" in the measuring instrument itself, not in the scores obtained by using the measuring instrument.
23. Measures of internal consistency will be too ___ (high/low) if a test is administered under highly speeded conditions.
24. Which of these statements is better: (a) We estimated the reliability of our data. (b) We determined the reliability of our data.

25. What is a really good one-word synonym for "valid"?

26. (T/F)  If reliability is very, very high . . . then validity must also be very, very high.
27. (T/F)  If validity is very, very high . . . then reliability must also be very, very high.

28. Content validity normally ____ (is/is not) expressed by means of a numerical coefficient.
29. The term "criterion-related validity" covers two approaches: predictive and _____.
30. A validity coefficient normally takes the form of a ______ .
    1. mean
    2. SD
    3. correlation
31. Which of these is a construct:
1. Height
2. Hair color
3. Happiness
4. Date of birth
32. (T/F)  To support the convergent and discriminant validity of a new test, correlation coefficients must turn out to be positive and negative in sign, respectively.
33. Which of the main validity procedures (content, criterion-related, or construct) is sometimes dealt with by the statistical technique of factor analysis?

34. (T/F)  Reliability is a necessary but not sufficient condition for validity.
35. Where does the validity of a new test reside, in the test itself or in the scores produced by an administration of the test?
36. What might cause an honest researcher to claim that his/her test has high content validity when in fact it has very little content validity?
37. What might cause an honest researcher to claim that his/her test has high criterion-related validity when in fact it has very little criterion-related validity?

38. How high should reliability and validity coefficients be before we can confidently call them "high enough?"
1. .5
2. .75
3. .90
4. .95
5. It depends
39. (T/F) If a researcher conducts a study wherein the data are perfectly reliable and valid, it's still possible for the researcher's data-based conclusions to be utterly worthless, even if it's the case that an important research question was under investigation.

Copyright © 2012

Schuyler W. Huck
All rights reserved.