In counseling research, reliability is an extremely important concept. Reliability of scores refers to how consistent scores remain across time, instruments, and conditions. The most commonly reported methods of assessing reliability are Cronbach’s alpha and Kuder-Richardson's index of reliability, otherwise known as measures of internal consistency. The purpose of this article is to describe internal consistency reliability coefficients, delineate the history of these coefficients, depict contemporary issues in counseling research, and present recommendations for practice.
AbelsonA. (1911). The measurement of mental ability of “backward” children. British Journal of Psychology, 4, 268–314. doi: 10.1111/j.2044-8295.1911.tb00047.x
2.
AllenM. J.YenW. M. (1979). Introduction to measurement theory. Monterey, CA: Brooks/Cole.
3.
American Psychological Association. (2010). Publication manual of the American Psychological Association. (6th ed.).
Washington, DC: Author.
4.
AshbyJ. S.DickinsonW. L.GnilkaP. B.NobleC. L. (2011). Hope as a mediator and moderator of multidimensional perfectionism and depression in middle school students. Journal of Counseling & Development, 89, 113–139.
5.
BeckA. T.SteerR. A.BrownG. K. (1996). Beck Depression Inventory. (2nd ed.).
San Antonio, TX: The Psychological Corporation.
6.
BrennanR. L. (2001). An essay on the history and future of reliability from the perspective of replications. Journal of Educational Measurement, 38, 295–317. doi: 10.1111/j.1745-3984.2001.tb01129.x
7.
BriefA. P.BurkeM. J.GeorgeG. M.RobinsonB. S.WebsterJ. (1988). Should negative affectivity remain an unmeasured variable in the study of job stress?. Journal of Applied Psychology, 73, 193–198.
8.
BrownW. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3, 296–322.
9.
CapraroR. M.CapraroM. M. (2002). Myers-Briggs Type Indicator score reliability across studies: A meta-analytic reliability generalization study. Educational and Psychological Measurement, 62, 590–602.
10.
CrockerL.AlginaJ. (1986). Introduction to classical modern test theory. Orlando, FL: Holt, Rinehart & Winston.
11.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. doi: 10.1007/BF02310555
12.
DanielL. G. (1998). Statistical significance testing: A historical overview of misuse and misinterpretation with implications for editorial policies of educational journals. Research in the Schools, 5, 23–32.
13.
EriksenK.McAuliffeG. (2003). A measure of counselor competency. Counselor Education & Supervision, 43, 120–133.
14.
HelmsJ. E.CarterR. T. (1990). Development of the White Racial Identity Attitudes Scale. In HelmsJ. E. (Ed.), Black and White racial identity: Theory, research, and practice (pp. 67–80). Westport, CT: Greenwood.
15.
HelmsJ. E.HenzeK. T.SassT. L.MifsudV. A. (2006). Treating Cronbach’s alpha reliability coefficients as data in counseling research. The Counseling Psychologist, 34, 630–660.
16.
HensonR. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177–189.
17.
Hillenbrand-GunnT. L.HeppnerM. J.MauchP. A.ParkH. (2010). Men as allies: The efficacy of a high school rape prevention intervention. Journal of Counseling & Development, 88, 43–51.
18.
HoganT. P.BenjaminA.BrezinskiK. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Methods, 60, 523–531. doi: 10.1177/ 00131640021970691
19.
HopkinsK. D. (1998). Educational and psychological measurement and evaluation. (8th ed.).
Needham Heights, MA: Allyn & Bacon.
20.
HoytC. (1941). Test reliability obtained by analysis of variance. Psychometrika, 6, 153–160.
21.
KelleyT. L. (1921). The reliability of test scores. Journal of Educational Research, 3, 370–379.
22.
KuderG. F.RichardsonM. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151-160. doi: 10.1007/BF02288391
23.
MeierS. T.DavisS. R. (1990). Trends in reporting psychometric properties of scales used in counseling psychology research. Journal of Counseling Psychology, 37, 113–115. doi: 10.1037//0022-0167.37.1.113
24.
MorrowJ. R.JacksonA. W. (1993). How “significant” is your reliability?. Research Quarterly for Exercise and Sport, 64, 352–355.
25.
MyersI. B.McCaulleyM. H. (1989). Manual: A guide to the development and use of the Myers-Briggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press.
26.
OnwuegbuzieA. J.DanielL. G. (2002). A framework for reporting and interpreting internal consistency reliability estimates. Measurement and Evaluation in Counseling and Development, 35, 89–103.
27.
OnwuegbuzieA. J.DanielL. G. (2004). Reliability generalization: The importance of considering sample specificity, confidence intervals, and subgroup differences. Research in the Schools, 11, 60–71.
28.
OnwuegbuzieA. J.RobertsJ. K.DanielL. G. (2005). A proposed new ‘What If’ reliability analysis for assessing the statistical significance of bivariate relationships. Measurement and Evaluation in Counseling and Development, 37, 228–239.
29.
OnwuegbuzieA. J.WeemsG. H. (2004). Response categories on rating scales. Characteristics of item respondents who frequently utilize midpoint. Research in the Schools, 11, 51–60.
30.
PayneD. L.LonswayK. A.FitzgeraldL. F. (1999). Rape myth acceptance: Exploration of its structure and its measurement using the Illinois Rape Myth Acceptance Scale. Journal of Research in Personality, 33, 27-68.
31.
ReinhardtB. (1996). Factors affecting coefficient alpha: A mini Monte Carlo study. In ThompsonB. (Ed.), Advances in social science methodology (pp. 3–20). Vol. 4, Greenwich, CT: JAI Press.
32.
RobertsJ. K.OnwuegbuzieA. J. (2003). Alternative approaches for interpreting alpha with homogeneous subsamples. Research in the Schools, 10, 63–69.
33.
Schönrock-AdemaJ.Van der MolenH. T.van der ZeeK. I. (2009). Effectiveness of a self instruction program for microcounseling skills training. Teaching of Psychology, 36, 246–252.
34.
SchweisheimerW.WalbergH. J. (1976). A peer counseling experiment: High school students as small-group leaders. Journal of Counseling Psychology, 23, 398–401.
35.
SimonsL.GiorgioT.HoustonH.JacobucciR. (2007). An exploration of students' perceptions of empirically supported treatments: The significance of gender and ethnicity. Journal of Alcohol and Drug Education, 51, 63–85.
36.
SpanierG. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38, 15–28. doi: 10.2307/350547
37.
SpearmanC. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15, 72–101.
38.
SpearmanC. (1907). Demonstration of formulae for true measurement of correlation. American Journal of Psychology, 18, 161–169. doi: 10.2307/1412408
39.
SpearmanC. (1910). Correlation calculated for faulty data. British Journal of Psychology, 3, 171–295.
40.
ThompsonB.SnyderP. A. (1998). Statistical significance and reliability analyses in recent. Journal of Counseling & Development research articles. Journal of Counseling & Development, 76, 436–441.
41.
ThompsonB.Vacha-HaaseT. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174–195. doi: 10.1177/00131640021970448
42.
ThrunM.CookP. F.Bradley-SpringerL. A.GardnerL.MarksG.WrightJ., … GolinC. (2009). Improved prevention counseling by HIV care providers in a multisite, clinic-based intervention: Positive STEPs. AIDS Education and Prevention, 21, 55–66.
43.
Vacha-HaaseT. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6–20. doi: 10.1177/0013164498058001002
44.
Vacha-HaaseT.KoganL.TaniC. R.WoodallR. A. (2001). Reliability generalization: Exploring reliability coefficients of MMPI clinical scales scores. Educational and Psychological Measurement, 61, 45–59.
45.
Vacha-HaaseT.KoganL. R.ThompsonB. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509–522.
46.
VogtW. P. (1999). Dictionary of statistics and methodology: A nontechnical guide for the social sciences. (2nd ed.).
Thousand Oaks, CA: Sage.
47.
WallaceK. A.WheelerA. J. (2002). Reliability generalization of the life satisfaction index. Educational and Psychological Measurement, 62, 674–684.
48.
WeemsG. H.OnwuegbuzieA. J. (2001). The impact of midpoint responses and reverse coding on survey data. Measurement and Evaluation in Counseling and Development, 34, 166–176.
49.
WeemsG. H.OnwuegbuzieA. J.LustigD. C. (2003). Profiles of respondents who respond inconsistently to positively- and negatively-worded items on rating scales. Evaluation and Research in Education, 17, 45–60. doi: 10.1080/14664200308668290
50.
WeemsG. H.OnwuegbuzieA. J.SchreiberJ. B.EggersS. J. (2003). Characteristics of respondents who respond differently to positively- and negatively-worded items on rating scales. Assessment and Evaluation in Higher Education, 28, 587-607. doi: 10.1080/0260293032000130234
51.
WilkinsonL. & American Psychological Association (APA)Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist,
54, 594–604. doi: 10.1037//0003-066X.54.8.594 (Reprint available through the APA Home Page: http://www.apa.org/journals/amp/amp548594.html)
52.
WilsonV. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9, 5–10. doi: 10.2307/1175221
53.
WittaE. L.DanielL. G. (1998, April). The reliability and validity of test scores: Are editorial policy changes reflected in journal articles?Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
54.
YinP.FanX. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201–223.
