Unnumbered graphic rating scales theoretically can lead to greater score variance and thus greater score reliability. The use of Web-based graphic sliders overcomes the practical problems of administering graphic scales on paper measures: the excessive costs of scoring and excessive scoring errors arising from the tediousness of scoring done by hand. The study investigated whether graphic scales realize theoretically expected improved score reliability and how coarseness in slider scoring affects reliability. A large sample size (n = 4,407) and diverse respondent groups were employed so that more confidence could be invested in results.
Get full access to this article
View all access options for this article.
References
1.
Bendig, A. W. (1953). The reliability of self-ratings as a function of the amount of verbal anchorings and of the number of categories on the scale. Journal of Applied Psychology, 37, 38-41.
2.
Bendig, A. W. (1954). Reliability and the number of rating scale categories. Journal of Applied Psychology, 38, 38-40.
3.
Beniger, J. R. (1998). Survey and market research confront their futures on the World Wide Web. Public Opinion Quarterly, 62, 442-452.
4.
Burke, M. J. (1993). Computerized psychological testing, impacts on measuring predictor constructs and future job behavior. In N. Schmitt & W. Borman (Eds.), Personnel selection in organizations (pp. 203-239). San Francisco: Jossey-Bass.
5.
Champney, H. , & Marshall, H. (1939). Optimal refinement of the rating scale. Journal of Applied Psychology, 23, 323-331.
6.
Cook, C. , Heath, F., Thompson, B., & Thompson, R. L. (2001). The search for new measures: The ARL “LibQUAL+” study—A preliminary report. Portal: Libraries and the Academy, 1, 103-112.
7.
Cook, C. , & Thompson, B. (2000). Higher-order factor analytic perspectives on users’ perceptions of library service quality. Library Information Science Research, 22, 393-404.
8.
Cook, C. , & Thompson, B. (2001). Psychometric properties of scores from the Web-based LibQUAL+ study of perceptions of library service quality. Library Trends, 49, 245-257.
9.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
10.
Cunningham, G. K. (1986). Educational and psychological measurement. New York: Macmillan.
11.
Dwight, S. A. , & Feigelson, M. E. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60, 340-360.
12.
Ferrell, C. M. , & Daniel, L. G. (1995). A frame of reference for understanding behaviors related to the academic misconduct of undergraduate teacher education students. Research in Higher Education, 36, 345-375.
13.
Freyd, M. (1923). The graphic rating scale. Journal of Educational Psychology, 14, 83-102.
14.
Guilford, J. (1954). Psychometric methods (2nd ed.). New York: McGraw-Hill.
15.
Guilford, J. P. , & Fruchter, B. (1978). Fundamental statistics in psychology and education (6th ed.). New York: McGraw-Hill.
16.
Kerlinger, F. N. (1986). Foundations of behavioral research (3rd ed.). New York: Holt, Rinehart & Winston.
17.
Komorita, S. S. , & Graham, W. K. (1965). Number of scale points and the reliability of scales. Educational and Psychological Measurement, 25, 987-995.
18.
Krieg, E.F., Jr. (1999). Biases induced by coarse measurement scales. Educational and Psychological Measurement, 59, 749-766.
19.
Linn, R. L. , & Gronlund, N. E. (1995). Measurement and assessment in teaching (7th ed.). Englewood Cliffs, NJ: Prentice Hall.
20.
Mattell, M. S. , & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and validity. Educational and Psychological Measurement, 31, 657-674.
21.
Parasuraman, A. , Berry, L. L. & Zeithaml, V. A. (1991). Refinement and reassessment of the SERVQUAL scale. Journal of Retailing, 67, 420-450.
22.
Pedhazur, E. J. , & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum.
23.
Reinhardt, B. (1996). Factors affecting coefficient alpha: A mini Monte Carlo study. In B. Thompson (Ed.), Advances in social science methodology (Vol. 4, pp. 3-20). Greenwich, CT: JAI.
24.
Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725.
25.
Symonds, P. M. (1924). On the loss of reliability in ratings due to coarseness of the scale. Journal of Experimental Psychology, 7, 456-461.
26.
Thompson, B. , Cook, C., & Heath, F. (2001). How many dimensions does it take to measure users’ perceptions of libraries? A “LibQUAL+” study. Portal: Libraries and the Academy, 1, 129-138.
27.
Thompson, B. , & Dennings, B. (1993, November). The unnumbered graphic scale as a data-collection method: An investigation comparing three measurement strategies in the context of Q-technique factor analysis. Paper presented at the annual meeting of the Mid-South Educational Research Association, New Orleans. (ERIC Document Reproduction Service No. ED 364 589)
28.
Thompson, B. , & Miller, A. H. (1978). Dissonance theory and education students’ attitudes toward teachers. Journal of Experimental Education, 47, 55-59.
29.
Thompson, B. , & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
30.
Wilkinson, L. , & APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. [reprint available through the APA Home Page: http://www.apa.org/journals/amp/amp548594.html]
31.
Wylie, P. B. (1976). Effects of coarse groupings and skewed marginal distributions on the Pearson product moment correlation coefficient. Educational and Psychological Measurement, 36, 1-7.
