A Test of the Measurement Assumptions Underlying Behaviorally Anchored Rating Scales

Abstract

Foundational and outcome assumptions of behaviorally anchored rating scales (BARS) are reviewed. Two studies were then conducted to examine two untested foundational assumptions of BARS. The first study examined the assumption that the critical incident methodology produces behavioral specific incidents with which to anchor BARS. Qualitative and quantitative analyses were conducted. Contrary to this assumption about BARS, results indicated that there was considerable variation in the specificity of the behavioral anchors. The second study examined the assumption that the interpretation of critical incidents is invariant across nonstimulus bound developmental and the actual stimulus bound rating contexts. The assumption was not satisfied. Results indicated that the evaluation of critical incidents was significantly different across rating contexts. Implications are discussed.

Get full access to this article

View all access options for this article.

References

Bernardin, H. (1977). Behavioral expectation scales versus summated scales: A fairer comparison. Journal of Applied Psychology, 62, 422-427.

Bernardin, H. (1979). The predictability of discrepancy measures of role constructs. Personnel Psychology, 32, 139-153.

Bernardin, H. , LaShells, M. , Smith, P. , and Alvares, K. (1976). Behavioral expectation scales: Effects of developmental procedures and formats. Journal of Applied Psychology, 61, 75-79.

Bernardin, H. and Smith, P. (1981). A clarification of some issues regarding the development and use of behaviorally anchored rating scales (BARS). Journal of Applied Psychology, 66, 458-463.

Blood, M. (1974). Spin-offs from behavioral expectation scale procedures. Journal of Applied Psychology, 59, 513-515.

Borman, W. (1979). Format and training effects on rater accuracy and rater errors. Journal of Applied Psychology, 64, 410-421.

Fiske, S. and Taylor, S. (1984). Social cognition. Reading, MA: Addison-Wesley.

Flanagan, J. (1954). The critical incident technique. Psychological Bulletin, 51, 327-357.

Hom, P. , DeNisi, A. , Kinicki, A. , and Bannister, B. (1982). Effectiveness of performance feedback from behaviorally anchored rating scales. Journal of Applied Psychology, 67, 568-576.

10.

Jacobs, R. , Kafry, D. , and Zedeck, S. (1980). Expectations of behaviorally anchored rating scales. Personnel Psychology, 33, 595-640.

11.

Kavanagh, M. and Duffy, J. (1978). An extension and field test of the retranslation method for developing rating scales. Personnel Psychology, 31, 461-470.

12.

Kingstrom, P. and Bass, A. (1981). A critical analysis of studies comparing behaviorally anchored rating scales (BARS) and other rating formats. Personnel Psychology, 34, 263-289.

13.

Kinicki, A. , Bannister, B. , Hom, P. , and DeNisi, A. (1985). Behaviorally anchored rating scales vs. summated rating scales: Psychometric properties and susceptibility to rating bias. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 45, 535-549.

14.

Landy, F. and Farr, J. (1980). Performance ratings. Psychological Bulletin, 87, 72-107.

15.

Landy, F. and Guion, R. (1970). Development of scales for the measurement of work motivation. Organizational Behavior and Human Performance, 5, 93-103.

16.

Schwab, D. , Heneman, H. , and DeCotiis, T. (1975). Behaviorally anchored rating scales: A review of the literature. Personnel Psychology, 28, 549-562.

17.

Smith, P. and Kendall, L. (1963). Retranslation of expectation: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149-155.

18.

Taylor, M. , Fisher, C. , and Ilgen, D. (1984). Individuals' reaction to performance feedback in organizations: A control theory perspective. In K. Rowland and G. Ferris (Eds.), Research in personnel and human resources management (Vol. 2). Greenwich, CT: JAI Press.

19.

Taylor, S. and Fiske, S. (1978). Salience, attention, and attribution: Top of the head phenomena. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology (Vol. 11). New York: Academic Press.

20.

Upshaw, H. (1969). The personal reference scale: An approach to social judgment. In L. Berkowitz (Ed.), Advances in Experimental Social Psychology (Vol. 4). New York: Academic Press.

21.

Zedeck, S. , Imparato, M. , Krausz, M. , and Oleno, T. (1974). Development of behaviorally anchored scales as a function of organizational level. Journal of Applied Psychology, 59, 249-252.