Process-Oriented Variables and Measurement of Job Performance: An Examination of Raters' Weighting Strategy

Abstract

This study explored the impart of process-oriented variables on the measurement of job performance by examining raters' weighting strategy. Weighting strategy was defined as the manner in which raters weight or combine their evaluations of a subordinate's job performance to reach an over-all judgment on a dimension. Specifically, three issues were examined: (1) the amount of relative weight raters place on favorable, neutral, and unfavorable information, (2) the number of pieces of information raters utilize in reaching an over-all judgment on a performance dimension, and (3) the extent to which raters agree on which piece(s) of information should be most heavily weighted in arriving at such a decision. An analysis of the job-performance ratings of 132 police officers showed that the most favorable information receives the most weight, raters utilize multiple pieces of information when arriving at an over-all judgment, and raters disagree on the relative importance they attribute to each piece of information in the decision-making process. The discussion centered on ways of improving the psychometric properties of measures for performance appraisal.

References

Bernarden

H. J.

Smith

P. C.

A clarification of some issues regarding the development and use of behaviorally anchored rating scales (BARS). Journal of Applied Psychology, 1981, 66, 458–463.

Borman

W. C.

Exploring upper limits of reliability and validity in job performance ratings. Journal of Applied Psychology, 1978, 63, 135–144.

Feldman

J. M.

Beyond attribution theory: Cognitive processes in performance appraisal. Journal of Applied Psychology, 1981, 66, 127–148.

Ferguson

G. A.

Statistical analysis in psychology and education. New York: McGraw-Hill, 1971.

Goldberg

L. R.

Five models of clinical judgment: An empirical comparison between linear and non-linear representations of the human inference process. Organizational Behavior and Human Performance, 1971, 6, 458–479.

Hollman

T. D.

Employment interviewers' errors in processing positive and negative information. Journal of Applied Psychology, 1972, 56, 130–134.

Latham

G. P.

Fay

Saari

L. M.

The development of behavioral observation scales for appraising the performance of foremen. Personnel Psychology, 1979, 32, 299–311.

Latham

G. P.

Saari

L. M.

Fay

BOS, BES, and baloney: Raising Kane with Bernarden. Personnel Psychology, 1980, 33, 815–821.

Mayfield

E. C.

Carlson

R. E.

Interview decisions: First results from a long-term research project. Personnel Psychology, 1966, 19, 41–55.

10.

Miller

J. W.

Rowe

P. M.

Influence of favorable and unfavorable information upon assessment decisions. Journal of Applied Psychology, 1967, 51, 432–435.

11.

Schmidt

F. L.

Johnson

R. H.

Gugel

J. F.

Utility of policy capturing as an approach to graduate admissions decision making. Applied Psychological Measurement, 1978, 2, 347–359.

12.

Schneier

C. E.

Operational utility and psychometric characteristics of behavioral expectation scales: A cognitive reinterpretation. Journal of Applied Psychology, 1977, 62, 541–548.

13.

Smith

P. C.

Behaviors, results, and organizational effectiveness: The problem of criteria. In Dunnette

M. D.

(Ed.), Handbook of industrial and organizational psychology. Chicago, IL: Rand McNally, 1976. Pp. 745–776.

14.

Smith

P. C.

Kendall

L. M.

Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 1963, 47, 149–155.

15.

Wiggins

J. S.

Personality and prediction: Principles of personality assessment. Reading, MA: Addison-Wesley, 1973.