In this study, a proposed extension to the job component validity model from the Position Analysis Questionnaire was tested. Job component validity, a form of synthetic validation, allows researchers to select useful predictors and to estimate the criterion-related validity of tests based on conducting a job analysis which includes the Position Analysis Questionnaire. Morris and colleagues described a method for estimating the multiple correlation of a test battery assembled via job component validity estimates. In the current study, job component validity estimates, derived from the multiple correlation procedure proposed by Morris, et al., were compared to unit-weighted validity estimates obtained in a criterion-related validity study of six job progressions. The multivariate job component validity estimates were comparable to unit-weighted validity coefficients obtained using supervisory ratings as criteria. Multivariate job component validity estimates were conservative compared to corrected unit-weighted validity coefficients.
Get full access to this article
View all access options for this article.
References
1.
CascioW. F. (1995) Whither industrial and organizational psychology in a changing world of work?American Psychologist, 50, 928–939.
2.
CascioW. F.ValenziE. R., & SilbeyV. (1978) Validation and statistical power: implications for applied research. Journal of Applied Psychology, 63, 589–595.
3.
CrockerL., & AlginaJ. (2008) Introduction to classical and modern test theory. Mason, OH: Cengage.
4.
Equal Employment Opportunity Commission. (1978) Uniform guidelines on employee selection procedures. Washington, DC: U.S. Government Printing Office. Federal Register, 43, 38,290–38,315.
5.
GuionR. M. (1991) Personnel assessment, selection, and placement. In DunneteM. D. & HoughL. M. (Eds.), Handbook of industrial and organizational psychology. Vol. 2. (2nd ed.) Palo Alto, CA: Consulting Psychologists Press. Pp. 327–397.
6.
GutenbergR. L.ArveyR. D.OsburnH. G., & JeanneretP. R. (1983) Moderating effects of decision-making/information processing job dimensions on test validities. Journal of Applied Psychology, 36, 237–247.
7.
HoffmanC. C. (1995) Applying range restriction corrections using published norms: three case studies. Personnel Psychology, 48, 913–923.
8.
HoffmanC. C., & HoldenL. M. (1997) An extension of “A comparison of validation criteria.”Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
9.
HoffmanC. C.HoldenL. M., & GaleE. K. (2000) So many jobs, so little ‘N’: applying expanded validation models to support generalization of cognitive test validity. Personnel Psychology, 53, 955–991.
10.
HoffmanC. C., & McPhailS. M. (1998) Exploring options for supporting test use in situations precluding local validation. Personnel Psychology, 51, 987–1003.
11.
HoffmanC. C.NathanB. R., & HoldenL. M. (1991) A comparison of validation criteria: objective versus subjective and self versus supervisor ratings. Personnel Psychology, 44, 601–619.
12.
HoffmanC. C.RashkovskyB., & D'EgidioE. (2007) Job component validity: background, current research, and applications. In McPhailS. M. (Ed.), Alternative validation strategies: developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass. Pp. 82–121.
13.
HunterJ. E. (1983) Validity generalization for 12,000 jobs: an application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, DC: U.S. Department of Labor, Employment Service.
14.
JeanneretP. R. (1992) Applications of job component/synthetic validity to construct validity. Human Performance, 5, 81–96.
15.
JohnsonJ. W. (2007) Synthetic validity: a technique of use (finally). In McPhailS. M. (Ed.), Alternative validation strategies: developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass. Pp. 122–158.
16.
LandyF. J. (1986) Stamp collecting versus science: validation as hypothesis testing. American Psychologist, 41, 1183–1192.
17.
LawleyD. N. (1943) A note on Karl Pearson's selection formulae. Proceedings of the Royal Society of Edinburgh, 62, 28–30.
18.
LawsheC. H. (1952) Employee selection. Personnel Psychology, 6, 31–34.
19.
McCormickE. J.DeNisiA. S., & ShawJ. B. (1979) Use of the Position Analysis Questionnaire for establishing the job component validity of tests. Journal of Applied Psychology, 64, 51–56.
20.
McCormickE. J.JeanneretP. R., & MechamR. C. (1972) A study of job characteristics and job dimensions based on the Position Analysis Questionnaire (PAQ). Journal of Applied Psychology, 56, 347–368.
21.
McPhailS. M. (2007) Development of validation evidence. In McPhailS. M. (Ed.), Alternative validation strategies: developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass. Pp. 1–25.
22.
MorrisD.HoffmanC. C., & ShultzK. (2002) Exploring job component validity: comparing JCV estimates to meta-analytic validity estimates, and developing JCV battery validity estimates. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Canada.
23.
MurphyK. R., & SchiarellaA. H. (1997) Implications of the multidimensional nature of job performance for the validity of selection tests: multivariate frameworks for studying test validity. Personnel Psychology, 50, 823–854.
24.
PearlmanK.SchmidtF. L., & HunterJ. E. (1980) Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology, 65, 373–406.
25.
ReeM. R.CarrettaT. R.EarlesJ. A., & AlbertW. (1994) Sign changes when correcting for range restriction: a note on Pearson's and Lawley's selection formulas. Journal of Applied Psychology, 79, 298–301.
26.
SackettP. R., & OstgaardD. J. (1995) Job-specific applicant pools and national norms for cognitive ability tests: implications for range restriction corrections in validation research. Journal of Applied Psychology, 79, 680–684.
27.
ScherbaumC. A. (2005) Synthetic validity: past, present, and future. Personnel Psychology, 58, 481–515.
28.
SchmidtF. L., & HunterJ. E. (1977) Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529–540.
29.
SchmidtF. L.HunterJ. E., & PearlmanK. (1980) Task difference and validity of aptitude tests in selection: a red herring. Journal of Applied Psychology, 66, 166–185.
30.
Society for Industrial and Organizational Psychology. (2003) Principles for the validation and use of personnel selection procedures. (4th ed.) College Park, MD: Author.
31.
SteelP. D. G.HuffcuttA. I., & Kammeyer-MuellerJ. (2006) From the work, one knows the worker: a systematic review of the challenges, solutions, and steps to creating synthetic validity. International Journal of Selection and Assessment, 14, 16–36.
32.
SteelP. D. G., & Kammeyer-MuellerJ. (2009) Using a meta-analytic perspective to enhance job component validation. Personnel Psychology, 62, 533–552.
33.
TabachnickB. G., & FidellL. S. (1997) Using multivariate statistics. (3rd ed.) Northridge, CA: HarperCollins.
34.
ThomsonD. E., & ThomsonT. A. (1982) Court standards for job analysis in selection. Personnel Psychology, 35, 872–873.
35.
ThorndikeR. L. (1949) Personnel selection: test and measurement techniques. New York: Wiley.
36.
U.S. Department of Labor. (1977) Dictionary of occupational titles. (4th ed.) Washington, DC: Author.
