Construct Validity: A Unitary Concept for Occupational Therapy Assessment and Measurement

Abstract

Background

In the traditional view of validity, it is divided it into three sub-types: content, criterion-related, and construct. Validity has recently been reconceptualized as a unitary factor known as construct validity. Five sources of construct validity evidence have been specified: test content, response processes, internal structure, relationships to other variables, and consequential aspects of construct validity. They function either as general validity criteria or as standards for all assessment and measurement.

Purpose

The purpose of the manuscript is to present an overview of the contemporary conceptualization of construct validity and its relevance to occupational therapy practice, education, and research.

Key Issues & Implications

Taken together, the five components of construct validity evidence provide a way of addressing the multiple and interrelated validity questions that need to be answered in order to justify occupational therapy test score interpretation and use by practitioners. Given the context of evidence-based practice, professional standards of practice, increasing calls for accountability, and the fact that validity of a test, instrument, or scale is now seen as being a dynamic process, it is important for occupational therapists to be conversant with this contemporary view of construct validity, and the body of validity evidence related to the assessment tools used in clinical practice.

Keywords

Assessment Construct validity Measurement Testing Occupational therapy

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

American Psychological Association. (1954). Technical recommendations for psychological tests and diagnostic techniques. Washington, DC: American Psychological Association.

American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1966). Standards for education and psychological tests and manuals. Washington, DC: American Psychological Association.

Anastasi

(1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.

Anastasi

(1988). Psychological testing. New York: Macmillan Publishing.

Anastasi

, & Urbina

(1997). Psychological testing. Upper Saddle River, NJ: Prentice Hall.

Angoff

W. H.

(1988). Validity: An evolving concept. In Wainer

, & Braun

H. I.

(Eds.), Test validity (pp. 19–32). Hillsdale, NJ: Lawrence Erlbaum.

Asher

I. E.

(2007). Occupational therapy assessment: An annotated index. Bethesda, MD: American Occupational Therapy Association.

10.

Benson

, & Schnell

B. A.

(1997). Measurement theory: Application to occupational and physical therapy. In Van Deusen

, & Brunt

(Eds.), Assessment in occupational therapy and physical therapy (pp. 3–24). Philadelphia, PA: W. B. Saunders.

11.

Bundy

A. C.

, Lane

S. L.

, & Murray

E. A.

(1991). Sensory integration: Theory and practice. Philadelphia, PA: F. A. Davis.

12.

Campbell

D. T.

, & Fiske

D. W.

(1959). Convergent and discriminant validity in the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.

13.

Chudowsky

, & Behuniak

(1998). Using focus groups to examine the consequential aspects of validity. Educational Measurement: Issues and Practice, 17(4), 28–38.

14.

Craik

, Davis

, & Polatajko

(2007). Introducing the Canadian Practice Process Framework. In Townsend

E. A.

, & Polatajko

H. J.

(Eds.), Enabling occupation II: Advancing an occupational therapy vision for health, well-being, & justice through occupation (pp. 229–246). Ottawa, ON: Canadian Association of Occupational Therapists.

15.

Cronbach

L. J.

(1971). Test validation. In Thorndike

R. L.

(Ed.), Educational measurement (pp. 443–507). Washington, DC: American Council on Education.

16.

Cronbach

L. J.

(1988). Five perspectives on validity argument. In Wainer

, & Baum

H. I.

(Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Erlbaum.

17.

Cronbach

L. J.

(1989). Construct validation after thirty years. In Linn

R. L.

(Ed.), Intelligence: Measurement, theory, and public policy (pp. 147–171). Urbana, IL: University of Illinois Press.

18.

Cronbach

L. J.

, & Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

19.

Downing

S. M.

(2003). Validity: On the meaningful interpretation of assessment data. Medical Education, 37, 830–837.

20.

Downing

S. M.

(2004). Validity threats: Overcoming interference with proposed interpretations of assessment data. Medical Education, 38, 327.

21.

Fawcett

A. L.

(2007). Principles of assessment and outcome measurement for occupational therapists and physiotherapists: Theory, skills and application. Hoboken, NJ: Wiley.

22.

Geisinger

K. F.

(1992). The metamorphosis of test validation. Educational Psychologist, 27, 197–222.

23.

Goodwin

L. D.

(1997). Changing conceptions of measurement validity. Journal of Nursing Education, 36(3), 102–107.

24.

Goodwin

L. D.

(2002a). Changing conceptions of measurement validity: An update on the new Standards. Journal of Nursing Education, 41, 100–106.

25.

Goodwin

L. D.

(2002b). The meaning of validity. Journal of Pediatric Gastroenterology & Nutrition, 35(1), 6–7.

26.

Goodwin

L. D.

, & Leech

N. L.

(2003). The meaning of validity in the New Standards for Educational and Psychological Testing: Implications for measurement courses. Measurement and Evaluation in Counseling and Development, 36, 181–191.

27.

Gorin

J. S.

(2007). Reconsidering issues in validity theory. Educational Researcher, 36, 456–462.

28.

Guion

R. M.

(1980). On trinitarian doctrines of validity. Professional Psychology, 11, 385–398.

29.

Hinojosa

, & Kramer

(1998). Evaluation: Obtaining and interpreting data. Bethesda, MD: American Occupational Therapy Association.

30.

Jonson

J. L.

, & Plake

B. S.

(1998). A historical comparison of validity standards and validity practices. Educational and Psychological Measurement, 58(5), 736–753.

31.

Kane

M. T.

(1992). An argument-based approach to validation. Psychological Bulletin, 11, 527–535.

32.

Kane

M. T.

(1994). Validating interpretive arguments for licensure and certification examinations. Evaluation & the Health Professions, 17(2), 133–159.

33.

Kane

M. T.

(2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319–342.

34.

Kane

, Crooks

, & Cohen

(1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18(2), 5–17.

35.

Kielhofner

(2006). Developing and evaluating quantitative data collection instruments. In Kielhofner

(Ed.), Research in occupational therapy: Methods of inquiry for enhancing practice (pp. 155–176). Philadelphia, PA: F. A. Davis Company.

36.

Langenfeld

T. E.

, & Crocker

L. M.

(1994). The evolution of validity theory: Public school testing, the courts, and incompatible interpretations. Educational Assessment, 2, 149–165.

37.

Law

, & Baum

(2005). Measurement in occupational therapy. In Law

, Baum

, & Dunn

(Eds.) Measuring occupational performance: Supporting best practice in occupational therapy (pp. 3–20). Thorofare, NJ: Slack.

38.

Law

, Baum

, & Dunn

(2005). Measuring occupational performance: Supporting best practice in occupational therapy. Thorofare, NJ: Slack.

39.

Lissitz

R. W.

, & Samuelsen

(2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437–448.

40.

Markus

K. A.

(1998). Science, measurement, and validity: Is completion of Samuel Messick's synthesis possible? Social Indicators Research, 45, 7–34.

41.

Messick

(1988). The once and future uses of validity: Assessing the meaning and consequences of validity. In Wainer

, & Braun

H. I.

(Eds.), Test validity (pp. 33–45). Hillsdale, NJ: Lawrence Erlbaum.

42.

Messick

(1989). Validity. In Linn

(Ed.), Educational measurement (pp. 13–104). New York: American Council on Education & Macmillan Publishing.

43.

Messick

(1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13–23.

44.

Messick

(1995). Validity of psychological assessment: Validation of inferences from persons' and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.

45.

Mislevy

R. J.

(2007). Validity by design. Educational Researcher, 36, 463–469.

46.

Nunnally

, & Bernstein

(1994). Psychometric theory. New York: McGraw-Hill.

47.

Portney

, & Watkins

(2000). Foundations of clinical research. Sydney, NSW: Prentice-Hall.

48.

Shepard

L. A.

(1993). Evaluating test validity. Review of Research in Education, 19, 405–450.

49.

Sireci

S. G.

(2007). On validity theory and test validation. Educational Researcher, 36, 477–481.

50.

Smith

E. V.

(2001). Evidence for the reliability of measures and validity of measure interpretation: A Rasch measurement perspective. Journal of Applied Measurement, 2, 281–311.

51.

Streiner

D. L.

, & Norman

G. R.

(1995). Health measurement scales: A practical guide to their development and use. Oxford, UK: Oxford University Press.

52.

Wolfe

E. W.

, & Smith

E. V.

(2007a). Instrument development tools and activities for measure validation using Rasch models: Part I – instrument development tools. Journal of Applied Measurement, 8(1), 97–123.

53.

Wolfe

E. W.

, & Smith

E. V.

(2007b). Instrument development tools and activities for measure validation using Rasch models: Part II – validation activities. Journal of Applied Measurement, 8(2), 204–234.

54.

Yun

, & Ulrich

D. A.

(2002). Estimating measurement validity: A tutorial. Adapted Physical Activity Quarterly, 19, 32–47.