Psychometric evaluation of the Relative Mastery Scale: An Occupational Adaptation instrument

Abstract

This study evaluated the psychometric properties of the Relative Mastery Scale (RMS). Valid and reliable client-centered instruments support practice in value-based health care and community-based settings. Participants were 368 community-dwelling adults aged 18 to 95 years. Researchers conducted validity and reliability examinations of the RMS using classical test theory and Rasch measurement model. A partial credit model allowed exploration of individual scale properties. Spearman’s correlation coefficients between items were statistically significant at the .01 level. Cronbach’s alpha coefficient was .94 showing strong internal consistency. In exploratory factor analysis, Factor 1 accounted for 71% of variance with an eigenvalue of 4.26. In Rasch analysis, the 5-point rating scale demonstrated adequate functioning, confirmed unidimensionality, and person/item separation. The RMS instrument demonstrates sound psychometric characteristics. A valid and reliable measure of internal occupational adaptation supports application to monitor progress of internal occupational adaptation across a variety of individuals.

Keywords

measurement occupational adaptation Rasch analysis self-awareness treatment effectiveness

Get full access to this article

View all access options for this article.

References

American Occupational Therapy Association. (2011). Occupational therapy research agenda. American Journal of Occupational Therapy, 65(6_Suppl.), S4–S7. https://doi.org/10.5014/ajot.2011.65S4

American Occupational Therapy Association. (2018). Occupational therapy education research agenda—Revised. American Journal of Occupational Therapy, 72(Suppl._2), 7212420070p1–7212420070p5. https://doi.org/10.5014/ajot.2014.685S06

American Occupational Therapy Association. (2020). Occupational therapy practice framework: Domain and process—Fourth edition. American Journal of Occupational Therapy, 74(Suppl._2), 7412410010p1–7412410010p87. https://doi.org/10.5014/ajot.2020.74S2001

Arnadóttir

Fisher

A. G.

(2008). Rasch analysis of the ADL scale of the A-ONE. The American Journal of Occupational Therapy, 62(1), 51–60. https://doi.org/10.5014/ajot.62.1.51

Bond

T. G.

Yan

Heene

(2020). Applying the Rasch Model: Fundamental measurement in the human sciences (4th ed.). Routledge. https://doi.org/10.4324/9780429030499

Bravini

Franchignoni

Giordano

Sartorio

Ferriero

Vercelli

Foti

(2015). Classical test theory and Rasch analysis validation of the Upper Limb Functional Index in subjects with upper limb musculoskeletal disorders. Archives of Physical Medicine and Rehabilitation, 96(1), 98–104. https://doi.org/10.1016/j.apmr.2014.08.022

Comrey

A. L.

Lee

H. B.

(2013). A first course in factor analysis. Psychology Press.

Conover

W. J.

Iman

R. L.

(1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician, 35(3), 124–129. https://doi.org/10.1080/00031305.1981.10479327

Costello

A. B.

Osborne

J. W.

(2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(1), Article 7. https://doi.org/10.7275/jyj1-4868

10.

Cronbach

L. J.

(1946). Response sets and test validity. Educational and Psychological Measurement, 6, 475–494. https://doi.org/10.1177/001316444600600405

11.

DeCarlo

L. T.

(1997). On the meaning and use of kurtosis. Psychological Methods, 2(3), 292–307. https://doi.org/10.1037/1082-989X.2.3.292

12.

DiStefano

Zhu

Mindrila

(2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1–11. https://doi.org/10.7275/da8t-4g52

13.

Dykema

Garbarski

Wall

I. F.

Edwards

D. F.

(2019). Measuring trust in medical researchers: Comparing agree-disagree and construct-specific survey questions. Journal of Official Statistics, 35(2), 353–386. https://dx-doi-org.web.bisu.edu.cn/10.2478/jos-2019-0017

14.

Field

(2017). Discovering statistics using IBM SPSS statistics: North American edition. SAGE.

15.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. https://doi.org/10.1037/1040-3590.7.3.286

16.

George

Mallery

(2018). IBM SPSS statistics 25 step by step: A simple guide and reference (15th ed.). Routledge. https://doi.org/10.4324/9781351033909

17.

George

L. A.

Schkade

J. K.

Ishee

J. H.

(2004). Content validity of the relative mastery measurement scale: A measure of occupational adaptation. OTJR: Occupation, Participation & Health, 24(3), 92–102. https://doi.org/10.1177/153944920402400303

18.

George-Paschal

Grajo

(2019). Adaptation as a measure of occupational participation. In Grajo

L. C.

Boiselle

(Eds.), Occupation and adaptation: Multidimensional perspectives. Slack.

19.

Höhne

J. K.

Krebs

(2018). Scale direction effects in agree/disagree and item-specific questions: A comparison of question formats. International Journal of Social Research Methodology, 21(1), 91–103. https://doi.org/10.1080/13645579.2017.1325566

20.

Höhne

J. K.

Revilla

Lenzner

(2018). Comparing the performance of agree/disagree and item-specific questions across PCs and smartphones. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 14(3), 109–118. https://doi.org/10.1027/1614-2241/a000151

21.

Huck

S. W.

(2012). Reading statistics and research (6th ed.). Pearson Education.

22.

IBM Corp. (2020). IBM Statistical Package of Social Sciences (SPSS) statistics for Windows (Version 27.0.) [Software]. IBM Corp. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-27

23.

Kielhofner

Fan

C. W.

Morley

Garnham

Heasman

Forsyth

Lee

S. W.

Taylor

(2010). A psychometric study of the Model of Human Occupation Screening Tool (MOHOST). Hong Kong Journal of Occupational Therapy, 20(2), 63–70. https://doi.org/10.1016/S1569-18611170005-5

24.

Ledesma

R. D.

Valero-Mora

(2007). Determining the number of factors to retain in EFA: An easy-to-use computer program for carrying out parallel analysis. Practical Assessment, Research & Evaluation, 12(2), 1–11. https://doi.org/10.7275/wjnc-nm63

25.

Lelkes

Weiss

(2015). Much ado about acquiescence: The relative validity and reliability of construct-specific and agree–disagree questions. Research and Politics, 2(3), 1–8. https://doi.org/10.1177/2053168015604173

26.

Linacre

J. M.

(2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.

27.

Linacre

J. M.

(2021a). Winsteps® (Version 4.8.0) [Computer Software]. Winsteps.com

28.

Linacre

J. M.

(2021b). Winsteps® Rasch measurement computer program user’s guide. Winsteps.com

29.

K. H.

(1971). A measure of agreement among subjective judgements. Educational and Psychological Measurement, 31, 75–84. https://doi.org/10.1177/001316447103100105

30.

Masters

G. N.

(2019). Partial credit model. In van der Linden

W. J.

(Eds.), Handbook of item response theory: Three volume set (pp. 109–126). Chapman and Hall.

31.

Mok

M. M. C.

(2004). Validation of scores from self-learning scales for primary students using true-score and Rasch measurement methods. Journal of Applied Measurement, 5, 258–286.

32.

Montanelli

R. G.

Humphreys

L. G.

(1976). Latent roots of random data correlation matrices with squared multiple correlations on the diagonal: A Monte Carlo study. Psychometrika, 41(3), 341–348. https://doi.org/10.1007/BF02293559

33.

Nejati

Fan

C. W.

Broome

W. J.

Lin

C.-Y.

Griffiths

M. D.

Pakpouir

A. H.

(2020). Validating Persian Intuitive Eating Scale-2 in breast cancer survivors with overweight. Evaluation & the Health Professions, 44(4), 385–394. https://doi.org/10.1177/0163278720965688

34.

Osborne

J. W.

Costello

A. B.

(2004). Sample size and subject to item ratio in principal components analysis. Practical Assessment, Research & Evaluation, 9, Article 11. https://doi.org/10.7275/ktzq-jq66

35.

Pan

A. W.

Fan

C. W.

Chung

Chen

T. J.

Kielhofner

M. Y.

Chen

Y. L.

(2011). Examining validity of the Model of Human Occupation Screening Tool (MOHOST): Using classical test theory and item response theory. British Journal of Occupational Therapy, 74(1), 34–40.

36.

Schkade

J. K.

Schultz

(1992). Occupational adaptation: Toward a holistic approach for contemporary practice, Part 1. American Journal of Occupational Therapy, 46(9), 829–837. https://doi.org/10.5014/ajot.46.9.829

37.

Schultz

Schkade

J. K.

(1992). Occupational adaptation: Toward a holistic approach for contemporary practice, Part 2. American Journal of Occupational Therapy, 46(10), 917–925. https://doi.org/10.5014/ajot.46.10.917

38.

Stewart

A. L.

Ware

J. E.

(Eds.). (1992). Measuring functioning and well-being: The medical outcomes study approach. Duke University Press. https://doi.org/10.7249/CB361

39.

Streiner

D. L.

(2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99–103. https://doi.org/10.1207/S15327752JPA8001_18

40.

Tabachnick

B. G.

Fidell

L. S.

(2019). Using multivariate statistics. Pearson Education.

41.

Tavakol

Dennick

(2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

42.

Yeomans

Golder

(1982). The Guttman-Kaiser criterion as a predictor of the number of common factors. Journal of the Royal Statistical Society, 31(3), 221–229. https://doi.org/10.2307/2987988

43.

Yong

A. G.

Pearce

(2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9(2), 79–94. https://doi.org/10.20982/tqmp.09.2.p079