Development and initial validation of ability levels to interpret pediatric spinal cord injury activity measure and pediatric measure of participation scores

Abstract

PURPOSE:

To develop ability levels and descriptions to interpret Pediatric Spinal Cord Injury Activity Measure (PEDI-SCI AM) and Pediatric Measure of Participation (PMoP) scores.

METHODS:

Mixed-methods approach to 1.) create item maps using PEDI-SCI AM and PMoP calibration data; 2.) employ bookmarking methods and an iterative consensus process including semi-structured interviews, focus groups and surveys to establish PEDI-SCI AM and PMoP level score cut-points and descriptions; 3.) use calibration sample data to examine mean score differences across levels [analyses of variance (ANOVAs)] and assess the sample distribution (%) across levels; 4.) repeat in a separate validation sample; 5.) develop level descriptions. Throughout the mixed methods approach, the sample participants include children with spinal cord injury (SCI), parents of children with SCI, and professionals who work with children with SCI.

RESULTS:

Four or five ability levels were identified for each PEDI-SCI AM and PMoP domain along with descriptions for each level. ANOVA results revealed significant overall differences for level mean scores and pairwise comparisons (p < 0.05). Consensus (>80%) was achieved for all PEDI-SCI AM and some PMoP level descriptions.

CONCLUSION:

PEDI-SCI AM and PMoP score cut-points identify different levels of activity and participation among children with SCI. These levels and descriptions provide clinical relevance for PEDI-SCI and PMoP numeric scores.

Keywords

Pediatric spinal cord injury patient reported outcome measure rehabilitation computer adaptive testing bookmarking

Abbreviations

SCI

Spinal cord injury

PEDI-SCI AM

Pediatric Spinal Cord Injury Activity Measure

PMoP

Pediatric Measure of Participation

IRT

Item Response Theory

CAT

Computerized Adaptive Test

Short Form

MWC

Manual Wheelchair

PWC

Power Wheelchair

Standard Deviation

PROM

Patient Reported Outcome Measure

PROMIS

Patient Reported Outcome Measurement Information System

WD-FAB

Work Disability Functional Assessment Battery

SCI-FI

Spinal Cord Injury Functional Index

1 Introduction

Reliable and sensitive outcome measures are critical to document outcomes for children with spinal cord injury (SCI) and support effective evidence-based rehabilitation practice. The Pediatric Spinal Cord Injury Activity Measure (PEDI-SCI AM) and the Pediatric Measure of Participation (PMoP) are child and parent-reported outcome measures developed specifically for the pediatric SCI population to meet this need.

The PEDI-SCI AM and PMoP were developed using item response theory (IRT) to create comprehensive calibrated item banks comprised of items hierarchically organized along a continuum of “ability” in unidimensional domains. These calibrated item banks can be administered as Computer Adaptive Tests (CATs) or short forms (SFs). Both assessments were developed to assess children with traumatic and non-traumatic SCI and dysfunction. Respondents are children age 8–21 and parents of children age 4–21 and can be used in a variety of clinical settings including but not limited to the hospital, rehabilitation centers, outpatient clinics and home. The PEDI-SCI AM includes assessments designed to measure activity performance of children with spinal cord injury and dysfunction. As background, activity performance as it relates to the PEDI-SCI AM is defined as a child’s execution of complex functions which represent specific tasks that can be completed alone or with others. This assessment focuses on the individual perspective of function. Each assessment is organized into four mobility domains [General Mobility, Manual Wheelchair (MWC) Mobility, Power Wheelchair (PWC) Mobility, and Ambulation] and a Daily Routines domain, comprised of items that assess self-care, school, play/leisure, and chores/work. The PMoP is intended to evaluate participation, defined as a child’s involvement in life situations across physical, social, spiritual and virtual environments including home, school and community. The PMoP assesses two domains: Self Participation (participation compared to what the child wants to do) and Friend Participation (participation compared to what the child’s friends do). Each domain includes separate assessments for child and parent respondents. The child and parent PEDI-SCI AM and PMoP assessments are separate instruments, owing to the importance of obtaining both child and parent reported outcomes. Each domain of the PEDI-SCI and PMoP is also a separate instrument and can be administered in combination or isolation. Because the PEDI-SCI AM and PMoP are separate instruments, scores cannot be compared, nor can scores be compared between the domains or respondents. Development and validation of the PEDI-SCI AM and PMoP are described in detail in previous publications [1 –6]. In summary, measurement development began with a review of existing literature, established assessments and medical charts to identify aspects of activity and participation that are important to children with SCI. An initial conceptual framework was developed and used to help identify and write items. Draft items were cognitively tested and refined to yield an initial item pool. Calibration studies administered activity and participation item pools to samples of children with SCI and their parents. Factor analysis identified unidimensional domains of activity and participation, and IRT analyses (graded response model) were conducted to yield calibrated item banks for each unidimensional domain. Follow up studies established the validity and reliability of the PEDI-SCI AM and PMoP assessments. Currently a platform is in development to maintain a web-based PEDI-SCI AM and PMOP CATs. The SFs as well as corresponding manuals are freely available and can be obtained from the website for the Jefferson College of Rehabilitation Sciences Center for Outcomes and Measurement (https://www.jefferson.edu/academics/colleges-schools-institutes/rehabilitation-sciences/departments/outcomes-measurement/measures-assessments.html).

Scores from IRT-based measures, including the PEDI-SCI AM and PMoP estimate an individual’s ability along a continuum from the lowest to the highest level of the specific trait assessed. Scores are calculated based on logit values that are transformed to a standard metric (T-score) with an average of 50 (based on calibration sample scores) and each ten-point interval represents one standard deviation (SD) from the mean. Scores from IRT-based measures, such as the PEDI-SCI AM and PMoP, provide interval level data, which is a significant advantage for quantitative analyses of outcomes data. Unfortunately, a numeric score based on a mean of 50 does not provide a meaningful clinical context for score interpretation [7]. There is growing recognition that difficulty interpreting and communicating patient reported outcome measure (PROM) scores presents a significant barrier to effective use of these tools in clinical settings [8]. PROMs have the potential to play an important role in clinical practice, including helping with needs assessments, shared decision-making and symptom management [9]; however, it is critical to address the issue of score interpretation to achieve these goals. Score interpretation is one of eight scientific criteria established by the Medical Outcomes Trust Scientific Advisory Board for high-quality, standardized health outcomes measurement instruments [10]. Providing a clinical context for interpreting PEDI-SCI AM and PMoP numeric scores will allow clinicians to communicate assessment findings to children with SCI and their parents/caregivers to initiate discussions about the child’s level of ability and potential treatment goals.

One strategy for interpreting IRT-based assessments employs bookmarking methods. Bookmarking methods are commonly used in educational testing to interpret scores from IRT-based standardized assessments by establishing score ranges associated with different levels of academic performance and describing the ability level (e.g. basic, proficient, advanced). The bookmarking process involves developing item maps, which display the distribution of IRT-calibrated items and responses for each item. Content experts review item maps to identify scores associated with different levels of ability and describe characteristics that define the different ability levels [11, 12]. Bookmarking methods have been adapted to establish ability levels for several PROMs, including Patient Reported Outcome Measure Information Systems (PROMIS) assessments of physical function, cognitive function and sleep disturbance, the Work Disability Functional Assessment Battery (WD-FAB), and the Spinal Cord Injury Functional Index (SCI-FI) [8 , 13–15].

This study aims to identify PEDI-SCI AM and PMoP cut-points associated with score ranges that distinguish different ability levels for children with spinal cord injury (SCI) and describe each level. PEDI-SCI AM and PMoP ability levels and descriptions will provide a context for interpreting numeric scores and may improve communication between clinicians and parents/children regarding meaningful and relevant information about a child’s abilities in a specific domain. PEDI-SCI AM and PMoP ability levels and descriptions will better enable clinicians, children with SCI and parents/caregivers of children with SCI to use these assessments to monitor change over time, guide goal setting and engage in shared decision making about rehabilitation and habilitation. Integrating ability levels and descriptions into reports of PEDI-SCI AM and PMoP scores will add value and enhance clinical utility of these assessments.

2 Methods

This study extends PEDI-SCI AM and PMoP development using a mixed-methods and an iterative approach to identify score ranges that distinguish different levels of ability for PEDI-SCI AM and PMoP domains. Figure 1 provides an overview of study methods. For each PEDI-SCI AM and PMoP domain, experts used bookmarking methods to identify scores, which are referred to as cut-points, associated with different levels of activity and participation ability. Focus groups established initial cut-points qualitatively, using the bookmarking process and these initial cut-points were examined in two samples using quantitative methods.

Fig. 1

Study Process.

2.1 Item maps and bookmarking procedure

An item map is a tool used in the bookmarking process to identify cut-points that distinguish different ability levels. PEDI-SCI AM and PMoP calibration studies [4 –6] included IRT analyses performed using PARSCALE (Scientific Software International, Chicago, IL) and ResidPlots-2 (University of Massachusetts, Center for Educational Assessment, Amherst, MA) analyses software to develop calibrated item banks. Item maps used during the bookmarking process were generated from the PEDI-SCI AM and PMoP calibrated item banks. Item maps provide a visual display of IRT-calibrated item banks. Figure 2 presents a sample item map demonstrating the following components: 1.) horizontal bars display the pattern of participant response choices (e.g., easy, a little hard, hard, really hard, can’t do); 2.) items are hierarchically ordered along the y-axis; 3.) PEDI-SCI AM and PMoP T-scores are presented on the x-axis; and 4.) cut-points distinguish different ability levels [16, 17]. Item maps were generated for each PEDI-SCI AM and PMoP domain and respondent (i.e., child, parent) for a total of 16 item maps.

Fig. 2

Item Map for PEDI-SCI AM General Mobility Child Report Items.

We convened a 4-hour expert panel consensus group meeting comprised of multi-disciplinary professionals with expertise in pediatric SCI rehabilitation. The consensus group meeting was conducted at Shriners Hospitals for Children (SHC) as an exempt project under 45 CFR 43.101(b)(2), and as such it was not formally supervised by an IRB. Expert panelists included 9 pediatric rehabilitation professionals: 1 physician, 3 occupational therapists, 4 physical therapists and 1 psychologist with 4 to 30 years of professional experience and different levels of education: BS degree (N = 1), MS degree (N = 3), clinical doctorate (N = 2), medical doctorate (N = 1) and PhD (N = 2)

The consensus group meeting began with a presentation to review the bookmarking process and provide examples from a study that used this process to demonstrate use of item maps to identify score cut-points associated with assessment items that define different ability levels [18]. Next, panelists were organized into three groups which were facilitated by one of three investigators (CCT, MJM, MDS). For each assigned item map, groups completed the following activities: 1.) determined the optimal number of ability levels; 2.) identified score cut-points that distinguish different ability levels; 3.) developed a description of each level that would be clear and understood by the respondent particularly by children as young as 8 for the child report. As a final step, each group reported initial findings to the entire group for review and discussion. The group reached consensus to establish level cut-points for all PEDI-SCI AM and PMoP domains. Consensus was achieved for PEDI-SCI AM level descriptions for each domain. However, consensus was not achieved for PMoP level descriptions and the expert panel recommended that investigators continue efforts to refine participation level descriptions.

2.2 Additional PMoP level description consensus processes

Efforts to achieve consensus for PMoP level descriptions included semi-structured interviews, a focus group and surveys. Figure 1 summarizes the process and Table 1 provides demographic information for participants involved in this multi-pronged consensus process. The semi-structured interviews conducted with children with SCI were approved by the Institutional Review Board at all participating sites. The focus groups, semi-structured interviews and surveys conducted with adults were undertaken at SHC as an exempt project under 45 CFR 43.101(b)(2) and as such it was not formally supervised by an IRB or exempt IRB approval was obtained from other participating sites. Semi-structured Interviews: Two investigators (CCT, MJM) conducted semi-structured interviews using child and parent-specific guides (Table 2) to explore characteristics that distinguish different levels and to determine preferred language to describe PMoP levels. Interviews were audio-recorded, transcribed verbatim and reviewed for content and themes. Focus Group: Investigators (CCT, MJM, MDS) conducted a focus group comprised of multi-disciplinary professionals working with children with SCI in order to confirm the characteristics that distinguish different levels of the PMoP and develop language to describe the levels. The focus group proceedings were audio-recorded, transcribed verbatim and reviewed for content and themes. An example of the semi-structured interview and focus group questions is included in Table 2. PMoP Survey: As a final step, two rounds of the survey were conducted using Qualtrics survey platform (Qualtrics, Provo, UT) to solicit feedback from experienced clinicians working with children with SCI. Clinicians could include, but were not limited to physicians, nurses, psychologists, occupational, physical, speech, and recreational therapists, child life specialists and social workers. Surveys presented each PMoP level description and asked the following questions: 1.) Does the level relate to children with SCI?; 2.) Does the level accurately describe participation of children with SCI?; 3.) Would the level be understood by children as young as 8 years? Since the child report assessments were completed by children as young as 8 years of age, it was necessary that the level descriptions could be understood by this population [2–3]. Additionally, the survey presented level descriptions in two formats: narrative and bulleted. Respondents were asked to select their preferred format.

Table 1
Semi-Structured Interview, Focus Group and Survey Participant Information

Method Participants Sample Description

Interviews Children N = 5 Age range: 10–17

Parents N = 6 Child with SCI age range: 5–17

Focus Group N = 10 Occupational therapists (N = 3), Physical therapists (N = 4), Nurses or nurse practitioners (N = 2), Social workers (N = 1)

Survey Round 1 N = 60 Physicians (N = 13), Occupational therapists (N = 9), Physical therapists (N = 24), Nurses or nurse practitioners (5), Social workers (N = 2), Psychologists (N = 4), Researchers (N = 3), Recreation therapists (N = 1)

Survey Round 2 N = 37 Physicians (N = 2), Occupational therapists (N = 9), Physical therapists (N = 14), Nurses or nurse practitioners (N = 2), Social workers (N = 2), Psychologists (N = 4), Researchers (N = 2), Speech therapists (N = 1)

Method	Participants	Sample Description
Interviews	Children N = 5	Age range: 10–17
	Parents N = 6	Child with SCI age range: 5–17
Focus Group	N = 10	Occupational therapists (N = 3), Physical therapists (N = 4), Nurses or nurse practitioners (N = 2), Social workers (N = 1)
Survey Round 1	N = 60	Physicians (N = 13), Occupational therapists (N = 9), Physical therapists (N = 24), Nurses or nurse practitioners (5), Social workers (N = 2), Psychologists (N = 4), Researchers (N = 3), Recreation therapists (N = 1)
Survey Round 2	N = 37	Physicians (N = 2), Occupational therapists (N = 9), Physical therapists (N = 14), Nurses or nurse practitioners (N = 2), Social workers (N = 2), Psychologists (N = 4), Researchers (N = 2), Speech therapists (N = 1)

Table 2

Semi-Structured Interview and Focus Group Guide Questions Related to PMoP Level Descriptions

The descriptions of the participation levels will help children and their parents/caregivers understand how much they are participating.
On your copy, you can see the participation levels and the corresponding description. Please take a moment to review each description.
Question	Helpful Probes
Do these levels relate to children with SCI?	Where do children with SCI participate?
Are the levels written well?	Would a child as young as 8 years old be able to read it?
	Would a child as young as 8 years old be able to understand it?
What is different about each level?	What is the difference between 1 and 2?
	What is the difference between 2 and 3?
	What is the difference between 3 and 4?
What is similar about each level?	What is the similar about 1 and 2?
	What is the similar about 2 and 3?
	What is the similar about 3 and 4?
Do you prefer the description in a narrative format or a bulleted format	What do you like about the narrative format?
	What do you dislike about the narrative format?
	What do you like about the bulleted format?
	What do you dislike about the bulleted format?
Do you have suggestions or a more preferred wording?	What wording would you use to describe someone in level 1 (2, 3, 4)?
Is there anything we may have missed concerning describing the participation of a child with SCI that you want to share?

2.3 PEDI-SCI AM and PMoP study samples and level validation

Ability levels were established and validated using data from earlier studies [4 –6]: 1.) PEDI-SCI AM and PMoP calibration study (Calibration Sample); 2.) PEDI-SCI AM and PMoP study to validate linking estimates (Validation Sample). From these earlier studies, in the Calibration Sample (N = 374 children with SCI; 320 parents of children with SCI), all PEDI-SCI AM and PMoP items were administered. For the Validation Sample (N = 106 children with SCI; 96 parents of children with SCI), study participants completed the PEDI-SCI AM and PMoP via a CAT (minimum 8 item, maximum 15 item).

Calibration study data were used to generate item maps and to conduct quantitative analyses of level cut-point recommendations that emerged from expert consensus. Analyses of Variance (ANOVAs) examined mean level scores for each PEDI-SCI AM and PMoP domain. If the F-ratio indicated an overall significant difference (p < 0.05) between levels, Tukey’s (HSD) Tests examined pairwise comparisons (p < 0.05). The distribution of the sample (% at each level) across levels for each PEDI-SCI AM and PMoP domain was also examined. All analyses were repeated using Validation Sample data.

3 Results

3.1 Cut points and ability levels

Four ability levels were established for all PMoP domains and for PEDI-SCI AM domains except Daily Routines (Parent & Child), Ambulation (Child) and MWC (Parent & Child) where five ability levels were established. Table 3 summarizes PEDI-SCI AM and Table 4 summarizes PMoP score level cut-points, level mean scores and Standard Deviation (SD), ANOVA results, and distribution of Calibration and Validation Samples (%) across the levels. ANOVA analyses demonstrated significant differences for level mean scores (P < 0.0001–0.0003) for all PEDI-SCI AM and PMoP domains in both samples. Mean scores increased progressively across all domains, and ranges across domains were as follows: Level 1 = 0–30.73, Level 2 = 31.25–40.85, Level 3 = 37.66–51.91, Level 4 = 46.7–71.62, and Level 5 = 59.12–66.55. A review of mean scores across levels revealed that the mean score based on IRT calibration (50) is located in Levels 3 and 4. Level 1 mean scores are 2 SD or more below the mean; Level 2 mean scores are 1–2 SD below the mean. For domains with 4 levels, Level 4 mean scores are 0.5–1 SD above the mean (except for Ambulation). For domains with 5 levels, Level 5 mean scores are 1–1.5 SD above the mean. Calibration and Validation Sample mean scores revealed that scores were similar with less than a 5-point difference for all domains and levels except: Daily Routines (Child & Parent) Level 1 & 2; PWC (Child & Parent) Level 1; Participation Self (Child & Parent) Level 1.

Table 3
PEDI-SCI AM score range for each level, means score, Standard deviation (SD) and ANOVA results

Domain Respondent Sample Level 1 Level 2 Level 3 Level 4 Level 5

General Mobility Child Cut Score Range ≤30^* 31–40^* 41–50^* ≥51^*

Calibration N (%) 21 (5.6) 44 (11.8) 122 (32.6) 187 (50.0)

N = 374 Mean (SD) 24.9 (4.2) 35.2 (2.7) 45.7 (3.0) 57.3 (4.4)

ANOVA F(3,370) = 791.7, p < 0.0001

Validation N (%) 4 (3.8) 4 (3.8) 39 (37.1) 58 (55.2)

N = 105 Mean (SD) 29.13 (2.7) 36.64 (3.1) 47.13 (2.3) 59.38 (4.8)

ANOVA F(3,101) = 152.1, p < 0.0001

Parent Cut Score Range ≤33^* 34–44^* 45–57^* ≥58^*

Calibration N (%) 28 (8.8) 62 (19.4) 147 (46.1) 82 (25.7)

N = 319 Mean (SD) 29.29 (2.5) 39.48 (3.4) 50.55 (3.3) 63.81 (4.2)

ANOVA F(3,315) = 939.9, p < 0.0001

Validation N (%) 4 (4.2) 9 (9.5) 47 (49.5) 35 (36.8)

N = 95 Mean (SD) 30.73 (2.7) 40.85 (3.4) 51.91 (3.5) 62.8 (3.7)

ANOVA F(3,91) = 175.3, p < 0.0001

Daily Routines Child Cut Score Range ≤25 26–36 37–45 46–51^* ≥52^*

Calibration N (%) 7 (1.9) 21 (5.6) 45 (12.1) 63 (16.9) 236 (63.5)

N = 372 Mean (SD) 13.18 (10.3) 30.32 (2.8) 40.38 (2.3) 47.84 (1.6) 59.12 (5.4)

ANOVA F(4,367) = 449.3, p < 0.0001

Validation N (%) 1 (1.0) 3 (2.9) 8 (7.6) 21 (20.0) 72 (68.6)

N = 105 Mean (SD) 23.1 (0) 39.2 (0.8) 40.9 (2.8) 49.7 (1.9) 59.9 (4.3)

ANOVA F(4,100) = 106.8, p < 0.0001

Parent Cut Score Range ≤27^* 28–37^* 38–44^* 45–54^* ≥55^*

Calibration N (%) 10 (3.1) 18 (5.7) 39 (12.3) 126 (39.6) 125 (39.3)

N = 318 Mean (SD) 20.1 (4.1) 31.8 (2.7) 40.9 (2.1) 50.0 (2.7) 61.3 (6.0)

ANOVA F(4,313) = 454.4, p < 0.0001

Validation N (%) 1 (1.1) 4 (4.2) 9 (9.5) 45 (47.4) 36 (37.9)

N = 95 Mean (SD) 24.9 (0) 38.9 (1.1) 42.3 (2.3) 50.7 (2.9) 62.0 (3.8)

ANOVA F(4,90) = 149.2, p < 0.0001

Ambulation Child Cut Score Range ≤32^* 33–45^* 46–51^* 52–59^* ≥60^*

Calibration N (%) 12 (9.8) 28 (22.8) 26 (21.1) 24 (19.5) 33 (26.8)

N = 123 Mean (SD) 22.9 (7.1) 40.3 (3.5) 48.4 (1.4) 54.5 (2.7) 66.6 (5.2)

ANOVA F(4,118) = 319.2, p < 0.0001

Validation N (%) 6 (12.2) 9 (18.4) 6 (12.2) 15 (30.6) 13 (26.5)

N = 49 Mean (SD) 25.8 (3.3) 39.8 (3.7) 49.8 (2.0) 56.2 (2.4) 64.5 (3.7)

ANOVA F(4,44) = 197.9, p < 0.0001

Parent Cut Score Range ≤28^* 29–35^* 36–41^* ≥42^*

Calibration N (%) 29 (25.0) 40 (34.5) 38 (32.8) 9 (7.7)

N = 116 Mean (SD) 23.6 (4.0) 32.2 (2.2) 37.7 (1.8) 46.9 (3.3)

ANOVA F(3,112) = 228.1, p < 0.0001

Validation N (%) 8 (19.1) 17 (40.5) 9 (21.4) 8 (19.1)

N = 42 Mean (SD) 23.1 (3.4) 32.7 (1.9) 39.4 (1.9) 46.7 (4.8)

ANOVA F(3,38) = 97.2, p < 0.0001

Power Wheelchair Child Cut Score Range ≤20^* 21–40^* 41–55 ≥56

Mobility Calibration N (%) 7 (7.9) 51 (57.3) 23 (25.8) 8 (9.0)

N = 89 Mean (SD) 0.3 (24.6) 31.3 (5.3) 46.1 (4.3) 61.3 (2.1)

ANOVA F(3,85) = 90.5, p < 0.0001

Validation N (%) 1 (7.7) 7 (53.9) 4 (30.8) 1 (7.7)

N = 13 Mean (SD) 14.3 (0) 33.5 (3.6) 51.9 (5.0) 60.6 (0)

ANOVA F(3,9) = 37.7, p < 0.0001

Parent Cut Score Range ≤16^* 17–39 40–54 ≥55^*

Calibration N (%) 2 (2.9) 42 (60.0) 17 (24.3) 9 (12.8)

N = 70 Mean (SD) 3.9 (0.4) 32.0 (6.0) 47.0 (4.2) 60.6 (3.6)

ANOVA F(3,66) = 117.4, p < 0.0001

Validation N (%) 1 (7.7) 1 (7.7) 8 (61.5) 2 (15.4)

N = 12 Mean (SD) 16.3 (0) 34.1 (0) 45.1 (5.02) 62.4 (0)

ANOVA F(3,8) = 23.3, p < 0.0003 (Continued)

Manual Wheelchair Mobility Child Cut Score Range ≤27^* 28–40^* 41–47^* 48–62^* ≥63^*

Calibration N (%) 10 (3.5) 25 (8.8) 58 (20.4) 146 (51.2) 46 (16.1)

N = 285 Mean (SD) 17.1 (10.0) 35.1 (3.2) 43.6 (1.9) 53.6 (4.4) 66.4 (3.2)

ANOVA F(4,280) = 604.6, p < 0.0001

Validation N (%) 3 (3.5) 13 (15.1) 10 (11.6) 53 (61.6) 7 (8.1)

N = 86 Mean (SD) 20.4 (1.6) 36.5 (3.4) 44.6 (1.8) 53.8 (3.6) 65.5 (0)

ANOVA F(4,81) = 188.8, p < 0.0001

Parent Cut Score Range ≤30^* 31–40^* 41–49^* 50–59^* ≥60^*

Calibration N (%) 11 (4.6) 17 (7.1) 73 (30.7) 99 (41.6) 38 (16.0)

N = 238 Mean (SD) 18.9 (7.2) 36.0 (2.9) 45.4 (2.6) 53.1 (2.6) 63.7 (3.6)

ANOVA F(4,233) = 604.6, p < 0.0001

Validation N (%) 2 (2.6) 7 (9.0) 25 (32.1) 39 (50.0) 5 (6.4)

N = 78 Mean (SD) 22.2 (0) 35.5 (2.0) 46.0 (2.7) 54.5 (2.8) 64.2 (2.9)

ANOVA F(4,73) = 181.5, p < 0.0001

Domain	Respondent	Sample		Level 1	Level 2	Level 3	Level 4	Level 5
General Mobility	Child		Cut Score Range	≤30^*	31–40^*	41–50^*	≥51^*
		Calibration	N (%)	21 (5.6)	44 (11.8)	122 (32.6)	187 (50.0)
		N = 374	Mean (SD)	24.9 (4.2)	35.2 (2.7)	45.7 (3.0)	57.3 (4.4)
			ANOVA	F(3,370) = 791.7, p < 0.0001
		Validation	N (%)	4 (3.8)	4 (3.8)	39 (37.1)	58 (55.2)
		N = 105	Mean (SD)	29.13 (2.7)	36.64 (3.1)	47.13 (2.3)	59.38 (4.8)
			ANOVA	F(3,101) = 152.1, p < 0.0001
	Parent		Cut Score Range	≤33^*	34–44^*	45–57^*	≥58^*
		Calibration	N (%)	28 (8.8)	62 (19.4)	147 (46.1)	82 (25.7)
		N = 319	Mean (SD)	29.29 (2.5)	39.48 (3.4)	50.55 (3.3)	63.81 (4.2)
			ANOVA	F(3,315) = 939.9, p < 0.0001
		Validation	N (%)	4 (4.2)	9 (9.5)	47 (49.5)	35 (36.8)
		N = 95	Mean (SD)	30.73 (2.7)	40.85 (3.4)	51.91 (3.5)	62.8 (3.7)
			ANOVA	F(3,91) = 175.3, p < 0.0001
Daily Routines	Child		Cut Score Range	≤25	26–36	37–45	46–51^*	≥52^*
		Calibration	N (%)	7 (1.9)	21 (5.6)	45 (12.1)	63 (16.9)	236 (63.5)
		N = 372	Mean (SD)	13.18 (10.3)	30.32 (2.8)	40.38 (2.3)	47.84 (1.6)	59.12 (5.4)
			ANOVA	F(4,367) = 449.3, p < 0.0001
		Validation	N (%)	1 (1.0)	3 (2.9)	8 (7.6)	21 (20.0)	72 (68.6)
		N = 105	Mean (SD)	23.1 (0)	39.2 (0.8)	40.9 (2.8)	49.7 (1.9)	59.9 (4.3)
			ANOVA	F(4,100) = 106.8, p < 0.0001
	Parent		Cut Score Range	≤27^*	28–37^*	38–44^*	45–54^*	≥55^*
		Calibration	N (%)	10 (3.1)	18 (5.7)	39 (12.3)	126 (39.6)	125 (39.3)
		N = 318	Mean (SD)	20.1 (4.1)	31.8 (2.7)	40.9 (2.1)	50.0 (2.7)	61.3 (6.0)
			ANOVA	F(4,313) = 454.4, p < 0.0001
		Validation	N (%)	1 (1.1)	4 (4.2)	9 (9.5)	45 (47.4)	36 (37.9)
		N = 95	Mean (SD)	24.9 (0)	38.9 (1.1)	42.3 (2.3)	50.7 (2.9)	62.0 (3.8)
			ANOVA	F(4,90) = 149.2, p < 0.0001
Ambulation	Child		Cut Score Range	≤32^*	33–45^*	46–51^*	52–59^*	≥60^*
		Calibration	N (%)	12 (9.8)	28 (22.8)	26 (21.1)	24 (19.5)	33 (26.8)
		N = 123	Mean (SD)	22.9 (7.1)	40.3 (3.5)	48.4 (1.4)	54.5 (2.7)	66.6 (5.2)
			ANOVA	F(4,118) = 319.2, p < 0.0001
		Validation	N (%)	6 (12.2)	9 (18.4)	6 (12.2)	15 (30.6)	13 (26.5)
		N = 49	Mean (SD)	25.8 (3.3)	39.8 (3.7)	49.8 (2.0)	56.2 (2.4)	64.5 (3.7)
			ANOVA	F(4,44) = 197.9, p < 0.0001
	Parent		Cut Score Range	≤28^*	29–35^*	36–41^*	≥42^*
		Calibration	N (%)	29 (25.0)	40 (34.5)	38 (32.8)	9 (7.7)
		N = 116	Mean (SD)	23.6 (4.0)	32.2 (2.2)	37.7 (1.8)	46.9 (3.3)
			ANOVA	F(3,112) = 228.1, p < 0.0001
		Validation	N (%)	8 (19.1)	17 (40.5)	9 (21.4)	8 (19.1)
		N = 42	Mean (SD)	23.1 (3.4)	32.7 (1.9)	39.4 (1.9)	46.7 (4.8)
			ANOVA	F(3,38) = 97.2, p < 0.0001
Power Wheelchair	Child		Cut Score Range	≤20^*	21–40^*	41–55	≥56
Mobility		Calibration	N (%)	7 (7.9)	51 (57.3)	23 (25.8)	8 (9.0)
		N = 89	Mean (SD)	0.3 (24.6)	31.3 (5.3)	46.1 (4.3)	61.3 (2.1)
			ANOVA	F(3,85) = 90.5, p < 0.0001
		Validation	N (%)	1 (7.7)	7 (53.9)	4 (30.8)	1 (7.7)
		N = 13	Mean (SD)	14.3 (0)	33.5 (3.6)	51.9 (5.0)	60.6 (0)
			ANOVA	F(3,9) = 37.7, p < 0.0001
	Parent		Cut Score Range	≤16^*	17–39	40–54	≥55^*
		Calibration	N (%)	2 (2.9)	42 (60.0)	17 (24.3)	9 (12.8)
		N = 70	Mean (SD)	3.9 (0.4)	32.0 (6.0)	47.0 (4.2)	60.6 (3.6)
			ANOVA	F(3,66) = 117.4, p < 0.0001
		Validation	N (%)	1 (7.7)	1 (7.7)	8 (61.5)	2 (15.4)
		N = 12	Mean (SD)	16.3 (0)	34.1 (0)	45.1 (5.02)	62.4 (0)
			ANOVA	F(3,8) = 23.3, p < 0.0003	(Continued)
Manual Wheelchair Mobility	Child		Cut Score Range	≤27^*	28–40^*	41–47^*	48–62^*	≥63^*
		Calibration	N (%)	10 (3.5)	25 (8.8)	58 (20.4)	146 (51.2)	46 (16.1)
		N = 285	Mean (SD)	17.1 (10.0)	35.1 (3.2)	43.6 (1.9)	53.6 (4.4)	66.4 (3.2)
			ANOVA	F(4,280) = 604.6, p < 0.0001
		Validation	N (%)	3 (3.5)	13 (15.1)	10 (11.6)	53 (61.6)	7 (8.1)
		N = 86	Mean (SD)	20.4 (1.6)	36.5 (3.4)	44.6 (1.8)	53.8 (3.6)	65.5 (0)
			ANOVA	F(4,81) = 188.8, p < 0.0001
	Parent		Cut Score Range	≤30^*	31–40^*	41–49^*	50–59^*	≥60^*
		Calibration	N (%)	11 (4.6)	17 (7.1)	73 (30.7)	99 (41.6)	38 (16.0)
		N = 238	Mean (SD)	18.9 (7.2)	36.0 (2.9)	45.4 (2.6)	53.1 (2.6)	63.7 (3.6)
			ANOVA	F(4,233) = 604.6, p < 0.0001
		Validation	N (%)	2 (2.6)	7 (9.0)	25 (32.1)	39 (50.0)	5 (6.4)
		N = 78	Mean (SD)	22.2 (0)	35.5 (2.0)	46.0 (2.7)	54.5 (2.8)	64.2 (2.9)
			ANOVA	F(4,73) = 181.5, p < 0.0001

N: Number SD: Standard Deviation ^*Post-hoc analysis pairwise comparisons demonstrated significant differences p < 0.05.

Table 4

PMoP score range for each level, means score, Standard deviation (SD) and ANOVA results

Domain	Respondent	Sample		Level 1	Level 2	Level 3	Level 4	Level 5
Participation Self	Child		Cut Score Range	≤23^*	24–40^*	41–58^*	≥59^*
		Calibration	N (%)	3 (0.8)	60 (16.0)	230 (61.5)	81 (21.7)
		N = 374	Mean (SD)	11.9 (3.2)	35.7 (4.2)	49.2 (4.9)	66.3 (7.6)
			ANOVA	F(3,370) = 425.9, p < 0.0001
		Validation	N (%)	X	27 (25.7)	69 (65.7)	9 (8.6)
		N = 105	Mean (SD)	X	37.4 (3.1)	49.5 (5.2)	63.1 (2.3)
			ANOVA	F(2,102) = 123, p < 0.0001
	Parent		Cut Score Range	≤19^*	20–36^*	37–63^*	≥64^*
		Calibration	N (%)	1 (0.3)	19 (5.9)	266 (83.1)	34 (10.7)
		N = 320	Mean (SD)	14.2	32.3 (3.9)	48.9 (6.7)	71.6 (7.2)
			ANOVA	F(3,316) = 178.7, p < 0.0001
		Validation	N (%)	X	11 (11.6)	82 (86.3)	2 (2.1)
		N = 95	Mean (SD)	X	32.8 (4.7)	51.5 (6.4)	68.2 (0)
			ANOVA	F(2,92) = 53.1, p < 0.0001
Participation Friend	Child		Cut Score Range	≤27^*	28–41^*	42–57^*	≥58^*
		Calibration	N (%)	5 (1.3)	63 (16.8)	220 (58.8)	86 (23.1)
		N = 374	Mean (SD)	24.9 (1.4)	36.8 (3.3)	48.8 (4.6)	64.3 (7.4)
			ANOVA	F(3,370) = 398.4, p < 0.0001
		Validation	N (%)	1 (1.0)	25 (23.8)	66 (62.9)	13 (12.4)
		N = 105	Mean (SD)	25.9 (0)	37.9 (3.1)	49.3 (4.7)	63.4 (3.0)
			ANOVA	F(3,101) = 119.6, p < 0.0001
	Parent		Cut Score Range	≤23^*	24–41^*	42–63^*	≥64^*
		Calibration	N (%)	2 (0.6)	57 (17.8)	214 (66.9)	47 (14.7)
		N = 320	Mean (SD)	16.6 (5.0)	36.6 (3.2)	50.4 (5.6)	69.7 (6.6)
			ANOVA	F(3,316) = 361.3, p < 0.0001
		Validation	N (%)	1 (1.1)	14 (14.7)	73 (76.8)	7 (7.4)
		N = 95	Mean (SD)	18.3	34.8 (5)	51.6 (5.3)	66.2 (2.6)
			ANOVA	F(3,91) = 79.4, p < 0.0001

N: Number SD: Standard Deviation ^*Post-hoc analysis pairwise comparisons demonstrated significant differences p < 0.05.

As demonstrated in Tables 3 4, participant scores were distributed across all levels, except the Participation-Self (Child & Parent) domain (Validation Sample) where no scores were located at Level 1. A comparison of the sample distribution across levels reveals that the Level 1 distribution was smaller than or equal to the distribution in other levels for all domains except Ambulation-Parents (smallest percentage = Level 4). Levels with the highest percentage of the sample vary across domains, samples and respondents.

All pairwise comparisons of level mean values for the Calibration sample demonstrated significant differences. Most pairwise comparisons of level mean values for the Validation sample were significant, with the following exceptions: Parent Power Wheeled Mobility, (Leve1s 2 and 3), Child Powered Wheeled Mobility (Levels 3 and 4) and Child Daily Routines (Levels 1 and 2, Levels 2 and 3).

3.2 PEDI-SCI AM and PMoP ability level descriptions

After the initial meeting, the expert panel reached consensus on PEDI-SCI AM level descriptions for all domains. Panelists agreed that PEDI-SCI AM descriptions captured key aspects of the domain and were appropriate for children of different ages. Investigators made minor revisions to the PEDI-SCI-AM level descriptions to eliminate jargon, ensure comprehension by children 8 years old, and establish consistent wording across domains. However, consensus was not achieved for PMoP level descriptions. Investigators made minor revisions to the PMoP to be consistent with PEDI-SCI AM wording and developed a bulleted format as an option for further review. Next, investigators conducted semi-structured interviews to review initial PMoP level descriptions and included PMoP response scale terms (e.g., a lot less, a little less) in level descriptions.

During the semi-structured interviews and focus group, versions of participation level descriptions were provided and participants were asked questions about the level descriptions (see Table 2). Children understood the purpose of the level and verbalized that it would describe their abilities. Children and parents provided feedback on wording to improve children’s comprehension. Professionals participating in the focus group felt it was important to match the language used in the level description with the language used in the response scale of the PEDI-SCI AM and the PMoP. Consensus on preference for the narrative or bulleted level description was not achieved.

During round 1 of the survey (N = 60), the majority of respondents agreed that level descriptions were written clearly and would be understood by a child as young as 8 years for both domains: Self Participation (66.1–84.1% respondent agreement) and Friend Participation (67.6–100% respondent agreement). Respondents indicated that descriptions were too wordy, the examples were helpful, and that children may not understand the term “all places” and suggested referring to specific places. Respondents also expressed concern that the term “community” would not be understood. Preferences for the bulleted and narrative descriptions were evenly split. Due to the lack of consensus, investigators further refined and developed level descriptions that combined the bullet and narrative versions to review in Survey Round 2.

Round 2 of the survey (N = 37) findings were similar to Round 1 with the majority agreeing that the level descriptions were appropriate for each domain: Self Participation (64.9–84.4% respondent agreement) and Friend Participation (82.8–87.5% respondent agreement). Key feedback provided by participants during this round of the survey was that children may not understand “all places” and what a “team” is. In both Rounds 1 and 2, a majority of the feedback centered around the use of “a lot less,” “a little less” and “as much as.” Survey participants did not agree that children as young as 8 years old would understand the terminology. Therefore, they disagreed with the statement that it was clearly written and could be understood by a child as young as 8 years old. Investigators revised the PMoP level descriptions based on the Round 2 feedback. While the majority of respondents agreed that the descriptions were adequate for all levels, the 80% target for agreement was not achieved for Participation Self Levels 1 (64.9%) and 2 (78.8%). Table 5 presents an example of a level description refinement resulting from the consensus activities and Table 6 presents the final level descriptions for all PEDI-SCI AM and PMoP domains.

Table 5
PMoP Friend, Child Level 2 Description Revision Timeline

Original Description from Consensus Meeting Presented during Semi-Structured Interviews and Focus Groups Presented during Survey: Round 1 Presented during Survey: Round 2 Final PMoP Level Descriptions

I participate a lot less than my friends in most unfamiliar environments. I participate a little less than desired in familiar environments. Narrative: I do things a little less than my friends at home and school. I do a lot less than my friends in all places outside of my home and school. Bullet: A little less than my friends: Home School A lot less than my friends: Community Narrative: I do things a little less than my friends at home and school. I do a lot less than my friends in all places outside of my home and school. Bullet: A little less than my friends: Home School A lot less than my friends: Community Sometimes I do what my friends do at: Home School I do a lot less than my friends at: Other places, like a friend’s house or with a team. Sometimes I do what my friends do at: Home School I do a lot less than my friends at: A friend’s house Other places like a restaurant or park.

Original Description from Consensus Meeting	Presented during Semi-Structured Interviews and Focus Groups	Presented during Survey: Round 1	Presented during Survey: Round 2	Final PMoP Level Descriptions
I participate a lot less than my friends in most unfamiliar environments. I participate a little less than desired in familiar environments.	Narrative: I do things a little less than my friends at home and school. I do a lot less than my friends in all places outside of my home and school. Bullet: A little less than my friends: Home School A lot less than my friends: Community	Narrative: I do things a little less than my friends at home and school. I do a lot less than my friends in all places outside of my home and school. Bullet: A little less than my friends: Home School A lot less than my friends: Community	Sometimes I do what my friends do at: Home School I do a lot less than my friends at: Other places, like a friend’s house or with a team.	Sometimes I do what my friends do at: Home School I do a lot less than my friends at: A friend’s house Other places like a restaurant or park.

Table 6

PEDI-SCI AM and PMoP Final Level Descriptions

	Level 1	Level 2	Level 3	Level 4	Level 5
PEDI-SCI AM General Mobility: Child	I can tell people how to move me and I can sit in my wheelchair.	I can move in bed and I can sit in a chair with a back.	I can sit in a chair move to a bed or another chair at the same height.	I can keep my balance, sit anywhere and move to a seat, even if it is at a different height.
PEDI-SCI AM General Mobility: Parent	My child can direct his/her own care and sit when he/she is fully supported.	My child can roll in bed and sit in a chair with some support.	My child can sit unsupported and move between two surfaces at the same height	My child can move between most surfaces, even if they are not at the same height such as to and from a seat of a car.
PEDI-SCI AM Daily Routines: Child	I can tell people how to help me and I get help for most tasks.	I can feed myself.	I can do some dressing and activities such as throwing a ball and raising my hand in class.	I can do most dressing and can help with my cathing and bowel program.	I can do most of my care, including all dressing and cathing and bowel program, school activities, and chores.
PEDI-SCI AM Daily Routines: Parent	My child can direct his/her own care. He/she needs help to set up tasks.	My child can feed him /herself.	My child can do some dressing and activities with 1 hand such as throwing a ball or overhead tasks such as raising a hand in class.	My child can do most dressing and can help with his/her cathing and bowel program.	My child can do most of his/her care, including all dressing and cathing and bowel management, school activities, and chores by him/herself.
PEDI-SCI AM Ambulation: Child	I only walk in therapy.	I can walk inside on a level floor, but it is hard to move in a crowded room.	I can walk outside when the path is smooth and clear.	I can walk in most places while carrying something, but it is hard to walk all day long.	I can walk almost anywhere, all day long.
PEDI-SCI AM Ambulation: Parent	My child walks in therapy with difficulty	My child walks in therapy and indoors on a level floor but has difficulty with a crowded or busy place.	My child walks in most places (indoors and outdoors) when the path is clear but has difficulty walking for an extended time.	My child walks in most places for most of the day.
PEDI-SCI AM Power Wheeled Mobility: Child	I can move my power wheelchair, but I cannot stay in it all day.	I can move my power wheelchair most places and can usually stay in my power wheelchair most of the day.	I can easily move my power wheelchair in all places, including ramps, turns and on/off lifts.	I can easily move my power wheelchair anywhere, all day long.
PEDI-SCI AM Power Wheeled Mobility: Parent	Your child can move his/her power wheelchair and tolerates being in the wheelchair several hours at a time.	Your child can move his/her power wheelchair in most places, and can usually stay in the wheelchair most of the day.	Your child can easily move his/her power wheelchair in all places, including ramps, turns and on/off lifts.	Your child can easily move his/her power wheelchair in all places, all day long.
PEDI-SCI AM Manual Wheeled Mobility: Child	I can move my manual wheelchair in some places, but it is hard.	I can move my manual wheelchair when the floor or ground is level, but it is a little hard.	I can move my manual wheelchair on level surfaces and inside but it is hard to move outside or on uneven areas.	I can do most manual wheelchair activities, including ramps and uneven surfaces.	I can move around in my manual wheelchair in most places all day long.
PEDI-SCI AM Manual Wheeled Mobility: Parent	Your child is able to move in his/her manual wheelchair on level surfaces but with difficulty.	Your child is able to move his/her wheelchair with some difficulty on level surfaces.	Your child can move his/her manual wheelchair on level surfaces and indoor environments but has difficulty outdoors or on uneven areas.	Your child can do most manual wheelchair activities, including ramps and uneven surfaces with minimal difficulty.	Your child is able to do all manual wheelchair activities in most places all day long.
PMoP Self: Child	I do a lot less than I want in all places. All places could include•My home•School•A friend’s house•Other places like a restaurant or park	Sometimes I do what I want at•Home•School I do a lot less than I want at•A friend’s house•Other places like a restaurant or spark	I almost always do what I want at•Home•School I do a lot less than I want at•A friend’s house•Other places like a restaurant or park	I do as much as I want in all places. All places could include•My home•School•A friend’s house•Other places like a restaurant or park
PMoP Self: Parent	My child does a lot less than he/she wants in all places. All places could include•Home•School•A friend’s house •Other places like a restaurant or park	Sometimes my child does what he/she wants at•Home•School My child does a lot less than he/she wants at•A friend’s house•Other places like a restaurant or park	My child almost always does what he/she wants at•Home •School My child does a lot less than he/she wants at•A friend’s house•Other places like a restaurant or park	My child does as much as he/she wants in all places. All places could include•Home •School•A friend’s house•Other places like a restaurant or park
PMoP Friend: Child	I do a lot less than my friends in all places. All places could include•My home•School•A friend’s house•Other places like a restaurant or park	Sometimes I do what my friends do at:•Home•School I do a lot less than my friends at:•A friend’s house•Other places like a restaurant or park	I almost always do what my friends do at:•Home•School I do a lot less than my friends at:•A friend’s house•Other places like a restaurant or park	I do as much as my friends do in all places. All places could include•Home•School•A friend’s house•Other places like a restaurant or park
PMoP Friend: Parent	My child does a lot less than friends in all places. All places could include•Home•School•A friend’s house•Other places like a restaurant or park	Sometimes my child does what friends do at:•Home•School My child does a lot less than friends at:•A friend’s house•Other places like a restaurant or park	My child almost always does what friends do at:•Home•School My child does a lot less than friends at:•A friend’s house•Other places like a restaurant or park	My child does as much as friends do in all places. All places could include•Home•School•A friend’s house •Other places like a restaurant or park

4 Discussion

We were able to meet the study objective and develop and validate clinically relevant ability levels for the PEDI-SCI AM and PMoP along with descriptions of activities that characterize each level. The bookmarking methods using item maps provided a sound method to establish cut-points that define different ability levels. A recent study used a similar methodology to identify different levels of participation among adults post-burn injury based on the Life Impact Burn Recovery Evaluation (LIBRE) Profile scores. The study identified LIBRE Profile score cut-points that characterize four levels of participation for Social Activities, Social Interactions, Romantic Relationships, Sexual Relationships, and Work and Employment domains and five levels for the Relationships with Family and Friends domain. Level cut-points for each domain were accompanied by descriptions of the abilities associated with the specified level [19]. In our study, for each PEDI-SCI AM and PMoP domain, level cut-points established qualitatively by the expert panel were confirmed by quantitative analyses and demonstrated significant differences between mean scores for all levels. Mean scores increased progressively across levels, and values corresponded to SDs above and below the mean. Thus, levels can be used to translate numeric PEDI-SCI AM and PMoP scores into clinically relevant language and provide a means to describe change over time. As an example, instead of reporting that child’s PEDI-SCI AM General Mobility (Parent-report) score changed from 35 (1.5 SDs below the mean) to 50 (at the mean), assessment results can be communicated by stating the child’s PEDI-SCI AM General Mobility score changed from Level 2 abilities described as “My child can roll in bed, move from the edge of the bed to lying down, and sit with some support” to Level 3 abilities described as “My child can sit unsupported and move between two surfaces at the same height.” This language provides opportunities to discuss the assessment findings, communicate progress, and discuss patient goals.

A review of the sample distribution across levels revealed that the percentage of the sample at Level 1 was very low and for some domains, no scores were located in Level 1. This finding may be viewed as evidence of the need to increase the upper limit of Level 1 cut-points to include greater sample representation. However, it is important to be able to identify children with lower scores who may be a higher risk for poor outcomes; therefore, the Level 1 cut-points were maintained to be better able to identify children at risk. There was also overlap when summarizing score ranges between levels 4 and 5. Because each domain is a unique metric with different ranges of high and low scores, the range of Level 5 is within Level 4. Additionally, there are 4 levels for some domains, while other domains include 5 levels. This difference between domains accounts for the overlap when summarizing the range of scores for levels 4 and 5.

A significant strength of the study is the replication of analyses in a separate Validation Sample; however, some study limitations resulted from including the Validation Sample. The relatively smaller size of the Validation Sample (N = 106 children with SCI; 96 parents of children with SCI) resulted in differences in the distribution of two samples across levels, especially for domains that were not administered to all participants (i.e., Ambulation, PWC, MWC). The use of different methods for administering items (full item bank versus CAT) of PEDI-SCI AM and PMoP administration in the two study samples is another study limitation. While all items were administered to the Calibration Sample, 15-item CATs were administered to the Validation Sample. However, these concerns are lessened due to the fact that previous work demonstrated a strong correlation between a 15-item stimulated CAT and the full item bank [4 –6].

A limitation of this study is underrepresentation of some groups of children with SCI such as those with high-level tetraplegia, power wheelchair users and those who ambulate. Furthermore, validation sample participants tended to be higher functioning with fewer children with SCI using power wheeled mobility, which may account for the non-significant differences between levels for some domains (i.e.: Power Wheeled Mobility and Daily Routines).

It was much more challenging to describe participation levels than activity levels. Activities are relatively easy to describe and the expert panel reached consensus on PEDI-SCI AM activity domain ability level descriptions after the first meeting. In contrast, participation is a complex construct that involves engaging in different social and community settings. Despite multiple efforts to define participation levels, our ability to achieve consensus targets for PMoP domain level descriptions remained elusive. Even though two level descriptions in one domain did not achieve 80% agreement, the 80% agreement was achieved for most level descriptions.

Survey participants were asked to comment on why they disagreed and most feedback was surrounding the use of the terms “a lot less,” “a little less” and “as much as.” Survey participants described these terms as vague, unclear, confusing and too abstract. It was their opinion that children as young as 8 years of age would not understand. While additional changes were made to the level descriptions following Round 2 of the survey, the investigators made the decision to finalize the level descriptions even though two levels did not reach 80% agreement. In previous work by the investigators, cognitive interviews were conducted with children with SCI. This work demonstrated an understanding by children as young as 8 years of the “a lot less,” “a little less” and “as much as” responses [2–3].

During the semi-structured interviews, focus group and Round 1 of the survey, we presented PMoP level descriptions in both a narrative and bulleted format. There was no consensus, and in the survey the results were split 50% for the narrative version and 50% for the bulleted. The investigators made the decision to combine both, enabling the level description to maintain the detail of the narrative version but reduce the words using the bullet.

5 Conclusion

Study findings enhance PEDI-SCI AM and PMoP assessments by providing clinical meaning for scores and enhancing communication between providers and children with SCI, parents and caregivers of children with SCI. When PEDI-SCI AM and PMoP assessments are administered longitudinally, changes in levels can be used to set goals, monitor goal achievement and assess plans of care.

Future research can extend these initial validation efforts to include a larger group of children with SCI with high level tetraplegia and those who ambulate. Next steps involve integrating PEDI-SCI AM and PMoP levels and descriptions into score reports to provide a context for interpreting numeric scores.

Footnotes

Acknowledgments

This work was supported by the following grants: Craig H Neilsen Foundation, Grant #360930, Mulcahey PI and Shriners Hospitals for Children, Grant #71003, Mulcahey PI.

Declaration of interest

The authors have no interests or conflicts to declare.

References

Calhoun

, Haley

, Riley

, Vogel

, McDonald

, Mulcahey

. Development of items designed to evaluate activity performance and participation in children and adolescents with spinal cord injury. Int J Pediatr. 2009;2009:854904. doi: 10.1155/2009/854904

Mulcahey

, Calhoun

, Riley

, Haley

. Children’s reports of activity and participation after sustaining a spinal cord injury: A cognitive interviewing study. Dev Neurorehabil. 2009;12(4):191–200. doi: 10.1080/17518420902998177

Mulcahey

, DiGiovanni

, Calhoun

, Homko

, Riley

, Haley

. Children’s and parents’ perspectives about activity performance and participation after spinal cord injury: Initial development of a patient-reported outcome measure. Am J Occup Ther. Jul-Aug. 2010;64(4):605–13. doi: 10.5014/ajot.2010.08148

Mulcahey

, Calhoun

, Tian

, Ni

, Vogel

, Haley

. Evaluation of newly developed item banks for child-reported outcomes of participation following spinal cord injury. Spinal Cord. 2012;50(12):915–9. doi: 10.1038/sc.2012.80

Bent

, Mulcahey

, Kelly

, Calhoun

, Tian

, Ni

, et al. Validity of computer adaptive tests of daily routines for youth with spinal cord injury. Top Spinal Cord Inj Rehabil. Spring. 2013;19(2):104–13. doi: 10.1310/sci1902-104

Slavin

, Mulcahey

, Calhoun Thielen

, Ni

, Vogel

, Haley

, et al. Measuring activity limitation outcomes in youth with spinal cord injury. Spinal Cord. 2016;54(7):546–52. doi: 10.1038/sc.2015.194

Tao

, Haley

, Coster

, Ni

, Jette

. An exploratory analysis of functional staging using an item response theory approach. Arch Phys Med Rehabil. 2008;89(6):1046–53. doi: 10.1016/j.apmr.2007.11.036

Rothrock

, Cook

, O’Connor

, Cella

, Wilder Smith

, Yount

. Establishing clinically relevant terms and severity thresholds for Patient Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting. Qual Life Res. 2019;28(12):3355–62. doi: 10.1007/s11136-019-02261-2

Jensen

, Rothrock

, DeWitt

, Spiegel

, Tucker

, Crane

, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53(2):153–9. doi: 10.1097/MLR.0000000000000289

10.

Lohr

, Aaronson

, Alonso

, Burnam

, Patrick

, Perrin

, et al. Evaluating quality-of-life and health status instruments: Development of scientific review criteria. Clin Ther. Sep-Oct. 1996;18(5):979–92. doi: 10.1016/s0149-2918(96)80054-3

11.

Jette

, Tao

, Norweg

, Haley

. Interpreting rehabilitation outcome measurements. J Rehabil Med. 2007;39(8):585–90. doi: 10.2340/16501977-0119

12.

Cizek

, Bunch

, Koons

. Setting performance standards: Contemporary methods. Educ Meas: Issues Pract. 2004;23(4):31–50. doi: 10.1111/j.1745-3992.2004.tb00166.x

13.

Cook

, Victorson

, Cella

, Schalet

, Miller

. Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning, and sleep disturbance using standard setting with patients and providers. Qual Life Res. 2015;24(3):575–89. doi: 10.1007/s11136-014-0790-9

14.

Marfeo

, Ni

, Chan

, Rasch

, McDonough

, Brandt

, et al. Interpreting physical and behavioral health scores from new work disability instruments. J Rehabil Med. 2015;47(5):394–402. doi: 10.2340/16501977-1947

15.

Sinha

, Slavin

, Kisala

, Ni

, Tulsky

, Jette

. Functional ability level development and validation: Providing clinical meaning for Spinal Cord Injury Functional Index Scores. Arch Phys Med Rehabil. 2015;96(8):1448–57. doi: 10.1016/j.apmr.2014.11.008

16.

Stelmack

, Szlyk

, Stelmack

, Babcock-Parziale

, Demers-Turco

, Williams

, et al. Use of Rasch person-item map in exploratory data analysis: A clinical perspective. J Rehabil Res Dev. 2004;41(2):233–41. doi: 10.1682/jrrd.2004.02.0233

17.

Ryser

, Wright

, Aeschlimann

, Mariacher-Gehler

, Stucki

. A new look at the Western Ontario and McMaster Universities Osteoarthritis Index using Rasch analysis. Arthritis Care Res. 1999;12(5):331–5. doi: 10.1002/1529-0131(199910)12:5<331::aid-art4>3.0.co;2-w

18.

Lin

. The Bookmark Procedure for Setting Cut-Score and Finalizing Performance Standards: Strengths and Weaknesses. Alberta Journal of Educational Research. 2006;52(1):36–52.

19.

Slavin

, Ryan

, Schneider

, Acton

, Amaya

, Saret

, et al. Interpreting Life Impact Burn Recovery Evaluation Profile Scores for Use by Clinicians, Burn Survivors, and Researchers. J Burn Care Res. 2021;42(1):23–31. doi: 10.1093/jbcr/iraa100