Abstract
Background:
Multiple studies have demonstrated the National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) to be a responsive and efficient measure for patients undergoing orthopaedic surgery. While these studies were rigorous in their protocol and methodology, no efforts in recent literature have been made to identify if these reference scores apply to elite athletes.
Purpose/Hypothesis:
The purpose of this study was to determine whether there is a difference in the baseline scores of elite athletes versus the general population. We hypothesized that athletes’ PROMIS upper extremity general function (PROMIS-UE) and general physical function (PROMIS-PF) scores would vary substantially from the mean health state of the general population. We further hypothesized that these scores would be affected by specific sport and level of competition
Study Design:
Cross-sectional study; Level of evidence, 3.
Methods:
Three PROMIS computer adaptive test (CAT) domains were administered to elite athlete (≥18 years) volunteers (either in person or through email). An elite athlete was defined as one participating in sports at the collegiate level or higher. Test domains included PROMIS-PF, PROMIS-UE, and pain interference (PROMIS-PI). PROMIS domain t scores were defined and assessed against NIH reference values to identify significant differences. Distribution analysis was conducted using histograms and normality assessments. Domains were also subject to correlation analysis. Finally, subgroup analysis was conducted for all athlete characteristics to identify any factors associated with variance.
Results:
In total, 196 elite athletes (mean age, 21.1 years; range, 18.0-36.7 years) completed all 3 PROMIS-CAT forms. Overall, the mean scores were 56.0 ± 6.4, 58.1 ± 7.7, and 47.1 ± 7.3 for PROMIS-UE, PROMIS-PF, and PROMIS-PI, respectively. Distribution analysis showed nonnormal distribution for all 3 PROMIS domains (Kolmogorov-Smirnov test, P < .001). Similarly, in all 3 PROMIS domains the athletes displayed more disparate scores than the NIH-reported reference values (1-way sign test, P < .001). Only the presence of pain and sport played showed association with variance in PROMIS domain scores (P < .001 and P = .003, respectively).
Conclusion:
Elite athletes displayed more disparate reference scores than the NIH-reported average of 50 for PROMIS-UE, PROMIS-PF, and PROMIS-PI. Furthermore, these forms were sensitive to varying levels of sport among collegiate athletes.
Patient-reported outcome measures (PROMs) are survey-based tools that allow physicians to quantify patient symptoms and complaints in both nonoperative and operative patients. 3 As medicine continues to focus on a patient-centered approach, these types of surveys have gained in popularity. The Patient-Reported Outcomes Measurement Information System (PROMIS) is a particular PROM that has emerged as an efficient and reliable measure of patient health in an orthopaedic setting. PROMIS was developed by the National Institutes of Health (NIH) in 2004 with the goal of standardizing and promoting a common PROM across all realms of clinical research. 3 The PROMIS computer adaptive test (CAT) system allows for specific questions to be tailored toward each patient’s unique presentation. This allows for decreased question burden on the patient and an easier facilitation into clinical practice.
PROMIS has shown validity across many orthopaedic surgery patient populations. Recently, studies have shown PROMIS to be valid in the arthroplasty, 13,20 upper extremity, 18 foot and ankle, 16 and pediatric 12,14 surgical populations. These studies have sought to identify reference scores for their patients, identify responsiveness to change after surgery, and validate the PROMIS domains in assessing different symptomatic states. While there has been considerable research done evaluating PROMIS in patients based on the nature of their surgical complaint, there have not been PROMIS studies on one of the most common subset of patient populations who are evaluated at sports medicine clinics—athletes.
Athletes represent a unique patient population. They are routinely exposed to high competition loads, increased amounts of mechanical stress, and pressure to be in peak physical form. 8,24 As a result of these factors, there can be many physical differences between athletes and nonathletes, including body composition, 15 amount of body fat, 17 and even health-related quality of life. 22 These differences can lead to different baseline levels of physical function and day-to-day activity in athletes compared with nonathletes. Thus, pre- and postsurgical symptom states can also differ significantly between athletes and nonathletes. However, physicians routinely use PROMIS-CAT domains to assess both athletes and nonathletes alike without assurance of the survey’s generalizability.
To our knowledge, there have been no studies examining the PROMIS scores of elite athletes undergoing evaluation for sports medicine–related complaints. The purpose of this study was to (1) determine whether there is a difference in the baseline scores of elite athletes versus the published values for the general population; (2) characterize baseline PROMIS upper extremity physical function, general physical function, and pain interference scores in elite athletes; and (3) identify any athlete factors that may influence these PROMIS domains. By assessing self-reported physical capabilities and functioning in the upper extremity and, more generally, through PROMIS-CAT domains, physicians can better understand the range of values that indicate baseline physical functioning in elite athletes. Similarly, the aforementioned differences in athletes may lead to distinctions between their self-assessments of the burden of pain on the relevant aspects of their life.
Methods
This study was approved by our institutional review board. Participants were recruited from a single metropolitan region by both in-person and electronic data collection. All participants were sourced through measures such as asking collegiate coaches, administrators, or athletes to distribute surveys among team members or through in-person administration at athlete-centered performance centers and school buildings. Questionnaires were emailed or administered on a tablet computer using Research Electronic Data Capture (REDCap), a web-based Health Insurance Portability and Accountability Act--compliant data management and collection application maintained by Vanderbilt University. 9 Inclusion criteria were participants at least 18 years of age who were actively competing in collegiate- or professional-level athletics and who could communicate adequately in English to complete the forms. Athletes participating at the collegiate level or higher were considered elite athletes for the purposes of this study.
The study was conducted between February 1, 2019, and August 1, 2019, during off-season training for each sport. Each participant completed a general intake form (Appendix 1) detailing their age, height, weight, sex, and preexisting pain or surgery, as well as a sports characteristics form (Appendix 2), before completing 3 PROMIS-CAT forms: PROMIS Upper Extremity physical function CAT V 2.0 (PROMIS-UE), PROMIS Physical Function CAT V 2.0 (PROMIS-PF), and PROMIS Pain Interference V 1.1 (PROMIS-PI). All PROMIS domains are calibrated to a mean t score of 50 and a standard deviation of 10. A greater score indicates a greater measurement of the domain in question; for example, a higher PF score demonstrates more function, while a higher PI score demonstrates a greater impact of pain on quality of life. Any participants who did not complete the entire questionnaire set (intake form, demographics form, and 3 PROMIS-CAT forms) were excluded from analysis. After exclusion, 196 out of 212 total forms (92.5%) were kept for statistical analysis.
Statistical Analysis
Participant characteristics were assessed using descriptive statistics, such as mean, standard deviation, maximum, and minimum. PROMIS domain t scores were also subject to summary statistics, such as median and interquartile range (IQR), to describe the central tendency and dispersion. One-sample sign tests were used to compare our athlete-centered data with the previously published normative value of 50. 2 Histograms were created for PROMIS domain scores and were analyzed for floor and ceiling effects, which measure the ability of a questionnaire to differentiate among those respondents at both extremes of the scale. For our analysis, a floor or ceiling effect of at least 15% was considered important. 23 Histograms were also visually examined to determine if normal distribution could be assumed. In addition, Kolmogorov-Smirnov and Shapiro-Wilk tests were used to verify the normality of these distributions.
Statistical significance was set at P < .05. Spearman rho correlations were used to evaluate the relationship between the 3 PROMIS domains. Correlation coefficients were interpreted as follows: high (>0.7), high-moderate (0.61-0.69), moderate (0.4-0.6), moderate-weak (0.31-0.39), or weak (≤0.3). 21 Finally, Kruskal-Wallis tests were performed to identify any significant variance among the PROMIS domains due to underlying factors, after which Mann-Whitney U tests were implemented to compare any individual groups that displayed significant variance. Significance values were adjusted by the Bonferroni correction for multiple tests. All analyses were performed using SPSS Statistics for Windows Version 35.0 software (IBM Corp).
Results
This study included 196 athlete participants (mean age, 21.1 years; range, 18.0-36.7 years), which is a similar cohort size to other PROMIS validation studies. 1,4,8,10,12 Participants competed in varying levels of competition: 6 professional athletes (3%), 47 National Collegiate Athletic Association (NCAA) Division I athletes (24%), and 143 Division II or III athletes (73%). The mean PROMIS-CAT scores were 56.0 ± 6.4, 58.1 ± 7.7, and 47.1 ± 7.3, for PROMIS-UE, PROMIS-PF, and PROMIS-PI, respectively. Full participant characteristics can be seen in Table 1. One-sample sign tests showed significant differences between the medians of each PROMIS domain and the reference population median of 50 (P < .001).
Participant Characteristics a
a Data are reported as mean ± SD (range) or n (%). BMI, body mass index; NCAA, National Collegiate Athletic Association.
PROMIS-CAT domains were assessed for their distribution normality in this cohort. Histograms were created and displayed a nonnormal distribution (Figure 1). Floor and ceiling effect analysis revealed a large ceiling effect (50%) in PROMIS-UE and a floor effect (33.7%) present in PROMIS-PI. Kolmogorov-Smirnov and Shapiro-Wilk tests also displayed nonnormal distributions (P < .001 for all 3 domains). Further distribution statistics, such as median and IQR, as well as floor and ceiling effects, are displayed in Table 2.
Histograms of Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive test (CAT) domain t score distribution in elite-level athletes. Normative reference values, provided by the NIH, are denoted as a means of comparison with the cohort. (A) PROMIS Upper Extremity Physical Function CAT. (B) PROMIS Physical Function CAT. (C) PROMIS Pain Interference CAT.
Distribution Analysis of PROMIS-CAT Domains a
a Substantial (>15%) ceiling and floor effects are shown in bold. CAT, computer adaptive test; IQR, interquartile range; PF, Physical Function; PI, Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System; UE, Upper Extremity Physical Function.
As summarized in Table 3, significant correlations were noted among all 3 PROMIS-CAT domains. PROMIS-UE and PROMIS-PF displayed a significant moderate correlation with one another (Spearman rho = 0.501; P < .001). Similarly, PROMIS-UE and PROMIS-PI showed a moderate negative correlation (–0.541; P < .001), while PROMIS-PF and PROMIS-PI showed a high-moderate negative correlation (–0.657; P < .001). No significant correlations were identified between age or body mass index and any of the 3 PROMIS-CAT domains.
Interdomain Correlations a
a CAT, computer adaptive test; PF, Physical Function; PI, Paint Interference; PROMIS, Patient-Reported Outcomes Measurement Information System; UE, Upper Extremity Physical Function.
Mann-Whitney U tests revealed significant differences between athletes experiencing pain and those not experiencing pain for PROMIS-PF and PROMIS-PI, across the entire cohort (P < .001). However, Mann-Whitney U tests did not show significant differences in any of the PROMIS domains among participants who had undergone surgery in the past 12 months compared with those who had not. Similarly, no differences were noted in any of the PROMIS domains based on the following factors: sex, position within sport, starter versus nonstarter, years active, or level of competition. Subgroup analysis did show unequal distributions for PROMIS-UE (Kruskal-Wallis, P = .004) when sport was considered. Specifically, baseball players were evaluated with significantly higher PROMIS-UE scores than football players (Mann-Whitney U test, +3.2; P = .003).
Discussion
The principal finding of our study was that PROMIS-CAT physical function scores significantly exceeded the stated reference of 50. For PROMIS-UE, the average score in elite athletes was 56, while the score for PROMIS-PF was approximately 58. Both of these values represent a large deviation from the NIH-reported mean values from the general public. If NIH-reported standard deviations are used, athlete scores were between 0.5 and 1.0 SD away from reported norms. These differences can play an important role when evaluating athletic patients’ outcomes after orthopaedic interventions. Significant ceiling effects (50.0%) for PROMIS-UE and floor effects (33.7%) for PROMIS-PI were found in this cohort, which suggests that higher-level questioning may be needed to further differentiate high-level athletes. These ceiling and floor effects speak to the nonnormal distributions we found when assessing these PROMIS domains, further distinguishing our elite athlete population from the reference population.
Significant ceiling effects for PROMIS-UE and floor effects for PROMIS-PI represent a limited ability for PROMIS-CAT domains to accurately distinguish between the higher physical function and lower pain levels of asymptomatic elite athletes. This suggests that the primary uses of the PROMIS-CAT domains should be to assess patients with known or suspected functional or pain limitations and not to distinguish between functional deficits in asymptomatic elite athlete cohorts. When functionally impaired knee or shoulder surgery patients were surveyed, 1 study found negligible ceiling and floor effects (<2% for PROMIS-PF and PROMIS-PI). 6 Still, some studies have identified varying levels of ceiling and floor effects for PROMIS-CAT in impaired patient populations undergoing orthopaedic procedures. Ceiling effects as great as 28% were identified in patients undergoing upper extremity fracture care, 5 and floor effects as great as 19% were identified in patients undergoing cervical spine surgery. 1 Understandably, our reported values were higher because of the nonclinical nature of our cohort. Kaat et al 10 echoed our findings with their own PROMIS-UE ceiling effects of 57% in an unimpaired, albeit nonathlete, group. While these ceiling and floor effects limit PROMIS-CAT domain use, a recent finding suggests that these domains are still as good as or better than comparable-length measures, such as the Functional Index of Hand Osteoarthritis or Michigan Hand Questionnaire Activities of Daily Living subscale, with regard to ceiling effects. 7 The lack of generalizability at the upper extremes hints at a need for higher-level questioning in order to differentiate high-level function and low-level pain in the elite athlete.
To date, the literature regarding the assessment of PROMIS outside of the clinical realm is slim. Rather than baseline scores from a healthy population, clinicians are provided with reference values by the NIH that were obtained using both clinical and general public cohorts. 19 Although initial validation studies did underline an association between worse scores and patient comorbidities, they did not provide clinicians with a healthy, asymptomatic reference with which to compare their patient populations. Franovic et al 4 assessed a pain- and disability-free cohort (N = 294) that displayed similar values, to the current study, for PROMIS-UE (56) and PROMIS-PF (60), although without controlling for physical activity levels. Madsen et al 11 assessed 60 healthy NCAA Division I and III athletes and revealed a value of 60 for PROMIS-PF. The limited literature available assessing physical function and pain through PROMIS domains reflects a stark difference from the NIH-provided reference values. These differences highlight the heterogeneity in orthopaedic patient populations, and physicians are urged to consider these differences when using PROMIS to evaluate feasibility for surgery or recovery from surgery. The present study analyzed a large elite athlete cohort, with analysis of athlete characteristics to provide orthopaedic surgeons with a comparison of the athletic population to the general patient population. Baseline values in the athletic population were found to significantly differ from baseline values of the general population as reported by the NIH. Our findings suggest that the elite athlete generally has a higher level of physical function and lower pain scores compared with the general population. This is important in establishing the appropriate baseline for athletes before intervention and when judging improvement or decline after surgical intervention.
While substantial ceiling and floor effects may govern the efficacy of PROMIS use in healthy elite athletes, previous studies have shown great sensitivity and specificity to injury in athletes. 11 Madsen et al 11 recruited 153 NCAA Division I and III athletes and divided them into healthy, injured and participating, and injured and not participating groups. Their analysis revealed that PROMIS-PF was able to distinguish between all 3 groups and thus was a viable tool toward assessing injuries among collegiate athletes. Our study results showed extensive interdomain correlation as one means of validity and our subgroup analysis uncovered significant differences in PROMIS-PF and PROMIS-PI scores between athletes experiencing and not experiencing pain. Furthermore, significantly lower average scores in PROMIS-UE were noted for football players when compared with baseball players. While no other athlete factors seemed to influence PROMIS domains, these results demonstrate the ability of PROMIS to differentiate between symptomatic states and between sports.
Our study does present with notable limitations. First, we acknowledge that although data were collected during respective sports’ off-seasons, the time from the previous season and the time until the next season were not evaluated or controlled for. Data taken during off-season may therefore not directly correlate with athlete states during different phases of their season. Furthermore, the PROMIS-CAT domains were presented in English, thereby excluding individuals not able to communicate adequately in English and limiting the generalizability of our results. Also, recruitment of female elite-level athletes, as well as professional-level athletes, was far less fruitful than their respective counterparts. The lack of recruitment in these 2 fields suggests a potential of selection bias in our recruiting methods. Because of the nature of our methodology and the protection of student-athlete information, much of our data collection was contingent on coaches and school-authorized personnel distributing survey links to their respective student-athletes. Thus, the true response rate and distribution of surveys were mediated by individuals outside of our research team and present as notable limitations of this study. These subgroups may not offer clinicians with true and statistically significant reference scores, and these limitations may have influenced the results, although no statistically significant data between the sexes were noted.
Conclusion
Elite athletes displayed more disparate reference scores than the NIH-reported average of 50 for PROMIS-UE, PROMIS-PF, and PROMIS-PI. Furthermore, these forms were sensitive to varying levels of sport among collegiate athletes.
Footnotes
Final revision submitted July 2, 2020; accepted July 9, 2020.
One or more of the authors has declared the following potential conflict of interest or source of funding: K.R.O. has received educational support from Medwest and Smith & Nephew and hospitality payments from Medical Device Business Services, Stryker, Wright Medical, and Zimmer Biomet. E.C.M. has received educational support from Arthrex and Pinnacle and consulting fees from Smith & Nephew. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Henry Ford Health System (ref No. 11361).
Appendix 1
General Intake Form for This Study
Appendix 2
Sports Demographics Form for This Study