Outcome Instruments in Spinal Trauma Surgery: A Bibliometric Analysis

Introduction

Patient-reported outcome instruments (PROIs) have become the gold standard for measuring treatment success in evidence-based medical research. The advantage of PROIs over other outcome measurements, such as radiographs, complication rates, or process of care measures, is that only PROIs directly address the patient's perception of their quality-of-life improvement. Studies of patient-reported outcomes are therefore integral to proving the effectiveness of treatments as our health care system changes, yet at this juncture no PROI has been designed for the specific needs of spinal trauma surgery research. ¹ , ²

High costs and broad impacts on patient quality of life make spine surgery particularly well suited for the application of PROIs. An estimated 160,000 spinal injuries occur per year in North America. The majority of these injuries are in the young or working population, ³ , ⁴ so return to work and sustained quality-of-life outcomes, measured by many PROIs, are of great importance to evaluating the effectiveness of an intervention.

Another important clinical outcome in the treatment of spinal injuries is neurologic deficit. A large retrospective study at a level 1 trauma center in the United States found neurologic deficit present in approximately 20% of spinal trauma cases. ⁴ Neurologic deficit is measured using neurologic function scales, such as the Frankel grade scale or American Spinal Injury Association Impairment Scale (AIS), which are administered by medical professionals rather than self-reported by patients.

As more studies incorporate patient outcomes as end points, it is important for researchers to use the available tools in a consistent fashion so that their results can be interpreted and compared across studies. To standardize the use of existing instruments in spinal trauma surgery research or to develop a spinal trauma–specific PROI, it is first necessary to consider how PROIs are currently used in the field. Therefore, a review of spinal trauma literature over the past 10 years was performed to identify the outcome instruments currently in use, their frequency, and usage trends. We hypothesize that a wide variety of PROIs are currently used in spinal trauma surgery and that no instrument captures all the domains of interest in spinal trauma research.

Methods

A review was conducted of articles published in five top orthopedics journals—Spine, European Spine Journal (ESJ), The Spine Journal, The Journal of Bone and Joint Surgery American (JBJS [Am]), and The Journal of Bone and Joint Surgery (British Volume) ^* (JBJS [Br])—from 2004 to 2013. Journals were selected based on impact factor and a track record of publishing relevant, high-quality spine research. The titles of all clinical articles published in these journals were screened on PubMed. The titles referring to clinical studies of surgical interventions in which outcomes were measured by PROIs or neurologic function scales were included for the analysis (Fig. 1). If the inclusion criteria could not be assessed from the title alone, the abstract was reviewed. The full-text article was reviewed if ambiguity remained. Review articles were excluded as were articles that had level 5 evidence. Level of evidence (LOE) was determined according to the definitions provided by the Oxford Center for Evidence Based Medicine. ⁵ Only articles having LOEs 1 to 4 were included in our study. For each study that met the inclusion criteria, the following variables were recorded: title, author, year, LOE, sample size, general diagnosis, and outcome instruments used. The literature search and assignment of the LOE was performed by a spine fellow and three medical students he trained for these tasks. The analysis of outcome measures and their domains was performed by an attending spine surgeon, two fellows, and three medical students.

OPEN IN VIEWER

Fig. 1

Flow diagram depicting literature search strategy used to screen and identify clinical studies of surgical interventions in spinal trauma that report use of outcomes instruments (patient-reported outcome instruments or neurologic function scales). Abbreviation: LOE, level of evidence.

All studies in this review are of patients with the diagnoses of spinal trauma. Specific diagnoses within spinal trauma included cervical, thoracic, and lumbar burst fracture, odontoid fracture, cervical transverse process fracture, sacral fracture, cervical dislocation, and spinopelvic dissociation. The trend in outcome instrument usage over time was reported along with the number of studies published by journal and frequency of each LOE.

PROIs encountered were categorized into subgroups of pain scales (e.g., visual analog scale [VAS]), functional/disability scales (e.g., Oswestry Disability Index [ODI]), and general health-related quality of life (HRQoL) questionnaires (e.g., Short Form 36 [SF-36]). The top five most frequently used outcome instruments were recorded and analyzed. Additionally, errors and inconsistencies of PROI score reporting that were identified as problematic for interstudy comparisons were analyzed and are reported in Table 1.

OPEN IN VIEWER

Table 1

Errors and inconsistencies of patient outcome score reporting

Articles excluded for:

1. Not reporting numerical data (e.g., graphical or statistical reporting of PROI scores)

2. Reporting only change in scores without preoperative or postoperative scores

3. Failing to report preoperative or postoperative scores

4. Significantly modifying a standard PROI

5. Reporting only domain or component scores rather than total score for a PROI

6. Baseline scores measuring health state prior to the spinal injury as recalled by patient

Abbreviation: PROI, patient-reported outcome instrument.

Results

From 2004 to 2013, Spine, ESJ, The Spine Journal, JBJS (Am), and JBJS (Br) published 19,736 articles. We identified 1,079 clinical studies of surgical interventions that made use of one or more outcome measures. Of these, 58 articles focused on spinal trauma. Outcome instrument use increased over the past 10 years from 3 studies in 2004 to 11 studies in 2013, although not in a consistent progression (Fig. 2). ESJ published the most studies using outcome instruments with 30 (51.7%) of the 58 total articles in this study (Fig. 3). ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³

OPEN IN VIEWER

Fig. 2

Number of studies reporting use of outcome instruments by year.

OPEN IN VIEWER

Fig. 3

Number of studies using outcome instruments by journal. Abbreviations: ESJ, European Spine Journal; JBJS, Journal of Bone and Joint Surgery.

Overall, 33 unique outcome instruments were used, including 26 named instruments and 7 improvised questionnaires. Table 2 lists the instruments that were used more than once. PROI use statistics are broken down by PROI category and type in Table 3. The top five outcome instruments were the VAS pain scale (25 uses, 43.1% of articles), SF-36 (20 uses, 34.5%), Frankel grade scale (15 uses, 25.9%), ODI (12 uses, 20.7%) and the American Spinal Injury AIS (9 uses, 15.5%; Fig. 4). The total usage of these top five instruments accounted for 66.9% (81 uses) of all instrument uses. An average of 2.1 outcome instruments were used per study (range 1 to 8). In descending order, the top five most common combinations of two PROIs were SF-36 with VAS (9 studies), ODI with VAS (7 studies), SF-36 with ODI (6), VAS with AIS (5), and tied for fifth were VAS with Frankel, SF-36 with Frankel, and SF-36 with AIS (4 studies each). These results are expected given the frequency of use of these instruments individually.

OPEN IN VIEWER

Fig. 4

Outcome instrument frequency as a percentage of total articles (n = 58). Abbreviations: AIS, American Spinal Injury Association Impairment Scale; ODI, Oswestry Disability Index; SF-36, Short Form 36; VAS, visual analog scale.

OPEN IN VIEWER

Table 2

Outcome instruments used more than once (2004–2013)

Instrument	Uses	%
Visual analog scale	25	43.1
36-Item Short Form	20	34.5
Frankel grade scale	15	25.9
Oswestry Disability Index	12	20.7
American Spinal Injury Association Impairment Scale	9	15.5
Denis Pain Scale	5	8.6
Neck Disability Index	4	6.9
Roland Morris Disability Questionnaire	4	6.9
Hannover Functional Ability Questionnaire	2	3.4

Reporting percentages of total spinal trauma studies (n = 58).

OPEN IN VIEWER

Table 3

Spinal trauma outcome instrument usage summary

Statistic	Value
No. of uses of PROIs by category
Pain scales a	31 (3 unique scales)
Functional and disability outcome instruments b	30 (12 unique scales)
General HRQOL questionnaires c	22 (3 unique scales)
Other	13 (13 unique scales)
No. of uses of neurologic function scales d	24 (2 unique scales)
Total instrument uses	120
Studies using outcome instruments	58
Average no. of outcome instruments per study	2.1 (range 1–8)
No. of unique named instruments	26
No. of improvised questionnaires	7

Abbreviation: HRQOL, Health Related Quality of Life; PROI, patient-reported outcome instrument.

a

Includes visual analog scale, Denis Pain Scale, and McGill Pain Questionnaire.

b

Includes Cervical Spine Outcomes Questionnaire, Functional Independence Measure, Hannover Functional Ability Questionnaire, Gibbons Nerve Functional Outcome Criteria, Majeed Functional Outcome Criteria, North American Spine Society Cervical Spine Outcome Instrument, Neck Disability Index, Neck Pain and Disability Scale, Nurick Score, Oswestry Disability Index, Roland Morris Disability Questionnaire, and Smiley-Webster Scale.

c

Includes Short Form 36, EQ-5D, Every Day Life Questionnaire.

d

Includes Frankel grade scale and American Spinal Injury Association Impairment Scale.

A subanalysis comparing studies that focused on injuries of the thoracolumbar spine (45 studies) and cervical spine (13 studies) showed far less use of VAS among cervical studies (Fig. 5). ODI, which measures low back pain, was appropriately seen only in thoracolumbar studies.

OPEN IN VIEWER

Fig. 5

Comparison of frequency of use of common patient-reported outcome instruments by region of injury (cervical versus thoracolumbar). Abbreviations: AIS, American Spinal Injury Association Impairment Scale; ODI, Oswestry Disability Index; SF-36, Short Form 36; VAS, visual analog scale.

Level 4 evidence was most common among the articles (22 articles, 37.9%; Fig. 6). LOE 1 was least common, with 8 (10.3%) LOE 1 articles published in the past 10 years. ESJ published the most studies using outcome instruments and had the highest-quality evidence as determined by the greatest number of LOE 1 and 2 studies (Fig. 7).

OPEN IN VIEWER

Fig. 6

Level of evidence (LOE) of studies using outcome instruments as a percentage of total articles (n = 58).

OPEN IN VIEWER

Fig. 7

Level of evidence (LOE) of studies using outcome instruments categorized by journal. Abbreviations: ESJ, European Spine Journal; JBJS, Journal of Bone and Joint Surgery.

Discussion

The growing demands for evidence-based medical care have driven increased use of PROIs in spinal trauma surgery research. For the results of these studies to be comparable and useful, standards of PROI use are important. However, at the current juncture, no such standards exist, nor has a PROI been developed that addresses the specific needs of spinal trauma surgery research. Our review represents the first study, to our knowledge, that identifies the top outcome instruments in spinal trauma surgery, their relative frequency of use, and the trends in their use. These statistics will be useful for investigators selecting among PROIs for a study, for guideline and consensus builders in need of a starting point for drafting standards of PROI use, and for the spinal trauma research community at large to take a closer look at the validity and reliability of PROIs currently relied upon for proving the effectiveness of new or existing treatments.

Our review of five top orthopedic journals found 58 studies of spinal trauma surgery that used PROIs. Thirty-three unique PROI were used and an average of 2.1 PROIs were used per study, which reveals a lack of standardization in PROI selection. In order of frequency of use, the top five instruments were VAS, SF-36, Frankel grade scale, ODI, and AIS. The VAS was the preferred pain scale and PROI overall. There was no consensus regarding a preferred functional/disability outcome instrument as 12 different PROIs were used to measure functional/disability outcomes in the 58 studies.

A review of the methods and results sections of the studies that used the top instruments revealed a variety of errors and inconsistencies of score reporting. The failure to report numerical data (i.e., reporting scores only graphically or statistically), reporting only change in scores, reporting only postoperative scores, and significantly modifying standard outcome instruments prevent valid interstudy comparisons from being possible. These errors and inconsistencies of score reporting need to be recognized by authors and reviewers and corrected to ensure that studies being considered for publication will add to, rather than obfuscate, the evidence for a particular treatment.

This review is the first of its kind in the spinal trauma literature; however, Hunt and Hurwit conducted a similar study of PROIs in foot and ankle surgery research. ⁶⁴ Stadhouder et al identified 21 outcome measures commonly used in spinal trauma research and reviewed the literature for validation studies of these instruments in the spinal trauma population. ² The researchers found an overall lack of research into the applicability and validity of existing outcome measures in spinal trauma research, but, as a road map for future work, the authors highlight key HRQoL domains from the World Health Organization's comprehensive International Classification of Functioning, Disability and Health that should be measured by a new instrument. Among other studies of spinal trauma surgery, the outcomes considered most important in patients with or without spinal cord injury (SCI) are neurologic recovery, stability, return to work time, and pain. ³ , ¹⁷ , ⁶⁵ , ⁶⁶ The top instruments from the present study are reviewed next in regard to their validity and relevance to these core outcomes of interest.

Conclusion

An earlier review of PROI in spine surgery by McCormick et al proposed that well-designed studies should utilize a general HRQoL instrument, pain scale, and disease-specific PROI. ⁶⁷ Neurologic status is the most important outcome to assess in spine trauma, and in those patients who are neurologically intact long-term, back pain becomes most important. The current review of the spinal trauma literature shows a preference for the SF-36 as a general HRQoL instrument and the VAS as a pain scale, while highlighting the need for a disease-specific PROI to be developed. Until that goal is realized, the best approach to studying outcomes in a standardized fashion is a commonly used battery of PROIs that minimizes respondent and administrative burden where possible. To that end, we suggest VAS for pain, SF-36 for general HRQoL, and Frankel for neurologic status.