The Glasgow Outcome at Discharge Scale: An Inpatient Assessment of Disability after Brain Injury

Abstract

This study assesses the validity and reliability of the Glasgow Outcome at Discharge Scale (GODS), which is a tool that is designed to assess disability after brain injury in an inpatient setting. It is derived from the Glasgow Outcome Scale-Extended (GOS-E), which assesses disability in the community after brain injury. Inter-rater reliability on the GODS is high (quadratic-weighted kappa 0.982; 95% confidence interval [CI] 0.968, 0.996) as is concurrent validity with the Disability Rating Scale (DRS) (Spearman correlation −0.728; 95% CI −0.819, −0.601). The GODS is significantly associated with physical and fatigue subscales of the short form (SF)-36 in hospital. In terms of predictive validity the GODS is highly associated with the GOS-E after discharge (Spearman correlation 0.512; 95% CI 0.281, 0.687), with the DRS, and with physical, fatigue, and social subscales of the SF-36. The GODS is recommended as an assessment tool for disability after brain injury pre-discharge and can be used in conjunction with the GOS-E to monitor disability between hospital and the community.

Introduction

B rain-injured patients are cared for in a range of acute hospital wards, including orthopedic and general medicine, where there is limited training and expertise in the sequelae of brain injury. Most are discharged when physically able, without assessment for persisting cognitive and emotional problems and without assessment of rehabilitation or care needs or appropriate referral.^1,2 An assessment of disability in inpatient settings with a simple and robust measure usable by non-specialist ward staff would be useful in addressing these issues in several ways. For example, such a measure might inform about disability when decisions about the appropriateness of hospital transfer or discharge are being considered. If categorizing disability during prolonged hospital stays, it may indicate the need for specialist assessment or referral to allied health professionals or clinical neuropsychology or the need for a social work assessment before discharge.

An association with function in the community would also make it potentially useful as a pre-discharge predictor of later needs and, hence, of use in discharge decision making, discharge planning (including identifying a need for rehabilitation and care), and in clinical audit. If coherently linked to an outpatient measure, an inpatient measure might also be used to assess change in disability across inpatient and community settings, including before and after interventions. In research in acute care, it may form a baseline to monitor “natural” outcome or outcome in response to treatment or act as a “surrogate” early end-point that may, to an extent, compensate for loss to follow-up.

The Glasgow Outcome Scale (GOS)³ is the most widely cited assessment of outcome in the community after brain injury.⁴ The GOS and its more sensitive extended form, the Glasgow Outcome Scale-Extended (GOS-E)^5,6 can be administered in busy clinical settings by professionals from a range of backgrounds and does not require training in neurology or neuropsychology. Both the GOS and GOS-E are simple and quick to administer and are valid and reliable when used in face to face interview, telephone interview, or in amended form by postal return.^6

–10 The GOS is not valid in a hospital setting, however, because a key criterion is restitution of independence in society. Despite this, the GOS continues to be used inappropriately in studies on hospitalized patients, emphasizing a demand for an inpatient version.

Although a variety of scales have been developed to assess the recovery and progress of inpatients with brain injury, attention has largely focused on neurorehabilitation where the scales commonly require training and are time consuming and less reliable than the GOS.^11,12 Few scales assess outcome across acute hospital and community settings. The Disability Rating Scale (DRS) was developed to assess change during rehabilitation, but it has been reported to be less sensitive than the GOS-E.¹³

Hence, we have devised an inpatient version of the GOS-E—the Glasgow Outcome at Discharge Scale (GODS)—to enable assessment of disability after brain injury in an inpatient setting including by general ward staff who are instructed in its use, but need not be specialists in brain injury or rehabilitation. We report data on validity and reliability.

Methods

Adult patients admitted to a Glasgow hospital for 24 h or more after a brain injury were assessed near to discharge. Written consent was obtained near to discharge to home when the hospital had decided that the patient had mental capacity and was fit for discharge. Ethics approval was obtained from the West of Scotland Research Ethics Committee. To be included, potential participants had to be age 16 years or more, to be fluent in English, and to have been admitted to hospital overnight or longer because of a brain injury. The pre-discharge assessment involved a short interview by the research worker with nursing staff that knew the patient well (and who would have a key role in completing the GODS should it be adopted in clinical practice) in addition to interviewing the patient. At the pre-discharge assessment, in addition to the GODS and the DRS,¹⁴ indicators of severity of brain injury were obtained. These were duration of loss of consciousness and minimum Glasgow Coma Scale score during hospital admission.

The follow-up appointment was arranged at the time of the inpatient assessment and planned within a few weeks of discharge. The post-discharge assessment involved the patient and a significant other (as is the norm for the GOS-E). The post-discharge assessment included the GOS-E and DRS as measures of general outcome. As an independent, subjective indicator of health outcome, the 36-item Short Form Health Survey (SF-36)¹⁵ was administered pre- and post-discharge. Concurrent and convergent validity was assessed by comparing GODS scores with the DRS and SF-36, respectively, in the inpatient setting. Predictive validity was assessed by comparing the GODS with outpatient administration of the GOS-E, the DRS, and the SF-36. A checklist regarding rehabilitation provision and attendance was administered at the community follow-up assessment, because this could be a confounding factor that affects outcome between discharge and follow-up.

Given that brain injury is often associated with excess habitual alcohol drinking and that an early return to drinking might have a rapid and negative effect on outcome, the Alcohol Use Disorders IdentificationTest (AUDIT)¹⁶ was administered at post-discharge assessment. Inter-rater reliability was assessed using the GODS on two separate occasions by two members of our research team within a 24-h period after the patient was deemed fit for discharge.

Sample size and statistics

The GOS-E has inter-rater reliability between telephone interview and in-person assessment of 0.84 (95% confidence interval [CI] 0.58 to 1.00) as measured by the quadratic-weighted kappa statistic.¹⁰ Cohen¹⁷ proved that for a general tabulation of the results from two raters on a scale with several categories, the quadratic-weighted kappa is exactly equivalent to the standard product-moment correlation coefficient if the marginal distributions are identical. Table 2 in the article by Pettigrew and associates¹⁰ shows that in the context of the GOS-E, the marginal distributions from each assessment are indeed very similar. We therefore base the sample size calculation for quadratic-weighted kappa on that for the product-moment correlation coefficient r. Algina and Olejnik¹⁸ provide tables for the product-moment correlation coefficient in terms of the sample size needed to achieve a given width of CI for a range of possible true values of r. Hence, we estimated that about 60 patients were needed for the reliability study and, from previous studies, approximately 80 to assess validity.^6

–10

Inter-rater reliability on the GODS was assessed using the quadratic-weighted kappa and 95% CI. Concurrent and convergent validity (association between GODS and each of inpatient DRS and SF-36, respectively) was quantified by the Spearman rank correlation coefficient and 95% CI. Predictive validity (association between GODS pre-discharge and GOS-E shortly after hospital discharge) was measured using the Spearman rank correlation coefficient and 95% CI. Predictive validity for moderate or severe disability on GOS-E was assessed using sensitivity, specificity, positive predictive value and negative predictive value with exact binomial CIs. The characteristics of those who were and were not successfully followed up were compared for categorical variables using the chi-square test and for continuous variables by two sample t test or Wilcoxon-Mann-Whitney test as appropriate.

Results

Pilot study

Ten health professionals (medicine; surgery; nursing; occupational therapy) commented on the construct validity of the GODS. All had some clinical experience with brain injury, and eight did not specialize in brain injury (to reflect admission in Scotland—usually to a medical or a surgical ward and not to a specialist brain injury team). The overall view was that the GODS is a useful tool that seems straightforward to use and that would predict outcome in the community. On the basis of comments, minor changes to the GODS were made. Where possible, the wording of the GODS was kept the same as or consistent with that of the GOS-E. The GODS is available free to download (http://www.gla.ac.uk/researchinstitutes/healthwellbeing/staff/tommcmillan).

Characteristics of the sample

Recruitment approximated to the estimated sample size, with 77 persons being recruited to the study and inter-rater reliability assessed in 56. Follow-up after discharge was achieved in 53/77 (69%). Those followed up (F) and lost to follow-up (NF) did not differ significantly in terms of age (two sample t test, p=0.66; mean 50 (F) versus mean 52 (NF) 95% CI for difference −8.0, 12.6), male sex (64% F, 71% NF: p=0.57, odds ratio 0.74, 95% CI 0.26–2.09), or GODS (F: median 5; NF: median 4.5; Wilcoxon Mann Whitney test=0.20) or DRS scores (F: median 3; NF: median 3; Wilcoxon Mann Whitney test 0.48).

Initial characteristics are presented in Table 1. On the GODS, 41.6% had severe disability, 29.9% moderate disability, and 28.6% good recovery. At follow-up on the GOSE, 30.2% were severely disabled (SD), 39.6% moderately disabled (MD), 30.2% had good recovery (GR), with data missing in 31.2% who were not available at follow-up (Table 2). GODS scores were not associated with age (Spearman r −0.069; 95% CI −0.288, 0.158), p=0.55). There was a slight trend toward greater disability in females (median upper SD) than males (median upper MD; Wilcoxon Mann Whitney, p=0.059).

Table 1.

Personal and Injury Characteristics and Glasgow Outcome at Discharge Scale Outcome ( n =77)

Characteristic
Age in years (mean (SD); range)	50.59 (21.01); 17–94
Sex (male/female)	51 (66.2%)/26 (33.8%)
Disability before admission	3 (3.9%)
GCS in hospital
Mild (13–15)	59 (76.6%)
Moderate (9–12)	4 (5.2%)
Severe (3–8)	10 (13.0%)
Unknown	4 (5.2%)
Loss of consciousness:	31 (40.3%)
None/minimal
1 to 30 min	27 (35.1%)
>30 mins	14 (18.2%)
Unknown	5 (6.5%)
Glasgow Outcome at Discharge Scale
Upper good recovery	8 (10.4%)
Lower good recovery	14 (18.2%)
Upper moderate disability	13 (16.9%)
Lower moderate disability	10 (13.0%)
Upper severe disability	19 (24.7%)
Lower severe disability	13 (16.9%)

SD, standard deviation; GCS, Glasgow Coma Scale.

Table 2.

Spearman Correlations between the Glasgow Outcome at Discharge Scale and Measures of Disability and Function in Hospital

	Correlation; p value	95% CI
Disability Rating Scale	−0.728; p<0.0001	−0.819, −0.601
SF-36 Physical function	0.459; p<0.0001	0.258, 0.623
SF-36 Role limitations physical	0.227; p=0.016	0.051, 0.475
SF-36 Emotional well-being	0.003; p=0.98	−0.227, 0.233
SF-36 Role limitations emotional	0.177; p=0.13	−0.053, 0.390
SF-36 Energy/fatigue	0.230; p=0.049	−0.001, 0.437
SF-36 Social function	0.220; p=0.058;	−0.009, 0.427
SF-36 Pain	0.116; p=0.32	−0.116, 0.335
SF-36 General health	−0.028; p=0.81	−0.257, 0.203

CI, confidence interval; SF, short form.

Inter-rater reliability

The quadratic weighted kappa on the GODS between two observers (a research nurse and a research psychologist) was 0.982 (95% CI: 0.968, 0.996; n=56). The full range of GODS scores from lower SD to upper GR were represented, with perfect agreement in 22 SD, 11 MD, and 17 GR cases and disagreement in 3 SD and in 3 MD cases, all by one point.

Validity

In terms of concurrent validity, the GODS was highly correlated with the in-hospital DRS, and in terms of convergent validity, the GODS significantly correlated with the in-hospital SF-36 factors physical function, role limitations because of physical health, energy/fatigue, and had a borderline significant association with social functioning (Table 2).

In terms of predictive validity, a moderately strong and highly significant relationship was found between GODS scores in hospital and GOSE scores at follow-up (Spearman r=0.512, p<0.0001; CI 0.281, 0.687). If intending to operate a simple clinical decision making rule using the GODS, (such as those with MD or SD at discharge need intervention or assessment for intervention and those with good recovery need a brief follow-up such as by telephone or letter),^19,20 the GODS correctly predicts disability (GOS-E) at follow-up with a sensitivity of 89% (95% CI, 75% to 97%), specificity of 75% (95% CI, 48% to 93%), with a positive predictive value of 89% (95% CI, 75% to 97%) and negative predictive value of 75% (95% CI, 48% to 93%).

It did not predict disability at follow-up in four (7%), and four others (7%) had false positives results (i.e., predicts disability but had good recovery). Of the four patients who deteriorated from GR to disabled (Table 3), one had a heart attack after discharge, severe anxiety in relation to symptom persistence developed in one, and two deteriorated and needed further CT brain investigation. In terms of the four who improved from disabled to GR, the context was resolution of symptoms of vomiting in one case and resolution of impaired balance in three.

Table 3.

Glasgow Outcome at Discharge Scale and Glasgow Outcome Scale-Extended Scores ( n =53)

	GOS-E Lower SD	Upper SD	Lower MD	Upper MD	Lower GR	Upper GR
GODS Lower SD	2	2		1		1
Upper SD	1	6	4	1	1	1
Lower MD			5	2
Upper MD	1	1	3	4	1
Lower GR	1	1	1		5	2
Upper GR		1				5

GOS-E, Glasgow Outcome Scale-Extended; GODS, Glasgow Outcome at Discharge Scale; SD, severe disability; MD, moderate disability; GR, good recovery.

The GODS correlated highly with the post-discharge DRS and more modestly with the SF-36 factors physical function and role limitations because of physical health and social functioning (Table 4).

Table 4.

Spearman Correlations between the Glasgow Outcome at Discharge Scale and Measures of Disability Post-Discharge

	Correlation; p value	95% CI
Disability Rating Scale	−0.476; p=0.00025	−0.663, −0.234
SF-36 Physical function	0.426; p=0.0015	0.170, 0.628
SF-36 Role limitations physical	0.400; p=0.0031	0.139, 0.608
SF-36 Emotional well-being	0.002; p=0.99	−0.274, 0.277
SF-36 Role limitations emotional	0.094; p=0.51	−0.187, 0.360
SF-36 Energy/fatigue	0.150; p=0.29	−0.131, 0.409
SF-36 Social function	0.336; p=0.014	0.067, 0.560
SF-36 Pain	0.202; p=0.32	−0.078, 0.452
SF-36 General health	0.035; p=0.80	−0.243, 0.308

SF, short form; CI, confidence interval.

Outcome at Follow-up

The earliest follow-up was 12 days after discharge, and the median time to follow-up was 22 days (interquartile range 19 days). Brain injury was subjectively viewed by participants as the most important factor affecting outcome at follow-up in 43/53 (81%). The remaining 10 participants (19%) thought that other injury or illness best explained their outcome. Scores on the AUDIT were not associated with differences in scores between GODS and GOS-E (Spearman r −0.049; 95% CI −0.317, 0.227, p=0.73) or GOS- E scores at follow-up (Spearman r 0.001; 95% CI −0.272, 0.274, p=0.99). At follow-up, 13 (25%) had seen their general practitioner about the brain injury. Eleven had been admitted to specialist inpatient neurorehabilitation pre-discharge (9/36 of those who were disabled at discharge) and an additional four to outpatient therapy services post-discharge. In those who were disabled on the GODS, there was no significant difference in change scores (GODS minus GOS-E) between those who received specialist rehabilitation and those who did not (Mann Whitney test, p=0.97).

Discussion

We were able, with input from expert colleagues and from potential inexpert users to adapt the GOS-E for use with brain injury patients before discharge from hospital care. The results of studies reported here demonstrate the merits of high validity and high inter-rater reliability. These initial findings suggest that the GODS will be useful in research or clinical contexts. On the basis of the pilot study with ward staff and from our other studies involving acute wards, our perception is that the GODS is more likely to achieve routine use by ward staff than is, for example, a measure of post-traumatic amnesia that might involve assessment of attention and memory. There are three reasons for this. First, in our experience, ward staff are not keen to perform cognitive assessment routinely, especially if test equipment has to be used. Second, the GODS is relevant to all persons admitted with a brain injury and not just those where post-traumatic amnesia is an issue. Finally, the GODS focuses on disability rather than impairment and, given this, the GODS has direct, practical implications that may reveal potential problems to ward staff and help in decision making and plans for discharge.

The study has several strengths. The patients were sampled from across the spectrum of severities of brain injury admitted to hospital; within this, the large proportion with a mild brain injury is consistent with the demographics of hospitalized brain injury,¹ and with the potential use of the GODS in the non-specialist wards where these patients tend to be admitted. Similarly, patients covered the range of categories in the GODS from upper GR to lower SD. The loss of patients to follow-up limited the analysis of relationships between the GODS and the GOS-E, but is typical of these patients.² The follow-up rate of almost 70% is high for studies of this kind and, of note, is the absence of any systematic loss at follow-up in relation to age, sex, GODS, or DRS scores.

The GODS has a very high inter-rater reliability, using raters from different professional backgrounds, and this attests to its potential in non-specialist settings. The GODS has a high concurrent validity with the DRS, which is a well-established measure of disability. Convergent validity was demonstrated with the SF-36 in terms of physical but not emotional concerns during hospital stay. This may reflect a focus on physical complaints and fatigue early after injury in a general hospital setting. This may be because of the evident nature of the physical symptoms in hospital whereas emotional problems or demands associated with cognitive functions such as memory or problem solving or associated with daily function are limited for the patient and can be less obvious in a ward environment, which by its nature is essentially supportive and non-challenging.

The GODS predicts outcome on the GOS-E and the DRS soon after discharge. Predictive accuracy on the GOS-E is particularly high if considering outcome in terms of GOS-E categories of disability (SD or MD) or GR. These categories are potentially of practical use in relation to discharge planning,¹⁸ and in the simple and practical terms of deciding about the need for brief follow-up (GR) or for planning a more detailed assessment of neurorehabilitation (disabled). The degree of association between the GODS and the GOS-E was reduced when there was early recovery in the community; for example, from complaints that were largely physical such as impaired balance and also by deterioration in a few patients.

Nevertheless, the sensitivity of the GODS in terms of simple clinical decision making was high and the specificity adequate. Outcome prediction was not confounded by alcohol drinking on return to the community in this sample. As in hospital, the associations between the GODS and the SF-36 at early follow-up in the community were dominated by a focus on physical symptoms, and the trend toward a significant association with social function in hospital became significant in the community. Specialist provision of support or rehabilitation was infrequent in the sample, and there was no indication that this was a confounding factor.

The study is limited by use of research staff rather than ward staff in assessing the reliability and validity of the GODS and by the limited data pertaining to the clinical utility of the GODS in facilitating decision making with regard to planning of discharge or hospital transfer to rehabilitation. In terms of further work, the validation of the GODS would benefit from the development of training materials with online accessibility and validation of its use by ward staff trained using these materials. Further development of the clinical use of the GODS as a guide to discharge planning and independent replication of this validity study are also important. Future work might consider the relationships between GODS score and later outcome after discharge. If ability of the GODS to predict outcome at later time points is of interest (e.g., 6 or 12 months), additional factors that might influence outcome should be taken into account, such as provision of rehabilitation, social support, and any further health difficulties.

Conclusion

We conclude that the GODS is a valid and reliable tool for use in acute hospital wards to assesses disability after brin injury. It has a good association with the community version, the GOS-E, and has potential uses to audit early outcome in patients in hospital, to enhance decisions about discharge planning, to evaluate the outcome of interventions in inpatient and community settings, and to support longitudinal assessment of change in disability across inpatient and community settings.

Footnotes

Acknowledgments

This study was funded by the Chief Scientist Office for Scotland (CZH/4/658). Thanks are due to Dr. Maria Gardani for reliability assessments. Thanks are also due to Prof. Sir Graham Teasdale for comments on an earlier draft of the article.

Author Disclosure Statement

No competing financial interests exist.

References

Thornhill

, Teasdale

G.M.

, Murray

G.D.

, McEwen

, Roy

C.W.

, Penny

K.I.

2000. Disability in young people and adults one year after head injury: prospective cohort study. BMJ, 320:1631–1635.

Whitnall

, McMillan

T.M.

, Murray

, Teasdale

GM.

2006. Disability in young people with head injury: 5-7 year follow-up of a prospective cohort study. J. Neurol. Neurosurg. Psychiatry, 77:640–645.

Jennett

, Bond

M.R.

1975. Assessment of outcome after severe brain damage. A practical scale. Lancet, 305:480–484.

King

J.T.

, Carlier

P.M.

, Marion

D.W.

2005. Early Glasgow Outcome Scale scores predict long-term functional outcome in patients with severe traumatic brain injury. J. Neurotrauma, 22:947–954.

Jennett

, Snoek

, Bond

M.R.

, Brooks

1981. Disability after severe head injury: observations on the use of the Glasgow Outcome Scale. J. Neurol. Neurosurg. Psychiatry, 44:285–293.

Wilson

J.T.

, Pettigrew

E.L.

, Teasdale

G.M.

1998. Structured interviews for the Glasgow Outcome Scale and the Extended Glasgow Outcome Scale: Guidelines for their use. J. Neurotrauma, 15:573–585.

Teasdale

G.M.

, Pettigrew

L.E.

, Wilson

J.T.

, Murray

, Jennett

1998. Analyzing outcome of treatment of severe head injury: a review and update on advancing the use of the Glasgow Outcome Scale. J. Neurotrauma, 15:587–597.

Wilson

J.T.

, Edwards

, Fiddes

, Stewart

, Teasdale

G.M.

2002. Reliability of postal questionnaires for the Glasgow Outcome Scale. J. Neurotrauma, 19:999–1005.

Levin

H.S.

, Boake

, Song

, Mccauley

, Contant

, Diaz-Marchan

, Brundage

, Goodman

, Kottrla

K.J.

2001. Validity and sensitivity to change of the extended Glasgow Outcome Scale in mild to moderate traumatic brain injury. J. Neurotrauma, 18:575–584.

10.

Pettigrew

L.E.

, Wilson

J.T.

, Teasdale

G.M.

2003. Reliability of ratings on the Glasgow Outcome Scales from in-person and telephone structured interviews. J. Head Trauma Rehabil., 18:252–258.

11.

Hammond

, Grattan

, Sasser

, Corrigan

, Bushnik

, Zafonte

. 2001. Long-term recovery course after traumatic brain injury: A comparison of the functional independence measure and disability rating scale. J. Head Trauma Rehabil., 16:318–329.

12.

Skinner

, Turner-Stokes

2006. The use of standardized outcome measures in rehabilitation centres in the UK. Clin. Rehabil., 20:609–615.

13.

Choi

S.C.

, Clifton

G.L.

, Marmarou

, Miller

E.R.

2002. Misclassification and treatment effect on primary outcome measures in clinical trials of severe neurotrauma. J. Neurotrauma, 19:17–22.

14.

Rappaport

, Hall

K.M.

, Hopkins

, Belleza

, Cope

D.N.

1982. Disability rating scale for severe head trauma patients: coma to community. Arch. Phys. Med. Rehabil., 63:118–123.

15.

Ware

J.E.

Jr. , Sherbourne

C.D.

The MOS 36-item short-form health survey (SF-36): I. Conceptual framework, item selection. 1992. Med Care, 30:473–483.

16.

Babor

, Ramon de la Fuente

, Saunders

. 1992. AUDIT: The Alcohol Use Disorders Test: Guidelines for Use in Primary Care. Geneva: World Health Organisation.

17.

Cohen

1968. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull., 70:213–220.

18.

Algina

, Olejnik

2003. Sample size tables for correlation analysis with applications in partial correlation and multiple regression analysis. Multivariate Behav. Res., 38:309–323.

19.

Turner-Stokes

, Disler

P.B.

, Nair

, Wade

D.T.

2005. Multi-disciplinary rehabilitation for acquired brain injury in adults of working age. Cochrane Database of Syst. Rev., 3:CD004170.

20.

Scottish Intercollegiate Guidelines Network (SIGN). 2009. Early Management of Patients with a Head Injury. A National Clinical Guideline. http://guideline.gov/content.aspx?id=15207. 2013 My 16 .