Consequences of Using SF-12 and RAND-12 when Examining Levels of Well-Being and Psychological Distress

Abstract

Objective: To explore the consequences of using the Short-Form Health Survey (SF-12) and the RAND-12 Health Status Inventory (RAND-12) for estimation of associations between its component scores and the levels of well-being and psychological distress.

Methods: Data from a rural community mental health survey of 5641 participants were analysed. Physical and mental health component scores of the SF-12 and RAND-12 were compared between participants with different rating on the Satisfaction with Life Scale and the Kessler-10. Descriptive graphical methods were utilized to explore the relationship between SF-12 and RAND-12 components scores against well-being and psychological distress. Proportional odds model was utilized to estimate the quantitative relationship between component scores of the SF-12 and RAND-12 against categories of well-being and psychological distress.

Results: Both SF-12 and RAND-12 component scores were generally positively associated with well-being and negatively associated with psychological distress. Median scores were similar despite scoring techniques used. However, distribution of scores differed whereby the RAND-12 yielded wider spread of scores in measures of well-being and psychological distressed when compared to the SF-12. A larger proportion of participants was classified as having moderate and severe disability under the RAND-12 compared to the SF-12.

Conclusion: It is recommended that users of the SF-12 and the RAND-12 are aware of the implications of utilizing either of the scoring techniques. Scoring techniques used should ideally be based on the theoretical basis of the study with consideration of the target population. Researchers may wish to use the SF-12 if distinct uncorrelated physical and mental constructs are required and for studies on clinical populations. In contrast, RAND-12 should be used if correlated physical and mental constructs are required and if the study is on community or general populations.

Keywords

epidemiologic measurements health status indicators mental health quality of life SF-12

Over the last decade, research into the quality of life has gained widespread interest. Some researchers focused on the objective component of quality of life, which involved the study of factors external to the individual such as income and health status [1]. Other researchers emphasized that quality of life is a subjective assessment of individual satisfaction with life [2], [3]. The latter concept is also known as ‘subjective well-being’. Health-related questionnaires have been widely utilized to examine subjective well-being [4–6].

The Short-Form Health Survey (SF-12) is one of such questionnaires [7], [8]. The SF-12 was adapted from the SF-36 developed for the Medical Outcomes Study to survey health status [9]. The SF-12 is a multipurpose short-form generic measure of health status that measures eight concepts: physical functioning, role limitations due to physical health problems, bodily pain, general health, vitality, social functioning, role limitation due to emotional problems, and mental health [8]. From these eight concepts two distinct component scores are derived: the physical component score and the mental component score [8].

The Kessler-10 questionnaire assesses global measures of non-specific psychological distress [10]. It was designed to function as a general purpose mental health survey [11]. The Kessler-10 has previously been utilized in a large number of surveys [10–13] and in the 1997 Australian Survey of Mental Health and Well-being study. Other assessments such as the Satisfaction with Life Scale (SWLS) [2] has also been widely utilized to measure quality of life [14], [15]. The SWLS is a multi-item scale that measures global life satisfaction by tapping into the individual's cognitive judgement process [15]. This judgement of how satisfied people are with their life ‘is based on a comparison with a standard that each individual sets for him or herself; it is not externally imposed’ [15].

The aforementioned questionnaires (SF-36, SF-12, Kessler-10 and SWLS) have been said to have adequate validity and reliability [3], [7], [10], [16], but there has been much debate over the method of scoring the SF-36 and its derivative, the SF-12 [6], [17], [18]. The scoring method of both the SF-36 and SF-12, physical and mental component scores were derived utilizing principal component analysis and orthogonal rotation, which produced factor score coefficients. These coefficients (some which are negatively weighted) are then used to compute the physical and mental component scores. This approach assumes that physical and mental health dimensions are uncorrelated [8], [19].

When the standard scoring method is used for the SF-36 and SF-12, isolated improvements in the SF-36 subscales, which contribute to the physical component score, produced worsening overall mental health (lower scores on the mental health component) and vice versa [6]. Treatment leading to large reduction in psychological distress and modest reduction in physical limitation would also produce a score pattern of no apparent change in the physical component summary score [6].

These findings were confirmed by another study in which participants who were taking physical health medication yielded a higher mental health component score and those on mental health medication yielded a higher physical component score [20]. Other research have also highlighted similar problems related to the scoring techniques used in the SF-36 [18], [21].

As an alternative to the SF-36 and SF-12 scoring system, the RAND-36 and RAND-12 Health Status Inventory scoring system was developed [22].

The RAND-36 is an exact replica of the SF-36 and the RAND-12 is an exact replica of the SF-12, but the physical and mental component scores are produced utilizing different scoring algorithms/techniques. The scoring system of the RAND-36 and RAND-12 uses a non-orthogonal rotation or oblique rotation that assumes correlation between the physical and mental health component scores [22]. Differences of results when utilizing the different scoring techniques have been shown in studies examining a sample with physical illnesses [17], [18] and a study examining community survey of mental health [23].

In the present paper we explore the consequences of using the SF-12 and the RAND-12 for estimation of associations between its component scores against levels of well-being and psychological distress.

Methods

Participants and procedures

A survey utilizing stratified random sampling of persons from the year 2000 State Electoral Role was undertaken by the Gippsland Research and Information Services (GRIS), a section with expertise in health surveys within Monash University Gippsland Campus. Introductory letter and questionnaire was mailed out to 23 000 residents in the Gippsland area in late July 2002. Follow-up letters were sent out in mid-August. In November 2002 5641 questionnaires were returned, yielding a response rate of 25%. Ethics approval was granted by the Monash University Standing Committee on Ethics in Research Involving Humans.

Measures

Gippsland community mental health survey

The Gippsland Community Mental Health Survey (GCMH) contains sociodemographic items and questions assessing life satisfaction, psychological distress, mental and physical disability. Explanations of research variables relevant to this study are provided here.

Satisfaction with life scale

The SWLS has seven items scored on a 7-point Likert scale (score anchors of ‘strongly disagree’ through to ‘strongly agree’), which measures global judgement of life satisfaction [2]. Final SWLS score ranged from 5 (low satisfaction) to 35 (high satisfaction) [2]. The scores are classified as follows: 5–9 = extremely dissatisfied, 10–14 = dissatisfied, 15–19 = slightly dissatisfied, 20 = the neutral point, 21–25 = slightly satisfied, 26–30 = satisfied and 31–35 = extremely satisfied [3]. In the present sample the scale had an internal reliability of α = 0.90. Nunnaly and Bernstein indicated 0.7 to be an acceptable reliability [24].

Kessler psychological distress scale

The Kessler-10 has 10 items, each scored on a 4-point Likert scale, which assess populations on psychological distress, specifically anxiety and depressive disorders [10]. To enable comparability the scale was modified to reflect the now commonly used 5-point Likert scale with anchors of ‘all of the time’, ‘most of the time’, ‘some of the time’, ‘a little of the time’ and ‘none of the time’. Score classifications developed by Clinical Research Unit for Anxiety and Depression (CRUFAD) were utilized, whereby scores under 20 indicate that a person is likely to be well, 20–24, likely to have a mild mental disorder, 25–29, a moderate mental disorder, and 30–50, a severe mental disorder [25]. This scale had an acceptable internal reliability of α = 0.90 [24] in the present sample.

Short Form Health Survey

The SF-12 contains 12 items that produce two component scores [7]: Physical Component Score (PCS) and Mental Component Score (MCS), utilized to assess physical and mental health of participants [8]. The component scores are produced via computation involving negative coefficient weights and orthogonal rotation, which assumes no correlation between the physical and the mental health component [8].

RAND-12 health status inventory

The RAND-12 contains 12 items that are identical to the SF-12; it also produces two component scores: Physical Health Component (PHC) and Mental Health Component (MHC) [22]. The component scores are constructed from an oblique factor solution, which assumes correlation between the physical and mental health component scores [22].

Score classifications developed by CRUFAD were utilized for both the SF-12 and the RAND-12. Scores above 50 indicate that a person is likely to be well, scores of 40–49, likely to have a mild disability, 30–39, a moderate disability, and scores below 30, severe disability [25].

Statistical analysis

We compared the distributions of the physical and mental component scores based on the two scoring methods (SF-12 and RAND-12) using descriptive graphical methods, which included histograms and density–distribution sunflower plots [26]. The latter are modifications of the traditional scatterplot that allow for the display of bivariate data with areas of high density. ‘Sunflowers’ represent areas of medium- or high-density regions in the graph, while ‘petals’ are line segments of equal length radiating from the central point [26]. Light sunflowers represent areas of medium density and each petal represents one observation. High-density regions are represented by dark sunflowers and each petal represents several observations.

We applied locally weighted regression techniques (locally weighted scatterplot smoothing or LOWESS) [27] to explore the general relationships of well-being and psychological distress scores against SF-12 and RAND-12 component scores. LOWESS provides a graphical representation of non-linear relationships between bivariate data utilizing an iterative process based on weighted least-square regression [27].

We then estimated the quantitative relationship between categorical outcomes of well-being and psychological distress against levels of disability based on SF-12 and RAND-12 component scores, while adjusting for potential putative confounders using a proportional odds model.

We excluded participants with missing information in any of the measures of well-being and in the potential confounding variables (age, gender, living circumstances, education and employment status). Where possible, we present point estimates and 95% confidence intervals. All statistical tests were implemented in Stata/SE 10.0 (College Station, Texas, USA). All p were two-sided and p < 0.05 was defined as statistically significant.

Results

Descriptive statistics

Descriptive characteristics of the participants are presented in Table 1. Approximately two-thirds were female, approximately half were older than 35 years and 58% were in full- or part-time employment (including students). Approximately three-quarters of the subjects expressed satisfaction with life (Table 2), and approximately 80% did not report any symptoms based on the Kessler-10.

Table 1.

Subject characteristics

	n	%
Gender
Female	3038	62.8
Male	1796	37.2

Age group (years)
18–35	720	14.9
36–50	1693	35.0
51–65	1436	29.7
≥ 66	985	20.4

Highest educational attainment
Did not complete high school	1685	34.7
Completed high school	830	17.2
Commenced but did not complete trade certificate	485	10.0
Completed trade certificate or TAFE course	1834	37.9

Employment
Full-time employment or student	1219	25.2
Part-time employment	1565	32.4
Unemployed	190	3.9
Home duties, maternity leave or carer	679	14.0
Disability pension or Workcover	256	5.3
Retired, aged pension	925	19.1

Living circumstances
Alone or with unrelated others	775	16.0
Partner and no children	1719	35.6
Partner with children	1904	39.4
Single parent	223	4.6
With parents	165	3.4
Other	48	1.0

TAFE, Technical Further Educational.

Table 2.

Subject mood and well-being characteristics

	N	%
Satisfaction with Life Scale
Extremely dissatisfied	176	3.6
Dissatisfied	397	8.2
Slightly dissatisfied	551	11.4
Neutral	155	3.2
Slightly satisfied	918	19.0
Satisfied	1878	38.8
Extremely satisfied	759	15.7

Kessler-10
No symptoms	3870	80.1
Mild symptoms	510	10.6
Moderate symptoms	275	5.7
Severe symptoms	179	3.7

SF-12 Physical
Well	2780	57.5
Mild disability	1435	29.7
Moderate disability	475	9.8
Severe disability	144	3.0

SF-12 Mental
Well	3215	66.5
Mild disability	1221	25.3
Moderate disability	352	7.3
Severe disability	46	1.0

RAND-12 Physical
Well	2634	54.5
Mild disability	1057	21.9
Moderate disability	674	13.9
Severe disability	469	9.7

RAND-12 Mental
Well	2500	51.7
Mild disability	1202	24.9
Moderate disability	719	14.9
Severe disability	413	8.5

SF-12, Short-Form Health Survey.

Differences between RAND-12 and SF-12 scoring systems

The distributions of SF-12 and RAND-12 component scores in Figure 1, illustrated a wider spread of scores and greater ‘peaked-ness’ under the RAND-12 method. Although the median scores (at approximately 50) were virtually similar regardless of scoring system, longer negative tails of the distributions based on the RAND-12 method demonstrated that a larger proportion of participants were classified as having moderate and severe disability compared to the SF-12 scoring method (Table 2).

Figure 1.

Differences in distribution between physical and mental component scores of the Short-Form Health Survey (SF-12) and RAND-12.

As expected, SF-12 component scores were generally positively associated with SWLS scores and negatively associated with Kessler-10 scores (Figures 2 and 3). The ability of sunflower plots to reduce the concealing effect of dense regions of the scatterplot is obvious. However, a more complex relationship emerges when LOWESS methods were applied to improve the signal-to-noise ratio (Figure 4).

Figure 2.

Distribution of physical and mental health scores for Short-Form Health Survey (SF-12) and RAND-12 against the Kessler-10.

Figure 3.

Distribution of physical and mental health scores for Short-Form Health Survey (SF-12) and RAND-12 against the Satisfaction with Life Scale.

Figure 4.

(a) Locally weighted scatterplot smoothing (LOWESS) curves of physical and mental health scores for Short-Form Health Survey (SF-12) and RAND-12 against Kessler-10. (b) LOWESS curves of physical and mental health scores for SF-12 and RAND-12 against the Satisfaction with Life Scale.

Smoothed curves for mental health component scores generally show parallel courses for both SF-12 and the RAND-12, producing a leftward shift in values. That is to say, for an equivalent SWLS or Kessler-10 score, the RAND-12 MHC score will be lower (higher severity), on average, compared to the SF-12 MCS. In contrast, although this is seen in the SF-12 and RAND-12 scores for the physical component, there are wide areas of overlap for scores <55. For higher scores, a reversal in trend is seen in the SF-12 PCS.

Relationships based on clinical categories

Increasing disability in physical and mental component scores is associated with symptom severity as measured by the Kessler-10 (Table 3). Categories of physical disability based on RAND-12 scores tend to be associated with larger proportional odds of greater symptom severity compared to scores derived from SF-12. In contrast, mental disability categories based on RAND-12 scores are associated with smaller proportional odds of greater symptoms severity compared to SF-12. Generally, similar results are seen for SWLS.

Table 3.

Multivariable adjusted ORs from proportional odds models

	Kessler-10	SWLS
	OR (95%CI)	OR (95%CI)
Physical
SF-12
Well	1.00	1.00
Mild	2.40 (2.01–2.85)	0.57 (0.50–0.64)
Moderate	6.06 (4.72–7.77)	0.36 (0.30–0.43)
Severe	10.68 (7.32–15.59)	0.24 (0.17–0.34)

RAND-12
Well	1.00	1.00
Mild	3.66 (2.99–4.47)	0.48 (0.42–0.54)
Moderate	7.05 (5.61–8.86)	0.34 (0.30–0.40)
Severe	18.76 (14.43–24.40)	0.20 (0.16–0.24)

Mental

SF-12
Well	1.00	1.00
Mild	20.39 (16.23–25.63)	0.19 (0.17–0.22)
Moderate	227.03 (166.17–310.18)	0.05 (0.04–0.06)
Severe	3272.60 (1369.71–7819.12)	0.02 (0.01–0.03)

RAND-12
Well	1.00	1.00
Mild	13.92 (9.26–20.91)	0.32 (0.28–0.36)
Moderate	110.88 (74.26–165.56)	0.11 (0.09–0.13)
Severe	741.80 (476.90–1153.84)	0.04 (0.04–0.05)

CI, confidence interval; OR, odds ratio; SF-12, Short-Form Health Survey; SWLS, Satisfaction with Life Scale.

Discussion

The aim of the present study was to ascertain the consequences of using the SF-12 and the RAND-12 for estimation of associations between its component scores and the levels of well-being and psychological distress. The present results indicate that both the SF-12 and RAND-12 component scores are generally positively associated with SWLS scores and negatively associated with Kessler-10 scores. But a larger proportion of participants was classified as having moderate and severe mental and physical disability under the RAND-12 compared to the SF-12 scoring method. Exceptions to this were detected for the SF-12 PCS for scores of mid-50s and above.

To our knowledge there has not been a study that specifically looks at the consequences of using the SF-12 and the RAND-12 for estimation of associations between the component scores and the levels of well-being and psychological distress. The present study had a response rate of 25%, which is not uncommon for mail-out surveys. Due to the high non-response rate, there may be a non-response bias, whereby people with low levels of life satisfaction, physical disability, mental disability and high levels of psychological distress may have chosen not to participate. Despite the 25% response rate the present study still had a large sample size of 5641 participants. All scales utilized have previously been validated and have adequate psychometric properties.

The present results, illustrated in Figures 1–3, demonstrate that the utilization of the RAND-12 compared to the SF-12 allowed for more variation in participant scores in both the SWLS and the Kessler-10. That is to say, there may be potential loss of information in the SWLS and the Kessler-10 when utilizing the SF-12, rather than the RAND-12.

The impact of the negatively weighted SF-12 items is illustrated in Figure 4(a). It is most evident in the SF-12 PCS, where the LOWESS line peaked steeply past the score of mid-50s. This sudden change in trend in the SF-12 PCS creates difficulty when interpreting the SF-12 PCS, because it is unclear whether the high SF-12 PCS reflects good physical health or high psychological distress and lower levels of well-being. That is to say, participant in studies identifying as having no physical disability or who scored >55 points on the SF-12 PCS might inaccurately be identifying as having low levels of well-being, higher levels of psychological distress or higher levels of mental health disability.

The differences between the SF-12 and RAND-12 noted here have also been replicated in previous studies [6], [21], [23]. These differences illustrate the implication of applying factor scores based on orthogonal rotation, which assumes no association, to constructs that have actual association [23]. The artificially imposed orthogonal rotation to the physical and mental component of the SF-12 ignores the natural nature of the correlation between physical and mental health, which has been documented in previous studies [6], [21], [23]. The reciprocal effects and negative weights utilized to construct the physical and mental health component of the SF-12 are likely to produce inaccurate results [21] because presence of symptoms in one domain is likely to inflate the scores in another domain.

When examining the clinical categorical cut-off score of the physical disability in the present population sample, the RAND-12 scores tended to be associated with larger proportional odds of lower level of well-being and greater psychological distress compared to scores derived from SF-12. Based on Table 3, the trend scores in the present study for both the SF-12 and RAND-12 are similar and were all statistically significant. The differences in magnitude in the proportional odds are large at the extreme end. Despite the peak in the SF-12 PCS of mid-50s and above, the overall SF-12 PCS still had the same trend as the RAND-12 PHC. That is to say, in the population we sampled, the inferences obtained from the SF-12 and the RAND-12 are similar. In clinical populations it would appear that there are no differences between using the SF-12 and the RAND-12 because in a clinical setting the focus is on patients identifying with severe symptoms, and there were no evident anomalies for those with component scores <50. However, if the SF-12 were to be utilized for a community or population survey with the aim of capturing data on persons with mild and moderate symptoms in order to promote mass behavioural change [28], the differences in magnitude in the proportional odds at the extreme end would be highly notable. The steep dip in levels of well-being of those with no physical disability indicated in the SF-12 PCS will provide an inaccurate population curve. In light of Rose's perspective that the study of a population as a whole attempts to ‘lower the mean level of risk factor’ [28] and remove the underlying causes, an accurate population curve is imperative before strategies can even be introduced to the population to address the issue.

Conclusion

We have discussed two scoring techniques to an instrument that may be utilized to measure health-related quality of life. We explored the consequences of using the SF-12 and the RAND-12 for estimation of associations between component scores and the levels of well-being and psychological distress. Differing outcomes were obtained based on the scoring technique utilized. Negative weights utilized to construct the physical and mental health component of the SF-12 are likely to produce inaccurate results, especially for those who scored above mid-50s in the physical component score.

Given the common use of the SF-12, we recommend that users familiarize themselves with the implications associated with the different scoring techniques. Users of the SF-12 are encouraged to identify the methodology and targeted population of their study prior to deciding on the scoring technique to utilize.

Footnotes

Acknowledgements

This research was funded by Beyond Blue. Researchers retained independent control of all aspects of the study and its submission for publication.

References

1. Smith

Langa

Kabeto

Ubel

. Health, wealth, and happiness: financial resources buffer subjective well-being after the onset of a disability. Psychol Sci 2005; 16: 663–666.

2. Diener

Emmons

Larsen

Griffin

. The Satisfaction with Life Scale. J Pers Assess 1985; 49: 71–75.

3. Pavot

Diener

. Review of the Satisfaction with Life Scale. Psychol Assess 1993; 5: 164–172.

4. Bosch

Tybout

van Spronsen

de Valk

Wijburg

Grootenhuis

. The course of life and quality of life of early and continuously treated Dutch patients with phenylketonuria. J Inherit Metab Dis 2007; 30: 29–34.

5. Painter

Luetkemeier

Moore

. Health-related fitness and quality of life in organ transplant recipients. Transplantation 1997; 64: 1795–1800.

6. Simon

Revicki

Grothaus

LMA

Vonkorff

. SF-36 summary scores: are physical and mental health truly distinct. Med Care 1998; 36: 567–572.

7. Ware

JEJ

Kosinski

Keller

. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996; 34: 220–233.

8. Ware

JEJ

Kosinski

Keller

. SF-12; How to score the SF-12 Physical and Mental Health Summary Scales. QualityMetric Incorporated, Lincoln, RI 1998.

9. Ware

Jr Sherbourne

. The MOS 36-Item Short-Form Health Survey (SF-36): I. conceptual framework and item selection. Med Care 1992; 30: 473–483.

10.

10. Kessler

Morczek

. Final version of our Non-Specific Psychological Distress Scale. Survey Research Centre Institute for Social Research University of Michigan, Ann Arbor, MI 1994.

11.

11. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–976.

12.

12. Andrews

Slade

. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health 2001; 25: 494–497.

13.

13. Furukawa

Kessler

Slade

Andrews

. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychol Med 2003; 33: 357–362.

14.

14. Arrindell

Heesink

Feij

. The Satisfaction With Life Scale (SWLS): appraisal with 1700 healthy young adults in The Netherlands. Pers Individ Differ 1999; 26: 815–826.

15.

15. Hart

Hanks

Bogner

Millis

Esselman

. Blame attribution in intentional and unintentional traumatic brain injury: longitudinal changes and impact on subjective well-being. Rehabil Psychol 2007; 52: 152–161.

16.

16. Jenkinson

Wright

Coulter

. Criterion validity and reliability of the SF-36 in a population sample. Qual Life Res 1994; 3: 7–12.

17.

17. Johnson

Maddigan

. Performance of the RAND-12 and SF-12 summary scores in type 2 diabetes. Qual Life Res 2004; 13: 449–456.

18.

18. Nortvedt

Riise

Myhr

K-M

Nyland

. Performance of the SF-36, SF-12, and RAND-36 summary scales in a multiple sclerosis population. Med Care 2000; 38: 1022–1028.

19.

19. Ware

Kosinski

Bayliss

. Comparison of methods for the scoring and statistical analysis of SF-36 health profile and summary measures: summary of results from the Medical Outcomes Study. Med Care 1995; 33(Suppl 4)AS279–AS264.

20.

20. Wilson

Parsons

Tucker

. The SF-36 summary scales: problems and solutions. Soc Prev Med 2000; 45: 239–246.

21.

21. Taft

Karlsson

Sullivan

. Do SF-36 summary component scores accurately summarize subscale scores?. Qual Life Res 2001; 10: 395–404.

22.

22. Hays

. RAND-36 Health Status Inventory. The Psychological Corporation, San Antonio, TX 1998.

23.

23. Windsor

Rodgers

Butterworth

Anstey

Jorm

. Measuring physical and mental health using the SF-12: implications for community surveys of mental health. Aust N Z J Psychiatry 2006; 40: 797–803.

24.

24. Nunnaly

Bernstein

. Psychometric theory. McGraw-Hill, New York 1994.

25.

25. Andrews

. Outcome measures suitable for patients with mental disorders seen in general practice. Sydney: CRUFAD, 2003. [cited 1 October 2007.] Available from URL: http://www.crufad.com/phc/outcome.htm .

26.

26. Dupont

Plummer

. Density distribution sunflower plots. J Stat Soft 2003; 8: 1–5.

27.

27. Cleveland

. Robust locally weighted regression and smoothing scatterplots. J Am Stat Assoc 1979; 74: 829–836.

28.

28. Rose

. Sick individuals and sick populations. Int J Epidemiol Assoc 1985; 30: 427–432.