Reliability,Validity,and Responsiveness of the Impact of Weight on Self-Perceptions Questionnaire (IW-SP) in Individuals with Type 2 Diabetes and Obesity

Abstract

Background:

The Impact of Weight on Self-Perceptions Questionnaire (IW-SP) assesses an individual's self-perception related to his or her weight. The primary objective of this study was to provide evidence of the reliability, validity, and responsiveness of the IW-SP.

Materials and Methods:

Study participants were individuals with type 2 diabetes mellitus (T2DM) and obesity enrolled in clinical weight-loss programs in the United States. Data were obtained for clinical measures, IW-SP, and other patient-reported outcome measures. An intraclass correlation coefficient (ICC) and Cronbach's α were calculated for test–retest reliability and internal consistency, respectively. For validity, correlations and t tests were performed. For responsiveness, baseline and 6-month data for a subgroup of patients were analyzed using the paired t test and calculation of effect size (ES).

Results:

Reliability data for 106 study participants (mean age, 52 years; 69% female; 31% white; mean body mass index, 38 kg/m²) yielded an ICC of 0.85 and Cronbach's α values of >0.89. IW-SP scores were associated with obesity-related quality of life, mental health, and vitality (r>0.50, P<0.001). In the subgroup (n=40) used to estimate responsiveness, weight was significantly less at end point than at baseline (mean, baseline=231.9 vs. end point=222.0 pounds; P<0.001; ES=0.23), and IW-SP scores were significantly better than at baseline (mean, baseline=61.0 vs. end point=72.1 [on a scale of 0–100]; P=0.01; ES=0.34). Mean IW-SP change scores significantly discriminated between those achieving >5% body weight loss and those who achieved <5% (mean change, 23.6 vs. 5.7; P=0.03).

Conclusions:

The IW-SP has demonstrated reliability, validity, and responsiveness in individuals with T2DM and obesity, thereby making it a potentially valuable tool in the evaluation of weight-loss interventions targeted toward patients with T2DM.

Introduction

In two previous articles published in Diabetes Technology and Therapeutics, Hayes et al.^1,2 described the development and validation of a new patient-reported outcome (PRO) instrument, the Ability to Perform Physical Activities of Daily Living Questionnaire (APPADL), in individuals with type 2 diabetes mellitus (T2DM) and obesity (body mass index [BMI] ≥30 kg/m²). This PRO measure was developed in response to the need for providing the patient's perspective of the benefit of antihypergylcemic medications (e.g., incretin mimetics) that not only produce improvements in glycemic control but also potentially moderate weight loss (e.g., 5% body weight).^3,4 In a qualitative study involving individuals with T2DM and obesity, participants indicated that an expected benefit from a 5% weight loss would be an improvement in their ability to perform physical activities of daily living.⁵ Participants also identified improvements in their weight-related self-perception as an expected benefit.³ Therefore, along with the APPADL, the authors developed and psychometrically tested an instrument designed to assess the impact of weight on self-perceptions of individuals with T2DM and obesity. The purpose of this brief report is to provide support for the reliability, validity, and responsiveness of that measure, the Impact of Weight on Self-Perceptions Questionnaire (IW-SP).

Research Design and Methods

Study participants

Study participants were recruited primarily from four sites in the United States offering weight-loss programs. Details of these sites have been published elsewhere.² Eligibility criteria included (1) currently engaged in a weight-loss intervention, (2) diagnosed with T2DM at least 6 months prior to screening, (3) 25–65 years of age, and (4) BMI ≥30 kg/m². Study approval was obtained from the Independent Investigational Review Board, Inc.

Procedure

At baseline (Visit 1), site coordinators provided study participants with a study packet that included a sociodemographic form, the IW-SP, and other PRO instruments: the Short-Form 36 Version 2.0 (SF-36),⁶ the Weight-Related Symptom Measure,^7,8 the Obesity and Weight Loss-Quality of Life Measure (OWQOL),^7,8 and the APPADL.^1,2 Site coordinators captured clinical data including height, weight, comorbidities, diabetes medications, and verification of T2DM (glucose reading or confirmation of prescription for diabetes medication) on a standardized form. At Visit 2, approximately 5 days postbaseline, participants were administered the IW-SP. Six months postbaseline (Visit 3), a subgroup of participants was administered the IW-SP, the PRO instruments administered at Visit 1, and a global impression of change in weight-related self-perceptions (since last visit) question. Site coordinators also captured clinical variables (i.e., weight) at Visit 3.

IW-SP

The three-item IW-SP is designed to assess an individual's self-perceptions related to his or her weight. Transcripts from six focus groups (n=54 individuals with T2DM; BMI, 30–40 kg/m²) served as the basis for IW-SP item generation. Sixteen items in four areas—dissatisfaction with appearance, self-consciousness in social settings, avoidance of social settings, and overall self-perception—were developed along with alternate response sets and recall periods. These items were pretested and refined through two iterative rounds of cognitive interviewing (n=24 individuals with T2DM; BMI, 25–40 kg/m²). Cognitive interviews indicated that all 16 items should be retained for further psychometric testing.

To reduce unnecessary or poor items, to identify factor structure, and to estimate internal consistency of the IW-SP, a Web-based survey was conducted in 349 individuals with self-reported T2DM and obesity. Details about the participants and the survey procedures are published elsewhere.² The IW-SP factor analysis results suggested that the three “avoidance of social situations” items assess a concept different from what is assessed by the 13 other items so they were eliminated from further analysis. Of the remaining 13 items, the majority of inter-item correlations were >0.75, suggesting redundancy in measurement. Therefore, to minimize respondent burden, the decision was made to include only one item each in the areas of unhappiness with appearance, self-consciousness in social situations, and overall self-perception in the final version of the IW-SP. Factor analysis of the three items showed one factor accounting for 87% variance with factor loadings ≥0.93. Cronbach's α calculated for the IW-SP was 0.92. Each of the final items asks respondents to indicate how often they feel unhappy or self-conscious about the situation represented in the item on a 5-point scale ranging from 1=“always” to 5=“never.” Total IW-SP scores are derived by summing the item scores and dividing by the number of items (n=3). Higher IW-SP scores correspond to better self-perception.

Statistical analysis

Descriptive statistics (e.g., mean and SD) were calculated for demographics, clinical variables, and each PRO instrument's domain and total scores. Total and domain scores were reverse-scored when necessary so that higher scores for all PRO instruments corresponded to better outcome (e.g., better self-perception, better health status). IW-SP test–retest reliability (i.e., the degree to which the scores of an instrument are reproducible over time) was estimated with intraclass correlation coefficient (ICC) (absolute agreement two-way random effects model)⁹ using data from Visits 1 and 2. IW-SP internal consistency reliability (i.e., the degree to which the items of an instrument tend to assess the same underlying variable or construct) was estimated with Cronbach's α¹⁰ using data from Visits 1 and 2. Reliability coefficients of 0.80 were considered satisfactory. Individual IW-SP item scores are reported as raw scores (1–5), but to more easily compare IW-SP mean scores with those of the other PRO instrument scores reported by Hayes et al,² IW-SP total scores were also linearly transformed to a 0–100 scale.

To provide support of convergent validity (i.e., scores of a new instrument correlate with scores of one or more instruments hypothesized to assess the same or similar constructs), Pearson correlation coefficients were calculated between all PRO instrument scores at Visit 1. A priori hypotheses were that the IW-SP scores would correlate highly with OWQOL and SF-36 Mental Health domain scores. To provide support for known group or discriminative validity (i.e., scores of a new instrument are significantly different, or discriminate, between subgoups hypothesized to have different levels of the construct) in the subgroup of participants who completed Visits 1 and 3, independent t tests were performed using percentage body weight loss and participants' perception of change in self-perception from the last visit as independent variables and IW-SP change scores as the dependent variable. For the first t test, because a 5% body weight loss is considered to be important to individuals with T2DM and obesity,⁵ is clinically beneficial,^11,12 and is likely to produce a positive effect on weight-related self-esteem,¹³ participants were divided into those achieving ≥5% body weight loss from Visit 1 to Visit 3 and those who did not. For the second t test, participants were divided into those who reported better self-perception and those who reported no change/worsening since the last visit.

To provide support of IW-SP responsiveness (i.e., the ability to detect within-patient change over time), first, a paired t test was performed to detect whether there was a significant weight loss in the 40 study participants from Visit 1 to 3. Then, a paired t test was performed to detect whether there was a significant change in IW-SP total scores. Effect size (ES) (mean change in score/SD of baseline scores) was calculated for both weight loss and IW-SP scores. The α level for all analyses was set at P<0.05.

Results

Study participants

Study participants (n=106) with complete data for Visits 1 and 2 were included in the test–retest analyses. These participants were mostly African American (54%), female (69%), middle-aged (mean [SD], 52 [10] years), and moderately to severely obese (BMI mean [SD], 38 [6] kg/m²). Additional participant characteristics are reported by Hayes et al.² As reported by Hayes et al.,² three of the four study sites ceased operation. Therefore, complete data needed for the responsiveness analysis (Visits 1 and 3 data) were only available for 40 participants from the one operational study site. Characteristics of the subgroup were similar to those of the larger test–retest group with the exception that the responsiveness group was more racially diverse than the larger group (15% white vs. 31% white).

Item and scale statistics, internal consistency, and test–retest reliability

At Visit 1, IW-SP item scores ranged from 2.8 to 3.3, and item-to-total correlations ranged from 0.77 to 0.81. The raw mean IW-SP total score was 3.1, and the linearly transformed mean IW-SP total score was 51.7. Ceiling effects (percentage of participants responding “never” to an item) for the IW-SP total score were 11% (Table 1). Cronbach's α coefficients (internal consistency) calculated for Visits 1 and 2 and the ICC calculated between Visits 1 and 2 were >0.89 and 0.85, respectively.

Table 1.

Item Statistics and Reliability Coefficients for the Impact of Weight on Self-Perceptions Questionnaire (n=106)

Item number	Floor (always=1) (%)	Ceiling (never=5) (%)	Mean	SD	Median	Item-total correlation
1. How often do you feel unhappy with your appearance due to your weight?	19	15	2.8	1.4	2.0	0.77
2. When going out in public, how often do you feel self-conscious due to your weight?	13	22	3.2	1.4	3.0	0.79
3. When comparing yourself to others, how often do you feel unhappy, due to your weight?	10	26	3.3	1.4	4.0	0.81
Raw IW-SP total score (1–5)	7	11	3.1	1.3	3.0	—
Transformed IW-SP total score (0–100)	7	11	51.7	31.6	50.0	—

Visit 1 Cronbach's α=0.89 (n=106); Visit 2 Cronbach's α=0.91 (n=104); test–retest reliability (Visit 1 and 2)=0.85 (n=104).

IW-SP, Impact of Weight on Self-Perceptions Questionnaire.

Convergent validity

As hypothesized, correlation coefficients calculated between the IW-SP scores and both the OWQOL scores and SF-36 Mental Health domain scores were significant and relatively high (r=0.85 and 0.54, respectively; P<0.001). IW-SP scores also significantly and moderately correlated with the scores of the SF-36 domains Vitality, General Health, and Physical Function, APPADL scores, and Weight-Related Symptom Measure scores (all r>0.40; P<0.001).

Discriminant validity

The mean change from baseline to end point in IW-SP scores for participants who achieved a 5% or more weight loss (n=12) was significantly greater than for those individuals who did not (n=28) (mean change, 23.6 vs. 5.7; P<0.05). The mean change in IW-SP scores for those participants who reported having better weight-related self-perceptions since the last visit (n=17) was significantly greater than for those who reported no change/worsening (n=23) (mean change, 21.6 vs. 3.3; P<0.05).

Responsiveness

A significant difference was found in participants' (n=40) mean weight between Visits 1 and 3 (231.9 vs. 222.0 pounds; P<0.001). The ES for change in weight was 0.23. A significant difference was also found in participants' mean IW-SP total scores between Visits 1 and 3 (61.0 vs. 72.1; P<0.01). The ES for change in IW-SP total score was 0.34.

Discussion

The overall goal of this study was to provide support for use of the IW-SP as a potential end point in trials of weight-loss interventions targeted at individuals with T2DM and moderate to severe obesity. This could include interventions such as clinic- or commercially provided programs, as well as antihyperglycemic medications that produce weight loss. The first objective was to demonstrate acceptable item statistics and reliability with the IW-SP measure. IW-SP item statistics (e.g., ceiling effects) were acceptable, and both internal consistency and test–retest reliability of the IW-SP were quite high.

The IW-SP total scores demonstrated convergent validity with other assessments of weight-related quality of life and mental health status. In addition, significant associations were found with measures of physical functioning and vitality. Several studies have shown that weight loss of at least 5% results in improvements in aspects of mental health (e.g., self-perception) that are accompanied by improvements in physical health (e.g., ability to perform physical daily activities). This has been observed for different types of weight-loss interventions and using various PRO instruments to assess the mental and physical concepts.^14

–18

Evidence of the known group or discriminative validity of the IW-SP was found in the differences in change in IW-SP scores between the two weight-loss groups (≥5% loss vs. <5% loss) and two global impression of change in self-perceptions groups (better vs. no change/worse). The difference in the magnitude of change, approximately 15 points on a 0–100 scale in both analyses, may serve as a preliminary estimate of meaningful change in IW-SP scores for future studies.

Support for the responsiveness of the IW-SP was provided by the significant change in Visit 1 and 3 IW-SP scores that mirrored that of weight change. The ES values for both variables were similar, suggesting a close correspondence between weight change and IW-SP scores and confirming the association between weight loss and self-perception that was anticipated by individuals with T2DM and obesity in a qualitative study.³

Limitations

The current sample was mostly African American (54%) and therefore not necessarily representative of the general population of individuals with T2DM and obesity. However, given that initial IW-SP developmental work was conducted in a primarily white sample, recruitment in this study was aimed at providing support for IW-SP validation in a more racially diverse sample. The dramatic loss of study participants from Visit 1 to Visit 3 due to the close of operation for three of the four sites affected the robustness of the analysis for responsiveness. However, despite the dramatic reduction in sample size, a change in weight loss and a corresponding improvement in IW-SP scores were detected. Future research is needed in larger samples to confirm these findings.

Conclusions

The IW-SP has shown the potential to be a useful tool in the evaluation of weight-loss interventions that target patients with T2DM and obesity. This may include antihyperglycemic medications that produce weight loss. The IW-SP has been developed according to regulatory guidance,¹⁹ which highlights the necessity for obtaining input from the target population of interest about their disease or condition for instrument development. The IW-SP has a Flesch Kincaid reading level of 6.2 and has been linguistically validated in several languages. The IW-SP is copyrighted by Eli Lilly and Company. For permission to use the IW-SP, please contact copyright@lilly.com.

Footnotes

Acknowledgments

This study was funded by Eli Lilly and Company.

Author Disclosure Statement

R.P.H. is a retired full-time employee and shareholder of Eli Lilly and Company. A.M.D. is a full-time employee and shareholder of Eli Lilly and Company.

References

Hayes

, Nelson

, Meldahl

, Curtis

: Ability to perform daily physical activities in individuals with type 2 diabetes and moderate obesity: a preliminary validation of the Impact of Weight on Activities of Daily Living Questionnaire. Diabetes Technol Ther, 2011; 13:705–712.

Hayes

, Schultz

, Naegeli

, Curtis

: Test–retest, responsiveness, and minimal important change of the Ability to Perform Physical Activities of Daily Living Questionnaire in individuals with type 2 diabetes and obesity. Diabetes Technol Ther, 2012; 14:1118–1125.

Amori

, Lau

, Pittas

: Efficacy and safety of incretin therapy in type 2 diabetes. JAMA, 2007; 298:194–206.

Vilsboll

, Christensen

, Junker

, Knop

, Gluud

: Effects of glucagon-like peptide-1 receptor agonists on weight loss: systematic review and meta-analyses of randomised controlled trials. BMJ, 2012; 344:d7771.

Curtis

, Hayes

, Fehnel

, Zografos

: Assessing the effect of weight and weight loss in obese persons with type 2 diabetes. Diabetes Metab Syndr Obes, 2008; 1:13–23.

Ware

. SF-36^® Health Survey Update. www.sf-36.org/tools/SF36.shtml (accessed January 13, 2012 ).

Niero

, Martin

, Finger

, Lucas

, Mear

, Wild

, Glauda

, Patrick

: A new approach to multicultural item generation in the development of two obesity-specific measures: the Obesity and Weight Loss Quality of Life (OWLQOL) questionnaire and the Weight-Related Symptom Measure (WRSM). Clin Ther, 2002; 24:690–700.

Patrick

, Bushnell

, Rothman

: Performance of two self-report measures for evaluating obesity and weight loss. Obes Res, 2004; 12:48–57.

McGraw

, Wong

: Forming inferences about some intraclass correlation coefficients. Psychol Methods, 1996; 1:30–46.

10.

Cronbach

: Coefficient alpha and the internal structure of tests. Psychometrika, 1951; 16:297–334.

11.

de Leiva

: What are the benefits of moderate weight loss?. Exp Clin Endocrinol Diabetes, 1998; 106(Suppl 2):10–13.

12.

U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER). Guidance for Industry: Developing Products for Weight Management. 2007. www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm071612.pdf (accessed March 25, 2012 ).

13.

Kolotkin

, Norquist

, Crosby

, Suryanwanshi

, Teixeira

, Heymsfield

, Erondu

, Nguyen

: One-year health-related quality of life outcomes in weight loss trial participants: comparison of three measures. Health Qual Life Outcomes, 2009; 7:53.

14.

Rippe

, Price

, Hess

, Kline

, DeMers

, Damitz

, Kreidieh

, Freedsoni

: Improved psychological well-being, quality of life, and health practices in moderately overweight women participating in a 12-week structured weight loss program. Obes Res, 1998; 6:208–218.

15.

Fontaine

, Barofsky

, Bartlett

, Franckowiak

, Andersen

: Weight loss and health-related quality of life: results at 1-year follow-up. Eat Behav, 2004; 5:85–88.

16.

Wouters

, Van Nunen

, Geenen

, Kolotkin

, Vingerhoets

: Effects of aquajogging in obese adults: a pilot study. J Obes 2010;, 2010:231074.

17.

Desiderio

, Trastulli

, Scalercio

, Mirri

, Grandone

, Cirocchi

, Penzo

, Santoro

, Redler

, Boselli

, Noya

, Fatati

, Parisi

: Effects of laparoscopic sleeve gastrectomy in patients with morbid obesity and metabolic disorders. Diabetes Technol Ther, 2013; 15:1004–1009.

18.

Desiderio

, Trastulli

, Scalercio

, Cirocchi

, Carloni

, Moriconi

, Boselli

, Noya

, Parisi

: Laparoscopic sleeve gastrectomy and medical management for the treatment of type 2 diabetes mellitus in non-morbidly obese patients: a single-center experience. Diabetes Technol Ther, 2013; 15:281–288.

19.

Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological Health (CDRH), Food and Drug Administration, U.S. Department of Health and Human Services: Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. 2009. www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf (accessed March 25, 2012 ).