Development and Psychometric Validation of the Novel Glucose Monitoring Experiences Questionnaire Among Adults with Type 1 Diabetes

Abstract

Background:

Glucose monitoring is an essential aspect of self-care for people with type 1 diabetes. With technologies developing rapidly, valid assessment of user experiences and satisfaction is needed. Our aim was to develop a novel measure: the Glucose Monitoring Experiences Questionnaire (GME-Q).

Methods:

Questionnaire design was informed by exploratory and cognitive debriefing interviews. The GME-Q was included in a large online survey enabling psychometric validation.

Results:

The interview sample included 17 adults (aged [mean ± SD] 46 ± 11 years, 53% women) with type 1 diabetes duration of 26 ± 14 years. The proposed conceptual framework included three domains: “Effectiveness”, “Intrusiveness”, and “Convenience”, assessed with 25 items plus a single, overview item. The validation sample included 589 adults (aged 44 ± 15 years; 64% women) with type 1 diabetes (duration: 22 ± 14 years, self-monitoring blood glucose [SMBG] using finger-prick devices: median [IQR] 6 [4–7] daily checks). Questionnaire acceptability was indicated: 98% (n = 578) completion rate. After deleting 3 redundant items, principal components analysis supported a 22-item questionnaire with 3 domains (“Effectiveness” [9 items]; “Intrusiveness” [6 items]; “Convenience” [7 items]), accounting for 55% of variance, with good internal consistency reliability (α = 0.83–0.88). Subscales correlated significantly (r _s = ±0.44–0.66, P < 0.001) with the single, overview item, together explaining 51% of the total variance in the single item score. Associations with demographic and clinical characteristics supported convergent and discriminant validity.

Conclusions:

Overall, the 22-item GME-Q is a brief, acceptable, valid, and reliable measure of satisfaction with glucose monitoring in adults with type 1 diabetes using SMBG, and this needs to be assessed among those using continuous glucose monitoring.

Introduction

Glucose monitoring is an essential aspect of self-care for people with type 1 diabetes. Guidelines indicate that adults with type 1 diabetes may need to monitor 6–10 times per day, for example, to inform insulin dose adjustments before meals/snacks and bed, to identify and inform treatment of hypoglycemia, and before driving or during long journeys.¹ Higher daily frequency of monitoring is associated with lower glycated hemoglobin (HbA1c),² which reduces the risk of the development of long-term complications.³ The past decade alone has witnessed impressive developments, including more reliable and advanced continuous glucose monitoring (CGM) and “flash” monitoring systems.⁴ Such innovation needs to be evaluated rigorously, not only in terms of the safety, accuracy, and efficacy of devices but also in terms of user experience.

Satisfaction is an important aspect of the user experience and can be conceptualized as the individual's evaluation of the convergence of their experience with their expectations. One of the most useful definitions in the health literature suggests that satisfaction with a medication (or medical device) “deals specifically with the evaluative aspect of the patient's experience…[which] involves both the process and the outcomes associated with [it].”⁵ The emphasis on both process and outcomes is important, and implicit in this definition is the notion that the person attaches value to specific attributes of the medication (or device) in relationship to his or her own priorities and preferences, and that these may be unique to each individual's experience. This being the case, it then follows that satisfaction can only be assessed by means of a self-report measure, as any form of proxy report (via clinician or carer) will involve bias, based on their own value judgments or lack of appreciation of issues important to the individual.

Satisfaction is an important outcome in clinical trials and real-world evaluations of medical treatments and devices.⁵ It has been shown consistently that positive experiences are associated with clinical effectiveness.⁶ Rising interest in the assessment of satisfaction⁷ parallels the rising recognition of the person with diabetes as an “active consumer” rather than “passive recipient” of health care services, treatments, and technologies.⁵ Indeed, satisfaction is an important proxy of future or longer term use, as people are unlikely to continue using a medication or device that they perceive to be ineffective; and, even if they perceive it to be effective, may not continue with it (or use it optimally) if they find it inconvenient or intrusive. Thus, assessing satisfaction can help manufacturers, policy makers, and clinicians distinguish between therapeutic modalities, which have equivalent biomedical efficacy.⁵

In diabetes, several self-report measures of satisfaction exist. Some focus broadly on treatment satisfaction, taking into account various aspects of self-care, such as diet, tablets, and insulin, but typically excluding glucose monitoring.⁸ Some focus on a specific glucose monitoring technology,⁹ thus limiting opportunity for comparisons between technologies. Other measures focus on the related concept of emotional/behavioral burden rather than on satisfaction.¹⁰ Finally, some measures are relatively lengthy and may be more suited to in-depth research rather than as brief tools for use in clinical trials and/or routine clinical practice.^9,11 Thus, novel measures are needed that (a) have a strong focus on satisfaction with the glucose monitoring experience; (b) enable comparison of user experiences across a range of monitoring devices; and (c) are brief, valid, reliable, and easy to complete.

The two-fold aims of this study were (a) to design a new conceptually valid and clinically relevant self-report measure of satisfaction with glucose monitoring for adults with type 1 diabetes, focused on the user's evaluation of various attributes of monitoring devices and (b) assess the psychometric properties of the new measure in adults with type 1 diabetes using self-monitoring of blood glucose (SMBG).

Methods

Study 1: design and debriefing of the Glucose Monitoring Experiences Questionnaire

The questionnaire design and content was informed by the literature and exploratory interviews with adults with type 1 diabetes. It was refined based upon the findings of cognitive debriefing interviews with adults with type 1 diabetes. Ethical and site-specific approvals were in place (NRES Committee North East—Tyne and Wear South [REC reference 07/Q0904/11]).

Participants and recruitment

Eligible participants were adults (aged 18–75 years) with established type 1 diabetes (≥12 months' duration); registered with one of two U.K. specialist type 1 diabetes centers (Newcastle and Manchester); with experience of SMBG and CGM use (to be able to offer insights about finger-prick and CGM); able to read/write in English and provide informed content to participate. A total of 20 adults with type 1 diabetes were approached, and three declined (all from Newcastle) for logistical reasons.

Procedure

Participants registered were invited to participate by their local diabetologist. If willing, they signed an informed consent form. Interviews were conducted by an academic health psychologist (J.S. or S.M.B.), not involved in the participants' diabetes care; three interviews were conducted jointly (J.S. and S.M.B.) to maximize comparability. Interviews were arranged at a time and location convenient to the participant: most were conducted in a private room at the hospital, but some were conducted by telephone. Demographic data were collected by self-report, with HbA1c retrieved from medical records.

All participants completed exploratory, semistructured interviews (mean [range] duration: 52 [34–69] min), during which they shared their experiences of diabetes self-management, including finger-prick and CGM. All interviews were audio recorded, transcribed, and subjected to thematic analysis.¹² The goal was to identify themes (concepts) that were relevant to the experience of glucose monitoring, so that these could inform the questionnaire design. In the first stage (familiarization), J.S. and S.M.B. listened to each interview recording several times. In the second stage, they reviewed transcripts and generated concepts for each participant, making notes independently about relevant concepts and extracting illustrative quotes. In the third and fourth stages, they discussed the concepts, and then summarized them for, and discussed them with, the wider research team (H.S., S.L. and J.A.M.S.). In the final stage, further checks (by J.S. and S.M.B.) confirmed no additional concepts emerging from the data.

After the initial five interviews, the first iteration of the Glucose Monitoring Experiences Questionnaire (GME-Q) was developed. A conceptual framework was proposed based upon our reading of the literature, with three domains (Effectiveness, Intrusiveness, and Convenience) informed by the theoretical underpinnings of what comprises satisfaction.¹³ Concepts for each domain were identified from the literature, the exploratory interviews and the cognitive debriefing interviews, and illustrated with participant quotes.

Twelve of the 17 participants, after a brief break following their exploratory interview, completed the draft GME-Q unaided, and then participated in a cognitive debriefing (“think aloud”) interview to discuss the questionnaire (instructions, items, response options) in terms of its relevance, comprehensiveness, comprehensibility, and ease of use.¹⁴ Participants were encouraged to advise the interviewer of any words that were difficult to understand, and any issues that were missing from their questionnaire.

Following each set of exploratory and debriefing interviews, J.S. and S.M.B. listened to interview recordings and reviewed their notes and the completed draft questionnaire for relevant issues. The GME-Q was refined through several iterations of exploratory interviews, modifications, and cognitive debriefing. Consensus about item revisions was reached through discussion of the issues arising in the interviews (J.S., S.M.B., H.S., S.L., and J.A.M.S.). To ensure that the questionnaire was relevant to those using either finger-prick or CGM devices, words such as “blood glucose” and “finger-prick” were avoided, as they are indicative of a particular monitoring method. The questionnaire was drafted to include both positively and negatively worded statements.

Study 2: psychometric validation of the GME-Q

This study used data from the YourSAY (Self-management And You): Glucose Monitoring study, a national online survey of Australian adults with insulin-treated diabetes, which aimed to explore glucose monitoring practices, attitudes, and barriers. Ethical approval was granted by Deakin University Human Ethics Committee (2015-270).

Participants and recruitment

Eligibility for the current study included adults (aged ≥18 years) self-reporting a diagnosis of type 1 diabetes, English speaking, currently residing in Australia, and completion of ≥1 of the GME-Q items. Due to the broader study focus on SMBG, participants were ineligible if they were currently using a CGM device. A convenience sample was recruited via social media (Twitter, Facebook), e-newsletter, and website advertisements facilitated by two leading national diabetes charities, Diabetes Australia, and associated state-based organizations, and JDRF. Data collection took place from September 9, 2015 to October 21, 2015.

Procedure

Potential participants were directed to a website that featured a plain language description of the study, a consent form, and the online survey (securely hosted by Qualtrics, Copyright © 2015, Provo, UT). After indicating consent and being screened for eligibility, participants proceeded to the survey.

Measures

In addition to the newly developed GME-Q (described in Results section: Study 1), participants in Study 2 were invited to respond to two free-text questions to indicate the best and worst aspects (up to three for each) of “your current method of monitoring.” These were included to capture additional information that might confirm existing items and/or prompt future revisions.

The survey also included several study-specific items, including demographic and clinical characteristics, such as age, diabetes duration, self-reported HbA1c in the past 6 months, typical daily SMBG frequency, and frequency of “mild” self-treated hypoglycemia event in the past week (day or night).

Statistical analyses

All statistical analyses were conducted using IBM SPSS version 23 (Chicago, IL). An alpha level of P < 0.05 was taken to indicate significance. Descriptive statistics [mean ± SD, n (%)] were calculated to describe participants' demographic and clinical characteristics. Missing data vary per variable, and valid percentage is reported. The Kolmogorov–Smirnov test of normality was applied to assess the distribution of GME-Q data. All data were distributed nonnormally, necessitating the use of nonparametric statistics.

Acceptability of the scale was assessed by examining questionnaire completion rates, where high overall completion rates (≥90%) were taken as evidence of acceptability. On subsequent analysis, loss of data was minimized by pairwise deletion of participants with missing data. Descriptive statistics were used to identify response patterns and item floor/ceiling effects (i.e., >20% scoring minimum/maximum response).¹⁵

Barlett's Test of Sphericity was assessed to check for intercorrelation between items, and the determinant was screened for multicollinearity. Two-tailed interitem Spearman's rho correlations were used to identify items with very high (r _s > 0.7) or very low (r _s < 0.3) interitem correlations. The Kaiser-Meyer-Olkin statistic (0.93) indicated that the sample size was adequate to proceed with dimension reduction analyses.¹⁶ Structural validity of the questionnaire was assessed using principal components analysis (PCA) with direct oblimin rotation. The number of questionnaire factors (i.e., domains) to retain was informed by Kaiser's eigenvalue (≥1), percentage variance explained by each factor and factor loadings. Factor loadings were considered meaningful if ≥0.3 and high if ≥0.6.¹⁷ Double loadings, where items loads ≥0.3 on two or more factors, were identified. An iterative scale reduction process was undertaken to remove any problematic items following which the factor analysis was recalculated. Finally, a forced one-factor solution was conducted to assess the validity of the calculation of a total score.

Cronbach's alpha was used to assess internal consistency reliability of the identified factors (domains) and a forced one-factor solution (total scale), where ≥0.7 was considered satisfactory and ≥0.95 would indicate some item redundancy.¹⁸

Following item reduction, factor (domain) and total scores were calculated (sum of scores, divided by number of items completed). To assess how many complete responses (i.e., not missing) per participant are required to retain strong internal consistency for each domain, Cronbach's alpha was recalculated iteratively (after deleting the item with the strongest item-total correlation) until reliability was compromised (α < 0.70). Participant response patterns on the final scales were examined using descriptive statistics.

In the absence of a gold-standard measure of glucose monitoring satisfaction, construct validity was assessed by regressing the domain subscale scores against a single overview item (“my current method of glucose monitoring suits me well”). All domains were expected to be significantly and independently associated with the single item. In addition, Spearman's rho correlations were calculated between the three subscales (domains). To demonstrate convergent validity, Spearman's rho correlations among the three GME-Q domain scores were expected to be moderate-to-large (r _s > ±0.3), but weak (r _s < ±0.3) between GME-Q scores and self-reported frequency of daily SMBG and self-treated (“mild”) hypoglycemia in the past week. To demonstrate discriminant validity, Spearman's rho correlations were expected to be weak (r _s < ±0.3) between GME-Q scores and age/diabetes duration.¹⁹

Qualitative responses to the free-text questions were reviewed to provide a further check of the content validity of the GME-Q, that is, that items adequately assess the most commonly reported glucose monitoring perceptions, and to identify relevant issues that might be missing and could be incorporated into future revisions of the measure.

Results

Study 1: design and debriefing of the GME-Q

The sample comprised 17 adults [including 9 (53%) women], aged (mean ± SD) 46 ± 11 years, with a type 1 diabetes duration of 26 ± 14 years and an HbA1c of 61 ± 11 mmol/mol (7.7% ± 0.8%). Nine (53%) were using an insulin pump and, in addition to using SMBG routinely, all had experience of CGM.

The themes and issues raised by participants confirmed a conceptual framework of glucose monitoring experience with three domains: “Effectiveness,” “Intrusiveness,” and “Convenience.” Table 1 provides illustrative quotes for the concepts identified within each domain, which informed item development. Cognitive debriefing enabled instructions and item wording to be refined, and additional items to be added in an iterative process.

Table 1.

Conceptual Framework for the Glucose Monitoring Experiences Questionnaire, Including Illustrative Participant Quotes for Each Potential Domain (Subscale) and Concept (Item)

Domain	Concept	Examples of participants' comments
Effectiveness	Avoid hypos (while asleep)	“We identified that I was generally going hypo in the night, and some nights it was a lot lower than others”
	Avoid hypos (while asleep)	“Now with a blood sugar of 6 or 7 at night I am confident to go to bed because I know it's going to wake me up. If it starts to drop, it's going to wake me up. So, yeah, it's been a huge benefit”
	Avoid hypos (when awake)	“At least you feel safe from the risk of hypos”
	Avoid hypos (when awake)	“I felt safer from the point of view of hypos”
	Avoid high blood glucose	“Now I have the CGM on, I can respond to high blood sugars more quickly because it alerts me so you're not waiting two and three hours between your blood sugar testing to know that it's on the rise”
	Monitor glucose trends	“Technically it was fantastic because if your blood sugar was dropping rapidly… it would have a double arrow on it so that you knew… likewise if it was going up”
	Monitor glucose trends	“Although you know it's not keeping up with where you actually are, you can see the trend of what's happening and that's really useful on a drop”
	Optimal glucose control	“I thought it would keep you at almost a perfect blood sugar and it was nowhere near that”
	Optimal glucose control	“My HbA1c improved dramatically”
	Personal control	“When I ran out [of sensors], I felt completely lost and at odds and felt as if I had no idea of what was going on, because you're just getting these little snapshots…”
		“I've felt for the first time in my life, after 40 years, where I've felt I know what's going on all the time”
		“It's the control. Where you know you're constantly at the right level”
	Reassurance	“[I wore it] to keep me safe, really. That was the only reason”
		“Often, I'd blame myself for it [but] I feel much more confident, since I've had the CGM on, when I can look at, you know, the patterns every day and through the night, all day, I feel much more confident that actually it's not anything I'm doing or not doing”
		“It just gives you peace of mind, really”
		“Sometimes the sensor can fail. It can just give you the wrong readings. It can also disconnect through dislodging and you have to be careful with that”
	Confidence (to change treatment)	“We altered my long-term rates… we did find a couple of things out”
		“If it's going up, I know I can give myself a bolus to bring it back down, so I feel much more in control of high blood sugars…”
		“I feel confident about making changes to remedy problems and I feel like a weight's been lifted”
	Confidence (to be active)	“I've used it when I knew I was going to do some serious physical exertion, because that's when I feel I'm at greatest risk”
	Confidence (to be active)	“I would never exercise before I had this, never. I don't mean just walking, but, you know, exercise to keep healthy, and it was just a complete disaster for me because I had to probably eat twice the amount, probably three times, of calories to burn off either in anticipation that I was going to go hypo, or afterwards when I did go hypo. So, it wasn't a pleasant experience… Whereas now I've started to do stuff with friends, go for a jog, I've never been for a jog [Laughs], and do a power walk and felt completely confident”
	Freedom in daily life	“It didn't benefit my lifestyle at all”
		“I wasn't worried that I was going to end up in a heap on the floor”
		“Because I've been cycling for a year and a half now, I can see that could be really useful to me”
		“What's made the huge life, completely life-changing benefit has been the CGM”
		“To have it go off in the middle when you're having sexual intercourse isn't great. It sort of stops things dead while you're fumbling about”
		“All the limitations of not being able to be left on your own of a night time”
Intrusiveness	Feeling like a robot	“Well, it's another thing to sort of plug onto yourself”
		“You've got the tape taping it on, so you end up with a huge square patch on your stomach where it's taped to you”
		“It was a bit awkward but not uncomfortable”
	Sleep disruption	“A couple of nights ago it was alarming and my blood sugar was sky high, 18 or 20. I gave myself the corrective bolus but it alarms—I can't remember what it's set for—but every so often it alarms you when it's still above 10. So, me and my husband are woken up every hour by the alarm going off and I wasn't acting on it because it was gradually coming down and I knew the insulin was taking effect but, you know, we did have a sleepless night”
	Body image	“It wasn't something you'd want to wear all the time, though you'd be tempted because it meant you didn't have to do blood sugars”
	Body image	“It looked a bit cumbersome and I still think the design, being an engineer, it makes me think they could have done a better job”
	Information (overload)	“I like having that amount of information, yeah”
	Information (overload)	“It would give me a lot more help in making decisions. Peace of mind. There's no harm in knowing that little bit more. Maybe some people might think it's too much information. I like to know a little bit more about what's happening”
	Other people's reactions	“A couple of times people sort of pointed it out and I would just say to them, yes, I'm a doctor on call, what's it got to do with you?”
		“When you're carrying that around, you know, people don't know what it is. It could be your phone or pager or anything”
		“If I'm at work, or if I'm in a public place, I don't really like to have to do that publicly. I have to try and do it discreetly”
	Preoccupation with glucose levels	“The downside is that if it's going up, I'm constantly looking every half an hour, every 15 minutes, to see what it's doing and I know it's when it's had time to have an effect and I have got to, and I think, oh, you know, I've had this really high blood sugar for, I don't know, two hours now, whereas before I'd only been checking my blood sugar every couple of hours”
	Preoccupation with glucose levels	“You're much more aware all the time of what your blood sugar is, so it's much more, you know, work. In between seeing people, I would usually get it out and have a quick look, whereas I wouldn't ever get my blood sugar out every 10 minutes and check it. Whereas, I have a quick look, feel better, I feel confident to carry on”
	Anxiety about glucose levels	“You might be looking at it every five minutes thinking, oh, well, it's 13.2, or you might even think it's going up, you know, it was 13 ten minutes ago and now it's 13.2, and feel you have to act on that. Whereas, the reality is it's not going to be exactly the same all the time. It's more a pattern over an hour or two that you notice things. So, I suppose that could be very anxiety-provoking. I think it was for me”
Convenience	Easy	“I've got diabetes but I haven't because I've got these things that are going to do it all for me, you know. But what I found was it wasn't as simple as that.”
		“It's a lot of hard work”
		“At the moment, possibly, I think I test about 6 to 8 times a day where you could just look continuously, even every half hour, you know. It takes nothing to look at it and see what your blood reading is”
	Time-consuming	“You had a to calibrate it continuously”
		“I don't find [finger-prick glucose monitoring] a problem at all. I mean, 5 seconds you know, that's all it takes”
		“Its definitely the way forward but you have to put time aside, its not something you can just fit in”
	Convenience	“The sensor actually helped me because I didn't need to do as many blood tests”
		“You had to do blood sugars to calibrate it, so it was doubling up on everything”
		“If you're busy at work… you think, oh, I can't come out of this meeting to go and get my blood tests or whatever and you think, I'll just finish the meeting. Whereas if you can look [at the CGM], you don't disturb the meeting, you don't do anything, you just carry on, where you always feel like you've got to stop things to do a blood test and that”
	Discrete	“Its tiny and that is absolutely wonderful”
		“The sensors are so discreet now”
		“But the alarms on it. It kept alarming… if the sensor moved, it alarmed”
		“It bleeps like hell. It gives you loud bleeps… it bleeping all the time”
	Discomfort/pain	“I got a reaction to the glue”
		“Occasionally I hit a nerve but it's generally painless”
		“Sometimes it can be painful… and one time it bled as well”
		“You end up with more sore fingertips”

CGM, continuous glucose monitoring; HbA1c, glycated hemoglobin.

The newly formed GME-Q included 26 items (9 of which were negatively worded). The scale was structured such that a single stem (“My current method of monitoring…”) was followed by 25 simply worded items (e.g., “…is easy to do”). Items were constructed as statements with responses on a five-point Likert scale (strongly disagree—strongly agree). The 26th item is a single, overview item asking the person to rate (on the same scale) the extent to which “my current method of monitoring suits me well.”

Study 2: psychometric validation of the GME-Q

Sample characteristics

A total of 592 adults with type 1 diabetes participated in the YourSAY: Glucose Monitoring study. Of these, three (0.5%) participants were excluded due to noncompletion of any GME-Q item (see Acceptability section for further details). Demographic and clinical characteristics of the eligible sample (N = 589) are reported in Table 2. Participants had been living with type 1 diabetes for 23 ± 14 years and around half (57%) were administering their insulin via multiple daily injections. Self-reported HbA1c was 57 ± 14 mmol/mol, and SMBG frequency was 6 ± 3 times per day.

Table 2.

Demographic and Clinical Characteristics of Psychometric Validation Sample (N = 589)

	Missing, N (%)	Mean ± SD, Median (IQR) or n (%) ^a
Demographic characteristics
Gender: Women	0 (0)	378 (64.2)
Age, years	0 (0)	43.5 ± 14.6
Geographic area: Metropolitan	7 (1.2)	351 (60.3)
Relationship status: In a relationship	7 (1.2)	419 (72.0)
Employment status: In paid employment	1 (0.2)	400 (68.0)
Education: Less than year 12 completion		61 (10.6)
Completed high school	15 (2.5)	89 (15.5)
Vocational education/diploma		141 (24.6)
University—undergraduate		156 (27.2)
University—postgraduate		127 (22.1)
Country of birth: Australia	0 (0)	490 (83.2)
Main language: English	0 (0)	584 (99.2)
Aboriginal or Torres Strait Islander descent	0 (0)	6 (1.0)
Clinical characteristics
Diabetes duration, years	0 (0)	22.5 ± 14.4
Insulin administration type: pump	0 (0)	249 (42.3)
Diabetes-related complications, number	0 (0)	0.6 ± 1.0
Diabetes-related complications, ≥1		191 (32.4)
Self-reported HbA1c in past 6 months: %, mmol/mol	87 (14.8)	7.4 ± 1.2,
Self-reported HbA1c in past 6 months: %, mmol/mol	87 (14.8)	57 ± 14
Mild hypoglycemia: episodes in past week	101 (17.1)	2.5 ± 2.3
Episode of severe hypoglycemia in past 6 months: ≥1	100 (17.0)	84 (17.2)
Number of glucose checks per day	7 (1.2)	5.9 ± 3.0
GME-Q^b
Domain scores
Effectiveness		3.7 (3.2–4.0)
Intrusiveness		2.7 (2.2–3.2)
Convenience		3.4 (2.9–4.0)
Total satisfaction score	11 (1.9)	3.4 (3.1–3.8)

Valid percentage reported.

Items scored from 1 to 5, where 5 indicates more positive glucose monitoring experience, except for intrusiveness scale where 5 indicates a less positive experience. Intrusiveness domain reverse scored before calculating total score.

GME-Q, Glucose Monitoring Experiences Questionnaire.

Response patterns and descriptive statistics

Descriptive statistics and missing data per item for the final 23-item scale and single overview item are displayed in Supplementary Table S1. The full range of response options was utilized for every item, although for most, the distribution of responses was negatively skewed. Four items (1, 9, 15, and 16) displayed ceiling effects, while items 11 and 12 displayed floor effects.

Acceptability

Among participants who attempted the GME-Q (i.e., completed ≥1 item), 579 (97.8%) respondents completed the 25 specific items and 13 (2.2%) had missing data, including 6 participants who skipped just 1 item. A total of 588 (99.3%) completed the single overview item (skipped by 5 participants).

Item screening

Very high interitem Spearman's rho correlations were not observed. However, the determinant value (4.003 e-6) suggests that this may be due to multicollinearity. Very low inter-item correlations were common across items (r _s < 0.3 for a third of all interitem relationships). In particular, items 8, 12, and 21 displayed low interitem correlations with ≥50% of all other items. However, Barlett's Test of Sphericity was significant indicating that the variables were intercorrelated [χ ² (300) = 7131, P < 0.001].

Scale reduction, structure, and internal consistency reliability

An unforced PCA was performed on the 25-item scale. Four factors were extracted using the eigenvalue cut-point of ≥1, accounting for 59% variance overall. However, only two items loaded ≥0.3 on to the fourth factor, both double loading elsewhere. Furthermore, the fourth factor contributed just 5% total variance with an initial eigenvalue of 1.3. Thus, a forced three-factor solution was sought. The three-factor structure accounted for 53% of the total variance and reflected the conceptual framework proposed in Study 1 (“Effectiveness,” “Intrusiveness,” ”Convenience”), with all items loading ≥0.3 and four double loadings (item 6, 7, 19 and 20). An iterative scale reduction process was undertaken to remove problematic items and to reduce the number of items overall:

Item 8 (“…it's accurate”): displayed a low factor loading (0.32), low commonalities after extraction (<0.3), and low interitem correlations with 50% of the items. In addition, 82% of participants agreed or strongly agreed, with a further 16% neither agreeing nor disagreeing, that their current method of glucose monitoring was accurate, suggesting low discriminant validity. Thus, item 8 was deleted.

The remaining 24 items were subjected again to a forced three-factor solution, which accounted for 55% of the variance. All 24 items loaded >0.3 and 2 items (19 and 25) double loaded.

Item 19 (“…gives me enough information”) double loaded across two factors (“Convenience” and “Effectiveness”). Furthermore, item 19 is conceptually related to item 14 (“…gives me too much information”) on the “Intrusiveness” factor. Consequently, this item was removed due to redundancy. However, item 25 (“…gives me freedom in my everyday life”) was retained for the purposes of content validity. It was considered to assess a concept highlighted by interview participants (Study 1) and not otherwise captured in the remaining items.

On reviewing the remaining items, item 3 (“…gives me confidence to do what I want”) was identified for deletion as it was found to be redundant, and reflective of other more specific items included in the scale with higher loadings (e.g., “…gives me the confidence to make changes in my treatment” = 0.65, compared to 0.47).

The iterative process above led to the removal of items 3, 8, and 19, and a resulting forced three-factor solution for the 22-item questionnaire, accounting for 55% of the variance. The final forced three-factor solution is presented in Table 3. Item 25 continued to double load on two factors (“Convenience” and “Effectiveness”). Given the marginally stronger factor loading, and because the item content is more similar to other items in this factor, from here on item 25 is to be included in the “Effectiveness” domain for scoring purposes. The three-factor solution was otherwise stable, with findings being replicated on two randomly selected subsamples (n = 276 and n = 313).

Table 3.

Glucose Monitoring Experiences Questionnaire Forced Three-Factor and One-Factor Structure and Internal Consistency Reliability

Item No. and wording		Forced 3-factors ^a			Forced 1-factor
Item No. and wording		1 Effectiveness	2 Intrusiveness	3 Convenience	Total satisfaction
1	…is easy to do			0.680	0.506
2^b	…takes a lot of time			−0.600	−0.586
4	…is convenient			0.781	0.647
5	…is discrete			0.648	0.636
6^b	…disrupts my sleep^c			−0.434^c	−0.513
7	…lets me be as physically active as I like			0.416	0.683
9	…reassures me	0.564			0.583
10^b	…makes me feel like a robot/machine		0.744		−0.577
11^b	…makes me feel uncomfortable about how my body looks		0.789		−0.589
12^b	…gives me too much information		0.578		−0.401
13^b	…causes other people to react negatively (e.g., to stare/ask intrusive questions)		0.594		−0.564
14^b	…means I am constantly looking at my glucose levels		0.537		−0.300
15	…means I have an accurate snapshot of my glucose levels	0.623			0.595
16	…helps me keep my glucose levels within target	0.784			0.601
17^b	…makes me feel anxious about my glucose levels		0.601		−0.549
18	…helps me avoid hypos during the day	0.761			0.529
20^b	…causes me pain or discomfort			−0.428	−0.481
21	…helps me to avoid high glucose	0.762			0.570
22	…helps me know in which direction my glucose is moving (falling/rising)	0.516			0.544
23	…helps me to avoid hypos when I am asleep	0.581			0.596
24	…gives me confidence to make changes in my treatment	0.636			0.671
25	…gives me freedom in my everyday life^d	0.437^d		0.339^d	0.744
No. of items per scale		9	6	7	22
Eigenvalue		7.9	2.5	1.7	7.9
Total variance explained %		35.9	11.5	7.6	35.9
Internal consistency reliability: Cronbach's alpha		0.881	0.827	0.835	0.908

Items 3, 8, and 19 are not presented in this final structure (see details in text).

Factor loadings <0.3 suppressed.

Negatively worded item. Rotation converged in 9 iterations.

Item 6 originally conceptualized as “Intrusiveness” but factor loading indicates it an aspect of “Convenience.”

Item 25 is included in the scoring of the “Effectiveness” subscale but not in the “Convenience” subscale, due to the higher factor loading.

All three subscales demonstrated strong internal consistency (α range = 0.827–0.881). Furthermore, the “alpha if item removed” for all items demonstrated that their removal would not improve the reliability of the scales.

A forced one-factor solution accounted for 36% of the total scale variance with all items loading >0.3 with the exception of item 14 (loading = 0.299). Internal consistency reliability of the overall scale was very strong (α = 0.908). Item 14 displayed a weak corrected item-total correlation and its removal would result in a marginally better alpha (α = 0.910). However, the decision was made to retain the item for the purposes of content validity.

GME-Q scoring

Composite scores were calculated for the GME-Q “Total satisfaction” and three subscale domain scores by summing the relevant items and dividing by the number of items completed in the relevant scale to create a score from 1 to 5, with higher scores indicating greater endorsement of the concept being measured. Scores for items 2, 7, and 20 are reversed before summing the “Convenience” or “Total Satisfaction” scores, while the “Intrusiveness” items are reversed only for the “Total Satisfaction” score. Given the very small amount of missing data, (sub)scale scores were calculated for those with complete data. However, iterative recalculation of Cronbach's alpha suggests ≤5, ≤2, and ≤1 missing items can be tolerated for “Effectiveness,” “Convenience,” and “Intrusiveness” (sub)scale scores, respectively. The median score (lower and upper interquartile range [IQR]) for each (sub)scale is reported in Supplementary Table S1.

Validity

Construct validity was demonstrated, with the three GME-Q subscale scores (domains) together accounting for 51% (adjusted r ²) of the variance in the single overview item, and all three domains significantly independently contributed to the model [F (3,574) = 202.7, P < 0.001]. Large significant correlations were observed between the overview item and “Effectiveness” (r _s = 0.66, P < 0.001) and “Convenience” (r _s = 0.60, P < 0.001), while a moderate significant correlation was observed between “Intrusiveness” and the overview item (r _s = −0.44, P < 0.001). Table 4 displays correlations between the GME-Q (sub)scale scores as well as between (sub)scale scores and the single item “overview item” and various self-reported demographic and clinical characteristics. Hypothesized convergent validity was evidenced in the observed moderate-to-large correlations among GME-Q subscales (r _s range = ±0.37–0.56), and the expected weak associations between GME-Q scores and frequency of “mild” hypoglycemia events in the past week (r _s range = ±0.17–0.26). No significant relationships between GME-Q scores and frequency of finger-prick glucose monitoring were observed. Hypothesized discriminant validity was evidenced by the weak (r _s ≤ ±0.3) associations observed with age and diabetes duration.

Table 4.

Correlations Among Glucose Monitoring Experiences Questionnaire Scores and with Self-Reported Demographic and Clinical Characteristics

	GME-Q intrusiveness	GME-Q convenience	GME-Q total satisfaction	GME-Q overview item	SMBG daily frequency	Mild hypo. in past week	Age	Diabetes duration
GME-Q effectiveness	−0.37^***	0.56^***	0.76^***	0.60^***	0.08	−0.25^***	0.23^***	0.12^**
GME-Q intrusiveness		−0.52^***	−0.78^***	−0.44^***	−0.01	0.17^***	−0.17^***	−0.06
GME-Q convenience			0.85^***	0.66^***	−0.04	−0.24^***	0.16^**	0.10^*
GME-Q total satisfaction				0.70^***	0.00	−0.26^***	0.25^***	0.13^**

P < 0.05, ^** P < 0.01, ^*** P < 0.001.

SMBG, self-monitoring blood glucose.

Content validity: qualitative responses

In total, 522 (89%) participants provided 1248 qualitative responses about the best aspects of their current method of monitoring, and 487 (82%) participants provided 1156 qualitative responses about the worst aspects (Supplementary Table S2). Most of the issues raised most frequently were generally reflected well in the GME-Q. Consideration could be given in future to whether certain issues warrant revision of existing items or addition of new items, for example, connectivity with other devices, including smartphones (n = 112; 9.0%); supporting insulin/dietary calculations (n = 79 mentions 6.3%); and expense (n = 72; 6.2%).

Discussion

This study demonstrates that the GME-Q is an acceptable, valid, and reliable measure of satisfaction with glucose monitoring in adults with type 1 diabetes. It comprises three brief scales (“Effectiveness,” “Intrusiveness,” and “Convenience”), which can be used individually or combined to form a composite scale: “Total satisfaction.” In addition, a single overview item reflects overall satisfaction, and may be useful in situations where a very brief measure is necessary.

Satisfaction (with a treatment or device) is an important secondary endpoint in clinical trials due to the insight it offers about the person's experience, and its recognized association with an individual's willingness to continue using that intervention. Furthermore, given that device manufacturers and regulatory authorities are inherently interested in demonstrating the value of a given product, satisfaction is a very useful endpoint. However, the concept of satisfaction is abstract and, like any other patient-reported outcome, its assessment needs to be rigorous. This applies not only to the demonstration of psychometric properties but, fundamentally, to the design of the measure. The qualitative research underpinning the GME-Qs design demonstrates that it has satisfactory face and content validity as a measure of satisfaction with monitoring devices.

The face and content validity of a measure relies heavily on the use of a strong conceptual model/framework. In the past decade, in parallel with rapid technological advances in technologies, several new measures have been developed, each claiming to focus explicitly on satisfaction with glucose monitoring.^9,11,20 However, none of these measures has a strong conceptual model or framework underpinning the questionnaire design, and they typically invite a broader “psychosocial rating” of the monitoring system. For example, the 15-item Glucose Monitoring System Satisfaction survey includes four subscales focused on quality of life (assessing the positive/negative impact of the monitoring method) and health beliefs (perceived value of and trust in monitoring), with only one subscale (four items) focused on the hassle of the monitoring method. All these issues are relevant and valuable when assessing the experience of day-to-day glucose monitoring. However, they do not assess “satisfaction” per se, which is the value placed by the individual on the intersection between outcomes achieved, and the process/experience of the intervention. In the design of the GME-Q, we operationalized this as the perceived “effectiveness,” “intrusiveness,” and “convenience” of the glucose monitoring method. The clear advantage of this is that any given monitoring system can be rated against another and, for example, found to be equivalent in terms of perceived effectiveness, but less intrusive and more convenient or vice versa. Future research comparing satisfaction with different devices may enable assessment known-groups validity; while research investigating satisfaction following change in monitoring device or education/support may enable assessment of responsiveness to change.

The acceptability of the GME-Q is evident from the extremely low number of missing responses to the questionnaire items. The iterative approach to determining the structural validity of the questionnaire yielded a 22-item questionnaire, with satisfactory internal consistency reliability for all subscales and the “total satisfaction” composite scale. The Cronbach's alpha for the 22-item scale indicates that it is reliable but with little redundancy, showing that all items make an important contribution to the scale.

The correlational analyses demonstrate that the GME-Q has satisfactory convergent validity through moderate-to-large correlations among subscales and the hypothesized weak but significant correlation with frequency of self-treated hypoglycemia. No significant relationship with finger-prick monitoring frequency was seen. Given that participants reported an average of six tests per day, it may be that these self-selecting participants are particularly committed to monitoring their diabetes regardless of their satisfaction with the monitoring technology. It is also possible that the self-report of glucose checking is subject to social desirability bias. Finally, it is likely that cross-sectional data are not optimal for assessing this relationship, and that the GME-Q would be a stronger predictor of change in monitoring behaviors over time. This discordance underlines the importance of assessing satisfaction, and not assuming high frequency of monitoring to be evidence of satisfaction with monitoring. Further research is needed.

The strengths of Study 1 include the use of both exploratory and debriefing interviews and the development of a conceptual model of satisfaction to underpin the design of the questionnaire. In the design phase, a relatively large sample of participants was selected purposefully for their experience of both SMBG and CGM, enabling them to consider the issues related to both monitoring modalities and for the design of the questionnaire to be suitable for completion by someone using either modality. Furthermore, Study 2 provided opportunity to confirm content validity of the final GME-Q items against the perceived best/worst aspects of participants' current glucose monitoring device in a large sample, as well as highlight areas for future scale extension or revision. Study 2 included a large sample of respondents providing highly satisfactory sample sizes for the psychometric analyses.

Both studies are limited by the relative homogeneity of the samples (English-speaking middle-aged adults with long-standing type 1 diabetes), with around half using an insulin pump, which is overrepresentative of the proportion accessing this technology in both the U.K. and Australia. Although the sample for Study 1 was relatively small and homogeneous, and we were satisfied that data saturation occurred, it remains the case that the experiences of other demographic and cultural groups need to be explored to assess content validity in those populations. There are four key limitations of Study 2. First, psychometric validation was limited to those using SMBG, so further research is needed to validate the measure with those using CGM. Second, no other validated measures of satisfaction or experience were included in the study, nor objective biomedical data, so there remains a need to examine the convergent and divergent validity of the GME-Q against validated measures and clinical data. The sample was recruited online as part of a national survey about glucose monitoring, which may have led to a self-selection bias. Finally, participants self-reported checking their glucose levels with a finger-prick, on average, six times per day. Without verifiable, objective data, we cannot be sure of this and, as noted above, the data may be subject to social desirability bias.

Further research with the GME-Q is needed to add to the evidence accumulated in the current study. Additional evidence of the convergent and discriminant validity of the GME-Q is needed, particularly using validated measures of satisfaction and other psychological constructs. Preliminary evidence of the GME-Qs responsiveness has been demonstrated elsewhere, reported in the HypoCOMPaSS study, which included participants using SMBG alone and adjuvant CGM.²¹ In addition, it will be important to examine predictive validity, that is, its ability to determine who continues with, and is successful with, a particular monitoring approach. Furthermore, the psychometric properties of the GME-Q need to be examined in other populations, for example, among adults with type 1 diabetes using CGM, and among adults with type 2 diabetes. Finally, the GME-Q has been assessed in isolation and comparisons of its performance relative to other measures of satisfaction with monitoring devices would be useful.

Notwithstanding these limitations and the need for further research, this study has implications for the ongoing development of monitoring technologies and for clinical practice. The overview item has the strongest correlation with the total scale score (r _s = 0.70), while Convenience is the domain most closely associated with both the total score (r _s = 0.85) and the overview item (r _s = 0.66). This suggests that, while effectiveness and lack of intrusiveness are important, convenience appears to be a key factor driving satisfaction with a monitoring device, which makes sense given how many times a day it needs to be used. This has clear implications for industry to involve users early in the development of new products, and to ensure that satisfaction is included as an outcome in efficacy and real-world implementation trials. In terms of the implications for clinical practice, the full questionnaire may be too long for routine use. However, its intermittent use (potentially around the time of a change in glucose monitoring modality, such as initiation of CGM) may provide useful insights that can gathered more efficiently and quantitatively in this way than through a clinical interview. The strong correlations of the domain and total scores with the single overview item suggest that the latter may have great utility in routine clinical practice to enable holistic diabetes care.

This study has described the development and psychometric validation of a 22-item measure of satisfaction with modern glucose monitoring devices. Our findings indicate that the GME-Q is conceptually valid (based on the experiences of people with diabetes), psychometrically valid, acceptable, and reliable. It is now suitable for inclusion in other studies, which will be able to add to this evidence. Given the importance of satisfaction as an indicator of willingness to continue optimal use of a monitoring device, there is considerable scope for the GME-Q to make an important contribution in future studies and clinical trials.

Footnotes

Acknowledgments

We thank the people with type 1 diabetes who participated in this study. We also acknowledge the following people for their contributions to this study: Dr. Martin K. Rutter (Manchester), for recruitment of participants to Study 1; Dr. Shalleen M. Barendse (formerly of AHP Research), for data collection and preliminary analysis of Study 1; Dr. Steven Trawley (Cairnmillar Institute) for his support of the YourSAY: Glucose monitoring survey development and data management. The research was supported by the National Institute for Health Research Newcastle Biomedical Research Centre.

Authors' Contributions

J.S., S.A.L., and J.A.M.S. conceived the idea for designing the GME-Q as part of their work on the HypoCOMPaSS study. J.S. collected and analyzed data in Study 1 and designed the GME-Q; H.S., S.L., and J.A.M.S. reviewed Study 1 findings iteratively and consulted on the design of the GME-Q. J.S. and E.H.-T. conceived the idea for the YourSAY: Glucose Monitoring study, designed and oversaw the study. E.H.-T. and J.S. developed Study 2, the statistical analysis plan, and EHT conducted data cleaning and analysis. J.S., E.H.-T., and J.A.M.S. provided input into Study 2 results interpretation and item reduction process. J.A.M.S. and E.H.-T. developed the first draft of the article. All authors contributed to article revisions and approved the final version.

Author Disclosure Statement

J.S. is a director of AHP Research Ltd., which owns the copyright of the GME-Q. J.S. has participated in advisory boards for Medtronic; her research center (ACBRD) has received honoraria in respect of these activities, as well as unrestricted educational grants from Abbott Diabetes Care; her research has received in-kind support (products and consumables) for studies from Abbott Diabetes Care, Medtronic, and Roche Diabetes Care; speaker fees from Roche Diabetes Care; sponsorship to attend educational meetings; and consultancy income from Roche Diabetes Care. E.H.-T. has received unrestricted educational grants from Abbott Diabetes Care; her research has received in-kind support (products and consumables) for studies from Abbott Diabetes Care and Roche Diabetes Care. J.A.M.S. has previously participated in advisory boards for Medtronic. No other potential conflicts of interest relevant to this article were reported.

Funding Information

The design of the GME-Q (Study 1) was funded by Diabetes U.K. (07/0003556) as part of the HypoCOMPaSS study. The data collection for the psychometric validation (Study 2) was funded by Abbott Diabetes Care as part of the YourSAY: Glucose Monitoring study, but Abbott had no role in these analyses or the preparation of the current article. The psychometric validation work (Study 2) and preparation of this article were funded by Diabetes Victoria and Deakin University through the core funding provided to the Australian Centre for Behavioural Research in Diabetes.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

References

American Diabetes Association: Diabetes Technology: Standards of Medical Care in Diabetes-2019. Diabetes Care, 2019; 42:S71–S80.

Miller

, Beck

, Bergenstal

, et al.: Evidence of a strong association between frequency of self-monitoring of blood glucose and hemoglobin A1c levels in T1D exchange clinic registry participants. Diabetes Care, 2013; 36:2009–2014.

Nathan

, Genuth

, Lachin

, et al.: The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med, 1993; 329:977–986.

Olczuk

, Priefer

: A history of continuous glucose monitors (CGMs) in self-monitoring of diabetes mellitus. Diabetes Metab Syndr, 2018; 12:181–187.

Shikiar

, Rentz

: Satisfaction with medication: an overview of conceptual, methodologic, and regulatory issues. Value Health, 2004; 7:204–215.

Doyle

, Lennox

, Bell

: A systematic review of evidence of the links between patient experience and clinical safety and effectiveness. BMJ Open, 2013; 3:e001570.

Speight

: Assessing patient satisfaction: concepts, applications, and measurement. Value Health, 2005; 8(Suppl 1):S6–S8.

Bradley

: The Diabetes Treatment Satisfaction Questionnaire: DTSQ. In: Bradley

, ed. Handbook of Psychology and Diabetes: A Guide to Psychological Measurement in Diabetes Research and Practice. Chur: Switzerland Harwood Academic, 1994.

Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group: Validation of measures of satisfaction with and impact of continuous and conventional glucose monitoring. Diabetes Technol Ther, 2010; 12:679–684.

10.

Polonsky

, Fisher

, Hessler

, Edelman

: Development of a new measure for assessing glucose monitoring device-related treatment satisfaction and quality of life. Diabetes Technol Ther, 2015; 17:657–663.

11.

Peyrot

, Rubin

: Patient-reported outcomes for an integrated real-time continuous glucose monitoring/insulin pump system. Diabetes Technol Ther, 2009; 11:57–62.

12.

Boyatzis

: Transforming Qualitative Information: Thematic Analysis and Code Development: Sage Publications, Inc., Cleveland, Ohio, USA. 1998.

13.

Atkinson

, Kumar

, Cappelleri

, Hass

: Hierarchical construct validity of the Treatment Satisfaction Questionnaire for Medication (TSQM Version II) among Outpatient Pharmacy Consumers. Value Health, 2005; 8:S9–S24.

14.

Willis

: Cognitive Interviewing: A Tool for Improving Questionnaire Design. Sage Publications, California, USA, 2004.

15.

Streiner

, Norman

, Cairney

: Health Measurement Scales: A Practical Guide to Their Development and Use. USA: Oxford University Press, 2015.

16.

Hutcheson

, Sofroniou

: The Multivariate Social Scientist: Introductory Statistics Using Generalized Linear Models. Sage, Wiltshire, UK. 1999.

17.

Kline

: An Easy Guide to Factor Analysis. Routledge, NY, USA, 1994.

18.

Kline

: The Handbook of Psychological Testing, 2nd ed. Florence, KY: Taylor & Frances/Routledge, 1994.

19.

Cohen

: A power primer. Psychol Bull, 1992; 112:155–159.

20.

Polonsky

, Hessler

: What are the quality of life-related benefits and losses associated with real-time continuous glucose monitoring? A survey of current users. Diabetes Technol Ther, 2013; 15:295–301.

21.

Speight

, Holmes-Truscott

, Little

, et al.: Satisfaction with the use of different technologies for insulin delivery and glucose monitoring among adults with long-standing type 1 diabetes and problematic hypoglycaemia: 2-year follow-up in the HypoCOMPaSS randomised clinical trial. Diabetes Technol Ther, 2019. [Epub ahead of print]; doi: 10.1089/dia.2019.0152.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB