“It Is Definitely a Game Changer”: A Qualitative Study of Experiences with In-home Overnight Closed-Loop Technology Among Adults with Type 1 Diabetes

Study design

This qualitative study was part of a randomized controlled crossover trial comparing overnight in-home use of a hybrid closed-loop insulin delivery system (closed-loop) with that of a sensor-augmented pump incorporating a low glucose suspend function (open-loop). Details of the clinical trial protocol and outcomes have been published elsewhere. ⁷ St. Vincent's Hospital Human Research Ethics Committee (HREC-D120/13) and Deakin Human Research Ethics Committee (DUHREC2013-326) approved the study. All participants provided written informed consent.

The prototype closed-loop system comprised an insulin pump (Veo™, Medtronic, Northridge, CA), a glucose sensor, and a transmitter (Enlite™, Medtronic) linking the pump with the programmable smart phone (controller). One week before using the devices in-home, the participants stayed overnight in the clinical trial center for training on the closed-loop or open-loop configuration. They were randomly assigned, first to four nights' closed-loop or open-loop in-home, and then crossed over to the alternate study arm.

Participants were eligible if they were aged ≥24 years with type 1 diabetes for >1 year and using insulin pump therapy for >3 months. The 24-year age cut-off was incorporated into the study design with the intention to minimize the influence of persisting adolescent behavior on their interaction with the technology. ^8,9

At home, participants activated the closed-loop from 2 h after dinner until pre-breakfast the next morning. Participants had to undertake an activation process (typically taking 20 min), navigating through several steps, including a glucose calibration. If the system could not be activated, it would return to open-loop and the process needed to be repeated. It was recommended to keep the controller and transmitter connected overnight to the battery charger.

There were no restrictions regarding food or physical activity. In both open- and closed-loop study phases, all participants were instructed to drink 300 mL of orange juice between 02:00 am and 03:00 am on study night four only to challenge the system. Overnight, participants were monitored remotely in real-time by trial clinicians and a technology team. A 24-h helpline was available. During the four nights of closed-loop at home, participants were required to have an adult with knowledge of diabetes and able to administer glucagon present.

Interviews

A semi-structured interview schedule was developed informed by the expertise of a clinical psychologist (C.H.) and a health psychologist (J.S.), both with long-standing experience in type 1 diabetes, and with reference to the literature. Questions focused on participants' experiences of using the closed-loop system, including its perceived effect on glucose levels, sleep, and day-time functioning; comparison with their usual treatment; their understanding of the control-algorithm; difficulties/problems with technology; their confidence in and concerns about closed-loop use; beliefs about technology controlling their diabetes; their views on future closed-loop use, their expectations of its impact on their diabetes care and daily life, and whether they would recommend closed-loop to other people with type 1 diabetes.

Interviews were conducted in a private room in the clinical trial center on the morning after completing the closed-loop phase, with one exception where a telephone interview was conducted 3 days postcompletion. All were conducted by a psychologist (C.H. or J.S.) with no involvement in the participants' clinical care, and lasted on average 45 min (range 23–68 min). Participants were advised that they could skip questions, refuse to answer, or terminate the interview at any time. Participants had no difficulties understanding the questions.

Transcription and analyses

Interviews were audio recorded and transcribed verbatim by a professional service, with each transcript checked and corrected as necessary (by L.A.P. and C.H.). Deidentified transcriptions were imported into NVivo 10 (QSR). Inductive thematic analysis was conducted to make sense of the qualitative data corpus. After familiarization with the content by listening to audio recordings and repeated reading of transcripts, initial codings, and main themes were identified based on two transcripts, analyzed independently by two authors (L.A.P. and C.H.). The two sets of codings were compared and any differences discussed and resolved, with input from J.S., before producing a final structure for the thematic analysis. Transcript excerpts were then indexed in NVivo (by L.A.P.) and, in the final stage, reviewed by C.H. who confirmed no further themes emerging.

Participant characteristics

All 16 participants were interviewed. Their characteristics are described elsewhere. ⁷ In brief, their age was 42 ± 10 (mean ± SD) years; nine were women. Ten had a university degree and 14 were employed. All had long-standing type 1 diabetes (duration 27 ± 7 years) and HbA1c was 56 ± 4 mmol/mol. All were using an insulin pump and had CGM experience; three used CGM routinely, whereas others had used CGM in previous studies, or intermittently to optimize management, during travel or sports training. As previously reported, diabetes distress and fear of hypoglycemia scores were very low, whereas diabetes treatment satisfaction scores were very high. ⁷

Four main themes were identified: Use and understanding of the closed-loop system; trusting the technology; perceived impact on diabetes outcomes, sleep and daily functioning; and future with closed-loop technology.

Use and understanding of the closed-loop system

Overall, participants found the system easy to use. They remarked that their routine insulin pump use and previous experience with CGM were advantageous in preparing them for closed-loop:

I think that I bring a number of years of experience with this. Yes, this is a big leap as far as technology goes, but I've been on the journey of it being developed. #1

They had a good understanding of the underlying algorithm: they observed how closed-loop dealt with higher glucose levels post-dinner and reduced insulin delivery when their glucose levels were trending down:

I think I had a good understanding of what was going on and why it was deciding to deliver zero units per hour and other times it was delivering much more. #16

While most reported that the closed-loop performed as they expected, one participant had a different experience:

As soon as I closed it down at 4.30 this morning my blood sugar made it back to normal … If my blood sugar was 14 it was giving me potentially less insulin with what I would normally have if my blood sugar was 6 or 7 overnight. I just don't think it was giving me enough [insulin] at all. #8

Participants evaluated the graphics on the smartphone screen as very informative, providing useful feedback. They did not feel overwhelmed with the amount of information available to them and were fascinated being able to follow the “whole metabolic process” on their smartphone:

The screen layout on the phone is amazing. I love that ability to review, increase, or decrease timeframes. You can pinch time on the phone so you can see exactly what the flow is now or what it was like over the past six, eight, 10, 12, hours… You can drag along your glucose reading to see exactly what the reading was at a particular time. #4

Realizing that there was no need to respond to these data was a new and positive experience:

I didn't feel that I needed to do anything with the information. That's actually a really good point because with the pump you feel you have to do stuff with the information “okay my finger prick is ‘x’ amount, I need to program that into the pump because I really need a correction bolus”. This was just doing it for you so it was actually less onerous on that side. #1

Some indicated that they checked the screen more often than they needed, out of curiosity, rather than concern about reliability:

I probably did check [the smartphone screen] a fair bit, but probably more just out of curiosity to see what it was doing. Also, just to look at the changes in the rates that it was giving. #13

Reflecting on the algorithm highlighted how different the experience of closed-loop was from their insulin pump, for example, the ability to predict the trend, the micro amounts of insulin delivered, the more gradual response to glucose levels:

I have got to have faith in there that the algorithm is also set up to following the trend rather than just the single digit. #3

I was quite fascinated by the fact that it would give you constant little tiny bits … #12

[The algorithm] is managing it and calculating and doing all of that matter rather than relying on my imperfect science.#1

Another observation was the proactive nature of managing diabetes with the closed-loop, compared with their usual treatment (for most, relying more on fragmented information from finger-prick monitoring) and carbohydrate counting:

[The closed-loop] was always adjusting whether it was high or low, the algorithm knew “this could be a low in three hours” … The predictive and constant monitoring, the constant checking, that algorithm is always there, which is everything we don't have now with open-loop… It picks up before it becomes a problem. It is just streets ahead. #10

Although some perceived the alarms as annoying, others reported that there were fewer alarms than with their usual care, because their glucose levels were more often within target range:

Because the closed-loop keeps my blood sugar in a nice steady sort of range, I don't get all of the alerts that I tend to get with the pump and CGM … So that's a lot better, … because again they're things that wake me up or wake my husband up. #1

Most participants also reported disadvantages or technical problems with the closed-loop system, including: inconvenience of multiple devices and transmission problems, especially when moving around the house and turning in bed; closed-loop system disengaging, often for reasons that were not apparent to the participant; being unable to activate the closed-loop function; ongoing alarms despite addressing low glucose levels; sensor inaccuracy, sometimes resulting in unnecessary alarms; inability to calibrate the sensor; algorithm not responding as expected.

Some of these issues resulted in sleep disturbances and sleep deprivation, leading to tiredness throughout the day:

It's been quite exhausting with multiple wake-ups overnight, either from malfunctions with the closed-loop system, not life-threatening, but minor malfunctions… So it's really just sleep deprivation-wise that's been difficult. #2

Trusting the technology

Participants felt generally confident that the technology was doing what it was meant to do. Their confidence in the system strengthened over the four nights, as they had more opportunities to observe how the closed-loop worked:

At one stage, I thought it would give me a bit more insulin than that, but it was taking something else into account and it got it right. So once I'd watched a couple of those I thought “yes, this is doing a brilliant job”. #7

One participant described losing confidence after a problematic night:

Things just didn't seem to work and I couldn't really understand what was going on as the closed-loop was not reducing my blood sugar at a rate that I thought it probably could have… and then I ended up crashing with an even more severe low after I stopped the closed-loop that morning. #16

Two participants perceived the closed-loop to be “inaccurate”. On two nights, the insulin delivery reverted automatically to “safe basal” because the translator was out of range. They were concerned about their glucose levels falling too low and not being identified by the system, despite the fact that the default basal insulin delivery was less than what they would have received with their open-loop settings.

Having hypoglycemic awareness and being monitored remotely by health professionals provided reassurance regarding their safety:

I felt safe, I knew they were all up there watching and I have hypo awareness as well. So if the sensor was wrong and I was going low without the remote monitoring showing that, I felt comfortable that I would know myself and be able to check and treat. #2

[I was] not concerned at all because I knew they were in control … I handed my diabetes over to somebody for the first time in my life and I had no concerns because they know what they are doing. #15

Sensor accuracy was perceived as crucial for a reliable closed-loop system. Experiences varied with some participants reporting calibration issues or mismatch between their blood glucose meter and the sensor, and others reporting the sensor to be very accurate:

Yesterday, it was saying my blood sugar was three point something and it was 11. #8

95% of them were pretty close. I mean occasionally they just get a little bit out and that's probably the only thing. #11

It's all about the sensor being accurate and, up until last night, I thought it was really pretty accurate, because I was pricking my finger and it was pretty well spot-on. #7

On the final night, the latter participant experienced inconsistencies:

It was reading that my blood glucose was 16. I'm pricking my finger and it was 13 and it didn't like the fact that it was so far apart. #7

One participant commented on the delay:

The readings it's getting from the sensor are actually about 20 minute delayed from your finger prick readings. When I see 5.3 it's like, well, is it going down or going up? Where is that 20 minute delay? What is the effect of that? So am I actually at 4.1 and it thinks I'm at 5.3? #3

Feeling confident with the system made it possible for participants to hand over control of their diabetes management to the closed-loop system without feeling they were giving up personal control. They felt their diabetes to be “under control,” observing that the system was doing a better job than they could:

The technology had the control, but… I'm in charge of the technology. I've decided whether I'm using it or not, so ultimately I have the control in saying “yes, I'm using that technology”. #3

Provided the sensor is working properly, it's kind of nice to not have to think about it too much, and let the system take care of it, which it did… When it's working well, I'm quite happy to let it take over a little bit of it. #13

For some, the four nights with the closed-loop was like “taking a break” from their diabetes:

It's been nice to have ‘set and forget’ for a couple of nights. #1

Had my brain just said “I am handing over to the technology, I don't need to worry about you?.”.. I think I'd switched off my self-survival stuff. #3

Perceived impact of closed-loop on diabetes outcomes, sleep and daily functioning

When asked about the effect of closed-loop on their glucose levels, many were impressed by the “flat line” they observed. With a well-functioning system for most, the glucose levels stayed within recommended range:

… just that stability, the flat lining of it was unbelievable. #10

The nights where we did not challenge the system [drinking orange juice], I sat in the range the whole time, which was extraordinary. #1

For one participant, stable levels were not in the middle of the target zone, for another they were at the higher end:

[On] Wednesday night it got down to 9 and it just stayed there, so I would have liked it to have come down to 6 or 7. #6

It kept me within, I suppose, its upper and its lower barriers, but it didn't keep me in the middle of those barriers. It just went “okay, you're in the barriers, let's keep you on that level now”. #4

For some, the observed glucose levels were not particularly different from, or more stable than with, their usual treatment:

I don't necessarily believe that they were anywhere much better than what I've been achieving through my usual control. #16

Probably only really about the same I think because I've been using CGM like quite a lot and because that has a large part to do with it. #5

More stable overnight glucose levels resulted in improved glucose levels the following day:

Even during the day afterwards … [I] got up and went to work and I'd never seen it so stable, like from breakfast… up until 6 o'clock, it was between 5 and 7 all day and normally … I would never get it that good. #10

When the closed-loop system was functioning well, some participants reported fewer hypoglycemic episodes than usual, resulting in “peace of mind”:

The four days I was on the closed-loop I didn't hypo which was great… I was running around the house erratically and didn't hypo and usually I would. So I was very impressed by that. #9

It gave me great peace of mind and confidence that I wasn't likely to drop low without realizing because it had the low glucose alarm … I found that a huge benefit. #12

Overnight knowing that you are not going to go low, once you are comfortable with it, it is actually really good. It is something you don't have to think about. #11

One person made the observation that their frequency of hypoglycemia had reduced in the run-in period, which was attributed to having used the CGM before the closed-loop.

Overnight disruptions were largely due to problems maintaining connectivity between the various technological components or due to the demands of the study protocol. Such disruptions limited participants' ability to evaluate the full impact of closed-loop on their sleep quality. However, some slept better, reassured that the closed-loop was managing their diabetes, and they were being monitored remotely:

If anything, [I was] perhaps a bit more relaxed than normal, because I knew that if there was any problem, it would let me know so I did not have to worry. #12

Participants commented about waking up much fresher and more alert, feeling better during the day because of consistent overnight glucose levels, feeling better emotionally or during sport.

The challenge with the orange juice was a positive experience with a well-functioning device: the closed-loop responded as expected.

Future with closed-loop technology

Participants did not anticipate major lifestyle changes should closed-loop become part of their routine diabetes management. However, they perceived their diabetes would require less attention and they would have more stable and within-target glucose levels. So, they predicted that life would become much easier and more flexible. They expected the closed-loop to be better able to manage their glucose levels when stressed, ill, or emotional; with impromptu physical exertion (e.g., running for a train); while driving (with no finger prick needed before or during travel).

Participants believed some aspects and limitations of diabetes management would not change with the closed-loop: for example, detaching device(s) for physical activities and showering; needing to remain vigilant in the event of technological failure (e.g., carrying a glucose meter, insulin pen, and hypoglycemia treatment); the need to deal with unusual activities (e.g., long-distance travel). Furthermore, one participant anticipated wanting to take “breaks” from the technology from time to time.

Participants did not foresee major differences between wearing an insulin pump or closed-loop routinely. However, there was consensus that “smaller is better” when it comes to the size of future closed-loop devices:

I just wonder, even though the closed-loop is set up on the Android, whether that can be incorporated into the pump system itself in some way, shape, or form. #5

I'm just imagining it will be fairly, not too much, different to wearing the pump because I would have thought they'd incorporate it into a pump. #7

Participants expressed various opinions about how people with type 1 diabetes without previous experience of advanced technologies would cope with closed-loop. Some thought it would be too big a step from insulin injections to closed-loop, whereas others considered it a matter of personal preference and feeling reassured regarding safety:

That would be hard I think. I think you'd almost have to be on a pump because otherwise you'd be trying to educate people on how to use a pump and then you go “well, we're going to add this other thing in and now the pump's running itself” and I think that might be a bridge too far for some people. #6

I know a lot of people simply wouldn't contemplate it… there's people who still won't have a pump and people who try a pump and aren't prepared to persevere with it and just say “no, I just want to live with injections”. #3

I think a lot of people just want [to] trust that the device that they're getting isn't going to kill them, that it's doing what it's supposed to do. So, for people who aren't really tech savvy they will just want assurance, that the technology has been tested and it works well; possibly even little stories, from early users saying how well it has worked for them. #4

Most would recommend other people with type 1 diabetes to take part in similar trials, for personal benefit/learnings, or to contribute altruistically to research:

It's for research and, to get to improvements, everyone needs to occasionally throw their support behind these things. #2

I would tell them that it's going to be a little bit time consuming and something that you need to focus on a bit. But there's real benefits I feel. I mean, despite the fact that I sort of needed to drop out of a few things, it still made my life easier whilst I was on this and had all the sensors and everything else. During the day, it made my life much easier, so there's real benefits there. So, yes, I would definitely recommend anybody to go on it. #12

One person would only recommend it under particular circumstances:

If somebody was having a lot of overnight hypos then maybe, yeah, but if they're going fine then you'd kind of say “why bother?” #8