Qualitative Study of User Experiences with Loop,an Open-Source Automated Insulin Delivery System

Abstract

Background:

Loop is an open-source automated insulin delivery (AID) system, used by more than 9,000 people with type 1 diabetes. Understanding the pros and cons of Loop use may help improve disease management and support population level innovation.

Methods:

Focus groups revealed 72 new and existing users' perspectives on Loop uptake, use, and persistence. A subsample of participants from a mixed-methods, observational cohort study shared first-hand accounts of their experiences using Loop. Participants were predominately white (95%), male (50%), privately insured (94%), and reported annual household income ≥$100K (73%) and education exceeding a bachelor's degree (87%) with a mean HbA1c of 6.6% ± 0.8%. Data were analyzed and synthesized by a multidisciplinary team.

Results:

Participants detailed their experiences with (1) Loop technical support and troubleshooting, (2) decreased mental/behavioral burden, (3) technical issues with parts of the system, (4) glycemic control, (5) personalizing settings, and (6) providers while using Loop. Decreased burden was the most endorsed benefit defined by less worry, stress, and cognitive effort and less time spent on diabetes management tasks. Participants highlighted the benefits of Loop overnight and their introduction to “Loop communities” during use. The most discussed challenges involved technical issues. A range of provider attitudes and knowledge about Loop complicated users' clinical experiences and disclosure.

Conclusions:

This sample of new and experienced Loop users reported benefits to quality of life and glycemic control that outweighed challenges of setting up system components, customizing the system to suit one's lifestyle and habits, and adjusting system settings. Challenges related to system setup and calibrating settings are remediable and, if addressed, may better serve Loop users. Users reported feeling empowered by the customizability of and the educational effects facilitated by the open-source AID system. Loop helped users learn more about their chronic illness and physiology in an acceptable format.

Clinical Trial Registration number: NCT03838900.

Introduction

Open-source, automated insulin delivery (AID) systems were developed as part of the “Do-It-Yourself” (DIY) movement led by people with diabetes to ease burden and increase safety of living with Type 1 diabetes mellitus. Loop, initially developed by Nate Racklyeft, is among the more prominent open-source AID systems, used by more than 9,000 people with Type 1 diabetes.^1,* The Loop app is designed to allow users to enter and modify carbohydrate content to determine and deliver a bolus, create presets for situations like exercise or illness, and connect with a continuous glucose monitor (CGM) to predict and modify (increase, decrease, or suspend) insulin delivery.¹ Different from commercial AID systems, Loop users must enter carbohydrate data by estimating carbohydrate consumption and confirming system-recommended dosages before insulin delivery. Loop has been described extensively elsewhere.¹

Upon this study's commencement, Loop had not received regulatory approval by the U.S. Food and Drug Administration (FDA) and no formal research had been conducted to evaluate Loop's efficacy, safety, or reported experiences and outcomes. An observational, mixed-methods, longitudinal study was undertaken to assess the experiences of those using Loop to inform future efforts of regulators, providers, and those interested in open-source AID.¹

This study employed qualitative methods to assess multidimensional aspects of Loop user experiences in the management of type 1 diabetes mellitus. Specifically, qualitative methodology provides a platform to ask open-ended questions of this unique cohort of individuals to learn from their real-world experiences and identify the barriers and facilitators to using these open-source AID systems. Users' perspectives on the use of cutting-edge technology and the processes involved in setup, use, and maintenance are imperative to implementing effective, patient-centered care as patients increasingly take their health into their own hands.

Methods

As part of a larger, observational study of 898 people with type 1 diabetes using Loop, participants were recruited for longitudinal data collection to learn more about their experiences and outcomes. The larger study has been described by Lum et al.¹ and the study protocol is summarized on clinicaltrials.gov. Study participants used Loop with both Medtronic and Omnipod insulin pumps and Medtronic and Dexcom CGMs. Quantitative data collection included measures of psychosocial characteristics and treatment satisfaction, as well as clinical measures and device data by electronic survey questionnaires at baseline, after 6 months, and after 12 months of study participation by Loop system users and/or their caregivers. New Loop system users also completed a 3-month survey. In all, 558 new Loop users and 340 people who had used Loop for at least 9 months participated in the research project.

All enrolling study participants received information about opportunities to participate in electronic focus groups designed to complement quantitative data collection and to obtain rich detail about user experiences. Focus groups are an effective methodology for exploring shared experiences and sensitive topics.^2,3 Upon expressing interest in focus group participation, potential participants were telephoned or emailed to schedule their participation. We aimed to recruit 50 new and 50 experienced Loop users; however, recruitment ended upon theoretical saturation, when no new concepts emerged during qualitative data collection. Qualitative research participants represented three groups: new users, experienced users, and discontinuers, and included both adult users and parents of child users.

Most of the qualitative sample contributed to focus groups; 19 discontinuers participated in semistructured interviews about their experiences with and reasons for discontinuing Loop (reported elsewhere).⁴ Open-ended questions explored experiences with and reactions to Loop, quality-of-life impacts of using Loop, and procedures and activities during continued use. Focus group data from 72 qualitative research participants are summarized in this study.

Ethical considerations

The study protocol was reviewed by the IRB of the Jaeb Center for Health Research (JCHR) and conducted in compliance with standards of Good Clinical Practice (GCP). Informed consent and authorization to release personal information were obtained electronically from participants or parents/legal guardians. Inclusion criteria for study eligibility included the following: diagnosed with Type 1 diabetes; currently have or have ordered the hardware devices necessary to use Loop; currently use Loop or have plans to start using Loop for insulin delivery; willing and able to provide informed consent and complete questionnaires that are part of the protocol; and resident of the United States. Participants received minor honoraria ($60 or less) for research participation.

Data collection

Qualitative data were collected remotely using Zoom meetings and recorded. Table 1 presents examples of questions covered during baseline and follow-up focus group discussions. Focus groups included two to five participants (or parents/caregivers). Trained study coordinators observed the focus groups and took field notes. Audio recordings were transcribed and de-identified by Medikin. Transcripts were uploaded into NVivo, version 12, and classified based on participant characteristics of study cohort, parent vs. adult user, and user/discontinuer status.

Table 1.

Sample Items from Electronic Focus Group Discussion Guides

Sample electronic focus group items
Items for new users Tell us about your decision to use Loop. What were your expectations for Loop? Describe the process of starting on Loop and accessing everything needed. How has Loop impacted your day-to-day life? When you have an issue or problem, how do you get help? Describe how your doctor/medical team works with you now that you (or your child) are on Loop.
Items for experienced users and new users at follow-up What were your expectations for Loop? In which ways has Loop met/exceeded/not met expectations? When do you think Loop is most useful? Least useful? While Looping, did the time you spend doing diabetes-related tasks change? Has anything happened while using Loop that surprised you, which was unexpected (both positive and negative)?

Data analysis and synthesis

Data were analyzed using inductive thematic analysis and a structured process. Analyses interpreted users' experiences as their assessments of their worlds; however, researchers acknowledge that user reports depend on context. Each transcript was coded by two members of the research team. The original codebook was developed in a previous study of closed-loop system users and modified during weekly team meetings. Weekly meetings were used to discuss transcripts, reconcile problematic codes, review tricky coding examples, and add codes to help address conceptual gaps. The principal investigator used the codebook to ascertain when saturation was met and no new theme was being discussed.

The final codebook encompassed 51 codes organized by 10 coding categories. For example, “decreased mental or behavioral burden” is one code in the “Benefits of Loop” category. Fifty-three transcripts were coded by paragraph, with only the three most salient codes retained per coding segment resulting in 9,338 coding references within the dataset. In the first week, each researcher coded 2 transcripts. Week 1 coding was revised after a weekly meeting where the definitions of codes were refined as a group. In subsequent weeks, each researcher coded 3–5 randomly assigned transcripts. Inter-rater agreement was measured with Cohen's Kappa coefficients for each pair of coders.

Kappas were run after weekly coding assignments and re-run after all transcripts had been coded and redundant or conceptually overlapping codes were merged. In the final week of coding, all transcripts with Kappa coefficients below 0.6 were randomly reassigned and recoded. Recoding accounted for the fact that codes had been added and/or redefined during subsequent weekly coding meetings. Ten transcripts were recoded. The mean Kappa for the study was 0.70 (standard deviation = 0.075; range: 0.51–0.91). These mean Kappas may be interpreted as substantial inter-rater agreement with a range of moderate to almost perfect agreement across transcripts.

Beyond inter-rater agreement, interpretative and rank order consensus was achieved among coders and other members of the research team in weekly meetings as well as larger research team meetings focused on data analysis and synthesis. Transcripts were not only analyzed as one dataset but also by study subgroup to understand differential salience of themes across groups. Co-occurrence of themes was also analyzed. Summative documents on analytical and interpretative discussions form a replicable audit trail.

Results

Participants in this qualitative study (N = 72) were predominately white (95%), male (50%), and privately insured (94%), and reported annual household income ≥$100,000 (73%), education exceeding a bachelor's degree (87%), and mean HbA1c of 6.6% ± 0.8%. Table 2 presents demographic characteristics of participants by study subgroup: new users, experienced users, and new users after 12 months of use.

Table 2.

Demographic Characteristics of Qualitative Study Participants

	New users at baseline	New users at follow-up	Experienced users at baseline
Total study participants	45	35	27
Adults with type 1 diabetes	36	28	21
Parents or child with type 1 diabetes	9	7	6
Age at enrollment-mean (range)	33.2 (1–61)	32.6 (4–60)	32.3 (6–66)
Sex (female)	53%	54%	37%
Race (white)	98%	97%	96%
Diabetes duration-mean	21.1 (2–51.5)	20.4 (2–51.5)	21.0 (3–61)
Education³ Bachelor's	87%	86%	81%
Annual Household, Inc., ³$100K	69%	69%	78%
Private insurance	96%	97%	96%
Hemoglobin A1c-mean (SD)	6.7% (0.9%)	6.8% (0.9%)	6.2% (0.6%)

SD, standard deviation.

Nine broad themes described Loop experiences: start-up, access, benefits, drawbacks, interactions with the system, provider factors, health outcomes, effects on life and relationships, and recommended improvements. The most frequently discussed were (1) technical support and troubleshooting, (2) decreased mental and behavioral burden, (3) technical issues with any component of the system, (4) overall glycemic control, and (5) changing settings. Other highly endorsed topics were the effects of Loop on users' overnight experiences, the roles that communities of other users play during Loop use, and interactions with providers while using Loop. Table 3 lists the most salient themes by participant subgroup. New users described their start-up process as well as benefits and drawbacks experienced while using Loop. The salience of themes discussed by new users after ∼1 year of Loop use converged with those described by more experienced Loopers.

Table 3.

Commonly Discussed Qualitative Themes by Study Subgroup and Time Period

Most discussed themes among experienced and new Loop users
Experienced Loop users	New Loop users
	Initiation	Follow-up
Decreased mental and behavioral burden Relying on the system to support management Overnight improvement Adjusting or changing settings Technical issues and system difficulties Overall glycemic control	Decreased mental and behavioral burden System start-up System expectations Characteristics of people likely to succeed with Loop Overall glycemic control	Technical support and system troubleshooting Decreased mental and behavioral burden Loop community interactions or support Overnight improvement Technical issues and system difficulties Overall glycemic control

More experienced users discussed the benefits of decreased burden, overall glycemic control, and improved overnight quality of life, while also describing the technical knowledge and experience gained through managing system technical difficulties and troubleshooting issues and customizable settings. Descriptive analyses of participants' perspectives organized around four general themes—benefits, drawbacks, health outcomes, and provider factors—follow. Exemplar quotes are embedded within analytical descriptions of themes and have been edited for clarity and conciseness. Qualitative research findings neither depict nor seek to depict idealized use of Loop but highlight the perspectives and experiences of participants based on their real-world experiences and contexts.

Benefits of Loop

The most reported benefit of Loop was decreased mental or behavioral burden associated with diabetes management. Other benefits included fewer nighttime episodes requiring management and better sleep, the technical and psychological support provided by the Loop community, and the relative ease of system start-up and day-to-day system management.

Decreased mental and behavioral burden

Decreased burden was characterized by less worry, stress, and cognitive effort, less time spent on diabetes management tasks, more space to think about nondiabetes aspects of life and trusting Loop to help users remain in range. In the words of one participant:

“I was shocked, I guess, not shocked, but surprised at the number of adjustments that [Loop] was making for me, and it made me realize, you know… We all know what a burden it is, to be making all those decisions yourself and I was amazed at the burden lift that I felt. I feel like it has my back if that makes any sense. So that was a big, a big thing for me.” [New user]

Other participants described similar reactions to reduced mental or behavioral burden and glycemic control that was easier to attain.

“I have a similar A1c now, but it is much easier to achieve, and I think I probably haven't felt this healthy and safe and good and in control since before I was diagnosed as a kid. and I think the automation [of Loop] has helped reduce the 24 × 7 demoralizing job that is diabetes management… I cannot imagine being without the technology now.” [Experienced user]

Some participants also described how using Loop allowed for greater discretion with diabetes management in social situations. This discretion also alleviated some of the mental burden of diabetes during other activities of life.

“Well, I think when you're out and about, it's way easier to go on your phone, than pulling out a pump… when I'm going out with friends for dinner or something like that, it's nice to be able just pull up my phone… So, I think that's also less of a social strain as well.” [New user]

Loop overnight

The benefits of using Loop overnight included improvements in nighttime glycemic control, better quality sleep, and greater peace of mind during overnight hours. Participants reported fewer nighttime episodes requiring hands-on management, feeling secure about the system's ability to manage glycemic control overnight, and multiplier effects of higher quality sleep on factors such as mood the next day, mental and physical health.

“The overnights have been steady. Whereas before, I would go high and then I would get these lows. I was always up and down. But now with Loop it's been more, more steady. So there's no really like crazy highs or very like crazy lows, it's been more steady than it was before. So that's something that I think [has] really improved my management for diabetes.” [New user]

“Sleep. Definitely sleep [has been the most important change]. I don't get up at all anymore, and I was probably up two times a night for the last 15 years either dealing with highs, lows, needing to eat, highs that led to full bladder, so then I had to pee. I don't think I've slept this good in 15 years, honestly, and I wake up every morning and my blood sugar's between 88 and 102.” [New user]

Support of the Loop community

A plurality of participants lauded the technical and psychological/emotional support provided by other Loop users through use of a Facebook group and other internet-enabled communication platforms, as well as less frequent in-person meet ups. The broader community of Loop users served as an informal help desk—users posted and received responses to questions about using the system—as well as support groups where shared experiences of living with type 1 diabetes forged social and emotional bonds.

“I think the two biggest [benefits] for me: one, is the level of support that's available. I don't enjoy technology and [I spent] a long time on insulin pens just because I didn't want to engage with it, and it's not fun for me. And the level of support that was there just through the Facebook group is incredible. I don't think I ever went 10 minutes without a response when I needed to like reach out and had an issue like on Friday nights… That blew my mind and made me feel really comfortable with it. And the other thing was the amount of nights that I was able to sleep through the night again.” [New user at follow up]

Start-up and ability of Loop to ease day-to-day management

Getting Loop running and using Loop are different processes with each requiring different skills. Although experiences with system start-up and system management varied, many new users reported being surprised by the relative ease of getting started on Loop. Most users relied on support from others to help set up Loop. Since discussions about Loop involve algorithms and re-engineering technology, several new users were worried about acquiring parts and whether they were “tech savvy” enough to run Loop. For about half of those new users, the start-up process was easier than expected, whereas others reported challenges in the start-up phase.

After an initial period of adjustment following system setup, participants generally described the day-to-day management of the Loop system as easy. Descriptions of Loop's ease of use included mentions of the user-friendly interface, minimal minute-to-minute monitoring, blood glucose variation smoothing facilitated by algorithmic predictions and changes, and the adaptability of Loop to various conditions. The quote below describes this common experience arc: an initial period of uncertainty, followed by focus on customizing one's system settings, and relief when Loop assists by reliably adjusting in the background.

“For me, I was surprised really at how easy it was to do setup… I'm not a programmer, I'm not techy, I'm a retired accountant so I can follow instructions… I was kind of determined, [I thought,] ‘Oh, I got to have my settings perfect to begin with before I start,’ and they were decent, they were pretty good… I had a friend that kind of encouraged me, ‘go ahead, turn it on, turn it on,’ so I turned it on and it was great from the outset for me… I slept much better… I feel like it kind of is something that just has my back if you will, both at night and even during the day if I, you know, misjudged my carbs which unless you are perfect, we all do quite often. [Laughs.] [Loop] can smooth things out for me, so it has been a very good experience.” [New user at follow-up]

Parents expressed relief at the ease of leaving instructions for Loop with sitters, grandparents, or people other than routine caregivers. For example,

“It is so user friendly that you know if he is with grandparents, or you know friends it is very easy to train people even if they have never seen it before like this is how you do the very basic you know diabetes stuffs.” [Parent of experienced user]

Challenges of using Loop

The most reported challenges using Loop were technical issues related to Loop settings, the app, algorithm, signal loss among system components, and/or acquiring old pumps that were compatible with Loop. Participants also reported frustrations with insulin pumps, infusion sites, and CGMs, but these challenges may not have been directly attributable to Loop. Many users described a process of recognizing a need to improve their knowledge about type 1 diabetes, pros and cons of diabetes management regimens, and their physiological responses to different regimens. Although infrequently labeled a lack of education, this revelation suggests that information or education gaps may also undergird some of the operational challenges confronted while using Loop.

Changing settings and troubleshooting

Adjusting and changing settings for optimal system operation was the technical issue that participants discussed most both in the study focus groups and when searching Loop community notes or posting inquiries on the Loop Facebook group.

“It took me a long time to dial in the right insulin sensitivity and things like that and I wasn't expecting it to be that difficult, but in hindsight the complexity of the system gets a bit harder to do testing for things like your sensitivity because the basal rate isn't a constant in the background, like it makes sense. So, once I got over that frustration just kind of worked methodically to get the right settings, dialed in like with small changes and seeing how they worked, it has been stable for months and months now.” [New user at follow up]

The customizability of Loop means that multiple parameters may be changed and set for more personalized diabetes management. Examples of modifiable parameters include one's insulin sensitivity factor, basal rate, and insulin-to-carb ratio. For some participants, learning how to adjust those parameters required more knowledge and awareness than standardized diabetes management devices and drug regimens. Likewise, troubleshooting system challenges and malfunctions were perceived to require technological knowledge, not simply knowledge about type 1 diabetes management. Due to the open-source/DIY nature of Loop, users are unable to call a system manufacturer for troubleshooting or technical support. Loop users assume responsibility, individually and collectively, by way of community support, for resolving system issues or malfunctions. Participants who stopped using Loop during the study period were deterred by challenges they faced adjusting system settings or attempting to synchronize Loop's operation with their lifestyle and habits.⁴

Unplanned physical activity, without a priori announcement, still challenges AID systems and Loop is no exception. However, users noted use of various Loop branches that had improved management of exercise as well as the benefit of Loop's continuous improvement.

“For me, exercise is the biggest challenge. I wish that there was a way to say I am going to do be doing this type of exercise at this time… I am generally pretty aware of what I need to do differently between you know cardio versus [other exercise], but the system still cannot always quite figure that out.” [Experienced user]

“[O]ne thing that's surprised me is how active [Loop's] development still is. I was complaining awhile back to someone about exercise thresholds not working well for me and like the new dev branch has like really nice like toggles that weren't there a year ago and now they've been developed. [Managing] exercise is much better than it was year ago. So, I've been happy to see people are still working on it constantly.” [New user at follow up]

Moreover, since Loop comprises multiple devices communicating with one another across different radio frequencies, loss of connectivity was another drawback for users. Connectivity issues were most frequently ascribed to the RileyLink (transmission distance is most limited on the sub-GHz carrier used between the pump and RileyLink). However, it is not always clear which components are having issues communicating.

“The only real problems I have with it are when, kind of, the communication link breaks for whatever reason, and you need to troubleshoot that. So anything that could be done to improve the reliability of the communication piece of it. And I guess that's mostly the RileyLink, maybe.” [New user]

Health outcomes described in Loop use

In terms of health outcomes, Loop users frequently reported increased time in range and tighter control (fewer highs or lows), while using the system. Many also reported achieving lower HbA1c levels, while using Loop. Users were overwhelmingly satisfied with improvements in HbA1c that were accompanied by less concern about complications, sleeping, and provider surveillance.

“[Looping] far exceeded our expectations. My son's A1c, within three or four weeks, dropped a full point. It went from seven to six. And that was at our first endocrinologist appointment with it, with Loop. So, we were so ecstatic, and we did get our nights back, and, yeah, it's been, I could just stand on a picnic table and scream into the world that it was amazing, it changed everything I'd hoped and more.” [New user]

“I have definitely thought about my diabetes a lot less. But, on the other hand, my time in range: my A1c, just overall control has probably been the best that it's ever been and so that's been an exciting thing!” [New user]

Loop participants reporting learning more about their diabetes by reviewing data provided by Loop. Reviewing Loop data and adjusting settings based on the data, in turn, helped some users create individualized continuous quality improvement cycles. Loop users also reported becoming more comfortable having a system run in the background (compared with past foreground efforts to manage with other regimens), which resulted in demonstrable improvements or steadiness in overall glycemic control.

“For me, I think it's a lot of just thinking, thinking less about the effects of something after it happens. So, like in the past, I would, you know eat something, and then kinda worry about what my blood sugar is doing afterward, whether that carb estimate was right or wrong or that take another action to correct it afterwards. And I kind of just hands off let loop do its own thing and then no longer think about that… Overall, my a1c went from, you know, seven down to six of the first three months of using loop, which is pretty awesome. I did have a lower carb diet, which helps with that but just, just overall better, better blood sugars, better management, um, less time trying to, you know, manually fix everything.” [Experienced user]

Provider-related factors and Loop use

A range of provider attitudes and knowledge about Loop complicated users' clinical experiences and disclosure to providers. Study participants largely reported interactions with providers who primarily sought to optimize patients' glycemic control and prevention of disease complications. Some providers approved of the results achieved by their patients on Loop, while also expressing concerns about Loop not yet being approved as an interoperable automated glycemic controller (iAGC) by the FDA. Few providers discouraged patients' use of Loop; however, many providers' attitudes were described as ambivalent by participants. Provider ambivalence was characterized by perceptions of provider disinterest, limited discussion of Loop technical operations, and providers' limited understanding about Loop or other alternatives to conventional insulin pumps.

Several participants described switching providers because a provider ostensibly refused to care for them, while on Loop. Reports of providers enthusiastically learning about Loop alongside their patients and proactively communicating with patients curious about using Loop were also reported.

“So we had our first appointment, about three weeks, three weeks after we started, and our doctor was very excited. I didn't know how he would accept it, but he saw the clarity report, and he told us he had never seen a flatter, tighter line at night, ever, from anybody. So he was like, so he was excited, and he, he wanted to learn, and he listened, and he had heard about it, but did not have a patient on Loop, and when we're all done, he's like, ‘I'm not sure what I'm going to be able to do for you,’ but he did help write a prescription for supplies, for a pump that we didn't, you know, get a prescription for, so he was happy to do whatever he could to help support us.” [New user]

“I think my biggest surprise is how hard it would be to find a medical provider willing to work with me on Loop. I had two endos who wouldn't even kind of keep following me and then my third endo said, I will prescribe you your test strips and pump supplies, but I am not messing with your settings, and you are kind of you are on your own. So, I am getting my insulin and I am getting my pump supplies, but like any sort of guidance I know that they can't do, that's the nature of the system, but [it was] surprising to me.” [Experienced user]

Providers and patients resolved to collaborate encountered knowledge and equipment/supply challenges. Many participants conveyed that their providers did not possess enough information about Loop, its specifications, or operations to help patients troubleshoot system settings or errors. Participants also reported challenges obtaining Loop-compatible pumps, supply prescriptions, or other supplies due to Loop's status as an unapproved iAGC in the context of current insurance rules.

Discussion

This sample of Loop users reported benefits to quality of life and glycemic control that outweighed challenges (Table 4). Users reported being empowered by the customizability and educational effects of the open-source AID system. Improved quality of life attributed to Loop, decreased mental and behavioral burden, fewer overnight awakenings, and overall glycemic control were benefits most endorsed by Loop users. Favorable health outcomes have also been noted in other assessments of Loop's effectiveness.^1,5

Table 4.

Qualitative, Thematic Synthesis of Loop User Experiences

Benefits of Loop use	Challenges of Loop use
Improved glycemic control	System learning curve
Decreased mental and behavioral burden	Calibrating baseline and on-the-go settings
Support of Loop community	Lack of regulatory approval
System customization

Most participants expressed gratitude for the Loop community that formed around empathy, collaboration, and shared experiences. Many participants described an ease using Loop that reinforced the system's appeal. Users favored the flexibility to adapt system settings such as time in range limits or insulin sensitivity factors, which led many to think of Loop as a technological member of their diabetes management team. Likewise, by using Loop, many participants reported gaining new insights about how their body responds to insulin, food, and exercise. A few realized how limited information or analysis of data in their past may have resulted in suboptimal diabetes management behaviors.

Despite predominantly favorable views of Loop, as expected, some users also reported difficulty with acquiring system parts, and discomfort with the level of technical knowledge needed to set up and manage the assemblage of devices and parts. Some of these barriers will disappear when Loop receives FDA approval and devices transition toward all Bluetooth connections. Themes related to the absence of regulatory guidance for Loop as well as interacting with providers while using Loop support current efforts to develop international consensus guidelines for open-source AID, which could facilitate dialog and legal clarifications for systems that have proven safe, efficacious, and ethical.⁶

Those who used Loop in this study were far more likely to continue using Loop than to stop. Reasons for discontinued use complement the current findings.⁴ For the larger Loop observational cohort, the majority received help with building the application and reported relative ease getting their systems running. Rates of continuity and satisfaction demonstrate high acceptability among Loop users.

To our knowledge, this study represents the largest qualitative analysis of Loop users' experiences and perspectives to date. Findings corroborate results from prior research findings suggesting that advancement of closed-loop technologies may improve diabetes management and reduce mental burden and diabetes distress.^7

–13 As in Schipp et al. and Adams et al., participants asserted that benefits of using Loop outweighed perceived risks and challenges.^7,13 Moreover, participants' accounts of Loop drawbacks affirm Lawton et al's suggestion that providing additional support and guidance around accounting for and managing food choices while using Loop could be beneficial.¹⁰ Uptake among this sample of users also demonstrates interest in regimen and device innovation among some people living with diabetes as well as demands unmet by the most common and readily available devices at the time of the study.

Limitations of the study include the relative homogeneity of the sample as well as aspects of the study design. Participant characteristics do not mirror the variety of people living with type 1 diabetes. As in much diabetes technology research, the sample is overwhelmingly wealthy, white, well educated, and privately insured. Although these disparities may be due to sample bias, this sample is similar to other Loop study populations suggesting that these characteristics may be linked to awareness, appeal, and/or accessibility of Loop. Future studies could explore how innovative technologies may be experienced differently by different demographic populations. Some demographic characteristics may be associated with stopping Loop as discontinuers in this study were older, more female, and more likely to rely on public insurance.⁴ Findings suggest that Loop may require additional access and training considerations to appeal to broader populations of users and providers.

The study design did not include a comparison group. To expand the robustness of collected evidence, future studies of Loop users could implement quasi-experimental designs where the experiences of Loop users are compared to those using commercial diabetes management systems and/or include the perspectives of providers working with patients interested in using Loop. Studies with comparison groups will help identify aspects of diabetes management that are similar across regimens as well as elements unique to experiences of Loop. Moreover, inclusion of provider perspectives may help round out understandings of this regulatory environment and provider concerns about off-label management.

This study gathered experiences of Loop users and is not able to convey providers' perspectives or experiences with patients using Loop. Given the regulatory status of Loop, provider concerns were anticipated. Participant experiences of sharing their Loop use with providers highlighted the potential need for additional patient-provider communication about diabetes management pathways and objectives. As a final limitation, this study synthesizes user reports of their experiences with Loop and user accounts were not compared with data reports of their system settings.

While findings were not returned to study participants for verification, the research team is confident in the findings based on work on parallel, qualitative, and mixed-methods studies with similar research populations. At least one other international study of open-source AID uncovered user experiences and themes similar to those reported in this study for Loop.^14,15 The composition and experience of the research team intended to optimize teamwork, consensus building, as well as translation of research findings to clinical and behavioral interventions. The research team includes experienced diabetes and qualitative researchers, physicians and mental health clinicians who routinely serve people living with diabetes, as well as individuals or family members living with type 1 diabetes. The relatively large qualitative sample and theoretical saturation support study research rigor.

Conclusion

Loop delivered favorable experiences, which may be scaled in future devices/systems in an acceptable format. Challenges related to system setup and calibrating settings are remediable and, if addressed, may better serve Loop users and enhance system appeal. Providers' responses are partially dependent upon changes in the regulatory environment, yet current users' experiences suggest that some providers are open to learning more about Loop. Future research will document the evolution of Loop users' experiences within the dynamic diabetes management marketplace.

Footnotes

Authors' Contributions

S.C.S. wrote and revised this article and contributed to data analysis and interpretation. J.J.W. contributed to data collection, analysis, and interpretation, and article preparation. M.S.L. was involved in data collection, analysis, and interpretation. A.D. was involved in data analysis and interpretation. S.J.H. and K.K.H. contributed to study design, data collection and interpretation, and article review. R.A.L. contributed to data interpretation and article refinement. D.N. contributed to study conceptualization, study design, data collection, analysis, and interpretation, and article review and preparation.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was supported by The Leona M. and Harry B. Helmsley Charitable Trust and NIH grant K23-DK121771 and was coordinated by the JCHR.

References

Lum

, Bailey

, Barnes-Lomen

, et al.: A real-world prospective study of the safety and effectiveness of the loop open-source automated insulin delivery system. Diabetes Technol Ther, 2021; 23:367–375.

Freeborn

, Dyches

, Roper

, Mandleco

: Identifying challenges of living with type 1 diabetes: child and youth perspectives. J Clin Nurs, 2013; 22:1890–1898.

Guest

, Namey

, Taylor

, et al.: Comparing focus groups and individual interviews: findings from a randomized study. Int J Soc Res Methodol, 2017; 20:693–708.

Wong

, Suttiratana

, Lal

, et al.: Discontinued use of the loop insulin dosing system: a mixed-methods investigation. Diabetes Technol Ther 2021;Nov15(ja).

Jeyaventhan

, Gallen

, Choudhary

, et al.: A real-world study of user characteristics, safety and efficacy of open-source closed-Loop systems and medtronic 670G. Diabetes Obes Metabol, 2021; 23:1989–1994.

Braune

, Lal

, Petruželková

, et al.: Open-source automated insulin delivery: international consensus statement and practical guidance for health-care professionals. Lancet Diabetes Endocrinol, 2022; 10:58–74.

Adams

, Tanenbaum

, Hanes

, et al.: Psychosocial and human factors during a trial of a hybrid closed loop system for type 1 diabetes management. Diabetes Technol Ther, 2018; 20:648–653.

Garg

, Weinzimer

, Tamborlane

, et al.: Glucose outcomes with the in-home use of a hybrid closed-loop insulin delivery system in adolescents and adults with type 1 diabetes. Diabetes Technol Ther, 2017; 19:155–163.

Hendrieckx

, Poole

, Sharifi

, et al.: “It is definitely a game changer”: a qualitative study of experiences with in-home overnight closed-loop technology among adults with type 1 diabetes. Diabetes Technol Ther, 2017; 19:410–416.

10.

Lawton

, Blackburn

, Rankin

, et al.: The impact of using a closed-loop system on food choices and eating practices among people with Type 1 diabetes: a qualitative study involving adults, teenagers and parents. Diabetic Med, 2019; 36:753–760.

11.

Lawton

, Blackburn

, Rankin

, et al.: Participants' experiences of, and views about, daytime use of a day-and-night hybrid closed-loop system in real life settings: longitudinal qualitative study. Diabetes Technol Ther, 2019; 21:119–127.

12.

Rankin

, Kimbell

, Hovorka

, et al.: Adolescents' and their parents' experiences of using a closed-loop system to manage type 1 diabetes in everyday life: qualitative study. Chronic Illn. 2021 Jan, 20:1742395320985924. [Epub ahead of print]; DOI: 10.1177/1742395320985924. PMID: 33472409.

13.

Schipp

, Skinner

, Holloway

, et al.: How adults with type 1 diabetes are navigating the challenges of open-source artificial pancreas systems: a qualitative study. Diabetes Technol Ther, 2021; 23:546–554.

14.

Cohen

, Barnard-Kelly

, Suttiratana

, et al.: Do-it-yourself automated insulin dosing: qualitative study exploring user experiences. Diabetes Technol Ther, 2021; 23:A94–A94.

15.

Cohen

, Naranjo

, Suttiratana

, et al.: Do-it-yourself automated insulin dosing: qualitative study exploring healthcare professional experiences. Diabetes Technol Ther, 2021; 23:A23–A24.