Abstract
Objective:
We evaluated the safety and feasibility of open-source automated insulin delivery AndroidAPS in adolescents and young adults with type 1 diabetes (T1D) and compared its efficacy in three different scenarios: hybrid closed loop (HCL) with meal boluses, meal announcement only (MA), and full closed loop (FCL).
Research Design and Methods:
In an open-label, prospective, randomized crossover trial (
Results:
The glycemia was controlled by the system 95% time of the study and the proportion of time below 3 mmol/L did not exceed 1% over the whole study period (0.72%). The HCL scenario reached significantly higher percentage of time below 3 mmol/L (HCL 1.05% vs. MA 0.0% vs. FCL 0.0%; P = 0.05) compared to other scenarios. No difference was observed among the scenarios in the percentage of time between 3.9 and 10 mmol/L (HCL 83.3% vs. MA 79.85% vs. FCL 81.03%, P = 0.58) corresponding to mean glycemia (HCL 6.65 mmol/L vs. MA 7.34 mmol/L vs. FCL 7.05 mmol/L, P = 0.28). No difference was observed in the mean daily dose of insulin or in the daily carbohydrate intake. No serious adverse event occurred during the study period.
Conclusions:
Our pilot study showed that FCL might be a realistic mode of treatment for people with T1D.
Introduction
Commercial hybrid closed-loop (HCL) systems (also referred to as automated insulin delivery “AID”) are increasingly applied in type 1 diabetes (T1D) management and have been shown to improve glycemic control in children and adolescents with T1D. 1 –3 As observed in previous studies, the AID systems were particularly beneficial overnight. 4 However, applying premeal insulin is still warranted to achieve optimal glucose control 5 and the users are still burdened by carbohydrate counting.
The omission of or delayed insulin boluses with meals or snacks is commonly reported in up to 50% of adolescents on insulin pump therapy. 5 –7 The benefit that has been seen from HCL systems is uncertain in these users, 6 although the recent study by Tornese et al. reported a well-tolerated unannounced snack of <20 g of carbohydrates by the advanced HCL system in the pediatric population. 8 Some real-world studies have even documented a high rate of discontinuation of HCL therapy in youths because of “too much work.” 6
The alternatives to the official approved systems are the open-source AID programs, including OpenAPS, AndroidAPS, an Android-based system using the OpenAPS algorithm, and Loop, an iOS-based system using a different algorithm. 9 These systems still cannot be universally recommended over commercial options because there is still the question of the responsibility in case of serious adverse events resulting from the lack of an official approval of these systems, results in an ambivalent attitude of health care providers toward them. 10
However, the education led by community of people with T1D increased general awareness of these systems and the first RCTs trials exhibited positive results, and practical guidance was published.
9,11,12
Moreover, the open-source systems introduced extra features enabling increased user flexibility and improved outcomes with less effort. This includes the “unannounced meal” feature and the supermicroboluses (SMB), which adjust insulin dosing as soon as the postprandial glucose rise is detected by the continuous glucose monitoring (CGM) with no need of manual meal bolusing. These features used in combination enable some OpenAPS and AndroidAPS users to utilize the open-source system in a full closed-loop (FCL) mode (13,
In this study, we present the results of open-label randomized crossover trial of the open-source AndroidAPS system, evaluating the safety and feasibility of the system and comparing its effectiveness in three different scenarios; traditional HCL, meal announcement only (MA), and the first-ever assessment of FCL in adolescents and young adults. In addition, we assessed the users' quality of life (QoL) and satisfaction with different scenarios.
Research Design and Methods
Study population and design
All adolescents and young adults enrolled in the study met the following inclusion criteria; (1) duration of T1D ≥1 year, (2) age 15–20 years, (3) baseline glycated hemoglobin (HbA1c) value between 43 and 75 mmol/mol International Federation of Clinical Chemistry (IFCC), (4) the use of CGM and insulin pump therapy ≥3 months, and (5) willingness to use the AID system continuously throughout the study. Exclusion criteria included pregnancy and any concomitant medication affecting glucose control. Eligible study adolescents were recruited at Children Diabetes Centrum of Motol University Hospital on first come, first-served basis.
The study was conducted in an inpatient setting at Motol University Hospital due to COVID-19 restrictions, but the stay simulated free living conditions by maintaining eating habits and the usual daily routine, including online education and personalized physical activity. There was no correction or education in carbohydrate counting by the study team during the whole study period.
The study consisted of a run-in phase lasting 1 day and followed by 9 days of different pump setting scenarios, which were divided into three study periods (for details see Fig. 1a, b). At enrollment, written consent was obtained from all adolescents, young adults, and their parents/caregivers and baseline data were collected. During the run-in period, participants received the investigation devices (Fig. 1c): insulin pump: DANA RS CGM: Dexcom G6 and Samsung S10 smartphone, with the control algorithm and were instructed to work with the system in an open mode. After 1 day of run-in, the 3-day HCL period started, followed by two 3-day scenarios (MA or FCL), which were assigned at random on a 1:1 basis.

The study was conducted in compliance with the International Organization for Standardization 14155:2012 and the Declaration of Helsinki. The study was approved by the institutional Ethics Committee and the State Institute for Drug Control SUKL (sukls11123/2021) and registered at
AID system
In the modified version of AndroidAPS 3.1.03 was reduced the part of “AndroidAPS learning objectives,” which consists of eight compulsory learning points you must pass as a mandatory part of the standard AndroidAPS configuration process and serve as user education (
OpenAPS and AndroidAPS enable automated insulin coverage of meals without carbohydrate announcement using the “unannounced meal” feature, whereby meals are anticipated based on available data suggesting an otherwise unanticipated glycemic excursion. These unannounced meals may then be acted upon by an SMB wherein tiny boluses are delivered to affect rising glucose levels more rapidly. 13
The algorithm enables switching among different scenarios of closed loop (HCL, MA, or FCL) depending on the amount of inputs entered into the system. HCl scenario is working with SMBs but only half of the calculated bolus is delivered before the meal while the rest of the insulin is delivered in form of SMBs afterwards. MA scenario working with SMBs without meal bolus delivery. The algorithm receives the carbohydrate announcement (the participants entered the estimated amount of carbohydrates) and performs all prandial insulin dosing using subsequent SMBs. FCL scenario is working with SMBs without any meal bolus with all the prandial insulin delivered in the form of SMBs after a increase in BG is detected. 13
The initial setting of all required values (basal profile, insulin-to-carbohydrate ratio, and insulin sensitivity) was calculated by one of the AndroidAPS app features, called the “profile assistant,” which uses the total daily dose of insulin, weight, or age for calculation. Further adjustment of the above-mentioned values was performed by the supervisors during the run-in and before each intervention period. The target glycemia was set to 5.5 mmol/L for all participants. The temporary target 8.0 mmol/L and a temporary profile (50% of basal rate, insulin-to-carbohydrate ratio, and insulin sensitivity) were set during the physical activity. Participants used the Pancreas4ALL app for mealtime bolus calculation and administration during HCL scenario, for mealtime bolus administration in MA scenario, and for fully AID in FCL scenario.
All participants used the fast-acting insulin aspart (Fiasp, NovoNordisk, Denmark). Participants were recommended to check their blood glucose using glucose meter in case of glycemia below 3 and blood glucose and ketone bodies, while glycemia was above 15 mmol/L during the entire study period. Hypoglycemia [real hypoglycemia defined as <3.9 mmol/L (70 mg/dL); prevention of hypoglycemia <4.9 mmol/L (88 mg/dL)] was treated with dextrose only according to the hypoglycemia levels and glycemic trends assessed by CGM.
In case of hyperglycemia (<15 mmol/L, 270 mg/dL) and positive ketone, the cannula replacement was considered and the insulin on board was deleted through the AndroidAPS app feature. All participants were provided with a 24-h online chat as a helpline. The data from the insulin pump and AndroidAPS system were uploaded automatically through NightScout to study cloud. All glucose outcomes were computed based on the CGM records. At the end of the study, all participants returned to their original treatment modality.
Statistical analyses
The differences between the tested scenarios were analyzed using nonparametric tests (Kruskal–Wallis and analysis of variance). The data of mean glucose levels in all three scenarios, including repeated measurements over three nights and days, were compared using a linear mixed-effects model. P-values <0.05 were considered statistically significant. The analyses were conducted using the R statistical package, version 3.1.1.
Study objectives
The primary objective was to demonstrate the feasibility and safety of AndroidAPS. Feasibility was defined as >90% of the study time on the AndroidAPS and safety as the percentage of time spent in level 2 hypoglycemia (<3 mmol/L) below 1% over the whole study period. The occurrence of severe adverse events such as severe hypoglycemia or diabetic ketoacidosis was also monitored during the whole study period. In addition, any adverse event occurring during the study was noted.
The secondary objective was to demonstrate the effectiveness of AndroidAPS by exceeding the time in range (TIR) over 70% in all three tested scenarios. Furthermore, the following values were also compared between the three tested scenarios (HCL, MA, and FCL): TIR (3.9–10 mmol/L), time above range level 1 (10.1–13.9 mmol/L), time above range level 2 (>13.9 mmol/L), mean sensor glycemia, time below range level 1 (3.0–3.8 mmol/L), time below range level 2 (<3.0 mmol/L), and glycemic variability [coefficient of variation (CV) and standard deviation of glycemia (SD)]. We also compared the day- and night-time performance of all three tested scenarios. The values of total daily dose of insulin [IU/(kg·d)] and daily carbohydrate intake were also collected by the study team.
In addition, a national language modified version (certified by the MAPI Research Trust, Lyon, France) of the 7-day QoL questionnaire PedsQL 3.2 Diabetes Module 14 and the Closed-Loop Experience Questionnaire 15 were administered to the participants at the beginning and end of the study to observe the psychosocial outcomes and user satisfaction.
Results
A total of 19 participants were screened for the study participation between February 26, 2021, and June 3, 2021. Three did not fulfill the inclusion criteria. Sixteen participants (10 females) with mean age of 17 years (range 15–20) and mean HbA1c 56 mmol/mol IFCC (range 43–75) [7.3% Diabetes Control and Complications Trial (range 6.1–9.0)] entered the study (for detailed characteristics see Table 1).
Baseline Characteristics of Study Participants
BMI, body mass index; DCCT, Diabetes Control and Complications Trial; HbA1c, glycated hemoglobin; IFCC, International Federation of Clinical Chemistry.
All the participants completed the study. Insulin was administered autonomously by the AndroidAPS for a median of 95% (92%–99%) of the study period. No serious adverse event such as severe hypoglycemic episode or diabetic ketoacidosis was observed throughout the study period (for all documented adverse events see Table 2). The time spent in hypoglycemia <3 mmol/L did not exceed 1% throughout the entire study (0.72%), with the HCL scenario having more time <3 mmol/L compared to other scenarios (HCL 1.05% vs. MA 0% vs. FCL 0%; P = 0.05). No difference was observed among the scenarios in time spent in hypoglycemia 3–3.8 mmol/L during whole day period (HCL 3.6% vs. MA 2.4% vs. FCL 1.4%; P = 0.3). The subanalysis comparing daytime and nighttime results showed more time in hypoglycemia during the daytime in HCL scenario (for details see Table 3).
Summary of Adverse Events and Setting Adjustment of Input Parameters Over Whole Study Period
AE, adverse events; CGM, Continuous Glucose Monitoring; HCL, hybrid closed loop; IR, insulin ratio; ISF, insulin sensitivity factor.
Comparison of Glycemic Outcomes (CGM Data), TDD, and Total Carbohydrate Intake between Three Different Tested Scenarios of Pancreas4ALL Automated Insulin Delivery
TIR (3.9–10 mmol/L), TAR1 (10.1–13.9 mmol/L), TAR2 (>14 mmol/L), mean sensor glycemia, TBR1 (3.0–3.8 mmol/L), TBR2 (<3.0 mmol/L), glycemic variability [CV and SD], TTD [U/(kg·d)]. Data are presented as mean ± SD or medians (IQRs).
CV, coefficient of variation; FCL, full closed loop; HCL, hybrid closed loop with premeal boluses; IQRs, interquartile ranges; MA, meal announcement only; SD, standard deviation of glycemia; TAR1, time above range level 1; TAR2, time above range level 2; TBR1, time below range level 1; TBR2, time below range level 2; TIR, time in range; TTD, total daily insulin dose.
All tested scenarios of Pancreas4ALL system showed excellent TIR (HCL 82.88% vs. MA 79.85% vs. FCL 81.03%; P = 0.58), which corresponds to mean glycemia (HCL 6.9 mmol/L vs. MA 7.34 mmol/L vs. FCL 7.08 mmol/L, P = 0.28). The course of mean glucose levels during the day in all three tested scenarios is displayed in Figure 2. In all scenarios, glycemic variability measured as the SD was <2.5 mmol/L (HCL 2.15 vs. MA 2.3 vs. FCL 2.15; P = 0.78) and <34% if measured as the CV (HCL 33% vs. MA 32% vs. FCL 32%; P = 0.46). No difference was observed in the mean daily dose of insulin (HCL 48.15 U vs. MA 48.1 U vs. FCL 43.5 U; P = 0.90) as well as in the daily carbohydrate intake in any tested scenario (HCL 182 g vs. MA 155 g vs. FCL 165 g; P = 0.63).

Comparison of median of mean glycemia (25–75 percentile) over whole 3 days for all tested scenarios; HCL—red line, MA—blue line, FCL—green line, Time in range (3.9–10 mmol/L) borders marked with a dashed line.
All 16 participants completed the Closed-Loop Experience Questionnaire for each tested scenario (the results are displayed in Supplementary Fig. S1). The participants enjoyed the automatic regulation of their blood glucose in all tested scenarios; they spent the least time managing their diabetes during the FCL. Interestingly, the participants expressed fewer diabetes-related worries during the HCL scenario compared to the MA and FCL. Participants also reported improvement in diabetes-related symptoms (P < 0.01) and worries about diabetes dimensions of the QoL questionnaire (P < 0.01) (for details see Supplementary Table S1).
Discussion
To our knowledge, this is the first randomized trial evaluating the algorithm of open-source AID AndroidAPS in different scenarios, including the FCL. We demonstrated that AndroidAPS can achieve excellent TIR in all the tested scenarios without excessive time in hypoglycemia, limitation of carbohydrate intake, or increasing the daily dose of insulin.
Our results are in concordance with the in silico trial testing the same algorithm 13 as well as with previously published studies with retrospective or survey designs. 9,16 –19 The prospective single-arm study by Gawrecki et al. 11 (Time <3 mmol/L 0.35% and TIR 79%) and randomized parallel arm Create study by Burnside et al. 12 (Time <3.9 mmol/L 2.1% and TIR 71%) presented similar results of HCL, but they did not evaluate the FCL scenario.
The recent study by Russell presented the results of a Bionic Pancreas, which uses meal announcements without carbohydrate counting, similar to the MA scenario in our study. The Bionic Pancreas displayed less time hypoglycemia compared to our study, but it was accompanied by a significantly lower TIR (65%). 20
The data evaluating glucose control by AID systems in fully closed regimen are scarce. 21 The currently available commercial systems exhibited increased postprandial glycemia, especially in the morning during the FCL regimen, despite using concomitant pharmacotherapy with Sodium-Glucose Transport Protein 2 inhibitors 22 or pramlintide. 23 Interestingly, Garcia-Tirado et al. showed that with a “priming dose of insulin” as a response to glucose elevation (similar approach to SMB), it is possible to maintain good postprandial glucose levels when bolus before dinner was omitted. 24 In our study, we demonstrated that the AndroidAPS was capable of achieving excellent glucose control during the whole study period, including the FCL regimen, which is reflected in comparable time in mild hyperglycemia (10.0–13.9 mmol/L) of 8%–14% in all tested scenarios (for details see Table 3).
Despite the excellent results of the tested AID, we have observed several pitfalls. First, we encountered postprandial hyperglycemia in FCL scenario when the premeal glucose level was <4 mmol/L or physical exercise preceded the meal. In those situations, the safety mechanism of the algorithm tried to prevent hypoglycemia by delaying additional insulin dosing of SMBs until the glycemia was above 8 mmol/L (There is a default rule in AndroidAPS of setting higher temporary target during and after hypoglycemia.), which was too late to maintain postprandial euglycemia. Applying a premeal bolus might thus be beneficial for the user if the net insulin on board is below zero.
Similarly, we observed prolonged hyperglycemia after cannula malfunction, as the algorithm decreased the correction SMBs due to perceived insulin on board from the previous cannula. The solution may be to remove this “false” insulin from the system (which is one of the features of the tested system) or to apply the correction bolus manually. Concern and special attention should be paid to cannula problems in all full closed regiment.
Finally, the amount of insulin delivered by SMBs highly depends on correct insulin ratio and especially insulin sensitivity factor (settings). For the optimal function of the algorithm, these values need to be optimized (for this process during the study; Table 2). Integration of an automated artificial intelligence-based decision support tool for optimizing the setting might pave the way for the future to reach fully artificial pancreas working with no user support. 25
Despite the pitfalls, all study participants enjoyed having glucose levels managed by Pancreas4ALL system. Interestingly, they reported feeling guilty for omitting the meal bolus and were worried not to count the meal content during FCL scenario. As they were used to it for several years, they found it difficult to change their routine.
The main limitations of the study were small sample size and short duration and the level of supervision of the clinical trial required by the local authorities and the short time span of the study. Regardless of the effort to imitate the camp regimen with unlimited food intake and physical activity, the pandemic restrictions in place during the time did not allow for such close simulation of free-living conditions. Thus, a longer out-patient study is warranted.
In addition, while the order of FCL and MA group was assigned at random, there was a fixed HCL period at the beginning of the study. The HCL period was preceded by only one a day of the run-in period for the adjustment of input settings. Although the profile assistant function was used for the initial setting of the system, the used input values, such as total daily dose, were inaccurate mostly due to missing boluses in prestudy period, which could have been adjusted, had the run-in period been longer. Strengths of this study were the randomized design of the trial with several different intervention groups, including the FCL. The extensive description of pitfalls that we met during the study can have practical implications for further FCL studies.
Our study not only showed that FCL is a realistic mode of treatment for people with T1D but also reflected the barriers and limitations of this scenario. The results of the FCL scenario showed promise for adolescents and young adults, as well as adults who are often burdened by the magnitude of required diabetes self-management. Furthermore, our study showed the importance of conducting RCTs to address pitfalls and challenges that users may encounter, while using do-it-yourself AID systems.
Footnotes
Acknowledgments
We would like to thank George Grunberger and Dana Lewis for their very valuable comments.
Authors Contributions
L.P.: principal investigator, designed and coordinated the study, wrote the article, and is the guarantor of the article. V.N., L.P., B.O., and S.K.: supervised and controlled the patients during the study and wrote the article. M.K.: study device creator, technical support, and helpline coordinator. S.P. and Z.S.: wrote the article.
Author Disclosure Statement
The national regulatory authority SUKL defines the regulatory pathway and restriction of testing the new medical device. The open-source device is not permitted to test, because the clear clarification of responsibility in case of a severe adverse event must be stated. M.K., the main developer of the AndroidAPS system, built the company CLOSED LOOP systems and took on the responsibility of the producer. The Motol University Hospital was the main investigator. The open-source version of AndroidAPS app was amended, while the number of objectives and time required to close the loop were reduced. The modified version was called Pancreas4ALL. All described steps were performed for study purposes. We have obtained permission from our national regulatory authority SUKL to provide the first feasibility and safety study after a year of effort.
M.K. is a holder of GPL license of AndroidAPS software. Other co-authors, including principal investigator L.P., have no competing financial interests.
Funding Information
This study was supported by Technology Agency of the Czech Republic TACR TJ02000342 and by Ministry of Health of the Czech Republic, Grant No. 64203.
Supplementary Material
Supplementary Data
Supplementary Figure S1
Supplementary Table S1
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
