Performance of a New Blood Glucose Monitoring System in the Hands of Intended Users

Abstract

Background:

This study assessed the performance of a blood glucose monitoring system (BGMS) in development that uses a new generation of blood glucose test strips with capillary and venous blood in the hands of its intended users, people with diabetes and healthcare professionals (HCPs).

Subjects and Methods:

In total, 93 subjects ≥18 years old (median age, 33 years) with type 1 (78%) or type 2 (22%) diabetes participated. Untrained subjects performed self-test fingersticks using a Microlet^®2 lancing device (Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY) followed by testing of their own capillary blood on the BGMS. HCPs performed fingersticks (using a Tenderlett^® lancing device [International Technidyne Corp., Edison, NJ]) and venipunctures on subjects and tested both capillary and venous samples from subjects on the BGMS. All BGMS results were compared with Yellow Springs Instruments (YSI) (YSI Life Sciences, Inc., Yellow Springs, OH) laboratory results. Analytical accuracy was assessed according to International Organization for Standardization (ISO) 15197:2003 guidelines (i.e., within ±15 mg/dL or ±20% of the YSI results for samples with glucose concentrations <75 mg/dL and ≥75 mg/dL, respectively) and more stringent criteria (i.e., within ±15 mg/dL or ±15% of the YSI results for samples with glucose concentrations <100 mg/dL and ≥100 mg/dL, respectively).

Results:

Overall, 98.9% (180/182) of subject Microlet2 capillary fingerstick results, 99.5% (182/183) of HCP Tenderlett capillary fingerstick results, and 100% (186/186) of venous results met current ISO criteria and more stringent criteria. The average hematocrit was 44%, with values ranging from 32% to 52%.

Conclusions:

Test results from both capillary fingerstick and venous samples with a new BGMS in development met current accuracy guidelines as well as proposed tighter criteria.

Introduction

S elf-monitoring of blood glucose (SMBG) is an integral component of comprehensive diabetes therapy and helps people with diabetes achieve optimal glycemic control.^1

–4 SMBG has been associated with improved clinical outcomes,^3,5,6 and because blood glucose values may help guide treatment decisions, accuracy is an important consideration when selecting a blood glucose monitoring system (BGMS).⁷ Standards from the International Organization for Standardization (ISO) are widely used to assess the accuracy of SMBG devices.⁸ In recent years, several organizations and regulatory agencies have recommended tighter accuracy criteria.^9
–11

Different factors contribute to variations in results from blood glucose testing. For instance, results obtained by individuals with diabetes in their homes may differ from those obtained in the office of a healthcare professional (HCP).¹² The source of a blood sample must be considered, as studies have shown that significant differences can be found in blood glucose levels for capillary versus venous blood samples, especially in nonfasting individuals.^13

–17 Thus, accuracy comparisons should be based on like samples (e.g., capillary to capillary, venous to venous). In addition to accuracy, ease of use of the BGMS may be another important consideration for people with diabetes who use SMBG to manage their blood glucose.

A BGMS is currently in development for use with a new generation of blood glucose test strips that utilize the flavin adenine dinucleotide–glucose dehydrogenase enzyme and a proprietary electron mediator to measure capillary or venous blood glucose levels. This BGMS has a glucose range of 10–600 mg/dL and a 0.6-μL sample volume. The test strip chemistry eliminates galactose and maltose interference, and the system compensates for hematocrit and has decreased sensitivity to common interferents. The primary objective of this study was to evaluate the performance of the new BGMS with capillary blood in the hands of intended users, people with diabetes and HCPs, and with venous blood based on ISO 15197:2003 guidelines⁸ and proposed more stringent criteria. Additional objectives were to obtain subject feedback on BGMS ease of use and to evaluate subject comprehension of BGMS labeling materials.

Subjects and Methods

Study design

This study was conducted at a single clinical site in the United States from November 2010 to December 2010. Eligible subjects were males and nonpregnant females ≥18 years old with type 1 or type 2 diabetes who regularly performed SMBG at home. The study protocol, informed consent forms, and BGMS labeling materials were approved by an institutional review board prior to study initiation. All subjects completed the informed consent process.

During the single clinic visit, untrained subjects learned to use the BGMS by reading the labeling materials provided, with no assistance from the study staff. For testing on the new BGMS, three test strip lots were used. Subjects performed self-test fingersticks using a Microlet^®2 lancing device (Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY) and tested their own capillary blood on the BGMS. HCPs performed fingersticks (using a Tenderlett^® lancing device [International Technidyne Corp., Edison, NJ]) and venipunctures on subjects and tested both capillary and venous samples from subjects on the BGMS. All results were compared with those obtained using a Yellow Springs Instruments (YSI) blood glucose analyzer (YSI Life Sciences, Inc., Yellow Springs, OH). The venous sample was also used to measure the subject's hematocrit on the CritSpin Reader (Iris Sample Processing, Westwood, MA).

Assessments and analyses

BGMS results obtained by subjects and HCPs for capillary blood samples and by HCPs for venous blood samples were compared with YSI laboratory reference results using regression analysis. Accuracy was assessed based on ISO 15197:2003 criteria⁸ (i.e., the percentage of results within ±15 mg/dL or ±20% of the mean YSI reference result for samples with glucose concentrations <75 mg/dL and ≥75 mg/dL, respectively).

Accuracy was also assessed based on proposed more stringent criteria: the percentage of results within ±15 mg/dL or ±15% of the YSI results for samples with glucose concentrations <100 mg/dL and ≥100 mg/dL, respectively. To determine whether there was a significant difference in accuracy based on blood collection methodology, a χ² test was performed. Parkes consensus error grid analysis¹⁸ was used to classify results based on the clinical significance of differences in BGMS results from YSI reference results.

Subjects completed a questionnaire rating the BGMS; the possible ratings were “no opinion,” “unacceptable,” “poor,” “good,” “very good,” and “excellent.” After reviewing the labeling information provided, subjects were rated by HCPs on their ability to perform basic tasks with the BGMS. Adverse events were monitored throughout the study.

Results

Subjects

This study enrolled 96 subjects with type 1 or type 2 diabetes; 93 subjects had evaluable blood glucose results (Table 1). In total, 92 subjects completed sample testing from HCP Tenderlett fingersticks, 91 completed sample testing from subject Microlet2 fingersticks, and 93 had results from HCP venous blood testing. The glucose concentrations ranged from 55 mg/dL to 432 mg/dL for capillary blood samples and from 48 mg/dL to 429 mg/dL for venous blood samples. Duplicate hematocrit values were obtained for each subject; the average hematocrit ranged from 32% to 52% with a mean (SD) of 43.9% (3.8%).

Table 1.

Subject Demographic and Baseline Characteristics

Characteristic	From n=93
Type of diabetes [n (%)]
Type 1	73 (78)
Type 2	20 (22)
Gender [n (%)]
Female	47 (51)
Male	46 (49)
Age (years)
Mean (SD)	37.8 (15.9)
Median (range)	33 (18–73)
Ethnicity [n (%)]^{a b}
White/Caucasian	85 (93)
Black/African American	3 (3)
Asian	1 (1)
Other	2 (2)
Duration of diabetes [n (%)]
≤2 years	5 (5)
3–5 years	9 (10)
6–10 years	15 (16)
>10 years	64 (69)
Diabetes therapy [n (%)]^c
Insulin injection	36 (39)
Pump use	49 (53)
Oral agents	20 (22)
Diet	31 (33)
Exercise	35 (38)
Other	2 (2)
Frequency of daily SMBG [n (%)]^b
>4	51 (55)
4	15 (16)
3	12 (13)
2	8 (9)
1	7 (8)

n=91.

Percentages may not total 100% because of rounding.

Use of diabetes therapies was not mutually exclusive.

SMBG, self-monitoring of blood glucose.

Accuracy

Regression analyses showed good correlation between subject Microlet2 fingerstick (R² =0.988, n=182) BGMS results and YSI laboratory reference results (Fig. 1A). Results from HCP Tenderlett fingerstick (R² =0.986, n=183) and venous (R² =0.996, n=186) blood samples also showed good correlation (Fig. 1B and C).

FIG. 1.

System accuracy regression analysis of blood glucose monitoring system results versus laboratory reference results for (A) subject Microlet2 fingerstick, (B) healthcare professional (HCP) Tenderlett fingerstick, and (C) venous blood samples. YSI, Yellow Springs Instruments.

Overall, 98.9% (180/182) of subject Microlet2 capillary fingerstick results met current ISO accuracy criteria as well as proposed more stringent criteria (Tables 2 and 3). Evaluation of HCP Tenderlett capillary fingersticks showed that 99.5% (182/183) of results met both current and proposed more stringent criteria (Tables 2 and 4). Overall, 100% (186/186) of venous results met both current ISO accuracy criteria and proposed more stringent criteria (Tables 2 and 5). Statistical analysis showed no significant difference in the percentage of accurate results among subject Microlet2 capillary fingerstick, HCP Tenderlett capillary fingerstick, and venous measurements using either current ISO accuracy criteria or proposed more stringent criteria.

Table 2.

Total Number of Results Within International Organization for Standardization 15197:2003 Minimum Acceptable Performance Criteria and Proposed More Stringent Criteria

	Number of results within the specified limits
Sample type	Current ISO criteria ^a	Proposed criteria ^b
Subject Microlet2 capillary	180 of 182 (98.9%)	180 of 182 (98.9%)
HCP Tenderlett capillary	182 of 183 (99.5%)	182 of 183 (99.5%)
Venous	186 of 186 (100%)	186 of 186 (100%)

Current International Organization for Standardization (ISO) 15197:2003 minimum acceptable performance criteria⁸ (i.e., ≥95% of results shall fall within ±15 mg/dL or ±20% for samples with glucose concentrations <75 mg/dL and ≥75 mg/dL, respectively).

Proposed more stringent criteria^9
–11 (i.e., ≥95% of results shall fall within ±15 mg/dL or ±15% for samples with glucose concentrations <100 mg/dL and ≥100 mg/dL, respectively).

HCP, healthcare professional.

Table 3.

Subject Microlet2 Fingerstick Results Within International Organization for Standardization 15197:2003 Minimum Acceptable Performance Criteria and More Stringent Criteria

Glucose concentration, data comparison	Number of readings within specified error limits
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<75 mg/dL, subject vs. YSI (n=18)	13 (72.2%)	17 (94.4%)	17 (94.4%)	18 (100%)^a	18 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥75 mg/dL, subject vs. YSI (n=164)	116 (70.7%)	155 (94.5%)	160 (97.6%)	162 (98.8%)	162 (98.8%)^a
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<100 mg/dL, subject vs. YSI (n=42)	26 (61.9%)	41 (97.6%)	41 (97.6%)	42 (100%)^b	42 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥100 mg/dL, subject vs. YSI (n=140)	103 (73.6%)	132 (94.3%)	136 (97.1%)	138 (98.6%)^b	138 (98.6%)

Within the International Organization for Standardization (ISO) 15197:2003 minimum acceptable performance criteria.⁸

Within the more stringent criteria.^9
–11

YSI, Yellow Springs Instruments.

Table 4.

Healthcare Professional Tenderlett Fingerstick Results Within International Organization for Standardization 15197:2003 Minimum Acceptable Performance Criteria and More Stringent Criteria

Glucose concentration, data comparison	Number of readings within specified error limits
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<75 mg/dL, HCP vs. YSI (n=20)	19 (95.0%)	20 (100%)	20 (100%)	20 (100%)^a	20 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥75 mg/dL, HCP vs. YSI (n=163)	135 (82.8%)	160 (98.2%)	161 (98.8%)	162 (99.4%)	162 (99.4%)^a
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<100 mg/dL, HCP vs. YSI (n=44)	40 (90.9%)	42 (95.5%)	43 (97.7%)	44 (100%)^b	44 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥100 mg/dL, HCP vs. YSI (n=139)	117 (84.2%)	138 (99.3%)	138 (99.3%)	138 (99.3%)^b	138 (99.3%)

Within the International Organization for Standardization (ISO) 15197:2003 minimum acceptable performance criteria.⁸

Within the more stringent criteria.^9
–11

HCP, healthcare professional; YSI, Yellow Springs Instruments.

Table 5.

Venous Results Within International Organization for Standardization 15197:2003 Minimum Acceptable Performance Criteria and More Stringent Criteria

Glucose concentration, data comparison	Number of readings within specified error limits
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<75 mg/dL, HCP vs. YSI (n=16)	14 (87.5%)	16 (100%)	16 (100%)	16 (100%)^a	16 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥75 mg/dL, HCP vs. YSI (n=170)	141 (82.9%)	168 (98.8%)	170 (100%)	170 (100%)	170 (100%)^a
	± 5 mg/dL	± 10 mg/dL	± 12.5 mg/dL	± 15 mg/dL	± 20 mg/dL
<100 mg/dL, HCP vs. YSI (n=46)	37 (80.4%)	46 (100%)	46 (100%)	46 (100%)^b	46 (100%)
	± 5%	± 10%	± 12.5%	± 15%	± 20%
≥100 mg/dL, HCP vs. YSI (n=140)	120 (85.7%)	139 (99.3%)	140 (100%)	140 (100%)^b	140 (100%)

Within the International Organization for Standardization (ISO) 15197:2003 minimum acceptable performance criteria.⁸

Within the more stringent criteria.^9
–11

HCP, healthcare professional; YSI, Yellow Springs Instruments.

Clinical accuracy was demonstrated by Parkes consensus error grid analyses. Overall, 98.9% (180/182) of subject Microlet2 fingerstick results were within Zone A; the remainder were within Zone B (Fig. 2A). For HCP Tenderlett fingerstick results, 99.5% (182/183) were within Zone A; the remainder was within Zone B (Fig. 2B). For venous samples, 100% (186/186) of results were within Zone A (Fig. 2C).

FIG. 2.

Parkes consensus error grid analyses of blood glucose monitoring system results versus laboratory reference results for (A) subject Microlet2 fingerstick, (B) healthcare professional (HCP) Tenderlett fingerstick, and (C) venous blood samples. YSI, Yellow Springs Instruments.

Subject questionnaire and labeling evaluation

Of the 94 subjects who completed the subject questionnaire and labeling evaluation, 93 (98.9%) rated their overall satisfaction with the BGMS as “good” to “excellent”; the remaining subject rated his or her overall satisfaction as “poor.” Similarly, 97.9% (92/94) of subjects rated the clarity of user guide instructions as “good” to “excellent”; the remaining two subjects rated the clarity of user guide instructions as “poor.” Additional subject ratings are summarized in Figure 3.

FIG. 3.

Percentage of subject ratings of (A) ease of use and (B) specific features of the new blood glucose monitoring system.

Subjects were rated by HCPs on their ability to perform basic tasks with the BGMS, including turning the system on and off, setting the date and time, performing a control solution test, and performing a fingerstick self-test. Overall, 97.9% (92/94) of subjects were able to perform all tasks either without assistance from the HCP, with additional review of the user guide, or with verbal cues from the HCP directing the subject to review the user guide. Only two subjects performed one of the tasks incorrectly (setting the date and time).

Adverse events

Overall, 14 non-serious, non–device-related adverse events were reported (hypoglycemia, n=13; feeling faint at the sight of blood, n=1). All adverse events were anticipated, considered mild, and resolved before subjects left the clinic.

Discussion

The accuracy of SMBG devices is becoming increasingly important, especially in light of more stringent guidelines under discussion by the ISO and other regulatory agencies as well as efforts to help people with diabetes achieve tighter glycemic control. Several factors can affect the accuracy of blood glucose results. Results can vary depending on whether testing is done by trained (HCPs) or untrained (subjects) users as well as the source of the blood sample (e.g., capillary or venous). In many cases, variations between HCP and subject results can be attributed to differences in training and familiarity with proper testing technique.¹² Ease of use of the BGMS must also be considered because it may allow untrained users to obtain valid blood glucose results and eliminate some of the sources of error associated with at-home versus in-clinic testing.

Results from this study showed that the BGMS in development for use with a new generation of test strips is accurate in different clinical settings, whether the blood sample was obtained by trained HCPs from either capillary or venous sampling or by untrained subjects from capillary fingersticks. Almost all of the subject Microlet2 fingerstick (98.9%) and HCP Tenderlett fingerstick (99.5%) capillary results and 100% of venous results met both current ISO criteria⁸ and proposed more stringent criteria.^9
–11 There was no difference seen in the proportion of accurate results obtained using the different blood collection methodologies, but it should be noted that all comparisons between meter and reference results were made using like samples (e.g., capillary to capillary, venous to venous). Furthermore, there was no difference seen in the proportion of accurate results obtained from trained or untrained users. Parkes consensus error grid analysis¹⁸ showed that 100% of capillary and venous results were within Zone A or Zone B, indicating no effect on clinical outcome. Furthermore, based on questionnaire results, the majority of subjects were satisfied with the ease of use and features of the BGMS. The labeling evaluation showed that untrained subjects were able to learn to use the BGMS using only the labeling information provided.

In conclusion, the new BGMS demonstrated a high level of accuracy in the hands of intended users, both people with diabetes and HCPs. Glucose results met current ISO guidelines as well as proposed tighter accuracy criteria, regardless of the blood collection methodology used to obtain the sample—venous, HCP Tenderlett capillary fingerstick, or subject Microlet2 capillary fingerstick. In addition, subjects found the BGMS easy to use and were able to obtain accurate results without prior training. The accuracy of this new device, therefore, can be expected to be similar in the hands of people with diabetes at home or in the HCP office. The higher level of accuracy found in this study demonstrates a significant improvement compared with data generated with prior generations of BGMSs. Because BGMSs play a critical role in diabetes care, including determining insulin doses and serving as a calibration reference for continuous glucose monitoring devices, the findings of this study demonstrate potential for improvements in the care of people with diabetes.

Footnotes

Acknowledgments

This study was supported by Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY. Medical writing assistance was provided by Allison Michaelis, Ph.D., of MedErgy, Yardley, PA, and was supported by Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY.

Author Disclosure Statement

T.S.B. has received honoraria from Roche, sanofi-aventis, Novo Nordisk, and Amylin as well as research support from Animas, Bayer, BD, Boehringer Ingelheim, Corcept, Bristol-Myers Squibb, Dexcom, GlaxoSmithKline, Halozyme, Lifescan, Lilly, Medtronic, Merck, Novo Nordisk, Resmed, Roche, sanofi-aventis, and Xoma. J.F.W., J.L.P., S.P., J.Y., H.C.S., D.A.S., and A.C. are full-time employees of Bayer HealthCare LLC, Diabetes Care.

References

Murata

, Duckworth

, Shah

, Wendel

, Mohler

, Hoffman

. Blood glucose monitoring is associated with better glycemic control in type 2 diabetes: a database study. J Gen Intern Med, 2009; 24:48–52.

Guerci

, Drouin

, Grange

, Bougneres

, Fontaine

, Kerlan

, Passa

, Thivolet

, Vialettes

, Charbonnel

. Self-monitoring of blood glucose significantly improves metabolic control in patients with type 2 diabetes mellitus: the Auto-Surveillance Intervention Active (ASIA) study. Diabetes Metab, 2003; 29:587–594.

Karter

, Ackerson

, Darbinian

, D'Agostino

Jr , Ferrara

, Liu

, Selby

. Self-monitoring of blood glucose levels and glycemic control: the Northern California Kaiser Permanente Diabetes Registry. Am J Med, 2001; 111:1–9.

American Diabetes Association. Clinical practice recommendations. Diabetes Care, 2010; 33,Suppl 1:S1–S99.

Fera

, Bluml

, Ellis

. Diabetes Ten City Challenge: final economic and clinical results. J Am Pharm Assoc (2003), 2009; 49:383–391.

Martin

, Schneider

, Heinemann

, Lodwig

, Kurth

, Kolb

, Scherbaum

. Self-monitoring of blood glucose in type 2 diabetes and long-term outcome: an epidemiological cohort study. Diabetologia, 2006; 49:271–278.

Bode

. The accuracy and interferences in self-monitoring of blood glucose. US Endocr Rev, 2007; 2:46–48.

International Organization for Standardization: ISO 15197. In Vitro Diagnostic Test Systems—Requirements for Blood-Glucose Monitoring Systems for Self-Testing in Managing Diabetes Mellitus. Geneva: International Organization for Standardization, 2003.

American Diabetes Association. Self-monitoring of blood glucose (consensus statement) Diabetes Care, 1994; 17:81–86.

10.

Klonoff

. The Food and Drug Administration is now preparing to establish tighter performance requirements for blood glucose monitors. J Diabetes Sci Technol, 2010; 4:499–504.

11.

Klonoff

. Regulatory controversies surround blood glucose monitoring devices. J Diabetes Sci Technol, 2010; 4:231–235.

12.

Ginsberg

. Factors affecting blood glucose monitoring: sources of errors in measurement. J Diabetes Sci Technol, 2009; 3:903–913.

13.

Kumar

, Sng

, Kumar

. Correlation of capillary and venous blood glucometry with laboratory determination. Prehosp Emerg Care, 2004; 8:378–383.

14.

Boyd

, Leigh

, Stuart

. Capillary versus venous bedside blood glucose estimations. Emerg Med J, 2005; 22:177–179.

15.

Shearer

, Boehmer

, Closs

, Dela

, Hamilton

, Horton

, McGrath

, Schulman

. Comparison of glucose point-of-care values with laboratory values in critically ill patients. Am J Crit Care, 2009; 18:224–230.

16.

Colagiuri

, Sandbaek

, Carstensen

, Christensen

, Glumer

, Lauritzen

, Borch-Johnsen

. Comparability of venous and capillary glucose measurements in blood. Diabet Med, 2003; 20:953–956.

17.

Petersen

, Graves

, Tacker

, Okorodudu

, Mohammad

, Cardenas

Jr . Comparison of POCT and central laboratory blood glucose results using arterial, capillary, and venous samples from MICU patients on a tight glycemic protocol. Clin Chim Acta, 2008; 396:10–13.

18.

Parkes

, Slatin

, Pardo

, Ginsberg

. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care, 2000; 23:1143–1148.