A Review of Blood Glucose Monitor Accuracy

Abstract

The aim of this study was to assess the accuracy of blood glucose monitors (BGMs) from studies reported in the medical literature. A literature review was performed of publications between 2010 and 2017 that presented data about the accuracy of BGMs using ISO 15197 2003 and/or ISO 15197 2013 as target standards. We found 58 publications describing the performance of 143 unique BGM systems, 59 of which were Food and Drug Administration (FDA) cleared. When compared with non-FDA-cleared BGMs, FDA-cleared BGMs were significantly more likely to pass both ISO 15197 2003 (OR = 2.39, CI 1.45–3.92, P < 0.01) and ISO 15197 2013 standards (OR = 2.20, CI 1.51–3.27, P < 0.01). Newer meters were more likely to pass both ISO 15197 2003 and ISO 15197 2013 standards. Many of the studies were supported by BGM manufacturers, and when compared with independent studies, an FDA-cleared BGM was significantly more likely to pass in a manufacturer-supported study for both ISO 15197 2003 (OR = 22.4, CI 8.73–21.57, P < 0.001) and ISO 15197 2013 (OR = 23.08, CI 10.16–60.03, P < 0.001). BGM accuracy should be assessed independently following regulatory clearance to ensure accurate performance. Failure to meet performance levels mandated by standards can result in deleterious clinical and economic effects.

Introduction

Self-monitoring of blood glucose (SMBG) with blood glucose monitor (BGM) systems plays a central role in the optimal management of diabetes mellitus. Many BGMs on the international market today, however, do not perform according to the current international standard¹ set by the International Organization for Standardization (ISO), known as ISO 15197 2013,² or even to its less rigorous predecessor, ISO 15197 2003.³ Since 2003 in the United States, ISO 15197 2003 had been used for all BGMs intended to perform SMBG until the Food and Drug Administration (FDA) released a draft guidance in 2014 for personal over-the-counter BGMs,⁴ which was replaced by a final guidance in 2016.⁵

The analytical accuracy requirements for compliance with ISO 15197 2003 (previously recognized both internationally and by FDA), ISO 15197 2013 (recognized internationally but not by FDA) and FDA 2016 Over-The-Counter (recognized by FDA but not internationally) are found in Table 1.

Table 1.

International and Food and Drug Administration Analytical Accuracy Requirements for Blood Glucose Monitors for Self-Monitoring

Accuracy standard	Minimum accuracy requirements
ISO 15197 2003	95% of glucose values must be within ±15 mg/dL of reference for values <75 mg/dL or within 20% of reference for values ≥75 mg/dL
ISO 15197 2013	95% of glucose values must be within ±15 mg/dL of reference for values <100 mg/dL or within 15% of reference for values ≥100 mg/dL
FDA 2016 for over-the-counter BGMs (FDA 2016)	95% of glucose values must be within 15% of reference, and 99% of glucose values must be within 20% of reference

BGMs, blood glucose monitors; FDA, Food and Drug Administration.

Additionally, for ISO 15197 2013, there is a clinical accuracy requirement that ≥99% of results must fall within zones A and B of the Parkes Error Grid.

Some BGMs that have previously been cleared by the FDA have been tested by independent investigators with protocols similar to (but not necessarily exactly the same as) those used by the manufacturer in pivotal trials, and have been found to not meet the latest standards (ISO 15197 2013 and/or FDA 2016 for personal BGMs). Furthermore, some do not even meet the older ISO 15197 2003 standard under which they were initially cleared.⁶

If BGMs do not meet accuracy standards, then they pose a risk to the patients that rely on this information for: (1) making treatment decisions about insulin dosing and taking other actions based on the blood glucose level⁷; (2) calibrating continuous glucose monitors which can inform treatment decisions⁸; and (3) controlling closed-loop insulin delivery systems that depend on accurate calibration of continuous glucose monitors.⁹

To analyze the accuracy of BGMs, we reviewed current research from 2010 through July 2017 addressing the accuracy of personal BGMs according to the ISO 15197 2003, ISO 15197 2013, and FDA 2016 standards. The results were analyzed according to the following five factors: (1) was the BGM cleared by the FDA; (2) if cleared, then what was the year of clearance; (3) what was the BGM test strip chemistry; (4) was the article either: (a) a manufacturer's study, which we defined as a BGM's manufacturer either performing the testing or not being declared to have been excluded from significant participation in the design of the study protocol for an independent investigator (defined as not employed by the manufacturer) to test its product; or (b) not a manufacturer's study, which we called an independent study; and (5) if a product was tested in a manufacturer's study, then was the product tested by its own manufacturer.

Methods

In 2015, Klonoff and Prahalad identified 11 studies published between 2010 and 2014 that addressed the accuracy of BGMs relative to ISO 15197 2003,² ISO 15197 2013,¹ or the then-current FDA 2014 draft guidance for over-the-counter BGM accuracy⁴ (that was later replaced by a final version in 2016⁵). To update the information presented in that review, we performed a new literature search.

We determined that to be included in our review, articles must be: (1) written in English; (2) published between 2010 and July 31, 2017; (3) include the name of the BGM product tested; (4) be accessible in the PubMed database; and (5) address the analytical accuracy of a BGM against ISO 15197 2003, ISO 15197 2013, and/or FDA over-the-counter 2016 by comparing the values obtained by each system with a reference method. Accessibility in PubMed was defined as including a link to purchase the article or open access. Articles were included only if they addressed BGM accuracy in humans (as opposed to animals). We also excluded articles with limited study populations (including those addressing only professional BGMs for hospital use or testing the performance of BGMs only on pregnant women), articles focusing on the performance of continuous glucose monitors, and articles that only used mean absolute relative difference (MARD) as a metric of BGM analytical accuracy instead of the criteria defined by ISO and FDA.

For articles that addressed BGMs designed for personal self-monitoring use and other BGMs intended for professional use, data were extracted for only the self-monitoring (personal use) portion. Articles were also excluded if they only used venous blood as the substrate to test BGM systems, although articles using venous blood for the comparator method were included. In studies where data from laboratory accuracy studies (healthcare professionals assessed BGMs) and user performance accuracy studies (lay users and healthcare professionals assessed BGMs) was reported, they were treated as separate studies. For the user performance accuracy studies, data from lay users and healthcare professionals were combined. A BGM needed to pass ISO requirements in both studies to receive a “pass” for that article. It should also be noted that we did not exclude articles in which the total number of data points and/or the distribution of data points did not conform to ISO recommendations.^2,3 In some articles the reference method used differs from the one recommended by the manufacturer.^{10

–24} If a BGM was compared against two different reference methods, then the BGM was assessed with respect to the reference method specified by the manufacturer.

To begin our search, we used PubMed and Google Scholar to review the initial 11 articles which were in the previous review article on this topic by Klonoff and Prahalad. We also reviewed all the references from each of these 11 articles. We then used Google Scholar to find any newer articles that cited any of the original 11. From these searches, we identified 19 new articles that met our inclusion criteria. Next, we reviewed the references from these 19 articles and performed a Google Scholar search to find even newer articles that cited any of these 19. This step yielded another 23 new articles for inclusion. We then repeated the review of references and Google Scholar search on these 23 articles and identified 5 more articles for inclusion. We repeated the review of the references and Google Scholar search on these 5 articles and found no new articles for inclusion.

In addition, a PubMed search was performed on July 12, 2017 for “blood glucose monitor analytical accuracy” with a publication date between January 1, 2010 and July 12, 2017. Forty-one articles were returned, but 36 did not meet the inclusion criteria. Three articles had already been discovered in the literature review based off of Klonoff and Prahalad, so only two new articles were found whose eligibility for inclusion was confirmed by two reviewers (F.K. and V.H.). We repeated our searches on August 7, 2017 (with publication date set from January 1, 2010 to August 7, 2017) and no new articles were returned. 58 articles in total met our inclusion criteria (see Table 7).

BGMs were determined to be FDA cleared if they were accessible in the FDA 510(k) searchable database²⁵ or the CLIA database.²⁶ We searched each BGM name and extracted the clearance year and 510(k) number from each entry. BGMs were marked “pass” if they were within the minimum accuracy limits in every trial of the studies in which they were tested. For example, if three lots were tested against ISO 15197 2013 in one study, then all three lots had to individually satisfy the 95% accuracy criteria for the BGM to receive a “pass” for that study. A BGM received either a pass or fail for each study in which it was assessed. This approach was used to determine the pass rate (N studies passed/N studies reported) for each of the subgroups of BGMs analyzed in this review (FDA-cleared, clearance date, etc.). For each study, we determined the number of tests per study (N tested) and the number passing ISO 15197 2003 and/or ISO 15197 2013 (N compliant). From these two numbers we were able to generate the compliance proportion (N compliant/N tested). Finally, we determined whether each study was manufacturer supported, and if so, whether the product was tested by its own manufacturer.

BGMs passing ISO 15197 2013 are assumed to pass ISO 15197 2003 because ISO 15197 2013 is stricter than ISO 15197 2003; however, those that fail ISO 15197 2013 are not assumed to pass or fail ISO 15197 2003. Similarly, in studies that only address ISO 15197 2003, BGMs that fail are assumed to also fail ISO 15197 2013; however, for BGMs that passed ISO 15197 2003, no assumption is made about ISO 15197 2013. Similarly, a product failing to meet ISO 15197 2013 would fail the FDA 2016 guidance; however, a product passing ISO 15197 2013 will not necessarily pass the FDA 2016 guidance. Of note, we only found a single study in the current literature that addressed the analytical accuracy of BGM systems with respect to the recently published FDA 2016 guidance, and this study showed identical results when using the ISO 15197 2013 and FDA 2016 standards.¹⁰ Evaluation against the two ISO standards remains most practical at this current point in time for assessing compliance with current and past BGM analytical accuracy levels.

We conducted bias-reduced logistic regression (Firth²⁷) to analyze the relationships between predictors of passing each standard, using pigeonhole bootstrap (Owen²⁸) to adjust for clustering of meters and studies. Statistical significance is defined as a P-value of 0.05 or less. We used a Spearman correlation to measure monotone relationships with respect to manufacturer bias.

Results

One hundred forty-three different BGMs were reviewed in a total of 58 studies.^{10

–24,29

–71} The minimum number of BGMs analyzed per study was 1, and the maximum was 43. The mean was 4.86 BGMs per study.

Around 93/137 (67.9%) of evaluated BGMs passed ISO 15197 2003, and 55/136 (40.4%) evaluated BGMs passed ISO 15197 2013. Out of the 143 total BGMs that we reviewed, 59 BGMs had received FDA clearance. Around 43/57 (75.4%) of FDA-cleared BGMs passed ISO 15197 2003, whereas only 26/56 (46.4%) of FDA-cleared BGMs passed the stricter ISO 15197 2013 standard. Among non-FDA-cleared BGMs, 50/80 (62.5%) passed ISO 15197 2003 and 29/80 (36.3%) passed ISO 15197 2013. FDA-cleared BGMs were found to be significantly more likely to meet both the ISO 15197 2003 (OR = 2.39, CI 1.45–3.92, P < 0.01) and ISO 15197 2013 standards (OR = 2.20, CI 1.51–3.27, P < 0.01).

We also analyzed the 59 FDA-cleared BGMs with respect to their year of clearance. The range of clearance years spanned from 2001 to 2017. We divided these BGMs into four groupings of 4-year increments, with the exception of the oldest category, which was a 5-year increment. The results are presented in Table 2 and Figure 1. For BGMs cleared from 2001 to 2005, 4/7 (57.1%) passed ISO 15197 2003, whereas 3/8 (37.5%) passed ISO 15197 2013. For BGMs cleared between 2006 and 2009, 13/23 (56.5%) passed ISO 15197 2003, whereas 7/23 (30.4%) passed ISO 15197 2013. For BGMs cleared between 2010 and 2013, 11/12 (91.7%) passed ISO 15197 2003, and 6/12 (50%) passed ISO 15197 2013. For BGMs cleared between 2014 and 2017, 15/15 (100%) passed ISO 15197 2003, and 10/13 (76.9%) passed ISO 15197 2013. BGMs approved more recently performed significantly better for both ISO 15197 2003 (OR 1.06, CI 1.06–1.07, P < 0.001, odds ratio reported for a 1-year period) and ISO 15197 2013 standards (OR = 1.06, CI 1.03–1.06, P < 0.01, odds ratio reported for a 1-year period).

FIG. 1.

Percentage of BGMs passing ISO 2003 and ISO 2013 over four intervals of time. BGMs, blood glucose monitors. (Color graphics available at www.liebertonline.com/dia)

Table 2.

Performance of Blood Glucose Monitor Systems Stratified by Clearance Year Intervals

Always means a BGM passed in at least one study but also failed in at least one study.

In addition, FDA-cleared meters and the total sample were analyzed and sorted by their test strip enzyme: either glucose dehydrogenase (GDH) or glucose oxidase (GOX). We found that among our entire cohort of tested BGMs, those that used a GDH test strip chemistry outperformed those that used GOX test strip chemistry for both ISO 15197 2003 and ISO 15197 2013. Around 46/57 (80.7%) and 32/55 (58.2%) of GDH systems passed ISO 15197 2003 and ISO 15197 2013, respectively (Table 3), whereas only 47/80 (58.8%) and 23/81 (28.4%) of GOX systems passed ISO 15197 2003 and 2013, respectively (Table 4). GDH's improved performance over GOX was statistically significant for both the ISO 15197 2003 (OR = 0.23, CI 0.13–0.38, P < 0.001) as well as the ISO 15197 2013 standard (OR = 0.24, CI 0.15–0.35, P < 0.001). This same statistically significant trend occurred when only considering FDA-cleared meters, with 25/28 (89.3%) of FDA-cleared BGMs using any type of GDH chemistry passing ISO 15197 2003, and 16/25 (64%) passing ISO 15197 2013. For BGMs using GOX chemistry, 18/29 (62.1%) passed ISO 15197 2003, whereas 10/31 (32.3%) passed ISO 15197 2013. FDA-cleared BGMs using GDH significantly outperformed FDA-cleared BGMs using GOX in both ISO 15197 2003 (OR = 0.17, CI 0.06–0.38, P < 0.001) and ISO 15197 2013 (OR = 0.24, CI 0.13–0.45, P < 0.001). The effect of test strip enzyme on accuracy as it relates to clearance year is presented in Figure 2.

FIG. 2.

Percentage of BGMs passing ISO 2003 and ISO 2013 based on clearance year and test strip enzyme. (Color graphics available at www.liebertonline.com/dia)

Table 3.

Performance of Blood Glucose Monitor Systems Using Glucose Dehydrogenase Test Strip Chemistry (With and Without Cofactors)

Always means a BGM passed in at least one study but also failed in at least one study.

GDH, glucose dehydrogenase; FAD, flavin adenine dinucleotide; NAD, nicotinamide adenine dinucleotide; PQQ, pyrroloquinoline quinone.

Table 4.

Performance of Blood Glucose Monitor Systems Using Glucose Oxidase Test Strip Chemistry

Always means a BGM passed in at least one study but also failed in at least one study.

GOX, glucose oxidase.

We also looked at the association of the recently popular GDH enzymatic cofactors flavin adenine dinucleotide (FAD) and nicotinamide adenine dinucleotide (NAD) compared with GDH without these new cofactors and containing the earlier used cofactor, pyrroloquinoline quinone (PQQ), on performance (Tables 5 and 6). With respect to all analyzed BGMs using GDH-FAD or GDH-NAD chemistry irrespective of FDA clearance status, 25/31 (80.6%) passed ISO 15197 2003, and 18/28 (64.3%) passed ISO 15197 2013. Among all analyzed BGMs using GDH but not specifying FAD or NAD as the enzymatic cofactor irrespective of FDA status, 21/26 (80.8%) passed ISO 15197 2003 and 13/27 (48.1%) passed ISO 15197 2013. We do not find evidence that FAD/NAD is more likely to be associated with passing the ISO 2003 standard (OR = 0.66, CI 0.24–1.50, P = 0.29) or ISO 2013 standard (OR = 0.91, CI 0.50–1.65, P = 0.76). Among FDA-cleared BGMs using GDH-FAD or GDH-NAD, 15/17 (88.2%) passed ISO 15197 2003, and 9/14 (64.3%) passed ISO 15197 2013. Among FDA-cleared BGMs using GDH but not specifying FAD or NAD as the cofactor, 10/11 (90.1%) passed ISO 15197 2003 and 7/11 (63.6%) passed ISO 15197 2013. These differences between BGMs with GDH enzymes containing FAD or NAD compared with those containing PQQ for noncleared BGMs were also not statistically significant for the ISO 2003 standard (OR = 0.51, CI 0.15–3.24, P = 0.38) or the ISO 2013 standard (OR = 0.62, CI 0.23–1.48, P = 0.29).

Table 5.

Performance of Blood Glucose Monitor Systems Using Glucose Dehydrogenase Test Strip Chemistry Without Cofactors or Using Pyrroloquinoline Quinone Cofactor

Always means a BGM passed in at least one study but also failed in at least one study.

Table 6.

Performance of Blood Glucose Monitor Systems Using Glucose Dehydrogenase Test Strip Chemistry with Flavin Adenine Dinucleotide or Nicotinamide Adenine Dinucleotide Cofactor

Always means a BGM passed in at least one study but also failed in at least one study.

Among FDA-cleared products tested in manufacturer-supported studies,^{11,14

–21,23,29

–65,67

–71} 95/107 (88.8%) passed in tests as per ISO 15197 2003 and in 59/88 (67%) passed in tests as per ISO 15197 2013. FDA-cleared products tested in independent studies^{10,12,13,21,24,66} passed in 19/23 (82.6%) of tests as per ISO 15197 2003 and in 16/27 (59.3%) of tests as per ISO 15197 2013. The likelihood of an FDA-cleared product passing in a manufacturer-supported test compared with an independent test was significantly greater according to both ISO 15197 2003 (OR = 22.4, CI 8.73–21.6, P < 0.001) and ISO 15197 2013 (OR = 23.08, CI 10.2–60.0, P < 0.001). For non-FDA-cleared products tested in manufacturer-supported studies, 78/106 (73.6%) passed in tests as per ISO 15197 2003, and 48/95 (50.5%) passed in tests as per ISO 15197 2013. The likelihood of a noncleared product passing in a manufacturer-supported test compared with an independent test was significantly greater according to both ISO 15197 2003 (OR = 37.8, CI 15.7–35.8, P < 0.001) and ISO 15197 2013 (OR = 163, CI 68.4–158, P < 0.001).

To assess whether other products tested in a manufacturer-supported study besides the manufacturer's own products were more or less likely to receive a passing score than products tested by their own manufacturer, and we determined the passing percentage for products made by competing manufacturers compared with products tested by their own manufacturer. Seventeen manufacturer-supported studies also tested one or more products from other manufacturers.^{14,17,18,20,23,33,36,38,46

–50,59,63,70,71} We also compared this with independent tests to investigate the possibility of a monotonic relationship between manufacturers testing their own products, independent tests, and manufacturers testing other products. We would expect there to be a relationship in that order from most passing to least passing. The passing percentages for these three classes of products (against ISO 15197 2003 and ISO 15197 2013 were 80/80 (100%) and 67/68 (98.5%) for products' own manufacturer, 32/39 (82.1%) and 28/63 (44.4%) for independent tests, and 93/133 (69.9%) and 40/115 (34.8%) for alternative manufacturers. We concluded that the difference between independent tests and tests by alternative manufacturers was statistically significant as per ISO 15197 2003 (OR = 1.88, CI 1.02–4.08, P = 0.04), but was not statistically significant as per ISO 15197 2013 (OR = 1.50, CI 0.93–2.41, P = 0.08). We also concluded that there was a statistically significant difference between manufacturers testing their own products versus a competitor's product according to both ISO 15197 2003 (OR = 37.2, CI 4.62–299, P < 0.01) and ISO 15197 2013 (OR = 56.1, CI 11.9–263, P < 0.01).

Discussion

The purpose of this review was to analyze factors contributing to BGM compliance with ISO and FDA 2016 guidelines for analytical accuracy in hopes of promoting development and use of BGMs with greater accuracy. The factors analyzed were FDA clearance (yes or no), time period of clearance (2001–2005, 2006–2009, 2010–2013, or 2014–2017), the type of chemistry used by the BGM (GOX vs. GDH and GDH-FAD and GDH-NAD vs. GDH not using FAD or NAD), and manufacturer support of the study (which could potentially introduce bias).

We would expect FDA-cleared BGMs to perform better than non-FDA-cleared BGM's in the medical literature, and FDA-cleared BGMs did in fact perform significantly better than noncleared BGMs. Even so, only 75.4% of FDA-cleared BGMs managed to pass the older ISO 15197 2003 standard by which they had initially been cleared. When evaluated according to the newer ISO 15197 2013 standard, only 46.4% of FDA-cleared BGMs passed the test. Furthermore, we can deduce that even fewer would pass the more rigorous 2016 FDA Guidance.

One possible limitation to our approach is a slight bias toward the fail mark. BGMs that are close to meeting the ISO accuracy standards, and may be on average meeting them, will be marked as “fail” in our review if one lot falls below the requirement. Our review was designed as such because patients depend on every lot fulfilling minimum accuracy standards.

Some studies adhered strictly to ISO recommendations for the distribution of data points, and BGMs in these studies may also be at a disadvantage when compared with those who used tighter distributions with fewer extreme blood glucose (BG) values at which some BGMs may be less accurate. More studies are needed to determine the accuracy of BGMs at extreme hyperglycemic and hypoglycemic ranges.

Many stakeholders in the diabetes community expect or assume that all cleared meters will perform to the standards under which they were approved, as well as to current standards. Parkin⁷² speculates that many FDA-cleared BGMs are failing to meet the current standard because: (1) perhaps in some cases quality standards in the manufacturing process have declined or (2) perhaps in some cases companies are falsifying data to present to the FDA (who might lack the resources to investigate, especially when the companies are overseas). We recommend that additional surveillance studies be conducted on BGMs that have already been cleared by the FDA to ensure that BGMs continue to adhere to accuracy standards following the initial clearance. Failure to remain accurate over time can put some patients who depend on this technology at risk of harm.

With respect to how long an existing blood glucose monitoring system (BGMS) has been on the market and cleared by the FDA, our analysis showed that newer BGMs performed significantly better for both ISO 15197 2003 (OR 1.06, CI 1.06–1.07, P < 0.001, odds ratio reported for a 1-year period) and ISO 15197 2013 standards (OR = 1.06, CI 1.03–1.06, P < 0.01, odds ratio reported for a 1-year period). A point of caution, however, is that while newer BGMs certainly outperformed older BGMs, 23.1% of BGMs cleared by the FDA in the most recent 4 years still did not meet the ISO 15197 2013 standard, and even fewer would likely pass the more rigorous 2016 FDA guidance.

Which technological advances are contributing most to the improved performance of newer SMBGs? Pfützner⁷³ identified the following four factors: (1) changes in test strip chemistry; (2) the elimination of hematocrit interference; (3) improved electrode design; and (4) improved mathematical algorithms. Our review demonstrated that approved BGMs utilizing GDH as their test strip enzyme significantly outperformed those using GOX for ISO 15197 2003 (OR = 0.23, CI 0.13–0.38, P < 0.001) and ISO 15197 2013 (OR = 0.24, CI 0.15–0.35, P < 0.001), which could be why GDH is increasingly replacing GOX over time as the test strip enzyme of choice. Unlike GOX meters, BGMs utilizing GDH are not dependent on the partial pressure of oxygen in the blood.⁷⁴ Looking deeper, our analysis of FAD/NAD as an enzymatic cofactor for GDH (as opposed to the older PQQ cofactor) did not find a significant correlation with compliance to either ISO standards (ISO 15197 2003 OR = 0.66, CI 0.24–1.50, P = 0.29; ISO 15197 2013 OR = 0.91, CI 0.50–1.65, P = 0.76). It is also possible that BGMs using GDH are on average newer than those using GOX and the perceived improved accuracy of BGMs using GDH cannot be entirely attributed to the enzyme itself.

Lastly, our analysis of manufacturer-supported studies revealed that BGMs supported by the manufacturer significantly outperformed BGMs from other manufacturers, both according to ISO 15197 2003 (OR = 37.2, CI 4.62–299, P < 0.01) and ISO 15197 2013 (OR = 56.1, CI 11.9–263, P < 0.01). While some studies we reviewed were performed by independent investigators, many were carried out or funded by a manufacturer with protocol input (to the extent that the manufacturer was not declared to have been excluded from significant participation in the design of the study protocol for an independent investigator). We think it is possible that a manufacturer might be more likely to publish a study with favorable results and less likely to publish such a study with unfavorable results regarding the accuracy of their BGM. Therefore, it is possible that even fewer BGMs would pass ISO or FDA guidance if all studies were completed by independent investigators and not subject to being selectively reported by a manufacturer.

Additionally, even in independent studies, supplies (such as the BGM system and test strips) are frequently supplied by the manufacturer. We believe that BGM supplies for accuracy studies should ideally be obtained through the same avenues that are used by patients, such as retail or mail-order pharmacies. Doing so would best recreate a patient's “real-world” experience and prevent the possibility of nonrandom selection of products.

The most accurate assessment of a blood glucose monitoring system (BGMS)-measuring capillary blood is obtained by also measuring capillary blood on a comparator method. In some cases, comparison between capillary blood tested on a BGM and venous blood tested on a comparator method could have a negative impact on the perceived accuracy of a BGMS. In other instances, the difference between the two matrices could cause a BGM to appear to be more accurate than it actually is. Whether the impact would be negative or positive would depend on whether the BGM-measuring capillary blood is biased away from or toward the direction where the reference method measuring venous blood is biased.⁷⁵

BGM accuracy is of the utmost importance because patients rely on the information obtained from their BGMs to make treatment decisions. When results obtained are inaccurate, clinical consequences can be potentially life threatening and the economic repercussions can be costly. Modeling studies have demonstrated the adverse outcomes of inaccurate BGMs, such as hypoglycemia and overall poor glucose control⁷⁶ and insulin dosing errors.⁷

Table 7.

Studies Included with N Blood Glucose Monitors and N Lots Tested

Study name	Publication year	PubMed ID	Manufacturer supported?	N BGMs assessed	LotN2003	LotN2013
Angelides et al.	2015	26002836	Yes	1	3505	3505
Bailey et al.^a	2012	22853721	Yes	1	1152
Bailey et al.^a	2015	26070512	Yes	1		1036
Bailey et al.^a	2017	28617617	Yes	1		868
Baumstark et al.	2012	23063033	Yes	5	800	800
Baumstark et al.	2014	25107707	Yes	1		600
Baumstark et al.	2016	26428935	Yes	1		600
Baumstark et al.^b	2017	28704063	Yes	6
Berti et al.	2015	26224761	Yes	6		200
Brazg et al.	2013	23439170	Yes	7	600^b	600^b
Brazg et al.	2016	27271844	Yes	1		600
Caswell et al.^a	2015	25494048	Yes	1	1033	1033
Chandnani et al.	2017	28658800	Yes	1		1005
Charoensri et al.	2015	26591392	Yes	1	153	153
Chen et al.	2012	22266375	Yes	1
Christiansen et al.^a	2017	28745099	Yes	1		1261
Frank et al.	2011	21303645	Yes	1	838
Freckmann et al.	2010	20151773	Yes	25	200^b
Freckmann et al.	2012	23063032	Yes	43	1
Freckmann et al.	2014	24205978	Yes	10	180–200	180–200
Freckmann et al.	2014	24681433	Yes	1	600	600
Freckmann et al.	2015	26110670	Yes	9	600	600
Freckmann et al.^a	2017	28225633	Yes	4	300	300
Harrison et al.	2011	21880245	Yes	1	600	600
Hasslacher et al.	2013	24133899	No	19		238–303
Hasslacher et al.	2014	24876607	No	27		248–303
Hehmke et al.	2017	27707915	Yes	3		200
Huang et al.	2014	24613515	Yes	2		224/672
Jeanny and Hope	2016	26424240	Yes	2	600	600
Jendrike et al.	2015	25901021	Yes	2	600	600
Jendrike et al.	2017	28459160	Yes	1		600
Ji et al.	2016	26053011	Yes	1		362
Kamecke et al.	2017	28627219	Yes	1		600
Katz et al.^a	2015	25702769	Yes	5		773–876
Katz et al.^a	2017	28459033	Yes	1		600
Kuo et al.	2011	21406013	Yes	6	107
Link et al.	2014	24876597	Yes	3	600	600
Link et al.	2015	25872967	Yes	4	600	600
List et al.^a	2011	22027308	Yes	1	296
Lock et al.^a	2011	21142975	Yes	1	394
Pfützner et al.	2012	22435798	Yes	6	212
Pfützner et al.	2014	24156775	Yes	6	212^b
Pfützner	2016	26553022	Yes	1		630
Philis-Tsimikas et al.^a	2011	22226284	Yes	1	1188
Pleus et al.	2014	24876447	Yes	4		200^b
Pleus et al.	2016	26809001	Yes	2	600	600
Robinson and Sharp	2012	22768885	Yes	5	214–216	214–216
Sachse et al.^a	2012	22900743	No	1	260
Salzsieder and Berg	2016	26394775	Yes	1		600
Setford et al.^a	2015	26488139	Yes	1		930
Setford et al.	2017	28406040	Yes	1		73600
Sølvik et al.^a	2015	25737533	No	10	152–262	152–262
Sonmez et al.	2010	20615103	No	5	144^b
Tack et al.	2012	22176154	Yes	5	480–510	480–510
Yon Hin et al.	2017	27578824	Yes	1		708
Yu-Fei et al.	2017	28480733	Yes	19	400	400
Zijlstra et al.	2016	27605592	Yes	5		330–342
Zueger et al.	2012	22791566	No	3	150

Data from lab and lay user accuracy was included.

One or more BGMs was tested with a different amount of lots.

An empiric study has also been conducted to demonstrate the real-world consequences of BGM inaccuracy.⁷⁷ Retrospectively studied subjects were classified retrospectively according to various levels of measurement error by BGMs they were using. The investigators did not select the BGMs. Consistently accurate BGMs had the best combination of low mean glycemia and low incidence of hypoglycemia, whereas BGMs providing falsely lower or higher readings resulted in higher HbA1c values and/or increased hypoglycemia, respectively. Furthermore, BGMs with greater accuracy have been shown in modeling studies to be economically attractive.^78,79

Finally, as technology continues to advance with the increasing adoption of continuous glucose monitors (CGMs) and closed-loop systems approaching on the horizon, BGM accuracy is more important than ever. BGMs are frequently relied upon for calibrating CGMs that can be used either as a standalone device or as part of a closed-loop system. The use of an inaccurate BGM when calibrating a CGM can lead to adverse changes in HbA1c levels as well as increase the likelihood of an adverse event, such as hypoglycemia. CGMs themselves must be accurate, and inaccurate CGM performance can have adverse outcomes.^8,9

Health risks and economic burdens accrue from inaccurate BGMs. According to our review of more than 7 years of the medical literature, many FDA-cleared BGMs used in the United States and uncleared BGMS used internationally do not deliver levels of accuracy called for by current standards. The diabetes community expects and needs consistently accurate systems for determining treatment. We believe that independent testing and the addition of postmarket surveillance testing will play a vital role in achieving that goal.

Footnotes

Author Disclosure Statement

D.A., V.H., F.K., and T.P. have nothing to disclose. D.K. is a consultant to Ascensia, EOFlow, Intarcia, Lifecare, Novo Nordisk, AstraZeneca, and Voluntis.

References

Klonoff

, Reyes

. Do currently available blood glucose monitors meet regulatory standards? 1-day public meeting in Arlington, Virginia. 2013. J Diabetes Sci Technol. 2013; 7(4):1071–1083.

International Organization for Standardization. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. ISO 15197:2013.

DIN EN ISO 15197: In Vitro Diagnostic Test Systems—Requirements for Blood Glucose Monitoring Systems for Self-Testing in Managing Diabetes Mellitus (ISO 15197:2003). Brussels: European Committee for Standardization, 2003.

US Food and Drug Administration. Self-monitoring blood glucose test systems for over-the-counter use: draft guidance for industry and Food and Drug Administration staff. www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM380327.pdf (accessed July 14, 2017 ).

US Food and Drug Administration. Self-monitoring of blood glucose test systems for over-the-counter use: guidance for Industry and Food and Drug Administration Staff. https://www.fda.gov/downloads/ucm380327.pdf (accessed July 30, 2017 ).

Klonoff

, Prahalad

. Performance of cleared blood glucose monitors. J Diabetes Sci Technol, 2015; 9:895–910.

Boyd

, Bruns

. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem, 2001; 47:209–214.

Thomas

, Signal

, Harris

, et al. Continuous glucose monitoring in newborn infants: how do errors in calibration measurements affect detected hypoglycemia?. J Diabetes Sci Technol, 2014; 8:543–550.

Campos-Náñez

, Breton

. Effect of BGM accuracy on the clinical performance of CGM: an in-silico study. J Diabetes Sci Technol, 2017; 11:1196–1206.

10.

Sølvik

UØ

, Risa

, Jacobsen

, et al. Performance of 10 systems for self-monitoring of blood glucose by trained healthcare professionals and in the hands of the users. Clin Chem, 2015; 61:772–774.

11.

Freckmann

, Schmid

, Pleus

, et al. System accuracy evaluation of systems for point-of-care testing of blood glucose: a comparison of a patient-use system with six professional-use systems. Clin Chem Lab Med, 2014; 52:1079–1086.

12.

Hasslacher

, Kulozik

, Platten

. Accuracy of self monitoring blood glucose systems in a clinical setting: application of new planned ISO-standards. Clin Lab, 2013; 59:727–733.

13.

Hasslacher

, Kulozik

, Platten

. Analytical performance of glucose monitoring systems at different blood glucose ranges and analysis of outliers in a clinical setting. J Diabetes Sci Technol, 2014; 8:466–472.

14.

Huang

, Chang

, Tsao

, et al. Evaluation of accuracy of FAD-GDH-and mutant Q-GDH-based blood glucose monitors in multi-patient populations. Clin Chim Acta, 2014; 433:28–33.

15.

Jendrike

, Rittmeyer

, Pleus

, et al. ISO 15197: 2013 accuracy evaluation of two CE-marked systems for self-monitoring of blood glucose. J Diabetes Sci Technol, 2015; 9:934–935.

16.

, Guo

, Liu

. Accuracy and precision assessment of a new blood glucose monitoring system. Clin Chem Lab Med, 2016; 54:181–188.

17.

Kuo

, Hsu

, Ho

, et al. Accuracy and precision evaluation of seven self-monitoring blood glucose systems. Diabetes Technol Ther, 2011; 13:596–600.

18.

Pfützner

, Mitri

, Musholt

, et al. Clinical assessment of the accuracy of blood glucose measurement devices. Curr Med Res Opin, 2012; 28:525–531.

19.

Pfützner

, Hengesbach

, Demircik

, et al. Performance of blood glucose meters in compliance with current and future clinical ISO15197 accuracy criteria. Curr Med Res Opin, 2014; 30:185–190.

20.

Robinson

, Sharp

. Tighter accuracy standards within point-of-care blood glucose monitoring: how six commonly used systems compare. J Diabetes Sci Technol, 2012; 6:547–554.

21.

Setford

, Grady

, Phillips

, et al. Seven-year surveillance of the clinical performance of a blood glucose test strip product. J Diabetes Sci Technol, 2017; 11:1155–1162.

22.

Sonmez

, Yilmaz

, Uckaya

, et al. The accuracy of home glucose meters in hypoglycemia. Diabetes Technol Ther, 2010; 12:619–626.

23.

Tack

, Pohlmeier

, Behnke

, et al. Accuracy evaluation of five blood glucose monitoring systems obtained from the pharmacy: a European multicenter study with 453 subjects. Diabetes Technol Ther, 2012; 14:330–337.

24.

Zueger

, Schuler

, Stettler

, et al. Assessment of three frequently used blood glucose monitoring devices in clinical routine. Swiss Med Wkly, 2012; 142:w13631.

25.

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm (accessed October 12, 2018 ).

26.

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCLIA/search.cfm (accessed October 12, 2018 ).

27.

Firth

. Bias reduction of maximum likelihood estimates. Biometrika, 1993; 80:27–38.

28.

Owen

. The pigeonhole bootstrap. Ann Appl Stat, 2007; 1:386–411.

29.

Angelides

, Matsunami

, Engler

. Performance and system validation of a new cellular-enabled blood glucose monitoring system using a new standard reference measurement procedure of isotope dilution UPLC-MRM mass spectrometry. J Diabetes Sci Technol, 2015; 9:1061–1070.

30.

Bailey

, Chang

, Rosenblit

, et al. A comprehensive evaluation of the performance of the test strip technology for OneTouch Verio glucose meter systems. Diabetes Technol Ther, 2012; 14:701–709.

31.

Bailey

, Wallace

, Greene

, et al. Accuracy and user performance evaluation of the Contour® Next Link 2.4 blood glucose monitoring system. Clin Chim Acta, 2015; 448:139–145.

32.

Bailey

, Wallace

, Pardo

, et al. Accuracy and user performance evaluation of a new, wireless-enabled blood glucose monitoring system that links to a smart mobile device. J Diabetes Sci Technol, 2017; 11:736–743.

33.

Baumstark

, Pleus

, Schmid

, et al. Lot-to-lot variability of test strips and accuracy assessment of systems for self-monitoring of blood glucose according to ISO 15197. J Diabetes Sci Technol, 2012; 6:1076–1086.

34.

Baumstark

, Schmid

, Pleus

, et al. Accuracy assessment of an advanced blood glucose monitoring system for self-testing with three reagent system lots following ISO 15197: 2013. J Diabetes Sci Technol, 2014; 8:1241–1242.

35.

Baumstark

, Kraft

, Hattemer

, et al. Accuracy evaluation of an integrated blood glucose monitoring system with improved test cassettes following ISO 15197: 2013. J Diabetes Sci Technol, 2016; 10:242–244.

36.

Baumstark

, Jendrike

, Pleus

, et al. Evaluation of accuracy of six blood glucose monitoring systems and modeling of possibly related insulin dosing errors. Diabetes Technol Ther, 2017; 19:580–588.

37.

Berti

, Scuffi

, Valgimigli

. Accuracy evaluation of two blood glucose monitoring systems following ISO 15197: 2013. J Diabetes Sci Technol, 2015; 9:1354–1356.

38.

Brazg

, Klaff

, Parkin

. Performance variability of seven commonly used self-monitoring of blood glucose systems: clinical considerations for patients and providers. J Diabetes Sci Technol, 2013; 7:144–152.

39.

Brazg

, Klaff

, Sussman

. New generation blood glucose monitoring system exceeds international accuracy standards. J Diabetes Sci Technol, 2016; 10:1414–1415.

40.

Caswell

, Frank

, Vigggiani

, et al. Accuracy and user performance evaluation of a blood glucose monitoring system. Diabetes Technol Ther, 2015; 17:152–158.

41.

Chandnani

, Ramakrishna

, Dave

, et al. A post-marketing surveillance study to evaluate performance of the EXIMO™ blood glucose monitoring system. J Clin Diagn Res, 2017; 11:FC29.

42.

Charoensri

, Khahmahpahte

, Thepsuthammarat

, et al. An evaluation of blood glucose measurement using TRUEresult blood glucose monitoring system. J Med Assoc Thai, 2015; 98:839–846.

43.

Chen

, Lin

, Hung

, Chun, et al. The clinical performance of the EGV1 self-monitoring blood glucose system. Clin Chim Acta, 2012; 413:1039–1044.

44.

Christiansen

, Greene

, Pardo

. A new, wireless-enabled blood glucose monitoring system that links to a smart mobile device: accuracy and user performance evaluation. J Diabetes Sci Technol, 2017; 11:567–573.

45.

Frank

, Wallace

, Pardo

, et al. Performance of the CONTOUR® TS blood glucose monitoring system. J Diabetes Sci Technol, 2011; 5:198–205.

46.

Freckmann

, Baumstark

, Jendrike

, et al. System accuracy evaluation of 27 blood glucose monitoring systems according to DIN EN ISO 15197. Diabetes Technol Ther, 2010; 12:221–231.

47.

Freckmann

, Schmid

, Baumstark

, et al. System accuracy evaluation of 43 blood glucose monitoring systems for self-monitoring of blood glucose according to DIN EN ISO 15197. J Diabetes Sci Technol, 2012; 6:1060–1075.

48.

Freckmann

, Baumstark

, Schmid

, et al. Evaluation of 12 blood glucose monitoring systems for self-testing: system accuracy and measurement reproducibility. Diabetes Technol Ther, 2014; 16:113–122.

49.

Freckmann

, Link

, Schmid

, et al. System accuracy evaluation of different blood glucose monitoring systems following ISO 15197: 2013 by using two different comparison methods. Diabetes Technol Ther, 2015; 17:635–648.

50.

Freckmann

, Baumstark

, Jendrike

, et al. Accuracy evaluation of four blood glucose monitoring systems in the hands of intended users and trained personnel based on ISO 15197 requirements. Diabetes Technol Ther, 2017; 19:246–254.

51.

Harrison

, Leazenby

, Halldorsdottir

. Accuracy of the CONTOUR® blood glucose monitoring system. J Diabetes Sci Technol, 2011; 5:1009–1013.

52.

Hehmke

, Berg

, Salzsieder

. Accuracy evaluation of a CE-marked glucometer system for self-monitoring of blood glucose with three reagent lots following ISO 15197: 2013. J Diabetes Sci Technol, 2017; 11:635–636.

53.

Jeanny

, Hope

. Surveillance of the system accuracy of two systems for self-monitoring of blood glucose after market approval. J Diabetes Sci Technol, 2016; 10:240–241.

54.

Jendrike

, Baumstark

, Chen

, et al. Introduction of a novel smartphone-coupled blood glucose monitoring system. J Diabetes Sci Technol, 2017; 11:1231–1233.

55.

Kamecke

, Baumstark

, Jendrike

, et al. Accuracy assessment of an improved version of an established blood glucose monitoring system for self-testing following ISO 15197: 2013. J Diabetes Sci Technol, 2017; 11:851–853.

56.

Katz

, Macleod

, Grady

, et al. A comprehensive evaluation of strip performance in multiple blood glucose monitoring systems. Expert Rev Med Devices, 2015; 12:263–271.

57.

Katz

, Grady

, Stewart

, et al. Patient and health-care professional satisfaction with a new, simple, high accuracy blood glucose meter with color range indicator. Indian J Endocrinol Metab, 2017; 21:322.

58.

Link

, Pleus

, Schmid

, et al. Accuracy evaluation of three systems for self-monitoring of blood glucose with three different test strip lots following ISO 15197. J Diabetes Sci Technol, 2014; 8:422–424.

59.

Link

, Schmid

, Pleus

, et al. System accuracy evaluation of four systems for self-monitoring of blood glucose following ISO 15197 using a glucose oxidase and a hexokinase-based comparison method. J Diabetes Sci Technol, 2015; 9:1041–1050.

60.

List

, Starks

, Baum

, et al. Performance evaluation and labeling comprehension of a new blood glucose monitoring system with integrated information management. J Diabetes Sci Technol, 2011; 5:1144–1153.

61.

Lock

, Brazg

, Bernstein

, et al. Performance of a new test strip for freestyle blood glucose monitoring systems. Diabetes Technol Ther, 2011; 13:1–10.

62.

Pfützner

, Demircik

, Ramljak

, et al. Evaluation of system accuracy of the GlucoMen LX plus blood glucose monitoring system with reference to ISO 15197: 2013. J Diabetes Sci Technol, 2015; 10:618–619.

63.

Philis-Tsimikas

, Chang

, Miller

. Precision, accuracy, and user acceptance of the OneTouch SelectSimple blood glucose monitoring system. J Diabetes Sci Technol, 2011; 5:1602–1609.

64.

Pleus

, Schmid

, Link

, et al. Accuracy assessment of two novel systems for self-monitoring of blood glucose following ISO 15197: 2013. J Diabetes Sci Technol, 2014; 8:906–908.

65.

Pleus

, Baumstark

, Rittmeyer

, et al. Performance of two updated blood glucose monitoring systems: an evaluation following ISO 15197: 2013. Curr Med Res Opin, 2016; 32:847–855.

66.

Sachse

, Bolstad

, Jonsson

, et al. The Accu-Chek Mobile blood glucose monitoring system used under controlled conditions meets ISO 15197 standards in the hands of diabetes patients. Scand J Clin Lab Invest, 2012; 72:374–379.

67.

Salzsieder

, Berg

. Accuracy evaluation of a CE-marked system for self-monitoring of blood glucose with three reagent system lots following ISO 15197: 2013. J Diabetes Sci Technol, 2015; 10:238–239.

68.

Setford

, Smith

, McColl

, et al. Evaluation of the performance of the OneTouch Select® Plus blood glucose test system against ISO 15197: 2013. Expert Rev Med Devices, 2015; 12:771–781.

69.

Yon Hin

, Bueno

, Lowe

, et al. Clinical accuracy study of an GDH-NAD blood glucose monitoring system using the performance criteria of ISO 15197: 2013. J Diabetes Sci Technol, 2016; 11:444–445.

70.

Yu-Fei

, Wei-Ping

, Ming-Hsun

, et al. Accuracy evaluation of 19 blood glucose monitoring systems manufactured in the Asia-Pacific region: a multicenter study. J Diabetes Sci Technol, 2017; 11:953–965.

71.

Zijlstra

, Heinemann

, Fischer

, et al. A comprehensive performance evaluation of five blood glucose systems in the hypo-, eu-, and hyperglycemic range. J Diabetes Sci Technol, 2016; 10:1316–1323.

72.

Parkin

. If SMBG accuracy is critical to patient safety, why are inaccurate meters still on the market?. J Diabetes Sci Technol, 2017; 11:574–576.

73.

Pfützner

. Advances in patient self-monitoring of blood glucose. J Diabetes Sci Technol, 2016; 10:101–103.

74.

Baumstark

, Schmid

, Pleus

, et al. Influence of partial pressure of oxygen in blood samples on measurement performance in glucose-oxidase-based systems for self-monitoring of blood glucose. J Diabetes Sci Technol, 2013; 7:1513–1521.

75.

Bailey

, Klaff

, Wallace

, et al. Fundamental importance of reference glucose analyzer accuracy for evaluating the performance of blood glucose monitoring systems (BGMSs). J Diabetes Sci Technol, 2016; 10:872–875.

76.

Breton

, Kovatchev

. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol, 2010; 4:562–570.

77.

Boettcher

, Dost

, Wudy

, et al. Accuracy of blood glucose meters for self-monitoring affects glucose control and hypoglycemia rate in children and adolescents with type 1 diabetes. Diabetes Technol Ther, 2015; 17:275–282.

78.

Budiman

, Samant

, Resch

. Clinical implications and economic impact of accuracy differences among commercially available blood glucose monitoring systems. J Diabetes Sci Technol, 2013; 7:365–380.

79.

Schnell

, Erbach

. Impact of a reduced error range of SMBG in insulin-treated patients in Germany. J Diabetes Sci Technol, 2014; 8:479–482.