Computerized Decision Support Systems in Primary Care for Type 2 Diabetes Patients Only Improve Patients' Outcomes when Combined with Feedback on Performance and Case Management: A Systematic Review

Abstract

Purpose:

Computerized decision support systems (CDSSs) are often part of a multifaceted intervention to improve diabetes care. We reviewed the effects of CDSSs alone or in combination with other supportive tools in primary care for type 2 diabetes mellitus (T2DM).

Materials and Methods:

A systematic literature search was conducted for January 1990–July 2011 in PubMed, Embase, and the Cochrane Database and by consulting reference lists. Randomized controlled trials (RCTs) in general practice were selected if the interventions consisted of a CDSS alone or combined with a reminder system and/or feedback on performance and/or case management. The intervention had to be compared with usual care. Two pairs of reviewers independently abstracted all available data. The data were categorized by process of care and patient outcome measures.

Results:

Twenty RCTs met inclusion criteria. In 14 studies a CDSS was combined with another intervention. Two studies were left out of the analysis because of low quality. Four studies with a CDSS alone and four studies with a CDSS and reminders showed improvements of the process of care. CDSS with feedback on performance with or without reminders improved the process of care (one study) and patient outcome (two studies). CDSS with case management improved patient outcome (two studies). CDSS with reminders, feedback on performance, and case management improved both patient outcome and the process of care (two studies).

Conclusions:

CDSSs used by healthcare providers in primary T2DM care are effective in improving the process of care; adding feedback on performance and/or case management may also improve patient outcome.

Introduction

M any patients with type 2 diabetes mellitus (T2DM) do not meet the targets for good glycemic and cardiovascular control.^1
–3 In facing the management problems with chronic illnesses in general practice, structured and regular review of patients⁴ and feedback on performance given to general practitioners^5,6 are effective in improving diabetes care. Both interventions can easily be combined with computerized decision support systems (CDSSs). Therefore these information technology systems may be an important tool in successful diabetes management.

Unfortunately, there is no international definition of CDSS, but in most diabetes management systems physicians, practice nurses, or patients manually enter patient characteristics into the CDSS, or the electronic medical record is electronically searched for patient characteristics. These individual patient characteristics are then used in software algorithms and/or matched to a computerized knowledge base, to generate treatment recommendations.

Garg et al.⁷ performed a review on CDSS in clinical care, showing mainly improvements in the practitioner's performance. Another review, evaluating the effects of interactive computer-assisted technology in T2DM care, concluded that there is growing evidence that information technology improves diabetes care.⁸ However, this study evaluated both randomized controlled trials (RCTs) and observational studies with a broad range of interventions, such as education, disease management, telephone automated calls, and telemedicine; it aimed at both healthcare provider and patient and included both primary and secondary care. Because of this heterogeneity general inferences were impaired.

We aimed to study whether a CDSS alone or a CDSS in combination with a reminder system or with feedback on performance or as part of a case management system improves both patients' outcome and practitioners' performance.

Materials and Methods

Eligibility criteria

Eligible studies were randomized clinical trials published in peer-reviewed journals in English that compared the effectiveness of T2DM care with a CDSS against that of T2DM care without a CDSS on clinical performance (measure of process of care) and/or patient outcome. We searched for management interventions developed for use by a diabetes care provider in general practice/primary care. The CDSS should contain a computer system that used patient characteristics to generate decision support by a software algorithm based on a diabetes guideline. The CDSS could also function as a recall system and/or make it possible to give feedback on performance on patient level and/or healthcare professional level and/or be integrated in a so-called case management system.

Computerized glucose monitoring systems, diabetes self-management programs, digital eye screening programs, or patient education systems were excluded. The studies should include only T2DM patients and have a follow-up of at least 6 months.

Search strategy

Published studies were identified by searching the electronic databases of PubMed, Embase, and the Cochrane Library. A universal definition of a CDSS is not available. However, based on our above-mentioned operational definition of a CDSS, the following search terms were used: “diabetes” AND (“decision support” OR “computer-assisted decision making” OR “computer” OR “artificial intelligence” OR “electronic intervention” OR “Internet” OR “reminder systems” OR “recall system” OR “feedback” OR “benchmark”) AND (“randomised” OR “randomized” OR “RCT” OR “trial” OR “evaluation studies”). Because a more widespread use of CDSS started about two decades ago, we included articles published between January 1990 and July 2011. Finally, manual searches were performed by screening the reference sections of the relevant review articles and of the selected RCTs.

Study selection

Titles and abstracts were independently reviewed by two pairs of investigators for eligibility. The first 200 titles were reviewed by all investigators. The results were compared and discussed in order to reduce the variation in interpretation of inclusion and exclusion criteria between the reviewers. Full text articles were retrieved if any reviewer considered a citation potentially relevant. Two investigators then independently judged the full text of potentially eligible articles. Disagreements were resolved by discussion. If no consensus could be achieved, a third investigator was asked. When comparable outcome data of a study were published twice, we cited the publication providing most data and with the longest follow-up.

Data extraction

Two reviewers independently extracted the following data from all included studies: study setting, study methods, study intervention characteristics, and study outcomes. Study outcome was categorized by process of care measures and by patient outcomes. Process of care measures are, for example, the frequency of glycosylated hemoglobin (HbA1c) testing or starting medication when treatment goals are not met. Patient outcomes measures are, for example, the actual change in HbA1c, cholesterol level, or blood pressure. The same pairs of investigators worked together, as in study selection. Disagreements were resolved by consensus, and where no consensus could be achieved, a third investigator decided.

Methodological validity

All studies were scored for methodological validity on a 2-point scale: yes (1 point) or no or unclear (0 points). The nine methodological validity indicators from the Dutch Cochrane Center were used: (1) intervention randomized; (2) randomization order not known by person who included patients/practices; (3) patients blinded; (4) therapist blinded; (5) outcome assessor blinded; (6) groups comparable; (7) proportion of follow-up of all included patients high enough; (8) included patients analyzed in group of inclusion; and (9) groups equally treated, except for the intervention.⁹ Studies could be cluster randomized or patient randomized. Whenever studies were cluster randomized, we identified whether appropriate analysis methods (e.g., generalized estimated equations) were used in order to correct for clustering. Only studies that randomized patients or studies that were cluster randomized and applied appropriate methods to take cluster effects into account scored 1 point for the first item of the Cochrane Center list. A 10^th indicator was added: the use of power calculations. Adding all validity indicators the studies could score a maximum of 10 points. Only the results of studies that scored 5 or more points were used. Furthermore, we reported country, commercial funding of studies, and the number of patients.

We presented studies as a group, depending on the type of intervention or combination of interventions.

Statistical analysis

First, we assessed the eligibility of the study. Reviewer agreement on study eligibility was quantified using the Cohen κ value.

We divided the eligible studies into six categories, based on the intervention used: (1) a CDSS alone; (2) a CDSS combined with a reminder system; (3) a CDSS with feedback on performance; (4) a CDSS with case management or with case management and reminders; (5) a CDSS with a reminder system and feedback on performance; or (6) a CDSS with a reminder system, feedback on performance, and case management. The 10 validity indicators were used to express study methodological validity. We calculated means, the SD, and the range. Finally, the effectiveness of the interventions was compared by describing the measures as mentioned in the articles. We distinguished “process measures” and “patient outcomes.”

Results

Selection of studies

The electronic database search revealed 2,290 citations, when duplicate citations between databases were removed. The titles of these citations were reviewed and revealed 548 abstracts. After abstract selection 121 articles remained for full text review. Eventually, 26 articles met our inclusion criteria. There were two duplicate publications: Glasgow et al.^10,11 with 6 months of follow-up¹⁰ and 12 months of follow-up¹¹ and Lobach et al.^12,13 with baseline compliance levels¹² and 6 months of follow-up.¹³ The study from Phillips et al.¹⁴ and Ziemer et al.⁶ regarded the same study population, with different outcome measures. This also applies to the three studies of Cleveringa et al.^15
–17 and to the studies from Khan et al.¹⁸ and MacLean et al.¹⁹ Therefore eventually 20 RCTs were included. Ninety-five articles were excluded because of different reasons, for example, review article (n=14), no RCT (n=22), no CDSS used in the intervention (n=26), glucose monitoring system (n=10), or diabetes self-management program (n=11) (Fig. 1).

FIG. 1.

Summary of the literature search. CDSS, computerized decision support system; RCT, randomized controlled trial.

There was substantial agreement between the reviewers for article inclusion, with a change-corrected agreement between two pairs of independent investigators of κ=0.75 versus κ=0.76.

Methodological validity assessment (Table 1)

In eight trials patients were randomized^20

–26; one of them also corrected for clustering.²⁵ The other 12 trials had a cluster randomized design,^{6,11,13

–17,19,27

–34} and 10 trials adjusted for clustering in the analysis.^{6,11,14,16,30

–35} Twelve trials reported a power calculation for a specified difference between groups and a specific outcome.^{6,11,14,20,22,26,29,31

–34,36}

Table 1.

Methodological Validity

							Blinded
Reference (year)	Total patients (n)	Country	Funding	Randomization level	GEE	Randomized ^a	Randomization order	Patient	Physician	Outcome assessor	Groups comparable	Follow-up of <70%	Patients analyzed in group of inclusion	Groups equally treated	Power calculation	Total ^b
Hurwitz et al.²⁴ (1993)	209	United Kingdom	Public	Patient	NA	1	1	0	0	?	1	1	1	1	0	6
Lobach et al.¹³ (1997)	359	United States	Public	Cluster	0	0	1	1	1	?	1	?	?	?	0	4^c
Hetlevic et al.²⁹ (2000)	1,034	Norway	Public	Cluster	0	0	?	0	0	?	1	0	?	1	1	3^c
Lafata et al.²⁵ (2002)	3,309	United States	None	Patient	1	1	1	1	0	1	1	1	1	1	0	8
Hirsch et al.³⁰ (2002)	109	United States	Private	Cluster	1	1	1	?	0	0	1	1	1	1	0	6
Meigs et al.³² (2003)	598	United States	Private	Cluster	1	1	1	?	0	1	1	?	1	1	1	7
Ilag et al.³¹ (2003)	284	United States	Public	Cluster	1	1	1	0	0	?	1	0	1	1	1	6
Glasgow et al.¹¹ (2005)	886	United States	Public	Cluster	1	1	1	0	0	0	1	1	1	1	1	7
Sequist et al.³⁴ (2005)	4,549	United States	Public	Cluster	1	1	1	?	0	?	1	?	?	1	1	5
Phillips et al.¹⁴ (2005)	4,138	United States	Public	Cluster	1	1	1	1	0	?	1	?	1	1	1	7
Ziemer et al.⁶ (2006)	4,138	United States	Public	Cluster	1	1	1	1	0	?	1	?	1	1	1	7
Cho et al.²³ (2006)	80	Korea	Public	Patient	NA	1	1	0	0	1	1	1	1	1	0	7
Bond et al.²⁰ (2007)	62	United States	Public	Patient	NA	1	1	0	0	?	1	1	1	1	1	7
Grant et al.²⁷ (2008)	244	United States	Public	Cluster	0	0	1	0	0	?	1	1	1	1	0	5
Peterson et al.³³ (2008)	7,101	United States	Public	Cluster	1	1	1	0	0	1	1	1	1	1	1	8
Cleveringa et al.³⁵ (2008)	3,391	The Netherlands	Private	Cluster	1	1	1	0	0	1	1	1	1	1	1	8
Ralston et al.²⁶ (2009)	83	United States	Private	Patient	0	1	1	0	?	?	1	1	1	1	1	7
Holbrook et al.³⁶ (2009)	511	Canada	Public	Patient	NA	1	1	0	0	1	1	1	1	1	1	8
MacLean et al.¹⁹ (2009)	7,412	United States	Public	Cluster	1	1	1	0	0	?	1	1	1	1	0	6
Cleveringa et al.¹⁷ (2010)	3,391	The Netherlands	Private	Cluster	1	1	1	0	0	1	1	0	1	1	1	7
Cleveringa et al.¹⁵ (2010)	3,391	The Netherlands	Private	Cluster	1	1	1	0	0	1	1	1	1	1	0	7
Khan et al.¹⁸ (2010)	7,268	United States	Public	Patient	NA	1	1	0	0	?	1	1	1	1	0	6
Guldberg et al.²⁸ (2011)	2,716	Denmark	Public	Cluster	1	1	1	0	0	?	1	1	1	1	0	6
O'Connor et al.²² (2011)	2,556	United States	Public	Patient	1	1	1	0	0	?	1	1	1	1	1	7

Cleveringa et al.,^15
–17 Phillips et al.¹⁴ and Ziemer et al.,⁶ and MacLean et al.¹⁹ and Khan et al.¹⁸ represent the same respective study with different outcome parameters.

Value in randomized column is 1 if patients were randomized or when cluster randomization with the generalized estimating equation (GEE) was used.

Total validity score is the sum of randomized, blinded (randomization order, patient, physician, and outcome assessor), groups comparable, follow-up of <70%, patients analyzed in group inclusion, groups equally treated, and power calculation columns.

Not meeting minimal validity score.

NA, not applicable.

Positive scores on the methodological validity indicators blinding of patient, therapist, and outcome assessor were poor: 17%, 4%, and 33%, respectively. On the methodological validity scale the mean score was 6.4 (SD, 1.3) with a range from 3 to 8. Two studies scored less than 5 points^13,29 and were excluded from the analysis (Table 1).

Categories of studies

The 20 included studies were published between 1993 and 2011. The number of trials increased with time: one in 1990–1994, one in 1995–1999, five in 2000–2004, eight in 2005–2009, and five in 2009–2011. Fourteen studies were conducted in the United States, one in Canada, one in the United Kingdom, one in Norway, one in Denmark, one in Korea, and one in The Netherlands. The number of patients included varied between 62²⁰ and 7,412.¹⁹ Sixteen of the studies described funding from the public sector, and four obtained funding from the private sector. In six studies the only intervention was a CDSS (Table 2); the other studies regarded a multifaceted intervention in which the CDSS was combined with a reminder system (Table 3), CDSS with feedback on performance (Table 4), CDSS with case management or CDSS with case management and reminders (Table 5), CDSS with a reminder system and feedback on performance (Table 6), and CDSS with a reminder system, feedback on performance, and case management (Table 7).

Table 2.

Computerized Decision Support Systems

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results for control vs. intervention	P value
Meigs et al.³² (2003)	598	12	Usual diabetes care	Web-based decision support tool. The diabetes management application displays interactive patient-specific clinical data, treatment advice, and links to Web-based care sources.	Patient outcome
					HbA1c (%)	0.14 vs. −0.23	NS
					LDL-cholesterol (mg/dL)	−9.4 vs. −14.7	NS
					SBP (mm Hg)	−2.2 vs. 0.8	0.03
					DBP (mm Hg)	−0.8 vs. −1.8	NS
					Process of care
					1 HbA1c test/year	−1.0 vs. 1.6	NS
					1 LDL test/year	3.4 vs. 7.2	NS
					1 blood pressure measurement/year	−1.4 vs. 1.0	NS
					Eye exam	1.7 vs. 5.5	NS
					Foot exam	0.7 vs. 9.8	0.003
Ilag et al.³¹ (2003)	174	12	Usual diabetes care	Annual diabetes assessment program. Results reviewed with patients, mailed to providers, and incorporated into EMR with guideline-generated suggestions for treatment and follow-up	Patient outcome
					HbA1c (%)
					<7%	30 vs. 28	NS
					7.1–8.0%	32 vs. 32	NS
					8.1–10.0%	30 vs. 28	NS
					>10.1%	8 vs. 12	NS
					SBP <135 mm Hg	58 vs. 60	NS
					DBP <80 mm Hg	75 vs. 77	NS
					LDL-cholesterol
					<2.5 mmol/L	35 vs. 48	NS
					2.5–3.3 mmol/L	34 vs. 23	NS
					>3.3 mmol/L	31 vs. 29	NS
					Process of care
					Increase mean sum of measures	0 vs. 1.5	0.014
Glasgow et al.¹¹ (2005)	886	12	Computer touch screen with print out focusing on general health risks	Computer touch screen assessment and action plan including a summary of assays/checks for which the patient might be due and a copy of the patient's self-management plan	Process of care
					Lab procedures completed (n)	3.97 vs. 4.29	0.001
					Blood pressure measurements (%)	99.7% vs. 100%	NS
					Eye exam (%)	72.4% vs. 77.2%	NS
					Foot exam (%)	83.7% vs. 93.6%	NS
					Microalbumin measurements (%)	81.4% vs. 91.3%	NS
					HbA1c ≤9.5% (%)	97.4% vs. 93.9%	NS
					Patient-centered activities completed	3.32 vs. 3.73	0.001
					HbA1c (%)	−0.13 vs. −0.22	NS
					Total cholesterol/HDL-cholesterol ratio	−0.23 vs. −0.21	NS
Grant et al.²⁷ (2008)	244	12	Usual diabetes care	Diabetes specific Web-based personal health record, providing patient-tailored decision support and enabling the patient to author a “diabetes care plan” for electronic submission to his or her physician prior to upcoming appointments	Patient outcome
					HbA1c (%)	0.26 vs. 0.16	NS
					HbA1c <7% (%)	25 vs. 45	NS
					Process of care
					Medication adjustments (%)
					DM-related	15 vs. 53	<0.001
					Hyperglycemia	15 vs. 29	NS
					Hypertension	0 vs. 13	0.02
					Hyperlipidemia	0 vs. 11	0.03

DBP, diastolic blood pressure; DM, diabetes mellitus; EMR, electronic medical record; HbA1c, glycosylated hemoglobin; HDL, high-density lipoprotein; LDL, low-density lipoprotein; NS, not significant; SBP, systolic blood pressure.

Table 3.

Computerized Decision Support Systems with Reminders

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results for control vs. intervention	P value
Lafata et al.²⁵ (2002)	3,309	12	Web-based diabetes care management support system	Web-based diabetes management support system and mailed patient reminder with tailored recommendations for actions to be taken by the patient, a self-care handbook, and a preventive care checklist	Process of care
					1 HbA1c test	OR 1.21	0.05
					2 HbA1c tests	OR 1.04	NS
					Retinal exam	OR 1.23	0.01
					Fasting lipid profile	OR 1.14	NS
					All 3 tests	OR 1.25	0.01
					HbA1c <8%	OR 1.14	NS
					HbA1c >9.5%	OR 0.83	0.01
					LDL <130 mg/dL	OR 1.11	NS
Sequist et al.³⁴ (2005)	4,549	6	Usual care	Evidence-based electronic reminders within patients' EMR	Process of care
					LDL testing	OR 1.41	<0.001
					ACE in hypertension	OR 1.42	NS
					Biennial HbA1c	OR 1.14	NS
					Eye exam	OR 1.38	NS
					Statin if LDL-cholesterol >130 mg/dL	OR 1.10	NS
					Composite	OR 1.3 (1.01–1.67)	<0.05
IPCAAD: Phillips et al.¹⁴ (2005)^a	4,138	36 average 15	Usual care	Hard copy computerized reminders that provided patient-specific recommendations for management	Patient outcome
					HbA1c (%)	−0.16 vs. −0.3	NS
					SBP (mm Hg)	−2.4 vs. 1.2	NS
					LDL-cholesterol (mg/dL)	−15 vs. −15	NS
IPCAAD: Ziemer et al.⁶ (2006)^a	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	Process of care
					Treatment intensification when glucose level >8.3 mmol/L	41 vs. 39	NS
					Any intensification of therapy (%)	24 vs. 26	<0.02
					Intensification of therapy with recommendations (%)	10 vs. 11.5	<0.02
Holbrook et al.³⁶ (2009)	511	6	Usual care	Shared access by the primary care provider and the patient to a Web-based color-coded diabetes tracker	Process of care	Mean difference
					HbA1c (%)	0.19 (0.09–0.29)	NS
					Blood pressure (mm Hg)	0.34 (0.19–0.49)	NS
					LDL-cholesterol (mmol/L)	0.18 (0.07–0.28)	NS
					Albuminuria (mg/mol)	0.27 (0.16–0.39)	NS
					BMI (kg/m²)	0.17 (0.02–0.32)	NS
					Feet no neuropathy	0.16 (0.06–0.25)	NS
					Exercise (min/week)	−0.01 (−0.09 to 0.07)	NS
					Nonsmoker	−0.03 (−0.12 to 0.70)	NS
MacLean et al.¹⁹ (2009)^a	7,412	1 day–47 months Average 32	Usual care	The VDIS receives laboratory results, maintains a registry, generates reminders for patients and providers, and produces reports with guideline-based recommendations for primary care providers and their patients.	Process of care (lab monitoring)	Adjusted effect (OR)
					HbA1c (%)	1.17 (0.80–1.72)	0.43
					LDL (mmol/L)	1.39 (1.08–1.80)	0.012
					Serum creatinine (mg/dL)	1.40 (1.06–1.84)	0.018
					Microalbumin (mg/mol)	1.74 (1.13–2.69)	0.012
Khan et al.¹⁸ (2010)^a	See MacLean et al.¹⁹	See MacLean et al.¹⁹	See MacLean et al.¹⁹	See MacLean et al.¹⁹	Process of care
					Number of hospital admissions	0.20 vs. 0.17	0.01
					Length of stay (days)	1.1 vs. 0.99	0.01
					Number of emergency department visits	0.36 vs. 0.27	0.0001

Same study population, same intervention, different outcome. The study compared four interventions. Here the results of the computerized decision support system with reminders are compared with usual care.

ACE, angiotensin converting enzyme; BMI, body mass index; EMR, electronic medical record; HbA1c, glycosylated hemoglobin; IPCAAD, Improving Primary Care of African Americans with Diabetes; LDL, low-density lipoprotein; NS, not significant; OR, odds ratio; SBP, systolic blood pressure; VDIS, Vermont Diabetes Information System.

Table 4.

Computerized Decision Support Systems and Feedback on Performance

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results for control vs. intervention	P value
IPCAAD: Phillips et al.¹⁴ (2005)^a	4,138	36 (average 15)	Usual care	Computerized patient-specific recommendations for management and individual face-to-face feedback on performance on providers' actions and patient-specific outcome, for 5 min every 2 weeks	Patient outcome
					HbA1c (%)	−0.16 vs. 0.4	NS
					SBP (mm Hg)	−2.4 vs. −3.2	NS
					LDL-cholesterol (mg/dL)	−15 vs. −14	NS
IPCAAD: Ziemer et al.⁶ (2006)^a	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	Process of care
					Treatment intensification when glucose level >8.3 mmol/L	42 vs. 50	<0.001
					Any intensification of therapy (%)	28 vs. 40	<0.005
					Intensification of therapy met recommendations (%)	11 vs. 17	<0.005
Cho et al.²³ (2006)	80	30	Conventional office visits	Internet-based individual electronic chart system. Patients entered self-monitored blood glucose levels, current medication, blood pressure, and weight. GP, nurse, or dietician sent treatment recommendations, education, and patient feedback every 2 weeks. Three monthly face-to-face visits	Patient outcome
					HbA1c (%)	−0.1 vs. −1.0	0.009
					Total cholesterol (mmol/L)	−0.31 vs. −0.14	NS
					HDL-cholesterol (mmol/L)	0.01 vs. 0.08	NS
Guldberg et al.²⁸ (2011)^b	2,716	15	Usual care	Electronic feedback system. Presents register data on T2DM population, giving the option either to use the data during individual diabetes consultations or to gain an overview of the quality of their diabetes care and compare it with the corresponding quality of colleagues.	Process of care	Difference (CI)
					Oral antidiabetes treatment and no insulin	20.9 (7.9–34.8)	0.002
					Oral antidiabetes sustained	−2.3 (−5.3 to 0.3)	NS
					Insulin sustained	−0.3 (−3.2 to 2.5)	NS
					Lipid-lowering initiated	19.7 (6.1 to 33.2)	0.004
					Lipid-lowering sustained	2.4 (1.0 to 3.7)	0.001
					Blood pressure-reducing treatment initiated	11.3 (1.4 to 21.2)	0.026
					Blood pressure-reducing treatment sustained	0.3 (−2.0 to 2.7)	NS

Same study population, same intervention, different outcome. The study compared four interventions. Here the results of CDSS with feedback on performance is compared with usual care.

Not all the study outcomes are listed here.

CI, confidence interval; GP, general practitioner; HbA1c, glycosylated hemoglobin; HDL, high-density lipoprotein; IPCAAD, Improving Primary Care of African Americans with Diabetes; LDL, low-density lipoprotein; NS, not significant; SBP, systolic blood pressure; T2DM, type 2 diabetes mellitus.

Table 5.

Computerized Decision Support Systems with Case Management or with Case Management and Reminders

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results for control vs. intervention	P value
Bond et al.²⁰ (2007)	62	6	Usual diabetes care	Web-based diabetes management intervention by the nurse. Patients entered blood sugar readings, exercise programs, weight changes, blood pressure, and medication data. GP retained full responsibility and control.	Patient outcome
					HbA1c (%)	−0.05% vs. −0.62%	<0.01
					HDL-cholesterol (mg/dL)	−0.16 vs. 6.4	<0.05
					Total cholesterol (mg/dL)	−5.1 vs. −11.4	<0.05
					Weight (pounds)	2.5 vs. −4.5	<0.001
					SBP (mm Hg)	−1.0 vs. −6.8	<0.01
					DBP (mm Hg)	−2.5 vs. −5.2	NS
Ralston et al.²⁶ (2009)	83	12	Usual care	Case manager, computerized decision support, clinical reminders, ability to upload glucose data by Web, and viewing patients' own health record. Active follow-up by healthcare provider	Patient outcome
					HbA1c (%)	0.2 vs. −0.9	<0.01
					HbA1c <7% (%)	11 vs. 33	0.03
					Process of care
					Outpatient visits	−2.1 vs. 0.6	NS
					Primary care provider visits	−0.2 vs. 0.0	NS
					Specialty physician visits	−1.9 vs. 0.6	NS
					Inpatient days	−0.3 vs. 0.2	NS

DBP, diastolic blood pressure; GP, general practitioner; HbA1c, glycosylated hemoglobin; HDL, high-density lipoprotein; NS, not significant; SBP, systolic blood pressure.

Table 6.

Computerized Decision Support Systems, Reminders, and Feedback on Performance

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results for control vs. intervention	P value
Hurwitz et al.²⁴ (1993)	209	24	Hospital diabetes clinic	Computerized prompting system. Reminders for lab testing and doctors' visits. Clinical review feedback form	Patient outcome
					Mean HbA1c (%)	10.6 vs. 10.0	NS
					Process of care
					Patients without doctor's review (%)	15.2 vs. 3.4	0.013
					Mean number of HbA1c tests	0.9 vs. 2.4	<0.001
Hirsch et al.³⁰ (2002)	109	14	Usual diabetes care	Reminder system, staged diabetes management protocol, computerized feedback, didactic teaching	Patient outcome
					HbA1c (%)	0.64 vs. −0.07	0.02
					SBP (mm Hg)	3.1 vs. −1.2	NS
					DBP (mm Hg)	−0.8 vs. −3.7	NS
IPCAAD: Phillips et al.¹⁴ (2005)^a	4,138	36 (average 15)	Usual care	Hard copy computerized reminders providing patient-specific recommendations for diabetes management. Individual face-to-face feedback on performance on providers' actions and patient-specific outcome, for 5 min every 2 weeks	Patient outcome
					HbA1c (%)	−0.16 vs. −0.56	0.01
					SBP (mm Hg)	−2.4 vs. −3.4	NS
					LDL-cholesterol (mg/dL)	−15 vs. −18	NS
IPCAAD: Ziemer et al.⁶ (2006)^a	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	See Phillips et al.¹⁴	Process of care
					Treatment intensification when glucose level >8.3 mmol/L	42 vs. 51	<0.001
					Any intensification of therapy (%)	28 vs. 40	<0.005
					Intensification of therapy met recommendations (%)	11 vs. 17	<0.005
O'Connor et al.²² (2011)			Usual care	Electronic health records Diabetes Wizard. Used by GP every visit of patient. Gives feedback to GP on treatment possibilities and frequency of follow-up	Patient outcome
					HbA1c (mean)	−0.32 vs. −0.58	0.01
					SBP (mean)	−10.1 vs. −10.8	0.56
					DBP (mean)	−7.5 vs. −8.3	0.38
					LDL (mean)	−25.8 vs. −24.4	0.62

DBP, diastolic blood pressure; GP, general practitioner; HbA1c, glycosylated hemoglobin; LDL, low-density lipoprotein; IPCAAD, Improving Primary Care of African Americans with Diabetes; NS, not significant; SBP, systolic blood pressure.

Table 7.

Computerized Decision Support Systems, Reminders, Feedback on Performance, and Case Management

					Study outcomes
Reference (year)	Sample (n)	Duration (months)	Control	Intervention	Measures	Results of control vs. intervention	P value
Peterson et al.³³ (2008)	7101	12	Usual diabetes care	Electronic diabetes patient registry providing patient-specific decision support, visit reminders, audit and feedback monthly, site coordinator	Process of care
					HbA1c tests	−5.3 vs. 2.8	<0.001
					Blood pressure monitoring	−2.1 vs. 1.3	0.05
					LDL-cholesterol testing	0.3 vs. 8.9	<0.001
					Eye exam	1.2 vs. 27	<0.001
					Foot exam	−5.6 vs. 29.4	<0.001
					Renal testing	−5.3 vs. 23.2	<0.001
					Composite of above	0.22 vs. 1.29	<0.001
					Patient outcome
					Composite of HbA1c <7.0%, SBP <130 mm Hg, LDL-cholesterol <100 mg/dL	0.02 vs. 0.17	0.002
Cleveringa et al.¹⁶ (2008)^a	3,391	12	Usual diabetes care	Diabetes consultation hour run by a practice nurse, computerized decision support providing patient-specific feedback, recall system, and feedback on performance 3 times monthly	Patient outcome
					HbA1c (%)	−0.1 vs. −0.2	NS
					Blood pressure:
					SBP (mm Hg)	−2 vs. −6	<0.05
					DBP (mm Hg)	0 vs. −3	<0.05
					Total cholesterol (mmol/L)	−0.1 vs. −0.4	<0.05
					LDL-cholesterol (mmol/L)	−0.2 vs. −0.3	<0.05
					10-year UKPDS CHD risk (%)^b	−0.1 vs. −1.9	<0.05
					Composite of HbA1c <7%, SBP <140 mm Hg, LDL <2.5 mmol/L	2.5 vs. 8.6	<0.05
Cleveringa et al.¹⁷ (2010)^a	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶ for noninferiority trial on health status	Patient care	Mean difference (CI)
					DHP total score	−0.88 (−1.94 to 0.12)	NS
					DHP barriers to activity	−1.16 (−2.34 to 0.03)	NS
					DHP psychological stress	−0.63 (−1.72 to 0.43)	NS
					DHP disinhibiting eating	−1.83 (−3.64 to −0.07)	NS
					SF-36 all items		NS
Cleveringa et al.¹⁵ (2010)^a	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶	See Cleveringa et al.¹⁶ for cost-effectiveness analysis	Process of care
					Diabetes-related costs (excluding CHD)	1,698 (187 to 3,209)	NS
					CHD costs	−587 (−880 to −294)	NS
					DCP costs	316 (315 to 318)	NS
					Total costs	1.415 (−130 to 2,961)	NS
					Total costs per QALY gained	38,243	NS

Same study population, same intervention, different outcome.

10-year United Kingdom Prospective Diabetes Study (UKPDS) coronary heart disease (CHD) risk is the estimated risk on death from CHD calculated by using the UKPDS risk engine.

CI, confidence interval; DBP, diastolic blood pressure; DCP, diabetes care protocol; DHP, Diabetes Health Profile; HbA1c, glycosylated hemoglobin; LDL, low-density lipoprotein; NS, not significant; QALY, quality-adjusted life-years; SBP, systolic blood pressure; SF-36, Short Form 36.

Because Phillips et al.¹⁴ and Ziemer et al.⁶ compared both usual care, CDSS with reminders, CDSS with feedback on performance, and CDSS with feedback on performance and reminders, these studies appear in three tables. In all these different studies one intervention group is compared with the usual-care control group.

Effectiveness of a CDSS alone (Table 2)

The four studies either used a CDSS or a Web-based diabetes management support system. They were performed between 2004 and 2008.

The studies show improvements in the process of care, like the number of completed foot exams,³² an increase in the mean sum of measures,³¹ the number of completed laboratory tests, and completed patient-centered activities.¹¹ The most recent study showed more medication adjustments by the general practitioner because patients took more initiative to improve their blood values.²⁷ Improvements in patient outcome were not significant. However, in one study the systolic blood pressure increased significantly more in the intervention group.³²

Effectiveness of a CDSS with reminders (Table 3)

Five studies, reported in seven articles between 2002 and 2010, used a CDSS with reminders. Three studies only showed improvements in the process of care. Improvements were found in yearly HbA1c testing, retinal exams, and the composite of three tests, as well as in laboratory monitoring,¹⁸ low-density lipoprotein testing, and the composite of all process measures.³⁴ One study showed that significantly more treatment adjustments were made when glucose levels exceeded 8.3 mmol/L,^6,14 and one study showed that the intervention was associated with reduced hospital and emergency department utilization and expenses.¹⁹ Two other two studies showed little improvement in patient outcome: blood pressure and HbA1c³⁶ and in fewer patients having HbA1c >9.5%.²⁵

Effectiveness of a CDSS and feedback on performance (Table 4)

In three studies (four publications), a CDSS with feedback on performance was compared with usual care. In one study the HbA1c level improved significantly,²³ but not in another study.¹⁴ The process of care improved by better prescription patterns and a better stimulation to follow the guidelines more closely in one study²⁸ and intensification of therapy in another study.⁶ In the latter a multivariable analysis showed that feedback on performance independently facilitated attainment of American Diabetes Association goals for HbA1c (<7%) and systolic blood pressure (<130 mm Hg) and also independently contributed to therapy intensification and consequently to a fall in HbA1c level.⁶

Effectiveness of a CDSS with case management or with case management and reminders (Table 5)

In both studies that assessed this type of intervention, performed in 2007 and 2009, respectively, patient outcome improved. The first study with only 62 patients led to significant improvements in HbA1c level, total cholesterol, high-density lipoprotein-cholesterol, weight, and systolic blood pressure.²⁰ In the second study both HbA1c level and the percentage of patients reaching HbA1c <7% improved significantly, but no differences were found in general practitioner visits, specialist visits, or inpatient days.²⁶

Effectiveness of a CDSS with reminders and feedback on performance (Table 6)

In four studies (five publications), performed between 1993 and 2011, a CDSS was combined with reminders and feedback on performance. In three of them the patient outcome HbA1c level was improved.^14,22,30 The process of care improved by treatment intensification in one of the three studies mentioned before.⁶ The oldest study did not show improvements in the patient outcome HbA1c level; however, the process of care did by a significant decrease in both the percentage of patients who had no doctor's review and the percentage of patients without HbA1c testing.²⁴

Effectiveness of a CDSS with reminders, feedback on performance, and case management (Table 7)

In two more recent large cluster randomized trials all four interventions were combined. In both studies patient outcome improved either by an improved composite end point of HbA1c level, systolic blood pressure, and low-density lipoprotein-cholesterol³³ or by an improved 10-year United Kingdom Prospective Diabetes Study coronary heart disease risk estimate.¹⁶ The process of care also significantly improved in one study.³³ For one intervention it was shown that it was not cost-effective¹⁵ and that there was no negative influence on health status.¹⁷

Discussion

We evaluated RCTs that studied the effectiveness of a CDSS alone or in combination with other supportive tools to improve the quality of primary T2DM care. We distinguished “process measures” and “patient outcome.” A CDSS alone seems ineffective in improving patient outcome.

Comparison with other studies

Our findings are in accordance with earlier reviews that concluded that information technology alone, like a CDSS, mainly improves the process of diabetes care.^7,8

Our finding that a CDSS with reminders improves the process of care, but not the patient outcome, is supported by an earlier review. Reminders facilitate a structured and regular review of patients; they improve the process of care.⁴

Our conclusion with regard to the effectiveness of a CDSS combined with feedback on performance is ambiguous. This seems to be in line with earlier findings. A Cochrane review regarding audit and feedback reported positive effects on the process of care but not on patient outcome³⁷; however, this review was hampered by inadequate reporting of study methods for almost all studies. Looking at the results of the Improving Primary Care of African Americans with Diabetes Study,⁶ we might conclude more optimistically that the combination of CDSS and feedback on performance is probably an important tool to improve patient outcome in diabetes care.

The combination of a CDSS with feedback on performance, reminders, and case management seems to be the most effective. This finding is supported by evidence on the Chronic Care Model,³⁸ in which making care delivery more team-based and planned and making better use of registry-based information belong to the key determinants of improved patient outcomes.⁴

Adding patient education and nurses that function as case-manager also improves patient outcome.^4,39 This review also concludes that case-managers can improve patient outcome. The effects of electronic patient education systems were, however, excluded in this review.

Strengths and limitations

This is the first review on the effectiveness of a CDSS that focused on primary care T2DM management programs. Because most CDSSs are part of a broader intervention, we distinguished six different combinations of interventions. Doing so, we could find the interventions in which a CDSS is most likely to improve both process and outcome of diabetes care.

The methodological quality of the studies was assessed. It appeared that the scores for blinding of patients, therapists, and outcome assessors were low, which may be caused by the complexity of these interventions. By excluding two studies the results of our review are more robust.

The funding of the studies was recorded. Four trials were privately funded; however, the authors of these studies did not disclose business relationship or financial gain from the technology they were studying. Most studies were publicly funded. Because of this we think we might conclude that conflict of interest is not a very important issue in this research domain.

Our study has also limitations. First, we studied interventions from a two-decade period. The technology of early CDSS is quite different from the more recent ones, and therefore the overall conclusions will be influenced by the effects of CDSSs that are no longer existing or whose technology may now be obsolete. Second, we included only published articles, all with some significant results. Because studies not showing a statistically significant superior effect of a CDSS may be less easily accepted for publication, a publication bias cannot be ruled out. Third, because of the heterogeneity of the interventions and outcome measures, a meta-analysis was not possible. Finally, because the follow-up period of most studies was only 1 year, it was not possible to assess the long-term outcome of a CDSS with or without additional support.

Our search strategy revealed only one cost-effectiveness study regarding a CDSS-based primary diabetes care management system. This hampers conclusions on the economic aspects of CDSSs.

Conclusions

CDSSs in primary T2DM care are effective in improving the process of care. The combination of a CDSS with feedback on performance, reminders, and case management is likely to be the most effective in improving patient outcome.

The long-term effectiveness and the cost-effectiveness of a CDSS in a primary care–based multifaceted diabetes management program remain to be elucidated.

Footnotes

Acknowledgments

This study was made possible by an unrestricted grant from Pfizer B.V.

Author Disclosure Statement

No competing financial interests exist. F.G.W.C. had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

References

Shah

, Hux

, Laupacis

, Zinman

, van Walraven

. Clinical inertia in response to inadequate glycemic control: do specialists differ from primary care physicians? Diabetes Care, 2005; 28:600–606.

Khunti

, Gadsby

, Millett

, Majeed

, Davies

. Quality of diabetes care in the UK: comparison of published quality-of-care reports with results of the Quality and Outcomes Framework for Diabetes. Diabet Med, 2007; 24:1436–1441.

Rutten

. [Care of patients with type 2 diabetes mellitus in primary care] Ned Tijdschr Geneeskd, 2008; 152:2389–2394.

Renders

, Valk

, Griffin

, Wagner

, van Eijk

, Assendelft

. Interventions to improve the management of diabetes in primary care, outpatient, and community settings: a systematic review. Diabetes Care, 2001; 24:1821–1833.

Kiefe

, Allison

, Williams

, Person

, Weaver

, Weissman

. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA, 2001; 285:2871–2879.

Ziemer

, Doyle

, Barnes

, Branch

Jr , Cook

, El-Kebbi

, Gallina

, Kolm

, Rhee

, Phillips

. An intervention to overcome clinical inertia and improve diabetes mellitus control in a primary care setting: Improving Primary Care of African Americans with Diabetes (IPCAAD) 8. Arch Intern Med, 2006; 166:507–513.

Garg

, Adhikari

, McDonald

, Rosas-Arellano

, Devereaux

, Beyene

, Sam

, Haynes

. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA, 2005; 293:1223–1238.

Jackson

, Bolen

, Brancati

, Batts-Turner

, Gary

. A systematic review of interactive computer-assisted technology in diabetes care. Interactive information technology in diabetes care. J Gen Intern Med, 2006; 21:105–110.

Formulier II, voor het beoordelen van een Randomised Controlled Trial (RCT)

www.cochrane.nl/Files/documents/Checklists/RCT.pdf. 2009 January 1.

10.

Glasgow

, Nutting

, King

, Nelson

, Cutter

, Gaglio

, Rahm

, Whitesides

, Amthauer

. A practical randomized trial to improve diabetes care. J Gen Intern Med, 2004; 19:1167–1174.

11.

Glasgow

, Nutting

, King

, Nelson

, Cutter

, Gaglio

, Rahm

, Whitesides

. Randomized effectiveness trial of a computer-assisted intervention to improve diabetes care. Diabetes Care, 2005; 28:33–39.

12.

Lobach

, Hammond

. Development, evaluation of a Computer-Assisted Management Protocol (CAMP): improved compliance with care guidelines for diabetes mellitus. Proc Annu Symp Comput Appl Med Care, 1994; 787–791.

13.

Lobach

, Hammond

. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med, 1997; 102:89–98.

14.

Phillips

, Ziemer

, Doyle

, Barnes

, Kolm

, Branch

, Caudle

, Cook

, Dunbar

, El-Kebbi

, Gallina

, Hayes

, Miller

, Rhee

, Thompson

, Watkins

. An endocrinologist-supported intervention aimed at providers improves diabetes management in a primary care site: Improving Primary Care of African Americans with Diabetes (IPCAAD) 7. Diabetes Care, 2005; 28:235–260.

15.

Cleveringa

, Welsing

, Van den Donk

, Gorter

, Niessen

, Rutten

, Redekop

. Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes Care, 2010; 33:258–263.

16.

Cleveringa

, Gorter

, Van den Donk

, Rutten

. Combined task delegation, computerized decision support, and feedback improve cardiovascular risk for type 2 diabetic patients: a cluster randomized trial in primary care. Diabetes Care, 2008; 31:2273–2275.

17.

Cleveringa

, Minkman

, Gorter

, van den

, Rutten

. Diabetes care protocol: effects on patient-important outcomes. A cluster randomized, non-inferiority trial in primary care. Diabet Med, 2010; 27:442–450.

18.

Khan

, MacLean

, Littenberg

. The effect of the Vermont Diabetes Information System on inpatient and emergency room use: results from a randomized trial. Health Outcomes Res Med, 2010; 1:e61–e66.

19.

MacLean

, Gagnon

, Callas

, Littenberg

. The Vermont diabetes information system: a cluster randomized trial of a population based decision support system. J Gen Intern Med, 2009; 24:1303–1310.

20.

Bond

, Burr

, Wolf

, Price

, McCurry

, Teri

. The effects of a web-based intervention on the physical outcomes associated with diabetes among adults age 60 and older: a randomized trial. Diabetes Technol Ther, 2007; 9:52–59.

21.

Holbrook

, Thabane

, Keshavjee

, Dolovich

, Bernstein

, Chan

, Thabane

, Gerstein

, Troyan

. Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ, 2009; 181:37–44.

22.

O'Connor

, Sperl-Hillen

, Rush

, Johnson

, Amundson

, Asche

, Ekstrom

, Gilmer

. Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med, 2011; 9:12–21.

23.

Cho

, Chang

, Kwon

, Choi

, Ko

, Moon

, Yoo

, Song

, Son

, Kim

, Lee

, Cha

, Son

, Yoon

. Long-term effect of the Internet-based glucose monitoring system on HbA1c reduction and glucose stability: a 30-month follow-up study for diabetes management with a ubiquitous medical care system. Diabetes Care, 2006; 29:2625–2631.

24.

Hurwitz

, Goodman

, Yudkin

. Prompting the clinical care of non-insulin dependent (type II) diabetic patients in an inner city area: one model of community care. BMJ, 1993; 306:624–630.

25.

Lafata

, Baker

, Divine

, McCarthy

, Xi

. The use of computerized birthday greeting reminders in the management of diabetes. J Gen Intern Med, 2002; 17:521–530.

26.

Ralston

, Hirsch

, Hoath

, Mullen

, Cheadle

, Goldberg

. Web-based collaborative care for type 2 diabetes: a pilot randomized trial. Diabetes Care, 2009; 32:234–239.

27.

Grant

, Wald

, Schnipper

, Gandhi

, Poon

, Orav

, Williams

, Volk

, Middleton

. Practice-linked online personal health records for type 2 diabetes mellitus: a randomized controlled trial. Arch Intern Med, 2008; 168:1776–1782.

28.

Guldberg

, Vedsted

, Kristensen

, Lauritzen

. Improved quality of Type 2 diabetes care following electronic feedback of treatment status to general practitioners: a cluster randomized controlled trial. Diabet Med, 2011; 28:325–332.

29.

Hetlevik

, Holmen

, Krüger

, Kristensen

, Iversen

, Furuseth

. Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess Health Care, 2000; 16:210–227.

30.

Hirsch

, Goldberg

, Ellsworth

, Evans

, Herter

, Ramsey

, Mullen

, Neighbor

, Cheadle

. A multifaceted intervention in support of diabetes treatment guidelines: a cont trial. Diabetes Res Clin Pract, 2002; 58:27–36.

31.

Ilag

, Martin

, Tabaei

, Isaman

, Burke

, Greene

, Herman

. Improving diabetes processes of care in managed care. Diabetes Care, 2003; 26:2722–2727.

32.

Meigs

, Cagliero

, Dubey

, Murphy-Sheehy

, Gildesgame

, Chueh

, Barry

, Singer

, Nathan

. A controlled trial of web-based diabetes disease management: the MGH diabetes primary care improvement project. Diabetes Care, 2003; 26:750–757.

33.

Peterson

, Radosevich

, O'Connor

, Nyman

, Prineas

, Smith

, Arneson

, Corbett

, Weinhandl

, Lange

, Hannan

. Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care, 2008; 31:2238–2243.

34.

Sequist

, Gandhi

, Karson

, Fiskio

, Bugbee

, Sperling

, Cook

, Orav

, Fairchild

, Bates

. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc, 2005; 12:431–437.

35.

Cleveringa

, Gorter

, van den Donk

, Rutten

36.

Holbrook

, Thabane

, Keshavjee

, Dolovich

, Bernstein

, Chan

, Troyan

, Foster

, Gerstein

. Individualized electronic decision support and reminders can improve diabetes care in the community: COMPETE II randomized trial. CMAJ, 2009; 181:37–44.

37.

Thomson O'Brien

, Oxman

, Davis

, Haynes

, Freemantle

, Harvey

. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2000; 2:CD000259.

38.

Coleman

, Austin

, Brach

, Wagner

. Evidence on the Chronic Care Model in the new millennium. Health Aff (Millwood), 2009; 28:75–85.

39.

Shojania

, Ranji

, McDonald

, Grimshaw

, Sundaram

, Rushakoff

, Owens

. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA, 2006; 296:427–440.