Technology-Enabled Outreach to Patients Taking High-Risk Medications Reduces a Quality Gap in Completion of Clinical Laboratory Testing

Abstract

Clinical laboratory quality improvement (QI) efforts can include population test utilization. The authors used a health care organization's Medical Data Warehouse (MDW) to characterize a gap in guideline-concordant laboratory testing recommended for safe use of antirheumatic agents, then tested the effectiveness of laboratory-led, technology-enabled outreach to patients at reducing this gap. Data linkages available through the Kaiser Permanente Colorado MDW and electronic health record were used to identify ambulatory adults taking antirheumatic agents who were due/overdue for alanine aminotransferase (ALT), aspartate aminotransferase (AST), complete blood count (CBC), or serum creatinine (SCr) testing. Outreach was implemented using an interactive voice response system to send patients text or phone call reminders. Interrupted time series analysis was used to estimate reminder effectiveness. Rates of guideline-concordant testing and testing timeliness in baseline vs. intervention periods were determined using generalized linear models for repeated measures. Results revealed a decrease in percentage of 3763 patients taking antirheumatic agents due/overdue for testing at any given time: baseline 24.3% vs. intervention 17.5% (P < 0.001). Among 3205 patients taking conventional antirheumatic agents, concordance for all ALT testing was baseline 52.8% vs. intervention 65.4% (P < 0.001) among patients chronically using these agents and baseline 20.6% vs. intervention 26.1% (P < 0.001) among patients newly starting these agents. The 95^th percentiles for days to ALT testing were baseline 149 vs. intervention 117 among chronic users and baseline 134 vs. intervention 92 among new starts. AST, CBC, and SCr findings were similar. Technology-enabled outreach reminding patients to obtain laboratory testing improves health care system outcomes.

Introduction

Laboratory quality improvement (QI) efforts are moving beyond activities that emphasize specimen analysis and into test utilization. This expanded QI agenda can address health care system outcomes (eg, appropriateness of test ordering, test completion, timeliness of testing) and/or clinical outcomes (eg, test results elucidating clinical status) at a population level.

Prerequisite to expanded laboratory QI activities are identifying and assessing relationships across laboratory (eg, test results), administrative (eg, patient demographics), and clinical (eg, diagnosis) data, but often administrative and clinical data are not readily available to laboratory personnel. Because medical data warehouses (MDWs) are comprehensive repositories of data originating from sources such as clinical data from electronic health records (EHRs), laboratory information systems (LIS), diagnosis and procedure claims (administrative data), and pharmacy dispensing information, MDWs are potentially useful tools for laboratory QI activities. Moreover, MDWs are structured to enable linkages across data types and to assess associations and outcomes.^1

–4

Although MDW data have been used for research, surveillance, and patient care,^{5

–20} they have not been employed to answer laboratory test utilization questions. The research team previously described the multidisciplinary process by which the Kaiser Permanente Colorado (KPCO) MDW, the Virtual Data Warehouse (VDW), was used to characterize quality gaps in laboratory testing and to select one gap for a laboratory-based intervention.²¹ The team selected laboratory monitoring of high-risk medications in ambulatory care, specifically alanine aminotransferase (ALT), aspartate aminotransferase (AST), complete blood count (CBC), and serum creatinine (SCr) testing among the rheumatology patient population receiving conventional or targeted synthetic disease-modifying antirheumatic drugs (conventional DMARDs; cDMARDs) and CBC testing among the rheumatology patient population receiving tumor necrosis factor (TNF) and non-TNF biologic antirheumatic drugs (biologic DMARDs; bDMARDs). Throughout this paper the more general term “antirheumatic agents” is used to refer to both conventional and biologic antirheumatic agents, while cDMARD or bDMARD refers to the indicated antirheumatic agents.

The KPCO antirheumatic agents laboratory testing guidelines are based on the American College of Rheumatology recommendations.²² However, there was a gap in timeliness to guideline-concordant testing²² – an important gap in that abnormal results for these laboratory tests are actionable and prompt intervention can reduce toxicity. As an example of the timeliness gap, our guidelines recommend monitoring CBC, ALT, AST, and SCr at 12-week intervals for patients taking methotrexate >6 months, and at 2–4 week intervals for patients taking methotrexate <3 months.²² Among 1520 ambulatory patients at KPCO taking methotrexate between January 1, 2014 and September 9, 2016, 60.7% had at least 1 ALT testing gap exceeding 100 days, although only 14% had at least 1 ALT testing gap exceeding 210 days.²¹ Similar timeliness of testing gaps were noted for AST, CBC, and SCr. The fact that most patients eventually completed testing reflected our “safety net” resource-intensive process. When a patient requests a refill of an antirheumatic agent, nursing personnel review the patient's record to determine whether the patient is overdue for laboratory testing. If so, the nurse reminds the patient to get the testing done and reinforces that an antirheumatic agent refill may not be authorized until testing is completed. In preliminary work to determine a QI intervention appropriate to address the timeliness of testing gaps, the research team identified that, for most patients taking antirheumatic agents, the tests had been ordered (ie, lack of current laboratory orders was not contributing to the gap). This finding steered the team to develop a laboratory-led intervention intended to improve the timeliness of test completion by intervening before the patient was overdue for testing. Herein is described the direct-to-patient interactive voice response (IVR) intervention that was developed and implemented. Findings from health care system outcomes also are presented.

Methods

Setting, population, and antirheumatic agents

This work was conducted at KPCO, an integrated health care delivery organization in the United States that in 2017 had about 700,000 members, approximately 600,000 of whom resided in a single metropolitan area. Medical offices offer integrated clinical, pharmacy, radiology, and laboratory services. All offices have a fully integrated ambulatory EHR in all patient care areas that is accessible from any other facility within KPCO.

Based on usage at KPCO, the antirheumatic agents included in this project were the cDMARDs methotrexate, leflunomide, sulfasalazine, tofacitinib, and azathioprine. The bDMARDs included were tocilizumab, adalimumab, certolizumab pegol, etanercept, golimumab, infliximab, and rituximab. The KPCO VDW was used to identify patients enrolled in the health plan who had medical and drug benefits, were aged 18 years or older, and whose current antirheumatic agent prescription was from a rheumatologist (to ensure the patient had a rheumatologic indication for use of the medication). For daily outreach programming, a current antirheumatic agent prescription was defined by the outreach date falling within the date range of the most recent dispensed days' supply plus 45 days or, in the case of an infused antirheumatic agent, within 45 days of the date of the most recent infusion. Additionally, there could be no EHR documentation that the antirheumatic agent had been discontinued. Iteratively applying these criteria provided real-time lists of patients preliminarily eligible for intervention.

VDW, EHR, and IVR system

The VDW was used to extract many data elements needed. VDW content areas include patient demographics, enrollment, encounters, diagnoses, procedures, death, cause of death, tumor, census, pharmacy, vital signs, social history, provider data, orders, and LIS data. VDW data tables are linked by a common, unique patient identifier that differs from the patient's health record number, with the crosswalk between the VDW identifier and the patient's health record number maintained in a separate table. VDW data are typically updated monthly or quarterly. VDW implementation and operations are governed by an operations committee. Ongoing processes ensure VDW quality is assessed and improved through programming and crowdsourcing via the user base.²

Because VDW tables are not updated daily and the intervention requires near real-time data, the EHR was used to gather data that were new since the last VDW update for patients preliminarily eligible for intervention. Specifically, the EHR was employed to obtain current laboratory orders and the most recent test completion data, to identify patients started on an antirheumatic agent since the last VDW pharmacy table update, and to confirm that there was no documentation of recent antirheumatic agent discontinuation.

The KPCO IVR system was employed for the intervention. The system uses a commercial database to distinguish cellular phones from landlines and to determine whether a phone is text enabled. The system sends text messages to text-enabled phones and calls to landlines and cellular phones that are not text enabled. Text messages are prioritized for text-enabled phones, but patients can request not to receive text messages. This IVR system has been used in multiple previous population-based projects, including several that subsequently were incorporated into usual health care operations.^23
–25

Determining the laboratory testing intervention was applicable to each patient

Whether patients were due or overdue for testing was based on the following: (1) methotrexate, leflunomide, sulfasalazine, or azathioprine: ALT, AST, CBC, and SCr every 4 weeks for patients newly started on the cDMARD with <3 months of therapy and every 12 weeks for patients on chronic therapy ≥3 months; (2) tocilizumab or tofacitinib: ALT, AST, SCr, and CBC every 6 weeks for patients with <3 months of therapy and every 12 weeks for patients with ≥3 months of therapy; (3) adalimumab, certolizumab pegol, etanercept, golimumab, or infliximab: CBC every 6 months; and (4) rituximab: CBC every 3 months. These tests and testing frequencies were based on published recommendations^22,26,27 along with consensus within the study institution rheumatology department about how to operationalize published recommendations for antirheumatic agent laboratory testing.

When a patient's last completed testing date exceeded the aforementioned antirheumatic agent-specific interval (overdue) or would exceed that interval within 3 days (due), the patient was confirmed to be eligible for intervention. Patients with orders for the due/overdue laboratory tests and currently taking an antirheumatic agent received the intervention.

Because of the recommended testing frequency, and because the intervention continued for 9 months (September 6, 2017, through June 5, 2018), many patients with ongoing antirheumatic agent use became due for testing more than once. The next due date for each laboratory test was calculated from the date that laboratory test was last completed and the recommended testing frequency for the antirheumatic agent. If a patient was taking more than 1 antirheumatic agent, and the guideline-recommended testing frequencies differed between the agents, the patient received the intervention when testing was due for the antirheumatic agent with the shorter testing interval. For example, a patient taking methotrexate and etanercept who completed CBC testing on January 1, 2018, was next due for CBC testing 12 weeks later, on March 26, 2018.

Intervention outreach approach

A daily list of patients to receive the intervention was sent electronically to the IVR system Monday through Friday; the Monday list included patients confirmed eligible for intervention on Saturday or Sunday. The IVR database was used to identify patient phone numbers. If the phone number was identified as text enabled, outreach was delivered by a text reminding the patient to obtain the testing; a phone message was delivered if the phone was not text enabled. Reminders were sent 3 days prior to the laboratory testing due date. If the patient had not completed the testing within 28 days following the reminder, the patient received a second reminder. Reminders were provided in English or Spanish based on language preference determined using the VDW.

Reminders maintained patient confidentiality. The text message stated: “This message is for <first name>. You are due for a lab test to continue taking your medicine safely. Go to any Kaiser Permanente lab within the next week. No appt is needed. Call <XXX-XXX-XXXX> for a list of lab locations and hours or with questions. Reply STOP to stop receiving texts from this system.” The phone message was similar. If patients called the <XXX-XXX-XXX> number, they reached a helpline staffed by a trained research assistant.

With any population-based intervention, change management issues should be encouraged to surface and then be addressed.²⁸ Therefore, comments were collected from patients who called the helpline and input was sought from rheumatology clinicians.

Analyses

The research team examined the effect of the intervention on timeliness of testing using interrupted time series analyses. These analyses identified patients on antirheumatic agents and whether they were due for testing during the baseline through intervention periods. The team modeled an outcome of due/overdue versus not due for testing in binomial models that included time and a variable indicating baseline or intervention period.

The team also compared (1) overall rates of patients in concordance with guidelines for testing during the one-year baseline (September 6, 2016, through September 5, 2017) and 9-month intervention (September 6, 2017, through June 5, 2018) periods and (2) timeliness of testing during baseline and intervention. Because this project was conducted in an ambulatory setting (eg, patients might have to coordinate attendance at the laboratory for testing around work schedules, patients must arrange transportation), in analyses of baseline and intervention periods “grace periods” were incorporated into definitions of guideline concordance. One additional week was allowed to complete testing when testing was recommended every 4 or 6 weeks (ie, testing within 5 or 7 weeks was defined as guideline concordant). When testing was recommended every 12 weeks (84 days), testing within 100 days was defined as guideline concordant. When testing was recommended every 6 months (183 days), testing within 200 days was defined as guideline concordant.

Generalized linear models were used to account for the repeated measures of individuals over time. For test completion within an interval, the team analyzed completed versus not completed in models that included a random subject effect. Eligible individuals and starts of time periods were based on the first date of antirheumatic agent dispensing for individuals newly starting the agent, or the date of the previous specific laboratory test result being analyzed for individuals with chronic use. For inclusion, patients had to have evidence of antirheumatic agent continuation when testing was due, either from adequate days of antirheumatic agent dispensed or subsequent dispensings or infusions. Linear models with a random subject effect were used for estimates of time between laboratory testing. Time between testing was calculated for intervals with both a start and end date for the interval and were limited to a maximum of 273 days (ie, the 9-month intervention time frame). All analyses were conducted using SAS Version 9.04 SAS Studio (SAS Institute Inc., Cary, NC).

The KPCO Institutional Review Board determined that this project did not meet the regulatory definition of research involving human subjects.

Results

Across the combined baseline and intervention periods, the cohort included 3763 patients. The cohort had a median age of 60.1 years and the majority were female (Table 1).

Table 1.

Rheumatology Patients Prescribed Antirheumatic Agents, September 6, 2016, Through June 5, 2018

Characteristic	Patients N = 3763
Age in years, median (5^th, 95^th percentile)	60.1 (31.7, 81.4)
Female (%)	2677 (71.1)
Race/Ethnicity^a (%)
Asian	106 (2.8)
Black	186 (4.9)
Hispanic	653 (17.4)
White	2621 (69.7)
Other	108 (2.9)
Diagnosis (%)
Rheumatoid arthritis	2466 (65.5)
Psoriatic arthritis	270 (7.2)
Inflammatory polyarthritis	349 (9.3)
Ankylosing spondylitis	132 (3.5)
Other	546 (14.5)
Project period with antirheumatic agent use^b (%)
Both baseline and intervention	2829 (75.2)
Baseline only	510 (13.6)
Intervention only	424 (11.3)

Race/ethnicity information was missing for 89 (2.4%) patients.

Baseline: September 6, 2016–September 5, 2017; intervention: September 6, 2017–June 5, 2018.

Three-fourths of the cohort (N = 2829, 75.2%) had antirheumatic agent use in both the baseline and intervention periods. During the baseline and/or intervention periods, antirheumatic agents were used chronically (≥3 months) by 2113 (56.2%) patients, whereas 1332 (35.4%) patients newly started (<3 months) 1 or more agents. For 318 (8.5%) patients it was not clear whether antirheumatic agent use was new or chronic because they had <6 months of KPCO membership prior to the first dispensing/administration. Fully 1324 (35.2%) patients were dispensed more than 1 antirheumatic agent.

Across the baseline and intervention periods, among the 3763 patients, the most commonly used cDMARD was methotrexate (N = 2385 [63.4%]) followed by leflunomide (N = 701 [18.6%]), sulfasalazine (N = 652 [17.3%]), and azathioprine (N = 195 [5.2%]). The bDMARDs were used far less often, with the most commonly used bDMARD being etanercept (N = 609 [16.2%]), followed by adalimumab (N = 421 [11.2%]), infliximab (N = 204 [5.4%]), and rituximab (N = 114 [3.0%]). Each of the other cDMARDs or bDMARDs was used by fewer than 75 (<2.0%) patients; these infrequently used antirheumatic agents were included in the interrupted time series analyses, but otherwise were not analyzed further.

Figure 1 displays the interrupted time series results. The models demonstrated a drop in the percentage of patients due/overdue for testing at any given time from 24.3% (95% CI 23.8, 24.9) during baseline to 17.5% (95% CI 16.9, 18.1) during intervention (P < 0.001).

FIG. 1.

Interrupted time series analysis showing proportions of patients taking antirheumatic agents, including methotrexate, leflunomide, sulfasalazine, azathioprine, adalimumab, etanercept, infliximab, rituximab, tofacitinib, tocilizumab, certolizumab pegol, and/or golimumab, who were due or overdue for recommended laboratory testing during baseline (September 6, 2016, through September 5, 2017) and intervention (September 6, 2017, through June 5, 2018) time periods (each data point represents every seventh day).

Among patients chronically taking cDMARDs, all ALT testing was completed at guideline-concordant frequencies for 52.8% during baseline vs. 65.4% during intervention (P < 0.001) (Table 2). Across patients newly starting one of the 4 commonly used cDMARDs, all ALT testing was completed at guideline-concordant frequencies by 20.6% during baseline vs. 26.1% during intervention (P < 0.001) (Table 2). Similar statistically significant increases in the proportions of patients with all AST, CBC, and SCr testing completed at the recommended frequencies were observed across cDMARDs during the intervention compared to baseline (Table 2).

Table 2.

Percentages of Patients Completing All Recommended Laboratory Testing at Guideline-Concordant Frequencies Among Patients Taking Antirheumatic Agents

Laboratory test type	Patients newly-started on antirheumatic agent therapy^a		P value^b	Patients chronically taking antirheumatic agent therapy^c		P value^b
Laboratory test type	Baseline % (95% CI)	Intervention % (95% CI)	P value^b	Baseline % (95% CI)	Intervention % (95% CI)	P value^b
Methotrexate, Leflunomide, Sulfasalazine, and Azathioprine (N = 3205)^d
Alanine aminotransferase	20.6	26.1	0.02	52.8	65.4	<0.001
Alanine aminotransferase	(17.8, 23.7)	(22.5, 30.0)	0.02	(50.6, 54.9)	(63.3, 67.5)	<0.001
Aspartate aminotransferase	20.6	25.5	0.04	52.6	65.3	<0.001
Aspartate aminotransferase	(17.8, 23.7)	(21.9, 29.3)	0.04	(50.5, 54.7)	(63.2, 67.4)	<0.001
Serum creatinine	21.6	27.5	0.01	55.0	67.0	<0.001
Serum creatinine	(18.7, 24.7)	(23.9, 31.5)	0.01	(52.9, 57.1)	(65.0, 69.0)	<0.001
Complete blood count	21.7	27.4	0.02	55.0	66.9	<0.001
Complete blood count	(18.9, 24.8)	(23.8, 31.4)	0.02	(52.9, 57.1)	(64.8, 68.9)	<0.001
Adalimumab, Etanercept, and Infliximab (N = 1138)^e
Complete blood count	N/A^e	N/A	N/A	87.3	89.5	0.19
Complete blood count	N/A^e	N/A	N/A	(84.7, 89.6)	(86.8, 91.6)	0.19
Rituximab (N = 114)^f
Complete blood count	N/A^f	N/A	N/A	70.8	70.3	0.94
Complete blood count	N/A^f	N/A	N/A	(59.4, 80.1)	(57.9, 80.3)	0.94

Newly-started therapy defined as initiated within <3 months.

Estimated from generalized linear mixed models with patient identifier as a random effect.

Chronic antirheumatic agent therapy defined as ongoing for ≥3 months.

Completion of recommended laboratory testing for patients who newly started therapy defined as ALT, AST, CBC, and SCr each completed within every 35 days (ie, every 28 days plus allowing a 7-day grace period) for the first 3 months of therapy. Completion of recommended laboratory testing for patients taking chronic therapy defined as ALT, AST, CBC, and SCr each completed within every 100 days (ie, every 12 weeks [84 days] plus allowing a 16-day grace period) during the baseline or intervention period.

CBC testing is not applicable to patients newly starting adalimumab, etanercept, or infliximab because CBC testing is recommended every 6 months.

CBC testing is not applicable to patients newly starting rituximab because CBC testing is recommended every 3 months.

ALT, alanine aminotransferase; AST, aspartate aminotransferase; CBC, complete blood count; CI, confidence interval; N/A, not applicable; SCr, serum creatinine.

In contrast, among patients chronically taking cDMARDs, ALT testing was completed at the guideline-concordant frequency at least once by 88.8% during baseline vs. 89.8% during intervention (P = 0.28). Across patients newly starting cDMARDs, ALT testing was completed at the guideline-concordant frequency at least once by 58.9% during baseline vs. 64.3% during intervention (P = 0.05). Among patients chronically taking bDMARDs, the percentage of patients completing CBC testing during the baseline and intervention periods did not differ (adalimumab, etanercept, and infliximab P = 0.19; rituximab P = 0.94) (Table 2).

Among patients newly starting cDMARDs, the mean days to ALT test completion was shorter during the intervention than the baseline period (49.4 vs. 54.5; P = 0.007) (Table 3). More important to intervention effectiveness for patients newly starting cDMARDs, the 95^th percentile for days to ALT test completion was shorter during the intervention (92 days) than the baseline (134 days) period (Figure 2). Results for AST, CBC, and SCr testing for patients newly starting cDMARDs were similar (Table 3 and Figure 2).

FIG. 2.

Median and 95^th percentile days to completion of recommended laboratory testing among patients taking the conventional antirheumatic agents methotrexate, leflunomide, sulfasalazine, and/or azathioprine (boxes show medians; 95^th percentiles are the upper boundaries of the vertical lines).

Table 3.

Mean Days to Completion of Recommended Laboratory Testing Among Patients Taking Antirheumatic Agents

Laboratory test type	Newly-started antirheumatic agent therapy^a		P value^b	Chronic antirheumatic agent therapy^c		P value^b
Laboratory test type	Baseline	Intervention	P value^b	Baseline	Intervention	P value^b
Methotrexate, Leflunomide, Sulfasalazine, or Azathioprine
Alanine aminotransferase, mean (SD)	54.5	49.4	0.007	69.7	70.4	0.22
Alanine aminotransferase, mean (SD)	(51.7, 57.3)	(46.5, 52.4)	0.007	(68.4, 70.9)	(69.1, 71.8)	0.22
Aspartate aminotransferase, mean (SD)	54.7	49.7	0.007	69.6	70.7	0.10
Aspartate aminotransferase, mean (SD)	(51.9, 57.6)	(46.7, 52.6)	0.007	(68.4, 70.9)	(69.4, 72.0)	0.10
Complete blood count, mean (SD)	53.2	48.6	0.01	66.9	66.0	0.14
Complete blood count, mean (SD)	(50.4, 56.0)	(45.7, 51.5)	0.01	(65.6, 68.2)	(64.7, 67.3)	0.14
Serum creatinine, mean (SD)	53.4	48.2	0.005	65.7	64.3	0.02
Serum creatinine, mean (SD)	(50.6, 56.2)	(45.3, 51.1)	0.005	(64.4, 67.0)	(63.0, 65.7)	0.02
Adalimumab, Etanercept, Infliximab
Complete blood count, mean (SD)	N/A	N/A	N/A	82.8	86.7	0.008
Complete blood count, mean (SD)	N/A	N/A	N/A	(79.3, 86.1)	(83.3, 90.1)	0.008
Rituximab
Complete blood count, mean (SD)	N/A	N/A	N/A	65.2	59.1	0.09

Newly starting antirheumatic agent therapy: <3 months. Days between antirheumatic agent initiation and completion of laboratory testing for newly-started antirheumatic agent therapy.

Estimated from generalized linear mixed models with subject identifier as a random effect.

Chronic antirheumatic agent therapy: ≥3 months. Days since previous laboratory testing completion date to next laboratory testing completion date for patients taking chronic antirheumatic agent therapy.

N/A, not applicable; SD, standard deviation.

Among patients with chronic use of cDMARDs, the mean days to ALT test completion was <12 weeks during both intervention and baseline (70.4 vs. 69.7 days; P = 0.22) (Table 3), reflecting patients who completed testing well within the guideline-concordant time frame. Most important to intervention effectiveness for patients chronically taking cDMARDs, the 95^th percentile for ALT test completion was much shorter during the intervention (117 days) than during the baseline (149 days) period (Figure 2). In other words, fewer patients chronically taking cDMARDs were far overdue for ALT testing during intervention than during baseline. Results for AST, CBC, and SCr testing for patients chronically taking cDMARDs were similar (Table 3 and Figure 2).

Among patients taking the bDMARDs adalimumab, etanercept, or infliximab (CBC every 6 months or 183 days), mean time to CBC completion was less than 6 months during both baseline and intervention (82.8 vs. 86.7 days; P = 0.008) (Table 3). For patients chronically taking these bDMARDs, the 95^th percentile for CBC completion was shorter during intervention (182 days) than during baseline (196 days). For patients taking rituximab (CBC every 3 months), the mean time to CBC completion was <3 months during both baseline and intervention (65.2 vs. 59.1 days; P = 0.09). For rituximab, the 95^th percentile for CBC completion was much shorter during intervention (93 days) than during baseline (174 days). Again, fewer patients taking bDMARDs were far overdue for testing during intervention than during baseline.

No change management issues were identified. Patients, physicians, nurses, and pharmacists were satisfied with the intervention. None of the 94 patients who called the helpline complained about the reminders. Moreover, 59 expressed appreciation for the reminders. For example, one patient commented “I have trouble keeping track of my labs when they are three months out, so I appreciate the reminder and would love reminders for all of my labs from various departments.” Physicians requested the intervention be implemented into routine operations, “This is a worthwhile investment in patients' care and coordination of care. We need to think deeply about how to fund this further for the organization….” A clinical pharmacy specialist stated that this program improved quality by supporting safe use of medication, improved service by providing less disruptive messaging to patients, and improved affordability by supporting adherence and appropriate use of medications. Finally, the rheumatology department nursing manager commented that nurses had to remind fewer patients to obtain overdue laboratory testing and that this project reduced that aspect of their workload.

Discussion

This study found that technology-enabled reminders to rheumatology patients to obtain recommended laboratory testing for antirheumatic agents reduced the proportion of patients with testing gaps, substantially reduced the occurrence of long-overdue testing, and was liked by patients and health care professionals. The reminders were effective among patients newly starting antirheumatic agents and among patients taking antirheumatic agents chronically. Further, this study demonstrated the usefulness of an MDW in addressing this quality of care problem. Importantly, it established that the clinical laboratory can take a lead role in a QI intervention that addresses test utilization and links to outcomes.

The intervention was consistently effective at reducing the proportions of patients with testing gaps across laboratory test types and across antirheumatic agents. Potential explanations for this are that patients forget to complete their laboratory tests or underestimate the length of time since their last testing. This explanation is bolstered by the observation that the most common question patients asked when calling the helpline was essentially “Am I really due for lab tests?”

This work also identified patients who consistently completed testing at shorter time frames than recommended by guidelines. The laboratory tests examined here are commonly ordered, and the fact that the mean days to test completion generally were less than the guideline-recommended time frame suggests that some patients had these tests completed for reasons other than antirheumatic agent monitoring.

Consistent and timely laboratory testing for cDMARDs appears difficult for many patients. For example, although 64.3% of new users had at least 1 interval of timely ALT testing, the proportion who completed all ALT testing in a guideline-concordant manner was 26.1% (eg, testing every 4 weeks).

This work targeted safe use of antirheumatic agents in ambulatory care. Although American College of Rheumatology guidelines²² recommending laboratory monitoring with antirheumatic agent use are consensus based, they are rooted in evidence that risk of toxicity can be minimized if laboratory tests that identify potential organ system toxicity are monitored and antirheumatic agent dosage(s) adjusted or discontinued until test result values return to normal. Using ALT/AST as the example, if testing identifies newly-abnormal results, the antirheumatic agent dosage can be reduced and ALT/AST result values subsequently monitored until they normalize, thereby avoiding serious hepatotoxicity.

The national burden associated with antirheumatic agent laboratory test completion gaps is significant. For example, ALT/AST abnormalities occur with methotrexate in up to 15% of patients at 12 to 52 weeks of therapy; as many as 5% discontinue methotrexate because of hepatotoxicity.²⁹ Similarly, leflunomide carries an approximately 9% risk of ALT/AST rising to 2- to 3-fold the upper limits of normal, with 2 to 4.9 per 100,000 patients per year experiencing hepatotoxicity resulting in hospitalization.^30,31 The importance of underutilization of laboratory testing for antirheumatic agents is clear when taken to scale. Rheumatoid arthritis is present in 0.5%-1% of the US population. Methotrexate or other agents with similar risk profiles are used for most patients with rheumatoid arthritis, as well as for patients with other immune-related diseases. Underutilization of testing is linked to the inability to increase the antirheumatic agent dosage to maximize efficacy (eg, when ordered testing is not completed, test results are not available to confirm lack of toxicity) and with lack of early recognition of emerging toxicity (eg, non-completed testing fails to identify abnormalities that should lead to dosage adjustment to avoid serious toxicity).

Although adverse health outcomes associated with laboratory testing and underutilization of testing among patients taking antirheumatic agents are clear, to the research team's knowledge this work is the first to examine concordance with published guidelines and regulatory labeling,^26,27 as well as the first to identify a gap in timeliness of testing. Other strengths of this work include the direct-to-patient outreach to improve testing adherence, and that the outreach is laboratory led. These factors, coupled with employing MDW data resources to achieve the outreach contribute to this novel project.

A final strength of this work is that the standardized data structure of the VDW is beneficial for translating this approach to other sites and other health care settings. The VDW data structure is shared across member organizations of the Health Care Systems Research Network (HCSRN) and is similar to that of distributed data networks including PCORnet and the US Food and Drug Administration Sentinel Distributed Database. Further, the EHR and LIS platforms on which this outreach was built are Epic and Cerner respectively, the most common health information technology platforms in the United States. Clinical laboratories at organizations with MDW and access to near real-time pharmacy and laboratory data could implement this intervention in their organizations or utilize this approach in an array of other QI interventions targeted to specific needs.

This work also has limitations. A larger patient population is necessary to have sufficient power to assess patient outcomes than is necessary to assess health care system outcomes. This 9-month intervention had sufficient power to assess system outcomes, but insufficient power to assess patient outcomes. A possible limitation to the generalizability of this work is that patients similar to the study population rarely obtain medications from out-of-plan pharmacies.³² Further, the research team reviewed the medical records of multiple patients and found only 1 instance in which a patient had testing completed at an external laboratory and the test results were not accessible using the VDW. Thus, although neither out-of-plan dispensing nor external laboratory testing were of concern in the study setting, it likely would be inefficient to implement this approach at health plans where such occurrences are commonplace because of the missing pharmacy and laboratory testing data. Also, health plan settings that do not have near real-time access to relatively complete dispensing and laboratory testing information could be unable to accurately identify patients for outreach. Finally, this intervention design is not generalizable to settings with a quality gap in test ordering, as the intervention is triggered when current orders for the laboratory tests are present.

The cost-effectiveness of this intervention was not formally assessed. However, several factors point to its potential cost-effectiveness. Costs include development, operationalization, and maintenance of the intervention. In similar technology-enabled interventions at KPCO, the internal cost to build and operationalize the outreach using the IVR system was budgeted at $11,000 and the yearly ongoing maintenance costs were about $5000.^23,33 Savings in the present project include nursing, physician, and pharmacist work/rework avoided attributable to the intervention. Informally, nursing personnel estimated they avoided 20 to 30 minutes per patient contacted per month; physician and pharmacist work avoided was not approximated. Implementation costs in other settings will vary related to differences in system costs and workflow patterns, but technology can support professional staff and decrease the time they must devote to routine tasks.

The research team concludes that technology-enabled outreach to remind patients to obtain laboratory testing for safe use of antirheumatic agents improves health care system outcomes and is well accepted by patients and health care professionals. Although this project reduced gaps associated with non-completed laboratory orders in a population taking antirheumatic agents, the project itself serves only as an example of this type of QI intervention. This approach is scalable locally and nationally. Opportunities exist for clinical laboratories to develop, implement, and evaluate population-based projects that employ an MDW to characterize quality gaps, facilitate QI interventions, and assess the effectiveness of those interventions.

Footnotes

Acknowledgements

We thank the following individuals from Kaiser Permanente Colorado: Lisa Pieper, MBA, MSHA, for project management; Megan Baldwin, BS, for project assistance; Glenn K. Goodrich, MS, for programming expertise. We also thank David H. Smith, PhD, from the Center for Health Research, Kaiser Permanente Northwest, Portland, Oregon, for his scientific contributions to project development.

Author Disclosure Statement

The authors declare that there are no conflicts of interest. This project was funded through a cooperative agreement from the Centers for Disease Control and Prevention (5NU47OE000102-02-00) to Kaiser Permanente Colorado. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention or the US Agency for Toxic Substances and Disease Registry.

References

Krasowski

, Schriever

, Mathur

, Blau

, Stauffer

, Ford

. Use of a data warehouse at an academic medical center for clinical pathology quality improvement, education, and research. J Pathol Inform, 2015; 6:45.

Ross

, Ng

, Brown

, et al. The HMO Research Network Virtual Data Warehouse: a public data model to support collaboration. EGEMS (Wash DC), 2014; 2:1049.

Sentinel Coordinating Center. Sentinel common data model. 2017. https://www.sentinelinitiative.org/sentinel/data/distributed-database-common-data-model Accessed October 26, 2017 .

Curtis

, Weiner

, Boudreau

, et al. Design considerations, architecture, and use of the mini-sentinel distributed data system. Pharmacoepidemiol Drug Saf, 2012; 21 suppl 1:23–31.

Raebel

, Zeng

, Cheetham

, et al. Risk of breast cancer with long-term use of calcium channel blockers or angiotensin-converting enzyme inhibitors among older women. Am J Epidemiol, 2017; 185:264–273.

Andrade

, Raebel

, Brown

, et al. Use of antidepressant medications during pregnancy: a multisite study. Am J Obstet Gynecol, 2008; 198:194.e1–194.e5.

Raebel

, Ross

, Xu

, et al. Diabetes and drug-associated hyperkalemia: effect of serum potassium monitoring. J Gen Intern Med, 2010; 25:326–333.

Raebel

, Newcomer

, Reifler

, et al. Chronic use of opioid medication before and after bariatric surgery. JAMA, 2013; 310:1369–1376.

Raebel

, Ellis

, Schroeder

, et al. Intensification of antihyperglycemic therapy among patients with incident diabetes: a Surveillance Prevention and Management of Diabetes Mellitus (SUPREME-DM) study. Pharmacoepidemiol Drug Saf, 2014; 23:699–710.

10.

Nichols

, Schroeder

, Karter

, et al. Trends in diabetes incidence, 2006–2011, in seven million insured US adults: the SUPREME-DM project. Am J Epidemiol, 2015; 181:32–39.

11.

Elston Lafata

, Karter

, O'Connor

, et al. Medication adherence does not explain black-white differences in cardiometabolic risk factor control among insured patients with diabetes. J Gen Intern Med, 2016; 31:188–195.

12.

, Zang

, Lakoma

, et al. Changes in youth antidepressant use and suicidal behavior following FDA warnings and media coverage. BMJ, 2014; 348:g3596.

13.

Neugebauer

, Fireman

, Roy

, O'Connor

, Selby

. Dynamic marginal structural modeling to evaluate the comparative effectiveness and safety of more or less aggressive treatment intensification strategies in type 2 diabetes. Pharmacoepidemiol Drug Saf, 2012; 21:99–113.

14.

Nichols

, Desai

, Elston Lafata

, et al. Construction of a multisite DataLink using electronic health records for the identification, surveillance, prevention, and management of diabetes mellitus: the SUPREME-DM project. Prev Chronic Dis, 2012; 9:E110.

15.

Schroeder

, Goodrich

, Newton

, Schmittdiel

, Raebel

. Implications of different laboratory-based incident diabetic kidney disease definitions on comparative effectiveness studies. J Comp Eff Res, 2014; 3:359–369.

16.

Yih

, Lieu

, Kulldorff

, et al. Intussusception risk after rotavirus vaccination in US infants. N Engl J Med, 2014; 370:503–512.

17.

Schmittdiel

, Raebel

, Dyer

, et al. Medicare STAR excludes patients with poor CVD risk factor control. Am J Manag Care, 2014; 20:e573–e581.

18.

Flory

, Roy

, Gagne

, et al. Missing laboratory results data in electronic health databases: implications for monitoring diabetes risk. J Comp Eff Res, 2017; 6:25–32.

19.

Raebel

, Shetterly

, Lu

, et al. Methods for using clinical laboratory test results as baseline confounders in multi-site observational database studies when missing data are expected. Pharmacoepidemiol Drug Saf, 2016; 25:798–814.

20.

Raebel

, Penfold

, McMahon

, et al. Adherence to guidelines for glucose assessment in starting second-generation antipsychotics. Pediatrics, 2014; 134:e1308–e1314.

21.

Raebel

, Quintana

, Schroeder

, et al. Identifying preanalytic and postanalytic laboratory quality gaps using a data warehouse and structured multidisciplinary process. Arch Pathol Lab Med. 2018. [Epub ahead of print]; DOI: 10.5858/arpa.2018-0093-OA.

22.

Singh

, Saag

, Bridges

Jr , et al. 2015 American College of Rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Rheumatol, 2016; 68:1–26.

23.

Shoup

, Madrid

, Koehler

, et al. Effectiveness and cost of influenza vaccine reminders for adults with asthma or chronic obstructive pulmonary disease. Am J Manag Care, 2015; 21:e405–e413.

24.

Kempe

, Shetterly

, France

, Levin

. Automated phone and mail population outreach to promote colorectal cancer screening. Am J Manag Care, 2012; 18:370–378.

25.

Bender

, Cvietusa

, Goodrich

, et al. Pragmatic trial of health care technologies to improve adherence to pediatric asthma treatment: a randomized clinical trial. JAMA Pediatr, 2015; 169:317–323.

26.

Product Information: ARAVA(R) oral tablets, leflunomide oral tablets. Bridgewater, NJ: Sanofi-Aventis U.S. LLC, 2015.

27.

Product Information: RASUVO(TM) subcutaneous injection, methotrexate subcutaneous injection. Chicago, IL: Medac Pharma, Inc., 2014.

28.

Raebel

, Chester

, Brand

, Magid

. Imbedding research in practice to improve medication safety. In: Henriksen

, Battles

, Keyes

, Grady

, eds. Advances in patient safety: new directions and alternative approaches (vol 4: technology and medication safety). Rockville, MD: Agency for Healthcare Research and Quality, 2008.

29.

Lopez-Olivo

, Siddhanamatha

, Shea

, Tugwell

, Wells

, Suarez-Almazor

. Methotrexate for treating rheumatoid arthritis. Cochrane Database Syst Rev, 2014; CD000957.

30.

Suissa

, Ernst

, Hudson

, Bitton

, Kezouh

. Newer disease-modifying antirheumatic drugs and the risk of serious hepatic adverse events in patients with rheumatoid arthritis. Am J Med, 2004; 117:87–92.

31.

Alcorn

, Saunders

, Madhok

. Benefit-risk assessment of leflunomide: an appraisal of leflunomide in rheumatoid arthritis 10 years after licensing. Drug Saf, 2009; 32:1123–1134.

32.

Delate

, Albrecht

, Olson

. Out-of-plan pharmacy use by members of a managed care organization. Perm J, 2012; 16:15–21.

33.

Cvietusa

, Goodrich

, Shoup

, et al. Implementing health care technology research into practice to improve adult asthma management. J Allergy Clin Immunol Pract, 2019; 7:908–914.