Abstract
Self-reported data suggest American adults with multiple chronic conditions account for 25.7% to 60% of the population. Despite emphasis on information technology to improve quality in health care, data addressing outcomes of clinically focused, provider-oriented dashboards are limited. To explore integrating performance platforms into clinical care, the authors designed a platform-based intervention to address 2 prevalent chronic conditions with significant long-term burden. This study used a performance platform to enhance clinicians' management of patients with diabetes and osteoporosis. Descriptive statistics were used to summarize patients' surveys and quality metrics, and to analyze clinicians' knowledge, attitudes, and beliefs in the pre and post time frames. The frequency of screening for osteoporosis in women improved post intervention (40% vs. 44%, P < 0.0001), whereas other quality metrics did not. Clinician respondents were primarily physicians (82%), white (73%), internal medicine specialists (58%), with an average of 18 years' experience, and nearly equally male and female. Their percent of correctly answered knowledge questions increased slightly in the postintervention phase for osteoporosis and hypoglycemia (0.53 and 1.74, respectively); however, results were not statistically significant (P > 0.4). Post intervention, clinicians reported that their attitudes and beliefs regarding disease management had changed in the past 6 months in a positive direction. Although few outcomes studied changed over time, results suggest that performance platforms may have a role to play in managing chronic conditions. However, their efficacy must continue to be evaluated in order to improve understanding of optimal approaches to integrating technology into patient care.
Introduction
The burden of chronic conditions in the United States is increasing. Although rates vary depending on the conditions considered “chronic,” national sources report estimates of the number of community-dwelling American adults with multiple chronic conditions ranging from 25.7% to 60%, based on self-reported data collected in 2014. 1,2 Medicare data from 2017 reported that at least one quarter of beneficiaries across racial categories had 2 to 3 chronic conditions. 3 Managing even a single chronic disease like diabetes is complex given multiple treatment options and contributing lifestyle factors that can help or hinder disease progression. 4,5 Clinical platforms or dashboards represent one method to help providers categorize and track data needed to manage patients with multiple chronic conditions. Platform designs and components vary, but tools typically include an interactive, dynamic view of aggregated data on key indicators of clinical care or risk management or operational metrics; allow for benchmarking against other providers or systems; and visualize trends over time. 6
Despite the emphasis on using information technology to improve quality in health care in influential reports, 7 data addressing outcomes of clinically focused, provider-oriented dashboards are surprisingly limited. Recent reviews using systematic evidence identification techniques addressed different aspects of dashboards including use in intensive care settings, 8 the landscape of various kinds of dashboards, 6,9 and evidence for clinical- and quality-focused systems. 10 Some studies have demonstrated that use of dashboards results in positive effects on provider workload and efficiency of data collection, accurate and speedy clinical decision-making, compliance with care recommendations/quality of care, and an enhancement of visibility and integration of information and communication. 6,8,10 Across reports, however, data are limited and the extent of the effects of dashboards on clinical outcomes, provider knowledge and decision-making, and process changes are not well understood. Recent evaluations of dashboards/platforms similarly report mixed findings—some improvements in metrics tracked after implementation of the tools but dashboard use was typically limited. 11 –14 Researchers note, however, that performance platforms alone likely are not sufficient to effect change in care. 14
To continue to explore integrating performance platforms into clinical care, the study team designed a platform-based intervention to address 2 prevalent chronic conditions with significant long-term burden: diabetes and osteoporosis. For diabetes, the focus was on hypoglycemia in particular because hypoglycemia has been associated with increased risk of death and the full burden of hypoglycemia is unknown. 15,16 Likewise, osteoporosis may lead to debilitating fractures, yet patients at risk may not be placed on the optimal evidence-based treatments even after a diagnosis of osteoporosis. 17 –20 In addition, because performance platforms alone may be insufficient to direct behavior change, the intervention was augmented with complementary provider education and a patient-reported assessment tool. Given the short intervention time frame of 6 months, this study focused on the outcomes of clinician knowledge, attitudes, and beliefs (KAB) as a proxy marker for changes in management of osteoporosis and diabetes mellitus. Changes in attitudes and beliefs have been posited to predict an individual's engagement with a given construct. 21 –24
Methods
This study used a pre–post evaluation design to assess outcomes prior to the intervention and 6-months post intervention. The intervention studied was clinicians' use of a performance platform to manage diabetes and osteoporosis patients. Outcomes included changes in: (1) clinician KAB about osteoporosis and hypoglycemia management via clinician surveys; (2) population care quality measures via extraction of data from the electronic health record (EHR); and (3) clinician and patient engagement in hypoglycemia management via patient surveys.
Setting
The study was conducted at 2 outpatient practices affiliated with integrated delivery networks that represented different practice models (ie, for-profit, physician-owned system; not-for-profit academic medical center). Clinicians treating adult patients with diabetes and/or osteoporosis at each outpatient practice were the primary intervention targets. Both sites were located in suburban settings in the United States.
Intervention components
Dashboard
The dashboard, delivered via an electronic platform, displayed clinicians' monthly performance on quality metrics for patients with diabetes and osteoporosis (Table 1). Figure 1 displays dashboard capabilities. To access the platform, each clinician had to execute approximately 4 clicks. The performance platform was not embedded in the EHR.

Platform example. Figure can be viewed in greater detail online.
Quality of Care Metrics Tracked in Dashboard
Education
Clinicians were required to complete self-paced study modules/webinars addressing aspects of care for hypoglycemia in diabetes (ie, medication management using the Beers criteria, virtual experience, management of hypoglycemia) and osteoporosis (fracture risk assessment). All materials were accessible via the performance platform during the intervention period.
Peer-to-peer education calls
All participating site clinicians were invited to attend three 60-minute telephone conference calls offered over the course of the intervention. A study team facilitator led each call and a subject matter expert presented information on hypoglycemia assessment or management or osteoporosis education resources.
Patient-reported hypoglycemia survey
Because successful diabetes mellitus management requires frequent and comprehensive input from patients, patients who were continuously associated with the practice were approached to complete a hypoglycemia assessment at study initiation and 3 months later. Assessment results and risk scores were available in real time via the platform. The study team developed the assessment through literature review; drafting candidate questions addressing frequency, symptoms, and symptom burden associated with hypoglycemia; and face validity evaluation of the assessment by a panel of 5 physicians with expertise in managing diabetes. Questions were revised or omitted based on consensus opinion of the physician panel. Surveys were sent to available patient email addresses using an encrypted message with a secure link to the survey and informed consent and opt-out information. Assessments were scored within the platform to provide a risk stratification (high, medium, low) for patients in the platform. Higher risk reflected greater perceived burden of hypoglycemia.
Evaluation tools
The study team evaluated clinicians' KAB via pre and post surveys (delivered electronically via REDCap) developed through review of the literature and project team review and consensus; questions comprised true/false, multiple choice, and Likert-scale items. Surveys are available in the online Supplementary Materials.
The team did not attempt to fully validate the surveys. However, in order to determine if the instrument could detect changes between pre- and postintervention KAB, 13 clinicians who care for patients in the populations under study reviewed and tested the survey. The Likert scale (5 = Strongly Agree, 4 = Agree, 3 = Neither, 2 = Disagree, 1 = Strongly Disagree) was sensitive to change with percentages for the strongly agree option ranging from 0–62%. Strongly agree/agree responses were considered positive/correct; thus, higher mean scores indicate better “performance.”
Clinicians had unique identifiers so that pre and post surveys could be linked. Pre surveys assessed participant characteristics (eg, area of practice, age range, sex) and baseline KAB regarding hypoglycemia and osteoporosis management (eg, signs of severe hypoglycemia include confusion, odd behavior, speech difficulty). Postintervention surveys duplicated several pre-survey questions but also included additional questions to assess changes in KAB during the intervention period (eg, “Over the past 6 months, I have been more able to identity if a patient is at risk for osteoporosis during routine wellness visits”).
Data analysis
The study team conducted descriptive analyses of data collected over the study period. The team calculated the number/percentage of correct knowledge-related questions (11 hypoglycemia, 4 osteoporosis) at the pre- and postintervention periods and assessed differences using a 2-sided paired t test. For attitude and belief questions (12 hypoglycemia, 13 osteoporosis), average Likert scores were used to examine each clinician's overall viewpoints regarding management of each chronic condition. EHR data were used to compare preintervention and postintervention attainment of quality metrics using a chi-square test. The mean of the final HbA1c measurements in the pre- and postintervention periods are reported for each patient to provide the proportion of patients in each period with HbA1c <7% (very well controlled), and 7.5%, 8%, and 9%, the latter of which denotes worsening control.
Approval for the study was granted by the WIRB-Copernicus Group Institutional Review Board.
Results
Population characteristics
Table 2 outlines characteristics of the 55 participating clinicians. Most participants were physicians in internal medicine, self-identified as non-Hispanic Whites, and were located in the southern US. Participants had an average of 18 years in clinical practice. Study modules were completed by all participants. The 3 Peer-to-Peer education calls were attended by 22 (40%), 12 (22%), and 15 (27%) of participants, respectively.
Participant Characteristics
Number (%) unless otherwise specified.
IQR, interquartile range; SD, standard deviation.
Intervention effects on knowledge, attitudes, and beliefs
All 55 clinicians responded to the knowledge-related questions on the preintervention survey and 47 completed the postintervention survey. Knowledge scores were low, particularly for osteoporosis questions (Table 3). Both pre and post scores on the osteoporosis knowledge questions ranged from 0% to 75% with a median of 25%. Baseline scores on the hypoglycemia knowledge questions ranged from 9% to 91% with a median of 64%, and postintervention scores from 36% to 100% (median 64%). While the overall mean knowledge score improved slightly in the postintervention phase for osteoporosis and hypoglycemia, mean differences were not statistically significant (P > 0.4).
Knowledge Scores (% Correct)
All P = not significant.
The pre and post surveys contained 11 hypoglycemia knowledge questions and 4 osteoporosis knowledge questions.
SD, standard deviation.
For clinicians who completed the survey at both time points, preintervention attitudes and beliefs were positive (Table 4). In the full sample, mean score regarding management of osteoporosis (3.86, standard deviation (SD) = 0.46) was slightly higher than that for hypoglycemia (3.36, SD = 0.41). Post intervention, clinicians reported that their attitudes and beliefs regarding disease management had changed in the past 6 months in a positive direction.
Attitudes and Beliefs Mean Scores (1–5 Likert scale * )
Range: 1 = strongly disagree to 5 = strongly agree.
The pre and post surveys contained 12 hypoglycemia and 13 osteoporosis attitude and beliefs questions.
Dashboard use and quality of care metrics
All clinicians engaged in the study logged into the platform at least once over the course of the study. Frequency of access was higher in the first month (the average rate of log-in was about 50% over the 6 months of the project) and tapered to 22% in the final month. Clinicians could log in more than once in a month. If a clinician-participant logged in more than once a month, the multiple log-ins did contribute to the percentage reported.
Overall, the measure for frequency of screening for osteoporosis in women improved significantly post-intervention, whereas the recommended pharmacologic treatment of osteoporosis in men and women significantly declined (Table 5). There was no significant change in management of women who had a fracture. HbA1c testing and control measures for diabetes both worsened over the study period, with fewer patients having HbA1c values measured and fewer patients reaching HbA1c <9% in the postintervention period. Measures were not associated with any value-based payment or quality measure incentives for clinicians at both sites.
Comparison of Proportion of Patients Meeting Each Quality of Care Measure *
For improvement in care, the proportion of patients should be higher at post intervention compared to pre intervention.
For improvement in care, the proportion of patients should be lower at post intervention compared to pre intervention.
P < 0.01.
Engagement with the hypoglycemia survey
Hypoglycemia surveys were sent to patients with diabetes twice over the course of the study period. Patients with diabetes completed 911 assessments of the 12,026 sent over the intervention period. Because results were aggregated, it is unknown how many patients completed the assessment once versus both times. Most respondents (77%) were considered low risk for hypoglycemia and roughly half (53%) reported no hypoglycemia symptoms at all over the preceding 3 months. Among those reporting any hypoglycemia symptoms, nausea was the primary symptom experienced (14%) followed by feeling warm or hot (10%) and trembling or shakiness (4%). Nineteen percent of patients reported having symptoms that affected daily activities at least once over a 3-month period. Surveys were accessed via the dashboard by 10 clinician-participants but viewing surveys via direct messaging could not be tracked.
Discussion
Information technologies such as dashboards that identify, integrate, and categorize data may help to improve the quality of care and population management for patients with chronic conditions, but an understanding of the efficacy of these tools—and approaches for optimal integration into care—remain nascent. 7,25,26 This study explored the use of a performance platform in the context of diabetes mellitus and osteoporosis care delivery, both prevalent and complicated chronic conditions to manage. While this study demonstrated a significant increase in one osteoporosis screening measure, improvements did not carry across all quality outcomes. Measures assessing appropriate pharmacologic management of osteoporosis decreased during the intervention. To understand physician behavior around management of these conditions, this study evaluated KAB of diabetes and hypoglycemia and found only a slight improvement in clinician-participants' attitudes and beliefs around the management of hypoglycemia and osteoporosis; there was no change in clinician-participant knowledge of how to manage these two conditions.
There are many potential explanations for the lack of change in knowledge, and the only slight perceived change in attitudes and beliefs around the management of diabetes and osteoporosis. Clinicians who are busy managing multiple comorbid conditions in one visit may prioritize aspects of these conditions that they feel that they can most impact. With that reasoning, hypoglycemia, which is not often considered during a visit, particularly in patients with uncontrolled diabetes and elevated hemoglobin A1c, may be an example of a condition that is not often prioritized. In support of this theory, previous research studies addressing behavior change have demonstrated an association between perceived control over an action/behavior and intention to complete the action. If clinicians perceived limited control or ability to affect their patients' osteoporosis or episodes of hypoglycemia, that may have affected their attention to some elements of care. 21,27 –31 Clinicians' perceptions that their attitudes toward hypoglycemia and osteoporosis management did improve modestly, which suggests that the intervention may have raised awareness of the importance of these issues in care.
Osteoporosis and hypoglycemia knowledge scores were low at baseline and only improved minimally on the post-intervention assessment. This study did not analyze knowledge results by years of clinical experience but may be worthwhile to evaluate in future studies. While the low knowledge scores highlight a need for additional provider education on these areas, pre and post survey design likely also affected scores. Some of the knowledge questions had multiple correct answers (ie, choose all that apply), and the scoring system required that all correct responses be selected to count the item as correctly answered. The study team also assessed changes in KAB 6 months after participants completed educational materials, which may have lessened recall but reflects real-world practice, in which knowledge gained may not be used immediately to inform care and may diminish over time if not routinely used. This finding of limited effects of education on provider knowledge aligns with other research that reported small effects of digital education versus traditional learning approaches on knowledge of guidelines and diabetes management and limited effects of printed educational materials on areas including thiazide prescribing, glucose management, and retinal screening. 32 –37
This study's performance platform also included a patient-reported hypoglycemia survey to promote engagement in diabetes self-management and to provide patients' input on an often underreported and undertreated side effect of diabetes care. More than 900 hypoglycemia assessment surveys were completed by patients with type 1 or type 2 diabetes, but few clinicians viewed the responses. The low use is likely due to difficulties in accessing survey results, which required extra steps within the EHR or platform to view. In addition, few patients completing the survey were flagged as “high risk” (based on their responses) for hypoglycemia, so clinicians may have felt little urgency to view the results.
Despite limited positive findings, this study can provide powerful lessons to researcher and quality improvement practitioners who may seek to use dashboard/platform technology to enhance care management. In particular, the platform required additional access steps for clinicians, which directed users to sign in to a separate interface. This lack of integration with the EHR was likely a major barrier to use, and despite multiple efforts to promote engagement (eg, email reminders, meetings), log-in rates declined over the intervention period. In the future, it may be beneficial to include some form of clinician incentive to encourage platform access/use as well as improved measure performance.
In addition, targeting platform use toward support staff, who may be tasked with obtaining information to inform pre-visit planning, may be a more effective approach than adding more systems into clinicians' workflow. Future research could explore optimal ways to target performance platforms.
The platform serves as an example of integrating multiple approaches—education, patient engagement, and a performance dashboard—into tools to drive improvements in care delivery. This research did not demonstrate a significant change in knowledge around diabetes and osteoporosis management, and one possibility for this may be that some chronic conditions may be better suited for technology-enhanced approaches or are particularly primed for improvement than those conditions observed in this study. 17,38,39 Diseases that have high social, clinical, and economic burden might benefit from technology-based interventions as a way of improving care pathways and reducing disease outcomes and costs.
The limitations to the study process likely constrained the ability to show an overall improvement in clinician KAB. The small sample size and brief intervention period of 6 months limited the amount of time to observe changes in clinical outcomes. A longer study period, greater patient survey completion, additional reinforcement of knowledge, and easier platform access may have improved findings. The study team also analyzed data in aggregate only and did not assess whether outcomes differed by clinician or patient. Similarly, data were not collected on insurance type so potential payor effects are unknown. The study also lacked a control group; thus, attribution of outcomes to the intervention versus temporal trends or other causes is limited.
Conclusions
Although not statistically significant, study results suggest that performance platforms may have a role to play in managing chronic conditions; however, their efficacy must continue to be evaluated. Efforts to increase engagement with the platform and ensure physicians utilize patient inputs (such as the patient-reported surveys) could increase and improve the impact of these platforms on chronic disease management. Finally, this study highlights a continued need for osteoporosis and hypoglycemia education in non-specialist care and for additional exploration of methods to improve engagement with technology-enhanced approaches to improve population health management.
While this research suggested that there may be some utility in the use of dashboards, the challenges identified serve as helpful examples to others who seek to employ platforms and dashboards in their efforts to improve patient care delivery.
Footnotes
Authorship Confirmation Statement
All authors confirm that they made substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND participated in drafting the work or revising it critically for important intellectual content; AND give final approval of the version to be published; AND agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Acknowledgments
We thank the participating heath care providers and patients for their time and feedback on the use of the platform.
Author Disclosure Statement
The authors have no institutional or commercial affiliations that might pose a conflict of interest regarding the publication of this manuscript.
Funding Information
This project and manuscript was funded by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA (MSD).The project agreement ensured the authors' independence in designing the study, interpreting the data, and writing and publishing the report, with additional input on design and content including unbranded resources for the Chronic Care Application from MSD.
Supplementary Material
Supplementary Material
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
