Health Care Worker Perspectives Inform Optimization of Patient Panel-Support Tools: A Qualitative Study

Abstract

Electronic decision-support systems appear to enhance care, but improving both tools and work practices may optimize outcomes. Using qualitative methods, the authors' aim was to evaluate perspectives about using the Patient Panel-Support Tool (PST) to better understand health care workers' attitudes toward, and adoption and use of, a decision-support tool. In-depth interviews were conducted to elicit participant perspectives about the PST—an electronic tool implemented in 2006 at Kaiser Permanente Northwest. The PST identifies “care gaps” and recommendations in screening, medication use, risk-factor control, and immunizations for primary care panel patients. Primary care physician (PCP) teams were already grouped (based on performance pre- and post-PST introduction) into lower, improving, and higher percent-of-care-needs met. Participants were PCPs (n=21), medical assistants (n=11), and quality and other health care managers (n=20); total n=52. Results revealed that the most commonly cited benefit of the PST was increased in-depth knowledge of patient panels, and empowerment of staff to do quality improvement. Barriers to PST use included insufficient time, competing demands, suboptimal staffing, tool navigation, documentation, and data issues. Facilitators were strong team staff roles, leadership/training for tool implementation, and dedicated time for tool use. Higher performing PCPs and their assistants more often described a detailed team approach to using the PST. In conclusion, PCP teams and managers provided important perspectives that could help optimize use of panel-support tools to improve future outcomes. Improvements are needed in tool function and navigation; training; staff accountability and role clarification; and panel management time. (Population Health Management 2013;16:107–119)

Introduction

P anel-support tools (PSTs) are decision-support instruments linked to electronic medical record (EMR) data, and can be used to manage care of individual patients and of entire patient panels. For example, a primary care physician (PCP) can query which patients are due for screening tests, immunizations, and flag multiple other patient management issues. PSTs have been shown to be effective in improving care,^1,2 but many such tools address single or small groups of health care needs³ and do not necessarily promote “whole person” care across multiple clinical areas. Enhanced PSTs, or “quality dashboards,”^4,5 address the need for a tool that can be accessed by a variety of clinical staff and that can query a more comprehensive set of databases.

Enhanced PSTs allow clinical staff to look at all the patients on their panel simultaneously, as well as at individual patients and gaps in care. Before enhanced PSTs, clinical staff did not necessarily know, and could not easily find out, which patients were on their panel. They could not categorize their patients by chronic disease, care needs met, or other metrics.

In 2006 the authors evaluated the effect of implementing a quality dashboard-like tool, the PST,^6,7 on PCPs' delivery of care recommendations for patients with diabetes mellitus (DM) and/or cardiovascular disease (CVD). The PST uses EMR data to graphically display “care gaps” for each patient, based on current evidence, and provides summary information to assist practitioners and care teams to evaluate gaps and order services. Comparing pre-PST (2005) to post-PST performance, the mean percent of care recommendations met in PCP practices (the “care score”) increased for both DM (from 63.5 to 70.6) and CVD (from 67.9 to 72.6). These results, while significant, show a fairly small effect.

However, despite the fact that PST-type tools are widely used, the authors found very few previous studies that examined how work practices and barriers affect use of PSTs. One Finnish study identified “obscured responsibilities” and the possibility of increased workloads as barriers,⁸ while another study in the Netherlands concluded that decision-support systems were not effective when users were asked to perform tasks they considered to be outside their scope of work.⁹ Compared to articles detailing studies on barriers and facilitators to PST use, however, there are more publications describing the lack of such studies, documenting the thus-far minor success of PSTs at actually improving outcomes,^10,11 and calling for qualitative research to investigate the root causes of these issues. The United States federal government, as well as the United Kingdom's National Health System, and informatics researchers from Europe and Australia, have called for prioritizing the integration of EMRs into health care delivery, specifically asking for research to address the chasm between technological advancements such as PSTs and the delivery of improved outcomes.^12
–14

This article describes^{5,15

–21} qualitative data and analyses about how care teams use the PST, and assesses teams' approaches to using the PST relative to their success in delivering needed care to their panels. The findings should help optimize the utility of EMRs and other electronically linked databases by lowering barriers and capitalizing on facilitators to the use of an important feature of integrated medical records: enhanced decision-support tools.

Methods

Study site and systems

The study was approved by the institutional review board within the study health maintenance organization (HMO), and clinician and staff participants provided written informed consent.

Study site

The study was conducted in 2008–2009 at a nonprofit group-model HMO, Kaiser Permanente Northwest, in Washington and Oregon. The ∼450,000 members are demographically similar to the area population.²² Electronic databases provide administrative and clinical data and a full EMR (Kaiser Permanente HealthConnect, Epic System Inc., St. Louis, MO) has been operational since 1996. The EMR is an EPIC systems product. When a patient record is opened, a button takes a clinician to the patient's PST data with a single click. The PST was developed by Kaiser Permanente and is proprietary.

The panel-support tool

The PST has been described in detail previously.^6,7,23 It graphically displays “care gaps” (eg, for screening, medication use, monitoring, risk-factor control, immunizations) for each patient in a PCP's panel, based on current evidence of recommended care ^24
–26 (Fig. 1). The PST gathers administrative data every 24 hours so that changes in patient status such as completed tests (based upon laboratory data systems) or medication initiated (based on dispensed medication systems) are largely up to date. There is the potential for these data to be inaccurate for a very small percentage (<5%) of patients who receive medications or services elsewhere and do not inform the health plan of this. PCPs and teams were trained to use the PST in 2006, before implementation, and then again a year later. The HMO expected PCPs and their staff (eg, medical assistants [MAs]) to use the PST, and use by PCP teams gradually increased to 50% by August 2006, 65% by December 2006, and more than 95% by February 2007.

FIG. 1.

Panel-support tool displays. CVD, cardiovascular disease; DM, diabetes mellitus; PCP, primary care physician.

Recruitment and study participants

Participants were recruited using up to 3 e-mail requests, with telephone follow-up for those expressing interest. Family practice and internal medicine PCPs were recruited for interviews using a stratified sampling method based upon percent of care needs met for their patient panel members with DM. A total of 159 PCPs were identified who had continuously had patient panels from the pre-PST period (2005) through the post-PST period (2007), who had at least 20 eligible DM patients during each study month, and who were still available at the study site for recruitment in 2008. The mean percent of care recommendations met by PCPs were determined, details of which have been described previously.⁷ Higher and lower performance in 2005 and 2007 were then defined separately as performance above and below the mean values in those years (≥64% vs. <64% care recommendations met in 2005, and ≥71% vs. <71% in 2007). PCPs were recruited from 3 lists of providers who met the following criteria: lower performing in 2005 and 2007 (“consistently lowest”), lower performing in 2005 but higher performing in 2007 (“improving”), and higher performing in both 2005 and 2007 (“consistently highest”); the aim was to interview at least 5 members of each group, and about half this number of MAs across performance groups. Twenty-one semi-structured, in-depth individual interviews were completed with PCPs (24% participation rate) during 2008–2009: 6 in the consistently lowest group, 5 improving, 10 consistently highest. Eleven MAs (42% participation rate) also were interviewed during this time frame. Scheduling conflicts and lack of time were the primary reasons given for not participating.

Health plan leaders with quality improvement responsibility in a range of geographic areas also were interviewed. Eight group interviews were completed with 20 individuals (5 quality leaders and 15 managers; 100% participation rate). A total of 52 staff (17 physicians, 4 allied health providers [nurse practitioners and physician assistants], 11 MAs, and 20 other managerial staff) participated.

Data collection and analysis

PCP performance-group differences were assessed using the Kruskal-Wallis test. The research team developed guides (based on prior experience^24

–27 and a literature review) that were refined following the first few interviews. The interview guides elicited information about barriers and facilitators to PST use, and overall advice on how to improve it and surrounding work practices. Interviews were conducted by 2 of the authors (AF and JS) who are trained in qualitative research,^28,29 and were tape-recorded and professionally transcribed for analysis. The interviewers and a qualitative analyst (JS) were blinded to participant performance.

Each stakeholder group was interviewed until no new content was elicited.³⁰ Analyses focused on representing, describing, and interpreting data, using standard techniques ^28,29,31 and a qualitative research software package, Atlas.ti 5.0 (Atlas.ti Scientific Software Development GmbH, Belin, Germany), to code data and generate reports of coded text for analysis. A coding dictionary was developed based on the interview guide and review of the transcribed interviews. Transcribed interviews were coded by marking passages of text with phrases indicating the content of the discussions. Using the report and query functions of Atlas.ti, coded text was further reviewed through an iterative process, resulting in refined themes. ^31,32

Results

Nearly 43% (42.9%) of the PCPs were internal medicine practitioners (the remainder were family practice); 19% were allied health practitioners. The final PCP participants represented all 3 performance groups in that each had significantly different mean percent of care needs met across both 2005 and 2007 (the study years). In 2005, the consistently lowest PCP performance group met 59±4% of care needs, the improving group met 63±1%, and the consistently highest group met 69±3% (P<0.001). In 2007, those same measures were 64±3%, 73±3%, and 77±3%, respectively (P=0.001).

Reported benefits of the PST

The most common, compelling benefit of the tool (Table 1), mentioned by all clinicians, was the ability to clearly identify patients on their panels. Prior to the PST, understanding patient needs—and which care needs were not being met—was dependent upon scheduled patient visits and caregivers opening individual records. Interviewees stated that, prior to the PST, they often overlooked individual patient care needs unrelated to the reason for the visit, particularly needs that were difficult to identify without the PST. Previously, identifying certain care gaps required manually looking in multiple areas of the EMR.

Table 1.

Participant Perspectives on Global Benefits of the Panel-support Tool (n=52)

Common Themes and Key Findings	Illustrative Quotes
In·depth Knowledge of Patient Panel • Knowledge of who is on providers' patient panel • Knowledge of what patients' care needs may be in a way not easily known before • Awareness of patient panel needs no longer dependent on a patient visit or disparate sources	“Other than the patient you are looking at, you don't know anything else about your other patients or even how to find them. So it's a crucial piece to moving that idea that if you are going to manage your panel; there is a small portion you see every day, but there is that huge portion that is just out there. Who has needs? Who doesn't? We haven't had any way of doing that [before]-we just relied on people contacting us when they were in trouble. And so it [PST] has really helped in that whole process of knowing who your panel is.” (PCP) “I have found it very helpful…before I would have had to do a complete chart review on a lot of different things to try to find information that is now on the PST…before [the PST] identified it, I didn't even know half the stuff that I could do [for patients].” (MA)
Provision of Information • Identifies care needs of individual patients and populations of interest • Makes relevant data in patients' charts convenient and accessible in a 1-page synopsis • Reminds providers and support staff regarding important gaps in care that may be unknown or overlooked • Brings primary and secondary preventive care needs to the forefront	“It's something that should have been here a long time ago, I think. It's really good. It's practical. It has everything easily readable and only what I want, not any of the other data, so I like it…It's just a one page document with exactly what I need and nothing else.” (PCP) “Basically for the first time it [PST] alerted me to what I wasn't paying attention to…it kept important things on the radar that would have slipped through the cracks. And, it gave me a perspective of how I was doing relative to everybody else, which we don't get a sense for otherwise.” (PCP) “The benefit is that it lets me know what I can do as far as filling their gaps where patients' are lacking—before you had to do a chart review and here it is all laid out for you in a straightforward way.” (MA)
Quality/Organizational Goals • Provides means to more easily meet and improve upon organizational performance goals • Helps providers meet their own personal care goals for their panel • Assists the organization as a whole to be more cost-effective by efficiently prompting and addressing needed patient services during office visits	“If you're looking at how you're doing on some of the diabetes measures and you want to know how to improve on measure x or y, or whatever, it's the fact that it is so easy to then go and find out who are the people that are not meeting the measure that you can intervene on. So that's very helpful.” (PCP) “When I first started we had a couple of hundred patients that needed Pap smears. So we just brought them in and after six months it was done and now we try to stay on top of it. And so those patients wouldn't have been called if we didn't have the Panel Support Tool.” (PCP) “Having staff look at and use this tool for clinical quality and also for utilization and cost opportunities…to have staff think of it as a comprehensive tool that will improve our resource stewardship and help get people to the right care [they need]…that's been huge.” (Manager)
Patient Care • Allows providers to be proactive rather than reactive • Assists providers and support staff with monitoring and prompting regular screening and prevention services • Facilitates better chronic care management and prevention • Reduces fragmentation of care and improves coordination and integration of care • Facilitates patients experiencing comprehensive care • Assists providers and support staff in team efforts toward increasing patient compliance and reinforcing patient behaviors related to needed services	“Care feels more broad scoped. It's like, well, we know you're here for your finger, but it looks like you're due for your diabetic foot check. So patients are like, ‘Oh my gosh, wow, they're really checking me out and taking care of me.’ So I think that the PST really does help to give a picture of we're caring for the whole person and not just you're here for your finger.” (PCP) “My blood pressure patients would be under worse control without it. The diabetics, I think benefit a lot too, because I can see exactly which ones have the LDLs under control and which ones don't. Who is on the statins and who is not?” (PCP) “In our practice patients have had a good reaction to the PST. They are amazed that we know this or that when we contact them by phone or remind during a visit. They feel taken care of…and patients' needs don't fall through the cracks any longer.” (MA)
Efficiency • Quickly brings forth all needed information from patient's chart “at a glance,” reducing the time needed for chart review • Maximizes what can be accomplished in a visit • Allows for quick and easy search and retrieval of information for an individual patient or subgroup of patients • Makes process of identifying and reaching out to a subgroup of patients more systematic and achievable • Improves workflow by increasing the role of medical assistants and other support staff in patient care	“I think it's made me more efficient, as far as time is concerned. Because I don't have to go around fishing for when did they have their last fasting blood work? You know, what are they due for?…Sometimes I used to sit down before patients came in and on paper sit and write down when they had work done the last time…That wasn't very efficient. So I think in that way it's helped.” (PCP) “I took one look at that and I thought, this is why we are putting that data into the computer. This is the first truly functional aid…This is why computerized medical records exist.” (PCP) “It is so much easier to have the PST right there and just be able to list off all of a patient's needs. We don't have to flip through all these other papers to try and figure it out. So it is just very convenient and handy to have everything right there at your fingertips.” (MA)
Staff Empowerment/Job Satisfaction • Allows providers and support staff to be curious about & investigate the patients on their panels • Allows for support staff to experience increased job satisfaction and gratification due to expansion of skill set and roles in assisting provider and patients in meeting care needs • Belief by providers & support staff that, if the tool was discontinued, it would make their jobs more challenging and decrease what they could accomplish in patient care	“I think it's been really useful, because when we first got the EMR twelve or fifteen years ago, I remember asking does this mean I'm going to be able to look up all my patients' with diabetes or cardiovascular problems. And the answer was yes. But, of course, in fact it was no…So with the Panel Support Tool now, I can go and see all my patients with diabetes, and then if I click on it I can see, all the care gaps there in the PST, whether it's prevention, diabetes, or anything else. So it's very patient-centric now and really helpful.” (PCP) “I think it's a great tool. I'm certainly glad we have it. I'd hate to lose it. It's made a huge difference. And I really, really think having it available for every patient is the real plus.” (PCP) “I enjoy the PST because you can really use your ability to think ahead and determine what else can I do to make [PCPs] job easier and help patients. I really enjoy being able to do more things…As you do it more, it gets easier, so you are ready to take on more.” (MA)

LDL, low-density lipoprotein; MA, medical assistant; PCP, primary care physician; PST, panel-support tool.

Many participants also noted that the tool empowered them at both PCP and team levels to help meet local or organizational performance goals. In addition, many providers said the tool helped them set priorities for quality improvement (ie, determining which patients were in need of certain services and acting upon those needs).

Staff members reported that they had gained additional appropriate responsibilities by actively using the PST. Both MAs and PCPs felt that workflow was improved, and that everyone was able to better communicate and reinforce patients' behaviors relating to needed services. MAs enjoyed the expanded number and diversity of tasks the PST brought to their jobs. They talked about experiencing positive reactions from most patients, which reinforced their use of the PST on a regular basis.

Overall, respondents felt that the PST helped them provide more comprehensive and more proactive patient care. Many noted that the tool increased efficiency by providing synthesized information at a glance, allowing providers to be maximally productive during a patient visit. Preventive and chronic-care management coordination was improved because the tool refreshed frequently, providing staff with near real-time status of care needs.

Reported barriers to PST use

The most commonly described—and most significant—barrier to optimal PST use was lack of time and competing demands (Table 2). Often providers were able to utilize the PST during patient visits, but had little time for related outreach (by phone, mail, or e-mail). Also, if a practice performed outreach for many care needs that required patient visits, the PST potentially led to scheduling difficulties.

Table 2.

Participant Perspectives on the Global Barriers to Use of the Panel-support Tool (N=52)

Common Theme and Key Findings	Illustrative Quotes
Lack of Time • No time in providers' day to practice using the tool or systematically integrate it into practice • Often no extra time to conduct and execute patient outreach activities • Sometimes no extra time during patient visits to execute all the prompted PST work	“It's been months since I really had the luxury of sitting down and looking at my Panel Support Tool and looking for ways I could improve the panel…But, at the heart of it all, is that we just don't have time. As useful and as important and as wonderful as the tool is, I don't have time to use it.”(PCP) “I just do it [outreach) in my spare time—squeeze it in at the end of the day seeing patients, or maybe a little bit of time before lunch. It's hard to fit in though, when schedules are tight or when we're short staffed, it's really difficult to fit in.” (MA)
Competing Demands • Responding to patient-driven acute needs reduces the opportunity to respond to PST-related care needs • Responding to clinic or organizational issues such as: changes in staffing or hours; participation in required trainings; or experiencing an escalating and overall sense of burden in primary care	“There's just a point, when you've gone through a problem list that has 7 things at the end of the visit, that you just simply sometimes have no energy to bring up that colorectal screening is due. When you've already spent thirty-five minutes or so on a patient and you're running behind, there is just absolutely no way that you say, by the way, you're due for this and that.” (PCP) “There have been times where I couldn't do my outreach, because we were out a month and there was no way I was going to call people and be like, come see us for this, you know.…At different times, we've gone through some crunches, shortages, people being out on leave and stuff. And so, a lot of those times, I end up being pulled to work with somebody else.” (MA)
Staffing Issues • Lack of consistent and/or competent staffing • Lack of training and skill mentoring for staff • Not enough staff to conduct and complete the work generated by the tool • Presence of long-time staff who are more resistant to incorporating the tool into their established workflows and approach to patient care • Lack of rewards/selection for support staff for productivity, self-sufficiency, and motivation	“I guess you can have someone who is really interested in it and uses it and they tend to do it well. And then some days I work with some medical assistants who I think could use a little bit more training. So I think it's dependent on the individual…most MAs will do it, but some of them still have some troubles with it.” (PCP) “I think where we still struggle for both doctors and support staff is around the accountability to use it. Some people take it and run with it—the MA and doctor are working together, and some just don't really see it as their job.…Part of what physicians have to do is understand they need to make it known to their MA that this is an expectation…Some doctors have gotten that and some say they can't or don't know how to make their MA do anything.” (Manager)
Workload For Patient Visits • Use during patient visits can increase daily workload and time spent with each patient during office visits • Visits more complex now as PCP attempts to address patient stated issue and PST identified needs • Reviewing and executing PST data may take time and attention away from patients' original need for a visit • Items in the PST can compete with each other and create competing access issues	“The PST is important stuff, but if you're dealing with the problem they came in with and then also having to discuss what they're overdue for, all of that takes time. I don't know if it was intended or unintended, but one of the consequences is it often makes the visit more complex.” (PCP) “I would say that every single thing that we put in the PST completely competes with each other. So a prime example of this is cervical cancer screening and Medicare. You might think, wait a second, those are two completely different things, right? But what do you have to do for both of them? You have to have a face-to-face visit. So, okay, what are you going to compete with? Are you going to have Medicare come in or are you going have cervical cancer screening come in?” (PCP)
Workload For Outreach Efforts • Lack of clear and reinforced communication from organizational leaders regarding the work expectations and accountabilities of the tool for outreach purposes • Unclear expectations at the PCP and support staff level as to who completes what task and how to best coordinate work • Use for outreach purposes can create clinician access issues and increase ongoing workload • Patients may be noncompliant and unresponsive regardless of the outreach effort put forth	“I think it is really important to fix our organizational structure regarding who is accountable for the education and updating of support staff, because it is not clear.…In order to have them do outreach, they need to understand [the PST]—the why and how…so we need to make sure there is an education system for every medical office and they are hit throughout the year with multiple modes of communication.” (Manager) “When you bring patients in, do you do it on the phone? Do I outreach by mail?…I can't bring them all in, because then there's the access problem again. So, how do I balance that?” (PCP) “There are those few patients, where the gaps are greater, and they're the ones that have been difficult to pin down. You can look and find out that they've been sent many letters. They've been contacted by phone. They've had things ordered for them and not done it. Those [patients] are challenging.” (MA)

MA, medical assistant; PCP, primary care physician; PST, panel-support tool.

Staff expressed that the PST seemed to increase the workload during a patient visit. Visits became more complex because PCPs were attempting to address acute care needs in addition to multiple chronic illness needs identified by the PST. In this sense, care gaps identified by PST prompts became new competing needs. If the patient had many concerns at the clinical visit, there was insufficient time to address the care needs prompted by the tool. However, PCPs stated that much of the work generated by PST prompts could be executed by nonphysician staff such as pharmacists, registered nurses, allied clinicians, and MAs.

PCPs noted that lack of consistent and well-trained non-PCP staff (eg, MAs) hindered optimal assistance with PST care needs. They also felt that non-PCP staff were not selected or rewarded for their productivity, self-sufficiency, and motivation to use this complex informatics tool. Lack of consistent and well-trained help exacerbated PCPs' sense of being expected to address an increasing number of care needs with continually less time and fewer resources. Interestingly, MAs were less fearful of the PST workload than PCPs. They clearly stated that their initial fear of not being able to get the PST-generated work done was ultimately unjustified. Many asked themselves how they could ease the PCPs' PST-generated workload, and felt that it was important to do all they could to assist.

Despite the barriers to using the tool, PCPs and non-PCP staff stated that they did not wish to return to pre-PST days. The overall sentiment was that the tool so improved what could be understood and accomplished in patient care that, although workload increased, the tool's benefits outweighed the negatives.

Factors facilitating panel-support tool integration into clinical practice

Respondents identified additional features that helped increase integration of the PST into practice (Table 3). PCPs noted that the most important facilitator of tool integration was having a consistently assigned and dedicated MA trained and skilled in monitoring and executing PST work. MAs who were detail oriented and had good time management and organizational skills were particularly effective. The strength of the PCP–MA partnership was viewed as important and necessary to execute work tasks generated by the PST. Regular communication was important for ongoing joint monitoring and development of goals for PST use.

Table 3.

Participant-identified Facilitators of Panel-support Tool Integration into Clinical Practice (N=52)

Common Theme	Key Findings
PCP and Support Staff Roles	• Support staff who are assigned and dedicated to work consistently with a designated PCP • Support staff trained and skilled to monitor and execute the work generated by the PST • Support staff who are proactive and work independently to assist PCP in meeting patient care gaps using the PST through in-reach and outreach activities • Support staff who are detailed oriented with good time management and organizational skills • Already established workflow patterns and partnership between support staff and PCP • Regular communication and joint monitoring of PST-related goals and needs between support staff and PCP
Organizational Leadership	• Organizational leaders/managers setting clear and concrete expectations and accountabilities regarding the use and work of the PST during a patient visit • Organizational leaders/managers communicating these expectations and accountabilities via multiple and repeated avenues—e-mail, staff meetings, face-to-face check-ins with clinic leaders, and trainings • Organizational leaders/managers clearly communicating high-priority care gap measures to give guidance and focus to both inpatient work and outreach efforts
Clinic/Local Support	• Coordinated clinic leadership by a dedicated staff to provide structure, guidance, and suggestions on PST outreach efforts • PCP and support staff leaders within clinics that serve as champions and expert resources for PST use and workflow
Time	• Having dedicated time for training and developing proficiency with the tool • Having dedicated time once a month to conduct outreach efforts on patient care needs identified by the tool
Implementation Approach	• Training demonstrated the utility and capacity power of the tool to assist providers in their work • Training increased awareness of tool and improved proficiency of use • Overall flexibility provided to PCPs to blend their use of the PST with their own practice style, preferences, and goals
PCP and Support Staff Belief and Approach	• Having a preventive medicine orientation and approach to patient care • Believing that the work generated by PST (screenings, reducing care gaps) is important and necessary work • Having an overall positive regard for the tool itself

PCP, primary care physician; PST, panel-support tool.

Interviewees noted the importance of organizational leadership and of monitoring use of the PST to improve team performance. Clear communication and specific expectations, followed by managers making staff accountable for the use, were helpful. Managers confirmed that this was done consistently for use of the PST during visits, but not for outreach activities.

PCPs and their staff reported that they found it helpful for organizational leaders to develop and communicate priority clinical areas that should be addressed through the PST, particularly for patients with multiple care needs. Local clinic leadership sometimes provided dedicated staff (eg, nurse practitioner) to assist with prioritizing PST needs by monitoring clinic performance, making outreach calls, or sending letters/e-mails to patients. These individuals would work on behalf of all the clinicians in the clinic, and serve as champions and expert resources for utilizing the PST effectively. Another useful model implemented at some clinics was providing PCPs and their staff with dedicated PST time (eg, 4 hours each month) to conduct outreach and develop proficiency with the tool.

Both formal and informal training was helpful for improving efficiency. Although all staff appreciated receiving information about particularly successful patterns of PST use, they also appreciated being able to individualize how they used the PST.

Thematic differences across PCP performance group

Differences in themes were compared across performance groups (a full version of Table 4 that includes participant quotes is available from the authors upon request). The consistently highest performing group frequently endorsed several sentiments that other groups did not. This group often expressed frustration at insufficient planning time to conceptualize the best outreach approach for PST use. They also stressed the importance of clearly communicating and following up on non-PCP staff roles in accomplishing PST work, as well as recognizing non-PCP staff efforts and results. Members of this group frequently stated that investing time and training in fostering the PST use goals of non-PCP staff was important. The consistently highest performing PCPs also routinely described engaging non-PCP staff as partners in meeting patient needs, as well as allowing staff to take the lead in determining outreach approaches. PCPs and non-PCP staff in this group often described sharing a competitive drive to excel at their jobs and meet as many patient care needs as possible. The non-PCP staff of the consistently highest performing clinicians often endorsed the value of being familiar with the tool, and of learning how to check PCP performance and guide outreach activities. These non-PCP staff often described a flexible and personalized approach to outreach, based upon their knowledge of specific patients, and were overall more likely to use language that described ownership and commitment to the patient panels they served.

Table 4.

Team Performance: Group Differences Across Key Categories of Themes

Theme and Related Attributes	Consistently Lowest Performance (N=10) ^*	Improving Performance (N=6) ^*	Consistently Highest Performance (N=16) ^*	Endorsing (PCP/SS)
Global Barriers to Panel-support Tool Use
Lack of Time
• To conceptualize best outreach topic			./	PCP/SS
• To conceptualize best outreach strategy			./	PCP/SS
Panel Composition
• Large patient panel size	./			PCP/SS
• Open panel (new members)	./			PCP/SS
Global Facilitators to Panel-support Tool Use
Fostering Team Partnership
• Communicate tool work expectations to SS			./	PCP
• Recognition of SS work effort in the tool			./	PCP
• Investment of time/training with the tool			./	PCP
• Regular engagement of SS as a partner/collaborator in meeting patient care needs			./	PCP
• Shared competitive drive to excel/be the best			./	PCP/SS
Performance Data
• Tool allows for easy access to/tracking of performance data		./	./	PCP/SS
• Tracking/improving performance numbers facilitates better patient care		./	./	PCP/SS
• Display of performance data creates accountability and motivation to use the tool		./	./	PCP/SS
• Reduction of patient care gaps is a “fun” challenge		./	./	PCP/SS
Technical and Human Factors Related to Panel-support Tool Use
Knowledge Fatigue
• Tool generates a feeling of having to do it all	./			PCP
• Tool generates an endless identification of needs	./			PCP
Tool Functionality
• Certain navigation/documentation functions lack efficiency		./	./	PCP/SS
• Tool data loads/refreshes too slowly for workflow preference		./	./	PCP/SS
Training
• Introduction to/training on tool was too minimal	./			PCP
• Training sufficient/tool self-explanatory		./	./	PCP
Teamwork Practices Related to Panel-support Tool Use During a Patient Visit
• Logs into/opens tool everyday		./	./	PCP/SS
• Familiarity with the many functions of the tool		./	./	PCP/SS
• Discusses care gap needs with patients prior to PCP entering the room			./	SS
• Use of related smart set tools for ordering tool-identified needs			./	SS
• Double-checks tool data for accuracy against other sources			./	SS
• Pends orders on majority of tool-identified prevention/chronic needs		./	./	SS
Teamwork Practices Related to Panel-support Tool Use for Outreach Activities Not During a Patient Visit
• Empowers SS as leader on outreach strategy, focus, and timeline			./	PCP
• Outreach strategy and focus determined by PCP and given to SS	./	./		PCP
• Outreach conducted at every possible opportunity during the day/week			./	PCP/SS
• Outreach conducted about once a month or more frequently if time allows	./	./		PCP/SS
• Engages in a detailed/organized approach to outreach activities		./	./	SS
• Engages in a follow-up process following initial outreach efforts		./	./	SS
• Knows how to check PCP performance data in the tool to guide outreach activities			./	SS
• Uses a flexible/personalized outreach approach (phone calls, letters, e-mail)			./	SS

Consistently Lowest Performance (PCP=6; SS=4); Improving Performance (PCP=5; SS=1); Consistently Highest Performance (PCP=10; SS=6) PCP=primary care physician (internal medicine; family practice; allied health); SS=support staff (medical assistants).

./=Defined as more than half of the participants in the performance group often bringing up and endorsing the theme during the interview process.

No./=Defined as less than half of the participants in the performance group not bringing up or regularly endorsing the theme during the interview process.

Several themes were expressed frequently among the groups with improving performance and the consistently highest performance. Both groups endorsed that the PST allowed for easy access to and tracking of performance, and felt that this and related follow-up improved patient care. They felt that the display of performance data in the PST motivated them and made them feel more accountable. These group members often noted that the tool was “fun,” as opposed to being burdensome, and said they used it every day. These same groups were most interested in future improvements in the PST, such as improved navigation and documentation functions and more up-to-date data. The non-PCP staff supporting these 2 groups often endorsed pending orders to make it more efficient for clinicians to follow-up on PST needs.

The consistently lowest-performing PCP group did not frequently endorse the above themes. This group often noted that they were hindered in their use of the PST by their larger patient panel size or by having a panel open to new patients. They often expressed a sense of knowledge fatigue, describing how the tool generated a feeling of being overwhelmed by too many patient care needs. They also felt the training and introduction to the tool provided by the organization was inadequate.

In contrast to the consistently highest-performing group, who frequently delegated PST leadership and responsibility to their non-PCP staff, the consistently lowest performing and the improving groups often endorsed that they, the PCPs, determined the outreach strategy and focus and instructed their staff to carry it out. Both of these groups often stated that they tried to conduct outreach about once a month, and more frequently only if time allowed, rather than weekly as the highest-performing group did.

Technical facilitators and barriers to PST use

PCPs and their non-PCP staff provided insight in the areas of PST design and function, data, navigation, and documentation (Table 5). The visual layout of the tool was pleasing and, in general, interviewees felt that it facilitated efficient and easy access to information and decision support. The sorting and searching functions also were described by most as efficient and easy to use. Clinicians appreciated that the tool and information technology support were consistently available and that they could give feedback on data integrity (a mechanism provided within the tool itself). Respondents generally felt that the data were reliable and accurate most of the time. Overall, participants mostly said that the PST was self-explanatory and intuitive to use.

Table 5.

PCP and Support Staff Perspectives on Technical Facilitators and Barriers to Use of the Panel-support Tool (N=32)

Common Theme
	Technical Facilitators to Use
Design and Functions	• Visual layout of tool facilitates efficient and easy access and interpretation of information• Search, sorting, and retrieval functions easy to use, allowing for efficient means to research panel needs• Tool not cluttered with too much or unnecessary information• Tool mostly self-explanatory and intuitive to use
Data	• Data are reliable and accurate most of the time• Data are timely and almost instantaneously available• Sources of data are known and considered trustworthy• Opportunities are provided to offer feedback on data integrity
Technical Support/Improvements	• Tool is consistently available• Information Technology support is consistently and efficiently available• Tool continues to evolve over time in useful ways, such as quicker access and improved interface with electronic medical record
	Technical Barriers to Use
Navigation and Documentation	• Experience too many log-in points and time-consuming steps when maneuvering from patient electronic chart to the tool• Lack of easy and efficient way to document and track patient refusal of services or medications and team effort made to meet a care need• Coordinating patient outreach is compromised by a lack of documentation in the PST regarding refusals or outreach by other centralized organizational programs (such as automated reminder calls)
Data	• Data are sometimes missing or not integrated from other external sources (claims or hospital data)• Sudden or unannounced changes in formatting of PST data and reports disrupts established routines and PST workflows• PST continues to remind for medicines or services that may have been refused or adversely tolerated by patients• High gap score information is frustrating and less helpful as it is not reflective of PCP efforts or obstacles outside of PCP control (people with dual coverage; change in gap measurement guidelines)

PCP, primary care physician; PST, panel-support tool.

Interviewees said several design elements could be improved. They described too many log-in points, and said it was time consuming to maneuver from the patient's electronic chart to the PST. In particular, clinicians wanted to be able to input orders directly from the tool as opposed to toggling to the medical record. Most PCPs described the “gap scores” or notations of patients with high scores as not very useful. Getting to a score based on care needs seemed somewhat arbitrary and not intuitive, and did not change the follow-up approach. PCPs said that they were much more likely to pay attention to a care need versus an applied numerical indicator of a weighted gap. Additionally, those patients with high gap scores often had missing data.

Advice for improving the PST

Participants advised continuing to improve the accuracy and timeliness of the PST data (Table 6). In particular, they suggested better tracking and documentation of outside care and requested that all data elements be updated (preferably daily) with the most current information available. They recommended adding new clinical areas to the tool, such as adult vaccinations, and prompts for colorectal cancer screening, chlamydia, and prostate cancer screening. PCPs and MAs were particularly anxious to access better methods for documenting and tracking service refusals and outreach efforts. Interviewees also wanted the ability to place care orders directly from the PST.

Table 6.

Participant Advice for Improving the Panel-support Tool Experience (N=52)

Common Theme	Key Findings
Function and Navigation	• Continue to improve the accuracy and timeliness of the data• Potentially add new clinical areas to tool such as adult vaccines, CRC screening test, chlamydia, and prostate screening• Create an easy and efficient method to see, document, and track refused services• Create an easy and efficient way to see and track outreach efforts that have been conducted by either central programs (ie, automated phone calls) or by PCP/support staff• Create the ability to order labs and services directly from the tool• Reduce extra steps needed to maneuver between the tool and electronic chart—improve integration by making it a 1-step process in both directions• Identify and highlight in the tool key areas the organization is focusing on each year to increase awareness and prioritize work effort
Training Opportunity and Needs	• Organizational leaders provide clear communication, support, and training to PCPs to direct, teach, and delegate work expectations and competencies of the tool to their support staff• Provide yearly in-person and/or Web-based refresher training that addresses useful functions (ie, checking panel vitals) and recent changes• Provide organizational guidelines to support staff regarding intensity and frequency of repeated outreach efforts, particularly for patients who tend not to respond• Include a reiteration of work expectations and use of PST for both in-reach and outreach efforts for all staff• Include motivational information as to why it is important to use the PST
Creation of Panel Management Time and Ability for Using the Tool	• Off-load prevention and screening work to non-PCP staff and utilize more affiliate clinicians, RNs, and MAs for this type of work• Formally involve and train membership services and specialists in use of the tool during points of contact with patients• Create dedicated and protected time (such as 4 hours per month) for panel management efforts, including outreach activities• Provide dedicated staff to help PCPs with panel management and outreach efforts, such as: RN/MA teams at each clinic who help to organize and execute outreach efforts for all clinicians at that location

CRC, colorectal cancer; PCP, primary care physician; PST, panel-support tool.

Interviewees noted that it would be helpful if the PST could highlight areas of organizational priority in terms of quality improvement each year, and provide guidance to all staff regarding the intensity and frequency of repeated outreach efforts. Participants suggested yearly refresher training, especially for non-PCP staff, to include general information on the importance of population-based care facilitated by the PST and progress made with PST use. They also advised that non-PCP staff (affiliated clinicians, registered nurses, and MAs) should be the ones to usually address care needs identified by the PST, and that more dedicated time be created for panel management activities. Additionally, ongoing communication from organizational leaders regarding work expectations and accountabilities for PST use for outreach efforts was cited as a needed improvement.

Discussion

Findings from interviews with PCPs, MAs, and health care managers yielded valuable perspectives on the PST. In general, respondents felt that the PST was transformative in its ability to affect quality improvement activities at the front lines of primary care. This was largely because of the most notable benefit of the PST—in-depth knowledge of patient panels. The impact of this tool on clinical practice rose to nearly the same level of importance as the implementation of the EMR itself. Although participants felt the PST could be improved, the tool helped to overcome many barriers typically associated with teams' implementation of evidence-based practices: not knowing a gap exists, not having all the data necessary to make a decision, and not having an efficient method for ordering and follow-up.³³ Interestingly, these findings are of particular consequence with regard to using electronic health records for patient management. Our HMO has a long-standing (>10 years), well-integrated EMR, yet clinical staff expressed that the decision support provided by the enhanced PST was a marked improvement compared to standard alerts and reminders in the EMR. A secondary benefit of the PST was development of non-PCP staff skills and improved job satisfaction.

Respondents also outlined some remaining facilitators of and challenges to more effective use of PSTs. Facilitators for effective PST use included that the PST offered important roles for non-PCP staff, leadership opportunity, training for tool implementation, and dedicated time for tool use. Barriers to PST use included lack of time, competing demands, suboptimal staffing, tool navigation, documentation, and data issues related to accuracy and timeliness. This feedback regarding tool function and navigation, optimizing time, and training and staffing for panel-management needs should be integrated into future initiatives to maximize PST use and effectiveness.

Findings across performance groups provide additional insights to improve PST implementation strategies. Higher-performing PCPs and their assistants described a detailed team approach to using the PST more often. Their specific partnership models, their staff facilitation of the work (including outreach), and their teams' frequent consultation of the PST and tracking of performance data in the tool all serve as a “best practices” guide. Lower-performing teams reported less frequent tool use, less frequent delegating of PST activities to staff, generally feeling overwhelmed by information from the PST, and needing more training. These clinicians may benefit from future PST enhancements such as simplification of data displays and assistance with activity prioritization. They also may have improved experiences with more extensive training, development of specific workflow policies and accountability, and more non-PCP staff support.

The results indicate that both communication regarding health care team roles and the facilitation of population-based care provided by the PST were equally critical to successful adoption. Participants in this study supported the importance of specific aspects of informatics systems previously described as being correlated with improved outcomes: close exchange of data with an EMR, allowing for large-scale screening and aggregation of data, the presence of population management tools, provision of performance feedback, and order templates.^34,35

This study adds to the literature^8,9,36
–38 in several ways. Facilitators of and barriers to the use of the PST for preventive care and for chronic disease management were evaluated both during care in the office and during outreach. Also, prior published reports examined barriers to decision support use experienced solely by physicians and/or nurses. This study surveyed MAs in addition to nurses and physicians and sought to understand the impact/use of a decision-support tool on an entire medical team. The importance of this distinction was borne out by the results—team dynamics (eg, whether and how physicians delegated or failed to delegate PCP-related workflow to assistants) emerged as an important factor related to successful use of the PST.

This study has limitations. The findings may not be completely generalizable to other PSTs or to other settings. However, many of the requirements for successful PST implementation that the participants of this study highlighted are important in many primary care settings. Another possible confounder of these data is that higher-performing PCPs were easier to recruit; hence, there is more input from them than from lower-performing PCPs. The findings across performance groups are associations and should not imply causation. Non-PCP staff was especially challenging to recruit for this study because they were difficult to reach and had little flexible time. There also may be discrepancies between what staff say they do and what they actually do. It is hoped that, in the long term, qualitative translational information such as that presented in this article will improve patient outcomes and health care efficiency by improving how clinical staff use these important tools. Future studies could examine long-term outcomes and cost, and observe how diverse teams use the tool in various natural settings.

Conclusion

The authors believe that most of the findings highlighted in this article are generalizable, and that implementation strategies based on these findings could improve care delivery by increasing efficiency and uptake of these EMR tools and ensuring that the addition of such decision tools does not add to clinical staff workloads. These findings are especially important when viewed within the context of the increasing pressure for health care providers to address gaps in guideline-based care using EMRs.

Footnotes

Author Disclosure Statement

Drs. Feldstein, Unitan, Perrin, Smith, and Nichols, and Ms. Schneider and Ms. Lee declared no conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclosed the following funding for the research, authorship, and/or publication of this article. This project was supported by a Kaiser Permanente electronic medical record research initiative. The funding organization was not involved in the design or conduct of the research; the collection, management, analysis, or interpretation of the data; or the preparation or approval of this manuscript.

Acknowledgments

The authors would like to acknowledge Gail Morgan and Mary Rix, RN, for project management; Carrie Davino, MD and Yvonne Zhou, PhD for their input into the methods; Leslie Bienen for editorial; and Dixie Sweo for administrative support. The authors would like to thank the many other individuals who assisted with the design and conduct of the study and who were critical organizational advocates for the Panel-Support Tool: Maureen Wright, MD; Homer Chin, MD; Thomas Hickey, MD; Wiley Chan, MD; Michael Krall, MD; and Trung Vu.

References

Bodenheimer

, Grumbach

. Electronic technology: A spark to revitalize primary care? JAMA, 2003; 290:259–264.

Conway

, Clancy

. Transformation of health care at the front line. JAMA, 2009; 301:763–765.

Balas

, Weingarten

, Garb

, Blumenthal

, Boren

, Brown

. Improving preventive care by prompting physicians. Arch Intern Med, 2000; 160:301–308.

Linder

, Schnipper

, Palchuk

, Einbinder

, Li

, Middleton

. Improving care for acute and chronic problems with smart forms and quality dashboards. AMIA Annu Symp Proc, 2006; 1193.

Jung

, Schnipper

, Li

et al. The coronary artery disease quality dashboard: A chronic care disease management tool in an electronic health record. AMIA Annu Symp Proc, 2007; 999.

Livaudais

, Unitan

, Post

. Total panel ownership and the panel support tool—“It's all about the relationship.” Permanente J, 2006; 10:72–79.

Feldstein

, Perrin

, Unitan

et al. Effect of a patient panel-support tool on care delivery. Am J Manag Care, 2010; 16:e256–e266.

Varonen

, Kortteisto

, Kaila

. What may help or hinder the implementation of computerized decision support systems (CDSSs): A focus group study with physicians. Fam Pract, 2008; 25:162–167.

Goud

, van Engen-Verheul

, de Keizer

et al. The effect of computerized decision support on barriers to guideline implementation: A qualitative study in outpatient cardiac rehabilitation. Int J Med Inform, 2010; 79:430–437.

10.

Fung

, Lim

, Mattke

, Damberg

, Shekelle

. Systematic review: The evidence that publishing patient care performance data improves quality of care. Ann Intern Med, 2008; 148:111–123.

11.

Liu

, Wyatt

, Deeks

et al. Systematic reviews of clinical decision tools for acute abdominal pain. Health Technol Assess, 2006; 10:1–167iii–iv.

12.

Garg

, Adhikari

, McDonald

et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA, 2005; 293:1223–1238.

13.

Sintchenko

, Magrabi

, Tipper

. Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med, 2007; 32:225–240.

14.

Shojania

, Jennings

, Mayhew

, Ramsay

, Eccles

, Grimshaw

. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev, 2009; CD001096.

15.

Kimura

, DaSilva

, Marshall

. Population management, systems-based practice, and planned chronic illness care: Integrating disease management competencies into primary care to improve composite diabetes quality measures. Dis Manag, 2008; 11:13–22.

16.

Follen

, Castaneda

, Mikelson

, Johnson

, Wilson

, Higuchi

. Implementing health information technology to improve the process of health care delivery: A case study. Dis Manag, 2007; 10:208–215.

17.

Olsha-Yehiav

, Einbinder

, Jung

et al. Quality dashboards: Technical and architectural considerations of an actionable reporting tool for population management. AMIA Annu Symp Proc, 2006; 1052.

18.

Linder

, Schnipper

, Tsurikova

, Melnikas

, Volk

, Middleton

. Barriers to electronic health record use during patient visits. AMIA Annu Symp Proc, 2006; 499–503.

19.

Zai

, Grant

, Estey

et al. Lessons from implementing a combined workflow-informatics system for diabetes management. J Am Med Inform Assoc, 2008; 15:524–533.

20.

MacLean

, Littenberg

, Gagnon

. Diabetes decision support: Initial experience with the Vermont diabetes information system. Am J Public Health, 2006; 96:593–595.

21.

Aetna. Aetna program has helped physicians over 350,000 times; MedQuery program finds care inprovement opportunities not likely to be discovered from paper medical records. http://www.businesswire.com/news/home/20050331005601/en/Aetna-Program-Helped-Physicians-350000-Times-MedQuery. 2012 July 23.

22.

Freeborn

, Pope

. Promise and Performance in Managed Care: The Prepaid Group Practice Model. Baltimore, MD: Johns Hopkins University Press, 1994.

23.

Zhou

, Unitan

, Wang

et al. Improving population care with an integrated electronic panel support tool. Popul Health Manag, 2011; 14:3–9.

24.

Feldstein

, Glasgow

. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf, 2008; 34:228–243.

25.

Feldstein

. Designing computerized physician order entry (CPOE) and academic detailing interventions to enhance outpatient prescribing safety. Presented at: 9th Annual HMO Research Network Conference, April 3, 2001, Denver, CO.

26.

Feldstein

, Smith

, Robertson

et al. Design and Implementation of Decision Support for Outpatient Prescribing in an Electronic Medical RecordAHRQ Publication No. 050021Rockville, MD: Agency for Healthcare Research and Quality, 2005.

27.

Feldstein

, Schneider

, Smith

et al. Harnessing stakeholder perspectives to improve the care of osteoporosis after a fracture. Osteoporos Int, 2008; 19:1527–1540.

28.

Erlandson

, Harris

, Skipper

, Allen

. Doing Naturalistic Inquiry: A Guide to Methods. Newbury Park, CA: Sage Publications, Inc., 1993.

29.

Marshall

, Rossman

. Designing Qualitative Research, 2nd. Thousand Oaks, CA: Sage Publications, Inc., 1995.

30.

Lincoln

, Guba

. Naturalistic Inquiry. Newbury Park, CA: Sage Publications, Inc., 1985.

31.

Strauss

, Corbin

. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications, 1990.

32.

Wolcott

. Transforming Qualitative Data: Description, Analysis, and Interpretation. Thousand Oaks, CA: Sage Publications, Inc., 1994.

33.

McDonald

. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med, 1976; 295:1351–1355.

34.

Dorr

, Bonner

, Cohen

et al. Informatics systems to promote improved care for chronic illness: A literature review. J Am Med Inform Assoc, 2007; 14:156–163.

35.

Chaudhry

, Wang

, Wu

et al. Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med, 2006; 144:742–752.

36.

Randell

, Dowding

. Organisational influences on nurses' use of clinical decision support systems. Int J Med Inform, 2010; 79:412–421.

37.

Chen

, Liao

, Chang

, Lai

, Chen

. Cost-effectiveness analysis for determining optimal cut-off of immunochemical faecal occult blood test for population-based colorectal cancer screening (KCIS 16) J Med Screen, 2007; 14:191–199.

38.

Peiris

, Joshi

, Webster

et al. An electronic clinical decision support tool to assist primary care providers in cardiovascular disease risk management: Development and mixed methods evaluation. J Med Internet Res, 2009; 11:e51.