Test results management and distributed cognition in electronic health record–enabled primary care

Participants and setting

We interviewed 37 participants: 16 primary care providers (PCPs; including 11 MDs, 5 nurse practitioners), 12 registered nurses (RNs), and 9 licensed practical nurses (LPNs) to understand how they use communication, coordination practices, and artifacts in the management of cancer-related abnormal test results. PCPs, RNs, and LPNs represent three of the four roles in a PACT core. Based on our earlier work and input from key informants, the fourth core PACT role (clerical associate) is not regularly involved in follow-up of abnormal test results.

We recruited participants from primary care clinics at five sites dispersed across three states within one VHA regional network. Site size and location varies, including one large suburban VA Medical Center (VAMC), two large urban VAMCs, one small rural VAMC, and one rural community-based outpatient clinic (CBOC). Sampling was purposive according to role (RN, LPN, PCP). Recruiting was done by members of the research team.

Data collection

Prior to these interviews, we worked with local leadership and research partners to identify each site’s general diagnostic testing and follow-up process (as related to common cancers), and the typical roles for PACT members; this was used to refine the set of semi-structured interview questions.

We developed interview guides specific to each role (PCP, RN, LPN), focusing on how they learned of test results, tracked patients’ status, and communicated as part of test results management. We targeted abnormal test results for cancers that require timely follow-up. PCPs’ and RNs’ interviews focused principally on a recent case they could recall of an abnormal test result related to one of five common cancers (lung, colorectal, liver, breast, or prostate). By focusing on one recent case, we were able to obtain concrete details (similar to the critical decision method ²⁷ ) while also covering the management process across those five cancers.

We pilot tested the interview guides with PACT PCPs, RNs, and LPNs at non-participating VHA sites. Table 1 shows key interview questions.

OPEN IN VIEWER

Table 1.

Representative key interview questions.

Role	Question
PCP, RN	In the past few months, which one of the following abnormal results has most been an issue you have had to deal with a lot?  Iron deficiency anemia (60 days)  Positive FOBT or FIT (60 days)  Abnormal PSA (90 days)  Mammogram with BIRADS ⩾ 3 (60 days)  Abnormal chest x-ray or chest CT where radiologist indicates a suspicion for malignancy (30 days)  An abnormal alpha-fetoprotein (AFP) level (> 20) in a patient at risk for liver cancer (hepatitis B or C, cirrhosis; 60 days)
PCP, RN	Think about a recent case of a patient with [abnormal result type] and where a decision was made about how to follow-up?  What was the process for following-up on that result?  Who needs to know what, and how does that happen?  What information was used to determine what follow-up, if any, is required?
PCP	At what part or parts of your work day do you review lab results and the status of follow-up actions? At the times you started looking at follow-up for a patient with an abnormal test result, what has been going on?
RN	When in the course of your work do you review lab results and the status of follow-up actions?
PCP, RN	In terms of staying informed about the status of the PACT patients and their care, what information sources do you use? Email, secure messaging, or other?
LPN	How does communication between the teamlet members happen in the PACT?
PCP, RN	What working condition factors would make it likely for a PACT to have problems with timely follow-up for abnormal test results?
PCP, RN, LPN	In what ways do you or other members of the PACT get informed about changes in a patient’s status or issue with care?
LPN	Is there any system for monitoring or tracking patients who are at risk, or who need to have follow-up?

PCP: primary care provider; RN: registered nurse; LPN: licensed practical nurse; PACT: Patient Aligned Care Teams; FOBT: fecal occult blood test; FIT: fecal immunochemical test; PSA: prostate-specific antigen; BIRADS: breast imaging-reporting and data system; CT: computed tomography.

The interviews occurred in June and September 2014. All were conducted on-site except for one telephone interview. Interviews were conducted in private rooms by a human factors engineer (M.W.S.) and an industrial/organization psychology graduate student (C.B.); both interviewers took notes independently. Interviews were audio recorded and transcribed. The interviews typically lasted 30–60 min (longer for PCPs, shorter for LPNs).

Data analysis

Two authors (M.W.S. and C.B.) coded the data using content analysis and managed the data with ATLAS.ti software. First, the two coders independently coded the same interview using a preliminary framework based on the literature (on distributed cognition,^23,28,29 healthcare teamwork, ³⁰ and PACTs^31,32), as well as open coded any emergent concepts. They then compared results, refined the coding framework, and repeated with another interview. In this manner, the team coded the same one-third of the interviews, merged the data, and reached consensus on disagreements. The remaining two-thirds of the interviews were distributed between the two coders. As an additional cross-check, each coder evaluated the other’s results by comparing them to their own interview notes and post-interview assessments (which serve as an independent summary for comparison). The coordination and communication actions and artifacts, identified via the coding, were then assembled into patterns of how information is transmitted from one person or artifact to others. ³³

Dealing with information overload

A main source of information overload related to test results management are the notifications that appear in the provider’s EHR “inbox” (known as View Alerts in the VHA). The View Alerts includes diagnostic test results and other communications for all care activities the provider is involved with across all patients. The PCPs depend on the originating diagnostic service to specify the importance of each test result through structured coding of the abnormality. This provides a method for organizing and prioritizing the View Alerts. Depending on the test, this coding may be done automatically based on reference values, or by a pathologist’s or radiologist’s interpretation. Variations across individuals’ coding tendencies were reported. One PCP said of their site’s radiology service, “… Every single one they write in, they put as abnormality.” This undermines the codes’ utility.

The coding of test result importance is an example of preliminary processing of information to facilitate the PCP’s intake of the information. Similar functions exists elsewhere. One example was that when a patient visit note from a specialist outside of the VHA is scanned into the EHR, the LPN writes a summary note, including the reason for the visit and recommended follow-up for abnormal test results or other issues. An example of this function involving physical artifacts is when the LPN is preparing the documentation for a patient visit and notices an abnormal test result that lacks patient notification or other follow-up; the LPN will note this and list the current and prior lab results on the “routing sheet” (paper form used for a patient visit).

Coordination across distributed care

A key part of coordination in test results management is getting the right information to the right people within the right time windows. This involves (1) the PCP learning of the results, (2) the patient requesting the results, and (3) the results being communicated to the patient.

PCP learns of results

Figure 1 shows the communication pathways mentioned regarding ways PCPs learn about specific abnormal test results for their patients (non-exhaustive nor universal across sites). The six numbered pathways are described below.

OPEN IN VIEWER

Figure 1.

How PCP learns of results.

There are many different sources of test results shared with the PCP. If from that site’s diagnostic services, the results are typically shared directly. If not, then they are mediated by another party (i.e. an intermediary). We uncovered the following pathways:

Notification to ordering PCP: A provider sends a test/imaging order to diagnostic services and later receives the results back via View Alert notification. Various diagnostic services do not send a View Alert. RNs and LPNs typically do not receive test result View Alerts.

Ordered by another provider: An emergency department physician, medical resident, or other provider can order tests and have the results sent to the patient’s PCP. Not all providers ensure results are sent to the PCP. One attending physician mentioned doing a review every Friday night of the results of tests ordered by residents in order to verify nothing critical was missed and potentially neglected until Monday.

Critical Results: Diagnostic services directly contact the PCP for a critical result, or a PACT nurse if the PCP is unreachable.

Monitoring by RN: In some PACTs, the RN monitors a few basic issues for some patients and informs the PCP of important results.

From another VAMC: Results from a PACT patient seeing specialty care at another VAMC are sent to the RN Care Coordinator of the local VAMC, who sends them to the PACT. Representatives from the other VAMC may summarize or highlight information for the receiving PACT.

Mammograms: For mammography, a fax from the outside mammography service is sent to the VHA’s Women’s Health Clinic, where results are coded for staging breast cancer (BIRADS), entered into the EHR, and a View Alert and a paper letter are sent to the PCP.

Patient requests results

The process for notifying the patient can be initiated in different ways. It may be a PCP’s usual process of handling notifications or a response to another communication such as a patient call. Figure 2 shows different interconnected paths involved in handling a patient’s request for results; these are described below.

OPEN IN VIEWER

Figure 2.

Patient call for results.

The patient has different ways of reaching the PCP, often via a PACT member who acts as an intermediator. The call from the patient is received by the PACT nurses or the phone triage nurse (Pathway 1: call from patient). The triage nurse can route the call to the PACT or take a message and share it with the PACT via a region-wide message system (Pathway 2: routing call). The PACT nurses will inform the PCP directly or via EHR note (Pathway 3: informing PCP). Due to the branching in these pathways, getting the request from the patient to the PCP can be as simple as patient to LPN to PCP (two transitions) or as complex as patient to phone triage nurse to message system to LPN to EHR to PCP (five transitions).

Results shared with patient

A patient call is one way to initiate the process of communicating results to the patient, shown in Figure 3, with descriptions of the pathways below:

Notification to PCP: A communication to the PCP from the message system or EHR regarding a call from a patient can initiate the process.

Reminder from nurse: Alternately, a reminder from a PACT nurse can initial the process.

PCP talks to patient: There are various ways of contacting the patient. For significantly abnormal results, the PCP will want to speak directly with the patient on the phone.

RN calling patient: In some sites, RNs can make calls to notify patient about abnormal results that involve straightforward follow-up actions (e.g. “lab was abnormal, please come in for a repeat check”). In most sites, only the PCP can inform patients of abnormal results.

Alternate ways to contact: If the RN cannot eventually reach the patient (e.g. patient’s phone number does not work), the nurse may call an emergency contact or send a certified letter to the patient’s address on file.

LPN sets up visit: The PCP may want to discuss the results in person. The LPN will call the patient and schedule an appointment.

Patient visits: When the patient arrives for the appointment, the PCP and patient will discuss the results.

OPEN IN VIEWER

Figure 3.

Communicating results to patient.

Enhancing prospective memory

A single PACT provides care to a large number of patients, often over a thousand. Management of test results for a given patient can be challenging, especially, when working with technologies which fail to support memory of pending tasks. Providers reported several ways in which they used resources or strategies outside of the EHR’s functions for assistance (e.g. lists, team members).

Some providers refrain from signing-off on View Alerts for results until after they have taken care of the relevant tasks for that set of results. Many providers will add their own note or addendum to the patient’s chart and intentionally leave it unsigned to serve as a reminder. One PCP confessed, “The way the physicians are so overwhelmed; in order to not to forget about these patients … actually I put a unsigned addendum honestly.” Likewise, some LPNs mentioned using unsigned notes to keep track of patients whom they have been unable to call. However, some sites discourage the presence of unsigned documentation in the EHR.

One RN reported another example of repurposing an artifact to support prospective memory. For certain tests, the Women’s Health Clinic will print a paper letter to mail to the patient. This RN will keep a paper copy of the letter as a reminder of results that need follow-up.

Some PCPs ask their LPNs to remind them about completing follow-up tasks. One LPN described her role:

… she’ll get them [hardcopy mammography results] and say “make sure I sent a lab letter” and I [LPN] do double check. She’s kind of that way. She wants to make sure, “make sure I sent this lab letter out. Make sure I said that it was negative.” And then I’ll look at the report cause she’ll give me the report and I’ll make sure “oh yeah you sent the lab letter, yeah you put negative, yeah it’s negative.” [laughing] So that’s, I think it’s her double check –

In many PACTs, the RN keeps track of the status of care process steps (e.g. an appointment scheduled or procedure completed). Most of this information is pulled by the RN frequently viewing the patient’s chart to see what has changed—a time-consuming process.

While documenting any medical-related information outside of the EHR is discouraged, many nurses rely on lists to keep track of the status of patients in the testing and follow-up process. Nurses’ lists may include patients who need to be called, results that need to be checked, among other tasks. One PACT’s method was revealed by its PCP, saying, “I don’t know if I can say the ‘list’ word without you know—repercussions—but we do keep an um- an eye on that so.”

The use of spreadsheet-based-lists has been institutionalized into an official tool at one site. ³⁴ Each PACT has a spreadsheet on the local server called the High-Risk Patient Tracker, accessed only by the RN. Per an RN, this tool typically contains information for 6–7 patients, with 15–20 as a maximum. The PCP decides a patient’s case is complex enough to warrant intensive tracking and requests the RN to add the patient. Multiple times per week, the RN looks at the spreadsheet and uses each patient’s chart to update the status of his or her follow-up.

Even with the official spreadsheet tool, further adaptations were used. One RN uses the spreadsheet plus unsigned notes in the EHR. Another RN does not delete patients from the spreadsheet but uses color to mark them as not actively tracked because often patients will be returned to the list. This prevents having to re-enter that patient’s data.

Main findings

We identified how primary care teams manage test results in a large, integrated healthcare system with a well-established EHR. Using a distributed cognition approach, we uncovered various strategies PACTs use to cope with the challenges of information overload, coordination across distributed settings, and demands on prospective memory. From these, we identify opportunities for health information technology (HIT) improvements for distributed healthcare settings, such as primary care.

There are some straightforward and reliable pathways for test results follow-up and patient notification of results. However, many situations do not fit into these pathways; more complex communication pathways are required. We speculate that the Pareto principle would describe the distribution of cases over pathways. ³⁵ This would imply that approximately 80 percent of the cases can be managed by the simplest 20 percent of the possible pathways but that the remaining 20 percent of the cases are complex and varied enough to require the messier 80 percent of the pathways.

Distribution of healthcare teams across space and time requires the use of various communication methods for coordination. This results in a large number of potential communication pathways through which notification and information exchange occur. One common characteristic of communication pathways identified in this study is their support for asynchronous communication (e.g. View Alerts, EHR notes, and message system). This contrasts with communications patterns previously identified in less distributed settings such as the Intensive Care Unit.^36,37 There, the work is mostly co-located, as well as temporally continuous and often short-term. The slower and more sporadic tempo of outpatient test results management entails use of asynchronous communication. The workflow of each party can proceed independently of the other, avoiding the need to disrupt workflow to communicate.

Various providers, especially nurses, play significant intermediary roles and relay information to both patients and other providers. Intermediaries can also aid in information exchange by filtering information, preventing the intended recipient (e.g. PCP) from being distracted with low priority or irrelevant messages, or act as buffers or barriers to information exchange by holding on to information until there is a good time for the recipient to receive it. ³⁸

Risks

Some risks occur with the strategies used by PACTs to adapt to the challenges in test results management. They may contribute to coordination failures rather than enhance PACT ability to notify patients of their results in a timely manner. Various sociotechnical factors have been identified as related to likelihood of missed or delayed follow-up,^15,39 including ambiguity about responsibility, ⁴⁰ information overload, ¹² and difficulty maintaining awareness of current and future status of follow-up. ⁴¹

Distributing the work among multiple parties may contribute to diminished awareness of task allocation and completion (a factor important for timely follow-up). ⁴¹ In this situation, PACTs may distribute test management tasks among members of the team; however, if they fail to request and/or receive updates on task completion, information will fall through the cracks. Organizational conditions and practices may also inhibit effective PACT coordination practices. For instance, a common policy for notifying providers of an abnormal test result includes sending the View Alert to more than one provider. Without defined roles and responsibilities for who should follow up with the patient, there can be ambiguity and the notified providers may fail to coordinate for successful follow-up. Furthermore, without adequate cognitive support from HIT and teamwork principles, intermediaries may unintentionally prevent important communication with the PCP or patient from occurring, resulting in delay in follow-up.

The cognitive work of test results management can be strained by information overload. ¹² Consistent with the previous literature, we found that non-patient facing work, such as managing information transmitted in View Alerts ¹⁴ and secure messaging, can contribute to information overload.^42,43 Further overload may also stem from the loosely distributed and interdependent nature of the work in test results management. The disjointed nature of the workflow and reliance on intermediary communication may impose additional cognitive burdens.

A specific part of awareness of the state of follow-up is the prospective memory for pending tasks. Our results demonstrated that current HIT fails to support prospective memory, making monitoring of test result task status more effortful. Whether involving View Alerts or RNs monitoring a chart for updates to a patient’s status, this can be a demanding process. Participants used lists, and adapted paper letters and unsigned EHR notes, to serve as external reminders. One risk of using unsigned EHR notes is that the more a provider is affected by information overload in the EHR, the harder it may become to find the unsigned notes.

Many PCPs relied on nurses as sources of external reminders. Having another person help remember a task does provide redundancy. However, each provider’s likelihood of prospective memory failure may not be independent. When a PACT’s PCP is experiencing high cognitive workload, it is likely the PACT’s nurses are as well, which weakens the safety benefit of redundancy.

Contributions and recommendations

This study is among the first to identify team member intermediaries as examples of adaptive strategies to challenges in test results management within a PACT environment. The functions fulfilled by the intermediaries, their needs, and the vulnerabilities that exist all help identify important opportunities for HIT in support of distributed healthcare delivery.

More generally, test results management and other distributed healthcare services require HIT support for team cognition (“the interplay of the individual cognition of each team member and team process behaviors” ⁴⁴ which is traditionally noted through team communication). ⁴⁵ Thereby, it is no surprise that recommendations regarding tracking of test results include suggested team communication strategies to bolster an accurate knowledge of task status—who will do or has done what. ⁴⁶ HIT should support these functions, especially, during conditions of information overload, fragmented care, and unreliable prospective memory.

Limitations

Due to the temporally and spatially distributed nature of the test results management process, we collected data through interviews, not observations. Consequentially, our data lack some of the specific details about artifact use that studies using observations provide.^59,60 However, the interviews did enable us to capture a more holistic view of the test results management system.

There is also the possibility that respondents could not recall or chose not to divulge relevant information. However, most respondents discussed recent cases in detail, and many disclosed unauthorized use of unsigned notes or paper lists.

We did not compare different sites or PACTs, nor did we compare the presence of challenges or use of mechanisms with measures of timely follow-up rates. We focused on the process from test result availability to patient contact; we did not evaluate primary care’s role in coordinating specialty follow-up care.