Abstract
Patient-centered medical homes (PCMHs) have the potential to improve patient experience of care. Since 2006, Geisinger Health System has implemented its own version of an advanced PCMH model, referred to as ProvenHealth Navigator (PHN). To evaluate the impact of PHN on patient experience of care, the authors conducted a survey of patients whose primary care clinics had been transformed to “PHN sites” and were under case management at the time of the survey. A comparable survey of patients from non-PHN sites also was conducted for comparison. The results suggest that patients in PHN sites were significantly more likely to report positive changes in their care experience and quality; moreover, they were more likely to cite the physician's office as their usual source of care rather than the emergency room (83% vs. 68% for physician's office; 11% vs. 23% for emergency room). However, the results also suggest that there was no significant difference between PHN and non-PHN patients in their perceptions of access to care or primary care physician performance in terms of patient-centered care (eg, listening, explaining, involving patients in decision making). These findings are consistent with the expectation that transformation of primary care into PCMH can lead to improved patient experience of care. (Population Health Management 2013;16:157—163)
Introduction
To date, however, little is known about the impact of PCMH on patient experience of care. It has been shown that communication with patients and coordination of care across providers is often inadequate in traditional primary care settings. 10 Also, improvement in access and personalized care are considered to be critical to provide an optimal patient experience. PCMH transformation seeks to improve on the traditional model of primary care precisely by improving access, communication, and coordination. Thus, one may expect to see significant differences in terms of how patients in a PCMH perceive and rate their care experiences relative to those who are not in a PCMH. In reality, however, because some of these transformations are done “behind the scenes,” patients may not even notice any changes in terms of their care experience, even if the changes are real.
Thus far, only limited literature examines the association between PCMH transformation and patient experience. 1,14,15 Measurement of patient experience within the PCMH context is also in its infancy; it was only in October 2011 that the Agency for Healthcare Research and Quality (AHRQ) released CAHPS (Consumer Assessment of Healthcare Providers and Systems) Patient-Centered Medical Home Item Set, which is a survey tool specifically designed to capture patient experience of care in clinics that have been transformed into PCMH. 16 Of note, the release of this particular survey instrument occurred well after the completion of this study.
This study aims to expand this body of literature by comparing patients' reported care experiences between Geisinger's version of advanced PCMH, referred to as ProvenHealth Navigator (PHN), and traditional primary care, using original data from a survey conducted among patients who are members of a single regional health plan.
Background
PHN blends aspects of the chronic care and patient-centered primary care models to offer fully integrated population management for all patients' needs. The model aims to provide complete patient-focused care and seeks ultimately to achieve the following goals: (1) optimization of health outcomes of every patient; (2) delivery of evidence-based care in a patient-centered care model; (3) delivery of “value” in health care as evidenced by improved quality, patient experience, and efficiency; (4) improvement in the viability of primary care; and (5) not only bridging the gaps among different “silos” in the health care system, but seeking to optimize the flow of patients through all of the various silos. Table 1 summarizes the core components of the PHN intervention.
HIT, health information technology; P4P, pay for performance; PHN, ProvenHealth Navigator.
Designing this unique care model required an integrated approach that included a cooperative partnership between Geisinger's Community Practice Service Line and the organization's health insurance arm (Geisinger Health Plan, or GHP), with each entity focusing on its core area of strength—from population management to development and implementation of clinical best practices. This partnered approach built a new model founded on an alliance between an insurance company, patients and their families, primary care physicians (PCPs), and other health care partners. The pilot was conducted at 2 primary care practices in November of 2006. The PHN intervention expanded over the next several years, and by 2011, there were 43 PHN sites, which included 36 Geisinger-owned primary care practices, as well as 7 contracted primary care practices in GHP's provider network.
Effects of PHN in terms of significant reductions in hospitalization, improvements in certain patient outcomes, and total cost savings have been reported previously. 2 –4 However, any quantifiable or demonstrable impact on patient experience has not been made available until now.
Data Collection
GHP provides health insurance services to its members, who reside throughout central and northeastern Pennsylvania. Almost half of its membership receives primary care from Geisinger-owned primary care clinics, all of which had been converted to PHN sites by 2011. The other half of the membership receives primary care from GHP's contracted network of PCPs. In general, relative to the Geisinger-owned primary care clinics, there is wider variation among the contracted primary care sites in terms of physician leadership; systems of care for comprehensive chronic and preventive disease management; cultural evolution to physician-directed, team-delivered care; implementation of electronic health records; integration with other health care providers in the community; and other aspects of care management practices that contribute to care experiences.
At the same time, the patient population is geographically and culturally homogeneous by the virtue of the fact that virtually all GHP members reside in rural central Pennsylvania and are predominantly white (93%). The GHP claims data indicate that in 2010, GHP's “gatekeeper” plans had approximately 80,000 non-Medicare adult (ie, 18 years of age and older) enrollees, 24% of whom received primary care from PHN sites. The data also indicate that GHP had roughly 38,000 enrollees in its Medicare Advantage gatekeeper plans in 2010, 48% of whom received primary care from PHN sites. This study focuses on GHP members with gatekeeper plans because their PCPs and the clinics can be conclusively identified.
To measure patient experience of care, an original survey instrument was developed. Ideally, an existing, validated survey instrument (such as CAHPS PCMH items, which had not been released by the time of this study) would have been preferred and used. However, after a thorough review of the existing literature and discussions with experts in the industry, the authors were unable to find an existing survey instrument that was well suited for this study and decided to develop an original survey instrument instead (the actual survey instruments are available from the author upon request). The survey focused on collecting information that was not already available from GHP's member profile database; thus, standard demographic characteristics such as age and sex were obtained directly from GHP's member profile database.
A comparable survey of patients from non-PHN sites also was conducted for comparison. The questions included in the non-PHN comparison group survey were virtually identical to the questions included in the PHN intervention group survey. The only difference was that all references to PHN or “medical homes” were either removed or replaced with “primary care provider” in the non-PHN comparison group survey.
For the PHN intervention group, a total of 1262 eligible respondents were identified based on the inclusion criteria (detailed in the Analysis section); a total of 1415 eligible respondents were identified for the non-PHN control group. Surveys were mailed to all the eligible respondents starting in April of 2011. Returned and completed surveys were collected and recorded for analysis until July of 2011. Only a paper version was used; respondents were not offered any incentives for completing the survey.
Data Analysis
Based on the survey data, patient experience was assessed in terms of the following 4 domains of patient care experience: perceived changes in care delivery, usual source of care, access to care, and PCP performance. In particular, patients' perceived changes in care delivery were considered because there may be differences between PHN and non-PHN sites that might be undetectable in terms of the other domains of care experience. The use of an ambiguous time frame reference (“lately”) is a deliberate attempt to recognize the fact that the clinics had been transformed into PHN at different points in time. Table 2 lists the specific survey questions.
For the analysis, responses from the intervention group (PHN sites) were compared to the responses from the control group (non-PHN). However, a naïve comparison between these 2 groups has the potential to be misleading because of nonresponse bias. That is, if patients in PHN sites were more satisfied with their primary clinics than their counterparts in non-PHN sites, then PHN patients may be more likely to respond to the survey and give positive responses to the survey questions, thus leading to false-positive findings in favor of PHN. Furthermore, it may be that the relationship between patient experience and PHN is confounded by differences in patient case mix across PHN and non-PHN clinics. As will be discussed, the authors attempted to reduce these confounding factors by applying a strict set of inclusion criteria and statistical adjustments in their estimates.
The following set of inclusion criteria were used to select the final analytic sample: GHP enrollees in gatekeeper plans who (1) had been members of GHP for at least 1 year; (2) were at least 18 years of age at the time of the survey; (3) were under case management at the time of the survey; and (4) had claims related to 1 or more of the following 10 chronic conditions: chronic kidney disease (CKD), end-stage renal disease, diabetes, asthma, congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), coronary artery disease, hypertension, cancer, or depression.
The main purpose of these inclusion criteria was to identify retrospectively those respondents from GHP's patient population who had adequate health care experiences and interactions with their PCPs to give reliable survey responses. Also, the fact that the sample was further restricted to those members who were under case management at the time of the survey ensured that any difference between the PHN intervention group and the non-PHN control group is not related to case management. Limiting the sampling frame to case managed patients implied that the intervention and control groups are similar to each other in terms of case mix as well as health plan and health plan support.
Geisinger's case management program seeks to identify higher risk patients and to provide them with personalized and well-coordinated care. Within the PHN sites, however, nurse case managers are physically embedded within the practices to provide even more personalized and coordinated care by building long-term relationships with those at-risk patients. Given that the goal of this analysis is to evaluate the impact of the PHN intervention as a package of interventions (as described in Table 1), it is therefore important to focus the analysis on those case managed patients who presumably benefit from all the key elements of PHN, particularly the enhanced care coordination and long-term personalized care.
Note that care coordination and personalized care are 2 critical elements of not only PHN but also of PCMH principles in general. 5 To the extent that other PCMH interventions seek to provide care coordination and long-term personalized care via similar case management models, the current findings are generalizable to other such PCMH interventions. Thus, restricting the sample to case managed patients ensures not only that the intervention and control groups are similar to each other in terms of patient case mix but also that the findings are representative of the care experiences of the types of patients that PCMH is designed to benefit the most (ie, those with severe or multiple chronic conditions requiring case management).
A propensity score matching (PSM) technique was used to reduce the magnitude of potential bias in the estimates. First, the probability of each respondent belonging to the treatment group (ie, enrolled in PHN sites) or the control group (ie, enrolled in non-PHN sites) was calculated using a multivariate logistic regression model. The covariates included respondent age, sex, education level, as well as overall level of satisfaction with the primary care clinic (“How satisfied are you overall with the quality of care at your primary care site?”). This overall satisfaction variable was included as a covariate rather than an outcome variable because it was expected that those who report greater overall satisfaction, particularly those in PHN sites, would be more likely to respond to the survey. Thus, this variable was used to account for the potential nonresponse bias.
Other covariates included presence of claims related to the 10 chronic conditions specified, as well as MEDai (Medical Artificial Intelligence, Inc., Orlando, FL) risk scores (1=least severe; 5=most severe). MEDai has been used previously to capture patient case mix in other contexts. 11 Based on the propensity score obtained from the logistic regression model, the respondents were stratified into 5 blocks. Within each block, the control and intervention groups were balanced in terms of the propensity score as well as each covariate.
Results
Table 3 summarizes the differences in terms of the covariates as well as the response rates. Columns (1) and (2) show the descriptive statistics of the covariates, while column (3) shows the results of the propensity score logistic regression model, in which the dependent variable is the binary indicator variable that equals 1 if the respondent was in a PHN clinic and 0 if in a non-PHN clinic. Column (4) shows the 95% confidence interval around the logistic regression coefficient estimates.
P<0.05
CAD, coronary artery disease; CHF, congestive heart failure; CI, confidence interval; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; ESRD, end-stage renal disease; GED, general equivalency diploma; PCP, primary care provider; PHN, ProvenHealth Navigator; PS, propensity score; SD, standard deviation.
Table 3 shows that the response rate among the members in PHN sites is about 15 percentage points higher than among those in non-PHN sites. This implies that the survey data, particularly those of non-PHN comparison group, are likely to be subject to nonresponse bias because only a minority of the potential respondents actually did respond. The propensity score logistic regression model as shown in Column (3) confirms that the PHN and non-PHN respondents are systematically different from each other: The PHN respondents were more likely to be older, more satisfied with their clinics, more educated, and more likely to have claims related to CKD and depression; they also were less likely to have claims related to CHF, COPD, and hypertension.
Respondents were approximately 75 years of age on average, suggesting that the majority of the respondents were Medicare patients, even though the sampling frame included both Medicare and non-Medicare populations. This was mostly because the inclusion criteria specifically targeted case managed patients; at the time of the survey, about a third of GHP's Medicare patients had been under case management at any point during 2011, whereas only about 7% of the non-Medicare patients were under case management during the same year. Medicare patients also were more like to respond to the survey (38%) than non-Medicare patients (27%).
Table 4 shows the comparison between PHN and non-PHN respondents in terms of the 4 domains of patient care experience. After applying the inclusion and exclusion criteria, the final sample includes 499 respondents in the PHN intervention group and 356 respondents in the non-PHN control group. Both the PSM-adjusted and unadjusted differences are reported to show the magnitude of potential bias in each estimate. Table 4 suggests that the most significant difference exists in terms of the perceived changes in care delivery and usual source of care: PHN respondents were roughly twice as likely as non-PHN respondents to have noticed differences in their care, care coordination, and service. They also were more likely to report that the quality of care at their primary clinic site is different and has improved. Related to the noted improvement in care was a higher likelihood of citing their primary care office as their usual source of care (83% vs. 68%) and a lower likelihood of citing the emergency room (ER) as their usual source of care (11% vs. 23%).
P<0.05
ER, emergency room; IP, inpatient; PCP, primary care provider; PHN, ProvenHealth Navigator.
These differences exist despite the lack of a significant difference in reported access to care and patient perception of PCP performance. Specifically, the PHN and non-PHN respondents had similar assessments of their PCPs' interaction with them, including: listening to their concerns, explaining issues, and involving patients in the decision-making process, among others. Also, there were no significant differences in terms of respondents' reported ability to access their primary care clinics and specialists, or the availability of test results within a 1-week period.
Comparing the PSM-adjusted differences against the unadjusted difference, the unadjusted differences are consistently larger in magnitude than the adjusted differences. In particular, all the adjusted differences in terms of the PCP performance measures are smaller in magnitude and closer to zero than the unadjusted differences. This suggests that, without the PSM adjustment, the raw differences reflect the nonresponse bias likely resulting from patients who are more satisfied with their PCPs being more likely to respond to the survey and giving more favorable responses overall.
Discussion
This study is one of the early attempts to examine the impact of PCMH on patient experience of care. The results showed that patients at PHN sites were significantly more likely to perceive changes in terms of care, care coordination, and services in their primary clinics; they were also more likely to report that the quality of care was better than before. Consistent with these findings, the PHN patients were more likely to cite PCP offices—rather than the ER—as their usual source of care.
Somewhat paradoxically, however, the study also found that PHN patients were not necessarily more likely to report improved access or higher PCP performance. One possible explanation for this is the multiple possible definitions of quality for patients. Existing literature suggests that when patients are asked to define “quality,” as was done in this survey, they are likely to give multiple definitions. Most of these can be grouped into several categories: patient-centered care, access, communication and information, courtesy and emotional support, technical quality, efficiency of care/organization, and structure and facilities. 12 Among these, “patient-centeredness” is arguably the most relevant dimension of patient-defined quality for the purposes of this study, given that it is one of the core components of the PHN intervention. Berwick defines “patient-centered care” as “[t]he experience (to the extent the informed, individual patient desires it) of transparency, individualization, recognition, respect, dignity, and choice in all matters, without exception, related to one's person, circumstances, and relationships in health care.” 13
Thus, one possible explanation for the seemingly paradoxical finding is that the dimension of patient-reported quality for which the PHN patients have perceived and reported improvements was neither access nor PCP performance; rather, the PHN patients may have responded to how their primary care clinics have created greater focus on each individual patient's unique needs and personalization of care. It is possible that the survey questions were not adequately designed to capture the other relevant dimensions of patient-defined quality. This does not imply that other dimensions of patient-reported quality are less important or are beyond the scope of the PHN intervention. It does imply, however, that when measuring the success of other PCMH interventions in terms of enhancing patient experience, a more refined set of questions specifically targeted at capturing all elements of patient-defined quality is needed and should be the subject of future studies.
As noted, there is now a validated CAHPS PCMH Item Set developed by AHRQ that is specifically designed to capture patient experience of care within PCMH contexts. Future research should use this new survey instrument to measure longitudinal changes in patient experience before and after a PCMH implementation, preferably using the same cohort of patients over time, to draw more robust inferences about the impact of PCMH on patient experience of care.
Another possible explanation for the lack of significant differences between PHN and non-PHN groups in terms of access to care and PCP performance is that there already may be an existing trusted relationship between patients and their PCPs that persisted post PHN intervention. It may be that when patients and their providers already have an ongoing long-term relationship, a PCMH transformation such as PHN strengthens and formalizes this relationship, which may not be manifested in terms of any detectable differences in the standard measures of access and PCP performance, as demonstrated in this study. If this explanation is correct, it argues for transforming existing practices into PCMHs (as was the case for PHN) to leverage the existing patient-provider relationships rather than moving patients from non-PCMH practices to new PCMH practices where no such relationship exists.
This study focused on patients in GHP's “gatekeeper” plan types whose PCPs could be readily identified. However, the reality is that because of the popularity of non-gatekeeper plan types (eg, fee for service, preferred provider organizations), there is a substantial portion of the patient population not affiliated with any PCP. The results suggest that there may be benefits to these patients in terms of enhanced care experience if they can be successfully integrated into PCMH via, for example, accountable care organizations (ACOs). PCMH may be implemented within the context of the ACO to further enhance care coordination and improve care experience. 17
This study did not explore whether there may be other clinic or physician factors not related to PHN that might influence patients' care experiences. Conceptually, patients' care experiences also depend on the clinic's existing organizational structure as well as physician practice styles and patterns. The current data, however, do not capture these factors. Future research should examine the interaction between PCMH primary care redesign and other clinic and physician characteristics that influence patient experience of care.
Footnotes
Author Disclosure Statement
Drs. Maeng, Davis, and Graf, and Ms. Tomcavage, and Ms. Procopio disclosed the following conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors are employees of Geisinger Health System. The authors received no funding for the research, authorship, and/or publication of this article
