Abstract
Patient satisfaction is a key metric used to measure quality in health care. However, patient satisfaction measures in the pediatric population are less studied and understood than in the adult population. The purpose of this study was to evaluate the impact of telephone follow-up on patient satisfaction in an outpatient pediatric neurosurgery clinic. A standardized telephone follow-up call was performed within 1 week of a child’s clinic visit. Pearson’s χ2 or Fisher’s exact tests were used to assess changes in patient satisfaction measures after implementation of the telephone follow-up call initiative. The proportion of overall “top-box” physician rating significantly increased from 85.5% in 2017 to 95.6% in 2018 (P = .04). There was also a nonsignificant upward trend in the proportion of respondents noting that they would recommend this provider, as well as in all measures of physician communication quality and office staff quality. A simple telephone call to new patients after an outpatient pediatric neurosurgery clinic visit resulted in statistically significant and clinically meaningful changes in patient satisfaction scores.
Introduction
The Institute of Medicine considers patient-centered health care a core measurement of quality health care delivery (1 –3). Therefore, the patient and/or caregiver’s evaluation of care is an essential component of quality health care. Currently, patient satisfaction is primarily measured through patient satisfaction surveys (1,4). These surveys are valuable tools that provide multiple benefits and opportunities for improvement, such as enhancing strategic decision-making, reducing costs, improving management, and monitoring health-care performance (4).
At this time, the most common tool for assessing patient satisfaction in an outpatient setting is the Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) Clinician and Group Survey. According to the Agency for Healthcare Research and Quality (AHRQ), the CG-CAHPS survey allows providers and health-care institutions to evaluate the patient experience based on measures, such as timely appointments, care, information, physician communication, coordination of care, helpful and courteous staff, and rating of the health-care provider (3,5 –7).
Evaluating patient experience in a pediatric setting is more complex; the experience of the family as a whole must be considered. However, additional considerations when surveying pediatric patients, such as difficulty in contacting children and variance in cognitive ability, have resulted in the common use of parent/caregiver reports to assess pediatric health-care quality (4). Although the CG-CAHPS child visit survey is one of the most commonly utilized survey for outpatient pediatric usage, the psychometric properties of this tool are delineated for the adult version only (8). The CG-CAHPS survey, which is applicable to both adult and pediatric patients, is standardized and validated (3,9). The CAHPS survey is available for any organization to use once the survey has been released into the public domain by AHRQ (10). In 2016, approximately 81 practices used the CG-CAHPS Child 3.0 Combined Survey, resulting in 12 488 respondents (11). The objective of this study was to evaluate the effect of telephone follow-up on patient satisfaction scores in a pediatric neurosurgery clinic.
Methodology
Instrument
The CG-CAHPS survey allows patients to report and evaluate their experiences of health care in the outpatient setting based on the components of access to care, doctor communication, staff quality, and 2 global ratings—recommend provider and overall doctor rating (3,6). In this study, the CG-CAHPS child visit survey (version 2.0) was used to survey parents/caregivers as a proxy for patient experience in the pediatric outpatient setting. No modifications were made to the CG-CAHPS Child Visit survey by our site. The CG-CAHPS survey was offered in 3 languages: English, Spanish, and Creole.
New patients evaluated in the clinic setting were randomly selected to receive a CG-CAHPS survey. All patients were less than 18 years old. Parents/caregivers of these randomly selected patients then submitted the survey to the national patient experience company, Press Ganey, via either mail or e-mail the day following the appointment. Surveys were processed daily by Press Ganey, and the satisfaction data were sent to health-care institutions on a monthly basis. The following survey components were used in our study: doctor communication, staff quality, recommend provider, and overall doctor rating. For overall doctor rating, response ratings of 9 or 10 were considered the “top-box score”; therefore, data pertaining to overall doctor rating were evaluated as top-box score for ratings of 9 or 10 and non-top-box score for ratings <9 (6). Each component of the survey was analyzed before and after telephone intervention to determine whether the intervention had an effect on patient satisfaction scores. Of note, there were no significant changes in providers or staff during the time period of this study.
Procedures
In January 2018, the pediatric neurosurgery clinic implemented a standardized telephone follow-up questionnaire. The parents or caregivers of all new patients were called after their child’s first visit. The CG-CAHPS data were not collected for postoperative and follow-up patients at our institution; therefore, these patients were excluded from this study. Medical assistants who were bilingual in English and Spanish performed the telephone follow-up calls within 1 week of the child’s initial clinic visit.
Three questions were asked to the parent or caregiver of a new patient: (1) How was your clinic visit experience? (2) Do you have any suggestions or feedback of how we can improve your experience? (3) Do you have any questions or concerns about your child’s care?
A voicemail was left for every parent or caregiver who could not be contacted. Medical-related questions were triaged to the clinic’s midlevel providers to be addressed. Feedback responses were documented in a continuous telephone feedback log and were updated weekly. This project received institutional review board approval from the University of Florida (UFIRB#201801070).
Statistical Analysis
This study used a pretest/posttest design to assess whether there were changes in patient satisfaction scores after implementation of the telephone follow-up call initiative. The monthly CG-CAHPS data for January 2017 through December 2017 were analyzed as the baseline patient satisfaction data before the telephone follow-up intervention. After the follow-up call, questionnaire had been used for 1 year, we compiled and analyzed the CG-CAHPS data from January 2018 to December 2018.
Patient satisfaction measures (overall top-box doctor rating, recommend provider, measures of physician communication quality, and staff quality) were assessed using Pearson’s χ2 or Fisher’s exact test. All data analysis was conducted using Stata SE version 15.1 (12). Statistical significance was set at P value <.05. Post hoc power analyses were conducted for all nonsignificant findings using G*Power 3.1.9.2.
Results
The CG-CAHPS satisfaction data were evaluated before and after telephone follow-up calls were initiated. Overall, there were an equal number of surveys in the pre and post periods (n = 69), resulting in a total analytic sample of 138 surveys. Results of individual satisfaction measures are presented in Table 1. Pre-post, there was a statistically significant increase in the proportion of top-box scores for overall physician rating, which increased from 85.5% in 2017 to 95.6% in 2018 (P = .04). Nonsignificant increases were also observed in scores of whether patient/caregiver would recommend this provider and among physician communication quality measures examined (ie, physician explained in a way that was easy to understand, physician listened carefully, physician gave easy-to-understand instruction, physician knew important medical history, physician showed respect for what the patient/caregiver said, and the physician spent enough time with the patient). Lastly, an upward trend was also noted among office staff quality measures, including the proportion of respondents with reports of clerks being helpful and of clerks treating patients with courtesy or respect. The estimated achieved power for all tests with nonsignificant results ranged from 0.10 to 0.50; suggesting that these statistical tests were underpowered and less likely to detect small differences.
Study Outcomes Before and After Implementation of a Follow-Up Telephone Protocol.a
a Variables may not total 100% due to rounding-up.
b Denotes statistical significance at P < .05.
Discussion
This pre-post intervention study evaluated the effect of implementation of a telephone follow-up on parent/caregiver satisfaction in a pediatric neurosurgery clinic setting. The survey components of the CG-CAHPS survey—physician communication quality, staff quality, recommend provider, and overall physician rating—were evaluated before and after the implementation of a telephone follow-up initiative. There was a statistically significant improvement in respondents noting a top-box score for overall doctor rating. However, there was no pre-post difference in scores in other areas, such as recommending the provider, physician communication quality measures, and office staff quality measures. Although results in these other areas were not statistically significant, providers in our clinic were already performing well in a vast majority of these measures during the preintervention period; potentially pointing to a ceiling effect.
Patient satisfaction scores have become a critical component of health-care delivery because these surveys are being linked to reimbursement and public reporting of outcomes (2,4). Therefore, it is essential to assess the patient experience with reliable and valid measures, such as the CG-CAHPS survey. Current literature supports the use of the CG-CAHPS survey to assess the patient and family experience because it has high construct validity, internal consistency, and reliability in measuring the patient and family experience (4,6,13,14). Health care organizations use the CG-CAHPS surveys for several purposes such as comparing survey data to aggregate results, improving the patient experience, and reporting scores to consumers (13 –15).
One intervention that could have a positive effect on parent/caregiver satisfaction is the implementation of a telephone follow-up call to the patient’s parent or caregiver after the clinic visit. A telephone follow-up can keep patients and families connected outside of the clinic setting, create positive social change through improved sense of well-being, and provide continuity of care, thus improving patient satisfaction. Furthermore, studies have shown that this intervention also has the potential to decrease unnecessary emergency department visits and/or unscheduled clinic visits and could ultimately be a cost-effective tool for health-care institutions (2,16 –18). Lack of follow-up may lead to adverse events such as postvisit problems or patient and family dissatisfaction. Parents or caregivers may also feel overwhelmed by their child’s complex condition, and this may be exacerbated when there is lack of support or resources from the child’s health-care provider. Telephone follow-up is a preferred tool in adult populations, but there is limited evidence available to support this practice in the pediatric setting (2,6,19).
In the pediatric surgical population, most of the studies that utilize follow-up calls implement them in the postoperative period to minimize the amount of time between discharge from hospital and the next clinic visit (2,17,18). Of the interventions that are done in the outpatient clinic setting, the majority of the interventions involve enhancing perioperative education (20). To our knowledge, there is no study looking at patient follow-up calls for measuring and improving patient satisfaction in an outpatient pediatric surgical clinic, making our study the first pediatric follow-up call study that focuses on the overall outpatient experience to help shape patient satisfaction.
The patient satisfaction calls were implemented in this practice largely because the authors wanted to try to improve the patient/caregiver experience. The calls did result in positive improvements in the patient experience as reflected in increased top-box scores for overall physician rating with a trend toward improvement in measures of provider recommendation, physician communication quality measures, and office staff quality. Feedback from the staff making the calls has become important in our practice because patients who are less than satisfied are able to express their frustrations with their experience in a phone call. As a practice, we are then able to make corrections such as improved communication or better service that would make the patient experience better in the future. In an era where reimbursements are increasingly tied to patient experience, being able to address patient concerns in a timely manner is crucial to being able to run a successful practice.
There are limitations to consider for this study. First, this study has a small sample size, which may limit the ability to detect small effect sizes and statistical significance, and was obtained from a single institution, meaning results may not be generalizable. In line with this, our survey had low response rates which may have introduced nonresponse bias; therefore, results of our study may not be representative of our entire patient population. Secondly, as the CG-CAHPS is administered by a third party company and is anonymized, we were unable to follow the same patients in the before and after periods; therefore, we are unable to determine whether patient satisfaction measures would improve with a person-level analysis because of implementation of the telephone follow-up initiative. In line with this, because of the observational nature of this study and anonymity of the data, we also had very little control over external factors that may have influenced this study’s results, such as patient-perceived satisfaction with individual providers in our practice, as well as the response rate for each individual provider. Despite these external factors, it is important to note that our practice physicians were not made aware of the intervention in order to avoid altering of behavior. Although the CG-CAHPS child visit survey is one of the most commonly utilized outpatient survey, separate psychometric analysis of this survey has not been done, as was done with the adult version of the survey. This is a significant limitation that needs to be addressed by the designers of the survey if it is to continue as the most common tool for child outpatient satisfaction measures. Lastly, given the short duration of this study, more time may be needed to see the full effect of this initiative on the study’s outcome measures. Despite these limitations, the results of our study provide some initial indication that a telephone follow-up initiative may improve overall top-box doctor rating in a pediatric surgical outpatient clinic.
Conclusion
To the best of our knowledge, this is the first study that evaluates a strategy for improving patient satisfaction among new patients in an outpatient pediatric neurosurgical clinic. Our results show that in a relatively short period of time (1 year), we were able to see positive changes in physician ratings. Further research with larger samples and increased follow-up time is needed to determine whether telephone follow-up calls may have a positive effect on respondents’ recommendation of a provider or their assessment of physician communication quality and office staff quality.
Footnotes
Authors’ Note
Data underlying this article are available upon request. Please contact Memorial Healthcare System Office of Human Research at 954-265-1847. This project received Institutional Review Board approval from the University of Florida (UFIRB#201801070).
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
