Risk Factor Model to Predict a Missed Clinic Appointment in an Urban,Academic,and Underserved Setting

Introduction

A missed clinic appointment (whether cancelled at the last minute or “no-show”) decreases continuity, adversely affects practice efficiency, and can harm quality of care. In a primary care practice, missed appointments result in lost revenue, increased cost to deliver health care, and decreased patient satisfaction. ^{1 –3}

Many factors have been identified previously as contributing to missed appointments. These can be divided into patient, appointment, and/or environmental characteristics such as age, race, insurance status or payer, behavioral comorbidities, comorbidities index, patient beliefs, immediate symptoms, previously kept visits, provider type (attending vs. resident), continuity of care, specialty, appointment wait times, time of the day, day of the week, weather, transportation methods, and even reminder methods. ^{2,4 –28} No single factor can adequately predict an individual's likelihood of missing an appointment and evidence for some factors, especially demographic and socioeconomic factors, is conflicting. ²

Observational studies have attempted to identify those factors that explain a missed appointment. Norris et al most recently provided a comprehensive literature review. ² They found that 4 factors alone have the greatest association with missed appointments in an adult population: appointment wait time (or lead time), patient age, financial payer, and patient rate of previously missed appointments. Multiple studies also agree that these are potent independent predictors, ^{2,4 –17} and have proposed models to predict a missed appointment using the derivation and validation techniques. ^{5 –7}

A number of interventions to decrease missed appointments have been tried and described in the literature. An early successful demonstration to decrease appointment wait time had significant results. ²⁹ Another more sophisticated implementation of a predictive model that created a scheduling algorithm based on individual patient characteristics was reported by Dove and Schneider. ¹⁷ However, most facilities address the problem of missed appointments by overbooking. The estimated national missed appointment rate ranges from 14% to 50%, ^{27,30 –32} so the higher the no-show rate, the bigger the potential variability that will be experienced. Double-booking for a no-show rate of 50% could be disastrous if everyone shows up.

In recent times, the industrial and systems engineering literature has reported applicable methods that incorporate known previously identified independent variables and applied them effectively onto highly sophisticated advanced scheduling models. For example, Daggy et al used the Muthuraman and Lawley stochastic method to describe and model a scheduling system that could incorporate patient characteristics and probabilities of missed appointments into the scheduling system. ^6,33 Clinical Decision Support (CDS) systems have surged as a system solution in which characteristics of individual patients are matched to a computerized knowledge base. ³⁴ The effective use and system integration of CDS is a critical component established by the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009. ³⁵

The objectives of this study were to identify the predictors of a missed clinic appointment and to develop a model that effectively predicts an individual's likelihood of missing an appointment in adult patients at an urban, academic, internal medicine primary care practice that serves a mostly Medicaid population. The study was approved by Baystate Medical Center's Institutional Review Board.

Methods

Results

The sample contained 11,546 patients and 163,554 encounters; 8084 (70%) patients were assigned to the derivation set and 3462 (30%) to the validation set. Patient characteristics are shown in Table 1. Patients identified their race/ethnicity as Hispanic (47%), white (27%), African-American (18%), other (2%), and no disclosure (6%). Although most patients identified themselves as English proficient, a sizeable proportion was non-English proficient (22% Spanish-speaking only). Most of the care was provided by residents under faculty supervision. The most common insurance type was Medicaid. The mean number of missed appointments per patient and the average time from booking to actual appointment are shown in Table 1.

In the derivation set, 45% of patients missed their appointment. Factors associated with missed appointments appear in Table 2. In the multivariable model, the strongest predictors were percentage of previously missed appointments (OR per 10% increase 1.20, 95% CI 1.17,1.23) and time from booking to actual appointment (OR per day 1.02, 95% CI 1.02,1.02). Other variables associated with a missed appointment are male sex, provider type, and summer, fall, and winter seasons when compared to spring. The probability of a missed appointment was higher on Mondays compared to other weekdays. Age was inversely related to the probability of a missed appointment—the older the age of the patient, the lower the probability. Non-English proficient speakers also had a lower probability of missing an appointment.

In the validation set, there was generally consistent agreement in model-predicted and observed values across all deciles of risk (Fig. 1). Overall model prediction accuracy measured by the c-statistic was 71% and it calibrated well (P=0.63) (Fig. 1). The multivariable model produced deciles of mean predicted risk from 16% to 74%, while the mean observed risk over the same deciles ranged from 14% to 71%. The predicted and observed rates of a missed appointment were both 41%.

OPEN IN VIEWER

FIG. 1.

Observed vs. predicted percentage of missed appointments in the validation cohort.

Discussion

In nearly 164,000 patient encounters in an urban, academic clinic serving a mostly Medicaid population in western Massachusetts, it was found that the rate of missed appointments in the study derivation sample—45%—was much higher than anticipated. Both patient and appointment variables predicted missed appointments. The percentage of previously missed appointments, in particular, and the wait time from booking to actual appointment were the 2 strongest predictors. Other variables, such as season, provider type, patient sex, English proficiency, and younger age, also were associated with missed appointments. Race and insurance payer or status have been associated with an increased rate of missed appointments but this was not the case in the study cohort. Using this model, visits could be accurately stratified into probabilities of a missed appointment from 15% to 75%.

Research of “no-show behavior” is relevant in today's post Affordable Care Act (ACA) ecosphere. However, most studies available in the medical literature originated after Congress established the Medicare and Medicaid programs in 1965. As a result of the law then, demand for primary care services increased significantly and access to care declined. In that era, Goldman et al ⁵ developed a 4-factor logistic regression model that accurately predicted no-show behavior based on age, race, psychosocial, or social issues identified by the treating physician and previous appointment-keeping behavior among 376 patients in an urban clinic. Goldman's model successfully stratified patients into 3 risk categories, ranging from a 7% to 24% risk of missing an appointment. The present study, performed in a much larger sample, was able to identify many more variables and to stratify patients across a much wider range of probabilities of missed appointments. The research team also confirmed that past appointment-keeping behavior predicted future behavior and that younger age was associated with a missed appointment. However, in this sample, in which socioeconomic status was more homogeneous, the team found no association between race and missed appointments, and patients who exhibited English proficiency were more likely to miss an appointment. The team did not measure psychosocial and socioeconomic factors because this information was not easily retrievable from the database systems and the team did not survey patients or perform manual chart review.

More recently, Bennett ⁹ used logistic regression in an urban teaching clinic to determine whether advance access scheduling, or ability to schedule an appointment within 48 hours, decreased missed appointments. They found advance access scheduling was not associated with missed appointments, but other factors, such as wait time to appointment, number of previous visits, provider type (resident vs. faculty), continuity with a provider, race, insurance status, and season were predictors of a missed appointment. They did not use the model to attempt to risk stratify patients.

Current systems engineering literature has clearly shown that the methods presented herein are not only valid and reproducible but also can be incorporated into advance scheduling solutions. The study results concur with previously cited literature and, most interestingly, show that the strongest predictors of a missed appointment are the rate of the individual's previously missed appointments, and wait time from booking to the actual appointment. Age also was a factor. ^2,6,7 In the present study cohort, insurance payer was not found to be a predictor, and this was most likely because of the insurance homogeneity of the patient population.

Many facilities do not consider the risk factors that predict a missed appointment and simply address the issue by overbooking. Although on average this tactic increases the chances that a physician will accomplish budgeted productivity per session, it also increases the variability in volume for any particular session. For example, if a practitioner or practice has an average no-show rate of 20%, and the session productivity goal is 10 patients, then 12 patients will be booked. On days when all the overbooked patients keep their appointments, the clinic overtime increases ^1,16 and both provider and patient satisfaction decreases. ^16,23 The higher the no-show rate, the larger the potential variability that will be experienced. Being able to predict an individual's risk of missing an appointment would therefore allow for more efficient scheduling by reducing the variability associated with overbooking. ¹⁶ In essence, a patient at low risk for a missed appointment merits receiving a 0.9 to 1.0 appointment slot. A patient at high risk for missed appointments merits a fractional slot depending on his or her individual no-show rate. Three reliable patients plus four 50% no show would thus fill 3×(1.0) plus 4×(0.5) or a total of 5 appointment slots for 7 patients.

Advanced interventions to decrease missed appointments have been described and tried, but few have been prospectively studied for longer than just a few months. It is argued that non-sustenance of advance access and scheduling efforts can be attributed to strongly held beliefs in the medical community and/or lack of leadership or support. ^3,39 However, Daggy et al demonstrated that sophisticated stochastic methods that incorporate patient characteristics and probabilities of missed appointments into the scheduling system can significantly improve access, session throughput, operational revenue, and decrease provider and allied services overtime. These positive outcomes came at relative minimal software and training costs. ⁶ CDS systems can potentially improve clinical practice, if implemented at the proper time and location of the decision making process and appropriate node within the physician or practice workflow. ^33,40,41

This study has several limitations. First, the data came from Massachusetts, where about 97% of the cohort had health care coverage, most with low co-pays, and no penalty for missed appointments. Second, the clinic is located in an underserved area where other socioeconomic factors not measured in this study may increase the probability of a missed clinic appointment. Thirdly, the missed appointment definition might have been too sensitive as the research team did not distinguish a cancellation versus a missed appointment within 24 hours. However, the team believes that both cancellations and missed appointments are equally disruptive to the provider schedule and clinical operations. Lastly, this study was limited to a single clinic site with a relatively high baseline rate of missed appointments. Thus, the results may not be generalizable.

Studies that consider the patient and his or her individual characteristics as the unit of study are inherently representative of the local population with its regional circumstances and within a given time frame. Nonetheless, this study infers that a model that selects just a few universal variables that are easily electronically retrieved and reproduced, and also remain constant and valid across multiple regions, is highly desirable.

In this study, a simple risk factor model successfully predicted the likelihood that an individual patient will miss an appointment, stratifying patients across a 5-fold range of missed visit probabilities. The data for the model were reliably collected through readily available electronic sources. Use of the model in practice should allow for a more targeted and precise advanced scheduling system at an urban, academic, internal medicine primary care practice that serves a primarily Medicaid population. This, in turn, should improve patient-centered and operational outcomes such as improved access, efficiency, patient experience, risk adjustment, revenue, provider satisfaction, and quality of care of a distinct patient population.

To the research team's knowledge, none of the described modeling tools that retrospectively attempts to predict a missed appointment have been implemented prospectively in any scheduling system. The team's next step is to incorporate the model directly into the scheduling system and measure practice efficiency and population health outcomes by reducing the variability related to missed appointments.