How prescribing available pharmacotherapies for alcohol use disorder can impact the healthcare system: A retrospective quality improvement study

Abstract

Alcohol use disorder is a multifactorial undertreated chronic disorder influenced by genetic, psychological, and environmental factors. Numerous pharmacotherapies are available and effective but are underutilized in healthcare. The purpose of this retrospective quality improvement study is to determine the impact of education sessions on the availability and efficacy of medications (focusing on Naltrexone) to treat alcohol use disorder in the healthcare system. Control charts were implemented to monitor the system change in two comparable urban areas. Dispensing rates increased at three points after a series of presentations. The first increase from baseline was 2.47 times, the second 3.7, and the third 4.81. Coinciding with these, weekly visits to the emergency department also decreased by 35% and stabilized at a 15% reduction. It was also observed that alcohol use disorder hospital admission rates decreased by 21%, but bounced back once the education sessions ended. Combined with counselling, pharmacotherapies can be effective in combating alcohol use disorder, while potentially reducing demands on the healthcare system.

Introduction

Alcohol Use Disorder (AUD) is a worldwide public health concern that results in high mortality and high social and economic costs when left untreated.¹ Globally, alcohol consumption contributes to 3 million deaths each year, and harmful use of alcohol accounts for 5.1% of the burden of disease across low-, middle-, and high-income countries, as reported by the World Health Organization.² In Canada, in 2017, alcohol use accounted for $16.6 billion, or 36.2%, of the overall cost of substance use, nearly triple that of opioid use ($5.9 billion). One-third ($5.4 billion) of this overall cost of alcohol use was acquired in the healthcare system, including inpatient hospitalizations, day surgeries, Emergency Department (ED) visits, and specialized treatment.³

The urgency to address AUD has become more pressing during the COVID-19 pandemic due to heightened financial difficulties, social isolation, and general uncertainty which resulted in atypical alcohol consumption behaviours.^4–6 In a recent Nano poll commissioned by the Canadian Centre on Substance Use and Addiction, 11.7% of the respondents indicated exceeding the recommended low-risk drinking guideline. Furthermore, 8% indicated that, on at least one occasion, they were unable to stop once they started consuming alcohol.⁷ These increased rates of alcohol consumption were reported to be a coping mechanism to deal with the stress and boredom experienced as a result of the pandemic. In addition to alcohol consumption at home, the number of ED visits related to alcohol use, including visits due to severe withdrawal and suicide attempts, has also increased.^8–10 In England, 28.6% of individuals hospitalized for COVID-19 had moderate levels of alcohol intake.¹¹ This provides an additional level of complexity in an already overtaxed system because those who struggle with AUD are often immunocompromised and may contract the virus more easily and suffer more severe symptoms.¹²

The British Columbia (BC) Provincial Guideline for the Clinical Management of High-risk Drinking and Alcohol Use Disorder, released in 2019, recommend four medications (Naltrexone, Acamprosate, Topiramate, and Gabapentin) that can be used to treat the disorder in conjunction with counselling, Naltrexone being the most common and a first-line pharmacotherapy.¹³ Despite the availability and efficacy of these pharmacotherapies, many providers are not aware of them or are hesitant to prescribe due to lack of confidence in medication effectiveness, perceived low patient demand, and/or inadequate training.¹⁴

This article reports on a retrospective quality improvement initiative aimed to determine the impact of raising awareness on the availability of these medications and their efficacy on the prescribing behaviours of the medical community, including the ED. A quality improvement design and analysis techniques were applied to evaluate the impact of 17 information sessions delivered in Penticton between August 1, 2016, and August 31, 2018, on the prescribing rates of Naltrexone, as per dataset obtained from the provincial PharmaNet database. We anticipated that providing education to the medical community would result in an increase in dispensing rates of Naltrexone and reduced visits to the ED for AUD.

Methods

Ethics review

As this is a quality improvement initiative with minimal risk, as per A pRoject Ethics Community Consensus Initiative (ARECCI), it does not require research ethics review.

Engagement sessions presentation

The presenter, Jeff Harries, is a well-known, award-winning, physician champion with 27 years of experience in family medicine and has been treating AUD with pharmacotherapies in conjunction with psychosocial approaches for more than 14 years. He is an active change agent in BC and has held numerous leadership roles, including electronic medical record Lead, Division of Family Practice Lead, and Head of Family Practice at Penticton Regional Hospital (PRH). Dr. Harries was commissioned to participate in the revision of the recently released Provincial Guideline for the Clinical Management of High-risk Drinking and Alcohol Use Disorder, including treating AUD with available pharmacotherapies.¹³ His contributions have been recognized by the British Columbia Patient Safety and Quality Council in 2020 by awarding him the Everyday Champion award. Recently, the British Columbia Centre for Substance Use created and awarded Jeff Harries with the inaugural Primary Care and Substance Use Trail Blazer Award in April 2021; an award inspired by his leadership efforts to raise awareness about AUD and available pharmacotherapies. No financial or other support was provided to Jeff Harries by pharmaceutical companies to deliver these presentations.

Dr. Harries’s goal with the engagement sessions was to bridge the existing gap between what is known in literature and what is practiced. From September 1, 2017, to January 1, 2021, he delivered a total of 153 presentations in 45 communities to more than 4,000 participants. The first 17 presentations were delivered between December 12, 2016, and August 31, 2018, to care providers, in Penticton, the intervention site for this study. This study summarizes the impact of these initial presentations. The objectives of these sessions were to raise awareness about the existing pharmacotherapies used to treat AUD and to motivate the audience to try this approach with their patients. Each session started with presenting scientific evidence of the efficacy of these medications (currently summarized in the BC Provincial Guideline for the Clinical Management of High-Risk Drinking and Alcohol Use Disorder),¹³ followed by treatment success stories, and ending with a question and answer period. These presentations lasted from 30 to 60 minutes, depending on time availability and audience interest. Sessions were delivered to as many groups as possible, starting with family physicians, due to their ability to adopt the treatment protocol. Audience size ranged from 7 to 57 individuals per session.

Upon delivery, Dr. Harries evaluated each session by collecting feedback from the audience. Subsequent presentations were modified to include answers to the most frequently asked questions. A sample recording of the presentation can be found by following this link: https://youtu.be/cHftRZdzKbQ. Accompanying supplemental content including an FAQ page, medication summaries, and medication decision aids were made available to those who were interested. As well, Dr. Harries would follow up and stay in contact via text or e-mail to answer any questions from attending clinicians as they began to change their practice.

Data source and statistical approach

The dispensing data were acquired from British Columbia’s PharmaNet system for the period from August 1, 2016, to August 31, 2018, and included total weekly counts for the dispensing of Naltrexone at community pharmacies in the Penticton LHA, with a served population of 41,800,¹⁵ and a comparable LHA, Kelowna, with a served population of 194,900.¹⁶ The comparable location was determined based on geographical proximity and on the comparable structure of the healthcare system. Both geographies are served by the same health authority and have the same primary care structure; they are similar in healthcare delivery, including constancy in clinical competencies, standards of work, and clinical treatment algorithms. It is not necessarily relevant to have comparable general population characteristics, as the focus of the study was on modifying behaviours of care providers and subsequent impacts on the healthcare system. Considering that the two locations are geographically approximate and within the same healthcare region, the population and demographics of the clinicians who attended the presentations can also be assumed as similar. The clinicians have the same clinical requirements in both locations. Also, it is well known that healthcare workers often live in the Penticton LHA and travel to comparable location for work, or vice versa.

An analytical approach was used to analyze the data, rather than enumerative, as expected in quality improvement studies.^17–19 The aim of an analytical approach is to improve future performance of the system and allow visualization of the temporal nature of the improvement.¹⁹ Data are sampled from a frame that changes over time, rather than at a point in time, and show patterns in future performance, rather than an estimate of the current situation.^17–22 Hence, judgment sampling is preferred over random sampling.²¹ A detailed description of this approach and the difference between analytic and enumerative studies is provided by Provost.^17–22

Recently, an analytic approach has been recommended for use in an ED setting²⁰ and as a real-time assessment of local conditions.²³ In the ED, quality improvement is often accelerated. By using the analytic approach, the benefits of intervention can be observed at the system level earlier. This earlier observation of impact can lead to lowering the cost of interventions and a more responsive system to the interest of patients, researchers, and decision-makers.^18,19 Further, an analytic approach provides a method to understand whether day-to-day variation in outcomes may be attributed to meaningful signals of change. Control charts were used to observe daily variation in reported COVID-19 deaths.²³ A chart detected the start of exponential growth in death rates and identified early signals that the growth phase was ending, providing a reliable signal that the epidemic was waxing or waning. The authors concluded that without these charts, we potentially lack a responsive, scientific method for a real-time assessment of local conditions.²³

Visits to the ED and hospital admission data were obtained for the same time period from the Discharge Abstract Database that captures administrative, clinical, and demographic information from acute care facilities. International Classification of Diseases (ICD) 9 and 10 codes were used that are 100% attributable to AUD, including alcohol psychosis, alcohol abuse, alcohol dependence syndrome, alcohol polyneuropathy, degeneration of nervous system due to alcohol, alcoholic myopathy, alcohol cardiomyopathy, alcoholic gastritis, alcoholic liver disease, fetal alcohol syndrome, fetus and newborn affected by maternal use of alcohol, and alcohol-induced chronic pancreatitis.

Using control charts, we plotted the rates of Naltrexone dispensing and visits to the ED over the course of engagement session delivery. Control charts have four main features: data displayed over time, a centre line (average), Upper Control Limit (UCL), and a Lower Control Limit (LCL). The control limits represent three sigma limits calculated from the data which are analogous to three standard deviations. When a process has data falling between the UCL and LCL and with no unusual distribution patterns, it is said to be stable (predictable). However, if there is a disruption to the process, for example, in case of an intervention, and there has been a significant change in the process or system, the model will be recalculated with the modified average and limits. As the model is defined by pre-set rules founded in statistics, it can be concluded that the impact of the intervention is statistically different.^18,19

Study limitations

Although it would have been preferable for this study to have been a prospective cohort study, this was not feasible at the time. Therefore, a retrospective observational approach was used, and conservative measures were implemented to enable the authors to make the conclusions claimed in this article.

Additional limitations include the regulated access to provincial data and a lack of ED visits and admissions data in the comparable location. The removal of counts below 5 per week, as per the provincial guideline relating to the use of PharmaNet data, created limitations to the comparative analysis. It would have been preferable to use the absolute counts that were obtained, especially as source data were de-identified of any patient information, alleviating any potential concerns of identification. Further, the limited access and retrospective approach of the study made it not feasible to access the needed data from a comparable location.

Results

In the Penticton LHA, with a relatively small population of 41,800 people,¹⁵ the rate of consumption of standard alcoholic drinks per capita is higher than the provincial average, 2.6 versus 1.5 drinks per day.³ Furthermore, Penticton residents have a 60% higher early death rate where alcohol was a contributing factor, as compared to the rest of the province.¹⁵ This high consumption is in the high-risk category of use, according to Canada’s Low-Risk Drinking Guideline.²⁴

The 17 face-to-face education sessions reached 262 physicians, nurse practitioners, and medical staff within the Penticton LHA, which approximates the entire medical community associated with the treatment of AUD at the time (Table 1). Participants included family medicine physicians, substance use clinicians, acute care clinical nurse managers, social workers, psychiatrists, community substance use program leaders, and Department of Surgery staff. Of these attendees, approximately 90 were potential prescribers; however, it is likely that some of the prescribers attended more than one presentation. Overall, presentations were well received by the audience, as indicated in feedback and numerous direct communications with Dr. Harries.

Table 1.

Distribution and attendance of education sessions

Education phase	Week number	Date	Medical staff audience	Attendance
Education phase	Week number	Date	Medical staff audience	N
1	6	February 7, 2017	Family Medicine	28
2	15	April 10, 2017	Substance Use clinicians	7
	16	April 21, 2017	Internal medicine doctors	11
	17	April 27, 2017	Emergency Department – Penticton Regional Hospital	11
	18	May 4, 2017	Acute Care Clinical Nurse Managers	13
	19	May 10, 2017	Integrated Mental Health and Substance Use staff meeting	26
	30	July 25, 2017	Lower Similkameen Health Centre	6
3	44	November 2, 2017	Social Workers at Penticton Regional Hospital	10
	45	November 6, 2017	Psychiatrists – Penticton Regional Hospital	8
	45	November 7, 2017	Family Medicine Continuing Medical Education	26
4	5	February 1, 2018	Pathways Community Substance Use Program	10
	6	February 6, 2018	Department of Surgery – Penticton Regional Hospital	12
	10	March 8, 2018	Hospital and Community Pharmacists	16
5	34	August 21, 2018	Chemical Dependency Response Team, Psychiatry	23
5	34	August 24, 2018	Mental Health Diagnosis Group	8
6	51	December 19, 2018	Mental Health and Substance Use and Community Partners	27
6	3	January 15, 2019	Department of Family Medicine	20
			Total	262

Over the course of the study period, roughly equal numbers of patients were treated with Naltrexone: 1,032 (2.5%) in the Penticton LHA and 1,182 (0.61%) in the comparable LHA, despite that the population in the comparable LHA was 4.5 times greater.

In total, there were 1,360 Naltrexone dispensing events for the Penticton LHA and 1,182 for the comparator LHA. Data were summarized in order of occurrence by week over the study period. As shown in Figure 1, three significant increases in rates occurred over the period of education delivery. Prior to the commencement of presentations, at baseline, the average prescription rate for Naltrexone was µ = 1.11 prescriptions per week. Once the first two presentations were delivered, weekly dispensing of Naltrexone increased which triggered the first significant increase in uptake to be 2.47 (UCL = 3.26) times higher than the baseline. It is evident that the change was maintained until a set of next presentations, after which the average dispensing rate increased to 3.7 (UCL = 5.12; LCL = 2.29). Interestingly, the rate of dispensing increased again at week 24, in 2018, to µ = 4.81 (UCL = 7.03; LCL = 2.58) prescriptions per week. Simply stated, the average dispensing rate (average weekly dispensing per 10,000) increased from about 1 to 4.8 by the end of the study period in the area where prescribers received education versus no change (rate stable around 1) in the comparison geography, during the same time period. Moreover, the rate at this final phase of presentations was 4.3 times the rate of the comparable LHA (Figure 2), where no substantial changes in dispensing rates occurred.

Figure 1.

Weekly dispensing of naltrexone at community pharmacies per 10,000 in the intervention Local Health Area (LHA) from December 12, 2016, to August 31, 2018.

Figure 2.

Weekly dispensing of naltrexone at community pharmacies per 10,000 in a comparable Local Health Area (LHA) from December 12, 2016, to August 31, 2018.

As seen in Figures 3 and 4, coincidentally, there was a substantial decrease in hospital admissions and weekly visits to the ED for patients identified with AUD substance misuse, as per the ICD-9 and 10 codes during the time of the presentations. A total of 37,840 ED visits were reported in 2018 in the PRH that services the Penticton LHA and surrounding area of the South Okanagan and Similkameen. Weekly visits to the ED initially decreased by 35%, from µ = 12.5 (UCL = 25.5; LCL = 0.3) to µ = 8.4 (UCL = 14.9; LCL = 1.9) and stabilized at overall 15% decrease, as compared to baseline, µ = 11.0 (UCL = 23.9; LCL = NA). In a smaller community, such as Penticton, this represents 78 fewer patient visits per year, or 213 visits had the initial decrease been maintained. Similarly, AUD hospital admissions also initially decreased by 21% during the time of the presentations, from µ = 28.0 (UCL = 47.9; LCL = 8.1) to µ = 22.3 (UCL = 34.8; LCL = 9.7). However, the rates returned to pre-intervention levels after six months, µ = 30.9 (UCL = 50.4; LCL = 11.4). Although this trend was observed, no direct conclusions can be made.

Figure 3.

Weekly emergency department (ED) visits for substance misuse in the Penticton Regional Hospital from April 16, 2017, to December 31, 2018.

Figure 4.

Monthly hospitalizations for substance misuse from January 1, 2015, to December 31, 2018.

Assessing the impact of the cost of prescribing Naltrexone in a primary care setting on the healthcare system was not an objective of this study. However, it should be noted that in British Columbia, Naltrexone costs $105 per month and is fully covered for clients under the Fair PharmaCare program.²⁵ In 2018 to 2019, the cost of a standard hospital stay in BC was $6,464, which is slightly higher than Canada at $6,162.²⁶ If one assumes that increased prescribing resulted in decreased hospitalizations, even with a conservative cost savings estimate, it is likely that the Penticton LHA avoided $504,192 per year through the avoidance of just 78 patient hospital visits per year.

Discussion

As per the conducted scoping review, this is the first study that investigates the impact of targeted messaging and knowledge brokering on the healthcare system in the treatment of AUD and uses control charts analytic approach to monitor system change. Seventeen face-to-face education sessions were delivered to the medical community over 18 months in the Penticton LHA. Naltrexone dispensing rates, ED visits, and hospital admission rates due to AUD, as per ICD-9 and 10 codes, were analyzed and related to the impact of presentations. Although causality is not implied in this study, a preliminary observation showed that alongside a raising of awareness, dispensing rates increased, and admissions and patient visits to the ED decreased. To the authors’ knowledge, no other initiatives were undertaken in that hospital at the time which could account for these observations. Therefore, the authors believe that the changes in admissions and visits to the ED could be attributable to the intervention. We believe that the lack of sustained decreased hospitalization rates could have been due to the lack of established internal treatment processes and standardized work. As well, patients who received medications while in the ED did not get the needed continuity of care once back in the primary care setting. Further, providers not yet familiar with the nuances of prescribing may not have tailored medications to the needs of their patients. For example, providers may have been more familiar with prescribing Naltrexone, as it is a first-line medication; it does not, however, work for everyone. Based on ongoing experience administering AUD pharmacotherapies, some patients may need one of the four other available medications, or a combination thereof to be fully effective.

Based on our findings, it is believed that there was a lag period after presentations to a measurable impact on the healthcare system. Practically, this could be a result of providers having an opportunity to implement their learnings and patients having sufficient time to fill their prescriptions. As well, findings showed a substantial effect after more than half (53%) of all the medical AUD community attended the education sessions. The findings confirm that presenting to prescribing providers first enables system change and presenting to non-prescribing personnel ensures a more supportive culture change.

As this is a retrospective study, there is a possibility that the dispensing patterns in the Penticton LHA are not relatable to those in the comparable LHA. We do not believe this is the case, however, as the data obtained from PharmaNet show continuously low numbers of dispensing throughout the education session time period. The resulting lack of data during the initial months of study caused us to start the graphic interpretation of data the week of December 11, 2016 (week 51) instead of at the start of the data collection period August 1, 2016 (week 31). Therefore, if these effects were due to external factors, they would have been evident in earlier data. It must be noted that the comparable LHA also saw slight changes in the dispensing pattern. Albeit, these were at a much smaller scale and some of which may be a carry-over effect from the Penticton LHA. It is well known that healthcare providers, including general practitioners, often travel between these locations.

Sustainability is a key consideration in improvement projects. Although the current study indicates that system change is possible, additional investigation of the mechanism of this change would provide an in-depth understanding and provide insight into key criteria needed to sustain this change. This is particularly the case when considering the impact on ED visits and admissions rates. It is likely that additional quality improvement activities are required, such as the development of necessary treatment pathways, to maintain a sustainable reduction in admission rates and ED visits due to AUD. Recently, our team has embarked on a project which aims to further embed the use of these treatments in a rural ED setting. The work entails developing care pathways that enable an effective start to treatment for patients with AUD as they enter the ED, and to promote AUD treatment and care in the primary care setting. Furthermore, to sustain and expand our efforts, our research team has created a national non-profit organization, the Canadian Alcohol Use Disorder Society (www.cauds.org), with the mission of spreading awareness about AUD and available treatments. An introductory package can be downloaded and presented to health leaders and providers.

Taken together, the relatively simple approach of providing information sessions to prescribing and non-prescribing care providers to dispel the stigma about AUD and the availability of AUD pharmacotherapies may have had a substantial impact on the healthcare system. In the current environment, during the COVID-19 pandemic, those who struggle with AUD are more likely to drink more and more often, which leads them to visit the ED or be admitted to the hospital. Such preventable visits and admissions are not only a risk to already immunocompromised AUD patients but are also not always necessary, as AUD can be effectively treated in the primary care setting. We encourage healthcare providers and leaders to make progress in improving AUD care within their areas of influence by raising awareness and making these pharmacotherapies available for their patients. This process will help patients and alleviate some of the strain on the healthcare system, especially during the ongoing pandemic and as post-epidemic concerns emerge.

Footnotes

ORCID iD

Izabela Szelest, PhD

References

Thompson

Ownes

Phshpakom

Faizal

Pirmohamed

. Pharmacotherapy for alcohol dependence: a stratified approach. Pharmacol Ther. 2015;153:10–24. doi:10.1016/j.pharmthera.2015.05.010

World Health Organization. The SAFER Technical Package: Five Areas of Intervention at National and Subnational Levels. World Health Organization; 2019. Accessed April 17, 2021. Available at: https://apps.who.int/iris/handle/10665/330053. License: CC BY-NC-SA 3.0 IGO

Canadian Substance Use Costs and Harms Scientific Working Group. Canadian Substance Use Costs and Harms 2015–2017. (Prepared by the Canadian Institute for Substance Use Research and the Canadian Centre on Substance Use and Addiction.). Canadian Centre on Substance Use and Addiction; 2020.

Rehm

Kilian

Ferreira-Borges

, et al. Alcohol use in times of the COVID 19: implications for monitoring and policy. Drug Alcohol Rev. 2020;39(4):301–304. doi:10.1111/dar.13074

Clay

Parker

. Alcohol use and misuse during the COVID-19 pandemic: a potential public health crisis?. Lancet Public Health. 2020;5(5):e259. doi:10.1016/S2468-2667(20)30088-8

Koob

Powell

White

. Addiction as a coping response: hyperkatifeia, deaths of despair, and COVID-19. Am J Psychiatry. 2020;177(11):1031–1037. doi:10.1176/appi.ajp.2020.20091375

Conducted by Nanos for the Canadian Centre on Substance Use and Addiction. Boredom and stress drives increased alcohol consumption during COVID-19: NANOS poll summary report. 2020. Updated July 1, 2021. Accessed April 17, 2021. Available at: https://www.ccsa.ca/boredom-and-stress-drives-increased-alcohol-consumption-during-covid-19-nanos-poll-summary-report

Balhara

YPS

Singh

Narang

. The effect of lockdown following COVID19 pandemic on alcohol use and help seeking behaviour: observations and insights from a sample of alcohol use disorder patients under treatment from a tertiary care centre. Psychiatry Clin Neurosci. 2020;74(8):440–441. doi:10.1111/pcn.13075

Narasimha

Shukla

Mukherjee

, et al. Complicated alcohol withdrawal—an unintended consequence of COVID-19 lockdown. Alcohol. 2020;55(4):350–353. doi:10.1093/alcalc/agaa042

10.

Smalley

Malone

Meldon

, et al. The impact of COVID-19 on suicidal ideation and alcohol presentations to emergency departments in a large healthcare system [published on-line ahead of print]. Am J Emerg Med. 2020;41:237–238. doi:10.1016/j.ajem.2020.05.093

11.

Hamer

Kivimäki

Gale

Batty

. Lifestyle risk factors for cardiovascular disease in relation to COVID-19 hospitalization: a community-based cohort study of 387,109 adults in UK. medRxiv [preprint]. 2020. doi:10.1101/2020.05.09.20096438

12.

Sarkar

Jung

Wang

. Alcohol and the immune system. Alcohol Res. 2015;37(2):153–155.

13.

British Columbia Centre on Substance Use, B.C. Ministry of Health and B.C. Ministry of mental health and addictions. Provincial Guideline for the Clinical Management of High-Risk Drinking and Alcohol Use Disorder. British Columbia Centre on Substance Use; 2019. Updated July 1, 2021. Accessed April 17, 2021. Available at: https://www.bccsu.ca/clinical-care-guidance

14.

Oliva

Harris

. If pharmacotherapies for alcohol use disorders are effective, why are they underutilized? Evid Based Med. 2014;19(6):230–231.

15.

Penticton

. Community health profile; 2021. Accessed March 16, 2021. Available at: http://communityhealth.phsa.ca/HealthProfiles/HealthReport/penticton?.archiveYear=2019

16.

Kelowna

. Community health profile. [Internet]; 2021. Accessed March 16, 2021. Available at: http://communityhealth.phsa.ca/HealthProfiles/HealthReport/Kelowna?.archiveYear=2019

17.

Portela

Pronovost

Woodcock

Carter

Dixon-Woods

. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf. 2015;24(5):325–336.

18.

Provost

Murray

. The Health Care Data Guide: Learning From Data for Improvement. Jossey-Bass; 2011.

19.

Provost

. Analytical studies: a framework for quality improvement design and analysis. BMJ Qual Saf. 2011;20(suppl 1):i92–i96. doi:10.1136/bmjqs.2011.051557

20.

Harries

Filiatrault

Abu-Laban

. Application of quality improvement analytic methodology in emergency medicine research: a comparative evaluation. CJEM. 2019;21(2):253–260. doi:10.1017/cem.2018.379

21.

Perla

Provost

. Judgement sampling: a health care improvement perspective. Qual Manag Health Care. 2012;31(3):169–175.

22.

Neuhauser

Provost

Bergman

. The meaning of variation to healthcare managers, clinical and health-services researchers, and individual patients. BMJ Qual Saf. 2011;20(suppl_1):i36–i40.

23.

Perla

Provost

Parry

Little

Provost

. Understanding variation in reported covid-19 deaths with a novel Shewhart chart application. Int J Qual Health Care. 2021;33(1):1. doi:10.1093/intqhc/mzaa069

24.

Canadian Centre on Substance Use and Addiction. Canada’s Low-Risk Alcohol Drinking Guidelines. Canadian Centre on Substance Use and Addiction; 2018. Updated July 1, 2021. Accessed March 16, 2021. Available at: https://www.ccsa.ca/canadas-low-risk-alcohol-drinking-guidelines-brochure

25.

British Columbia Centre on Substance Use, B.C. Ministry of Health and B.C. Ministry of Mental Health and Addictions. Pharmacotherapy Tables. British Columbia Centre for Substance Use; 2019. Updated July 1, 2021. Accessed March 16, 2021. Available at: https://www.bccsu.ca/wp-content/uploads/2020/04/AUD-Pharmacotherapy-Tables.pdf

26.

Canadian Institute for Health Information. Cost of standard hospital stay; 2021. Updated on July 1, 2021. Accessed April 24, 2021. Available at: https://yourhealthsystem.cihi.ca/hsp/inbrief?lang=en#!/indicators/015/cost-of-a-standard-hospital-stay/;mapC1;mapLevel2;overview;/