Abstract
Background:
Project ECHO™ (Extension for Community Healthcare Outcomes) is a form of online interactive teaching, which has gained international traction. This project evaluates the effectiveness of an ECHO-delivered palliative care education program for the South Dublin region of Ireland. Our aim was to measure project success by quantifying gains in staff confidence.
Methods:
The educational program consisted of 10 interactive sessions over a five-month period on palliative care topics ranging from pain management to advance care planning. Twenty nursing homes took part in the education program. Of these, a subgroup of six nursing homes were randomly selected for assessment. Likert scale-based questionnaires assessed staff confidence before and after each lecture and assessment was repeated at least six weeks postlecture. Five of the 10 sessions were assessed in this way. Other characteristics such as staff role and years of experience were also collected.
Results:
Twenty nursing homes and 353 staff participated in the education sessions. Of the 6 nursing homes chosen for assessment, an average of 42 questionnaires were returned per session (n = 211), representing 83% of attendees at these 6 selected nursing homes. Seventy-seven percent of questionnaires were successfully followed up for six weeks. Average confidence increased by 27% pre- to postlecture (6.4 [SD = 1.4] to 8.1 [SD = 2.1], p < 0.005). Confidence gains persisted at six weeks; 8.1 of 10 (SD = 1.4), with no significant drop-off (−0.01/10, p = 0.95). All staff groups (nursing vs. non-nursing) exhibited equal confidence gains (nursing gain of 27%, non-nursing gain 22%, p = 0.16), and all confidence gains persisted at six weeks.
Conclusion:
This interactive, novel, training program significantly improved nursing home staff confidence in managing palliative care situations, and this confidence was sustained at least six weeks after the sessions.
Introduction
Nursing homes today provide complex care to residents with life-limiting conditions. Emphasis on patient self-determinism and emergency department overcrowding has increasingly directed care from the acute to the residential setting. Quality specialized care remote from expert services, however, requires upskilling of nursing home staff and this is challenging in sites remote to expertise. Previous studies have shown deficiencies in palliative care knowledge among nurses working in long-term care on both sides of the Atlantic.1,2 A U.K. study 3 cited staff shortages, rostering, cost, and language difficulties among barriers to providing palliative care education to nursing home staff. This same study also reported nursing home managers felt attendance at education sessions would be higher if they were provided “in-house.” ECHO (Extension for Community Healthcare Outcomes) provides an affordable solution that addresses the growing need to provide community-based palliative care training and support to a large group of candidates in a time-efficient manner and without the geographical obstacles inherent in traditional models of education provision.
Project ECHO is an internationally recognized collaborative model of health education and care management (Fig. 1). It utilizes modern videoconferencing technology to link expert specialist teams at a “hub” site to that of frontline staff at “spoke” sites (usually their place of work). Together, they participate in “TeleECHO” clinics, which are interactive sessions and include spoke case presentations and emphasize on interspoke (rather than hub–spoke) interaction, with hub oversight and teaching. ECHO-delivered care commenced in 2003 and is now used in >65 health care areas in 23 countries. 4 It has been shown that the quality of care provided by Project ECHO-trained clinicians was equal to that of care provided by university-based specialists. 5 A 2016 systematic review concluded that ECHO may be an effective and potentially cost-saving model. 6
An illustration of ECHO model of education with emphasis on spoke–spoke interaction. Reprinted with permission from Project ECHO CYMH Ontario. ECHO CYMH, Extension for Community Healthcare Outcomes Child and Youth Mental Health.
The purpose of this study was to evaluate the effectiveness of an ECHO model-delivered palliative care training program to nursing homes in the South Dublin region of Ireland. Specifically, we assessed nursing home staff confidence in implementing palliative care strategies with their residents.
Methods
Project ECHO in Dublin
The ECHO program consisted of an initial face-to-face half-day workshop with participating nursing home staff. Their learning needs were determined, and the curriculum was derived according to their wishes (Table 1). Ten videoconferencing sessions (90 minutes each) then took place over a five-month period. The “Hub” site was the All Ireland Institute for Hospice and Palliative Care at Our Lady's Hospice and Care Services, Dublin. Using Zoom teleconferencing technology, the hub linked to 20 spoke sites (nursing homes in the South Dublin area). The sessions consisted of a specialist-led training lecture followed by case presentations from two or more of the nursing homes with clear emphasis on spoke–spoke discussion postcase presentation rather than hub–spoke discussion to encourage confidence and debate and enable spoke sites to cross-pollinate knowledge, supported but not lead by the hub site.
Number of Nursing Home Attendees Per Session
Assessed sessions in bold.
Assessment of ECHO
Questionnaires were developed to assess participants' confidence on the topic of the day. Participants were asked to rate their confidence on a Likert scale, for example, “Using a score between 0 and 10, how confident are you that you can support and facilitate advance care planning?” Two questions were asked on each topic, and participants were asked to include their staff role, their years of experience, and their contact details to allow for six-week follow-up (all questionnaires are available in Supplementary Data). Although not validated, the questions were derived from a previous study, 7 which looked at palliative care education using ECHO technology to nursing homes on the island of Ireland, and modified if necessary, under direction from the expert(s) leading each session. Likert scales were also used in this previous study, 7 and have been used in other, similarly structured palliative care education programs using ECHO. 8
All staff working at participating nursing homes were invited to attend, but it was up to individual nursing homes to decide how many staff could be excused from duty to attend. Staff from all professional backgrounds attended, including nurses, health care assistants, medical staff, and allied health professionals. Participating staff were requested to complete these questionnaires before and after each session regarding their confidence in managing the topic of the day. The questionnaire was repeated at least six weeks after the lecture (at least three attempts were made to contact each participant through telephone call/text). It was estimated based on distribution of scores from self-rated confidence on a previous ECHO program, 7 that a sample size of 30 participants per lecture would yield >90% power to detect a 20% difference in scores postintervention. 9 Consequently, 6 of 20 participating nursing homes were randomly selected to aim for a minimum sample size of 30 participants per session. A nominated contact person in each nursing home provided staff with the Likert scale-based questionnaires for 5 of the 10 sessions.
Descriptive statistics and Student's t-test were performed to analyze the relationship between the variables. A hierarchical strategy was used where covariates were entered into the model in a series of planned steps. The Pearson correlation coefficient was used to assess test-retest reliability. Data were entered in a Microsoft Excel Workbook and analyzed using Stata 13.
Results
A total of 20 nursing homes and 353 nursing home staff attended at least one ECHO session (two nursing homes amalgamated and one nursing home withdrew because of information technology (IT) difficulties from an initial registration of 22). There was a mean average of 83 attendees per session (Table 1), a median of 78 attendees at each session, and staff attended a mean average of 2.6 sessions each. Table 2 provides the characteristics of all those who completed the questionnaires. The collected demographics of the participants in the six selected nursing homes did not significantly differ from the collected demographics in the other nonselected, participating nursing homes (years of experience, p = 0.4; staff role nursing vs. non-nursing, p = 0.78).
Summary of Professional Role of Participants
The six selected nursing homes averaged 51 attendees per session and returned 42 questionnaires per session (83%), although some of the remaining 17% were unable to attend all or some of the lectures. Six-week follow-up data were successfully acquired for 77% participants (162/211 questionnaires).
Changes in staff confidence through ECHO participation
Prelecture confidence among nursing staff was significantly higher than non-nursing staff (6.7 vs. 5.3/10, p < 0.005). However, prelecture confidence did not differ based on years of experience (6.3 < 15 years, 6.4 > 15 years, p = 0.75). Prelecture Pearson correlation coefficient between question 1 and question 2 for all 5 lectures was 0.84, suggesting high positive correlation, which adds to the validity of the questionnaire.
Confidence scores immediately postlecture
Participants' mean confidence scores increased by 27% compared with prelecture values (8.1 vs. 6.4/10, p < 0.005). This change was observed for nursing (26% increase, p < 0.005) and non-nursing (34% increase, p < 0.005) groups. It was also present for those with >15 years of experience (29% increase, p < 0.005) and those with <15 years of experience (27%, p < 0.005) (Table 3).
Summary of Changes in Staff Confidence, Expressed as x/10 on Likert Scale
Confidence scores six weeks postlecture
Follow-up data at least six weeks postlecture were obtained in 77% participants (n = 162). Confidence gains persisted. The mean confidence score was 8.1 across all staff groups (27% increase from prelecture scores, p < 0.005). Confidence scores six weeks postlecture did not differ from confidence scores immediately postlecture (8.13 vs. 8.12, respectively, p = 0.95). This was the case for all subgroups; nursing 0.14 difference, p = 0.28; non-nursing 0.41 difference, p = 0.31; >15 years of experience 0.07 difference, p = 0.77; <15 years of experience 0.02 difference, p = 0.89 (Fig. 2). In addition, test-retest reliability was assessed postlecture and six weeks postlecture. The Pearson correlation coefficient was 0.71 for question 1 of 2, and 0.73 for question 2 of 2, suggesting a high positive correlation between postlecture and six-week postlecture scores.
Confidence scores of participants
Missing data
As mentioned, 77% (n = 162) participants were successfully followed up to acquire repeat confidence scores at least six weeks postlecture. Prelecture confidence scores did not differ between those who were successfully followed up and those who were not (p = 0.15). In addition, immediate postlecture scores did not differ between the two groups (p = 0.17).
Discussion
Project ECHO is an internationally recognized method of upskilling staff in remote settings with specialist knowledge, overcoming some of the logistical and resource barriers to education. Its interactive format is superior to a didactic lecture style as it encourages participant-directed learning and enhances confidence. It has been shown in previous studies that clinician confidence ratings are related positively to the validity of judgments. 10 Previous work highlighted confidence improvements in staff following ECHO-delivered training. 9 Our research reaffirms this benefit. Of importance, this study confirms that confidence benefits are sustained for more than six weeks after training. A surprising level of consistency was observed between scores immediately postlecture and scores six weeks later, further reaffirming the validity of reported sustained enhanced confidence. Of note, all staff gained similarly from the program, regardless of professional background or years of experience.
Isolated staff in nursing homes are subject to increasing scrutiny and transparency from health care provider monitoring and increasing medicolegal exposure. Yet, formalized educational programs to nursing home staff have not increased in tandem. 11 Previous research highlighted that many dying nursing home residents experience potentially avoidable symptoms. 12 Inappropriate emergency transfers to hospitals are an inevitable consequence, with emergency departments ill equipped to meet the challenging needs of such vulnerable patient groups.
There are plentiful data on terminal patients' preferred place of death; however, these data mainly pertain to patients in hospital or at home. Nearly 90% of patients in these situations wish to die in a nonhealth care setting. 13 A study of all inpatient deaths in a U.K. hospital over a year determined 69% of those who died after being admitted from a nursing home could have stayed in the nursing home to die. 14 Further study should be carried out as to whether remote upskilling will facilitate preventative strategies to avoid such transfers, whether it could yield greater patient and family satisfaction, reduce acute hospital costs, and improve nursing home staff morale and confidence. However, there is evidence that developing health care workers' skills and confidence in formulating advance care plans with patients and discussing sensitive issues such as ceilings of care, unplanned hospital admission, and personal goals positively impact the quality of end-of-life care. 15 There is also evidence that nursing homes with less training have higher transfer rates to emergency departments. 16
This study has several limitations. First, 6 of 20 nursing homes were selected randomly for the survey; although they had similar demographics to those not selected and power was calculated in advance, generalizability may be compromised. It must also be taken into account that overall session attendance was low (staff attended a mean of 2.6 sessions each out of 10 sessions). Follow-up data were incomplete as 77% participants were followed up at least six weeks postlecture. It is possible that the missed 23% may have had relatively lower confidence scores, but neither prelecture nor immediate postlecture scores differed significantly (p = 0.15 and 0.17, respectively). Although questionnaires were anonymized, participants may have exaggerated postlecture confidence to please supervisors, as contact details were included.
The questionnaires were not validated, although they were utilized in a previous ECHO-derived program. 7 Consequently, caution must be exercised in generalizing results to actual confidence improvements and we cannot assume that enhanced confidence culminates in meaningful outcomes for patients. Further work is ongoing in a separate ECHO lecture series to determine whether remote education impacts emergency hospital transfers and validated quality indicators.
Summary
An ECHO-delivered palliative program improves confidence for nursing home staff in managing residents with palliative care needs. Confidence remains sustained for at least six weeks and all staff benefit, regardless of profession or experience. Further work is required to identify whether an ECHO-delivered palliative education program reduces impacts on emergency hospital transfers and improves other quality indicators of health and well-being in residents.
Footnotes
Author Disclosure Statement
No competing financial interests exist.
This article has not been published and will not be submitted elsewhere for publication while being considered by the Journal of Palliative Medicine. The unfinished work to date was presented orally at the Irish Gerontological Society in September 2017, and thus is an abstract in Age and Ageing:
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
