A Tool for the Evaluation of Clinical Needs and Eligibility to Pediatric Palliative Care: The Validation of the ACCAPED Scale

Abstract

Background:

Despite their importance, pediatric palliative care (PPC) services are still scantly diffused. In addition, eligibility criteria for PPC are quite complex. Consequently, clinicians require a tool that suggests how to refer patients with life-limiting diseases to the most appropriate service and how to properly allocate health care resources.

Objective:

Recently, the Accertamento dei bisogni Clinico-Assistenziali Complessi in PEDiatria (ACCAPED) scale has been developed by a group of experts in PPC to evaluate the specific clinical needs of pediatric patients with a life-limiting disease. This study presents the validation of the ACCAPED scale.

Design:

Validation of ACCAPED scale was pursued by means of description and analysis of clinical vignettes representing patients with challenging-to-evaluate needs who have to be referred to the most appropriate service (community care, general PPC, and specialized PPC). The evaluation of vignettes according to the clinical experience of the experts represented the gold standard against which the validity of the ACCAPED scale was tested by groups with different levels of experience (experts, pediatricians, and health care providers (HCPs) not involved in PPC).

Results:

Results show a very high concordance between the evaluation of the vignettes through the ACCAPED scale and the evaluation by the clinical experience for experts in PPC and pediatricians. A less favorable grade of concordance has been recorded for HCPs not involved in PPC, suggesting that educational efforts to improve basic knowledge of PPC within the medical community are needed.

Conclusions:

Overall, this study suggests that the ACCAPED scale is a useful tool to improve rationalization of resources and eligibility criteria for PPC.

Introduction

Pediatric palliative care (PPC) aims to respond to physical and psychological needs of children who are affected from life-limiting diseases, supporting their family at the same time.^1–4 The goal of PPC is to ensure the best possible quality of life for children and their caregivers, evaluating all their needs, without jeopardizing the efficacy and safety of the proposed interventions.^2,5 Remarkably, the realm of PPC extends far beyond end-of-life care. According to current models, curative, palliative, and bereavement care coexist for each patient and his/her family. Therefore, a multidisciplinary team is necessary to address these multiple needs.⁶ Of note, eligibility criteria to PPC are quite complex, since these patients generally present with very specific case histories, thus, making it unfeasible to directly adopt existing palliative protocols established for adult patients.⁷ Moreover, the severity of needs influences the level of service provided to the patient within community care, general PPC (i.e., provided by specialist of a given disease with training in PPC), or specialized PPC (i.e., provided in a dedicated setting by a multidisciplinary team of experts in PPC).⁷

Despite its importance, PPC is still scantly diffused. According to an Italian survey conducted in 2016, only 5%–15% of 30,000 children who required PPC had access to PPC service, thus showing a very limited availability of dedicated services.⁸ Therefore, it is necessary to properly allocate available health care resources; health care providers (HCPs) have the responsibility to refer patients and their families to the correct service.

At present, only scant evidence exists on tools that are aimed at evaluating the extent of the assistance burden in addressing the needs and complexities of incurable children and their families.⁷ The pediatric palliative screening scale has been previously described in the literature.^9,10 This scale helps to define a global support burden through a bio–psycho–social evaluation of each specific situation. However, this scale does not help to refer each patient and his/her caregivers to the most appropriate service according to their specific needs.

We have recently developed the ACCAPED Accertamento dei bisogni Clinico-Assistenziali Complessi in PEDiatria scale; scale for the assessment of complex needs in pediatrics), to assess the multifaceted needs of each patient who requires PPC.¹¹ In this way, it is also possible to identify the most appropriate approach of PPC for the patient—that is, community care, general PPC, or specialized PPC service.^11,12 However, this scale needs to be validated to ensure its correct application.

We have, therefore, assessed the validity of the ACCAPED scale through a study involving experts in PPC, pediatricians, and HCPs who are not directly involved in PPC.

Methods

The ACCAPED scale: an overview

The ACCAPED scale is a questionnaire, written in Italian, aimed to evaluate the specific clinical needs of a single pediatric patient with a life-limiting or life-threatening disease, developed by a multidisciplinary group of experts in PPC. A HCP fills the questionnaire and, according to the result obtained, can address the patient and his/her caregivers to community care, general PPC, or a specialized PPC service. A detailed description of the ACCAPED scale has been provided elsewhere.¹¹

In brief, the ACCAPED questionnaire evaluates 10 domains of PPC needs: respiration, feeding, seizures/altered consciousness, skin integrity, mobility, communication, rest and sleep, continence and evacuation, administration of drugs, and pain. For each area, the most relevant needs are defined by a score that increases with the complexity of the need (possible scores: 1–2–3–4–5–10–15–20–25–50). Moreover, the scale includes a modified version of the “surprise question” (“Would you be surprised if this patient died within the next 12 months?”), which was suggested as a trigger for referral to specialist palliative care.^13,14 This question accounts for a maximum of 50 points. The scores for each domain and the surprise question are then summed up, and the child is referred to community care in the case of low-complexity needs (score ≤29), to general PPC in the case of middle-complexity needs (score 30–49), or to a specialized PPC service in the case of high-complexity needs (score ≥50). The English version of the questionnaire (not validated in the English language) is provided in the Supplementary Data.

Validation of the ACCAPED scale

At the Pediatric Palliative Care Service of Padua (Veneto region, northern Italy), we have conducted a validation of the ACCAPED scale using the approach applied by Ramos et al. in a previous study on the development of an algorithm to aid triage decisions for intensive care unit admission, based on the evaluation of clinical vignettes by a group of experts.¹⁵

The process we followed can be summarized as follows (Fig. 1).

FIG. 1.

Validation of the ACCAPED scale. ACCAPED, Accertamento dei bisogni Clinico-Assistenziali Complessi in PEDiatria. PPC, pediatric palliative care.

Development of clinical vignettes

In total, 57 (see statistical section for the determination of sample size) clinical vignettes describing a patient with a life-limiting or life-threatening condition and peculiar clinical characteristics and needs were prepared. At the end of each vignette, the reader is asked to refer the patient to outpatient care, hospital care, or PPC services. Real cases have been used but had to be modified to make them not identifiable. The vignettes were developed by two experts in PPC and revised by a qualified methodologist. Some examples of vignettes are provided in the Supplementary Data.

Evaluation of clinical vignettes by experts without the ACCAPED scale

The clinical vignettes were evaluated by eight experts in PPC, four pediatricians, and four nurses, all working in a specialized PPC service recognized by the Italian Healthcare System for at least five years. All experts did not participate in the development of the vignettes, without the ACCAPED scale and relying only on their clinical experience. Each expert was blinded to the others and defined his/her own distribution. This step of the process aimed to produce the reference response—the “gold standard”—to each vignette on the proper referral of the patient described in the vignette. In case of discrepancy among experts, consensus was achieved by discussion during several web-based meetings involving all experts.

Evaluation of clinical vignette by the ACCAPED scale

The clinical vignettes were then evaluated using the ACCAPED scale, to refer patients described in the vignettes to the proper service according to the score calculated with the ACCAPED questionnaire. After proper training on the use of the questionnaire, three different groups of HCPs participated in this step: (1) the same experts who evaluated the vignettes without the ACCAPED scale; (2) a group of nine pediatricians working in the Veneto region without formal training in PPC; and (3) a group of nine HCPs working in adult palliative care (physicians and nurses).

Statistical analysis

Data were collected in an Excel spreadsheet and analyzed with SPSS v.21.0.

Assuming a K statistic equal to 0.80 under the null hypothesis and an expected value equal to 0.95 under the alternative hypothesis, with α = 0.05, β = 0.20, the proportion of vignettes with cases to be assigned to basic service, hospital service, and CPP service equal to 0.20, 0.30, and 0.50, respectively, the needed sample size of vignettes was estimated to be 57: 11 for community care, 17 for general PPC service, and 29 for specialized PPC. The number of participants to each group was selected by convenience method.

We conducted all the statistical analyses stratifying the sample by operator expertise—expert, pediatrician, and HCP without specific experience in pediatrics. To estimate the overall agreement between the scores assigned by each operator in each group with the reference distribution, we applied the Cohen's Kappa statistics with its relative 95% confidence interval (CI). Owing to few unsuccessful results observed in the group, we decided to study the classification assigned in this group in depth. Therefore, to evaluate the contribution of each scoring category to the overall agreement (i.e., the agreement between the score given by each NP_i and that assigned by the group of expert) the kappa category-specific (K_cs) statistic¹⁶ and its 95% CI were calculated. Each K statistic was interpreted in a qualitative manner based on the Landis and Koch classification criteria¹⁷ (K < 0 poor agreement; K = 0.00–0.20 slight agreement; 0.21–0.40 fair agreement; 0.41–0.60 moderate agreement; 0.61–0.80 substantial agreement; 0.80–1.00 almost perfect agreement).

Results

According to the expert assessment of the vignettes without using the ACCAPED scale and relying only on their clinical experience, 13 (23%) vignettes were classified as cases to be referred to community care services, 15 (26%) to general PPC service, and 29 (51%) to specialized PPC service. These values represent the “gold standard” for the validation of the ACCAPED scale.

Table 1 reports the overall concordance results with the gold standard when using the ACCAPED scale. We observed an “almost perfect agreement” with the gold standard in five out of eight experts and a “substantial agreement” in the remaining three. In the pediatrician group, “almost perfect agreement” was shown by four out of nine participants and a “substantial agreement” in the remaining five. These results proved a good performance of the instrument. However, in the HCP group, only four out of eight participants showed an “almost perfect” or “substantial” agreement, whereas the other four showed a “moderate” or “fair” agreement.

Table 1.

Overall Concordance Results

Group	Cases (n)	Raw agreement (%)	Cohen's Kappa	95% CI	p	Qualitative interpretation of agreement according to Fleiss¹⁶
Experts (E_i)
E1	57	89.5	0.829	0.699–0.958	<0.001	Almost perfect
E2	57	89.5	0.832	0.705–0.959	<0.001	Almost perfect
E3	57	93.0	0.884	0.775–0.994	<0.001	Almost perfect
E4	57	84.2	0.738	0.581–0.895	<0.001	Substantial
E5	57	93.0	0.885	0.777–0.994	<0.001	Almost perfect
E6	57	91.2	0.859	0.741–0.977	<0.001	Almost perfect
E7	57	87.7	0.796	0.655–0.938	<0.001	Substantial
E8	57	87.7	0.794	0.652–0.937	<0.001	Substantial
Pediatricians (P_i)
P1	57	82.5	0.723	0.567–0.879	<0.001	Substantial
P2	57	82.5	0.720	0.562–0.878	<0.001	Substantial
P3	57	82.5	0.709	0.545–0.873	<0.001	Substantial
P4	57	77.2	0.643	0.473–0.814	<0.001	Substantial
P5	57	89.5	0.831	0.703–0.959	<0.001	Almost perfect
P6	57	89.5	0.831	0.703–0.0959	<0.001	Almost perfect
P7	57	89.5	0.829	0.700–0.959	<0.001	Almost perfect
P8	57	86.0	0.773	0.627–0.919	<0.001	Substantial
P9	57	93.0	0.885	0.777–0.994	<0.001	Almost perfect
HCP (NP_i)
HCP1	57	89.5	0.831	0.703–0.959	<0.001	Almost perfect
HCP2	57	75.4	0.595	0.411–0.779	<0.001	Moderate
HCP3	57	75.4	0.621	0.448–0.793	<0.001	Substantial
HCP4	57	64.9	0.468	0.281–0.656	<0.001	Moderate
HCP5	57	68.4	0.516	0.331–0.701	<0.001	Moderate
HCP6	57	49.1	0.265	0.077–0.452	<0.001	Fair
HCP7	57	75.4	0.603	0.423–0.784	<0.001	Substantial
HCP8	57	89.5	0.829	0.700–0.959	<0.001	Almost perfect

CI, confidence interval; HCP, health care provider.

To explain this result, we calculated the kappa category-specific statistic values and its relative 95% CI to indicate how each category of referral (community care, general PPC service, and specialized PPC service) contributed to the overall agreement in the HCP group (Table 2). For vignettes with a gold standard of “referral to specialized PPC service,” seven out of eight participants showed “substantial agreement” with the gold standard. However, only two out of eight HCPs showed a substantial agreement with the gold standard for vignettes, suggesting a referral to general PPC service. In contrast, no differences among different types of referral were disclosed in the expert and pediatrician groups (data not shown).

Table 2.

K_cs Statistic in the Health Care Provider Group

HCP	Community care				General PPC service				Specialized PPC service
HCP	K_cs	95% CI	p	Qualitative interpretation of agreement^a	K_cs	95% CI	p	Qualitative interpretation of agreement^a	K_cs	95% CI	p	Qualitative interpretation of agreement^a
HCP1	0.820	0.650–0.990	<0.001	Almost perfect	0.703	0.478–0.928	<0.001	Substantial	0.930	0.834–1.000	<0.001	Almost perfect
HCP2	0.657	0.437–0.877	<0.001	Substantial	0.194	0–0.561	0.073	Slight	0.789	0.629–0.949	<0.001	Substantial
HCP3	0.608	0.387–0.829	<0.001	Substantial	0.341	0.030–0.656	0.008	Fair	0.825	0.679–0.972	<0.001	Almost perfect
HCP4	0.444	0.213–0.675	<0.001	Moderate	0.047	0–0.427	0.680	Slight	0.755	0.586–0.925	<0.001	Substantial
HCP5	0.608	0.387–0.828	<0.001	Substantial	0.208	0–0.537	0.109	Fair	0.650	0.454–0.847	<0.001	Substantial
HCP6	0.375	0.146–0.605	<0.001	Fair	−0.257	Not estimable^a	0.037	Poor	0.513	0.291–0.735	<0.001	Moderate
HCP7	0.675	0.437–0.877	<0.001	Substantial	0.415	0.104–0.725	0.001	Moderate	0.684	0.494–0.873	<0.001	Substantial
HCP8	0.854	0.694–1.000	<0.001	Almost perfect	0.716	0.502–0.931	<0.001	Substantial	0.895	0.779–1.000	<0.001	Almost perfect

Observed concordance is smaller than expected chance concordance.

K_cs, kappa category-specific; PPC, pediatric palliative care.

Discussion

In the PPC setting, resources are particularly limited and, therefore, proper referral of patients—and of his/her caregivers—to a specific service should be based according to his/her specific needs. However, a correct evaluation of needs of patients with life-limiting diseases is challenging, given the complexity of the clinical status of these patients, their natural physical and psychological development from child to adolescents, and the heavy burden on their families.⁷

To our knowledge, dedicated tools able to suggest clinicians how to refer patients with life-limiting diseases to the most appropriate service are not commonly used in clinical practice.

At the Pediatric Palliative Care Service of Padua, we have developed the ACCAPED scale, a user-friendly tool able to evaluate the specific needs of each child requiring PPC, according to 10 domains of PPC need and the “surprise question.” Each domain and the “surprise question” are given a score, and the sum of scores indicates the most appropriate service to which the patient should be referred—that is, community care, general PPC, or specialized PPC service.¹¹ However, it is important to point out that the ACCAPED assesses clinical needs only, but a properly based decision on the most appropriate service for each patient should also consider social, psychological, and ethical needs.

In this study, we have tested the validity of the ACCAPED scale by means of description and analysis of clinical vignettes representing patients with challenging-to-evaluate needs who have to be referred to an outpatient service in the case of low-complexity needs, hospital service in the case of middle-complexity needs, or to a dedicated PPC service in the case of high-complexity needs. The evaluation of vignettes by a group of recognized Italian experts in PPC according to their clinical experience represented the gold standard against which the validity of the ACCAPED scale was tested.

The ACCAPED scale was tested by three different groups of clinicians: the very same experts who have assessed the vignettes, a group of pediatricians working in the Veneto region, and a group of HCPs without formal training in PPC.

First, we reported a very high level of concordance between the evaluation of the vignettes through the ACCAPED scale and the evaluation by recognized experts according to their clinical experience only. This finding is of major importance, since it represents the first formal validation of the ACCAPED scale, in line with the aims of this study.

Second, we found a high level of agreement in the assessment of vignettes between a group of pediatricians working in community care and the gold standard represented by the clinical evaluations of recognized experts in PPC. This observation has immediate implications to clinical practice: indeed, by using the ACCAPED, a specialist in pediatrics without specific training in PPC can evaluate a complex situation as deeply as an expert of PPC, and refer the patient to the most appropriate service, potentially allowing a better allocation of resources. ACCAPED can also be well used in clinical practice since it requires only short (∼1 hour) training and only 10 minutes or less are necessary to complete. However, it must be noted that the ACCAPED scale is specific to the Italian scenario, and its use in different settings would require proper validation, although the different levels of assistance required may not vary across different countries.

Third, the results we observed are less favorable for the group of HCPs without specific experience in PPC, at least when the case is more challenging—for example, vignettes describing cases to allocate to general PPC service, a kind of “gray area” between community care and specialized PPC facilities. This likely reflects the poor level of knowledge of the pediatric setting of HCPs working in adult palliative care. Indeed, specific experience in the pediatric setting, with its peculiar needs and diverse diseases, is crucial for the management of pediatric patients eligible to PPC. However, HCPs working in adult palliative care, but without a direct involvement in pediatrics, are less likely to manage children with life-limiting or life-threatening diseases in clinical practice. These patients are more likely to be managed by multidisciplinary teams. In contrast, the ACCAPED scale would assist HCPs without specific involvement in pediatrics in referring patients with high-complexity needs directly to specialized PPC services.

Of note, educational efforts to improve basic knowledge of pediatrics and PPC appear necessary for all specialists of palliative care, and not only for those specifically involved in PPC.^2,3

Much effort is indeed required to improve PPC. We hope that the ACCAPED scale may represent only a step toward a refinement of PPC in Italy and in other countries.

Footnotes

Acknowledgments

Editorial assistance was provided by Aashni Shah (Polistudium SRL, Milan, Italy) and Simonetta Papa, and was supported by internal funds.

The ACCAPED Study Group

Caterina Agosto, Centro Regionale Veneto di Terapia del Dolore e Cure Palliative Pediatriche, Padova, Italy; Antuan Divisic, Centro Regionale Veneto di Terapia del Dolore e Cure Palliative Pediatriche, Padova, Italy; Rusalen Francesca, Centro Regionale Veneto di Terapia del Dolore e Cure Palliative Pediatriche, Padova, Italy; Moro Nicoletta, Centro Regionale Veneto di Terapia del Dolore e Cure Palliative Pediatriche, Padova, Italy; Pellizzon Monica, Centro Regionale Veneto di Terapia del Dolore e Cure Palliative Pediatriche, Padova, Italy; Igor Catalano, Casa Sollievo Bimbi, Vidas, Milano, Italy; Marco Bolognani, ULSS 9 Scaligera, Verona, Italy; Mariella Rubin, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Roberta Battagin, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Cristina Zaggia, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Rossella Ranzato, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Pamela Ballotta, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Anna Mercante, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Martinolli Francesco, Dipartimento di Salute della Donna e del Bambino, Padova, Italy; Mazza Alessandro, Dipartimento di Salute della Donna e del Bambino, Padova, Italy.

Author Disclosure Statement

No competing financial interests exist.

Supplementary Material

References

WHO: Cancer pain relief and palliative care in children. 1998. https://www.who.int/cancer/palliative/definition/en/ (Last accessed July 1, 2020).

Benini

, Cauzzo

, Congedi

, et al.: Training in pediatric palliative care in Italy: Still much to do. Ann Ist Super Sanita, 2019; 55:240–245.

Kaye

, Rubenstein

, Levine

, et al.: Pediatric palliative care in the community. CA Cancer J Clin, 2015; 65:316–333.

Norris

, Minkowitz

, Scharbach

: Pediatric palliative care. Prim Care, 2019; 46:461–473.

Constantinou

, Garcia

, Cook

, et al.: Children's unmet palliative care needs: A scoping review of parents' perspectives. BMJ Support Palliat Care, 2019; 9:439–450.

Michelson

: Pediatric end-of-life issues and palliative care. Clin Pediatr Emerg Med, 2007; 8:212–219.

Jankovic

, De Zen

, Pellegatta

, et al.: A consensus conference report on defining the eligibility criteria for pediatric palliative care in Italy. Ital J Pediatr, 2019; 45:89.

Benini

, Bertolotti

, De Zen

, et al.: Conferenza di Consenso. Le cure palliative pediatriche: Dalla diagnosi di inguaribilità al lutto. La definizione della eleggibilità alle Cure Palliative Pediatriche. Commissione della Società Italiana di Cure Palliative (SICP) delle Cure Palliative https://www.sicp.it/documenti/sicp/2016/04/conferenza-di-consenso-le-cure-palliative-pediatriche-dalla-diagnosi-di-inguaribilita-al-lutto/ (Last accessed July 1, 2020).

Bergstraesser

, Hain

, Pereira

: The development of an instrument that can identify children with palliative care needs: The paediatric palliative screening scale (PaPaS scale): A qualitative study approach. BMC Palliat Care, 2013; 12:20.

10.

Bergstraesser

, Paul

, Rufibach

, et al.: The paediatric palliative screening scale: Further validity testing. Palliat Med, 2014; 28:530–533.

11.

Lazzarin

, Giacomelli

, Biscaglia

, et al.: La valutazione della Complessità assistenziale in cure palliative pediatriche. La scala ACCAPED come strumento per la valutazione dei bisogni [The evaluation of complex needs in PPC: the ACCAPED scale as a tool for the assessment of needs.]. Riv It Cure Palliative, 2019; 21:162–165.

12.

Integrating Palliative Care and Symptom Relief into Paediatrics: A WHO Guide for Health-Care Planners, Implementers and Managers. Geneva: World Health Organization, 2018. Licence: CC BY-NC-SA 3.0 IGO.

13.

White

, Kupeli

, Vickerstaff

, et al.: How accurate is the ‘Surprise Question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med, 2017; 15:139.

14.

Weissman

, Meier

: Identifying patients in need of a palliative care assessment in the hospital setting: A consensus report from the Center to Advance Palliative Care. J Palliat Med, 2011; 14:17–23.

15.

Ramos

, Perondi

, Dias

, et al.: Development of an algorithm to aid triage decisions for intensive care unit admission: A clinical vignette and retrospective cohort study. Crit Care, 2016; 20:81.

16.

Fleiss

: Statistical Methods for Rates and Proportions, 2nd ed. New York: Wiley and Sons, 1981.

17.

Landis

, Koch

: The measurement of observer agreement for categorical data. Biometrics, 1977; 33:117–127.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.09 MB