Adaptation and Validation of the Standardized Swallowing Assessment Tool for Patients With Moderate-Severe Traumatic Brain Injury and Cervical Spinal Cord Injury

Abstract

Oropharyngeal dysphagia is common in moderate-severe traumatic brain injury (TBI) and cervical spinal cord injury (SCI) patients and can have serious consequences. Delaying feeding in these patients can also be detrimental. Nonetheless, the psychometric properties of screening tools that can promptly identify dysphagia have never been tested in these neurotrauma populations. This study aimed to: (1) adapt, translate, and validate the content of the French-Canadian version of the the Standardized Swallowing Assessment (SSA) tool to meet the needs of moderate-severe TBI and cervical SCI patients, (2) examine its inter-rater reliability and criterion-concurrent validation, and (3) evaluate its clinical utility from the perspectives of critical care nurses. The SSA tool was adapted and translated using an integrated method for the cultural adaptation and translation of tools. Eleven experts participated in the adaptation of the SSA tool, which led to the clarification of one item, as well as a new step and instructions for the screening procedure. Content validation (i.e., item and scale relevance) was evaluated by multidisciplinary team members (n = 17). The mean content validity index (CVI) score was 0.97 for the entire scale, while the mean CVI scores for individual items ranged from 0.82 to 1.0. A total of 60 neurotrauma patients were enrolled for inter-rater reliability and criterion-concurrent validation. Interrater reliability was determined by comparing two scores: one score from nurses responsible for the care of enrolled patients and one score from the research nurse. The weighted kappa coefficients for inter-rater reliability were 0.86 for moderate-severe TBI patients (n = 30) and 0.73 for cervical SCI patients (n = 30). A speech language therapist (SLT) also assessed dysphagia and results were used as the standard clinical reference criterion to determine concurrent validity (sensibility and specificity) of the adapted SSA tool. The sensitivity and specificity were 92% and 50% for moderate-severe TBI, and 77% and 75% for cervical SCI, respectively. The positive predictive value (PPV) and the negative predictive value (NPV) were 65% and 87% in TBI patients, and 75% and 76% in SCI patients. Test accuracy was 71% and 77% for these same groups. The clinical utility of the tool was evaluated according to the following domains: appropriateness, accessibility of the required material, applicability, perceived effectiveness, and acceptability. Acceptability was the only domain with a level of agreement <80% (74%) among trauma critical care nurses (n = 49). Findings support the content validation and inter-rater reliability of the adapted French-Canadian version of the SSA tool in moderate-severe TBI and cervical SCI patients. Sensitivity was acceptable in both groups, but the specificity was lower, especially in moderate-severe TBI patients. Further validation of the adapted French-Canadian version of the SSA tool is needed in neurotrauma patients to confirm these results and to ensure safe dysphagia screening while avoiding oral feeding deferrals.

Introduction

Eating and swallowing are based on complex processes involving the voluntary and reflex actions of >30 nerves and muscles, to move substances from the oral cavity to the stomach while ensuring airway protection.¹ The swallowing process is classified in three stages, according to the location of the bolus: oral (preparatory and propulsive), pharyngeal, and esophageal.² Dysfunction of one or more parts of the swallowing apparatus may result in dysphagia. Functional or structural abnormalities affecting the oral cavity, larynx, pharynx, and upper esophageal sphincter, in isolation or combination, result in dysphagia affecting either or both transit and airway-protective functions, whereas diseases affecting the esophagus mainly result in a disruption of peristalsis function.¹

Dysphagia is estimated to affect up to 16% of the general population.³ However, dysphagia affects the neurotrauma population more heavily, especially those with moderate-severe traumatic brain injury (TBI) and cervical spinal cord injury (SCI), with an incidence rate of 30% on average when considering risk factors such as age, severity of injury, time since injury, and different methods to assess dysphagia.^4
–6 Moderate-severe TBI and cervical SCI primarily impair the oropharyngeal axis of the swallowing process, as a result of injuries to the central nervous system, to the pharyngolaryngeal structures or their peripheral tissues, after prolonged endotracheal and tracheal intubation or surgery.^4,6
–8 In addition, cognitive deficits associated with TBI may affect voluntary behaviors that make it possible to swallow food safely.⁹

The fact that there is a large proportion of moderate-severe TBI and cervical SCI patients with dysphagia is a significant clinical problem, given the associated risk of complications. Considering the biological role of the swallowing process in protecting the airway, a rate of up to 40% of pneumonia has been reported in TBI patients with dysphagia,^10,11 compared with <10% in those without dysphagia.¹¹ Similarly, studies conducted among SCI patients have shown pneumonia rates varying from 55% to 80% in those with dysphagia, compared with 20–30%^12,13 in those without dysphagia. In addition, a 30% mortality rate has been associated with aspiration pneumonia following a neurological event,¹⁴ and it may be 15 times higher in patients who have undergone anterior spine surgery, which is often required after cervical SCI.¹⁵

Beyond its impact on the respiratory function, dysphagia can also result in delayed oral feeding contributing to malnutrition, which in turn can lead to an increased risk of infection¹⁶ and pressure ulcers.¹⁷ Low visceral protein levels (e.g. albumin, transferrin) were shown to increase the risk of nosocomial infection and sepsis up to three times^18,19 in severely injured trauma patients. Moreover, nearly 60% of SCI patients with stage III and IV pressure ulcers had lower than normal albumin levels and anemia.²⁰ Finally, lack of screening for dysphagia can be detrimental to organizational efficiency. Patients who are not screened have been shown to require specialized resources [e.g., speech-language therapist (SLT)] and to have longer hospital stays because the resources needed are not always available outside of normal working hours.^21
–23

Accordingly, recognized associations and experts recommend that patients with moderate to severe TBI or cervical SCI should be screened for oropharyngeal dysphagia by a trained healthcare provider.^24
–26 Dysphagia screening is defined as a pass/fail procedure to identify individuals who require a comprehensive assessment of swallowing function or a referral to other professionals and/or medical services.²⁷ In this regard, guidelines recommend that if there are signs of oropharyngeal dysphagia or aspiration, a clinical swallowing assessment should be conducted by an SLT to determine the appropriate feeding strategy.²⁴

However, although several dysphagia screening tools are available, none have been validated among moderate-severe TBI and cervical SCI patients, or specifically recommended for these neurotrauma populations.^4,10 The Standardized Swallowing Assessment (SSA) tool^28,29 was selected by our expert advisory committee (i.e., trauma nurses, critical care physicians, a neurosurgeon, SLTs, nutritionists, and trauma decision makers) because of its relevance to these neurotrauma populations, its psychometric properties for similar conditions (e.g., stroke), and its ease of implementation.^23,30 The aims of this study were to: (1) adapt/translate, pre-test, and validate the content of the French-Canadian version of the SSA tool to meet the needs of patients with moderate-severe TBI and cervical SCI, (2) examine its inter-rater reliability and criterion-concurrent validation, and (3) evaluate the clinical utility of the SSA tool from the perspective of healthcare providers who use it in their everyday practice. Our hypotheses were that the adapted French-Canadian version of the SSA tool would have similar psychometric properties in the targeted neurotrauma population, as in the stroke population, and that a large proportion of healthcare providers would agree with its clinical utility.

Methods

To meet study objective 1, we used an integrated method for the cultural adaptation and translation of tools³¹ and followed validation content guidelines.³² Study objectives 2 and 3 were achieved through a prospective study in a level I trauma center in Montreal, Canada from 2018 to 2019. The study was approved by the Research Ethics Board (# 2016-1223).

Adaptation/translation, pre-testing, and content validation of the French-Canadian version of the SSA tool

The SSA was developed by nurses in the acute care setting for the early detection of dysphagia in stroke patients.³³ The original SSA tool consists of 11 items divided into two sections. Section A describes the criteria for dysphagia screening (i.e., level of consciousness, ability to be positioned upright with head control, ability to cough when asked to, capacity to manage saliva, ability to lick top and bottom lip, adequate oxygenation/ventilation parameters, voice quality) (7 items) and Section B includes an algorithm describing the procedures for dysphagia screening (i.e., give a teaspoon of water three times and half a glass of water followed by an assessment of signs of dysphagia) (4 items).²⁹ Failure to meet a criterion in Section A is a contraindication to dysphagia screening, whereas failing one of the steps in Section B indicates a positive screening for dysphagia. With the authorization of the tool's author, we proceeded to the adaptation and translation of the SSA tool into French-Canadian according to established guidelines: (1) assessment of the cultural conceptual equivalence by our expert advisory committee, (2) forward translation of the SSA from English to French-Canadian, and (3) pre-testing the items for understanding and cultural relevance.³¹ These steps were followed by the content validation³² of the adapted French-Canadian version.

For the conceptual equivalence assessment, a cognitive interview³¹ was conducted with a purposive sample of experts (i.e., trauma nurses, critical care physicians, neurosurgeons, SLTs, nutritionists, trauma decisions makers) who understood and spoke English fluently, to determine the acceptability and relevance of the items to the targeted culture and neurotrauma populations. Experts had to answer the following questions. What does the item mean to you? How relevant is this item or step as a criterion to screen for dysphagia in moderate-severe TBI and cervical SCI patients? What words can be used in your own language to express the idea conveyed in this item?

The adapted SSA tool was translated from English to French-Canadian independently by a researcher and an SLT who were bilingual and native French-Canadian speakers, to ensure the accuracy of clinical conceptual equivalence.³¹ Afterwards, these two French-Canadian speakers met with three other experts from the advisory committee to review the two initial French-Canadian translated versions, discuss discrepancies, and suggest alternative wording for the items to reach optimal conceptual equivalence. There was no back translation of the adapted French-Canadian version of the SSA tool into English, as the items are pathophysiological concepts described in simple medical terminology.³¹

The adapted French-Canadian version of the SSA tool was then pre-tested with the other experts from the advisory committee and with French-Canadian monolingual nurses working in neurotrauma. They were asked to complete the dysphagia screening using the French-Canadian version of SSA tool, and to provide suggestions to improve the clarity of items that they deemed unclear as well as recommendations to improve their cultural relevance.³¹

A form listing the adapted French-Canadian version of the SSA items was developed for content validation based on expert judgement. We recruited trauma nurses and other healthcare providers involved in dysphagia screening (i.e., physicians and SLTs) to conduct content validation. Eligible healthcare providers needed to be considered professionally competent; that is, to have worked for at least 2 years full time³⁴ in neurotrauma. We aimed to recruit 10–20 healthcare providers as per recommendations to evaluate the relevance of the screening items using a four-point descriptive scale (4 = very relevant and 1 = not relevant). Healthcare providers' sociodemographic information (e.g., profession, last degree obtained, years of experience with the populations) was also collected. Content validation was established by calculating the content validity index (CVI) for each of the items (I-CVI) and for the total scale (S-CVI with all items combined). The CVI rates the relevance of individual items (I-CVI) and of the items taken all together (S-CVI) according to content experts, ranking them as relevant or highly relevant (i.e., scores of 3 and 4).³² Content validation was confirmed in the presence of I-CVI ≥0.75 for individual items³⁵ and ≥0.90 for the total scale (S-CVI).³⁶

Examination of the inter-rater reliability and criterion-concurrent validation of the adapted French-Canadian version of the SSA tool

Participants

Patients were recruited in trauma critical care units (i.e., intermediate care and intensive care units [ICU]) using a consecutive sampling to test inter-rater reliability and criterion validity. The inclusion criteria were: (1) being >18 years of age, (2) the presence of moderate (Glasgow Coma Score 9–12) or severe (Glasgow Coma Score 3–8)³⁷ TBI as determined by an intensivist or the TBI consulting physician, or the presence of a complete (American Spinal Cord Injury Association [ASIA]-A) or incomplete cervical (ASIA B, C, D) SCI³⁸ diagnosed in the emergency unit or during hospitalization, and (3) being extubated. Patients with a tracheotomy, maxillofacial trauma, or a past history of dysphagia were excluded, because those conditions usually require a complete assessment of swallowing ability by a trained specialist.

High sensitivity and specificity (0.97 and 0.90, respectively) have been reported for the SSA in stroke patients.²⁶ Based on sample size recommendations to determine the sensibility and specificity of a test, we aimed to recruit 70 patients, given a prevalence of dysphagia of 0.50, a sensitivity and specificity of 0.90, and a minimal acceptable lower confidence level of 0.80.^39
–41

Data collection

The psychometric properties of the adapted French-Canadian version of the SSA tool were examined using different validation strategies. Patients' sociodemographic data were collected, as well as clinical data that might play a role in dysphagia, including the type of neurotrauma injury and duration of intubation.^4,6,7,11

Inter-rater reliability

To determine inter-rater reliability, two nurses screened each patient: one bedside nurse from the trauma intermediate care unit or ICU and one research nurse. Their results were noted on the SSA form. Participating nurses received a 3-h training session on the physiological mechanisms involved in dysphagia, the clinical signs of dysphagia, how to use the adapted SSA tool, and under which conditions the patient could resume feeding. These nurses needed to score at least 80% on a written test to be certified for dysphagia screening. The nurse from the research team (clinical nurse in the trauma program), also received training for dysphagia screening. Both nurses assessed the patient at the same time, independently of each other. Nurses involved in the research project were informed that their result; that is, whether the patient was dysphagic or not, was not to be shared at the time of the assessment. Instead, the two dysphagia screening test results were placed in an envelope and sent to the research team.

Criterion-concurrent validation

For criterion-concurrent validation, the dysphagia screening results from the bedside nurses were compared with those reported by a SLT on the research team who had 10 years' experience in the targeted neurotrauma populations. The dysphagia assessment by the SLT included: history of dysphagia, patient behavior, ability to sit, respiratory assessment, ability to manage secretions, peripheral oral mechanisms, communication, coughing and gaging reflexes, and swallowing processes. SLTs are healthcare providers with specialized training in the assessment of swallowing function. Therefore, their evaluation was used as the standard clinical reference criterion. The SLT's assessment needed to be completed within 4 h after the nurse's screening to avoid fluctuations in the patient's condition and to ensure study feasibility. Likewise, the SLT was blinded to the screening result.

Data analysis

Data were analyzed using SPSS version 28.0 software (Statistical Package for the Social Sciences v28, IBM, Armonk, NY, USA). Sociodemographic and clinical data were analyzed by calculating proportions and mean scores with standard deviations. Weighted kappa coefficients along with 95% confidence intervals (CI) were computed to evaluate the inter-rater reliability of the adapted SSA tool between the two raters. A coefficient >0.60 was considered acceptable.⁴² Criterion-concurrent validation of the SSA tool was evaluated using sensitivity, specificity, positive and negative predictive values (PPV and NPV) as well as test accuracy analysis. A sensitivity and specificity ≥0.80 established clinical criterion validation.⁴³ Inter-rater reliability and criterion-concurrent validation analyses were performed using the combined sample (i.e., TBI and SCI groups). We also stratified data according to each group.

Evaluation of the clinical utility of the adapted French-Canadian version of the SSA tool

Participants

Trauma critical care nurses were recruited using a purposing sampling method. They needed to have used the SSA tool at least three times in moderate-severe TBI or cervical SCI patients to be eligible to participate.

Data collection

Clinical utility (i.e., healthcare providers' perspectives about the usefulness, benefits, and drawbacks of a new tool for their everyday practice) of the adapted French-Canadian version of the SSA tool⁴⁴ was assessed by healthcare providers according to the following domains: appropriateness (i.e., evidence of effectiveness, benefits associated with use), accessibility of the tool and associated materials (i.e., materials are readily available and costs are reasonable), applicability (i.e., ease of use, clear delineation of purpose), effectiveness (i.e., ease of use, clear delineation of the role of the stakeholders involved) and acceptability (i.e., professionals' belief that the tool provides benefits to users, acceptance of the tool's use by peers).⁴⁴ Clinical utility was rated on a five-point Likert scale (5 = strongly agree and 1 = strongly disagree). Nurses' sociodemographic information (i.e., number of years of experience with the target populations and number of times they had used the SSA tool) was also collected.

Data analysis

Proportions were calculated to determine the nurses' level of agreement with the items evaluating utility. The average of the proportions obtained for the items comprising each of the utility domain was also computed. To be considered clinically useful, the research team determined a priori that ≥80% of nurses had to state that they strongly agreed or agreed with the various items of the clinical utility questionnaire as well as with utility domains.

Results

Adaptation/translation, pre-testing, and content validation of the adapted French-Canadian version of the SSA tool

Eleven experts (three trauma nurses, two critical care physicians, one neurosurgeon, two SLTs, one nutritionist, two trauma decision makers) participated to a cognitive interview for the conceptual equivalence assessment. Following this step, the item “Is the patient able to breathe freely?” was changed to: “Is the patient able to breathe freely, with a stable O₂ concentration and SpO₂?” In addition, based on physicians' recommendations, details were added to the dysphagia screening procedure for patients with a high flow humidified nasal cannula (i.e., screen for dysphagia with the cannula and repeat the screening at the end of treatment). Likewise, based on recommendations from SLTs and empirical data,^45,46 another item was added to the swallowing assessment: using semi-solid and solid textures in addition to water at the time of dysphagia screening. The instruction to say “aaah” was also added after each dysphagia screening step to better assess voice quality after swallowing. Directives on which healthcare providers should be informed of the screening results and the timeline for screening after admission (i.e., within 24 h) and extubation (i.e., within 4–24 h) before initiating oral feeding were also added to the original version of the tool. Hence, the adapted French-Canadian version of the SSA tool included 12 items (Fig. S1).

Two experts from the advisory committee and three French monolingual bedside nurses from the trauma intermediate care unit pre-tested the adapted French-Canadian version of the SSA tool. Questions arose from nurses about how to conduct screening, but none of the items were modified at this stage.

A total of 17 healthcare providers evaluated the content relevance of the adapted French-Canadian version of the SSA. Among these, 76% (n = 12) were critical care nurses, 18% (n = 3) were SLTs (18%), and 12% (n = 2) were physicians. Of those, 88% (n = 15) had >5 years of experience with the targeted populations (Table S1). As outlined in Table 1, the S-CVI score was 0.97 for the adapted SSA tool, while the mean I-CVI score for individual items ranged from 0.82 to 1.0. Item 5 (Is the patient able to lick the upper top and lower lip?) had the lowest CVI mean score (0.82). These CVI results supported the content validation of the adapted French-Canadian version of the SSA tool, which was used to achieve objectives 2 and 3.

Table 1.

Content Validation of the Adapted French-Canadian Version of the SSA Tool

Step A: Pre-swallow screening	CVI
1. Is the patient awake and alert, or responding to speech?	1.0
2. Is the patient able to be positioned upright with head control?	1.0
3. Can the patient cough when asked to?	0.94
4. Is the patient able to maintain some control of saliva?	1.0
5. Is the patient able to lick the upper and lower lip?	0.82
6. Is the patient able to breathe freely, with a stable O₂ concentration and SpO₂?	1.0
7. Does the patient have a “WET”, “HOARSE”, or “SOFT” sounding voice or is she/he “VOICELESS”?	0.94

Step B: Screening for dysphagia
8. Give first teaspoonful of water (5mL) and ask the patient to say AAAH after having swallowed.	1.0
9. Give second teaspoonful of water (5 mL) and ask the patient to say AAAH after having swallowed.	1.0
10. Give third teaspoonful of water (5 mL) and ask the patient to say AAAH after having swallowed.	0.98
11. Give a small glass of water (125 mL) and ask the patient to say AAAH after having swallowed.	1.0
12. Give a serving of apple sauce followed by a cookie and ask the patient to say AAAH after having swallowed each of the textures.	1.0
S-CVI average	0.97

SSA, Standardized Swallowing Assessment; CVI: content validity index; S-CVI: scale-content validity index.

Inter-rater reliability and criterion-concurrent validation of the adapted French-Canadian version of the SSA tool

A total of 30 moderate-severe TBI and 30 cervical SCI patients were recruited for the examination of inter-rater reliability, of which 15 TBI and 17 SCI patients were assessed to determine the criterion-concurrent validation, a smaller sample size than that established a priori. Screening for dysphagia can occur at any time of the day, 7 days a week, and the research team was not always available when patients began oral feeding, which explains why fewer patients were enrolled in the project during the targeted time period. Patients' sociodemographic and clinical characteristics are presented in Table 2. A third of patients (35%, n = 21) had a moderate TBI and one quarter (25%, n = 15) presented an ASIA-C injury. The mean age was 57 years (standard deviation [SD]: 21). Most patients were intubated prior to dysphagia screening (86%, n = 50) and patients were mechanically ventilated for a mean of 57 h (SD: 94). Patients with moderate-severe TBI were intubated for a longer period than patients with cervical SCI (90 h, SD: 118 vs. 27 h, SD: 48).

Table 2.

Patients' Sociodemographic and Clinical Characteristics

Characteristics	Inter-rater reliability (n = 60)	Concurrent validation (n = 32)
Diagnostic, n (%)
Moderate TBI	21 (35%)	10 (31%)
Severe TBI	9 (15%)	5 (16%)
ASIA-A cervical SCI	5 (8%)	5 (16%)
ASIA-B cervical SCI	2 (3%)	1 (3%)
ASIA-C cervical SCI	15 (25%)	8 (25%)
ASIA-D cervical SCI	8 (13%)	3 (9%)
Age, mean (SD, min-max)
Both populations	57 (21, 18-93)	58 (20, 18-87)
Moderate-severe TBI	52 (22, 18-93)	56 (21, 18-80)
Cervical SCI	62 (19, 18-87)	61 (20, 18-87)
Intubation prior to dysphagia screening, n (%)
Both populations	50 (86%)^a	27 (90%)^b
Moderate-severe TBI	24 (86%)	10 (78%)
Cervical SCI	26 (87%)	17 (100%)
Intubation duration in hours, mean (SD, min-max)
Both populations	57 (94, 0-600)	78 (119, 0-600)
Moderate-severe TBI	90 (118, 0-600)	125 (160, 0-600)
Cervical SCI	27 (48, 0-235)	41 (58, 0-235)

Proportion calculated from 58 patients.

Proportion calculated from 30 patients.

TBI, traumatic brain injury; ASIA, American Spinal Cord Injury Association; SCI, spinal cord injury, SD, standard deviation.

Inter-rater reliability

The bedside nurses reported a 50% (n = 30) rate of dysphagia, whereas the research nurse reported a 58% (n = 35) rate among all the patients screened. The inter-rater reliability between the bedside nurses and the research nurse for all the patients reached a weighted kappa coefficient of 0.80 (95% CI 0.65–0.95). When stratified by group, the weighted kappa coefficient was 0.86 (95% CI 0.68–1.04) for moderate-severe TBI patients and 0.73 (95% CI 0.49–0.97) for cervical SCI patients.

Criterion-concurrent validation

The prevalence of dysphagia was 81% (n = 26) for both groups, 87% (n = 13) for moderate-severe TBI and 76% (n = 13) for cervical SCI patients. As detailed in Figure 1, the sensitivity of the adapted SSA tool, for both groups, was 85% and the specificity was 67%. When stratified by group, the sensitivity was 92% for moderate-severe TBI and 77% for cervical SCI, while the specificity reached 50% and 75%, respectively. Because the prevalence of dysphagia was significantly higher than that documented in previous studies conducted in TBI and SCI patients (i.e., ±80% vs. 50%) and that the PPV and NPV as well as the test accuracy are influenced by the prevalence of the disorder being assessed, a Bayes correction was applied to these tests.⁴³ As such, the PPV and NPV of the adapted SSA tool were 65% and 87% in TBI patients and 75% and 76% in SCI patients, respectively. The test accuracy scores showed that 71% of TBI patients and 77% of SCI patients were correctly classified as having or not having dysphagia with the adapted French-Canadian version of the SSA tool.

FIG. 1.

Results for the criterion-concurrent validation.

Clinical utility of the adapted French-Canadian version of the SSA tool

The clinical utility of the adapted French-Canadian version of the SSA tool was evaluated by 49 critical care nurses. Of these, 65% (n = 32) had >5 years of experience with the target populations. Also, 37% (n = 18) had used the adapted French-Canadian version of the SSA tool from 3 to 10 times in their practice, 47% (n = 23) had used it 10–20 times, and 16% (n = 8) had used it >20 times. The scores for each item and for the utility domains are presented in Table 3. Nurses reported being in complete agreement or in agreement at ≥80% with the majority of the clinical utility questionnaire items. The following items did not reach these levels of agreement: “the tool allows for more efficient use of specialized resources (e.g., SLTs)” (78%, n = 38), “the tool reduces the number of complications related to dysphagia” (74%, n = 36), “conducting standardized dysphagia screening is an accepted practice among my team members” (68%, n = 33), “there are rules that support dysphagia screening” (63%, n = 31), and “I think patients appreciate undergoing standardized dysphagia screening” (47%, n = 23). For the clinical utility domains, only acceptability was <80% (74%, n = 36) when considering the mean proportion of nurses who completely agreed and agreed with each item of this domain.

Table 3.

Frequency (%) of the Clinical Utility Evaluation

Items	Completely agree	Agree	Neutral	Disagree	Completely disagree
1: The standardized dysphagia screening tool screens for dysphagia.	19 (39%)	25 (51%)	3 (6%)	2 (4%)	-
2: The tool allows for a reduction in delays in the detection of dysphagia.	25 (51%)	18 (37%)	4 (8%)	2 (4%)	-
3: The tool helps determine whether or not it is appropriate to start oral feeding.	18 (37%)	26 (53%)	3 (6%)	1 (2%)	-
4: The tool allows rapid resumption of oral feeding.	19 (39%)	24 (49%)	4 (8%)	2 (4%)	-
5: The tool reduces the number of complications related to dysphagia (e.g., pulmonary aspiration, pneumonia, malnutrition).	16 (33%)	20 (41%)	10 (20%)	3 (6%)	-
6: The tool integrates well into nursing practice.	13 (27%)	30 (61%)	4 (8%)	2 (4%)	-
7: The tool allows for more efficient use of specialized resources (e.g., speech language therapist).	19 (39%)	19 (39%)	8 (16%)	2 (4%)	1 (2%)

Appropriateness domain, mean%	38%	47%	10%	4%	1%
8: The tool is easily accessible.	27 (55%)	19 (39%)	1 (2%)	1 (2%)	-
9: The equipment required to conduct standardized dysphagia screening is easily accessible.	24 (49%)	21 (43%)	4 (8%)	-	-
10: Standardized screening for dysphagia is a low-cost practice	27 (55%)	17 (35%)	1 (2%)	2 (4%)	-

Accessibility domain, mean %	53%	39%	4%	2%	-
11: The tool is easy to use.	27 (55%)	17 (35%)	4 (8%)	-	-
12: The steps for standardized dysphagia screening are appropriate.	22 (45%)	20 (41%)	5 (10%)	-	1 (2%)
13: The role of each professional in standardized dysphagia screening is clear.	23 (47%)	18 (37%)	6 (12%)	1 (2%)

Practicability domain, mean %	49%	38%	10%	1%	1%
14: I encounter barriers in implementing standardized dysphagia screening.	1 (2%)	10 (20%)	11 (22%)	21 (43%)	5 (10%)
15: I had appropriate training in using the standardized dysphagia screening tool.	20 (41%)	24 (49%)	4 (8%)	-	-
16: Standardized screening for dysphagia is essential for the well-being of patients.	22 (45%)	20 (41%)	6 (12%)	-	-
17: I think patients appreciate having a standardized dysphagia screening done on them.	8 (16%)	15 (31%)	22 (45%)	2 (4%)	1 (2%)
18: There are legal rules supporting the practice of dysphagia screening.	11 (22%)	20 (41%)	16 (33%)	1 (2%)	-
19: Conducting standardized dysphagia screening is an accepted practice among my team members.	13 (27%)	20 (41%)	10 (20%)	5 (10%)	-
20: I feel comfortable conducting the standardized dysphagia screening.	23 (47%)	19 (39%)	6 (12%)	-	-
Acceptability domain, mean %	33%	41%	18%	5%	1%

Discussion

In this psychometric study, we adapted and translated the SSA tool into French-Canadian, and its content was validated by healthcare providers involved in dysphasia screening. Then, we examined its inter-rater reliability, criterion-concurrent validation, and clinical utility in moderate-severe TBI and cervical SCI patients. The inter-rater reliability of the tool's scores between trained bedside nurses and the research nurse was substantial in both these groups. Sensitivity met the threshold to support criterion validity in moderate-severe TBI and cervical SCI patients, but specificity did not, especially in TBI patients. Nevertheless, the NPV was high in the latter population, decreasing the probability of identifying patients as positive for dysphagia when they are not. At the same time, a quarter of moderate-severe TBI and cervical SCI patients could be incorrectly classified as having or not having dysphagia when using the adapted SSA tool. Finally, most critical care nurses considered the adapted SSA tool to be clinically useful, although consensus was not as strong when it came to the acceptability of dysphagia screening.

So far, most studies on the psychometric properties of dysphagia screening tools have focused on sensitivity and specificity, assessing the ability of these tools to identify patients at risk for dysphagia while ruling out those who are not.²³ This is meant to reduce the risk of adverse events that can occur when an individual with swallowing difficulties is not identified, and to avoid any delays in oral feeding and discomfort in those who are unlikely to have dysphagia. The inter-rater reliability of the original version of the SSA tool has not been previously examined.^28,29 However, the few studies published on the inter-rater reliability of dysphagia screening tools, with the exception of those involving a complex dysphagia protocol (e.g., Bedside Swallowing Assessment), had satisfactory scores, similar to those reported in our study.²³ These findings demonstrate that the screening tool must be easy to administer (e.g., clear items, not too many steps), but must also include items that are relevant for the target population. In this regard, content validation is essential as an initial step to enhance the validity of an instrument and to ensure that there are no clinical inconsistencies in its application.⁴⁸ Thus, the strong content validity of the adapted SSA may have contributed to the good inter-rater reliability reported in this study.

Regarding criterion-concurrent validation, a recent study⁴⁹ using a homemade dysphagia screening tool developed for patients with cervical spine injuries demonstrated higher sensitivity and specificity scores than those obtained in our study. However, in this previous study, <30% of the patients had neurological injuries, and no subgroup analysis was performed, making comparison with our results difficult. Compared with the original SSA tool scores obtained from stroke patients, the sensitivity score of the adapted French-Canadian version of the SSA tool was similar in TBI patients but lower in SCI patients. The specificity score of the adapted SSA tool was much lower in both neurotrauma groups than the specificity for the original version of the SSA tool with stroke patients, but comparable to other dysphagia screening tools.¹⁹ The brain structures that are impaired and that affect swallowing ability in stroke and TBI patients share several similarities,⁶ which may explain the comparable sensitivity scores in these two populations. However, the mechanisms involved in swallowing are different in SCI patients, specifically when it comes to the paralysis of the coughing muscles that help protect the airway and the neuronal damage associated with the injury and surgery.³ These mechanisms could inhibit responses associated with positive dysphagia screening such as coughing, choking, or a hoarse voice after swallowing,⁴ making dysphagia screening more complex, and potentially explaining the lower sensitivity score in this population. Criterion-concurrent validation of the original version of the SSA tool was assessed retrospectively, with a chart review of the dysphagia screening completed by the nurses and the swallowing disorder assessment completed by SLTs, or by the videofluroscopic swallowing assessment.²⁶ It is therefore possible that healthcare providers were not completely blinded to the different evaluations being conducted, which could have been a contributing factor in the diagnostic accuracy testing, thereby influencing the high specificity scores obtained with the original version of the SSA tool. It should also be noted that the nurses and the SLT in our study identified very few patients without dysphagia, particularly among TBI patients, which may have contributed to the low specificity. Only a subsample of patients was evaluated for criterion-concurrent validation, and dysphagia was much more prevalent in this subsample than in the total sample. Coordinating the screening nurse and the SLT was challenging, and the SLT may only have been called by the nurses after positive dysphagia screening or to confirm dysphagia when in doubt. Considering the high prevalence of dysphagia identified by nurses and SLTs, the confidence intervals were acceptable for sensitivity, but close to 200 patients would have been required to estimate specificity with the same level of accuracy.²⁸

Up to now, little was known about the clinical utility of dysphagia screening tools from the perspectives of healthcare providers. Our study provides new insights into this process, showing that nurses' acceptance of dysphagia screening could be a concern for teams implementing this practice. Tool implementation involving several disciplines for a clinical condition with a high risk of complications can generate misgivings among healthcare providers. Close multidisciplinary collaboration is therefore recommended for the implementation of a dysphagia screening tool in order to propose solutions in a timely manner and improve the comfort level of the healthcare providers involved.^19,45 Further, other validation studies of dysphagia screening tools in neurotrauma populations may also promote greater professional agreement on their utility in practice.

Study strengths and limitations

This study fills a research gap and addresses an important clinical need. Indeed, although early screening for dysphagia has been recommended for moderate-severe TBI and cervical SCI patients, none of the available tools underwent psychometric testing in these populations prior to this study. Further, this is the first study to report results on the perceived clinical utility of a dysphagia screening tool. Additionally, rigorous methods were used, with recognized clinical reference standards, to determine the reliability and validity of the tool.

However, several limitations must be considered. First, the reliability of the adapted French-Canadian version of the SSA tool was tested with the participation of one research nurse. Therefore, the results obtained do not support inter-rater reliability among the bedside nurses themselves. However, this limitation is mitigated by the fact that >20 bedside nurses were involved in the inter-rater reliability assessment along with the research nurse. Second, because we wanted to reduce the risk associated with multiple tests in patients who might present with dysphagia, the research nurse and bedside nurses screened concurrently, instead of one after the other, which may have contributed to better agreement between the two. Nevertheless, the nurses were told not to share their respective screening results during the study period. Third, dysphagia assessed by a SLT was used as the clinical reference criterion for concurrent validation. Although the assessment was rigorously performed by a specialist, the use of a gold standard test, such as the flexible endoscopic evaluation of swallowing,⁴⁶ could have increased the level of confidence for our findings. The use of this test and a larger sample size could also allow for a better estimate of the accuracy of the adapted SSA tool. Fourth, we did not assess the predictive validity of the adapted French-Canadian version of the SSA tool. Therefore, its ability to predict adverse outcomes, such as aspiration pneumonia, has yet to be demonstrated. Finally, our study assessed the psychometric qualities of the adapted SSA tool within two specific neurotrauma groups. Therefore, further validation of the tool is required in other vulnerable populations (e.g., elderly with other types of injuries, patients with maxillofacial trauma),^50,51 as validity is not transferable from one population to another.⁴³

Conclusion

This study supported the content validation and inter-rater reliability of the adapted French-Canadian version of the SSA tool in moderate-severe TBI and cervical SCI patients. Sensitivity was acceptable in both neurotrauma groups, demonstrating the tool's ability to correctly identify patients with dysphagia and supporting its use by trained healthcare professionals shortly after patient admission to trauma critical care units or after extubation. Specificity was lower, particularly in moderate-severe TBI patients, indicating the need for more vigilance in the presence of a negative dysphagia screening in this group, which may require dysphagia evaluation by a specialist. However, the specificity results and the screening accuracy of the adapted French-Canadian version of the SSA tool in neurotrauma and other trauma populations remain to be confirmed in a larger validation study that could also include a gold standard assessment tool for dysphagia. Regardless of future validation procedures of the adapted French-Canadian version of the SSA tool, support from the multidisciplinary care team involved seems paramount to the successful implementation of dysphagia screening by nurses and to improve acceptability.

Authors' Contributions

Mélanie Bérubé secured funding for the project, was responsible for conception and design, and did most of the data analysis and interpretation. She drafted the manuscript, revised it multiple times, gave final approval and agreed to act as guarantor of the work. Simon Ouellet contributed to data analysis. He critically revised the manuscript, gave his final approval, and agreed to act as guarantor of the work. Valérie Turcotte contributed to data collection and interpretation. She critically revised the manuscript, gave her final approval, and agreed to act as guarantor of the work. Annick Gagné contributed to data collection and interpretation. She critically revised the manuscript, gave her final approval, and agreed to act as guarantor of the work. Céline Gélinas contributed to the study design, data analysis, and data interpretation. She critically revised the manuscript, gave her final approval, and agreed to act as guarantor of the work.

Funding Information

This study was funded by the Institut national d'excellence en santé et en services sociaux (no grant number provided) and the Trauma Program of the Hôpital du Sacré-Coeur de Montréal (no grant number provided).

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Supplementary Material

Supplementary Figure S1

Supplementary Table S1

References

Matsuo

, Palmer

. Anatomy and physiology of feeding and swallowing: normal and abnormal. Phys Med Rehabil Clin N Am, 2008; 19(4):691–707; doi: 10.1016/j.pmr.2008.06.001

Dodds

, Stewart

, Logemann

. Physiology and radiology of the normal oral and pharyngeal phases of swallowing. AJR Am J Roentgenol, 1990; 154(5):953–963; doi: 10.2214/ajr.154.5.2108569

Adkins

, Takakura

, Spiegel

BMR

, et al. Prevalence and characteristics of dysphagia based on a population-based survey. Clin Gastroenterol Hepatol, 2020; 18(9):1970–1979.e2; doi: 10.1016/j.cgh.2019.10.029

McRae

, Morgan

, Wallace

, et al. Oropharyngeal dysphagia in acute cervical spinal cord injury: a literature review. Dysphagia, 2023; 38(4):1025–1038; doi: 10.1007/s00455-022-10535-0

Iruthayarajah

, McIntyre

, Mirkowski

, et al. Risk factors for dysphagia after a spinal cord injury: a systematic review and meta-analysis. Spinal Cord, 2018; 56(12):1116–1123; doi: 10.1038/s41393-018-0170-3

Takizawa

, Gemmell

, Kenworthy

, Speyer

. A systematic review of the prevalence of oropharyngeal dysphagia in stroke, Parkinson's disease, Alzheimer's disease, head injury, and pneumonia. Dysphagia, 2016; 31(3):434–41; doi: 10.1007/s00455-016-9695-9

Borders

, Gibson

, Grayev

, et al. Predictors of dysphagia in critically injured patients with neck trauma. J Crit Care, 2018; 44:312–317; doi: 10.1016/j.jcrc.2017.12.004

Hayashi

, Fujiwara

, Ariji

, et al. Mechanism of dysphagia after acute traumatic cervical spinal cord injury. J Neurotrauma, 2020; 37(21):2315–9; doi: 10.1089/neu.2020.6983

Alhashemi

. Dysphagia in severe traumatic brain injury. Neurosciences (Riyadh), 2010; 15(4):231–236

10.

Yan

, Jiang

, Liu

, et al. Evidence-based bundled care for patients with dysphagia after severe traumatic brain injury: a randomized controlled trial. Am J Transl Res, 2021; 13(7):7819–7828.

11.

Brown

CVR

, Hejl

, Mandaville

, et al. Swallowing dysfunction after mechanical ventilation in trauma patients. J Crit Care, 2011; 26(1):9–108; doi: 10.1016/j.jcrc.2010.05.036

12.

Shem

, Castillo

, Wong

, et al. Dysphagia and respiratory care in individuals with tetraplegia: incidence, associated factors, and preventable complications. Top Spinal Cord Inj Rehabil, 2012; 18(1):15–22; doi: 10.1310/sci1801-15

13.

Chaw

, Shem

, Castillo

, et al. Dysphagia and associated respiratory considerations in cervical spinal cord injury. Top Spinal Cord Inj Rehabil, 2012; 18(4):291–9; doi: 10.1310/sci1804-291

14.

Hinchey

, Shephard

, Furie

, et al. Formal dysphagia screening protocols prevent pneumonia. Stroke, 2005; 36(9):1972–1976; doi: 10.1161/01.STR.0000177529.86868.8d

15.

Starmer

, Hillel

, Akst

, et al. Dysphagia, short-term outcomes, and cost of care after anterior cervical disc surgery. Dysphagia, 2014; 29(1):68–77; doi: 10.1007/s00455-013-9482-9

16.

Abugroun

, Nayyar

, Abdel-Rahman

, Patel

. Impact of malnutrition on hospitalization outcomes for older adults admitted for sepsis. Am J Med, 2021; 134(2):221–226; doi: 10.1016/j.amjmed.2020.06.044

17.

Norman

, Pichard

, Lochs

, Pirlich

. Prognostic impact of disease-related malnutrition. Clin Nutr, 2008; 27(1):5–15; doi: 10.1016/j.clnu.2007.10.007

18.

Wilson

, Lunati

, Grabel

, et al. Hypoalbuminemia Is an independent risk factor for 30-day mortality, postoperative complications, readmission, and reoperation in the operative lower extremity orthopaedic trauma patient. J Orthop Trauma, 2019; 33(6):284–91; doi: 10.1097/BOT.0000000000001448

19.

Goiburu

, Jure Goiburu

, Bianco

, et al. The impact of malnutrition on morbidity, mortality and length of hospital stay in trauma patients. Nutrición Hospitalaria; 2006. Available from: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S0212-16112006000800007 [Last accessed: 10/13/2021 ].

20.

Guihan

, Bombardier

. Potentially modifiable risk factors among veterans with spinal cord injury hospitalized for severe pressure ulcers: a descriptive study. J Spinal Cord Med, 2012; 35(4):240–250; doi: 10.1179/2045772312Y.0000000016

21.

See

, Peng

, Phua

, Sum

, Concepcion

. Nurse-performed screening for postextubation dysphagia: a retrospective cohort study in critically ill medical patients. Crit Care (London, England), 2016; 20(1):326; doi: 10.1186/s13054-016-1507-y

22.

Patel

, Krishnaswami

, Steger

, et al. Economic and survival burden of dysphagia among inpatients in the United States. Dis Esophagus, 2018; 31(1):1–7; doi: 10.1093/dote/dox131

23.

Donovan

, Daniels

, Edmiaston

, et al. Dysphagia screening: state of the art: invitational conference proceeding from the State-of-the-Art Nursing Symposium, International Stroke Conference 2012. Stroke, 2013; 44(4):24–31; doi: 10.1161/STR.0b013e3182877f57

24.

Bayley

, Lamontagne

, Kua

, et al. Unique features of the INESSS-ONF Rehabilitation Guidelines for Moderate to Severe Traumatic Brain Injury: Responding to Users' Needs. J Head Trauma Rehabil, 2018; 33(5):296–305; doi: 10.1097/HTR.0000000000000428

25.

McRae

, Smith

, Beeke

, Emmanuel

. Development of a swallowing risk screening tool and best practice recommendations for the management of oropharyngeal dysphagia following acute cervical spinal cord injury: an international multi-professional Delphi consensus. Disabil Rehabil, 2022; 44(26):8311–24; doi: 10.1080/09638288.2021.2012607

26.

Consortium for Spinal Cord Medicine. Early Acute Management in Adults with Spinal Cord Injury: A Clinical Practice Guideline for Health-Care Providers. Consortium for Spinal Cord Medicine: Washington, DC; 2008.

27.

American-Speech-Language-Hearing-Association. Preferred Practice Patterns for the Profession of Speech-Language Pathology. American Speech-Language-Hearing Association: Rockville, MD; 2004.

28.

Perry

. Screening swallowing function of patients with acute stroke. Part one: identification, implementation and initial evaluation of a screening tool for use by nurses. J Clin Nurs, 2001; 10(4):463–473; doi: 10.1046/j.1365-2702.2001.00501.x

29.

Perry

. Screening swallowing function of patients with acute stroke. Part two: detailed evaluation of the tool used by nurses. J Clin Nurs, 2001; 10(4):474–81; doi: 10.1046/j.1365-2702.2001.00502.x

30.

Jiang

, Fu

, Wang

, et al. Validity and reliability of swallowing screening tools used by nurses for dysphagia: a systematic review. Ci Ji Yi Xue Za Zhi, 2016; 28(2):41–8; doi: 10.1016/j.tcmj.2016.04.006

31.

Sidani

, Guruge

, Miranda

, et al. Cultural adaptation and translation of measures: an integrated method. Res Nurs Health, 2010; 33(2):133–143; doi: 10.1002/nur.20364

32.

Polit

, Beck

. The content validity index: Are you sure you know what's being reported? critique and recommendations. Res Nurs Health, 2006; 29(5):489–497; doi: 10.1002/nur.20147

33.

Ellul

, Barer

, Fall

. Improving detection and management of swallowing problems in acute stroke: a multi-centre study. Cerebrovasc Dis, 1997; 7(Suppl. 4):18.

34.

Benner

PE.

From Novice to Expert: Excellence and Power in Clinical Nursing Practice. Addison–Wesley Pub. Company, Nursing Division: Menlo Park, CA; 1984.

35.

Polit

, Beck

, Owen

. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health, 2007; 30(4):459–467; doi: 10.1002/nur.20199

36.

Waltz

, Strickland

, Lenz

. Measurement in Nursing and Health Research. Springer Publishing: New York; 2017.

37.

Teasdale

, Jennett

. Assessment of coma and impaired consciousness: A Practical Scale. Lancet, 1974; 304(7872):81–84; doi: 10.1016/s0140-6736(74)91639-0

38.

Marino

, Barros

, Biering-Sorensen

, et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med, 2003; 26 (Suppl. 1):50–6; doi: 10.1080/10790268.2003.11754575

39.

Flahault

, Cadilhac

, Thomas

. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol, 2005; 58(8):859–862; doi: 10.1016/j.jclinepi.2004.12.009

40.

Buderer

NMF

. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med, 1996; 3(9):895–900; doi: 10.1111/j.1553-2712.1996.tb03538.x

41.

McHugh

. Interrater reliability: the kappa statistic. Biochem Med, 2012; 22(3):276–282.

42.

Landis

, Koch

. The measurement of observer agreement for categorical data. Biometrics, 1977:159–174.

43.

Fischer

, Bachmann

, Jaeschke

. A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med, 2003; 29(7):1043–1051; doi: 10.1007/s00134-003-1761-8

44.

Smart

. A multi-dimensional model of clinical utility. Int J Qual Health Care, 2006; 18(5):377–382; doi: 10.1093/intqhc/mzl034

45.

Marques

CHD

, De Rosso

ALZ

, Andre

. Bedside assessment of swallowing in stroke: Water tests are not enough. Dysphagia, 2009; 24(2):249–250; doi: 10.1310/tsr1504-378

46.

Ellis

, Hannibal

. Nursing swallow screens: why is testing water only not enough?. J Neurosci Nurs, 2013; 45(5):244–253; doi: 10.1097/JNN.0b013e31829d8b5b

47.

Boateng

, Neilands

, Frongillo

, Melgar-Quiñonez

. Best Practices For Developing And Validating Scales For Health, Social, And Behavioral Research: A Primer. Front Public Health, 2018; 6:149; doi:10.3389/fpubh.2018.00149

48.

Haynes

, Richard

DCS

, Kubany

. Content validity in psychological assessment: A functional approach to concepts and methods. Psychol Assess, 1995; 7(3):238–247; doi: 10.1037/1040-3590.7.3.238

49.

Posillico

, Golob

, Rinker

, et al. Bedside dysphagia screens in patients with traumatic cervical injuries: an ideal tool for an under-recognized problem. J Trauma Acute Care Surg, 2018; 85(4):697–703; doi: 10.1097/TA.0000000000002035

50.

Kregel

, Attia

, Pedroza

, et al. Dysphagia is associated with worse clinical outcomes in geriatric trauma patients. Trauma Surg Acute Care Open, 2022; 7(1):e001043; doi: 10.1136/tsaco-2022-001043

51.

Solomon

, Dietsch

, Dietrich-Burns

. Predictors of swallowing outcomes in patients with combat-injury related dysphagia. J Trauma Acute Care Surg, 2020; 89(2S Suppl 2):S192–S199; doi: 10.1097/TA.0000000000002623

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB

0.01 MB