Validity,inter-rater reliability,and feasibility of the Chelsea Physical Assessment Tool for assessing physical function in post-acute COVID-19 patients: A cross-sectional study

Abstract

BACKGROUND:

Various tools have been created to measure physical function during intensive care unit (ICU) stay and after ICU discharge, but those have not been validated in coronavirus 2019 (COVID-19) patients. There is a need for a reliable, valid and feasible tool to define the rehabilitation needs of post-ICU COVID-19 patients entering the acute wards and then rehabilitation clinics.

OBJECTIVE:

This study aims to investigate the validity, inter-rater reliability and feasibility of Chelsea Physical Assessment Tool (CPAx) in assessing the functional status of COVID-19 patients after discharge from the ICU.

METHODS:

Demographic and clinical characteristics of the patients were recorded. Patients were evaluated using the modified Medical Research Council (MRC) dyspnea scale, Functional Oral Intake Scale, Glasgow Coma Scale, CPAx, Barthel Index, Katz Index and MRC sum score, measurements of grip strength obtained by dynamometer, the 5 time sit-to-stand test and 30 seconds and sit-to-stand test. CPAx and the other functional assessment tools were administered to 16 patients within 48 hours following ICU discharge. For inter-rater reliability, another physiatrist independently re-assessed the patients. MRC sum score, Barthel and Katz indexes were used to assess construct validity of CPAx. The discriminative validity of CPAx was determined by its ability to differentiate between patients with and without ICU acquired muscle weakness based on MRC sum score. The intra-class correlation coefficients (ICC) were calculated to determine inter-rater reliability for total scores of the functional assessment tools. Cohen’s Kappa ( $\kappa$ ) coefficient and weighed Kappa ( $\kappa_{w}$ ) were calculated to determine inter-rater reliability of individual CPAx items. Ceiling and flooring effects were calculated by percentage frequency of lowest or highest possible score achieved. The number and percentages of the patients who were able to complete each tool were calculated to assess feasibility. The CPAx score was strongly correlated with MRC sum score (rho: 0.83), Barthel Index (rho: 0.87) and Katz Index (rho: 0.89) ( $p<$ 0.001) showing construct validity. Area under the ROC curve demonstrated that cut off score for CPAx was $\leqslant$ 12 to discriminate patients with MRC sum score $<$ 48, with a sensitivity and a specificity of 100% and 63%, respectively (AUC $=$ 0.859, $p<$ 0.001). ICC was high for CPAx, MRC sum score, Barthel and Katz indexes, Glasgow Coma Scale, and hand grip strength measurement, with the highest value observed for CPAx (ICC, 0.96; 95% confidence interval (CI), 0.71–0.98). $\kappa$ and $\kappa_{w}$ analysis showed good to excellent inter-rater reliability for individual CPAx items. No floor or ceiling effect was observed at CPAx while floor effect was observed at Barthel Index scores (25%) and Katz Index scores (37.5%). All patients could be evaluated using CPAx while less were physically able to complete the 5 time sit-to-stand, 30 seconds sit-to-stand tests ( $n=$ 4) and MRC sum score ( $n=$ 14).

CONCLUSION:

CPAx is a valid, reliable, and feasible tool to assess the physical functional state in COVID-19 patients following discharge from the ICU.

Keywords

COVID-19 post-intensive care physical performance function rehabilitation validity reliability

1. Introduction

Patients with coronavirus 2019 (COVID-19) who survive after intensive care unit (ICU) support are at risk for several physical, mental, and psychosocial problems such as ICU acquired weakness, dysphagia, chronic inflammation, hypovitaminosis D, neuromyopathy, delirium, anxiety, and functional limitations, known as post intensive care syndrome [1, 2, 3, 4]. Post-acute COVID-19 patients are often transferred from the ICU to post-acute specialized COVID-19 rehabilitation units or other hospital wards. In this context, an adequate transfer of information regarding patients’ condition and functional status at the time of ICU discharge during handover is mandatory for both ICU and rehabilitation physicians [5]. After an increasing number of patients who display functional impairments after COVID-19 discharged from the ICU, there has been a surge in the literature regarding the evaluation of the functional status of COVID-19 patients, as well as rehabilitation interventions that can improve the functional status of COVID-19 patients after discharge from the ICU [6, 7]. As a result, practical, feasible, valid, and reliable tools for assessing the functional status of COVID-19 patients discharged from the hospital should be used to make a definite conclusion about the efficacy of rehabilitation interventions.

The present assessment tools to assess the functional status of patients following ICU discharge could be categorized as strength test, walk tests, functional tests, and health related quality of life (HRQOL) assessment [8]. Among manual muscle strength tests, Medical Research Council (MRC) sum score is the suggested tool to screen ICU acquired weakness. Walk tests such as six-minute walk test or timed up and go test could be difficult to carry out as the patients could not even sit due to severe impairments during the early period [8, 9]. Additionally, these tests require space to perform and may require the management of several drips, drains, and oxygen delivery systems, which complicates test execution while the patient is walking and turning. Because of the increase in patient admissions due to the unexpected pandemic, hospital rooms have become crowded. This made it difficult to perform walk tests like these due to limited space in patient rooms.

Physical functional assessments include evaluation of muscle strength, mobility, physical and functional activity [8]. Among functional tests, the Physical Function in ICU Test (PFIT), Functional Status Score for the ICU and the Chelsea Critical Care Physical Assessment Tool (CPAx) are specifically developed to assess function after ICU stay. Barthel and Katz indexes are general tools to assess functional daily activities in various diseases [8]. According to a systematic review of 26 different outcome measures, the CPAx and the PFIT demonstrated the strongest clinimetric properties with established reliability, validity, and responsiveness; however, the PFIT has a greater floor effect than CPAx [10, 11]. The PFIT and CPAx may be more suitable for the assessment of patients who may never reach the ability to perform submaximal exercise tests [12].

Among these functional assessment tools, CPAx is easy to use in the clinical setting which covers respiratory function, coughing capacity, moving within the bed, sitting, and standing balance, sit to stand, transferring from bed to chair, stepping, and grip strength. Duration of assessment time is short and relatively few equipment (hand dynamometer for grip strength) is required. Many numbers of patients who were delivering non-invasive ventilation (e.g. continuous positive airway pressure) had to be discharged from the ICU to other outpatient wards because ICU beds had to be reserved for COVID-19 patients requiring invasive ventilation [13]. Although there are several tools that were constructed to measure physical function during ICU stay and following ICU discharge [14, 15, 16], they had not been validated in patients with COVID-19. To define rehabilitation needs of post-ICU COVID-19 patients who had been stepping down to acute wards and onwards into rehabilitation there was a need for a reliable, valid and feasible tool. We aimed to investigate the validity and the inter-rater reliability of CPAx in the assessment of the functional status of COVID-19 patients discharged from the ICU. Our secondary aim was to investigate feasibility of commonly used assessment tools for assessing physical function after ICU in COVID patients discharged from the ICU.

2. Materials and methods

2.1 Study design and setting

This cross-sectional observational study was conducted between March 01 and May 30 2021 at Koç University Hospital and Gaziosmanpaşa Training and Research Hospital among COVID-19 survivors discharged from the ICU to the wards. The study was approved by the Research Ethical Board of the University of Health Sciences (approval number: 245/2021). The study was registered on clinicaltrials.gov.tr (NCT04762056) and conforms to the STROBE checklist.

2.2 Study participants

Patients who were discharged from the ICU with a diagnosis of COVID-19 pneumonia were enrolled in this study. Inclusion criteria were: a) age $\geqslant$ 18 years; b) transferred from the ICU to hospital ward; c) able to follow at least 2 of the following commands from De Jonghe et al. [17] awakening criteria: “Open/close your eyes.”, “Look at me.”, “Open your mouth and stick out your tongue.”, “Nod your head.”, “Raise your eyebrows when I have counted to 5.”. Exclusion criteria were previous neurologic impairment, pregnancy and inability evaluate to grip muscle strength such as dominant extremity amputation.

2.3 Sample size calculation

MedCalc Statistical Software version 19.1 (MedCalc Software bv Ostend, Belgium) was used for sample size analysis. A sample size calculation was completed for a Pearson’s correlation with a two-sided test, alpha $=$ 0.05 and a power of 80% (1- $\beta$ ). At least 16 participants were required to find a statistically significant correlation of 0.65 between MRC sum score and CPAx score [18].

2.4 Outcome measures

Demographic characteristics including age, gender, body mass index, comorbidities, sequential organ failure assessment score (SOFA score) and acute physiology and chronic health evaluation (APACHE II) on ICU admission, length of ICU stay, history and duration of invasive mechanical ventilation, history of extra corporeal membrane oxygenation (ECMO), and presence of tracheostomy were recorded. Modified MRC dyspnea scale was used to quantify disability associated with dyspnea and Functional Oral Intake Scalewas used to evaluate the swallowing function. Patients’ level of consciousness and responsiveness were evaluated with Glasgow Coma Scale.

Chelsea Critical Care Physical Assessment Tool, Barthel Index, Katz Index and MRC sum score, grip strength measurement via dynamometer, the 5 time sit-to-stand test and 30 seconds sit-to-stand test were used to assess the physical status of the patients within 48 hours following ICU discharge by a physiatrist. Another physiatrist independently re-assessed the patients for inter-rater reliability.

2.4.1 Validity

To assess construct validity MRC sum score, Barthel Index, Katz Index and modified MRC dyspnea scale were used. The discriminant validity of CPAx was determined by its ability to discriminate between patients with and without ICU acquired muscle weakness based on MRC sum score.

2.4.2 Inter-rater reliability

To assess inter-rater reliability another physiatrist completed the CPAx and other assessments including MRC sum score, Barthel and Katz indexes on the same day without communicating the first rater and was blinded to the first rater’s score.

2.4.3 Feasibility

To assess and compare the feasibility of existing tools for functional status assessment of COVID-19 patients discharged to rehabilitation wards, the number and the percentages of the patients who were able to complete each tool were recorded.

2.4.4 Sequential Organ Failure Assessment Score (SOFA score)

Sequential Organ Failure Assessment Score is used to describe the degree of organ dysfunction from 0 (normal) to 4 (most abnormal) for six vital organs [19].

2.4.5 Acute Physiology and Chronic Health Evaluation (APACHE II)

APACHE II is a classification system representing a general measure of severity of disease which is based on age, initial values of 12 physiologic measurements and previous health status. It is also used to compare the efficacy of intensive care over time. The score ranges between 0 and 71 and higher score is correlated to subsequent risk of hospital death [20].

2.4.6 Glasgow Coma Scale

The Glasgow Coma Scale is a structured method for assessment of the level of consciousness via evaluating motor, verbal and eye responses, that enable comparison of the changes over time. These three components are combined in a sum score between 3 (worst) and 15 (best) [21].

2.4.7 Functional Oral Intake Scale

The Functional Oral Intake Scale is a tool used to document change in the functional eating abilities of the patients. Swallowing abilities were assessed with the FOIS, which consists of seven choices to describe the quality of oral intake, ranging from 1 (worst) to 7 (normal) as below [22]:

Tube Dependent (levels 1–3)

1.
No oral intake
2.
Tube dependent with minimal/inconsistent oral intake
3.
Tube supplements with consistent oral intake

Total Oral Intake (levels 4–7)

1.
Total oral intake of a single consistency
2.
Total oral intake of multiple consistencies requiring special preparation
3.
Total oral intake with no special preparation, but must avoid specific foods or liquid items
4.
Total oral intake with no restrictions

2.4.8 MRC dyspnea scale

Disability attributable to breathlessness was quantified using modified MRC dyspnea scale that stratifies severity of dyspnea in respiratory diseases, particularly COPD [14]. Dyspnea in daily living was evaluated by the modified MRC dyspnea scale which include five statements that describe almost the entire range of dyspnea from none (Grade 0) to almost complete incapacity (Grade 4) as follows [23]:

•
Grade 0: I only get breathless with strenuous exercise
•
Grade 1: I get short of breath when hurrying on level ground or walking up a slight hill
•
Grade 2: On level ground, I walk slower than people of the same age because of breathlessness, or I have to stop for breath when walking at my own pace on the level
•
Grade 3: I stop for breath after walking about 100 yards or after a few minutes on level ground
•
Grade 4: I am too breathless to leave the house or I am breathless when dressing

2.4.9 MRC sum score

Manual muscle strength of six muscle groups (shoulder abduction, elbow flexion, wrist extension, hip flexion, knee extension, and ankle dorsiflexion) was evaluated on both sides using MRC scale. MRC sum score was calculated by summing the scores. This score, ranging from 0 to 60, reliably identifies significant weakness ( $<$ 48) and even better in severe weakness ( $<$ 36) [15, 16].

2.4.10 Hand grip strength measurement

Hand grip strength was measured using a handheld dynamometer (JAMAR Plus+ electronic dynamometer, part number: 563213, serial number: 2019070814, Sutton-in-Ashfield, Nottinghamshire, UK) in the standardized recommended position by American Society of Hand Therapy (with the patient as seated as possible, with the elbow as close as to 90 ${}^{\circ}$ ) with a rest period of 20 seconds. The patient’s dominant hand was tested, with the patient as seated as possible, with the elbow as close as to 90 ${}^{\circ}$ . Three measurements were taken, and mean values were recorded [24].

2.4.11 CPAx

The CPAx is a bedside assessment tool firstly reported to measure physical morbidity in critical care population in 2013 and recommended for use in post-acute COVID-19 patients [18, 25]. It consists of 10 items (respiratory function, cough, moving within the bed, supine to sitting on the edge of bed, dynamic sitting, standing balance, sit to stand, transferring from bed to chair, stepping, and grip strength) rated on a 6-point scale from complete dependency (level $=$ 0) to independency (level $=$ 5). Therefore, the CPAx sum score ranges from 0 (worst condition) to 50 (best functioning/independence).

2.4.12 Barthel Index

The Barthel Index is a measure of performance in activities of daily living (ADL). This ordinal scale includes 10 items of mobility and self-care ADL, assessing the degree of physical assistance required and time taken to perform each item. Scores range from 0 to 100. A higher score reflects greater ability to function independently (Scores 0–20: “total” dependency, 21–61: “severe” dependency, 62–90: “moderate” dependency, 91–99: “slight” dependency, 100: “full” independency) [26].

2.4.13 Katz Index

The Katz Index of Independence in Activities of Daily Living, commonly referred to as the Katz ADL, assesses the ability to perform activities of daily living independently, in the six functions of bathing, dressing, toileting, transferring, continence, and feeding. Each item is scored yes/no for independence. A score of 6 indicates full function, 4 indicates moderate impairment, and 2 or less indicates severe functional impairment [27].

2.4.14 The 5 time sit-to-stand test

The 5 repetition sit-to-stand test evaluates the exercise capacity and lower limb strength and assesses the changes in exercise capacity in COPD [28]. Normative values of the 5-repetition sit-to-stand test are reported as 11.4 sec (60 to 69 years), 12.6 sec (70 to 79 years), and 14.8 sec (80 to 89 years) in healthy individuals [29].

2.4.15 30 seconds sit-to-stand test

The 30 seconds sit-to-stand test evaluates the muscular endurance or strength endurance and the functional status of the patients those at risk for mobility. The score ranges from 0 (cannot complete even one stand) to 20 or more in highly fit individuals. Less than 10 times sit to stand indicates lower extremity strength and can be used as a measure of lower body strength in community-residing older adults [28].

2.5 Statistical analysis

Statistical analysis was performed using SPSS Statistics for Windows, version 27.0. (IBM Corp., Armonk, NY, USA). Histogram, normality plots and Shapiro-Wilk normality test were performed to assess the distribution of the data. Descriptive data were presented as median, interquartile range (IQR), minimum, maximum, frequency and percentage.

For construct validity, the Spearman Rho correlation coefficients were used to evaluate the correlation between CPAx and Barthel, Katz indexes (Spearman Rho coefficients were classified as follows: $\pm$ 1 as “perfect”, above $\pm$ 0.7 as “strong”, between $\pm$ 0.69 and 0.3 as “moderate”, and below $\pm$ 0.29 as “weak”) [30]. To evaluate the discrimination ability of CPAx for muscle weakness (MRC sum score $<$ 48 scores) receiver operating characteristics (ROC) curve was performed and the area under curve (AUC) value has been evaluated [31]. The cut-off point of CPAx was determined by the maximum value of the Youden Index.

The intra-class correlation coefficient (ICC, range 0.00–1.00) were calculated to determine inter-rater reliability for total scores. ICC values less than 0.5 were considered as poor reliability, values between 0.5 and 0.75 were considered as moderate reliability, values between 0.75 and 0.9 were considered as good reliability, and values greater than 0.90 were considered as excellent reliability [30]. Cohen’s Kappa ( $\kappa$ ) coefficient and weighed Kappa ( $\kappa_{w}$ ) were calculated to determine inter-rater reliability of individual CPAx items [11]. Cohen’s K is a reliability index used for nominal and categorical data. K is a chance-corrected measure of agreement in which all disagreements are given equal weight. In contrast, the $\kappa_{w}$ takes into account the degree of disagreement among raters. $\kappa_{w}$ differentially weighs discrepancies between pairs of scores so that the further apart the two scores, the more effect that observation has on lowering the reliability [32]. Overall interpretations of the simple $\kappa$ and $\kappa_{w}$ statistic were based on the criteria described by Fleiss et al. [33, 34]. The level of reliability was defined as follows: $\kappa$ or $\kappa_{w}$ values of $<$ 0.40 reflect poor agreement; values of 0.40–0.75 reflect fair to good agreement, and values of $>$ 0.75 reflect excellent agreement [33, 34]. The ceiling and flooring effects were calculated by percentage frequency of lowest or highest possible score achieved. The ceiling and flooring effects of more than 15% were significant. The number and the percentages of the patients who were able to complete each tool were calculated to assess feasibility of the tools for functional status assessment of COVID-19 patients discharged from the ICU.

3. Results

Demographic, clinical characteristics, and baseline data of 16 patients with COVID-19 transferred from the ICU to the ward are presented in Table 1. The flowchart of the study participants is depicted in Fig. 1.

Table 1
Demographic, clinical characteristics and baseline data of the patients

Variable

Post-ICU COVID-19 survivors

(

n=

16)

Age (years)

63 (52–72) 28–84

Gender,

n

(%) Man Woman

10 (62.5%) 6 (37.5%)

BMI (kg/m

{}^{2}

)

28.8 (26.7–32.4) 19.9–35.9

Systemic comorbidities,

n

(%) HT DM Coronary artery disease Arrythmia Chronic pulmonary disease Cancer Chronic kidney disease

8 (50%) 4 (25%) 3 (19%) 2 (13%) 5 (31%) 2 (13%) 1 (6%)

APACHE II on ICU admission

8 (6–14) 2–28

SOFA on ICU admission

4 (2–5) 2–9

History of IMV,

n

(%)

11 (69%)

IMV duration, days

13 (7–17) 0–30

History of ECMO,

n

(%)

1 (6%)

LOS in ICU, days

22 (10–28) 2–49

Tracheostomy,

n

(%)

4 (25%)

Functional Oral Intake Scale,

n

(%) 1 No oral intake 2 Tube dependent with minimal/inconsistent oral intake 3 Tube supplements with consistent oral intake 4 Total oral intake of a single consistency 5 Total oral intake of multiple consistencies requiring special preparation 6 Total oral intake with no special preparation, but must avoid specific foods or liquid items 7 Total oral intake with no restrictions

5 (31%) 1 (6.3%) 1 (6.3%) 1 (6.3%) 1 (6.3%) 1 (6.3%) 7 (43.8%)

Glasgow Coma Scale total score Eyes 3 4 Verbal 2 5 Motor 6

15 (0) 11–15 1 (6.3%) 15 (93.8%) 1 (6.3%) 15 (93.8%) 16 (100%)

Modified MRC dyspnea scale, n (%) 1 2 3 4 5

1 (6%) 3 (19%) 2 (13%) 1 (6%) 9 (56%)

MRC sum score

<

48,

n

(%)

5 (31%)

CPAx

14.5 (10.5–36.0)

Barthel Index

17.5 (1.3–53.8)

Katz Index

1 (0–2)

Values are presented as median (interquartile range) and min – max, unless mentioned otherwise. BMI: body mass index, HT: hypertension, DM: diabetes mellitus, APACHE II: Acute Physiology and Chronic Health Evaluation II, ICU: intensive care unit, SOFA: Sequential Organ Failure Assessment, IMV: invasive mechanical ventilation, ECMO: extra corporeal membrane oxygenation, LOS: length of stay, MRC: Research Council, CPAx: Chelsea Critical Care Physical Assessment Tool.

Figure 1.

Flowchart of the study participants.

CPAx total score was strongly correlated with MRC sum score, Barthel and Katz indexes (Table 2).

Table 2

Correlations of Chelsea Critical Care Physical Assessment Tool with MRC sum score, Barthel Index and Katz Index

Variable	Rho	$p$
MRC sum score	0.832	$<$ 0.001
Barthel Index
Total score	0.876	$<$ 0.001
Feeding	0.658	0.006
Bathing	0.680	0.004
Grooming	0.672	0.004
Dressing	0.691	0.003
Bowel	0.735	0.002
Bladder	0.755	$<$ 0.001
Toilet use	0.653	0.006
Transfers	0.757	$<$ 0.001
Mobility	0.758	$<$ 0.001
Stairs	0.683	0.004
Katz Index
Total score	0.896	$<$ 0.001
Bathing	0.680	0.004
Dressing	0.680	0.004
Toileting	0.680	0.004
Transferring	0.680	0.004
Continence	0.466	0.069
Feeding	0.700	0.003

MRC: Medical Research Council.

Area under the ROC curve demonstrated that cut off score for CPAx was $\leqslant$ 12 to discriminate patients with MRC sum score $<$ 48 scores, with a sensitivity and a specificity of 100% and 63%, respectively (AUC $=$ 0.859, $p<$ 0.001) (Fig. 2).

Interclass correlation coefficients were high for CPAx, MRC sum score, Barthel and Katz indexes, Glasgow Coma Scale, and hand grip strength measurement, with the highest value observed for CPAx (Table 3). ICC of CPAx was calculated as 0.96 (95% CI, 0.71–0.98) indicating excellent inter-rater reliability. Each item of the CPAx demonstrated good to excellent inter-rater reliability (Table 4). $\kappa_{w}$ values for “respiratory” and “moving within the bed” were calculated as highest among the all items ( $\kappa_{w}=$ 1, $p=$ 0.0001; $\kappa_{w}=$ 0.85, $p=$ 0.0001, consecutively) indicating excellent agreement. “cough” and “supine to sitting on the edge of the bed” items were the two items with utmost lowest $\kappa_{w}$ value among all items ( $\kappa_{w}=$ 0.64, $p=$ 0.004; $\kappa_{w}=$ 0.62, $p=$ 0.004, consecutively) demonstrating good agreement. Floor effect was observed at Barthel Index scores (25%) and Katz score (37.5%) while no floor or ceiling effect was observed at CPAx.

Table 3

Inter-rater reliability results for tools assessing physical function after intensive care unit (ICU) discharge in COVID patients discharged from the ICU

Tool	Interclass correlation	95% CI
	coefficient (ICC)	Lower bound	Upper bound
CPAx	0.96	0.71	0.98
Barthel Index	0.93	0.77	0.98
Katz Index	0.89	0.62	0.97
MRC sum score	0.88	0.50	0.97
Glasgow Coma Scale	1	1	1
Grip strength measurement	0.97	0.87	0.99

CPAx: Chelsea Critical Care Physical Assessment Tool, MRC: Medical Research Council, CI: Confidence interval.

Table 4

Inter-rater agreement for the items of Chelsea Critical Care Physical Assessment Tool

Domain	Kappa	95% CI		$p$ value	Weighed	95% CI		$p$ value
		Lower bound	Upper bound		Kappa	Lower bound	Upper bound
Respiratory	1	1	1	0.0001	1	1	1	0.0001
Cough	0.46	0.64	0.83	0.012	0.64	0.38	0.91	0.004
Moving within the bed (rolling)	0.78	0.46	1	0.0001	0.85	0.64	1	0.0001
Supine to sitting on the edge of the bed	0.40	0.70	0.71	0.003	0.62	0.28	0.95	0.004
Dynamic sitting	0.63	0.35	1	0.0001	0.71	0.40	1.02	0.001
Standing balance	0.59	0.0001	1	0.004	0.76	0.38	1.14	0.001
Sit to stand	0.61	0.0001	1	0.0001	0.78	0.44	1.11	0.001
Transferring from bed to chair	0.57	0.057	0.872	0.002	0.79	0.54	1.05	0.001
Stepping	0.71	0.42	1	0.002	0.77	0.51	1.02	0.001
Grip strength	0.59	0.21	0.87	0.002	0.73	0.49	0.98	0.001

CI: Confidence interval.

Among the 16 patients, only 4 (25%) of the patients were physically able to complete the 5 time sit-to-stand, 30 seconds sit-to-stand tests. 2 (12.5%) of the patients were unable to complete the MRC sum score. All patients could perform grip strength test and CPAx.

4. Discussion

This cross-sectional study demonstrated that CPAx is a valid, reliable, and feasible tool with a high inter-rater reliability that can be used in the functional assessment of COVID-19 patients during the 48 hours following discharge from the ICU. CPAx showed a strong correlation with MRC sum score, which is an important screening tool for ICU-acquired weakness. The validity of CPAx as a functional assessment tool after ICU is further confirmed by its strong correlation with Barthel and Katz indexes which were previously found suitable for this population [35]. In addition, contrary to what is seen in the Barthel and Katz Indices, no ceiling and floor effects were observed in CPAx scores.

Figure 2.

Receiver operating characteristics (ROC) curve for Chelsea Critical Care Physical Assessment Tool to discriminate post-ICU COVID-19 patients with MRC sum score $<$ 48.

Validity is the degree that a measured result reflects the measured content. The more consistent the measured result is with the measured content, the higher is the validity [36]. The construct validity results including the correlation coefficient between the CPAx score and MRC-score in the present study were considered good ( $n=$ 0.832) and correspond to earlier studies by Zhang et al. ( $r=$ 0.60) [36], Eggmann et al. ( $r=$ 0.809) [19] and Corner et al. ( $r=$ 0.65) [37]. Functional Independence Measure (FIM), Barthel and Katz indexes assess the ability to perform ADL [35]. They are frequently used for clinical decision-making for rehabilitation plans. Also, unlike previous validation studies of CPAx, in the current study, its validity has been demonstrated by also measuring the correlation coefficient between CPAx, Barthel, and Katz indexes which are commonly used tools in rehabilitation settings for measuring functional independence. Similarly, Eggmann et al. has shown high correlation coefficient between the CPAx and FIM indicating good construct validity [19]. The clinical implication of these results is that the CPAx seems to be a valid tool in assessing the functional status of COVID-19 patients after discharge from the ICU. Reliability results for CPAx were detected as excellent in this study and were consistent with previous studies. [11, 19] Different from previous studies, comparative reliability of the existing tools in assessing function after ICU including MRC sum score, Barthel and Katz indexes, Glasgow Coma Scale, and hand grip strength measurement were calculated. CPAx showed the highest reliability result. Among items of CPAx, inter-rater reliability assessed by Kappa value was highest for the respiratory item. “Cough” and “supine to sitting on the edge of the bed” items showed fair to good interrater reliability. Inter-rater reliability of the respiratory item was also found to be one of the items with the highest reliability in previous studies [11, 19, 38]. Similar to the results of the present study, Milton et al. detected moderate agreement for cough and supine to sitting items and almost perfect agreement for respiratory function [38].

Contrary to Barthel and Katz indexes, items for respiratory function and cough in the CPAx provides important information regarding the respiratory status of the critically ill patients who are recently discharged from the ICU. In case of acute respiratory distress patients such as COVID-19, majority of the patients continue to suffer from respiratory symptoms, as 75% of our study population had MRC dyspnea 3 or greater. These items displayed good-excellent reliability between two raters. Similar to the results of the present study, inter-rater reliability of individual CPAx items was detected high in critically ill patients and in patients who discharged from the ICU [11, 19].

Another advantage of CPAx is the inclusion of hand grip strength, which is an important indicator for general health, sarcopenia, and mortality [39]. This item might increase the value of CPAx as a prognostic indicator. It might have a role in screening the patients at risk for ICU acquired weakness. In our study population, a cut off value of 12 points in CPAx was found 100% sensitive and 63% specific to identify patients with MRC sum scores lower than 48. Zhang et al. have found that the cut-off value for diagnosis of ICU acquired weakness for CPAx is 31.5 with a sensitivity of 87% and specificity of 77% [31]. In the present study cut off value of CPAx for ICU acquired muscle weakness were detected lower than the study by Zhang et al. [31] but sensitivity was 100%. Sensitivity refers to a test’s ability to identify an individual with disease as positive. The more sensitive a test the more likely to detect an individual with a positive test and thus the more accurate to be used in screening. Since it is important to screen and correctly define patients with ICU-acquired muscle weakness, a lower cut-off value with high sensitivity may have utility in the assessment of post-ICU COVID-19 patients.

In CPAx, there are no items for continence, toileting, feeding, dressing, bathing or grooming, although Barthel and Katz indexes question the level of independency in these activities of daily living. CPAx mainly concentrates on the physical mobility within the bed, transfer from bed to chair, sitting, standing, and stepping. As it was developed for use in ICU and during ICU stay, the primary activity of concern is mainly transfer and mobility after breathing. ADL care is already provided by the nurses in the ICU. During the early period following discharge from the ICU, physical mobility is still one of the main parameters both for physicians and caregivers [39].

The aforementioned assessment tools mainly consider physical function, considering the potential sequalae of COVID-19 in terms of functioning [40, 41, 42, 43, 44, 45]. Patients with post-intensive care syndrome are also at high risk for mental and psychological disorders which are not questioned in any of these scales. Future scales for post-ICU discharge should consider the mental and psychological aspects as well as the swallowing domain under the physical function. Functional assessments require awakening and appropriate mental health. Possible mental impairments in the early post-ICU period may interfere with the cooperation of the patients [46]. In the present study, to prevent the impact of a possible mental problem, the patient’s awakening and response to basic commands are checked before CPAx administration. Moreover, a high prevalence of swallowing disorders was reported in COVID-19 patients with acute respiratory distress syndrome after discharge from the ICU. As mentioned before, the involvement of swallowing status in the assessment tools would be a better representation of the functional status [47]. To the best of our knowledge, CPAx tool can be strengthened by adding an item related to cognitive and swallowing impairment.

In line with previous studies, we did not detect a floor or ceiling effect of CPAx [19, 37, 48]. This was previously explained by the superiority of CPax’s measurement features, including evaluation of both physical function (whole body activities and hand grip strength) and respiratory (ventilation, oxygenation, and secretion clearance) measures [49].

COVID-19 patients who are discharged alive from intensive care units (ICUs) suffer from intensive care syndrome which is a multisystem disability that encompasses rapid acute muscle wasting and associated impaired mobility; cognitive problems relating to impaired short-term memory and executive function; depression, anxiety and post-traumatic stress-disorder; and dysphonia and dysphagia in those with and without tracheostomies. It can be hard to assess the functional status of these patients due to physical and cognitive impairments because the existing tools to assess functional status of these patients may require awakening, appropriate cognitive and conscious status. Also, several of functional assessment tests can be impractical because they require space to perform and may require management of several drips, drains, and oxygen delivery systems while the patient is walking and turning that render the tests difficult to carry out. During COVID-19 pandemic, there was a surge of patients admitting to hospital intensive care units and stepping down to other wards of hospitals due to shortage of intensive care beds. Due to all aforementioned reasons, to plan personalized rehabilitation, the tools for assessing functional status must be practical and feasible. In our sample, CPAx was the most feasible assessment tool including assessment of wide range of disability related to post-intensive care syndrome. Very few of the patients could be physically and mentally able to complete function tests such as the 5 time sit-to-stand, 30 seconds sit-to-stand tests. Also, several of the patients could not complete MRC sum score which is a frequent assessment tool to screen ICU acquired muscle weakness.

Walk tests or tools for functional status assessment of critical care survivors which include walking or shuttling items may show ceiling or floor effects if the patients have extreme (very low or very high) levels of physical performance. For patients who demonstrate high levels of physical performance, stride length and speed may limit the distance walked resulting in a ceiling effect. In contrast, after discharge from the intensive care unit, greatly disabled patients may not be able to walk at all, resulting in a “floor effect”. For example, in an observational study ( $n=$ 69), almost 40% of patients were unable to walk or needed assistance 4 days after discharge from the ICU [8]. In the present study, CPAx did not show floor and ceiling effects. We were able to calculate ceiling or floor effect of neither 5 time sit-to-stand test nor 30 seconds sit-to-stand test because only two patients were physically able to perform these tests.

A specific scale has been proposed for COVID-19 to measure functional status after hospital discharge termed the post-COVID-19 functional status scale. It mainly defines the need of assistance at work and at home and questions the presence of depression and anxiety [50], considering that long COVID patients might be also affected by a severe fatigue affecting the other clinical manifestations [51]. Therefore, this scale does not seem to be appropriate for use in critically ill patients early after discharge from the ICU.

Another specific scale which has been proposed to measure functional status of the COVID-19 patients with post-intensive care syndrome is the Post- ICU Presentation Screen (PICUPS), which was developed by the National Post-Intensive Care Rehabilitation Collaborative formed by the British Society of Rehabilitation Medicine’s (BSRM’s) and The Intensive Care Society (ICS) to address the rehabilitation needs of post-ICU COVID-19 patients as well as of all intensive care unit survivors as patients discharge from the ICU and acute hospital care [6, 7, 52]. Furthermore, by using PICUPS, to support triage and handover of patients discharging from the ICU to acute wards into rehabilitation, to inform the immediate care plan on the acute ward, to identify problems that are likely to require further detailed assessment/evaluation by members of the multi-disciplinary team, and thus, to develop a personalized rehabilitation plan. The PICUPS tool consists of four domains: medical and essential care, breathing and nutrition, physical movement and communication, cognition and behavior. PICUPS domains were developed to target to assess whole post-intensive care syndrome related impairments. The PICUPS Plus also include additional items related to ICU-acquired dysphagia, dysphonia or upper airway dysfunction that helps identifying breathing, voice and swallowing impairments thereby triggering further evaluation and intervention and referral for speech and language therapy [52]. PICUPS tool has been shown to have strong robust scaling properties including face and content validity, utility, feasibility, structural validity and responsiveness but its reliability has not been studied. Similar to CPAx, PICUPS did not show floor or ceiling effect [7]. This scale has been proposed during patient recruitment of the present study. That is why the authors of the present study were not aware of it during study protocol planning. We have chosen CPAx for the assessment of post-ICU COVID-19 patients taking into account the short time required for being administered and the relatively minimal use of equipment and space [53]. It may take more time to complete The PICUPS than CPAx. The PICUPS has the potential for use in assessing the functional status of COVID-19 patients after discharge from the intensive care unit but comparative feasibility, validity and reliability of PICUPS should be investigated in further studies.

Nowadays, the rehabilitative approach and evaluation has changed in order to cope with the COVID-19 pandemic [31, 54, 55]. However, the validity of CPAx in post-ICU COVID-19 patients might be considered valuable, considering severely ill neurological patients, early discharged from the ICU, requiring assessment tools more focused on physical early function improvements rather than ADL. From this point of view, demonstrating reliability, validity, and feasibility of CPAx compared to existing functional assessment tools in post-ICU COVID-19 patients can be viewed as the strength of the study.

The small sample size can be viewed as the limitation of the present study. We did not calculate the sample size for discriminative validity, but ROC analysis was performed and 12 or lower scores in CPAx was found to discriminate patients with muscle weakness (MRC sum score $<$ 48); this result is suggested to be repeated with a larger sample size.

5. Conclusions

The findings showed that CPAx might be considered as a valid, reliable and feasible tool that might be used to assess the physical functional state in COVID-19 patients during the 48 hours following discharge from the ICU. It is a useful tool that can be used for supporting triage and handover of ICU patients stepping down to rehabilitation units or other hospital wards where individualized rehabilitation programs of patients are planned. Further observational studies are mandatory to improve the knowledge on the use of this instrumental tool.

Ethical approval

This study was approved by the Research Ethical Board of University of Health Sciences of Koç University Hospital (approval number: 245/2021). The study was also registered on clinicaltrials.gov.tr (NCT04762056).

Funding

This research received no external funding.

Informed consent

Not applicable.

Author contributions

Conceptualization, E.G., A.d.S., O.O.T.; methodology, E.G., A.d.S., O.O.T.; formal analysis, Z.T., M.T., A.B., C.C.; investigation, Z.T., M.T., A.B.; resources, O.O.T.; data curation, A.d.S., O.O.T.; writing – original draft preparation, E.G., A.d.S., writing – review and editing, O.O.T.; visualization, Z.T., M.T., A.B., C.C.; supervision, A.d.S, O.O.T. All authors have read and agreed to the published version of the manuscript.

Data availability statement

Due to the nature of this research, the supporting data are not available to be shared publicly due to ethical restrictions.

Footnotes

Acknowledgments

None to report.

Conflict of interest

The authors have no competing interests to declare.

References

Daste

et al., Post-intensive care syndrome in patients surviving COVID-19. Ann Phys Rehabil Med. 2021; 64(6): 101549.

Gallelli

et al., Vitamin D Serum Levels in Subjects Tested for SARS-CoV-2: What Are the Differences among Acute, Healed, and Negative COVID-19 Patients? A Multicenter Real-Practice Study. Nutrients, 2021; 13(11).

Eyigör

Umay

. Dysphagia management during COVID-19 pandemic: A review of the literature and international guidelines. Turk J Phys Med Rehabil. 2021; 67(3): 267-274.

Roberts

et al., Identification of Functional Limitations and Discharge Destination in Patients With COVID-19. Arch Phys Med Rehabil. 2021; 102(3): 351-358.

Halpin

et al., Postdischarge symptoms and rehabilitation needs in survivors of COVID-19 infection: A cross-sectional evaluation. J Med Virol. 2021; 93(2): 1013-1022.

Puthucheary

et al., The Post-ICU presentation screen (PICUPS) and rehabilitation prescription (RP) for intensive care survivors part II: Clinical engagement and future directions for the national Post-Intensive care Rehabilitation Collaborative. 2021; p. 1751143720988708.

Turner-Stokes

et al., The post-ICU presentation screen (PICUPS) and rehabilitation prescription (RP) for intensive care survivors part I: development and preliminary clinimetric evaluation. 2021; p. 1751143720988715.

Elliott

et al., Assessing physical function and activity for survivors of a critical illness: a review of instruments. Aust Crit Care. 2011; 24(3): 155-66.

Denehy

et al., A physical function test for use in the intensive care unit: validity, responsiveness, and predictive utility of the physical function ICU test (scored). Phys Ther. 2013; 93(12): 1636-45.

10.

Parry

et al., Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med. 2015; 41(5): 744-62.

11.

Holdar

et al., Cross-cultural adaptation and inter-rater reliability of the Swedish version of the Chelsea critical care assessment tool (CPAX-Swe) in critically ill patients. Disabil Rehabil. 2021; 43(11): 1600-1604.

12.

Nordon-Craft

et al., The physical function intensive care test: implementation in survivors of critical illness. Phys Ther. 2014; 94(10): 1499-507.

13.

Torjesen

. Covid-19: Intensive care units asked to take extra patients as hospitals struggle to find beds. Bmj. 2022; 376: o125.

14.

Bestall

et al., Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax. 1999; 54(7): 581-6.

15.

Hermans

et al., Interobserver agreement of Medical Research Council sum-score and handgrip strength in the intensive care unit. Muscle Nerve. 2012; 45(1): 18-25.

16.

Turan

Topaloglu

Ozyemisci Taskiran

. Medical Research Council-sumscore: a tool for evaluating muscle weakness in patients with post-intensive care syndrome. Crit Care. 2020; 24(1): 562.

17.

De Jonghe

et al., Paresis acquired in the intensive care unit: a prospective multicenter study. Jama. 2002; 288(22): 2859-67.

18.

Corner

et al., The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study. Physiotherapy. 2013; 99(1): 33-41.

19.

Eggmann

et al., German version of the Chelsea Critical Care Physical Assessment Tool (CPAx-GE): translation, cross-cultural adaptation, validity, and reliability. Disabil Rehabil. 2021; pp. 1-10.

20.

Knaus

et al., APACHE II: a severity of disease classification system. Crit Care Med. 1985; 13(10): 818-29.

21.

Reith

et al., The reliability of the Glasgow Coma Scale: a systematic review. Intensive Care Med. 2016; 42(1): 3-15.

22.

Tanıgör

Eyigör

. Evaluation of dysphagia in patients with sarcopenia in a rehabilitation setting: insights from the vicious cycle. Eur Geriatr Med. 2020; 11(2): 333-340.

23.

Launois

et al., The modified Medical Research Council scale for the assessment of dyspnea in daily living in obesity: a pilot study. BMC Pulm Med. 2012; 12: 61.

24.

Giray

Karali-Bingul

Akyuz

. The Effectiveness of Kinesiotaping, Sham Taping or Exercises Only in Lateral Epicondylitis Treatment: A Randomized Controlled Study. PMR. 2019; 11(7): 681-693.

25.

Curci

et al., Authors’ reply to: Rivera-Lillo et al. comment on: Early rehabilitation in post-acute COVID-19 patients: data from an Italian COVID-19 rehabilitation unit and proposal of a treatment protocol. Eur J Phys Rehabil Med. 2021; 57(1): 172-173.

26.

Küçükdeveci

et al., Adaptation of the modified Barthel Index for use in physical medicine and rehabilitation in Turkey. Scand J Rehabil Med. 2000; 32(2): 87-92.

27.

Arik

et al., Validation of Katz index of independence in activities of daily living in Turkish older adults. Arch Gerontol Geriatr. 2015; 61(3): 344-50.

28.

Jones

et al., The five-repetition sit-to-stand test as a functional outcome measure in COPD. Thorax. 2013; 68(11): 1015-20.

29.

Bohannon

, Reference values for the five-repetition sit-to-stand test: a descriptive meta-analysis of data from elders. Percept Mot Skills. 2006; 103(1): 215-22.

30.

Koo

, A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016; 15(2): 155-63.

31.

Zhu

et al., Reconfigure rehabilitation services during the Covid-19 pandemic: best practices from Southwest China. Disabil Rehabil. 2021; 43(1): 126-132.

32.

Florence

et al., Intrarater reliability of manual muscle test (Medical Research Council scale) grades in Duchenne’s muscular dystrophy. Physical Therapy. 1992; 72(2): 115-122.

33.

Fleiss

Levin

Paik

. Statistical methods for rates and proportions. 2013; John Wiley & Sons.

34.

Giray

et al., Validity and reliability of the Turkish version of the prosthesis donning and doffing questionnaire for persons with transtibial amputations. 2022; 34(2): 122-130.

35.

Silveira

et al., Assessing functional status after intensive care unit stay: the Barthel Index and the Katz Index. Int J Qual Health Care. 2018; 30(4): 265-270.

36.

Zhang

et al., Chinesisation, adaptation and validation of the Chelsea Critical Care Physical Assessment Tool in critically ill patients: a cross-sectional observational study. BMJ Open. 2021; 11(4): e045550.

37.

Corner

et al., Construct validity of the Chelsea critical care physical assessment tool: an observational study of recovery from critical illness. Crit Care. 2014; 18(2): R55.

38.

Milton

et al., ICU discharge screening for prediction of new-onset physical disability-A multinational cohort study. Acta Anaesthesiol Scand. 2020; 64(6): 789-797.

39.

Hamasaki

et al., Association of handgrip strength with hospitalization, cardiovascular events, and mortality in Japanese patients with type 2 diabetes. Sci Rep. 2017; 7(1): 7041.

40.

de Sire

et al., Rehabilitation and COVID-19: the Cochrane Rehabilitation 2020 rapid living systematic review. Update as of August 31st 2020. Eur J Phys Rehabil Med. 2020; 56(6): 839-845.

41.

Negrini

et al., Rehabilitation and COVID-19: the Cochrane Rehabilitation 2020 rapid living systematic review. Update as of July 31st 2020. Eur J Phys Rehabil Med. 2020; 56(5): 652-657.

42.

de Sire

et al., Rehabilitation and COVID-19: a rapid living systematic review by Cochrane Rehabilitation Field updated as of December 31st, 2020 and synthesis of the scientific literature of 2020. Eur J Phys Rehabil Med. 2021; 57(2): 181-188.

43.

Curci

et al., Early rehabilitation in post-acute COVID-19 patients: data from an Italian COVID-19 Rehabilitation Unit and proposal of a treatment protocol. Eur J Phys Rehabil Med. 2020; 56(5): 633-641.

44.

Curci

et al., Functional outcome after inpatient rehabilitation in postintensive care unit COVID-19 patients: findings and clinical implications from a real-practice retrospective study. Eur J Phys Rehabil Med. 2021; 57(3): 443-450.

45.

Albu

et al., Multidisciplinary outpatient rehabilitation of physical and neurological sequelae and persistent symptoms of covid-19: a prospective, observational cohort study. Disabil Rehabil. 2021; pp. 1-8.

46.

Farì

et al., Impact of COVID-19 on the mental health in a cohort of Italian rehabilitation healthcare workers. J Med Virol. 2022; 94(1): 110-118.

47.

Lagier

et al., Swallowing function after severe COVID-19: early videofluoroscopic findings. Eur Arch Otorhinolaryngol. 2021; 278(8): 3119-3123.

48.

Corner

et al., The responsiveness of the Chelsea Critical Care Physical Assessment tool in measuring functional recovery in the burns critical care population: an observational study. Burns. 2015; 41(2): 241-7.

49.

Huang

et al., Functional Status Score for the ICU: An International Clinimetric Analysis of Validity, Responsiveness, and Minimal Important Difference. Crit Care Med. 2016; 44(12): e1155-e1164.

50.

Klok

et al., The Post-COVID-19 Functional Status scale: a tool to measure functional status over time after COVID-19. Eur Respir J. 2020; 56(1).

51.

Ferraro

et al., COVID-19 related fatigue: Which role for rehabilitation in post-COVID-19 patients? A case series. J Med Virol. 2021; 93(4): 1896-1899.

52.

https://www.kcl.ac.uk/cicelysaunders/resources/tools/picups-and-picups-plus-v10.pdf.

53.

de Sire

Giray

Ozyemisci Taskiran

. Chelsea physical assessment tool for evaluating functioning in post-intensive care unit COVID-19 patients. J Med Virol. 2021; 93(5): 2620-2622.

54.

D’Souza

et al., Barriers leading to increased disability in neurologically challenged populations during COVID-19 pandemic: a scoping review. Disabil Rehabil. 2021; pp. 1-14.

55.

Khoo

Jesudason

FitzGerald

. Catching our breath: reshaping rehabilitation services for COVID-19. Disabil Rehabil. 2021; 43(1): 112-117.

Validity,inter-rater reliability,and feasibility of the Chelsea Physical Assessment Tool for assessing physical function in post-acute COVID-19 patients: A cross-sectional study

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

CONCLUSION:

Keywords

1. Introduction

2. Materials and methods

2.1 Study design and setting

2.2 Study participants

2.3 Sample size calculation

2.4 Outcome measures

2.4.1 Validity

2.4.2 Inter-rater reliability

2.4.3 Feasibility

2.4.4 Sequential Organ Failure Assessment Score (SOFA score)

2.4.5 Acute Physiology and Chronic Health Evaluation (APACHE II)

2.4.6 Glasgow Coma Scale

2.4.7 Functional Oral Intake Scale

2.4.10 Hand grip strength measurement

2.4.11 CPAx

2.4.12 Barthel Index

2.4.13 Katz Index

2.4.14 The 5 time sit-to-stand test

2.4.15 30 seconds sit-to-stand test

2.5 Statistical analysis

3. Results

Table 1 Demographic, clinical characteristics and baseline data of the patients

Ethical approval

Funding

Informed consent

Author contributions

Data availability statement

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Demographic, clinical characteristics and baseline data of the patients