The post-ICU presentation screen (PICUPS) and rehabilitation prescription (RP) for intensive care survivors part I: Development and preliminary clinimetric evaluation

Introduction

Surviving critical care impacts many aspects of physical, cognitive and psychological function, often described as ‘Post intensive care unit [ICU] Syndrome.^1,2 Rehabilitation needs can thus be complex.^3,4 In particular, COVID-19 ICU survivors face all the general impacts of ICU care, but with additional disease specific features ⁵ and possible sustained post-infective elements (‘Long COVID’),^6–9 As yet, however, there is no systematic collection of data to identify the individual rehabilitation needs of patients as they leave critical and acute hospital care, or to inform the planning and development of rehabilitation services.

Meanwhile, despite national guidance published being some 11 years ago, ¹⁰ there is still no UK national registry which captures patient-level data on the rehabilitation requirements for critical care survivors. Provision of post-ICU rehabilitation remains the responsibility of local Trusts, with no central co-ordination of pathways or collation of data to inform care. One exception to this is the use of a Rehabilitation Prescription (RP) established within the Trauma Networks to identify the rehabilitation needs of seriously-injured patients leaving major trauma centres. ¹¹ This patient-held record sets out the individual’s requirements for ongoing rehabilitation and the plan to provide for them. Collated nationally through the Trauma Audit and Research Network (TARN: www.tarn.ac.uk), the associated dataset can be used at a population level to examine gaps between capacity and demand for services, in hospital and in the community. The UK Rehabilitation Outcomes Collaborative (UKROC) is commissioned by NHS England to provide the national clinical registry for specialist rehabilitation. It collates patient-level data on needs, inputs and outcomes and provides reports on activity and quality bench-marking for all specialist inpatient services in England. A recent National Clinical Audit successfully used linkage between TARN and UKROC to quantify the shortfall in provision of specialist inpatient rehabilitation beds in England and the cost of rectifying it. ¹² This approach could equally apply to all critical care survivors.

Against this background the Post-ICU Presentation Screen (PICUPS) was developed. It is designed as a clinical tool to support decision-making from ICU, through acute hospital care and into rehabilitation. Its purpose is (a) to inform the immediate plan for care on the acute ward, (b) to identify problems likely to require further, more detailed assessment and evaluation by members of the multi-disciplinary team and so prompt appropriate referrals, (c) to help inform the development of a personalised Rehabilitation Prescription (RP), and (d) to facilitate gap analysis between services provided and demand for those services. ¹³ Together these also support the systematic collection of data on needs for rehabilitation, during recovery and as patients leave intensive and acute care.

It is important to establish the scaling properties of any measurement tool. Complex clinical conditions are typically multidimensional, and this can create a tension within the science of clinical measurement. ¹⁴ Psychometricians emphasise the importance of uni-dimensionality and interval-level measurement, but clinicians typically place greater value on the content and clinical usefulness of a tool to describe the condition within the heterogeneity of real-life clinical practice. The term ‘clinimetrics’ was coined to accommodate both ‘standardisation’ (reliability and construct validity) and ‘sensibility’ (face and content validity) in the evaluation of clinical tools.

This article describes the development of the PICUPS tool and presents a preliminary exploration of its clinimetric properties to support its validity for clinical implementation. The accompanying paper explores in more detail the ‘system utility’ of the PICUPS and RP, their potential application in clinical practice and directions for future development.

Methods

Design and development

The task was to develop a practical tool that addressed the broad range of rehabilitation needs across the post-ICU syndrome spectrum, while minimising data burden. It had to be simple enough to be completed by junior members of the clinical team (medical, nursing and therapy), so not requiring any particular specialist knowledge.

Development and validation of a clinical tool is normally a lengthy and involved process. By necessity, the PICUPs tool was developed in haste in order to identify the rehabilitation needs of critical illness survivors from the first wave of the 2020 COVID-19 pandemic. A pragmatic approach thus had to be taken, drawing very rapidly on clinical expertise in critical care and rehabilitation medicine, to deliver timely development and initial piloting during the pandemic.

Development started in early May 2020 and was led by a core multidisciplinary group of experienced clinicians in the various disciplines involved in post-ICU rehabilitation, including occupational therapy, physiotherapy, speech and language therapy, psychology, rehabilitation medicine and critical care medicine, nursing and dietetics – all brought together through the National Post-ICU Rehabilitation Collaborative, led by the Intensive Care Society and the British Society of Rehabilitation Medicine. ¹⁵ The key constructs and item-level descriptors were identified by group consensus to produce a first draft in mid-May. The PICUPS drew on existing validated measures for the identified constructs, adapted where necessary to fit the 6-point scale structure. There followed an iterative process of feedback and adjustment until consensus was reached at version 9 (28.5.2020). As part of this process, exploratory field testing was conducted in five centres across London, and teams provided constructive feedback on its utility and usefulness, reporting that the tool was practical and quick (3 min or so per patient) to complete. Encouragingly the range of scores generated resonated with clinical experience and changed as the patient’s condition improved during acute care, suggesting that it might also be used to capture the trajectory of recovery (data available on request).

The PICUPS tool and rehabilitation prescription

The PICUPS tool consists of two main components:

The PICUPS-Basic supports initial triage and handover of patients stepping down from ITU into the acute wards. It comprises 14 items in four domains: (a) Medical and essential care, (b) Breathing and nutrition; (c) Physical movement and (d) Communication, cognition and behaviour. Tracheostomy care and weaning were separated to ensure that data informing planning of patient destination (ICU, high dependency or general ward setting) were captured in the setting of recurrent inter-institutional transfers during a pandemic.

The PICUPS-Plus identifies potential higher-level items that may need to be addressed as the patient progresses during acute care and onwards into rehabilitation. It comprises 10 additional items in three domains; (a) Upper airway, (b) Physical and activities of daily living, and (c) Symptoms that interfere with activities.

Each of the 24 PICUPS items is rated on a 6-point ordinal scale that describes the patient’s level of function ranging from 0 (most dependent) to 5 (near-normal).

Thresholds set within the score-range for each item trigger referrals to the various different disciplines for further evaluation. Each of those disciplines will make their own assessment using more detailed assessment tools. (The PICUPs does not replace those tools, but simply acts as a screening tool and overall functional assessment.) These then inform the Rehabilitation Prescription on step down from acute care to indicate their needs for ongoing rehabilitation and the plans to provide for them, whether in inpatient- or community-based services.

A standardised data collection tool was developed to collate the data (see the online supplementary material for more details (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715)). This tool may now be downloaded, together with the PICUPS, from the Intensive Care Society’s web page. ¹⁵ As part of this development, the data collection tool (including the PICUPS and RP minimum dataset) was incorporated into the UKROC database and a freely-available dedicated software package (built in Microsoft Excel) was developed and piloted to support their use and local data collection by clinical teams.

Piloting and wider engagement

The next stage of development was to extend the pilot activity to a wider community of clinicians across England, representing the various different contexts in which the tools would be used. The primary aim of this extended pilot was to explore the utility of the PICUPS and RP as clinical decision-making tools, making them as useful and practical as possible for this post-ICU context, while minimising data burden. Secondary aims were to explore the clinimetric properties of the PICUPs (presented below) and to use the data to gather preliminary information about the rehabilitation needs of patients following treatment in intensive care (whether following COVID-19 infection or for other reasons) (presented in Part II).¹⁶

Data were collected at Acute NHS Trusts as part of a clinical and/or service evaluation process and site leads were responsible for arranging local permissions in line with their own Trust policies. According to the UK Health Research Authority, the publication of research findings from de-identified data gathered in the course of routine clinical practice does not require research ethics permission. UKROC has permissions in place to collate identifiable data centrally for clinical, audit and commissioning purposes. In this pilot, the UKROC team, collated the de-identified data only on behalf of the UK Intensive Care Society (ICS) as a trusted data environment in line with the emergency information governance arrangements during the COVID pandemic of 2020 (https://www.nhsx.nhs.uk/information-governance/guidance/covid-19-ig-advice/).

Participating centres

Acute NHS Trusts were approached via the National Post-Intensive Care Rehabilitation Collaborative through meetings and webinars. Volunteer sites were recruited in a rolling fashion between the 1 and 31 July 2020. The 26 participating centres represented a wide geographic spread across England ¹⁶ and encompassed a range of different settings including ICUs and acute wards in district general, teaching and single specialty hospitals.

Data collection

Following step down from ICU, patients recover on different trajectories. Each participating centre was asked to record PICUPS and RP on 10 patients – ideally five making a “rapid recovery” and five who were on the “slower recovery” pathway who would have a second PICUPs recorded, but this judgement was at the discretion of the treating clinicians. The data collection points are illustrated in Figure 1. Patients could include those with COVID-19 or those with prolonged (>7 days) stays on ICU for other reasons.

OPEN IN VIEWER

Figure 1.

Data collection points for the pilot. Following step down from ICU, patients recover on different trajectories. Some patients make a very rapid recovery. Others follow a slower trajectory to the point of discharge to the community or on to further in-patent rehabilitation. Patients making a rapid recovery would have a single PICUPs and PICUPs-Plus recorded at transition to the acute wards and then a Rehabilitation Prescription recorded at discharge to the community. Those on a slower recover trajectory would have a second PICUPs and PICUPs-Plus recorded at the point when they become ‘Rehabilitation Ready’ (i.e. when their medical condition has stabilised and rehabilitation became the primary focus for intervention) and then a Rehabilitation Prescription recorded at discharge to the community.

Teams who were unable to collect live data during the short time window were invited to submit scores applied retrospectively (through multidisciplinary team discussion) about patients who had recently passed through the service. Data were either entered directly into the dedicated supporting software or on standard de-identified paper forms sent to UKROC by secure NHS mail.

Although the PICUPS-Plus items were originally intended to be optional, for use on a ‘pick ‘n’ mix’ basis to score the relevant items only, teams were asked to complete all 24 items during the pilot.

Utility

Teams also provided feedback about the feasibility and utility of the tool through online or paper-based questionnaires and through the online “catch up” sessions that ran throughout the piloting phase. In addition to providing general comments, we asked how well does the PICUPS:

Describe the patients as they transition out of critical care?

Help to trigger engagement of other members of the multi-professional team?

Support the construction of the Rehabilitation Prescription?

To gain insight into the likely legal basis for future data collection, teams were also asked to estimate what percentage of their patients would have been able to consent to the data collection (a) on step down from critical care and (b) on discharge from acute care.

Clinimetric analysis

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative has published a framework to encourage transparent methodology in the evaluation of outcome measurement tools for research and clinical practice. ¹⁷ This framework is used to describe the different components of clinimetric evaluation of the PICUPs using classical test theory – the parameters of interest being its face and content validity, utility, structural validity and responsiveness to change.

Statistical methods

See online supplementary material for details (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715) of the methods for statistical analysis and associated results tables.

Results

Data extraction is summarised in Figure 2. A total of 552 records were collated from 314 patients across the 26 participating centres, of which 121 patients (38.5%) had COVID-19. Data on age, sex and ethnicity were held locally but not collated centrally in this pilot to preserve patient anonymity.

OPEN IN VIEWER

Figure 2.

Summary of extracted data.

Score distribution, internal consistency and scalability

The tabulated results for score distribution, internal consistency and factor analysis may be found in the online supplementary material Tables A to C (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715).

Online supplementary material Table A (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715) shows the distribution of scores across the whole sample (including all time-points). All items except ‘Family distress’ covered the entire score range (0–5). As expected, there were some ceiling effects in less commonly applicable items such as ventilation and tracheostomy care weaning, but otherwise no obvious floor or ceiling effects were apparent. The median (IQR i.e. 25th–75th percentile) scores for the PICUPS Basic, Plus and Total scores were respectively 53 (43–62), 33 (22–41) and 84 (64–101).

Online supplementary material Table B (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715) shows the Cronbach’s alpha and item-total correlations for the PICUPS-Basic- and Plus Subscale, and the full-scale scores. Alpha values were respectively 0.92, 0.91 and 0.95. Item-total correlations were high except for ‘Breathing’, ‘Mental Health’ and ‘Family Distress’ and Cronbach’s alpha improved when ‘Breathing’ and ‘Mental Health’ were deleted, but only very marginally.

Bartlett’s test of sphericity (significant at p < 0.001) and the Kaiser–Meyer–Olkin test (0.940) both indicated the correlation matrix was suitable for factor analysis. Online supplementary material Table C (https://journals.sagepub.com/doi/suppl/10.1177/1751143720988715) summarises the results of principal components analysis. All items loaded strongly on the first component except for ‘Breathing’ and ‘Mental health’ which loaded only weakly. Four factors had eigenvalues >1, which were 12.4, 2.2, 1.5, and 1.2 – respectively accounting for 51%, 9%, 6% and 5% of the variance (72% in total) – see Scree plot, Figure 3. The rotated factor solution suggested three main subscales (Physical, Respiratory and Psychosocial) with a possible further comprising one item only (Breathing). When Breathing and Cognition were included respectively in factors 2 and 3, Cronbach’s alphas for the three subscales were 0.96, 0.85 and 0.71, illustrated in Figure 4.

OPEN IN VIEWER

Figure 3.

Eigenvalues of the components of PCA of the PICUPs (n = 306). The Scree plot of eigenvalues from the principal components analysis (PCA) shows a very striking drop after the first factor (which accounts for 51% of the variance), and falls more slowly after the third.

OPEN IN VIEWER

Figure 4.

Factor structure of the PICUPS tool according to exploratory factor analysis.

Responsiveness

Centres represented various stages in the pathway (within ICU, around stepdown, in acute care, etc.). Data were not therefore captured systematically at the time-points originally intended. Nevertheless, of the 200 patients who had a PICUPS rated on more than one occasion, 92 had complete scores at two different time points (labelled Time 1 and Time 2). The median (IQR) length of stay in ICU for this group was 31 (10–46), range 1–86 days, and on the acute ward was 11 (7–15 days) range 2–47 days. Tables 1 and 2 summarise the changes in total and item-level scores for these patients. The median (IQR) total PICUPs score changed from 82 (67–92) to 105 (94–114). After correcting for multiple tests, statistically significant changes were seen in all but two items (Tracheostomy weaning and Behaviour). Figure 5 shows a radar chart illustrating the change in median scores.

OPEN IN VIEWER

Table 1.

Median (IQR) change in item-level scores between Time 1 and Time 2 (n = 92).

Score	Time 1		Time 2		Wilcoxon signed rank tests
Item	Median	25th–75thCentile	Median	25th–75thCentile	z	P value*
PICUPS-Basic	52	42–58	65	57–68	–7.02	<0.001
PICUPS-Plus	28	21–36	40	33–46	–6.82	<0.001
Total PICUPS	82	67–92	105	94–114	–7.07	<0.001
PICUPS-Physical	34	23–43	55	43–60	–7.11	<0.001
PICUPS-Respiratory	29	25–33	33	30–35	–6.31	<0.001
PICUPS-Psychosocial	17	14–18	18	16–20	–5.15	<0.001

*Significance level <0.008 allowing for multiple tests.

OPEN IN VIEWER

Table 2.

Median (IQR) change in item-level scores.

	Score	Time 1		Time 2		Wilcoxon signed rank tests
	Item	Median	25th–75thCentile	Median	25th–75thentile	z	P value*
1	Medical stability	2	1–3	5	3–5	–8.59	<0.001
2	Medical care	4	4–5	4	3–5	–6.96	<0.001
3	Ventilator	5	5–5	5	5–5	–6.11	<0.001
4	Tracheostomy care	5	5–5	5	5–5	–3.02	0.003
5	Tracheostomy weaning	5	4–5	5	5–5	–3.09	0.002
6	Cough	2	0–5	5	5–5	–5.10	<0.001
7	Nutrition	3	2	5	3–5	–6.55	<0.001
8	Repositioning	2	1–3	5	4–5	–7.45	<0.001
9	Transfers	5	3–5	4	3–5	–7.52	<0.001
10	Communication	4	3–5	5	4–5	–4.32	<0.001
11	Cognition	5	4–5	5	4–5	–4.53	<0.001
12	Behaviour	4	3–5	5	4–5	–2.85	0.004
13	Mental health	4	3–5	4.5	4–5	–3.56	<0.001
14	Family distress	3	0–5	5	4–5	–3.79	<0.001
15	Breathing	4	2–5	4	2–5	–5.47	<0.001
16	Voice	4	0–5	5	4–5	–5.14	<0.001
17	Swallow	4.5	3–5	5	4–5	–4.99	<0.001
18	Posture	2	1–3	5	0–5	–7.65	<0.001
19	Personal hygiene	2	1–3	4	3–5	–6.52	<0.001
20	Physical care	2.5	1–3	4	3–5	–6.94	<0.001
21	Mobility	4	2–5	4	3–5	–6.23	<0.001
22	Upper limb	1	1–1	5	4–5	–7.43	<0.001
23	Fatigue	1	1–1	3	2–4	–5.27	<0.001
24	Pain	2	1–3	5	4–5	–5.47	<0.001

*Significance level <0.0021 allowing for multiple tests.

OPEN IN VIEWER

Figure 5.

Radar chart of change in median item-level ratings in patients with paired complete scores (n = 92). The radar chart (or “PICUPs splat”) provides a graphic representation of the functional profile from the PICUPS data. The 24 scale items are arranged as spokes of a wheel. Scoring levels from 1 (total dependence) to 5 (total independence) run from the centre outwards. Thus a perfect score would be demonstrated as a large circle. This composite radar chart illustrates the median scores on admission and discharge. The yellow shaded portion represents the median scores at Time 1 for each item. The blue-shaded area represents the change in median score from Time 1 to Time 2.

Discussion

This article describes the development and preliminary clinimetric evaluation of the PICUPS tool (version 9) to explore its face and content validity, utility, feasibility, structural validity and responsiveness.

Face validity refers to the extent to which the tool looks valid to those who will use it and content validity addresses whether it includes the relevant items to cover the construct comprehensively. Utility reflects whether it provides useful information. Feasibility addresses whether it can be implemented in clinical practice – is it timely and practical to apply in clinical care? Face and content validity of the PICUPS were conferred through the iterative development and consensus process, conducted by an experienced multi-professional team followed by the wider testing feedback and testing from a first national pilot study. Utility and feasibility were explored through qualitative feedback from the participating centres involved in the pilot study. Overall the feedback was very positive. Once familiar with it, teams on the ground reported that the PICUPS did not take long to record, and they generally found it to be useful for describing patient’s needs, triggering referrals and informing a rehabilitation prescription. Some teams also saw the value of the PICUPS as a way of providing a gap analysis for under-resourced members of the multi-professional team in order to develop future business cases for service improvement and reported that they were already starting to use it locally for this purpose. Constructive criticism of the tool itself led to some small adjustments to produce the current PICUPs version 10, and also provided valuable insights into the challenges for wider implementation and how these might be addressed.¹⁶

Structural validity reflects the degree to which the scores of a tool are an adequate reflection of the dimensionality of the construct to be measured and internal consistency is a measure of the uni-dimensionality of a scale or its subscales. The PICUPS is based on a formative model (in which the different items together form the construct) and is expected to be multi-dimensional, so it was not anticipated that the individual items would correlate very closely with each other. In fact, the PCA demonstrated much greater uni-dimensionality than expected. The very strong loading on the first factor (eigenvalue 12.4 accounting for 51% of the variance); the large gap between this and the second factor; and the high degree of internal consistency of the full scale scores (Cronbach’s alpha 0.95) together provide evidence of uni-dimensionality, suggesting that it is acceptable to sum the items into a single total score. The three-factor solution only accounted for a further 21% of the variance, but the items grouped within those factors do make sense from a clinical perspective.

Responsiveness refers to the ability of an instrument to detect clinically important changes (or stability) over time or as the result of an intervention. The PICUPS demonstrated sensitivity to change over time both at item-level, and in terms of the total scores.

Conclusions

Taken together, our findings provide positive evidence that the PICUPS has robust scaling properties as a clinical measure and is potentially useful as a screening tool for identifying rehabilitation needs as patients step down from ICU and acute hospital care. The pilot was able to provide sufficient data on both COVID and non-COVID ICU survivors for these tools to be applied to both populations. Part II addresses the practical use of the PICUPS and its further implementation in clinical practice.¹⁶ As a result, it has been actively deployed. The PICUPS and RP minimum dataset have now been incorporated into the UKROC national clinical dataset. The freely-available dedicated UKROC software package has in-built functionality to support clinicians in the preparation of a personalised RP. Data extracts can be generated from this software that either contain no identifiable data (pseudonymised IDs only), or encrypted identifiable data (NHS no and data of birth) that can be sent by secure transmission to the UKROC central database. The latter have the potential for use in data linkage to track patients from one service to another, or for inclusion in the NHS Digital’s central National Clinical Data Registry.

This work would not have occurred without the support of the critical care community, and we would specifically like to thank Megan Richardson, Katie Dowling, David McWilliams, Ioan Morgan, Tamara Pendry, Charlotte Pereira, Monica Trivedi, Michela Frankland, Rebecca Vokes, Sarah Linford, Callum Skillen, Wendy Willingham, Vicky Newey, Laura Grimsey, Kate Tantam, Louise Mitchell, Rachel Waddington, Sally Westhead, Jacqui Wakefield, Mandie Thomas, Sean Hanley, Natalie Scott, Lee Bolton and Emmeline Maidment-Fullard.

Supplemental Material