The Validity and Applicability of the Revised Delirium Rating Scale (DRS-R98) for Delirium Severity Assessment in a Critical Care Setting

Abstract

Background

Delirium is a neuropsychiatric syndrome common in critical illness. Worsening delirium severity is associated with poorer clinical outcomes, yet its assessment remains under-reported with most severity assessment tools not validated for critical care. The DRS-R98 is a widely applied and validated tool. The aim of this project is to report the validation and utility of the DRS-R98 in critical illness.

Methods

This prospective, cohort study was conducted in adults with delirium admitted to a critical care unit and predicted to stay for ≥ 24 h. We excluded patients with severe neurological or communication barriers that would have interfered with the DRS-R98 assessment. Patients were screened using a delirium detection algorithm and the Confusion Assessment Method for the Intensive Care Unit. Eligible patient informations were collected and reported to qualified assessor/s before visiting clinical areas, confirming delirium presence and undertaking DRS- R98 assessments. To assess the tool's construct validity, an intensivist completed the Clinical Global Impression-Scale (CGI-S). To calculate the inter-rater reliability (IRR) a subset of patients were simultaneously evaluated by two assessors.

Results

We assessed 22 patients, 73% were male, with a median age of 65 years (IQR14). The DRS -R98 mean (SD) severity score was 24 (+/-7.7), total scale was 29 (+/18.0), and CGI-S 3.5 (+/11.5). Assessment duration was 90 min (+/-55) and 15 min (+/-5) for record data extraction and clinical assessment respectively. The CGI-S significantly correlated with DRS-R98 severity (r = 0.626) and total (r = 0.628) scales. The DRS-R98 Cronbach's alpha was 0.896 for severity scale and 0.886 for total scale. The inter-rater reliability (IRR) was assessed in six patients and reported an inter-correlation coefficient of 0.505 (p = 0.124) and 0.565 (p = 0.93) for the severity and total scale respectively.

Conclusions

In critical care, the Delirium Rating Scale R98 had good construct validity, excellent internal consistency, and moderate inter-rater reliability.

Keywords

delirium severity validation critical care intensive care

Background

Delirium is a neuropsychiatric presentation common in critical illness. The incidence is as high as 89% in mechanically ventilated patients.¹ The cause of this is multifactorial and a mixture of predisposing risk factors including older age (> 65 years), frailty, alcohol dependency, medication with a high anticholinergic burden and precipitating risk factors sepsis, exposure to deliriogenic medications (eg, benzodiazepines and opioids) and disturbance in circadian rhythm.^2,3

Several outcomes are used as primary measures in delirium prevention and treatment. Prior to around 2010, there was greater emphasis on perceiving delirium as a binary condition, with outcomes that focused on delirium resolution dominating the literature.⁴

After 2010, there has been greater interest in considering delirium as a continuum and in reporting outcomes which include delirium intensity and changes over time. Such outcomes can capture the different dimensions of delirium with unique features which binary or categorical outcomes can overlook.

The concept of delirium as a continuum can support clinicians in monitoring a patients’ progress. Greater delirium severity is generally associated with poorer outcomes, both during hospitalisation and on discharge⁵ ,including increase in long term cognitive disturbance.^6,7 Therefore delirium severity assessment could be a suitable outcome measure for clinical and research purposes. Rose et al recently included delirium severity as a core delirium outcome in their proposed core outcome set which was developed by a group of international delirium experts and organisations that support those surviving critical care illness “Del-COrS group.”^8,9

Although a number of psychometric tools have been designed and validated to assess and numerically score delirium severity in hospital inpatients.^4,10,11 The use of delirium severity assessment scales in critical illness remains limited.^12,13 The majority of these tools were not designed or validated to assess delirium severity in patients admitted to a critical care setting.¹⁴

The revised delirium rating scale (DRS-R98) remains one of the most validated, widely translated, and extensively applied delirium severity assessment scales outside of critical care, including medical, surgical, and older age population.^13,15–47 The DRS-R98 has been applied in critical care, yet as far as we are aware, it has not been validated as a delirium severity assessment tool in this setting.^13,14 The validation of a given tool in a specific setting is an essential step to justify its adaptation in research and integration into clinical practice.

Study aim

The aim of this study is to report our experience in validating the DRS-R98 in a critical care setting.

Method

Study Design

Prospective, cross-sectional, single-centre cohort project.

Study Setting

The study assessment was undertaken in adult tertiary referral critical care unit at Guy's and St Thomas (GSTT) NHS Foundation Trust, London, UK from 01/02/2016 to 20/09/2018.

Ethical Approval

Delirium severity assessment was included as a core aspect of the GSTT 2016 critical care analgesia, sedation, and delirium guidelines.⁴⁸ A clinical service evaluation authorisation request was submitted to GSTT research and development department (project 5711) to authorise the study. The study was approved and full application for ethical approval was deemed unnecessary by research and development. The intensivist in charge of the patient's care were contacted before each assessment and the patient, family and visitors were appraised whenever possible.

Participants

Patients eligible for delirium severity assessment included patients who were admitted to critical care and, expected to stay for longer than 24 h. A member of the assessment team confirmed they met the delirium DSM-IV diagnostic criteria on assessment day.⁴⁹ Patients were not eligible if there was a neurological or communication barrier which could significantly interfere with the ability to collect data needed to evaluate most of the DRS-R98 scale items. This included severe cerebrovascular accident (CVA), post-traumatic brain injury (TBI) with evidence of hypoxic brain damage, and if the patient was deeply sedated [Richmond Agitation Sedation Scale (RASS) ≤ -3].

Procedures

The core team were :- research fellow (EA), psychiatric nurse/psychologist (MB), neuropsychiatry registrar (AP), consultant neuropsychiatrist (MP) and consultant pharmacist (CM).

Prior to the study; a copy of the DRS-R98 administration manual was provided to the team.⁵⁰ EA facilitated a meeting with primary assessors (MP and MB) to discuss the approach to assess different items in selected patients.

A delirium detection algorithm for ICU was used to screen and identify eligible patients.⁵¹ This included applying collected data to perform a delirium detection chart review and was followed by formal delirium screening whenever feasible, using the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU).^52,53 Patients were screened, on average twice weekly over the entire study period, although sometimes considerably less depending on the team's clinical demands which took precedence.

The collected data included delirium relevant multidisciplinary team (MDT) notes from previous 24 h, medication history and selected neurological assessments (eg, CAM-ICU and RASS). All data was collected and stored in an anonymised case report form (CRF). Historical, social, and clinical information relevant to delirium assessment and management was extracted by EA.

The assessors (MP and AP or MB) liaised with EA and arranged a time to visit the patient within 2 h of initial screening. While visiting, the primary assessors (MP or MB) confirmed the patient met the delirium DSM-IV diagnostic criteria.⁴⁹ The DRS-R98 scale items were then scored by assessors’ (MP and AP or MB) using extracted data, clinical observations, interviewing staff, family, and/or caregivers. The DRS-R98 form was completed concurrently, collected by EA and stored securely for analysis.

Concurrent validity of the DRS-R98 was evaluated through direct comparison of the DRS-R98 score with the (CGI-S) Score assessed by the consultant intensivist in charge of patient care.⁵⁴ To assess inter-rater reliability (IRR), a subset of patients were evaluated by MP and AP. The primary assessor (MP) would undertake a clinical interview, and DRS-R98 forms were independently and simultaneously completed by both assessors.

To assess the tool's ability to detect change over time, EA tracked recruited patients to identify those who had recovered from delirium before critical care discharge, and to evaluate post recovery using DRS-R98 severity subscale, and the Clinical Global improvement scale (CGI-I).

Figure 1 illustrates the study flow chart.

Figure 1.

DRS-R98 process, from screening to validation.

Data Management

All patient identifiers (eg, name, hospital number, gender, critical care admission and discharge dates) were coded and saved in a separate encrypted folder.

Definitions and Measurement Tools

Revised Delirium Rating Scale- -R98

The DRS-R98 consisted of 16 items split into diagnostic and severity scales.⁵⁵ The diagnostic scale includes three items and severity scale 13 items. Each item is rated on a scale of 0 to 2 or 3, with higher scores indicating greater severity. At baseline, both diagnostic and severity subscales are assessed to give total score, and a follow-up assessment is completed using severity scale only. The total scale score maximum is 46, and the severity scale score maximum is 39.

The DRS-R98 severity scale can then be subclassified into non-cognitive scores (scores from items 1-8) and cognitive scores (scores from items 9-13), which provided additional delirium severity monitoring strategies.^26,42,56

Figure 2 demonstrates the DRS-R98 content, structure, sub-classifications, the different severity stages thresholds and ranges used in this study.

Figure 2.

DRS-R98indivdual components and subclassifications.

Clinical Global Impression-Severity and Improvement Scales

The clinical global impression severity (CGI-S) and the clinical severity improvement (CGI-I) are clinician rated scales commonly used in psychiatry, both are applied by clinicians without training.⁵⁷ The CGI-S scale is a single overall impression of patient's psychiatric illness. It rates severity on a Likert-type scale ranging from 1 (normal) to 7 (most severely observed).

The CGI-I scale score reflects the degree of (change) in patient's psychiatric illness status from baseline and is rated by 7 points. Likert scale ranging from 1 (very much improved) to 7 (very much worse).

Motor Subtype Classification

A motor subtype classification was applied using the DRS-R98 severity scale scores on items 7 (motor agitation) and item 8 (motor retardation) scored. Patients with a score of ≥ 1 on item 7 of the DRS-R98 were classified as hyperactive, those with score of ≥ 1 on i 8 were classified as hypoactive, while score ≥ 1 on both items 7 and 8 were classified as having a mixed motor subtype.⁵⁸

Delirium Recovery

Defined as resolution of delirium after treatment was initiated by the clinician in charge over the same critical care admission. It was also determined by documentation of two consecutive negative CAM-ICU recordings within the previous 24 h and applying the delirium detection algorithm. Recovery was further confirmed by MP or MB using the DSM-IV criteria.

Sample Size

The sample size was calculated, a priori, with advice from medical statistics at Kings College London (KCL).⁵⁹ The analysis was based on values from the original validation study, where the DRS-R98 total score was correlated with the CGI-S (clinical global impression severity scale) with a correlation coefficient (r) of 0.62.⁵⁹

To demonstrate the DRS-R98's validity to measure delirium severity in critical illness a minimum of 18 patients were required to detect a correlation of 0.62 or higher between DRS-R98 total and CGI-S scores, (p ≤ 0.05, Power = 80%).⁶⁰ To determine the tools’ inter-rater reliability (IRR) a minimum sample of 10 patients was needed to detect a concordance between assessors of 0.7 or more via calculating the scales intraclass correlation.^60,61

Statistical Analysis

Continuous variables (eg, age and scales scores) were described as a mean (with standard deviation) or median (with a range). Categorical variables (eg, gender) were presented using actual number and /or percentages.

Construct validity was assessed by correlating DRS-R98 total and severity scale scores with the CGI-S scores using Pearson's correlation coefficient, while the internal consistency was evaluated by Cronbach's alpha coefficient.

The inter rater-reliability (IRR) was estimated by calculating Intra-Class Correlation (ICC) between paired assessors scores [using two way mixed model-reported for single measure].⁶²

The missing or non-assessable items were substituted with Pearson mean value calculated from the remaining scored items values in the assessed case.⁶³ All analyses was undertaken with Graph-pad Prism 7 and IBM SPSS 24 software.

Results

Participants and Scales

Twenty-two critical care patients were assessed with the DRS-R98 over the study period of 31.5 months. Although the actual recruitment time was about six months of the 31.5, because clinical work took precedence. The recruited patients were largely admitted to level 3 (highest acuity) critical care unit (n = 18, 82%),⁶⁴ most were males (n = 16, 73%), with a mean age of 65 years (SD+/−14), two had a previous medical history (PMH) of dementia (9%), five with a history of psychiatric and/or neurological disease (23%) and three reported alcohol or drug dependence (14%).

Most were able to communicate verbally (n = 16,73%), included patients communicating via a tracheostomy placed speaking valve.

Fourteen patients (64%) had an atypical antipsychotic prescribed for agitation and /or delirium on assessment. The most frequently prescribed agent was quetiapine (n = 8), followed by olanzapine (n = 4).

Thirteen patients (59%) were prescribed a sedative on assessment day, the most frequently prescribed agent was propofol (n = 7). Seventeen patients (77%) were also prescribed an analgesic, with the most frequently prescribed agents being paracetamol (n = 11) and fentanyl (n = 4). Patients’ characteristics including delirium relevant previous medical history and treatments are described in [Table 1].

Table 1.

Patients Demographics, Administered Medications and Delirium Motor Subtype.

Demographics ^◊	N = 22
Age (years) *	65 (14)
Sex: Male**	16 (73%)
PMH**
Psychiatric/ Neurological disorder	5 (23%)
Dementia	2 (9%)
Alcohol or drug abuse	3 (14%)
Admission characteristics ** ^◊
Admitted to CRITICAL CARE (level 3)	18 (82%)
Communicate verbally ^┬	16 (73%)
Administered AAP	14 (64%)
Quetiapine	8 (57%)
Olanzapine	4 (29%)
Risperidone	2 (14%)
Administered Sedatives	13 (59%)
Propofol	7 (32%)
Clonidine	6 (27%)
Lorazepam	4 (18%)
Administered Analgesics	17 (77%)
Paracetamol	11 (50%)
Fentanyl	4 (18%)
Delirium Motor subtype^▴
Hyperactive	4 (18%)
Hypoactive	1 (5%)
Mixed	17 (77%)

N number of recruited subjects, PMH pre-admission medical history, * mean (standard deviation), ** count (percentage), ◊ on assessment day, ^┬ communicate by mouthing or through speaking valve in mechanically intubated patients, ^▴ using DRS-R98 motor subtype items 7 and 8 classification criteria

Thirty five percent patients were classified with mild, 40% moderate and 25% severe delirium.

In 17 patients (77%) the motor subtype was reported as mixed, four (18%) were hyperactive delirium and one (5%) hypoactive delirium.

A summary of the mean (+/− standard deviation) of DRS-R98 (total and severity) overall and individual items scores calculated and Clinical Global Impression severity scale scores, are provided in [Table 2].

Table 2.

Mean and Standard Deviation of the DRS-R98 (Total and Severity) Overall and Individual Items Scores, Internal Consistency, and CGI-S Scores.

Scale/ item	Mean (SD)*	Missing data** N (%)	r for DRS-R98 severity scale	r for DRS-R98 total scale
1. Sleep-wake Cycle Disturbance	1.9 (0.9)	0 (0)	0.883	0.874
2. Perceptual Disturbance	1.4 (0.9)	1 (5)	0.913	0.901
3. Delusions	1.2 (0.9)	5 (23)	0.905	0.893
4. Affect lability	1.8 (1.0)	0 (0)	0.894	0.882
5. Language	2.1 (0.9)	4 (18)	0.883	0.874
6. Thought process	1.8 (0.8)	8 (36)	0.882	0.872
7 Motor agitation	1.8 (0.9)	0	0.891	0.880
8. Motor retardation	1.6 (1.1)	0	0.889	0.876
9. Orientation	2.2 (0.9)	1 (5)	0.884	0.874
10.Attention	2.1 (0.8)	2 (9)	0.880	0.870
11 Short term memory	2.0 (0.8)	6 (27)	0.879	0.870
12. Long term Memory	2.1 (0.7)	7 (32)	0.879	0.870
13 Visuospatial ability	1.9 (0.9)	8 (36)	0.877	0.867
14.Temporal onset of symptoms	2.5 (0.5)	0	NA	0.903
15.Fluctuation of symptoms severity	1.2 (0.4)	0	NA	0.881
16.Physical disorder	1.5 (0.5)	0	NA	0.882
DRS-R98 severity scale score	24 (7.7)	NA	NA	NA
DRS-R98 scale total score	29 (8.0)	NA	NA	NA
CGI-S	3.5 (1.5)	NA	NA	NA
Overall Cronbach alpha	NA	NA	0.896	0.886

* Mean and standard deviation calculated using imputed data set, N number of patients, ** missing data analysed using the non-imputed data set scored by primary assessors, r Cronbach's Alpha coefficient after item deletion, NA not applicable, DRS-R98 severity scale includes items 1 to 13, DRS-R98 total scale includes items 1 to 16

Main Results

The DRS-R98 severity scale score (r = 0.626, p = 0.002) and total score (r = 0.628, p = 0.002) correlated with CGI-S score. The DRS-R98's internal consistency was high, with Cronbach's alpha coefficient of 0.896 for DRS-R-98 severity scale items and 0.886 for the total scale. When the effect of individual items were deleted, coefficients of DRS-R98 severity scale ranged from 0.877 to 0.913 and for the total scale items from 0.867 to 0.903. Table 2 provides a summary of DRS-R98 internal consistency for both total and the severity scales (including item deletion).

The intraclass correlation coefficient (ICC) calculated between scores of paired assessors in six patients was moderate. The ICC was 0.505 (95% CI −0.988-0.961, p = 0.124) for the DRS-R98 severity scale and for 0.565 (95% CI −0.403-0.912, p = 0.093) for the total scale.

The scale's sensitivity to change overtime was not calculated, this was because none of the pre-assessed patients fully recovered from delirium and therefore qualified for post-assessment evaluation during their critical care admissions and approval was only given for critical care areas.

Scale Implantation, Duration and Missing Data

The mean time for data extraction and collection process was 90 min (SD +/- 55), and mean duration for clinical assessment and DRS-R98 scoring by primary assessor was 15 min (SD+/- 5).

Fifteen (68%) of assessed cases had incomplete data, with a total of 43 missing values (12.2%). The most common reason (n = 40, 93%) for incomplete data was inability to assess.

The DRS-R98 scale items with no missing data were item 1 (sleep awake cycle), item 4 (affect liability), item 7 (motor agitation), item 8 (motor retardation), item 14 (temporal onset), item 15 (fluctuation) and item 16 (physical disorder). The DRS-R98 items with most missing data values were item 13 (visual spatial ability, 33% and 38% respectively), item 12 (long term memory, 33% and 31%) and item 11 (short term memory, 33% and 25%) as rated by MP and MB sucessively.

The percentage of missing data values for DRS-R98 items scored varied between experts MP and MB (19.7% and 9.3%, respectively).

Discussion

In this project, we describe the validation of the DRS-R98 in 22 critical care patients in delirium. As far as we are aware, this is the first validation report for the DRS-R98 in a critical illness setting.

Our work demonstrates that the DRS-R98, when applied in critical care, exhibited a strong and significant correlation with the CGI-severity scale; this suggests excellent construct validity. The DRS-R98's Cronbach's alpha coefficient which assesses for internal consistency and homogeneity was excellent, even after individual item deletion. These findings were aligned with the DRS-R98 original validation work and validation studies in different inpatient settings.^18,19,59

In 2018, Jones et al published a systematic review of delirium severity instruments and DRS-R98 had one of the highest rating.⁶⁵ The DRS-R98's multidimensional structure gives a detailed description of delirium phenomenology and prevention/treatment strategies.⁶⁶

Many critically ill patients (eg, intubated patients, severely ill) have limited ability to engage in interviews and assessments especially those with a predominant verbal component.⁶⁷ One of the advantages of the DRS-R98 is that it integrates information resources including direct observation, staff interview and medical records,⁵⁰ and thus more favourable to critical care. Furthermore, the DRS-R98 is one of the few severity assessment tools with a detailed application manual, where individual items are anchored with clear descriptions.⁵⁰ This further supports the severity assessment process. Finally using a numeric scale rather than a categorical screen. eg, CAM-ICU gives the opportunity to assess the severity of a patient's delirium over time and could, one could argue be better suited to delirium's fluctuant nature.

The project had several strengths including pragmatic design which reflected the less restrictive recruitment criteria, its implementation by primary assessors who were experts and able to complete the assessment using the DRS-R98 manual without additional training. Moreover, the recruited patient scores included patients with different stages of delirium severity and with a higher mean delirium severity scores than those reported in earlier validation studies where severe delirium is frequently underrepresented.^{13,15,16,18,20–22,59}

We acknowledge that the project had its limitations. Firstly, the DRS-R98 construct validity was only assessed via correlation with CGI-S. At the design stage, other concurrent validity evaluation methods were considered. This included correlating the DRS-R98 scores with other delirium severity assessment scales, but the investigators considered this impossible to achieve as, at the time, there were no delirium severity scales that were validated in critical illness.

A second limitation was that the DRS-R98 scores were not correlated with clinical outcomes such as length of stay and mortality. This was too difficult to achieve because of the study's irregular collection schedule and lack of access to patient data once discharged from the critical care area (our approval to undertake the DRS-R98 as a service evaluation was strictly for critical care).

Moreover, the inter-rater reliability was determined by calculating the intra-class correlation (ICC) coefficient. The ICC calculated between MP and AP scores was moderate and not statistically significant. This was lower than that reported in the original and majority of validation studies, which consistently reported excellent (≥ 0.9) and statistically significant ICC.^{15,16,18,19–22,59} Reasons for that were that we were unable to recruit our a priori agreed sample size for the assessment and despite being undertaken by experts it was not preceded with extensive training. The IRR was only assessed in six patients, because the second assessor had additional clinical commitments that took precedence, and we were unable to recruit a qualified replacement.

Additionally, despite planning we were unable to assess patients after delirium resolution. This is because patients were discharged from critical care area before assessors could evaluate them on a second occasion. As the aims of this study were to describe our entire experience of utilising the DRS-R98 in critical illness; we believe it was essential to report all planned activity including logistical challenges that prevented our pre-planned goals. Future delirium severity validation projects should consider reassessing the IRR with a larger sample size; extensive training on the application of the DRS-R98; and assessing the DRS-R98's ability to detect changes in delirium over time in a critical care setting.

Finally, the principal reason for selecting the DRS-R98 is that it integrates data from multiple resources. However, this requires time and effort to collect, compare, and evaluate each of the individual items or symptoms. The DRS-R98 collection process consisted of two stages: stage one required detailed review of medical records, which was time consuming, although could be conducted in part, remotely from the ICU (Figure 3, describes assessment stages locations and durations). This extraction process could be improved by designing a data extraction form and improving the skill and knowledge of the extractor. Further, the duration for this stage is expected to reduce over time, with greater familiarity with process, patient and automated electronic capture of key data.

Figure 3.

DRS-R98 assessment location, duration and impacting factors.

The second stage of data collection. Unlike the first stage, was short with a median duration of 15 min. The DRS-R98 is designed to be implemented by a psychiatrist or other healthcare professional who must be trained in assessing psychiatric conditions and who can solicit information expertly from patients. This was demonstrated, as several assessed patients could not communicate verbally, cannot move, and/or declined to interact. Despite the assessor's best efforts, this wasn’t always attainable and thus we were not surprised to report that items dependent on interview and clinical assessment (eg, visual spatial ability, memory) were the most challenging to evaluate. Whereas items that depended on other information resources (eg, sleep disorder, motor agitation), were easier to evaluate and complete. On reflection, discussion after each assessment session, may help determine how missing items may best be assessed.

Our overall impression is that the DRS-R98 is a potential tool in research and for complex delirium case evaluation in a critical care setting. Although its lengthy process and expert staff to extract and assess the relevant information could limit its application for routine clinical use.

Conclusion

This study has demonstrated that the DRS-R98 can successfully assess delirium severity in a critical care setting. It has demonstrated good construct validity, excellent internal consistency, and moderate inter-rater reliability. In clinical practice, the DRS-R98 could be applied as structured approach to assess complex delirium in critical care. In research, the tool could have a valuable role in describing delirium phenomenology and assessing response to delirium prevention and treatment approaches.

Footnotes

Acknowledgment and Credits

The authors recognise the contribution of Prof Paula T. Trzepacz.

Declaration of Conflicting Interest

All authors though have no competing interests to declare that are relevant to the content of this article.

Funding

The research is part of the first author's PhD degree, except for the student bench fess, the author(s) received no financial support for the research or authorship of this article. The article open access was provided by Southhampton Unversity.

ORCID iDs

Eiman Saeed Almuhairi

Cathrine A McKenzie

References

Ely

Girard

Shintani

, et al. Apolipoprotein E4 polymorphism as a genetic predisposition to delirium in critically ill patients. Crit Care Med. 2007;35(1):112-117.

Zaal

Spruyt

Peelen

, et al. Intensive care unit environment may affect the course of delirium. Intensive Care Med. Mar 2013;39(3):481-488. doi:10.1007/s00134-012-2726-6

Vasilevskis

Han

Hughes

Ely

. Epidemiology and risk factors for delirium across hospital settings. Best Pract Res Clin Anaesthesiol. 2012;26(3):277-287. https://doi.org/10.1016/j.bpa.2012.07.003

Adamis

Sharma

Whelan

PJP

MacDonald

AJD

. Delirium scales: A review of current evidence. Aging Ment Health. 2010;14(5):543-555. doi:10.1080/13607860903421011

Rosgen

Krewulak

Stelfox

, et al. The association of delirium severity with patient and health system outcomes in hospitalised patients: a systematic review. Age Ageing. 2020; 49(4):549–557. doi:10.1093/ageing/afaa053

Marcantonio

Kiely

Simon

, et al. Outcomes of older people admitted to postacute facilities with delirium. Article; Proceedings Paper. J Am Geriatr Soc. Jun 2005;53(6):963-969. doi:10.1111/j.1532-5415.2005.53305.x

Cole

McCusker

Dendukuri

Han

. The prognostic significance of subsyndromal delirium in elderly medical inpatients. J Am Geriatr Soc. Jun 2003;51(6):754-760. doi:10.1046/j.1365-2389.2003.51255.x

Rose

Agar

Burry

, et al. Reporting of outcomes and outcome measures in studies of interventions to prevent and/or treat delirium in the critically ill: A systematic review. Crit Care Med. Apr 2020;48(4):e316-e324. doi:10.1097/ccm.0000000000004238

Rose

Agar

Burry

, et al. Development of core outcome sets for effectiveness trials of interventions to prevent and/or treat delirium (Del-COrS): study protocol. BMJ Open. 2017;7(9):e016371. doi:10.1136/bmjopen-2017-016371

10.

Vreeswijk

Timmers

de Jonghe

JFM

Kalisvaart

. Assessment scales for delirium. Aging Health. 2009;5(3):409-425. doi:10.2217/ahe.09.10

11.

Grover

Kate

. Assessment scales for delirium: A review. World J Psychiatry. Aug 22 2012;2(4):58-70. doi:10.5498/wjp.v2.i4.58

12.

Skrobik

Bergeron

Dumont

Gottfried

. Olanzapine vs haloperidol: Treating delirium in a critical care setting. Intensive Care Med. Mar 2004;30(3):444-449. doi:10.1007/s00134-003-2117-0

13.

Sharma

Malhotra

Grover

Jindal

. Incidence, prevalence, risk factor and outcome of delirium in intensive care unit: A study from India. Gen Hosp Psychiatry. Nov-Dec 2012;34(6):639-646. doi:10.1016/j.genhosppsych.2012.06.009

14.

Khan

Perkins

Gao

, et al. The confusion assessment method for the ICU-7 delirium severity scale: A novel delirium severity instrument for use in the ICU. Crit Care Med. 2017;45(5):851-857. doi:https://dx-doi-org-s.web.bisu.edu.cn/10.1097/CCM.0000000000002368

15.

Fonseca

Bulbena

Navarrete

, et al. Spanish version of the Delirium Rating Scale-Revised-98: Reliability and validity. J Psychosom Res. Sep 2005;59(3):147-151. doi:10.1016/j.jpsychores.2004.04.373

16.

de Negreiros

da Silva Meleiro

Furlanetto

Trzepacz

. Portuguese Version of the Delirium Rating Scale-Revised-98: Reliability and validity. Int J Geriatr Psychiatry. May 2008;23(5):472-477. doi:10.1002/gps.1906

17.

Franco

Trzepacz

Mejia

Ochoa

. Factor analysis of the Colombian translation of the Delirium Rating Scale (DRS), Revised-98. Psychosomatics. May-Jun 2009;50(3):255-262. doi:10.1176/appi.psy.50.3.255

18.

Kato

Kishi

Okuyama

Trzepacz

Hosaka

. Japanese Version of the Delirium Rating Scale, Revised-98 (DRS-R98-J): Reliability and validity. Psychosomatics. Sep-Oct 2010;51(5):425-431.

19.

Lee

Ryu

Lee

, et al. Korean Version of the delirium rating scale-revised-98: Reliability and validity. Psychiatry Investig. Mar 2011;8(1):30-38. doi:10.4306/pi.2011.8.1.30

20.

Huang

Lee

Lai

Kao

Lin

Chen

. Chinese Version of the Delirium Rating Scale-Revised-98: Reliability and validity. Compr Psychiatry. Jan-Feb 2009;50(1):81-85. doi:10.1016/j.comppsych.2008.05.011

21.

de Rooij

van Munster

Korevaar

, et al. Delirium subtype identification and the validation of the Delirium Rating Scale–Revised-98 (Dutch version) in hospitalized elderly patients. Int J Geriatr Psychiatry. Sep 2006;21(9):876-882. doi:10.1002/gps.1577

22.

Cinar

Ozmenler

Ozsahin

Trzepacz

. Reliability and validity of the Turkish translation of the Delirium Rating Scale-Revised-98. Düþünen Adam. 2014;27(3):186-193. doi:10.5350/DAJPN2014270301

23.

Grover

Kate

Agarwal

, et al. Delirium in elderly people: A study of a psychiatric liaison service in north India. Int Psychogeriatr. Jan 2012;24(1):117-127. doi:10.1017/s1041610211001815

24.

Leonard

Raju

Conroy

, et al. Reversibility of delirium in terminally ill patients and predictors of mortality. Palliat Med. Oct 2008;22(7):848-854. doi:10.1177/0269216308094520

25.

Meagher

Ryan

O'Regan

O'Caoimhe

Timmons

. A point prevalence study of delirium in an adult acute hospital population: Predictors of accurate detection. Eur Psychiatry. 2012;27(S1):1.

26.

Meagher

Leonard

Donnelly

Conroy

Adamis

Trzepacz

. A longitudinal study of motor subtypes in delirium: Relationship with other phenomenology, etiology, medication exposure and prognosis. J Psychosom Res. 2011;71(6):395-403. https://dx-doi-org.web.bisu.edu.cn/10.1016/j.jpsychores.2011.06.001

27.

Slor

Adamis

Jansen

, et al. Delirium motor subtypes in elderly hip fracture patients: Risk factors, outcomes and longitudinal stability. J Psychosom Res. May 2013;74(5):444-449. doi:10.1016/j.jpsychores.2012.12.007

28.

Grover

Agarwal

Sharma

, et al. Symptoms and aetiology of delirium: A comparison of elderly and adult patients. Eas As A Psyc. 2013;23(2):56-64.

29.

Grover

Sharma

Aggarwal

, et al. Comparison of symptoms of delirium across various motoric subtypes. Psychiatry Clin Neurosci 2014;68(4):283-291. doi:10.1111/pcn.12131

30.

Leonard

Donnelly

Conroy

Trzepacz

Meagher

. Phenomenological and neuropsychological profile across motor variants of delirium in a palliative-care unit. J Neuropsychiatry Clin Neurosci. 2011;23(2):180-188.

31.

Trzepacz

Franco

Meagher

, et al. Phenotype of subsyndromal delirium using pooled multicultural Delirium Rating Scale–Revised-98 data. J Psychosom Res. Jul 2012;73(1):10-17. doi:10.1016/j.jpsychores.2012.04.010

32.

Tieges

Brown

LJE

MacLullich

AMJ

. Objective assessment of attention in delirium: A narrative review. Int J Geriatr Psychiatry. Dec 2014;29(12):1185-1197. doi:10.1002/gps.4131

33.

de Jonghe

JFM

Kalisvaart

Dijkstra

, et al. Early symptoms in the prodromal phase of delirium: A prospective cohort study in elderly patients undergoing hip surgery. Am J Geriatr Psychiatry. 2007;15(2):112-121. doi:10.1097/01.JGP.0000241002.86410.c2

34.

Kalisvaart

de Jonghe

Bogaards

, et al. Haloperidol prophylaxis for elderly hip-surgery patients at risk for delirium: A randomized placebo-controlled study. J Am Geriatr Soc. 2005;53:1658-1666.

35.

Larsen

Kelly

Stern

, et al. Administration of olanzapine to prevent postoperative delirium in elderly joint-replacement patients: A randomized, controlled trial. Psychosomatics. 2010;51(5):409-418. doi:10.1016/s0033-3182(10)70723-4

36.

Mouzopoulos

Vasiliadis

Lasanianos

Nikolaras

Morakis

Kaminaris

. Fascia iliaca block prophylaxis for hip fracture patients at risk for delirium: A randomized placebo-controlled study. J Orthop Traumatol. 2009;10(3):127-133. doi:10.1007/s10195-009-0062-6

37.

Hempenius

Slaets

JPJ

van Asselt

de Bock

Wiggers

van Leeuwen

. Outcomes of a geriatric liaison intervention to prevent the development of postoperative delirium in frail elderly cancer patients: Report on a multicentre, randomized, controlled trial. PloS One. 2013;8(6):e64834.

38.

Zisapel

. Melatonin-dopamine interactions: From basic neurochemistry to a clinical setting. Cell Mol Neurobiol. Dec 2001;21(6):605-616. doi:10.1023/a:1015187601628

39.

Bee Gek Tay

Chew Chan

Sian Chong

. Functional improvement in hospitalized older adults is independent of dementia diagnosis: Experience of a specialized delirium management unit. J Hosp Med. 2013;8(6):321-327. doi:10.1002/jhm.2035

40.

Lee

K-U

Won

W-Y

Lee

H-K

, et al. Amisulpride versus quetiapine for the treatment of delirium: A randomized, open prospective study. Int Clin Psychopharmacol. 2005;20(6):311-314.

41.

Straker

Shapiro

Muskin

. Aripiprazole in the treatment of delirium. Psychosomatics. 2006;47(5):385-391. doi:10.1176/appi.psy.47.5.385

42.

Tahir

Eeles

Karapareddy

, et al. A randomized controlled trial of quetiapine versus placebo in the treatment of delirium. J Psychosom Res. Nov 2010;69(5):485-490. doi:10.1016/j.jpsychores.2010.05.006

43.

Yoon

Park

Choi

, et al. Efficacy and safety of haloperidol versus atypical antipsychotic medications in the treatment of delirium. BMC Psychiatry. 2013;13:240. doi:10.1186/1471-244X-13-240

44.

Yoo

Kim

Bae

, et al. Risperidone versus olanzapine for the treatment of delirium. Eur Neuropsychopharmacol. Sep 2011;21(S3):S493-S493. doi:10.1016/S0924-977X(11)70800-8

45.

Sakong

Jung

Kim

Koo

. P.5.e.002 comparison between effectiveness of aripiprazole and risperidone for treatment of delirium in hospitalized patients. Eur Neuropsychopharmacol. 2010;20(S3):S568-S569.

46.

Maneeton

Srisurapanont

Chittawatanarat

. Quetiapine versus haloperidol in the treatment of delirium: A double-blind, randomized, controlled trial. Drug Des Devel Ther. 2013;7:657-667. doi:10.2147/DDDT.S45575

47.

Campbell

Khan

Farber

, et al. Improving delirium care in the intensive care unit: The design of a pragmatic study. Trials. 2011;12:139. doi:10.1186/1745-6215-12-139

48.

Andrew Retter

Yassin

. Guideline for the management of analgesia, sedation and delirium (ASD) in critical care – Main guidance. 2016.

49.

Bell

. DSM-IV: Diagnostic and statistical manual of mental disorders. JAMA. 1994;272(10):828-829. doi:10.1001/jama.1994.03520100096046

50.

Trzepacz

Kean

Abell

Meagher

. DRS-R98 Administration Manual, A guide to increase understanding of how to solicit delirium symptoms when administering the DRS-R98. December 2010.

51.

Pisani

Araujo

Van Ness

Zhang

Ely

Inouye

. A research algorithm to improve detection of delirium in the intensive care unit. Crit Care. 2006;10(4):R121. doi:10.1186/cc5027

52.

Inouye

Leo-Summers

Zhang

Bogardus

Leslie

Agostini

. A chart-based method for identification of delirium: Validation compared with interviewer ratings using the confusion assessment method. J Am Geriatr Soc. 2005;53(2):312-318. doi:10.1111/j.1532-5415.2005.53120.x

53.

Ely

Inouye

Bernard

, et al. Delirium in mechanically ventilated patients: Validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA. 2001;286(21):2703-2710.

54.

Busner

Targum

. The clinical global impressions scale: Applying a research tool in clinical practice. Psychiatry (Edgmont). 2007;4(7):28-37.

55.

Trzepacz

. The delirium rating scale-revised-98: Comparison with the delirium rating scale and the cognitive test for delirium (vol 13, pg 229, 2001). J Neuropsychiatry Clin Neurosci 2001;13(3):433-433. doi:10.1176/jnp.13.2.229

56.

Meagher

Leonard

Donnelly

Conroy

Saunders

Trzepacz

. A comparison of neuropsychiatric and cognitive profiles in delirium, dementia, comorbid delirium-dementia and cognitively intact controls. J Neurol Neurosurg Psychiatry. Aug 2010;81(8):876-881. doi:10.1136/jnnp.2009.200956

57.

Padhi

Fineberg

. Clinical global impression scales. In: Stolerman

, ed. Encyclopedia of Psychopharmacology. Springer; 2010:303-303.

58.

Meagher

Moran

Raju

, et al. Motor symptoms in 100 patients with delirium versus control subjects: Comparison of subtyping methods. Psychosomatics: J Consult Liaison Psychiatry. 2008;49(4):300-308.

59.

Trzepacz

Mittal

Torres

Kanary

Norton

Jimerson

. Validation of the delirium rating scale-revised-98: Comparison with the delirium rating scale and the cognitive test for delirium. J Neuropsychiatry Clin Neurosci. 2001;13(2):229-242. doi:10.1176/appi.neuropsych.13.2.229

60.

Bujang

Baharum

. Sample size guideline for correlation analysis. World. 2016;3(1):37-46.

61.

Trzepacz

Mittal

Torres

Kanary

Norton

Jimerson

. Validation of the Delerium Rating Scale-Revised-98 (DRS-R-98). Eur Neuropsychopharmacol. September 2000;10(Supplement 3):S389-S390. doi:10.1016/S0924-977X(00)80541-6

62.

Hallgren

. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34.

63.

Hawthorne

Elliott

. Imputing cross-sectional missing data: Comparison of common techniques. Aust N Z J Psychiatry. 2005;39(7):583-590. doi:10.1080/j.1440-1614.2005.01630.x

64.

NHS. NHS data model and dictionary. NHS, 2022. https://www.datadictionary.nhs.uk/data_elements/critical_care_level.html. Accessed June 20, 2022.

65.

Jones

Cizginer

Pavlech

, et al. Assessment of instruments for measurement of delirium severity: A systematic review. JAMA Intern Med. 2019;179(2):231–239. doi:10.1001/jamainternmed.2018.6975

66.

Vasunilashorn

Schulman-Green

Tommet

, et al. New delirium severity indicators: Generation and internal validation in the better assessment of illness (BASIL) study. Dement Geriatr Cogn Disord. 2020;49(1):77-90. doi:10.1159/000506700

67.

Eddleston

JGDMJ

. Levels of Critical Care Adult patients. Intensive Care Soc Joint descriptive document 2009. https://icmwk.com/wp-content/uploads/2014/02/Revised-Levels-of-Care-21-12-09.pdf