The feasibility of videoconferencing for neuropsychological assessments of rural youth experiencing early psychosis

Introduction

Although telepsychiatry has been used for clinical assessment in rural and remote areas, there is limited evidence for its reliability with neuropsychological assessment in young people with early psychosis. One study comparing videoconferencing and face-to-face neuropsychological assessment for adults with mental health problems reported similar results for both modes of assessment and high consumer satisfaction for videoconferencing. ¹ Conventionally, neuropsychological assessments have been conducted using pencil and paper or computer-based testing. If these assessments could be done by videoconferencing instead, it is likely that sample sizes for research could be increased by improving recruitment and retention rates, and that clinicians would have greater access to specialist services.

A review of telepsychiatry research showed that clinical diagnostic assessments by telepsychiatry were reliable, resulted in improved clinical outcomes, and were favourably received by clinicians and consumers. ² In addition to clinical assessments, neuropsychological assessments including tests of attention, memory and learning are an important component of research, contribute to clinical outcomes in early psychosis ³ and may be predictive of functional outcome among young people with early-onset symptoms. ⁴ We have conducted a pilot study of the feasibility of using videoconferencing to administer neuropsychological assessment to young people with early psychosis in rural Australia.

Method

Participants aged 14–30 years who met the criteria for early psychosis were recruited through mental health services within a 170 km radius (for accessibility of face-to-face assessment) of Orange, New South Wales. Early psychosis was defined as being within two years of the onset of psychotic symptoms at the time of assessment. Diagnosis was confirmed with the Diagnostic Interview for Psychosis Diagnostic Module (DIP-DM). ⁵ Participants were excluded if there was evidence of an organic mental disorder or an inadequate command of English (which was necessary for the neuropsychological assessment). Participants received $20 reimbursement for time and travel. The study was approved by the appropriate ethics committees.

Recruitment was conducted through community health and community mental health services, psychiatric inpatient units, non-government crisis services, general practitioners and youth organizations. Following informed consent, the DIP-DM was administered face-to-face to all participants to confirm diagnosis. Participants were each assessed using videoconferencing and face-to-face administration with the following instruments:

Results

The 11 participants (5 male) had a mean age of 20 years (range 14–27) and most were unemployed (n = 9). Two participants had completed the Higher School Certificate (Year 12); one had completed the School Certificate (Year 10); one had a trade or technical qualification; and seven had left school prior to completing Year 10. All participants met DSM-IV diagnostic criteria for a psychotic disorder (schizophrenia, 5; schizoaffective disorder, 2; psychosis not otherwise specified, 3; major depressive disorder with psychotic features, 1). The median duration of illness was 104 weeks (range 16–364).

The Opiate Treatment Index (OTI) was excluded from the analysis owing to missing data. Mean scores for the face-to-face and videoconference assessment measures are shown in Table 1.

Correlations between the face-to-face and videoconference modes of assessment were significant for the Wechsler Test of Adult Reading (WTAR), the Controlled Oral Word Association Test (COWAT), the Logical Memory subtest of the Wechsler Memory Scale and the BPRS (Table 1). There was perfect agreement between modes for the SOFAS and thus it was not included in further analyses. Non significant correlations were found for the Digit Span subtest. The Bland-Altman plots (Figure 1) and analysis (Table 1) indicated that in general the mean difference between face-to-face and videoconference modes of assessment was close to zero with significant bias only evident for WTAR, where videoconference produced higher ratings than face-to-face assessments. However, despite the mean differences between face-to-face and videoconference being close to zero for most assessment items, the limits of agreement were rather wide. This was partly due to the small sample size but also indicated poor reproducibility for some assessments. This was particularly problematic in the case of WTAR and to a lesser degree BPRS.

OPEN IN VIEWER

Figure 1

Bland-Altman plots for (a) BPRS, (b) AQOL, (c) WTAR scores, (d) Digit Span, (e) COWAT, (f) Logical Memory. Horizontal lines indicate the mean difference (mean bias) and 95% limits of agreement

Consumer satisfaction questionnaires were completed by six participants who reported that they felt comfortable and could understand the instructions during the videoconference assessment. Five stated that they would recommend a videoconference interview for someone who needed help and only one said that they definitely did not prefer the videoconference mode to the face-to-face mode of assessment.

Discussion

The majority of cognitive tasks completed by young people with early psychosis demonstrated equivalence of face-to-face and videoconference methods of administration. This is important because it suggests that these clients can perform equally well on cognitive tasks, whether administered by a clinician in the same room or by videoconference. Difficulties were encountered when administering the quality of life measure by videoconference owing to the multiple-choice format. We recommend providing a respondent booklet for participant reference in future. For cognitive assessment, the participants performed equally across both modes of assessment on the demanding verbal memory task. While there was a strong positive association between assessment modes for scores on the WTAR, analyses showed assessment mode differences such that participants performed better on the WTAR by videoconference than face-to-face. The clinical importance of this finding requires further investigation with a larger sample.

The present study improved service delivery and research in rural and remote communities by identifying videoconference facilities in the rural Central West region of NSW. It also increased exposure, awareness and experience among clinicians on the feasibility of videoconferencing for neuropsychological assessments with young people experiencing early psychosis.

The present study had some limitations, including the small sample size and the reluctance of clinicians to refer clients considered too unwell, limiting the reliability and generalizability of the findings to the wider population. The lack of reliable medication ratings for participants impedes assessment of the influence of medication on test performance. The low referral rate may indicate that young people experiencing early psychosis are managed outside the mainstream mental health services. The low referral rate may also indicate an underlying culture of resistance to technologically mediated clinical interaction ¹⁵ and its value for clients. The increasing use of videoconferencing in rural areas for professional development activities as well as studies such as this may increase clinicians' familiarity with the technology and facilitate access to specialist services. In addition, researchers may build on the present study by researching a full range of neuropsychological measures useful for early psychosis research and service provision.

While only tentative conclusions can be drawn from our study due to the small sample size, the results suggest that the two modes of neuropsychological assessment produce equivalent results on some measures. There are clear benefits of neuropsychological assessments via videoconferencing where face-to-face options are unavailable including reduced service costs. ¹⁶ Anecdotally, clinicians have expressed a clear interest in having a neuropsychological assessment for early psychosis clients due to the benefit of such an assessment for treatment planning and the ability to access clinical expertise that may not be readily available in the local clinical setting.