Assessing Executive Functioning in Schizophrenia: Concurrent and Discriminative Validity of a Novel Virtual Cooking Task

Abstract

Deficits in executive functions (EF) are strongly related to real-life functioning and negative symptoms (NS) in schizophrenia. Recently, virtual reality has enabled more ecologically valid approaches to assess EF in simulated “real-life” scenarios among which the virtual cooking task (VCT) has gained attention. However, the clinical implications of the VCT in schizophrenia have not been investigated exhaustively. In this study, clinically stable individuals with schizophrenia (n = 38) and healthy controls (n = 42) completed a novel VCT and a set of computerized standard EF tools (CST) to primarily investigate concurrent and discriminant validity. In addition, the study explored links between EF assessments, functioning, and NS while controlling for antipsychotic intake, clinical stability, and age. This VCT consisted of four tasks with increasing difficulty and time constraints. The most relevant findings indicate that (1) the VCT showed moderate to strong correlations with CST, (2) the VCT discriminated EF performance between both the groups, (3) the VCT predicted interpersonal functioning, and (4) the VCT predicted NS in greater extent than CST. Accordingly, the findings give support to the concurrent and discriminant validity of the VCT to assess EF and indicate its value to deepen the study of collateral functional deficits and NS in schizophrenia.

Introduction

Schizophrenia is a heterogeneous disorder characterized by the confluence of three main symptoms domains: positive, negative, and cognitive.¹ It is associated with a high global burden of disability, as many patients experience difficulties in securing employment, living independently, and maintaining interpersonal relationships.² In this sense, greater functional impairment is associated with higher negative symptoms (NS) and greater cognitive dysfunction.³ Quantifying the cognitive deficits associated with loss of functionality is challenging. Until now, this has been done using traditional executive function (EF) analysis tasks, although they may not reflect the actual impairment in real-life situations.⁴ Therefore, virtual reality (VR) systems have emerged as a promising methodology for analyzing these functions in realistic contexts.⁵ In this study, we assess the concurrent and discriminative validity of an innovative VR system for evaluating EF in schizophrenia.

Deficits in EF have been identified as a critical dimension of cognitive dysfunctions in schizophrenia.^6–8 They comprise higher-order processes including cognitive flexibility, working memory, inhibitory control, planning, reasoning, and problem-solving.⁹ According to the Frith’s neurocognitive theory of schizophrenia,¹⁰ the impairment of EF would condition disordered “willed behavior,” generating deficits in goal-directed actions and leading to NS of schizophrenia, such as abulia, social withdrawal, or alogia.¹¹ Specifically, deficits in EF result in patients failing to (1) form willed intentions based on current goals, (2) make a sufficient link between goals and the actions required for the action, or (3) inhibit stimulus-driven behaviors not related to the goal. The theory is supported by the literature, demonstrating the concurrence between deficits in EF and the NS of schizophrenia,^11–14 which are predictors of functional performance.^14,15

Traditionally, EF impairments have been evaluated by standard performance-based tasks under repetitive conditions (e.g., the STROOP test¹⁶). However, despite its extended use and value, previous research has indicated moderate effectiveness in predicting real-life functioning, thus, questioning their ecological validity^17,18 and pointing to the need for developing new methods to assess EF.¹⁹ In this context, VR tools have emerged as a promising ecologically valid methodological approach with potential to improve the efficiency and reliability of standard performance-based tasks.⁵ Different research groups have developed VR paradigms to explore their capabilities in assessing EF in schizophrenia.^20–22 These paradigms are based on nonimmersive environments, where interaction occurs with elements displayed on screens or projectors, which emulate real-life scenarios, such as supermarkets,^23–25 medication management,^26,27 cities,²⁸ preparing a meeting,^29,30 choosing a bus route,³¹ or scheduling work-related activities.³² These studies have confirmed the ability of VR systems to detect the presence of EF impairments in schizophrenia,²² mainly in planning,^{24,25,29,31,33} working,^{21,25,29,31,34} and prospective memory.^31,35 In addition, some studies have identified limitations in other domains such as decision-making,^21,31 multitasking,²⁹ switching, problem-solving, or stimulus integration.^21,34 However, these works leave unresolved questions regarding VR systems, including (1) their ability to detect features related to EF deficits, such as NS or functional impairment²²; (2) the influence of antipsychotic treatment on these assessments, as these medications could affect symptoms and performance^36,37; and (3) the degree of impairment in crucial EF domains, such as inhibitory control.

To our knowledge, no previous studies have focused on evaluating EF in individuals with schizophrenia through fully immersive VR systems (involving the use of head-mounted displays). These immersive systems have generated interest for their utilization in clinical intervention settings.²² In addition, they have greater capacity to generate a sense of presence, realism, and higher ecological validity than nonimmersive systems.³⁸ However, in scientific research, immersive environments have been underutilized, given the difficulty of controlling the basic characteristics for a strict experimental design.²² The study here approaches these issues by using our own immersive VR cooking task, the virtual cooking task (VCT), developed to assess EF. The VCT has been tested on healthy participants, demonstrating good ability to produce high levels of presence, feasibility, and usability.³⁹ It has also shown the capacity to analyze selective attention, cognitive shifting, or planning when assessing patients with alcohol use disorders.⁴⁰ Furthermore, a previous study has observed promising results of another immersive VR cooking task in assessing cognition in mood disorders and psychosis spectrum disorders.⁴¹ They found significant correlations between the VR system and computerized standard tasks (CST), besides sensitivity to cognitive impairments when comparing with healthy controls, and significant associations with functional assessments. Nevertheless, the results must be interpreted with caution, as the sample was only partially composed of patients with schizophrenia who were also in an early stage of the disease, often associated with mild cognitive impairment.⁴² In addition, it also exhibits the limitations described earlier, such as the absence of analyses exploring the association with schizophrenia symptomatology,²² the lack of control for antipsychotic treatment,⁴³ or failing to differentiate between EF subdomains.

The specific goals of this study were the comprehensive assessment of EF in individuals with schizophrenia and healthy controls through a battery of CST and the VCT, to compare EF in individuals with schizophrenia and healthy controls in both CST and VCT assessments, and to analyze the association between EF measured by VCT with daily functioning and NS of schizophrenia. The hypotheses were (1) VCT assessments will be significantly associated with CST and^39,40 (2) participants with schizophrenia will perform worse on both assessments compared with healthy control.²² Finally, and following the Frith’s neurocognitive theory of schizophrenia,¹⁰ EF deficits will predict impaired functioning^14,15 (3) and NS of schizophrenia (4).^11–14

Methods

Participants

Eighty participants ranging 21–60 years old were recruited from March 2021 to February 2022 (Table 1).

Table 1.

Demographics

	Total	SCH (n = 38)	CON (n = 42)	p value
Age
M (SD)	44.7 (9.6)	44.68 (9.59)	43.19 (10.90)	0.52
Gender
Woman	26.3%	5 (23.8%)	16 (38.1%)
Man	73.8%	33 (86.8%)	26 (61.9%)	0.021
Education
Elementary	77.5%	33 (86.8%)	29 (69%)
Higher	22.5%	5 (13.2%)	13 (31%)	0.07
Clinical stability (in months)		41.79
Positive symptoms		20.52 (7.8)
Negative symptoms		41.1 (8.6)
CPZ ED (average mg/day)		645.8

Clinical stability was defined as the total number of months in which no clinical decompensation was found, thus not requiring an acute care or attention by emergency units. Antipsychotic intake was measured as chlorpromazine equivalent dose (CPZ ED); the p value regarding age was calculated by a t test, t(78) = 0.65; p values regarding gender and education were calculated by the Fisher’s exact test.

SCH, clinical group; CON, control group; SD, standard deviation.

The clinical group (SCH group, n = 38) comprised clinically stable patients diagnosed of schizophrenia according to DSM-5⁴⁴ criteria. They were referred by their main psychiatrist from a specialized day rehabilitation center for patients with chronic psychosis. The diagnosis was carried out by their regular psychiatrist and confirmed through a semi-structured interview based on the Structural Clinical Interview for DSM-5⁴⁵ by another psychiatrist. Individuals with comorbid psychiatric disorders or who had electroconvulsive treatment within the past 3 months were not included.

A control group (CON group, n = 42) was recruited through media advertisements. Individuals were included if they never had received psychiatric care, diagnoses, or undergone psychopharmacological treatment.

Exclusion criteria for both groups included (1) major medical and neurological disorders, (2) substance abuse disorders (except tobacco use disorder), (3) cognitive impairment assessed by the Mini-Mental State Examination (MMSE), and (4) using nonpsychiatric drugs that may affect cognition (e.g., corticosteroids).

The total sample is composed of more men and more participants who had completed up to elementary education (Table 1). The two groups did not significantly differ regarding age or education, but they did regarding gender (more men in the clinical group). The investigation was in accordance with the later Declaration of Helsinki and approved by the corresponding ethical committee (2022-862-1). Participants signed a written informed consent before the study.

Measures

Computerized standard tasks

Participants completed five different CST: the DOT probe task, STROOP test, Go/No-Go task, Trail Making Test (TMT), and Tower Of London (TOL). Supplementary Table S1 reports the CST parameters assessed in the study.

Virtual cooking task

The VCT was developed using the Unity 5.5.1f1 and Visual Studio software. It was displayed through the Vive Pro Eyeglasses. Two manual controllers served to interact with the contents. The VCT recreates a kitchen that, after a tutorial, involves four increasingly difficult tasks. The tasks consist of cooking different ingredients while avoiding them overcooking or cooling down. See Figure 1 for a detailed description of the task. Table 2 displays the parameters analyzed by the task. The remaining information about the VCT can be found elsewhere.^39,40

FIG. 1.

Four tasks (“levels”) conforming the virtual cooking task. All ingredients are displayed at the right side of a virtual table to be cooked in the pan(s). Once cooked, they must be placed at the dish on the left side of the table. Before each subtask, the system explains the specific activity, the total cooking time available, the cooking times for each ingredient, and alerts about burning or cooling them down. Visual feedback regarding the cooking times is continuously displayed over the pans during the tasks using a graphical progression bar and a counter in seconds. The counter turns green when the cooking time finishes, thus indicating that ingredient can be moved to the dish.

Table 2.

Virtual Cooking Task Parameters

Task (time threshold)	Ingredients (cooking time thresholds in seconds)	Parameters	Executive function
1. (2 minutes). Cooking three ingredients in one pan.	Meat (40 seconds)Carrots (30 seconds)Courgettes (30 seconds.)	Task cooking time (TCT)Burning ratio (BR)Cooling ratio (CR)	Selective attentionAttention shifting
2. (3 minutes) Cooking five ingredients in two pans.	Meat (50 seconds)Carrots (20 seconds)Courgettes (10 seconds)Potatoes (30 seconds)Eggs (60 seconds)	TCTBRCRCooking order efficiency (COE)*	Selective attentionAttention shiftingCognitive flexibility
3. (4 minutes)Subtask a: Cooking five ingredients in two pans.Subtask b: Using dressings.	Meat (50 seconds)Carrots (20 seconds)Courgettes (10 seconds)Potatoes (30 seconds)Eggs (60 seconds)“Go” dressings: salt and pepper.“No-go” dressings: sugar, vanilla, and cinnamon	TCTBRCRCOEDressing latencies (DL)Dressing accuracy (DA)	Selective attentionAttention shiftingCognitive flexibilityInhibition of control
4. (5 minutes)Subtask a: Cooking five ingredients in two pans.Subtask b: Set the table.	Meat (50 seconds)Carrots (20 seconds)Courgettes (10 seconds)Potatoes (30 seconds)Eggs (60 seconds)	TCTBRCRCOETable setting efficiency (TSE)*	Selective attentionAttention shiftingCognitive flexibilityPlanning

Asterisks (*) denote novel parameters included within the virtual cooking task (VCT). TCT 1–4 = time needed to finish the tasks; BR = proportion of overcooked (burned) ingredients; CR = proportion of ingredients let to cool down; DA = proportion of correct dressings (including “go” and “no-go”); DL = latencies of “go” dressings; TSE = addresses whether participants set the table while cooking (saving time), or before/after cooking (leading to longer task timing); COE = one of the pans could be overused in tasks 2–4, leading also to longer task timing. An optimal COE could be achieved by distributing the ingredients proportionally to the two pans (i.e., a combination of ingredients leading to 80 seconds in a pan and 90 seconds in the other).

In general, all tasks addressed selective attention. Burning Ratio (BR) and cooling ratio (CR) aimed to assess shifting attention to the different ingredients’ timings so that participants should avoid burning or cooling them down. The dressing accuracy (DA) and dressing latencies (DL) in task 3 followed the inhibition of control reasoning: the right “Go” dressings should be identified and used while avoiding the wrong “No-go” dressings. Finally, the cooking order efficiency (COE, tasks 2 to 4) and table setting efficiency (TSE,task 4) aimed to assess cognitive flexibility and planning since the performance could be more efficient depending on how the actions were performed.

Functional Assessment Short Test

The Functional Assessment Short Test (FAST) is a clinician-administered scale designed to assess daily-life functioning.⁴⁶ It evaluates six dimensions of functioning: autonomy, occupational functioning, cognitive functioning, financial issues, interpersonal relationships, and leisure time. The reliability of the instruments was adequate (Cronbach’s α = 0.77).

Positive and Negative Syndrome Scale

The Positive and Negative Syndrome Scale (PANSS) is a 30-item clinician-administered instrument for measuring schizophrenia symptomatology.⁴⁷ It is composed of three subscales (positive symptoms, NS, and general psychopathology). The reliability of the instrument was high (Cronbach’s α = 0.95).

Experimental procedure

The assessment was conducted individually in a quiet room. Each participant was scheduled to complete the experiment. After signing informed consent, they completed the demographic interview. The clinical group participated in a clinical interview carried out by a psychiatrist and responded to the clinical and functional scales, and the control group was asked about their psychiatric history.

Later, participants completed the CST and the VCT in a random order. They completed the CST while seated in a comfortable chair approximately 60 cm in front of a laptop. Each of the 5 CST was presented randomly by participant (ca. 25 minutes in total). The VCT assessment began once the VR-glasses were calibrated (ca. 5–10 minutes). Before performing the VCT, participants were given the eyeglasses and manual controllers, ensuring they put them on properly and without causing discomfort. Afterward, they were allowed to practice with a short tutorial that explained the main actions for the time necessary to adapt to the virtual environment. Information was provided before each task, specifying the to-be-performed activity, and indicating the timings associated with the task (Figure 2). Participants’ performance was monitored through a monitor screen displaying the virtual environment. A manipulation check indicated that the performance was congruent with the VCT task timings (see Supplementary Table S2). The entire VCT assessment took approximately 30 per participant.

FIG. 2.

Illustration of task 4 instructions. “In this activity you will use 2 pans to cook 5 ingredients (each one with their respective cooking time). In addition, you will have to set the table for 1 person. Be careful of not burning out the ingredients of letting them to cool down. You have 5 minutes to complete the task. Press the button to continue.”

The total procedure took approximately 90 minutes per participant. The experimenter was present in the room throughout the entire assessment in order to ensure a smooth execution.

Statistical analyses

The analyses were performed with SPSS v25 (Chicago, IL, USA). Correlation analyses inspected associations between the CST and the VCT parameters. Multivariate analyses of covariance (MANCOVAs) served to compare the groups’ performance within the CST and VCT. A generalized linear model (GLM) approach served to explore whether and how the CST and VCT assessments predicted the FAST and PANSS scores within the clinical group. This modeling considered a Gaussian data distribution with a log link and a robust estimator for standard errors.⁴⁸

A power analysis approach via Gpower (v.3.1.9.6) was performed based on 18 repeated measures multivariate analysis of variance (MANOVA), including the CST/VCT parameters and age, and the between factor “group” (control vs. clinical). It indicated that a sample of n = 70 was adequate to detect a medium-to-large effect of f2 = 0.20 with 80% power at the α = 0.05 level (a small-to-medium correlation [0.30] was assumed among the repeated measures). The sample in the study was completed with 10 additional participants to prevent potential dropouts.

Results

Correlational analysis

Table 3 shows Spearman rank correlations indicating that, in general, the CST parameters correlated positively with each other but negatively with the TOL (i.e., correct “planning” responses were associated with faster attentional responses). TMT-A only correlated positively with TMT-B. VCT timings also correlated positively with each other and the burning ratio but negatively with the cooking order efficiency and table setting efficiency, thus pointing to their internal efficiency validity. Noteworthy, the VCT timings and burning ratio correlated positively with the “attentional” CST timings (DOT, STROOP, GO-NOGO, or TMT-B) and negatively with the “planning” TOL. Finally, the table setting efficiency correlated positively with the TOL, supporting its planning role.

Table 3.

Parameters’ Descriptives and Correlations Between the CST and the VCT

	CST (1–6)						VCT (7–17)
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17
1. DOT	—
2. STROOP	0.68^**	—
3. GO	0.61^**	0.63^**	—
4. TMT-A	0.15	0.18	0.09	—
5. TMT-B	0.58^**	0.82^**	0.52^**	0.38^**	—
6. TOL	−0.48^**	−0.63^**	−0.32^**	0.12	−0.58^**	—
7. TT	0.37^**	0.63^**	0.41^**	0.06	0.45^**	−0.40^**	—
8. TCT1	0.32^**	0.59^**	0.36^**	0.11	0.44^**	−0.30^*	0.80^**	—
9. TCT2	0.50^**	0.58^**	0.37^**	0.07	0.48^**	−0.40^**	0.84^**	0.69^**	—
10. TCT3	0.30^**	0.54^**	0.43^**	0.02	0.35^**	−0.32^**	0.82^**	0.55^**	0.65^**	—
11. TCT4	0.29^*	0.50^**	0.27^*	0.14	0.41^**	−0.42^**	0.80^**	0.57^**	0.65^**	0.56^**	—
12. BR	0.38^**	0.59^**	0.40^**	−0.01	0.38^**	−0.27^*	0.72^**	0.71^**	0.64^**	0.60^**	0.51^**	—
13. CR	−0.08	−0.01	−0.18	0.03	−0.05	−0.04	0.17	0.16	0.10	0.15	0.15	0.28^*	—
14. DL	0.18	0.31^**	0.17	0.20	0.26^*	−0.12	0.40^**	0.36^**	0.31^**	0.34^**	0.40^**	0.34^**	0.09	—
15. DA	−0.05	−0.10	0.001	0.03	−0.10	−0.05	−0.11	−0.06	−0.11	−0.14	0.07	−0.24^*	−0.15	0.17	—
16. COE	−0.29^*	−0.23^*	−0.16	0.01	−0.17	.13	−0.40^**	−0.28^*	−0.41^**	−0.34^**	−0.36^**	−0.35^**	−0.29^*	−0.14	0.06	—
17. TSE	−0.20	−0.47^**	−0.29^*	−0.01	−0.37^**	0.35^**	−0.56^**	−0.35^**	−0.46^**	−0.45^**	−0.51^**	−0.28^*	0.01	−0.38^**	−0.06	0.18	—
Min	0.37	0.97	0.33	0.00	37.9	5.0	441	113	107	105	106	0.0	0.0	0.00	40	0.0	0.0
Max	0.78	4.99	0.72	99.8	180	10	1,491	287	562	434	475	90	80.2	53.7	100	100	100
M total	0.53	1.9	0.44	42.8	89.5	9.01	701	151	169	183	181	13.8	8.4	10.5	82.9	60.3	51.9
SD	0.09	0.81	0.07	26.8	45.2	1.4	187	35.4	67.9	59.6	62	17.3	15.9	9.3	19.4	32.8	50.3
\|s\|	2.19	4.9	6.12	1.22	3.17	5.79	6.72	6.51	12.08	7.89	8.25	7.47	10.7	4.12	3.18	1.48	0.60
\|k\|	0.08	3.56	8.68	0.30	0.59	3.20	8.81	6.30	28.26	12.07	16.30	10.83	18.5	1.73	0.55	1.47	3.62

The shaded area represents correlations between CST and VCT. Multivariate Shapiro–Wilkinson tests showed a departure from normality in the CST (W = 0.819, p < 0.001) and the VCT (W = 0.572, p < 0.001) assessments. Accordingly, Spearman rank correlations were performed. In the study, DOT_LT (latencies), STROOP_LT, GO_LT, TMT_TTA (total time), TMT_TTB, and TOL_CA (correct answers) were considered based on their closer association with the VCT parameters (see Supplementary Table S2). |s| and |k| represent standardized skewness and kurtosis. BR, CR, and COE represent averaged scorings from the four tasks.

p ≤ 0.05.

p ≤ 0.01.

CST, Computerized Standard Task; VCT, Virtual Cooking Task; TMT, Trail Making Test; TOL, Tower Of London; TT, Total Time; TCT1, Total Cooking Time of task 1; TCT2, Total Cooking Time of task 2; TCT3, Total Cooking Time of task 3; TCT4, Total Cooking Time of task 4; BR, Burning Ratio; CR, Cooling Ratio; DL, Dressing Latencies; DA, Dressing Accuracy; COE, Cooking Order Efficiency; TSE, Table Setting Efficiency.

Group comparisons

MANCOVAs (with age as covariate) and follow-up analyses of covariance (ANCOVAs) indicated that the control group performed better, on average, in the CST and the VCT (Table 4). Effect sizes ranged from 0.06 to 0.55. The CST outcomes revealed group differences in all variables except for the TMT-A. In addition, the VCT showed significant group differences in all variables except for the cooling ratio, dressing accuracy, and cooking order efficiency.

Table 4.

Group Comparisons Regarding the CST and VCT Assessments

	DOT	STRP	GO	TMTA	TMTB	TOL	TT	TCT1	TCT2	TCT3	TCT4	BR	CR	DL	DA	COE	TSE
M_CON	0.48	1.45	0.41	38.3	60.7	9.7	617.6	136.6	143.8	167.4	158.7	6.8	8.3	8.5	82.1	64.7	73.3
M_SCH	0.59	2.60	0.48	48.4	126.3	8.2	800.8	168.6	198.5	201.1	208.3	22.1	8.5	12.9	83.7	55.0	26.3
F_Levene	1.5	25.3	6.4	73.1	10.5	34.6	7.4	18.7	15.2	1.2	3.5	16.2	0.55	4.5	1.1	4.9	0.05
p_Levene	0.22	0.001	0.013	0.001	0.002	0.001	0.008	0.001	0.001	0.28	0.07	0.001	0.46	0.04	0.30	0.03	0.83
F	33.2	87.5	24.0	2.69	83.5	25.6	29.4	21.8	16.6	7.0	16.1	21.9	0.004	4.6	0.119	1.9	21.5
p	0.001	0.001	0.001	0.11	0.001	0.001	0.001	0.001	0.001	0.010	0.001	0.001	0.95	0.035	0.73	0.17	0.001
η_p ²	0 0.32	0.55	0.25	—	0.54	0.26	0.28	0.23	0.18	0.09	0.18	0.23	—	0.06	—	—	0.22
F_Quades	34.3	113.8	26.2	4.7	107.0	26.4	36.4	29.2	27.3	11.6	24.3	22.6	0.21	4.8	0.001	1.72	21.6
p_Quades	0.001	0.001	0.001	0.11	0.001	0.001	0.001	0.001	0.001	0.001	0.001	0.001	0.65	0.030	0.98	0.19	0.001
η_p ²	0.32	0.60	0.26	—	0.60	0.26	0.33	0.28	0.27	0.13	0.24	0.23	—	0.06	—	—	0.22

Five and three participants from the clinical group did not complete the VCT and CST, respectively. Accordingly, MANCOVAs were carried out per each task to prevent data over-exclusion. Box’s M (M_CST = 140.78; M_VCT = 236.3), p < 0.001, indicated unequal covariance matrices in both tasks, which is congruent with the reported Levene’s testing of equal error variances. Accordingly, the multivariate Pillai’s Trace (V_CST = 0.64, p < 0.001; V_VCT = 0.44, p < 0.001) parameter was interpreted.⁵⁰ Age also showed statistical significance in both tasks (V_{age_CST} = 0.21, p < 0.001; V_{age_VCT} = 0.31, p < 0.001). Bonferroni correction was applied for multiple comparison testing.

CST, Computerized Standard Task; VCT, Virtual Cooking Task; CON, control group; SCH, clinical group; TMT, Trail Making Test; TOL, Tower Of London; TT, Total Time; TCT1, Total Cooking Time of task 1; TCT2, Total Cooking Time of task 2; TCT3, Total Cooking Time of task 3; TCT4, Total Cooking Time of task 4; BR, Burning Ratio; CR, Cooling Ratio; DL, Dressing Latencies; DA, Dressing Accuracy; COE, Cooking Order Efficiency; TSE, Table Setting Efficiency.

Predicted daily functioning

The GLM analysis included age, months of clinical stability, and antipsychotic intake as covariates (Table 5). Regarding the CST, slower GO latencies predicted lower total FAST scores, particularly in the cognitive and occupational domains. Moreover, longer TMT-B performance was consistently associated with worsened FAST reports. The TMT-A showed a negative association with autonomy.

Table 5.

Predicted Daily Functioning (FAST) by the CST and VCT

	Autonomy	Cognitive	Interpersonal	Occupational	Financial	Leisure	Total
FAST	ß (OR)	ß (OR)	ß (OR)	ß (OR)	ß (OR)	ß (OR)	ß (OR)
CST
Age	−0.04	0.040	−0.034	0.033	0.094	−0.090	0.026
CPZ ED	−0.10	−0.052	−0.017	0.008	−0.045	−0.061	−0.063
Clin. Stability	0.02	0.057	0.065	−0.021	−0.041	0.098	0.048
DOT	0.04	0.007	−0.020	0.003	0.165^***	−0.006	0.028
STROOP	.001	−0.027	.004	.022	−0.035	.185	−0.011
GO	−0.10	−0.14^* (12.8%)	−0.048	−0.41^** (33.7%)	−0.170	−0.281^***	−0.201^* (18.2%)
TMT-A	−0.18^* (16.8%)	−0.047	−0.078	−0.17	−0.173^***	−0.128	−0.154
TMT-B	0.23^** (25.7%)	0.21^** (23.9%)	0.19^* (20.9%)	0.26^***	0.136	0.27^* (30.5%)	0.27^** (30%)
TOL	0.03	0.04	0.058	0.11	0.076	0.150	0.091
VCT
Age	0.078	0.080	0.001	0.153	0.133	0.086	0.097
CPZ ED	0.080	−0.010	0.012	0.058	0.074^***	0.040	0.041
Clin. Stability	0.083	0.039	0.211^** (23.6%)	0.254	0.015	0.114	0.177
TT	−0.16	0.002	−0.10	−0.221	−0.071	−0.007	−0.145
BR	0.12	0.009	0.079^***	−0.026	0.089	0.063	0.054
CR	−0.01	0.066	−0.023	0.037	0.020	0.018	0.029
DL	−0.09	−0.038	0.090^* (9.4%)	0.116	−0.120	−0.142	0.018
DA	−0.006	−0.010	−0.150^** (13.9%)	−0.219	−0.042	−0.043	−0.116
COE	−0.10	0.068	0.042	0.100	−0.039	0.072	0.010
TSE	0.001	0.044	−0.138^* (12.9%)	−0.087	−0.033	0.023	−0.036

Results rely on participants with complete clinical information (n_CST = 23; n_VCT = 27). In addition, sensitivity residual analyses revealed three extreme cases potentially biasing the results. The reported results excluded these cases. Supplementary Table S3 shows results before the exclusions. Percentages (%) represent the natural exponential function of ß and are interpreted in terms of odds ratio (OR).⁵¹

p ≤ 0.05.

p ≤ 0.01.

***

p ≤ 0.1.

FAST, Functional Assessment Short Test; CST, Computerized Standard Task; CPZ ED, Chlorpromazine Equivalent Dose (average mg/day); Clin. Stability, Clinical Stability (in months); TMT, Trail Making Test; TOL, Tower Of London; VCT, Virtual Cooking Task; TT, Total Time; BR, Burning Ratio; CR, Cooling Ratio; DL, Dressing Latencies; DA, Dressing Accuracy; COE, Cooking Order Efficiency; TSE, Table Setting Efficiency.

The VCT showed consistent predictive capabilities within the interpersonal domain. Specifically, increased dressing latencies predicted worse interpersonal functioning while selecting the correct dressing (Dressing Accuracy) and setting the table while cooking (Table Setting Efficiency) were both associated with better interpersonal functioning.

Predicted NS

TMT-B was the only statistically significant CST parameter predicting NS (Table 6). Longer timing to finish the task was associated with worse NS reports. In contrast, several VCT parameters were sensitive to NS: Total Time and Cooking Order Efficiency predicted lowered NS, whereas increased burning ratio predicted higher NS.

Table 6.

Predicted NS (PANSS) by the CST and VCT

	ß (OR)
CST
Age	0.006
CPZ	−0.077
Clin. Stability	0.223^** (25%)
DOT	0.101
STROOP	−0.071
GO	−0.044
TMTA	−0.137^***
TMTB	0.279^** (32%)
TOL	0.030
VCT
Age	−0.052
CPZ	0.072^***
Clin. Stability	0.283^** (32.6%)
TT	−0.232^** (20.7%)
BR	0.172^** (18.8%)
CR	−0.004
DL	0.034
DA	0.008
COE	−0.194^** (17.7%)
TSE	−0.025

Results rely on participants with complete clinical information (same exclusion criteria as for the FAST analyses). Supplementary Table S4 shows results before such exclusions.

p ≤ 0.05.

p ≤ 0.01.

***

p ≤ 0.1.

NS, negative symptoms; PANSS, Positive and Negative Syndrome Scale; CST, Computerized Standard Task; CPZ ED, Chlorpromazine Equivalent Dose (average mg/day); Clin. Stability, Clinical Stability (in months); TMT, Trail Making Test; TOL, Tower Of London; VCT, Virtual Cooking Task; TT, Total Time; BR, Burning Ratio; CR, Cooling Ratio; DL, Dressing Latencies; DA, Dressing Accuracy; COE, Cooking Order Efficiency; TSE, Table Setting Efficiency.

Discussion

This study introduced a novel VR tool to investigate, from an ecologically valid framework, EF in schizophrenia. The main findings are summarized as follows: (1) the VCT showed moderate-to-strong correlations with the CST, (2) the VCT discriminated between clinical and control groups, (3) the VCT predicted interpersonal functioning, and (4) the VCT predicted NS to a greater extent than the CST. Taken together, these results point to the interest and usefulness of the VCT when assessing individuals with schizophrenia in a virtual everyday environment, as it evaluates different key domains in this condition such as EF, NS, and functioning.

First, most of the VCT parameters in the current study showed moderate-to-high correlations with the CST. In this regard, these findings support the VCT concurrent validity. This is consistent with the previous research testing the VCT in individuals with alcohol use disorder and healthy controls.⁴⁰ Hence, this system can assess EF in a comparable manner to traditional validated tasks, conducting such analysis while participants engage in behaviors typical of a real-life situation. Importantly, this approach increases the ecological validity as it represents the interaction between the individual and the situational context.^17,18

Second, both types of evaluations indicated worse performance in the clinical group than healthy controls. Regarding the VCT performance, the clinical group required longer time to complete all the tasks. Interestingly, these timing differences were qualified by three parameters: the burning ratio, the dressing latencies, and table setting efficiency. In other words, individuals with schizophrenia (1) burned more ingredients, thus showing less capacity to shift and sustain attention within the task; (2) were slower to select the correct dressings, indicating worse inhibitory control; and (3) were less efficient setting the table, which implies a worsened planning and multitasking ability. In summary, these findings support discriminant validity of the VCT indicating the tool’s ability to detect EF deficits in individuals with schizophrenia. The discriminative value of these parameters adds a quality component to a task performed in a context closer to “daily-life” actions than standard tasks.^4,17,18 Moreover, the VCT also adds to the previously tested VR systems by incorporating its ability to explore and discern a wide range of EF subdomains, including sustain attention, planning, multitasking, or inhibitory control.²²

Third, the study also explored the ability of CST and VCT to predict daily functioning and NS of schizophrenia, controlling for age, stability time, and antipsychotic intake. Regarding daily functioning, regression analyses on the FAST scores revealed a differential “global vs. focal” result-pattern linked to the CST and the VCT. On one hand, the TMT-B appeared as the most relevant predictor across almost all functional activity subdomains. On the other hand, the VCT rather showed a predictive effect highly focused on interpersonal functioning. This effect was driven by table setting efficiency and by the dressing accuracy and dressing latencies. It should be noted that EF mediates between the information-processing system and the execution of complex behaviors¹³; thus, they are critical in modulating human interactions,^52,53 a central and disabling feature of schizophrenia.⁵⁴ In this regard, the developers of VR systems for assessing functionality in schizophrenia could take this finding into account and create immersive social environments. As for the NS of schizophrenia, longer TMT-B also showed a greater predictive capability of NS. Interestingly, the VCT points to greater sensitivity than the CST to predict NS, as total time, burning ratio, and cooking order efficiency were associated to NS variations. In this sense, the results could support Frith’s neurocognitive theory, which suggests that NS reflect a deterioration of EF because of impaired awareness of goals (including the intention for communication and interpersonal interaction).^10,11,14 Importantly, previous studies using VR technology to analyze EF in schizophrenia did not assess the ability of these systems to predict NS.²²

This work has some strengths to highlight. First, the sample is mostly composed of individuals with chronic schizophrenia. Given that schizophrenia is a deteriorating disorder,⁵⁵ in chronic schizophrenia the cognitive decline would already be established,⁵⁶ as well as the presence of NS and functional disability. Second, the subjects were clinically stable, avoiding acute psychopathology that could interfere with neurocognition.⁵⁷ Third, the results take into account age, duration of stability, and antipsychotic consumption, which was a limitation in previous literature on VR.²² Importantly, antipsychotic medications, especially when used in polypharmacy or at high doses, can lead to the worsening of NS and further cognitive and global dysfunction.^36,37 Finally, in contrast to previous research on this topic,^23–32 the VCT is based on an fully immersive system. This would enhance its ecological validity by narrowing the gap between interactions in the real and the virtual world.⁵⁸

Nonetheless, some limitations should also be pointed out. First, although sociodemographic features were similar between groups, the total sample was composed of few individuals with higher education. In this regard, it is worth taking into account that individuals with schizophrenia are less likely to access to higher education.⁵⁹ Accordingly, further VCT research should investigate individuals with different educational status. Second, prior exposure to video games and VR tools and thus its potential impact on the VCT performance was not quantified. Aiming to mitigate this potential confounding factor, before starting the VCT, users were allowed to practice navigating the virtual environment. Finally, proper performance in VCT requires complex cognitive functions. Aiming to exclusively analyze EF, the MMSE was used when evaluating patients, trying to avoid those subjects with alterations in higher functions. Including the ability to analyze skill in task execution, as well as incorporating other cognitive domains such as memory, would be useful for the development of future VR systems designed for clinical settings.

Conclusions

The VCT shows concurrent and discriminatory validity when investigating individuals with schizophrenia. In addition, the VCT predicts NS and interpersonal functioning. Overall, this study provides additional data suggesting the potential for analyzing EF and the symptomatology of schizophrenia through systems in contexts more closely resembling real-world scenarios. Specifically, this study contributes to the growing literature exploring innovative assessments of EF in schizophrenia and represents a further advancement in conferring psychometric value to the VCT. Further research should assess the ability of this tool to capture changes over time, as well as evaluating its usefulness in clinical research as an outcome measure (e.g., when studying novel therapeutic agents or psychotherapeutic interventions for cognition) or in clinical practice, guiding personalized clinical decisions.

Footnotes

Acknowledgments

The authors would like to offer their thanks for the support provided by the Division of Psychiatry and Clinical Psychology of La Fe University and Polytechnic Hospital, especially to the day hospital and the biopsychosocial rehabilitation center. We are also grateful for the contribution of the Human-tech Institute of the Polytechnic University of Valencia. Finally, we extend our gratitude to all the participants of the study.

Authors’ Contributions

A.A.: Conceptualization, investigation, resources, data curation, and writing—original draft. S.C.T.: Conceptualization, formal analyses, investigation, and writing—original draft. Y.C.: Conceptualization, data curation, and funding acquisition. I.A.C.G.: Project administration. A.G-B.: Supervision and funding acquisition. P.S.: Supervision. M.L.-C.: Investigation and resources. B.R.C.: Investigation and resources. P.N.: Conceptualization, methodology, validity, supervision, writing—reviewing and editing, and project administration. A.R.M.: Supervision and project administration.

Author Disclosure Statement

The authors declare that there is no conflict of interest.

Funding Information

This work was supported by Instituto Carlos III grants (Spanish Ministry of Economy and Innovation) and co-financed by the European Union: AGB (CP21/00085, PI21/00549), PN (JR22/00030, AP2020-34), and by La Fe Health Research Institute: YC (2020-616-1). The funding sources have not been implicated in the design or publication of the study.

Supplementary Material

References

Tamminga

. Domains of dysfunction in schizophrenia: Implications for diagnosis. World Psychiatry, 2008; 7(1):34–35; doi: 10.1002/j.2051-5545.2008.tb00147.x

GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: A systematic analysis for the Global Burden of disease study 2019. Lancet Psychiatry, 2022; 9(2):137–150; doi: 10.1016/S2215-0366(21)00395-3

Keefe

RSE

, Harvey

. Novel antischizophrenia treatments—Cognitive impairment in schizophrenia. Handb Exp Pharmacol, 2012; 213:11–37; doi: 10.1007/978-3-642-25758-2

Suchy

, Ziemnik

, Niermeyer

. Assessment of Executive Functions in Clinical Settings. Elsevier Inc.; Amsterdam. 2017; doi: 10.1016/B978-0-12-803676-1.00022-2

Parsey

, Schmitter-Edgecombe

. Applications of technology in neuropsychological assessment. Clin Neuropsychol, 2013; 27(8):1328–1361; doi: 10.1080/13854046.2013.834971

Reichenberg

, Harvey

. Neuropsychological impairments in schizophrenia: Integration of performance-based and brain imaging findings. Psychol Bull, 2007; 133(5):833–858; doi: 10.1037/0033-2909.133.5.833

Weickert

, Goldberg

, Gold

, et al. Cognitive impairments in patients with schizophrenia displaying preserved and compromised intellect. Arch Gen Psychiatry, 2000; 57(9):907–913; doi: 10.1001/archpsyc.57.9.907

Orellana

, Slachevsky

. Executive functioning in schizophrenia. Front Psychiatry, 2013; 4:15–35; doi: 10.3389/fpsyt.2013.00035

Diamond

. Executive functions. Annu Rev Psychol, 2013; 64:135–168; doi: 10.1146/annurev-psych-113011-143750

10.

Frith

. The Cognitive Neuropsychology of Schizophrenia. 1992. Available from: https://psycnet.apa.org/record/1993-97187-000 [Last accessed: April 4, 2024].

11.

Donohoe

, Robertson

. Can specific deficits in executive functioning explain the negative symptoms of schizophrenia? A review. Neurocase, 2003; 9(2):97–108; doi: 10.1076/NEUR.9.2.97.15075

12.

Green

, Harvey

. Cognition in schizophrenia: Past, present, and future. Schizophr Res Cogn, 2014; 1(1):e1–e9; doi: 10.1016/J.SCOG.2014.02.001

13.

Jurado

, Rosselli

. The elusive nature of executive functions: A review of our current understanding. Neuropsychol Rev, 2007; 17(3):213–233; doi: 10.1007/S11065-007-9040-Z

14.

Dibben

CRM

, Rice

, Laws

, et al. Is executive impairment associated with schizophrenic syndromes? A meta-analysis. Psychol Med, 2009; 39(3):381–392; doi: 10.1017/S0033291708003887

15.

Bowie

, Harvey

. Cognitive deficits and functional outcome in schizophrenia profile of cognitive impairments in schizophrenia. Neuropsychiatr Dis Treat, 2006; 2(4):531–536; doi: 10.2147/nedt.2006.2.4.531

16.

Stroop

. Studies of interference in serial verbal reactions. J Exp Psychol, 1935; 18(6):643–662; doi: 10.1037/h0054651

17.

Chaytor

, Schmitter-Edgecombe

, Burr

. Improving the ecological validity of executive functioning assessment. Arch Clin Neuropsychol, 2006; 21(3):217–227; doi: 10.1016/j.acn.2005.12.002

18.

Burgess

, Alderman

, Forbes

, et al. The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology. J Int Neuropsychol Soc, 2006; 12(2):194–209; doi: 10.1017/S1355617706060310

19.

Chaytor

, Schmitter-Edgecombe

. The ecological validity of neuropsychological tests. Neuropsychol Rev, 2003; 13(4):181–197; doi: 10.1023/b:nerv.0000009483.91468.fb

20.

, Cho

, Kim

, et al. The development of a VR system for the cognitive and behavioral assessment of schizophrenia. Stud Health Technol Inform, 2004; 98:180–182; doi: 10.3233/978-1-60750-942-4-180

21.

, Cho

, Kim

J-J

, et al. A virtual environment for investigating schizophrenic patients’ characteristics: Assessment of cognitive and navigation ability. Cyberpsychol Behav, 2003; 6(4):397–404; doi: 10.1089/109493103322278781

22.

Tyburski

, Mak

, Sokołowski

, et al. Executive dysfunctions in schizophrenia: A critical review of traditional, ecological, and virtual reality assessments. J Clin Med, 2021; 10(13):1–26; doi: 10.3390/jcm10132782

23.

Larøi

, Canlaire

, Mourad

, et al. Relations between a computerized shopping task and cognitive tests in a group of persons diagnosed with schizophrenia compared with healthy controls. J Int Neuropsychol Soc, 2010; 16(1):180–189; doi: 10.1017/S1355617709991159

24.

Josman

, Schenirderman

, Klinger

, et al. Using virtual reality to evaluate executive functioning among persons with schizophrenia: A validity study. Schizophr Res, 2009; 115(2–3):270–277; doi: 10.1016/j.schres.2009.09.015

25.

Plechatá

, Hejtmánek

, Fajnerová

., Virtual supermarket shopping task for cognitive rehabilitation and assessment of psychiatric patients: Validation in chronic schizophrenia. Cesk Psychol, 2021; 65(1):14–30; doi: 10.51561/cspsych.65.1.14

26.

Baker

, Kurtz

, Astur

. Virtual reality assessment of medication compliance in patients with schizophrenia. Cyberpsychol Behav, 2006; 9(2):224–229; doi: 10.1089/cpb.2006.9.224

27.

Kurtz

, Baker

, Pearlson

, et al. A virtual reality apartment as a measure of medication management skills in patients with schizophrenia: A pilot study. Schizophr Bull, 2007; 33(5):1162–1170; doi: 10.1093/schbul/sbl039

28.

Siddiqui

, Saperia

, Fervaha

, et al. Goal-directed planning and action impairments in Schizophrenia evaluated in a virtual environment. Schizophr Res, 2019; 206:400–406; doi: 10.1016/j.schres.2018.10.012

29.

Laloyaux

, Van der Linden

, Levaux

, et al. Multitasking capacities in persons diagnosed with schizophrenia: A preliminary examination of their neurocognitive underpinnings and ability to predict real world functioning. Psychiatry Res, 2014; 217(3):163–170; doi: 10.1016/j.psychres.2014.03.026

30.

Laloyaux

, Van der Linden

, Nuechterlein

, et al. A direct examination of the cognitive underpinnings of multitasking abilities: A first study examining schizophrenia. Psychiatry Res, 2018; 268:288–296; doi: 10.1016/j.psychres.2018.06.060

31.

Han

, Young Kim

, Kim

. Assessment of cognitive flexibility in real life using virtual reality: A comparison of healthy individuals and schizophrenia patients. Comput Biol Med, 2012; 42(8):841–847; doi: 10.1016/j.compbiomed.2012.06.007

32.

Holt

, Rodewald

, Rentrop

, et al. The plan-a-day approach to measuring planning ability in patients with schizophrenia. J Int Neuropsychol Soc, 2011; 17(2):327–335; doi: 10.1017/S1355617710001712

33.

Aubin

, Béliveau

, Klinger

. An exploration of the ecological validity of the Virtual Action Planning-Supermarket (VAP-S) with people with schizophrenia. Neuropsychol Rehabil, 2018; 28(5):689–708; doi: 10.1080/09602011.2015.1074083

34.

Siemerkus

, Irle

, Schmidt-Samoa

, et al. Egocentric spatial learning in schizophrenia investigated with functional magnetic resonance imaging. Neuroimage Clin, 2012; 1(1):153–163; doi: 10.1016/J.NICL.2012.10.004

35.

Man

DWK

, Ganesan

, Yip

CCK

, et al. Validation of the virtual-reality prospective memory test (Hong Kong Chinese version) for individuals with first-episode schizophrenia. Neuropsychol Rehabil, 2018; 28(7):1197–1210; doi: 10.1080/09602011.2016.1251949

36.

Gallego

, Nielsen

, De Hert

, et al. Safety and tolerability of antipsychotic polypharmacy. Expert Opin Drug Saf, 2012; 11(4):527–542; doi: 10.1517/14740338.2012.683523

37.

Mosolov

, Yaltonskaya

. Primary and secondary negative symptoms in schizophrenia. Front Psychiatry, 2021; 12:766692; doi: 10.3389/FPSYT.2021.766692

38.

Parsons

, Carlew

, Magtoto

, et al. The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychol Rehabil, 2017; 27(5):777–807; doi: 10.1080/09602011.2015.1109524

39.

Chicchi Giglioli

, Bermejo Vidal

, Alcañiz Raya

. A virtual versus an augmented reality cooking task based tools: A behavioral and physiological study on the assessment of executive functions. Front Psychol, 2019; 10:2529–2512; doi: 10.3389/fpsyg.2019.02529

40.

Chicchi Giglioli

, Pérez Gálvez

, Gil Granados

, et al. The virtual cooking task: A preliminary comparison between neuropsychological and ecological virtual reality tests to assess executive functions alterations in patients affected by alcohol use disorder. Cyberpsychol Behav Soc Netw, 2021; 24(10):673–682; doi: 10.1089/cyber.2020.0560

41.

Miskowiak

, Jespersen

, Kessing

, et al. Cognition assessment in virtual reality: Validity and feasibility of a novel virtual reality test for real-life cognitive functions in mood disorders and psychosis spectrum disorders. J Psychiatr Res, 2022; 145:182–189; doi: 10.1016/j.jpsychires.2021.12.002

42.

Fett

AKJ

, Reichenberg

, Velthorst

. Lifespan evolution of neurocognitive impairment in schizophrenia—A narrative review. Schizophr Res Cogn, 2022; 28:100237; doi: 10.1016/j.scog.2022.100237

43.

Rehse

, Bartolovic

, Baum

, et al. Influence of antipsychotic and anticholinergic loads on cognitive functions in patients with schizophrenia. Schizophr Res Treatment, 2016; 2016:8213165; doi: 10.1155/2016/8213165

44.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. American Psychiatric Association; Virginia. 2013; doi: 10.1176/appi.books.9780890425596

45.

First

. Structured Clinical Interview for the DSM (SCID). In: The Encyclopedia of Clinical Psychology. John Wiley & Sons; New York. 2015; pp. 1–6; doi: 10.1002/9781118625392.wbecp351

46.

Rosa

, Sánchez-Moreno

, Martínez-Aran

, et al. Validity and reliability of the Functioning Assessment Short Test (FAST) in bipolar disorder. Clin Pract Epidemiol Ment Health, 2007; 3:5; doi: 10.1186/1745-0179-3-5

47.

Lin

, Lin

, et al. Early improvement in PANSS-30, PANSS-8, and PANSS-6 scores predicts ultimate response and remission during acute treatment of schizophrenia. Acta Psychiatr Scand, 2018; 137(2):98–108; doi: 10.1111/acps.12849

48.

Cervera-Torres

, Ruiz-Fernández

, Godbersen

, et al. Influence of resilience and optimism on distress and intention to self-isolate: Contrasting lower and higher COVID-19 illness risk samples from an extended health belief model. Front Psychol, 2021; 12:662395–662310; doi: 10.3389/fpsyg.2021.662395

49.

Meyers

, Gamst

, Guarino

. Applied Multivariate Research. SAGE Publications, London; 2006.

50.

Conover

, Iman

. Analysis of covariance using the rank transformation. Biometrics, 1982; 38(3):715–724.

51.

Fernández

, Kastner

, Cervera-Torres

, et al. Moved by emotions: Affective concepts representing personal life events induce freely performed steps in line with combined sagittal and lateral space-valence associations. Front Psychol, 2019; 10:2787–2713; doi: 10.3389/fpsyg.2019.02787

52.

Gao

, Jia

, Zhao

, et al. Inhibitory control in excessive social networking users: Evidence from an event-related potential-based go-nogo task. Front Psychol, 2019; 10:1810; doi: 10.3389/fpsyg.2019.01810

53.

Perry

, Braren

, Rincón-Cortés

, et al. Enhancing executive functions through social interactions: Causal evidence using a cross-species model. Front Psychol, 2019; 10:2472–2411; doi: 10.3389/fpsyg.2019.02472

54.

Pinkham

, Penn

. Neurocognitive and social cognitive predictors of interpersonal skill in schizophrenia. Psychiatry Res, 2006; 143(2–3):167–178; doi: 10.1016/j.psychres.2005.09.005

55.

Lieberman

. Is schizophrenia a neurodegenerative disorder? A clinical and neurobiological perspective. Biol Psychiatry, 1999; 46(6):729–739; doi: 10.1016/S0006-3223(99)00147-X

56.

Lewandowski

, Cohen

, Öngur

. Evolution of neuropsychological dysfunction during the course of Schizophrenia and bipolar disorder. Psychol Med, 2011; 41(2):225–241; doi: 10.1017/S0033291710001042

57.

Zhang

, Fang

, Fan

, et al. Brain-derived neurotrophic factor as a biomarker for cognitive recovery in acute schizophrenia: 12-week results from a prospective longitudinal study. Psychopharmacology (Berl), 2018; 235(4):1191–1198; doi: 10.1007/s00213-018-4835-6

58.

Fusco

, Tieri

. Challenges and perspectives for clinical applications of immersive and non-immersive virtual reality. J Clin Med, 2022; 11(15); doi: 10.3390/jcm11154540

59.

Crossley

, Alliende

, Czepielewski

, et al. The enduring gap in educational attainment in Schizophrenia according to the last 50 years of published research: A meta-analysis. Lancet Psychiatry, 2022; 9(7):565–573; doi: 10.1016/S2215-0366(22)00121-3

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.56 MB