An immersive virtual reality serious game set for the clinical assessment of spatial attention impairments: Effects of avatars on perspective?

Abstract

Background

Hemineglect (HN) is a post-stroke condition, frequently defined as an impairment in finding spatial targets within an egocentric frame of reference, where the stimulus is coded relative to the self. However, the egocentric reference frame can change with the presence of another person. Immersive virtual reality (IVR) offers several advantages over paper-and-pencil tests typically used to assess HN, such as a realistic and controlled environment, standardised stimulus presentation, and sensitive response acquisition. We developed a new serious game in IVR to assess HN. Here we investigated (1) the feasibility and user experience of this serious game, and (2) the effect of the presence of an avatar on HN.

Methods

We first tested a group of 61 control individuals (CI), followed by a group of post-stroke 11 individuals without HN (SI:HN-), 6 with HN (SI:HN+), and 17 resampled age matched group of control individuals (CI). All participants performed the “Peach test”, which required them to find and respond to a target presented among distractors, either alone or in the presence of an avatar. We measured response time (RT) and omissions. The SI:HN- and SI:HN + groups also completed a paper-and-pencil test for HN and a user experience questionnaire.

Results

The first analysis of results with CI showed no differences in responding to the target when in ipsi- compared to contra-lateral spaces, nor in peri- compared to extra-personal spaces. There were also no differences in responding to the target in the no-avatar condition relative to the two avatars conditions. In the second analysis, SI:HN + were slower than SI:HN- and CI. Although an interaction between group and Laterality was predicted, the results showed that there was no Laterality effect for any of the groups. An interaction between group and proximity, showed that both SI groups were slower in the extra- compared to peri-personal spaces. The user experience was globally positively rated by the SI.

Conclusion

We developed a serious game in IVR for the assessment of HN. Although most our findings were inconclusive, the Peach test showed excellent user experience results.

Trial registration http://www.clinicaltrials.gov ; Unique identifier: NCT04694833, Date of registration: 11/24/2020.

Keywords

serious game immersive virtual reality hemineglect perspective-taking stroke

1 Introduction

Hemineglect (HN), also known as unilateral neglect, spatial neglect, or hemispatial neglect, is a complex disorder of spatial cognition (perception and motor planning), attention (spatial and non-spatial; inhibition) and arousal, caused by parietal-frontal damage of the superior longitudinal fasciculus.^1–4 It is a condition characterised by a failure to attend or respond to a stimulus or stimuli presented on the opposite side of space to a brain lesion (contra-lesional space), and cannot be attributed to sensory or motor deficits.^5,6 The estimated prevalence of HN ranges from 30% following stroke of either or both hemispheres (Esposito, Shekhtman and Chen⁷ and 50% after a right-hemisphere lesion versus 30% after a left-hemisphere lesion.⁸ In addition to lateralised attentional bias, HN can manifest differentially in peri-personal space (i.e., space within reach) and extra-personal space (i.e., space outside of reach).^9,10 For example, Van der Stoep, Visser-Meily¹¹ reported 109 stroke patients showing 47% had HN in both peri- and extra-personal spaces, 25% had HN only in extra-personal space, with the remaining patients showing a mix of profiles depending on the task used for the assessment.

HN is typically assessed using paper-and-pencil tests (or more recently, digitised paper-and-pencil tests).^12,13 Checketts, Mancuso¹⁴ administrated a survey examining the tools used for hemineglect cognitive assessment by 454 clinicians. The results showed that most responders were psychologists and occupational therapists, and most reported using the Behavioural Inattention Test battery¹⁵ (BIT) consisting of 15 sub-tests, including cancellation tests.¹⁴ In a similar survey, Evald, Wilms and Nordfang¹⁶ reported that 49% of their respondents (psychologists, occupational therapists, and speech therapists) used paper-and-pencil assessment, while few respondents used computerised tests. This latter finding is surprising because computerised tests bring several advantages to HN cognitive assessment, mostly addressing weaknesses in the use of conventional paper-and-pencil tests such as ecological validity, sensitivity, and standardisation.^17–19 Computerised tests, such as the Test Attentional Performance (TAP) battery,²⁰ involves precise control of the stimuli presentation, stimuli randomisation (increasing reliability), multiple response measures (omission, error, response time) with increased accuracy (increasing test sensitivity and specificity) and automated score analyses and data comparison relative to control participant performance (norms).^21–23 One potential explanation for the slow adoption of computerised tests by clinicians might be that the tests are often digitised forms of the paper-and-pencil tests, lacking ecological validity, and not bringing significant advancement to justify changing their standard clinical routine.

HN is thought to result from impairment to the spatial coordinate systems and specific frames of reference used to code the location of stimuli in space.^24–26 Typically, HN is defined by an egocentric frame of reference, where the location of a stimulus is spatially coded relative to the self^27,28 (e.g., stimuli are typically neglected when presented on the contra-lesional relative to ipsilesional side of space relative to body position). While eye movements can be used to evaluate HN,²⁹ making eye movements rarely influences HN, suggesting no impairment of the retinotopic frame of reference^30–32 (i.e., moving the eyes does not reduce HN). The egocentric reference frame can change with the presence of another person in the scene.^33–37 When alone, environment spatial stimuli are coded with reference to an egocentric frame of reference (own perspective). However, when another person is present, the viewer may spontaneously take the perspective of the other person, representing spatial stimuli from the other's perspective.^38–40 This effect causes congruency conflicts between peripersonal and extrapersonal space for the self-perspective compared to other-perspective (i.e., stimuli within physical reach from the viewer become coded in extrapersonal space from the other person's perspective, and vice versa). Furthermore, stimuli presented on the egocentric left of the viewer becomes egocentric right when viewing the same stimuli from the other's perspective. These reversal effects present interesting opportunities that could be useful for better understanding HN, as well as using the effects caused by the presence of another person (an avatar) to facilitate diagnosis.

The effect of perspective in the expression of HN has already been reported for representational (memory) HN.^41,42 In Della Sala, Logie,⁴² patients with representational hemineglect viewed a visual scene and then recalled objects from the scene, either from their own perspective, or from an opposite perspective (where they imagined themselves stood at a different location in the room facing the direction of their physical position). The data showed that the patients failed to recall stimuli presented on the contra-lesional side of space relative to their (own or opposite) perspective, demonstrating that items were neglected based on the relative position of the stimuli. For example, when patients with a right lesion causing left HN recalled stimuli using their own perspective, they neglected to recall stimuli to the left of their perspective. However, when they used the opposite imagined perspective, they again neglected the stimuli to the left of the opposite perspective (i.e., to the right of their physical position), and they recalled the stimuli to the right of the opposite perspective (i.e., to the left of their physical position; the stimuli that they neglect from their own perspective). A similar study was reported by Becchio, Del Giudice.⁴³ They asked left HN patients to recall the positions of objects placed in a hidden grid consisting of three rows and two columns, either from their own first-person perspective or from the perspective of an avatar. Their findings demonstrated that objects presented on the left (contra-lesional) side were omitted when viewed from their own perspective, but not when the same stimuli were perceived from the opposite avatar perspective, suggesting a strong effect of perspective on HN. The authors argued that the transformation of HN was caused by a remapping of space with reference to the avatar's frame of reference (e.g., an altercentric frame of reference).

Perspective taking has been systematically investigated by Samson, Apperly⁴⁴ in a series of experiments in which participants were asked to perceive environmental stimuli from their own perspective versus an avatar's perspective. The coloured stimuli where either congruent (the same) for the self (own perspective) and avatar perspective (e.g., two dots placed in front of the avatar), or different (incongruent) for the self and avatar perspectives (e.g., one dot placed in front of the avatar and one dot placed behind of the avatar creating a perception of two dots for the self and one dot for the avatar perspective). The results showed that participants were slower and less accurate when perceiving the stimuli from their own perspective, particularly in the incongruent condition. The authors proposed that the avatar's perspective automatically interfered with self-perspective, even when the avatar's perspective was irrelevant to the task. In another series of experiments, Cavallo, Ansuini⁴⁵ tested right-handed participants that were asked to verbally report the location of a stimulus presented on the right versus left, and near versus far space, from either their own perspective or from an avatar's perspective (present opposite). The results showed that when participants responded from their own perspective, mean response time was faster when the stimulus was on the right than left, and in near than far space. However, the effect reversed when participants responded from the perspective of the avatar, with faster responses now made when the stimulus was on the left than right, and when in far than near space⁴⁵ showing a remapping of space relative to perspective.^39,46,47 Other research shows that perspective-taking can be modulated by numerous other factors (e.g., emotions,^48,49 prosocial behaviour,⁵⁰ culture⁵¹), and social perception (i.e., the capacity to analyse the intentions and dispositions of others⁵²).

Virtual reality is a sophisticated computerised technology and a promising solution to enhance the quality of cognitive assessment.^53,54 It is a combination of hardware and software allowing for the manipulation of objects,^55,56 and the monitoring of users’ performance and behaviour within a virtual environment.⁵⁷ Thus, bringing all the advantages of computerised tests with ecologically relevant stimuli. Here, we use the term “Immersive Virtual Reality” (IVR) to refer to technology that allows users to immerse themselves into a three-dimensional simulation, giving the feeling of being situated within the virtual environment (i.e., sense of presence).^58,59 The application of IVR to clinical assessment has been the subject of several reviews.^60–63 For example, Tsirlin, Dupierrix⁶⁴ analysed the scientific literature and summarised that IVR offers opportunities to display ecologically relevant and attractive stimuli within a controllable, safe, familiar and meaningful context to users, increasing engagement. Engagement can be further gained through IVR-based serious games,^65–67 which can be defined as entertaining or fun exercises that motivate users to engage in performance or learning. They can be theory-driven and based on existing psychometric tests, providing multisensorial feedback, and meaningful gameplay through intriguing and enjoyable storytelling.^67–70 Serious games can be designed to provoke users to perform at their maximal level of ability (i.e., to achieve a high score; improving test/re-test reliability), as well as provide a means to simulate everyday life situations that use the assessed cognitions. Several studies have already demonstrated the added value that serious games bring for cognitive assessment.^71–73

In the present paper, we present a new IVR-based computerised serious game (the “Peach test”), which combines the assessment of egocentric, peri- and extra-personal HN, including perspective manipulation (self-versus other perspective; and the inclusion of social manipulation within the avatar). The Peach test simulates a daily living (ecological) activity involving a simple and playful storyline. The participant has to search for a target peach that is presented within a hidden grid, allowing systematic manipulation of target spatial position, with equal presentation of the target on the ipsilateral versus contralateral side, and peri- versus extra-personal distance across trials (randomised) (requiring similar behaviour to cancellation tasks; the most frequently used HN assessment method).^74,75 The task is performed three times, once with no avatar, and twice with an avatar stood at the opposite side of the table to the participant's viewpoint, one block with a friendly avatar and one block with a non-friendly avatar.

We performed three analyses of data. In the first analysis, we examined the feasibility of the “Peach test” with a large group of healthy control individuals (CI). For the dependent measures of omissions and response time, we hypothesised that (1) there would be no evidence of HN, showing no differences in target responses between ipsilateral compared to contralateral space, and for response time; (2) an interaction between the avatar and proximity showing that for the no avatar condition, slower response times to targets placed in extra- than peri-personal space, whereas in the avatar condition, faster response times to targets placed in extra- than peri-personal space (showing a reversal) ((3) perhaps further moderated by the social characteristics of the avatar). In analysis 2, we tested a group of stroke individuals with hemineglect (SI:HN+), a group of stroke individuals without hemineglect (SI:HN-), and a re-sampled age-matched group of healthy control individuals (CI) (selected from the previous CI group to match the SI groups). In addition to the same hypotheses of analysis 1, we hypothesised that, (1) omissions would be greater and response time would be slower for the SI:HN + and SI:HN- groups relative to the CI group; (2) an interaction between group and Laterality showing that the SI:HN + group would be slower to respond to contralateral compared to ipsilateral targets (i.e., showing HN), whereas the SI:HN- and CI would show no lateral differences; (3) an interaction between group, Laterality and avatar showing that for the SI:HN + group, the no avatar condition would show more omissions and slower response times to contralateral compared to ipsilateral targets (showing HN), but that in the avatar condition, the effect would be reversed, showing less omissions and faster response times to contralateral compared to ipsilateral targets (a perspective shift). We predicted that there would be no lateral effects for the SI:HN- and CI groups, and consequently, no reversal. (4) This interaction may be further moderated by the friendliness of the avatar. Finally, in the third analysis, we evaluated the user experience in the two SI groups.

2 Methodology

2.1 Participants

We tested 61 healthy control individuals (CI) (32 females, 3 left-handed; aged between 30 and 77 years, M = 55, SD = 10.6) and 17 post-stroke individuals (SI) (3 females, 9 left-handed (less-affected); aged between 46 and 79 years, M = 60, SD = 10). The CI were recruited using convenience sampling from the University of Louvain participation panel and social media groups. The inclusion criteria were: (1) corrected-to-normal vision, and (2) a good understanding of the task instructions. They were excluded if they reported a history of neurological conditions. The first analysis considered all 61 participants of the CI group, whereas the second analysis involved the 17 SI and a selection of 17 CI from the 61 participants that were matched by age to the SI participants. The SI were recruited from the physical medicine and rehabilitation department of the Cliniques universitaires Saint-Luc in Brussels. The inclusion criteria were: (1) presence of an ischaemic or haemorrhagic first stroke according to the World Health Organisation, with lesions confirmed by medical imagery, (2) clinical diagnosis of HN and HP from a clinical neuropsychological or physical medicine evaluation report, and (3) a good understanding of the task instructions. They were excluded if they (1) presented other neurological or orthopaedical conditions (such as dementia) that would interfere with the use of IVR, or (2) had uncorrected vision. All the SI participants had a hemiparesis, and 6 SI additionally had a clinical diagnosis of HN documented in an evaluation medical report (SI:HN+) (see Table 1 for precise details of each anonymised post-stroke individual). The SI:HN + and SI:HN- groups were contrasted to the re-sampled age-matched CI group. The three groups (SI:HN+, SI:HN- and CI) consisted of 6, 11 and 17 participants (SI:HN+: 0 females, 3 left-handed post-stroke; aged between 49 and 67 years, M = 59, SD = 7.01; SI:HN-: 3 females, 6 left-handed post-stroke, aged between 47 and 79 years, M = 60.5, SD = 12.01). A t-test showed no differences between groups for age (t (32) = -0.069, p = 0.95). All procedures were approved by the Saint–Luc UCLouvain-Hospital–Faculty Ethics Committee, and registered on clinicaltrial.org (NCT04694833). All participants volunteered to participate, and provided written informed consent prior to the experiment. The third analysis of user experience was tested only with the SI groups.

Table 1.
The demographic characteristics of the post-stroke individuals (SI groups).

Diagnosed Asymmetry

Hemineglect Apples

Months (side L/R) Test

Id Gender Age range Stroke site Stroke type post-onset (note 1) (note 2)

Individuals with HP and No HN (SI:HN-) SI01 F 50–55 Left middle cerebral artery ischaemic 3,5 No (L) 0

SI02 M 55–60 Right thalamus ischaemic 1,5 No (R) 0

SI05 M 65–70 Left paramedian pontine ischaemic 38 No (L) 1

SI06 M 45–50 Left middle cerebral artery ischaemic 22,8 No (L) 0

SI07 M 60–65 Right lacunar internal capsule ischaemic 18 No (R) 0

SI09 F 75–80 Left posterior capsulo-thalamic ischaemic 1,6 No (L) 0

SI12 M 70–75 Right posterior branch of the right sylvian artery ischaemic 10 No (R) 2

SI13 M 70–75 Right sylvian fissure ischaemic 2 No (R) −1

SI15 M 45–50 Left temporal lobe ischaemic 0,2 No (L) −1

SI16 M 45–50 Left lenticulostriate artery haemorrhagic 7 No (L) 0

SI17 F 60–65 Right sylvian fissure ischaemic 20 No (R) 0

Individuals with HP and HN (SI:HN+) SI03 M 45–50 Left middle cerebral artery ischaemic 15 Yes (L) 1

SI04 M 60–65 Left parietal intraparenchymal haemorrhagic 13 Yes (L) 0

SI08 M 50–55 Right superficial and deep sylvian and secondary occipital ischaemic 17,5 Yes (R) 0

SI10 M 60–65 Left capsulo-thalamic haemorrhagic 4,6 Yes (L) 1

SI11 M 60–65 Right superficial middle cerebral vein ischaemic 5,8 Yes (R) 0

SI14 M 65–70 Right sylvian fissure ischaemic 19,4 Yes (R) 3*

Mean All (SD) 60 (10.30) 11,75 (10.12)

Mean SI: HP-HN (SD) 60.54 (12.01) 11.32 (12.02)

Mean SI: HP + HN (SD) 59 (7.01) 12.55 (6.10)

							Diagnosed	Asymmetry
Individuals with HP and No HN (SI:HN-)	SI01	F	50–55	Left middle cerebral artery	ischaemic	3,5	No (L)	0
	SI02	M	55–60	Right thalamus	ischaemic	1,5	No (R)	0
	SI05	M	65–70	Left paramedian pontine	ischaemic	38	No (L)	1
	SI06	M	45–50	Left middle cerebral artery	ischaemic	22,8	No (L)	0
	SI07	M	60–65	Right lacunar internal capsule	ischaemic	18	No (R)	0
	SI09	F	75–80	Left posterior capsulo-thalamic	ischaemic	1,6	No (L)	0
	SI12	M	70–75	Right posterior branch of the right sylvian artery	ischaemic	10	No (R)	2
	SI13	M	70–75	Right sylvian fissure	ischaemic	2	No (R)	−1
	SI15	M	45–50	Left temporal lobe	ischaemic	0,2	No (L)	−1
	SI16	M	45–50	Left lenticulostriate artery	haemorrhagic	7	No (L)	0
	SI17	F	60–65	Right sylvian fissure	ischaemic	20	No (R)	0
Individuals with HP and HN (SI:HN+)	SI03	M	45–50	Left middle cerebral artery	ischaemic	15	Yes (L)	1
	SI04	M	60–65	Left parietal intraparenchymal	haemorrhagic	13	Yes (L)	0
	SI08	M	50–55	Right superficial and deep sylvian and secondary occipital	ischaemic	17,5	Yes (R)	0
	SI10	M	60–65	Left capsulo-thalamic	haemorrhagic	4,6	Yes (L)	1
	SI11	M	60–65	Right superficial middle cerebral vein	ischaemic	5,8	Yes (R)	0
	SI14	M	65–70	Right sylvian fissure	ischaemic	19,4	Yes (R)	3*

	Mean All (SD)		60 (10.30)			11,75 (10.12)
	Mean SI: HP-HN (SD)		60.54 (12.01)			11.32 (12.02)
	Mean SI: HP + HN (SD)		59 (7.01)			12.55 (6.10)

SI = Post-stroke individuals; F = Female; M = Male; SD = Standard deviation; (note 1) Hemineglect was determined by clinical evaluation during the acute phase post-stroke; (note 2) At the time of the present study, patients were evaluated using the Apple Cancellation Test to assess hemineglect status on the day of the experiment.

2.2 Materials, stimuli and experimental design

We used a VR headset (Oculus Quest 2) and one Oculus Quest motion controller. The visual display presented to participants was monitored by the experimenter through live stream of the Oculus App on a digital tablet (Huawei MediaPad T, model AGS2-W09). The virtual environment was built with Unity 2019.3 software (in C # language), using virtual objects purchased in Unity Asset Store. We purchased two 3D avatars, consisting of middle-aged male and female humanoids (Figure 1(d)). The avatar animations (e.g., head movements, eye blinks, speech) were performed using SALSA Lip Sync². The IVR simulated a 3D kitchen environment consisting of worktops, wall cupboards, a cooker, fridge-freezer, and a table (Figure 1(a)), on which a hidden grid (6 columns and 4 rows; 24 cells) was placed. The table dimensions were 120 cm (L), 120 cm (W), and 73 cm (H), and the cells of the hidden grid measured 20 cm (L) and 20 cm (W). The target stimulus was a Peach (presented in 3D), and the distractor stimuli were different fruits and vegetables (all presented in 3D). The target Peach was developed in-house using Blender. The distractor fruit objects were taken from two Unity Asset Store packages that we purchased: Food Pack Mixed and Modern Supermarket.^76,77 All stimuli were placed within the centre of each cell, with the target stimulus appearing in each of the 24 cells (randomly) across the trial set. The distractors were presented in the other 23 cells, with random allocation on every trial (Figure 1(b)).

Figure 1.

(a) The 3D simulation of the kitchen where the participants were immersed. The kitchen contained a table on which the stimuli appeared. (b) Before each trial, the participant had to fixate the red basket to regulate the eye starting position for each trial. When fixated, the basket turned from red to green. (c) The target Peach and the controller button for responding (shown as an animated gif). (d) the selection of either the female or male avatar. (e) An example of the avatar presented within the kitchen scene (see coloured version online).

The CI held the controller in their dominant hand, and the SIs held the controller in their less-affected hand. Before each trial, a red basket was displayed floating in front of the participant, positioned along their sagittal axis, at the level of the table (Figure 1(b)). To initiate a trial, the participant had to fixate the red basket (alignment measured with the Oculus head position tracker). Once the head position was correctly located, the basket turned green, and the stimuli appeared. This fixation procedure assured a central fixation starting point for each trial. The participants were instructed to push on a button of the Oculus Quest motion controller when they saw the target stimulus (Figure 1(c)). All the trial stimuli appeared for 7000 milliseconds maximum, and if the participants made no response during target presentation trials, the serious game registered an omission (failure to find the target). On 5 additional trials (randomly presented within the 24-target trial set), no target was presented (catch trials; on these trials, 24 distractors were presented). The participant was instructed to make no response, but if a response was made, a false-positive error was recorded.

There were three blocks of trials, with each block composed of 29 trials (24 target trials and 5 catch-trials). In the control condition, no avatar was presented (the participant was the only person in the virtual room). In the avatar conditions, one of two avatars entered the virtual kitchen and stood facing the participant, at the opposite side of the table (the virtual distance between the participant and the avatar was 1m40) (Figure 1(e)). The sex of the avatar was selected by the participant before the experiment, and the two avatars were of the selected sex. In one block, the selected avatar acts as the friendly avatar, it positively interacted with the participant by giving verbal and physical encouragement, while in a second block, the avatar that was not chosen acts as the non-friendly avatar, it does not interact with the participant (no display of verbal and physical encouragement). The order of these three blocks were randomised across participants (see Table 2 for a summary of the Peach test scenario and task). Before each trial block, written and audio instructions were played, and the participants were consistently instructed to respond as fast as they could to the target Peach.

Table 2.

A summary of the Peach test sequence. The participant was exposed to a series of visual scenes, in which the participant performed a task.

Scene	Content	Task	Interactions and software
1	The study begins with the experimenter using the Oculus Quest 2. A simple user interface (UI) containing the title of the serious game and a dialog box is presented: “Enter the participant's ID”.	The experimenter indicates the participant's (anonymised) ID (for data logging).	Interactions with the UI are made using a virtual laser controlled using the Oculus controller (implemented using a prefab of the Oculus library; UIHelpers).
2	The participant is now wearing the Oculus. A gif animation shows which controller button must be pressed to respond to the target, accompanied by written and auditory instructions. They are then invited to “Press the button to start”.	The participant listens to the instructions and then presses the virtual button to initiate the study.	The written and audio instructions are presented. The audio was recorded using iOS Voice Memos.To press the virtual button, the same laser-based control is used to interact with the UI.
3	A second set of written and auditory instructions are presented with the same gif as the previous scene. They are requested to click “continue” to move forward.	The second set of instructions explain that the participant can choose the gender of the avatar (“sous-chef” in the SG storyline).	The same laser-based button is used to interact with the UI and move to the next screen.
4	Pictures of the two avatars and their names are presented (Figure 1(d)), along with a “continue” button.	The participants chose one of the avatars by clicking on the button to select the gender of their “sous-chef” avatar.	The laser-based button is aimed to one of the avatar pictures. A visual feedback is shown to indicate selection. They then press continue.
5	The participants enter the virtual 3D visual kitchen. They are situated in front of a wooden table, with a semi-transparent grey screen hanging in front of them, on which written and illustrated instructions are displayed. A “continue” button is also presented.	The text-audio instructions are displayed explaining the storyline to the participants (“A Pastry Chef needs help to make a Peach Pie”). They must gather peaches as fast as they can. They will be accompanied by their chosen sous-chef, and by a second sous-chef (i.e., the avatar that they did not choose).The participant needs to press “continue” button to move forward.	The audio is presented in a male or female voice based on the choice of avatar (i.e., Philippe vs Françoise).The same laser-based button is used to interact with the UI.
6	Another instruction is displayed on the semi-transparent screen using image and audio. It shows a red basket turning green, and a 2D illustration of the target Peach. A “continue” button is also presented.	Participants are instructed to fixate the red basket until it turns green to launch the serious game.They need to press “continue” button to move forward.	To detect if the participants is looking at certain objects in the virtual environment, the Unity function Raycasting is used (a virtual ray from the headset central camera to the basket).
7	A tutorial is launched after pressing a “continue” button UI.The tutorial is composed of 12 sample trials. The table is first presented with the red basket, at the level of their hand. After fixation, the stimuli appear on the table (1 target and 23 distractors, and no avatar).	Before each trial, participants fixate the red basket, and upon turning green, the trial begins. Participants are requested to search and respond to the target Peach presented among other fruits and vegetables by pressing a button when they see the target (responding as fast as they can).	The participant has up to 7000 ms to respond to the target, otherwise the trial ends and another red basket fixation task appears.The target appears in every cell of the hidden grid, in every block.
8	A final text-audio instruction is displayed, with a “continue” button.Once the serious game is launched, three consecutive blocks are displayed. A break is provided between blocks, to allow the change of avatars and for participant rest.	Participants launch the serious game by pressing the “continue” button when they feel ready.	The presentation of the stimuli is similar to the tutorial, but additionally containing catch trials (no target) and trial with the presence of the avatar.The animations of the avatars were taken from Mixamo data base, and the facial expressions were created using Blend Shapes Editor in Unity.
9	Once the three blocks are complete, the participant is presented with an empty table, with the semi-transparent screen giving written-audio instructions.	The text-audio instructions inform the participant that they gathered enough peaches, and that this is the end of the game. The instruction thanks the participant for their participation.	This is the end of the experiment. The participant removes the headset.

The performance was logged and stored in “CSV” format in the headset storage. This contained various standard data entries including participant (anonymised) ID, trial number, and block number. Target and Catch Trials were registered, and depending upon the participant's response, the result of the response was recorded (i.e., correct response to the target; omission - failure to respond to the target; error - response to a distractor; correct non-response to a catch trial, and a false positive error to a catch trial). The row and column coordinates of the target location were registered. Response time was recorded in milliseconds.

We used the Apple Test,⁷⁸ a standardised hemineglect diagnostic test, to assess the visuo-spatial attention performance of SIs. It is a cancellation test involving marking up to 50 apples from 100 distractors consisting of incomplete apples. The complete and incomplete apples were equally distributed across an invisible grid constituted of 5 columns and 2 rows. The analysed measure included the total omissions (independently of the space) and the omission asymmetry between the contra- and the ipsi-lateral spaces (e.g., the difference of omissions committed between the two contra-lateral columns and the two ipsi-lateral columns).

To evaluate users’ experience, we used the User Experience Questionnaire (UEQ) considering hedonic and pragmatic measures,⁷⁹ represented by 6 scales with twenty-six items in total: (1) Attractiveness: refers to the overall impression of the serious game (Do users like or dislike it?); (2) Perspicuity: refers to how easy it is to get familiar with the serious game (Is it easy to understand?); (3) Efficiency: refers to the ease with which interactions can be conducted within the serious game (Is the interaction efficient and fast?); (4) Dependability: refers to the sentiment of control over the serious game (Does the user feel in control of their interactions?); (5) Stimulation: refers to the motivation and pleasure that the user experience when using the serious game (Is it exciting and motivating to use the serious game?); (6) Novelty: refers to the novelty of the serious game (Is the serious game novel and creative?).^79,80 The Attractiveness scale correspond to 6 items, whereas all the other scales correspond to 4 items. The items format consisted of semantic differential with a 7-points Likert scale. The questionnaire is accessible and available for free in multiple languages, and a data analysis tool is provided to facilitate the evaluation.

2.3 Procedure

The study was conducted in controlled laboratories based at the Institute of Psychological Sciences Research Institute, University of Louvain and in the Cliniques universitaires Saint-Luc, Brussels. The experiment session lasted one hour. At the beginning of the session, the participants were instructed on the experimental design, and they were invited to sign a consent form. They were then invited to sit on a chair at the edge of a physical table, wearing the IVR headset and holding the controller in their dominant (Less-affected) hand. They were then immersed in the virtual environment. They were told a back story of a pastry chef that needed help to make a peach pie. To assist the pastry chef, the participant had to collect as many peaches as they could during the game. They were also told that they would sometimes be accompanied by a sous-chef (the avatar). They were immersed in the virtual environment and invited to select their sous-chef avatar by pressing on a button (i.e., allowing the selection of a male or female avatar) (see Figure 1; each having the friendly and non-friendly versions). Once their choice was made, they were instructed to push on a start button allowing them to enter the virtual kitchen containing the virtual table that corresponded in position to the physical table. The participants could view and visually explore the kitchen and the empty table (containing no stimuli). Written and auditory instructions were then displayed to the participant, and they were invited to push on a button to initiate a training session containing 12 trials (including 2 catch-trials). The objective of this training session was to confirm that the participants understood the instructions, that they could correctly identify the target, and that they engaged and experienced the immersion of the task. At the end of this training session, they were then invited to again push a button to initiate the experiment (three blocks of 29 trials) (see Table 2 for the sequence of the serious game). After completing the three blocks of the Peach test, the patients were then asked to complete UEQ. At the end of the session, the participants were thanked for their participation, and the CI received a payment of 10 EUROS for their participation. All participants completed the experiment.

2.4 Methods of data analyses

For analyses 1 and 2, a Shapiro-Wilk normality test indicated that the data were normally distributed (W (25) = 0.92, p > 0.05). Therefore, the data were analysed using repeated measures ANOVAs (using SPPS IBM). The choice of ANOVA was deemed appropriate for analysing the data, as it can handle multiple independent variables and their interactions. The independent variables were Group (CI, SI:HN+, SI:HN-), Laterality (ipsilateral vs contralateral space relative to the dominant hand in CI/ non-HP hand in SI), Proximity (peri-personal vs extra-personal space), and Avatar (no avatar, friendly avatar, non-friendly avatar). The targets were semi-randomly presented across a 6 column and 4 row hidden grid. Targets presented in columns 1–3 and columns 4–6 were defined as ipsi- or contra-lateral space relative to the dominant hand. For right-handed participants (SI with a left HP), contralateral space corresponded to targets displayed in columns 1–3 and ipsilateral space corresponded to targets displayed in columns 4–6, whereas for left-handed participants (SI with a right HP), contralateral space corresponded to targets displayed in columns 4–6 and ipsilateral space corresponded to targets displayed in columns 1–3. For proximity, peri-personal space corresponded to targets presented in rows 1 and 2, and extra-personal space corresponded to targets presented in rows 3 and 4 (see Figure 1(b)). The dependant variables were the number of omissions for target trials (i.e., a failure to respond to the target by pressing the button when the target stimulus appeared) and mean response time (RT) to correctly respond to target stimuli (i.e., the time between the presentation of the target stimuli and the moment the participant correctly pressed the button; measured in milliseconds), and the number of omissions for target trials (i.e., a failure to respond to the target by pressing the button when the target stimulus appeared). Post-hoc analyses were performed using Bonferroni correction.

In the first analyses, all response to catch trials (i.e., 3 blocks or 5 trials x 61 CI participants, 915 trials) were removed from the total data set (i.e., 3 blocks of 29 trials x 61 participants; 5307 total trials). All the data from three participants were excluded as they pressed the response button when no target stimulus was presented for more than 6/15 catch trials. From the remaining 4176 total trials (58 participants), omissions (38 trials) and abnormal responses (< 250 ms) (2 trials) were removed. Outlier data were identified numerically using a confidence interval of three standard deviations above and below the mean, causing a further 92 trials to be deleted. The analysis was performed on the remaining data set of 4044 trials.

In the second analyses, all response to catch trials (i.e., 15 trials x 34 participants, 510 trials) were removed from the total data set (i.e., 3 blocks of 29 trials x 34 participants; 2958 total trials). From the re-sampled CI data set (i.e., 82 trials x 17 CI; 1224 total trials), 15 omission trials were removed (and there were no abnormal responses, < 250 ms). From the SI:HP- groups (i.e., 82 trials x 11 SI:HN-; 627 total trials), 32 omission trials and 2 trials showing abnormal responses (< 250 ms) were removed. From SI:HN + data set (i.e., 82 trials x 6 SI:HN+; 432), 24 omissions (see Table 1) and 13 abnormal responses (< 250 ms) were removed. Outliers were identified using a three standard deviation confidence interval above and below the mean, causing a further 26 trials to be delated in CI group data. The analysis was performed on the remaining 2336 data set (see Table 3).

Table 3.
The number of omissions made by the SIs.

Omissions in Omissions Omissions

Total ipsi- and in near and with Omissions with non-

number of (contra-) (far) friendly with no friendly

ID omissions lateral space space avatar avatar avatar

Individuals with HP and No HN (SI: HN-) SI01 0 0 (0) 0 (0) 0 0 0

SI02 0 0 (0) 0 (0) 0 0 0

SI05 0 0 (0) 0 (0) 0 0 0

SI06 2 1 (1) 0 (2) 2 0 0

SI07 2 1 (1) 1 (1) 0 1 1

SI09 26 11 (15) 6 (20) 5 11 10

SI12 2 1 (1) 0 (2) 0 1 1

SI13 0 0 (0) 0 (0) 0 0 0

SI15 0 0 (0) 0 (0) 0 0 0

SI16 0 0 (0) 0 (0) 0 0 0

SI17 0 0 (0) 0 (0) 0 0 0

Individuals with HP and HN (SI: HN+) SI03 13 6 (7) 1 (12) 5 3 5

SI04 7 6 (1) 2 (5) 1 1 5

SI08 2 1 (1) 2 (0) 0 0 2

SI10 0 0 (0) 0 (0) 0 0 0

SI11 1 1 (0) 0 (1) 1 0 0

SI14 1 1 (0) 0 (1) 1 0 0

			Omissions in		Omissions		Omissions
Individuals with HP and No HN (SI: HN-)	SI01	0	0 (0)	0 (0)	0	0	0
	SI02	0	0 (0)	0 (0)	0	0	0
	SI05	0	0 (0)	0 (0)	0	0	0
	SI06	2	1 (1)	0 (2)	2	0	0
	SI07	2	1 (1)	1 (1)	0	1	1
	SI09	26	11 (15)	6 (20)	5	11	10
	SI12	2	1 (1)	0 (2)	0	1	1
	SI13	0	0 (0)	0 (0)	0	0	0
	SI15	0	0 (0)	0 (0)	0	0	0
	SI16	0	0 (0)	0 (0)	0	0	0
	SI17	0	0 (0)	0 (0)	0	0	0
Individuals with HP and HN (SI: HN+)	SI03	13	6 (7)	1 (12)	5	3	5
	SI04	7	6 (1)	2 (5)	1	1	5
	SI08	2	1 (1)	2 (0)	0	0	2
	SI10	0	0 (0)	0 (0)	0	0	0
	SI11	1	1 (0)	0 (1)	1	0	0
	SI14	1	1 (0)	0 (1)	1	0	0

The UEQ was completed by the SI only, as they are the end-users. Their scores were analysed using the data analysis tool (an excel sheet) provided by the authors.⁷⁹ The results were encoded in the tool, then an automatic transformation was performed, scaling the items from −3 to +3. Then, the mean and standard deviation of each scale were computed. Scores ranging from −1 to 1 indicate a neutral evaluation of the corresponding scale, scores below −1 indicate a negative evaluation, and scores above 1 represent a positive evaluation of the corresponding scale. These results were compared to a classification benchmark value, containing 468 evaluation study (i.e., these studies concern various products, such as software, web shops and websites). The evaluated product (here, Peach test) was classified, based on the benchmark, into 5 categories per scale: Excellent, Good, Above average, Below average, and Bad.^81,82

3 Results

3.1 Analysis 1: General effects for the large Ci group

The analysis of mean RT showed no main effects for Laterality, F (1, 57) = 2.29, p = 0.14, η²= 0.04, Proximity, F (1, 57) = 0.12, p = 0.74, η²= 0.002, and Avatars, F (2, 114) = 0.27, p = 0.76, η²= 0.005. There were also no significant interactions between Laterality and Avatars, F (2, 114) = 2.04, p = 0.13, η²= 0.034, Proximity and Avatars, F (2, 114) = 1.34, p = 0.27, η²= 0.02, and the three-way interaction between Laterality, Proximity and Avatars, F (2114) = 0.48, p = 0.62, η²= 0.008. However, there was a significant interaction between Laterality and Proximity, F (1, 57) = 8.08, η², p < 0.001, η²= 0.12 (Figure 2(a)). A separate ANOVA was run for each proximity space. As hypothesised, this showed no significant effect of Laterality in both the peri-personal space, F (1,57) = 3.70, p = 0.06, η²= 0.06 and extra-personal space, F (1, 57) = 1.65, p = 0.20, η²= 0.03 (Figure 2(b)).

Figure 2.

(a) Violin plots with boxplots illustrating mean response time (milliseconds) to the peach target in ipsi- and contra-lateral spaces, and in peri- and extra-personal spaces. (b) Boxplots illustrating mean response time (milliseconds) to the Peach in ipsi- and contra-lateral spaces, and in peri- and extra-personal spaces, in the three avatar conditions.

3.2 Analysis 2: Group comparison study

Table 3 shows the total omission scores. As omissions were too few, analyses by ANOVA were not possible. The analysis of mean response time showed no main effect of Laterality, F (1, 31) = 1.62, p = 0.21, η²= 0.05, or for Avatars, F (2, 62) = 0.81, p = 0.45, η²= 0.02. There were no significant interactions between Avatars and Group, F (4, 62) = 0.68, p = 0.61, η²= 0.04, Avatar and Laterality, F (2, 62) = 0.01, p = 0.99, η²= 0.00, Avatar and Proximity, F (2, 62) = 0.87, p = 0.42, η²= 0.03, and Proximity and Laterality, F (1, 31) = 0.48, p = 0.49, η²= 0.01. There were also no significant three-way interactions between Avatars, Laterality, and Group, F (4,62) = 1.00, p = 0.41, η²= 0.06, Avatars, Proximity, and Group, F (4,62) = 0.56, p = 0.69, η²= 0.03, and Avatars, Laterality, and Proximity, F (2,62) = 0.02, p = 0.98, η²= 0.00.

As predicted, there was a main effect of Group, F (2,31) = 10.49, p < 0.001, η²= 0.40. This showed that the SI:HN + group was the slowest, then the SI:HN- group, and that both SI groups were slower than the CI group (SI:HN+: M = 2961.26, SD = 247.17; SI:HN-: M = 2290.36, SD = 182.55; CI: M = 1691.17, SD = 146.84). Also, as predicted, there was a main effect of Proximity, F (1,31) = 22.44, p < 0.001, η²= 0.42, with all participants being slower in extra-personal compared to peri-personal space (Peri-personal space: M = 2164.87, SD = 100.13; Extra-personal space: M = 2463.66, SD = 133.18). There were significant interactions between Group and Laterality, F (2,31) = 3.28, p < 0.001, η²= 0.17, and Group and Proximity, F (2,31) = 8.57, p < 0.001, η²= 0.36 (Figures 3). Separated ANOVAs were run for each Group. For each of the three groups, there was no effects of Laterality for CI, F (1,16) = 0.03, p = 0.86, η²= 0.00; SI:HN-, F (1,10) = 0.44, p = 0.52, η²= 0.04, and; SI:HN+, F (1,5) = 0.4.41, p = 0.09, η²= 0.47. For proximity, the CI group showed no effects, F (1,16) = 0.33, p = 0.57, η²= 0.02, but there were significant effects for both SI groups: SI:HN-, F (1,10) = 7.74, p < 0.001, η²= 0.43, and SI:HN + group, F (1,5) = 7.27, p < 0.001, η²= 0.59. Both analyses showed that the SI:HN + and SI:HN- were slower in extra-personal (SI:HN+: M = 3251.76, SD = 289.97 vs. SI:HN-: M = 2460.09, SD = 214.16) than in peri-personal (SI: HP + HN: M = 2670.77, SD = 218.02 and SI: HP-HN: M = 2120.62, SD = 161.02).

Figure 3.

(a) Violin plots with boxplots illustrating mean response time (milliseconds) to the peach (target) in ipsi- and contralateral space in SI:HN+, SI:HN- and CI groups. (b) Violin plots with boxplots illustrating mean response time (milliseconds) to the Peach (target) in peri- and extra-personal spaces in SI:HN+, SI:HN- and CI groups.

3.3 Analysis 3: Peach test user experience

The analysis of the 17 SI responses to the UEQ showed that overall, the Peach test had positive evaluations. For all the evaluated dimensions, our results were in the positive range (Figure 4(a)). The highest values obtained were on the Perspicuity dimension (M = 2.69; SD = 0.24), indicating that the Peach test is clear, very easy to understand and participants easily learn how to perform the tasks. The Attractiveness dimension showed a positive rating (M = 1.88; SD = 1.62), showing that the Peach test was enjoyable and attractive, and gave an excellent overall impression. On the Efficiency dimension (M = 1.84; SD = 0.72), participants found the interaction with the test to be efficient and fast. On the Stimulation dimension (M = 1.38; SD = 1.83), participants thought that the Peach test was exciting, motivating, and fun to use. On the Novelty dimension (M = 1.37; SD = 1.95), participants found the Peach test to be innovative and creative. Finally, on the Dependability dimension (M = 1,29; SD = 0.95), participants reported a feeling of safety, and control of their interaction with the virtual environment.

Figure 4.

(a) A graphical representation of the UEQ dimension values given by the SIs. (b) A graphical representation of the benchmark on Peach test.

Based on the UEQ Data Analysis tool, a benchmark was conducted, comparing the Peach test to the 468 products included in the data set. The results demonstrated that Peach test had an excellent mean value on the Attractiveness and Perspicuity dimensions (M = 1.88, & M = 2.69, respectively), meaning that Peach test lies in the range of the 10% best results. It has a good mean value on the Efficiency, Stimulation, and Novelty dimensions (M = 1.84, M = 1.38, & M = 1.37, respectively), meaning that 10% of the products included in the benchmark data set have better results than Peach test, and 75% of these products have worst results. The analysis also showed an above average mean value on the Dependability dimensions (M = 1.29), meaning that 50% of the products included in the benchmark data set had better results than the Peach test, and 25% of these results had worst results (Figure 4(b)).

4 Discussion

In this paper, we presented a new IVR serious game to evaluate spatial attention by contrasting responses to targets presented in ipsi- versus contra-lateral, and peri- versus extra-personal spaces. The assessment used an interactive serious game, where participants were immersed in a kitchen simulation, and requested to perform a simple visual search task in the presence of avatars. Our first objective was to evaluate the feasibility and user experience of this serious game. For this, we tested a group of sixty healthy control individuals. We hypothesised that the CI would show no Laterality effect, but show a proximity effect, and that there would be an interaction between avatar and proximity. Our findings indicated no significant differences when responding to a target presented in the different space contrasts (ipsilateral / contralateral, and peri-personal / extra-personal spaces), and no significant differences in responding to the target presented in the peri- and extra-personal spaces in the no-avatar and the two avatar conditions for both omissions and response time. These results were surprising, because there is a substantial body of evidence showing that perspective taking of a third person presented in front of a participant can automatically influence response performance.^83–86 Further, several papers have demonstrated evidence that participants spontaneously took the spatial perspective of another's perspective causing reversal effects for right/left or near/far spaces.^39,44,45,87 Despite these null effects, the advantage is a stable data profile on which to contrast patients.

In the second analysis of this paper, we contrasted results from a group of post-stroke individuals with hemiparesis without hemineglect, a group of post-stoke individuals with hemiparesis and hemineglect, and a resampled selection of age-matched controls. We hypothesised that SI:HN + would show a Laterality effect, whereas SI:HN- and CI would not show Laterality effects, and further, that there would be an interaction between group, Laterality and avatar. Our findings indicated that both SI groups were slower than CI, with SI:HN + being the slowest group, but that there was no significant effect of Laterality in the SI:HN + group for both omissions and response time measures, thereby not supporting our hypotheses. This lack of lateralised effect can likely be explained by the SI no longer showing HN at the time of testing, evaluated by standardised test (e.g., Apples test⁷⁸). Despite these inconclusive results for the SI groups, the present Peach test could still be used to detect spatial attention impairments from omission and response time measures, contrasting target ipsi- and contra-lateral spaces as well as peri-personal and extra-personal spaces. The predicted hypotheses may have more likely demonstrated significant results with acute patients who showed HN at the time of testing.

Our results showed an effect of proximity for both SI groups, with slower responses to the target presented in extra- compared to the peri-personal space. However, there were no interaction effects between group, proximity and avatar, and between group, Laterality and avatar. This suggest that there was no shift in the proximity effects for both SI groups with the presence of the avatar, and that the predicted shift for Laterality with the SI:HN + for perspective was not found. This reversal effect may have been better demonstrated in neurological patients showing representational hemineglect^41–43 or as proposed earlier, with patients having HN at the time of testing. It is possible that acute HN individuals would have shown a lateral bias without the avatar, and a reversal lateralised effect with the presence of the avatar. The same argument can be made for responses to targets in peri- and extra-personal spaces, with the effect for proximity with the SI groups not showing reversal effects with compared to without the avatar due to the SI not being acute. Alternatively, this result may indicate that the avatars were not suitable to provoke reverse effects. Clearly, more studies are needed to test whether the presence of animated 3D avatars can cause perspective taking shifts. These additional studies would add to growing evidence and challenges for the perspective taking paradigm.^88–90

In traditional neuropsychology assessment, omission measures are currently the standard metric to assess asymmetry bias in spatial attention.^91–94 When combined with response time, the test diagnostic sensitivity to detect spatial attention impairment is increased, by providing more measures to distinguish between patients with and without hemineglect.^20,95 HN is a heterogenous syndrome that is more frequent and/or severe after a right lesion, but there is also evidence of right HN after a left lesion.^2,96–99 Furthermore, HN can differentially moderate in peri-personal and/or extra-personal space.^9,100 It is possible that these different sub-types of HN are not systematically evaluated with the current tools used in the clinics. Our peach test can be used to detect biases in responding to ipsi- and contra-lateral space, and it can also simultaneously assess peri- and extra-personal spaces, thus increasing future clinical utility with a broader post-stroke population. An important advantage of using IVR here was that it allowed for the use of large virtual spaces that simulate real-world environments, which can lead to a more ecological task performance.

We evaluated the user experience of the post-stroke patients using UEQ.⁷⁹ These results showed that the Peach test was easy to understand and use, due to clear and intuitive interactions. The post-stroke individuals, as the end-users, were able to perform the serious game fully without aid from the experimenter. The overall experience was rated positively, and both SI groups enjoyed the serious game and were motivated by the implementation of a clear and fun storyline, that they engaged with. This aligns with Ong, Weibin and Vallabhajosyula,¹⁰¹ who suggest that the use of serious games increases patients’ satisfaction with neuropsychological assessment. It is also compatible with several recommendations that support the development of ecologically relevant and entertaining neuropsychological measures to enhance the adherence of the participants and present usable and efficient devices.^102–104 These results suggest that post-stroke individuals can easily interact with sophisticated technologies, enjoying the experience, thereby confirming the usability of the Peach test as a viable concept for assessment of SI.

This paper had several limitations that need to be addressed in future studies. Testing the feasibility of the Peach test with neurological patients showing HN at the time of testing is needed, demonstrated using standardised measures. These neurological hemineglect patients must show lateral bias effects in order to test if the presence of an avatar causes reversal effects. It could also be that a larger sample of patients are tested, perhaps showing differences for the avatar perspective taking effects. Another limitation in this paper is related to the statistical analysis performed. A repeated analysis of variance (ANOVA) was performed on the dataset, and outliers were removed from the CI dataset using a normal distribution method. While this is a common practice in the field, there may be better statistical method that allow for more effective contrast between clinical and control participants. Future studies should consider the use of an alternative methods, such as Mixed-Effects Models, which are more robust in dealing with sphericity violations and can handle unbalanced data and missing values.¹⁰⁵ Additional improvements could be implemented to the Peach test, such as replacing the key-button response by a more naturalistic interaction with the targets, dispensing the need for catch trials. For this, the inclusion of hand tracking that is available with the head mounted display could be more efficient and useful, since it would enhance the user-experience and provide with action metrics. The compromise will be that the test can only be performed in peri-personal space. Another improvement could be to vary the difficulty of the task to increase test sensitivity for different severities of patients, allowing detection of HN in chronic patients. Moreover, the presence of different difficulty levels could help personalising the adaptation of the assessment to a treatment based serious game for patients.^106,107

5 Conclusion

To sum up, we have used a novel serious game in IVR for the assessment of spatial attention in hemineglect. Although our findings were inconclusive, the Peach test showed excellent usability and acceptance by SI. The serious game comprehended a scenario where participants interacted with avatars. Future studies should focus on investigating the impact that the presence of an avatar has on spatial attention performance, and clinical validity and reliability of Peach test should be studied to allow the use of this virtual test in clinical settings.

Footnotes

Acknowledgements

We would like to thank Stéphane Grade, Camille Ganci and Anthony Garcia for technical support for the project. We also would like to thank all the clinicians of the Cliniques universitaires Saint-Luc who helped with the recruitment of the patients, and all the participants who took part of this study.

ORCID iD

Khawla Ajana

Authors’ contributions

The authors confirm their contribution to the paper as follows: K.A. and M.G.E. developed and designed the study, K.A. conducted the experiment, K.A. and M.G.E. analysed and interpreted the results, M.G.E. substantially contributed in the manuscript editing, T.L. and G.E. participated in the manuscript editing and result interpretation. All authors provided critical feedback and helped shape the research, analysis and approved the final manuscript.

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The Saint–Luc–UCLouvain-Hospital–Faculty Ethics Committee approved the study (reference number: 2015/10FEV/053). All participants provided written informed consent before enrolment.

Consent for publication

Not applicable

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Availability of data and materials

The raw data that support the findings of this study are openly available in the open data archive of University of Louvain (10.14428/DVN/CH500 M).

References

Barrett

Buxbaum

Coslett

, et al. Cognitive rehabilitation interventions for neglect and related disorders: moving from bench to bedside in stroke patients. J Cogn Neurosci. 2006; 18: 1223–1236.

Vallar

. Spatial hemineglect in humans. Trends Cogn Sci. 1998; 2: 87–97.

Mesulam

M-M

. Spatial attention and neglect: parietal, frontal and cingulate contributions to the mental representation and attentional targeting of salient extrapersonal events. Philos Trans R Soc London Ser B 1999; 354: 1325–1346.

Duncan

Bundesen

Olson

, et al. Systematic analysis of deficits in visual attention. J Exp Psychol: Gen 1999; 128: 450.

Buxbaum Ferraro

Veramonti

, et al. Hemispatial neglect: subtypes, neuroanatomy, and disability. Neurology 2004; 62: 749–756.

Kerkhoff

. Spatial hemineglect in humans. Prog Neurobiol. 2001; 63: 1–27.

Esposito

Shekhtman

Chen

. Prevalence of spatial neglect post-stroke: a systematic review. Ann Phys Rehabil Med. 2021; 64: 101459.

Chen

Hreha

, et al. Kessler foundation neglect assessment process uniquely measures spatial neglect during activities of daily living. Arch Phys Med Rehabil. 2015; 96: 869–76.e1.

Ten Brink

Biesbroek

Oort

, et al. Peripersonal and extrapersonal visuospatial neglect in different frames of reference: a brain lesion-symptom mapping study. Behav Brain Res. 2019; 356: 504–515.

10.

Butler

Eskes

Vandorpe

. Gradients of detection in neglect: comparison of peripersonal and extrapersonal space. Neuropsychologia 2004; 42: 346–358.

11.

Van der Stoep

Visser-Meily

Kappelle

, et al. Exploring near and far regions of space: distance-specific visuospatial neglect after stroke. J Clin Exp Neuropsychol. 2013; 35: 799–811.

12.

Bailey

Riddoch

Crome

. Evaluation of a test battery for hemineglect in elderly stroke patients for use by therapists in clinical practice. NeuroRehabilitation 2000; 14: 139–150.

13.

Grattan

. Do neglect assessments detect neglect differently? Am J Occup Ther 2017; 71: 7103190050p1–p9.

14.

Checketts

Mancuso

Fordell

, et al. Current clinical practice in the screening and diagnosis of spatial neglect post-stroke: findings from a multidisciplinary international survey. Neuropsychol Rehabil. 2021; 31: 1495–1526.

15.

Wilson

Cockburn

Halligan

. Development of a behavioral test of visuospatial neglect. Arch Phys Med Rehabil. 1987; 68: 98–102.

16.

Evald

Wilms

Nordfang

. Assessment of spatial neglect in clinical practice: a nationwide survey. Neuropsychol Rehabil. 2021; 31: 1374–1389.

17.

Azouvi

. The ecological assessment of unilateral neglect. Ann Phys Rehabil Med. 2017; 60: 186–190.

18.

Reynolds

Mason

. Measurement and statistical problems in neuropsychological assessment of children. In: Reynolds CR and Fletcher-Janzen E (eds) Handbook of clinical child neuropsychology. 3rd ed. New York, NY: Springer, 2009, pp.203–230.

19.

Terruzzi

Albini

Massetti

, et al. The neuropsychological assessment of unilateral spatial neglect through computerized and virtual reality tools: a scoping review. Neuropsychol Rev. 2024; 34: 363–401.

20.

Zimmermann

Fimm

. Test for attentional performance (TAP). PsyTest, Herzogenrath 1995; 1995: 76–77.

21.

Kane

Kay

. Computerized assessment in neuropsychology: a review of tests and test batteries. Neuropsychol Rev. 1992; 3: 1–117.

22.

Parsey

Schmitter-Edgecombe

. Applications of technology in neuropsychological assessment. Clin Neuropsychol. 2013; 27: 1328–1361.

23.

Rengachary

d'Avossa

Sapir

, et al.

Is the posner reaction time test more accurate than clinical tests in detecting left neglect in acute and chronic stroke?

Arch Phys Med Rehabil. 2009; 90: 2081–2088.

24.

Mozer. Frames of reference in unilateral neglect and visual perception: a computational perspective. Psychol Rev. 2002; 109: 156.

25.

Farah

Brunn

Wong

, et al. Frames of reference for allocating attention to space: evidence from the neglect syndrome. Neuropsychologia 1990; 28: 335–347.

26.

Behrmann

Moscovitch

. Object-centered neglect in patients with unilateral neglect: effects of left-right coordinates of objects. J Cogn Neurosci. 1994; 6: 1–16.

27.

Vallar

Calzolari

. Unilateral spatial neglect after posterior parietal damage. In: Vallar

and Coslett

(eds) Handbook of clinical neurology, Vol. 151. Amsterdam, NL: Elsevier, 2018, pp.287–312.

28.

Driver

Pouget

. Object-centered visual neglect, or relative egocentric neglect? J Cogn Neurosci. 2000; 12: 542–545.

29.

Kaiser

Villadsen

Samani

, et al. Virtual reality and eye-tracking assessment, and treatment of unilateral spatial neglect: systematic review and future prospects. Front Psychol. 2022; 13: 787382.

30.

Ladavas

. Is the hemispatial deficit produced by right parietal lobe damage associated with retinal or gravitational coordinates? Brain 1987; 110: 167–180.

31.

Stein

. The representation of egocentric space in the posterior parietal cortex. Behav Brain Sci. 1992; 15: 691–700.

32.

Chokron

. Right parietal lesions, unilateral spatial neglect, and the egocentric frame of reference. NeuroImage 2003; 20: S75–S81.

33.

Vogeley

Fink

. Neural correlates of the first-person-perspective. Trends Cogn Sci. 2003; 7: 38–42.

34.

Wang

Chen

Knierim

. Egocentric and allocentric representations of space in the rodent brain. Curr Opin Neurobiol. 2020; 60: 12–20.

35.

Zaehle

Jordan

Wüstenberg

, et al. The neural basis of the egocentric and allocentric spatial frame of reference. Brain Res. 2007; 1137: 92–103.

36.

Southgate

. Are infants altercentric? The other and the self in early social cognition. Psychol Rev. 2020; 127: 505–523.

37.

Kampis

Southgate

. Altercentric cognition: how others influence our cognitive processing. Trends Cogn Sci. 2020; 24: 945–959.

38.

Mainwaring

Tversky

Ohgishi

, et al. Descriptions of simple spatial scenes in English and Japanese. Spat Cogn Comput. 2003; 3: 3–42.

39.

Tversky

Hard

. Embodied and disembodied cognition: spatial perspective-taking. Cognition 2009; 110: 124–129.

40.

St. Jacques

Iriye

. Putting things into perspective. Abingdon: Taylor & Francis, 2022, pp.1–8.

41.

Bisiach

Luzzatti

. Unilateral neglect of representational space. Cortex 1978; 14: 129–133.

42.

Della Sala

Logie

Beschin

, et al. Preserved visuo-spatial transformations in representational neglect. Neuropsychologia 2004; 42: 1358–1364.

43.

Becchio

Del Giudice

Dal Monte

, et al. In your place: neuropsychological evidence for altercentric remapping in embodied perspective taking. Soc Cogn Affect Neurosci 2013; 8: 165–170.

44.

Samson

Apperly

Braithwaite

, et al. Seeing it their way: evidence for rapid and involuntary computation of what other people see. J Exp Psychol 2010; 36: 1255.

45.

Cavallo

Ansuini

Capozzi

, et al. When far becomes near: perspective taking induces social remapping of spatial relations. Psychol Sci. 2017; 28: 69–79.

46.

Frith

. The social brain: allowing humans to boldly go where no other species has been. Philos Trans R Soc, B. 2010; 365: 165–176.

47.

Tosi

Pickering

Branigan

. Speakers’ use of agency and visual context in spatial descriptions. Cognition 2020; 194: 104070.

48.

Yang

M-L

Yang

C-C

Chiou

W-B

. When guilt leads to other orientation and shame leads to egocentric self-focus: effects of differential priming of negative affects on perspective taking. Soc Behav Pers: Int J 2010; 38: 605–614.

49.

Bukowski

Samson

. Can emotions influence level-1 visual perspective taking? Cogn Neurosci. 2016; 7: 182–191.

50.

Carlo

Allen

Buhman

. Facilitating and disinhibiting prosocial behaviors: the nonlinear interaction of trait perspective taking and trait personal distress on volunteering. Basic Appl Soc Psych. 1999; 21: 189–197.

51.

Keysar

. The effect of culture on perspective taking. Psychol Sci. 2007; 18: 600–606.

52.

Allison

Puce

McCarthy

. Social perception from visual cues: role of the STS region. Trends Cogn Sci. 2000; 4: 267–278.

53.

Riva

Mancuso

Cavedoni

, et al. Virtual reality in neurorehabilitation: a review of its effects on multiple cognitive domains. Expert Rev Med Devices. 2020; 17: 1035–1061.

54.

Wiley

Khattab

Tang

. Examining the effect of virtual reality therapy on cognition post-stroke: a systematic review and meta-analysis. Dis Rehabil: Assistive Technol 2022; 17: 50–60.

55.

Kalawsky

. Exploiting virtual reality techniques in education and training: Technological issues. http://www man ac uk/MVC/SIMA/vrtech/title html. 1996.

56.

Anthes

García-Hernández

Wiedemann

Kranzlmüller

(eds). State of the art of virtual reality technology. In: 2016 IEEE aerospace conference, 2016: IEEE.

57.

Riva

Gamberini

. Virtual reality as telemedicine tool: technology, ergonomics and actual applications. Technol Health Care. 2000; 8: 113–127.

58.

Ryan

Cornick

Blascovich

, et al. Virtual reality: whence, how and what for. In: Rizzo

Bouchard

(eds) Virtual reality for psychological and neurocognitive interventions. New York, NY: Springer New York, 2019, pp.15–46.

59.

Kardong-Edgren

Farra

Alinier

, et al. A call to unify definitions of virtual reality. Clin Simul Nurs 2019; 31: 28–34.

60.

Rizzo

Koenig

. Is clinical virtual reality ready for primetime? Neuropsychology 2017; 31: 877–899.

61.

Garrett

Taverner

Gromala

, et al. Virtual reality clinical research: promises and challenges. JMIR Serious Games 2018; 6: e10839.

62.

Pourmand

Davis

Lee

, et al. Emerging utility of virtual reality as a multidisciplinary tool in clinical medicine. Games Health J. 2017; 6: 263–270.

63.

Massetti

Da Silva

Crocetta

, et al. The clinical utility of virtual reality in neurorehabilitation: a systematic review. J Cent Nerv Syst Dis. 2018; 10: 1179573518813541.

64.

Tsirlin

Dupierrix

Chokron

, et al. Uses of virtual reality for diagnosis, rehabilitation and study of unilateral spatial neglect: review and analysis. Cyberpsychol Behav. 2009; 12: 175–181.

65.

Zheng

. Virtual reality and serious games in healthcare. In: Advanced computational intelligence paradigms in healthcare 6: Virtual reality in psychotherapy, rehabilitation, and assessment. Berlin, DE: Springer, 2011, pp.169–192.

66.

Feng

González

Amor

, et al. Immersive virtual reality serious games for evacuation training and research: a systematic literature review. Comput Educ. 2018; 127: 252–266.

67.

Burke

McNeill

Charles

, et al. Optimising engagement for stroke rehabilitation using serious games. Vis Comput. 2009; 25: 1085–1099.

68.

Alankus

Lazar

May

, et al. (eds). Towards customizable games for stroke rehabilitation. In: Proceedings of the SIGCHI conference on human factors in computing systems, 2010.

69.

Tong

Chignell

(eds). Developing a serious game for cognitive assessment: choosing settings and measuring performance. In: Proceedings of the second international symposium of Chinese CHI, 2014.

70.

Mubin

Alnajjar

Al Mahmud

, et al. Exploring serious games for stroke rehabilitation: a scoping review. Dis Rehabil: Assistive Technol 2022; 17: 159–165.

71.

Knobel

Kaufmann

Gerber

, et al. Immersive 3D virtual reality cancellation task for visual neglect assessment: a pilot study. Front Hum Neurosci. 2020; 14: 180.

72.

Huygelier

Schraepen

Lafosse

, et al. An immersive virtual reality game to train spatial attention orientation after stroke: a feasibility study. Appl Neuropsychol: Adult 2022; 29: 915–935.

73.

Ajana

Everard

Lejeune

, et al. A feature and conjunction visual search immersive virtual reality serious game for measuring spatial and distractor inhibition attention using response time and action kinematics. J Clin Exp Neuropsychol. 2023: 1–12.

74.

Bowen

McKenna

Tallis

. Reasons for variability in the reported rate of occurrence of unilateral spatial neglect after stroke. Stroke 1999; 30: 1196–1202.

75.

Golisz

. Dynamic assessment and multicontext treatment of unilateral neglect. Top Stroke Rehabil. 1998; 5: 11–28.

76.

Technologies U. Food Pack - Mixed. [Available from: https://assetstore.unity.com/packages/3d/props/food/food-pack-mixed-154349. n.d.

77.

Technologies

. Modern Supermarket n.d. [Available from: https://assetstore.unity.com/packages/3d/environments/modern-supermarket-186122.

78.

Bickerton

Samson

Williamson

, et al. Separating forms of neglect using the apples test: validation and functional prediction in chronic and acute stroke. Neuropsychology 2011; 25: 567–580.

79.

Laugwitz

Held

Schrepp

(eds). Construction and evaluation of a user experience questionnaire. In: Symposium of the Austrian HCI and usability engineering group, 2008: Springer.

80.

Schrepp

Hinderks

Thomaschewski

(eds). Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In: Design, User Experience, and Usability Theories, Methods, and Tools for Designing the User Experience: Third International Conference, DUXU 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22–27, 2014, Proceedings, Part I 3: Springer.

81.

Schrepp

. User experience questionnaire handbook. All you need to know to apply the UEQ successfully in your project. 2015.

82.

Schrepp

Thomaschewski

Hinderks

. Construction of a benchmark for the user experience questionnaire (UEQ). 2017.

83.

Gunalp

Moossaian

Hegarty

. Spatial perspective taking: effects of social, directional, and interactive cues. Mem Cognit. 2019; 47: 1031–1043.

84.

Surtees

Apperly

Samson

. Similarities and differences in visual and spatial perspective-taking processes. Cognition 2013; 129: 426–438.

85.

Freundlieb

Sebanz

Kovács

ÁM

. Out of your sight, out of my mind: knowledge about another person’s visual access modulates spontaneous visuospatial perspective-taking. J Exp Psychol: Hum Percept Perf 2017; 43: 1065.

86.

von Salm-Hoogstraeten

Bolzius

Müsseler

. Seeing the world through the eyes of an avatar? Comparing perspective taking and referential coding. J Exp Psychol: Hum Percept Perf 2020; 46: 264.

87.

Furlanetto

Cavallo

Manera

, et al. Through your eyes: incongruence of gaze and action increases spontaneous perspective taking. Front Hum Neurosci. 2013; 7: 55.

88.

Cole

Millett

. The closing of the theory of mind: a critique of perspective-taking. Psychon Bull Rev. 2019; 26: 1787–1802.

89.

Cole

Atkinson

, et al.

Do humans spontaneously take the perspective of others?

Acta Psychol (Amst). 2016; 164: 165–168.

90.

Santiesteban

Catmur

Hopkins

, et al.

Avatars and arrows: implicit mentalizing or domain-general processing?

J Exp Psychol: Hum Percept Perf 2014; 40: 929.

91.

Demeyere

Sun

Milosevich

, et al. Post-stroke cognition with the Oxford cognitive screen vs montreal cognitive assessment: a multi-site randomized controlled study (OCS-CARE). AMRC Open Res 2019; 1: 12.

92.

Basagni

De Tanti

Damora

, et al. The assessment of hemineglect syndrome with cancellation tasks: a comparison between the bells test and the apples test. Neurol Sci. 2017; 38: 2171–2176.

93.

Rorden

Karnath

H-O

. A simple measure of neglect severity. Neuropsychologia 2010; 48: 2758–2763.

94.

Plummer

Morris

Dunai

. Assessment of unilateral neglect. Phys Ther. 2003; 83: 732–740.

95.

Montedoro

Alsamour

Dehem

, et al. Robot diagnosis test for egocentric and allocentric hemineglect. Arch Clin Neuropsychol. 2018; 34: 481–494.

96.

Vallar

Rusconi

Barozzi

, et al. Improvement of left visuo-spatial hemineglect by left-sided transcutaneous electrical stimulation. Neuropsychologia 1995; 33: 73–82.

97.

Azouvi

Samuel

Louis-Dreyfus

, et al. Sensitivity of clinical and behavioural tests of spatial neglect after right hemisphere stroke. J Neurol Neurosurg Psychiatry. 2002; 73: 160–166.

98.

Smania

Martini

Gambina

, et al. The spatial distribution of visual attention in hemineglect and extinction patients. Brain J Neurol. 1998; 121: 1759–1770.

99.

Ogden

. Anterior-posterior interhemispheric differences in the loci of lesions producing visual hemineglect. Brain Cogn. 1985; 4: 59–75.

100.

Aimola

Schindler

Simone

, et al. Near and far space neglect: task sensitivity and anatomical substrates. Neuropsychologia 2012; 50: 1115–1123.

101.

Ong

DSM

Weibin

Vallabhajosyula

. Serious games as rehabilitation tools in neurological conditions: a comprehensive review. Technol Health Care. 2021; 29: 15–31.

102.

Krohn

Tromp

Quinque

, et al. Multidimensional evaluation of virtual reality paradigms in clinical neuropsychology: application of the VR-check framework. J Med Internet Res 2020; 22: e16724.

103.

Lumsden

Edwards

Lawrence

, et al. Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 2016; 4: e11.

104.

Bauer

Iverson

Cernich

, et al. Computerized neuropsychological assessment devices: joint position paper of the American academy of clinical neuropsychology and the national academy of neuropsychology. Clin Neuropsychol. 2012; 26: 177–196.

105.

Bates

Mächler

Bolker

, et al. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:14065823. 2014.

106.

Bilder

Reise

. Neuropsychological tests of the future: how do we get there from here? Clin Neuropsychol 2019; 33: 220–245.

107.

Brooks

Strauss

Sherman

, et al. Developments in neuropsychological assessment: refining psychometric and clinical interpretive methods. Can Psychol 2009; 50: 196.

							Diagnosed	Asymmetry
							Hemineglect	Apples
						Months	(side L/R)	Test
	Id	Gender	Age range	Stroke site	Stroke type	post-onset	(note 1)	(note 2)
Individuals with HP and No HN (SI:HN-)	SI01	F	50–55	Left middle cerebral artery	ischaemic	3,5	No (L)	0
	SI02	M	55–60	Right thalamus	ischaemic	1,5	No (R)	0
	SI05	M	65–70	Left paramedian pontine	ischaemic	38	No (L)	1
	SI06	M	45–50	Left middle cerebral artery	ischaemic	22,8	No (L)	0
	SI07	M	60–65	Right lacunar internal capsule	ischaemic	18	No (R)	0
	SI09	F	75–80	Left posterior capsulo-thalamic	ischaemic	1,6	No (L)	0
	SI12	M	70–75	Right posterior branch of the right sylvian artery	ischaemic	10	No (R)	2
	SI13	M	70–75	Right sylvian fissure	ischaemic	2	No (R)	−1
	SI15	M	45–50	Left temporal lobe	ischaemic	0,2	No (L)	−1
	SI16	M	45–50	Left lenticulostriate artery	haemorrhagic	7	No (L)	0
	SI17	F	60–65	Right sylvian fissure	ischaemic	20	No (R)	0
Individuals with HP and HN (SI:HN+)	SI03	M	45–50	Left middle cerebral artery	ischaemic	15	Yes (L)	1
	SI04	M	60–65	Left parietal intraparenchymal	haemorrhagic	13	Yes (L)	0
	SI08	M	50–55	Right superficial and deep sylvian and secondary occipital	ischaemic	17,5	Yes (R)	0
	SI10	M	60–65	Left capsulo-thalamic	haemorrhagic	4,6	Yes (L)	1
	SI11	M	60–65	Right superficial middle cerebral vein	ischaemic	5,8	Yes (R)	0
	SI14	M	65–70	Right sylvian fissure	ischaemic	19,4	Yes (R)	3*

	Mean All (SD)		60 (10.30)			11,75 (10.12)
	Mean SI: HP-HN (SD)		60.54 (12.01)			11.32 (12.02)
	Mean SI: HP + HN (SD)		59 (7.01)			12.55 (6.10)

			Omissions in		Omissions		Omissions
		Total	ipsi- and	in near and	with	Omissions	with non-
		number of	(contra-)	(far)	friendly	with no	friendly
	ID	omissions	lateral space	space	avatar	avatar	avatar
Individuals with HP and No HN (SI: HN-)	SI01	0	0 (0)	0 (0)	0	0	0
	SI02	0	0 (0)	0 (0)	0	0	0
	SI05	0	0 (0)	0 (0)	0	0	0
	SI06	2	1 (1)	0 (2)	2	0	0
	SI07	2	1 (1)	1 (1)	0	1	1
	SI09	26	11 (15)	6 (20)	5	11	10
	SI12	2	1 (1)	0 (2)	0	1	1
	SI13	0	0 (0)	0 (0)	0	0	0
	SI15	0	0 (0)	0 (0)	0	0	0
	SI16	0	0 (0)	0 (0)	0	0	0
	SI17	0	0 (0)	0 (0)	0	0	0
Individuals with HP and HN (SI: HN+)	SI03	13	6 (7)	1 (12)	5	3	5
	SI04	7	6 (1)	2 (5)	1	1	5
	SI08	2	1 (1)	2 (0)	0	0	2
	SI10	0	0 (0)	0 (0)	0	0	0
	SI11	1	1 (0)	0 (1)	1	0	0
	SI14	1	1 (0)	0 (1)	1	0	0