Using Virtual Reality to Assess Associations Between Paranoid Ideation and Components of Social Performance: A Pilot Validation Study

Abstract

Virtual reality (VR) enables the real-time assessment of paranoid ideation and of associated social performance. In this two-phase study, we aimed to recruit a general population sample to investigate the association between trait paranoia and cognitive, emotional, and behavioral components of social performance. In phase 1, a general population online survey (N = 609) investigated how trait paranoia related to fear of negative evaluation (FNE), interpersonal sensitivity, social avoidance and distress (SAD), mood, and demographic characteristics. In phase 2, we piloted a new VR social situation paradigm (a party in a bar) with a subsample of participants who scored high or low in trait paranoia. Eighty-nine participants entered the VR party in a bar scenario to evaluate the acceptability of the task and the relationship between paranoid ideation and social performance. As hypothesized, in phase 1, trait paranoia was associated with FNE, SAD, interpersonal sensitivity, mood, and demographics (all small-to-medium effects); in phase 2, participants found the VR environment acceptable and immersive; exposure to the VR environment elicited a range of cognitive, emotional, and behavioral components of social performance; and high trait paranoia participants reported higher state paranoia and greater negative components of social performance (all medium effects). The study tests a novel sample recruited predominantly online and validates the virtual environment for psychological assessment and treatment. This result suggests that the new VR scenario could be used as a psychological assessment and treatment tool for people who experience paranoia in social situations.

Introduction

Paranoid ideation has been conceptualized as a cognitive response to the perception of interpersonal threat, and can be understood as comprising a spectrum of beliefs concerning ideas of reference and persecution and involving the thought that others intend harm, related to appraisals of changed and confusing experiences of anomalous internal states.¹ Research shows that components of social performance, such as fear of negative evaluation (FNE),² interpersonal sensitivity,³ and social avoidance and distress (SAD),⁴ are important indicators of an individual's ability to initiate and enter into social interactions and are adversely affected in people with paranoid ideation.^3–6 Indeed, people with high levels of paranoid ideation experience greater social defeat⁷ and can inhibit their interactions with others, experience greater isolation, and abandonment of activities.⁸

Virtual reality (VR) offers a promising new way to research real-time paranoid ideation and components of social performance, by providing ecologically valid environments with modifiable environmental conditions.^9–12 Previous VR research of social performance in psychosis has been either non-immersive¹³ or focused on one-to-one interactions.¹⁴ An important development of VR research for people with psychosis would be to examine the role of cognitive processes and the associated emotional and physiological responses that relate to social performance, within an immersive, interactive, multiavatar environment that explicitly requires participants to engage in individual or group social interaction tasks in an overtly social setting.¹¹ Furthermore, any newly designed VR environment will need to be validated as a measurement tool if it is to be used for psychological assessment.

In the current study, we aimed to address these needs. In phase 1, using a general population survey, we aimed to investigate trait paranoia, its demographic correlates, and its associations with social performance. In phase 2, our aims were to evaluate the acceptability of a new VR social environment; to validate the VR environment as a measure of state paranoia; and, controlling for baseline trait paranoid ideation, to test the relationship between state paranoia and social performance. Our aim was not to trigger paranoid ideation in participants but rather to evaluate whether high trait paranoia participants experienced higher state paranoia in a neutral VR environment, compared with lower trait paranoia participants.

Our hypotheses were as follows. (1) In phase 1, trait paranoia would be associated with higher levels of FNE, interpersonal sensitivity, SAD, depression, anxiety, and with demographic characteristics. (2) In phase 2, exposure to the VR social scenario would be acceptable, immersive, and elicit paranoid ideation. (3) High trait paranoia would be associated with high state paranoia. (4) High trait paranoia would be related to greater negative components of social performance in the VR social scenario.

Methods

Phase 1. Survey study

Procedure

This was a cross-sectional cohort study. Participants completed an online survey, advertised as a survey of “thoughts and feelings about social situations.” Survey instructions did not refer to paranoid ideation. Following Green et al.,¹⁵ using a similar methodology, we aimed to recruit a minimum of 300 participants to ensure a representative spread of paranoia scores.

Participants

The online survey was open for 2 months. Participants (N = 609) were working-age adults aged 18–65 years, fluent English speakers, and willing to be invited to the VR laboratory, in the event that they were selected for the phase 2 VR study. Participants were excluded if they self-reported diagnosis of a serious mental health condition (psychosis/bipolar disorder), a neurological disorder, learning disability, or epilepsy. The study was advertised predominantly online, using social media, e-mails, Internet forums, and mailing lists; flyers were distributed in southeast London. Participants were entered into a prize draw to win 4 × £25 shopping vouchers.

Measures

Participants self-reported demographic characteristics and completed the Green et al. Paranoid Thoughts Scale (GPTS), which measures trait paranoid ideation.¹⁵ GPTS consists of 32 items measured on a 5-point scale, from 1 (“Not at all”) to 5 (“Totally”), referring to the past month, with a minimum score of 16 and a maximum score of 160. Parts A and B of GPTS each consists of 16 items that measure ideas of reference (GPTS^REF) and ideas of persecution (GPTS^PERS), respectively. In addition, participants completed the SAD scale, which measures state and trait SAD⁴; the brief fear of negative evaluation (BFNE) scale, which measures apprehension of being negatively evaluated by other people²; the Interpersonal Sensitivity Scale, which measures interpersonal sensitivity³; the Patient Health Questionnaire-8, which measures depression¹⁶; and the Generalized Anxiety Disorder 7, which measures generalized anxiety.¹⁷

Phase 2. VR study

Procedure

This was a cross-sectional comparison study with an experimental manipulation. Following Green et al.,¹⁵ we aimed to recruit a subsample of at least 70 participants from phase 1: 35 participants with high paranoid ideation and 35 participants with low paranoid ideation. After exposure to a VR social scenario, participants of high and low trait paranoid ideation were compared to establish levels of state paranoid ideation and components of social performance.

Participants

Two approximately equal subsamples of participants who demonstrated high (≥85th percentile) or low (≤15th percentile) trait paranoia (GPTS^TOTAL) were identified from phase 1 participants. A random 35 participants from each subsample were invited by e-mail to participate. If participants declined or did not respond within 2 weeks, reserve list participants were invited to replace initial invitees, until recruitment goals were achieved. Participants were paid £10.

A calculation of GPTS scores from phase 1 identified 96 eligible participants (mean GPTS^TOTAL = 86.35) from a high paranoia subsample (≥85th percentile), and 100 eligible participants (mean GPTS^TOTAL = 32.44) from a low paranoia subsample (≤15th percentile). Researchers and participants were blind to group status. Participant number lists for the high paranoia sample and low paranoia sample were each randomized by researchers SR and CE using Microsoft Excel. Two random samples of 35 participants from both the high paranoia sample and low paranoia sample, along with accompanying reserve lists, were identified to be invited for the VR task. The two sets of 35 participant numbers were then combined and randomized to make a list of 70 participants. Participant identity and contact details were only revealed to researchers conducting phase 2 data collection once the combined subsamples had been randomized.

All mean group differences were statistically significant with large effect sizes. During recruitment, all 96 high paranoia subsample participants and 74 of the 100 low paranoia subsample participants were invited for the VR task. An initial pilot phase with 12 participants (8 high paranoia and 4 low paranoia) led to minor modifications of the VR audio in which some negative background stimuli were reduced. Pilot data were not included in the analysis. Thirty-seven participants with high paranoia and 40 with low paranoia entered into the VR task. One high paranoia participant dropped out during the task. Figure 1 demonstrates how eligible participants were recruited from phase 1 and participated in phase 2.

FIG. 1.

Phase 2 (virtual reality study) recruitment process. GPTS, Green et al. Paranoid Thoughts Scale; VR, virtual reality.

VR environment and apparatus

Participants wore an Oculus Rift Developer version 2 head-mounted display, with noise canceling headphones, and moved in the VR with a combination of Xbox control pad and by physically turning their body direction.

The virtual social scenario was a party in a bar-room (Fig. 2), lasting ∼5 minutes, where computer-programmed virtual agents interacted with the participant. Participants were initially in a street and invited to look around the street, using the joypad to move themselves forward to a mark on the ground. This part of the task served as a demo for participants to familiarize themselves with the environment and controls. Participants were directed to a bar that they were to enter and then instructed to walk through the bar according to marks on the ground. The bar was populated by female and male avatars that appeared to be in their twenties or thirties, and represented various ethnicities. Background audio played throughout and included ambiguous stimuli with positive (“he/she's so nice!”), negative (“what a loser!”), or neutral interpretations (“what a joke!”).

FIG. 2.

Screenshot of virtual bar environment. The VR scenario was commissioned by King's College London, designed by software company Virtualware, using the Unity software platform, and ran on an Alienware PC.

Following previous research, participants were specifically given the following instruction: “While you are in the bar please try to get an impression of what the people in the bar think about you and what you think about them. If someone asks you a question, try to reply to them.”⁷ In the bar, participants engaged in one individual greeting with the host of the party who invited them to meet the other guests and had four brief group interactions that could be interpreted as positive, mildly negative, or neutral.

The initial greeting with the host and the final conversation both had an interactive component where participants were invited to speak out loud in response to questions they were asked by avatars in the VR. At the initial greeting, participants were invited to introduce themselves and then avatars in the pub turned toward them and greeted them in return. The second and third interactions could be interpreted as neutral or mildly negative insofar as avatars did not respond positively to participants. At the final conversation, which could be interpreted as positive, participants were invited to a table by a male avatar. Once at the table, a female avatar asked the participant what their favorite television program was and asked them to tell her about it. Participant reactions to each specific interaction were not measured.

Measures

Pre-VR, state paranoid ideation was measured with the State Paranoia Measure (SPM)¹⁸; stress, anxiety, sadness, and happiness were measured with visual analog scales (VAS) on a 10-point scale; and heart rate was measured using a finger pulse oximeter.

Post-VR, SPM, heart rate, and baseline VAS were repeated; immersion, or sense of presence (the impression of “being there” in the virtual environment), was measured with a VAS and the Slater-Usoh-Steed Sense of Presence Questionnaire (SUS) modified to our VR scenario¹⁹; state social paranoid ideation about the virtual environment and avatars was measured with the State Social Paranoia Scale (SSPS) subscales, which examine persecutory (SSPS^PERS), neutral (SSPS^NEU), and positive ideation (SSPS^POS) about avatars²⁰; additional VAS on a 10-point scale measured situation-specific paranoid ideation, friendliness of other people, neutrality of other people, hostility of other people, anxiety in the social situation, desire to avoid social interaction (these two items measured SAD and were modified from two SAD items), fear that other people would disapprove, worries of saying or doing the wrong thing (these two items measured FNE and were modified from two BFNE items), how positively or negatively other people were thinking, and enjoyment.

Previous VR use and computer gaming regularity were recorded as potential confounders: participants were asked if they had used VR before and if they played computer games regularly. A “yes” or “no” answer was recorded.

Analyses

All analyses were conducted using SPSS v22 (Chicago, IL). Internal reliability of scales was calculated using Cronbach's alpha. As GPTS data were not normally distributed, Spearman's correlation coefficient was used for all correlation analyses. Demographic differences between VR groups were calculated using χ² tests for categorical data or independent samples t tests for continuous data. Independent samples t tests were used to test mean differences between groups. Paired samples t tests were used to test mean differences within the whole group between two time points. Effect sizes (Cohen's d) for independent samples t tests were calculated using t values and degrees of freedom, and for paired samples, t tests were calculated using means and standard deviations. Effect sizes were measured at thresholds of 0.1 (small), 0.3 (medium), and 0.5 (large).²¹

Results

Phase 1. Survey study

Table 1 reports demographic characteristics and mean (SD, range) of measures. All scales and subscales demonstrated adequate internal consistency (α's < 0.88). Table 2 reports associations between paranoid ideation and social performance. Table 3 reports that paranoid ideation is related to education level and relationship status, although not to age (comparing ≤35 and >35 years of age), gender, or ethnicity (comparing white with black and other minorities).

Table 1.

Phase 1 (Survey Study) Demographic Characteristics and Summary of Scores

	N = 609, M (SD, range)
Age (years)
	29.33 (9.24, 18–65)
Gender, n (%)
Male	166 (27.3)
Female	439 (72.1)
Other	4 (0.7)
Ethnicity, n (%)
Asian/Asian British	67 (11)
Black African/Caribbean/British	19 (3.1)
White	463 (76)
Mixed/multiple ethnic groups	30 (4.9)
Other	30 (4.9)
Education, n (%)
No formal qualifications	1 (0.2)
O-Levels, GC(S)Es	15 (2.5)
AS-, A-levels, (G)NVQ	124 (20.4)
Undergraduate degree	219 (36)
Master's degree	181 (29.7)
Doctorate	69 (11.3)
Employment, n (%)
Student	226 (37.1)
Full-time paid	298 (48.9)
Part-time paid	47 (7.7)
Full-time unpaid	3 (0.5)
Part-time unpaid	5 (0.8)
Unemployed	18 (3)
Other	12 (2)
Relationship/marital status, n (%)
Single	225 (41)
In relationship, not cohabiting	112 (18.4)
Cohabiting	127 (20.9)
Married	95 (15.6)
Separated	3 (0.5)
Divorced	8 (1.3)
Widowed	2 (0.3)
Other	7 (1.1)
Living arrangements, n (%)
Live with parents	65 (10.7)
House/flat owner	156 (25.6)
Renting house/flat	223 (36.6)
Renting bedsit	9 (1.5)
Renting room in house share	134 (22)
Staying in hostel	12 (2)
Homeless	1 (0.2)
Other	9 (1.5)
GPTS^REF	27.60 (11.724, 16–78)
GPTS^PERS	20.62 (9.768, 16–75)
GPTS^TOTAL	48.22 (19.84, 32–152)
SAD	9.05 (6.90, 0–28)
BFNE	38.92 (10.527, 12–60)
ISPM	95.13 (14.84, 53–132)
PHQ8	5.66 (5.04, 0–24)
GAD7	5.05 (4.79, 0–21)

BFNE, brief fear of negative evaluation; GAD7, Generalized Anxiety Disorder 7; GPTS^PERS, ideas of persecution; GPTS^REF, ideas of reference; GPTS^TOTAL, Green et al. Paranoid Thoughts Scale; IPSM, Interpersonal Sensitivity Scale; PHQ8, Patient Health Questionnaire-8; SAD, social avoidance and distress.

Table 2.

Phase 1 (Survey Study) Associations Between Paranoid Ideation and Components of Social Performance

	GPTS^REF	GPTS^PERS	GPTS^TOTAL	SAD	BFNE	ISPM	PHQ8	GAD7
GPTS^REF	—	—	—	—	—	—	—	—
GPTS^PERS	0.686^*	—	—	—	—	—	—	—
GPTS^TOTAL	0.981^*	0.786^*	—	—	—	—	—	—
SAD	0.335^*	0.219^*	0.325^*	—	—	—	—	—
BFNE	0.352^*	0.158^*	0.328^*	0.457^*	—	—	—	—
ISPM	0.424^*	0.257^*	0.417^*	0.459^*	0.751^*	—	—	—
PHQ8	0.512^*	0.369^*	0.512^*	0.488^*	0.443^*	0.526^*	—	—
GAD7	0.499^*	0.380^*	0.496^*	0.485^*	0.510^*	0.572^*	0.786^*	—

Correlation r_s is significant at the 0.01 level (one tailed).

Table 3.

Phase 1 (Survey Study) Paranoid Ideation and Demographic Characteristics

	N	GPTS^TOTAL	Test	p	Effect
	N	M (SD, range)	Test	p	Effect
Age
≤35	503	48.18 (18.74, 32–152)	t(607) = 0.115	0.91	—
>35	106	48.42 (24.49, 32–133)
Gender
Male	166	50.42 (21.61, 32–133)	t(603) = 1.642	0.10	—
Female	439	47.45 (19.15, 32–152)
Ethnicity
White	463	47.42 (19.759, 32–152)	t(607) = 1.788	0.07	—
BAME	146	50.78 (19.933, 32–128)
Education level
Nongraduate	140	54.42 (22.327, 32–129)	t(607) = 4.273	<0.0001	0.391
Graduate	469	46.37 (18.661, 32–152)
Relationship status
In relationship	334	46.56 (19.531, 32–133)	t(607) = 2.286	<0.05	0.186
Not in relationship	275	50.24 (20.051, 32–152)

BAME, Black, Asian, and Minority Ethnic.

Phase 2. VR study

Thirty-seven participants with high paranoia and 40 with low paranoia were entered into VR, as reported above. Trait paranoia scores for VR participants were significantly different between high (mean GPTS^TOTAL = 81.08) and low (mean GPTS^TOTAL = 32.53) paranoia groups. All scales and subscales demonstrated adequate internal consistency for the total sample (α's > 0.7), apart from SPM, which demonstrated low internal consistency for the low paranoia group pre-VR (α = 0.191) and post-VR (α = 0.417). Table 4 reports that group differences for gender, ethnicity, and employment status, way of finding out about the study (online/offline), and previous VR and gaming experience were not statistically significant; whereas group differences for age, education, and relationship status were statistically significant.

Table 4.

Phase 2 (Virtual Reality Study) Baseline Paranoid Ideation, Demographic Characteristics, and Postvirtual Reality Task Outcomes

	Total sample, N = 76	High paranoia group, n = 36	Low paranoia group, n = 40	Test	p	Effect
	M (SD)	M (SD)	M (SD)	Test	p	Effect
GPTS^TOTAL, M (SD, range)	—	81.08 (18.433, 64–152)	32.53 (0.506, 32–33)	t(74) = 16.666	<0.001	0.89
Age (years), M (SD, range)	—	28.86 (9.84, 18–54)	33.78 (11.04, 24–65)	t(74) = −2.039	<0.05	—
Gender, n (%)
Male	—	13 (36.1)	14 (35)	χ²(1) = 0.0102	0.9195	—
Female	—	23 (63.9)	26 (65)
Ethnicity, n (%)
White	—	29 (80.6)	36 (90)	χ²(1) = 1.3653	0.243	—
Black and minority ethnic	—	7 (19.4)	4 (10)
Education level, n (%)
Secondary/higher education	—	10 (27.8)	3 (7.5)	χ²(1) = 14.084	<0.0001	—
Undergraduate degree	—	16 (44.4)	9 (22.5)
Postgraduate degree(s)	—	10 (27.8)	28 (70)
Employment status, n (%)
Student	—	14 (38.9)	9 (22.5)	χ²(1) = 5.3073	0.070	—
In paid employment	—	17 (47.2)	29 (72.5)
Unpaid/unemployed	—	5 (13.9)	2 (5)
Relationship status, n (%)
In a relationship	—	14 (38.9)	27 (67.5)	χ²(1) = 6.243	<0.05	—
Not in a relationship	—	22 (61.1)	13 (32.5)
SUS1	4.53 (1.519)	4.61 (1.536)	4.45 (1.518)	—		—
SUS2	3.93 (1.636)	4.03 (1.748)	3.85 (1.545)	—		—
SUS3	4.20 (1.862)	4.19 (1.704)	4.20 (2.015)	—		—
SUS4	4.34 (1.694)	4.33 (1.690)	4.35 (1.718)	—		—
SUS5	4.45 (1.587)	4.42 (1.610)	4.48 (1.585)	—		—
SUS6	3.76 (1.607)	3.81 (1.653)	3.73 (1.585)	—		—
SUS^TOTAL	25.21 (8.021)	25.39 (7.980)	25.05 (8.155)	t(74) = 0.183	0.856	—
SPM	—	10.39 (6.33)	6.43 (1.43)	t(74) = 3.855	<0.001	0.41
SSPS^PERS	—	21.19 (8.998)	14.35 (5.137)	t(74) = 4.124	<0.001	0.43
SSPS^POS	—	12.67 (2.859)	15.35 (3.80)	t(74) = −3.448	<0.001	0.37
SSPS^NEU	—	10.92 (3.842)	11.40 (3.727)	t(74) = −0.556	0.58	—
VAS
Presence	6.09 (2.39)	6.17 (2.274)	6.03 (2.516)	t(74) = 0.256	0.798	—
Enjoyment	6.91 (2.246)	6.64 (2.016)	7.15 (2.434)	t(74) = −0.99	0.325	—
Paranoia	—	4.78 (2.474)	2.7 (1.757)	t(74) = 4.253	<0.001	0.44
Friendliness of people	—	3.78 (1.681)	3.98 (1.527)	t(74) = −0.543	0.589	0.06
Neutrality of people	—	4.36 (1.759)	4.68 (1.913)	t(74) = −0.742	0.461	0.09
Hostility of people	—	4.58 (2.02)	4.18 (2.438)	t(74) = 0.790	0.432	0.09
Socially anxious (SAD)	—	5.86 (2.416)	4.75 (2.509)	t(74) = 1.962	0.054	—
Social avoidance (SAD)	—	5.64 (2.84)	4.08 (2.702)	t(74) = 2.459	<0.05	0.27
Others do not approve (FNE)	—	4.94 (2.529)	2.98 (2.27)	t(74) = 3.578	<0.001	0.38
Say the wrong thing (FNE)	—	5.72 (2.7)	4.05 (2.511)	t(74) = 2.797	<0.01	0.31
Others thought of me positively	—	4.22 (1.758)	4.90 (1.878)	t(74) = −1.619	0.110	—

All p values two tailed.

FNE, fear of negative evaluation; SPM, State Paranoia Measure; SSPS, State Social Paranoia Scale (persecution, neutral, and positive ideation); SUS, Slater-Usoh-Steed Sense of Presence Questionnaire; VAS, visual analog scale.

Independent-samples t tests compared mean sense of presence and enjoyment scores between groups. Table 4 reports that sense of presence and enjoyment were at acceptable levels; scores were comparable with the original SUS study.²² Group differences were not statistically significant. Paired-samples t tests compared pre-VR and post-VR mean VAS scores and heart rate for the whole sample. Table 5 reports that the VR task elicited a statistically significant change in stress (medium effect), anxiety (large effect), sadness (small effect), and heart rate (small effect) for the whole sample.

Table 5.

Phase 2 (Virtual Reality Study) Pre- and Postvirtual Reality Mood and Heart Rate

	Pre-VR	Post-VR	Test	p	Effect
	M (SD)	Post-VR	Test	p	Effect
Stress VAS	2.57 (1.644)	3.30 (1.987)	t(75) = −3.499	<0.001	0.40
Anxiety VAS	2.42 (1.56)	3.43 (2.15)	t(75) = −4.809	<0.001	0.52
Sadness VAS	1.79 (1.236)	2.01 (1.669)	t(75) = −2.06	<0.05	0.15
Happiness VAS	5.82 (2.108)	5.45 (2.241)	t(75) = −1.746	0.085	—
Heart rate	83.55 (15.431)	86.76 (16.55)	t(75) = −2.131	<0.05	0.20

All p values two tailed.

VR, virtual reality.

An independent-samples t test compared mean pre-VR SPM scores between groups to evaluate validity of groupings. There was a significant mean difference between high paranoia group (10.47, SD 6.25) and low paranoia group in their pre-VR state paranoia (6.15, SD 0.483); t(74) = 4.363, p < 0.01. Independent-samples t tests compared mean post-VR state paranoia scores between high and low paranoia groups. Table 4 reports that, post-VR, the high paranoia group was significantly higher in state paranoia (SPM), and state paranoia specifically about the virtual social scenario and avatars (SSPS) when compared with the low paranoia group (medium effects).

Independent-samples tests were conducted to compare post-VR FNE and SAD mean scores between groups. Table 4 reports that the high paranoia group was significantly more socially avoidant, more concerned about others' disapproval, and more concerned with saying something wrong (medium effects). Group differences in appraisals of friendliness, neutrality, and hostility of avatars, and of social anxiety were not significant.

Discussion

Trait paranoia and its associations with social performance

In line with our first hypothesis, trait paranoia was associated with higher levels of self-reported fear, negative evaluation, interpersonal sensitivity, SAD, mood, and with demographic characteristics.

Our survey findings of correlations between trait paranoia and components of social performance were consistent with previous reports in clinical and nonclinical populations. However, our analysis of demographic characteristics does not replicate results, in showing higher paranoia in males,²³ nor shows higher paranoia in the younger age group (≤35), which research demonstrates is a group typically more “at-risk” of developing serious mental health conditions, such as psychosis.²⁴

Survey participants ranged across the working age life span but were not totally representative of it in that they were predominantly younger adults, female, and of white ethnicity; half were in full-time employment and just over two-thirds were students, while just over half were in a relationship. The survey sample incurs similar limitations highlighted in previous general population studies of paranoid ideation, insofar as directions of effect cannot be substantiated and associations could be a consequence of unmeasured variables.²⁵ Our recruitment used convenience sampling, disproportionately targeting southeast London and those affiliated with the university. The younger age and high prevalence of females are noteworthy and the sample may be disproportionally interested in VR when compared with the general population.

Trait paranoia (mean GPTS) was markedly similar to Green et al.'s¹⁵ nonclinical sample, which used university students and employees recruited by e-mail rather than the predominantly online sample used in this study. One can tentatively conclude that paranoid ideation in our sample is likely to be broadly representative of the general population, noting we largely used online recruitment and not the predominately student sample used by previous studies.^15,26 However, mean GPTS^PERS and GPTS^TOTAL in our high paranoia sample were markedly lower than scores for the same scale and subscale in Green et al.¹⁵ Our high paranoia sample was comparable with a clinical sample in ideas of reference but less comparable in persecutory beliefs. Nevertheless, given the very wide range of scores in our high paranoia group, we note that at the higher end there is overlap with clinical scores, suggesting that it is a partial comparator to a clinical sample.

Acceptability and validity of new VR environment to assess real-time paranoid ideation

Our participants found the newly designed VR environment acceptable and immersive. No adverse effects were reported. A small minority of participants reported mild, brief cybersickness, a well-known phenomenon found in previous studies.²⁷ Overall, while our VR task elicited mild stress and anxiety, albeit with low baselines, effects on sadness were small, happiness was unaffected, and enjoyment was at a moderate-to-high level, therefore supporting the acceptability and feasibility of using this environment for assessment and treatment.

As hypothesized, higher trait paranoia was associated with higher state paranoia and greater negative components of social performance in the VR social scenario. The post-VR group differences in SSPS scores support our conclusion that the high paranoia group experienced greater state paranoia about the VR environment than the low paranoia group. These results were not confounded by greater familiarity with VR or computer games in either group.

The VR environment was therefore shown to be valid for measuring paranoid ideation and social performance by replicating phase 1 associations of high trait paranoia with both high state paranoia and greater negative appraisals of social performance. This result builds on previous research that demonstrates measurement of the relationship between paranoia and environmental stress in VR environments,^28,29 but, in this study, validates a new immersive, interactive, multiavatar bar-room environment and targets specific components of social performance known to be associated with paranoia.

Strengths and limitations

A strength of the study is that it explores the link between paranoid ideation and social performance in an ecologically valid VR environment, which has the potential to be manipulated experimentally and used therapeutically in personalized treatments.¹¹ The VR study controlled for potential confounders, such as group differences in gender, ethnicity, employment status, social anxiety, way of finding out about the study, and previous VR or gaming experience. Statistically significant group differences in age, education, and relationship status may have contributed to effects on social performance and paranoid ideation.

Methodological limitations include lack of power calculation to determine sample size. A post hoc power analysis revealed that with the recruited sample size of 37 and 40, respectively, we had 80 percent power to detect differences between two groups with moderate to large effect sizes (d = 0.65) at alpha = 0.05. Nonsignificant results are therefore difficult to interpret in that we might have lacked power to detect effects that were present.

Technological limitations precluded the opportunity to explore systematically, and in real-time, the impact of environmental and behavioral characteristics on paranoid ideation, for example, it was not possible to individually evaluate each interaction or participant behavior, engagement, or eye gaze within the VR task. We were limited to pre-/postmeasurement.

Conclusions

This study supports the safe and effective use of a new VR social scenario for the assessment of people who experience high paranoid ideation in social situations, and validates the virtual environment for use with existing psychometric tools.

Footnotes

Acknowledgments

The study proposal was reviewed by service user researchers at the Service User Advisory Group, Biomedical Research Centre, IoPPN. The study was approved by the Psychiatry, Nursing and Midwifery Research Ethics Subcommittee, King's College London (HR-14/15-0859). The authors acknowledge financial support from the National Institute for Health Research (NIHR) Biomedical Research Centre for Mental Health at South London and Maudsley NHS Foundation Trust and King's College London (PI: Valmaggia, Garety). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.

Author Disclosure Statement

The authors have declared that there are no conflicts of interest in relation to the subject of this study.

References

Freeman

, Thompson

, Vorontsova

, et al. Paranoia and post-traumatic stress disorder in the months after a physical assault: a longitudinal study examining shared and differential predictors. Psychological Medicine, 2013; 43:2673–2684.

Leary

. A brief version of the fear of negative evaluation scale. Personality and Social Psychology Bulletin, 1983; 9:371–375.

Boyce

, Parker

. Development of a scale to measure interpersonal sensitivity. Australian and New Zealand Journal of Psychiatry, 1989; 23:341–351.

Watson

, Friend

. Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 1969; 33:448.

Kesting

M-L

, Bredenpohl

, Klenke

, et al. The impact of social stress on self-esteem and paranoid ideation. Journal of Behavior Therapy and Experimental Psychiatry, 2013; 44:122–128.

Taylor

, Stopa

. The fear of others: a pilot study of social anxiety processes in paranoia. Behavioural and Cognitive Psychotherapy, 2013; 41:66–88.

Valmaggia

, Day

, Garety

, et al. Social defeat predicts paranoid appraisals in people at high risk for psychosis. Schizophrenia Research, 2015; 168:16–22.

Freeman

. Suspicious minds: the psychology of persecutory delusions. Clinical Psychology Review, 2007; 27:425–457.

Parsons

. (2011) Neuropsychological assessment using virtual environments: enhanced assessment technology for improved ecological validity. In: Brahnam

, Jain

, eds. Advanced computational intelligence paradigms in healthcare 6. Virtual reality in psychotherapy, rehabilitation, and assessment. Berlin Heidelberg: Springer, pp. 271–289.

10.

Rus-Calafell

, Garety

, Sason

, et al. Virtual reality in the assessment and treatment of psychosis: a systematic review of its utility, acceptability and effectiveness. Psychological Medicine, 2017; 48:362–391.

11.

Valmaggia

, Latif

, Kempton

, et al. Virtual reality in the psychological treatment for mental health problems: an systematic review of recent evidence. Psychiatry Research, 2016; 236:189–195.

12.

Freeman

, Reeve

, Robinson

, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 2017; 47:2393–2400.

13.

Smith

, Fleming

, Wright

, et al. Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophrenia Research, 2015; 166:86–91.

14.

Rus-Calafell

, Gutierrez-Maldonado

, Ribas-Sabate

. A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. Journal of Behavior Therapy and Experimental Psychiatry, 2014; 45:81–89.

15.

Green

, Freeman

, Kuipers

, et al. Measuring ideas of persecution and social reference: the Green et al. Paranoid Thought Scales (GPTS). Psychological Medicine, 2008; 38:101–111.

16.

Kroenke

, Spitzer

. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals, 2002; 32:509–515.

17.

Spitzer

, Kroenke

, Williams

, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of Internal Medicine, 2006; 166:1092–1097.

18.

Garety

, Waller

, Emsley

, et al. Freeman, cognitive mechanisms of change in delusions: an experimental investigation targeting reasoning to effect change in paranoia. Schizophrenia Bulletin, 2015; 41:400–410.

19.

Slater

, Usoh

, Steed

. Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 1994; 3:130–144.

20.

Freeman

, Pugh

, Green

, et al. A measure of state persecutory ideation for experimental studies. The Journal of Nervous and Mental Disease, 2007; 195:781–784.

21.

Field

. (2009) Discovering statistics using SPSS. London, UK: Sage Publications.

22.

Usoh

, Catena

, Arman

, et al. Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments, 2000; 9:497–503.

23.

Johns

, Cannon

, Singleton

, et al. Prevalence and correlates of self-reported psychotic symptoms in the British population. The British Journal of Psychiatry, 2004; 185:298–305.

24.

Kessler

, Amminger

, Aguilar-Gaxiola

, et al. Ustun, age of onset of mental disorders: a review of recent literature. Current Opinion in Psychiatry, 2007; 20:359.

25.

Freeman

, McManus

, Brugha

, et al. Concomitants of paranoia in the general population. Psychological Medicine, 2011; 41:923–936.

26.

Freeman

, Garety

, Bebbington

, et al. Psychological investigation of the structure of paranoia in a non-clinical population. The British Journal of Psychiatry, 2005; 186:427–435.

27.

Rebenitsch

, Owen

. Review on cybersickness in applications and visual displays. Virtual Reality, 2016; 20:101–125.

28.

Veling

, Pot-Kolder

, Counotte

, et al. Environmental social stress, paranoia and psychosis liability: a virtual reality study. Schizophrenia Bulletin, 2016; 42:1363–1371.

29.

Freeman

, Pugh

, Antley

, et al. Virtual reality study of paranoid thinking in the general population. The British Journal of Psychiatry, 2008; 192:258–263.