Being There in the Midst of the Story: How Immersive Journalism Affects Our Perceptions and Cognitions

Abstract

Immersive journalism in the form of virtual reality (VR) headsets and 360°-video is becoming more mainstream and is much touted for inducing greater “presence” than traditional text. But, does this presence influence psychological outcomes of reading news, such as memory for story content, perceptions of credibility, and empathy felt toward story characters? We propose that two key technological affordances of VR (modality and interactivity) are responsible for triggering three presence-related cognitive heuristics (being-there, interaction, and realism), which influence news readers' memory and their perceptions of credibility, empathy, and story-sharing intentions. We report a 3 (storytelling medium: VR vs. 360°-video vs. Text) × 2 (story: “The displaced” and “The click effect”) mixed-factorial experiment, in which participants (N = 129) experienced two New York Times stories (that differed in their emotional intensity) using one of three mediums (VR, 360°-video, Text). Participants who experienced the stories using VR and 360°-video outperformed those who read the same stories using text with pictures, not only on such presence-related outcomes as being-there, interaction, and realism, but also on perceived source credibility, story-sharing intention, and feelings of empathy. Moreover, we found that senses of being-there, interaction, and realism mediated the relationship between storytelling medium and reader perceptions of credibility, story recall, and story-sharing intention. These findings have theoretical implications for the psychology of virtual reality, and practical applications for immersive journalism in particular and interactive media in general.

Introduction

Immersive journalism provides consumers a first-person experience of events, locations, and stories.¹ Recent advancements in omnidirectional (or 360°) video² have made it possible to view news stories using either a mobile headset with smartphone as a virtual reality (VR) display or an interactive (computer or mobile) screen with keyboard/mouse/touchpad. The VR headset and 360°-video have received much attention because they can induce presence (“the perceptual illusion of nonmediation”),³ situating readers within stories. But, does presence translate to other psychological outcomes of reading news, such as memory for story content, perceptions of credibility, and empathy felt toward story characters? To date, only a few studies have examined the effect of immersive videos on user engagement, but in the advertising context.⁴ It remains unclear whether these newer methods differ from the traditional textual medium in influencing user engagement with news, and if so, why.

In this study, we investigate the difference between experiencing 360°-video using a VR headset vs. on a screen, and compare both to reading the same news story on a traditional web-based platform. Using the Modality-Interactivity-Agency-Navigability (MAIN) model, we propose that the three mediums, categorized by two technological affordances, modality and interactivity, elicit divergent outcomes from readers by triggering three presence-related cognitive heuristics—being-there, interaction, and realism—as explained below.

Modality, interactivity and presence

While modality refers to the means by which information is conveyed,⁵ interactivity is the degree to which individuals can act and control the mediated presentation.⁶ The former determines the vividness or “representational richness” of the mediated environment and the latter dictates the extent to which users can modify that environment in real-time.⁶ Studies have shown that the differential provision of modality and interactivity affordances can influence user perception of news. Individuals perceive the news to be of higher quality and recall the story details better when it is presented in text only and text-plus-picture modalities compared to multimedia presentations.⁷ Similarly, news featured on highly interactive Web sites is considered more credible⁸ and involving.⁹

What mechanisms explain the effects of technological affordances on user perceptions of news?

The MAIN model¹⁰ proposes that affordances serve as cues that trigger cognitive heuristics, or mental shortcuts, about the quality and credibility of media content. Three presence-related heuristics relevant to immersive news are being-there, interaction, and realism.^5,10 Being-there heuristic —“I am part of the action, therefore I am present”^5,10—is triggered when a user is drawn into the mediated environment—what presence researchers call “self-location” in an external, physical space.¹¹ Once it is triggered, the user takes into account the “authenticity and intensity of the experience”¹⁰ (p. 81) for making evaluations. The interaction heuristic (the more interaction, the better) is triggered when a given medium enables user action with content¹⁰—intuitive action when immersed is an inherent component of presence.¹² Lastly, realism is based on the rule that “seeing is believing,” that pictures cannot lie–“it is so real that I am present.”^5,10 This heuristic is said to be triggered when the mediated presentation closely approximates physical reality and thereby provides a compelling experience.¹³ The positive effects of presence^14–16 may be explained by the operation of these heuristics, whereby the authenticity, interactivity, and realism of the portrayal factor into evaluations of content.

The three storytelling mediums were selected for study based on their relative levels of modality and interactivity, with VR being richer in both, followed by 360°-video, then text,^5,6 thereby making it more likely to trigger presence-related heuristics. VR as a multimedia modality presented through a head-mounted display is more likely to draw individuals into the mediated environment, allowing individuals to feel as if they are part of the action. Moreover, when using VR, all reminders of the physical world (such as light) are blocked off, enabling full immersion in the mediated environment.¹⁷ As for interactivity, VR enables users to navigate the story environment by moving their heads in different directions, as opposed to mouse and keyboard-based interactions, thereby providing a more natural and intuitive interaction. Lastly, in VR and 360°-video, individuals are watching actual video footage, in conjunction with story characters' voices. Thus, these two mediums are more likely to enhance sense of realism, compared to text. Furthermore, the “telepresence”⁶ (or transportation of the user to the mediated environment) felt in VR is likely to enhance the sense of realism. Therefore, we propose:

H1. VR will be highest, followed by 360°-video, then text, in eliciting a sense of being-there (H1a), sense of interaction (H1b), and sense of realism (H1c).

As the MAIN model posits, heightening the senses of being-there, interaction, and realism can trigger positive heuristics, which carry over to readers' judgments of content credibility. Therefore, we propose the following:

H2. Senses of being-there, interaction, and realism will mediate the relationship between the storytelling medium and source credibility.

We examine the effects of storytelling mediums on other aspects of reader engagement as well, namely story recall, empathy, and story-sharing intention. A wealth of studies has demonstrated a positive relationship between presence and story recall.^16,18 We hypothesize that the three presence-related heuristics will affect memory in a similar way as credibility. The sense of being in the story environment and interacting among seemingly real entities is likely to enhance memory for story details in VR condition, compared to 360°-video and text.

H3. Senses of being-there, interaction, and realism will mediate the relationship between the storytelling medium and story recall.

Aside from enhancing credibility and memory, VR is known for being an “empathy machine.”¹⁹ Journalistic excitement over VR storytelling was in part premised on the belief that it can “give viewers a unique sense of empathic connection to people and events.”²⁰ VR has been a tool to teach empathy (e.g., autistic children).²¹ Moreover, studies have shown that VR can increase empathy toward characters presented in a virtual environment.^22,23 Therefore, we hypothesize that the heightened senses of being-there, interaction, and realism associated with VR will translate to greater empathy for story characters and relatedly, to greater tendency to share the story with others.

H4. Senses of being-there, interaction, and realism will mediate the relationship between the storytelling medium and empathy (H4a) and story-sharing intention (H4b).

Methods

We conducted a mixed-factorial experiment with modality as a between-subjects factor and story as a within-subjects factor. Participants were randomly assigned to one of three storytelling mediums (VR vs. 360°-video vs. Text)^a to read two stories from the New York Times, “The displaced”^24–26 and “The click effect.”²⁷ Story order was counterbalanced. Participants in VR condition experienced the stories using a smartphone and a Cardboard VR headset. VR stories were accessed from the New York Times VR mobile app and YouTube channel “The Daily 360°.”^b In 360°-video condition, participants experienced the stories on traditional desktop computers. Participants in Text condition read both stories from the New York Times' online news sites (www.nytimes.com) using the same computers.

Stimuli

We used “The displaced” and “The click effect” as our story stimuli for two reasons. First, both stories were published in all three storytelling mediums, allowing us to keep story content approximately constant across conditions. Second, they differ on emotional intensity, allowing us to examine whether story's emotional factors moderate the hypothesized relationships, as argued by presence researchers.^13,28 Emotional intensity is related to emotional valence, with negative emotions being more dominant.^29,30 “The displaced” immersed readers in the lives of three child refugees–Hana,²⁴ Oleg,²⁵ and Chuol.²⁶ Learning about the hardships faced by these children is likely to elicit negative responses (e.g., sadness, heartache). In contrast, “The click effect” evokes more benign responses (e.g., cheer, surprise), as readers learn about the diving method used by marine researchers studying the vocalizations of dolphins and whales. In addition to negative valence, “The displaced” is associated with greater emotional intensity given the relative seriousness of the issue. Modifications were made to the Text condition, including removal of sidebars, to reduce content variations across the three conditions.

Procedure

Participants (N = 129) were recruited from a large U.S. university (M_age = 20.44, SD = 1.08; Female = 100). They first completed individual-difference measures on a computer. Afterward, they were asked to read two stories. Participants in the VR and 360°-video conditions were provided with headphones, given a brief tutorial and instructed to explore the story environment as they liked. After reading/watching each story, participants completed a postquestionnaire.

Measures

Individual difference variables

Dispositional empathy was measured with perspective-taking and empathic concern subscales of Interpersonal Reactivity Index.³¹ Existing source credibility for the New York Times was measured using trustworthy and expertise scales.³² See Table 1 for measures and their reliabilities.

Table 1.

A List of Individual Items Comprising the Measured Variables and Their Reliability

Measure	Items
Prequestionnaire
Dispositional empathy (Interpersonal Reactivity Index) (Cronbach's α = 0.84)	2 Sample Items: I often have tender, concerned feelings for people less fortunate than me; I sometimes find it difficult to see things from the “other guy's” point of view
Trustworthy (Prior source credibility) (Cronbach's α = 0.91)	Not dependable—Dependable; Dishonest–Honest; Unreliable–Reliable; Insincere–Sincere; Untrustworthy–Trustworthy; Not credible—Credible
Expertise (Prior source credibility) (Cronbach's α = 0.91)	Not expert–Expert; Inexperienced–Experienced; Not Knowledgeable–Knowledgeable; Unqualified–Qualified; Not skilled—Skilled
Postquestionnaire
Emotional valence (Cronbach's α = 0.96)	Unhappy–Happy; Sad–Elated; Bitter–Cheerful; Gloomy–Lively; Melancholy–Positive; Down–Upbeat; Depressed–Joyful
Emotional intensity (Cronbach's α = 0.69)	I was bored by the story; The story seemed superficial; I was moved by the story at a deeper level
Empathy Adjective Scale (Cronbach's α = 0.91)	Sympathetic; Moved; Compassionate; Warm; Softhearted; Tender
Story-sharing intention (Cronbach's α = 0.79)	I am going to tell my friends and family about the story; I will post the story on my social media to let people know about the story; I will let other students on campus know about the story
Trustworthy (Source credibility) (Cronbach's α = 0.93)	Not dependable—Dependable; Dishonest–Honest; Unreliable–Reliable; Insincere–Sincere; Untrustworthy–Trustworthy; Not credible—Credible
Expertise (Source credibility) (Cronbach's α = 0.93)	Not expert–Expert; Inexperienced–Experienced; Not Knowledgeable–Knowledgeable; Unqualified–Qualified; Not skilled—Skilled

Presence

Standard measures from the presence literature^33–38 were used to capture the senses of being-there, interaction, and realism (Table 2).

Table 2.

List of Presence Items Used for Measuring Senses of Being-There, Interaction, and Realism

Variables	Items
Being-there (Cronbach's α = 0.98)	I felt like I was actually there in the story environment; It was as though my true location had shifted into the story environment; I felt as though I was physically present in the story environment; It seemed as though I actually took part in the action of the story; I felt like I was immersed in places that I couldn't physically visit; I had a sense of being present in the story environment; I thought I experienced the story event in person without actually being there; I had a sense of “being there” in the story environment; During the story, I often thought I was really standing in the story environment; When the story ended, I felt like I came back to the “real world” after a journey; During the story, I felt I was in the world that was presented in the story; During the story, I often forgot I was in the middle of an experiment; During the story, my body was in the room, but my mind was inside the world that was presented in the story
Interaction (Cronbach's α = 0.91)	I had the impression that I could be active in the story environment; The objects in the story gave me the feeling that I could do things with them; I had a sense of being together with the characters in the story; There were times during which I felt like I was directly interacting with characters in the story; I felt the characters in the story were aware of my presence
Realism (Cronbach's α = 0.85)	I thought the objects I read about in the story seemed natural; Objects, situations and people that were presented in the story were realistic; Events in the story were portrayed vividly; I know the story was real and not made up

Note: Items derived from scales for Self-location,³⁰ Being-there,²⁷ and Telepresence³²(for Being-there); Possible action,³⁰ Sense of togetherness,²⁸ and Social presence²⁹ (for Interaction); Realism²⁷ (for Realism).

Outcome variables

Empathy was assessed using an empathy adjective scale by Batson.³⁹ Emotional valence and intensity were measured via two scales,⁴⁰ after reading each story. Source credibility for the New York Times was measured using the same trustworthy and expertise scales as in the prequestionnaire³² (Table 1). Story recall was assessed via five cued-recall and five free-recall items pertaining to content that overlapped across the three storytelling mediums. The total number of correctly answered questions was indexed and used for analyses.

See Table 3 for measures that were used as covariates.

Table 3.

Questionnaire Items Used for Covariate Measures

Variable	Items
Suspension of disbelief (Cronbach's α = 0.59)	I concentrated on whether there were any inconsistencies in the story; I didn't really pay attention to the existence of errors or inconsistencies in the story; I took a critical viewpoint of the story; It was not important for me whether the story contained errors or contradictions
Attention allocation (Cronbach's α = 0.94)	I devoted my whole attention to the story; I concentrated on the story; The story captured my senses; I dedicated myself completely to the story
Enjoyment (Cronbach's α = 0.94)	I enjoyed the story content; I liked the story content; The story content was captivating

Note: Items obtained from scales for Suspension of disbelief³⁰ and Attention Allocation.³⁰

Results

First, we found significant differences in the emotional intensity of the two stories, with the “The displaced” (M = 5.90, SE = 0.09) scoring significantly higher than “The click effect” (M = 4.90, SE = 0.09), F (1, 125) = 92.80, p < 0.001. As expected, it scored on the negative end of the valence scale (M = 2.89, SE = 0.09) compared to “The click effect” (M = 4.88, SE = 0.09), F (1, 125) = 283, p < 0.001. We found a significant story-order effect on some of the variables, so it was entered as a covariate for all analyses.

Presence: being-there, interaction, and realism

Table 4 shows the main and interaction effects. As hypothesized in H1a, H1b, and H1c, we found a linear pattern of main effects of storytelling medium on senses of being-there, interaction, and realism. For being-there, participants in the VR condition scored higher than those in the 360°-video condition, who in turn scored higher than those in the Text condition. Interestingly, participants in the VR and 360°-video conditions did not differ from each other significantly on interaction and realism, while both were higher than those in Text. There was a significant interaction of medium with story for all three outcomes. For “The displaced,” participants in the VR and 360°-video conditions scored similarly on all three presence measures, with both of them significantly higher than those in the Text condition, but for “The click effect,” there was a significant difference between VR and 360°-video. Participants in the VR condition felt higher senses of being-there and interaction than those in 360°-video. On sense of realism, participants in the VR condition rated higher than those in the Text, with those in 360°-video in the middle.

Table 4.

The Effects of Storytelling Medium and Story on Senses of Being-There, Interaction, and Realism

		Presence-related outcome variables
Effects	Condition	Being-there	Interaction	Realism
Storytelling medium	Text	3.57^a (0.19)	3.16^a (0.20)	4.88^a (0.15)
	360°-video	4.78^b (0.19)	4.08^b (0.19)	5.57^b (0.15)
	VR	5.45^c (0.19)	4.70^b (0.20)	5.78^b (0.15)
		F (2, 122) = 25.10, p < 0.001, partial η² = 0.27	F (2, 122.8) = 15.06, p < 0.001, partial η² = 0.17	F (2, 124.7) = 9.97, p < 0.001, partial η² = 0.08
Story	The displaced	4.62^a (0.12)	4.20^a (0.13)	5.64^a (0.09)
	The click effect	4.58^a (0.12)	3.76^b (0.13)	5.18^b (0.10)
		F (1, 124.2) = 0.11, p = 0.74, partial η² = 0.002	F (1, 124.9) = 14.52, p < 0.001, partial η² = 0.03	F (1, 126.8) = 29.03, p < 0.001, partial η² = 0.06
Storytelling medium × Story	The displaced (Text)	3.57^a (0.21)	3.31^a (0.22)	5.08^bc (0.17)
	The displaced (360°-video)	5.00^bc (0.21)	4.50^b (0.22)	5.95^a (0.16)
	The displaced (VR)	5.28^bc (0.21)	4.78^b (0.22)	5.88^a (0.17)
	The click effect (Text)	3.56^a (0.21)	3.02^a (0.22)	4.68^c (0.17)
	The click effect (360°-video)	4.56^b (0.21)	3.66^a (0.22)	5.19^bc (0.16)
	The click effect (VR)	5.62^c (0.22)	4.61^b (0.22)	5.67^ab (0.17)
		F (2, 122.5) = 3.84, p < 0.05, partial η² = 0.02	F (2, 123.3) = 3.37, p < 0.05, partial η² = 0.01	F (2, 125.1) = 3.66, p < 0.05, partial η² = 0.02

Note: Within each column, condition means for a given effect term that do not have a letter in common in their superscripts differ at p < 0.05, according to Tukey-Kramer HSD post hoc test.

Source credibility

Participants in the VR and 360°-video conditions rated The New York Times as significantly more trustworthy than those in the Text, but no such main effect was found for perceived expertise (Table 5).

Table 5.

The Effects of Storytelling Medium and Story on Credibility (Trustworthiness and Expertise)

		Source credibility
Effects	Condition	Trustworthiness	Expertise
Storytelling medium	Text	5.85^a (0.07)	6.27^a (0.06)
	360°-video	6.17^b (0.07)	6.38^a (0.06)
	VR	6.22^b (0.07)	6.45^a (0.06)
		F (2, 122) = 7.38, p < 0.001, partial η² = 0.07	F (2, 122) = 2.29, p = 0.11, partial η² = 0.001
Story	The displaced	6.12^a (0.05)	6.37^a (0.04)
	The click effect	6.04^a (0.05)	6.36^a (0.04)
		F (1, 125) = 3.04, p = 0.08, partial η² = 0.003	F (1, 125) = 0.02, p = 0.90, partial η² = 0.07
Storytelling medium × Story	The displaced (Text)	5.85^a (0.08)	6.24^a (0.07)
	The displaced (360°-video)	6.25^b (0.08)	6.44^a (0.07)
	The displaced (VR)	6.25^b (0.08)	6.42^a (0.07)
	The click effect (Text)	5.85^a (0.08)	6.30^a (0.07)
	The click effect (360°-video)	6.08^ab (0.08)	6.31^a (0.07)
	The click effect (VR)	6.19^ab (0.08)	6.49^a (0.07)
		F (2, 125) = 1.25, p = 0.29, partial η² = 0.02	F (2, 125) = 2.23, p = 0.11, partial η² = 0.001

Note: Within each column, condition means for a given effect term that do not have a letter in common in their superscripts differ at p < 0.05, according to Tukey-Kramer HSD post hoc test.

Story recall

A significant two-way interaction for free recall revealed that participants in Text condition performed better than those in the VR condition, with those in 360°-video scoring in the middle for “The click effect,” but there were no such condition-based differences for “The displaced” (Table 6). The same pattern was found for cued recall.

Table 6.

The Effects of Storytelling Medium and Story on Cued Recall, Free Recall, and Attention Allocation

		Cognitive outcome variables
Effects	Condition	Cued Recall	Free Recall	Attention Allocation
Storytelling medium	Text	3.90^a (0.13)	3.25^a (0.16)	5.37^a (0.15)
	360°-video	3.74^a (0.13)	3.27^a (0.16)	5.47^ab (0.15)
	VR	3.52^a (0.13)	2.90^a (0.16)	5.89^b (0.15)
		F (2, 125) = 2.08, p = 0.13, partial η² = 0.008	F (2, 122.4) = 1.63, p = 0.20, partial η² = 0.001	F (2, 124.3) = 3.28, p < 0.05, partial η² = 0.03
Story	The displaced	4.12^a (0.09)	3.34^a (0.12)	6.00^a (0.11)
	The click effect	3.33^b (0.09)	2.95^b (0.12)	5.16^b (0.11)
		F (1, 142.4) = 43.87, p < 0.001, partial η² = 0.04	F (1, 139.7) = 7.17, p < 0.01, partial η² = 0.07	F (1, 125) = 42.78, p < 0.001, partial η² = 0.10
Storytelling medium x Story	The displaced (Text)	4.05^a (0.16)	3.11^ab (0.20)	5.58^ab (0.19)
	The displaced (360°-video)	4.27^a (0.16)	3.74^a (0.20)	6.20^a (0.19)
	The displaced (VR)	4.03^a (0.16)	3.16^ab (0.20)	6.21^a (0.19)
	The click effect (Text)	3.75^ab (0.16)	3.39^ab (0.20)	5.15^bc (0.19)
	The click effect (360°-video)	3.21^bc (0.16)	2.81^b (.20)	4.74^c (0.19)
	The click effect (VR)	3.02^c (0.16)	2.64^b (0.20)	5.58^ab (0.19)
		F (2, 126.4) = 4.96, p < 0.05, partial η² = 0.06	F (2, 123.8) = 6.68, p < 0.05, partial η² = 0.07	F (2, 125) = 6.21, p < 0.01, partial η² = 0.03

Note: Within each column, condition means for a given effect term that do not have a letter in common in their superscripts differ at p < 0.05, according to Tukey-Kramer HSD post hoc test.

Empathy and story-sharing intention

A significant main effect shows that participants in the VR and 360°-video conditions were more empathetic toward the story characters than their counterparts in the Text condition (Table 7).

Table 7.

The Effects of Storytelling Medium and Story on Empathy and Story-Sharing Intention

		Prosocial outcome variables
Effects	Condition	Empathy	Story-sharing intention
Storytelling medium	Text	4.70^a (0.13)	2.80^a (0.19)
	360°-video	5.15^b (0.13)	3.49^b (0.19)
	VR	5.33^b (0.13)	3.82^b (0.19)
		F (2, 123) = 6.30, p < 0.01, partial η² = 0.07	F (2, 123) = 7.35, p < 0.001, partial η² = 0.10
Story	The displaced	5.76^a (0.10)	3.68^a (0.12)
	The click effect	4.36^b (0.10)	3.06^b (0.12)
		F (1, 125) = 109.09, p < 0.001, partial η² = 0.38	F (1, 125) = 40.30, p < 0.001, partial η² = 0.06
Storytelling medium × Story	The displaced (Text)	5.37^ab (0.17)	3.02^bc (0.21)
	The displaced (360°-video)	6.01^a (0.17)	3.98^a (0.21)
	The displaced (VR)	5.90^a (0.17)	4.03^a (0.21)
	The click effect (Text)	4.04^d (0.17)	2.58^c (0.21)
	The click effect (360°-video)	4.29^cd (0.17)	2.99^bc (0.21)
	The click effect (VR)	4.76^bc (0.17)	3.60^ab (0.21)
		F (2, 125) = 1.65, p = 0.20, partial η² = 0.02	F (2, 125) = 3.64, p < 0.05, partial η² = 0.01

Note: Within each column, condition means for a given effect term that do not have a letter in common in their superscripts differ at p < 0.05, according to Tukey-Kramer HSD post hoc test.

Similarly, participants in the VR and 360°-video conditions reported higher story-sharing intention than those in Text. A significant interaction showed that while participants in VR and 360°-video conditions were equally likely to share “The displaced” than those in Text, it is only the former who were more likely to share “The click effect” than those in Text.

See Figure 1 for a visual summary of main effects of storytelling medium.

FIG. 1.

Main effect of storytelling medium on key variables.

Indirect effects

We used 10,000 bootstrap samples and a 95% confidence interval⁴¹ to test H2 to H4. We conducted mediation analyses for each story separately, using Model 4 of Hayes' macro,⁴² to obtain indirect effects of senses of being-there, interaction, and realism.

For “The displaced,” we found that being-there and realism were significant mediators for trustworthiness in the VR and 360°-video conditions. Interestingly, the direction of these indirect effects was negative for being-there (i.e., higher sense of being-there was associated with lower trustworthiness toward the New York Times) and positive for sense of realism. On the measure of expertise, only realism was a significant positive mediator. Those in the VR and 360°-video conditions reported higher sense of realism, in comparison to Text; this was in turn positively associated with perceived expertise of the New York Times, in support of H2 (Figs. 2 and 3). Senses of being-there, interaction, and realism were not significant mediators for cued recall and free recall, thereby failing to support H3 (Figs. 4 and 5). For story-sharing intention, participants in the VR and 360°-video conditions reported higher sense of being-there, which in turn, was associated with higher story-sharing intention, in comparison to Text, in support of H4b (Figs. 6 and 7).

FIG. 2.

“The displaced”: Indirect effect of storytelling medium on trustworthiness and expertise (VR-to-Text comparison).

FIG. 3.

“The displaced”: Indirect effect of storytelling medium on trustworthiness and expertise (360°-video-to-Text comparison).

FIG. 4.

“The displaced”: Indirect effect of storytelling medium on cued and free recall (VR-to-Text comparison).

FIG. 5.

“The displaced”: Indirect effect of storytelling medium on cued and free recall (360°-video-to-Text comparison).

FIG. 6.

“The displaced”: Indirect effect of storytelling medium on empathy and story-sharing intention (VR-to-Text comparison).

FIG. 7.

“The displaced”: Indirect effect of storytelling medium on empathy and story-sharing intention (360°-video-to-Text comparison).

The same mediation analyses with “The click effect” showed that, for participants in VR condition, relative to those in Text, realism mediated the relationship with trustworthiness and expertise, again supporting H2 (Fig. 8), whereas for 360°-video-to-Text comparison, the mediation was significant for expertise, but not trustworthiness (Fig. 9). For 360°-video-to-Text comparison, we found that sense of interaction significantly mediated the relationship with cued recall, but in the negative direction, disconfirming H3 (Fig. 11), although this was not found for the VR-to-Text comparison (Fig. 10). Those in the 360°-video condition reported higher sense of interaction, which in turn was associated with poorer cued story recall, compared to Text. None of the three presence-related factors were significant mediators for empathy (Figs. 12 and 13), but sense of being-there was a significant mediator of the effect of VR on story-sharing intention, thereby partially supporting H4b.

FIG. 8.

“The click effect”: Indirect effect of storytelling medium on trustworthiness and expertise (VR-to-Text comparison).

FIG. 9.

“The click effect”: Indirect effect of storytelling medium on trustworthiness and expertise (360°-video-to-Text comparison).

FIG. 10.

“The click effect”: Indirect effect of storytelling medium on cued and free recall (VR-to-Text comparison).

FIG. 11.

“The click effect”: Indirect effect of storytelling medium on cued and free recall (360°-video-to-Text comparison).

FIG. 12.

“The displaced”: Indirect effect of storytelling medium on empathy and story-sharing intention (VR-to-Text comparison).

FIG. 13.

“The click effect”: Indirect effect of storytelling medium on empathy and story-sharing intention (360°-video-to-Text comparison).

Discussion

It is clear that VR stories via headsets and/or 360°-videos provide significantly greater sense of being in the midst of the story than text with pictures—quite remarkable, considering that the text we used was of high quality, rich with imagery. On all presence-related factors, both outperformed the text medium, with notable carryover effects on perception (credibility), cognition (memory), and conation (story-sharing intention). The strength and consistency of these effects suggest that telepresence is the key value added over text by 360°-video and over 360°-video by VR headset, implying the joint operation of modality and interactivity affordances in shaping user perceptions of being transported into the mediated environment.^5,10 The more intuitively that individuals can interact with the medium, the higher the presence, in keeping with the principle of “free energy minimization.”^12,43 When individuals receive multimodal inputs to their senses, as well as ability to explore the story environment by their actions, they experience greater sense of embodiment in the story.^44,45 By affording readers richer modality and interactivity, VR stories can truly shift the location of the readers into the story. Our data suggest that 360°-video can do the same, but to a lesser extent.

Our results indicate that the degree to which the VR headset and 360°-video can induce the three presence factors depends on story's emotional intensity. While for “The displaced,” the 360°-video was comparable to VR headset in its effects, it was clearly inferior to VR in “The click effect.” This finding can be attributed to the powerful emotional impact of “The displaced”—stronger narratives are known to enhance presence outcomes.^13,28 The differential effects of VR headset and 360°-videos for the two stories suggest that when stories are emotionally powerful and also richly narrated, they may override the capacity of the technological factors (i.e., interactivity/modality affordances) in eliciting presence,¹³ as these stories can provide meanings to individuals' mediated experience.⁴⁶ When stories are less emotionally powerful and not as richly narrated, however, the technological factors may play a bigger role in determining presence.

It is possible the story's emotional intensity may have affected attention allocation (Table 6). Perhaps the powerful emotional response to “The displaced” elicited stronger physiological arousal, resulting in greater attention allocation,^47,48 regardless of the medium of storytelling. On the other hand, “The click effect,” which is regarded as less emotional, required more than 360°-video to command user attention, as evidenced by a significant two-way interaction. However, this heightened attention appears to come at a cost. Memory for the more informational story is lower (Table 6), probably because VR and 360°-video compel users to pay attention at the cost of storage, thereby exhausting the limited capacity of the human information processing system.⁴⁹

Our mediation models show that the three presence-related outcomes are not always positively related to outcomes of interest. While inducing a greater sense of being-there enhances story sharing, it can negatively affect trust in the news organization, perhaps because it triggers the “bells and whistles” heuristic¹⁰ (“all flash and no substance”), which is negatively associated with credibility. Evoking a sense of realism, on the other hand, is associated with greater trust and perceived expertise in the news source, as predicted by MAIN model. In sum, while the ability of VR and 360°-video to trigger the “being-there” heuristic is beneficial for user experience, the realism heuristic is important for enhancing credibility. This has practical implications. For example, if credibility is the goal, designers could focus user interaction on those aspects of the portrayed environment that emphasize the reality of the situation, rather than fantasy (or game-like) elements of VR story-telling.

Limitations

Aside from the usual external-validity limitations of a laboratory experiment, the current study suffers from the fact that the exploration of story content was not constant within or across conditions, thus introducing noise. Second, the VR condition required the experimenter to slip a smartphone into the VR headset, thereby contributing to novelty effects.

Future studies would do well to take these limitations into account when examining the psychological effects of VR experiences in particular and interactive media in general.

Notes

a. We decided to focus on VR, 360°-video and Text, instead of regular video, because in these three mediums, it is the readers who control the story navigation. In VR and 360°-video, readers independently navigate the direction of the video while in Text, readers decide the pace of the story as well as the order of which paragraphs to read first. This reader agency in story navigation is largely absent from non-360°-video because the viewer simply witnesses the scenes navigated by the cameraperson and producer rather than interacting with them.

b. www.youtube.com/channel/UCqnbDFdCpuN8CMEg0VuEBqA

Footnotes

Acknowledgment

The authors would like to thank Maria Molina, Carly Chiavaroli, and Natalie Guarna for their assistance with data collection.

Author Disclosure Statement

No competing financial interests exist.

References

de la Peña

, Weil

, Llobera

, et al. Immersive journalism: immersive virtual reality for the first-person experience of news. Presence: Teleoperators and Virtual Environments, 2010; 19:291–301.

Aronson-Rath

, Milward

, Owen

, et al. Virtual reality journalism. Tow Center for Digital Journalism Columbia University. 2015. http://towcenter.org/research/virtual-reality-journalism (accessed Mar. 8, 2016).

Lombard

, Ditton

. At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication, 1997; 3.

Scatena

, Nicolo

, Riva

. (2016) Virtual reality vs television vs web exposure: the impact on brand experience. A preliminary study. In: Wiederhold

, Riva

, Wiederhold

, Kirwan

, eds. Annual Review of Cybertherapy And Telemedicine 2016, Being Different: The Transformative Potential of Virtual Reality. San Diego, CA: Interactive Media Institute, p. 214.

Sundar

, Oeldorf-Hirsch

, Garga

. (2008) A cognitive-heuristics approach to understanding presence in virtual environments. In Proceedings of the 11th Annual International Workshop on Presence. Padova, Italy.

Steuer

. Defining virtual reality: dimensions determining telepresence. Journal of Communication, 1992; 42:73–93.

Sundar

. Multimedia effects on processing and perception of online news: a study of picture, audio, and video downloads. Journalism & Mass Communication Quarterly, 2000; 77:480–499.

Bucy

. (2004) The interactivity paradox: closer to the news but confused. In: Bucy

and Newhagen

, eds. Media Access: social and psychological dimensions of new technology use. Mahwah, NJ: Lawrence Erlbaum, pp. 47–72.

Ketterer

. Links engage readers of online crime stories. Newspaper Research Journal, 2001; 22:2–13.

10.

Sundar

. (2008) The MAIN model: a heuristic approach to understanding technology effects on credibility. In: Metzger

, Flanagin

. eds. Digital media, youth, and credibility. Cambridge, MA: The MIT Press, pp. 72–100.

11.

Riva

, Mantovani

. (2014) Extending the self through the tools and the others: a general framework for presence and social presence in mediated interactions. In Riva

, Waterworth

, Murray

, ed. Interacting with Presence, HCI and Sense of Presence in Computer-mediated Environments. Berlin, Germany: De Gruyter Open Ltd., pp. 9–31.

12.

Riva

. The neuroscience of body memory: From the self through the space to the others. Cortex, 2017; 1–20.

13.

Gorini

, Capideville

, De Leo

, et al. The role of immersion and narrative in mediated presence: The virtual hospital experience. Cyberpsychology, Behavior, and Social Networking, 2011; 14:99–105.

14.

Hou

, Nam

, Peng

, et al. Effects of screen size, viewing angle, and players' immersion tendencies on game experience. Computers in Human Behavior, 2012; 28:617–623.

15.

Skalski

, Whitbred

. Image versus sound: A comparison of formal feature effects on presence and video game enjoyment. PsychNology Journal, 2010; 8:67–84.

16.

Lin

, Duh

HBL

, Parker

, et al. (2002) Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings of the IEEE Virtual Reality Conference. Washington, DC, USA.

17.

Slater

, Wilbur

. A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 1997; 6:603–616.

18.

Fox

, Bailenson

, Binney

. Virtual experiences, physical behaviors: the effect of presence on imitation on an eating avatar. Presence: Teleoperators & Virtual Environments, 2009; 18:294–303.

19.

Manly

. (2015) A virtual reality revolution, coming to a headset near you. www.nytimes.com/2015/11/22/arts/a-virtual-reality-revolution-coming-to-a-headset-near-you.html?_r=0 (accessed Feb. 22, 2016)

20.

The New York Times. (2015) The New York Times launches NYT VR; delivers over one million Google cardboard viewers. http://investors.nytco.com/press/press-releases/press-release-details/2015/The-New-York-Times-Launches-NYT-VR-Delivers-Over-One-Million-Google-Cardboard-Viewers/default.aspx (accessed Mar. 6, 2017)

21.

Cheng

, Chiang

, Ye

, et al. Enhancing empathy instruction using a collaborative virtual learning environment for children with autistic spectrum conditions. Computers & Education, 2000; 55:1449–1458.

22.

Gillath

, McCall

, Shaver

, et al. What can virtual reality teach us about prosocial tendencies in real and virtual environments?. Media Psychology, 2008; 11:259–282.

23.

Hasler

, Hirschberger

, Shani-Sherman

, et al. Virtual peacemakers: mimicry increases empathy in simulated contact with virtual outgroup members. Cyberpsychology, Behavior, and Social Networking, 2014; 17:766–771.

24.

Dominus

. (2015) The displaced: Hana. www.nytimes.com/2015/11/08/magazine/the-displaced-hana.html (accessed Sept. 5, 2016).

25.

Kramer

. (2015) The displaced: Oleg. www.nytimes.com/2015/11/08/magazine/the-displaced-oleg.html (accessed Sept. 5, 2016).

26.

Santora

. (2015) The displaced: Chuol. www.nytimes.com/2015/11/08/magazine/the-displaced-chuol.html (accessed Sept. 5, 2016).

27.

Nestor

. (2016) Conversation with whales. www.nytimes.com/interactive/2016/04/16/opinion/sunday/conversation-with-whales.html (accessed Sept. 5, 2016).

28.

Riva

, Mantovani

, Capideville

, et al. Affective interactions using virtual reality: the link between presence and emotions. CyberPsychology & Behavior, 2007; 10:45–56.

29.

Rubin

, Hoyle

, Leary

. Differential predictability of four dimensions of affect intensity. Cognition & Emotion, 2012; 26:25–41.

30.

Posner

, Russell

, Peterson

. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 2005; 17:715–734.

31.

Davis

. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology, 1980; 10:85.

32.

Lee

, Sundar

. To tweet or to retweet? That is the question for health professionals on Twitter. Health Communication, 2013; 28:509–524.

33.

Baños

, Botella

, Garcia-Palacios

, et al. Presence and reality judgment in virtual environments: A unitary construct?. CyberPsychology & Behavior, 2000; 3:327–335.

34.

Sundar

, Limperos

. Uses and grats 2.0: New gratifications for new media. Journal of Broadcasting & Electronic Media, 2013; 57:504–525.

35.

, Basdogan

, Slater

, et al. (1998) An experiment on the influence of haptic communication on the sense of being together. In Proceedings of the British Telecom Workshop on Presence in Shared Virtual Environments, Ipswich, UK.

36.

Bailenson

, Aharoni

, Beall

, et al. (2004) Comparing behavioral and self-report measures of embodied agents' social presence in immersive virtual environments. In Proceedings of the 7th Annual International Workshop on PRESENCE. Valencia, Spain.

37.

Wirth

, Hartmann

, Böcking

, et al. A process model of the formation of spatial presence experiences. Media Psychology, 2007; 9:493–525.

38.

Kim

, Biocca

. Telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion. Journal of Computer-Mediated Communication, 1997; 2.

39.

Batson

, Sager

, Garst

, et al. Is empathy-induced helping due to self–other merging?. Journal of Personality and Social Psychology, 1997; 73:495–509.

40.

Burgoon

, Hale

. Validation and measurement of the fundamental themes of relational communication. Communications Monographs, 1987; 54:19–41.

41.

Hayes

, Preacher

. Statistical mediation analysis with a multicategorical independent variable. British Journal of Mathematical and Statistical Psychology, 2014; 67:451–470.

42.

Hayes

. (2012). My macros and code for SPSS and SAS. www.afhayes.com/spss-sas-and-mplus-macros-and-code.html (accessed Mar. 3, 2017).

43.

Riva

, Waterworth

. (2014) Being present in a virtual world. In M. Grimshaw, ed. The oxford handbook of virtuality. New York: Oxford University Press, pp. 205–221.

44.

Waterworth

, Waterworth

. (2014) Altered, expanded and distributed embodiment: the three stages of interactive presence. In Riva

, Waterworth

, Murray

, ed. Interacting with Presence, HCI and Sense of Presence in Computer-mediated Environments. De Gruyter Open Ltd., pp. 32–45.

45.

Carroll

, Shih

, Hoffman

, et al. (2014) Presence and hyperpresence: implications for community awareness. In Riva

, Waterworth

, Murray

, ed. Interacting with Presence, HCI and Sense of Presence in Computer-mediated Environments. De Gruyter Open Ltd, pp. 70–82.

46.

Villani

, Repetto

, Cipresso

, et al. May I experience more presence in doing the same thing in virtual reality than in reality? An answer from a simulated job interview. Interacting with Computers, 2012; 24:265–272.

47.

Lang

, Bolls

, Potter

, Kawahara

. The effects of production pacing and arousing content on the information processing of television message. Journal of Broadcasting & Electronic Media, 1999; 43:451–476.

48.

Bolls

, Lang

, Potter

. The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Communication Research, 2001; 28:627–651.

49.

Lang

. The limited capacity model of mediated message processing. Journal of Communication, 2000; 50:46–70.