The Effect of Auditory and Visual Recommendations on Choice

Abstract

We explore the effect of recommendation modality on recommendation adherence. Results from five experiments run on various online platforms (N = 6,103 adults from TurkPrime and Prolific) show that people are more likely to adhere to recommendations that they hear (auditory) than recommendations that they read (visual). This effect persists regardless of whether the auditory recommendation is spoken by a human voice or an automated voice and holds for hypothetical and consequential choices. We show that the effect is in part driven by the relative need for closure—manifested in a sense of urgency—that is evoked by the ephemerality of auditory messages. This work suggests that differences in the physical properties of auditory and visual modalities can lead to meaningful psychological and behavioral consequences.

Keywords

judgment decision-making open data preregistered modality open materials

New technologies have increased the accessibility of information in diverse modalities. Indeed, smart speakers are among the world’s fastest-growing technology segments and allow people to hear the same information that in the past they could only read on a browser (Canalys, 2018; Molla, 2018; Munz & Morwitz, 2020). For instance, prior to choosing a restaurant, a person might read about a recommended restaurant online, or they might hear the same recommendation from Alexa or Siri. In this article, we examine the effect of the modality of a recommendation on choice.

Prior research shows that the modality used to convey information influences perceptions of the communicator (Chaiken & Eagly, 1983; Schroeder & Epley, 2015). Much of the existing literature on modality effects explores the ways in which the auditory mode enhances the salience of the communicator of a message and that communicator’s attitudes, even when the content of the information is held constant (Chaiken & Eagly, 1983). Recipients perceive a communicator to be more competent, thoughtful, and intelligent when they hear the communicator’s message rather than when they read it (Schroeder & Epley, 2015, 2016; Schroeder et al., 2017). Nonverbal cues, including the tone and volume of communication, have been shown to have a significant impact on perceptions of the communicator’s attitude and confidence (Mehrabian & Wiener, 1967; Van Zant & Berger, 2020).

In contrast, we focus on how the physical properties of auditory and visual information affect the recipient, independent of the recipient’s perceptions of the communicator. Specifically, we show that the ephemerality of auditory messages relative to visual messages influences persuasiveness. When people receive recommendations through the visual channel, the recommendation remains visually present; that is, people continue to be exposed to the recommendation even after reading it. Conversely, spoken language is limited to a specific time span (Auer, 2009; Leahy & Sweller, 2011; Singh et al., 2012). When people receive recommendations through the auditory channel, the recommendation disappears after it is heard.

We suggest that the ephemerality of auditory messages evokes a sense of relative urgency by creating a time-pressure cue (Morlok et al., 2018). Although we are unaware of literature that links ephemerality and urgency per se, there is reason to believe that such a relationship exists. Prior research suggests that ephemeral content on online messaging platforms drives quicker responses (Bayer et al., 2016; Flecha-Ortíz et al., 2019). When auditory messages disappear, they evoke the perception of a restricted time during which a response is expected (Berger & Iyengar, 2013)—a speaker’s question or statement during a phone conversation warrants a near-immediate response. In contrast, the permanence of visual messages does not evoke a similar perception of urgency (Berger & Iyengar, 2013)—in most cases, people are not expected to respond right away to a text message or an email. The mere presence of a time restriction can create a sense of urgency (Swain et al., 2006; Zhu et al., 2018). For example, Zhu et al. (2018) found that participants felt a greater urgency to complete tasks with nominally shorter expiration times, even when these tasks were less important.

Urgency is one of the two primary inclinations underlying the need for closure, which refers to the tendency to seize on information quickly and maintain, or freeze on, that information (Kruglanski & Webster, 1996, p. 265; Roets & Van Hiel, 2007, 2011). Need for closure influences judgments and decision-making, including impression formation, stereotypic judgments, anchoring effects, attribution, the extent of information processing, and—most relevant to our context—persuasion. For instance, Kruglanski et al. (1993) found that in the absence of prior information, a heightened need for closure led to greater persuasion. Before forming a crystallized opinion, the urgency tendency associated with the need for closure affects persuasion by leading individuals to exhibit a speeded-up reliance on early cues and seize on early information when committing to a judgment (Kruglanski & Webster, 1996; Kruglanski et al., 1993; Mayseless & Kruglanski, 1987). We predict that in the context of our studies, recommendations serve as these cues so that need for closure—manifested as the urgent desire to reach swift decisions—will result in recommendation adherence. To capture the specific aspect of need for closure that is relevant to our theory, we focus on measuring participants’ sense of urgency when making their choices using measures adapted from the Need for Closure Scale.

Overview of Studies

To isolate the effect of recommendation modality, our experimental paradigm held other aspects of the choice process constant, including the modality in which choices are presented and the response modality. Studies 1a and 1b presented contexts in which auditory recommendations were followed more than visual recommendations for a variety of choices and provided evidence that the effect was not solely driven by perceptions of the recommender. Study 2 provided evidence that the effect holds for consequential choices. Study 3 demonstrated an attenuation of the effect when visual recommendations are made ephemeral. Finally, Study 4 demonstrated that perceived urgency can mediate the modality effect on recommendation adherence. For every study, we report all manipulations and all measures in the main text. Sample sizes were determined prior to data collection, and we aimed for at least 200 participants per cell (Simmons et al., 2011). We did not exclude any observations among people who completed the studies. To recruit participants with diverse demographic backgrounds and boost the generality of our findings, we conducted our studies on online populations. For every study presented below, we preregistered our sample size, methods, predictions, and analysis plan at AsPredicted. Our stimuli and data on all measures have been made available on OSF at https://osf.io/49v5z/. We complied with American Psychological Association, national, and institutional guidelines for experimental conduct and ethical treatment of participants.

Statement of Relevance

New technologies (e.g., Apple’s Siri, Amazon’s Alexa, Microsoft’s Cortana, and Google’s Google Assistant) that utilize voice allow people convenient and interactive ways to exchange information. The purpose of these technologies is to provide a convenient, hands-free means through which people can receive information and interact with the Internet. But do people weight and act on auditory and visual information identically, regardless of the modality through which they receive it? Our research suggests that modality matters: Physical properties that are unique to certain modalities can result in different psychological and behavioral experiences for the information recipient. We focus on auditory messages and show that regardless of whether the voice is human or automated, people are more likely to adhere to information presented aurally than visually.

Study 1a

In Study 1a (preregistration: https://aspredicted.org/ar3v7.pdf), we examined whether people adhere to auditory recommendations more than visual recommendations, independent of their perceptions of the recommender. Prior research implies that the intonation and varied pace of auditory messages emphasize the humanness, competence, intelligence, and thoughtfulness of the communicator and make such messages persuasive (Schroeder & Epley, 2015, 2016; Schroeder et al., 2017). However, we propose that the ephemerality of auditory messages operates separately from perceptions of the communicator and causes differences in message effectiveness even when the spoken voice is automated and lacks paralinguistic cues.

Method

We recruited participants from the Prolific online platform in exchange for payment. Three hundred ninety-three individuals (48.9% female; age: M = 33.7 years; SD = 12.5) completed this study. Each participant was randomly assigned to one of two recommendation modality conditions (auditory or visual) in a between-subjects design. Participants received recommendations and made choices in four categories: beer, credit card, extension cord, and restaurant. For example, for the category of beer, participants were told: “Imagine that you are looking for a recommendation for the best beer that goes with ramen.” In the visual condition, participants read a recommendation that stated, “I recommend the Jack Hammer IPA.” In the auditory condition, participants heard the same recommendation spoken using text-to-speech software, in which the software employs a robotic-sounding female voice to read text aloud (see Schroeder & Epley et al., 2016). In all studies presented in the main text, we employed female voices to match the default option provided by smart speakers and voice assistants such as Apple’s Siri, Amazon’s Alexa, Microsoft’s Cortana, and Google’s Google Assistant. (In Study S1, reported in the Supplemental Material available online, we found that the effect was robust to the gender of the automated voice.) To equate the level of effort and action required in both modality conditions, we designed the study so that the recommendations did not appear or sound until participants indicated their readiness by clicking a button.

After receiving their recommendation, participants proceeded to the following page and were asked to make a choice among three options for that category, one of which was the recommended option. For the category of beer, participants were asked to make a choice among the following three options: Jack Hammer IPA, Cocoa Psycho Stout, or Dead Pony Club—Pale Ale. Across all categories, all options were presented with additional details (such as average ratings or price) to ensure that participants felt that they were making a realistic choice. The recommended option was not always the one with the lowest price or highest ratings.

After participants made their choices in all categories, to confirm that perceptions of the recommender were not driving recommendation adherence, we measured the extent to which the participants found the recommender competent, intelligent, and thoughtful and formed a composite measure of perceived intellect using these items (Schroeder & Epley, 2015; α = .87). All items used a 9-point Likert-type scale ranging from 1, not at all, to 9, very.

Results

The results confirmed our prediction that participants would be more likely to follow auditory recommendations than visual recommendations. We ran a mixed-effects logistic regression model with recommendation adherence as the dependent variable (1 = followed, 0 = not followed) and the modality of the recommendations as the fixed effect. Following the preregistration, we used random intercepts for participants and the recommendation categories.¹ There was a main effect of modality—auditory recommendations were more likely to be adhered to than visual recommendations, b = 0.79, SE = 0.21, z = 3.81, p < .001, 95% confidence interval (CI) = [0.39, 1.22]. This observed pattern was consistent across all four categories (see Fig. 1). Whether the recommendations were auditory or visual did not significantly interact with the order of the categories in this study or any other study.

Fig. 1.

Percentage of participants who adhered to the recommendation in each condition (Study 1a). Note that the main text presents analyses in terms of log odds coefficients, and this figure presents percentages for ease of interpretability. Error bars represent 95% confidence intervals.

There was also a significant main effect of modality on participants’ ratings of intellect: The recommender was viewed more favorably among participants in the visual condition than among participants in the auditory condition, b = 0.52, SE = 0.16, t(391) = 3.19, p = .002, 95% CI = [0.20, 0.84]. When we controlled for intellect (this analysis was not preregistered), the modality effect on recommendation adherence remained significant, b = 1.09, SE = 0.20, z = 5.54, p < .001, 95% CI = [0.71, 1.49]. Given that auditory recommendations were followed more despite the fact that the recommender was rated less favorably, this study suggests that the modality effect on persuasion is not driven solely by perceptions of the communicator.

Study 1b

In Study 1b (preregistration: https://aspredicted.org/qc9nj.pdf), we provided additional evidence that perceptions of the recommender—humanness and intellect (Schroeder & Epley, 2015, 2016)—do not fully explain the effect of computerized voices on recommendation adherence. In order to test this, we replaced the computerized font in the visual condition with human handwriting. We also measured participants’ perceptions of the novelty of the recommendations to gauge whether computerized auditory recommendations drew more attention to the stimuli. In addition, given that prior literature on modality primarily utilizes human voices, we added a second auditory condition in which the recommendations were spoken by a female actor.

Method

We recruited 1,177 participants from the Prolific online platform (49.7% female; age: M = 31.2 years; SD = 11.8). Each participant was randomly assigned to one of three recommendation-modality conditions: auditory-human, auditory-computer, or visual-handwritten. Participants received recommendations and made choices for the same four categories used in Study 1a.

After participants made their choices in all categories, we measured perceived competence, intelligence, and thoughtfulness of the recommender (α = .91) as in Study 1a. In a measure adapted from Schroeder and Epley (2016), participants also reported the degree to which they believed that the recommendations were originally generated by a human compared with a computer. Participants in all conditions read the following: “As you may know, computer technology is now attempting to mimic human language. For the recommendations you received, do you believe that the content of the recommendations was originally created by a computer or by a human?” (1 = computer, 9 = human). For each condition, we explicitly reminded participants that the recommendations that they received were either read by a human, written by a human, or read by a computer and that their job was not to determine who voiced or wrote the recommendations. Finally, participants reported how novel and unique they found the form in which they received the recommendations, which we used to create an index of novelty (r = .82, p < .001).

Results

The results of this study showed that recommendation adherence is greater for auditory recommendations, whether delivered by a human or computer voice. This effect remained despite the fact that participants rated the humanness and intellect of the recommender and the novelty of the recommendations to be lower in the auditory-computer condition.

Perceived intellect

The recommender in the auditory-computer condition was rated less favorably than the recommenders in the visual-handwritten condition and the auditory-human condition—visual-handwritten: b = −0.61, SE = 0.13, t(1174) = −4.84, p < .001, 95% CI = [−0.86, −0.36]; auditory-human: b = −0.89, SE = 0.13, t(1174) = −7.06, p < .001, 95% CI = [−1.14, −0.64] (see Fig. 2a). The recommender in the visual-handwritten condition was rated less favorably than the recommender in the auditory-human condition, consistent with past literature, b = −0.28, SE = 0.13, t(1174) = −2.21, p = .028, 95% CI = [−0.53, −0.03] (Schroeder & Epley, 2015).

Fig. 2.

Results of Study 1b. (a) Participants’ mean ratings of intellect of the recommender, humanness of the recommender, and novelty of the recommendations. Error bars represent 95% confidence intervals. (b) Percentage of participants who adhered to the recommendation in each condition. Note that the main text presents analyses in terms of log odds coefficients, and this figure presents percentages for ease of interpretability. Error bars represent 95% confidence intervals.

Perceived humanness

The auditory-computer recommender was perceived as less human than both the visual-handwritten recommender and the auditory-human recommender—visual-handwritten: b = −1.35, SE = 0.17, t(1174) = −7.98, p < .001, 95% CI = [−1.68, −1.02]; auditory-human: b = −2.20, SE = 0.17, t(1174) = −13.00, p < .001, 95% CI = [−2.53, −1.86] (see Fig. 2a). The visual-handwritten recommender was perceived as less human than the auditory-human recommender, also consistent with findings reported in past literature, b = −0.84, SE = 0.17, t(1174) = −4.99, p < .001, 95% CI = [−1.18, −0.51] (Schroeder & Epley, 2016).

Perceived novelty

The auditory-computer condition was perceived as less novel than the visual-handwritten condition, b = −0.51, SE = 0.16, t(1174) = −3.18, p = .002, 95% CI = [−0.83, −0.20] (see Fig. 2a) and marginally less novel than the auditory-human condition, b = −0.28, SE = 0.16, t(1174) = −1.70, p = .089, 95% CI = [−0.59, 0.04]. Those who saw visual-handwritten recommendations perceived them to be equally as novel as the auditory-human condition, b = 0.24, SE = 0.16, t(1174) = 1.48, p = .140, 95% CI = [−0.08, 0.55].

Recommendation adherence

Despite decreased perceptions of the recommenders’ intellect, the recommenders’ humanness, and the novelty of the recommendations, participants were still marginally more likely to adhere to recommendations in the auditory-computer condition than in the visual-handwritten condition, b = 0.26, SE = 0.14, z = 1.85, p = .065, 95% CI = [−0.02, 0.54] (see Fig. 2b). Indeed, when controlling for these variables (this analysis was not preregistered), we found the effect of computerized voices on recommendation adherence to be highly significant, b = 0.68, SE = 0.13, z = 5.28, p < .001, 95% CI = [0.43, 0.93].²

Participants were significantly more likely to adhere to recommendations in the auditory-human condition than in the visual-handwritten condition, b = 0.65, SE = 0.14, z = 4.64, p < .001, 95% CI = [0.38, 0.94] (see Fig. 2b), illustrating that the modality effect is robust to different types of voices and writing formats. This effect was significant even after we controlled for perceptions of intellect, humanness, and novelty, b = 0.46, SE = 0.12, z = 3.69, p < .001, 95% CI = [0.22, 0.71].

As one might expect, anthropomorphic attributes and ephemerality have additive effects, so recommendation adherence was highest in the auditory-human condition. Results comparing the two auditory conditions to one another are reported in the Supplemental Material.

In addition, we conducted a posttest in which we measured the degree to which the recommendations in this study felt personalized, the likability of the recommender, and the degree to which participants could relate to the recommender (see Study S2 in the Supplemental Material). The auditory-computer condition scored lowest on all dimensions, casting doubt on these alternative explanations.

Study 2

Our goal in Study 2 (preregistration: https://aspredicted.org/bk68q.pdf) was to replicate the modality effect from Studies 1a and 1b for consequential choices.

Method

We recruited 400 (45.0% female; age: M = 37.6 years; SD = 11.9) participants from the TurkPrime online platform in exchange for payment. Given that TurkPrime workers routinely make choices about which online tasks to complete for payment, we gave participants one such choice in this study.³ Participants were told that they would be randomly assigned a topic and would have to choose between two games to play for that topic: a word search or a word unscramble. In addition, they were told that participants from a previous study had completed both games for that topic and had been asked which game they would recommend to a future participant. Current participants were told that the computer would relay to them a randomly selected recommendation from a previous participant. This explicitly equated the recommendation agent across conditions so that perceptions of the recommender would be unlikely to influence choice. In reality, all participants were given the topic of sports and were recommended the word search. We manipulated the modality of the recommendation (auditory or visual) using an automated female voice for the auditory recommendations and typed text for the visual recommendations.

After participants made their choices, they had up to 60 seconds to complete their chosen game. Participants were incentivized to complete the game as quickly as possible so as to increase their overall wage rate. They could advance in the survey only after correctly entering the three words required to finish the game. If they had not correctly completed the game after 60 seconds, the correct answers would appear automatically on their screen. Each of the two games had exactly three correct responses. For both games, the three correct responses were the same.

Results

Consistent with Studies 1a and 1b, our results showed that the auditory recommendation was more persuasive than the visual recommendation. A logistic regression model revealed that participants were significantly more likely to adhere to the auditory recommendation (84.7%) than the visual recommendation (75.6%), b = 0.58, SE = 0.26, z = 2.27, p = .023, 95% CI = [0.08, 1.09], thus replicating the main effect from Studies 1a and 1b and confirming that the effect also holds for consequential choices.

Study 3

In Study 3 (preregistration: https://aspredicted.org/3ac4f.pdf), we provide evidence that ephemerality contributes to the effect of modality on recommendation adherence. If the modality effect is in part driven by ephemerality, then eliminating the perceived permanence of visual information should increase the likelihood of following visual recommendations. We tested this hypothesis by making visual recommendations ephemeral.

Method

We recruited 2,106 individuals (53.8% female; age: M = 37.8 years; SD = 12.9) from the TurkPrime online platform. In this study, we used the same paradigm that was used in Studies 1a and 1b with four different categories: brownie recipe, hotel, shampoo, and toaster oven. Each participant was randomly assigned to one of three modality conditions: auditory, disappearing visual, or permanent visual. Each participant was told that they would be using either a smart speaker (auditory condition) or an Internet search engine (visual conditions) to ask for various recommendations. Recommendations in the auditory condition were presented by an automated female voice. In the disappearing-visual condition, recommendations were presented visually in the form of sequentially appearing words. Each word of the sentence appeared on the screen one at a time and disappeared as the next word appeared on the screen at approximately the same pace as the words spoken in the auditory condition. In the permanent-visual condition, each word of the recommendations appeared sequentially on the screen at approximately the same pace; however, the words did not disappear after they were shown. Thus, by the end, the entire recommendation was visible. This design served to equate the visual conditions as much as possible—including the way in which words appeared—while manipulating only the ephemerality of the words.

Results

As predicted, participants in the ephemeral conditions, both visual and auditory, were more likely to adhere to the recommendations than participants in the permanent-visual condition (see Fig. 3). We conducted pairwise comparisons between the three modality conditions using a mixed-effects logistic regression model. Participants in the auditory condition were no more likely to adhere to the recommendations than participants in the disappearing-visual condition (b = 0.05, SE = 0.09, z = 0.54, p = .591, 95% CI = [−0.13, 0.24]), but participants in both ephemeral conditions were more likely to adhere to the recommendations than participants in the permanent-visual condition (auditory: b = 0.43, SE = 0.09, z = 4.56, p < .001, 95% CI = [0.24, 0.61]; disappearing visual: b = 0.38, SE = 0.09, z = 4.02, p < .001, 95% CI = [0.19, 0.56]). Thus, when the visual recommendations were made ephemeral, participants were just as likely to follow visual recommendations as auditory recommendations.

Fig. 3.

Percentage of participants adhering to the recommendation in each condition (Study 3). Note that the main text presents analyses in terms of log odds coefficients, and this figure presents percentages for ease of interpretability. Error bars represent 95% confidence intervals.

Study 4

In Study 4, we measured perceived urgency as a mediator for the modality effect. We predicted that auditory recommendations would increase perceptions of urgency, which in turn would lead to recommendation adherence.

Method

We recruited 2,027 individuals (50.7% female; age: M = 37.2 years; SD = 12.5) from the TurkPrime platform (preregistration: https://aspredicted.org/z5m6g.pdf) and used the same paradigm as in prior studies, with four different categories: puzzle, set of towels, set of frying pans, and blender. Each participant was randomly assigned to receive either auditory or visual recommendations. After participants made their choices, we measured perceived urgency using two items adapted from the Need for Closure Scale (Roets and Van Hiel, 2011; r = .78, p < .001) to create an index: “To what extent did you feel rushed when making your choices?” and “To what extent did you feel like you had to make your choices quickly?” Both items used a 7-point scale ranging from not at all to very.

Results

Consistent with findings of prior studies, results revealed a main effect of modality: Auditory recommendations were more likely to be adhered to than visual recommendations, b = 0.41, SE = 0.07, z = 5.63, p < .001, 95% CI = [0.27, 0.56] (see Fig. 4). In addition, as predicted, participants in the auditory condition reported experiencing a greater sense of urgency than participants in the visual condition, b = 0.36, SE = 0.08, t(2025) = 4.25, p < .001, 95% CI = [0.19, 0.52].

Fig. 4.

Percentage of participants adhering to recommendation in each condition (Study 4). Note that the main text presents analyses in terms of log odds coefficients, and this figure presents percentages for ease of interpretability. Error bars represent 95% confidence intervals.

To test whether recommendation adherence is mediated by participants’ perceptions of urgency, we used the mdt_simple and add_index functions in the JSmediation R package to first examine the component paths by means of a joint significance test and then examine the magnitude and Monte Carlo 95% CI of the indirect effect (Yzerbyt et al., 2018).⁴ We note that participants in the auditory condition followed more recommendations than those in the visual, c = 0.24, SE = 0.04, t(2025) = 5.45, p < .001. As predicted, the auditory condition increased perceptions of urgency, a = 0.19, SE = 0.04, t(2025) = 4.25, p < .001, and perceptions of urgency were associated with greater recommendation adherence, b = 0.23, SE = 0.02, t(2024) = 10.54, p < .001. The effect of modality on recommendation adherence was reduced, c′ = 0.20, SE = 0.04, t(2024) = 4.59, p < .001. Consistent with this analysis, the indirect effect was significant (indirect effect = 0.04, 95% CI = [0.02, 0.06]; see Fig. 5). To test that participants’ self-reported urgency was not influenced by the choices they made, in Study S4 (see the Supplemental Material), we measured self-reported urgency prior to asking participants to make their choices. We replicated the mediation results found in Study 4.

Fig. 5.

Effect of modality on the number of recommendations followed, as mediated by perceived urgency (Study 4). The value in parentheses is the effect of modality on number of recommendations followed when we controlled for the mediator (***p < .001).

Timing Analyses

As we mentioned in the introduction, individuals under elevated need for closure experience an urgent desire to reach swift decisions (Roets & Van Hiel, 2011). In Study 4, we used a subjective measure of self-reported urgency to test the association between the urgency that ephemerality evokes and recommendation adherence. In Studies 2, 3, and 4, our data provide us an objective manifestation of urgency: decision time.⁵ We would expect participants in the ephemeral conditions to make their choices more quickly. Indeed, we found that participants took significantly less time to make their choices in the ephemeral conditions relative to the permanent visual condition in each study (see Table 1). And, as we expected, faster decision times were associated with greater levels of recommendation adherence (see Table 2).

Table 1.

Difference in Log-Transformed Choice Time Between the Ephemeral Conditions and the Permanent Visual Condition for All Participants and All Stimuli (Studies 2–4)

Study and condition	b	SE	t	p	95% CI
Study 2	−0.12	0.06	−2.03	.043	[−0.234, −0.004]
Study 3
Auditory	−0.13	0.04	−3.58	< .001	[−0.207, −0.061]
Disappearing visual	−0.09	0.04	−2.30	.021	[−0.159, −0.013]
Study 4	−0.10	0.03	−3.24	.001	[−0.153, −0.038]

Table 2.

Differences in Recommendation Adherence Based on Log-Transformed Choice Time for All Participants and All Stimuli (Studies 2–4)

Study	b	SE	t	p	95% CI
Study 2	−0.59	0.21	−2.84	.005	[−0.997, −0.182]
Study 3	−1.20	0.04	−27.43	< .001	[−1.288, −1.116]
Study 4	−0.87	0.04	−20.70	< .001	[−0.944, −0.776]

General Discussion

Voice assistants are revolutionizing the way people receive information. This work illustrates that fundamental differences in the properties of auditory and visual modalities can lead to psychological and behavioral consequences, even when the information content is held constant. Across five studies, we found evidence that auditory recommendations are adhered to more than visual recommendations. We show that the effect is robust to the pace of auditory information and to the gender of the automated voice (see Study S1 in the Supplemental Material). The results of Studies 1a and 1b additionally suggest that the effect cannot be explained simply by perceptions of the recommender or novelty of the recommendation form. In Study 1b, auditory recommendations were adhered to more than visual recommendations, regardless of whether the auditory recommendations were spoken by a human or a computer. Participants tended to adhere to handwritten recommendations less than to recommendations that used text-to-speech software, even after we accounted for perceptions of the humanness and intellect of the recommender and the novelty of the recommendations. Study 2 demonstrates that the modality effect on recommendation adherence holds when choices are consequential. In addition, the automated voice used in the auditory condition relayed recommendations ostensibly generated by other people, effectively holding the recommender constant across conditions; this provides evidence that challenges the alternative explanation that more sophisticated technology provides better recommendations. Studies 3 and 4 demonstrate that the modality effect is in part driven by a need for closure that manifests as relative urgency: the ephemeral nature of auditory messages creates a window of time in which a response is expected. When visual recommendations were made ephemeral, the modality effect was attenuated (Study 3). In Study 4, participants in the auditory condition reported feeling a greater sense of urgency than participants in the visual condition, which was associated with increased recommendation adherence. In all our studies, we examined the case in which participants receive a recommendation and then make a choice among multiple options. For generalizability, in Study S5 (see the Supplemental Material), we examined the case in which participants receive a recommendation to purchase a product and then simply choose whether to buy it. We found that across three recommendation categories, participants who received auditory (vs. visual) recommendations were more likely to choose to purchase the product, b = 0.16, SE = 0.08, z = 2.09, p = .037, 95% CI = [0.01, 0.31].

An alternative explanation that we test indirectly in our studies is that people pay more attention to ephemeral stimuli than to permanent ones, process them more thoroughly (see Barnea et al., 2019), and as a result are more likely to adhere to ephemeral recommendations. However, note that in Study 1b, we found that handwritten recommendations were perceived as more novel—and thus ostensibly more attention-grabbing—than auditory-computer recommendations. Yet auditory recommendations still tended to be followed more.

In our studies, we were able to moderate the modality effect by making visual recommendations ephemeral. In theory, one should also be able to moderate the modality effect by making auditory recommendations permanent. Although auditory information by its very nature is ephemeral and there is no way to make purely auditory information equally permanent to visual information, adding a degree of permanence to auditory recommendations might attenuate the modality effect. We attempted to do this in a study (Study S6 in the Supplemental Material) in which the permanent visual text was presented simultaneously with the auditory recommendations. Thus, we included three conditions: auditory only, visual only, and auditory and visual. The auditory-and-visual condition consisted of the standard auditory manipulation with the added element of the permanent visual manipulation (participants heard and read the recommendations simultaneously). Although 60.7% of recommendations were followed in the auditory-only condition, 57.7% of recommendations were followed in the visual-only condition and 57.9% of recommendations were followed in the auditory-and-visual condition. A comparison between the auditory-only condition and the other two permanent conditions reveals that recommendations were significantly more likely to be adhered to when received only in the auditory mode rather than when they were accompanied by any element of (visual) permanence, b = 0.18, SE = 0.08, z = 2.13, p = .033, 95% CI = [0.01, 0.34].

Another possible explanation for increased adherence to auditory recommendations is that something fleeting is valued more than something permanent. Because people assign higher values to stimuli that are scarce (Lynn, 1991; Sharma & Alter, 2012), it is common to find promotion tactics that underscore the degree to which products and events are limited, either because of quantity or time restrictions (Kristofferson et al., 2017; Ku et al., 2012). However, it is unlikely that assigning greater value to the auditory recommendation would result in participants feeling more rushed to make their choices, as they reported feeling in Study 4 (and in Study S4 in the Supplemental Material). Furthermore, if value is associated with recommendation adherence, the direction of causality is unclear: That is, perceived value could lead to recommendation adherence, or recommendation adherence could lead to perceived value. Future research on value and its potential position in the causal chain is warranted.

The current research is related to ongoing work by Munz and Morwitz (2020) on the processing of auditory information. Munz and Morwitz suggest that receiving choice options in an auditory (vs. a visual) manner might make comparisons of options difficult and require more working memory, which could ultimately lead to greater recommendation adherence. This is in line with the notion that the transitory nature of auditory information increases cognitive difficulty for longer and more complex messages (Leahy & Sweller, 2011; Singh et al., 2012). In our studies, the paradigm in which we present recommendations is relatively short and easy to understand. Furthermore, we equate the way in which the options are presented to isolate the impact of recommendation modality. We show that even when the choice options are presented in the same modality and even when the information to be processed does not require significant cognitive load, the auditory nature of the recommendation increases adherence.

Although some studies we conducted included consequential choices (Study 2 and Supplemental 3), one potential limitation of our work is that some of our conclusions are drawn from studies involving hypothetical choices made by participants from online platforms. Although we attempted to test these findings on a variety of stimuli (e.g., hedonic and utilitarian, experiential and material), naturally we recognize that these stimuli are not representative of the full range of recommendation categories available, and it is possible that the specific features of the recommendation categories we presented our participants influenced their choices. Additionally, to isolate the effect of recommendation modality on recommendation adherence, we presented only textual information. However, in certain situations, people receiving recommendations might receive additional information about the product or service at hand (e.g., in the form of images). Future research might explore the effect of adding images to textual recommendations on adherence. Finally, our paradigm utilized a short time frame in which people were asked to make choices soon after they had been exposed to recommendations. Our studies do not speak to the length of time for which the impact of auditory recommendations persists, a question we leave for future research.

The modality effect we observed raises other possibilities for future research on the ways people might be impacted by the ephemerality of auditory information. For instance, even in contexts that do not involve recommendations, we found preliminary evidence that participants are more likely to incorporate information that was delivered aurally (vs. visually) into their evaluations, regardless of their perception of the veracity of the information (see Study S7 in the Supplemental Material). The sense of imminence of aural information might also affect perceptions of time. If ephemerality evokes a sense of urgency, perhaps people’s perceptions of elapsed time shift so that future events seem more imminent when described aurally.

Supplemental Material

sj-docx-1-pss-10.1177_09567976221106349 – Supplemental material for The Effect of Auditory and Visual Recommendations on Choice

Supplemental material, sj-docx-1-pss-10.1177_09567976221106349 for The Effect of Auditory and Visual Recommendations on Choice by Shwetha Mariadassou, Christopher J. Bechler and Jonathan Levav in Psychological Science

Footnotes

Transparency

Action Editor: Krishnankutty Sathian

Editor: Patricia J. Bauer

Author Contributions

S. Mariadassou and C. J. Bechler conceived and designed the experiments. S. Mariadassou collected the stimuli and data for the experiments and performed the data analysis under the supervision of C. J. Bechler and J. Levav. S. Mariadassou, C. J. Bechler, and J. Levav wrote the manuscript. All the authors approved the final version of the manuscript for submission.

ORCID iD

Jonathan Levav

Supplemental Material

Additional supporting information can be found at

Notes

References

Auer

(2009). On-line syntax: Thoughts on the temporality of spoken language. Language Sciences, 31(1), 1–13.

Barnea

Meyer

Nave

(2019). You only get one shot: How content ephemerality affects information processing and viewing behavior. In Bagchi

Block

Lee

(Eds.), Advances in consumer research (Vol. 47, pp. 448–449). Association for Consumer Research.

Bayer

J. B.

Ellison

N. B.

Schoenebeck

S. Y.

Falk

E. B.

(2016). Sharing the small moments: Ephemeral social interaction on Snapchat. Information, Communication & Society, 19(7), 956–977.

Berger

Iyengar

(2013). Communication channels and word of mouth: How the medium shapes the message. Journal of Consumer Research, 40(3), 567–579.

Canalys. (2018). Smart-speakers are the fastest-growing consumer tech; shipments to surpass 50 million in 2018. Canalys Newsroom. https://www.canalys.com/newsroom/smart-speakers-are-fastest-growing-consumer-tech-shipments-surpass-50-million-2018

Chaiken

Eagly

A. H.

(1983). Communication modality as a determinant of persuasion: The role of communicator salience. Journal of Personality and Social Psychology, 45(2), 241–256.

Flecha-Ortíz

Santos-Corrada

Dones-González

López-González

Vega

(2019). Millennials & Snapchat: Self-expression through its use and its influence on purchase motivation. Journal of Business Research, 125, 798–805.

Kristofferson

McFerran

Morales

A. C.

Dahl

D. W.

(2017). The dark side of scarcity promotions: How exposure to limited-quantity promotions can induce aggression. Journal of Consumer Research, 43(5), 683–706.

Kruglanski

A. W.

Webster

D. M.

(1996). Motivated closing of the mind: “Seizing” and “freezing.” Psychological Review, 103(2), 263–283.

10.

Kruglanski

A. W.

Webster

D. M.

Klem

(1993). Motivated resistance and openness to persuasion in the presence or absence of prior information. Journal of Personality and Social Psychology, 65(5), 861–876.

11.

H. H.

Kuo

C. C.

Kuo

T.-W.

(2012). The effect of scarcity on the purchase intentions of prevention and promotion motivated consumers. Psychology & Marketing, 29(8), 541–548. https://doi.org/10.1002/mar.20541

12.

Leahy

Sweller

(2011). Cognitive load theory, modality of presentation and the transient information effect. Applied Cognitive Psychology, 25(6), 943–951.

13.

Lynn

(1991). Scarcity effects on value: A quantitative review of the commodity theory literature. Psychology & Marketing, 8(1), 43–57. https://doi.org/10.1002/mar.4220080105

14.

Mayseless

Kruglanski

A. W.

(1987). What makes you so sure? Effects of epistemic motivations on judgmental confidence. Organizational Behavior and Human Decision Processes, 39(2), 162–183.

15.

Mehrabian

Wiener

(1967). Decoding of inconsistent communications. Journal of Personality and Social Psychology, 6(1), 109–114.

16.

Molla

(2018, November 12). Voice tech like Alexa and Siri hasn’t found its true calling yet: Inside the voice assistant “revolution.” Vox. https://www.vox.com/2018/11/12/17765390/voice-alexa-siri-assistant-amazon-echo-google-assistant

17.

Morlok

T. N.

Constantiou

Hess

(2018). Gone for better or for worse? Exploring the dual nature of ephemerality on social media platforms. Research Papers, 15. https://aisel.aisnet.org/ecis2018_rp/15

18.

Munz

K. P.

Morwitz

V. G.

(2020). Not-so easy listening: Roots and repercussions of auditory choice difficulty in voice commerce. In Argo

Lowrey

T. M.

Jensen Schau

(Eds.), Advances in consumer research (Vol. 48, pp. 1011–1016). Association for Consumer Research.

19.

Oberski

(2014). lavaan.survey: An R package for complex survey analysis of structural equation models. Journal of Statistical Software, 57(1). https://doi.org/10.18637/jss.v057.i01

20.

Roets

Van Hiel

(2007). Separating ability from need: Clarifying the dimensional structure of the Need for Closure Scale. Personality and Social Psychology Bulletin, 33(2), 266–280.

21.

Roets

Van Hiel

(2011). Item selection and validation of a brief, 15-item version of the Need for Closure Scale. Personality and Individual Differences, 50(1), 90–94.

22.

Schroeder

Epley

(2015). The sound of intellect: Speech reveals a thoughtful mind, increasing a job candidate’s appeal. Psychological Science, 26(6), 877–891.

23.

Schroeder

Epley

(2016). Mistaking minds and machines: How speech affects dehumanization and anthropomorphism. Journal of Experimental Psychology: General, 145(11), 1427–1437.

24.

Schroeder

Kardas

Epley

(2017). The humanizing voice: Speech reveals, and text conceals, a more thoughtful mind in the midst of disagreement. Psychological Science, 28(12), 1745–1762.

25.

Sharma

Alter

A. L.

(2012). Financial deprivation prompts consumers to seek scarce goods. Journal of Consumer Research, 39(3), 545–560.

26.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.

27.

Singh

A. M.

Marcus

Ayres

(2012). The transient information effect: Investigating the impact of segmentation on spoken and written text. Applied Cognitive Psychology, 26(6), 848–853.

28.

Swain

S. D.

Hanna

Abendroth

L. J.

(2006). How time restrictions work: The roles of urgency, anticipated regret, and deal evaluations. In Pechmann

Price

(Eds.), ACR North American Advances (Vol. 33, pp. 523–525). Association for Consumer Research.

29.

Van Zant

A. B.

Berger

. (2020). How the voice persuades. Journal of Personality and Social Psychology, 118(4), 661–682.

30.

Yzerbyt

Muller

Batailler

Judd

C. M.

(2018). New recommendations for testing indirect effects in mediational models: The need to report and test component paths. Journal of Personality and Social Psychology, 115(6), 929–943.

31.

Zhu

Yang

Hsee

C. K.

(2018). The mere urgency effect. Journal of Consumer Research, 45(3), 673–690.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

6.80 MB