YouTube as a Research Tool: Three Approaches

Abstract

The present paper provides empirical data to support the use of social media as research environment. YouTube was chosen as a most appropriate format to target adolescents in experimental and cross-sectional designs given its popularity as well as its plasticity. We uniquely applied the YouTube format as (a) an environment to present manipulated media materials in controlled experimental designs; (b) an environment to study effects of peer feedback on various media contents; (c) a format to design a media-based questionnaire, specifically, the Media, Morals and Youth Questionnaire (MMaYQue). Various studies have been conducted that demonstrate the appropriateness of our YouTube transformations for each of these three purposes. The focus in the present paper is on the methodology of these studies to illustrate how we effectively transformed YouTube as a research tool.

Introduction

Today's adolescents grow up in a media-saturated world and actually are the first generation growing up with continuous use of new media from childhood onwards.¹ To some, today's youngsters even ‘live in media’—expressing the omnipresent availability and habitual use of interactive media. Therefore, it is important to study the impact contemporary media fare may have on them. However, existing tools and research methodology are limited in addressing this specific target group, for example because most are text-driven, not available online, unnatural, and hard to manipulate for proper use in controlled experiments. Therefore, an up-to-date research environment that appeals to these media-savvy youngsters is needed.

The abundant outreach and high popularity of YouTube² inspired us to explore YouTube as a research environment. YouTube has more than 10 million unique users a month who are younger than 18 years of age.³ Thus, presenting materials for study purposes in a YouTube setting would provide an accommodating and convincing research setting for this age group.

A second reason for choosing YouTube was the dissemination of almost any kind of media fare. Thus, it is suitable for studying any kind of media content, whether it be antisocial or prosocial content, commercials, sad tears, or happy laughter. Furthermore, variations in presentation format are possible, ranging from still images and text only to rich audiovisual materials and movies.

A third important reason for selecting YouTube as the research environment was the ease with which it is possible to edit and manipulate material for controlled experimental designs. Such designs require presenting the same material to each respondent except for the factor under study. Furthermore, various segments or episodes of movies can be edited and uploaded as YouTube clips for further study.

Finally, because YouTube allows its users to post comments to the materials presented, which are publicly visible, “peer influence” can be studied in direct relation to the contents displayed. During adolescence, both media and peers have a strong influence.^4,5 Through YouTube, the combined effects of media and peer feedback can now be studied.⁶

In the following, we present several studies in which we transformed YouTube as a research environment in two separate sections. A first section focuses on YouTube to manipulate media materials for controlled experimental designs. Additionally, we manipulated YouTube's feature to post comments to study the effects of peer feedback. Then, we present a YouTube-based measurement device to assess adolescents' media preferences and moral judgment.

The purpose of the present paper is to focus on the methodology of several studies to demonstrate the effectiveness of our YouTube transformations. Each section below briefly describes the aim and design of the studies, explains how YouTube was transformed to meet the research purposes (visualized in Figures), and discusses the obtained results.

YouTube to Create Experimental Materials and Peer Feedback

Aim and study design

We developed materials in YouTube to examine how peer feedback on variously sized media models would affect adolescent girls' responses and body perceptions. Current society faces a paradox when it comes to weight issues: the media overwhelmingly portray thin females as highly attractive and successful role models,⁷ while overweight and obesity rapidly increase.⁸ Many studies^9,10 show that particularly adolescent females are negatively affected by the media's thin body ideal. Research further shows that adolescents who diet and undertake unhealthy weight-control behaviors are especially at risk for future overweight.^11,12 Therefore, it is important to study how detrimental effects of the media's thin body ideal can be counteracted in adolescents.

Previous research¹³ has shown that accurately informing adolescent girls about the actual (under)weight of the displayed models could redirect such undesired effects. However, in adolescent girls, peers are also very influential.^9,14–16 Therefore, the present research focused on possible counteracting effects from peers' feedback.

Study peer influence in combination with the thin body ideal requires a format in which peers can actually comment on the media models as presented. Thus far, peer influence has mainly been studied as an indirect manner through asking how much one responds to one's peers (e.g., using a perceived peer influence scale¹⁷). In this study, it was considered important to test the causal effects of peer feedback to variously sized media models on adolescent girls' body perceptions directly.⁶

Based on theoretical reasoning,^13,14 we expected that peers providing accurate weight information on (extremely) thin models would reduce negative body perceptions. Furthermore, when peers added “normal weight” information, adolescent girls may be led to believe that the presented (extremely) thin model is considered of “normal” shape, thereby increasing negative body perceptions.

To study causal effects, we systematically combined media models varying in three body shapes (i.e., extremely thin, thin, normal) with feedback from peers referring to the weight of the model in YouTube. The manipulation of peer feedback claimed the model to be either “6 kg underweight/extremely thin,” “3 kg underweight/thin,” or “normal weight.” Participants were 216 adolescent girls (M_age=14.15, SD_age=1.47), and they completed the questionnaires after exposure.

YouTube as a research tool—Media content and peer feedback

To create peer feedback, we mimicked “common” peer comments such as those that can be found in response to YouTube clips. We consistently varied peer comments in accordance with the experimental design: comments referring to the (under)weight status of the model (e.g., “yes, she for sure is 6 kilos underweight”) or just neutral (filler) comments (e.g., “tatidaiada, nice weather”). The peer feedback was presented to reflect the model's (under)weight status from an adolescent's perspective.

To create the materials, we borrowed pretested and digitally available media models.¹³ In accordance with girls' evaluations, media models were selected who significantly differed in looking “extremely thin,” “thin,” or “normally” shaped, while they did not differ in attractiveness.

The combinations of media models' body shapes and online peer comments covered the experimental conditions of our study. All models were named Lauren. Formulated in adolescents' jargon, comments were similar across conditions, apart from the weight-related comments. In accordance with YouTube conventions, the responding peers were given original names (e.g., “Roxxie”). Ten comments were created to provide feedback as if from peers, on each of the selected models.

All created materials, media models, and peer feedback were presented as an incorporated entity in YouTube. A screenshot of a full-color YouTube page, including a title, media model, and 10 comments providing feedback, was pasted in Adobe Photoshop. The screenshot further included both parts of a YouTube screen: the upper half and graphics part, as well as the accompanying written comments in the lower half. From there, we manipulated the materials such that all content was related to our research theme. Thus, the displayed media model, the side clips, and ads were all adjusted to show media models and related (neutral) text in addition to variations in the factors under study. The final results, looking like original full-color YouTube-screenshots, were included in an online questionnaire (Fig. 1).

FIG. 1.

Visual image of a YouTube setting to present study materials and to simulate peer feedback on media content.

Results and discussion

Given the paper's purpose, we will focus on the results as related to the effectiveness of YouTube as a research tool. Results showed that peer feedback on media models clearly guided the effects of thin body ideals in the perceptions of adolescents. For example, a multivariate analysis of variance (MANOVA) to compare differences between experimental groups who received different media-model by peer feedback combinations (see above) not only showed a significant main effect of peer comments (Wilks' Λ=0.939, F(6, 404)=2.14, p<0.05), but also a significant two-way interaction effect between peer comments and media models' body shape (Wilks' Λ=0.929, F(6, 404)=2.52, p<0.04. Subsequent simple effects analyses revealed that when an extremely thin media model was combined with peer feedback expressing that this model looks just “3 kg underweight,” this combination exerted significantly more body dissatisfaction than extremely thin media models with “normal weight” peer comments (p<0.05) or “6 kg underweight” peer comments (p=0.05). Although the results were partly different from what was expected,¹³ the YouTube setting was clearly effective.

The YouTube setting as designed in the present study made it possible to study this combined influence by integrating both the peer feedback and the media models into one set of visual materials. Applying the YouTube format with online peer feedback appeared an ecologically valid approach that worked well in our studies.

Future research may apply YouTube to a variety of research goals, in particular studying media effects in relation to peer feedback. Furthermore, YouTube is a highly accessible and appealing format to target adolescents. Importantly, YouTube makes it much easier than ever before to design AV-materials to be used in controlled experiments. Therefore, this format should be further tested in future research, including variations of feedback to various media images. Likewise, variations of media content could be included, such as various clips, sound, and any kind of media content relevant to the study. For example, setting peer norms through manipulating pro- versus antisocial peer comments in response to violent media content, or analyzing actual online peer conversations through automatic content analyses. Next, we will present another application of YouTube as a research tool in designing an appealing measurement device.

YouTube as a Measurement Device

Aim and study design

Given the popularity among youth of media content portraying antisocial and norm-crossing behavior such as unhealthy and risky behaviors (e.g., reckless driving, drugs abuse, bullying^5,18,19), there is a need for research on factors that might elicit adolescent preferences for antisocial, violent, and risky media content. Differential media content preferences may provide important insights in how specific media effects are established. A measurement device was needed to study adolescents' preferences and responses to particular media content, for which we used YouTube.

Both media exposure to morally adverse content and peer rejection are associated with negative developmental outcomes,^20,21 while it is unclear how media use among adolescents and peer rejection are related. In brief, we argued that adolescents who are rejected by peers are particularly prone to detrimental media effects in showing a preference for antisocial media content, and that rejection-based anger would impair adolescents' moral judgment of antisocial behavior.^22–24 Therefore, in a between-participants experimental design, we manipulated peer-acceptance versus rejection in adolescents (M_age=13.88; n=74) and young adults (M_age=21.37; n=75) through the Cyberball paradigm.^2,25 Thereafter, participants completed a paper-and-pencil questionnaire and the newly devised Media, Morals and Youth Questionnaire (MMaYQue) to assess media preferences and moral judgment. YouTube served as an appropriate platform.

YouTube as a research tool—Measurement device

In testing causal relations, existing measurement tools fall short. Often, media use or preference is measured by asking participants how much time they spend on a certain medium or a certain genre, or to list their favorite programs, movies, or video games. These tools are not appropriate to study what precedes media preference or what the underlying mechanisms are through which media preferences are formed. Using the newly devised MMaYQue makes it possible to measure media preference as a consequence of preceding events, moods, and situational or social influences.²³

In constructing the MMaYQue, we collected episodes of commonly portrayed media content on YouTube. Popular clips were observed and rated in terms of portraying antisocial and prosocial behavior. Our observation scheme was based on antisocial and risk behavior categories²⁶ and the popularity of clips (i.e., # of hits, likings, reviews). Hence, we selected a large pool of clips and described each in only a few lines. We pretested the descriptions of the clips on likeability and “want to see this clip” in order to make a selection of equally attractive yet clearly distinguishable episodes of anti- versus prosocial media content.

The inital MMaYQue contained 22 descriptions of highly popular YouTube clips. Fourteen descriptions of anti-social behaviors were based on four domains of risk-taking behavior: violence/aggression, sexual harassment/sexual self-exposure, substance abuse/binge drinking, and reckless driving.²⁶ A sample item is: “Youngsters scolding at a police officer and pushing him off his motorbike.” Eight items described prosocial/neutral media content, and were added as filler items (e.g., “Cat playing a song on the piano”). Participants indicated on 5-point scales how much they would like to watch the clip (1=“not at all”, 5=“very much”).

The same materials were used to measure moral judgment of media content; that is, participants indicated how acceptable and how “ok” he/she judged the behavior in the clip to be on two 5-point rating scales.

We designed two formats of the MMaYQue. In one, all content descriptions were formatted as in the original YouTube layout, based on screenshots from YouTube clips and edited in Photoshop. The results were then presented as if they were original YouTube clips in an online questionnaire (Fig. 2). A second format, also applicable in cross-sectional designs, was created in a more condensed format with fewer graphics (Fig. 3), which is more useful, and less costly, for paper-and-pencil questionnaires. Content wise, the items as construed in the MMaYQue may also be used in different formats.

FIG. 2.

Visual image of a YouTube setting designed as a measurement device (MMaYQue)

FIG. 3.

Example of items using visual images of YouTube designed as a measurement device (MMaYQue).

Psychometric qualities of MMaYQue showed the intended two factors and good reliabilities, with Cronbach's alphas ranging from 0.85 to 0.94 and 0.73 for preference for antisocial and neutral/social media content, respectively, throughout the various studies. The items measuring moral judgment also formed separate and reliable scales for antisocial and neutral/social media content, with Cronbach's alphas ranging from 0.74 to 0.93 and 0.89 respectively.

Results and discussion

Results of bootstrapping analyses²⁷ showed the expected double mediation in adolescents. Higher levels of state anger in peer-rejected adolescents (b=−0.49, p<0.05) induced more tolerable moral judgments of antisocial media content (b=0.27, p<0.01), subsequently instigating a preference for antisocial media content (b=0.82, p<0.01). Bootstrap estimates showed a significant indirect effect of peer rejection on antisocial media preferences, via anger and moral judgment, point estimate=−0.11, 95% CI [−0.25; −0.01], supporting a full double mediation model in the adolescent sample. The young adult sample showed no relation between peer rejection and preferences for antisocial media content.

Additional validation studies with various samples showed that the MMaYQue consistently showed a good internal structure and good criterion and discriminant validity.²⁸ For example, in line with the extant literature,^29–31 adolescents high on trait aggressiveness and sensation seeking were also high on preferences for antisocial media content (r_aggression=0.48, p<0.01; r_{sensation seeking}=0.40, p<0.01) and showed a more lenient moral judgment of antisocial media content (r_aggression=0.44, p<0.01; r_{sensation seeking}=0.29, p<0.01). In contrast, trait empathy was negatively related to antisocial media content (r=−0.38, p<0.01), and adolescents high on empathy judged antisocial media content as less tolerable (r=−0.35, p<0.01).

Given the results, the MMaYQue seems a promising YouTube-based research tool. Little is known thus far about who prefers which kind of media content under which circumstances or about how media content is evaluated in terms of moral values. The MMaYQue may therefore play a relevant role in future research.

Future research may make more sophisticated use of the MMaYQue by including colorful content-related images in the graphics-screen part, or even moving images, to increase further the appeal to the adolescent target group. Caution is directed at possible confounds in view of the measurements. Furthermore, the MMaYQue may include peer feedback to study peer influence on adolescents' judgment of media content. This may be further developed into a Wizard-of-Oz-like interactive environment,³² to increase ecological validity further and study instantaneous feedback in processing mediated interactions. The wide applicability and effectiveness of the MMaYQue demonstrates the value of YouTube as research tool.

General Discussion

This paper focused on transforming YouTube into a research tool in various settings. YouTube appeared a most appropriate format to target adolescents in experimental and cross-sectional designs. We have presented several studies in which YouTube served as a research setting for manipulated media content in experimental designs. Additionally, we simulated peer feedback as if it was real (i.e., looked like original YouTube sites) to study peer influence in adolescents' responses to specific media content. Finally, studies showed that YouTube-based measurement devices were effective in assessing differential preferences for antisocial media content and moral judgments.

The YouTube setting made it possible to investigate systematically the combined influence of immediate peer feedback and a media model's weight status on adolescent girls' responses and body perceptions in experimental designs. By “mimicking” YouTube's feedback option with ostensible peer feedback, we created a convincing way to study peer influence systematically. The graphic and textual nature of YouTube, as well as its plasticity, allows the exertion of extensive control over both the media images and peer comments. The promising results open up new horizons for experimental research into the combined influence of media and peers, for example how peer norms in response to violent media content may affect moral reasoning.

The YouTube setting provides an ecologically valid research environment, as it is a media channel through which people can actively deliver comments on media content directly visible to others. Research materials delivered through YouTube have a realistic feel because they are rich in their presentation like common YouTube sites. Using nicknames and jargon that conform to YouTube conventions strengthens this. We believe that this format makes the experimental manipulations less obvious.

YouTube's characteristics provide many opportunities for further research, not only the effects of varying contents but also of varying senders of information. For example, variations in expert and peer senders could be studied, as well as different subcultures (e.g., by using different nicknames and slang). Furthermore, varying the side clips and other features of YouTube sites would allow the study of the effects of context and framing on message processing. Future research may further include multiple-exposure designs because comments on YouTube proceed in time. Research efforts may also be undertaken by analyzing demographic indicators for those who are more active YouTube users than others, and relate this information to breaking down data and analyze differential effects on the outcome. The format could also be extrapolated to contemporary (social) media settings such as Twitter or Facebook. Interactive research settings may also be created in YouTube with real-time peer comments from actual peers such as classmates responding to each other within a further controlled setting. The various adjustment options in content, format, and sender information makes YouTube highly useful for a broad range of research topics, for example studying cyberbullying, collective social and political uprising, viral marketing, and so on.

Promising implications can also be derived from the YouTube-based measurement device MMaYQue with which we examined adolescents' preferences for antisocial media content and a morally lenient attitude as a function of being rejected by peers. Not only does the MMaYQue itself appear to be a promising YouTube-based research tool, but similar or extended measurement devices may also be construed to advance the field of media research. MMaYQue contributes to the media research methodology in that no widely applicable measurement devices were yet available to assess media preferences or moral judgment in experimental designs. Most media-based research thus far studied media exposure in a rather global sense through frequency of exposure to media, such as hours of watching television or playing video games.^33,34 Studies more specific mainly focused on violent media content asking how often one played/watched violent games/movies, or to name favorite games/shows.^35,36 Such measurements have been useful in past research but fall short in the current media landscape. Almost any content is available anywhere (e.g., same movie on TV, Internet, smartphones). Hence, tools are needed to assess exposure to and preferences for specific content. Our YouTube-based MMaYQue is a step in that direction, and may inspire researchers in the field. We focused on pro-versus antisocial content, while others might be interested in other content, for example humorous versus nonhumorous commercials.

In conclusion, the various studies presented in this paper demonstrate the appropriateness of YouTube transformations for the purpose of creating media materials as experimental stimuli and studying the effects of peer feedback, as well as constructing a YouTube-based measurement device. In these studies, we successfully transformed YouTube into a research tool with promising perspectives for advancing the field of media research for both experimental and cross-sectional designs.

Footnotes

Acknowledgments

We would like to thank Rianne van der Veen and Mayke Huiberts for their support and inspiration in developing and pre-testing the materials. Furthermore, the schools and adolescents who participated in our studies are highly acknowledged.

Author Disclosure Statement

No competing financial interests exist.

References

Rideout

, Foehr

, Roberts

. 2010. Generation M2: media in the Lives of 8- to 18-year-olds. Menlo Park: Henry J. Kaiser Family Foundation.

Madden

. 2009. The audience for online video sharing sites shoots up. Pew Internet and American life project. http://fe01.pewinternet.org/Reports/2009/13–The-Audience-for-Online-VideoSharing-Sites-Shoots-Up.aspx. 2010 Jun 20.

YouTube Fact Sheet. 2007. www.youtube.com/t/fact_sheet. 2007 Dec 12.

Laible

. Attachment with parents and peers in late adolescence: links with emotional competence and social behavior. Personality & Individual Differences, 2007; 43:1185–97.

Strasburger

, Jordan

, Donnerstein

. Health effects of media on children and adolescents. Pediatrics, 2010; 125:756–67.

Veldhuis

, Konijn

, Seidell

. Counteracting media's thin-body ideal for adolescent girls: informing is more effective than warning. Media Psychology, 2013in press.

Bergstrom

, Neighbors

. Body image disturbance and the social norms approach: an integrative review of the literature. Journal of Social & Clinical Psychology, 2006; 25:975–1000.

World Health Organization. 2012. Obesity and overweightFact sheet No. 311www.who.int/mediacentre/factsheets/fs311/en/. 2012 Jun 29.

Grabe

, Ward

, Hyde

. The role of the media in body image concerns among women: a meta-analysis of experimental and correlational studies. Psychological Bulletin, 2008; 134:460–76.

10.

Groesz

, Levine

, Murnen

. The effect of experimental presentation of thin media images on body satisfaction: a meta-analytic review. International Journal of Eating Disorders, 2002; 31:1–16.

11.

Neumark-Sztainer

, Wall

, Guo

et al.

Obesity, disordered eating, and eating disorders in a longitudinal study of adolescents: how do dieters fare 5 years later?

Journal of the American Dietetic Association, 2006; 106:559–68.

12.

Neumark-Sztainer

, Wall

, Haines

et al. Why does dieting predict weight gain in adolescents? Findings from Project EAT-II: a 5-year longitudinal study. Journal of the American Dietetic Association, 2007; 107:448–55.

13.

Veldhuis

, Konijn

, Seidell

. Weight information labels on media models reduce body dissatisfaction in adolescent girls. Journal of Adolescent Health, 2012; 50:600–6.

14.

Dohnt

, Tiggemann

. Body image concerns in young girls: the role of peers and media prior to adolescence. Journal of Youth & Adolescence, 2006; 35:141–51.

15.

Dohnt

, Tiggemann

. The contribution of peer and media influences to the development of body satisfaction and self-esteem in young girls: a prospective study. Developmental Psychology, 2006; 42:929–36.

16.

Jones

, Vigfusdottir

, Lee

. Body image and appearance culture among adolescent girls and boys: an examination of friend conversations, peer criticism, appearance magazines, and the internalization of appearance ideals. Journal of Adolescent Research, 2004; 19:323–39.

17.

Keery

, van den Berg

, Thompson

. An evaluation of the Tripartite Influence Model of body dissatisfaction and eating disturbance with adolescent girls. Body Image, 2004; 1:237–51.

18.

Brown

, Witherspoon

. The mass media and American adolescents' health. Journal of Adolescent Health, 2002; 31,Suppl:153–70.

19.

Funk

, Buchman

. Playing violent video and computer games and adolescent self-concept. Journal of Communication, 1996; 46:19–32.

20.

Juvonen

, Graham

, Schuster

. Bullying among young adolescents: the strong, the weak, and the troubled. Pediatrics, 2003; 112:1231–37.

21.

Strasburger

. Why do adolescent health researcher ignore the impact of the media? Journal of Adolescent Health, 2009; 44:203–5.

22.

Eisenberg

. Emotion, regulation, and moral development. Annual Review of Psychology, 2000; 51:665–97.

23.

Plaisier

, Konijn

. Rejected by peers—attracted to antisocial media content: rejection-based anger impairs moral judgment among adolescents. Developmental Psychology, 2012[Epub ahead of print]10.1037/a0029399.

24.

Ugazio

, Lamm

, Singer

. The role of emotions for moral judgments depends on the type of emotion and moral scenario. Emotion, 2012; 12:579–590.

25.

Williams

, Cheung

, Choi

. Cyberostracism: effects of being ignored over the Internet. Journal of Personality & Social Psychology, 2000; 79:748–62.

26.

Gardner

, Steinberg

. Peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: an experimental study. Developmental Psychology, 2005; 41:625–35.

27.

Preacher

, Hayes

. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 2004; 36:717–31.

28.

Plaisier

, Konijn

. Constructing and validating a scale to assess moral judgment of antisocial media content and media preferences among youth. Paper presented at the Etmaal-Conference Communication Science, February 7–8, 2013, Rotterdam, The Netherlands. Erasmus University.

29.

Bushman

. Moderating role of trait aggression in the effects of violent media on aggression. Journal of Personality & Social Psychology, 1995; 69:950–60.

30.

Pizarro

. Nothing more than feelings? The role of emotions in moral judgment. Journal for the Theory of Social Behaviour, 2000; 30:355–75.

31.

Slater

, Henry

, Swaim

et al. Vulnerable teens, vulnerable times: how sensation seeking, alienation, and victimization moderate the violent media content–aggressiveness relation. Communication Research, 2004; 31:642–68.

32.

Akers

. 2006. Wizard of Oz for participatory design: inventing a gestural interface for 3D selection of neural pathway estimates. CHI ’06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22–27, 2006. New York: CHI ’06. ACM Press, 454–9.

33.

Schechter

, Gross

, Willheim

et al.

Is maternal PTSD associated with greater exposure of very young children to violent media?

Journal of Traumatic Stress, 2009; 22:658–62.

34.

Möller

, Krahé

. Exposure to violent video games and aggression in German adolescents: a longitudinal analysis. Aggressive Behavior, 2009; 35:75–89.

35.

Anderson

, Sakamoto

, Gentile

et al. Longitudinal effects of violent video games on aggression in Japan and the United States. Pediatrics, 2008; 122:1067–72.

36.

Boxer

, Huesmann

, Bushman

et al. The role of violent media preference in cumulative developmental risk for violence and general aggression. Journal of Youth & Adolescence, 2008; 38:417–428.