Picture me in person: Personalization and emotionalization as political campaign strategies on social media in the German federal election period 2021

(Visual) campaign strategies on different social media platforms

Research has shown that political campaign strategies are influenced by “a platform’s network structure, functionality, algorithmic filtering, and data processing” (Bossetta, 2018: 471), thus attuned to platform characteristics (Haßler et al., 2023). Focusing on visual campaigning, Farkas and Bene (2021) accordingly found that campaign strategies differed between Facebook and Instagram. Although such results underline that inferences about political communication on social media clearly benefit from examining multiple platforms, Gerodimos (2019) demonstrated that most studies on visual political communication focus on a single social media platform, while only few studies take a cross-platform approach (Bossetta, 2018; Farkas and Bene, 2021). Yet, as each platform has developed its own logic (Haßler et al., 2023) and even “culture” (Gerodimos, 2019: 63), which fosters different strategies of personalization and emotionalization, examining multiple platforms seems key to identify overarching patterns in visual political content. In our study, we therefore follow a comparative perspective and look at Facebook and Instagram.

Multimodal visual–text interactions in social media posts

As many other media, social media has seen a shift to visual communication, with visual-centric platforms such as Instagram rising in popularity, and formerly text-based platforms such as Facebook incorporating more options to use visuals. Politicians have adapted to this change and their visual communication has attracted research interest (Grusell and Nord, 2023; Russmann et al., 2019). This is noteworthy because it has long been complained that social media analyses have been primarily text-based (Meeks, 2017), while images have been seen primarily as complements to verbal/textual information that help viewers to build trust and legitimacy, to achieve coherence and create meaning more quickly (Russmann et al., 2019). However, multimodal communication, “with the visual as an especially important mode,” has long played a “dominant role in public communication” (Kress and Van Leeuwen, 2020: 3). And along the line, research has shown that images are not just decorative accessories and “amplifiers” of textual communication, but are culturally meaning-making and meaningful practices providing intimate connections that can serve as important aspects of identification, identity representation, and self-expression (Andreallo, 2022: 29, 31). In addition, visuals can activate image-specific cognitive and affective processes (Geise and Baden, 2015) that can further shape political thinking and acting (Geise et al., 2021; Powell et al., 2015; Sülflow and Maurer, 2019). Recent research therefore takes more account of the multimodality of digital communication, described as “the communicative interaction of meaning coded in different modalities (e.g., sound, image, text)” in a certain media context (Geise and Baden, 2015: 4). From this perspective, visuals and texts are two sovereign communication modalities that constantly interact with each other during reception (Powell et al., 2015). This includes the option that they complement and reinforce each other’s message, but also that both offer contradictory information—in a multimodal post, “which uses images and writing, the writing may carry one set of meanings and the images another” (Kress and Van Leeuwen, 2020: 41). In our study, we therefore inspect the consistency between the textual and visual personalization embedded in the multimodal posts, treating them as (1) a composition in which “all elements are spatially co-present” (Kress and Van Leeuwen, 2020: 182) and as (2) politically relevant communicative artifacts that “simultaneously fulfil the function of representing” the political subject, and of “enacting certain social relations between the maker and the potential audience” (Zhao and Zappavigna, 2018: 1737).

Personalization of politics in social media

Most scholars see personalization as a complexity-reducing strategy that offers voters a choice between different political personalities rather than complex problem-solving approaches (Raupp, 2021). From a theoretical perspective, personalization is the result of long-term social and political transformation processes that span individualization and mediatization. With Bennett (2012), we see the increasing presence of personalized politics as part of a broader social transition that emphasizes the achievements of personal lifestyles, while at the same time being accompanied by a disengagement from traditional social structures, with which party allegiances fade and must be renegotiated from election to election. Following Mazzoleni (2008), this tendency is simultaneously enabled and promoted by the ongoing mediatization of politics: Along with the differentiation of media offerings, the reach and dynamics of journalistic and social media, mediatization entails that politicians increasingly have to meet the requirements of media logic to gain media attention at all, so that political communication increasingly focuses on politicians’ personal characteristics (Esser and Strömbäck, 2014; Mazzoleni, 2008). Social media thus not only create an arena for the ongoing construction of political actors’ personal identities (Enli and Skogerbø, 2013), they also provide voters with a forum for individual policy centricity.

Research on personalization in political communication can be divided in the following three subfields: studies on (1) the media coverage of politicians, (2) effects on recipients, and (3) the party’s and politician’s self-presentation (Raupp, 2021). Our study focuses on the latter. We understand personalization as a strategic way for candidates to foreground themselves as persons instead of issues in order to satisfy media logic and public demands—and, if necessary, to decouple themselves from their party to some extent. In that sense, self-personalized posts and particularly photographic self-portrayals do not only provide a medium for candidates “to represent themselves as ‘objects of seeing’”; by enacting intersubjectivity and by construing perspectives they also help articulate the candidates own perspectives on political events, issues, and agendas (Zhao and Zappavigna, 2018: 1749). The detachment from strategic party decisions can be especially important for local runners who are not the focus of the official campaigns.

Recent studies show that politicians make massive use of the direct route of personalized self-presentation on social media (Larsson, 2017). In addition, researchers have shown that personalization is not a uniform communication strategy, but a phenomenon that can be differentiated into various dimensions or sub-strategies that encompass different types, such as professional personalization, individualization, privatization (Grusell and Nord, 2023; Metz et al., 2020), and intimization (Graham et al., 2018). Accordingly, Hermans and Vergeer (2013) examined 738 candidate-websites during European Parliament elections and observed three different dimensions of personalization across 17 countries, namely “professional,” “home and family,” and “personal preferences.” Whereas the sub-strategy professional personalization uses the candidate’s career background to display his or her competence, home and family concentrates on the candidate’s private life, and the focus of the rarely used personal preference dimension is on the candidate’s personal interests and private sphere. Building on these findings, Larsson (2017) analyzed Instagram accounts of Norwegian parties and party leaders, showing that the identified sub-strategies are also used in social media. Metz et al. (2020) as well as Peng (2021) identified similar subtypes of personalization on social media, namely “professional,” “private,” and “emotional” political personalization. Here again, professional personalization addresses instances in which individual qualities and experiences are related to the political role the candidate seeks, while private personalization, which subsumes the formerly described “home and family” and “personal preferences,” delivers intimate information of the politician’s private life. As third sub-strategy, the authors observed an emotional personalization, putting an emphasis on the politician’s feelings and emotive expressions.

However, as these and other studies suggest, candidates rarely focus on privatizing their content, but prefer a professional personalization that reminds audiences of traditional media campaigns (Carlson and Håkansson, 2022; Peng, 2021). Research also shows that personalized political posts commonly embed media images (Jungblut and Haim, 2023), which are considered predestined agents of self-personalization (Metz et al., 2020), increase audience engagement and identification (Grusell and Nord, 2023). Particularly the use of (staged) “selfies,” common in political self-representation (Bast, 2021), may appear as “micro-autobiography of the present moment” (Schleser, 2014: 155), suggesting authenticity, and creating forms of “liveness” and expressing “visual co-presence” that bring the audience closer to the situation being represented (Zappavigna, 2016; Zhao and Zappavigna, 2018: 1739).

Nonetheless, even though some researchers have turned more attention to the visual aspects of political social media communication in the US context (Casas and Williams, 2019) and the Scandinavian countries (Filimonov et al., 2016; Larsson, 2017; Russmann et al., 2019) in recent years, the self-personalization of political personalities has mainly been studied using text-based social media platforms as examples (Raupp, 2021). Especially for Germany, where personalization has always played a major role in political campaigning, relatively few studies examine the role of images in multimodal election campaigns online (Haßler et al., 2023). Our first study therefore aims to answer the following question:

Emotionalization of politics in social media

Personalization often is accompanied by emotionalization, a communicative strategy in which candidates prioritize expressions of emotion and affect more than political debates and issues (Bronstein et al., 2018). Emotionalization has been used successfully in traditional election campaigns, but mostly understood as an overarching scheme of the parties, not as part of the self-personalization strategy of the candidates themselves (Raupp, 2021). Because emotion-based messages capture audience attention (Bronstein et al., 2018), lead to more positive voter evaluations (Gabriel and Masch, 2017; Sülflow and Maurer, 2019), foster social connections and identification (Andreallo, 2022), increase audience engagement, and even support voter mobilization (Bossetta and Schmøkel, 2023), such an “emotional personalization” (Metz et al., 2020: 1483) has also been shown to be effective as part of an individual personalization strategy (Peng, 2021).

Emotions can be transferred through different modalities, such as text, images, or sound. Yet, images—particularly photographs—are considered the most effective way to communicate emotions (Casas and Williams, 2019). While social media images invite “ambient viewers to approach the image as if they were sharing in the photographer’s subjective experience” (Zappavigna, 2016: 289), recipients even tend to emotionally “mirror” the affects shown in visuals, a process known as emotional contagion (Gabriel and Masch, 2017). As such images have “the power to make us laugh, cry or simply recall,” and their sharing “connects people beyond words” (Andreallo, 2022: 3), the visual representation of emotions is not only an effective strategy to emphasize a political point of view, but can also lead to a further diffusion of these emotions in audiences (Gabriel and Masch, 2017).

While campaign teams seem aware of this power of imagery (Sülflow and Maurer, 2019), studying the visual display of emotions on social media is extremely time- and resource-intensive. Established and reliable coding schemes (such as the Facial Action Coding System [FACS]) require a lot of training and time (Ekman and Rosenberg, 2005). Studies therefore often examine extreme poles of emotion (e.g. intense joy, great anger) while less intense emotions have so far widely gone unnoticed (Casas and Williams, 2019). To date, however, only a few studies have examined the emotion display of politicians in Germany and the European Union (Carlson and Håkansson, 2022; Gabriel and Masch, 2017) and even fewer studies address self-representation through social media (Jungblut and Haim, 2023; Metz et al., 2020). Extending this line of research, we further investigate the process of emotionalization in the context of personalization on social media and ask the following:

Gender stereotypes in political communication on social media

Examining typical, personalized portrayals of politicians in traditional media, research has identified significant differences between male and female candidates. Female politicians are, for example, often portrayed in their role as mothers and caregivers, which can diminish their suitability as competent candidates (Andrich and Domahidi, 2023; Bauer, 2022). “Designated ‘female’ traits” such as “passive, dependent, non-competitive, gentle, weak leader, emotional, compassionate, kind, honest, warm, attractive, honest, altruistic” (Aaldering and Van Der Pas, 2018: 915) seem to imply that women cannot handle the competitive political system (Andrich and Domahidi, 2023). The representation of such gender stereotypes in traditional media has been explained by structures across media organizations, routines in journalistic reporting as well as potential cognitive biases of newscasters (Bauer, 2022).

While candidates are actively involved in the initial presentation and sharing on their accounts (Zappavigna, 2016; Zhao and Zappavigna, 2018), the act of independently creating representational images “credits more agency to the person in the photograph” as he or she is—in many cases—the portrayed, the photographer, and the publisher (Andreallo, 2022: 17). Because this transformation disrupts traditional power relations (Andreallo, 2022; Zhao and Zappavigna, 2018), the representation of politicians on social media was expected to be less characterized by gender-stereotypical representations. But gender differences arise on social media, too: McGregor et al. (2017) as well as Meeks (2017) found that female politicians use personalization, emotionalization, and intimization online less than their male colleagues. Because female candidates are judged by different evaluative standards than male opponents (Andrich and Domahidi, 2023), this strategy can be wise, whereas running overly personalized and emotionalized campaigns tends to be a risky choice (Raupp, 2021). In this respect, socially established stereotypes of women also limit the possibilities of female representation on social media: If female politicians do not submit to representational expectations, they seem to risk being seen as “cold,” “bossy,” or “unsympathetic” (Andrich and Domahidi, 2023).

Nonetheless, Tsichla et al. (2023) observed that female candidates used more expressive images and more emoticons in their accompanying text-based Facebook communications than male candidates. As Carlson and Håkansson (2022) as well as Jungblut and Haim (2023) showed, female European politicians were also more likely to present themselves smiling/happy in campaign pictures than their male counterparts. Stronger emotionalized communication among female candidates was also observed by Boussalis et al. (2022) in self-posted images of US candidates, who, regardless of party affiliation, were more likely to demonstrate joy, but less anger, as well as fewer neutral expressions, than men. In order to address RQ3 and based on the findings of Carlson and Håkansson (2022), Jungblut and Haim (2023) as well as Boussalis et al. (2022) we assume the following:

Research design

We conducted two studies. Attempting to answer RQ1, study 1 employs a qualitative content analysis of the multimodal postings, combining a picture type analysis with a qualitative text analysis. Study 2 analyzes which role emotions play within personalization, applying an automated content analysis of the embedded visuals using deep learning procedures and a manual text analysis. We explicitly included the analysis of both, textual and visual information, since we consider social media postings as an interplay of textual and visual content, with both components creating meaning and mutually contextualizing one another (Graber, 1989; Kress and Van Leeuwen, 2020).

Data for both studies were collected during the German 2021 federal election by two coders through a systematic screening process. The coders extracted data from Instagram and Facebook on 3 days of each week (Monday, Wednesday, Friday) from 14 June to 26 September. Using a hashtag and keyword search, they selected the 10 most recent posts for each platform on each of the three most important campaign topics (Corona, climate change, the campaign itself; Infratest dimap, 2021) and then took a screenshot of the posts using the Evernote web clipper for further coding.

Study 1: qualitative content analysis of multimodal posts

The resulting set of posts (n_total = 1959; n_Instagram = 1032, n_Facebook = 927) is the source for this analysis (Figure 1). In accordance with our research questions, however, we are primarily interested in personalized candidate posts. We thus further analyzed only those 584 posts from the full sample that (1) were published by the candidates themselves on their public channels (n_Instagram = 272, n_Facebook = 312), and that (2) depicted the candidates in a personalized way, finally limiting our sample to 401 postings (n_Instagram = 166, n_Facebook = 235). Figure 1 illustrates this multi-layered sampling strategy.

OPEN IN VIEWER

Figure 1.

Data selection and exclusion.

All images that depicted some form of personalization by the candidates were further analyzed applying a qualitative content analysis. In order to differentiate personalization strategies on social media, we conducted a qualitative image type and text analysis. The applied image type analysis, as suggested by Grittmann and Ammann (2011), takes into account that the meanings of an image insist on the contextual understanding and relationships displayed by the individual visual elements and their configuration to each other (Kress and Van Leeuwen, 2020). The basic idea of image type analysis is the classification of visual content according to its motives (and their inherent meanings) into certain image types; it is based on the iconographical insight that visual representations of politicians follow certain archetypes (Bucy and Grabe, 2009).

We accordingly started with previously observed subtypes derived from research literature (e.g. private vs. formal personalization; Metz et al., 2020), as such “archetypes” provide a framework for the classification of the visual material according to already observed categories. However, because already established image types do not necessarily reflect selection and presentation routines of political self-presentation on social media, we aimed to further discriminate forms of personalization in multimodal postings. Our qualitative analysis therefore follows a directed approach in which existing categories are further refined inductively (see Mayring, 2000).

Sample 1

The sample consists of all postings by candidates, which feature images depicting the candidate in a personalized way. From 584 postings of candidates on Facebook and Instagram nearly 70% depicted the candidate in a personalized way (n = 401). In total, 235 were published on Facebook and 166 on Instagram. This sample size is comparable to existing studies analyzing personalization on social media (e.g. Metz et al., 2020, n = 435).

Measures and coding process in Study 1

Drawing on existing research, we began by distinguishing between “formal,” “private,” and “intimate” forms of personalization. For formal personalization, we adopted the categorization by Farkas and Bene (2021), according to which the candidates present themselves in their working context, often with formal clothing. Following Peng (2021), we defined the depiction of politicians in non-political settings such as bars, stores, or nature as private personalization. With Larsson (2017), we consider representations of politicians with their close family to be intimate personalization; as an extension, we also included depictions of illness and/or death in this category.

When analyzing the material, the coders examined whether assignment to these deductively derived forms of self-personalization was feasible; if not, a new category was inductively formed. The coders met in several meetings, discussed and adopted revisions as a formative check of reliability (Mayring, 2000). While the private and intimate forms of personalization could be subsumed under the already established categories, the representation with regard to formal strategies was further differentiated between “professional” and four additional subcategories: “life on stage,” “interaction/mobilization,” “in action,” and “behind the scenes” (see results for more explanations of the categories). In addition to the visual and textual personalization strategies, the candidates’ party affiliation and their channel of communication were also coded.

Results of Study 1

The qualitative content analysis led to the extraction of seven image and text personalization strategies, whereby five of these seven strategies can be assigned to the formal spectrum and two reflect a private personalization (visual examples: Appendix Figure 7). The ratio of formal versus private personalization indicates that a private style—both in image and text—was the exception for the 2021 German federal election. In line with previous studies for other national contexts (Carlson and Håkansson, 2022; Metz et al., 2020; Peng, 2021) candidates mainly relied on a formal type of personalization.

According to prior research, we differentiated the “private spectrum” into intimate and private personalization. Private-personalized images (n = 50) are characterized by the explicit depiction of politicians in their very private surroundings or on highly private occasions. Intimate visualizations (n = 4) even go one step further and show close insights, for example by depicting the candidates’ family, their children, or health issues. On the textual level, intimate postings (n = 4) convey insights into health or family life (e.g. “I took my son to the acropolis in Athens”), whereas private text postings (n = 49) focus on the individual politician’s views, values and personality (e.g. “As a young candidate from rural areas, I campaign especially for . . .”).

Considering formal strategies, we distinguished the following five subtypes: professional, life on stage, interaction/mobilization, in action, and behind the scenes (see Appendix Figure 8). Images within the professional category (n = 139) depict the candidates in a formal setting—usually showing an image of the face or the upper body half in formal attire. They typically contain the campaign slogan and a clear campaign branding, thus strongly resembling official election posters. The accompanying professional textual postings (n = 70) often present the candidate as part of the campaign and their party (e.g. “We the [party] stand for . . .”).

Candidates’ life on stage utilizes images (n = 34) that present them in large campaign arenas. These images usually have a documentary but idealizing character, and often convey a certain distance to the viewer. The textual personalization “on stage” (n = 19) oftentimes supports the visual depiction, by either stressing how many people attended the event (e.g. “over 1000 citizens did not want to miss [the candidate]”) or thanking for the inspiring dialogue (e.g. “thanks for the invitation to speak”).

Closely intertwined are the categories in action and interaction/mobilization. In-action images (n = 72) show candidates in active acts of campaigning, while “interaction/mobilization” visuals (n = 39) depict the candidates in dialogue with voting citizens. Correspondingly, “in action” texts (n = 75) describe election campaign actions (e.g. “today my team canvassed”), whereas textual “interaction/mobilization” posts (n = 90) formulate invitations, activating citizens to further interact with the politicians (e.g. “come discuss with us . . .”).

The last, and maybe most interesting, discovered formal personalization strategy is behind the scenes. With these images (n = 77), candidates offer otherwise unseen insights into political work, for example, through backstage pictures of photo shoots or after political speeches. Such images are particularly thought-provoking because, at first glance, they seem like private snapshots that one would rather share with close acquaintances. Yet, when considering the accompanying textual “behind the scenes” posts (n = 62)—which usually provide background on the event depicted (e.g. “we shot my campaign videos”)—it becomes clear that these are strategic occasions. Nevertheless, recipients could get the impression that considerably more pictures reflect the candidates’ private everyday life and provide “hidden views” than is actually the case.

Considering multimodality, most postings show strong consistency between the textual and visual personalization (see Figure 2). An exception is the professional personalization: while images in this category are combined with appropriate texts, too, they are also often used to illustrate texts that call for further interaction and mobilization, provide a private insight, or address political issues. If one takes a closer look at these individual postings, politicians seem to adapt to the visual imperatives of social media by using their official campaign images in situations where a posting otherwise would not contain an image.

OPEN IN VIEWER

Figure 2.

Visual personalization strategies (vertical) and textual personalization strategies (horizontal).

Overall, the results for Instagram and Facebook differ only slightly. Although, on Instagram—both within the textual and the visual posts—the candidate is shown somewhat more frequently “on the big stage.” On Facebook, however, the candidates more strongly focused on interaction and a formal professional impression in their posts.

Since due to the platform logic visual personalization is somewhat more prominent on Instagram, more emphasis is placed here on high-quality photos than on Facebook, where candidates present themselves more formally and more often call for further mobilization (see Figure 3). In addition, candidates present themselves more closely on Instagram, yet again relying more on the visual power of symbolic images in which they pose, for example, in an elevated position on the podium, while the crowd looks up at them. Our comparative analysis thus shows that candidates seem to use the individual channels for different purposes and adapt their strategies accordingly to the platform logic (Haßler et al., 2023).

OPEN IN VIEWER

Figure 3.

Personalization strategies within images on Facebook and Instagram.

Looking at the visual personalization, we also find interesting differences with regard to the candidates’ gender: while male candidates primarily present themselves in professional personalization and on stage, female candidates tend to show themselves in direct interaction with their voters or in active campaigning (see Figure 4).

OPEN IN VIEWER

Figure 4.

Personalization strategies of male and female candidates.

Study 2: automated content analysis of emotions displayed in visuals

In study 2, we combined a quantitative automated content analysis to classify the emotional expressions displayed in posted photographs using deep learning neural networks (Torres and Cantú, 2022) with standardized manual coding procedures to capture emotional articulations in text messages. Such a combined, multimodal approach seems particularly useful for the analysis of emotion, since contextual factors such as textual cues can affect the perception of facial emotions (Feldman Barrett et al., 2011).

In order to analyze the textual articulations of emotion, we manually coded the final sample (n = 401). To detect emotional expressions in the visual parts of the posts, we used a computer vision approach based on the Facial Action Coding System (FACS, Ekman and Friesen, 1978). In FACS, several muscle groups responsible for facial expressions are classified as action units (AU). These muscle groups occur in the upper (12 AUs) and lower half of the face (12 AUs) and include movement of the brows, eyelids, cheeks, eyes, nose and lips. The movements include, among other, the raising (e.g. AU1 inner brow raise), tightening (e.g. AU23 lip tightener), wrinkling (e.g. AU9 nose wrinkle), and clenching (e.g. AU31 jaw clench) of facial parts. The combination of certain actions units (e.g. anger is constituted by lowered and drawn together brows, raised upper eyelids and tightened lips) allow inferences about the expression of six different basic emotions (anger, disgust, fear, sadness, surprise, and happiness; Ekman et al., 2002; Ekman and Friesen, 1978).

FACS is considered an established emotion recognition approach, but the manual coding is resource-intensive and requires extensive training (Ekman and Rosenberg, 2005). As computational methods can significantly improve the efficiency of visual coding (Sülflow and Maurer, 2019), we used FaceReader 8, a trained deep learning algorithm that detects facial AUs and infers to basic human emotions. FaceReader automatically detects structures of the face (e.g. nose, mouth, eyes), then models the face (Figure 5) before it compares the detected facial structures with a database of 20,000 manually coded images to determine patterns within the face. Those patterns are then used to automatically classify the face according to the six basic emotions and a neutral facial expression. Studies that have tested FaceReader against manual-FACS coding recommend it as a valid instrument to detect emotions (Skiendziel et al., 2019). It enables researchers to reliably analyze large samples with regard to the visually displayed emotions. The automated coding of facial emotions not only enables a faster analysis but also provides researchers with more detailed and consistent data (Ahn et al., 2010). In addition to these methodological aspects, there is also a conceptual criterion in favor of focusing on the emotions depicted in faces, since faces are considered to have a very high emotionalization potential (Müller and Kappas, 2010).

OPEN IN VIEWER

Figure 5.

Analysis visualization of FaceReader.

Sample 2

In study 2, we were mostly interested in the emotional expression of the candidates. As automated emotion recognition in image content places demands on image quality and, in particular, the size and resolution of the faces depicted, we excluded all images without satisfactory accuracy and in which the face was not clearly recognizable (e.g. long distance shots, half-profile, out of focus, from behind, in shadow, with face masks). We omitted 242 images which left us with a sample of 159 images for our second study. This dropout rate of around 60% is comparatively low for the automated analysis of non-standardized images (Küntzler et al., 2021).

Measures and automated coding procedure in Study 2

Applying FaceReader 8, we automatically coded expressions of anger, disgust, fear, sadness, surprise, happiness, contempt and neutral expressions (as absence of emotional cues). For each participant, the strength of expression of these emotions was determined on a scale from zero (not expressed at all) to one (perfectly expressed). In most observations we detected mixed emotions, with one emotion dominating the expression.

We also automatically detected the politicians’ gender. To ensure measurement accuracy, we additionally annotated the candidates’ gender manually based upon their outer appearance and their chosen name. A comparison of both codings revealed a deviation in only six cases. We corrected these cases manually, resulting in 98 postings of male candidates and 61 of female candidates.

In addition, a trained coder manually categorized the emotions conveyed in the textual posts. The coder classified individual emotions whenever they were explicitly expressed textually or through the use of an emoticon. Only text statements that unambiguously reflected the applicant’s emotional experience (“I am happy”; “I am afraid”) were coded; figurative speech or emotionalizing language (“it becomes unbearable”) was not recorded as expressions of emotion. Due to this distinction, and aiming to capture as many emotional expressions as possible, we made a conscious decision to apply a qualitative manual analysis rather than automated approaches. While the detection of visually expressed emotions by automated FACS is considered a reliable computational approach (Skiendziel et al., 2019), the computational identification of individually experienced emotions expressed in text messages is still a methodological challenge due to the high need for interpretation (Seyeditabari et al., 2018).

Results of Study 2

In Study 2, we were interested in whether and to what extent candidates convey their own emotions and express them in the images and texts embedded in their posts. Interestingly, even though social media benefits negative tonality and emojis (Heiss et al., 2019), German politicians refrain from depicting negative emotions themselves. With regard to the images, the strongest emotion observed in multimodal posts was happiness (M = .54), followed at a great distance by the emotions anger (M = .04) and disgust (M = .03). With respect to the textual expression of personal emotions, we see an even clearer picture. Negative emotions are virtually absent in the textual posts on social media, while the main emotion observed again was happiness, oftentimes articulated through expressions such as “I’m happily looking forward to it” or “I’m happy about.” This, however, is in line with previous research (Heiss et al., 2019) which found that the expression of positive emotions by politicians is typical as it is expected to encourage user engagement.

Based upon the findings of Tsichla et al. (2023) and others, RQ3 as well as H1 and H2 address gender differences between female and male candidates. As Figure 6 reveals, female and male candidates indeed display emotions differently. A Welch Two Sample t-test testing the difference of the depiction of happy emotions by the candidate’s gender (M_male = 0.43, M_female = 0.70) reveals a significant and medium-strong effect of the candidate’s gender on their visually expressed happiness, difference = 0.27, 95% CI = [–0.39, –0.14], t(135.50) = –4.14, p < .001; Cohen’s d = –0.67, 95% CI = [–0.99, –0.34]. Hypothesis 1, that female candidates compared to male candidates are more likely to express the discrete emotion happiness, was thus confirmed in our study. In comparison, testing for the difference of the depiction of neutral emotions by gender (M_male = 0.42, M_female = 0.22), we find a significant effect of the candidate’s gender being male on the depiction of neutral emotions, difference = 0.19, 95% CI = [0.09, 0.30], t(139.30) = 3.68, p < .001; Cohen’s d = 0.58, 95% CI = [0.25, 0.91]. Accordingly, on their self-selected posts, male candidates show neutral faces to a greater extent than female candidates do, thus confirming H2.

OPEN IN VIEWER

Figure 6.

Automatically detected emotions by gender.

Taking a platform comparative approach regarding the display of emotions, we did not find significant differences between Instagram and Facebook, although descriptive findings by trend suggest that Instagram personalized postings contain more happy emotions, whereas more neutral facial expressions are displayed on Facebook.

Discussion

The aim of our research was to examine the social media posts of political candidates in order to gain a more differentiated picture of personalization and emotionalization through multimodal posts using the example of the 2021 German federal election campaign. Study 1 focused on the question of which sub-strategies of personalization are applied by the political candidates on Facebook and Instagram. We conducted a qualitative visual content and image type analysis of candidate posts and identified key multimodal personalization strategies. Aiming to gain a deeper understanding of the role emotion play within personalization on social media, in study 2, we analyzed how visual and textual information is used by political candidates to convey emotion. Using deep learning algorithms, we classified the emotional expressions transferred in visuals and analyzed emotional articulations in text messages employing manual coding procedures.

Study 1 supports the observation that political personalization in social media is a complex and multi-faceted phenomenon. As previous researchers (e.g. Farkas and Bene, 2021; Grusell and Nord, 2023; Peng, 2021), we found that politicians use Facebook and Instagram to present themselves as suitable candidates, and often rely on professional personalization, while very intimate and personal postings form the exception. Our qualitative approach unveiled previously undetected sub-strategies of multimodal personalization—namely behind the scenes, interaction/mobilization, in action, and on stage—thus adding to the current literature (e.g. Hermans and Vergeer, 2013; Metz et al., 2020) and suggesting a more differentiated view of the concept.

Regarding the role of gender, we found interesting differences in the personalization strategy of male and female candidates: while male candidates seem to present themselves in a rather distancing manner by using professional personalization and on stage shots, females tend to present themselves as more approachable, signified by the use of interaction scenes. It is obvious that such a distanced staging symbolically endows the male candidates with power and authority (Kress and Van Leeuwen, 2020), because the audience literally “has to look up” to them. At the same time, such a distant vie evokes less intimacy than close-ups and interaction scenes preferred by female candidates who tend to signal social proximity and closeness (Andreallo, 2022).

Comparing the platforms with regard to the applied personalization strategies, our results reveal a high degree of consistency and little platform adherence. Although, on Instagram—both within the text and in the images—the candidates are shown somewhat more frequently “on the big stage.” This may be because Instagram focuses more on the symbolic power of images, or because such appearances seem more suitable for the Insta aesthetic as Bast (2021) has suggested. On Facebook, howbeit, the candidates more strongly focused on interaction with their voters and fostered a more formal impression (Farkas and Bene, 2021).

Regarding the multimodal nature of personalization on social media, candidates seem to use the distinct potential of text and images strategically. Our mixed-methods approach shows that images and texts are mostly used in coherence with each other, but sometimes also complementarily, conveying different communicative appeals. Such functional picture–text differentiations are present, for example, when candidates combine rather formal textual announcements with highly personalized photographs, most likely in order to lend more vividness to the rather abstract information. Whether these strategic considerations are intended by the politicians and whether they are effective at the recipient level, must be clarified by future studies. Implementing this research seems all the more urgent as studies across European countries suggest that particularly visual personalization is on the rise (Carlson and Håkansson, 2022). And despite the fact that the iconic dimension of political communication has long been treated as rather trivial and as an “ornamental appendage,” visual practices of displaying, looking and seeing must considered as inherently political because “they shape and constrain what we think it is possible to see, what we are allowed to see, what we are made to see, how we are seen, what is worth seeing, and what is unseen or made invisible” (Andreallo, 2022: 6) in the political sphere.

Further examining the role that emotions play within personalization on social media, with study 2 we showed that German politicians rarely show negative emotions in their pictures. When emotions were displayed at all, happiness dominated. Here, too, the supplementary analysis of textual posts revealed a nuanced picture, as expressions of negative emotions were virtually absent in the textual messages, too. As studies have attested positive emotions a mobilizing potential in social media campaigns (Bossetta and Schmøkel, 2023), the expression of happiness seems logical from a strategic perspective. However, since studies have also pointed to the strong mobilizing effect of negative emotions in election campaigns (Bil-Jaruzelska and Monzer, 2022), the finding of the virtual absence of negative emotional expressions within personalization strategies in the German 2021 Federal election is somewhat surprising. Both male and female candidates, at least, did not aim to mobilize their voters with the deliberate display of individual emotional expressions of fear, anger, sadness, disgust, or contempt. Nonetheless, we found that male candidates presented neutral faces to a greater extent than female candidates in their self-selected posts, while female candidates expressed significantly more happiness. These findings are in line with Boussalis et al. (2022) who identified gender as the largest predictor of visual emotionalization in politicians’ depictions in the US context, and align with Carlson and Håkansson (2022) who replicated such gender differences for 27 European countries. However, since candidates on social media can seemingly decide autonomously how to present themselves on their accounts, such gendered portrayals are not necessarily to be expected—especially as they enforce existing gender stereotypes in politics (Aaldering and Van Der Pas, 2018). As a possible explanation, some authors have suggested that politicians anticipate established stereotypes, and meet these expectations to their advantage (McGregor et al., 2017). This points to a larger discussion of the alleged agency social media and self-portrayals provide to candidates (see Andreallo, 2022: 17). In any case, our results indicate an inequality of visual expression repertoire related to heteronormative views of gender, with female candidates in particular being limited in their repertoire in accordance with platform dynamics, opportunities, and societal expectations. Thinking about future research, interviews with male and female candidates about their strategic considerations would be highly illuminating to explain the observed differences and the motives behind them.

Limitations

Of course, our study has limitations, and some are related to the innovative use of automated emotion detection using computer vision: We had few precedents to draw from, especially since we chose an unusual combination of qualitative and computer-based approaches. Our results therefore are only tentatively comparable to other studies—at least, the different setups have to be taken into account when comparing the observed findings.

It is also important to acknowledge that automated emotion detection, like human coding, is not free from biases in correctly identifying emotions of different genders, age groups, and ethnicities (Ahmad et al., 2022). In this respect, observed differences, for example, between genders, may also result from a bias in the automatic analysis of emotions. Preliminary studies show that misclassification, such as misclassifying women as happier, occurs particularly when the algorithm’s training dataset is unbalanced (more happy women compared to men) or incorrectly annotated (human coders perceive women as happier due to their internal bias and therefore mislabel the data; Chen and Joo, 2021). The applied FaceReader is based on a balanced training set, was validated with the Standardized and Motivated Facial Expression of Emotion (SMoFEE) and provided similar good results as trained human coders (Skiendziel et al., 2019). Nevertheless, further research is needed to investigate possible biases in automatic encoding of facial expressions, especially when dealing with less standardized representations that may lead to more misclassifications (Küntzler et al., 2021). Furthermore, although the system as a whole has been validated (Skiendziel et al., 2019), standard parameters such as precision and recall are missing for FaceReader, which complicates quality assessments of individual emotion measurements.

Another limitation is related to our sample: we made a conscious decision to structure the search according to the most important political issues rather than the top politicians of the political parties. We chose to do this because it allowed us to ensure that we included a broad sample of political candidates, including politicians from regional and local associations—and not just the political elite. In our opinion, this approach also tends to reflect what is offered from a user perspective, as this was not based on which postings individual politicians published, but which postings were actually offered to users. However, this sampling decision also makes our study less comparable to other studies, as most analyses are guided by a focus on top-level personnel. For future studies, it would be valuable to additionally include the accounts of the parties’ top candidates in order to then analyze the politicians’ self-representation in a more stratified way, and to compare the influence of contextual factors (e.g. regional identity, ethnicity, party affiliation). In addition, we only distinguished between two gender expressions, thus our work adheres to heteronormative assumptions.

Methodological limitations also include the fact that we could only consider images for automated image analysis that met the technical requirements, resulting in a smaller sample in Study 2 than in Study 1. Nevertheless, our qualitative–quantitative approach combining manual and automated coding steps led to interesting results that complement the current state of research and may also illustrate how such innovative methodological approaches can be used in future studies.