Multivariate Testing Confirms the Effect of Age–Gender Congruence on Click-Through Rates from Online Social Network Digital Advertisements

Introduction

A key benefit of web-based technology is the computational ability to target users and customize communication using context-aware approaches and explicit or implicit personalization techniques. ¹ This study uses multivariate testing to assess the extent to which an online ad treatment, which has been designed to be age and gender congruent with the target audience, increases the likelihood of the ad being clicked on. A particularly important feature of this study is that the data are derived from a real-life, real-time digital advertising campaign. This study used the execution of an ad campaign with users whose authentic reactions to the ads were captured. The time-bound ad campaign was hosted on the online social network Facebook, in one region of the United Kingdom.

Background

Personalized messages can be defined as messages that are “delivered to each individual user through paid media based on personal information (such as user names, past buying history, demographics, psychographics, locations, and lifestyle interests).”² Online social media “provide web services which facilitate users maintaining a public or semi-public profile within a bounded system” and through which they can “articulate a list of other users with whom they share a connection.” ³ On the basis of the number of active users worldwide, Facebook remains the largest online social network (Statista). ⁴ According to a 2015 social media marketing industry report, only 45 percent of marketers think that their Facebook efforts are effective. ⁵ A particular attribute of advertising on Facebook is its capability to target users of specific gender, age, and other criteria. Demographics are the most common variables used to target online users, and age and gender are the most common demographic variables used due to the ease with which an online user's gender and age can be determined and used.^6,7 Tailoring ad content to reflect the demographic profile of the target audience relates to the concept of user congruence.

Congruence is defined as the extent to which content expressions coincide with self-concept ⁸ and developing advertising appeals that are congruent with the target's self-concept is thought to be more effective than appeals which might be deemed incongruent. ⁹ Such appeals are considered to reinforce self-concepts ¹⁰ and generally “for-me” self-congruent messages are more persuasive than “not-for-me” self-incongruent messages. ¹¹

Hypotheses

The study addresses a number of hypotheses that deal with gender congruence, age congruence, and gender and age congruence combined. The hypotheses are grouped according to these variables and the first hypothesis relates to gender congruence solely. As outlined previously, the gender of the central figure in an ad is one factor that effects a process of self-categorizing ^12,13 and is one of a number of factors that contributes to overall self-congruence. As such, the H1 hypothesis and its associated hypotheses are shown in Table 1a.

Similar to gender congruence, age congruence is another variable contributing to overall self-congruence. Therefore, the congruency between the model's age and the audience's age may also influence the effect the ad has on the audience. ¹¹ Therefore, the H2 hypothesis and its associated hypotheses are shown in Table 1b.

While it is proposed that gender and age, respectively, will have positive effects on advertising effectiveness, what is of further interest is the combined effect of these two variables. It can be assumed that an ad exhibiting both age and gender congruence will produce the best effects, leading to the H3 hypotheses in Table 1c.

Leading on from these hypotheses emerges the further proposition that either gender or age congruency is better than neither variable being congruent. This extends to the H4 hypotheses dealing with first, gender congruency but not age congruency as shown in Table 1d, and second, age congruency but not gender congruency, as shown in Table 1e.

Methodology

The broad design principle of this study is the manipulation of a direct response advertising campaign to allow ad treatments to be varied along two variables: gender and age (of model) and to be served to a population that had been segmented by the same two variables. A two-by-two consumer profile design was combined with multivariate testing to address the hypotheses just outlined.

For this study, in Facebook, we created four campaigns each containing four ad sets. Four ads containing facial imagery reflecting a different age and gender persona were shown equally to four similar populations of cohorts that were targeted at an ad set level within each campaign. We displayed only one ad, reflective of a specific age range and gender persona, in each of the four campaigns.

To maximize external validity, it was important to use a real business. The product offering needed to be appropriate to all ages and to both men and women. It was also important that the product offering was such that its depiction within the ads was the same in all four treatments, while ensuring that each resultant ad was realistic. With this in mind, a teeth whitening procedure was selected to be the focus of the ads. The ads were created such that their creative execution differed only by gender or age of the model. Further details on the targeting criteria are provided later. The overall design of the experiment is presented in Figure 1 below.

OPEN IN VIEWER

FIG. 1.

Design content for the experiment.

Designing content for the experiment

The ad content was identical in every respect except for the image. Four different images were used which represented one of four different age and gender personas: older female; younger female; older male; younger male. The age and gender of the models included in the images were objectively verified using a publicly available face detection application program interface (API) developed by “Face++.” ¹⁴ A recent study investigating engagement rates on Instagram found the “Face++” API to show high accuracy. ¹⁵ One of the ad treatments (“older female”) is shown in Figure 2 below.

OPEN IN VIEWER

FIG. 2.

Example of ad treatments for “older female.”

Results

This study set out to investigate if ad effectiveness is influenced by the extent to which the gender and age of the model depicted in an ad execution was congruent with the target audience. To test the hypotheses, 95% confidence intervals and an “N −1” χ² test, as originally proposed by Pearson ¹⁷ and recommended by Campbell, ¹⁸ were used to determine the statistical significance (α = 0.05). Accordingly, differences in click-through ratios with a p value less than 5 percent, were regarded as representing significant differences.

In the experiment, a total of 659,522 Facebook users were exposed to the ads (number of impressions). We recorded 1,733 unique clicks on the ads, representing an overall click-through ratio for the whole campaign of 0.26 percent. The click-through ratios across the 16 combinations is shown in Table 2. Before turning to the hypotheses individually, some initial observations can be made.

First, older users exhibited almost twice the click-through rate of ads (0.36 percent) of younger users (0.18 percent). Furthermore, females clicked on a greater proportion of ads (0.31 percent) than males (0.23 percent). As shown in Table 2, older females are more likely than any other cohort to click through the ad, regardless of which ad was being presented (0.47 percent). Further analysis shows that the difference between this click-through rate (CTR) and the CTRs of the other three audiences is statistically significant (p < 0.05).

Second, as per Table 3, the ad treatment which elicited the greatest click-through, across user profiles, was the ad depicting a younger male model (0.32 percent) and again this CTR was significantly higher than that generated by the other three ad treatments (p < 0.05).

Turning now to the hypotheses, the first hypothesis of the study was that gender congruence will have a positive effect on advertising effectiveness (H1), and within this, two separate hypotheses were presented. The findings in relation to these hypotheses are presented in Table 4 below. It is shown that H1a was supported, in that men clicked on a significantly greater proportion of ads featuring a male model compared with a female model, but that H1b was not supported, in that female users did not click on a significantly greater proportion of ads featuring a female model compared with a male model. As such, H1 cannot be fully supported.

The second hypothesis was that age congruence will have a positive effect on advertising effectiveness (H2). From Table 5 it can be seen that the first of the two associated hypotheses, H2a, was supported in that older users clicked on a significantly greater proportion of ads featuring an older model compared with a younger model. Similarly, H2b was supported in that younger users clicked on a significantly greater proportion of ads featuring a younger model compared with an older model. As such, the second hypothesis was supported.

The next hypotheses considered age and gender congruence together. It was first proposed that when the ad is both age and gender congruent, the click-through rate is higher than with any other ad treatment (H3). Findings in relation to the subhypotheses contributing to H3 are shown in Table 6 and Figure 3. H3a proposed that older female users will click on a significantly greater proportion of ads featuring an older female model compared with any of the other three ad treatments. This was fully supported. H3b proposed that younger female users will click on a significantly greater proportion of ads featuring a younger female model compared with any of the other three ad treatments and this was also fully supported. H3c held that older male users will click on a significantly greater proportion of ads featuring an older male model compared with any of the other three ad treatments and it too was fully supported. Lastly, H3d proposed that younger male users will click on a significantly greater proportion of ads featuring a younger male model compared with any of the other three ad treatments and it also was fully supported. Overall, therefore, there is full support for H3, indicating that the most effective ad is one that is both age and gender congruent.

OPEN IN VIEWER

FIG. 3.

Click-through rates for all ads.

The last set of hypotheses proposed that either gender or age congruency (but not both) is better than neither variable being congruent (H4). Within this, the first hypothesis is that when the ad is gender congruent but not age congruent, the click-through rate is higher than if the ad is neither age nor gender congruent (H4a). The findings in relation to the associated hypotheses are presented in Table 7, where it can be seen that none of four subhypotheses was supported and as such H4a cannot be supported.

The second hypothesis associated with H4 is that when the ad is age congruent but not gender congruent, the click-through rate is higher than if the ad is neither age nor gender congruent. Findings in relation to this are shown in Table 8.

As per Table 7, none of the associated hypotheses finds support. Indeed, one of the differences is significant but in the opposite direction to that proposed. So, instead of older male users clicking on a significantly greater proportion of ads featuring an older female model than a younger female model (H4biii), they clicked on a significantly greater proportion of ads featuring the younger female model. As such, H4b cannot be supported. Moreover, the fourth hypothesis (H4) is not supported by the findings, thus indicating that gender congruency alone, without age congruency, does not produce better results. These findings are discussed next.

Discussion

The findings have yielded important new insights about the topic of personalization in online advertising and content presentation and specifically on the effect of age and gender congruence on ad click-through rates. The findings showed that older users proportionately clicked on almost twice as many ads as younger users (0.36 percent vs. 0.18 percent). This may support other research that found that older people are more drawn to faces on a Web page than younger users.^19–21 It was also revealed that older females, with a click-through rate of 0.47 percent are more likely than any other cohort to click through the ad, regardless of which ad was being presented. The gender aspect of this finding supports previous studies that revealed that females showed a more positive attitude toward online ads and indicated a greater intention to click through and learn more than males.^22,23 Similarly, other research found that females were more likely to be drawn to photos on a Web page. ¹⁹ Again, the results from our study appear to corroborate these past studies as our female cohort were statistically significantly more likely than our male cohort to click on an ad with facial imagery. In addition, from both a gender and age perspective, it supports other research, which found that females 50 years of age and over have a 31.2 percent higher click-through rate on ads than younger females (age 18–29). ²⁴

A second key finding is that the ad treatment which elicited the greatest click-through (0.32 percent) across all user profiles was the ad depicting a younger male model. This would seem to support other researchers who argue that older models are less appealing than younger models ²⁵ and relates to a belief held by some that featuring elderly models in advertising can alienate users. ²⁶

Turning to the hypotheses of the study, the findings indicated that age congruence, regardless of gender congruence has a positive effect on advertising effectiveness. However, the proposition that gender congruence, regardless of age congruence, would have a positive effect was rejected. It was therefore instructive to drill down further into these variables to explore their individual effects and their potential cumulative effects.

There was clear, definitive support for the hypothesis that age and gender congruence together will have the most positive effect. It was found that an age- and gender-congruent ad was the most clicked on compared with any other treatment across the four audience segments.

The last hypothesis is perhaps the most interesting and potentially useful. It was proposed that when the ad is either gender or age congruent, but not both, the click-through rate would be higher than if neither variable is congruent. The clear conclusion is that this is not the case. Specifically, gender congruency with age incongruency does not produce better results and equally, age congruency with gender incongruency did not lead to higher click-through rates. Taken together, this study provides compelling evidence that age and gender congruence are not just complementary but actually require each other to serve any significant benefit.

Conclusion

This study clearly demonstrates that marketers who wish to improve click-through rates on social media or just indeed design content for users of their content ought to take into account both the age and gender congruency of the content and the target audience. Rather than having a single ad or content that all the target audience sees, it is recommended that a number of treatments are created with imagery tailored to reflect the age and gender of the recipient. In our study, ensuring the ad was both age and gender congruent led to almost a doubling in click-through rates for most of our cohorts. Using age or gender alone cannot be relied on to improve performance.

This study used multivariate testing of an advertisement campaign on the online social network, Facebook, to investigate the extent to which digital advertising, personalized to specific age and gender group demographics (age and gender congruent) influences user engagement and increases click-through rates.

The study achieved a total of 659,522 impressions (i.e., number of users who were exposed to the personalized advertisements and had the opportunity to engage). Moreover, a total of 1,733 unique clicks were recorded. Using N-1 χ² testing, this study found that a combined age and gender congruency yielded statistically significantly greater click-through ratios in comparison to noncongruent (nonpersonalized) online advertisements (p < 0.05).

As an example, the click-through rates by younger male users increased by over threefold when a young male model appeared in the imagery. The implication is that online content that is personalized to the user's age and gender demographic increases active user engagement.

This study indicates that the additional time taken by digital marketers and web developers to personalize content to reflect the age and gender of each user is a worthwhile and useful investment, given the relative computational ease with which online content can be customized for different user groups on an interactive platform.

Study Limitations

This study benefited from being based upon a real-life, real-time, online advertising campaign and thus, avoided the limitations of similar studies that rely on “indirect findings.” ²⁷ Care was taken to ensure that like-for-like comparisons could be made within the two-by-two ad treatment/audience profile design. However, there are a number of factors that may be considered limitations.

First, while there is no reason to believe that the geographical location of the study—the Glasgow metropolitan area, a region of the UK—is atypical, the fact that the sample was drawn from such a specific region could be construed as limiting. Related to this, all the people featured in the ads were white. It is the case that the vast majority of people residing in the area chosen for this experiment are white; according to 2011 census data, over 88 percent of people living in “Glasgow City” are white. ²⁸

Our experiment was scheduled during a 4-hour time period on a Sunday. Being limited to any single time period could inadvertently introduce bias by favoring one age or gender group over another. That said, all cohorts in our study did receive equal ad impressions, and so this issue was, to a great extent, legislated for.

This study used click-through rates as a measure of ad effectiveness. Previous studies have shown that women are more likely to click on ads to gather information, whereas males, being more task oriented, are more likely to click only when there is a “pressing need.”^29,30