Algorithmic brand culture: participatory labour,machine learning and branding on social media

Abstract

Social media platforms like Facebook and Instagram have transformed how brands operate as cultural processes. Over the past decade, these platforms have engineered an algorithmic brand culture that combines the participatory affordances and data-processing power of digital media. In the brand culture of social media, the creative narration of cultural experience doubles as data that trains platform algorithms. We develop an account of the algorithmic brand culture of social media via a case study of the Splendour in the Grass music festival, the activations sponsoring brands build at the festival, and the participatory mediation of the festival on Instagram. Brands play a critical role in both funding and engineering social media platforms, and integrating them into cultural experience. We argue for the development of critical cultural and computational approaches that examine how the use of machines to make judgments about cultural life is intrinsic to capitalising on the participatory nature of brand culture. Media researchers must go beyond describing the content of user-generated content on platforms to critically simulate and scrutinise media platforms’ algorithmic infrastructure.

Keywords

affective labour algorithmic culture brand culture branding image classification Instagram media platforms music festivals social media

Introduction

When attendees arrived at Splendour in the Grass music festival near Byron Bay in Australia in 2011, some may have seen a small sign that stated ‘Your entry beyond this point constitutes your consent to your likeness to be used for Virgin Mobile promotional purposes anywhere in the world, at any time, without litigation’. Similarly, those entering the Smirnoff Cocktail Bar at the festival in 2014 might have read ‘Smile! You’re on camera! Filming in progress. By entering this area I understand that I may appear on camera and my image or voice may be used for advertising and/or trade purposes’. The two notices disclosed to attendees that entry to the festival embedded them in a media infrastructure, involved them in the creation of brand value and harnessed the social relations they would jointly create as they partied at the event.

It was just over a decade ago that digital cameras, and then smartphones, became an ordinary part of cultural events like concerts and music festivals. The experience of live performances became characterised by using a web-enabled smartphone to capture and circulate images on the social web. This instigated a change in the role cultural experiences played in the production of brands. Cultural spaces like music festivals became integrated into the emerging advertising model of platforms like Facebook, Instagram and Snapchat. Brands used these cultural events to harness the participatory labour of attendees. They began by creating opportunities for attendees to incorporate the brand within the images they created and shared on social media (Carah, 2017). Over time, this narration of social life in images, videos, likes and comments came to double as the production of data about individuals, their social networks and cultural worlds. That data fed the development of social media’s algorithmic engagement and targeted advertising models. When a festival attendee shares images of their experience partying in a branded bar at a festival, they both incorporate the brand in a user-generated story about their cultural experience and generate data that connect the brand to a social network and set of cultural associations.

This production of content and data has been understood as the increasingly ordinary participatory labour of promotional culture. We represent brands as part of our lives, and we generate data about ourselves. An often neglected aspect of this labour however, is that a key product of our participation on social media is the platform infrastructure itself. Our participatory engagements congeal in the form of platform interfaces, protocols, algorithms and advertising models. Platforms like Facebook, Instagram, Snapchat and Google are among the largest and fastest growing advertising-funded media businesses in the world. Facebook and Google effectively constitute a duopoly accounting for the majority of online advertising revenue, and most of the growth. The more we document our lives on these platforms, the more we enhance their capacity to engineer a new kind of advertising and branding. In this article, we argue that brands are the critical, yet under-examined, process in the ongoing development of social media platforms, their commercial models and cultures. The objectives of advertisers underpin the development of platform interfaces, protocols, algorithms, and the participatory cultures they afford.

The relationship between the participatory culture and data-processing power of social media is one where lived experience becomes gradually more sensible to machines. The advertising model of social media – viewed in the longer term – seems geared to developing a media infrastructure with the calculative capacity to experiment with the experience of reality (Carah, 2017). To understand this process, critical media studies must go beyond using computational approaches to describe what people do or say on social media, to explore how machines learn to make judgments about lived experience. This requires developing an account of both how machine learning works and how the labour of training machines is dispersed into cultural life. In this article, we develop a collaborative approach between media studies and computer science to examine the emergence of algorithmic brand cultures using the case of a music festival.

The algorithmic brand culture of social media

Media platforms like Facebook and Instagram are a key site for experimentation in the data-driven optimisation of participatory branding. Brands are participatory in the sense that they operate as open-ended cultural processes that rely on the capacity of consumers to incorporate them into their self-narratives, everyday practices and cultural spaces (Banet-Weiser, 2012; Fisher, 2015; Hearn, 2008; Zwick et al., 2008). Brands operate not so much as defined symbolic messages but rather as cultural platforms and programming devices (Lury, 2004). By ‘platform’, we mean that a brand offers symbolic resources and builds social spaces that structure the participatory incorporation of brands into cultural life. And, by ‘programming device’, we mean that a brand draws on algorithmic media to judge, modulate and capitalise on that participation (Hallinan and Striphas, 2014; Lury, 2004). Media platforms can therefore be understood as brand-funded engineering projects where participatory and culturally embedded modes of branding intersect with the capacities of algorithms to make judgements about cultural life (Banet-Weiser, 2012; Hallinan and Striphas, 2014; McStay, 2013). Platforms like Facebook and Instagram create interfaces, protocols, databases and algorithms that engineer and optimise user participation for commercial-ends (Van Dijck, 2013).

In this article, we describe this formation as algorithmic brand culture, where brand value is created by managing the interplay between the open-ended and creative capacities of participants and the calculative capacities of media platforms. We develop our account via a case study of the Splendour in the Grass music festival, sponsoring brands and Instagram. Music festivals are significant sites where the engineering of feedback loops between material cultural spaces, lived cultural practices and the databases of media platforms can be critically observed. Brands like Smirnoff, Virgin, Jagermeister and Spotify are major sponsors of music festivals like Splendour in the Grass in part because they can use them as a location to get attendees to generate engagement, content and data on platforms like Instagram. The festival site is purposefully designed to stimulate, channel and leverage these capacities. Art installations and brand activations are purpose-built spaces that generate a shared sensibility at the festival. Festival attendees undertake the productive activity of generating shared sociality, mediating sociality and registering sociality as machine-readable data on media platforms. Brands, and the music festivals and media platforms they fund and sponsor, stimulate and leverage each of these activities.

Participants in an algorithmic brand culture undertake the affective labour of both affecting humans and training machines. The concept of affective labour has been critical to accounts of how networked forms of cultural production, like social media and brand culture, rely on the communicative and cooperative capacities of social life (Terranova, 2000; Wissinger, 2007). In our account, affective labour refers to both the conscious narration of affect and the pre-conscious, non-symbolic and embodied capacity to affect and be affected (Clough et al., 2007; Wissinger, 2007). Festival attendees undertake affective labour when they generate shared sociality at the festival and use their smartphones to register their experience as images on media platforms. When they dance at a performance, bump into other bodies, sing the lyrics of a song, wear a costume, or capture video on their smartphone, they do the productive labour of stimulating and channelling the attention of others. As they snap, swipe, tap and glance at flows of images on their smartphones they do the work of generating archives of digital material that inform brand interventions in cultural lives and spaces. Users undertake the ‘work of being watched’ in both the sense of producing their lives as images others consume and as data that machines process (Andrejevic, 2002). Critical analysis of the commercial logic of algorithmic media cultures must account for the embodied capacity to register human experience as machine-readable data, to train machines, and to knit them into cultural experiences that were once the sole reserve of human judgement. Brands are critical actors in the engineering of media platforms, and so a critical cultural account of brands needs to engage with the technical development of systems of algorithmic judgement interwoven with living bodies and lived cultural experiences.

We propose critical simulation as a technique for understanding affective labour and human-machine interplay in algorithmic brand culture. Simulation approaches, common in computer science, are necessary to critical studies of algorithmic media and brand cultures. In engineering and computer science, simulations are computational models are developed as analogues for physical and biological systems. The development of computational methods to study social media has focussed predominantly on documenting the content of those platforms (Highfield and Leaver, 2016). While this work is important, it must be complemented by the development of computational methods that enable critical public scrutiny of the algorithmic sorting and classification of cultural life. Critical simulation places emphasis not on documenting the content of media platforms, but rather on seeking to engineer and publicly scrutinise their calculative infrastructure.

Splendour in the grass: music festivals as media infrastructure

Splendour in the Grass, owned by multinational music group Live Nation, is a 3-day music festival staged in a nature reserve in the hinterland of the Australian coastal town Byron Bay. Media and brand infrastructure are an important part of the festival. Festival organisers install video screens, wifi and mobile communication towers that amplify and distribute the performances on site and social media platforms. Festivals like Splendour are sites of experimentation with participatory branding and media technology (Carah 2017). For example, in 2013, Splendour had attendees wear RFID chipped wristbands that could be connected to their Facebook account. As they moved around the site, they could swipe the band at sensor points, which generated an automatic status update for their Facebook profile alerting friends to where they were and what they were watching.

Brands are key actors in connecting cultural spaces like music festivals with media platforms like Facebook, Instagram and Snapchat. Brands initially built activations at music festivals as a way to engage with hip consumers. ‘Activation’ is the marketing industry term for the infrastructure that brands build at cultural events like music festivals. At Splendour in the Grass in recent years brands have built elaborate themed bars: Strongbow cider built a bar on the deck of an antique sailing ship they had trucked into the festival, Smirnoff vodka built a multi-level cocktail bar themed as a retro ‘nanna’s’ house and Jagermiester built a rustic hunting lodge. In each case, the activation is designed to create a memorable aesthetic and affective space where festival attendees enjoy partying and are encouraged to mediate the experience using smartphones.

In the sections to follow, we develop an account of the relationship between the design of the festival site, brand activations at the festival and the archive of images uploaded to Instagram using #splendourinthegrass. We develop this account from a combination of participant-observation at the festival (2012, 2013 and 2014), an informal tour of the festival site with festival organisers (2012), 15 interviews with informants who went to the 2011 or 2014 festival, and an analysis of 1644 images uploaded to Instagram using #splendourinthegrass over a 24-hour period during the 2014 festival. This project is a collaboration between a media and cultural studies researcher and a computer scientist. A singular pronoun is used in the account of the Splendour fieldwork because it was conducted by one researcher. Informants met with the researcher before attending the festival. They agreed to catalogue the images they captured during their festival experience. These image practices changed substantially between 2011 and 2014. In 2011, attendees took large ‘albums’ of photos that they then loaded onto Facebook after the festival. In 2014, they used their smartphone to both create and consume images during the festival, using platforms like Instagram and Snapchat. Following the festival, the informants met with the researcher for an open-ended interview about the festival experience and to critically reflect on the images they collected. In 2014, informants also followed their social media feeds in the weeks following the festival, documenting how the festival was represented in images circulated by peers.

The analysis to follow critically examines how a sense of shared sociality is created, mediated and registered as digital images that double as machine-readable data. We examine in particular the role that art installations and brand activations play in generating an atmosphere at the festival that is translated into flows of images on platforms like Instagram. The purpose of this article is not to present an ethnographic analysis of Splendour in the Grass or a close textual analysis of the images produced by festival attendees. Rather, by drawing on a combination of participant-observation, informant interviews and analysis of an archive of Instagram images, we aim to conceptualise the infrastructure of an algorithmic brand culture. We conceptualise the feedback loop between the design of the purpose-built festival site, art installations and brand activations, the creative capacities of festival attendees, and the use of smartphones and Instagram to mediate and process the experience. In the final part of the article, we consider the challenges for critical scholarship of algorithmic brand cultures and social media in analysing and theorising the emerging algorithmic capacities of media. At no point do we argue a deterministic relationship between brands, algorithmic media platforms, and the design of cultural space. Our intention is to open up ways of conceptualising the dynamic relationship between the commercial objectives of brands, the creative capacities of humans in cultural space, the design of cultural spaces, and the calculative logics of media platforms. The participatory affordances of social media and smartphones appear to both intensify the enjoyment of cultural experiences, at the same time they employ those cultural experiences as productive brand-building activities.

Engineering connectivity in algorithmic brand culture

Media platforms like Facebook and Instagram are socio-technical architectures that engineer and broker connectivity (Van Dijck, 2013). Platforms are commercial engineering projects that aim to ‘activate relational impulses’ for ‘input into algorithmically configured connections’ (Van Dijck, 2013: 161). That engineering process extends beyond the platform itself to cultural spaces like music festivals. This section is organised around three moments in the sequence of engineering connectivity at Splendour in the Grass. First, we examine how the design of the festival site, art installations and brand activations generate shared sociality at the event. Second, we explore how the ‘atmosphere’ at the festival is mediated by attendees using their smartphones. And finally, we present a, necessarily speculative, critical analysis of how an archive of images generated from the festival might be ‘algorithmically recognisable’ (Gillespie, 2014).

Our aim is to consider how these three moments constitute the feedback loop of an algorithmic brand culture. The design of the festival space weaves brands into cultural experiences, and the mediation of those experiences generates archives of data, which in turn enables brands and cultural producers to optimise the design of cultural space as a domain for the generation of brand value. In conclusion, we argue that a critical simulation of the algorithmic capacities of media platforms, and in particular image classification, is necessary to account for cultural formations characterised by commercial engineering of human-machine judgement. Our deliberate focus on this sequence comes at the expense of a detailed analysis and complete presentation of the fieldwork for each moment. We acknowledge this as a necessary limitation, our aim in this article is to develop an account of the infrastructure of an algorithmic brand culture, rather than a detailed account of human experiences of the cultural space this infrastructure facilitates.

Creating shared sociality

Informants routinely described a sense of shared sociality at Splendour. One explained that

Splendour for me always had … this sense of wonder to it. … all these people, really friendly with each other, really cool, all into good music, just sort of gallivanting in the woods for a few days, where nothing else kind of matters. (Festival attendee 10)

Another described the festival as a ‘little Disneyland. You forget everything. It’s very well set up’ (festival attendee 12). Festival attendees arrive with the intention of setting aside usual routines of study and work. They plan to pursue pleasure, meet new people, consume alcohol and drugs. In their accounts, they perceive that their desires are reflected in the aesthetic production of the site, its performances and the actions of fellow attendees.

Art installations are an important part of the ‘wonderland’ or ‘theme park’ Splendour creates. The festival runs an arts programme where it commissions artists to construct large installations around the festival site. Over two consecutive festivals, the contemporary artist Bennett Miller built the art installation and performances ‘Barnraiser’ (2013) and ‘Rumspringer’ (2014) (Lismore Regional Gallery, n.d.). The 2013 installations involved a staged Amish barn-raising. Actors dressed as Amish posed for photos with festival attendees as they went about raising a barn, sharing meals and staging a wedding. The barn-raising went on throughout the weekend as the festival raged on around it, the barn sitting amid the rainforest site between a rough-hewn Bundaberg Rum brand activation and a festival stage.

The 2014 ‘Rumspringer’ opened the now finished barn to festival attendees. The installation played on the Amish rite of passage where young people leave their community for a period of time before freely choosing to re-enter and commit to Amish life. The period of Rumspringer is characterised by access to freedoms and forms of consumption denied within Amish life. Amish-dressed performers invited festival attendees in to drink and sit on hay bales around the barn. The performers played folk music and worked on traditional crafts like weaving and woodcarving. ‘Rumspringer’ clearly referenced the ‘vibe’ festival attendees described in their descriptions of Splendour as a space removed from the routines of everyday life.

The festival site has many art performances and installations like ‘Rumspringer’ that invite the participation of attendees. Describing a competition Splendour ran where they asked music fans to dress their cars up as a unicorn one attendee explained,

Some cars went full out, like Where’s Wally or Harry Potter or like, it was so funny. It’s kind of like an ice-breaker, because you’re coming from somewhere, you don’t know, you’re in the middle of nowhere, you only know some people, but you make friends. Everyone is so nice and out-going. And so many people told me that. I think they wanted to do that. My friends were like, ‘why is everyone so nice here?’ (Festival attendee 12)

The informant illustrates how the festival’s effort to involve people in art performances or decorating cars and campsites generates a social atmosphere. They encourage strangers to interact in playful and unexpected ways. Informants described being approached in their campsites by ‘eco cops’ and ‘eco fairies’ who undertook theatrical inspections of their campsites to ensure they were sorting their recycling from their rubbish. At the 2014 festival, I wandered into an area set up as a used car lot. I was approached by a man in a gaudy 1970s suit who played the part of a crooked used car dealer. As evening fell I stood for a while watching the Skywhale, a hot air balloon whale with several breasts for wings, designed by the sculptor Patricia Piccinni, being inflated between the festival site and the campgrounds (Piccinni, n.d.). Many festival goers stopped to watch the balloon take shape. Some, appearing to be affected by alcohol and drugs, had visceral reactions. Many took images and videos. Splendour is a series of interlinked spaces and performances that festival attendees flow through, each cultivating and affirming a sense of wonder and escape that attendees both desire and co-create.

Sponsoring brands are then layered into these artful social engagements. In the first instance, the festival is valuable to brands because it generates a cultural and affective atmosphere they can inhabit. In 2011 Jagermeister built a wooden hunting lodge with taxidermy moose heads, chandeliers, a fireplace, a bar, screens, wooden stools and benches, DJs and a dance floor. Several artists playing at the festival performed DJ sets at the hunting lodge. These intimate performances attracted fans to the branded venue. One festival attendee explained that Jagermeister’s hunting lodge was a ‘massive dance floor’ that fit the ‘aesthetics of a winter festival’ (festival attendee 1). Offering a similar perception, another attendee described Jagermeister’s installations at the festival as ‘aesthetically cooler’ (festival attendee 4) than other brands. She explained that while it ‘was really cold outside’ inside the Jagermeister activation ‘it was really warm’. The aesthetics of the winter festival are invoked by the bodily sensation of being warm in a woodland setting in the middle of winter.

The same year, Strongbow’s antique sailing ship sat in a central location between two of the major stages where bands perform. The ship had a large deck, bars, promotional staff taking photos, and served Strongbow. At night, it was lit up with fairy lights. One informant spoke to me about how brands evoked a sense of the ‘old’ Splendour, a sense of the feelings and tastes that attract audience members to the festival. She evaluated the Strongbow installation by saying,

I like Strongbow the best because it spoke to the old Splendour, you’re just on a deck drinking a bit of cider. … It captures the night time, you’re in the forest, kind of feel. You’re drinking cider. I think it was just a little bit more true to the original of what I recall from the festival years ago. (Festival attendee 4)

By being woven into the material festival site, the brands become part of the way fans imagine, remember and affectively relate to the festival experience. This fan remembered the brand installations as part of the feelings of wonder and escape as they drink, dance and socialise in the festival site. The Strongbow ship was woven into the festival, close enough to stages for audience members to see bands on stage as they drank cider on the deck and imagined themselves ‘sailing’ through the festival. Being on the deck of the Strongbow ship didn’t mean you ‘give up music or being in the thick of things’ (festival attendee 4), rather the branded experience was woven into the feeling and experience of the festival.

The brands become, in the words of an artist who worked at the festival, part of the ‘cultural mega-mash’ (festival attendee 11). Art installations and brand activations are frequently woven together. Artists perform in branded bars, like the Red Bull Music Academy venue or the Jagermeister Hunting Lodge. Artists painted large murals on the walls of the Smirnoff Cocktail Bar in 2012. The Bundaberg Rum barn and the Amish Barn sat side by side in 2013. The ‘Rumspringer’ barn doubled as a bar where sponsoring alcohol brands were sold in 2014. The brand activations and art installations at the festival are each visually rich social spaces. They are each attuned to the atmosphere of the festival and use similar forms and devices (Böhme, 1993). One of the important roles they play is offering cultural resources and performances that festival goers curate and structure as flows of images on social media.

The festival site establishes what Alison Hearn (2008) refers to as a general ‘ambience’ for branding. After the festival, some informants expressed ambivalence about these activations. Recalling the sign at the entrance of the festival that licenced brands to use images of all attendees, one informant explained that ‘we’re too comfortable with branding’ (festival attendee 10). But, in the words of another informant, during the festival ‘no one would have given a shit… [they are] just another environment to drink in’ (festival attendee 5). These perceptions echo Banet-Weiser’s (2012) conceptualisation of brands as a recognisable part of cultural life. Attendees are aware of them, evaluate their intentions and objectives, and respond to them in ambivalent ways. These forms of critical distance though do not thwart brands. Brands generate value not so much because attendees acknowledge their authenticity, but rather because they are woven into the festival atmosphere and lived cultural practices. Brands act as infrastructure that facilitate the affective flow of the festival. For instance, the vitamin brand Berocca opened up a shady brand activation during the first morning of the festival. As hung-over festival goers stumbled about the site, brand promo workers invited festival goers in to drink a revitalising Berocca, sit in a bean bag and charge their phones. In this moment, the brand was acting as affective infrastructure, attuned to the embodied feelings and desires of the festival crowd at that moment. By nightfall, the Berocca activation was in darkness, and the alcohol brand activations were pulsating.

Mediating shared sociality

Music, art and brand performances at the festival stimulate a sense of play among festival goers. Smartphones enable festival attendees to translate this play into media content and data. One attendee described to me a performance at Splendour that had the audience enthralled. Attempting to explain the shared feeling he said, ‘it was that thickness in the atmosphere, and the body responding to music as it’s happening and that builds that ambient experience you’re then responding to’ (festival attendee 11). He then offered an explanation of how that ‘ambience’ and ‘thickness’ is mediated and circulated by festival attendees:

Well, it is that thing about creating a memory. So I guess we have these tools now to really create these memories. And I guess we want to create these memories instantly, and maybe that’s why Instagram and stuff is really interesting, because you can … you can make your memories cooler immediately. You can kind of get an instant nostalgia. (Festival attendee 11)

The music festival becomes a space where material cultural practices like dancing to live music can be extended, expressed and archived on media platforms.

Digital cameras and then smartphones made it commonplace for live music audiences to capture images and video of performances. While the quality of these images and video is often shaky with blurred images and distorted sound, they capture the intensity of a live music performance. These images especially circulate on mobile platforms like Snapchat where the technical quality of images and video is less important than their capacity to convey the energy and feeling of a specific moment in time. The use of smartphones and mobile social media are woven into the action on the festival site. Images are captured and circulated at music performances, art installations, brand activations, and campsites. In 2014, attendees queued up to have a photo taken in front of a blow-up, oversized, impression of Lionel Ritchie’s head. At the entrance to the nearby Smirnoff cocktail bar a large screen encouraged patrons to use #vipublic when posting images from the bar on Instagram. Cultural intermediaries the brand invited to party in the bar used smartphones, drones and go-pros to capture images and circulate them online. Later in the night at the main stages when the US hip hop act Outkast played their hit song Hey Ya the crowd in the amphitheatre was lit up by the screens of thousands of phones pointed at the stage to capture the moment in images and videos.

The festival infrastructure – art installations, brand activations, musical performances, and mobile reception towers – stimulate and facilitate the process through which attendees translate a shared sense of sociality into digital information on media platforms. Art installations, music performances and brand activations at music festivals can be understood as market devices that link together the creative capacities of attendees, smartphones, and media platforms to generate valuable archives of digital data. The images festival goers upload generate engagement in the form of views, likes, comments, and tags; and they are available for open-ended data analysis. Art installations and brand activations at Splendour in the Grass operate in both discursive and material ways. They create an aesthetic and cultural atmosphere recognisable to participants, and they enable the translation of those affective responses into media content and data.

Converting shared sociality into machine-classifiable data

At the 2014 festival, several ‘cut out’ wooden frames were placed around the festival site. They evoked the retro frames of cartoon characters at family theme parks, with the character’s face cut out so that you could stand behind the frame, placing your head in the hole, to pose for a photograph. The frames at Splendour included mock vintage postcards, picture frames and oversized sunglasses. On a large hill leading into the festival, organisers erected large wooden ‘Splendour in the Grass’ lettering, similar to the famous ‘Hollywood’ sign. The postcards, sunglasses and Splendour sign each invited festival goers to stand in and around them, posing for photographs that could be uploaded to social media platforms. Many of these objects appeared to either ‘address’ algorithms (Hallinan and Striphas, 2014) or be ‘algorithmically recognisable’ (Gillespie, 2014).

During the 2014 festival, we collected and catalogued 1644 images shared on Instagram using #splendourinthegrass over a 24-hour period beginning at 9 pm on the first evening of the festival. The archive of images is the product of the purpose-built festival site, including the activations of sponsoring brands. In this section, we explore the extent to which the images created by festival attendees depict machine-classifiable objects and repeated perspectives. As festival attendees translate the festival into an archive of digital images, they potentially play a part in training image-classification algorithms to make judgments about their cultural life. These judgments may enable brands to make more fine-grained, real-time and targeted engagements with consumers.

We categorised the Instagram accounts that posted each of the 1644 images collected as: festival attendees, brands, bands, the Splendour in the Grass official account, and cultural intermediaries. Cultural intermediaries included influencers, models, fashion bloggers, photographers and celebrities (Abidin, 2016; Marwick, 2015). They were accounts with over a thousand followers who regularly appeared at cultural events and endorsed brands in their posts. In all, 73.5% of images were posted by ordinary festival attendees. These images generated 16.1% of total likes. Images posted by cultural intermediaries (16.4%) and brands (8.7%) accounted for approximately the other quarter of the images posted and generated 50,039 and 45,101 likes respectively (or 72.6% of total likes). Most of the content is posted by a ‘long tail’ of ordinary users and seen only by their peers, while the few ‘celebrity’ users generate the bulk of engagement. Similar to previous analyses of image collections on social media platforms (Carah and Dobson, 2016), images were more than twice as likely to feature only people who appeared female than they were to feature only people who appeared male (30.4% to 12.4%).

We then manually analysed the 1644 images uploaded under #splendourinthegrass to consider the feasibility of a supervised image-classification of the archive. Image classification refers to the range of supervised and unsupervised processes through which machines can be trained to make judgments about the content of images. Image classification is present in many increasingly ordinary aspects of everyday media use. For instance, when Facebook suggests we tag ourselves in a photo or when Google’s photo app sorts our personal library of images by faces, landmarks and objects. Supervised-image classification begins with a pre-classified dataset to train an algorithm to detect known features – such as a particular logo or object – in a group of images. Once trained, the algorithm can then be used to organise and classify previously unclassified images and provide confidence scores as to the suspected presence or absence of known objects. The approach offers a way to automate the classification of large datasets of images.

Image classification is a rapidly developing computer science field, where recent advancements in machine learning techniques such as deep neural networks have seen a profound increase in the reliability of computational judgement (Krizhevsky et al., 2012). The capabilities of image classification now extend to highly reliable identification and localisation of text labels, faces, landmarks, logos, and objects. These image-classification techniques are constantly evolving and full details – such as algorithmic architecture, system parameters, and training data – are often not open to public scrutiny. A critical account of brand culture must address the development of machine judgments like image classification and the role they play in optimising the participatory brand culture of commercial media platforms.

Supervised-image classification systems are comprised of a series of inputs, fed through a network of mathematical functions, and mapped onto a series of outputs. Recent approaches, such as ‘deep’ neural networks, contain recurrent feedback loops and many layers between the input and output. The power of supervised image-classification systems is their ability to mathematically model latent features in an input dataset working only with an indication that the feature is present. They do not require precise definition of the feature. For example, the network could be configured to reliably detect the presence or absence of the yellow letter ‘S’ in the Splendour in the Grass sign in an input image. The network will create mathematical functions that can abstract features such as the rounded edges, hue, and other patterning of such a feature. The network will not know it is a letter of the alphabet, nor that it is yellow, and roughly 1 metre high. It has no notion of such grammars, just the pixel arrangement that might lead a human to classify an image as containing this feature.

This classification ability is developed through training. Training involves repetitious feeding of gold standard human-classified data into the system. The system is given a set of images and told which ones contain the feature, but not what the feature is. The system is then given an unclassified set of images to classify, and then told which it classified correctly. The algorithm is trained by repeating this process. When first instantiated these networks are configured randomly. Training involves the modification of the internal network parameters, such as modifying weights of connections or adding or removing connections. Specific training algorithms allow the network to learn the mathematical relationships that define certain known features of an input dataset. As in the ‘S’ example above, a high value at one system output might represent a high confidence of this feature being present in an image fed into the input of the network. The network in the beginning will get many of these classifications incorrect; however, it is continually updated to minimise incorrect classifications and boost its classification accuracy. The ability of the network to achieve good results is a function of the inherent subtlety of the feature, amount of data on hand to train, complexity of the network, and computational resources available.

We determined, via our manual analysis, that 836 of the 1644 images (50.8%) of the images contained a potentially classifiable non-human object, and 927 of the 1644 images (56.3%) contained a potentially classifiable human face. Only 207 (12.5%) of the images did not contain either a recognisable object or face. The images generated over this period at Splendour are highly classifiable due to both the presence of classifiable objects on site and the relatively confined range of image making practices of festival attendees. For instance, 347 of 836 images judged to be classifiable (41.5%) featured an element of the stage: a sign on the stage, stage lights or the signage and shape of the main stage. These images are classifiable because the stage has unique features like festival logos and signage, band insignia and coloured down lights, and, because cultural practices at the festival are routine, festival attendees routinely point their smartphones at the stage, take photos and upload those photos to media platforms.

Image classification potentially enables the generation of archives that extend beyond the partial classification possible using textual metadata like hashtags and comments added by users. The set of images analysed here is of course a partial record. Informants explained that most images taken at the festival were shared via private accounts on Instagram, privately messaged, uploaded after the festival to Facebook albums, or shared on ephemeral apps like Snapchat. Therefore, we can assume that the set of 347 images of the stage are a small fraction of the total number of images taken and circulated of the stage at the festival, and they are only those images where the poster had a public account and tagged them with #splendourinthegrass. Most images taken would not be tagged with the public festival hashtag because they serve more intimate purposes in the experience of the festival among friends. But, from this public archive, a larger archive could be developed by searching for a classifiable feature of the stage across all images on the platform, and if available also using other image metadata such as date, time, and geolocation.

The same argument can be made for other unique objects on the site like Splendour in the Grass signs, brand logos and objects, art objects like Lionel Ritchie’s head, Skywhale or the Amish barn. While there are less images of these unique objects, for instance, only 95 of the 1644 images featured art objects, these images are absolutely unique to the Splendour site. While such a small number of images push the limits of many image classification approaches which rely on much larger corpora for accurate training, the addition of metadata which would be available to the platform such as location, date and social connections, could readily boost the reliability of such approaches (Johnson et al., 2015).

The images generated at Splendour in the Grass are not just examples of the symbolic narration of lived experience. While they are the product of the human capacity to apprehend, record and share lived experience with one another, they also serve as an archive of machine readable data. Image processing techniques are developing rapidly, driven by advances in classification algorithms and investment from major commercial platforms. Image classification is one part of the larger engineering efforts of media platforms like Instagram that are funded and driven by the imperatives of brands. As image classification techniques develop the capacity to identify brand logos, objects and places in our images this will enable the further optimisation of native, real-time and contextual advertising models. Image classification algorithms already have 90% accuracy in identifying brand logos, when trained on a gold standard logo dataset (Iandola et al., 2015).

Many of the unique objects on the Splendour site such as art installations, signage and brand activations may be classifiable by machines. This could enable a platform like Instagram to identify links between people, brands, and cultural experiences, tastes and affects in real time. Rather than just rely on metadata like location, hashtags or text comments, platforms could begin to programmatically respond to the content of images. As one example, Google is now using classifiable image content to inform search rankings. If a platform identified a brand object in images circulated at Splendour that might enable real-time retargeting of those users, similar users, or proximate friends. Or, it might enable follow-up next time those targets are proximate to similar cultural spaces. Recent research published in the field of computer science also demonstrates the emerging capacity of image classification techniques to be trained to determine proxies for human affects like mood (Reeece and Danforth, 2016). The commercial applications of this are simple enough to imagine. The colour, light, or blurriness of our images, combined with other contextual information like our presence at a music festival, could indicate to brands optimum times to target consumers to take advantage of mood, location, movement, or even a predicted state of intoxication.

While we have focussed above on supervised classification (networks pre-trained on human classified information), archives of images can also be used to train unsupervised image classification approaches that utilise the same deep neural network architectures. These unsupervised approaches seek to discover clusters of images based on common (or uncommon) visual features, but do not impose pre-calculated classifications or semantic data to these features. Rather, they rely on latent features in the data to act as determinants for machine classifications. That is, they do not depend on a human classifying the first archive of images in order for the algorithm to train. Here, the whole sequence from the creation of an image, to its classification and then actions a platform might take in response can unfold without any moment of symbolic articulation.

The presence of dark colours, object-dense scenes, and distinctive objects, might all be the basis of an unsupervised image cluster. Meaning, however, does not need to be imposed on any of these features. Instead, the machine denotes a pattern of action in the way that human actors mediate their experience of cultural space. The machine could detect the proximity of those practices – or more precisely, the residues of those practices in image features – to particular brands or modes of consumption and uses that as a basis to engage in targeted real time engagements. In an algorithmic brand culture, the human actor is an intrinsic part of the process by which brands engage with and generate value from cultural life, but not only because of their capacity to narrate cultural experience in a meaningful narrative. And, not because the machine needs to make sense of the human activity at a symbolic level. Image classification is one instance of machines determining patterns in rich cultural texts. While machines do not make sense of these texts on human terms, they are able to determine features that enable intervention in cultural life. Brand culture is open-ended in several senses: participatory meaning-making, the capacity of bodies to affect one another, and non-human machine judgments.

Media platforms are pursuing the development of unsupervised classification because they reveal emergent patterns within the data that may be overlooked by human users. This approach is increasingly used by platforms like Instagram, Netflix and Spotify to organise cultural material (Van den Oord, 2013). For example, the music streaming service Spotify is attempting to bypass human judgments about who will like a particular song with machine-learning techniques that analyse the frequency spectrum data of audio files to determine patterns between individual users listening habits and emerging music not at the level of genre or critical appraisal, but at the structure of the waveform itself. The machine does not judge music in the way a human user would, but it can efficiently arrive at the same outcome (Van den Oord, 2013). The designers of such systems see this as a necessary step in the evolution of the platform given the increasing volume of cultural material requiring classification. It is not that the machine is better at performing the work of classification and curation, rather programmers see it as a necessary tool to deal with the volume of media created in participatory digital media cultures. There is a subtle turn here though, being that artificial intelligence techniques can create machine-based ways of ordering culture rather than simulate human understandings of culture (Russell and Norvig, 1995). Critical media studies needs to contend with non-human actors like image-classification algorithms shaping the capacity of media platforms and brands to act in the world.

Image archives are critical to the open-ended experimentation with cultural life by brands and media platforms. Platforms don’t need to know what those images mean to human participants, rather they create non-human ways of making judgments about them that inform market processes. A friend may share a blurry image from the middle of a late night performance at the festival with other friends. The image is affecting and possibly meaningful to the people who create, view, share and engage with that image. And, it may also be classifiable by a non-human algorithm. The algorithm doesn’t make any judgement about the specific meaning shared, but might recognise features of the image that predict elevated mood, intoxication, proximity to a branded bar, or presence at a music performance. Any of these features make the image operational within an algorithmic brand culture. The platform and brand can act programmatically to engage or stimulate the person, or people like them, in that moment. Or, store the information as a template for engagement in similar cultural settings or moments in the future. The human capacity to affect here does double duty, affecting other humans immediately and training a media infrastructure to develop the capacity to affect indefinitely.

The critical simulation of image classification

While brands might have initially built themed activations at music festivals to capture the imagination and storytelling capacities of festival attendees, to weave themselves into the symbolic landscape of the festival, we must now also account for the role these objects play in the creation of data archives. Festival attendees undertake the affective labour of converting lived experience into classifiable data and training the non-human classification and exploitation of cultural life by media platforms. A critical account of branding in an algorithmic culture needs to explicate how media are learning to process, classify and make judgments about cultural life. Meaningful critical analysis and public scrutiny of branding, media platforms and algorithmic culture must unravel the infrastructure itself for examination.

We argue for the critical simulation of image classification as a necessary intervention in accounts of media platforms and the brands that fund them. The engineering of image creation, circulation and classification is critical to the commercial imperatives of platforms like Instagram. Despite this, the development of computational approaches by media and cultural researchers remains predominantly textual, descriptive and focussed on platforms like Twitter. While we agree with Highfield and Leaver (2016) that methods developed on these platforms serve as a useful starting point for analysis of image-based social media platforms, we argue that the engineering of image-classification, and the analysis of the infrastructure of platforms themselves, remain problematic blind-spots in the field. Computational approaches need to contend with not only what users do on platforms but how platforms themselves broker, classify and modulate those practices. Critical scholars need to invest in computational approaches that enable the critical simulation and analysis of the machine learning and classification capacities of platforms themselves. We need to examine not only what users do on platforms, but develop approaches that enable public scrutiny of the cultural-computational architecture of platforms themselves. This requires a shift from analysis of content that circulates on platforms to analysis of the algorithms that increasingly shape cultural life and optimise market exchange.

Conclusion

Drawing on Hayles (1999: 287), we argue that the limiting factor in an algorithmic brand culture is

not the speed of computers … or the amount of data that can be generated and stored. Rather, the scarce commodity is human attention … an obvious solution is to design intelligent machines to attend to the choices and tasks that do not have to be done by humans.

The critical issue to contend with then is the information processing power of media. To understand how brands create value, shape culture and exercise power, we do not need more sophisticated accounts of their methods of symbolic seductions, or analysis of the participatory co-creation of brand content on social media. What we need is frameworks for analysing how the use of non-human machines to make judgments about cultural life is intrinsic to capitalising on the open-ended and participatory aspects of brand culture. The fundamental business problem of platforms like Facebook and Instagram is the optimisation of human affect and attention. The strategic response to this process is investment in software and hardware engineering projects to enable machines to make more timely, contextual and operational judgments about human life. Algorithmic brand cultures are characterised by open-ended, experimental and continuous training of machines. In this formation what matters is that we make our lived experience and living bodies available to the information processing capacities of media platforms and the brands that fund them.

We are in a ‘training period’ where a critical account of algorithmic brand culture must address the parallel human and non-human engineering of computational media infrastructure. In this environment, brands will depend less on our judgments about them and more on their capacity to use machines to make automatic, real-time judgments about us. Brands that depend on our generalised capacity to affect thrive on our ambivalent, ironic, creative and critical responses to and appropriation of brand culture (Carah, 2014). The moment of value is created where we enable algorithmic media to train on the data we continuously stream, enabling them to make more fine-grained judgments about us. Media platforms are characterised by the logic of simulation, to experiment with and program reality (Bogard, 1996; Packer, 2013). The task for critical media studies then is to simulate the simulators, to build the infrastructure platforms and brands use to orchestrate human–machine relations, build it in the public domain, and subject it to meaningful forms of scrutiny.

Footnotes

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Nicholas Carah

References

Abidin

(2016) Visibility labour: engaging with Influencers’ fashion brands and #OOTD advertorial campaigns on Instagram. Media International Australia 161: 86–100.

Andrejevic

(2002) The work of being watched: Interactive media and the exploitation of self-disclosure. Critical Studies in Media Communication 19(2): 230-248.

Banet-Weiser

(2012) Authentic™: The Politics of Ambivalence in a Brand Culture. New York: New York University Press.

Bogard

(1996) The Simulation of Surveillance: Hypercontrol in Telematic Societies. Cambridge: Cambridge University Press.

Böhme

(1993) Atmosphere as the fundamental concept of a new aesthetics. Thesis Eleven 36(1): 113–126.

Carah

(2014) Brand value: how affective labour helps create brands. Consumption Markets & Culture 17(4): 346-366.

Carah

Shaul

(2016) Brands and Instagram: Point, tap, swipe, glance. Mobile Media & Communication 4(1): 69-84.

Carah

Dobson

(2016) Algorithmic hotness: Young women’s “promotion” and “reconnaissance” work via social media body images. Social Media + Society 2(4): 1-10.

Carah

(2017) Algorithmic brands: A decade of brand experiments with mobile and social media. New Media & Society 19(3): 384-400.

10.

Clough

Goldberg

Schiff

et al . (2007) Notes towards a theory of affect-itself. Ephemera: Theory & Politics in Organization 7(1): 60–77.

11.

Fisher

(2015) ‘You media’: audiencing as marketing in social media. Media, Culture & Society 37(1): 50–67.

12.

Gillespie

(2014) The Relevance of Algorithms, Media Technologies: Essays on Communication, Materiality, and Society. Cambridge, MA: The MIT Press.

13.

Hallinan

Striphas

(2014) Recommended for you: the Netflix prize and the production of algorithmic culture. New Media & Society 18: 117–137.

14.

Hayles

(1999) How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago, IL: University of Chicago Press.

15.

Hearn

(2008) Meat, mask, burden: probing the contours of the branded ‘self’. Journal of Consumer Culture 8(2): 197–217.

16.

Highfield

Leaver

(2016) Instagrammatics and digital methods: studying visual social media, from selfies and GIFs to memes and emoji. Communication Research and Practice 2(1): 47–62.

17.

Iandola

Shen

Gao

et al . (2015) DeepLogo: hitting logo recognition with the deep neural network hammer. Available at: https://arxiv.org/abs/1510.02131

18.

Johnson

Ballan

Fei-Fei

(2015) Love thy neighbors: image annotation by exploiting image metadata. In: Proceedings of the IEEE international conference on computer vision, pp. 4624–4632. New York: IEEE. Available at: https://arxiv.org/abs/1508.07647

19.

Krizhevsky

Sutskever

Hinton

(2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the advances in neural information processing systems, Lake Tahoe, NV, pp. 1097–1105. New York: ACM.

20.

Lury

(2004) Brands: The Logos of the Global Economy. London: Routledge.

21.

Lismore Regional Gallery (n.d.) Offsite. Available at: https://www.lismoregallery.org/cp_themes/default/page.asp?p=DOC-WSL-07-81-63#.WlvtyTtx1tQ (accessed 15 January 2018).

22.

McStay

(2013) Creativity and Advertising: Affect, Events and Process. London: Routledge.

23.

Marwick

(2015) Instafame: luxury selfies in the attention economy. Public Culture 27(175): 137–160.

24.

Packer

(2013) Epistemology not ideology or why we need new Germans. Communication and Critical/Cultural Studies 10(2–3): 295–300.

25.

Piccinni

(n.d.) Skywhale. Available at https://www.patriciapiccinini.net/215/48# (accessed 15 January 2018).

26.

Reeece

Danforth

(2016) Instagram photos reveal predictive markers of depression. Available at: https://arxiv.org/abs/1608.03282

27.

Russell

Norvig

(1995) Artificial Intelligence: A Modern Approach. Upper Saddle River, NJ: Prentice Hall.

28.

Terranova

(2000) Free labor: producing culture for the digital economy. Social Text 18(2): 33–58.

29.

Van den Oord

Dieleman

Schrauwen

(2013) Deep content-based music recommendation. Advances in Neural Information Processing Systems 26 (NIPS 2013). Available at: http://ai2-s2-pdfs.s3.amazonaws.com/7145/44b7cf35a3b8bdc12fb1967624a38f257a42.pdf (accessed 15 January 2018).

30.

Van Dijck

(2013) The Culture of Connectivity: A Critical History of Social Media. Oxford: Oxford University Press.

31.

Wissinger

(2007) Modelling a way of life: immaterial and affective labour in the fashion modelling industry. Ephemera: Theory and Politics in Organization 7: 1250–1269.

32.

Zwick

Bonsu

Darmody

(2008) Putting consumers to work: ‘Co-creation’ and new marketing govern-mentality. Journal of Consumer Culture 8(2): 163–197.