Abstract
In this article, Dutch artist Sander Veenhof shares his fascination for the rapidly expanding augmented reality (AR) sphere from an artistic point of view. With ‘AR’ being a merger of our physical environment and the omnipresent geodata reality, mobile AR in particular allows an unbounded reinterpretation of space, as will be addressed by zooming in on the geospatial aspects of a couple of recent projects.
With the massive spread of smartphones across a global audience, augmented reality (AR) applications and experiences are often launched with a global target audience in mind. Spatial limitations no longer exist in the augmented world; time is the only constraint. The 24/7 global audience does not spend its time online in a synchronous way. But even that is not a problem, as digital data are patient. It can be stored indefinitely, waiting to be accessed. With mobile global positioning system (GPS)-enabled technology, the physical space has become a container for interactive data, just as space is a container for physical material. The locally stored data have different ways of manifesting itself. Interacting with the data can be done in a text-based style, but, when nearby, the visual manifestation of the local data reality is what the general audience considers to be ‘AR’. More fundamental than the actual visual mixing of superimposed computer-generated content on live imagery is that AR is about the omnipresent data reality that has integrated itself into our physical domain. And with the rapid developments in cloud-based image-recognition technologies, the ties have grown even stronger. Every recognizable or definable item in our world, be it real or virtual, is getting its own data set, consisting of intrinsic data, historic data, usage data, location data and thus context data too. Everything becomes part of the interconnected ‘Internet of things’. And because interacting with the data belonging to these items does not require a physical presence, there is a significant upscale of scope, reach and thus impact of projects designed for this new hybrid reality.
At first sight, this might not sound radically new, as projects on the Internet are theoretically capable of achieving the same reach. But the Web generally lacks a precise form of location awareness. Publicizing content and attracting website visitors require either a top-down strategy or a grass-roots initiative taking enormous momentum to make itself known around the world. Whereas AR is de facto localized, present at your doorstep, in your face, within your back garden. We might be surrounded by virtual realms, and there is no proper trigger yet to let us know that we are. There’s a strong need for a proper indexer for location-based realities as well as an indicator in the menu bar of our smartphones, notifying us about the number of realities available in your current surroundings. A more probable situation is that such an indicator will be realized beyond the phone display. Smartphone AR should be seen as a transitional phase – a temporary medium, although a preliminary indicator of something fundamental: the interweaving of data and matter.
In the past, we used to have clearly separated modes of activity, such as sitting behind our desktop computers for a session of emailing. This has changed to being interrupted continuously while on the go by incoming mails and messages and reacting to those with short intervals of concentration. As the Google Glass promotion video shows, in our future lives we will have to deal with a continuous blur of triggers possibly requiring our actions. We will need filters to control the triggers, to process the massive influx of data approaching us from innumerable sources. We might be present in multiple parallel social and visual realities at the same time, universes that are both local and global.
The possibilities to develop new entertainment, gaming or advertising formats are obvious. And as art always seeks to discover the boundaries and reflect on what’s happening in society, these technical developments with their social and spatial implications present new challenges for artists, as well as great opportunities to make use of the infiniteness of the semi-digital reality. Artist groups such as Manifest.AR have been showcasing what’s possible from an artistic perspective, ever since they staged their own uninvited exhibition of AR art within the walls of the iconic Museum of Modern Art (New York, USA), in 2010. AR could be causing a shift in the dynamics between venues, curators, audience and artists. With AR, artists can choose where and how to display their work; audiences can choose whether or not to see it.
New social dynamics also arise because of the fact that local experiences can be shared with a global participating audience, even though games are appearing on a sidewalk in front of a player. Multiplayer tic-tac-toe with 100, 1000 or 100,000 people at the same time might not profit from a massive audience, but bouncing pixelated balls back and forth in a global pong game makes use of the connectedness in a more meaningful way than just sharing high scores. Collective virtual outdoor sculpturing turns the whole world into a massive creative environment, allowing people to work on one single Gesamtkunstwerk that exists at a multitude of locations at the same time, when in collaborative building mode.
The outdoor theatrical experience Meet Your Stranger is another example of a project that is location independent but at the same time highly localized, thanks to mobile and AR technology. It consists of a number of pre-written roles and short scenes available to a worldwide audience. A detection mechanism helps participants scan for fellow participants in their vicinity, then brings them together using an unconventional (in this digital era that is) meeting mechanism. When near enough to each other, participating ‘actors’ receive a gesture pattern that has to be made in the air, holding up the phone. This visible gesture overcomes – in a very analogue way – the impreciseness of GPS technique. In addition to a dialogue text appearing on their mobile phones, the device is also used to view a virtual decor that surrounds the actors during their scene, showing characteristic props relevant to the contents of the scene.
Despite the global scale of projects such as Meet Your Stranger, the actual encounters can in contrast feel very local and personalized. Especially when experienced through social interaction with real people, not with devices. It is a misunderstanding often repeated by technophobes that smartphone technology shuts us off from the world we live in. It is true that there’s a risk of being connected to the whole world while ignoring what’s around us. However, when applications succeed in offering a meaningful interchange with local data or ‘augments’, mobile phones do reconnect us to our direct surroundings. Not to forget the social encounters that happen unintentionally, when technology does not work or is not understood: people helping each other to configure apps on their devices and then sharing the fascination for a synchronised AR appearance, for example. A moment of experience sharing – which is starting to become uncommon in today’s society where we stare at our 24/7 personalised media display continuously.
Assuming that society will gradually develop a proper social code for dealing with technology that offers us continuous on-the-go connectivity, it is time to look through the technology, literally. As with any storytelling medium, the presentation layer of AR has to be bypassed in order to get to the level of accepting a fictional story or a conceptual creation. Assuming that technology will progress further and AR of the future will be presented to us through an invisible unnoticeable mediator possibly called Glass 2.0, there will be concepts that still go beyond what can be seen and interpreted as ‘really there’. The world’s biggest interactive artwork BiggAR is such a conceptual piece. Consisting of 7,463,185,678 virtual cubes floating in the skies across the globe, the artwork can never be seen as a whole. When viewing just a part of it, the full scale of the earth-encapsulating maze has to be imagined. It might help that the artwork registers its most recent alterations by people across the globe. The colour of all cubes can be changed by anyone pointing his or her phone at the blocks and choosing a new colour for all of them at once. Looking at the sky full of cubes in Amsterdam and knowing that a person in Melbourne has just changed the colours 7 min ago helps to understand that this is a global super gadget.
Although people pointing into the sky in search of AR are still not numerous, this creative universe is gaining ground. And even when there’s no real person in sight, one is never alone again in the spatially flexible, globally shared universe. It takes some advanced conceptual willingness to immerse oneself in, but the Dance.AR choreography for ‘persons holding an iPhone’, potentially brings the whole world population together into one harmonious choreography, thanks to a time-synchronized, AR instruction cube, floating around the heads of Dance.AR participants. Originally inspired by the quirky looking movements of the lone lunatics immersed in their own secret, imagined AR world, the choreography makes the individual experience of AR into a collective one, because the chances are high that in the global audience someone is joining you right now, wherever you are. AR brings you a globally local world to explore, right at your feet.