Studying the Elusive Experience in Pervasive Games

Abstract

Studying pervasive games is inherently difficult and different from studying computer or board games. This article builds upon the experiences of staging and studying several playful pervasive technology prototypes. It discusses the challenges and pitfalls of evaluating pervasive game prototypes and charts methods that have proven useful in previous research. The aim is to open discussion on the situated methodology of qualitative study of evaluating and researching pervasive play.

Keywords

ambiguity design design-play relationship elusiveness evaluation methods experience game genres game studies pervasive games pervasive play play experience qualitative study research methodology situated methodology

Many reasons to study play experiences exist. We may be looking to understand what people learn from games or how they learn to play. We may be looking at the social dynamics, for example, how people collaborate or compete in a game. In this article, we focus on studies that look at what people do when they play games, scrutinizing the relationship between design and actual play.

We focus on a relatively new genre, pervasive games, which has recently gained attention as a growing cultural phenomenon. These games are not played on computer screens (even if screens can be used in games) and seldom on predefined places or set times. Although these games are interesting in themselves, they are also connected to contemporary culture. The recent proliferation of pervasive games exists at the conjunction of three cultural shifts: the struggle over public space, blurring of the real and the fictive, and the rise of ludus in society (Montola, Stenros, & Waern, 2009). Studying these games helps us understand wider trends in media culture, urbanism, mobility, and the player-participant-spectators tension in such phenomena.

Although pervasive games share many properties with computer and board games, they possess fundamental qualities that make them inherently hard to study. This article builds on our experiences of staging and evaluating numerous prototypes (e.g., Ahmet & Waern, 2011; Bichard & Waern, 2008; Fischer, Lindt, & Stenros, 2006; Jonsson, Montola, Waern, & Ericsson, 2006; Montola et al., 2006; Stenros, Montola, Waern, & Jonsson, 2007). We first explain the fundamental issues involved and then chart practical approaches that benefit the study of pervasive games. The aim is to discuss a situated methodology (Seale, Gobo, Gubrium, & Silverman, 2004) of evaluating and researching pervasive play to better understand design, play, and the connection between the two.

The Pervasive Game Experience

Pervasive games have been defined according to their technological enablers and to their gameplay (Nieuwdorp, 2007). They have been called by numerous names, such as hybrid games, location-based games, and mobile games. In this article, we build on our earlier work in Pervasive Games: Theory and Design (Montola et al., 2009), seeing pervasive games as a broad category of games in which what is actually in the game and what is outside it is somewhat ambiguous. The play area or the time frame may be ambiguously defined, or players may drift in and out of the game in ambiguous ways. Pervasive games expand the contractual magic circle of play (Montola et al., 2009) that is characteristic to other types of games (see Huizinga, 1938; Salen & Zimmerman, 2004).

Pervasive games include numerous subgenres. Genres such as treasure hunts, assassination games, pervasive larps, and alternate reality games are already established as game cultures. Others, such as smart street sports and playful public performances are mostly created by researchers, artists, and marketers (Montola et al., 2009). In practice, these games range from small mobile phone games such as INSECTOPIA to large street performances such as UNCLE ROY ALL AROUND YOU and transmedia games such as SANNINGEN OM MARIKA.

Unlike some previous definitions (e.g., Magerkurth, Cheok, Mandryk, & Nilsen, 2005), we do not assume that pervasive games always make use of pervasive technology: The campus assassination games, such as KILLER, emerged in the 1960s (Johnson, 1981), and the predecessors to GEOCACHING are easily traced back to mid-19th century (Hall, 2004). Numerous contemporary games are also informed by work done in the realm of art (Adams, 2009; de Souza e Silva & Hjorth, 2009, Montola et al., 2009; also Rodriguez, 2006). Still, the recent advancements in wireless technologies have made designing, staging, and restaging pervasive games much easier. Technology has also expanded the design space for such games, adding the opportunity to “enchant” physical space using virtual content (Waern, Montola, & Stenros, 2009) and to automate cumbersome game administration.

What makes pervasive games fun, and different from other types of games, is their ambiguity. Pervasive games blur the boundary between play and everyday life (Montola et al., 2009; Nieuwdorp, 2007). They empower the players to see their everyday surroundings through ludic lenses, transforming the quotidian city into a fun and exciting site (McGonigal, 2006a; Montola et al., 2009). As they are embodied experiences, they are subject to contextual coincidences. Experiences such as changing weather conditions, mistaking players for nonplayers, or noticing game-related advertisements are often reported as the key moments of a game. Players have physical and tangible experiences, which are often social as they encounter other players and bystanders. They are activities where the players’ physical and mental resources all can come in play in order to address the game challenges. These aspects also make them difficult to study.

Whatever the reason we have for studying a pervasive game, studies tend to face common challenges, most often arising from the way the game is enacted in the everyday world. Designing a game always means designing an activity. The game’s rule system (and related paraphernalia) must provide a platform for a meaningful activity to emerge:

As a game designer, you are tackling a second-order design problem. The goal of successful game design in meaningful play, but play is something that emerges from the functioning of the rules. As a game designer, you can never directly design play. You can only design the rules that give rise to it. Game designers create experiences, but only indirectly. (p.168) (Salen & Zimmerman, 2004, p.168)

Although this dilemma affects all games, in pervasive games, it is exacerbated by the unpredictable complexity of the world outside the game. It is not only just the game that invades everyday life, but also the ordinary life that invades the game. Thus, it is theoretically possible to experience anything life has to offer during a game. Yet the game needs to be a self-supporting and self-motivating activity as the decision to keep playing is continuously reiterated—especially in games where players might be spread individually around town. In order to fully reflect this activity-oriented nature, the study of pervasive games needs to look at the experience of enacting the game, play.

Approaches to Pervasive Game Studies

Researchers studying pervasive games typically come from one of the following three fields: ubiquitous/pervasive computing, digital game studies, or cultural studies. These fields have had a tremendous influence on the perception of pervasive games. Although each field has much to offer, they have their blind spots as well.

Research founded in ubiquitous and pervasive computing has mostly approached the field through prototypes. According to Montola (2011a), this research is driven by the desire to develop new technology, and games are of secondary interest. McGonigal (2006b) considers such prototypes as a genre of their own, ubicomp gaming. These researchers “use ubicomp technologies to put games into new objects and spaces, and to use the medium of games to put ubicomp technologies into more contexts and into the hands of more users (p. 60).” From the perspective of studying game design and play experiences, the immaturity of the prototype technologies and the low emphasis on innovative game design often limit the value of studying such games (see Koivisto & Ollila, 2006; Montola, 2011a)—the focus is on the technology.

Pervasive games have been approached from the point of view of “the social and cultural implications of mobile technologies in contemporary society” (de Souza e Silva & Sutko, 2009). These studies contemplate the social and societal changes in urbanism and mobility that pervasive games foster. This field usually operates on a higher level of abstraction, seeking larger trends, and as a result, sometimes, ignores the individual player, his or her experience, and his or her interpretation—or even that the interesting activity is created by playing games.

Digital game studies takes an interest both in the game design and the play activity, but it also tends to approach games as systems (see, for example, Juul, 2005; Montola, 2011b [this issue]; Salen & Zimmerman, 2004), which makes it possible to study games also as objects and artifacts that can be studied independent of players (Myers, 2009). However, if the individual player and his or her experience are bracketed, then much of what constitutes play in pervasive games is dismissed. Mäyrä (2008) has pointed out that no game exists without a player. In pervasive games, the affordances offered by screen interaction are often less important than what players do beyond it and even beyond the designed game (Reid, 2008). The impact of players’ seemingly nonludic actions can radically change both their experience and the game itself. As pervasive games can be extremely physical (see, for example, Ericsson, 2003), the embodied sports research could prove to be a relevant reference point.

We have also seen exploratory game design research, in which technology—or other design elements—have been used to create experiments with new forms of play—be it for entertainment, artistic experience, education, or simulation purposes (see, for example, Adams, 2009; Sweeny & Patton, 2009).

The approach that we support is an amalgam of the above approaches. Such an approach has primarily been employed by sociocultural studies, exemplified by work conducted on the alternate reality game (ARG) genre. Some of these studies show a strong connection to play and design (Dena, 2008; McGonigal, 2006b), whereas others operate on a higher level: they analyze what these games mean based on the textual fragments left behind (Taylor & Kolko, 2003). However, all are anchored in the study design as well as play and players.

We emphasize that this article explores the evaluation of pervasive games as games. If the gaming activity is to be used for teaching, training, and communicating a political message or an artistic agenda, those functions need to be addressed with additional methods.

Central Challenges for Studies

The interplay between design and play is central to understanding all play experiences. Documenting and analyzing the design, the individual sessions of games, and the player activities and experiences in these are key in cultivating this understanding. Although this is challenging for all types of games, for pervasive games, the game-as-played can only be observed in the actual context of play, where the borderline between game and the world outside is blurred. This poses unique challenges.

First, since the ordinary invades the ludic, and this is a central part of the experience, the play activity is governed by more than just rules and goals. Many of the players’ decisions are guided by experiences in the everyday world, rather than by the rules and objectives of the game. Sometimes players enjoy taking the game into ordinary life situations, and sometimes, they are pleasantly surprised by coincidences between the game and the everyday world. At times, the experiences clash.

This is tied to the social contexts outside the frame of playing. The bystander experiences must be carefully considered. Some pervasive games involve and affect bystanders who are unaware of the playful nature of the activities they are observing (Montola & Waern, 2006; also Benford, Giannachi, Koleva, & Rodden, 2009). In general, the fact that pervasive games are socially expanded is ethically the most challenging aspect of their design (Harvey, 2006; Montola et al., 2009; Niemi, Sawano, & Waern, 2005). In a sense, bystanders are (unwilling) players in pervasive games, and they can be offered numerous roles: as audience, judge, obstacle, invited to play, and so on. Designers of pervasive games must consider the bystander experience, and empirical studies should take them into account. Such outsider probes also contribute to the discussion on the ethics of pervasive gaming. However, it is often very difficult to track down the random bystander, as they might not even know that they have participated in a game.

Studying games on the move is difficult

It is much harder to observe and capture the experience of a moving player than a stationary one. Whereas the computer games researcher can mount a set of cameras to record play (see, for example, Linderoth, 2004), the pervasive games researcher must be prepared to follow the players wherever they decide to go—without intruding on the game experience or breaking privacy regulations, for example, by filming outsiders without permission.

Core elements of gameplay occur outside the reach of technology even in the most technology-heavy pervasive games. Although PRISONER ESCAPE FROM THE TOWER (Reid, 2008) appears to be a mobile phone game, the central part of the fun is being in the Tower of London, sneaking to avoid the yeoman guards. The spatiophysical experiences of being and moving (even waiting; see Calleja, 2007) do not show up in logs and nor does random coincidences between the physical world and the designed game content (Reid, 2008). Even when a pervasive game has a computer interface, it cannot be reduced to a predictable case of program use.

Finally, the open nature of pervasive games leads to particular challenges when designing and studying them. Play-testing pervasive games is extremely challenging. In order to see how they function, they must be tested in practice. Although core technologies and features can be tested with mock-ups (e.g., Koivisto & Palm, 2005), elements involving outsiders and emergent coincidences are only seen when the game is actually played. This is not unique to pervasive games, the same applies to MMORPGs and games where a sufficient mass of players are needed before the proper play patterns emerge.

As these games must be fully formed and robust enough to endure testing in the real world, the iterative game design (e.g., Salen & Zimmerman, 2004) loop of designing, testing, evaluating, and redesigning is often impractical. Instead, development happens on the run. Games such as MOMENTUM (Stenros, Montola, Waern, et al., 2007, Stenros & Montola, 2009) can adapt their later stages based on the lessons of early gameplay, just like the Facebook games that are stuck in the perpetual beta of constant tweaks. Alternatively, the iterative cycle can be long and slow: Prototypes are designed, played, and then studied, with the study influencing the design of future games. For a large range of pervasive games, this goes so far as to erasing the difference between the game and a play session (Björk & Holopainen, 2005): The game is either only staged once, or every staging is so different that it is not quite the same game.

Approaches to Play Experience

Pervasive games offer a whole range of gameplay experiences, some of which rarely occur in digital games. Similarly, others that often appear in computer games make less sense for pervasive games. They are often not situated in a dedicated and stable game arena.

Although it has been criticized (e.g., Juul, 2004), the concept of flow (Csikszentmihalyi, 1975) seems to largely dominate traditional digital game design (e.g., Bateman & Boon, 2006; Fullerton, Swain, & Hoffman, 2004; Rabin, 2005). The flow concept captures the experience of being fully engrossed in an activity and is presumably induced by balancing the player’s abilities and the challenges provided by the game. A similar term is immersion (Brown & Cairns, 2004; Jennett et al., 2008), which focuses on the sensory and cognitive immersions in a (virtual) environment. The immersion model proposed by Ermi and Mäyrä (2007) adds a third alternative, imaginative immersion.

Certain pervasive games can offer strong moments of flow and immersion, yet neither model can capture the full gameplay experience. Whereas computer games may offer the possibility for complete indulgence, pervasive game players constantly shift in and out of the play experience, the frame (Fine, 1983; Goffman, 1974; Stenros, Montola, & Mäyrä, 2007) of the game. As these games are often played in appropriated places, the player needs to have a dual awareness, to be aware of the game and to pay attention to the nonstatic environment (see also Mäyrä & Lankoski, 2009). Indeed, the random coincidences between the two frames are often reported as the most fun moments (Montola, 2007; Stenros, Montola, Waern, et al., 2007). Thus, pervasive games have a sort of built-in Verfremdungseffekt. This makes models such as flow and immersion insufficient for capturing the experience of playing pervasive games.

Methods for Data Acquisition

Although designing a full study requires more than choosing the methods for data gathering, the data determine the scope of the study. Thus, we will structure our discussion around data acquisition. The methods charted in this section originate in social sciences and psychology, and they represent good experimental practices for design-oriented research in general. Our focus is on how these methods are usefully applied to pervasive games, highlighting both their potential and their shortcomings for particular purposes.

Playing and Observing the Game

In order to understand how a game functions and what it is like to play a game, it is imperative to play the game: Studying participatory culture requires participatory methodology. The importance of this cannot be overstated. No amount of reading about playing and participation in trial runs will compare with the firsthand experience of playing the game together with actual players.

Playing a game may not seem like a proper scientific method of extracting knowledge: It is hard to measure, and the insights gained seem less sound than, for example, interviews with “real” players. Yet in game studies its importance is widely recognized (Aarseth, 2003; Mäyrä, 2008) to the point that discussion has moved to addressing how playing needs to be done (Karppi & Sotamaa, 2011 [this issue]; Kücklich, 2007). The only way to access a game is by playing it. It is instrumental in finding the key issues with a particular game.

Playing also gives a possibility for participatory observation: The researcher can act as an observer of the social interplay in and around the game. Thus, it makes sense to make time for a researcher to participate in a comparable expanded game experience (Kultima & Stenros, 2010) as the other players. This means that a researcher should go through the same preliminary steps as the players, spend time with them during possible downtime, and participate in possible after-game activities. It is one thing to access the game and another to access the social frames (Goffman, 1974) of the game and the players.

The participatory observer of play must play actively, instead of just “watching” or “helping occasionally” (Delamont, 2004). This creates tension with the researchers’ role as an observer and evaluator, forcing the researchers to play almost fully—they should try not to steer the game or draw too much attention to themselves. In order to not affect the game too much, the researchers’ skill level also needs to resemble a “typical player.” Another difference to traditional ethnography is that the observer can rarely pause the game to interview players. Even when players are aware of being observed, they do not appreciate being interrupted. Field notes, of course, are still important.

One ethical dilemma¹ is that the observer will often witness more authentic reactions, if his or her role as a researcher is not disclosed. The research group needs to carefully consider whether to use undisclosed observation or not. One way to side step the problem is to inform the players of an observer in their midst without informing them of who it is. The risk is that, in the already playful context of the game, guessing who this researcher is turns into a game in itself.

Documenting Design and Orchestration

Documenting the design of a pervasive game can vary from describing a simple mobile phone application (e.g., Peitz, 2009) to collecting data from various sources (e.g., Montola, 2007; Stenros, Montola, Waern, et al., 2007; Waern, Montola, et al., 2009). In this section, we document the most important sources.

Designer interviews are, especially for the more complex games, very important. If possible, these should be carried out already during the design phase. It is not always possible to deduce the design from the play experience—and the design documents often neglect to mention “taken-for-granted” knowledge. Furthermore, the design intentions and goals of the design team are often not easily discernable from playing the game or even reading the game design documentation. Denward (2008) has even shown that participant observation of a production can be crucial in understanding the goals and structure of a game design.

If the researchers have access to the design process, it may be possible to include design features that address certain research questions (e.g., Bichard & Waern, 2008). However, note the following pitfalls of such methods that Koivisto and Ollila (2006) have listed: design by committee, too many research questions, and immature technology. Weighing game design quality against research goals and technological constraints can become a source of conflict in a research team. Hence, the development process will benefit from having all these goals laid out early in the process.

Once the game is set to run, the focus of the observation should not be moved entirely to the players. The game masters should also be observed. Even technology-heavy pervasive games tend to have runtime staff. The runtime game masters will often not be able to express how they ended up making certain decisions after the game has ended. This makes it important to study them in the act and interview them on the spot when something interesting happens. In general, having a firm grasp of how a game is run helps bridge the analysis of the design of the game structure and the play experience.

Participating researchers tend to, to some extent, become included in the game mastering team. This leads to ethical dilemmas regarding data: Can the researchers use all game mastering information as data (for example, player communications) in their analysis? A practical solution is to include the research into the player contract that most pervasive games should use in any case.

The “objective distance” of removing oneself from players and game masters to a separate observation location usually means having cripplingly little data and a very meager context for analyzing it (see, for example,. Fischer et al., 2006, Fischer, Lindt, & Stenros, 2007).

Postgame Interviews and Surveys

The simplest way of accessing the experience of playing a game is by interviewing the players after a game has ended. For nondigital games that offer no possibility for researchers to participate, this is also often the only available method. For pervasive games with short duration, the postgame interviews and surveys may work very well (e.g., Benford et al., 2004; Rowland et al., 2009).

The major problem with doing retrospective interviews is that they do not capture the experience as it happens. The experiences are narrativized and turned into stories in the context of what happened later in the game: A spectacular payoff can flip the opinions on a repetitive boring task. The narrative reframing changes the meaning of the experience and thus the very memory of the experience.

Even so, players’ stories are important for understanding how they experienced the games—at least after the game has ended. However, in long games, people start to forget their experiences. If players hear the stories of other players, this also influences their narratives. If the game experience depends on the players’ skills, players may be reluctant to report bad experiences because this would mean that they were bad players. Finally, if they make huge investments in a game, be it in terms of money, time, or status, they are bound to oversell their experience: “I spent so much time with this game so it has got to have been good” (see also Schell, 2009).

Taken together, these factors can foster a consistent, but false picture of what people experienced in the game, a postgame lie. The phenomenon is not created out of malice or deceptive intent, but constructed in the process of reflecting on the game. It can affect postgame surveys as well as interviews, but especially group interviews tend to foster the fabrication of narratives. It is often a good idea to complement postgame interviews with individual surveys.

In some game cultures, players produce written debriefs postgame for the game masters (see, for example, Hopeametsä, 2008), in some others—such as the ARG community—the players discuss the game extensively online. Debriefs written for game masters can provide interesting data, even though such documents are prone to contain some fabrications. In addition to the postgame lie, players may sacrifice details to fit the form of the debrief text or flatter the game organizers.

Participatory play and observation are particularly valuable methods when used in concert with postgame interviews. Playing the game allows the researcher to ask the right questions and contextualize the answers, although interviews allow a more general view on experiences on the game.

The postgame interview is the chief method for capturing the experience of nonplayers. The challenge here is finding interview subjects. While some games have used informational business cards (McGonigal, 2006b; Stenros, Montola, Waern, et al., 2007), pieces of paper with researcher contact info handed out to bystanders, we are not aware of any situation in which a previously unaware participant has used one to contact a researcher.

Tapping Into the Runtime Game Experience

The risk of encountering a postgame lie makes it desirable to tap into the player experience while the game is underway. In long-duration games, it is sometimes possible to interview players during play (see, for example, Stenros, Montola, Waern, et al., 2007). Although such interviews and surveys also show effects of contextualization, in a more immediate context, they are much less likely to suffer from the postgame lie, especially if players are interviewed individually.

An alternative is to use a tool that encourages the players to self-report on their activities and experiences during the game. A runtime self-reporting system need not be textual. The Babylon system (Waern, Ahmet, & Sundström, 2009) uses an easy, graphical interface where the players quickly log their feelings and activities. The tool also includes a visualization tool that can be used during postgame interviews to aid recall of particular events in the game. One study (McMillan, Morrison, Brown, Hall, & Chalmers, 2010) shows promising results for giving players game benefits for providing qualitative data.

Runtime reporting can also be done diegetically, as a part of the game. The method can be recommended for game-mastered, story-driven games, such as MOMENTUM. Such reports create an excellent understanding of how much players have seen and understood of the game content and are a valuable resource for game mastering. From the perspective of studying the experience, the problem with diegetic reports is that they often become strongly narrativized; players tend to write literary short stories based on their game experience. They can still provide useful information: If players are confused or slowing down, this will almost certainly show up in in-game reports. Some players will stop reporting, why frequency of reports can be a good indicator, whereas others will report that not much has happened or that they are confused (see Stenros, Montola, Waern, et al., 2007).

Runtime reporting may also affect player engagement: As they are forced to reflect on the play experience, it becomes more difficult to be captivated by play (Stenros, Montola, Waern, et al., 2007). For example, the Babylon users not only found it an easy and convenient tool, but also that using it slightly disrupted the play experience (Waern, Ahmet, et al., 2009).

Indirect Access to the Runtime Experience

In addition to direct observation, numerous indirect methods for gathering data on play activity exist. A common method of orchestrating pervasive games is for the game masters to use spies to track the players. These same people can also be valuable sources of information for a researcher. Informants (see, for example, Stenros, Montola, Waern, et al., 2007) participate as players, but work as a connection between players and game masters. They often know more about the game context and objectives than the players do. Specially instructed participants (see, for example, Bichard & Waern, 2009) play special characters as part of the game mastering team, with the objective of informing the players or influencing them to do a particular thing. Runners (see, for example, Bedwell, Schnäderlback, Benford, Rodden, & Koleva, 2009) spy on the game group, but try to stay hidden from it. All three roles will gather a certain understanding of the ongoing game activities that is much less susceptible to effects of narration. They can act as additional pairs of eyes for the researchers and can be useful in identifying moments of particular importance that can be brought up in postgame interviews.

Another method is using activity logs. Technical logs can yield a wealth of statistics, but although they can seem impressive, translating them into knowledge is difficult. The SMS game DAY OF THE FIGURINES was evaluated largely on the basis of detailed logs of player activity (Benford, 2007) in the form of messages sent and received. Although these rich logs generated useful information in terms of activity levels and game progression, the player experiences were not reliably captured. In general, activity logs tend to be the most meaningful when used together with other information sources—especially with qualitative data.

Surveillance is another way of producing logs. Tapping into player communication is the most reliable means of capturing player confusion and frustration, and it is also a great source for understanding the gameplay activity. The performance effect is low: While playing, players tend to forget or ignore that their communication is being recorded. However, both technical and privacy issues must be considered. The INTERFERENCE study (Ahmet & Waern, 2012) monitored player discussions through a portable recorder carried by one of the players. This simple setup worked, since INTERFERENCE is designed to make the players stick together.

If players stay in dedicated areas, it is possible to mount fixed surveillance cameras and microphones. This does not work nearly as well: It is difficult to use fixed equipment in a way that provides useful data, the video quality is typically low, and monitoring the cameras requires a lot of work (Jonsson et al., 2006). The privacy regulations typically restrict monitoring of dedicated play areas, so when the players move, stationary cameras become useless.

Some designs allow the evaluators to tap into the communication between players through the game setup itself. In UNCLE ROY ALL AROUND YOU (Benford et al., 2004), online players collaborated with on-street players using a communication channel that was part of the game, and in PROSOPOPEIA (Jonsson et al., 2006) and MOMENTUM (Stenros, Montola, Waern, et al., 2007), players communicated with game masters through diegetic communication channels. Although performative, these kinds of messages can be valuable and were also used in the evaluation of these games. ARGs tend to be easy to study this way, as a big part of the gameplay happens on a few web forums (McGonigal, 2006b). Such forums are monitored by game masters who fine-tune the game design based on what players are talking about. The players are also aware of this, which makes the forum a site for metagaming.

The communication can take place in a wider media context as well. Media responses, blog posts, and community discussions are valuable resources—indeed, many worthwhile sociocultural takes on pervasive games have been constructed based mostly on players’ public self-reporting and reflection. If the game is publicized heavily and publicly accessible, you might have the chance to gather player and spectator comments from public resources: media reviews, blog posts by players, and online community discussions. For large-scale pervasive games, these sources are invaluable in capturing the general reception and interpretation of the game. That being said, it is still recommended to do a postgame survey targeting the players.

Sometimes publicity can be unplanned, unwanted, and negative from the perspective of the organizers. Such cases can open up interesting research opportunities, creating a possibility to compare and contrast the game design, player experience, and bystander perception (see, for example, VEM GRÅTER in Montola et al., 2009).

Conclusions

In this article, we have outlined, based on our 6 years of experience in studying pervasive games, a basis for a situated methodology of studying pervasive games and play, concentration on the design and playing of these games, and the way the two connect. It is our hope that these concrete guidelines and theoretical groundings help others avoid the mistakes we have made while striving for a deeper understanding of pervasive play. Although our focus has been on studying these games for their own sake, the basic methodology remains the same when games are developed for particular purposes, such as educational games.

The difficulty of studying pervasive games is inherently connected to their highly ephemeral nature: They are not replayable, reproducible, or accessible in retrospect (Montola, 2011b). The elusive experience cannot be studied as play-tests or use-cases: They must be staged under real settings.

The ephemeral nature of pervasive games impacts the generalizability, reliability, and validity of the research. As some games cannot be replayed, replicating results is difficult. Long trials of large-scale prototypes (e.g., McMillan et al., 2010) are one way of tapping into the elusive experiences. However, in practice, the understanding of pervasive game design often comes from many detailed studies, synthesized through comparative metastudies (e.g., Montola, 2011a) that can compare findings and substantiate them through repeated occurrences of phenomena in new studies. These syntheses need well-executed descriptions and analyses on individual games to generalize on game design, meaning of the experience, or the social and cultural implications of pervasive play.

It is worth noticing that almost all of the methods discussed here are inherently qualitative. They favor in-depth detailed studies of the experiences of a few players, rather than the average experience of a large number of players.

Footnotes

Authors’ Note

The authors studied many of the pervasive games mentioned in this article during a 3.5-year period when the authors were working in the EU-funded Integrated Project of Pervasive Games (IPerG).

Declaration of Conflicting Interests

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

Bios

Jaakko Stenros, MSocSc, works as a game researcher and a doctoral candidate at Game Research Lab (University of Tampere). He is an author of Pervasive Games: Theory and Design (2009) as well as an editor of three books on role-playing games, Nordic Larp (2010), Playground Worlds (2008), and Beyond Role and Play (2004). He lives in Helsinki, Finland.

Contact: jaakko.stenros@uta.fi.

Annika Waern, PhD, works as senior researcher at Mobile Life, Stockholm University, and as studio director for the Game Studio at Interactive Institute. Working together with game designers, media theorists, artists, and technicians, her research today is highly multidisciplinary with a focus on theory, design, production, evaluation, and analysis of technology-supported pervasive games.

Contact: annikaw@dsv.su.se.

Markus Montola, MSocSc, has worked as a game researcher at University of Tampere and at Nokia Research Center. He’s an author of Pervasive Games: Theory and Design (2009) and an editor of three books on role-playing, Nordic Larp (2010), Playground Worlds (2008), and Beyond Role and Play (2004). He is currently finishing his dissertation on pervasive role-playing.

Contact: markus.montola@uta.fi.

References

Aarseth

(2003). Playing research: Methodological approaches to game analysis. Paper presented at the 5th Digital Arts & Culture Conference, Melbourne, Australia.

Adams

(2009). Mixed reality arts. In Montola

Stenros

Waern

(Eds.), Pervasive games. Theory and design (pp. 236-240). Amsterdam, Netherlands: Morgan Kaufmann.

Ahmet

Waern

(2011). The engagement of a play collective: Collaborative play in a pervasive game. Manuscript under review.

Bateman

Boon

(2006). 21st century game design. Hingham, MA: Charles River Media.

Bedwell

Schnäderlback

Benford

Rodden

Koleva

(2009). In support of city exploration. In Proceedings of the 27th international conference on human factors in computing systems. Boston, MA: ACM.

Benford

(Ed.). (2007). Evaluation of day of the figurines II (IPerG Deliverable D12.6). Retrieved from http://iperg.sics.se/press1.php

Benford

Flintham

Drozd

Anastasi

Rowland

Tandavanitj

. . .Sutton

(2004). Uncle Roy all around you: Implicating the city in a location-based performance. In Proceedings of the ACE conference. Singapore: ACM.

Benford

Giannachi

Koleva

Rodden

(2009, April 4-9). From interaction to trajectories: Designing coherent journeys through user experiences. Proceedings of CHI 2009. Boston, MA: ACM.

Bichard

J.-P.

Waern

(2008). Pervasive play, immersion and story: Designing interference. In Proceedings of DIMEA ’08. Athens, Greece. ACM New York, NY, USA.

10.

Björk

Holopainen

(2005). Patterns in game design. Hingham, MA: Charles River Media.

11.

Brown

Cairns

(2004). A grounded investigation of game immersion. In CHI ’04 extended abstracts on human factors in computing systems (pp. 1297-1300). New York, NY: ACM.

12.

Calleja

(2007). Digital game involvement. A conceptual model. Games and Culture, 2, 236-260.

13.

Csikszentmihalyi

(1975). Beyond boredom and anxiety. The experience of play in work and games. San Francisco, CA: Jossey-Bass.

14.

DAY OF THE FIGURINES. (2005). [Developed by Mixed Reality Lab]. United Kingdom. Blast Theory.

15.

Delamont

(2004). Ethnography and participant observation. In Seale

Gobo

Gubrium

J. F.

Silverman

(Eds.), Qualitative research practice (pp. 217-229). London, England: SAGE.

16.

Dena

(2008). Emerging participatory culture practices: Player-created tiers in alternate reality games. Convergence, 14, 41-57.

17.

Denward

(2008). Broadcast culture meets role-playing culture: Consequences for audience participation in a cross-media production. In Proceedings of IAMCR 2008. Stockholm, Sweden.

18.

de Souza e Silva

Hjorth

(2009). Playful urban spaces: A historical approach to mobile games. Simulation & Gaming, 40, 602-625.

19.

de Souza e Silva

Sutko

D. M.

(Eds.). (2009). Digital cityscapes: Merging digital and urban playscapes. New York, NY: Peter Lang.

20.

Ericsson

(2003). Enchanting reality: A vision of big experiences on small platforms. Level Up Conference Proceedings. Digra.

21.

Ermi

Mäyrä

(2007). Fundamental components of the gameplay experience: Analyzing immersion. In De Castell

Jenson

(Eds.), Worlds in play: International perspectives on digital games research (pp. 37-53). New York, NY: Peter Lang.

22.

Fine

G. A.

(1983). Shared fantasy: Role-playing games as social worlds. Chicago, IL: University of Chicago Press.

23.

Fischer

J. E.

Lindt

Stenros

(2006). Epidemic menace II evaluation report (IPerG Deliverable D8.8 part II). Retrieved from http://iperg.sics.se/press1.php

24.

Fischer

J. E.

Lindt

Stenros

(2007). Evaluation of crossmedia gaming experiences in epidemic menace. In Proceedings of the 4th International Symposium on Pervasive Gaming Applications PerGames 2007. Salzburg, Austria.

25.

Fullerton

Swain

Hoffman

(2004). Game design workshop: Designing, prototyping, and playtesting games. San Francisco, CA: CMP Books.

26.

Goffman

(1974). Frame analysis: An essay on the organization of experience. Boston, MA: Northeastern University Press.

27.

Hall

(2004). The letterboxer’s companion. Helena, MN: Globe Pequot Press.

28.

Harvey

(2006). The liminal magic circle: Boundaries, frames, and participation in pervasive mobile games. Wi: Journal of the Mobile Digital Commons Network, 1. Retrieved from http://wi.hexagram.ca/1_1_html/1_1_pdf/wi.1.1.alison.harvey.pdf

29.

Hopeametsä

(2008). 24 hours in a bomb shelter: Player, character and immersion in ground zero. In Montola

Stenros

(Eds.), Playground worlds (pp. 187-198). Ropecon ry, Helsinki.

30.

Huizinga

(1938). Homo Ludens. Proeve Ener Bepaling Van Het Spelelement Der Cultuur [Homo ludens: A study of play element in culture]. Boston, MA: Beacon Press.

31.

INSECTOPIA. (2005). [Developed by Peitz, J.]. Gothenburg, Sweden: GAME Studio.

32.

INTERFERENCE. (2008). [Developed by Bichard, J. P., & Waern, A.]. Stockholm, Sweden.

33.

Jennett

Cox

A. L.

Cairns

Dhoparee

Epps

Tijs

Walton

(2008). Measuring and defining the experience of immersion in games. International Journal of Human-Computer Studies, 66, 641-661.

34.

Johnson

J. W.

(1981). A scholar looks at killer. Afterword in Killer: The Game of Assassination. Austin, TX: Steve Jackson Games.

35.

Jonsson

Montola

Waern

Ericsson

(2006, June 14-16). Prosopopeia: Experiences from a pervasive larp. In Proceedings of ACM SIGGCHI ACE 2006 Conference. West Hollywood, CA: ACM.

36.

Juul

(2004). Introduction to game time. In Wardrip-Fruin

Harrigan

(Eds.), First person: New media as story, performance, and game (pp. 131-142). Cambridge, MA: MIT Press.

37.

Juul

(2005). Half-real: Video games between real rules and fictional worlds. Cambridge, MA: MIT Press.

38.

Karppi

Sotamaa

(2011). Rethinking playing research: Methodological observations from behind the decks. Simulation & Gaming.

39.

Koivisto

Ollila

(2006, November 15-16). How to fail: Mobile game design in a research project involving software prototype development. In Game Design & Technology Workshop ’06. Liverpool, UK: ACM.

40.

Koivisto

Palm

(2005, November 8-9). Iterative design of mobile games. In Computer Game Design and Technology Workshop ’05. Liverpool, UK: ACM.

41.

Kücklich

(2007). Homo deludens: Cheating as a methodological tool in digital games research. Convergence, 12, 355-367.

42.

Kultima

Stenros

(2010). Designing games for everyone: The expanded game experience model. In Proceedings of Future Play 2010. Vancouver, Canada.

43.

Linderoth

(2004). Spelelement bortom idén om den interaktiva illusionen. (Game elements beyond the idea of an interactive illusion.) (Unpublished doctoral dissertation). Göteborg University, Göteborg. Retrieved from http://gupea.ub.gu.se/handle/2077/16217

44.

Magerkurth

Cheok

A. D.

Mandryk

R. L.

Nilsen

(2005). Pervasive games: Bringing computer entertainment back to the real world. Computers in Entertainment, 3(1).

45.

Mäyrä

(2008). An introduction to game studies: Games in culture. Los Angeles, CA: SAGE.

46.

Mäyrä

Lankoski

(2009). Play in hybrid reality: Alternative approaches to game design. In de Souza e Silva

Sutko

D. M.

(Eds.), Digital cityscapes: Merging digital and urban playscapes (pp. 129-147). New York, NY: Peter Lang.

47.

McGonigal

(2006a, January 24). The puppet master problem: Design for real-world, mission based gaming. Lecture given at University of Art and Design Helsinki, as part of Games and Storytelling lecture series.

48.

McGonigal

(2006b). This might be a game: Ubiquitous play and performance at the turn of the twenty-first century (Unpublished doctoral dissertation). University of California, Berkeley.

49.

McMillan

Morrison

Brown

Hall

Chalmers

(2010). Further into the wild: Running worldwide trials of mobile systems. In Proceedings of Pervasive 2010. Lecture Notes in Computer Science 6030 (pp. 210-227). Springer.

50.

MOMENTUM. (2007). [Developed by Ericsson

Jonsson

.] Stockholm, Sweden. Retrieved from http://momentum.sics.se

51.

Montola

(2007, September 24-28). Tangible pleasures of pervasive role-playing. In Baba

(Ed.), Proceedings of DiGRA 2007 Situated Play Conference (pp. 178-185). Tokyo, Japan: The University of Tokyo.

52.

Montola

(2011a). A ludological view on the pervasive mixed-reality game research paradigm. Personal and Ubiquitous Computing, 15, 3-12.

53.

Montola

(2011b). Social constructionism for ludology. Simulation & Gaming.

54.

Montola

Jäppinen

Lahti

Lankoski

Waern

Holopainen

(2006). Pervasive game design and evaluation. Guidelines for IPerG Phase Two, IPerG Deliverable D5.4. Retrieved from http://iperg.sics.se/press1.php

55.

Montola

Stenros

Waern

(2009). Pervasive games: Theory and design. Amsterdam, Netherlands: Morgan Kaufmann.

56.

Montola

Waern

(2006, October 6-8). Ethical and practical look at unaware game participation. In Santorineos

(Ed.), Gaming realities: A challenge for digital culture (pp. 185-193). Athens, Greece: Mediaterra 2006 Festival, Fournos Centre for the Digital Culture.

57.

Myers

(2009). In search of a minimalist game. In breaking new ground: Innovation in games, play, practice, and theory. In Proceedings of DiGRA. DiGRA. Retrieved from http://www.digra.org/dl/db/09287.39520.pdf

58.

Niemi

Sawano

Waern

(2005). Involving non-players in pervasive games. In Proceedings of Critical Computing. Aarhus, Denmark.

59.

Nieuwdorp

(2007). The pervasive discourse: An analysis. Computers in Entertainment, 5(2).

60.

Peitz

(2009). Insectopia. In Montola

Stenros

Waern

(Eds.), Pervasive games. Theory and design (pp. 175-177). Amsterdam, Netherlands: Morgan Kaufmann.

61.

PRISONER ESCAPE FROM THE TOWER. (2006). [Developed by Historic Royal Palaces.]. Mscape Team.

62.

PROSOPOPEIA. (2005). [Developed by Ericsson

Jonsson

.] Stockholm, Sweden. Retrieved from http://prosopopeia.sics.se/

63.

Rabin

(Ed.). (2005). Introduction to game development. Hingham, MA: Charles River Media.

64.

Reid

(2008). Design for coincidence: Incorporating real world artifacts in location based games. In Proceedings of DIMEA’08. Athens, Greece.

65.

Rodriguez

(2006). The playful and the serious: An approximation to Huizinga’s Homo Ludens. Game Studies, 6. Retrieved from http://gamestudies.org/0601/articles/rodriges

66.

Rowland

Flintham

Opperman

Marshall

Chamberlain

Koleva

. . .Perez

(2009). Ubikequitous computing: Designing interactive experiences for cyclists. In Proceedings of Mobile HCI. Bonn, Germany.

67.

Salen

Zimmerman

(2004). The rules of play: Game design fundamentals. Cambridge, MA: MIT Press.

68.

Schell

(2009). Design outside the box. Presentation at DICE2009. Retrieved from http://g4tv.com/videos/44277/DICE-2010-Design-Outside-the-Box-Presentation/

69.

Seale

Gobo

Gubrium

J. F.

Silverman

(Eds.). (2004). Qualitative research practice. London, England: SAGE.

70.

Stenros

Montola

(2009) Momentum. In Montola

Stenros

Waern

(Eds.), Pervasive games: Theory and design (pp. 111-116). Amsterdam, Netherlands: Morgan Kaufmann.

71.

Stenros

Montola

Mäyrä

(2007). Pervasive games in ludic society. In Proceedings of Future Play 2007. New York, NY: ACM.

72.

Stenros

Montola

Waern

Jonsson

(2007). Momentum evaluation report (IPerG Deliverable 11.8 Appendix C). Retrieved from http://iperg.sics.se/press1.php

73.

Sweeny

R. W.

Patton

R. M.

(2009). CitySneak: Play, pedagogy, and surveillance. In de Souza e Silva

Sutko

D. M.

(Eds.), Digital cityscapes: Merging digital and urban playscapes (pp. 204-216). New York, NY: Peter Lang.

74.

Taylor

T. L.

Kolko

(2003). Boundary spaces: Majestic and the uncertain status of knowledge, community and self in a digital age. Information, Communication & Society, 6, 497-522.

75.

THE TRUTH ABOUT MARIKA. (2007). [Developed by Ericsson

Lägersten

]. SVT and Company P.

76.

UNCLE ROY ALL AROUND YOU. (2003). United Kingdom. Blast Theory. Retrieved from http://www.blasttheory.co.uk/bt/work_uncleroy.html

77.

VEM GRÅTER. (2005). Student project. University of Gotland, Sweden.

78.

Waern

Ahmet

Sundström

(2009). An in-game reporting tool for pervasive games. In Proceedings of ACE’09. Athens, Greece: ACM.

79.

Waern

Montola

Stenros

(2009). The 360 illusion: Designing for immersion in pervasive games. In Proceedings of CHI’09. Boston, MA: ACM.