From interactive surfaces to interactive game pieces in hybrid board games

Abstract

Recent advances in interactive surfaces and Tangible User Interfaces (TUIs) have created new opportunities for hybrid board games that aim at mixing the social affordances of traditional board games with the interactivity of video games. Within this area of research, we propose an approach centered on the concepts of tokens, constraints, spatial expressions, and interaction events. Rather than using interactive surfaces as the primary interaction medium, our approach relies on physical manipulation of interactive, computer-augmented game pieces on conventional surfaces. The design of our approach has been informed by a literature review that took into account 27 hybrid board games from both academia and industry. This review allowed us to identify technology strands used to implement interactive board games and discuss the pros and cons of the different alternatives in the design space. After describing our approach, we report how it was applied to the design and development of a game for training emergency workers. Building on feedback from user evaluations and our experience with the development, we outline design opportunities and challenges of the approach.

Keywords

Tangible user interface hybrid board game interactive objects pervasive gaming ubiquitous computing

1. Introduction

Playing board games is an engaging social experience characterized by two levels of interaction: between the players themselves (e.g., discussing strategies) and mediated by physical artifacts representing information and actions (e.g., rolling a die or drawing a card). Such rich experience is facilitated by the social and physical affordances of the main elements common to any board game: board and game pieces such as pawns and cards. While sitting around a board affords face-to-face and gestural communication, game pieces allow for tangible interaction and physical feedback. For example, pawns representing entities and actions can be disposed of, moved on and around the board, and also manipulated by inactive players; they can be kept hidden or exchanged among players, therefore enabling simultaneous play and leaving room for unintended uses. Figure 1 depicts the main interactions in a classical board game.

With the development of desktop PCs, arcade games, and game consoles, digital computer games were introduced. Board games hence got the opportunity to be translated through this new medium. In this paper, we adopt the concept of translation as defined by Latour [32], that is, a process of arranging heterogeneous interests into a new order, thereby creating something new. By using the term translation, we underline that when game designers and developers create a computer-based hybrid board game from its paper-based version, they make interpretations during the translation that impact the game experience (e.g., using or not using tangible components or virtual avatars). This is demonstrated, for example, in [25], where the authors compare different versions of Settlers of Catan.

Initially, physical board games were translated into completely virtualized games. The use of computers allowed for advantages such as the usage of multimedia to promote a richer and more interactive playing experience, artificial intelligence (AI) to make intelligent nonplaying characters, and tracking of the activities to create leaderboards, thus stimulating competition. At first, keyboard and point-and-click interaction replaced physical actions around the board. Many board games could be played only in single-player mode (against an AI character), even though some of them also offered a multiplayer mode consisting of two or more players sharing a keyboard and mouse in a turn-by-turn, collocated but screen-mediated interaction (Fig. 2, left). Notably, this approach does not facilitate face-to-face interaction (it is rather a shoulder-to-shoulder interaction) [36] and simultaneous actions. Over the years, virtualized digital board games encompassed higher-definition graphics and sound. Taking advantage of increased connectivity, multiplayer games started supporting remote players around a virtual board (Fig. 2, right). Still, the translation to the digital domain lacked the social and physical affordances of traditional board games: the lack of tangible interactions impacted the game experience [48], and the computer or the console was still acting as a mediator [35], impacting the social experience. Comparing Figs 1 and 2, the reader can observe the reduced number of interactions when moving from traditional to virtualized board games.

Fig. 1.

Rich interactions in a traditional board game.

Taking advantage of advances in ubiquitous and ambient technology, mainly interactive surfaces and Tangible User Interfaces (TUIs), the translation of board games entered a new phase. Several works introduced interactivity in hybrid board games by replacing the game board with a computer-interactive surface capable of sensing touch inputs and allowing for the manipulation of conventional or technology-augmented objects. This is achieved by replacing the game cardboard with a touchscreen computer (e.g., iPad or tabletop). The screen becomes an interactive board capable of graphical and auditory stimuli and reacts to touch inputs and manipulation of technology-augmented artifacts on and around it. Although this approach has become mainstream, it mainly confines interactivity to a touchscreen area, imposing a trade-off between the size of the touchscreen (the interactive space) and the cost.

Fig. 2.

Interaction in a computer-based board game.

The contribution of this paper is twofold. First, we present an extensive review of the state of the art in this area of research, focusing on how ubiquitous and ambient technology has been used to create hybrid board games.

Building on the current state of the art and addressing the identified limitations, the second contribution of the paper is an approach to the digitalization of board games inspired by the Token+Constraint (T+C) paradigm [51]. Rather than focusing on interactive surfaces, we focus on transforming game pieces into interactive tokens, preserving the board as a passive element. The actions players can perform on game pieces are driven by the physical and visual constraints provided by the board and game rules, as they are in traditional board games.

In order to prove its validity, we applied our approach to the translation of Don’t Panic [14], an existing board game to train emergency workers in panic management. Starting from a cardboard prototype, we redesigned Don’t Panic around interactive tokens and built a working prototype, leveraging digital manufacturing techniques. This game was chosen because it implements elements that are generic to board games, like pawns and cards, and has a complex but limited set of rules to illustrate our approach in a concise way.

The paper is structured as follows. We first review the state of the art of current digital board games, identifying current technology strands. Our design approach is then presented and grounded in existing conceptual frameworks. Next we describe its application to Don’t Panic and the related evaluation. Finally, we discuss lessons learned in the form of design opportunities and challenges.

2. State of the art

In this section, we present research on digital board games with elements of physical interaction. The research presented hereafter can be set in the broader field of pervasive gaming, which aims to bring physical and social interaction back to computer games [34].

2.1. Literature review methodology

We started our review by exploring papers and online articles, published over the last 15 years, about digital board games without any specific constraint on the type of platform used for implementation, considering both research and commercial documentation. This initial screening helped us to get an overview of different translations of board games so that we could better define the limits of our review in terms of both sources to be used and domain boundaries. First, we decided to continue our analysis by using research papers as the main source of information and commercial games only for exemplification purposes. In order to identify material to be included in the analysis, we conducted automatic searches in relevant databases, including the ACM Digital Library, Springerlink, and IEEE Xplore, and manually searched for literature in relevant conferences, including ACE, TEI, CHI, INTERACT, and ITS.

Second, we decided to focus on ubiquitous technologies that have actually been used to implement board games, rather than technologies that could potentially be used for game design.

Third, we decided to focus on translations aiming at preserving the strengths of board games, that is, preserving and enriching social and physical affordances of cardboard games.

From this process, we retained articles that we read in detail and analyzed to identify common trends and challenges. Within this research, we identified two main technology strands: (i) stationary interactive surfaces and (ii) mobile interactive surfaces. These two strands are described in the following sections, with a summary presented in Table 1.

2.2. Stationary interactive surfaces

At the beginning of the 21st century, tabletop computers (large horizontal touchscreens) were considered an ideal platform for digital board game development [19] as they were able to combine some of the advantages of low-tech board games with the benefits of video games [3]. Indeed, sitting around a tabletop computer allows players to be closer to the digital information, and at the same time, enhances collaboration and communication among the users [44], re-creating the experience of face-to-face gaming. Although tabletops can be augmented with technology for playing games using different flavors of augmented reality, as in [5,13], or 3-D projections [20], with the advances in touchscreen technology, a number of works have used computer vision-based tabletop computers, such as the Microsoft Surface (2001), DiamondTouch (2001) [15], and Reactable (2007) [27], as platforms for digital board game development.

The computer engine developed for Reactable, reacTIVision [28], has been used to implement several board games. Sheridan et al. [46], for example, present two interesting variations of reacTIVision usage, one implementing tangible interaction and another using tags attached to gloves to identify players. Several other examples are available in the literature, such as fostering game-based learning [1,24,42]. For a review, see [19].

Table 1
Technology strands for digital board games

Game Year Technology Strands

Stationary Surfaces Augmented Objects Mobile Surfaces

Augmented Reality Tabletop Displays Smartphones/Tablets MDE ADI

DiamondTouch Games [15] 2001 ∙

False Prophet [38] 2002 ∙ ∙

Monopoly STARS [37] 2003 ∙ ∙

Augmented Reality Chinese Checkers [13] 2004 ∙

DiamondSpin [45] 2004 ∙

Monkey Bridge [5] 2005 ∙ ∙

PlayAnywhere [53] 2005 ∙

Matching, Poetry and Classification Table [42] 2006 ∙

Weathergods [4] 2007 ∙ ∙

YellowCab [52] 2007 ∙ ∙

Totti [23] 2008 ∙ ∙

Asterocks [54] 2008 ∙

IncrediTable [33] 2009 ∙ ∙

ePawn 2010 ∙ ∙

Futura [1] 2011 ∙

Portico [2] 2011 ∙ ∙ ∙

Of BATs and APEs [24] 2012 ∙

Pong+ [11] 2012 ∙ ∙

zAPPed Monopoly 2012 ∙ ∙ ∙

Toy Vision [39] 2013 ∙ ∙

iPieces 2013 ∙ ∙

Scrabble for iOS 2013 ∙ ∙

Projected Play [20] 2014 ∙ ∙

Capture the Flag [43] 2015 ∙ ∙ ∙

Towering Defense [49] 2015 ∙ ∙ ∙

Airsteroids [18] 2015 ∙ ∙ ∙ ∙

RapID Tic-Tac-Toe [47] 2016 ∙ ∙

Even though the direct manipulation of virtual objects supported by touchscreens makes these games more similar to “analog” board games and allows for a face-to-face social experience, the resulting global experience is still different from the 3-D sensory feedback experienced by playing with dice, pawns, and cards. For example, we still miss physical objects that allow for peripheral interaction during the game and permit passive players (in turn-based games) to manipulate game pieces as long as they do not break the rules [31]. In interactive-surface implementations of board games, technology is usually employed to virtualize pieces’ representations by means of computer graphics and sound. The player’s physical interactions with game pieces are often substituted for traditional GUI (Graphical User Interfaces) metaphors. For example, the action of rolling a die or drawing a card is implemented in touchscreen gestures such as pushing a button or pinching a virtual die. Still, it is not possible to do simultaneous actions, manipulate other players’ objects, and so on.

Figure 3 shows in a visual way the reintroduction of social interactions and the lack of tangible interactions in the case of touch-based surfaces.

Fig. 3.

Interactions in a touchscreen-based board game.

To address the lack of tangible play, game designers started combining the touch-based interaction of tabletop computers with interactions through physical objects placed on top of the screen surface as a means of controlling virtual game elements. In this way, conventional objects, such as pawns and cards, can become game elements by attaching active or passive tags recognizable by a computer vision system. Several works have introduced augmented objects in board game editions for tabletop computers. For example, in Weathergods [4] and Totti [23], players use different iconic or symbolic physical artifacts as their avatars and to perform actions during the game. In False Prophets [38], YellowCab [52], and the STARS edition of Monopoly [37], tangible objects act as characters in the game. In IncrediTable [33], players can modify the game board with smartpens and combine physical and virtual objects to solve puzzles in the game. In order to facilitate the implementation of tabletop-based tangible games, toolkits are available, for example [39,45]. This type of approach allows, at the same time, the reintroduction of social affordances and physical manipulation typical of traditional board games (Fig. 4).

Finally, tabletop computers or surfaces suffer from two major drawbacks: mobility issues (they are bulky and heavy) and high cost; therefore, most of the developed games aim at school or museum audiences, and, for this reason, are serious games (e.g., see relevant experiences at the National Museum of Natural History in Paris). Low-cost alternatives [54] or mobile options [53] are not yet available outside research labs. However, in recent years, smaller but more affordable touchscreen devices have hit the market: smartphones and tablets.

Fig. 4.

Typical interactions when using tangible objects on a touch-based surface.

2.3. Mobile interactive surfaces

Fueled by new mobile platforms, board games were revitalized: remarkably, iPad editions of board games such as Monopoly, Scrabble, and Ticket to Ride are constantly among the top 100 grossing games in the United States.1

¹
Top 100 iOS applications, https://hec.su/dEL5.

One of the reasons for this renaissance is the ubiquitous play that mobile devices allow. Like the larger screens, these games can also make use of tangible interaction with objects, for instance, using active [55] or passive [11] tags. Physical pawns for playing board games on tablet PCs have recently been commercialized by board game companies, for example, under the names of iPieces2

iPieces, http://jumbo.eu/ipieces/.

and ePawn.3

ePawn, http://epawn.fr/.

These solutions attempt to re-create a tabletop-like setting using a tablet, but the small touchscreen area (compared to a traditional paper board or tabletop computer) can diminish the gaming experience due to information occlusion and overloading. To avoid these issues, provide the social affordances of board games, and enable collocated multiplayer mode and private/public spaces, two trends have recently emerged: multi-device environments (MDEs) and around-device interaction (ADI).

2.3.1. Multi-device environment (MDE)

MDEs make use of several displays orchestrated closely together, enabling games to, for example, extend the interaction surface by splitting a virtualized game board on several devices. Furthermore, MDEs can establish public and private spaces by advertising public information on shared screens and private content on personal smartphones. This approach is showcased in Scrabble for iOS.4

⁴
Scrabble for iPad, http://ea.com/scrabble-paid-ipad.

In this game, near the online multiplay option, players can choose collocated multiplay, using an iPad as a public shared surface (to display the board) and iPhones as private spaces (to display letter tiles). In Capture the Flag [43], the popular video game has been implemented using augmented reality on personal smartphones to provide a private view of a battlefield displayed on a shared tablet device. Similarly, in Towering Defense [49], gesturing and juxtaposition of a smartphone screen onto a tablet screen reveal hidden layers and provide a deeper perspective into the game. MDEs have found successful application in implementing card and gambling games, such as Magic: The Gathering [16], in which players display private cards on their devices before sharing them on a public screen. Even if this latter case is far removed from board game dynamics, the private/public use of the devices in the game context is very similar to that shown in the Scrabble example.

2.3.2. Around-device interaction (ADI)

ADI [12] expands the area of play outside the device’s screen by enabling interaction using gestures or manipulation of passive or active augmented objects over (aerial interaction) or next to the interactive surface. This approach affords parallel interaction, allowing, for example, two players to interact with a tablet, both using physical objects and touch inputs at the same time without occluding each other’s actions [17].

ADI can be implemented with different technologies whether involving interaction “over” or “on the side” of a device. Aerial interaction often makes use of optical recognition of fingers and objects hovered over devices’ front cameras with or without external hardware. In [22], users can control content displayed on-screen by gesturing in space wearing ring-shaped visual tags on their fingers, while [30] allows controlling digital content using hand gestures over smartphones and wearable devices. Notably, both of these approaches require equipping the interactive surface with external hardware (IR sensors), limiting ubiquitous play. Further extending interaction to the sides of an interactive surface and preserving portability, Portico [2] enables interaction with multiple objects on top and around the interactive surface by embedding two small external cameras in the tablet’s bezel. Portico has ported several games to its platform, including Tic-Tac-Toe and popular arcade-type games.

A different strand of research has implemented ADI-exploiting sensors commonly found in mobile devices without requiring external hardware. MagiTact [29] and Magnetic Appcessories [8] make use of the magnetic (compass) sensor to enable interaction around the mobile screen with objects with magnetic properties. Abracadabra [21] enables high-fidelity aerial interaction using a finger-worn magnetic ring. Although the technique has been demonstrated on smart-watches, the approach could be ported to larger screens. Airsteroids [18] showcases an application incorporating both ADI and MDE techniques and has redesigned a traditional spaceship arcade game: a ship must destroy asteroids before they crash into it. The game space extends across several tablets brought in by players, which can control the ship’s heading and shoot hovering cards using fiducial markers recognizable by tablets’ top cameras. RFID (Radio Frequency IDentification) technology has been also employed to implement ADI games. RapID [47] shows how to achieve low-latency movement sensing of interactive objects using arrays of RFID tags. The company demonstrates its framework implementing game pieces for a hybrid version of the Tic-Tac-Toe game. RFID is also used to enhance gaming experiences in console games, as exemplified by commercial products like Activision Skylanders5

⁵
Skylanders video game, http://skylanders.com.

and Disney Infinity.6

⁶

Disney Infinity video game, http://infinity.disney.com.

Finally, in Hasbro’s Monopoly zAPPed edition,7

⁷

Monopoly zAPPED edition, http://www.hasbro.com/en-us/product/monopoly-zapped-edition.

the original Monopoly board is augmented with digital content produced by an iPad. In this implementation, some interactions are low-tech, such as moving pawns on the board, and some others are mediated by technology, such as buying a property. Notably, the digital and physical representations of the state of the game are disconnected.

Despite technological advances, many MDE and ADI techniques are often limited to single-user applications. Reasons are that the interactions still call for proximity to a device (due to the limited recognition range of magnetic or optical sensors), although requiring the user to move closer to a target could cause occlusion and interference in MDEs [17].

Research in pervasive games has also explored, for example, how to use location information to enhance computer games (e.g., see Can You See Me Now?[7]). In the field of ubiquitous computing, even if some research atelier were conducted in an attempt to bridge ubiquitous computing and games [9], as far as we know, this aspect has not been addressed in depth.

2.4. Discussion

Table 1 reports in chronological order the games analyzed in the previous sections. It provides an overview of the technological solutions used in the identified technology strands and specific examples. Analyzing the different works reviewed, it is apparent that over the years, creating interactive board games has become an increasingly complex task for developers.

While works in the early 2000s mainly made use of advances in tabletop computers and augmented reality, recent works employ a diverse set of technologies that span the different research strands proposed; see, for example, Towering Defense and Airsteroids. Therefore, we can consider the different technologies introduced over the years as complementary rather than mutually exclusive. Also 16 of the 27 works featured the use of augmented objects either on top of or around interactive surfaces, illustrating the importance of reintroducing physical affordances in board games.

Whereas developing games for interactive surfaces could rely on established toolkits and processes, adding tangible components with emerging MDE and ADI technologies required a radical change in the development process.

Developing board games for interactive surfaces mainly consisted of mapping keyboard/mouse commands with multi-touch gestures provided by the new technology medium (e.g., sliding a finger over a virtual die rather than mouse-clicking on it) and exploiting increased computation capability to produce richer graphical representations of game pieces. This translation was largely helped by the established toolkits available, together with specific hardware platforms (e.g., Reactable/reacTIVision). In this way, developers had a clear vocabulary of multi-touch gestures to replace the keyboard/mouse or other input peripherals, together with formal guidelines.

More recent technology requires developers to largely rethink board game design from scratch. MDEs and ADI dramatically expanded the design space by providing more degrees of freedom in interaction design. The user interface is no longer constrained by an interactive surface, but blends with the surrounding physical environment, creating both design and technology challenges. From a design perspective, there is a lack of established design processes and metaphors to map the new types of user interaction allowed by technology, such as aerial interaction or multi-screen juxtaposition, with game dynamics. The set of unintended actions [6] the designer has to account for is also much broader. Very little work has been done to provide tools to support the game design and implementation phases.

An integrated game design toolkit for augmented board games could help developers in mapping game dynamics with opportunities provided by technology and provide a common language between game designers and software developers to describe players’ interactions and to abstract technology details. From a technology perspective, the new technological solutions require dealing with scalability, embedded hardware, and wireless ecologies. Skills at the intersection between software, hardware engineering, and digital manufacturing are needed.

In the following section, we investigate how frameworks from the field of TUIs could provide theory tools to support board game translation. From this perspective, establishing theoretical constructs to model digital board game interactions is the first step toward building a technology toolkit to support game developers. With that aim, in the following section, we describe the theoretical approach to board games that we adopted.

3. The interactive-token approach to board games

As detailed in Section 2, the dominant paradigm for designing hybrid board games consists of adding technology-augmented active or passive objects on, next to, or over one or more interactive surfaces. These objects complement and facilitate interaction with the interactive surface by offering affordances proper to the physical world for manipulating virtual artifacts and controls (e.g., buttons and menus).

We propose a different approach: the game pieces are the means to interactivity and not the board per se. Embedding interactivity across multiple components opens up a wider space of possibility and a higher degree of flexibility in shaping the game experience. For example, game pieces can influence the state of a game not only when they sit on the interactive surface, but also when they are manipulated over and around it, without requiring an external infrastructure for sensing. In this way, the board is mainly used to stage the game and establish context for the use of the pieces, as in traditional board games. Also, the interactive area of the board is less limited by size, which also determines the portability of the game (and cost). Our approach merges the advantages of using ADI and MDE technologies, in that augmented objects are not just used to control information displayed on an interactive surface, but become autonomous points of interaction with input and output capabilities. Interactivity is hence provided by multiple interfaces rather than just one interactive surface. Hereafter, we use the term interactive token to refer to game pieces with added interactivity.

3.1. Control and representation

Our investigation aims to augment the two intrinsic roles commonly found in board games: control and representation. For example, pawns serve as a visual representation of players, and shared items (e.g., houses in Monopoly) as the representation of a resource count. The action of rolling a die or drawing a card acts as a control for a (random) variable, allowing the game to evolve from one representational state to another. Each game piece can serve in one role (as in Monopoly) or both (as in chess). Pieces usually represent players and resources via iconic or symbolic artifacts; moreover, the spatial configuration of game pieces on the board provides the players with a shared awareness of the state of the game.

In interactive-surface implementations of board games, technology is often employed to virtualize pieces’ representations by means of computer graphics and sound. The players’ physical interactions with game pieces are often substituted with traditional GUI metaphors. For example, the action of rolling a die or drawing a card is implemented in touchscreen gestures such as pushing a button or pinching a virtual die.

In our approach, the role of technology is twofold. On one hand, it brings interactivity by augmenting, not virtualizing, pieces’ material representations; on the other, sensor technology is used to capture players’ tangible interactions with control pieces aiming at preserving their traditional physical affordances. For example, an accelerometer embedded in dice can sense the result of a dice throw and update a digital variable in a way that is transparent to the player.

Interactivity is a consequence of players’ interaction with control pieces, and it is regulated by game rules. Interactivity can be provided, for example, by means of small LEDs or LCDs embedded in pieces to convey graphic and video content, or through auditory or haptic feedback. Game pieces might still preserve their traditional aspects, having a tangible representation that complements an intangible or ephemeral representation provided by technology. As a matter of fact, pieces in board games are used to convey both static and dynamic information; for example, players’ identities and roles do not change throughout the game and are often represented by a set of distinguishing pawns (or tokens), while resources or scores associated with each player vary and are usually represented by a number of shared artifacts (e.g., houses and hotels in Monopoly). In our approach, designers can define the trade-off between the two representations as balanced using static information provided by the tangible representation and dynamic information provided by intangible representations. For example, a revisited version of Monopoly tokens might preserve their physical semblances to identify players but might embed an intangible representation of the number of properties owned by the player (e.g., in digits, icons, or symbols on an LCD display).

3.2. Architectural view of interactive-token board games

From an architectural point of view (Fig. 5), a digital model of game variables and rules (stored in a computer game engine) mirrors the spatial configuration of physical game tokens on the board. Each token has a tangible representation (i.e., shape and color) that identifies the piece and defines its affordances; in addition, it might have an intangible representation (graphic or auditory), controlled by the digital model, that is updated anytime the manipulation of a piece with control power pushes a change in the model. The interaction with pieces is based on a double loop [26].

Fig. 5.

Double interaction loop in interactive board games.

A first interaction loop consists of the passive haptic and visual feedback the player perceives when manipulating pieces on the board. This loop is in common with traditional board games. A second loop adds interactivity by means of graphical and auditory feedback conveyed via the tokens’ intangible representation. This loop requires technology for sensing manipulations of tokens as well as providing visual/audio feedback (Fig. 5). Our approach is conservative toward traditional game mechanics. Technology is used for augmenting players’ interactions with the pieces rather than reinventing them. The set of valid interactions with game pieces is defined by the affordances of pieces and by game rules. To formalize these rules, we build on two theories: the Token+Constraint framework [51], providing a powerful descriptive language, and the MCRit (Model-Control-Representation, intangible and tangible) [50], proposed by Ullmer and Ishii, addressing issues of representation and control in TUI.

3.3. Theory grounding

The Token+Constraint framework defined by Ullmer et al. [51] defines tokens as discrete physical objects that represent digital information and constraints as either mechanical or visual confining regions that are mapped to digital operations. By the interaction phases of association and manipulation of tokens within a system of constraints, it is possible to map physical actions to a set of computational operations; for example, the presence or absence of a token in a constrained area could be easily digitized in binary information.

Besides the T+C paradigm focus on the use of tokens and constraints as a means to trigger digital operations, physical artifacts are also characterized by their appearance. Indeed, the “seamless integration of control and representation” [50] is a distinctive characteristic of TUIs over traditional GUIs, where control and representation are decoupled in input (e.g., keyboard or mouse) and output (e.g., screen or printer) devices. Aiming at going beyond the traditional MVC (Model-View-Controller) paradigm, in [50] Ullmer and Ishii proposed an interaction model for TUIs called MCRit. They redefined the view concept of graphical interfaces as a balance between a physical representation (the tokens’ shapes and affordances) and an intangible representation (e.g., computer graphics and sounds). This approach allows for blending the flexibility offered by the graphical elements of GUIs with the natural manipulation offered by TUIs.

The presented interaction paradigms can be integrated to drive the design of digital board games. The Token+Constraint approach provides conceptual tools for building TUIs that leverage interaction with physical game pieces for controlling digital representation of game elements, hence preserving the affordances of board games. The MCRit paradigm allows for adding interactivity by augmenting, not replacing or virtualizing, the physical representation of game pieces with an intangible representation of digital information.

3.4. Key design constructs

Aiming at extending the T+C paradigm, we define a game, which is composed by game dynamics (the sum of game logic and rules), as a sequence of player-initiated interaction events that modify spatial configurations of tokens with respect to board constraints and other tokens. Sequences of interaction events describe players’ interaction during the game and allow a game to evolve through states. In the following, we describe how we extended T+C to address the design of interactive board games.

Tokens are technology-augmented artifacts capable of triggering digital operations that can activate game dynamics. Tokens may be capable of sensing information (e.g., proximity to other tokens) and displaying computer graphics and sound. Some tokens are the personal embodiment of the players on the board, whereas others are public and can be handed around during the game. Tokens conceptualize all the tangible pieces traditionally used in board games. They range from elements of chance (e.g. augmented dice in backgammon or RFID-enabled cards in Monopoly) to game pieces, for example, a pawn augmented with an LCD displaying the player’s rank in the game.

Board constraints are physical or visual confining regions in the (physical) board space. The association or dissociation of a token within a constraint can be mapped to digital operations to activate game dynamics. Constrained regions are determined by a perimeter that could be visual or physical; the structure of the perimeter might permit a certain degree of freedom for the token (e.g. allowing for translation or rotation). Examples of constraints are checks for chess pieces and territories in Risk.

Spatial configurations are static relationships of tokens with respect to both constraints and to other tokens. They limit the space of interaction of players to a set of valid token-constraint and token-token relationships defined by a grammar of game-specific rules. For example, certain tokens can be associated only with selected constraints; the relationship of proximity among tokens can be meaningful or not. Spatial configurations are used to validate players’ interaction events against game rules, narrowing the set of actions that are valid for activating game rules.

Interaction events are player-triggered manipulations of tokens, recognizable with sensor technology that modifies the (digital and physical) state of a game. We identified three types of events:

solo-token event (T): the manipulation of a single token over or on the board. For example, the action of rolling a die or drawing a card.

token-constraint event (T-C): the operation of building transient token-constraint associations by adding or removing tokens to or from a constrained region of the board. T-C events can have different consequences depending on game rules: in Risk, moving army pieces beyond a territory line is an attack action; in the Mancala solitaire game, the marble can only fit in an empty space and moving it in an occupied space implies to eat another marble.

token-token event (T-T): the operation of building transient token-token adjacency relationships, achieved by moving tokens on the board, for example moving a token next to another token to unlock special powers or to exchange a resource between two players. Creating a king in the Draughts game, for example, requires that the player put a game piece on top of another.

The sequence of valid interaction events activates specific game dynamics, thus allowing the game to evolve from one state to another and triggering a change in intangible representation produced by tokens. For example, we can model the act of capturing a piece in chess as a sequence of interaction events that modify proximity between two chess tokens within checkers constraints.

4. Applying the interactive-token approach

In this section, we illustrate the approach by describing how it has been used to design interactive tokens, board constraints, and interaction events for translating an existing board game called Don’t Panic [14]. The game shares similarities with many board games, such as the use of pawns to represent players, items to trigger game mechanics, and cards as elements of chance. This effort allowed us to evaluate the feasibility of the approach.

4.1. Don’t Panic game dynamics and rules

Don’t Panic is a collaborative game inspired by Pandemic,8

⁸
Pandemic game, http://zmangames.com.

a game about fighting viral infections spreading across nations. Four players start the Don’t Panic game as members of a panic management team that must work together to manage panicking crowds in turn-based actions. A map representing a city is displayed on the game board, and the territory is divided into sectors. Each sector contains a number of people (PO) characterized by a panic level (PL). During the game’s panicking events (e.g., fires and explosions), randomly triggered by card drawing, PLs increase in determinate sectors. In addition, the panic increases at regular intervals. Each player is represented on the board space by a pawn token and gets a limited number of actions with the goal of lowering the panic level in the city. Using the “Calm!” and “Move!” tokens, a player can either reduce the panic in a specific sector or move panicked people to an adjacent sector (with a lower PL). Information cards distributed during each turn can lower the panic in multiple sectors; for example, the action “TV broadcast” reduces the PL in all the sectors. Players collectively win the game when the PL in all sectors is zero. For a full description of game rules, see [14].

Fig. 6.

Don’t Panic interactive tokens.

4.2. Design of tokens, and board constraints

Don’t Panic, in its augmented version, is composed by a cardboard and a set of tokens. In the following we describe the objects and their meaning as game pieces.

The board (Fig. 6(a)) is cardboard and visualizes a map portraying a territory divided in nodes, sectors, and paths. Nodes are edges between sectors and are connected by paths, as in closed cyclic graphs. Nodes feature physical constraints and no degree of freedom for the hosted tokens; sectors and paths provide visual constraints, allowing tokens’ translation and rotation within the perimeter.

The card deck (Fig. 6(c)) consists of dynamically printed information cards. Each card has a textual description of how it affects the game and a barcode that links the card to its digital representation. The top surface of the card deck can read the barcode on the card and trigger actions in the game (Fig. 6(d)). Therefore cards don’t affect game dynamics immediately after they are produced; they can be kept or exchanged by players, until when they are activated by the card deck.

Pawn tokens (Fig. 6(b)) embody the players’ presence on a node; each player interacts with a personal pawn during a game. Pawns can be dragged from node to node, as long as a path directly connects the two. Each pawn provides static and dynamic information via an LCD display. The static information shows icons linking to a specific player. The dynamic representation visualizes the number of people present in sectors adjacent to each of the four pawns’ sides and their panic level (symbolized by colors). This information is contextually updated according to a pawn’s location, since different nodes face different sectors. Besides their representational functions, pawns also have a control role: in order to activate nested actions with other tokens, the player has to reach the relevant node.

The Calm! token (Fig. 6(e)) represents the action of going into the field and calming people by talking to them, thus reducing the PL in a specific sector. The top display shows a numeric representation of how effective the action of calming people is, given the player’s role in the active turn. When it is activated by proximity to a pawn’s side, it provides visual and auditory feedback.

The Move! token (Fig. 6(f)) simulates moving people between sectors. In this way the people who are moved acquire the panic level of the recipient sector. The top display shows the number of people that can be moved, given the player’s role in the active turn. It also provides visual and auditory feedback.

4.3. Design of spatial configurations

After designing tokens and constraints, we defined valid token-constraint and token-token configurations and interaction events (Table 2). Token-constraint relationships are defined by univocal, transient associations created by the add/remove interaction event. Token-token relationships are defined by adjacency achieved via the move interaction event. The types of constraint limit the interaction events that tokens can afford. For example, physically confined tokens can only afford the add/remove (association with constraint) event, while visually constrained tokens leave the player free to manipulate the token, for example, to build proximity relationships with other tokens.

Table 2
Valid spatial configurations and interaction events

Token Allowed Constraint Allowed proximity with other token

Pawn Nodes (physical) Adjacency to Calm! and Move!

Calm!, Move! Sectors (visual) Adjacency to Pawn

Cards Card Deck (visual) None

Token	Allowed Constraint	Allowed proximity with other token
Pawn	Nodes (physical)	Adjacency to Calm! and Move!
Calm!, Move!	Sectors (visual)	Adjacency to Pawn
Cards	Card Deck (visual)	None

4.4. Mapping valid interaction events to game dynamics

Table 3 presents the mapping between Don’t Panic! game dynamics and sequences of interaction events validated against spatial configurations. Each sequence of interaction events results in a new physical configuration of tokens on the board and in an update of the digital representation of tokens in the game engine. It also produces a change in tokens’ intangible representations (graphic and audio). Links to videoclips showing each sequence of interaction events are available.9

⁹
Interaction events demonstrated, http://research.idi.ntnu.no/dontpanic/video.

Table 3

Mapping game dynamics to interaction events

Game Dynamic	Interaction Event (Type)	Digital Feedbacks
Move from node A to node B	1-Remove pawn from node A (T-C)2-Add pawn to node B (T-C)	Panic and people display update
Calm down people in sector A (Fig. 6(e))	1-Add the Calm! tool to sector A (T-C)2-Move the Calm! tool toward a pawn’s side facing sector A (T-T)	Panic display update, auditory feedback
Move people between sectors A and B (Fig. 6(f))	1-Add the Move! tool to sector A (T-C)2-Move the Move! tool toward the pawn’s side facing sector A (T-T)3-Repeat steps 1 and 2 in sector B	People display update, auditory feedback
Calm down people in multiple sectors	1-Positioning a card in contact with the card deck top surface (T-C)	Panic display update, auditory feedback

5. Technologies and tools for implementation

Don’t Panic has been implemented in a fully functional prototype. We designed the hardware and the software with the help of several commercial and open source toolkits. The system we implemented, a loosely coupled modular architecture, is composed of a game engine and a set of token handlers. The game engine implements game rules and stores a digital representation of game variables (e.g., PO and PL levels); token handlers bridge players’ physical interaction with game pieces to their digital representations. Modules exchange information over an event-based messaging system via the socketIO protocol. As an example, when a player associates a pawn with a node, the relative location of the node to the board surface is acquired by sensors on the pawn, encoded in a JSON message and sent to the game engine. The engine updates the digital representation of the game state and messages back to the pawn the list of sectors adjacent to the node and relative PO and PL variables; the pawn uses the data to update information on the LCD display (Fig. 6(b)).

5.1. The game engine

The game engine was implemented in JavaScript using the node.js framework. Besides activating game rules, the game engine also acts as a server for handling communication with token handlers; moreover, it exposes an HTML-based interface for remotely administering game sessions and customizing game rules. The game engine runs on a Raspberry Pi, which is configured as a WiFi hotspot to handle TCP/IP connections with TUI clients and with remote clients for game administration. The game engine also produces auditory feedback and music.

5.2. Token handlers

Pawns, Calm!, and Move! tokens were implemented using the first generation of Sifteo Cubes [41]. Each cube is capable of sensing acceleration, sensing proximity to other cubes on any of its four sides, and displaying graphics on the top surface. The cubes’ behavior is wirelessly controlled. Sifteo Cubes do not provide any API for sensing their location relative to visual constraints on a 2-D board space. In order to make the cubes recognize discrete locations on a board (required to use the cubes as pawns reacting to node constraints), we exploited in an unconventional way the data from the three-axis accelerometer embedded in each of them. We designed sockets for the cubes, each featuring a combination of unique horizontal tilt angles over two axes; the aggregated value of tilt angles is used as a fingerprint for the socket. We 3-D-printed and embedded the sockets into the board as nodes constraints (Fig. 6(a)) and coded the relation between the socket fingerprints and node locations. In this way, when a player associates a pawn with a socket, the cube senses the surface tilt over two axes using the accelerometer; the aggregated value is matched against socket fingerprints, and thus the position of the pawn on the board is updated in the game engine. As a current limitation, after each interaction with a cube, the player has to push the upper side of the cube to confirm the action.

The card deck is crafted in the form of a wooden box that encloses a thermal printer, a CCD barcode scanner and a Raspberry Pi (which also runs the game engine). The card deck allows for printing and recognizing cards during a game. A card manager module developed in Python allows for information exchange with the game engine. Although both the game engine and card manager are deployed to the same hardware (Raspberry Pi), they are loosely coupled, allowing flexibility for future development. Each printed card displays text, graphics, and a distinct barcode that is used to link the physical card to its intangible representation stored in the game engine. When the action of drawing a random card is activated by a game rule, the game engine notifies the card handler to print text and a unique barcode on the card. When a player plays a card by waving it toward the barcode reader, the engine is notified and triggers an update in the set of variables and thus in the representation of panic levels on the pawn’s display. The card deck area also features a push button and an LCD display. In the current implementation, these devices are used to pass the turn and display status information.

6. Evaluation

The Don’t Panic prototype (Fig. 7(b)) was tested with 16 players, aged 20 to 59. The goal was to identify strengths and challenges connected with the proposed approach. The evaluation of Don’t Panic game dynamics is beyond the scope of this paper.

Each evaluation session was composed of four participants in a controlled environment. After a game walk-through, players were asked to play for 30 min, then to fill in usability (SEQ) questionnaires. In addition, sessions were observed and video-recorded with the consent of the participants. Two researchers conducted a qualitative and quantitative analysis of the collected data (2 h of video footage and questionnaires) following [10]. When relevant, we also briefly compared the results of this evaluation with the evaluation of the cardboard-based prototype reported in [14].

Utterances analysis showed that 65% of statements concerned game strategies. Only 18% (most of them in the first minutes of the game) concerned game management tasks (e.g., rules), demonstrating a steep learning curve. The remaining 17% of utterances concerned the use of tokens, often about usability issues, also noticed by the observers. Some groups used a lot of verbal interaction (202 utterances in 30 min), whereas others kept verbal utterances very low (52 in 30 min). It is interesting to note that the group with the fewest utterances won the game, while the one with the most lost. This is coherent with the game as what matters is the quality of the interaction not the quantity.

Fig. 7.

The Don’t Panic prototype tested during evaluation (b) and former cardboard prototype (a).

Comparing the results with the evaluation of the earlier paper version of Don’t Panic, reported in [14] and shown in Fig. 7(a), there is no difference between the two prototypes in terms of perceived communication and cooperation. In both cases, all participants felt the game dynamics pushed them to communicate and collaborate. This proves that the original social affordances of the board game were preserved in the translation. The analysis of the utterances, however, shows a higher number of verbal utterances (283 in 30 min) for one of the groups when playing the cardboard-based prototype. Also, the verbal utterances mostly concerned the rules (32%). In general, it seems that the addition of computer interactivity lowered the time of appropriation of game rules (learning curve) and increased perceived ease of play. Similar to many board games, in our low-tech prototype, the flow of play was negatively influenced by the need for the players to absolve to game management tasks (e.g., track information, hand out cards). The addition of computer interactivity, as implemented with the interactive-token approach, proved to promote better game flow.

Regarding physical manipulations of tokens, the usability of the system was rated quite high, with half of the participants in the study finding the system easy to use; only three participants found the system hard or very hard to use. Despite these results, we observed some usability issues. Metaphors used for representing panic levels and people per sector were difficult to learn. Moreover, several players had problems with reading and manipulating tokens when associated with nodes with high tilt angles.

Most players created private spaces to store their game pieces, although all the information displayed was public. At the same time, players manipulated each other’s cards to suggest strategies. Most of the players used deictic utterances accompanied by gestures when manipulating pieces to visually propose game strategies to other players.

When asked about how interactive tokens fostered gameplay, players answered that they were helpful as (i) memory helpers “[tokens’ LCD display] let rules be clearer and there is no need to remember them,” and (ii) facilitators for social interaction “[tokens] add more interaction with people and make it easier to remember actions,”

Regarding the display of information, in the cardboard prototype, all information was displayed using multiple but static paper displays. Conversely, with the introduction of the interactive-token prototype, players could see only the information shown on tokens’ displays. This information was automatically updated by the system, reflecting the game status. This resulted in a sense of challenge for most of the players. For example, to make up for the perceived lack of control, a player used colored candies to track missing information on the board space.

The result of adding computer interactivity in the token-based prototype was an added sense of “tension” or “suspense” largely experienced by all the players. This was an intended feature of the game as a reminder of the stress experienced in real emergency situations. We surveyed players about their emotions and stress levels during the game. In the cardboard experiment, only 30% of the players perceived the game as “stressful.” For the technology-augmented prototype, we had stronger reactions, as demonstrated by several utterances, for example, “This game is making me anxious!” and “I’m starting to panic now! These sounds are really…” As revealed by the last quotation, auditory feedback played a strong role not only in creating an ambience but also in giving information to the players (e.g., when a particular event or result of an action was triggered). Another element of tension was injected by events printed on cards during the game; the stress derived from not knowing which event would happen next.

Finally, when players were asked to compare Don’t Panic with respect to a traditional board game experience (open question), they considered it to be “less repetitive, quicker, more reactive,” amusing, and more interesting because of the interactive tokens: “[Interactive] objects add to the realism of the game.”

7. Discussion

Our evaluation highlights that the introduction of technology did not alter the traditional social affordances of board games. Even if the interactive tokens were richer in terms of action and feedback than traditional game pieces, this choice did not disrupt the flow of action in the game. Regarding physical interaction, game dynamics were successfully implemented through sequences of interaction events.

In this section, we reflect on our proposed approach, highlighting strengths and discussing the challenges posed by the design and implementation steps for digital board games.

7.1. Strengths

7.1.1. Blending strengths from the physical and digital domain

The blend of elements taken from the digital and analog worlds introduced new design opportunities that, in our experience, resulted in added interactivity and fun for the players. For example, by adopting a card printer, we were able to mix in the power of the digital domain to sort and select a huge number of choices, yet preserve the physicality of tangible interaction with cards, their flexibility of manipulation, and their extended visibility. We observed that game cards printed “on the fly” brought elements of excitement and surprise due to the players’ anticipation while a card was (slowly) being printed. Furthermore, information on cards can be designed to be highly contextual with the status of the game or random or tailored to the role of the active player or the level in the game. The physicality of cards also allows for playful interaction not conventionally available in traditional board gaming: cards can be annotated, kept by the players for future reference, or tossed.

7.1.2. Unconstrained interactivity

Besides the analog affordances of board gaming, in our approach, video game interactivity (e.g., 3-D graphics and audio), useful in conveying rich information and creating ambience, can still be exploited by designers to a certain degree. Interactivity, rather than being confined to a single interactive surface, becomes mobile as it is being distributed across an ecology of tokens, implementing a multi-display environment. This opens two new design opportunities. First, the role of computer graphic representations provided by tokens’ LCDs can serve as both private and public displays. For example, a token can provide secret information when it is sheltered in a player’s hand, yet become a display of public information when it sits on a board constraint. Tokens can be scattered around the board to provide dynamic information over static regions of space and can be positioned side by side for extending the display surface. This opportunity could be further exploited in designing games that make use of single-player interaction with tokens when they are off the board and multiplayer interaction once they are back within board edges.

7.2. Design challenges

7.2.1. Balancing physical and digital control and representation

In designing token-interactive games, designers face the challenge of understanding how to adopt technology in their games, balancing different aspects. The first and most important challenge is how to preserve as much as possible the rich, collaborative sensory experience of interacting with physical pieces, using technology to provide extended control and representation capabilities; rather than just virtualizing those pieces.

For example, the design of tangible and intangible representations is critical to avoid usability issues. In Don’t Panic, a single token (Fig. 6(b)) captures information about the player (role and number of actions left) and information about the state of the game (distribution of people and panic levels). Although providing a quick awareness of the game’s status, in our experience this design has been perceived by the players as overloading and confusing. This issue presents a wider design challenge: how to find the right balance between the information encoded in tangible representations and information represented in dynamic intangible ones (e.g., on small, embedded displays). Furthermore, it is important to pay attention to which symbolic and iconic representations are adopted. For example, we used a discrete color-coded scale to symbolize ranges of values (panic levels). Because the information only updated when a threshold was reached, most of the players experienced this design choice as a frustrating lack of feedback from the system. Though this is a general Human Computer Interaction problem, it takes a different connotation when using a TUI approach, which imposes stricter limitations on the design space compared to GUIs [26]. The design choices for tokens’ intangible representations can be influenced by a specific technology, by the physical affordances of the token, or by adding a fun factor. Further research is necessary to understand how to create equilibrium between objects, interactions, and technologies so that the sensory experience is enhanced and does not become overwhelming.

7.2.2. Deciding task automation

One of the roles of technology in any kind of game is to take over game management tasks and keep track of achievements rather than requiring a human player to act as game master. Furthermore, technology might be used to facilitate rule learning, allowing players to avoid the onset of boredom before they finish reading a manual. However, completely delegating rule mastery to the machine could negatively impact the game experience, especially for expert players. For example, in Pandemic, one of the rules requires players to add colored cubes to infected areas. In Don’t Panic, in the same vein, panic levels increase when a panicking event happens. In a first paper prototype, players were obliged to update these levels by hand. These physical actions, which take a certain amount of time, have the role of making the players reflect on what is happening and start to think about counterattacking strategies. When the rule is automated, this awareness moment is suppressed. It should also be noted that learning game rules might be important in the case of serious games, when the rules are part of the intended learning process. Finally, by hard-coding rules in games and objects, delegating them to the computer can restrain players’ freedom. For a designer, it is therefore important to automate or eliminate only tasks that do not involve interesting or social decisions, thus automating only the boring, repetitive tasks. Though this is an issue relevant to any translation of games, it raises new issues when augmenting game pieces. Designers should pay particular attention to the virtualization and/or augmentation of game pieces used for making strategic choices, while virtualization and/or augmentation of pieces used for simple mundane tasks, such as those required for board setup, seem more straightforward. Further research is needed to provide design guidelines specific to augmented board games.

7.3. Implementation challenges

7.3.1. Choosing the right technology

Our review of the state of the art in digital board games (Section 2) reveals a wide range of complementary technologies that can be used for implementing board games. We want to remark that there is no technology solution that is better a priori. When translating a game to the digital domain, it is important to carefully choose how to augment game pieces and game dynamics to best exploit the opportunities provided by specific technologies.

The interactive-token approach implemented in Don’t Panic indeed makes use of multiple displays embedded in game tokens as well as sensors and actuators typical of ADI interaction. It could even be extended by technologies such as stationary or mobile interactive surfaces. For example, touchscreens might be better than a paper booklet at providing visual context for storytelling. The action of rolling dice, which delivers rich sensory feedback, might be kept closer to its traditional version by putting sensors in traditional physical dice, rather than rolling a virtualized version of them on a touchscreen. The right choice depends on the specific game and the game experience that the designer wants to promote. Further research is needed to identify, through large-scale evaluations, for example, how different technologies can be combined to provide engaging playing experiences and to provide designers with best practices about the use of different technologies for augmenting board games.

7.3.2. Implementing augmented board games

Moving from design to implementation, there is a set of challenges to be addressed. Developing digital board games has traditionally required coding software using game engines and interface-builder toolkits, such as Unity10

¹⁰
Unity game engine, https://unity3d.com.

or Phaser,11

¹¹

Phaser game engine, http://phaser.io.

or using toolkits for tabletop computers such as reacTIVision.

The interactive-token approach, as well as recent trends in ADI, requires developers to also build game pieces augmented with electronics and crafted in wood, metal, or plastic. This requires skills and tools that go beyond traditional software development, although, as emerged from our analysis of the state of the art, few technology toolkits have been specifically designed to facilitate this task.

In implementing Don’t Panic, we were challenged by the current lack of technology tools to support the transition between design and implementation (e.g., hardware components and software libraries). In order to build tokens that allow for the interaction events required in Don’t Panic, we had to use multiple hardware platforms, use different coding languages, and hack the Sifteo platform. Although this modus operandi was coherent with the goal of rapid-prototyping the game to validate our approach, it imposed limitations on the generalization of our approach and high entry barriers for designers. The lack of a technology toolkit might create barriers to the implementation of a planned sequence of interaction events. For example, the use of Sifteo Cubes as tokens in Don’t Panic required adding a final step: pushing the upper surface of the cube to signal that the operation was terminated when moving the token between nodes, thus creating a breakdown in the user experience.

Closer cooperation with the Internet of Things (IoT) and the maker movement could help to identify and develop technology tools suitable for supporting developing hybrid board games. Research in those areas provides generic hardware tools, such as Arduino [40], and novel technologies that can be used to build augmented game pieces. For example, conductive ink or fabric could be used to create interactive boards without employing touchscreen technology. IoT technologies could be used to augment traditional game pieces. In Don’t Panic, for example, RFID tags could be used to recognize cards more reliabily than using barcodes, although at the cost of added complexity for developers. Mobile devices, such as smartphones and tablets, could also be used to complement interaction with interactive tokens (e.g., running game engine code or displaying extended controls to complement tokens’ interactivity). IoT and maker movements count on large communities based on sharing of knowledge and open source hardware and software. This model, shared by many open source communities, could facilitate the development of the interactive-token approach to board games by lowering the threshold for developing and circulating new concepts.

7.3.3. Reducing the cost factor

Finally, there are challenges related to the cost of developing prototypes. While normally, cost is not considered before commercialization, it could strongly impact where the game could be used. Also, high costs and limited availability of the needed technology might influence the feasibility of arranging usability studies during development. Implementing an interactive-token game is still very expensive because of the cost of the different technologies (e.g., Sifteo Cubes) that are not mainstream.

Moreover, augmented objects are often custom designed and implemented for a certain game, making reusability across different games unlikely. Further research is needed to understand how to develop modular and reusable components and to balance hybrid components that can be reused across different games with custom-designed elements.

8. Conclusions and future work

In this paper, we present the interactive-token approach to the design of digital board games. The proposed approach provides a change in perspective from mainstream works in interactive board games, which center design on interactive surfaces. Our approach, borrowing elements from ADI and MDE technologies, relies on physical manipulation of interactive objects on conventional surfaces, with the aim of preserving the physical and social affordances that are the basis of the success of traditional board games.

The contribution of the paper is twofold. First, we provide an extensive state of the art in the area of digital board games. The literature review identifies two main approaches that build on stationary or mobile interactive surfaces to realize the board.

The second and main contribution of this paper is in the extension of the Token+Constraint interaction approach [51], providing constructs that can be used by designers to augment board games with interactivity in accordance with the game rules. These constructs are intended as a way to describe games, supporting the transition to implementation.

The approach proposed in the paper successfully supported the design of Don’t Panic in terms of tokens, constraints, and interaction events. Results of the evaluation reveal that the social affordances of traditional board games are preserved and the addition of computer interactivity is well accepted. The design and implementation of the game served as our evaluation of the feasibility of the approach and allowed us to identify a set of challenges and opportunities that can be useful to other designers.

Starting with the experience discussed in this paper, we aim to formalize a design process for the creation of interactive-token board games and to provide a toolkit supporting game designers in the implementation of hardware and software for systems of tokens and constraints.

Footnotes

Acknowledgements

We thank the students who helped with the development of our prototype and the volunteers who joined our user studies, in particular, Mr. Gianni Della Valle for helping with the organization of the evaluation studies.

References

A.N.

Antle,

Bevans,

Tanenbaum,

Seaborn and

Wang, Futura: Design for collaborative learning and game play on a multi-touch digital tabletop, in: Proceedings of TEI, ACM Press, 2011, pp. 93–100.

Avrahami,

J.O.

Wobbrock and

Izadi, Portico: Tangible interaction on and around a tablet, in: Proceedings of UIST, ACM, 2011, pp. 347–356.

Bakker,

Vorstenbosch,

Van Den Hoven,

Hollemans and

Bergman, Tangible interaction in tabletop games, in: Proceedings of ACE, ACM Press, 2007, pp. 163–170.

Bakker,

Vorstenbosch,

Van Den Hoven,

Hollemans and

Bergman, Weathergods: Tangible interaction in a digital tabletop game, in: Proceedings of TEI, ACM Press, 2007, pp. 151–152.

Barakonyi,

Weilguny,

Psik and

Schmalstieg, MonkeyBridge, in: Proceedings of ACE, ACM Press, 2005, pp. 172–175.

Bellotti,

Back,

W.K.

Edwards,

R.E.

Grinter,

Henderson and

Lopes, Making sense of sensing systems: Five questions for designers and researchers, in: Proceedings of CHI, ACM, 2002, pp. 415–422.

Benford,

Crabtree,

Flintham,

Drozd,

Anastasi,

Paxtonet al., Can you see me now?ACM Transactions on Computer-Human Interaction (TOCHI)2006 (2010), 100–133.

Bianchi and

Oakley, Designing tangible magnetic appcessories, in: Proceedings of TEI, ACM, 2013, pp. 255–258.

Björk,

Holopainen,

Ljungstrand and

K.-P.

Åkesson, Designing ubiquitous computing games – A report from a workshop exploring ubiquitous computing entertainment, Personal and Ubiquitous Computing6 (2002), 443–458. doi:10.1007/s007790200048.

10.

Bødker, Applying activity theory to video analysis: How to make sense of video data in human-computer interaction, in: Context and Consciousness, Massachusetts Institute of Technology, 1995, pp. 147–174.

11.

Burnett,

Coulton and

Lewis, Providing both physical and perceived affordances using physical games pieces on touch based tablets, in: Proceedings of IE, ACM, 2012.

12.

Butler,

Izadi and

Hodges, SideSight: Multi-“touch” interaction around small devices, in: Proceedings of UIST, ACM Press, 2008, pp. 201–204.

13.

Cooper,

Keatley,

Dahlquist,

Mann,

Slay,

Zuccoet al., Augmented reality Chinese checkers, in: Proceedings of ACE, ACM Press, 2004, pp. 117–126.

14.

Di Loreto,

Mora and

Divitini, Don’t Panic: Enhancing soft skills for civil protection workers, in: Proceedings of SGDA, Springer, Berlin, Heidelberg, 2012, pp. 1–12.

15.

Dietz and

Leigh, DiamondTouch: A multi-user touch technology, in: Proceedings of UIST, ACM, 2001, pp. 219–226.

16.

Dippon and

Klinker, Natural interaction for card games on multiple devices, in: Proceedings of ITS, ACM, 2013, pp. 409–412.

17.

Garcia-Sanjuan,

Catala,

Fitzpatrick and

Jaen, Around-device interactions: A usability study of frame markers in acquisition tasks, in: Proceedings of INTERACT, Springer International Publishing, 2015, pp. 195–202.

18.

Garcia-Sanjuan,

Jaen,

Catala and

Fitzpatrick, Airsteroids: Re-designing the arcade game using MarkAirs, in: Proceedings of ITS, ACM, 2015, pp. 413–416.

19.

Haller,

Forlines,

Koeffel,

Leitner and

Shen, Tabletop games: Platforms, experimental games and design recommendations, in: Art and Technology of Entertainment Computing and Communication, 2010, pp. 271–297. doi:10.1007/978-1-84996-137-0_11.

20.

Halskov,

Dalsgaard and

L.B.

Stolze, Analysing engaging experiences with a tangible 3D tabletop, in: Proceedings of ACE, ACM, 2014, pp. 29–10.

21.

Harrison and

S.E.

Hudson, Abracadabra: Wireless, high-precision, and unpowered finger input for very small mobile devices, in: Proceedings of UIST, ACM, 2009, pp. 121–124. doi:10.1145/1622176.1622199.

22.

Hasan,

Ahlström and

Irani, Ad-binning: Leveraging around device space for storing, browsing and retrieving mobile device content, in: Proceedings of CHI, ACM, 2013, pp. 899–908.

23.

Heijboer and

Van Den Hoven, Keeping up appearances: Interpretation of tangible artifact design, in: Proceedings of NordiCHI, ACM, 2008.

24.

Horn,

Atrash Leong,

Block,

Diamond,

E.M.

Evans,

Phillipset al., Of BATs and APEs: An interactive tabletop game for natural history museums, in: Proceedings of CHI, ACM Press, 2012, pp. 2059–2068.

25.

Ip and

Cooperstock, To virtualize or not? The importance of physical and virtual components in augmented reality board games, in: Proceedings of ICEC, Springer, Berlin, Heidelberg, 2011, pp. 452–455.

26.

Ishii, Tangible bits: Beyond pixels, in: Proceedings of TEI, ACM, 2008.

27.

Jordà, The reactable: Tangible and tabletop music performance, in: Proceedings of CHI Extended Abstracts, ACM, 2010, pp. 2989–2994.

28.

Jordà,

Geiger,

Alonso and

Kaltenbrunner, The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces, in: Proceedings of TEI, ACM, 2007, pp. 139–146.

29.

Ketabdar,

K.A.

Yüksel and

Roshandel, MagiTact: Interaction with mobile devices based on compass (magnetic) sensor, in: Proceedings of IUI, ACM, 2010, pp. 413–414.

30.

Kratz and

Rohs, Hoverflow: Exploring around-device interaction with IR distance sensors, in: Proceedings of MobileHCI, ACM Press, 2009.

31.

Krzywinski,

Chen and

Røsjø, Digital board games: Peripheral activity eludes ennui, in: Proceedings of ITS, ACM, 2011, pp. 280–281.

32.

Latour, Circulating reference: Sampling the soil in the Amazon forest, in: Pandora’s Hope: Essays on the Reality of Science Studies, Harvard University Press, 1999, pp. 24–79.

33.

Leitner,

Haller,

Yun,

Woo,

Sugimoto,

Inamiet al., Physical interfaces for tabletop games, ACM Computers in Entertainment (CIE)7 (2009).

34.

Magerkurth,

A.D.

Cheok,

R.L.

Mandryk and

Nilsen, Pervasive games: bringing computer entertainment back to the real world, ACM Computers in Entertainment (CIE)3 (2005).

35.

Magerkurth,

Engelke and

Memisoglu, Augmenting the virtual domain with physical and social elements: Towards a paradigm shift in computer entertainment technology, in: Proceedings of ACE, ACM, 2004.

36.

Magerkurth,

Memisoglu,

Engelke and

Streitz, Towards the next generation of tabletop gaming experiences, in: Proceedings of Graphics Interface 2004, 2004, pp. 73–80.

37.

Magerkurth,

Stenzel and

Prante, STARS – A ubiquitous computing platform for computer augmented tabletop games, in: Proceedings of UBICOMP, 2003.

38.

R.L.

Mandryk and

D.S.

Maranan, False prophets: Exploring hybrid board/video games, in: Proceedings of CHI Extended Abstracts, ACM, 2002, pp. 640–641.

39.

Marco,

Baldassarri and

Cerezo, ToyVision: A toolkit to support the creation of innovative board-games with tangible interaction, in: Proceedings of TEI, ACM, 2013.

40.

Mellis,

Banzi,

Cuartielles and

Igoe, Arduino: An open electronic prototyping platform, in: Proceedings of CHI Extended Abstracts, Proc. CHI, 2007.

41.

Merrill,

Sun and

Kalanithi, Sifteo cubes, in: Proc. of CHI EA 2012, 2012, pp. 1015–1018.

42.

M.R.

Morris,

Cassanego,

Paepcke,

Winograd,

A.M.

Piper and

Huang, Mediating group dynamics through tabletop interface design, IEEE Computer Graphics and Applications26 (2006), 65–73. doi:10.1109/MCG.2006.114.

43.

Mueller,

Dippon and

Klinker, Capture the flag: Engaging in a multi-device augmented reality game, in: Proceedings of ITS, ACM, 2015, pp. 277–282.

44.

Rogers and

Rodden, Configuring spaces and surfaces to support collaborative interactions, in: Public and Situated Displays, Springer, Netherlands, Dordrecht, 2003, pp. 45–79.

45.

Shen,

F.D.

Vernier,

Forlines and

Ringel, DiamondSpin: An extensible toolkit for around-the-table interaction, in: Proceedings of CHI, ACM, 2004.

46.

J.G.

Sheridan,

Tompkin,

Maciel and

Roussos, DIY design process for interactive surfaces, in: Proceedings of HCI, British Computer Society, 2009, pp. 485–493.

47.

Spielberg,

Sample,

S.E.

Hudson,

Mankoff and

McCann, RapID: A framework for fabricating low-latency interactive objects with RFID tags, in: Proceedings of CHI, ACM, New York, New York, USA, 2016, pp. 5897–5908.

48.

Terrenghi,

Kirk,

Sellen and

Izadi, Affordances for manipulation of physical versus digital media on interactive surfaces, in: Proceedings of CHI, ACM, 2007, pp. 1157–1166.

49.

Tolstoi and

Dippon, Towering defense: An augmented reality multi-device game, in: Proceedings of CHI Extended Abstracts, ACM, 2015, pp. 89–92.

50.

Ullmer and

Ishii, Emerging frameworks for tangible user interfaces, IBM Systems Journal39 (2000), 915–931. doi:10.1147/sj.393.0915.

51.

Ullmer,

Ishii and

R.J.K.

Jacob, Token+constraint systems for tangible interaction with digital information, ACM Transactions on Computer-Human Interaction (TOCHI)12 (2005). doi:10.1145/1057237.1057242.

52.

van Loenen,

Bergman,

Buil,

van Gelder,

Groten,

Hollemanset al., Entertaible: A solution for social gaming experiences, in: Workshop on Research and Design for Tangible and Tabletop Games at IUI, 2007.

53.

A.D.

Wilson, PlayAnywhere: A compact interactive tabletop projection-vision system, in: Proceedings of UIST, ACM, 2005, pp. 83–92.

54.

Wolfe,

J.D.

Smith and

T.C.

Graham, A low-cost infrastructure for tabletop games, in: Proceedings of FuturePlay, ACM, 2008, pp. 145–151.

55.

N.-H.

Yu,

L.-W.

Chan,

S.Y.

Lau,

S.-S.

Tsai,

I.-C.

Hsiao,

D.-J.

Tsaiet al., TUIC: Enabling tangible interaction on capacitive multi-touch displays, in: Proceedings of CHI, ACM, 2011, pp. 2995–3004.

Game	Year	Technology Strands

		Stationary Surfaces		Augmented Objects	Mobile Surfaces

		Augmented Reality	Tabletop Displays		Smartphones/Tablets	MDE	ADI
DiamondTouch Games [15]	2001		∙
False Prophet [38]	2002		∙	∙
Monopoly STARS [37]	2003		∙	∙
Augmented Reality Chinese Checkers [13]	2004	∙
DiamondSpin [45]	2004		∙
Monkey Bridge [5]	2005	∙		∙
PlayAnywhere [53]	2005		∙
Matching, Poetry and Classification Table [42]	2006		∙
Weathergods [4]	2007		∙	∙
YellowCab [52]	2007		∙	∙
Totti [23]	2008		∙	∙
Asterocks [54]	2008		∙
IncrediTable [33]	2009		∙	∙
ePawn	2010			∙	∙
Futura [1]	2011		∙
Portico [2]	2011			∙	∙		∙
Of BATs and APEs [24]	2012		∙
Pong+ [11]	2012			∙	∙
zAPPed Monopoly	2012			∙	∙	∙
Toy Vision [39]	2013		∙	∙
iPieces	2013			∙	∙
Scrabble for iOS	2013				∙	∙
Projected Play [20]	2014		∙	∙
Capture the Flag [43]	2015	∙			∙	∙
Towering Defense [49]	2015	∙			∙	∙
Airsteroids [18]	2015			∙	∙	∙	∙
RapID Tic-Tac-Toe [47]	2016			∙			∙

From interactive surfaces to interactive game pieces in hybrid board games

Abstract

Keywords

1. Introduction

2.1. Literature review methodology

2.2. Stationary interactive surfaces

1 Top 100 iOS applications, https://hec.su/dEL5.

4 Scrabble for iPad, http://ea.com/scrabble-paid-ipad.

5 Skylanders video game, http://skylanders.com.

3. The interactive-token approach to board games

3.1. Control and representation

3.2. Architectural view of interactive-token board games

3.4. Key design constructs

4. Applying the interactive-token approach

4.1. Don’t Panic game dynamics and rules

8 Pandemic game, http://zmangames.com.

4.3. Design of spatial configurations

Table 2 Valid spatial configurations and interaction events Token Allowed Constraint Allowed proximity with other token Pawn Nodes (physical) Adjacency to Calm! and Move! Calm!, Move! Sectors (visual) Adjacency to Pawn Cards Card Deck (visual) None

9 Interaction events demonstrated, http://research.idi.ntnu.no/dontpanic/video.

5.1. The game engine

5.2. Token handlers

6. Evaluation

7.1. Strengths

7.1.1. Blending strengths from the physical and digital domain

7.1.2. Unconstrained interactivity

7.2. Design challenges

7.2.1. Balancing physical and digital control and representation

7.2.2. Deciding task automation

7.3. Implementation challenges

7.3.1. Choosing the right technology

7.3.2. Implementing augmented board games

10 Unity game engine, https://unity3d.com.

8. Conclusions and future work

Footnotes

Acknowledgements

References

¹
Top 100 iOS applications, https://hec.su/dEL5.

⁴
Scrabble for iPad, http://ea.com/scrabble-paid-ipad.

⁵
Skylanders video game, http://skylanders.com.

⁸
Pandemic game, http://zmangames.com.

Table 2
Valid spatial configurations and interaction events

Token Allowed Constraint Allowed proximity with other token

Pawn Nodes (physical) Adjacency to Calm! and Move!

Calm!, Move! Sectors (visual) Adjacency to Pawn

Cards Card Deck (visual) None

⁹
Interaction events demonstrated, http://research.idi.ntnu.no/dontpanic/video.

¹⁰
Unity game engine, https://unity3d.com.