Training Joint Attention Skills and Facilitating Proactive Interactions in Children With Autism Spectrum Disorder: A Loosely Coupled Collaborative Tabletop-Based Application in a Chinese Special Education Classroom

Abstract

Joint attention skills, broadly speaking, refer to the ability to share experiences and interests about objects and events with others in a social environment. Due to their impairment in joint attention skills, children with autism are usually facing communication challenges, for example, being reluctant to socialize and share items with others. Previous research, aiming to enhance and promote collaboration among children in this population, had designed collaborative games for children to share a single workspace with highly coupled collaborative tasks, which thus could result in forced collaboration and reduce game playability. To address this issue, we design a collaborative puzzle game to indirectly foster players’ collaboration and proactive interaction, to train their joint attention skills gradually. The application is different from previous ones in two aspects: (a) It supports loosely coupled collaboration between two players and (b) it focuses on measuring/promoting the joint attention skills among players. Specifically, some puzzle pieces of a player will be systematically put in another player’s workspace, and they are expected to notice and share them with the other player to complete the whole game. In addition, the game complexity can also be adaptively adjusted according to the players’ playing behavior.

Keywords

autism joint attention skills puzzle game special education

Introduction

Joint attention (i.e., shared visual attentional focus on an object) is not only regarded as a key developmental building block in infants’ play, social skills, learning, and language development but also considered as a necessary precursor for Theory of Mind (Baron-Cohen, Leslie, & Frith, 1985; Tomasello, 1995). It is known that individuals with autism spectrum disorder (ASD) have delayed development in joint attention skills that play a critical role in their social-emotional development, subsequent language development as well as emotion regulation skills (Dawson, Bernier, & Ring, 2012; Lai, Lombardo, & Baron-Cohen, 2013; Morales, Mundy, Crowson, Neal, & Delgado, 2005; Mundy, 1995; Yuill & Rogers, 2012). Children with ASD prefer not to socialize (Silva, Raposo, & Suplino, 2015), which in turn further prohibit them from gaining social-emotional skills that are key in social activities. And those with low-functioning ASD (LFASD) exhibit higher degree of deficiency in social attention skills and demonstrate more indifference to social interaction, which thus lead to higher degree of isolation (Wing, 1988). Although stronger connections with their loved ones are formed as individuals grow older, their social communication skills remain superficial compared with their counterparts with typical development (TD; Lai et al., 2013; Rehg et al., 2013).

Over the past several years, a number of collaborative technology-enabled training applications emerged to provide a safer, affordable, and repetitive environment for children with ASD (Chen, 2012; Kientz, Goodwin, Hayes, & Abowd, 2013; Millen et al., 2011). Among them, collaborative games have gained prominence to train children joint attention skills (e.g., Battocchi et al., 2009; Boyle, Arnedillo-Sanchez, & Zahid, 2015; Gal, Lamash, Bauminger-Zviely, Zancanaro, & Weiss, 2016; Giusti, Zancanaro, Gal, & Weiss, 2011; Goh, Shou, Tan, & Lum, 2012; Millen et al., 2011; Piper, O’Brien, Morris, & Winograd, 2006; Rehg et al., 2013), mainly due to its inherent multiuser features to promote and facilitate collaborative work and information sharing (Edward, Greenberg, Shen, & Forlines, 2007; Silva et al., 2015). However, the majority of these games are often developed to enhance children’s social communication skills and promote cooperative works (Battocchi et al., 2009; Ben-Sasson et al., 2013; Boyle et al., 2015; Gal et al., 2016; Giusti et al., 2011; Goh et al., 2012; Hourcade, Bullock-Rest, & Hansen, 2012; Millen et al., 2011; Rehg et al., 2013; Silva et al., 2015); as such, these applications contain only one workspace where collaboration is tightly coupled (Battocchi et al., 2009; Giusti et al., 2011; Piper et al., 2006).

However, social interactions cannot be solely measured by the performance of a particular task but rather are defined by the degree of the reciprocity between engaging participants (Rehg et al., 2013). Prior works had revealed that individuals with ASD showed diminished social motivation that has led them to be intrinsically less willing to seek acceptance and thus socialize with others (Cage, Pellicano, Shah, & Bird, 2013; Chevallier, Kohls, Troiani, Brodkin, & Schultz, 2012; Izuma, Matsumoto, Camerer, & Adolphs, 2011). Therefore, in contrast to previous works that focus on the interaction patterns as afforded by such a platform, our proposed games take further steps by adaptively strengthening the reciprocity between children and facilitating proactive interactions (Tentori & Hayes, 2010). We hope these can help them develop joint attention skills accordingly in a more unobtrusive and natural way; that is, unlike previous studies that implemented stronger degree of enforced collaboration (EC) for players to share a single workspace with highly coupled collaborative tasks, our games allow players to focus on their own separate workspace while helping each other to finish his or her own task.

Little is known, however, about whether collaborative games could be effective in promoting and enhancing children’s joint attention skills. Furthermore, similar games as well as general games specifically designed for Chinese children with ASD are rare (Silva et al., 2015), which together motivates the development of our games in the present study. Instead of performing human studies to find the most effective collaborative games, our goal is to design loosely coupled game platforms that can be configured to train children joint attention skills through in-game prompting (Whalen & Schreibman, 2003; Wong & Kasari, 2012). Our pilot testing enabled us to assess the learning benefits of such games in relation to players’ joint attention skills and understand their behavioral patterns exhibited during the rounds of repeated game-play. Following feedback from special education teachers, we redesigned our game so as to provide in-game data recording capabilities that, in the future, could help teachers to monitor students’ learning progress.

The organization of this article is as follows: In the next section, we present earlier works on the application of collaborative game in ASD and present our design rationale; we then provide detailed discussions on our design process and conclude this article by reflecting the research down this path in China. It is our hope that the study would also provide early glimpse into the research landscape of technology-oriented applications for individuals with ASD in China.

Related Works

Multiuser Large-Screen Collaborative Games and Their Benefits

Numerous prior studies such as Alghanim (2013); El-Nasr et al. (2010); Gross, Fetter, and Liebsch (2009); Khaled, Barr, Johnston, and Biddle (2009); Morris, Huang, Paepcke, and Winograd (2006); Piper et al. (2006) have highlighted the advantageous features afforded by multiuser large-screen tabletops for a general population: (a) direct manipulation of digital objects afforded through simple touch; (b) large enough social spaces for all participants including onlookers; (c) large enough screen prevents a user from easily reaching to the other end of the screen to manipulate screen objects, thus providing partitioned spaces for both players; and (d) programmable to force the design of joint actions in the applications. Moreover, there is a growing recognition that in such a play environment, successful collaborations largely depend on tight and high awareness of each other’s actions and intentions that can be obtained via a common view of the workspace and a constant view of each other (Dietz & Leigh, 2001; Tang, Wang, You, Huang, & Chen, 2015). Collaborative games are especially popular among children and teenagers between 6 and 16 years old (El-Nasr et al., 2010). While these previous studies differ in their focuses and tasks supported, they unanimously pointed out that the degree of reciprocity and engagement between participating individuals is crucial to allow both parties to collaborate effectively.

Recently, the attention has been shifted to the design of such games to improve social communication skills for children with ASD, among many, Battocchi et al. (2009), Boyle et al. (2015), Chen (2012), Gal et al. (2016), Giusti et al. (2011), Goh et al. (2012), Hourcade et al. (2012), Millen et al. (2011), and Rehg et al. (2013). In the next subsection, we will survey these notable studies and compare them with ours.

Collaborative Applications for Children With ASD

The intrinsic deficits of ASD individuals in initiating and maintaining social interactions with their peers during game-play have made them tend to engage in more parallel play than their TD peers who engage more in cooperative play (Bauminger et al., 2008). Hence, the multiuser collaborative works and information sharing afforded by large-screen tabletops offer unprecedented opportunities for this population (Silva et al., 2015). The major differences among collaborative games including ours center on the constraints embedded in the interaction patterns and game items (Battocchi et al., 2009; Ben-Sasson et al., 2013; Gal et al., 2016; Giusti et al., 2011; Goh et al., 2012; Piper et al., 2006; Silva et al., 2015). Based on the design patterns in these earlier works, we categorize two broad types of interaction patterns: EC and free play (FP) patterns. Our discussions on these previous games will be unfolded from these two patterns.

In the puzzle game presented and studied by Battocchi et al. (2009), all children must touch and drag the puzzle pieces together at the same time to move them, which requires a high degree of joint attention for involving children to finish it, and therefore EC must be executed where players are forced to take joint actions on screen objects. The experiment demonstrated that such execution constraints had provided an excellent opportunity for on-demand training on children’s social interaction and attention skills. However, due to the large number of nonfunctional free moves, children might get frustrated, and frequent ECs might prevent players from enjoying the game (Battocchi et al., 2009).

Similarly, the game in Piper et al. (2006) requires both higher degree of joint attention and stronger coupling of interaction patterns: A group of four children with Asperger’s syndrome (a type of high-functioning autism) need to build a path for a frog on a tabletop surface starting with performing rudimentary tasks such as determining the number of tiles on the path and the placement of them. EC and a set of other design constraints had been imposed to help children facilitate the negotiation (voting through buttons). To be able to make progress in the game, all votes must be casted unanimously, which tend to again conversely affect game playability.

Other game used in Goh et al. (2012) has also implemented EC in their tabletop games where as many as four children aim to clean the ocean mess by moving floating cans, logs, and so forth, which further encourage larger group interactions. And Gal et al. (2016) empirically deployed two tabletop-enabled collaborative tasks (puzzle game and collage) to assess the level of social interaction skills of high-functioning ASD (HFASD) children following the Friendship Observation Scale (Bauminger et al., 2008); EC patterns (referred to joint performance in the article) were adopted. A small-scale empirical study revealed that the game can positively enhance social skills.

While EC could be helpful to promote collaboration, it may not be effective when the participants have different cognitive skills because those with better skills have to wait for other(s), putting more pressure to the latter, and eventually frustrate all participants. Grouping children with similar cognitive skills become a necessary condition for playing such games, which is not always possible in many learning or training centers, especially in China. Besides, it is unclear whether the improved social attention and joint attention skills learned from playing these games are transferrable to real-life situations (Boyle et al., 2015; Chen, 2012). In addition, for children with ASD, Rehg et al. (2013) has pointed out that the play environment should be designed to foster easy manipulation of the screen elements, thus minimize the cognitive workload accordingly (Haller, Forlines, Koeffel, Leitner, & Shen, 2010).

The tabletop collaborative puzzle game presented in Ben-Sasson et al. (2013) is similar to that in Battocchi et al. (2009) and employed both FP and EC interaction patterns; a small-scale pilot testing with six HFASD boys revealed that the EC interaction pattern can potentially promote goal-oriented joint tasks among children; however, negative effects were also uncovered with inconclusive results due to the limited number of samples. Finally, Giusti et al. (2011) constructed a number of well-design interaction patterns (play constraints) based on the cognitive-behavioral therapy (Hart & Morgan, 1993). Such patterns including “choosing together,” “constraints on objects,” and “different role” have to be used with interventions from a specially trained teacher or a therapist who act as a facilitator during the collaborative task-performing process.

All aforementioned games are developed to promote social communication skills among children; yet, only a few collaborative games have emerged to focus and assess the level of joint attention skills children exhibit during game-play. However, such advanced preferential attention, perceptual and semantic processing skills are essential indicator of the joint attention skills (Ames & Fletcher-Watson, 2010). Our present work tends to fill the void by employing an FP-based tabletop collaborative puzzle game to focus and assess joint attention skills through observation and in-game behavioral data analysis. Another reason to adopt FP is to allow two players with different cognitive skills to collaborate. In fact, having two players with different cognitive skills (e.g., an LFASD child and an HFASD/TD child) could be more effective to promote cooperative play (Bauminger et al., 2008).

Table 1 provides a summary of employed interaction patterns and evaluation dimensions of these studies to position ours in the landscape of collaborative multitouch game area for individuals with ASD.

Table 1.

Tabletop Applications on Promoting Social Skills Among Children With ASD.

Applications	Design pattern	Promoted social skills	Research work	Remarks
Four-player route-planning game (SIDES)	EC	Group work skills including negotiation and turn-taking	Piper et al. (2006)	HFASD (Asperger’s) only
Two-player collaborative (jigsaw) puzzle game	EC	Cooperation manifested in coordinated finger movement; positive social interaction, in particular goal-related behaviors	Battocchi et al. (2009)	Both HFASD and LFASD
Two-player collaborative (jigsaw) puzzle game	EC		Ben-Sasson et al. (2013)	HFASD only
Three-user games to support CBT (Join-In Suite)	CBT	Joint performance, sharing, and mutual planning	Giusti et al. (2011)	HFASD only
Two-player coordination game (PAR)	EC and FP	Passive/active sharing, joint performance, and turn-taking	Silva et al. (2015)	LFASD only
Collaborative storymaking (StoryTable)	EC	Joint performance, turn-taking, and negotiating	Gal et al. (2016)	HFASD only
Our proposed game: two-player (jigsaw) puzzle game	Loosely coupled	Joint attention		Both HFASD and LFASD

Note. ASD = autism spectrum disorder; SIDES = shared interfaces to develop effective social skills; EC = enforced collaboration; HFASD = high-functioning ASD; LFASD = low-functioning ASD; CBT = cognitive-behavioral therapy; FP = free play.

Attention and Joint Attention Skills in Children With ASD and Intervention Strategies

In the autism research community, relevant attention skills include shifting attention of targeted item(s), broadening or narrowing attention focus and areas of targeted items, and preferential attentional cueing guided by visual or audio stimuli (Ames & Fletcher-Watson, 2010). These attention skills, if studied in a social context, would encompass the social environment where the user situates; such attention could then be elaborated into triadic, dyadic, or shared gaze, which are known as the three levels of joint attention skills (Ames & Fletcher-Watson, 2010). Broadly speaking, these skills refer to the ability to share interests and experiences on objects and events with others in a social environment (Paparella & Freeman, 2015).

In the context of this study, we are interested in the joint attention where two individuals are expected to jointly coordinating their visual attention to an item (i.e., shared gaze), each other and others who could be in the environment. However, such coordination is often achieved through levels of responses to joint attention bids by another individual (i.e., a partner, a therapist, the parent) who is present in the social environment. In a natural classroom setting where the primary goal is to encourage children’s independence, Wong and Kasari (2012) revealed through empirical studies that teachers should not just provide functional assistance (i.e., problem solving) but also offer additional guidance and structured assistance to help children with ASD to seek social help and engage with others, which could in turn enhance their joint attention skills.

Prior works have proposed a wide variety of intervention and training strategies. In their work, Whalen and Schreibman (2003) documented two phases of joint attention intervention in a noncomputerized setting: response and initiation training. The former sponsors five levels of responses as “response to hand on object,” “response to object being tapped,” “response to showing of object,” “eye contact,” “following a point,” and “following a gaze.” The latter includes “coordinated gaze shifting” and “prodeclarative pointing.” Physical (i.e., taking the child’s hand to point), verbal (utterances such as “point to here”), and gestural prompts were adopted to further assist the children to engage with others especially during the response training phase (Whalen & Schreibman, 2003). Such response (responding to joint attention [RJA]) and initiation attention (initiating joint attention [IJA]) behaviors to joint attention bids had also been studied in a parent–child intervention setting known as Parent-Mediated Communication-Focused Treatment in Children with Autism (Green, Charman, & McConachie, 2010) and caregiver or parent-mediated behavioral intervention known as joint attention-mediated learning (Kasari et al., 2014; Odom, Schertz, & Baggett, 2011; Schertz, Odom, Baggett, & Sideris, 2013). These intervention methods usually combined an array of developmental behavioral approaches including reciprocal imitation training, interpersonal synchrony on prosocial behavioral, and the joint attention and symbolic play/engagement and regulation treatment (see Paparella & Freeman, 2015 for a recent survey).

The gesture-enabled collaborative block-building game in Boyle et al. (2015) is one of the few targeting joint attention training in technology-enabled intervention area; active and passive sharing, joint performance design patterns were employed to encourage such skills that linked to joint attention as turn-taking, active sharing, and shared gaze toward items. The game, which is a single-player turn-taking game, had been evaluated by teachers, parents, and therapists (including occupational, special language, and Applied Behavior Analysis/ABA therapist) with largely positive feedback. One notable suggestion is naturally to extend the game to allow at least two players to engage (Boyle et al., 2015).

Although the measurement for social collaborative works is different from that for the more advanced joint attention skills, they somehow share some overlapping items, as shown in Figure 1 where the right figure shows our measurement metrics and the training environment.

Figure 1.

The comparison of assessment dimensions in our application supporting joint attention skills training (right) with that of others supporting and promoting collaborative work (left).

Our Games

The puzzle games presented and studied in Battocchi et al. (2009) and Ben-Sasson et al. (2013) bear the strongest similarity to ours (in terms of a jigsaw puzzle game) and the most distant design approach to ours (in terms of the design goals and approach). In this section, we will elaborate these issues.

In contrast to earlier studies that largely tend to enhance and promote children’s collaborative works, ours focuses on their level of shared social play, joint attention, and reciprocal skills in a loosely coupled collaborative environment by allowing each child his or her own space. Hence, we blend the two interaction patterns (EC and FP), propose and adopt a loosely coupled interaction pattern in a collaborative puzzle game, which therefore might be appropriate for either HFASD or LFASD children. Two types of equipment are proposed: tabletop computer and tablet computer (see Figure 2(a) and (b)).

Figure 2.

Proposed equipment to support collaborative game-play. (a) Seeking and receiving helps in the tablet game; (b) seeking and receiving helps in the tabletop game (Version 1).

There are four reasons for us to implement a tablet game: (a) many children with ASD prefer parallel play instead of associative or cooperative one, hence using tablet will accommodate their needs; (b) tablets (including those designed for kids) are affordable by most households in China; (c) many commercial tablet games require players’ collaboration, so our game could serve as an introductory one; and (d) it is easy to develop and maintain mobile applications due to abundant resources (skillful programmers, integrated development environment/software development kit, courseware, cloud services, etc.). However, the focus of our discussion in this article will be on the design of tabletop games because they are more challenging and relevant in promoting joint attention skills.

Unlike earlier studies that require children to stand side by side around the tabletop (Ben-Sasson et al., 2013; Boyle et al., 2015; Giusti et al., 2011; Piper et al., 2006; Silva et al., 2015), ours expect children to play at the two edges of the tabletop facing each other (see Figure 2(b)). The purpose is to create two separated but overlapping workspaces for them, hence facilitating both parallel and associative play and avoiding coordination/turn-taking issues.

Design Consideration: Interaction Pattern Cultivated in Our Games—EC or FP?

We followed the collaboration patterns in Giusti et al. (2011) but adopted a loosely coupled interaction patterns so as to align with our design goal; such a pattern is referred to as LC. Figure 3 illustrates the desired interactions. Design constraints and facilitators were subsequently programmed and embedded in our game so as to assess, promote, and train children to be aware of task-specific items, each other, and the social environment.

Figure 3.

The interaction pattern supported in (a) tabletop—when Player 1 press “Help” button, Player 2 may notice a flashing piece (arrow) and pass it to Player 1, and (b) tablet games—Player 2 responded by swiping to “send” the puzzled to Player 1.

Silva et al. (2015) recommended four general interaction patterns as practical guidance for tabletop-enabled collaborative applications, that is, Passive Sharing, Active Sharing, Joint Performance, and Unrestricted Interaction Patterns. Joint Performance pattern extends the Active Sharing one by incorporating the cooperative “simultaneous” actions to enforce tighter collaboration and social communication between pairs, which had been criticized in Cage et al. (2013), Chevallier et al. (2012), and Izuma et al. (2011) that individuals with ASD should not be forced to socialize. Hence, our proposed design pattern (LC) aims at encouraging them to actively share items with others and in practice is similar to Active Sharing for both tablets and tabletop (see Figure 3: users collaborate while working on their own space).

Figure 4 shows the two play moments in the 27-inch tabletop, which is less popular than tablets. However, because tabletop games require coexistence of both players, it may promote associative play and verbal communication between players, hence would be useful for joint attention training and is desirable especially at school. Because most children receiving early intervention or training on joint attention skills are preschoolers (age 3 to 5 years), the games we choose must be easy but attractive enough for them. For these reasons, we designed the simplest puzzle games that can be manipulated easily through drag-and-drop operations only. The joint attention training is achieved by scattering some puzzle pieces that belong to a player to his or her opposite player’s working area such that both players are expected to pay joint attention to find misplaced pieces to complete the game. Joint attention bid could be achieved by tapping a “request” button that will trigger an animation of the misplaced piece to attract both players’ attention. Figures 5 and 6 show some modified versions of it after observing pilot testing results and recommendation from teachers/parents.

Figure 4.

A player who finished the game earlier must wait for the other (tabletop game Version 1).

Figure 5.

Tabletop game Version 2 with colored border indicates puzzle ownership and enclosed territory.

Figure 6.

Tabletop game Version 3 with colored border in each puzzle pieces.

Table 2 summarizes the five versions of tabletop games, including their unique features and purposes. Version 1, 2, and 3 have been tested, Version 4 has been implemented but not tested yet, and Version 5 has partially developed.

Table 2.

Tabletop Versions and Their Unique Features.

Ver.	Features	Design rationales	Remarks
1	– Ill-defined (borderless) playing area – Displaying target pictures near players – Using some photographs and illustrations – Flashing arrow near requested pieces (inducing joint attention) – Both players play the same game and can only play the subsequent one (up to 10 different games) after both players finish the present one.	We assume children are capable of observing their working area without border and be guided using the displayed target picture during game-play. The design is intended for children who like both illustrations and photographs with comparable playing skill. Children are incentivized to collaborate by preventing them to play the subsequent games without solving the present one.	It is the first prototype; after two pilot tests, we developed Versions 2 and 3.
2	– Well-defined (bordered) playing area with two colors (purple and blue) – No additional target pictures shown – Using large illustrations (see Figure 5) – Flashing pieces upon a player request to induce joint attention	Intended for preschooler or children with LFASD because they may ignore flashing arrows (used in Version 1), confuse with similar pieces without additional cue(s), and prefer larger illustrations. Two colors (purple and blue) are used as the border to differentiate players’ pieces; hence, easily be recognized.
3	– Borderless playing area but with bordered puzzle pieces – Displaying B/W target pictures in the background (see Figure 6) – Flashing pieces upon a player request when the missing pieces are near the other player, or otherwise a puzzle piece will automatically move to fill the correct position (quick-finish mode).	Intended to accommodate two children with a large playing skill gap. When the player with lower skill could not find a correct puzzle piece, she or he may tap help (“?”) button that will either move a piece to its correct place or flash a piece near the other player requesting him or her to pass it.
4	Similar to Version 3, except: – teachers can change/upload images; and – two players do not need to play the same game at the same time.	Considering various children and teachers’ preference, it would be better to allow them to edit/upload their chosen puzzle images. To reduce players’ waiting time, a more skillful player may be given more challenging games (e.g., 4 × 6 puzzles), while a less skillful player is given easier ones (e.g., 2 × 3 puzzles).	Completed but has not been tested
5	– Puzzle pieces will be redistributed (shuffled) according to the training progress. – A “desperate” request may freeze other pieces (EC).	We categorize requests by a player into “ordinary” and “desperate,” that is, whether or not the request is made after a long unsuccessful attempt. A “desperate” request for help should be addressed by another player; hence, it will interrupt his or her game-play by freezing all pieces but requested ones that is a strong EC design.	Partially completed

Note. LFASD = low-functioning autism spectrum disorder; EC = enforced collaboration.

Game Elements as Constraints to Support Joint Attention Skills

Recognizing the different design goals, we employed different patterns as those collaborative patterns in Silva et al. (2015) and those linked with joint attention skills in Boyle et al. (2015) in a loosely coupled large interactive space where each player retains his or her own workspace. As such, the interaction pattern, LC, adopted in our game has been transferred to several design constraints. For example, EC is happening only when a player seeks help from another player who may or may not be interrupted, and the puzzle pieces are distributed so that they will increase the collaboration opportunities.

In addition, because joint attention skills should be assessed within a collaborative work, we also followed a few patterns laid out in the general computer-supported cooperative work (CSCW) platform (Yuill & Rogers, 2012). Specifically, although we did not employ EC, certain aspects unique to EC were embedded in the digital objects. For example, the puzzles are clearly marked to signify “ownership” (with colored border in each puzzle pieces, see Figures 5 and 6), which thus provide “awareness” to both players (Yuill & Rogers, 2012). Furthermore, players who have finished present game must wait for their cohorts before they could play subsequent games, hence incentivize them to help their cohorts. In the latest version, any “desperate” request by a player will freeze all pieces of the other player for a few seconds so she or he could pay attention to requested pieces.

We also incorporated several play constraints in the game to provide implicit prompts in an attempt to replace the human facilitator’s prompts (Paparella & Freeman, 2015; Whalen & Schreibman, 2003) to players to act in certain ways, which in turn indirectly foster their collaboration, to train their joint attention skills gradually (see Table 3). Such in-game elements coupled with the automated in-game player behavioral data recording and analysis are expected to offer us an unprecedented window of opportunity into their temporal behavioral patterns. The latter of the approach is little known in the literature, as the majority of earlier works rely on expert observations. Table 3 shows the list of design features that are aligned with the goals and rationales of the game for the purpose of joint attention training within a CSCW platform based on Boyle et al. (2015) and Silva et al. (2015). Note, audio cue (reminder and reward/praise) were also used in the last three versions of our games.

Table 3.

Design Features and Their Associated Joint Attention Training.

Design feature	Joint attention	Collaborative pattern
– Flashing puzzle pieces – Automatically moving pieces	Visual attention and shared gaze	Passive sharing
– Selecting subsequent games – Passing desperately needed pieces upon request (when other pieces are frozen)	Share gaze and require dyadic joint attention skill (waiting for another player to tap a button)	Active sharing
– Passing pieces upon request (without interrupt/frozen pieces) – Passing pieces without any request – Helping cohort to finish the game	Require triadic joint attention skill	Active sharing and joint performance

For children with ASD, Rehg et al. (2013) has pointed out that the play environment should be designed to foster easy manipulation of the screen elements, thus minimize the cognitive workload accordingly (Haller et al., 2010). Hence, in Versions 3 and 4 of our game, we enabled an intelligent “quick-finish” mode. This feature is intended to help players with low cognitive skills to solve the puzzle.

Game Development and Running Environment

All games were developed using C# (managed code) in Visual Studio Ultimate 2015. The two Windows tablets are running Windows 8.1 with Bluetooth enabled; the tabletop used is Lenovo Horizon 2 (27”) running Windows 8.1.

Pilot Studies

Because ASD are heterogeneous, very unlikely two children using our equipment will belong to the same homogeneous subgroup with comparable cognitive skills. Therefore, instead of finding the most effective educational game (software), our focus is to develop configurable platform that can be easily adjusted and used by as many children as possible. Consequently, we are not interested in conducting control studies but collecting teachers’ feedback to improve the design. Hence, we employed a participatory design approach in which teachers, children, and their parents were frequently involved in the design iterations. Ethnographic research method was primarily used to evaluate our systems, that is, through observation and discussions with teachers and parents. Both participatory design and ethnography process are commonly used in CSCW research where the designed systems are intended be used by the participants only (Frauenberger, Good, & Keay-Bright, 2010; Kensing & Blomberg, 1998; Simonsen & Kensing, 1997).

Because the hardware (a commercial 27-inch touch screen tabletop) can be used with ease to play many preload multiplayer games, and our software requires only users’ single operation (dragging items) and has been tested well before used by children, we did not perform usability study to evaluate the software. Also, given the fact that some children are slower or less skillful in solving puzzle compared with others, and some are less interested in puzzle games, quantitative measurement (e.g., time to solve a problem, the mistake rate, number of requested helps, etc.) is less important compared with qualitative data in our interactive evaluation throughout this participatory design process. Nevertheless, these quantitative data recording is made available for teachers who may want to monitor their students’ progress (see Figure 7) but not used in our usability study or design iterations.

Figure 7.

An example of recorded gameplay data for teachers.

The second row in Figure 7 shows the IDs of two players, and the first column contains the piece IDs of puzzle pieces been touched/dragged by either the left or right-player; for examples, “pcsl11” in the fifth row represents “piece #11 on the left,” and “pcsr8” in the eighth row represents “piece #8 on the right”; “autoFillL” in the third row means the left-player had tapped “autofill” button. The second column contains the information of right/wrong attempts (1 or 2, respectively), and the third column contains the dragging duration in seconds. The fourth and fifth columns contain the screen coordinates of the puzzle piece’s initial location. In fact, only the first three columns (piece ID, right/wrong attempts, and duration) were suggested by some teachers. The last two columns were added to identify active sharing between two players. For example, the x-coordinate of the left-player’s puzzle pieces should be less than 990 (on the left-half of the screen), while those of the right-player should be greater than 990. However, some left-player’s puzzle pieces may be on the right-player’s workspace (x-coordinate > 990). An active sharing by the right player is plausibly happened when these left-player’s puzzle pieces were moved from the right-half of the screen to a random location in the left-half of the screen. A similar situation could be applied to identify an active sharing by the left-player. However, we have not implemented a classifier to identify the plausibility of such active sharing behavior, which is left for our future work.

Both tablet and tabletop games were pilot tested in a privately owned children’s autism educational development center and a public special education school. We installed the equipment in a common area, and some teachers (and parents) brought some children to try it during recess period. In addition, seven children accompanied by their parents and teachers have tried the system during their visit to our laboratory. Figures 4, 5, and 8 show some testing moments. All children, parents at present, and teachers showed high enthusiasm at the games, with a particular interest in the tabletop game because it is their first encounter and interaction with such a large-screen interactive tablet PC.

Figure 8.

The pilot testing moment including HFASD and LFASD children. (a) Two HFASD children were observed to play the tabletop game Version 1 at ease; (b) one child was playing the tablet game with another; (c) two LFASD children were observed to play the tabletop game Version 2.

We largely followed the subjective evaluation strategies as in previous works (Battocchi et al., 2009; Ben-Sasson et al., 2013; Gal et al., 2016; Giusti et al., 2011; Goh et al., 2012; Piper et al., 2006; Silva et al., 2015), including observation and teachers’ feedback. Some verbal comments recorded during the testing moments can be categorized into three parts: software features, contents, and usages. Some examples include (translated from Chinese):

Features:

– “Better come with voice feedback … praising and encouragement, many children cannot read Chinese characters.”

– “Now he could play it well, after played for a while and understood the game rule.”

– “It may not be easy to find two children with the same skill, maybe a more difficult puzzle in one side and an easier one in another side.”

– “Group the contents into different categories, so they can choose what to play. Maybe Animals and Vehicles?”

– “Could be more useful if we could record their progress, helping us to evaluate them, so to convince their parents.”

Contents:

– “Maybe we could use real photos, not illustrations. It could be more useful for children to recognize real items they will use everyday.”

– “What he tried to match is the pattern at the edge of two pieces, not the whole picture. He does not pay attention to the picture or color.”

– “Children may want to play longer if the images are what they like … . He likes dinosaurs.”

– “Many children like cars or moving vehicles.”

Usages:

– “He is eleven now and likes puzzle and computer very much. He is very patient to play with it.”

– “She is just trying to match it one by one, not really playing the puzzle.”

– “This device (tabletop) could be useful for our family.” (by a mother of twins with autism)

Some suggestions provided by the teachers include the following: (a) allow three or more children to play together; (b) allow the teacher to select the number of categories of pictures (configurable); (c) allow customization of the number of pieces (from 2 × 3 at present to 3 × 4); (d) use voice as feedback; (e) replace Chinese characters with images/symbols; and (f) use familiar picture/photos. Except for suggestion No. 1, the rests have been incorporated into our later game versions.

According to our observation, children were inclined to work on their own (parallel play moments appeared more frequently than associative play ones), while some HFASD children were being able to spot puzzle pieces that did not belong to themselves and share them proactively (with the verbal and gestural prompts from teachers). Overall, the levels of engagement and independence were high; goal-oriented coorporative behaviors had also been observed. For instance, during one moment, one child finished his own puzzle; because he was prohibited from continuing, instead of waiting for the other player to finish his tasks, the child went to his partner’s corner and help his partner to finish his tasks; whether or not his intention to actively offer help is driven by his desire to proceed to the next level or intrinsic cooperative intention is not clear. Further emprical studies to assess the social reptuation of children with ASD, especially those in China, would be necessary as insensitivity to establish, retain, and maintain social reputation has been offered as a plausible explaination for this population’s lack of Theory of Mind and social communication skills (Izuma et al., 2011).

In addition, compared with the earlier version (Version 1) where puzzle ownership was not explicit, the colored border wrapping each puzzle and the workspace had been shown to greatly facilitate sharing process and provide visual aid to encouarge nonverbal interactions (i.e., eye gaze). Teachers’ reactions were observed to greatly change from high skepticism (because at first glance, the game is no different from an ordinary puzzle game) to complete surprises with total acceptance. They especially expressed great enthusiasm when the first pair of LFASD children were observed from being lost in their own workspace and reluctant to play together to actively engage in the game-play with some prompts; in the later stages, they exchanged puzzle pieces. Two HFASD boys were observed being able to exchange gestural prompts to declare ownership of the puzzles in another’s workspace; and such achievement of providing gestural prompts indicate that their visual attentional skills had been employed during the game-play.

Overall, four primary lessons could be derived from our design process:

Given varying children cognitive skills and mental ages, adopting participatory design with proxy (teachers and parents) is the most suitable method in designing such learning application.

Input from both teachers and parents are equally important in which parents could provide more specific individual requirement (such as children’s preferences on music, objects, pictures, etc.), while teachers could suggest more suitable learning strategy (such as voice feedback, levels of difficulties, etc.).

Most children, including LFASD, could use touch-based application easily with minimum training, although some may not understand the game rules. Most of them have used smartphones and played with similar games. Therefore, instead of focusing on the UI manipulation, the designers should pay more attention on the game content and instruction to fit user’s need.

Configurable learning application is essential for children with special education needs. Both teachers and parents should be allowed to alter or add learning contents to fit their children needs and to alter other settings (voice feedback, level of difficulty, background sound, etc.) to keep their children motivated.

Conclusions

Although it is still too early to draw conclusions about the full potentials of such a loosely coupled game in offering benefits in enhacing ASD children’s joint attention skills, the series of our studies nevertheless underline the importance of such an envionrment, especially regarding the special design concerns of our application. In particular, Table 4 summarizes and compares the different and unique aspects of our design concerns in such an environment.

Table 4.

A Summary of Our Collaborative Environment and Previous Ones.

Applications	Promoted social skills	Focus on joint attention skills?	In-game behavioral data collection	Subjects
Four-player route-planning game (SIDES) (Piper et al., 2006)	Group work skills including negotiation and turn-taking	N	N	HFASD (Asperger’s) only
Two-player collaborative (jigsaw) puzzle game (Battocchi et al., 2009, 2010)	Cooperation manifested in coordinated finger movement; positive social interaction, in particular goal-related behaviors	Y	N	Both HFASD and LFASD
		Y	N	HFASD only
Three-user games to support CBT (Join-In Suite) (Giusti et al., 2011)	Joint performance, sharing, and mutual planning	N	N	HFASD only
Two-player coordination game (PAR) (Silva et al., 2015)	Passive/active sharing, joint performance, and turn-taking	Y	N	LFASD only
Collaborative storymaking (StoryTable) (Gal et al., 2016)	Joint performance, turn-taking, and negotiating	N	N	HFASD only
Our proposed game: two-player (jigsaw) puzzle game	Joint attention	Y	Y	Both HFASD and LFASD

Note. SIDES = shared interfaces to develop effective social skills; HFASD = high-functioning autism spectrum disorder; LFASD = low-functioning autism spectrum disorder; CBT = cognitive-behavioral therapy.

Moreover, the analysis and the overwhelmingly positive feedback from teachers on how fast LFASD children could pick up the skills and engaged in collaborative works (passing requested puzzle pieces) provide complementary evidence to the effectiveness of such a loosely coupled game approach and adds to the range of our understanding and data available for researchers to pursue down this path.

The magnitude and probability of gains in joint attention skills would be maximinzed if the intervention and training can be offered at home (Bruder, 2010; Paparella & Freeman, 2015). Our games have the potential of offering such an opportunities because (a) they are easily configurable and affordable and (b) they are equipped with in-game automatic data recording capabilities that, in the long term, could provide therapists rich data to adjust and personalize intervention strategies accordingly and gradually.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Pinata Winoto is an assistant professor in computer science at Wenzhou-Kean University. He received his PhD degree from University of Saskatchewan. His current research interests are recommendation systems, multiagent systems, assistive technology, and edutainment. He has published over 60 high-quality conference papers in IEEE/ACM AAMAS, ACM Ubicomp, ACM IDC, ACM CHI, ACM CSCW,, and so forth; book chapters published by Springer and CRC Press; and journal papers in Journal of Educational Computing Research, Expert Systems With Applications, New Generation Computing, and so forth. His early research on cross-domain and emotion-aware recommendation systems has been widely cited. He is currently the founding director of the Computer Science Playground and the Assistive Technology Research & Development Center at Wenzhou-Kean University.

Tiffany Y. Tang is an assistant professor in the Department of Computer Science at Wenzhou-Kean University. Her recent research focuses on affective computing, sensing and wearable technologies for children with autism spectrum disorder (ASD) and other special needs, human factors for smart recommendations, and artificial intelligence in education. She has previously served as the editor to special issues in Expert Systems: The Journal of Knowledge Engineering (Wiley) and Journal of Educational Computing Research (SAGE). She currently serves as a coeditor in the special issue on Leveraging Wearable and Sensing Technologies for Assistive Learning Environment in Special Education in Journal of Educational Computing Research. She was the cochair of the track for PACIS’2014 and PACIS’ 2016 on Human and Social Factors in Decision Support and Recommendation Systems. In May 2015, she cofounded the Autism Research Network in Wenzhou as the city’s first and one of China’s few dedicated research and development centers for individuals with autism. She has coauthored conference papers published in ACM Ubicomp, ACM IDC, and HCII and journal papers on technology-enabled intervention for individuals with autism.

References

Alghanim, F. L. (2013). Investigating the impact of co-located and distributed collaboration using multi-touch tables (Doctoral thesis). Durham University, England.

Ames

Fletcher-Watson

(2010) A review of methods in the study of attention in autism. Developmental Review 30(1): 52–73.

Baron-Cohen

Leslie

A. M.

Frith

(1985) Does the autistic-child have a theory of mind. Cognition 21(1): 37–46.

Battocchi, A., Pianesi, F., Tomasini, D., Zancanaro, M., Esposito, G., Venuti, P., … Weiss, P. L. (2009). Collaborative puzzle game: A tabletop interactive game for fostering collaboration in children with autism spectrum disorders (ASD). Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’09), 197–204.

Battocchi, A., Ben-Sasson, A., Esposito, G., Gal, E., Pianesi, F., Tomasini, D., … Zancanaro, M. (2010). Collaborative puzzle game: A tabletop interface for fostering collaborative skills in children with autism spectrum disorders. Journal of Assistive Technology, 4(1), 4–13. Retrieved from http://www.emeraldinsight.com/doi/abs/10.5042/jat.2010.0040.

Bauminger

Solomon

Aviezer

Heung

Brown

Rogers

S. J.

(2008) Friendship in high-functioning children with autism spectrum disorder: Mixed and non-mixed dyads. Journal of Autism and Developmental Disorders 38(7): 1211–1229.

Ben-Sasson

Lamash

Gal

(2013) To enforce or not to enforce? The use of collaborative interfaces to promote social skills in children with high functioning autism spectrum disorder. Autism 17(5): 608–622.

Boyle, B., Arnedillo-Sanchez, I., & Zahid, A. (2015). AutBlocks: Using collaborative learning to develop joint attention skills for children with autism spectrum disorder. Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference, 2, 529–532.

Bruder

M. B.

(2010) Early childhood intervention: A promise to children and families for their future. Exceptional Children 76(3): 339–355.

10.

Cage

Pellicano

Shah

Bird

(2013) Reputation management: Evidence for ability but reduced propensity in autism. Autism Research 6: 433–442.

11.

Chen, W. (2012). Multitouch tabletop technology for people with autism spectrum disorder: A review of the literature. Proceedings of the 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI’12), 14, 198–207.

12.

Chevallier

Kohls

Troiani

Brodkin

E. S.

Schultz

R. T.

(2012) The social motivation theory of autism. Trends in Cognitive Sciences 16(4): 231–239.

13.

Dawson

Bernier

Ring

R. H.

(2012) Social attention: A possible early indicator of efficacy in autism clinical trials. Journal of Neurodevelopmental Disorders 4: 11.

14.

Dietz, P., & Leigh, D. (2001). DiamondTouch: A multi-user touch technology. Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST’01), 219–226.

15.

Edward

Greenberg

Shen

Forlines

(2007) Multimodal multiplayer tabletop gaming. ACM Computers in Entertainment (CIE) 5(2): 1–12.

16.

El-Nasr, M. S., Aghabeigi, B., Milam, D., Erfani, M., Lameman, B., Maygoli, H., & Mah, S. (2010, April). Understanding and evaluating cooperative games. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’10) (pp. 253–262), Atlanta, GA. Retrieved from https://doi-acm-org.web.bisu.edu.cn/10.1145/1753326.1753363.

17.

Frauenberger, C., Good, J., & Keay-Bright, W. (2010). Phenomenology, a framework for participatory design. Proceedings of the 11th Biennial Participatory Design Conference (PDC ‘10), 187–190.

18.

Gal

Lamash

Bauminger-Zviely

Zancanaro

Weiss

P. L. T.

(2016) Using multitouch collaboration technology to enhance social interaction of children with high-functioning autism. Physical & Occupational Therapy in Pediatrics 36(1): 46–58.

19.

Giusti, L., Zancanaro, M., Gal, E., & Weiss, P.L.T. (2011). Dimensions of collaboration on a tabletop interface for children with autism spectrum disorder. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11), 3295–3304.

20.

Goh

W. B.

Shou

Tan

Lum

G. T. J.

(2012) Interaction design patterns for multi-touch tabletop collaborative games. CHI ‘12 Extended Abstracts on Human Factors in Computing Systems (CHI’12). 141–150.

21.

Green

Charman

McConachie

Aldred

Slonims

Howlin

PACT consortium (2010) Parent-mediated communication-focused treatment in children with autism (PACT): A randomized controlled trial. Lancet 375: 2152–2160.

22.

Gross, T., Fetter, M., & Liebsch, S. (2009, April). The cuetable: Cooperative and competitive multi-touch interaction on a tabletop. Paper presented at the CHI ‘08 Extended Abstracts on Human Factors in Computing Systems (CHI’09) (pp. 3465–3470), Florence, Italy. Retrieved from https://doi-acm-org.web.bisu.edu.cn/10.1145/1358628.1358875.

23.

Haller, M., Forlines, C., Koeffel, C., Leitner, J., & Shen, C. (2010). Tabletop games: Platforms, experimental games and design recommendations. In A. D. Cheok (Eds.), Art and technology of entertainment computing and communication (pp. 271–297). London, England: Springer-Verlag.

24.

Hart, K. J., & Morgan, J. R. (1993). Cognitive behavioral therapy with children: Historical context and current status. In A. J. Finch, W. M. Nelson, & E. S. Ott (Eds.), Cognitive behavior procedures with children and adolescents: A practical guide (pp. 1–24). Boston, MA: Allyn Bacon.

25.

Hourcade

J. P.

Bullock-Rest

N. E.

Hansen

T. E.

(2012) Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders. Personal and Ubiquitous Computing 16(2): 157–168.

26.

Izuma

Matsumoto

Camerer

C. F.

Adolphs

(2011) Insensitivity to social reputation in autism. PNAS 108(42): 17302–17307.

27.

Kasari

Siller

Huynh

L. N.

Shih

Swanson

Hellemann

G. S.

Sugara

C. A.

(2014) Randomized controlled trial of parental responsiveness intervention for toddlers at high risk for autism. Infant Behavior and Development 37(4): 711–721.

28.

Kensing

Blomberg

(1998) Participatory design: Issues and concerns. Computer Supported Cooperative Work 7: 167–185.

29.

Khaled, R., Barr, P., Johnston, H., & Biddle, R. (2009). Let’s clean up this mess: Exploring multi-touch collaborative play. Extended Abstracts on Human Factors in Computing Systems (CHI’09), 4441–4446.

30.

Kientz, J. A., Goodwin, M. S., Hayes, G. R., & Abowd, G. D. (2013). Interactive technologies for autism. Synthesis lectures on assistive, rehabilitative, and health-preserving technologies. Retrieved from http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6813124.

31.

Lai

M.-C.

Lombardo

M. V.

Baron-Cohen

(2013) Autism. The Lancet 383(9920): 896–910.

32.

Millen, L., Hawkins, T., Cobb, S., Zancanaro, M., Glover, T., Weiss, P. L., & Gal, E. (2011). Collaborative technologies for children with autism. Proceedings of the 10th International Conference on Interaction Design and Children (IDC’11), 246–249.

33.

Morales

Mundy

Crowson

M. M.

Neal

A. R.

Delgado

(2005) Individual differences in infant attention skills, joint attention, and emotion regulation behavior. International Journal of Behavioral Development 29: 259–263.

34.

Morris, M. R., Huang, A., Paepcke, A., & Winograd, T. (2006). Cooperative gestures: Multi-user gestural interactions for co-located groupware. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’2006), 1201–1210.

35.

Mundy

(1995) Joint attention and social-emotional approach behavior in children with autism. Development and Psychopathology, special issue on Emotions in Developmental Psychopathology 7(1): 63–82.

36.

Odom, S. L., Schertz, H. H., & Baggett, K. B. (2011). Single case design and growth curve analysis of an intervention to promote joint attention for toddlers with autism spectrum disorders. Paper presented at the Proceedings of the International Meeting of Autism Research, San Diego, CA.

37.

Paparella

Freeman

S. F. N.

(2015) Methods to improve joint attention in young children with autism: A review. Pediatric Health, Medicine and Therapeutics 6: 65–78.

38.

Piper, A. M., O’Brien, E., Morris, M. R., & Winograd, T. (2006). SIDES: A cooperative tabletop computer game for social skills development. Proceedings of the 2006 20th conference on Computer Supported Cooperative Work (CSCW’06), 1–10.

39.

Rehg, J. M., Abowd, G. D., Rozga, A., Romero, M., Clements, M., Sclaroff, S., … Ye, Z. (2013). Decoding children’s social behavior. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’13), 3414–3421.

40.

Schertz

H. H.

Odom

S. L.

Baggett

K. M.

Sideris

J. H.

(2013) Effects of joint attention mediated learning for toddlers with autism spectrum disorders: An initial randomized controlled study. Early Childhood Research Quarterly 28(2): 249–258.

41.

Silva

G. F.

Raposo

Suplino

(2015) Exploring collaboration patterns in a multitouch game to encourage social interaction and collaboration among users with autism spectrum disorder. Journal of Computer Supported Cooperative Work 24: 149–175.

42.

Simonsen

Kensing

(1997) Using ethnography in contextual design. Communication of the ACM 40(7): 82–88.

43.

Tang, T. Y., Wang, R. Y., You, Y., Huang, L. Z, & Chen, C. P. (2015). Supporting collaborative play via an affordable touching + singing plant for children with autism in China. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing/the International Symposium on Wearable Computers (ACM UbiComp/ISWC ‘15), 373–376.

44.

Tentori, M., & Hayes, G. (2010). Designing for interaction immediacy to enhance social skills of children with autism. Proceedings of the 12th ACM International Conference on Ubiquitous Computing (ACM Ubicomp’2010), 51–60.

45.

Tomasello

(1995) Joint attention as social cognition. In: Moore

Dunham

P. J.

(eds) Joint attention: Its origin and role in development, Cambridge, England: Cambridge University Press, pp. 103–130.

46.

Whalen

Schreibman

(2003) Joint attention training for children with autism using behavior modification procedures. Journal of Child Psychology and Psychiatry 44(3): 456–468.

47.

Wing, L. (1988). The continuum of autistic characteristics. In E. Schopler & G. B. Mesibov (Eds.), Diagnosis and assessment in autism. Current issues in autism (pp. 91–110). New York, NY: Springer US.

48.

Wong

Kasari

(2012) Play and joint attention of children with autism in the preschool special education classroom. Journal of Autism and Developmental Disorders 42(10): 2152–2161.

49.

Yuill

Rogers

(2012) Mechanisms for collaboration: A design and evaluation framework for multi-user interfaces. ACM Transactions on Computer-Human Interaction (TOCHI) 19(1): 1.