A basic tactile language to support leader-follower dancing

Abstract

Work is needed to advance the current understanding of tactile interaction among humans through haptic technologies. We introduce a novel language that has been designed to guide users in leader-follower dances. This language is based on a nine-word vocabulary that corresponds to nine dance steps, following the metaphor of a leader-follower dance. Our work benefits from a haptic coding that is commonly used by couples when dancing, and explores the potential of wearable technology in this scenario. A wearable prototype consisting of four vibrotactile actuators was used to test the idea. Two user studies show a high recognition rate (more than 90%) of the intended tactile vocabulary. This particular work highlights the feasibility of a haptic vocabulary to exchange full, understandable commands between users, the importance of dance as a case study, and the potential of using wearable technology to support haptic communications in scenarios similar to those in the real world, such as partner dancing. Current results show it is viable to successfully guide someone to follow dance through communication using a basic vibrotactile language.

Keywords

Wearable technology haptic language leader-follower dancing natural user interfaces

1 Introduction

Tactile devices and wearable technologies both result from and have prompted research in novel forms of communications. A key concept in communication is the language involved in the haptic interaction occurring between two humans or between human and technology. This concept is commonly known as tactile or vibrotactile language [1 –3] and gives formality to what humans experience throughout their sense of touch, how they identify and respond to it.

Wearable technology consists of various components, which include sensors, such as accelerometers, and also actuators, like vibrating motors. For this technology to be considered wearable, it needs to be as non-obtrusive as possible, meaning humans are not constantly aware of its use, as well as non-disruptive, as daily activities should be performed naturally. People interact with wearable technology in novel ways, as the classic user meets computer interface no longer applies. In this scenario, one of the most common interactions occurs through vibrotactile actuators. Vibrotactile is the term used for technology that uses vibrations that humans perceive using their cutaneous sense of touch. Investigating the potential of tactile languages and their possible implementations through wearable technologies is a particular interest of this work. As an initial application domain, communication in dance has been selected.

Out of the many different dance styles that exist, this research focuses on leader-follower dance, also referred to as couple's dance or partner dance. The main reason is the communication process that takes place in a couple’s dance between two human partners involved, since it is purely haptic. Haptic is the term used to refer to the human sense of touch. In a partner dance, the person in charge of leading sends step instructions to the person that takes the role of a follower. However, the leader does not talk to the follower, instead they mostly use haptic cues and haptic feedback [4]. This particular interaction between humans is possible because both agents share the same predefined vocabulary of movements [5]. One of the agents, a leader, emits a signal of the move or combination of moves the follower must perform. The follower agent offers certain resistance that communicates how the received signal was understood and how the movements will adapt to his physical characteristics [6]. So, any human participating in a couple’s dance is actually interacting with their partner using a successful and well proven haptic or, more specifically, tactile language. Thus, a fundamental question is why haptic interaction between human and technology does not use the already proven and functional communication that occurs in a human leader-follower dance? Dance involves a lot of movement and fluidity from humans, meaning that if any technology will be used to benefit from this activity, it should not interfere with human mobility. This is where wearable technology comes in handy, and could help people learn how to dance by themselves, using technology to guide them.

With all the previous considerations in mind, this work is an initial exploration of how these important concepts (vibrotactile wearable technology and tactile languages) can benefit from one another when applied in the context of communication in dance. Thus, a basic tactile language to be used for communication through wearable technology in leader-follower dance is proposed.

The main contribution is a mechanism to indicate humans dance steps to execute, using only wearable technology as a communication tool. That is, a mean to provide support the leader’s role in a couple’s dance with technology, achieving dancing through a synthetic tactile language using the wearable technology as a mediator. This requires the codification of dance moves using vibrations, so the wearable technology can give cues to a human. The human will feel the instruction, decode it and perform the specified dance step, engaging in communication process with the use of wearable technology. An application was prototyped that implements the proposed language and is used to evaluate its potential by applying user studies in an iterative manner.

The remainder of this paper is organized as follows. Section 2 covers related work in detail, Section 3 presents conceptual design, Section 4 the implementation, Section 5 describes two studies with their results, and finally, Sections 6 and 7 provide discussion and conclusion.

2 Related work

Two major areas can be identified as relevant to this work: tactile languages and technology in dance. While both are very broad areas, this paper is focused on more specific sub-areas of each one.

2.1 Tactile languages

Some examples of tactile language theory can be found in the literature [1 –3], other applications range from situational awareness to navigation [7 –9] and even to enhance communication [10, 11]. Dance applications focusing on the intrinsic tactile language have not been found. Recently, a common practice for conveying messages in environments where visual interaction is not available, includes the application of tactons or tactile icons [2]. A specific message can be transmitted by different signals, for example, tactile pulses with particular frequency, strength and duration. Therefore, a basic structure for developing tactile language can be tactons that should improve non-visual interaction, particularly, support interface concepts, objects and actions through well-structured signal messages with specific rhythm and location [1 –3].

A useful methodology for designing a tactile language has been proposed in an application for squad level soldier communication, where particular combination and manipulation by tactons’ parameters is used for conveying meaning in real world contexts [3]. A good example of developing understandable tactile language has been proposed in an application for assisting visually-impaired people in situational awareness [7]. According to three main concepts introduced by them: tactile learning, tactile language and tactile memory, it was possible to define the process of learning any type of language and recognize several words coded with haptic stimuli, as well as to progress to even more complex sentences.

The assessment of tactile languages involving navigation in 3D environments is discussed by the work presented in [9], applying prefixes in tactons that enhance the level of expressiveness of language, while keeping it simple to learn. User studies based on three different tactile vocabularies (with and without prefixation), evidence the correlation between number of used vibrotactile parameters, tactile language complexity and amount of messages to be transmitted [12].

Some applications do not focus particularly on vibrotactile languages, but still use them to convey information for visually impaired people. The first one is reported in [13], where the main goal is to aid the visually impaired in unknown locations, developing a navigation system that is cheap and low power consuming, although only sighted users participated in the validation. In contrast, in [14], the guidance was performed by human operator, that sent vibrotactile cues to visually impaired participant through two bracelets. The operator also used camera glasses to obtain feedback about the participant environment. In addition to navigation, authors in [15] propose to use haptic feedback to further complement auditory instructions to a visually impaired student.

Non-wearable technology, or grounded technology, is also widely used; for example, [16] focused on a vibrotactile array mounted on the back side of a chair facilitating directional attention to sighted users by haptic feedback. As can be seen, there are several works concerning tactile communication, but research on tactile communication based on synthetic tactile languages, specifically for dance, is still needed. Related works on tactile languages is summarized, extended, and organized by application in Table 1.

Table 1
Related works: Comparison of tactile languages and their features

Technology Social Touch Environment Applications

Wearable Grounded Simulated Mediated Virtual Real

Haritaipan [17] • • • • Emotions - Understand

Morrison, [18] • • • • • Emotions - Emulate

Velázquez, [7] • • • Actions-Assistance

Oliveira, [9] • • • Actions-Simulation

Oliveira, [12] • • • Actions-Simulation

Riddle, [2] • • • Actions-Simulation

Spelmezan, [19] • • • • Actions-Entertainment

Enriquez, [20] • • • Communication-Word

Brewster, [2] • • • • Communication-Word

Schelle, [11] • • • Communication-Message

MacLean, [1] • • • • Communication-Display

	Technology	Social Touch	Environment	Applications
Haritaipan [17]	•	•		•		•	Emotions - Understand
Morrison, [18]	•	•	•	•		•	Emotions - Emulate
Velázquez, [7]	•		•			•	Actions-Assistance
Oliveira, [9]	•		•			•	Actions-Simulation
Oliveira, [12]		•	•		•		Actions-Simulation
Riddle, [2]	•		•			•	Actions-Simulation
Spelmezan, [19]	•	•	•		•		Actions-Entertainment
Enriquez, [20]		•	•			•	Communication-Word
Brewster, [2]	•	•	•			•	Communication-Word
Schelle, [11]		•		•		•	Communication-Message
MacLean, [1]		•	•	•		•	Communication-Display

2.2 Technology in dance

There are several well-known dance applications such as dancing with robots [21], leader-follower interactions with a grounded haptic device [5, 6] and supporting dancers to improve their technique [22 –24]. There are also useful systems that help deaf, blind, and deaf-blind people to experience a dance presentation [25 –27] as well as others being mainly recreational and artistic are also found in literature [28 –32]. From all of these applications, only few are based on wearable technology. In both [4, 5], a Phantom haptic device is used to guide a human arm’s motion imitating a leader-follower dance. Another project with the same objective of having a haptic robot lead a human in dance is described by Hölldampf et al, where the mobile haptic interface was developed and used with a head-mounted display for the user to be guided using virtual reality [6].

The work introduced by Alizadeh et al. supports learning and shared dance space using the VICON tracking system for shoes [29]. Authors of that work chose line dancing, when a dance does not involve touching the dance partner because dancers are remotely located. Another interesting system that gives dancers some control over music by receiving musical feedback from both dancers’ movements is reported by Brown and Paine [30]. They present an interactive system with real-time analysis that includes sonification of social tango providing new expressions of tango tradition by immersion and feeling as one with the dance partner. Other work focused only on dance presentation is discussed by Tsuchida, where dance with multiple robots of the spherical shape (Sphero2.0) is provided with better performance in entertainment and sports activities [28]. Another example of dance presentation can be found in [31], where the main goal was to give presentation connecting music and dancers all around the world. Authors used Ultragrid for video transmission as well as two dancers were recruited for this performance with final proposal to place all participants on a single screen. It can be seen that there is a good opportunity to study non-verbal communication using leader-follower dances, since it is indeed what happens when dancing with a couple. Table 2 summarizes and extends the related works using technology in dance.

Table 2
Related works: Technology in dance

Technology Medium Applications

Robots Visual Wearable Presentation System

Giomi, [33] • • • Generative-dance moves

Chiang, [34] • • Generative-dance moves

Danjoux, [32] • • • Generative-choreography

Molina, [22] • • Improvement-assistive feedback

Yan, [23] • • • Improvement-assistive feedback

Trajkova, [35] • • Improvement-learning

Ros, [36] • • Improvement-learning

Florian, [25] • • Inclusion-hearing impaired

Shibasaki, [26] • Inclusion-hearing impaired

Hossny, [27] • • • • Inclusion-hearing impaired

McLean, [37] • • • • Inclusion-genderless

Bryant, [38] • • Inclusion-rural minorities

Alizadeh, [29] • • Recreational-remote dancing

Tsuchida, [28] • • Recreational-enhance experience

Park, [31] • • Recreational-enhance experience

Brown, [30] • • Recreational-enhance experience

	Technology	Medium	Applications
Giomi, [33]		•	•		•	Generative-dance moves
Chiang, [34]		•			•	Generative-dance moves
Danjoux, [32]		•	•	•		Generative-choreography
Molina, [22]		•			•	Improvement-assistive feedback
Yan, [23]		•	•		•	Improvement-assistive feedback
Trajkova, [35]		•			•	Improvement-learning
Ros, [36]	•				•	Improvement-learning
Florian, [25]			•		•	Inclusion-hearing impaired
Shibasaki, [26]				•		Inclusion-hearing impaired
Hossny, [27]	•	•	•	•		Inclusion-hearing impaired
McLean, [37]	•		•	•	•	Inclusion-genderless
Bryant, [38]	•				•	Inclusion-rural minorities
Alizadeh, [29]		•			•	Recreational-remote dancing
Tsuchida, [28]	•			•		Recreational-enhance experience
Park, [31]		•		•		Recreational-enhance experience
Brown, [30]			•		•	Recreational-enhance experience

Based on previously described related works, the following opportunities have been identified. Couple’s dancing involves a leader and a follower requiring communication between partners that mainly uses touch sense. However, work was not found that focuses on how to use synthetic tactile language to provide guided leader-follower dance. Currently available technology used in dance mainly focuses on enhancing the experience, either for dancers, for audience or both. On a very different research direction there exist several works that study communication properties of haptic stimuli. Their applications are quite wide covering motion activities such as sport and navigation. The main opportunity of this work consists in exploring how synthetic haptic communication can benefit from interaction occurring when two people dance together. The main questions to be addressed are: can users understand dance commands via vibrotactile stimuli? Is wearable technology suitable to communicate dance commands? Is this communication indeed similar to leader-follower dance? Can basic vocabulary be regarded as foundation to develop tactile language to support motion activities? To answer these questions, the following concept was conceived.

3 Conceptual design

This work proposes a basic language conveyed by a wearable device that mediates leader-follower dance between human agent (the follower) and wearable technology (the leader). In leader-follower dances the user that takes a role of the follower receives a cue from leader to perform specific move from a known vocabulary of movements. In real scenario (regular setup), these cues are haptic feedbacks sent by body movements of the leader and in turn are sensed by body of the follower, who proceeds to interpret cues and perform movement in response.

In the proposed setup, the couple of the leader-follower dance is no longer human-human, and thus no longer can send cues with their body language. Instead, a wearable vibrotactile device is used to convey tactile words to the follower, keeping the haptic nature. This wearable device sends vibrotactile signals at strategic parts of the body that are commonly the ones in contact in regular leader-follower dance.

The follower is expected to identify different cues depending on which parts of a body the vibrotactile signal has been received. It is intended that these haptic patterns are as similar as possible to the actual body cues received in the regular setup, and to be as intuitive as possible to the follower. To limit scope of a work, a vocabulary consisting only of nine moves were defined. They are: right foot forward, right foot backward, left foot forward, right foot backward, sideways left, sideways right, complete left turn, complete right turn and close.

Six moves consisted in a single step from only one foot in one of three directions: front, back or sideways. For these moves, user was expected to move only the specified foot in direction indicated along with their bodyweight. The close movement consists also of a single step from only one foot, but in this case to place both feet close together to balance their bodyweight, closing where it currently is.

The two final moves were turn around to each side: right turn and left turn. For these turns user was expected to make full turn, meaning they should end facing the same direction where they started. These nine moves need to be coded in vibrotactile language understandable to humans, described in detail in the test section. To prove the concept, two iterative studies were made using a prototype described in the next section.

4 Implementation

The wearable device used for validation of the language was a prototype [39] that consists of four components, each equipped with Inertial Measurement Unit (IMU sensors) and vibrotactile actuators presented in Fig. 1. In this experiment, only actuators were used. The components were placed near the shoulder area: two in the front near the clavicle, and two in the back. The components were attached to the body of user with the help of suspenders and belt. The actuators can vibrate with different intensities for a specified period of time.

Fig.1

a) MACHINA wearable prototype (actuators); b) device placement on the back; c) device placement on the front.

In this setup the maximum vibration intensity was used as well as the 0.5s duration for each vibrotactile feedback applied. Each of several encoded cues is sent to the actuators via a graphical user interface (GUI) in Unity, which is presented in Fig. 2.

Fig.2

Designed GUI for control of actuators in dance.

Each button represents a different cue, two sliders control the intensity and duration of each cue. The red and green buttons will be specified in detail later. Using the installed hardware and designed framework two following studies have been carried out.

5 Experimentation and results

The two studies as part of the same iterative process were carried out in a lab setting, where a 1.8m by 3.0m rectangular space shown in Fig. 3a) was provided so that participants could move freely. The first study (study 1) was meant to evaluate the intuitiveness of the proposed tactile communication system and the second one (study 2) implemented all the improvements found in study 1.

Fig.3

a) Study space used in tests; b) Layout of the study.

In the study ambient a facilitator was responsible for placing the device on each participant, explaining everything from the main idea of the study to the message coding; also, adjusting the device if necessary or answering on-the-go questions of participants. An operator was responsible of sending the cues via the Unity interface, thus producing a kind of Wizard of Oz environment. The general layout of the study is provided in Fig. 3b).

5.1 Study 1

The methodology adopted for study 1 was to define vibration patterns that represent each of the nine movements according to the defined vocabulary presented in Table 3. The pairing of movement and actuator was designed based on the metaphor that vibration represents a gentle push (e.g. vibration on the front means to go backwards). Then, a three-phase experiment was performed with each participant. Seven participants with ages ranging from 20–30 years old were selected for this study, particularly, five women and two men, everyone with previous experience in leader-follower dances.

Table 3
Vocabulary of movements and message coding

Movement Actuators

Right Step Forward Right-Back

Left Step Forward Left-Back

Right Step Backward Right-Front

Left Step Backward Left-Front

Right Sideways Both Left

Left Sideways Both Right

Right Turn Right- Front &Left-Back

Left Turn Left-Front &Right-Back

Close All together

Movement	Actuators
Right Step Forward	Right-Back
Left Step Forward	Left-Back
Right Step Backward	Right-Front
Left Step Backward	Left-Front
Right Sideways	Both Left
Left Sideways	Both Right
Right Turn	Right- Front &Left-Back
Left Turn	Left-Front &Right-Back
Close	All together

Their demographics are summarized in Table 4. To all participants were handed written agreement to be signed and a post-test questionnaire to be filled after completing study.

Table 4

Demographics of participants in Study 1

Demographics
Participants	Age	Gender	Dance Lessons	Role	Expertise	Wearable usage
P1	20	F	Salsa, Danzon, Bachata, Mexican Folk, Ballet	Both	Intermediate	None
P2	21	F	Salsa, Bachata, Tango	Follower	Intermediate	None
P3	20	F	Salsa, Danzon, Bachata, Tango, Cumbia	Both	Intermediate	None
P4	20	F	Salsa, Danzon, Mexican Folk	Both	High	None
P5	30	F	Tango	Both	High	None
P6	20	M	Salsa, Danzon, Mexican Folk	Leader	Intermediate	Fitness Tracker
P7	21	M	Salsa, Danzon, Bachata, Tango, Mexican Folk	Leader	High	None

The phases of the study were:

Phase1: Natural Response. All nine expected movements of the vocabulary were described to the participants, without explaining which actuators represent each movement. Participants were told to respond to received signal with one of these moves, the one that they felt most natural according to the vibration received. The vibrotactile signals were sent twice for each participant in a random order.

Phase 2: Code Response. This phase is similar to the previous one in the sense that the signals were once more sent twice in a random order. The difference is that now the full vocabulary was explained to the participants, meaning they now knew which movement is expected from each vibrotactile cue.

Phase 3: Sequence Response. This phase was designed to assess the understanding of the basic tactile language, applying it to two simple choreographies. The first one was inspired by popular line dance in Mexico, chosen because of its popularity in parties and weddings, and also because it is repetitive and simple. The second one was designed to include the basic steps of salsa and tango. Compared to the previous sequence, this one is a bit more difficult because it is less repetitive. A rhythmic Cuban beat [40] was played during this phase only to induce the right mood to participants however, they were told that following the rhythm of the beat was not mandatory in order to avoid extra complications to the participants. Each sequence was repeated twice.

From the Natural Response phase each of the final outcome of the step performed of every participant was kept using the following notation. Good: participant responded following the vocabulary correctly, according to push metaphor. Inverted: participant understood the expected movement, but misunderstood direction (e.g. participant performed right step forward instead of right step backward), meaning the vibration is being interpreted as a gentle pull (instead of the gentle push). Wrong: Participant performed a movement that does not correspond to the vibration sent, not even considering a pull metaphor. From the Code Response phase participants responses were kept with the same notation described previously. Results are presented in Table 5. Because each of the nine movement was sent twice, the total number of correct movements are eighteen for each participant in each phase. The accuracy column represents the percentage of correct movements performed, meaning the participant understood the vibration sent.

Table 5

Natural response and message coding results

Participants	Natural Response				Code Response
	Good	Inverted	Wrong	Accuracy	Good	Inverted	Wrong	Accuracy
P1	11	3	4	61%	15	0	3	83%
P2	2	4	12	11%	8	3	7	44%
P3	0	5	13	0%	10	3	5	56%
P4	1	9	8	6%	6	4	8	33%
P5	2	6	10	11%	15	2	1	83%
P6	3	11	4	17%	14	2	2	78%
P7	0	7	11	0%	14	0	4	78%

Analyzing the results of the Natural Response phase, only one participant had more good than inverted, while everyone else had more inverted than good. This implies that the push metaphor is not as natural to users as the hypothesis suggested, instead users feel the pull metaphor as more natural. This finding will be the foundation for Study 2. Four out of the seven participants had more wrong than good and inverted together. This implies that the proposed vocabulary is not as intuitive as it is, even though it was proposed by two of the authors who are regular couple dancers. The wrong outcomes varied significantly from participant to participant, including movements not considered in the vocabulary and misunderstanding of what the close step meant. The accuracy ranged from 0% to 61%, average being 15%. All of these findings suggest a more structured training is needed for participants to understand the proposed basic tactile language correctly, and was also considered for Study 2.

In the Code Response phase, once the vocabulary was explained, there was a significant improvement in the accuracy rate of all participants, ranging from 33% to 83%, average being 65%. This is a major improvement from the 15% obtained in the previous phase, although it is far from being acceptable in a real life application. Also, inverted movements were still observed, even after the push metaphor was explained to the participants. Most of them justified themselves by saying it was confusing following a push metaphor. In various cases, the wrong movements were misinterpreted closes, returning the bodyweight instead of going to it.

In the Sequence Response phase participants had a downgrade in their performance, compared to the previous phase. These results are summarized in Table 6, were accuracy ranged from 16% to 84% , average being 46%. These results were very evident when none of the participants succeeded in performing any of the two choreographies, or recognizing any of the steps they were doing. In fact, everyone looked confused as to what they were doing. Thus, the communication system is less effective when used in situ, as in a real activity and not only recognizing the meaning of the vibrotactile cues.

Table 6

Results of Sequence Response phase

Participants	First Sequence				Second Sequence
	Good	Inverted	Wrong	Accuracy	Good	Inverted	Wrong	Accuracy
P1	19	3	10	59%	19	1	8	59%
P2	10	11	11	31%	5	2	21	16%
P3	14	12	6	44%	15	8	5	47%
P4	5	7	20	16%	7	7	14	22%
P5	27	2	3	84%	18	5	5	56%
P6	16	14	2	50%	20	2	6	63%
P7	12	0	20	38%	18	1	9	56%

In general, it was observed that all participants performed each move one or two seconds after each signal was sent. Some of them said that they needed to be focused to identify which actuator had been activated, and also to remember which of the nine movements it meant. Even after explaining the vocabulary, participants inverted the movements, and only afterwards they remembered the coding used and corrected themselves. The most challenging movements for the participants were the turns, sometimes performing them instead of close, or inverting the direction. Also, the close step presented many difficulties to the participants, such as inverting it or performing a turn (the same problems as the turns). A possible explanation is that identifying two actuators instead of only one confused the participants. A final remark of the observations was the influence of the music, participants seemed to enjoy it, swaying their hip and their arms in sync with the beat. If it caused a distraction or motivation is uncertain.

At the end of the three phases, participants filled out a post-test questionnaire, which among other questions included the System Usability Scale (SUS). A total mean score of 69.29 was obtained which is above average and means the proposed approach has good potential of acceptance. Regarding a question of the placement of the actuators, some participants agreed that when the back and front actuators of the same side activated, it was difficult to tell if one or both were vibrating, and thus they suggested an alternative position may come in handy. A good result regarding the activity was that all participants agreed it was highly entertaining, and very similar to dancing. For example, a comment of one of the participants stated: “Could be an entertaining videogame similar to Guitar Hero, but different to Just Dance because there are vibrations in the body”. Based on all these findings Study 2 was developed to improve the proposed basic tactile language.

5.2 Study 2

The same hardware, software and layout from Study 1, were used. Significant changes were made in the methodology, inspired by the findings of the previous study, such as changing the first phase from natural response to vocabulary training. The purpose of this change is to improve the results by ensuring participants are successfully trained in understanding the language. There are minor changes in every phase, mainly the addition of auditory feedback for good/wrong performed moves. The core difference is the message coding, now a pull metaphor is being considered. The new vocabulary considering the new message coding is represented in Table 7.

Table 7
New message coding for Study 2

Movement Actuators

Right Step Forward Right-Front

Left Step Forward Left-Front

Right Step Backward Right-Back

Left Step Backward Left-Back

Right Sideways Both Right

Left Sideways Both Left

Right Turn Left-Front &Right-Back

Left Turn Right-Front &Left-Back

Close All together

Movement	Actuators
Right Step Forward	Right-Front
Left Step Forward	Left-Front
Right Step Backward	Right-Back
Left Step Backward	Left-Back
Right Sideways	Both Right
Left Sideways	Both Left
Right Turn	Left-Front &Right-Back
Left Turn	Right-Front &Left-Back
Close	All together

Eleven participants of ages ranging from 20–29 years old were selected for this study, particularly, six women and five men, each with previous experience of leader-follower dance. Three of these participants also participated in Study 1, in order to evaluate if they considered there was a significant improvement between studies. Only one of them had previous experience with wearable technology. Only three of the male participants had experience as leaders. All the demographic info is summarized in Table 8.

Table 8

Demographics of participants in Study 2

Demographics
Participants	Age	Gender	Dance Lessons	Role	Expertise	Wearable usage
P1	22	F	Salsa, Bachata	Both	High	None
P2	20	F	Salsa, Danzon, Bachata	Both	High	None
P3	20	F	Salsa, Danzon, Bachata	Follower	Intermediate	None
P4	20	M	Danzon, Mexican Folk	Leader	High	Fitness Tracker
P5	20	M	Salsa, Danzon, Bachata, Tango, Mexican Folk	Both	Intermediate	None
P6	20	M	Salsa, Bachata, Tango	Both	Intermediate	None
P7	26	M	Salsa, Bachata, Mexican Folk	Leader	Intermediate	None
P8	29	F	Salsa, Danzon, Bachata, Tango	Both	High	None
P9	22	F	Salsa, Bachata, Tango	Both	High	None
P10	22	F	Salsa, Bachata	Follower	High	None
P11	21	M	Salsa, Danzon, Bachata, Tango, Mexican Folk	Leader	High	None

Phase 1: Vocabulary Training. This phase was added to ensure the correct understanding of moves that were expected from each participant. Wearable technology was not used, but instead the operator gave verbal instructions of the expected movement to the participant. Direct feedback from the operator was given to the participant on whether the performed movement was correct or incorrect. In the case it was incorrect, the facilitator made the correct movement together with the participant. As stated previously, gaming characteristics were also added to improve learning and motivation. This was achieved by adding two sounds as feedback, and also keeping the participants total score of correct movements. An encouraging chime-like sound was played by the operator when the participant understood the instruction correctly, thus adding a point to their score. If the participant interpreted the instruction performing the wrong movement, a negative sound was played as feedback. The final significant change was pre-defining the sequence of movements, not making it random. This change corrected the problem of a random order occasionally placing two steps together that do not make much sense (e.g. performing right foot forward twice).

Phase 2: Message Coding Training. This phase’s purpose is to continue the training for the participants. Now that they know which dance moves are expected from them and how to perform them, the new message coding based on the pull metaphor was explained to them. In this phase, the participants wore the wearable technology and felt the vibrations corresponding to each move. Again, a different predefined sequence of each step repeated twice was given to them via the actuators, both the auditory and facilitator feedback were kept throughout this phase.

Phase 3: Sequence Test. In order to compare the two studies, the same two choreographies from Study 1 were used with the same Cuban rhythmic beat [42]. The only difference was the auditory feedback. Since each sequence was performed twice, one was with audio feedback, and the other without feedback. In both cases, now the facilitator did not made any comment or corrections to the moves. At the end, participants were given their scores.

The results of every phase of Study 2 are presented in Table 9. Each value corresponds to the percentage of total correct interpretations. Overall, every participant had an excellent performance, with few mistakes. A peculiarity observed is that most participants got better results when no sound feedback was given to them. One possible reason would be that they had previously done the sequence with sound feedback, and knew if they were mistaken the first time. Another explanation could be there were no distractors and they stayed more focused on the task. Either way, further research is needed in order to reach significant conclusions from this setup.

Table 9

Performance of participants in Study 2

Participants	Training		Phase 3
	Phase 1	Phase 2	Sequence 1		Sequence 2
			Sound	No Sound	Sound	No Sound
P1	94%	100%	100%	94%	100%	100%
P2	100%	94%	94%	94%	93%	100%
P3	100%	89%	100%	100%	100%	100%
P4	100%	100%	100%	100%	100%	100%
P5	100%	100%	100%	100%	86%	100%
P6	94%	72%	63%	100%	79%	79%
P7	94%	94%	81%	75%	93%	100%
P8	100%	94%	94%	94%	86%	93%
P9	100%	94%	100%	100%	93%	100%
P10	100%	89%	75%	75%	86%	71%
P11	94%	89%	69%	100%	93%	100%

From both experiments it can be concluded that using a pull metaphor is more intuitive than the push variation. Whether this is a general rule or it only applies to people with experience in leader-follower dances is still uncertain. Further proof can be found on a comment made by a participant who was in both studies: “Actually, the vibration is better than last time (Study 1). It is easier to dance with the feeling of being pulled than pushed”. As final evidence, a direct comparison of correct outcomes from the sequences of phase 3 in both studies is presented in Table 10.

Table 10

Comparative performance of participants in Study 1 and Study 2

Mean	Value		Percentage
	Seq. 1	Seq. 2	Seq. 1	Seq. 2
Study 1	14.71	14.57	46%	52%
Study 2	29.18	26.09	91%	93%
Improvement	14.47	11.52	45%	41%

A significant improvement of +45% and +41% was observed from Study 1 to Study 2 for each sequence. A possible explanation of the success could be the pull metaphor, the new proposed training method to learn the basic language, and the gamification. From the post-test results, every participant perceived the vibration of the technology as comfortable, an important issue to be considered when wearing a vibrotactile actuator. As in Study 1 both right and left turns were identified as the most confusing steps to perform. This could occur because two actuators are being used instead of only one, although future research is also needed to find out whether a difficulty lies in the movement or the coding.

Another result of the post-test consistent with Study 1 was that the study was considered very entertaining for all participants. Also some of them commented directly to the facilitator and operator that the first and second phases (the training ones) could be helpful for people that are beginners learning how to dance. This also opens up a new research direction considering this basic tactile language as an assistive tool in teaching leader-follower dances.

A non-conclusive question was related to the amount of movements in the vocabulary, since answers varied from participant to participant, therefore suggesting it is dependent on each participant. One participant gave an interesting suggestion for the proposed language: “it helps working on capacity of motricity, it could be useful for children that are candidates to high performance training”. This possibility could also be further explored. Another participant (who teaches tango lessons) commented: “It could also be used to create reasoning in dancers as warmup exercise since it requires you to think and respond quickly to a stimulus instead of imitating the teacher”.

This also suggests that this basic language can be used to improve the abilities of dance students who are not necessarily beginners. SUS was again included in the post-test, yielding average total of 77.95 points, which is higher than the value obtained in the previous study, thus improving its potential acceptance. Overall results for the post-test questionnaire are presented in Fig. 4. A five-point Likert scale was used for each of the questions, and since there were eleven participants the maximum score possible was 55 points.

Fig.4

Results from the post-test describing acceptance of studies by participants.

6 Discussion

Comparing the obtained results to the reported accuracy rates (of comprehension of the language) of some works discussed in the related work section, in [7] they report four experiments with average recognition rates ranging from 70%–90%, 65%–90%, 70%–85% and 55%–75% respectively.

In [20] their reported accuracy rates on average are 76% in reinforced learning stage, and 77% on the test stage. Moreover, they state that a minimum average performance of 80% is considered conservative in a real application. Finally, in [19] they divided their test in stationary and active, the first regarding only recognition, and the other involved doing the action. Their average recognition rates are over 90% in stationary and over 75% in active mode. Therefore, the obtained results are promising in the field.

The placement of technology in the shoulder and back area with suspenders and belt gave a positive outcome because participants differentiated vibrations better if the actuators were placed as far from each other as possible. A crucial aspect is that vibrotactile cues are easily and quickly understood by the users, because in a real scenario, users need to be in time with the music, which will not be achieved if a lot of time is being used to understand which dance step to perform.

This proposed vocabulary helps to understand natural responses to haptic cues for movement, and could potentially benefit other applications described previously, such as helping visually-impaired people navigate. Regarding the proposed language, even though the set of nine cues reported may not be complex enough to be referred to as a full language, our work shows it is a well-structured basic set of words that were successfully learned and understood by the participants.

7 Conclusion and future work

This work has proven that it is possible to be guided in a leader-follower dance using a basic vocabulary conveyed through haptic feedback from a wearable device. Guiding is possible because a defined vocabulary of movements can be communicated through the interpretation of haptic coded messages, through a shared synthetic language. Participants adapted rapidly to the use of this communication system (only one session was enough to get good results) because the message coding was designed to be intuitive and as similar as possible to a real leader-follower dance setup. Also training was performed using game-like characteristics adding motivation.

This novel way of dancing seems to be appealing to dancers especially, because it is a new way to experience an activity they enjoy. Many were interested in the progress of the work and asked us to contact them to further experience with the proposed system.

This work is also relevant compared with reported average recognition rates reported in literature for other activities using vibrotactile cues as means of haptic communication using a predefined language.

Future work shall address to formalize the proposed synthetic tactile language, as to further prove whether or not it can be considered a language. Also, other user experiments will be carried out to discover to what extent participants can learn a tactile vocabulary related to dance moves, and successfully apply them to a specific dance. Some other rather interesting directions can include assessing the long-term memory of humans interacting with this kind of haptic language, whether humans still recall the vocabulary after certain time has elapsed. Not only that, but also how much training and interaction is required in order to learn this language in full, and if it is indeed feasible to use the language as a learning and assistive tool in dance and other motion-related activities.

References

MacLean

K.E.

, Pasquero

and Smith

, Building a haptic language: Communication through touch, Research Gate38(7) (2015), 1434–1452.

Brewster

and Brown

L.M.

, Tactons: Structured tactile messages for non-visual information display, In Proceedings of the Fifth Conference on Australasian User Interface-Volume 28, Australian Computer Society, Inc, pp. 15–23.

Riddle

D.L.

and Chapman

R.J.

, Tactile language design, In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Sage CA: Los Angeles, CA: Sage Publications, Vol. 56, No. 1, 2012, pp. 478–482.

Gentry

S.E.

, Dancing cheek to cheek: Haptic communication between partner dancers and swing as a finite state machine (Doctoral dissertation, Massachusetts Institute of Technology), 2005.

Gentry

and Murray-Smith

, Haptic dancing: Human performance at haptic decoding with a vocabulary, In Systems, Man and Cybernetics, 2003 IEEE International Conference on, Vol. 4, 2003, pp. 3432–3437. IEEE.

Hölldampf

, Peer

and Buss

, Synthesis of an interactive haptic dancing partner, In RO-MAN, 2010 IEEE, 2010, pp. 527–532. IEEE.

Velázquez

and Pissaloux

, Constructing Tactile Languages for Situational Awareness Assistance of Visually Impaired People, In Mobility of Visually Impaired People, Cham, Springer, 2018, pp. 597–616.

Morrison

, Knoche

and Manresa-Yee

, Designing a vibrotactile language for a wearable vest, In International Conference of Design, User Experience, and Usability, Springer, Cham, 2015, pp. 655–666.

Oliveira

V.A.D.J.

and Maciel

, Assessment of tactile languages as navigation aid in 3d environments, In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, Springer, Berlin, Heidelberg, 2014, pp. 104–111.

10.

Haritaipan

and Mougenot

, Cross-Cultural Study of Tactile Interactions in Technologically Mediated Communication, In International Conference on Cross-Cultural Design, Springer, Cham, 2016, pp. 63–69.

11.

Schelle

K.J.

, Gomez Naranjo

, Ten Bhömer

, Tomico

and Wensveen

, Tactile dialogues: Personalization of vibrotactile behavior to trigger interpersonal communication, In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, 2015, pp. 637–642. ACM.

12.

Oliveira

V.A.D.J.

and Maciel

, Introducing the modifier tactile pattern for vibrotactile communication, In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, Springer, Berlin, Heidelberg, 2014, pp 309–316.

13.

Altini

, Farella

, Pirini

and Benini

, A Cost-effective Indoor Vibrotactile Navigation System for the Blind, In HEALTHINF, 2011, pp. 477–481.

14.

Scheggi

, Talarico

and Prattichizzo

, A remote guidance system for blind and visually impaired people via vibrotactile haptic feedback, In Control and Automation (MED), 2014 22nd Mediterranean Conference of, 2014, pp. 20–23. IEEE.

15.

Aggravi

, Salvietti

and Prattichizzo

, Haptic assistive bracelets for blind skier guidance, In Proceedings of the 7th Augmented Human International Conference, 2016, p. 25. ACM.

16.

Tan

, Gray

, Young

J.J.

and Taylor

, A haptic back display for attentional and directional cueing, 2003.

17.

Haritaipan

and Mougenot

, Cross-Cultural Study of Tactile Interactions in Technologically Mediated Communication, In International Conference on Cross-Cultural Design, Springer, Cham, 2016, pp. 63–69.

18.

Morrison

, Knoche

and Manresa-Yee

, Designing a vibrotactile language for a wearable vest, In International Conference of Design, User Experience, and Usability, Springer, Cham, 2015, pp. 655–666.

19.

Spelmezan

, Hilgers

and Borchers

, A language of tactile motion instructions, In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2009, p. 29. ACM.

20.

Enriquez

, MacLean

and Chita

, Haptic phonemes: Basic building blocks of haptic communication, in Proc Conference on Multimodal Interfaces, Banff, Canada, 2006, pp. 302–309.

21.

Kosuge

, Dance Partner Robot: An engineering approach to human-robot interaction, in Proc Conference on Human-Robot Interaction, Osaka, Japan, 2010, pp. 201–201.

22.

Molina-Tanco

, García-Berdonés

and Reyes-Lecuona

, The delay mirror: A technological innovation specific to the dance studio, in Proc Conference on Movement Computing, New York, USA, 2017, 9:1–9:6.

23.

Yan

S.G.

, Ding

, Guan

, Sun

and Li

, Zhang, OutsideMe: Augmenting dancer’s external self-image by using a mixed reality system, in: Proc Conf on Human Factors in Comp Systems, Seoul, Reublic of Korea, 2015, pp. 965–970.

24.

Großhauser

, Bläsing

, Spieth

and Hermann

, Wearable sensor-based real-time sonification of motion and foot pressure in dance teaching and training, Journal of the Audio Engineering Society60(7/8) (2012), 580–589.

25.

Florian

, Mocanu

, Vlasin

, Machado

, Carvalho

, Soares

and Avram

, Deaf people feeling music rhythm by using a sensing and actuating device, Sensors and Actuators A: Physical267 (2017), 431–442.

26.

Shibasaki

, Kamiyama

and Minamizawa

, Designing a haptic feedback system for hearing-impaired to experience tap dance, in Proc Symposium on User Interface Software and Technology, Tokyo, Japan, 2016, pp. 97–99.

27.

Hossny

, Nahavandi

, Fielding

, Mullins

, Creighton

and Hutchison

, Haptically-enabled dance visualisation framework for deafblind-folded audience and artists, in Proc Conference on Systems, Man, and Cybernetics, Kowloon, China, 2015, pp. 446–450.

28.

Tsuchida

, Terada

and Tsukamoto

, A dance performance environment in which performers dance with multiple robotic balls, in Proc Augmented Human International Conference, Geneva, Switzerland, 2016, pp. 1–8.

29.

Alizadeh

, Witcraft

, Tang

and Sharlin

, HappyFeet: Embodiments for joint remote dancing, in Proc Graphics Interface Conf, British Columbia, Canada, 2016, pp. 117–124.

30.

Brown

and Paine

, Interactive Tango Milonga: Designing internal experience, in Proc Workshop on Movement and Computing, Vancouver, Canada, 2015, pp. 17–20.

31.

Park

S.I.

, Goo

, Noor

F.N.M.

and Kim

D.Y.

, Dancing queen: dancing in space (Connecting the Czech Republic, Spain, the United States of America and Taiwan), in Proc of the Asia-Pacific Advanced Network39 (2015), 40–53.

32.

Danjoux

, Choreography and sounding wearables, Scene2(1-2) (2014), 197–122.

33.

Giomi

, From haptic perception to movement: A phenomenological study on sound and technological mediation in dance pedagogy, Repertorio28 (2017), 342–361.

34.

Chiang

I.K.

, Spiro

, Lee

, Lees

, Liu

, Bregler

and Liu

, Dancing with Turks, in Proc Conference on Multimedia, New York, USA, 2015, pp. 241–250.

35.

Trajkova

and Cafaro

, E-ballet: Designing for remote ballet learning, In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, pp. 213–216. ACM.

36.

Ros

, Coninx

, Demiris

, Patsis

, Enescu

and Sahli

, Behavioral accommodation towards a dance robot tutor, in Proc Conference on Human-robot Interaction, Bielefeld, Germany, 2014, pp. 278–279.

37.

McLean

, Susko

, Harlow

and Bianchini

, Dancing robots: A collaboration between elementary school and university engineering students, in Proc on Creativity and Fabrication in Education, Stanford, USA, 2017, pp. 1–4.

38.

Bryant

D.A.G.

, Liles

K.R.

and Beer

J.M.

, Developing a robot hip-hop dance game to engage rural minorities in computer science, in Proc Conference on Human-Robot Interaction, Vienna, Austria, 2017, pp. 89–90.

39.

MACHINA: An advanced wearable technology, Smart clothing. Retrieved at August, 30, 2018 from https://www.machina.cc/.

40.

PercTube Base de Bolero Cha a 50 (2016). Retrieved from https://www.youtube.com/watch?v=8VIUXqiH48.