A basic tactile language to support leader-follower dancing

2.1 Tactile languages

Some examples of tactile language theory can be found in the literature [1–3], other applications range from situational awareness to navigation [7–9] and even to enhance communication [10, 11]. Dance applications focusing on the intrinsic tactile language have not been found. Recently, a common practice for conveying messages in environments where visual interaction is not available, includes the application of tactons or tactile icons [2]. A specific message can be transmitted by different signals, for example, tactile pulses with particular frequency, strength and duration. Therefore, a basic structure for developing tactile language can be tactons that should improve non-visual interaction, particularly, support interface concepts, objects and actions through well-structured signal messages with specific rhythm and location [1–3].

A useful methodology for designing a tactile language has been proposed in an application for squad level soldier communication, where particular combination and manipulation by tactons’ parameters is used for conveying meaning in real world contexts [3]. A good example of developing understandable tactile language has been proposed in an application for assisting visually-impaired people in situational awareness [7]. According to three main concepts introduced by them: tactile learning, tactile language and tactile memory, it was possible to define the process of learning any type of language and recognize several words coded with haptic stimuli, as well as to progress to even more complex sentences.

The assessment of tactile languages involving navigation in 3D environments is discussed by the work presented in [9], applying prefixes in tactons that enhance the level of expressiveness of language, while keeping it simple to learn. User studies based on three different tactile vocabularies (with and without prefixation), evidence the correlation between number of used vibrotactile parameters, tactile language complexity and amount of messages to be transmitted [12].

Some applications do not focus particularly on vibrotactile languages, but still use them to convey information for visually impaired people. The first one is reported in [13], where the main goal is to aid the visually impaired in unknown locations, developing a navigation system that is cheap and low power consuming, although only sighted users participated in the validation. In contrast, in [14], the guidance was performed by human operator, that sent vibrotactile cues to visually impaired participant through two bracelets. The operator also used camera glasses to obtain feedback about the participant environment. In addition to navigation, authors in [15] propose to use haptic feedback to further complement auditory instructions to a visually impaired student.

Non-wearable technology, or grounded technology, is also widely used; for example, [16] focused on a vibrotactile array mounted on the back side of a chair facilitating directional attention to sighted users by haptic feedback. As can be seen, there are several works concerning tactile communication, but research on tactile communication based on synthetic tactile languages, specifically for dance, is still needed. Related works on tactile languages is summarized, extended, and organized by application in Table 1.

There are several well-known dance applications such as dancing with robots [21], leader-follower interactions with a grounded haptic device [5, 6] and supporting dancers to improve their technique [22–24]. There are also useful systems that help deaf, blind, and deaf-blind people to experience a dance presentation [25–27] as well as others being mainly recreational and artistic are also found in literature [28–32]. From all of these applications, only few are based on wearable technology. In both [4, 5], a Phantom haptic device is used to guide a human arm’s motion imitating a leader-follower dance. Another project with the same objective of having a haptic robot lead a human in dance is described by Hölldampf et al, where the mobile haptic interface was developed and used with a head-mounted display for the user to be guided using virtual reality [6].

The work introduced by Alizadeh et al. supports learning and shared dance space using the VICON tracking system for shoes [29]. Authors of that work chose line dancing, when a dance does not involve touching the dance partner because dancers are remotely located. Another interesting system that gives dancers some control over music by receiving musical feedback from both dancers’ movements is reported by Brown and Paine [30]. They present an interactive system with real-time analysis that includes sonification of social tango providing new expressions of tango tradition by immersion and feeling as one with the dance partner. Other work focused only on dance presentation is discussed by Tsuchida, where dance with multiple robots of the spherical shape (Sphero2.0) is provided with better performance in entertainment and sports activities [28]. Another example of dance presentation can be found in [31], where the main goal was to give presentation connecting music and dancers all around the world. Authors used Ultragrid for video transmission as well as two dancers were recruited for this performance with final proposal to place all participants on a single screen. It can be seen that there is a good opportunity to study non-verbal communication using leader-follower dances, since it is indeed what happens when dancing with a couple. Table 2 summarizes and extends the related works using technology in dance.

Based on previously described related works, the following opportunities have been identified. Couple’s dancing involves a leader and a follower requiring communication between partners that mainly uses touch sense. However, work was not found that focuses on how to use synthetic tactile language to provide guided leader-follower dance. Currently available technology used in dance mainly focuses on enhancing the experience, either for dancers, for audience or both. On a very different research direction there exist several works that study communication properties of haptic stimuli. Their applications are quite wide covering motion activities such as sport and navigation. The main opportunity of this work consists in exploring how synthetic haptic communication can benefit from interaction occurring when two people dance together. The main questions to be addressed are: can users understand dance commands via vibrotactile stimuli? Is wearable technology suitable to communicate dance commands? Is this communication indeed similar to leader-follower dance? Can basic vocabulary be regarded as foundation to develop tactile language to support motion activities? To answer these questions, the following concept was conceived.

The methodology adopted for study 1 was to define vibration patterns that represent each of the nine movements according to the defined vocabulary presented in Table 3. The pairing of movement and actuator was designed based on the metaphor that vibration represents a gentle push (e.g. vibration on the front means to go backwards). Then, a three-phase experiment was performed with each participant. Seven participants with ages ranging from 20–30 years old were selected for this study, particularly, five women and two men, everyone with previous experience in leader-follower dances.

Their demographics are summarized in Table 4. To all participants were handed written agreement to be signed and a post-test questionnaire to be filled after completing study.

The phases of the study were:

Phase1: Natural Response. All nine expected movements of the vocabulary were described to the participants, without explaining which actuators represent each movement. Participants were told to respond to received signal with one of these moves, the one that they felt most natural according to the vibration received. The vibrotactile signals were sent twice for each participant in a random order.

From the Natural Response phase each of the final outcome of the step performed of every participant was kept using the following notation. Good: participant responded following the vocabulary correctly, according to push metaphor. Inverted: participant understood the expected movement, but misunderstood direction (e.g. participant performed right step forward instead of right step backward), meaning the vibration is being interpreted as a gentle pull (instead of the gentle push). Wrong: Participant performed a movement that does not correspond to the vibration sent, not even considering a pull metaphor. From the Code Response phase participants responses were kept with the same notation described previously. Results are presented in Table 5. Because each of the nine movement was sent twice, the total number of correct movements are eighteen for each participant in each phase. The accuracy column represents the percentage of correct movements performed, meaning the participant understood the vibration sent.

Analyzing the results of the Natural Response phase, only one participant had more good than inverted, while everyone else had more inverted than good. This implies that the push metaphor is not as natural to users as the hypothesis suggested, instead users feel the pull metaphor as more natural. This finding will be the foundation for Study 2. Four out of the seven participants had more wrong than good and inverted together. This implies that the proposed vocabulary is not as intuitive as it is, even though it was proposed by two of the authors who are regular couple dancers. The wrong outcomes varied significantly from participant to participant, including movements not considered in the vocabulary and misunderstanding of what the close step meant. The accuracy ranged from 0% to 61%, average being 15%. All of these findings suggest a more structured training is needed for participants to understand the proposed basic tactile language correctly, and was also considered for Study 2.

In the Code Response phase, once the vocabulary was explained, there was a significant improvement in the accuracy rate of all participants, ranging from 33% to 83%, average being 65%. This is a major improvement from the 15% obtained in the previous phase, although it is far from being acceptable in a real life application. Also, inverted movements were still observed, even after the push metaphor was explained to the participants. Most of them justified themselves by saying it was confusing following a push metaphor. In various cases, the wrong movements were misinterpreted closes, returning the bodyweight instead of going to it.

In the Sequence Response phase participants had a downgrade in their performance, compared to the previous phase. These results are summarized in Table 6, were accuracy ranged from 16% to 84% , average being 46%. These results were very evident when none of the participants succeeded in performing any of the two choreographies, or recognizing any of the steps they were doing. In fact, everyone looked confused as to what they were doing. Thus, the communication system is less effective when used in situ, as in a real activity and not only recognizing the meaning of the vibrotactile cues.

In general, it was observed that all participants performed each move one or two seconds after each signal was sent. Some of them said that they needed to be focused to identify which actuator had been activated, and also to remember which of the nine movements it meant. Even after explaining the vocabulary, participants inverted the movements, and only afterwards they remembered the coding used and corrected themselves. The most challenging movements for the participants were the turns, sometimes performing them instead of close, or inverting the direction. Also, the close step presented many difficulties to the participants, such as inverting it or performing a turn (the same problems as the turns). A possible explanation is that identifying two actuators instead of only one confused the participants. A final remark of the observations was the influence of the music, participants seemed to enjoy it, swaying their hip and their arms in sync with the beat. If it caused a distraction or motivation is uncertain.

At the end of the three phases, participants filled out a post-test questionnaire, which among other questions included the System Usability Scale (SUS). A total mean score of 69.29 was obtained which is above average and means the proposed approach has good potential of acceptance. Regarding a question of the placement of the actuators, some participants agreed that when the back and front actuators of the same side activated, it was difficult to tell if one or both were vibrating, and thus they suggested an alternative position may come in handy. A good result regarding the activity was that all participants agreed it was highly entertaining, and very similar to dancing. For example, a comment of one of the participants stated: “Could be an entertaining videogame similar to Guitar Hero, but different to Just Dance because there are vibrations in the body”. Based on all these findings Study 2 was developed to improve the proposed basic tactile language.

The same hardware, software and layout from Study 1, were used. Significant changes were made in the methodology, inspired by the findings of the previous study, such as changing the first phase from natural response to vocabulary training. The purpose of this change is to improve the results by ensuring participants are successfully trained in understanding the language. There are minor changes in every phase, mainly the addition of auditory feedback for good/wrong performed moves. The core difference is the message coding, now a pull metaphor is being considered. The new vocabulary considering the new message coding is represented in Table 7.

Eleven participants of ages ranging from 20–29 years old were selected for this study, particularly, six women and five men, each with previous experience of leader-follower dance. Three of these participants also participated in Study 1, in order to evaluate if they considered there was a significant improvement between studies. Only one of them had previous experience with wearable technology. Only three of the male participants had experience as leaders. All the demographic info is summarized in Table 8.

Phase 1: Vocabulary Training. This phase was added to ensure the correct understanding of moves that were expected from each participant. Wearable technology was not used, but instead the operator gave verbal instructions of the expected movement to the participant. Direct feedback from the operator was given to the participant on whether the performed movement was correct or incorrect. In the case it was incorrect, the facilitator made the correct movement together with the participant. As stated previously, gaming characteristics were also added to improve learning and motivation. This was achieved by adding two sounds as feedback, and also keeping the participants total score of correct movements. An encouraging chime-like sound was played by the operator when the participant understood the instruction correctly, thus adding a point to their score. If the participant interpreted the instruction performing the wrong movement, a negative sound was played as feedback. The final significant change was pre-defining the sequence of movements, not making it random. This change corrected the problem of a random order occasionally placing two steps together that do not make much sense (e.g. performing right foot forward twice).

The results of every phase of Study 2 are presented in Table 9. Each value corresponds to the percentage of total correct interpretations. Overall, every participant had an excellent performance, with few mistakes. A peculiarity observed is that most participants got better results when no sound feedback was given to them. One possible reason would be that they had previously done the sequence with sound feedback, and knew if they were mistaken the first time. Another explanation could be there were no distractors and they stayed more focused on the task. Either way, further research is needed in order to reach significant conclusions from this setup.

From both experiments it can be concluded that using a pull metaphor is more intuitive than the push variation. Whether this is a general rule or it only applies to people with experience in leader-follower dances is still uncertain. Further proof can be found on a comment made by a participant who was in both studies: “Actually, the vibration is better than last time (Study 1). It is easier to dance with the feeling of being pulled than pushed”. As final evidence, a direct comparison of correct outcomes from the sequences of phase 3 in both studies is presented in Table 10.

A significant improvement of +45% and +41% was observed from Study 1 to Study 2 for each sequence. A possible explanation of the success could be the pull metaphor, the new proposed training method to learn the basic language, and the gamification. From the post-test results, every participant perceived the vibration of the technology as comfortable, an important issue to be considered when wearing a vibrotactile actuator. As in Study 1 both right and left turns were identified as the most confusing steps to perform. This could occur because two actuators are being used instead of only one, although future research is also needed to find out whether a difficulty lies in the movement or the coding.

Another result of the post-test consistent with Study 1 was that the study was considered very entertaining for all participants. Also some of them commented directly to the facilitator and operator that the first and second phases (the training ones) could be helpful for people that are beginners learning how to dance. This also opens up a new research direction considering this basic tactile language as an assistive tool in teaching leader-follower dances.

A non-conclusive question was related to the amount of movements in the vocabulary, since answers varied from participant to participant, therefore suggesting it is dependent on each participant. One participant gave an interesting suggestion for the proposed language: “it helps working on capacity of motricity, it could be useful for children that are candidates to high performance training”. This possibility could also be further explored. Another participant (who teaches tango lessons) commented: “It could also be used to create reasoning in dancers as warmup exercise since it requires you to think and respond quickly to a stimulus instead of imitating the teacher”.

This also suggests that this basic language can be used to improve the abilities of dance students who are not necessarily beginners. SUS was again included in the post-test, yielding average total of 77.95 points, which is higher than the value obtained in the previous study, thus improving its potential acceptance. Overall results for the post-test questionnaire are presented in Fig. 4. A five-point Likert scale was used for each of the questions, and since there were eleven participants the maximum score possible was 55 points.

OPEN IN VIEWER

Fig.4

Results from the post-test describing acceptance of studies by participants.