On the use of dance notation systems to generate movements in humanoid robots: The utility of Laban notation in robotics

Abstract

French

This article raises the issue of collaboration between the field of dance and that of robotics. Dance notation systems are developed in the choreographic field as a method to describe and record human movements. Translating movements into symbols is of particular concern for computer science and robotics, which are interested in generating anthropomorphic motions in robots or animated avatars. We examine different research projects on the use of Laban notation for this purpose, and present our own attempt at generating dance movements in a humanoid robot from a Laban score. We discuss the interests and difficulties related to the use of a dance notation system in a robotic context.

Keywords

dance notation humanoid robot Laban Laban Movement Analysis motion generation

Introduction

There are many ways to study human motion. For instance, non-verbal communication emphasizes the importance of gesture, body posture and facial expression when communicating and interacting with others (Kendon, 1981). In psychology, human motion is analyzed in terms of the realization of tasks or goal-oriented actions (Bril & Goasdoué, 2009). Biomechanics is concerned with the mechanisms and physiological aspects of human motion (Blanchi, 2000). Ergonomics aims to find the most efficient gestures for using machines (Falzon, 2004). Neuroscientists consider the integration between motion and perception (Berthoz, 2008). Anthropology and sociology are interested in the social determination and construction of gestures and postures (Bourdieu, 1980; Mauss, 1934). In robotics, especially humanoid robotics, the goal is to generate human-like motions and to design generation programming for this purpose. On this matter, choreography may be a source of inspiration for roboticists. Dancers and choreographers can be considered as specialists in human motion. They experiment with human motion and expand the repertoire of gestures and dynamic postures, developing their own methods to conceptualize motion. Among these methods, the dance notation system, developed since the 15th century, is a rich and flexible tool for exploring and characterizing the richness of human motion.

The objective of this article is to delineate robotics and dance notation. Robotics is technology driven, whereas dance notation elaborates on human experiences and observations. Both fields share a common interest in human motions, and they can learn from each other about the ways they describe, analyze and eventually model human movements.

We will first explain what a dance notation system is, focusing especially on Kinetography Laban. We will then describe the ongoing collaborative research between dance notation systems, robotics and computer science. In the third section, we will present our own attempt to design a motion generation program for humanoid robots starting from a Kinetography Laban score. This work has been developed within the Gepetto research team, at LAAS (Laboratoire des Analyses d’Architecture et des Systèmes), in Toulouse. Finally, we will examine the points of agreement and disagreement between the field of robotics and that of choreography with respect to understanding human movement.

What is a dance notation system?

Dance notation is a system for recording and analyzing human movements using specific symbols. Today, about 90 notation systems exist in Europe and North America (Hutchinson Gest, 1989). In the same way musical notation matches sounds to symbols, dance notation translates four-dimensional movements (3 dimensions and time) into signs written on a page (Hutchinson Gest, 1984). Dance notation serves mainly as a way to store information, to teach technical dance exercises and to develop a literature of dance compositions.

Historically dance notation has been used to describe traditional dances (e.g. ballet, court dance, country dance). Since the 19th century, the field of choreography has explored any type of movement that might be included in dance performances. The most currently used dance notation systems are Kinetography Laban (Labanotation) (1928), Benesh Movement Notation (1956), and Eshkol Wachman Movement Notation (1958). All of them allow the notation of every kind of human movement. In this article, we will focus on Kinetography Laban, and we will consider its applications to the field of robotics.

What is Kinetography Laban?

How can we describe our movements? Even during simple ordinary actions such as walking, running, raising a hand or grasping something, it is not easy to describe exactly what is happening in the body. Kinetography Laban enables us to break down these actions and to describe any kind of movement (e.g. dynamic movements, postures, relationship with others or objects, group movements) using symbols. Kinetography Laban was created by Rudolf Laban, an Austro-Hungarian choreographer and dance theorist, and was first published in 1928.

Kinetography Laban describes movements using three factors: body, time and space. The notation score is composed of a vertical staff and several abstract symbols. When describing a movement, a notator should answer four principal questions: (1) which body (or parts of the body) makes the movement? (2) What is happening (e.g. in which direction is a part of the body going)? (3) When does it happen? (4) How long does it last? (Knust, 2011). Theoretically, all observable movements can potentially be noted in the system.

Rudolf Laban’s initial design was developed and finalized by Ann Hutchinson in the USA and by Albercht Knust in Europe. Today the system is considered complete; it is known as ‘Labanotation’ in the USA and ‘Kinetography Laban’ in Europe. The Laban notation system was constructed observing human movements and subjective body experiences. The system therefore contains an empirical knowledge of human movement. The rules in the Laban notation system are based on a human body and its movement capacity.

Overview of the attempts at automating Kinetography Laban

As Kinetography Laban is structured around a systematic method of describing movements, the idea of providing the notation system with a computational basis occurred naturally with the advent of technology. Attempts to elaborate a computational Laban system began as early as the 1970s when Laban notation started to gain popularity. A computational Laban system has three main applications: (1) notation editing; (2) automatic translation of notations into animations; and (3) automatic translation of motion-capture data into notations. The Laban system has a structure and a framework for representing human movements; it defines three components of the movement: space, body and time. The systematic way of describing movement made computer scientists think of automating the notation system. However, these attempts, especially automatic human generation from Laban notation and generation of Laban score from motion captured data, are still an ongoing project, and further development is expected.

Notation editing

A notation editor is a system allowing a notator to write Laban scores directly on a computer. One of the earliest attempts to design such a graphic editor for Laban notation dates back to the 1970s (Brown, 1976; Zalla, 1974). The motivation was to make the notator’s life easier by abridging the laborious work of manual writing (Brown et al., 1978; Smoliar, 1978). Of course, such a notation editor does not entirely replace the work of a notator; it merely offers a quick and easy way to take notes and store them in a computer.

In this vein, the ‘LED project’ initiated by the University of Technology Sydney aimed to create a graphic interactive editor for Labanotation (Hunt et al., 1989). There are many examples of related projects, some of them still in progress: ‘LabanPad’, a program for interactive Labanotation input (Griesbeck, 1996); ‘Labanator’, an editor for AutoCad (Gabor, 2014); ‘Calaban’, an editing software developed for Windows (Adamson, 2004); ‘LabanGraph 4P’, an application program interface for AutoCAD (Fugedi, 2011); and ‘LabanWriter’, an editor for the Macintosh developed by Ohio State University. An iPad application, ‘KineScribe’, has also been created by Reed College to edit scores in Labanotation.

Automatic translation of notation into animation

An interesting prospect regarding the computation of the Laban notation system is to create software that makes it possible to read Laban scores automatically and to generate movements in the form of computer animations.

In the domain of computer animation or video games, it is crucial to determine the most efficient way to represent the human body and its movements. In this regard, Laban notation may be used as a tool to elaborate more expressive animation characters using a kinetic formalism.

In its early stages, the generation of animations from Laban scores was implemented in stick figures or in crude humanlike avatars (Badler, 1979; Zalla, 1974). More recently, realistic 3D avatars have replaced the rough figures, such as in ‘LabanEditor’, an interactive graphic editor for Labanotation scores (Choensawat et al., 2016), or ‘LabanDancer’, a program for translating notation into animation developed through a collaboration between the Dance Notion Bureau,¹ Simon Fraser University, the University of Waterloo and Credo interactiv Inc. (Wilke et al., 2005).

In addition to helping create more realistic movements, notation-to-animation software has a role in heritage preservation, with the possibility of representing and storing emblematic choreographies (Hachimura & Nakamura, 2001; Wilke et al., 2005). It is also a pedagogical tool, making it possible to reduce the complexity and difficulty of learning movement notations (Calvert, 1986).

Confronted with the abundance and complexity of human motion, the possibility of generating movements from Laban scores is still limited. Both LabanEditor and LabanDancer can only display body translations, as well as leg and arm gestures. These limitations aside, the development of an automatic translation system is innovative and could foster the adoption of notation systems by a larger public.

Automatic translation of motion-capture data into notation

Moving in the opposite direction from the previous application, it is possible to generate Laban notation from movements recorded using the motion-capture technique. While several teams are working on this endeavor, the project is still in its infancy, and it remains difficult to generate correct Laban scores (Chen et al., 2005; Chen et al., 2013; Shen et al., 2005).

Laban Movement Analysis and robotics

Laban Movement Analysis (LMA) is a different system from Kinetography Laban (Labanotation). LMA focuses on the classification and analysis of the movement qualities, while Kinetography Laban aims at transcribing all observable movements. Specialists in humanoid robotics have noted the particular capacities of LMA to give robots more human-like movements. LMA was also conceived by Laban and developed by his colleagues, Lisa Ullmann, Irmgard Bartenieff and Warren Lamb.

LMA is structured around four concepts: Effort, Shape, Body and Space. Effort deals with qualitative change according to certain motion factors (time, space, weight and flow). Shape concerns the body’s capacity to change volume and shape, the way it adapts to the environment as well as to internal transformations. Body focuses on gesture, posture and the way they merge into whole body organizations. Space examines the general, personal and interpersonal spaces, and reveals the way movement unfolds into these different spaces (Loureiro de Souza, 2016).

Computer scientists initially attempted to create an automatic LMA system, which was called EMOTE (Expressive MOTion Engine). EMOTE is a computational system that makes it possible to adjust the parameters of Effort and Shape to generate movements for animation characters (Chi et al., 2000). EMOTE allows the input of parameters which make the movements of animation characters more natural, expressive and human-like. Inspired by the EMOTE project, subsequent research has examined the possibility of creating a computational model that could automatically extract motion qualities from human performance (Zhao & Badler, 2004).

In the domain of computer animation, where creating expressive movements is paramount, LMA may be used to describe and define styles of motion (Torresani et al., 2006) or to characterize human behavior (Santos & Dias, 2010). LMA is also used to translate human movement features into mathematical formulas, for instance to retrieve movements in a motion database for the creation of 3D CG animations (Okajima et al., 2012; Wakayama et al., 2010). A similar objective of retrieving and indexing motion also exists in the domain of dance (Aristidou & Chrysanthou, 2014). In the domain of intangible cultural heritage, LMA is also used to extract the nuance of motion in order to compare and evaluate dance performances (Aristidou et al., 2015).

LMA also finds an application in the domain of eLearning for athletes, dancers or rehabilitation specialists. Effort, one of the four LMA parameters, may contribute to the creation of a guiding language system to judge whether or not learners’ movements are correct, even in the absence of expert advice (Chen et al., 2011).

In the area of human–robot interaction or human–machine interaction, Masuda et al. (2009) designed the Laban feature value set, inspired by LMA, to determine bodily expression associated with specific emotions. LMA has also been used to classify human movements for the design of a visual tracking system able to capture and analyze human movements from multiple perspectives (Rett et al., 2008).

LMA makes it possible to identify different qualities of movement associated with certain emotions, for example the particular walking pace associated with being happy, angry, sad, etc. In robotics, LMA can be useful in translating emotions into algorithms, helping model affective motions (Burton et al., 2016). This kind of computation could endow a humanoid robot with the capacity to detect the emotional movements of the person with whom it interacts. (Lourens et al., 2010).

To summarize, LMA finds in robotics a domain of application. Having been developed to account for the information transmitted via body movements, LMA naturally integrates recent thinking on the way robots can convey emotions and attitudes through non-verbal communication. LMA thus makes it possible to classify, evaluate and label movements according to their qualities, which can then be associated with emotions or personality traits.

Using Kinetography Laban to program a humanoid robot

With the aim of designing a system to automate Kinetography Laban, our own research was motivated by the desire to create a motion-generation program for humanoid robots using Laban notation (Salaris et al., 2016). To evaluate the feasibility of such a project, we chose a simple dance sequence, called ‘tutting’, a kind of hip-hop dance that involves only the upper body, especially the arms and the hands. The plan was to notate the tutting dance in the form of Kinetography Laban, and then to translate the Laban score into the framework of a motion-generation program called ‘Stack of Tasks’ (Mansard & Chaumette, 2007). After having implemented the Laban notation into the Stack of Tasks, we simulated the tutting dance movements in a humanoid robot called Romeo. We could then observe the differences between Romeo’s movements and a human dancer. After presenting the method we used to implement the Laban notation in the Stack of Tasks, we will discuss the reasons for these differences between robots and humans.

Notating a tutting dance

The tutting dance² is notated by Kinetography Laban. The score is shown in Figure 1. The score contains nine columns. Each column is associated with a body part. The symmetrical staff represents the symmetry of the body. The central columns on both sides of the main vertical axis represent the support of the body. Then the second column is dedicated to the movement notation of the right and left legs. The third column, immediately outside the staff, is used for the torso and its parts. The tutting dance concerns primarily the arms and the hands. This is why the second and third columns do not contain any symbol, while the support columns contain only two small circles, which represent the ‘hold weight sign’, just after the double line indicating the start of the movement. In this case, these symbols indicate that the actor has to maintain a standing posture with the weight on the feet.

Figure1.

Laban tutting dance score.

The fourth column, on the right and on the left of the body columns, corresponds to the right and left arm gestures, respectively. The fifth column corresponds to both forearm gestures; the sixth to both upper arm gestures; the seventh to the right and left hand gestures; the eighth to the back and palm of the right and left hands. Finally the last columns on the right and the left correspond to the edge of fingers.

The duration of the sequence is broken down into 16 intervals according to our movement segmentation. Moreover, based on our observations, we deduced that no movement is faster or slower than the others. As a consequence, in the Laban score the direction symbols have the same length, which means that all the movements have the same duration.

A detailed description of each movement is reported here:

0. This is a starting point. An actor is standing. Both arms are stretched out along the body.

1. The right arm goes in the right-middle direction.

2. The left arm goes in the left-middle direction.

3. The right upper arm goes in the forward-middle. The right forearm goes in the left-middle direction.

4. The left upper arm goes in the forward-middle direction. The left forearm goes in the right-middle direction.

5. The left forearm goes in the forward-middle direction.

6. The right forearm goes in the forward-middle direction.

7. The left hand goes down.

8. The right hand goes up.

9. Both forearms go up. During this movement, both hands maintain their configurations with respect to the forearms. (Pause on both hands.)

10. Both hands go in the right-middle direction.

11. The right upper arm goes in the left-forward-middle direction. The left upper arm goes in the right-forward-middle direction. Both upper arms are in contact. The left and right palms are also in contact.

12. The right and left hands, while maintaining the contact, change their direction to the left-middle.

13. Holding of the contact is cancelled. The right upper arm goes in the intermediary direction between forward-middle and left-forward-middle. The left upper arm goes in the intermediary direction between forward-middle and right-forward-middle. As a consequence, the palms separate. The edge of the right finger touches the edge of the left hand.

14. The left hand goes in the right-middle direction. The left palm is in contact with the back of the right hand.

15. Both upper arms go in the forward-middle direction. The edges of both fingers are in contact.

16. The upper right arm goes in the right-forward-low direction. The right forearm goes in the left-middle direction. The upper left arm goes in to the left-forward-middle direction. The left forearm goes to the right-middle direction. The hands go up. Both palms are in contact.

From tutting dance notation to Romeo via Stack of Tasks

After the tutting dance is transcribed into Laban notation, the score is translated into a motion-generation program, the Stack of Tasks, in order to generate suitable control signals for the robot Romeo’s motors to execute the movements (Salaris et al., 2016).

Comparison of the information provided in the Laban scores with the information sent to the robot

Humanoid robots use around 30 motors to perform their action (Kajita et al., 2009). In general, there are two ways to control a humanoid robot: either the robot is controlled at a distance – this is called teleoperation – by transfering human motion-capture data to the robot or by using computational programming. Controlling the robot via a graphic interface does not make it autonomous (Kajita et al., 2009), while using a computational programming, even though a more complex procedure, means that the robot can be granted a certain degree of autonomy (avoid obstacles, plan a trajectory, etc.).

The Stack of Tasks belongs to the latter approach, with a motion-generation program that sends control signals to the robot’s motors. The Stack of Tasks considers each action the robot should execute as a series of tasks, and describes the hierarchy of these tasks. Robot actions are redundant, they do not have a limit on movement as a human does. For example, a robot could turn its head 360 degrees, a feat impossible for a human being. A robot’s actions may also conflict, for instance if one hand tries to reach an object while the other hand is trying to reach the opposite side of the object, preventing the other hand from grasping it. It is therefore necessary to order a robot’s actions. The Stack of Tasks serves this purpose, giving the appropriate succession of movements for the robot to execute an action.

In the case of Romeo’s tutting dance, the first and most important task is to maintain a static equilibrium: Romeo should stand and not fall down. For the first action of the tutting dance, there is no displacement of the entire body, Romeo keeps standing. In the Laban score, keeping standing corresponds to the ‘hold sign’, which specifies the continuation of the starting position. In the Stack of Tasks, the first task, called ‘weight on the feet’, is programmed in order to guarantee that the weight of the body rests on the feet. Then the arm movement ‘right arm goes in the middle-right direction’ is programmed, followed by the left arm ‘stay in down position’. The fourth task consists in not moving the torso position during the arm movements. The last task consists in keeping a reference configuration, which is the natural standing position.

One difficulty with translating a Laban score into a motion-generation program is that there is not a complete correspondence between the symbols of a Laban score and the tasks defined in the Stack of Tasks. To generate the desired action, it is necessary to program each task the robot has to accomplish. However, in the Laban system, a symbol is placed only when the dancer has to move. Moreover the rules of the system are based on the human standing position. Such a default position does not exist in robotics, and the roboticist needs to explicitly send the commands for the robot to hold a standing position. The information given to a human dancer and the information given to a robot are thus extremely different, and it is necessary at some point to resolve the discrepancy between the symbols of the notation and the inputs sent to the robot.

Implementation of direction symbols into the Stack of Tasks

One of the most basic and principal symbols in Laban notation is the direction symbol. There are 27 principal direction symbols. Translating these symbols into the Stack of Tasks was the starting point of our work. The direction symbol placed in the limb column (in our study, the limbs correspond both to the entire arm and parts of the arm) indicates in which direction the limbs’ free end should move from the current position.

For a robot to move its arm, the Stack of Tasks sends control signals to reduce the distance between the current position and the desired position, while at the same time performing other tasks. The control signals correspond to either velocities or accelerations, with the acceleration control providing a smoother movement than the velocity control (Salaris et al., 2016).

Comparison of the robot and the human dancer

The tutting dance was first virtually generated in a 3D simulation of Romeo. We could observe some differences between the way Romeo and a human dancer performed the dance. In a Laban score, the direction symbols are placed continuously and successively. This implies that the movements performed by the dancer occur one after the other. This is not the case with the robot, which demonstrates some movement overlapping: before the previous movement has ended, the next movement has already started. Moreover, Romeo cannot prevent his torso from moving when making an arm movement. This is due to the fact that Romeo has only 37 degrees of freedom, and also to the way the actions are programmed in the Stack of Tasks.

Path rules for arm movements

One of the principal differences observed between the robot and the dancer regards the way the arm moves. In the Stack of Tasks, direction symbols are interpreted as positions in space. Each of Romeo’s motors is controlled so that a body segment’s free end moves in a straight line, from the initial position to the final one. However, in the Laban score, the right forearm’s free end should move along a circular arc centered on the elbow, while the rest of the body remains fixed. This arm movement is called ‘peripheral movement’ in the Laban system (Salaris et al., 2016).

In Kinetography Laban, the rules for path movement of the limbs are clearly defined à la fin de phrase, ajouter (see Figure 2). For example, if the arm, held straight in front of the body, moves 45° to the right, a ‘first-degree distance’, the arm’s free end (i.e. the hand) describes a circular arc whose center is the shoulder. This is called a ‘peripheral movement’ in Kinetography Laban (Knust, 2011). All movements between the forward position and the second-degree distance point will produce this sort of path, without any special flexion of the arm unless otherwise specified by the addition of a particular sign (e.g. the straight path sign). If a limb moves to the ‘third-degree distance’ or the ‘fourth-degree distance’, the limb’s free end will move along a trajectory close to the body, and not a peripheral path. For instance, if the arm moves from a forward position to a backward position, the arm retracts itself in the direction of the body (‘in place’), and then extends again in the direction opposite to the initial direction. This type of movement is called ‘central movement’ in Kinetography Laban (Knust, 2011). If the arm moves from a forward position to the ‘third-degree distance’, the arm’s path will move between the periphery and ‘in place’, an ‘intermediate situation or transversal movement’ (Knust, 2011). All these rules for path movement are based on the naturalness of human movement.

Figure 2.

Path degree in Laban notation.

The differences between the robot’s and the dancer’s arm movements are due to lack of information in the way direction symbols are implemented in the Stack of Tasks. In a Laban score, the direction symbols do not indicate only a position in space, they also contain information about the duration and the path to take for the limb to reach a desired position. If there is no specific indication about the way to move and no special symbol added, the movement should be ‘natural’. This does not mean that the dancer can do whatever s/he wants, but that the dancer should adopt a natural position.³

Romeo’s arm movements are programmed according to a control law that gives rise to a straight line path for the free end of the arms, whereas the dancer’s movements follow peripheral and intermediate paths. The control signal used to move Romeo’s arm does not contain enough information to accurately reproduce a human movement. It is of course possible to program a robot’s arm movements in such a way that they perform peripheral and intermediate paths, but this requires the addition of more specific control signals.

Differences between Kinetography Laban and robotics regarding the definition of movement

The observations we collected during the implementation of a tutting dance in Romeo illustrate a discrepancy between the ways dance notation and robotics envision movement. In robotics, a movement is defined as a series of instructions sent to a motor, while Laban notation considers a movement in terms of displacement in a physical space.

This experience illuminates the problematic of each field, dance notation and robotics, even if human motion is the common topic of both. The main issue concerning movement in robotics is the movement in motor-control level, and not the movement in the physical space, while Laban notation does not give any information concerning the movement in motor control level, even if it gives all information pertaining to physical space. The point of view on the generation of movements is thus completely different in the two fields.

Physical space and motor-control space

A dance notation system translates observable body movements into abstract symbols. A dancer trained with a dance notation system is able to ‘embody’ the motion symbols: when reading a score, s/he ‘sees’, for instance a movement of the right hand in the physical space; s/he knows what to do to perform a slow movement forward. The details of muscle activations are implicit, and the dancer does not need to think about the muscular organization that underlies his/her performance. A robot, on the other hand, does not know what to do in order to move its hand forward, but it possesses a list of motor-control instructions. When translating a dance notation into a motion-generation program, the fundamental problem is therefore to interpret actions expressed in the physical space in terms of movements expressed in the motor-control space (Salaris et al., 2016).

Possible obstacles to a dialogue between dance and robotics

We have seen that one of the principal difficulties in a collaborative project involving dancers and roboticists is that they do not conceive movement in the same manner. As an example of one such conceptual discrepancy, the use of the word ‘rotation’ turns out to be very different in the two disciplines. In Kinetography Laban, movement is classified as either a ‘displacement’ or a ‘rotation’. A displacement means that the whole body, or a body part, moves from a given point to another point in space. A rotation indicates that a limb needs to rotate on its own axis, or the body to spin on its central axis. In robotics, limb movements are expressed in terms of ‘rotation of the articulation’. A movement, for example lifting the arm, would be expressed in the Laban system as the displacement of the arm’s free-end, while in a robot, the movement would be expressed as a rotation of the articulation at the shoulder level. This example shows clearly the different points of view of the two disciplines. In the Laban system, the language and the rule are constructed from the point of view of the human body and what is considered a natural movement. Roboticists express the movement from the point of view of the motors that need to be activated to produce a motion. The distinction between the physical spaces within which the movement is realized and perceived, and the motor space from which the movement is performed is a fundamental distinction making it difficult to translate a dance notation system into a robot-programming system.

Conclusion and discussion: Toward the principles of human movement

We have seen that Kinetography Laban does not provide the information in terms of motor control that would be required for straightforward motion generation in a robot. If Kinetography Laban is not useful for this purpose, how can dance notation contribute to robotics research? First, Kinetography Laban has the advantage of providing information about complex actions that would be difficult to describe otherwise. A dance notation system makes it possible to break down and to segment a complex action, augmenting its clarity. Roboticists can use this information to get a sense of the complexity of a particular action, even if the action is only described in a physical space. The second interest offered by Kinetography Laban is the notion of the ‘naturalness’ of a movement, on which a great deal of a dance interpretation is based. Kinetography Laban is constructed around the notion of natural body states and movements, and offers a good vantage point on this problematic. What constitutes a natural movement, and what are the underlying principles that make a movement natural? Does naturalness come from a certain economy of energy, ease at the level of the articulations? Is it possible to translate the notion of naturalness in terms of the way a human body occupies the space around it, in relation to gravity? It would be extremely interesting to define this notion of naturalness more precisely, to express it in a suitable mathematical manner and to use it to determine the laws of control for a humanoid robot (Salaris et al., 2016).

Altogether, beyond the obvious differences between dance and robotics, a common interest in the nature of human movement should foster more collaborative research aimed at the principles of movement.

Footnotes

Funding

This collaborative study was done in the framework of the European Research Council project Actanthrope (N°340050), which aims to explore the computational foundations of anthropomorphic action.

Notes

Author biographies

Naoko Abe received her Bachelor’s and Master’s degrees in Sociology from Paris Descartes University. She obtained a Doctorate in Sociology from the École des Hautes Études en Sciences Sociales (EHESS) in Paris in 2012. Her thesis research was carried out in collaboration with the RATP (Parisian Public Transportation Authority) from 2008 to 2012 on subway users’ behavior, using Kinetography Laban to analyze human behavior. She obtained an advanced teaching and notation certificate in Kinetography Laban in 2011 from the Conservatoire de Paris. She held a post-doctoral fellowship at the LAAS (Laboratory for Analysis and Architecture of Systems) in Toulouse from April 2014 to September 2015. She was a Renault research fellow from October 2015 to September 2016 and since October 2016 is a research fellow at the EHESS France-Japan Foundation.

Jean-Paul Laumond, IEEE Fellow, is a roboticist. He is Directeur de Recherche at LAAS–CNRS (team Gepetto) in Toulouse, France. His research is devoted to robot motion. In the 1990s he was the coordinator of two European Esprit projects – PROMotion (Planning RObot Motion) and MOLOG (Motion for Logistics) – both dedicated to robot motion planning and control. In the early 2000s he created and managed Kineo CAM, a spin-off company from LAAS–CNRS devoted to developing and marketing motion-planning technology. Kineo CAM was awarded the French Research Ministry prize for innovation and enterprise in 2000 and the third IEEE–IFR prize for Innovation and Entrepreneurship in Robotics and Automation in 2005. Siemens acquired Kineo CAM in 2012. In 2006, Laumond launched the research team Gepetto, dedicated to human-motion studies along three perspectives: artificial motion for humanoid robots, virtual motion for digital actors and mannequins, and natural motions of human beings. He teaches Robotics at the École Normale Supérieure in Paris. He has edited three books and has published more than 150 papers in international journals and conferences on robotics, computer science, automatic control and recently on neurosciences. He was the 2011–12 recipient of the Chaire d’Innovation technologique Liliane Bettencourt at the Collège de France in Paris. His current project, Actanthrope (ERC–ADG 340050), is devoted to the computational foundations of anthropomorphic action.

Paolo Salaris is currently a permanent junior researcher (CR2) at INRIA–Sophia Antipolis, France. He received the ‘Laurea’ in Electrical Engineering in 2007 and his doctoral degree in Robotics, Automation and Bioengineering, in 2011 at the E. Piaggio Research Center of the University of Pisa. He was a visiting scholar at the Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana-Champaign in 2009. He was a post-doctoral fellow at the E. Piaggio Research Center from January 2011 to January 2014, and spent February 2014 to July 2015 at LAAS–CNRS in Toulouse. His main research interests within robotics are in optimal motion-planning and control, control for nonholonomic systems, visual servoing and motion generation for humanoid robots.

Florent Levillain is a researcher in cognitive psychology. During the preparation of his doctoral thesis, he studied mental imagery and the understanding of dynamic scenes, conducting research both at the International School for Advanced Studies (Trieste, Italy) and at the Paris 8–Saint Denis University. After a postdoctoral fellowship at Johns Hopkins University, where he examined the limitations of short-term memory as well as the perception of causal relationships, he resumed an investigation into the interpretation of action. What are the motion cues that enable an event to be perceived? Which cognitive systems contribute to the spontaneous understanding of behavior in terms of intentional actions? These questions arose in the context of a collaboration with the Diip Ensadlab at the École Nationale Supérieure des Arts Décoratifs in Paris.

References

Adamson

(2004) Calaban LT: Computer-aided-labanotation: A guide to producing labanotated dance scores using computer-aided design. USA: AJ Adamson.

Aristidou

Chrysanthou

(2014) Feature extraction for human motion indexing of acted dance performances. In: Proceedings of the 9th International Conference on Computer Graphics Theory and Applications (GRAPP 2014), Lisbon, 5–8 January, 277–287.

Aristidou

Stavrakis

Charalambous

Chrysanthou

. (2015) Folk dance evaluation using Laban Movement Analysis. ACM Journal on Computing and Cultural Heritage 8(4): 20:1–20:19.

Badler

(1979) Digital representations of human movement. Computing Surveys 11(1): 19–38.

Berthoz

(2008) Le sens du mouvement. Paris: Odile Jacob.

Blanchi

J-P

(2000) Biomécanique du mouvement et APS. Paris: Vigot.

Bourdieu

(1980) Le sens pratique. Paris: Minuit.

Bril

Goasdoué

(2009) Du mouvement sans sens ou du sens sans mouvement: Rôle des finalités et des contextes dans l’étude de comportements moteurs. Intellectica 51(1): 273–293.

Brown

(1976) A graphic editor for Labanotation, Master’s thesis, University of Pennsylvania.

10.

Brown

Smoliar

Weber

(1978) Preparing dance notation scores with a computer. Computer & Graphics 3(1): 1–7.

11.

Burton

Samadani

A-A

Gorbet

Kulic

(2016) Laban movement analysis and affective movement generation for robots and other near-living creatures. In: Laumond

Abe

(eds.) Dance Notations and Robots Motion. Cham, Heidelberg, New York, Dordrecht, London: Springer. 25–48.

12.

Calvert

(1986) Toward a language for human movement. Computers and the Humanities 20(2): 35–43.

13.

Chen

Miao

Zhu

Zhang

. (2013) Generating Labanotation from motion-capture data. International Conference on Culture and Computing (Culture Computing), Kyoto, 16–18 September, 222–223. Available at: http://ieeexplore.ieee.org/abstract/document/6680389/?reload=true&section=abstract.

14.

Chen

Qian

James

(2005) An autonomous dance scoring system using marker-based motion capture. IEEE 7th Workshop on Multimedia Signal Processing, Shanghai, 30 October–2 November 2, 1–4. Available at: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4014087&isnumber=4013942.

15.

Chen

J-F

Lin

W-C

Tsai

K-H

Dai

S-Y

(2011) Analysis and evaluation of human movement based on Laban Movement Analysis. Tamkang Journal of Science and Engineering 14(3): 255–264.

16.

Chi

Costa

Zhao

Badler

(2000) The EMOTE model for effort and shape. In: Proceedings of the 27th annual conference on Computer Graphics and Interactive Techniques SIGGRAPH ‘00, 23–28 July, New Orleans, 173–182. Available at: http://repository.upenn.edu/cgi/viewcontent.cgi?article=1044&context=hms.

17.

Choensawat

Nakamura

Hachimura

(2016) Applications for recording and generating human body motion with Labanotation. In: Laumond

Abe

(eds) Dance Notations and Robot Motion. Cham, Heidelberg, New York, Dordrecht, London: Springer, 391–416.

18.

Falzon

(2004) Ergonomie. Paris: Presses Universitaires de France.

19.

Fugedi

(2011) LabanGraph 4P – An(other) computer editor for Labanotation. In: 27th International Council of Kinetography Laban/Labanotation. 31 July–7 August, Budpest, 326–331.

20.

Gabor

(2014) Labanatory. Available at: http://www.labanatory.com/.

21.

Griesbeck

(1996) Introduction to Labanotation. Available at: http://user.uni-frankfurt.de/~griesbec/LABANE.HTML.

22.

Hachimura

Nakamura

(2001) Method of generating coded description of human body motion from motion-captured data. In: 10th IEEE International Workshop on Robot and Human Interactive Communication. 18– 21 September, Bordeaux and Paris. Available at: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=981889&isnumber=21147.

23.

Hunt

Politis

Herbison-Evans

DH-E

(1989) LED: An interactive graphical editor for labanotation. Technical Report 343, University of Sydney, Basser Departement of Computer Sciences.

24.

Hutchinson Gest

(1984) Dance Notation. The process of recording movement on paper. New York: Dance Horizons.

25.

Hutchinson Gest

(1989) Choreo-Graphics, A comparison of dance notation systems from the fifteenth century to the present. New York, NY: Gordon and Breach Science Publishers.

26.

Kajita

Hirukawa

Harada

Yokoi

(2009). Introduction à la commande des robots humanoïdes: De la modélisation à la génération du mouvement. Paris: Springer-Verlag.

27.

Kendon

(1981) Nonverbal Communication, Interaction and Gesture. The Hague: Mouton.

28.

Knust

(2011) Dictionnaire usuel de Cinétographie Laban (Labanotation). Villers-Cotterets: Ressouvenances.

29.

Loureiro de Souza

(2016) Laban Movement Analysis – Scaffolding human movement to multiply possibilitie and choices. In: Laumond

Abe

(eds) Dance Notations and Robot Motion. Cham, Heidelberg, New York, Dordrecht, London: Springer, 283–297.

30.

Lourens

Berkel

von Barakova

(2010) Communicating emotions and mental states to robots in a real time parallel framework using Laban Movement Analysis. Robotics and Autonomous Systems 58(12): 1256–1265.

31.

Mansard

Chaumette

(2007) Task sequencing for high-level sensor-based control. IEEE Transactions on Robotics 23(1): 60–72.

32.

Masuda

Kato

Itoh

(2009) Emotion detection from body motion of human form robot based on Laban Movement Analysis. Principles of Practice in Multi-Agent Systems. LNCS 5925: 322–334.

33.

Mauss

(1934) Les techniques du corps. Journal de Psychologie, 32(3–4 (Communication présentée à la Société de Psychologie le 17 mai 1934). Also in (1950) Sociologie et anthropologie. Paris: Presses Universitaires de France, 363–386.

34.

Okajima

Wakayama

Okada

(2012) Human motion retrieval system based on LMA features using interactive evolutionary computation method. In: Watanabe

Jain

(eds) Innovations in Intelligent Machines – 2. Berlin, Heidelberg: Springer Verlag, 117–130.

35.

Rett

Santos

Dias

(2008) Laban Movement Analysis for multi-ocular systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. 22–26 September, Nice, 761–766. Available at: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4650717&isnumber=4650570.

36.

Salaris

Abe

Laumond

J-P

(2016) A worked-out experience in programming humanoid robots via the Kinetography Laban. In: Laumond

Abe

(eds) Dance Notations and Robot Motion. Cham, Heidelberg, New York, Dordrecht, London: Springer, 339–359.

37.

Santos

Dias

(2010) Laban Movement Analysis towards behavior patterns. Emerging trends in technological innovation. In: Camarinha-Matos

Pereira

Ribeiro

(eds) Emerging Trends in Technological Innovation. First IFIP WG 5.5/SOCOLNET Doctoral Conference on Computing, Electrical and Industrial Systems, DoCEIS 2010m Costa de Caparica, Portugal, February 2010, Proceedings. Advances in Information and Communication Technology, vol 314. Berlin, Heidelberg: Springer, 187–194.

38.

Shen

Geng

. (2005) Mocap data editing via movement notations. CAD/Graphics 6: 463–470.

39.

Smoliar

(1978) A lexical analysis of Labanotation with an associated data structure. ACM ‘78. Proceedings of the 1978 annual conference 2: 727–730.

40.

Torresani

Hackney

Bregler

(2006) Learning motion style synthesis from perceptual observations. Proceedings of NIPS’06, 1393–1400. Available at: https://papers.nips.cc/paper/3109-learning-motion-style-synthesis-from-perceptual-observations.pdf.

41.

Wakayama

Okajima

Takano

Okada

(2010) IEC-based motion retrieval system using Laban Movement Analysis. In: Knowledge-Based and Intelligent Information and Engineering Systems, 14th International Conference, KES 2010, Cardiff, 8–10 September, Proceedings, Part IV 6279: 251–260. Available at: https://link-springer-com.web.bisu.edu.cn/chapter/10.1007/978–3–642–15384–6_27.

42.

Wilke

Calvert

Ryman

Fox

(2005) From dance notation to human animation: The LabanDancer project. Computer Animation and Virtual Worlds 16(3–4): 201–211.

43.

Zalla

(1974) Computer Interpretation of Selected Labanotation Commands. Master’s thesis, Simon Fraser University.

44.

Zhao

Badler

(2004) Acquiring and validating motion qualities from live limb gestures. Graphical Models 64(1): 1–16.