Hand Posture Prediction Using Neural Networks within a Biomechanical Model

Abstract

This paper proposes the use of artificial neural networks (ANNs) in the framework of a biomechanical hand model for grasping. ANNs enhance the model capabilities as they substitute estimated data for the experimental inputs required by the grasping algorithm used. These inputs are the tentative grasping posture and the most open posture during grasping. As a consequence, more realistic grasping postures are predicted by the grasping algorithm, along with the contact information required by the dynamic biomechanical model (contact points and normals). Several neural network architectures are tested and compared in terms of prediction errors, leading to encouraging results. The performance of the overall proposal is also shown through simulation, where a grasping experiment is replicated and compared to the real grasping data collected by a data glove device.

Keywords

Grasp human hand artificial neural networks biomechanical model robotic hand

1. Introduction

Most human mechanical interactions with the environment are performed by the hands. Very different tasks can be carried out with them because of its complex kinematics, with more than 20 degrees of freedom (dof) controlled by muscles, tendons and ligaments. The study of the human grasp is very interesting because of the knowledge it can provide regarding human/robot grasping and manipulation [1].

Over the past two decades, a lot of research has been carried out in the field of robotic grasping [1 -4] and it is currently a very hot topic. Several of the existing techniques can be extended to the field of biomechanics due to the similarities between human and robotic hands, although the former are kinematically simpler than the latter. In this sense, many researchers state that advances in the field of robot grasping and manipulation require a better knowledge of human grasping [5]. This is especially true in the field of service robotics, where robots move in human environments and interact with daily-life objects [6]. In fact, humanoid robots tend to imitate the morphology and operation of the human body in terms of motion and actuation. Thus, developments in the biomechanical field can shine a light on developments in the robotics field. A promising research area lies ahead, with scientists aiming to obtain a more powerful biomechanical model of the hand, integrating knowledge and developments from the fields of biomechanics and robotics.

Hand biomechanical models describe the hand as a mechanical device: the different elements are defined as rigid bodies, joints and actuators, and the mechanical laws are applied. These models are used to perform analyses which cannot be done directly on humans or which have unfeasible experimental costs such as, for instance, the study of new alternatives for restoring hand pathologies. To be useful, hand biomechanical models should be able to properly simulate the hand while grasping. However, existing models still present important limitations on this matter which could be addressed using the current techniques and knowledge of robot grasping.

The latest hand biomechanical models were developed for very different and specific purposes [7 -23], namely to understand the role of anatomical elements, to study the causes and effects of pathologies, to plan rehabilitation, to simulate surgeries, to analyse the energetics of human movement and athletic performance, to design prosthetics and biomedical implants, and to design functional electric stimulation controllers, to name a few. To simulate a task, these models need the hand posture, the forces on the hand and their application points as inputs, and they allow the estimation of the muscle forces needed to perform the task. Such input data have to be experimentally registered, which significantly reduces the applicability of the models. Therefore, incorporating the prediction of grasping postures and contact information with the grasped object will increase their utility.

Hand posture prediction is an important issue as it allows the evaluation of biomechanical and ergonomic parameters related to grasping [4, 24]. The computation of grasping postures associated with particular objects is a very challenging topic, as it implies the fulfilment of a large number of constraints that relate not only to hand structure and the object, but also to the requirements of the environment and the task to be performed.

The biomechanical studies in the literature usually measure hand postures using specific devices – such as data gloves [25] – rather than predicting hand postures. Very few biomechanical works have tackled the prediction of grasping postures for given objects and tasks. Artificial intelligence techniques were used in [26] for learning a hand's inverse kinematics given the fingertips' positions and in [2] for estimating the hand shape needed to grasp various shaped objects in a virtual environment, with encouraging results. In [26], a grasping algorithm was proposed to automatically generate a natural grasping motion path of the hand model – from a fully opened state to a clenched one -obtained from the collision detection between the fingers and the object when the finger joints are rotated. Nevertheless, the posture predictions were highly affected by the rotation rates of the finger joints [28]. In [24], an optimization model was proposed based on the assumption that the best prehensile configuration of the hand in a power grasp optimally conforms to the shape of the object, which is not generalizable for other grasps.

From the robotics' viewpoint, the questions posed by grasping are slightly different to those of human grasping. For a given object, grasp synthesis must provide the most appropriate set of contact points and hand posture for grasping the object, with regard to object stability and manipulability. This problem has been studied using analytical approaches that had nearly been ruled out due to their poor results when it came to real implementations [29]. Currently, artificial intelligence algorithms are used, such as those based on artificial neural networks (ANNs) [30]. These sensor-based approaches learn the underlying rules of robot grasping without the need of explicit kinematic models by means of exploration. For instance, an ANN-based strategy was developed in [31] for a 5-dof gripper. Similarly, a grasping model was designed in [32] for the grasp synthesis of a 7-dof planar hand for circular and rectangular objects. ANN methods have also been used in feasible contact point determination [2, 32] and in order to guide robot grasp synthesis, as in [2] where a data glove was used for training a neural network that produced robot grasping postures from previously computed contact zones for different objects.

Collision detection is also required in robotics in order to address robot manipulation, and may also be required to tackle human hand grasping simulation. Many different collision detection algorithms have been used in the past in the robotics literature [33, 34]. Recently, in [35], it was shown that modelling the geometry of the robot and the grasped object using the spherical extension of polytopes (s-topes) allows fast and efficient collision detection. Collision detection was carried out by calculating the minimum distance between s-topes, based on the Gilbert– Johnson–Keerthi algorithm [36].

Along these lines, the present work proposes the use of ANNs in combination with a grasping algorithm to predict the grasping posture and contact information required by an already validated 3D scalable biomechanical model of the human hand developed by the authors in previous works [14 -17]. ANNs enhance the model capabilities, as they provide estimated data that substitutes for the experimental input required by the grasping algorithm. The paper is structured as follows: Section 2 explains the complete grasping model proposal, detailing the kinematic hand model used as well as the grasping algorithm; Section 3 addresses the different ANNs which were tested for posture prediction, along with the experiments carried out; Section 4 discusses the results obtained and Section 5 states the conclusions and future work.

2. Model Proposal

In [37], we presented a proposal for a self-contained and realistic hand biomechanical model. In particular, a method for incorporating the grasping posture prediction into an existing 3D biomechanical model of the hand was sketched. This proposal is developed in detail in the present work by means of incorporating ANNs in the generation of the input data required for the dynamic biomechanical model of the hand (Figure 1). The requirements for the grasping posture prediction model in the self-contained and realistic hand model are:

Figure 1.

Block diagram of the model proposal.

The model has to simulate the complete hand in order to allow the study of any grasp.

The model has to be scalable so as to allow the simulation of different population groups.

The model has to simulate and show the grasping of an object in a realistic way.

The model has to be able to predict feasible grasping postures for a given object and provide the contact information required for evaluating the grasp.

These requirements imply the availability of a geometric model of the hand and the object to be grasped, and a kinematic model for movement. In addition, a grasping posture algorithm must be included to automatically predict the hand posture during grasping, as well as the contact points and normals, required by the dynamic biomechanical model.

The model proposed in this section has been developed in a scalable way, choosing two very well known anthropometric parameters of the hand that are easy to measure and representative of the hand's size. The parameters are the hand length (HL) and hand breadth (HB), as shown in Figure 2.

Figure 2.

Parameters used to scale the model: HL (hand length) and HB (hand breadth).

The grasping posture algorithm (Figure 1) is based on the calculation of appropriate rotation rates for the joint angles from the use of two characteristic hand postures obtained by ANNs: the most open posture (MOP) and the tentative grasping posture (TGP). Different feed-forward networks have been tested for automatically providing these characteristic hand postures given the size of the hand, the features of the object to be grasped and the task to be performed. The outputs of each ANN are the fingers' joint angles for each characteristic posture. The ANNs have been trained with data collected from grasping experiments with daily-life objects (bottles) and tasks (moving and pouring). Once trained, the network is able to predict the MOP when the hand approaches the object to grasp it, and the TGP for objects and subjects different to those used in the training phase with an acceptable error. Both postures are then used by the grasping posture algorithm to generate the angle rotation rates for all hand joints. Hand segments are rotated until collision with the object is detected, which allows the prediction of an improved grasping posture, the contact points and normals, required as inputs of the biomechanical model. Collision detection is efficiently performed by the use of spherical models to represent the object and the hand. Finally, we can then compute muscle forces, contact forces and torques needed for the biomechanical and ergonomic evaluation of grasping postures through the biomechanical 3D model.

2.1 Kinematic Model

Care has been taken in the selection of the appropriate DOF among the different hand bones in order to allow the simulation of realistic grasping postures. The hand has been considered as five skeletal open chains of rigid bodies connected to the carpus through different joints which characterize the kinematic behaviour of the chains (Figures 3 and 4).

Figure 3.

Joints of the hand.

Figure 4.

Movements of the hand fingers.

The proximal and distal interphalangeal joints (PIPs and DIPs) of the fingers and the interphalangeal joint (IP) of the thumb are of the trochlear type. They allow only flexion-extension movements, which are basically a rotation of the distal bone about an axis fixed to the proximal bone [38]. Therefore, we have modelled them as hinge joints.

All the metacarpophalangeal joints (MCPs) are of the condylar type allowing both flexion-extension and abduction-adduction movements, whereby they correspond to a rotation of the distal bone about a flexion-extension axis that is fixed with respect to the proximal bone and a rotation about an abduction-adduction axis that is fixed with respect to the distal bone [38]. The carpometacarpal joint (CMC) of the thumb is a saddle joint, resulting in similar kinematic behaviour to that of the MCPs [38]. All these joints have been modelled as universal joints.

Finally, the model considers the little and ring CMC joints, which are of the arthrodial type. They allow a very limited movement range [39], and we have chosen to model them as hinge joints.

The data for the axes location and orientation were obtained from [40 -42]. This data, along with the segment lengths' data, was appropriately scaled with respect to the parameters HB and HL [16]. The Denavit-Hartenberg method – from the robotics field [42] – was adapted to define the position of any segment point of interest.

2.2 Grasping Posture Algorithm

When trying to simulate the grasping of an object with a hand biomechanical model, the grasping postures predicted with ANNs do not exactly match the grasped object – becoming a non-conforming grasp – so that contact information (contact points and normals) cannot be obtained. To avoid this, we propose to use the predictions of postures using ANNs within a grasping algorithm based upon that of Choi [27]. This algorithm uses a function to automatically generate a natural grasping path of the hand model from a fully opened state to a clenched one. The goal is to find contacts between the surface hand skin and the object surface while rotating the joint angles of the fingers. It is very important to choose appropriate rotation rates for the finger joints, as they affect the final posture prediction [44]. We have solved this difficulty by using rotation rates that try to match those that are experimentally observed. The rotation rates are defined by the difference between the angles of the most open posture (MOP) observed when approaching the hand to grasp the object, and the clenched posture once the grasp is performed, which will be used as a tentative grasping posture (TGP). These postures are not experimentally measured, but obtained as predictions with ANNs, as described in the next section.

A geometric model of the hand and a contact model are required to generate the grasp because, at each rotation step, contact has to be checked between the surface skin model and the surface of the object model. In real grasping, the surface of a hand deforms in a nonlinear way when making contact with the object. To avoid long execution times, we considered a geometric collision-detection algorithm. The hand segments are considered as rigid bodies and their deformation is simulated, allowing the penetration of the surface skin model and the object model. This penetration is limited by a tolerance that relates to the hand stiffness at each contact region. A maximum penetration of 3 mm has been considered for all hand segments as a first approximation.

The distances between the points on the skin surface and the object are calculated while each joint rotates according to the specific joint rotation algorithm. When this distance reaches the given maximum penetration tolerance for a given segment, the contact of that segment is achieved and the joint rotation ends together with the rotation of all the proximal joints. When the distal segments of all four fingers make contact with the object, the grasping simulation terminates.

In order to perform these calculations in an efficient way, the geometry of the hand surface and the grasped object have been modelled using the spherical extension of polytopes (s-topes). This graphical representation has been successfully used previously in robotics [35], allowing for fast and efficient collision detection between the grasping hand and the grasped object while displaying a sufficient level of realism (Figure 5). The collision detection is performed by calculating the minimum distance between s-topes, based on the Gilbert-Johnson-Keerthi algorithm [36]. The algorithm also calculates the minimum distance points that define the normal direction to the contact surface.

Figure 5.

External geometrical representation of the hand with s-topes.

3. Model Development and Assessment

The previously described grasping algorithm requires the knowledge postures MOP and TGP in order to compute a feasible grasping posture, along with the contact points and normals. Usually, these postures are experimentally measured by means of data gloves or motion capture systems. However, the experiments are tedious and time-consuming as they have to be carried out for every subject, task and object. For this reason, there are no hand valid models for the study of general grasping. In order to overcome such a drawback and to automatically generate the characteristic postures MOP and TGP for a set of daily-life objects (bottles), we have used ANNs.

3.1 Neural Networks for Characteristic Posture Prediction

An ANN can be described as a large number of simple processing elements – neurons – running in parallel. The system performance depends on the characteristics of the network: the architecture (the number of layers and neurons), the number of synaptic connections and the processing function for every neuron (the activation function). Experimental knowledge is stored in the strength of the neuron connections – known as synaptic weights – in a manner similar to the human brain. The method used to iteratively compute these weights as the network receives new information is the training algorithm. It is applied to train the network in learning a particular task. If the inputs and targets (the desired outputs) are known, supervised learning takes place. Otherwise, if only the inputs are known, an unsupervised learning process is carried out.

There are three typical problems that are usually addressed by means of ANNs: the approximation of functions, pattern recognition and clustering [45, 46]. Multiple-layer networks are quite powerful for the approximation of functions, which is the subject of this specific work. Kolmogorov [47] demonstrated that any continuous function can be computed with three layers of neurons between the inputs and outputs, given enough neurons in each layer. Moreover, the connectivity plays an important role in successfully solving the problem (see [45, 46] for detailed information).

In the present work, the design of the different ANNs was inspired by the works presented in [2] and [26]. In particular, we have chosen a single neural network in order to predict the joint angles of the hand, as in [2]. Most of the existing biomechanical works [4, 30, 32] use different networks for the different fingers of the hand, due to the better performance of smaller neural networks. However, the coupling between fingers could hardly be predicted by means of independent networks, but should instead be predicted through a single network for the whole hand.

Various ANN architectures have been tested with data provided by different grasping experiments. Both topics are thoroughly described in the following subsections.

3.1.1 Data Acquisition Device and Preliminary Study

The design of a neural network requires an in-depth study of the process to be modelled. The more knowledge we have about the process, the better we can find a proper neural network to model its behaviour.

The authors have previous experience in human grasping and have carried out several studies in the field of biomechanics [48 -53]. These works have focused on the experimental characterization of common grasps in daily life, including not only grasping postures but also the forces produced by the different hand segments. These studies have been performed from data collected by a Cyberglove™ device, among others.

The Cyberglove™ system provides the main joint angles of the hand with an acquisition rate of 15Hz. Figure 6 displays an image of the glove with the abbreviations used for the joint angles studied in this work: the thumb (ROLL, ABD1–2, MCP1, IP1), index finger (ABD2–3, MCP2, PIP2), medium finger (MCP3, PIP3), ring finger (ABD3–4, MCP4, PIP4) and little finger joint angles (ABD4–5, MCP5, PIP5). This glove has been proven valid for the measurement of hand postures during grasping tasks, ranging its sensors' repeatability errors from 1.2° to 5°. In particular, in [49] a quantitative study was carried out to determine the influence of different parameters for the grasping of several bottles of different sizes, weights and materials. It was demonstrated that the hand posture was repeatable for the same subject, bottle, filling level and task. Another remarkable result was that the weight did not have a significant influence on the grasping posture.

Figure 6.

Cyberglove joint sensors.

3.1.2 Set of Experiments

The data from the experiments developed in the previous work [49] was used. They consisted of sixteen grasping experiments with the Cyberglove™ device over 4 bottles of different sizes, weights and materials (Figure 7). Six right-handed subjects with different anthropometric characteristics were selected for the experiments, which consisted in 2 grasping tasks with each bottle and 2 different filling levels, all of them carried out randomly within the same session (4 bottles x 2 tasks x 2 filling levels). The tasks were: to move the bottle to a different position and to pour the bottle's contents into a recipient vessel. Each experiment implied 5 repetitions of the same task, bottle and weight, carried out after a rehearsal session (5 identical tests). The subjects were asked to perform the tasks in a natural way. Table 1 shows the bottles' features: the bottle number, material, height (h), diameter at the contact area (d), empty weight (W₀), and weights at filling levels 1 (W₁) and 2 (W₂).

Table 1.

Features of the bottles employed in the experiments.

Bottle	Material	h(m)	d(m)	W₀(kg)	W₁(kg)	W₂(kg)
1	glass	0.300	0.08	0.5235	0.550	1.00
2	plastic	0.350	0.08	0.0490	0.150	0.55
3	plastic	0.245	0.075	0.0445	0.150	0.55
4	Plastic	0.222	0.065	0.0285	0.150	0.55

Figure 7.

Bottles B1, B2, B3 and B4 (left to right).

Table 2 shows the subjects' data, including HB and HL.

Table 2.

Features of the subjects participating in the experiments.

Subject	Gender	Age	Height(m)	Weight (kg)	HL (m)	HB (m)
1	Male	47	1.74	82	0.186	0.088
2	Male	38	1.85	78	0.202	0.081
3	Female	28	1.64	69	0.165	0.077
4	Male	40	1.73	84	0.190	0.086
5	Male	25	1.77	76	0.195	0.085
6	Female	35	1.71	60	0.180	0.080

Figure 8 shows an image of one of the 16 experiments. Each experiment implied 5 new input-target sequences for the ANN, yielding a set of 480 data sequences for training and test purposes.

Figure 8.

Experiment example: pouring water into a glass.

A statistical analysis over the output data standard deviations (SDs) was performed. For each subject grasping each bottle, the SD was computed at each joint angle. The overall SD values of each joint angle were computed in order to compare the ANNs' errors with the deviations of the real data.

3.1.3 Neural Network Design

In this case, the inputs and targets were available from grasping experiments. Therefore, supervised learning was the best option for learning the nonlinear relationships between them. In supervised learning, the optimum weights are obtained by an optimization technique for data not used in the training phase. This is the test phase.

In order to decide the inputs to the neural network, the results from the preliminary grasping study were considered. As the weight was not a significant parameter [49], it was not taken into account as an input to the neural network. Therefore, the input variables were: the hand parameters HL and HB, the object size (diameter in the case of bottles) and the task to be performed. The outputs of the neural network were the joint angles for the characteristic hand postures during the grasping process: the MOP and the TGP. A different neural network was used for predicting each posture.

Accounting for the nature of our problem, we decided to use a multi-layer fully-connected feed-forward network, which was able to learn complex relationships. The selected training algorithm was the backpropagation technique [45, 46], which is the generalization of the Least Mean Squares algorithm for multiple-layer networks. It minimizes the mean square error (MSE) between the network outputs and the targets, where a set of examples of proper network behaviour are provided for learning.

We developed, trained and tested ANNs for the TGP and the MOP with different architectures. The number of hidden layers ranged from one to three. Different numbers of neurons were tested for each layer, as well as different activation functions commonly used in the literature (linear and sigmoid [45, 46]). Over each ANN, two tests were performed:

Test B2: The training set was composed of data from all the subjects and tasks, except for bottle 2. The test set was composed of the data not used for training (from all subjects and tasks, but only from bottle 2).

Test S4: The training set was composed of data from all the bottles and tasks, except for subject 4. The test set was composed of the remaining data (from all bottles and tasks but only from subject 4).

The ANNs performances were computed based on the prediction results of the abovementioned tests, in terms of the mean RMS (root mean square) errors and standard deviation SD of the RMS errors for each joint angle of the TGP and MOP.

3.2 Posture Prediction Assessment

The global model proposed for the prediction of grasping postures and contact information was assessed through the simulation of the two tasks of moving a bottle and pouring its contents, using the grasping posture algorithm. The simulations were performed with the prediction results of the tasks for subject 4 and bottle 2, and consisted in using the MOP and the TGP estimated by the ANNs (the selected ones among the ANNs developed after their assessment) as input for the grasping posture algorithm.

The errors in the joint angles were calculated as the difference between the predictions and the measured values. The RMS errors and standard deviations of these errors for the whole hand were used to globally validate the technique, and the RMS errors and standard deviation of these errors per finger were used for a detailed analysis of the distribution of the errors and their relative importance in the subsequent analysis of dynamic grasping with the hand biomechanical model.

The visual representation of the input postures and the predicted grasping posture was also made in order to visually corroborate the validity of the model.

4. Results and Discussion

Before assessing whether the estimations made by the grasping algorithm are good enough, we must take into account the variability in the grasping postures when performed by the same subject several times. Less precision must be demanded of joints with large

variability. Table 3 shows the joint angles' variability whereby the higher values are found in IP1, MCP5 and PIP5. The little finger posture is the least repetitive. Moreover, the role of this finger in grasping is minor compared with the other fingers.

Table 3.

Mean standard deviation (SD) values for the joint angles of the experimental grasping postures (variability).

Joint Angle	Mean SD (°)
ROLL	1.49
MCP1	2.27
IP1	4.57
ABD1–2	3.09
MCP2	2.92
PIP2	3.05
MCP3	2.91
IFP3	3.04
ABD2–3	1.22
MCP4	3.35
PIP4	3.69
ABD3–5	2.54
MCP5	7.87
PIP5	5.73
ABD4–5	2.22

4.1 Selection and Performance of the Neural Network

Tables 4 and 5 summarize the best ANN performances from the different tested architectures. These values are given in terms of the mean RMS errors of the test phase and their associated standard deviations (SD) for the joint angles' prediction for the TGP and MOP. It can be seen that in Test B2 the minimum mean error is attached to ANN7, a four-layer feed-forward network. However, the results are very close to those from ANN1, a two-layer feed-forward network. The high computational cost involved in the use of a four-layer ANN did not compensate for the slightly better results of ANN7. In Test S4, ANN1 clearly displays the best performance for both the TGP and MOP. Nevertheless, this test has considerably higher errors than Test B2, which suggests that we may lack experimental data regarding the subjects for proper generalization. Therefore, the selected architecture was a two layer feed-forward neural network with 100 neurons in the hidden layer, for the prediction of both TGP and MOP postures.

Table 4.

ANNs' performance for the TGP and MOP in Test B2.

ANN ID	Hidden Layers	Neurons per layer	Activ. function	Mean RMS error TGP(°)	SD TGP (°)	Mean RMS error MOPO	SD MOP (°)
1	1	100	Sigmoid	8.64	4.07	4.63	2.03
2	2	75;50	Sigmoid	8.84	3.89	8.01	3.38
3	2	100;75	Sigmoid	8.70	3.87	7.51	3.27
4	2	150;100	Sigmoid	8.64	3.80	4.72	2.27
5	3	35;35;35	Sigmoid	8.71	3.67	4.82	2.00
6	3	100;75;50	Sigmoid	8.80	3.83	4.85	2.20
7	3	75;100;150	Sigmoid	8.50	3.93	5.02	2.27
8	3	150;150;150	Sigmoid	8.71	3.97	4.89	2.43

Table 5.

ANNs' performance for the TGP and MOP in Test S4.

ANN ID	Hidden Layers	Neurons per layer	Activ. function	Mean RMS error TGP(°)	SD TGP (°)	Mean RMS error MOPO	SD MOP (°)
1	1	100	Sigmoid	12.76	7.93	14.51	12.36
2	2	75;50	Sigmoid	16.94	6.79	20.03	13.45
3	2	100;75	Sigmoid	15.25	6.75	19.21	13.23
4	2	150;100	Sigmoid	15.12	6.70	14.95	14.35
5	3	35;35;35	Sigmoid	14.93	9.01	16.12	12.49
6	3	100;75;50	Sigmoid	15.01	9.25	17.22	12.14
7	3	75;100;150	Sigmoid	14.88	8.93	16.07	12.32
8	3	150;150;150	Sigmoid	14.97	9.45	17.73	12.67

To illustrate the performance of the ANN1, the detailed results for each joint angle prediction are shown in Tables 6 -8 for both the moving and pouring tasks carried out by subject 4 with bottle 2.

Table 6.

ANN1 Prediction results (°) for task MOVE.

MOVE	TGP			MOP
Joint Angle	Mean Exp. Value	ANN Prediction	Error	Mean Exp. Value	ANN Prediction	Error
ROLL	96.46	96.94	0.48	93.62	93.66	0.05
MCP1	14.18	3.91	10.27	1.93	−0.01	1.94
IP1	30.45	38.98	8.53	−19.40	−13.15	6.25
ABD1–2	63.36	66.74	3.38	20.77	14.34	6.43
MCP2	32.28	32.53	0.25	2.04	3.99	1.95
PIP2	20.55	22.61	2.06	−12.94	−11.13	1.80
MCP3	32.28	32.53	0.24	2.04	4.01	1.98
IFP3	20.55	22.61	2.06	−12.94	−11.18	1.76
ABD2–3	11.83	13.77	1.94	2.70	2.77	0.07
MCP4	31.11	23.55	7.56	5.19	5.38	0.19
PIP4	11.99	23.34	11.34	−17.08	−14.95	2.13
ABD3–5	13.89	20.08	6.19	4.35	4.76	0.41
MCP5	29.94	14.55	15.40	8.34	6.76	1.58
PIP5	16.05	16.05	0.00	−9.96	−8.09	1.87
ABD4–5	8.62	17.37	8.74	4.78	6.09	1.31

Table 7.

ANN1 Prediction results (°) for task POUR.

POUR	TGP			MOP
Joint Angle	Mean Exp. Value	ANN Prediction	Error	Mean Exp. Value	ANN Prediction	Error
ROLL	96.08	95.68	0.40	94.10	93.79	0.31
MCP1	18.02	9.24	8.78	2.67	4.36	1.69
IP1	25.09	36.19	11.09	−5.65	−8.65	3.01
ABD1–2	64.23	63.88	0.36	26.71	20.69	6.02
MCP2	23.93	32.71	8.79	9.02	5.66	3.36
PIP2	10.70	26.06	15.36	−10.18	−7.35	2.83
MCP3	23.93	32.72	8.79	9.02	5.62	3.40
IFP3	10.70	26.02	15.32	−10.18	−7.32	2.86
ABD2–3	26.32	13.50	12.82	2.57	2.36	0.20
MCP4	32.66	32.83	0.18	9.15	7.50	1.65
PIP4	1.24	26.84	25.60	−16.30	−11.17	5.13
ABD3–5	15.65	14.57	1.08	3.58	3.74	0.16
MCP5	41.38	32.91	8.47	9.29	9.36	0.08
PIP5	2.88	11.78	8.90	−7.20	−6.35	0.85
ABD4–5	6.59	9.79	3.19	3.33	3.12	0.22

Table 8.

Summary of the ANN1 prediction results when performing tasks MOVE and POUR for subject 4 and bottle 2.

	TGP		MOP
Task	RMS Error (°)	SD(°)	RMS Error (°)	SD(°)
Move	7.02	4.85	2.72	1.92
Pour	10.98	7.05	2.80	1.89

We can observe that the MOP predictions are very accurate, with errors lower than 3°. The TGP predictions present higher errors, although they are lower than 11°. Taking into account the real variability in the repetition of the postures as experimentally observed (Table 3), which reaches 8° in some joints, the errors obtained by the neural network are reasonable. The maximum errors for the TGP prediction for the task MOVE occur in the MCP5 joint, which is the joint with a higher repeatability error. The errors for the task POUR are larger than those for the task MOVE, which may be due to the difficulty involved in the selection of the representative instant of the task, since the hand does not hold the bottle in a static posture but rather it changes its grasping posture for the bottle's manipulation.

4.2 Performance of the Grasping Algorithm

The TGP and MOP predictions of the two-layer feed-forward network for subject 4 and bottle 2 were introduced in the grasping algorithm for the assessment of the global model in the realization of the two tasks: MOVE and POUR. This model produced a final grasping posture and contact through the simulation of both tasks. Table 9 shows the RMS error, the standard deviation and the maximum error of the joint angles for the final grasping posture's computation. Table 10 displays the RMS errors for every finger obtained from the simulations of both tasks, as well as the global mean RMS error per finger. It is important to note that these values are satisfactory when taking into account the joint angles' variability mentioned above (Table 3).

Table 9.

RMS errors and SDs of the grasping algorithm in the computation of the grasping posture from the MOP and TGP obtained predictions.

Subject	Task	RMS Error (°)	SD(°)	Maximum Error (°)
4	Move	11.43	7.02	22.91
4	Pour	13.23	7.49	25.60

Table 10.

RMS errors and SDs of the grasping algorithm described for each finger in the computation of the grasping posture from the MOP and GP predictions.

Subject	Task	Thumb	Index	Middle	Ring	Little
4	Move	9.44	2.53	8.68	14.17	15.79
4	Pour	12.59	12.61	17.03	14.81	8.65
Mean RMS Error		11.02	7.57	12.39	10.49	10.66

Figure 9a-b displays the MOP and TGP prediction results provided by the ANNs. Note that the phalanxes of the fingers in the TGP do not make contact with the object (Figure 9b), so it is not possible to use this posture to directly obtain the contact information required for a dynamic analysis of the grasp. Figure 9c-d shows the grasping posture obtained from the grasping algorithm. Here, the hand has finally contacted the object and is able to grasp it in a realistic way. In addition, the muscle forces, contact forces and torques could be computed by means of the dynamic biomechanical model.

Figure 9.

Simulation results of the grasping algorithm for the task MOVE of Test B2: a) MOP, b) TGP, c) and d) different views of the grasping posture.

5. Conclusions and Future Work

This paper has described a valid methodology for the computation of grasping posture by introducing artificial neural networks in a biomechanical grasping model. The artificial neural networks are used to predict two characteristic hand postures needed in the grasping algorithm: the tentative grasping posture and the most open posture. The results show that a relatively simple architecture is enough to obtain satisfactory posture predictions. In fact, two layer feed-forward neural networks with 100 neurons in the hidden layer are used for posture prediction. The performance of the neural networks, in terms of prediction errors for the joint angles, is similar to the joint angles variability, which implies that a good prediction has been made. However, the results also indicate that more experimental data regarding the subjects is needed.

The use of the artificial neural networks within a grasping algorithm allows the computation of a feasible and realistic grasping posture. Moreover, this algorithm produces the contact information needed for the biomechanical analysis of the grasping process. The global error increases slightly when compared to the error of the artificial neural networks (around 10°), although this result is reasonable if we take account of the joint angles' repeatability when grasping objects in daily life.

Future works would include the use of this methodology for objects of a non-cylindrical shape – but with a similar grasp configuration – and the realization of more experimental work for the generalization of proper grasping.

Footnotes

6. Acknowledgments

The research leading to these results has received funding from the Spanish Research and Innovation Ministry and the FEDER through the project DPI2010–18177; Fundació Caixa-Castelló and the Universitat Jaume I through the project P1–1B2011–25.

References

Miller

Allen

Santos

Valero-Cuevas

(2005) From robot hands to human hands: A visualization and simulation engine for grasping research. Industrial Robot 32(1): 55–63.

Kyota

Watabe

Saito

Nakajima

(2005) Detection and evaluation of grasping positions for autonomous agents. Int. Conference on Cyberworlds. pp. 453–460.

Gorce

Rezzoug

(2005) Grasping posture learning with noisy sensing information for a large scale of multifingered robotic systems. Journal of Robotic Systems 22(12): 711–724.

Buchholz

Armstrong

(1995) A kinematic model of the hand to evaluate its prehensile capabilities. Journal of biomechanics 25: 149–162.

(2001) Toward better grasping and manipulation by multifingered robotic hand. Doctoral Dissertation. Harbin Institute of Technology. China.

Kemp

Edsinger

Torres-Jara

(2007) Challenges for robot manipulation in human environments. IEEE Robotics and Automation Magazine 14(1): 20–29.

Fok

Chou

(2010) Development of a finger biomechanical model and its considerations. Journal of Biomechanics 43(4): 701–713.

Kamper

Fischer

Cruz

(2006) Impact of finger posture on mapping from muscle activation to joint torque. Clinical Biomechanics 21(4): 361–369.

Kurita

Onoue

Ikeda

Ogasawara

(2009) Biomechanical analysis of subjective pinching effort based on tendon-skeletal model. Proc. IEEE Eng. in Medicine and Biology Society, pp. 5231–5234.

10.

Lee

Chen

Towles

Kamper

(2008) Effect of finger posture on the tendon force distribution within the finger extensor mechanism. Journal of Biomechanical Engineering 130(5): 051014–051023.

11.

Lee

Chen

Towles

Kamper

(2008) Estimation of the effective static moment arms of the tendons in the index finger extensor mechanism. Journal of Biomechanics 41(7): 1567–1573.

12.

Qiu

Fischer

Kamper

(2009) Muscle activation patterns during force generation of the index finger. Proc. IEEE Eng. in Medicine and Biology Society, pp. 3987–3990.

13.

Roloff

Schöffl

Vigouroux

Quaine

(2006) Biomechanical model for the determination of the forces acting on the finger pulley system. Journal of Biomechanics 39(5): 915–923.

14.

Sancho-Bru

Perez-Gonzalez

Vergara-Monedero

Giurintano

(2001) A 3-D dynamic model of human finger for studying free movements. Journal of Biomechanics 34(11): 1491–1500.

15.

Sancho-Bru

Perez-Gonzalez

Vergara

Giurintan

(2003) A 3D biomechanical model of the hand for power grip. Journal of Biomechanical Engineering 125(1): 78–83.

16.

Sancho-Bru

Giurintano

Pérez-González

Vergara

(2003) Optimum tool handle diameter for a cylinder grip. Journal of Hand Therapy 16(4): 337–342.

17.

Sancho-Bru

Vergara

Rodríguez-Cervantes

Giurintano

Pérez-González

(2008) Scalability of the Muscular Action in a Parametric 3D Model of the Index Finger. Annals of Biomedical Engineering 36 (1): 102–107.

18.

Valero-Cuevas

(2000) Predictive modulation of muscle coordination pattern magnitude scales fingertip force magnitude over the voluntary range. Journal of Neurophysiology 83 (3): 1469–1479.

19.

Valero-Cuevas

Towles

Hentz

(2000) Quantification of fingertip force reduction in the forefinger following simulated paralysis of extensor and intrinsic muscles. Journal of Biomechanics 33(12): 1601–1609.

20.

Valero-Cuevas

(2005) An integrative approach to the biomechanical function and neuromuscular control of the fingers. Journal of Biomechanics 38(4): 673–684.

21.

Vigouroux

Quaine

Labarre-Vila

Moutet

(2006) Estimation of finger muscle tendon tensions and pulley forces during specific sport-climbing grip techniques. Journal of Biomechanics 39(14): 2583–2592.

22.

Vigouroux

Ferry

Colloud

Paclet

Cahouet

Quaine

(2008) Is the principle of minimization of secondary moments validated during various fingertip force production conditions? Human Movement Science 27(3): 396–407.

23.

Cutlip

Dong

(2010) A practical biomechanical model of the index finger simulating the kinematics of the muscle/tendon excursions. Bio-Medical Materials and Engineering 20(2): 89–97.

24.

Lee

Zhang

(2005) Development and evaluation of an optimization-based model for power-grip posture prediction. Journal of Biomechanics 38(8): 1591–1597.

25.

Bae

Chan S

(2011) Investigation of the hand posture during reach and grasp for ergonomic applications. Doctoral Dissertation. University of Michigan. EEUU.

26.

Rezzoug

Gorce

(2008) Prediction of fingers posture using artificial neural networks. Journal of Biomechanics 41(21): 2743–2749.

27.

Choi

(2008) Developing a 3-Dimensional kinematic model of the hand for ergonomic analyses of hand posture, hand space envelope, and tendon excursion. Doctoral Dissertation. University of Michigan. EEUU.

28.

Armstrong

Best

Bae

Choi

Grieshaber

Park

Woolley

Zhou

(2009) Development of a kinematic hand model for study and design of hose installation. Proc. 2^nd Int. Conf. on Digital Human Modeling, pp. 85–94.

29.

Bicchi

(2000) Hand for dexterous manipulation and robust grasping: A difficult road toward simplicity. IEEE Trans. Robotics & Automation 16(6): 652–662.

30.

Molina-Vilaplana

López-Coronado

(2007) Neural modelling of hand grip formation during reach to grasp. Neurocomputing 71(1): 411–416.

31.

Kuperstein

Rubinstein

(1989) Implementation of an adaptive controller for sensory-motor condition. IEE Control Systems Magazine: 25–30.

32.

Moussa

Kamel

(1998) An Experimental approach to robotic grasping using a connectionist architecture and generic grasping functions. IEEE Transactions on Systems, Man and Cybernetics 28: 239–253.

33.

Jiménez

Thomas

Torras

(1998) Collision detection algorithms for motion planning. In: Laumond

J-P

editor. Robot Motion Planning and Control. Springer-Verlag. pp. 305–343.

34.

Schwarzer

Saha

Latombe

(2004) Exact collision checking of robot paths. In: Boissonnat

J-D

editor. Algorithmic Foundations of Robotics V, STAR 7. Springer-Verlag. pp. 25–41.

35.

Bernabeu

Tornero

(2002) Hough Transform for Distance Computation and Collision Avoidance. IEEE Trans. Robotics & Automation 18(3): 393–398.

36.

Gilbert

Johnson

Keerthi

(1988) A fast procedure for computing the distance between complex objects in 3-dimensional space. IEEE Journal of Robotics & Automation 4(2): 193–203.

37.

Sancho-Bru

Pérez-González

Mora

León

Vergara

Iserte

Rodríguez-Cervantes

Morales

(2011) Towards a realistic and self-contained biomechanical model of the hand. In: Klika

Vaclav

editor. Theoretical Biomechanics. In-Tech.

38.

Brand

Hollister

(1992) Clinical mechanics of the hand. Third ed. St. Louis: Elsevier Science Health Science div.

39.

Kapandji

(1998) Fisiologie articulaire, membre supérieur. Paris: Editions Maloine.

40.

Chao

Cooney

Linscheid

(1979) Normative model of human hand for biomechanical analysis. Journal of Biomechanics 12(10): 775–788.

41.

Buchholz

Armstrong

Goldstein

(1992) Anthropometric data for describing the kinematics of the human hand. Ergonomics 35(3): 261–273.

42.

Hollister

Giurintano

Buford

Myers

Novick

(1995) The axes of rotation of the thumb interphalangeal and metacarpophalangeal joints. Clin. Orthop. Relat. Res. (320): 188–93.

43.

Denavit

Hartenberg

(1955) A kinematic notation for lower-pair mechanisms based on matrices. Journal of Applied Mechanics 77: 215–221.

44.

Armstrong

Best

Bae

Choi

Grieshaber

Park

Woolley

Zhou

(2009) Development of a Kinematic Hand Model for Study and Design of Hose Installation. Proc. 2^nd Int. Conf. on Digital Human Modeling, pp. 85–94.

45.

Cerdá-Boluda

Gadea-Gironés

(2009) Introducció als sistemes complexos, als autòmats cel.lulars i a les xarxes neuronals, Universidad Politécnica de Valencia Ed.

46.

Haykin

(2009) Neural networks and learning machines: A comprehensive foundation. Prentice Hall Ed.

47.

Kůrková

(1992) Kolmogorov's theorem and multilayer neural networks. Neural Networks 5(3): 501–506.

48.

Mora

Sancho-Brú

Iserte-Vilar

Pérez-González

Porcar

(2011) Protocolo de Calibración para Guante Instrumentado en la Caracterización Cinemática del Agarre Humano. X Congreso Iberoamericano de Ingeniería Mecánica.

49.

Pérez-González

Mora

Sancho-Brú

Iserte-Vilar

Vergara

Porcar

(2011) Caracterización Experimental de la Biomecánica del Agarre Humano: Manipulación de Botellas. X Congreso Iberoamericano de Ingeniería Mecánica.

50.

Sancho-Brú

Mora

León

Pérez-González

Iserte-Vilar

(2011) Modelo de la Mano para la Evaluación de Agarres. X Congreso Iberoamericano de Ingeniería Mecánica.

51.

Sancho-Bru

Mora

León

Pérez-González

Morales

(2012) Grasp modelling with a biomechanical model of the hand. Computer Methods in Biomechanics and Biomedical Engineering.

52.

Pérez-González

Sancho-Brú

Jurado-Tovar

Mora

Iserte-Vilar

(2012). Application of a modified elastic foundation contact model for simulating hand-object interaction in grasping tasks. 10th Int. Symp. on Computer Methods In Biomechanics & Biomedical Engineering.

53.

Pérez-González

Jurado-Tovar

Sancho-Bru

Mora

(2012). Static force response of the palmar side of the human hand. Journal of Biomechanics 45: 569.