Force-Sensing Glove System for Measurement of Hand Forces during Motorbike Riding

2.1. The Force-Sensing Glove System

The prototype system is a lightweight measurement device designed to measure hand performance automatically. This system is composed of a portable subsystem and a remote unit (Figure 1). The portable device consists of two gloves with 4 tactile sensors attached onside each glove and connected to a microcontroller through front-end electronics. The sensor was mounted on the palms and fore fingers to measure the major activities during motorbike riding. The management of the sensor measurement is assigned to the microcontroller that performs several functions: interfacing and conditioning of signals from the sensor block and data transmission. During the operation, the remote unit (the smartphone used in this research) converts, records, and displays the received force measures data.

OPEN IN VIEWER

Figure 1

(a) Block diagram of the proposed system (① four flexi-sensors for one glove, ② chipboard with a MCF51 chip, ③ a Bluetooth transceiver, and ④ a smartphone or a laptop). (b) The prototype force-sensing gloves.

The most important requirements for the force-sensing glove system include low cost and lightweight construction, unencumbered movement on operating motorbike during data collection, and unobstructed sense of touch on the palms. In the present configuration, four A401 Flexi-Force tactile sensors (Tekscan, Inc., South Boston, MA) were attached as in Figure 1(b) to measure tactile forces. Two tactile sensors were placed on index finger distal phalanx (ID) and middle finger distal phalanx (MD), respectively, so that they are applied in the monitoring of the activity of two-finger braking [12]. Such braking action is used most commonly by riders because of its benefits including reduced braking reaction time and simultaneous use of throttle and brakes. It generally gives riders a firm grip on the throttle. The other two force sensors are integrated on palm (P) and thumb proximal phalanx (TP) of the glove surrounding the throttle; therefore the rotation of throttle can be observed from two force signals [3]. The A401 Flexi-Force sensor is a commercial light-weighted foil product which weighs only 0.33 g, with an active area diameter of 25.4 mm and 0.2 mm thickness. All tactile sensors are separately connected to the conditioning/signal-processing circuit block using twisted pair cable. The electrical noise going into or coming from the cable can be prevented to an extent. All tactile sensors are interfaced to the smartphone wirelessly via Bluetooth protocol.

The microcontroller is the MCF51JM128VLH (Freescale Semiconductor Inc., West Austin, TX). This commercialized device is a 32-bit reduced instruction set computing (RISC) microprocessor and an ultralow power microcontroller. It operates at processor core speeds up to 50.33 MHz. The operating supply voltage is between 2.7 V and 5.5 V. The Bluetooth transceiver is a Parani-ESD 200D (Sena Technologies, Inc., San Jose, CA) and its operating supply voltage is 3.3 V. Bluetooth standard was specifically proposed by the SIG to fulfill a need for extremely lower power, lower data transmissions for wireless sensor networks monitoring, and most of glove-based system [5]. Bluetooth technology feasibly supports real-time and wireless hand activity recognition with a mobile device. The adopted bidirectional transmission is not encrypted and is asynchronous. Authentication and error detection are not implemented as well. Parani-ESD200 has a built-in on-board antenna and provides an antenna connector for longer distance communication. Another reason for adopting Bluetooth standard is to perform data collection from both gloves synchronously. With the Android 4.0 version operation system, a smartphone can communicate well with the Bluetooth hardware and millisecond-level synchronization between the wireless devices can be achieved.

The wearable portion of the system weighs approximately 145 g (four sensors + wires: 40 g, signal conditioning circuit with packaging + battery: 105 g). The entire system (including sensors and signal conditioning hardware) is powered by a Li-Ion battery, which is sufficient for a minimum of 8 hours of data collection. There are sensor nodes that connect to the body and record physiological parameters before wirelessly transmitting the recorded signals to a smartphone, PC, or another computer. The power consumption of the sensor node is a critical factor in realizing the smaller device size with the longer operating lifetime, which in turn affects the ease of use. Table 1 summarizes the specifications of three typical batteries which are potentially suitable for powering glove-based system. The physical size, which is discussed above, dominates the device volume and the energy storage capacity, typically expressed in mA-hours. Though capacity is lower than others, Li-Ion battery is better in size and weight. Duration tests also show that over 8 hours of battery life is enough for daily use of our glove system with Li-Ion battery. Hence Li-Ion battery is the best choice for our force-sensing glove system.

OPEN IN VIEWER

Table 1

Specification of batteries potential for wireless glove-based system.

Name	Type	Rechargeable	Amount	Nominal voltage (V)	Capacity (mAh)	Size ( L × W × H , mm)	Weight (g)
2A everyday use [6]	Alkaline	No	2	1.5	1500	50.5 × 14.5 × 14.5	23
2A recharge [8]	NiMH	Yes	2	1.5	1500	50.5 × 14.5 × 14.5	30
BL 4D	Li-Ion	Yes	1	3.7	1200	60 × 44 × 4	24

Data acquisition software (as shown in Figure 2 a real-time monitoring of the glove activity) was also developed for hardware setup. Human interaction with the system is facilitated through the smartphone software. The four plots demonstrated the clutch, brake, steer, and throttle of the motorbike, respectively. F 1 and 2 represent two force sensors, which have been detailed at left column accurately based on 1 s, 2 s, and 5 s intervals.

OPEN IN VIEWER

Figure 2

Real-time data acquisition interface of the force-sensing gloves.

The dual purpose smartphone software is responsible for logging the data collected from the glove and visualizing the data. The user interface was built using JDK and eclipse IDE in Android 4.0.4 development environment. The application is capable of performing common Bluetooth operations. The raw data was logged to comma separated value (CSV) files. The software incorporates visual displays of the tactile sensors; the amount force of each hand activity is indicated with color and any emergent event is indicated with sound alarms [6]. In this research, raw data sampling was initially performed at 20 Hz, as generally suggested sampling frequency in other similar works [5, 13].

2.2. Calibration Tests

It is necessary for force-sensing glove system to be calibrated before application to reduce inaccuracies. Most existing force-sensing gloves are calibrated under static conditions [14]. However, since loadings are dynamic in typical applications, dynamic measurements are indispensable in this study. The calibration tests herein include both static and dynamic protocols.

2.2.2. Dynamic Calibration Tests

During the calibration, operator wearing the glove upright placed the sensor on a load cell and smoothly applied a continuously changing force. The load cell (Xtran Load Cells S1W force transducer, Applied Measurement Australia Pty. Ltd., Melbourne, Australia) was fixed on a vibration isolation table with screws. Data of load cell were transmitted to NI USB-6002 data acquisition adapter (National Instruments Corporation, Austin, USA) via an amplifier (Figure 3(a)) [15], then uploaded, and recorded by Ni LabVIEW signal-express 2013 software (National Instruments Corporation, Austin, USA) at 20 Hz, which was the same as the sampling frequency of the force-sensing glove. Then, the measurements of the force-sensing glove system were synchronized with corresponding load cell measurements (Figure 3(b)). Data stream synchronization was confirmed by a tap with the first finger so that a large spike in the sensor data streams from both the force-sensing glove and the load cell marks the start and/or the end of the measurement streams. Unlike manual operation in traditional measurements under static conditions, streams from both the force-sensing glove and the load cell are synchronously and automatically collected in the same window. One advantage is that it avoids operative error.

OPEN IN VIEWER

Figure 3

Calibration. (a) The total setup of testing environment. (b) For the force sensor calibration, finger puts on the surface of a real motorbike brake 3D printed model. (c) Built clutch models in SolidWorks. (d) The load cell which was fixed with a 3D printed model which provides similar touching surface of natural environment.

Since the tactile sensors intrinsically react to pressure, the contact area had to be restricted (made constant) by the introduction of an intermediate activator surface. It was necessary to provide a contact area close to the real environment use for the force-sensing glove system. A model (Figure 3(c)) capturing true geometrical aspects of a motorbike handle or clutch was developed with SolidWorks 2013 (Dassault Systèmes SolidWorks Corp., France) and then 3D printed in ABS plastic. This realistic solid model was fixed to the working plate of the load cell (Figure 3(d)). With the assistance of 3D printed model, the sensing area of the sensor under calibration is fully covered for a reliable calibration.

Four healthy subjects, aged 28–40 years, with no movement disorders in the hands participated in the calibration and reliability studies. All subjects received verbal and written descriptions for all procedures. Calibration was performed for each subject matching with each sensor. Subjects were familiarized with the apparatus and given a demonstration by the operator. In every test set, subjects were required to continuously apply vertical and gradually increasing pressure on the load cell for 10 cycles following an acoustic start signal.

2.3. Force-Sensing Glove Performance Evaluation

2.3.1. Hand Movement Recognition

Our investigation in tactile motion instructions addressed the composition of tactile patterns that could be used for representing hand movements in an intuitive way. Based on force-sensing, we can distil recommendations for composing tactile patterns that could represent specific hand movements.

The algorithm in Figure 4 worked as follows.

OPEN IN VIEWER

Figure 4

Flowchart of the activity classification algorithm for hand performance evaluation. For hand movements that could serve as general instructions. Every movement has a corresponding countermovement.

First, the sensing element recorded on the ID, the MD, the P, and the TP of the left hand (L) and the right hand (R) was computed:

e i t , i ∈ LID , LMD , LP , LTP , RID , RMD , RP , RTP .

(1)

Then, a threshold of 10% of the maximum expected signal e m a x was used to filter out the noncontacting sensing. As a result, the sensing on which contact is detected ( e i > 0.1 × e m a x ) was

F i t = e i t - 0.1 × e m a x .

(2)

As a result, the value 0 became the new reference value for all measurements. Then the forces measured under different places.

For brake and clutch, each of the two sensors is placed and works in the same direction. The resultant force F t is the amplitude combination of two sensors F a ( t ) and F b ( t ) :

F t = F a t + F b t ,

(3)

where F a ( t ) and F b ( t ) are the forces from LID and LMD for clutch activity recognition, while F a ( t ) and F b ( t ) are the forces from RID and RMD for brake activity recognition.

For the steer, it is rotating about a fixed symmetry axis, and its angular momentum is expressed as the product of the moment of inertia of the object and its angular velocity vector:

F c t - F d t R s Δ t = I s Δ ω s ,

(4)

where F c ( t ) is the force from LTP, F d ( t ) is the force from RTP, R s is the distance between the handgrip and the axis, I s is the moment of inertia of one side wheel, and ω s is the angular velocity of the steer.

For the throttle, it also is rotating about a fixed symmetry axis. But the rotation is caused by friction, and thus the angular momentum is expressed as the product of the moment of inertia of the object and its angular velocity vector:

μ F e t + F f t r t Δ t = I t Δ ω t ,

(5)

where F e ( t ) is the force from RP, F f ( t ) is the force from RTP, I t is the moment of inertia of throttle wheel, r t is the radius of throttle, and ω t is the angular velocity.

2.3.2. Verification Tests

One healthy subject was enrolled in the below tests, his age is 22 years, and he has no hand movement disorders. He drove a motorbike in a 400 m straight road, wearing a pair of force-sensing gloves and a Gopro2 camera. The rider repeated trials between the start point and the end point at different final velocity (60 km/h, 80 km/h, and 100 km/h in different trials); each time he started from motionless and then accelerated to the assigned speed. The rider repeated all of these movements eight times for every velocity. A smartphone was attached to the back seat of the motorbike (Figure 5) and simultaneously collected data from all tactile sensors.

OPEN IN VIEWER

Figure 5

Overview for hand performance test.

There are three main metrics that are used to assess the operation.

The sensitivity is the main performance metric and shows how many of the actual events are correctly detected:

Sensitivity = TP TP + FN × 100 % ,

(6)

where TP is the number of correct detection incidents (true positives) and FN is the number of real events that are missed and not detected (false negatives).

The specificity shows how many nonevents that should not be detected are indeed not detected:

Specificity = TN TN + FP × 100 % ,

(7)

where FP is the number of incorrect, false, detection incidents of an event (false positives) and TN is the number of nondetection incidents (true negatives). Often it is not easy to define what a true negative actually is.

The selectivity shows what fraction of all of the detection incidents made is in fact correct:

Selectivity = TP TP + FP × 100 % .

(8)

3.1. Tactile Sensing Performance

The static calibrations in this study validate the feasibility of the wireless force-sensing glove system in force measurement for solid and flat surfaces. Similar to most relevant studies [13, 15, 16], the static calibration tests as recommended in the User's Manual were efficiently accurate and reproducible. The tactile sensors demonstrated reasonable accuracy (MD% = 0.14~0.88%) and linearity ( R 2 > 0.99 ) in static calibration tests (Table 2). Figure 6 indicates measurements made for the retrospective calibration of test data; a significant linear relationship exists between weight and the measurement. Moreover, the tactile sensors produce consistent measurements in repeated test conditions. The coefficient of determination was checked to ensure its statistical significance at the 99.9% confidence level, which it comfortably exceeded.

OPEN IN VIEWER

Table 2

Results that come from calibration tests.

Session	Content	Left hand				Right hand
		ID	MD	P	TP	ID	MD	P	TP
Static calibration	Linearity ( R 2 )	0.994	0.992	0.990	0.986	0.996	0.992	0.992	0.996
	Accuracy (MD%)	0.62%	0.88%	0.43%	0.36%	0.14%	0.37%	0.47%	0.55%
Dynamic calibration	Linearity ( R 2 )	0.945	0.913	0.929	0.940	0.949	0.939	0.927	0.964
	Accuracy (MD%)	2.33%	4.11%	3.14%	4.34%	5.66%	3.85%	2.87%	3.51%

OPEN IN VIEWER

Figure 6

Data for static calibration test.

Tekscan sensors are useful and reliable tools once the calibration has been optimized for their application, and calibration should be conducted under conditions as similar as possible to those used in force measurement [13–15]. A new calibration method was developed for synchronous measurements, for forces applied to an uneven surface, that is, motorbike handle. In the development of this calibration process, diverse behaviors had to be considered in order to obtain an acceptable force measurement in the natural environment.

Hysteresis, although not eliminated, is effectively compensated when data from force-sensing glove and load cell are acquired simultaneously. Hysteresis is the difference in the sensor output response during increased loading and decreased loading at the same force [13]. Sensor drift compensation was considered for static measurements, while hysteresis compensation was performed for dynamic measurements [15]. Since data acquisition depended on the experimental protocol (sensor type, interface shape and materials, sensor range in use, loading method, etc.), sensor physical characteristics and behavior should be investigated separately to avoid any potential errors. Hysteresis for all tactile sensors varied from 5.45% to 13.10% in static calibration while they varied from 1.46% to 4.27% in dynamic calibration. The effect is significant enough to warrant calibration of the force-sensing glove for each use to permit reliable repetition of dynamic force data acquisition.

Curved surfaces such as the motorbike handles were of concern since the force sensors would be subject to physical deformation even when no force was applied. Since interface forces happen between support surfaces, glove garments, and the skin, the surfaces of the skin are commonly uneven, and the curvature unavoidably changes the output of the tactile sensors [13]. As shown in Figures 7(b) and 7(d), it has a strong linear connection between the data from force-sensing glove system and the load cell. As noncontact area and slipping occur on curved surfaces of clutches, wrong detection consequently occurred which caused outlier points in scatter plots for sensor placed on the thumb proximal phalanx (Figure 7). Unlike flat brake or clutch surface touched by the sensor on the index finger distal phalanx, handlebar surface is more uneven and similar to cylinder. Ferguson-Pell et al. [13] conclude that the outputs of the sensor changed when the pressures were applied to the curved surface. In this study, the results show that there is no significant difference from the flat surface when the radius of the curved surface was less than 10 mm (the radius of the 3D model is 10 mm). Compared with the recognition results in Table 3, there is no significant influence by different surface in our calibration measurements; the reliability of the tactile sensors shows some decrease in dynamic calibration (Table 2). The simulated surface in our dynamic calibration exposes the influence to a great extent. When comparing our results with the literature [16], the accuracy (MD% = 2.33~5.66%) and the linearity ( R 2 > 0.91 ) are acceptable based on the accuracy and linearity standard in static calibration tests.

OPEN IN VIEWER

Table 3

Recognition results with respect to maximum velocity change.

Movements	Sensitivity (%)			Specificity (%)			Selectivity (%)
	60 km/h	80 km/h	100 km/h	60 km/h	80 km/h	100 km/h	60 km/h	80 km/h	100 km/h
Brake	90.0	95.0	100.0	0.0	0.0	0.0	90.0	95.0	100.0
Steer	81.3	87.5	75.0	16.7	66.7	25.0	83.9	95.5	80.0
Clutch	95.0	95.0	97.5	0.0	0.0	100.0	95.0	95.0	100.0
Throttle	89.6	89.3	87.5	40.0	33.3	37.5	93.5	92.6	91.8

OPEN IN VIEWER

Figure 7

Typical dynamic calibration results for sensors on index finger distal phalanx and thumb proximal phalanx of the same subject's left hand. (a) Data of sensor on index finger distal phalanx by both measures. (b) Correlation of data from two measures. (c) Data of sensor on thumb proximal phalanx by both measures. (d) Correlation of data from two measures.

Despite considerable effort and many attempts to calibrate these sensors, the results were not perfectly satisfactory. The new calibration method was deemed necessary for force-sensing glove that would be sufficiently accurate and reproducible to enable quality research to be conducted in motorbike hand performance measurement. The force-sensing glove system is a potential attempt for motorbike operation after calibrations in static and dynamic conditions.

3.2. Force-Sensing Gloves' Performance

Off-line data analysis reveals that it is possible to differentiate hand movements based on the results between the velocity changes and the force data (Figure 8). In fact, hand movements recognition is challenging from the data stream of the force-sensing glove. The velocity changes during riding are superimposed on the hand movements making separation for recognition difficult. Compared to force information, hand movements messages are minor. The algorithm reported data in four groups.

OPEN IN VIEWER

Figure 8

Typical performance results from a straight speed up to 60 km/h and speed down trial. (a) Speed data integral from accelerate of smartphone, (b) force data result from force-sensing gloves. As shown in this demonstration. During 0–4 s, there was a rapid acceleration and a bit right turn after shift up. During 4–8 s, there was an adaption which included a left turn and a bit brake. Accelerate during 8–16 s and keep the speed over 10 seconds during 16–28 s. Before brake for speed down, there was a continuous shift down. Finally there was brake movement for about 8 seconds.

The accuracy data of the recognition algorithm were assessed through a one-by-one comparison of its output with the expected results derived from manual annotations of videos. Table 3 summarizes the recognition accuracy for the test set. Results indicated that overall accuracy of the recognition algorithm is over 80%. The force-sensing glove performs good sensitivity in hand activity recognition.

The developed algorithm used a threshold test and empirically chosen parameters for the hand movement classification. This method is simple when compared to machine learning algorithms that consider sensor data variability. Even so, most of hand movements were correctly recognized in the test set, which comprised records with different speeds. The rate of wrong detection increased particularly for hand movements of steer and throttle. It was necessary to improve the setup of the force sensors and adjust threshold parameters to improve recognition. Although our algorithm correctly classified hand movements with high accuracy, we conjecture that other external sensors are required to assist with event detection, for example, the built-in accelerometer in the smartphone. Future work will focus on more movement experiments under real motorbike riding tests coupled with further sensors for event detection and movement recognition.

After providing a comprehensive understanding of the performance characteristics of the various force glove measurement system, the aim of this study is to estimate the influence of finger force and predict potential accidents to handle the racing motorbike activities. This force-sensing glove system furnishes sufficient data to build a feedback sensor network for the safety of motor divers and determine an optimal handle shape for motorbike riding.

Furthermore, the alerting threshold could be customized by the riders' subjective assessment during the test. These types of riding assistance improve the judgment of a safe driving approach and achieve the intelligent speed adaptation. This force-sensing glove system furnishes sufficient data to build a feedback sensor network for the safety of motor divers and determines an optimal handle shape for motorbike riding.