Design and control of intelligent bionic artificial hand based on image recognition

Abstract

BACKGROUND:

At present, the popular control method for intelligent bionic prosthetic hands is EMG control. However, the control accuracy of this method is low. It is a trend to integrate computer vision into the prosthetic hand.

OBJECTIVE:

The purpose of this paper is to design an intelligent prosthetic hand based on image recognition, improve the control accuracy and the quality of life of the disabled.

METHODS:

Convolutional neural network is used to recognize the object to be grasped, and the recognition result is used as a trigger signal to control our intelligent prosthetic hand. We have designed a four-bar linkage mechanism and a side swing mechanism in the structure, which can not only achieve the flexion and extension of fingers but also realize the adduction and abduction of the four fingers and the lateral swing of the thumb.

RESULTS:

Through the method of image recognition, the new intelligent bionic hand can achieve five kinds of Human action. Including grasp, side pinch, three-finger pinch, two-finger pinch, and pinch between fingers.

CONCLUSIONS:

The experiment result proves that the precision of image recognition control is very excellent, the intelligent prosthetic hand can be completed the corresponding task.

Keywords

Deformable convolution image recognition prosthetic hand

1. Introduction

Human hands are the most complex and exquisite organ, they cooperate with the eye and brain to achieve various movements. It plays an important role in our life, series of physical and mental injuries will be caused because of the loss of hand. It directly affects the external appearance of the patient and reduces the quality of life. In recent years, the research of intelligent bionic hands is one of the hotspots in human-machine communicative robots [1].

The sophisticated design of the structure is one of the critical properties of the artificial hand. Transmission methods can be divided into connecting rod transmission, tendon rope transmission, and gear drive. Jing et al. [2] proposed a prosthetic hand that used tendon rope transmission, only applied three motors to achieve 13 types of prosthetic hand grips. Laffranchi et al. [3] developed an underdrive hand with a rope drive structure to realize fingers and wrist function. Said et al. [4] designed a 3D printed prosthetic hand, driven by a linear motor, which can achieve 13 degrees of freedom of the prosthetic hand, and the overall mass was 428 grams. However, its structure was simple and only performed a simple grasping function. Kai et al. [5] designed a five-finger human-like mechanical artificial hand based on a continuum differential mechanism. Xiong et al. [6] created a five-finger imitation hand driven by four motors. The Michelangelo prosthetic hands’ metacarpophalangeal joints were responsible for flexing each finger’s flexion and extension, not including the adduction and abduction movements of the fingers [7]. The use of linear motion relationships [8] in BeBionic prosthetic hands to govern the coordination of each finger’s movement had a particular impact on the coordination of human hands. Abayasiri et al. [9] designed an underactuated hand prosthesis with finger abduction and adduction to grasp larger objects. However, it used gear transmission, which was difficult to control the motion ratio, and the cutting mechanism occupied a lot of internal spaces.

The control methods are the core to enhance the function of the prosthetic hand. Li et al. [10] preprocessed the sEMG of the forearm muscles with the 20-order comb filter and wavelet, the hand movements were realized successfully through GRNN and SVM. At present, the application of residual limbs to provide electromyographic signal (EMG) is a popular control method. However, electrodes over time are susceptible to a decline in signal amplitude, which minimizes their clinical utility [11]. Vu et al. [12] found that the signal generated by regenerating peripheral nerve tissue in upper limb amputees had long-term stability. They successfully controlled the prosthetic hand in real-time for up to 300 days without recalibrating the control algorithm. With the rise of artificial intelligence, people began to combine neural networks with artificial hand control. It aims to improve the accuracy of pattern recognition. Manfredo et al. [13] improved prosthetic hand control accuracy based on convolutional neural networks (CNN) to classify EMG signals. Mukhopadhyay et al. [14] presented the detailed experience exploration of the EMG signal classification system based on the deep neural network (DNN) with the upper limbs’ position unchanged. They used a fully connected feedforward DNN model to classify eight different hand EMG signals, which outperforms other existing classifiers. Zhang et al. [15] applied the sliding window method to segment the collected EMG signals, and sent the data sets to the feedforward neural network for training, the average recognition rate was about 98.7%. Although the application of surface EMG signals can greatly improve the bionics of the prosthetic hand, the accumulation of sweat under the surface electrodes [16] or electrode displacement or damage to the soft tissues [17] of the limbs will affect the real-time control accuracy of the prosthetic hand. Image recognition technology based on neural networks is an intelligent control method that has emerged in recent years. It integrates autonomous visual control and can reduce the necessary visual attention during the grasping process of amputees. The Karlsruhe Institute of Technology (KIT) prosthetic hand [18] had an embedded camera which can realize real-time object recognition, and it was the first to incorporate a camera into a prosthetic hand. However, its control algorithm took a long time to identify the object correctly. CNN are an effective method of image recognition. From the earliest LeNet5v [19] network model proposed by LeCun, AlexNet [20], GoogleNet [21], MSRA-Net [22], ResNet [23] and an ultra-lightweight network called Squeezenet [24] have been gradually developed. However, these traditional image recognition network models are mainly for common objects, and their performance was not very good for objects with large deformation.

In this paper, a new prosthetic hand was designed, the more important was a new method based on the currently popular image recognition in computer vision was proposed as the way to control it. A four-bar linkage structure was designed, which can realize the adduction and abduction movements of the prosthetic hand. The method of tendon and rope transmission was adopted to improve the utilization efficiency of the internal space. A reversing structure was designed to realize the side swing of the thumb and improve the prosthetic hand’s grip performance. Deformable convolution [25] was incorporated into the ResNet-18 model which was the basic model to improve irregular objects’ recognition accuracy.

2. Overall design

In terms of a human hand, its 25 degrees of freedom include a single interphalangeal (IP), metacarpophalangeal (MCP), and carpometacarpal (CMC) joints of the thumb, as well as two IP and one MCP each of the other four fingers as shown in Fig. 1.

Sufficient degrees of freedom can significantly reduce the rejection rate of artificial hands. Therefore, in the prosthetic hand design, the degrees of freedom (DOFs) of each finger were preserved as much as possible. The distribution of the DOFs was shown in Table 1, and “I to V” represent the thumb, index, middle, ring, and little fingers, respectively. “1 to 3” represent the number of DOFs, respectively.

Table 1
The distribution of the DOFs

I	/	/	/	1	3
	DIP	PIP	MCP	IP	CMC
II	1	1	2	/	/
III	1	1	2	/	/
VI	1	1	2	/	/
V	1	1	2	/	/

Figure 1.

Human hand DOFs. There are 1, 2, and 3 degrees of freedom (DOFs) in DIP, PIP, and MCP joints. The wrist of the hand also has 3 DOFs, and the wrist of our prosthetic hand only retains the DOFs of flexion and extension.

In addition to improve the bionic hand’s normal gripping function, its opposing-palm movement and fingertip pinching movement are also essential for a pair of flexible hands. To realize these features, a prosthetic hand was designed in this paper, and the structures were described as follows.

Weight is one of the critical factors in designing intelligent bionic hands [26]. Moreover, the function of the wrist can be approximately compensated by the role of the human arm. Therefore, our prosthetic hand’s wrist only retained one degree of freedom for flexion and extension, a single steering gear was applied to achieve its function. The overall structure was shown in Fig. 2A.

Figure 2.

Hand design. The whole prosthetic hand consists of fingers, palms, and arms. (A) The two-drive steering gear of the thumb was placed in the palm. (B) The thumb (left) mechanical structure and the routing mode of thumb transmission (right) were indicated by the red dotted line. (C) The mechanical structure (right) of the index finger (representing four fingers) and the transmission routing mode (left) of the index finger were indicated by the red dotted line. (D) The steering gear of the other three fingers, except the index finger, was placed in the arm. (E) This linkage mechanism can realize the adduction and abduction function of four fingers. In order to reduce the difficulty of control, and in the foue-finger adduction and abduction movement, the movement angle of the middle finger is very small, so it was fixed on the palm in our design.

A new cross-type rotating mechanism was designed, which can realize the function of flexion, extension and posture adjustment of the thumb. A new rotating body was intended on the MCP joint of the thumb, driven by a small steering gear through tendon rope, which can change the attitude angle of the thumb relative to the entire palm. Its flexion, extension, and adjustment of posture were equipped with a steering gear drive, respectively. Pulley was installed to change the transmission direction of the tendon rope, and a small spring was installed at each joint of the finger to realize the automatic reset of the thumb’s position. Its structure is shown in Fig. 2B.

The transfer method of the four fingers was similar to the thumb, and the flexion and extension functions of each finger were realized by the steering gear pulling the tendon rope. The critical point was that a rotation degree of freedom perpendicular was added to the palm of the four-finger metacarpophalangeal joint root. The structure of the four fingers except for the thumb was similar. Taking the index finger as an example, the structure was shown in Fig. 2C. To make reasonable use of the prosthetic hand’s spatial distribution, each drive source of the four fingers was set in the palm or at the end of the arm and use tendon cords to transmit power in the middle. Its structure was shown in Fig. 2D.

Figure 3.

Control system. (A) The transmission process of the signal between devices. (B) The prosthetic hand equipment is made of high toughness resin material for 3D printing, and the overall weight is 336.5 g. (C) The control system consists of three parts: central control unit, PWM signal shunt module, and power supply regulation module. (D) The experimental platform. A RGB camera was used to grasp the object and the control system would be worked after the image recognition to make the hand complete the prescribed action.

A new four-bar linkage mechanism was designed to achieve the four-finger adduction and abduction function, the middle finger was fixed, and the index finger and ring finger rely on only one link to achieve joint movement. The active movement of the four fingers was realized with only one steering gear. The structure was shown in Fig. 2E.

The overall size of prosthetic hand: it was about 165 mm from the longest middle finger to the base of the palm. When the prosthetic hand was in the natural flat rest position, its maximum width is almost 120 mm from the palm side to the thumb. The maximum size of the arm was about 118 mm.

3. Control platform construction

A desktop software operating system was developed as a communication medium between the upper-end recognition signal and the lower-end real-time control. In other respects, using the STM32 development board as the central control unit, and using the PCA9685 module to divide the specified PWM signal into multiple channels, and control our prosthetic hand to perform movements. An external power supplier was applied as the power source. To ensure the overall circuit’s safety, a step-down module was used to reduce the output voltage and current. The overall control framework of the system was shown in Fig. 3.

The Logitech C920e RGB camera was used to collect image data at present and in future the mobile phone’s camera will be applied to capture images because it is more convenient. After a simple size transformation, the collected pictures will be directly inputted into the CNN for classification and recognition. The network was written with Tensorflow as the framework. The resulting label would be directly sent to the central controller via serial communication. The central control unit would then match this instruction with the preset instruction to control the prosthetic hand perform corresponding movements. If the command failed to match, a feedback signal would be sent to retake the picture information. The overall interaction method was shown in Fig. 4.

Five hand movements corresponding to different labels were designed to facilitate the comparison of the central control and the analysis of instructions. The corresponding relationship is shown in Table 2.

Table 2
Labels and functions

Gesture	Label
Grasp	1
Side pinch	2
Three-finger pinch	3
Two-finger pinch	4
Pinch between fingers	5

Figure 4.

Communication method. The image information was collected by the computer camera, sent to the neural network for identification. The discriminating tag was sent to the software and then delivered to the control unit after processed to control the prosthetic hand performs movements.

4. Network design

Geometric changes caused by scale, posture, viewpoint, and part deformation are the main challenges of object recognition. At present, convolutional neural networks for image recognition mostly use block convolution, scanning images with a fixed shape and size, extracting features from the images. However, its ability to cope with geometric changes is limited. Deformable convolution is one of the main methods to solve this problem. It can change the size, shape, and position of the window size according to the target’s shape features, focus on the image target object, and extract the essential features. Compared with the standard convolution, the Deformable convolution divides the convolution process into two paths and adds a fully connected layer to learn the offset. Its operation process was shown in Fig. 5A.

Figure 5.

Overall network model. (A) Deformable convolution module used to identify deformed objects’ characteristics. (B) Based on ResNet-18 designed a new model combined with deformable convolution, which can improve the feature extraction ability of deformed objects.

The model structure in this article was improved based on ResNet-18 and combined with the characteristics of Deformable convolution, designed our network model. The overall network model designed in this paper was shown in Fig. 5B. In the shortcut block of the typical ResNet-18 network structure, the input was simply down-sampled. To fully extract the features of the recognized object and reduce the influence of background factors, Deformable convolution was applied in the shortcut block to extract features from the input and add the result to the main road to increase the feature extraction capability.

5. Experiments

5.1 Datasets collection

In order to supply the datasets for the CNN model designed in this paper, a image grasp platform was made. The object which will be grasped by our hand was placed on the rotating platform that can make a 360 degree rotation. At the same time, the Logitech RGB camera will be used to record this process. Then, the video will be divided into 108 images by our algorithm as a part of the training datasets. The overall process was shown in Fig. 6.

Table 3
Part of the dataset

Figure 6.

Image collection. (A) The object demo which will be grasped, only showed a part of the objects. (B) The equipment for collect image which include a rotating platform and a Logitech C920e RGB camera, the platform can make a 360 degree rotation and the camera will record this process. (C) The Video File. It is used to save the video so that our program can extract images from this video automatically. (D) The part of images extracted from the video which will be as the training datasets.

To verify the performance of the network, a small data set was made to train the neural network model proposed in this article, including columnar objects, pens, napkins, etc. They are all items that are frequently used by human hands. Some of the data sets are shown in Table 3. The classification rules [27] for the data set were made according to the usage habits of human hands, not according to the shape of the object or other factors.

5.2 Function test

To show the control and structural performance of our bionic hand, we did a few simple experiments, and the experimental results are shown in Fig. 7. The verified execution actions include grasping, pinching between sides, three-finger grasping, two-finger grasping, and pinch between fingers. During the experiment, the five-finger grasping and pinching gestures between the sides showed excellent performance. Although the three-finger pinch action may occasionally appear unstable during the execution process, it can eventually achieve its function without affecting the overall performance. The successful realization of the Yes gesture benefited from a new type of four-bar linkage transmission structure in our mechanical design, as shown in Fig. 2E. It also showed that our artificial hand can not only realize the normal grasping function of the human hand but also complete the adduction and abduction movement of the four fingers of the human hand, and cooperate with the thumb to increase the overall grasping performance of the artificial hand. It proved that our prosthetic hands had better bionics.

Figure 7.

Performance verification. Each row in the figure shows the execution of different actions, and from left to right in each row was the state at a certain moment in the movement’s performance.

5.3 Success rate test

Given the completion of the prosthetic hand movement mentioned in this article, we conducted 50 sets of experiments, and the results are shown in Fig. 8. The experimental results showed that our bionic hand had a higher success rate when performing normal grasping, two-finger grasping, and pinch between fingers. However, the three-finger pinch and the side pinch had a lower success rate. The reason was that the grasping force of the prosthetic hand was not very large. Although the object can be grasped successfully, the stability of maintaining the corresponding posture was too low. The action failure mentioned here was not a failure in the complete sense, but the action stability was not good. We will continue to improve the stability of its actions and the grasping performance in the subsequent work.

Figure 8.

Action success rate. The picture above (left) shows 50 tests for different actions. The light blue represents the number of successes, and the dark blue represents the number of failures. The form (Right) is the success rate of each action.

6. Discussion

The analysis of our bionic hand showed that most of the prosthetic hand dimensions correspond to the standard size of the human body. In appearance, except that the fingertip joint of the thumb had a smaller installation angle relative to the middle finger joint, the rest are similar to the human hand. A tiny corner of the thumb was designed to increase the contact area between the thumb and the other four fingers when the fingertip pinch action occurred to improve the stability of the pinch. Laffranchi et al. [3] developed an underdrive hand only with one rope, so the prosthetic hand can only achieve a simple grasping function. However, each finger was provided with tendons for transmission about our prosthetic hand, which can achieve more functions.

To realize the movement of adduction and abduction, we innovatively designed a four-bar linkage mechanism. Although Abayasiri et al. [9] also achieved the abduction and adduction of the fingers, they used gear transmission which will take up more space and it was difficult to assemble. The application of four-bar linkage mechanism can control the movement angle of the prosthetic hand more precisely.

The rationality of our prosthetic hand structure design had verified by us in the preliminary work. The fingers designed in this paper all have the same structure, so only the index finger was used as an example for analysis. The index finger structure was theoretically analyzed by D-H notation, and the three-dimensional model of the index finger was imported into ADAMS software for simulation analysis at the same time. The result showed that their motion curves were basically same. To give an example, the movement curve of the index fingers is shown in Fig. 9.

Figure 9.

Movement analysis. The MATLAB software was used to analyze the theoretical model, at the same time, the ADAMS software was used to simulate and verify the structure so that we can get a comparison result between the two ways.

In this paper, the adduction and abduction motion of each finger was driven by connecting rods, and the analysis methods were similar. Therefore, only the adduction and abduction motion of the index finger was analyzed. The kinematic model of the index finger was analyzed by MATLAB software, and the three-dimensional structure of the connecting rod was imported into the ADAMS software for analysis. It was found that the adduction and abduction angles of the index finger, ring finger, and little finger were between 0–25 ${}^{\circ}$ , 0–15 ${}^{\circ}$ , and 0–35 ${}^{\circ}$ , respectively (Fig. 10).

Figure 10.

Angle analysis. The MATLAB software was used to analyze the theoretical model, at the same time, the ADAMS software was used to simulate and verify the structure so that we can get a comparison result between the two ways.

The self-made dataset was used to train our network, in order to prove the performance of our model, the accuracy and loss curve were output at the final time (Fig. 11). The train accuracy and the train loss are 0.96 and 0.001, respectively. The validation accuracy and the validation loss are 0.95 and 0.18, respectively.

Although we can achieve our ultimate goal, the adduction and abduction movement angles of the four fingers cannot fully reach the movement angle of the human hand. Follow-up work will continue to improve on this problem. The overall weight of the prosthetic hand is 336.5 g. To maximize the wearing comfort of the patient, we discarded the partial freedom of the prosthetic wrist to reduce the overall weight of the prosthetic hand. To some extent, the function of the artificial wrist can be compensated by the human arm [28]. At present, our prosthetic hand only retains the function of flexion and extension. The comparison of parameters between our prosthetic hand and some typical hands were showed in Table 4. The hand that we made was named MyHand.

Table 4

Labels and functions

Name (year)	Weight (g)	Degrees of freedom	Numbers of actuators	Actuation type
Normal Human Hand	400	21	/	/
MyHand (2021)	336.5	19	8	Digital steering gear
Hannes (2020)	/	14	1	DC motor
KIT (2018)	/	10	2	DC motors
iLimb (2010)	563	6	5	DC Brushed motors-Worm gear
BeBionic (2011)	682	6	5	DC Brushed motors-Lead Screw
Michelangelo (2012)	572	2	2	DC Brushless motors-planetary gear head

Figure 11.

Accuracy and Loss curve. The left is training and validation accuracy curve, the right is training and validation loss curve.

The traditional EMG signal control method has low precision, and the introduction of computer vision technology was a current research direction. The KIT hand [18] was a successful example, however, its control accuracy was not very high. In the bionic hand control recognition, we used image recognition method instead of traditional EMG signal control and the integration of deformable convolution made our network model perform better. The prosthetic hand’s functional test experiment showed that the control of the prosthetic hand through the features identified by the network model can achieve the corresponding actions well. After many experiments and statistics, the success rate of prosthetic hands performing movements was much better, but sometimes it failed. On the one hand, because the feature extraction ability of the network model is not very stable. On the other hand, it is caused by the mechanical structure of the artificial hand.

In the future, the function of adduction and abduction and internal rotation and external rotation will be added to the structure to improve the prosthetic hand’s bionic performance. The larger the image database, the stronger the generalization ability of the trained network model. Therefore, we will collect more object images to expand our dataset in the later work. The multi-source signal cross fusion control method is also a popular research field. In the later work, we will explore the fusion of EMG signal control and image recognition control to form a dual-mode control method to improve the control performance of the bionic prosthetic hand.

7. Conclusion

A multi-degree-of-freedom intelligent bionic prosthetic hand was designed in this study, which can realize the normal grasping and pinching functions of the human hand, and innovatively designed a four-link structure, which can realize the adduction and abduction movement of the four fingers of the bionic hand. The problem of poor dexterity of the existing bionic hand is solved. We innovatively applied image recognition technology as the primary control method of the bionic hand. In order to improve the feature extraction ability of the CNN, we used the deformable convolutional in conjunction with the regular CNN and chose ResNet-18 as the basic network that builds the neural network model in this article. The neural network model proposed in this article can recognize images well and control our prosthetic hands to perform corresponding movements. The functions involved in the article include grasping, pinching between sides, three-finger grasping, two-finger grasping, and pinch between fingers. Its execution success rates were 96%, 86%, 76%, 98%, and 96%, respectively.

Footnotes

Acknowledgments

This work was supported by a grant from the National Key Research and Development Program of China (2018YFB1307200).

Conflict of interest

None to report.

Ethical approval

Not applicable.

References

Shi

Zhu

Song

. Dexterous Prosthetic Hand with Force Perception and Electrical Stimulation. J Mech Eng (China). 2019; 55(11): 10-8.

Jing

Yong

Jiang

Yokoi

. Anthropomorphic Prosthetic Hand with Combination of Light Weight and Diversiform Motions. Appl Sci-Basel. 2019; 9(20): 26. doi: 10.3390/app9204203.

Laffranchi

Boccardo

Traverso

Lombardi

Canepa

Lince

, et al. The Hannes hand prosthesis replicates the key biological properties of the human hand. Science Robotics. 2020; 5(46): 15-30. doi: 10.1126/scirobotics.abb0467.

Said

Boulkaibet

Sheikh

Karar

Alkork

Nait-ali

. Machine-Learning-Based Muscle Control of a 3D-Printed Bionic Arm. Sensors. 2020; 20(11): 16-31. doi: 10.3390/s20113144.

Liu

Zhu

. A Single-Actuator Prosthetic Hand Using a Continuum Differential Mechanism. IEEE International Conference on Robotics and Automation (ICRA); May 26-30 Seattle, WA. LOS ALAMITOS: IEEE Computer Soc; 2015. pp. 6457-62.

Xiong

Chen

Sun

Liu

Yue

Chen

. Design and Implementation of an Anthropomorphic Hand for Replicating Human Grasping Functions. IEEE Trans Robot. 2016; 32(3): 652-71.

Liu

Zhao

Liu

. Design of a Lightweight Single-Actuator Multi-Grasp Prosthetic Hand With Force Magnification. J Mech Robot. 2020; 12(5): 10. doi: 10.1115/1.4047438.

Devi

Udupa

Sreedharan

. A novel underactuated multi-fingered soft robotic hand for prosthetic application. Robotics and Autonomous Systems. 2018; 100(3): 267-77. doi: 10.1016/j.robot.2017.11.005.

Abayasiri

RAM

Abayasiri

RST

Gunawardhana

Premakumara

RMC

Mallikarachchi

Gopura

, et al. An Under-Actuated Hand Prosthesis with Finger Abduction and Adduction for Human Like Grasps. 6th International Conference on Control, Automation and Robotics (ICCAR); Apr 20–23; Electr Network. NEW YORK: IEEE; 2020. pp. 574-80.

10.

Jiang

Chen

Liu

. Surface EMG data aggregation processing for intelligent prosthetic action recognition. Neural Comput Appl. 2020; 32(22): 16795-806. doi: 10.1007/s00521-018-3909-z.

11.

Wendelken

Page

Davis

Wark

HAC

Kluger

Duncan

, et al. Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves. J Neuroeng Rehabil. 2017; 14(1): 121-34. doi: 10.1186/s12984-017-0320-4.

12.

Vaskov

Irwin

Henning

Lueders

Laidlaw

, et al. A regenerative peripheral nerve interface allows real-time control of an artificial hand in upper limb amputees. Sci Transl Med. 2020; 12(533): 1-11. doi: 10.1126/scitranslmed.aay2857.

13.

Atzori

Cognolato

Muller

. Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands. Front Neurorobotics. 2016; 10(1): 1-10. doi: 10.3389/fnbot.2016.00009.

14.

Mukhopadhyay

Samui

. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed Signal Process Control. 2020; 55(2): 101669-76. doi: 10.1016/j.bspc.2019.101669.

15.

Zhang

Yang

Qian

Zhang

. Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network. Sensors (Basel). 2019; 19(14): 15. doi: 10.3390/s19143170.

16.

Bates

Fergason

Pierrie

. Technological Advances in Prosthesis Design and Rehabilitation Following Upper Extremity Limb Loss. Curr Rev Musculoskelet Med. 2020; 13(4): 485-93.

17.

Krausz

Lamotte

Batzianoulis

Hargrove

Micera

Billard

. Intent Prediction Based on Biomechanical Coordination of EMG and Vision-Filtered Gaze for End-Point Control of an Arm Prosthesis. IEEE Trans Neural Syst Rehabil Eng. 2020; 28(6): 1471-80. doi: 10.1109/tnsre.2020.2992885.

18.

Weiner

Starke

Hundhausen

Beil

Asfour

. The KIT Prosthetic Hand: Design and Control. 25th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); Oct 01-05; Madrid, SPAIN. NEW YORK: IEEE; 2018. pp. 3328-34.

19.

Lecun

Bottou

Bengio

Haffner

. Gradient-based learning applied to document recognition. Proc IEEE. 1998; 86(11): 2278-324.

20.

Krizhevsky

Sutskever

Hinton

. ImageNet classification with deep convolutional neural networks. Commun ACM (USA). 2017; 60(6): 84-90.

21.

Szegedy

Liu

Jia

Sermanet

Reed

Anguelov

, et al. Going Deeper with Convolutions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Jun 07-12 Boston, MA. NEW YORK: IEEE; 2015. pp. 1-9.

22.

Sun

. Convolutional Neural Networks at Constrained Time Cost. IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Jun 07-12 Boston, MA. NEW YORK: IEEE; 2015. pp. 5353-60.

23.

Kaiming

Xiangyu

Shaoqing

Jian

. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 IEEE; 2016. pp. 770-8.

24.

Shen

Albanie

Sun

. Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell. 2020; 42(8): 2011-23.

25.

Ceballos

Daya

. Endometrioid cystadenofibroma of the ovary with pelvic lymph node involvement. International Journal of Gyncological Pathology. 2006; 25(4): 344-6.

26.

Controzzi

Cipriani

Carrozza

. Design of artificial hands: A review. In: Balasubramanian

Santos

. (eds) editor. The Human Hand as an Inspiration for Robot Hand Development: Springer; 2014; pp. 219-46.

27.

Ghazaei

Alameer

Degenaar

Morgan

Nazarpour

. Deep learning-based artificial vision for grasp classification in myoelectric hands. J Neural Eng. 2017; 14(3): 18. doi: 10.1088/1741-2552/aa6802.

28.

Carlson

Hock

. Kinematic analysis of coupled arm prostheses. Journal of Biomechanical Engineering-Transaction of the ASME. 1977; 99(2): 110-5.

Design and control of intelligent bionic artificial hand based on image recognition

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1. Introduction

2. Overall design

Table 1 The distribution of the DOFs

Table 2 Labels and functions

5.1 Datasets collection

Table 3 Part of the dataset

Footnotes

Acknowledgments

Conflict of interest

Ethical approval

References

Table 1
The distribution of the DOFs

Table 2
Labels and functions

Table 3
Part of the dataset