Flexible Electronic Skin for Monitoring of Grasping State During Robotic Manipulation

Abstract

Electronic skin for robotic tactile sensing has been studied extensively over the past years, yet practical applications of electronic skin for the grasping state monitoring during robotic manipulation are still limited. In this study, we present the fabrication and implementation of electronic skin sensor arrays for the detection of unstable grasping. The piezoresistive sensor arrays have the advantages of facile fabrication, fast response, and high reliability. With the tactile data from the sensor array, we propose two quantitative indicators, correlation coefficient and wavelet coefficient, to identify grasping with variable forces and slippage. Those two indicators reflect both time and frequency domain characteristics in the contact forces from the sensor array and can be obtained without large amount of calculation. We demonstrate the utility of this method under various conditions, the results indicate grasping with variable forces, and slippage can be distinguished by this method. The flexible sensor arrays are adopted for tactile sensing on a bionic hand, and the effectiveness of this method in detecting various grasping states has been verified. The electronic skin sensor array and the grasping state monitoring method are promising for applications in robotic dexterous manipulation.

Introduction

Similar as the “sense of touch” to human, tactile sensing is critical for robots to understand and interact with real-world objects and environment.^1–5 Tactile sensing becomes an increasingly important capability for robots as their workplace has been shifting from structured industrial sectors to complex and dynamically changing environment.^6–10 For instance, robotic dexterous manipulation is a typical scenario where tactile sensation is critical as the touch condition is constantly evolving and the grasping state has to be continuously monitored.^11–15 Unstable grasping may lead to catastrophic failure and damage to operated objects, robots, as well as personnel.

Driven by the demand for robotic tactile perception, the development of electronic skin has been attracting significant interest over the past decade.^16–20 Inspired by human skin, electronic skin refers to flexible sensors that can detect force,^21–25 strain,^26–28 temperature,²⁹ etc.³⁰ And electronic skin with high sensitivity, spatial resolution, and that can cover large area have been reported.^31–36 However, due to the complexity in sensor data acquisition and analysis, especially for sensor arrays with a large number of units, electronic skin sensors actually adopted in robotic manipulation for grasping state monitoring are rather limited.

Flexible sensors that can measure both normal and shear forces have been developed to predict slippage during grasping^37–39; however, this approach requires the prior knowledge of friction coefficients of contact interfaces, which differ significantly for objects to be operated and can experience variations even on the surface of the same object.^40–42 In contrast, slippage detection approaches based on machine learning^43–45 and neural network^46,47 have been proposed through the analysis of the time and frequency domain features of sensor data. However, those approaches can be computationally costly due to the large amount of data, and the applications are typically limited to the scenarios similar to the training process.

To tackle these challenges, herein we design and fabricate flexible electronic skin sensor arrays and develop a highly effective approach for the monitoring of grasping state based on the sensor data. The piezoresistive sensor array and the data acquisition method can be easily scaled up. Due to the flexibility, the sensor array can be attached to curvy surfaces of robotic end effectors. In the grasping state monitoring approach, we propose two numerical indices by extracting both the time and frequency domain characteristics of the sensor array signals. Unstable grasping states, such as grasping with variable forces and slippage, can be distinguished by the magnitude of those two indices. The effectiveness of this method is demonstrated during various grasping tasks of a bionic hand. Owing to facile fabrication, easy implementation, and low computational cost, the flexible sensor array and the grasping state monitoring method are promising for applications in robotic dexterous manipulation.

Methods and Results

Fabrication of electronic skin sensor array and characterizations

We fabricated the electronic skin sensor array through the development of a flexible and porous piezoresistive film, laying out column and row electrode lines, followed by encapsulation using polymeric thin film. The piezoresistive film is porous thermoplastic polyurethanes matrix (TPU) doped with conductive carbon black (CB) particles. A piece of square film of 25 × 25 × 0.6 mm size was produced for the fabrication of a 5 × 5 sensor array. The intersection of each horizontal and longitudinal electrode line is a piezoresistive sensor unit, as shown in Figure 1a. Details of the fabrication processes can be found in the Supporting Note S1.

FIG. 1.

(a) Conductive wires as electrodes arranged longitudinally and horizontally on the front and back surface of the piezoresistive film, respectively, for a 5 × 5 sensor array; (b) SEM image of the cross section of the piezoresistive film, showing the porous structures; (c) photograph of a piezoresistive film during bending by two fingers; (d) conductivity of composites fabricated with different weight ratios of CB and TPU; (e) force-resistance curve of a piezoresistive sensor unit. CB, carbon black; SEM, standard error of the mean; TPU, thermoplastic polyurethanes matrix.

We adopted different weight ratios of CB and TPU, from 0.01:1 to 0.4:1, to fabricate the piezoresistive films and tested their force-resistance response. It can be seen from Supplementary Figure S1 that the ratio of percolation thresholds to achieve conductivity in this type of material combination is 0.03:1. The resistance drops rapidly when the ratio of CB exceeds the threshold. The measurement sensitivity of a piezoresistive sensor is defined as $S = Δ R ∕ R_{0} Δ P$ , where $Δ P$ is the change in applied force, $Δ R$ and R₀ are the change of resistance and original resistance, respectively.

Sodium chloride (NaCl) particles were added to the mixed powder to render the film porous and flexible. After dissolution of the NaCl particles, the porous structure results in lower modulus and larger deformation, and thus, larger change of resistance will occur under the same loading condition, leading to higher measurement sensitivity and high flexibility, as shown in Figure 1b and c. We adopted 1:1 weight ratio of NaCl and TPU powder to achieve the balance between structure integrity and measurement sensitivity. The conductivity of the composites composed of different weight ratios of CB and TPU is shown in Figure 1d. The best measurement performance of the film appears when the ratio of CB, NaCl, TPU is 0.3:1:1, which enables highly linear response and rather high sensitivity of 0.13/N, as shown in Figure 1e and Supplementary Figure S1.

The performances of the piezoresistive sensor were characterized using a unit in the 5 × 5 sensor array. The resistance of the unit varies linearly with applied force. The original resistance is 1400 Ω and drops to 495 Ω when the applied force is 5 N (Fig. 1e). The force-resistance curves of the units in the 5 × 5 sensor array are similar and individually calibrated before applications. The sensor also maintains high stability and repeatability as the resistance keeps almost constant after 2000 load-release cycles as shown in Supplementary Figure S2. Although we only demonstrate the 5 × 5 sensor array, the sensor array can be easily scaled up to large areas with large number of units. It is noted that sensor array can be obtained through facile fabrication schemes, and the flexibility allows easy attachment to even curvy surfaces.

As illustrated in Figure 2a, the equipotential circuit approach is adopted to collect the force data applied on the array. The details of process of data collection are described in Supporting Note S2. It is noted that only m + n wires are needed to cover m × n sensor units using the equipotential method, while 2 m × n connecting wires will be needed through traditional method to connect each electrode, which will significantly increase the complexity of data processing circuit. Figure 2b shows the pressure distribution images when a triangle and a pentagon wood block loaded by an iron cube was placed on the 5 × 5 array, respectively. It is observed that the force map is consistent with object shape with low level of cross talk. And the total forces from the sensor units are 2.241 and 2.303 N, respectively; both agree with the applied force of 2.254 N by the wood piece and iron cube (total weight 230 g), indicating the accuracy of the measurements. Moreover, the cross talk between sensing units in the array is also minimum, as shown in Supplementary Figure S4.

FIG. 2.

(a) Circuit design for force data acquisition from the 5 × 5 sensor array; (b) distribution of applied force when a triangle and a pentagon wood block are placed on the 5 × 5 array, respectively; (c) continuous force data from 5 U under different conditions.

Figure 2c presents the force data on five sensor units when a wood cube was pushed on the sensor array with variable forces without sliding, and sliding against the array. The experiment details are shown in Supplementary Figure S3. It is clear that there is difference in contact forces for the two cases. Typically, large variation in contact forces can occur during both variable forces without sliding and sliding against array; however, it is difficult to distinguish those two conditions based on the magnitude of force data.

Method for monitoring robotic grasping state

Robotic manipulation can be very challenging as various conditions can occur in highly dynamic environment when the manipulator performs grasping tasks.^48,49 In a stable grasping process, the force at each contact point remains stable, although tiny fluctuations may happen. The contact force may change as the manipulator follows the command instructions to open or close, and the change can be predicted as it occurs when the operation is executed. In this study, the unstable grasping state considered includes two types of unexpected conditions. The first is that the grasping is subject to external force interferences but the contact between the robot and grasped object is still sustained, and the other case is that the force applied is insufficient to provide enough friction to maintain contact, and slippage occurs at the contact point. Those two unstable grasping events need to be detected in real time and avoided for the safety of operation.

As shown in Figure 2d, when the electronic skin sensor array is adopted, the unstable grasping state typically causes short-term variation on the contact forces. To identify those unstable grasping state, it is necessary to propose quantitative indicators which can comprehensively represent the data changes of all sensor units and directly correlate to grasping state. In this study, we come up with two indicators which reflect both time and frequency domain changes in the contact forces from the sensor array. The complex change in contact forces during unstable state is manifested in the drastic change of the center of force (CoF) and the pressure distribution. In time domain analysis, we apply the calculation of correlation coefficient of the pressure distribution vector as the indicator for monitoring grasping state. In time-frequency domain analysis, we calculate CoF and process it with one-dimensional discrete wavelet transformation (DWT) to obtain the second indicator. The force data from the 5 × 5 array is adopted to illustrate the derivation of those two indicators.

Analysis in time domain

Correlation coefficient of the pressure distribution vector, which represents real-time changes in contact force distribution, is calculated from distribution vectors of the sensor array at two adjacent detection moments. Arrange each row of the 5 × 5 array sensor data collected at t₀ in order and convert it into a vector F_dis,0 with 25 elements for correlation calculation. F_dis,0 represents the ratio of the pressure F_i,0 at each unit to the total force F_{sum, 0} on the sensor array at moment t₀: $F_{s u m, 0} = \sum_{i = 1}^{n} F_{i, 0} (i = 1, 2, 3, \dots, 25)$ $F_{d i s, 0} (i) = \frac{F_{i, 0}}{F_{s u m, 0}}$

F_dis,1 is calculated in the same manner at t₁ (t₁ = t₀ + 100 ms), and the correlation coefficient is calculated based on the two vectors: $c o e_{t_{0}, t_{1}} = \frac{c o v (F_{d i s, 0}, F_{d i s, 1})}{\sqrt{v a r [F_{d i s, 0} {]}\cdot v a r{[} F_{d i s, 1}]}}$

$c o e_{t_{0}, t_{1}}$ is the correlation coefficient of two adjacent data acquisition time t₀ and t₁, cov represents the covariance of two vectors, and var represents the variance of each vector. The value of coe tends to be 1 during stable grasping and will fluctuate during variable forces or slippage. To highlight the difference from stable state, the value (1−coe) is used to indicate the pressure distribution change. The correlation coefficients within the period [t₁−400 ms, t₁] are calculated and are summed up to obtain Sum_coe,1: $S u m_{c o e, 1} = \sum_{i = 1}^{N} (1 - c o e [i])$

where N is the total length of data collected in the 400 ms period. Repeat the above calculation after 100 ms at t₂ (t₂ = t₁ + 100 ms) to obtain Sum_coe,2 and then calculate the relative change: $R a t i o_{c o e, t_{2} ∕ t_{1}} = \frac{S u m_{c o e, 2}}{S u m_{c o e, 1}}$

It is noted that a time segment of 400 ms is adopted to calculate $S u m_{c o e}$ to cover a rather long time segment to reduce the impact of measurement noise, while the calculation of Ratio_coe is refreshed once every 100 ms to capture any unstable state in a responsive manner.

Analysis in time-frequency domain

It has been known that high frequency signals are generated when unstable grasping such as slippage occurs during robotic manipulation, which may be utilized as the indicator of the grasping state.⁵⁰ Fourier transformation is widely used for the analysis of frequency features. However, the time characteristics of the signal is lost in Fourier transformation, that is, the time of occurrence of unstable grasping cannot be obtained using Fourier transformation. In this study, we use DWT to obtain both the temporal and frequency features in the sensor signal. Each sensor unit in the sensor arrays has a virtual coordinate based on the sampling order. We define CoF as the virtual center of the applied forces: $X_{c} = \sum_{i = 1}^{n} x_{i} \cdot F_{i} ∕ F_{s u m}$ $Y_{c} = \sum_{i = 1}^{n} y_{i} \cdot F_{i} ∕ F_{s u m}$

C o F = \sqrt{X_{c}^{2} + Y_{c}^{2}}

where F_i is the measured force of each unit, F_sum is the total force, x_i and y_i are the coordinates of each unit, and X_c and Y_c are the coordinates of the force center along two dimensional orthogonal directions. Daubechies wavelet is adopted to perform DWT on the CoF data sequence in the period [t₁−400 ms, t₁], and the approximate coefficient dw_1,L and the detail coefficient dw_1,H are calculated by Mallat algorithm⁵¹: $d w_{1, L} [n] = \sum_{a = 1}^{K} C o F [2 n - a] \cdot g (a) (n < \frac{N}{2})$ $d w_{1, H} [n] = \sum_{a = 1}^{K} C o F [2 n - a] \cdot h (a) (n < \frac{N}{2})$

where g and h are the low-pass filter and high-pass filter of Daubechies wavelet function, K is the length of the filter, and N is the total length of the data to be decomposed. The absolute value of detail coefficient represents the magnitude of high frequency component at the moment. During unstable grasping state, there will be drastic change in the CoF and thus large increase in dw_1,H. We adopt V_center, the square of dw_1,H, to enlarge the fluctuation: $V_{c e n t e r} [n] = d w_{1, H} {[n]}^{2}$

and the sum of all the V_center values calculated during the [t₁−400 ms, t₁] period by this part of the data is defined as Sum_dw,1; $S u m_{d w, 1} = \sum^{V_{c e n t e r}} [n]$

Repeat the above calculation at t₂ (t₂ = t₁ + 100 ms) to obtain Sum_dw,2 and then calculate the relative change: $R a t i o_{d w, t_{2} ∕ t_{1}} = \frac{S u m_{d w, 2}}{S u m_{d w, 1}}$

The relative amplitude change, Ratio_dw and Ratio_coe, is used to detect the unstable grasping state. For the stable state, there is little force difference between data at the two periods analyzed; therefore, these two ratios are close to 1. However, in the unstable grasping state, the two ratios will far exceed 1. To focus on the difference between the unstable state and the stable cases, the above two ratios are processed as described below to achieve state indicators R_dw and R_coe: $R_{d w} = {(R a t i o_{d w} - 1)}^{2}$ $R_{c o e} = {(R a t i o_{c o e} - 1)}^{2}$

R_dw and R_coe are both close to zero during stable grasp and increase during unsteady grasp. It is noted that R_coe represents the spatial change of grasping force distribution, while R_dw also captures the variation of applied forces in frequency domain during unstable grasping. In addition, the time of large change in R_dw corresponds to the occurrence time of grasping state change due to the adoption of wavelet analysis. Therefore, the combination of R_dw and R_coe represents a comprehensive approach for unstable state detection as together they take account of the variation in both time and frequency domain.

Demonstration of the algorithm in detecting unstable states

We set up several scenarios to test the effectiveness of the algorithm in the detection of unstable contact between the sensor array and objects. Briefly, a cuboid wood block (size 20 × 20 × 20 mm) was placed on the 5 × 5 sensor array, and then the applied force on the sensor array by the block was adjusted in the range of 4–10 N. It is noted that the block covers at least 15 sensor units on the array. The block was pulled or pushed by a linear motor (LinMot HF01-23) at several constant speeds to create sliding with the array. The contact force and sliding speed were set up at different values for the following cases:

Case 1: Applied force varies from 6 to 10 N without sliding;

Case 2:Applied force 8 N, slide once with speed 10 mm/s;

Case 3:Applied force 8 N, slide once with speed 3 mm/s;

Case 4: Applied force 4 N, slide once with speed 10 mm/s;

Case 5: Applied force 4 N, slide once with speed 3 mm/s;

Case 6: Applied force 6 N, slide four times back and forth with speed 5 mm/s.

It can be seen from Figure 3a that when the sensor array is subjected to variable pressure but no slippage occurs, the correlation coefficient has relatively large variation due to the large change in magnitude of the applied forces. The amplitude of the wavelet detail coefficient is high, leading to the rather high amplitude of $V_{c e n t e r}$ , which agrees with the fact that high frequency component in the sensor signal is present due to the force change. The two indicators R_dw and R_coe have a high amplitude at 3.8 s, corresponding to the initial application of external forces. Both R_coe and R_dw fluctuate afterward, reflecting the varied forces.

FIG. 3.

Data calculated from the sensor array at different scenarios. (a)–(f) Correspond to Case 1–6, respectively. The five graphs in each row correspond to the calculation results of applied force, CoF, coe, Vcenter, and two ratios of the sensor array data; the horizontal axis in each figure is time in the unit of second.

In case 2–6, corresponding to Figure 3b–f, it is clear that slippage occurs, and the time of occurrence can be identified from the two indicators (9.4 s for case 2, for instance). Generally, there are two major peaks in the figures of coe, $V_{c e n t e r}$ , and ratios. The first peak is caused by initial contact, which leads to a drastic change in the pressure center and pressure distribution. This change can be predicted by the control command. The second peak is caused by the block sliding against the sensor array. It can be seen that larger magnitude in applied force or sliding speed corresponds to higher amplitude of R_dw and R_coe. When the applied force is large (8 N) and the slip speed is high (10 mm/s), the amplitudes of $V_{c e n t e r}$ , coe, and two ratios shown in Figure 3b are the largest among all six conditions. The decrease in applied force or sliding speed will lead to the decrease in R_dw and/or R_coe (case 3–5).

There are four sliding peaks in the Figures 3f (case 6), indicating that the algorithm remains effective even in the presence of multiple sliding. Based on the results in Figure 3, it is found that variable force without slippage (case 1) leads to relatively lower magnitude of R_dw and R_coe, and when slippage occurs both the ratios exceed 2.5. This threshold may be used to distinguish variable force condition and slippage.

Detection of unstable grasping by a bionic hand

We further demonstrate the applications of the electronic skin sensor array and the utility of the unstable state detection approach in the manipulation tasks by a bionic hand. To measure the grasping forces, the sensor array was fabricated to fit the size of the hand. The grasping experiment setup is shown in Figure 4a. The system consists of a UR5 robotic arm (Universal Robot Co., LTD), and a bionic hand is installed as the end effector. The bionic hand has five fingers with six degree of freedom for manipulation tasks. During grasping, the areas in contact with objects are mainly from the fingertips of the index, middle, ring, and little fingers and the palm. Therefore, four 1 × 5 sensor arrays were arranged on the four fingertips as shown in Figure 4a and Supplementary Figure S5 to measure the grasping forces, forming a 4 × 5 sensor array. Due to the flexibility, the sensor array could be readily attached to the fingertips. The bionic hand was programmed to go through six actions during the manipulation of a glass of water (Fig. 4b):

FIG. 4.

(a) Experiment setup for the manipulation tasks of the bionic hand; (b) actions performed by the bionic hand during the task; (c–f) is the calculated force, CoP, coe, V_center during the manipulation of a glass of water. (g) The evolution of the two indicators during the task. Actions 2–6 are marked in the graph, and the red lines correspond to the beginning of each action.

(1)

Approach the glass;

(2)

Grab the glass;

(3)

Pick up the glass;

(4)

Pour a little water into the glass;

(5)

Move the glass horizontally back and forth;

(6)

Pour more water until the glass slips.

The details of data analysis are shown in Figure 4c–f and Supplementary Figure S6. The variation of R_dw and R_coe is shown in Figure 4g. Supplementary Video S1 shows the whole process.

Before grasping, the sensor array is not in contact with the glass in Action 1 and the two ratios maintain 0. A peak occurs in both R_dw and R_coe at Action 2 due to the force applied when grabbing the glass. The applied forces are not changed during the pickup of the glass; thus, the ratios are 0 at Action 3, indicating a stable state. There is a low peak of R_coe at Action 4 due to the flow-in of the small amount of water. At Action 5, the horizontal motion of the hand leads to acceleration and deceleration of the glass, resulting in relatively large variation in the handling force (Fig. 4c). The sensor array can effectively detect the fluctuation as there is notable increase in both R_dw and R_coe. As more water is gradually poured into the glass, the handling force is not enough to overcome gravity at Action 6 and slippage occurs, which leads to a dramatic decrease in handling force and significantly high peaks in both R_dw and R_coe.

In this manipulation task, the evolution of the two indicators is in line with expectations, verifying the accuracy and effectiveness of detection algorithm. Consistent with the results shown in Figure 3, Action 5 leads to variable handling forces with no relative sliding; thus, the increase in the magnitude of R_dw and R_coe is rather limited, while slippage causes dramatic peaks in those two indicators in Action 6.

Conclusion

In summary, we have presented the fabrication and implementation of electronic skin sensor arrays for the detection of unstable grasping during robotic manipulation. The piezoresistive sensor arrays have the advantages of facile fabrication, fast response, and high reliability, while the equipotential circuit approach enables highly efficient data acquisition. The sensor array and the data acquisition method can be easily scaled up for a large number of sensing units. To reduce the complexity in tactile data analysis, we propose two quantitative indicators, correlation coefficient and wavelet coefficient, to identify unstable grasping states such as grasping with variable forces and slippage. Those two indicators reflect both time and frequency domain changes in the contact forces from the sensor array and can be obtained without large amount of calculation. With a low-cost microprocessor for signal analysis, the response time for unstable state monitoring can be as short as 100 ms in this study, which can be compared favorably with previous studies using data-intensive deep neural networks or sophisticated data acquisition systems, as shown in Supplementary Table S1.

We demonstrate the utility of this method under various unstable grasping conditions, and the results indicate that variable grasping force and slippage can be distinguished by this method. It is noted that the sensor array can be facilely fabricated; both the sensor array and data acquisition can be easily scaled up. Due to the low computation cost, the grasping state detection method demonstrates satisfactory performances with a low-cost microprocessor and no advanced electronic components. With dedicated data acquisition and analysis system, such as the embedded system, the efficiency of detection can be significantly improved. The flexible sensor arrays are adopted for tactile sensing on a bionic hand, and the effectiveness of this method in detecting various grasping states has been verified during manipulation tasks. The flexible electronic skin and the method for detecting unstable grasping have great potential for practical applications in robotic manipulation.

Footnotes

Acknowledgment

The authors acknowledge the support from Flexible Electronics Research Center of HUST for providing experiment facility.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

We acknowledge the support from National Natural Science Foundation of China (nos. U2013213, 51820105008, 92048302), the Technology Innovation Project of Hubei Province of China under grant 2019AEA171.

Supplementary Material

References

Johansson

, Flanagan

. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci, 2009; 10:345–359.

Yousefa

, Boukallela

, Althoeferb

. Tactile sensing for dexterous in-hand manipulation in robotics—a review. Robot Auton Syst, 2011; 167:171–187.

Tegin

, Wikander

. Tactile sensing in intelligent robotic manipulation—a review. Ind Robot, 2010; 32:64–70.

Dahiya

, Metta

, Valle

, et al. Tactile sensing—From humans to humanoids. In: IEEE Transactions on Robotics. IEEE,, 2010; 26:1–20.

, Xiao

, Yang

, et al. Stretchable triboelectric–photonic smart skin for tactile and gesture sensing. Adv Mater, 2018; 30:1800066.

Bartolozzi

, Natale

, Nori

, et al. Robots with a sense of touch. Nat Mater, 2016; 15:921–925.

Chen

, Khamis

, Birznieks

, et al. Tactile sensors for friction estimation and incipient slip detection—towards dexterous robotic manipulation: a Review. IEEE Sens J, 2018; 18:9049–9064.

Dollar

, Howe

. Simple, robust autonomous grasping in unstructured environments. In: 2007 IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2007:4693–4700.

Jiang

, Li

, Wu

, et al. Grab and heat: highly responsive and shape adaptive soft robotic heaters for effective heating of objects of three-dimensional curvilinear surfaces. ACS Appl Mater Interfaces, 2019; 11:47476–47484.

10.

Zhao

, Bu

, Zhang

, et al. Intrinsically stretchable organic-tribotronic-transistor for tactile sensing. Research, 2020; 2020:1398903.

11.

Deimel

, Brock

. A novel type of compliant and underactuated robotic hand for dexterous grasping. Int J Robot Res, 2016; 35:161–185.

12.

Billard

, Kragic

. Trends and challenges in robot manipulation. Science, 2019; 364:eaat8414.

13.

Mahler

, Matl

, Goldberg

, et al. Learning ambidextrous robot grasping policies. Sci Robot, 2019; 4:eaau4984.

14.

, Qi

, Hu

, et al. Towards model-free tool dynamic identification and calibration using multi-layer neural network. Sensors, 2019; 19:3636.

15.

Huang

, Guo

, Liu

, et al. Flexible mechanical metamaterials enabled electronic skin for real-time detection of unstable grasping in robotic manipulation. Adv Funct Mater, 2022; 2109109.

16.

Liu

, He

, Chen

, et al. Nature-inspired structural materials for flexible electronic devices. Chem Rev, 2017; 117:12893–12941.

17.

Hammock

, Chortos

, Bao

, et al. 25th anniversary article: the evolution of electronic skin (e-skin): a brief history, design considerations, and recent progress. Adv Mater, 2013; 25:5997–6038.

18.

, Kim

. Flexible and stretchable electronics paving the way for soft robotics. Soft Robot, 2014; 1:53–62.

19.

Dahiya

. E-skin: from humanoids to humans. P IEEE, 2019; 107:247–252.

20.

Zeng

, Zhao

, Li

, et al. Triboelectric nanogenerators as active tactile stimulators for multifunctional sensing and artificial synapses. Sensors, 2022; 22:975.

21.

Gong

, Wang

, Cheng

, et al. A wearable and highly sensitive pressure sensor with ultrathin gold nanowires. Nat Commun, 2014; 5:3132.

22.

Zhang

, Liu

, Wu

, et al. A stretchable dual-mode sensor array for multifunctional robotic electronic skin. Nano Energy, 2019; 62:164–170.

23.

Yao

, Wang

, Wu

, et al. Bioinspired triboelectric nanogenerators as self-powered electronic skin for robotic tactile sensing. Adv Funct Mater, 2019; 30:1907312.

24.

Tang

, Wu

, Gan

, et al. Multilevel microstructured flexible pressure sensors with ultrahigh sensitivity and ultrawide pressure range for versatile electronic skin. Small, 2019; 15:1804559.

25.

Zhou

, Ji

, Wei

, et al. A bio-inspired cilia array as the dielectric layer for flexible capacitive pressure sensors with high sensitivity and a broad detection range. J Mater Chem A, 2019; 7:27334–27346.

26.

, Wu

, Pan

, et al. Anti-freezing and moisturizing conductive hydrogels for strain sensing and moist-electric generation applications. J Mater Chem A, 2020; 8:3109–3118.

27.

Muth

, Vogt

, Truby

, et al. Embedded 3D printing of strain sensors within highly stretchable elastomers. Adv Mater, 2014; 26:6307–6312.

28.

Gao

, Li

, Wu

, et al. Laser direct writing of ultrahigh sensitive SiC-based strain sensor arrays on elastomer toward electronic skins. Adv Funct Mater, 2019; 29:1806786.

29.

Yang

, Zhao

, Cheng

, et al. Stretchable and healable conductive elastomer based on PEDOT:PSS/natural rubber for self-powered temperature and strain sensing. Acs Appl Mater Inter, 2020; 13:14612–14624.

30.

Zhu

, Lou

, Ding

, et al. Highly shape adaptive fiber based electronic skin for sensitive joint motion monitoring and tactile sensing. Nano Energy, 2020; 69:104429.

31.

Lee

, Franklin

, Someya

, et al. Nanomesh pressure sensor for monitoring finger manipulation without sensory interference. Science, 2020; 370:966–970.

32.

Won

, Park

, Lee

, et al. Stretchable and transparent kirigami conductor of nanowire percolation network for electronic skin applications. Nano Lett, 2019; 19:6087–6096.

33.

Wang

, Xu

, Bao

, et al. Skin electronics from scalable fabrication of an intrinsically stretchable transistor array. Nature, 2018; 555:83–88.

34.

Lee

, Tan

, Tee

BCK

, et al. A neuro-inspired artificial peripheral nervous system for scalable electronic skins. Sci Robot, 2019; 4:eaax2198.

35.

Wang

, Zhang

, Wang

, et al. Self-powered high-resolution and pressure-sensitive triboelectric sensor matrix for real-time tactile mapping. Adv Mater, 2016; 28:2896–2903.

36.

Zhao

, Guo

, Pang

, et al. Flexible organic tribotronic transistor for pressure and magnetic sensing. ACS Nano, 2017; 11:11566–11573.

37.

Boutry

, Negre

, Vardoulis

, et al. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci Robot, 2018; 3:eaau6914.

38.

Wang

, Chen

, Mei

. Flexible Tactile sensor array for slippage and grooved surface recognition in sliding movement. Micromachines, 2019; 10:579.

39.

Sun

, Sun

, Li

, et al. Flexible tactile electronic skin sensor with 3D force detection based on porous CNTs/PDMS nanocomposites. Nano-Micro Lett. 11:57.

40.

Kaboli

, Cheng

. Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin. IEEE T Robot, 2018; 34:985–1003.

41.

Chuang

, Liou

, Chen

. Detection system of incident slippage and friction coefficient based on a flexible tactile sensor with structural electrodes. Sensor Actuat A-Phys, 2012; 188:48–55.

42.

Kaboli

, Walker

, Cheng

. Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2016:2242–2247.

43.

James

, Pestell

, Lepora

. Slip detection with a biomimetic tactile sensor. In: IEEE Robotics and Automation Letters. IEEE,, 2018; 3:3340–3346.

44.

Stabilizing novel objects by learning to predict tactile slip. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,, 2015:5065–5072.

45.

Zapata-Impata

, Gil

, Torres

. Learning Spatio temporal tactile features with a ConvLSTM for the direction of slip detection. Sensors, 2019; 19:523.

46.

, Dong

, Adelson

. Slip detection with combined tactile and visual information. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE,, 2018:7772–7777.

47.

Shirafuji

, Hosoda

. Detection and prevention of slip using sensors with different properties embedded in elastic artificial skin on the basis of previous experience. 2011 15th International Conference on Advanced Robotics (ICAR). IEEE,, 2011:459–464.

48.

Kappassov

, Corrales

, Perdereau

. Tactile sensing in dexterous robot hands—review. Robot Auton Syst, 2015; 74:195–220.

49.

Sanchez

, Bouzgarrou

, Mezouar

, et al. Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. Int J Robot Res, 2018; 37:688–716.

50.

Choi

, Kuchenbecker

. Vibrotactile display: perception, technology, and applications. P IEEE, 2013; 101:2093–2104.

51.

Mallat

. A theory for multiresolution signal decomposition: the wavelet representation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE,, 1989; 11:674–693.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

2.70 MB

73.62 MB

0.00 MB