Design of physical user–robot interactions for model identification of soft actuators on exoskeleton robots

Abstract

Recent breakthroughs in wearable robots, such as exoskeleton robots with soft actuators and soft exosuits, have enabled the use of safe and comfortable movement assistance. However, modeling and identification methods for soft actuators used in wearable robots have yet to be sufficiently explored. In this study, we propose a novel approach for obtaining accurate soft actuator models through the design of physical user–robot interactions for wearable robots, in which the user applies external forces to the robot. To obtain an accurate soft actuator model from the limited amount of data acquired through an interaction, we leverage an active learning framework based on Gaussian process regression. We conducted experiments using a two-degree-of-freedom upper-limb exoskeleton robot with four pneumatic artificial muscles (PAMs). Experimental results showed that physical interactions between the exoskeleton robot and the user were successfully designed to allow PAM models to be identified. Furthermore, we found that data acquired through an interaction could result in more accurate soft actuator models for the exoskeleton robots than data acquired without a physical interaction between the exoskeleton robot and the user.

Keywords

Soft actuators exoskeleton robots physical human–robot interactions human-in-the-loop Gaussian process active learning

1. Introduction

Wearable robots can be used in various situations, including assistance with daily life or with rehabilitation (Dollar and Herr, 2008; Gopura et al., 2016; Yan et al., 2015). Recent studies have developed exoskeleton robots with soft actuators and soft exosuits. These soft actuators, which include series elastic actuators (SEAs) (Wang et al., 2015; Yu et al., 2015), variable stiffness actuators (VSAs) (Mghames et al., 2017) or pneumatic artificial muscles (PAMs) (Cao et al., 2018; Ferris et al., 2005; Liu et al., 2017), have attracted significant attention owing to their safe and compliant physical properties suitable for wearable robots. The soft exosuits (Asbeck et al., 2015; Ding et al., 2018) are composed of soft and lightweight textiles, and can thus easily fit to the shape of the user’s body.

However, modeling and identification methods for soft actuators used in wearable robots have yet to be sufficiently explored. In these actuators or suits composed of soft materials, force transmission mechanisms tend to introduce highly nonlinear dynamics (Asbeck et al., 2015; Lagoda et al., 2010; Liu et al., 2017; Mghames et al., 2017). Moreover, they are stretched by external forces because of the compliance. Therefore, rich datasets generated using various external forces are required to identify such models that include nonlinearity and compliance. The standard approach is to apply a load to an actuator by placing various weights (Merola et al., 2018), or to employ antagonistic muscles (Teramae et al., 2013) to generate external forces. However, such approaches do not necessarily take into account actual situations in which the robot is worn by a human user.

In our approach, the external forces applied to soft actuators are generated through user–robot physical interactions to tackle this model identification problem. In other words, we developed a user–robot interaction design strategy to efficiently generate external forces for identifying soft actuator models from a limited number of data collection trials. Accordingly, we collected data under versatile circumstances close to those in practical use for the model identification, and avoided collecting unnecessary data through unnecessary experimental trials.

Figure 1 shows a schematic diagram of our proposed method. We designed the user–robot interactions by providing a human user with a way to generate an external force based on the current status (e.g., model uncertainty) of the learned model. Our proposed system designs action patterns for the human user and robot based on the current soft actuator model, and then asks the user to execute the actions by physically moving the wearable robot. Because the action patterns for the human user and robot are different, the user ends up applying external forces to the soft actuators equipped in the wearable robot. The data generated through physical user–robot interaction are used to update the soft actuator model.

Fig. 1.

Schematic diagram of our proposed method. We designed user–robot interactions by providing the human user with a way to generate external force based on the current status (e.g., model uncertainty) of the learned model. Our proposed system designs action patterns for the human user and robot based on the current soft actuator model, and then asks the user to execute actions by physically moving the wearable robot. Because action patterns for the human user and robot are different, the user ends up applying external forces to soft actuators equipped in the wearable robot. Data generated through physical user–robot interaction are used to update the soft actuator model.

In this study, we used PAMs as the soft actuators. To design the user–robot interaction and acquire a soft actuator model, we leveraged a Gaussian process (GP) regression and GP active learning framework. GP can express the model uncertainty as the predictive variance and nonlinearity of a PAM. GP active learning sequentially selects a data point, which reduces the model uncertainty. It can be used to accelerate the PAM model identification as well as avoid overfitting. In our learning framework, the reference trajectories that maximize the information gain are provided to the human user and robot. The user then tries to follow the provided trajectory to generate external forces used to update the GP-based PAM (GP-PAM) model efficiently.

In our preliminary study (Hamaya et al., 2017b), we explored how the GP active learning framework can be useful for soft actuator identification using a simple one-degree-of-freedom (one-DoF) elbow-joint assistive robot (Noda et al., 2013). Although the GP active learning framework was successfully applied, the applicability of our previous method was limited to identifying only one PAM model attached to the one-DoF assistive robot, owing to the necessity of manually tuning GP the hyperparameters that determine the shape of the approximated function.

In this study, we newly propose a GP active learning framework that can be applicable to further practical situations in which a multi-DoF robot and multiple actuators are involved. The contributions of this study are as follows.

We developed a systematic procedure to simultaneously identify multiple soft actuator models on a wearable robot through a physical human–robot interaction.

We newly adopted an active learning method that can learn the hyperparameters of the GP-PAM models such that the shape of the approximated function can be adaptively learned through the learning process.

We conducted model identification experiments using a 2-DoF upper-limb exoskeleton robot with four PAMs. The results demonstrate that our method outperforms standard model identification approaches that do not take the user–robot interactions into account.

Our proposed method is composed of two components: the human-in-the-loop optimization and GP active learning. However, they are not independent contributions; rather, its combination is essentially important because our human-in-the-loop data collection process requires the criterion of model uncertainty, and it requires the GP-PAM model owing its capability for evaluating the model uncertainty.

The remainder of the paper is organized as follows. Section 2 describes related studies regarding physical user–robot interaction designs and PAM-driven exoskeleton robots. Section 3 introduces our proposed method. Section 4 describes our experimental setup. Section 5 provides the experimental results. Section 6 discusses an additional off-line analysis. Finally, Section 7 provides some concluding remarks regarding this research.

2. Related work

In this section, we describe recent physical user–robot interaction and user-in-the-loop optimization methods for wearable robots. We also present PAM-driven exoskeleton robots and discuss the model identification methods developed.

2.1. Physical user–robot interaction design

Physical user–robot interactions have been investigated in collaborative manipulation tasks. Ghadirzadeh et al. (2016) proposed the modeling of physical user–robot collaborations, and the use of such modeling to select the control output through Bayesian optimization. Rozo et al. (2016) developed a task-parameterized interaction model and a stiffness estimation method for a transport task. Peternel et al. (2016b) utilized human user’s electromyography (EMG) to modulate the joint stiffness of a robot for a collaborative cut-sawing task. Bajcsy et al. (2017) proposed estimating the objective function of a manipulation task by observing how a user physically guides a robot to generate hand trajectories. Maeda et al. (2017) estimated the user’s movement intentions and generate robot joint trajectories through the use of dynamic movement primitives in handover tasks. Khoramshahi and Billard (2018) employed task-parameterized dynamical systems and switched the dynamical systems according to the user’s movement intentions. The above methods designed interactions for completing collaborative tasks. The robots optimize their behavioral strategies according to the users’ movement intentions. In contrast, in our study, human users provide necessary data to identify a soft actuator model according to the uncertainty of the actuator model learned using the robots.

A way for designing physical user–robot interactions to enhance the user’s motor performance has also been recently explored. User-in-the-loop frameworks have been proposed to optimize the assistive strategies through physical user–robot interactions. These user-profiling assistive strategies have successfully been used to reduce metabolic costs when walking (Ding et al., 2018; Koller et al., 2016; Zhang et al., 2017), reduce muscle activities (Hamaya et al., 2017a), or generate preferred walking movements on an active prosthesis (Thatte et al., 2017).

In this study, we utilized a user-in-the-loop framework for the actuator model identification, rather than for optimizing assistive strategies. The user-in-the-loop framework can be extremely useful in acquiring model identification data because the identified model needs to represent the actuator behaviors when a user is wearing a robot. However, difficulties are encountered when using the user-in-the-loop framework for the model identification, because we need to ask human users to conduct many movement trials to provide a sufficient amount of data. In addition, it is not clear what data collection methods of user–robot physical interaction data are required to improve the actuator models.

To solve these problems, we propose an active-learning based approach using GP models (Garnett et al., 2014) to efficiently design a physical user–robot interaction and provide concrete instructions to the users regarding how to generate data for model identification such that the human users do not need to conduct too many movement trials.

2.2. PAM-driven exoskeleton robots

PAMs have attracted significant attention as an actuator for a wearable robot owing to their compliance and safety with a high power-to-weight ratio. Ferris et al. (2005) proposed controlling the PAMs in proportion to the amplitude of the user’s EMG signals without conducting model identification. Liu et al. (2017) used a static PAM model under the assumption that the muscle tension linearly changes with respect to the input pressure at the same muscle length. The parameters of the model were identified through modulation of the muscle length, contraction force, and pressure. The antagonistic muscle was used to provide external forces for the identification process.

Cao et al. (2018); Merola et al. (2018); Park et al. (2014) used linear dynamical models. Teramae et al. (2018) used a static PAM model proposed for position control and employed a PAM model identification method proposed by Teramae et al. (2013). Ugurlu et al. (2015) proposed the use of nonlinear dynamical PAM models and applied them to the control of a lower-limb exoskeleton robot Ugurlu et al. (2016). Koller et al. (2016), Peternel et al. (2016a), and Hamaya et al. (2017a) developed black-box optimization methods for the control of PAM-driven exoskeleton robots without conducting PAM model identification.

As introduced above, most of the previously proposed PAM-driven exoskeleton robot control methods have used parametric PAM models to determine the pressure command to generate the desired contraction force. Otherwise, PAM controllers are directly derived through learning trials to accomplish the given tasks without identifying the PAM model. These parametric PAM models, which are typically derived through a consideration of the physical property of the PAM mechanisms, cannot fully capture the behavior of the PAMs owing to the occurrence of modeling errors between the real PAMs and the method used to parameterize them. Therefore, in our study, we introduced a GP-based PAM model to represent the mechanical and dynamical properties of PAMs using data sampled from a real system. In contrast, we were able to apply direct PAM controller learning approaches for PAM-driven exoskeleton robot control. However, when we use this approach, the PAM controllers need to be learned for each target task. Therefore, this approach is not practical if we need to use an exoskeleton robot for a wide variety of different tasks. In our study, we identified a GP-PAM model through physical user–robot interactions such that the PAM-driven exoskeleton robot can generate the desired force according to the target problems.

3. Proposed framework

We introduce the interaction design strategy for the collaborative soft actuator model identification with the exoskeleton robot. The schematic diagram of our implementation is depicted in Figure 2: (1) identifying GP models of the soft actuators from collected data; (2) selecting an interaction strategy by the GP active learning algorithm from a pre-constructed user–robot trajectory bank, and (3) the user is asked to track the proposed reference joint trajectory and the robot generates joint torques using the selected pressure command to collect the data of the soft actuators. We repeat the procedure above to improve the soft actuator model.

Fig. 2.

Overview of our implementation for physical user–robot interactions to identify soft actuator models: (1) we learn GP models of actuators; (2) select optimal interaction strategy that maximizes mutual information from pre-constructed user–robot trajectory bank by GP active learning; and (3) user and robot execute interactions to collect the soft actuator’s data.

3.1. GP soft actuator model identification

The soft actuator model is assumed as

y_{t} = f (x_{t}) + ϵ_{t}, ϵ_{t} ~ N (0, σ_{n}^{2})

(1)

where $x$ and y are inputs (e.g., actuator’s length, velocity, input commands) and outputs (e.g., contraction force) of the soft actuator, and $ϵ$ is an additive Gaussian noise. We applied GP regression (Rasmussen and Williams, 2006) to model the soft actuator dynamics. The GP can represent the model uncertainty with the predictive variance; thus, we can derive the entropy of the acquired model distribution analytically. With a input $X = [x_{1}, \dots, x_{n}]$ ( $x \in R^{d}$ ) and output $y = [y_{1}, \dots, y_{n}]^{⊤}$ ( $y \in R$ ) dataset, a predictive distribution of $f (x_{*})$ for the new input $x_{*}$ can be represented as follows:

p (f (x_{*}) | X, y, θ) = N (f (x_{*}) | m_{f | θ} (x_{*}), σ_{f | θ}^{2} (x_{*}))

(2)

where $θ$ is a hyperparameter vector and the mean and the variance can be derived as (Murphy, 2013)

m_{f | θ} (x_{*}) = k_{*}^{T} (K + σ_{n}^{2} I)^{- 1} y

(3)

σ_{f | θ}^{2} (x_{*}) = k_{* *} - k_{*}^{T} (K + σ_{n}^{2} I)^{- 1} k_{*}

(4)

Here, $k (\cdot)$ is a kernel function ( $k_{*} : = k (X, x_{*})$ , $k_{* *} : = k (x_{*}, x_{*})$ ), $K$ is a kernel matrix, each of whose element follows $K_{ij} = k (x_{i}, x_{j})$ . In this study, we used a squared exponential kernel:

k (x_{p}, x_{q}) = σ_{k}^{2} \exp (- \frac{1}{2} {(x_{p} - x_{q})}^{T} Λ (x_{p} - x_{q}))

(5)

where $Λ = diag [a_{1}^{- 1}, \dots, a_{d}^{- 1}]$ is a precision matrix and controls the horizontal scale over which the function changes and $σ_{k}$ controls the vertical scale of the function (Murphy, 2013). We define the hyperparameter vector as $θ = [a_{1}, \dots, a_{d}, σ_{k}, σ_{n}]^{T}$ .

3.2. User–robot interaction strategy

3.2.1. Defining reference trajectories as candidate data sampling points

Before we collect the user–robot interaction data, we define a set of reference trajectories as candidate data sampling points for the active learning framework. These reference trajectories for the human user and the robot must include the ranges where the actuators operate for the target tasks. In our previous work (Hamaya et al., 2017b), we simply designed the reference trajectories based on the mechanically reachable area of a single-DoF robot. However, for multi-DoF robots, this simple reference design strategy would generate many unnecessary trajectories that include irrelevant movement ranges for the target tasks. Thus, in this study, we came up another approach: we prepared the reference trajectories by moving a human experimenter’s arm while he was wearing an arm exoskeleton robot given different pressures. The resulting reference trajectories naturally represented the range of the human experimenter’s movements that are close to practical situations.

Here, the set of reference trajectories is represented as $X_{ref} = {X_{ref}^{(1, 1)}, \dots, X_{ref}^{(J, M)}}$ , where each trajectory is defined as $X_{ref}^{(j, m)} = [x_{1}^{(j, m)}, \dots, x_{t}^{(j, m)}, \dots, x_{T}^{(j, m)}]$ . $j = 1, \dots, J$ is an index of a trajectory, and $m = 1, \dots, M$ is an index of an actuator.

3.2.2. GP active learning for designing user–robot interactions

We applied a GP active learning method to select the reference trajectory for designing physical interactions. Kapoor et al. (2007) and Tanaka et al. (2014) demonstrated that efficient object categorizations could be achieved by GP active learning methods. However, most of the GP active learning methods have been implemented with fixed GP hyperparameters while these hyperparameters were manually designed based on the experts’ knowledge. If these predetermined hyperparameters are inappropriate for the learning procedure, they can result in the overfitting of the model to the training dataset. Therefore, we leveraged an active learning of GP hyperparameters (Garnett et al., 2014). Based on uncertainty of the hyperparameters, the algorithm actively selects the reference trajectory to update the hyperparameters.

To select a new data point, the mutual information between the hyperparameter $θ$ and mean output of the model $f (x)$ was used as the objective function:

I (f (x); θ) = H (f (x)) - H (f (x) | θ)

(6)

where $H (\cdot)$ is the entropy. We can derive the entropy $H (f (x) | θ)$ because $p (f (x) | X, y, θ)$ is a Gaussian distribution as in (2). Meanwhile, the entropy $H (f (x))$ cannot be easily obtained because the marginal distribution $p (f (x) | X, y) = \int p (f (x) | X, y, θ) p (θ | X, y) d θ$ cannot be analytically derived. We then calculate the approximated marginal distribution $q (f (x) | X, y)$ from integrating an approximated linear Gaussian model of $p (f (x) | X, y, θ)$ with respect to $θ$ and an approximated posterior of the hyperparameter $q (θ | X, y)$ . Finally, we can obtain the approximated form of (6) as follows Garnett et al. (2014):

I (x) \approx \ln \frac{{σ^{2}}_{f} (x)}{{σ^{2}}_{f | θ} (x)},

(7)

where $I (f (x); θ)$ is denoted as $I (x)$ for simplicity. Here $σ_{f | θ}^{2} (x)$ is the predictive variance given the hyperparameters $θ$ . We utilized $θ = \hat{θ}$ to evaluate the predictive variance, where $\hat{θ}$ was estimated by the maximum a posteriori as in Appendix A. ${σ^{2}}_{f} (x)$ is the predictive variance of the marginalized distribution and can be derived as presented in Appendix B. We applied the nearest Semi Positive Definite (SPD) algorithm¹ that approximates the covariance matrix to the nearest positive define matrix to avoid numerical problems pertaining to optimization of the hyperparameter $θ$ . The algorithm subsequently selects reference trajectories to conduct user–robot interactions as follows:

L_{I} (X_{ref}^{(j, \cdot)}) = \sum_{m = 1}^{M} \sum_{t = 1}^{T} I (x_{t}^{(j, m)})

(8)

j^{*} = \arg max_{j} L_{I} (X_{ref}^{(j, \cdot)})

(9)

We summarize the algorithm of our proposed method in Algorithm 1. First, we prepare the priors of the hyperparameters for M actuators, initial dataset, and reference trajectories. The hyperparameters of the GP models were estimated by the maximum a posteriori with using the dataset and priors $p (θ)$ as described in Appendix A (line 2). We compute the mutual information Garnett et al. (2014) between the function $f (x)$ and the hyperparameters $θ$ at each sample point for each actuator (line 3). Then, the index of reference trajectory $j^{*}$ is selected to design user–robot interaction that maximizes the mutual information as in (9) (line 4). Subsequently, the user and robot generate movements by following the reference trajectories. Then, input $X_{s}$ and output $y_{s}$ data are sampled (line 5). The newly sampled data is merged to the input $X$ and output $y$ dataset (line 6). When we merge the newly acquired data, we applied a data subsampling procedure as explained in Appendix C.

Algorithm 1 User–robot collaborative model identification
Require: Input $X$ and output $y$ dataset; Hyperprior $p (θ)$ ; Reference trajectories $X_{r ef}$
1: for physical interaction trials do
2: $\hat{θ}$ ← LearnGPmodels( $p (θ), X, y$ )
3: $I (x)$ ← MutualInformation( $p (f (x) \| X, y, θ), p (θ \| X, y), \hat{θ}$ )
4: $j^{*}$ ← SelectInteraction( $I (x), X_{r ef}$ )
5: $X_{s}, y_{s}$ ← DataSampling $(X_{r ef}^{(j^{*}, \cdot)})$
6: Add the pair of newly sampled input $X_{s}$ and output $y_{s}$ data to the dataset $X$ and $y$ after a data subsampling procedure.
7: end for
8: return $\hat{θ}$

4. Experiments

To evaluate our proposed method, we conducted PAM model identification experiments and an offline analysis. We utilized two joints out of our four-DoF upper-limb exoskeleton robot driven by PAMs. Two healthy adults participated in the experiments. All participants were provided written informed consent before participation. Wearing the exoskeleton robot, the participants were asked to track the reference trajectories selected by the active GP algorithm while the robot also used the selected desired pressure profiles to collect data for the identification (Figure 3(a)). For comparison, the standard PAM identification procedure that does not consider the user–robot interaction was employed. We further conducted an offline analysis to investigate how the tracking errors affected the model identification performances.

Fig. 3.

Experimental setup. (a) Experimental platform composed of exoskeleton robot and visual feedback system. (b) Upper-limb exoskeleton robot. Our exoskeleton robot contains four DoFs: shoulder abduction/adduction (SAA), shoulder flexion/extension (SFE), elbow flexion/extension (EFE), and wrist flexion/extension (WFE). In this study, we utilized two DoFs, SFE and EFE, out of four. To move these two joints, we activated four PAMs.

4.1. Experimental setup

We used our upper-limb exoskeleton robot (Furukawa et al., 2017; Noda et al., 2014), as shown in Figure 3(b). Our exoskeleton robot contains four DoFs: shoulder abduction/adduction (SAA), shoulder flexion/extension (SFE), elbow flexion/extension (EFE), and wrist flexion/extension (WFE). In this study, we utilized two DoFs, SFE and EFE, out of the four joints. To move these two joints, we activated four PAMs, and two sets of agonist and antagonist PAMs. The agonist muscles used 40 mm PAMs (Festo MAS-40) and the antagonist muscles used 10 mm PAMs (Festo MAS-10). The servo valves are equipped with each muscle. The contraction force of each PAM is transmitted to a robot joint via Bowden cables. Encoders are mounted on each joint to measure the joint angles. Load cells are attached to the tips of the PAMs to measure the contraction force. The PAM states were defined as $x = [L, \overset{\cdot}{L}, P]^{⊤}$ , where L is the muscle length and $\overset{\cdot}{L}$ is the change rate of the length. Here P is the pressure command to the servo valve. The output of the PAM model y is the contraction force.

We provided visual feedback to the participants. The reference joint angles and the current posture of the robot were displayed using the Open GL library. The red link in Figure 3(a) shows the reference joint angles and the green one shows the current robot posture.

4.2. Defining a set of reference trajectories

In this experiment, we defined the reference trajectories by moving a human experimenter’s arm while he was wearing the arm exoskeleton robot. Simultaneously, the robot was activated by generating the pressure command to the PAMs. The rth reference pressure profile $P_{r} = [P_{r} (0), \dots, P_{r} (t), \dots, P_{r} (T)]$ was represented with a sinusoidal function $P_{r} (t) = A \sin (2 π α t) + B$ , where the duration for one movement trial T was 4.0 s. We designed the parameters of the sinusoidal function as $A = [- 5 / 3, 5 / 3]$ , $B = [8 / 3, 13 / 3]$ , and $α = 0 (t < 1.0)$ , $0.5 (1.0 \leq t < 2.0)$ , $0.25 (2.0 \leq t \leq 4.0)$ Hz such that these parameter ranges encompass the region where the robot operates. This yields four sinusoidal input pressure patterns for each PAM. Because we used four PAMs, we had to prepare $256 (= 4^{4})$ pressure command profiles. The sampling frequency was 250 Hz. In a later section, we show the actual profile of the reference joint angle trajectories and the pressure command profiles in Figure 5.

When we collected the data for the PAM model identification, we also measured the force applied to the PAMs. However, the force data were only used for evaluating the PAM model learning process in the offline analysis. In other words, activating the PAMs is not necessary for preparing the reference joint trajectories and pressure command profiles.

4.3. Designing user–robot interaction for PAM model identification

In this study, we adopted a Matlab software package for the GP hyperparameter active learning (Garnett et al., 2014).² The duration for one trial was 4.0 s. The participants conducted 10 trials to generate the physical user–robot interaction data.

For comparison, we employed a standard PAM model identification procedure that does not consider the user–robot interaction. We adopted a linear parametric PAM model and utilized an automatic PAM identification technique (Teramae et al., 2013) in which antagonist PAMs were used for generating the external forces. The details of the model and the identification process are presented in Appendix D. We evaluated the force output estimation error of the learned PAM models on a test dataset. As the test dataset, we utilized all of the data points that were collected when we prepared the reference trajectories, as described in Section 4.2.

5. Results

5.1. PAM model identification through user–robot interactions

The participants observed the current desired posture displayed on the monitor that follows the reference trajectories selected by the GP active learning method (Figure 4). Simultaneously, the participants were asked to track the displayed posture by moving their arm with the exoskeleton robot through a physical user–robot interaction. Figure 5 shows an example of a sequence of selected reference trajectories of the shoulder joint, and the pressure command profiles of the PAM1 by our active GP learning method for one experimental sequence.

Fig. 4.

Snapshots of the user–robot interactive PAM model identification experiment. Wearing an exoskeleton robot, participants were asked to track reference trajectories selected by an active GP algorithm while the robot used selected desired pressure profiles to collect data for identification. We provided visual feedback to the participants. Reference joint angles and the current posture of the robot were displayed using Open GL library. Red links show reference joint angles and green links show the current robot posture.

Fig. 5.

Example sequence of selected reference trajectories of shoulder joint and pressure command profiles of PAM1 by our active GP learning method for one experimental sequence. (Top) Selected reference trajectories. (Bottom) Pressure command profiles.

Figure 6 shows the learning curves of our proposed approach and that of the standard PAM procedure. The total number of the data points to the represent GP-PAM model was 30. The vertical axis shows the force prediction error rate. The error rates were derived by normalizing the error with the maximum contraction force of the test data. The error bars show the standard deviations of the predicted errors on eight experiments. The red lines show the learning performances of our proposed method. The gray lines show that of the standard identification method that does not consider the user–robot interaction. Our proposed method successfully learned the PAM model through the physical user–robot interactions. Although the error rates were reduced even with the standard identification process, The progresses of model learning were significantly worse than with our proposed approach. This result indicates that user–robot interactions successfully enhanced the progress of actuator model identification.

Fig. 6.

Learning curves of our proposed approach and that of the standard PAM procedure. The vertical axis shows the force prediction error rate. Error rates were derived by normalizing the error with the maximum contraction force of test data. Error bars show standard deviations of predicted errors on eight experiments. Red lines show learning performances of our proposed method. Gray lines show that of the standard identification method, which does not consider user–robot interactions. Our proposed method successfully learned the PAM model through physical user–robot interactions. Although error rates were reduced even with the standard identification process, progresses of model learning were significantly worse than with our proposed approach. This result indicates that user–robot interaction successfully enhanced progress of actuator model identification.

5.2. Comparisons of actuator model identification performances

Figure 7 shows the comparisons of the actuator model identification performances. The red bar shows the contraction force prediction performance of our proposed method with the two participants. The dark gray bar shows that of the standard linear PAM model without user–robot interactions. The light gray bar shows that of the GP–PAM model without user–robot interactions. Because ordinal GP model non-interactive batch learning with many data points cannot be used because of heavy computational burden, we adopted the sparse GP (SPGP) model (Snelson and Ghahramani, 2006) for this comparison. A detailed explanation is provided in Appendix E. Our proposed method outperformed two other standard non-interactive identification procedures. This result indicates the importance of physical user–robot interactions to acquire better actuator models for wearable robots.

Fig. 7.

Comparisons of actuator model identification performances. The red bar shows contraction force prediction performance of our proposed method with two participants. The dark gray bar shows that of the standard linear PAM model without user–robot interactions. The light gray bar shows that of the GP-PAM model without user–robot interactions. An ordinal GP-PAM model with non-interactive batch training cannot be applied for whole training data due to its heavy computational burden since it does not include any subsampling methods unlike our proposed method. To cope with this problem, we adopted SPGP model for this comparison. Our proposed method outperformed two other standard non-interactive identification procedures. This result indicates importance of user–robot physical interactions to acquire better actuator models for wearable robots.

In addition, to validate how our proposed actuator model identification procedure with physical user–robot interactions can be used to operate properly on different participants, we individually evaluated the contraction force prediction performances of the acquired model through the interaction of the two different participants. We used the same set of reference trajectories and the pressure command profiles for the experiments with these two participants.

Figure 8 shows the model identification performances with different participants for eight experiments. The error bar shows the standard deviation. We applied Welch’s t-test (5% significance level) and did not find any statistically significant difference between the prediction performances of the models acquired through the interactions with the two different participants. This result indicates that the performances are comparable between these two users in the experiment.

Fig. 8.

Model identification performances with different participants. Error bar shows standard deviation. We applied Welch’s t-test and did not find any statistically significant difference between the prediction performances of the models acquired through interactions with two different participants. This result indicates that the performances are comparable between these two users in the experiment.

5.3. Investigation of how tracking errors affect model identification performances

In our user–robot interaction experiment, the human participant was asked to track the reference trajectories that were selected by our active GP learning method. However, precisely following the displayed reference movements is difficult for a participant. Therefore, we conducted an offline analysis to investigate how the tracking error affects the PAM identification performance. For the PAM model identification, our method exploits the most informative data point that maximizes the approximated mutual information. To simulate the tracking error, we randomly shifted the recorded data that has the maximum mutual information. The amount of the data time shifts $Δ$ were sampled from a uniform distribution with the range of $[- 250, 250]$ corresponding to 1.0 second ahead and delay. We selected the data points $x_{t_{1}^{I} + Δ}$ and $y_{t_{1}^{I} + Δ}$ . We set this value so that the subjects would show the tracking error within this range. The specified range represents the possible deviation of human subject movements. We used the same number of the collected data and hyperpriors as the actual online experiment introduced in Section 5.1. Figure 9 shows the learning curves of the force prediction errors with the shifted and without-shifted data. In both conditions, the force prediction errors were decreased successfully through the simulated learning process. The results indicate the robustness of our model identification procedure against the tracking error of a subject to the reference trajectories. Meanwhile, the learning curve without the data shift indicates better learning performance. This result indicates that the selected reference trajectory by our active learning method contains more information for PAM model identification than other data points.

Fig. 9.

Robustness of the model-learning process. The vertical axis shows force prediction error. To simulate tracking error, we randomly shifted recorded data that has maximum mutual information. The amplitude of data shifts were sampled from a uniform distribution. The cyan line shows learning performance with data shift. The blue line shows learning performance without data shift. Results indicate robustness of our model identification procedure against tracking error of participants to reference trajectories. Results also indicate that the reference trajectory selected by our active learning method contains more information for PAM model identification than other data points.

6. Discussion

In this section, we further discuss how the hyperprior distribution $p (θ)$ affects the model identification performance through offline analyses. In addition, we investigate the learned GP-PAM model in this experiment.

6.1. Effects of the hyperpriors

The hyperprior settings of each actuator for the GP hyperparameter learning in our experiments are summarized in Table 1. Even though we manually selected these hyperpriors, the settings of the hyperpriors are not highly sensitive.

Table 1.

Hyperprior settings.

$p (θ) = N (m_{h}, σ_{h}^{2})$	PAM1		PAM2		PAM3		PAM4
$p (θ) = N (m_{h}, σ_{h}^{2})$	$m_{h}$	$σ_{h}$	$m_{h}$	$σ_{h}$	$m_{h}$	$σ_{h}$	$m_{h}$	$σ_{h}$
$a_{1}$	1.5	10.0	1.8	8.0	2.3	8.0	2.5	8.0
$a_{2}$	2.3	10.0	1.0	8.0	2.6	8.0	2.9	8.0
$a_{3}$	2.2	10.0	2.1	8.0	2.7	8.0	2.0	8.0
$σ_{k}$	1.8	10.0	0.7	8.0	2.9	8.0	1.6	8.0
$σ_{n}$	5.0	1.4	5.0	1.0	5.0	1.0	5.0	1.0

Using the dataset we collected for the offline analysis in Section 5.3, we investigated how the model identification performance can be affected by the different settings of the hyperpriors. First, we modulated the hyperpriors for the mean value of the hyperparameters. Instead of using the specific mean values in Table 1, to validate the sensitivity of the value for the performance, we additionally evaluated other parameters locally and randomly sampled at around $m_{h} = 2.0$ from $N (2, 1)$ at each experiment. In this offline analysis, we used $K = 30$ points of the subsampled data to update the hyperparameters at each trial (see Appendix C). We conducted the comparison between the contraction force prediction performances of the fixed hyperprior used in our experiments and that of the stochastically sampled hyperpriors. We did not find any statistically significant difference between these two performances (Wilcoxon’s signed-rank test, 5% significance level). The result suggests that the performance of our method is not very sensitive to the choice in $m_{h}$ .

Next, we tested much smaller hyperpriors for the variance compared with the settings used in the online experiments. In the online experiments, to avoid overfitting, we used large hyperpriors for the variance. Here, we used smaller hyperpriors to assess the robustness of our proposed method. We conducted the comparison between the contraction force prediction performances of the large hyperpriors for the variance used in our experiments and the newly tested small hyperpriors. We did not find any statistically significant difference between these two performances (Wilcoxon’s signed-rank test, 5% significance level). The results indicate that our proposed method is not highly sensitive to the settings of hyperpriors for the variances.

6.2. Learning GP-PAM models

Here, we first investigated which reference trajectory was frequently chosen by our active learning framework. Figure 10 shows the top three most frequently selected reference trajectories. A common tendency can be observed among these trajectory profiles, cf., Figure 5. This result indicates the consistency of the learning process of our proposed approach.

Fig. 10.

Top three most frequently selected reference trajectories. A common tendency can be observed among the top three trajectory profiles, cf., Figure 5.

Figure 11 shows the learned GP-PAM model for PAM1. We plotted the function in Figure 11(a) by taking the PAM pressure P and the PAM velocity $\overset{\cdot}{L}$ as the inputs and the contraction force F as the output with the fixed PAM length of $L = 240.0$ mm that is around the intermediate value of the operated range. The learned function clearly depends on the velocity $\overset{\cdot}{L}$ especially in the high-pressure input region while the static PAM model (see Appendix D) cannot represent velocity-dependent variation of the contraction force. To further investigate the acquired function from a different view, we also plotted the function in Figure 11(b) by taking the PAM velocity $\overset{\cdot}{L}$ and the PAM length L as the inputs and the contraction force F as the output with the fixed PAM pressure of $P = 4.0$ bar, which is around the intermediate value of the operated range. Figure 11(a) and (b) indicate that the GP-PAM model successfully captured the nonlinear characteristics of the PAM through our proposed model identification strategy.

Fig. 11.

Learned GP-PAM model (PAM1): (a) taking PAM pressure P and PAM velocity $\overset{\cdot}{L}$ as inputs and contraction force F as output with fixed PAM length of $L = 240.0$ mm, and (b) taking PAM velocity $\overset{\cdot}{L}$ and PAM length L as inputs and contraction force F as output with fixed PAM pressure of $P = 4.0$ bar. The GP-PAM model successfully captured the nonlinear characteristics of PAM through our proposed model identification strategy.

7. Conclusion

In this study, we have proposed a design for physical user–robot interactions for modeling soft actuators equipped with wearable robots. We have applied our approach to a user–robot collaborative PAM model identification. We have adopted a GP active learning method for an efficient model identification. We have conducted interactive model identification experiments with two participants using a two-DoF exoskeleton robot containing four PAMs. These experimental results indicate that our proposed GP active learning method with the interactive identification process is more suitable for soft actuator modeling than the standard PAM modeling procedure that did not consider physical user–robot interactions.

We have further investigated how the user’s tracking performances based on the presented reference trajectories affected the model identification process through offline analysis. We found that the proposed approach is robust against the user’s tracking error. However, if no tracking error is present, the learning performance is better than that of the simulated tracking errors. This indicated that our active learning approach successfully selected informative reference trajectories for model identification to the user.

Currently, we are using the ordinal GP model for our active learning framework. However, we cannot directly use a large number of samples to train the ordinal GP model because of the heavy computational burden. For future work, we will derive an active learning algorithm for a sparse GP model such that we can utilize information extracted from all the collected data, but can simultaneously reduce the effective number of samples to concisely represent the actuator models for practical implementation.

Recently, novel soft actuators have been developed (Koizumi et al., 2018; Li et al., 2017; Marchese et al., 2015; Robertson and Paik, 2017). They have soft materials and are driven by fluid; our approach could be applied to model identification for wearable robots with soft actuators. In addition, our approach could be employed in model identification for soft manipulators that physically interact with users (Bicchi and Tonietti, 2004; Ohta et al., 2018). Recently, wearable sensing devices that can measure human movements (Mengüç et al., 2014; Michael and Howard, 2018) and stiffness (Yagi et al., 2018) have also been developed. We could combine these kinds of devices and our proposed method in future work.

Footnotes

Appendix A. MAP estimation of the GP hyperparameter

The hyperparameter can be obtained from the maximum a posteriori of $p (θ | X, y) \propto p (y | X, θ) p (θ)$ . The hyperprior model $p (θ)$ is provided in Table 1. We consider minimizing the negative log likelihood:

(10)

\begin{matrix} \log (p (θ | X, y)) = \frac{1}{2} y^{T} (K + σ_{n}^{2} I)^{- 1} y + \frac{1}{2} \log | K + σ_{n}^{2} I | \\ + \frac{n}{2} \log 2 π - \log p (θ) \end{matrix}

(11)

\hat{θ} = \arg min_{θ} {- \log (p (θ | X, y))}

The gradient of $\log (p (θ | X, y))$ can be calculated analytically. We used the conjugate gradient method to update $\hat{θ}$ .

Appendix B. Approximated marginal predictive variance

The marginalization ${σ^{2}}_{f}$ can be given as

(12)

\begin{matrix} {σ^{2}}_{f} (x) = \frac{4}{3} {σ^{2}}_{f | θ} (x) + {\frac{\partial m_{f | θ} (x)}{\partial θ}}^{T} Σ \frac{\partial m_{f | θ} (x)}{\partial θ} \\ + \frac{1}{3 {σ^{2}}_{f | θ} (x)} {\frac{\partial {σ^{2}}_{f | θ} (x)}{\partial θ}}^{T} Σ \frac{\partial {σ^{2}}_{f | θ} (x)}{\partial θ} \end{matrix}

where $m_{f | θ} (x)$ and ${σ^{2}}_{f | θ} (x)$ are the predictive mean and variance of the distribution $p (f (x) | X, y, θ)$ . Partial derivatives with respect to the hyperparameters $θ$ are derived around $θ = \hat{θ}$ . Here $Σ$ is the covariance matrix of $p (θ | X, y)$ that is obtained by Laplace approximation:

(13)

Σ^{- 1} = - \nabla \nabla^{T} \log (p (θ | X, y)) |_{θ = \hat{θ}}

Appendix C. Data subsampling

As the computational complexity of GP active learning is $O (n^{2})$ , the computational cost increases significantly with a large number of training data points. To cope with this issue, we subsampled data points from the pair of the acquired input $X_{s}$ and output $y_{s}$ data with considering the mutual information between the function f and the hyperparameters $θ$ . Concretely, we first extract the most informative data point in terms of the mutual information:

(14)

t_{1}^{I} = \arg max_{t} {I (x_{t}^{s})}

where $X_{s} = [x_{1}^{s}, \dots, x_{T}^{s}]$ . We then additionally subsampled data points by randomly selected data indexes ${t_{2}^{I}, \dots, t_{K}^{I}}$ . The number of subsampled data points were $K = 3$ for our online experiments. As the pair of the input $X_{s}$ and output $y_{s}$ data are acquired through the informative reference trajectory as explained in Algorithm 1, here we assume that data points in $X_{s}$ and $y_{s}$ are equivalently informative to estimate the hyperparameter $θ$ except for the most informative data point $t_{1}^{I}$ . By using the above subsampling strategy, we have a reduced number of data points ${x_{t_{1}^{I}}, \dots, x_{t_{K}^{I}}}$ and ${y_{t_{1}^{I}}, \dots, y_{t_{K}^{I}}}$ . We add these subsampled data points to the input $X$ and output $y$ dataset.

Appendix D. Parametric models

We developed our pneumatic actuators based on FESTO PAMs (see https://www.festo.com). FESTO provided a data sheet that shows the relationship between the contraction rate C and force output F. From Chou and Hannaford (1996) and the data sheet provided by FESTO, we modeled the relationship between the contraction rate and the force output by a quadratic function $f = α_{1} C^{2} + α_{2} C + α_{3}$ (see Figure 12). We identified the two sets of parameters for the quadratic models ${α_{1}^{l}, α_{2}^{l}, α_{3}^{l}}$ and ${α_{1}^{u}, α_{2}^{u}, α_{3}^{u}}$ under two different pressure inputs to PAM. Then, force outputs were estimated by using a linear approximation of the data sheet profiles Teramae et al. (2013) as

(15)

F = \frac{(f_{u} - f_{l}) P + P_{u} f_{l} - P_{l} f_{u}}{P_{u} - P_{l}}

(16)

f_{l} = α_{1}^{l} C^{2} + α_{2}^{l} C + α_{3}^{l}

(17)

f_{u} = α_{1}^{u} C^{2} + α_{2}^{u} C + α_{3}^{u}

where $f_{u}$ and $f_{l}$ are the contraction force at the upper and lower pressures, respectively. In this study, we designed $P_{u} = 6.0$ bar and $P_{l} = 2.0$ bar, but these upper and lower pressure value can be arbitrarily selected as long as these two values are not very close and within a reasonable range for the target PAMs.

Appendix E. Sparse GP

Snelson and Ghahramani (2006) proposed the SPGP that is parameterized by the pseudo-inputs l ( $l << n$ ). The SPGP can reduce the computational cost ( $O (l n^{2})$ for training and $O (l^{2})$ for prediction) compared with the normal GP ( $O (n^{3})$ for training and $O (n^{2})$ for prediction). Given pseudo-inputs $\bar{X} = [{\bar{x}}_{1}, \dots, {\bar{x}}_{l}]$ , the predictive distribution of the SPGP can be expressed as follows:

(18)

p (y_{*} | x_{*}, D, \bar{X}) = N (y_{*} | μ_{*}, σ_{*}^{2})

where

(19)

μ_{*} = E [y_{*}] = k_{*}^{T} Q_{\ln}^{- 1} (K_{\ln} + σ_{n}^{2} I)^{- 1} y

(20)

σ_{*}^{2} = Var [y_{*}] = k_{* *} - k_{*}^{T} (K_{l} - Q_{l}^{- 1}) k_{*}

(21)

Q_{l} = K_{l} + K_{\ln} (Λ + σ_{n}^{2} I)

The marginal likelihood can be maximized with respect to hyperparameters ( $Λ$ and $σ_{n}$ ) and pseudo-inputs $\bar{X}$ by a gradient method.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by “Research and development of technology for enhancing functional recovery of elderly and disabled people based on non-invasive brain imaging and robotic assistive devices,” the Commissioned Research of National Institute of Information and Communications Technology (NICT), Japan, by ImPACT Program of Council for Science, Technology and Innovation (Cabinet Office, Government of Japan), by AMED (Grant Number JP18he1902005), and by JSPS KAKENHI (Grant Numbers JP16H06565 and JP17J02979).

Notes

ORCID iD

Masashi Hamaya

References

Asbeck

De Rossi

Holt

Walsh

(2015) A biologically inspired soft exosuit for walking assistance. The International Journal of Robotics Research 34(6): 744–762.

Bajcsy

Losey

O’Malley

Dragan

(2017) Learning robot objectives from physical human interaction. In: Proceedings of the Conference on Robot Learning, pp. 217–226.

Bicchi

Tonietti

(2004) Fast and “soft-arm” tactics [robot arm design]. IEEE Robotics and Automation Magazine 11(2): 22–33.

Cao

Xie

Das

(2018) MIMO sliding mode controller for gait exoskeleton driven by pneumatic muscles. IEEE Transactions on Control Systems Technology 26(1): 274–281.

Chou

Hannaford

(1996) Measurement and modeling of Mckibben pneumatic artificial muscles. IEEE Transactions on Robotics and Automation 12(1): 90–102.

Ding

Kim

Kuindersma

Walsh

(2018) Human-in-the-loop optimization of hip assistance with a soft exosuit during walking. Science Robotics 3(15): eaar5438.

Dollar

Herr

(2008) Lower extremity exoskeletons and active orthoses: Challenges and state-of-the-art. IEEE Transactions on Robotics 24(1): 144–158.

Ferris

Czerniecki

Hannaford

(2005) An ankle–foot orthosis powered by artificial pneumatic muscles. Journal of Applied Biomechanics 21(2): 189–197.

Furukawa

Noda

Teramae

Morimoto

(2017) Human movement modeling to detect biosignal sensor failures for myoelectric assistive robot control. IEEE Transactions on Robotics 33(4): 846–857.

10.

Garnett

Osborne

Hennig

(2014) Active learning of linear embeddings for Gaussian processes. In: Proceedings of the Conference on Uncertainty Artificial Intelligence, pp. 230–239.

11.

Ghadirzadeh

Bütepage

Maki

Kragic

Björkman

(2016) A sensorimotor reinforcement learning framework for physical human–robot interaction. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2682–2688.

12.

Gopura

Bandara

Kiguchi

Mann

(2016) Developments in hardware systems of active upper-limb exoskeleton robots: A review. Robotics and Autonomous Systems 75: 203–220.

13.

Hamaya

Matsubara

Noda

Teramae

Morimoto

(2017a) Learning assistive strategies for exoskeleton robots from user–robot physical interaction. Pattern Recognition Letters 99: 67–76.

14.

Hamaya

Matsubara

Noda

Teramae

Morimoto

(2017b) User–robot collaborative excitation for PAM model identification exoskeleton robots. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3063–3068.

15.

Kapoor

Grauman

Urtasun

Darrell

(2007) Active learning with Gaussian processes for object categorization. In: Proceedings IEEE International Conference on Computer Vision, pp. 1–8.

16.

Khoramshahi

Billard

(2018) A dynamical system approach to task-adaptation in physical human–robot interaction. Autonomous Robots, in press.

17.

Koizumi

Kurumaya

Nabae

Endo

Suzumori

(2018) Braiding thin Mckibben muscles to enhance their contracting abilities. IEEE Robotics and Automation Letters 3(4): 3240–3246.

18.

Koller

Gates

Ferris

Remy

(2016) ‘Body-in-the-loop’ optimization of assistive robotic devices: A validation study. In: Proceedings of Robotics: Science and Systems.

19.

Lagoda

Schouten

Stienen

Hekman

van der Kooij

(2010) Design of an electric series elastic actuated joint for robotic gait rehabilitation training. In: Proceedings IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 21–26.

20.

Vogt

Rus

Wood

(2017) Fluid-driven origami-inspired artificial muscles. Proceedings of the National Academy of Sciences 114(50): 13132–13137.

21.

Liu

Meng

Xie

(2017) Hierarchical compliance control of a soft ankle rehabilitation robot actuated by pneumatic muscles. Frontiers in Neurorobotics 11: 64.

22.

Maeda

Ewerton

Neumann

Lioutikov

Peters

(2017) Phase estimation for fast action recognition and trajectory generation in human–robot collaboration. The International Journal of Robotics Research 36(13–14): 1579–1594.

23.

Marchese

Katzschmann

Rus

(2015) A recipe for soft fluidic elastomer robots. Soft Robotics 2(1): 7–25.

24.

Mengüç

Park

Pei

, et al. (2014) Wearable soft sensing suit for human gait measurement. The International Journal of Robotics Research 33(14): 1748–1764.

25.

Merola

Colacino

Cosentino

Amato

(2018) Model-based tracking control design, implementation of embedded digital controller and testing of a biomechatronic device for robotic rehabilitation. Mechatronics 52: 70–77.

26.

Mghames

Laghi

Della Santina

, et al. (2017) Design, control and validation of the variable stiffness exoskeleton flexo. In: Proceedings IEEE International Conference on Rehabilitation Robotics, pp. 539–546.

27.

Michael

Howard

(2018) Gait reconstruction from motion artefact corrupted fabric-embedded sensors. IEEE Robotics and Automation Letters 3(3): 1918–1924.

28.

Murphy

(2013) Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT Press.

29.

Noda

Furukawa

Teramae

Hyon

Morimoto

(2013) An electromyogram based force control coordinated in assistive interaction. In: IEEE International Conference on Robotics and Automation, pp. 2657–2662.

30.

Noda

Teramae

Ugurlu

Morimoto

(2014) Development of an upper limb exoskeleton powered via pneumatic electric hybrid actuators with Bowden cable. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3573–3578.

31.

Ohta

Valle

King

, et al. (2018) Design of a lightweight soft robotic arm using pneumatic artificial muscles and inflatable sleeves. Soft Robotics 5(2): 204–215.

32.

Park

Chen

Pérez-Arancibia

, et al. (2014) Design and control of a bio-inspired soft wearable robotic device for ankle–foot rehabilitation. Bioinspiration & Biomimetics 9(1).

33.

Peternel

Noda

Petric

Ude

Morimoto

Babic

(2016a) Adaptive control of exoskeleton robots for periodic assistive behaviours based on emg feedback minimisation. PLoS ONE 11(2): 1–26.

34.

Peternel

Tsagarakis

Ajoudani

(2016b) Towards multi-modal intention interfaces for human–robot co-manipulation. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2663–2669.

35.

Rasmussen

Williams

(2006) Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press.

36.

Robertson

Paik

(2017) New soft robots really suck: Vacuum-powered systems empower diverse capabilities. Science Robotics 2(9): eaan6357.

37.

Rozo

Calinon

Caldwell

Jimenez

Torras

(2016) Learning physical collaborative robot behaviors from human demonstrations. IEEE Transactions on Robotics 32(3): 513–527.

38.

Snelson

Ghahramani

(2006) Sparse Gaussian processes using pseudo-inputs. Advances in Neural Information Processing Systems 18: 1257.

39.

Tanaka

Matsubara

Ichien

Sugimoto

(2014) Object manifold learning with action features for active tactile object recognition. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 608–614.

40.

Teramae

Noda

Hyon

Morimoto

(2013) Modeling and control of a pneumatic–electric hybrid system. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4887–4892.

41.

Teramae

Noda

Morimoto

(2018) EMG-based model predictive control for physical human–robot interaction: Application for assist-as-needed control. IEEE Robotics and Automation Letters 3(1): 210–217.

42.

Thatte

Duan

Geyer

(2017) A sample-efficient black-box optimizer to train policies for human-in-the-loop systems with user preferences. IEEE Robotics and Automation Letters 2(2): 993–1000.

43.

Ugurlu

Doppmann

Hamaya

, et al. (2016) Variable ankle stiffness improves balance control: Experiments on a bipedal exoskeleton. IEEE Transactions on Mechatronics 21(1): 79–87.

44.

Ugurlu

Forni

Doppmann

Morimoto

(2015) Torque and variable stiffness control for antagonistically driven pneumatic muscle actuators via a stable force feedback controller. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1633–1639.

45.

Wang

Meijneke

, et al. (2015) Design and control of the mindwalker exoskeleton. IEEE Transactions on Neural Systems and Rehabilitation Engineering 23(2): 277–286.

46.

Yagi

Suzuki

Mochiyama

(2018) Human joint impedance estimation with a new wearable device utilizing snap-through buckling of closed-elastica. IEEE Robotics and Automation Letters 3(3): 1506–1513.

47.

Yan

Cempini

Oddo

Vitiello

(2015) Review of assistive strategies in powered lower-limb orthoses and exoskeletons. Robotics and Autonomous Systems 64: 120–136.

48.

Huang

Chen

Pan

Guo

(2015) Human-robot interaction control of rehabilitation robots with series elastic actuators. IEEE Transactions on Robotics 31(5): 1089–1100.

49.

Zhang

Fiers

Witte

, et al. (2017) Human-in-the-loop optimization of exoskeleton assistance during walking. Science 356(6344): 1280–1284.