Passive dynamic walking exhibits human-like and energy-efficient gait. Biologically inspired compliance introduced to flexible passivity-based robot would be helpful to generate stable locomotion. However, designing adaptive controller for flexible biped on compliant ground still remains a challenge. This paper aims to design an adaptive and model-free stiffness controller for passivity-based flexible biped on compliant ground, where the hip stiffness is modulated by double deep Q network. One benefit of the double deep Q network is that adaptive stiffness control policy could be directly learned from inputs. At first, passive dynamic walking gait is utilized as a reference trajectory during double deep Q network training. Then the trained double deep Q network is used as adaptive stiffness controller for biped on compliant ground. Simulation results show that the passivity-based biped robot could walk in such walking cases as disturbed initial condition, level compliant ground, downslope slippery compliant surface, and varying compliance environments. The adaptive stiffness controller would be used to make the passivity-based biped robot adapt to the environmental changes.
Sakagami Y, Watanabe R, Aoyama C, et al. The intelligent Asimo: System overview and integration. In: IEEE/RSJ international conference on intelligent robots and systems, vol. 3, 2002, pp.2478–2483. New York: IEEE.
2.
Kaneko K, Kanehiro F, Morisawa M, et al. Cybernetic human hrp-4c. In: 9th IEEE-RAS international conference on humanoid robots, 2009, pp.7–14. New York: IEEE.
3.
VukobratovićMStepanenkoJ. On the stability of anthropomorphic systems. Math Biosci1972; 15: 1–37.
4.
FarzadpourFDaneshMTorkLarkiSM. Development of multi-phase dynamic equations for a seven-link biped robot with improved foot rotation in the double support phase. Proc IMechE, Part C: J Mechanical Engineering Science2015; 229: 3–17.
5.
McGeerT. Passive dynamic walking. Int J Robot Res1990; 9: 62–82.
6.
Schwab A and Wisse M. Basin of attraction of the simplest walking model. In: Proceedings of the ASME design engineering technical conference, Pittsburgh, PA, USA, vol. 6, pp.531–539.
7.
SpongMWHolmJKLeeD. Passivity-based control of bipedal locomotion. IEEE Robot Autom Mag2007; 14: 30–40.
8.
TorricelliDGonzalezJWeckxMet al.Human-like compliant locomotion: State of the art of robotic implementations. Bioinspir Biomim2016; 11: 051002–051002.
GuoZPanYSunTet al.Adaptive neural network control of serial variable stiffness actuators. Complexity2017; 2017: 5361246–5361246.
11.
GuoZPanYWeeLBet al.Design and control of a novel compliant differential shape memory alloy actuator. Sens Actuat A2015; 225: 71–80.
12.
PanDGaoFMiaoY. Dynamic research and analyses of a novel exoskeleton walking with humanoid gaits. Proc IMechE, Part C: J Mechanical Engineering Science2014; 228: 1501–1511.
13.
HobbelenDGWisseM. Ankle actuation for limit cycle walkers. Int J Robot Res2008; 27: 709–735.
14.
HuangYVanderborghtBVan HamRet al.Step length and velocity control of a dynamic bipedal walking robot with adaptable compliant joints. IEEE/ASME Trans Mechatron2013; 18: 598–611.
15.
WhittingtonBSilderAHeiderscheitBet al.The contribution of passive-elastic mechanisms to lower extremity joint kinetics during human walking. Gait Posture2008; 27: 628–634.
16.
NarukawaTTakahashiMYoshidaK. Efficient walking with optimization for a planar biped walker with a torso by hip actuators and springs. Robotica2011; 29: 641–648.
17.
SeyfarthAIidaFTauschRet al.Towards bipedal jogging as a natural result of optimizing walking speed for passively compliant three-segmented legs. Int J Robot Res2009; 28: 257–265.
18.
PlestanFGrizzleJWWesterveltERet al.Stable walking of a 7-dof biped robot. IEEE Trans Robot Autom2003; 19: 653–668.
19.
WesterveltERGrizzleJWChevallereauCet al.Feedback control of dynamic bipedal robot locomotion2007; vol. 28, Boca Raton, FL: CRC Press.
20.
YooSJParkJBChoiYH. Robust control of planar biped robots in single support phase using intelligent adaptive backstepping technique. Int J Control Autom Syst2007; 5: 269–282.
21.
SunCHeWGeWet al.Adaptive neural network control of biped robots. IEEE Trans Syst Man Cybernet Syst2017; 47: 315–326.
22.
WangSChaovalitwongseWBabuskaR. Machine learning algorithms in bipedal robot control. IEEE Trans Syst Man Cybernet C2012; 42: 728–743.
23.
SuttonRSBartoAG. Reinforcement learning: An introduction, Cambridge: MIT Press, 1998.
24.
Mnih V, Kavukcuoglu K, Silver D, et al. Playing Atari with deep reinforcement learning. arXiv: Learning 2013. Preprint arXiv:1312.5602v1.
25.
MnihVKavukcuogluKSilverDet al.Human-level control through deep reinforcement learning. Nature2015; 518: 529–533.
26.
ArulkumaranKDeisenrothMPBrundageMet al.Deep reinforcement learning: A brief survey. IEEE Signal Process Mag2017; 34: 26–38.
27.
Schuitema E, Hobbelen DG, Jonker PP, et al. Using a controller based on reinforcement learning for a passive dynamic walking robot. In: 2005 5th IEEE-RAS international conference on humanoid robots, 2005, pp.232–237. New York: IEEE.
28.
CollinsSRuinaATedrakeRet al.Efficient bipedal robots based on passive-dynamic walkers. Science2005; 307: 1082–1085.
29.
LiCLoweRZiemkeT. A novel approach to locomotion learning: Actor-critic architecture using central pattern generators and dynamic motor primitives. Front Neurorobot2014; 8: 23–23.
30.
Van Hasselt H, Guez A and Silver D. Deep reinforcement learning with double q-learning. In: Proceedings of the thirtieth AAAI conference on artificial intelligence (AAAI-16), Phoenix, AZ, USA, 2016, pp.2094–2100.
31.
GilardiGSharfI. Literature survey of contact dynamics modelling. Mech Mach Theory2002; 37: 1213–1239.
32.
NingLJunfengLTianshuW. The effects of parameter variation on the gaits of passive walking models: Simulations and experiments. Robotica2009; 27: 511–528.
33.
HuntKCrossleyF. Coefficient of restitution interpreted as damping in vibroimpact. J Appl Mech1975; 42: 440–445.
34.
WuYYaoDXiaoX. The effects of ground compliance on flexible planar passive biped dynamic walking. J Mech Sci Technol2018; 32: 1793–1804.
35.
QiFWangTLiJ. The elastic contact influences on passive walking gaits. Robotica2011; 29: 787–796.
36.
BatheKSaundersH. Finite element procedures in engineering analysis, Englewood Cliffs, New Jersey, USA: Prentice-Hall, Inc, 1982.
37.
Lin LJ. Reinforcement learning for robots using neural networks. Technical Report, Carnegie-Mellon University, USA, 1993.
38.
He FS, Liu Y, Schwing AG, et al. Learning to play in a day: Faster deep reinforcement learning by optimality tightening. arXiv: Learning 2016. Preprint arXiv:161101606.
39.
Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv: Learning 2016. Preprint arXiv: 160304467.
40.
PanYYuH. Composite learning robot control with guaranteed parameter convergence. Automatica2018; 89: 398–406.
