In this paper we propose a method for interactive recognition of household objects using proprioceptive and auditory feedback. In our experiments, the robot observed the changes in its proprioceptive and auditory sensory streams while performing five exploratory behaviors (lift, shake, drop, crush, and push) on 50 common household objects (e.g. bottles, cups, balls, toys, etc.). The robot was tasked with recognizing the objects it was manipulating by feeling them and listening to the sounds that they make without using any visual information. The results show that both proprioception and audio, coupled with exploratory behaviors, can be used successfully for object recognition. Furthermore, the robot was able to integrate feedback from the two modalities, to achieve even better recognition accuracy. Finally, the results show that the robot can boost its recognition rate even further by applying multiple different exploratory behaviors on the object.
AtkesonCAnCHollerbachJ (1986) Estimation of inertial parameters of manipulator loads and links. The International Journal of Robotics Research5: 101–119.
BergquistTSchenckCOhiriUSinapovJGriffithSStoytchevA (2009) Interactive object recognition using proprioceptive feedback. In Proceedings of the 2009 IROS Workshop: Semantic Perception for Robot Manipulation, St Louis, MO.
5.
ChanAPampalkE (2002) Growing hierarchical self organizing map (ghsom) toolbox: visualizations and enhancements. In Proceedings of the 9th International Conference on Neural Information Processing (NIPS), pp. 2537–2541.
6.
DietterichT (2000) An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Machine Learning40: 139–157.
7.
ErnstMBulthofH (2004) Merging the Senses into a Robust Percept. Trends in Cognitive Science8: 162–169.
8.
GaverW (1993) What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology5: 1–29.
9.
GiordanoBMcAdamsS (2006) Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and plexiglass plates. Journal of the Acoustical Society of America119: 1171–1181.
10.
GrassiM (2005) Do we hear size or sound? Balls dropped on plates. Perception and Psychophysics67: 274–284.
11.
HellerM (1992) Haptic dominance in form perception: vision versus proprioception. Perception21: 655–660.
12.
HollerbachJWamplerC (1996) The calibration index and taxonomy for robot kinematic calibration methods. The International Journal of Robotics Research15: 573–591.
KrabbesMDöschnerC (1999) Modelling of robot dynamics bsed on multi-dimensional RBF-like neural network. In Proceedings of International Conference on Information Intelligence and Systems (ICIIS), pp. 180–187.
15.
KrotkovEKlatzkyRZumelN (1996) Robotic perception of material: Experiments with shape-invariant acoustic measures of material type. In Experimental Robotics IV (Lecture Notes in Control and Information Sciences, Vol. 223). Berlin: Springer, pp. 204–211.
16.
KubusDKrogerTWahlF (2007) On-line rigid object recognition and pose estimation based on inertial parameters. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1402–1408.
17.
KubusDWahlF (2008) Estimating inertial load parameters using force/torque and acceleration sensor fusion. In Robotic 2008, VDI-Beritche 2012, Munchen, Germany, pp. 29–32.
18.
KunchevaLWhitakerC (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning51: 181–207.
19.
LeeKHonHReddyR (1990) An overview of the SPHINX speech recognition system. IEEE Transactions on Acoustics, Speech, and Signal Processing38: 35–45.
20.
LorenzK (1996) Innate bases of learning. Learning as Self-Organization. Mahwah, NJ: Lawrence Erlbaum and Associates.
21.
LynottDConnellL (2009) Modality exclusivity norms for 423 object properties. Behavior Research Methods41: 558–564.
22.
MettaGFitzpatrickP (2003) Early integration of vision and manipulation. Adaptive Behavior11: 109–128.
23.
NakamuraTNagaiTIwahashiN (2007) Multimodal object categorization by a robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2415–2420.
24.
NanayakkaraTWatanabeKIzumiK (1999) Evolving Runge–Kutta–Gill RBF networks to estimate the dynamics of a multi-link manipulator. In Proceedings of Systems, Man, and Cybernetics, pp. 770–775.
25.
NataleLMettaGSandiniG (2004) Learning haptic representation of objects. In Proceedings of the International Conference on Intelligent Manipulation and Grasping.
26.
NavarroG (2001) A guided tour to approximate string matching. ACM Computing Surveys33: 31–88.
27.
NeedlemanSWunschC (1970) A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology48: 443–453.
28.
PetersRJenkinsOBodenheimerR (2006) Sensory–motor manifold structure induced by task outcome: experiments with robonaut. In Proceedings of IEEE International Conference on Humanoid Robots, pp. 484–489.
29.
PowerT (2000) Play and Exploration in Children and Animals. Mahwah, NJ: Lawrence Erlbaum Associates.
30.
QuigleyMBergerENgA (2007) STAIR: Hardware and software architecture. Presented at AAAI 2007 Robotics Workshop.
31.
RasolzadehBBjörkmanMHuebnerKKragicD (2010) An active vision system for detecting, fixating and manipulating objects in the real world. The International Journal of Robotics Research29: 133–154.
32.
RichmondJ (2000) Automatic Measurement and Modelling of Contact Sounds. Master’s thesis, University of British Columbia.
33.
RichmondJPaiD (2000) Active measurement of contact sounds. In Proceedings of ICRA, pp. 2146–2152.
34.
RusuRBMartonZCBlodowNDolhaMBeetzM (2008) Towards 3D point cloud based object maps for household environments. Robotics and Autonomous Systems56: 927–941.
35.
SappFLeeKMuirD (2000) Three-year-olds’ difficulty with the appearance-reality distinction. Developmental Psychology36: 547–60.
36.
SinapovJStoytchevA (2009) From acoustic object recognition to object categorization by a humanoid robot. In Proceedings of the RSS 2009 Workshop on Mobile Manipulation, Seattle, WA.
37.
SinapovJStoytchevA (2010) The boosting effect of exploratory behaviors. In Proceedings of the 24th National Conferrence on Artificial Intelligence (AAAI), pp. 1613–1618.
38.
SinapovJWeimerMStoytchevA (2009) Interactive learning of the acoustic properties of household objects. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2518–2524.
39.
SrinivasaSFergusonCHelfrichDBerensonDColletADiankovR (2009) Herb: a home exploring robotic butler. Autonomous Robots28: 5–20.
40.
Torres-JaraENataleLFitzpatrickP (2005) Tapping into touch. In Proceedings of the 5th International Workshop on Epigenetic Robotics, pp. 79–86.
