A visual servoing scheme consists of a closed-loop control approach that uses visual information feedback to control the motion of a robotic system. Probably the most popular visual servoing method is image-based visual servoing (IBVS). This kind of method uses geometric visual features extracted from the image to design the control law. However, extracting, matching, and tracking geometric visual features over time significantly limits the versatility of visual servoing controllers in various industrial and medical applications, in particular for “low-structured” medical images, e.g. ultrasounds and optical coherence tomography modalities. To overcome the limits of conventional IBVS, one can consider novel visual servoing paradigms known as “direct” or “featureless” approaches. This paper deals with the development of a new generation of direct visual servoing methods in which the signal control inputs are the coefficients of a multiscale image representation. In particular, we consider the use of multiscale image representations that are based on discrete wavelet and shearlet transforms. Up to now, one of the main obstacles in the investigation of multiscale image representations for visual servoing schemes was the issue of obtaining an analytical formulation of the interaction matrix that links the variation of wavelet and shearlet coefficients to the spatial velocity of the camera and the robot. In this paper, we derive four direct visual servoing controllers: two that are based on subsampled respectively non-subsampled wavelet coefficients and two that are based on the coefficients of subsampled respectively non-subsampled discrete shearlet transforms. All proposed controllers were tested in both simulation and experimental scenarios (using a six-degree-of-freedom Cartesian robot in an eye-in-hand configuration). The objective of this paper is to provide an analysis of the respective strengths and weaknesses of wavelet- and shearlet-based visual servoing controllers.
AbolmaesumiPSalcudeanSEZhuW-HSirouspourDiMaioSP (2002) Image-guided control of a robot for medical ultrasound. IEEE Transactions on Robotics and Automation18(1): 11–23.
2.
AllenJ (1977) Short term spectral analysis, synthesis, and modification by discrete fourier transform. IEEE Transactions on Acoustics, Speech, and Signal Processing25(3): 235–238.
3.
BarajasMDávalos-ViverosJPGarcia-LumbrerasSGordilloJL (2013) Visual servoing of UAV using cuboid model with simultaneous tracking of multiple planar faces. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 596–601.
4.
BernardC (2001) Discrete wavelet analysis for fast optic flow computation. Applied and Computational Harmonic Analysis11(1): 32–63.
5.
BeylkinGCoifmanRRokhlinV (1991) Fast wavelet transforms and numerical algorithms I. In Communications on Pure and Applied Mathematics44: 141–183.
6.
BurtPAdelsonE (1983) The Laplacian pyramid as a compact image code. IEEE Transactions on communications31(4): 532–540.
7.
CandesEDonohoD (2000) Curvelets: A surprisingly effective nonadaptive representation for objects with edges. Technical report, Department of Statistics, Stanford University, CA.
8.
CandèsEDonohoD (2004) New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities. Communications on Pure and Applied Mathematics57(2): 219–266.
9.
ChaumetteFBoukirS (1992) Structure from motion using an active vision paradigm. In Proceedings 11th IAPR International Conference on Pattern Recognition, pp. 41–44.
10.
ChaumetteFHutchinsonS (2006) Visual servo control. I. Basic approaches. IEEE Robotics and Automation Magazine13(4): 82–90.
11.
CollewetCMarchandE (2011) Photometric visual servoing. IEEE Transactions on Robotics27(4): 828–834.
12.
CrombezNCaronGMouaddibEM (2015) Photometric Gaussian mixtures based visual servoing. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015, pp. 5486–5491.
DaubechiesI (1988) Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics41(7): 909–996.
15.
DoMVetterliM (2003) Contourlets. In WellandGV (ed.), Beyond wavelets (Studies in Computational Mathematics, vol. 10). San Diego, CA: Academic Press/Elsevier Science, pp. 1–27.
16.
DonohoD (2001) Sparse components of images and optimal atomic decompositions. Constructive Approximation17(3): 353–382.
17.
DuflotL-AKrupaATamadazteBAndreffN (2016a) Toward ultrasound-based visual servoing using shearlet coefficients. In IEEE International Conference on Robotics and Automation, pp. 3420–3425.
18.
DuflotL-AKrupaATamadazteBAndreffN (2016b) Shearlet-based vs. photometric-based visual servoing for robot-assisted medical applications. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Daejeon, pp. 4099–4104.
19.
EasleyGLabateDColonnaF (2009) Shearlet-based total variation diffusion for denoising. IEEE Transactions on Image Processing18(2): 260–268.
20.
FlandersH (1973) Differentiation under the integral sign. The American Mathematical Monthly80(6): 615–627.
21.
GenzelMKutyniokG (2014) Asymptotic analysis of inpainting via universal shearlet systems. In SIAM Journal on Imaging Sciences7(4): 2301–2339.
22.
GrohsPKeiperSKutyniokGSchäferM (2016) -Molecules. Applied and Computational Harmonic Analysis41(1): 297–336.
23.
GuoKLabateD (2007) Optimally sparse multidimensional representation using shearlets. SIAM Journal on Mathematical Analysis39(1): 298–318.
24.
HaarA (1910) Zur theorie der orthogonalen funktionensysteme. Mathematische Annalen69(3): 331–371.
25.
HanBKutyniokGShenZ (2011) Adaptive multiresolution analysis structures and shearlet systems. SIAM Journal on Numerical Analysis49(5): 1921–1946.
26.
HäuserSSteidlG (2012) Fast finite shearlet transform. arXiv preprint arXiv:1202.1773.
HutchinsonSHagerGCorkeP (1996) A tutorial on visual servo control. IEEE Transactions on Robotics and Automation12(5): 651–670.
29.
KingEKutyniokGLimW-Q. Image inpainting: Theoretical analysis and comparison of algorithms. In SPIE Optical Engineering and Applications 885802. International Society for Optics and Photonics.
30.
KrupaAGangloffJDoignonC. (2003) Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Transactions on Robotics and Automation19(5): 842–853.
31.
KutyniokGLabateD (2012a) Introduction to shearlets. In Shearlets. New York: Springer, pp. 1–38.
32.
KutyniokGLabateD (2012b) Shearlets: Multiscale analysis for multivariate data. New York: Springer Science & Business Media.
33.
KutyniokGLimW-Q (2011) Compactly supported shearlets are optimally sparse. Journal of Approximation Theory163(11): 1564–1589.
34.
KutyniokGPetersenP (2015) Classification of edges using compactly supported shearlets. Applied and Computational Harmonic Analysis42(2): 245–293.
35.
KutyniokGShahramMZhuangX (2012) Shearlab: A rational design of a digital parabolic scaling algorithm. SIAM Journal on Imaging Sciences5(4): 1291–1332.
36.
KutyniokGLimW-QReisenhoferR (2016) Shearlab 3D: Faithful digital shearlet transforms based on compactly supported shearlets. ACM Transactions on Mathematical Software42(1): 5.
37.
LabateDLimW-QKutyniokGWeissG (2005) Sparse multidimensional representation using shearlets. In Optics and Photonics 2005, 59140U. International Society for Optics and Photonics.
38.
LippielloVSicilianoBVillaniL (2007) Position-based visual servoing in industrial multirobot cells using a hybrid camera configuration. IEEE Transactions on Robotics23(1): 73–86.
39.
MallatS (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence11(7): 674–693.
40.
MarturiNTamadazteBDembéléSPiatN (2014) Visual servoing schemes for automatic nanopositioning under scanning electron microscope. In IEEE International Conference on Robotics and Automation, pp. 981–986.
41.
MarturiNTamadazteBDembéléSPiatN (2016) Image-guided nanopositioning scheme for SEM. IEEE Transactions on Automation Science and Engineering, 99: 1–12.
42.
MáthéKBuşoniuLBarabásLIugaCIMicleaLBrabandJ (2016) Vision-based control of a quadrotor for an object inspection scenario. In International Conference on Unmanned Aircraft Systems (ICUAS), pp. 849–857.
43.
MeyerY (1992) Wavelets and operators. Translated from the 1990 French original by SalingerD. H.Cambridge Studies in Advanced Mathematics, vol. 37. Cambridge: Cambridge University Press, 1992.
44.
OurakMTamadazteBAndreffN. Partitioned camera-oct based 6 dof visual servoing for automatic repetitive optical biopsies. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2337–2342.
45.
OurakMTamadazteBLehmannOAndreffN. Wavelets-based 6 dof visual servoing. In IEEE International Conference on Robotics and Automation, pp. 3414–3419.
46.
ReisenhoferRKieferJKingE (2016) Shearlet-based detection of flame fronts. Experiments in Fluids57(3): 1–14.
47.
RenaudPAndreffNDhomeMMartinetP (2002) Experimental evaluation of a vision-based measuring device for parallel machine-tool calibration. In IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2, pp. 1868–1873.
48.
RichaRSznitmanRTaylorRHagerG (2011) Visual tracking using the sum of conditional variance. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2953–2958.
49.
SilveiraGMalisE (2012) Direct visual servoing: Vision-based estimation and control using only nonmetric information. IEEE Transactions on Robotics28(4): 974–980.
50.
SimoncelliEFreemanW (1995) The steerable pyramid: A flexible architecture for multi-scale derivative computation. In ICIP (3), pp. 444–447.
51.
TahriOTamtsiaAYMezouarYDemonceauxC (2015) Visual servoing based on shifted moments. IEEE Transactions on Robotics31(3): 798–804.
52.
TamadazteBPiatNLFMarchandE (2012) A direct visual servoing scheme for automatic nanopositioning. IEEE/ASME Transactions on Mechatronics17(4): 728–736.
53.
YiSLabateDEasleyGKrimH (2008) Edge detection and processing using shearlets. In IEEE International Conference on Image Processing, pp. 1148–1151.
