This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.
BandelloFLanzettaPMenchiniU (1999) When and how to do a grid laser for diabetic macular edema. Documenta Ophthalmologica97(3–4): 415–419.
2.
BeckerBCMacLachlanRALobesLAet al. (2013) Vision-based control of a handheld surgical micromanipulator with virtual fixtures. IEEE Transactions on Robotics29(3): 674–683.
3.
BergelesCKratochvilBENelsonBJ (2012) Visually servoing magnetic intraocular microdevices. IEEE Transactions on Robotics28(4): 798–809.
4.
BergelesCShamaeiKAbbottJJet al. (2010) Single-camera focus-based localization of intraocular devices. IEEE Transactions on Biomedical Engineering57(8): 2064–2074.
5.
BradskiG (2000) OpenCV Library. Dr. Dobb’s. Journal of Software Tools Available at: https://opencv.org/ (accessed 15 May 2015).
6.
BraunDYangSMartelJNet al. (2018) EyeSLAM: Real-time localization and mapping of retinal vessels during intraocular microsurgery. International Journal of Medical Robotics and Computer Assisted Surgery14(1): e1848.
7.
BrooksHL (2000) Macular hole surgery with and without internal limiting membrane peeling. Ophthalmology107(10): 1939–1948.
8.
BuchsNCVolonteFPuginFet al. (2013) Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. Journal of Surgical Research184(2): 825–831.
9.
CastañoAHutchinsonS (1994) Visual compliance: Task-directed visual servo control. IEEE Transactions on Robotics and Automation10(3): 334–342.
10.
ChenQWuHWadaT (2004) Camera calibration with two arbitrary coplanar circles. In: PajdlaTMatasJ (eds.) Computer Vision—ECCV 2004. (Lecture Notes in Computer Science, vol. 3023). Berlin: Springer, pp. 521–532.
11.
CorkePIHutchinsonSA (2001) A new partitioned approach to image-based visual servo control. IEEE Transactions on Robotics and Automation17(4): 507–515.
12.
DasHZakHJohnsonJet al. (1999) Evaluation of a telerobotic system to assist surgeons in microsurgery. Computer Aided Surgery4(1): 15–25.
13.
Diabetic Retinopathy Study Research Group (1981) Photocoagulation treatment of proliferative diabetic retinopathy: Clinical application of Diabetic Retinopathy Study (DRS) findings, DRS report number 8. Ophthalmology88(7): 583–600.
14.
FineHFWeiWGoldmanRet al. (2010) Robot-assisted ophthalmic surgery. Canadian Journal of Ophthalmology45(6): 581–584.
15.
FischlerMABollesRC (1981) Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM24(6): 381–395.
16.
GijbelsAVander PoortenEBGorissenBet al. (2014a) Experimental validation of a robotic comanipulation and telemanipulation system for retinal surgery. In: 5th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics, Sao Paulo, Brazil, 12–15 August 2014, pp. 144–150. Piscataway, NJ: IEEE.
17.
GijbelsAVander PoortenEBStalmansPet al. (2014b) Design of a teleoperated robotic system for retinal surgery. In: IEEE international conference on robotics and automation, Hong Kong, China, 31 May–7 June 2014, pp. 2357–2363. Piscataway, NJ: IEEE.
18.
HutchinsonSHagerGDCorkePI (1996) A tutorial on visual servo control. IEEE Transactions on Robotics and Automation12(5): 651–670.
19.
IdaYSugitaNUetaTet al. (2012) Microsurgical robotic system for vitreoretinal surgery. International Journal of Computer Assisted Radiology and Surgery7(1): 27–34.
20.
MacLachlanRARiviereCN (2009) High-speed microscale optical tracking using digital frequency-domain multiplexing. IEEE Transactions on Instrumentation and Measurement58(6): 1991–2001.
21.
MacLachlanRABeckerBCTabarésJCet al. (2012) Micron: An actively stabilized handheld tool for microsurgery. IEEE Transactions on Robotics28(1): 195–212.
22.
MacLarenREEdwardsTXueKet al. (2017) Results from the first use of a robot to operate inside the human eye. Investigative Ophthalmology & Visual Science58(8): 1185.
23.
MalisEChaumetteFBoudetS (1999) 2 1/2D visual servoing. IEEE Transactions on Robotics and Automation15(2): 238–250.
24.
MeeninkHCMHendrixRNausGJLet al. (2012) Robot-assisted vitreoretinal surgery. In: GomesP (ed.) Medical Robotics: Minimally Invasive Surgery. Oxford: Woodhead Publishing, pp. 185–209.
25.
MukherjeeSYangSMacLachlanRAet al. (2017) Toward monocular camera-guided retinal vein cannulation with an actively stabilized handheld robot. In: IEEE international conference on robotics and automation, Singapore, 29 May–3 June 2017, pp. 2951–2956. Piscataway, NJ: IEEE.
26.
NambiMBernsteinPSAbbottJJ (2016) A compact telemanipulated retinal-surgery system that uses commercially available instruments with a quick-change adapter. Journal of Medical Robotics Research1(2): 1630001.
27.
NasseriMAEderMNairSet al. (2013) The introduction of a new robot for assistance in ophthalmic surgery. In: 35th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013, pp. 5682–5685. Piscataway, NJ: IEEE.
28.
NooFClackdoyleRMennessierCet al. (2000) Analytic method based on identification of ellipse parameters for scanner calibration in cone-beam tomography. Physics in Medicine and Biology45(11): 3489–3508.
29.
RahimyEWilsonJTsaoT-Cet al. (2013) Robot-assisted intraocular surgery: Development of the IRISS and feasibility studies in an animal model. Eye (London, England)27: 972–978.
30.
RichaRBalickiMSznitmanRet al. (2012) Vision-based proximity detection in retinal surgery. IEEE Transactions on Biomedical Engineering59(8): 2291–301.
31.
TaylorRJensenPWhitcombLet al. (1999) A steady-hand robotic system for microsurgical augmentation. The International Journal of Robotics Research18(12): 1201–1210.
32.
UetaTYamaguchiYShirakawaYet al. (2009) Robot-assisted vitreoretinal surgery: Development of a prototype and feasibility studies in an animal model. Ophthalmology116(8): 1538–1543.
33.
ÜneriABalickiMAHandaJet al. (2010) New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In: 3rd IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob), Tokyo, Japan, 26–29 September 2010, pp. 814–819. Piscataway, NJ: IEEE.
34.
WillekensKGijbelsASchoevaerdtsLet al. (2017) Robot-assisted retinal vein cannulation in an in vivo porcine retinal vein occlusion model. Acta Ophthalmologica95(3): 270–275.
35.
WilsonJTGerberMJPrinceSWet al. (2018) Intraocular robotic interventional surgical system (IRISS): Mechanical design, evaluation, and master-slave manipulation. International Journal of Medical Robotics and Computer Assisted Surgery14(1): e1842.
36.
WrightCHGBarrettSFWelchAJ (2006) Design and development of a computer-assisted retinal laser surgery system. Journal of Biomedical Optics11(4): 41127.
37.
YangSBalickiMMacLachlanRAet al. (2012) Optical coherence tomography scanning with a handheld vitreoretinal micromanipulator. In: Annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), San Diego, CA, USA, 28 August–1 September 2012, pp. 948–951. Piscataway, NJ: IEEE.
38.
YangSLobesLAMartelJNet al. (2015a) Handheld-automated microsurgical instrumentation for intraocular laser surgery. Lasers in Surgery and Medicine47(8): 658–668.
39.
YangSMacLachlanRAMartelJNet al. (2016) Comparative evaluation of handheld robot-aided intraocular laser surgery. IEEE Transactions on Robotics32(1): 246–251.
40.
YangSMacLachlanRARiviereCN (2014) Toward automated intraocular laser surgery using a handheld micromanipulator. In: IEEE/RSJ international conference on intelligent robots and systems, Chicago, IL, USA, 14–18 September 2014, pp. 1302–1307. Piscataway, NJ: IEEE.
41.
YangSMacLachlanRARiviereCN (2015b) Manipulator design and operation of a six-degree-of-freedom handheld tremor-canceling microsurgical instrument. IEEE/ASME Transactions on Mechatronics20(2): 761–772.
42.
YuHShenJHJoosKMet al. (2013) Design, calibration and preliminary testing of a robotic telemanipulator for OCT guided retinal surgery. In: IEEE international conference on robotics and automation (ICRA), Karlsruhe, Germany, 6–10 May 2013, pp. 225–231. Piscataway, NJ: IEEE.
43.
YuHShenJHJoosKMet al. (2016) Calibration and integration of B-mode optical coherence tomography for assistive control in robotic microsurgery. IEEE/ASME Transactions on Mechatronics21(6): 2613–2623.
44.
ZhouYNelsonBJ (1999) Calibration of a parametric model of an optical microscope. Optical Engineering38(12): 1989–1995.
