Remote Sonography in Routine Clinical Practice Between Two Isolated Medical Centers and the University Hospital Using a Robotic Arm: A 1-Year Study

Abstract

Background: A robotic arm was developed by our laboratory for tele-operated echography on patients in locations isolated from a trained sonographer. The objective of the study was to evaluate, over a 1-year period, the use of the robotic arm for telesonography performed by a sonographer located at the University Hospital (Tours, France) on patients in two isolated medical centers 50 km away linked via the Internet. Materials and Methods: A nonsonographer operator (physician or paramedic) located the ultrasound probe attached to the robotic arm over the appropriate acoustic window for the organ of interest by rolling the whole robotic arm and mechanical support across the floor. The expert sonographer then telemanipulated the robotic arm via an Internet connection and adjusted the orientation of the probe until the most appropriate organ view for delivering a diagnosis was obtained. Results: Three hundred telesonography examinations were performed within 1 year: 68 (22.7%) on abdominal organs, 20 (6.7%) on pelvic organs, 138 (46%) on supraaortic vessels (carotid artery), 33 (11%) on the thyroid, 30 (10%) on leg veins, and 11 (3.7%) on the kidney and urinary tract. Telesonography could not be achieved in 10 of the 300 cases due to poor image quality on obese patients or those presenting poor echogenicity. These cases were re-examined at the university hospital by a sonographer. The rate of telesonography exams over the 1-year period was 1.5 per day for the “general population” medical site and 1 per week for the “elderly patient” medical site. Conclusions: This study demonstrated that telesonography using a robotic arm can be routinely used for providing echographic diagnoses on patients isolated from imaging centers.

Introduction

Robotic telesonography is based on the telemanipulation of a robotic arm to which an echographic probe is attached.^1,2 The robotic arm is tele-operated in real time by an expert sonographer located at the university hospital separated from the isolated medical center where the patient is located. The expert sonographer moves a dummy probe, with the movements being captured and transmitted via the Internet to the robotic arm, which then replicates the movements with the real echographic probe in contact with the patient. Echographic video captured at the patient site is transmitted in near real time by videoconference to the expert center. Currently, there is a slight delay of approximately 1–2 s for the movement of the dummy probe to be processed by the robotic arm and the resulting echographic video to be transmitted back to the expert center; however, the expert sonographers in this study have been trained to accommodate for this lag time.

The robotic telesonography technique has been previously validated in studies where the diagnoses provided by tele-operated sonography, using the robotic arm, were compared with those delivered by a sonographer acting directly on the patient. These studies confirmed that robotic telesonography can provide a reliable diagnosis in approximately 85% of the cases of abdominal, vascular, and cardiac investigations.^2
–4 There was no false-positive or false-negative results because from the same echographic image both the telesonographer and the radiologist on-site saw the same thing. In the case of a poor image the telesonographer did not deliver any diagnosis. Consequently the tool was considered sufficiently accurate and safe by the Regional Health Administration and is now used routinely.

On the basis of these encouraging results, and in order to answer questions from the Regional Health Administration, our team started a new study to evaluate the demand for telesonography in routine clinical use over a 1-year period. Two sites were chosen: a senior living community for patients 75–95 years old (Etablissement d'Hospitalisation pour Personnes Agées et Dependantes, Montoir sur Loir, France) and a general patient population 20–80 years (Richelieu Medical Center, Richelieu, France), where ultrasound experts are not located in the medical centers or in the immediate vicinity. The main objective was to evaluate the impact of the access to robotic telesonography “on demand” and to quantify the real need for echographic examination in both of these sites.

Materials and Methods

The current study was conducted over a 1-year period with tele-operated examinations being conducted between the senior living community (Etablissement d'Hospitalisation pour Personnes Agées et Dependantes) at Montoire sur Loir (7,000 habitants; 50 km away from Tours) and the medical center for the general population at Richelieu (8,000 habitants; 50 km away from Tours), with the expert sonographer at the University Hospital in Tours.

Expert and Isolated Site Equipment

The two isolated sites were connected through the Internet to the Ultrasound-Doppler Department of the University Hospital of Tours. This connection allowed for a videoconference to be established between the expert sonographer in Tours and the patient where both the patient and expert could see and talk with each other. In addition, this link allowed the expert sonographer to communicate with the nonsonographer operator at the patient center. For the echography examination, the expert used the videoconference to verify correct probe placement on the patient and to provide guidance for adjustments of the probe location if necessary. The images generated by the echograph were converted into an Ethernet format through a video converter (AXIS 243 video server; AXIS, Paris, France) and sent to the expert center via the Internet, with the delay to reach the expert remaining constant at 1–2 s. The videos were sent as Moving Picture Experts Group files collected at the Video Graphics Array output of the echograph; Digital Imaging and Communications in Medicine files were not used because of their excessive size.

At the University Hospital, the expert sonographer sent commands to the robot via an electronic module that converted the information generated by the gyroscope inside the dummy probe into a binary file. This file was sent to the patient site where it was interpreted by the electronic module controlling the robotic arm so that the movement of the dummy probe was fully replicated on the real echographic probe attached to the robotic arm.^3,4 Thus, the equipment at the expert center consisted of a dummy probe similar in size and shape to an echographic curved array probe, connected to an electronic module (Internet interface), a computer for visualizing the echographic image from the patient site, and an Internet protocol camera (AXIS 207 camera; AXIS). The robotic arm could apply the following movements to the real echographic probe: self-rotation (±180°), tilt (±90°), and rotation around the vertical axis “precession” (±180°). For the abdominal and superficial organ examinations that were conducted during this study, there was no need to have a tele-operated translation mode because the acoustic window is rather large, in contrast to tele-echocardiography, in which the acoustic window is very narrow, requiring a very accurate positioning of the probe.⁴

Each patient center was equipped with a “MELODY” robotic arm (3.5 kg; 40 × 40 × 35 cm³; Adechotech Vendome France), which was placed in contact with the patient skin by a 7-cm rigid half-circle through which the probe passed (Fig. 1). The robotic arm was either held by the nonsonographer operator (physician or paramedic) at the patient's side or suspended to a mechanical arm that could be rolled across the floor. The robotic arm was compatible with any kind of commercial probe that allowed each isolated center to use a different echograph. The first center used an echograph (model ATL 5000; Philips, Paris) with a 3.5-MHz curved array probe for deep organ examinations and a linear 7.5-MHz probe for superficial targets. The second one used an Image Point (Philips) equipped with a 3.5-MHz curved array probe for deep organ examinations and a linear 7.5-MHz probe for superficial targets.

Fig. 1.

Robotic arm (a) held by the nonsonographer or (b) suspended by a mechanical support. (c) Expert center with the dummy probe in the expert's hand and the computer with the echographic video and ambient video of the isolated site.

Patient Recruitment

Each patient was considered for telesonography examination according to the clinical findings and suspected diagnosis after the initial clinical examination. Each patient was informed of the telesonography procedures and had the choice to sign a consent form for telesonography or to ask for conventional echography to be performed by a sonographer in the closest imaging center.

Procedure for Telesonography

After the initial clinical examination, the physician from the patient site called the expert site to determine when the telesonography examination would occur. In the case of an emergency (suspicion of deep vein thrombosis, appendicitis, or renal cavity dilation) the exam was scheduled within 30 min. For all other pathologies without acute symptoms or potential complications, the examination was scheduled at a later time convenient for both testing centers.

At the time of the telesonography, the expert sonographer and the nonsonographer operator at the patient site (physician or paramedic) set up their respective devices and checked that all the connection were working properly (videoconferencing, movement of the robot). The nonsonographer operator located the echographic probe over the acoustic window of the organ of interest by translating (rolling on the floor) the support to which the robotic arm was suspended (Figs. 1 and 2). The expert at the University Hospital manipulated the dummy probe to change the orientation of the real probe on the patient and made additional probe adjustments based on the echographic video that was transmitted back to the expert site from the patient site through the videoconference. The expert sonographer asked the nonsonographer operator to adjust the image settings (gain, depth, etc.), to trigger color or pulsed wave Doppler function, and to adjust the Doppler line, angle, and sample volume location on the echograph if necessary. When the expert obtained the appropriate view of the organ required for the diagnosis, the image was captured from the computer at the expert center. At the end of the exam (which lasted approximately 25 min) the diagnosis was provided orally, by phone, to the requesting physician. Afterward an anonymized report was sent by the expert to the requesting physician via e-mail.

Fig. 2.

(a) Acoustic windows (right side). (b) Acoustic windows (left side).

Results

Telesonography examinations were performed 300 times over the 1-year interval (Table 1): 68 (22.7%) on abdominal organs, 20 (6.7%) on pelvic organs, 138 (46%) on supraaortic vessels (carotid artery), 33 (11%) on the thyroid, 30 (10%) on leg veins, and 11 (3.7%) on the kidney and urinary tract (Fig. 3). Telesonography could not be achieved in 10 of the 300 cases (3.3%), which involved obese patients or patients with absorbent tissues or when the telesonographer could not push hard enough with the robotic arm as the sonographer does directly on the patient. These cases concerned pancreas, gallbladder, or appendicitis investigation in obese patients, search for deep vein thrombosis (femoral, popliteal) in a patient with enormous and not echogenic legs, and the left kidney in a patient who was difficult to move. These patients were re-examined at the University Hospital by an expert sonographer.

Fig. 3.

(a) Atheromatous plaque, (b) gallbladder with lithiasis, and (c) kidney with a cyst.

Table 1.

Distribution of the Organs Investigated by Telesonography

ORGAN	NUMBER
Abdomen	68
Lower limb vein	30
Supraaortic vessels	138
Thyroid	33
Pelvis	20
Kidneys	7
Small parts	3
Obstetrics	1
Total tele-echography	300

The average duration of a tele-echographic examination was 24 ± 5 min. The average rate of tele-echography examination over 1 year was 1.5/day for the general population medical center and 1/week for the elderly patient medical site. Of the 300 examinations, 10% of the telesonography examinations were performed within 30 min after the patient site called the expert (potential emergency situation), with the others being scheduled within the following 24–36 h. The Internet connection allowed for a good transfer of controls sent by the expert to the robotic arm and for the return of the echography video to the expert center. The images received by the expert center were of lower quality compared with the originals but were still of sufficient quality for making a reliable diagnosis. The echographic image after being compressed when leaving the patient site and then decompressed when reaching the expert center had lost gray levels, brightness, and resolution. Nevertheless, even though the image reaching the expert was not as clean as on the patient echograph, the limits and content of the organs were still interpretable. There was a slight delay (1–2 s) between the movement of the dummy probe by the expert sonographer and the change in the echographic video, but the expert sonographer was able to overcome this limitation after 1 h of training.

Discussion

This study demonstrated that robotic telesonography can be used in routine clinical practice for acquiring echographic diagnoses on patients in medical centers without trained sonographers or in areas where imaging centers are not available. Telesonography provided echographic examinations with a short delay and delivered the diagnoses in real time to the requesting physician during the examination.

In this study the robotic arm device was used to perform 300 telesonography examinations on patients located 45–50 km from the expert sonographer. The robotic arm, which moved the real echographic probe on the patient, operated like an extension of the expert's hand toward the patient. The contribution of the physician or paramedic at the isolated site was limited to positioning and maintaining the robotic arm on the patient at the location indicated by the expert and to adjusting setting and activating functions on the echograph. In addition, the nonsonographer's presence at the side of the patient provided a human interface between the robotic arm and the patient, not only enabling the nonsonographer to move the robot over the patient's body but also to remove it quickly from the patient at any moment. The expert, the nonsonographer operator, and the patient were able to communicate in real time through videoconference, which allowed the expert not only to guide the positioning of the robotic arm on the patient's body, but also to talk with the patient and direct the patient to perform different maneuvers such as to hold a breath or take a deep breath. This link also allowed the expert sonographer to talk with the physician or paramedic to obtain additional medical information. None of the patients referred for echographic examinations refused to undergo the robotic telesonography. In addition, no patients undergoing this examination reported any discomfort or unease when the nonsonographer applied the rather impressive robotic arm on their body.

The delay, although short, between the order sent by the expert to the robotic arm and the return of the video image induced by this order was responsible for the longer duration of the telesonography compared with direct echography on the patient. When the expert moved the dummy probe, the effect of such maneuver on the echographic image appeared only 1–2 s later. Thus if the expert applied with his hand several rapid movement on the probe, the changes in the image related to these movements reached him in cascade, and he was unable to return to any of the previous positions of the probe. Consequently the expert had to apply on the dummy probe only movement of low amplitude and at low speed, in contrast to the sonographer who can quickly move the probe on the patient's skin.

The selected echographic images for the medical report and a copy of the echographic report were stored at the expert center and sent via e-mail to the physician of the isolated site. The patient name did not appeared on the echographic views, and the expert sent the report without the name of the patient, only a reference number given by the isolated site physician. When this physician received the report, the patient's name was entered, and the report was stored in his computers of the isolated medical site.

In the current study, there were no false-positive diagnoses among the 300 tele-echographic diagnoses as the expert only made a diagnosis on images of organs perfectly visualized in the long and short axis and scanned entirely. Abnormal findings (tumor, lithiasis, or thrombosis) diagnosed by the telesonography right after the first clinical examination at the isolated site were confirmed later by an expert sonographer during the assessment and treatment of the abnormalities.

Other methods have been developed to remotely investigate abdominal organs and the heart, with most based on “remote guidance.” The principle is that the operator by the side of the patient is trained to perform basic views of each organ (long axis and short axis) and is guided by an expert sonographer through a videoconference system.^5

–8 Remote guidance has been successfully tested on normal adult subjects and infants but not on adult patients. Additionally, the operator by the side of the patient was always trained to perform basic echography, which is not necessary when using robotic tele-operated echography.

Telesonography provided significant advantages to the patient: there was no need to move the patient by ambulance to the closest radiology center, the patient did not have to wait several days before the echography occurred, and the patient did not spend hours waiting at the radiology center as is the case today. Additionally, telesonography could be performed right after the first clinical examination, and the treatment could start very quickly. In a limited number of cases the diagnosis provided by the telesonography convinced the physician to emergency transfer the patient to the University Hospital. This system also provided additional benefits to the healthcare system as money was saved by not having to transport the patient to an imaging facility. Additionally, the main hospital was less overcrowded by patients arriving just for echography examinations, which is normal in a significant number of cases. Lastly, telesonography provided early diagnoses of conditions leading to faster prescriptions of treatments, which should result in improved patient outcomes and potentially less time spent in long-term care facilities.

Limitation of the System

The main limitation consisted in the limited movement of the robotic arm compared with the sonographer's hand. Also, the volume, weight, and the ergonomics of the robotic arm suspended by the mechanical arm made difficult the positioning of the probe in front of the acoustic windows, even when moving the patient. Another limitation of the telesonography was related to the physical limitation of the patient when he or she was obese, unable to hold his or her breath, or nonmovable. Also limiting was the impossibility of pushing hard with the probe on the patient's skin as the sonographer, which in some cases could make it more difficult to get echographic images of good quality. Finally, it was not possible to control the settings and function of the echograph, which increased the duration of the telesonography as it required the operator to ask and be guided by the side of the patient to do this.

At present, the medical reliability and usefulness of telesonography for the isolated patient has been demonstrated, but it raises some financial concern not solved yet even in our country. Funds for the telesonography procedure will be paid to the hospital where the expert is located, but still unclear is how to pay the physician for the 20 min he or she spent managing the robotic arm on the patient, as well as who will pay for the equipment (approximately 100,000 euros): the local administration (Mairie) Regional Health Administration or the national healthcare administration Ministry of Health. At present, the Ministry of Health has purchased two systems for the first center.

Conclusions

The present study confirmed that telesonography, conducted over a period of 1 year, provided similar information as conventional echography performed in an imaging center. The number of telesonography examinations requested by the isolated medical centers remained at a reasonable level despite this imaging mode being available upon demand. The availability of telesonography in general medicine practice facilitated the medical decision-making process, making it faster and saving several days of waiting for the patient. At the patient site, telesonography did not generate any additional costs and did not require additional manpower. It also contributed to saving money for the healthcare system by avoiding patient transportation costs and allowing for earlier treatment of conditions, potentially resulting in improved patient outcome and less time in care facilities.

Footnotes

Acknowledgments

The authors want to acknowledge Joel Blouin (hospital engineer) and Maryannick Porcher (hospital sonographer) for their active contribution to this study.

Disclosure Statement

No competing financial interests exist.

References

Vieyres

, Poisson

, Courrege

, Merigeaux

, Arbeille

. The TERESA project: From space research to ground tele-echography. Ind Robot, 2003; 30:77–82.

Arbeille

, Poisson

, Ayoub

, Vieyres

, Chevillot

, Hervé

, Porcher

, Boulay

. Echographic examination in isolated sites controlled from an expert using a 2D echography guided by a robotic arm. Ultrasound Med Biol, 2003; 29:993–1000.

Arbeille

, Capri

, Ayoub

, Kieffer

, Georgescu

, Poisson

. Use of a robotic arm to perform tele operated abdominal ultrasound. Am J Roentgenol, 2007; 188:317–322.

Arbeille

, Provost

, Zuj

, Dimouro

, Georgescu

. Tele-operated echocardiography using a robotic arm and an Internet connection. Ultrasound Med Biol, 2014; 40:2521–2529.

Awadallah

, Halaweish

, Kutayli

. Tele-echocardiography in neonates: Utility and benefits in South Dakota primary care hospitals. S D Med, 2006; 59:97–100.

Hamilton

, Sargsyan

, Martin

, Garcia

, Melton

, Feiveson

, Dulchavsky

. On-orbit prospective echocardiography on International Space Station crew. Echocardiography, 2011; 28:491–501.

Otto

, Shemenski

, Drudi

. Real-time tele-echocardiography: Diagnosis and management of a pericardial effusion secondary to pericarditis at an Antarctic research station. Telemed J E Health, 2012; 18:521–524.

Randolph

, Hagler

, Khandheria

, Lunn

, Cook

, Seward

, O'Leary

. Remote telemedical interpretation of neonatal echocardiograms: Impact on clinical management in a primary care setting. J Am Coll Cardiol, 1999; 34:241–245.