Investigating the Need for Point-of-Care Robots to Support Teleconsultation

Abstract

Background:

Development of a point-of-care (POC) consultation system based on telepresence robots is needed to enable effective decision-making by medical staff at care sites.

Introduction:

This study aimed to identify essential features and functional requirements of teleconsultation robot systems and predict potential administrative and clinical issues.

Materials and Methods:

Surveys were conducted with 90 health care professionals and additional focus group interviews with 4 physicians and 5 nurses. The questionnaire for the surveys was developed by the authors. Survey results were analyzed using descriptive statistics, and content analysis was used to extract themes from the unabridged transcripts of focus group interviews.

Results:

The most desired functionalities were related to patient evaluation and facilitation of communication, including measuring vital signs, and medical record sharing and delivery. Nurses and physicians reported different needs for human-robot interactions. Nurses valued robotic functions such as voice command, automatic camera movement, voice recognition with contextual perception, and recognition of nonverbal signals. The thematic analysis of the interviews yielded four themes: major functions, usability, expected effects, and potential issues. The results indicated that robots should primarily be employed to support communication between medical professionals. The major expected effects included prevention of treatment delays and decision-making assistance. Participants believed that teleconsultation robots would be helpful, but had concerns, including anxiety about the robots and judgment errors.

Conclusions:

Using robots in health care institutions may support effective communication among medical staff, thus contributing to health care improvement. In the future, an actual POC robot system will be developed and its effectiveness evaluated.

Introduction

Telehealth technologies eliminate location as a barrier to accessing quality treatment and care, creating new opportunities for health care providers to engage with patients and other professionals across distances in real time. Several studies have reported that medical treatment and consultation using videoconferencing results have improved health care quality, and are effective for educating medical staff.^1
–3 Most studies, however, have been limited because the videoconferencing was conducted at a fixed place and time. Developing a point-of-care (POC) consultation system based on telepresence robots may provide value to enable effective decision-making by medical staff at care sites. Telepresence robots allow offsite medical professionals to move, look around, communicate, and participate from remote locations.⁴ A telepresence system basically consists of a movable robot and a remote station. The health care provider of a remote station can remotely control robot movement and camera angle. Voice recognition technology of the robot enables the recognition and translation of verbal information into text, which then can be used in automated robot control. Care quality could be improved by establishing a POC services system that considers immediacy, onsite presence, and patient safety.

POC means that health care providers deliver health services to patients at the site where the patient is located at the time of care.^5,6 If real-time collaboration can be conducted at the place where the patient is, then diagnosis and treatment planning can be achieved more quickly. Accordingly, we are developing a POC robot as a tool for teleconsultation and interprofessional education, and this study's goal was to identify functions and traits to be included in the system. Specifically, it aimed to identify essential features and functional requirements of teleconsultation robot systems for patient support, identify which medical departments require teleconsultation and education, and predict potential administrative and clinical issues.

Materials and Methods

We conducted a survey to better understand how teleconsultation robots could fill health care professionals' needs and identify issues in implementing such technology. Quantitative data were collected through online surveys, followed up by focus group interviews for qualitative data. The study received approval from the Institutional Review Board of the Seoul National University Hospital (H-1607-174-779).

Participants

Convenience sampling was used to obtain the participants. All participants were health care professionals (physicians and nurses) currently working in University Hospital in Korea. The survey period was September 2 to November 7, 2016.

Respondents were recruited through a bulletin board notice posted in a university hospital in Korea and group chat rooms on social media services for the surveys. The notice asked people to visit a given web link for additional instructions. Incomplete and insincere responses were excluded from the 182 surveys logged, resulting in a total of 90 respondents for the surveys.

We recruited physicians and nurses to participate in the interview separately from the survey participants. Purposive sampling was then used to select participants for the focus group interviews to obtain a variety of opinions from different medical and nursing fields. Among the participants, we recruited physicians specializing in plastic surgery, radiology, internal medicine, and emergency medicine, and nurses working in the intensive care unit, general ward, operating room, and consignment hospitals abroad. We divided the participants into two groups: four physicians and five nurses and held two sets of interviews in September and October 2016.

Quantitative Data

The initial online questionnaire was developed by researchers in health care and nursing informatics (Table 1).^{2,7

–14} It included 2 questions about which departments would use the robots and 38 on essential features of the teleconsultation robot.

Table 1.

Questionnaire Categories

CATEGORIES	NUMBER OF ITEMS	EXAMPLES
General functions (task)	12	• Sphygmomanometer • Electronic stethoscope • Laser pointer (for indicating) • Medical information delivery (e.g., admission note)
Autonomy and interfacing	17	• Bidirectional videoconferencing system • Wireless system • Touch screen • Automatic navigation • Obstacle avoidance: onboard sensors detect impending collisions and prevent the robot from coming into contact with objects in close proximity
Social ability	2	• Recognition of nonverbal signals (e.g., users' facial expressions) • Voice interaction (context-aware speech recognition)
Form factor (shape and appearance)	7	• Limb (tool) movements • Robot's human likeness (choices: 5 different pictures) • Robot's gender (choices: male, female, intersex, or presented on the screen without defining gender) • Robot's method of movement (choices: two legs, wheels, or treads)
Total number of items	38

For questions on the application of teleconsultation robots, participants could select multiple medical departments (internal medicine, general surgery, and so on) and divisions (general wards, emergency room, intensive care unit, and so on). For questions on essential features of teleconsultation robots, 38 questions explored general functions, traits related to autonomy and interfacing, social functions, and form factors.

The content validity index (CVI)¹⁵ was obtained from 6 experts, including professors in medical and nursing informatics, and after deleting 10 items with CVI <0.8, a total of 38 items were finally selected. The average scale-CVI of the final items was 0.87. In addition, we distributed a preliminary questionnaire to 10 graduate students in nursing informatics, and the final questionnaire incorporated their comments on the survey's readability.

We asked respondents how important the functions and traits of teleconsultation robots were (Table 1). Responses were measured using a four-point Likert scale ranging from “Not important at all” (1 point) to “Very important” (4 points) for 35 items. The three questions on robots' human likeness considering form factors, gender, and method of movement were presented with response options using photographs. Before the online survey, participants viewed an ∼2-min video explaining the teleconsultation robot (Fig. 1). They could access the questionnaire either through quick response codes or the URL and complete the survey at any time or place, thus ensuring their privacy. Only participants who agreed to the instructions were invited to complete the survey. If participants attempted to take the survey twice, they received a message indicating that the questionnaire was already completed, thus preventing duplicate responses. The questionnaire was administered using the online survey site SurveyMonkey.

Fig. 1.

Screenshots of used video clip. SNUH, Seoul National University Hospital.

The results were processed using IBM SPSS^® 22.0 (IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp.). The demographic characteristics and responses to items relating to application area and robots' essential elements and functions were analyzed using constants, percentages, means, and standard deviations. A t-test was used to compare the needs of the physicians and nurses.

Qualitative Data

We then collected qualitative data through additional focus group interviews with five nurses and four physicians. All participants consented to participate. They were presented with structured open-ended questions regarding their needs and the possible issues in instituting a teleconsultation robot.

Before the interviews, participants watched a short video explaining the use of robots (Fig. 1). Each group interview lasted for 60–90 min, and ended when it no longer yielded new ideas or opinions. Each focus group was recorded completely. To remove bias and improve the credibility and rigor of the results, a team consisting of two or more researchers, including the moderator and research assistants in charge of field notes, attended the interviews, allowing for researcher triangulation.¹⁶ Interview questions were based on previous research^11,17 and included the following open-ended items:

What are the current limitations of teleconsultation?

What are the expected outcomes of developing and using a robot?

What are the most important needs that such a robot should fulfill?

What form(s) should such a robot have?

What are the potential administrative issues in using such a robot?

What are the potential clinical issues when using such a robot?

All recordings were transcribed and coded based on similar concepts. The data were then reviewed to determine similarities, differences, meaning, and structures. This allowed us to identify what health care professionals believed to be the functional and technical specifications and expected issues of teleconsultation robots.

Results

Survey Participants

Ninety participants, including 39 physicians and 51 nurses, were surveyed; 65.6% were female and the mean age was 32 ± 6.5 years. The physicians had various specialties and the nurses worked in various departments (Table 2).

Table 2.

Survey Participants (N = 90)

TYPE	n	%
Gender
Male	31	34.4
Female	59	65.6
Age (Mean ± SD)	32.0 ± 6.5
Position
Physician	39	43.3
Nurse	51	56.7
Specialty (Physicians only)
Respiratory	5	12.82
Surgery	5	12.82
Plastic surgery	4	10.26
Family medicine	3	7.69
Pediatrics	3	7.69
Neurology	2	5.13
Emergency medicine	2	5.13
Rehabilitation medicine	2	5.13
Orthopedics	2	5.13
Other	11	28.16
Departments (Nurses only)
ICU	13	23.21
Internal medicine	11	19.64
Emergency room	7	12.5
Surgery	6	10.71
Pediatrics	4	7.14
Outpatient	3	5.36
Other	7	21.44

ICU, intensive care unit; SD, standard deviation.

Differences in Priorities Between Physicians and Nurses

The 10 most important functional requirements or traits for teleconsultation robots were as follows: heart rate monitor, wireless system, sphygmomanometer, medical information delivery, screen zoom in/out, oximeter, examination record delivery, touch screen, voice commands, and screen sharing (Table 3). A face presented on a monitor was preferred over a humanoid form (61.0%). Among the participants, 66.7% preferred screen displays without labeling the robot's gender. Wheels (71.1%) were preferred for the method of movement, followed by treads (20.0%) and two legs (8.9%).

Table 3.

Functional Priority of the Teleconsultation Robots

FUNCTIONALITY	TOTAL (n = 90)	PHYSICIANS (n = 39)	RN (n = 51)	DIFFERENCES
FUNCTIONALITY	M ± SD			t(p)
Heart rate monitor	3.44 ± 0.56	3.31 ± 0.57	3.55 ± 0.54	2.015 (0.043)
Wireless system	3.37 ± 0.64	3.28 ± 0.65	3.43 ± 0.64	1.091 (0.278)
Sphygmomanometer	3.36 ± 0.66	3.26 ± 0.64	3.43 ± 0.67	1.253 (0.214)
Medical information delivery (e.g., admission note)	3.33 ± 0.56	3.45 ± 0.64	3.23 ± 0.54	1.925 (0.057)
Screen zoom in/out	3.33 ± 0.54	3.28 ± 0.56	3.37 ± 0.53	0.785 (0.434)
Oximeter	3.32 ± 0.58	3.23 ± 0.58	3.39 ± 0.57	1.320 (0.190)
Examination record delivery (e.g., laboratory test result)	3.29 ± 0.55	3.13 ± 0.52	3.41 ± 0.54	2.524 (0.013)
Touch screen	3.29 ± 0.64	3.26 ± 0.55	3.31 ± 0.71	0.433 (0.666)
Human verbally provides commands to robot	3.28 ± 0.65	3.10 ± 0.72	3.41 ± 0.57	2.274 (0.025)
Screen sharing	3.27 ± 0.60	3.23 ± 0.54	3.29 ± 0.64	0.498 (0.620)
Thermometer	3.26 ± 0.59	3.21 ± 0.57	3.29 ± 0.61	0.705 (0.482)
Bidirectional videoconferencing system	3.26 ± 0.66	3.26 ± 0.64	3.25 ± 0.68	0.011 (0.992)
Onscreen messages to notify the user of such conditions as loss of far end video, incomplete or dropped connections, mute, etc.	3.24 ± 0.64	3.08 ± 0.70	3.37 ± 0.56	2.213 (0.029)
Video and image play	3.24 ± 0.71	3.18 ± 0.72	3.29 ± 0.70	0.759 (0.450)
Image file sharing	3.24 ± 0.57	3.18 ± 0.56	3.29 ± 0.58	0.950 (0.345)
Automatic camera movement	3.23 ± 0.58	3.08 ± 0.62	3.35 ± 0.52	2.283 (0.025)
Videotaping or sound recording	3.22 ± 0.56	3.13 ± 0.57	3.29 ± 0.54	1.409 (0.162)
Obstacle avoidance: onboard sensors detect impending collisions and prevent the robot from coming into contact with objects in close proximity	3.21 ± 0.66	3.22 ± 0.61	2.97 ± 0.63	1.040 (0.301)
Option to view the picture sent as well as the picture received simultaneously (known as “picture-in-picture” or PIP)	3.18 ± 0.59	3.13 ± 0.77	3.27 ± 0.57	1.055 (0.294)
Height adjustment	3.18 ± 0.59	3.10 ± 0.55	3.24 ± 0.59	1.091 (0.278)
Blood glucose meter	3.17 ± 0.52	3.10 ± 0.55	3.22 ± 0.50	1.014 (0.313)
Video screen capture	3.16 ± 0.54	3.10 ± 0.55	3.20 ± 0.53	0.815 (0.417)
Sending message	3.16 ± 0.54	3.08 ± 0.58	3.22 ± 0.61	1.092 (0.278)
Docking station: manual or automatic drive to a battery charging station connected to an electrical outlet	3.10 ± 0.64	2.95 ± 0.72	3.22 ± 0.54	2.004 (0.048)
Direction change	3.10 ± 0.56	3.05 ± 0.51	3.14 ± 0.60	0.717 (0.475)
Voice recognition with contextual perception (voice interaction)	3.03 ± 0.68	2.82 ± 0.68	3.20 ± 0.63	2.694 (0.008)
Movement of robot's arms and legs	3.01 ± 0.68	2.77 ± 0.67	3.20 ± 0.63	3.096 (0.003)
Statistical value display	3.00 ± 0.67	2.95 ± 0.72	3.04 ± 0.63	0.632 (0.529)
Pen light	2.94 ± 0.62	2.72 ± 0.51	3.12 ± 0.65	3.156 (0.010)
Electronic stethoscope	2.92 ± 0.75	2.69 ± 0.80	3.10 ± 0.67	2.615 (0.010)
Recognition of nonverbal signals (e.g., users' facial expressions)	2.92 ± 0.69	2.72 ± 0.72	3.08 ± 0.63	2.527 (0.013)
Automatic navigation	2.91 ± 0.73	2.92 ± 0.70	2.90 ± 0.76	0.135 (0.893)
Ruler	2.84 ± 0.58	2.79 ± 0.52	2.88 ± 0.62	0.708 (0.481)
Laser pointer (for indicating)	2.61 ± 0.73	2.67 ± 0.66	2.57 ± 0.78	0.629 (0.531)
Stress evaluation	2.57 ± 0.67	2.41 ± 0.68	2.69 ± 0.65	1.964 (0.053)

RN, registered nurses; SD, standard deviation.

Nurses generally scored these items higher than did physicians. Items with statistically significant differences included heart rate monitor (p = 0.043), examination record delivery (p = 0.013), verbal commands to robots (p = 0.025), onscreen messages to notify the user of problematic conditions (p = 0.029), automatic camera movement (p = 0.025), docking station (p = 0.048), voice recognition with contextual perception (voice interaction) (p = 0.008), movement of robot's arms and legs (p = 0.003), pen light (p = 0.010), electronic stethoscope (p = 0.010), and recognition of nonverbal signals (p = 0.013) (Table 3).

The internal medicine and surgical wards and the operating room were identified as having the greatest needs for the robot, although the differences in need among the treatment divisions were not significant (Fig. 2).

Fig. 2.

Priority of the department and treatment divisions to which the robot would be applied.

Focus Groups

The thematic analysis of the focus group interviews with five nurses and four physicians yielded four themes: (1) major functions, (2) usability, (3) expected effects, and (4) potential issues.

Major functions

Interviewees agreed that the robots' most important function is to support communication between medical professionals. They emphasized that the robots need to be able to send high-definition videos so that patients can be evaluated in detail. In emergency situations, the ability to communicate details quickly through voice or video rather than a text-based consultation was suggested as a potential strength of robotic teleconsultation. The participants also said that various devices, including tablet PCs and cellphones, need to be used so that medical professionals can view patient data on the move. One respondent remarked as follows:

“The robot will need all the possible functions available to assess the patient. To compensate for the fact that the physician can't touch the patient directly, the delivered video must be high definition. The robot also needs to effectively eliminate background noise to facilitate communication through the microphone. If the robot can communicate remotely and send and receive the necessary data quickly, it can improve the efficiency of consultation.” [Physician]

Usability

User convenience must be considered to maximize a robot's usability. A simple user interface should be implemented, and a manual is especially needed so that even beginners can use the robot. In addition, easy battery charging mechanisms such as auto-docking could be considered.

The robots' movement can also affect their usability. Some said that robots' strengths are quick patient evaluations by auto-navigation during emergency situations or a tracking function to follow medical professionals. In contrast, others mentioned that self-driving should be optional in the interest of patient safety and frequent robot failures. One participant said the following:

“You can consider using it to not only help medical professionals but also to use auto-navigation to help patients locate the hospital. Or, the robot can do the patient rounds instead. It would be nice if the nurse could get the camera footage taken by the robot transferred to the station monitor, or the nurse could accompany the robot during patient rounds. Physicians can also check the condition of their patients from anywhere.” [RN]

Expected effects

The major expected effects included prevention of treatment delays and assistance in decision-making regarding transferring patients. Participants agreed that instant consultation with other health care professionals would not only provide fast and accurate treatment but also help greatly with time-sensitive decisions, including transferring patients to higher-level medical institutions. One respondent remarked as follows:

“It can help reduce the drawbacks of medical facilities being densely concentrated in metropolitan areas. Particularly, it is difficult to have all the specialists in each department unless it is a large hospital, so it will be of great help in cases that require a second opinion in a timely manner. It will be cost-effective because robots will eliminate the need for unnecessary patient transfers just for short consultations, and it will also positively impact patients' clinical outcomes.” [Physician]

Potential issues

The major issues anticipated included anxiety about robots, judgment errors, and limited use in some divisions. Pediatric patients or those with psychological disorders may be repulsed by robots or experience increased anxiety. Patients may also not trust the decision of medical professionals who evaluate them through robots. To reduce anxiety, robotic consultations should be explained in layman terms. In terms of practical application, it is also necessary to consider that other patients may have to wait during remote consultations.

From the perspective of medical professionals, there is a danger of incorrect judgments based on limited information and that the source of responsibility may be unclear in such cases. Remotely transferred data may not be as reliable as that collected in person. Some participants questioned the application in certain medical departments. One mentioned the following:

“There may be cases in which patient data and telecommunication are insufficient to evaluate a patient. For example, it will be difficult in cases that require direct touching for data collection. Also, children and elderly patients may not communicate well and that may interfere with the medical evaluation interview.” [Physician]

However, some interviewees opined that robots can be used in all treatment divisions if they can transfer high-quality videos without delay.

Finally, other possible issues included the inability of medical professionals to join the consultation in real time owing to problems such as time zone differences in intercountry remote consultations, and administrative issues, including the appropriate fee to charge for robotic consultations. Possible ethical and privacy problems were also emphasized.

Discussion and Conclusions

This study showed that the functional and technical aspects of telepresence robots must facilitate communication in real time. Carefully assessing the potential needs and involving physicians and nurses early in the planning process was a key strategy for ensuring future compliance. This study also sought ways to increase acceptance by identifying solutions to the expected clinical and behavioral issues in advance.

The results show that the most highly demanded functions for robots are patient evaluation and the facilitation of smooth communication between medical professionals working together for joint consultations. Of the two telemedicine transport modes—store and forward and real time—the benefits of using the real-time mode is that patients can be seen and evaluated immediately. In addition, using peripheral devices, including electronic stethoscopes and otoscopes to share patients' images or obtain clinical data immediately, is a great advantage.¹⁸ Of the highest priority functions in this study, the heart rate monitor, sphygmomanometer, and oximeter measure clinical data, while wireless systems, medical information delivery, and examination record delivery concern effective instant communication regarding the robot's autonomy and interfacing. Participants said developing a program that allows communication through a tablet PC or smartphone is necessary to facilitate real-time consultations. It was also reported that live teleconsultations need to be in the proximity of physicians, nurses, and patients or go completely mobile, such as through secured tablets and iPods.¹⁹ Response time is an important factor for facilitating rapid identification and intervention for diseases, especially for emergency consultations.³ In the management of crises or emergencies, even personal digital assistants can be used for data transfer and video calls.³

The results of the survey and interviews emphasize the importance of a wireless system, ability to transfer large volume medical videos quickly, and high-quality images. A study examining the differences in video quality, audio quality, and overall satisfaction between consultations using a wireless tablet and a wired telemedicine cart for consultation found that the latter was significantly higher in all three.²⁰ The medical professionals in this study noted the audio drop and poor connection quality as limitations of a wireless tablet, and the high video quality and stability of connection as the main advantages of wired telemedicine carts. Therefore, there must be an option to choose either a wireless or wired network connection depending on the consultation case and type of medical information to be shared.

Nurses and physicians had different requirements for human-robot interaction (HRI). Nurses were more likely to value robotic functions, including voice command, automatic camera movement, voice recognition with contextual perception, and recognition of nonverbal signals, than physicians. In other words, nurses valued HRI more than physicians, probably because nurses have more need for HRI to alleviate some of the demands for direct care or the assistive role they play, but future studies are required to verify this claim.

Moreover, nurses valued maintenance-related functions such as onscreen messages to notify the user of problematic conditions or docking stations more than physicians did. This discrepancy seems to be because nurses are usually responsible for maintaining medical devices²¹ and seek assistance with that responsibility.

The kind of clinical disciplines that can be added to the program depends on many factors, but these are mostly dictated by the clinical needs of regional hospitals and physician leaders.¹⁹ No significant difference in the need for the system was observed among the treatment divisions. This means that a robot can be widely used in all treatment divisions. However, some voiced the contrasting opinion that introducing robots may be difficult in cases where the physician needs physical contact to properly evaluate or the patient is unable to communicate clearly.

While the economic effect of telemedicine has been previously addressed in many health care systems, finding a way to compensate consulting physicians and other medical personnel for their time spent answering telemedicine referrals has not been adequately explored in the literature.¹⁹ Particularly, the introduction and usage of telehealth in Korea is behind that in other countries, largely due to a lack of policy and adequate legislation.²² As such, there has been little discussion of the compensation for telehealth, including teleconsultation, in South Korea. The medical professionals in this study also emphasized the need for adequate motivation to improve acceptance among health care providers. Physicians who provide robotic consultations need to be compensated appropriately, and additional personnel will be needed to maintain such services.

One author points to medical professionals' lack of technological experience owing to the hurdle of telemedicine services, problems of trust between medical professionals in cases of interinstitutional joint consultations, and resistance to technology that stems from the misconception that teleconsultation is time-consuming.²³ In addition, technical problems and political and legal issues such as the low quality of transferred images or patient data, and different institutions using different telemedicine or teleconsultation platforms may also prevent the service from being implemented. Finding solutions to these potential issues before implementing the service will greatly affect the acceptance of this technology. Some studies have noted that the telehealth systems apply artificial intelligence (AI) through various means, such as an automated interpretation system or predictive models, to detect potential medical problems and to support decision-making.^24,25 Despite these attempts, there is still noticeable skepticism regarding such applications,²⁴ and we could not find a high level of demand for process innovation such as AI in this study. To boost the adoption of AI by the medical community, large-scale studies are required to validate these results.²⁵

This study showed that the key function of telepresence robots for POC services is to facilitate communication and that they can be widely used in all treatment divisions. The POC robot service will support effective communication among medical staff, thereby contributing to the improvement of health care. In the future, an actual POC robot system will be developed and its effectiveness must be evaluated. Through repetitive simulations, user needs should be continuously reflected in the development process. Based on the results of the needs assessment, the development of consultation/training programs, including processes and guidelines, operation of a test bed, and evaluation of the usefulness of the system are essential.

Despite our valuable findings, this study has several limitations. First, the survey was conducted in a single university hospital. As the environment, severity of patients' conditions, and purpose of admission may differ depending on the size of the hospital, further research is needed to analyze the demand for robots in health care institutions of various sizes. Moreover, the demand for robots may differ depending on the characteristics of physicians and nurses. As we were unable to analyze the differences in demand stemming from these variables, future studies should assess demand according to the age, gender, department, and assigned wards of health care professionals. Finally, all the survey responses were made using a four-point Likert scale, which produces distortions in the results obtained due to the absence of a mid-point. Therefore, the results should be carefully interpreted.

Footnotes

Acknowledgments

This material is based on work supported by the Ministry of Trade, Industry & Energy (MOTIE, Korea) under the Industrial Technology Innovation Program. No.10063098, “Telepresence robot system development for the support of POC (Point Of Care) service associated with ICT technology.”

The authors greatly appreciate HyunJin Joo and EunJin Hwang for their support of this study.

Disclosure Statement

The authors have no conflicts of interest to disclose.

References

Dharmar

, Romano

, Kuppermann

, Nesbitt

, Cole

, Andrada

, et al. Impact of critical care telemedicine consultations on children in rural emergency departments. Crit Care Med, 2013; 4:2388–2395.

Moffatt

, Eley

. The reported benefits of telehealth for rural Australians. Aust Health Rev, 2010; 34:276–281.

Wang

, Li

, Zhu

, Hong

, An

, Yang

, et al. Assessment of utilization and cost-effectiveness of telemedicine program in western regions of China: A 12-year study of 249 hospitals across 112 cities. Telemed J E Health, 2016; 22:909–920.

Virk

. The autobots are here to stay: Robotics in medicine. J Postgrad Med Educ Res, 2015; 49:2.

Ebell

. Information at the point of care: Answering clinical questions. J Am Board Fam Pract, 1999; 12:225–235.

Lee

, Kim

. Scenario-based assessment of user needs for point-of-care robots. Healthc Inform Res, 2018; 24:12–21.

Beer

, Prakash

, Mitzner

, Rogers

. Understanding robot acceptance. Georgia Institute of Technology; 2011. Available at https://smartech.gatech.edu/bitstream/handle/1853/39672/HFA-TR-1103-RobotAcceptance.pdf?sequence=1&isAllowed=y (last accessed September 18, 2018 ).

Boudreaux

, Haskins

, Harralson

, Bernstein

. The remote brief intervention and referral to treatment model: Development, functionality, acceptability, and feasibility. Drug Alcohol Depend, 2015; 155:236–242.

Krupinski

, Burdick

, Pak

, Bocachica

, Earles

, Edison

, et al. American Telemedicine Association's practice guidelines for teledermatology. Telemed J E Health, 2008; 14:289–302.

10.

Sabesan

, Larkins

, Evans

, Varma

, Andrews

, Beuttner

, et al. Telemedicine for rural cancer care in North Queensland: Bringing cancer care home. Aust J Rural Health, 2012; 20:259–264.

11.

Shore

. Telepsychiatry: Videoconferencing in the delivery of psychiatric care. Am J Psychiatry, 2013; 170:256–262.

12.

Smith

, Gray

. Telemedicine across the ages. Med J Aust, 2009; 190:15–19.

13.

Tsao

, Demaerschalk

. Teleneurology in practice: a comprehensive clinical guide. New York, NY: Springer, 2015.

14.

Yellowlees

, Shore

, Roberts

. Practice guidelines for videoconferencing-based telemental health–October 2009. Telemed J E Health, 2010; 16:1074–1089.

15.

Polit

, Beck

. The content validity index: Are you sure you know what's being reported? Critique and recommendations. Res Nurs Health, 2006; 29:489–497.

16.

Speziale

, Streubert

, Carpenter

. Qualitative research in nursing: advancing the humanistic imperative. Philadelphia, PA: Lippincott Williams & Wilkins, 2011.

17.

Khan

. Attitudes towards intelligent service robots. Technical Report No. TRITA-NA-P9821, NADA, KTH, Stockholm, Sweden, 1998. Available at http://wproj.nada.kth.se/midhistoric/ftp.nada.kth.se/IPLab/TechReports/IPLab-154.pdf (last accessed September 18, 2018 ).

18.

Hoyt

, Yoshihashi

. Health informatics: practical guide for healthcare and information technology professionals. Pensacola, FL: Informatics Education, 2014.

19.

Latifi

, Gunn

, Bakiu

, Boci

, Dasho

, Olldashi

, et al. Access to specialized care through telemedicine in limited-resource country: Initial 1,065 teleconsultations in Albania. Telemed J E Health, 2016; 22:1024–1031.

20.

Beck

, Jensen

, Putzier

, Stubert

, Stuart

, Mohammed

, et al. Developing a newborn resuscitation telemedicine program: A comparison of two technologies. Telemed J E Health, 2018; 24:481–488.

21.

Money

, Barnett

, Kuljis

, Craven

, Martin

, Young

. The role of the user within the medical device design and development process: medical device manufacturers' perspectives. BMC Med Inform Decis Mak, 2011; 11:15.

22.

, Park

, Jo

, Kim

. Current status and progress of telemedicine in Korea and other countries. Healthc Inform Res, 2015; 21:239–243.

23.

de Souza

CHA

, Morbeck

, Steinman

, Hors

, Bracco

, Kozasa

, et al. Barriers and benefits in telemedicine arising between a high-technology hospital service provider and remote public healthcare units: A qualitative study in Brazil. Telemed J E Health, 2017; 23:527–532.

24.

Larburu

, Artetxe

, Escolar

, Lozano

, Kerexeta

. Artificial intelligence to prevent mobile heart failure patients decompensation in real time: Monitoring-based predictive model. Mob Inf Syst, 2018; 2018:1–11.

25.

Das

, Topalovic

, Janssens

. Artificial intelligence in diagnosis of obstructive lung disease: Current status and future potential. Curr Opin Pulm Med, 2018; 24:117–123.