AI support in prehospital telemedicine: Perspectives of tele-emergency physicians and ethical considerations

Section 1: Demographics and background information

The demographics and background information section consists of questions about the age, gender and the professional experiences of the participants as physicians, emergency physicians and tele-EMS physicians. The basic requirements for admission to specialist training as a tele-EMS physician in Germany include recognised specialist status in a field closely related to clinical and rescue service emergency and intensive care medicine, along with additional training in emergency medicine. ¹² Candidates must also provide proof of at least 2 years of regular and continuous activity as an emergency physician, but at least 500 independently completed emergency physician missions. ¹² These requirements assume that tele-EMS physicians should also have experience working as on-site emergency physicians. The questionnaire therefore included questions to determine the number of years of experience the participants had in the relevant areas.

Section 2: Working profile as a tele-EMS physician

The second part of the interview focused on the routine tasks and responsibilities of the tele-EMS physicians. Specific items were used to gather information on their professional activities, as well as the positive and negative aspects of their work. Participants were also asked whether they have access to contacts in the workplace who could provide professional advice if required, and to specify the nature of these contacts. The objective of this section was to provide a basic understanding of the role of tele-EMS physicians within emergency medical services, including the challenges faced by practitioners. Furthermore, it sought to identify areas for improvement and to gain insight into how practitioners seek an access professional support.

Section 3: Knowledge about AI

To ensure that all participants possessed at least a basic understanding of AI, a brief definition of the term was provided at the beginning of the interview: “AI is the term used to describe technologies that imitate the intelligent behaviour of humans. Similar to a human, an AI should learn with the help of data and experience to independently find solutions to problems in order to support its users as well as possible.” ¹³ After receiving this definition, participants had the opportunity to ask clarifying questions, which were addressed by the interviewer. Subsequently, they were asked to name AI systems they were familiar with, in order to facilitate access to the topic. If respondents were unable to name any systems, the interviewer provided concrete examples (e.g. voice assistants such as Alexa and Google Assistant, route suggestions such as Google Maps, or chatbots such as ChatGPT).

Section 4: Experience with AI

Participants’ prior experience with AI, both in their private and professional contexts, was also documented. They were asked to share their experiences with AI systems they had previously encountered, along with the aspects they found positive and negative. These questions served to assess the participants’ familiarity with AI and to identify their preferences and attitudes towards its use.

Section 5: Description of the KIT²-AI

The fifth section of the questionnaire included a description of the KIT²-AI system: “The aim of this system is to support tele-EMS physicians in their work by providing them with suggestions for suspected diagnoses, therapeutic measures, target clinics and other resources, such as emergency physicians, carrying assistance, police, firefighters, etc… The tele-EMS physician enters the available patient and medical history data into the system. The system makes suggestions based on this data, information from past emergency operations and local procedural instructions. The system's suggestions are sorted by probability. The tele-EMS physician has the option of approving or rejecting the suggestions made by the decision support system. Accordingly, the tele-EMS physician is responsible for the treatment and the role of the AI is exclusively supportive.” After the description of the system by the interviewer, participants were given the opportunity to ask clarifying questions to ensure their understanding of the system's purpose and functionality.

Section 6: Opinion on the KIT²-AI

Section 6 constituted the main part of the semi-structured interview and aimed to capture the opinions, needs and concerns of tele-EMS physicians regarding the previously described AI system. Participants were encouraged to spontaneously name advantages and disadvantages they associated with the system. Subsequently, they were asked to provide specific feedback on the individual functions of the AI, or the individual tasks for which the AI provides support—such as suggestions for suspected diagnoses, therapeutic measures, target clinics and other resources (Figure 1).

OPEN IN VIEWER

Figure 1.

Interview questions to gather the opinions of tele-EMS physicians on the individual functions of the KIT²-AI system, exemplified by the AI feature to suggest suspected diagnoses. Questions marked with an asterisk (*) could be omitted if the necessary information is already available. AI: artificial intelligence; EMS: emergency medical system.

In addition, more general questions were posed regarding the utilisation and implications of AI. The corresponding items, which do not inquire about particular system requirements or concerns related to its use, were based on the Model for Ethical Evaluation of Socio-Technical Arrangements (MEESTAR)^14,15 (Figure 2). Five MEESTAR dimensions for ethical evaluation—safety, autonomy, justice, participation and self-conception—were operationalised in questions about the KIT²-AI system (Figures 1 and 2).

OPEN IN VIEWER

Figure 2.

Interview questions to gather the opinions of tele-EMS physicians on the use and impact of AI. Questions marked with an asterisk (*) could be omitted if the necessary information is already available. Prompt questions in blue brackets were employed when responses to the preceding questions lacked sufficient detail. AI: artificial intelligence; EMS: emergency medical system.

The dimension of “safety” was operationalised in the questionnaire as the competence of the AI system—specifically, its ability to make correct predictions and to support physicians in delivering high-quality patient care. ¹⁵ The dimension of “autonomy” was conceptualised as the freedom of decision of the individual ¹⁶ or, more precisely, the freedom of decision of tele-EMS physicians during a rescue operation. The concept of “justice” was understood as fairness and the absence of disadvantage to the physician. ¹⁷ “Participation” was defined as the involvement of patients in decisions that affect their health. ¹⁸ The dimension of “self-conception” was interpreted as the perception of potential changes, including those in professional fields due to technological progress ¹⁹ or the implementation of AI.

Section 7: Closing questions

The final questions “Is there anything that has not come up in the interview so far that you would still like to share with me?” and “Now that we have discussed all these aspects, how do you feel about the AI? Could you imagine working with it?” provided participants the opportunity to address additional topics they considered important in the context of AI in tele-emergency medicine. The question of whether they could imagine working with the KIT²-AI system enables a concluding reflection on the potential use of AI in practice.

Interview process and evaluation

The qualitative interview study was conducted between October 2023 and February 2024 at the Fire and Rescue Service Department in Aachen, Germany, where the tele-EMS centre of Aachen is located. Fourteen tele-EMS physicians from Aachen were recruited via email by HS, a tele-EMS physician and medical director EMS, city of Aachen. All of the recruited physicians agreed to participate in the interview study. Due to sickness and shift changes, however, 11 interviews were conducted. The final sample size was deemed sufficient once data saturation was reached. Data saturation was reached when consecutive interviews no longer provided new information relevant to the research question. ²⁰ After the 11 planed interviews were conducted and yielded similar responses as well as recurring themes, it was determined that additional data would not contribute further insights. ²⁰ Therefore, participant recruitment was concluded at the point of thematic redundancy. All participants were interviewed during their working hours, while a tele-EMS physician involved in the project provided coverage at the tele-EMS centre. The interviews were conducted face-to-face in a quiet room from ND, a psychologist with experience of conducting interviews. She is employed as a research assistant at the RWTH Aachen University Hospital and not acquainted with the participants. Prior to the commencement of an interview, the interviewer introduced herself as a research assistant on the KIT² project at the Institute for History, Theory and Ethics of Medicine at Aachen University Hospital and provided a brief overview of the study and eliciting the participants’ personal opinion. No additional characteristics of the interviewer were mentioned. The mean duration of the interviews was 42 minutes (range: 28:42–54:59 minutes). Prior to the interview, each participant provided written consent for both the interview and its audio recording. Superiors or colleagues were not present during the interviews. The study was approved by the Ethics Commission of the Faculty of Medicine of the RWTH Aachen University (EK 23–256). A pilot interview with a tele-EMS physician (MI), who did not participate in further interviews, was conducted in order to assure the relevance and the quality of the questionnaire items.^21,22

After conducting the interviews, ND transcribed them verbatim, and anonymised them; the audio files were then deleted. A qualitative analysis according to Mayring ²³ was conducted. An initial inductive categorisation was carried out in QCAmap ²⁴ by ND. A secondary inductive categorisation was conducted with the help of ChatGPT 4.0 and specially developed prompts (Figure 3) based on extant research.^25–27 Both categorisations were then subjected to a comparative analysis, followed by a summary by ND. Discrepancies were resolved through discussion and consensus with SW, a PhD in bioethics experienced in qualitative research in the field of medicine, employed as a scientist at the RWTH Aachen University Hospital, thereby ensuring coding reliability and enhancing the credibility of the findings. ²⁸ The final categorisations were subsequently analysed and interpreted by ND and SW.

OPEN IN VIEWER

Figure 3.

Prompts used in ChatGPT to support the inductive categorisation of the interview transcripts, exemplified by the research question “What do tele-EMS physicians like about their job?”. EMS: emergency medical system.

Demographic characteristics of the sample

A total of 11 tele-EMS physicians, working in the city of Aachen, Germany, were interviewed. Five of them identified as male and six as female. Ten of those surveyed were also working as on-site emergency physicians at the time of the interview, eight of them additionally as anaesthetists, two in an intensive care unit (Table 1). One participant was working sole as a tele-EMS physician at the time of the interview due to parental leave.

OPEN IN VIEWER

Table 1.

Sample demographics (N = 11).

	M	SD	Range
Age in years	35.82	4.22	31–46
Years of experience as a physician	9.27	3.01	6–16
Years of experience as an emergency physician	6.00	3.81	2–14
Years of experience as a tele-EMS physician	3.90	2.91	1–10

EMS: emergency medical system.

Working as tele-EMS physician—advantages, disadvantages and contacts

In response to the question “What do you like about your job as a tele-EMS physician?” the study participants expressed their appreciation for the efficiency of the tele-EMS. They highlighted that this system enables them to treat emergency patients more quickly and to safe resources by avoiding the need for an emergency physician to visit all patients in person. They also enjoyed the tele-medical teamwork with the paramedics and the emergency physicians, who treat the patients on site, as well as the interaction with the control centre. The variety of emergency situations handled by tele-EMS physicians, the different rescue regions in which they operate, the opportunity to develop skills in prioritising tasks (especially in situations involving the simultaneous treatment of multiple patients) and effective communication in the tele-medical context were also identified as positive attributes of the role.

On the other hand, technical and organisational issues, particularly during inter-hospital patient transfers, were perceived as negative aspects of the tele-EMS physician job. Furthermore, one interviewee reported that in cases where patient's condition is so critical that tele-medical treatment is insufficient and an emergency physician is needed on site, it can be very unsatisfactory to have to hand over the patient and not be able to treat them personally. The office environment was perceived as both a benefit and a drawback. On the one hand, it was considered advantageous in the sense of a safe, warm and dry working environment. On the other hand, it was viewed as disadvantageous, as one typically works alone in the office and remains in the same environment throughout the entire day.

When asked whether they have any contacts they could consult for professional advice during a rescue operation, participants gave mixed responses. Tele-EMS physicians approach various contact persons, with management staff generally available to answer questions at any time by telephone or smartphone using instant messaging and voice-over services (such as WhatsApp Messenger). It should be noted, however, that these contact options are not structurally embedded. Managers and colleagues are contacted infrequently, and primarily in cases of uncertainty regarding inter-hospital patient transfers. However, as tele-EMS physicians gain experience, they tend to require less support with logistical issues. During shift handovers, there is the opportunity to discuss previous cases with colleagues. External contacts—such as clinics, the poison control centre or technical support—are rarely used and usually only for specific issues. Participants stated that for them contacting someone during a rescue operation is not a practicable option, as emergency medicine requires time-critical action, and external support structures are therefore not foreseen.

Private and professional experience with AI

Eight out of 11 interviewees stated that they use AI systems in their personal lives; two revealed that they only experimented with AI, and one admitted to never having used AI. In the context of their professional lives as physicians, six participants indicated that they currently use or have previously used AI, while five reported never having done so. However, the interviews revealed that participants had divergent conceptions of what AI is. Some associated AI specifically with chatbots or voice assistants that enable human-like communication, while others linked it more closely to features such as fingerprint sensors or facial recognition software, automatically activated on smartphones, for example. Moreover, participants were unaware whether the systems available to them in their workplace are AI-based, making it difficult to determine the extent of their experience with medical AI. Accordingly, the results mirror the general beliefs of tele-EMS physicians regarding the use of AI in their professional field rather than their actual use of and experience with this technology. In both their personal and professional lives, interviewees particularly valued the time savings offered by AI, its ability to take on tasks and simplify workflows, the extensive knowledge base of AI systems, and the fact that in some cases they can respond more effectively and quickly than humans. However, concerns were raised when AI systems generate false or discriminatory information, and when users lack insight into how these systems function or what happens to their own or patient's data. Participants were also concerned about the absence of clear regulations and guidelines for AI usage, along with the potential for external manipulation of AI systems.

What advantages and disadvantages do tele-EMS physicians believe the use of the KIT²-AI could have for their work?

Based on the description of the KIT²-AI, participants believed that it could be beneficial for them in various ways. For example, the system may help avoid errors in diagnosis, medication, and treatment while also increasing efficiency. With the support of its database, physicians could identify a suitable target hospital more quickly—especially in unfamiliar regions—and receive assistance in managing rare or unusual cases. Another perceived advantage is that the AI system performs consistently, regardless of the user's mood, stress level or physical condition, and can also help maintain an overview during simultaneous rescue operations (Table 2).

OPEN IN VIEWER

Table 2.

Advantages and disadvantages of the KIT²-AI, identified during interviews with tele-EMS physicians from Aachen, Germany (October 2023–February 2024).

Theme	Subtheme	Supporting quote (Participant ID)
Advantages of the KIT2-AI	Avoidance of errors	“Well, as a human being you often make a fixation error: an ambulance [worker] calls and says, ‘We have a patient with a renal colic’ and in the end it's something else, but you immediately stick to what is suggested to you. I believe that the system is much better in this because it approaches the matter neutrally (…) and doesn’t allow itself to be guided by such human aspects, I would say.” (P157)
	Increased efficiency and time saving	“I think it [the KIT2-AI use] will shorten the duration of rescue operations, which means that you will be faster, and I also think that you will always choose the right dosage of medication for patient's body weight, because that is something that the system can easily calculate.” (P259)
	Correct medication dosage
	AI works always equally well	“And of course, the system is always equally good. So, if I wake up at 3 a.m. and listen to a quick report while I’m asleep and then decide something, I’m not as good as if I’m sitting there in the morning after my first coffee, and I’m fit, let's put it that way. And I think the system is helpful when there are a lot of rescue operations going on at the same time. If you start to mix things up and don’t have that much time for the individual case, then the system is of course equally meticulous and can help you a lot, I would say.” (P157)
	Helpful for parallel rescue operations
	Suggestions for target clinics in unfamiliar areas	“Especially in regions in which I am not so familiar with the local area, the option of using a suggestion to find a target clinic that matches the right medical department (…) is a huge advantage.” (P216)
	Support in unfamiliar cases	“Perhaps in areas of knowledge that you might not be very familiar with, you could be reminded to ‘look here, look there’ and so on. I think that would be really good.” (P389)
Disadvantages of the KIT2-AI	Inaccurate recommendations	“Disadvantages … depends, of course, on how good the proposals are. (…) In other words, I think it [the KIT2-AI system] will only be accepted or used well if the probability that what comes out of it is real. If it constantly presents any special cases or produces nonsense, then I don't think it will be used so well. I think that's a problem. And then, of course, it has to work quickly (…), if this is accompanied by a time delay or a waiting time in general, I don’t think that's good for acceptance either.” (P157)
	Time delay
	Devaluation of the work of the tele-EMS physician	“Yes, well, the danger would probably be that some people would say: If you just have AI, then do you even need a human sitting there making the decision or (…) as a source of knowledge when the AI actually already figures out everything on its own?” (P389)
	The AI makes too many recommendations	“Depending on how many recommendations you get, you may be overwhelmed by the amount that pops up, and you have to press it away again.” (P216)
	Discrimination/bias	“The fact that the [AI] system in Germany, for example, will mainly be fed with [data from] white, Caucasian people and that other factors will not be well taken into account. Therefore, I think that there will be a bias in the data. I don’t necessarily believe that it will help to provide better care for the patient as an individual, but it will help to provide fast care for the wider society and quickly get them to the right clinic.” (P259)

AI: artificial intelligence; EMS: emergency medical system.

At the same time, participants expressed that the AI system could generate inaccurate or discriminatory recommendations, potentially endangering patients if these are not carefully verified. If the system tends to offer too many suggestions (e.g. multiple suspected diagnoses), it could negatively impact physicians’ work or the rescue operation. Participants also indicated that an overload of information from the AI may distract them and cause treatment delays. Additionally, interviewees were concerned that the use of AI in tele-EMS could negatively affect the image of tele-EMS physicians and diminish the perceived value of their work and expertise (Table 2).

How do tele-EMS physicians rate the competence of the KIT²-AI?

In general, participants believed that the KIT²-AI would be capable of making accurate suggestions regarding suspected diagnoses, treatments and medication. Most participants also indicated that the AI would likely propose suitable target clinics; however, one interviewee pointed out challenges in this regard, noting that clinics and hospitals do not always offer all the treatments they should in theory. Opinions also differed concerning the AI's ability to recommend additional resources during a rescue operation, such as on-site emergency physician, police or additional carrying assistance. Some of the tele-EMS physicians believed that, with proper training, the AI would be able to correctly identify such needs. Others, however, were more sceptical, emphasising that the decision to request additional resources during a rescue operation is typically individual and sometimes subjective (Table 3).

OPEN IN VIEWER

Table 3.

Perceived competence of the KIT²-AI, identified during interviews with tele-EMS physicians from Aachen, Germany (October 2023–February 2024).

Theme	Subtheme	Supporting quote (Participant ID)
Perceived competence of the KIT2-AI	Correct suggestions for suspected diagnoses	“I do believe that this is possible [for the AI to make correct suspected diagnoses] if you provide the relevant data and then, for example, have labelled training data, and you say ‘OK, we now have many, many cases in which the patients have shown the following vital parameters, the following symptoms etc.’, and then have the corresponding verified diagnoses, then I do believe that you can create and train a system, that can make correct diagnoses.” (P843)
	Correct treatment suggestions	“If the suggested diagnosis is good – yes, [the treatment recommendation will be correct,] because it is pretty standardised. We have procedural instructions for that. I think it [the KIT2-AI system] is also good in this. It's also clearly defined: If the blood pressure is above that, give this. If not, then no. Technically, it's actually relatively simple, but of course it depends on the suspected diagnosis. If that's correct, I think the treatment instructions will be good.” (P157)
	Correct suggestions for target clinics	“Yes, it [the KIT2-AI] would [make correct suggestions for target clinics], but that is usually a very obvious decision. So, it's not something you have to think about for a long time.” (P775)
	The KIT2-AI has no knowledge of the quality of the target clinics	“All of us who work on the street [as emergency physicians] have experienced that a certain hospital is suitable for something on paper, but not in real life (…). A hospital that is connected to the trauma network can theoretically treat seriously injured patients on paper, but I know that (…) the trauma room is used as a storage room for cleaning buckets, and they probably saw their last seriously injured patient five years ago by chance. I know that, but I don’t know if the AI knows that.” (P548)
	Further resources (emergency physician, police, carrying assistance)	“Yes, I could imagine [that the AI makes correct suggestions for further resources] if it is fed with this information, of course. I think that … yes, I can imagine that it would work well using an AI, but of course you would have to make sure that it [the AI] gets this information beforehand – when do I need police, when do I need carrying assistance or something else. But then it would probably be easy, yes.” (P674) “I think, carrying assistance, if I enter this somewhere … I can enter the weight, maybe it [the KIT2-AI] will suggest it. But then of course the system doesn’t know which floor we’re on and everything around it. I think that's difficult because it varies greatly – sometimes I request a stretcher for a person of normal weight on the first floor because the staircase is a total pain, or they’re in a lot of pain and I want to bring them down with a turntable ladder. But sometimes they are on the ground floor at ground level and are easily accessible and then even 120-130 kilograms can be transported by the ambulance alone, depending on whether there are two or three of them, I would say. I think that makes it difficult.” (P157)

AI: artificial intelligence; EMS: emergency medical system.

Possible impact on physicians’ working habits and medical knowledge

Participants believed that the KIT²-AI would positively impact their working habits, particularly in making diagnoses, choosing treatment procedures, and identifying the most appropriate target clinic for the patient. They anticipated that using the AI would encourage them to engage in more reflective thinking when considering its suggestions and would also help them overlook fewer steps, for example when following procedural instructions or conducting specific diagnostics. The tele-EMS physicians, however, did not believe that the AI would influence the way they order additional resources, such as an on-site emergency physician or police, as these decisions are complex and usually made in consultation with paramedics on site. One interviewee stated that the use of the KIT²-AI could reduce the perceived effort in performing their tasks and contribute to a sense of devaluation of their profession (Table 4).

OPEN IN VIEWER

Table 4.

Possible impact of the KIT²-AI on users’ working habits and medical knowledge, identified during interviews with tele-EMS physicians from Aachen, Germany (October 2023–February 2024).

Theme	Subtheme	Supporting quote (Participant ID)
Impact of the KIT2-AI on the working habits of tele-EMS physicians	Impact on making suspected diagnoses	“Perhaps the AI could help me to have a nudge in the right direction, or it could say: Before you diagnose X, why don’t you check parameter Y to see if it matches your suspected diagnosis?” (P548)
	Impact on the prescription of treatment measures	“I believe that the AI brings you closer to the procedural instructions or the guidelines because it stubbornly works them through.” (P157)
	Impact on the selection of a target clinic	“When someone calls from Frankfurt, I don’t know any hospitals there, I have no idea which are capable of what. I think that it is an extreme relief to quickly find the nearest suitable hospital with very little work.” (P157)
	Impact on the request for further resources (emergency physician, police, carrying assistance)	“No, [I don’t think that the AI's suggestions for additional resources would change the way I do it myself], because I usually ask the staff [on site] first, ‘What is it like? What does it look like on site?’. And whether I then enter in the AI ‘Here's what it looks like’, or whether I pick up the phone myself and say to the control centre ‘Here, they have a turnaround staircase, they can’t come down. They told me, they have to use the window.’ (…) [The AI] has little influence on me.” (P216)
	The AI suggestions provoke thought	“I believe that the first [working] steps [of a rescue operation] will probably be the same and then as soon as I have the proposal [from the AI], I have to think about it again, consider whether I think it makes sense and can then either accept or reject it.” (P843)
	The AI use ensures that less gets forgotten	“It could contribute to some things that are often forgotten so that this is no longer the case. (…) Simply because the AI thinks along with you, I’d say less gets forgotten.” (P216)
	Perceived devaluation of the own work	“I realise that whenever you have something that somehow reduces your work and thinking, then you also stop. Somehow the feeling of the value of your own work is reduced, because –I think – I’m sitting at an office desk, that's not … of course it's a physician's task, but it's no longer primarily a physician's task. And if I then also have a computer system where I just click through, that is, I only do (…) a desk job – I transfer patient data into a form and at the end a computer programme tells me what I should do with the patients. But I’m still liable for it, I think now … so … I think it would lead to a perceived devaluation of my work.” (P259)
Impact of the KIT2-AI on the medical knowledge of tele-EMS physicians	Positive impact on medical knowledge	“I believe that this [the KIT2-AI] would foster guideline adherence and make you aware of treatment standards a bit more.” (P775)
	Negative impact on medical knowledge	“If it [the KIT2-AI] works well, I could imagine that at some point one will start to rely on it and perhaps become less, I don’t want to say careful, but … that you will tend to rest a little on the fact that the AI takes care of everything, and perhaps you will look up things less often when you are unsure or yes … that you will then let the AI take the lead and when you realise that it works accurately, you will (…) become a little lazy when it comes to your own professional knowledge or your own further training. I could imagine that.” (P674)
	No (major) impact on medical knowledge	“I don't believe that it [the KIT2-AI] would affect my knowledge in any positive or negative way.” (P259)

AI: artificial intelligence; EMS: emergency medical system.

Regarding the influence of the KIT²-AI use on their medical knowledge, participants expressed mixed opinions. Some believed that the AI would have a positive impact, enabling them to learn new things from its suggestions. Others were of the opposite opinion, expressing concern that physicians might lose knowledge if they rely too much on the suggestions provided by the KIT²-AI and follow them without making efforts to verify their accuracy. Other participants believed that the AI would not have any impact on their medical knowledge (Table 4).

Informing emergency patients about the use of an AI system

The most common opinion regarding whether patients should be informed about the use of AI was negative. Physicians provided various reasons for this—ranging from a lack of time during emergency situations and the practical impossibility of providing information about AI during rescue operations, to the belief that patients would not notice the AI usage and therefore would not be affected by whether the tele-EMS physician works with or without AI. Some also argued that informing patients about it is irrelevant as they were receiving appropriate care. Several participants expressed concerns that patients might not understand information regarding the physician's use of AI, or that such information could confuse or frighten them. As long as the AI suggests appropriate treatment and the tele-EMS physician makes the final decision, there was seen to be no necessity to inform patients about the AI. While some believed that informing patients about the use of AI was only a data protection issue, others were convinced that from a legal perspective, patients should be informed that the physician is supported by AI (Table 5).

OPEN IN VIEWER

Table 5.

Opinions on the need to inform patients about the use of an AI system, identified during interviews with tele-EMS physicians from Aachen, Germany (October 2023–February 2024).

Theme	Subtheme	Supporting quote (Participant ID)
Informing patients about the use of an AI system	Patients should not be informed about the use of an AI system	“I believe that in cases of emergency it is completely secondary [whether the patient is informed about the use of AI]. As long as you use systems in a way that is profitable for patients, whether it is the tele-EMS physician as a person or an AI that supports this tele-EMS physician, it is totally secondary whether the patients are informed or agree or similarly, especially given how many of the patients (…) understand this at all (…). It is not practicable and a bit unrealistic. Most people who really need emergency help, who even need an ambulance and perhaps a tele-EMS consultation, have completely different problems than thinking about whether someone will be connected by phone or whether a digital AI system will support them. Secondary.” (P843) “I think this could create prejudice or raise concerns for many people. Since, in the end, the tele-EMS physician still has sole decision-making authority, or let's say is solely responsible, I’m not sure – I don’t think it's absolutely necessary [to inform patients about the use of AI].” (P974) “No, [patients should] not separately [be informed about the use of AI]. (…) I mean, if I’m not being supported by AI but just quickly googling ECG images, I’m not informing patients that I’m checking Google to see what might be wrong with them either. They’re being medically consulted – that's what they should be informed about, and what the physician does with that is up to them.” (P775)
	Patients should be informed about the use of an AI system	“Hm, that's a good question. So, legally speaking, it's probably the case that you currently have to [inform patients about the use of AI]. Practically speaking, I find it quite difficult. Yes, every patient who is treated by us is asked whether it's OK. By doing so, they have given their consent.” (P216)
	Informing patients is only a question of data protection	“Well that's a difficult question. If the data processing does not take place someplace in the middle of nowhere, but instead the data is collected, processed and then deleted again or only remains in Germany, I think that is only a data protection issue, if at all, but otherwise I would say it doesn’t matter.” (P157)

AI: artificial intelligence; EMS: emergency medical system.

Willingness and requirements to use the KIT²-AI

Five of the 11 interviewed tele-EMS physicians could imagine working with the KIT²-AI in the future, while another five would prefer to trial the system first to evaluate its functionality before deciding whether to utilise it, depending on their experiences. Only one participant could not envision working with the system, despite acknowledging its potential benefits: “In fact, I can’t imagine that [working with the KIT²-AI]. (…) I’m sceptical about it, but I know that it can certainly provide valuable support” (P548).

For the interviewees, it was crucial that the diagnoses proposed by the AI are logical and applicable to the specific patient. The same applied to the treatment suggestions—they should be appropriate and consistent with the suspected diagnosis. It was considered important that the recommended treatment measures be adapted to the respective emergency medical service district: proposed medications must be available, treatment procedures must be authorised for paramedics, and paramedics must actually be capable of performing them. The recommended target hospital should be open and have available capacity in a department suitable for the patient's needs. Any additional resources suggested by the AI should be reasonable, and agreed upon by the paramedics on site.

Safety

The dimension of safety—understood as patient safety during a rescue operation—is linked to the ethical principle of non-maleficence, which emphasises avoiding harm to patients during medical interventions, ²⁹ or in the case of the KIT²-AI, during a rescue operation. As most interviewees believed that the AI system would be capable of making accurate suggestions regarding diagnosis, treatment and target clinic, its competence was perceived as high and therefore beneficial for emergency patients’ safety. The perception that suggestions made by the KIT²-AI could reduce errors, such as fixation errors or incorrect medication dosage, is a significant aspect of the principle of non-maleficence. Furthermore, both perceived safety and non-maleficence of the system—based on participants’ perspectives, regardless of whether such safety objectively exists—are essential for its acceptance and successful integration into medical practice. ¹⁵ Our findings on safety align with existing research, in which physicians consider AI beneficial for diagnostics,^30–34 medication dosage^32,35,36 and patient care^34,37 with the potential to reduce treatment errors.^36,37

Autonomy

In the context of this study, autonomy is understood as the freedom of decision of the individual, ¹⁶ or more precisely of tele-EMS physicians and their right to make responsible decisions regarding their work. ³⁸ Participants had different perceptions on how the KIT²-AI might influence their working habits and medical knowledge. While most expressed that using the system could have a positive or no impact on their working habits and knowledge, some were concerned that overreliance on the AI might lead to loss of skills and expertise. In the literature, this phenomenon is often referred to as “deskilling” ³⁹ and has been recognised by physicians in other studies as a significant issue in implementing medical AI.^30,32 Loss of practical experience, coupled with a decline in active participation in decision-making processes, can gradually erode existing competencies, which in turn undermines meaningful professional autonomy. ⁴⁰ To prevent dependencies on the AI system and inability to act during a potential system breakdown caused by deskilling, new technologies should only be employed if it is ensured that users continue to regularly practice their skills. ³⁹ In the case of the KIT²-AI, this could mean that tele-EMS physicians should continue to work as emergency physicians on site.

Justice

In the context of this study, the dimension of justice is also related to the physicians, suggesting that the implemented AI system should be fair to them by easing their workload rather than adding to it.^17,41 Participants believed that, if well-designed, the AI system has the potential to enhance their work efficiency, save time, and support them during simultaneous rescue operations or in unfamiliar rescue areas. However, concerns were raised that the system might produce misleading or excessive suggestions, potentially leading to confusion and delays. Similar perceptions have been reported in other studies, with physicians recognising that medical AI could ease their work ³⁰ and improve time^32,42 and cost ³² efficiency, while also expressing concerns that AI implementation in medicine may cause more workload for its users. ⁴³ To ensure AI fairness towards physicians, it is crucial to focus on user-friendly design,^31,44 compatibility with existing systems, ³¹ and to ensure that AI adds value to their work ³⁶ by involving them early in the development process.^39,44,45

Participation

The dimension of participation is important to patients, as it involves their engagement in their own health and the decisions about their treatment. ¹⁸ However, most participants felt that patients should not be informed about AI use in tele-EMS, perceiving that emergency situations allow limited time for providing information about AI, and saving a person's life should be the highest priority regardless of AI involvement. Yet, the deployment of AI in decision making contributes to a more complex situation. If patients are unaware of the contribution of an algorithm to the decision-making process, they are unable to meaningfully engage with or contest that process. ⁴⁶ This conflicts with patients’ right to receive all essential information about their treatment and to make an informed decision regarding consent. ⁴⁷ Emergency medicine, on the other hand, often involves decisions limited by time and available information, ⁴⁸ with emergency physicians frequently acting according to the patient's presumed will, for example, when the patient is unconscious or non-responsive. ⁴⁹ In most cases, this presumed will is likely aligned with saving the patient's life or performing medically indicated treatments at that moment. ⁵⁰ This creates an ethical tension in acute settings, as the urgency of treatment can limit opportunities for full informed consent. ⁵¹

Although it can be challenging to inform patients about the use of AI during an emergency operation, transparency regarding its use remains important, ³⁹ because it fosters accountability, enables patients to better understand the decision-making process, and burst trust in AI supported medical care. ⁵² As do efforts to keep the public adequately informed and to provide easy access to information for those who are interested.

Self-conception

Self-conception in the context of this study relates to the perception of potential changes in the field of the prehospital tele-EMS ¹⁹ and the willingness of the participants to incorporate AI into their work. ²⁹ Despite certain concerns, the interviewees were generally receptive to implementing AI in tele-EMS, believing that if integrated effectively, it could enhance patient treatment and ease their work. This aligns with other research where physicians recognise both benefits and risks associated with AI use,^33,36,43,53 yet maintaining a generally optimistic outlook,^43,45 viewing AI as a potentially useful clinical tool ⁵³ with the potential to positively transform (emergency) medicine.^44,54 Both our study participants and existing research stress that AI should not replace physicians but rather support them.^30,34 AI is anticipated to change the way they work, ⁴⁴ necessitating education among medical professionals on its utilisation.^34,37,44,45 The “future of medicine will depend on the knowledge and the use of tools based on AI,” ⁵⁴ underscoring the importance of managing it consciously and responsibly.

These findings imply that medical decision support systems must emphasise decision support features rather than fully automated recommendations, thereby preserving physician agency and supporting integration into existing workflows. ⁵⁵ Creators should prioritise transparency in algorithmic reasoning and enable the physician to customise or override suggestions, ensuring that the system complements rather than supplants their judgement. ⁷

For ethical education among tele-EMS physicians in this context, curricula should address the shifting role of AI in decision-making, emphasise physicians’ responsibility for interpreting and validating AI output, and build competence in digital literacy and bias mitigation. ⁵⁶