Evaluation of a Rapid Implementation of Telemedicine for Delivery of Obstetric Care During the COVID-19 Pandemic

Abstract

Introduction:

The aim of this evaluation was to assess the rapid implementation of obstetric telemedicine during the COVID-19 pandemic using the Consolidated Framework in Implementation Research (CFIR) evaluation framework.

Study Design:

Following 1 month of telemedicine implementation, obstetric providers at the University of South Florida clinic completed qualitative surveys and in-depth interviews about the implementation of obstetric telemedicine in the clinic guided by the CFIR evaluation framework.

Results:

Overall, providers considered obstetric telemedicine comparable to traditional in-person clinic visits and acknowledged that they were adequately prepared for the telemedicine implementation. They perceived that obstetric telemedicine mostly met the needs of patients in terms of convenience and comfort of visits, decreased exposure to COVID-19 infection, and the ability of the patient to listen to fetal heart sounds if at-home doppler monitoring was available.

Conclusions:

The implementation of the obstetric telemedicine care model was deemed a favorable alternative option for patients during the COVID-19 pandemic.

Introduction

The 2019 novel coronavirus (COVID-19), originating in Wuhan, China, subsequently became a global pandemic infecting over 250 million people and resulting in over 5 million deaths to date.¹ In the United States, >50 million people have tested positive for COVID-19 and over 700,000 deaths have occurred.² In the initial stages of the COVID-19 pandemic, lockdowns, quarantines, and social distancing measures were implemented because of the high transmissibility of the COVID-19 virus, shortage of personal protective equipment, and the absence of curative treatments or a vaccine.

As a result, these measures impeded the provision and participation of traditional in-person health care appointments. For obstetric patients, the need to limit infection exposure, reduced access to childcare because of school closures, and restrictions to working from home were barriers likely to prevent access to in-person obstetric care. Likewise, in response to the COVID-19 pandemic, preventive social measures were likely to impact health-related policies and practices thereby limiting the delivery of comprehensive obstetric care services.³ To support the continuity of health care services during the COVID-19 pandemic while adhering to pandemic restrictions, there was an increase in the adoption and utilization of telemedicine for the delivery of health care services.

Telemedicine involves two-way audio and visual electronic communication between patients and providers in real-time at different remote locations.⁴ The American Hospital Association reported that in 2017, 76% of U.S. hospitals connected patients remotely with health care providers using video or other technology.⁵ Research has also shown that before the COVID-19 pandemic, Radiology specialists had the highest utilization rates of telemedicine (39.5%), while Obstetrician-Gynecologist specialists had among the lowest utilization rates of telemedicine (9.3%) in their practices.⁶

Telemedicine has been shown to be advantageous for both patients and health care providers, due to its ease of accessibility to attend office visits from different geographical locations, reduced expenses related to travel costs and childcare costs, increased time efficiency for consultations, and limited exposure to infectious diseases.⁷

In addition, obstetric telemedicine has been found to have similar safety and effectiveness to in-person care,⁸ and no differences have been found in the rates of health outcomes such as gestational age at birth, preterm births, cesareans, birthweight, and neonatal intensive care admissions when comparing obstetric telemedicine visits to in-person visits.⁹

Although most studies have assessed the implementation of telemedicine in various clinical settings without the guidance of theory,^10

–16 we aimed to assess the rapid implementation, feasibility, and satisfaction of obstetric telemedicine during the COVID-19 pandemic using the Consolidated Framework in Implementation Research (CFIR) evaluation framework.¹⁷

Subjects and Methods

OVERVIEW

This qualitative evaluation was developed using CFIR, which provides constructs that have been associated with effective implementation (Table 1).¹⁸ Key CFIR constructs guiding this evaluation included Intervention Characteristics (Evidence Strength and Quality, Relative Advantage, Adaptability, and Complexity), Outer Setting (Patient Needs and Resources), Inner Setting (Implementation Climate and Readiness for Implementation), Characteristics of Individuals (Knowledge and Beliefs about the Intervention), and Processes (Engaging Intervention Participants) related to the implementation of obstetric telemedicine.

Table 1.

Consolidated Framework for Implementation Research Constructs Used to Assess Implementation of Obstetric Telemedicine Visits Guided by Consolidated Framework for Implementation Research

DOMAIN	CONSTRUCT	DEFINITION
Intervention Characteristics	Evidence Strength and Quality	Perception about the quality and validity of evidence supporting the belief the intervention will have desired outcomes
	Relative Advantage	Perception of the advantage of implementing the intervention compared to other programs
	Adaptability	Degree to which intervention can be changed to meet local needs
	Complexity	Perception about the difficulty of implementing the intervention
Outer Setting	Patient Needs and Resources	Extent to which the needs of patients are known and prioritized
Inner Setting	Implementation Climate	Shared receptivity of individuals to an intervention
	Readiness for implementation	Actual organizational commitment to implement the intervention
	Available Resources	Level of resources allocated for implementation and continued use of intervention
	Access to Knowledge and Information	Access and use of knowledge and information about the intervention
Characteristics of Individuals	Knowledge and Beliefs about the Intervention	Individual's attitudes and values toward the intervention
Process	Engaging	Encouragement of individuals in the implementation and use of the intervention

These CFIR items¹⁸ were specifically intended for this study and constructs are listed in the order they appear in the text.

CFIR, Consolidated Framework in Implementation Research.

DATA COLLECTION

We recruited obstetric providers delivering obstetric care visits via telemedicine at the University of South Florida clinic from May to November 2020. Purposive sampling was used to recruit obstetric providers via email who were provided with a link to complete an anonymous online survey. Eligibility criteria included health care providers that currently provided obstetric care at University of South Florida clinic. Nineteen participants completed the online survey out of 31 total prenatal care providers in the practice (Table 2).

Table 2.

Demographic Characteristics of Obstetric Providers Implementing Telemedicine

	SURVEY QUESTIONNAIRE ( n = 19)	INTERVIEW ( n = 13)
Years in practice	N	n
0–5	7 (37%)	5 (38%)
6–10	0 (0%)	2 (15%)
11–15	1 (5%)	1 (8%)
16–20	3 (16%)	3 (23%)
21–25	3 (16%)	0 (0%)
26–30	3 (16%)	1 (8%)
30+	2 (11%)	0 (0%)
Age/years
20–30	0 (0%)	0 (0%)
31–40	6 (32%)	6 (46%)
41–50	5 (26%)	5 (38%)
51–60	5 (26%)	1 (8%)
61–70	3 (16%)	1 (8%)
70+	0 (0%)	0 (0%)
Race/ethnicity
White	14 (74%)	10 (77%)
Black or African American	2 (11%)	1 (8%)
Hispanic	2 (11%)	1 (8%)
Asian	1 (5%)	1 (8%)
American Indian or Alaskan Native	0 (0%)	0 (0%)

The anonymous online Qualtrics survey included a demographic questionnaire on providers' age, race, and years in practice postresidency or fellowship training, and open-ended questions guided by key CFIR constructs.¹⁹ We selected eight key CFIR constructs most relevant to implementation of obstetric telemedicine. Some open-ended survey questions included: “What advantages do telemedicine visits have compared to existing programs/in-patient visits?” (Intervention Characteristics); “What barriers do you believe obstetric patients face in participating in telemedicine visits?” (Outer Setting); “What was the general level of receptivity in your clinic to implementing telemedicine visits?” (Inner Setting); “What steps were taken to encourage patients to commit to using telemedicine visits?” (Process) (full survey available on request).

Following the completion of surveys, we invited providers meeting the inclusion criteria to participate in semistructured interviews to ascertain more in-depth feedback regarding their experience in implementation of obstetric telemedicine. A total of 13 providers, some of whom previously completed the anonymous online survey, participated in the in-depth interviews. These interviews consisted of questions identical to those in the online survey, with opportunity to expand upon answers. Interviews lasted ∼20 to 30 min and were audio recorded and transcribed for data analysis. To preserve confidentiality, each transcript was assigned a pseudonym. This study was reviewed and determined to be exempt as program evaluation by the University of South Florida Institutional Review Board.

DATA ANALYSIS

We developed an initial codebook based on a priori structural codes guided by CFIR. This codebook was revised for clarity and the relevance of codes and definitions. Two evaluation team members independently coded four transcripts to consensus (C.R. and K.F.), and a kappa statistic of 0.8 was calculated, indicating good agreement between coders.²⁰ Any coding discrepancies were identified and discussed to achieve agreement on the final codebook. One coder (C.R.) coded the remaining surveys and transcripts. A thematic analysis approach was used to analyze the data²¹ and all coding and data analysis was conducted using MAXQDA 2020 software.²² Recruitment of providers was ended once it was determined that data saturation was achieved; participant comments were repeated from previous surveys/interviews and no new information was being added.

In addition, trustworthiness was enhanced by conducting peer debriefing (credibility), having two evaluation team members independently code at least 10% of the interviews with a good kappa (reliability), using provider quotes to represent themes (confirmability), and by having audit trails throughout the process.^23,24

Results

The evaluation results from survey and interview participants (Table 2) are summarized below along with representative quotes found in Table 3 (quote ID noted in parentheses). These findings are aligned with CFIR constructs that help to inform the pros and cons of obstetric telemedicine implementation from the viewpoints of a diverse range of providers who are newly conducting this practice. Doximity and Microsoft Teams programs on HIPAA-secure computer devices were used to conduct telemedicine consultations.

Table 3.

Participant Quotes

CONSTRUCT AND INTERVIEW QUESTION(S)	ID	QUOTES
Quality Do you have any concerns about the safety and efficacy of telemedicine visits?	IC1	“On the safety perspective, the only thing I would say is that I think it's really increasingly important on telemedicine that because we are not able to visualize the patient, that we give them precautions—so, for example, if someone's coming in with abdominal pain and we're not able to fully assess it, that we want to make sure we have close follow-up, so that these patients aren't lost to follow up and two, that we give them precautions of this, “If your pain is not getting better or you experience X, Y, and Z, that you return for care,”…I do get a little worried, especially if we're prescribing other treatment or working up a problem that we haven't fully assessed, that the patient may not make or be lost a follow up.”—T1 (0–5 years)
Adaptability What changes allowed telehealth visits to work effectively in your clinic?	IC2	“Well, obviously the technology portion, getting all the technology arranged so that we could do that. Workflow in terms of having the staffing available to pre-triage with the patient and get them prepared for the telemedicine, and on the back-end our nursing staff being able to then help distribute whatever orders we have to give, so if lab needs to be done or prescriptions need to be mailed and that sort of thing, after the fact.”—T11 (16–20 years)
Relative Advantage How does obstetric telehealth compare to in-patient visits in your setting?	IC3	“So, decreased risk of COVID exposure for both the patient and the provider. Improved efficiency of time because the patient does not have to travel to the clinic so it's faster. There's less distraction to the provider. So, I'm not in my office, I'm not being pulled to answer a bunch of patient or questions about other patients. I can really just focus on the patient. Then I typically do telemedicine from home. So, I didn't have to travel, so the decreased risk of exposure for me.”—T1 (0–5 years)
	IC4	“I guess just the fact that they're in their home and they could even share—that's an even more personal connection, like they can literally move their phone around and say, “Oh, this is the baby's room,” and just kind of give you that even more personal connection, thinking about bringing the baby home to this space, and being able to connect in that way.”—T7 (16–20 years) I found some tremendous gains because the patient is comfortable in their own homes. Any of the anxieties that they might have coming to see a doctor was dramatically reduced. I felt a different sense of connection with a patient because of their comfort level, I think that they were able to speak more clearly or addressing things they have on their mind with more ease. What's also is interesting is…you could see living conditions or maybe they could have safe living conditions or adequate food or these sorts of things, so you get a window into their life a little bit more to see how their living environment might be a little bit, you know? In some regard, that's helpful…I felt like they felt very comfortable in their ability to that type of conversation with the doctor, more so than they do in the office setting. It just felt more intimate...because it's just the patient and the doctor that there's not any other staff around or the noise from the hallway or coming from a waiting room setting or you don't have privacy. I feel like they felt a sense of intimacy and privacy.”—T11 (16–20 years)
ComplexityHow complicated are telehealth visits?	IC5	“It's maybe slightly more complicated than an in-person visit only in that you're doing all of the data entry in terms of vital signs, reviewing their medications, reviewing their history and a lot of times, that's done by a medical assistant or a nurse in the office. So, you're more responsible for the entirety of the visit and documenting more things than we would in-person. So I would say the complexity is just slightly more complex only in that there's more work involved.”—T13 (16–20 years)
Patient Needs and ResourcesHow well did telemedicine visits meet the needs of obstetric patients? In what ways did telemedicine visits meet their needs? What barriers do you believe obstetric patients face in participating in telemedicine visits?	OS1	“Definitely travel time is a major expense, but I think that probably the moment that I felt like telemedicine was really the most useful was there were some postpartum patients that were depressed and they were really having a hard time, and I just don't know how they would have left the house. First of all, especially given the pandemic, they wouldn't have been able to bring their newborn in with them. So, they would have to leave their newborn at home to come and talk to us. So, telemedicine provided an avenue that they could talk to us and, also, we could get visitors like their spouses, their mothers and their parents to chime in and say, “I'm really worried about the person and I think they need help.” That would not have been possible given our current model where we weren't allowing visitors, and people were having to drive in and park and all that. So, yes, reduced times, the ability to have spouses and loved ones be a part of the telemedicine experience than in the actual visits if the patient wanted that.”—T2 (11–15 years)
	OS2	“Some of them, they thought their insurance would pay for it. Their insurance doesn't pay for it and they ended up saying, “Well, I'm not going to buy it myself.”—T8 (31+ years) “Well, it is OB so more patients are still comfortable seeing the provider and being able to hear their baby's heartbeat, and that they can't do. Technically, if they want to, they would have to buy a monitor, a Doppler, to be able to listen to the baby, and we would be, on the other side, telling them, ‘Yes. That's your baby.’ Then the other thing is being prenatal and OB, we do a lot of in-patient where we need to draw labs, do ultrasound. There are limitations of when we can do with telemedicine.”—T12 (6–10 years)
	OS3	“…we definitely have patients that have a lack of a solid internet connection…Reliable internet—that's true, yes; which could make the visit hard.”—T5 (0–5 years) “Every once in a while, the connection will be a little difficult and that the screen will freeze or you can't hear everything they say but…It depends on their Wi-Fi connection…Probably maybe like 10 to 20% of visits, there's some like freezing of the screen or we have to repeat things, or you have to wait for it to catch up in real time or whatever.”—T13 (16–20 years)
	OS4	“If they need to talk about something very sensitive and they are in their homes and feel uncomfortable about that, that might be a barrier to it [telemedicine].”—T8 (31+ years) “Many times, it took longer for me because I don't know, I think for patients there were like some extra distractions, or they just want to talk a lot more than they normally do. You know trying to work from home and having like a cat walk in across the computer and then my patients want to talk to my cat, or having my elementary school kids e-learning in the office next to me and they want to say hi to my patients and my patients going to say hi to them, that slows it down a bit. Like it's cute the first time but the 30th time, I'm just like ‘No, keep going I have 10 more patients this morning.’”—T7 (16–20 years)
	OS5	“If you are someone that depends on public transportation and you missed the bus. For whatever reason, your Uber doesn't show or whatever you want. Here, there's no reason why you'll have missed your appointment if you're at your home, okay? So it provides that—it can eliminate the concern for transportation. It's also for patients who have mobility issues; wheelchairs, walkers, all that sort…I also think that it also allows for—I'll just say people who acted as an interpreter for us, they could just sit right alongside the patient and interpret off a phone if they know ahead of time. We've had patients who are deaf that we have their speech individuals and they're pairing people right there with the visit. So they don't have to take off work themselves. So, all that made it a great opportunity to make sure they got an appointment.”—T8 (31+ years)
Implementation ClimateWhat was the general level of receptivity in your clinic to implementing telemedicine visits?	IS1	“I think we probably had a handful of people who were nervous about it [telemedicine],but I think everybody else thought it was a great idea, and I think even the people that were nervous about it, once they started doing it, found that it was very doable and a very good option. So, I would say that the reception was overall very, very good or excellent.”—T5 (0–5 years)
Available ResourcesWhat supports were available to help you implement and use telemedicine visits? How did you access these materials?	IS2	“The ability to dial the patient's phone number or not—I'm using through Doximity—I had not done that prior to this roll-out. That was really useful…Obviously, it's Microsoft Teams scheduling that was really key because there are people that work for the telehealth clinic, they get on and they text me in the chat to say this person is ready or this person is not ready yet and they have a question about this with their support people that they have devoted to telehealth that make sure that it all runs smoothly and then all this billing stuff gets taken care of, so they wouldn't be able to do telehealth without them and they actually connect the patient and collect their information and their payment and all that kind of stuff…and then there's the charge nurses…also are people that I go to for support, like if I have a question about the appropriateness of whether this person should really be a telehealth patient. I go talk to them and they talk me down to the ledge and they say ‘Why don't you find out what they need and we'll try to get them in the office as quickly as possible?’…they're kind of problem solvers and troubleshooters as well.”—T2 (11–15 years)
	IS3	“Also the other thing that's missing with telemedicine is there's no medical assistance, so I was doing all the things that I don't necessarily normally do like documenting vital signs, updating their medical list or pharmacy when needed. So I was missing having a medical assistance do those things.”—T10 (6–10 years)
	IS4	“So, I think instead of having 15 or 20-minute patient encounters, like having the 30 minutes blocks really allowed for a little troubleshooting and not having patients delayed in their appointment time if there's a problem came up, because there was a little bit extra time for that kind of trouble buffer.”—T5 (0–5 years) “I would say there would have to be specific schedules made for telehealth so the physician director has to set aside time where they specifically schedule providers for telehealth versus trying to filter in telehealth into our regular schedule…there would have to be a commitment on the part of the physician to prioritize telehealth.”—T2 (11–15 years)
	IS5	“I would say lacking certain resources, the availability of needing a quick lab or if fetal non-stress test is indicated, checking a blood pressure. I mean we encourage everybody to get blood pressure cuffs and we prescribed them for patients but not everybody got them…Some of them say it wasn't covered by insurance and it was too expensive.”—T10 (6–10 years)
Access to Knowledge and InformationWhat information and materials were made available to help you implement and use telemedicine visits? What was missing? What continued training is needed?	IS6	“We have a lot of more technologically savvy faculty, like we have a lot of young faculty who were trained with a lot of newer technology. So, I think it was a little bit easier for us...”—T5 (0–5 years) “We kind of just got thrown into it and say, ‘Hey, download this app and you're going to be using it like this.’ We've got a short, kind of—like if you have questions, you can get into these sessions to be able to understand it more and how to use it…but I think the whole process is pretty easy if you are young and you're used to using the internet, and the ability of the modern technology.”—T12 (6–10 years)
	IS7	“It was really helpful to kind of compare notes with other institutions across the country about what a pre-natal care schedule would look like—maybe look a little differently when they would come in person, when they would go telehealth. So, those resources comparing with other big institutions—that was helpful.”—T7 (16–20 years)
	IS8	“I think just more education on the financial part of telemedicine like the billing and how we can optimize billing…but also not have it become an increase financial burden on our patients, and kind of finding that sweet spot is I think somewhere we can improve.”—T1 (0–5 years)
	IS9	“I think that's going be something long term that we would probably benefit from a discussion of how best to use audio/visual resources in healthcare. Things like eye contact and interpersonal interactions over a less natural medium would probably be helpful for most people so that we can optimize how patients see us...”—T6 (0–5 years)
	IS10	“If any, [needed] training would be to troubleshoot, you know, how come—I can't hear the patient or the patient can't hear me, those little glitches that could happen during a visit.”—T12 (16–20 years)
Knowledge and Beliefs about the InterventionHow did you feel about the plan to implement telemedicine? How did you feel about telemedicine visits being used in your clinic?	IN1	“I think it's great. I think it's a great alternative to let's say patients who just can't—let's say they're 30 minutes or 45 minutes away, and they have work, and they have a 30-minute break, but they can have a visit in. It allows more access to care. It's a great addition. I don't think we can take away in-person visits ever, but I definitely think it's a great addition, great option to have, especially for low-risk patients.”—T4 (0–5 years) “I thought it was a good idea especially since we were pretty much mandated that we were only going to see the very necessary patients in the office and I was worried about our patients not having access to care. I saw it as a great alternative. So I was looking forward to learning it and adopting it for our care model.”—T13 (16–20 years)
	IN2	“As I said before, in the setting in the onset of the pandemic and the restrictions that were being applied, it was life-saving and it saved all of our jobs, and probably a lot of our lives because we were able to reduce visits...Moving forward, in a post-pandemic world, I do feel like it should play a role…I feel very positive about it. I don't think it's the answer to every problem, but I definitely want to work in a place where telemedicine is an option for patients that want that.”—T2 (11–15 years)
EngagingWhat steps were taken to encourage patients to commit to using telemedicine visits?	Pr1	“Then, explaining what's going to happen at each visit…giving them information about how to take a blood pressure at home, checking their weight at home…to replace what we do in the office at those visits. Educating the patient about how that is a reasonable option and it doesn't compromise their care by doing it that way… we have this telemedicine option for you…We're just trying to create a process that is safer for you and maybe advantageous to you in many ways like that. I didn't want any patient to feel like we didn't want to see them. I always constantly reassure them that if there's an emergency or you have an issue, you need us to look at something or you're not comfortable on the phone, you let us know and we're happy to adjust.”—T11 (16–20 years)
	Pr2	“That is because a lot of our offices closed. It was probably less encouragement…At one point they [patients] didn't have a choice and now it's being presented as a normal part of real care as opposed to an optional unique aspect of it.”—T6 (0–5 years)

OB, obstetrics.

PROVIDER DESCRIPTION

The participants in this study were predominantly white (74%), with black (11%), Hispanic (11%), and Asian (5%) providers represented as well (Table 2). Over a third (37%) had been in practice for up to 5 years, while over 58% had been in practice for >15 years.

PROVIDER PERCEPTIONS OF IMPLEMENTATION

Before this implementation of obstetric telemedicine, almost all providers had never used telemedicine in practice before. With the rapid implementation of obstetric telemedicine early in the COVID-19 pandemic, obstetric providers shared their perceptions.

QUALITY

The quality of telemedicine visits referred to providers' perception of the safety (avoiding risk or harm) and efficacy of telemedicine visits to have desired outcomes. Most providers did not have any concerns about the safety and efficacy of telemedicine visits. However, a few providers (11 years or more in practice) had concerns about solely using telemedicine for new patients with unrecognized urgent conditions or for patients who required a physical examination or ultrasound (IC1).

ADAPTABILITY

The adaptability of telemedicine visits entailed the degree to which changes could be made to telemedicine visits for it to work effectively in a clinic. Such changes included the use of multiple platforms (e.g., Microsoft Teams, Doximity dialer, etc.) to conduct telemedicine visits efficiently, the flexibility of both scheduling patient appointments and allotting extra time to transition between scheduled in-person and telemedicine appointments, or simply for troubleshooting technical issues if necessary. Likewise, the ability to incorporate support staff during telemedicine visits to improve workflow made telemedicine work effectively (IC2).

RELATIVE ADVANTAGE

The relative advantage of telemedicine visits referred to providers' perception of the advantages of implementing obstetric telemedicine compared to in-person obstetric visits. Generally, providers regarded telemedicine as comparable to in-person obstetric visits. Convenience, comfort, and decreased exposure to COVID-19 were the most common themes associated with the relative advantage of obstetric telemedicine.

Convenience was described by providers as less travel commute and parking issues, less visit wait times, reduced need for childcare, fewer missed or late appointments, more flexibility in scheduling appointments, and longer visits for counseling. In addition, providers expressed added benefits of telemedicine for both patients and providers to include the convenience of less distractions and a decreased risk of exposure to COVID-19 infection (IC3). Providers also discussed the personal connections they were able to have with patients because of how comfortable patients were in their home environment during telemedicine visits (IC4).

Overall, providers described telemedicine as having several advantages over traditional in-person visits. Barriers such as limitations in performing physical examinations, laboratories, or ultrasounds when deemed necessary were addressed by scheduling the patient for an in-person visit.

COMPLEXITY

The complexity of telemedicine visits was defined as the perceived difficulty of its implementation. Most providers described telemedicine visits as simple or not complicated to implement. However, a few providers in practice for 16 years or longer found telemedicine to be slightly more complex in terms of the multiple steps involved in the process of conducting a telemedicine visit (IC5).

PATIENT NEEDS AND RESOURCES

This construct encompassed the extent to which patient needs, and barriers and facilitators to meet those needs, were known and prioritized by the clinic. In the context of the COVID-19 pandemic, which restricted access to health care services, telemedicine offered an alternative option to access obstetric care.

Themes encompassing this construct—patients' need for convenience, comfort, decreased exposure to COVID-19 infection, and physical examination, were similar to those under the relative advantage construct. For example, one provider discussed how the convenience of obstetric telemedicine visits and the flexibility to include family members for support at these visits was helpful to patients (OS1). Many providers mentioned the need expressed by patients to hear the fetus' heartbeat during visits. However, this was only possible if the patient had an at-home fetal heart doppler machine.

Barriers to meeting these needs included a lack of patient access to at-home monitoring devices such as BP machines, weight scales, and fetal heart doppler machines to measure vital signs; health insurance policies not covering the purchase of these at-home monitoring devices; the impracticality of having a physical examination, laboratory work or ultrasound done if required (OS2); and access to technology such as reliable internet service, which was required to participate in telemedicine visits (OS3).

Some providers mentioned patient privacy and distractions as barriers to patient comfort and a more efficient telemedicine visit (OS4). Overall, providers felt that telemedicine as an alternative option to in-person obstetric visits met most of these patients' needs. This was particularly for low-risk obstetric patients not requiring a physical examination and for patients with certain hearing or mobility limitations (OS5).

IMPLEMENTATION CLIMATE

Implementation climate referred to the shared receptivity of providers to telemedicine visits within their clinics. Overall, providers were receptive to the implementation of telemedicine visits. One provider reported that although some providers were initially nervous about the implementation of telemedicine, it was, however, well received (IS1).

READINESS FOR IMPLEMENTATION

The tangible and immediate indicators of overall commitment to the decision to implement telemedicine visits are described by the readiness for implementation construct. Regarding the question “Did you feel the training prepared you to carry out telemedicine visits?,” all providers acknowledged feeling prepared to implement telemedicine visits. We further investigated two of the three subconstructs under the main construct readiness for implementation, that is, (1) available resources and (2) access to knowledge and information.

AVAILABLE RESOURCES

This subconstruct included resources devoted for the implementation and ongoing operation of telemedicine visits. Generally, themes associated with resources available for implementation as reported by providers included technology access and support, staff support, time availability, and access to at-home monitoring devices (e.g., BP machine and so on). For some providers, resources for telemedicine implementation were considered adequate, while for others the opposite held true. For example, access to telemedicine platforms such as Microsoft Teams and Doximity and the availability of staff to troubleshoot both technical issues and assist with the telemedicine visit process were considered ample (IS2). However, some providers did not have adequate staff support (IS3).

Likewise, provision of time allotted for telemedicine visits was sufficient and appropriate for some, but not all providers (IS4). Finally, some patients had access to BP machines, weight scales, and fetal heart dopplers provided to them by the clinic, while other patients did not have access because they were not covered by insurance and were too expensive (IS5). This created a barrier for patients' desire to hear the baby's heartbeat and for providers to complete comprehensive telemedicine visits.

ACCESS TO KNOWLEDGE AND INFORMATION

This construct included access to available information and knowledge associated with obstetric telemedicine and its use. Providers reported generally gaining knowledge and access to information about telemedicine visits through in-person and online trainings. In particular, providers with fewer years in practice revealed that the process of familiarizing oneself with telemedicine was easier for younger or more tech savvy colleagues (IS6). One provider sought obstetric telemedicine information provided by their clinic and compared it to information provided at other larger institutions (IS7).

Although most providers gained knowledge and information from trainings offered, some indicated that there were no need for continued trainings, while others indicated certain aspects they considered missing or needing additional trainings. These included information and trainings related to telemedicine billing and reimbursements (IS8), improvements in provider–patient interaction (IS9), and troubleshooting technological issues (IS10).

KNOWLEDGE AND BELIEFS ABOUT THE INTERVENTION

The construct of knowledge and beliefs about obstetric telemedicine referred to providers' attitudes and values placed toward telemedicine visits and its implementation in their clinics. Overall, all providers had positive reactions to implementation as it related to their knowledge and beliefs about telemedicine. Providers mostly considered telemedicine a usefulness alternative to deliver obstetric care especially for low-risk obstetric patients (IN1). In addition, some providers believed the implementation of telemedicine enabled obstetric providers to continue employment during the COVID-19 pandemic, and despite few providers believing that its implementation was rushed, some providers expressed the desire for telemedicine to continue beyond the pandemic (IN2).

ENGAGING

The question “What steps were taken to encourage patients to commit to using telemedicine visits?” referred to the construct of engaging. Mostly, providers reported that patients were encouraged to commit to participating in telemedicine visits by educating them about the simplicity and advantages of telemedicine (similar to those mentioned under the relative advantage construct), explaining to them what the process of a typical telemedicine visit entailed, as well as reassuring them of adequate patient care (Pr1). One provider reported that because of the COVID-19 pandemic, patients were not given an option to opt-out of telemedicine visits. Nevertheless, it has now become part of routine care in the clinic (Pr2).

Discussion

The implementation of the obstetric telemedicine care model in our clinical setting was deemed favorable, safe, and an alternative option for patients during the COVID-19 pandemic. The majority of providers found this model of care to be easy to use and implement even though they did not have past exposure to telemedicine. Barriers to telemedicine noted included lack of home monitoring devices for blood pressure and fetal heart tones, knowledge of billing, and concerns regarding patient privacy. These advantages and barriers are similar to those reported in other studies analyzing rapid telemedicine implementation in the post-COVID era.^10

–16

It is encouraging that most of the barriers identified by providers can be addressed. Furthermore, the pros noted in this evaluation suggest that it may be advantageous to extend telemedicine beyond the pandemic to improve the efficiency and access of health care services provision in the future. The American College of Obstetricians and Gynecologists has provided guidance to providers in the form of a telemedicine implementation guide²⁵ and guide for telemedicine implementation during the pandemic.²⁶

STRENGTHS AND LIMITATIONS

One of the major strengths of this study was the ability to quickly evaluate a rapid implementation of telemedicine during a pandemic. We were able to capture this feedback during the rollout of a new model of care and in a prospective manner. Our results compare favorably with the current research, which shows that telemedicine was well received during the COVID pandemic.^10

–16 In addition, we were able to perform this evaluation with a robust theoretical framework. Weaknesses include that this study is single site within the context of a unique situation (rapid implementation due to the COVID-19 pandemic); therefore, the results may differ from future implementation of telemedicine where more advanced planning and training would be possible before implementation.

FUTURE RESEARCH

The research on telemedicine-enhanced prenatal care during the pandemic is a growing field, but solid randomized trials are scant in the literature.^27,28 Further evaluation is also needed to explore this care model from patient perspectives and randomized controlled trial is needed to show safety and efficacy. In addition, more research is needed on how to integrate telemedicine into care for low-risk pregnancies, patient privacy concerns, and access to internet and at-home monitoring devices.

Telemedicine shows promise for certain populations, but may not be ideal for all due to insurance reimbursement issues, need for in-personal monitoring for high-risk pregnancies, and access to high-speed internet. Additional funding is needed to provide patients with at home monitoring devices through insurance reimbursement or other funding mechanisms, especially for low-income women who cannot afford these devices due to high out-of-pocket costs.

Further guidance and standardization of a telemedicine prenatal care model for low-risk women from national professional organizations such as American College of Obstetricians and Gynecologist and the American College of Nurse-Midwives would be advantageous for both providers and patients in implementation of this model across the country. Some clinics may find that telemedicine is best used to provide specific patient education or remote monitoring.^8,26,29

Conclusions

From the provider perspective, the rapid implementation of the obstetric telemedicine care model in a single clinical site was deemed a favorable option for patients during the COVID-19 pandemic and shows promise for further study. Future research and evaluation are needed to assess patient satisfaction, safety, and compare maternal/neonatal outcomes compared to in-person only care.

Footnotes

Disclosure Statement

No competing financial interests exist.

Funding Information

This work was supported by University of South Florida Internal Seed Grant (Grant number IS415-11).

References

Johns Hopkins University & Medicine. Coronavirus Resource Center. 2021. Available at https://coronavirus.jhu.edu (last accessed February 12, 2021 ).

Centers for Disease Control and Prevention. CDC COVID Data Tracker. 2021. Available at https://covid.cdc.gov/covid-data-tracker/#cases_casesper100klast7days (last accessed February 12, 2021 ).

Fryer

, Delgado

, Foti

, Reid

, Marshall

. Implementation of obstetric telehealth during COVID-19 and beyond. Matern Child Health J, 2020; 24:1104–1110.

Centers for Medicare & Medicaid Services. Telemedicine. 2021. Available at https://www.medicaid.gov/medicaid/benefits/telemedicine/index.html (last accessed January 19, 2021 ).

American Hospital Association. Fact Sheet: Telehealth. 2019. Available at https://www.aha.org/system/files/2019-02/fact-sheet-telehealth-2-4-19.pdf (last accessed January 20, 2021 ).

Robeznieks

Which medical specialties use telemedicine the most? 2019. Available at https://www.ama-assn.org/practice-management/digital/which-medical-specialties-use-telemedicine-most (last accessed January 20, 2021 ).

Colbert

, Venegas-vera

, Lerma

. Utility of telemedicine in the COVID-19 era. Rev Cardiovasc Med, 2020; 21:583–587.

DeNicola

, Grossman

, Marko

, et al. Telehealth interventions to improve obstetric and gynecologic health outcomes: A systematic review. Obstet Gynecol, 2020; 135:371–382.

Pflugeisen

, McCarren

, Poore

, Carlile

, Schroeder

. Virtual visits: Managing prenatal care with modern technology. MCN Am J Matern Nurs, 2016; 41:24–30.

10.

Barney

, Buckelew

, Mesheriakova

, Raymond-Flesch

, Barney

. The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: Challenges and opportunities for innovation. J Adolesc Health, 2020; 67:164–171.

11.

Loeb

, Rao

, Ficke

, Morris

, Riley

, Levin

. Departmental experience and lessons learned with accelerated introduction of telemedicine during the COVID-19 crisis. J Am Acad Orthop Surg, 2020; 28:e469–e476.

12.

Smith

, Atala

, Terlecki

, Kelly

, Matthews

. Implementation guide for rapid integration of an outpatient telemedicine program during the COVID-19 pandemic. J Am Coll Surg, 2020; 231:216–222.e2.

13.

Bishop

, Jackson

, Vats

, Azzuqa

. Prenatal neonatology telemedicine consultation for patients with fetal anomalies during the COVID-19 pandemic era: Rapid implementation and lessons learned. J Perinatol, 2020; 40:1451–1452.

14.

Yellowlees

, Nakagawa

, Pakyurek

, Hanson

, Elder

, Kales

. Rapid conversion of an outpatient psychiatric clinic to a 100% virtual telepsychiatry clinic in response to COVID-19. Psychiatr Serv, 2020; 71:749–752.

15.

Peahl

, Smith

, Moniz

. Prenatal care redesign: Creating flexible maternity care models through virtual care. Am J Obstet Gynecol, 2020; 223:389.e1–e389.e10.

16.

Aziz

, Zork

, Aubey

, et al. Telehealth for high-risk pregnancies in the setting of the COVID-19 pandemic. Am J Perinatol, 2020; 37:800–808.

17.

CFIR Research Team. The Consolidated Framework for Implementation Research. 2021. Available at https://cfirguide.org (last accessed February 12, 2021 ).

18.

CFIR Research Team. Constructs—The Consolidated Framework for Implementation Research. 2021. Available at https://cfirguide.org/constructs (last accessed February 12, 2021 ).

19.

CFIR Research Team. Welcome to the Interview Guide Tool. 2021. Available at https://cfirguide.org/guide/app/# (last accessed February 12, 2021 ).

20.

Bernard

, Ryan

. Analyzing qualitative data: systematic approaches. Thousand Oaks, CA: SAGE, 2010.

21.

Saldana

. The coding manual for qualitative researchers, 3rd ed. USA: Arizona State University, 2015.

22.

VERBI Software. MAXQDA 2020 [computer software]. 2019. Available at maxqda.com

23.

Creswell

, Creswell

. Research design: Qualitative, quantitative, and mixed methods approaches, 5th ed. Los Angeles, CA: SAGE, 2018.

24.

Lincoln

, Guba

. Naturalistic inquiry. Newbury Park, CA: SAGE Publications, 1985.

25.

American College of Obstetricians and Gynecologists. COVID-19 FAQs for obstetrician-gynecologists, telehealth. Washington, DC: ACOG, 2020. Available at https://www.acog.org/clinical-information/physician-faqs/covid-19-faqs-for-ob-gyns-telehealth (last accessed February 10, 2021 ).

26.

American College of Obstetricians and Gynecologists. Implementing Telehealth in Practice. Committee Opinion No. 798. Obs Gynecol . 2020; 135:e73–e79.

27.

Theiler

, Butler-Tobah

, Hathcock

, Famuyide

. OB Nest randomized controlled trial: A cost comparison of reduced visit compared to traditional prenatal care. BMC Pregnancy Childbirth, 2021; 21:1–6.

28.

Butler Tobah

, LeBlanc

, Branda

, et al. Randomized comparison of a reduced-visit prenatal care model enhanced with remote monitoring. Am J Obstet Gynecol, 2019; 221:638.e1–e638.e8.

29.

Marko

, Ganju

, Brown

, Benham

, Gaba

. Remote prenatal care monitoring with digital health tools can reduce visit frequency while improving satisfaction [3]. Obstet Gynecol, 2016; 127(Supplement 1):1S.