The analysis of the effect of satisfaction on telehealth utilization with a structural equation model

Abstract

BACKGROUND:

While COVID-19 has significantly impacted how healthcare is provided, telehealth services with remote access have dramatically reduced the chance of in-person contact and the costs of services for patients with increased healthcare quality.

OBJECTIVE:

As the COVID-19 is still a very prevalent part of people’s lives, it is critical to examine what factors affect telehealth, impacts the choice of the prominent and efficient healthcare service platform, and utilizes telehealth effectively and efficiently.

METHODS:

Grounded on the Self-determination theory (SDT), this research analyzes a sample of 142 response data for the effects of access, need, knowledge, and technology skills, on telehealth utilization mediated by satisfaction via Structural Equation Modeling (SEM).

RESULTS:

The empirical results indicated that telehealth satisfaction significantly increased the use of telehealth services during the pandemic. Meanwhile, the access, needs, and knowledge of telehealth also significantly increase telehealth utilization directly and indirectly through telehealth satisfaction. Additionally, an individual’s technological skill has no significant effect on telehealth use; instead, it can increase telehealth satisfaction, which increases telehealth utilization.

CONCLUSION:

The study with its theoretical and practical implications may provide researchers and public health officials with new options and strategies for telehealth services regarding the pandemic issue.

Keywords

COVID-19 telehealth utilization telehealth satisfaction self-determination theory structural equation model

Dr. Xiaohui You is an Assistant Professor in the Department of Health Professions at the Metropolitan State University of Denver. Her research interests include Health Economics, Strategic Management, and Health Management.

Dr. Erin Seedorf is an Assistant Professor with the Department of Health Professions at the Metropolitan State University of Denver. Dr Seedorf is the program coordinator for the MSU Denver Public Health Program. Her teaching and research focus on community and population health, health policy, and program planning and evaluation. Previously, Dr Seedorf served as a program manager and evaluation specialist at the Colorado School of Public Health, University of Colorado Denver at Anschutz

1 Introduction

Healthcare is a constantly and rapidly changing industry, with new and upcoming technologies being woven into our lives at a remarkable pace. Our dependence on technology is unquestionable, and the adoption of these technologies, while sometimes challenging, has been critical for societal progress. This was never more evident than in 2020 and the impact of the global Coronavirus (COVID-19) pandemic. During the pandemic, employers worldwide were faced with moving staff to virtual work environments, including the healthcare industry. As a result, healthcare industries were forced to learn how to shift over to telehealth services within weeks quickly. Telehealth is a new avenue barely being paved within healthcare; however, it has been pivotal in reducing healthcare costs and expenditures [1 –3]. For example, Telehealth allows healthcare providers to connect, visit, and communicate electronically, thus potentially reducing the number of visits a patient may have and expanding access to care for patients in rural areas [4, 5]. Additionally, Telehealth can reduce overall health care costs by reducing travel expenses for both patients and providers [3].

The impact of COVID-19 was felt by everyone over the past two years, including practices of wearing masks, social distancing, and being restricted on what one could or could not do. For example, growing science demonstrated that the transmission mechanism was direct contact with individuals with COVID-19 through airborne particles and droplets [6]. Acknowledging this and subsequent stay-at-home orders, the US and most of the world went into a lockdown, which resulted in the cancelation of many activities, including doctor’s appointments. Without Telehealth, many clinics would have had to shut down entirely [7]. However, due to the availability of telehealth in many clinics and practices, most in-person appointments were able to be moved to virtual appointments, thus there was a continuation of needed health care services [8]. This would allow for minimized contact with people outside of the home, thus reducing exposure in an effort to slow down the COVID-19 curve.

Further, as reported in February 2022 by the Assistant Secretary for Planning and Evaluation, Office of Health Policy, approximately 1 in 4 adults in the US reported having an appointment by video or phone in the past four weeks [9]. Telehealth allows providers to collaborate more efficiently, and working from home offices, home-based teleworking has become the “new normal” [10, 11]. By 2020 and 2021, those accessing Telehealth are expected to grow more than before. Recent policy changes during the COVID-19 pandemic have reduced telehealth access barriers and have promoted telehealth use to deliver acute, chronic, primary, and specialty care [12]. While the use of Telehealth has been an idea for a few years, it is still evolving and probably will continue to grow through to the continuation of this pandemic.

While COVID-19 has significantly impacted how healthcare can and is being provided, telehealth services with remote access have dramatically reduced the chance of in-person contact and the costs of services for patients while increasing healthcare quality [1, 13]. Therefore, analyzing telehealth utilization during the pandemic and identifying factors influencing telehealth services could provide researchers and public health officials new options and strategies. While previous findings are relevant, the literature will be more focused on the common factors of patient satisfaction related to health outcomes, cost, and preference.

This research aimed to address the current knowledge gap on how patient satisfaction with Telehealth could increase telehealth utilization, mainly focusing on the effects of need, accessing healthcare, the knowledge, and technology skills around Telehealth, using the robust methodology through structural equation modeling (SEM). Specifically, this paper investigates the relationship between the individuals’ telehealth utilization and telehealth satisfaction, access to Telehealth, needs, knowledge, and technology skills, using the Self-determination theory [14 –16]. We hypothesize that access, needs, knowledge, and technology skills have direct and indirect effects (through telehealth satisfaction) on telehealth utilization.

2 Literature review

2.1 Literature and research hypotheses

Telehealth has been used limitedly for decades, primarily to make healthcare available to people who live in rural or isolated communities and people with limited time, mobility, and transportation options [17]. Widespread use of Telehealth was often connected to factors such as cost, technology, and ambiguous reimbursement regulations [18, 19]. However, during the pandemic, Telehealth helped providers and patients avoid unnecessary contact and minimized the risk of COVID-19 transmission. Thus, the need for telehealth services has driven up the use of Telehealth, and people use it because they need it.

According to the previous literature, several factors influence the use of Telehealth, including areas like satisfaction, comfort, ease of use, and convenience [20 –24]. It has been identified that satisfaction of Telehealth can powerfully increase telehealth utilization [20 –24]. How comfortable, easy, and convenient it is for patients to use Telehealth services and express their concerns are also important factors influencing satisfaction and utilization. Studies have found that several factors could impact the satisfaction with telehealth services; patients have different barriers (language, knowledge, etc.) that make it difficult to express their concerns or lack of connection or relationship with providers [20, 21]. Other studies found that patients expressed dissatisfaction with telehealth services due to the lack of “hands-on” care, making them feel like their needs were not met [22]. Meanwhile, looking at non-English speaking and elderly patients, researchers found that they were dissatisfied with telehealth services and preferred in-person visits due to feelings of being misunderstood or generally not feeling confident in the quality of the virtual visit [23]. Further, poor user accessibility, such as technological limitations, may outweigh the need and affect overall satisfaction [24].

With telehealth use trending more after the COVID-19 pandemic, it seems more people are acquiring resources and preferring the convenience of telehealth services [18 , 26]. For example, the American Psychiatric Association found that nearly four in ten Americans, or approximately 38%, have been using telehealth services –which increased by 7% from fall 2020 or the pandemic’s beginning [25]. In addition, patients report the convenience of Telehealth contributes to their satisfaction due to not having to be absent from work and/or travel to these appointments –saving individuals gas, time, parking, physical limitations, etc., particularly for individuals in rural communities or cities [18 , 27]. Studying 1734 patients, Polinski et al. found that 95% of patients appreciated Telehealth because of its convenience [28]. Similarly, using a study of 84 participants, Ruelos et al. pointed out that 86.7% of patients were using Telehealth in place of in-person visits due to convenience, again a primary driver of telehealth satisfaction and utilization [26]. Therefore, access to Telehealth is crucial for telehealth utilization; increasing access by making care more convenient may increase telehealth utilization and health care spending [29 –31].

Again, using Telehealth requires technological and language skills, as language could preclude some individuals from using the service. A study conducted in California found that patients with language barriers (i.e., those patients with limited English proficiency) had lower rates of telehealth use than proficient English speakers (4.8% versus 12.3%) [32]. In a retrospective cohort study, investigators found fewer rheumatology appointments were kept during COVID-19 and associated the preference for the Spanish language or other non-English languages with the decline in video use [33].

The technology and the user’s technical skill level are critical factors influencing telehealth utilization. While Telehealth ensured patients continued HIV care services in South Carolina during the COVID-19 pandemic, technological challenges, digital literacy, client/provider experiences, and factors were barriers to using Telehealth [34]. Similarly, while Telehealth improves health outcomes and quality of life indicators for older adults, using the technology was often an obstacle [35]. High rates of non-participation in Telehealth have been noted in aging populations. The population refused acceptance because they believed there were requirements for technical competence and operation of equipment [36].

Investigators have been interested in determining the link between telehealth usage and knowledge of Telehealth. One study found that interest in telehealth utilization was higher when patients had more confidence in using the associated technology and perceived more advantages than disadvantages when using Telehealth [37]. While different forms of Telehealth were more prevalent before 2020, most respondents were willing to use videoconferencing visits [38]. However, there was a caveat to the findings; participants who identified as Black, were 65 years or older, or had less education (high school or less) were less likely to be willing to use videoconferencing [38]. Even when the benefits of Telehealth are known, some indicated a preference for in-person visits. For example, families of kids with disabilities who use early intervention therapy prefer in-person visits, feeling that they have to compromise the relationship between family, child, and therapist and that treatment would not be as effective if done through Telehealth [39].

The knowledge of Telehealth among providers and the clinicians also has been shown to impact the utilization of Telehealth. While most physicians indicated that they would be interested in using the technology, some also expressed they had little understanding of Telehealth and perceived the advantages of Telehealth at a moderate level [40]. Non- or underutilization of Telehealth exists among clinicians as well. For example, while clinical video telehealth (CVT) enables care to be provided directly to veterans’ home, it remains underutilized despite training initiatives [41]. While the majority of the mental health providers endorsed the use of CVT, less than half used it for patient care because of a lack of training or administrative burden. Training with a multimodal approach to nurse practitioners found that they were more likely to embrace telehealth usage [42]. Thus, increased training would likely increase telehealth utilization overall.

Meanwhile, knowledge sharing significantly and positively affected organizational performance when organizations were going virtually during the COVID-19 pandemic [43]. As the nature of healthcare is, patients must rely on physicians to make healthcare decisions, such is the so-called physician-induced demand. Thus, the telehealth providers’ utilization is connected patients’ utilization.

Through the exiting literature, some of the previous literature suggest that access, need, technology knowledge, and skills influence telehealth utilization while some indicate that those factors could influence telehealth satisfaction; however, few indicate that those factors could influence both utilization and satisfaction. To date, no study examines that access, need, technology skill, and knowledge could directly influence both utilization and satisfaction with telehealth; meanwhile, those factors also indirectly influence utilization through satisfaction with telehealth services. Therefore, this research fills this gap by employing the structural equation modeling (SEM) approach to capture the direct and indirect effects on telehealth utilization.

Therefore, the hypotheses in the following research are:

As the access to telehealth services increases, telehealth utilization increases (H1d), and telehealth satisfaction may also increase (H1i), which in turn increases telehealth utilization (H5).

As the need for telehealth services increases, telehealth utilization increases (H2d), and telehealth satisfaction may also increase (H2i), which in turn increases telehealth utilization (H5).

As the individuals’ knowledge of telehealth services increases, telehealth utilization may increase (H2d), and telehealth satisfaction may also increase (H2i), which in turn increases telehealth utilization (H5).

As the individuals’ technology skills increase, telehealth utilization may increase (H2d), and telehealth satisfaction may also increase (H2i), which in turn increases telehealth utilization (H5).

2.2 Self-determination theory

Self-determination theory (SDT) is a theory of human motivation and personality that suggests people can become self-determined when their needs for competence, relatedness, and autonomy are fulfilled [44]. Self-determination refers to a person’s ability to manage themselves, make confident choices, and think independently [45]. Deci and Ryan first introduced self-determination theory in their book Self-Determination and Intrinsic Motivation in Human Behavior [14]. Self-determination significantly impacts motivation; for example, if the individual believes he/she can manage himself/herself correctly, he/she would have more motivation to carry out some tasks.

Self-determination theory posits that three innate and universal psychological needs drive people and personal well-being is a direct function of satisfying these basic psychological needs [46]. First, autonomy, the autonomous motivation is crucial as the individuals feel that they have the choice and willingly endorse their behavior. Second, relatedness, the need for growth as a human being, drives behavior as people always actively seek to grow and improve [44]. And third, competence, mastery over challenges, knowledge, and skill, are essential for developing a sense of self. As a result, the quality of motivation is key to both satisfaction (telehealth satisfaction) and sustained success in achieving one’s goals (telehealth utilization). The more pleasure and satisfaction the individuals receive from telehealth services, the more they would use these service options.

Self-determination theory has been widely used in health behavior, physical, and psychology studies regarding the effects on motivation [47 –49]. For example, during the COVID-19 crisis, researchers investigated emotion regulation and psychological needs based on Self-determination theory to examine the individuals’ mental health and students’ motivation to engage in online learning [50, 51].

In this study, the access to telehealth services is the autonomy factor; the need for telehealth services is precisely the relatedness motivation; then, knowledge of telehealth and technology skills necessary to use the telehealth services are factors to estimate the competence in SDT. As hypothesized, access, need, knowledge, and technology skills can affect telehealth utilization directly or indirectly through telehealth satisfaction. Based on SDT, the theoretical model and factors in this study were depicted graphically in Fig. 1. (See Fig. 1: Conceptual model based on Self-Determination Theory)

Fig. 1

Conceptual model based on Self-Determination Theory (SDT).

3 Methods

3.1 Research design and planning

This study aims to look at how often and how well people utilize telehealth in the United States. Many variables will contribute to how often and well these services are used, such as satisfaction, accessibility, type of healthcare, etc. With COVID-19 being an integration of everyone’s daily life, we wanted to analyze why people in the United States are not utilizing telehealth to its full potential. This existing gap could lead to better implementation and improvement of the telehealth services that are currently in place in the United States. Thus, existing literature were investigated for this research to see any similarities that this research and others may have curated. This study consists of distributing an online survey to consenting adult volunteers in the United States. The survey participants anonymously participated by answering questions related to telehealth services in the US. By distributing the survey link on social platforms and to colleagues, the data were collected anonymously and effectively via an electronic survey.

3.2 Sampling, recruitment, and instruments

The primary research tool administered for this study was an anonymous survey to ascertain customer satisfaction with telemedicine services. This instrument was based on survey instruments pulled from several studies on telehealth research before or during COVID-19 [33, 48]. The IRB protocol “Telehealth utilization in the US during COVID-19” was submitted and approved by Metropolitan State University Denver’s Institutional Review Board. Survey questions were modified to fit the current research focus but were influenced by telehealth literature examples. The survey was distributed to a sample of patients, primarily from two states in the US, Colorado (CO) and Texas (TX), through social media and email recruitment. In addition, the email provided potential participants with an anonymous survey link administered through the Qualtrics survey program.

3.3 Data

In the survey, participants were asked a series of questions regarding basic demographic information, information on health behaviors, and then perceptions of telehealth satisfaction and telehealth utilization. Participants were instructed that they could choose not to answer any of the questions or end their participation at any time for a reason. From February to March 2022, we received 175 observations, but some were incomplete and contained missing values. Regarding the physician-induced demand for healthcare services, survey participants include healthcare providers. After data cleaning, 142 observations remained as complete data. Using the structural equation modeling (SEM) approach, an analysis was conducted with the data to capture the mediator effect with the bootstrapping method. The skewness and kurtosis for each factor suggest a normal distribution for each individual variable; however, the Multivariate for Kurtosis and c.r. (>5) indicates that the multivariate normality may not hold for the dataset. With bootstrapping technique, Bollen-Stine Bootstrap, we could handle this issue. (See Table 1: Descriptive Statistics)

Table 1
Descriptive Statistics

Variable (label) Question Min Max Mean Std. Dev. skew c.r. kurtosis c.r.

Access to Telehealth (access) a1 1 3 2.366 0.785 –0.566 –2.753 –0.861 –2.095

a2 1 3 2.634 0.601 –1.410 –6.860 0.904 2.199

a3 1 3 2.113 0.716 –0.167 –0.813 –1.029 –2.502

Needs (need) n1 1 5 3.817 1.183 –0.855 –4.160 –0.092 –0.225

n2 1 5 3.824 1.093 –0.725 –3.527 –0.184 –0.447

n3 1 5 3.627 1.108 –0.479 –2.331 –0.479 –1.166

Knowledge (know) k1 1 5 3.324 1.089 0.026 0.128 –0.635 –1.545

k2 1 5 3.197 1.245 0.043 0.209 –1.05 –2.553

k3 1 3 1.359 0.737 –0.358 –1.744 –0.626 –1.523

Technology skills (tech) t1 1 5 3.676 1.062 –0.502 –2.443 –0.315 –0.765

t2 1 5 2.514 0.814 –0.006 –0.029 0.67 1.629

t3 1 5 3.690 1.143 –0.545 –2.649 –0.557 –1.356

Telehealth Satisfaction (satis) sa1 1 5 3.204 0.964 0.012 0.057 0.13 0.317

sa2 1 5 3.190 0.960 –0.049 –0.237 0.058 0.142

sa3 1 5 3.394 1.078 –0.015 –0.073 –0.665 –1.618

sa4 1 5 3.380 0.966 –0.014 –0.069 –0.423 –1.028

Telehealth Utilization (use) u1 1 5 3.303 1.197 –0.251 –1.220 –0.801 –1.947

u2 1 5 3.373 1.269 –0.476 –2.316 –0.771 –1.874

u3 1 5 3.430 1.145 –0.239 –1.163 –0.588 –1.429

u4 1 4 2.486 0.797 –0.587 –2.858 –0.491 –1.195

u5 0 1 0.782 0.415 –1.364 –6.635 –0.14 –0.341

Multivariate 40.935 7.847

Valid N 142

Variable (label)	Question	Min	Max	Mean	Std. Dev.	skew	c.r.	kurtosis	c.r.
Access to Telehealth (access)	a1	1	3	2.366	0.785	–0.566	–2.753	–0.861	–2.095
	a2	1	3	2.634	0.601	–1.410	–6.860	0.904	2.199
	a3	1	3	2.113	0.716	–0.167	–0.813	–1.029	–2.502
Needs (need)	n1	1	5	3.817	1.183	–0.855	–4.160	–0.092	–0.225
	n2	1	5	3.824	1.093	–0.725	–3.527	–0.184	–0.447
	n3	1	5	3.627	1.108	–0.479	–2.331	–0.479	–1.166
Knowledge (know)	k1	1	5	3.324	1.089	0.026	0.128	–0.635	–1.545
	k2	1	5	3.197	1.245	0.043	0.209	–1.05	–2.553
	k3	1	3	1.359	0.737	–0.358	–1.744	–0.626	–1.523
Technology skills (tech)	t1	1	5	3.676	1.062	–0.502	–2.443	–0.315	–0.765
	t2	1	5	2.514	0.814	–0.006	–0.029	0.67	1.629
	t3	1	5	3.690	1.143	–0.545	–2.649	–0.557	–1.356
Telehealth Satisfaction (satis)	sa1	1	5	3.204	0.964	0.012	0.057	0.13	0.317
	sa2	1	5	3.190	0.960	–0.049	–0.237	0.058	0.142
	sa3	1	5	3.394	1.078	–0.015	–0.073	–0.665	–1.618
	sa4	1	5	3.380	0.966	–0.014	–0.069	–0.423	–1.028
Telehealth Utilization (use)	u1	1	5	3.303	1.197	–0.251	–1.220	–0.801	–1.947
	u2	1	5	3.373	1.269	–0.476	–2.316	–0.771	–1.874
	u3	1	5	3.430	1.145	–0.239	–1.163	–0.588	–1.429
	u4	1	4	2.486	0.797	–0.587	–2.858	–0.491	–1.195
	u5	0	1	0.782	0.415	–1.364	–6.635	–0.14	–0.341
	Multivariate							40.935	7.847
	Valid N	142

3.4 Variable selection

The dependent variable in this study is the utilization of telehealth services (use), with five exogenous indicators, including the possibility, frequency, convenience, variety, and repeated use of telehealth services. In addition, there are four endogenous variables, access to Telehealth (access), the need for Telehealth (need), knowledge about Telehealth (know), and necessary technical skills to use telehealth services (tech). Finally, the mediator, telehealth satisfaction (satis), refers to the satisfaction with the telehealth system and services, comfort, and ease of use of telehealth services. We hypothesize that access, need, know, and tech could directly affect use or indirectly through satis.

Each variable was measured by three to five indicators, as shown in Table 2. With the four indicators, the intermediate variable, satis, is a mediator with an intervention effect between the four independent variables and the dependent variable, use. Specifically, the endogenous variables (access, need, know, and tech) directly affect use and indirectly through affecting satis. First, individuals’ satisfaction with Telehealth will affect their possibility and frequency of telehealth utilization. Meanwhile, the access, needs, knowledge, and technical skills may also affect telehealth utilization. In addition, access, needs, knowledge, and technical skills can affect individuals’ satisfaction, which, in turn, affects their utilization of telehealth services. As shown in Fig. 1, we sketch the direct and indirect paths from the endogenous variables to the intermediate variable to telehealth utilization in the Structural equation modeling diagram. (See Table 1: Descriptive Statistics and Table 2: Constructs and measurement factors from the Questionnaire)

Table 2
Constructs and measurement factors from the Questionnaire

Variable (label) Question Indicator

Access to Telehealth (access) a1 Do you have a device to use for telehealth services? (For example, smartphone, internet, computer, tablet, etc.) 1 = No, not at all. 2 = No internet. 3 = Yes, I have all resources.

a2 Does your primary care physician offer telehealth services? No = 1, Unsure = 2, Yes = 3.

a3 Are you currently covered by any types of health insurance? No = 0, Some type = 2, Yes = 3.

Needs (need) n1 I need Telehealth services in emergent situations, such as COVID-19. 1–5, from highly disagree to highly agree.

n2 I need Telehealth services regardless of emergent situations such as COVID-19. 1–5, from highly disagree to highly agree.

n3 Telehealth services can replace at least part of my in-person visits. 1–5, from highly disagree to highly agree.

Knowledge (know) k1 I know the purpose of Telehealth services. 1–5, from highly disagree to highly agree.

k2 I understand the advantages and disadvantages of Telehealth services. 1–5, from highly disagree to highly agree.

k3 Can you explain the benefits and recommend Telehealth to another person? No = 1, unsure = 2, Yes = 3.

Technology skills (tech) t1 Would you consider yourself “Tech-Savvy?” 1–5, from highly disagree to highly agree.

t2 I was able to express myself effectively (language skills) over the Telehealth services system are the same as in-person visits. 1–5, from highly disagree to highly agree.

t3 I was able to use the Telehealth services system is the same as in-person visits. 1–5, from highly disagree to highly agree.

Telehealth Satisfaction (satis) sa1 Overall, I am satisfied with this Telehealth system. 1–5, from highly disagree to highly agree.

sa2 How satisfied are you with telehealth services? 1–5, from highly disagree to highly agree.

sa3 What is your personal comfort in using the telehealth system? 1–5, from highly disagree to highly agree.

sa4 Do you feel telehealth visits, and appointments, are easy to use or schedule? 1–5, from highly disagree to highly agree.

Telehealth Utilization (use) u1 How likely do you use telehealth services? (Possibility) 1 = Never, 2 = Unlikely, 3 = No Preference, 4 = Likely, 5 = Very Likely.

u2 How often do you use telehealth services? (Frequency) 1 = Never, 2 = very few, 3 = No Preference, 4 = Somewhat often, 5 = Very often.

u3 I would use telehealth services again. 1–5, from highly disagree to highly agree.

u4 What kind of healthcare have you use via Telehealth? 1 = None, 2 = General Check-Up 3 = General Check-Up and one of the others (include Chronic Illness Check-In, Dental, or Mental Health), 4 = More than two types.

u5 Telehealth service is convenient to use compared with in-person visits. No = 0, Yes = 1.

Variable (label)	Question	Indicator
Access to Telehealth (access)	a1	Do you have a device to use for telehealth services? (For example, smartphone, internet, computer, tablet, etc.) 1 = No, not at all. 2 = No internet. 3 = Yes, I have all resources.
	a2	Does your primary care physician offer telehealth services? No = 1, Unsure = 2, Yes = 3.
	a3	Are you currently covered by any types of health insurance? No = 0, Some type = 2, Yes = 3.
Needs (need)	n1	I need Telehealth services in emergent situations, such as COVID-19. 1–5, from highly disagree to highly agree.
	n2	I need Telehealth services regardless of emergent situations such as COVID-19. 1–5, from highly disagree to highly agree.
	n3	Telehealth services can replace at least part of my in-person visits. 1–5, from highly disagree to highly agree.
Knowledge (know)	k1	I know the purpose of Telehealth services. 1–5, from highly disagree to highly agree.
	k2	I understand the advantages and disadvantages of Telehealth services. 1–5, from highly disagree to highly agree.
	k3	Can you explain the benefits and recommend Telehealth to another person? No = 1, unsure = 2, Yes = 3.
Technology skills (tech)	t1	Would you consider yourself “Tech-Savvy?” 1–5, from highly disagree to highly agree.
	t2	I was able to express myself effectively (language skills) over the Telehealth services system are the same as in-person visits. 1–5, from highly disagree to highly agree.
	t3	I was able to use the Telehealth services system is the same as in-person visits. 1–5, from highly disagree to highly agree.
Telehealth Satisfaction (satis)	sa1	Overall, I am satisfied with this Telehealth system. 1–5, from highly disagree to highly agree.
	sa2	How satisfied are you with telehealth services? 1–5, from highly disagree to highly agree.
	sa3	What is your personal comfort in using the telehealth system? 1–5, from highly disagree to highly agree.
	sa4	Do you feel telehealth visits, and appointments, are easy to use or schedule? 1–5, from highly disagree to highly agree.
Telehealth Utilization (use)	u1	How likely do you use telehealth services? (Possibility) 1 = Never, 2 = Unlikely, 3 = No Preference, 4 = Likely, 5 = Very Likely.
	u2	How often do you use telehealth services? (Frequency) 1 = Never, 2 = very few, 3 = No Preference, 4 = Somewhat often, 5 = Very often.
	u3	I would use telehealth services again. 1–5, from highly disagree to highly agree.
	u4	What kind of healthcare have you use via Telehealth? 1 = None, 2 = General Check-Up 3 = General Check-Up and one of the others (include Chronic Illness Check-In, Dental, or Mental Health), 4 = More than two types.
	u5	Telehealth service is convenient to use compared with in-person visits. No = 0, Yes = 1.

3.5 Empirical method for data analysis

With the theoretical framework based on Self-Determination Theory (SDT) design, we analyzed the data using structural equation modeling (SEM). In addition, various statistical analyses, such as reliability, validity, exploratory, and confirmatory analyses, were undertaken using SmartPLS, SPSS, and AMOS to assess the applicability of the measurement model and scales utilized. To determine the convergent validity of each construct, we performed a factor analysis to ensure the survey items could produce the expected number of factors and that each item loaded on the relevant construct. In addition, using a reliability test, we test the internal consistency of the research instrument. Then, we also check the new model fit based on the criteria. At last, we computed the indirect, direct, and total effects using the bias-corrected bootstrapping method to examine further whether the mediating effect was significant.

3 Empirical results

The results from the SEM estimation for the theoretical model are reported in Fig. 2: Results from the theoretical model based on SDT (after SEM) and Table 3. PLS estimated path coefficients with the standardized SEM coefficients. According to Fig. 2 results, the standardized loading factors in SEM produce the following findings. First, satisfaction has a positive effect on telehealth utilization and is statistically significant (β= 0.363, p < 0.01), which confirms our hypothesis H5 (See Table 3). When individuals feel satisfied with the telehealth services and the system, which may or may not occur during the COVID-19, they are more likely to frequently and repeatedly use telehealth services.

Fig. 2

Results from the conceptual model based on SDT (after SEM).

Table 3

PLS estimated path coefficients with regression weight

Hypothesis		Path Coefficients	P-values	Supported or not
H1i	access ->satis	0.412	0.000	Supported
H1d	access ->use	0.305	0.000	Supported
H2i	need ->satis	0.022	0.778	Not Supported
H2d	need ->use	0.252	0.000	Supported
H3i	know ->satis	0.347	0.000	Supported
H3d	know ->use	0.138	0.036	Supported
H4i	tech ->satis	0.185	0.003	Supported
H4d	tech ->use	–0.050	0.299	Not Supported
H5	satis ->use	0.363	0.000	Supported

Second, access, needs, and knowledge positively affect telehealth utilization and are statistically significant (p < 0.05); tech, on the contrary, has a negative effect on telehealth utilization but is insignificant. (See Fig. 2). According to the results, individuals with access to Telehealth, and those who know more about Telehealth, have higher utilization of telehealth services. However, the evidence does not support individuals with higher technological skills in using more telehealth services. In other words, individuals’ higher skills in technology may have no connection to the utilization of Telehealth. These findings support Hypotheses H1d, H2d, and H3d, but not for H4d (The same information is also reported in Table 3).

As was expected, access (0.412, p < 0.01), know (0.347, p < 0.01), and tech (0.185, p < 0.01) are positively related to telehealth satisfaction, while the effect of need (–0.022, p = 0.765) is insignificant. (Fig. 2). Understandably, the individuals with access, knowledge, and necessary technical skills feel more satisfied with the services. In contrast, people who are in need of telehealth services seem to have no particular feeling about Telehealth. These results support Hypothesis H1i, H3i, and H4i, but not for H2i (The same information is also reported in Table 3).

Overall, while access, need, knowledge, and satisfaction significantly increase telehealth utilization, tech (technology skills) has an insignificant effect on utilization (direct effects). Meanwhile, as a means of competence, the combination of knowledge and technology skills significantly increases satisfaction. At the same time, the autonomy (access) also significantly increase satisfaction, and the relatedness of need insignificantly decreases the satisfaction; in turn, satisfaction significantly increases telehealth utilization (indirect effects).

The component factor analysis and the value of Cronbach’s alpha, composite reliability (CR), and average variance value (AVE) for each indicator in the research models are reported in Table 4. The model fit indices with a sample size of 142 are reported in Table 5. According to Table 4, the composite reliability values (>0.7) indicate that the scale reliability overall with CFA is good and adequate; values for average variance extracted (AVE) suggest an adequate convergent level, with all AVEs > 0.50. In addition, Cronbach’s alpha values (>0.7) also confirm a good and acceptable level of the internal consistency of each group. The model fit results in Table 5 show that the fit measures are good, with the standardized root mean square residual (SRMR), 0.051, falling within the recommended range between 0 and 0.08. The Normed Fit Index (NFI), 0.906, which is closer to 1, indicating a better fit. The squared Euclidean distance, d_ULS and the geodesic distance, d_G are both insignificant, indicating a good model fit. The Bollen-Stine bootstrap results suggest that the model fits better in 5,000 bootstrap samples, and the model is correct with p = 0.04.

Table 4

Component Factor Analysis

Variable (label)	Questions	Factor	CR	AVE	Cronbach’s Alpha
Loadings
Access (access)	a1	0.832	0.823	0.610	0.800
	a2	0.806
	a3	0.698
Needs (need)	n1	0.828	0.880	0.709	0.879
	n2	0.874
	n3	0.823
Knowledge (know)	s1	0.787	0.811	0.589	0.781
	s2	0.788
	s3	0.725
Technology skills (tech)	t1	0.872	0.865	0.684	0.857
	t2	0.682
	t3	0.910
Satisfaction (satis)	sa1	0.835	0.889	0.667	0.888
	sa2	0.838
	sa3	0.820
	sa4	0.773
Utilization (use)	u1	0.846	0.921	0.703	0.896
	u2	0.905
	u3	0.935
	u4	0.786
	u5	0.699

Table 5

Fit indices

	Saturated Model	Estimated Model
SRMR	0.051	0.051
d_ULS	0.390	0.390
d_G	0.449	0.449
Chi-Square	317.305	317.305
NFI	0.906	0.906
Bollen-Stine bootstrap	p = 0.04

Note: N = 142. Bootstrapping sample size = 5,000.

The Standardized indirect effects, direct effects, and total effects are reported in Table 6. According to the results, all indirect effects are significant, except for the model pathway “ Need->Satisfaction ->Utilization.” The total indirect effects, 0.806 (p < 0.05), and total direct effects, 1.77 (p < 0.01), suggest a partial mediation and confirm the significant mediating roles of telehealth satisfaction on telehealth utilization.

Table 6

Standardized indirect effects, direct effects, and total effects

Model pathways	Estimate	Bias-Correlated 95% CI		P-value
		Lower	Upper
Access->Satisfaction ->Utilization	0.450	0.054	1.038	0.034
Need->Satisfaction ->Utilization	–0.068	–0.303	0.037	0.167
Knowledge ->Satisfaction ->Utilization	0.355	–0.026	0.960	0.034
Tech skill ->Satisfaction ->Utilization	0.070	–0.006	0.221	0.074
Total indirect effects	0.806	0.073	1.699	0.044
Total direct effects	1.770	1.271	2.232	0.004
Total effects	2.575	1.884	3.235	0.002

Note: N = 142. Bootstrapping sample size = 5,000.

4 Discussion and conclusion

Using structural equation modeling (SEM) based on the Self-Determination Theory (SDT), this research analyzed the association between satisfaction and telehealth utilization from possibility, frequency, and repeated use during the COVID-19 crisis. This study sought to address gaps in the current literature on this topic. To capture the direct and indirect effects on telehealth utilization, we specifically focus on the relationships among access, needs, knowledge, technology skills, and telehealth satisfaction.

The results suggest that individuals’ satisfaction with Telehealth will significantly increase the utilization of telehealth services. In addition, the access to Telehealth, needs, and knowledge of telehealth services also positively affect telehealth utilization directly. In addition, access to Telehealth and the combination of knowledge and technology skills have synergistic positive effects on satisfaction, and lastly, satisfaction significantly increases telehealth utilization. Despite significant effects on satisfaction, individuals’ advances in technology were shown to have no significant influence on their telehealth utilization.

It was not until COVID-19 restricted the ability of healthcare organizations to see patients face-to-face that providers were forced to place more attention and perhaps even resources on Telehealth. Access to telehealth services was demonstrated to be crucial for satisfaction and utilization, along with adequate resources, insurance, and devices. However, access to Telehealth is still an issue for multiple populations that need it the most. For example, older individuals and individuals who live in rural areas are some of the people that struggle with accessing Telehealth [53]; however, they are also the population where it could be most beneficial to reduce substantial barriers to care. Moreover, lack of health insurance, socioeconomic status, lack of education, and other social determinants often affect access to healthcare [54]. Therefore, developing and implementing more equitable policies and interventions to improve access to telehealth services and reduce related disparities is needed [31]. Further, loosening legislative reimbursement requirements for telehealth services could significantly lift the barrier to Telehealth utilization [55].

In addition to access issues, lack of technology, lack of overall knowledge about Telehealth, and generational ideas are also significant barriers [35]. Clinicians reported families having a lack of available Internet access (62%), a lack of a technology device (58%), and a lack of affordable Internet access (49%) as major barriers [56]. These barriers have become a substantial problem as healthcare organizations cannot serve these specific populations and therefor healthcare organizations are not fulfilling their potential or needed volume of patients.

This research identified primary factors and barriers to implementing telehealth services in the United States. According to our findings, providing telehealth services is an excellent solution to increase accessibility and quality of care. Telehealth services allow patients and other health care professionals to collaborate virtually, thus increasing accessibility. However, it will be necessary to continue building infrastructure to expand access to Telehealth. One way to support the growth of telehealth services is to increase bandwidth for the internet, especially in rural areas.

Besides, knowledge sharing among telehealth providers and increasing the connection or relationship between providers and patients would positively affect telehealth utilization [21, 57]. Furthermore, though knowing about the technology does not always lead to using the technology [40, 58], the knowledgeability of the physicians, teaching, and education to meet the need could increase well-being at work and improve patients’ satisfaction, which could increase telehealth utilization [58, 59]. Meanwhile, management groups may use overarching frameworks rather than industry/work-specific literature to help practitioners effectively meet the current crisis through virtual working modes [60].

There are limitations to this survey-based study, one being the composition of the study sample. The final participant was a convenience sample, and it had an over-representation of more highly educated young people (See Appendix Table 1: Demographic Characteristics of Respondents). Thus, findings from the analysis may be biased, given this characteristic of the study population. To expand on this research and the literature in this area in general, future studies should collect more diverse samples in terms of social and demographic factors. However, despite the limitations, the study expands the existing knowledge base and adds to the availability of literature on Telehealth. The expansion of telehealth utilization has been of exceptional importance during the COVID-19 crisis, and improvements need to be made in how telehealth services are offered in the healthcare industry.

Footnotes

Acknowledgments

The authors have no acknowledgments.

Author contributions

CONCEPTION: Xiaohui You and Erin Seedorf.

METHODOLOGY: Xiaohui You.

DATA COLLECTION: Xiaohui You.

INTERPRETATION OR ANALYSIS OF DATA: Xiaohui You.

PREPARATION OF THE MANUSCRIPT: Erin Seedorf and Xiaohui You.

REVISION FOR IMPORTANT INTELLECTUAL CONTENT: Erin Seedorf and Xiaohui You.

SUPERVISION: Xiaohui You and Erin Seedorf.

Appendix

Table 1

Demographic Characteristics of Respondents

Demographic	Category	Frequency	Percentage
Characteristics
Age	1 = 18–34	62	43.7
	2 = 35–44	36	25.4
	3 = 45–54	24	16.9
	4 = 55–64	14	9.9
	5 = 65+	6	4.2
Gender	1 = male	32	22.5
	2 = female	108	76.1
	3 = other	2	1.4
Race	1 = White	87	61.3
	2 = Black	21	14.8
	3 = Hispanic	21	14.8
	4 = Asian	11	7.7
	5 = Other	2	1.4
Employment	1 = Full Time	21	14.8
	2 = Part Time	37	26.1
	3 = Unemployed	78	54.9
	4 = Retired	6	4.2
Income	1 = Less than $25,000	21	14.8
	2 = $25,000 to $50,000	35	24.6
	3 = $50,000 to $100.000	50	35.2
	4 = $100,000 to $150,000	26	18.3
	5 = $150,000 to $200,000	7	4.9
	6 = More than $200,000	3	2.1
Education	1 = less than high School	3	2.1
	2 = High school	22	15.5
	3 = Two-year College	34	23.9
	4 = Undergraduate	67	47.2
	5 = Graduate	16	11.3
Marriage	1 = Not married	87	61.3
	2 = Married	42	29.6
	3 = Widowed	6	4.2
	4 = Divorced	7	4.9

References

Dorsey

, Topol

. State of telehealth, New England Journal of Medicine2016;375(2):154–61.

Jong

, Mendez

, Jong

. Enhancing access to care in northern rural communities via telehealth, International Journal of Circumpolar Health2019;78(2):1554174.

Snoswell

, Taylor

, Comans

, Smith

, Gray

, Caffery

. Determining if telehealth can reduce health system costs: scoping review, Journal of medical Internet Research2020;22(10):e17298.

Mishori

, Antono

. Telehealth, rural America, and the digital divide, The Journal of Ambulatory Care Management2020;43(4):319–22.

Nagata

. Rapid scale-up of telehealth during the COVID-19 pandemic and implications for subspecialty care in rural areas, The Journal of Rural Health2021;37(1):145–5.

Jayaweera

, Perera

, Gunawardana

, Manatunge

. Transmission of COVID-19 virus by droplets and aerosols: A critical review on the unresolved dichotomy, Environmental Research2020;188, 109819.

Wahezi

, Kohan

, Spektor

, Brancolini

, Emerick

, Fronterhouse

, Luedi

, Colon

, Kitei

, Anitescu

, Goeders

. Telemedicine and current clinical practice trends in the COVID-19 pandemic, Best Practice & Research Clinical Anaesthesiology2021;35(3):307–19.

Mishkind

, Shore

, Bishop

, D’Amato

, Brame

, Thomas

, Schneck

. Rapid conversion to Telemental health services in response to COVID- experiences of two outpatient mental health clinics, Telemedicine and e-Health2021;27(7):778–84.

Karimi

, Lee

, Couture

, Gonzales

, Grigorescu

, Smith

, De Lew

, Sommers

. National survey trends in telehealth use in 2021: disparities in utilization and audio vs. video services.

10.

Ahmed

, Qamar

, Soomro

. Ergonomic work from home and occupational health problems amid COVID-19. Human Systems Management. 2022(Preprint):1-7.

11.

Sahai

, Ciby

, Dominic

. Workplace isolation amongst home-based teleworkers: Can psychological capital make a difference?Human Systems Management2022;41(3):327–39. doi: 10.3233/HSM-211178.

12.

Koonin

, Hoots

, Tsang

, Leroy

, Farris

, Jolly

, Antall

, McCabe

, Zelis

, Tong

, Harris

. Trends in the use oftelehealth during the emergence of the COVID-19pandemic— United States, January–March Morbidityand Mortality Weekly Report2020;69(43):1595.

13.

Young

, Badowski

. Telehealth: increasing access to high quality care by expanding the role of technology in correctional medicine, Journal of Clinical Medicine2017;6(2):20.

14.

Deci

, Ryan

. The general causality orientations scale: Self-determination in personality, Journal of Research in Personality1985;19(2):109–34.

15.

Deci

, Ryan

. A motivational approach to self: Integration in Personality..

16.

Deci

, Ryan

. Self-determination theory: A macrotheory of human motivation, development, and health, Canadian Psychology/Psychologie Canadienne2008;49(3):182.

17.

Sharma

, Kala

, Guleria

, Jaiswal

. IoT-Based Data Management and Systems for Public Healthcare. In Assistive Technology Intervention in Healthcare 2022, pp. 189-224. CRC Press.

18.

Gajarawala

, Pelkowski

. Telehealth benefits and barriers, The Journal for Nurse Practitioners2021;17(2):218–21.

19.

Lin

, Dievler

, Robbins

, Sripipatana

, Quinn

, Nair

. Telehealth in health centers: key adoption factors, barriers, and opportunities, Health Affairs2018;37(12):1967–74.

20.

Gordon

, Solanki

, Bokhour

, Gopal

. “I’m not feeling like I’m part of the conversation” patients’ perspectives on communicating in clinical video telehealth visits, Journal of General Internal Medicine2020;35(6):1751–8.

21.

Ladin

, Porteny

, Perugini

, Gonzales

, Aufort

, Levine

, Wong

, Isakova

, Rifkin

, Gordon

, Rossi

. Perceptions of Telehealth vs In-Person Visits Among Older Adults With Advanced Kidney Disease, Care Partners, and Clinicians, JAMA Network Open2021;4(12):e2137193.

22.

Donovan

, Mog

, Blanchard

, Magid

, Syed

, Kelley

, Palen

, Parsons

, McCall

, Thompson

, Charlton

. Patient experiences with telehealth in sleep medicine: a qualitative evaluation, Journal of Clinical Sleep Medicine2021;17(8):1645–51.

23.

Julie

, Reynolds

, Olbricht

, McGee

. Moving forward with teledermatology: Operational challenges of a hybrid in-person and virtual practice, Clinics in Dermatology2021;39(4):707–9.

24.

Gogia

, Maeder

, Mars

, Hartvigsen

, Basu

, Abbott

. Unintended consequences of tele health and their possible solutions, Yearbook of Medical Informatics2016;25(01):41–6.

25.

New Nationwide Poll Shows an Increased Popularity for Telehealth Services. American Psychiatric Association. (2022, May 25). Retrieved from https://psychiatry.org/news-room/news-releases/new-nationwide-poll-shows-an-increased-popularity.

26.

Ruelos

, Puzzitiello

, Menendez

, Moverman

, Pagani

, Rogerson

, Ryan

, Salzler

. Patient Perceptions of Telehealth Orthopedic Services in the Era of COVID-19 and Beyond, Orthopedics2021;44(5):e668–74.

27.

Powell

, Henstenburg

, Cooper

, Hollander

, Rising

. Patient perceptions of telehealth primary care video visits, The Annals of Family Medicine2017;15(3):225–9.

28.

Polinski

, Barker

, Gagliano

, Sussman

, Brennan

, Shrank

. Patients’ satisfaction with and preference for telehealth visits, Journal of General Internal Medicine2016;31(3):269–75.

29.

Ashwood

, Mehrotra

, Cowling

, Uscher-Pines

. Direct-to-consumer telehealth may increase access to care but does not decrease spending, Health Affairs2017;36(3):485–91.

30.

Harvey

, Valenta

, Simpson

, Lyles

, McElligott

. Utilization of outpatient telehealth services in parity and nonparity states –Telemedicine and e-Health2019;25(2):132–6.

31.

Zhang

, Shi

, Han

, Li

, Jalajel

, Patel

, Chen

, Wen

, Li

, Chen

. Disparities in telehealth utilization during theCOVID-19 pandemic: Findings from a nationally representative survey in the United States. Journal of Telemedicine and Telecare. 2021:1357633X211051677..

32.

Rodriguez

, Saadi

, Schwamm

, Bates

, Samal

. Disparities In Telehealth Use Among California Patients With Limited English Proficiency: Study examines disparities in telehealth use among California patients with limited English proficiency, Health Affairs2021;40(3):487–95.

33.

Thomason

, Bays

, Mantilla

, Huang

, Najjar

, Singh

, Wysham

, Hughes

. Non-English Language Preference AssociatedWith Decreased Rheumatology Telehealth Use During the COVID-19 Pandemic. ACR Open Rheumatology.

34.

Yelverton

, Qiao

, Weissman

, Olatosi

, Li

. Telehealth for HIV Care Services in South Carolina: Utilization, Barriers, and Promotion Strategies During the COVID-19 Pandemic, AIDS and Behavior2021;25(12):3909–21.

35.

Kruse

, Fohn

, Wilson

, Patlan

, Zipp

, Mileski

. Utilization barriers and medical outcomes commensurate with the use of telehealth among older adults: systematic review, JMIR Medical Informatics.e2020;8(8):20359.

36.

Sanders

, Rogers

, Bowen

, Bower

, Hirani

, Cartwright

, Fitzpatrick

, Knapp

, Barlow

, Hendy

, Chrysanthaki

. Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: a qualitative study, BMC Health Services Research2012;12(1):1–2.

37.

Edwards

, Thomas

, Gregory

, Yardley

, O’Cathain

, Montgomery

, Salisbury

. Are people with chronic diseases interested in using telehealth? A cross-sectional postal survey, Journal of medical Internet Research2014;16(5):e3257.

38.

Fischer

, Ray

, Mehrotra

, Bloom

, Uscher-Pines

. Prevalence and characteristics of telehealth utilization in the United States, JAMA Network Open2020;3(10):e2022302.

39.

Yang

, Burke

, Isaacs

, Rios

, Schraml-Block

, Aleman-Tovar

, Tompkins

, Swartz

. Family perspectives toward using telehealth in early intervention, Journal of Developmental and Physical Disabilities2021;33(2):197–216.

40.

Ayatollahi

, Sarabi

, Langarizadeh

. Clinicians’ knowledge and perception of telemedicine technology. Perspectives in Health Information Management. 2015;12(Fall).

41.

Perry

, Gold

, Shearer

. Identifying and addressing mental health providers’ perceived barriers to clinical video telehealth utilization, Journal of Clinical Psychology2020;76(6):1125–34.

42.

Rutledge

, Kott

, Schweickert

, Poston

, Fowler

, Haney

. Telehealth and eHealth in nurse practitioner training: current perspectives, Advances in Medical Education and Practice2017;8:399.

43.

Ahmed

, Qamar

, Soomro

. Ergonomic work from home and occupational health problems amid COVID-19. Human Systems Management. 2022(Preprint): 1-7.

44.

Ryan

, Deci

. Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications; 2017 Feb 14.

45.

Deci

. Effects of externally mediated rewards on intrinsic motivation, Journal of Personality and Social Psychology1971;18(1):105.

46.

Frederick

, Ryan

. Self-determination in sport: A review using cognitive evaluation theory, International Journal of Sport Psychology. Jan1995).

47.

Behzadnia

, Alizadeh

, Haerens

, Aghdasi

. Changes in students’ goal pursuits and motivational regulations toward healthy behaviors during the pandemic: A Self-Determination Theory perspective, Psychology of Sport and Exercise2022;59:102131.

48.

Gillison

, Rouse

, Standage

, Sebire

, Ryan

. A meta-analysis of techniques to promote motivation for health behaviour change from a self-determination theory perspective, Health Psychology Review2019;13(1):110–30.

49.

Ntoumanis

, Ng

, Prestwich

, Quested

, Hancox

, Thøgersen-Ntoumani

, Deci

, Ryan

, Lonsdale

, Williams

, A meta-analysis of self-determination theory-informed interventionstudies in the health domain: effects on motivation, healthbehavior, physical, and psychological health, Health PsychologyReview2021;15(2):214–44 .

50.

Chiu

. Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic, Journal of Research on Technology in Education.S2022;54(sup1):14–30.

51.

Waterschoot

, Morbée

, Vermote

, Brenning

, Flamant

, Vansteenkiste

, Soenens

. Emotion regulation in times of COVID-A person-centered approach based on self-determination theory, Current Psychology2022;19:1–5.

52.

Park

, Kwon

, Jun

, Jung

, Kwon

. Satisfaction survey of patients and medical staff for telephone-based telemedicine during hospital closing due to COVID-19 transmission, Telemedicine and e-Health2021;27(7):724–32.

53.

Kichloo

, Albosta

, Dettloff

, Wani

, El-Amir

, Singh

, Aljadah

, Chakinala

, Kanugula

, Solanki

, Chugh

. Telemedicine, the current COVID-19 pandemic and the future: a narrative review and perspectives moving forward in the USA, Family Medicine and Community Health2020;8(3).

54.

Bergmark

, Sedaghat

. Disparities in health in the United States: an overview of the social determinants of health for otolaryngologists, Laryngoscope Investigative Otolaryngology2017;2(4):187–93.

55.

Lieneck

, Weaver

, Maryon

. Outpatient telehealth implementation in the United States during the COVID-19 global pandemic: a systematic review, Medicina2021;57(5):462.

56.

Campbell

, Goldstein

. Evolution of telehealth technology, evaluations, and therapy: Effects of the COVID-19 pandemic on pediatric speech-language pathology services, American Journal of Speech-Language Pathology2022;31(1):271–86.

57.

Ahmed

, Khan

, Thitivesa

, Siraphatthada

, Phumdara

. Impactof employees engagement and knowledge sharingon organizational performance: Study of HR challenges in COVID-19pandemic, Human Systems Management2020;39(4):589–601.

58.

Turja

, Krutova

, Melin

. Telework and perceived workload: Learnings prior to the COVID-19 upheaval. Human Systems Management. 2022:1-5.

59.

Cloutier

, Cappelli

, Glennie

, Keresztes

. Mental health services for children and youth: a survey of physicians’ knowledge, attitudes and use of telehealth services, Journal of Telemedicine and Telecare2008;14(2):98–101.

60.

Bhargava

. Virtuality and teams: Dealing with crises and catastrophes, Human Systems Management2020;39(4):537–47.