Patient Experience Correlates with Willingness to Retain Service Utilization in Post-Pandemic Times Among Telemedicine Users in Indonesia

Abstract

Background:

COVID-19 pandemic increases the use of telemedicine worldwide. Sustaining its use during post-pandemic times is important to overcome health care disparities, especially in countries with an inadequate number or uneven distribution of health care workers. This study aims to analyze factors associated with the willingness to retain telemedicine utilization after the COVID-19 pandemic.

Methods:

An online survey was administered during the COVID-19 pandemic. Demographic characteristics and patient experience were used as predictors. Chi-square was used to examine the relationship between the outcome variable and the predictors. Finally, binary logistics regression was conducted to determine factors associated with willingness to retain telemedicine utilization after the COVID-19 pandemic.

Results:

A total of 784 responses were included in the analysis. The result showed a high rate of willingness to retain telemedicine utilization (81%). Factors associated with outcome variable were satisfaction (adjusted odds ratio [aOR] 7.893; 95% confidence interval [CI]: 3.600–17.308; p < 0.001), telemedicine efficacy (aOR 1.747; 95% CI: 1.144–2.667; p = 0.010), ease of use (aOR 6.570; 95% CI: 3.029–14.250; p < 0.001), time efficiency (aOR 1.666; 95% CI: 1.092–2.540; p = 0.018), and cost efficiency (aOR 1.852; 95% CI: 1.005–3.411; p = 0.048). In contrast, patients who first used telemedicine during the COVID-19 pandemic were less likely to retain telemedicine utilization (aOR 0.437; 95% CI: 0.281–0.679; p < 0.001).

Conclusions:

The high willingness to retain utilization is a good indication of the sustainability of telemedicine services during post-COVID-19 pandemic. The stakeholders should focus on factors revealed in this study to increase the service uptake.

Introduction

The COVID-19 pandemic, which has caused more than five million deaths globally, has changed the face of health care delivery in most countries. In almost all nations, preventative measures and limitations on medical visits and surgeries were put in place to prevent patients and health care workers from contracting the virus.¹ This situation promoted the advancement of information technology to deliver health care services from a distance. In Indonesia and globally, the use of telemedicine and remote health care delivery has been acknowledged as public health measures to combat the virus.²

In Indonesia, the United States, and Australia, additional regulations were introduced to urge stakeholders to implement telemedicine services during the COVID-19 pandemic.^3
–5 In England, 85% of primary health care was delivered remotely through telephone and video calls, whereas before the pandemic 95% of care was delivered face-to-face.⁶ The accelerated utilization of telemedicine was also found in Canada, Spain, and the United Arab Emirates.¹ This condition is in contrast with the telemedicine uptake before the COVID-19 pandemic, which was very slow.⁶

In the context of Indonesia, telemedicine should not be regarded merely as an emergency response to the COVID-19 pandemic. Indonesia's vast landscape, which consists of more than 17,000 islands, has caused disparities in health services due to transportation issues and uneven distribution of health practitioners.^7,8 About 54.5% of the total number of health professionals in Indonesia are congested in Java and Bali islands, whereas 45.5% of them work in other parts of the country.⁷

Moreover, most of these health professionals are concentrated in health care facilities in urban settings and a few in rural and remote areas, especially specialist physicians.⁹ Therefore, people in remote and rural areas often have to travel a long distance to access health services. This circumstance undeniably causes delays in health care delivery and excessive health expenditures by the patients.⁷

One of the solutions to these obstacles is the utilization of telemedicine since it enables health practitioners to deliver medical consultations to patients from distance. Telemedicine also allows information exchange between health care professionals.⁷ In nonpandemic conditions, telemedicine has been proven to reduce patient queues in health facilities and help to overcome the bottleneck problem in health care delivery.¹⁰

Previous studies have shown that telemedicine has almost the same clinical efficacy as face-to-face consultations, especially for chronic diseases that require periodic examinations. Telemedicine has also proven to be time-efficient and cost-effective, both in clinical costs and travel costs.¹¹ The development of telemedicine in Indonesia is supported by the increasing internet coverage in rural and remote areas. The latest data reported that in 2021, internet connection has penetrated almost 85% of the total villages.¹²

Although telemedicine has been introduced since 1985 in Indonesia, its progress before the COVID-19 pandemic was slow and rarely evaluated.⁷ The pandemic has drastically increased the utilization of telemedicine in Indonesia. The challenge for the Indonesian stakeholders is retaining the interest of telemedicine users when the pandemic is over while promoting the services to nonusers.

Evidence from the previous study demonstrated that future use intention was determined by patients' experience.¹ Listening to patients' perspectives is essential to enable the safe long-term adoption of telemedicine.⁶ In light of the aforementioned, this study aims to analyze factors associated with the willingness to retain telemedicine utilization after the COVID-19 pandemic in Indonesia.

Methods

This is a cross-sectional study that utilized the data from the telemedicine survey by the Ministry of Health of Indonesia at the initial stage of COVID-19 pandemic. Owing to the mobility restriction during the COVID-19 pandemic, the survey was conducted online between July 3 and 17, 2020. The survey link was distributed by using online platforms such as WhatsApp, Facebook, Twitter, and Telegram. The poster and survey link were advertised on the Ministry of Health's official social media pages.

Indonesian citizens aged 18 and above were eligible to participate. Respondents were recruited voluntarily from all 34 provinces of Indonesia. Before participation, they were given a written explanation about the study and signed electronic consent to participate. The collected data were then cleaned from duplication by checking the IP address and the respondents' names. Duplicate and incomplete responses were deleted from the data set.

VARIABLES

The outcome variable of this study was the willingness to retain telemedicine utilization when the COVID-19 pandemic is over. This variable was assessed through the question “will you still use the telemedicine services when the COVID-19 pandemic is over?” The response to this question was dichotomous “yes” or “no.” Code “1” was used for “yes” responses and “0” for “no” responses.

The predictors included in this study were chosen based on the literature and classified into two categories. The first category was participants' demographic characteristics that included gender, age, education, employment, health insurance ownership, monthly income, and distance to the nearest health care facility.^1,13,14 The second category was the patients' experience on telemedicine services, which involved length of experience using telemedicine, overall satisfaction, telemedicine efficacy, patient data safety, medical record safety, ease of use, time efficiency, and cost efficiency.^2,15,16

ANALYSIS

Before the analysis, the data were tested for multicollinearity issues. The Variance Inflation Factor indicated that there was no multicollinearity problem with the data. In the bivariate stage, chi-square was used to determine the relationship between the willingness to retain telemedicine utilization after the COVID-19 pandemic with the predictors. Factors that showed significant relationships were taken into the multivariable logistic regression stage.

Finally, binary logistic regression was used to determine factors associated with willingness to retain telemedicine utilization after the COVID-19 pandemic in Indonesia. The significance for the statistical test was set at p < 0.05. All statistical tests were conducted using IBM SPSS version 25. The visualization of study participant distribution was generated using QGIS version 3.34.1.

ETHICS APPROVAL

The ethical clearance for this study was granted by the National Health Research Ethics Commission, Ministry of Health Republic of Indonesia (KE.416/2020).

Results

The cleaning stage yielded a total of 784 observations for analysis. The distribution of study participants in 34 Indonesian provinces is illustrated in Figure 1. Among all participants, 635 (81%) of them reported that they would like to retain using telemedicine after the COVID-19 pandemic. Whereas only 149 (19%) respondents conveyed that they would not continue using telemedicine after the COVID-19 pandemic.

Fig. 1.

Distribution of study participants in 34 provinces of Indonesia.

Based on the demographic characteristics, the participants were mostly female (75.6%), aged 25–34 years (37.4%), finished education until a bachelor's degree (48.1%), and employed (73.2%). The highest rate of willingness to retain telemedicine utilization was found among participants in the age range of 18–24 (85.5%).

Regarding the length of experience using telemedicine, most of the respondents used telemedicine before the COVID-19 pandemic (76.8%). About 52.5% of participants selected “fair” in overall satisfaction and 50.1% conveyed that telemedicine has similar efficacy as a face-to-face consultation. In terms of data safety, 63.0% of respondents agreed that the patients' data were safely stored and 57.0% agreed that patients' medical records were safely stored.

Furthermore, most respondents conveyed that the telemedicine application was easy to operate (95.3%) and agreed that telemedicine saved time (52.3%). Unlike the other patient experience factors, most participants (79.5%) disagreed that telemedicine was cost-efficient. The participants' demographic characteristics and patient experience factors are presented in Tables 1 and 2.

Table 1.

Participants' Characteristics and Their Relationship with Willingness to Retain Telemedicine Utilization After COVID-19 (N = 784)

DEMOGRAPHIC CHARACTERISTICS	n (%)	WILLINGNESS TO RETAIN TELEMEDICINE SERVICES		p
DEMOGRAPHIC CHARACTERISTICS	n (%)	NO, n (%)	YES, n (%)	p
Gender				0.362
Female	593 (75.6)	117 (19.7)	476 (80.3)
Male	191 (24.4)	32 (16.8)	159 (83.2)
Age				0.068
18–24	145 (18.5)	21 (14.5)	124 (85.5)
25–34	293 (37.4)	53 (18.1)	240 (81.9)
35–44	213 (27.2)	39 (18.3)	174 (81.7)
45–54	100 (12.8)	29 (29.0)	71 (71.0)
>54	33 (4.2)	7 (21.2)	26 (78.8)
Education				0.446
Primary and secondary school	136 (17.3)	25 (18.4)	111 (81.6)
Associate degree	104 (13.3)	16 (15.4)	88 (84.6)
Bachelor's degree	377 (48.1)	80 (21.2)	297 (78.8)
Master's and doctoral degree	167 (21.3)	28 (16.8)	139 (83.2)
Employment				0.823
Unemployed	210 (26.8)	41 (19.5)	169 (80.5)
Employed	574 (73.2)	108 (18.8)	466 (81.2)
Health insurance				0.907
Not insured	65 (8.3)	12 (18.5)	53 (81.5)
Insured	719 (91.7)	137 (19.1)	582 (80.9)
Monthly income (IDR)				0.656
<5,000,000	478 (61.0)	86 (18.0)	392 (82.0)
5,000,000–10,000,000	221 (28.2)	46 (20.8)	175 (79.2)
>10,000,000	85 (10.8)	17 (20.0)	68 (80.0)
Distance to the nearest health care facility (km)				0.654
<1	206 (26.3)	39 (18.9)	167 (81.1)
1–2	263 (33.5)	48 (18.3)	215 (81.7)
3–5	198 (25.3)	43 (21.7)	155 (78.3)
>5	117 (14.9)	19 (16.2)	98 (83.8)

IDR, Indonesian rupiahs.

Table 2.

Telemedicine Patient Experience and Their Relationship with Willingness to Retain Service Utilization After COVID-19 (N = 784)

PATIENT EXPERIENCE	n (%)	WILLINGNESS TO RETAIN TELEMEDICINE SERVICES		p
PATIENT EXPERIENCE	n (%)	NO, n (%)	YES, n (%)	p
Using telemedicine since				<0.001
Before COVID-19 pandemic	602 (76.8)	93 (15.4)	509 (84.6)
During COVID-19 pandemic	182 (23.2)	56 (30.8)	126 (69.2)
Overall satisfaction				<0.001
Unsatisfied	50 (6.4)	23 (46.0)	27 (54.0)
Fair	411 (52.4)	106 (25.8)	305 (74.2)
Satisfied	323 (41.2)	20 (6.2)	303 (93.8)
Efficacy compared to a face-to-face medical consultation				<0.001
Dissimilar	391 (49.9)	99 (25.3)	292 (74.7)
Similar	393 (50.1)	50 (12.7)	343 (87.3)
Patients' personal data safety				<0.001
Unsafe	290 (37.0)	79 (27.2)	211 (72.8)
Safe	494 (63.0)	70 (14.2)	424 (85.8)
Patient's medical records safety				0.001
Unsafe	337 (43.0)	82 (24.3)	255 (75.7)
Safe	447 (57.0)	67 (15.0)	380 (85.0)
Telemedicine application ease of use				<0.001
Uneasy	37 (4.7)	23 (62.2)	14 (37.8)
Easy	747 (95.3)	126 (16.9)	621 (83.1)
Time saving				<0.001
Disagree	374 (47.7)	96 (25.7)	278 (74.3)
Agree	410 (52.3)	53 (12.9)	357 (87.1)
Cheaper service costs				0.001
Disagree	623 (79.5)	133 (21.3)	490 (78.7)
Agree	161 (20.5)	16 (9.9)	145 (90.1)

The highest rate of willingness to retain telemedicine utilization was observed among respondents who stated satisfaction with telemedicine services. Among 41.2% of respondents who conveyed being satisfied with telemedicine, 93.8% of them stated willingness to retain telemedicine utilization after COVID-19. The results of the chi-square test showed that none of the demographic factors were significantly associated with willingness to retain telemedicine services. In contrast, all patient experience factors were shown to have significant relationships with the outcome variable. Thus, these variables were taken into binary logistic regression (Table 3).

Table 3.

Factors Associated with Willingness to Retain Telemedicine Utilization (N = 784)

VARIABLES	aOR	95% CI		p
VARIABLES	aOR	LOWER	UPPER	p
Using telemedicine since
Before the COVID-19 pandemic (Ref.)	—	—	—	—
During COVID-19 pandemic	0.437	0.281	0.679	<0.001
Overall satisfaction
Unsatisfied (Ref.)	—	—	—	—
Fair	1.790	0.929	3.447	0.082
Satisfied	7.893	3.600	17.308	<0.001
Efficacy compared to a face-to-face medical consultation
Dissimilar (Ref.)	—	—	—	—
Similar	1.747	1.144	2.667	0.010
Patients' personal data safety
Unsafe (Ref.)	—	—	—	—
Safe	1.245	0.737	2.101	0.412
Patient's medical records safety
Unsafe (Ref.)	—	—	—	—
Safe	1.031	0.615	1.728	0.908
Telemedicine application ease of use
Uneasy (Ref.)	—	—	—	—
Easy	6.570	3.029	14.250	<0.001
Time saving
Disagree (Ref.)	—	—	—	—
Agree	1.666	1.092	2.540	0.018
Cheaper service costs
Disagree (Ref.)	—	—	—	—
Agree	1.852	1.005	3.411	0.048

aOR, adjusted odds ratio; CI, confidence interval; Ref, reference category.

The results of multivariable logistic regression indicated that satisfaction was the strongest predictor of willingness to retain telemedicine utilization. The other categories that showed significant association with outcome variable were those who agreed that telemedicine had similar efficacy as a face-to-face consultation, respondents who conveyed that telemedicine application was easy to operate, respondents who reported that telemedicine saved time and participants who agreed that the telemedicine was cost-efficient were positively associated with the willingness to retain telemedicine utilization after the COVID-19 pandemic.

Whereas participants who started to use telemedicine during the COVID-19 pandemic were shown to have a significant inverse association with telemedicine utilization retention. Factors associated with willingness to retain telemedicine services are presented in Table 3.

Discussion

This study revealed a high rate of willingness to retain telemedicine utilization (81%) after the COVID-19 pandemic in Indonesia. The enthusiasm of Indonesian to use telemedicine after the COVID-19 pandemic is in line with Indonesia's increasing number of internet users. According to the Indonesia National Statistics Agency, individuals who have access to the internet and possess mobile phones are 98.7%.¹⁷ This finding is essential to inform decision-making by the Indonesian government regarding the upcoming administration of telemedicine services. The increasing telemedicine popularity is an opportunity to reduce the gaps in health care delivery for people in remote areas and regions with sparse health workers.

This study discovered a significant inverse association between willingness to retain telemedicine utilization with respondents who started to use telemedicine during the COVID-19 pandemic. This means people who started to use telemedicine during the COVID-19 pandemic are less likely to use telemedicine after the pandemic. For this group of people, conventional face-to-face consultation was the most preferred mode of health service delivery. In Indonesia, the opportunity to have a face-to-face medical consultation was hindered during the COVID-19 pandemic due to mobility restrictions.

The Indonesian government even instigated a telemedicine-use policy to provide medical consultation for people with COVID-19 symptoms to protect health workers and suppress the spread of the virus. Thus, for those who preferred conventional face-to-face consultation, telemedicine was seen merely as a short-term solution for the pandemic. This finding echoes the result of a previous study that reported a negative correlation between the preference for seeking face-to-face health services and the intention to use telemedicine in the future.¹

In addition, people who used telemedicine before COVID-19 are more likely to have better experience and knowledge of the benefits of telemedicine and thus have a better attitude and intent of use. This result is consistent with the finding of a study among Israeli and Uruguayan that found low intent of future use on participants that first used telemedicine during the COVID-19 pandemic.²

The result of the multivariable logistic regression indicated that user satisfaction had the strongest association with the willingness to use telemedicine in the future. This result confirms the existing argumentation that states patient satisfaction is the most important predictor of future telemedicine utilization among current users.¹⁸ A comparable result was found in an earlier study in the United States that discovered, among patients who reported satisfaction with telemedicine services, 87.2% of them expressed a desire to continue utilizing telemedicine in the future.¹⁹

Similarly, research in Korea, India, and Israel also indicated a high rate of utilization retention among telemedicine users who were satisfied with the service.^20
–22 To enhance user satisfaction, it is advised that both the Indonesian government and health care providers address the existing challenges encountered by telemedicine users in Indonesia, particularly the issues related to expensive and slow internet connections.

Despite the significant growth of telemedicine after the COVID-19 pandemic, individuals in rural areas of Indonesia still face limited access due to inadequate network coverage and connectivity challenges.^9,10 Overcoming these barriers is crucial not only for retaining current telemedicine users but also for encouraging adoption among nonusers.

Moreover, this study found a significant association between respondents who reported that telemedicine has similar efficacy as a face-to-face consultation and the willingness to retain telemedicine utilization. This finding corroborates the result of previous research in the United Arab Emirates that found a relationship between patients' willingness to use and the perceived efficacy of telemedicine.²³

Another international study during the COVID-19 pandemic also indicated that the most influencing factor in the use of telemedicine services was that the service offered similar benefits as a face-to-face consultation and should not necessitate an extra face-to-face consultation in person.^2,6 Telemedicine experts assert that telemedical consultation is equivalent to face-to-face clinical meetings in terms of patient outcomes.^24,25 However, they also agree that it could not fully substitute physical health care delivery, especially for patients with unknown or unstable conditions.²⁶

This study additionally discovered that ease of use was significantly associated with willingness to retain telemedicine utilization. Ease of use has been recognized as an important component of user experience that determines attitude and willingness of retaining technology utilization.²⁷ This finding confirms the results of previous studies in Indonesia and China that showed a positive association between application ease of use and telemedicine utilization.^10,27,28 Therefore, the telemedicine application should be designed simple and easy to operate to retain current users' willingness to use telemedicine services.

The analysis result displayed a positive significant association between willingness to retain telemedicine utilization with participants who reported that telemedicine saved time. Telehealth services have been proven to be time efficient since patients are not required to travel to health care facilities to get medical consultations.²⁹

This finding is in line with the result of a recent study in India, which demonstrated that time efficiency for not traveling a long distance for health care had been the main reason for people in rural areas to utilize telemedicine services.³⁰ Accordingly, a study in New Zealand suggested that participants construed telemedicine as important since it could save them time in getting health care services.³¹

A significant association was also found between willingness to retain telemedicine utilization and participants who conveyed that telemedicine saved their money. Budgetary consideration has been shown as one of the key factors affecting the future prevalence of telemedicine services.³² Recent research in the United States reported a significant cost reduction among telemedicine patients compared with conventional face-to-face health care service.²⁶ Evidence from previous studies showed that telemedicine was proven to save money since patients were not required to travel to health care facilities.^29,33

Consistently, the results of studies in the United States, Saudi Arabia, The Netherlands, and New Zealand revealed a positive association between willingness to use telemedicine and the cheaper cost of telemedicine.^26,33
–35 Nevertheless, the descriptive result (Table 1) suggested that 79.5% of study participants conveyed that telemedicine had not saved them money. Therefore, the telemedicine providers are suggested to reformulate the service charge to increase the uptake.

Although the result of this study showed a good prospect for telemedicine during post-COVID-19 pandemic, telemedicine services that require patients to provide personal data have risen the data safety issue.³⁶ In the past couple of years, at least 277 cyberattacks that resulted in data breaches occurred in Indonesia.³⁷ The result of logistics regression revealed that personal data and medical records safety were not significantly associated with the willingness to retain telemedicine utilization.

This phenomenon portrays inadequate knowledge of the importance of data safety among Indonesian citizens. This finding is in contrast with the result of a previous study in the United States, which showed a positive significant correlation between data safety and intention of using telemedicine.¹³ Although data safety was not the main concern for the study's respondents, medical records and personal data safety remain to be one of the most critical issues in telemedicine services. Therefore, the Indonesian government is expected to formulate regulations to protect patients' data and medical records.

This study has limitations. First, as with the nature of the online survey, selection bias is the most common limitation mentioned since people with poor internet connection are less likely to have taken part in the survey. Second, the absence of stratified sampling has made the result of this study to be less generalizable. Third, reporting bias is also suspected to occur because this study was a self-report survey that examined self-assessed behavior. Apart from its limitations, this study has notable implications at national, regional, and international levels.

At the national level, it has been observed that post-pandemic telemedicine users are highly willing to continue utilizing telemedicine services. This suggests a positive reception of telemedicine and encourages policymakers to invest more in telemedicine infrastructure and integrate it into the health care system. The study also highlights the differing development trajectories of telemedicine between developed and developing nations.

Although similarities exist, certain issues, such as private data security, may be more prominent in developed countries. The findings of this research could benefit nations that share similar characteristics with Indonesia, particularly in Southeast Asia. At the international level, this study contributes to discussions on best practices and challenges in telemedicine adoption, fostering cross-country comparisons and knowledge exchange to enhance global health care sustainability and quality.

Conclusions

The willingness to retain telemedicine utilization in Indonesia was quite high (81%). This is a good indication of the development of telemedicine services in this country. Patients' satisfaction, teleconsultation efficacy, ease of use, time efficiency, cost efficiency, and the time of telemedicine first use were identified as factors associated with willingness to use telemedicine in the future.

Considering the enthusiasm of participants to use telemedicine, the government is advised to formulate regulation that specifically addresses the data security issue. The patients' lack of awareness of the importance of personal data and medical records safety within teleconsultation services is an indication of deficient health and digital literacies.³⁸ Therefore, the government must take part to protect the people from the danger of data misuse.

Increasing the uptake of telemedicine, while maintaining the current users, is also important to reduce the burden in health care facilities. Therefore, it is advised that experts in health communication and decision-makers create educational campaigns outlining the advantages of telemedicine that target those who are hesitant and ambivalent about using telemedicine.

Footnotes

Acknowledgments

The authors thank the Health Policy Agency, Ministry of Health of Indonesia for funding this research and allowing the use of the data for publication.

Authors' Contributions

Conceptualization (lead), data curation (lead), formal analysis (lead), methodology (lead), investigation (lead), visualization (lead), writing—original draft (lead), and writing—review and editing (equal) by D.E.E. Data curation (supporting), formal analysis (supporting), investigation (supporting), writing—original draft (supporting), and writing—review and editing (equal) by I.A. Formal analysis (supporting), investigation (supporting), writing—original draft (supporting), and writing—review and editing (equal) by S.H. and A.P.N. Formal analysis (supporting), writing—original draft (supporting), and writing—review and editing (equal) by Y.F. and E.D.M.

Disclosure Statement

All authors affirm that they have no conflict of interest.

Funding Information

This study was funded by Ministry of Health of Indonesia under grant number HK/02.03/1/1800/2020. No funding was received for the publication.

References

Al Meslamani

, Aldulaymi

, El Sharu

, et al. The patterns and determinants of telemedicine use during the COVID-19 crisis: A nationwide study. J Am Pharm Assoc, 2022; 62(6):1778–1785; doi: 10.1016/j.japh.2022.05.020.

Benis

, Banker

, Pinkasovich

, et al. Reasons for utilizing telemedicine during and after the COVID-19 pandemic: An internet-based international study. J Clin Med, 2021; 10(23):5519; doi: 10.3390/jcm10235519.

Ministry of Health Republic of Indonesia. Preventing the Spread of Covid-19, Health Services Are Delivered Through Telemedicine. 2020. Available from: https://www.kemkes.go.id/article/view/20043000002/cegah-penyebaran-covid-19-pelayanan-kesehatan-dilakukan-melalui-telemedicine.html [Last accessed: November 16, 2022 ].

Taylor

, Caffery

, Gesesew

, et al. How Australian Health Care Services adapted to telehealth during the COVID-19 pandemic: A survey of telehealth professionals. Front Public Health, 2021; 9:648009; doi: 10.3389/fpubh.2021.648009.

Hyder

, Razzak

. Telemedicine in the United States: An introduction for students and residents. J Med Internet Res, 2020; 22(11):e20839; doi: 10.2196/20839.

Neves

, van Dael

, O'Brien

, et al. Use and impact of virtual primary care on quality and safety: The public's perspectives during the COVID-19 pandemic. J Telemed Telecare, 2024; 30(2):393–401; doi: 10.1177/1357633X211066235.

Indria

, Alajlani

, Sf. Fraser H. Clinicians perceptions of a telemedicine system: A mixed method study of Makassar City, Indonesia. BMC Med Inform Decis Mak, 2020; 20(1):233; doi: 10.1186/s12911-020-01234-7.

Effendi

, Handayani

, Nugroho

, et al. Adolescent pregnancy prevention in rural Indonesia: A participatory action research. Rural Remote Health, 2021; 21(3):6639; doi: 10.22605/RRH6639.

Meliala

, Hort

, Trisnantoro

. Addressing the unequal geographic distribution of specialist doctors in Indonesia: The role of the private sector and effectiveness of current regulations. Soc Sci Med, 2013; 82:30–34; doi: 10.1016/j.socscimed.2013.01.029.

10.

Alexandra

, Handayani

, Azzahro

. Indonesian hospital telemedicine acceptance model: The influence of user behavior and technological dimensions. Heliyon, 2021; 7(12):e08599; doi: 10.1016/j.heliyon.2021.e08599.

11.

Garattini

, Badinella Martini

, Zanetti

. More room for telemedicine after COVID-19: Lessons for primary care?. Eur J Health Econ, 2021; 22(2):183–186; doi: 10.1007/s10198-020-01248-y.

12.

Statistics Indonesia. Number of Villages Having Base Transceiver Station (BTS) Tower by Province and Urban Rural Classification (Villages), 2019–2021. 2022. Available from: https://www.bps.go.id/indicator/2/1677/1/-banyaknya-desa-kelurahan-yang-memiliki-menara-base-transceiver-station-bts-menurut-provinsi-dan-klasifikasi-daerah.html [Last accessed: November 17, 2022 ].

13.

Kato-Lin

Y-C

, Thelen

. Privacy concerns and continued use intention of telemedicine during COVID-19. Telemed J E Health, 2022; 28(10):1440–1448; doi: 10.1089/tmj.2021.0603.

14.

Andersen

, Felix

, Su

, et al. Factors associated with Arkansans' first use of telehealth during the COVID-19 pandemic. Int J Telemed Appl, 2022; 2022:5953027; doi: 10.1155/2022/5953027.

15.

Serwe

, Hersch

, Pancheri

. Feasibility of using telehealth to deliver the “powerful tools for caregivers” program. Int J Telerehabilitation, 2017; 9(1):15–22; doi: 10.5195/ijt.2017.6214.

16.

Parmanto

, Lewis

. AN, Graham KM, et al. Development of the telehealth usability questionnaire (TUQ). Int J Telerehabilitation, 2016; 8(1):3–10. doi: 10.5195/ijt.2016.6196.

17.

Statistics Indonesia. Telecommunication Statistics in Indonesia 2021, 1st ed. Badan Pusat Statistik: Jakarta, Indonesia; 2022.

18.

Thomas

, Haydon

, Mehrotra

, et al. Building on the momentum: Sustaining telehealth beyond COVID-19. J Telemed Telecare, 2022; 28(4):301–308; doi: 10.1177/1357633X20960638.

19.

Ngo

. A survey of patient satisfaction with telemedicine during the COVID-19 pandemic at a student-run free clinic. Free Clin Res Collect, 2020; 6(1):1–5.

20.

Park

, Kim

Y-M

, Yi

, et al. Coronavirus disease outbreak in call center, South Korea. Emerg Infect Dis, 2020; 26(8):1666–1670; doi: 10.3201/eid2608.201274.

21.

Reicher

, Sela

, Toren

. Using telemedicine during the COVID-19 pandemic: Attitudes of adult health care consumers in Israel. Front Public Health, 2021; 9:653553; doi: 10.3389/fpubh.2021.653553.

22.

Priyadarshani

, Purwar

, Pipal

, et al. Patient satisfaction with telemedicine services in obstetrics and gynecology during the COVID-19 pandemic. J South Asian Fed Obstet Gynaecol, 2022; 13(6):382–386; doi: 10.5005/jp-journals-10006-1974.

23.

Taha

, Shehadeh

, Alshehhi

, et al. The integration of mHealth technologies in telemedicine during the COVID-19 era: A cross-sectional study. Ravi R. ed. PLoS One, 2022; 17(2):e0264436; doi: 10.1371/journal.pone.0264436.

24.

Buvik

, Bugge

, Knutsen

, et al. Patient reported outcomes with remote orthopaedic consultations by telemedicine: A randomised controlled trial. J Telemed Telecare, 2019; 25(8):451–459; doi: 10.1177/1357633X18783921.

25.

Said

, Ngo

, Hwang

, et al. Navigating telemedicine for facial trauma during the COVID-19 pandemic. Laryngoscope Investig Otolaryngol, 2020; 5(4):649–656; doi: 10.1002/lio2.428.

26.

Langabeer

, Champagne-Langabeer

, Alqusairi

, et al. Cost-benefit analysis of telehealth in pre-hospital care. J Telemed Telecare, 2017; 23(8):747–751; doi: 10.1177/1357633X16680541.

27.

Manda

, Salim

. Analysis of the influence of perceived usefulness, perceived ease of use and attitude toward using technology on actual to use Halodoc application using the technology acceptance model (TAM) method approach. Int Res J Adv Eng Sci, 2021; 6(1):135–140.

28.

Zhou

, Zhao

, Kong

, et al. Factors influencing behavior intentions to telehealth by Chinese elderly: An extended TAM model. Int J Med Inf, 2019; 126:118–127; doi: 10.1016/j.ijmedinf.2019.04.001.

29.

Bokolo

. Exploring the adoption of telemedicine and virtual software for care of outpatients during and after COVID-19 pandemic. Ir J Med Sci, 2021; 190(1):1–10; doi: 10.1007/s11845-020-02299-z.

30.

Acharya

, Rai

. Evaluation of patient and doctor perception toward the use of telemedicine in Apollo Tele Health Services, India. J Fam Med Prim Care, 2016; 5(4):798–803; doi: 10.4103/2249-4863.201174.

31.

Imlach

, McKinlay

, Middleton

, et al. Telehealth consultations in general practice during a pandemic lockdown: Survey and interviews on patient experiences and preferences. BMC Fam Pract, 2020; 21(1):269; doi: 10.1186/s12875-020-01336-1.

32.

Uei

S-L

, Kuo

Y-M

, Tsai

C-H

, et al. An exploration of intent to use telehealth at home for patients with chronic diseases. Int J Environ Res Public Health, 2017; 14(12):1544; doi: 10.3390/ijerph14121544.

33.

Abdulwahab

, Zedan

. Factors affecting patient perceptions and satisfaction with telemedicine in outpatient clinics. J Patient Exp, 2021; 8:237437352110637; doi: 10.1177/23743735211063780.

34.

Serper

, Cubell

, Deleener

, et al. Telemedicine in liver disease and beyond: Can the COVID-19 crisis lead to action?. Hepatology, 2020; 72(2):723–728; doi: 10.1002/hep.31276.

35.

de Veer

AJE

, Peeters

, Brabers

, et al. Determinants of the intention to use e-Health by community dwelling older people. BMC Health Serv Res, 2015; 15(1):103; doi: 10.1186/s12913-015-0765-8.

36.

Mangku

DGS

, Yuliartini

NPR

, Suastika

, et al. The personal data protection of internet users in Indonesia. J Southwest Jiaotong Univ, 2021; 56(1):23; doi: 10.35741/issn.0258-2724.56.1.23.

37.

Sudarwanto

, Kharisma

DBB

. Comparative study of personal data protection regulations in Indonesia, Hong Kong and Malaysia. J Financ Crime, 2021; 29(4):1443–1457; doi: 10.1108/JFC-09-2021-0193/FULL/PDF.

38.

Effendi

, Laksono

, Pranata

, et al. Prevalence and factors associated with belief in COVID-19 vaccine efficacy in Indonesia: A cross-sectional study. Asian Pac J Trop Med, 2022; 15(7):308–313; doi: 10.4103/1995-7645.351769.