Abstract
This study aims to explore patient perceptions and interactions with virtual consultation (VC) systems to understand the factors influencing their adoption and satisfaction. We analyzed 21,839 patient reviews from four major virtual consultation platforms—MDLive, Doctor on Demand, Maple, and HealthTap—collected from publicly accessible sources. Sentiment analysis, word frequency analysis, topic modeling using Latent Dirichlet Allocation (LDA), and association rule mining were used to extract insights. The findings reveal a generally positive sentiment among patients, with recurring themes focusing on app functionality and the important role of doctors in the virtual consultation experience. Virtual consultation systems were found to play a dual role: as a communicator during initial interactions and as a medium facilitating patient-doctor communication. The analysis also identified key doctor-related factors, categorized by the Theory of Planned Behavior, including attitudes (e.g., empathy), subjective norms (e.g., cultural competence), and perceived behavioral control (e.g., time management). The study provides valuable insights for enhancing healthcare system design and improving virtual consultation quality. However, limitations include potential bias in patient reviews, limited platform focus, and the lack of demographic data. Future research should explore advanced machine learning techniques and investigate relationships between different factors to improve virtual healthcare.
Keywords
Introduction
Virtual consultation (VC) refers to a type of telemedicine service that enables patients to receive treatment via digital platforms through video, audio, or text-based communication mode in their own environment, such as at home or work. 1 The rapid proliferation of virtual consultation is challenging the norm of traditional face-to-face doctor visits, offering treatment from the office or home, equal healthcare access in urban and rural areas, and reduced charges for patients and caregivers.2,3 Despite these benefits, patients may still doubt about the diagnosis accuracy of virtual consultation, and the associated financial and time costs compared to traditional doctor visits. 4 Such doubts and negative perceptions can influence patient cognitions, emotions, and decision-makings, 5 ultimately impacting their willingness to adopt virtual consultation systems.
Given this, we were motivated to explore patients’ perceptions and interactions with healthcare systems and doctors within virtual consultation, to help virtual consultation providers improve their service. Patient reviews have emerged as a valuable resource for understanding patient perceptions, preferences and behaviors.6,7 While sentiment analysis and basic text mining techniques are broadly used to analyze patient reviews,8,9 more context-specific analysis with large scale of data are needed to delve deeper into patient experiences with particular healthcare settings, such as virtual consultation.9,10 This paper addresses this gap by investigating: How do patients perceive and engage with healthcare systems in the context of virtual consultation? Through an analysis of 21,839 patient reviews across four prominent virtual consultation systems, this study explores patient-centric interactions, with a focus on the roles of app functionality and doctor involvement.
In the following sections, after the literature review, we begin by describing the methods used for data collection and analysis, including sentiment analysis, word frequency analysis, topic modeling, and association rule mining. The results section presents key findings on patient perceptions of virtual consultation systems, followed by a discussion that interprets these findings in the context of patient-system-doctor communication. Finally, we conclude by summarizing the main contributions of this study, addressing limitations, and suggesting directions for future research.
Literature review
Virtual consultation
Virtual consultation, an integral component of telemedicine, has garnered substantial attention in literature due to its transformative impact on healthcare delivery. The evolving landscape of digital health technologies has propelled the exploration of virtual consultation, encompassing diverse aspects ranging from patient experiences to technological innovations. 11 Some studies focus on the benefits and challenges of virtual consultation. 12 For example, studies by Smith et al. (2022) and Chen et al. (2020) emphasize the positive reception of virtual consultations among patients, citing convenience, accessibility, and reduced wait times as key drivers of satisfaction,13,14 while Greenfield et al. (2021), highlight concerns related to the quality of doctor-patient communication and the need for personalized approaches to address patient preferences 15 .
Some studies focus on the regulation and technology aspects of virtual consultation. Scholars like Vera et al. (2022) and Neves et al. (2021) investigate the regulatory frameworks governing virtual consultations, emphasizing the need for standardized guidelines to ensure patient safety, data security, and ethical considerations.16,17 Research by Mirbabaie et al. (2021) explores the integration of Artificial Intelligence (AI) to enhance diagnostic accuracy, while Pallavicini et al. (2022) delve into the impact of Virtual Reality (VR) in simulating physical examinations during remote consultations.18,19 Other studies focus on not so common aspect of virtual consultation. For instance, Fortney et al. and Hwang et al. highlight the potential of virtual healthcare to overcome geographical barriers and improve healthcare accessibility, particularly in underserved regions.20,21 The ongoing evolution of virtual consultation necessitates continuous research to adapt to emerging challenges and opportunities in the realm of digital health.
Patient-centric design in healthcare systems
In the context of virtual consultation, patient-centric design principles are fundamental for ensuring optimal patient experiences. Greenhalgh et al. (2016) outline principles such as user accessibility, clear communication interfaces, and seamless integration with patients’ daily lives. These principles guide the development of virtual consultation platforms that prioritize the unique needs and preferences of patients. 2 Patient-centric design in virtual consultation systems has a profound impact on patient experience. Studies find that user-friendly interfaces, personalized communication tools, and empathetic design significantly enhance patient satisfaction and engagement.22,23 The research underscores the pivotal role of patient-centric features in shaping positive virtual consultation experiences.
Communication between patients, healthcare systems, and doctors is central to virtual consultations.24,25 Many studies investigate the role of patient-centric design in fostering effective communication. Findings emphasize features such as secure messaging, video quality, and interactive tools that facilitate a meaningful and patient-centered exchange between patients and healthcare professionals.26,27 Understanding patient engagement and perceptions during virtual consultation can enhance patient-centric design of the virtual consultation systems prioritizing the needs and experiences of patients. 28
Looking ahead, the future of patient-centric design in virtual consultation systems involves ongoing innovation. The work of Gupta et al. (2022) envisions the integration of artificial intelligence, predictive analytics, and personalized health insights to further enhance patient experiences during virtual consultations. These advancements position patient-centric virtual consultation design as an evolving and adaptive approach. 29
This study builds on existing literature by focusing on patient reviews as a data source to explore the factors that influence patient satisfaction, trust, and engagement with virtual consultation systems. By applying various text mining techniques, including sentiment analysis, topic modeling, and association rule mining, this study aims to contribute to the understanding of patient-centric factors that affect the adoption and success of virtual consultations.
Methods
Study design
This study used an observational design, relying on patient-generated content from reviews of virtual consultation platforms. The research aimed to explore patient experiences by employing text-mining techniques on the reviews collected from four major virtual consultation platforms. By analyzing sentiment, word frequency, topic modeling, and association rules within the reviews, we aimed to gain insights into the factors that influence patient satisfaction, trust, and engagement with virtual consultation systems.
Data collection
Patient reviews were collected from four prominent virtual consultation platforms: MDLive, Doctor on Demand, Maple, and HealthTap. These platforms were selected due to their wide adoption and large user bases. MDLive serves over 60 million users in the U.S., 30 Doctor on Demand is available to around 98 million people, 31 Maple has handled over 3 million consultations in Canada, 32 and HealthTap has a global user base of tens of millions, with over 140,000 licensed doctors. 33
The reviews were retrieved from publicly accessible sources, including Google Play Store, Apple App Store, and the official websites of the platforms, using Python for data collection. Reviews were retrieved using a PC, although some patients explicitly mentioned using the mobile apps in their feedback. To capture a comprehensive range of patient experiences, reviews from all platform versions (PC, mobile apps, etc.) were included in the analysis. The dataset covers reviews posted between March 25, 2015, and July 27, 2023. In total, 25,763 reviews were collected, and after data cleansing, 21,839 reviews were deemed suitable for analysis.
Data cleansing
To ensure the relevance and quality of the data, reviews were filtered using the following criteria: • Irrelevant Reviews: Reviews that were off-topic or lacked meaningful information (e.g., “I didn’t use this app”) were excluded. • Non-Patient Reviews: Reviews written by healthcare providers (e.g., doctors, nurses) or individuals other than patients were removed to ensure that the dataset represented patient perspectives exclusively. • Duplicate Reviews: Any repeated reviews were eliminated to avoid bias.
Two researchers manually removed irrelevant and non-patient reviews to enhance the data quality and reduce potential bias. This data-cleansing process ensured that only meaningful and patient-relevant reviews were included in the final dataset.
Data analysis techniques
The analysis involved several text-mining techniques, including sentiment analysis, word frequency analysis, topic modeling, and association rule mining.
Sentiment analysis was conducted to determine the emotional tone of reviews as positive or negative using SentiWordNet, a lexicon-based approach chosen for its effectiveness with large text datasets and structured sentiment quantification. 34 A manual review of 50 reviews prior to the analysis showed that SentiWordNet had a higher level of agreement compared to the VADER model, which is another commonly used lexicon-based sentiment analysis approach.
Word frequency analysis involved breaking down patient reviews into individual tokens, removing common English stopwords (e.g., “the,” “is,” “and”), and applying the Snowball stemming technique, which is used to reduce words to their base or root forms for more consistent analysis (e.g., “doctor” and “doctors” were counted together). We also treated consecutive tokens as cohesive units (e.g., “I am”) to better capture patient expressions. Only words with three or more characters were included in the analysis.
Topic modeling was conducted with Latent Dirichlet Allocation (LDA) method to extract topics from the reviews. LDA is a probabilistic generative model that infers underlying topics and their distributions based on the probability of words belonging into a topic. 35 Given the varying lengths of patient reviews and their potential to cover multiple topics, LDA effectively captured these complexities. 36 The optimal number of topics was determined by evaluating perplexity and coherence scores, with nine topics providing the best balance. These topics were used to identify key themes in patient feedback, such as app usability, doctor interactions, and privacy concerns.
At last, association rule mining was conducted to uncover key relationships between frequently mentioned terms in patient reviews. The FP-growth algorithm was used to identify patterns, and association rules were generated based on a minimum support value of 0.1 and a minimum confidence value of 0.7. These rules helped reveal relatively important relationships between patient experiences, such as links between doctor professionalism and patient satisfaction.
Results
Sentiment mining
Sentiment analysis of the 21,839 patient reviews showed 65% were positive and 15% negative. Positive reviews often mentioned terms like “professional,” “helpful,” and “convenient.” Negative reviews highlighted issues like “technical difficulties” and “long wait times.” This provides an overview of patient satisfaction with virtual consultations. Figure 1 illustrates the sentiment analysis results, excluding neutral reviews, along with total annual review counts. The sentiment analysis results of patient reviews.
The figure reveals trends in the adoption of online consultation and changes in sentiment over the years. Before 2019, yearly review counts increased steadily, but the adoption of online consultation was still limited. During the COVID-19 pandemic (2020–2022), there was a surge in reviews, peaking at 6904 in 2020, likely due to the lack of access to traditional healthcare. Post-pandemic, while review counts have decreased, they remain higher than pre-pandemic levels, with 1037 reviews by July 27, 2023.
Both positive and negative reviews followed similar trends over time. Positive reviews consistently outnumbered negative ones, reflecting overall patient satisfaction. However, the rise in negative reviews during the pandemic suggests emerging issues like technical challenges and dissatisfaction with aspects of care. Additionally, the greater drop in positive reviews compared to negative ones post-pandemic implies that patients may have become less tolerant of service issues as restrictions eased.
Word frequency analysis
The word frequency results (top 30).
From Table 1, we observe that words such as “app,” “doctor,” and “great” are the most frequently mentioned terms, each appearing over 8000 times in patient reviews. The word “use” and its variants appeared over 5000 times, followed by “help,” which was mentioned more than 5000 times. Even the least occurring word, “connect,” in the table appeared over 1000 times.
The high frequency of these terms demonstrates the primary focus of patients on app functionality and doctor interactions. Additionally, the frequent use of words like “care,” “quick,” “professional,” and “prescription” indicates recurring themes among patient feedback.
These results provide a basis for further thematic analysis to understand how patients perceive the system and their interaction experiences. By understanding the frequently mentioned aspects, developers and healthcare providers can identify areas that may require improvement to align better with patient needs.
Topic modeling
The LDA clustering results.
The outcomes in Table 2 capture various facets of patient experiences, including app usability, doctor interactions, and privacy concerns. For instance, topic 0 highlights challenges related to app usage, such as scheduling appointments and receiving details. Topic 3 focuses on multimedia features, like video and audio quality, which are critical for effective consultations. Topic 4 covers the app’s role in providing medical information and managing symptoms. Topic 5 indicates that patients consider the app a mean of receiving care at home, potentially reducing the need for physical office visits. Topic 7 discusses payment-related aspects, including app fees, doctor fees, and insurance coverage. Topic 8 addresses concerns related to privacy and data security.
Topics 2 and 6 emphasize doctor-related aspects, including appointments, prescriptions, and positive interactions with doctors. Topic 1 captures overall patient satisfaction, highlighting words like “easy”, “helpful”, and “convenient”, which point to a generally user-friendly experience.
These findings are consistent with previous studies on the complexity of patient experiences with virtual consultation, which range from technical usability to patient-doctor communication.35,37 Our analysis extends these insights by identifying sub-topics such as “video quality”, “appointment scheduling”, and “privacy concerns” that are critical to improving virtual consultation services. Understanding these themes helps developers and healthcare providers prioritize improvements to better align with patient needs and preferences.
Association rule mining
Item sets with support values and association rules with confidence values (top 15).
The support values in Table 3 indicate the frequency of word pair occurrences in patient reviews. Notably, the term “doctor” is prevalent across most itemsets, with support values ranging from 0.19 to 0.45, indicating that “doctor” frequently appears alongside terms like “recommend” and “great”. This underscores the significant role of doctors in shaping patient experiences.
The association rules provide insights into the connections between various aspects of patient experiences. Rules with confidence values below 0.80, such as the link between “professional,” “experience,” and “doctor,” indicate moderate associations, suggesting that professionalism and positive experiences are key factors in shaping patient satisfaction. Rules with confidence values above 0.80 reveal strong associations, including links between doctors and factors like appointment scheduling, prescription management, and consultation speed. These associations emphasize the importance of effective doctor-patient communication and efficient service delivery in enhancing patient satisfaction.
In summary, the association rule mining results highlight the critical role of doctors and their interactions with patients in driving positive experiences. The strong associations identified between terms like “doctor,” “great,” and “recommend” suggest that enhancing doctor-related factors—such as professionalism, communication, and efficiency—can significantly improve patient satisfaction. These findings provide actionable insights for developers and healthcare providers to refine virtual consultation services to better meet patient expectations and preferences.
Discussions
Is healthcare system a communicator or a medium?
The Shannon-Weaver model of communication, developed by Claude Shannon and Warren Weaver in 1949, conceptualizes communication as a process of information transmission.38,39 This theoretical construct can be applied to the virtual consultation process, conceptualizing it as two distinct patient-centric communication periods characterized by different senders and receivers. The initial phase, spanning from the onset of the consultation until the involvement of the doctor, portrays a communication scenario where patients interact with the healthcare system. The second phase, starting with the involvement of the doctor until the end of the consultation, focuses on patient-doctor communication. This framing is substantiated by the findings of word frequency and topic extraction analyses, which revealed the pivotal roles of “app” and “doctor” in patient reviews.
In the initial consultation period, patients log in to the system, seek pertinent doctor information, complete data entry, and schedule an appointment. During this phase, the healthcare system acts as both sender and receiver of information in communication with the patient. In this role, the healthcare system can be analyzed as a communicator, responsible for implementing strategies to reduce uncertainty and foster trust, such as providing comprehensive and clear information, ensuring user-friendly navigation, offering engaging interaction formats, and aligning with patients’ cognitive expectations.40,41
The findings from association rule mining, presented in Table 3, emphasize the term “doctor,” indicating that patient perceptions are notably shaped by interactions with doctors. This highlights the pivotal role of doctors as communicators during the second phase of the consultation, with the healthcare system serving as a medium to support patient-doctor communication. Developers and healthcare providers can focus on enhancing features that support the initial system-patient communication (e.g., seamless information flow, minimizing uncertainty) and optimizing patient-doctor interactions. This patient-centric approach is likely to align more closely with patient expectations and improve overall satisfaction.
What specific doctor-related factors are there?
In the virtual healthcare landscape, our data analysis highlights several doctor-related factors that impact patient experience during virtual consultation. These factors were categorized based on the theory of planned behavior (TPB), developed by Icek Ajzen in 1991, which posits that human behavior is primarily guided by three key components: attitudes, subjective norms, and perceived behavioral control. 42
Doctors’ attitudes—including professionalism, empathy and adaptability to the virtual environment—are crucial in shaping patient experiences during virtual consultation. Unlike physical doctor visits, virtual consultation often lacks non-verbal and non-visual cues, making empathetic communication a significant challenge. Doctors need to demonstrate a higher degree of verbal and visual empathy and active listening to bridge the gap caused by the absence of body language and physical presence. Our findings (association rules 1, 3, 6, and 14, topic 6) highlight that positive interactions with doctors who exhibit empathetic communication and adapt to the patient’s needs lead to higher satisfaction. This aligns with sentiments related to the use of positive words like “helpful,” “understanding,” “professional,” and “friendly” in patient reviews.
Subjective norms in virtual consultation refer to patients’ expectations about how doctors communicate and handle sensitive issues. Word frequency analysis revealed frequent mentions of “communication” and “helpful,” highlighting the importance of effective communication skills. Association rules 7 and 8, which linked terms like “communication” and “appointment” to “doctor,” showed the importance of adaptable communication styles. In virtual settings, communication is more challenging without physical interaction, making it crucial for doctors to adapt their communication styles to ensure clarity and engagement. Topic modeling results suggested that doctors who adjust their communication methods to meet patient needs contribute positively to patient satisfaction.
Cultural competence also emerged as an important factor in the category of subjective norm. Patients expect doctors to understand and respect their cultural backgrounds, especially in diverse virtual environments. This includes doctors’ ability to adjust communication to fit the patient’s cultural context, aligning with broader societal expectations and norms. Our findings also identified privacy as a significant concern, linked to terms like “privacy” and “data.” Association rule mining linked privacy concerns to patient trust, emphasizing the need for strong data security measures. In virtual consultation, patients have heightened concerns about the safety of their personal and medical information. Addressing these concerns with transparent data policies and strong security measures is essential for maintaining patient trust. Doctors should explain data privacy measures proactively, enhancing transparency and comfort for patients, which ultimately improves patient outcomes.
Perceived behavioral control refers to doctors’ ability to manage the consultation effectively, including technical proficiency and time management. Our analysis reveals that a doctor’s ability to handle technical aspects, such as guiding patients on using the system, ensuring video and audio quality, and managing technological disruptions, is crucial for a successful virtual consultation (topics 3 and 5). These are not typically concerns in physical consultations, making them unique to virtual environments. Additionally, time management is critical, as doctors need to effectively navigate virtual queues and waiting times which is expected by patients who seek care remotely (Topics 2 and 3, association rules 4 and 11). Terms like “quick,” “efficient,” and “wait” in patient reviews (as highlighted in the word frequency analysis) indicate that patients value promptness and efficient management of time in virtual consultation.
In summary, to enhance virtual consultation quality, healthcare providers should focus on training doctors not only in clinical skills but also in effective virtual communication, cultural competence, and technical adaptability. This training can bridge the gap created by the virtual medium and ensure that patient expectations are met consistently. Developers should continue improving system features that support doctors in managing consultations effectively, such as streamlined interfaces and integrated technical support tools.
Limitations
Some limitations exist within our data and analysis methods. The dataset consisted of patient reviews, which may introduce bias since patients with extreme experiences are more likely to leave reviews. This bias may result in overrepresenting extreme opinions, potentially skewing sentiment trends and affecting the balance of insights regarding patient satisfaction and concerns.
The devices used for virtual consultation (e.g., mobile vs computer) were not specified in patient reviews, which could have influenced patient experiences and outcomes. Device type can impact the user experience in virtual consultations. For example, mobile users may face different usability challenges, such as smaller screen sizes affecting navigation, or greater connection issues compared to desktop users, which could affect their overall satisfaction and feedback. The focus on four specific systems may limit the generalizability of our findings, while the lack of demographic information restricts our understanding of how different groups experience virtual consultations.
Additionally, the absence of a formal sample size calculation is a notable limitation. While the dataset of 21,839 reviews is considerably larger than most comparable studies - from 4872 to 10,000 reviews36,43,44 – this limitation could impact the generalizability of the findings. Future research should ensure a systematic sample size determination to strengthen the validity and reliability of the insights obtained.
Furthermore, the chosen minimum support threshold in association rule mining may have affected the detection of less frequent patterns, potentially influencing the reliability and inclusiveness of our conclusions. Finally, our use of a frequency-based bag-of-words model presents limitations, as it lacks the ability to capture contextual nuances, handle synonyms effectively, or accurately interpret complex sentiments such as negations.
Conclusions
We conducted sentiment analysis, word frequency, topic modeling, and association rule mining on 21,839 patient reviews from four virtual consultation platforms. This analysis provides valuable insights for enhancing healthcare system design and the quality of virtual consultation.
The findings reveal the dual role of virtual consultation systems as both a communicator and a medium. Initially, the system interacts directly with patients, while later it facilitates patient-doctor communication. Effective communication at both stages is key to reducing uncertainty and building patient trust. Doctor-related factors, categorized using the Theory of Planned Behavior, include attitudes, subjective norms, and perceived behavioral control. Key factors such as professionalism, empathy, technical skills, cultural competence, and time management are crucial for positive patient experiences in virtual consultations.
Based on these findings, developers should prioritize addressing common pain points such as login issues, appointment scheduling, and app interface usability to improve the patient experience. They could also strengthen the features related to patient-doctor interaction – such as video quality and reducing technical issues – can significantly enhance patient satisfaction. System developers can also design specific features for unique user groups, such as the elderly or caregivers to make the system more inclusive and accessible. At last, given the frequent mentions of privacy-related issues, providing better transparency regarding data handling and ensuring secure data management practices are crucial for fostering patient trust.
Future research could utilize advanced machine learning techniques, such as BERT-based models, for more nuanced topic modeling and sentiment analysis. Developing predictive models to forecast patient concerns and conducting correlation studies between different factors, such as sentiment and specific word frequencies, can further strengthen the rigor of future studies. Using metadata, such as examining repeated patient interactions to assess loyalty, would provide deeper insights into patient behavior. These directions will help generate more precise insights, facilitating evidence-based improvements in virtual healthcare systems.
Footnotes
Author contributions
Dr Yuxi Shi was responsible for the conceptualization, study design, data collection, data analysis, interpretation of results, manuscript writing, and final revisions. Dr Sherrie Komiak contributed to data cleansing and assisted with manuscript revisions. Both authors reviewed and approved the final version of the manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article
Prior submissions
This manuscript has not been previously published and has not been submitted as a preprint, conference paper, or thesis.
