Enhancing Digital Wallet Services Based on Electronic Word of Mouth Using Social Media Reviews

User Satisfaction

In the fast-paced and rapidly evolving digital payment landscape, user satisfaction is a critical factor for the success of digital wallets (Ilieva et al., 2023). Oliver (2014) defines satisfaction as the consumer's response to fulfillment. It constitutes an evaluation of whether a product or service feature or the entire product or service, has provided or is providing a satisfactory level of consumption-related fulfillment, which may include both underfulfillment and overfulfillment. User satisfaction depends on the quality of the digital wallet services (Ajina et al., 2023). Users need to experience holistic satisfaction with digital wallets, taking into account factors such as reliability and the availability of desired features (Dhaigude et al., 2023). User experience determines whether users will continue to use a digital wallet or switch to the competitors (Ilieva et al., 2023).

Measuring user satisfaction is important. Ajina et al. (2023) posit that the organization must ensure the quality of the digital wallet service and enhance the specifications and quality of services to guarantee customer satisfaction and loyalty. Digital wallet providers must strive to understand their customers’ needs and preferences to guarantee digital wallet success (Ilieva et al., 2023). Among the various tools for measuring user satisfaction is SERVQUAL.

Service in a business or industry context is mainly intangible and challenging to measure (Al-Laymoun et al., 2019). The evaluation of user satisfaction with a service can be accomplished by employing the SERVQUAL framework, encompassing dimensions like tangibles, reliability, responsiveness, assurance, and empathy, as delineated by Parasuraman et al. (2005). The SERVQUAL approach is widely utilized or adopted for measuring service quality and management in practical business settings and academic research (Gathoni & Van der Walt, 2016). However, the theory can only be applied to conventional services. It cannot be fully applied in technology-based services (Huang et al., 2015). This study measures the level of satisfaction of digital wallet users using factors that have a significant influence on user satisfaction of a technology adopted from previous studies. These factors are reliability (Ganguli & Roy, 2011; Theodosiou et al., 2019), economic benefit (Bagla & Sancheti, 2018; Sutticherchart & Rakthin, 2023), assurance (Kim, 2021) and responsiveness (Ganguli & Roy, 2011; Huang et al., 2015; Theodosiou et al., 2019).

The first factor is Reliability, which refers to the capacity to consistently and accurately deliver the services that have been promised (Parasuraman et al., 1988). The correct and standardized provision of primary services is of utmost importance for users. Safi’i et al. (2019) stated that reliable services play a crucial role in mitigating transaction errors that could adversely impact users in terms of time and financial aspects. Moreover, Kim (2021) added reliability as the complete set of platform services, including the security and privacy of the platform service, can be trusted. The second factor is Economic benefit, which refers to the reciprocal advantages users derive from engaging in transactions facilitated by digital wallet applications (Sutticherchart & Rakthin, 2023). Typically, corporations offer various incentives such as cashback, discounts, rewards, and vouchers to provide advantages to their customers (Bagla & Sancheti, 2018; Sutticherchart & Rakthin, 2023). The third factor is Assurance, which refers to the capacity of a company to instill trust in users by providing secure and seamless application experiences (Kim, 2021). Ensuring the security and seamless functioning of applications should be paramount to the company in order to uphold user data privacy. The fourth factor is Responsiveness, which refers to the capacity to assist users and deliver efficient service promptly (Kim, 2021). This dimension pertains to how employees demonstrate a proactive attitude in assisting users in resolving issues encountered while utilizing digital payment applications. Responsiveness encompasses the ease of locating contact information for the customer service center, prompt responses, and the usefulness of the answers provided by the platform's customer service center (Kim, 2021).

Previous research has identified various benefits of service quality in relation to user satisfaction. Kim (2021) observed a noteworthy influence of service quality on satisfaction and loyalty within the context of content providers. Likewise, Dhaigude et al. (2023) underscores the importance of user satisfaction in fostering loyalty. Maintaining the satisfaction of digital wallet users will lead to an augmented market share, enhanced competitive capabilities, and an opportunity to expand the customer base (Ajina et al., 2023). Moreover, users with satisfaction are likely to continue using digital wallets and when they have a chance to share their experiences with others, which could lead to positive WOM (Sutticherchart & Rakthin, 2023). Al-Okaily (2023) showed that satisfaction and WOM were found to be determinants of loyalty toward digital wallet payment apps.

Electronic Word-of-Mouth

Electronic word of mouth (e-WOM) is defined as the dynamic and ongoing information exchange process between potential, actual, or former consumers regarding a product, service, brand, or company, which is available to a multitude of people and institutions via the Internet (Ismagilova et al., 2017). E-WOM, which often includes online reviews, recommendations and/or opinions, has gained significance with the emergence of innovative technologies such as social media (Al-Okaily, 2023). The main difference between e-WOM and traditional word of mouth (WOM) is that e-WOM communications are spread via the Internet while traditional WOM happens face-to-face. Also, e-WOM communications have a wider reach, stay in public repositories for a long time, and can be easily accessed by other consumers, which, in the case of negative e-WOM, can damage or even ruin the business (Ismagilova et al., 2017).

Understanding e-WOM can contribute to various aspects of business operations, including branding, user acquisitions, product development, and quality control. It gives companies valuable insights into user attitudes and behaviors, which can be leveraged to enhance services and establish a sustainable competitive advantage (Ismagilova et al., 2017; Luo & Xu, 2019). People rely on e-WOM to make various decisions, from gathering information about others' experiences with a specific product or service, sharing their reviews based on their personal experience, and forwarding reviews from others to their relatives (Zamil et al., 2022). When users are satisfied with the provided service, positive e-WOM is likely to emerge; conversely, in the event of user dissatisfaction, negative e-WOM may arise (Chang et al., 2019).

In order to survive in the services industry, service providers must analyze both positive and negative e-WOM. Individuals, in particular, closely observe online reviews detailing others’ experiences, encompassing favorable and unfavorable feedback (Al-Okaily, 2023). Positive e-WOM not only mitigates potential users’ status quo bias but also enhances the likelihood of continuous usage (Talwar et al., 2021). Similarly, positive e-WOM proves beneficial, fostering loyalty to digital wallet applications and prompting service providers to elevate the reputation of these applications (Al-Okaily, 2023). Conversely, negative e-WOM is recognized for its detrimental impact on a firm's customers, including past, current, and potential customers, potentially leading to negative attitudes or even defection from the brand (Talwar et al., 2021). Motivated by negative experiences of a service or company, users tend to disseminate negative e-WOM swiftly, contributing to the spread of negative news within society (Anderson, 1998; Chang et al., 2019). It is imperative to handle such instances properly to avoid any harm to the image of service providers (Singh et al., 2023).

Twitter, a form of e-WOM (Hodeghatta & Sahney, 2016), has emerged as one of the most extensively utilized social media platforms globally (Majumdar & Bose, 2019). Twitter has a global user base of 353 million users in 2023 (Dixon, 2023). Indonesia ranked as the fifth largest Twitter user country, with a total of 24 million users (Statista, 2023). Utilizing social media platforms for marketing has become a pivotal strategic undertaking by global brands (Vidya et al., 2015). In recent decades, numerous researchers have endeavored to extract valuable insights from various social media platforms, focusing on Twitter.

Methodology

The methods applied in this research are Support Vector Machine (SVM) and K-Nearest Neighbors (KNN) to classify the sentiments from the Twitter community regarding digital payment services. Data was collected using the Twitter Application Programming Interface (API) and then analyzed using RapidMiner software, which provides Python Script extensions and Execute operators. The SVM and KNN were used to classify the sentiments of the collected data.

The preference for the SVM algorithm in sentiment analysis is based on several factors. First, SVM effectively handles high-dimensional data, commonly in text data such as social media (Gaye et al., 2021). SVM also performs well in text classification, including sentiment analysis, due to its ability to generalize to unseen data (Yadav & Vishwakarma, 2020). In addition, implementing SVM using kernel functions such as the Radial Basis Function (RBF) allows for efficient training and testing processes, which is especially important in sentiment analysis with large datasets (Ding et al., 2021). Furthermore, another advantage of SVM is its ability to provide clear separation boundaries between classes, which helps understand decision boundaries in sentiment classification and analyze the influence of different features on such classification decisions (Han et al., 2020). Yao et al. (2021) also conducted sentiment analysis in their research, where social network analysis and sentiment analysis were used to investigate knowledge sharing on Twitter as a new attempt to understand the social structure of Twitter networks related to construction safety. Moreover, Song et al. (2022) studied sentiment analysis to understand the social structure of English and French tweets about work safety and sentiment distribution. In addition, testing was conducted using the SVM algorithm to evaluate the accuracy of the developed system. This system is expected to create a more comprehensive understanding of the views of digital payment users on the services they get on social media.

Meanwhile, K-Nearest Neighbors (KNN) is a widely used algorithm in supervised machine learning, primarily employed for classification and regression tasks. The essence of KNN lies in its simplicity and intuitive approach to classifying data points based on their proximity to other data points in the feature space. In KNN, a data point is classified by looking at the “k” number of nearest neighbors from the training dataset and assigning the class that is the most frequent among those neighbors. This process typically employs distance metrics such as Euclidean distance, but other distance measures can also be utilized depending on the specific requirements of the application (Idouglid et al., 2024; Uçar et al., 2021; Wang & Lu, 2021).

One of the defining characteristics of KNN is that it operates without a formal training phase. Instead, it is classified as a “lazy learner,” meaning the computation of classifications occurs at the time of query rather than during the training phase (Qi et al., 2023; Rodriguez-Bazan et al., 2023). This attribute makes KNN particularly appealing in scenarios where rapid classification is required over accumulation of training data, as it eliminates the computational overhead of training (Uçar et al., 2021). Upon receiving a new data point for classification, the algorithm calculates the distance from this point to all other points in the training dataset before determining the class based on majority voting among the nearest neighbors (Idouglid et al., 2024).

The choice of “k,” the number of nearest neighbors to consider, significantly affects the classification’s accuracy and the model’s performance. A small value of “k” may introduce noise into the classification decision, particularly if the dataset exhibits a lot of overlap between classes. Conversely, a larger “k” may smooth out the classifications too much, potentially misclassifying points by diluting the influence of the nearest neighbors. Empirical studies often demonstrate that choosing an optimal ‘k’ through methods such as cross-validation leads to improved model performance (Guo & Sun, 2025).

In sentiment analysis, KNN is appreciated for its simplicity and effectiveness in handling multi-label classifications. This is particularly evident in applications involving social media comments and user reviews, where sentiments may not be distinctly positive or negative but rather nuanced expressions of opinion. KNN can adeptly classify these sentiments through majority voting among the nearest neighbors (Aljameel et al., 2021; Rasheed & Sadiq, 2021). For instance, Rasheed and Sadiq (2021) implemented KNN to analyze a dataset comprising 7,800 social media comments, highlighting how its application resulted in useful insights into public sentiment regarding various services.

The process of this research begins with determining the user review data, reviews from users of the DANA and LinkAja payment applications collected from Twitter. Twitter has been proven to provide accurate and reliable data in sentiment analysis research (Antonakaki et al., 2021). Before data retrieval is carried out, keywords relevant to the research topic are also selected to ensure that the data retrieved follows the research objectives. The data text collected was selected only from users, while those from administrators were not used. The data collected is still semi-structured. Hence, pre-processing must be done first to turn the data into structured data. This stage is essential because it can affect the accuracy of the model created. Pre-processing is done with the help of Kemangi software. The pre-processed data is then used to create a dimensional and sentiment classification model. The classification stage uses the SVM and KNN algorithm. The collected data is divided into training data and test data. Data is classified into four dimensions: reliability, economic benefit, assurance, and responsiveness. After that, classification is carried out to determine the review's sentiment. The analysis compares the sentiment results on each dimension of the two digital payment applications.

Data Collection

The data collected in this research are user reviews of DANA and LinkAja digital payment applications on Twitter, commonly referred to as tweets using the Indonesian language. The data collection process was conducted using web scraping techniques on the Twitter platform using the Twitter API. Web scraping techniques can be an alternative to obtaining reliable data when the required data is challenging to obtain (Luscombe et al., 2022). Twitter provides an API that can be used to access their data officially. Understanding and complying with the rules set out in the Twitter API documentation is essential. This research has acknowledged Twitter’s Terms of Service (TOS), including provisions related to collecting data automatically or using techniques such as web scraping. The statements in the TOS guide the use of web access, determining acceptable or unacceptable methods for collecting information found on the site (Stringam et al., 2023). All data from Twitter is collected using the Spyder program with the Python programming language. Python is beginner-friendly and can help low-level users with its many packages, but it can also lead novices to trust the output too much. Python is very popular with researchers and practitioners who find a need for web scraping (Valente & Marchetti, 2022). Spyder is an integrated development environment (IDE) or program in the Python programming language that was often used to perform web scraping in previous studies (Navlani et al., 2021). The keywords used are @danawallet and @linkaja. The data collection results include a user name, full name, identity number, URL address, text, replies, retweets, and likes. The tweets obtained will be used to measure the satisfaction value of users of the digital wallet applications DANA and LinkAja on the performance and services of the application manager.

Data Pre-Processing

The next stage is conducted after the raw data is collected from the data collection stage. The data change from semi-structured to more structured data is carried out at this stage. The data pre-processing stage aims to remove distractions and inappropriate content. Moreover, this stage is to present the form of the words to reduce their number so that the results in the next step will be more accurate. The data collected through the webscraping technique must be converted into structured data for text-processing applications (Nigam & Biswas, 2021). Cheema et al. (2023) applied the Natural Language Processing (NLP) technique in their research to help customize raw data into structured data. NLP techniques were applied to extract keywords and develop a data repository. NLP can automate tasks that usually require much manual work (Jafari & Far, 2022). However, the results of this stage will provide data ready to be used in the following steps. The data pre-processing stage is carried out manually with the help of Kemangi software version 1.0.0 because of the large amount of data obtained and the Indonesian language used in this study. Dewi et al. (2020) also used this software as a research tool in the data pre-processing stage and showed the best results. The pre-processing stage includes document selection and spelling normalization, which are done manually. Meanwhile, the stages of case folding, cleansing, tokenization filtering, and matrix creation were carried out using Kemangi 1.0.0 software due to the limited capabilities of the software. The document-finding stage is done by sorting the collection of texts obtained. Only tweets that come from users are used.

Similarly, tweets containing administrators' answers and company quiz answers were eliminated. The selected text data is still not standardized because of many abbreviations and non-standardized language. Therefore, an advanced stage is carried out, namely spelling normalization, which is changing the language of the text data obtained with standard and uniform language. The spelling normalization process is essential so the data obtained can be processed further.

However, after the text data has been standardized and uniform, the next step is to perform case folding and data cleansing. This phase converts all text data into lowercase letters so that it is uniform and can be processed, while data cleansing is done to remove unimportant words and symbols such as punctuation marks, hashtags (#), usernames, and others. Furthermore, in the data pre-processing stage, a tokenization process is carried out to break down the text stream into meaningful words so that they can be explored into sentences. Data in the form of words obtained from the tokenization process is then filtered to eliminate words that are not important. The algorithm used in the filtering stage is the stoplist algorithm (discarding less important words) or wordlist (keeping essential words). Meaningless words such as “which,”“and,”“in,”“from,” and so on are removed using the stoplist algorithm. In this study, the stages of the case folding process, data cleansing, tokenization, and filtering use Kemangi Software Version 1.0.0.

The stemming process is advanced in the pre-processing stage, where the process of normalizing the word pieces into their basic words is carried out in this stage. This process also groups words with similar root words and meanings but have different forms or forms because they use different affixes. The data obtained using the previous stages is finally converted into numerical form using the Term Frequency-Inverse Frequency (TF-IDF) weighting method. TF-IDF is used to represent the terms in the data so that the term-specific weights in the document vector can be represented as a vector space model (VSM) (Jiang et al., 2022). TF-IDF is useful for calculating the level of relevance between the input query and the available documents to derive a similarity index, and it can be further used in information retrieval approaches (Thakkar & Chaudhari, 2020). The TF-IDF metric mainly defines terms used in search engines, text summarization, or text classification (Zaware et al., 2021). The matrix formation stage uses RapidMiner 9.6.0 software. Text data is given a weight based on the term frequency of the pieces of words that appear in each tweet.

Furthermore, the text is given weight with the TF-IDF scheme, namely by calculating the frequency of pieces of words that appear in all tweets. Following the weighting with the TF-IDF method, a model is developed to process the data and produce outputs to answer the research objectives. Two models are used at this stage. The first model is used to classify categories or factors in tweets, while the second model is used to determine the sentiment in the tweet based on several SERVQUAL dimensions and additional dimensions, namely reliability, economic benefit, assurance, and responsiveness.

Labeling for Model Development

The labeling is done before developing the classification model. The labeling stage is conducted by manually building a data set given the suitability of dimensional labels. Dimension labeling is done based on keywords that describe the dimension. The first model development labeling is determining the category classification by manually building a set of data to be given the appropriate dimension label. Dimension labeling is done based on keywords that describe the dimension. The keywords used for labeling use terms often used in digital payments in Indonesia because the reviews to be analyzed in this study are Indonesian digital payment applications whose market is the Indonesian people. Labeling for the each dimension are:

The development labeling of the second model, in order to find out the sentiment in the tweet reviews, is done by using keywords that show the sentiment of the data and are used to label the data obtained.

Dimension Category Classification Model Using Tweet Reviews

Classification is done to get a model that can automatically classify large amounts of data. The classification at this stage is carried out to determine the dimensions or categories in the tweets that have been processed. Classification modeling is done with the SVM and KNN classification algorithm.

This research uses k-fold cross-validation to evaluate the classification model with a value of k = 10. Before performing classification, the parameters for the SVM algorithm must be determined. This research uses the Radial Basis Function (RBF) kernel type, so the parameters that need to be determined are the gamma and C values. The gamma value is determined in the interval 0.1 to 0.5 with an increase of 0.1. The C value is determined from 1 to 100 with an increase of 1. Evaluate the model's performance by looking at the accuracy value and the F1 value obtained. F1 is a trade-off between precision and recall. In the classification model for DANA, the gamma value used is 0.2, and the C value used is 3. The accuracy value obtained by the classification model for DANA is 84.24%, and the average F1 is 83.24%. The resulting accuracy value obtained by the classification model for LinkAja is 81.80%, and the average F1 is 80.36%. The resulting model is then used to classify new data that has not been labeled with categories, with 1031 data for DANA and 905 data for LinkAja. DANA customers review the responsiveness dimension more by about 42%, and the reliability dimension is reviewed more by LinkAja customers by about 48% on social media. Figures 1 and 2 show the data results classified into their respective categories for DANA and LinkAja.

OPEN IN VIEWER

Figure 1.

Dimensional proportion data of DANA: (a) Data processed using SVM; (b) Data processed using KNN.

OPEN IN VIEWER

Figure 2.

Dimensional proportion data of LinkAja; (a) Data processed using SVM; (b) Data processed using KNN.

KNN was conducted using 10-fold cross-validation on the same dataset for each method. The optimal k value for KNN is 5, obtained from testing k = 3, 5, and 7. KNN uses Euclidean distance as a distance metric. The classification result for DANA using KNN achieved an accuracy of 78.86% and an average F1-score of 68.6%, while for LinkAja using KNN made an accuracy of 70.28% and an F1-score of 55.84%. DANA customers review the responsiveness dimension more by about 71.87%, and the reliability dimension is reviewed more by LinkAja customers by about 62.87%.

Sentiment Classification Model using Tweets Reviews

The current stage involves conducting a process to obtain the results of data classification in order to determine whether the data falls under positive or negative sentiment. The data processing employed in the classification model process stage is similar, using the Support Vector Machine (SVM) classification algorithm with the help of RapidMiner 9.6.0 software and using k-fold cross-validation to evaluate the classification model with a value of k = 10. This stage uses a Radial Basis Function (RBF) kernel with a gamma value of 0.1 to 0.5 in 0.1 increments and a C value of 1 to 100 in 1 increment. The results of the sentiment classification model for DANA resulted in an accuracy of 88.55% and an average F1 of 88.3% with a gamma value of 0.1 and a C value of 3. Meanwhile, the sentiment classification results for LinkAja resulted in an accuracy of 83.65% and an average F1 of 82.62% with a gamma value of 0.5 and a C value of 3.

The results of the sentiment proportion for DANA using SVM show that neutral sentiment dominates the proportion of around 49.85% of the total tweets, or as many as 514 tweets, while positive sentiment is 309 tweets and 208 tweets for negative sentiment. Furthermore, the proportion results for LinkAja also show that neutral sentiment dominates the classification proportion, around 58.56% of the total tweets or as many as 530 tweets, while positive sentiment is 190 tweets and 185 tweets for negative sentiment.

The results of the sentiment proportion for DANA using KNN show that neutral sentiment dominates the proportion of around 43.94% of the total tweets, or as many as 453 tweets, while positive sentiment is 315 tweets and 263 tweets for negative sentiment. The proportion results for LinkAja also show that neutral sentiment dominates the classification proportion, around 48.29% of the total tweets or as many as 437 tweets, while positive sentiment is 179 tweets and 289 tweets for negative sentiment.

The comparison of the number of positive, neutral and negative sentiments based on tweeter reviews on each dimension can be seen in Table 1.

OPEN IN VIEWER

Table 1.

Comparison Between Negative, Neutral, and Positive Sentiments for DANA and LinkAja.

	DANA			LinkAJa
	Negative	Neutral	Positive	Negative	Neutral	Positive
SVM	208 (20.17%)	514 (49.85%)	309 (29.97%)	185 (20.44%)	530 (58.56%)	190 (20.99%)
KNN	263 (25.51%)	453 (43.94%)	315 (30.55%)	289 (31.93%)	437 (48.29%)	179 (19.87%)

The classification of tweet sentiments—negative, neutral, and positive—into the SERVQUAL dimensions reveals significant insights for both DANA and LinkAja. For DANA, the responsiveness dimension is the most frequently reviewed and discussed by users (Table 2). However, most of the discussions around this dimension tend to highlight problems and receive predominantly negative sentiments (Figure 3). Additionally, the reliability dimension also attracts negative sentiments from users. On the other hand, the economic benefit dimension stands out as the one that receives the most positive sentiments from users.

OPEN IN VIEWER

Table 2.

Negative, Neutral, and Positive Sentiments for DANA.

Dimension	Negative	Neutral	Positive	Source
Responsiveness	194	400	147	SVM
Economic benefit	1	50	154	SVM
Assurance	0	38	7	SVM
Reliability	13	26	1	SVM
Responsiveness	233	354	154	KNN
Economic benefit	1	50	154	KNN
Assurance	2	37	6	KNN
Reliability	27	12	1	KNN

OPEN IN VIEWER

Figure 3.

Difference between positive and negative reviews for DANA.

For LinkAja, the reliability dimension is the most discussed by users (Table 3), many users express dissatisfaction, resulting in significant negative sentiments (Figure 4). Interestingly, contrary to DANA, LinkAja’s responsiveness dimension receives mostly positive sentiments. Furthermore, similar to the findings for DANA, the economic benefit dimension is the one that garners the highest number of positive sentiments from users. This analysis indicates that while both platforms share positive user perceptions regarding economic benefits, their performances in reliability and responsiveness are perceived differently by their users.

OPEN IN VIEWER

Table 3.

Negative, Neutral, and Positive Sentiments for LinkAja.

Dimension	Negative	Neutral	Positive	Source
Responsiveness	28	119	55	SVM
Economic benefit	0	26	74	SVM
Assurance	1	10	2	SVM
Reliability	156	375	59	SVM
Responsiveness	40	107	55	KNN
Economic benefit	4	19	77	KNN
Assurance	5	6	2	KNN
Reliability	240	305	45	KNN

OPEN IN VIEWER

Figure 4.

Difference between positive and negative reviews for LinkAja.