Abstract
In today’s digital era, the generation and sharing of information are rapidly expanding. The increased volume of complex data is big data. YouTube is the primary source of big data. The proliferation of the internet and smart devices has led to a significant increase in content creators on social media platforms, with YouTube being a prominent example. There has been a substantial increase in content creators across various social media platforms, with YouTube emerging as one of the foremost platforms for content generation and sharing. YouTubers face challenges in enhancing content strategies due to the growing number of comments, such as big data on shared videos. Reading and finding viewers’ opinions of such a large amount of data through manual methods is time-consuming and challenging and makes it hard to understand people’s sentiments. To address this, spark-based machine learning algorithms have emerged as a transformative tool for content creators to understand the audience. The Improved Novel Ensemble Method (INEM) algorithm is designed to predict viewers’ sentiments and emotional responses based on the content they interact through the comments. The proposed results provide valuable insights for content creators, helping them refine the strategies to optimize the channel’s revenue and performance. Fit Tuber Channel is analyzed to perform the sentiment of user comments.
Introduction
In recent years, online social media platforms like Facebook (FB), WhatsApp, Twitter, YouTube, Instagram, and Google
To have a popular and well-established content creator, one must be aware of the feedback and sentiments of the targeted customers or individuals who have reacted to it via social media. In today’s technologically advanced world, expressing emotions, feelings, and opinions in any situation is much easier because of the digital world, the internet. Customers’ reactions on social media are open-ended, and they may provide feedback in written language. As a result, what better way to track the performance of a product’s promotion, a famous person, an event, or an organization’s accomplishment than through a social media platform such as YouTube, Facebook, Twitter, WhatsApp, etc. Analyzing the sentiment of a video or image comment is a critical challenge in many applications. Text sentiment analysis can be helpful in a variety of decision-making processes. One such tool analyses the widespread feelings of social media videos based on audience comments. However, social media comments do not adhere to rigid grammar requirements.
The emergence of social media platforms has resulted in a significant surge in data volume, often referred to as big data. This vast and continuously expanding data pool exhibits a multifaceted nature, encompassing structured, semi-structured, and unstructured data. Analyzing such diverse data types presents substantial challenges due to its complexity and rapid growth [2]. Conventional database management systems are ill-equipped to manage the sheer magnitude of data in big data scenarios. In contrast, big data frameworks such as Hadoop and Spark have been specifically designed to address this challenge. They excel in processing massive data volumes with remarkable efficiency, offering low latency, scalability, and robust fault tolerance mechanisms, making them the go-to solutions for handling big data tasks. The Fig. 1 shows the relationship between big data and YouTube.
Characteristics of big data and YouTube.
The proposed work introduces a novel model INEM that generates embedding representations for text data from YouTube comments, aiming to perform multiclass classification by assigning one of three labels to the comments. The model extracts essential features from the input data and uses a Machine Learning algorithm to classify the data into different categories based on these features. This process involves learning from the extracted features and the labels assigned to training data, grouping the data into distinct categories, and applying corresponding labels. To manage the challenge of classifying data into multiple classes, multiclass classifiers are employed for accurate categorization.
The summary of the paper is organized as follows. Section 2 discusses the previous relevant work of sentiment analysis in social media networks. The implemented overall framework structure of the proposed model is introduced in Section 3, which includes data collection, adding sentiment score to the data set, preprocessing, model classification, and evaluation. Performance metrics are reported in Section 4. Experimental setup and Interpretation of results are discussed in Section 5. The conclusion is discussed in Section 6.
Social media comments sentiment analysis
These days, it is imperative to analyze and comprehend the emotions conveyed in posts on social media sites like Facebook, Instagram, and Twitter. Emotional tone analysis of these messages provides insightful information about user perspectives and helps to better understand the general attitude these social networks are expressing. In the study, Al-Qablan et al. [3] investigated various methods for opinion mining, or sentiment and opinion analysis on social media platforms. Furthermore, they compiled and showcased an inventory of accuracy outcomes derived from diverse models employed in previous studies. In essence, they investigated different methods for interpreting and categorizing concepts on social media and summarized the degrees of accuracy these methods achieved in earlier research.
Social networking sites such as Facebook, Instagram, YouTube, etc. There are fewer comments on Facebook than there are on YouTube. Emotional comments are more likely to appear on Facebook and YouTube following a film’s release than before. Moreover, after a movie comes out, the emotional gap between Facebook and YouTube gets less. Hu et al. [4] examine the sentiment of the movie review by Analyzing data from several platforms.
Social support can offer advantages to both the person providing support and the one receiving it. In challenging situations like the COVID-19 pandemic, where stress can result from various internal, external, or environmental factors, social support can serve as a strategy for stress reduction. The pandemic has become a significant stressor impacting nearly every aspect of people’s lives worldwide. Extending help to others is a means of forming connections and developing social aid, and during a period when people felt increasingly isolated, social support became particularly essential. Research conducted by Wenjing Xie et al. [5]. indicated that emotional appeals in messages evolved.
As a YouTube channel expands, it gets harder and harder to find the most popular video. Several factors, including likes, dislikes, shares, and comments determine a video’s popularity on YouTube. Sometimes, it depends mainly on shares and likes. Natural language processing (NLP) techniques are important for assessing user opinions. Guraj et al. [6] study analyses the views left in the comments section of popular YouTube videos to rate channels.
At the time of the election, relevant political content tends to gain popularity, potentially exerting influence on people’s voting decisions. The author examined the sentiments conveyed by YouTube viewers concerning the performances of two Philippine presidential candidates and the content of news and political vloggers’ videos shared on the platform. In the sentiment analysis process, the research [7] collected over 300,201 user comments originating from more than 70 different videos. The machine learning method predicted the sentiment of posted video comments. Supervised machine algorithms determine the sentiment expressed in comments on YouTube videos [8].
Over the past few decades, there has been a more than 25% increase in suicide rates in the United States. Public service announcements (PSAs) are potent tools for promoting health, and social media can play a crucial role in disseminating these PSAs to individuals who are challenging to reach but could benefit from intervention initiatives. Harris [9] analyses the emotions of the suicide video comments [10]. Amid the COVID-19 pandemic, social media content witnessed a surge in popularity as it became a prominent public information source. In this study, a comprehensive analysis was conducted on a dataset comprising over 153,000 tweets [11], aiming to assess the level of popularity attained by these tweets during the pandemic. The research delved into the dynamics of social media engagement and its role in disseminating critical information to the broader audience during these challenging times. Outside the traditional classroom, students often turn to online platforms like YouTube to gain knowledge on various subjects. Identifying the most valuable and effective educational channels involves Analyzing the comments and feedback provided by users [12]. By scrutinizing user comments, students can determine which channels offer the best educational content and tailor the learning experiences to these online resources. These studies [13] presented sentiment analysis of non-English language.
Material and method
System Workflow.
In the study, supervised learning was employed as the most suitable approach for text classification, to analyze the sentiment of a multiclass-labeled dataset’. Supervised text classification relies on predefined categories, including a pre-labeled dataset with accurate values [14]. Figure 2 shows the system workflow. The first stage begins with data collection, in which the necessary data is obtained through an Application Programming Interface (API). The emotion score is not included in the YouTube video comments. The second stage adds the sentiment score to make the data set supervised. In the third stage of the research framework, data is cleaned and transformed into a vectorized format. Unstructured data is converted to machine-readable format during data transformation. The Term Frequency-Inverse Document Frequency (TF-IDF) technique was used to transform the text data. The categorization model is trained using machine learning models in the fourth stage. Finally, the model results are evaluated using several performance metrics. The framework of this research involves data collection, labeling the data set, cleaning the dataset, and transformation. The transformed dataset was then used to train machine learning models. To achieve this, different machine learning models are used, and the model’s performance is evaluated in the validation part.
Collecting vast amounts of data with complicated structures is known as big data. While standard management systems may not enable storing and analyzing such large amounts of data, big data processing solutions do. The few tools for handling big data are Hadoop, Spark, Impala, Sqoop, HBase, and MongoDB [21]. Spark is a distributed processing engine for big data. Hadoop and Spark are big data solutions for storing and analyzing large amounts of data. Spark is much faster than Hadoop. RDD is Resilient Distributed Data. which resides in spark to reduce the main memory access [20].
Data collection
The initial step is the data-gathering process. The collected data comprises comments from YouTube videos and is obtained in real-time using the YouTube API (Application Programming Interface). Due to restrictions on YouTube, downloading comments directly is not possible. Users must first register with the YouTube API to obtain a unique secret key to access the data. The API streamlines the user’s code and reduces the need for creating new code, ultimately minimizing the time required for data retrieval.
Data collection.
The Fit Tuber channel was chosen for the study. The channel reaches millions of people and receives more views and comments to conduct big data analysis. The channel comments are retrieved using a unique key and saved in local storage. Later, it was kept in a CSV file for future examination. Figure 3 contains the statistics of collected comments from the video.
The primary goal is to implement a supervised machine-learning approach. To achieve a supervised machine learning approach, the sentiment label is to be added to the data set. The data set is obtained from the YouTube API. The raw comments lack these sentiment scores. Adding sentiment score is making a data set as the supervised machine learning method. Online tools, software, and experts make the labeled data set. The VADER (Valence Aware Dictionary for Sentiment Reasoning) method assigns a sentiment score to each comment to make the data set a supervised approach. VADER is a popular and straightforward method to analyze the sentiment of user comments [19]. In this proposed work, the VADER technique is used for assigning the sentiment score to the data set.
Text preprocessing
Preprocessing is the foremost step for a machine learning algorithm. The second phase in the proposed framework is preprocessing. It is a crucial step where irrelevant data is eliminated. In the initial preprocessing stages, identifying and removing duplicates and handling missing values to ensure the dataset quality. Further into this stage, particularly in text data processing, actions are performed, including removing stop words, eliminating punctuation, deleting text that matches specific patterns, converting text to a consistent case, lemmatization, and stemming. These actions collectively contribute to the refinement of the data, making it more amenable for subsequent machine learning tasks by enhancing its clarity and reducing unnecessary noise.
Removal of emoji and punctuation
Emoticons often convey valuable information regarding sentence interpretation, especially when expressing emotions. Nevertheless, it’s essential to recognize that not all emojis are informative, and many lack practical relevance. To prepare the dataset for further analysis, irrelevant emojis and symbols, pictographs, transport-related symbols, map symbols, and flags were eliminated.
Punctuation is not comprehended by machine learning models; punctuation marks can introduce text noise. Elements like exclamation marks, apostrophes, and commas are frequently found in unstructured text. A systematic approach was used to detect and replace these punctuation marks with whitespace, employing regular expressions to mitigate this issue.
Stop-word removal, case conversion and lemmatization
Stop words encompass commonly occurring, irrelevant words that lack meaningful significance in sentiment analysis (SA). The stop words are characterized by limited importance and fewer grammatical constraints. Consequently, eliminating these words is essential to diminish the low-level information in the text and emphasize the crucial content. at, of what, who, etc., are examples of stop words. Removing these stop words improves the model’s accuracy. The lemmatization technique is used to create a comparable word to a common word. The primary purpose is to reduce words to the root form.
Feature transformation
Data can vary in format, such as structured, semi-structured, or unstructured. Text data is typically considered unstructured data that are not fixed format and makes it challenging to extract valuable information directly. To comprehend unstructured text, feature extraction techniques are employed to identify significant details, transforming text data into organized data for analysis.
TF-IDF is the most common feature transformation technique to transform and encode the dataset while extracting features. This method is a widely used technique for converting text into feature vectors, generating a machine-learning readable representation of patterns used for training machine algorithms to make predictions [15]. This method assigns the numerical value of each word using the multiplication of two distinct metrics: term frequency and inverse document frequency [19]. The Term Frequency metric quantifies how often a word (
Support vector machine
SVM is a Machine Learning approach used to discover the best boundary for decision-making between variables in a data set. It performs effectively in both small groups of data and complex data. The main goal is to assign the new data point to the proper group. It’s also utilized to solve regression problems [16]. SVM is originally a two-class model designed to discriminate between two classes, and numerous hyperplanes can be used to split these classes. This proposed study involved three different classes positive, negative, and neutral.
Naive base
Naive Bayes classification is a straightforward approach that relies on event probabilities. Despite its simplicity, it performs strongly in various text classification tasks, such as sentiment analysis [17]. This method is rooted in statistical classification and based on Bayes’ Theorem, expressed in Eq. (2).
The Random Forest algorithm is one of the popular algorithms used for classification and regression problems. It constructs decision trees using data samples and makes predictions using decision-making rules with each tree, Final prediction is the merging result of each tree in the forest through a voting mechanism [18]. It’s an ensemble method that outperforms individual decision trees by aggregating results to reduce the risk of overfitting.
Improved novel ensemble method
The novel Ensemble Method combines classic machine-learning approaches to improve prediction accuracy. Using an ensemble methodology like soft voting combines the Support Vector Machine (SVM) and Naive Bayes (NB) algorithms to create a powerful new machine learning algorithm such as the Improved Novel Ensemble method (INEM). The combined algorithm seeks to generate improved predictive capabilities by exploiting the strengths and diverse characteristics of SVM and NB.
Material and method
Precision(P) and Recall(R)
Precision measures the proportion of correctly identified sentiment among the selected videos, while recall measures the proportion of genuinely top-performing sentiment successfully identified by the algorithm. Higher precision indicates high performance, while higher recall suggests that the algorithm accurately identifies a larger portion of the opinion of videos. The precision, Recall, and F1 score equation are given by Eq. (3).
where
The recall equation is given by Eq. (4).
where
The F1 score is the harmonic mean of precision and recall, providing a balanced evaluation of the VA algorithm’s effectiveness. It considers precision and recall, making it a suitable metric for dealing with imbalanced datasets. A higher F1 score indicates a more balanced performance in identifying the sentiment of posted videos on YouTube. The F1 score equation is solved by Eq. (5).
where
Details of fit tuber channel
Details of fit tuber channel
Collected videos and comments
Apache Spark is a distributed parallel processing engine, that can handle massive volumes of data. In this study, the spark framework is utilized to analyze the comments of a social media video. Spark 3.1.2 standalone nodes with two cores and 8 GB RAM were installed. The proposed work uses Fit Tuber YouTube channel data to achieve SA. Tables 1 and 2 contain YouTube7 channels and information about the collected video data. The top video is on fitness-relevant data collected through API.
The Fit Tuber channel was employed in this study. It is a popular YouTube channel that covers physical activity videos. The channel statistic is included in Table 1. It contains information on subscribers, views, and the overall quantity of submitted videos. As of 08-11-2023, the channel had 7.4 million subscribers, 8,764,061 watchers, and 292 uploaded videos. Table 2 describes the collected comments on video details which contain the video caption, video URL, collected date, video comments, and collected comment. Due to the API access limitation the count of gathered comments differed slightly from the actual video comments.
Model comparison
The entire model comparison used in the proposed work is depicted in Table 3. The performance metrics precision, recall, and F1 score are utilized to evaluate the performance of various machine learning algorithms. The F1 score is considered for the final evaluation metrics. it is calculated using precision and recall. The INEM has the highest accuracy, although the other models perform admirably in the data set.
Model Comparison.
In Fig. 4 The Improved Novel Ensemble Method (INEM) classifier was selected, which produced high results compared to alternative classifiers such as Naive Bayes and Random Forest. These additional classifiers were included in the research to validate the results. The INEM has a better F1 score of 89 and the SVM, NB, RF perform well for the data set.
In this research paper, integrating Spark-based big data sentiment analysis into the YouTube revenue optimization framework has proven to be a valuable strategy. It is an automated approach to identify valuable videos by harnessing sentiment analysis of user comments. Big data is in a complex structure. YouTube is the primary source procedure for big data. The main objective is to provide content creators with valuable insights to aid in decision-making processes and revenue optimization using spark-based big data sentiment analysis of social media comments. The methodology involves a comprehensive analysis of YouTube comments gathered through the API. The experimental results show the pivotal role of user sentiments in this context, highlighting the effectiveness of the proposed approach. The proposed INEM method achieved an impressive accuracy rate of 89 percent in identifying the sentiment of YouTube video comments. Additionally, the graphical representation of the results offers content creators actionable insights to enhance the content strategy and improve the performance of the YouTube channel.