Bigdata clustering and classification with improved fuzzy based deep architecture under MapReduce framework

Abstract

The current state of economic, social ideas, and the advancement of cutting-edge technology are determined by the primary subjects of the contemporary information era, big data. People are immersed in a world of information, guided by the abundance of data that penetrates every element of their surroundings. Smart gadgets, the IoT, and other technologies are responsible for the data’s explosive expansion. Organisations have struggled to store data effectively throughout the past few decades. This disadvantage is related to outdated, expensive, and inadequately large storage technology. In the meanwhile, large data demands innovative storage techniques supported by strong technology. This paper proposes the bigdata clustering and classification model with improved fuzzy-based Deep Architecture under the Map Reduce framework. At first, the pre-processing phase involves data partitioning from the big dataset utilizing an improved C-Means clustering procedure. The pre-processed big data is then handled by the Map Reduce framework, which involves the mapper and reducer phases. In the mapper phase. Data normalization takes place, followed by the feature fusion approach that combines the extracted features like entropy-based features and correlation-based features. In the reduction phase, all the mappers are combined to produce an acceptable feature. Finally, a deep hybrid model, which is the combination of a DCNN and Bi-GRU is used for the classification process. The Improved score level fusion procedure is used in this case to obtain the final classification result. Moreover, the analysis of the proposed work has proved to be efficient in terms of classification accuracy, precision, recall, FNR, FPR, and other performance metrics.

Keywords

Map reduce framework improved feature fusion normalization fuzzy C-means clustering

1. Introduction

The availability of data increases every day due to recent technical developments. Websites of social networking and networks of sensors produce massive amounts of data [1, 2]. In other words, big data are generated quickly from several sources and formats. Big data is currently a significant research field as it is created quickly, making it challenging to store, process, or manage them using conventional software. Technologies for big data are equipment that can store valuable data in different formats [3, 4]. Several analytical systems are being made accessible to assist users in analysing complicated data, both structured and unstructured, so as to satisfy users’ needs to analyse and store complicated information. To obtain information within big data, many suggestions and designs for applications, models, methods, equipment, and algorithms are being made [5, 6]. Keeping accurate and dependable results for big data is the major objective of these technological advancements. Modern technology is additionally needed for big data to be effectively stored and handled within a constrained length of time [7].

Especially, the most effective meta-learning methods for proper analysis of the vast amounts of data produced by contemporary devices include clustering techniques. The fundamental objective of clustering is to organize points of information into clusters so that they are comparable in terms of certain parameters. There have been many efforts to evaluate and group the data in the field of clustering in many applications. However, the main problem with algorithmic clustering is that it frequently demands a lot of computer power, particularly when processing big data. Resources for interactive analysis, applications for processing streams, and resources for batch processing belong to three separate categories of big data systems [8]. Data processing in interactive settings and real-time information interaction require interactive data analysis technologies [9].

Also, the most significant and frequently utilized problem in ML is big data classification, whose goal is to develop a strategy for classifying data into a number of already existing groups based on a collection of learning sets. The SVM is one of the most intriguing approaches in ML for categorization, and it has been effectively used in many fields of science and engineering [10]. Numerous machine learning techniques have faced difficulties as a result of the introduction of large data [11]. Additionally, the two types of fundamental deep learning models include generative models and discriminative models. RBN, AE and DBN models are the key components of the conventional type and are frequently used to convey as bigdata categorization [12]. Order correlation or joint statistical distribution characterizing data, which mostly includes CNN, RNN, DSN, and long short-term memory network models to categorize the internal structure of the information or characterize the subsequent probability of the data [13, 14].

Though, one of the essential elements of safe information clustering is the safety of data throughout the clustering of multiple dimensions of big data, particularly unstructured and uncertain large data. Loss of information can happen during transmission, storage, and clustering itself owing to a variety of data mistakes and security assaults [15, 16, 17, 18]. To overcome this problem, there needs efficient data classification methods. Accordingly, this paper proposes a bigdata classification model and the main contribution of this work is brief as below:

•
Proposing an improved fuzzy C-means clustering algorithm for data partitioning, which enhances the classification performance.
•
Map Reduce framework is used to handle the bigdata, where the mapper phase handles the feature extraction and fusion process. Introducing an improved feature fusion process which depends on the weight average fusion method that fuses the entropy-based features and correlation-based features.
•
Recommending an improved score level fusion method obtaining accurate classification results by combining the results from DCNN and Bi-GRU models.

The remaining of this proposed work is organized as follows: The literature review and the problem statement are mentioned in Section 2. The proposed big data classification and classification model is explained in Section 3. Section 4 explains about the Results and Discussions and Section 5 represents the conclusion of the classification model.
2. Literature review

In 2023, Shuaiyin Ma et al., [19] stated with the development of cutting-edge technologies for information and communication, manufacturing information from the IoT achieved the size of big data in Industry 4.0. Big data presents a challenge to conventional clustering and correlational evaluation due to its size and low-value density. To address this issue, clustering based on a big data-driven correlation analysis was suggested in this research as a way to increase the effectiveness of resource and energy use. The industrial units with unusually high energy consumption were categorized in-depth using clustering analysis. The interaction between energy supply and demand is then balanced by correlational evaluation, which could decrease carbon emissions and improve sustainable competitiveness. Results of the sensitivity study indicate that, in comparison to the original analysis system, the technique for extracting features could increase the accuracy of the correlation evaluation.

In 2022, Abhilasha, A et al., [20] focused on Big data, which was crucial for information processing training, manipulation, and predicting. The training and extraction of information from these big datasets could lead to limited categorization outcomes and incorrect conclusions as a result of the imbalanced data supply. Conventional ML classifiers were able to handle the unbalanced datasets, but they still had issues with overfitting, cost of training, and specimen classification difficulty. The investigation effort offered a novel “Self-Boosted along with Dynamic Semi-Supervised Segmentation Method” to predict improved categorization. This approach was first preprocessed by building data segments using the Hybrid Related closest neighbour algorithm. After preparing the data, big data categorization necessitates large amounts of storage, which raises the cost of training. Thus, a heterogeneous weight ensemble classifier was proposed to compute the information space in every sample data by using closest neighbors and to address the training cost with problems in unstable samples depending on adaptive weight modification. Then the categorization performs poorly as a result of sample hardness. As a result, the project created a Self Managed Ensample Optimizer that improves classification results by sorting the specimen majority into bins based on their stiffness values. To provide balanced data with superior classification outcomes, the suggested model successfully categorized the unbalanced dataset with 99% of high accuracy.

In 2022, Dongwei Li et al., [21] stated that cloud computing was one of the most intriguing options because clustering Bigdata frequently takes enormous processing power. However, if it was not effectively managed, the cost of computation in the clouds could be surprisingly large. In the big data clustering field, the long tail effect was widely noticed, which suggests that a significant amount of time was frequently spent in the middle to late phases of grouping. To attain a sufficiently suitable precision at a minimal computing cost, the authors of this study attempt to reduce the unneeded long tail in the clustering process. A fresh strategy was put forth to achieve big data clustering on the cloud that was affordable. The k-means and EM algorithms were commonly used by the authors at the earliest stage when the necessary accuracy was achieved by learning the model of regression with the samples. The outcomes of tests on four well-known data sets show that both the k-means and EM algorithms might achieve great affordability in the cloud using our suggested methodology. In their experiments using the far more effective k-means method, for instance, the authors discovered that, in comparison to the less effective EM approach, reaching a 99% accuracy only requires 47.71% to 71.14% of the calculation effort needed to achieve a 100% accuracy.

In 2021, Chitrakant Banchhor and Srinivasu, N [22] proposed the efficient administration of storage and processing of a very high volume of data is referred to as the “big data handling” method. Both organized and unstructured data might be handled overall using a certain strategy. This study examines the CNB classifier, CGCNB, FCNB, and HCNB classifiers. These classifiers operate as intended since they depend on the Bayesian concept. The CNB was created by converting the normal naive Bayes classifier into an underlying hypothesis by applying correlation between the attributes. The CNB classifier’s efficiency was significantly improved by integrating the cuckoo searches and grey wolf optimization methods. The final classifier was referred to as a CGCNB. Additionally, the accuracy, specificity, sensitivity, recall, and time required for execution of the FCNB and HCNB classifications were taken into account while analyzing their performance with CNB and CGCNB.

In 2022, Jayasri N.P and R. Aruna [23] concentrated on Big data which was the collection of Big data from numerous sources, including sensor data, social media, online transaction information, etc. It was difficult to assess the relevance of such a large collection of created data using traditional processing methods. It was necessary to evaluate a large amount of noisy data to obtain important information due to the development and forthcoming trends in the health care field. This study’s objective in the field of healthcare was to assess the diabetic patients’ medical dataset by using a combination of cutting-edge modular decision attention networks, AR, and artificial intelligence through the MapReduce framework. In a MapReduce context, the association-based apriori algorithm takes into account health data to produce regulations. This analysis was conducted using a 50-variable diabetic machine-learning dataset from UCI. The proposed algorithm’s output was provided by metrics like accuracy, precision, recollection, and the F-score.

In 2021, Mario Juez-Gil et al., [24] stated that the Big Data used to describe ever-increasing datasets that are rising in both quantity and complexity at a rate never previously witnessed. The abundance of samples for the various classes could not be accurately balanced, which is a common difficulty for categorization, especially with Big Data. Due to this bias in preference for the majority class and disregard for the minority one, imbalanced categorization was first established several decades earlier. Although there were already more imbalanced classification algorithms than ever before, they still concentrated on small datasets rather than the current state of big data. This study uses two well-known ensemble families (Bagging and Boosting) to conduct extensive experiments with combined classifiers in the setting of unbalanced big data categorization. Moreover, Statistical experimental results show that the most recent was the Bayesian technique, which was compared with ensemble performance and time to execution throughout all of the research, which was conducted in the Spark cluster. One especially interesting finding of the study was the fact that when using Big Data to analyze unbalanced datasets, simpler strategies outperformed more complicated ones. Some advanced techniques that appear to be required to process and lessen imbalance in normal-sized databases failed as well for imbalanced Big Data due to their added complexity.

In 2022, Noha Mostafa et al., [25] stated the primary component of the Energy Internet was thought to be the use of large amounts of data in the energy industry. Particularly with the incorporation of energy from renewable sources and smart grids, significant and promising problems exist. The advantages and difficulties of applying big data analytics for power plants that produce renewable energy were discussed in this paper. The capability to gather the information and use it appropriately for improved decision-making is a significant element. For the possible use of big data processing for electric grids and environmentally friendly power utilities, a framework was created. The prediction of the smart grid’s stability using multiple ML techniques was suggested as a five-step process. Through the use of three different machine learning techniques, the long-term viability of the model was forecast by using the data collection from a distributed smart grid data system. According to the outcomes of training the weighted linear regression model was obtained 96% accuracy for 70% training data. Likewise, the decision tree model and randomly generated forest tree model of accuracy were 78% and 84% respectively. The classification model produced 87% across the CNN model and the one with a gradient-boosted decision tree model. Then the dataset’s amount of data was deemed very small for big data analysis, which was the main constraint of this effort. However, the immediate analysis of events and cloud computing services offered was appropriate for a big data analysis architecture.

In 2022, Fang Liu [26] stated the building of a multi-level language collection is first accomplished in this study using large-scale data technology including the DL technique. Then EKF model was developed to create a data query system through the examination of NN algorithms. Finally, simulation tests confirm that language bytes can be appropriately retrieved from the language database. Research demonstrates the great degree of result adaptation of the database using big data as well as the DL algorithm. In addition to being significantly superior to the other two approaches when the total amount of analyses exceeded 15, it also had the simplest convergence curve. The outcomes of the simulation demonstrate the effectiveness of the DL algorithm combined with the C-type travelling wave approach for building language databases to handle branch point attribute data. Additionally, it has an appropriately matching effect on Big data, allowing it to increase the database’s capacity for recognizing various linguistic systems and vastly enhancing retrieving effectiveness and precision.

In 2022, Xuegong Du et al., [27] described a CNN-based big data analysis and forecasting system. To investigate the distributed data layout of big data, continuous pattern matching technology is employed, along with the fusion of information processing of cloud-based gathered big data, comparison associated methods for frequent item identification, and rule of association extraction of features from large-dimensional fusion data. To achieve the categorization and processing of Big data from the cloud services collection, a method known as clustering is used. The CNN and camera combined for identifying the environment around it was currently a centre for research because the hardware technology of the vehicle to identify the nearby environment is challenging. However, CNN alone cannot solve the issues of long training times and poor accuracy. It must also analyze camera input. It is suggested to use a better CNN. The results of the experiment reveal that this technique’s mining of data accuracy is 12.43% as well as 21.76% greater than that of standard approaches. Additionally, because there are fewer iteration phases, mining is more timely. The network’s precision and learning speed can both be successfully increased due to the network’s architecture. It has been demonstrated that the CNN provides a higher accuracy as well as a greater rate of training.

In 2019 S. K. Lakshmanaprabu et al., [28] developed an IoT-based healthcare system of big data analytics by using MapReduce process and Random Forest Classifier. Here, various diseases affected patient’s e-health data was collected for the analysis of bigdata. For better classification, the enhanced factors were selected from the dataset by utilizing the Improved Dragonfly Algorithm. After that, based on the enhanced features, the e-health data is classified by using the Random Forest Classifiers. Moreover, the experimental results were carried out and noticed that the proposed model has attained 94.2% high precision. Also, various performance measures were evaluated for the proposed model and compared with other traditional methods. Table 1 provides examples of certain large data-related problems and difficulties.

Table 1
Features and challenges of conventional insider threat detection model

Author [citation]	Adopted methodology	Features	Challenges
Shuaiyin Ma et al., [19]	Big data driven correlation analysis-based clustering	The proposed model enhance the accuracy of the correlation analysis and it balance the connection among supply as well as demand	Need to enhance the correlation analysis by using Deep learning algorithm for analysing the relation among the real time resources and consumption of energy
Abhilasha, A et al., [20]	Self-Manages Ensample Optimizer	The imbalanced datasetcontains with 99% accuracy to achieve stableinformation with enhanced the classification outcome	Need to analyse various metrics and also consider the clustering algorithm
Dongwei Li et al., [21]	K-means, EM algorithm	The proposed algorithm was obtained 99% accuracy and required the complete accuracy of the computation time is 47.71%–71.14%	Have to consider the time-series data as well as spatiotemporal data for the cost effectiveness of the proposed algorithm
Chitrakant Banchhor and Srinivasu, N [22]	CNB, CGCNB	The efficiency of the classifiers was improved and better performance with respect to time consumption and accuracy	Need to analyse the classifiers through log loss function and training loss function for improved performance
Jayasri N.P and R. Aruna [23]	Hierarchical decision attention network, AR	The proposed algorithm’s output was provided by metrics like accuracy, precision, recollection, and the F-score (99% execution in dataset)	Need to allowed cloud computing framework for better performance of model in real time
Mario Juez-Gil et al., [24]	Bagging, Boosting	Big Data to analyze unbalanced datasets, simpler strategies outperformed more complicated ones.	Need to process and reduce imbalance data and also implemented a newclassifier for bigdata framework
Noha Mostafa et al., [25]	Smart grid and renewable energy power	The classification model formed 87% across the CNN model and with a gradient boosted decision tree model	Have to consider different energy source of larger datasets
Fang Liu [26]	NN, EKF	The model has an appropriately matching effect on Big data, allowing it to increase the database’s capacity for recognizing various linguistic systems	Need to consider the grammatical differences between various language systems
Xuegong Du et al., [27]	CNN, Fuzzy k-means clustering	The experimental outcome shows that this technique’s mining of data accuracy is 12.43% as well as 21.76% greater than that of standard approaches	Need to include various classifiers to enhance the prediction accuracy
S. K. Lakshmanaprabu et al., [28]	IDA	The efficiency of the proposed model was obtained recall as 89.99% and 94.2% of accuracy	Need to consider new disease datasets and also require different classifiers for better performance with speed computation

2.1 Problem statement

Numerous complications and concerns arise during the storage and big data process due to the rapid expansion of data. To fix these flaws and issues in a cloud environment, there aren’t many options accessible. Pig Latin, Dryad, MongoDB, Cassandra, and MapR are a few examples of technologies that cannot tackle these problems in big data processing. Users still lack access to databases and low-level infrastructure for processing information and administration, despite the aid of Hadoop and MapR. Big Data Analytics has been an important subject in the field of data science as numerous businesses are investing in creating solutions using it to solve their monitoring, testing, analysis of data, modelling, and other technical requirements. In the world of Big Data Analytics, the knowledge acquired by Deep Learning algorithms remains largely unexplored. Deep Learning has been extensively used in some Big Data areas, like computer vision and recognition of speech to enhance classification modelling outcomes. Deep Learning is appealing as a useful tool for Big Data Analytics because it can extract high-level, concepts and data representations from Big data, mostly uncontrolled data. Deep learning can be used to better tackle Big Data issues like biased modelling, rapid knowledge retrieval, linguistic indexing, and information labelling.

3. Proposed big data clustering and classification model

This paper proposes the bigdata clustering and classification model with improved fuzzy-based Deep architecture under the MapReduce framework. The following description are the step-by-step process of the proposed model.

•
Initially, Data partitioning is carried out in the pre-processing phase by using an improved C-Means clustering process.
•
Then, the MapReduce framework is applied to handle the pre-processed bigdata which contains the mapper and reducer phase. In the mapper phase, an improved normalization process takes place, subsequently, the feature fusion process, which fuses entropy-based features and correlation-based features, is carried out. Then all the mappers are merged together to obtain the combined feature set in the reducer phase.
•
Finally, the classification process takes place by the deep hybrid model with the combination of Deep Convolutional Network and Bidirectional Gated Recurrent Unit. Here, the final classification outcome is determined by the Improved score level fusion method. The overall architecture of bigdata classification is illustrated in Fig. 1.

Figure 1.
Overall architecture of the bigdata classification.

3.1 Pre-processing the big data via improved fuzzy C-means clustering

As there are different data sources, processing and cleansing data is challenging. Additionally, data sources could be incomplete or riddled, which makes the further process complicated for classification. To handle this problem, this work does partitioning of data (bigdata dataset $B_{D}$ ) by the improved fuzzy c-means clustering.

3.1.1 Improved fuzzy c-means clustering

Initially, the input big data, $B_{D}$ is split to form various clusters by using Improved fuzzy c-means clustering. One of the renowned data segmentation methods is fuzzy C-means [29]. The improved fuzzy c-means algorithm is briefly described in this section. Consider of a group of unlabelled patterns ${X}^{\prime}$ , where $n$ represents the total number of forms and $F$ represents a dimension of the pattern’s matrices. After that, let ${X}^{\prime}=\left\{{x_{1},x_{2},\ldots,x_{n}}\right\}$ . The objective function’s value is minimized using the FCM technique. The objective function evaluates the way a dataset was partitioned into $B$ groups. The procedure is repeated in the clustering technique, which minimizes the weighted across-groups sum of the squared error objective function $J_{m}$ to obtain an ideal $B$ partition, which is calculated in Eq. (1).

$\displaystyle J_{m}\left({V,C}\right)=\sum\limits_{j=1}^{B}{\sum\limits_{i=1}^% {n}{V_{ij}^{m}}}d_{ij}^{2}$ (1)

Where, $B$ indicates the number of clusters, $V_{ij}$ indicates the membership degree $X_{i}$ in the $j$ th cluster. $C_{i}$ represents the model of the centre of the cluster $j$ , $d_{ij}^{2}$ represents a distance measure among the object and the cluster centre. Here, the distance centre $d_{ij}^{2}$ is calculated in a new way of computation that combines Modified Euclidean Distance and the Average of Manhattan Distance evaluation.

A) Modified Euclidean distance

The spatial distance between the two sets of spatial vectors, $l=\left\{{l_{1},l_{2},\ldots,l_{n}}\right\}$ and $k=\left\{k_{1},k_{2}\right.$ , $\left.\ldots,k_{n}\right\}$ , can be determined using the Euclidean distance [30], which is defined as in Eq. (2). where $l_{i}$ and $k_{i}$ represent any elements in the vector $l$ and $k$ , and $q$ remains the Euclidean distance between two samples events.

$\displaystyle q=\sqrt{\sum\limits_{i=1}^{n}{\left({l_{i}-k_{i}}\right)^{2}}}$ (2)

By separating the Euclidean distance given in Eq. (2) the distance is normalized by the least value of the square root of the squared sum of two space vectors. The modified Euclidean distance is defined in Eq. (3) by multiplying each vector component by a constant stability factor $\beta$ since it is difficult to determine a correct outcome when the present sample value on a single side is 0. where $\beta$ represents a 0.6 stabilizing factor.

$\displaystyle q_{i}=\frac{\sqrt{\sum\limits_{i=1}^{n}{\left({\left|{l_{i}}% \right|-\left|{k_{i}}\right|}\right)^{2}}}}{\min\left({\sqrt{\sum\limits_{i=1}% ^{n}{l_{i}^{2}}}*\beta,\sqrt{\sum\limits_{i=1}^{n}{k_{i}^{2}}}*\beta}\right)+\beta}$ (3)

The improved Euclidean distance $q_{i}$ is bound to zero if all sampling values are treated as the associated absolute numbers. But for interior errors, the fluctuating currents on both sides differ significantly, so the improved Euclidean distance in Eq. (3) is much higher than 0.

B) Average of manhattan distance

The separation between two places in space is known as the Manhattan distance. However, it is the total of the vertical and horizontal distances between two variables rather than the linear distance among them. Using this as a basis, the approach is improved to provide the improved Manhattan distance among the two matrices. With the data matrix and the associated row component of the matrix, the Manhattan distance is calculated. The Manhattan distance can be obtained from the number of matrix paths, and the average value improves the Manhattan distance [31]. The computation of the distance to Manhattan is in Eq. (4). Where, $n$ represents the line number of identical arrays, $x_{i}$ and $y_{i}$ are column vectors that correspond to the two matrices one after the other. $\textit{MD}(t,m)$ represents the Manhattan distance The computation method for the average Manhattan distance $(\textit{MD})_{m}$ is provided in Eq. (5).

$\displaystyle\textit{MD}(t,m)=\sum\limits_{i-1}^{n}{\left|{x_{i}-y_{i}}\right|}$ (4) $\displaystyle(\textit{MD})_{m}=\frac{1}{n}\sum\limits_{i-1}^{n}{\textit{MD}% \left({t,m}\right)}$ (5)

According to the improved fuzzy c-means clustering process, the proposed distance measures are calculated by adding both the modified Euclidean distance in Eq. (3) and Manhattan distance in Eq. (5). Equations (6) and (7) signifies the improved distance measures among the object and the cluster centre.

$\displaystyle d_{ij}^{2}=\left[{q_{i}+\left({\textit{MD}}\right)_{m}}\right]$ (6) $\displaystyle d_{ij}^{2}=\left[{\frac{\sqrt{\sum\limits_{i=1}^{n}{\left({\left% |{l_{i}}\right|-\left|{k_{i}}\right|}\right)^{2}}}}{\min\left({\sqrt{\sum% \limits_{i=1}^{n}{l_{i}^{2}}}*\beta,\sqrt{\sum\limits_{i=1}^{n}{k_{i}^{2}}}*% \beta}\right)+\beta}+\frac{1}{n}\sum\limits_{i-1}^{n}{\textit{MD}\left({t,m}% \right)}}\right]$ (7)

Then substitute the overall distance from Eq. (7) to Eq. (1), to form the clusters $B_{D}^{P}$ .

3.2 MapReduce framework

After the clustering process of big data, the Map Reduce framework is applied to handle it. According to the Map Reduce framework, initially, the preprocessed data contain several clustered data and each clustered data is again split into the multiple mapper phase. The Mapper phase intends the process of improved data normalization and feature fusion process as well. Here, entropy-based features and correlation-based features are fused together by using the weighted average fusion method. After that, the Reducer phase combines the overall features from all mappers. Finally, the results of the suitable features are obtained from the output of the reducer phase. The normalization and feature extraction under the Map-Reduce framework is depicted in Fig. 2.

Figure 2.

Feature extraction under the Map-Reduce framework.

Mapper Phase: As specified above, the mapper phase includes the data normalization and feature fusion with the extracted features including entropy-based features and colleration-based features.

3.2.1 Improved normalization

A collection of data is transformed to be on an equivalent scale through normalization. According to the data themselves, the purpose of ML algorithms is typically to update and adjust the information to ensure it has a value in the range of 0 to 1 or between $-$ 1 and 1. To decide that the data implies a normal distribution, a typical approach for doing that is to determine the standard deviation and mean on a collection of data and then modify every sample by deducting the mean as well as splitting pod each sample by standard deviation. This method is beneficial because it allows to standardization of the data and attains a typical normal distribution. As a result of each characteristic being on similar scales after normalization, the slope descent step is able to be stabilized, enabling us to employ higher learning levels or causing models to converge more quickly for a particular learning rate.

According to this paper, improved normalization takes place, which is based on the tanh normalization process. Tanh estimation techniques are thought to be a more efficient and dependable normalization method. Along with convergent faster than Z-score normalization process, it is not susceptible to outliers. Thus, it produces values ranging from $-$ 1 to 1. Let the data is taken as the set of $S$ vectors (i.e., $S=\left\{{s_{1},s_{2},\ldots,s_{l},\ldots,s_{n}}\right\})$ . Equation (8) shows the tanh normalized score $s_{l}^{N}$ [32] calculation for the improved normalization.

$\displaystyle s_{l}^{N}=0.5\left\{{\tanh\left({0.01\left({\frac{s_{l}-\mu\left% ({S_{K}}\right)}{\sigma\left({S_{K}}\right)}}\right)}\right)+1}\right\}$ (8)

Where, $\mu\left({S_{K}}\right)$ represents the Arithmetic mean calculation from the preprocessed data, which is calculated as per Eq. (9). $\sigma\left({S_{K}}\right)$ represents the standard deviation derived from the preprocessed data, Arithmetic mean is termed as the ratio of the sum of all observations to the number of observations. ( $\textit{AM}=\frac{s_{1}+s_{2}+s_{3}+..s_{n}}{n})$

$\displaystyle\mu\left({S_{K}}\right)=\textit{Arithmetic mean}\left({\sigma% \left({s_{l}}\right)-\left({S_{K}}\right)}\right)$ (9)

From the normalized data $s_{l}^{N}$ , the features are extracted including entropy-based features and correlation-based features, and are fused together with the improved feature fusion process.

3.2.2 Improved feature fusion

{F}^{\prime}

via weighted average fusion

In this section, the improved feature fusion is takes place including the fusion of entropy-based features and correlation-based features. The stepwise procedure for the Improved feature fusion by using the weight average fusion method is explained as follows.

Step 1:
Feature extraction $\left(F\right)$

In the improved feature fusion method, feature extraction is the first step. Initially, entropy-based features and correlation-based features are extracted from the normalized data. Then the final extracted features are subjected to the feature selection. It is explained as follows

(a)
Entropy-based features $\left({F^{E}}\right)$

Entropy [33] is a measurement of a random variable’s unpredictability. Assume that $X$ is a discrete random variable and the probability mass functional is $P(x)=\Pr\left\{{X=x}\right\},x\in Z$ . Here, $P_{x}\left(x\right)$ is changed into $P(x)$ for the probability mass function. The terms $P(x)$ and $P(y)$ relate to two distinct random variables and are in fact, two distinct probability mass functions, $P_{x}\left(x\right)$ and $P_{y}\left(y\right)$ , accordingly. The discrete random parameter $X$ ’s entropy $H(X)$ is defined by using Eq. (10).

$\displaystyle H(X)=-\sum\limits_{x\in z}{P(x)\log p(x)}$ (10)

Since $x\log x\to 0$ , that is simply supported by continuity. Thus, the entropy is unaffected by the addition of terms of probability zero. If the logarithm’s base is $b$ , then entropy is represented as $H_{b}(X)$ . All logarithms will be taken to base 2 unless otherwise stated, hence all entropies will be expressed in bits. The entropy is a function of $X$ is distributed. It simply depends on the probabilities rather than the actual values that the $X$ random variable has chosen. $e$ will be used to indicate the expectancy. Thus, if $X$ tends to $P(x)$ , then Eq. (11) shows the expression of the expectation, $e_{P}G(X)$ or $eG(X)$ .

$\displaystyle e_{P}G(X)=\sum\limits_{x\in z}{G(x)P(x)}$ (11)

If $X$ is drawn using the probability-mass function $P(x)$ , then the predicted value of $\log^{\frac{1}{P(X)}}$ can likewise be used to evaluate the entropy of $X$ . Consequently, Eq. (12) shows the entropy calculation based on the probability mass function. Here the entropy feature $\left({F^{E}}\right)$ is taken as $H(X)$

$\displaystyle H(X)=e_{P}\log\frac{1}{P(X)}$ (12)
(b)
Correlation-based features $\left({F^{C}}\right)$

Next, the correlation-based features [34] are extracted accordingly, the Pearson correlation coefficient method is used in this proposed work. It is applied for the linear inter-relationship between two variables are identified. It is measured on ratio scales or intervals. Also, statistical measures are used in its frequency. If the data has a continuous scale, then the correlation coefficient is in the range of $-$ 1 to $+$ 1. If the value is positive, it indicates the positive interconnection between two variables, whereas, the negative value indicates the perfect negative correlation. If the value is 0 indicates there is no inter connection among the two variables. According to this work, the normalized data is extracted based on correlation-based features under the Pearson correlation coefficient. Eq. (13) shows the calculation of the Pearson correlation coefficient and it is denoted as $C(X,Y)$ .

$\displaystyle C(X,Y)=\frac{\sum{\left[{\left({x_{i}-{x}^{\prime}}\right)\times% \left({y_{i}-{y}^{\prime}}\right)}\right]}}{\sigma_{x}\times\sigma_{y}}$ (13)

Here, ${x}^{\prime}$ indicates the mean of $x$ variables in the normalized data, ${y}^{\prime}$ indicates the mean of $y$ variables in the normalized data. $F^{C}$ indicates the correlation-based feature which is based on the Pearson correlation coefficient $P(X,Y)$ . Therefore, the total features are extracted from the entropy and correlation-based features are termed as $F=\left[{F^{E}\,\,\,\,\,F^{C}}\right]$ .
Step 2:
Feature Selection by using PCA

In this step.2, from the extracted features, $F$ , the appropriate features are selected by using PCA. The proposed method clearly demonstrates that the PCA possesses the ability to execute the selection of features and has the ability to select several important individuals to every one of the feature elements $F$ .

Let $x$ be the eigen vector of the PCA covariance matrix [35], the results of the feature extraction in terms of $x$ , with an arbitrary sample vector $l$ is expressed in Eq. (14). Where, $x=\left[{x_{1}\ldots x_{n}}\right]^{T}$ , $l=\left[{l_{1}....l_{n}}\right]^{T}$ and the dimensionality of sample vector is represented as $n$ , $x_{i}$ indicates the $i$ th feature component of the sample from the extracted features. Here, it is easy to identify the smaller absolute value $x_{i}$ , which shows the minimum contribution of the $i$ th feature samples. Likewise, assuming the absolute value of $x_{k}$ is very small, $k$ th feature element is neglected which is not taken as the effect of the feature extraction outcome.

$\displaystyle z=l^{T}x=\sum\limits_{i=1}^{n}{l_{i}}x_{i}$ (14)

It is also noted that there are always multiple eigenvectors, once evaluating the important of a single feature component, there is more than one eigenvectors are considered. To perform the feature selection, the following methods are developed.

–
By using the original training data, the covariance matrix of PCA is calculated. After that, all the eigen values and eigen vectors are solved.
–
Based on the first largest eigen values $l$ , their eigen vectors are selected, which is denoted by $V_{1},V_{2},\ldots,V_{l}$
–
Then, calculate the contribution of $j$ th feature component from the result of feature extraction, which is calculated in Eq. (15). Where, $\left|{V_{pj}}\right|$ represents the absolute value of $V_{pj}$ , $j=1,2,\ldots,n$ and $p=1,2,\ldots,l$

$\displaystyle\textit{CT}_{j}=\sum\limits_{p=1}^{l}{\left|{V_{pj}}\right|}$ (15)
–
Arrange the contribution of the features in descending order and it is stored by using $d_{j}$ . If $\textit{CT}_{s}$ and $\textit{CT}_{t}$ are the first and second largest contribution, then $d_{1}=s$ and $d_{2}=t$ that indicates the $s$ th and $t$ th feature component in the original data. If there is $n$ number of dimensional features are needed, then the result of feature selection will be $d_{1}$ th, $d_{2}$ th, …, $d_{n}$ th feature components.

Here, select the top $k$ principal components, considering the selected features $F^{S}$ .
Step 3:
Normalization of selected features

Then, the selected features $\left({F^{S}}\right)$ are normalized by using min-max normalization to determine them between 0 and 1 range. The selected features is converted linearly using min-max normalization, also terms as scaling of features. This can be done using Eq. (16), where $x_{\min\textit{imum}}$ denotes the minimum value of the features and $x_{\max\textit{imum}}$ denotes the maximum value of the features. The associations between the original values of data are preserved by min-max normalization.

$\displaystyle x_{\textit{scal}}=\frac{x-x_{\min\textit{imum}}}{x_{\max\textit{% imum}}-x_{\min\textit{imum}}}$ (16)
Step 4:
Computation of weight

After the normalization of the selected features, the computed mutual Information score assigns weight to the selected features. According to mutual information [33], the amount of information is measured by one random variable consisting of another random variable. Here, a single random variable’s uncertainties are reduced due to the knowledge of the other random variable.

Consider two random variables $Q$ and $R$ that have respective probability mass functions $p\left(q\right)$ and $p\left(r\right)$ as well as a combined probability mass function $p\left({q,r}\right)$ . The mutual information $I\left({Q;R}\right)$ is the relative entropy across the combined distribution and the product distribution, which is calculated in Eq. (17)

$\displaystyle I\left({Q;R}\right)=\sum\limits_{q\in Q}{\sum\limits_{r\in R}{p% \left({q,r}\right)}}\log\frac{p\left({q,r}\right)}{p\left(q\right)p\left(r% \right)}$ (17)

Then, normalize the feature score, so that the summation of the normalized score is always equal to 1. The normalized mutual information score is defined as the ratio between the mutual information score of each feature and the summation of all mutual information score, which is mathematically expressed in Eq. (18)

$\displaystyle W_{i}^{\textit{norm}}=\frac{W_{i}}{\sum W}$ (18)

Where, $W_{i}=$ Mutual information score of the $i$ th feature

$W_{i}^{\textit{norm}}=$ Normalized weight of $i$ th feature

$\sum W=$ Sum of all Mutual Information Score
Step 5:
Weighted Averaging

Using the weighted average strength [36] of the relevant areas taken from the input and output data, the resulting fused feature is created. With the computed Normalized weight in Eq. (18), the weighted averaging of the selected as well as normalized features is performed. Each feature is multiplied by its equivalent Normalized weight and then concatenated to produce a fused and weighted feature ${F}^{\prime}$ , which is defined in Eq. (19). Here, $W_{\textit{avg}}$ represents the weighted average, $W_{1},W_{2},..,W_{n}$ represents the normalized weight calculated by using mutual information score, $F_{1},\,F_{2},F_{3},..,\,F_{n}$ represents the normalized features.

$\displaystyle W_{\textit{avg}}=\left[{W_{1}F_{1},\,\,\,\,W_{2}F_{2},\,\,\,\,% W_{3}F_{3},\,\,..,\,\,\,\,W_{n}F_{n}}\right]$ (19)

Reducer phase: the output of the mappers is shuffling together to get into the reducer phase. Then, the final feature $F^{F}$ is obtained from the Reducer phase, which is subjected to the classification process.
3.3 Classification process via DCNN and Bi-GRU

In this work, the final process is the classification process by the combination of the DCNN classifier and the Bi-GRU classifier. Here, the final features, $F^{F}$ obtained are subjected to the hybrid classifiers that combine DCNN and Bi-GRU. Then the final classification outcome is determined by using the Improved Score level fusion process. The following section explains the classification process of the proposed work.

3.3.1 DCNN

Initially, the feature $F^{F}$ is fed into the DCNN classifier. Figure 3 depicts the fundamental design of the CNN or DCNN. The convolution layer, pooling layer, fully connected layer and activation function make up its four basic components. In DCNN, the fully connected network categorizes the gathered features after the convolutional layer, pooling layer and activation function are mostly employed for feature extraction [37]. The following is a description of the four categories. Figure 3 shows the DCNN architecture.

Figure 3.

DCNN architecture.

(a)

Convolution Layer

Convolution primarily employs the data of the kernel of convolution to carry out the convolution process using a sliding window approach on the input matrices. The generated output matrix is inversely proportional to the input matrix’s convolutional kernel, steps and buffering sizes, which is expressed in Eq. (20).

$\displaystyle h_{o}=\left[{\frac{\left({h_{i}-k_{h}}\right)+2p}{s}}\right]+1,b% _{o}=\left[{\frac{\left({b_{i}-k_{b}}\right)+2p}{s}}\right]+1$ (20)

Where, $b_{o}$ and $h_{o}$ stands for the width and height of the resultant matrix, $h_{i}$ and $b_{i}$ for the height and width of the matrix being input, $p$ stands for the number of padded cycles, and $s$ stands for the distance travelled throughout the convolution kernel process.

(b)

Pooling Layer

The fundamental purpose of pooling is to decrease the data dimension while preserving the most important data. There are actually two typical approaches for calculating pooling. The initial method is known as maximum pooling, and unlike the others, it outputs the highest value from the mask. Average pooling is the second. The average among all the values in the filter is the output.

(c)

Fully Connected Layer

A multilayer neural network with fully connected layers is said to be fully connected. All feature maps are transformed into an array of one dimension for the fully linked layer’s networking input. For categorization or forecasting, the fully connected NN is employed last.

(d)

Activation Function

Both linear and non-linear functions make up the activation function. Compared to linear functions, non-linear functions can be represented more accurately. In common neural networks, non-linear functions are more frequently used. ReLU is more frequently utilized as a function that is not linear. Equation (21) illustrates the formula for the ReLU function. The outcome is $x$ if the value of input $x$ is larger than 0 and 0 for otherwise.

$\displaystyle f\left(x\right)=\left\{{{\begin{array}[]{*{20}c}{x,\,\,\,\,% \textit{if}\ (x>0)}\hfill\\ {0,\,\,\,\,\textit{otherwise}}\hfill\\ \end{array}}}\right.$ (21)

3.3.2 Bi-GRU

The GRU is a simpler version of the LSTM, and both models are enhanced RNNs with strong modelling skills for dependence over time. A reset gate $\left({R_{t}}\right)$ and an update gate $\left({S_{t}}\right)$ make up a GRU unit. According to this paper, the total feature $F^{F}$ is taken as input into the Bi-GRU model [38]. Both the current input $F^{F}$ and the prior state $h_{t-1}$ are used to calculate the output $h_{t}$ when these two gates are in control. The following formula in Eq. (3.3.2) is used to calculate the outputs of the gates and GRU unit

$\displaystyle R_{t}=\sigma\left({W_{R}F^{F}+P_{R}h_{t-1}+B_{R}}\right)$ $\displaystyle S_{t}=\sigma\left({W_{S}F^{F}+P_{S}h_{t-1}+B_{S}}\right)$ $\displaystyle\tilde{h}_{t}=\tanh\left[{W_{h}F^{F}+P_{h}\left({R_{t}\cdot h_{t-% 1}}\right)+B_{h}}\right]$ $\displaystyle h_{t}=\left({1-S_{t}}\right)\cdot h_{t-1}+S_{t}\cdot\tilde{h}_{t}$ (22)

Where, $W_{R}$ , $P_{R}$ , $W_{S}$ , $P_{S}$ , $W_{h}$ and $P_{h}$ represents the weight matrices. $B_{R}$ , $B_{S}$ and $B_{h}$ represents the bias vectors for input $F^{F}$ and previous sate, $h_{t-1}$ , $\sigma$ denotes the sigmoidal function, $\tanh$ represent the hyperbolic tangent activation function.

When working with the present data, models with a bi-directional structure have the capacity to acquire data from both past and subsequent data. Bi-GRU contains two GRUs. one processing the information forward and the other processing it backwards. Simply the input and forget gates are present in this bidirectional recurrent neural network. Figure 4 depicts the Bi-GRU model diagram’s structure. The two GRUs’ states, which are bidirectional and pointing in the opposite direction, are used to decide on the bi-GRU model. The first GRU moves forward, starting at the beginning of the data series, and the second GRU moves backwards, starting at the conclusion of the information sequence. This enables knowledge from the past as well as the future to affect the conditions of the present. Equation (3.3.2) describes the bi-GRU. Where, $\overrightarrow{h}_{t}$ indicates the forward GRU state, $\overleftarrow{h}_{t}$ indicates the backward GRU state, $\oplus$ indicates the concatenating two vectors. Then the final output from the output layer of the Bi-GRU classifier is denoted as $O^{F}$ .

$\displaystyle\overrightarrow{h}_{t}=\textit{GRU}_{\textit{fwd}}\left({F^{F},% \mathop{h_{t-1}}\limits^{\to}}\right)$ $\displaystyle\overleftarrow{h}_{t}=\textit{GRU}_{\textit{bwd}}\left({F^{F},% \mathop{h_{t-1}}\limits^{\leftarrow}}\right)$ $\displaystyle h_{t}=\overrightarrow{h}_{t}\oplus\overleftarrow{h}_{t}$ (23)

Therefore both the output obtained from the DCNN and Bi-GRU classifiers are combined by using the Improved score level fusion method to get the final classification output. Figure 4 shows the Bi-GRU architecture.

Figure 4.

Bi-GRU architecture.

3.3.3 Improved score level fusion

In this proposed work, the outcome from the hybrid classifiers is normalized by using z-improved min-max normalization. The score level fusion techniques [39], the classification process of the hybrid classifiers are separately by using Euclidean Distance and then normalized by using Eq. (24). After that the scores are fused together based on the addition rule as per Eq. (25). Here, $\textit{SC}_{i}$ , $\textit{SC}_{Ni}$ represents the score value and score normalization of specimen $i$ respectively. $\max\left({\textit{SC}_{i}}\right)$ , $\min\left({\textit{SC}_{i}}\right)$ indicates the maximum and minimum value of the score vector, $M$ represents the number of samples to be used.

$\displaystyle\textit{SC}_{Ni}=\frac{\textit{SC}_{i}-\max\left({\textit{SC}_{i}% }\right)}{\max\left({\textit{SC}_{i}}\right)-\min\left({\textit{SC}_{i}}\right)}$ (24) $\displaystyle f_{\textit{score}}=\sum\limits_{i}^{M}{\left({\textit{SC}_{Ni}}% \right)}$ (25)

According to this work, an improved score level fusion is proposed for getting more accuracy for the big data classification. Here both the classification models are normalized separately by using min-max normalization, Eq. (26) shows the min-max normalization of the DCNN classifier, ${X}^{\prime\prime}_{\textit{DCNN}}$ . Where the $\textit{Min}({X}^{\prime})$ and $\textit{Max}({X}^{\prime})$ of the samples in the DCNN classifier are taken as z-score normalized data. It is among the most widely used standardization methods. Then determine the parameters’ z-scores based on their unique distributions. It is only effective if the information is distributed in a Gaussian-like manner. The mathematical representation of the z-score normalization is expressed in Eq. (27). Where $\textit{std}(X)$ indicates the standard deviation of the sample, $\textit{Mean}(X)$ indicates the average of geometric mean (GM) and contra harmonic mean (CHM). The Geometric mean calculation is based on Eq. (28), which is defined as the $n$ th root of the product of $n$ numbers. Here, multiply the given data value and take the total number of data values in the root, here $a_{1}\times a_{2}\times..\times a_{n}$ represents the product of the samples. The contrahramonic mean of the variable is calculated as per Eq. (29), it is defined as the ratio of summation of the arithmetic mean and variance to the arithmetic mean. Consider the variance is greater than or equal to zero, then the contra harmonic mean is equal to or greater than the arithmetic mean. Equation (30) represents the mean of the sample which is the average of the geometric mean and the contra-harmonic mean values.

$\displaystyle{X}^{\prime\prime}_{\textit{DCNN}}=\frac{X-\textit{Min}({X}^{% \prime})}{\textit{Max}({X}^{\prime})-\textit{Min}({X}^{\prime})}$ (26) $\displaystyle{X}^{\prime}=\frac{X-\textit{Mean}(X)}{\textit{std}(X)}$ (27) $\displaystyle\textit{GM }=\sqrt[n]{a_{1}\times a_{2}\times..\times a_{n}}$ (28) $\displaystyle\textit{CHM }=\frac{\sum\limits_{i}^{n}{x_{i}^{2}}}{\sum\limits_{% i}^{n}{x_{i}}}$ (29) $\displaystyle\textit{Mean}(X)=\frac{\left[{\textit{GM}+\textit{CHM}}\right]}{2% }=\frac{\left[{\sqrt[n]{a_{1}\times a_{2}\times..\times a_{n}}+\frac{\sum% \limits_{i}^{n}{x_{i}^{2}}}{\sum\limits_{i}^{n}{x_{i}}}}\right]}{2}$ (30)

Similarly, the min-max normalization of the Bi-GRU classifier is taken and follows the same procedure as mentioned in DCNN classifiers. It is calculated by using Eq. (31), where ${X}^{\prime\prime}_{\textit{Bi }-\textit{GRU}}$ indicates the min-max normalization of the Bi-Gru classifier

$\displaystyle{X}^{\prime\prime}_{\textit{Bi }-\textit{GRU}}=\frac{X-\textit{% Min}({X}^{\prime})}{\textit{Max}({X}^{\prime})-\textit{Min}({X}^{\prime})}$ (31)

Then the final fusion score is calculated as per Eq. (32), which is the combination of the normalized DCNN classifier and Bi-GRU classifier. This is the final classification outcome of the bigdata classification model.

$\displaystyle f_{\textit{score}}=\left({{X}^{\prime\prime}_{\textit{DCNN}}+{X}% ^{\prime\prime}_{Bi-\textit{GRU}}}\right)$ (32)

4. Results and discussion

4.1 Experimental setup

The big data classification model was implemented using Python, specifically with Python version 3.7. The computational hardware used for this implementation was an AMD Ryzen 5 3450U processor with Radeon Vega Mobile Gfx, operating at 2.10 GHz. The system was equipped with a total of 16.0 GB of RAM, of which 13.9 GB was available for use. Further, the big data classification analysis was carried out using the Lung Cancer Dataset [40]and [41].

4.2 Dataset1 description

Hong and Young used the lung cancer dataset to demonstrate the effectiveness of the ideal discriminant plane even in ill-posed situations. Multivariate is a characteristic of the Lung Cancer dataset. Further, it includes 56 attributes and 32 instances. The associated task is classification. Class label is the first attribute. Each nominal predictive attribute has an integer value between 0 and 3. It consists of a totally 10 attributes. For the attribute the type is categorical and the role is the feature. For the variable name class, the role is the target and the type is categorical.

4.3 Dataset2 description

The raw network packets of the UNSW-NB 15 dataset were developed by the IXIA PerfectStorm tool in the Cyber Range Lab of UNSW Canberra for generating a hybrid of real modern normal activities and synthetic contemporary attack behaviours. The tcpdump utility was employed to capture 100 GB of the raw traffic (e.g., Pcap files). There are nine different kinds of attacks in this dataset: worms, reconnaissance, shellcode, DoS, backdoors, fuzzers, and exploits. The Argus, Bro-IDS tools are utilized and twelve algorithms are built to generate fully 49 features with the class label. The UNSW-NB15_features.csv file contains a description of these features.UNSW-NB15_GT.csv is the name of the ground truth table, and UNSW-NB15_LIST_EVENTS.csv is the name of the event list file. This dataset was partitioned, and UNSW_NB15_training-set.csv and UNSW_NB15_testing-set.csv, respectively, were set up as the training and testing sets. There are 175,341 records in the training set and 82,332 records in the testing set, which are divided into two categories: attack and normal.

4.4 Performance evaluation

The evaluation of classification performance involved comparing the DCNN+Bi-GRU approach with traditional methods. This assessment considered various performance metrics, including specificity, FNR, precision, FPR, MCC, sensitivity, F-measure, NPV, and accuracy. Additionally, we conducted a comparative analysis between the DCNN+Bi-GRU method and state-of-the-art approaches like ICNN [27] and RFC+MAPREDUCE [28], as well as traditional methods such as Bi-GRU, DCNN, LSTM, RF, RNN, and GRU.

4.5 Performance evaluation of positive measures for Dataset 1

In Fig. 5, the positive metric evaluation for the DCNN+Bi-GRU method is presented in comparison to Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU for the classification of big data. To achieve precise classification of big data, the ideal outcome is for the model to produce elevated positive metric scores. The primary observation is that the DCNN+Bi-GRU approach demonstrates a highly accurate classification of big data, with accuracy ratings consistently exceeding 90%. In contrast, Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU yield lower accuracy values. Especially, training rate=70, the DCNN+Bi-GRU method achieved an accuracy of 94.231, whilst the Bi-GRU is 87.360, DCNN is 83.705, LSTM is 79.812, RF is 83.535, RNN is 86.570, ICNN [27] is 89.712, RFC+MAPREDUCE [28] is 81.727 and GRU is 85.768, correspondingly. Additionally, when the training percentage was set at 90%, the DCNN+Bi-GRU approach demonstrated a specificity of 95.630, whereas Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU all achieved lower specificity values.

Concurrently, the evaluation of precision and sensitivity for DCNN+Bi-GRU and conventional methods is illustrated in Fig. 5(b) and 5(c). At the training rate 80, the DCNN+Bi-GRU obtained a sensitivity of 96.103, though the conventional strategies scored least sensitivity values, notably, Bi-GRU $=$ 90.174, DCNN $=$ 83.495, LSTM $=$ 79.709, RF $=$ 71.936, RNN $=$ 91.419, RFC+MAPREDUCE [28] $=$ 90.283 and GRU $=$ 86.102, respectively. In summary, the exceptional performance exhibited by the DCNN+Bi-GRU approach firmly reinforces its efficacy in exactly classifying big data. This high level of efficacious is achieved via the integration of improved fuzzy c-means clustering and improved normalization with deep hybrid classification strategy.

Figure 5.

Validation of DCNN+Bi-GRU versus conventional methods with respect to positive metrics.

4.6 Performance evaluation of negative measures for dataset 1

Figure 6 displays the contrasting evaluation of the DCNN+Bi-GRU approach and previous schemes regarding negative metrics for big data classification. Furthermore, the DCNN+Bi-GRU method is compared to Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. It is important to note that for accurate classification of big data, the negative metric value should be minimized. Similarly, the DCNN+Bi-GRU method achieved lower negative metric values compared to traditional methods. In particular, the False Positive Rate (FPR) of the DCNN+Bi-GRU approach stands at 4.369 when the training percentage is set to 90%. This value is significantly lower in comparison to Bi-GRU (12.070), DCNN (16.682), LSTM (14.026), RF (16.694), RNN (33.824), ICNN [27] (14.654), RFC+MAPREDUCE [28] (11.462), and GRU (9.465), respectively. Simultaneously, the RF algorithm achieved the highest FNR, followed by LSTM and DCNN, whereas the DCNN+Bi-GRU method recorded the lowest FNR of 3.896. The evaluation has unveiled remarkable achievements in the field of big data classification, largely attributable to the implementation of the DCNN+Bi-GRU scheme. This improvement can be attributed to the application of improved fuzzy c-means clustering, improved normalization techniques, and the combination of deep hybrid classification methods.

Figure 6.

Validation of DCNN+Bi-GRU versus conventional methods with respect to negative metrics.

4.7 Performance evaluation of other measures for dataset 1

To evaluate the effectiveness of big data classification, a comprehensive analysis of various metrics is conducted for the DCNN+Bi-GRU method, comparing it with Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU, as illustrated in Fig. 7. To guarantee the effective classification of big data, the DCNN+Bi-GRU approach should demonstrate superior scores across these other metrics. Likewise, when the training rate is set at 70%, the F-measure of the DCNN+Bi-GRU methodology stands at 95.080, even though the Bi-GRU is 88.805, DCNN is 84.818, LSTM is 81.004, RF is 84.870, RNN is 87.837, ICNN [27] is 90.899, RFC+MAPREDUCE [28] is 89.053 and GRU is 87.200, correspondingly. Furthermore, the NPV of the DCNN+Bi-GRU scheme significantly surpasses that of Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. Hence, The DCNN+Bi-GRU approach has consistently demonstrated superior performance over existing methods, consistently excelling across various metrics. This unquestionably highlights its exceptional potential for the accurate classification of big data.

Figure 7.

Validation of DCNN+Bi-GRU versus conventional methods with respect to other metrics.

4.8 Analysis of ROC curve for dataset 1

Figure 8 presents a comparative analysis of ROC curves for the DCNN+Bi-GRU method and Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU within the context of a big data classification framework. The ROC curve is generated by plotting the true positive rate against the false positive rate, with a particular focus on the 80% learning percentage. To improve the accuracy of big data classification, it is desirable for the ROC area to reach or exceed 95%. Similarly, when the false positive rate was set to 1.0, the DCNN+Bi-GRU method attained a true positive rate of 0.983, which exceeded the lower true positive rates achieved by Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. Hence, the remarkable performance observed in the ROC assessment highlights the DCNN+Bi-GRU method’s capability to achieve accurate classification of large-scale data.

Figure 8.

ROC curve analysis.

4.9 Ablation evaluation on DCNN+Bi-GRU for dataset 1

Table 2 presents the ablation analysis of the DCNN+Bi-GRU method, including models with conventional feature fusion, conventional normalization, conventional fuzzy C-means, and a model without feature extraction, all in the context of big data classification. A comprehensive ablation analysis is carried out to thoroughly assess the results of integrating or improving specific elements within the DCNN+Bi-GRU methodology. This process offers a deeper insight into the distinct contributions that these features bring to the overall performance of the DCNN+Bi-GRU framework. Furthermore, the specificity values for the different models are as follows: the DCNN+Bi-GRU scheme achieves a specificity of 0.945, the model with conventional feature fusion scores 0.900, the model with conventional normalization obtains 0.886, the model with conventional Fuzzy C-means records 0.805, and the model without feature extraction registers 0.859. In addition, the FPR of the DCNN+Bi-GRU $=$ 0.055, the model with conventional feature fusion $=$ 0.100, the model with conventional normalization $=$ 0.114, the model with conventional fuzzy C-means $=$ 0.195 and the model without feature extraction $=$ 0.141.

Table 2
Ablation study on DCNN+Bi-GRU, model with conventional feature fusion, model with conventional normalization, model with conventional Fuzzy C-means and model without feature extraction

Metrics	Model with conventional feature fusion	Model with conventional normalization	Model with conventional fuzzy c-means	Model without feature extraction	DCNN+Bi- GRU
Accuracy	0.914	0.899	0.829	0.850	0.942
FPR	0.100	0.114	0.195	0.141	0.055
F-measure	0.922	0.909	0.841	0.864	0.951
Specificity	0.900	0.886	0.805	0.859	0.945
FNR	0.075	0.091	0.152	0.157	0.060
NPV	0.906	0.885	0.822	0.807	0.916
Precision	0.920	0.910	0.834	0.885	0.962
MCC	0.851	0.825	0.672	0.727	0.879
Sensitivity	0.925	0.909	0.848	0.843	0.940

4.10 Statistical analysis of accuracy for dataset 1

The statistical evaluation of DCNN+Bi-GRU over Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28] and GRU for big data classification is illustrated in Table 3. To ensure the production of exceptionally precise outcomes, each model undergoes a stringent statistical evaluation procedure. This exhaustive assessment aims for accuracy and encompasses the scrutiny of critical statistical metrics, including minimum, average, standard deviation, median, and maximum values. Considering the median statistical metric, the DCNN+Bi-GRU gained an accuracy of 0.948, whilst the Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28] and GRU presented lower accuracy ratings of 0.881, 0.829, 0.805, 0.769, 0.825, 0.884, 0.879 and 0.852, respectively. In addition, under the maximum statistical metric, the DCNN+Bi-GRU scheme achieves an accuracy of 0.967, which surpasses the accuracy values of Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU.

Table 3
Statistical assessment on accuracy

Methods	Maximum	Standard deviation	Mean	Minimum	Median
Bi-GRU	0.893	0.022	0.874	0.843	0.881
DCNN	0.844	0.036	0.817	0.765	0.829
LSTM	0.858	0.028	0.817	0.798	0.805
RF	0.847	0.080	0.772	0.703	0.769
RNN	0.887	0.068	0.820	0.741	0.825
ICNN [27]	0.897	0.042	0.867	0.805	0.884
RFC+MAPREDUCE [28]	0.907	0.035	0.872	0.824	0.879
GRU	0.911	0.046	0.853	0.798	0.852
DCNN+Bi-GRU	0.967	0.014	0.950	0.936	0.948

4.11 K-fold analysis for dataset 1

Table 4 presents a comparative analysis of the K-fold evaluation conducted on the DCNN+Bi-GRU method and conventional strategies for the classification of big data. K-fold analysis, also known as K-fold cross-validation, is a method employed in the fields of machine learning and statistics to evaluate the effectiveness and consistency of a predictive model. In K-fold cross-validation, the initial dataset is partitioned into K equal-sized subsets, often called “folds.” The model is then subjected to K rounds of training and assessment, where each round employs a distinct fold as the validation set, while the remaining K-1 folds are employed for training. This method guarantees that each data point is employed for validation precisely once. Furthermore, the DCNN+Bi-GRU approach achieved the highest precision of 0.939, whereas Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU yielded lower precision scores.

Table 4
Analysis on K-Fold

Methods	Accuracy	FPR	F-measure	Sensitivity	NPV	Specificity	FNR	MCC	Precision
Bi-GRU	0.858	0.158	0.871	0.871	0.840	0.842	0.129	0.750	0.872
DCNN	0.801	0.219	0.812	0.817	0.793	0.781	0.183	0.691	0.807
LSTM	0.803	0.206	0.813	0.811	0.785	0.794	0.189	0.692	0.817
RF	0.769	0.253	0.783	0.787	0.755	0.747	0.213	0.617	0.780
RNN	0.825	0.217	0.838	0.861	0.836	0.783	0.139	0.698	0.817
ICNN [27]	0.851	0.194	0.864	0.888	0.865	0.806	0.112	0.738	0.841
RFC+MAPREDUCE [28]	0.851	0.152	0.863	0.854	0.823	0.848	0.146	0.736	0.873
GRU	0.828	0.196	0.842	0.846	0.814	0.804	0.154	0.695	0.839
DCNN+Bi-GRU	0.919	0.076	0.927	0.916	0.893	0.924	0.084	0.859	0.939

4.12 Statistical test for dataset 1

The difference between the two sets, given in standard error units, is measured by the $t$ -statistic. Since the $P$ -value quantifies the likelihood that an observation will fall between extreme $t$ -values, a low $P$ -value also suggests “significance.” Table 5 shows the statistical test analysis. The outcomes of the statistical tests verify whether or not the suggested model’s improvement over the baseline models is statistically significant. Using the $t$ -test with a two-tailed hypothesis and a 95% confidence level (i.e., 0.05), the improvement in the outcome is verified.

Table 5
Analysis of statistical test

	$T$ -test	$P$ -test
BIGRU	0.499411	0.617766
DCNN	$-$ 0.60179	0.547659
LSTM	0.599248	0.549349
RF	$-$ 0.80352	0.422154
RNN	$-$ 0.60179	0.547659
ICNN	$-$ 0.70256	0.482738
RFC+MAP REDUCE	$-$ 0.50117	0.616527
GRU	$-$ 0.60179	0.547659
DCNN+Bi-GRU	0.299737	0.764535

4.13 Wilcoxon test for dataset 1

There are two variations of the Wilcoxon test: the signed-rank test and the rank sum test. It compares two matched groups. Finding out if two or more sets of pairs differ from one another in a statistically significant way is the aim of the test. Table 6 shows the analysis of the Wilcoxon Test. The Wilcoxon value of the proposed model is ( $\sim$ 0.131668).

Table 6
Analysis of Wilcoxon test

Methods	Wilcoxon $p$ value
BIGRU	0.507801
DCNN	0.376344
LSTM	0.396144
RF	0.2278
RNN	0.376344
ICNN	0.317311
RFC+MAPREDUCE	0.674424
GRU	0.354539
DCNN+Bi-GRU	0.131668

4.14 Friedman test for dataset 1

A non-parametric test for examining randomized complete block designs is the Friedman test. When there might be more than two treatments, it is an expansion of the sign test. Table 7 shows the Friedman Test. There are k experimental treatments (k $\geqslant$ 2) according to the Friedman test’s assumptions. The frieman chi-square value of the proposed value is ( $\sim$ 0.838223).

Table 7
Analysis of friedman test

Methods	Friedmanchisquare $p$ value
BIGRU	0.644941
DCNN	0.457212
LSTM	0.486752
RF	0.233506
RNN	0.457212
ICNN	0.367879
RFC+MAPREDUCE	0.103031
GRU	0.424373
DCNN+Bi-GRU	0.838223

4.15 Performance analysis on Dataset 2

For Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU for the classification of huge data. Figure 9 presents the positive metric assessment for the DCNN+Bi-GRU approach. The model’s output should have high positive metric scores to accomplish accurate large data classification. Firstly, the DCNN+Bi-GRU technique consistently achieves accuracy ratings above 90%, indicating very accurate big data classification. Therefore, the accuracy values of Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU are lower. The DCNN+Bi-GRU approach, in particular, attained an accuracy of 98.43 at a training rate of 90. In contrast, the Bi-GRU is 82.360, DCNN is 73.705, LSTM is 75.812, RF is 80.535, RNN is 85.570, ICNN [27] is 84.822, RFC+MAPREDUCE [28] is 82.838, and GRU is 87.878. Furthermore, the DCNN+Bi-GRU strategy showed a specificity of 95.630 when the training percentage was set at 90%, while Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU all had lower specificity values.

Figure 9.

Analysis of positive metrics for dataset 2.

4.16 Performance evaluation of negative measures for dataset 2

Figure 10 compares the DCNN+Bi-GRU technique with earlier approaches in terms of negative metrics for big data categorization. Moreover, a comparison is made between the DCNN+Bi-GRU approach and Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. It is crucial to remember that the negative metric value needs to be as low as possible in order to accurately classify huge data. In a similar vein, the DCNN+Bi-GRU approach produced smaller negative metric values than the conventional approaches. Specifically, when the training percentage is set to 90%, the DCNN+Bi-GRU approach’s False Positive Rate (FPR) is 3.450. Compared to Bi-GRU (11.181), DCNN (13.793), LSTM (11.137), RF (12.785), RNN (24.932), ICNN [27] (14.654), RFC+MAPREDUCE [28] (11.462), and GRU (9.465), respectively, this number is much lower. The RF algorithm and LSTM and DCNN algorithms reached the greatest FNR simultaneously, whereas the DCNN+Bi-GRU approach recorded the lowest FNR of 3.796. The assessment has revealed impressive progress in the big data classification domain, mostly due to the application of the DCNN+Bi-GRU method. The implementation of enhanced normalization methods enhanced fuzzy c-means clustering, and the combination of deep hybrid classification approaches are responsible for this improvement.

Figure 10.

Analysis of negative measures for Dataset 2.

4.17 Performance evaluation of other measures for Dataset 2

As shown in Fig. 11, a thorough study of multiple metrics is carried out for the DCNN+Bi-GRU approach, comparing it with Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU to assess the efficacy of big data classification. The DCNN+Bi-GRU method needs to perform better on all of these additional criteria to ensure that huge data is classified effectively. The F-measure of the DCNN+Bi-GRU approach, meantime, is 96.190 when the training rate is set at 70%, even though the Bi-GRU is 88.805, the DCNN is 85.929, the LSTM is 81.004, the RF is 85.980, the RNN is 88.948, the ICNN [27] is 90.899, the RFC+MAPREDUCE [28] is 80.164, and the GRU is 88.312, respectively. Moreover, the DCNN+Bi-GRU strategy has a much higher net present value (NPV) than Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. Therefore, the DCNN+Bi-GRU strategy has continuously outperformed the state-of-the-art techniques, outperforming them on a wide range of parameters. This demonstrates its extraordinary potential for precise large data classification.

Figure 11.

Analysis on other measures for Dataset 2.

4.18 Statistical analysis of accuracy for dataset 2

Table 8 shows the statistical comparison of DCNN+Bi-GRU for big data classification vs Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU. This thorough analysis, which strives for precision, examines important statistical measures such as minimum, average, standard deviation, median, and maximum values. The DCNN+Bi-GRU achieved an accuracy of 0.9430 when the median statistical metric was taken into account. In contrast, the Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU demonstrated lower accuracy ratings of 0.8611, 0.8404, 0.8270, 0.8042, 0.8398, 0.7686, 0.8604, and 0.852, in that order. Furthermore, the DCNN+Bi-GRU scheme obtains an accuracy of 0.967 under the maximal statistical measure, outperforming the accuracy values of Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU.

Table 8
Statistical analysis for dataset 2

Method	Standard deviation	Mean	Median	Max	Min
BIGRU	0.028822	0.861191	0.867976	0.884966	0.823846
DCNN	0.03156	0.840429	0.844804	0.874034	0.798073
LSTM	0.031177	0.827014	0.818659	0.871304	0.799435
RF	0.054999	0.804286	0.801153	0.868746	0.74609
RNN	0.039096	0.839842	0.84016	0.882265	0.796784
ICNN	0.021095	0.768683	0.776209	0.784686	0.737628
RFC+MAPREDUCE	0.034359	0.860446	0.867631	0.889151	0.81737
GRU	0.033051	0.85245	0.857087	0.887628	0.807997
DCNN+Bi-GRU	0.018519	0.943078	0.939517	0.967304	0.925972

Table 9

Analysis of K-fold for dataset 2

	Accuracy	Sensitivity	Specificity	Precision	F_measure	MCC	NPV	FPR	FNR
BIGRU	0.8496	0.8618	0.8334	0.8634	0.8626	0.7420	0.8313	0.1665	0.1381
DCNN	0.7929	0.8089	0.7731	0.7989	0.8038	0.6842	0.7853	0.2268	0.1910
LSTM	0.7947	0.8025	0.7860	0.8083	0.8053	0.6847	0.7771	0.2139	0.1974
RF	0.7614	0.7787	0.7400	0.7725	0.7756	0.6108	0.7474	0.2599	0.2212
RNN	0.8167	0.8523	0.7748	0.8091	0.8300	0.6906	0.8272	0.2251	0.1476
ICNN	0.8425	0.8791	0.7981	0.8323	0.8550	0.7301	0.8566	0.2018	0.1208
RFC+MAPREDUCE	0.8421	0.8450	0.8392	0.8642	0.8543	0.7288	0.8150	0.1607	0.1549
GRU	0.8195	0.8378	0.7961	0.8304	0.8340	0.6879	0.8055	0.2038	0.1621
DCNN+Bi-GRU	0.9101	0.9070	0.9145	0.9296	0.9182	0.8506	0.8838	0.0854	0.0929

Table 10

Analysis of mann-whitney $U$ test

Methods	Mann-whitney $U$ -test
BIGRU	0.617511
DCNN	0.54733
LSTM	0.54902
RF	0.421758
RNN	0.54733
ICNN	0.482362
RFC+MAPREDUCE	0.616271
GRU	0.54733
DCNN+Bi-GRU	0.764498

4.19 K-fold analysis for dataset 2

K-fold analysis also referred to as K-fold cross-validation is a technique used to assess a prediction model’s efficacy and consistency. The original dataset is divided into K equal-sized subgroups, typically referred to as “folds,” for K-fold cross-validation. After that, the model is trained and evaluated for K rounds. In each round, a different fold is used as the validation set, while the remaining K-1 folds are used for training. Table 9 shows the analysis of the k-fold for dataset 2. This technique ensures that every data point is used exactly once for validation. Furthermore, the precision score of 0.929 was highest for the DCNN+Bi-GRU method, whereas lower values were obtained for Bi-GRU, DCNN, LSTM, RF, RNN, ICNN [27], RFC+MAPREDUCE [28], and GRU.

4.20 Statistical test for mann-whitney U test

The probability that assesses the evidence against the null hypothesis, or $p$ -value, is computed using the Mann-Whitney statistic. Table 10 shows the Mann-Whitney U Test. To compare two sample means from the same population and determine whether or not they are equal, one can use the non-parametric test known as $t$ . In situations where the $t$ -test assumptions are not satisfied or the data is ordinal, the Mann-Whitney U test is typically employed. If there is no real difference, there is a 5% chance of drawing the incorrect conclusion at a significance level of 0.05. To reject the null hypothesis, one must determine if the $p$ -value is less than or equal to the significance level.

4.21 Analysis of computational time

“Running time” or computation time is the amount of time needed to complete a computing task. When a computation is represented as a series of rule applications, the number of rule applications determines how long the computation takes. Table 11 shows the computational time analysis. The computational time of the proposed model is ( $\sim$ 43.07). As a result, the suggested model’s superiority over other existing models is demonstrated.

Table 11
Computational time analysis

Methods	Time (sec)
BIGRU	49.67724
DCNN	61.9715
LSTM	54.65746
RF	60.02476
RNN	64.99135
ICNN	73.37626
RFC+MAPREDUCE	80.49638
GRU	68.98085
DCNN+Bi-GRU	43.07433

5. Conclusion

In this paper, a Bigdata clustering and classification model is proposed with an improved fuzzy-based deep architecture under the Map Reduce framework. The initial step in the pre-processing stage was to split out the data from the bigdata collection by using an enhanced C-Means clustering method. Then the MapReduce framework was used to manage the preprocessed large data, which consists of mapper and reducer phases. Following the enhanced normalization method discussed in the mapper phase, a feature fusion strategy that blends correlation-based and entropy-based features was utilized. All of the mappers were then combined in the reduction process to create an acceptable feature. The classification procedure was completed by a deep hybrid model that combines a bidirectional gated recurrent unit with a deep convolutional network. In this instance, the improved score level fusion approach was applied to determine the classification’s ultimate outcome. In addition, regarding classification precision, recall, FNR, FPR, and other performance metrics, the suggested work outperforms as compared to conventional models in their performance.

Footnotes

Abbreviations

References

Jiang

. Health big data classification using improved radial basis function neural network and nearest neighbor propagation algorithm. IEEE Access. 2019; 7: 176782-9.

Dener

. RFSE-GRU: Data Balanced Classification Model for Mobile Encrypted Traffic in Big Data Environment. IEEE Access. 2023; 11: 21831-47.

Zhang

. Research and analysis for real-time streaming big data based on controllable clustering and edge computing algorithm. IEEE Access. 2019; 7: 171621-32.

Nafis

Biswas

. A secure technique for unstructured big data using clustering method. International Journal of Information Technology. 2022; 14(3): 1187-98.

Brahmane

Krishna

. Big data classification using deep learning and apache spark architecture. Neural Computing and Applications. 2021; 33: 15253-66.

Xing

Bei

. Medical health big data classification based on KNN classification algorithm. IEEE Access. 2019; 8: 28808-19.

Fattahi

Moattar

Forghani

. Improved cost-sensitive representation of data for solving the imbalanced big data classification problem. Journal of Big Data. 2022; 9(1): 1-24.

. Computational intelligence based sustainable computing with classification model for big data visualization on map reduce environment. Discover Internet of Things. 2022; 2(1): 2.

Weinberg

Last

. Selecting a representative decision tree from an ensemble of decision-tree models for fast big data classification. Journal of Big Data. 2019; 6(1): 1-7.

10.

Yang

Liu

Wang

Martínez

. A micro-extended belief rule-based system for big data multiclass classification problems. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2018; 51(1): 420-40.

11.

Angelov

Zhao

. Self-organizing fuzzy inference ensemble system for big streaming data classification. Knowledge-Based Systems. 2021; 218: 106870.

12.

Elkano Ilintxeta

Sanz Delgado

Barrenechea Tartas

Bustince Sola

Galar Idoate

. CFM-BD: a distributed rule induction algorithm for building compact fuzzy models in Big Data classification problems. IEEE Transactions on Fuzzy Systems, 2020; 28(1): 163-177.

13.

Hernández

Zamora

Sossa

Téllez

Furlán

. Hybrid neural networks for big data classification. Neurocomputing. 2020; 390: 327-40.

14.

Zhu

Chen

. Big data image classification based on distributed deep representation learning model. IEEE Access. 2020; 8: 133890-904.

15.

González

Pérez

Romero-Zaliz

. An incremental approach to address big data classification problems using cognitive models. Cognitive Computation. 2019; 11: 347-66.

16.

Banchhor

Srinivasu

. Integrating Cuckoo search-Grey wolf optimization and Correlative Naive Bayes classifier with Map Reduce model for big data classification. Data & Knowledge Engineering. 2020; 127: 101788.

17.

Sleeman IV

Krawczyk

. Multi-class imbalanced big data classification on spark. Knowledge-Based Systems. 2021; 212: 106598.

18.

Jaiswal

Tiwari

Garg

Hossain

. Entity-aware capsule network for multi-class classification of big data: A deep learning approach. Future Generation Computer Systems. 2021; 117: 1-1.

19.

Huang

Liu

Chen

Wang

. Big data-driven correlation analysis based on clustering for energy-intensive manufacturing industries. Applied Energy. 2023; 349: 121608.

20.

Abhilasha

. Self-boosted with dynamic semi-supervised clustering method for imbalanced big data classification. International Journal of Software Innovation (IJSI). 2022; 10(1): 1-24.

21.

Wang

Gao

Yang

. Cutting the unnecessary long tail: cost-effective big data clustering in the cloud. IEEE Transactions on Cloud Computing. 2019; 10(1): 292-303.

22.

Banchhor

Srinivasu

. Analysis of Bayesian optimization algorithms for big data classification based on Map Reduce framework. Journal of Big Data. 2021; 8(1): 81.

23.

Jayasri

Aruna

. Big data analytics in health care by data mining and classification techniques. ICT Express. 2022; 8(2): 250-7.

24.

Juez-Gil

Arnaiz-González

Rodríguez

García-Osorio

. Experimental evaluation of ensemble classifiers for imbalance in big data. Applied soft computing. 2021; 108: 107447.

25.

Mostafa

Ramadan

Elfarouk

. Renewable energy management in smart grids by using big data analytics and machine learning. Machine Learning with Applications. 2022; 9: 100363.

26.

Liu

. Language database construction method based on big data and deep learning. Alexandria Engineering Journal. 2022; 61(12): 9437-46.

27.

Cao

Zhang

. Big Data Analysis and Prediction System Based on Improved Convolutional Neural Network. Computational Intelligence and Neuroscience. 2022.

28.

Lakshmanaprabu

Shankar

Ilayaraja

Nasir

Vijayakumar

Chilamkurti

. Random forest for big data classification in the internet of things using optimal features. International journal of machine learning and cybernetics. 2019102609-18.

29.

Liu

. A novel modified kernel fuzzy c-means clustering algorithm on image segementation. In 2011 14th IEEE International Conference on Computational Science and Engineering 2011 (pp. 621-626. IEEE.

30.

Yang

Liao

Wang

Bak

Chen

. Improved Euclidean Distance Based Pilot Protection for Lines with Renewable Energy Sources. IEEE Transactions on Industrial Informatics. 2022; 18(12): 8551-62.

31.

Wang

Sun

. Application of a novel improved Manhattan distance on bearing fault diagnosis.

32.

Khalifa

Gazzah

BenAmara

. Adaptive score normalization: a novel approach for multimodal biometric systems. International Journal of Computer and Information Engineering. 2013; 7(3): 376-84.

33.

Souza

Premebida

Araújo

. High-order conditional mutual information maximization for dealing with high-order dependencies in feature selection. Pattern Recognition. 2022; 131: 108895.

34.

Peng

Han

Jia

. Pearson correlation and transfer entropy in the Chinese stock market with time delay. Data Science and Management. 2022; 5(3): 117-23.

35.

Song

Guo

Mei

. Feature selection using principal component analysis. In 2010 international conference on system science, engineering design and manufacturing informatization 2010 (Vol. 1, pp. 27-30). IEEE.

36.

Rani

Sharma

. Study of different image fusion algorithm. International Journal of Emerging Technology and Advanced Engineering. 2013; 3(5): 288-91.

37.

Lin

Jeng

. Using feature fusion and parameter optimization of dual-input convolutional neural network for face gender recognition. Applied Sciences. 2020; 10(9): 3166.

38.

Liu

Wang

Xin

. Bi-directional gated recurrent unit neural network based nonlinear equalizer for coherent optical communication system. Optics Express. 2021; 29(4): 5923-33.

39.

Hamd

Rasool

. Score level fusion technique for Human identification. In IOP Conference Series: Materials Science and Engineering 2020; (Vol. 990, No. 1, p. 012021). IOP Publishing.

40.

https://archive.ics.uci.edu/ml/datasets/lung+cancer.

41.

https://research.unsw.edu.au/projects/unsw-nb15-dataset.

Bigdata clustering and classification with improved fuzzy based deep architecture under MapReduce framework

Abstract

Keywords

1. Introduction

Table 1 Features and challenges of conventional insider threat detection model

3. Proposed big data clustering and classification model

3.1.1 Improved fuzzy c-means clustering

A) Modified Euclidean distance

B) Average of manhattan distance

3.3.1 DCNN

4.1 Experimental setup

4.2 Dataset1 description

4.3 Dataset2 description

4.4 Performance evaluation

4.5 Performance evaluation of positive measures for Dataset 1

Table 2 Ablation study on DCNN+Bi-GRU, model with conventional feature fusion, model with conventional normalization, model with conventional Fuzzy C-means and model without feature extraction

Table 3 Statistical assessment on accuracy

Table 4 Analysis on K-Fold

Table 5 Analysis of statistical test

Table 6 Analysis of Wilcoxon test

Table 7 Analysis of friedman test

Table 8 Statistical analysis for dataset 2

4.20 Statistical test for mann-whitney U test

4.21 Analysis of computational time

Table 11 Computational time analysis

Footnotes

Abbreviations

References

Table 1
Features and challenges of conventional insider threat detection model

Table 2
Ablation study on DCNN+Bi-GRU, model with conventional feature fusion, model with conventional normalization, model with conventional Fuzzy C-means and model without feature extraction

Table 3
Statistical assessment on accuracy

Table 4
Analysis on K-Fold

Table 5
Analysis of statistical test

Table 6
Analysis of Wilcoxon test

Table 7
Analysis of friedman test

Table 8
Statistical analysis for dataset 2

Table 11
Computational time analysis