Movie recommendation system via fuzzy decision making based dual deep neural networks

Abstract

The internet and social networks produce an increasing amount of data. There is a serious necessity for a recommendation system because exploring through the huge collection is time-consuming and difficult. In this study, a multi-modal classifier is introduced which makes use of the output from dual deep neural networks: GRU for text analysis and Faster R-CNN for image analysis. These two networks reduce overall complexity with minimal computational time while retaining accuracy. More precisely, the GRU network is utilized to process movie reviews and the Faster RCNN is used to recognize each frames of the movie trailers. Gated Recurrent Unit (GRU) is a well-known variety of RNN that computes sequential data across recurrent structures. Faster RCNN is an enhanced version of Fast RCNN, it combines with the rectangular region proposals and with the features is extract by the ResNet-101. Initially, the trailer of the movie is manually splitted into frames and these frames are pre-processed using fuzzy elliptical filter for image analysis and the movie reviews are also tokenized for text analysis. The pre-processed text is taken as an input for GRU to classify offensive and non-offensive movies and the pre-processed images are taken as an input for Faster R-CNN to classify violence and non- violence movies based on the extracted features from the movie trailer. Afterwards, the four classified outputs are given as input for fuzzy decision-making unit for recommending best movies based on the Mamdani fuzzy inference system with gauss membership functions. The performance of the dual deep neural networks was evaluated using the specific parameters like specificity, precision, recall, accuracy and F1 score measures. The proposed GRU yields accuracy range of 97.73% for reviews and FRCNN yields the accuracy range of 98.42% for movie trailer.

Keywords

Movie recommendation deep learning Mamdani fuzzy inference system Gated Recurrent Unit Faster R-CNN

1 Introduction

Individuals have been left perplexed by the proliferation of online information in recent years [1]. A recommender system is a technology that filters information so that it can direct users to goods or services they are interested in based on their preferences [2]. As one of the many information sharing systems, the recommender system is vital to enhancing businesses and assisting users [4]. The internet and social networks create an increasing amount of information. There is a serious need for a recommendation system because exploring through the huge collection is time-consuming and difficult [7]. Recently, several movie recommendation systems have been developed based on this context. They provide specific movies based on past user data instead of all types of genre movies [8]. Currently, the traditional recommendation system can make a suggestion based on specific characteristics, such as the popularity of a movie, user behaviors, or movie content [12].

The analysis of video automatically could be used to resolve a number of issues that are currently too expensive or too time-consuming for people to resolve themselves [15]. There are several effective ML techniques that can classify photos with around 94% accuracy according to 1,000 different labels, however video-based applications have proven to be more challenging [16]. An ML algorithm of this type is not capable of handling such a task effectively and efficiently due to its high level of complexity. The concept of automatic video analysis is wide and offers a variety of study opportunities, including context analysis, element detection, and action recognition [17, 18]. Recent research uses Deep Convolutional Neural Networks (CNNs) for video processing, yielding promising preliminary findings that may pave the way for further investigation of a wide range of applications [20, 21]. The most advanced technique for supervised image classification is convolutional neural networks (CNNs), which use ideas from image analysis to provide some level of size, location, and distortion partially invariant. These theories are based on the recognition of main features, such as aligned edges, end-points, and corners, which may be found anywhere in the image [26, 27]. After blending these visual features, the successive layers detect novel, higher-order characteristics, mimicking human learning. As a result, this study introduces a dual deep neural network with Gated Recurrent Unit (GRU) and Faster R-CNN for classifying offensive and non-offensive movies based on reviews and trailers [32]. The main contribution of our proposed model are as follows,

The main purpose of this study is to present a best movie recommendation system based on the novel fuzzy decision making based dual deep neural networks.

Initially, the movie reviews are gathered from the Internet Movie Database (IMDB) and the movie trailers are gathered from the MovieLens, in which this video sequences are splitted into frames.

In pre-processing stage, the noise artifacts of each frames are eliminated by fuzzy elliptical filter and tokenization is applied for text analysis.

The pre-processed text is taken as an input for GRU for classifying offensive and non-offensive movies based on the reviews and pre-processed images are taken as an input for Faster R-CNN for classifying violence and non-violence movies based on the extracted features from the movie trailer.

Afterwards, the four classified outputs are given as input for fuzzy decision-making unit for recommending best movies based on the Mamdani fuzzy inference system with gauss membership functions.

The proposed deep neural networks are evaluated based on its sensitivity, specificity, accuracy and F1 score.

The remainder of this study was organized with five sections as follows. Section-2 outlines with the literature survey, Section-3 includes the proposed Dual deep neural network, Section-4 comprises with results and discussion and finally Section-5 encloses with conclusion and future enhancement.

2 Literature survey

In recent days several deep learning and machine learning techniques were introduced by the researchers mainly to progress the accuracy for recommending movies. Some of those techniques are studied briefly in this section.

In 2022 Chen.X., et al., [3] devised an ensemble framework based on the analytic network process (ANP) and fuzzy comprehensive evaluation model for calculating the impact of the Arctic oil spill. The fuzzy comprehensive evaluation model was used to quantify the various factors and determine the level of risk associated with an oil spill. According to experiment results, this framework accurately quantified the influence of an oil spilling event in Arctic area, and thus the level of potential risk was accurately determined.

In 2021 Widiyaningtyas, et al., [4] had devised a User Profile Correlation-based Similarity (UPCSim) that analyze the genre data and the user profile data. Each user data was utilized to calculate the weights of similarity between the user rating values and the user behavioral values. To determine the weights of both similarities, correlation coefficients are used between user profile data and user ratings and behaviors. A reduction in MAE and RMSE of 1.4% makes this approach more accurate than the prior algorithm.

In 2021 Sujithra., et al., [5] had developed a hybrid method for recommending movies to the target user that takes into account both recent purchases and demographic information. The demographic characteristics help with the cold start issue. Based on the experimental results that the proposed approach was successful when more transaction data was taken into account along with higher ratings.

In 2021 Behera, et al., [6] had introduced a hybrid model for a constrained boltzmann machine and content K-nearest neighbors to propose movies. In order to capture both content-based filtering and collaborative filtering based on higher-order models, researchers developed a weighted hybrid approach based on content k-NN and RBM. The suggested method combines the impacts of both collaborative and content-based filtering with MovieLens benchmark datasets to suggest best films to the active users.

In 2021 Lang, et al., [9] had proposed the field-aware factorization machine (FFM) to address the issue of movie rating prediction and assist users in choosing the right films for academic purposes. Clustering technique is also implemented in FFM for adding additional fields, which will further increase the availability of the technique. In order to address the sparseness of the input data, the clusters are used as a special field in the FFM approach. The experimental findings show that the suggested work has minimize RMSE value.

In 2021 Tahmasebi, et al., [10] had developed a social movie recommendation system using deep autoencoder network with information from Twitter. This hybrid approach, which may be easily expanded to other aspects of product-based recommendation system relies on deep autoencoder networks, content-based and collaborative filtering. In order to attain improved accuracy, MovieTweetings and the Open Movie Database (OMDB) were employed that minimizes sparsity and improves the amount of data needed.

In 2021 Choudhury, et al., [11] had devised trust matrix measure, the combination of weighted trust propagation with user similarity. A non-cold user went through many algorithms with a trust filter, and a cold user produced the best possible score using their own preferences. To suggest the appropriate movie to the user, four different recommendation structures, including the BPNN model, SVD model, DNN, and DNN with Trust, were evaluated. In terms of accuracy, the DNN with trust model came out on top with 83% and a 0.74 MSE value.

In 2021 Wan.Z., et al., [15] had developed a fuzzy DEMATEL approach to classify 10 factors that have influenced Northern Sea Route development into four categories. An uncertain and complex decision-making environment was used to determine the cause-effect chain using fuzzy decision-making and the DEMATEL technique. A method that examines the relationship between political factors, sea ice conditions, and climate elements, as well as their policy implications that can influence the decisions of shipping corporations, was developed with this method.

In 2021 Yang., et al., [16] proposed an approach to categorizing spare parts via deep CNN based on transfer learning theory. Initially, the network was built using the pre-trained VGG-19. The hierarchical inference model of spare components was then constructed to visualize data and convert images. Based on the five-fold cross-validation, the model performed exceptionally well with an average accuracy of 96.36%.

In 2020 Bathla, G., et al., [17] had devised AutoTrustRec model which integrates deep learning architecture and increase the recommendation accurateness for large scale heterogeneous complicated data. A common layer was used in the autoencoder, this strategy feeds ratings, direct trust values, and indirect trust values into neural networks. The three public datasets were used to prove that MAE and RMSE are enhanced with AutoTrustRec model.

In 2020 Gupta, et al., [19] had developed Movie Recommendation System with collaborative filtering and K-NN models were employed primarily focus on improving the accuracy of output in contrast to content-based filtering. This method eliminates the shortcomings of content-based filtering while relying on cosine similarity and k-nearest neighbor in order to employ a collaborative filtering method.

In 2020 Singh, et al., [22] had proposed a movie recommendation approach that would provide all-encompassing suggestions based on movie popularity and/or genre. It involves several deep learning techniques to implement Content-Based Recommendation Systems. Moreover, this model describes the challenges faced by content-based recommendation systems and the solutions we have come up with in order to overcome those challenges.

In 2019 Aljunid and Manjaiah [23] had introduced a movie recommendation framework with ALS and Apache Spark. It focuses on the selection of ALS algorithmic variables that affect the effectiveness of a structure with a robust RS. The algorithmic elements of ALS that can affect the performance of a movie recommendation engine. In order to evaluate a model, different criteria were used, including execution time, RMSE of rating prediction, and rank at which the model was trained.

In 2014 Baziuke., et al., [24] had developed a fuzzy logic approach for merging rank and quantitative data with respect to initial correctness using the Marine Life Information Network. This method combines expert knowledge and fuzzy logic techniques, and it gives marine environmental decision support systems the option of utilizing numerical scales. It can also be used to assess sensitivity explicitly. This method can also be employed for other environmental indicators and assessments.

In 2012 Filippo., et al., [25] designed an Artificial Neural Network (ANN) to improve sea level predictions. The ANN was trained using hourly time series of atmospheric pressure, wind, and harmonically generated tides during 1982 as input data, and hourly time series of measured tides as output data. Among the Cananéia and Ilha Fiscal time series, the coefficients of correlation between projected and measured water levels were 0.88 and 0.98, respectively.

From the literature review, various deep and machine learning techniques were focused on reviews or trailers for recommending the best movies. Moreover, the existing techniques take more time since they require exploring a large collection of movies, which is the most challenging task. The proposed dual deep neural network addresses these issues by using both reviews and trailers as input without comprising on accuracy. The pre-processed text is taken as an input for GRU for classifying offensive and non-offensive movies based on the reviews and pre-processed images are taken as an input for Faster R-CNN for classifying violence and non-violence movies based on the extracted features from the movie trailer. By utilizing these two networks the overall complexity is reduced with minimal computational time while persevering the detection accuracy. Afterwards, the Mamdani fuzzy inference system with gauss membership functions is used for recommending best movies.

3 Proposed movie recommendation system

In this section, we proposed the dual deep neural networks with GRU and Faster R-CNN for recommending best movies based on fuzzy decision-making system. The overall workflow of the proposed methodology is depicted in Fig. 1.

Fig. 1

The overall workflow of the proposed movie recommendation system.

The movie reviews and movie trailers are gathered from the openly available datasets. Initially, the video sequences are splitted into frames and the noise artifacts of each frames are eliminated by fuzzy elliptical filter and tokenization is applied for text analysis. The pre-processed text is taken as an input for GRU for classifying offensive and non-offensive movies and pre-processed images are taken as an input for Faster R-CNN for classifying violence and non-violence movies based on the extracted features from the movie trailers. Mamdani fuzzy inference system with gauss membership functions is used for recommending best movies.

3.1 Dataset description

3.1.1 Dataset for movie reviews

IMDB dataset having 50000 movie reviews for text analytics. The dataset contains much more data than previous benchmark data sets for binary sentiment classification. The reviews are divided into 25000 each for training and testing. Each set contains 50 percent favorable reviews and 50 percent negative reviews. So, predict the number of negative and positive reviews used for classification algorithms.

3.1.2 Dataset for movie trailers

MovieLens is a social networking site and web-based recommendation system that recommends movies to users based on their movie ratings and movie trailers. There have been a number of studies based on the MovieLens data sets. It is estimated that there are approximately 11 million ratings for approximately 8500 movie trailers. In total, 270,000 users provided 26,000,000 ratings and 750,000 tag applications for 45,000 movies. Furthermore, it includes information for 1,100 tags with 12 million relevance ratings.

3.2 Data pre-processing

3.2.1 Pre-processing of movie reviews

A preprocessing method is crucial to enhancing the effectiveness of the models. To transform irregular texts into proper format and improve the quality of the text dataset, the pre-processing stage is the first step in text mining. In this study, the movie reviews are pre-processed by tokenization. Tokenization involves separating a stream of texts into tokens such as phrases, words, symbols, or other useful items. The goal of tokenization is to examine the words in a sentence or phrases.

3.2.2 Pre-processing of movie trailers

Afterwards, the movie trailers are splitted into frames and these frames (images) are pre-processed using fuzzy elliptical filter. Automatic movie trailer classification is one of the several tasks that may be accomplished through video analysis. In order to determine when a shot in a video occurs, each frame must be compared to its neighboring frame. In low-similarity frames, a scene border is detected. The HSV (Hue, Saturation, Value) colour system is used to calculate frame similarity through histogram intersection. Each histogram has 16 bins that include eight bins for hue, four bins for saturation, and four bins for value. Equation (1) describes the calculation of the histogram intersection. $M_{r} = \sum_{j ɛ allbins} \min (H_{r} (j), H_{r - 1} (j))$ (1)

Such a method is effective in identifying instantaneous scene changes. Consequently, when gentle transitions take place, the approach fails. To overcome this issue M_r is iteratively smoothen with the fuzzy elliptical filter. The fuzzy elliptical filter measures the degree of membership for processed data using a logic variable with multi-dimensional functional membership. By determining the ripple factor and linearity of the input data, it determines the linearity and angle of the network. The elliptical filter has a separate adjustment for equalizing ripple behaviour with a faster transition rate between the passband and stopband. The nth order of magnitude frequency response is expressed as; ${| M_{r} (j ω) |}^{2} = \frac{1}{1 + ɛ^{2} C_{r}^{2} (ω)}$ (2)

Where, C_r (ω) denotes the Chebyshev rational function of ω evaluated from definite ripple characteristics. In-order to evaluate the denoising efficiency, mean square error (MSE) is used as the measure. Filtering frequencies with significant attenuation with an elliptical filter with sharp cut-off frequency is ideal since errors on both sides of the cut-off frequency are minimised. As a result of fuzzy systems, the optimal combination of processed signals is selected to reduce the MSE of the reconstructed images.

3.3 Dual deep neural networks

The proposed Dual deep neural networks are combination of GRU shown in (Fig. 2) and Faster R-CNN (Fig. 3) for classifying movies based on pre-processed movie reviews and movie trailers. Based on the output of the both neural networks the non-offensive movies are recommended for the users.

Fig. 2

Structure of GRU cell.

Fig. 3

Structure of Faster R-CNN.

3.3.1 Gated Recurrent Unit for movie reviews

The GRU is a well-known recurrent neural network (RNN) that can compute sequential data using its recurrent architecture. The GRU architecture consists of five layers: input, embedding, recurrent, output, and classification as softmax. This architecture uses word embeddings as inputs and then sends them through a GRU learn layer to gather semantic information before passing them on to a classification layer. An LSTM unit with a gating system that regulates data flow, but without dedicated storage. In order to handle the flow of data from prior activation when calculating a new candidate state, GRU calculates two gates known as update and reset gates. The update gate ties past activation and new candidate activation into fresh activation. Every hidden state at time-step t is computed using these equations $Update gate : U_{t} = σ_{g} (W_{U} X_{t} + V_{U} Z_{t - 1} + {bias}_{U})$ (3) $Reset gate : R_{t} = σ_{g} (W_{R} X_{t} + V_{R} Z_{t - 1} + {bias}_{R})$ (4)

$\begin{matrix} Candidate state : \\ {\hat{Z}}_{t} = \tanh (W_{\hat{Z}} X_{t}) + V_{\hat{Z}} (R_{t} * Z_{t - 1}) \end{matrix}$ (5) $Final output : Z_{t} = U_{t} * Z_{t - 1} + (1 - U_{t}) * {\hat{Z}}_{t - 1}$ (6) where * denotes the element-wise multiplication and σ_g refers to the logistic sigmoid activation function. W_u, W_r and W_z are the feedforward weights and V_u, V_r and $V_{\hat{z}}$ are the recurrent weights of the reset gate, update gate and output candidate state respectively. bias_u and bias_r are the biases of these update and reset gates.

In GRU networks for text categorization, softmax regression is widely used to distinguish between offensive and non-offensive words. A probabilistic explanation is presented with the results of the computation. In this model, a fixed-dimensional input is provided by the lower layer, which is then refined by the classification layer, and then a softmax activation function is used to calculate the probability that two categories will predict each other and it is derived as, $℘ = \frac{exp (w_{o}^{t} s^{'} + b_{0})}{\sum_{t = 1}^{c} \exp (w_{o}^{t} s^{'} + b_{0})}$ (7) where ℘ refers to the predicted distribution of the texts, P_s is the chosen text labels, w_o^t and b_o are the variables of the learnable softmax classifier.

3.3.2 Faster R-CNN for movie trailers

In the faster R-CNN, a region proposal network (RPN) was used, and the input is the feature map from the first CNN, which creates a bounding box and the possibility that it contains an object. From the RPN we can deduce the most likely bounding boxes. Despite their lack of precision, these bounding boxes can still be analyzed by pooling regions of interest (RoI). The most accurate bounding box coordinates can be determined after RoI pooling for each region.

A small network is slides on top of the convolutional feature map to create the proposal and it generates a series of rectangular object proposals with the objectness score. The concept of anchor box is introduced to avoid pyramids of filters or pyramids of images. After that, each region is mapped to each reference anchor box, allowing for the object detection at different scales. An anchor is focused at the sliding window and it is linked with a scales and aspect ratios. At every sliding-window spot, in chorus forecast several region proposals, where k is denoted as the maximum number of possible proposals for each location region. So, the regression layer has 4k outcomes represents the coordinates of k anchor boxes, and the classification layer produces 2k scores outcomes that assess the probability of an object or not being an object for each proposition and the parameterization of k proposals is based on k reference boxes. In addition, for choosing an anchor, the anchors are categorized into two types that are foreground class and background class.

As a classification index, an anchor box is choosing with an intersection-over-union (IoU) relationship between ground-truth box and anchor box. The IoU overlap ratio is expressed as, $IoU = \frac{S_{an} \cap S_{gr}}{S_{an} \cup S_{gr}}$ (8) where S_an denotes anchor boxes and S_gr denotes ground-truth box. In FR-CNN the multi-task loss is minimized by an objective function by using ResNet-50, and the loss function is derived as follows, $L (x, y) = \frac{1}{N_{c}} \sum_{i} L_{c} (x, x^{*}) + δ \frac{1}{N_{r}} \sum_{i} L_{r} (y, y^{*})$ (9) where L_c refers to the classification loss function, and L_r refers to the regression loss function. N_c and N_r are the normalization coefficients of L_c and L_r respectively and δ denotes the weight parameter between L_c and L_r. Here, x is the probability of an object predicted in the anchor. The ground-truth value x* is 1 it is a positive anchor, and is 0 then it is a negative anchor, y is a vector containing the four parametric variables of the predicted bounding boxes, and y*is the ground-truth box related with a positive anchor. The loss function of object detection is defined as, $L_{obj} (a, b, c, d) = L_{cls} (a, b) + δ [b ⩾ 1] L_{reg} (c, d)$ (10)

Where w refers to predicted class value, x denotes the true class score, y is the true box coordinates and z is the predicted coordinates. The loss function of classification is defined as, $L_{cls} (x, x^{*}) = - \log (x, x^{*} + (1 - x) (1 - x^{*}))$ (11)

The loss function of regression is represented as, $L_{reg} (y, y^{*}) = G (y - y^{*})$ (12) where G defines the robust loss function and it is expressed as, $G (f) = {\begin{matrix} 0.5 f^{2}, | f | < 1 \\ | f | - 0.5, otherwise \end{matrix}$ (13)

Furthermore, this model accepts each feature map as input and reducing the computational cost significantly and enhances the learning capability of the model.

3.4 Fuzzy decision-making unit

Based on these classification results, Mamdani fuzzy is built with a minimum number of rules for the recommending the best movies for the users. The fuzzy logic controller enables the translation of a linguistic control approach based on expert data into an autonomous control strategy. MFIS consists of the following five blocks: Database, Rule Base, Decision–making Unit, Fuzzification interface unit and Defuzzification interface unit as shown in Fig. 4.

Fig. 4

Structure of Decision-making fuzzy inference system.

The fuzzification module converts the crisp value of the four classification outputs (violence, non-violence, non-offensive and offensive) into fuzzy quantities. The Decision-making unit includes the fuzzy rule base which comprises of three rules that could recommend the best movies by evaluating the inputs such as violence, non-violence, non-offensive and offensive. Mamdani Inference system with the gauss membership functions (MF) was used in the fuzzy rule base system that ranges from 0 to 1 for inputs. Four rules were implemented in this proposed system and it has shown in Table 1. These rules are crucial in the classification of various groups of subjects.

Table 1

Rules for fuzzy inference system

Rules	GRU		FRCNN		Output
	Offensive	Non-offensive	Violence	Non-violence
1	✓	x	✓	x	Not recommended
2	✓	x	x	✓	Recommended
3	x	✓	✓	x	Not recommended
4	x	✓	x	✓	Recommended

Table 2

The input parameters of fuzzy logic

Parameters Membership functions	GRU		FRCNN		Output values
	Offensive	Non-offensive	Violence	Non-violence	R	NR
Min	0.0	0.6	0.0	0.49	1	51
Max	0.59	1.0	0.48	1.0	50	100

Fuzzy rules

* If review is offensive and trailer is violence then movie is not recommended

* If review is offensive and trailer is non-violence then movie is recommended

* If review is non-offensive and trailer is violence then movie is not recommended

* If review is non-offensive and trailer is non-violence then movie is recommended

According to a set of rules, fuzzy inference interprets the values in an input and assigns values to the output. A fuzzy interference rule is applied to the output of the Dual deep neural networks. The created fuzzy system incorporates two membership functions inputs from the networks to represent an output for recommending best movies. There are two input elements, each with four linguistic terms, resulting in a total of 2^∧2 = 4 fuzzy rules. $R = ⋃_{i = 1}^{4} S_{1} (i) \land S_{1} (i) \land S_{1} (i) \land p (i)$ (14)

Where R refers to the rule databased and p(i) denotes the chance of i-th rule. The centre of gravity defuzzification method was used to transfer the Fuzzy Inference output to the crisp output, $Z = \frac{μ A (z) Zdz}{μ A (z) d}$ (15)

Whereas μA(z) denotes the Membership function of the gathered output. By using Equation (14) the crisp values of the fuzzy inference system outputs are calculated for recommending best movies to the users based on the classification results.

4 Results and discussion

In this section, the proposed Dual deep neural networks are evaluated with different measures namely accuracy, specificity, precision, recall and F1 score based on the collected datasets. The benchmark includes the total accuracy rate, which expressly specified and evaluated, and the performance of the proposed Dual deep neural networks. Furthermore, the evaluation of the proposed Dual deep neural networks with classic ML and DL models is also provided.

4.1 Performance analysis

The efficiency of the proposed Dual deep neural networks can be measured using evaluation metrics accuracy, specificity, precision, recall and F1 score. $Specificity = \frac{TN}{TN + FP}$ (16) $Precision = \frac{TP}{TP + FP}$ (17) $Recall = \frac{TP}{TP + FN}$ (18) $Accuracy = \frac{TP + TN}{TN + TP + FN + FP}$ (19) $F 1 score = 2 (\frac{Precision * Recall}{Precision + Recall})$ (20) where TP and TN represent true positives and negatives of the samples, FP and FN indicates false positives and negatives of the samples. The efficiency of the proposed Dual deep neural networks for the classification of offensive and offensive movies are illustrated in Table 3 and it is graphically represented in Fig. 5.

Table 3

Performance evaluation of the proposed network

Datasets	Classes	Specificity	Precision	Recall	F1 score	Accuracy
IMDB (GRU)	Offensive	95.42	96.34	96.25	96.29	97.95
	Non-offensive	94.26	96.02	95.15	95.58	97.52
MovieLens (FRCNN)	Violence	96.22	97.18	96.28	96.72	98.81
	Non-violence	95.14	97.01	96.04	96.52	98.03

Fig. 5

Performance analysis of the Proposed Dual deep neural networks.

The performance result acquired by the proposed Dual deep neural networks for two different datasets i.e., IMDB for reviews and MovieLens for movie trailers is exposed in Table 3.

Among these two datasets, the proposed FRCNN model has acquired high accuracy in MovieLens dataset as shown in Fig. 5. The performance is based on the specificity, precision, recall, f1 score and accuracy, and FRCNN obtains the overall accuracy of 98.42%. Likewise, the overall accuracy of the proposed GRU is 97.73% based on movie reviews from IMDB dataset.

The ROC generated for two datasets is depicted in Figs. 6 7. The MovieLens dataset has a higher AUC. The proposed FRCNN model achieved higher AUC of 0.98 for the MovieLens dataset that can be measured via TPR and FPR parameters.

Fig. 6

ROC of the proposed GRU based on the IMDB dataset.

Fig. 7

ROC of the proposed FRCNN based on the MovieLens dataset.

Figure 8 displays the accuracy curve with epochs on x-axis and accuracy on y-axis, the accuracy of the method improves when the epochs are increased.

Fig. 8

Testing and Training accuracy of GRU.

Figure 9 shows the epoch versus loss curve, which illustrates when the epochs are increased, the loss of the model diminishes. The GRU model achieves high accuracy range for IMDB dataset.

Fig. 9

Training and Testing loss curve of GRU.

Figure 10 displays the accuracy curve with epochs and accuracy on both the axis, the accuracy of the method improves when the epochs are increased. Figure 11 shows the epoch versus loss curve, which illustrates when the epochs are increased, the loss of the model diminishes. So, the predicted outcomes of the proposed FRCNN framework are highly reliable for MovieLens dataset. The study started by determining the number of training epochs necessary to attain the maximum level of testing accuracy. According to the results, the classification accuracy of GRU and FRCNN was achieved at 5 and 50 training epochs, with testing accuracy of 97.73% and 98.42% respectively.

Fig. 10

Training and Testing accuracy curve of FRCNN.

Fig. 11

Training and Testing loss curve of FRCNN.

4.2 Comparative analysis

The efficiency of each ML classifiers was estimated to validate that the results of the proposed Dual deep neural networks achieves high accuracy. The comparative evaluation was made between the proposed GRU and FRCNN with four machine learning classifiers such as SVM, MLP, ELM and RF. The performance evaluation was done using various metrics like specificity, precision, recall, f1 score and accuracy of each ML techniques and the accuracy attained by proposed classifier is 99.07%, which was higher than the traditional ML methods.

From Table 4, the comparison was made between four ML techniques based on the performance metrics for recommending best movies. The proposed GRU achieves the accuracy range of 97.73% for reviews and the proposed FRCNN achieves the accuracy range of 98.42% for movie trailers. However, the traditional techniques not performed well compared to the proposed classifier. GRU model progresses the overall accuracy range by 9.44%, 8.62%, 0.13% and 2.38%; the FRCNN progresses the overall accuracy range by 10.07%, 9.26%, 0.83% and 3.06% better than RF, SVM, MLP and ELM respectively.

Table 4
Comparison between ML classifiers

Classifiers Specificity Precision Recall F1 score Accuracy

RF 87.2 85.6 84.10 84.7 88.5

SVM 88.7 86.2 85.3 85.7 89.3

MLP 94.54 93.10 95.18 94.4 97.6

ELM 93.8 90.14 91.21 90.7 95.4

GRU 94.84 96.18 95.7 95.93 97.73

FRCNN 95.68 97.07 96.16 96.62 98.42

Classifiers	Specificity	Precision	Recall	F1 score	Accuracy
RF	87.2	85.6	84.10	84.7	88.5
SVM	88.7	86.2	85.3	85.7	89.3
MLP	94.54	93.10	95.18	94.4	97.6
ELM	93.8	90.14	91.21	90.7	95.4
GRU	94.84	96.18	95.7	95.93	97.73
FRCNN	95.68	97.07	96.16	96.62	98.42

From Table 5, the comparison was made between different DL networks based on the performance metrics by obtaining the appropriate percentage of accuracy in classification. However, the traditional networks not performed well compared to the proposed networks. GRU improves the overall accuracy range by 6.68%, 8.62%, 5.35% and 4.22%; the FRCNN the overall accuracy range by 7.33%, 9.06%, 6.01% and 4.87% better than AlexNet, LeNet, ResNet and LSTM respectively.

Table 5

Comparison between Deep neural networks

Classifiers	Specificity	Precision	Sensitivity	F1 score	Accuracy
AlexNet	89.2	87.5	84.6	86.0	91.2
LeNet	88.7	86.2	85.3	85.7	89.5
ResNet	90.6	89.0	88.7	88.8	92.5
LSTM	94.8	94.4	92.5	93.4	93.6
GRU	94.84	96.18	95.7	95.93	97.73
FRCNN	95.68	97.07	96.16	96.62	98.42

The proposed GRU and FRCNN outperforms in this spot and obtains higher accuracy range of 97.73% and 98.42%. This network was trained with two different datasets for different classification. Figure 12 result shows that the proposed model does not require a model with complex architecture to recommend best movies with high accuracy than the traditional networks. The proposed GRU yields accuracy range of 97.73% for reviews and FRCNN yields the accuracy range of 98.42% for movie trailer.

Fig. 12

Performance comparison between the traditional DL networks.

Table 6 illustrates the accuracy evaluation of proposed model with different datasets such as IMDB, OMDB, MovieLens, MovieTweeting and Flixter. Due to the inclusion of both reviews and trailers in the MovieLens dataset, the proposed network achieves low accuracy. The accuracy is 97.25% for MovieLens dataset which is comparatively 1.18% low than the attained results for movie trailers using same dataset. Moreover, the proposed model yields better accuracy for text analysis using IMDB than MovieLens. Compared to MovieTweeting and Flixter datasets the proposed model obtains better results in OMDB dataset. From this comparison, the proposed network shows high accuracy level in IMDB dataset, but it does not perform well in the different datasets.

Table 6

Accuracy comparison of Proposed network with different datasets for text analysis

Datasets	Accuracy (%)
IMDB	97.73
OMDB	96.02
MovieLens	97.25
MovieTweeting	92.46
Flixter	93.61

Table 7 portrayals the performance comparison of the proposed Dual deep neural network with the existing methods interms of accuracy. The proposed network attains the average accuracy of 98.07% for recommending best movies by using IMDB and MovieLens datasets. According to this comparison analysis, CNN-LSTM [28] achieves 9.91% lower accuracy than the proposed GRU based on IMDB dataset. Moreover, the prior models like VGG-16 [29], DNNRec [30] and LSTM-RNN [31] attains low level of accuracy than the proposed FRCNN based on MovieLens dataset. The FRCNN model progresses the overall accuracy range by 5.47%, 6.39% and 1.83% better than [29], [30] and [31] respectively. The proposed Dual deep neural network yields higher accuracy than the existing models for both the movie recommendation datasets.

Table 7

Accuracy comparison amid the proposed and existing models

Authors	Method	Accuracy
Achin &Vanita [28]	CNN-LSTM	88.35%
Kaushik., et al. [29]	VGG-16	92.7%
Kiran., et al. [30]	DNNRec	91.8%
Liu., et al. [31]	LSTM-RNN	96.27%
Proposed	Dual deep neural network	98.07%

5 Conclusion

In this study, a multi modal classifier is introduced which makes use of the output from dual deep neural networks with GRU for text analysis and Faster R-CNN for image analysis. Initially, the trailer of the movie is manually splitted into frames and these frames are pre-processed using fuzzy elliptical filter for image analysis and the movie reviews are also tokenized for text analysis. The pre-processed text is taken as an input for GRU to classify offensive and non-offensive movies and the pre-processed images are taken as an input for Faster R-CNN to classify violence and non- violence movies based on the extracted features from the movie trailer. Afterwards, the four classified outputs are given as input for fuzzy decision-making unit for recommending best movies based on the Mamdani fuzzy inference system with gauss membership functions.

The performance of the dual deep neural networks was evaluated using the specific parameters like specificity, precision, recall, accuracy and F1 score measures. The proposed GRU yields accuracy range of 97.73% for reviews and FRCNN yields the accuracy range of 98.42% for movie trailer. The proposed Dual deep neural networks attain the average accuracy of 98.07% for recommending best movies by using IMDB and MovieLens datasets. The proposed model progresses the overall accuracy range by 9.91%, 5.47%, 6.39% and 1.83% better than CNN-LSTM, VGG-16, DNNRec and LSTM-RNN respectively. From the experimental fallouts, the proposed dual deep neural networks produce better results in both text and image analysis. But the proposed model does not attain better accuracy in text analysis. The classification results can be improved using word embedding in conjunction with advanced deep learning networks using the MovieLens dataset. In future work, we will develop a novel 3D deep learning architecture that captures the relationship among frames within a trailer in order to recommend the best movies.

References

Khurana

and Dhingra

, An improved hybrid and knowledge based recommender system for accurate prediction of movies, In 8th International Conference on Computing for Sustainable Global Development (INDIACom), pp. 881–886, IEEE, 2021, March.

Gourammolla

and Gokila

, HCB Machine Learning Approach For Movie Recommendation System, In 6th International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 1186–1190, IEEE, 2022, May.

Chen

, Liu

R.W.

, Wu

, Han

and Zhao

, Quantifying Arctic oil spilling event risk by integrating an analytic network process and a fuzzy comprehensive evaluation model, Ocean & Coastal Management 228 (2022), 106326.

Widiyaningtyas

, Hidayah

and Adji

T.B.

, User profile correlation-based similarity (UPCSim) algorithm in movie recommendation system, Journal of Big Data 8(1) (2021), 1–21.

Sujithra Alias Kanmani

, Surendiran

and Ibrahim

S.P.

, Recency augmented hybrid collaborative movie recommendation system, International Journal of Information Technology 13(5) (2021), 1829–1836.

Behera

D.K.

, Das

, Swetanisha

and Sethy

P.K.

, Hybrid model for movie recommendation system using content K-nearest neighbors and restricted Boltzmann machine, Indones J Electr Eng Comput Sci 23(1) (2021), 445–452.

Bhalse

and Thakur

, Algorithm for movie recommendation system using collaborative filtering, Materials Today: Proceedings, 2021.

Shen

, Zhou

and Chen

, Collaborative filtering-based recommendation system for big data, International Journal of Computational Science and Engineering 21(2) (2021), 219–225.

Lang

, Liang

, Huang

, Chen

and Zhu

, Movie recommendation system for educational purposes based on field-aware factorization machine, Mobile Networks and Applications 26(5) (2021), 2199–2205.

10.

Tahmasebi

, Ravanmehr

and Mohamadrezaei

, Social movie recommender system based on deep autoencoder network using Twitter data, Neural Computing and Applications 33(5) (2021), 1607–1623.

11.

Choudhury

S.S.

, Mohanty

S.N.

and Jagadev

A.K.

, Multimodal trust-based recommender system with machine learning approaches for movie recommendation, International Journal of Information Technology 13(2) (2021), 475–482.

12.

Keshava

M.C.

, Srinivasulu

, Reddy

P.N.

and Naik

B.D.

, Machine learning model for movie recommendation system, International Journal of Engineering Research & Technology (IJERT) 9(04), 2021.

13.

Ahmed

, Imtiaz

M.T.

and Khan

, Movie recommendation system using clustering and pattern recognition network. In IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pp. 143–147. IEEE, 2018, January.

14.

Ali

S.M.

, Nayak

G.K.

, Lenka

R.K.

and Barik

R.K.

, Movie recommendation system using genome tags and content-based filtering, In Advances in Data and Information Sciences pp. 85–94. Springer, Singapore, 2018.

15.

Wan

, Nie

, Chen

, Ge

, Zhang

and Zhang

, Key barriers to the commercial use of the Northern Sea Route: View from China with a fuzzy DEMATEL approach, Ocean & Coastal Management 208 (2021), 105630.

16.

Yang

, Yang

, Yao

and Fan

S.D.

, A transfer learning-based convolutional neural network and its novel application in ship spare-parts classification, Ocean & Coastal Management 215 (2021), 105971.

17.

Bathla

, Aggarwal

and Rani

, AutoTrustRec: Recommender system with social trust and deep learning using autoencoder, Multimedia Tools and Applications 79(29) (2021), 20845–20860.

18.

Singh

, Nayyar

and Solanki

, Multilingual opinion mining movie recommendation system using RNN, In Proceedings of first international conference on computing, communications, and cyber-security (IC4S 2019) (pp. 589–605). Springer, Singapore, 2020.

19.

Gupta

, Thakkar

, Gupta

and Rathore

D.P.S.

20.

Zhang

, Wang

, Yuan

and Jin

, Personalized real-time movie recommendation system: Practical prototype and evaluation, Tsinghua Science and Technology 25(2) (2020), pp. 180–191, July.

21.

Pecune

, Murali

, Tsai

, Matsuyama

and Cassell

, A model of social explanations for a conversational movie recommendation system, In Proceedings of the 7th International Conference on Human-Agent Interaction (2019), pp. 135–143.

22.

Singh

R.H.

, Maurya

, Tripathi

, Narula

and Srivastav

, Movie recommendation system using cosine similarity and KN, International Journal of Engineering and Advanced Technology 9(5) (2020), 556–559.

23.

Aljunid

M.F.

and Manjaiah

D.H.

, Movie recommender system based on collaborative filtering using apache spark. In Data management, analytics and innovation, (2019), pp. 283–295. Springer, Singapore.

24.

Baziukė

, Jusčenko

and Šiaulys

, Switching to numerical scale in marine environmental decision support systems: Fuzzy logic approach, Ocean & Coastal Management 101 (2014), 35–41.

25.

Filippo

, Torres

A.R.

Jr , Kjerfve

and Monat

, Application of Artificial Neural Network (ANN) to improve forecasting of sea level, Ocean & Coastal Management 55 (2012), 101–110.

26.

Reddy

M.M.

, Kanmani

R.S.

and Surendiran

, Analysis of Movie Recommendation Systems; with and without considering the low rated movies. In 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE) (2020), pp. 1–4. IEEE., February

27.

Ahuja

, Solanki

and Nayyar

, Movie recommender system using K-Means clustering and K-Nearest Neighbor. In 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (2019), pp. 263–268. IEEE., January.

28.

Jain

and Jain

, In 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (2019), pp. 263–268. IEEE., January.

29.

Kaushik

, Gupta

and Bhatia

, A movie recommendation. System using Neural Networks, International Journal of Advance Research, Ideas and Innovations in Technology 4(2) (2018), 425–430.

30.

Kiran

, Kumar

and Bhasker

, DNNRec: A novel deep learning based hybrid recommender system, Expert Systems with Applications 144 (2020), 113054.6.

31.

Liu

, Choi

W.H.

and Liu

, Personalized movie recommendation method based on deep learning, Mathematical Problems in Engineering, 2021.

32.

Kumar

, De

and Roy

P.P.

, Movie recommendation system using sentiment analysis from microblogging data, IEEE Transactions on Computational Social Systems 7(4) (2020), 915–923.