MDEEPFIC: Food item classification with calorie calculation using modified dragonfly deep learning network

Abstract

Foods are very essential for living beings for providing energy, development and preserve their existence. It plays a vital role in promoting health and preventing illness. Nowadays, many people are suffered from obesity, they tend to maintain their body weight by consuming a sufficient number of calories in their routine life. In this research, a novel Modified Deep Learning-based Food Item Classification (MDEEPFIC) approach has been proposed to categorize the different food items from the dataset with their calorie values. Initially, the images are processed using the sigmoid stretching method to enhance the image quality and remove the noises. Consequently, the pre-processed images are segmented using Improved Watershed Segmentation (IWS2) algorithm. Recurrent Neural Network (RNN) is used to extract features like shape, size, textures, and color. The extracted features are then normalized using the modified dragonfly technique for same food calorie calculation. Bidirectional Long Short-Term Memory (Bi-LSTM) is utilized to classify food products based on these pertinent aspects. Finally, using food area volume and calorie and nutrition measures based on mass value, the calorie value of the categorized food item is calculated. The efficiency of the proposed method was calculated in terms of specificity, precision, accuracy, and recall F-measure. The proposed method improves the overall accuracy of 4.99%, 8.72%, and 10.4% better than existing Deep Convolution Neural Network (DCNN), Faster Recurrent convolution neural network (FRCNN), Local Variation Segmentation based Support Vector Machine (LSV-SVM) method respectively.

Keywords

Food bidirectional long short-term memory improved watershed recurrent neural network dragon fly algorithm

1 Introduction

Overweight and obesity are defined as an abnormal or excessive deposit of fat that is harmful to one’s health. [1]. Physical activity [2], calorie consumption that is too high, and hereditary factors are all major contributors to obesity [3, 4]. Dietary evaluation is now exceedingly challenging and largely rely on patient self-reported data [5]. The dietary evaluation typically entails many processes, including segmentation [6], recognition [7], weight calculation, and mass calculation [8, 9].

Calories are essential in determining how much energy is consumed by our bodies [10]. According to recent study, obese people are more likely to experience serious health issues like high blood pressure, cardiac arrest, type 2 diabetes, high blood cholesterol, breast and colon cancer, and breathing difficulties [11]. The patient is required to keep a daily food diary as part of obesity therapy, but most patients difficulties to quantify or manage their intake since they are malnourished and lack self-control [12].

Obesity prevalence and central obesity vary from 11.7 to 32.1 percent and 17 to 36.4 percent, respectively, as per the Indian Council of medical research [13]. The Journal Lancet conveys that the survey of 2018, India got the third rank in obesity. Early prevention of obesity means, people must take a balanced number of calories, nutrients, carbohydrates and do some physical activities and exercises [14]. A lot of techniques are involved to identify the number of calories, carbohydrates, and nutrients [15]. However, most of these systems have flaws, such as difficulty in using them or huge computation mistakes. Moreover, the existing techniques have some limitations such as low computational time and misclassification and this leads to get the error values of the measuring calories. To overcome this challenge, a novel Deep Learning-based Food Item Classification (MDEEPFIC) approach has been proposed to categorize the different food items from the dataset with their calorie values.

Initially, the images are processed using the sigmoid stretching method to enhance the image quality and remove the noises.

Consequently, the pre-processed images are segmented using Improved Watershed Segmentation (IWS2) algorithm.

A recurrent neural network (RNN) is utilized to extract characteristics like shape, size, textures, and color. The extracted features are then normalized using the modified dragonfly technique for same food calorie calculation.

Based on these relevant features the Bi-LSTM is used to classify the food items.

Finally, using food area volume and calorie and nutrition measures based on mass value, the calorie value of the categorized food item is calculated.

The remainder of the paper is arranged as follows. The literature review is summarized in Section 2, followed by a discussion of the proposed model and related algorithms in Section 3, then a performance analysis and protocol comparison of the suggested method in explained Section 4. Section 5 encloses with conclusion.

2 Related works

This section provides an overview of certain machine or deep learning-based algorithms for food image processing. Some of those techniques are illustrated briefly in this section.

In 2017 Minija et al., [16] proposed a classification of food images using a sphere-structured support vector machine. The segmented food items are classified by employing Sphere Shaped SVM classifiers based on an FCM algorithm. Calorie values are calculated automatically by identifying food items and calculating their nutritional values. Based on the proposed method, 95% of accuracy is achieved, which proves to be better in terms of classification.

In 2021 Manjunathan et al., [17] proposed a faster deep-learning algorithm for segmentation and calculating the calorie and nutrition content. Here, the histogram equalization method is utilized to enhance the pixel brightness. Next, the histograms of oriented gradients and local binary patterns are used to extract the features based on size, shape, and color. The goal of this work presents a feasible method for patients and nutritionists to compute and manage daily food consumption but, the processing speed is low.

In 2017 Kohila et al., [18] developed a fuzzy c-means algorithm was including segmentation for extracting image features have been used as a morphological technique. The standard deviation, mean, area, volume, variance, and density are the features retrieved in this technique, which will assist in the determination of calorific value. Here, this technique was not suitable to implement the smartphone application for calculating the calorie content of a variety of meals.

In 2017 Farooq et al., [19] developed a convolution neural network (CNN) for extracting the features from the food images. The images are collected from the Pittsburgh fast-food image database for training linear support vector machine (SVM) was utilized. For classification, features from three separate fully connected layers of convolution neural network (CNN) were employed. It secures poor accuracy and it processed only a small number of datasets.

In 2019 Wasif et al., [20] put forward to faster RCNN (F-RCNN) used for detection, and the grab-cut algorithm is used for segmentation and feature extraction. After that, the probing object is used to calculate the known volume by determining the volume of the food items. In this case, the link between mass and calories is used to compute the calories in the food items. After then, the volume is used to determine the calories in the food item, however, this process is highly complex.

In 2017 Minija et al., [21] developed a neural network classifier for classification and measuring calories. The segmentation procedure includes both quick rejection and multi-scale segmentation. Following the extraction of the characteristics, a feed-forward neural network is utilized to detect the segmented area. The proposed method’s results are 96% accurate, which results in superior classification performance.

In 2020 Latif et al., [22] put forward a deep convolution neural network (DCNN) for determining the automated fruit calories. The goal of this paper is to develop an app that can virtually provide an accurate assessment of the calories in fruits and vegetables particularly. People of various age groups will be able to use the applications. While using this application, the internet is not necessary. Here, it is cost-effective and the operating speed is slow.

In 2021 Deshmukh et al. [23] presented a calorie calculation utilizing a machine learning method. The suggested method is carried out in three stages: first, Faster R-CNN is used to identify the item, then the Grab algorithm is used to segment it. After segmenting the food product, they measure its volume. Finally, we calculate the food item’s calories.

In 2019, Sukvichai, K., et al. [24] proposed an easy way to determine nutrition by using Convolutional Neural Network (CNN). YOLO v3 network is chosen as the classifier and localizer for ingredients, and the Mobile Net network is chosen as the food categorization network. Although performance is extremely poor, the network can classify food, identify ingredients, and locate them with at least 80% accuracy.

In 2018 Minija et al., [25] developed a local variation segmentation (LVS) based multi kernel based SVM (LVS-SVM) in image processing for detection and segmentation. This paper also suggests a nutritional evaluation method to assist users and nutritionists in determining and maintaining healthy eating habits in everyday life. It increases categorization accuracy and aids in calorie value calculation.

The main drawbacks identified with the prior methodologies are poor accuracy, wide variability in calorie amounts, misclassification, and database sets with few entries. The proposed method employs a classification strategy with a novel Modified Deep Learning-based Modified Food Item Classification (MDEEPFIC) approach to overcome these drawbacks.

3 Proposed MDEEPFIC methodology

In this section, a novel Modified Deep Learning based Food Item Classification (MDEEPFIC) approach has been proposed to categorize the different food items from the dataset with their calorie values. Initially, the images are processed using the sigmoid stretching method to enhance the image quality and remove the noises. Consequently, the pre-processed images are segmented using Improved Watershed Segmentation (IWS2) algorithm. Recurrent Neural Network (RNN) is utilized to extract features like shape, size, textures, and color. The extracted features are then normalized using the modified dragonfly technique for same food calorie calculation. The Bi-LSTM is utilized to categorize the food items based on these relevant features. Finally, using I food area volume and (ii) calorie and nutrition measures based on mass value, the calorie value of the categorized food item is calculated. The architectural structure of the proposed technique is shown in Fig. 1.

Fig. 1

Overall block of MDEEPFIC framework.

3.1 Data acquisition

In this research, the dataset from [22] has taken; Twenty different types of fruits and vegetables, total of 20 varieties, make up the dataset used in this study. 8 classes for fruits and 8 classes for vegetables make up the 16 separate categories into which the data is divided. The five fruits and vegetables distributed to each class are different. Some classes, though, only include a combination of 4 fruits or vegetables. On various sized, shaped, and colored plates, fruits, and vegetables were photographed. Approximately 4000 images per class make up the dataset’s total of over 41,509 images.

3.2 Preprocessing

Generally, the preprocessing process is going to enhance the image quality and remove the noises. These images do not contain any noises and pixel brightness enhancement is needed. Here, initialized to enhance the image pixel quality using a pixel brightness transformation based on the sigmoid stretching method. The sigmoid stretching in pixel brightness transformation is under pixel brightness and transformations are determined by the attributes of the pixel itself. In pixel brightness transformation, the value of the output pixel is relying only on the congruent value of the input pixel. The sigmoid function can be thought of as a continuous, non-linear activation function. Additionally, it is known as the logistic function.g (u, v) $g (u, v) = \frac{1}{1 + e^{(c * (t - fs (u, v)))}}$ (1)

The above equation (1) implies fs (u, v) – original image; g (u, v) – improved pixel value; t – threshold value, c – contrast factor. By altering the ‘n’ and ‘t’ it is possible to tailor and regulate the overall contrast enhancement and adjust the determined amount of lightness and darkness.

3.3 Segmentation using Improved Watershed (IWS2) algorithm

The pre-processed images are segmented using the Enhanced Watershed segmentation method. Typically, watershed transformation is used to segment data based on zones. A mathematical method for segmenting morphology that is based on topology theory is called the Improved Watershed Segmentation (IWS2) algorithm. Watersheds are referred to as the boundaries between regions that gather rainwater in the geography that supports the idea. In watershed segmentation, each pixel’s grayscale value is taken into account as the point’s height and the image is assumed to be a topological landform in geodesy. Each local minimum and its surrounding regions are referred to as a basin, and a basin’s edge is referred to as a watershed. The improved watershed segmentation algorithm’s precise process is as follows:

Morphological Restoration Filter: Pre-smoothing filters often perform a proper job of eliminating noise and irregular features, but they lose information about the contour boundaries, altering the contour of the region. Filtering and denoising the food photos can greatly protect the objective’s edge shape data. Additionally, it does not result in a change in the rebuilt image’s contour. The definition of morphological restoration is

S_{x + 1} = (S_{x} \oplus S_{t e^{l}}) \cap^{n}

(2)

Where: S_{te
^l} denotes structural element, n denotes the actual image, and S_x denotes the outcome of the most recent iteration. Based on the initial and final acts, the morphological initial and final restoration action is defined as $H = {IN}_{S_{t e^{l}}}^{res} [{FI}_{S_{t e^{l}}}^{res} (n), n)$ (3) where: ${IN}_{S_{t e^{l}}}^{res}$ denotes the restorations initial action and ${FI}_{S_{t e^{l}}}^{res}$ denotes the restoration final action

Markup Extraction: Following morphological restoration filtering of food photos, nonsensical segmentation results with a high ratio can be accepted by marker extraction. Only the minimum value of the objective area is allowed to be kept in order to prevent over segmentation problems. Selecting the threshold Th^d and extracting the smallest value whose depth is smaller than Th^d is the key problem with this technique.

Phase 1: Assuming that Th^d is a threshold, Th^d divides the objective image into two groups: greyscale of f0, 1, 0: Th^d comprise the objective group V₀, while Th^d+1, Th^d+2, 0, and L-1 comprise the objective group V₀.

Phase 2: Identify the background group V₀ inter-group variance and the objective group V₀ intra-group variation. The variation within the group is as follows:

$\begin{matrix} V a^{2} (℘) = ℘_{0} V a^{2} (0) + ℘_{1} V a^{2} (1) \\ = \sum_{m}^{T h^{d}} {[m - \bar{R_{0}}]}^{2} ℘_{m} + \sum_{a = T h^{d} + 1}^{L - 1} {[m - \bar{R_{1}}]}^{2} ℘_{m} \end{matrix}$ (4) where the likelihood of the background group V₁ occurring is ℘₁ and the likelihood of the objective group V₀ occurring is ℘₀. Differentiation between classes is

$\begin{matrix} V a^{2} (x) = ℘_{0} {(\bar{R_{1}} - \bar{R_{t}})}^{2} + ℘_{0} {(\bar{R_{1}} - \bar{R_{t}})}^{2} \\ = ℘_{0} ℘_{1} {(\bar{R_{1}} - \bar{R_{0}})}^{2} \end{matrix}$ (5) The overall variance is $V a^{2} (t) = V a^{2} (x) + V a^{2} (℘)$ (6)

Phase 3: find the optimum threshold. The ideal threshold is reached when utilizing the sorting search strategy, Va² (℘) finds its lowest value or Va² (x) finds its highest value. ${Th}^{d} = \underset{0 ⩽ T h^{d} < L}{arg} {V a^{2} (x)}$ (7)

Watershed Transformation Based on Marks: To force the marker to appear at any rate value, the gradient-recreated image H is separated using the drawn-out least change technique after the method is used to construct the Th^d.

On the gradient image H^mk, the watershed segmentation process is carried out using the minimum value forced minimum approach.

H_{EWS} = EWS (H^{mk})

(8) where A transformation is shown by EWS (), and a value is H_EWS.

3.4 Classification using optimized RNN with Bi-LSTM

The segmented result is fed into Regression Neural Network (RNN) layer. The RNN is optimized using the modified dragonfly algorithm which helps to normalize the parameter of the network for achieving better accuracy. Here RNN is utilized to extract the features like shape, size, texture, and colour of the image. The proposed Optimized RNN with the BiLSTM method is shown in Fig. 2.

Fig. 2

Optimized RNN with BiLSTM method.

When using an RNN, all the components of the input vector have roughly the same weight, unlike feedforward neural networks. The model may handle sequences of different lengths by simply reusing the weights. Another advantage is that fewer network-learning parameters (weights) are required. Additionally, using both the input vector and those images from the segmented phase, a number of the outputs for the subsequent stage was calculated using the data from the segmented phase. The formulas used to calculate the intermediate findings are abbreviated as units (blocks). Therefore, the relation given in Equations (9) and (10) can be utilized to define a block for the most fundamental type of recurrent network: $R_{t} = e_{1} (Z_{00} R_{t - 1} + Z_{0 m} m_{t} + a_{0}$ (9) $\hat{L_{t}} = e_{2} (Z_{pm} p_{t} + a_{p})$ (10)

Where m_t is a vector of input sequences and t is the iteration utilized to generate recurrent relations, The activation functions are represented by e₁, e₂. Z₀₀, Z_0m, Z_pm, a₀ and a_p, are the weight matrices.

3.4.1 Optimized RNN for best features

The RNN network is optimized using Modified dragonfly algorithm which help to normalize the parameter of the network for achieving better accuracy. In the Modified dragon-fly algorithm is used for optimization and it is used to resolve a problem by changing the interior parameter weights of the neural network to achieve the certain goals. The flow chart of the modified dragonfly optimization is depicted in Fig. 3.

Fig. 3

Flowchart of Modified dragonfly optimization.

In this study, MDA is employed to enhance the system’s precision. The WOA is used in this MDA to update the optimal solution. Combining the dragon fly and whale optimization algorithms could speed up global convergence and improve the method’s viability. As a result, a combined approach is used to avoid the weaknesses brought on by the employment of individual optimization techniques. As a result, the WOA is used to update the optimal position. The next section provides a mathematical description of the modified dragonfly method and itslayers.

3.4.1.1 Layers of modified dragonfly optimization

Layer 1: A random input is generated using equation (11) in this layer $P = [\begin{matrix} S_{ig}^{11} & \begin{matrix} S_{ig}^{12} & \begin{matrix} \dots & S_{ig}^{1 a} \end{matrix} \end{matrix} \\ \begin{matrix} S_{ig}^{21} \\ \begin{matrix} \begin{matrix} . \\ . \end{matrix} \\ S_{ig}^{b 1} \end{matrix} \end{matrix} & \begin{matrix} \begin{matrix} S_{ig}^{22} & \begin{matrix} \dots & S_{ig}^{2 a} \end{matrix} \end{matrix} \\ \begin{matrix} \begin{matrix} \begin{matrix} . \\ . \end{matrix} & \begin{matrix} \begin{matrix} . \\ . \end{matrix} & \begin{matrix} . \\ . \end{matrix} \end{matrix} \end{matrix} \\ \begin{matrix} S_{ig}^{b 1} & \begin{matrix} \dots & S_{ig}^{ba} \end{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix}]$ (11) where S_ig is the combined generation of generators.

Layer 2: Each dragonfly has a specific objective function based on equation (12). $Objectiv e_{Fn} = Min f (i, l)$ (12)

Where f (i, l) represents the system’s objective. If the Objective_Fn value is minimum, then the optimal solution is obtained. A fittest solution is updated based on the following five factors. Separation of dragonfly is shown in Fig. 4.

Fig. 4

Separation of dragonfly.

Layer 3: The following equation is used to evaluate the separation Sep_i between each i^th individual dragonfly ${Sep}_{i} = - \sum_{i = 1}^{k} P - P_{j}$ (13)

According to the equation above, P is considered the neighbour of P_j if there is less distance between P and P_j than the distance at present. A neighboring individual has k neighbors.

Layer 4: Dragonfly alignment Alig_i is evaluated by using the following equation for each i^th individual dragonfly. Alignment of dragonfly is shown in Fig. 5. $Ali g_{i} = \frac{\sum_{i = 1}^{k} V_{j}}{k}$ (14)

Fig. 5

Alignment of dragonfly.

Where, V_j denotes the velocity consistency of the dragonfly

Layer 5: In each i^th individual dragonfly, the cohesion Coh_i is evaluated using the following equation ${Coh}_{i} = \frac{\sum_{i = 1}^{k} p_{j}}{k} - p$ (15)

Cohesion of dragonfly is shown in Fig. 6.

Fig. 6

Cohesion of dragonfly.

Layer 6: In order to evaluate the attraction towards food source FS_i for i^th individual dragonflies, the following equation is used: $δ_{j} = p^{fs} - p$ (16) $σ_{j} = p^{ey} - p$ (17) where p^fs is represents the food source, and p^ey is represents the enemy source.

Layer 7: In equation (18), the WOA is assisted in the updating process to increase DFA performance accuracy through the searching behavior. $H = | \vec{X} . \vec{P_{rand}} - \vec{P} |$ (18) $P (t + 1) = \vec{P_{rand}} - \vec{Y} . \vec{H}$ (19)

Where, $\vec{P_{rand}}$ is denotes the random whales, $\vec{H}$ , $\vec{Y}$ and $\vec{X}$ are denotes the coefficient vectors. Attraction to food is shown in Fig. 7.

Fig. 7

Attraction to food.

3.4.2 Bidirectional LSTM

The optimal feature set is fed into the BiLSTM layer. LSTM belongs to the class of recurrent neural networks in the field of AI. To classify the food item along with calories, we combine RNNs and BiLSTM in this work. An RNN-BiLSTM consists of five layers: the sequence input layer, 150 hidden-unit RNNs, 250 hidden-unit bidirectional LSTMs, FC layer, SoftMax layer, and classification output layer.

LSTM networks, a type of RNN structure with bidirectional long-term memory (BiLSTM), are effective and reliable at simulating sequences that have numerous dependencies, including time, and may be used for a variety of purposes. Afterward, a BiLSTM layer was applied. Segmented food images are arranged time-based, which means that the previous environment has a significant influence on the current condition.

Using the BiLSTM model, it is possible to solve this problem effectively. Many self-parameterized regulating gates within the BiLSTM module allow state information to be read, written and cleared. The cell’s complete informational capacity would be accessible if that door were to be opened. Recall that the forget gate was not activated, which resulted in the prior cell’s procedure failing. If nothing happens, the information from before could be “lost.” The output gate can independently decide whether to communicate the most recent cell output and the final state.

A dropout layer was added above the BiLSTM layer to mitigate the overfitting problem. By utilizing these dropout layers, we can prevent the issue of overfitting. Dropout helps achieve the goal of reducing overfitting, which is pursued in conjunction with the expansion of neural network layers. The BiLSTM layer is followed by the FC layer, the SoftMax layer, and finally the classification layer.

3.5 Calculating the volume of food

Every food item image is subjected to the square grid to determine its surface area. Each square grid has a comparable number of pixels, which stands for the quantity of food. Equation (13) represents the overall area of the food image. ${Total}_{area} = \sum_{j = 1}^{N} p_{j}$ (20) where, N is the segmented image which has the total number of square grids and the total number of pixels in the j^th the segmented image is represented by p_j. Moreover, the area and depth obtained by rotating the images are used to calculate the food item’s volume. By generating specific error values when completing the food item classification, this method addresses the problem of matching complexity. Equation (21) represents the volume of the food item in the food image. $volume = {Total}_{area} * D$ (21) whereas, Total_area indicated the total area of the food item in the food image and D indicates the depth. The values of calories are evaluated by the volume and mass of the food item. The mass of the food item is expressed in equation (22). $m = ρ * volume$ (22) Hither, m refers to the mass of the food item, ρ is the density value of the food item. The calorie values of each segmented image are expressed in equation (23). $C = \frac{t x m}{mt}$ (23)

Whereas, C is the calorie value, t is the table which has the calorie content of the food, m is the mass of the food item and mt is the table that has the mass of each food item. The diet food consumes only a minimum level of energy only.

4 Result and discussion

MATLAB2019b is used to implement the proposed methodology. As a result, food images are trained in a novel Modified Deep Learning based Food Item Classification (MDEEPFIC) approach to classify the food items and include their calories. The output performance has been calculated, examined, and contrasted with the existing used classification methods. Existing and proposed techniques are compared based on their variations.

4.1 Calorie estimation

The volume (weight) of food and an estimate of the calorie values are used to determine the calorie values. The calories are widely used to calculate the total amount of energy in any food portion that contains the three primary dietary elements such as carbohydrates, proteins, and fats. The sample of calories table is shown in Table 1. A specific number of calories must be consumed by every day in an individual. If too much of calories are consumed in a day, it will lead to excess body weight.

Table 1
Common calories table

Food Name Measure Weight (in grams) Energy

Apple with skin 1 130 75

Potato 1 128 69

Orange 1 112 64

Tomatoes 1 125 32

Strawberry 1 150 81

Cucumber, raw 1 113 18

Banana 1 102 108

Grape juice 1 cup 100 60

Brinjal 1 300 75

Food Name	Measure	Weight (in grams)	Energy
Apple with skin	1	130	75
Potato	1	128	69
Orange	1	112	64
Tomatoes	1	125	32
Strawberry	1	150	81
Cucumber, raw	1	113	18
Banana	1	102	108
Grape juice	1 cup	100	60
Brinjal	1	300	75

4.2 Performance analysis

Performance was tested with accuracy, precision, specificity, f measure, and recall using several assessment metrics for a comparison study. A statistical analysis of the parameters can be found below. $Accuracy = \frac{TP + TN}{total no . of datas}$ (24) $precision = \frac{TP}{TP + FP}$ (25) $Specificity = \frac{TN}{TN + FP}$ (26) $recall = \frac{TP}{TP + FN}$ (27) $f mesure = 2 (\frac{precision * recall}{precision + recall})$ (28) where TP and TN represent true positives and negatives of the samples, FP and FN represent false positives and negatives of the data. Figure 8 shows the experimental results of the proposed technique.

Fig. 8

Experimental results of the proposed method.

Figure 8 depicts the results of the proposed MDEEPFIC with a sample of six different fruits and vegetable images. The input images (column-1) are pre-processed to eliminate the unwanted noise from the image which is shown in (column-2). The segmented food images are displayed in column-3. The segmented food images are classified and the calorie value is shown in column-4. In the experiment, the proposed method utilizes modified dragonfly optimization algorithm to provide a more accurate estimation of calorie value with varying sizes of fruits and vegetables.

Figure 9 illustrates that the proposed model has attained high accuracy in both training and testing. Improving the epoch value improves the model’s performance. In Fig. 10, the loss curve and epoch are displayed to demonstrate how the model loss decreases with increasing epoch size. As a result, the model’s predictions of outcomes were very accurate.

Fig. 9

Training and testing accuracy of the proposed method.

Fig. 10

Training and testing loss of the proposed method.

4.3 Comparative analysis

The effectiveness of existing methods was assessed to demonstrate that the proposed novel Food Item Classification with Calorie Calculation (MDEEPFIC) via the Deep Learning Network approach method works effectively with a high level of accuracy. Based on the F measure, recall, specificity, sensitivity, and accuracy, the proposed technique performs well. The accuracy rate demonstrates that the proposed approach is more effective than the prior techniques. The proposed method is compared to the existing LSV-SVM [25], F-RCNN [20], and DCNN [22] methods, respectively. Table 2. Illustrates the comparison of proposed techniques with the existing techniques.

Table 2
Comparison of proposed technique with existing techniques

Techniques Accuracy Precision Recall Specificity F-measure

LSV-SVM 87.37 90.1 88.4 85.15 86.48

F-RCNN 91.4 90.4 89.7 90.47 87.25

DCNN 93.05 96.15 94.5 93.5 91.43

Proposed (DF-DLN) 99.36 98.9 99.15 98.07 99.04

Techniques	Accuracy	Precision	Recall	Specificity	F-measure
LSV-SVM	87.37	90.1	88.4	85.15	86.48
F-RCNN	91.4	90.4	89.7	90.47	87.25
DCNN	93.05	96.15	94.5	93.5	91.43
Proposed (DF-DLN)	99.36	98.9	99.15	98.07	99.04

Figure 11 illustrates the numerical analysis of the Proposed novel Food Item Classification with Calorie Calculation via Deep Learning Network (MDEEPFIC) approach and Existing methods LSV-SVM [25, 26], F-RCNN [20], DCNN [22] methods. This comparison demonstrates that the proposed MDEEPFIC performs better than the existing approaches. The accuracy of proposed MDEEPFIC has maximum accuracy of 99.36%, whereas existing models like LSV-SVM have 87.37%, F-RCNN has 91.4% and DCNN has 93.05%. It demonstrates that the proposed approach is an effective one and yields a highly accurate outcome. The precision of the proposed MDEEPFIC is 10.4%, 7.6%, and 4.8% increased by existing LSV-SVM, F-RCNN, and DCNN methods. The increased precision is due to the use of significant classification from a large database. The Recall of the proposed MDEEPFIC is 9.2%, 7.8%, and 4.78% better than the existing methods. The retrieval for positive results increases as recall increases. The proposed MDEEPFIC has a maximum F-measure value of 99.04%, which is relatively high compared to the existing methods. When compared to existing techniques, the proposed technique delivers better specificity of 98.07% respectively.

Fig. 11

Performance of the proposed with the existing method.

5 Conclusion

In this research, a novel Modified Deep Learning-based Food Item Classification (MDEEPFIC) approach has been proposed to categorize the different food items from the dataset with their calorie values. Initially, the images are processed using the sigmoid stretching method to enhance the image quality and remove the noises. Consequently, the pre-processed images are segmented using Improved Watershed Segmentation (IWS2) algorithm. Recurrent Neural Network (RNN) is used to extract features like shape, size, textures, and color. The extracted features are then normalized using the modified dragonfly technique for same food calorie calculation. The Bi-LSTM is utilized to classify food products based on these pertinent aspects. Finally, using food area volume and calorie and nutrition measures based on mass value, the calorie value of the categorized food item is calculated. The experimental findings were then verified and the performance was evaluated utilizing evaluation metrics such as accuracy, precision, recall, specificity, and f-measure. To improve classification performance, the suggested novel Modified Deep Learning-based Food Item Classification (MDEEPFIC) approach technique acquired a 99.15% accuracy value. Results indicate that our suggested approach performed comparably well and has a lot of potential for promoting a healthy diet and effective findings. In the future, the computational speed, processing time, and automatic diet calculator can be improved by developing algorithms.

References

Ege

, Shimoda

and Yanai

, A new large-scale food image segmentation dataset and its application to food calorie estimation based on grains of rice. In Proceedings of the 5th International Workshop on Multimedia Assisted Dietary Management, France (2019), 82–87.

Liang

and Li

, Computer vision-based food calorie estimation: dataset, method, and experiment, 3 (2017), 1–7.

Ege

, Ando

, Tanno

and Shimoda

, Image-based estimation of real food size for accurate food calorie estimation. In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), USA (2019), 274–279.

McAllister

, Zheng

, Bond

and Moorhead

, Combining deep residual neural network features with supervised machine learning algorithms to classify diverse food image datasets, Comput. Biol. Med. 95 (2018), 217–233.

Inunganbi

, Seal

and Khanna

, Classification of food imagesthrough interactive image segmentation, In: N. Nguyen, D. Hoang,T.Hong, H. Pham and B. Trawiński, Intelligent Information andDatabase Systems. ACIIDS 2018. Lect. Notes Comput. Sci. Springer (2018), 519–528.

Metwalli

A.S.

, Shen

and Wu

C.Q.

, Food image recognition based on densely connected convolutional neural networks. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Japan, (2020), 027–032. https://doi.org/10.1109/ICAIIC48513.2020.9065281.

Shen

, Shehzad

, Chen

and Sun

, Machine learning based approach on food recognition and nutrition estimation, Procedia Comput. Sci. 174 (2020), 448–453.

Emmanuel

W.S.

and Minija

S.J.

, Fuzzy clustering and Whale-basedneural network to food recognition and calorie estimation for dailydietary assessment, Sādhanā 43 (2018), 1–19.

Kumar

R.D.

, Julie

E.G.

, Robinson

Y.H.

and Vimal

, Recognition of food type and calorie estimation using neural network, The Journal of Supercomputing 77 (2021), 8172–8193.

10.

Turmchokkasam

and Chamnongthai

, the design and implementation of an ingredient-based food calorie estimation system using nutrition knowledge and fusion of brightness and heat information, IEEE Access 6 (2018), 46863–46876.

11.

Rewane

and Chouragade

P.M.

, Food Recognition and Health Monitoring System for Recommending Daily Calorie Intake. In IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT), India (2019), 1–5.

12.

Yunus

, Arif

, Afzal

and Amjad

M.F.

, A framework to estimate the nutritional value of food in real time using deep learning techniques, IEEE Access 7 (2018), 2643–2652.

13.

Naritomi

and Yanai

, CalorieCaptorGlass: Food calorie estimation based on actual size using hololens and deep learning. In IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), USA (2020), 818–819.

14.

Jeny

A.A.

, Junayed

M.S.

, Ahmed

and Habib

M.T.

, Local food recognition using deep residual neural networks. In International Conference on Information Technology (ICIT), India (2019), 184–189.

15.

Islam

M.T.

, Siddique

B.N.K.

, Rahman

and Jabid

, Image recognition with deep learning. In International conference on intelligent informatics and biomedical sciences (ICIIBMS), Thailand (2018), 106–110.

16.

Minija

S.J.

and Emmanuel

W.S.

, Food image classification using sphere shaped—Support vector machine. In International Conference on Inventive Computing and Informatics (ICICI), India (2017), 109–113.

17.

Manjunathan

, Lakshmi

, Ananthi

and Ramachandran

, Image Processing Based Classification of Energy Sources in Eatables Using Artificial Intelligence, Ann. Rom. Soc. Cell Biol. 1 (2021), 7401–7407.

18.

Kohila

and Meenakumari

, Predicting calorific value for mixed food using image processing. In International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS), India (2017), 1–4.

19.

Farooq

and Sazonov

, Feature extraction using deep learning forfood type recognition, In: I. Rojas, F. Ortuño, Bioinformaticsand Biomedical Engineering. Springer (2017), 464–472.

20.

Wasif

S.M.

, Thakery

, Nagauri

and Pereira

S.I.

, calorie estimation using machine learning and image processing, International Journal of Advance Research, Ideas and Innovations in Technology 5 (2019), 1627–1630.

21.

Minija

S.J.

and Emmanuel

W.S.

, Neural network classifier and multiple hypothesis image segmentation for dietary assessment using calorie calculator, Imaging Sci. J. 65 (2017), 379–392.

22.

Latif

, Alsalem

, Mubarky

and Mohammad

, Automatic Fruits Calories Estimation through Convolutional Neural Networks. In Proceedings of the 2020 6th International Conference on Computer and Technology Applications Turkey (2020), 17–21.

23.

Deshmukh

P.B.

, Metre

V.A.

and Pawar

R.Y.

, Caloriemeter: Food Calorie Estimation using Machine Learning. In International Conference on Emerging Smart Computing and Informatics (ESCI), India (2021), 418–422.

24.

Sukvichai

, Maolanon

and Muknumporn

, categories classification and Ingredients estimation using CNNs on Raspberry Pi 3. In 10th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES), IEEE (2019), 1–6.

25.

Minija

S.J.

and Emmanuel

W.S.

, Image processing based Classification and Segmentation using LVS based Multi-Kernel SVM. In International Conference on Smart Systems and Inventive Technology (ICSSIT), India (2018), 61–65.

26.

Shermila

P.J.

and Milton

, Estimation of protein from the images of health drink powders, Journal of Food Science and Technology 57(5) (2020), 1887–1895.

MDEEPFIC: Food item classification with calorie calculation using modified dragonfly deep learning network

Abstract

Keywords

1 Introduction

2 Related works

3 Proposed MDEEPFIC methodology

3.2 Preprocessing

3.5 Calculating the volume of food

4.1 Calorie estimation

Table 1 Common calories table Food Name Measure Weight (in grams) Energy Apple with skin 1 130 75 Potato 1 128 69 Orange 1 112 64 Tomatoes 1 125 32 Strawberry 1 150 81 Cucumber, raw 1 113 18 Banana 1 102 108 Grape juice 1 cup 100 60 Brinjal 1 300 75

Table 2 Comparison of proposed technique with existing techniques Techniques Accuracy Precision Recall Specificity F-measure LSV-SVM 87.37 90.1 88.4 85.15 86.48 F-RCNN 91.4 90.4 89.7 90.47 87.25 DCNN 93.05 96.15 94.5 93.5 91.43 Proposed (DF-DLN) 99.36 98.9 99.15 98.07 99.04

References

Table 1
Common calories table

Food Name Measure Weight (in grams) Energy

Apple with skin 1 130 75

Potato 1 128 69

Orange 1 112 64

Tomatoes 1 125 32

Strawberry 1 150 81

Cucumber, raw 1 113 18

Banana 1 102 108

Grape juice 1 cup 100 60

Brinjal 1 300 75

Table 2
Comparison of proposed technique with existing techniques

Techniques Accuracy Precision Recall Specificity F-measure

LSV-SVM 87.37 90.1 88.4 85.15 86.48

F-RCNN 91.4 90.4 89.7 90.47 87.25

DCNN 93.05 96.15 94.5 93.5 91.43

Proposed (DF-DLN) 99.36 98.9 99.15 98.07 99.04