Food-Type Recognition and Estimation of Calories Using Neural Network

Abstract

In the fast-moving world, obesity has become a major health issue to the human beings. BMI defines the obesity when it is greater than 30 kg/m². Obesity leads to many diseases like high cholesterol, liver failure, knee problems, diabetes, and sometimes cancer. When the patient eats healthy food, the obesity can be controlled. The obesity problem can be addressed when there is a system that monitors the food consumed by the patient automatically and gives the suggestion periodically to the patient in treatment of obesity. Many of the people find difficulty in monitoring their food intake periodically, due to less knowledge in nutrition and self-control. In this chapter, identification of food type is made and estimation of calorie is done using MLP and proposes the results. Single food item types were considered previously, but here mixed food item types are considered. Region of Interest (ROI) is used to identify the mixed food item type. The next step includes feature extraction process. The extracted feature image is fed into MLP classification to classify the food image. The volume of the food is used to calculate the calories present in the food. The implementation is processed in MATLAB with 1000 fruit images containing 6 food classes with good accuracy. The automatic dietary control is made available for the diabetic patients.

1 Introduction

In recent years, the numbers of diabetic patients are increasing due to lack of knowledge in diet control. The most important cause of diabetes is the less insulin production in the human body. This leads to many of the diseases like continuous urination, increased blood pressure, loss of weight, and sometimes cardiac arrest. These enable us to develop the system to educate the people to have proper healthy foods every day. The proper system is built in MATLAB to food-type recognition and estimate food calories consumed.

2 Related Work

The clustering techniques have been proposed to identify the food types. To segment the food item, affinity propagation and unsupervised clustering method have been adopted. The affinity propagation with agglomerative hierarchical clustering (AHC) obtains 95% of accuracy. The Monitoring of Ingestive Behavior model has built to monitor and estimate the calories [1]. The input food images have been taken from smartphones with single and mixed food items and fed into training and testing. The preprocessing steps are carried first, followed by vision-based segmentation done, and deep learning algorithm applied to estimate calories [2]. In the input food image, chopstick is used as a reference for measurement. The density-based database of the food is considered to evaluate the food volume, weight, and calorie estimation. The estimated weight and calories’ relative average error rate are 6.65% and 6.70% [3]. The two datasets have been collected and trained with single-task and multitask CNN. The above multitask CNN classifiers achieved good results in food identification and calorie estimation than single-task CNN [4]. The top 10 Thai curries are considered. The segmented image is fed into the fuzzy logic to identify the ingredients based on their intensities and boundaries. The calories are calculated by sum of all the ingredient calories [5].

3 Proposed Work

The proposed technique has been divided into two phases as the training phase and the testing phase. The input image is resized using the scaling technique. Feature extraction consists of the SIFT method, Gabor Filter, and color histogram. The feature extraction is converted into the classification that implements the segmented process and MLP, and these processes are implemented in testing phase. After the classification procedure, the total area computation and the volume measurements are identified for producing the calorie estimate. The entire process is demonstrated in Fig. 1.

Fig. 1

Proposed system architecture

3.1 Algorithm – Multilayer Perceptron Neural Networks

Initialize all neurons (v_ih) and weights (w_hj.)

Run=1

While (Run)

Do{

Store all w and v.

Epoch=epoch+1 /*training*/

For all (x^t, r^t) ∈ X_training in random order.

For all hidden nodes z_h,

$$ {\mathrm{z}}_{\mathrm{h}}=\mathrm{sigmoid}\left({\sum}_{\mathrm{j}=0}^{\mathrm{d}}{\mathrm{w}}_{\mathrm{h}\mathrm{j}}{\mathrm{x}}_{\mathrm{j}}^{\mathrm{t}}\right) $$

For all output nodes y_i

$$ {\mathrm{y}}_{\mathrm{i}}=\mathrm{softmax}\left({\sum}_{\mathrm{h}=0}^{\mathrm{H}}{\mathrm{v}}_{\mathrm{i}\mathrm{h}}{\mathrm{z}}_{\mathrm{h}}\right) $$

For all weights v_ih

$$ \Delta {\mathrm{v}}_{\mathrm{i}\mathrm{h}}=\upeta \left({\mathrm{r}}_{\mathrm{i}}^{\mathrm{t}}-{\mathrm{y}}_{\mathrm{i}}^{\mathrm{t}}\right){\mathrm{z}}_{\mathrm{h}} $$

$$ {\mathrm{v}}_{\mathrm{ih}}={\mathrm{v}}_{\mathrm{ih}}+\Delta {\mathrm{v}}_{\mathrm{ih}} $$

For all weights w_ih

$$ \Delta {\mathrm{w}}_{\mathrm{i}\mathrm{h}}=\upeta \left(\sum_{\mathrm{i}=1}^{\mathrm{K}}\left({\mathrm{r}}_{\mathrm{i}}^{\mathrm{t}}-{\mathrm{y}}_{\mathrm{i}}^{\mathrm{t}}\right){\mathrm{v}}_{\mathrm{i}\mathrm{h}}\right){\mathrm{z}}_{\mathrm{h}}\left(1-{\mathrm{z}}_{\mathrm{h}}\right){\mathrm{x}}_{\mathrm{j}}^{\mathrm{t}} $$

$$ {\mathrm{w}}_{\mathrm{ih}}={\mathrm{w}}_{\mathrm{ih}}+\Delta {\mathrm{w}}_{\mathrm{ih}} $$

For all (x^t, r^t) ∈ X_Validating

For all hidden nodes z_h,

$$ {\mathrm{z}}_{\mathrm{h}}=\mathrm{sigmoid}\left({\sum}_{\mathrm{j}=0}^{\mathrm{d}}{\mathrm{w}}_{\mathrm{h}\mathrm{j}}{\mathrm{x}}_{\mathrm{j}}^{\mathrm{t}}\right) $$

For all output nodes y_i

$$ {\mathrm{y}}_{\mathrm{i}}=\mathrm{softmax}\left({\sum}_{\mathrm{h}=0}^{\mathrm{H}}{\mathrm{v}}_{\mathrm{i}\mathrm{h}}{\mathrm{z}}_{\mathrm{h}}\right) $$

$$ \mathrm{err}\left(\mathrm{epoch}\right)=\frac{1}{2}{\sum}_{\mathrm{x}\upepsilon {\mathrm{X}}_{\mathrm{Validating}}}\Big({\sum}_{\mathrm{i}=1}^{\mathrm{K}}{\left({\mathrm{r}}_{\mathrm{i}}^{\mathrm{t}}-{\mathrm{y}}_{\mathrm{i}}^{\mathrm{t}}\right)}^2 $$

If err(epoch)>err (epoch-1)

Run=0

}

For all (x^t, r^t) ∈ X_testing

For all hidden nodes z_h,

$$ {\mathrm{z}}_{\mathrm{h}}=\mathrm{sigmoid}\left({\sum}_{\mathrm{j}=0}^{\mathrm{d}}{\mathrm{w}}_{\mathrm{h}\mathrm{j}}{\mathrm{x}}_{\mathrm{j}}^{\mathrm{t}}\right) $$

For all output nodes y_i

$$ {\mathrm{y}}_{\mathrm{i}}=\mathrm{softmax}\left({\sum}_{\mathrm{h}=0}^{\mathrm{H}}{\mathrm{v}}_{\mathrm{i}\mathrm{h}}{\mathrm{z}}_{\mathrm{h}}\right) $$

If y==r

“Successfully recognized”

Else

“Failed to recognize”

3.2 Multilayer Perceptron Neural Network

Multilayer perceptron neural network working principle is relatively based on the human brain and belongs to feed forward NN. Normally a human brain stores the information as the pattern and gains the knowledge to solve the complex problems by experience and it is demonstrated in Fig. 2.

Fig. 2

Multilayer perceptron neural network

The multilayer perceptron neural network recognizes the patterns by supervised training algorithm fed forward from input to output layers. Activation function is computed as given in Eq. (1).

$$ {x}_i^{m+1}(n)=f\ \left[\sum \limits_{j=1}^M{W}_{ij}^m{x}_j(n)\right] $$

(1)

4 Performance Evaluation

The proposed work is carried out in MATLAB with six food classes. Here, precision and recall are identified for different food classes for SVM and MLP based classifier to have better results. The accuracy is tested using F-measure. The F-measure is calculated by taking the precision and recall values of each class and is shown in Table 1.

Table 1 Performance comparison

5 Conclusion

In this proposed work, identification of food type is made and estimation of calorie is done using MLP and the results proposed. Single food item types were considered previously, but here mixed food item types are taken to have better results. The implementation is processed in MATLAB with 1000 fruit images containing 6 food classes with good accuracy. The automatic dietary control is made available for diabetic patients.