The Old Monks

Project Proposal - CaloRadar

Rohit Madhu
​Rohan Pathak
Gokul Nedunsezhian
Jai Krishna Mounaguru
Balajhi Shanmugam Selvakumar
OBJECTIVE​

To develop an innovative
model that accurately
identifies food items
from images, uncovers
the ingredients in
every dish, and promotes
healthy dietary
choices through
increased awareness​
DATA SOURCE​
 F​ lickr API: Using Flickr API, we
collect diverse food images
worldwide, enriching our dataset with
culinary variety
 Kaggle Datasets: Kaggle's open
datasets, featuring a wide array of
food images, are a valuable resource
for training and testing our
image classification system​
 ​Google Open Source: Google's open-
source data bolsters our dataset with a
comprehensive array of images,
spanning diverse cuisines and dishes​
Data Collection

​Flickr API

 Training Image Files: 25,459

 Test Image Files: 2,652
 Total Image Files: 28,111
Food Categories
BUSINESS CASE 1
Fitness Apps – Calorie & Nutrition Tracking

Queries USDA FoodData Central

API for detailed nutritional data

TensorFlow-based Xception model

identifies food items
 Allergen Alert
BUSINESS CASE 2
Recognize food, get recipe & notify users of allergens

Shazam for Food!

Recipe and Ingredient Retrieval

Leverages Spoonacular API for recipe

and ingredients
Data Preprocessing

Normalization

 Rescale pixel values to a range

between 0 and 1
 Ensures consistent data
representation for the model
Image Resizing:

Set target size to 224 x 224 pixels

All images standardized to the same
dimensions
Facilitates easier model learning
Building the Classifier (Training, Evaluation &
Deployment)

Divide the dataset into Load the Pre trained model Compile the model with
training, validation, and test and remove the top layers of appropriate loss and optimizer &
sets – 70:20:10 the model train the model

Evaluate model performance Use the Model to make Use R Shiny to create a
using metrics such as accuracy, predictions on new images simple UI to upload and
precision, recall & F1 score classify images
Model Activation Functions: Softmax

 Softmax Activation in the Output

Layer:
 Objective: Multi-class food classification
task.
 Why Softmax?
 Softmax converts raw scores into
probability distributions over classes.
 Ideal for assigning probabilities to
various food categories.
 Enables the model to provide a
confident prediction for a single food
class.
Model Activation Functions:
ReLU

 ReLU Activation in Hidden Layers:

 Objective: Enhance non-linearity and feature
learning
 Why ReLU?
 Introduces non-linearity, allowing the model to
capture complex patterns in the data
 Mitigates the vanishing gradient problem, improving
training stability
 Promotes sparsity, making the network more memory-
efficient
Why Softmax +
ReLU?

Softmax for Precision:

 Converts raw scores to precise probability distributions.
 Facilitates accurate and confident food category predictions.
ReLU for Feature Mastery:
 Introduces non-linearity for capturing intricate features.
 Enhances the model's capacity to learn complex patterns.
Conclusion:
 The combined power of Softmax and ReLU yields a high-
performance neural network for image food classification.
 Xception

 "Extreme Inception"
 Focuses on xtreme depthwise
separable convolutions
 Key Advantages:
 Reduced number of parameters
 Maintains or improves
representational power
 Enhanced computational
efficiency
Untraining: Break Free

 Recipe:
 Freeze early layers to retain pre-
learned features.
 Randomly initialize last layers to
disconnect from the original task.
 Train on your dataset to forge task-
specific knowledge.
Retraining: Seamless Transition

 Recipe:
 Remove final layers designed for
the original task.
 Inject custom output layers for the
new task.
 Retrain on your dataset, harnessing
the power of pre-learned features.
In a Snap - Hybrid Fine-tuning:

 Controlled Evolution:
 Gradually unfreeze layers to fine-tune pre-trained features.
 Customization without losing the essence of prior knowledge.
 Tailored for swift adaptation to new challenges.
 Conclusion:
 Untraining and retraining, quick and dynamic transformations.

Our approach, a fusion of the two, ensures rapid adaptation and

mastery for optimal model performance.
 Confusion Matrix
Apple Banana Bread Burger Egg Fries Pasta Rice Soup Steak Total
Apple 122 1 1 0 2 1 0 0 0 1 128
Banana 14 192 4 1 17 6 4 1 0 1 240
Bread 0 0 223 0 15 16 4 0 1 4 263
Burger 0 1 44 101 15 8 7 0 0 1 177
Egg 0 0 21 0 199 4 3 0 0 4 231
Fries 0 0 21 1 20 250 11 2 0 3 308

Model Pasta
Rice
Soup
1
0
0
0
0
0
17
0
6
0
0
0
23
3
5
6
2
6
150
1
5
6
89
2
3
2
473
6
1
1
212
98
498

Evaluation Steak
Total 137
0 0
194 362
25
103
0 28
327
27
326 191
6 1
101
2
481
408
430 2545
497
 FoodData Central API

Provides accurate Authoritative source for Utilized for fetching Ensures users receive up-
nutritional information. diverse food items. nutritional details. to-date data.
 Spoonacular API

Enhances user Facilitates ingredient Integration in our App: Fetches diverse recipes Promotes culinary
experience with recipes analysis based on identified food exploration and healthy
items choices
Choose an image c

Classify

Select Dietary Options

Search for recipes

CaloRadar Application
Future Enhancements

EXPANSION OF MULTI-FOOD QUANTITY INGREDIENT

FOOD DETECTION DETECTION ORDERING
CATEGORIES CAPABILITY
Key Learnings
References

 https://www.r-bloggers.com/2021/03/how-to-build-your-own-image-recognition-app-with-r-part-1/

 https://medium.com/@nutanbhogendrasharma/image-classification-with-xception-model-e8094a9de4d2

 Ng, Y.S., Xue, W., Wang, W. and Qi, P., 2019, October. Convolutional neural networks for food image recognition: An
experimental study. In Proceedings of the 5th International Workshop on Multimedia Assisted Dietary Management (pp. 33-
41).

 Wu, X., Liu, R., Yang, H. and Chen, Z., 2020, December. An xception based convolutional neural network for scene image
classification with transfer learning. In 2020 2nd international conference on information technology and computer
application (ITCA) (pp. 262-267). IEEE.
 ChatGPT
THE END