Product direction
Defined the initial problem, main use cases, and the idea of recognizing meals from photos.
Computer Vision
FitFood AI Nutrition App
A mobile nutrition prototype with a food model that recognizes 43 classes and reached 80% test accuracy.
- Client
- Team competition project
- Role
- Idea Initiator & ML Engineer
- Year
- 2024
- Status
- Competition prototype
FitFood home screen with BMI details, recommendations, and an option to analyze food
Screenshot 1 of 4
Overview
Food recognition built into a mobile nutrition flow
FitFood is an Android prototype created for Indonesia Inventors Day 2024. Users can photograph or upload a meal, receive a predicted food label, and continue to calorie information and meal recommendations. BMI, activity level, a nutrition chatbot, and water tracking complete the wider health flow.
The food model uses InceptionV3 with ImageNet weights and a custom classification head. I trained it on 10,179 images, validated it on 1,270 images, and evaluated the final model on a separate set of 1,280 images. It reached 80% test accuracy across 43 food classes.
I proposed the product idea and worked as the Machine Learning Engineer. I prepared the data, built and trained the classifier, converted it to TensorFlow Lite, and worked with the mobile team to connect its predictions to the application.
Technology stack
Mobile application
- Kotlin
- Jetpack Compose
- MVVM
Machine learning
- TensorFlow
- Keras
- InceptionV3
Model delivery
- TensorFlow Lite
- ImageNet Transfer Learning
- NVIDIA RTX 3080
ProblemManual food logging adds friction to nutrition tracking
People trying to balance diet and exercise often have to enter meals manually before an application can provide useful guidance. That makes routine tracking slow, especially when the user does not know the food name, portion, or nutritional value.
Generic recommendations also ignore differences in BMI and activity level. The team used FitFood to explore whether one mobile workflow could combine personal context, meal recognition, and practical diet or workout guidance.
Input friction
Users can start with a photo instead of searching for a food by name.
Personal context
BMI and activity level provide the basis for diet and exercise recommendations.
ContributionInitiated the concept and built the model that recognizes food
I proposed combining AI, nutrition, and fitness in a mobile app. The design lets users begin with a photo rather than manual food entry.
As the Machine Learning Engineer, I prepared the 12,729 image dataset, built the InceptionV3 classifier, ran both training stages, and evaluated it on the 1,280 image test set. I converted the trained model to TensorFlow Lite and worked with the mobile team to connect its output to the meal tracking flow.
Machine learning
Owned data preparation, model design, transfer learning, fine tuning, and test evaluation.
Team integration
Delivered a TensorFlow Lite model and aligned mobile preprocessing with the training pipeline.
ArchitectureInceptionV3 turns a meal photo into one of 43 food labels
The model accepts a 224 x 224 RGB image with pixel values scaled from 0 to 1. InceptionV3 uses ImageNet weights as the feature extractor. Its original classification head was replaced with global average pooling, a dense layer with 128 ReLU units, dropout at 0.5, and an output layer with 43 sigmoid units.
Training used two stages on an NVIDIA RTX 3080 with binary cross entropy as the loss function. I first froze the InceptionV3 base and trained the new head for 10 epochs. I then unfroze the network from layer 249 and trained for another 150 epochs with Adam at a learning rate of 0.00001. The complete model has 22,070,603 parameters.
After training, I converted the model to TensorFlow Lite. The Android pipeline applies the same resize, RGB conversion, and pixel scaling used during training, then selects the class with the highest output score. That food label leads to calorie information and recommendations in FitFood.
Prepare the image
Camera or upload
Meal image supplied by the user
Resize to 224 x 224
Matches the model input shape
RGB + normalization
Pixel values scaled to the 0 to 1 range
Classify the food
InceptionV3
Extracts visual features from the meal
Custom head
Pooling, dense, dropout, and 43 outputs
Highest score
Selects one predicted food label
Use the result
TensorFlow Lite
Runs the converted model in the mobile flow
Nutrition details
Shows food and calorie information
Recommendations
Connects the result to the wider FitFood experience
Reuse learned visual features
ImageNet weights provide the starting point, while the custom head learns the 43 FitFood classes.
Keep preprocessing consistent
Training and mobile inference both use 224 x 224 RGB inputs with values scaled to the 0 to 1 range.
Results80% accuracy on the separate test set
80.00%
test accuracy
from a separate set of 1,280 images
80.15%
weighted F1 score
from the same test set of 1,280 images
12,729
food images
split into training, validation, and testing
43
food classes
one folder label for each class
On the separate 1,280 image test set, the model reached 80.00% accuracy, 80.81% weighted precision, 80.00% weighted recall, and an 80.15% weighted F1 score. The recorded test loss was 0.0451. At 80% accuracy, about 1,024 test images received the correct class.
FitFood also received a Bronze Medal at Indonesia Inventors Day 2024 in Bali. The team built the prototype in approximately one and a half months, covering the mobile interface, machine learning work, chatbot, and application integration.
The current evaluation covers aggregate test results. It does not include performance for individual food classes, a confusion matrix, model size after TensorFlow Lite conversion, or inference time on a physical phone.
Separate test set
All published model metrics come from 1,280 images excluded from training and validation.
Balanced aggregate metrics
Accuracy, weighted precision, weighted recall, and weighted F1 all remained close to 80%.
Mobile delivery
The trained TensorFlow model was converted to TensorFlow Lite for Android integration.
LearningsWhat shaped the model and mobile flow
Train the new head before the base model
Freezing InceptionV3 for the first 10 epochs let the custom head adapt to the food classes before deeper layers were opened for another 150 epochs.
Match mobile input to training input
The Android pipeline must preserve the same image size, RGB format, and pixel scaling used during model training.
Keep product recognition and model quality separate
The Bronze Medal describes the complete prototype. Test accuracy and weighted F1 describe the classifier on images kept out of training and validation.
Next project
Hermes Agent
A personal AI agent that connects chat, tools, scheduled workflows, and structured data on a privately managed home server.