Scribd
Upload
9 views

Machine-Learning-in-Android-P

Uploaded by

Halima Hammougua
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Machine-Learning-in-Android-P

Uploaded by

Halima Hammougua
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 41

Machine Learning in Android App

Development

Fajar Ulin Nuha -


Principal Engineer Android
About Me
- Ilmu Komputer UGM 2014
- OmahTI Mobile Apps
- Lab Sistem Cerdas
- Skripsi: Training Dataset Reduction on GAN
- Research Assistant: Comet assay classification
- Tokopedia Android Engineer (2018-now)
Overview

1 Machine Learning Deployment Overview

2 Android ML: ML Kit & Tensorflow Lite

3 Mobile Apps: Implementation & Challenge

4 Careers at Tokopedia
Machine Learning Deployment
Overview
Generic Machine Learning Workflow

Evaluation of
Accuracy/Performance

Evaluation
with ops team
On-Device Inference Cloud-based Inference
Inference

source
ML Kit
- Available in Android and iOS
- On-device inference
- Powerful yet easy to use Vision and Natural Language APIs
- Powered by Google's best-in-class ML models and offered at no cost
- Google product
Key ML Development Areas

Design: https://pair.withgoogle.com/guidebook (ux)


Explore pre-trained models: https://tfhub.dev/s?deployment-format=lite
Training: local or cloud
ML Kit
Vision Natural Language

- Text Recognition - Identify Languages


- Face Detection - Translate Text
- Pose Detection (beta) - Smart Replies
- Selfie Segmentation (beta) - Entity Extraction (beta)
- Barcode Scanning
- Image Labelling
- Object Detection & Tracking
- Digital Ink Recognition
- Custom Models
ML Vision

IMAGE

PREDICTION OUTPUT

VIDEO
Face Detection

- Identify key facial features


- Get the contours of detected faces
- Detects faces, it does not recognize
people

Application:
- Perform tasks like embellishing selfies
and portraits
- Generating avatars from a user's photo
- Applications like video chat or games
that respond to the player's
expressions
Pose Detection
- Detect the pose of a subject's body in real time from
a continuous video or static image
- Full body tracking The model returns 33 key skeletal
landmark points, including the positions of the hands
and feet
- InFrameLikelihood score For each landmark, a
measure that indicates the probability that the
landmark is within the image frame. The score has a
range of 0.0 to 1.0, where 1.0 indicates high
confidence
- Z Coordinate for depth analysis This value can help
determine whether parts of the users body are in
front or behind the users' hips
Case: KYC
- Know Your Customer, ID Verification
- Typical flow: KTP & Selfie
- Many problems with false positive case
- Can we improve using Machine Learning?
- Liveness: Action Video
source
- Face Detection: Face feature probability
distribution
- Face Comparison
Case: Category Recommendation
Custom Models

● Before going further: Use a model published on TensorFlow Hub


Custom Models

FTF: Try to explore models published on TensorFlow Hub

TensorFlow Lite Model Maker

Re-train a model (transfer learning), takes less time and


requires less data than training a model from scratch Image Labeling API or the Object
Detection and Tracking API

Convert Tensorflow model to Tensorflow Lite

If you have an existing TensorFlow image classification model,


you can convert it using the TensorFlow Lite converter.
Lite

Vision Text

- Image Classification - BERT Question and Answer


- Object Detection - Smart Reply
- Pose Estimation - Text classification
- Segmentation - Recommendation
- Style Transfer
- Super resolution
Mobile Apps: Implementation &
Challenge
Let’s Code
- Object Detection and Tracking
- Base model & Custom model
- It detects up to five objects in the image along
with the position of each object in the image
- When detecting objects in video streams, each
object has a unique ID that you can use to track
the object from frame to frame
- You can also optionally enable coarse object
classification, which labels objects with broad
category descriptions
Let’s Code
Let’s Code
App Size Challenge
- App size is very important to a user
- ML model tends to fill great amount of size in the apk
- Each model will take MBs. Face Detection model can take up to
~16 MB
Base Model: Bundled vs Unbundled
- A bundled model which is part of your app and an unbundled model that depends on
Google Play Services. The two models are the same. If you select the unbundled
model your app will be smaller.

To opt-in:
Edit build.gradle

Install-time download:
Edit AndroidManifest.xml
Custom Model: Bundled vs Hosted

- There are two ways to integrate a custom model. You can bundle the
model by putting it inside your app’s asset folder, or you can dynamically
download it from Firebase
Custom Model

1. Copy the model file (usually ending in .tflite or .lite) to your app's assets/ folder
2. Then, add the following to your app's build.gradle
Custom Model
Explore more
- ML Kit : https://developers.google.com/ml-kit/guides
- ML Kit Samples: https://github.com/googlesamples/mlkit
- Tensorflow Lite : https://www.tensorflow.org/lite
- Tensorflow Lite Samples:
https://github.com/tensorflow/examples/tree/master/lite/examples
- Tensorflow Lite Model Maker: https://www.tensorflow.org/lite/guide/model_maker
Careers at Tokopedia
3 DNA of Tokopedia - our culture, our belief
Working @ Tokopedia

1 My Career

2 Environment

3 Perks

4 Data office
Careers @ Tokopedia
individual managerial

Fellow VP

Architect Manager

Principal Lead

Sr. Staff

Staff
Data Office

Business
Data Scientist Data Engineer Data Analyst
Intelligence

- analyze data for


- AI solution - build data pipeline business team
- research/scientist from source to data - business data - public data
- model warehouse visualization and
analysis
What makes you a good candidate?

1 AI/ML General Knowledge

2 Coding, Software Engineering -> python, pandas,


seldoncore, git, linux, ssh, etc
3 Problem Solving & Analytical Thinking

4 Be practical and have a business mindset

5 3 DNA
https://www.tokopedia.com/careers/

video
Thank You
Contact me [email protected], @fajarnuha

You might also like