TinyML Cookbook: Combine artificial intelligence and ultra-low-power embedded devices to make the world smarter

BIRMINGHAM—MUMBAI

TinyML Cookbook

Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Publishing Product Manager: Devika Battike

Senior Editors: Roshan Kumar, Nathanya Dias

Content Development Editor: Tazeen Shaikh

Technical Editor: Rahul Limbachiya

Copy Editor: Safis Editing

Project Coordinator: Aparna Ravikumar Nair

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Shankar Kalbhor

Marketing Coordinator: Abeer Dawe

First published: April 2022

Production reference: 2290322

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80181-497-3

www.packt.com

While the publisher and the author have taken every precaution to ensure that information contained in this book is accurate, neither the publisher nor the author assumes any responsibility for errors, omissions, or damages to persons or proprieties from the use of the information contained herein.

TinyML is a registered trademark of the TinyML foundation, and is used with permission.

It is because of the source of support from my wife, Eleonora, during the long nights of writing that I managed to complete this unique journey. I dedicate this book to her, who believed in this project from the very beginning.

Foreword

Without a doubt, the tech industry continues to have an ever-increasing impact on our daily lives. The changes are as rapid as they are constant and are happening all around us – in our phones, cars, smart speakers, and the micro gadgets we use to improve efficiency, wellbeing, and connectivity. Machine learning is one of the most transformative technologies of our age. Businesses, academics, and engineering communities continue to understand, evolve, and explore the capabilities of this incredible technology, and are unlocking the greater potential to enable new use cases across many industries.

I am a product manager for machine learning at Arm. In this role, I am at the center of the ML revolution that is happening in smartphones, the automotive industry, gaming, AR, VR, and other devices. It is clear to me that there will be ML functionality in every single electronics device in the near future – from the world's largest supercomputers, down to the smallest, low-powered microcontrollers. Working in ML has introduced me to some of the most brilliant and brightest minds in tech – those who challenge the orthodoxies that exist in traditional industries, ask the tough questions, and unlock new value through the use of ML.

When I first met Gian Marco, I could barely spell ML, yet at that time he was already a veteran in the space. I was astonished by the breadth and depth of his knowledge and his ability to solve difficult problems. Together with the team at Arm, he has worked to make Arm Compute Library (ACL) the most performant library available for ML on Arm. The success of ACL is unrivaled. It's deployed on billions of devices worldwide – from servers to flagship smartphones, to smart ovens.

When Gian Marco told me he was writing a book on ML, my immediate reaction was Which part? The ML ecosystem is so diverse, with many different technologies, platforms, and frameworks to consider. At the same time, I knew that he was the right person for the job due to his extensive knowledge of all aspects of ML. Additionally, Gian Marco has an amazing way of explaining things in a straightforward and logical manner.

Gian Marco's book demystifies the world of TinyML by guiding us through a series of practical, real-world examples. Each example is outlined like a recipe, with a clear and consistent format throughout, providing an easy-to-follow, step-by-step guide. Beginning with the first principles, he explains the basics of the electronics or software techniques that will be used in the recipe. The book then introduces the platforms and technologies used, followed by the ML – where neural network models are developed, trained, and deployed on the target device. This really is a soup to nuts guide. Each recipe is a little more challenging than the last, and there is a nice mix of established and nascent technologies. You don't just learn the how, you also get an understanding of the why. When it comes to edge devices, this book really does provide a panoramic view of the ML space.

Machine learning continues to disrupt all aspects of technology and getting started is a must for software developers. This book enables quick onboarding through the use of readily available and inexpensive technologies. Whether you are new to ML or have some experience, each recipe provides a steady ramp of knowledge and leaves enough scope for further self-development and experimentation. Whether you use this book as a guide or a reference, you will develop a strong foundation in ML for future development. It will empower your team to get new insights and to achieve new efficiencies, performance improvements, and even new functionality for your products.

– Ronan Naughton

Senior Product Manager for Machine Learning at Arm

Contributors

About the author

Gian Marco Iodice is team and tech lead in the Machine Learning Group at Arm, who co-created the Arm Compute Library in 2017. The Arm Compute Library is currently the most performant library for ML on Arm, and it's deployed on billions of devices worldwide – from servers to smartphones.

Gian Marco holds an MSc degree, with honors, in electronic engineering from the University of Pisa (Italy) and has several years of experience developing ML and computer vision algorithms on edge devices. Now, he's leading the ML performance optimization on Arm Mali GPUs.

In 2020, Gian Marco cofounded the TinyML UK meetup group to encourage knowledge-sharing, educate, and inspire the next generation of ML developers on tiny and power-efficient devices.

About the reviewers

Alessandro Grande is a physicist, an engineer, a communicator, and a technology leader with a visceral passion for connecting and empowering humans to build more efficient and sustainable technology. Alessandro is the director of product at Edge Impulse and cofounded the TinyML Meetups in the UK and in Italy. Prior to Edge Impulse, Alessandro worked at Arm as a developer evangelist and ecosystem manager with a focus on building the foundations for a smarter and more efficient IoT. He holds a master's degree in nuclear and electronic physics from the University of Rome, La Sapienza.

Daksh Trehan began his career as a data analyst. His love for data and statistics is unimaginable. Various statistical techniques introduced him to the world of ML and data science. While his focus is on being a data analyst, he loves to forecast given data using ML techniques. He understands the power of data in today's world and constantly tries to change the world using various ML techniques and his concrete data visualization skills. He loves to write articles on ML and AI, and these have bagged him more than 100,000 views to date. He has also contributed as an ML consultant to 365 Days as a TikTok creator, written by Dr. Markus Rach, available publicly on the Amazon e-book store.

Table of Contents

Preface

Chapter 1: Getting Started with TinyML

Technical requirements

Introducing TinyML

What is TinyML?

Why ML on microcontrollers?

Why run ML locally?

The opportunities and challenges for TinyML

Deployment environments for TinyML

tinyML Foundation

Summary of DL

Deep neural networks

Convolutional neural networks

Quantization

Learning the difference between power and energy

Voltage versus current

Power versus energy

Programming microcontrollers

Memory architecture

Peripherals

Presenting Arduino Nano 33 BLE Sense and Raspberry Pi Pico

Setting up Arduino Web Editor, TensorFlow, and Edge Impulse

Getting ready with Arduino Web Editor

Getting ready with TensorFlow

Getting ready with Edge Impulse

How to do it…

Running a sketch on Arduino Nano and Raspberry Pi Pico

Getting ready

How to do it…

Chapter 2: Prototyping with Microcontrollers

Technical requirements

Code debugging 101

Getting ready

How to do it...

There's more

Implementing an LED status indicator on the breadboard

Getting ready

How to do it...

Controlling an external LED with the GPIO

Getting ready

How to do it...

Turning an LED on and off with a push-button

Getting ready

How to do it...

Using interrupts to read the push-button state

Getting ready

How to do it...

Powering microcontrollers with batteries

Getting started

How to do it...

There's more

Chapter 3: Building a Weather Station with TensorFlow Lite for Microcontrollers

Technical requirements

Importing weather data from WorldWeatherOnline

Getting ready

How to do it…

Preparing the dataset

Getting ready

How to do it…

Training the ML model with TF

Getting ready

How to do it…

Evaluating the model's effectiveness

Getting ready

How to do it…

Quantizing the model with the TFLite converter

Getting ready

How to do it…

Using the built-in temperature and humidity sensor on Arduino Nano

Getting ready

How to do it…

Using the DHT22 sensor with the Raspberry Pi Pico

Getting ready

How to do it…

Preparing the input features for the model inference

Getting ready

How to do it…

On-device inference with TFLu

Getting ready

How to do it…

Chapter 4: Voice Controlling LEDs with Edge Impulse

Technical requirements

Acquiring audio data with a smartphone

Getting ready

How to do it…

Extracting MFCC features from audio samples

Getting ready

How to do it…

There's more…

Designing and training a NN model

Getting ready

How to do it…

Tuning model performance with EON Tuner

Getting ready

How to do it…

Live classifications with a smartphone

Getting ready

How to do it…

Live classifications with the Arduino Nano

Getting ready

How to do it…

Continuous inferencing on the Arduino Nano

Getting ready

How to do it…

Building the circuit with the Raspberry Pi Pico to voice control LEDs

Getting ready

How to do it…

Audio sampling with ADC and timer interrupts on the Raspberry Pi Pico

Getting ready

How to do it…

There's more…

Chapter 5: Indoor Scene Classification with TensorFlow Lite for Microcontrollers and the Arduino Nano

Technical requirements

Taking pictures with the OV7670 camera module

Getting ready

How to do it...

Grabbing camera frames from the serial port with Python

Getting ready

How to do it...

Converting QQVGA images from YCbCr422 to RGB888

Getting ready

How to do it...

Building the dataset for indoor scene classification

Getting ready

How to do it...

Transfer learning with Keras

Getting ready

How to do it...

Preparing and testing the quantized TFLite model

Getting ready

How to do it...

Reducing RAM usage by fusing crop, resize, rescale, and quantize

Getting ready

How to do it...

Chapter 6: Building a Gesture-Based Interface for YouTube Playback

Technical requirements

Communicating with the MPU-6050 IMU through I2C

Getting ready

How to do it…

Acquiring accelerometer data

Getting ready

How to do it…

Building the dataset with the Edge Impulse data forwarder tool

Getting ready

How to do it…

Designing and training the ML model

Getting ready

How to do it…

Live classifications with the Edge Impulse data forwarder tool

Getting ready

How to do it…

Gesture recognition on Raspberry Pi Pico with Arm Mbed OS

Getting ready

How to do it…

Building a gesture-based interface with PyAutoGUI

Getting ready

How to do it…

Chapter 7: Running a Tiny CIFAR-10 Model on a Virtual Platform with the Zephyr OS

Technical requirements

Getting started with the Zephyr OS

Getting ready

How to do it…

Designing and training a tiny CIFAR-10 model

Getting ready

How to do it…

Evaluating the accuracy of the TFLite model

Getting ready

How to do it…

Converting a NumPy image to a C-byte array

Getting ready

How to do it…

Preparing the skeleton of the TFLu project

Getting ready

How to do it…

Building and running the TFLu application on QEMU

Getting ready

How to do it…

Chapter 8: Toward the Next TinyML Generation with microNPU

Technical requirements

Setting up Arm Corstone-300 FVP

Getting ready

How to do it…

Installing TVM with Arm Ethos-U support

Getting ready

How to do it…

Installing the Arm toolchain and Ethos-U driver stack

Getting ready

How to do it…

Generating C code with TVM

Getting ready

How to do it…

Generating C-byte arrays for input, output, and labels

Getting ready

How to do it…

Building and running the model on Ethos-U55

Getting ready

How to do it…

Other Books You May Enjoy

Preface

This book is about TinyML, a fast-growing field at the unique intersection of machine learning (ML) and embedded systems to make AI work with extremely low-powered devices, such as microcontrollers.

TinyML is an exciting field full of opportunities. With a small budget, we can give life to objects that interact with the world around us smartly and transform the way we live for the better. However, this field can be hard to approach if we come from an ML background with little familiarity with embedded systems, such as microcontrollers. Therefore, this book aims to dispel these barriers and make TinyML also accessible to developers with no embedded programming experience through practical examples. Each chapter will be a self-contained project to learn how to use some of the technologies at the heart of TinyML, interface with electronic components such as sensors, and deploy ML models on memory-constrained devices.

TinyML Cookbook starts with a practical introduction to this multidisciplinary field to get you up to speed with some of the fundamentals for deploying intelligent applications on Arduino Nano 33 BLE Sense and Raspberry Pi Pico. As you progress, you'll tackle various problems that you may encounter while prototyping microcontrollers, such as controlling the LED state with GPIO and a push-button and supplying power to microcontrollers with batteries. After that, you'll cover recipes relating to temperature, humidity, and the three V (voice, vision, and vibration) sensors to gain the necessary skills to implement end-to-end smart applications in different scenarios. Then, you'll learn best practices to build tiny models for memory-constrained microcontrollers. Finally, you'll explore two of the most recent technologies, microTVM and microNPU, which will help you step up your TinyML game.

By the end of this book, you'll be well versed in best practices and ML frameworks to develop ML apps easily on microcontrollers and have a clear understanding of the key aspects to consider during the development phase.

Who this book is for

This book is for ML developers/engineers interested in developing ML applications on microcontrollers through practical examples quickly. The book will help you expand your knowledge of the revolution of TinyML by building end-to-end smart projects with real-world data sensors on the Arduino Nano 33 BLE Sense and the Raspberry Pi Pico. Basic familiarity with C/C++, Python programming, and a command-line interface (CLI) is required. However, no prior knowledge of microcontrollers is necessary.

What this book covers

Chapter 1, Getting Started with TinyML, provides an overview of TinyML, presenting the opportunities and challenges to bring ML on extremely low-power microcontrollers. This chapter focuses on the fundamental elements behind ML, power consumption, and microcontrollers that make TinyML unique and different from conventional ML in the cloud, desktop, or even smartphones.

Chapter 2, Prototyping with Microcontrollers, presents concise and straightforward recipes to deal with the relevant microcontroller programming basics. We will deal with code debugging and how to transmit data to the Arduino serial monitor. After that, we will discover how to program GPIO peripherals with the ARM Mbed API and use the breadboard to connect external components, such as LEDs and push-buttons. In the end, we will see how to power the Arduino Nano 33 BLE Sense and the Raspberry Pi Pico with batteries.

Chapter 3, Building a Weather Station with TensorFlow Lite for Microcontrollers, guides you through all the development stages of a TensorFlow-based application for microcontrollers and teaches you how to acquire temperature and humidity sensor data. The application developed in the chapter is an ML-based weather station for snow forecasts.

In the first part, we will focus on dataset preparation by acquiring historical weather data from WorldWeatherOnline. After that, we will present the relevant basics to train and test a model with TensorFlow. In the end, we will deploy the model on the Arduino Nano 33 BLE Sense and the Raspberry Pi Pico with TensorFlow Lite for Microcontrollers.

Chapter 4, Voice Controlling LEDs with Edge Impulse, shows how to develop an end-to-end keyword spotting (KWS) application with Edge Impulse and get familiar with audio data acquisition and analog-to-digital (ADC) peripherals. The application considered for this chapter voice controls the LED emitting color (red, green, and blue) and the number of times to make it blink (one, two, and three).

In the first part, we will focus on the dataset preparation, showing how to acquire audio data with a mobile phone. After that, we will design a model using the mel-frequency cepstral coefficient (MFCC) features and optimize the performance with EON Tuner. In the end, we will finalize the KWS application on the Arduino Nano 33 BLE Sense and the Raspberry Pi Pico.

Chapter 5, Indoor Scene Classification with TensorFlow Lite for Microcontrollers and the Arduino Nano, aims to show you how to apply transfer learning with TensorFlow and get familiar with the best practices to use a camera module with a microcontroller. For the purpose of this chapter, we will develop an application to recognize indoor environments with the Arduino Nano 33 BLE Sense and the OV7670 camera module.

In the first part, we will see how to acquire images from the OV7670 camera module. After that, we will focus on the model design, applying transfer learning with Keras to recognize kitchen and bathroom rooms. In the end, we will deploy the quantized TensorFlow Lite model on the Arduino Nano 33 BLE Sense with the help of TensorFlow Lite for Microcontrollers.

Chapter 6, Building a Gesture-Based Interface for YouTube Playback, aims to develop an end-to-end gesture recognition application with Edge Impulse and the Raspberry Pi Pico to get acquainted with inertial sensors, teach you how to use I2C peripherals, and write a multithreading application in Arm Mbed OS.

In the first part, we will collect the accelerometer data through the Edge Impulse data forwarder to prepare the dataset. After that, we will design a model using features in the frequency domain to recognize three gestures. In the end, we will deploy the application on the Raspberry Pi Pico and implement a Python program with the PyAutoGUI library to build a touchless interface for YouTube video playback.

Chapter 7, Running a Tiny CIFAR-10 Model on a Virtual Platform with the Zephyr OS, provides best practices to build tiny models for memory-constrained microcontrollers. In this chapter, we will be designing a model for the CIFAR-10 image classification dataset on a virtual Arm Cortex-M3-based microcontroller.

In the first part, we will install Zephyr, the primary framework used in this chapter to accomplish our task. After that, we will design a tiny quantized CIFAR-10 model with TensorFlow. This model will fit on a microcontroller with only 256 KB of program memory and 64 KB of RAM. In the end, we will build an image classification application with TensorFlow Lite for Microcontrollers and the Zephyr OS and run it on a virtual platform using Quick Emulator (QEMU).

Chapter 8, Toward the Next TinyML Generation with microNPU, helps familiarize you with microNPU, a new class of processors for ML workloads on edge devices. In this chapter, we will be running a quantized CIFAR-10 model on a virtual Arm Ethos-U55 microNPU with the help of TVM.

In the first part, we will learn how the Arm Ethos-U55 microNPU works and install the software dependencies to build and run the model on the Arm Corstone-300 fixed virtual platform. After that, we will use the TVM compiler to convert a pretrained TensorFlow Lite model to C