Back To Basics

The ultimate Yocto introduction

summit.23:~$ bitbake -e | grep ^WHOAMI

WHOAMI=”Talel BELHAJSALEM”

WHOAMI_DESC=”Yocto Enthusiast and Trainer”

WHOAMI_WORK=”Embedded Linux Engineer at Sofia Technologies”

WHOAMI_LINKS=”linkedin email stackoverflow extra”

WHOAMI_LINKS[linkedin] = “https://www.linkedin.com/in/bhstalel/”

WHOAMI_LINKS[stackoverflow] = “https://stackoverflow.com/users/7553704/talel-belhadjsalem”

WHOAMI_LINKS[email] = “[email protected]”

WHOAMI_LINKS[extra] = “https://yoctoleef.xyz/”
Summary

1. Linux Architecture
2. The Global Aim
3. Build System Philosophy
4. The Chef : Bitbake
5. The Story behind Metadata
6. Preparing The Kitchen : oe-init-build-env
7. The Cooking Process
8. Plating The Image
9. Useful Metadata
10. Board Support Package
11. Useful Cooking Techniques
12. What’s Next?
Linux Architecture : Basics

App App App Libs

C Lib
USERSPACE

Shell
syscalls
(ABI)

KERNELSPACE
Kernel

Hardware
Linux Architecture : What we need?

App App App Libs

ROOTFS C Lib
USERSPACE

BOOTLOADER
Shell

KERNELSPACE
Kernel DTB

ARCH ? Hardware
The Global Aim : Back to boot

❏ We need a way to:

ROM SPL
❏ Build Kernel and DTB
❏ Assemble Rootfs
KERNEL
❏ Handle packaging
❏ Handle dependencies Boot Partition
❏ Build SPL (Vendor Specific) DTB
❏ Assemble the final image

❏ We need a way to: ROOTFS

Rootfs
Partition

❏ Build all of that for the target ARCH

❏ Figure out the compilation type The typical image structure (SD, eMMC, …)
❏ Prepare an SDK (Toolchain)
Build System Philosophy

Linux Packages

Bootloader Kernel + DTB

In House Coreutils Bootloader

Build System
Upstream Busybox Rootfs

App Assembled Image

… SDK

Sources Configuration Output

Build System Philosophy : Configuration

● The build system needs some information in order to produce the output:
○ What Linux Kernel to use ?
○ What Bootloader to use ?
○ What Device Tree Source to compile from the Kernel dts sources ?
○ What is the Target architecture ?
■ This helps creating the right toolchain for the compilation process
○ What is the final image file type and structure ?
○ What C Library to use ?
○ What are packages to include in the final rootfs ?
○ What to use for the basic utilities (busybox, coreutils, …) ?
○ What init manager to use (systemd, sysvinit, …) ?
○ Other questions need to be answered
The Chef : BitBake
Timeline 2006 2010-2011

OpenEmbedded Linux Foundation +

Yocto Project
poky
Tools
bitbake Test Bitbake
Automate Builds/Tests
Reference Distro
License Handling

OpenEmbedded
Core (metadata)

● The Build engine (Python)

● Called: The make of distributions

 The Story Behind Metadata
● Bitbake metadata is just a form of specifying what the
developer wants from it and how to arrange that demand.
Upstream InHouse ● It is just a form of providing the Configuration and Sources
to the build system.
● It is like providing to the chef what raw food to use, how to
GIT - HTTPS - FTP - … cook and what to do with the final plates

Append
.bbappend .rpm .deb .ipk

Recipe Package
meta-* Bitbake ROOTFS
.bb
Layer Recipe Package

Shared Data Configuration

.bbclass .inc .conf
The Story Behind Metadata : Interconnection

Class Patch
<name>.bbclass <name>.patch

Recipe Include
<name>_<version>.bb <name>.inc

Append Conf
<name>_<version>.bbappend <name>.conf
The Story Behind Metadata : Recipe

meta-*
poky Recipe

recipes-*

base.bbclass *
Image Package
image.bbclass
*.bb

Contains all tasks needed to create a Package from fetching source to actual package creation.

Disables all package tasks and create new tasks to create and assembly the final Rootfs using list of package recipes.

A package recipe defines (Similar to a real world chef’s recipe):

❏ What sources to work with: SRC_URI
❏ How to deal with them: do_configure, do_compile, …
❏ What to go into the final package: do_install, FILES, …
 The Story Behind Metadata : Layer

# We have a conf and classes directory, add to BBPATH meta-*

BBPATH .= ":${LAYERDIR}"

# We have recipes-* directories, add to BBFILES

BBFILES += "${LAYERDIR}/recipes-*/*/*.bb
${LAYERDIR}/recipes-*/*/*.bbappend" conf recipes-* classes
BBFILE_COLLECTIONS += "skeleton"
BBFILE_PATTERN_skeleton = "^${LAYERDIR}/"

BBFILE_PRIORITY_skeleton = "1" layer.conf * *.bbclass

# This should only be incremented on significant changes

that will
# cause compatibility issues with other layers
LAYERVERSION_skeleton = "1" files *.bb *.inc

LAYERDEPENDS_skeleton = "core"
LAYERSERIES_COMPAT_skeleton = "kirkstone"
source files
The Story Behind Metadata : The big picture

.inc
meta- Packages
.bb
meta- Build System Images
.bbclass
… SDK
.conf

…
local.conf

bblayers.conf
 The chef (BitBake) needs to know:

Where to work (The kitchen) ?

How and where to find what to work with ?
 Preparing The Kitchen : oe-init-build-env
source poky/oe-init-build-env build
poky
BUILDDIR

oe-init-build-env bitbake scripts conf downloads tmp sstate-cache

lib bin wic bblayers.conf work

runqemu local.conf deploy

bitbake
devtool

bitbake-layers BB_ENV_PASSTHROUGH BUILDDIR PATH PYTHONPATH

● Source the env in every new opened terminal
● Sourcing in new folder will create a new kitchen
The Cooking Process : Entry

● Introduce the chef to the kitchen (source oe-init-build-env)

● The chef analyses the environment and collects final information
about:
○ What layers to consider: BBFILE_COLLECTIONS
○ Where recipes are: BBFILES
○ The rest of variables
● All configuration is set of variables used by the chef to know where to
find things:
○ MACHINE
○ DISTRO
○ …
The Cooking Process : Global configuration

poky BUILD_ARCH := "${@os.uname()[4]}"

HOST_ARCH = "${TARGET_ARCH}"
TARGET_ARCH = "${TUNE_ARCH}"
meta
TCLIBC ??= “glibc”
export BUILD_CC = "${CCACHE}${BUILD_PREFIX}gcc ${BUILD_CC_ARCH}"
export BUILD_CXX = "${CCACHE}${BUILD_PREFIX}g++ ${BUILD_CC_ARCH}"
conf export BUILD_CPP = "${BUILD_PREFIX}gcc ${BUILD_CC_ARCH} -E"
export BUILD_LD = "${BUILD_PREFIX}ld ${BUILD_LD_ARCH}"
export BUILD_AS = "${BUILD_PREFIX}as ${BUILD_AS_ARCH}"
export MAKE = "make"
bitbake.conf

● This is the default configuration with lot of other variables as well.

● This is updated according to type of compilation
○ Example: for cross compilation: HOST_ARCH will be same as BUILD_ARCH
● This is automatically handled, you can choose not to bother digging into the configuration.
The Cooking Process : Recipe

meta-example
SRC_URI = “git://github.com/bhstalel/yocto-hello-c-makefile;branch=main;protocol=https
SRCREV ?= “c6c81ae747fa72a60ab389c486fa84a09446fdb8”
recipes-example DEPENDS += “hellolib”
RDEPENDS:${PN} += “hellolib”
S = “${WORKDIR}/git”
example

hellolib_0.1.bb hello_0.1.bb hello_0.2.bb hello.inc

require hello.inc

require hello.inc
SRCREV = “3dc0d44b3881cb0a38ecdfc8dcb4b735bc5f7a02”

SRC_URI = “git://github.com/bhstalel/yocto-hello-c-dependency;branch=main;protocol=https”
SRCREV = “6114bcd982672421739cd1d5d9ddfd6d32dc0fe0”
# …
https://docs.yoctoproject.org/overview-manual/concepts.html#bitbake-tasks-map
raspberrypi4-poky-linux
build

x86_64-linux all-poky-linux aarch64-poky-linux

tmp
downloads
MULTIATCH_TARGET_SYS =
work “${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}

git2
MULTIARCH_TARGET_SYS
1- do_fetch

GitHub hello Package type is controlled by PACKAGE_CLASSES hellolib

2- do_unpack
4- do_populate_sysroot
0.1-r0 hello-0.1-r0-??.rpm

temp git image package* deploy-rpms recipe-sysroot sysroot-destdir

5- do_prepare_recipe_sysroot
10- do_package_write_rpm
3- do_patch Makefile usr lib usr

6- do_configure
7- do_compile hello.c bin include
9- do_package*
8- do_install

hello hello hellolib.so hellolib.h

Plating The Image build

custom-image.bb tmp
inherit core-image
IMAGE_INSTALL:append = “ hello”
work deploy
BitBake will:
MACHINE rpm images
● Make sure that all IMAGE_INSTALL recipes are built
correctly
custom-image <target> MACHINE
● Install all packages into the final rootfs
0.1-r0 *rpm
● Create image structure specified with WKS_FILE and
other variables
rootfs oe-rootfs-repo
● Place the final image in the deploy directory

… usr etc bin

Useful Metadata
● Poky, the core OE layer and the community provides very useful metadata:
○ https://git.yoctoproject.org/

Layers Classes

Board Support Package Software useradd.bbclass (poky)

meta-raspberrypi meta-openembedded systemd.bbclass (poky)

meta-intel meta-browser cmake.bbclass (poky)

meta-imx meta-virtualization qt6-qmake.bbclass (meta-qt6)

meta-st-stm32mp meta-aws packagegroup.bbclass (poky)

meta-ti meta-qt6 module.bbclass (poky)

meta-xilinx meta-java setuptools3.bbclass (poky)

Board Support Package : Machine configuration
● Machine configuration contains:

meta-bsp
○ Target architecture: poky/meta/conf/machine/include/tune*

○ Machine features MACHINE_FEATURES conf

○ Variables to customize the Kernel and the Bootloader

○ The choice of the Kernel and Bootloader machine

■ PREFERRED_PROVIDER_virtual/kernel
■ PREFERRED_PROVIDER_bootloader
myboard.conf
○ The configuration of the serial console
■ SERIAL_CONSOLE

local.conf MACHINE = “myboard”

Board Support Package : Machine configuration

● Kernel and Bootloader configurations:

○ Kernel DTB: KERNEL_DEVICETREE
○ Kernel Image name: KERNEL_IMAGETYPE
○ Uboot machine: UBOOT_MACHINE
○ Boot files: IMAGE_BOOT_FILES
○ Rootfs Image type: IMAGE_FSTYPES , WKS_FILE

● These variables are used by virtual/kernel and virtual/bootloader virtual recipes to do their internal
compilation.
 Board Support Package : raspberrypi4-64

IMAGE_FSTYPES = “tar.bz2 ext3 wic.bz2”

WKS_FILE ?= “sdimage-raspberrypi.wks”
This answers all the questions we asked in the
MACHINE_FEATURES += “usbhost wifi ..” beginning.
KERNEL_IMAGETYPE_UBOOT ?= “Image”
Now BitBake knows:
KERNEL_IMAGETYPE_DIRECT ?= “Image”
UBOOT_MACHINE = “rpi_4_config” ● What architecture to use, so CROSS
● What kernel to use
SDIMG_KERNELIMAGE ?= “kernel8.img” ● What DTB to pass to Kernel
RPI_KERNEL_DEVICETREE = “broadcom/bcm2711-rpi-4-b.dtb” ● What bootloader to use
● What Uboot machine to use
PREFERRED_PROVIDER_virtual/kernel ?= “linux-raspberrypi”
● What files to put in boot partition
IMAGE_CLASSES += “sdcard_image-rpi” ● What image file structure to build
require conf/machine/include/arm/armv8a/tune-cortexa72.inc
Advanced Cooking Techniques
Debugging and Useful techniques Comment

bitbake -e Expand the full environment bitbake -e <recipe> | grep ^VARIABLE

bitbake -s Show all recipes versions Useful to check available versions and test layers priorities on them

devshell Prepare dev shell in WORKDIR Useful when you want to work manually on the source code after do_unpack:
bitbake <recipe> -c devshell

pydevshell Python dev interpreter Same as devshell (more powerful) you have access to the d object:
d.getVar() d.getVarFlag() …

Log Functions Used inside tasks to log stuff bbwarn, bbinfo, …

bb.warn, bb.info, …

Anonymous Functions Used to prepare some python __anonymous(){}

information or do some checks at
parse-time.

Inline Python Used to do conditional IMAGE_INSTALL:append = “ ${@’m.service’ if d.getVar(‘INIT_MANAGER’)

Variables assignment == ‘systemd’ else ‘’}”
Useful Cooking Tools

Tool Comment

bitbake-layers Manage layers (create, add, remove, …)

devtool The most powerful utility after bitbake, it is used to add and modify recipes automatically.

Example: devtool add hello https://github.com/bhstalel/yocto-hello-c-makefile -B main

recipetool Used also to add and/or modify recipes and create bbappend files automatically

oe-pkgdata-util Used to check information about packages, example: find which recipe provides a file
What Is Next?

● PACKAGECONFIG
● Create and test various DISTROs
● Sharing downloads and cache and optimization
● Work with Package Management
● Work with auto wrappers like repo and Kas
● Populate and work with SDK
● Check Documentation: https://docs.yoctoproject.org/
● Welcome to IRC: https://web.libera.chat/?channels=#yocto
● Welcome to Mailing Lists: https://lists.yoctoproject.org/