Embedded Linux Systems With The Yocto Projects
Embedded Linux Systems With The Yocto Projects
Embedded
Linux Systems with
the Yocto Project"
�f, � � � �
Embedded Linux
Systems with the
TM
Yocto Project
This page intentionally left blank
Embedded Linux
Systems with the
TM
Yocto Project
Rudolf J. Streif
Boston • Columbus • Indianapolis • New York • San Francisco • Amsterdam • Cape Town
Dubai • London • Madrid • Milan • Munich • Paris • Montreal • Toronto • Delhi • Mexico City
São Paulo • Sidney • Hong Kong • Seoul • Singapore • Taipei • Tokyo
Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in this book, and the publisher was
aware of a trademark claim, the designations have been printed with initial capital letters or in all
capitals.
The author and publisher have taken care in the preparation of this book, but make no expressed
or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is
assumed for incidental or consequential damages in connection with or arising out of the use of the
information or programs contained herein.
For information about buying this title in bulk quantities, or for special sales opportunities (which
may include electronic versions; custom cover designs; and content particular to your business,
training goals, marketing focus, or branding interests), please contact our corporate sales depart-
ment at [email protected] or (800) 382-3419.
For government sales inquiries, please contact [email protected].
For questions about sales outside the U.S., please contact [email protected].
Visit us on the Web: informit.com
Cataloging-in-Publication Data is on file with the Library of Congress.
Copyright © 2016 Pearson Education, Inc.
All rights reserved. Printed in the United States of America. This publication is protected by copy-
right, and permission must be obtained from the publisher prior to any prohibited reproduction,
storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical,
photocopying, recording, or likewise. For information regarding permissions, request forms and
the appropriate contacts within the Pearson Education Global Rights & Permissions Department,
please visit www.pearsoned.com/permissions/.
ISBN-13: 978-0-13-344324-0
ISBN-10: 0-13-344324-8
Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana.
First printing, May 2016
❖
To Janan, Dominic, Daniel, and Jonas
❖
This page intentionally left blank
Contents
Foreword xv
Preface xvii
Acknowledgments xxi
About the Author xxiii
4.4.3 Inclusion 76
4.4.4 Inheritance 77
4.4.5 Executable Metadata 79
4.4.6 Metadata Attributes 85
4.4.7 Metadata Name (Key) Expansion 86
4.5 Source Download 86
4.5.1 Using the Fetch Class 87
4.5.2 Fetcher Implementations 88
4.5.3 Mirrors 94
4.6 HelloWorld—BitBake Style 95
4.7 Dependency Handling 99
4.7.1 Provisioning 99
4.7.2 Declaring Dependencies 101
4.7.3 Multiple Providers 101
4.8 Version Selection 102
4.9 Variants 103
4.10 Default Metadata 103
4.10.1 Variables 103
4.10.2 Tasks 107
4.11 Summary 107
4.12 References 108
5 Troubleshooting 109
5.1 Logging 110
5.1.1 Log Files 110
5.1.2 Using Logging Statements 114
5.2 Task Execution 116
5.2.1 Executing Specific Tasks 118
5.2.2 Task Script Files 118
5.3 Analyzing Metadata 119
5.4 Development Shell 120
5.5 Dependency Graphs 121
5.6 Debugging Layers 122
5.7 Summary 124
Index 429
Foreword
Ttechnology
he embedded Linux landscape is a little bit like the Old West: different outposts of
scattered here and there, with barren and often dangerous landscape in
between. If you’re going to travel there, you need to be well stocked, be familiar with
the territory, and have a reliable guide.
Just as people moved West during the Gold Rush in the mid-1800s, developers are
moving into the embedded Linux world with the rush to the Internet of Things. As
increased population brought law, order, and civilization to the Old West, important
new open source software projects are bringing order to embedded Linux.
The Yocto Project is a significant order-bringer. Its tools let you focus on designing
your project (what you want to build) and devote only the necessary minimum of your
time and effort to putting it all together (how you build what you want to build).
This book is your reliable guide. In logically ordered chapters with clear and com-
plete instructions, it will help you get your work done and your IoT project to market.
And with some luck, you’ll have fun along the way!
Enjoy your adventure!
Arnold Robbins
Series Editor
This page intentionally left blank
Preface
S mart home. Smart car. Smart phone. Smart TV. Smart thermostat. Smart lights.
Smart watch. Smart washer. Smart dryer. Smart fridge. Smart basketball. Welcome to
the brave new world of smart everything!
The proliferation of embedded computers in almost everything we touch and inter-
act with in our daily lives has moved embedded systems engineering and embedded
software development into the spotlight. Hidden from the direct sight of their users,
embedded systems lack the attractiveness of web applications with their f lashy user
interfaces or the coolness of computer games with their animations and immersive
graphics. It comes as no surprise that computer science students and software developers
hardly ever think of embedded software engineering as their first career choice. How-
ever, the “smart-everything revolution” and the Internet of Things (IoT) are driving
the demand for specialists who can bridge hardware and software worlds. Experts who
speak the language of electric schematics as well as programming languages are sought
after by employers.
Linux has become the first choice for an explosively growing number of embedded
applications. There are good reasons for this choice, upon which we will elaborate in the
coming chapters. Through my journey as an embedded software developer for vari-
ous industries, I have learned Linux for embedded systems the hard way. There is no
shortage of excellent development tools for virtually any programming language. The
vast majority of libraries and applications for Linux can easily be built natively because
of their tooling. Even building the Linux kernel from scratch is almost a breeze with
the kernel’s own build system. However, when it comes to putting it all together into a
bootable system, the choices are scarce.
The Yocto Project closes that gap by providing a comprehensive set of integrated
tools with the OpenEmbedded build system at its center. From source code to bootable
system in a matter of a few hours—I wish I had that luxury when I started out with
embedded Linux!
how the build system assembles the many different components into an operational
system), I do not go into the details of embedded Linux as such. If you are a beginning
embedded Linux developer, I strongly recommend Christopher Hallinan’s excellent
Embedded Linux Primer, published in this same book series.
In this book, you will learn how the OpenEmbedded build system works, how you
can write recipes to build your own software components, how to use and create Yocto
Project board support packages to support different hardware platforms, and how to
debug build failures. You will learn how to build software development kits for applica-
tion development and integrate them with the popular Eclipse integrated development
environment (IDE) for seamless round-trip development.
Chapter 7, “Building a Custom Linux Distribution,” details how to use the Yocto
Project to create your own customized Linux distribution. It starts with an overview of
the Linux distribution blueprints available with the build system and how to customize
them. It then demonstrates how to create a Linux distribution entirely from scratch
using the build system tools. After completing this chapter, you will know how to build
your own operating system images.
Chapter 8, “Software Package Recipes,” explains BitBake recipes and how to write
them to build your own software packages with the build system. The chapter provides
various real-world recipe examples that you can try.
Chapter 9, “Kernel Recipes,” examines the details of building the Linux kernel with
the OpenEmbedded build system. It explains how the build system tooling interacts
with the kernel’s own build environment to set kernel configuration and apply patches.
A discussion of how the build system handles out-of-tree kernel modules and incorpo-
rates building device trees with the build process closes this chapter.
Chapter 10, “Board Support Packages,” introduces how the build system supports
building for different hardware—that is, CPU architectures and systems. After an
explanation of the Yocto Project board support package concepts, the chapter details
how you can build a project using a board support package. We then look into the
internals of Yocto Project board support packages and explain how to create your own
with a practical example that you can put to use with actual hardware. The chapter
concludes with creating bootable media images for different hardware configurations.
Chapter 11, “Application Development,” describes Yocto Project support for
developing applications for Linux OS stacks created with the build system. It provides
hands-on instructions on how to build application development toolkits (ADT) that
include all the necessary tools for round-trip application development. Examples illus-
trate how to use an ADT for application development using the command-line tools as
well as with the Eclipse IDE. Step-by-step instructions teach how to remotely run and
debug applications on an actual hardware target.
Chapter 12, “Licensing and Compliance,” discusses requirements for compliance
with open source licenses and the tools the Yocto Project provides to facilitate meeting
them.
Chapter 13, “Advanced Topics,” introduces several tools that help you scale the
Yocto Project to teams. Toaster is a web-based graphical user interface that can be used
to create build systems that can be controlled remotely from a web browser. Build
history is a tool that provides tracking and audit capabilities. With source mirrors, you can
share source packages to avoid repeated downloads and to control source versions for
product delivery. Last but not least, Autobuilder provides an out-of-the-box continuous
build and integration framework for automating builds, quality assurance, and release
processes. Equipped with the knowledge from this chapter, you can effectively set up
team environments for the Yocto Project.
The appendices cover popular open source licenses and alphabetical references of
build system metadata layers and machines.
xx Preface
Hands-on Experience
The book is written to provide you with hands-on experience using the Yocto Project.
You will benefit the most if you follow along and try out the examples. The majority
of them you can work through simply with an x86-based workstation running a recent
Linux distribution (detailed requirements are provided in Chapter 2). For an even bet-
ter experience, grab one of the popular development boards, such as the BeagleBone,
the MinnowBoard Max, or the Wandboard. The BeagleBone makes an excellent low-
cost experimental platform. The other two boards offer more performance and let you
gain experience with multicore systems.
Analyze the code and try to understand the examples produced in the book. Follow the
steps and then veer off on your own by changing settings, applying your own configu-
ration, and more. It is the best way to learn, and I can tell you, it is a lot of fun too. It is
a great feeling to get your first own Linux distribution to work on a piece of hardware
of your choice.
Register your copy of Embedded Linux Systems with the Yocto Project at informit.com
TM
W hat you are holding in your hands is my first attempt at writing a technical book. Well,
any book, for that matter. I humbly have to admit that I greatly underestimated the effort
that goes into a project like this, the hours spent experimenting with things, finding the
best way to make them work, and documenting everything in a concise and understandable
fashion. During the process, I have come to truly appreciate the work of the many authors
and technical writers whose books and manuals I have read and continue reading.
Foremost, I want to express my gratitude to my family, my loving wife, Janan, and
my three wonderful boys, Dominic, Daniel, and Jonas. Without their support and their
understanding, it would not have been possible for me to spend the many hours writing
this text.
Special thanks go to the Yocto Project team. When I approached Dave Stewart,
Project Manager for the Yocto Project at the time, and Jeffrey Osier-Mixon, the Yocto
Project’s Community Manager, they immediately welcomed the idea for the book and
offered their support. Several individuals from the team were especially helpful with
advice and answers to my questions: Beth Flanagan for Autobuilder, Belen Barros Pena
and Ed Bartosh for Toaster, and Paul Eggleton and Khem Raj who jumped on many of
the questions I posted to the Yocto Project mailing list.
Special thanks to Christopher Hallinan whose Embedded Linux Primer: A Practical Real-
World Approach (Prentice Hall, 2006) inspired me to write this book on the Yocto Project.
I especially want to thank Debra Williams Cauley, Executive Acquisitions Editor,
for her guidance and particularly her patience while this book was in the works. It took
much longer than expected, and I am the only one to blame for the missed deadlines.
I cannot thank and praise enough my dedicated review team, Chris Zahn, Jeffrey
Osier-Mixon, Robert Berger, and Bryan Smith, for their valuable contributions to the
quality of the book in the form of corrections and suggestions for improvements.
I also want to thank the production team at Prentice Hall, Julie Nahil and Anna Popick,
for their coordination and guidance through the process, and in particular Carol Lallier
for her diligence in copyediting the manuscript.
Thanks also to the Linux Foundation and Jerry Cooperstein, who gave me the
opportunity to develop the Linux Foundation’s training course on the Yocto Project.
Nothing teaches as well as teaching somebody else. Thank you to the students of the
classes that I taught. Through your critical questions and feedback, I gained a lot of
understanding for the many different problems you are facing when developing prod-
ucts with embedded Linux. One of your most asked questions was, “Is there a book on
the Yocto Project?” Finally, I can say, “Yes.”
This page intentionally left blank
About the Author
Rudolf Streif has more than twenty years of experience in software engineering as a
developer as well as a manager leading cross-functional engineering teams with more
than one hundred members. Currently, he is an independent consultant for software
technology and system architecture specializing in open source.
He previously served as the Linux Foundation’s Director of Embedded Solutions,
coordinating the Foundation’s efforts for Linux in embedded systems. Rudolf devel-
oped the Linux Foundation’s training course on the Yocto Project, which he delivered
multiple times to companies and in a crash-course variant during Linux Foundation
events.
Rudolf has been working with Linux and open source since the early 1990s and
developing commercial products since 2000. The projects he has been involved with
include high-speed industrial image processing systems, IPTV head-end system and
customer premises equipment, and connected car and in-vehicle infotainment.
In 2014, Rudolf was listed by PC World among the 50 most interesting people in the
world of technology (http://tinyurl.com/z3tbtns).
Rudolf lives with his wife and three children in San Diego, California.
This page intentionally left blank
7
Building a Custom Linux
Distribution
In This Chapter
7.1 Core Images—Linux Distribution Blueprints
7.2 Building Images from Scratch
7.3 Image Options
7.4 Distribution Configuration
7.5 External Layers
7.6 Hob
7.7 Summary
In the preceding chapters, we laid the foundation for using the Yocto Project tools to
build custom Linux distributions. Now it is time that we put that knowledge to work.
Chapter 2, “The Yocto Project,” outlined the prerequisites for the build system and
how to set up your build host, configure a build environment, and launch a build that
creates a system ready to run in the QEMU emulator. In this chapter, we reuse that
build environment. If you have not yet prepared your build system, we recommend that
you go back to Chapter 2 and follow the steps. Performing a build using Poky’s default
settings validates your setup. It also downloads the majority of the source code packages
and establishes a shared state cache, both of which speed up build time for the examples
presented in this chapter.
In Chapter 3, “OpenEmbedded Build System,” and Chapter 4, “BitBake Build
Engine,” we explained the OpenEmbedded build system and the BitBake syntax. This
and following chapters show examples or snippets of BitBake recipes utilizing that syntax.
While the syntax is mostly straightforward and resembles typical scripting languages,
there are some constructs that are particular to BitBake. Referring to Chapter 4, you
find syntax examples and explanations.
When experimenting with the Yocto Project, you eventually encounter build fail-
ures. They can occur for various reasons, and troubleshooting can be challenging. You
may want to refer to Chapter 5, “Troubleshooting,” for the debugging tools to help you
track down build failures.
Chapter 6, “Linux System Architecture,” outlined the building blocks of a Linux
distribution. While bootloader and the Linux kernel are indispensable for a working
146 Chapter 7 Building a Custom Linux Distribution
Linux OS stack, user space makes up its majority. In this chapter, we focus on custom-
izing Linux OS stacks with user space libraries and applications from recipes provided
by the Yocto Project and other compatible layers from the OpenEmbedded project.
You can look at the core images as Linux distribution blueprints from which you
can derive your own distribution by extending them. All core image recipes inherit
the core-image class, which itself inherits from image class. All images set the IMAGE_
INSTALL variable to specify what packages are to be installed into the root filesystem.
IMAGE_INSTALL is a list of packages and package groups. Package groups are collections
7.1 Core Images—Linux Distribution Blueprints 147
of packages. Defining package groups alleviates the need to potentially list hundreds of
single packages in the IMAGE_INSTALL variable. We explain package groups in a coming
section of this chapter. Image recipes either explicitly set Image_INSTALL or extend its
default value provided by the core-image class, which installs the two package groups
packagegroup-core-boot and packagegroup-base-extended . The default creates a
working root filesystem that boots to the console.
Let’s have a closer look at the various core images:
The following three images are not reference images for embedded Linux systems.
We include them in this discussion for completeness purposes.
Studying the reference image recipes is a good way to learn how these images are
built and what packages comprise them. The core images are also a good starting point
for your own Linux OS stack. You can easily extend them by adding packages and
package groups to IMAGE_INSTALL. Images can only be extended, not shrunk. To build
an image with less functionality, you have to start from a smaller core image and add
only the packages you need. There is no simple way to remove packages. The majority
of them are added through package groups, and you would need to split up the package
group if you do not want to install a package included with it. Of course, if you are
removing a package, you also have to remove any other packages that depend on it.
There are several ways you can add packages and package groups to be included with
your root filesystem. The following sections explain them and also provide information
on why you would want to use one method over another.
As we have seen, image recipes set the IMAGE_INSTALL variable adding packages and
package groups. To extend an image, you have to append your packages and packages
group to the variable. You may wonder why we use the explicit _append operator
instead of the += or .+ operators. Using the _append operator unconditionally appends
the specified value to the IMAGE_INSTALL variable after all recipes and configuration
files have been processed. Image recipes commonly explicitly set the IMAGE_INSTALL
variable using the = or ?= operators, which may happen after BitBake processed the
settings in conf/local.conf.
For example, adding
IMAGE_INSTALL_append = " strace sudo sqlite3"
installs the strace and sudo tools as well as SQLite in the root filesystem. When using
the _append operator, you always have to remember to add a space in front of the first
package or package group, as this operator does not automatically include a space.
Using IMAGE_INSTALL in the conf/local.conf of your build environment uncondi-
tionally affects all images you are going to build with this build environment. If you
are looking to install additional packages only to a certain image, you can use condi-
tional appending:
IMAGE_INSTALL_append_pn-<image> = " <package> <package group>"
150 Chapter 7 Building a Custom Linux Distribution
This installs the specified packages and package groups only into the root filesystem of
image. For example,
installs the strace tool only into the root filesystem of core-image-minimal. All other
images are unaffected.
Using IMAGE_INSTALL also affects core images, that is, images that inherit from the
core-image class, as well as images that inherit directly from the image class. For conve-
nience purposes, the core-image class defines the variable CORE_IMAGE_EXTRA_INSTALL.
All packages and package groups added to this variable are appended to IMAGE_INSTALL
by the class. Using
CORE_IMAGE_EXTRA_INSTALL = "strace sudo sqlite3"
adds these packages to all images that inherit from core-image. Images that inherit
directly from image are not affected. Using CORE_IMAGE_EXTRA_INSTALL is a safer and
easier method for core images than appending directly to IMAGE_INSTALL.
n The round-trip time for launching QEMU is much quicker than deploying an
image to actual hardware.
n Frequently, hardware is not yet available when software development begins.
n Yocto Project board support packages (BSP) make it simple to switch from
QEMU to hardware and back.
In Chapter 2, when performing our first build, we used QEMU to verify the build
output. The Poky reference distribution provides the script runqemu that greatly simpli-
fies the task of launching QEMU by providing the necessary parameters. In its simplest
form, you launch the script with a single parameter
$ runqemu qemux86
which tells the script to locate the latest kernel and root filesystem image builds for the
provided QEMU machine and otherwise launch QEMU with default parameters. The
parameter values match the QEMU machine types in conf/local.conf.
When working with different root filesystem images, you probably want to select the
particular image when running QEMU. For example, you have built core-image-minimal
and core-image-base using the preceding command line, since runqemu launches what-
ever image you last built. Using the command as follows lets you choose the image:
$ runqemu qemux86 core-image-minimal
7.1 Core Images—Linux Distribution Blueprints 151
The script automatically selects the correct kernel and uses the latest core-image-
minimal root filesystem. For even more control, you can directly specify the kernel
image and root filesystem image file:
$ runqemu <path>/bzImage-qemux86.bin <path>/core-image-minimal-qemux86.ext3
QEMU and the runqemu script are handy tools for rapid round-trip application
development, which we explore in Chapter 11, “Application Development.”
to the conf/local.conf file of your build environment. Please note that INHERIT
(uppercase) is a variable to which you have to add the buildhistory class. It is different
from the inherit (lowercase) directive used by recipes and classes to inherit functional-
ity from classes. Every time you do a build, buildhistory creates a commit to the
Git repository with the changes.
The buildhistory Git repository is stored in a directory as defined by the BUILDHISTORY_
DIR variable. The default value of this variable is set to
BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
After enabling buildhistory and running a build, you see a buildhistory directory
added to the top-level directory of your build environment. The directory contains
the two subdirectories images and packages. The former contains build information
about the images you build, the latter information on the packages. We analyze the
buildhistory Git repository in Chapter 13, “Advanced Topics.” Here we just look at
the images subdirectory. Inside the images subdirectory, the images are sorted into
further subdirectories by target machine, target C library, and image name:
${TOPDIR}/buildhistory/images/<machine>/<clib>/<image>
For the build of our core-image-minimal for qemux86 using the default EGLIBC
target library, you find the image history in
${TOPDIR}/buildhistory/images/qemux86/eglibc/core-image-mininal
152 Chapter 7 Building a Custom Linux Distribution
The files in that directory give you detailed information on what makes up your
image:
require recipes-core/images/core-image-base.bb
The example includes the recipe for core-image-base and adds packages to IMAGE_
INSTALL and an image feature to IMAGE_FEATURES. We explain what image features are
and how to utilize them to customize image in the next section.
A couple of things to consider when extending images with recipes:
n Unlike classes, you need to provide the path relative to the layer for BitBake to
find the recipe file to include, and you need to add the .bb file extension.
n While you can use either include or require to include the recipe you are
extending, we recommend the use of require, since it causes BitBake to exit with
an explicit error message if it cannot locate the included recipe file.
n Remember to use the += operator to add to IMAGE_INSTALL. Do not use = or :=
because they overwrite the content of the variable defined by the included recipe.
7.1 Core Images—Linux Distribution Blueprints 153
For BitBake to actually be able to use this recipe as a build target, you have to add it
to a layer that is included into your build environment via the conf/bblayers.conf file.
It is not recommended that you add your recipes to the core Yocto Project layers, such
as meta, meta-yocto, and others, because it makes it hard to maintain your build envi-
ronment if you upgrade to a newer version of the Yocto Project. Instead, you should
create a layer in which to put your recipes.
Creating a layer for one recipe may seem like a lot of overhead, but hardly any proj-
ect ever stays small. What may start with one recipe eventually grows into a sophisti-
cated project with recipes for images, packages, and package groups. In Chapter 3, we
introduced the yocto-layer, which makes creating layers a breeze.
n Defined by image.bbclass:
n debug-tweaks : Prepares an image for development purposes. In particular, it sets
empty root passwords for console and Secure Shell (SSH) login.
n package-management : Installs the package management system according to the
works only if System V Init (SysVinit) system is used rather than sytemd.
n splash : Enables showing a splash screen instead of the boot messages during
boot. By default, the splash screen is provided by the psplash package, which
can be customized. You can also define an alternative splash screen package by
setting the SPLASH variable to a different package name.
154 Chapter 7 Building a Custom Linux Distribution
n Defined by populate_sdk_base.bbclass:
n dbg-pkgs : Installs the debug packages containing symbols for all packages
root filesystem.
n staticdev-pkgs : Installs the static development packages such as static library
files ending in *.a for all packages installed in the root filesystem.
n ptest-pkgs : Installs the package test (ptest) packages for all packages installed in
IDE, namely the GDB debugging server, the Eclipse Target Communication
Framework (TCF) agent, and the OpenSSH SFTP server.
n hwcodecs : Installs the hardware decoders and encoders for audio, images, and
patible with ssh-server-dropbear. Either one of the two, but not both, can be
used.
n tools-debug : Installs debugging tools, namely the GDB debugger, the GDB
remote debugging server, the system call tracing tool strace, and the memory
tracing tool mtrace for the GLIBC library if it is the target library.
n tools-profile : Installs common profiling tools such as oprofile , powertop,
packages like the telephony manager oFono and the connection manager
ConnMan.
n x11 : Installs the X11 server.
n x11-sato : Installs the OpenedHand Sato user experience for mobile devices.
7.1 Core Images—Linux Distribution Blueprints 155
It matters what classes define the image features when creating your own image
recipes and choosing the image class to inherit. The class image inherits populate_sdk_
base and thus all image features defined by those two classes are available to images
that inherit image. Image features defined by core-image are available only to images
that inherit that class, which in turn inherits image and with it also populate_sdk_base.
from the installation directory of the Yocto Project build system gives you a list of all
the package group recipes for the predefined package groups of the Yocto Project build
system.
Following are the most common predefined package groups:
the compiler collection cpp; gcc; g++; the GNU internationalization and localiza-
tion tool gettext; make; libstc++ with development packages; and pkgconfig.
n packagegroup-core-tools-debug : Provides the essential debugging tools, namely
the GDB debugger, the GDB remote debugging server, the system call tracing
tool strace, and, for the GLIBC target library, the memory tracing tool mtrace.
n packagegroup-core-sdk : This package group combines the packagegroup-core-
buildessential package group with additional tools for development such as
GNU Core Utilities coreutils with shell, file, and text manipulation utilities;
dynamic linker ldd; and others. Together with packagegroup-core-standalone-
sdk-target, this package group forms the tools-sdk image feature.
n packagegroup-core-standalone-sdk-target: Provides the GCC and standard
C++ libraries. Together with packagegroup-core-sdk, this package group forms
the tools-sdk image feature.
n packagegroup-core-eclipse-debug : Provides the GDB debugging server,
the Eclipse TCF agent, and the OpenSSH SFTP server for integration with
the Eclipse IDE for remote deployment and debugging. The image feature
eclipse-debug installs this package group.
n packagegroup-core-tools-testapps : Provides test applications such as tests for
X11 and middleware packages like the telephony manager oFono and the connec-
tion manager ConnMan. The tools-testapps image feature installs this package
group.
n packagegroup-self-hosted : Provides all necessary packages for a self-hosted build
system. The build-appliance image target uses this package group.
n packagegroup-core-boot: Provides the minimum set of packages necessary to
create a bootable image with console. All core-image targets install this package
group. The core-image-minimal installs just this package group and the postinstal-
lation scripts.
n packagegroup-core-nfs : Provides NFS server, utilities, and client. The nfs-server
image feature installs this package group.
n packagegroup-base : This recipe provides multiple package groups that depend
on each other as well as on machine and distribution configuration. The purpose
of these package groups is to add hardware, networking protocol, USB, filesystem,
and other support to the images dependent on the machine and distribution
configuration. The two top-level package groups are packagegroup-base and
packagegroup-base-extended . The former adds hardware support for Blue-
tooth, WiFi, 3G, and NFC only if both the machine configuration and the
distribution configuration require them. The latter also adds configuration for
those technologies if the distribution configuration requires them. However, the
machine configuration does not support them directly but provides support for PCI,
PCMCIA, or USB host. This package group allows you to create an image with
support for devices that can physically be added to the target device; for exam-
ple, via USB hotplug. Most commonly, images providing hardware support use
7.1 Core Images—Linux Distribution Blueprints 157
When explaining the different package groups, we used the terms provide and install
somewhat liberally, since the package group recipes actually do not provide or install
any packages. They only create dependencies that cause the build system to process the
respective package recipes, as we see in the next section.
Several of the package groups are used by image features, which raises the question
whether to use an image feature or to use the package group the image feature uses.
LICENSE = "MIT"
inherit packagegroup
PACKAGES = "\
packagegroup-databases \
packagegroup-python \
packagegroup-servers"
RDEPENDS_packagegroup-databases = "\
db \
sqlite3"
RDEPENDS_packagegroup-python = "\
python \
python-sqlite3"
RDEPENDS_packagegroup-servers = "\
openssh \
openssh-sftp-server"
7.1 Core Images—Linux Distribution Blueprints 159
RRECOMMENDS_packagegroup-python = "\
ncurses \
readline \
zip"
Names of package group recipes, although not enforced or required by the build sys-
tem, should adhere to the convention packagegroup-<name>.bb. You also would want
to place them in the subdirectory packagegroup of the recipe category the package
groups are integrating. If package groups span recipes and possibly package groups from
multiple categories, it is good practice to place them into the recipes-core category.
The basic structure of package group recipes is rather simple. As should any recipe
(and we go into the details of writing recipes in Chapter 8, “Software Package Reci-
pes”), a package group recipe should provide a SUMMARY of what the recipe does. The
DESCRIPTION, which can provide a longer, more detailed explanation, is optional, but
it is good practice to add it. Any recipe also needs to provide a LICENSE for the recipe
itself. All package group recipes must inherit the packagegroup class.
The names of the actual package groups are defined by the PACKAGES variable.
This variable contains a space-delimited list of the package group names. In the
case of Listing 7-3, these are packagegroup-databases, packagegroup-python, and
packagegroup-servers. By convention, package group names begin with packagegroup-.
Although the build system does not require it, it is good practice if you adhere to it for
your own package group names.
For each package group, the recipe must define its dependencies in a conditional
RDEPENDS_<package-group-name> variable. These variables list the required dependen-
cies, which can be packages or package groups.
The RRECOMMENDS_<package-group-name> definitions are optional. As we saw
in Chapter 3, recommendations are weak dependencies that cause a package to be
included only if it already has been built.
You can reference package groups from other variables, such as IMAGE_INSTALL,
which of course causes these package groups to be installed in a target image. You can
also use them to create dependencies for other package groups for a hierarchy. You must
avoid circular dependencies of package groups. That may sound simple and straightfor-
ward but can easily happen by mistake in rather complex environments. BitBake, how-
ever, aborts with an error message in the case of a circular package group dependency.
Package group recipes can also be directly used as BitBake build targets. For exam-
ple, if the name of the package group recipe is packagegroup-core-iot.bb, you can
build all the packages of the package groups defined by the recipe using
$ bitbake packagegroup-core-iot
Doing so allows testing the package groups before referencing them by image builds,
which simplifies debugging.
160 Chapter 7 Building a Custom Linux Distribution
LICENSE = "MIT"
inherit core-image
The recipe creates an image with the core packages to boot and hardware sup-
port for the target device because the core-image class adds the two package groups
packagegroup-core-boot and packagegroup-base-extended to IMAGE_INSTALL by
default. Also added to IMAGE_INSTALL by the class is the variable CORE_IMAGE_EXTRA_
INSTALL , which allows for simple image modification through conf/local.conf, as
described earlier.
The basic image with package-group-core-boot and package-base-extended pro-
vides a good starting point that easily can be extended by adding to IMAGE_INSTALL and
IMAGE_FEATURES, as shown in Listing 7-5.
LICENSE = "MIT"
# We are using the append operator (+=) below to preserve the default
# values set by the core-image class we are inheriting.
IMAGE_INSTALL += "mtd-utils"
IMAGE_FEATURES += "splash"
inherit core-image
7.3 Image Options 161
LICENSE = "MIT"
inherit core-image
At first glance, the image recipes of Listings 7-5 and 7-6 look rather similar. In fact,
the two recipes produce exactly the same image. The differences are subtle but signifi-
cant. Listing 7-5 uses the append operator += for IMAGE_INSTALL and IMAGE_FEATURES to
take advantage of the default values provided by the core-image class. Listing 7-6 uses
the assignment operator = to purposely overwrite the default values.
Overwriting the default values gives you the most control over the content of your
image, but you also have to take care of the basics yourself. For any image, you would
most likely always want to include packagegroup-core-boot to get a bootable image.
Whether you want the hardware support that packagegroup-base-extended provides
depends on your requirements. Also at your disposal is CORE_IMAGE_EXTRA_INSTALL : if
you do not explicitly add it to IMAGE_FEATURES, you will not be able to use this variable
in conf/local.conf for local customization of your target image, but it may make sense
to do so for a controlled build environment for production.
The same holds true for IMAGE_FEATURES and EXTRA_IMAGE_FEATURES. If you use the
assignment operator with IMAGE_FEATURES and purposely do not add EXTRA_IMAGE_
FEATURES, it is not included, which means that the debug-tweaks image feature is not
applied, and you need to provide passwords for shell and SSH logins. Again, this makes
sense for production build environments where you do not want local configuration
settings to override the settings of your production images.
adds the specific language packages for British English and Brazilian Portuguese to the
image. However, not all software packages provide locales separated by language and
territory. Some of them provide the locale files only by language. In this case, the build
system defaults to installing the correct language local files regardless of the territory.
The minimum default for all packages is en-us and is always installed. In addition,
the image class defines
IMAGE_LINGUAS ?= "de-de fr-fr en-gb"
Any additional locale packages, of course, occupy additional space in your root
filesystem image. Therefore, if your device does not require any additional language
support, it is good practice to set
IMAGE_LINGUAS = ""
in image recipes.
The build system ignores the languages for packages that do not provide them.
You can declare more than one packaging class, but you have to provide at least one.
The build system creates packages for all classes specified; however, only the first pack-
aging class in the list is used to create the root filesystem of your distribution images.
The first packaging class in the list must not be tar.
The build system stores the package feeds organized by the package management
system in separate directories in tmp/deploy/<pms>, where <pms> is the name of the
respective package management system. Inside those directories, the packages are fur-
ther subdivided into common, architecture, and machine-dependent packages.
What package management system should you choose for your project? That
depends on the requirements of your project. Here are some considerations you may
want to take into account:
7.3 Image Options 163
n Opkg creates and utilizes less package metadata than dpkg and RPM. That makes
building faster, and the packages are smaller.
n Dpkg and RPM offer better dependency handling and version management than
opkg because of the enhanced package metadata.
n The RPM package manager is written in Python and requires Python to be
installed on the target to install packages during runtime of the system.
By default, the build system does not install the package manager on your target
system. If you are looking to install packages during runtime of your embedded system,
you have to add the package manager using its image feature:
IMAGE_FEATURES += "package_management"
The build system automatically installs the correct package manager depending on
the first entry of PACKAGE_CLASSES.
The package management system for your root filesystem is ultimately controlled
by the variable IMAGE_PKGTYPE. This variable is set automatically by the order of the
packaging classes defined by PACKAGE_CLASSES. The first packaging class in the list sets
the variable. We recommend that you do not set this variable directly.
After the build system has created the root filesystem in the staging area, a directory
specified by the variable IMAGE_ROOTFS, it calculates its actual size in kilobytes using
du -ks ${IMAGE_ROOTFS}. The function _get_rootfs_size() computes the final root
filesystem image size, as shown by Listing 7-7 in pseudocode.
return IMG_SIZE
creates two root filesystem images, one using the ext3 filesystem and one that is a tar
archive compressed using the bzip2 algorithm.
The image_types class defines the variable IMAGE_TYPES, which contains a list of
all image types you can specify in IMAGE_FSTYPES. The list shows the filesystem types
ordered by core type. Commonly, some of the core types are also used in compressed for-
mats to preserve space. If a compression algorithm is used for the filesystem, the name of
the core type is appended with the compression type: <core name>.<compression type>.
n tar, tar.gz, tar.bz2, tar.xz, tar.lz3: Create uncompressed and compressed root
filesystem images in the form of tar archives.
n ext2, ext2.gz, ext2.bz2, ext2.lzma : Root filesystem images using the ext2 filesys-
tem without or with compression.
7.3 Image Options 165
n ext3, ext3.gz : Root filesystem images using the ext3 filesystem without or with
compression.
n btrfs: Root filesystem image with B-tree filesystem.
n jffs2, jffs2.sum : Uncompressed or compressed root filesystems based on the sec-
ond generation of the Journaling Flash File System ( JFFS2). Since JFFS2 directly
supports NAND f lash devices, it is a popular choice for embedded systems. It also
provides journaling and wear-leveling.
n cramfs : Root filesystem image using the compressed ROM filesystem (cramfs).
The Linux kernel can mount this filesystem without prior decompression. The
compression uses the zlib algorithm that compresses files one page at a time to
allow random access. This filesystem is read-only to simplify its design, as random
write access with compression is difficult to implement.
n iso : Root filesystem image type using the ISO 9660 standard for bootable
CD-ROM. This filesystem type is not a standalone format. It uses ext3 as the
underlying filesystem type.
n hddimg : Root filesystem image for bootable hard drives. It uses ext3 as the actual
filesystem type.
n squashfs, squashfs-xz : Compressed read-only root filesystem type specifically for
Linux, similar to cramfs but with better compression and support for larger files
and filesystems. SquashFS also has a variable block size from 0.5 kB to 64 kB over
the fixed 4 kB block size of cramfs, which allows for larger file and filesystem sizes.
SquashFS uses gzip compression, while squashfs-xz uses Lempel–Ziv–Markov
(LZMA) compression for even smaller images.
n ubi, ubifs: Root filesystem images using the unsorted block image (UBI) format
for raw f lash devices. UBI File System (UBIFS) is essentially a successor to JFFS2.
The main differences between the two is that UBIFS supports write caching.
Using ubifs in IMAGE_FSTYPES just creates the ubifs root filesystem image. Using
ubi creates the ubifs root filesystem image and also runs the ubinize utility to
create an image that can be written directly to a f lash device.
n cpio, cpio.gz, cpio.xz, cpio.lzma : Root filesystem images using uncompressed or
compressed copy in and out (CPIO) streams.
n vmdk : Root filesystem image using the VMware virtual machine disk format. It
uses ext3 as the underlying filesystem format.
n elf: Bootable root filesystem image created with the mkelf Image utility from the
Coreboot project (www.coreboot.org).
Once again, which image types to use depends entirely on the requirements of
your project, particularly on your target hardware. Boot device, bootloader, memory
constraints, and other factors determine what root filesystem types are appropriate for
your project. Our recommendation is to specify the root filesystem types ext3 and
tar, or better, one of the compressed formats such as tar.bz2, in the image recipe. The
166 Chapter 7 Building a Custom Linux Distribution
ext3 format allows you to easily boot your root filesystem with the QEMU emulator
for testing. The tar filesystem can easily be extracted onto partitioned and formatted
media. The machine configuration files for your target hardware can then add addi-
tional root filesystem types appropriate for it.
LICENSE = "MIT"
inherit core-image
inherit extrausers
EXTRA_USERS_PARAMS = "\
groupadd developers; \
useradd -p `openssl passwd ${DEV_PASSWORD}` developer; \
useradd -g developers developer; \
usermod -p `openssl passwd ${ROOT_PASSWORD}` root; \
"
The listing adds a group named developers and a user account named developer
and adds the user account to the group. It also changes the password for the root
account. Commands for adding and modifying groups, users, and passwords are added
to the variable EXTRA_USERS_PARMS, which is interpreted by the class. The commands
understood by the class are
The class executes the respective Linux utilities with the corresponding names.
Hence, the options are exactly the same and can easily be found in the Linux man
pages. Note that the individual commands must be separated with a semicolon.
Using the option -p with the commands useradd and usermod sets the password of
the user account. The password must be provided as the password hash. You can either
calculate the password hash manually and add it to the recipe or, as shown in the exam-
ple, have the recipe calculate it.
A word about the root user account: the build system sets up the root user for an
image with an empty password if debug-tweaks is included with IMAGE_FEATURES.
Removing debug-tweaks replaces the empty root password with *, which disables the
account, so logging in as root from the console is no longer possible. For production
use, we strongly recommend removing debug-tweaks from the build. If your embedded
system requires console login capability, you can either set the root password as shown
previously or add the sudo recipe and set up user accounts as sudoers.
For example, if you want to give the developer user account sudoer privileges, simply
add sudo to IMAGE_INSTALL and usermod -a -G sudo developer to EXTRA_USERS_PARAMS.
LICENSE = "MIT"
inherit core-image
The example adds the bash shell to /etc/shells. Be sure to always use the += operator
to add to ROOTFS_POSTPROCESS_COMMAND, as the build system adds its own postprocessing
commands to it.
Sudo Configuration
If you followed the example on giving a user sudoer privileges in the previous paragraph,
you probably noticed that it does not work unless you uncomment the line %sudo
ALL=(ALL) ALL in /etc/sudoers. A simple shell function added to ROOTFS_POSTPROCESS_
COMMAND takes care of that when the root filesystem image is created (see Listing 7-10).
The script first disables authentication with user name and password for SSH. It then
creates a key pair in tmp/deploy/keys inside the build environment using the name of
7.4 Distribution Configuration 169
the root filesystem image, essentially the name of the image recipe. If a previous build
has already created a set of keys, they are preserved. Finally, the script adds the public
key to the authorized_keys file in /home/root/.ssh, which is typical for SSH configu-
ration. Login keys for other users can be created in a similar way.
This method works well if you do not require different keys for each device that you
build, as every copy of the root filesystem of course contains the same keys. If you need
different keys or, in general, individual configuration for your devices, then you need
to devise a provisioning system for your device production.
The variable setting corresponds to a distribution configuration file whose base name
is the same as the variable’s argument with the file extension .conf. For the preceding
example, the build system searches for a distribution configuration file with the name
poky.conf in the subdirectory conf/distro in all metadata layers included by the build
environment.
n poky: Poky is the default policy for the Yocto Project’s reference distribution Poky.
It is a good choice for getting started with the Yocto Project and as a template for
your own distribution configuration files.
n poky-bleeding : This distribution configuration is based on poky but sets the
versions for all packages to the latest revision. It is commonly used by the Yocto
Project developers for integration test purposes. You may, of course, use it, but be
aware that there could be issues with packages with incompatible versions.
n poky-lsb : This distribution configuration is for a stack that complies with LSB. It
is preferably used with the core-image-lsb image target and image targets derived
from it. It inherits the base settings from poky and adds global configuration set-
tings to enable security and includes default libraries required for LSB compliance.
n poky-tiny: This distribution configuration tailors the settings to yield a very com-
pact Linux OS stack for embedded devices. It is based on poky but provides only
the bare minimum functionality necessary to support the hardware and a BusyBox
170 Chapter 7 Building a Custom Linux Distribution
environment. It does not support any video but only a serial console. Because of
its slim configuration, only the core-image-minimal image target and image tar-
gets based on it can be built with the poky-tiny distribution configuration.
The standard distribution policies, particularly poky, are good starting points for
your own distribution configuration. Let’s have a closer look at the poky distribution
configuration to understand how distribution policies are set and how we can use them
for our own projects.
# SDK Information
SDK_NAME = \
"${DISTRO}-${TCLIBC}-${SDK_ARCH}-${IMAGE_BASENAME}-${TUNE_PKGARCH}"
SDK_VERSION := \
"${@'${DISTRO_VERSION}'.replace('snapshot-${DATE}','snapshot')}"
SDK_VENDOR = "-pokysdk"
SDKPATH = "/opt/${DISTRO}/${SDK_VERSION}"
# Distribution Features
# Override these in poky based distros
POKY_DEFAULT_DISTRO_FEATURES = "largefile opengl ptest multiarch wayland"
POKY_DEFAULT_EXTRA_RDEPENDS = "packagegroup-core-boot"
POKY_DEFAULT_EXTRA_RRECOMMENDS = "kernel-module-af-packet"
# Dependencies
DISTRO_EXTRA_RDEPENDS += " ${POKY_DEFAULT_EXTRA_RDEPENDS}"
DISTRO_EXTRA_RRECOMMENDS += " ${POKY_DEFAULT_EXTRA_RRECOMMENDS}"
POKYQEMUDEPS = "${@bb.utils.contains( \
"INCOMPATIBLE_LICENSE", "GPLv3", "", "qemu-config",d)}"
DISTRO_EXTRA_RDEPENDS_append_qemuarm = " ${POKYQEMUDEPS}"
DISTRO_EXTRA_RDEPENDS_append_qemumips = " ${POKYQEMUDEPS}"
DISTRO_EXTRA_RDEPENDS_append_qemuppc = " ${POKYQEMUDEPS}"
DISTRO_EXTRA_RDEPENDS_append_qemux86 = " ${POKYQEMUDEPS}"
DISTRO_EXTRA_RDEPENDS_append_qemux86-64 = " ${POKYQEMUDEPS}"
# Source Mirrors
PREMIRRORS ??= "\
bzr://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
cvs://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
git://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
gitsm://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
hg://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
osc://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
p4://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
svk://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
svn://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n"
MIRRORS =+ "\
ftp://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
http://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \
https://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n"
CONNECTIVITY_CHECK_URIS ?= " \
https://eula-downloads.yoctoproject.org/index.php \
http://bugzilla.yoctoproject.org/report.cgi"
SANITY_TESTED_DISTROS ?= " \
Poky-1.4 \n \
Poky-1.5 \n \
Poky-1.6 \n \
Ubuntu-12.04 \n \
Ubuntu-13.10 \n \
Ubuntu-14.04 \n \
Fedora-19 \n \
Fedora-20 \n \
CentOS-6.4 \n \
CentOS-6.5 \n \
Debian-7.0 \n \
Debian-7.1 \n \
Debian-7.2 \n \
Debian-7.3 \n \
Debian-7.4 \n \
SUSE-LINUX-12.2 \n \
openSUSE-project-12.3 \n \
openSUSE-project-13.1 \n \
"
The file shown in the listing is from the head of the Yocto Project Git repository
at the writing of this book. Depending on what version of the Yocto Project tools you
are using, this file may look slightly different. The file is an example of a distribution
policy only. It provides the variable settings most commonly associated with the con-
figuration of a distribution. You are not limited to using just the settings shown in the
listing, and you can remove settings if you do not need them for your project.
Distribution Information
This section of the distribution policy file contains settings for general information
about the distribution.
7.4 Distribution Configuration 173
n DISTRO : Short name of the distribution. The value must match the base name of
the distribution configuration file.
n DISTRO_NAME : The long name of the distribution. Various recipes reference this
variable. Its contents are shown on the console boot prompt.
n DISTRO_VERSION : Distribution version string. It is referenced by various recipes and
used in filenames’ distribution artifacts. It is shown on the console boot prompt.
n DISTRO_CODENAME : A code name for the distribution. It is currently used only by
the LSB recipes and copied into the lsb-release system configuration file.
n MAINTAINER : Name and e-mail address of the distribution maintainer.
n TARGET_VENDOR : Target vendor string that is concatenated with various variables,
most notably target system (TARGET_SYS). TARGET_SYS is a concatenation of target
architecture (TARGET_ARCH), target vendor (TARGET_VENDOR), and target operat-
ing system (TARGET_OS), such as i586-poky-linux. The three parts are delimited
by hyphens. The TARGET_VENDOR string must be prefixed with the hyphen, and
TARGET_OS must not. This is one of the many unfortunate inconsistencies of the
OpenEmbedded build system. You may want to set this variable to your or your
company’s name.
SDK Information
The settings in this section provide the base configuration for the SDK.
n SDK_NAME : The base name that the build system uses for SDK output files. It is
derived by concatenating the DISTRO, TCLIBC, SDK_ARCH , IMAGE_BASENAME , and
TUNE_PKGARCH variables with hyphens. There is not much reason for you to change
that string from its default setting, as it provides all the information needed to
distinguish different SDKs.
n SDK_VERSION : SDK version string, which is commonly set to DISTRO_VERSION.
n SDK_VENDOR : SDK vendor string, which serves a similar purpose as TARGET_VENDOR .
Like TARGET_VENDOR , the string must be prefixed with a hyphen.
n SDKPATH : Default installation path for the SDK. The SDK installer offers this path
to the user during installation of an SDK. The user can accept it or enter an alter-
native path. The default value /opt/${DISTRO}/${SDK_VERSION} installs the SDK
into the /opt system directory, which requires root privileges. A viable alternative
would be to install the SDK into the user’s home directory by setting SDKPATH =
"${HOME}/${DISTRO}/${SDK_VERSION}".
Distribution Features
These feature settings provide specific functionality for the distribution.
Preferred Versions
Version settings prescribe particular versions for packages rather than the default
versions.
Dependencies
These settings are declarations for dependencies required for distribution runtime.
Toolchain Configuration
These settings configure the toolchain used for building the distribution.
n TCMODE : This variable selects the toolchain that the build system uses. The default
value is default, which selects the internal toolchain built by the build system
(gcc, binutils, etc.). The setting of the variable corresponds to a configuration file
tcmode-${TCMODE}.inc, which the build system locates in the path conf/distro/
include. This allows including an external toolchain with the build system by
including a toolchain layer that provides the necessary tools as well as the con-
figuration file. If you are using an external toolchain, you must ensure that it is
compatible with the Poky build system.
n TCLIBC : Specifies the C library to be used. The build system currently supports
EGLIBC, uClibc, and musl. The setting of the variable corresponds to a config-
uration file tclibc-${TCLIBC).inc that the build system locates in the path conf/
distro/include. These configuration files set preferred providers for libraries and
more.
n TCLIBCAPPEND : The build system appends this string to other variables to dis-
tinguish build artifacts by C library. If you are experimenting with different C
libraries, you may want to use the settings
TCLIBCAPPEND = "-${TCLIBC}"
TMPDIR .= "${TCLIBCAPPEND}"
7.4 Distribution Configuration 175
Mirror Configuration
The settings in this section configure the mirrors for downloading source packages.
n PREMIRRORS and MIRRORS : The Poky distribution adds these variables to set its mir-
ror configuration to use the Yocto Project repositories as a source for downloads.
If you want to use your own mirrors, you can add them to your distribution con-
figuration file. However, since mirrors are not strictly distribution settings, you
may want to add these variables to the local.conf file of your build environment.
Another alternative would be to add them to the layer.conf file of a custom layer.
n LOCALCONF_VERSION : Sets the expected or required version for the build environ-
ment configuration file local.conf. The build system compares this value to the
value of the variable CONF_VERSION in local.conf. If LOCALCONF_VERSION is a later
version than CONF_VERSION, the build system may be able to automatically upgrade
local.conf to the newer version. Otherwise, the build system exits with an error
message.
n LAYER_CONF_VERSION : Sets the expected or required version for the bblayers.conf
configuration file of a build environment. The build system compares this version
to the value of LCONF_VERSION set by bblayers.conf. If LAYER_CONF_VERSION is a
later version than LCONF_VERSION, the build system may be able to automatically
upgrade bblayers.conf to the newer version. Otherwise, the build system exits
with an error message.
n OELAYOUT_ABI : Sets the expected or required version for the layout of the output
directory TMPDIR . The build system stores the actual layout version in the file
abi_version inside of TMPDIR . If the two are incompatible, the build system exits
with an error message. This typically happens only if you are using a newer ver-
sion of the build system with a build environment that was created by a previous
version and the layout changed incompatibly. Deleting TMPDIR resolves the issue by
re-creating the directory.
n BB_SIGNATURE_HANDLER : The signature handler used for signing shared state cache
entries and creating stamp files. The value references a signature handler function
that, because of its complexity, is typically implemented in Python. The code in
meta/lib/oe/sstatesig.py implements OEBasic and OEBasicHash based on the BitBake
signature generators SignatureGeneratorBasic and SignatureGeneratorBasicHash
defined by bitbake/lib/bb/siggen.py and illustrates how to insert your own sig-
nature handler function. The two signature handlers are principally the same, but
OEBasicHash includes the task code in the signature, which causes any change to
176 Chapter 7 Building a Custom Linux Distribution
metadata to invalidate stamp files and shared state cache entries without explicitly
changing package revision numbers. Using the default value of OEBasicHash is
typically sufficient for most applications.
QA Checks
The QA checks are defined and implemented by meta/classes/insane.bbclass. This
class also defines the QA tasks that are included with the build process. QA checks are
performed after configuration, packaging, and other build tasks. The following two
variables define which QA checks cause warning messages and which checks cause the
build system to terminate the build with an error message:
n WARN_QA : A list of QA checks that create warning messages, but the build
continues
n ERROR_QA : A list of QA checks that create error messages, and the build terminates
The preceding list represents the most common variable settings used by a distribu-
tion configuration. For your own distribution configuration, you may add and/or omit
variables as needed.
adding x11 to DISTRO_FEATURES configures that software package so that it is built with
X11 support. Unlike the x11 image feature, this does not mean that the X11 packages
are installed in your target root filesystem. The distribution feature only prepares a
software package for X11 support so that it uses X11 on a system where the X11 base
packages are installed.
Using DISTRO_FEATURES gives you granular control over how software packages are
built. If you do not need a particular functionality, omitting the distribution feature
enabling it typically results in a smaller footprint for a particular software package.
Using
$ grep -R DISTRO_FEATURES *
from the installation directory of your build system gives you a list of all the recipes and
include files that use DISTRO_FEATURES to conditionally modify configuration settings
or build processes dependent on what distribution features are enabled.
Recipes typically scan DISTRO_FEATURES using
bb.utils.contains('DISTRO_FEATURES', <feature>, <true_val>, <false_val>)
n alsa : Enable support for the Advanced Linux Sound Architecture (ALSA), includ-
ing the installation of open source compatibility modules if available.
n bluetooth : Enable support for Bluetooth.
n cramfs : Enable support for the compressed filesystem CramFS.
n directfb : Enable support for DirectFB.
n ext2 : Enable support and include tools for devices with internal mass storage
devices such as hard disks instead of f lash devices only.
n ipsec : Enable support for authentication and encryption using Internet Protocol
Security (IPSec).
n ipv6: Enable support for Internet Protocol version 6 (IPv6).
178 Chapter 7 Building a Custom Linux Distribution
n irda : Enable support for wireless infrared data communication as specified by the
Infrared Data Association (IrDA).
n keyboard : Enable keyboard support, which includes loading of keymaps during
boot of the system.
n nfs: Enable client NFS support for mounting NFS exports on the system.
n opengl: Include the Open Graphics Library (OpenGL), which is an application
programming interface for rendering 2D and 3D graphics. OpenGL runs on dif-
ferent platforms and provides bindings for most common programming languages.
n pci: Enable support for the PCI bus.
n pcmcia : Enable PCMCIA and CompactFlash support.
n ppp : Enable Point-to-Point Protocol (PPP) support for dial-up networking.
n smbfs : Enable support and include clients for Microsoft’s Server Message Block
(SMB) for sharing remote filesystems, printers, and other devices over networks.
n systemd : Include support for the system management daemon (systemd) that
replaces the SysVinit script-based system for starting up and shutting down a
system.
n sysvinit: Include support for the SysVinit system manager.
n usbgadget: Enable support for the Linux-USB Gadget API Framework that allows
a Linux device to act like a USB device (slave role) when connected to another
system.
n usbhost: Enable USB host support allowing client devices such as keyboards,
mice, cameras, and more to be connected to the system’s USB ports and detected
by it.
n wayland : Enable support for the Wayland compositor protocol and include the
Weston compositor.
n wifi: Enable WiFi support.
n x11: Include the X11 server and libraries.
The list does not include the distribution features for the configuration of the C
library. These distribution features all begin with libc-. They enable support for
functionality provided by the C library if the C library is configurable like the Yocto
Project’s default C library glibc. If you are using glibc, then you do not have to worry
about setting these distribution features, as they are inherited from the default distribu-
tion setup, which is covered in the next section.
If you have already been working with the Yocto Project, you may have noticed
that there is also a variable called MACHINE_FEATURES and that the permissible list of
machine features has a large intersection with the distribution feature list. For example,
both MACHINE_FEATURES and DISTRO_FEATURES provide the feature bluetooth. Enabling
Bluetooth in DISTRO_FEATURES causes the Bluetooth packages for hardware support to
be installed and also enables Bluetooth support for various software packages. However,
7.4 Distribution Configuration 179
enabling Bluetooth in MACHINE_FEATURES only causes the Bluetooth packages for hard-
ware support to be installed. This gives you control over functionality on the machine
and the distribution level. We discuss machine features in detail when we are looking
into Yocto Project board support packages.
The first line installs systemd in the root filesystem. The second line enables it as the
system manager. Installing and enabling systemd does not remove SysVinit from your
root filesystem if it is also included in DISTRO_FEATURES. If you are using one of the
standard distribution configurations, such as poky, then you can remove it from DISTRO_
FEATURES with
DISTRO_FEATURES_BACKFULL_CONSIDERED = "sysvinit"
which is easier than redefining DISTRO_FEATURES altogether. For your own distribution
configuration, you can of course simply omit SysVinit from the DISTRO_FEATURES list.
The SysVinit initscripts to start the individual system services are typically part of
the package that provides the service. To conserve space in the root filesystem, you may
not want to install the initscripts if you want to use systemd exclusively. Use
VIRTUAL-RUNTIME_initscripts = ""
as poky.conf, which we discussed earlier. Instead, the file is included by BitBake’s main
configuration file, bitbake.conf.
Knowing about defaultsetup.conf and understanding its settings is important
because your own distribution policy configuration may extend or overwrite some of
the default variable settings provided by it. If you do not set up the default distribution
correctly, you may inadvertently lose important default settings, and your distribution
build may fail or not yield the desired results.
TCMODE ?= "default"
require conf/distro/include/tcmode-${TCMODE}.inc
TCLIBC ?= "eglibc"
require conf/distro/include/tclibc-${TCLIBC}.inc
USER_CLASSES ?= ""
PACKAGE_CLASSES ?= "package_ipk"
INHERIT_BLACKLIST = "blacklist"
INHERIT_DISTRO ?= "debian devshell sstate license"
INHERIT += "${PACKAGE_CLASSES} ${USER_CLASSES} ${INHERIT_DISTRO} \
${INHERIT_BLACKLIST}"
The file first includes three other files with default settings: default-providers.inc,
default-versions.inc, and default-distrovars.inc. The names for these files are
indicative of what the file content is providing.
The file default-distrovars.inc in particular provides default settings for DISTRO_
FEATURES, DISTRO_FEATURES_DEFAULT, DISTRO_FEATURES_LIBC, and DISTRO_FEATURES_
LIBC_DEFAULT. If you are going to set DISTRO_FEATURES in your own distribution policy
configuration file, you need to pay attention that you do not inadvertently remove the
default settings by overwriting the variable. A safe way of doing so is to use an assign-
ment like
DISTRO_FEATURES ?= "${DISTRO_FEATURES_DEFAULT} ${DISTRO_FEATURES_LIBC} \
${MY_DISTRO_FEATURES}"
MY_DISTRO_FEATURES = "<distro features>"
7.6 Hob 181
which includes all default settings and adds another variable to include additional distri-
bution features as needed.
The configuration file defaultsetup.conf also sets the defaults for TCMODE and
TCLIBC and includes their respective configuration files, as described earlier.
7.6 Hob
Hob is a graphical user interface for BitBake provided by the Yocto Project. It is one of
the Yocto Project’s subprojects and is maintained by the Yocto Project development team.
Why is it called Hob? In the early days of Hob, the three letters stood for
Human-Oriented Builder. However, that does not really sound too appealing and now the
name of the tool is commonly associated with hob, the British English word for cook-
top. And that fits well into the scheme of BitBake and recipes.
With Hob you can conveniently customize your root filesystem images using your
mouse rather than editing text files. If that’s the case, why didn’t we introduce Hob
first rather than explain how to build your custom Linux distribution the “hard” way?
There are a couple of reasons:
1. http://layers.openembedded.org
182 Chapter 7 Building a Custom Linux Distribution
Understanding how this is done manually helps you understand what Hob does in
particular if something goes wrong.
n Although Hob may hide some of the complexity, you still need to know the ter-
minology and how certain variable settings inf luence your build results.
Using Hob is rather simple. First, set up a build environment and then launch Hob
from inside it:
$ source oe-init-build-env build
$ hob
Hob launches and then verifies your build environment. After that check is com-
pleted, you see a screen similar to the one in Figure 7-1 (we already made choices for
the machine and image recipe).
The Hob user interface is easy to understand:
n Select a machine: From the drop-down menu, choose the machine you want
to build for. The list shows all the machines that are defined by any layer included
with the build environment. Selecting the machine changes the MACHINE variable
setting in the con/local.conf file.
n Layers: Click this button to open a graphical editor that lets you include layers
with and remove them from your build environment. Doing so modifies the
conf/bblayers.conf file in your build environment.
n Select an image recipe: From this drop-down menu, you can choose the image
that you want to build. This provides the image target to BitBake similar to run-
ning bitbake <image-target>. The menu contains image targets from all layers
included with your build environment.
n Advanced configuration: Clicking on this button opens a menu that lets you
select root filesystem types, packaging format, distribution policy, image size, and
more, as outlined in Sections 7.3 and 7.4. Hob adds these options to the conf/
local.conf file of the build environment.
n Edit image recipe: This button at the bottom of the screen lets you modify the
image recipe by adding and/or removing packages and/or package groups. Doing
so effectively modifies the IMAGE_INSTALL variable of the image target. You can-
not, however, define new package groups from the Hob user interface. For that
task, you have to write your package group recipe as explained in Section 7.1.6.
But, of course, if you wrote your package recipe and included the layer it resides
in with Hob, then you are able to select it from the package groups list.
n Settings: This button in the upper right corner of the user interface allows you to
modify general settings contained in conf/local.conf such as parallelism, down-
load directory, shared state cache, mirrors, and network proxies. Using the Others
tab, you can add any variable to conf/local.conf and assign a value to it.
n Images: This button next to the Settings button in the upper right corner of the
Hob user interface displays a list of previously built images. The list is created by
parsing the tmp/deploy/images/<machine> subdirectories of the build environment.
You can select an image from the list, run it if it is a QEMU image, or rebuild it.
n Build image: This button launches BitBake with the selected configuration and
image target. The user interface switches to the Log tab of the build view from
which you can follow the build process. This view has a major advantage over the
BitBake output when started from the command line: not only do you see the tasks
that are currently run but also the pending tasks and the ones that already have com-
pleted. If there are any build issues, warnings, or errors, they are logged underneath
the Issues tab. There you can examine build issues and directly view the entire log
file of a task without navigating through the build environment directory structure.
After the build finishes, Hob presents you with a summary page where you can view
the created files in the file browser of your build system. You can also examine a sum-
mary log showing the run results for each task as well as any notes, warnings, or error
messages. If you used Hob to build a root filesystem image and Linux kernel for the
QEMU emulator, you can launch QEMU directly from Hob to verify your image by
clicking on the Run image button in the lower right corner of the user interface. From the
summary page, you can also make changes to your configuration and run a new build.
184 Chapter 7 Building a Custom Linux Distribution
Whether you prefer Hob over configuring your build environment, customizing
your target images, and launching BitBake manually is entirely up to you. Hob is great
for rapid prototyping and to quickly enable somebody who is not all that familiar with
BitBake and the Yocto Project to build predefined root filesystem image targets. Hob
does not allow you to create your own image recipes, nor can you create your own
distribution policy files with it (or even edit them). For these tasks, you need to set up
your own layer and create the necessary files and recipes manually.
From Yocto Project version 2.1 on, Hob is being deprecated in favor of the web-
based Toaster, which we explore in detail in Chapter 13.
7.7 Summary
The largest building block of a Linux distribution is the user space that contains the
various libraries and applications that provide the essential functionality of the system.
This chapter presented the fundamental concepts on how the Poky build system creates
root filesystem images and how you can customize them to meet your requirements.
n The OpenEmbedded build system’s core images provide distribution blueprints
that you can extend and modify.
n Core images can easily be extended by appending packages and package groups to
the list contained in the variable IMAGE_INSTALL.
n The QEMU emulator is a convenient and quick way to test your root file before
booting it on an actual device.
n Enabling the build history lets you track changes to your images and compare
subsequent executions of the build process.
n Creating your own image recipes that build on core image recipes by including
them provides you with more control over what packages your root filesystem
image contains. Image recipes that directly inherit the core-image class let you
build root filesystem images from scratch.
n Package groups are a mechanism to bundle multiple packages and reference
them by a single name, which greatly simplifies image customization with the
IMAGE_INSTALL variable. Poky provides a series of predefined package groups that
organize common packages.
n The build system can produce root filesystem images in various output formats. Some
of them can be written directly to storage media such as flash devices to boot a system.
n Setting up a distribution policy allows operating system configuration indepen-
dent of the content of the root filesystem. It also provides the means to use an
external toolchain with the build system and to change the C library.
n Hob is a graphical user interface for BitBake. Launched from within an initialized
build environment, it allows configuring and building of root filesystem images
without modifying files using a text editor.
Index
Symbols A
" (double quote) ABI (application binary interface), 289
in assignments, 195–196 abi_version file, 52
variable delimiter, 72 --active parameter, 296
/ (forward slash), in symbolic names, 100 Administrative privileges for ordinary users,
- (hyphen), in variable names, 72 28
( ) (parentheses), in license names, 201 ADT (Application Development Toolkit), 26.
:= (colon equal sign), variable expansion, 74 See also SDK (software development kit).
?= (question mark equal), default value components, 302–304
assignment, 73 cross-development toolchain, 302
??= (question marks equal), weak default definition, 301
assignment, 73 description, 26
.= (dot equal), appending variables, 75 Eclipse IDE plugin, 302
' (single quote), variable delimiter, 72 environment setup, 302
@ (at sign), variable expansion, 74 integrating into Eclipse, 27
\ (backslash), line continuation, 195–196 ADT (Application Development Toolkit),
# (hash mark), comment indicator, 21, 71, 19 building applications
% (percent sign), in BitBake version strings, Autotools based, 316, 322–323
102 makefile based, 315–316
+= (plus equal), appending variables, 74 ADT (Application Development Toolkit),
= (equal sign), direct value assignment, 73 Eclipse integration
=. (equal dot), prepending variables, 75 Arguments tab, 326
=+ (equal plus), prepending variables, 74 Autotools-based applications, 322–323
${} (dollar sign curly braces), variable CMake-based applications, 321–322
expansion, 74 Common tab, 326–327
_ (underscore) configuration screen, 320–321
conditional variable setting, 76 Debugger tab, 328–329
in variable names, 72 debugging applications on the target,
. (dot) 327–330
in hidden file names, 226 developing applications, 321–323
in variable names, 72 GDB/CLI command line interpreter, 328
& (ampersand), concatenating license names, GDB/MI command line interpreter, 328
201, 337 Gdbserver Settings subtab, 329
| (pipe symbol) inspecting the target system, 324–325
concatenating license names, 201, 337 installing Eclipse IDE, 317–319
separating kernel names, 237 Main subtab, 329
~ (tilde), in variable names, 72 Main tab, 326
430 Index
Project-specific BitBake variables, 104 Question mark equal (?=), default value
protocol parameter assignment, 73
Git fetcher, 90 Question marks equal (??=), weak default
SVN (Subversion) fetcher, 92 assignment, 73
PROVIDES variable, 99–100, 106, 191
Provisioning R
BitBake dependency handling, 99–101 -r parameter, 64, 68–69, 293
explicit, 100 RANLIB variable, 309
implicit, 99–100 Raspberry Pi 2 B development board, 275
symbolic, 100–101 Raw mode, 292–293
Pseudo, description, 28 RCONFLICTS variable, 195
ptest-pkgs feature, 154 RDEPENDS variable, 101, 194
Public key infrastructure (PKI), 360 --read option, 64, 68–69
PUBLISH_BUILDS parameter, 371 README file, Yocto Project BSPs, 279
PUBLISH_SOURCE_MIRROR parameter, 371 README.sources file, Yocto Project BSPs, 280
PUBLISH_SSTATE parameter, 371 read-only-rootfs feature, 153
PV variable Real time operation, Linux, 2
build metadata, 191 Real-time systems, hard vs. soft, 2
building kernel recipes, 237 rebaseable parameter, 91
explicit provisioning, 100 Recipe files, 70–71, 281–282
runtime variable, 106 Recipes. See also Kernel recipes.
setting package version number, 243 appending files, listing, 123
PXELINUX bootloader, 133 building, 222
Python definition, 33
logging statements, example, 115 extending core images, 152–153
variable expansion, 74 filenames, 186
version verification, 19 formatting source code, 195
Python functions. See also Functions. listing, 123
accessing BitBake variables, 83 listing tasks, 116–117
anonymous, 80 metadata dependent, listing, 124
executable metadata, 79–80 OpenEmbedded workf low, 42–43
formatting guidelines, 196 package groups, 158–159
global, 80 tools and utilities, 7
python keyword, 79–80 updating, 223–224
Python virtual environment, Toaster, Recipes, creating
347–348 architecture-independent packaging, 210
PYTHONHOME variable, 309 common failures, 204
compiling source code, 203–204
Q configuring source code, 202–203
QEMU emulator custom installation scripts, 210–211
application development with, 331–333 establishing the recipe, 198–199
launching, 50 fetching source code, 199–200
launching applications, 333 host leakage, 204
purpose of, 302 installing the build output, 204–206
terminating, 24 licensing information, 201–202
qt4-pkgs feature, 154 machine-dependent packaging, 210
Index 449
missing headers or libraries, 204 Red Hat Package Manager (RPM), 29, 39,
overview, 196–198 162–163
package architecture adjustment, 210 RedBoot bootloader, 131, 134
package QA, 209–210 Release schedule, Yocto Project, 17
package splitting, 207–209 Releases, code names, 277
packaging the build output, 207–210 Relevant bodies. See Organizations.
parallel build failure, 204 Remote on-target debugging, 311–315
patching source code, 201 _remove operator, 75
plausibility and error checking, 209–210 Removing. See also Deleting.
from a script, 50 obsolete directories, 50
setup system services, 206–207 packages, 222
skeleton recipe, 198 values from BitBake metadata, 75
source configuration systems, 203 required directive, 77
systemd, setting up, 207 Required inclusion, 77
SysVinit, setting up, 206–207 Restricted mode, 129
tools for. See Devtool. rev parameter, 92
unpacking source code, 200 Root filesystems
variants, 211 OpenEmbedded system, 52
workf low, 197 tweaking, 167–169
Recipes, examples types of, 164–166
Autotools-based package, 216–217 Root user accounts, 167
C file software, 212–213 --rootfs-dir parameter, 293
CMake-based package, 215–216 ROOTFS_POSTPROCESS_COMMAND, 167–169
externally built package, 217–218 Routing network traffic, distributions for, 5
makefile-based package, 213–215 Royalties, embedded Linux, 2
Recipes, layout RPM (Red Hat Package Manager), 29, 39,
appends, 194 162–163
build metadata, 191–193 RPROVIDES variable, 195
class extensions, 194 RRECOMMENDS variable, 194
code sample, 187–189 RREPLACES variable, 195
descriptive metadata, 189 rsh parameter, 92
includes, 190 RSUGGESTS variable, 194
inheritance directives, 190 run.do file, 118–119
licensing metadata, 190 runqemu script, 50
overview, 186 Runtime dependencies, 99
package manager metadata, 189–190
packaging metadata, 193–194 S
prepends, 194 -s parameter, 284, 292
runtime metadata, 194 S variable, 106, 191, 236
task overrides, 194 SANITY_TESTED_DISTROS variable, 176
variants, 194 saved_tmpdir file, 52
recipes-bsp directory, 281 Scalability, embedded Linux, 3
recipes-core directory, 281 Scaling to teams. See Autobuilder; Build
recipes-graphics directory, 281 history; Mirrors; Toaster.
recipes-kernel directory, 282 Scheduling, 135, 368
Red Hat bootloader. See RedBoot bootloader. SCI (system call interface), 139–140
450 Index