Everand Logo

Welcome to Everand!

  • Discover millions of ebooks, audiobooks, and so much more with a free trial

    From $11.99/month after trial. Cancel anytime.

    Linux: Embedded Development
    Linux: Embedded Development
    Linux: Embedded Development
    Ebook2,183 pages15 hours

    Linux: Embedded Development

    By Alexandru Vaduva, Alex Gonzalez and Simmonds Chris

    ()

    Read preview

    About this ebook

    Leverage the power of Linux to develop captivating and powerful embedded Linux projects

    About This Book
    • Explore the best practices for all embedded product development stages
    • Learn about the compelling features offered by the Yocto Project, such as customization, virtualization, and many more
    • Minimize project costs by using open source tools and programs
    Who This Book Is For

    If you are a developer who wants to build embedded systems using Linux, this book is for you. It is the ideal guide for you if you want to become proficient and broaden your knowledge. A basic understanding of C programming and experience with systems programming is needed. Experienced embedded Yocto developers will find new insight into working methodologies and ARM specific development competence.

    What You Will Learn
    • Use the Yocto Project in the embedded Linux development process
    • Get familiar with and customize the bootloader for a board
    • Discover more about real-time layer, security, virtualization, CGL, and LSB
    • See development workflows for the U-Boot and the Linux kernel, including debugging and optimization
    • Understand the open source licensing requirements and how to comply with them when cohabiting with proprietary programs
    • Optimize your production systems by reducing the size of both the Linux kernel and root filesystems
    • Understand device trees and make changes to accommodate new hardware on your device
    • Design and write multi-threaded applications using POSIX threads
    • Measure real-time latencies and tune the Linux kernel to minimize them
    In Detail

    Embedded Linux is a complete Linux distribution employed to operate embedded devices such as smartphones, tablets, PDAs, set-top boxes, and many more. An example of an embedded Linux distribution is Android, developed by Google.

    This learning path starts with the module Learning Embedded Linux Using the Yocto Project. It introduces embedded Linux software and hardware architecture and presents information about the bootloader. You will go through Linux kernel features and source code and get an overview of the Yocto Project components available.

    The next module Embedded Linux Projects Using Yocto Project Cookbook takes you through the installation of a professional embedded Yocto setup, then advises you on best practices. Finally, it explains how to quickly get hands-on with the Freescale ARM ecosystem and community layer using the affordable and open source Wandboard embedded board.

    Moving ahead, the final module Mastering Embedded Linux Programming takes you through the product cycle and gives you an in-depth description of the components and options that are available at each stage. You will see how functions are split between processes and the usage of POSIX threads.

    By the end of this learning path, your capabilities will be enhanced to create robust and versatile embedded projects.

    This Learning Path combines some of the best that Packt has to offer in one complete, curated package. It includes content from the following Packt products:

    • Learning Embedded Linux Using the Yocto Project by Alexandru Vaduva
    • Embedded Linux Projects Using Yocto Project Cookbook by Alex Gonzalez
    • Mastering Embedded Linux Programming by Chris Simmonds
    Style and approach

    This comprehensive, step-by-step, pragmatic guide enables you to build custom versions of Linux for new embedded systems with examples that are immediately applicable to your embedded developments. Practical examples provide an easy-to-follow way to learn Yocto project development using the best practices and working methodologies. Coupled with hints and best practices, this will help you understand embedded Linux better.

    LanguageEnglish
    PublisherPackt Publishing
    Release dateSep 27, 2016
    ISBN9781787124455
    Linux: Embedded Development

    Related to Linux

    Related ebooks

    Operating Systems For You

    View More
    View More

    Related podcast episodes

    Related articles

    • UNLIMITED

      Observability Of The Kernel And Containers

      Apr 4, 2023

      Mihalis Tsoukalos is currently working on Time Series. You can reach him at: @mactsouk. For our final delve into eBPF, we’re tackling applications, the kernel and Docker containers. At the end of the day, all Linux machines execute code for applicat

      10 min read

    • UNLIMITED

      Mark Your Media Files

      Nov 17, 2020

      When using an embedded system as a media centre, you need to know about NFO files. The files keep track of the meta data for your titles. They’re XML files that contain information on the song, movie or other media that you’re accessing. For example,

      6 min read

    • UNLIMITED

      Build Your Own Distro With NixOS

      Apr 4, 2023

      Credit: https://nixos.org Matt Holder has been a fan of the open source methodology for over two decades and uses Linux and other tools where possible. In his spare time, he enjoys listening to music and reading. We’re continuing our look at next-gen

      10 min read

    • UNLIMITED

      Grasp The Kernel Basics

      Mar 8, 2022

      Grasp The Kernel Basics
      3 min read

    • UNLIMITED

      Grasp The Kernel Basics

      Mar 8, 2022

      Grasp The Kernel Basics
      3 min read

    • UNLIMITED

      Using Bsd For Linux Users

      Mar 10, 2020

      Using Bsd For Linux Users
      8 min read

    • UNLIMITED

      Installing Arch Linux

      Apr 19, 2021

      Installing Arch Linux
      5 min read

    • UNLIMITED

      Get Into Gentoo

      May 4, 2021

      Get Into Gentoo
      7 min read

    • UNLIMITED

      Installing Arch Linux

      Jan 5, 2021

      Installing Arch Linux
      5 min read

    • UNLIMITED

      Answers

      Apr 5, 2022

      fixes non-functional FOSS – fast! My computer is running Ubuntu 20.04.03, and Thunderbird is the latest from Ubuntu’s repositories. When I copy my home folder to an external drive I get this error message: This is the relevant part of my backup scrip

      8 min read

    • UNLIMITED

      Answers

      Apr 5, 2022

      fixes non-functional FOSS – fast! My computer is running Ubuntu 20.04.03, and Thunderbird is the latest from Ubuntu’s repositories. When I copy my home folder to an external drive I get this error message: This is the relevant part of my backup scrip

      8 min read

    • UNLIMITED

      Crux 3.7

      Nov 15, 2022

      Crux 3.7
      2 min read

    • UNLIMITED

      Answers

      Dec 15, 2020

      Answers
      8 min read

    • UNLIMITED

      Answers

      Jul 27, 2021

      Answers
      8 min read

    • UNLIMITED

      How Does The Kernel Work?

      Aug 24, 2021

      How Does The Kernel Work?
      2 min read

    • UNLIMITED

      Kernel Minification

      Mar 8, 2022

      Kernel Minification
      5 min read

    • UNLIMITED

      Kernel Minification

      Mar 8, 2022

      Kernel Minification
      5 min read

    • UNLIMITED

      Build And Compile For Embedded Systems

      Nov 17, 2020

      Mats Tage Axelsson and a whole pile of compiling time. Mats Tage Axelsson has spent decades figuring out reasons to use Linux computers while sacrificing real social interactions. Embedded systems are small, specialised units with a key attribute

      4 min read

    • UNLIMITED

      Doctor

      Mar 26, 2024

      > Deploy new PC > Office alternatives > Linux drives I was inspired by your Hydra Mini ITX Chassis build (Feb 2024 issue) to go out and spec a similar system to replace my three-year-old setup (also based on one of your builds, namely the AMD Ryzen 9

      6 min read

    • UNLIMITED

      Answers

      Nov 16, 2021

      Answers
      9 min read

    • UNLIMITED

      Answers

      Jul 25, 2023

      Answers
      9 min read

    • UNLIMITED

      Hot Picks

      Jun 1, 2021

      Hot Picks
      12 min read

    • UNLIMITED

      The Problem Solvers

      Apr 29, 2024

      The Problem Solvers
      6 min read

    Related categories

    Reviews for Linux

    0 ratings

    0 ratings0 reviews

    What did you think?

    Tap to rate

    Review must be at least 10 words

      Book preview

      Linux - Alexandru Vaduva

      (missing alt)

      Table of Contents

      Linux: Embedded Development

      Linux: Embedded Development

      Credits

      Preface

      What this learning path covers

      What you need for this learning path

      Who this learning path is for

      Reader feedback

      Customer support

      Downloading the example code

      Errata

      Piracy

      Questions

      1. Module 1

      1. Introduction

      Advantages of Linux and open source systems

      Embedded systems

      General description

      Examples

      Introducing GNU/Linux

      Introduction to the Yocto Project

      Buildroot

      OpenEmbedded

      Summary

      2. Cross-compiling

      Introducing toolchains

      Components of toolchains

      Delving into C libraries

      Working with toolchains

      Advice on robust programming

      Generating the toolchain

      The Yocto Project reference

      Summary

      3. Bootloaders

      The role of the bootloader

      Comparing various bootloaders

      Delving into the bootloader cycle

      The U-Boot bootloader

      Booting the U-Boot options

      Porting U-Boot

      The Yocto Project

      Summary

      4. Linux Kernel

      The role of the Linux kernel

      Delving into the features of the Linux kernel

      Memory mapping and management

      Page cache and page writeback

      The process address space

      Process management

      Process scheduling

      System calls

      The virtual file system

      Interrupts

      Bottom halves

      Methods to perform kernel synchronization

      Timers

      Linux kernel interaction

      The development process

      Kernel porting

      Community interaction

      Kernel sources

      Configuring kernel

      Compiling and installing the kernel

      Cross-compiling the Linux kernel

      Devices and modules

      Debugging a kernel

      The Yocto Project reference

      Summary

      5. The Linux Root Filesystem

      Interacting with the root filesystem

      Delving into the filesystem

      Device drivers

      Filesystem options

      Understanding BusyBox

      Minimal root filesystem

      The Yocto Project

      Summary

      6. Components of the Yocto Project

      Poky

      Eclipse ADT plug-ins

      Hob and Toaster

      Autobuilder

      Lava

      Wic

      Summary

      7. ADT Eclipse Plug-ins

      The Application Development Toolkit

      Setting up the environment

      Eclipse IDE

      QEMU emulator

      Debugging

      Profiling and tracing

      The Yocto Project bitbake commander

      Summary

      8. Hob, Toaster, and Autobuilder

      Hob

      Toaster

      Autobuilder

      Summary

      9. Wic and Other Tools

      Swabber

      Wic

      LAVA

      Summary

      10. Real-time

      Understanding GPOS and RTOS

      PREEMPT_RT

      Applying the PREEMPT_RT patch

      The Yocto Project -rt kernel

      Disadvantages of the PREEMPT_RT patches

      Linux real-time applications

      Benchmarking

      Meta-realtime

      Summary

      11. Security

      Security in Linux

      SELinux

      Grsecurity

      Security for the Yocto Project

      Meta-security and meta-selinux

      Meta-security

      Meta-selinux

      Summary

      12. Virtualization

      Linux virtualization

      SDN and NFV

      NFV

      ETSI NFV

      SDN

      OPNFV

      Virtualization support for the Yocto Project

      Summary

      13. CGL and LSB

      Linux Standard Base

      Carrier grade options

      Carrier Grade Linux

      Automotive Grade Linux

      Carrier Grade Virtualization

      Specific support for the Yocto Project

      Summary

      2. Module 2

      1. The Build System

      Introduction

      Setting up the host system

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Installing Poky

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Creating a build directory

      How to do it...

      How it works...

      There's more...

      Building your first image

      Getting ready

      How to do it...

      How it works...

      There's more...

      Explaining the Freescale Yocto ecosystem

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Installing support for Freescale hardware

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Building Wandboard images

      How to do it...

      How it works...

      See also

      Troubleshooting your Wandboard's first boot

      Getting ready

      How to do it...

      There's more...

      Configuring network booting for a development setup

      Getting ready

      Installing a TFTP server

      Installing an NFS server

      How to do it...

      Sharing downloads

      Getting ready

      How to do it...

      Sharing the shared state cache

      How to do it...

      There's more...

      Setting up a package feed

      Getting ready

      Versioning packages

      How to do it...

      See also

      Using build history

      How to do it...

      How it works...

      Looking at the build history

      There's more...

      Working with build statistics

      How to do it...

      How it works...

      There's more...

      Debugging the build system

      Getting ready

      Finding recipes

      Dumping BitBake's environment

      Using the development shell

      How to do it...

      Task log and run files

      Adding logging to recipes

      Looking at dependencies

      Debugging BitBake

      Error reporting tool

      There's more...

      2. The BSP Layer

      Introduction

      Creating a custom BSP layer

      How to do it...

      How it works...

      There's more...

      Adding a new machine

      Adding a custom device tree to the Linux kernel

      Adding a custom U-Boot machine

      Adding a custom formfactor file

      Introducing system development workflows

      How to do it...

      How it works...

      External development

      Working directory development

      External source development

      Adding a custom kernel and bootloader

      Getting Ready

      Finding the Linux kernel source

      Finding the U-Boot source

      Developing using a Git repository fork

      How to do it...

      How it works...

      Explaining Yocto's Linux kernel support

      How to do it...

      How it works...

      See also

      Describing Linux's build system

      How to do it...

      How it works...

      There's more...

      Configuring the Linux kernel

      Getting ready

      How to do it...

      There's more...

      Using configuration fragments

      Building the Linux kernel

      How to do it...

      How it works...

      External development

      External source development

      Working directory development

      Building external kernel modules

      Getting ready

      How to do it...

      There's more...

      Debugging the Linux kernel and modules

      How to do it...

      How it works...

      There's more...

      Using dynamic debug

      Rate-limiting debug messages

      See also

      Debugging the Linux kernel booting process

      How to do it...

      How it works...

      Dumping the kernel's printk buffer from the bootloader

      There's more...

      Using the kernel function tracing system

      Getting ready...

      How to do it...

      How it works...

      There's more...

      Filtering function traces

      Enabling trace options

      Using the function tracer on oops

      Getting a stack trace for a given function

      Configuring the function tracer at boot

      See also

      Managing the device tree

      Getting ready

      How to do it...

      How it works...

      The compatible property

      The Wandboard device tree file

      Defining buses and memory-addressable devices

      There's more...

      Modifying and compiling the device tree in Yocto

      See also

      Debugging device tree issues

      How to do it...

      How it works...

      Looking at the device tree from U-Boot

      Looking at the device tree from the Linux kernel

      3. The Software Layer

      Introduction

      Exploring an image's contents

      Getting ready

      How to do it...

      How it works...

      Adding a new software layer

      Getting ready

      How to do it...

      How it works...

      There's more...

      Selecting a specific package version and providers

      How to do it...

      How do we select which provider to use?

      How do we select which version to use?

      How do we select which version not to use?

      Adding supported packages

      Getting ready

      How to do it...

      How it works...

      There's more...

      Configuring packages

      Adding new packages

      Getting ready

      How to do it...

      How it works...

      Package licensing

      Fetching package contents

      Specifying task overrides

      Configuring packages

      Splitting into several packages

      Setting machine-specific variables

      Adding data, scripts, or configuration files

      How to do it...

      Managing users and groups

      Getting ready

      How to do it...

      There's more...

      Using the sysvinit initialization manager

      Getting ready

      How to do it...

      Using the systemd initialization manager

      Getting ready

      How to do it...

      There's more...

      Installing systemd unit files

      Installing package-installation scripts

      Getting ready

      How to do it...

      How it works...

      Reducing the Linux kernel image size

      How to do it...

      How it works...

      Reducing the root filesystem image size

      How to do it...

      How it works...

      Releasing software

      Getting ready

      How to do it...

      There's more…

      See also

      Analyzing your system for compliance

      How to do it...

      There's more

      Working with open source and proprietary code

      How to do it...

      How it works...

      The U-Boot bootloader

      The Linux kernel

      Glibc

      BusyBox

      The Qt framework

      The X Windows system

      There's more...

      See also

      4. Application Development

      Introduction

      Preparing and using an SDK

      Getting ready

      How to do it...

      Using the Application Development Toolkit

      Getting ready

      How to do it...

      How it works...

      Using the Eclipse IDE

      Getting ready

      How to do it...

      There's more...

      See also

      Developing GTK+ applications

      Getting ready

      How to do it...

      There's more...

      Using the Qt Creator IDE

      Getting ready

      How to do it...

      Developing Qt applications

      Getting ready

      How to do it...

      There's more...

      Describing workflows for application development

      How to do it...

      How it works...

      External development

      Working directory development

      External source development

      Working with GNU make

      How to do it...

      See also

      Working with the GNU build system

      Getting ready

      How to do it...

      See also

      Working with the CMake build system

      Getting ready

      How to do it...

      See also

      Working with the SCons builder

      Getting ready

      How to do it...

      See also

      Developing with libraries

      Getting ready

      Building a static library

      Building a shared dynamic library

      How to do it...

      How it works...

      There's more...

      See also

      Working with the Linux framebuffer

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Using the X Windows system

      Getting ready

      How to do it...

      There's more...

      See also

      Using Wayland

      Getting ready

      How to do it...

      There's more...

      See also

      Adding Python applications

      Getting ready

      How to do it...

      There's more...

      Integrating the Oracle Java Runtime Environment

      Getting ready

      How to do it...

      There's more...

      Integrating the Open Java Development Kit

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Integrating Java applications

      Getting ready

      How to do it...

      How it works...

      There's more...

      5. Debugging, Tracing, and Profiling

      Introduction

      Analyzing core dumps

      Getting ready

      How to do it...

      How it works...

      See also

      Native GDB debugging

      Getting ready

      How to do it...

      There's more...

      See also

      Cross GDB debugging

      Getting ready

      How to do it...

      How it works...

      Using strace for application debugging

      Getting ready

      How to do it...

      How it works...

      See also

      Using the kernel's performance counters

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Using static kernel tracing

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Using dynamic kernel tracing

      Getting ready

      How to do it...

      There's more...

      See also

      Using dynamic kernel events

      Getting ready

      How to do it...

      How it works...

      There's more...

      Exploring Yocto's tracing and profiling tools

      Getting ready

      How to do it...

      There's more...

      Tracing and profiling with perf

      Getting ready

      How to do it...

      How it works...

      Reading tracing data

      There's more...

      Profile charts

      Using perf as strace substitute

      See also

      Using SystemTap

      Getting ready

      How to do it...

      How it works...

      See also

      Using OProfile

      Getting ready

      How to do it...

      How it works...

      There's more...

      See also

      Using LTTng

      Getting ready

      How to do it...

      How it works...

      Extending application profiling

      There's more...

      See also

      Using blktrace

      Getting ready

      How to do it...

      How it works...

      There's more...

      3. Module 3

      1. Starting Out

      Selecting the right operating system

      The players

      Project lifecycle

      The four elements of embedded Linux

      Open source

      Licenses

      Hardware for embedded Linux

      Hardware used in this book

      The BeagleBone Black

      QEMU

      Software used in this book

      Summary

      2. Learning About Toolchains

      What is a toolchain?

      Types of toolchain - native versus cross toolchain

      CPU architectures

      Choosing the C library

      Finding a toolchain

      Building a toolchain using crosstool-NG

      Installing crosstool-NG

      Selecting the toolchain

      Anatomy of a toolchain

      Finding out about your cross compiler

      The sysroot, library, and header files

      Other tools in the toolchain

      Looking at the components of the C library

      Linking with libraries: static and dynamic linking

      Static libraries

      Shared libraries

      Understanding shared library version numbers

      The art of cross compiling

      Simple makefiles

      Autotools

      An example: SQLite

      Package configuration

      Problems with cross compiling

      Summary

      3. All About Bootloaders

      What does a bootloader do?

      The boot sequence

      Phase 1: ROM code

      Phase 2: SPL

      Phase 3: TPL

      Booting with UEFI firmware

      Moving from bootloader to kernel

      Introducing device trees

      Device tree basics

      The reg property

      Phandles and interrupts

      Device tree include files

      Compiling a device tree

      Choosing a bootloader

      U-Boot

      Building U-Boot

      Installing U-Boot

      Using U-Boot

      Environment variables

      Boot image format

      Loading images

      Booting Linux

      Automating the boot with U-Boot scripts

      Porting U-Boot to a new board

      Kconfig and U-Boot

      Board-specific files

      Configuration header files

      Building and testing

      Falcon mode

      Barebox

      Getting Barebox

      Building Barebox

      Summary

      4. Porting and Configuring the Kernel

      What does the kernel do?

      Choosing a kernel

      Kernel development cycle

      Stable and long term support releases

      Vendor support

      Licensing

      Building the kernel

      Getting the source

      Understanding kernel configuration

      Using LOCALVERSION to identify your kernel

      Kernel modules

      Compiling

      Compiling the kernel image

      Compiling device trees

      Compiling modules

      Cleaning kernel sources

      Booting your kernel

      BeagleBone Black

      QEMU

      Kernel panic

      Early user space

      Kernel messages

      Kernel command line

      Porting Linux to a new board

      With a device tree

      Without a device tree

      Additional reading

      Summary

      5. Building a Root Filesystem

      What should be in the root filesystem?

      Directory layout

      Staging directory

      POSIX file access permissions

      File ownership permissions in the staging directory

      Programs for the root filesystem

      The init program

      Shell

      Utilities

      BusyBox to the rescue!

      Building BusyBox

      ToyBox – an alternative to BusyBox

      Libraries for the root filesystem

      Reducing size by stripping

      Device nodes

      The proc and sysfs filesystems

      Mounting filesystems

      Kernel modules

      Transfering the root filesystem to the target

      Creating a boot ramdisk

      Standalone ramdisk

      Booting the ramdisk

      Booting with QEMU

      Booting the BeagleBone Black

      Mounting proc

      Building a ramdisk cpio into the kernel image

      Another way to build a kernel with ramdisk

      The old initrd format

      The init program

      Configuring user accounts

      Adding user accounts to the root filesystem

      Starting a daemon process

      A better way of managing device nodes

      An example using devtmpfs

      An example using mdev

      Are static device nodes so bad after all?

      Configuring the network

      Network components for glibc

      Creating filesystem images with device tables

      Putting the root filesytem onto an SD card

      Mounting the root filesystem using NFS

      Testing with QEMU

      Testing with BeagleBone Black

      Problems with file permissions

      Using TFTP to load the kernel

      Additional reading

      Summary

      6. Selecting a Build System

      No more rolling your own embedded Linux

      Build systems

      Package formats and package managers

      Buildroot

      Background

      Stable releases and support

      Installing

      Configuring

      Running

      Creating a custom BSP

      U-Boot

      Linux

      Build

      Adding your own code

      Overlay

      Adding a package

      License compliance

      The Yocto Project

      Background

      Stable releases and support

      Installing the Yocto Project

      Configuring

      Building

      Running

      Layers

      BitBake and recipes

      Customizing images via local.conf

      Writing an image recipe

      Creating an SDK

      License audit

      Further reading

      Summary

      7. Creating a Storage Strategy

      Storage options

      NOR flash

      NAND flash

      Managed flash

      MultiMediaCard and secure digital cards

      eMMC

      Other types of managed flash

      Accessing flash memory from the bootloader

      U-Boot and NOR flash

      U-Boot and NAND flash

      U-Boot and MMC, SD and eMMC

      Accessing flash memory from Linux

      Memory technology devices

      MTD partitions

      MTD device drivers

      The MTD character device, mtd

      The MTD block device, mtdblock

      Logging kernel oops to MTD

      Simulating NAND memory

      The MMC block driver

      Filesystems for flash memory

      Flash translation layers

      Filesystems for NOR and NAND flash memory

      JFFS2

      Summary nodes

      Clean markers

      Creating a JFFS2 filesystem

      YAFFS2

      Creating a YAFFS2 filesystem

      UBI and UBIFS

      UBI

      UBIFS

      Filesystems for managed flash

      Flashbench

      Discard and TRIM

      Ext4

      F2FS

      FAT16/32

      Read-only compressed filesystems

      squashfs

      Temporary filesystems

      Making the root filesystem read-only

      Filesystem choices

      Updating in the field

      Granularity: file, package, or image?

      Atomic image update

      Further reading

      Summary

      8. Introducing Device Drivers

      The role of device drivers

      Character devices

      Block devices

      Network devices

      Finding out about drivers at runtime

      Getting information from sysfs

      The devices: /sys/devices

      The drivers: /sys/class

      The block drivers: /sys/block

      Finding the right device driver

      Device drivers in user-space

      GPIO

      Handling interrupts from GPIO

      LEDs

      I2C

      SPI

      Writing a kernel device driver

      Designing a character device interface

      Anatomy of a device driver

      Compile and load

      Loading kernel modules

      Discovering hardware configuration

      Device trees

      Platform data

      Linking hardware with device drivers

      Additional reading

      Summary

      9. Starting up - the init Program

      After the kernel has booted

      Introducing the init programs

      BusyBox init

      Buildroot init scripts

      System V init

      inittab

      The init.d scripts

      Adding a new daemon

      Starting and stopping services

      systemd

      Building systemd with the Yocto Project and Buildroot

      Introducing targets, services, and units

      Units

      Services

      Targets

      How systemd boots the system

      Adding your own service

      Adding a watchdog

      Implications for embedded Linux

      Further reading

      Summary

      10. Learning About Processes and Threads

      Process or thread?

      Processes

      Creating a new process

      Terminating a process

      Running a different program

      Daemons

      Inter-process communication

      Message-based IPC

      Unix (or local) sockets

      FIFOs and named pipes

      POSIX message queues

      Summary of message-based IPC

      Shared memory-based IPC

      POSIX shared memory

      Threads

      Creating a new thread

      Terminating a thread

      Compiling a program with threads

      Inter-thread communication

      Mutual exclusion

      Changing conditions

      Partitioning the problem

      Scheduling

      Fairness versus determinism

      Timeshare policies

      Niceness

      Real-time policies

      Choosing a policy

      Choosing a real-time priority

      Further reading

      Summary

      11. Managing Memory

      Virtual memory basics

      Kernel space memory layout

      How much memory does the kernel use?

      User space memory layout

      Process memory map

      Swap

      Swap to compressed memory (zram)

      Mapping memory with mmap

      Using mmap to allocate private memory

      Using mmap to share memory

      Using mmap to access device memory

      How much memory does my application use?

      Per-process memory usage

      Using top and ps

      Using smem

      Other tools to consider

      Identifying memory leaks

      mtrace

      Valgrind

      Running out of memory

      Further reading

      Summary

      12. Debugging with GDB

      The GNU debugger

      Preparing to debug

      Debugging applications using GDB

      Remote debugging using gdbserver

      Setting up the Yocto Project

      Setting up Buildroot

      Starting to debug

      Connecting GDB and gdbserver

      Setting the sysroot

      GDB command files

      Overview of GDB commands

      Breakpoints

      Running and stepping

      Information commands

      Running to a breakpoint

      Debugging shared libraries

      The Yocto Project

      Buildroot

      Other libraries

      Just-in-time debugging

      Debugging forks and threads

      Core files

      Using GDB to look at core files

      GDB user interfaces

      Terminal user interface

      Data display debugger

      Eclipse

      Debugging kernel code

      Debugging kernel code with kgdb

      A sample debug session

      Debugging early code

      Debugging modules

      Debugging kernel code with kdb

      Looking at an oops

      Preserving the oops

      Additional reading

      Summary

      13. Profiling and Tracing

      The observer effect

      Symbol tables and compile flags

      Beginning to profile

      Profiling with top

      Poor man's profiler

      Introducing perf

      Configuring the kernel for perf

      Building perf with the Yocto Project

      Building perf with Buildroot

      Profiling with perf

      Call graphs

      perf annotate

      Other profilers: OProfile and gprof

      Tracing events

      Introducing Ftrace

      Preparing to use Ftrace

      Using Ftrace

      Dynamic Ftrace and trace filters

      Trace events

      Using LTTng

      LTTng and the Yocto Project

      LTTng and Buildroot

      Using LTTng for kernel tracing

      Using Valgrind for application profiling

      Callgrind

      Helgrind

      Using strace to show system calls

      Summary

      14. Real-time Programming

      What is real-time?

      Identifying the sources of non-determinism

      Understanding scheduling latency

      Kernel preemption

      The real-time Linux kernel (PREEMPT_RT)

      Threaded interrupt handlers

      Preemptible kernel locks

      Getting the PREEMPT_RT patches

      The Yocto Project and PREEMPT_RT

      High resolution timers

      Avoiding page faults in a real-time application

      Interrupt shielding

      Measuring scheduling latencies

      cyclictest

      Using Ftrace

      Combining cyclictest and Ftrace

      Further reading

      Summary

      A. Bibliography

      Index

      Linux: Embedded Development

      Linux: Embedded Development

      Leverage the power of Linux to develop captivating and powerful embedded Linux projects

      A course in three modules

      Linux: Embedded Development

      BIRMINGHAM - MUMBAI

      Linux: Embedded Development

      Copyright © 2016 Packt Publishing

      All rights reserved. No part of this course may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

      Every effort has been made in the preparation of this course to ensure the accuracy of the information presented. However, the information contained in this course is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this course.

      Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this course by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

      Published on: September 2016

      Published by Packt Publishing Ltd.

      Livery Place

      35 Livery Street

      Birmingham B3 2PB, UK.

      ISBN 978-1-78712-420-2

      www.packtpub.com

      Credits

      Authors

      Alexandru Vaduva

      Alex González

      Chris Simmonds

      Reviewers

      Peter Ducai

      Alex Tereschenko

      Burt Janz

      Dave (Jing) Tian

      Javier Viguera

      Robert Berger

      Tim Bird

      Mathieu Deschamps

      Mark Furman

      Klaas van Gend

      Behan Webster

      Content Development Editor

      Rohit Kumar Singh

      Graphics

      Jason Monteiro

      Production Coordinator

      Aparna Bhagat

      Preface

      An embedded system is a device with a computer inside that doesn't look like a computer. Washing machines, televisions, printers, cars, aircraft, and robots are all controlled by a computer of some sort, and in some cases, more than one. As these devices become more complex, and as our expectations of the things that we can do with them expand, the need for a powerful operating system to control them grows.

      Linux is only one component of the operating system. Many other components are needed to create a working system, from basic tools, such as a command shell, to graphical user interfaces, with web content and communicating with cloud services.

      The Linux kernel together with an extensive range of other open source components allow you to build a system that can function in a wide range of roles.

      The Linux kernel is at the heart of a large number of embedded products being designed today. Over the last 10 years, this operating system has developed from dominating the server market to being the most used operating system in embedded systems, even those with real-time requirements.

      But at the same time, an embedded Linux product is not only the Linux kernel. Companies need to build an embedded system over the operating system, and that's where embedded Linux was finding it difficult to make its place—until Yocto arrived.

      The Yocto Project brings all the benefits of Linux into the development of embedded systems. It provides a standard build system that allows you to develop embedded products in a quick, reliable, and controlled way.

      What this learning path covers

      Module 1, Learning Embedded Linux Using the Yocto, introduces you to embedded Linux software and hardware architecture, cross compiling, bootloader. You will also get an overview of the available Yocto Project components.

      Module 2, Embedded Linux Projects Using Yocto Project, helps you set up and configure the Yocto Project tools. You will learn the methods to share source code and modifications

      Module 3, Mastering Embedded Linux, takes you through the product cycle and gives you an in-depth description of the components and options that are available at each stage.

      What you need for this learning path

      Before reading this learning path, prior knowledge of embedded Linux and Yocto would be helpful, though not mandatory. In this learning path, a number of exercises are available, and to do them, a basic understanding of the GNU/Linux environment would be useful.

      The examples have been tested with an Ubuntu 14.04 LTS system, but any Linux distribution supported by the Yocto Project can be used. Any piece of i.MX-based hardware can be used to follow the examples.

      The versions of the main packages for the target are U-Boot 2015.07, Linux 4.1, Yocto Project 1.8 Fido, and Buildroot 2015.08.

      Who this learning path is for

      If you are a developer who wants to build embedded systems using Linux, this learning path is for you. It is an ideal guide for you to become proficient and broaden your knowledge with examples that are immediately applicable to your embedded developments. A basic understanding of C programming and experience with systems programming is needed. Experienced embedded Yocto developers will find new insight into working methodologies and ARM specific development competence.

      Reader feedback

      Feedback from our readers is always welcome. Let us know what you think about this learning path—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

      To send us general feedback, simply e-mail <[email protected]>, and mention the title in the subject of your message.

      If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

      Customer support

      Now that you are the proud owner of a Packt product, we have a number of things to help you to get the most from your purchase.

      Downloading the example code

      You can download the example code files for this learning path from your account at http://www.packtpub.com. If you purchased this learning path elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

      You can download the code files by following these steps:

      Log in or register to our website using your e-mail address and password.

      Hover the mouse pointer on the SUPPORT tab at the top.

      Click on Code Downloads & Errata.

      Enter the name of the learning path in the Search box.

      Select the learning path for which you're looking to download the code files.

      Choose from the drop-down menu where you purchased this learning path from.

      Click on Code Download.

      You can also download the code files by clicking on the Code Files button on the learning path's webpage at the Packt Publishing website. This page can be accessed by entering the learning path's name in the Search box. Please note that you need to be logged in to your Packt account.

      Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

      WinRAR / 7-Zip for Windows

      Zipeg / iZip / UnRarX for Mac

      7-Zip / PeaZip for Linux

      The code bundle for the learning path is also hosted on GitHub at https://github.com/PacktPublishing/Embedded-Linux-for-Developers. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

      Errata

      Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our learning path —maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this learning path. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your learning path, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

      To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the learning path in the search field. The required information will appear under the Errata section.

      Piracy

      Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

      Please contact us at <[email protected]> with a link to the suspected pirated material.

      We appreciate your help in protecting our authors and our ability to bring you valuable content.

      Questions

      If you have a problem with any aspect of this learning path, you can contact us at <[email protected]>, and we will do our best to address the problem.

      Part 1. Module 1

      Learning Embedded Linux Using the Yocto Project

      Develop powerful embedded Linux systems with theYocto Project components

      Chapter 1. Introduction

      In this chapter, you will be presented with the advantages of Linux and open source development. There will be examples of systems running embedded Linux, which a vast number of embedded hardware platforms support. After this, you will be introduced to the architecture and development environment of an embedded Linux system, and, in the end, the Yocto Project, where its Poky build system's properties and purposes are summarized.

      Advantages of Linux and open source systems

      Most of the information available in this book, and the examples presented as exercises, have one thing in common: the fact that they are freely available for anyone to access. This book tries to offer guidance to you on how to interact with existing and freely available packages that could help an embedded engineer, such as you, and at the same time, also try to arouse your curiosity to learn more.

      Note

      More information on open source can be gathered from the Open Source Initiative (OSI) at http://opensource.org/.

      The main advantage of open source is represented by the fact that it permits developers to concentrate more on their products and their added value. Having an open source product offers access to a variety of new possibilities and opportunities, such as reduced costs of licensing, increased skills, and knowledge of a company. The fact that a company uses an open source product that most people have access to, and can understand its working, implies budget savings. The money saved could be used in other departments, such as hardware or acquisitions.

      Usually, there is a misconception about open source having little or no control over a product. However, the opposite is true. The open source system, in general, offers full control over software, and we are going to demonstrate this. For any software, your open source project resides on a repository that offers access for everyone to see. Since you're the person in charge of a project, and its administrator as well, you have all the right in the world to accept the contributions of others, which lends them the same right as you, and this basically gives you the freedom to do whatever you like. Of course, there could be someone who is inspired by your project and could do something that is more appreciated by the open source community. However, this is how progress is made, and, to be completely honest, if you are a company, this kind of scenario is almost invalid. Even in this case, this situation does not mean the death of your project, but an opportunity instead. Here, I would like to present the following quote:

      Allowing access to others, having external help, modifications, debugging, and optimizations performed on your open source software implies a longer life for the product and better quality achieved over time. At the same time, the open source environment offers access to a variety of components that could easily be integrated in your product if there's a requirement for them. This permits a quick development process, lower costs, and also shifts a great deal of the maintenance and development work from your product. Also, it offers the possibility to support a particular component to make sure that it continues to suit your needs. However, in most instances, you would need to take some time and build this component for your product from zero.

      This brings us to the next benefit of open source, which involves testing and quality assurance for our product. Besides the lesser amount of work that is needed for testing, it is also possible to choose from a number of options before deciding which components fits best for our product. Also, it is cheaper to use open source software, than buying and evaluating proprietary products. This takes and gives back process, visible in the open source community, is the one that generates products of a higher quality and more mature ones. This quality is even greater than that of other proprietary or closed source similar products. Of course, this is not a generally valid affirmation and only happens for mature and widely used products, but here appears the term community and foundation into play.

      In general, open source software is developed with the help of communities of developers and users. This system offers access to a greater support on interaction with the tools directly from developers - the sort of thing that does not happen when working with closed source tools. Also, there is no restriction when you're looking for an answer to your questions, no matter whether you work for a company or not. Being part of the open source community means more than bug fixing, bug reporting, or feature development. It is about the contribution added by the developers, but, at the same time, it offers the possibility for engineers to get recognition outside their working environment, by facing new challenges and trying out new things. It can also be seen as a great motivational factor and a source of inspiration for everyone involved in the process.

      Instead of a conclusion, I would also like to present a quote from the person who forms the core of this process, the man who offered us Linux and kept it open source:

      Embedded systems

      Now that the benefits of open source have been introduced to you, I believe we can go through a number of examples of embedded systems, hardware, software, and their components. For starters, embedded devices are available anywhere around us: take a look at your smartphone, car infotainment system, microwave oven, or even your MP3 player. Of course, not all of them qualify to be Linux operating systems, but they all have embedded components that make it possible for them to fulfill their designed functions.

      General description

      For Linux to be run on any device hardware, you will require some hardware-dependent components that are able to abstract the work for hardware-independent ones. The boot loader, kernel, and toolchain contain hardware-dependent components that make the performance of work easier for all the other components. For example, a BusyBox developer will only concentrate on developing the required functionalities for his application, and will not concentrate on hardware compatibility. All these hardware-dependent components offer support for a large variety of hardware architectures for both 32 and 64 bits. For example, the U-Boot implementation is the easiest to take as an example when it comes to source code inspection. From this, we can easily visualize how support for a new device can be added.

      We will now try to do some of the little exercises presented previously, but before moving further, I must present the computer configuration on which I will continue to do the exercises, to make sure that that you face as few problems as possible. I am working on an Ubuntu 14.04 and have downloaded the 64-bit image available on the Ubuntu website at http://www.ubuntu.com/download/desktop

      Information relevant to the Linux operation running on your computer can be gathered using this command:

      uname –srmpio

      The preceding command generates this output:

      Linux 3.13.0-36-generic x86_64 x86_64 x86_64 GNU/Linux

      The next command to gather the information relevant to the Linux operation is as follows:

      cat /etc/lsb-release

      The preceding command generates this output:

      DISTRIB_ID=Ubuntu DISTRIB_RELEASE=14.04 DISTRIB_CODENAME=trusty DISTRIB_DESCRIPTION=Ubuntu 14.04.1 LTS

      Examples

      Now, moving on to exercises, the first one requires you fetch the git repository sources for the U-Boot package:

      sudo apt-get install git-core git clone http://git.denx.de/u-boot.git

      After the sources are available on your machine, you can try to take a look inside the board directory; here, a number of development board manufacturers will be present. Let's take a look at board/atmel/sama5d3_xplained, board/faraday/a320evb, board/freescale/imx, and board/freescale/b4860qds. By observing each of these directories, a pattern can be visualized. Almost all of the boards contain a Kconfig file, inspired mainly from kernel sources because they present the configuration dependencies in a clearer manner. A maintainers file offers a list with the contributors to a particular board support. The base Makefile file takes from the higher-level makefiles the necessary object files, which are obtained after a board-specific support is built. The difference is with board/freescale/imx which only offers a list of configuration data that will be later used by the high-level makefiles.

      At the kernel level, the hardware-dependent support is added inside the arch file. Here, for each specific architecture besides Makefile and Kconfig, various numbers of subdirectories could also be added. These offer support for different aspects of a kernel, such as the boot, kernel, memory management, or specific applications.

      By cloning the kernel sources, the preceding information can be easily visualized by using this code:

      git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

      Some of the directories that can be visualized are arch/arc and arch/metag.

      From the toolchain point of view, the hardware-dependent component is represented by the GNU C Library, which is, in turn, usually represented by glibc. This provides the system call interface that connects to the kernel architecture-dependent code and further provides the communication mechanism between these two entities to user applications. System calls are presented inside the sysdeps directory of the glibc sources if the glibc sources are cloned, as follows:

      git clone http://sourceware.org/git/glibc.git

      The preceding information can be verified using two methods: the first one involves opening the sysdeps/arm directory, for example, or by reading the ChangeLog.old-ports-arm library. Although it's old and has nonexistent links, such as ports directory, which disappeared from the newer versions of the repository, the latter can still be used as a reference point.

      These packages are also very easily accessible using the Yocto Project's poky repository. As mentioned at https://www.yoctoproject.org/about:

      The Yocto Project is an open source collaboration project that provides templates, tools and methods to help you create custom Linux-based systems for embedded products regardless of the hardware architecture. It was founded in 2010 as a collaboration among many hardware manufacturers, open-source operating systems vendors, and electronics companies to bring some order to the chaos of embedded Linux development.

      Most of the interaction anyone has with the Yocto Project is done through the Poky build system, which is one of its core components that offers the features and functionalities needed to generate fully customizable Linux software stacks. The first step needed to ensure interaction with the repository sources would be to clone them:

      git clone -b dizzy http://git.yoctoproject.org/git/poky

      After the sources are present on your computer, a set of recipes and configuration files need to be inspected. The first location that can be inspected is the U-Boot recipe, available at meta/recipes-bsp/u-boot/u-boot_2013.07.bb. It contains the instructions necessary to build the U-Boot package for the corresponding selected machine. The next place to inspect is in the recipes available in the kernel. Here, the work is sparse and more package versions are available. It also provides some bbappends for available recipes, such as meta/recipes-kernel/linux/linux-yocto_3.14.bb and meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.10.bbappend. This constitutes a good example for one of the kernel package versions available when starting a new build using BitBake.

      Toolchain construction is a big and important step for host generated packages. To do this, a set of packages are necessary, such as gcc, binutils, glibc library, and kernel headers, which play an important role. The recipes corresponding to this package are available inside the meta/recipes-devtools/gcc/, meta/recipes-devtools/binutils, and meta/recipes-core/glibc paths. In all the available locations, a multitude of recipes can be found, each one with a specific purpose. This information will be detailed in the next chapter.

      The configurations and options for the selection of one package version in favor of another is mainly added inside the machine configuration. One such example is the Freescale MPC8315E-rdb low-power model supported by Yocto 1.6, and its machine configuration is available inside the meta-yocto-bsp/conf/machine/mpc8315e-rdb.conf file.

      Note

      More information on this development board can be found at http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8315E.

      Introducing GNU/Linux

      GNU/Linux, or Linux as it's commonly known, represents a name that has a long line of tradition behind it, and is one of the most important unions of open source software. Shortly, you will be introduced to the history of what is offered to people around the world today and the choice available in terms of selecting personal computer operating systems. Most of all, we will look at what is offered to hardware developers and the common ground available for the development of platforms.

      GNU/Linux consists of the Linux kernel and has a collection of user space applications that are put on top of GNU C Library; this acts as a computer operating system. It may be considered as one of the most prolific instances of open source and free software available, which is still in development. Its history started in 1983 when Richard Stallman founded the GNU Project with the goal of developing a complete Unix-like operating system, which could be put together only from free software. By the beginning of the 1990s, GNU already offered a collection of libraries, Unix-like shells, compilers, and text editors. However, it lacked a kernel. They started developing their own kernel, the Hurd, in 1990. The kernel was based on a Mach micro-kernel design, but it proved to be difficult to work with and had a slow development process.

      Meanwhile, in 1991, a Finnish student started working on another kernel as a hobby while attending the University of Helsinki. He also got help from various programmers who contributed to the cause over the Internet. That student's name was Linus Torvalds and, in 1992, his kernel was combined with the GNU system. The result was a fully functional operating system called GNU/Linux that was free and open source. The most common form of the GNU system is usually referred to as a GNU/Linux system, or even a Linux distribution, and is the most popular variant of GNU. Today, there are a great number of distributions based on GNU and the Linux kernel, and the most widely used ones are: Debian, Ubuntu, Red Hat Linux, SuSE, Gentoo, Mandriva, and Slackware. This image shows us how the two components of Linux work together:

      Introducing GNU/Linux

      Although not originally envisioned to run on anything else then x86 PCs, today, the Linux operating system is the most widespread and portable operating system. It can be found on both embedded devices or supercomputers because it offers freedom to its users and developers. Having tools to generate customizable Linux systems is another huge step forward in the development of this tool. It offers access to the GNU/Linux ecosystem to new categories of people who, by using a tool, such as BitBake, end up learning more about Linux, its architecture differences, root filesystem construction and configuration, toolchains, and many other things present in the Linux world.

      Linux is not designed to work on microcontrollers. It will not work properly if it has less then 32 MB of RAM, and it will need to have at least 4 MB of storage space. However, if you take a look at this requirement, you will notice that it is very permissive. Adding to this is the fact that it also offers support for a variety of communication peripherals and hardware platforms, which gives you a clear image of why it is so widely adopted.

      Note

      Well, it may work on 8MB of RAM, but that depends on the application's size as well.

      Working with a Linux architecture in an embedded environment requires certain standards. This is an image that represents graphically an environment which was made available on one of free-electrons Linux courses:

      Introducing GNU/Linux

      The preceding image presents the two main components that are involved in the development process when working with Linux in the embedded devices world:

      Host machine: This is the machine where all the development tools reside. Outside the Yocto world, these tools are represented by a corresponding toolchain cross-compiled for a specific target and its necessary applications sources and patches. However, for an Yocto user, all these packages, and the preparation work involved, is reduced to automatized tasks executed before the actual work is performed. This, of course, has to be prioritized adequately.

      Target machine: This is the embedded system on which the work is done and tested. All the software available on the target is usually cross-compiled on the host machine, which is a more powerful and more efficient environment. The components that are usually necessary for an embedded device to boot Linux and operate various application, involve using a bootloader for basic initiation and loading of the Linux kernel. This, in turn, initializes drivers and the memory, and offers services for applications to interact with through the functions of the available C libraries.

      Note

      There are also other methods of working with embedded devices, such as cross-canadian and native development, but the ones presented here are the most used and offer the best results for both developers and companies when it comes to software development for embedded devices.

      To have a functional Linux operating system on an development board, a developer first needs to make sure that the kernel, bootloader, and board corresponding drives are working properly before starting to develop and integrate other applications and libraries.

      Introduction to the Yocto Project

      In the previous section, the benefits of having an open source environment were presented. Taking a look at how embedded development was done before the advent of the Yocto Project offers a complete picture of the benefits of this project. It also gives an answer as to why it was adopted so quickly and in such huge numbers.

      Using the Yocto Project, the whole process gets a bit more automatic, mostly because the workflow permitted this. Doing things manually requires a number of steps to be taken by developers:

      Select and download the necessary packages and components.

      Configure the downloaded packages.

      Compile the configured packages.

      Install the generated binary, libraries, and so on, on rootfs available on development machine.

      Generate the final deployable format.

      All these steps tend to become more complex with the increase in the number of software packages that need to be introduced in the final deployable state. Taking this into consideration, it can clearly be stated that manual work is suitable only for a small number of components; automation tools are usually preferred for large and complex systems.

      In the last ten years, a number of automation tools could be used to generate an embedded Linux distribution. All of them were based on the same strategy as the one described previously, but they also needed some extra information to solve dependency related problems. These tools are all built around an engine for the execution of tasks and contain metadata that describes actions, dependencies, exceptions, and rules.

      The most notable solutions are Buildroot, Linux Target Image Builder (LTIB), Scratchbox, OpenEmbedded, Yocto, and Angstrom. However, Scratchbox doesn't seem to be active anymore, with the last commit being done in April 2012. LTIB was the preferred build tool for Freescale and it has lately moved more toward Yocto; in a short span of time, LTIB may become deprecated also.

      Buildroot

      Buildroot as a tool tries to simplify the ways in which a Linux system is generated using a cross-compiler. Buildroot is able to generate a bootloader, kernel image, root filesystem, and even a cross-compiler. It can generate each one of these components, although in an independent way, and because of this, its main usage has been restricted to a cross-compiled toolchain that generates a corresponding and custom root filesystem. It is mainly used in embedded devices and very rarely for x86 architectures; its main focus being architectures, such as ARM, PowerPC, or MIPS. As with every tool presented in this book, it is designed to run on Linux, and certain packages are expected to be present on the host system for their proper usage. There are a couple of mandatory packages and some optional ones as well.

      There is a list of mandatory packages that contain the certain packages, and are described inside the Buildroot manual available at http://buildroot.org/downloads/manual/manual.html. These packages are as follows:

      which

      sed

      make (version 3.81 or any later ones)

      binutils

      build-essential (required for Debian-based systems only)

      gcc (version 2.95 or any later ones)

      g++ (version 2.95 or any later ones)

      bash

      patch

      gzip

      bzip2

      perl(version 5.8.7 or any later ones)

      tar

      cpio

      python(version 2.6 or 2.7 ones)

      unzip

      rsync

      wget

      Beside these mandatory packages, there are also a number of optional packages. They are very useful for the following:

      Source fetching tools: In an official tree, most of the package retrieval is done using wget from http, https, or even ftp links, but there are also a couple of links that need a version control system or another type of tool. To make sure that the user does not have a limitation to fetch a package, these tools can be used:

      bazaar

      cvs

      git

      mercurial

      rsync

      scp

      subversion

      Interface configuration dependencies: They are represented by the packages that are needed to ensure that the tasks, such as kernel, BusyBox, and U-Boot configuration, are executed without problems:

      ncurses5 is used for the menuconfig interface

      qt4 is used for the xconfig interface

      glib2, gtk2, and glade2 are used for the gconfig interface

      Java related package interaction: This is used to make sure that when a user wants to interact with the Java Classpath component, that it will be done without any hiccups:

      javac: this refers to the Java compiler

      jar: This refers to the Java archive tool

      Graph generation tools: The following are the graph generation tools:

      graphviz to use graph-depends and -graph-depends

      python-matplotlib to use graph-build

      Documentation generation tools: The following are the tools necessary for the documentation generation process:

      asciidoc, version 8.6.3 or higher

      w3m

      python with the argparse module (which is automatically available in 2.7+ and 3.2+ versions)

      dblatex (necessary for pdf manual generation only)

      Buildroot releases are made available to the open source community at http://buildroot.org/downloads/ every three months, specifically in February, May, August, and November, and the release name has the buildroot-yyyy-mm format. For people interested in giving Buildroot a try, the manual described in the previous section should be the starting point for installing and configuration. Developers interested in taking a look at the Buildroot source code can refer to http://git.buildroot.net/buildroot/.

      Note

      Before cloning the Buildroot source code, I suggest taking a quick look at http://buildroot.org/download. It could help out anyone who works with a proxy server.

      Next, there will be presented a new set of tools that brought their contribution to this field and place on a lower support level the Buildroot project. I believe that a quick review of the strengths and weaknesses of these tools would be required. We will start with Scratchbox and, taking into consideration that it is already deprecated, there is not much to say about it; it's being mentioned purely for historical reasons. Next on the line is LTIB, which constituted the standard for Freescale hardware until the adoption of Yocto. It is well supported by Freescale in terms of Board Support Packages (BSPs) and contains a large database of components. On the other hand, it is quite old and it was switched with Yocto. It does not contain the support of new distributions, it is not used by many hardware providers, and, in a short period of time, it could very well become as deprecated as Scratchbox. Buildroot is the last of them and is easy to use, having a Makefile base format and an active community behind it. However, it is limited to smaller and simpler images or devices, and it is not aware of partial builds or packages.

      OpenEmbedded

      The next tools to be introduced are very closely related and, in fact, have the same inspiration and common ancestor, the OpenEmbedded project. All three projects are linked by the common engine called Bitbake and are inspired by the Gentoo Portage build tool. OpenEmbedded was first developed in 2001 when the Sharp Corporation launched the ARM-based PDA, and SL-5000 Zaurus, which run Lineo, an embedded Linux distribution. After the introduction of Sharp Zaurus, it did not take long for Chris Larson to initiate the OpenZaurus Project, which was meant to be a replacement for SharpROM, based on Buildroot. After this, people started to contribute many more software packages, and even the support of new devices, and, eventually, the system started to show its limitations. In 2003, discussions were initiated around a new build system that could offer a generic build environment and incorporate the usage models requested by the open source community; this was the system to be used for embedded Linux distributions. These discussions started showing results in 2003, and what has emerged today is the Openembedded project. It had packages ported from OpenZaurus by people, such as Chris Larson, Michael Lauer, and Holger Schurig, according to the capabilities of the new build system.

      The Yocto Project is the next evolutionary stage of the same project and has the Poky build system as its core piece, which was created by Richard Purdie. The project started as a stabilized branch of the OpenEmbedded project and only included a subset of the numerous recipes available on OpenEmbedded; it also had a limited set of devices and support of architectures. Over time, it became much more than this: it changed into a software development platform that incorporated a fakeroot replacement, an Eclipse plug-in, and QEMU-based images. Both the Yocto Project and OpenEmbedded now coordinate around a core set of metadata called OpenEmbedded-Core (OE-Core).

      The Yocto Project is sponsored by the Linux Foundation, and offers a starting point for developers of Linux embedded systems who are interested in developing a customized distribution for embedded products in a hardware-agnostic environment. The Poky build system represents one of its core components and is also quite complex. At the center of all this lies Bitbake, the engine that powers everything, the tool that processes metadata, downloads corresponding source codes, resolves dependencies, and stores all the necessary libraries and executables inside the build directory accordingly. Poky combines the best from OpenEmbedded with the idea of layering additional software components that could be added or removed from a build environment configuration, depending on the needs of the developer.

      Poky is build system that is developed with the idea of keeping simplicity in mind. By default, the configuration for a test build requires very little interaction from the user. Based on the clone made in one of the previous exercises, we can do a new exercise to emphasize this idea:

      cd poky source oe-init-build-env ../build-test bitbake core-image-minimal

      As shown in this example, it is easy to obtain a Linux image that can be later used for testing inside a QEMU environment. There are a number of images footprints available that will vary from a shell-accessible minimal image to an LSB compliant image with GNOME Mobile user interface support. Of course, that these base images can be imported in new ones for added functionalities. The layered structure that Poky has is a great advantage because it adds the possibility to extend functionalities and to contain the impact of errors. Layers could be used for all sort of functionalities, from adding support for a new hardware platform to extending the support for tools, and from a new software stack to extended image features. The sky is the limit here because almost any recipe can be combined with another.

      All this is possible because of the Bitbake engine, which, after the environment setup and the tests for minimal systems requirements are met, based on the configuration files and input received, identifies the interdependencies between tasks, the execution order of tasks, generates a fully functional cross-compilation environment, and starts building the necessary native and target-specific packages tasks exactly as they were defined by the developer. Here is an example with a list of the available tasks for a package:

      OpenEmbedded

      Note

      More information about Bitbake and its baking process can be found in Embedded Linux Development with Yocto Project, by Otavio Salvador and Daiane Angolini.

      The metadata modularization is based on two ideas—the first one refers to the possibility of prioritizing the structure of layers, and the second refers to the possibility of not having the need for duplicate work when a recipe needs changes. The layers are overlapping. The most general layer is meta, and all the other layers are usually stacked over it, such as meta-yocto with Yocto-specific recipes, machine specific board support packages, and other optional layers, depending on the requirements and needs of developers. The customization of recipes should be done using bbappend situated in an upper layer. This method is preferred to ensure that the duplication of recipes does not happen, and it also helps to support newer and older versions of them.

      An example of the organization of layers is found in the previous example that specified the list of the available tasks for a package. If a user is interested in identifying the layers used by the test build setup in the previous exercise that specified the list of the available tasks for a package, the bblayers.conf file is a good source of inspiration. If cat is done on this file, the following output will be visible:

      # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf

      # changes incompatibly

      LCONF_VERSION = 6

       

      BBPATH = ${TOPDIR}

      BBFILES ?=

       

      BBLAYERS ?= " \

        /home/alex/workspace/book/poky/meta \

        /home/alex/workspace/book/poky/meta-yocto \

        /home/alex/workspace/book/poky/meta-yocto-bsp \

        "

      BBLAYERS_NON_REMOVABLE ?= " \

        /home/alex/workspace/book/poky/meta \

        /home/alex/workspace/book/poky/meta-yocto \

        "

      The complete command for doing this is:

      cat build-test/conf/bblayers.conf

      Here is a visual mode for the layered structure of a more generic build directory:

      OpenEmbedded

      Yocto as a project offers another important feature: the possibility of having an image regenerated in the same way, no matter what factors change on your host machine. This is a very important feature, taking into consideration not only that, in the development process, changes to a number of tools, such as autotools, cross-compiler, Makefile, perl, bison, pkgconfig, and so on, could occur, but also the fact that parameters could change in the interaction process with regards to a repository. Simply cloning one of the repository branches and applying corresponding patches may not solve all the problems. The solution that the Yocto Project has to these problems is quite simple. By defining parameters prior to any of the steps of the installation as variables and configuration parameters inside recipes, and by making sure that the configuration process is also automated, will minimize the risks of manual interaction are minimized. This process makes sure that image generation is always done as it was the first time.

      Since the development tools on the host machine are prone to change, Yocto usually compiles the necessary tools for the development process of packages and images, and only after their build process is finished, the Bitbake build engine starts building the requested packages. This isolation from the developer's machine helps the development process by guaranteeing the fact that updates from the host machine do not influence or affect the processes of generating the embedded Linux distribution.

      Another critical point that was elegantly solved by the Yocto Project is represented by the way that the toolchain handles the inclusion of headers and libraries; because this could bring later on not only compilation but also execution errors that are very hard to predict. Yocto resolves these problems by moving all the headers and libraries inside the corresponding sysroots directory, and by using the sysroot option, the build process makes sure that no contamination is done with the native components. An example will emphasize this information better:

      ls -l build-test/tmp/sysroots/ total 12K drwxr-xr-x 8 alex alex 4,0K sep 28 04:17 qemux86/ drwxr-xr-x 5 alex alex 4,0K sep 28 00:48 qemux86-tcbootstrap/ drwxr-xr-x 9 alex alex 4,0K sep 28 04:21 x86_64-linux/

       

       

      ls -l build-test/tmp/sysroots/qemux86/ total 24K drwxr-xr-x 2 alex alex 4,0K sep 28 01:52 etc/ drwxr-xr-x 5 alex alex 4,0K sep 28 04:15 lib/ drwxr-xr-x 6 alex alex 4,0K sep 28 03:51 pkgdata/ drwxr-xr-x 2 alex alex 4,0K sep 28 04:17 sysroot-providers/ drwxr-xr-x 7 alex alex 4,0K sep 28 04:16 usr/ drwxr-xr-x 3 alex alex 4,0K sep 28 01:52 var/

      The Yocto project contributes to making reliable embedded Linux

      Enjoying the preview?
      Page 1 of 1