COURSE: EMBEDDED SYSTEM

Chapter 7 Embedded System Development

Assoc. Prof. Dr. Truong Ngoc Son

Faculty of Electrical and Electronics Engineering
HCMC University of Technology and Education
Embedded System Development
• Building Linux kernel
• Building Device driver
Embedded System Development

• Building the kernel

• Customize the kernel to fit the hardware
• Building Device driver
• Write or modify device driver for new hardware
• Application developments
• Build the application for embedded system: IoT, AI, Edge computing, ….
 Embedded System Development
• Cross-Development Environment - cross-compiler
• A “cross-compiler” is a compiler that creates executable
code for a different architecture. In the case of developing
your Linux BSP, you’ll be compiling on your PC but the
code you make is targeting your custom board
• Toolchain: A toolchain is the set of tools that compiles
source code into executables that can run on your target
device, and includes a compiler, a linker, and run-time
libraries
• Cross compilation with Yocto
 Embedded System Development
• Cross-Development Environment
Building the kernel
Kernel Configuration
• The kernel contains thousands of device drivers, filesystem
drivers, network protocols and other configurable items
• Thousands of options are available, that are used to selectively
compile parts of the kernel source code
• The kernel configuration is the process of defining the set of
options with which you want your kernel to be compiled
• The set of options depends
• On the target architecture and on your hardware (for device drivers,
etc.)
• On the capabilities you would like to give to your kernel (network
capabilities,
filesystems, real-time, etc.). Such generic options are available in all
architectures.
Kernel configuration and build system

• The kernel configuration and build system is based on multiple

Makefiles
• One only interacts with the main Makefile, present at the top
directory of the kernel source tree
• Interaction takes place
• using the make tool, which parses the Makefile
• through various targets, defining which action should be done
(configuration,compilation, installation, etc.). Run make help to see all
available targets.
• Example
cd linux-4.14.x/
make <target>
Specifying the target architecture

 First, specify the architecture for the kernel to build

Set ARCH to the name of a directory under arch/:
export ARCH=arm
 By default, the kernel build system assumes that the kernel is
configured and built
for the host architecture (x86 in our case, native kernel
compiling)
 The kernel build system will use this setting to:
 Use the configuration options for the target architecture.
 Compile the kernel with source code and headers for the target
architecture.
 Choose a compiler
 The compiler invoked by the kernel Makefile is $
(CROSS_COMPILE)gcc
Specifying the compiler is already needed at configuration time, as
some kernel configuration options depend on the capabilities of the
compiler.
 When compiling natively
 Leave CROSS_COMPILE undefined and the kernel will be natively
compiled for the host architecture using gcc.
When using a cross-compiler
 To make the difference with a native compiler, cross-compiler executables
are prefixed by the name of the target system, architecture and sometimes
library.
Examples:
mips-linux-gcc: the prefix is mips-linuxarm-linux-gnueabi-gcc: the prefix is
arm-linux-gnueabi-
 So, you can specify your cross-compiler as follows:
export CROSS_COMPILE=arm-linux-gnueabi-
CROSS_COMPILE is actually the prefix of the cross compiling tools
(gcc, as, ld, objcopy, strip...).
Specifying ARCH and CROSS_COMPILE

There are actually two ways of defining ARCH and CROSS_COMPILE:

• Pass ARCH and CROSS_COMPILE on the make command line:
make ARCH=arm CROSS_COMPILE=arm-linux- ...
Drawback: it is easy to forget to pass these variables when you
run any make command, causing your build and configuration to be
screwed up.
• Define ARCH and CROSS_COMPILE as environment variables:
export ARCH=arm
export CROSS_COMPILE=arm-linux
Drawback: it only works inside the current shell or terminal.
You could put these settings in a file that you source every time you
start working on the project. If you only work on a single architecture
with always the same toolchain, you could even put these settings in
your ~/.bashrc file to make them permanent and visible from any
terminal.
Kernel configuration details
• The configuration is stored in the .config file at the root of
kernel sources
• Simple text file, CONFIG_PARAM=value (included by the kernel
Makefile)
• As options have dependencies, typically never edited by hand,
but through graphical or text interfaces:
• make xconfig, make gconfig (graphical)
• make menuconfig, make nconfig (text)
• You can switch from one to another, they all load/save the
same .config file, and show the same set of options
Initial configuration
Difficult to find which kernel configuration will work with your hardware
and root filesystem. Start with one that works!
• Desktop or server case:
• Advisable to start with the configuration of your running kernel, usually
available in
/boot:
cp /boot/config-`uname -r` .config
• Embedded platform case (at least on ARM 32 bit):
• Default configuration files are available, usually for each CPU family.
• They are stored in arch/<arch>/configs/, and are just minimal .config files
(only settings different from default ones).
• Run make help to find if one is available for your platform
• To load a default configuration file, just run
make cpu_defconfig
• This will overwrite your existing .config file!
Now, you can make configuration changes (make menuconfig...).
Create your own default configuration
To create your own default configuration file:
make savedefconfig
This creates a minimal configuration (non-default
settings)
mv defconfig arch/<arch>/configs/myown_defconfig
This way, you can share a reference configuration inside
the kernel sources.
Kernel or module ?
• The kernel image is a single file, resulting from the linking of
all object files that correspond to features enabled in the
configuration
• This is the file that gets loaded in memory by the bootloader
• All included features are therefore available as soon as the kernel
starts, at a time where no filesystem exists
• Some features (device drivers, filesystems, etc.) can however
be compiled as modules
• These are plugins that can be loaded/unloaded dynamically to
add/remove features to the kernel
• Each module is stored as a separate file in the filesystem, and
therefore access to a filesystem is mandatory to use modules
• This is not possible in the early boot procedure of the kernel,
because no filesystem
is available
make menuconfig
• Useful when no graphics are available.
Very efficient interface.
• Same interface found in other tools:
BusyBox, Buildroot...
• Convenient number shortcuts to jump
directly to search results.
• Required Debian packages:
libncurses-dev
Compiling and installing the kernel
• make
Run it in the main kernel source directory!
• Remember to run multiple jobs in parallel if you have multiple
CPU cores / threads. Our advice: ncpus * 2 or ncpus + 2, to
fully load the CPU and I/Os at all times.
Example: make -j 8
• No need to run as root!
• To recompile faster (7x according to some benchmarks),
use the ccache compiler cache:
export CROSS_COMPILE="ccache riscv64-linux-"
Kernel compilation results
• vmlinux, the raw uncompressed kernel image, in the ELF
format, useful for debugging purposes, but cannot be booted
• arch/<arch>/boot/*Image, the final compressed kernel image
that can be booted
• bzImage for x86, zImage for ARM, Image.gz for RISC-V,
vmlinux.bin.gz for ARC, etc.
• arch/<arch>/boot/Image, uncompressed kernel image that
can be booted too
• arch/<arch>/boot/dts/*.dtb, compiled Device Tree files (on
some architectures)
• All kernel modules, spread over the kernel source tree, as .ko
(Kernel Object) files.
Building Kernel for raspberry
 Building Kernel for raspberry Pi 4
Building kernel on Raspberry

Set up

Install Raspberry Image on SD Card (done)

Set up vnc-server
sudo apt-get install realvnc-vnc-server
sudo raspi-config, navigate to Interfacing Options > VNC and select Yes.
Setup FFT
Install vsftpd
Sudo apt-get install vsftpd
Nano /etc/vsftpd.conf
Uncomment # write enable
Service vsftpd restart
Conect Raspberry via VNC Client
Building Kernel for raspberry Pi 4
Install package required to building and compiling
Sudo apt install raspberrypi-kernel-headers build-essential bc git wget
bison flex libssl-dev make libncurses-dev
Create directory to work in
Sudo mkdir kernel
cd kernel
Clone the latest kernel source using git
Sudo git clone --depth=1 https://github.com/raspberrypi/linux
Set up the kernel
cd linux
KERNEL=kernel7l
Sudo make bcm2711_defconfig
Building Kernel for raspberry Pi 4
Configure the kernel
arch/arm/configs/bcm2711_deconfig
Manually configure the kernel (modify version and something else)
Sudo make menuconfig
Compile the kernel
make -j4 zImage modules dtbs
Install the compiled modules
Sudo make modules_install
Building Kernel for raspberry Pi 4
Copy the kernel, modules, and other files to the boot filesystem
Sudo cp arch/arm/boot/dts/*.dtb /boot/
Sudo cp arch/arm/boot/dts/overlays/*.dtb* /boot/overlays/
Sudo cp arch/arm/boot/dts/overlays/README /boot/overlays/
Sudo cp arch/arm/boot/zImage /boot/kernel-2021.img
Configure the PI to boot using the new kernel by modifying and adding
the below line to “/boot/config.txt”.
kernel=kernel-2021.img
 Building Kernel for raspberry Pi 4
Cross-compiling the kernel

Install Required Dependencies and Toolchain

sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-
dev
Install the 32-bit Toolchain for a 32-bit Kernel
sudo apt install crossbuild-essential-armhf
Building Kernel for raspberry Pi 4
Get the Kernel Sources
git clone --depth=1 https://github.com/raspberrypi/linux
cd linux
KERNEL=kernel7l
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
bcm2711_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage
modules dtbs
Building Kernel for raspberry Pi 4
Install the kernel into SD Card, insert SD Card, check the SD Card using
Lsblk
(sdb -> sdb1, sdb2)
with sdb1 being the FAT (boot) partition, and sdb2 being the ext4
filesystem (root) partition
Mount these first, adjusting the partition letter as necessary:
mkdir mnt
mkdir mnt/fat32
mkdir mnt/ext4
sudo mount /dev/sdb1 mnt/fat32
sudo mount /dev/sdb2 mnt/ext4
Building Kernel for raspberry Pi 4
Next, install the kernel modules onto the SD card
sudo env PATH=$PATH make ARCH=arm CROSS_COMPILE=arm-linux-
gnueabihf- INSTALL_MOD_PATH=mnt/ext4 modules_install
sudo cp mnt/fat32/$KERNEL.img mnt/fat32/$KERNEL-backup.img
sudo cp arch/arm/boot/zImage mnt/fat32/$KERNEL.img
sudo cp arch/arm/boot/dts/*.dtb mnt/fat32/
sudo cp arch/arm/boot/dts/overlays/*.dtb* mnt/fat32/overlays/
sudo cp arch/arm/boot/dts/overlays/README mnt/fat32/overlays/
sudo umount mnt/fat32
sudo umount mnt/ext4
Building Kernel for raspberry Pi 4
Another option is to copy the kernel into the same place, but with a
different filename - for instance, kernel-myconfig.img - rather than
overwriting the kernel.img file. You can then edit the config.txt file to
select the kernel that the Pi will boot into:

kernel=kernel-myconfig.img
Configuring the Kernel
Device Driver (Kernel Module)
• Building Device driver
• Install Device driver (Install module)
• Building Kernel with device driver
Device Driver (Kernel Module)
• Build and install Kernel Module on Ubuntu
• Build and install Kernel Module to control GPIO on
raspberry Pi 4
• Build and install Device Driver on Ubuntu
• Build and install Device Driver to control GPIO on
raspberry Pi 4
Trình điều khiển thiết bị (Device Driver, Kernel
Module)
• Biên dịch và cài đặt Kernel Module trên Ubuntu
• Biên dịch và cài đặt Kernel Module điều khiển
GPIO trên raspberry Pi 4
• Biên dịch và cài đặt Device Driver trên Ubuntu
• Biên dịch và cài đặt Device Driver điều khiển
GPIO trên raspberry Pi 4
Device Driver

Hardware Kernel
User Space
GPIO Space
Raspberry Pi 4 I/O
Device Driver (Kernel Module)
Install package
Sudo apt install raspberrypi-kernel-headers
Device Driver (Kernel Module)
Kernel module
Device driver
Kernel Module
File operation structure
The file_operations structure is defined in linux/fs.h, and holds
pointers to functions defined by the driver that perform various
operations on the device. Each field of the structure corresponds to
the address of some function defined by the driver to handle a
requested operation.
File operation structure
struct file_operations {
struct module *owner;
loff_t (*llseek) (struct file *, loff_t, int);
ssize_t (*read) (struct file *, char *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char *, size_t, loff_t *);
int (*readdir) (struct file *, void *, filldir_t);
unsigned int (*poll) (struct file *, struct poll_table_struct *);
int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *);
int (*release) (struct inode *, struct file *);
int (*fsync) (struct file *, struct dentry *, int datasync);
int (*fasync) (int, struct file *, int);
int (*lock) (struct file *, int, struct file_lock *);
ssize_t (*readv) (struct file *, const struct iovec *, unsigned long,
loff_t *);
ssize_t (*writev) (struct file *, const struct iovec *, unsigned long,
loff_t *);
};
Kernel Module - Write my-driver.c
Lab.01.Module
test module
Echo “hello” > /proc/my-driver
Dmeg
Kernel Module - Write my-driver.c
int alloc_chrdev_region ( dev_t * dev,
unsigned baseminor,
unsigned count,
const char * name);

In the kernel versions before 2.6 these two numbers were 8 bit
numbers thus allowing only 256 drivers with each controlling
maximum of 256 devices.

But this was not enough for the modern systems like the big servers
which have a large number of devices connected to them.

Thus from 2.6 these two numbers are combined and stored in one 32
bit number of the data type dev_t. Out of the 32 bits the MSB 12 bits
represent the major number and the LSB 20 bits represent the minor
number
Kernel Module
Install kernel module
Sudo insmod my-driver.ko
Debug
dmesg
Uninstall kernel module
Sudo rmmod my-driver
dmesg
Device Driver (Kernel Module)
Write Makefile

obj-m += test-driver.o
KDIR = /lib/modules/$(shell uname -r)/build
all:
make -C $(KDIR) M=$(shell pwd) modules
clean:
make -C $(KDIR) M=$(shell pwd) clean
Kernel Module
Device Driver (Kernel Module)
List the Installed modules
Lsmod
Install module
sudo insmod test-driver.ko
Check the installation
Lsmod | grep test

Display the kernel message

dmesg

Remove kernel module

Sudo rmmod test-driver
Device Driver (Kernel Module)
Memory Mapping
Device Driver (Kernel Module)
Peripherals Base address: FC00_0000
ioremap() function is used to map the physical addres of an I/O device
to the kernel virtual address.Kernel creates a page table i.e mapping of
virtual address to the physical address requested.When we do
iounmap() this mapping is destroyed.

void *ioremap(unsigned long phys_addr, unsigned long size);

void iounmap(void * addr);
Device Driver (Kernel Module)
create a proc entry which enables reading access, Example:
struct proc_dir_entry *proc_file_entry;
static const struct file_operations proc_file_fops = {
.owner = THIS_MODULE,
.open = open_callback,
.read = read_callback,
};
int __init init_module(void){
proc_file_entry = proc_create("proc_file_name", 0, NULL,
&proc_file_fops);
if(proc_file_entry == NULL)
return -ENOMEM;
return 0;
}
Device Driver (Kernel Module)
• The /proc File System
In Linux, there is an additional mechanism for the kernel and kernel
modules to send information to processes --- the /proc file system.
Originally designed to allow easy access to information about
processes (hence the name), it is now used by every bit of the kernel
which has something interesting to report, such as /proc/modules
which provides the list of modules and /proc/meminfo which stats
memory usage statistics.
Device Driver (Kernel Module)
• Access to the address space of the process
A driver for a device is the interface between an application and hardware.
As a result, we often have to access user-space data. Accessing it can not
be done directly (by de-referencing a user-space pointer). Direct access of a
user-space pointer can lead to incorrect behaviour
#include <asm/uaccess.h>
put_user(type val, type *address);
get_user(type val, type *address);
unsigned long copy_to_user(void __user *to, const void *from, unsigned long n);
unsigned long copy_from_user(void *to, const void __user *from, unsigned long n);

put_user store the value val to user-space address address; Type can be one on 8, 16, 32, 64 bit
(the maximum supported type depends on the hardware platform);
get_user analogue to the previous function, only that val will be set to a value identical to the value
at the user-space address given by address;
copy_to_user copies n bytes from the kernel-space, from the address referenced by from in user-
space to the address referenced by to;
copy_from_user copies n bytes from user-space from the address referenced by from in kernel-space
to the address referenced by to.
 Device Driver (Kernel Module)
Set GPIO
Device Driver (Kernel Module)
To Find the GIPO Address: 0xFE200000
cat /proc/iomem
Device Driver (Kernel Module)
Next, install the kernel modules onto the SD card
Device Driver (Kernel Module)
Next, install the kernel modules onto the SD card