Slides
Uploaded by
neeru29Slides
Uploaded by
neeru29Free Electrons
Linux
kernel and driver
development
training
Thomas Petazzoni
Michael Opdenacker
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux kernel
Linux device drivers
Free Electrons
Board support code Our services
Mainstreaming kernel code
Kernel debugging
Custom Development
System integration
Embedded Linux Training
Embedded Linux demos and prototypes
All materials released with a free license! System optimization
Unix and GNU/Linux basics Application and interface development
Linux kernel and drivers development
Realtime Linux, uClinux Consulting and technical support
Development and profiling tools Help in decision making
Lightweight tools for embedded systems System architecture
Root filesystem creation System design and performance review
Audio and multimedia Development tool and application support
System optimization Investigating issues and fixing tool bugs
Rights to copy
© Copyright 20042010, Free Electrons
feedback@freeelectrons.com
Electronic version of this document available on
http://freeelectrons.com/doc/training/linuxkernel
Document updates will be available on
http://freeelectrons.com/doc/training/linuxkernel
Attribution – ShareAlike 3.0 Corrections, suggestions,
You are free
contributions and translations are welcome!
to copy, distribute, display, and perform the work
to make derivative works Latest update: Mar 5, 2010
to make commercial use of the work
Under the following conditions
Attribution. You must give the original author credit.
Share Alike. If you alter, transform, or build upon this work, you
may distribute the resulting work only under a license identical to
this one.
For any reuse or distribution, you must make clear to others the license
terms of this work.
Any of these conditions can be waived if you get permission from the
copyright holder.
Your fair use and other rights are in no way affected by the above.
License text: http://creativecommons.org/licenses/bysa/3.0/legalcode
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Hardware used in this training session
Calao Systems USBA9263 AT91SAM9263 ARM CPU
64 MB RAM
256 MB flash
2 USB 2.0 host
1 USB device
100 Mbit Ethernet port
Powered by USB!
Serial and JTAG through
this USB port.
Multiple extension boards.
162 EUR
Supported in mainstream Linux since version 2.6.27!
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Hyperlinks in this document
Links to external sites Usable in the PDF and ODP formats
Example: http://kernel.org/ Try them on this page!
Kernel source files
Our links let you view them in your browser.
Example: kernel/sched.c
Kernel source code:
Identifiers: functions, macros, type definitions...
You get access to their definition, implementation and where they are
used. This invites you to explore the source by yourself!
click wait_queue_head_t queue;
init_waitqueue_head(&queue);
Table of contents
Directly jump to the corresponding sections.
Example: Kernel configuration
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Cooperate!
As in the Free Software and Open Source community,
cooperation during practical labs is valuable in this training session:
Don't hesitate to share your questions with other participants.
They can make some points easier to understand, by using their own
words.
Explaining what you understood to other participants
also helps to consolidate your knowledge.
If you complete your labs before other people, don't hesitate to help
other people and investigate the issues they face. The faster we
progress as a group, the more time we have to explore extra topics.
Don't hesitate to report potential bugs to your instructor.
Don't hesitate to look for solutions on the Internet as well.
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Free Electrons
Linux kernel
introduction
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20042009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelintro
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Kernel overview
Linux features
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux kernel in the system
User app B
Library A User app A Userspace
C library
Call to services Event notification,
information exposition
Linux Kernel
Manage
hardware Event notification
Hardware
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
History
The Linux kernel is one component of a system, which also
requires libraries and applications to provide features to end
users.
The Linux kernel was created as a hobby in 1991
by a Finnish student, Linus Torvalds.
Linux quickly started to be used as the kernel for free software
operating systems
Linus Torvalds has been able to create a large and dynamic
developer and user community around Linux.
Nowadays, hundreds of people contribute to each kernel release,
individuals or companies big and small.
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux license
The whole Linux sources are Free Software released
under the GNU General Public License version 2 (GPL v2).
For the Linux kernel, this basically implies that:
When you receive or buy a device with Linux on it,
you should receive the Linux sources, with the right to
study, modify and redistribute them.
When you produce Linux based devices, you must
release the sources to the recipient, with the same rights,
with no restriction.
See our http://freeelectrons.com/articles/freesw/ training for
exact details about Free Software and its licenses.
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux kernel key features
Portability and hardware Security
support It can't hide its flaws. Its code
Runs on most architectures. is reviewed by many experts.
Scalability Stability and reliability.
Can run on super computers
Modularity
as well as on tiny devices
Can include only what a
(4 MB of RAM is enough).
system needs even at run
Compliance to standards and time.
interoperability.
Easy to program
Exhaustive networking You can learn from existing
support. code. Many useful resources
on the net.
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Supported hardware architectures
2.6.31 status
See the arch/ directory in the kernel sources
Minimum: 32 bit processors, with or without MMU, and gcc support
32 bit architectures (arch/ subdirectories)
arm, avr32, blackfin, cris, frv, h8300, m32r, m68k,
m68knommu, microblaze, mips, mn10300, parisc, s390,
sparc, um, xtensa
64 bit architectures:
alpha, ia64, sparc64
32/64 bit architectures
powerpc, x86, sh
Find details in kernel sources: arch/<arch>/Kconfig,
arch/<arch>/README, or Documentation/<arch>/
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
System calls
The main interface between the kernel and userspace is the set
of system calls
About ~300 system calls that provides the main kernel services
File and device operations, networking operations, interprocess
communication, process management, memory mapping, timers,
threads, synchronization primitives, etc.
This interface is stable over time: only new system calls can be
added by the kernel developers
This system call interface is wrapped by the C library, and
userspace applications usually never make a system call directly
but rather use the corresponding C library function
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Virtual filesystems
Linux makes system and kernel information available in
userspace through virtual filesystems (virtual files not
existing on any real storage). No need to know kernel
programming to access such information!
Mounting /proc:
sudo mount t proc none /proc
Mounting /sys:
sudo mount t sysfs none /sys
Filesystem type Raw device Mount point
or filesystem image
In the case of virtual
filesystems, any string is fine
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
/proc details
A few examples:
/proc/cpuinfo: processor information
/proc/meminfo: memory status
/proc/version: kernel version and build information
/proc/cmdline: kernel command line
/proc/<pid>/environ: calling environment
/proc/<pid>/cmdline: process command line
... and many more! See by yourself!
Lots of details about the /proc interface are available in
Documentation/filesystems/proc.txt
(almost 2000 lines) in the kernel sources.
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Kernel overview
Linux versioning scheme and development process
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Until 2.6 (1)
One stable major branch every 2 or 3 years
Identified by an even middle number
Examples: 1.0, 2.0, 2.2, 2.4
One development branch to integrate new functionalities and
major changes
Identified by an odd middle number
Examples: 2.1, 2.3, 2.5
After some time, a development version becomes the new
base version for the stable branch
Minor releases once in while: 2.2.23, 2.5.12, etc.
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Until 2.6 (2)
Stable version
2.4.0 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.4.6 2.4.7 2.4.8
2.5.0 2.5.1 2.5.2 2.5.3 2.5.4 2.6.0 2.6.1
Development Stable
Note: in reality, many more minor
versions exist inside the stable and
development branches
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Changes since Linux 2.6 (1)
Since 2.6.0, kernel developers have been able to
introduce lots of new features one by one on a steady pace,
without having to make major changes in existing
subsystems.
Opening a new Linux 2.7 (or 2.9) development branch will
be required only when Linux 2.6 is no longer able to
accommodate key features without undergoing traumatic
changes.
Thanks to this, more features are released to users at a
faster pace.
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Changes since Linux 2.6 (2)
Since 2.6.14, the kernel developers agreed
on the following development model:
After the release of a 2.6.x version, a twoweeks merge window
opens, during which major additions are merged.
The merge window is closed
by the release of test version 2.6.(x+1)rc1
The bug fixing period opens, for 6 to 10 weeks.
At regular intervals during the bug fixing period,
2.6.(x+1)rcY test versions are released.
When considered sufficiently stable,
kernel 2.6.(x+1) is released, and the process starts again.
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Merge and bug fixing windows
2 weeks 6 to 10 weeks
Merge window Bug fixing period
2.6.21 2.6.22rc1 2.6.22rc3 2.6.22rc5
2.6.22rc2 2.6.22rc4 2.6.22
2.6.21.1 2.6.21.2 2.6.21.3 2.6.21.4 2.6.21.5
2.6.22.1
Bug fix updates
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
More stability for the 2.6 kernel tree
Issue: bug and security fixes only released for last (or last
two) stable kernel versions (like 2.6.16 and 2.6.17), and of
course by distributions for the exact version that you're
using.
Some people need to have a recent kernel, but with long
term support for security updates.
You could get long term support from a commercial
embedded Linux provider.
You could reuse sources for the kernel used in Ubuntu Long
Term Support releases (5 years of free security updates).
You could choose Linux 2.6.27 for your project, which will be
maintained by kernel.org for a long time, unlike other
versions.
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
What's new in each Linux release?
??!
commit 3c92c2ba33cd7d666c5f83cc32aa590e794e91b0
Author: Andi Kleen <[email protected]>
Date: Tue Oct 11 01:28:33 2005 +0200
[PATCH] i386: Don't discard upper 32bits of HWCR on K8
Need to use long long, not long when RMWing a MSR. I think
it's harmless right now, but still should be better fixed
if AMD adds any bits in the upper 32bit of HWCR.
Bug was introduced with the TLB flush filter fix for i386
Signedoffby: Andi Kleen <[email protected]>
Signedoffby: Linus Torvalds <[email protected]>
...
The official list of changes for each Linux release is just a
huge list of individual patches!
Very difficult to find out the key changes and to get the
global picture out of individual changes.
Fortunately, a summary of key changes
with enough details is available on
http://wiki.kernelnewbies.org/LinuxChanges
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Kernel
source code
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20042010, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelsources/
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Linux code and device drivers
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Supported kernel version
The APIs covered in these training slides
should be compliant with Linux 2.6.31.
We may also mention features in more recent kernels.
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Programming language
Implemented in C like all Unix systems.
(C was created to implement the first Unix systems)
A little Assembly is used too:
CPU and machine initialization, exceptions
Critical library routines.
No C++ used, see http://www.tux.org/lkml/#s153
All the code compiled with gcc, the GNU C Compiler
Many gccspecific extensions used in the kernel code, any ANSI C
compiler will not compile the kernel
A few alternate compilers are supported (Intel and Marvell)
See http://gcc.gnu.org/onlinedocs/gcc4.3.3/gcc/CExtensions.html
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
No C library
The kernel has to be standalone and can't use userspace
code.
Userspace is implemented on top of kernel services, not the
opposite.
Kernel code has to supply its own library implementations
(string utilities, cryptography, uncompression ...)
So, you can't use standard C library functions in kernel code.
(printf(), memset(), malloc()...).
You can also use kernel C headers.
Fortunately, the kernel provides similar C functions for your
convenience, like printk(), memset(), kmalloc() ...
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Portability
The Linux kernel code is designed to be portable
All code outside arch/ should be portable
To this aim, the kernel provides macros and functions to abstract
the architecture specific details
Endianness
(cpu_to_be32, cpu_to_le32, be32_to_cpu, le32_to_cpu)
I/O memory access
Memory barriers to provide ordering guarantees if needed
DMA API to flush and invalidate caches if needed
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
No floating point computation
Never use floating point numbers in kernel code. Your
code may be run on a processor without a floating point
unit (like on arm).
Don't be confused with floating point related configuration
options
They are related to the emulation of floating point
operation performed by the user space applications,
triggering an exception into the kernel
Using softfloat, i.e. emulation in userspace, is
however recommended for performance reasons
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
No stable Linux internal API (1)
The internal kernel API to implement kernel code can undergo changes
between two 2.6.x releases. A standalone driver compiled for a given
version may no longer compile or work on a more recent one.
See Documentation/stable_api_nonsense.txt
in kernel sources for reasons why.
Of course, the external API must not change (system calls, /proc, /sys),
as it could break existing programs. New features can be added, but kernel
developers try to keep backward compatibility with earlier versions, at least
for 1 or several years.
Whenever a developer changes an internal API, (s)he also has to update all
kernel code which uses it. Nothing broken!
Works great for code in the mainline kernel tree.
Difficult to keep in line for out of tree or closedsource drivers!
Kernel API changes reference: http://lwn.net/Articles/2.6kernelapi/
Feature removal schedule: Documentation/featureremovalschedule.txt
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
No stable Linux internal API (2)
USB example
Linux has updated its USB internal API at least 3 times (fixes,
security issues, support for highspeed devices) and has now the
fastest USB bus speeds (compared to other systems)
Windows XP also had to rewrite its USB stack 3 times. But,
because of closedsource, binary drivers that can't be updated, they
had to keep backward compatibility with all earlier implementation.
This is very costly (development, security, stability, performance).
See “Myths, Lies, and Truths about the Linux Kernel”, by Greg K.H.,
for details about the kernel development process:
http://kroah.com/log/linux/ols_2006_keynote.html
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel memory constraints
Who can look after the kernel? User
No memory protection process
SIGSEGV, kill
Accessing illegal memory Attempt
locations result in (often fatal) to access
kernel oopses.
Kernel
Fixed size stack (8 or 4 KB) Illegal Exception
Unlike in userspace, memory (MMU)
no way to make it grow. location
Kernel memory can't be Userspace memory
swapped out (for the same management
Used to implement:
reasons).
memory protection
stack growth
memory swapping to disk
demand paging
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux kernel licensing constraints
The Linux kernel is licensed under the GNU General Public License
version 2
This license gives you the right to use, study, modify and share the
software freely
However, when the software is redistributed, either modified or
unmodified, the GPL requires that you redistribute the software
under the same licence, with the source code
If modifications are made to the Linux kernel (for example to adapt it to
your hardware), it is a derivative work of the kernel, and therefore must
be released under GPLv2
The validity of the GPL on this point has already been verified in courts
However, you're only required to do so
At the time the device starts to be distributed
To your customers, not to the entire world
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Proprietary code and the kernel
It is illegal to distribute a binary kernel that includes statically
compiled proprietary drivers
The kernel modules are a grey area : are they derived works of
the kernel or not ?
The general opinion of the kernel community is that proprietary
drivers are bad
http://www.linuxfoundation.org/en/Kernel_Driver_Statement
From a legal point of view, each driver is probably a different case
Is it really useful to keep your drivers secret ?
There are some examples of proprietary drivers, like the Nvidia
graphics drivers
They use a wrapper between the driver and the kernel
Unclear whether it makes it legal or not
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Advantages of GPL drivers
From the driver developer / decision maker point of view
You don't have to write your driver Users and the community get a
from scratch. You can reuse code positive image of your company.
from similar free software drivers. Makes it easier to hire talented
developers.
You get free community
contributions, support, code review You don't have to supply binary
and testing. Proprietary drivers driver releases for each kernel
(even with sources) don't get any. version and patch version (closed
source drivers).
Your drivers can be freely shipped
by others (mainly by distributions). Drivers have all privileges. You
need the sources to make sure that
Closed source drivers often support
a driver is not a security risk.
a given kernel version. A system
with closed source drivers from 2 Your drivers can be statically
different sources is unmanageable. compiled into the kernel.
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Advantages of intree kernel drivers
Advantages of having your drivers in the mainline kernel sources
Once your sources are accepted in the mainline tree, they are
maintained by people making changes.
Costfree maintenance, security fixes and improvements.
Easy access to your sources by users.
Many more people reviewing your code.
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Userspace device drivers (1)
Possible to implement device drivers in userspace!
Such drivers just need access to the devices through
minimum, generic kernel drivers.
Examples
Printer and scanner drivers
(on top of generic parallel port / USB drivers)
X drivers: low level kernel drivers + user space X drivers.
Userspace drivers based on UIO,
framework introduced in 2.6.23
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Userspace device drivers (2)
Advantages
No need for kernel coding skills. Easier to reuse code between
devices.
Drivers can be written in any language, even Perl!
Drivers can be kept proprietary.
Driver code can be killed and debugged. Cannot crash the kernel.
Can be swapped out (kernel code cannot be).
Can use floatingpoint computation.
Less inkernel complexity.
Drawbacks
Less straightforward to handle interrupts.
Increased latency vs. kernel code.
See UIOHowto in the kernel documentation for detailed informations
and the «Using UIO on an Embedded platform» talk at ELC 2008,
http://www.celinux.org/elc08_presentations/uio080417celfelc08.pdf.
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Linux sources
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Location of kernel sources
The official version of the Linux kernel, as released by Linus
Torvalds is available at http://www.kernel.org
This version follows the welldefined development model of the
kernel
However, it may not contain the latest development from a specific
area, due to the organization of the development model and
because features in development might not be ready for mainline
inclusion
Many kernel subcommunities maintain their own kernel, with
usually newer but less stable features
Architecture communities (ARM, MIPS, PowerPC, etc.), device
drivers communities (I2C, SPI, USB, PCI, network, etc.), other
communities (realtime, etc.)
They generally don't release official versions, only development
trees are available 18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux sources structure (1)
arch/<arch> Architecture specific code
arch/<arch>/<mach> Machine / board specific code
block/ Block layer core
COPYING Linux copying conditions (GNU GPL)
CREDITS Linux main contributors
crypto/ Cryptographic libraries
Documentation/ Kernel documentation. Don't miss it!
drivers/ All device drivers except sound ones (usb, pci...)
fs/ Filesystems (fs/ext3/, etc.)
include/ Kernel headers
include/asm<arch> Architecture and machine dependent headers
include/linux Linux kernel core headers
init/ Linux initialization (including main.c)
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux sources structure (2)
ipc/ Code used for process communication
Kbuild Part of the kernel build system
kernel/ Linux kernel core (very small!)
lib/ Misc library routines (zlib, crc32...)
MAINTAINERS Maintainers of each kernel part. Very useful!
Makefile Top Linux makefile (sets arch and version)
mm/ Memory management code (small too!)
net/ Network support code (not drivers)
README Overview and building instructions
REPORTINGBUGS Bug report instructions
samples/ Sample code (markers, kprobes, kobjects)
scripts/ Scripts for internal or external use
security/ Security model implementations (SELinux...)
sound/ Sound support code and drivers
usr/ Code to generate an initramfs cpio archive.
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing development sources (1)
Useful if you are involved in kernel development
or if you found a bug in the source code.
Kernel development sources are now managed with git:
http://kernel.org/pub/software/scm/git/
You can browse Linus' git tree (if you just need to check a few files):
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux2.6.git;a=tree
You can also directly use git on your workstation
Debian / Ubuntu: install the gitcore package
If you are behind a proxy, set Unix environment variables defining
proxy settings. Example:
export http_proxy="proxy.server.com:8080"
export ftp_proxy="proxy.server.com:8080"
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing development sources (2)
Choose a git development tree on http://git.kernel.org/
Get a local copy (“clone”) of this tree.
Example (Linus tree, the one used for Linux stable releases):
gitclone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux2.6.git
Update your copy whenever needed (Linus tree example):
cd linux2.6
git pull
See our training materials on git:
http://freeelectrons.com/docs/git/
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Kernel source management tools
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Cscope
http://cscope.sourceforge.net/
Tool to browse source code
(mainly C, but also C++ or Java)
Supports huge projects like the Linux kernel
Takes less than 1 min. to index Linux 2.6.17
sources (fast!) Allows searching code for:
all references to a symbol
Can be used from editors like vim and emacs. global definitions
functions called by a function
In Linux kernel sources, run it with: functions calling a function
cscope Rk text string
regular expression pattern
(see man cscope for details) a file
files including a file
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Cscope screenshot
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
LXR: Linux Cross Reference
http://sourceforge.net/projects/lxr Takes a little time and patience to setup
(configuration, indexing, server
Generic source indexing tool
configuration).
and code browser
Indexing a new version is very fast:
Web server based
approximately 20 minutes with LXR 0.3.1
Very easy and fast to use
(old version, but scales very well).
Identifier or text search available
You don't need to set up LXR by yourself.
Very easy to find the declaration, Use our http://lxr.freeelectrons.com
implementation or usages of symbols server! Other servers available on the
Internet:
Supports C and C++ http://freeelectrons.com/community/kernel/lxr/
Supports huge code projects This makes LXR the simplest solution
such as the Linux kernel to browse standard kernel sources.
(264 M in version 2.6.26).
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
LXR screenshot
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Kernel sources
Get familiar with the sources
Use a kernel source indexing tool
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux kernel usage
Embedded Linux
kernel usage
(summary)
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20042009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelusage
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Linux kernel sources
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Getting Linux sources
Full tarballs
Contain the complete kernel sources
Long to download and uncompress, but must be done at least once
Example:
http://kernel.org/pub/linux/kernel/v2.6/linux2.6.14.7.tar.bz2
Incremental patches between versions
It assumes you already have a base version and you apply the
correct patches in the right order
Quick to download and apply
Examples
http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.bz2 (2.6.13 to 2.6.14)
http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.7.bz2 (2.6.14 to 2.6.14.7)
All previous kernel versions are available in
http://kernel.org/pub/linux/kernel/
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using the patch command
The patch command applies changes
to files in the current directory:
You can reverse
Making changes to existing files
a patch
Creating or deleting files and directories with the R
option
patch usage examples:
patch p<n> < diff_file
cat diff_file | patch p<n>
You can test a patch with
bzcat diff_file.bz2 | patch p<n> the dryrun
option
zcat diff_file.gz | patch p<n>
n: number of directory levels to skip in the file paths
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Anatomy of a patch file
A patch file is the output of the diff command
diff Nru a/Makefile b/Makefile diff command line
a/Makefile 20050304 09:27:15 08:00
+++ b/Makefile 20050304 09:27:15 08:00 File date info
@@ 1,7 +1,7 @@ Line numbers in
VERSION = 2 files
PATCHLEVEL = 6 Context info: 3 lines before the change
Useful to apply a patch when line numbers
SUBLEVEL = 11 changed
EXTRAVERSION = Removed line(s) if
+EXTRAVERSION = .1 any
Added line(s) if any
NAME=Woozy Numbat
Context info: 3 lines after the change
# *DOCUMENTATION*
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Applying a Linux patch
Linux patches...
Always to apply to the x.y.<z1> version
Downloadable in gzip You can make patch 30%
and bzip2 (much smaller) compressed files. faster by using sp1
instead of p1
Always produced for n=1 (silent)
(that's what everybody does... do it too!)
Tested on patch2.6.23.bz2
Linux patch command line example:
cd linux2.6.13
bzcat ../patch2.6.14.bz2 | patch p1
bzcat ../patch2.6.14.7.bz2 | patch p1
cd ..; mv linux2.6.13 linux2.6.14.7
Keep patch files compressed: useful to check their signature later.
You can still view (or even edit) the uncompressed data with vim:
vim patch2.6.14.bz2 (on the fly (un)compression)
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Kernel configuration
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel configuration (1)
The kernel contains thousands of device drivers, filesystem
drivers, network protocols and other configurable items
Thousands of options are available, that are used to selectively
compile parts of the kernel source code
The kernel configuration is the process of defining the set of
options with which you want your kernel to be compiled
The set of options depends
On your hardware
On the capabilities you would like to give to your kernel
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel configuration (2)
The configuration is stored in the .config file at the root of kernel
sources
Simple text file, key=value style
As options have dependencies, typically never edited by hand, but
through graphical interfaces :
make [xconfig|gconfig|menuconfig|oldconfig]
These are targets from the main kernel Makefile. Run make help to
get a list of all available targets.
To modify a kernel in a GNU/Linux distribution:
the configuration files are usually released in /boot/, together with
kernel images: /boot/config2.6.1711generic
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
make xconfig
make xconfig
The most common graphical interface
to configure the kernel.
Make sure you read
help > introduction: useful options!
File browser: easier to load configuration files
New search interface to look for parameters
Required Debian / Ubuntu packages:
libqt3mtdev, g++
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
make xconfig screenshot
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
make xconfig search interface
Looks for a keyword
in the description
string
Allows to select
or unselect found
parameters.
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel configuration options
Compiled as a module (separate file)
CONFIG_ISO9660_FS=m
Driver options
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
Compiled statically into the kernel
CONFIG_UDF_FS=y
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Corresponding .config file excerpt
# Section name
# CDROM/DVD Filesystems
# (helps to locate settings in the interface)
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y All parameters are prefixed
CONFIG_UDF_NLS=y with CONFIG_
#
# DOS/FAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel option dependencies
There are dependencies between kernel options
For example, enabling a network driver requires the network
stack to be enabled
Two types of dependencies
depends on dependencies. In this case, option A that depends on
option B is not visible until option B is enabled
select dependencies. In this case, with option A depending on
option B, when option A is enabled, option B is automatically
enabled
make xconfig allows to see all options, even those that cannot be
selected because of missing dependencies. In this case, they are
displayed in gray
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
make menuconfig
make menuconfig
Useful when no graphics
are available. Pretty
convenient too!
Same interface found in
other tools: BusyBox,
buildroot...
Required Debian
packages:
libncursesdev
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
make oldconfig
make oldconfig
Needed very often!
Useful to upgrade a .config file from an earlier kernel
release
Issues warnings for configuration parameters
that no longer exist in the new kernel.
Asks for values for new parameters
If you edit a .config file by hand, it's strongly
recommended to run make oldconfig afterwards!
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and installing the kernel
for the host system
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel compilation
make
in the main kernel source directory
Remember to run make j 4 if you have multiple CPU cores to
speed up the compilation process
No need to run as root !
Generates
vmlinux, the raw uncompressed kernel image, at the ELF format,
useful for debugging purposes, but cannot be booted
arch/<arch>/boot/*Image, the final, usually compressed,
kernel image that can be booted
bzImage for x86, zImage for ARM, vmImage.gz for Blackfin, etc.
All kernel modules, spread over the kernel source tree, as .ko files.
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel installation
make install
Does the installation for the host system by default, so needs to be
run as root
Installs
/boot/vmlinuz<version>
Compressed kernel image. Same as the one in
arch/<arch>/boot
/boot/System.map<version>
Stores kernel symbol addresses
/boot/config<version>
Kernel configuration for this version
Typically reruns the bootloader configuration utility to take into
account the new kernel.
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module installation
make modules_install
Does the installation for the host system by default, so needs to be
run as root
Installs all modules in /lib/modules/<version>/
kernel/
Module .ko (Kernel Object) files, in the same directory
structure as in the sources.
modules.alias
Module aliases for module loading utilities. Example line:
alias soundservice?0 snd_mixer_oss
modules.dep
Module dependencies
modules.symbols
Tells which module a given symbol belongs to.
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel cleanup targets
Cleanup generated files
(to force recompiling drivers):
make clean
Remove all generated files. Needed when switching
from one architecture to another
Caution: also removes your .config file!
make mrproper
Also remove editor backup and patch reject files:
(mainly to generate patches):
make distclean
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Linux device files
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Character device files
Accessed through a sequential flow of individual
characters
Character devices can be identified by their c type
(ls l):
crwrw 1 root uucp 4, 64 Feb 23 2004 /dev/ttyS0
crww 1 jdoe tty 136, 1 Feb 23 2004 /dev/pts/1
crw 1 root root 13, 32 Feb 23 2004 /dev/input/mouse0
crwrwrw 1 root root 1, 3 Feb 23 2004 /dev/null
Example devices: keyboards, mice, parallel port, IrDA,
Bluetooth port, consoles, terminals, sound, video...
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Block device files
Accessed through data blocks of a given size.
Blocks can be accessed in any order.
Block devices can be identified by their b type (ls l):
brwrw 1 root disk 3, 1 Feb 23 2004 hda1
brwrw 1 jdoe floppy 2, 0 Feb 23 2004 fd0
brwrw 1 root disk 7, 0 Feb 23 2004 loop0
brwrw 1 root disk 1, 1 Feb 23 2004 ram1
brw 1 root root 8, 1 Feb 23 2004 sda1
Example devices: hard or floppy disks, ram disks, loop
devices...
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Device major and minor numbers
As you could see in the previous examples,
device files have 2 numbers associated to them:
First number: major number
Second number: minor number
Major and minor numbers are used by the kernel to bind a
driver to the device file. Device file names don't matter to the
kernel!
To find out which driver a device file corresponds to,
or when the device name is too cryptic,
see Documentation/devices.txt.
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Device file creation
Device files are not created when a driver is loaded.
They have to be created in advance:
sudo mknod /dev/<device> [c|b] <major> <minor>
Examples:
sudo mknod /dev/ttyS0 c 4 64
sudo mknod /dev/hda1 b 3 1
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Overall system startup
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Traditional booting sequence
Bootloader
Executed by the hardware at a fixed location in ROM / Flash
Initializes support for the device where the kernel image is found (local storage,
network, removable media)
Loads the kernel image in RAM
Executes the kernel image (with a specified command line)
Kernel
Uncompresses itself
Initializes the kernel core and statically compiled drivers (needed to access the root
filesystem)
Mounts the root filesystem (specified by the root kernel parameter)
Executes the first userspace program (specified by the init kernel parameter)
First userspace program
Configures userspace and starts up system services
29
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel command line parameters
The Linux kernel can be given parameters at boot time
Kernel command line arguments are part of the bootloader
configuration settings.
They are copied to RAM by the bootloader,
to a location where the kernel expects them.
Useful to modify the behavior of the kernel
at boot time, without having to recompile it.
Useful to perform advanced kernel and driver initialization,
without having to use complex userspace scripts.
30
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel command line example
HP iPAQ h2200 PDA booting example:
root=/dev/ram0 \ Root filesystem (first ramdisk)
rw \ Root filesystem mounting mode
init=/linuxrc \ First userspace program
console=ttyS0,115200n8 \ Console (serial)
console=tty0 \ Other console (framebuffer)
ramdisk_size=8192 \ Misc parameters...
cachepolicy=writethrough
Hundreds of command line parameters described on
Documentation/kernelparameters.txt
31
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Drawbacks
Assumption that all device drivers needed to mount the root
filesystem (storage and filesystem drivers) are statically compiled
inside the kernel.
Assumption can be correct for most embedded systems, where
the hardware is known and the kernel can be finetuned for the
system.
Assumption is mostly wrong for desktop and servers, since a
single kernel image should support a wide range of devices and
filesystems
More flexibility was needed
Modules have this flexibility, but they are not available before
mounting the root filesystem
Need to handle complex setups (RAID, NFS, etc.)
32
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Solution
A solution is to include a small temporary root filesystem with
modules, in the kernel itself. This small filesystem is called the
initramfs.
This initramfs is a gzipped cpio archive of this basic root filesystem
A gzipped cpio archive is a kind of zip file, with a much simpler format
The initramfs scripts will detect the hardware, load the
corresponding kernel modules, and mount the real root filesystem.
Finally the initramfs scripts will run the init application in the real root
filesystem and the system can boot as usual.
The initramfs technique completely replaces init ramdisks (initrds).
Initrds were used in Linux 2.4, but are no longer needed.
33
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Booting sequence with initramfs
Bootloader
unchanged
Executed by the hardware at a fixed location in ROM / Flash
Initializes support for the device where the images are found (local storage, network, removable media)
Loads the kernel image in RAM
Executes the kernel image (with a specified command line)
Kernel
Uncompresses itself
Initializes the kernel core and statically compiled drivers
Uncompresses an initramfs cpio archive (if existing, in the kernel image or copied to memory by the
bootloader) and extracts it to the kernel file cache (no mounting, no filesystem).
If found in the initramfs, executes the first userspace program: /init
Userspace: /init script (what follows is just a typical scenario)
Runs userspace commands to configure the device
(such as network setup, mounting /proc and /sys...)
Mounts a new root filesystem. Switch to it ( switch_root)
Runs /sbin/init
Userspace: /sbin/init
Runs commands to configure the device (if not done yet in the initramfs)
Starts up system services (daemons, servers) and user programs
34
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Initramfs features and advantages
Root filesystem directly embedded in the kernel image, or copied
to RAM by the bootloader, simple solution.
Just a plain compressed cpio archive extracted in the file cache.
Neither needs a block nor a filesystem driver.
Simpler to mount complex filesystems from flexible userspace
scripts rather than from rigid kernel code. More complexity
moved out to userspace!
Possible to add non GPL files (firmware, proprietary drivers)
in the filesystem. This is not linking, just file aggregation
(not considered as a derived work by the GPL).
35
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to populate an initramfs
Using CONFIG_INITRAMFS_SOURCE
in kernel configuration (General Setup section)
Give a directory containing a root filesystem.
The kernel Makefile will take care of archiving this directory
and including the archive in the compressed kernel image.
Your kernel will then boot automatically on this initramfs
(provided it contains a /init file).
See Documentation/filesystems/ramfsrootfsinitramfs.txt
and Documentation/earlyuserspace/README in kernel sources.
See also http://www.linuxdevices.com/articles/AT4017834659.html for a nice
overview of initramfs (by Rob Landley).
36
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Summary
For embedded systems, two interesting solutions
No initramfs: all needed drivers are included inside the kernel,
and the final root filesystem is mounted directly
Everything inside the initramfs
37
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Host kernel compiling
Compile a recent Linux kernel for
your GNU/Linux workstation
Update the bootloader configuration
Update the initramfs archive used in
the early boot process.
38
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Root filesystem over NFS
39
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Usefulness of rootfs on NFS
Once networking works, your root filesystem could be a directory
on your GNU/Linux development host, exported by NFS (Network
File System). This is very convenient for system development:
Makes it very easy to update files (driver modules in particular)
on the root filesystem, without rebooting. Much faster than
through the serial port.
Can have a big root filesystem even if you don't have support
for internal or external storage yet.
The root filesystem can be huge. You can even build native
compiler tools and build all the tools you need on the target
itself (better to crosscompile though).
40
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
NFS boot setup (1)
On the host (NFS server)
Install an NFS server (example: Debian, Ubuntu)
sudo aptget install nfskernelserver
Add the exported directory to your /etc/exports file:
/home/rootfs 192.168.0.202(rw,no_root_squash,no_subtree_check)
client address NFS server options
Start or restart your NFS server (example: Debian, Ubuntu)
sudo /etc/init.d/nfskernelserver restart
41
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
NFS boot setup (2)
On the target (NFS client)
Compile your kernel with CONFIG_NFS_FS=y,
CONFIG_IP_PNP=y (configure IP at boot time)
and CONFIG_ROOT_NFS=y
Boot the kernel with the below command line options:
root=/dev/nfs
virtual device
ip=192.168.1.111
local IP address
nfsroot=192.168.1.110:/home/nfsroot
NFS server IP address Directory on the NFS server
42
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux usage
Compiling and booting Linux
Crosscompiling the kernel
43
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Crosscompiling the kernel
When you compile a Linux kernel for another CPU architecture
Much faster than compiling natively, when the target system is
much slower than your GNU/Linux workstation.
Much easier as development tools for your GNU/Linux
workstation are much easier to find.
To make the difference with a native compiler, crosscompiler
executables are prefixed by the name of the target system,
architecture and sometimes library. Examples:
mipslinuxgcc
m68klinuxuclibcgcc
armlinuxgnueabigcc
44
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Specifying crosscompilation
The CPU architecture and crosscompiler prefix are defined through
the ARCH and CROSS_COMPILE variables in the toplevel Makefile.
The Makefile defines CC = $(CROSS_COMPILE)gcc
See comments in Makefile for details
The easiest solution is to modify the Makefile.
Example, ARM platform, crosscompiler: armlinuxgcc
ARCH ?= arm
CROSS_COMPILE ?= armlinux
Other solutions
Pass ARCH and CROSS_COMPILE on the make command line
Define ARCH and CROSS_COMPILE as environment variables
Don't forget to have the values properly set at all steps, otherwise the
kernel configuration and build system gets confused
45
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Configuring the kernel
make xconfig
Same as in native compiling
The set of available options will be different
Don't forget to set the right board / machine type!
46
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Readymade config files
assabet_defconfig integrator_defconfig mainstone_defconfig
badge4_defconfig iq31244_defconfig mx1ads_defconfig
bast_defconfig iq80321_defconfig neponset_defconfig
cerfcube_defconfig iq80331_defconfig netwinder_defconfig
clps7500_defconfig iq80332_defconfig omap_h2_1610_defconfig
ebsa110_defconfig ixdp2400_defconfig omnimeter_defconfig
edb7211_defconfig ixdp2401_defconfig pleb_defconfig
enp2611_defconfig ixdp2800_defconfig pxa255idp_defconfig
ep80219_defconfig ixdp2801_defconfig rpc_defconfig
epxa10db_defconfig ixp4xx_defconfig s3c2410_defconfig
footbridge_defconfig jornada720_defconfig shannon_defconfig
fortunet_defconfig lart_defconfig shark_defconfig
h3600_defconfig lpd7a400_defconfig simpad_defconfig
h7201_defconfig lpd7a404_defconfig smdk2410_defconfig
h7202_defconfig lubbock_defconfig versatile_defconfig
hackkit_defconfig lusl7200_defconfig
arch/arm/configs example
47
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using readymade config files
Default configuration files available for many boards / machines!
Check if one exists in arch/<arch>/configs/ for your target.
Example: if you found an acme_defconfig file, you can run:
make acme_defconfig
Using arch/<arch>/configs/ is a very good good way of
releasing a default configuration file for a group of users or
developers.
Like all make commands, you must
run make <machine>_defconfig
in the toplevel source directory.
48
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Building the kernel
Run
make
Copy
arch/<arch>/boot/zImage
to the target storage
You can customize arch/<arch>/boot/install.sh so
that make install does this automatically for you.
make INSTALL_MOD_PATH=<dir>/ modules_install
and copy <dir>/lib/modules/ to /lib/modules/ on
the target storage.
49
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Lab – Module development environment
Set up a crosscompiling
environment
Crosscompile the kernel for an arm
target platform and boot it through
NFS
50
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux kernel usage
Using kernel modules
51
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Loadable kernel modules
Modules: add a given functionality to the kernel (drivers,
filesystem support, and many others).
Can be loaded and unloaded at any time, only when their
functionality is need.
Modules make it easy to develop drivers without rebooting:
load, test, unload, rebuild, load...
Useful to keep the kernel image size to the minimum (essential
in GNU/Linux distributions for PCs).
Also useful to reduce boot time: you don't spend time
initializing devices and kernel features that you only need later.
Caution: once loaded, have full access to the whole kernel
address space. No particular protection.
52
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module dependencies
Some kernel modules can depend on other modules,
which need to be loaded first.
Example: the usbstorage module depends on the scsi_mod,
libusual and usbcore modules.
Dependencies are described
in /lib/modules/<kernelversion>/modules.dep
This file is generated when you run make modules_install.
You can also update the modules.dep file
by yourself, by running (as root):
depmod a [<version>]
53
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel log
When a new module is loaded,
related information is available in the kernel log.
The kernel keeps its messages in a circular buffer
(so that it doesn't consume more memory with many messages)
Kernel log messages are available through the dmesg command.
(“diagnostic message”)
Kernel log messages are also displayed in the system console
(messages can be filtered by level using
/proc/sys/kernel/printk)
54
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module utilities (1)
modinfo <module_name>
modinfo <module_path>.ko
Gets information about a module: parameters, license,
description and dependencies.
Very useful before deciding to load a module or not.
sudo insmod <module_path>.ko
Tries to load the given module. The full path to the module
object file must be given.
55
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Understanding module loading issues
When loading a module fails,
insmod often doesn't give you enough details!
Details are often available in the kernel log.
Example:
> sudo insmod ./intr_monitor.ko
insmod: error inserting './intr_monitor.ko': 1
Device or resource busy
> dmesg
[17549774.552000] Failed to register handler for
irq channel 2
56
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module utilities (2)
sudo modprobe <module_name>
Most common usage of modprobe: tries to load all the
modules the given module depends on, and then this
module. Lots of other options are available. Modprobe
automatically looks in /lib/modules/<version>/ for the
object file corresponding to the given module name.
lsmod
Displays the list of loaded modules
Compare its output with the contents of
/proc/modules!
57
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module utilities (3)
sudo rmmod <module_name>
Tries to remove the given module.
Will only be allowed if the module is no longer in use
(for example, no more processes opening a device file)
sudo modprobe r <module_name>
Tries to remove the given module and all dependent
modules (which are no longer needed after
the module removal)
58
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Passing parameters to modules
Find available parameters:
modinfo sndintel8x0m
Through insmod:
sudo insmod ./sndintel8x0m.ko index=2
Through modprobe:
Set parameters in /etc/modprobe.conf or in any file in
/etc/modprobe.d/:
options sndintel8x0m index=2
Through the kernel command line,
when the driver is built statically into the kernel:
sndintel8x0m.index=2
driver name
driver parameter name
driver parameter value
59
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful reading
Linux Kernel in a Nutshell, Dec 2006
By Greg KroahHartman, O'Reilly
http://www.kroah.com/lkn/
A good reference book and guide on configuring,
compiling and managing the Linux kernel sources.
Freely available online!
Great companion to the printed book
for easy electronic searches!
Available as single PDF file on
http://freeelectrons.com/community/kernel/lkn/
60
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Embedded Linux
kernel and driver
development
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20042009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernel
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Contents
Driver development
Loadable kernel modules Sleeping, Interrupt management
Memory management Handling concurrency
I/O memory and ports Debugging
Character drivers mmap
Processes and scheduling Device model, sysfs
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Loadable kernel modules
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
hello module
/* hello.c */
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
static int __init hello_init(void)
{
printk(KERN_ALERT "Good morrow"); __init:
printk(KERN_ALERT "to this fair assembly.\n");
return 0; removed after initialization
} (static kernel or module).
static void __exit hello_exit(void)
{ __exit: discarded when
printk(KERN_ALERT "Alas, poor world, what treasure");
printk(KERN_ALERT "hast thou lost!\n"); module compiled statically
}
into the kernel.
module_init(hello_init);
module_exit(hello_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Greeting module");
MODULE_AUTHOR("William Shakespeare");
Example available on http://freeelectrons.com/doc/c/hello.c
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Module license
Several usages
Used to restrict the kernel functions that the module can use if it
isn't a GPLlicensed module
Difference between EXPORT_SYMBOL() and
EXPORT_SYMBOL_GPL()
Used by kernel developers to identify issues coming from
proprietary drivers, which they can't do anything about
(“Tainted” kernel notice in kernel crashes and oopses).
Useful for users to check that their system is 100% free
(check /proc/sys/kernel/tainted)
Values
GPL, GPL v2, GPL and additional rights, Dual MIT/GPL, Dual
BSD/GPL, Dual MPL/GPL, Proprietary
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Compiling a outoftree module
The below Makefile should be reusable for any outoftree
Linux 2.6 module
Intree modules are covered later
Just run make to build the hello.ko file
Caution: make sure there is a [Tab] character at the
beginning of the $(MAKE) line (make syntax) Either
full kernel
ifneq ($(KERNELRELEASE),) source directory
(configured and
objm := hello.o
compiled)
else
[Tab]! or just kernel
KDIR := /path/to/kernel/sources
(no spaces) headers directory
all: (minimum
$(MAKE) C $(KDIR) M=`pwd` modules needed)
endif
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Modules and kernel version
To be compiled, a kernel module needs access to the kernel
headers, containing the functions, types and constants definitions
Two solutions
Full kernel sources
Only kernel headers (linuxheaders* packages in
Debian/Ubuntu distributions)
The sources or headers must be configured
Many macros or functions depend on the configuration
A kernel module compiled against version X of kernel headers
will not load in kernel version Y
modprobe/insmod will say « Invalid module format »
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Symbols exported to modules
From a kernel module,
only a limited number of kernel functions can be called
Functions and variables have to be explicitly exported
by the kernel to be visible from a kernel module
Two macros are used in the kernel
to export functions and variables :
EXPORT_SYMBOL(symbolname), which exports a
function or variable to all modules
EXPORT_SYMBOL_GPL(symbolname), which exports a
function or variable only to GPL modules
A normal driver should not need any nonexported function.
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
hello module with parameters
/* hello_param.c */ Thanks to
#include <linux/init.h>
#include <linux/module.h> Jonathan Corbet
#include <linux/moduleparam.h>
for the example!
MODULE_LICENSE("GPL");
/* A couple of parameters that can be passed in: how many times we say
hello, and to whom */
static char *whom = "world";
module_param(whom, charp, 0);
static int howmany = 1;
module_param(howmany, int, 0);
static int __init hello_init(void)
{
int i;
for (i = 0; i < howmany; i++)
printk(KERN_ALERT "(%d) Hello, %s\n", i, whom);
return 0;
}
static void __exit hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel %s\n", whom);
}
module_init(hello_init);
module_exit(hello_exit);
Example available on http://freeelectrons.com/doc/c/hello_param.c
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Declaring a module parameter
#include <linux/moduleparam.h>
module_param(
name, /* name of an already defined variable */
type, /* either byte, short, ushort, int, uint, long,
ulong, charp, or bool.
(checked at compile time!) */
perm /* for /sys/module/<module_name>/parameters/<param>
0: no such module parameter value file */
);
Example
int irq=5;
module_param(irq, int, S_IRUGO);
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Declaring a module parameter array
#include <linux/moduleparam.h>
module_param_array(
name, /* name of an already defined array */
type, /* same as in module_param */
num, /* number of elements in the array, or NULL (no check?) */
perm /* same as in module_param */
);
Example
static int base[MAX_DEVICES] = { 0x820, 0x840 };
module_param_array(base, int, NULL, 0);
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Adding sources to the kernel tree
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
New driver in kernel sources (1)
To add a new driver to the kernel sources:
Add your new source file to the appropriate source directory.
Example: drivers/usb/serial/navman.c
Describe the configuration interface for your new driver
by adding the following lines to the Kconfig file in this directory:
config USB_SERIAL_NAVMAN
tristate "USB Navman GPS device"
depends on USB_SERIAL
help
To compile this driver as a module, choose M here: the
module will be called navman.
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
New driver in kernel sources (2)
Add a line in the Makefile file based on the Kconfig setting:
obj$(CONFIG_USB_SERIAL_NAVMAN) += navman.o
Run make xconfig and see your new options!
Run make and your new files are compiled!
See Documentation/kbuild/ for details
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to create Linux patches
Download the latest kernel sources
Make a copy of these sources:
cp a linux2.6.9rc2/ linux2.6.9rc2patch/
Apply your changes to the copied sources, and test them.
Run make distclean to keep only source files.
Create a patch file:
diff Nur linux2.6.9rc2/ \
linux2.6.9rc2patch/ > patchfile
Always compare the whole source structures
(suitable for patch p1)
Patch file name: should recall what the patch is about.
If you need to manage a lot of patches, use git or quilt instead
Thanks to Nicolas Rougier (Copyright 2003,
http://webloria.loria.fr/~rougier/) for the Tux image 15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Writing modules
Write a kernel module with several
capabilities, including module
parameters.
Access kernel internals from your
module.
Setup the environment to compile it
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Memory management
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Physical and virtual memory
Physical address space Virtual address spaces
0xFFFFFFFF 0xFFFFFFFFF 0xFFFFFFFF
Kernel
I/O memory 3 0xC0000000
Process1
I/O memory 2 Memory
Management
Unit 0x00000000 0x00000000
I/O memory 1
Flash MMU CPU
0xFFFFFFFF
Kernel
RAM 1 0xC0000000
All the processes have their Process2
RAM 0 own virtual address space, and
run as if they had access to the
whole address space. 0x00000000
0x00000000
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
kmalloc and kfree
Basic allocators, kernel equivalents of glibc's malloc and
free.
#include <linux/slab.h>
static inline void *kmalloc(size_t size, int flags);
size: number of bytes to allocate
flags: priority (explained in a few pages)
void kfree (const void *objp);
Example: (drivers/infiniband/core/cache.c)
struct ib_update_work *work;
work = kmalloc(sizeof *work, GFP_ATOMIC);
...
kfree(work);
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
kmalloc features
Quick (unless it's blocked waiting for memory to be freed).
Doesn't initialize the allocated area.
The allocated area is contiguous in physical RAM.
Allocates by 2n sizes, and uses a few management bytes.
So, don't ask for 1024 when you need 1000! You'd get 2048!
Caution: drivers shouldn't try to kmalloc
more than 128 KB (upper limit in some architectures).
Minimum memory consumption:
32 or 64 bytes (page size dependent).
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Main kmalloc flags
Defined in include/linux/gfp.h (GFP: __get_free_pages)
GFP_KERNEL
Standard kernel memory allocation. May block. Fine for most
needs.
GFP_ATOMIC
RAM allocated from code which is not allowed to block (interrupt
handlers) or which doesn't want to block (critical sections). Never
blocks.
GFP_DMA
Allocates memory in an area of the physical memory usable for
DMA transfers.
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Related allocation functions
Again, names similar to those of C library functions
static inline void *kzalloc(
size_t size, gfp_t flags);
Zeroes the allocated buffer.
static inline void *kcalloc(
size_t n, size_t size, gfp_t flags);
Allocates memory for an array of n elements of size size,
and zeroes its contents.
void * __must_check krealloc(
const void *, size_t, gfp_t);
Changes the size of the given buffer.
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Slab caches and memory pools
Slab caches: make it possible to allocate multiple
objects of the same size, without wasting RAM.
So far, mainly used in core subsystems,
but not much in device drivers
(except USB and SCSI drivers)
Memory pools: pools of preallocated objects,
to increase the chances of allocations to succeed.
Often used with file caches.
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Available allocators
Memory is allocated using slabs (groups of one or more continuous pages
from which objects are allocated). Several compatible slab allocators are
available:
SLAB: original, well proven allocator in Linux 2.6.
SLOB: much simpler. More space efficient but doesn't scale well. Saves
a few hundreds of KB in small systems (depends on
CONFIG_EMBEDDED)
SLUB: the new default allocator since 2.6.23, simpler than SLAB,
scaling much better (in particular for huge systems) and creating less
fragmentation.
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Allocating by pages
More appropriate when you need big slices of RAM:
A page is usually 4K, but can be made greater in some architectures
(sh, mips: 4, 8, 16 or 64K, but not configurable in i386 or arm).
unsigned long get_zeroed_page(int flags);
Returns a pointer to a free page and fills it up with zeros
unsigned long __get_free_page(int flags);
Same, but doesn't initialize the contents
unsigned long __get_free_pages(int flags,
unsigned int order);
Returns a pointer on an area of several contiguous pages in physical RAM.
order: log2(<number_of_pages>)
If variable, can be computed from the size with the get_order function.
Maximum: 8192 KB (MAX_ORDER=11 in include/linux/mmzone.h),
except in a few architectures when overwritten with CONFIG_FORCE_MAX_ZONEORDER.
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Freeing pages
void free_page(unsigned long addr);
void free_pages(unsigned long addr,
unsigned int order);
Need to use the same order as in allocation.
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
vmalloc
vmalloc can be used to obtain contiguous memory zones
in virtual address space (even if pages may not be
contiguous in physical memory).
void *vmalloc(unsigned long size);
void vfree(void *addr);
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Memory utilities
void * memset(void * s, int c, size_t count);
Fills a region of memory with the given value.
void * memcpy(void * dest,
const void *src,
size_t count);
Copies one area of memory to another.
Use memmove with overlapping areas.
Lots of functions equivalent to standard C library ones defined
in include/linux/string.h
and in include/linux/kernel.h (sprintf, etc.)
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel memory debugging
Debugging features available since 2.6.31
Kmemcheck
Dynamic checker for access to uninitialized memory.
Only available on x86 so far, but will help to improve architecture
independent code anyway.
See Documentation/kmemcheck.txt for details.
Kmemleak
Dynamic checker for memory leaks
This feature is available for all architectures.
See Documentation/kmemleak.txt for details.
Both have a significant overhead. Only use them in development!
29
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
I/O memory and ports
30
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Port I/O vs. MemoryMapped I/O
MMIO PIO
Same address bus to address Different address spaces for
memory and I/O devices memory and I/O devices
Access to the I/O devices Uses a special class of CPU
using regular instructions instructions to access I/O
devices
Most widely used I/O method
across the different Example on x86: IN and OUT
architectures supported by instructions
Linux
31
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Requesting I/O ports
/proc/ioports example (x86)
Tells the kernel which driver is using which
0000001f : dma1
00200021 : pic1
I/O ports
00400043 : timer0
00500053 : timer1
0060006f : keyboard
Allows to prevent other drivers from using the
00700077 : rtc
0080008f : dma page reg
same I/O ports, but is purely voluntary.
00a000a1 : pic2
00c000df : dma2 struct resource *request_region(
00f000ff : fpu
0100013f : pcmcia_socket0
unsigned long start,
01700177 : ide1 unsigned long len,
01f001f7 : ide0
03760376 : ide1 char *name);
0378037a : parport0
03c003df : vga+ Tries to reserve the given region and returns
03f603f6 : ide0
03f803ff : serial
NULL if unsuccessful.
0800087f : 0000:00:1f.0 request_region(0x0170, 8, "ide1");
08000803 : PM1a_EVT_BLK
08040805 : PM1a_CNT_BLK
0808080b : PM_TMR void release_region(
08200820 : PM2_CNT_BLK
0828082f : GPE0_BLK
unsigned long start,
... unsigned long len);
32
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing I/O ports
Functions to read/write bytes, word and longs to I/O ports:
unsigned in[bwl](unsigned long *addr);
void out[bwl](unsigned port, unsigned long *addr);
And the strings variants: often more efficient than the corresponding
C loop, if the processor supports such operations!
void ins[bwl](unsigned port, void *addr, unsigned long
count);
void outs[bwl](unsigned port, void *addr, unsigned long
count);
33
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Requesting I/O memory
/proc/iomem example
Functions equivalent to
000000000009efff : System RAM
0009f0000009ffff : reserved
request_region() and
000a0000000bffff : Video RAM area
000c0000000cffff : Video ROM
release_region(), but for I/O memory.
000f0000000fffff : System ROM
001000003ffadfff : System RAM struct resource * request_mem_region(
001000000030afff : Kernel code unsigned long start,
0030b000003b4bff : Kernel data
3ffae0003fffffff : reserved unsigned long len,
40000000400003ff : 0000:00:1f.1 char *name);
4000100040001fff : 0000:02:01.0
4000100040001fff : yenta_socket void release_mem_region(
4000200040002fff : 0000:02:01.1
4000200040002fff : yenta_socket unsigned long start,
40400000407fffff : PCI CardBus #03 unsigned long len);
4080000040bfffff : PCI CardBus #03
40c0000040ffffff : PCI CardBus #07
41000000413fffff : PCI CardBus #07
a0000000a0000fff : pcmcia_socket0
a0001000a0001fff : pcmcia_socket1
e0000000e7ffffff : 0000:00:00.0
e8000000efffffff : PCI Bus #01
e8000000efffffff : 0000:01:00.0
...
34
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Mapping I/O memory in virtual memory
To access I/O memory, drivers need to have a virtual
address that the processor can handle, because I/O
memory is not mapped by default in virtual memory.
The ioremap functions satisfy this need:
#include <asm/io.h>;
void *ioremap(unsigned long phys_addr,
unsigned long size);
void iounmap(void *address);
Caution: check that ioremap doesn't return a NULL
address!
35
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing MMIO devices
Directly reading from or writing to addresses returned by
ioremap (“pointer dereferencing”) may not work on some
architectures.
To do PCIstyle, littleendian accesses, conversion being done
automatically
unsigned read[bwl](void *addr);
void write[bwl](unsigned val, void *addr);
To do raw access, without endianess conversion
unsigned __raw_read[bwl](void *addr);
void __raw_write[bwl](unsigned val, void *addr);
36
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
New API for mixed accesses
A new API allows to write drivers that can work on either devices
accessed over PIO or MMIO. Some drivers use it, but there doesn't
seem to be a consensus in the kernel community around it.
Mapping
For PIO: ioport_map() and ioport_unmap(). They don't really
map, but they return a special cookie.
For MMIO: ioremap() and iounmap(). As usual.
Access, works both on addresses returned by ioport_map() and
ioremap()
ioread[8/16/32]() and iowrite[8/16/32] for single access
ioread_rep[8/16/32]() and iowrite_rep[8/16/32]() for
repeated accesses
37
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Avoiding I/O access issues
Caching on I/O ports or memory already disabled
Use the macros, they do the right thing for your architecture
The compiler and/or CPU can reorder memory accesses, which
might cause troubles for your devices is they expect one register
to be read/written before another one.
Memory barriers are available to prevent this reordering
rmb() is a read memory barrier, prevents reads to cross the
barrier
wmb() is a write memory barrier
mb() is a readwrite memory barrier
38
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
/dev/mem
Used to provide userspace applications
with direct access to physical addresses.
Usage: open /dev/mem and read or write at given offset.
What you read or write is the value
at the corresponding physical address.
Used by applications such as the X server
to write directly to device memory.
Since 2.6.26 (x86 only, 2.6.32 status): only nonRAM can
be accessed for security reasons, unless explicitly
configured otherwise (CONFIG_STRICT_DEVMEM).
39
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – I/O memory and ports
Make a remote connection to your
board through ssh.
Access the system console through
the network.
Reserve the I/O memory addresses
used by the serial port.
Read device registers and write data
to them, to send characters on the
serial port.
40
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Character drivers
41
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Usefulness of character drivers
Except for storage device drivers, most drivers for devices with
input and output flows are implemented as character drivers.
So, most drivers you will face will be character drivers
You will regret if you sleep during this part!
42
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Creating a character driver
Userspace
Userspace needs
The name of a device file in /dev to Read Write
buffer string
interact with the device driver through
regular file operations (open, read, write, read write
close...)
The kernel needs /dev/foo
To know which driver is in charge of device
Copy from user
Copy to user
major / minor
files with a given major / minor number pair
For a given driver, to have handlers (“file
operations”) to execute when userspace Read
handler
Write
handler
opens, reads, writes or closes the device Device driver
file.
Kernel space
43
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Implementing a character driver
Four major steps
Implement operations corresponding to the system calls an
application can apply to a file : file operations
Define a file_operations structure associating function pointers
to their implementation in your driver
Reserve a set of major and minors for your driver
Tell the kernel to associate the reserved major and minor to your
file operations
This is a very common design scheme in the Linux kernel
A common kernel infrastructure defines a set of operations to be
implemented by a driver and functions to register your driver
Your driver only needs to implement this set of welldefined
operations
44
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
File operations
Before registering character devices, you have to define
file_operations (called fops) for the device files.
The file_operations structure is generic to all files handled by the Linux
kernel. It contains many operations that aren't needed for character
drivers.
Here are the most important operations for a character driver. All of
them are optional.
struct file_operations {
[...]
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*open) (struct inode *, struct file *);
int (*release) (struct inode *, struct file *);
[...]
};
45
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
open() and release()
int foo_open (struct inode *i, struct file *f)
Called when userspace opens the device file.
inode is a structure that uniquely represent a file in the system (be
it a regular file, a directory, a symbolic link, a character or block
device)
file is a structure created every time a file is opened. Several file
structures can point to the same inode structure.
Contains informations like the current position, the opening
mode, etc.
Has a void *private_data pointer that one can freely use.
A pointer to the file structure is passed to all other operations
int foo_release(struct inode *i, struct file *f)
Called when userspace closes the file.
46
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
read()
ssize_t foo_read ( struct file *f, __user char *buf,
size_t sz, loff_t *off)
Called when userspace uses the read() system call on the
device.
Must read data from the device, write at most sz bytes in the
userspace buffer buf, and update the current position in the file
off. f is a pointer to the same file structure that was passed in
the open() operation
Must return the number of bytes read.
On Unix, read() operations typically block when there isn't
enough data to read from the device
47
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
write()
ssize_t foo_write( struct file *f,
__user const char *buf,
size_t sz ,loff_t *off)
Called when userspace uses the write() system call on the
device
The opposite of read, must read at most sz bytes from buf,
write it to the device, update off and return the number of
bytes written.
48
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Exchanging data with userspace (1)
Kernel code isn't allowed to directly access
userspace memory, using memcpy or
direct pointer dereferencing
Doing so does not work on some
architectures
If the address passed by the application
was invalid, the application would
segfault
To keep the kernel code portable and have
proper error handling, your driver must use
special kernel functions to exchange data
with userspace
49
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Exchanging data with userspace (2)
A single value
get_user(v, p);
The kernel variable v gets the value pointer by the userspace
pointer p
put_user(v, p);
The value pointed by the userspace pointer p is set to the contents
of the kernel variable v.
A buffer
unsigned long copy_to_user(void __user *to,
const void *from, unsigned long n);
unsigned long copy_from_user(void *to,
const void __user *from, unsigned long n);
The return value must be checked. Zero on success, nonzero on
failure. If nonzero, the convention is to return EFAULT.
50
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
read operation example
static ssize_t
acme_read(struct file *file, char __user *buf, size_t count, loff_t *ppos)
{
/* The acme_buf address corresponds to a device I/O memory area */
/* of size acme_bufsize, obtained with ioremap() */
int remaining_size, transfer_size;
remaining_size = acme_bufsize (int) (*ppos); // bytes left to transfer
if (remaining_size == 0) { /* All read, returning 0 (End Of File) */
return 0;
}
/* Size of this transfer */
transfer_size = min(remaining_size, (int) count);
if (copy_to_user(buf /* to */, acme_buf + *ppos /* from */, transfer_size)) {
return EFAULT;
} else { /* Increase the position in the open file */
*ppos += transfer_size;
return transfer_size;
}
}
Read method Piece of code available in
http://freeelectrons.com/doc/c/acme.c
51
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
write operation example
static ssize_t
acme_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos)
{
int remaining_bytes;
/* Number of bytes not written yet in the device */
remaining_bytes = acme_bufsize (*ppos);
if (count > remaining_bytes) {
/* Can't write beyond the end of the device */
return EIO;
}
if (copy_from_user(acme_buf + *ppos /* to */, buf /* from */, count)) {
return EFAULT;
} else {
/* Increase the position in the open file */
*ppos += count;
return count;
}
}
Write method Piece of code available in
http://freeelectrons.com/doc/c/acme.c
52
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
ioctl()
int ioctl(struct inode *i, struct file *f,
unsigned int cmd, unsigned long arg)
Associated to the ioctl() system call
Allows to extend the driver capabilities beyond the limited read/write
API
For example: changing the speed of a serial port, setting video
output format, querying a device serial number...
cmd is a number identifying the operation to perform
arg is the optional argument passed as third argument of the
ioctl() system call. Can be an integer, an address, etc.
The semantic of cmd and arg is driverspecific.
53
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
ioctl() example : kernel side
static long phantom_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
struct phm_reg r;
void __user *argp = (void __user *)arg;
switch (cmd) {
case PHN_SET_REG:
if (copy_from_user(&r, argp, sizeof(r)))
return EFAULT;
/* Do something */
break;
case PHN_GET_REG:
if (copy_from_user(&r, argp, sizeof(r)))
return EFAULT;
/* Do something */
break;
default:
return ENOTTY;
}
return 0;
}
54
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Ioctl() example : application side
int main(void)
{
int fd, ret;
struct phm_reg reg;
fd = open(“/dev/phantom”);
assert(fd > 0);
reg.field1 = 42;
reg.field2 = 67;
ret = ioctl(fd, PHN_SET_REG, & reg);
assert(ret == 0);
return 0;
}
55
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
file operations definition example (3)
Defining a file_operations structure:
#include <linux/fs.h>
static struct file_operations acme_fops =
{
.owner = THIS_MODULE,
.read = acme_read,
.write = acme_write,
};
You just need to supply the functions you implemented! Defaults for
other functions (such as open, release...) are fine if you do not
implement anything special.
56
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Information on registered devices
Registered devices are visible in /proc/devices:
Character devices: Block devices:
1 mem 1 ramdisk
4 /dev/vc/0 3 ide0
4 tty 8 sd
4 ttyS 9 md
5 /dev/tty 22 ide1
5 /dev/console 65 sd
5 /dev/ptmx 66 sd
6 lp 67 sd
10 misc 68 sd
13 input
14 sound
...
Major Registered
number name
57
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
dev_t data type
Kernel data type to represent a major / minor number pair
Also called a device number.
Defined in <linux/kdev_t.h>
Linux 2.6: 32 bit size (major: 12 bits, minor: 20 bits)
Macro to compose the device number:
MKDEV(int major, int minor);
Macro to extract the minor and major numbers:
MAJOR(dev_t dev);
MINOR(dev_t dev);
58
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering device numbers (1)
#include <linux/fs.h>
int register_chrdev_region(
dev_t from, /* Starting device number */
unsigned count, /* Number of device numbers */
const char *name); /* Registered name */
Returns 0 if the allocation was successful.
Example
static dev_t acme_dev = MKDEV(202, 128);
if (register_chrdev_region(acme_dev, acme_count, “acme”)) {
printk(KERN_ERR “Failed to allocate device number\n”);
...
59
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering device numbers (2)
If you don't have fixed device numbers assigned to your driver
Better not to choose arbitrary ones.
There could be conflicts with other drivers.
The kernel API offers a alloc_chrdev_region function
to have the kernel allocate free ones for you. You can find the
allocated major number in /proc/devices.
60
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Character device registration (1)
The kernel represents character drivers with a cdev structure
Declare this structure globally (within your module):
#include <linux/cdev.h>
static struct cdev acme_cdev;
In the init function, initialize the structure:
cdev_init(&acme_cdev, &acme_fops);
61
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Character device registration (2)
Then, now that your structure is ready, add it to the system:
int cdev_add(
struct cdev *p, /* Character device structure */
dev_t dev, /* Starting device major / minor number
*/
unsigned count); /* Number of devices */
Example (continued):
if (cdev_add(&acme_cdev, acme_dev, acme_count)) {
printk (KERN_ERR “Char driver registration failed\n”);
...
62
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Character device unregistration
First delete your character device:
void cdev_del(struct cdev *p);
Then, and only then, free the device number:
void unregister_chrdev_region(dev_t from,
unsigned count);
Example (continued):
cdev_del(&acme_cdev);
unregister_chrdev_region(acme_dev, acme_count);
63
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux error codes
The kernel convention for error management is
Return 0 on success
return 0;
Return a negative error code on failure
return EFAULT;
Error codes
include/asmgeneric/errnobase.h
include/asm/errno.h
64
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Char driver example summary (1)
static void *acme_buf;
static int acme_bufsize=8192;
static int acme_count=1;
static dev_t acme_dev = MKDEV(202,128);
static struct cdev acme_cdev;
static ssize_t acme_write(...) {...}
static ssize_t acme_read(...) {...}
static struct file_operations acme_fops =
{
.owner = THIS_MODULE,
.read = acme_read,
.write = acme_write
};
65
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Char driver example summary (2)
Shows how to handle errors and deallocate resources in the right order!
static int __init acme_init(void)
{
int err;
acme_buf = ioremap (ACME_PHYS, return 0;
acme_bufsize);
err_dev_unregister:
if (!acme_buf) { unregister_chrdev_region(
err = ENOMEM; acme_dev, acme_count);
goto err_exit; err_free_buf:
} iounmap(acme_buf);
err_exit:
if (register_chrdev_region(acme_dev, return err;
acme_count, “acme”)) { }
err=ENODEV;
goto err_free_buf; static void __exit acme_exit(void)
} {
cdev_del(&acme_cdev);
cdev_init(&acme_cdev, &acme_fops); unregister_chrdev_region(acme_dev,
acme_count);
if (cdev_add(&acme_cdev, acme_dev, iounmap(acme_buf);
acme_count)) { }
err=ENODEV;
goto err_dev_unregister;
}
Complete example code available on http://freeelectrons.com/doc/c/acme.c
66
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Character driver summary
Character driver writer
Kernel
Define the file operations callbacks for the device file: read, write, ioctl...
In the module init function, reserve major and minor numbers with
register_chrdev_region(), init a cdev structure with your file operations and add it to
the system with cdev_add().
In the module exit function, call cdev_del() and unregister_chrdev_region()
System administration
Userspace
Load the character driver module
Create device files with matching major and minor numbers if needed
The device file is ready to use!
System user
Open the device file, read, write, or send ioctl's to it.
Kernel
Kernel
Executes the corresponding file operations
67
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Character drivers
Writing a simple character driver, to
write data to the serial port.
On your workstation, checking that
transmitted data is received correctly.
Exchanging data between userspace
and kernel space.
Practicing with the character device
driver API.
Using kernel standard error codes.
68
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux Driver Development
Driver development
Processes and scheduling
69
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Process, thread ?
Confusion about the terms «process», «thread» and «task»
In Unix, a process is created using fork() and is composed of
An address space, which contains the program code, data, stack,
shared libraries, etc.
One thread, that starts executing the main() function.
Upon creation, a process contains one thread
Additional threads can be created inside an existing process,
using pthread_create()
They run in the same address space as the initial thread of the
process
They start executing a function passed as argument to
pthread_create()
70
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Process, thread: kernel point of view
The kernel represents each thread running in the system by a
structure of type task_struct
From a scheduling point of view, it makes no difference between
the initial thread of a process and all additional threads created
dynamically using pthread_create()
Thread Thread Thread
A A B
Address space Address space
Process after fork() Same process after pthread_create()
71
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
A thread life
EXIT_ZOMBIE
Thread created Task terminated but its
by fork() or The thread is elected resources are not freed yet.
pthread_create()
by the scheduler Waiting for its parent
to acknowledge its death.
TASK_RUNNING TASK_RUNNING
Ready but Actually running
not running The thread is preempted
by the scheduler to run
a higher priority task
Decides to sleep
The event occurs on a wait queue
TASK_INTERRUPTIBLE
or the process receives TASK_UNINTERRUPTIBLE for a specific event
a signal. Thread becomes or TASK_KILLABLE
runnable again Waiting
72
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Execution of system calls
The execution of system calls takes place in the
context of the thread requesting them.
Process continuing in user space...
Process executing in user space...
(or replaced by a higher priority process)
(can be preempted)
(can be preempted)
System call
or exception
Kernel code executed
Still has access to process
on behalf of user space
data (open files...)
(can be preempted too!)
73
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Sleeping
74
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Sleeping
Sleeping is needed when a process (user space or kernel space)
is waiting for data.
User space process... Other ...User space
processes
read device file are return
scheduled
System call... ... System call
ask for sleep wake up
data
Interrupt
handler
data ready notification
75
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to sleep (1)
Must declare a wait queue
A wait queue will be used to store the list of threads waiting
for an event.
Static queue declaration
useful to declare as a global variable
DECLARE_WAIT_QUEUE_HEAD (module_queue);
Or dynamic queue declaration
useful to embed the wait queue inside another data
structure
wait_queue_head_t queue;
init_waitqueue_head(&queue);
76
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to sleep (2)
Several ways to make a kernel process sleep
wait_event(queue, condition);
Sleeps until the task is woken up and the given C expression is true.
Caution: can't be interrupted (can't kill the userspace process!)
int wait_event_killable(queue, condition); (Since Linux 2.6.25)
Can be interrupted, but only by a “fatal” signal (SIGKILL). Returns
ERESTARSYS if interrupted.
int wait_event_interruptible(queue, condition);
Can be interrupted by any signal. Returns ERESTARSYS if interrupted.
int wait_event_timeout(queue, condition, timeout);
Also stops sleeping when the task is woken up and the timeout expired. Returns 0
if the timeout elapsed, nonzero if the condition was met.
int wait_event_interruptible_timeout(queue, condition,
timeout);
Same as above, interruptible. Returns 0 if the timeout elapsed, ERESTARTSYS
if interrupted, positive value if the condition was met
77
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to sleep Example
ret = wait_event_interruptible
(sonypi_device.fifo_proc_list,
kfifo_len(sonypi_device.fifo) != 0);
if (ret)
return ret;
78
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Waking up!
Typically done by interrupt handlers when data sleeping
processes are waiting for are available.
wake_up(&queue);
Wakes up one process in the given wait queue
wake_up_all(& queue);
Wakes up all processes in the given wait queue. Not
recommended.
wake_up_interruptible(&queue);
Wakes up one process waiting in an interruptible sleep
on the given queue
79
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Sleeping and waking up implementation
The scheduler doesn't keep evaluating the sleeping condition!
wait_event_interruptible(&queue, condition);
The process is put in the TASK_INTERRUPTIBLE state.
wake_up_interruptible(&queue);
For all processes waiting in queue, condition is evaluated.
When it evaluates to true, the process is put back
to the TASK_RUNNING state, and the need_resched flag for the
current process is set.
This way, several processes can be woken up at the same time.
80
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Interrupt management
81
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Interrupt handler constraints
Not run from a user context:
Can't transfer data to and from user space
(need to be done by system call handlers)
Interrupt handler execution is managed by the CPU, not
by the scheduler. Handlers can't run actions that may
sleep, because there is nothing to resume their
execution. In particular, need to allocate memory with
GFP_ATOMIC.
Have to complete their job quickly enough:
they shouldn't block their interrupt line for too long.
82
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering an interrupt handler (1)
Defined in include/linux/interrupt.h
int request_irq( Returns 0 if successful
unsigned int irq, Requested irq channel
irq_handler_t handler, Interrupt handler
unsigned long irq_flags, Option mask (see next page)
const char * devname, Registered name
void *dev_id); Pointer to some handler data
Cannot be NULL and must be unique for shared irqs!
void free_irq( unsigned int irq, void *dev_id);
dev_id cannot be NULL and must be unique for shared irqs.
Otherwise, on a shared interrupt line,
free_irq wouldn't know which handler to free.
83
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering an interrupt handler (2)
irq_flags bit values (can be combined, none is fine too)
IRQF_DISABLED
"Quick" interrupt handler. Run with all interrupts disabled on the current cpu
(instead of just the current line). For latency reasons, should only be used
when needed!
IRQF_SHARED
Run with interrupts disabled only on the current irq line and on the local cpu.
The interrupt channel can be shared by several devices. Requires a
hardware status register telling whether an IRQ was raised or not.
84
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Information on installed handlers
/proc/interrupts
CPU0
0: 5616905 XTPIC timer # Registered name
1: 9828 XTPIC i8042
2: 0 XTPIC cascade
3: 1014243 XTPIC orinoco_cs
7: 184 XTPIC Intel 82801DBICH4
8: 1 XTPIC rtc
9: 2 XTPIC acpi
11: 566583 XTPIC ehci_hcd, uhci_hcd,
uhci_hcd, uhci_hcd, yenta, yenta, radeon@PCI:1:0:0
12: 5466 XTPIC i8042
14: 121043 XTPIC ide0
15: 200888 XTPIC ide1
NMI: 0 Non Maskable Interrupts
ERR: 0 Spurious interrupt count
85
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Interrupt handler prototype
irqreturn_t foo_interrupt
(int irq, void *dev_id)
Arguments
irq, the IRQ number
dev_id, the opaque pointer passed at request_irq()
Return value
IRQ_HANDLED: recognized and handled interrupt
IRQ_NONE: not on a device managed by the module. Useful to
share interrupt channels and/or report spurious interrupts to
the kernel.
86
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
The interrupt handler's job
Acknowledge the interrupt to the device
(otherwise no more interrupts will be generated)
Read/write data from/to the device
Wake up any waiting process waiting for the completion
of this read/write operation:
wake_up_interruptible(&module_queue);
87
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Top half and bottom half processing (1)
Splitting the execution of interrupt handlers in 2 parts
Top half: the interrupt handler must complete as quickly
as possible. Once it acknowledged the interrupt, it just
schedules the lengthy rest of the job taking care of the
data, for a later execution.
Bottom half: completing the rest of the interrupt handler
job. Handles data, and then wakes up any waiting user
process.
Best implemented by tasklets (also called soft irqs).
88
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Top half and bottom half processing (2)
Declare the tasklet in the module source file:
DECLARE_TASKLET (module_tasklet, /* name */
module_do_tasklet, /* function */
data /* params */
);
Schedule the tasklet in the top half part (interrupt handler):
tasklet_schedule(&module_tasklet);
Note that a tasklet_hi_schedule function is available to
define high priority tasklets to run before ordinary ones.
By default, tasklets are executed right after all top halves
(hard irqs)
89
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Interrupt management fun
In a training lab, somebody forgot to unregister a handler on
a shared interrupt line in the module exit function.
? Why did his kernel oops with a segmentation fault
at module unload?
Answer...
In a training lab, somebody freed the timer interrupt handler
by mistake (using the wrong irq number). The system froze.
Remember the kernel is not protected against itself!
90
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Interrupt management summary
Device driver Tasklet
When the device file is first Process the data
open, register an interrupt
Wake up processes waiting
handler for the device's
for the data
interrupt channel.
Device driver
Interrupt handler
When the device is no longer
Called when an interrupt is
opened by any process,
raised.
unregister the interrupt
Acknowledge the interrupt handler.
If needed, schedule a tasklet
taking care of handling data.
Otherwise, wake up processes
waiting for the data.
91
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Interrupts
Adding read capability to the
character driver developed earlier.
Register an interrupt handler.
Waiting for data to be available in the
read file operation.
Waking up the code when data is
available from the device.
92
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Concurrent access to resources
93
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Sources of concurrency issues
The same resources can be accessed by several kernel
processes in parallel, causing potential concurrency issues
Several userspace programs accessing the same device data
or hardware. Several kernel processes could execute the same
code on behalf of user processes running in parallel.
Multiprocessing: the same driver code can be running on
another processor. This can also happen with single CPUs with
hyperthreading.
Kernel preemption, interrupts: kernel code can be interrupted at
any time (just a few exceptions), and the same data may be
access by another process before the execution continues.
94
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Avoiding concurrency issues
Avoid using global variables and shared data whenever
possible
(cannot be done with hardware resources).
Use techniques to manage concurrent access to
resources.
See Rusty Russell's Unreliable Guide To Locking
Documentation/DocBook/kernellocking/
in the kernel sources.
95
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Concurrency protection with locks
Process 1 Process 2
Failed
Acquire lock Wait lock release
Try again
Success Critical code section Success
Shared resource
Release lock
96
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux mutexes
The main locking primitive since Linux 2.6.16.
Better than counting semaphores when binary ones are
enough.
Mutex definition:
#include <linux/mutex.h>
Initializing a mutex statically:
DEFINE_MUTEX(name);
Or initializing a mutex dynamically:
void mutex_init(struct mutex *lock);
97
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
locking and unlocking mutexes
void mutex_lock (struct mutex *lock);
Tries to lock the mutex, sleeps otherwise.
Caution: can't be interrupted, resulting in processes you cannot kill!
int mutex_lock_killable (struct mutex *lock);
Same, but can be interrupted by a fatal (SIGKILL) signal. If interrupted, returns
a non zero value and doesn't hold the lock. Test the return value!!!
int mutex_lock_interruptible (struct mutex *lock);
Same, but can be interrupted by any signal.
int mutex_trylock (struct mutex *lock);
Never waits. Returns a non zero value if the mutex is not available.
int mutex_is_locked(struct mutex *lock);
Just tells whether the mutex is locked or not.
void mutex_unlock (struct mutex *lock);
Releases the lock. Do it as soon as you leave the critical section.
98
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Spinlocks
Locks to be used for code that is not allowed to sleep
(interrupt handlers), or that doesn't want to sleep (critical
sections). Be very careful not to call functions which can
sleep!
Originally intended for multiprocessor systems Still locked?
Spinlocks never sleep and keep spinning Spinlock
in a loop until the lock is available.
Spinlocks cause kernel preemption to be disabled
on the CPU executing them.
99
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Initializing spinlocks
Static
spinlock_t my_lock = SPIN_LOCK_UNLOCKED;
Dynamic
void spin_lock_init (spinlock_t *lock);
100
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using spinlocks
Several variants, depending on where the spinlock is called:
void spin_[un]lock (spinlock_t *lock);
Doesn't disable interrupts. Used for locking in process context
(critical sections in which you do not want to sleep).
void spin_lock_irqsave / spin_unlock_irqrestore
(spinlock_t *lock, unsigned long flags);
Disables / restores IRQs on the local CPU.
Typically used when the lock can be accessed in both process
and interrupt context, to prevent preemption by interrupts.
void spin_[un]lock_bh (spinlock_t *lock);
Disables software interrupts, but not hardware ones.
Useful to protect shared data accessed in process context
and in a soft interrupt (“bottom half”). No need to disable
hardware interrupts in this case.
Note that reader / writer spinlocks also exist. 101
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Deadlock situations
They can lock up your system. Make sure they never happen!
Don't call a function that can Holding multiple locks is risky!
try to get access to the same
lock
Get lock1 Get lock2
Get lock1 call
Dead
Get lock2 Lock!
Get lock1
Wait for lock1
Dead
Lock!
102
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel lock validator
From Ingo Molnar and Arjan van de Ven
Adds instrumentation to kernel locking code
Detect violations of locking rules during system life, such as:
Locks acquired in different order
(keeps track of locking sequences and compares them).
Spinlocks acquired in interrupt handlers and also in process
context when interrupts are enabled.
Not suitable for production systems but acceptable overhead in
development.
See Documentation/lockdepdesign.txt for details
103
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Alternatives to locking
As we have just seen, locking can have a strong negative
impact on system performance. In some situations, you could
do without it.
By using lockfree algorithms like Read Copy Update (RCU).
RCU API available in the kernel
(See http://en.wikipedia.org/wiki/RCU).
When available, use atomic operations.
104
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Atomic variables
Useful when the shared resource is an
Operations without return value:
integer value void atomic_inc (atomic_t *v);
void atomic_dec (atomic_t *v);
Even an instruction like n++ is not void atomic_add (int i, atomic_t *v);
guaranteed to be atomic on all void atomic_sub (int i, atomic_t *v);
processors!
Simular functions testing the result:
Header int atomic_inc_and_test (...);
int atomic_dec_and_test (...);
#include <asm/atomic.h> int atomic_sub_and_test (...);
Type Functions returning the new value:
int atomic_inc_and_return (...);
atomic_t int atomic_dec_and_return (...);
contains a signed integer (at least 24 int atomic_add_and_return (...);
bits) int atomic_sub_and_return (...);
Atomic operations (main ones)
Set or read the counter:
atomic_set (atomic_t *v, int i);
int atomic_read (atomic_t *v);
105
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Atomic bit operations
Supply very fast, atomic operations
On most platforms, apply to an unsigned long type.
Apply to a void type on a few others.
Set, clear, toggle a given bit:
void set_bit(int nr, unsigned long * addr);
void clear_bit(int nr, unsigned long * addr);
void change_bit(int nr, unsigned long * addr);
Test bit value:
int test_bit(int nr, unsigned long *addr);
Test and modify (return the previous value):
int test_and_set_bit (...);
int test_and_clear_bit (...);
int test_and_change_bit (...);
106
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Locking
Add locking to the driver to prevent
concurrent accesses to shared
ressources
107
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Debugging and tracing
108
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Debugging with printk
Universal debugging technique used since the beginning of
programming (first found in cavemen drawings)
Printed or not in the console or /var/log/messages
according to the priority. This is controlled by the loglevel
kernel parameter, or through /proc/sys/kernel/printk
(see Documentation/sysctl/kernel.txt)
Available priorities (include/linux/kernel.h):
#define KERN_EMERG "<0>" /* system is unusable */
#define KERN_ALERT "<1>" /* action must be taken immediately */
#define KERN_CRIT "<2>" /* critical conditions */
#define KERN_ERR "<3>" /* error conditions */
#define KERN_WARNING "<4>" /* warning conditions */
#define KERN_NOTICE "<5>" /* normal but significant condition */
#define KERN_INFO "<6>" /* informational */
#define KERN_DEBUG "<7>" /* debuglevel messages */
109
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Debugging with /proc or /sys
Instead of dumping messages in the kernel log, you can have your
drivers make information available to user space
Through a file in /proc or /sys, which contents are handled by
callbacks defined and registered by your driver.
Can be used to show any piece of information
about your device or driver.
Can also be used to send data to the driver or to control it.
Caution: anybody can use these files.
You should remove your debugging interface in production!
Since the arrival of debugfs, no longer the preferred debugging
mechanism
110
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Debugfs
A virtual filesystem to export debugging information to userspace.
Kernel configuration: DEBUG_FS
Kernel hacking > Debug Filesystem
Much simpler to code than an interface in /proc or /sys.
The debugging interface disappears when Debugfs is configured
out.
You can mount it as follows:
sudo mount t debugfs none /mnt/debugfs
First described on http://lwn.net/Articles/115405/
API documented in the Linux Kernel Filesystem API:
http://freeelectrons.com/kerneldoc/latest/DocBook/filesystems/index.html
111
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Simple debugfs example
#include <linux/debugfs.h>
static char *acme_buf; // module buffer
static unsigned long acme_bufsize;
static struct debugfs_blob_wrapper acme_blob;
static struct dentry *acme_buf_dentry;
static u32 acme_state; // module variable
static struct dentry *acme_state_dentry;
/* Module init */
acme_blob.data = acme_buf;
acme_blob.size = acme_bufsize;
acme_buf_dentry = debugfs_create_blob("acme_buf", S_IRUGO, // Create
NULL, &acme_blob); // new files
acme_state_dentry = debugfs_create_bool("acme_state", S_IRUGO, // in debugfs
NULL, &acme_state);
/* Module exit */
debugfs_remove (acme_buf_dentry); // removing the files from debugfs
debugfs_remove (acme_state_dentry);
112
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Debugging with ioctl
Can use the ioctl() system call to query information
about your driver (or device) or send commands to it.
This calls the ioctl file operation that you can register in
your driver.
Advantage: your debugging interface is not public.
You could even leave it when your system (or its driver) is in
the hands of its users.
113
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using Magic SysRq
Allows to run multiple debug / rescue commands even when the
kernel seems to be in deep trouble
On PC: Alt + SysRq + <character>
On embedded: break character on the serial line + <character>
. Example commands:
n: makes RT processes niceable.
t: shows the kernel stack of all sleeping processes
w: shows the kernel stack of all running processes
b: reboot the system
You can even register your own!
Detailed in Documentation/sysrq.txt
114
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
kgdb A kernel debugger
The execution of the kernel is fully controlled by gdb from
another machine, connected through a serial line.
Can do almost everything, including inserting breakpoints in
interrupt handlers.
Feature included in standard Linux since 2.6.26 (x86 and
sparc). arm, mips and ppc support merged in 2.6.27.
115
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using kgdb
Details available in the kernel documentation:
http://freeelectrons.com/kerneldoc/latest/DocBook/kgdb/
Recommended to turn on CONFIG_FRAME_POINTER to aid
in producing more reliable stack backtraces in gdb.
You must include a kgdb I/O driver. One of them is kgdb
over serial console (kgdboc: kgdb over console, enabled
by CONFIG_KGDB_SERIAL_CONSOLE)
Configure kgdboc at boot time by passing to the kernel:
kgdboc=<ttydevice>,[baud]. For example:
kgdboc=ttyS0,115200
116
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Using kgdb (2)
Then also pass kgdbwait to the kernel:
it makes kgdb wait for a debugger connection.
Boot your kernel, and when the console is initialized, interrupt the
kernel with [Alt][SyrRq][g].
On your workstation, start gdb as follows:
% gdb ./vmlinux
(gdb) set remotebaud 115200
(gdb) target remote /dev/ttyS0
Once connected, you can debug a kernel the way you would
debug an application program.
117
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Debugging with a JTAG interface
Two types of JTAG dongles
Those offering a gdb compatible interface, over a serial port or an Ethernet
connexion. Gdb can directly connect to them.
Those not offering a gdb compatible interface are generally supported by
OpenOCD (Open On Chip Debugger)
OpenOCD is the bridge between the gdb debugging language and the
JTAGdongle specific language
http://openocd.berlios.de/web/
See the very complete documentation: http://openocd.berlios.de/doc/
For each board, you'll need an OpenOCD configuration file (ask your
supplier)
See very useful details on using Eclipse / gcc / gdb / OpenOCD on Windows:
http://www2.amontec.com/sdk4arm/ext/jlynchtutorial20061124.pdf and
http://www.yagarto.de/howto/yagarto2/
118
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
More kernel debugging tips
Enable CONFIG_KALLSYMS_ALL
(General Setup > Configure standard kernel features)
to get oops messages with symbol names instead of raw addresses
(this obsoletes the ksymoops tool).
If your kernel doesn't boot yet or hangs without any message, you can
activate Low Level debugging (Kernel Hacking section, only available on
arm):
CONFIG_DEBUG_LL=y
Techniques to locate the C instruction which caused an oops:
http://kerneltrap.org/node/3648
More about kernel debugging in the free Linux Device Drivers book:
http://lwn.net/images/pdf/LDD3/ch04.pdf
119
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Tracing with SystemTap
http://sourceware.org/systemtap/
Infrastructure to add instrumentation to a running kernel:
trace functions, read and write variables, follow pointers, gather statistics...
Eliminates the need to modify the kernel sources to add one's own
instrumentation to investigated a functional or performance problem.
Uses a simple scripting language.
Several example scripts and probe points are available.
Based on the Kprobes instrumentation infrastructure.
See Documentation/kprobes.txt in kernel sources.
Linux 2.6.26: supported on most popular CPUs (arm included in 2.6.25).
However, lack of recent support for mips (2.6.16 only!).
120
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
SystemTap script example (1)
#! /usr/bin/env stap
# Using statistics and maps to examine kernel memory allocations
global kmalloc
probe kernel.function("__kmalloc") {
kmalloc[execname()] <<< $size
}
# Exit after 10 seconds
probe timer.ms(10000) { exit () }
probe end {
foreach ([name] in kmalloc) {
printf("Allocations for %s\n", name)
printf("Count: %d allocations\n", @count(kmalloc[name]))
printf("Sum: %d Kbytes\n", @sum(kmalloc[name])/1000)
printf("Average: %d bytes\n", @avg(kmalloc[name]))
printf("Min: %d bytes\n", @min(kmalloc[name]))
printf("Max: %d bytes\n", @max(kmalloc[name]))
print("\nAllocations by size in bytes\n")
print(@hist_log(kmalloc[name]))
printf("\n\n");
}
}
121
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
SystemTap script example (2)
#! /usr/bin/env stap
# Logs each file read performed by each process
probe kernel.function ("vfs_read")
{
dev_nr = $file>f_dentry>d_inode>i_sb>s_dev
inode_nr = $file>f_dentry>d_inode>i_ino
printf ("%s(%d) %s 0x%x/%d\n",
execname(), pid(), probefunc(), dev_nr, inode_nr)
}
Nice tutorial on http://sources.redhat.com/systemtap/tutorial.pdf
122
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel crash analysis with kexec/kdump
1. Copy debug Standard kernel
kexec system call: makes it possible to kernel to
reserved 2. kernel
call a new kernel, without rebooting and RAM panic, kexec
debug kernel
going through the BIOS / firmware.
3. Analyze
Idea: after a kernel panic, make the crashed Debug kernel
kernel RAM
kernel automatically execute a new,
clean kernel from a reserved location in
RAM, to perform postmortem analysis
of the memory of the crashed kernel.
See Documentation/kdump/kdump.txt
in the kernel sources for details.
Regular RAM
123
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel markers
Capability to add static markers to kernel code,
merged in Linux 2.6.24 by Matthieu Desnoyers.
Almost no impact on performance, until the marker is dynamically
enabled, by inserting a probe kernel module.
Useful to insert trace points that won't be impacted by changes in
the Linux kernel sources.
See marker and probe example
in samples/markers in the kernel sources.
See http://en.wikipedia.org/wiki/Kernel_marker
124
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
LTTng
http://lttng.org
The successor of the Linux Trace Toolkit (LTT)
Toolkit allowing to collect and analyze tracing information from
the kernel, based on kernel markers and kernel tracepoints.
So far, based on kernel patches, but doing its best to use intree
solutions, and to be merged in the future.
Very precise timestamps, very little overhead.
Useful documentation on http://lttng.org/?q=node/2#manuals
125
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
LTTV
Viewer for LTTng traces
Support for huge traces (tested with 15 GB ones)
Can combine multiple tracefiles in a single view.
Graphical or text interface
See http://lttng.org/files/lttvdoc/user_guide/
126
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Kernel debugging
Load a broken driver and see it crash
Analyze the error information
dumped by the kernel.
Disassemble the code and locate
the exact C instruction which caused
the failure.
Use the JTAG and OpenOCD to
remotely control the kernel execution
127
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
mmap
128
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
mmap (1)
Possibility to have parts of the virtual address space of a program
mapped to the contents of a file!
> cat /proc/1/maps (init process)
start end perm offset major:minor inode mapped file name
007710000077f000 rxp 00000000 03:05 1165839 /lib/libselinux.so.1
0077f00000781000 rwp 0000d000 03:05 1165839 /lib/libselinux.so.1
0097d00000992000 rxp 00000000 03:05 1158767 /lib/ld2.3.3.so
0099200000993000 rp 00014000 03:05 1158767 /lib/ld2.3.3.so
0099300000994000 rwp 00015000 03:05 1158767 /lib/ld2.3.3.so
0099600000aac000 rxp 00000000 03:05 1158770 /lib/tls/libc2.3.3.so
00aac00000aad000 rp 00116000 03:05 1158770 /lib/tls/libc2.3.3.so
00aad00000ab0000 rwp 00117000 03:05 1158770 /lib/tls/libc2.3.3.so
00ab000000ab2000 rwp 00ab0000 00:00 0
0804800008050000 rxp 00000000 03:05 571452 /sbin/init (text)
0805000008051000 rwp 00008000 03:05 571452 /sbin/init (data, stack)
08b4300008b64000 rwp 08b43000 00:00 0
f6fdf000f6fe0000 rwp f6fdf000 00:00 0
fefd4000ff000000 rwp fefd4000 00:00 0
ffffe000fffff000 p 00000000 00:00 0
129
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
mmap (2)
Particularly useful when the file is a device file!
Allows to access device I/O memory and ports without having to
go through (expensive) read, write or ioctl calls!
X server example (maps excerpt)
start end perm offset major:minor inode mapped file name
08047000081be000 rxp 00000000 03:05 310295 /usr/X11R6/bin/Xorg
081be000081f0000 rwp 00176000 03:05 310295 /usr/X11R6/bin/Xorg
...
f4e08000f4f09000 rws e0000000 03:05 655295 /dev/dri/card0
f4f09000f4f0b000 rws 4281a000 03:05 655295 /dev/dri/card0
f4f0b000f6f0b000 rws e8000000 03:05 652822 /dev/mem
f6f0b000f6f8b000 rws fcff0000 03:05 652822 /dev/mem
A more user friendly way to get such information: pmap <pid>
130
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
mmap overview
mmap
system
call (once) Device driver
Process mmap fop called
initializes the mapping
access
virtual
address access
MMU physical
address
Process virtual address space Physical address space
131
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to implement mmap User space
Open the device file
Call the mmap system call (see man mmap for details):
void * mmap(
void *start, /* Often 0, preferred starting address */
size_t length, /* Length of the mapped area */
int prot , /* Permissions: read, write, execute */
int flags, /* Options: shared mapping, private copy...
*/
int fd, /* Open file descriptor */
off_t offset /* Offset in the file */
);
You get a virtual address you can write to or read from.
132
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to implement mmap Kernel space
Character driver: implement a mmap file operation
and add it to the driver file operations:
int (*mmap) (
struct file *, /* Open file structure */
struct vm_area_struct * /* Kernel VMA structure */
);
Initialize the mapping.
Can be done in most cases with the remap_pfn_range()
function, which takes care of most of the job.
133
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
remap_pfn_range()
pfn: page frame number
The most significant bits of the page address
(without the bits corresponding to the page size).
#include <linux/mm.h>
int remap_pfn_range(
struct vm_area_struct *, /* VMA struct */
unsigned long virt_addr, /* Starting user virtual address */
unsigned long pfn, /* pfn of the starting physical address */
unsigned long size, /* Mapping size */
pgprot_t /* Page permissions */
);
134
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Simple mmap implementation
static int acme_mmap (
struct file * file, struct vm_area_struct * vma)
{
size = vma>vm_end vma>vm_start;
if (size > ACME_SIZE)
return EINVAL;
if (remap_pfn_range(vma,
vma>vm_start,
ACME_PHYS >> PAGE_SHIFT,
size,
vma>vm_page_prot))
return EAGAIN;
return 0;
}
135
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
devmem2
http://freeelectrons.com/pub/mirror/devmem2.c, by JanDerk Bakker
Very useful tool to directly peek (read) or poke (write) I/O addresses
mapped in physical address space from a shell command line!
Very useful for early interaction experiments with a device, without
having to code and compile a driver.
Uses mmap to /dev/mem.
Examples (b: byte, h: half, w: word)
devmem2 0x000c0004 h (reading)
devmem2 0x000c0008 w 0xffffffff (writing)
devmem is now available in BusyBox, making it even easier to use.
136
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
mmap summary
The device driver is loaded.
It defines an mmap file operation.
A user space process calls the mmap system call.
The mmap file operation is called.
It initializes the mapping using the device physical address.
The process gets a starting address to read from and write to
(depending on permissions).
The MMU automatically takes care of converting the process
virtual addresses into physical ones.
Direct access to the hardware!
No expensive read or write system calls!
137
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Driver development
Kernel architecture for device drivers
138
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel and device drivers
Application
Userspace
System call interface
Framework
Kernel
Driver
Bus infrastructure
Hardware
139
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel and device drivers
Many device drivers are not implemented directly as character
drivers
They are implemented under a « framework », specific to a given
device type (framebuffer, V4L, serial, etc.)
The framework allows to factorize the common parts of drivers for
the same type of devices
From userspace, they are still seen as character devices by the
applications
The framework allows to provide a coherent userspace interface
(ioctl, etc.) for every type of device, regardless of the driver
The device drivers rely on the « bus infrastructure » to enumerate
the devices and communicate with them.
140
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Kernel frameworks
141
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
« Frameworks »
Application Application Application
System call interface
Character Framebuffer V4L TTY Block
driver core core core core
Framebuffer V4L Serial IDE SCSI
driver driver core core core
Serial IDE USB storage
driver driver driver
Bus infrastructure
142
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Example: framebuffer framework
Kernel option CONFIG_FB
menuconfig FB
tristate "Support for frame buffer devices"
Implemented in drivers/video/
fb.c, fbmem.c, fbmon.c, fbcmap.c, fbsysfs.c,
modedb.c, fbcvt.c
Implements a single character driver and defines the user/kernel
API
First part of include/linux/fb.h
Defines the set of operations a framebuffer driver must implement
and helper functions for the drivers
struct fb_ops
Second part of include/linux/fb.h
(in #ifdef __KERNEL__)
143
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Framebuffer driver skeleton
Skeleton driver in drivers/video/skeletonfb.c
Implements the set of framebuffer specific operations defined
by the struct fb_ops structure
xxxfb_open() xxxfb_fillrect()
xxxfb_read() xxxfb_copyarea()
xxxfb_write() xxxfb_imageblit()
xxxfb_release() xxxfb_cursor()
xxxfb_checkvar() xxxfb_rotate()
xxxfb_setpar() xxxfb_sync()
xxxfb_setcolreg() xxxfb_ioctl()
xxxfb_blank() xxxfb_mmap()
xxxfb_pan_display()
144
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Framebuffer driver skeleton
After the implementation of the operations, definition of a struct
fb_ops structure
static struct fb_ops xxxfb_ops = {
.owner = THIS_MODULE,
.fb_open = xxxfb_open,
.fb_read = xxxfb_read,
.fb_write = xxxfb_write,
.fb_release = xxxfb_release,
.fb_check_var = xxxfb_check_var,
.fb_set_par = xxxfb_set_par,
.fb_setcolreg = xxxfb_setcolreg,
.fb_blank = xxxfb_blank,
.fb_pan_display = xxxfb_pan_display,
.fb_fillrect = xxxfb_fillrect, /* Needed !!! */
.fb_copyarea = xxxfb_copyarea, /* Needed !!! */
.fb_imageblit = xxxfb_imageblit, /* Needed !!! */
.fb_cursor = xxxfb_cursor, /* Optional !!! */
.fb_rotate = xxxfb_rotate,
.fb_sync = xxxfb_sync,
.fb_ioctl = xxxfb_ioctl,
.fb_mmap = xxxfb_mmap,
};
145
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Framebuffer driver skeleton
In the probe() function, registration of the framebuffer device
and operations
static int __devinit xxxfb_probe
(struct pci_dev *dev,
const struct pci_device_id *ent)
{
struct fb_info *info;
[...]
info = framebuffer_alloc(sizeof(struct xxx_par), device);
[...]
info>fbops = &xxxfb_ops;
[...]
if (register_framebuffer(info) < 0)
return EINVAL;
[...]
}
register_framebuffer() will create the character device
that can be used by userspace application with the generic
framebuffer API
146
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Device Model and Bus Infrastructure
147
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Unified device model
The 2.6 kernel included a significant new feature:
a unified device model
Instead of having different adhoc mechanisms in the various
subsystems, the device model unifies the description of the devices
and their topology
Minimizing code duplication
Common facilities (reference counting, event notification, power
management, etc.)
Enumerate the devices, view their interconnections, link the devices to
their buses and drivers, categorize them by classes.
148
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Bus drivers
The first component of the device model is the bus driver
One bus driver for each type of bus: USB, PCI, SPI, MMC, ISA, etc.
It is responsible for
Registering the bus type
Allowing the registration of adapter drivers (USB controllers, I2C
adapters, etc.), able of detecting the connected devices
Allowing the registration of device drivers (USB devices, I2C
devices, PCI devices, etc.), managing the devices
Matching the device drivers against the devices detected by the
adapter drivers.
149
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
List of device identifiers
Depending on the bus type, the method for binding a device to a
driver is different. For many buses, it is based on unique identifiers
The device driver defines a table with the list of device identifiers it
is able to manage :
static const struct pci_device_id rhine_pci_tbl[] = {
{ 0x1106, 0x3043, PCI_ANY_ID, PCI_ANY_ID, }, /* VT86C100A */
{ 0x1106, 0x3053, PCI_ANY_ID, PCI_ANY_ID, }, /* VT6105M */
{ } /* terminate list */
};
MODULE_DEVICE_TABLE(pci, rhine_pci_tbl);
Code on this slide and on the next slides are taken
from the viarhine driver in drivers/net/viarhine.c
150
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Defining the driver
The device driver defines a driver structure, usually specialized by
the bus infrastructure (pci_driver, usb_driver, etc.)
The structure points to: the device table, a probe function, called
when a device is detected and various other callbacks
static struct pci_driver rhine_driver = {
.name = DRV_NAME,
.id_table = rhine_pci_tbl,
.probe = rhine_init_one,
.remove = __devexit_p(rhine_remove_one),
#ifdef CONFIG_PM
.suspend = rhine_suspend,
.resume = rhine_resume,
#endif /* CONFIG_PM */
.shutdown = rhine_shutdown,
};
151
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering the driver
In the module initialization function, the driver is registered to the
bus infrastructure, in order to let the bus know that the driver is
available to handle devices.
static int __init rhine_init(void)
{
[...]
return pci_register_driver(&rhine_driver);
}
static void __exit rhine_cleanup(void)
{
pci_unregister_driver(&rhine_driver);
}
If a new PCI device matches one of the identifiers of the table, the
probe() method of the PCI driver will get called.
152
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Probe method
The probe() method receives as argument a structure
describing the device, usually specialized by the bus
infrastructure (pci_dev, usb_device, etc.)
This function is responsible for
Initializing the device, mapping I/O memory, registering the
interrupt handlers. The bus infrastructure provides methods to
get the addresses, interrupts numbers and other device
specific information.
Registering the device to the proper kernel framework, for
example the network infrastructure.
153
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Device driver (5)
static int __devinit rhine_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
struct net_device *dev;
[...]
rc = pci_enable_device(pdev);
[...]
pioaddr = pci_resource_start(pdev, 0);
memaddr = pci_resource_start(pdev, 1);
[...]
dev = alloc_etherdev(sizeof(struct rhine_private));
[...]
SET_NETDEV_DEV(dev, &pdev>dev);
[...]
rc = pci_request_regions(pdev, DRV_NAME);
[...]
ioaddr = pci_iomap(pdev, bar, io_size);
[...]
rc = register_netdev(dev);
[...]
}
154
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Global architecture
PCI bus driver Network
1 2
device
The device The bus driver
driver detects a
framework
registers matching device,
itself and the and calls the 3
device probe() method The probe()
identifiers to of the device method of the
the bus driver. device driver
driver initializes the
device and
viarhine registers a new
device driver network interface
155
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
sysfs
The bus, device, drivers, etc. structures are internal to the
kernel
The sysfs virtual filesystem offers a mechanism to export such
information to userspace
Used for example by udev to provide automatic module loading,
firmware loading, device file creation, etc.
sysfs is usually mounted in /sys
/sys/bus/ contains the list of buses
/sys/devices/ contains the list of devices
/sys/class enumerates devices by class (net, input,
block...), whatever the bus they are connected to. Very useful!
Take your time to explore /sys on your workstation.
156
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Platform devices
On embedded systems, devices are often not connected
through a bus allowing enumeration, hotplugging, and providing
unique identifiers for devices.
However, we still want the devices to be part of the device
model.
The solution to this is the platform driver / platform device
infrastructure.
The platform devices are the devices that are directly connected
to the CPU, without any kind of bus.
157
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Implementation of the platform driver
The driver implements a platform_driver structure
(example taken from drivers/serial/imx.c)
static struct platform_driver serial_imx_driver = {
.probe = serial_imx_probe,
.remove = serial_imx_remove,
.driver = {
.name = "imxuart",
.owner = THIS_MODULE,
},
};
And registers its driver to the platform driver infrastructure
static int __init imx_serial_init(void)
{
[...]
ret = platform_driver_register(&serial_imx_driver);
[...]
}
158
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Platform device instantiation (1)
In the boardspecific code, the platform devices are
instantiated (arch/arm/machimx/mx1ads.c):
static struct platform_device imx_uart1_device = {
.name = "imxuart",
.id = 0,
.num_resources = ARRAY_SIZE(imx_uart1_resources),
.resource = imx_uart1_resources,
.dev = {
.platform_data = &uart_pdata,
}
};
The match between the device and the driver is made using
the name. It must be unique amongst drivers !
159
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Platform device instantiation (2)
The device is part of a list
static struct platform_device *devices[] __initdata = {
&cs89x0_device,
&imx_uart1_device,
&imx_uart2_device,
};
And the list of devices is added to the system
during board initialization
static void __init mx1ads_init(void)
{
[...]
platform_add_devices(devices, ARRAY_SIZE(devices));
}
160
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
I/O resources
Each platform device is associated with a set of I/O resources,
referenced in the platform_device structure
static struct resource imx_uart1_resources[] = {
[0] = {
.start = 0x00206000,
.end = 0x002060FF,
.flags = IORESOURCE_MEM,
},
[1] = {
.start = (UART1_MINT_RX),
.end = (UART1_MINT_RX),
.flags = IORESOURCE_IRQ,
},
};
It allows the driver to be independent of the I/O addresses, IRQ
numbers ! See imx_uart2_device for another device using the
same platform driver.
161
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Inside the platform driver
When a platform_device is added to the system using
platform_add_device(), the probe() method of the platform
driver gets called
This method is responsible for initializing the hardware,
registering the device to the proper framework (in our case, the
serial driver framework)
The platform driver has access to the I/O resources :
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
base = ioremap(res>start, PAGE_SIZE);
sport>rxirq = platform_get_irq(pdev, 0);
162
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Framework and bus infrastructure
A typical driver will
Be registered inside a framework
Rely on a bus infrastructure and the device model
Example with the iMX serial driver, drivers/serial/imx.c
At module initialization time, the driver registers itself both to the
UART framework and to the platform bus infrastructure
static int __init imx_serial_init(void)
{
ret = uart_register_driver(&imx_reg);
[...]
ret = platform_driver_register(&serial_imx_driver);
[...]
return 0;
}
163
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
iMX serial driver
Definition of the iMX UART driver
static struct uart_driver imx_reg = {
.owner = THIS_MODULE,
.driver_name = DRIVER_NAME,
.dev_name = DEV_NAME,
.major = SERIAL_IMX_MAJOR,
.minor = MINOR_START,
.nr = ARRAY_SIZE(imx_ports),
.cons = IMX_CONSOLE,
};
Definition of the iMX platform driver
static struct platform_driver serial_imx_driver = {
.probe = serial_imx_probe,
.remove = serial_imx_remove,
[...]
.driver = {
.name = "imxuart",
.owner = THIS_MODULE,
},
};
164
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
iMX serial driver
When the platform device is instantiated, the probe()
method is called
static int serial_imx_probe(struct platform_device *pdev)
{
struct imx_port *sport;
sport = kzalloc(sizeof(*sport), GFP_KERNEL);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
base = ioremap(res>start, PAGE_SIZE);
/* sport initialization */
sport>port.irq = platform_get_irq(pdev, 0);
sport>port.ops = &imx_pops;
sport>clk = clk_get(&pdev>dev, "uart_clk");
clk_enable(sport>clk);
uart_add_one_port(&imx_reg, &sport>port);
}
165
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
iMX serial driver
The operation structure uart_ops is associated to each
port. The operations are implemented in the iMX driver
static struct uart_ops imx_pops = {
.tx_empty = imx_tx_empty,
.set_mctrl = imx_set_mctrl,
.get_mctrl = imx_get_mctrl,
.stop_tx = imx_stop_tx,
.start_tx = imx_start_tx,
.stop_rx = imx_stop_rx,
.enable_ms = imx_enable_ms,
.break_ctl = imx_break_ctl,
.startup = imx_startup,
.shutdown = imx_shutdown,
.set_termios = imx_set_termios,
.type = imx_type,
.release_port = imx_release_port,
.request_port = imx_request_port,
.config_port = imx_config_port,
.verify_port = imx_verify_port,
};
166
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
References
Kernel documentation
Documentation/drivermodel/
Documentation/filesystems/sysfs.txt
Linux 2.6 Device Model
http://www.bravegnu.org/devicemodel/devicemodel.html
Linux Device Drivers, chapter 14 «The Linux Device Model»
http://lwn.net/images/pdf/LDD3/ch14.pdf
The kernel source code
Full of examples of other drivers!
167
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Annexes
Quiz answers
168
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Quiz answers
Interrupt handling
Q: Why did the kernel segfault at module unload (forgetting to
unregister a handler in a shared interrupt line)?
A: Kernel memory is allocated at module load time, to host module
code. This memory is freed at module unload time. If you forget to
unregister a handler and an interrupt comes, the cpu will try to jump
to the address of the handler, which is in a freed memory area.
Crash!
169
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux device driver development
DMA
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20042009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/dma
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
DMA memory constraints
Need to use contiguous memory in physical space.
Can use any memory allocated by kmalloc (up to 128
KB) or __get_free_pages (up to 8MB).
Can use block I/O and networking buffers,
designed to support DMA.
Can not use vmalloc memory
(would have to setup DMA on each individual physical
page).
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reserving memory for DMA
To make sure you've got enough RAM for big DMA transfers...
Example assuming you have 32 MB of RAM, and need 2 MB for
DMA:
Boot your kernel with mem=30
The kernel will just use the first 30 MB of RAM.
Driver code can now reclaim the 2 MB left:
dmabuf = ioremap (
0x1e00000, /* Start: 30 MB */
0x200000 /* Size: 2 MB */
);
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Memory synchronization issues
Memory caching could interfere with DMA
Before DMA to device:
Need to make sure that all writes to DMA buffer are
committed.
After DMA from device:
Before drivers read from DMA buffer, need to make sure that
memory caches are flushed.
Bidirectional DMA
Need to flush caches before and after the DMA transfer.
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux DMA API
The kernel DMA utilities can take care of:
Either allocating a buffer in a cache coherent area,
Or making sure caches are flushed when required,
Managing the DMA mappings and IOMMU (if any).
See Documentation/DMAAPI.txt
for details about the Linux DMA generic API.
Most subsystems (such as PCI or USB) supply their own DMA
API, derived from the generic one. May be sufficient for most
needs.
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Coherent or streaming DMA mappings
Coherent mappings Streaming mappings
The kernel allocates a suitable buffer The kernel just sets the mapping for a
and sets the mapping for the driver. buffer provided by the driver.
Can simultaneously be accessed by Use a buffer already allocated
the CPU and device. by the driver.
So, has to be in a cache coherent Mapping set up for each transfer.
memory area. Keeps DMA registers free on the
hardware.
Usually allocated for the whole time
the module is loaded. Some optimizations also available.
Can be expensive to setup and use The recommended solution.
on some platforms.
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Allocating coherent mappings
The kernel takes care of both buffer allocation and mapping:
include <asm/dmamapping.h>
void * /* Output: buffer address */
dma_alloc_coherent(
struct device *dev, /* device structure */
size_t size, /* Needed buffer size in bytes */
dma_addr_t *handle, /* Output: DMA bus address */
gfp_t gfp /* Standard GFP flags */
);
void dma_free_coherent(struct device *dev,
size_t size, void *cpu_addr, dma_addr_t handle);
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Setting up streaming mappings
Works on buffers already allocated by the driver
<include linux/dmapool.h>
dma_addr_t dma_map_single(
struct device *, /* device structure */
void *, /* input: buffer to use */
size_t, /* buffer size */
enum dma_data_direction /* Either DMA_BIDIRECTIONAL,
DMA_TO_DEVICE or DMA_FROM_DEVICE */
);
void dma_unmap_single(struct device *dev, dma_addr_t
handle, size_t size, enum dma_data_direction dir);
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
DMA streaming mapping notes
When the mapping is active: only the device should access
the buffer (potential cache issues otherwise).
The CPU can access the buffer only after unmapping!
Use locking to prevent CPU access to the buffer.
Another reason: if required, this API can create an
intermediate bounce buffer (used if the given buffer is not
usable for DMA).
The Linux API also supports scatter / gather DMA streaming
mappings.
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
DMA summary
Most drivers can use the specific API provided by their subsystem: USB,
PCI, SCSI... Otherwise they can use the Linux generic API:
Coherent mappings Streaming mappings
DMA buffer allocated by the kernel DMA buffer allocated by the driver
Set up for the whole module life Set up for each transfer
Can be expensive. Not Cheaper. Saves DMA registers.
recommended.
Only the device can access the buffer
Let both the CPU and device when the mapping is active.
access the buffer at the same time.
Main functions:
Main functions: dma_map_single
dma_alloc_coherent dma_unmap_single
dma_free_coherent
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux PCI drivers
Linux
PCI drivers
Michael Opdenacker
Free Electrons
© Copyright 20042009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://en.wikipedia.org/wiki/Image:PCI_Slots_Digon3.JPG http://freeelectrons.com/docs/pcidrivers
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux PCI drivers
Understanding PCI
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI bus family
The following buses belong to the PCI family:
PCI PCI Express (PCIe or PCIE)
32 bit bus, 33 or 66 MHz Current generation of PCI. Serial
instead of parallel.
MiniPCI
Smaller slot in laptops PCI Express Mini Card
Replaces MiniPCI in recent
CardBus
laptops
External card slot in laptops
Express Card
PIX Extended (PCIX)
Replaces CardBus in recent
Wider slot than PCI, 64 bit, but
laptops
can accept a standard PCI card
These technologies are compatible and can be handled by the same kernel drivers.
The kernel doesn't need to know which exact slot and bus variant is used.
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device types
Main types of devices found on the PCI bus
Network cards (wired or wireless)
SCSI adapters
Bus controllers: USB, PCMCIA, I2C, FireWire, IDE
Graphics and video cards
Sound cards
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI features
For device driver developers
Device resources (I/O addresses, IRQ lines) automatically
assigned at boot time, either by the BIOS or by Linux itself (if
configured).
The device driver just has to read the corresponding
configurations somewhere in the system address space.
Endianism: PCI device configuration information is Little Endian.
Remember that in your drivers
(conversion taken care of by some kernel functions).
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device list (1)
lscpi
00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 01)
00:1c.2 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 3 (rev 01)
00:1d.0 USB Controller [...]
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7M) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
02:01.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev b4)
02:01.1 FireWire (IEEE 1394): Ricoh Co Ltd R5C552 IEEE 1394 Controller (rev 09)
02:01.2 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 18)
09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752 Gigabit Ethernet PCI Express (rev 02)
0c:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection (rev 61)
lspci tv
[0000:00]+00.0 Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub
+02.0 Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller
+02.1 Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller
+1b.0 Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller
+1c.0[0000:0b]
+1c.1[0000:0c]00.0 Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection
+1c.2[0000:09]00.0 Broadcom Corporation NetXtreme BCM5752 Gigabit Ethernet PCI Express
+1d.0 Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 [...]
+1e.0[0000:0206]+01.0 Ricoh Co Ltd RL5c476 II
| +01.1 Ricoh Co Ltd R5C552 IEEE 1394 Controller
| \01.2 Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter
+1f.0 Intel Corporation 82801GBM (ICH7M) LPC Interface Bridge
+1f.1 Intel Corporation 82801G (ICH7 Family) IDE Controller
\1f.3 Intel Corporation 82801G (ICH7 Family) SMBus Controller
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device list (2)
lspci enumerates all PCI devices
02:01.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev b4)
02:01.1 FireWire (IEEE 1394): Ricoh Co Ltd R5C552 IEEE 1394 Controller
02:01.2 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro
Function number
PCI device number
PCI bus number
lscpi t shows the bus device tree
[0000:00]+00.0 Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub
+02.0 Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller
+02.1 Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated Graphics Controller
+1b.0 Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller
+1c.0[0000:0b]
PCI domain
+1c.1[0000:0c]00.0 Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection
+1c.2[0000:09]00.0 Broadcom Corporation NetXtreme BCM5752 Gigabit Ethernet PCI Express
+1d.0 Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 [...]
PCI bus 0 +1e.0[0000:0206]+01.0 Ricoh Co Ltd RL5c476 II
| +01.1 Ricoh Co Ltd R5C552 IEEE 1394 Controller
| \01.2 Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter
PCI bridge PCI bus 2
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device list (3)
This tree structure reflects the structure in /sys:
/sys/devices/pci0000:00/0000:00:1e.0/0000:02:01.2
Bus 0 PCI bridge Bus 2
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device configuration (1)
Each PCI device has a 256 byte address space containing
configuration registers.
Device configuration can be displayed with lspci x:
0c:00.0 Network controller: Intel Corporation PRO/Wireless
4965 AG or AGN Network Connection (rev 61)
00: 86 80 29 42 06 04 10 00 61 00 80 02 10 00 00 00
10: 04 e0 df ef 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 21 11
30: 00 00 00 00 c8 00 00 00 00 00 00 00 05 01 00 00
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device configuration (2)
Standard information found in PCI configurations:
Offset 0: Vendor Id
Offset 2: Device Id
Offset 10: Class Id (network, display, bridge...)
Offsets 16 to 39: Base Address Registers (BAR) 0 to 6
Offset 44: Subvendor Id
Offset 46: Subdevice Id
Offsets 64 and up: up to the device manufacturer
Kernel sources: these offsets are defined in
include/linux/pci_regs.h
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Linux PCI drivers
Implementing Linux drivers
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering supported devices
From drivers/net/ne2kpci.c (Linux 2.6.27):
static struct pci_device_id ne2k_pci_tbl[] = {
{ 0x10ec, 0x8029, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_RealTek_RTL_8029 },
{ 0x1050, 0x0940, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Winbond_89C940 },
{ 0x11f6, 0x1401, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Compex_RL2000 },
{ 0x8e2e, 0x3000, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_KTI_ET32P2 },
{ 0x4a14, 0x5000, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_NetVin_NV5000SC },
{ 0x1106, 0x0926, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Via_86C926 },
{ 0x10bd, 0x0e34, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_SureCom_NE34 },
{ 0x1050, 0x5a5a, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Winbond_W89C940F },
{ 0x12c3, 0x0058, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Holtek_HT80232 },
{ 0x12c3, 0x5598, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Holtek_HT80229 },
{ 0x8c4a, 0x1980, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CH_Winbond_89C940_8c4a },
{ 0, }
};
MODULE_DEVICE_TABLE(pci, ne2k_pci_tbl);
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering a driver (1)
Declaring driver hooks and supported devices table:
static struct pci_driver ne2k_driver = {
.name = DRV_NAME,
.probe = ne2k_pci_init_one,
.remove = __devexit_p(ne2k_pci_remove_one),
.id_table = ne2k_pci_tbl,
#ifdef CONFIG_PM
.suspend = ne2k_pci_suspend,
.resume = ne2k_pci_resume,
#endif /* CONFIG_PM */
};
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Registering a driver (2)
static int __init ne2k_pci_init(void)
{
return pci_register_driver(&ne2k_driver);
}
static void __exit ne2k_pci_cleanup(void)
{
pci_unregister_driver (&ne2k_driver);
}
The hooks and supported devices are loaded at module loading
time.
The probe() hook gets called by the PCI generic code
when a matching device is found.
Very similar to USB device drivers!
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Marking driver hooks (1)
__init: module init function.
Code discarded after driver initialization.
__exit: module exit function.
Ignored for statically compiled drivers.
__devinit: probe function and all initialization functions
Normal function if CONFIG_HOTPLUG is set. Identical to __init
otherwise.
__devinitconst: for the device id table
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Marking driver hooks (2)
__devexit: functions called at remove time.
Same case as in __devinit
All references to __devinit function addresses should be
declared with __devexit_p(fun). This replaces the
function address by NULL if this code is discarded.
Example: same driver:
static struct pci_driver ne2k_driver = {
.name = DRV_NAME,
.probe = ne2k_pci_init_one,
.remove =
__devexit_p(ne2k_pci_remove_one),
.id_table = ne2k_pci_tbl,
...
};
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Device initialization steps
Enable the device
Request I/O port and I/O memory resources
Set the DMA mask size
(for both coherent and streaming DMA)
Allocate and initialize shared control data
(pci_allocate_coherent())
Initialize device registers (if needed)
Register IRQ handler (request_irq())
Register to other subsystems (network, video, disk, etc.)
Enable DMA/processing engines.
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Enabling the device
Before touching any device registers, the driver should first
execute pci_enable_device(). This will:
Wake up the device if it was in suspended state
Allocate I/O and memory regions of the device
(if not already done by the BIOS)
Assign an IRQ to the device
(if not already done by the BIOS)
pci_enable_device() can fail. Check the return value!
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
pci_enable_device example
From drivers/net/ne2kpci.c (Linux 2.6.27):
static int __devinit ne2k_pci_init_one
(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
...
i = pci_enable_device (pdev);
if (i)
return i;
...
}
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Enabling the device (2)
Enable DMA by calling pci_set_master(). This will:
Enable DMA by setting the bus master bit in the
PCI_COMMAND register. The device will then be able to act
as a master on the address bus.
Fix the latency timer value if it's set to something bogus by
the BIOS.
If the device can use the PCI MemoryWriteInvalidate
transaction (writing entire cache lines), you can also call
pci_set_mwi():
This enables the PCI_COMMAND bit for MemoryWrite
Invalidate
This also ensures that the cache line size register is set
correctly.
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing configuration registers (1)
Needed to access I/O memory and port information
#include <linux/pci.h>
Reading:
int pci_read_config_byte(struct pci_dev *dev,
int where, u8 *val);
int pci_read_config_word(struct pci_dev *dev,
int where, u16 *val);
int pci_read_config_dword(struct pci_dev *dev,
int where, u32 *val);
Example: drivers/net/cassini.c
pci_read_config_word(cp>pdev, PCI_STATUS, &cfg);
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing configuration registers (2)
Writing:
int pci_write_config_byte(struct pci_dev *dev,
int where, u8 val);
int pci_write_config_word(struct pci_dev *dev,
int where, u16 val);
int pci_write_config_dword(struct pci_dev *dev,
int where, u32 val);
Example: drivers/net/s2io.c
/* Clear "detected parity error" bit
pci_write_config_word(sp>pdev, PCI_STATUS, 0x8000);
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing I/O registers and memory (1)
Each PCI device can have up to 6 I/O or memory regions,
described in BAR0 to BAR5.
Access the base address of the I/O region:
#include <linux/pci.h>
long iobase = pci_resource_start (pdev, bar);
Access the I/O region size:
long iosize = pci_resource_len (pdev, bar);
Reserve the I/O region:
request_region(iobase, iosize, “my driver”);
or simpler:
pci_request_region(pdev, bar, “my driver”);
or even simpler (regions for all BARs):
pci_request_regions(pdev, “my driver”);
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Accessing I/O registers and memory (2)
From drivers/net/ne2kpci.c (Linux 2.6.27):
ioaddr = pci_resource_start (pdev, 0);
irq = pdev>irq;
if (!ioaddr || ((pci_resource_flags (pdev, 0) & IORESOURCE_IO) == 0))
{
dev_err(&pdev>dev, "no I/O resource at PCI BAR #0\n");
return ENODEV;
}
if (request_region (ioaddr, NE_IO_EXTENT, DRV_NAME) == NULL) {
dev_err(&pdev>dev, "I/O resource 0x%x @ 0x%lx busy\n",
NE_IO_EXTENT, ioaddr);
return EBUSY;
}
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Setting the DMA mask size (1)
Use pci_dma_set_mask() to declare any device with more (or
less) than 32bit bus master capability
In particular, must be done by drivers for PCIX and PCIe
compliant devices, which use 64 bit DMA.
If the device can directly address "consistent memory" in System
RAM above 4G physical address, register this by calling
pci_set_consistent_dma_mask().
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Setting the DMA mask size (2)
Example (drivers/net/wireless/ipw2200.c in Linux 2.6.27):
err = pci_set_dma_mask(pdev, DMA_32BIT_MASK);
if (!err)
err = pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK);
if (err) {
printk(KERN_WARNING DRV_NAME ": No suitable DMA available.\n");
goto out_pci_disable_device;
}
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Allocate consistent DMA buffers
Now that the DMA mask size has been allocated...
You can allocate your cache consistent buffers
if you plan to use such buffers.
See our DMA presentation
and Documentation/DMAAPI.txt for details.
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Initialize device registers
If needed by the device
Set some “capability” fields
Do some vendor specific initialization or reset
Example: clear pending interrupts.
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Register interrupt handlers
Need to call request_irq() with the IRQF_SHARED flag,
because all PCI IRQ lines can be shared.
Registration also enables interrupts, so at this point
Make sure that the device is fully initialized and ready to
service interrupts.
Make sure that the device doesn't have any pending
interrupt before calling request_irq().
Where you actually call request_irq() can actually
depend on the type of device and the subsystem it could be
part of (network, video, storage...).
Your driver will then have to register to this subsystem.
29
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device shutdown (1)
In the remove() function, you typically have to undo
what you did at device initialization (probe() function):
Disable the generation of new interrupts.
If you don't, the system will get spurious interrupts, and will
eventually disable the IRQ line. Bad for other devices on this
line!
Release the IRQ
Stop all DMA activity.
Needed to be done after IRQs are disabled
(could start new DMAs)
Release DMA buffers: streaming first and then consistent
ones.
30
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PCI device shutdown
Unregister from other subsystems
Unmap I/O memory and ports with io_unmap().
Disable the device with pci_disable_device().
Unregister I/O memory and ports.
If you don't, you won't be able to reload the driver.
31
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful resources
Documentation/PCI/pci.txt in the kernel sources
An excellent guide to writing PCI drivers, which helped
us to write these slides.
Book: Essential Linux device drivers (Prentice Hall)
A very good and recent book!
http://freeelectrons.com/redirect/elddbook.html
Book: Linux Device Drivers (O'Reilly)
Available under a free documentation license.
http://freeelectrons.com/community/kernel/ldd3/
Wikipedia:
http://en.wikipedia.org/wiki/Peripheral_Component_Interconnect
32
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful resources (2)
PCI Utilities: http://mj.ucw.cz/pciutils.html
lspci: shows PCI bus and device information
setpci: allows to manipulate PCI device configuration
registers
PCI vendor and device id repository:
http://pciids.ucw.cz/read/PC/
33
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel initialization
Kernel
initialization
Michael Opdenacker
Free Electrons
© Copyright 20072009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelinit
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
From bootloader to userspace
Bootloader
Low level hardware
initialization Kernel init process
System initialization
Fetch and copy
Linux kernel
initialization from userspace
to RAM
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel bootstrap (1)
How the kernel bootstraps itself appears in kernel building.
Sharp Spitz example (arm pxa270 based):
...
LD vmlinux
SYSMAP System.map
SYSMAP .tmp_System.map
MODPOST vmlinux
Kernel: arch/arm/boot/Image is ready
AS arch/arm/boot/compressed/head.o
GZIP arch/arm/boot/compressed/piggy.gz
AS arch/arm/boot/compressed/piggy.o
CC arch/arm/boot/compressed/misc.o
AS arch/arm/boot/compressed/headxscale.o
AS arch/arm/boot/compressed/headsharpsl.o
LD arch/arm/boot/compressed/vmlinux
OBJCOPY arch/arm/boot/zImage
Kernel: arch/arm/boot/zImage is ready
...
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel bootstrap (2)
objcopy gzip as ld objcopy
piggy.S
piggy.o
Image
vmlinux piggy.gz
zImage
vmlinux
head.o
misc.o
Stripped Compressed headcpu.o
“Kernel proper”: kernel kernel
binary headboard.o
Raw kernel binary
executable (binary (in arch/<arch> Kernel image
object) boot/compressed) for bootloader
(ELF object) asm wrapper
around piggy.gz Composite (binary object)
+ bootstrap kernel image
Details found by compiling
code (ELF object)
with make V=1
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Bootstrap code
head.o:
Architecture specific initialization code.
This is what is executed by the bootloader
headcpu.o (here headxscale.o):
CPU specific initialization code
headboard.o (here headsharpsl.o):
Board specific initialization code
misc.o:
Decompression routines
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Bootstrap code tasks
Main work done by head.o:
Check the architecture, processor and machine type.
Configure the MMU, create page table entries
and enable virtual memory.
Calls the start_kernel function in init/main.c.
Same code for all architectures.
Anybody interesting in kernel startup should study this file!
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
start_kernel main actions
Calls setup_arch(&command_line)
(function defined in arch/<arch>/kernel/setup.c), copying
the command line from where the bootloader left it.
On arm, this function calls setup_processor
(in which CPU information is displayed) and setup_machine
(locating the machine in the list of supported machines).
Initializes the console as early as possible
(to get error messages)
Initializes many subsystems (see the code)
Eventually calls rest_init.
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
rest_init: starting the init process
Starting a new kernel thread which will later become the init process
static void noinline __init_refok rest_init(void)
__releases(kernel_lock)
{
int pid;
kernel_thread(kernel_init, NULL, CLONE_FS | CLONE_SIGHAND);
numa_default_policy();
pid = kernel_thread(kthreadd, NULL, CLONE_FS | CLONE_FILES);
kthreadd_task = find_task_by_pid(pid);
unlock_kernel();
/*
* The boot idle thread must execute schedule()
* at least one to get things moving:
*/
preempt_enable_no_resched();
schedule();
preempt_disable();
/* Call into cpu_idle with preempt disabled */
cpu_idle();
}
Source: Linux 2.6.22
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
kernel_init
kernel_init does 2 main things:
Call do_basic_setup
Now that kernel services are ready, start device initialization:
static void __init do_basic_setup(void)
{
/* drivers will send hotplug events */
init_workqueues();
usermodehelper_init();
driver_init();
init_irq_proc();
do_initcalls();
}
Call init_post
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
do_initcalls
Calls pluggable hooks registered with the macros below.
Advantage: the generic code doesn't have to know about them.
/*
* A "pure" initcall has no dependencies on anything else, and purely
* initializes variables that couldn't be statically initialized.
*
* This only exists for builtin code, not for modules.
*/
#define pure_initcall(fn) __define_initcall("0",fn,1)
#define core_initcall(fn) __define_initcall("1",fn,1)
#define core_initcall_sync(fn) __define_initcall("1s",fn,1s)
#define postcore_initcall(fn) __define_initcall("2",fn,2)
#define postcore_initcall_sync(fn) __define_initcall("2s",fn,2s)
#define arch_initcall(fn) __define_initcall("3",fn,3)
#define arch_initcall_sync(fn) __define_initcall("3s",fn,3s)
#define subsys_initcall(fn) __define_initcall("4",fn,4)
#define subsys_initcall_sync(fn) __define_initcall("4s",fn,4s)
#define fs_initcall(fn) __define_initcall("5",fn,5)
#define fs_initcall_sync(fn) __define_initcall("5s",fn,5s)
#define rootfs_initcall(fn) __define_initcall("rootfs",fn,rootfs)
#define device_initcall(fn) __define_initcall("6",fn,6)
#define device_initcall_sync(fn) __define_initcall("6s",fn,6s)
#define late_initcall(fn) __define_initcall("7",fn,7)
#define late_initcall_sync(fn) __define_initcall("7s",fn,7s)
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
initcall example
From arch/arm/machpxa/lpd270.c
static int __init lpd270_irq_device_init(void)
{
int ret = sysdev_class_register(&lpd270_irq_sysclass);
if (ret == 0)
ret = sysdev_register(&lpd270_irq_device);
return ret;
}
device_initcall(lpd270_irq_device_init);
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
init_post
The last step of Linux booting
First tries to open a console
Then tries to run the init process,
effectively turning the current kernel thread
into the userspace init process.
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
init_post code
static int noinline init_post(void)
{
free_initmem();
unlock_kernel();
mark_rodata_ro();
Source:
system_state = SYSTEM_RUNNING;
numa_default_policy(); init/main.c
if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0)
printk(KERN_WARNING "Warning: unable to open an initial console.\n"); in Linux 2.6.22
(void) sys_dup(0);
(void) sys_dup(0);
if (ramdisk_execute_command) {
run_init_process(ramdisk_execute_command);
printk(KERN_WARNING "Failed to execute %s\n",
ramdisk_execute_command);
}
/*
* We try each of these until one succeeds.
*
* The Bourne shell can be used instead of init if we are
* trying to recover a really broken machine.
*/
if (execute_command) {
run_init_process(execute_command);
printk(KERN_WARNING "Failed to execute %s. Attempting "
"defaults...\n", execute_command);
}
run_init_process("/sbin/init");
run_init_process("/etc/init");
run_init_process("/bin/init");
run_init_process("/bin/sh");
panic("No init found. Try passing init= option to kernel.");
}
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel initialization graph
System System
Bootloader
initialization operation
head.o
(bootstrap code)
cpu_idle
kernel_init
(idle loop)
start_kernel
rest_init init_post init process
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Kernel initialization What to remember
The bootloader executes Initializes the console.
bootstrap code. Initializes kernel services
Bootstrap code initializes (memory allocation,
the processor and board, scheduling, file cache...)
and uncompresses the Creates a new kernel thread
kernel code to RAM, and (future init process) and
calls the kernel's continues in the idle loop.
start_kernel function.
Initializes devices and
Copies the command line execute initcalls.
from the bootloader.
Identifies the processor and
machine.
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Serial drivers
Serial drivers
Thomas Petazzoni
Free Electrons
© Copyright 2009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/serialdrivers
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Architecture (1)
System call interface
Line Character driver Character driver
discipline TTY Core
TTY driver
TTY driver
serial_core
Serial driver
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Architecture (2)
To be properly integrated in a Linux system, serial ports must be
visible as TTY devices from userspace applications
Therefore, the serial driver must be part of the kernel TTY
subsystem
Until 2.6, serial drivers were implemented directly behind the TTY
core
A lot of complexity was involved
Since 2.6, a specialized TTY driver, serial_core, eases the
development of serial drivers
See include/linux/serial_core.h for the main definitions of
the serial_core infrastructure
The line discipline that cooks the data exchanged with the tty
driver. For normal serial ports, N_TTY is used.
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Data structures
A data structure representing a driver : uart_driver
Single instance for each driver
uart_register_driver() and uart_unregister_driver()
A data structure representing a port : uart_port
One instance for each port (several per driver are possible)
uart_add_one_port() and uart_remove_one_port()
A data structure containing the pointers to the operations :
uart_ops
Linked from uart_port through the ops field
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_driver
Usually
Defined statically in the driver
Registered in module_init()
Unregistered in module_cleanup()
Contains
owner, usually set to THIS_MODULE
driver_name
dev_name, the device name prefix, usually “ttyS”
major and minor
Use TTY_MAJOR and 64 to get the normal numbers. But they might
conflict with the 8250reserved numbers
nr, the maximum number of ports
cons, pointer to the console device (covered later)
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_driver code example (1)
static struct uart_driver atmel_uart = {
.owner = THIS_MODULE,
.driver_name = "atmel_serial",
.dev_name = ATMEL_DEVICENAME,
.major = SERIAL_ATMEL_MAJOR,
.minor = MINOR_START,
.nr = ATMEL_MAX_UART,
.cons = ATMEL_CONSOLE_DEVICE,
};
static struct platform_driver atmel_serial_driver = {
.probe = atmel_serial_probe,
.remove = __devexit_p(atmel_serial_remove),
.suspend = atmel_serial_suspend,
.resume = atmel_serial_resume,
.driver = {
.name = "atmel_usart",
.owner = THIS_MODULE,
},
};
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_driver code example (2)
static int __init atmel_serial_init(void)
{ Warning: error
uart_register_driver(&atmel_uart); management
platform_driver_register(&atmel_serial_driver);
removed!
return 0;
}
static void __exit atmel_serial_exit(void)
{
platform_driver_unregister(&atmel_serial_driver);
uart_unregister_driver(&atmel_uart);
}
module_init(atmel_serial_init);
module_exit(atmel_serial_exit);
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_port
Can be allocated statically or dynamically
Usually registered at probe() time and unregistered at
remove() time
Most important fields
iotype, type of I/O access, usually UPIO_MEM for memory
mapped devices
mapbase, physical address of the registers
irq, the IRQ channel number
membase, the virtual address of the registers
uartclk, the clock rate
ops, pointer to the operations
dev, pointer to the device (platform_device or other)
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_port code example (1)
static int __devinit atmel_serial_probe(struct platform_device *pdev)
{
struct atmel_uart_port *port;
port = &atmel_ports[pdev>id];
port>backup_imr = 0;
atmel_init_port(port, pdev);
uart_add_one_port(&atmel_uart, &port>uart);
platform_set_drvdata(pdev, port);
return 0;
}
static int __devexit atmel_serial_remove(struct platform_device *pdev)
{
struct uart_port *port = platform_get_drvdata(pdev);
platform_set_drvdata(pdev, NULL);
uart_remove_one_port(&atmel_uart, port);
return 0;
}
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_port code example (2)
static void __devinit atmel_init_port(struct atmel_uart_port *atmel_port,
struct platform_device *pdev)
{
struct uart_port *port = &atmel_port>uart;
struct atmel_uart_data *data = pdev>dev.platform_data;
port>iotype = UPIO_MEM;
port>flags = UPF_BOOT_AUTOCONF;
port>ops = &atmel_pops;
port>fifosize = 1;
port>line = pdev>id;
port>dev = &pdev>dev;
port>mapbase = pdev>resource[0].start;
port>irq = pdev>resource[1].start;
tasklet_init(&atmel_port>tasklet, atmel_tasklet_func,
(unsigned long)port);
[... see next page ...]
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_port code example (3)
[... continued from previous page ...]
if (data>regs)
/* Already mapped by setup code */
port>membase = data>regs;
else {
port>flags |= UPF_IOREMAP;
port>membase = NULL;
}
/* for console, the clock could already be configured */
if (!atmel_port>clk) {
atmel_port>clk = clk_get(&pdev>dev, "usart");
clk_enable(atmel_port>clk);
port>uartclk = clk_get_rate(atmel_port>clk);
clk_disable(atmel_port>clk);
/* only enable clock when USART is in use */
}
}
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
uart_ops
Important operations
tx_empty(), tells whether the transmission FIFO is empty or not
set_mctrl() and get_mctrl(), allow to set and get the modem
control parameters (RTS, DTR, LOOP, etc.)
start_tx() and stop_tx(), to start and stop the transmission
stop_rx(), to stop the reception
startup() and shutdown(), called when the port is
opened/closed
request_port() and release_port(), request/release I/O or
memory regions
set_termios(), change port parameters
See the detailed description in
Documentation/serial/driver
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Implementing transmission
The start_tx() method should start transmitting characters
over the serial port
The characters to transmit are stored in a circular buffer,
implemented by a struct uart_circ structure. It contains
buf[], the buffer of characters
tail, the index of the next character to transmit. After transmit, tail
must be updated using tail = tail & (UART_XMIT_SIZE
1)
Utility functions on uart_circ
uart_circ_empty(), tells whether the circular buffer is empty
uart_circ_chars_pending(), returns the number of characters
left to transmit
From an uart_port pointer, this structure can be reached using
port>info>xmit
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Polledmode transmission
foo_uart_putc(struct uart_port *port, unsigned char c) {
while(__raw_readl(port>membase + UART_REG1) & UART_TX_FULL)
cpu_relax();
__raw_writel(c, port>membase + UART_REG2);
}
foo_uart_start_tx(struct uart_port *port) {
struct circ_buf *xmit = &port>info>xmit;
while (!uart_circ_empty(xmit)) {
foo_uart_putc(port, xmit>buf[xmit>tail]);
xmit>tail = (xmit>tail + 1) & (UART_XMIT_SIZE 1);
port>icount.tx++;
}
}
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Transmission with interrupts (1)
foo_uart_interrupt(int irq, void *dev_id) {
[...]
if (interrupt_cause & END_OF_TRANSMISSION)
foo_uart_handle_transmit(port);
[...]
}
foo_uart_start_tx(struct uart_port *port) {
enable_interrupt_on_txrdy();
}
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Transmission with interrupts (2)
foo_uart_handle_transmit(port) {
struct circ_buf *xmit = &port>info>xmit;
if (uart_circ_empty(xmit) || uart_tx_stopped(port)) {
disable_interrupt_on_txrdy();
return;
}
while (! uart_circ_empty(xmit)) {
if (! (__raw_readl(port>membase + UART_REG1) &
UART_TX_FULL))
Break;
__raw_writel(xmit>buf[xmit>tail],
port>membase + UART_REG2);
xmit>tail = (xmit>tail + 1) & (UART_XMIT_SIZE 1);
port>icount.tx++;
}
if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
uart_write_wakeup(port);
}
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception
On reception, usually in an interrupt handler, the driver must
Increment port>icount.rx
Call uart_handle_break() if a BRK has been received, and if it
returns TRUE, skip to the next character
If an error occurred, increment port>icount.parity, port
>icount.frame, port>icount.overrun depending on the
error type
Call uart_handle_sysrq_char() with the received character,
and if it returns TRUE, skip to the next character
Call uart_insert_char() with the received character and a
status
Status is TTY_NORMAL is everything is OK, or TTY_BREAK,
TTY_PARITY, TTY_FRAME in case of error
Call tty_flip_buffer_push() to push data to the TTY later
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Understanding Sysrq
Part of the reception work is dedicated to handling Sysrq
Sysrq are special commands that can be sent to the kernel to make it
reboot, unmount filesystems, dump the task state, nice realtime tasks,
etc.
These commands are implemented at the lowest possible level so that
even if the system is locked, you can recover it.
Through serial port: send a BRK character, send the character of the
Sysrq command
See Documentation/sysrq.txt
In the driver
uart_handle_break() saves the current time + 5 seconds in a variable
uart_handle_sysrq_char() will test if the current time is below the
saved time, and if so, will trigger the execution of the Sysrq command
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception code sample (1)
foo_receive_chars(struct uart_port *port) {
int limit = 256;
while (limit > 0) {
status = __raw_readl(port>membase + REG_STATUS);
ch = __raw_readl(port>membase + REG_DATA);
flag = TTY_NORMAL;
if (status & BREAK) {
port>icount.break++;
if (uart_handle_break(port))
Continue;
}
else if (status & PARITY)
port>icount.parity++;
else if (status & FRAME)
port>icount.frame++;
else if (status & OVERRUN)
port>icount.overrun++;
[...]
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception code sample (2)
[...]
status &= port>read_status_mask;
if (status & BREAK)
flag = TTY_BREAK;
else if (status & PARITY)
flag = TTY_PARITY;
else if (status & FRAME)
flag = TTY_FRAME;
if (uart_handle_sysrq_char(port, ch))
continue;
uart_insert_char(port, status, OVERRUN, ch, flag);
}
spin_unlock(& port>lock);
tty_flip_buffer_push(port>info>port.tty);
spin_lock(& port>lock);
}
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Modem control lines
Set using the set_mctrl() operation
The mctrl argument can be a mask of TIOCM_RTS (request to
send), TIOCM_DTR (Data Terminal Ready), TIOCM_OUT1,
TIOCM_OUT2, TIOCM_LOOP (enable loop mode)
If a bit is set in mctrl, the signal must be driven active, if the bit is
cleared, the signal must be driven inactive
Status using the get_mctrl() operation
Must return read hardware status and return a combination of
TIOCM_CD (Carrier Detect), TIOCM_CTS (Clear to Send),
TIOCM_DSR (Data Set Ready) and TIOCM_RI (Ring Indicator)
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_mctrl() example
foo_set_mctrl(struct uart_port *uart, u_int mctrl) {
unsigned int control = 0, mode = 0;
if (mctrl & TIOCM_RTS)
control |= ATMEL_US_RTSEN;
else
control |= ATMEL_US_RTSDIS;
if (mctrl & TIOCM_DTS)
control |= ATMEL_US_DTREN;
else
control |= ATMEL_US_DTRDIS;
__raw_writel(port>membase + REG_CTRL, control);
if (mctrl & TIOCM_LOOP)
mode |= ATMEL_US_CHMODE_LOC_LOOP;
else
mode |= ATMEL_US_CHMODE_NORMAL;
__raw_writel(port>membase + REG_MODE, mode);
}
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
get_mctrl() example
foo_get_mctrl(struct uart_port *uart, u_int mctrl) {
unsigned int status, ret = 0;
status = __raw_readl(port>membase + REG_STATUS);
/*
* The control signals are active low.
*/
if (!(status & ATMEL_US_DCD))
ret |= TIOCM_CD;
if (!(status & ATMEL_US_CTS))
ret |= TIOCM_CTS;
if (!(status & ATMEL_US_DSR))
ret |= TIOCM_DSR;
if (!(status & ATMEL_US_RI))
ret |= TIOCM_RI;
return ret;
}
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
termios
“The termios functions describe a general terminal interface that
is provided to control asynchronous communications ports”
A mechanism to control from userspace serial port parameters
such as
Speed
Parity
Byte size
Stop bit
Hardware handshake
Etc.
See termios(3) for details
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_termios()
The set_termios() operation must
apply configuration changes according to the arguments
update port>read_config_mask and port
>ignore_config_mask to indicate the events we are interested
in receiving
static void set_termios(struct uart_port *port,
struct ktermios *termios, struct ktermios *old)
port, the port, termios, the new values and old, the old values
Relevant ktermios structure fields are
c_cflag with word size, stop bits, parity, reception enable, CTS
status change reporting, enable modem status change reporting
c_iflag with frame and parity errors reporting, break event
reporting
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_termios() example (1)
static void atmel_set_termios(struct uart_port *port, struct ktermios *termios,
struct ktermios *old)
{
unsigned long flags;
unsigned int mode, imr, quot, baud;
mode = __raw_readl(port>membase + REG_MODE);
baud = uart_get_baud_rate(port, termios, old, 0, port>uartclk / 16);
quot = uart_get_divisor(port, baud);
Read current
switch (termios>c_cflag & CSIZE) { configuration
case CS5:
mode |= ATMEL_US_CHRL_5;
break;
case CS6: Compute the
mode |= ATMEL_US_CHRL_6; mode
break; modification for
case CS7: the byte size
mode |= ATMEL_US_CHRL_7;
parameter
break;
default:
mode |= ATMEL_US_CHRL_8;
break;
}
[...]
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_termios() example (2)
[...]
if (termios>c_cflag & CSTOPB)
mode |= ATMEL_US_NBSTOP_2;
if (termios>c_cflag & PARENB) {
/* Mark or Space parity */
if (termios>c_cflag & CMSPAR) {
if (termios>c_cflag & PARODD)
mode |= ATMEL_US_PAR_MARK; Compute the mode
else modification for
mode |= ATMEL_US_PAR_SPACE; ● the stop bit
} else if (termios>c_cflag & PARODD) ● Parity
mode |= ATMEL_US_PAR_ODD; ● CTS reporting
else
mode |= ATMEL_US_PAR_EVEN;
} else
mode |= ATMEL_US_PAR_NONE;
if (termios>c_cflag & CRTSCTS)
mode |= ATMEL_US_USMODE_HWHS;
else
mode |= ATMEL_US_USMODE_NORMAL;
[...]
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_termios() example (3)
Compute the read_status_mask and
ignore_status_mask according to the events
we're interested in. These values are used in
[...] the interrupt handler.
port>read_status_mask = ATMEL_US_OVRE;
if (termios>c_iflag & INPCK)
port>read_status_mask |= (ATMEL_US_FRAME | ATMEL_US_PARE);
if (termios>c_iflag & (BRKINT | PARMRK))
port>read_status_mask |= ATMEL_US_RXBRK;
port>ignore_status_mask = 0;
if (termios>c_iflag & IGNPAR)
port>ignore_status_mask |= (ATMEL_US_FRAME | ATMEL_US_PARE);
if (termios>c_iflag & IGNBRK) {
port>ignore_status_mask |= ATMEL_US_RXBRK;
if (termios>c_iflag & IGNPAR)
port>ignore_status_mask |= ATMEL_US_OVRE;
}
uart_update_timeout(port, termios>c_cflag, baud);
[...]
The serial_core maintains a timeout that corresponds to
the duration it takes to send the full transmit FIFO. This
timeout has to be updated.
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
set_termios() example (4)
Finally, apply the mode and baud rate
modifications. Interrupts, transmission
[...]
and reception are disabled when the
/* Save and disable interupts */ modifications are made.
imr = UART_GET_IMR(port);
UART_PUT_IDR(port, 1);
/* disable receiver and transmitter */
UART_PUT_CR(port, ATMEL_US_TXDIS | ATMEL_US_RXDIS);
/* set the parity, stop bits and data size */
UART_PUT_MR(port, mode);
/* set the baud rate */
UART_PUT_BRGR(port, quot);
UART_PUT_CR(port, ATMEL_US_RSTSTA | ATMEL_US_RSTRX);
UART_PUT_CR(port, ATMEL_US_TXEN | ATMEL_US_RXEN);
/* restore interrupts */
UART_PUT_IER(port, imr);
/* CTS flowcontrol and modemstatus interrupts */
if (UART_ENABLE_MS(port, termios>c_cflag))
port>ops>enable_ms(port);
}
29
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Console
To allows early boot messages to be printed, the kernel provides
a separate but related facility: console
This console can be enabled using the console= kernel argument
The driver developer must
Implement a console_write() operation, called to print
characters on the console
Implement a console_setup() operation, called to parse the
console= argument
Declare a struct console structure
Register the console using a console_initcall() function
30
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Console: registration
static struct console serial_txx9_console = {
.name = TXX9_TTY_NAME, Helper function from the
.write = serial_txx9_console_write,
serial_core layer
.device = uart_console_device,
.setup = serial_txx9_console_setup,
.flags = CON_PRINTBUFFER, Ask for the kernel
.index = 1, messages buffered
.data = &serial_txx9_reg, during boot to be printed
}; to the console when
activated
static int __init serial_txx9_console_init(void)
{
register_console(&serial_txx9_console); This will make sure the
return 0; function is called early
} during the boot process.
console_initcall(serial_txx9_console_init);
start_kernel() calls
console_init() that calls
our function
31
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Console : setup
static int __init serial_txx9_console_setup(struct console *co, char *options)
{
struct uart_port *port;
struct uart_txx9_port *up;
int baud = 9600;
int bits = 8;
int parity = 'n';
int flow = 'n'; Function shared with the
normal serial driver
if (co>index >= UART_NR)
co>index = 0;
up = &serial_txx9_ports[co>index];
port = &up>port;
if (!port>ops)
return ENODEV; Helper function from
serial_core that parses
serial_txx9_initialize(&up>port); the console= string
if (options)
uart_parse_options(options, &baud, &parity, &bits, &flow);
return uart_set_options(port, co, baud, parity, bits, flow);
}
Helper function from serial_core that calls the
>set_termios() operation with the proper
arguments to configure the port
32
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Console : write
static void serial_txx9_console_putchar(struct uart_port *port, int ch)
{
struct uart_txx9_port *up = (struct uart_txx9_port *)port; Busywait for transmitter
wait_for_xmitr(up); ready and output a
sio_out(up, TXX9_SITFIFO, ch); single character.
}
static void serial_txx9_console_write( struct console *co,
const char *s, unsigned int count)
{
struct uart_txx9_port *up = &serial_txx9_ports[co>index];
unsigned int ier, flcr;
/* Disable interrupts
ier = sio_in(up, TXX9_SIDICR);
sio_out(up, TXX9_SIDICR, 0);
/* Disable flow control */
flcr = sio_in(up, TXX9_SIFLCR);
if (!(up>port.flags & UPF_CONS_FLOW) && (flcr & TXX9_SIFLCR_TES))
sio_out(up, TXX9_SIFLCR, flcr & ~TXX9_SIFLCR_TES);
uart_console_write(&up>port, s, count, serial_txx9_console_putchar);
/* Reenable interrupts */
wait_for_xmitr(up); Helper function from
sio_out(up, TXX9_SIFLCR, flcr); serial_core that
sio_out(up, TXX9_SIDICR, ier); repeatedly calls the
} given putchar()
callback
33
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Serial drivers
Improve the character driver of
the previous labs to make it a
real serial driver
34
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Network drivers
Network drivers
Thomas Petazzoni
Free Electrons
© Copyright 2009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/networkdrivers
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Architecture
Network stack
sk_buff net_device
Network hardware driver
Bus infrastructure
(platform, pci, usb, etc.)
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
sk_buff
The struct sk_buff is the structure representing a network
packet
Designed to easily support encapsulation/decapsulation of data
through the protocol layers
In addition to the data itself, an sk_buff maintains
head, the start of the packet
data, the start of the packet payload
tail, the end of the packet payload
end, the end of the packet
len, the amount of data of the packet
These fields are updated when the packet goes through the
protocol layers
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Allocating a SKB
Function dev_alloc_skb() allows to allocate an SKB
Can be called from an interrupt handler.
Usually the case on reception.
On Ethernet, the size allocated is usually the length of the packet
+ 2, so that the IP header is wordaligned (the Ethernet header is
14 bytes)
skb = dev_alloc_skb(length + NET_IP_ALIGN);
data, head, tail
length +
NET_IP_ALIGN
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reserving space in a SKB
Need to skip NET_IP_ALIGN bytes at the beginning of the SKB
Done with skb_reserve()
skb_reserve(skb, NET_IP_ALIGN);
head
data, tail NET_IP_ALIGN
length
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Copy the received data
The packet payload must be copied from the DMA buffer to the
SKB, using
static inline void skb_copy_to_linear_data(struct sk_buff
*skb, const void *from, const unsigned int len);
static inline void skb_copy_to_linear_data_offset(struct
sk_buff *skb, const int offset, const void *from, const
unsigned int len);
skb_copy_to_linear_data(skb, dmabuffer,
length);
head
data, tail NET_IP_ALIGN
packet
length
data
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Update pointers in SKB
skb_put() is used to update the SKB pointers after copying the
payload
skb_put(skb, length);
head
data NET_IP_ALIGN
packet
length
data
tail
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
struct net_device
This structure represents a single network interface
Allocation takes place with alloc_etherdev()
The size of private data must be passed as argument. The pointer
to these private data can be read in net_device>priv
alloc_etherdev() is a specialization of alloc_netdev() for
Ethernet interfaces
Registration with register_netdev()
Unregistration with unregister_netdev()
Liberation with free_netdev()
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
struct net_device_ops
The methods of a network interface. The most important ones:
ndo_open(), called when the network interface is up'ed
ndo_close(), called when the network interface is down'ed
ndo_start_xmit(), to start the transmission of a packet
And others:
ndo_get_stats(), to get statistics
ndo_do_ioctl(), to implement device specific operations
ndo_set_rx_mode(), to select promiscuous, multicast, etc.
ndo_set_mac_address(), to set the MAC address
ndo_set_multicast_list(), to set multicast filters
Set the netdev_ops field in the struct net_device structure
to point to the struct net_device_ops structure.
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Utility functions
netif_start_queue()
Tells the kernel that the driver is ready to send packets
netif_stop_queue()
Tells the kernel to stop sending packets. Useful at driver cleanup of
course, but also when all transmission buffers are full.
netif_queue_stopped()
Tells whether the queue is currently stopped or not
netif_wake_queue()
Wakeup a queue after a netif_stop_queue().
The kernel will resume sending packets
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Transmission
The driver implements the ndo_start_xmit() operation
The kernel calls this operation with a SKB as argument
The driver sets up DMA buffers and other hardwaredependent
mechanisms and starts the transmission
Depending on the number of free DMA buffers available, the driver
can also stop the queue with netif_stop_queue()
When the packet has been sent, an interrupt is raised. The driver
is responsible for
Acknowledging the interrupt
Freeing the used DMA buffers
Freeing the SKB with dev_kfree_skb_irq()
If the queue was stopped, start it again
Returns NETDEV_TX_OK or NETDEV_TX_BUSY
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception: original mode
Reception is notified by an interrupt. The interrupt handler should
Allocate an SKB with dev_alloc_skb()
Reserve the 2 bytes offset with skb_reserve()
Copy the packet data from the DMA buffers to the SKB
skb_copy_to_linear_data() or
skb_copy_to_linear_data_offset()
Update the SKB pointers with skb_put()
Update the skb>protocol field with eth_type_trans(skb,
netdevice)
Give the SKB to the kernel network stack with netif_rx(skb)
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception: NAPI mode (1)
The original mode is nice and simple, but when the network traffic is
high, the interrupt rate is high. The NAPI mode allows to switch to
polled mode when the interrupt rate is too high.
In the network interface private structure, add a struct
napi_struct
At driver initialization, register the NAPI poll operation:
netif_napi_add(dev, &bp>napi, macb_poll, 64);
dev is the network interface
&bp>napi is the struct napi_struct
macb_poll is the NAPI poll operation
64 is the «weight» that represents the importance of the network
interface. It limits the number of packets each interface can feed to the
networking core in each polling cycle. If this quota is not met, the driver
will return back to interrupt mode. Don't send this quota to a value
greater than the number of packets the interface can store. 13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reception: NAPI mode (2)
In the interrupt handler, when a packet has been received:
if (napi_schedule_prep(&bp>napi)) {
/* Disable reception interrupts */
__napi_schedule(& bp>napi);
}
The kernel will call our poll() operation regularly
The poll() operation has the following prototype
static int macb_poll(struct napi_struct *napi, int budget)
It must receive at most budget packets and push them to the
network stack using netif_receive_skb().
If less than budget packets have been received, switch back to
interrupt mode using napi_complete(& bp>napi) and re
enable interrupts
Must return the number of packets received
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Communication with the PHY (1)
Usually, on embedded platforms, the SoC contains the Ethernet
controller, that takes care of layer 2 (MAC) communication.
An external PHY is responsible for layer 1 communication.
The MAC and the PHY are connected using a MII or RMII
interface
MII = Media Independent Interface
RMII = Reduced Media Independent Interface
This interface contains two wires used for the MDIO bus
(Management Data Input/Output)
The Ethernet driver needs to communicate with the PHY to get
information about the link (up, down, speed, full or half duplex)
and configure the MAC accordingly
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Communication with the PHY (2)
SoC (R)MII
Ethernet
MAC PHY connector
MDIO
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PHY in the kernel
The kernel provides a framework that
Exposes an API to communicate with the PHY
Allows to implement PHY drivers
Implements a basic generic PHY driver that works with all PHY
Implemented in drivers/net/phy/
Documented in Documentation/networking/phy.txt
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
MDIO bus initialization
The driver must create a MDIO bus structure that tells the PHY
infrastructure how to communicate with the PHY.
Allocate a MDIO bus structure
struct mii_bus *mii_bus = mdiobus_alloc();
Fill the MDIO bus structure
mii_bus>name = “foo”
mii_bus>read = foo_mii_bus_read,
mii_bus>write = foo_mii_bus_write,
snprintf(mii_bus>id, MII_BUS_ID_SIZE, "%x", pdev>id);
mii_bus>parent = struct net_device *
The foo_mii_bus_read() and foo_mii_bus_write() are
operations to read and write a value to the MDIO bus. They are
hardware specific and must be implemented by the driver.
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
MDIO bus initialization (2)
The >irq[] array must be allocated and initialized. To use
polling, set the values to PHY_POLL.
mii_bus>irq = kmalloc(sizeof(int)*PHY_MAX_ADDR, GFP_KERNEL);
for (i = 0; i < PHY_MAX_ADDR; i++)
bp>mii_bus>irq[i] = PHY_POLL;
Finally, register the MDIO bus. This will scan the bus for PHYs
and fill the mii_bus>phy_map[] array with the result.
mdiobus_register(bp>mii_bus)
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Connection to the PHY
The mdiobus_register() function filled the mii_bus
>phy_map[] array with struct phy_device * pointers
The appropriate PHY (usually, only one is detected) must be
selected
Then, connecting to the PHY allows to register a callback that will
be called when the link changes :
int phy_connect_direct(
struct net_device *dev,
struct phy_device *phydev,
void (*handler)(struct net_device *),
u32 flags,
phy_interface_t interface
)
interface is usually PHY_INTERFACE_MODE_MII or
PHY_INTERFACE_MODE_RMII
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Updating MAC capabilities
The MAC and the PHY might have different capabilities. Like a
PHY handling Gigabit speed, but not the MAC
The driver is responsible for updating phydev>advertise and
phydev>supported to remove any PHY capability that the
MAC doesn't support
A typical solution for a 10/100 controller is
phydev>supported &= PHY_BASIC_FEATURES
phydev>advertising = phydev>supported
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Handling link changes
The callback that handle link changes should have the following
prototype
void foo_handle_link_change(struct net_device *dev)
It must check the duplex, speed and link fields of the struct
phy_device structure, and update the Ethernet controller
configuration accordingly
duplex is either DUPLEX_HALF or DUPLEX_FULL
speed is either SPEED_10, SPEED_100, SPEED_1000,
SPEED_2500 or SPEED_10000
link is a boolean
22
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Starting and stopping the PHY
After set up, the PHY driver doesn't operate. To make it poll
regularly the PHY hardware, one must start it with
phy_start(phydev)
And when the network is stopped, the PHY must also be
stopped, using
phy_stop(phydev)
23
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
ethtool
ethtool is a userspace tool that allows to query lowlevel
information from an Ethernet interface and to modify its
configuration
On the kernel side, at the driver level, a struct ethtool_ops
structure can be declared and connected to the struct
net_device using the ethtool_ops field.
List of operations: get_settings(), set_settings(),
get_drvinfo(), get_wol(), set_wol(), get_link(),
get_eeprom(), set_eeprom(), get_tso(),
set_tso(), get_flags(), set_flags(), etc.
Some of these operations can be implemented using the PHY
interface (phy_ethtool_gset(), phy_ethtool_sset()) or
using generic operations (ethtool_op_get_link() for
example)
24
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Statistics
The network driver is also responsible for keeping statistics up to
date about the number of packets/bytes received/transmitted, the
number of errors, of collisions, etc.
Collecting these informations is left to the driver
To expose these information, the driver must implement a
get_stats() operation, with the following prototype
struct net_device_stats *foo_get_stats
(struct net_device *dev);
The net_device_stats structure must be filled with the driver.
It contains fields such as rx_packets, tx_packets,
rx_bytes, tx_bytes, rx_errors, tx_errors,
rx_dropped, tx_dropped, multicast, collisions,
etc.
25
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Power management
To support suspend and resume, the network driver must
implement the suspend() and resume() operations
These operations are referenced by the xxx_driver structure
corresponding to the bus on which the Ethernet controller is
The suspend() operation should
Call netif_device_detach()
Do the hardwaredependent operations to suspend the devices (like
disable the clocks)
The resume() operation should
Do the hardwaredependent operations (like enable the clocks)
Call netif_device_attach()
26
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
References
«Essential Linux Device Drivers», chapter 15
«Linux Device Drivers», chapter 17 (a little bit old)
Documentation/networking/netdevices.txt
Documentation/networking/phy.txt
include/linux/netdevice.h,
include/linux/ethtool.h, include/linux/phy.h,
include/linux/sk_buff.h
And of course, drivers/net/ for several examples of drivers
Driver code templates in the kernel sources:
drivers/usb/usbskeleton.c
drivers/net/isaskeleton.c
drivers/net/pciskeleton.c
drivers/pci/hotplug/pcihp_skeleton.c
27
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Network drivers
Implement a working network
driver for the MACB Ethernet
controller of the AT91SAM9263
CPU
28
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Porting the Linux kernel to an ARM board
Porting the Linux
kernel
to an ARM board
Thomas Petazzoni
Free Electrons
© Copyright 20092010, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelporting
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Porting the Linux kernel
The Linux kernel supports a lot of different CPU architectures
Each of them is maintained by a different group of contributors
See the MAINTAINERS file for details
The organization of the source code and the methods to port the
Linux kernel to a new board are therefore very architecture
dependent
For example, PowerPC and ARM are very different
PowerPC relies on device trees to describe hardware details
ARM relies on source code only
This presentation is focused on the ARM architecture only
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Architecture, CPU and machine
In the source tree, each architecture has its own directory
arch/arm for the ARM architecture
This directory contains generic ARM code
boot, common, configs, kernel, lib, mm, nwfpe,
vfp, oprofile, tools
And many directories for different CPU families
mach* directories : machpxa for PXA CPUs, machimx for
Freescale iMX CPUs, etc.
Each of these directories contain
Support for the CPU
Support for several boards using this CPU
Some CPU types share some code, in an entity called a platform
platomap contains common code from machomap1 and mach
omap2 3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Source code for Calao USB A9263
Taking the case of the Calao USB A9263 board, which uses a
AT91SAM9263 CPU.
arch/
arm/
machat91/
AT91 generic code
clock.c, leds.c, irq.c, pm.c
CPUspecific code for the AT91SAM9263
at91sam9263.c, at91sam926x_time.c,
at91sam9263_devices.c
Board specific code
boardusba9263.c
For the rest of this presentation, we will focus on board support
only
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Configuration
A configuration option must be defined for the board, in
arch/arm/machat91/Kconfig
config MACH_USB_A9263
bool "CALAO USBA9263"
depends on ARCH_AT91SAM9263
help
Select this if you are using a Calao Systems USBA9263.
<http://www.calaosystems.com>
This option must depend on the CPU type option corresponding
to the CPU used in the board
Here the option is ARCH_AT91SAM9263, defined in the same file
A default configuration file for the board can optionally be stored
in arch/arm/configs/. For our board, it's usb
a9263_defconfig
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Compilation
The source files corresponding to the board support must be
associated with the configuration option of the board
This is done in arch/arm/machat91/Makefile
obj$(CONFIG_MACH_USB_A9263) += boardusba9263.o
The Makefile also tells which files are compiled for every AT91
CPU
objy := irq.o gpio.o
obj$(CONFIG_AT91_PMC_UNIT) += clock.o
objy += leds.o
obj$(CONFIG_PM) += pm.o
obj$(CONFIG_AT91_SLOW_CLOCK) += pm_slowclock.o
And which files for our particular CPU, the AT91SAM9263
obj$(CONFIG_ARCH_AT91SAM9263) += at91sam9263.o at91sam926x_time.o
at91sam9263_devices.o sam9_smc.o
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Machine structure
Each board is defined by a machine structure
The word « machine » is quite confusing since every mach*
directory contains several machine definitions, one for each board
using a given CPU type
For the Calao board, at the end of arch/arm/mach
at91/boardusba9263.c
MACHINE_START(USB_A9263, "CALAO USB_A9263")
/* Maintainer: calaosystems */
.phys_io = AT91_BASE_SYS,
.io_pg_offst = (AT91_VA_BASE_SYS >> 18) & 0xfffc,
.boot_params = AT91_SDRAM_BASE + 0x100,
.timer = &at91sam926x_timer,
.map_io = ek_map_io,
.init_irq = ek_init_irq,
.init_machine = ek_board_init,
MACHINE_END
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Machine structure macros
MACHINE_START and MACHINE_END
Macros defined in arch/arm/include/asm/mach/arch.h
They are helpers to define a struct machine_desc structure
stored in a specific ELF section
Several machine_desc structures can be defined in a kernel,
which means that the kernel can support several boards.
The right structure is chosen at boot time
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Machine type number
In the ARM architecture, each board type is identified by a
machine type number
The latest machine type numbers list can be found at
http://www.arm.linux.org.uk/developer/machines/download.php
A copy of it exists in the kernel tree in arch/arm/tools/mach
types
For the Calao board
usb_a9263 MACH_USB_A9263 USB_A9263 1710
At compile time, this file is processed to generate a header file,
include/asmarm/machtypes.h
For the Calao board
#define MACH_TYPE_USB_A9263 1710
And a few other macros in the same file
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Machine type number
The machine type number is set in the MACHINE_START()
definition
MACHINE_START(USB_A9263, "CALAO USB_A9263")
At run time, the machine type number of the board on which the
kernel is running is passed by the bootloader in register r1
Very early in the boot process (arch/arm/kernel/head.S),
the kernel calls __lookup_machine_type in
arch/arm/kernel/headcommon.S
__lookup_machine_type looks at all the machine_desc
structures of the special ELF section
If it doesn't find the requested number, prints a message and stops
If found, it knows the machine descriptions and continues the boot
process
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Early debugging and boot parameters
Early debugging
phys_io is the physical address of the I/O space
io_pg_offset is the offset in the page table to remap the I/O
space
These are used when CONFIG_DEBUG_LL is enabled to provide
very early debugging messages on the serial port
Boot parameters
boot_params is the location where the bootloader has left the boot
parameters (the kernel command line)
The bootloader can override this address in register r2
See also Documentation/arm/Booting for the details of the
environment expected by the kernel when booted
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
System timer
The timer field point to a struct sys_timer structure, that
describes the system timer
Used to generate the periodic tick at HZ frequency to call the
scheduler periodically
On the Calao board, the system timer is defined by the
at91sam926x_timer structure in at91sam926x_time.c
It contains the interrupt handler called at HZ frequency
It is integrated with the clockevents and the clocksource
infrastructures
See include/linux/clocksource.h and
include/linux/clockchips.h for details
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
map_io()
The map_io() function points to ek_map_io(), which
Initializes the CPU using at91sam9263_initialize()
Map I/O space
Register and initialize the clocks
Configures the debug serial port and set the console to be on this
serial port
Called at the very beginning of the C code execution
init/main.c: start_kernel()
arch/arm/kernel/setup.c: setup_arch()
arch/arm/mm/mmu.c: paging_init()
arch/arm/mm/mmu.c: devicemaps_init()
mdesc>map_io()
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
init_irq()
init_irq() to initialize the IRQ hardware specific details
Implemented by ek_init_irq(), which calls
at91sam9263_init_interrupts() in at91sam9263.c,
which mainly calls at91_aic_init() in irq.c
Initialize the interrupt controller, assign the priorities
Register the IRQ chip (irq_chip structure) to the kernel generic
IRQ infrastructure, so that the kernel knows how to ack, mask,
unmask the IRQs
Called a little bit later than map_io()
init/main.c: start_kernel()
arch/arm/kernel/irq.c: init_IRQ()
init_arch_irq() (equal to mdesc>init_irq)
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
init_machine()
init_machine() completes the initialization of the board by
registering all platform devices
Called by customize_machines() in
arch/arm/kernel/setup.c
This function is an arch_initcall (list of functions whose
address is stored in a specific ELF section, by levels)
At the end of kernel initialization, just before running the first
userspace program init:
init/main.c: kernel_init()
init/main.c: do_basic_setup()
init/main.c: do_initcalls()
Calls all initcalls, level by level
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
init_machine() for Calao
For the Calao board, implement in ek_board_init()
Registers serial ports, USB host, USB device, SPI, Ethernet, NAND
flash, 2IC, buttons and LEDs
Uses at91_add_device_*() helpers, defined in
at91sam9263_devices.c
These helpers call platform_device_register() to register
the different platform_device structures defined in the same file
For some devices, the board specific code does the registration
itself (buttons) or passes boardspecific data to the registration
helper (USB host and device, NAND, Ethernet, etc.)
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Drivers
The at91sam9263_devices.c file doesn't implement the
drivers for the platform devices
The drivers are implemented at different places of the kernel tree
For the Calao board
USB host, driver at91_ohci, drivers/usb/host/ohciat91.c
USB device, driver at91_udc, drivers/usb/gadget/at91_udc.c
Ethernet, driver macb, drivers/net/macb.c
NAND, driver atmel_nand, drivers/mtd/nand/atmel_nand.c
I2C on GPIO, driver i2cgpio, drivers/i2c/busses/i2cgpio.c
SPI, driver atmel_spi, drivers/spi/atmel_spi.c
Buttons, driver gpiokeys, drivers/input/keyboard/gpio_keys.c
All these drivers are selected by the readymade configuration file
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Power management
Power
Management
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 20072009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/power
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
General observations
Power management development efforts in the Linux kernel
originally very PC centric (using ACPI or APM).
However, there is now substantial work to improve power
management in the embedded Linux world as well.
For example, it is now possible to implement suspend to disk and
even to flash (“hibernating”) wherever suspend to RAM is
available.
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PM building blocks
Several power management « building blocks »
Suspend and resume
CPUidle
Frequency and voltage scaling
Applications
Independent « building blocks » that can be improved gradually
during development
Presentation and « building blocks » term inspired from Kevin
Hilman talks on power management:
http://www.celinux.org/elc08_presentations/PM_Building_Blocks1.pdf
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Clock framework (1)
Generic framework to manage clocks used by devices in the
system
Allows to reference count clock users
and to shutdown the unused clocks to save power
Simple API described in
http://freeelectrons.com/kerneldoc/latest/DocBook/kernelapi/ch17.html
clk_get() to get a reference to a clock
clk_enable() to start the clock
clk_disable() to stop the clock
clk_put() to free the clock source
clk_get_rate() to get the current rate
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Clock framework (2)
The clock framework API and the clk structure are usually
implemented by each architecture
See arch/arm/machat91/clock.c for an example
Clocks are identified by a name string specific to a given platform
Drivers can then use the clock API.
Example from drivers/net/macb.c:
clk_get() called when device is probed,
clk_put() when removed
clk_enable() and clk_disable()
on power management notifications
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Clock disable implementation
From arch/arm/machat91/clock.c: (2.6.32)
static void __clk_disable(struct clk *clk)
{
BUG_ON(clk>users == 0);
if (clk>users == 0 && clk>mode)
clk>mode(clk, 0);
if (clk>parent)
__clk_disable(clk>parent);
} Call the hardware function
Example mode function (same file): switching off this clock
static void pmc_sys_mode(struct clk *clk, int is_on)
{
if (is_on)
at91_sys_write(AT91_PMC_SCER, clk>pmc_mask);
else
at91_sys_write(AT91_PMC_SCDR, clk>pmc_mask);
}
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Suspend and resume
Infrastructure in the kernel to support suspend and resume
Platform hooks
prepare(), enter(), finish(), valid()
in a platform_suspend_ops structure
Registered using the suspend_set_ops() function
See arch/arm/machat91/pm.c
Device drivers
suspend() and resume() hooks in the *_driver structures
(platform_driver, usb_driver, etc.)
See drivers/net/macb.c
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Board specific power management
Typically takes care of battery and charging management.
Also defines presuspend and postsuspend handlers.
Example:
arch/arm/machpxa/spitz_pm.c
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
arch/arm/mach<cpu>/sleep.S
Assembly code implementing CPU specific
suspend and resume code. Note: only found on arm, just 3 other
occurrences in other architectures, with other paths.
First scenario: only a suspend function. The code goes in sleep
state (after enabling DRAM selfrefresh), and continues with
resume code.
Second scenario: suspend and resume functions.
Resume functions called by the bootloader.
Examples to look at:
arch/arm/machomap2/sleep.S (1st case)
arch/arm/machpxa/sleep.S (2nd case)
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Triggering suspend
Whatever the power management implementation, CPU
specific suspend_ops functions are called by the
enter_state function.
enter_state also takes care of executing the suspend
and resume functions for your devices.
The execution of this function can be triggered from
userspace. To suspend to RAM:
echo mem > /sys/power/state
Can also use the s2ram program
from http://suspend.sourceforge.net/
Read kernel/power/suspend.c
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Saving power in the idle loop
The idle loop is what you run
when there's nothing left to run in the system.
Implemented in all architectures
in arch/<arch>/kernel/process.c
Example to read: look for cpu_idle in
arch/arm/kernel/process.c
Each ARM cpu defines its own arch_idle function.
The CPU can run power saving HLT instructions, enter NAP
mode, and even disable the timers (tickless systems).
See also http://en.wikipedia.org/wiki/Idle_loop
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Managing idle
Adding support for multiple idle levels
Modern CPUs have several sleep states offering different power savings
with associated wake up latencies
Since 2.6.21, the dynamic tick feature allows to remove the periodic tick
to save power, and to know when the next event is scheduled, for smarter
sleeps.
CPUidle infrastructure to change sleep states
Platformspecific driver defining sleep states and transition operations
Platformindependent governors (ladder and menu)
Added in 2.6.24 for x86/ACPI, work in progress on various ARM cpus.
(look for cpuidle* files under arch/arm/)
See Documentation/cpuidle/ in kernel sources.
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PowerTOP
http://www.lesswatts.org/projects/powertop/
With dynamic tick, new interest in fixing the kernel code and
applications that wake up the system too often
PowerTOP application allows to track the worst offender
Experimental availability on arm (OMAP cpus)
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Frequency and voltage scaling (1)
Frequency and voltage scaling possible through the cpufreq kernel infrastructure.
Generic infrastructure
drivers/cpufreq/cpufreq.c
include/linux/cpufreq.h
Generic governors, responsible for deciding frequency and voltage transitions
performance: maximum frequency
powersave: minimum frequency
ondemand: measures CPU consumption to adjust frequency
conservative: often better than ondemand.
Only increases frequency gradually when the CPU gets loaded.
userspace: leaves the decision to an userspace daemon.
This infrastructure can be controlled from
/sys/devices/system/cpu/cpu<n>/cpufreq/
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Frequency and voltage scaling (2)
CPU support code in architecture dependent files.
Example to read: arch/arm/platomap/cpuomap.c
Must implement the operations of the cpufreq_driver structure
and register them using cpufreq_register_driver()
init() for initialization
exit() for cleanup
verify() to verify the userchosen policy
setpolicy() or target()
to actually perform the frequency change
etc.
See Documentation/cpufreq/ for useful explanations
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
PM QoS
PM QoS is a framework developed by Intel introduced in 2.6.25
It allows kernel code and applications to set their requirements
in terms of
CPU DMA latency
Network latency
Network throughput
According to these requirements, PM QoS allows kernel drivers
to adjust their power management
See Documentation/power/pm_qos_interface.txt
and Mark Gross' presentation at ELC 2008
Still in very early deployment (only 4 drivers in 2.6.32).
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Regulator framework
Modern embedded hardware have hardware
responsible for voltage and current regulation
The regulator framework allows to take advantage of this
hardware to save power when parts of the system are unused
A consumer interface for device drivers (i.e users)
Regulator driver interface for regulator drivers
Machine interface for board configuration
sysfs interface for userspace
Merged in Linux 2.6.27.
See Documentation/power/regulator/ in kernel sources.
See Liam Girdwood's presentation at ELC 2008
http://freeelectrons.com/blog/elc2008report#girdwood
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
BSP work for a new board
In case you just need to create a BSP for your board, and your CPU
already has full PM support, you should just need to:
Create clock definitions and bind your devices to them.
Implement PM handlers (suspend, resume) in the drivers for your
board specific devices.
Implement board specific power management if needed
(mainly battery management)
Implement regulator framework hooks for your board if needed.
All other parts of the PM infrastructure should be already there:
suspend / resume, cpuidle, cpu frequency and voltage scaling.
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful resources
Documentation/power/ in the Linux kernel sources.
Will give you many useful details.
http://lesswatts.org
Intel effort trying to create a Linux power saving community.
Mainly targets Intel processors.
Lots of useful resources.
http://freeelectrons.com/redirect/elc2007suspend.html
ELC 2007, Toshiba: indepth review of their Suspend to
RAM implementation on Freescale 74xx, without PMU.
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Power management
Suspend and resume your
Linux system
Change the CPU frequency of
your system
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Free Electrons
Kernel
advice and
resources
Michael Opdenacker
Thomas Petazzoni
Free Electrons
© Copyright 2009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/kernelresources
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Solving issues
If you face an issue, and it doesn't look specific to your work
but rather to the tools you are using, it is very likely that
someone else already faced it.
Search the Internet for similar error reports.
You have great chances of finding a solution or workaround,
or at least an explanation for your issue.
Otherwise, reporting the issue is up to you!
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Getting help
If you have a support contract, ask your vendor.
Otherwise, don't hesitate to share your questions and issues
Either contact the Linux mailing list for your architecture (like linux
armkernel or linuxshdev...).
Or contact the mailing list for the subsystem you're dealing with
(linuxusbdevel, linuxmtd...). Don't ask the maintainer directly!
Most mailing lists come with a FAQ page. Make sure you read it
before contacting the mailing list.
Useful IRC resources are available too
(for example on http://kernelnewbies.org).
Refrain from contacting the Linux Kernel mailing list, unless you're
an experienced developer and need advice.
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Getting contributions
Applies if your project can interest other people:
developing a driver or filesystem, porting Linux on a new
processor, board or device available on the market...
External contributors can help you a lot by
Testing
Writing documentation
Making suggestions
Even writing code
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Encouraging contributions
Open your development process: mailing list, Wiki, public
CVS read access
Let everyone contribute according to their skills and
interests.
Release early, release often
Take feedback and suggestions into account
Recognize contributions
Make sure status and documentation are up to date
Publicize your work and progress to broader audiences
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Reporting Linux bugs
First make sure you're using the latest version
Make sure you investigate the issue as much as you can:
see Documentation/BUGHUNTING
Make sure the bug has not been reported yet. A bug tracking
system
(http://bugzilla.kernel.org/) exists but very few kernel developers
use it. Best to use web search engines (accessing public mailing
list archives)
If the subsystem you report a bug on has a mailing list, use it.
Otherwise, contact the official maintainer (see the MAINTAINERS
file). Always give as many useful details as possible.
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to submit patches or drivers
Don't merge patches addressing different issues
You should identify and contact the official maintainer
for the files to patch.
See Documentation/SubmittingPatches for details. For trivial
patches, you can copy the Trivial Patch Monkey.
See also http://kernelnewbies.org/UpstreamMerge for very helpful
advice to have your code merged upstream (by Rik van Riel).
Patches related to embedded systems can be submitted on the linux
embedded mailing list:
http://vger.kernel.org/vgerlists.html#linuxembedded
ARM platform: it's best to submit your ARM patches to Russell King's
patch system: http://www.arm.linux.org.uk/developer/patches/
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
How to become a kernel developer?
Greg KroahHartman wrote useful guidelines in the kernel
documentation:
Documentation/HOWTO
How to Participate in the Linux Community (by Jonathan Corbet)
A Guide To The Kernel Development Process
http://ldn.linuxfoundation.org/documentation/howparticipatelinuxcommunity
Read this last document first.
It is probably sufficient!
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Advice and resources
References
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Specific training materials
More training materials available on http://freeelectrons.com/docs:
New features in 2.6
Kernel initialization
Porting Linux to new hardware
Power management
USB device drivers
Block drivers
More will be available in the next months.
Don't hesitate to ask us to create the ones you need for a training
session!
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Information sites (1)
Linux Weekly News
http://lwn.net/
The weekly digest off all Linux and free software information
sources
In depth technical discussions about the kernel
Subscribe to finance the editors ($5 / month)
Articles available for non subscribers
after 1 week.
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Information sites (2)
Kernel Podcast
http://www.kernelpodcast.org/
Daily podcast about Linux kernel development
Load it on your media player!
KernelTrap
http://kerneltrap.org/
Forum website for kernel developers
News, articles, whitepapers, discussions, polls, interviews
Perfect if a digest is not enough!
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful reading (1)
Essential Linux Device Drivers, April 2008
http://freeelectrons.com/redirect/elddbook.html
By Sreekrishnan Venkateswaran, an embedded IBM
engineer with more than 10 years of experience
Covers a wide range of topics not covered by LDD :
serial drivers, input drivers, I2C, PCMCIA and
Compact Flash, PCI, USB, video drivers, audio
drivers, block drivers, network drivers, Bluetooth,
IrDA, MTD, drivers in userspace, kernel debugging,
etc.
« Probably the most wide ranging and complete
Linux device driver book I've read » Alan Cox
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful reading (2)
Writing Linux Device drivers, September 2009
http://www.coopj.com/
Self published by Jerry Cooperstein
Available like any other book (Amazon and others)
Though not as thorough as the previous book on
specific drivers, still a good complement on multiple
aspects of kernel and device driver development.
Based on Linux 2.6.31
Multiple exercises. Solutions available.
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful reading (3)
Linux Device Drivers, 3rd edition, Feb 2005
By Jonathan Corbet, Alessandro Rubini,
Greg KroahHartman, O'Reilly
http://www.oreilly.com/catalog/linuxdrive3/
Freely available online!
Great companion to the printed book
for easy electronic searches!
http://lwn.net/Kernel/LDD3/ (1 PDF file per chapter)
http://freeelectrons.com/community/kernel/ldd3/ (single PDF file)
Getting outdated
But still useful for Linux device driver writers!
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful reading (4)
Linux Kernel Development, 2nd Edition, Jan 2005
Robert Love, Novell Press
http://freeelectrons.com/redirect/lkd2book.html
A very synthetic and pleasant way to learn about kernel
subsystems (beyond the needs of device driver writers)
Understanding the Linux Kernel, 3rd edition, Nov 2005
Daniel P. Bovet, Marco Cesati, O'Reilly
http://oreilly.com/catalog/understandlk/
An extensive review of Linux kernel internals,
covering Linux 2.6 at last.
Unfortunately, only covers the PC architecture.
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Useful online resources
Linux kernel mailing list FAQ
http://www.tux.org/lkml/
Complete Linux kernel FAQ
Read this before asking a question to the mailing list
Kernel Newbies
http://kernelnewbies.org/
Glossary, articles, presentations, HOWTOs,
recommended reading, useful tools for people
getting familiar with Linux kernel or driver development.
Kernel glossary:
http://kernelnewbies.org/KernelGlossary
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
International conferences
Japan Linux Symposium: Tokyo, Oct 2123, 2009
http://events.linuxfoundation.org/events/japanlinuxsymposium
A conference right after the kernel summit. Many key kernel
hackers should still be there!
linux.conf.au: http://linux.org.au/conf/ (Australia / New Zealand)
Features a few presentations by key kernel hackers.
Don't miss our free conference videos on
http://freeelectrons.com/community/videos/conferences/!
18
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
ARM resources
ARM Linux project: http://www.arm.linux.org.uk/
Developer documentation:
http://www.arm.linux.org.uk/developer/
armlinuxkernel mailing list:
http://lists.infradead.org/mailman/listinfo/linuxarmkernel
FAQ: http://www.arm.linux.org.uk/armlinux/mlfaq.php
How to post kernel fixes:
http://www.arm.uk.linux.org/developer/patches/
ARMLinux @ Simtec: http://armlinux.simtec.co.uk/
A few useful resources: FAQ, documentation and Who's who!
ARM Limited: http://www.linuxarm.com/
Wiki with links to useful developer resources
19
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Embedded Linux driver development
Advice and resources
Last advice
20
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Use the Source, Luke!
Many resources and tricks on the Internet find you will, but solutions
to all technical issues only in the Source lie.
Thanks to LucasArts
21
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Introduction to Git
Introduction to Git
Thomas Petazzoni
Free Electrons
© Copyright 2009, Free Electrons.
Creative Commons BYSA 3.0 license
Latest update: Mar 5, 2010,
Document sources, updates and translations:
http://freeelectrons.com/docs/git
Corrections, suggestions, contributions and translations are welcome!
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
What is Git?
A version control system, like CVS, SVN, Perforce or ClearCase
Originally developed for the Linux kernel development, now used
by a large number of projects, including UBoot, GNOME,
Buildroot, uClibc and many more
Contrary to CVS or SVN, Git is a distributed version control
system
No central repository
Everybody has a local repository
Local branches are possible, and very important
Easy exchange of code between developers
Wellsuited to the collaborative development model used in open
source projects
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Install and setup
Git is available as a package in your distribution
sudo aptget install gitcore
Everything is available through the git command
git has many commands, called using git <command>, where
<command> can be clone, checkout, branch, etc.
Help can be found for a given command using
git help <command>
Setup your name and email address
They will be referenced in each of your commits
git config global user.name 'My Name'
git config global user.email [email protected]
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Clone a repository
To start working on a project, you use Git's clone operation.
With CVS or SVN, you would have used the checkout
operation, to get a working copy of the project (latest version)
With Git, you get a full copy of the repository, including the
history, which allows to perform most of the operations offline.
Cloning Linus Torvalds' Linux kernel repository
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux
2.6.git
git:// is a special Git protocol. Most repositories can also be
accessed using http://, but this is slower.
After cloning, in linux2.6/, you have the repository and a
working copy of the master branch.
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Explore the history
git log will list all the commits. The latest commit is the first.
commit 4371ee353c3fc41aad9458b8e8e627eb508bc9a3
Author: Florian Fainelli <[email protected]>
Date: Mon Jun 1 02:43:17 2009 0700
MAINTAINERS: take maintainership of the cpmac Ethernet driver
This patch adds me as the maintainer of the CPMAC (AR7)
Ethernet driver.
Signedoffby: Florian Fainelli <[email protected]>
Signedoffby: David S. Miller <[email protected]>
git log p will list the commits with the corresponding diff
The history in Git is not linear like in CVS or SVN, but it is a graph
of commits
Makes it a little bit more complicated to understand at the beginning
But this is what allows the powerful features of Git (distributed,
branching, merging)
5
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Visualize the history
gitk is a graphical tool that represents the history of the current
Git repository
Can be installed from the gitk package
6
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Visualize the history
Another great tool is the Web interface to Git. For the kernel, it is
available at http://git.kernel.org/
7
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Update your repository
The repository that has been cloned at the beginning will change
over time
Updating your local repository to reflect the changes of the
remote repository will be necessary from time to time
git pull
Internally, does two things
Fetch the new changes from the remote repository (git fetch)
Merge them in the current branch (git merge)
8
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Tags
The list of existing tags can be found using
git tag l
To check out a working copy of the repository at a given tag
git checkout <tagname>
To get the list of changes between a given tag and the latest
available version
git log v2.6.30..master
List of changes with diff on a given file between two tags
git log v2.6.29..v2.6.30 MAINTAINERS
With gitk
gitk v2.6.30..master
9
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Branches
To start working on something, the best is to make a branch
It is localonly, nobody except you sees the branch
It is fast
It allows to split your work on different topics, try something and
throw it away
It is cheap, so even if you think you're doing something small and
quick, do a branch
Unlike other version control systems, Git encourages the use of
branches. Don't hesitate to use them.
10
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Branches
Create a branch
git branch <branchname>
Move to this branch
git checkout <branchname>
Both at once (create and switch to branch)
git checkout b <branchname>
List of local branches
git branch l
List of all branches, including remote branches
git branch a
11
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Making changes
Edit a file with your favorite text editor
Get the status of your working copy
git status
Git has a feature called the index, which allows you to stage your
commits before committing them. It allows to commit only part of
your modifications, by file or even by chunk.
On each modified file
git add <filename>
Then commit. No need to be online or connected to commit.
git commit
If all modified files should be part of the commit
git commit a
12
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Sharing changes by email
The simplest way of sharing a few changes is to send patches by
email
The first step is to generate the patches
git formatpatch n master..<yourbranch>
Will generate one patch for each of the commits done on
<yourbranch>
The patch files will be 0001...., 0002...., etc.
The second step is to send these patches
git sendemail compose to [email protected] 00*.patch
Assumes that the local mail system is properly configured
Needs the gitemail package in Ubuntu.
Or git config allows to set the SMTP server, port, user and
password if needed
13
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Sharing changes: your own repository
If you do a lot of changes and want to ease collaboration with
others, the best is to have your own repository
Create a bare version of your repository
cd /tmp
git clone bare ~/project project.git
touch project.git/gitdaemonexportok
Transfer the contents of project.git to a publiclyvisible place
(reachable readonly by HTTP for everybody, and readwrite by
you through SSH)
Tell people to clone http://yourhost.com/path/to/project.git
Push your changes using
git push ssh://yourhost.com/path/toproject.git
srcbranch:destbranch
14
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Tracking remote trees
In addition to the official Linus Torvalds tree, you might want to use
other development or experimental trees
The OMAP tree at
git://git.kernel.org/pub/scm/linux/kernel/git/tmli
nd/linuxomap2.6.git
The realtime tree at
git://git.kernel.org/pub/scm/linux/kernel/git/rost
edt/linux2.6rt.git
The git remote command allows to manage remote trees
git remote add rt git://git.kernel.org/pub/scm/li
nux/kernel/git/rostedt/linux2.6rt.git
Get the contents of the tree
git fetch rt
Switch to one of the branches
git checkout rt/master
15
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
About Git
We have just seen the very basic features of Git.
A lot more interesting features are available (rebasing, bisection,
merging and more)
References
Git Manual
http://www.kernel.org/pub/software/scm/git/docs/usermanual.html
Git Book
http://book.gitscm.com/
Git official website
http://gitscm.com/
James Bottomley's tutorial on using Git
http://freeelectrons.com/pub/video/2008/ols/ols2008jamesbottomleygit.ogg
16
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Practical lab – Git
Clone a Git repository and
explore history
Make and share changes to a
project managed in Git
17
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Archive your lab directory
Clean up files that are easy to
retrieve, remove downloads.
Generate an archive of your lab
directory.
1
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Related documents
All our technical presentations
on http://freeelectrons.com/docs
Linux kernel
Device drivers
Architecture specifics
Embedded Linux system development
2
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Life after training
Here are things we could do to support you in your embedded Linux
and kernel projects:
BSP development for your hardware
(drivers, bootloader, toolchain)
Make the official Linux sources support your hardware
System development and integration
System optimization
Hunting and fixing nasty bugs
More training: see http://freeelectrons.com/training/. Your
colleagues who missed this class could go to our public sessions.
See http://freeelectrons.com/development
and http://freeelectrons.com/services
3
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
Last slides
Thank you!
And may the Source be with you
4
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http//freeelectrons.com
You might also like
- Protect Critical Iot Devices With Vxworks Secure Boot and Secure LoadingNo ratings yetProtect Critical Iot Devices With Vxworks Secure Boot and Secure Loading7 pages
- Memory Management in Vxworks and RtlinuxNo ratings yetMemory Management in Vxworks and Rtlinux12 pages
- (FREE PDF Sample) Mastering Embedded Linux Programming Second Edition Unleash The Full Potential of Embedded Linux With Linux 4 9 and Yocto Project 2 2 Morty Updates Chris Simmonds Ebooks100% (5)(FREE PDF Sample) Mastering Embedded Linux Programming Second Edition Unleash The Full Potential of Embedded Linux With Linux 4 9 and Yocto Project 2 2 Morty Updates Chris Simmonds Ebooks52 pages
- The Good, The Bad and The Ugly: On Threads, Processes and CoprocessesNo ratings yetThe Good, The Bad and The Ugly: On Threads, Processes and Coprocesses35 pages
- Real-Time Operating Systems An Ongoing ReviewNo ratings yetReal-Time Operating Systems An Ongoing Review4 pages
- Embedded Linux Handson Tutorial ZedBoardNo ratings yetEmbedded Linux Handson Tutorial ZedBoard46 pages
- Sitara Linux Training - Hands On With QTNo ratings yetSitara Linux Training - Hands On With QT43 pages
- Linux Foundation Courses Delivered by Enea h2 2018No ratings yetLinux Foundation Courses Delivered by Enea h2 20181 page
- IEEE Standard For POSIX Realtime and Embedded ApplicationNo ratings yetIEEE Standard For POSIX Realtime and Embedded Application189 pages
- CUDA Compute Unified Device ArchitectureNo ratings yetCUDA Compute Unified Device Architecture26 pages
- Debugging Core Files Crash Dumps UNIX LinuxNo ratings yetDebugging Core Files Crash Dumps UNIX Linux91 pages
- STM32 Security Workshop 03 SBSFU PresentationNo ratings yetSTM32 Security Workshop 03 SBSFU Presentation61 pages
- U-Boot-v2: Sascha Hauer Marc Kleine-BuddeNo ratings yetU-Boot-v2: Sascha Hauer Marc Kleine-Budde36 pages
- DRBD-Cookbook: How to create your own cluster solution, without SAN or NAS!From EverandDRBD-Cookbook: How to create your own cluster solution, without SAN or NAS!No ratings yet
- Formal Methods Applied to Complex Systems: Implementation of the B MethodFrom EverandFormal Methods Applied to Complex Systems: Implementation of the B MethodNo ratings yet
- Wireless Communications Security: Solutions for the Internet of ThingsFrom EverandWireless Communications Security: Solutions for the Internet of ThingsNo ratings yet
- SOFTWARE/ORACLE Key. This Contains Registry Entire For All OracleNo ratings yetSOFTWARE/ORACLE Key. This Contains Registry Entire For All Oracle7 pages
- SP - MDT Feature Introduction and User GuideNo ratings yetSP - MDT Feature Introduction and User Guide26 pages
- High Available Data Center On The Laptop/Desktop With Xen: Martin Bracher Cyrill Müller Linuxworld 2006No ratings yetHigh Available Data Center On The Laptop/Desktop With Xen: Martin Bracher Cyrill Müller Linuxworld 200638 pages
- [FREE PDF sample] Linux Kernel Development 1st Edition Robert Love ebooks100% (1)[FREE PDF sample] Linux Kernel Development 1st Edition Robert Love ebooks55 pages
- Tab 2-Cimplicity Startup and Navigation-Fr100% (1)Tab 2-Cimplicity Startup and Navigation-Fr18 pages
- SINUMERIK 810D/840Di/840D Sinumerik PCU Basesoftware (IM6)No ratings yetSINUMERIK 810D/840Di/840D Sinumerik PCU Basesoftware (IM6)100 pages
- Chapter 10: Windows Installation: Instructor MaterialsNo ratings yetChapter 10: Windows Installation: Instructor Materials41 pages
- CICS Commands: Indicates You Must Specify One of The OptionsNo ratings yetCICS Commands: Indicates You Must Specify One of The Options10 pages
- Protect Critical Iot Devices With Vxworks Secure Boot and Secure LoadingProtect Critical Iot Devices With Vxworks Secure Boot and Secure Loading
- Unix / Linux FAQ: with Tips to Face InterviewsFrom EverandUnix / Linux FAQ: with Tips to Face Interviews
- (FREE PDF Sample) Mastering Embedded Linux Programming Second Edition Unleash The Full Potential of Embedded Linux With Linux 4 9 and Yocto Project 2 2 Morty Updates Chris Simmonds Ebooks(FREE PDF Sample) Mastering Embedded Linux Programming Second Edition Unleash The Full Potential of Embedded Linux With Linux 4 9 and Yocto Project 2 2 Morty Updates Chris Simmonds Ebooks
- The Good, The Bad and The Ugly: On Threads, Processes and CoprocessesThe Good, The Bad and The Ugly: On Threads, Processes and Coprocesses
- Linux Foundation Courses Delivered by Enea h2 2018Linux Foundation Courses Delivered by Enea h2 2018
- IEEE Standard For POSIX Realtime and Embedded ApplicationIEEE Standard For POSIX Realtime and Embedded Application
- Bug tracking system Complete Self-Assessment GuideFrom EverandBug tracking system Complete Self-Assessment Guide
- DRBD-Cookbook: How to create your own cluster solution, without SAN or NAS!From EverandDRBD-Cookbook: How to create your own cluster solution, without SAN or NAS!
- Formal Methods Applied to Complex Systems: Implementation of the B MethodFrom EverandFormal Methods Applied to Complex Systems: Implementation of the B Method
- Wireless Communications Security: Solutions for the Internet of ThingsFrom EverandWireless Communications Security: Solutions for the Internet of Things
- SOFTWARE/ORACLE Key. This Contains Registry Entire For All OracleSOFTWARE/ORACLE Key. This Contains Registry Entire For All Oracle
- High Available Data Center On The Laptop/Desktop With Xen: Martin Bracher Cyrill Müller Linuxworld 2006High Available Data Center On The Laptop/Desktop With Xen: Martin Bracher Cyrill Müller Linuxworld 2006
- [FREE PDF sample] Linux Kernel Development 1st Edition Robert Love ebooks[FREE PDF sample] Linux Kernel Development 1st Edition Robert Love ebooks
- SINUMERIK 810D/840Di/840D Sinumerik PCU Basesoftware (IM6)SINUMERIK 810D/840Di/840D Sinumerik PCU Basesoftware (IM6)
- Chapter 10: Windows Installation: Instructor MaterialsChapter 10: Windows Installation: Instructor Materials
- CICS Commands: Indicates You Must Specify One of The OptionsCICS Commands: Indicates You Must Specify One of The Options