Reducing Boot Time:

Techniques for Fast Booting

Chris Hallinan
Field Applications Engineer
Author: Embedded Linux Primer
 Agenda

➔ Introduction and Overview

➔ Hardware Discussion
➔ Software Components
➔ Overview of Boot Sequence
➔ Bootloader Issues
➔ Linux Kernel Optimizations
➔ Tools
➔ Applications/Userland
 Introduction
➔ Fast Boot is Important to Many Products
➔ Consumer, Automotive, Medical Devices, etc.
➔ Significant gains w. minimal investment
➔ First, define “boot”. Does it mean:
➔ Splash screen?
➔ 1st user process started?
➔ Device fully up and running, connected?
➔ Boot time is affected by many factors:
➔ Hardware Design
➔ Bootloader Implementation
➔ Kernel Configuration
➔ Application Profile
Typical Boot Sequence
Power Applied
➔ Optimization Opportunities
➔ Bootloader
➔ May be multiple stages
Pwr/Clk/Reset ➔ Remove support for unused features
➔ Modify/remove hardware probing features
➔ Keep it Simple, Small :)
Bootloader
➔ Kernel
➔ Many opportunities for optimization
Kernel Init ➔ Low hanging fruit can be easy to 'pluck'
➔ Applications
Most are up to you!
Root FS Mount
➔

➔ Most initialization is serial

Userland Apps ➔ Take measurements to decide where
to focus!
© 2008 MontaVista Software ­ Confidential 4
 Bootloader Considerations

➔ Bootloader has two primary

responsibilities:

➔ Initialize CPU/Hardware (minimally)

➔ Locate, load and execute a kernel image
➔ May involve several steps, including device I/O,
decompression, etc.
 Bootloader Considerations
➔ Most bootloaders have many more features
➔ Esp. look for any type of bus enumeration
➔ Lots of useful “development” functionality
➔ dhcp, tftp, pci scan, mem utils
➔ device initialization, Flash utilities, etc
➔ In a production system, many of these features may be
unnecessary
➔ Disabling these features can have a significant impact on
boot time
➔ For fast boot, get the bootloader out of the
way quickly
➔ Remember, small == fast
 Kernel Image Optimization
➔ Use Uncompressed Kernel
➔ Decompression can take several seconds!
➔ Tradeoff: more Flash storage required
➔ Kernel build produces two images*
➔ Image and zImage (ARM)
➔ Obviously, zImage is the compressed
version
➔ On i.MX31, this saved on average ~750 mS
➔ YMMV, depending on CPU speed, FLASH
speed...
*Details vary for each architecture, ARM discussed here

© 2008 MontaVista Software ­ Confidential 7
 Linux Kernel Configuration
➔ Eliminate Unnecessary Kernel Options
➔ Reduces kernel size
➔ Speeds up kernel loading
➔ In some cases, will reduce kernel init time
once kernel is loaded
➔ Typical default kernel config contains lots
of 'stuff' (i.e. features) you may not need:
➔ MD/Raid support, IPv6, Numerous File
Systems, Extended Partition support, etc.
➔ Debug features such as kernel symbols,
ikconfig, etc.
➔ Many are compiled in features and increase
kernel size
© 2008 MontaVista Software ­ Confidential 8
Kernel Config Options to Consider Disabling
➔ CONFIG_MD ➔ CONFIG_IKCONFIG
➔ RAID/LVM support ➔ Removes support for
➔ CONFIG_IDE config info, makes kernel
➔ Saves init time if smaller
not used on HW w/ ➔ CONFIG_DEBUG_KERNEL
IDE ctrlr
➔ Can also use ➔ CONFIG_KALLSYMS
hdx=noprobe
➔ CONFIG_BUG
➔ CONFIG_PCCARD ➔ Used for debug – can be
➔ CONFIG_HOTPLUG removed if desired
➔ Remove support for ●

hotplug if not
required
© 2008 MontaVista Software ­ Confidential 9
 More Interesting Kernel Config Options

➔ Check Networking options

➔ Lots of functionality – do you need it all?
➔ ie. kernel autoconf, multicast, advanced router, etc.
➔ Remove support for unnecessary FS features
➔ Default configs often have much of this enabled
➔ CONFIG_DNOTIFY
➔ CONFIG_INOTIFY
➔ CONFIG_XFS
➔ CONFIG_AUTOFS4_FS (Automounter)
➔ etc

➔ Remember: Smaller kernel = Faster load

If in doubt, make small changes. Test early and 
often so you can rollback breakages easily!
© 2008 MontaVista Software ­ Confidential 10
 Calibration Routines

➔ Many hardware platforms spend

considerable time in calibration routines
➔ “Calculating BogoMips...”
➔ Allows precise µdelay() routines
➔ Can take significant time
➔ Use kernel command line: loops-per-jiffy:
➔ lpj=xxxxx
➔ Easy to use: most platforms will display
correct value in kernel log (and to console)
on startup

© 2008 MontaVista Software ­ Confidential 11
 Driver Configuration
➔ Consider your system requirements:
➔ What functionality must be available
immediately?
➔ What functionality can be deferred?
➔ What drivers are not needed?
➔ Modules (CONFIG_*=M), if unused, are
irrelevant
➔ Won't affect kernel startup time

© 2008 MontaVista Software ­ Confidential 12
 Driver Configuration
➔ Drivers can be pre-compiled into kernel or
built as modules for loading later
➔ Use statically-linked drivers for functions
that must be immediately available
➔ Use Loadable Modules for deferred
functionality
➔ Caveat: if you can avoid CONFIG_MODULES,
kernel will be smaller*, thus faster to load.
There is probably also a minor hit to reading
the driver from the FS instead of it being in
the kernel

*Assumes minimal system, few drivers.
© 2008 MontaVista Software ­ Confidential 13
 File System Selection

➔ Consider CRAMFS for initial read-only FS

➔ Compact and fast
➔ No journaling entries to scan on initial mount
➔ Use tmpfs for /tmp, possibly /var, others
➔ Mount writable FS later, such as JFFS2 on
NOR Flash
➔ Consider your tolerance to sudden power off
➔ Journaling file systems can protect but at a
cost of increased startup times

© 2008 MontaVista Software ­ Confidential 14
 XIP – Execute in Place
➔ Processor does not copy Kernel image to DRAM
➔ Executes directly from (NOR) Flash
➔ Advantages
➔ Reduces DRAM requirements (and thus power)
➔ Eliminates time-consuming copy from Flash
➔ Disadvantages
➔ Depending on h/w architecture, could be much
slower
➔ i.e. burst/cache performance, etc.
➔ Cost of Flash – kernel must be stored
uncompressed
➔ Your Mileage May Vary
© 2008 MontaVista Software ­ Confidential 15
 Remove Support for printk()
➔ The “Brute Force” approach - CONFIG_PRINTK
➔ Completely eliminates calls to printk()
➔ Advantages
➔ Saves significant kernel size, and therefore load time
➔ Eliminates many boot messages - decreasing boot
time
➔ Disadvantage
➔ No kernel status message are available!
➔ Makes kernel debugging very difficult
➔ A thoroughly tested kernel should work well here
➔ Alternatively, use “quiet” command line parameter
➔ suppresses printk output during boot, preserving the
printk infrastructure to be used post-boot
© 2008 MontaVista Software ­ Confidential 16
 Tools
➔ KFT: Kernel Function Timing
➔ Requires KALLSYMS, mentioned above
➔ Provides function call tracing and timing
 Entry      Duration   Local       Pid    Trace
­­­­­­­­­­ ­­­­­­­­­­ ­­­­­­­­­­ ­­­­­­­ ­­­­­­­­­­­­­­­­­­­­­­­­­­­­­
       162      11523          0       0 paging_init
       162      11523          0       0 |  free_area_init_nodes
       162      11523         12       0 |  |  free_area_init_node
       162      11511      11511       0 |  |  |  _etext+0x2f0
       162      11511          0       0 __alloc_bootmem_node
       162      11511      11511       0 !  __alloc_bootmem_core
     11787       2307       2307       0 vfs_caches_init_early
     11787       1531         69       0 vfs_caches_init_early
     11787        686          0       0 [  alloc_large_system_hash
     11787        686          0       0 [  [  __alloc_bootmem
     11787        686          0       0 [  [  [  __alloc_bootmem_nopanic
     11787        686        686       0 [  [  [  [  __alloc_bootmem_core
     13318        776        776       0 [  inode_init_early
     14094       1208        641       0 mem_init
     14094        567          4       0 #  free_all_bootmem
     14094        563        563       0 #  #  free_all_bootmem_core
     15607       3851       3851       0 schedule
     15607       3848       3848       0 schedule
     15630    1573099    1573099       1 kernel_init
     15666      81152      81152       4 schedule
     15666      81139      81139       4 schedule
© 2008 MontaVista Software ­ Confidential 17
 Tools
➔ Linux Trace Toolkit

© 2008 MontaVista Software ­ Confidential 18
 Tools
➔ printk timestamps (CONFIG_PRINTK_TIME)
➔ Appends time info to printk() output
➔ Enables measurement of long operations,
esp. at boot time
[    1.321054] md: linear personality registered for level ­1
[    1.326629] md: raid0 personality registered for level 0
[    1.331964] md: raid1 personality registered for level 1
[    1.342289] TCP cubic registered
[    1.345936] NET: Registered protocol family 1
[    1.350403] NET: Registered protocol family 17
[    1.355816] RPC: Registered udp transport module.
[    1.360571] RPC: Registered tcp transport module.
[    1.366034] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[    2.880506] IP­Config: Complete:
[    2.883575]      device=eth0, addr=192.168.1.201, mask=255.255.255.0, gw=255...
[    2.892227]      host=8349itx, domain=, nis­domain=(none),
[    2.897798]      bootserver=192.168.1.9, rootserver=192.168.1.9, rootpath=
[    2.906152] md: Autodetecting RAID arrays.

© 2008 MontaVista Software ­ Confidential 19
 Tools

➔ initcall_debug
➔ Great way to get a coarse view of system
init timing

4 msecs: initcall c02c7da0 t linear_init
5 msecs: initcall c02c6bbc t init_sd
6 msecs: initcall c02c7d78 t mpc83xx_wdt_init
7 msecs: initcall c02cc450 t init_sunrpc
10 msecs: initcall c02c01a8 t slab_sysfs_init
15 msecs: initcall c02c8d50 t genl_init
24 msecs: initcall c02c55c4 t serial8250_init
30 msecs: initcall c02c6364 t gfar_init
34 msecs: initcall c02c743c t physmap_init
72 msecs: initcall c02c9c60 t inet_init
127 msecs: initcall c02c4e4c t pty_init
4597 msecs: initcall c02cabe0 t ip_auto_config

© 2008 MontaVista Software ­ Confidential 20
An example of boot time reduction

➔ i.MX31 Reference board

➔ Initial configuration:
➔ 0:36 power on to command prompt
➔ Redboot bootloader takes 10 seconds
➔ Final Configuration
➔ Networking, static IP
➔ Busybox userland
➔ Many kernel optimizations
➔ 2.7 Seconds - Kernel start to shell prompt
➔ Minimal engineering investment

© 2008 MontaVista Software ­ Confidential 21
 Applications/Userland
➔ Startup scripts:
➔ Avoid Perl/Python dependencies (will reduce FS
size as well)
➔ One custom startup script instead of using SysV
Init and /etc/init.d
➔ Caches (big impact if you have a big
userland):
➔ Library Cache setup
➔ ICON
➔ Font
➔ Prelink libraries
➔ Use tools to profile execution
➔
strace and ltrace can be useful here
© 2008 MontaVista Software ­ Confidential 22
Additional Ideas and Resources

➔ Parallelize the init tasks

➔ “boot” everything at once
➔ drivers and tasks can do their several
seconds of waiting at the same time
➔ Provide user feedback early
➔ Splash screens, etc
➔ give impression that unit is booted while
initialization continues
➔ Gets back to defining what “booted”
means...

© 2008 MontaVista Software ­ Confidential 23
Additional Ideas and Resources

➔ Save a canned memory and system state

to non-volatile storage
➔ so that booting can occur as fast as the
memory image can be read from disk
(hibernation).
➔ Requires significant storage
➔ Suspend/Resume – many consumer
devices almost never cold boot.
➔ e.g. many “Smart Phones” only cold boot if
you remove & replace battery

© 2008 MontaVista Software ­ Confidential 24
Additional Ideas and Resources

➔ Oprofile – Systemwide Stat. Profiling for Linux

➔ On one project, this helped ID Flash as a bottleneck
➔ Focused on accelerating Flash I/O reduced boot time ~25%
➔ http://elinux.org/Boot_Time
➔ Tim Bird's OLS 2004 presentation
➔ Methods to Improve Bootup Time in Linux
➔ http://kernel.org/doc/ols/2004/ols2004v1-pages-79-88.pdf
➔ Arjan van de Ven, Linux Plumbers Conference
Booting Linux in 5 Seconds (x86/Desktop focused)
(Sept 18, 2008)
➔ http://lwn.net/Articles/299483/
➔ Meld (next slide)

© 2008 MontaVista Software ­ Confidential 25
 A Meld 
discussion 
on Linux   

startup...

© 2008 MontaVista Software ­ Confidential 26
 Thank you!

Chris Hallinan, Field Apps Engineer
MontaVista Software, Inc.
[email protected]