EmbeddedLinuxoptimizations

EmbeddedLinux optimizations
Size,RAM,speed, power,cost
MichaelOpdenacker ThomasPetazzoni FreeElectrons
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec20,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/optimizations Corrections,suggestions,contributionsandtranslationsarewelcome!

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Penguinweightwatchers
Makeyourpenguinslimmer,faster,andreduceitsconsumptionoffish! Before 2weeksafter

2
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

CELinuxForum
http://celinuxforum.org/ Nonprofitorganization,whosemembersareembeddedLinux companiesandConsumerElectronics(CE)devicesmakers. Mission:developtheuseofLinuxinCEdevices HostsmanyprojectstoimprovethesuitabilityofLinuxforCE devicesandembeddedsystems.Allpatchesaremeanttobe includedinthemainlineLinuxkernel. Mostoftheideasintroducedinthispresentationhavebeen gatheredorevenimplementedbyCELinuxForumprojects!

3
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Contents
IdeasforoptimizingtheLinuxkernelandexecutables Increasingspeed Reducingsize:diskfootprintandRAM Reducingpowerconsumption Globalperspective:costandcombinedoptimizationeffects Theultimateoptimizationtool!

4
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Increasingspeed
Reducingkernelboottime

5
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Measuringkernelboottime
CONFIG_PRINTK_TIME ConfigureitintheKernelHackingsection. Addstiminginformationtokernelmessages.Simpleandrobust. Notaccurateenoughonsomeplatforms(1jiffy=10msonarm!) Seehttp://elinux.org/Printk_Times
... [42949372.970000]Memory:64MB=64MBtotal [42949372.970000]Memory:54784KBavailable(1404Kcode,296Kdata,72Kinit) [42949373.180000]Mountcachehashtableentries:512 [42949373.180000]CPU:Testingwritebuffercoherency:ok [42949373.180000]checkingifimageisinitramfs...itisn't(badgzipmagicnumb ers);lookslikeaninitrd [42949373.200000]Freeinginitrdmemory:8192K [42949373.210000]NET:Registeredprotocolfamily16 ...

6
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Boottracer
CONFIG_BOOT_TRACERinkernelconfiguration IntroducedinLinux2.6.28 Basedontheftracetracinginfrastructure Allowstorecordthetimingsofinitcalls Bootwiththeinitcall_debugandprintk.time=1parameters, rundmesg>boot.logandonyourworkstation,run catboot.log|perlscripts/bootgraph.pl>boot.svg togenerateagraphicalrepresentation ExampleonaboardwithatAtmelAT91CPU:

5s

tty_init

pty_init

atmel_nand_init

ip_auto_config

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Grabserial
FromTimBird http://elinux.org/Grabserial Asimplescripttoaddtimestampstomessagescomingfroma serialconsole. Keyadvantage:startscountingveryearly(bootloader), anddoesn'tjuststartwhenthekernelinitializes. Anotheradvantage:nooverheadonthetarget,becauserunon thehostmachine.

8
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

DisableIPautoconfig
StoppedinitializingtheIPaddressonthekernelcommandline (oldremainsfromNFSbooting,wasconvenientnottohardcode theIPaddressintherootfilesystem.) Instead,diditinthe/etc/init.d/rcSscript. Thissaved1.56sonourAT91board. Youwillsaveevenmoreifyouhadotherrelatedoptionsinyour kernel(DHCP,BOOP,RARP)

9
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

ReducingthenumberofPTYs
PTYsareneededforremoteterminals(throughSSH) Theyarenotneededinourdedicatedsystem! ThenumberofPTYscanbereducedthroughthe CONFIG_LEGACY_PTY_COUNTkernelparameter. Ifthisnumberissetto4,wesave0.63sonourAtmelboard. Aswe'renotusingPTYsatallinourproductionsystem, wedisabledthemwithcompletelywithCONFIG_LEGACY_PTYS. Wesaved0.64s. Notethatthiscanalsobeachievedwithoutrecompilingthe kernel,usingthepty.legacy_countkernelparameter.

10
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Disableconsoleoutput
Theoutputofkernelbootupmessagestotheconsoletakes time!Evenworse:scrollingupinframebufferconsoles! Consoleoutputnotneededinproductionsystems. Consoleoutputcanbedisabledwiththequiet argumentintheLinuxkernelcommandline(bootloadersettings) Example: root=/dev/ram0rwinit=/startup.shquiet Benchmarks:canreduceboottimeby30oreven50%! Seehttp://elinux.org/Disable_Console

11
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Presetloops_per_jiffy
Ateachboot,theLinuxkernelcalibratesadelayloop(forthe udelayfunction).Thismeasuresaloops_per_jiffy(lpj) value.Thistakesabout25jiffies(1jiffy=timebetween2timer interrupts). Inembeddedsystems,itcanbeabout250ms! Youjustneedtomeasurethisonce!Findthelpjvalueinkernel bootmessages(ifyoudon'tgetitintheconsole,bootLinuxwith theloglevel=8parameter).Example:
Calibratingusingtimerspecificroutine...187.59 BogoMIPS(lpj=937984)

Atthenextboots,startLinuxwiththebelowoption: lpj=<value>
12
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LZOkerneldecompression
LZOisacompressionalgorithmthatismuchfasterthangzip, atthecostofaslightlydegradecompressionratio(+10%). Itwasalreadyinuseinthekernelcode(JFFS2,UBIFS...) AlbinTonnerrefromFreeElectronsaddedsupportforLZO compressedkernels.Hispatchesarewaitingforinclusionin mainstreamLinux.Getthemfromhttp://lwn.net/Articles/350985/

13
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LZOdecompressionresults
Savesapproximately0.25sofboottime Seehttp://freeelectrons.com/blog/lzokernelcompression/ OurpatchalsoallowsLZOtobeusedforinitramfs decompression(CONFIG_INITRAMFS_COMPRESSION_LZO=y) Anothersolutionistouseanuncompressedkernel (anotherpatchwillbesent),inwhichcasekernelexecutionisjust marginallyfasterthanwithLZO,attheexpenseofadoublesize.

14
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

DirectlybootLinuxfrombootstrapcode
Idea:makeaslightchangetoat91bootstraptodirectlyloadand executetheLinuxkernelimageinsteadoftheUbootone. RatherstraightforwardwhenbootUbootandthekernelare loadedfromNANDflash. Requirestohardcodethekernelcommandlineinthekernel image(CONFIG_CMDLINE) RequiresmoredevelopmentworkwhenUbootisloadedfroma differenttypeofstorage(SPIdataflash,forexample). Inthiscase,youcankeepUboot,butremoveallthefeaturesnot neededinproduction(USB,Ethernet,tftp...) Timesavings:about2s Seehttp://freeelectrons.com/blog/at91bootstraplinux/
15
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Reducethekernelsize
ThroughtheCONFIG_EMBEDDEDoption Removethingsthatarenotneededinyourdedicatedsystem (features,debuggingfacilitiesandmessages) Makesureyouhavenounusedkerneldrivers Disablesupportforloadablekernelmodulesandmakeallyour driversstatic(unlesstherearemultipledriversthancanbe loadedlater). Asmallerkernelisfastertoload Asimplerkernelexecutesfaster Atleast,compiledriversasmodulesfordevicesnotusedatboot time.Thisreducestimespentinitializingdrivers.
16
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Fasterrebooting(1)
kexecsystemcall:executesanewkernelfromarunningone. Mustfasterrebooting:doesn'tgothroughbootstrap/bootloader code. Greatsolutionforrebootingaftersystem(firmware)upgrades. Usefulforautomaticrebootingafterkernelpanics. Seehttp://developer.osdl.org/andyp/kexec/whitepaper/kexec.pdf andDocumentation/kdump/kdump.txtinkernelsources.

17
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Fasterrebooting(2)
Anotheroption:usereboot=softinthekernelcommandline Whenyoureboot,thefirmwarewillbeskipped. Drawback:unlikekexec,cannotbechosenfromuserspace. Supportedplatforms:i386,x86_64,arm,arm26(Aug.2006) SeeDocumentation/kernelparameters.txt inthekernelsourcesfordetails.Notsupportedonallplatforms.

18
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Skipmemoryallocation
Idea:sparememoryatboottimeandmanageitbyyourself! Assumeyouhave32MBofRAM Bootyourkernelwithmem=30 Thekernelwilljustmanagethefirst30MBofRAM. Drivercodecannowreclaimthe2MBleft: buf=ioremap( 0x1e00000, /*Start:30MB*/ 0x200000 /*Size:2MB*/ );

Thissavestimeallocatingmemory. CriticaldriversarealsosuretoalwayshavetheRAMtheyneed.

19
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

KernelboottimeOtherideas
CopykernelandinitramfsfromflashtoRAMusingDMA (UsedbyMontaVistainDellLatitudeON) Fastboot,asynchronousinitcalls:http://lwn.net/Articles/314808/ Mainlined,butAPIstillusedbyveryfewdrivers. MostlyusefulwhenyourCPUhasidletimeinthebootprocess. Usedeferredinitcalls Seehttp://elinux.org/Deferred_Initcalls NAND:justcheckforbadblocksonce Atmel:seehttp://patchwork.ozlabs.org/patch/27652/ Seehttp://elinux.org/Boot_Timeformoreresources

20
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Increasingspeed
Systemstartuptimeandapplicationspeed

21
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Startingsystemservices
SysVinit: Startsservicessequentially.Waitsforthecurrentstartupscriptto becompletetostartthenextone!Whiledependenciesexist, sometaskscanberuninparallel! Initng:http://initng.org NewalternativetoSysVinit,whichcanstartservicesinparallel, assoonastheirpreconditionsaremet.

22
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Initngvs.SysVinit:bootcharts

Initngwins! Systemutilizationismuch better. YoucanhuntsystemstartuptroublebyusingtheBootchartprogram (http://www.bootchart.org/). Bootchartisslow(Java)andnotveryaccurate. Seehttp://elinux.org/Bootchartforsolutionsforembeddedsystems.

Readingahead
LinuxkeepsthecontentsofallthefilesitreadsinRAM(inthe pagecache),aslongasitdoesn'tneedtheRAMpagesfor somethingelse. Idea:loadfiles(programsandlibrariesinparticular)inRAM cachebeforeusingthem.Bestdonewhenthesystemisnot doinganyI/O. Thankstothis,programsarenotstuckwaitingforI/O. UsedtheKnoppixdistributiontoachieveverynicebootspeed ups. AlsoplannedtobeusedbyInitng. NotveryusefulforsystemswithverylittleRAM: cachedpagesarerecycledbeforethefilesareaccessed.
24
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Implementingreadahead
Youcanusethesys_readahead()systemcall inyourCprograms.Seemanreadaheadfordetails. Youcanalsousethereadaheadlistutility,whichreadsafile containingthelistoffilestoloadincache. Availableon:http://freshmeat.net/projects/readaheadlist/. InembeddedsystemsusingBusybox,youcanusethe readaheadcommand(implementedbyFreeElectrons).

25
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Compilerspeedoptimizations
Bydefault,mosttoolsarecompiledwithcompileroptimizations. Makesureyouusethemforyourownprograms! O2isthemostcommonoptimizationswitchofgcc. Lotsofoptimizationtechniquesareavailable. Seehttp://en.wikipedia.org/wiki/Compiler_optimization O3canbealsobeusedforspeedcriticalexecutables. However,thereisdoneattheexpenseofcodesize(forexample inlining:replacingfunctioncallsbythefunctioncodeitself).

26
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Usingprocessoraccelerationinstructions
liboilhttp://liboil.freedesktop.org/ Libraryoffunctionsoptimizedforspecialinstructions fromseveralprocessors(Altivec,MMX,SSE,etc.) Mainlyfunctionsimplementingloopsondataarrays: typeconversion,copying,simplearithmetics,directcosine transform,randomnumbergeneration... Transparent:keepsyourapplicationportable! Sofarmainlysupportsdesktopprocessors License:BSDtype

27
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Prelinking(1)
Appliestoexecutablesusingsharedlibraries Toloadandstartanexecutable,thedynamiclinkerhasa significantamountofworktodo(mainlyaddressrelocation) Itcantakealotoftimeforexecutablesusingmanyshared libraries! Inmanysystemsinwhichexecutablesandsharedlibrariesnever change,thesamejobisdoneeverytimetheexecutableis started.

28
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Prelinking(2)
prelink http://people.redhat.com/jakub/prelink/ prelinkmodifiesexecutablesandsharedlibrariestosimplify thedynamiclinkerrelocationwork. Thiscangreatlyreducestartuptimeforbigapplications(50% lessforKDE!).Thisalsosavesmemoryconsumedby relocations. CanbeusedtoreducethestartuptimeofaLinuxsystem. Justneedstoberunagainwhenlibrariesorexecutablesare updated. Detailsonhttp://elinux.org/Pre_Linking
29
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

UsesimplerUnixexecutables
Big,featurerichexecutablestaketimetoload. Particularlytrueforshellscriptscallingthebashshell! Idea:replacestandardUnix/GNUexecutablesbylightweight,simplified implementationsbybusybox(http://busybox.net). ImplementedbyUbuntu6.10toreduceboottime,replacingbash(649K) bydash(79K,seehttp://en.wikipedia.org/wiki/Debian_Almquist_shell). Thisbrokevariousshellscriptswhichusedbashspecificfeatures (bashisms). InnonembeddedLinuxsystems wherefeaturerichexecutablesarestillneeded, shouldatleastusebusyboxashforsystemscripts.

30
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Shells:reducingforking
fork/execsystemcallsareveryheavy. Becauseofthis,callstoexecutablesfromshellsareslow. Evenexecutingechoinbusyboxshellsresultsinaforksyscall! SelectShells>Standaloneshellinbusybox configurationtomakethebusyboxshellcallappletswhenever possible. Pipesandbackquotesarealsoimplementedbyfork/exec. Youcanreducetheirusageinscripts.Example: cat/proc/cpuinfo|grepmodel Replaceitwith:grepmodel/proc/cpuinfo Seehttp://elinux.org/Optimize_RC_Scripts

31
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Usefasterfilesystems
Runfasterbyusingthemostappropriatefilesystems! Compressedreadonlyfilesystem(blockdevice): useSquashFS(http://squashfs.sourceforge.net) insteadofCramFS(muchslower,gettingobsolete). NANDflashstorage:youshouldtryUBIFS (http://www.linuxmtd.infradead.org/doc/ubifs.html),the successorofJFFS2.Itismuchfaster.Youcouldalsouse SquashFS.SeeourChoosingfilesystemspresentation (http://freeelectrons.com/docs/filesystems).

32
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Usefasterfilesystems(2)
UseRAMfilesystemsfortemporary,speedcriticalfiles withnoneedforpermanentstorage.Detailsinthekernel sources:Documentation/filesystems/tmpfs.txt Benchmarkyoursystemandapplicationoncompeting filesystems!Reiser4ismoreinnovativeandbenchmarks founditfasterthanext3. GoodtobenchmarkyoursystemwithJFSorXFStoo. XFSisreportedtobethefastesttomount(goodfor startuptime),andJFStohavethelowestCPUutilization. Seehttp://www.debianadministration.org/articles/388
ext4isalsoreadytobeusednow.

33
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Speedupapplicationswithtmpfs
WhenenoughRAMisavailable,theOSkeepsrecentlyaccessedfilesand applicationsinRAM(pagecache).Thissignificantlyspeedsupanynewusage. However,dependingonsystemactivity,thismaynotlastlong. Forprogramsthatneedfaststartupeveniftheyhaven'tbeenrunforalongtime: copythemtoatmpfsfilesystematsystemstartup!Thismakessuretheyare alwaysaccessedfromthefilecacheinRAM(providedyoudonothaveaswap partition). SeeDocumentation/filesystems/tmpfs.txtinkernelsourcesfordetailsabout tmpfs. Caution:don'tuseramdisksinstead! RamdisksduplicatefilesinRAMandunusedspacecannotbereclaimed. Caution:usewithcare.Mayimpactoverallperformance. Notneededifthere'senoughRAMtocacheallfilesandprograms.

34
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Bootfromahibernateimage
Theultimatetechniqueforinstantboot! Indevelopment:startthesystem,requiredapplicationsandthe userinterface.Hibernatethesystemtodisk/flashinthisstate. Inproduction:bootthekernelandrestorethesystemstatefrom withthispredefinedhibernationimage. Thisway,youdon'thavetoinitializetheprogramsonebyone. Youjustgetthebacktoavalidstate. UsedinSonycamerastoachieveinstantpowerontime. UnlikeSuspendtoRAM,stillallowstoremovebatteries!

35
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Useaprofiler
Usingaprofilercanhelptoidentifyunexpectedbehaviordegrading applicationperformance. Forexample,aprofilercantellyouinwhichfunctionsmostofthe timeisspent. Possibletostartwithstraceandltrace AdvancedprofilingwithValgrind:http://valgrind.org/
Compileyourapplicationforx86architecture YoucanthenprofileitwiththewholeValgrindtoolsuite: Cachegrind:sourcesofcachemissesandfunctionstatistics. Massif:sourcesofmemoryallocation.

SeeourSoftwareDevelopmentpresentationfordetails: http://freeelectrons.com/docs/swdev/
36
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Reducingsize
KernelsizeandRAMusage

37
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LinuxTiny
Goal:reducethediskfootprintandRAMsizeoftheLinuxkernel http://elinux.org/Linux_Tiny SetofpatchesagainstthemainstreamLinuxkernel. Mergeabilityinmainstreamisapriority. Manychangeshavealreadybeenmergedinrecentkernels. Allfeaturescanbeselectedinkernelconfiguration (CONFIG_EMBEDDED). Alsoshipsutilitiesorpatchesfortrackingsources ofmemoryusageorcodesize.

38
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LinuxTinyideas(1)
Removekernelmessages(printk,BUG,panic...) Huntexcessinlining(speedvs.sizetradeoff) 2.6.26:canallowgcctouninlinefunctionsmarkedasinline: (CONFIG_OPTIMIZE_INLINING=y).Onlyusedbyx86sofar. Huntexcessmemoryallocations Memory(slobinsteadofslab)allocatormorespaceefficientfor smallsystems. Reducethesizeofkerneldatastructures(mayimpact performance) Simpleralternativeimplementationsofkernelfunctionalities withlessfeatures,ornotsupportingspecialcases.
39
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LinuxTinyideas(2)
Removesomefeatureswhichmaynotbeneeded insomesystems. Compilingoptimizationsforsize. AsmallerkernelexecutablealsosavesRAM (unlessexecutedinplacefromstorage).

40
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LinuxTiny:kernelconfigurationscreenshot

Manyfeatures configuredout

41
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

WithandwithoutCONFIG_EMBEDDED
TestsonLinux2.6.29,onaminimalisticbutworkingx86kernel
1800

1600

1400

1200

1000

Raw Compressed

800

600

400

200

0 CONFIG_EMBEDDED=n CONFIG_EMBEDDED=y

Raw:272KB(17%),Compressed:136KB(20%)

42

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Replaceinitrdbyinitramfs
Replaceinitramdisks(initrd)withinitramfs: muchlessoverheadandramwaste!
Access tofile VirtualFile System Filesystem driver Block driver Copy Blockstorage File cache

Regular blockdevice

Access tofile VirtualFile System Filesystem driver Block driver

Ramdisk blockdevice

Access tofile VirtualFile System

ramfs

File cache Copy Blockstorage RAM

File cache

RAM

RAM

43
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

ramfsadvantagesoverramdisks
Noblockandfilesystemoverhead. NoduplicationinRAM. Filescanberemoved(reclaimingRAM)afteruse. Initramfs:ramfsarchiveembeddedintheLinuxkernelfile.

44
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Reducingsize
ApplicationsizeandRAMusage

45
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Staticordynamiclinking?(1)
Staticlinking Allsharedlibrarycodeduplicatedintheexecutables AllowsnottocopytheClibraryinthefilesystem. Simplerandsmallerwhenveryfewexecutables(busybox) Librarycodeduplication:badforsystemswithmoreexecutables (codesizeandRAM) Bestforsmallsystems(<12MB)withfewexecutables!

46
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Staticordynamiclinking?(2)
Dynamiclinking Sharedlibrarycodenotduplicatedintheexecutables Makesmuchsmallerexecutables SavesspaceinRAM(biggerexecutablestakemoreRAM) Requiresthelibrarytothecopiedtothefilesystem Bestformediumtobigsystems(>500KB1MB)

47
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

UsingalighterClibrary
glibc(GNUClibrary):http://www.gnu.org/software/libc/ FoundonmostcomputertypeGNU/Linuxmachines Sizeonarm:approx1.7MB uClibc:http://www.uclibc.org/ FoundinmoreandmoreembeddedLinuxsystems! Sizeonarm:approx400KB(yousave1.2MB!) Executablesareslightlysmallertoo:
Cprogram Plainhelloworld Busybox Compiledwithsharedlibraries glibc 4.6K 245K uClibc 4.4K 231K Compiledstatically glibc 475K 843K uClibc 25K 311K

48
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

HowtouseuClibc?
NeedtocompileallyourexecutableswithauClibctoolchain. Readytousetoolchainscanbefoundon http://freeelectrons.com/community/tools/uclibc Youcanveryeasilybuildyourownwithbuildroot: http://buildroot.uclibc.org/ YoualsohavetocopytheuClibcfilesfromthetoolchaintothe /libdirectoryinthetargetrootfilesystem. Readytousefilesystemscanalsobegeneratedbybuildroot. Youjustneedtoaddyourspecificstuffthen.

49
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Needforstripping
Compiledexecutablesandlibrariescontainextrainformation whichcanbeusedtoinvestigateproblemsinadebugger. Thiswasusefulforthetooldeveloper,butnotforthefinaluser. Toremovedebugginginformation,usethestripcommand. Thiscansaveaverysignificantamountofspace! gccohellohello.c (outputsize:4635bytes) striphello (outputsize:2852bytes,38.5%) Don'tforgettostriplibrariestoo!

50
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Aremyexecutablesstripped?
Youcanusethefilecommandtogettheanswer gccohellohello.c filehello hello:ELF32bitLSBexecutable,Intel80386,version1 (SYSV),forGNU/Linux2.2.5,dynamicallylinked(uses sharedlibs),notstripped striphello hello:ELF32bitLSBexecutable,Intel80386,version1 (SYSV),forGNU/Linux2.2.5,dynamicallylinked(uses sharedlibs),stripped Youcanusefindstrip(http://packages.debian.org/stable/source/perforate) tofindallexecutablesandlibrariesthatneedstrippinginyoursystem.

51
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Howtostrip
Somelightweighttools,likebusybox,areautomaticallystripped whenyoubuildthem. Makefilesformanystandardtoolsofferaspecialcommand: makeinstallstrip Caution:strippingisarchitecturedependent. Usethestripcommandfromyourcrosscompilingtoolchain: armlinuxstrippotato

52
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

sstrip:superstrip
http://muppetlabs.com/~breadbox/software/elfkickers.html Goesbeyondstripandcanstripoutafewmorebitsthatare notusedbyLinuxtostartanexecutable. Canbeusedonlibrariestoo.Minorlimitation:processed librariescannolongerbeusedtocompilenewexecutables. CanalsobefoundintoolchainsmadebyBuildroot(optional)

Hello World Regular stripped sstripped 4691 B 2904 B (-38 %) 1392 B (-70 %)

Busybox 287783 B

Inkscape 11397 KB

230408 B (-19.9 %) 9467 KB (-16.9 %) 229701 B (-20.2 %) 9436 KB (-17.2 %)

Bestfortiny executables!

53
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

LibraryOptimizer
http://libraryopt.sourceforge.net/ ContributedbyMontaVista Examinesthecompletetargetfilesystem,resolvesallshared librarysymbolreferences,andrebuildsthesharedlibrarieswith onlytheobjectfilesrequiredtosatisfythesymbolreferences. Canalsotakecareofstrippingexecutablesandlibraries. However,requirestorebuildallthecomponentsfromsource. WouldbenicertoachievethisonlywithELFmanipulations.

54
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Compilerspaceoptimizations
Regularcompileroptimizationssimplifyingcodealsoreducesize Youcanalsoreducethesizeofexecutablesbyaskinggccto optimizegeneratedcodesize: gccOsohusbandhusband.c OscorrespondstoO2optimizationsexcepttheonesincreasing size,plusextrasizespecificones. Osisalreadyusedbydefaulttobuildbusybox. Possibletofurtherreducethesizebycompilingandoptimizingall sourcesatonce,withthefwholeprogramcombinegcc options. Seehttp://gcc.gnu.org/onlinedocs/gcc/OptimizeOptions.html forallgccoptimizationoptions.
55
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Simplegccoptimizationbenchmark
Executable size
250000

200000

150000

None O2 (generic) O3 (speed) Os (size)

100000

50000

Busybox

Dropbear

56
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Restartableapplications
WhenRAMisscarce,canbeusefultoabortapplicationsthatare notinuse(forexamplehiddengraphicalinterfaces). BettertodoitbeforetheLinuxKernelOOM (OutOfMemory)killercomesandmakesbaddecisions. YoucanusetheLinuxCheckpoint/Restartprojecttohavethe Linuxkernelsavethestateofarunningapplicationsothatitcan laterresumeitsexecutionfromthetimeatwhichitwas checkpointed. Seehttp://www.linuxcr.org/fordetails.

57
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Compressingfilesystems
Cansignificantlyincreaseyourstoragecapacity MTD(flashorROM)storage:useUBIFS orJFFS2forsmallpartitions. Blockstorage:useSquashFS(http://squashfs.sourceforge.net) insteadofCramFSforreadonlypartitions.Itcompressesmuch betterandismuchfastertoo.

58
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Mergingduplicatefiles
Softwarecompilingandinstallingoftencreateduplicatefiles... Checkthatyourrootfilesystemdoesntcontainany! dupmerge2:http://sourceforge.net/projects/dupmerge Replacesduplicatefilesbyhardlinks. clink:http://freeelectrons.com/community/tools/utils/clink Replacesduplicatefilesbysymboliclinks. Example:saves4%oftotalspaceinFedoraCore5. finddup:http://www.shelldorado.com/scripts/cmds/finddup Findsduplicatefiles.

59
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Reducingpowerconsumption

60
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Ticklesskernel
Kernelconfiguration:NO_HZsettinginProcessortypeandfeatures Toimplementmultitasking,theprocessorreceivesatimerinterrupt atagivenfrequency(every4msbydefaultonLinux2.6).Onidlesystems, thiswakesuptheprocessorallthetime,justtorealizethereisnothingtodo! Idea:whenallprocessorsareidle,disablethetimerinterrupt,andreenableit whensomethinghappens(arealinterrupt).Thissavespowerinlaptops,in embeddedsystemsandwithvirtualservers! 2.6.24:supportsx86,arm,mipsandpowerpc

61
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

PowerTOP
http://www.lesswatts.org/projects/powertop/ Withdynamicticks,allowstofixpartsofkernelcodeand applicationsthatwakeupthesystemtoooften. PowerTOPallowstotracktheworstoffenders NowavailableonARMcpusimplementingCPUidle Alsogivesyouusefulhints forreducingpower.

62
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

PowerTOPinaction

63
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

cpufreq
Configuration:CPU_FREQinPowermanagementoptions AllowstochangetheCPUfrequencyonthefly Supportedarchitectures(2.6.20): i386,sh,ia64,sparc64,x86_64,powerpc,arm(i.MXonly). Usuallycontrolledfromuserspacethrough/sysbyauser configurablegovernorprocess,accordingtoCPUload,heat,battery status...Themostcommoniscpuspeed: http://carlthompson.net/software/cpuspeed/ Savesasignificantamountofbatterylifeinnotebooks.

64
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

SuspendhiddenGUIs
Idea:suspendhiddenuserinterfacestosaveCPUandpower. Sendasuspend(stop)signal: killSIGTSTP<pid> Sendacontinuesignal: killSIGCONT<pid>

65
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Softwaresuspend
http://www.suspend2.net/ Lotsofgreatfeaturesfornotebookusers,suchasRAMsuspend orhibernatetodisk. Unfortunately,restrictedonsomeIntelcompatibleprocessors andtargetingonlymachineswithAPMorACPI(rarelyfoundin nonPCembeddedsystems!). Notaddressingtherequirementsofembeddedsystems(support forotherCPUs,voltagereduction...).

66
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Powermanagementresources
http://freeelectrons.com/docs/power/ OurpresentationonpowermanagementintheLinuxkernel WhatyouneedtoimplementinyourBSPanddevicedrivers. http://lesswatts.org IntelefforttryingtocreateaLinuxpowersavingcommunity. MainlytargetsIntelprocessors. Lotsofusefulresources. http://wiki.linaro.org/WorkingGroups/PowerManagement/ OngoingdevelopmentontheARMplatform. Tipsandideasforprolongingbatterylife: http://j.mp/fVdxKh

67
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

EmbeddedLinuxOptimizations

Globalperspective
Costandcombinedoptimizationeffects

68
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Combinedbenefits
Speedincrease Morespeed LessRAM
Fasterallocations Lessswapping Sometimesless cacheflushing Fasterapplication loadingfromstorage andinRAM. Sometimes,simpler, fastercode. LessRAMusage

RAMreduction Powerreduction
CPUcanrunslowerorstaylonger inpowersavingmode Fewer/smallerRAMchips:less dynamicandstandbypower. CPUwithlesscache:lesspower Fewer/smallerstoragechips:less power

Costreduction
Slower,cheaper CPU Fewer/cheaper RAMchips CPUwithless cache:cheaper Fewer/cheaper storage

Lessspace

Lesspower

Cheaperbatteries orcheaperAC/DC converter

69
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Theultimateoptimizationtool!
Wehaveseenmanywaystooptimize anexistingsystem. However,nothingreplacesagooddesign! So,firstcarefullydesignandimplement yoursystemandapplicationswiththeirrequirements inmind. Then,usetheoptimizationtechniquestofurtherimprove yoursystemandthepartsthatyoureused(kerneland applications).

70
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Relateddocuments

Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

Howtohelp
Youcanhelpustoimproveandmaintainthisdocument... Bysendingcorrections,suggestions,contributionsand translations Byaskingyourorganizationtoorderdevelopment,consulting andtrainingservicesperformedbytheauthorsofthese documents(seehttp://freeelectrons.com/). Bysharingthisdocumentwithyourfriends,colleagues andwiththelocalFreeSoftwarecommunity. Byaddinglinksonyourwebsitetoouronlinematerials, toincreasetheirvisibilityinsearchengineresults.

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Linuxkernel Linuxdevicedrivers Boardsupportcode Mainstreamingkernelcode Kerneldebugging EmbeddedLinuxTraining Allmaterialsreleasedwithafreelicense! UnixandGNU/Linuxbasics Linuxkernelanddriversdevelopment RealtimeLinux,uClinux Developmentandprofilingtools Lightweighttoolsforembeddedsystems Rootfilesystemcreation Audioandmultimedia Systemoptimization

FreeElectrons
Ourservices
CustomDevelopment Systemintegration EmbeddedLinuxdemosandprototypes Systemoptimization Applicationandinterfacedevelopment Consultingandtechnicalsupport Helpindecisionmaking Systemarchitecture Systemdesignandperformancereview Developmenttoolandapplicationsupport Investigatingissuesandfixingtoolbugs