EmbeddedLinuxdevelopment

EmbeddedLinuxkernelanddriverdevelopment MarcoStornelli

CreatedwithOpenOffice.org3.1.1
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Rightstocopy
MarcoStornelli Copyright20092010 http://www.webalice.it/firefox82

AttributionShareAlike3.0 Youarefree tocopy,distribute,display,andperformthework tomakederivativeworks tomakecommercialuseofthework Underthefollowingconditions Attribution.Youmustgivetheoriginalauthorcredit. ShareAlike.Ifyoualter,transform,orbuilduponthiswork, youmaydistributetheresultingworkonlyunderalicense identicaltothisone. Foranyreuseordistribution,youmustmakecleartoothersthe licensetermsofthiswork. Anyoftheseconditionscanbewaivedifyougetpermissionfrom thecopyrightholder. Yourfairuseandotherrightsareinnowayaffectedbytheabove. Licensetext:http://creativecommons.org/licenses/bysa/3.0/legalcode

BasedonaworkofFreeElectronsCopyright20042008http://freeelectrons.com/
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Bestviewedwith...
ThisdocumentisbestviewedwitharecentPDFreader orwithOpenOffice.orgitself! Takeadvantageofinternalorexternalhyperlinks. So,donthesitatetoclickonthem!Seenextpage. Findpagesquicklythankstoautomaticsearch Usethumbnailstonavigateinthedocumentinaquickway

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Courseprerequisites
Skillstomaketheselecturesprofitable FamiliaritywithUnixconceptsanditscommandlineinterface Essentialtomanipulatesourcesandfiles Essentialtounderstandanddebugthesystemthatyoubuild ExperiencewithCprogramming OnlineCcoursescanbefoundon http://dmoz.org/Computers/Programming/Languages/C/Tutorials/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Contents(1)
Kerneloverview Linuxfeatures Kernelcode Kernelsubsystems Linuxversioningschemeanddevelopmentprocess Legalissues Kerneluserinterface

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Contents(2)
Compilingandbooting Linuxkernelsources Kernelsourcemanagers Kernelconfiguration Crosscompilingthekernel Overallsystemstartup Bootloaders Linuxdevicefiles Basicdriverdevelopment Loadablekernelmodules Moduleparameters Addingsourcestothetree

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Contents(3)
Driverdevelopment Memorymanagement I/Omemoryandports Characterdrivers Debugging Processesandscheduling Sleeping,Interruptmanagement Handlingconcurrency mmap,DMA Asyncoperations

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Contents(4)
Driverdevelopment Newdevicemodel,sysfs udev,mdevandhotplug Adviceandresources Gettinghelpandcontributions Bugreportandpatchsubmission References

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Contents(5)
Annexes Kernelsources Slabcachesandmemorypools Ubootdetails Initrunlevels EmbeddedDistributionsandTools EmbeddedFilesystems

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

EmbeddedLinuxdriverdevelopment

Kerneloverview
Linuxfeatures

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

10

Studiedkernelversion:2.6
Linux2.6 Linux2.6.0wasreleasedinDecember2003. Lotsoffeaturesandnewdrivers havebeenaddedataquickpacesincethen. Itisgettingmoreandmoredifficulttogetsupportordrivers forrecenthardwarein2.4.Nocommunitysupportatall! ThesetrainingslidesarecompliantwithLinux2.6.32. It'sbesttostarttolearnaboutthemostrecentfeaturesand updates!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

11

Linuxkernelkeyfeatures
Portabilityandhardwaresupport Runsonmostarchitectures. Scalability Canrunonsupercomputersas wellasontinydevices (8MBofRAMisenough). Compliancetostandardsand interoperability. Exhaustivenetworkingsupport. Security Itcan'thideitsflaws.Itscodeis reviewedbymanyexperts. Stabilityandreliability. Modularity Canincludeonlywhatasystem needsevenatruntime. Easytoprogram Youcanlearnfromexistingcode. Manyusefulresourcesonthenet.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

12

Supportedhardwarearchitectures
Seethearch/directoryinthekernelsources Minimum:32bitprocessors,withorwithoutMMU 32bitarchitectures(arch/subdirectories) alpha,arm,avr32,blackfin,cris,frv,h8300,m32r,m68k, m68knommu,mn10300,mips,parisc,powerpc,s390,sh, sparc,um,x86,xtensa,microblaze,S+core 64bitarchitectures: ia64,mips,powerpc,sh64,sparc64,x86_64,parisc,s390 Seearch/<arch>/Kconfig,arch/<arch>/README,or Documentation/<arch>/fordetails

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

13

EmbeddedLinuxdriverdevelopment

Kerneloverview
Kernelcode

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

14

ImplementedinC
ImplementedinClikeallUnixsystems. (CwascreatedtoimplementthefirstUnixsystems) AlittleAssemblyisusedtoo: CPUandmachineinitialization,exceptions, andcriticallibraryroutines. Seehttp://www.tux.org/lkml/#s153 forreasonsfornotusingC++ (mainreason:thekernelrequiresefficientcode).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

15

Programminglanguagesinthekernelsources
Linux2.6.32reportbySLOCCount(http://dwheeler.com/sloccount/)
Totalsgroupedbylanguage(dominantlanguagefirst): ansic:7680247(96.91%) asm:225175(2.84%) perl:9015(0.11%) sh:3404(0.04%) yacc:2964(0.04%) lex:1824(0.02%) cpp:1795(0.02%) python:493(0.01%) awk:109(0.00%) sed:30(0.00%) Totallinesofcode:7925056
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

16

CompiledwithGNUC
NeedGNUCextensionstocompilethekernel. So,youcannotuseanyANSICcompiler! YoucanalsousetheIntelandMarvellcompilers(onlyontheir respectiveplatforms)whichidentifythemselvesasaGNUcompiler. SomeGNUCextensionsusedinthekernel: InlineCfunctions Inlineassembly Structurememberinitializationinanyorder(alsoinANSIC99) Branchannotation(seenextpage)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

17

Helpgcctooptimizeyourcode!
Usethelikelyandunlikelystatements (include/linux/compiler.h) Example: if(unlikely(err)){ ... } TheGNUCcompilerwillmakeyourcodefaster forthemostlikelycase. Usedinmanyplacesinkernelcode! Don'tforgettousethesestatements!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

18

NoClibrary
Thekernelhastobestandaloneandcan'tuseuserspacecode. Userspaceisimplementedontopofkernelservices,nottheopposite. Kernelcodehastosupplyitsownlibraryimplementations (stringutilities,cryptography,uncompression...) So,youcan'tusestandardClibraryfunctionsinkernelcode. (printf(),memset(),malloc()...). YoucanalsousekernelCheaders. Fortunately,thekernelprovidessimilarCfunctionsforyour convenience,likeprintk(),memset(),kmalloc()...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

19

Managingendianism
Linuxsupportsbothlittleandbigendianarchitectures Eacharchitecturedefines__BIG_ENDIANor__LITTLE_ENDIAN in<asm/byteorder.h> Canbeconfiguredinsomeplatformssupportingboth. Tomakeyourcodeportable,thekerneloffersconversionmacros (thatdonothingwhennoconversionisneeded).Mostusefulones: u32cpu_to_be32(u32); //CPUbyteordertobigendian u32cpu_to_le32(u32); //CPUbyteordertolittleendian u32b32_to_cpu(u32); //BigendiantoCPUbyteorder u32le32_to_cpu(u32); //LittleendiantoCPUbyteorder

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

20

Kernelcodingguidelines
Neverusefloatingpointnumbersinkernelcode.Yourcode mayberunonaprocessorwithoutafloatingpointunit(likeon arm).Floatingpointcanbeemulatedbythekernel,butthisis veryslow. Defineallsymbolsasstatic,exceptexportedones (toavoidnamespacepollution) SeeDocumentation/codingStyleformoreguidelines

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

21

EmbeddedLinuxdriverdevelopment

Kerneloverview
Kernelsubsystems

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

22

Kernelarchitecture
App1 App2 Clibrary Systemcallinterface Process management Memory management Filesystem support Filesystem types CPUsupport code CPU/MMU supportcode Storage drivers Character devicedrivers Network devicedrivers Hardware CPU RAM Storage Device control Networking ... User space

Kernel space

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

23

Kernelmemoryconstraints
Whocanlookafterthekernel? Nomemoryprotection Accessingillegalmemory locationsresultin(oftenfatal) kerneloopses. Fixedsizestack(8or4KB) Unlikeinuserspace, nowaytomakeitgrow. Kernelmemorycan'tbeswapped out(forthesamereasons).
User User process process Attempt toaccess

SIGSEGV,kill

Kernel Illegal memory location Exception (MMU) Userspacememorymanagement Usedtoimplement: memoryprotection stackgrowth memoryswappingtodisk demandpaging

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

24

I/Oschedulers
MissionofI/Oschedulers:reorderreadsandwritestodiskto minimizediskheadmoves(timeconsuming!)

Slower

Faster

Notneededinembeddedsystemswithnoharddisks (dataaccesstimeindependentoflocationonflashstorage) BuildyourkernelwithnoopI/Oschedulerthen!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

25

EmbeddedLinuxdriverdevelopment

Kerneloverview
Linuxversioningschemeanddevelopmentprocess

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

26

Until2.6(1)
Onestablemajorbranchevery2or3years Identifiedbyanevenmiddlenumber Examples:1.0,2.0,2.2,2.4 Onedevelopmentbranchtointegratenewfunctionalitiesandmajor changes Identifiedbyanoddmiddlenumber Examples:2.1,2.3,2.5 Aftersometime,adevelopmentversionbecomesthenewbase versionforthestablebranch Minorreleasesonceinwhile:2.2.23,2.5.12,etc.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

27

Until2.6(2)

Stableversion 2.4.0 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.4.6 2.4.7 2.4.8

2.5.0

2.5.1 2.5.2

2.5.3

2.5.4

2.6.0

2.6.1 Stable

Development

Note:inreality,manymoreminor versionsexistinsidethestableand developmentbranches

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

28

ChangessinceLinux2.6(1)
Since2.6.0,kerneldevelopershavebeenabletointroducelots ofnewfeaturesonebyoneonasteadypace,withouthavingto makemajorchangesinexistingsubsystems. OpeninganewLinux2.7(or2.9)developmentbranchwillbe requiredonlywhenLinux2.6isnolongerabletoaccommodate keyfeatureswithoutundergoingtraumaticchanges. Thankstothis,morefeaturesarereleasedtousersatafasterpace.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

29

ChangessinceLinux2.6(2)
Since2.6.14,thekerneldevelopersagreedonthefollowingdevelopmentmodel Afterthereleaseofa2.6.xversion,atwoweeksmergewindowopens, duringwhichmajoradditionsaremerged Themergewindowisclosed bythereleaseoftestversion2.6.(x+1)rc1 Thebugfixingperiodopens,forsixtotenweeks Atregularintervalsduringthebugfixingperiod, 2.6.(x+1)rcYtestversionsarereleased Whenconsideredsufficientlystable, kernel2.6.(x+1)isreleased,andtheprocessstartsagain

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

30

Mergeandbugfixingwindows

2weeks
Mergewindow 2.6.21 2.6.22rc1 2.6.22rc2

6to10weeks
Bugfixingperiod 2.6.22rc3 2.6.22rc4 2.6.22rc5 2.6.22

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

31

NostableLinuxinternalAPI(1)
Ofcourse,theexternalAPImustnotchange(systemcalls,/proc,/sys),asit couldbreakexistingprograms.Newfeaturescanbeadded,butkerneldeveloperstry tokeepbackwardcompatibilitywithearlierversions,atleastfor1orseveralyears. TheinternalkernelAPIcannowundergochangesbetweentwo2.6.xreleases.A standalonedrivercompiledforagivenversionmaynolongercompileorworkona morerecentone. SeeDocumentation/stable_api_nonsense.txt inkernelsourcesforreasonswhy. WheneveradeveloperchangesaninternalAPI,(s)healsohastoupdateallkernel codewhichusesit.Nothingbroken! Worksgreatforcodeinthemainlinekerneltree. Difficulttokeepinlineforoutoftreeorclosedsourcedrivers!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

32

NostableLinuxinternalAPI(2)
USBexample LinuxhasupdateditsUSBinternalAPIatleast3times(fixes,security issues,supportforhighspeeddevices)andhasnowthefastestUSBbus speeds(comparedtoothersystems) WindowsXPalsohadtorewriteitsUSBstack3times.But,becauseof closedsource,binarydriversthatcan'tbeupdated,theyhadtokeep backwardcompatibilitywithallearlierimplementation.Thisisverycostly (development,security,stability,performance). SeeMyths,Lies,andTruthsabouttheLinuxKernel,byGregK.H.,for detailsaboutthekerneldevelopmentprocess: http://kroah.com/log/linux/ols_2006_keynote.html

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

33

Morestabilityforthe2.6kerneltree
Issue:securityfixesonlyreleasedforlast(orlasttwo)stable kernelversions,andofcoursebydistributionsfortheexact versionthatyou'reusing. Somepeopleneedtohavearecentkernel,butwithlongterm supportforsecurityupdates.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

34

What'snewineachLinuxrelease?(1)
commit 3c92c2ba33cd7d666c5f83cc32aa590e794e91b0 Author: Andi Kleen <[email protected]> Date: Tue Oct 11 01:28:33 2005 +0200 [PATCH] i386: Don't discard upper 32bits of HWCR on K8 Need to use long long, not long when RMWing a MSR. I think it's harmless right now, but still should be better fixed if AMD adds any bits in the upper 32bit of HWCR. Bug was introduced with the TLB flush filter fix for i386 Signed-off-by: Andi Kleen <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>

? ?!

...

TheofficiallistofchangesforeachLinuxreleaseisjusta hugelistofindividualpatches! Verydifficulttofindoutthekeychangesandtogetthe globalpictureoutofindividualchanges.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

35

What'snewineachLinuxrelease?(2)
Fortunately,asummaryofkeychanges withenoughdetailsisavailableon http://wiki.kernelnewbies.org/LinuxChanges

? ?!

Foreachnewkernelrelease,youcanalsogetthe changesinthekernelinternalAPI: http://lwn.net/Articles/2.6kernelapi/ What'snext? Documentation/featureremovalschedule.txt liststhefeatures,subsystemsandAPIsthatare plannedforremoval(announced1yearinadvance).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

36

EmbeddedLinuxdriverdevelopment

Kerneloverview
Legalissues

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

37

Linuxlicense
ThewholeLinuxsourcesareFreeSoftwarereleased undertheGNUGeneralPublicLicenseversion2(GPLv2). Seehttp://freeelectrons.com/articles/freesw/trainingfor detailsaboutFreeSoftwareanditslicenses.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

38

Linuxkernellicensingconstraints
Constraintsatreleasetime(noconstraintbefore!) ForanydeviceembeddingLinuxandFreeSoftware,youhavetoreleasesources totheenduser.Youhavenoobligationtoreleasethemtoanybodyelse! AccordingtotheGPL,onlyLinuxdrivers withaGPLcompatiblelicenseareallowed. Proprietarydriversarelessandlesstolerated. Lawyerssaythattheyareillegal. Proprietarydriversmustnotbestaticallycompiledintothekernel. Youarenotallowedtoreusecodefromotherkerneldrivers(GPL) inaproprietarydriver.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

39

AdvantagesofGPLdrivers
Fromthedriverdeveloper/decisionmakerpointofview
Youdon'thavetowriteyourdriver fromscratch.Youcanreusecodefrom similarfreesoftwaredrivers. Yougetfreecommunitycontributions, support,codereviewandtesting. Proprietarydrivers(evenwithsources) don'tgetany. Yourdriverscanbefreelyshippedby others(mainlybydistributions). Closedsourcedriversoftensupporta givenkernelversion.Asystemwith closedsourcedriversfrom2different sourcesisunmanageable. Usersandthecommunitygetapositive imageofyourcompany.Makesit easiertohiretalenteddevelopers. Youdon'thavetosupplybinarydriver releasesforeachkernelversionand patchversion(closedsourcedrivers). Drivershaveallprivileges.Youneed thesourcestomakesurethatadriver isnotasecurityrisk. Yourdriverscanbestaticallycompiled intothekernel.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

40

Advantagesofintreekerneldrivers
Advantagesofhavingyourdriversinthemainlinekernelsources Onceyoursourcesareacceptedinthemainlinetree,theyare maintainedbypeoplemakingchanges. Costfreemaintenance,securityfixesandimprovements. Easyaccesstoyoursourcesbyusers. Manymorepeoplereviewingyourcode.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

41

LegalproprietaryLinuxdrivers(1)
WorkingaroundtheGPLbycreatingaGPLwrapper:

Wrapper SpecialAPI

Linuxkernel

Binary Binary blob blob (proprietary (proprietary driver) driver)

Theproprietaryblobisnotbrokenwhenyourecompileorupdatethe kerneland/ordriver.Hence,theproprietarydrivermaynotbeconsideredas aderivativework.However,thekernelismonolithicandtheblobstill belongstoasingleexecutable.Thisisstillcontroversial!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

42

LegalproprietaryLinuxdrivers(2)
2examplecases Nvidiagraphiccarddrivers Supportingwirelessnetworkcards usingWindowsdrivers. TheNdisWrapperproject (http://ndiswrapper.sourceforge.net/) implementstheWindowskernelAPI andNDIS(NetworkDriverInterface Spec.)APIwithintheLinuxkernel. Usefulforusingcardsforwhichno specificationsarereleased. Drawbacks Stillsomemaintenanceissues. Example:Nvidiaproprietarydriver incompatiblewithX.org7.1. Performanceissues. Wrapperoverheadandoptimizationsnot available. Securityissues.Thedriversareexecuted withfullkernelprivileges. ...andallotherissueswithproprietary drivers.UserslosemostbenefitsofFree Software.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

43

Softwarepatentissuesinthekernel
LinuxKerneldriverissuesbecauseofpatentedalgorithms Checkforsoftwarepatentwarningswhenyouconfigureyourkernel!
Patentwarningsissuedinthe documentationofdrivers,showninthe kernelconfigurationinterface. FlashTranslationLayer drivers/mtd/ftl.c IntheUSA,thisdrivercanonlybe usedonPCMCIAhardware (MSystemspatent). NandFlashTranslationLayer IntheUSA,canonlybeusedon DiskOnChiphardware. Networkingcompression drivers/net/bsd_comp.c Can'tsendaCCPresetrequestasa resultofanerrordetectedafter decompression(Motorolapatent). OtherdriversnotacceptedinLinux releasesoralgorithmsnot implementedbecauseofsuchpatents! Otherwise,moreexampleswouldbe availableinthesourcecode.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

44

EmbeddedLinuxdevelopment

Kerneloverview
Kerneluserinterface

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

45

Mountingvirtualfilesystems
Linuxmakessystemandkernelinformationavailableinuser spacethroughvirtualfilesystems(virtualfilesnotexistingonany realstorage).Noneedtoknowkernelprogrammingtoaccessthis!

Mounting/proc: sudomounttprocnone/proc Mounting/sys: sudomounttsysfsnone/sys

Filesystemtype Mountpoint Rawdevice orfilesystemimage Inthecaseofvirtual filesystems,anystringisfine

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

46

Kerneluserspaceinterface
Afewexamples: /proc/cpuinfo:processorinformation /proc/meminfo:memorystatus /proc/version:kernelversionandbuildinformation /proc/cmdline:kernelcommandline /proc/<pid>/environ:callingenvironment /proc/<pid>/cmdline:processcommandline ...andmanymore!Seebyyourself!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

47

Userspaceinterfacedocumentation
Lotsofdetailsaboutthe/procinterfaceareavailablein Documentation/filesystems/proc.txt (almost2000lines)inthekernelsources. Youcanalsofindotherdetailsintheprocmanualpage: manproc SeetheNewDeviceModelsectionfordetailsabout/sys

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

48

Userspacedevicedrivers(1)
Possibletoimplementdevicedriversinuserspace! Suchdriversjustneedaccesstothedevicesthrough minimum,generickerneldrivers. Examples: Printerandscannerdrivers (ontopofgenericparallelport/USBdrivers) Xdrivers:lowlevelkerneldrivers+userspaceXdrivers. Userspacedriversintroducedin2.6.23.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

49

Userspacedevicedrivers(2)
Advantages Noneedforkernelcodingskills.Easiertoreusecodebetweendevices. Driverscanbewritteninanylanguage,evenPerl! Driverscanbekeptproprietary. Drivercodecanbekilledanddebugged.Cannotcrashthekernel. Canbeswappedout(kernelcodecannotbe). Canusefloatingpointcomputation. Lessinkernelcomplexity. Drawbacks Lessstraightforwardtohandleinterrupts. Increasedlatencyvs.kernelcode. SeeUIOHowtointhekerneldocumentationfordetailedinformationsandthe UsingUIOonanEmbeddedplatformtalkatELC2008, http://www.celinux.org/elc08_presentations/uio080417celfelc08.pdf.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

50

EmbeddedLinuxdriverdevelopment

CompilingandbootingLinux
Linuxkernelsources

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

51

kernel.org

Downloadkernel sourcesfrom http://kernel.org

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

52

Linuxsourcesstructure(1)
arch/<arch>
Architecturespecificcode arch/<arch>/mach<mach> Machine/boardspecificcode block/ Blocklayercore COPYING Linuxcopyingconditions(GNUGPL) CREDITS Linuxmaincontributors crypto/ Cryptographiclibraries Documentation/ Kerneldocumentation.Don'tmissit! drivers/ Alldevicedriversexceptsoundones fs/ Filesystems(fs/ext3/,etc.) include/ Kernelheaders include/asm<arch> Architectureandmachinedependentheaders include/linux Linuxkernelcoreheaders init/ Linuxinitialization(includingmain.c) firmware/ Firmwaresneededtoworkwithdrivers virt/ Codeforvirtualizationsupport tools/ Performacemonitor/countertools
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

53

Linuxsourcesstructure(2)
ipc/ Kbuild kernel/ lib/ MAINTAINERS Makefile mm/ net/ README REPORTINGBUGS samples/ scripts/ security/ sound/ usr/ Codeusedforprocesscommunication Partofthekernelbuildsystem Linuxkernelcore(verysmall!) Misclibraryroutines(zlib,crc32...) Maintainersofeachkernelpart.Veryuseful! TopLinuxmakefile(setsarchandversion) Memorymanagementcode(smalltoo!) Networksupportcode(notdrivers) Overviewandbuildinginstructions Bugreportinstructions Examplekernelmodules Scriptsforinternalorexternaluse Securitymodelimplementations(SELinux...) Soundsupportcodeanddrivers Codetogenerateaninitramfscpioarchive
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

54

Linuxkernelsize(1)
Linux2.6.32sources: Rawsize:349MB(approx8millionlinesofcode) gzipcompressedtararchive:75MB bzip2compressedtararchive:61MB(better) MinimumcompiledLinuxkernelsize(withLinuxTinypatches) approx300KB(compressed),800KB(raw)(Linux2.6.14) Whyarethesesourcessobig? Becausetheyincludethousandsofdevicedrivers,manynetworkprotocols, supportmanyarchitecturesandfilesystems... TheLinuxcore(scheduler,memorymanagement...)isprettysmall!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

55

Linuxkernelsize(2)
SizeofLinux2.6.32sourcedirectories(KB)
drivers arch fs sound net include Documentation firmware kernel mm scripts crypto security lib tools block ipc virt init samples usr 0 20000 40000 60000 80000 100000 120000 140000 160000 180000 200000

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

56

GettingLinuxsources:2possibilities
Fullsources Theeasiestway,butlongertodownload. Example: http://kernel.org/pub/linux/kernel/v2.6/linux2.6.28.1.tar.bz2 Orpatchagainstthepreviousversion Assumingyoualreadyhavethefullsourcesofthepreviousversion Example: http://kernel.org/pub/linux/kernel/v2.6/patch2.6.28.bz2(2.6.27to2.6.28) http://kernel.org/pub/linux/kernel/v2.6/patch2.6.28.1.bz2(2.6.28to2.6.28.1)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

57

Downloadingfullkernelsources
Downloadingfromthecommandline Withawebbrowser,identifytheversionyouneedonhttp://kernel.org Intherightdirectory,downloadthesourcearchiveanditssignature (copyingthedownloadaddressfromthebrowser):
wgethttp://kernel.org/pub/linux/kernel/v2.6/linux2.6.11.12.tar.bz2

Extractthecontentsofthesourcearchive:
tarjxflinux2.6.11.12.tar.bz2
~/.wgetrcconfigfileforproxies: ~/.wgetrcconfigfileforproxies:
http_proxy=<proxy>:<port> http_proxy=<proxy>:<port> ftp_proxy=<proxy>:<port> ftp_proxy=<proxy>:<port> proxy_user=<user>(ifany) proxy_user=<user>(ifany) proxy_password=<passwd>(ifany) proxy_password=<passwd>(ifany)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

58

Downloadingkernelsourcepatches(1)
Assumingyoualreadyhavethelinuxx.y.<n1>version Identifythepatchesyouneedonhttp://kernel.orgwithawebbrowser Downloadthepatchfilesandtheirsignature:
Patchfrom2.6.10to2.6.11
wgetftp://ftp.kernel.org/pub/linux/kernel/v2.6/patch2.6.11.bz2

Patchfrom2.6.11to2.6.11.12(lateststablefixes)
wgethttp://www.kernel.org/pub/linux/kernel/v2.6/patch2.6.11.12.bz2

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

59

Downloadingkernelsourcepatches(2)
Applythepatchesintherightorder:
cdlinux2.6.10/ bzcat../patch2.6.11.bz2|patchp1 bzcat../patch2.6.11.12.bz2|patchp1 cd.. mvlinux2.6.10linux2.6.11.12

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

60

Anatomyofapatchfile
Apatchfileistheoutputofthediffcommand
diffcommandline diffNrua/Makefileb/Makefile a/Makefile2008120409:25:1708:00 Filedateinfo +++b/Makefile2008120409:27:1508:00 @@1,7+1,7@@ Linenumbersinfiles VERSION=2 Contextinfo:3linesbeforethechange PATCHLEVEL=6 Usefultoapplyapatchwhenlinenumberschanged SUBLEVEL=27 EXTRAVERSION= Removedline(s)ifany +EXTRAVERSION=.1 Addedline(s)ifany

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

61

Usingthepatchcommand
Thepatchcommandapplieschangestofilesinthecurrentdirectory: Makingchangestoexistingfiles Creatingordeletingfilesanddirectories patchusageexamples: patchp<n><diff_file catdiff_file|patchp<n> bzcatdiff_file.bz2|patchp<n> zcatdiff_file.gz|patchp<n> n:numberofdirectorylevelstoskipinthefilepaths
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Youcanreverse Youcanreverse apatch apatch withtheR withtheR option option

Youcantestapatchwith Youcantestapatchwith thedryrun thedryrun option option

62

ApplyingaLinuxpatch
Linuxpatches... Alwaystoapplytothex.y.<z1>version Downloadableingzip andbzip2(muchsmaller)compressedfiles. Alwaysproducedforn=1 (that'swhateverybodydoes...doittoo!) Linuxpatchcommandlineexample: cdlinux2.6.10 bzcat../patch2.6.11.bz2|patchp1 cd..;mvlinux2.6.10linux2.6.11 Youcanmakepatch30% Youcanmakepatch30% fasterbyusingsp1 fasterbyusingsp1 insteadofp1 insteadofp1 (silent) (silent)
Testedonpatch2.6.23.bz2

Keeppatchfilescompressed:usefultochecktheirsignaturelater. Youcanstillview(orevenedit)theuncompresseddatawithvi: vipatch2.6.11.bz2(onthefly(un)compression)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

63

Accessingdevelopmentsources(1)
Kerneldevelopmentsourcesarenowmanagedwithgit: http://kernel.org/pub/software/scm/git/ YoucanbrowseLinus'gittree(ifyoujustneedtocheckafewfiles): http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux2.6.git;a=tree Ifyouarebehindaproxy,setUnixenvironmentvariablesdefiningproxy settings.Example: exporthttp_proxy="proxy.server.com:8080" exportftp_proxy="proxy.server.com:8080"

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

64

Accessingdevelopmentsources(2)
Pickupagitdevelopmenttreeonhttp://git.kernel.org/ Getalocalcopy(clone)ofthistree. Example(Linustree,theoneusedforLinuxstablereleases):
gitclonegit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux2.6.git

Updateyourcopywheneverneeded(Linustreeexample):
cdlinux2.6 gitpullgit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux2.6.git

Moredetailsavailable onhttp://git.or.cz/orhttp://linux.yyz.us/githowto.html

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

65

EmbeddedLinuxdevelopment

CompilingandbootingLinux
Kernelsourcemanagementtools

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

66

Cscope
http://cscope.sourceforge.net/ Tooltobrowsesourcecode (mainlyC,butalsoC++orJava) SupportshugeprojectsliketheLinuxkernel Takeslessthan1min.toindexLinux2.6.17 sources(fast!) Canbeusedfromeditorslikevimandemacs. InLinuxkernelsources,runitwith: makeARCH=<arch>cscope
Allowssearchingcodefor: Allowssearchingcodefor: allreferencestoasymbol allreferencestoasymbol globaldefinitions globaldefinitions functionscalledbyafunction functionscalledbyafunction functionscallingafunction functionscallingafunction textstring textstring regularexpressionpattern regularexpressionpattern afile afile filesincludingafile filesincludingafile

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

67

Cscopescreenshot

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

68

KScope
http://kscope.sourceforge.net AgraphicalfrontendtoCscope MakesiteasytobrowseandedittheLinuxkernelsources Candisplayafunctioncalltree Niceeditingfeatures:symbolcompletion,spellingchecker, automaticindentation... Usageguidelines: UsetheKernelsettingtoignorestandardCincludes. Makesuretheprojectnamedoesn'tcontainblankcharacters!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

69

KScopescreenshots(1)

Symbols incurrent file Mainwindow

Project files

Querywindow

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

70

KScopescreenshots(2)

Calledfunctionstree
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

71

LXR:LinuxCrossReference
http://sourceforge.net/projects/lxr Genericsourceindexingtool andcodebrowser Webserverbased Veryeasyandfasttouse Identifierortextsearchavailable Veryeasytofindthedeclaration, implementationorusagesofsymbols SupportsCandC++ Supportshugecodeprojects suchastheLinuxkernel (264Minversion2.6.26). Takessometimeandpatiencetosetup (configuration,indexing,server configuration). Initialindexingveryslow: Linux2.6.17:severalhoursonaserver withanAMDSempron2200+CPU. UsingKscopeistheeasiestandfastest solutionformodifiedkernelsources.

Youdon'tneedtosetupLXRby yourself. Useourhttp://lxr.freeelectrons.com server! OtherserversavailableontheInternet: http://freeelectrons.com/community/kernel/l ThismakesLXRthesimplestsolution tobrowsestandardkernelsources.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

72

LXRscreenshot

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

73

KetchupEasyaccesstokernelsourcetrees
http://www.selenic.com/ketchup/wiki/ Makesiteasytogetthelatestversionofagivenkernelsourcetree (2.4,2.6,2.6rc,2.6git,2.6mm,2.6rt...) Onlydownloadstheneededpatches. Revertspatcheswhenneededtoapplyamorerecentpatch. Alsochecksthesignatureofsourcesandpatches.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

74

Ketchupexamples
Gettheversioninthecurrentdirectory: >ketchupm 2.6.10 Upgradetothelateststableversion: >ketchupG2.6tip 2.6.10>2.6.12.5 Applyingpatch2.6.11.bz2 Applyingpatch2.6.12.bz2 Applyingpatch2.6.12.5.bz2 Youcangetbackto2.6.8: >ketchupG2.6.8 TheGoptionofketchup TheGoptionofketchup disablessourcesignature disablessourcesignature checking. checking. SeeKernelsourcesannex SeeKernelsourcesannex fordetailsaboutenabling fordetailsaboutenabling kernelsource kernelsource integritychecking. integritychecking.

Moreonhttp://selenic.com/ketchup/wiki/index.cgi/ExampleUsage

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

75

EmbeddedLinuxdevelopment

CompilingandbootingLinux
Kernelconfiguration

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

76

Kernelconfiguration
Defineswhatfeaturestoincludeinthekernel: Storedinthe.configfileattherootofkernelsources. Mostusefulcommandstocreatethisconfigfile: make[xconfig|gconfig|menuconfig|oldconfig] TomodifyakernelinaGNU/Linuxdistribution: configfilesusuallyreleasedin/boot/,togetherwithkernelimages: /boot/config2.6.1711generic Theconfigurationfilecanalsobefoundinthekernelitself: >zcat/proc/config.gz (ifenabledinGeneralSetup>Kernel.configsupport)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

77

makexconfig
makexconfig NewQtconfigurationinterfaceforLinux2.6:qconf. MucheasiertousethaninLinux2.4! Makesureyouread help>introduction:usefuloptions! Filebrowser:easiertoloadconfigurationfiles Newsearchinterfacetolookforparameters

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

78

makexconfigscreenshot

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

79

makexconfigsearchinterface
Looksforakeyword inthedescriptionstring Allowstoselectorun selectfoundparameters.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

80

Kernelconfigurationoptions
Compiledasamodule(separatefile) CONFIG_ISO9660_FS=m

Driveroptions CONFIG_JOLIET=y CONFIG_ZISOFS=y

Compiledstaticallyintothekernel CONFIG_UDF_FS=y
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

81

Corresponding.configfileexcerpt
# #CDROM/DVDFilesystems # CONFIG_ISO9660_FS=m CONFIG_JOLIET=y CONFIG_ZISOFS=y CONFIG_UDF_FS=y CONFIG_UDF_NLS=y

Sectionname (helpstolocatesettingsintheinterface)

Allparametersareprefixed withCONFIG_

# #DOS/FAT/NTFilesystems # #CONFIG_MSDOS_FSisnotset #CONFIG_VFAT_FSisnotset CONFIG_NTFS_FS=m #CONFIG_NTFS_DEBUGisnotset CONFIG_NTFS_RW=y

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

82

makegconfig
makegconfig NewGTKbasedgraphical configurationinterface. Functionalitysimilartothat ofmakexconfig. Justlackingasearch functionality.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

83

makemenuconfig
makemenuconfig Sameoldtextinterface asinLinux2.4. Usefulwhennographics areavailable.Pretty convenienttoo! Sameinterfacefoundin othertools:BusyBox, buildroot...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

84

makeoldconfig
makeoldconfig Neededveryoften! Usefultoupgradea.configfilefromanearlierkernel release. Issueswarningsforobsoletesymbols. Asksforvaluesfornewsymbols. Ifyouedita.configfilebyhand,it'sstronglyrecommended torunmakeoldconfigafterwards!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

85

makeallnoconfig
makeallnoconfig Onlysetsstronglyrecommendedsettingstoy. Setsallothersettingston. Veryusefulinembeddedsystemstoselectonlytheminimum requiredsetoffeaturesanddrivers. Muchmoreconvenientthanunselectinghundredsoffeatures onebyone!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

86

makelocalmodconfig
makelocalmodconfig Itperformsthefollowingsteps:
Itruns"lsmod"tofindallthemodulesloadedonthecurrent runningsystemanditwillreadalltheMakefilestomapwhich CONFIGenablesamodule; itreadstheKconfigfilestofindthedependenciesandselects thatmaybeneededtosupportaCONFIG. itreadsthe.configfileandremovesanymodule"=m"thatis notneededtoenablethecurrentlyloadedmodules.

There'sanadditional"makelocalyesconfig"target,incase youdon'twanttousemodulesand/orinitrds.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

87

Cloningaconfigfile
Wecaninsertinthekerneltheconfigfileenablinga configurationoption. Toextracttheinformationfromakernelimagewecanuse scripts/extractikconfigscript. Wecanuse(ifenabled)/proc/config.gz. Inthiswaywecanalwaysknowtheconfigurationofa runningkernel!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

88

Undoingconfigurationchanges
Afrequentproblem: Afterchangingseveralkernelconfigurationsettings, yourkernelnolongerworks. Ifyoudon'trememberallthechangesyoumade, youcangetbacktoyourpreviousconfiguration: >cp.config.old.config Alltheconfigurationinterfacesofthekernel (xconfig,menuconfig,allnoconfig...) keepthis.config.oldbackupcopy.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

89

makehelp
makehelp Listsallavailablemaketargets. Usefultogetareminder,ortolookforneworadvanced options!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

90

Customizingtheversionstring
Toidentifyyourkernelimagewithothersbuiltfromthesame sources(butadifferentconfiguration),usethe LOCALVERSIONsetting(inGeneralSetup). Example: # #Generalsetup # CONFIG_LOCALVERSION="acme1" Theunamercommand(intherunningsystem)willreturn: 2.6.20acme1

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

91

EmbeddedLinuxdriverdevelopment

CompilingandbootingLinux
Crosscompilingthekernel

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

92

Crosscompilingthekernel
WhenyoucompileaLinuxkernelforanotherCPUarchitecture Muchfasterthancompilingnatively,whenthetargetsystemis muchslowerthanyourGNU/Linuxworkstation. MucheasierasdevelopmenttoolsforyourGNU/Linux workstationaremucheasiertofind. Tomakethedifferencewithanativecompiler,crosscompiler executablesareprefixedbythenameofthetargetsystem, architectureandsometimeslibrary.Examples: mipslinuxgcc m68klinuxuclibcgcc armlinuxgnueabigcc
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

93

Specifyingacrosscompiler(1)
TheCPUarchitectureandcrosscompilerprefixaredefinedthroughthe ARCHandCROSS_COMPILEvariablesinthetoplevelMakefile. TheMakefiledefinesCC=$(CROSS_COMPILE)gcc SeecommentsinMakefilefordetails TheeasiestsolutionistomodifytheMakefile. Example,ARMplatform,crosscompiler:armlinuxgcc ARCH ?=arm CROSS_COMPILE ?=armlinux

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

94

Specifyingacrosscompiler(2)
AnothersolutionistosetARCHandCROSS_COMPILE throughthemakecommandline Explanation:anyvariablesetthroughthemakecommandline overridesanysettingintheMakefile. Examples:
makeARCH=armCROSS_COMPILE=armlinuxxconfig makeARCH=armCROSS_COMPILE=armlinux makeARCH=armCROSS_COMPILE=armlinuxmodules_install

Bigdrawback: Youshouldneverforgetthesesettingswhenyourunmake! That'serrorproneandnotconvenientatall.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

95

Specifyingacrosscompiler(3)
Anothersolution:setARCHandCROSS_COMPILE
asenvironmentvariablesinyourterminal: exportARCH=arm exportCROSS_COMPILE=armlinux Canbesetinprojectspecificenvironments. NothardcodedintheMakefile. Donotinterferewithpatches. Youdon'tforgettosetthemwhenyourunanymakecommand. Caution:onlyapplytoshells inwhichthesevariableshavebeenset.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

96

Readymadeconfigfiles
assabet_defconfigintegrator_defconfigmainstone_defconfig assabet_defconfigintegrator_defconfigmainstone_defconfig badge4_defconfigiq31244_defconfigmx1ads_defconfig badge4_defconfigiq31244_defconfigmx1ads_defconfig bast_defconfigiq80321_defconfigneponset_defconfig bast_defconfigiq80321_defconfigneponset_defconfig cerfcube_defconfigiq80331_defconfignetwinder_defconfig cerfcube_defconfigiq80331_defconfignetwinder_defconfig clps7500_defconfigiq80332_defconfigomap_h2_1610_defconfig clps7500_defconfigiq80332_defconfigomap_h2_1610_defconfig ebsa110_defconfigixdp2400_defconfigomnimeter_defconfig ebsa110_defconfigixdp2400_defconfigomnimeter_defconfig edb7211_defconfigixdp2401_defconfigpleb_defconfig edb7211_defconfigixdp2401_defconfigpleb_defconfig enp2611_defconfigixdp2800_defconfigpxa255idp_defconfig enp2611_defconfigixdp2800_defconfigpxa255idp_defconfig ep80219_defconfigixdp2801_defconfigrpc_defconfig ep80219_defconfigixdp2801_defconfigrpc_defconfig epxa10db_defconfigixp4xx_defconfigs3c2410_defconfig epxa10db_defconfigixp4xx_defconfigs3c2410_defconfig footbridge_defconfigjornada720_defconfigshannon_defconfig footbridge_defconfigjornada720_defconfigshannon_defconfig fortunet_defconfiglart_defconfigshark_defconfig fortunet_defconfiglart_defconfigshark_defconfig h3600_defconfiglpd7a400_defconfigsimpad_defconfig h3600_defconfiglpd7a400_defconfigsimpad_defconfig h7201_defconfiglpd7a404_defconfigsmdk2410_defconfig h7201_defconfiglpd7a404_defconfigsmdk2410_defconfig h7202_defconfiglubbock_defconfigversatile_defconfig h7202_defconfiglubbock_defconfigversatile_defconfig hackkit_defconfiglusl7200_defconfig hackkit_defconfiglusl7200_defconfig

arch/arm/configsexample
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

97

Usingreadymadeconfigfiles
Defaultconfigurationfilesavailableformanyboards/machines! Checkifoneexistsinarch/<arch>/configs/foryourtarget. Example:ifyoufoundanacme_defconfigfile,youcanrun: makeacme_defconfig Usingarch/<arch>/configs/isaverygoodwayofreleasinga defaultconfigurationfileforagroupofusersordevelopers. Likeallmakecommands,youmust runmake<machine>_defconfig inthetoplevelsourcedirectory.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

98

Kernelcompilingtips
Viewthefull(gcc,ld...)commandline: makeV=1 Cleanupgeneratedfiles(toforcerecompilingdrivers): makeclean Removeallgeneratedfiles.Caution:alsoremovesyour.configfile! makemrproper Alsoremoveeditorbackupandpatchrejectfiles(mainlytogenerate patches): makedistclean

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

99

Crosscompilingsetup
Example IfyouhaveanARMcrosscompilingtoolchain in/usr/local/arm/3.3.2/ YoujusthavetoaddittoyourUnixsearchpath: exportPATH=/usr/local/arm/3.3.2/bin:$PATH (Caution:thescopeofthisdefinitionislimitedtothecurrentshell). Choosingatoolchain SeetheDocumentation/Changesfileinthesources fordetailsaboutminimumtoolversionsrequirements. Moreabouttoolchains:FreeSoftwaretoolsforembeddedsystemstraining: http://freeelectrons.com/training/devtools/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

100

Buildingthekernel
Run make Copy arch/<arch>/boot/zImage(oruImageforUBoot) tothetargetstorage

Youcancustomizearch/<arch>/boot/install.shsothatmake installdoesthisautomaticallyforyou. makeINSTALL_MOD_PATH=<rootfsprefix> modules_install

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

101

Dependencymanagement
Whenyoumodifyaregularkernelsourcefile,makeonly rebuildswhatneedsrecompiling.That'swhatitisusedfor. However,theMakefileisquitepessimisticabout dependencies.Whenyoumakesignificantchangestothe .configfile,don'tbesurprisedifmakeoftenrecompiles mostfiles,evenwhenitdoesn'tseemnecessary.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

102

Generatedfiles
Createdwhenyourunthemakecommand vmlinux RawLinuxkernelimage,noncompressed. arch/<arch>/boot/zImage zlibcompressedkernelimage (defaultimageonarm)

arch/<arch>/boot/bzImage (defaultimageoni386) Alsoazlibcompressedkernelimage. Caution:bzmeansbigzippedbutnotbzip2compressed! arch/<arch>/boot/uImage CompressedkernelimagereadytouseforUBoot.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

103

Filescreatedbymakemodules_install(1)
<rootfsprefix>/lib/modules/<version>/:Kernelmodules +extras build/ Everythingneededtobuildmoremodulesforthiskernel:Makefile, .configfile,modulesymbolinformation(module.symVers), kernelheaders(include/andinclude/asm/) kernel/ Module.ko(KernelObject)files,inthesamedirectorystructureasin thesources.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

104

Filescreatedbymakemodules_install(2)
<rootfsprefix>/lib/modules/<version>/(continued) modules.alias Modulealiasesformoduleloadingutilities.Exampleline: aliassoundservice?0snd_mixer_oss modules.dep Moduledependencies(seetheLoadablekernelmodulessection) modules.symbols Tellswhichmoduleagivensymbolbelongsto. Allthefilesinthisdirectoryaretextfiles. Don'thesitatetohavealookbyyourself!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

105

Compilingfasterwithccache
http://ccache.samba.org/ CompilercacheforCandC++,alreadyshippedbysomedistributions Muchfasterwhencompilingthesamefileasecondtime! Veryusefulwhen.configfilechangearefrequent. Useitbyaddingaccacheprefix totheCCandHOSTCCdefinitionsinMakefile: CC=ccache$(CROSS_COMPILE)gcc HOSTCC=ccachegcc Performancebenchmarks: 63%:withaFedoraCore3configfile(manymodules!) 82%:withanembeddedLinuxconfigfile(muchfewermodules!)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

106

Compilingfasteronmultiprocessorhosts
Ifyouareusingaworkstationwithnprocessors,youmayroughly divideyourcompiletimebynbycompilingseveralfilesinparallel. makej<n> Runsseveraltargetsinparallel,wheneverpossible Usingmakej2ormakej3onsingleprocessorworkstations. Thisdoesn'thelpmuch.Intheory,severalparallelcompilejobskeep theprocessorbusywhileotherprocessesarewaitingforfilestobe readofwritten.Inpractice,youdon'tgetanysignificantspeedup(not morethan10%),unlessyourI/Osareveryslow.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

107

Kernelheaders
Sometimeswhenwewantcrosscompileanapplicationweneed thekernelheaders,wherearethey?
makeARCH=<targetarch>INSTALL_HDR_PATH=<kernel headerpath>headers_install

Wecanuseaspath<rootfsprefix>/usr/includeand removethatfolderbeforetoinstalltherootfsinthetarget.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

108

Crosscompilingsummary
EditMakefile:setARCHandCROSS_COMPILE Getthedefaultconfigurationforyourmachine: make<machine>_defconfig(ifexistinginarch/<arch>/configs). Refinetheconfigurationsettingsaccordingtoyourrequirements: makexconfig AddthecrosscompilerpathtoyourPATHenvironmentvariable. Compilethekernel:make Copythekernelimagefromarch/<arch>/boot/tothetarget. Copymodulestoadirectorywhichyoureplicateonthetarget: makeINSTALL_MOD_PATH=<dir>modules_install

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

109

EmbeddedLinuxdevelopment

CompilingandbootingLinux
Linuxdevicefiles

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

110

Characterdevicefiles
Accessedthroughasequentialflowofindividualcharacters. Characterdevicescanbeidentifiedbytheirctype(lsl):
crwrw1rootuucp4,64Feb232004/dev/ttyS0 crww1jdoetty136,1Feb232004/dev/pts/1 crw1rootroot13,32Feb232004/dev/input/mouse0 crwrwrw1rootroot1,3Feb232004/dev/null

Exampledevices:keyboards,mice,parallelport,IrDA, Bluetoothport,consoles,terminals,sound,video...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

111

Blockdevicefiles
Accessedthroughdatablocksofagivensize.Blockscanbe accessedinanyorder. Blockdevicescanbeidentifiedbytheirbtype(lsl):
brwrw1rootdisk3,1Feb232004hda1 brwrw1jdoefloppy2,0Feb232004fd0 brwrw1rootdisk7,0Feb232004loop0 brwrw1rootdisk1,1Feb232004ram1 brw1rootroot8,1Feb232004sda1

Exampledevices:hardorfloppydisks,ramdisks,loopdevices...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

112

Devicemajorandminornumbers
Asyoucouldseeinthepreviousexamples, devicefileshave2numbersassociatedtothem: Firstnumber:majornumber. Secondnumber:minornumber. Majorandminornumbersareusedbythekerneltobindadriver tothedevicefile.Devicefilenamesdon'tmattertothekernel! Tofindoutwhichdriveradevicefilecorrespondsto, orwhenthedevicenameistoocryptic, seeDocumentation/devices.txt.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

113

Devicefilecreation
Devicefilesarenotcreatedwhenadriverisloaded. Theyhavetobecreatedinadvance: sudomknod/dev/<device>[c|b]<major><minor> Examples: sudomknod/dev/ttyS0c464 sudomknod/dev/hda1b31

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

114

EmbeddedLinuxdevelopment

CompilingandbootingLinux
Overallsystemstartup

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

115

Linux2.4bootingsequence
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash Initializessupportforthedevicewherethekernelimageisfound(localstorage,network, removablemedia) LoadsthekernelimageinRAM Executesthekernelimage(withaspecifiedcommandline)

Kernel
Uncompressesitself Initializesthekernelcoreandstaticallycompileddrivers(neededtoaccesstherootfilesystem) Mountstherootfilesystem(specifiedbytherootkernelparameter) Executesthefirstuserspaceprogram(specifiedbytheinitkernelparameter)

Firstuserspaceprogram
Configuresuserspaceandstartsupsystemservices

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

116

Linux2.6bootingsequence
Bootloader

Kernel

Uncompressesitself Initializesthekernelcoreandstaticallycompileddrivers Uncompressestheinitramfscpioarchiveincludedinthekernelfilecache(nomounting,nofilesystem). Iffoundintheinitramfs,executesthefirstuserspaceprogram: /init

Userspace:/initscript(whatfollowsisjustatypicalscenario)

Runsuserspacecommandstoconfigurethedevice(suchasnetworksetup,mounting /procand/sys...) Mountsanewrootfilesystem.Switchtoit( switch_root) Runs/sbin/init(orsometimesanew/linuxrcscript)

Userspace:/sbin/init

Runscommandstoconfigurethedevice(ifnotdoneyetintheinitramfs) Startsupsystemservices(daemons,servers)anduserprograms

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

unchanged

ExecutedbythehardwareatafixedlocationinROM/Flash Initializessupportforthedevicewheretheimagesarefound(localstorage,network,removablemedia) LoadsthekernelimageinRAM Executesthekernelimage(withaspecifiedcommandline)

117

Linux2.6bootingsequencewithinitrd
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash Initializessupportforthedevicewheretheimagesarefound(localstorage,network,removablemedia) Loadsthekernelandinitramdisk(initrd)imagesinRAM Executesthekernelimage(withaspecifiedcommandline)

Kernel

Uncompressesitself Initializesstaticallycompileddrivers Uncompressestheinitramfscpioarchiveincludedinthekernel.Mountsit.No /initexecutablefound. Sofallsbacktotheoldwayoftryingtolocateandmountarootfilesystem. Mountstherootfilesystemspecifiedbythe rootkernelparameter(initrdinourcase) Executesthefirstuserspaceprogram:usually /linuxrc

Userspace:/linuxrcscriptininitrd(whatfollowsisjustatypicalsequence)

Runsuserspacecommandstoconfigurethedevice(suchasnetworksetup,mounting /procand/sys...) Loadskernelmodules(drivers)storedintheinitrd,neededtoaccessthenewrootfilesystem. Mountsthenewrootfilesystem.Switchtoit( pivot_root) Runs/sbin/init(orsometimesanew/linuxrcscript)

Userspace:/sbin/init

Runscommandstoconfigurethedevice(ifnotdoneyetintheinitrd) Startsupsystemservices(daemons,servers)anduserprograms

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

118

Linux2.4bootingsequencedrawbacks
Tryingtomountthefilesystemspecified bytherootkernelparameteriscomplex: Needdeviceandfilesystemdriverstobeloaded. Specifyingtherootfilesystemrequiresuglyblackmagicdevice naming(suchas/dev/ram0,/dev/hda1...),while/doesn't existyet! Canrequireacomplexinitializationtoimplementwithinthe kernel.Examples:NFS(setupanIPaddress,connecttothe server...),RAID(rootfilesystemonmultiplephysicaldrives)... Inanutshell:toomuchcomplexityinkernelcode!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

119

Extrainitramdiskdrawbacks(1)
Initramdisksareimplementedasstandardblockdevices Needaramdiskandfilesystemdriver. Fixedinsize:cannoteasilygrowinsize. Anyfreespacecannotbereusedbyanythingelse. Needstobecreatedandmodifiedlikeanyblockdevice: formatting,mounting,editing,unmounting. Rootpermissionsneeded. Likeinanyblockdevice,filesarefirstreadfromthestorage, andthencopiedtothefilecache. SlowandduplicationinRAM!!!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

120

Extrainitramdiskdrawbacks(2)
initrdisaseparetefilewhileinitramfsismanagedwiththeLinux kernelimage. initrdfileisagzippedfilesystemimage(itneedsadriverbuiltinto thekernel),whilethenewinitramfsarchiveisagzippedcpio archive(liketaronlysimpler).Thekernel'scpioextractioncode isextremelysmall,it'salso__inittextanddatathatcanbe discardedduringthebootprocess. Theprogramrunbytheinitrddoessomesetupandthenreturnsto thekernel,whiletheinitprogramfrominitramfsisnotexpectedto returntothekernel.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

121

Initramfsfeaturesandadvantages(1)
Rootfilesystembuiltinthekernelimage (embeddedasacompressedcpioarchive). Veryeasytocreate(atkernelbuildtime). Noneedforrootpermissions(formountandmknod). Comparedtoinitramdisks, just1filetohandleinthebootloader. AlwayspresentintheLinux2.6kernel(emptybydefault). Justaplaincompressedcpioarchive. Neitherneedsablocknorafilesystemdriver.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

122

Initramfsfeaturesandadvantages(2)
ramfs:implementedinthefilecache.NoduplicationinRAM,nofilesystem layertomanage.Justusesthesizeofitsfiles.Cangrowifneeded.
Access tofile VirtualFile System Filesystem driver Block(disk) driver Copy Blockstorage File cache

Regular blockdevice

Access tofile VirtualFile System Filesystem driver Block (ramdisk) driver

Ramdisk blockdevice

Access tofile VirtualFile System

ramfs

File cache Copy Blockstorage RAM

File cache

RAM

RAM

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

123

Initramfsfeaturesandadvantages(3)
Loadedbythekernelearlier. Moreinitializationcodemovedtouserspace! Simplertomountcomplexfilesystemsfromflexibleuserspace scriptsratherthanfromrigidkernelcode.Morecomplexity movedouttouserspace! Nomoremagicnamingoftherootdevice. pivot_rootnolongerneeded.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

124

Initramfsfeaturesandadvantages(4)
PossibletoaddnonGPLfiles(firmware,proprietarydrivers) inthefilesystem.Thisisnotlinking,justfileaggregation (notconsideredasaderivedworkbytheGPL). Possibilitytoremovethesefileswhennolongerneeded. Moretechnicaldetailsaboutinitramfs: seeDocumentation/filesystems/ramfsrootfsinitramfs.txt andDocumentation/earlyuserspace/READMEinkernelsources. Seealsohttp://www.linuxdevices.com/articles/AT4017834659.htmlforanice overviewofinitramfs(byRobLandley,theexnewBusyboxmaintainer).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

125

Howtopopulateaninitramfs
UsingCONFIG_INITRAMFS_SOURCE inkernelconfiguration(GeneralSetupsection) Eithergiveanexistingcpioarchive. Orgivealistoffilesordirectories tobeaddedtothearchive. Orgiveatextspecificationfile(seenextpage). YoucanbuildyourinitramfswithatinyClibrary andthetinyexecutablesitships: klibc:http://en.wikipedia.org/wiki/Klibc

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

126

Initramfsspecificationfileexample
dir/dev75500 nod/dev/console64400c51 permissions nod/dev/loop064400b70 dir/bin75510001000 file/bin/busybox/stuff/initramfs/busybox75500 slink/bin/shbusybox77700 dir/proc75500 dir/sys75500 dir/mnt75500 file/init/stuff/initramfs/init.sh75500 Noneedforrootuseraccess! userid groupid major minor

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

127

Howtohandlecompressedcpioarchives
Usefulwhenyouwanttobuildthekernelwithareadymadecpio archive.Betterletthekerneldothisforyou!

Extracting: gzipdcinitramfs.img|cpioid Creating:

find<dir>printdepth|cpioov|gzipc> initramfs.img

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

128

Howtocreateaninitrd
Incaseyouneedaninitrd: sudomkdir/mnt/initrd ddif=/dev/zeroof=initrd.imgbs=1kcount=2048 mkfs.ext2Finitrd.img sudomountoloopinitrd.img/mnt/initrd Filltheramdiskcontents:BusyBox,modules,/linuxrcscript MoredetailsintheFreeSoftwaretoolsforembeddedsystemstraining! sudoumount/mnt/initrd gzipbestcinitrd.img>initrd MoredetailsonDocumentation/initrd.txtinthekernel sources!Alsoexplainspivotrooting.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

129

Bootingvariants
XIP(ExecuteInPlace) Thekernelimageisdirectlyexecutedfromthestorage CanbefasterandsaveRAM However,thekernelimagecan'tbecompressed Noinitramfs/initrd Directlymountingthefinalrootfilesystem (rootkernelcommandlineoption) Nonewrootfilesystem Runningthewholesystemfromtheinitramfs.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

130

EmbeddedLinuxdevelopment

CompilingandbootingLinux
Bootloaders

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

131

2stagebootloaders
Atstartup,thehardwareautomaticallyexecutesthebootloader viaXIPfromagivenlocation,usuallywithverylittlespace (suchasthebootsectoronaPCharddisk) Becauseofthislackofspace,2stagesareimplemented:
1st stage:minimumfunctionality.ConfigureRAMcontrollerand someCPUregisters.RelocatethecodeandjumpinRAM. 2nd stage:offersthefullbootloaderfunctionality.Nolimitinwhat canbeimplemented.Canevenbeanoperatingsystemitself!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

132

x86bootloaders(PCworld)
LILO:LInuxLOad.OriginalLinuxbootloader.Nowrare. http://freshmeat.net/projects/lilo/ Supports:x86 GRUB:GRandUnifiedBootloaderfromGNU.Morepowerful. http://www.gnu.org/software/grub/ Supports:x86 SeeourGrubdetailsannexfordetails. SYSLINUX:Utilitiesfornetworkandremovablemediabooting http://syslinux.zytor.com Supports:x86

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

133

Bootloadersforembeddedworld
DasUBoot:UniversalBootloaderfromDenxSoftware Themostusedonarm. http://www.denx.de/wiki/UBoot/WebHome Supports:arm,powerpc,mips,x86,m68k,nios... SeeUbootdetailsannexfordetails. RedBoot:eCosbasedbootloaderfromRedHat http://sources.redhat.com/redboot/ Supports:x86,arm,powerpc,mips,sh,m68k... uMon:MicroMonitorgeneralpurpose,multiOSbootloader http://microcross.com/html/micromonitor.html Supports:ARM,ColdFire,SH2,m68k,MIPS,PowerPC,Xscale...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

134

EmbeddedLinuxdriverdevelopment

CompilingandbootingLinux
Kernelbooting

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

135

Kernelcommandlineparameters
LikemostCprograms,theLinuxkernelaccepts commandlinearguments Kernelcommandlineargumentsarepartofthebootloader configurationsettings. Usefultomodifythebehaviorofthekernel atboottime,withouthavingtorecompileit. Usefultoperformadvancedkernelanddriverinitialization, withouthavingtousecomplexuserspacescripts.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

136

Kernelcommandlineexample
HPiPAQh2200PDAbootingexample: root=/dev/ram0\ rw\ init=/linuxrc\ console=ttyS0,115200n8\ console=tty0\ ramdisk_size=8192\ cachepolicy=writethrough Rootfilesystem(firstramdisk) Rootfilesystemmountingmode Firstuserspaceprogram Console(serial) Otherconsole(framebuffer) Miscparameters...

Hundredsofcommandlineparametersdescribedon Documentation/kernelparameters.txt

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

137

UsefulnessofrootfsonNFS
Oncenetworkingworks,yourrootfilesystemcouldbeadirectoryon yourGNU/Linuxdevelopmenthost,exportedbyNFS(NetworkFile System).Thisisveryconvenientforsystemdevelopment: Makesitveryeasytoupdatefiles(drivermodulesinparticular)on therootfilesystem,withoutrebooting.Muchfasterthanthroughthe serialport. Canhaveabigrootfilesystemevenifyoudon'thavesupportfor internalorexternalstorageyet. Therootfilesystemcanbehuge.Youcanevenbuildnativecompiler toolsandbuildallthetoolsyouneedonthetargetitself(betterto crosscompilethough).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

138

NFSbootsetup(1)
Onthehost(NFSserver) Addthebelowlinetoyour/etc/exportsfile:
/home/rootfs192.168.0.202(rw,no_root_squash,no_subtree_check)

StartorrestartyourNFSserver(Example:Debian,Ubuntu)
/etc/init.d/nfskernelserverrestart

clientaddress

NFSserveroptions

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

139

NFSbootsetup(2)
Onthetarget(NFSclient) CompileyourkernelwithCONFIG_NFS_FS=y, CONFIG_IP_PNP=y(configureIPatboottime) andCONFIG_ROOT_NFS=y Bootthekernelwiththebelowcommandlineoptions:
root=/dev/nfs virtualdevice ip=192.168.1.111:192.168.1.110:192.168.1.100:255.255.255.0:at91:eth0 localIPaddress serverIPaddress gateway netmask hostnamedevice nfsroot=192.168.1.110:/home/nfsroot NFSserverIPaddressDirectoryontheNFSserver

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

140

Firstuserspaceprogram
Specifiedbytheinitkernelcommandlineparameter Examples:init=/bin/shorinit=/sbin/init Executedattheendofbootingbythekernel. Takescareofstartingallotheruserspaceprograms (systemservicesanduserprograms). Getsthe1processnumber(pid) Parentorancestorofalluserspaceprograms Thesystemwon'tletyoukillit.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

141

/linuxrc
Programexecutedbydefaultwhenbootingfromaninit ramdiskandnoinitparameterisgiventothekernel. Ismostofthetimeashellscript,basedonaverylightweight shellsuchasnashandbusyboxsh Thisscriptcanimplementcomplextasks:detectingdriversto load,settingupnetworking,mountingpartitions,switching toanewrootfilesystem.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

142

Theinitprogram
/sbin/initistheseconddefaultinitprogram Takescareofstartingsystemservices,andeventuallytheuser interfaces(sshd,Xserver...). Alsotakescareofstoppingsystemservices. Lightweight,partialimplementationavailablethroughBusyBox. SeetheInitrunlevelsannexsectionformoredetailsaboutstarting andstoppingsystemserviceswithinit. However,simplestartupscriptsareoftensufficient inembeddedsystems.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

143

EmbeddedLinuxdriverdevelopment

Driverdevelopment
Loadablekernelmodules

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

144

Loadablekernelmodules(1)
Modules:addagivenfunctionalitytothekernel(drivers, filesystemsupport,andmanyothers). Canbeloadedandunloadedatanytime,onlywhentheir functionalityisneed.Onceloaded,havefullaccesstothe wholekerneladdressspace.Noparticularprotection. Usefultokeepthekernelimagesizetotheminimum (essentialinGNU/LinuxdistributionsforPCs).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

145

Loadablekernelmodules(2)
Usefultodeliverbinaryonlydrivers(badidea) withouthavingtorebuildthekernel. Modulesmakeiteasytodevelopdriverswithoutrebooting: load,test,unload,rebuild,load...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

146

Symbolsexportedtomodules
Fromakernelmodule,onlyalimitednumberofkernelfunctionscan becalled. Functionsandvariableshavetobeexplicitlyexported bythekerneltobevisiblefromakernelmodule. Twomacrosareusedinthekerneltoexportfunctionsandvariables: EXPORT_SYMBOL(symbolname),whichexportsafunctionor variabletoallmodules EXPORT_SYMBOL_GPL(symbolname),whichexportsa functionorvariableonlytoGPLmodules Anormaldrivershouldnotneedanynonexportedfunction.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

147

Moduledependencies
Moduledependenciesstoredin /lib/modules/<version>/modules.dep Theyareautomaticallycomputedduringkernelbuildingfrommodule exportedsymbols.module2dependsonmodule1ifmodule2 usesasymbolexportedbymodule1. Example:usb_storagedependsonusbcore, becauseitusessomeofthefunctionsexportedbyusbcore. Youcanalsoupdatethemodules.depfile byyourself,byrunning(asroot): depmoda[<version>]

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

148

hellomodule
/*hello.c*/ /*hello.c*/ #include<linux/init.h> #include<linux/init.h> #include<linux/module.h> #include<linux/module.h> #include<linux/kernel.h> #include<linux/kernel.h> staticint__inithello_init(void) {staticint__inithello_init(void) { printk(KERN_ALERT"Goodmorrow"); printk(KERN_ALERT"Goodmorrow"); printk(KERN_ALERT"tothisfairassembly.\n"); printk(KERN_ALERT"tothisfairassembly.\n"); return0; }return0; } staticvoid__exithello_exit(void) {staticvoid__exithello_exit(void) { printk(KERN_ALERT"Alas,poorworld,whattreasure"); printk(KERN_ALERT"Alas,poorworld,whattreasure"); printk(KERN_ALERT"hastthoulost!\n"); }printk(KERN_ALERT"hastthoulost!\n"); } module_init(hello_init); module_init(hello_init); module_exit(hello_exit); module_exit(hello_exit); MODULE_LICENSE("GPL"); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Greetingmodule"); MODULE_DESCRIPTION("Greetingmodule"); MODULE_AUTHOR("WilliamShakespeare"); MODULE_AUTHOR("WilliamShakespeare");

__init: removedafterinitialization (statickernelormodule). __exit:discardedwhen modulecompiledstatically intothekernel.

Exampleavailableonhttp://freeelectrons.com/doc/c/hello.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

149

Modulelicenseusefulness
Usedbykerneldeveloperstoidentifyissuescomingfrom proprietarydrivers,whichtheycan'tdoanythingabout (Taintedkernelnoticeinkernelcrashesandoopses). Usefulforuserstocheckthattheirsystemis100%free (check/proc/sys/kernel/tainted). UsefulforGNU/Linuxdistributorsfortheirreleasepolicy checks.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

150

Possiblemodulelicensestrings
Availablelicensestringsexplainedininclude/linux/module.h GPL GNUPublicLicensev2orlater GPLv2 GNUPublicLicensev2 GPLandadditional rights DualMIT/GPL GNUPublicLicensev2orMIT DualBSD/GPL GNUPublicLicensev2orBSD DualMPL/GPL GNUPublicLicensev2 orMozilla Proprietary Nonfreeproducts

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

151

Compilingamodule
ThebelowMakefileshouldbereusableforanyLinux2.6module. Justrunmaketobuildthehello.kofile Caution:makesurethereisa[Tab]characteratthebeginningof the$(MAKE)line(makesyntax).
#Makefileforthehellomodule #Makefileforthehellomodule objm:=hello.o objm:=hello.o KDIR:=/lib/modules/$(shellunamer)/build KDIR:=/lib/modules/$(shellunamer)/build PWD:=$(shellpwd) PWD:=$(shellpwd) all: all: $(MAKE)C$(KDIR)SUBDIRS=$(PWD)modules $(MAKE)C$(KDIR)SUBDIRS=$(PWD)modules Exampleavailableonhttp://freeelectrons.com/doc/c/Makefile
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

[Tab]! (nospaces)

Either fullkernelsource directory (configuredand compiled) orjustkernel headersdirectory (minimumneeded)

152

Kernellog
Ofcourse,thekerneldoesn'tstoreitslogintoafile! Filesbelongtouserspace. Thekernelkeepsprintkmessagesinacircularbuffer (sothatdoesn'tconsumemorememorywithmanymessages). Kernellogmessagescanbeaccessedfromuserspacethroughsystem calls,orthrough/proc/kmsg Kernellogmessagesarealsodisplayedinthesystemconsole.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

153

Accessingthekernellog
Manywaysareavailable!
Watchthesystemconsole syslogd/klogd Daemongatheringkernelmessages in/var/log/messages Followchangesbyrunning: tailf/var/log/messages Caution:thisfilegrows! Uselogrotatetocontrolthis cat/proc/kmsg Waitsforkernelmessagesand displaysthem. Usefulwhennoneoftheabove userspaceprogramsareavailable (tinysystem) dmesg(diagnosticmessage) Foundinallsystems Displaysthekernellogbuffer

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

154

Usingthemodule
Loadthemodule: sudoinsmod./hello.ko Youwillseethefollowinginthekernellog: Goodmorrow tothisfairassembly Nowremovethemodule: sudormmodhello Youwillsee: Alas,poorworld,whattreasure hastthoulost!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

155

Understandingmoduleloadingissues
Whenloadingamodulefails, insmodoftendoesn'tgiveyouenoughdetails! Detailsareavailableinthekernellog. Example:
>sudoinsmod./intr_monitor.ko insmod:errorinserting'./intr_monitor.ko':1 Deviceorresourcebusy >dmesg [17549774.552000]Failedtoregisterhandlerfor irqchannel2

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

156

Moduleutilities(1)
modinfo<module_name> modinfo<module_path>.ko Getsinformationaboutamodule:parameters,license, descriptionanddependencies. Veryusefulbeforedecidingtoloadamoduleornot. sudoinsmod<module_path>.ko Triestoloadthegivenmodule.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

157

Moduleutilities(2)
sudomodprobe<module_name> Mostcommonusageofmodprobe:triestoloadallthe modulesthegivenmoduledependson,andthenthismodule. Lotsofotheroptionsareavailable. lsmod Displaysthelistofloadedmodules Compareitsoutputwiththecontentsof/proc/modules!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

158

Moduleutilities(3)
sudormmod<module_name> Triestoremovethegivenmodule sudomodprober<module_name> Triestoremovethegivenmoduleandalldependentmodules (whicharenolongerneededafterthemoduleremoval)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

159

Createyourmoduleswithkdevelop
http://kdevelop.orgAvailableinmostdistros. Makesiteasytocreate amodulecodeskeleton fromareadymade template. Canalsobeusedto compileyourmodule.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

160

EmbeddedLinuxdevelopment

Driverdevelopment
Moduleparameters

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

161

hellomodulewithparameters
/*hello_param.c*/ /*hello_param.c*/ #include<linux/init.h> #include<linux/init.h> #include<linux/module.h> #include<linux/module.h> #include<linux/moduleparam.h> #include<linux/moduleparam.h> MODULE_LICENSE("GPL"); MODULE_LICENSE("GPL"); /*Acoupleofparametersthatcanbepassedin:howmanytimeswesay /*Acoupleofparametersthatcanbepassedin:howmanytimeswesay hello,andtowhom*/ hello,andtowhom*/ staticchar*whom="world"; staticchar*whom="world"; module_param(whom,charp,0); module_param(whom,charp,0); staticinthowmany=1; staticinthowmany=1; module_param(howmany,int,0); module_param(howmany,int,0); staticint__inithello_init(void) { staticint__inithello_init(void) { inti; inti; for(i=0;i<howmany;i++) for(i=0;i<howmany;i++) printk(KERN_ALERT"(%d)Hello,%s\n",i,whom); printk(KERN_ALERT"(%d)Hello,%s\n",i,whom); return0; } return0; } staticvoid__exithello_exit(void) { staticvoid__exithello_exit(void) { printk(KERN_ALERT"Goodbye,cruel%s\n",whom); } printk(KERN_ALERT"Goodbye,cruel%s\n",whom); } module_init(hello_init); module_init(hello_init); module_exit(hello_exit); module_exit(hello_exit);

Thanksto JonathanCorbet fortheexample!

Exampleavailableonhttp://freeelectrons.com/doc/c/hello_param.c

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

162

Passingmoduleparameters
Throughinsmod: sudoinsmod./hello_param.kohowmany=2whom=universe Throughmodprobe: Setparametersin/etc/modprobe.conforinanyfilein/etc/modprobe.d/: optionshello_paramhowmany=2whom=universe Throughthekernelcommandline, whenthemoduleisbuiltstaticallyintothekernel: optionshello_param.howmany=2hello_param.whom=universe

modulename moduleparametername moduleparametervalue

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

163

Declaringamoduleparameter
#include<linux/moduleparam.h> module_param( name, /*nameofanalreadydefinedvariable*/ type, /*eitherbyte,short,ushort,int,uint,long, ulong,charp,orbool. (checkedatcompiletime!)*/ perm /*for/sys/module/<module_name>/parameters/<param> 0:nosuchmoduleparametervaluefile*/ ); Example intirq=5; module_param(irq,int,S_IRUGO);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

164

Declaringamoduleparameterarray
#include<linux/moduleparam.h> module_param_array( name, /*nameofanalreadydefinedarray*/ type, /*sameasinmodule_param*/ num, /*numberofelementsinthearray,orNULL(nocheck?)*/ perm /*sameasinmodule_param*/ ); Example staticintbase[MAX_DEVICES]={0x820,0x840}; module_param_array(base,int,NULL,0);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

165

EmbeddedLinuxdevelopment

Driverdevelopment
Addingsourcestothekerneltree

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

166

Newdriverinkernelsources(1)
Toaddanewdrivertothekernelsources: Addyournewsourcefiletotheappropriatesourcedirectory. Example:drivers/usb/serial/navman.c Describetheconfigurationinterfaceforyournewdriver byaddingthefollowinglinestotheKconfigfileinthisdirectory:
configUSB_SERIAL_NAVMAN tristate"USBNavmanGPSdevice" dependsonUSB_SERIAL help Tocompilethisdriverasamodule,chooseMhere:the modulewillbecallednavman.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

167

Newdriverinkernelsources(2)
AddalineintheMakefilefilebasedontheKconfigsetting:
obj$(CONFIG_USB_SERIAL_NAVMAN)+=navman.o

Runmakexconfigandseeyournewoptions! Runmakeandyournewfilesarecompiled!
SeeDocumentation/kbuild/fordetails

EmbeddedLinuxkernelanddriverdevelopment Copyright2009 2010 ,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20 ,2010

111

HowtocreateLinuxpatches
Downloadthelatestkernelsources Makeacopyofthesesources: rsyncalinux2.6.9rc2/linux2.6.9rc2patch/ Applyyourchangestothecopiedsources,andtestthem. Runmakedistcleantokeeponlysourcefiles. Createapatchfile: diffNurplinux2.6.9rc2/\ linux2.6.9rc2patch/>patchfile Alwayscomparethewholesourcestructures (suitableforpatchp1) Patchfilename:shouldrecallwhatthepatchisabout.
ThankstoNicolasRougier(Copyright2003,http://webloria.loria.fr/~rougier/)fortheTuximage

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

169

EmbeddedLinuxdevelopment

Driverdevelopment
Memorymanagement

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

170

Physicalandvirtualmemory
Physicaladdressspace
0xFFFFFFFFF
0xFFFFFFFFF

Virtualaddressspaces
0xFFFFFFFFF 0xC0000000

Kernel

I/Omemory3 I/Omemory2 I/Omemory1

Memory Management Unit

Process1

0x00000000

0x00000000

Flash

MMU

CPU
0xFFFFFFFFF

RAM1 RAM0 Alltheprocesseshavetheir ownvirtualaddressspace,and runasiftheyhadaccesstothe wholeaddressspace.

Kernel

0xC0000000

Process2

0x00000000

0x00000000
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

171

kmallocandkfree
Basicallocators,kernelequivalentsofglibc'smallocandfree.
#include<linux/slab.h> staticinlinevoid*kmalloc(size_tsize,intflags);

size:numberofbytestoallocate flags:priority(seenextpage)
voidkfree(constvoid*objp); Example:(drivers/infiniband/core/cache.c) structib_update_work*work; work=kmalloc(sizeof*work,GFP_ATOMIC); ... kfree(work);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

172

kmallocfeatures
Quick(unlessit'sblockedwaitingformemorytobefreed). Doesn'tinitializetheallocatedarea. TheallocatedareaiscontiguousinphysicalRAM. Allocatesby2nsizes,andusesafewmanagementbytes. So,don'taskfor1024whenyouneed1000!You'dget2048! Caution:driversshouldn'ttrytokmalloc morethan128KB(upperlimitinsomearchitectures). Minimumallocation:32or64bytes(pagesizedependent).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

173

Mainkmallocflags(1)
Definedininclude/linux/gfp.h(GFP:__get_free_pages) GFP_KERNEL Standardkernelmemoryallocation.Mayblock.Fineformostneeds. GFP_ATOMIC RAMallocatedfromcodewhichisnotallowedtoblock(interrupt handlers)orwhichdoesn'twanttoblock(criticalsections).Neverblocks. GFP_USER Allocatesmemoryforuserprocesses.Mayblock.Lowestpriority.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

174

Mainkmallocflags(2)
Extraflags(canbeaddedwith|) __GFP_DMAorGFP_DMA AllocateinDMAzone __GFP_ZERO Returnsazeroedpage. __GFP_NOFAIL Mustnotfail.Nevergivesup. Caution:useonlywhen mandatory! __GFP_NORETRY Ifallocationfails,doesn'ttryto getfreepages. Example: GFP_KERNEL|__GFP_DMA Note:almostonly__GFP_DMA orGFP_DMAusedindevice drivers.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

175

Relatedallocationfunctions
Again,namessimilartothoseofClibraryfunctions staticinlinevoid*kzalloc( size_tsize,gfp_tflags); Zeroestheallocatedbuffer. staticinlinevoid*kcalloc( size_tn,size_tsize,gfp_tflags); Allocatesmemoryforanarrayofnelementsofsizesize, andzeroesitscontents. void*__must_checkkrealloc( constvoid*,size_t,gfp_t); Changesthesizeofthegivenbuffer.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

176

Availableallocators
Memoryisallocatedusingslabs(groupsofoneormorecontinuouspagesfrom whichobjectsareallocated).Severalcompatibleslaballocatorsareavailable: SLAB:original,wellprovenallocatorinLinux2.6. SLOB:muchsimpler.Morespaceefficientbutdoesn'tscalewell.Savesafew hundredsofKBinsmallsystems(dependsonCONFIG_EMBEDDED) SLUB:thenewdefaultallocatorsince2.6.23,simplerthanSLAB,scaling muchbetter(inparticularforhugesystems)andcreatinglessfragmentation.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

177

Slabcachesandmemorypools
Slabcaches:makeitpossibletoallocatemultiple objectsofthesamesize,withoutwastingRAM. Sofar,mainlyusedincoresubsystems, butnotmuchindevicedrivers (exceptUSBandSCSIdrivers) Memorypools:poolsofpreallocatedobjects, toincreasethechancesofallocationstosucceed. Oftenusedwithfilecaches. SeeourSlabcachesandmemorypoolsannexfordetails.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

178

Allocatingbypages
MoreappropriatewhenyouneedbigslicesofRAM: Apageisusually4K,butcanbemadegreaterinsomearchitectures (sh,mips:4,8,16or64K,butnotconfigurableini386orarm). unsignedlongget_zeroed_page(intflags); Returnsapointertoafreepageandfillsitupwithzeros unsignedlong__get_free_page(intflags); Same,butdoesn'tinitializethecontents unsignedlong__get_free_pages(intflags, unsignedintorder); ReturnsapointeronanareaofseveralcontiguouspagesinphysicalRAM. order:log2(<number_of_pages>) Ifvariable,canbecomputedfromthesizewiththeget_orderfunction. Maximum:8192KB(MAX_ORDER=11ininclude/linux/mmzone.h), exceptinafewarchitectureswhenoverwrittenwithCONFIG_FORCE_MAX_ZONEORDER.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

179

Freeingpages
voidfree_page(unsignedlongaddr); voidfree_pages(unsignedlongaddr, unsignedintorder); Needtousethesameorderasinallocation.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

180

vmalloc
vmalloccanbeusedtoobtaincontiguousmemoryzones invirtualaddressspace(evenifpagesmaynotbe contiguousinphysicalmemory). void*vmalloc(unsignedlongsize); voidvfree(void*addr);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

181

Memoryutilities
void*memset(void*s,intc,size_tcount); Fillsaregionofmemorywiththegivenvalue. void*memcpy(void*dest, constvoid*src, size_tcount); Copiesoneareaofmemorytoanother. Usememmovewithoverlappingareas. LotsoffunctionsequivalenttostandardClibraryonesdefinedin include/linux/string.h

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

182

MemorymanagementSummary
Smallallocations kmalloc,kzalloc (andkfree!) Slabcachesandmemorypools Biggerallocations __get_free_page[s], get_zeroed_page, free_page[s] vmalloc,vfree Libclikememoryutilities memset,memcopy, memmove...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

183

EmbeddedLinuxdevelopment

Driverdevelopment
I/Omemoryandports

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

184

RequestingI/Oports
/proc/ioportsexample(x86) 0000001f:dma1 00200021:pic1 00400043:timer0 00500053:timer1 0060006f:keyboard 00700077:rtc 0080008f:dmapagereg 00a000a1:pic2 00c000df:dma2 00f000ff:fpu 0100013f:pcmcia_socket0 01700177:ide1 01f001f7:ide0 03760376:ide1 0378037a:parport0 03c003df:vga+ 03f603f6:ide0 03f803ff:serial 0800087f:0000:00:1f.0 08000803:PM1a_EVT_BLK 08040805:PM1a_CNT_BLK 0808080b:PM_TMR 08200820:PM2_CNT_BLK 0828082f:GPE0_BLK ...

structresource*request_region( unsignedlongstart, unsignedlonglen, char*name); TriestoreservethegivenregionandreturnsNULLif unsuccessful.Example: request_region(0x0170,8,"ide1"); voidrelease_region( unsignedlongstart, unsignedlonglen); Seeinclude/linux/ioport.hand kernel/resource.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

185

Reading/writingonI/Oports
Theimplementationofthebelowfunctions andtheexactunsignedtypecanvaryfromplatformtoplatform! bytes unsignedinb(unsignedport); voidoutb(unsignedcharbyte,unsignedport); words unsignedinw(unsignedport); voidoutw(unsignedcharbyte,unsignedport); "long"integers unsignedinl(unsignedport); voidoutl(unsignedcharbyte,unsignedport);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

186

Reading/writingstringsonI/Oports
OftenmoreefficientthanthecorrespondingCloop, iftheprocessorsupportssuchoperations! bytestrings
voidinsb(unsignedport,void*addr,unsignedlongcount); voidoutsb(unsignedport,void*addr,unsignedlongcount);

wordstrings
voidinsw(unsignedport,void*addr,unsignedlongcount); voidoutsw(unsignedport,void*addr,unsignedlongcount);

longstrings
voidinsl(unsignedport,void*addr,unsignedlongcount); voidoutsl(unsignedport,void*addr,unsignedlongcount);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

187

RequestingI/Omemory
/proc/iomemexample 000000000009efff:SystemRAM 0009f0000009ffff:reserved 000a0000000bffff:VideoRAMarea 000c0000000cffff:VideoROM 000f0000000fffff:SystemROM 001000003ffadfff:SystemRAM 001000000030afff:Kernelcode 0030b000003b4bff:Kerneldata 3ffae0003fffffff:reserved 40000000400003ff:0000:00:1f.1 4000100040001fff:0000:02:01.0 4000100040001fff:yenta_socket 4000200040002fff:0000:02:01.1 4000200040002fff:yenta_socket 40400000407fffff:PCICardBus#03 4080000040bfffff:PCICardBus#03 40c0000040ffffff:PCICardBus#07 41000000413fffff:PCICardBus#07 a0000000a0000fff:pcmcia_socket0 a0001000a0001fff:pcmcia_socket1 e0000000e7ffffff:0000:00:00.0 e8000000efffffff:PCIBus#01 e8000000efffffff:0000:01:00.0 ...

Equivalentfunctionswiththesameinterface
structresource*request_mem_region( unsignedlongstart, unsignedlonglen, char*name); voidrelease_mem_region( unsignedlongstart, unsignedlonglen);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

188

ChoosingI/Oranges
I/Oportandmemoryrangescanbepassedasmodule parameters.Aneasywaytodefinethoseparametersis through/etc/modprobe.conf. Modulescanalsotrytofindfreerangesbythemselves (makingmultiplecallstorequest_regionor request_mem_region.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

189

MappingI/Omemoryinvirtualmemory
ToaccessI/Omemory,driversneedtohaveavirtualaddress thattheprocessorcanhandle. Theioremapfunctionssatisfythisneed: #include<asm/io.h>; void*ioremap(unsignedlongphys_addr, unsignedlongsize); voidiounmap(void*address); Caution:checkthatioremapdoesn'treturnaNULLaddress!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

190

Differenceswithstandardmemory
Readsandwritesonmemorycanbecached. Thecompilermaychoosetowritethevalueinacpuregister, andmayneverwriteitinmainmemory. Thecompilermaydecidetooptimizeorreorderreadand writeinstructions.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

191

AvoidingI/Oaccessissues
CachingonI/Oportsormemoryalreadydisabled,eitherbythe hardwareorbyLinuxinitcode. UsethevolatilestatementinyourCcodetopreventthecompiler fromusingregistersinsteadofwritingtomemory. Memorybarriersaresuppliedtoavoidreordering. Hardwareindependent #include<asm/kernel.h> voidbarrier(void); Onlyimpactsthebehaviorofthe compiler.Doesn'tpreventreordering intheprocessor! Hardwaredependent #include<asm/system.h> voidrmb(void); voidwmb(void); voidmb(void); Safeonallarchitectures!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

192

AccessingI/Omemory
Directlyreadingfromorwritingtoaddressesreturnedbyioremap (pointerdereferencing)maynotworkonsomearchitectures. Usethebelowfunctionsinstead.Theyarealwaysportableandsafe:
unsignedintioread8(void*addr);(samefor16and32) voidiowrite8(u8value,void*addr);(samefor16and32)

Toreadorwriteaseriesofvalues:
voidioread8_rep(void*addr,void*buf,unsignedlongcount); voidiowrite8_rep(void*addr,constvoid*buf,unsignedlongcount);

Otherusefulfunctions:
voidmemset_io(void*addr,u8value,unsignedintcount); voidmemcpy_fromio(void*dest,void*source,unsignedintcount); voidmemcpy_toio(void*dest,void*source,unsignedintcount);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

193

/dev/mem
Usedtoprovideuserspaceapplicationswithdirectaccessto physicaladdresses. Usage:open/dev/memandreadorwriteatgivenoffset. Whatyoureadorwriteisthevalueatthecorresponding physicaladdress. UsedbyapplicationssuchastheXserver towritedirectlytodevicememory. Since2.6.26(x86onlysofar):onlynonRAMcanbe accessed,unlessexplicitlyconfiguredotherwise(forsecurity reasons).
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

194

EmbeddedLinuxdevelopment

Driverdevelopment
Characterdrivers

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

195

Creatingacharacterdriver
Userspaceneeds Thenameofadevicefilein/devtointeract withthedevicedriverthroughregularfile operations(open,read,write,close...) Thekernelneeds
Copyfromuser Copytouser

Userspace
Read buffer read /dev/foo major/minor Write string write

Toknowwhichdriverisinchargeofdevice fileswithagivenmajor/minornumberpair

Foragivendriver,tohavehandlers(file operations)toexecutewhenuserspaceopens, reads,writesorclosesthedevicefile.

Read handler

Write handler

Devicedriver Kernelspace
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

196

Declaringacharacterdriver
Devicenumberregistration Needtoregisteroneormoredevicenumbers(major/minorpairs), dependingonthenumberofdevicesmanagedbythedriver. Needtofindfreeones! Fileoperationsregistration Needtoregisterhandlerfunctionscalledwhenuserspaceprograms accessthedevicefiles:open,read,write,ioctl,close...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

197

Informationonregistereddevices
Registereddevicesarevisiblein/proc/devices:
Characterdevices: Characterdevices: 1mem 1mem 4/dev/vc/0 4/dev/vc/0 4tty 4tty 4ttyS 4ttyS 5/dev/tty 5/dev/tty 5/dev/console 5/dev/console 5/dev/ptmx 5/dev/ptmx 6lp 6lp 10misc 10misc 13input 13input 14sound 14sound ... ... Blockdevices: Blockdevices: 1ramdisk 1ramdisk 3ide0 3ide0 8sd 8sd 9md 9md 22ide1 22ide1 65sd 65sd 66sd 66sd 67sd 67sd 68sd 68sd

Canbeusedto findfreemajor numbers

Major number

Registered name

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

198

dev_tdatatype
Kerneldatatypetorepresentamajor/minornumberpair Alsocalledadevicenumber. Definedin<linux/kdev_t.h> Linux2.6:32bitsize(major:12bits,minor:20bits) Macrotocreatethedevicenumber: MKDEV(intmajor,intminor); Macrotoextracttheminorandmajornumbers: MAJOR(dev_tdev); MINOR(dev_tdev);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

199

Allocatingfixeddevicenumbers
#include<linux/fs.h> intregister_chrdev_region( dev_tfrom, /*Startingdevicenumber*/ unsignedcount, /*Numberofdevicenumbers*/ constchar*name); /*Registeredname*/ Returns0iftheallocationwassuccessful. Example
if(register_chrdev_region(MKDEV(202,128), acme_count,acme)){ printk(KERN_ERRFailedtoallocatedevicenumber\n); ...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

200

Dynamicallocationofdevicenumbers
Safer:havethekernelallocatefreenumbersforyou! #include<linux/fs.h> intalloc_chrdev_region( dev_t*dev, /*Output:startingdevicenumber*/ unsignedbaseminor, /*Startingminornumber,usually0*/ unsignedcount, /*Numberofdevicenumbers*/ constchar*name); /*Registeredname*/ Returns0iftheallocationwassuccessful. Example
if(alloc_chrdev_region(&acme_dev,0,acme_count,acme)){ printk(KERN_ERRFailedtoallocatedevicenumber\n); ...
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

201

Creatingdevicefiles
Issue:youcannolongercreate/deventriesinadvance! Youhavetocreatethemontheflyafterloadingthedriveraccordingto theallocatedmajornumber. Trick:thescriptloadingthemodulecanthenuse/proc/devices and/proc/misc:
module=foo;name=foo;device=foo rmf/dev/$device insmod$module.ko major=`awk"\\$2==\"$name\"{print\\$1}"/proc/devices` minor=`awk"\\$2==\"$name\"{print\\$1}"/proc/misc` mknod/dev/$devicec$major$minor

Better:useudevtocreatethedevicefileautomatically. Seeudev,mdevandhotplugsection.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

202

Fileoperations(1)
Beforeregisteringcharacterdevices,youhavetodefine file_operations(calledfops)forthedevicefiles. Herearethemainones: int(*open)( structinode*,/*Correspondstothedevicefile*/ structfile*);/*Correspondstotheopenfiledescriptor*/ Calledwhenuserspaceopensthedevicefile. int(*release)( structinode*, structfile*); Calledwhenuserspaceclosesthefile.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

203

Thefilestructure
Iscreatedbythekernelduringtheopencall.Representsopenfiles. mode_tf_mode; Thefileopeningmode(FMODE_READand/orFMODE_WRITE) loff_tf_pos; Currentoffsetinthefile. structfile_operations*f_op; Allowstochangefileoperationsfordifferentopenfiles! structdentry*f_dentry Usefultogetaccesstotheinode:f_dentry>d_inode.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

204

Fileoperations(2)
ssize_t(*read)( structfile*, /*Openfiledescriptor*/ __userchar*, /*Userspacebuffertofillup*/ size_t, /*Sizeoftheuserspacebuffer*/ loff_t*); /*Offsetintheopenfile*/ Calledwhenuserspacereadsfromthedevicefile. ssize_t(*write)( structfile*, __userconstchar*, /*Openfiledescriptor*/ /*Userspacebuffertowrite tothedevice*/ size_t, /*Sizeoftheuserspacebuffer*/ loff_t*); /*Offsetintheopenfile*/ Calledwhenuserspacewritestothedevicefile.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

205

Exchangingdatawithuserspace(1)
Indrivercode,youcan'tjustmemcpybetween anaddresssuppliedbyuserspaceand theaddressofabufferinkernelspace! Correspondtocompletelydifferent addressspaces(thankstovirtualmemory) Theuserspaceaddressmaybeswappedouttodisk Theuserspaceaddressmaybeinvalid (userspaceprocesstryingtoaccessunauthorizeddata)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

206

Exchangingdatawithuserspace(2)
Youmustusededicatedfunctionssuchasthefollowingones inyourreadandwritefileoperationscode: include<asm/uaccess.h> unsignedlongcopy_to_user(void__user*to, constvoid*from, unsignedlongn); unsignedlongcopy_from_user(void*to, constvoid__user*from, unsignedlongn); Makesurethatthesefunctionsreturn0! Anotherreturnvaluewouldmeanthattheyfailed.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

207

Fileoperations(3)
int(*ioctl)(structinode*,structfile*, unsignedint,unsignedlong); Canbeusedtosendspecificcommandstothedevice,whichareneither readingnorwriting(e.g.changingthespeedofaserialport,setting videooutputformat,queryingadeviceserialnumber...).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

208

Fileoperationsspecifictoeachopenfile!
Usingthepossibilitytoredefinefileoperationsforeachopenfile.
getPAL video getNTSC video!

Process1 read Openfile1 open

Process2 ioctl:change readfop Openfile2 read

open /dev/video

2differentprocesses canreaddifferentdata fromthesamedevicefile!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

209

Fileoperations(4)
int(*mmap)(structfile*, structvm_area_struct*); Askingfordevicememorytobemapped intotheaddressspaceofauserprocess. Moreinourmmapsection. Thesewerejustthemainones: about25fileoperationscanbeset,correspondingtoall thesystemcallsthatcanbeperformedonopenfiles.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

210

readoperationexample
staticssize_t staticssize_t acme_read(structfile*file,char__user*buf,size_tcount,loff_t*ppos) {acme_read(structfile*file,char__user*buf,size_tcount,loff_t*ppos) { /*Theacme_bufaddresscorrespondstoadeviceI/Omemoryarea*/ /*Theacme_bufaddresscorrespondstoadeviceI/Omemoryarea*/ /*ofsizeacme_bufsize,obtainedwithioremap()*/ /*ofsizeacme_bufsize,obtainedwithioremap()*/ intremaining_size,transfer_size; intremaining_size,transfer_size; remaining_size=acme_bufsize(int)(*ppos);//byteslefttotransfer remaining_size=acme_bufsize(int)(*ppos);//byteslefttotransfer if(remaining_size==0){/*Allread,returning0(EndOfFile)*/ if(remaining_size==0){/*Allread,returning0(EndOfFile)*/ return0; } return0; } /*Sizeofthistransfer*/ /*Sizeofthistransfer*/ transfer_size=min(remaining_size,(int)count); transfer_size=min(remaining_size,(int)count); if(copy_to_user(buf/*to*/,acme_buf+*ppos/*from*/,transfer_size)){ if(copy_to_user(buf/*to*/,acme_buf+*ppos/*from*/,transfer_size)){ returnEFAULT; returnEFAULT; }else{/*Increasethepositionintheopenfile*/ }else{/*Increasethepositionintheopenfile*/ *ppos+=transfer_size; *ppos+=transfer_size; returntransfer_size; returntransfer_size; } }} }

Readmethod

Pieceofcodeavailablein http://freeelectrons.com/doc/c/acme.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

211

writeoperationexample
staticssize_t staticssize_t acme_write(structfile*file,constchar__user*buf,size_tcount,loff_t*ppos) {acme_write(structfile*file,constchar__user*buf,size_tcount,loff_t*ppos) { intremaining_bytes; intremaining_bytes; /*Numberofbytesnotwrittenyetinthedevice*/ /*Numberofbytesnotwrittenyetinthedevice*/ remaining_bytes=acme_bufsize(*ppos); remaining_bytes=acme_bufsize(*ppos); if(count>remaining_bytes){ if(count>remaining_bytes){ /*Can'twritebeyondtheendofthedevice*/ /*Can'twritebeyondtheendofthedevice*/ returnEIO; } returnEIO; } if(copy_from_user(acme_buf+*ppos/*to*/,buf/*from*/,count)){ if(copy_from_user(acme_buf+*ppos/*to*/,buf/*from*/,count)){ returnEFAULT; returnEFAULT; }else{ }else{ /*Increasethepositionintheopenfile*/ /*Increasethepositionintheopenfile*/ *ppos+=count; *ppos+=count; returncount; returncount; } }} }

Writemethod

Pieceofcodeavailablein http://freeelectrons.com/doc/c/acme.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

212

fileoperationsdefinitionexample(3)
Definingafile_operationsstructure:
#include<linux/fs.h> staticstructfile_operationsacme_fops= { .owner=THIS_MODULE, .read=acme_read, .write=acme_write, };

Youjustneedtosupplythefunctionsyouimplemented! Defaultsforotherfunctions(suchasopen,release...) arefineifyoudonotimplementanythingspecial.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

213

Characterdeviceregistration(1)
Thekernelrepresentscharacterdriverswithacdevstructure Declarethisstructureglobally(withinyourmodule): #include<linux/cdev.h> staticstructcdevacme_cdev; Intheinitfunction,initializethestructure: cdev_init(&acme_cdev,&acme_fops);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

214

Characterdeviceregistration(2)
Then,nowthatyourstructureisready,addittothesystem: intcdev_add( structcdev*p, /*Characterdevicestructure*/ dev_tdev, /*Startingdevicemajor/minornumber*/ unsignedcount); /*Numberofdevices*/

Example(continued):
if(cdev_add(&acme_cdev,acme_dev,acme_count)){ printk(KERN_ERRChardriverregistrationfailed\n);

...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

215

Characterdeviceunregistration
Firstdeleteyourcharacterdevice: voidcdev_del(structcdev*p); Then,andonlythen,freethedevicenumber: voidunregister_chrdev_region(dev_tfrom, unsignedcount); Example(continued): cdev_del(&acme_cdev); unregister_chrdev_region(acme_dev,acme_count);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

216

Linuxerrorcodes
Trytoreporterrorswitherrornumbersasaccurateaspossible! Fortunately,macronamesareexplicitandyoucanremember themquickly. Genericerrorcodes: include/asmgeneric/errnobase.h Platformspecificerrorcodes: include/asm/errno.h

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

217

Chardriverexamplesummary(1)
staticvoid*acme_buf; staticvoid*acme_buf; staticintacme_bufsize=8192; staticintacme_bufsize=8192; staticintacme_count=1; staticintacme_count=1; staticdev_tacme_dev; staticdev_tacme_dev; staticstructcdevacme_cdev; staticstructcdevacme_cdev; staticssize_tacme_write(...){...} staticssize_tacme_write(...){...} staticssize_tacme_read(...){...} staticssize_tacme_read(...){...} staticstructfile_operationsacme_fops= staticstructfile_operationsacme_fops= { { .owner=THIS_MODULE, .owner=THIS_MODULE, .read=acme_read, .read=acme_read, .write=acme_write .write=acme_write }; };

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

218

Chardriverexamplesummary(2)
Showshowtohandleerrorsanddeallocateresourcesintherightorder!
staticint__initacme_init(void) {staticint__initacme_init(void) { interr; interr; acme_buf=ioremap(ACME_PHYS, acme_buf=ioremap(ACME_PHYS, acme_bufsize); acme_bufsize); if(!acme_buf){ if(!acme_buf){ err=ENOMEM; err=ENOMEM; gotoerr_exit; gotoerr_exit; } } if(alloc_chrdev_region(&acme_dev,0, if(alloc_chrdev_region(&acme_dev,0, acme_count,acme)){ acme_count,acme)){ err=ENODEV; err=ENODEV; gotoerr_free_buf; gotoerr_free_buf; } } cdev_init(&acme_cdev,&acme_fops); cdev_init(&acme_cdev,&acme_fops); if(cdev_add(&acme_cdev,acme_dev, if(cdev_add(&acme_cdev,acme_dev, acme_count)){ acme_count)){ err=ENODEV; err=ENODEV; gotoerr_dev_unregister; gotoerr_dev_unregister; } }

return0; return0; err_dev_unregister: err_dev_unregister: unregister_chrdev_region( unregister_chrdev_region( acme_dev,acme_count); acme_dev,acme_count); err_free_buf: err_free_buf: iounmap(acme_buf); iounmap(acme_buf); err_exit: err_exit: returnerr; }returnerr; } staticvoid__exitacme_exit(void) {staticvoid__exitacme_exit(void) { cdev_del(&acme_cdev); cdev_del(&acme_cdev); unregister_chrdev_region(acme_dev, unregister_chrdev_region(acme_dev, acme_count); acme_count); iounmap(acme_buf); }iounmap(acme_buf); }

Completeexamplecodeavailableonhttp://freeelectrons.com/doc/c/acme.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

219

Characterdriversummary
Definethefileoperationscallbacksforthedevicefile:read,write,ioctl... Inthemoduleinitfunction,getmajorandminornumberswithalloc_chrdev_region(), initacdevstructurewithyourfileoperationsandaddittothesystemwithcdev_add(). Inthemoduleexitfunction,callcdev_del()andunregister_chrdev_region()

Loadthecharacterdrivermodule In/proc/devices,findthemajornumberituses. Createthedevicefilewiththismajornumber Thedevicefileisreadytouse!

Kernel

Executesthecorrespondingfileoperations

EmbeddedLinuxkernelanddriverdevelopment Copyright2009 2010 ,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20 ,2010

Kernel

Systemuser

Openthedevicefile,read,write,orsendioctl'stoit.

Userspace

Systemadministration

Kernel

Characterdriverwriter

111

EmbeddedLinuxdevelopment

Driverdevelopment
Debugging

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

221

Usefulnessofaserialport
ForpeopleportingLinuxonconsumerdevices(nodevelopmentboard) Mostprocessorsfeatureaserialportinterface(usuallyverywell supportedbyLinux).Justneedthisinterfacetobeconnectedtothe outside. Easywayofgettingthefirstmessagesofanearlykernelversion,even beforeitboots.Aminimumkernelwithonlyserialportsupportis enough. Oncethekernelisfixedandhascompletedbooting,possibletoaccessa serialconsoleandissuecommands. Theserialportcanalsobeusedtotransferfilestothetarget.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

222

Whenyoudon'thaveaserialport
Onthehost Notanissue.YoucangetaUSBtoserialconverter.Usuallyvery wellsupportedonLinux.Thedeviceappearsas/dev/ttyUSB0 onthehost. Onthetarget CheckwhetheryouhaveanIrDAport.It'susuallyaserialporttoo. Youmayalsotrytomanuallyhookuptheprocessorserial interface(checktheelectricalspecificationsfirst!)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

223

Debuggingwithprintk
Universaldebuggingtechniqueusedsincethebeginningof programming(firstfoundincavemendrawings). Printedornotintheconsoleor/var/log/messages accordingtothepriority.Thisiscontrolledbytheloglevel kernelparameter,orthrough/proc/sys/kernel/printk (seeDocumentation/sysctl/kernel.txt) Availablepriorities(include/linux/kernel.h):
#defineKERN_EMERG"<0>"/*systemisunusable*/ #defineKERN_ALERT"<1>"/*actionmustbetakenimmediately*/ #defineKERN_CRIT"<2>"/*criticalconditions*/ #defineKERN_ERR"<3>"/*errorconditions*/ #defineKERN_WARNING"<4>"/*warningconditions*/ #defineKERN_NOTICE"<5>"/*normalbutsignificantcondition*/ #defineKERN_INFO"<6>"/*informational*/ #defineKERN_DEBUG"<7>"/*debuglevelmessages*/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

224

Debuggingwith/procor/sys(1)
Insteadofdumpingmessagesinthekernellog,youcanhaveyour driversmakeinformationavailabletouserspace. Throughafilein/procor/sys,whichcontentsarehandledby callbacksdefinedandregisteredbyyourdriver. Canbeusedtoshowanypieceofinformationaboutyourdevice ordriver. Canalsobeusedtosenddatatothedriverortocontrolit. Caution:anybodycanusethesefiles. Youshouldremoveyourdebugginginterfaceinproduction!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

225

Debuggingwith/procor/sys(2)
Examples cat/proc/acme/stats(dummyexample) Displaysstatisticsaboutyouracmedriver. cat/proc/acme/globals(dummyexample) Displaysvaluesofglobalvariablesusedbyyourdriver.
echo600000>/sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed

AdjuststhespeedoftheCPU(controlledbythecpufreqdriver).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

226

Debugfs
Avirtualfilesystemtoexportdebugginginformationtouserspace. Kernelconfiguration:DEBUG_FS Kernelhacking>DebugFilesystem Muchsimplertocodethananinterfacein/procor/sys. ThedebugginginterfacedisappearswhenDebugfsisconfiguredout. Youcanmountitasfollows: sudomounttdebugfsnone/mnt/debugfs Describedonhttp://lwn.net/Articles/334068/ APIdocumentedintheLinuxKernelFilesystemAPI: http://freeelectrons.com/kerneldoc/latest/DocBook/filesystems/index.html
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

227

Simpledebugfsexample
#include<linux/debugfs.h> #include<linux/debugfs.h> staticchar*acme_buf; staticchar*acme_buf; staticunsignedlongacme_bufsize; staticunsignedlongacme_bufsize; staticstructdebugfs_blog_wrapperacme_blob; staticstructdebugfs_blog_wrapperacme_blob; staticstructdentry*acme_buf_dentry; staticstructdentry*acme_buf_dentry; staticu32acme_state; staticu32acme_state; staticstructdentry*acme_state_dentry; staticstructdentry*acme_state_dentry; //modulebuffer //modulebuffer

//modulevariable //modulevariable

/*Moduleinit*/ /*Moduleinit*/ acme_blob.data=acme_buf; acme_blob.data=acme_buf; acme_blob.size=acme_bufsize; acme_blob.size=acme_bufsize; acme_buf_dentry=debugfs_create_blog("acme_buf",S_IRUGO, acme_buf_dentry=debugfs_create_blog("acme_buf",S_IRUGO, NULL,&acme_blob); NULL,&acme_blob); acme_state_dentry=debugfs_create_u32("acme_state",S_IRUGO, acme_state_dentry=debugfs_create_u32("acme_state",S_IRUGO, NULL,&acme_state); NULL,&acme_state); /*Moduleexit*/ /*Moduleexit*/ debufs_remove(acme_buf_dentry); debufs_remove(acme_buf_dentry); debugfs_remove(acme_state_dentry); debugfs_remove(acme_state_dentry);

//Create //Create //newfiles //newfiles //indebugfs //indebugfs

//removingthefilesfromdebugfs //removingthefilesfromdebugfs

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

228

Debuggingwithioctl
Canusetheioctl()systemcalltoqueryinformation aboutyourdriver(ordevice)orsendcommandstoit. Thiscallstheioctlfileoperationthatyoucanregisterin yourdriver. Advantage:yourdebugginginterfaceisnotpublic. Youcouldevenleaveitwhenyoursystem(oritsdriver)isin thehandsofitsusers.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

229

Debuggingwithgdb
Ifyouexecutethekernelfromadebuggeronthesamemachine, thiswillinterferewiththekernelbehavior. However,youcanaccessthecurrentkernelstatewithgdb: gdb/usr/src/linux/vmlinux/proc/kcore uncompressedkernelkerneladdressspace Youcanaccesskernelstructures,followpointers...(readonly!) RequiresthekerneltobecompiledwithCONFIG_DEBUG_INFO (Kernelhackingsection)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

230

Kgdb
Kgdbisinmainlinesince2.6.26 Theexecutionofthekernelisfullycontrolledbygdbfrom anothermachine,connectedthroughaseriallineorethernet. Candoalmosteverything,includinginsertingbreakpointsin interrupthandlers. Supportedarchitecturessparc,sparc64,x86,powerpc, armandmips.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

231

UsingKgdb(1)
Detailsavailableinthekerneldocumentation: http://freeelectrons.com/kerneldoc/latest/DocBook/kgdb/ RecommendedtoturnonCONFIG_FRAME_POINTERto aidinproducingmorereliablestackbacktracesingdb. YoumustincludeakgdbI/Odriver.Oneofthemiskgdb overserialconsole(kgdboc:kgdboverconsole,enabledby CONFIG_KGDB_SERIAL_CONSOLE) Configurekgdbocatboottimebypassingtothekernel: kgdboc=<ttydevice>,[baud].Forexample: kgdboc=ttyS0,115200
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

232

UsingKgdb(2)
Thenalsopasskgdbwaittothekernel: itmakeskgdbwaitforadebuggerconnection. Nowbootyourkernelanditwillwait.... Onyourworkstation,startgdbasfollows: %gdb./vmlinux (gdb)setremotebaud115200 (gdb)targetremote/dev/ttyS0 Onceconnected,youcandebugakernelthewayyouwould debuganapplicationprogram.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

233

Kernelcrashanalysiswithkexec
kexecsystemcall:makesitpossibleto callanewkernel,withoutrebootingand goingthroughtheBIOS/firmware. Idea:afterakernelpanic,makethe kernelautomaticallyexecuteanew, cleankernelfromareservedlocationin RAM,toperformpostmortemanalysis ofthememoryofthecrashedkernel. Kdump,however,canbeveryram expensive! SeeDocumentation/kdump/kdump.txt inthekernelsourcesfordetails.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

1.Copydebug kernelto reserved RAM 3.Analyze crashed kernelRAM

Standardkernel 2.kernel panic,kexec debugkernel Debugkernel

RegularRAM

234

DebuggingwithSystemTap
http://sourceware.org/systemtap/ Infrastructuretoaddinstrumentationtoarunningkernel: tracefunctions,readandwritevariables,followpointers,gatherstatistics... Eliminatestheneedtomodifythekernelsourcestoaddone'sowninstrumentationto investigatedafunctionalorperformanceproblem. Usesasimplescriptinglanguage. Severalexamplescriptsandprobepointsareavailable. BasedontheKprobesinstrumentationinfrastructure. Linux2.6.20:supportedonmostpopularCPUs. armandmipspatchesavailablefromhttp://elinux.org/Patch_Archive

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

235

SystemTapscriptexample(1)
#!/usr/bin/envstap #!/usr/bin/envstap #Usingstatisticsandmapstoexaminekernelmemoryallocations #Usingstatisticsandmapstoexaminekernelmemoryallocations globalkmalloc globalkmalloc probekernel.function("__kmalloc"){ probekernel.function("__kmalloc"){ kmalloc[execname()]<<<$size kmalloc[execname()]<<<$size } } #Exitafter10seconds #Exitafter10seconds probetimer.ms(10000){exit()} probetimer.ms(10000){exit()} probeend{ probeend{ foreach([name]inkmalloc){ foreach([name]inkmalloc){ printf("Allocationsfor%s\n",name) printf("Allocationsfor%s\n",name) printf("Count:%dallocations\n",@count(kmalloc[name])) printf("Count:%dallocations\n",@count(kmalloc[name])) printf("Sum:%dKbytes\n",@sum(kmalloc[name])/1000) printf("Sum:%dKbytes\n",@sum(kmalloc[name])/1000) printf("Average:%dbytes\n",@avg(kmalloc[name])) printf("Average:%dbytes\n",@avg(kmalloc[name])) printf("Min:%dbytes\n",@min(kmalloc[name])) printf("Min:%dbytes\n",@min(kmalloc[name])) printf("Max:%dbytes\n",@max(kmalloc[name])) printf("Max:%dbytes\n",@max(kmalloc[name])) print("\nAllocationsbysizeinbytes\n") print("\nAllocationsbysizeinbytes\n") print(@hist_log(kmalloc[name])) print(@hist_log(kmalloc[name])) printf("\n\n"); printf("\n\n"); } } } }
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

236

SystemTapscriptexample(2)
#!/usr/bin/envstap #!/usr/bin/envstap #Logseachfilereadperformedbyeachprocess #Logseachfilereadperformedbyeachprocess probekernel.function("vfs_read") probekernel.function("vfs_read") { { dev_nr=$file>f_dentry>d_inode>i_sb>s_dev dev_nr=$file>f_dentry>d_inode>i_sb>s_dev inode_nr=$file>f_dentry>d_inode>i_ino inode_nr=$file>f_dentry>d_inode>i_ino printf("%s(%d)%s0x%x/%d\n", printf("%s(%d)%s0x%x/%d\n", execname(),pid(),probefunc(),dev_nr,inode_nr) execname(),pid(),probefunc(),dev_nr,inode_nr) } }

Nicetutorialonhttp://sources.redhat.com/systemtap/tutorial.pdf
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

237

DebuggingwithFtrace(1)
Itisaselfcontainedsolution,requiringnouserspacetools orsupport. Itisusefulfortrackingdownproblems,notonlyinthe kernel,butinitsinteractionswithuserspaceaswell. Namecomesfrom"functiontracer",whichwasitsoriginal purpose,butitcandomorethanthat. Additionaltracershavebeenaddedtolookatthingslike contextswitches,howlonginterruptsaredisabled,howlong ittakesforhighprioritytaskstorunaftertheyhavebeen wokenup,andsoon.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

238

DebuggingwithFtrace(2)
EnablethefollowingoptioninthekerneltouseFtrace:
CONFIG_FUNCTION_TRACER CONFIG_FUNCTION_GRAPH_TRACER CONFIG_STACK_TRACER CONFIG_DYNAMIC_TRACE

Withdynamictraceoption,mcount()callsarenop,andonly whenthefunction(orfunctiongraph)tracerisenabledthey arereplacedbythetracecode.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

239

UsingFtrace(1)
Viewtraceravailableviadebugfs:
#cat/debug/tracing/available_tracers powerwakeupirqsofffunctionsysprofsched_switch initcallnopfunction_graph

Enablingtracer:
#echofunction>/debug/tracing/current_tracer #echo1>/debug/tracing/tracing_on ...somecommandsoractivitytotrace... #echo0>/debug/tracing/tracing_on

Thisproducesatraceofeachkernelfunctioncalled,along withatimestamp,whichwillallowakernelhackertosee whatisgoingoninsidethekernel.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

240

UsingFtrace(2)
Outputfromftracecanbereadfromoneofseveralfilesin thetracingdirectory:
trace:containshumanreadableoutputofthetrace latency_trace:outputfromthesametrace,butorganizedso thatsystemlatencyissuescanbediagnosed,alsoinhuman readableformat trace_pipe:containsthesameoutputastrace,butismeantto bepipedintoacommand.Unliketheothertwo,readingfrom thisfileconsumestheoutput.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

241

UsingFtrace(3)
tracing_enabledgovernswhetherfunctionentryisrecorded aspartofthetrace.Toenableit: echo1>/debug/tracing/tracing_enabled Wecanusefilterstoselectonlysomefunctionstotraceorto excludethemfromthetraceactivity:
echo'sys_*'>/debug/tracing/set_ftrace_filter echo'sys_*'>/debug/tracing/set_ftrace_nofilter

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

242

Ftraceoutput
tracefile:
#tracer:function # #TASKPIDCPU#TIMESTAMPFUNCTION #||||| bash3330[000]147.799029:sys_open<syscall_call bash3330[000]147.799030:do_sys_open<sys_open bash3330[000]147.799030:getname<do_sys_open bash3330[000]147.799031:kmem_cache_alloc<getname

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

243

Ftraceoutput(2)
latency_tracefile:
#tracer:function # functionlatencytracev1.1.5on2.6.290.215.rc7.fc11.i586 latency:0us,#120119/5425477,CPU#0|(M:desktopVP:0,KP:0,SP:0HP:0#P:2) |task:0(uid:0nice:0policy:0rt_prio:0) #_=>CPU# #/_=>irqsoff #|/_=>needresched #||/_=>hardirq/softirq #|||/_=>preemptdepth #||||/ #|||||delay #cmdpid|||||time|caller #\/|||||\|/ bash33300....3531221us:sys_open(syscall_call) bash33300....3531222us:do_sys_open(sys_open) bash33300....3531222us:getname(do_sys_open) bash33300....3531223us:kmem_cache_alloc(getname)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

244

OtherFtracetracers
power:tracestheCPUpowerstatetransitions function_graph:tracesbothfunctionentryandexit,whichallowsa callgraphtobegenerated sysprof:periodically(governedbysysprof_sample_period) generatestacktracesfortherunningprocessorkernelthread initcall:tracestheentryandexitofinitializationfunctioncalls duringsystemboot branch:tracesbranchpredictionandexecution hwbranchtracer:usestheBranchTargetStack(BTS)featureof x86CPUstotracebranchexecution mmiotrace:tracesmemorymappedI/O
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

245

Usingtrace_printk()(1)
printk()hasaproblem:ifyouaredebuggingahigh volumeareaitcanleadtoboggingdownthesystemorcan evencreatealivelock. Itisalsoquitecommontoseeabug"disappear"whenadding afewprintk().Thisisduetothesheeroverheadthatit introduces!! Writingwithtrace_printk()onlytakesaroundatenth ofamicrosecondorsoinsteadofseveralmillisecondsof printk(). trace_printk()writesinaringbufferandwecansee theoutputviatracefile. Itcanbeusedinanycontext.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

246

Usingtrace_printk()(2)
Forexampleyoucanaddsomethinglikethistothekernelor module: trace_printk("readfoo%d\n",foo);
#cattrace #tracer:nop # #TASKPIDCPU#TIMESTAMPFUNCTION #||||| <...><1000>[001]17279.332920::readfoo15

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

247

Debugginguserspaceinteractions
WithFtracewecaninaveryeasywaydebugapplications andwecanseetheapplication/kernelinteractions. ToperformitourapplicationcanwritedirectlyintheFtrace ringbufferviatrace_markerfile. Noticethatthewritestotrace_markerareshowedupas commentsinthefunctiongraphtracer. Whenweusemulticoreenvironmentitusefultoreadthe traceviaper_cputrace.Theper_cputracefilesarelocatedin thedebugfstracingdirectoryunderper_cpu. ThereexistsatracefileineachoneoftheseCPUdirectories thatonlyshowthetraceforthatCPU.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

248

Ftraceonoops
Whenftrace_dump_on_oopsisenabledinthekernel bootparameters,orbyechoinga"1"into /proc/sys/kernel/ftrace_dump_on_oops, Ftracecandumptotheconsoletheentiretracebufferon oopsorpanic. Havingtheconsoleoutputtoaseriallogmakesdebugging crashesmucheasier.Youcannowtracebacktheeventsthat leduptothecrash. Dumpingtotheconsolemaytakealongtimesincethe defaultFtraceringbufferisoveramegabyteperCPU.To shrinkthesize,writethenumberofkilobytesyouwantthe ringbuffertobetobuffer_size_kb.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

249

UsingMagicSysRq
Linuxalsohas3fingerkeystosaveyourwork Allowstorunmultipledebug/rescuecommandsevenwhenthekernel seemstobeindeeptrouble.Examplecommands: [ALT][SysRq][d]:killsallprocesses,exceptinit. [ALT][SysRq][n]:makesRTprocessesniceable. [ALT][SysRq][s]:attemptstosyncallmountedfilesystems. [ALT][SysRq][b]:immediatelyrebootwithoutsyncingand unmounting. Typicalcombination:[ALT][SysRq][s] andthen[ALT][SysRq][b] DetailedinDocumentation/sysrq.txt
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

250

DebugwithJTAG
FortheFOSScommunity,bestdonewithOpenOCD (OpenOnChipDebugger):http://openocd.berlios.de/web/ OpenOCDsupportsmultipleJTAGhardwaredongles. OpenOCDcaneitherbeinterfacedbyaserialconsole,adebugger,orbyanEclipse basedinterface. Ofcourse,itsupportsallthecapabilitiesofferedbyaJTAGinterface. Seetheverycompletedocumentation:http://openocd.berlios.de/doc/ Foreachboard,you'llneedanOpenOCDconfigurationfile(askyoursupplier) SeeveryusefuldetailsonusingEclipse/gcc/gdb/OpenOCDonWindows: http://www2.amontec.com/sdk4arm/ext/jlynchtutorial20061124.pdfand http://www.yagarto.de/howto/yagarto2/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

251

__log_buf
Incaseofveryearlysystemcrash,thekernelcanhangbefore printingtheerrormessageontheconsole. Toseethemessagesstoredbykernel,butthatdon'tyetprinton theconsole,wecandumpthememorywithJTAG. Themessagesarestoredin__log_buf. Memoryaddresstouseistheaddressof__log_buf,wecan readitfromSystem.mapfile,butwehavetoconvertitfrom virtualtophysical. Toperformthatwecanseethecodeofvirt_to_phys() functionforourarchitectureinthekernel. Wecansetthelengthofthebufferinthekernelconfiguration underGeneralSetupmenu.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

252

LTTng
http://ltt.polymtl.ca/ ThesuccessoroftheLinuxTraceToolkit(LTT) Toolkitallowingtocollectandanalyzetracinginformation fromthekernel,basedonkernelmarkersandkernel tracepoints. Sofar,basedonkernelpatches,butdoingitsbesttousein treesolutions,andtobemergedinthefuture. Veryprecisetimestamps,verylittleoverhead. Usefulguidelinesin http://ltt.polymtl.ca/svn/trunk/lttv/QUICKSTART
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

253

LTTV
ViewerforLTTngtraces Supportforhugetraces(testedwith15GBones). Cancombinemultipletracefilesinasingleview. Graphicalortextinterface. Seehttp://ltt.polymtl.ca/lttvdoc/user_guide/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

254

Morekerneldebuggingtips
EnableCONFIG_KALLSYMS_ALL (GeneralSetup>Configurestandardkernelfeatures) togetoopsmessageswithsymbolnamesinsteadofrawaddresses (thisobsoletestheksymoopstool). Ifyourkerneldoesn'tbootyetorhangswithoutanymessage,youcanactivate LowLeveldebugging(KernelHackingsection,onlyavailableonarm): CONFIG_DEBUG_LL=y Moreaboutkerneldebugginginthefree LinuxDeviceDriversbook(Referencessection)!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

255

Kernelerrors
KernelOops! Softerrorcondition,thesystemusuallywillbestillupand running. KernelPanic Criticalerrorcondition,thesystemwillberebootedaftera configurabletimeout. Sometimesisuseful,togranthighavailabilityandstability, tomanageoopslikekernelpanic.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

256

Quickkerneldebugging(1)
Unable to handle kernel NULL pointer dereference at virtual address 00000e14 printing eip: c0162887 OOPS! *pde = 00000000 Oops: 0000 [#1] PREEMPT Modules linked in: eeprom snd_seq_oss snd_seq_midi_event.......... CPU: 0 EIP: 0060:[<c0162887>] Not tainted EIP information EFLAGS: 00010206 (2.6.7-nf2) EIP is at prune_dcache+0x147/0x1c0 eax: 00000e00 ebx: d1bde050 ecx: f1b3c050 edx: f1b3ac50 esi: f1b3ac40 edi: c1973000 ebp: 00000036 esp: c1973ef8 ds: 007b es: 007b ss: 0068 Process kswapd0 (pid: 65, threadinfo=c1973000 task=c1986050) Stack: d7721178 c1973ef8 0000007a 00000000 c1973000 f7ffea48 c0162d1f 0000007a c0139a2b 0000007a 000000d0 00025528 049dbb00 00000000 000001fa 00000000 c0364564 00000001 0000000a c0364440 c013add1 00000080 000000d0 00000000 Call Trace: Call Trace [<c0162d1f>] shrink_dcache_memory+0x1f/0x30 [<c0139a2b>] shrink_slab+0x14b/0x190 ........................... Code: 8b 50 14 85 d2 75 27 89 34 24 e8 4a 2b 00 00 8b 73 0c 89 1c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

257

Quickkerneldebugging(2)
prune_dcache()isdefinedinfs/dcache.c #objdumpdfs/dcache.o>fs/dcache.disasm #makefs/dcache.s Nowinfs/wehave:
dcache.csourcecode dcache.ocompiledobjectfile dcache.sassembleroutputofCcompiler('halfcompiled' code) dcache.disasmdisasembledobjectfile
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

258

Quickkerneldebugging(3)
Opendcache.disasmandfind"prune_dcache" 00000540<prune_dcache>: 540:55push%ebp Weneedtofindprune_dcache+0x147.Usingshell, #printf"0x%x\n"$((0x540+0x147)) 0x687

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

259

Quickkerneldebugging(4)
andindcache.disasm:
683:85c0test%eax,%eax

685:7407je68e<prune_dcache+0x14e> 687:8b5014mov0x14(%eax),%edx 68a:85d2test%edx,%edx 68c:7527jne6b5<prune_dcache+0x175> 68e:893424mov%esi,(%esp) 691:e8fcffffffcall692<prune_dcache+0x152> 696:8b730cmov0xc(%ebx),%esi 699:891c24mov%ebx,(%esp) 69c:e89ff9ffffcall40<d_free>

OOPS

Comparingwith"Code:8b501485d27527"match!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

260

Quickkerneldebugging(5)
Weneedtofindmatchinglineindcache.sand,eventually,in dcache.c. It'seasytofindprune_dcacheindcache.s:
.L517: movl68(%ebx),%eax testl%eax,%eax je.L532 movl20(%eax),%edx testl%edx,%edx jne.L594

OOPS

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

261

Quickkerneldebugging(6)
NowwecanfindtheassemblerlineintheCcodewitha trick:wecanuseasm(#A),asm(#B)andsoon.
staticinlinevoiddentry_iput(structdentry*dentry)

{ ........................... asm(#A); if(dentry>d_op&&dentry>d_op>d_iput) { asm("#B"); dentry>d_op>d_iput(dentry,inode); } else ..............................

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

262

Quickkerneldebugging(7)
Whichcorrespondstothispartofnewdcache.s:
.L517: #APP #A #NO_APP movl68(%ebx),%eax testl%eax,%eax je.L532 movl20(%eax),%edx testl%edx,%edx jne.L594 .L532: #APP #B #NO_APP.

OOPS

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

263

Quickkerneldebugging(8)

Thisis"if(dentry>d_op&&dentry>d_op>d_iput)" conditioncheck,anditisoopsingtryingtodosecondcheck. dentry>d_opcontainsboguspointervalue0x00000e00!!! InalternativewecanuseafeatureofGCCtobeabletomix Candassemblercode: gcccdcache.cgWa,a,ad>dcache.s Examplefrom:http://kerneltrap.org/node/3648

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

264

Postmortemdebugging
Forthekernel:kdump. Fortheapplications:Elfcoredumps. Dumpscanbereallybig,even100MB! Alternativeapproach:useacustomtooltoselectonly relevantinformation. DisableElfcoredumpssupportinthekernel.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

265

Libcrash
Createadebugprocessconnectedtotheapplicationviapipe. Whenthecrashhappensdebugprocessgathersalltherelevant informationanditsavesthem. TransparentapproachusingLDpreloading. Itusedecentralizedapproach,butitcouldbeusedtocreatea centralprocessthatmangesalltheinformation. See
http://tuxology.net/lectures/crashandburnwritinglinuxapplicationfaulthandlers/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

266

DebuggingwithKprobes
Kprobesareabletocollectdebuggingandperformance informationwithverylowoverheadoveranykernelroutine. Wecaninsertpreandposthandlersoverakernelroutine, caughtitsreturnaddressorbypassitwritinganewonewith thesamesignature. TherearethreetypesofKprobes:
Kernelprobe:itcanbeinsertedonvirtuallyanyinstructioninthekernel. Jumpprobe:itisinsertedattheentrytoakernelfunction,andprovides convenientaccesstothefunction'sarguments. Returnprobe:itfireswhenaspecifiedfunctionreturns.

WecoulduseKprobesascoarsekernelpatchingtechnique.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

267

ConfiguringKprobes
EnableCONFIG_KPROBESunderInstrumentationSupport. NowwecanloadandunloadKprobesbasedmodules! Ifweusethefunctionsymbolnamethenwehavetoenable CONFIG_KALLSYMSandCONFIG_KALLSYMS_ALL orwecanusetheSystemMaptofindtheroutineaddress. Wecaninsertaprobeinthemiddleofafunction,ifwe compilethekernelwithdebuginformation,sowecanuse objdumpdlvmlinuxtoseethesourcetoobject codemapping. Moreinformation:Documentation/Kprobe.txt
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

268

EmbeddedLinuxDevelopment

Driverdevelopment
Processesandscheduling

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

269

Processes
Aprocessisaninstanceofarunningprogram Multipleinstancesofthesameprogramcanberunning. Programcode(textsection)memoryisshared. Eachprocesshasitsowndatasection,addressspace, processorstate,openfilesandpendingsignals. Thekernelhasaseparatedatastructureforeachprocess.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

270

Threads
InLinux,threadsarejustimplementedasprocesses! Newthreadsareimplementedasregularprocesses, withtheparticularitythattheyarecreatedwiththesame addressspace,filesystemresources,filedescriptorsand signalhandlersastheirparentprocess.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

271

Aprocesslife
Parentprocess Parentprocess Callsfork()
Callsfork() andcreates andcreates anewprocess anewprocess Theprocessiselected bythescheduler Taskterminatedbutits resourcesarenotfreedyet. resourcesarenotfreedyet. Waitingforitsparent Waitingforitsparent toacknowledgeitsdeath. toacknowledgeitsdeath.

EXIT_ZOMBIE EXIT_ZOMBIE Taskterminatedbutits

TASK_RUNNING TASK_RUNNING Readybut

Readybut notrunning notrunning Theprocessispreempted bytoschedulertorun ahigherprioritytask

TASK_RUNNING TASK_RUNNING Actuallyrunning

Actuallyrunning

Theeventoccurs ortheprocessreceives asignal.Processbecomes runnableagain

TASK_INTERRUPTIBLE, TASK_INTERRUPTIBLE, TASK_UNINTERRUPTIBLE, TASK_UNINTERRUPTIBLE, TASK_KILLABLE TASK_KILLABLE Waiting

Waiting
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Decidestosleep onawaitqueue foraspecificevent

272

Processcontext
Userspaceprogramsandsystemcallsarescheduledtogether

Processexecutinginuserspace... (canbepreempted) Systemcall orexception

Processcontinuinginuserspace... (orreplacedbyahigherpriorityprocess) (canbepreempted)

Kernelcodeexecuted onbehalfofuserspace (canbepreemptedtoo!)

Stillhasaccesstoprocess data(openfiles...)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

273

Kernelthreads
Thekerneldoesnotonlyreactfromuserspace(systemcalls,exceptions) orhardwareevents(interrupts).Italsorunsitsownprocesses. Kernelthreadsarestandardprocessesscheduledandpreemptedinthesame way(youcanviewthemwithtoporps!)Theyjusthavenospecial addressspaceandusuallyrunforever. Kernelthreadexamples: pdflush:regularlyflushesdirtymemorypagestodisk (filechangesnotcommittedtodiskyet). migration/<n>:PerCPUthreadstomigrateprocessesbetween processors,tobalanceCPUloadbetweenprocessors.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

274

Processpriorities
Regularprocesses Prioritiesfrom20(maximum)to19(minimum) Onlyrootcansetnegativepriorities (rootcangiveanegativeprioritytoaregularuserprocess) Usethenicecommandtorunajobwithagivenpriority: nicen<priority><command> Usetherenicecommandtochangeaprocesspriority: renice<priority>p<pid>

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

275

Realtimepriorities
ProcesseswithrealtimeprioritycanbestartedbyrootusingthePOSIXAPI Availablethrough<sched.h>(seemansched.hfordetails) 100realtimeprioritiesavailable SCHED_FIFOschedulingclass: TheprocessrunsuntilcompletionunlessitisblockedbyanI/O,voluntarily relinquishestheCPU,orispreemptedbyahigherpriorityprocess. SCHED_RRschedulingclass: Difference:theprocessesarescheduledinaRoundRobinway. Eachprocessisrununtilitexhaustsamaxtimequantum.Thenother processeswiththesamepriorityarerun,andsoandso...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

276

Timerfrequency
TimerinterruptsareraisedeveryHZthofsecond(=1jiffy) HZisnowconfigurable(inProcessortypeandfeatures): 100,250(i386default),300or1000(architecturedependent) Seekernel/Kconfig.hz. Compromisebetweensystemresponsivenessandglobalthroughput. Caution:notanyvaluecanbeused.Constraintsapply! AnotherideaistocompletelyturnoffCPUtimerinterruptswhenthesystemis idle(dynamictick).Thiscapabilityisavailablesince2.6.21,togetherwithhigh resolutiontimers. Seeourhttp://freeelectrons.com/articles/realtimepresentationfordetails.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

277

Timeslices
Theschedulerprioritizeshighpriorityprocesses bygivingthemabiggertimeslice. Initialprocesstimeslice:parent'stimeslicesplitin2 (otherwiseprocesswouldcheatbyforking). Minimumpriority:5msor1jiffy(whicheverislarger) Defaultpriorityinjiffies:100ms Maximumpriority:800ms Note:actuallydependsonHZ. Seekernel/sched.cfordetails.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

278

Whenisschedulingrun?
Eachprocesshasaneed_reschedflagwhichisset: Afteraprocessexhausteditstimeslice. Afteraprocesswithahigherpriorityisawakened. Thisflagischecked(possiblycausingtheexecutionofthescheduler) Whenreturningtouserspacefromasystemcall Whenreturningfrominterrupts(includingthecputimer), whenkernelpreemptionisenabled. Schedulingalsohappenswhenkernelcodeexplicitlyruns schedule()orexecutesanactionthatsleeps.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

279

EmbeddedLinuxdevelopment

Driverdevelopment
Sleeping

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

280

Sleeping
Sleepingisneededwhenaprocess(userspaceorkernelspace) iswaitingfordata.
Userspaceprocess... readdevicefile Systemcall... askfor data sleep wakeup Interrupt handler datareadynotification
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Other processes are scheduled ...Systemcall

...Userspace return

281

Howtosleep(1)
Mustdeclareawaitqueue Staticqueuedeclaration DECLARE_WAIT_QUEUE_HEAD(module_queue); Ordynamicqueuedeclaration wait_queue_head_tqueue; init_waitqueue_head(&queue);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

282

Howtosleep(2)
Severalwaystomakeakernelprocesssleep
wait_event(queue,condition); SleepsuntilthegivenCexpressionistrue. Caution:can'tbeinterrupted(i.e.bykillingtheclientprocessinuserspace) wait_event_interruptible(queue,condition); Canbeinterrupted wait_event_timeout(queue,condition,timeout); Sleepsandautomaticallywakesupafterthegiventimeout. wait_event_interruptible_timeout(queue,condition,
timeout);

Sameasabove,interruptible. wait_event_killable(queue,condition); Sameaswait_event,buttheprocesscanreceivethesigkillsignal.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

283

HowtosleepExample
Fromdrivers/ieee1394/video1394.c wait_event_interruptible( d>waitq, (d>buffer_status[v.buffer] ==VIDEO1394_BUFFER_READY) ); if(signal_pending(current)) returnEINTR;
Currentlyrunningprocess

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

284

Wakingup!
Typicallydonebyinterrupthandlerswhendatasleeping processesarewaitingforareavailable. wake_up(queue); Wakesupallthewaitingprocessesonthegivenqueue wake_up_interruptible(queue); Doesthesamejob.Usuallycalledwhenprocesseswaited usingwait_event_interruptible.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

285

Sleepingandwakingupimplementation
Theschedulerdoesn'tkeepevaluatingthesleepingcondition! wait_event_interruptible(queue,condition); TheprocessisputintheTASK_INTERRUPTIBLEstate. wake_up_interruptible(queue); Forallprocesseswaitinginqueue,conditionisevaluated. Whenitevaluatestotrue,theprocessisputback totheTASK_RUNNINGstate,andtheneed_reschedflagfor thecurrentprocessisset. Thisway,severalprocessescanbewokenupatthesametime.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

286

EmbeddedLinuxdevelopment

Driverdevelopment
Interruptmanagement

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

287

Interrupthandlerconstraints
Notrunfromausercontext: Can'ttransferdatatoandfromuserspace (needtobedonebysystemcallhandlers) InterrupthandlerexecutionismanagedbytheCPU,notby thescheduler.Handlerscan'trunactionsthatmaysleep, becausethereisnothingtoresumetheirexecution. Inparticular,needtoallocatememorywithGFP_ATOMIC. Havetocompletetheirjobquicklyenough: theyshouldn'tblocktheirinterruptlinefortoolong.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

288

Registeringaninterrupthandler(1)
Definedininclude/linux/interrupt.h intrequest_irq( Returns0ifsuccessful unsignedintirq, Requestedirqchannel irqreturn_thandler, Interrupthandler unsignedlongirq_flags, Optionmask(seenextpage) constchar*devname, Registeredname void*dev_id); Pointertosomehandlerdata CannotbeNULLandmustbeuniqueforsharedirqs! voidfree_irq(unsignedintirq,void*dev_id); dev_idcannotbeNULLandmustbeuniqueforsharedirqs. Otherwise,onasharedinterruptline, free_irqwouldn'tknowwhichhandlertofree.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

289

Registeringaninterrupthandler(2)
irq_flagsbitvalues(canbecombined,noneisfinetoo) IRQF_DISABLED "Quick"interrupthandler.Runwithallinterruptsdisabledonthecurrentcpu (insteadofjustthecurrentline).Forlatencyreasons,shouldonlybeusedwhen needed! IRQF_SHARED Runwithinterruptsdisabledonlyonthecurrentirqlineandonthelocalcpu. Theinterruptchannelcanbesharedbyseveraldevices. RequiresahardwarestatusregistertellingwhetheranIRQwasraisedornot. IRQF_SAMPLE_RANDOM Interruptscanbeusedtocontributetothesystementropypoolusedby /dev/randomand/dev/urandom.Usefultogenerategoodrandomnumbers. Don'tusethisiftheinterruptbehaviorofyourdeviceispredictable!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

290

Informationoninstalledhandlers
/proc/interrupts
CPU0 0:5616905XTPICtimer#Registeredname 1:9828XTPICi8042 2:0XTPICcascade 3:1014243XTPICorinoco_cs 7:184XTPICIntel82801DBICH4 8:1XTPICrtc 9:2XTPICacpi 11:566583XTPICehci_hcd,uhci_hcd, uhci_hcd,uhci_hcd,yenta,yenta,radeon@PCI:1:0:0 12:5466XTPICi8042 14:121043XTPICide0 15:200888XTPICide1 NMI:0 NonMaskableInterrupts ERR:0 Spuriousinterruptcount
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

291

Totalnumberofinterrupts
cat/proc/stat|grepintr intr819076760929671037701102775520196...
Totalnumber ofinterrupts IRQ1 total IRQ2 IRQ3 total ...

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

292

Theinterrupthandler'sjob
Acknowledgetheinterrupttothedevice (otherwisenomoreinterruptswillbegenerated). Read/writedatafrom/tothedevice. Wakeupanywaitingprocesswaitingforthecompletionof thisread/writeoperation: wake_up_interruptible(&module_queue);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

293

Interrupthandlerprototype
irqreturn_t(*handler)( int, //irqnumberofthecurrentinterrupt void*dev_id, //Pointerusedtokeeptrack //ofthecorrespondingdevice. //Usefulwhenseveraldevices //aremanagedbythesamemodule ); Returnvalue: IRQ_HANDLED:recognizedandhandledinterrupt. IRQ_NONE:notonadevicemanagedbythemodule.Usefultoshare interruptchannelsand/orreportspuriousinterruptstothekernel. IRQ_WAKE_THREAD:seenextslides.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

294

ThreadedInterruptHandling(1)
Startingfrom2.6.30adevicedrivercanrequestthatitsmain interrupthandlerrunsinathread. Newfunctionrequest_threaded_irq()provided. Thehandlerfunctioniscalledinhardinterruptcontextand needstocheckwhethertheinterruptoriginatedfromthe device. Iftheinterruptoriginatedfromthedevicethenthehandler caneitherreturnIRQ_HANDLEDor IRQ_WAKE_THREAD. IRQ_HANDLEDisreturnedwhennofurtheractionis required.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

295

ThreadedInterruptHandling(2)
IRQ_WAKE_THREADinvokesthethreaded(main) handler. WhenIRQ_WAKE_THREADisreturnedhandlermusthave disabledtheinterruptonthedevicelevel. Thisismandatoryforsharedinterrupthandlers,butitneedto doitaswellforobscurex86hardwarewheredisablingan interruptontheIO_APIClevelredirectstheinterrupttothe legacyPICinterruptlines.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

296

Registeringathreadedinterrupthandler
Definedininclude/linux/interrupt.h intrequest_threaded_irq( Returns0ifsuccessful unsignedintirq, Requestedirqchannel irq_handler_thandler, Interrupthandler irq_handler_tthread_fn Interruptthreadfunction unsignedlongirq_flags, Optionmask(seenextpage) constchar*devname, Registeredname void*dev_id); Pointertosomehandlerdata CannotbeNULLandmustbeuniqueforsharedirqs!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

297

Tophalfandbottomhalfprocessing(1)
Splittingtheexecutionofinterrupthandlersin2parts Tophalf:theinterrupthandlermustcompleteasquicklyas possible.Onceitacknowledgedtheinterrupt,itjust schedulesthelengthyrestofthejobtakingcareofthedata, foralaterexecution. Bottomhalf:completingtherestoftheinterrupthandlerjob. Handlesdata,andthenwakesupanywaitinguserprocess. Bestimplementedbytasklets(alsocalledsoftirqs).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

298

tophalfandbottomhalfprocessing(2)
Declarethetaskletinthemodulesourcefile: DECLARE_TASKLET(module_tasklet,/*name*/ module_do_tasklet,/*function*/ 0/*data*/ ); Schedulethetaskletinthetophalfpart(interrupthandler): tasklet_schedule(&module_tasklet); Notethatatasklet_hi_schedulefunctionisavailableto definehighprioritytaskletstorunbeforeordinaryones. Bydefault,taskletsareexecutedrightafteralltophalves(hardirqs)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

299

Disablinginterrupts
Maybeusefulinregulardrivercode... Canbeusefultoensurethataninterrupthandlerwillnotpreemptyour code(includingkernelpreemption). DisablinginterruptsonthelocalCPU:
unsignedlongflags; local_irq_save(flags); //Interruptsdisabled ... local_irq_restore(flags);//Interruptsrestoredtotheirpreviousstate.

Note:mustberunfromwithinthesamefunction!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

300

Maskingoutaninterruptline
Usefultodisableinterruptsonaparticularline voiddisable_irq(unsignedintirq); Disablestheirqlineforallprocessorsinthesystem. Waitsforallcurrentlyexecutinghandlerstocomplete. voiddisable_irq_nosync(unsignedintirq); Same,exceptitdoesn'twaitforhandlerstocomplete. voidenable_irq(unsignedintirq); Restoresinterruptsontheirqline. voidsynchronize_irq(unsignedintirq); Waitsforirqhandlerstocomplete(ifany).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

301

Checkinginterruptstatus
Canbeusefulforcodewhichcanberunfrombothprocessor interruptcontext,toknowwhetheritisallowedornottocall codethatmaysleep. irqs_disabled() Testswhetherlocalinterruptdeliveryisdisabled. in_interrupt() Testswhethercodeisrunningininterruptcontext. in_irq() Testswhethercodeisrunninginaninterrupthandler.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

302

Interruptmanagementsummary
Devicedriver Whenthedevicefileisfirstopen orinthedriverinit,registeran interrupthandlerforthedevice's interruptchannel. Interrupthandler Calledwhenaninterruptisraised. Acknowledgetheinterrupt. Ifneeded,scheduleatasklettaking careofhandlingdata.Otherwise, wakeupprocesseswaitingforthe data. Tasklet Processthedata. Wakeupprocesseswaitingfor thedata. Devicedriver Whenthedeviceisnolonger openedbyanyprocessorinthe driverexit,unregisterthe interrupthandler.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

303

EmbeddedLinuxdevelopment

Driverdevelopment
Concurrentaccesstoresources

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

304

Sourcesofconcurrencyissues
Thesameresourcescanbeaccessedbyseveralkernelprocessesin parallel,causingpotentialconcurrencyissues Severaluserspaceprogramsaccessingthesamedevicedataor hardware.Severalkernelprocessescouldexecutethesamecodeon behalfofuserprocessesrunninginparallel. Multiprocessing:thesamedrivercodecanberunningonanother processor.ThiscanalsohappenwithsingleCPUswithhyperthreading. Kernelpreemption,interrupts:kernelcodecanbeinterruptedatany time(justafewexceptions),andthesamedatamaybeaccessbyanother processbeforetheexecutioncontinues.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

305

Avoidingconcurrencyissues
Avoidusingglobalvariablesandshareddatawheneverpossible (cannotbedonewithhardwareresources). Usetechniquestomanageconcurrentaccesstoresources. SeeRustyRussell'sUnreliableGuideToLocking Documentation/DocBook/kernellocking/ inthekernelsources.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

306

Concurrencyprotectionwithlocks
Process1
Failed Acquirelock Success Criticalcodesection Tryagain Success
Waitlockrelease

Process2

Sharedresource

Releaselock

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

307

Linuxmutexes
ThemainlockingprimitivesinceLinux2.6.16. Betterthancountingsemaphoreswhenbinaryonesareenough. Mutexdefinition: #include<linux/mutex.h> Initializingamutexstatically: DEFINE_MUTEX(name); Orinitializingamutexdynamically: voidmutex_init(structmutex*lock);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

308

lockingandunlockingmutexes
voidmutex_lock(structmutex*lock); Triestolockthemutex,sleepsotherwise. Caution:can'tbeinterrupted,resultinginprocessesyoucannotkill! intmutex_lock_interruptible(structmutex*lock); Same,butcanbeinterrupted.Ifinterrupted,returnsanonzerovalueand doesn'tholdthelock.Testthereturnvalue!!! intmutex_trylock(structmutex*lock); Neverwaits.Returnsanonzerovalueifthemutexisnotavailable. intmutex_is_locked(structmutex*lock); Justtellswhetherthemutexislockedornot. voidmutex_unlock(structmutex*lock); Releasesthelock.Makesureyoudoitasquicklyaspossible!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

309

Reader/writersemaphores
Allowsharedaccessbyunlimitedreaders,orbyonly1writer.Writersget priority. voidinit_rwsem(structrw_semaphore*sem); voiddown_read(structrw_semaphore*sem); intdown_read_trylock(structrw_semaphore*sem); intup_read(structrw_semaphore*sem); voiddown_write(structrw_semaphore*sem); intdown_write_trylock(structrw_semaphore*sem); intup_write(structrw_semaphore*sem); Wellsuitedforrarewrites,holdingthesemaphorebriefly.Otherwise,readersget starved,waitingtoolongforthesemaphoretobereleased.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

310

Whentousemutexesorsemaphores
Beforeandafteraccessingsharedresources. Insituationswhensleepingisallowed. Semaphoresandmutexesmustonlybeusedinprocess context(managedbythescheduler),andnotininterrupt context(managedbytheCPU,sleepingnotsupported).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

311

Spinlocks
Lockstobeusedforcodethatisnotallowedtosleep(interrupt handlers),orthatdoesn'twanttosleep(criticalsections).Bevery carefulnottocallfunctionswhichcansleep! Originallyintendedformultiprocessorsystems. Spinlocksneversleepandkeepspinning inaloopuntilthelockisavailable.
Spinlock Stilllocked?

Spinlockscausekernelpreemptiontobedisabled ontheCPUexecutingthem.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

312

Initializingspinlocks
Static spinlock_tmy_lock=SPIN_LOCK_UNLOCKED; Dynamic voidspin_lock_init(spinlock_t*lock);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

313

Usingspinlocks(1)
Severalvariants,dependingonwherethespinlockiscalled: voidspin_[un]lock(spinlock_t*lock); Doesn'tdisableinterrupts.Usedforlockinginprocesscontext (criticalsectionsinwhichyoudonotwanttosleep). voidspin_lock_irqsave/spin_unlock_irqrestore (spinlock_t*lock,unsignedlongflags); Disables/restoresIRQsonthelocalCPU. Typicallyusedwhenthelockcanbeaccessedinbothprocessand interruptcontext,topreventpreemptionbyinterrupts.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

314

Usingspinlocks(2)
voidspin_[un]lock_bh(spinlock_t*lock); Disablessoftwareinterrupts,butnothardwareones. Usefultoprotectshareddataaccessedinprocesscontext andinasoftinterrupt(bottomhalf).Noneedtodisablehardware interruptsinthiscase. Notethatreader/writerspinlocksalsoexist.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

315

Deadlocksituations
Theycanlockupyoursystem.Makesuretheyneverhappen!
Don'tcallafunctionthatcantry togetaccesstothesamelock Holdingmultiplelocksisrisky!

Getlock1 Getlock1 call Getlock2 Waitforlock1

Dead Lock!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Getlock2

Dead Lock!

Getlock1

316

Kernellockvalidator
FromIngoMolnarandArjanvandeVen Addsinstrumentationtokernellockingcode. Detectviolationsoflockingrulesduringsystemlife,suchas: Locksacquiredindifferentorder (keepstrackoflockingsequencesandcomparesthem). Spinlocksacquiredininterrupthandlersandalsoinprocesscontext wheninterruptsareenabled. Notsuitableforproductionsystemsbutacceptableoverheadin development. SeeDocumentation/lockdepdesign.txtfordetails

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

317

Alternativestolocking
Aswehavejustseen,lockingcanhaveastrongnegativeimpacton systemperformance.Insomesituations,youcoulddowithoutit. ByusinglockfreealgorithmslikeReadCopyUpdate(RCU). Whenavailable,useatomicoperations.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

318

ReadCopyUpdate
Algorithmrequirements:
Referencestosharedresourcesviapointers. Wecanclearlysplittheprocessesinreadersandwriters. Mostoftimetherearereadoperations. Readerscanmanageolddata.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

319

Howitworks(1)
Readerscallrcu_read_lock()beforeaccessingtothe resource. Readerscallrcu_dereference()togetapointertothe resource. Whentheyhavefinished,theycallrcu_read_unlock() Nocontentiononthesefunctionsbecausetheyarenottrue mutexesbuttheyworklikeflags. Tomodifyaresource,writersacquirealock(formorethan onewriter),theycopytheresourceandmodifyit,thenthey claimtheupdate.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

320

Howitworks(2)
Readersalreadyinacriticalsectionwillseetheold resourceyet. Newreaders,aftertheupdate,willseethenewresource whentheywillenterinthecriticalsectionagain. Whenalloldreaderswillexitfromtheircriticalsection, theoldresourcewillbeerased.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

321

UsingRCU
rcu_read_lock() Usedbyreaderstoenterinacriticalsection rcu_read_unlock() Usedbyreaderstoexitfromacriticalsection synchronize_rcu()/call_rcu() Usedbywritertobesynchronizedwithreadersanddeletethe oldresource rcu_assign_pointer() Usedbywritertochangethepointertothenewresource rcu_dereference() Usedbyreaderstogetareferencetotheresource
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

322

RCUReferences
Wikipediaarticle http://en.wikipedia.org/wiki/RCU KernelDocumentationfolder Documentation/RCU/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

323

Atomicvariables
Usefulwhenthesharedresourceisan integervalue. Evenaninstructionliken++isnot guaranteedtobeatomiconallprocessors! Header #include<asm/atomic.h> Type atomic_t containsasignedinteger(atleast24bits)

Operationswithoutreturnvalue:
voidatomic_inc(atomic_t*v); voidatomic_dec(atomic_t*v); voidatomic_add(inti,atomic_t*v); voidatomic_sub(inti,atomic_t*v);

Simularfunctionstestingtheresult:
intatomic_inc_and_test(...); intatomic_dec_and_test(...); intatomic_sub_and_test(...);

Functionsreturningthenewvalue:
intatomic_inc_and_return(...); intatomic_dec_and_return(...); intatomic_add_and_return(...); intatomic_sub_and_return(...);

Atomicoperations(mainones) Setorreadthecounter:
atomic_set(atomic_t*v,inti); intatomic_read(atomic_t*v);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

324

Atomicbitoperations
Supplyveryfast,atomicoperations Onmostplatforms,applytoanunsignedlongtype. Applytoavoidtypeonafewothers. Set,clear,toggleagivenbit: voidset_bit(intnr,unsignedlong*addr); voidclear_bit(intnr,unsignedlong*addr); voidchange_bit(intnr,unsignedlong*addr); Testbitvalue: inttest_bit(intnr,unsignedlong*addr); Testandmodify(returnthepreviousvalue): inttest_and_set_bit(...); inttest_and_clear_bit(...); inttest_and_change_bit(...);
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

325

EmbeddedLinuxdevelopment

Driverdevelopment
mmap

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

326

mmap(1)
Possibilitytohavepartsofthevirtualaddressspaceofaprogram mappedtothecontentsofafile!
>cat/proc/1/maps(initprocess) startend permoffsetmajor:minorinodemappedfilename 007710000077f000rxp0000000003:051165839/lib/libselinux.so.1 0077f00000781000rwp0000d00003:051165839/lib/libselinux.so.1 0097d00000992000rxp0000000003:051158767/lib/ld2.3.3.so 0099200000993000rp0001400003:051158767/lib/ld2.3.3.so 0099300000994000rwp0001500003:051158767/lib/ld2.3.3.so 0099600000aac000rxp0000000003:051158770/lib/tls/libc2.3.3.so 00aac00000aad000rp0011600003:051158770/lib/tls/libc2.3.3.so 00aad00000ab0000rwp0011700003:051158770/lib/tls/libc2.3.3.so 00ab000000ab2000rwp00ab000000:000 0804800008050000rxp0000000003:05571452/sbin/init(text) 0805000008051000rwp0000800003:05571452/sbin/init(data,stack) 08b4300008b64000rwp08b4300000:000 f6fdf000f6fe0000rwpf6fdf00000:000 fefd4000ff000000rwpfefd400000:000 ffffe000fffff000p0000000000:000

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

327

mmap(2)
Particularlyusefulwhenthefileisadevicefile! AllowstoaccessdeviceI/Omemoryandportswithouthavingtogo through(expensive)read,writeorioctlcalls! Xserverexample(mapsexcerpt)
startend permoffsetmajor:minorinodemappedfilename 08047000081be000rxp0000000003:05310295/usr/X11R6/bin/Xorg 081be000081f0000rwp0017600003:05310295/usr/X11R6/bin/Xorg ... f4e08000f4f09000rwse000000003:05655295/dev/dri/card0 f4f09000f4f0b000rws4281a00003:05655295/dev/dri/card0 f4f0b000f6f0b000rwse800000003:05652822/dev/mem f6f0b000f6f8b000rwsfcff000003:05652822/dev/mem

Amoreuserfriendlywaytogetsuchinformation:pmap<pid>

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

328

mmapoverview
mmap system call(once)

Process
access virtual address

Devicedriver
mmapfopcalled initializesthemapping

MMU

access physical address

Processvirtualaddressspace
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Physicaladdressspace

329

HowtoimplementmmapUserspace
Openthedevicefile Callthemmapsystemcall(seemanmmapfordetails): void*mmap( void*start, /*Often0,preferredstartingaddress*/ size_tlength, /*Lengthofthemappedarea*/ intprot, /*Permissions:read,write,execute*/ intflags, /*Options:sharedmapping,privatecopy...*/ intfd, /*Openfiledescriptor*/ off_toffset /*Offsetinthefile*/ ); Yougetavirtualaddressyoucanwritetoorreadfrom.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

330

HowtoimplementmmapKernelspace
Characterdriver:implementammapfileoperation andaddittothedriverfileoperations:
int(*mmap)( structfile*, structvm_area_struct* ); /*Openfilestructure*/ /*KernelVMAstructure*/

Initializethemapping. Canbedoneinmostcaseswiththeremap_pfn_range() function,whichtakescareofmostofthejob.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

331

remap_pfn_range()
pfn:pageframenumber Themostsignificantbitsofthepageaddress (withoutthebitscorrespondingtothepagesize). #include<linux/mm.h> intremap_pfn_range( structvm_area_struct*, /*VMAstruct*/ unsignedlongvirt_addr, /*Startinguservirtualaddress*/ unsignedlongpfn, /*pfnofthestartingphysicaladdress*/ unsignedlongsize, /*Mappingsize*/ pgprot_t /*Pagepermissions*/ );

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

332

Simplemmapimplementation
staticintacme_mmap( structfile*file,structvm_area_struct*vma) { size=vma>vm_startvma>vm_end; if(size>ACME_SIZE) returnEINVAL; if(remap_pfn_range(vma, vma>vm_start, ACME_PHYS>>PAGE_SHIFT, size, vma>vm_page_prot)) returnEAGAIN; return0; }
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

333

devmem2
http://freeelectrons.com/pub/mirror/devmem2.c,byJanDerkBakker Veryusefultooltodirectlypeek(read)orpoke(write)I/Oaddresses mappedinphysicaladdressspacefromashellcommandline! Veryusefulforearlyinteractionexperimentswithadevice,without havingtocodeandcompileadriver. Usesmmapto/dev/mem. Examples(b:byte,h:half,w:word) devmem20x000c0004h(reading) devmem20x000c0008w0xffffffff(writing)

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

334

mmapsummary
Thedevicedriverisloaded. Itdefinesanmmapfileoperation. Auserspaceprocesscallsthemmapsystemcall. Themmapfileoperationiscalled. Itinitializesthemappingusingthedevicephysicaladdress. Theprocessgetsastartingaddresstoreadfromandwriteto(depending onpermissions). TheMMUautomaticallytakescareofconvertingtheprocessvirtual addressesintophysicalones. Directaccesstothehardware! Noexpensivereadorwritesystemcalls!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

335

EmbeddedLinuxdevelopment

Driverdevelopment
DMA

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

336

DMAmemoryconstraints
Needtousecontiguousmemoryinphysicalspace. Canuseanymemoryallocatedbykmalloc(upto128KB). CanuseblockI/Oandnetworkingbuffers, designedtosupportDMA. Maynotusevmallocmemory (wouldhavetosetupDMAoneachindividualpage).

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

337

ReservingmemoryforDMA
Tomakesureyou'vegotenoughRAMforbigDMAtransfers... Exampleassumingyouhave32MBofRAM,andneed2MBfor DMA: Bootyourkernelwithmem=30 Thekernelwilljustusethefirst30MBofRAM. Drivercodecannowreclaimthe2MBleft: dmabuf=ioremap( 0x1e00000, /*Start:30MB*/ 0x200000 /*Size:2MB*/ );

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

338

Memorysynchronizationissues
MemorycachingcouldinterferewithDMA BeforeDMAtodevice: NeedtomakesurethatallwritestoDMAbufferarecommitted. AfterDMAfromdevice: BeforedriversreadfromDMAbuffer,needtomakesurethat memorycachesareflushed. BidirectionalDMA NeedtoflushcachesbeforeandaftertheDMAtransfer.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

339

LinuxDMAAPI
ThekernelDMAutilitiescantakecareof: Eitherallocatingabufferinacachecoherentarea, Ormakesurecachesareflushedwhenrequired, ManagingtheDMAmappingsandIOMMU(ifany). SeeDocumentation/DMAAPI.txt fordetailsabouttheLinuxDMAgenericAPI. Mostsubsystems(suchasPCIorUSB)supplytheirownDMAAPI, derivedfromthegenericone.Maybesufficientformostneeds.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

340

LimitedDMAaddressrange?
Bydefault,thekernelassumesthatyourdevice canDMAtoany32bitaddress.Nottrueforalldevices! Totellthekernelthatitcanonlyhandle24bitaddresses: if(dma_set_mask(dev, /*devicestructure*/ 0x00ffffff /*24bits*/ )) use_dma=1; /*AbletouseDMA*/ else use_dma=0; /*WillhavetodowithoutDMA*/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

341

CoherentorstreamingDMAmappings
Coherentmappings Thekernelallocatesasuitablebuffer andsetsthemappingforthedriver. Cansimultaneouslybeaccessedby theCPUanddevice. So,hastobeinacachecoherent memoryarea. Usuallyallocatedforthewholetime themoduleisloaded. Canbeexpensivetosetupanduse onsomeplatforms. Streamingmappings Thekerneljustsetsthemappingfora bufferprovidedbythedriver. Useabufferalreadyallocated bythedriver. Mappingsetupforeachtransfer. KeepsDMAregistersfreeonthe hardware. Someoptimizationsalsoavailable. Therecommendedsolution.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

342

Allocatingcoherentmappings
Thekerneltakescareofboththebufferallocationandmapping: include<asm/dmamapping.h> void* /*Output:bufferaddress*/ dma_alloc_coherent( structdevice*dev, /*devicestructure*/ size_tsize, /*Neededbuffersizeinbytes*/ dma_addr_t*handle,/*Output:DMAbusaddress*/ gfp_tgfp /*StandardGFPflags*/ ); voiddma_free_coherent(structdevice*dev, size_tsize,void*cpu_addr,dma_addr_thandle);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

343

DMApools(1)
dma_alloc_coherentusuallyallocatesbufferswith __get_free_pages(minimum:1page). YoucanuseDMApoolstoallocatesmallercoherentmappings: <includelinux/dmapool.h> CreateaDMApool: structdma_pool* dma_pool_create( constchar*name, structdevice*dev, size_tsize, size_talign, size_tallocation );

/*Namestring*/ /*devicestructure*/ /*Sizeofpoolbuffers*/ /*Hardwarealignment(bytes)*/ /*Addressboundariesnottobecrossed*/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

344

DMApools(2)
Allocatefrompool void*dma_pool_alloc( structdma_pool*pool, gfp_tmem_flags, dma_addr_t*handle );

Note
DMApoolsonly usedbyUSBcore and2SCSI drivers

Freebufferfrompool voiddma_pool_free( structdma_pool*pool, void*vaddr, dma_addr_tdma); Destroythepool(freeallbuffersfirst!) voiddma_pool_destroy(structdma_pool*pool);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

345

Settingupstreamingmappings
Worksonbuffersalreadyallocatedbythedriver
<includelinux/dmapool.h> dma_addr_tdma_map_single( structdevice*, /*devicestructure*/ void*, /*input:buffertouse*/ size_t, /*buffersize*/ enumdma_data_direction /*EitherDMA_BIDIRECTIONAL, DMA_TO_DEVICEorDMA_FROM_DEVICE*/ ); voiddma_unmap_single(structdevice*dev,dma_addr_t handle,size_tsize,enumdma_data_directiondir);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

346

DMAstreamingmappingnotes
Whenthemappingisactive:onlythedeviceshouldaccessthe buffer(potentialcacheissuesotherwise). TheCPUcanaccessthebufferonlyafterunmapping! UselockingtopreventCPUaccesstothebuffer. Anotherreason:ifrequired,thisAPIcancreateanintermediate bouncebuffer(usedifthegivenbufferisnotusableforDMA). TheLinuxAPIalsosupportsscatter/gatherDMAstreaming mappings.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

347

DMAsummary
MostdriverscanusethespecificAPIprovidedbytheir subsystem:USB,PCI,SCSI...OtherwisetheycanusetheLinuxgenericAPI:
Coherentmappings DMAbufferallocatedbythekernel Setupforthewholemodulelife Canbeexpensive.Notrecommended. LetboththeCPUanddevice accessthebufferatthesametime. Mainfunctions: dma_alloc_coherent dma_free_coherent Streamingmappings DMAbufferallocatedbythedriver Setupforeachtransfer Cheaper.SavesDMAregisters. Onlythedevicecanaccessthebuffer whenthemappingisactive. Mainfunctions: dma_map_single dma_unmap_single

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

348

EmbeddedLinuxdevelopment

Driverdevelopment
Asyncoperations

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

349

Workqueue
Allowskernelcodetodeferprocessingtoalatertime. Workqueuesarecharacterizedbytheexistenceofoneormore dedicatedprocesseswhichexecutequeuedjobs. Workqueuescanalsodelaytheexecutionofspecificjobsfora callerspecifiedperiod. Taskletaresoftirq=>interruptcontext Workqueuesareexecutedbykernelthreads=>processcontext Workqueuescansleeponamutex.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

350

Usingworkqueue
Runtimeinittheworkqueueforthefirsttime
INIT_WORK(structwork_structwork,work_func_t func);

Wherework_func_tis
typedefvoid(*work_func_t)(structwork_struct *work);

Createasingleworkqueuethread
create_singlethread_workqueue(name); Createarealtimeworkqueuethread create_rt_workqueue(name);

Queuethework
intqueue_work(structworkqueue_struct*queue, structwork_struct*work);
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

351

AsyncI/O
TwokindofasyncI/O:
fasyncmethod. Aiointerface(POSIX1.bstandard).

VeryusefultoperformnonblockingI/Ofromapplication pointofview. Applicationscouldparallelizetheirworkflow.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

352

Fasync
Driversmustimplementfasyncmethodinthe file_operationsstruct: int(*fasync)(intfd,structfile *filp,intmode) Whendataarereadythedrivercallkill_fasyncand theapplicationsreceiveSIGIOsignal(default)as notification.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

353

AIO
Driversmustimplementthreefunctionsinthe file_operationsstruct:
ssize_t(*aio_read)(structkiocb*,const structiovec*,unsignedlong,loff_t); ssize_t(*aio_write)(structkiocb*,const structiovec*,unsignedlong,loff_t); int(*aio_fsync)(structkiocb*,int datasync);

NotificationviacustomUnixsignalorwiththecreationofa threadtomanageit.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

354

Asyncfunctioncalls
Codeinmainlinestartingfrom2.6.29,butit'senabledby defaultonlystartingfrom2.6.30. Designedtospeedupthebootprocess. Duringregularkernelboot,theseasynchronouscallsare actuallyexecutedsynchronously(fornow). Exampleuse:asyncprobingofSCSIdevice. Ifyouwantsimplytoschedulesomedelayedwork,use workqueuesinstead.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

355

EmbeddedLinuxdevelopment

Driverdevelopment
NewDeviceModel

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

356

Unifieddevicemodel
The2.6kernelincludedasignificantnewfeature: aunifieddevicemodel. Insteadofhavingdifferentadhocmechanismsinthevarious subsystems,thedevicemodelunifiesthedescriptionofthe devicesandtheirtopology:
Minimizationofcodeduplication Commonfacilities(referencecounting,eventnotification, powermanagement,etc.) Enumeratethedevices,viewtheirinterconnections,linkthe devicestotheirbusesanddrivers,categorizethembyclasses

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

357

Busdrivers
Thefirstcomponentofthedevicemodelisthebusdriver.
Onebusdriverforeachtypeofbus:USB,PCI,SPI,MMC, ISA,etc.

Itisresponsiblefor:
Registeringthebustype. Allowingregistrationofadapterdrivers(USBcontrollers,I2C adapters,etc.),ableofdetectingtheconnecteddevices. Allowingregistrationofdevicedrivers(USBdevices,I2C devices,PCIdevices,etc.),managingthedevices. Matchingthedevicedriversagainstthedevicesdetectedbythe adapterdrivers.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

358

Listofdeviceidentifiers
Dependingonthebustype,themethodforbindingadevice toadriverisdifferent.Formanybuses,itisbasedonunique identifiers. Thedevicedriverdefinesatablewiththelistofdevice identifiersitisabletomanage:
staticconststructpci_device_idrhine_pci_tbl[]={ {0x1106,0x3043,PCI_ANY_ID,PCI_ANY_ID,},/*VT86C100A*/ {0x1106,0x3053,PCI_ANY_ID,PCI_ANY_ID,},/*VT6105M*/ {}/*terminatelist*/ }; MODULE_DEVICE_TABLE(pci,rhine_pci_tbl);

Code on this slide and the next slides are taken from the via-rhine driver in drivers/net/viarhine.c
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

359

Definingthedriver
Thedevicedriverdefinesadriverstructure,usually specializedbythebusinfrastructure(pci_driver, usb_driver,etc.). Thestructurepointsto:thedevicetable,aprobefunction, calledwhenadeviceisdetectedandvariousothercallbacks
staticstructpci_driverrhine_driver={ .name=DRV_NAME, .id_table=rhine_pci_tbl, .probe=rhine_init_one, .remove=__devexit_p(rhine_remove_one), #ifdefCONFIG_PM .suspend=rhine_suspend, .resume=rhine_resume, #endif/*CONFIG_PM*/ .shutdown=rhine_shutdown, };
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

360

Registeringthedriver
Inthemoduleinitializationfunction,thedriverisregistered tothebusinfrastructure,inordertoletthebusknowthatthe driverisavailabletohandledevices.
staticint__initrhine_init(void) staticint__initrhine_init(void) { { [...] [...] returnpci_register_driver(&rhine_driver); returnpci_register_driver(&rhine_driver); } } staticvoid__exitrhine_cleanup(void) staticvoid__exitrhine_cleanup(void) { { pci_unregister_driver(&rhine_driver); pci_unregister_driver(&rhine_driver); } }

IfanewPCIdevicematchesoneoftheidentifiersofthe table,theprobe()methodofthePCIdriverwillgetcalled.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

361

Probemethod
Theprobe()methodreceivesasargumentastructure describingthedevice,usuallyspecializedbythebus infrastructure(pci_dev,usb_device,etc.). Thisfunctionisresponsiblefor
Initializingthedevice,mappingtheI/Omemory,registering theinterrupthandlers.Thebusinfrastructureprovidesmethods togettheaddresses,interruptsnumbersandotherdevice specificinformation. Registeringthedevicetotheproperkernelframework,for examplethenetworkinfrastructure. It'saninitfunction!It'scalledprobeforhistoricalreasons.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

362

Globalarchitecture

PCIbusdriver PCIbusdriver

Thedevice driver registersitself andthedevice identifiersto thebusdriver

Thebusdriver detectsamatching device,andcalls theprobe()method ofthedevice driver.

Network Network device device framework framework 3

Theprobe() methodofthe devicedriver initializesthe deviceandregisters anewnetwork interface

viarhine viarhine devicedriver devicedriver

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

363

sysfs
Thebus,device,drivers,etc.structuresareinternaltothekernel. Thesysfsvirtualfilesystemprovidesamechanismtoexportsuch informationtouserspace. Usedforexamplebyudevtoprovideautomaticmoduleloading, firmwareloading,devicefilecreation,etc.
sysfsisusuallymountedin/sys /sys/bus/containsthelistofbuses. /sys/devices/containsthelistofdevices.

/sys/classenumeratesdevicesbyclass(net,input,block...), whateverthebustheyareconnectedto.Veryuseful! Takeyourtimetoexplore/sysonyourworkstation.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

364

Platformdevices
Onembeddedsystems,devicesareoftennotconnected throughabusallowingenumeration,hotplugging,and providinguniqueidentifiersfordevices. However,westillwantthedevicestobepartofthedevice model. Thesolutiontothisistheplatformdriver/platformdevice infrastructure. Theplatformdevicesarethedevicesthataredirectly connectedtotheCPU,withoutanykindofbus.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

365

Implementationofaplatformdriver
Thedriverimplementsaplatform_driverstructure (exampletakenfromdrivers/serial/imx.c)
staticstructplatform_driverserial_imx_driver={ staticstructplatform_driverserial_imx_driver={ .probe=serial_imx_probe, .probe=serial_imx_probe, .remove=serial_imx_remove, .remove=serial_imx_remove, .driver={ .driver={ .name="imxuart", .name="imxuart", .owner=THIS_MODULE, .owner=THIS_MODULE, }, }, }; };

Andregistersitsdrivertotheplatformdriverinfrastructure
staticint__initimx_serial_init(void) staticint__initimx_serial_init(void) { { [...] [...] ret=platform_driver_register(&serial_imx_driver); ret=platform_driver_register(&serial_imx_driver); [...] [...] } }
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

366

Instantiationoftheplatformdevice(1)
Intheboardspecificcode,theplatformdevicesare instantiated(arch/arm/machimx/mx1ads.c):
staticstructplatform_deviceimx_uart1_device={ staticstructplatform_deviceimx_uart1_device={ .name="imxuart", .name="imxuart", .id=0, .id=0, .num_resources=ARRAY_SIZE(imx_uart1_resources), .num_resources=ARRAY_SIZE(imx_uart1_resources), .resource=imx_uart1_resources, .resource=imx_uart1_resources, .dev={ .dev={ .platform_data=&uart_pdata, .platform_data=&uart_pdata, } } }; };

Thematchbetweenthedeviceandthedriverismadeusing thename.Itmustbeuniqueamongstdrivers!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

367

Instantiationoftheplatformdevice(2)
Thedeviceispartofalist
staticstructplatform_device*devices[]__initdata={ staticstructplatform_device*devices[]__initdata={ &cs89x0_device, &cs89x0_device, &imx_uart1_device, &imx_uart1_device, &imx_uart2_device, &imx_uart2_device, }; };

Andthelistofdevicesisaddedtothesystem duringboardinitialization
staticvoid__initmx1ads_init(void) staticvoid__initmx1ads_init(void) { { [...] [...] platform_add_devices(devices,ARRAY_SIZE(devices)); platform_add_devices(devices,ARRAY_SIZE(devices)); } }

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

368

I/OResources
EachplatformdeviceisassociatedwithasetofI/Oresources, referencedintheplatform_devicestructure
staticstructresourceimx_uart1_resources[]={ staticstructresourceimx_uart1_resources[]={ [0]={ [0]={ .start=0x00206000, .start=0x00206000, .end=0x002060FF, .end=0x002060FF, .flags=IORESOURCE_MEM, .flags=IORESOURCE_MEM, }, }, [1]={ [1]={ .start=(UART1_MINT_RX), .start=(UART1_MINT_RX), .end=(UART1_MINT_RX), .end=(UART1_MINT_RX), .flags=IORESOURCE_IRQ, .flags=IORESOURCE_IRQ, }, }, }; };

ItallowsthedrivertobeindependentfortheI/Oaddresses,IRQ numbers!Seeimx_uart2_deviceforanotherdeviceusingthe sameplatformdriver.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

369

Insidetheplatformdriver
Whenaplatform_deviceisaddedtothesystemusing platform_add_device(),theprobe()methodofthe platformdrivergetscalled. Thismethodisresponsibleforinitializingthehardware, registeringthedevicetotheproperframework(inourcase, theserialdriverframework). TheplatformdriverhasaccesstotheI/Oresources:
res=platform_get_resource(pdev,IORESOURCE_MEM,0); res=platform_get_resource(pdev,IORESOURCE_MEM,0); base=ioremap(res>start,PAGE_SIZE); base=ioremap(res>start,PAGE_SIZE); sport>rxirq=platform_get_irq(pdev,0); sport>rxirq=platform_get_irq(pdev,0);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

370

sysfstools
http://linuxdiag.sourceforge.net/Sysfsutils.html libsysfsThelibrary'spurposeistoprovideaconsistentand stableinterfaceforqueryingsystemdeviceinformationexposed throughsysfs.Usedbyudev(seelater). systoolAutilitybuiltuponlibsysfsthatlistsdevicesby bus,class,andtopology.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

371

DeviceModelreferences
Kerneldocumentation Documentation/drivermodel/ Documentation/filesystems/sysfs.txt Linux2.6DeviceModel http://www.bravegnu.org/devicemodel/devicemodel.html LinuxDeviceDrivers,chapter14TheLinuxDevice Model http://lwn.net/images/pdf/LDD3/ch14.pdf Thekernelsourcecode Fullofexamplesofotherdrivers!
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

372

EmbeddedLinuxdevelopment

Driverdevelopment
udev,mdevandhotplug

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

373

/devissuesandlimitations
OnRedHat9,18000entriesin/dev! Allentriesforallpossibledevices hadtobecreatedatsysteminstallation. Neededanauthoritytoassignmajornumbers http://lanana.org/:LinuxAssignedNamesandNumbersAuthority Notenoughnumbersin2.4,limitsextendedin2.6. Userspaceneitherknewwhatdeviceswerepresentinthesystem, norwhichrealdevicecorrespondedtoeach/deventry.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

374

devfssolutionandlimitations
devfs:afirstsolutionimplementedinLinux2.3. Onlyshowedpresentdevices. Butuseddifferentnamesasin/dev,causingissuesinscripts. Butnoflexibilityindevicenames,unlikewith/dev/,e.g.the1st IDEdiskdevicehadtobecalledeither/dev/hdaor /dev/ide/hd/c0b0t0u0. Butdidn'tallowdynamicmajorandminornumberallocation. Butrequiredtostorethedevicenamingpolicyinkernelmemory. KeptforeverinkernelRAMevenwhennolongerneeded. devfswascompletelyremovedinLinux2.6.18.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

375

devtmpfs
Inmainlinestartingfrom2.6.32. Thebasicideaistocreateatmpfsearlyinthekernel initializationbeforethedrivercorehasinitialized. Aseachdeviceregisterswiththedrivercore,itsmajorand minornumbersanddevicenamecanbeusedtocreatean entryinthatfilesystem. Nouserspacesupport,usefulforembeddeddevice

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

376

Theudevsolution
TakesadvantageofsysfsintroducedbyLinux2.6. CreatedbyGregKroahHartman,ahugecontributor. Otherkeycontributors:KaySievers,DanStekloff. Entirelyinuserspace. Automaticallycreates/removesdeviceentries in/dev/accordingtoinserted/removeddevices. Majorandminordevicetransmittedbythekernel. Requiresnochangetodrivercode. Fast:writteninC Smallsize:udevdversion108:61KBinUbuntu7.04
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

377

Themdevsolution
mdevisanappletofbusybox. Samefeaturesofudev. Verysimpleconfigurationwithmdev.conf. Itcanstart/stopcommandsorscriptswhenadevicefileis created/removed.Forexampleanentryinthemdev.confcouldbe: mmc.*0:0777*(/usr/bin/automount_sdcard.sh) Wecanuse@,$,*beforethepathtostartthecommandwhenthedevice isconnected,whenit'sdisconnectedorboth,respectively.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

378

udevandmdevresources
Homepage http://kernel.org/pub/linux/utils/kernel/hotplug/udev.html Sources http://kernel.org/pub/linux/utils/kernel/hotplug/ Recentstateofudev,byKaySievers(verygoodarticle): http://vrfy.org/log/recentstateofudev.html Theudevmanualpage: manudev

mdevdocumentationinthebusyboxdocfolder

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

379

EmbeddedLinuxdevelopment

Adviceandresources
Gettinghelpandcontributions

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

380

Solvingissues
Ifyoufaceanissue,anditdoesn'tlookspecifictoyourworkbut rathertothetoolsyouareusing,itisverylikelythatsomeoneelse alreadyfacedit. SearchtheInternetforsimilarerrorreports. Youhavegreatchancesoffindingasolutionorworkaround,orat leastanexplanationforyourissue. Otherwise,reportingtheissueisuptoyou!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

381

Gettinghelp
Ifyouhaveasupportcontract,askyourvendor. Otherwise,don'thesitatetoshareyourquestionsandissues EithercontacttheLinuxmailinglistforyourarchitecture(likelinuxarm kernelorlinuxshdev...). Orcontactthemailinglistforthesubsystemyou'redealingwith(linuxusb devel,linuxmtd...).Don'taskthemaintainerdirectly! MostmailinglistscomewithaFAQpage.Makesureyoureaditbefore contactingthemailinglist. UsefulIRCresourcesareavailabletoo (forexampleonhttp://kernelnewbies.org). RefrainfromcontactingtheLinuxKernelmailinglist,unlessyou'rean experienceddeveloperandneedadvice.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

382

Gettingcontributions
Appliesifyourprojectcaninterestotherpeople: developingadriverorfilesystem,portingLinuxonanew processor,boardordeviceavailableonthemarket... Externalcontributorscanhelpyoualotby Testing. Writingdocumentation. Makingsuggestions. Evenwritingcode.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

383

Encouragingcontributions
Openyourdevelopmentprocess:mailinglist,Wiki,publicCVS readaccess. Leteveryonecontributeaccordingtotheirskillsandinterests. Releaseearly,releaseoften. Takefeedbackandsuggestionsintoaccount. Recognizecontributions. Makesurestatusanddocumentationareuptodate. Publicizeyourworkandprogresstobroaderaudiences.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

384

EmbeddedLinuxdevelopment

Adviceandresources
Bugreportandpatchsubmission

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

385

ReportingLinuxbugs
Firstmakesureyou'reusingthelatestversion. Makesureyouinvestigatetheissueasmuchasyoucan: seeDocumentation/BUG_HUNTING. Makesurethebughasnotbeenreportedyet.Abugtrackingsystem (http://bugzilla.kernel.org/)existsbutveryfewkerneldevelopersuseit. Besttousewebsearchengines(accessingpublicmailinglistarchives). Ifthesubsystemyoureportabugonhasamailinglist,useit. Otherwise,contacttheofficialmaintainer(seetheMANTAINERSfile). Alwaysgiveasmanyusefuldetailsaspossible.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

386

Howtosubmitpatchesordrivers
Don'tmergepatchesaddressingdifferentissues. Youshouldidentifyandcontacttheofficialmaintainer forthefilestopatch. SeeDocumentation/SubmittingPatchesfordetails.Fortrivial patches,youcancopytheTrivialPatchMonkey. Seealsohttp://kernelnewbies.org/UpstreamMergeforveryhelpfuladviceto haveyourcodemergedupstream(byRikvanRiel). Specialsubsystems: ARMplatform:it'sbesttosubmityourARMpatchestoRussellKing's patchsystem:http://www.arm.linux.org.uk/developer/patches/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

387

Howtobecomeakerneldeveloper?
GregKroahHartmanwroteusefulguidelinesinthekerneldocumentation:

Documentation/HOWTO.
HowtoParticipateintheLinuxCommunity(byJonathanCorbet) AGuideToTheKernelDevelopmentProcess
http://ldn.linuxfoundation.org/documentation/howparticipatelinuxcommunity

Readthislastdocumentfirst. Itisprobablysufficient!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

388

EmbeddedLinuxdevelopment

Adviceandresources
References

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

389

Informationsites(1)
LinuxWeeklyNews http://lwn.net/ TheweeklydigestoffallLinuxandfreesoftware informationsources. Indepthtechnicaldiscussionsaboutthekernel. Subscribetofinancetheeditors($5/month). Articlesavailablefornonsubscribers after1week.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

390

Informationsites(2)
KernelTrap http://kerneltrap.org/ Forumwebsiteforkerneldevelopers. News,articles,whitepapers,discussions,polls,interviews. Perfectifadigestisnotenough!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

391

Usefulreading(1)
LinuxDeviceDrivers,3rd edition,Feb2005 ByJonathanCorbet,AlessandroRubini, GregKroahHartman,O'Reilly http://www.oreilly.com/catalog/linuxdrive3/ Freelyavailableonline! Greatcompaniontotheprintedbook foreasyelectronicsearches! http://lwn.net/Kernel/LDD3/(1PDFfileperchapter) http://freeelectrons.com/community/kernel/ldd3/(singlePDFfile) AmusthavebookforLinuxdevicedriverwriters!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

392

Usefulreading(2)
LinuxKernelinaNutshell,Dec2006 ByGregKroahHartman,O'Reilly http://www.kroah.com/lkn/ Agoodreferencebookandguideonconfiguring, compilingandmanagingtheLinuxkernelsources. Freelyavailableonline! Greatcompaniontotheprintedbook foreasyelectronicsearches! AvailableassinglePDFfileon http://freeelectrons.com/community/kernel/lkn/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

393

Usefulreading(3)
LinuxKernelDevelopment,2nd Edition,Jan2005 RobertLove,NovellPress http://freeelectrons.com/redirect/lkd2book.html Averysyntheticandpleasantwaytolearnaboutkernel subsystems(beyondtheneedsofdevicedriverwriters) UnderstandingtheLinuxKernel,3rd edition,Nov2005 DanielP.Bovet,MarcoCesati,O'Reilly http://oreilly.com/catalog/understandlk/ AnextensivereviewofLinuxkernelinternals, coveringLinux2.6atlast. Unfortunately,onlycoversthePCarchitecture.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

394

Usefulonlineresources
LinuxkernelmailinglistFAQ http://www.tux.org/lkml/ CompleteLinuxkernelFAQ Readthisbeforeaskingaquestiontothemailinglist KernelNewbies http://kernelnewbies.org/ Glossary,articles,presentations,HOWTOs, recommendedreading,usefultoolsforpeople gettingfamiliarwithLinuxkernelordriverdevelopment. Kernelglossary: http://kernelnewbies.org/KernelGlossary

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

395

EmbeddedLinuxWiki
TheembeddedLinuxWikicontainsloadsofusefulresources forembeddedsystemsdevelopers: ManyHOWTOdocumentsofallkinds,coveringtopicslikesystem size,boottime,multimedia,powermanagement,toolchains... Kernelpatchesnotavailableinmainstreamyet(e.g.LinuxTiny). Communityresource:hackerinterviews,bookreviews, eventcoverage... Isopentoeveryone.Contributionsarewelcome! http://elinux.org

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

396

ARMresources
ARMLinuxproject:http://www.arm.linux.org.uk/ Developerdocumentation:http://www.arm.linux.org.uk/developer/ armlinuxkernelmailinglist: http://lists.arm.linux.org.uk/mailman/listinfo/linuxarmkernel FAQ:http://www.arm.linux.org.uk/armlinux/mlfaq.php Howtopostkernelfixes: http://www.arm.uk.linux.org/developer/patches/ ARMLinux@Simtec:http://armlinux.simtec.co.uk/ Afewusefulresources:FAQ,documentationandWho'swho! ARMLimited:http://www.linuxarm.com/ Wikiwithlinkstousefuldeveloperresources

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

397

Internationalconferences(1)
UsefulconferencesfeaturingLinuxkernelpresentations OttawaLinuxSymposium(July):http://linuxsymposium.org/ Lotsofkerneltopicsbymajorkernelhackers. Freelyavailableproceedings. Fosdem:http://fosdem.org(Brussels,February) Fordevelopers.Kernelpresentationsfromwellknownkernelhackers. EmbeddedLinuxConference:http://embeddedlinuxconference.com/ OrganizedbytheCELinuxForum:California (SanJose,April),inEurope(November).Frequenttechnicalsessions inJapan.Veryinterestingkerneltopicsforembeddedsystems developers.Presentationslidesfreelyavailable.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

398

Internationalconferences(2)
linux.conf.au:http://conf.linux.org.au/(Australia/NewZealand) Featuresafewpresentationsbykeykernelhackers.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

399

EmbeddedLinuxdevelopment

Annexes
Kernelsources

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

400

Checkingtheintegrityofsources
KernelsourceintegritycanbecheckedthroughOpenPGPdigitalsignatures. Fulldetailsonhttp://www.kernel.org/signature.html DetailsaboutGnuPG:http://www.gnupg.org/gph/en/manual.html ImportthepublicGnuPGkeyofkerneldevelopers:
gpgkeyserverpgp.mit.edurecvkeys0x517D0F0E

Ifblockedbyyourfirewall,lookfor0x517D0F0Eonhttp://pgp.mit.edu/, copyandpastethekeytoalinuxkey.txtfile: gpgimportlinuxkey.txt Downloadthesignaturefilecorrespondingtoyourkernelversion (atthesamelocation),andrunthesignaturecheck: gpgverifylinux2.6.11.12.tar.bz2.sign

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

401

EmbeddedLinuxdevelopment

Annexes
Slabcachesandmemorypools

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

402

Slabcaches
Alsocalledlookasidecaches Slabcaches:Objectsthatcanholdanynumber ofmemoryareasofthesamesize. OptimumuseofavailableRAMandreducedfragmentation. MainlyusedinLinuxcoresubsystems:filesystems(openfiles,inode andfilecaches...),networking...Livestatson/proc/slabinfo. Maybeusefulindevicedriverstoo,thoughnotusedsooften. Linux2.6:usedbyUSBandSCSIdrivers.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

403

SlabcacheAPI(1)
#include<linux/slab.h> Creatingacache: cache=kmem_cache_create( name, /*Namefor/proc/slabinfo*/ size, /*Cacheobjectsize*/ align, /*Cachealignment*/ flags, /*Options:DMA,debugging,tracing...*/ constructor, /*Optional,calledaftereachallocation*/ destructor); /*Optional,calledbeforeeachrelease*/ Example:drivers/usb/host/uhcihcd.c uhci_up_cachep=kmem_cache_create( "uhci_urb_priv",sizeof(structurb_priv), 0,0,NULL,NULL);
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

404

SlabcacheAPI(2)
SinceLinux2.6.22,amacrocansimplifycachecreationinmostcases: #defineKMEM_CACHE(__struct,__flags)\ kmem_cache_create(#__struct,\ sizeof(struct__struct),\ __alignof__(struct__struct),\ (__flags),NULL,NULL) Example:kernel/pid.c pid_cachep=KMEM_CACHE(pid,SLAB_PANIC);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

405

SlabcacheAPI(3)
Allocatingfromthecache: object=kmem_cache_alloc(cache,flags); orobject=kmem_cache_zalloc(cache,flags); Freeinganobject: kmem_cache_free(cache,object); Destroyingthewholecache: kmem_cache_destroy(cache); MoredetailsandanexampleintheLinuxDeviceDriversbook: http://lwn.net/images/pdf/LDD3/ch08.pdf

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

406

Memorypools
Usefulformemoryallocationsthatcannotfail Kindoflookasidecachetryingtokeepaminimumnumber ofpreallocatedobjectsaheadoftime. Usewithcare:otherwisecanresultinalotofunused memorythatcannotbereclaimed!Useothersolutions wheneverpossible.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

407

MemorypoolAPI(1)
#include<linux/mempool.h> Mempoolcreation: mempool=mempool_create( min_nr, alloc_function, free_function, pool_data);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

408

MemorypoolAPI(2)
Allocatingobjects: object=mempool_alloc(pool,flags); Freeingobjects: mempool_free(object,pool); Resizingthepool: status=mempool_resize( pool,new_min_nr,flags); Destroyingthepool(caution:freeallobjectsfirst!): mempool_destroy(pool);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

409

Memorypoolimplementation
mempool_create mempool_create Callalloc functionmin_nr times

mempool_alloc mempool_alloc

Callalloc Callalloc function function

No Success? Success? Yes

Takean Takean objectfrom objectfrom thepool thepool

mempool_free mempool_free

poolcount poolcount <min_nr? <min_nr? No Callfree Callfree function function onobject onobject

Yes

Addfreed Addfreed objecttopool objecttopool

Newobject Newobject

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

410

Memorypoolsusingslabcaches
Idea:useslabcachefunctionstoallocateandfreeobjects. Themempool_alloc_slabandmempool_free_slabfunctions supplyalinkwithslabcacheroutines. So,youwillfindmanycodeexampleslookinglike: cache=kmem_cache_create(...); pool=mempool_create( min_nr, mempool_alloc_slab, mempool_free_slab, cache); There'sashorthandpoolcreationfunctionforthiscase: pool=mempool_create_slab_pool(min_nr,cache);

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

411

EmbeddedLinuxdevelopment

Annexes
Ubootdetails

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

412

PostprocessingkernelimageforUboot
TheUbootbootloaderneedsextrainformationtobeaddedto thekernelandinitrdimagefiles. mkimagepostprocessingutilityprovidedinUbootsources Kernelimagepostprocessing: makeuImage

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

413

PostprocessinginitrdimageforUboot
mkimage ninitrd\ Aarm\ Olinux\ Tramdisk\ Cgzip\ drdext2.gz\ uInitrd Name Architecture OperatingSystem Type Compression Inputfile Outputfile

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

414

CompilingDasUboot
GettheUbootsourcesfromhttp://www.denx.de/wiki/UBoot IntheUbootsourcedirectory: Findthenameoftheconfigfileforyourboardininclude/configs (forexample:omap1710h3.h) ConfigureUboot: makeomap1710h3_config(.hreplacedby_config) Ifneeded,changethecrosscompilerprefixinMakefile: ifeq($(ARCH),arm) CROSS_COMPILE=armlinux endif Compile: make
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

415

CompilingUbootmkimage
IfyoujustneedmkimageandUbootisalreadyinstalledonyourboard: mkimageiscompletelyarchitectureandboardindependent. ConfigureUbootsourcesforanyboardonanyarchitecture (seepreviousslide). Compile: make(ormakekifyouhaveminorfailures) Installmkimage: cptools/mkimage/usr/local/bin/

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

416

Ubootprompt
Connectthetargettothehostthroughaserialconsole. Poweruptheboard. Ontheserialconsole,youwillseesomethinglike: UBoot1.3.4(Dec4200817:31:20) RAMConfiguration: Bank#0:000000008MB Flash:2MB In:serial Out:serial Err:serial uboot#
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

417

Boardinformation
uboot#bdinfo DRAMbank=0x00000000 >start=0x00000000 >size=0x00800000 ethaddr=00:40:95:36:35:33 ip_addr=10.0.0.11 baudrate=19200bps

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

418

Environmentvariables(1)
uboot#printenv baudrate=19200 ethaddr=00:40:95:36:35:33 netmask=255.255.255.0 ipaddr=10.0.0.11 serverip=10.0.0.1 stdin=serial stdout=serial stderr=serial uboot#printenvserverip serverip=10.0.0.2
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

Networksettings ForTFTP andNFS

uboot#setenvserverip10.0.0.2

419

Environmentvariables(2)
Environmentvariablechangescanbestored toflashusingthesaveenvcommand. Youcanevencreatesmallshellscripts storedinenvironmentvariables: setenvmyscript'tftp0x21400000uImage; bootm0x21400000' Youcanthenexecutethescript: runmyscript

Moreelaboratescriptingisavailablewithscriptfiles, tobeprocessedwithmkimage.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

420

Networkcommands
uboot#tftp8000uboot.bin Fromserver10.0.0.1;ourIPaddressis 10.0.0.11 Filename'uboot.bin'. Loadaddress:0x8000 Loading:################### done Bytestransferred=95032(17338hex) Theaddressandsizeofthedownloadedfilearestoredinthe fileaddrandfilesizeenvironmentvariables.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

421

Flashcommands(1)
uboot#flinfo Bank#1:AMDAm29LV160DB16KB,2x8KB,32KB,31x64KB Size:2048KBin35Sectors SectorStartAddresses: S00@0x01000000!S01@0x01004000! S02@0x01006000!S03@0x01008000! S04@0x01010000!S05@0x01020000! S06@0x01030000S07@0x01040000 Protectedsectors ... S32@0x011D0000S33@0x011E0000 S34@0x011F0000

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

422

Flashcommands(2)
uboot#protectoff1:04 UnProtectFlashSectors04inBank#1 uboot#erase1:04 EraseFlashSectors04inBank#1 [email protected] [email protected] [email protected] [email protected] [email protected]

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

423

Flashcommands(3)
Storingafileinflash Downloadingfromthenetwork: uboot#tftp8000uboot.bin Copytoflash(0x01000000:firstsector) uboot#cp.b${fileaddr}1000000${filesize} CopytoFlash...................done Restoreflashsectorprotection: uboot#protecton1:04 ProtectFlashSectors05inBank#1

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

424

bootcommands
Specifykernelbootparameters: Continueson uboot#setenvbootargsmem=64M\ thesameline console=ttyS0,115200init=/sbin/init\ root=/dev/mtdblock0 Executethekernelfromagivenphysicaladdress(RAMorflash): bootm0x01030000

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

425

Usefullinks
Uboothomepage: http://www.denx.de/wiki/UBoot/WebHome VeryniceoverviewaboutUboot (whichhelpedtocreatethissection): http://linuxdevices.com/articles/AT5085702347.html TheUbootmanual: http://www.denx.de/wiki/view/DULG/UBoot

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

426

EmbeddedLinuxdevelopment

Annexes
Initrunlevels

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

427

SystemVinitrunlevels(1)
IntroducedbySystemVUnix MuchmoreflexiblethaninBSD Makeitpossibletostartorstop differentservicesforeach runlevel Correspondtotheargumentgiven to/sbin/init. Runlevelsdefinedin /etc/inittab.
/etc/initabexcerpt: /etc/initabexcerpt:
id:5:initdefault: id:5:initdefault: #Systeminitialization. #Systeminitialization. si::sysinit:/etc/rc.d/rc.sysinit si::sysinit:/etc/rc.d/rc.sysinit l0:0:wait:/etc/rc.d/rc0 l0:0:wait:/etc/rc.d/rc0 l1:1:wait:/etc/rc.d/rc1 l1:1:wait:/etc/rc.d/rc1 l2:2:wait:/etc/rc.d/rc2 l2:2:wait:/etc/rc.d/rc2 l3:3:wait:/etc/rc.d/rc3 l3:3:wait:/etc/rc.d/rc3 l4:4:wait:/etc/rc.d/rc4 l4:4:wait:/etc/rc.d/rc4 l5:5:wait:/etc/rc.d/rc5 l5:5:wait:/etc/rc.d/rc5 l6:6:wait:/etc/rc.d/rc6 l6:6:wait:/etc/rc.d/rc6

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

428

SystemVinitrunlevels(2)
Standardlevels init0 Haltthesystem init1 Singleusermodeformaintenance init6 Rebootthesystem initS Singleusermodeformaintenance. Mountingonly/.Oftenidenticalto1 Customizablelevels:2,3,4,5 init3 Oftenmultiusermode,withonly commandlinelogin init5 Oftenmultiusermode,with graphicallogin

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

429

initscripts
Accordingto/etc/inittabsettings,init<n>runs: First/etc/rc.d/rc.sysinitforallrunlevels Thenscriptsin/etc/rc<n>.d/ Startingservices(1,3,5,S): runsS*scriptswiththestartoption Killingservices(0,6): runsK*scriptswiththestopoption Scriptsareruninfilenamelexicalorder Justuselsltofindouttheorder!

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

430

/etc/init.d
Repositoryforallavailableinitscripts /etc/rc<n>.d/onlycontainslinkstothe/etc/init.d/ scriptsneededforrunleveln /etc/rc1.d/example(fromFedoraCore3) K01yum>../init.d/yum K02atd>../init/atd K05anacron>../init.d/anacron S00single>../init.d/single S01sysstat>../init.d/sysstat S06cpuspeed>../init.d/cpuspeed

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

431

Handlinginitscriptsbyhand
Simplycallthe/etc/init.dscripts! sudo/etc/init.d/sshdstart Startingsshd:[OK] sudo/etc/init.d/nfsstop ShuttingdownNFSmountd:[FAILED] ShuttingdownNFSdaemon:[FAILED] ShuttingdownNFSquotas:[FAILED] ShuttingdownNFSservices:[OK] sudo/etc/init.d/pcmciastatus cardmgr(pid3721)isrunning... sudo/etc/init.d/httpdrestart Stoppinghttpd:[OK] Startinghttpd:[OK]

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

432

EmbeddedLinuxdevelopment

Annexes
EmbeddedDistributionsandTools

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

433

ELDK
EmbeddedLinuxDevelopmentkitprovidedbyDENX. providesacompleteandpowerfulsoftwaredevelopment environmentforembeddedandrealtimesystems. ItisavailableforARM,PowerPCandMIPSprocessorsand consistsof:
CrossDevelopmentTools(Compiler,Assembler,Linkeretc.). NativeTools(Shell,commandsandlibraries). Firmware(UBoot). Linuxkernel. XenomaiRTOSEmulationframeworkforsystemsrequiringhardrealtime responses. SELF(SimpleEmbeddedLinuxFramework)tobuildyourembeddedsystemson.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

434

Buildroot
Buildrootisacompletebuildingsystem:toolchain,firmware, kernelandrootfilesystem. Hugenumberofpackageswhichcanbecompiledintothe image. Itisausefultoolevenonlytogenerateanewtoolchain. ItshouldbenotedhoweverthatitonlysupportsuClibc! Graphicmenulikemenuconfigofthekernel.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

435

Crosstool
Scriptwhichdownloadssourcetarballsandbuildssimple gcc/glibccrosstoolchains. Buildmatrixwhichshowswhichversionsofgcc/glibcwork togetherwithvariousarchitectures. It'seasytoselectwhichversionsofgcc/glibcshouldbeused togenerateatoolchainforaparticulararchitecture.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

436

CrosstoolNG
Forkofcrosstool. Easierconfiguration,refactoredcode,andalearningbaseon howtoolchainsarebuilt. SupportbothuClibcandglibc,debugtools(gdb,strace, dmalloc...),andawiderangeofversionsforeachtools.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

437

Busybox
Implementationofhundredsofunix/linuxcommandsina singleexecutable. AmustintheLinuxembeddedworld. Designedtobesizecritical. Wecouldbuildacompleterootfilesystemonlywithit! Oneexecutableandseverallinks. Securitysettingsforeachappletcanbeconfiguredatrun timeinthebusybox.conffile. SeeBusyboxHomepage
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

438

EmbeddedLinuxdevelopment

Annexes
EmbeddedFilesystems

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

439

Mostimportantembeddedfilesystems
CRAMFSCompressedROMfilesystem SQUASHFS AXFSAdvancedXIPFileSystem JFFS2JournallingFlashFileSystem YAFFS2YetAnotherFlashFileSystem UBIFSUnsortedBlockImagesFileSystem LogFSLogstructuredFilesystem PRAMFSPersistentandProtectedRAMFilesystem
ReadOnly

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

440

CRAMFS
Filesysteminmainline. Metadatanotcompressed. Filesizesarelimitedtolessthan16MBandmaxfilesystem sizeisalittleover256MB. Notimestampsarestoredinacramfs. Doesn'tsupport16/32bitsuid/gid.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

441

SquashFS
Itisinmainlinestartingfrom2.6.29. Data,inodesanddirectoriesarecompressed. Itstoresfulluid/gids(32bits),andfilecreationtime. Filesupto2^64bytesandfilesystemscanbeupto2^64 bytes. Inodeanddirectorydataarehighlycompacted. Fileduplicatesaredetectedandremoved. Bothbigandlittleendianarchitecturesaresupported. Onlyforblockdevicebutreadonlysoit'sfinetouse mtdblockdriver
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

442

AXFS
DesignedspecificallytosupportExecuteinplaceoperations. Biphasedapproach:
Firstphase:filesysteminflashandrunittocollectprofiledata, statingwhatpagesareused. Secondphase:buildafilesystemusingtheseprofiledata.All pagesmentionedintheprofilefileasXIPdata.

CompressedregionsplusXIPregionsapproach. It'spossibleasplitacrossXIPNORandnonXIPNAND.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

443

JFFS2/YAFFS2
JFFS2 YAFFS2

Wearleveling,ECC Powerdownresistant Ontheflycompression Hugemounttimes MetadatainRAM Mainline

SpecificforNANDdevice (butitworksevenwithNOR) Wearleveling,ECC Powerdownresistant Nocompression Veryquickmounttime AvailableasaLinuxpatch

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

444

UBIFS
UBIFSinmainlinestartingfrom2.6.27. VolumemanagementsystemontopofMTDdevices. Allowstocreatemultiplelogicalvolumesandspreadwritesacross allphysicalblocks. Takescareofmanagingtheeraseblocksandwearleveling. Separationbetweenlogicaleraseblock(UBI),groupedbyvolumes andphysicaleraseblock(MTD). BestperformancecomparedtoJFFS2andYAFFS2... Seehttp://freeelectrons.com/pub/conferences/2008/elce/flashfilesystems.pdf ....butithasgotsomeperformanceissueduetobadblocktable builtoneachmount.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

445

LogFS
Inmainlinestartingfrom2.6.34. UnlikeJFFS2,LogFSstorestreeinformationonflashinstead ofreadingthemediumatmounttime,readingitinO(1). Thedeviceistreatedlikeasortofcircularbuffer,withnew dataalwaysbeingwrittentotheendlikealog. Itimplementsthe"wanderingtree"algorithm:changesatthe bottomofthefstreewillpropagateupwardallthewaytothe root.Oneoftheadvantagesisthattheoldfilesystem structureremainsvaliduntiltherootisrewritten. Itusesafixedlocationdatainthemediumwhereversioned pointerstotherootinodecanbefoundcalledanchorarea.
LogFSinfo:http://lazybastard.org/~joern/logfs1.pdf
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

446

PRAMFS
FSlivesonlyinRAM. Itisdesignedtoberesistanttosystemshutdown(forexample withanSRAMbatterybacked)orsystemreboot. UseDirectIOtoavoidpagecaching. Filedimensionupto2^30. Itusehardwareprotection(MMU)toavoidcorruptionofthefs againstkernelbugs. ItsupportsXIPfeature. Itstoresfulluid/gids(32bits),andfilecreationtime. Betterperformancescomparedwithdiskfs(ext2/3/4)overram.
EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

447

Legalstatements
LinuxisaregisteredtrademarkofLinusTorvalds. LinuxlogobasedonoriginallogobyLarryEwing. Othercompany,product,andservicenamesmaybe trademarksorservicemarksofothers.

EmbeddedLinuxkernelanddriverdevelopment Copyright20092010,MarcoStornelli CreativeCommonsAttributionShareAlike3.0license Mar20,2010

448