E Linu Slides

FreeElectrons
EmbeddedLinux training
GregoryClement ThomasPetazzoni MichaelOpdenacker
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Rightstocopy
Copyright20042010,FreeElectrons feedback@freeelectrons.com Electronicversionofthisdocumentavailableon http://freeelectrons.com/doc/training/beagle Updateswillbeavailableon http://freeelectrons.com/doc/training/beagle Corrections,suggestions, contributionsandtranslationsarewelcome! Latestupdate:Dec7,2010
AttributionShareAlike3.0 Youarefree tocopy,distribute,display,andperformthework tomakederivativeworks tomakecommercialuseofthework Underthefollowingconditions Attribution.Youmustgivetheoriginalauthorcredit.
ShareAlike.Ifyoualter,transform,orbuilduponthiswork,you maydistributetheresultingworkonlyunderalicenseidenticalto thisone. Foranyreuseordistribution,youmustmakecleartoothersthelicense termsofthiswork. Anyoftheseconditionscanbewaivedifyougetpermissionfromthe copyrightholder. Yourfairuseandotherrightsareinnowayaffectedbytheabove. Licensetext:http://creativecommons.org/licenses/bysa/3.0/legalcode
Linuxkernel Linuxdevicedrivers Boardsupportcode Mainstreamingkernelcode Kerneldebugging EmbeddedLinuxTraining Allmaterialsreleasedwithafreelicense! UnixandGNU/Linuxbasics Linuxkernelanddriversdevelopment RealtimeLinux,uClinux Developmentandprofilingtools Lightweighttoolsforembeddedsystems Rootfilesystemcreation Audioandmultimedia Systemoptimization
FreeElectrons
Ourservices
CustomDevelopment Systemintegration EmbeddedLinuxdemosandprototypes Systemoptimization Applicationandinterfacedevelopment Consultingandtechnicalsupport Helpindecisionmaking Systemarchitecture Systemdesignandperformancereview Developmenttoolandapplicationsupport Investigatingissuesandfixingtoolbugs
Hardwareusedinthistrainingsession
TIBeagleBoard TIOMAP3530ARMCPU 256MBRAM,256MBflash RS232serial USBHost USBOTG JTAG DVID,SVideo AudioInandOut MMC/SD Only150USD
1
Participate!
Duringthelectures... Don'thesitatetoaskquestions.Otherpeopleintheaudience mayhavesimilarquestionstoo. Thishelpsthetrainertodetectanyexplanationthatwasn'tclear ordetailedenough. Don'thesitatetoshareyourexperience,forexampletocompare Linuxwithotheroperatingsystemsusedinyourcompany. Yourpointofviewismostvaluable,becauseitcanbesimilarto yourcolleagues'anddifferentfromthetrainer's. Yourparticipationcanmakeoursessionmoreinteractiveand makethetopicseasiertolearn.
Practicallabguidelines
Duringpracticallabs... Wecannotsupportmorethan8workstationsatonce(eachwith itsboardandequipment).Havingmorewouldmakethewhole classprogressslower,compromisingthecoverageofthewhole trainingagenda(exceptionforpublicsessions:upto10people). So,ifyouaremorethan8participants,pleaseformupto8 workinggroups. Opentheelectroniccopyofyourlecturematerials,anduseit throughoutthepracticallabstofindtheslidesyouneedagain. Don'tcopyandpastefromthePDFslides. TheslidescontainUTF8charactersthatlookthesameasASCII ones,butwon'tbeunderstoodbyshellsorcompilers.
Commandmementosheet
Thismementosheetgives commandexamplesforthemost typicalneeds(lookingforfiles, extractingatararchive...) Itsavesus1dayofUNIX/Linux commandlinetraining. Ourbesttip:inthecommandline shell,alwayshitthe[Tab]keyto completecommandnamesandfile paths.Thisavoids95%oftyping mistakes. Getanelectroniccopyon http://freeelectrons.com/docs/commandline
vibasiccommands
Thevieditorisveryuseful tomakequickchangesto filesinaembeddedtarget. Thoughnotveryuser friendlyatfirst,viisvery powerfulanditsmain15 commandsareeasyto learnandaresufficientfor 99%ofeveryone'sneeds! Youcanalsotakethequicktutorialbyrunningvimtutor. Thisisaworthyinvestment! Getanelectroniccopyon http://freeelectrons.com/docs/commandline
Cooperate!
AsintheFreeSoftwareandOpenSourcecommunity, cooperationduringpracticallabsisvaluableinthistrainingsession: Ifyoucompleteyourlabsbeforeotherpeople,don'thesitateto helpotherpeopleandinvestigatetheissuestheyface.Thefaster weprogressasagroup,themoretimewehavetoexploreextra topics. Explainwhatyouunderstoodtootherparticipantswhenneeded. Italsohelpstoconsolidateyourknowledge. Don'thesitatetoreportpotentialbugstoyourinstructor. Don'thesitatetolookforsolutionsontheInternetaswell.
PracticallabInstallingLinux
InstallUbuntuDesktop10.04onyourPC MakespaceforLinux Seehoweasyitistoinstall Ubuntu Applythelatestupdates
FreeElectrons
Introduction toembedded Linux

MichaelOpdenacker ThomasPetazzoni FreeElectrons
Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!
Embeddedsystem?
An embedded system is a specialpurpose computer system designed to perform one or a few dedicated functions, often with realtime computing constraints. It is usually embedded as part of a complete device including hardware and mechanical parts. In contrast, a general purpose computer, such as a personal computer, can do many different tasks depending on programming. Embeddedsystemscontrolmanyofthecommondevices inusetoday.
Wikipedia,http://en.wikipedia.org/wiki/Embedded_system
2
Manydifferentsystems
Averygenericdefinition Coversverydifferenttypesofsystems Fuzzyborderwithstandardsystems. Consumerelectronics(CE)products Homerouters,DVDplayers,TVsets,digitalcameras,GPS, camcorders,mobilephones,microwaveovens... Industrialproducts Machinecontrol,alarms,surveillancesystems,automotive, rail,aircraft,satellite...
3
Manydifferentproducts
4
EmbeddedLinux
TheFreeSoftwareandOpenSourceworldoffersabroad rangeoftoolstodevelopembeddedsystems. Advantages
Reuseofexistingcomponentsforthebasesystem. Allowstofocusontheaddedvalueoftheproduct. Highquality,provencomponents(Linuxkernel,Clibraries...) Completecontrolonthechoiceofcomponents. Modificationspossiblewithoutexternalconstraints. Communitysupport:tutorials,mailinglists... Lowcost,inparticularnoperunitroyalties. Potentiallylesslegalissues. Easieraccesstosoftwareandtools.
5
Deviceexamples
GPS:TomTomandGarmin Homenetworkrouters:Linksys,Netgear PDA:Zaurus,NokiaN8x0 TVs,camcorders,DVDplayers:Sony,Philips Mobilephones:Motorola,Android,OpenMoko Industrialmachinery Andmanyotherproductsyoudon'tevenimagine...
6
Quiz
ItworkswithLinux,butwhatisitfor?
7
Answer
Tomilkcows!
8
Globalarchitecture
Application Application
Tools
Library
Library
Library
StandardClibrary Linuxkernel Bootloader Hardware

9
Embeddedhardware
Hardwareforembeddedsystemsisoftendifferentfrom hardwareforclassicalsystems.
OftenadifferentCPUarchitecture: oftenARM,MIPSorPowerPC.x86isalsoused. Storageonflashstorage,NORorNANDtype,oftenwith limitedcapacity(fromafewMBtohundredsofMB) LimitedRAMcapacity(fromafewMBtoseveraltensofMB) Manyinterconnectbusnotoftenfoundonthedesktop:I2C, SPI,SSP,CAN,etc.
Developmentboardsstarting fromafewhundredsofEUR/USD
Oftenusedasabasisforthefinalboarddesign.
10
Examples
Picotux100 ARM755MHz,Netsilicon NS7520 2MBofflash 8MBofRAM Ethernet 5GPIOs Serial OpenMoko ARM920T400MHz, Samsung2442B 2MBofNORflash 128MBofRAM 256MBofNANDflash 640x480touchscreen, Bluetooth,GSM,serial, GPS,sound,2buttons, Wifi,USB,etc.
11
Minimumrequirements
ACPUsupportedbygccandtheLinuxkernel
32bitCPU MMUlessCPUsarealsosupported,throughtheuClinuxproject.
AfewMBofRAM,from4MB. 8MBareneededtodoreallydosomething. AfewMBofstorage,from2MB. 4MBtoreallydosomething. Linuxisn'tdesignedforsmallmicrocontrollersthatjusthavea fewtensorhundredsofKBofflashandRAM.

Basemetal,noOS Reducedsystems,suchasFreeRTOS
12
Softwarecomponents
Crosscompilationtoolchain
Compilerthatrunsonthedevelopmentmachine,butgenerates codeforthetarget
Bootloader
Startedbythehardware,responsibleforbasicinitialization,loading andexecutingthekernel
LinuxKernel
Containstheprocessandmemorymanagement,networkstack, devicedriversandprovidesservicestouserspaceapplications
Clibrary
Theinterfacebetweenthekernelandtheuserspaceapplications
Librariesandapplications
Thirdpartyorinhouse
13
EmbeddedLinuxwork
Severaldistincttasksareneededwhendeploying embeddedLinuxinaproduct BoardSupportPackagedevelopment
ABSPcontainsabootloaderandkernelwiththesuitabledevice driversforthetargetedhardware PurposeofourKernelDevelopmenttraining
Systemintegration
Integrateallthecomponents,bootloader,kernel,thirdpartylibraries andapplicationsandinhouseapplicationsintoaworkingsystem Purposeofthistraining
Developmentofapplications
NormalLinuxapplications,butusingspecificallychosenlibraries
14
Rootfilesystem
InaLinuxsystem,several filesystemsaremountedandcreatea globalhierarchyoffilesand directories Aparticularfilesystem,theroot filesystem,ismountedas/ Onembeddedsystems,thisroot filesystemcontainsallthelibraries, applicationsanddataofthesystem Therefore,buildingtheroot filesystemisoneofthemaintasksof integratingembeddedLinux componentsintoadevice Thekernelisusuallykeptseparate
15
Flashcontents
Bootloader Kernel
Root filesystem
Developmentenvironment
TwowaystoswitchtoembeddedLinux
UsesolutionsprovidedandsupportedbyvendorslikeMontaVista, WindRiverorTimeSys.Thesesolutionscomewiththeirown developmenttoolsandenvironment Usecommunitysolutions
InFreeElectronstrainings,wedonotpromoteaparticular vendor,andthereforeusecommunitysolutions
However,knowingtheconcepts,switchingtovendorsolutionswill beeasy
DoingembeddedLinuxdevelopmentrequiresLinuxonthe desktop
ThecommunitysolutionsusuallyonlyexistonLinux UnderstandingLinuxonthedesktopallowsyoutobetter understandLinuxonthedevice
16
EmbeddedLinuxTraining
System administration basics

Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/commandline Corrections,suggestions,contributionsandtranslationsarewelcome!
1
Systemadministrationbasics
Networking
2
Networksetup(1)
ifconfiga Printsdetailsaboutallthenetworkinterfaces availableonyoursystem. ifconfigeth0 Listsdetailsabouttheeth0interface ifconfigeth0192.168.0.100 Assignsthe192.168.0.100IPaddress toeth0(1IPaddressperinterface). ifconfigeth0down Shutsdowntheeth0interface (freesitsIPaddress).
3
Networksetup(2)
routeadddefaultgw192.168.0.1 Setsthedefaultrouteforpacketsoutsidethelocal network.Thegateway(here192.168.0.1)is responsibleforsendingthemtothenextgateway,etc., untilthefinaldestination. routen Liststheexistingroutes
noption:immediatelydisplaysipaddresses insteadoftryingtofindtheirdomain names
routedeldefault orroutedel<IP> Deletesthegivenroute Usefultoredefineanewroute.
4
Networksetup(3)
YourprogramsneedtoknowwhatIPaddresscorrespondsto agivenhostname(suchaskernel.org) DomainNameServers(DNS)takecareofthis. YoujusthavetospecifytheIPaddressof1ormoreDNS serversinyour/etc/resolv.conffile: nameserver217.19.192.132 nameserver212.27.32.177 Thechangestakeeffectimmediately!
5
Networktesting
First,trytopingtheIPaddressofyourgateway. Thiswillconfirmthatyournetworkadapterworksfine. Then,makesureyoucanpingthenameserverIPaddress, whichwillconfirmthatyourgatewayisconfiguredproperly. Finally,makesureyoucanpinganyhostusingitsname,which willconfirmthatthenameserverconfigurationiscorrect.
6
Filesystemsanddevices
7
Creatingfilesystems
Examples mkfs.ext2/dev/sda1 FormatsyourUSBkey(/dev/sda1:1stpartitionrawdata)inext2 format. F:force.Executeeven mkfs.ext2Fdisk.img Formatsadiskimagefileinext2format
ifnotarealdevicefile.
mkfs.vfatvF32/dev/sda1(v:verbose) FormatsyourUSBkeybacktoFAT32format. mkfs.vfatvF32disk.img FormatsadiskimagefileinFAT32format. Blankdiskimagescanbecreatedasinthebelowexample(64MBfile): ddif=/dev/zeroof=disk.imgbs=1Mcount=64
8
Mountingdevices(1)
Tomakefilesystemsonanydevice (internalorexternalstorage) visibleonyoursystem,youhavetomountthem. Thefirsttime,createamountpointinyoursystem: mkdir/mnt/usbdisk(example) Now,mountit: mounttvfat/dev/sda1/mnt/usbdisk /dev/sda1:physicaldevice t:specifiesthefilesystem(format)type (ext2,ext3,vfat,reiserfs,iso9660...)
rawdata
(filedescriptors andfilecontents)
mount
file structure
9
Mountingdevices(2)
Youcanalsomountafilesystemimage storedinaregularfile(loopdevices) Usefultodevelopfilesystemsforanothermachine UsefultoaccessthecontentsofanISOcdromimage withouthavingtoburnit. UsefultohaveaLinuxfilesysteminsideafileinaWindows partition. cp/dev/sda1usbkey.img mountolooptvfatusbkey.img/mnt/usbdisk
10
Listingmountedfilesystems
Justusethemountcommandwithnoargument:
/dev/hda6on/typeext3(rw,noatime) noneon/proctypeproc(rw,noatime) noneon/systypesysfs(rw) noneon/dev/ptstypedevpts(rw,gid=5,mode=620) usbfson/proc/bus/usbtypeusbfs(rw) /dev/hda4on/datatypeext3(rw,noatime) noneon/dev/shmtypetmpfs(rw) /dev/hda1on/wintypevfat(rw,uid=501,gid=501) noneon/proc/sys/fs/binfmt_misctypebinfmt_misc(rw)
11
Unmountingdevices
umount/mnt/usbdisk Commitsallpendingwritesandunmountsthegivendevice, whichcanthenberemovedinasafeway. Tobeabletounmountadevice,youhavetocloseallthe openfilesinit:
Closeapplicationsopeningdatainthemountedpartition Makesurethatnoneofyourshellshaveaworkingdirectoryin thismountpoint. Youcanrunthelsof<mountpoint>command(list openfiles)toviewwhichprocessesstillhaveopenfilesinthe mountedpartition.
12
Packagemanagement
13
Bewareofthedarksideofroot
rootuserprivilegesareonlyneededforveryspecific taskswithsecurityrisks:mounting,creatingdevice files,loadingdrivers,startingnetworking, changingfileownership,packageupgrades... Evenifyouhavetherootpassword,yourregular accountshouldbesufficientfor99.9%ofyourtasks (unlessyouareasystemadministrator). Inatrainingsession,itisacceptabletouseroot. Inreallife,youmaynotevenhaveaccesstothis account,orputyoursystemsanddataatriskifyou do.
14
Usingtherootaccount
Incaseyoureallywanttouseroot... Ifyouhavetherootpassword: su(switchuser) Inmoderndistributions,thesudocommandgivesyouaccesstosome rootprivilegeswithyourownuserpassword. Example:sudomount/dev/hda4/home
15
Softwarepackages
ThedistributionmechanismforsoftwareinGNU/Linuxisdifferent fromtheoneinWindows Linuxdistributionsprovidesacentralandcoherentwayof installing,updatingandremovingapplicationsandlibraries: packages Packagescontainstheapplicationorlibraryfiles,andassociated metainformation,suchastheversionandthedependencies
.debonDebianandUbuntu,.rpmonMandriva,Fedora,OpenSUSE
Packagesarestoredinrepositories,usuallyonHTTPorFTP servers Oneshouldonlyusepackagesfromofficialrepositoriesofits distribution,unlessstrictlyrequired.

16
Managingsoftwarepackages(1)
InstructionsforDebianbasedGNU/Linuxsystems (Debian,Ubuntu...) Packagerepositoriesarespecifiedin /etc/apt/sources.list Toupdatepackagerepositorylists: sudoaptgetupdate Tofindthenameofapackagetoinstall,thebestistousethe searchengineonhttp://packages.debian.orgoron http://packages.ubuntu.com.Youmayalsouse: aptcachesearch<keyword>
17
Managingsoftwarepackages(2)
Toinstallagivenpackage: sudoaptgetinstall<package> Toremoveagivenpackage: sudoaptgetremove<package> Toinstallallavailablepackageupdates: sudoaptgetdistupgrade Getinformationaboutapackage: sudoaptcacheshow<package> Graphicalinterfaces
SynapticforGNOME AdeptforKDE
Furtherdetailsonpackagemanagement: http://www.debian.org/doc/manuals/apthowto/
18
Shuttingdown
halt Immediatelyhaltsthesystem. reboot Immediatelyrebootsthesystem. [Ctrl][Alt][Del] AlsoworksonGNU/Linuxtoreboot. Embeddedsystems:youmustuseanimplementationof initandcanspecifyanykeycombinationin /etc/inittab.
19
EmbeddedLinuxsystemdevelopment
Crosscompiling toolchains
ThomasPetazzoni MichaelOpdenacker FreeElectrons
Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/toolchains Corrections,suggestions,contributionsandtranslationsarewelcome!
Definition(1)
TheusualdevelopmenttoolsavailableonaGNU/Linuxworkstationisa nativetoolchain Thistoolchainrunsonyourworkstationandgeneratescodeforyour workstation,usuallyx86 Forembeddedsystemdevelopment,itisusuallyimpossibleornot interestingtouseanativetoolchain Thetargetistoorestrictedintermsofstorageand/ormemory Thetargetisveryslowcomparedtoyourworkstation Youmaynotwanttoinstallalldevelopmenttoolsonyourtarget. Therefore,crosscompilingtoolchainsaregenerallyused.Theyrun onyourworkstationbutgeneratecodeforyourtarget.
2
Definition(2)
Sourcecode Crosscompiling toolchain x86 Execution machine ARM Compilation machine
Nativetoolchain
x86binary x86
ARMbinary
3
Components
Binutils
Kernelheaders
C/C++libraries
GCCcompiler
GDBdebugger (optional)
4
Binutils
Binutilsisasetoftoolstogenerateandmanipulatebinariesfora givenCPUarchitecture
as,theassembler,thatgeneratesbinarycodefromassembler sourcecode ld,thelinker ar,ranlib,togenerate.aarchives,usedforlibraries objdump,readelf,size,nm,strings,toinspectbinaries. Veryusefulanalysistools! strip,tostripuselesspartsofbinariesinordertoreducetheirsize
http://www.gnu.org/software/binutils/ GPLlicense
5
Kernelheaders(1)
TheClibraryandcompiledprograms needstointeractwiththekernel
Availablesystemcallsandtheir numbers Constantdefinitions Datastructures,etc.
Kernel Kernelheaders CLibrary Applications
Therefore,compilingtheClibrary requireskernelheaders,andmany applicationsalsorequirethem.
Availablein<linux/...>and<asm/...>andafewother directoriescorrespondingtotheonesvisibleininclude/inthe kernelsources

6
Kernelheaders(2)
Systemcallnumbers,in<asm/unistd.h>
#define__NR_exit1 #define__NR_fork2 #define__NR_read3
Constantdefinitions,herein<asmgeneric/fcntl.h>,included from<asm/fcntl.h>,includedfrom<linux/fcntl.h>
#defineO_RDWR00000002
Datastructures,herein<asm/stat.h>
structstat{ unsignedlongst_dev; unsignedlongst_ino; [...] };
7
Kernelheaders(3)
ThekerneltouserspaceABIisbackwardcompatible
Binariesgeneratedwithatoolchainusingkernelheadersolderthan therunningkernelwillworkwithoutproblem,butwon'tbeableto usethenewsystemcalls,datastructures,etc. Binariesgeneratedwithatoolchainusingkernelheadersnewer thantherunningkernelmightworkoniftheydon'tusetherecent features,otherwisetheywillbreak Usingthelatestkernelheadersisnotnecessary,unlessaccessto thenewkernelfeaturesisneeded
Thekernelheadersareextractedfromthekernelsourcesusing theheaders_installkernelMakefiletarget.
8
GCCcompiler
GNUCCompiler,thefamousfreesoftwarecompiler CancompileC,C++,Ada,Fortran,Java,Objective C,ObjectiveC++,andgeneratecodeforalarge numberofCPUarchitectures,includingARM,AVR, Blackfin,CRIS,FRV,M32,MIPS,MN10300, PowerPC,SH,v850,i386,x86_64,IA64,Xtensa,etc. http://gcc.gnu.org/ AvailableundertheGPLlicense, librariesundertheLGPL.
9
Clibrary
TheClibraryisanessentialcomponentofa Linuxsystem
Interfacebetweentheapplicationsandthe kernel ProvidesthewellknownstandardCAPIto easeapplicationdevelopment
Kernel
SeveralClibrariesareavailable: glibc,uClibc,eglibc,dietlibc,newlib,etc. ThechoiceoftheClibrarymustbemadeat thetimeofthecrosscompilingtoolchain generation,astheGCCcompileriscompiled againstaspecificClibrary.
CLibrary Applications
10
glibc
http://www.gnu.org/software/libc/ License:LGPL ClibraryfromtheGNUproject Designedforperformance,standardscomplianceand portability FoundonallGNU/Linuxhostsystems Ofcourse,activelymaintained Quitebigforsmallembeddedsystems:approx2.5MB onarm(version2.9libc:1.5MB,libm:750KB)
11
uClibc
http://www.uclibc.org/fromCodePoetConsulting License:LGPL LightweightClibraryforsmallembeddedsystems Highconfigurability:manyfeaturescanbeenabledor disabledthroughamenuconfiginterface WorksonlywithLinux/uClinux,worksonmostembedded architectures NostableABI,differentABIdependingonthelibrary configuration Focusonsizeratherthanperformance Smallcompiletime
12
uClibc(2)
MostoftheapplicationscompilewithuClibc.Thisappliestoall applicationsusedinembeddedsystems. Size(arm):4timessmallerthanglibc! uClibc0.9.30.1:approx.600KB(libuClibc:460KB,libm:96KB) glibc2.9:approx2.5MB Usedonalargenumberofproductionembeddedproducts,including consumerelectronicdevices Activelymaintained,largedeveloperanduserbase NowsupportedbyMontaVista,TimeSysandWindRiver.
13
uClibc(3)
Aftercompilationandinstallation,thefollowingcomponentsareavailable Standardheaders,stdio.h,stdlib.h,unistd.handothers, andLinuxkernelheaders,integratedwiththeClibraryheaders. Thelibrariesthemselves,withmainly libuClibc,theClibraryitself lduClibc,thedynamicloader,responsibleforloadingthe sharedlibrariesatthebeginningofaprogram'sexecution librt,thelibraryimplementingtherealtimerelatedfunctions libstdc++,theC++standardlibrary libpthread,thethreadslibrary libm,themathematiclibrary
14
Honey,Ishrunktheprograms!
Cprogram Plainhelloworld (stripped) Busybox (stripped)
Compiledwithsharedlibraries glibc uClibc 5.6K 5.4K (glibc2.9) (uClibc0.9.30.1) 245K (olderglibc) 231K (olderuClibc)
Compiledstatically glibc 472K (glibc2.9) 843K (olderglibc) uClibc 18K (uClibc 0.9.30.1) 311K (olderuClibc)
ExecutablesizecomparisononARM
15
eglibc
Embeddedglibc,undertheLGPL VariantoftheGNUCLibrary(GLIBC)designedtoworkwellon embeddedsystems StrivestobesourceandbinarycompatiblewithGLIBC eglibc'sgoalsincludereducedfootprint,configurablecomponents, bettersupportforcrosscompilationandcrosstesting. CanbebuiltwithoutsupportforNIS,locales,IPv6,andmanyother features. Supportedbyaconsortium,withFreescale,MIPS,MontaVistaand WindRiverasmembers. TheDebiandistributionisswitchingtoeglibctoo: http://blog.aurel32.net/?p=47 http://www.eglibc.org
16
OthersmallerClibraries
SeveralothersmallerClibrarieshavebeendeveloped,but noneofthemhavethegoalofallowingthecompilationof largeexistingapplications Theyneedspeciallywrittenprogramsandapplications Choices:
Dietlibc,http://www.fefe.de/dietlibc/.Approximately70KB. Newlib,http://sourceware.org/newlib/
Klibc,http://www.kernel.org/pub/linux/libs/klibc/,designed foruseinaninitramfsorinitrdatboottime.
17
Buildingatoolchain
Threemachinesmustbedistinguishedwhendiscussing toolchaincreation
Thebuildmachine,wherethetoolchainisbuilt. Thehostmachine,wherethetoolchainwillbeexecuted. Thetargetmachine,wherethebinariescreatedbythe toolchainwillbeexecuted.
Fourbuildtypesarepossible
18
Buildingatoolchain(2)
build host Nativebuild
usedtobuildthenormalgccofa workstation
target
build
host Crossbuild
target
usedtobuildatoolchainthatrunsonyour workstationbutgeneratesbinariesforthe target Themostcommonsolutioninembedded
build
host
target
build
host
target
Crossnativebuild
usedtobuildatoolchainthatrunsonyour targetandgeneratesbinariesforthe target
Canadianbuild
usedtobuildonarchitectureAatoolchain thatrunsonarchitectureBandgenerates binariesforarchitectureC 19
Manydecisionsmustbemadewhenbuildingatoolchain ChoosingtheClibrary Choosingtheversionofthedifferentcomponents Choosingtheconfigurationofthetoolchain
WhichABIshouldbeused?ToolchainsfortheARM architectureforexample,cangeneratebinariesusingthe OABI(OldABI)ortheEABI(EmbeddedABI),thatare incompatible Shouldthetoolchainsupportsoftwarefloatingpoint,ordoes thehardwaresupportfloatingpointoperations? Shouldthetoolchainsupportlocales,IPv6,orotherspecific features?
20
Crosstoolbuildreports:http://kegel.com/crosstool/crosstool0.43/buildlogs
21
Basicsteps
Extractandinstallkernelheaders Extract,configure,compileandinstallbinutils Extract,configureandcompileafirstversiongccthat generatesbinariesforthetarget.Itwillbeusedtocross compiletheClibrary. Extract,configureandcompiletheClibraryusingthe previouslygeneratedcompiler. Reconfigureandcompilethefinalgcccrosscompiler.
22
Homebuilttoolchains
Buildingacrosscompilingtoolchainbyyourselfisadifficult andpainfultask!Cantakedaysorweeks!
Lotsofdetailstolearn.Severalcomponentstobuild. Lotsofdecisionstomake (suchasClibraryversionandconfigurationforyourplatform) NeedkernelheadersandClibrarysources Needtobefamiliarwithcurrentgccissuesandpatchesonyour platform Usefultobefamiliarwithbuildingandconfiguringtools
http://www.aleph1.co.uk/armlinux/docs/toolchain/toolchHOWTO.pdf canshowyouhowfunitcanbe!
23
Getaprecompiledtoolchain
Solutionthatmostpeoplechoose,becauseitisthesimplest andmostconvenientsolution First,determinewhattoolchainyouneed:CPU,endianism,C library,componentversions,ABI,softfloatorhardfloat,etc. Manyprecompiledtoolchainsarefreelyavailable: CodeSourcery,http://www.codesourcery.com,isa referenceinthatarea,buttheyonlyprovideglibc toolchains. Linarowillgiveyouthemostoptimizedtoolchainson ARM:https://wiki.linaro.org/WorkingGroups/ToolChain (LinarohiredCodeSourceryandaddedotherfeatures) Seealsohttp://elinux.org/Toolchains
24
Installingandusingaprecompiledtoolchain
Followtheinstallationprocedureproposedbythevendor Usually,itissimplyamatterofextractingatarballatthe properplace Toolchainsusednottoberelocatable! Youmustinstalltheminthelocationtheywerebuiltfor. Thisisnolongertruewithgcc4.x,thankstosysroot support,butitisstillmoreconvenienttoinstallthemat theproperplace. Then,addthepathtotoolchainbinariesinyourPATH: exportPATH=/path/to/toolchain/bin/:$PATH
25
Toolchainbuildingutilities
Anothersolutionistouseutilitiesthatautomatetheprocessof buildingthetoolchain Sameadvantageastheprecompiledtoolchains:youdon'tneed tomessupwithallthedetailsofthebuildprocess Butalsooffersmoreflexibilityintermsoftoolchainconfiguration, componentversionselection,etc. Theyalsousuallycontainseveralpatchesthatfixknownissues withthedifferentcomponentsonsomearchitectures Identicalprinciple:shellscriptsorMakefilethatautomatically fetch,extract,configure,compileandinstallthedifferent components
26
Toolchainbuildingutilities(2)
Crosstool
Theprecursor,writtenbyDanKegel Setofscriptsandpatches,glibconly Notreallymaintainedanymore http://www.kegel.com/crosstool
Crosstoolng
RewriteofCrosstool,withamenuconfiglikeconfigurationsystem Featurefull:supportsuClibc,glibc,eglibc,hardandsoftfloat,many architectures Activelymaintained http://ymorin.isageek.org/dokuwiki/projects/crosstool
27
Toolchainbuildingutilities(3)
Manyrootfilesystembuildingsystemsalsoallowtheconstructionof crosscompilingtoolchain Buildroot Makefilebased,uClibconly,maintainedbythecommunity http://buildroot.uclibc.org PTXdist Makefilebased,uClibcorglibc,maintainedmainlyby Pengutronix http://www.pengutronix.de/software/ptxdist/index_en.html OpenEmbedded Thefeaturefull,butcomplexbuildingsystem http://www.openembedded.org/
28
PracticallabUsingCrosstoolNG
Timetobuildyourtoolchain ConfigureCrosstoolNG Runittobuildyourowncross compilingtoolchain
29
Bootloaders
Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/bootloaders Corrections,suggestions,contributionsandtranslationsarewelcome!
Bootloaders
Thebootloaderisapieceofcoderesponsiblefor
Basichardwareinitialization Loadingofanapplicationbinary,usuallyanoperatingsystem kernel,fromflashstorage,fromthenetwork,orfromanothertypeof nonvolatilestorage. Possiblyuncompressionoftheapplicationbinary Executionoftheapplication
Besidesthesebasicfunctions,mostbootloadersprovideashell withvariouscommandsimplementingdifferentoperations.
Loadingofdatafromstorageornetwork,memoryinspection, hardwarediagnosticsandtesting,etc.
2
Bootloadersonx86(1)
Thex86processorsaretypicallybundledonaboardwithanon volatilememorycontainingaprogram,theBIOS. ThisprogramgetsexecutedbytheCPUafterreset,andis responsibleforbasichardwareinitializationandloadingofa smallpieceofcodefromanonvolatilestorage.
Thispieceofcodeisusuallythefirst512bytesofanharddisk
Thispieceofcodeisusuallya1ststagebootloader,whichwill loadthefullbootloaderitself. Thebootloadercanthenofferallitsfeatures.Ittypically understandsfilesystemformatssothatthekernelfilecanbe loadeddirectlyfromanormalfilesystem.

3
Bootloadersonx86(2)
GRUB,GrandUnifiedBootloader,themostpowerfulone. http://www.gnu.org/software/grub/ Canreadmanyfilesystemformatstoloadthekernelimage andtheconfiguration,providesapowerfulshellwithvarious commands,canloadkernelimagesoverthenetwork,etc. Seeourdedicatedpresentationfordetails: http://freeelectrons.com/docs/grub/ LILO,theoriginalLinuxLoader http://freshmeat.net/projects/lilo/ Syslinux,fornetworkandremovablemediabooting http://syslinux.zytor.com
4
Bootloadersonembeddedarchitectures(1)
Onembeddedarchitectures,thelowlevelbootingprocessisveryCPUand boarddependent SomeboardshaveaNORflashfromwhichtheCPUstartsexecuting instructionsafterreset.Inthatcase,thebootloadermustdirectlybeflashed insidetheNORattheproperlocation SomeCPUshaveanintegratedbootcodeinROMthatautomaticallyloadsa smallportionofaDataFlashorNANDflash,usuallytoastaticRAM.Inthat case,aminimalfirststagebootloaderisrequired,thatwillloadthemain bootloader(BootROMonAT91SAMCPUs,SteppingstoneonS3C24xx CPUs,etc.). ThebootloaderonembeddedarchitecturesstartsrightafterCPUreset,soit mustinitializeallthedevices,includingthememorycontrollerinordertoaccess theDRAM. AsthebootprocessisveryCPUandboarddependent, refertothevendordocumentation.
5
Bootloadersonembeddedarchitectures(2)
Wewillfocusonthegenericpart,themainbootloader,offering themostimportantfeatures. Thereareseveralopensourcegenericbootloaders. Herearethemostpopularones:
UBoot,theuniversalbootloaderbyDenx ThemostusedonARM,alsousedonPPC,MIPS,x86,m68k,NIOS,etc. Thedefactostandardnowadays.Wewillstudyitindetail. http://www.denx.de/wiki/UBoot Barebox,anewarchitectureneutralbootloader,writtenasasuccessorof UBoot.Betterdesign,bettercode,activedevelopment,butdoesn'tyet haveasmuchhardwaresupportasUBoot. http://www.barebox.org
Therearealsoalotofotheropensourceorproprietary bootloaders,oftenarchitecturespecific
RedBoot,Yaboot,PMON,etc.
6
Accessingaserialconsole
7
Minicom(1)
Definition:serialcommunicationprogram AvailableinallGNU/Linuxdistributions Capabilities(allthroughaseriallink):
SerialconsoletoaremoteUnixsystem Filetransfer Modemcontrolanddialup Serialportconfiguration
8
Minicom(2)
Startbyrunning minicomstosetup Minicom Abitaustereatfirstglance, butquicklygetsfriendly (seethelabsfordetails)
9
Otherterminalemulators
GTKTerm:http://www.jlsinfo.com/julien/linux/ Graphical.LesspowerfulthanMinicom,butwithasimplerand moreattractiveinterface.Availableinrecentdistros. CuteCom:http://cutecom.sourceforge.net/ Anothergraphicalanduserfriendlyterminalemulator. Availableinrecentdistros. picocom:http://freshmeat.net/projects/picocom/ Tinyterminalemulator(20K),canbeusedinembeddedsystems. GNUScreen:canalsobeusedonaserialconsole: screen<device><baudrate> Example: screen/dev/ttyS0115200
10
TheUbootbootloader
TheUboot bootloader
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/uboot Corrections,suggestions,contributionsandtranslationsarewelcome!
UBoot
UBootisatypicalfreesoftwareproject Freelyavailableathttp://www.denx.de/wiki/UBoot Documentationavailableat http://www.denx.de/wiki/UBoot/Documentation ThelatestdevelopmentsourcecodeisavailableinaGit repository: http://git.denx.de/cgibin/gitweb.cgi?p=uboot.git;a=summary Developmentanddiscussionshappenaroundanopenmailing listhttp://lists.denx.de/pipermail/uboot/ Sincetheendof2008,itfollowsafixedintervalreleaseschedule. Everytwomonths,anewversionisreleased.Versionsare namedYYYY.MM.
2
CompilingUBoot(1)
Getthesourcecodefromthewebsite,anduncompressit Theinclude/configs/directorycontainsoneconfiguration fileforeachsupportedboard
ItdefinestheCPUtype,theperipheralsandtheirconfiguration,the memorymapping,theUBootfeaturesthatshouldbecompiledin, etc. Itisasimple.hfilethatsetspreprocessorconstants.Seethe READMEfileforthedocumentationoftheseconstants.
AssumingthatyourboardisalreadysupportedbyUBoot,there shouldbeonefilecorrespondingtoyourboard,forexample include/configs/omap2420h4.h.
3
CompilingUBoot(2)
UBootmustbeconfiguredbeforebeingcompiled
makeBOARDNAME_config WhereBOARDNAMEisthenameoftheconfigurationfilein include/configs/,withoutthe.h
MakesurethatthecrosscompilerisavailableinPATH
exportPATH=/usr/local/uclibc0.9.292/arm/bin/:$PATH
CompileUBoot,byspecifyingthecrosscompilerprefix. Example,ifyourcrosscompilerexecutableisarmlinuxgcc: makeCROSS_COMPILE=armlinux
4
InstallingUBoot
UBootmustusuallybeinstalledinaFlashmemorytobe executedbythehardware.Dependingonthehardware,the installationofUBootisdoneinadifferentway
Theboardprovidessomekindofspecificbootmonitor,which allowstoflashthesecondstagebootloader.Inthiscase,referto theboarddocumentationandtools UBootisalreadyinstalled,andcanbeusedtoFlashanewversion ofUBoot.However,becareful:ifthenewversionofUBootdoesn't work,theboardisunusable TheboardprovidesaJTAGinterface,whichallowstowritetothe Flashmemoryremotely,withoutanysystemrunningontheboard.It alsoallowstorescueaboardifthebootloaderdoesn'twork.
5
Ubootprompt
Connectthetargettothehostthroughaserialconsole Poweruptheboard.Ontheserialconsole,youwillsee somethinglike: UBoot1.1.2(Aug3200417:31:20) RAMConfiguration: Bank#0:000000008MB Flash:2MB In:serial Out:serial Err:serial uboot# TheUBootshelloffersasetofcommands.Wewillstudy themostimportantones,seethedocumentationfora completereferenceorthehelpcommand.
6
Informationcommands
UBoot>flinfo DataFlash:AT45DB021 Nbpages:1024 PageSize:264 Size=270336bytes Logicaladdress:0xC0000000 Area0:C0000000toC0001FFF(RO)Bootstrap Area1:C0002000toC0003FFFEnvironment Area2:C0004000toC0041FFF(RO)UBoot UBoot>nandinfo Device0:NAND256MiB3,3V8bit,sectorsize128KiB UBoot>version UBoot2009.08(Nov15200914:48:35)
Flash information
NANDFlash information UBoot information
Canvaryfromoneboardtotheother (accordingtotheUBootcompileconfiguration)
7
Environmentvariables(1)
UBootcanbeconfiguredthroughenvironmentvariables,which affectthebehaviorofthedifferentcommands. Seethedocumentationforthecompletelistofenvironment variables. Theprintenvcommandalsotodisplayallvariablesorone:
uboot#printenv baudrate=19200 ethaddr=00:40:95:36:35:33 netmask=255.255.255.0 ipaddr=10.0.0.11 serverip=10.0.0.1 stdin=serial stdout=serial stderr=serial uboot#printenvserverip serverip=10.0.0.2
Networkconfiguration
Environmentvariables(2)
Thevalueoftheenvironmentvariablescanbechangedusing thesetenvcommand: uboot#setenvserverip10.0.0.2 Environmentvariablechangescanbestoredtoflashusingthe saveenvcommand.Thelocationinflashisdefinedatcompile timeintheUBootconfigurationfile. Youcanevencreatesmallscriptsstoredinenvironment variables: setenvmmcboot'mmcinit0;iffatloadmmc0 80000000boot.ini;thensource;elseif fatloadmmc080000000uImage;thenrunmmc bootargs;bootm;fi;fi' Youcanthenexecutethescript: runmmcboot
9
Transferringfilestothetarget
UBootismostlyusedtoloadandbootakernelimage,butitalso allowstochangethekernelimageandtherootfilesystemstored inflash. Filesmustbeexchangedbetweenthetargetandthe developmentworkstation.Thisispossible:
ThroughthenetworkifthetargethasanEthernetconnection,and UBootcontainsadriverfortheEthernetchip.Ifso,theTFTP protocolcanbeusedtoexchangefiles ThroughtheseriallineifnoEthernetconnectionisavailable.
Target UBootTFTP client Ethernetconnection
Host TFTP server

10
Configuringandtestingtftp
OnGNU/LinuxsystemsbasedonDebian:Ubuntu,Knoppix Installthetftpdhpapackage(tftpserver): aptgetinstalltftpdhpa Copyfilestotherootdirectoryofthetftpserver.Example: cparch/arm/boot/uImage/var/lib/tftpboot Totesttheserver,installatftpclientonyourworkstation: aptgetinstalltftphpa Useittodownloadafile(4toforcetheuseofIPv4) tftp4localhost >getuImage
11
Ubootmkimage
ThekernelimagethatUBootloadsandbootsmustbeprepared, sothatanUBootspecificheaderisaddedinfrontoftheimage ThisisdonewithatoolthatcomesinUBoot,mkimage Debian/Ubuntu:justinstalltheubootmkimagepackage. Or,compileitbyyourself:simplyconfigureUBootforanyboard ofanyarchitectureandcompileit.Theninstallmkimage: cptools/mkimage/usr/local/bin/ ThespecialtargetuImageofthekernelMakefilecanthenbe usedtogenerateakernelimagesuitableforUBoot.
12
Flashingakernelimage
CompileyourkernelandgeneratetheUBootheader runningmakeuImage CopythekernelimagetothedirectoryexportedbytheTFTPserver Ontheboard,inUBoot,downloadthekernelimagetomemory: uboot#tftp8000uImage UnprotectNORflash uboot#protectoff1:04 EraseNORflash uboot#erase1:04 CopytoNORflash(0x01000000:firstsector) uboot#cp.b${fileaddr}1000000${filesize} RestoreNORflashsectorprotection: uboot#protecton1:04 SeeourpracticallabsfordetailshandlingNANDflash.
13
bootcommands
Specifykernelbootparameters: uboot#setenvbootargsmem=64M\ console=ttyS0,115200init=/sbin/init\ root=/dev/mtdblock0 Executethekernelfromagivenphysicaladdress (RAMorflash): bootm0x01030000
Continueson thesameline
14
OMAPbootcomponents
AtleastofOMAP3andOMAP4 Romcode
FoundinROM ExecutedfromROMatboot FoundinNANDorinMMC(MLOfile) CopiedtotheCPU'sinternalRAM. InitializestheexternalDRAMcontroller+otherdevices FoundinNANDorinMMC(uboot.binfile) CopiedtoDRAM Canalsoinitializesomedevices FoundinNAND,MMC,USB,network,serial CopiedtoDRAM (dependingonUboot'sfeatures)
1
Xloader
Uboot
Linuxkernel
PracticallabUBoot
Timetostartthepracticallab! ConfigureandcompileUboot foryourARMboard. Flashitonyourboard
1
FreeElectrons
Linuxkernel introduction
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelintro Corrections,suggestions,contributionsandtranslationsarewelcome!
EmbeddedLinuxdriverdevelopment
Kerneloverview
Linuxfeatures
2
Linuxkernelinthesystem
UserappB LibraryA UserappA Clibrary Calltoservices LinuxKernel Manage hardware Hardware
3
Userspace
Eventnotification, informationexposition
Eventnotification
History
TheLinuxkernelisonecomponentofasystem,whichalso requireslibrariesandapplicationstoprovidefeaturestoend users. TheLinuxkernelwascreatedasahobbyin1991 byaFinnishstudent,LinusTorvalds.
Linuxquicklystartedtobeusedasthekernelforfreesoftware operatingsystems
LinusTorvaldshasbeenabletocreatealargeanddynamic developerandusercommunityaroundLinux. Nowadays,hundredsofpeoplecontributetoeachkernelrelease, individualsorcompaniesbigandsmall.
4
Linuxlicense
ThewholeLinuxsourcesareFreeSoftwarereleased undertheGNUGeneralPublicLicenseversion2(GPLv2).
FortheLinuxkernel,thisbasicallyimpliesthat: WhenyoureceiveorbuyadevicewithLinuxonit, youshouldreceivetheLinuxsources,withtherightto study,modifyandredistributethem. WhenyouproduceLinuxbaseddevices,youmust releasethesourcestotherecipient,withthesamerights, withnorestriction. Seeourhttp://freeelectrons.com/articles/freesw/trainingfor exactdetailsaboutFreeSoftwareanditslicenses.

5
Linuxkernelkeyfeatures
Portabilityandhardware support Runsonmostarchitectures. Scalability Canrunonsupercomputers aswellasontinydevices (4MBofRAMisenough). Compliancetostandardsand interoperability. Exhaustivenetworking support. Security Itcan'thideitsflaws.Itscode isreviewedbymanyexperts. Stabilityandreliability. Modularity Canincludeonlywhata systemneedsevenatrun time. Easytoprogram Youcanlearnfromexisting code.Manyusefulresources onthenet.
6
Supportedhardwarearchitectures
2.6.31status Seethearch/directoryinthekernelsources Minimum:32bitprocessors,withorwithoutMMU,andgccsupport 32bitarchitectures(arch/subdirectories) arm,avr32,blackfin,cris,frv,h8300,m32r,m68k, m68knommu,microblaze,mips,mn10300,parisc,s390, sparc,um,xtensa 64bitarchitectures: alpha,ia64,sparc64 32/64bitarchitectures powerpc,x86,sh Finddetailsinkernelsources:arch/<arch>/Kconfig, arch/<arch>/README,orDocumentation/<arch>/
7
Systemcalls
Themaininterfacebetweenthekernelanduserspaceistheset ofsystemcalls About~300systemcallsthatprovidesthemainkernelservices
Fileanddeviceoperations,networkingoperations,interprocess communication,processmanagement,memorymapping,timers, threads,synchronizationprimitives,etc.
Thisinterfaceisstableovertime:onlynewsystemcallscanbe addedbythekerneldevelopers ThissystemcallinterfaceiswrappedbytheClibrary,and userspaceapplicationsusuallynevermakeasystemcalldirectly butratherusethecorrespondingClibraryfunction
8
Virtualfilesystems
Linuxmakessystemandkernelinformationavailablein userspacethroughvirtualfilesystems(virtualfilesnot existingonanyrealstorage).Noneedtoknowkernel programmingtoaccesssuchinformation! Mounting/proc: sudomounttprocnone/proc Mounting/sys: sudomounttsysfsnone/sys
Filesystemtype Mountpoint Rawdevice orfilesystemimage Inthecaseofvirtual filesystems,anystringisfine
9
/procdetails
Afewexamples: /proc/cpuinfo:processorinformation /proc/meminfo:memorystatus /proc/version:kernelversionandbuildinformation /proc/cmdline:kernelcommandline /proc/<pid>/environ:callingenvironment /proc/<pid>/cmdline:processcommandline ...andmanymore!Seebyyourself! Lotsofdetailsaboutthe/procinterfaceareavailablein Documentation/filesystems/proc.txt (almost2000lines)inthekernelsources.
10
EmbeddedLinuxusage
Kerneloverview
Linuxversioningschemeanddevelopmentprocess
11
Until2.6(1)
Onestablemajorbranchevery2or3years Identifiedbyanevenmiddlenumber Examples:1.0,2.0,2.2,2.4 Onedevelopmentbranchtointegratenewfunctionalitiesand majorchanges Identifiedbyanoddmiddlenumber Examples:2.1,2.3,2.5 Aftersometime,adevelopmentversionbecomesthenew baseversionforthestablebranch Minorreleasesonceinwhile:2.2.23,2.5.12,etc.
12
Until2.6(2)
Stableversion 2.4.0 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.4.6 2.4.7 2.4.8
2.5.0
2.5.1 2.5.2
2.5.3
2.5.4
2.6.0
2.6.1 Stable
Development
Note:inreality,manymoreminor versionsexistinsidethestableand developmentbranches
13
ChangessinceLinux2.6(1)
Since2.6.0,kerneldevelopershavebeenableto introducelotsofnewfeaturesonebyoneonasteadypace, withouthavingtomakemajorchangesinexisting subsystems. OpeninganewLinux2.7(or2.9)developmentbranchwill berequiredonlywhenLinux2.6isnolongerableto accommodatekeyfeatureswithoutundergoingtraumatic changes. Thankstothis,morefeaturesarereleasedtousersata fasterpace.
14
ChangessinceLinux2.6(2)
Since2.6.14,thekerneldevelopersagreed onthefollowingdevelopmentmodel: Afterthereleaseofa2.6.xversion,atwoweeksmergewindow opens,duringwhichmajoradditionsaremerged. Themergewindowisclosed bythereleaseoftestversion2.6.(x+1)rc1 Thebugfixingperiodopens,for6to10weeks. Atregularintervalsduringthebugfixingperiod, 2.6.(x+1)rcYtestversionsarereleased. Whenconsideredsufficientlystable, kernel2.6.(x+1)isreleased,andtheprocessstartsagain.
15
Mergeandbugfixingwindows
2weeks
Mergewindow 2.6.21 2.6.22rc1 2.6.22rc2
6to10weeks
Bugfixingperiod 2.6.22rc3 2.6.22rc4 2.6.22rc5 2.6.22
2.6.21.1 2.6.21.2
2.6.21.3 2.6.21.4
2.6.21.5
2.6.22.1
Bugfixupdates
16
Morestabilityforthe2.6kerneltree
Issue:bugandsecurityfixesonlyreleasedforlast(orlast two)stablekernelversions(like2.6.16and2.6.17),andof coursebydistributionsfortheexactversionthatyou're using. Somepeopleneedtohavearecentkernel,butwithlong termsupportforsecurityupdates. Youcouldgetlongtermsupportfromacommercial embeddedLinuxprovider. YoucouldreusesourcesforthekernelusedinUbuntuLong TermSupportreleases(5yearsoffreesecurityupdates). YoucouldchooseLinux2.6.27foryourproject,whichwillbe maintainedbykernel.orgforalongtime,unlikeother versions.
17
What'snewineachLinuxrelease?
commit3c92c2ba33cd7d666c5f83cc32aa590e794e91b0 Author:AndiKleen<ak@suse.de> Date:TueOct1101:28:332005+0200 [PATCH]i386:Don'tdiscardupper32bitsofHWCRonK8 Needtouselonglong,notlongwhenRMWingaMSR.Ithink it'sharmlessrightnow,butstillshouldbebetterfixed ifAMDaddsanybitsintheupper32bitofHWCR. BugwasintroducedwiththeTLBflushfilterfixfori386 Signedoffby:AndiKleen<ak@suse.de> Signedoffby:LinusTorvalds<torvalds@osdl.org> ...
??!
TheofficiallistofchangesforeachLinuxreleaseisjusta hugelistofindividualpatches! Verydifficulttofindoutthekeychangesandtogetthe globalpictureoutofindividualchanges. Fortunately,asummaryofkeychanges withenoughdetailsisavailableon http://wiki.kernelnewbies.org/LinuxChanges
18
EmbeddedLinuxkernelusage
EmbeddedLinux kernelusage
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelusage Corrections,suggestions,contributionsandtranslationsarewelcome!
Contents
Compilingandbooting Linuxkernelsources Kernelconfiguration Compilingthekernel Overallsystemstartup Linuxdevicefiles Crosscompilingthekernel
2
EmbeddedLinuxusage
CompilingandbootingLinux
Linuxkernelsources
3
Locationofkernelsources
TheofficialversionoftheLinuxkernel,asreleasedbyLinus Torvaldsisavailableathttp://www.kernel.org
Thisversionfollowsthewelldefineddevelopmentmodelofthe kernel However,itmaynotcontainthelatestdevelopmentfromaspecific area,duetotheorganizationofthedevelopmentmodeland becausefeaturesindevelopmentmightnotbereadyformainline inclusion
Manykernelsubcommunitiesmaintaintheirownkernel,with usuallynewerbutlessstablefeatures
Architecturecommunities(ARM,MIPS,PowerPC,etc.),device driverscommunities(I2C,SPI,USB,PCI,network,etc.),other communities(realtime,etc.) Theygenerallydon'treleaseofficialversions,onlydevelopment treesareavailable
4
Linuxkernelsize(1)
Linux2.6.31sources: Rawsize:350MB(30,900files,approx12,000,000lines) gzipcompressedtararchive:75MB bzip2compressedtararchive:59MB(better) lzmacompressedtararchive:49MB(best) MinimumLinux2.6.29compiledkernelsizewithCONFIG_EMBEDDED, forakernelthatbootsaQEMUPC(IDEharddrive,ext2filesystem, ELFexecutablesupport): 532KB(compressed),1325KB(raw) Whyarethesesourcessobig? Becausetheyincludethousandsofdevicedrivers,manynetwork protocols,supportmanyarchitecturesandfilesystems... TheLinuxcore(scheduler,memorymanagement...)isprettysmall!
5
Linuxkernelsize(2)
SizeofLinuxsourcedirectories(KB)
arch block crypto Documentation drivers fs include init ipc kernel lib mm net scripts security sound usr 0 50000 100000 150000
Linux2.6.17
Measuredwith: dusapparentsize
6
GettingLinuxsources
Fulltarballs
Containthecompletekernelsources Longtodownloadanduncompress,butmustbedoneatleastonce Example: http://kernel.org/pub/linux/kernel/v2.6/linux2.6.14.7.tar.bz2 Incrementalpatchesbetweenversions Itassumesyoualreadyhaveabaseversionandyouapplythe correctpatchesintherightorder Quicktodownloadandapply Examples
http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.bz2(2.6.13to2.6.14) http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.7.bz2(2.6.14to2.6.14.7)
Allpreviouskernelversionsareavailablein http://kernel.org/pub/linux/kernel/
Usingthepatchcommand
Thepatchcommandapplieschanges tofilesinthecurrentdirectory: Makingchangestoexistingfiles Creatingordeletingfilesanddirectories patchusageexamples: patchp<n><diff_file catdiff_file|patchp<n> bzcatdiff_file.bz2|patchp<n> zcatdiff_file.gz|patchp<n>
Youcantestapatchwith thedryrun option Youcanreverse apatch withtheR option
n:numberofdirectorylevelstoskipinthefilepaths
8
Anatomyofapatchfile
Apatchfileistheoutputofthediffcommand
diffcommandline diffNrua/Makefileb/Makefile a/Makefile2005030409:27:1508:00 Filedateinfo +++b/Makefile2005030409:27:1508:00 @@1,7+1,7@@ Linenumbersinfiles VERSION=2 Contextinfo:3linesbeforethechange PATCHLEVEL=6 Usefultoapplyapatchwhenlinenumbers SUBLEVEL=11 changed EXTRAVERSION= Removedline(s)ifany +EXTRAVERSION=.1 Addedline(s)ifany NAME=WoozyNumbat
#*DOCUMENTATION*
Contextinfo:3linesafterthechange
9
ApplyingaLinuxpatch
Linuxpatches... Alwaystoapplytothex.y.<z1>version Downloadableingzip andbzip2(muchsmaller)compressedfiles. Alwaysproducedforn=1 (that'swhateverybodydoes...doittoo!) Youcanmakepatch30% fasterbyusingsp1 insteadofp1 (silent)
Testedonpatch2.6.23.bz2
Linuxpatchcommandlineexample: cdlinux2.6.13 bzcat../patch2.6.14.bz2|patchp1 bzcat../patch2.6.14.7.bz2|patchp1 cd..;mvlinux2.6.13linux2.6.14.7
Keeppatchfilescompressed:usefultochecktheirsignaturelater. Youcanstillview(orevenedit)theuncompresseddatawithvim: vimpatch2.6.14.bz2(onthefly(un)compression)
10
KetchupEasyaccesstokernelsources
http://www.selenic.com/ketchup/ Makesiteasytodownloadaspecificversion. Takescareofdownloadingandapplyingpatches Example:downloadingthelatestkernelversion >mkdirlinux2.6.31 >cdlinux2.6.31 >ketchupG2.6tip None>2.6.31.6 Downloadinglinux2.6.31.6.tar.bz2 Unpackinglinux2.6.31.6.tar.bz2
TheGoptionofketchup disablessourcesignature checking. See http://kernel.org/signature.html fordetailsaboutenabling kernelsource integritychecking.
Nowgettingbacktoanolderversion(fromthesamedirectory) >ketchupG2.6.30 2.6.31.6>2.6.30 Downloadingpatch2.6.31.6.bz2 Applyingpatch2.6.31.bz2R Downloadingpatch2.6.30.bz2 Applyingpatch2.6.30.bz2R

11
PracticallabKernelsources
Getthesources Applypatches
12
EmbeddedLinuxusage
Kernelconfiguration
13
Kernelconfiguration(1)
Thekernelcontainsthousandsofdevicedrivers,filesystem drivers,networkprotocolsandotherconfigurableitems Thousandsofoptionsareavailable,thatareusedtoselectively compilepartsofthekernelsourcecode Thekernelconfigurationistheprocessofdefiningthesetof optionswithwhichyouwantyourkerneltobecompiled Thesetofoptionsdepends
Onyourhardware Onthecapabilitiesyouwouldliketogivetoyourkernel
14
Kernelconfiguration(2)
Theconfigurationisstoredinthe.configfileattherootofkernel sources Simpletextfile,key=valuestyle Asoptionshavedependencies,typicallynevereditedbyhand,but throughgraphicalinterfaces:
make[xconfig|gconfig|menuconfig|oldconfig] ThesearetargetsfromthemainkernelMakefile.Runmakehelpto getalistofallavailabletargets.
TomodifyakernelinaGNU/Linuxdistribution: theconfigurationfilesareusuallyreleasedin/boot/,togetherwith kernelimages:/boot/config2.6.1711generic

15
makexconfig
makexconfig Themostcommongraphicalinterface toconfigurethekernel. Makesureyouread help>introduction:usefuloptions! Filebrowser:easiertoloadconfigurationfiles Newsearchinterfacetolookforparameters RequiredDebian/Ubuntupackages: libqt3mtdev,g++
16
makexconfigscreenshot
17
makexconfigsearchinterface
Looksforakeyword inthedescription string Allowstoselect orunselectfound parameters.
18
Kernelconfigurationoptions
Compiledasamodule(separatefile) CONFIG_ISO9660_FS=m
Driveroptions CONFIG_JOLIET=y CONFIG_ZISOFS=y
Compiledstaticallyintothekernel CONFIG_UDF_FS=y
19
Corresponding.configfileexcerpt
# #CDROM/DVDFilesystems # CONFIG_ISO9660_FS=m CONFIG_JOLIET=y CONFIG_ZISOFS=y CONFIG_UDF_FS=y CONFIG_UDF_NLS=y
Sectionname (helpstolocatesettingsintheinterface)
Allparametersareprefixed withCONFIG_
# #DOS/FAT/NTFilesystems # #CONFIG_MSDOS_FSisnotset #CONFIG_VFAT_FSisnotset CONFIG_NTFS_FS=m #CONFIG_NTFS_DEBUGisnotset CONFIG_NTFS_RW=y

20
Kerneloptiondependencies
Therearedependenciesbetweenkerneloptions Forexample,enablinganetworkdriverrequiresthenetwork stacktobeenabled Twotypesofdependencies
dependsondependencies.Inthiscase,optionAthatdependson optionBisnotvisibleuntiloptionBisenabled selectdependencies.Inthiscase,withoptionAdependingon optionB,whenoptionAisenabled,optionBisautomatically enabled makexconfigallowstoseealloptions,eventhosethatcannotbe selectedbecauseofmissingdependencies.Inthiscase,theyare displayedingray
21
makegconfig
makegconfig NewGTKbased graphicalconfiguration interface.Functionality similartothatofmake xconfig. Justlackingasearch functionality. RequiredDebian packages: libglade2dev
22
makemenuconfig
makemenuconfig Usefulwhennographics areavailable.Pretty convenienttoo! Sameinterfacefoundin othertools:BusyBox, buildroot... RequiredDebian packages: libncursesdev
23
makeoldconfig
makeoldconfig Neededveryoften! Usefultoupgradea.configfilefromanearlierkernel release Issueswarningsforconfigurationparameters thatnolongerexistinthenewkernel. Asksforvaluesfornewparameters Ifyouedita.configfilebyhand,it'sstrongly recommendedtorunmakeoldconfigafterwards!
24
makeallnoconfig
makeallnoconfig Onlysetsstronglyrecommendedsettingstoy. Setsallothersettingston. Veryusefulinembeddedsystemstoselectonlythe minimumrequiredsetoffeaturesanddrivers. Muchmoreconvenientthanunselectinghundredsof featuresonebyone!
25
Undoingconfigurationchanges
Afrequentproblem: Afterchangingseveralkernelconfigurationsettings, yourkernelnolongerworks. Ifyoudon'trememberallthechangesyoumade, youcangetbacktoyourpreviousconfiguration: >cp.config.old.config Alltheconfigurationinterfacesofthekernel (xconfig,menuconfig,allnoconfig...) keepthis.config.oldbackupcopy.
26
Configurationperarchitecture
Thesetofconfigurationoptionsisarchitecturedependent
Someconfigurationoptionsareveryarchitecturespecific Mostoftheconfigurationoptions(globalkerneloptions,network subsystem,filesystems,mostofthedevicedrivers)arevisibleinall architecture
Bydefault,thekernelbuildsystemassumesthatthekernelis beingbuiltforthehostarchitecture,i.enativecompilation Thearchitectureisnotdefinedinsidetheconfiguration,butatan higherlevel Wewillseelaterhowtooverridethisbehaviour,toallowthe configurationofkernelsforadifferentarchitecture
27
Overviewofkerneloptions(1)
Generalsetup
Promptfordevelopment/incompletecodeallowstobeableto enabledriversorfeaturesthatarenotconsideredascompletely stableyet Localversionappendtokernelreleaseallowstoconcatenatean arbitrarystringtothekernelversionthatanusercangetusing unamer.Veryusefulforsupport! Supportforswap,canusuallybedisabledonmostembedded devices Configurestandardkernelfeatures(forsmallsystems)allowsto removefeaturesfromthekerneltoreduceitssize.Powerful,use withcare!
28
Loadablemodulesupport
Allowstoenableorcompletelydisablemodulesupport.Ifyour systemdoesn'tneedkernelmodules,besttodisablesinceitsaves asignificantamountofspaceandmemory
Enabletheblocklayer
IfCONFIG_EMBEDDEDisenabled,theblocklayercanbe completelyremoved.EmbeddedsystemsusingonlyFlashstorage cansafelydisabletheblocklayer
Processortypeandfeatures(x86)orSystemtype(ARM)orCPU selection(MIPS)
AllowstoselecttheCPUormachineforwhichthekernelmustbe compiled Onx86,onlyoptimizationrelated,onotherarchitecturesvery importantsincethere'snocompatibility
29
Kernelfeatures
Ticklesssystem,whichallowstodisabletheregulartimertickand useondemandticksinstead.Improvespowersavings Highresolutiontimersupport.Bydefault,theresolutionoftimeris thetickresolution.Withhighresolutiontimers,theresolutionisas preciseasthehardwarecangive Preemptiblekernelenablesthepreemptioninsidethekernelcode (theuserspacecodeisalwayspreemptible).Seeourrealtime presentationfordetails
Powermanagement
Globalpowermanagementoptionneededforallpower managementrelatedfeatures SuspendtoRAM,CPUfrequencyscaling,CPUidlecontrol, suspendtodisk
30
Networkingsupport
Thenetworkstack Networkingoptions
Unixsockets,neededforaformofinterprocesscommunication TCP/IPprotocolwithoptionsformulticast,routing,tunneling,Ipsec, Ipv6,congestionalgorithms,etc. OtherprotocolssuchasDCCP,SCTP,TIPC,ATM Ethernetbridging,QoS,etc.
Supportforothertypesofnetwork
CANbus,Infrared,Bluetooth,Wirelessstack,WiMaxstack,etc.
31
Devicedrivers
MTDisthesubsystemforFlash(NOR,NAND,OneNand,battery backedmemory,etc.) Parallelportsupport Blockdevices,afewmiscblockdriverssuchasloopback,NBD, etc. ATA/ATAPI,supportforIDEdisk,CDROMandtapes.Anewstack exists SCSI
TheSCSIcore,needednotonlyforSCSIdevicesbutalsoforUSB massstoragedevices,SATAandPATAharddrives,etc. SCSIcontrollerdrivers
32
Devicedrivers(cont)
SATAandPATA,thenewstackforharddisks,reliesonSCSI RAIDandLVM,toaggregateharddriversanddoreplication Networkdevicesupport,withthenetworkcontrollerdrivers. Ethernet,WirelessbutalsoPPP Inputdevicesupport,foralltypesofinputdevices:keyboards, mices,joysticks,touchscreens,tablets,etc. Characterdevices,containsvariousdevicedrivers,amongstthem
serialportcontrollerdrivers PTYdriver,neededforthingslikeSSHortelnet
I2C,SPI,1wire,supportforthepopularembeddedbuses Hardwaremonitoringsupport,infrastructureanddriversforthermal sensors

33
Devicedrivers(cont)
Watchdogsupport Multifunctiondriversaredriversthatdonotfitinanyothercategory becausethedeviceoffersmultiplefunctionalityatthesametime Multimediasupport,containstheV4LandDVBsubsystems,for videocapture,webcams,AM/FMcards,DVBadapters Graphicssupport,infrastructureanddriversforframebuffers Soundcardsupport,theOSSandALSAsoundinfrastructuresand thecorrespondingdrivers HIDdevices,supportforthedevicesthatconformtotheHID specification(HumanInputDevices)
34
Devicedrivers(cont)
USBsupport
Infrastructure Hostcontrollerdrivers Devicedrivers,fordevicesconnectedtotheembeddedsystem Gadgetcontrollerdrivers Gadgetdrivers,tolettheembeddedsystemactasamassstorage device,aserialportoranEthernetadapter
MMC/SD/SDIOsupport LEDsupport RealTimeClockdrivers Voltageandcurrentregulators Stagingdrivers,crappydriversbeingcleanedup

35
Forsomecategoriesofdevicesthedriverisnotimplemented insidethekernel
Printers Scanners GraphicsdriversusedbyX.org SomeUSBdevices
Forthesedevices,thekernelonlyprovidesamechanismto accessthehardware,thedriverisimplementedinuserspace
36
Filesystems
ThecommonLinuxfilesystemsforblockdevices:ext2,ext3,ext4 Lesscommonfilesystems:XFS,JFS,ReiserFS,GFS2,OCFS2, Btrfs CDROMfilesystems:ISO9660,UDF DOS/Windowsfilesystems:FATandNTFS Pseudofilesystems:procandsysfs Miscellanousfilesystems,withamongstotherFlashfilesystems suchasJFFS2,UBIFS,SquashFS,cramfs Networkfilesystems,withmainlyNFSandSMB/CIFS
Kernelhacking
Debuggingfeaturesusefulforkerneldevelopers
37
EmbeddedLinuxusage
Compilingandinstallingthekernel forthehostsystem
38
Kernelcompilation
make
inthemainkernelsourcedirectory Remembertorunmakej4ifyouhavemultipleCPUcoresto speedupthecompilationprocess Noneedtorunasroot!
Generates
vmlinux,therawuncompressedkernelimage,attheELFformat, usefulfordebuggingpurposes,butcannotbebooted arch/<arch>/boot/*Image,thefinal,usuallycompressed, kernelimagethatcanbebooted
bzImageforx86,zImageforARM,vmImage.gzforBlackfin,etc.
Allkernelmodules,spreadoverthekernelsourcetree,as.kofiles.
39
Kernelinstallation
makeinstall
Doestheinstallationforthehostsystembydefault,soneedstobe runasroot
Installs /boot/vmlinuz<version> Compressedkernelimage.Sameastheonein arch/<arch>/boot /boot/System.map<version> Storeskernelsymboladdresses /boot/config<version> Kernelconfigurationforthisversion Typicallyrerunsthebootloaderconfigurationutilitytotakeinto accountthenewkernel.
40
Moduleinstallation
makemodules_install
Doestheinstallationforthehostsystembydefault,soneedstobe runasroot
Installsallmodulesin/lib/modules/<version>/ kernel/ Module.ko(KernelObject)files,inthesamedirectory structureasinthesources. modules.alias Modulealiasesformoduleloadingutilities.Exampleline: aliassoundservice?0snd_mixer_oss modules.dep Moduledependencies modules.symbols Tellswhichmoduleagivensymbolbelongsto.
41
Kernelcleanuptargets
Cleanupgeneratedfiles (toforcerecompilingdrivers): makeclean Removeallgeneratedfiles.Neededwhenswitching fromonearchitecturetoanother Caution:alsoremovesyour.configfile! makemrproper Alsoremoveeditorbackupandpatchrejectfiles: (mainlytogeneratepatches): makedistclean
42
EmbeddedLinuxusage
Linuxdevicefiles
43
Characterdevicefiles
Accessedthroughasequentialflowofindividual characters Characterdevicescanbeidentifiedbytheirctype (lsl):
crwrw1rootuucp4,64Feb232004/dev/ttyS0 crww1jdoetty136,1Feb232004/dev/pts/1 crw1rootroot13,32Feb232004/dev/input/mouse0 crwrwrw1rootroot1,3Feb232004/dev/null
Exampledevices:keyboards,mice,parallelport,IrDA, Bluetoothport,consoles,terminals,sound,video...
44
Blockdevicefiles
Accessedthroughdatablocksofagivensize. Blockscanbeaccessedinanyorder. Blockdevicescanbeidentifiedbytheirbtype(lsl):
brwrw1rootdisk3,1Feb232004hda1 brwrw1jdoefloppy2,0Feb232004fd0 brwrw1rootdisk7,0Feb232004loop0 brwrw1rootdisk1,1Feb232004ram1 brw1rootroot8,1Feb232004sda1
Exampledevices:hardorfloppydisks,ramdisks,loop devices...
45
Devicemajorandminornumbers
Asyoucouldseeinthepreviousexamples, devicefileshave2numbersassociatedtothem: Firstnumber:majornumber Secondnumber:minornumber Majorandminornumbersareusedbythekerneltobinda drivertothedevicefile.Devicefilenamesdon'tmattertothe kernel! Tofindoutwhichdriveradevicefilecorrespondsto, orwhenthedevicenameistoocryptic, seeDocumentation/devices.txt.
46
Devicefilecreation
Devicefilesarenotcreatedwhenadriverisloaded. Theyhavetobecreatedinadvance: sudomknod/dev/<device>[c|b]<major><minor> Examples: sudomknod/dev/ttyS0c464 sudomknod/dev/hda1b31
47
PracticallabConfiguringandcompiling
Configureyourkernel Compileit BootitonavirtualPC Modifyarootfilesystemimageby addingentriestothe/dev/ directory
48
EmbeddedLinuxusage
Overallsystemstartup
49
Traditionalbootingsequence
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash Initializessupportforthedevicewherethekernelimageisfound(localstorage, network,removablemedia) LoadsthekernelimageinRAM Executesthekernelimage(withaspecifiedcommandline)
Kernel
Uncompressesitself Initializesthekernelcoreandstaticallycompileddrivers(neededtoaccesstheroot filesystem) Mountstherootfilesystem(specifiedbytherootkernelparameter) Executesthefirstuserspaceprogram(specifiedbytheinitkernelparameter)
Firstuserspaceprogram
Configuresuserspaceandstartsupsystemservices
50
Kernelcommandlineparameters
TheLinuxkernelcanbegivenparametersatboottime Kernelcommandlineargumentsarepartofthebootloader configurationsettings. TheyarecopiedtoRAMbythebootloader, toalocationwherethekernelexpectsthem. Usefultomodifythebehaviorofthekernel atboottime,withouthavingtorecompileit. Usefultoperformadvancedkernelanddriverinitialization, withouthavingtousecomplexuserspacescripts.
51
Kernelcommandlineexample
HPiPAQh2200PDAbootingexample: root=/dev/ram0\ rw\ init=/linuxrc\ console=ttyS0,115200n8\ console=tty0\ ramdisk_size=8192\ cachepolicy=writethrough Rootfilesystem(firstramdisk) Rootfilesystemmountingmode Firstuserspaceprogram Console(serial) Otherconsole(framebuffer) Miscparameters...
Hundredsofcommandlineparametersdescribedon Documentation/kernelparameters.txt
52
Drawbacks
Assumptionthatalldevicedriversneededtomounttheroot filesystem(storageandfilesystemdrivers)arestaticallycompiled insidethekernel. Assumptioncanbecorrectformostembeddedsystems,where thehardwareisknownandthekernelcanbefinetunedforthe system. Assumptionismostlywrongfordesktopandservers,sincea singlekernelimageshouldsupportawiderangeofdevicesand filesystems
Moreflexibilitywasneeded Moduleshavethisflexibility,buttheyarenotavailablebefore mountingtherootfilesystem Needtohandlecomplexsetups(RAID,NFS,etc.)
53
Solution
Asolutionistoincludeasmalltemporaryrootfilesystemwith modules,inthekernelitself.Thissmallfilesystemiscalledthe initramfs. Thisinitramfsisagzippedcpioarchiveofthisbasicrootfilesystem
Agzippedcpioarchiveisakindofzipfile,withamuchsimplerformat
Theinitramfsscriptswilldetectthehardware,loadthe correspondingkernelmodules,andmounttherealrootfilesystem. Finallytheinitramfsscriptswillruntheinitapplicationintherealroot filesystemandthesystemcanbootasusual. Theinitramfstechniquecompletelyreplacesinitramdisks(initrds). InitrdswereusedinLinux2.4,butarenolongerneeded.
54
Bootingsequencewithinitramfs
ExecutedbythehardwareatafixedlocationinROM/Flash Initializessupportforthedevicewheretheimagesarefound(localstorage,network,removablemedia) LoadsthekernelimageinRAM Executesthekernelimage(withaspecifiedcommandline)
Kernel
Uncompressesitself Initializesthekernelcoreandstaticallycompileddrivers Uncompressesaninitramfscpioarchive(ifexisting,inthekernelimageorcopiedtomemorybythe bootloader)andextractsittothekernelfilecache(nomounting,nofilesystem). Iffoundintheinitramfs,executesthefirstuserspaceprogram:/init
Userspace:/initscript(whatfollowsisjustatypicalscenario)
Runsuserspacecommandstoconfigurethedevice (suchasnetworksetup,mounting/procand/sys...) Mountsanewrootfilesystem.Switchtoit(switch_root) Runs/sbin/init
Userspace:/sbin/init
Runscommandstoconfigurethedevice(ifnotdoneyetintheinitramfs) Startsupsystemservices(daemons,servers)anduserprograms
unchanged
55
Bootloader
Initramfsfeaturesandadvantages
Rootfilesystemdirectlyembeddedinthekernelimage,orcopied toRAMbythebootloader,simplesolution. Justaplaincompressedcpioarchiveextractedinthefilecache. Neitherneedsablocknorafilesystemdriver. Simplertomountcomplexfilesystemsfromflexibleuserspace scriptsratherthanfromrigidkernelcode.Morecomplexity movedouttouserspace! PossibletoaddnonGPLfiles(firmware,proprietarydrivers) inthefilesystem.Thisisnotlinking,justfileaggregation (notconsideredasaderivedworkbytheGPL).
56
Howtopopulateaninitramfs
UsingCONFIG_INITRAMFS_SOURCE inkernelconfiguration(GeneralSetupsection) Eithergiveanexistingcpioarchive (filenameendingwith.cpio) Orgiveadirectorytobearchived. Anyotherregularfilewillbetakenasatextspecificationfile (seenextpage).
seeDocumentation/filesystems/ramfsrootfsinitramfs.txt andDocumentation/earlyuserspace/READMEinkernelsources. Seealsohttp://www.linuxdevices.com/articles/AT4017834659.htmlforanice overviewofinitramfs(byRobLandley).
57
Initramfsspecificationfileexample
dir/dev75500 nod/dev/console64400c51 permissions nod/dev/loop064400b70 dir/bin75510001000 file/bin/busybox/stuff/initramfs/busybox75500 slink/bin/shbusybox77700 dir/proc75500 dir/sys75500 dir/mnt75500 file/init/stuff/initramfs/init.sh75500 Noneedforrootuseraccess! userid groupid major minor
58
Summary
Forembeddedsystems,twointerestingsolutions Noinitramfs:allneededdriversareincludedinsidethekernel, andthefinalrootfilesystemismounteddirectly Everythinginsidetheinitramfs
59
EmbeddedLinuxusage
RootfilesystemoverNFS
60
UsefulnessofrootfsonNFS
Oncenetworkingworks,yourrootfilesystemcouldbeadirectory onyourGNU/Linuxdevelopmenthost,exportedbyNFS(Network FileSystem).Thisisveryconvenientforsystemdevelopment: Makesitveryeasytoupdatefiles(drivermodulesinparticular) ontherootfilesystem,withoutrebooting.Muchfasterthan throughtheserialport. Canhaveabigrootfilesystemevenifyoudon'thavesupport forinternalorexternalstorageyet. Therootfilesystemcanbehuge.Youcanevenbuildnative compilertoolsandbuildallthetoolsyouneedonthetarget itself(bettertocrosscompilethough).
61
NFSbootsetup(1)
Onthehost(NFSserver) InstallanNFSserver(example:Debian,Ubuntu)
sudoaptgetinstallnfskernelserver
Addtheexporteddirectorytoyour/etc/exportsfile:
/home/rootfs192.168.1.111(rw,no_root_squash,no_subtree_check)
clientaddress NFSserveroptions
StartorrestartyourNFSserver(example:Debian,Ubuntu)
sudo/etc/init.d/nfskernelserverrestart
62
NFSbootsetup(2)
Onthetarget(NFSclient) CompileyourkernelwithCONFIG_NFS_FS=y, CONFIG_IP_PNP=y(configureIPatboottime) andCONFIG_ROOT_NFS=y Bootthekernelwiththebelowcommandlineoptions: root=/dev/nfs virtualdevice ip=192.168.1.111 localIPaddress nfsroot=192.168.1.110:/home/nfsroot NFSserverIPaddressDirectoryontheNFSserver
63
EmbeddedLinuxusage
Crosscompilingthekernel
64
Crosscompilingthekernel
WhenyoucompileaLinuxkernelforanotherCPUarchitecture Muchfasterthancompilingnatively,whenthetargetsystemis muchslowerthanyourGNU/Linuxworkstation. MucheasierasdevelopmenttoolsforyourGNU/Linux workstationaremucheasiertofind. Tomakethedifferencewithanativecompiler,crosscompiler executablesareprefixedbythenameofthetargetsystem, architectureandsometimeslibrary.Examples: mipslinuxgcc m68klinuxuclibcgcc armlinuxgnueabigcc
65
Specifyingcrosscompilation
TheCPUarchitectureandcrosscompilerprefixaredefinedthrough theARCHandCROSS_COMPILEvariablesinthetoplevelMakefile. TheMakefiledefinesCC=$(CROSS_COMPILE)gcc SeecommentsinMakefilefordetails TheeasiestsolutionistomodifytheMakefile. Example,ARMplatform,crosscompiler:armlinuxgcc ARCH ?=arm CROSS_COMPILE ?=armlinux Othersolutions
PassARCHandCROSS_COMPILEonthemakecommandline DefineARCHandCROSS_COMPILEasenvironmentvariables Don'tforgettohavethevaluesproperlysetatallsteps,otherwisethe kernelconfigurationandbuildsystemgetsconfused
66
Configuringthekernel
makexconfig Sameasinnativecompiling Thesetofavailableoptionswillbedifferent Don'tforgettosettherightboard/machinetype!
67
Readymadeconfigfiles
assabet_defconfigintegrator_defconfigmainstone_defconfig badge4_defconfigiq31244_defconfigmx1ads_defconfig bast_defconfigiq80321_defconfigneponset_defconfig cerfcube_defconfigiq80331_defconfignetwinder_defconfig clps7500_defconfigiq80332_defconfigomap_h2_1610_defconfig ebsa110_defconfigixdp2400_defconfigomnimeter_defconfig edb7211_defconfigixdp2401_defconfigpleb_defconfig enp2611_defconfigixdp2800_defconfigpxa255idp_defconfig ep80219_defconfigixdp2801_defconfigrpc_defconfig epxa10db_defconfigixp4xx_defconfigs3c2410_defconfig footbridge_defconfigjornada720_defconfigshannon_defconfig fortunet_defconfiglart_defconfigshark_defconfig h3600_defconfiglpd7a400_defconfigsimpad_defconfig h7201_defconfiglpd7a404_defconfigsmdk2410_defconfig h7202_defconfiglubbock_defconfigversatile_defconfig hackkit_defconfiglusl7200_defconfig
arch/arm/configsexample
68
Usingreadymadeconfigfiles
Defaultconfigurationfilesavailableformanyboards/machines! Checkifoneexistsinarch/<arch>/configs/foryourtarget. Example:ifyoufoundanacme_defconfigfile,youcanrun: makeacme_defconfig Usingarch/<arch>/configs/isaverygoodgoodwayof releasingadefaultconfigurationfileforagroupofusersor developers. Likeallmakecommands,youmust runmake<machine>_defconfig inthetoplevelsourcedirectory.
69
Buildingthekernel
Run make Copy arch/<arch>/boot/zImage tothetargetstorage Youcancustomizearch/<arch>/boot/install.shso thatmakeinstalldoesthisautomaticallyforyou. makeINSTALL_MOD_PATH=<dir>/modules_install andcopy<dir>/lib/modules/to/lib/modules/on thetargetstorage.
70
PracticallabCrosscompiling
Setupacrosscompiling environment ConfigurethekernelMakefile accordingly Crosscompilethekernelforanarm targetplatform Onthisplatform,interactwiththe bootloaderandbootyourkernel.
71
EmbeddedLinuxkernelusage
Usingkernelmodules
72
Loadablekernelmodules
Modules:addagivenfunctionalitytothekernel(drivers, filesystemsupport,andmanyothers). Canbeloadedandunloadedatanytime,onlywhentheir functionalityisneed. Modulesmakeiteasytodevelopdriverswithoutrebooting: load,test,unload,rebuild,load... Usefultokeepthekernelimagesizetotheminimum(essential inGNU/LinuxdistributionsforPCs). Alsousefultoreduceboottime:youdon'tspendtimeinitializing devicesandkernelfeaturesthatyouonlyneedlater. Caution:onceloaded,havefullaccesstothewholekernel addressspace.Noparticularprotection.
73
Moduledependencies
Somekernelmodulescandependonothermodules, whichneedtobeloadedfirst. Example:theusbstoragemoduledependsonthescsi_mod, libusualandusbcoremodules. Dependenciesaredescribed in/lib/modules/<kernelversion>/modules.dep Thisfileisgeneratedwhenyourunmakemodules_install. Youcanalsoupdatethemodules.depfile byyourself,byrunning(asroot): depmoda[<version>]
74
Kernellog
Whenanewmoduleisloaded, relatedinformationisavailableinthekernellog. Thekernelkeepsitsmessagesinacircularbuffer (sothatitdoesn'tconsumemorememorywithmanymessages) Kernellogmessagesareavailablethroughthedmesgcommand. (diagnosticmessage) Kernellogmessagesarealsodisplayedinthesystemconsole (messagescanbefilteredbylevelusing /proc/sys/kernel/printk) Notethatyoucanwritetothekernellogfromuserspacetoo: echoDebuginfo>/dev/kmsg
75
Moduleutilities(1)
modinfo<module_name> modinfo<module_path>.ko Getsinformationaboutamodule:parameters,license, descriptionanddependencies. Veryusefulbeforedecidingtoloadamoduleornot. sudoinsmod<module_path>.ko Triestoloadthegivenmodule.Thefullpathtothemodule objectfilemustbegiven.
76
Understandingmoduleloadingissues
Whenloadingamodulefails, insmodoftendoesn'tgiveyouenoughdetails! Detailsareoftenavailableinthekernellog. Example: >sudoinsmod./intr_monitor.ko insmod:errorinserting'./intr_monitor.ko':1 Deviceorresourcebusy >dmesg [17549774.552000]Failedtoregisterhandlerfor irqchannel2
77
Moduleutilities(2)
sudomodprobe<module_name> Mostcommonusageofmodprobe:triestoloadallthe modulesthegivenmoduledependson,andthenthis module.Lotsofotheroptionsareavailable.Modprobe automaticallylooksin/lib/modules/<version>/forthe objectfilecorrespondingtothegivenmodulename. lsmod Displaysthelistofloadedmodules Compareitsoutputwiththecontentsof /proc/modules!
78
Moduleutilities(3)
sudormmod<module_name> Triestoremovethegivenmodule. Willonlybeallowedifthemoduleisnolongerinuse (forexample,nomoreprocessesopeningadevicefile) sudomodprober<module_name> Triestoremovethegivenmoduleandalldependent modules(whicharenolongerneededafter themoduleremoval)
79
Passingparameterstomodules
Findavailableparameters: modinfosndintel8x0m Throughinsmod: sudoinsmod./sndintel8x0m.koindex=2 Throughmodprobe: Setparametersin/etc/modprobe.conforinanyfilein /etc/modprobe.d/: optionssndintel8x0mindex=2 Throughthekernelcommandline, whenthedriverisbuiltstaticallyintothekernel: sndintel8x0m.index=2 drivername driverparametername driverparametervalue
80
Usefulreading
LinuxKernelinaNutshell,Dec2006 ByGregKroahHartman,O'Reilly http://www.kroah.com/lkn/ Agoodreferencebookandguideonconfiguring, compilingandmanagingtheLinuxkernelsources. Freelyavailableonline! Greatcompaniontotheprintedbook foreasyelectronicsearches! AvailableassinglePDFfileon http://freeelectrons.com/community/kernel/lkn/
81
BusyBox
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/busybox Corrections,suggestions,contributionsandtranslationsarewelcome!
Generalpurposetoolbox:BusyBox
http://www.busybox.net/ MostUnixcommandlineutilitieswithinasingleexecutable! Itevenincludesawebserver! Sizeslessthan<500KB(staticallycompiledwithuClibc)orlessthan 1MB(staticallycompiledwithglibc). Easytoconfigurewhichfeaturestoinclude. Thebestchoicefor Initramfs/initrdwithcomplexscripts Smallandmediumsizeembeddedsystems Seehttp://www128.ibm.com/developerworks/linux/library/lbusybox/ foraniceintroduction.
2
BusyBoxcommands!
[,[[,addgroup,adduser,adjtimex,ar,arp,arping,ash,awk,basename,bbconfig,bbsh, brctl,bunzip2,busybox,bzcat,bzip2,cal,cat,catv,chat,chattr,chcon,chgrp,chmod, chown,chpasswd,chpst,chroot,chrt,chvt,cksum,clear,cmp,comm,cp,cpio,crond,crontab, cryptpw,cttyhack,cut,date,dc,dd,deallocvt,delgroup,deluser,depmod,devfsd,df, dhcprelay,diff,dirname,dmesg,dnsd,dos2unix,dpkg,dpkg_deb,du,dumpkmap,dumpleases, e2fsck,echo,ed,egrep,eject,env,envdir,envuidgid,ether_wake,expand,expr,fakeidentd, false,fbset,fbsplash,fdflush,fdformat,fdisk,fetchmail,fgrep,find,findfs,fold,free, freeramdisk,fsck,fsck_minix,ftpget,ftpput,fuser,getenforce,getopt,getsebool,getty, grep,gunzip,gzip,halt,hd,hdparm,head,hexdump,hostid,hostname,httpd,hush,hwclock, id,ifconfig,ifdown,ifenslave,ifup,inetd,init,inotifyd,insmod,install,ip,ipaddr, ipcalc,ipcrm,ipcs,iplink,iproute,iprule,iptunnel,kbd_mode,kill,killall,killall5, klogd,lash,last,length,less,linux32,linux64,linuxrc,ln,load_policy,loadfont, loadkmap,logger,login,logname,logread,losetup,lpd,lpq,lpr,ls,lsattr,lsmod,lzmacat, makedevs,man,matchpathcon,md5sum,mdev,mesg,microcom,mkdir,mke2fs,mkfifo,mkfs_minix, mknod,mkswap,mktemp,modprobe,more,mount,mountpoint,msh,mt,mv,nameif,nc,netstat, nice,nmeter,nohup,nslookup,od,openvt,parse,passwd,patch,pgrep,pidof,ping,ping6, pipe_progress,pivot_root,pkill,poweroff,printenv,printf,ps,pscan,pwd,raidautorun, rdate,rdev,readahead,readlink,readprofile,realpath,reboot,renice,reset,resize, restorecon,rm,rmdir,rmmod,route,rpm,rpm2cpio,rtcwake,run_parts,runcon,runlevel, runsv,runsvdir,rx,script,sed,selinuxenabled,sendmail,seq,sestatus,setarch, setconsole,setenforce,setfiles,setfont,setkeycodes,setlogcons,setsebool,setsid, setuidgid,sh,sha1sum,showkey,slattach,sleep,softlimit,sort,split,start_stop_daemon, stat,strings,stty,su,sulogin,sum,sv,svlogd,swapoff,swapon,switch_root,sync,sysctl, syslogd,tac,tail,tar,taskset,tcpsvd,tee,telnet,telnetd,test,tftp,tftpd,time,top, touch,tr,traceroute,true,tty,ttysize,tune2fs,udhcpc,udhcpd,udpsvd,umount,uname, uncompress,unexpand,uniq,unix2dos,unlzma,unzip,uptime,usleep,uudecode,uuencode, vconfig,vi,vlock,watch,watchdog,wc,wget,which,who,whoami,xargs,yes,zcat,zcip
CommandsavailableinBusyBox1.13
ApplethighlightBusyBoxvi
IfyouareusingBusyBox,addingvisupportsonlyadds20K. (builtwithsharedlibraries,usinguClibc). Youcanselectwhichexactfeaturestocompilein. Usershardlyrealizethattheyareusingalightweightvi version! Tip:youcanlearnvionthedesktop, byrunningthevimtutorcommand.
4
ConfiguringBusyBox
Getthelateststablesourcesfromhttp://busybox.net ConfigureBusyBox(createsa.configfile): makedefconfig GoodtobeginwithBusyBox. ConfiguresBusyBoxwithalloptionsforregularusers. makeallnoconfig Unselectsalloptions.Goodtoconfigureonlywhatyouneed. makexconfig(graphical)ormakemenuconfig(text) SameconfigurationinterfacesastheonesusedbytheLinux kernel.
5
BusyBoxmakexconfig
Youcanchoose: thecommands tocompile, andeventhe command optionsand featuresthatyou need!
6
CompilingBusyBox
Setthecrosscompilerprefixintheconfigurationinterface: BusyBoxSettings>BuildOptions>CrossCompiler prefix Example:armlinux Settheinstallationdirectoryintheconfigurationinterface: BusyBoxSettings>InstallationOptions>BusyBox installationprefix AddthecrosscompilerpathtothePATHenvironmentvariable: exportPATH=/usr/local/arm/3.3.2/bin:$PATH CompileBusyBox: make Installit(thiscreatesaUnixdirectorystructuresymboliclinkstothebusybox executable): makeinstall
7
AlternativetoBusyBox:embutils
http://www.fefe.de/embutils/ Fromthecreatorofdietlibc Asimilarsetoftinyutilitiesforembeddedsystems. Version0.19(Aug.2008):90commoncommandsare implemented. Canonlybebuiltstaticallywithdietlibc! ComparedtoBusyBox:Muchlessmomentum, useranddeveloperbase.Stillmisseskeycommandsandfeatures (ifconfig,forexample) ButcanachievesmallersizethanBusyBox onstandaloneexecutables.
8
PracticallabAtinyembeddedsystem
MakeLinuxbootonadirectoryon yourworkstation,sharedbyNFS Createandconfigurea minimalisticLinuxembedded system InstallanduseBusyBox Systemstartupwith/sbin/init Setupasimplewebinterface Usesharedlibraries
9
Blockfilesystems
Blockfilesystems
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/blockfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!
Blockvs.Flash
Storagedevicesareclassifiedintwomaintypes:blockdevices andflashes
Theyarehandledbydifferentsubsystemsanddifferentfilesystems
Blockdevicescanbereadandwrittenonaperblockbasis, withouterasing,anddonotwearoutwhenbeingusedforalong time

Harddisks,floppydisks,RAMdisks USBkeys,CompactFlash,SDcard,thesearebasedonflashes, buthaveanintegratedcontrollerthatemulatesablockdevice
Flashes,canberead,butwritingrequireserasing,andoften occursonalargersizethantheblocksize
NORflashes,NANDflashes
2
Traditionalblockfilesystems
Traditionalfilesystems Canbeleftinanoncoherentstateafterasystemcrash orsuddenpoweroff,whichrequiresafullfilesystem checkafterreboot. ext2:traditionalLinuxfilesystem (repairitwithfsck.ext2) vfat:traditionalWindowsfilesystem (repairitwithfsck.vfatonGNU/LinuxorScandisk onWindows)
3
Journaledfilesystems
Designedtostayina correctstateeven aftersystemcrashes orasuddenpoweroff Allwritesarefirst describedinthe journalbeforebeing committedtofiles
Application
Userspace Kernelspace (filesystem) Writetofile
Writeanentry inthejournal Write tofile Clear journalentry
4
Filesystemrecoveryaftercrashes
Reboot Reboot
No Discard Discard incomplete incomplete journalentries journalentries
Journal Journal empty? empty?
Thankstothejournal, thefilesystemisnever leftinacorrupted state Recentlysaveddata couldstillbelost
Yes Execute Execute journal journal FilesystemOK FilesystemOK
5
Journaledblockfilesystems
Journaledfilesystems ext3:ext2withjournalextension ext4:thenewgenerationwithmanyimprovements. Readyforproduction.Theyarethedefaultfilesystemsforall Linuxsystemsintheworld. TheLinuxkernelsupportsmanyotherfilesystems:reiserFS, JFS,XFS,etc.Eachofthemhavetheirowncharacteristics,but aremoreorientedtowardsserverorscientificworkloads btrfs(ButterFS) Thenextgeneration.Inmainlinebutstillexperimental.
6
Creatingext2/ext3volumes
Tocreateanemptyext2/ext3filesystemonablockdeviceor insideanalreadyexistingimagefile
mkfs.ext2/dev/hda3 mkfs.ext3/dev/sda2 mkfs.ext2disk.img
Tocreateafilesystemimagefromadirectorycontainingallyour filesanddirections
Usethegenext2fstool,fromthepackageofthesamename genext2fsdrootfs/rootfs.img Yourimageisthenreadytobetransferredtoyourblockdevice
7
Cramfs
Simple,small,readonlycompressedfilesystem designedforembeddedsystems Limitations Maximumfilesystemsize:256MB Maximumfilesize:16MB Associatedtocramfsprogs,whichcontainsmkcramfsto createanimageofacramfsfilesystem mkcramfsrootfs/rootfs.img SeeDocumentation/filesystems/cramfs.txt inkernelsources.
8
Squashfs
Squashfs:http://squashfs.sourceforge.net AmustusereplacementforCramfs!Alsoreadonly. Maximumfilesystemandfilesize:264bytes! Achievesbettercompressionandmuchbetterperformance. Itsupportsblocksizesupto64K(insteadof4K)forgreater compression,andevendetectsduplicatefiles! AvailableinmainstreamLinuxsinceversion2.6.29. Patchesavailableforallearlierversions. Benchmarks:(roughly3timessmallerthanext3,and24timesfaster) http://elinux.org/Squash_Fs_Comparisons
9
SquashfsHowtouse
Needtoinstallthesquashfstoolspackage Creationoftheimage Onyourworkstation,createyourfilesystemimage: mksquashfsrootfs/rootfs.sqfs
Caution:iftheimagealreadyexistsremoveitfirst, orusethenoappendoption.
Installationoftheimage Let'sassumeyourpartitiononthetargetisin/dev/sdc1 Onyourtarget,copythefilesystemimageonthedevice (CAUTION:don'trunthisonyourworkstation! Youcoulddestroycriticalsystempartitions.) ddif=rootfs.sqfsof=/dev/sdc1 Mountyourfilesystem: mounttsquashfs/dev/sdc1/mnt/root
10
tmpfs
UsefultostoretemporarydatainRAM: systemlogfiles,connectiondata,temporaryfiles... Don'tuseramdisks!Theyhavemanydrawbacks:fixedinsize, RemainingspacenotusableasRAM,filesduplicatedinRAM(inthe blockdeviceandfilecache)! tmpfsconfiguration:Filesystems>Pseudofilesystems LivesintheLinuxfilecache.Doesn'twasteRAM:growsandshrinksto accommodatestoredfiles.SavesRAM:noduplication;canswapout pagestodiskwhenneeded. Howtouse:chooseanametodistinguishthevarioustmpfsinstances youcouldhave.Examples: mountttmpfsvarrun/var/run mountttmpfsudev/dev SeeDocumentation/filesystems/tmpfs.txtinkernelsources.
11
Mixingreadonlyandreadwritefilesystems
Goodideatosplityourblockstorageinto
Areadwritepartitionwithajournaledfilesystem(likeext3) Usedtostoreuserorconfigurationdata. Guaranteesfilesystemintegrityafterpowerofforcrashes. Ramstoragefortemporaryfiles(tmpfs)
readwrite userand configuration data
ext3
readwrite volatiledata
RAM
12
tmpfs
BlockStorage
readonly Acompressedreadonlypartition(Squashfs) compressed Typicallyusedfortherootfilesystem(binaries,kernel...). root Compressionsavesspace.Readonlyaccessprotectsyour filesystem systemfrommistakesanddatacorruption.
Squashfs
PracticallabBlockfilesystems
Creatingpartitionsonyourblock storage Bootingyoursystemwithamixof filesystems:SquashFSfor applications,ext3forconfiguration anduserdata,andtmpfsfor temporarysystemfiles.
13
Flashfilesystems
Flash filesystems
MichaelOpdenacker ThomasPetazzoni GregoryClement FreeElectrons
Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/flashfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!
TheMTDsubsystem
MTD:MemoryTechnologyDevices(flash,ROM,RAM)
Linuxfilesysteminterface MTDUsermodules
UBI yaffs2 jffs2 Chardevice Blockdevice
FlashTranslationLayers forblockdeviceemulation Caution:patentedalgorithms!
Readonlyblockdevice
FTL
NFTL
INFTL
MTDChipdrivers
NORflash NANDflash DiskOnChipflash
RAMchips Blockdevice ROMchips Virtualmemory Virtualdevicesappearing asMTDdevices
Memorydevices hardware
MTDdevices
MTDdevicesarevisiblein/proc/mtd ThemtdchardrivercreatesacharacterdeviceforeachMTD deviceofthesystem
Usuallynamed/dev/mtdX,major90.Evenminorsforreadwrite access,oddminorsforreadonlyaccess Provideioctl()toeraseandmanagetheflash Usedbythemtdutils
ThemtdblockdrivercreatesablockdeviceforeachMTDdevice ofthesystem
Usuallynamed/dev/mtdblockX,major31.Minoristhenumber oftheMTDdevice Allowsread/writeblocklevelaccess.Butbadblocksarenot handled,andnowearlevelingisdoneforwrites.
3
MTDpartitioning
MTDdevicesareusuallypartitioned
Itallowstousedifferentareasoftheflashfordifferentpurposes: readonlyfilesystem,readwritefilesystem,backupareas, bootloaderarea,kernelarea,etc.
Unlikeblockdevices,whichcontainstheirownpartitiontable,the partitioningofMTDdevicesisdescribedexternally
Hardcodedintothekernelcode Specifiedthroughthekernelcommandline
EachpartitionbecomesaseparateMTDdevice
Differentfromblockdevicelabeling(hda3,sda2) /dev/mtd1iseitherthesecondpartitionofthefirstflashdevice,or thefirstpartitionofthesecondflashdevice
4
DefinitionofMTDpartitions
MTDpartitionsaredefinedinthekernel,intheboarddefinitions: arch/arm/machat91/boardusba9263.cexample:
staticstructmtd_partition__initdataek_nand_partition[]={ { .name="LinuxKernel", .offset=0, .size=SZ_16M, }, { .name="RootFS", .offset=MTDPART_OFS_NXTBLK, .size=120*SZ_1M, }, { .name="FS", .offset=MTDPART_OFS_NXTBLK, .size=120*SZ_1M, } }; 5
ModifyingMTDpartitions(1)
MTDpartitionscanfortunatelybedefined throughthekernelcommandline. FirstneedtofindthenameoftheMTDdevice. Lookatthekernellogatboottime: NANDdevice:ManufacturerID:0xec,ChipID: 0xda(SamsungNAND256MiB3,3V8bit) Scanningdeviceforbadblocks Baderaseblock2000at0x0fa00000 Creating3MTDpartitionson"atmel_nand": 0x000000000x01000000:"LinuxKernel" 0x010000000x08800000:"RootFS" 0x088000000x10000000:"FS"
6
ModifyingMTDpartitions(2)
Youcannowusethemtdpartskernelbootparameter Example:
mtdparts=atmel_nand:2m(kernel)ro,1m(rootfs)ro,(data)
We'vejustdefined3partitionsintheatmel_nanddevice:
kernel(2M) rootfs(1M) data
Partitionsizesmustbemultipleoftheeraseblocksize. Youcanusesizesinhexadecimaltoo.Rememberthebelowsizes: 0x20000=128k,0x100000=1m,0x1000000=16m roliststhepartitionasreadonly isusedtousealltheremainingspace.

7
mtdutils
AsetofutilitiestomanipulateMTDdevices
mtdinfotogetdetailedinformationaboutaMTDdevice flash_erasealltocompletelyeraseagivenMTDdevice flashcptowritetoNORflash nandwritetowritetoNANDflash UBIutilities Flashfilesystemimagecreationtools:mkfs.jffs2, mkfs.ubifs
Usuallyavailableasthemtdutilspackageinyourdistribution Seehttp://www.linuxmtd.infradead.org/
8
jffs2
http://www.linuxmtd.infradead.org/doc/jffs2.html Today'sstandardfilesystemforMTDflash Nicefeatures:ontheflycompression(savesstorage spaceandreducesI/O),powerdownreliable,wear levelingandECC. Drawbacks:doesn'tscalewell Mounttimedependingonfilesystemsize: thekernelhastoscanthewholefilesystemat mounttime,toreadwhichblockbelongstoeach file. NeedtousetheCONFIG_JFFS2_SUMMARYkernel optiontostoresuchinformationinflash.This dramaticallyreducesmounttime(from16sto0.8s fora128MBpartition). MTDdriver Standardfile API JFFS2 filesystem
Flashchip
9
jffs2Howtouse
OntheLinuxtarget NeedeitherthemtdutilsfromtheMTDproject,ortheir embeddedvariantsfromBusybox Eraseandformatapartitionwithjffs2: flash_eraseallj/dev/mtd2 Mountthepartition: mounttjffs2/dev/mtdblock2/mnt/flash Fillthecontentsbywriting (copyingfromNFSorfromexternalstorage) Otherpossibility:useajffs2image(seenextpagetoproduceit): flash_eraseall/dev/mtd2 nandwritep/dev/mtd2rootfs.jffs2
10
Howtocreateajffs2image
mkfs.jffs2commandavailableinthemtdutilspackage. Caution:unlikesomemkfscommands,itdoesn'tcreatea filesystem,butafilesystemimage. First,findtheeraseblocksizefromUbootnandinfo:
Device0:NAND256MiB3,3V8bit,sectorsize128KiB
Thencreatetheimageonyourworkstation: mkfs.jffs2padnocleanmarkers eraseblock=128drootfs/orootfs.jffs2 Thepadoptionpadsthejffs2imagecontents untiltheendofthefinaleraseblock. Itisfineifthejffs2imageissmallerthantheMTDpartition. Thejffs2filesystemwillusetheentirepartitionanyway. ThenocleanmarkersoptionisforNANDflashonly.

11
Mountingajffs2imageonyourhost
Usefultoeditjffs2imagesonyourdevelopmentsystem MountinganMTDdeviceasaloopdeviceisabitcomplextask. Here'sanexampleforjffs2,foryourreference: Firstfindtheeraseblocksizeusedtocreatethejffs2image. Let'sassumeitis128KiB(131072bytes). Createablockdevicefromtheimage losetup/dev/loop0root.jffs2 EmulateanMTDdevicefromablockdevice, usingtheblock2mtdkernelmodule modprobeblock2mtdblock2mtd=/dev/loop0,131072 Finally,mountthefilesystem(create/mnt/jffs2ifneeded) mounttjffs2/dev/mtdblock0/mnt/jffs2
12
Initializingjffs2partitionsfromUboot
Youmaynotwanttohavemtdutilsonyourtarget! CreateaJFFS2imageonyourworkstation IntheUBootprompt: Downloadthejffs2imagetoRAMwithtftp OrcopythisimagetoRAMfromexternalstorage (UbootunderstandsFATfilesystemsandsupportsUSBstorage) FlashitinsideanMTDpartition (exactinstructionsdependingonflashtype,NORorNAND, reusetheinstructionsusedtoflashyourkernel).Makesuretowriteonly thesizeoftheimage,notmore! Ifyoubootonajffs2rootfilesystem,addroot=/dev/mtdblock<x>and rootfstype=jffs2totheLinuxcommandlinearguments. Limitation:needtosplitthejffs2imageinseveralchunks ifbiggerthantheRAMsize.
13
yaffs2
http://www.yaffs.net/ MainlysupportsNANDflash Nocompression Wearleveling,ECC,powerfailureresistant Fastboottime Codeavailableseparatelythroughgit (DualGPL/Proprietarylicense fornonLinuxoperatingsystems)
MTDdriver Standardfile API YAFFS2 filesystem
Flashchip
14
yaffs2Howtouse
Eraseapartition: flash_eraseall/dev/mtd2 Thefilesystemisautomaticallyformattedatthefirstmount: mounttyaffs2/dev/mtdblock2/mnt/flash Imagescanbecreatedwiththemkyaffstool,fromyaffsutils http://www.aleph1.co.uk/cgibin/viewvc.cgi/yaffs/utils/
15
UBI(1)
UnsortedBlockImages http://www.linuxmtd.infradead.org/doc/ubi.html VolumemanagementsystemontopofMTDdevices. Allowstocreatemultiplelogicalvolumes andspreadwritesacrossallphysicalblocks. Takescareofmanagingtheeraseblocksandwear leveling.Makesfilesystemeasiertoimplement.
16
UBI(2)
UBI Logical EraseBlocks MTD Physical EraseBlocks
Volume1
Volume2
LEB
LEB
LEB
LEB
LEB
LEB
LEB
PEB PEB PEB PEB PEB PEB PEB PEB PEB

Freeblock Freeblock
17
UBIFS
http://www.linuxmtd.infradead.org/doc/ubifs.html Thenextgenerationofthejffs2filesystem, fromthesamelinuxmtddevelopers. AvailableinLinux2.6.27 WorksontopofUBIvolumes Hasanoticeablemetadataoverheadonvery smallpartitions(4M,8M)
Standardfile API UBIFS UBI MTDdriver
Flashchip
18
UBIlayout
UBIFSmountedon/ (fromubi0:rootfs)
UBIFS mountedon /myapp
JFFS2 mounted on
/appsettings
UBIFSmountedon /var/log (fromubi1:log)
UBI Volume: raw
(from
ubi0:data)
(from
/dev/mtd1)
UBIVolume:rootfs
UBIVolume: data
UBIVolume:log
UBIdeviceubi0
MTDpartition0
MTD partition1
UBIdeviceubi1
MTDpartition2
Flashdevice
19
UBIPreparation
Eraseyourflashpartitionwhilepreservingyourerasecounters ubiformat/dev/mtd1 Seehttp://www.linuxmtd.infradead.org/faq/ubi.htmlifyoufaceproblems Needtocreatea/dev/ubi_ctrlchardevice(ifyoudon'thaveudev) ThisspecialcharacterdeviceisusedbyotherUBIutilities Majorandminornumberallocatedinthekernel.Findthesenumbers in/sys/class/misc/ubi_ctrl/dev(e.g.:10:63) Orrunubinfo: UBIversion:1 CountofUBIdevices:1 UBIcontroldevicemajor/minor:10:63 PresentUBIdevices:ubi0
Thesestepsaredoneonceforall
20
UBIAttaching
AttachUBItoone(ofseveral)oftheMTDpartitions: ubiattach/dev/ubi_ctrlm1 Thiscommandcreatestheubi0device,whichrepresentthefull UBIspacestoredonMTDdevice1
FindthemajorandminornumbersusedbyUBI: cat/sys/class/ubi/ubi0/dev(e.g.253:0) CreatetheUBIdevicefile: mknod/dev/ubi0c2530 ThisUBIspacecancontainseveralvolumes
21
UBIVolumemanagement
Volumecreationwithubimkvol
ubimkvol/dev/ubi0Ntests116MiB ubimkvol/dev/ubi0Ntestm(maxavailablesize) Thevolumeisthenidentifiedasubi0:testforthemount/umount commands
Volumeremovalwithubirmvol
ubirmvol/dev/ubi0Ntest
22
UBIFSHowtouse
WhenaUBIvolumeiscreated,creatinganemptyUBIFS filesystemisjustamatterofmountingit
mounttubifsubi0:test/mnt/flash
ImagesofUBIFSfilesystemscanbecreatedusingthe mkfs.ubifsutility
mkfs.ubifsm512e128KiBc100r/opt/img ubifs.img CanbewrittentoaUBIvolumeusingubiupdatevolandthe /dev/ubiX_Ydevices
ImagesofafullUBIspace,containingseveralvolumescanbe createdusingtheubinizeutility
CanbewrittentoarawMTDusingnandwrite
23
UBIFSHowtopreparearootfs
CreatetheUBIFSimagefromthetargetdirectory WritetheconfigurationfilefortheUBIdevice
[RFSvolume] mode=ubi image=rootfs.ubifs vol_id=1 vol_size=30MiB vol_type=dynamic vol_name=rootfs vol_flags=autoresize vol_alignment=1
CreatetheUBIdeviceimage Flashitusingabadblockawarecommandfromthebootloader PassUBIlayoutinformationtothekernel:

rootfstype=ubifsubi.mtd=1root=ubi0:rootfs
24
LogFS
Newcomerinmainline(integratedin2.6.34) Still(atleastin2.6.36) Designedtobeveryfastatmounttime (evenfasterthanUBIFS):O(1)mounttime SupposedtoconsumelessRAMthanJFFS2 Abilitytorunonblockdevicesbutwithpoorperformance.
25
SquashFS
http://squashfs.sourceforge.net/ Filesystemforblockstorage,soitdoesn'tsupporttheMTDAPI. However,asitisreadonly,itworksfinewithmtdblock, aslongastheflashchipdoesn'thaveanybadblocks. Youcoulduseitforreadonlysectionsinyourfilesystem, butyoucannotrelyonit(badblockscanalwayshappen).
26
SquashFSHowtouse
Verysimple! Onyourworkstation,createyourfilesystemimage: mksquashfsrootdirrootdir.sqfs Caution:iftheimagealreadyexistsremoveitfirst, orusethenoappendoption. Eraseyourflashpartition: flash_eraseall/dev/mtd2 Makeyourfilesystemimageavailabletoyourdevice (NFS,copy,etc.)andflashyourpartition: ddif=rootdir.sqfsof=/dev/mtdblock2 Mountyourfilesystem: mounttsquashfs/dev/mtdblock2/mnt/flash
27
Ourbenchmarks
jffs2 Dramaticallyoutperformedbyubifsin mostaspects. Hugemount/boottimeunless CONFIG_SUMMARYisused. yaffs2 Alsooutperformedbyubifs. Maynotfitallyourdata Uglyfileremovaltime (poordirectoryupdate performance?) Memoryusagenotscaling ubifsleavesnoreason tosticktoyaffs2.
28
ubifs Greatperformanceinallcorner cases. SquashFS Bestornearbestperformance inallreadonlyscenarios.
Fullbenchmarkdetailson http://freeelectrons.com/pub/conferences/2008/elce/flashfilesystems.pdf
Conclusions
Convertyourjffs2partitionstoubifs! Itmayonlymakesensetokeepjffs2 forMTDpartitionssmallerthan10MB, incasesizeiscritical. Noreasonlefttouseyaffs2insteadofjffs2? YoumayalsouseSquashFStosqueezemore stuffonyourflashstorage.Advisabletouseiton topofUBI,toletallflashsectorsparticipateto wearleveling. SquashFS MTDblock MTDAPI UBI MTDdriver Flashchip
29
Issueswithflashbasedblockstorage
Flashstoragemadeavailableonlythroughablockinterface. Hence,nowaytoaccessalowlevelflashinterface andusetheLinuxfilesystemsdoingwearleveling. Nodetailsaboutthelayer(FlashTranslationLayer)theyuse. Detailsarekeptastradesecrets,andmayhidepoor implementations. Hence,itishighlyrecommendedtolimitthenumberofwritesto thesedevices.
30
Reducingthenumberofwrites
Ofcourse,donotuseyourflashstorageasswaparea (rareinembeddedsystemsanyway) Mountyourfilesystemsasreadonly,orusereadonlyfilesystems (SquashFS),wheneverpossible. KeepvolatilefilesinRAM(tmpfs) Usethenoatimemountoption,toavoidupdatingthefilesystem everytimeyouaccessafile.Oratleast,ifyouneedtoknowwhether fileswerereadaftertheirlastchange,usetherelatimeoption (defaultsettingsinceLinux2.6.30). Don'tusethesyncmountoption(commitswritesimmediately).Use thefsync()systemcallforperfilesynchronization. Youmaydecidetodowithoutjournaledfilesystems.Theycausemore writes,butarealsomuchmorepowerdownresistant(tradeoff).
31
Usefulreading
IntroductiontoJFFS2andLogFS: http://lwn.net/Articles/234441/ NiceUBIpresentationfromToshiba: http://freeelectrons.com/redirect/celfubi.html Documentationonthelinuxmtdwebsite: http://www.linuxmtd.infradead.org/
32
Filesystemchoicesummary
Volatile Volatile data? data?
No
Readonly Readonly files? files?
No Storagetype Storagetype
Block
Contains Contains flash? flash?
Yes MTD Yes choosesquashfs choosesquashfs
Yes
No
noatimeoption noatimeoption
chooseext2 chooseext2
Choosetmpfs Choosetmpfs
chooseUBIFS chooseUBIFS orJFFS2 orJFFS2
Chooseext3orext4 Chooseext3orext4
SeeDocumentation/filesystems/inkernelsourcesfordetails aboutallavailablefilesystems.
33
PracticallabFlashfilesysetms
Creatingpartitionsinyour internalflashstorage. Formatingthemainpartitionwith SquashFSonmtdblock. Usingjffs2forsystemdata.
34
EmbeddedLinux system development

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/sysdev Corrections,suggestions,contributionsandtranslationsarewelcome!
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http://freeelectrons.com
Contents
Usingopensourcecomponents Toolsforthetargetdevice Networking Systemutilities Languageinterpreters Audio,videoandmultimedia Graphicaltoolkits Databases Webbrowsers Systembuilding Emulators Commercialtoolsetsanddistributions
2
Leveragingopensourcecomponentsinan EmbeddedLinuxsystem
3
Thirdpartylibrariesandapplications
OneoftheadvantagesofembeddedLinuxisthewiderangeof thirdpartylibrariesandapplicationsthatonecanleverageinits product
Theyarefreelyavailable,freelydistributable,andthankstotheir opensourcenature,theycanbeanalyzedandmodifiedaccording totheneedsoftheproject
However,efficientlyreusingthesecomponentsisnotalways easy.Onemust:
Findthesecomponents Choosethemostappropriateones Crosscompilethem Integratethemintheembeddedsystemandwiththeother applications
4
Findexistingcomponents
Freshmeat,awebsitereferencingmostopensourceprojects http://www.freshmeat.net FreeSoftwareDirectory http://directory.fsf.org LookatotherembeddedLinuxproducts,andseewhattheir componentsare Lookatthelistofsoftwarepackaged byembeddedLinuxbuildsystems
Thesearetypicallychosenfortheirsuitabilitytoembeddedsystems
AskthecommunityorGoogle Thispresentationwillalsofeaturealistofcomponentsfor commonneeds

5
Choosingcomponents
Notallfreesoftwarecomponentsarenecessarilygoodtoreuse. Onemustpayattentionto:
Vitalityofthedeveloperandusercommunities.Thisvitalityensures longtermmaintenanceofthecomponent,andrelativelygoodsupport. Itcanbemeasuredbylookingatthemailinglisttrafficandtheversion controlsystemactivity. Qualityofthecomponent.Typically,ifacomponentisalready availablethroughembeddedbuildsystems,andhasadynamicuser community,itprobablymeansthatthequalityisrelativelygood. License.Thelicenseofthecomponentmustmatchyourlicensing constraints.Forexample,GPLlibrariescannotbeusedinproprietary applications. Technicalrequirements.Ofcourse,thecomponentmustmatchyour technicalrequirements.Butdon'tforgetthatyoucanimprovethe existingcomponentsifafeatureismissing!
6
Licenses(1)
Allsoftwarethatareunderafreesoftwarelicensegivefour freedomstoallusers
Freedomtouse Freedomtostudy Freedomtocopy Freedomtomodifyanddistributemodifiedcopies
Seehttp://www.gnu.org/philosophy/freesw.htmlforadefinitionof FreeSoftware OpenSourcesoftware,asperthedefinitionoftheOpenSource Initiative,aretechnicallysimilartoFreeSoftwareintermsof freedoms Seehttp://www.opensource.org/docs/osdforthedefinitionof OpenSourceSoftware

7
Licenses(2)
FreeSoftwarelicensesfallintwomaincategories
Thecopyleftlicenses Thenoncopyleftlicenses
Theconceptofcopyleftistoaskforreciprocityinthe freedomsgiventoanuser. Theresultisthatwhenyoureceiveasoftwareunderacopyleft freesoftwarelicenseanddistributemodifiedversionsofit,you mustdosounderthesamelicense

Samefreedomstothenewusers It'sanincentivetocontributebackyourchangesinsteadofkeeping themsecret
Noncopyleftlicenseshavenosuchrequirements,andmodified versionscanbekeptproprietary,buttheystillrequireattribution
8
GPL
GNUGeneralPublicLicense Covers~55%ofthefreesoftwareprojects
IncludingtheLinuxkernel,Busyboxandmanyapplications
Isacopyleftlicense
Requiresderivativeworkstobereleasedunderthesamelicense ProgramslinkedwithalibraryreleasedundertheGPLmustalsobe releasedundertheGPL
Someprogramscoveredbyversion2(Linuxkernel,Busyboxand others) Moreandmoreprogramscoveredbyversion3,releasedin2007

Majorchangefortheembeddedmarket:therequirementthatthe usermustbeabletorunthemodifiedversionsonthedevice,ifthe deviceisaconsumerdevice
GPL:redistribution
Noobligationwhenthesoftwareisnotdistributed
Youcankeepyourmodificationssecretuntiltheproductdelivery
Itisthenauthorizedtodistributebinaryversions,ifoneofthe followingconditionsismet:
Conveythebinarywithacopyofthesourceonaphysicalmedium Conveythebinarywithawrittenoffervalidfor3yearsthatindicates howtofetchthesourcecode Conveythebinarywiththenetworkaddressofalocationwherethe sourcecodecanbefound Seesection6.oftheGPLlicense
Inallcases,theattributionandthelicensemustbepreserved
Seesection4.and5.
10
LGPL
GNULesserGeneralPublicLicense Covers~10%ofthefreesoftwareprojects Acopyleftlicense
Modifiedversionsmustbereleasedunderthesamelicense But,programslinkedagainstalibraryundertheLGPLdonotneed tobereleasedundertheLGPLandcanbekeptproprietary However,theusermustkeeptheabilitytoupdatethelibrary independentlyfromtheprogram,sodynamiclinkingmustbeused
UsedinsteadoftheGPLformostofthelibraries,includingtheC libraries
Someexceptions:MySQL,orQt<=4.4
Alsoavailableintwoversions,v2andv3
11
Licensing:examples
YoumakemodificationstotheLinuxkernel(toadddriversor adapttoyourboard),toBusybox,UBootorotherGPLsoftware
Youmustreleasethemodifiedversionsunderthesamelicense, andbereadytodistributethesourcecodetoyourcustomers
YoumakemodificationstotheClibraryoranyotherLGPLlibrary
Youmustreleasethemodifiedversionsunderthesamelicense
YoucreateanapplicationthatreliesonLGPLlibraries
Youcankeepyourapplicationproprietary,butyoumustlink dynamicallywiththeLGPLlibraries
Youmakemodificationstoanoncopyleftlicensedsoftware
Youcankeepyourmodificationsproprietary,butyoumuststill credittheauthors
12
Noncopyleftlicenses
Alargefamilyofnoncopyleftlicensesthatarerelativelysimilarin theirrequirements Afewexamples
Apachelicense(~4%) BSDlicense(~6%) MITlicense(~4%) X11license Artisticlicense(~9%)
13
BSDlicense
Copyright(c)<year>,<copyrightholder> Allrightsreserved. Redistributionanduseinsourceandbinaryforms,withor withoutmodification,arepermittedprovidedthatthefollowing conditionsaremet: *Redistributionsofsourcecodemustretaintheabove copyrightnotice,thislistofconditionsandthefollowing disclaimer. *Redistributionsinbinaryformmustreproducetheabove copyrightnotice,thislistofconditionsandthefollowing disclaimerinthedocumentationand/orothermaterialsprovided withthedistribution. *Neitherthenameofthe<organization>northenamesofits contributorsmaybeusedtoendorseorpromoteproductsderived fromthissoftwarewithoutspecificpriorwrittenpermission. [...]
14
Isthisfreesoftware?
Mostofthefreesoftwareprojectsarecoveredby~10wellknown licenses,soitisfairlyeasyforthemajorityofprojecttogeta goodunderstandingofthelicense Otherwise,readthelicensetext CheckFreeSoftwareFoundation'sopinion http://www.fsf.org/licensing/licenses/ CheckOpenSourceInitiative'sopinion http://www.opensource.org/licenses
15
Respectfreesoftwarelicenses
FreeSoftwareisnotpublicdomainsoftware,thedistributorshave obligationsduetothelicenses
Beforeusingafreesoftwarecomponent,makesurethelicense matchesyourprojectconstraints Makesuretokeepacompletelistofthefreesoftwarepackagesyou use,theoriginalversionyouusedandtokeepyourmodificationsand adaptationswellseparatedfromtheoriginalversion Conformtothelicenserequirementsbeforeshippingtheproductto thecustomers
FreeSoftwarelicenseshavebeenenforcedsuccessfullyincourts
GPLviolations.org,http://www.gplviolations.org SoftwareFreedomLawCenter,http://www.softwarefreedom.org/ Askyourlegaldepartment!
16
Keepingchangesseparate(1)
Whenintegratingexistingopensourcecomponentsinyour project,itissometimesneededtomakemodificationstothem
Betterintegration,reducedfootprint,bugfixes,newfeatures,etc.
Insteadofmixingthesechanges,itismuchbettertokeepthem separatefromtheoriginalcomponentversion
Ifthecomponentneedstobeupgraded,easiertoknowwhat modificationsweremadetothecomponent Ifsupportfromthecommunityisrequested,importanttoknowhow differentthecomponentwe'reusingisfromtheupstreamversion Makescontributingthechangesbacktothecommunitypossible
Itisevenbettertokeepthevariouschangesmadeonagiven componentseparate
Easiertoreviewandtoupdatetonewerversions
17
Keepingchangesseparate(2)
ThesimplestsolutionistouseQuilt
Quiltisatoolthatallowstomaintainastackofpatchesoversource code Makesiteasytoadd,removemodificationsfromapatch,toadd andremovepatchesfromstackandtoupdatethem Thestackofpatchescanbeintegratedintoyourversioncontrol system https://savannah.nongnu.org/projects/quilt/
Anothersolutionistouseaversioncontrolsystem
Importtheoriginalcomponentversionintoyourversioncontrol system Maintainyourchangesinaseparatebranch
18
Toolsforthetargetdevice
Networking
19
sshserverandclient:dropbear
http://matt.ucc.asn.au/dropbear/dropbear.html Verysmallmemoryfootprintsshserverforembeddedsystems Satisfiesmostneeds.Bothclientandserver! Size:110KB,staticallycompiledwithuClibconi386. (OpenSSHclientandserver:approx1200KB, dynamicallycompiledwithglibconi386) Usefulto: Getaremoteconsoleonthetargetdevice Copyfilestoandfromthetargetdevice(scporrsynce ssh).
20
Benefitsofawebserverinterface
Manynetworkenableddevicescanjusthaveanetworkinterface Examples:modems/routers,IPcameras,printers... Noneedtodevelopdriversandapplicationsforcomputers connectedtothedevice.Noneedtosupportmultipleoperating systems! JustneedtodevelopstaticordynamicHTMLpages (possiblywithpowerfulclientsideJavaScript). Easywayofprovidingaccesstodeviceinformationand parameters. Reducedhardwarecosts(noLCD,verylittlestoragespace needed)
21
thttpd
Tiny/Turbo/ThrottlingHTTP server http://acme.com/software/thttpd/ Simple ImplementstheHTTP/1.1 minimum(orjustalittlemore) Simpletoconfigureandrun. Small Executablesize:88K(version 2.25b),Apache2.0.52:264K Verylowmemoryconsumption: doesnotforkandverycareful aboutmemoryconsumption. Portable Compilescleanlyonmost Unixlikeoperatingsystems Fast Aboutasfastasfullfeatured servers.Muchfasteronvery highloads(becausereduces theserverloadforthesame amountofwork) Secure Designedtoprotectthe webservermachinefrom attacks.
22
Otherwebservers
BusyBoxhttpserver:http://busybox.net Tiny:onlyadds9KtoBusyBox1.5(dynamicallylinkedwithglibcon i386,withallfeaturesenabled.)!Sufficientfeaturesformanydevices withawebinterface,includingCGI,httpauthenticationandscript support(likePHP). License:GPL Boa:http://www.boa.org/ Designedtobesimple,fastandsecure. Unlikethttpd,noparticularcareformemoryordiskfootprintthough. Embeddedsystems:prettypopular,thoughnottargetedbydevelopers. lighthttpd:http://lighttpd.net Lowfootprintservergoodatmanaginghighloads. Maybeusefulinembeddedsystemstoo.
23
Networkutilities(1)
avahiisanimplementationofMulticastDNSServiceDiscovery, thatallowsprogramstopublishanddiscoverservicesonalocal network bind,aDNSserver iptables,theuserspacetoolsassociatedtotheLinuxfirewall, Netfilter iwandwirelesstools,theuserspacetoolsassociatedto Wirelessdevices netsnmp,implementationoftheSNMPprotocol openntpd,implementationoftheNetworkTimeProtocol,for clocksynchronization openssl,atoolkitforSSLandTLSconnections
24
Networkutilities(2)
pppd,implementationofthePointtoPointProtocol,usedfor dialupconnections samba,implementstheSMBandCIFSprotocols,usedby Windowstosharefilesandprinters coherence,aUPnP/DLNAimplementation vsftpd,proftpd,FTPservers
25
Systemutilities
26
Systemutilities
dbus,aninterapplicationobjectorientedcommunicationbus gpsd,adaemontointerpretandshareGPSdata hal,theHardwareAbstractionLayersuite.Adaemonthat receiveshardwarenotifications,maintainsadatabaseof availablehardwaredevicesandoffersaDBusinterface libraw1394,rawaccesstoFirewiredevices libusb,auserspacelibraryforaccessingUSBdeviceswithout writinganinkerneldriver Utilitiesforkernelsubsystems:i2ctoolsforI2C,inputtoolsfor input,mtdutilsforMTDdevices,usbutilsforUSBdevices
27
Languageinterpreters
28
Languageinterpreters
Interpretersforthemostcommonscriptinglanguagesare available.Usefulfor
Applicationdevelopment Webservicesdevelopment Scripting
Languagessupported
Lua Python Perl Ruby TCL PHP
29
Audio,videoandmultimedia
30
Audio,videoandmultimedia
gstreamer,amultimediaframework
Allowstodecode/encodeawidevarietyofformats Supportshardwareencodersanddecodersthroughplugins
alsalib,theuserspacetoolsassociatedtotheALSAsound kernelsubsystem Encodinganddecodinglibrariessuchasflac,libogg,libtheora, libvorbis,libmad,libsndfile,speex,etc.
31
Graphicaltoolkits
32
Graphicaltoolkits Lowlevelsolutionsandlayers
33
DirectFB(1)
Lowlevelgraphicallibrary
Lines,rectangles,trianglesdrawingandfilling Blitting,flipping Textdrawing Windowsandtransparency Imageloadingandvideodisplay
Butalsohandlesinputeventhandling:mouse,keyboard, joystick,touchscreen,etc. Providesacceleratedgraphicoperationsonvarious hardware,morecanbeaddedinaneasyway Integratedwindowinginfrastructure

34
DirectFB(2)
Singleapplicationbydefault,butmultipleapplicationscan sharetheframebufferthankstofusion Developmentandcommunity:veryactive License:LGPL2.1 http://www.directfb.org
35
DirectFB:sizeanddependencies
DirectFB library ~450k plugins ~200k Total:~650k Total:~1.4m
Utils freetype ~380k libpng ~130k libjpeg ~130k libts ~6k zlib ~64k
Total:~710k Someofthese dependenciesareoptional. Thisisatypicalsetup.

36
DirectFB:architecture
Application
DirectFB fbdev devmem
FB driver
Kernel
/dev/ mem
Hardware
37
DirectFB:usage(1)
Multimediaapplications
ForexampletheDiskoframework,forsettopboxrelated applications
Simplegraphicalapplications
Industrialcontrol Devicecontrolwithlimitednumberofwidgets
Visualizationapplications Asalowerlayerforhigherlevelgraphicallibraries
38
DirectFB:usage(2)
39
SDL
Alibraryoriginallydesignedforgamedevelopment Inadditiontographicdisplay,alsoprovidesinputevent management,sound,CDROMaudio,threads,timers,etc. CanworkontopofX11,theframebufferorDirectFB
AndDirectFBcanworkontopofSDLaswell:)
TheAPIisroughlythesamelevelastheoneofDirectFB DevelopedinC,CAPI,manybindingsavailable Activelymaintained DirectFBisprobablymorecommoninembeddedsystems, whileSDLismorecommonforsmalldesktopgames License:LGPL http://www.libsdl.org/

40
SDLscreenshots
QEMU:CPUandsystememulator
Pig:ademoarcadegame.7000lines.
41
X.orgKDrive
StandalonesimplifiedversionoftheXserver,forembedded systems
FormerlyknowasTinyX KdriveisintegratedintheofficialX.orgserver
WorksontopoftheLinuxframebuffer,thankstotheXfbdev variantoftheserver RealXserver

FullysupportstheX11protocol:drawing,inputeventhandling,etc. AllowstouseanyexistingX11applicationorlibrary
Activelydevelopedandmaintained X11license http://www.x.org

42
Kdrive:sizeanddependencies
Xserver Xfbdev ~1.2m Fonts
fromafewkb toseveralmb
Utils dbus lib:~200k bin:~350k libsysfs ~27k libpng ~130k zlib ~64k pixman ~130k
Xlibraries libxcb ~300k liblbxutil ~156k libX11 ~920k libXfont ~380k Misclibs ~770k Total:2.5m
expat ~120k freetype ~380k fontconfig ~165k Total:1.5m Total,withoutX toolkit:5.4m

43
Xtoolkit(optional) libXaw6,7,8 libXt ~900k ~330k
Kdrive:architecture
Application Application Toolkit
X.orgKdriveserver fbdev
FB driver
Kernel
Hardware
44
Kdrive:usage
CanbedirectlyprogrammedusingXlib/XCB
Lowlevelgraphiclibrary Probablydoesn'tmakesensesinceDirectFBisamorelightweight solutionforanAPIofroughlythesamelevel(nowidgets)
Or,usuallyusedwithatoolkitontopofit
Gtk Qt Fltk WxEmbedded
45
Graphicaltoolkits Highlevelsolutions
46
Gtk
Thefamoustoolkit,providingwidgetbased highlevelAPIstodevelopgraphical applications StandardAPIinC,butbindingsexistfor variouslanguages:C++,Python,etc. TwoGDKbackends
TheclassicalXorgbackend TheDirectFBbackend,whichremovesthe needforanXorgserver
GtkDFB DirectFB Kernel Hardware
Gtk X.orgKdrive Kernel Hardware

47
Nowindowingsystem,alightweightwindow managerneededtorunseveral applications.Possiblesolution:Matchbox. License:LGPL http://www.gtk.org
GtkDFB:dependenciesandsize
Gtkgalaxy libgtk2 ~3.2m atk ~100k pango ~400k cairo ~350k libglib2 ~1.4m
Imagelibraries
libpng ~130k jpeg ~130k tiff ~320k
directfb ~750k Total:~9MB
Utils gettext ~510k libintl ~30k fontconfig ~165k zlib ~64k freetype ~365k expat ~120k pixman ~130k
48
Gtkstackcomponents
Glib,coreinfrastructure
ObjectorientedinfrastructureGObject Eventloop,threads,asynchronousqueues,plugins,memory allocation,I/Ochannels,stringutilities,timers,dateandtime, internationalization,simpleXMLparser,regularexpressions Datatypes:memoryslicesandchunks,linkedlists,arrays,trees, hashtables,etc.
Pango,internationalizationoftexthandling ATK,accessibilitytoolkit Cairo,vectorgraphicslibrary Gtk+,thewidgetlibraryitself TheGtkstackisacompleteframeworktodevelopapplications

49
GNOMEMobile
TheGtkstackispartofthelargerGNOMEMobileplatform
50
Gtkexamples
OM2007.2platformon OpenMokophone
Maemoplatformon NokiaInternettablets
InterfaceofVernierdata acquisitionand visualizationsystems

51
Qt(1)
Theotherfamoustoolkit,providingwidgetbasedhighlevel APIstodevelopgraphicalapplications
QtforEmbeddedLinux,formerlyknownasQtopiaCore,is theversionofQtthatrunsontopofaframebuffer,onembedded devices.Itincludesawindowingsystem QtExtended,formerlyknownasQtopia,extendsQtfor EmbeddedLinuxwithusefulcomponentsonembedded devices:communication,contents,applicationspecificanduser experiencecomponents.
ImplementedinC++
theC++libraryisrequiredonthetargetsystem standardAPIinC++,butbindingsarealsoavailableforother languages http://www.qtsoftware.com/products/platform/qtforembeddedlinux
52
Qt(2)
Workseitherontopof
Framebuffer X11 DirectFBbackendintegratedinversion4.4,whichallowstotake advantageoftheaccelerationprovidedbyDirectFBdrivers
Qtismorethanjustagraphicaltoolkit,italsooffersacomplete developmentframework:datastructures,threads,network, databases,XML,etc. QtEmbeddedhasanintegratedwindowingsystem,allowing severalapplicationstosharethesamescreen Verywelldocumented Sinceversion4.5,availableundertheLGPL,allowingproprietary applications

53
Qt:sizeanddependencies
Qtgalaxy QtCore ~2.6m QtGui ~8.6m QtNetwork ~740k QtScript ~1.1m QtTest ~64k QtXml ~290k Imgplugins ~750k Libraries jpeg ~130k libpng ~130k zlib ~64k freetype ~365k libstdc++ ~675k
Fonts ~160k
Total:~15.5MB
54
Qt'susage
QtontheDash Expressnavigation system
QtontheNetflix PlayerbyRoku
QtontheOpenMoko phone
55
Otherlessusedsolutions
Enlightenmentfoundationlibraries
Verypowerful,butcomplicatedtouseduetothelackof documentation http://www.enlightenment.org/p.php?p=about/efl
FLTK
LightweightC++toolkit.Version2.xworksonlyontopofX.org. Usedbyrelativelyfewapplications. http://www.fltk.org
WxEmbedded
TheembeddedversionofWxWindows.WorksontopofX.organd DirectFB http://www.wxwidgets.org/docs/embedded.htm
56
Databases
57
LightweightdatabaseSQLite
http://www.sqlite.org SQLiteisasmallClibrarythatimplementsaselfcontained, embeddable,lightweight,zeroconfigurationSQLdatabaseengine ThedatabaseengineofchoiceforembeddedLinuxsystems Canbeusedasanormallibrary Canbedirectlyembeddedintoaapplication,evenaproprietaryone sinceSQLiteisreleasedinthepublicdomain
58
Webbrowsers
59
Fastandtinywebbrowser:Dillo
http://www.dillo.org/ Veryfast,lightweightwebbrowserwritten inC/C++,withaFLTK2interface TheDillobinaryfitsin940KBonx86 (doesn'tincludetheFLTK2sharedlibraries) License:GPL Supportsmanystandardfeatures:cookies,images,tables,CSS... Extensiblethroughplugins(e.g.ftp) Fitswellonsmallscreens Stillmissing:frames,javascript AgoodsolutionifyourproductjustneedstodisplayitsownHTML pages.Notsufficientasageneralpurposewebbrowser.
60
Links
http://links.twibright.com/ Portablebrowserwithmanyfeatures: frames,tables,SSL,Javascript... Runseitherintextmodeorwith severalgraphicalbackends: X,DirectFB,SDL,SVGAlib. Size:4MB(i386,glibc)+shared libraries(libpng,libssl...)
61
Fullfeaturedbrowser:MozillaFirefox
http://www.mozilla.org/products/firefox/ LightweightandfastbrowserbasedonMozilla Fullfeatured:CSS,SSL,Javascript,tabbedbrowsing,pop upblocking...,butveryeasytoconfigure. Takesaround40MBofRAMwith8tabsopen. Need25MBofstoragespace(SharpZaurus) Designedtobecrossplatform.Alreadyusedinembedded systemswithsufficientscreenresolution(webpads,high endPDAs) Greatforconsumersappliances.Looksfamiliarto consumers:thedefaultthemerecallsIE.
62
WebKit
http://webkit.org/ Webbrowserengine. Applicationframeworkthatcanbeused todevelopwebbrowsers. License:portionsinLGPLandothersinBSD. Proprietaryapplicationsallowed. Usedeverywhere(MacOSX,iPhone,GoogleAndroidandChrome...) Manyapplications(browsers,emailclients...)arealreadyusingWebKit: http://trac.webkit.org/projects/webkit/wiki/Applications%20using%20WebKit Multiplegraphicalbackends:Qt4,GTK... Lightweightwebbrowsers:Midori(GTK),Arora(Qt) Youcoulduseittocreateyourcustombrowser.
63
Exampleofcomponentsusedinrealdevices
64
Industrialapplications
Inmanyindustrialapplications,thesystemisonlyresponsiblefor monitoringandcontroladevice Suchasystemisusuallyrelativelysimpleintermsofcomponents
Kernel BusyBox Clibrary ApplicationsrelyingdirectlyontheClibrary,sometimesusingthe realtimecapabilitiesoftheLinuxkernel SometimesaWebserverforremotecontrol,oranotherserver implementingacustomprotocol
65
DigitalPhotoFrame:requirements
ExampletakenfromaconferenceofMattPorter,Embedded AlleyatELC2008 Hardware:ARMSoCwithDSP,audio,800x600LCD,MMC/SD, NAND,buttons,speakers Thephotoframemustbeableto
DisplaytotheLCD DetectSDcardinsertion,notifyapplicationsoftheinsertionsothat applicationscanbuildacatalogofthepicturesontheSDcard Modern3DGUIwithnicetransitions Navigationthroughbuttons Supportaudioplayback(MP3,playlists,ID3tag) JPEGresizeandrotation
66
DigitalPhotoFrame:components(1)
Basesystem
ComponentspresentinvirtuallyallembeddedLinuxsystems TheUBootbootloader LinuxKernel
DriversforSD/MMC,framebuffer,sound,inputdevices
Busybox Buildsystem,inthiscasewasOpenEmbedded Components:uboot,linux,busybox
67
EventhandlingtodetectSDcardinsertion
udev,thatreceiveseventsfromthekernel,createsdevicenodes, andsendseventstoHAL HAL,whichmaintainsadatabaseofavailabledevicesandprovides aDBusAPI DBustoconnectHALwiththeapplication.Theapplication subscribestoHALeventthroughDBusandgetsnotifiedwhenthey aretriggered Components:udev,hal,dbus
68
JPEGdisplay
libjpegtodecodethepictures jpegtrantoresizeandrotatethem FIM(FbiImproved)fordithering
MP3support
libmadforplaying libid3forID3tagsreading libm3utosupportplaylists UsedvendorprovidedcomponentstoleveragetheDSPtoplay MP3
69
3Dinterface
Vincent,anopensourceimplementationofOpenGLES Clutter,higherlevelAPItodevelop3Dapplications
Applicationitself
Managesmediaevents UsestheJPEGlibrariestodecodeandrenderpictures ReceivesLinuxinputeventsfrombuttonsanddrawsOpenGL basedUIdevelopedwithClutter Manageauserdefinedconfiguration PlaythemusicwiththeMP3relatedlibraries Displayphotoslideshow
70
Systembuilding
71
Systembuilding:goalandsolutions
Goal
Integrateallthesoftwarecomponents,boththirdpartyandin house,intoaworkingrootfilesystem Itinvolvesthedownload,extraction,configuration,compilation andinstallationofallcomponents,andpossiblyfixingissues andadaptingconfigurationfiles
Severalsolutions
Manually Systembuildingtools Distributionsorreadymadefilesystems
72
Systembuilding:manually
Manuallybuildingatargetsysteminvolvesdownloading, configuring,compilingandinstallingallthecomponentsofthe system. Allthelibrariesanddependenciesmustbeconfigured,compiled andinstalledintherightorder. Sometimes,thebuildsystemusedbylibrariesorapplicationsis notverycrosscompilefriendly,sosomeadaptationsare necessary. Thereisnoinfrastructuretoreproducethebuildfromscratch, whichmightcauseproblemsifonecomponentneedstobe changed,ifsomebodyelsetakesovertheproject,etc.
73
Systembuilding:manually(2)
Manualsystembuildingisnotrecommendedforproduction projects However,usingautomatedtoolsoftenrequiresthedeveloperto digintospecificissues Havingabasicunderstandingofhowasystemcanbebuilt manuallyisthereforeveryusefultofixissuesencounteredwith automatedtools
Wewillfirststudymanualsystembuilding,andduringapractical lab,createasystemusingthismethod Then,wewillstudytheautomatedtoolsavailable,anduseoneof themduringalab
74
Systemfoundations
Abasicrootfilesystemneedsatleast
Atraditionaldirectoryhierarchy,with/bin,/etc,/lib,/root,
/usr/bin,/usr/lib,/usr/share,/usr/sbin,/var,/sbin
Asetofbasicutilities,providingatleasttheinitprogram,ashelland othertraditionalUnixcommandlinetools.Thisisusuallyprovided byBusybox TheClibraryandtherelatedlibraries(thread,math,etc.)installed in/lib Afewconfigurationfiles,suchas/etc/inittab,andinitialization scriptsin/etc/init.d
OntopofthisfoundationcommontomostembeddedLinux system,wecanaddthirdpartyorinhousecomponents
75
Targetandbuildspaces
Thesystemfoundation,BusyboxandClibrary,arethecoreof thetargetrootfilesystem However,whenbuildingothercomponents,onemustdistinguish twodirectories
Thetargetspace,whichcontainsthetargetrootfilesystem, everythingthatisneededforexecutionoftheapplication Thebuildspace,whichwillcontainalotmorefilesthanthe targetspace,sinceitisusedtokeepeverythingneededto compilelibrariesandapplications.Sowemustkeeptheheaders, documentation,andotherconfigurationfiles
76
Buildsystems
Eachopensourcecomponentcomeswithamechanismto configure,compileandinstallit
AbasicsimpleMakefile
NeedtoreadtheMakefiletounderstandhowitworksandhowto tweakitforcrosscompilation
AbuildsystembasedontheAutotools
Asthisisthemostcommonbuildsystem,wewillstudyitindetails
CMake,http://www.cmake.org/
Newerandsimplerthantheautotools.Usedbylargeprojectssuchas KDEorSecondLife
Scons,http://www.scons.org/ Waf,http://code.google.com/p/waf/ Othermanualbuildsystems

77
Autotoolsandfriends
Afamilyoftools,whichassociatedtogetherformacompleteand extensiblebuildsystem
autoconfisusedtohandletheconfigurationofthesoftware package automakeisusedtogeneratetheMakefilesneededtobuildthe softwarepackage pkgconfigisusedtoeasecompilationagainstalreadyinstalled sharedlibraries libtoolisusedtohandlethegenerationofsharedlibrariesina systemindependentway
Mostofthesetoolsareoldandrelativelycomplicatedtouse,but theyareusedbyamajorityoffreesoftwarepackagestoday.One musthaveabasicunderstandingofwhattheydoandhowthey work.
78
automake/autoconf/autoheader
configure.in
autoconf autoheader
configure config.h.in
Makefile.am
automake
Makefile.in
configure
Makefile
Writtenbythedeveloper Generatedbythedeveloperusingtheautotools Generatedbytheuserbyrunningtheconfigurescript
config.h
79
automake/autoconf
Fileswrittenbythedeveloper
configure.indescribestheconfigurationoptionsandthechecks doneatconfiguretime Makefile.amdescribeshowthesoftwareshouldbebuilt
TheconfigurescriptandtheMakefile.infilesare generatedbyautoconfandautomakerespectively.
Theyshouldnevermodifieddirectly Theyareusuallyshippedpregeneratedinthesoftwarepackage, becausethereareseveralversionsofautoconfandautomake, andtheyarenotcompletelycompatible
TheMakefilefilesaregeneratedatconfiguretime,before compiling
Theyarenevershippedinthesoftwarepackage.
80
Configuringandcompiling(1)
Thetraditionalstepstoconfigureandcompileanautotoolsbased packageare: ./configure make makeinstall Thesestepsworkwellfornativecompilation.Forcrosscompilation, thingsarealittlebitmorecomplicated.
AtleastsomeoftheenvironmentvariablesAR,AS,LD,NM,CC,GCC, CPP,CXX,STRIP,OBJCOPYmustbedefinedtopointtothepropercross compilationtools.Thehosttupleisalsobydefaultusedasprefix. Thehostargumentmustbepassedtotheconfigurescript.The buildargumentisautomaticallydetected,andtargetisonlyfortools generatingcode. Itisrecommendedtopasstheprefixargument.Itdefinesfromwhich locationthesoftwarewillruninthetargetenvironment.Usually,/usrisfine.
81
Configuringandcompiling(2)
Ifonesimplyrunsmakeinstall,thesoftwarewillbeinstalledinthe directorypassedasprefix.Forcrosscompiling,onemustpass theDESTDIRargumenttospecifywherethesoftwaremustbe installed. Makingthedistinctionbetweentheprefix(aspassedwith prefixatconfiguretime)andthedestinationdirectory(as passedwithDESTDIRatinstallationtime)isveryimportant.
Example:
exportPATH=/usr/local/armlinux/bin:$PATH exportCC=armlinuxgcc exportSTRIP=armlinuxstrip ./configurehost=armlinux make makeDESTDIR=/home/<user>/work/rootfsinstall
82
Installation
Theautotoolsbasedsoftwarepackagesprovidebothainstall andinstallstripmaketargets,usedtoinstallthesoftware, eitherstrippedorunstripped. Forapplications,thesoftwareisusuallyinstalledin <prefix>/bin,withconfigurationfilesin<prefix>/etcand datain<prefix>/share/<application>/. Thecaseoflibrariesisalittlemorecomplicated:
In<prefix>/lib,thelibraryitself(a.so.<version>),afew symboliclinks,andthelibtooldescriptionfile(a.lafile) Thepkgconfigdescriptionfilein<prefix>/lib/pkgconfig Includefilesin<prefix>/include/ Sometimesa<libname>configprogramin<prefix>/bin Documentationin<prefix>/share/manor <prefix>/share/doc/
83
Installation(2)
Contentsof/usrafterinstallationofzlibandlibpng
./lib ./lib/libpng12.la ./lib/libpng.la ./lib/libpng12.a ./lib/libpng.a ./lib/libpng.so.3.32.0 ./lib/libpng12.so.0.32.0 ./lib/libpng12.so.0 ./lib/libpng12.so ./lib/libpng.so ./lib/libpng.so.3 ./lib/pkgconfig ./lib/pkgconfig/libpng.pc ./lib/pkgconfig/libpng12.pc ./lib/libz.so.1.2.3 ./lib/libz.so ./lib/libz.so.1 >libpng12.la >libpng12.a >libpng12.so.0.32.0 >libpng12.so.0.32.0 >libpng12.so >libpng.so.3.32.0 >libpng12.pc >libz.so.1.2.3 >libz.so.1.2.3 Libtool description file Static versionof thelibrary Dynamic versionof thelibrary
Pkgconfig description file Zlib dynamic library
84
Installationinthebuildandtargetspaces
Fromallthesefiles,everythingexceptdocumentationis necessarytobuildanapplicationthatreliesonlibpng.
Thesefileswillgointothebuildspace
However,onlythelibrarybinaryin<prefix>/libandsome symboliclinksareneededtoexecutetheapplicationonthe target.

Onlythesefileswillgointhetargetspace
Thebuildspacemustbekeptinordertobuildotherapplications orrecompileexistingapplications.
85
Let'sfindthelibraries
Whencompilinganapplicationoralibrarythatreliesonother libraries,thebuildprocessbydefaultlooksin/usr/libfor librariesand/usr/includeforheaders. ThefirstthingtodoistosettheCFLAGSandLDFLAGS environmentvariables: exportCFLAGS=I/my/build/space/usr/include/ exportLDFLAGS=L/my/build/space/usr/lib Thelibtoolfiles(.lafiles)mustbemodifiedbecausetheyinclude theabsolutepathsofthelibraries: libdir='/usr/lib' +libdir='/my/build/space/usr/lib' ThePKG_CONFIG_PATHenvironmentvariablemustbesettothe locationofthe.pcfilesandthePKG_CONFIG_SYSROOT_DIR variablemustbesettothebuildspacedirectory.
86
pkgconfig
pkgconfigisatoolthatallowstoqueryasmalldatabasetoget informationonhowtocompileprogramsthatdependonlibraries Thedatabaseismadeof.pcfiles,installedbydefaultin <prefix>/lib/pkgconfig/. pkgconfigisusedbytheconfigurescripts togetthelibraryconfigurations Itcanalsobeusedmanuallytocompileanapplication:
armlinuxgccotesttest.c$(pkgconfiglibscflagsthelib)
Bydefault,pkgconfiglooksin/usr/lib/pkgconfigforthe *.pcfiles,andassumesthatthepathsinthesefilesarecorrect. PKG_CONFIG_PATHallowstosetanotherlocationforthe*.pc filesandPKG_CONFIG_SYSROOT_DIRtoprependaprefixtothe pathsmentionedinthe.pcfiles.

87
PracticallabManualcrosscompiling
Manuallycrosscompiling applicationsandlibraries Learningaboutcommon techniquesandissues.
88
Systembuildingtools:principle
Differenttoolsareavailabletoautomatetheprocessof buildingatargetsystem,includingthekernel,and sometimesthetoolchain. Theyautomaticallydownload,configure,compileandinstall allthecomponentsintherightorder,sometimesafter applyingpatchestofixcrosscompilingissues. Theyalreadycontainalargenumberofpackages,that shouldfityourmainrequirements,andareeasilyextensible. Thebuildbecomesreproducible,whichallowstoeasily changetheconfigurationofsomecomponents,upgrade them,fixbugs,etc.
89
Availablesystembuildingtools
Largechoiceoftools Buildroot,developedbythecommunity http://www.buildroot.net PTXdist,developedbyPengutronix http://www.pengutronix.de/software/ptxdist/index_en.html OpenWRT,originallyaforkofBuildrootforwirelessrouters,nota moregenericproject http://www.openwrt.org LTIB,developedmainlybyFreescale.GoodsupportforFreescale boards,butsmallcommunity http://www.bitshrine.org/ OpenEmbedded,moreflexiblebutalsofarmorecomplicated http://www.openembedded.org Vendorspecifictools(siliconvendororembeddedLinuxvendor)
90
Buildroot(1)
Allowstobuildatoolchain,arootfilesystemimagewithmany applicationsandlibraries,abootloaderandakernelimage
Oranycombinationofthepreviousitems
SupportsbuildinguClibctoolchainsonly,butcanuseexternal uClibcorglibctoolchains Over500+applicationsorlibrariesintegrated,frombasicutilities tomoreelaboratesoftwarestacks:X.org,Gstreamer,Qt,Gtk, Webkit,etc. Goodforsmalltomediumembeddedsystems,withafixedsetof features

Nosupportforgeneratingpackages(.debor.ipk) Needscompleterebuildformostconfigurationchanges.
Activecommunity,releasespublishedevery3months.
91
Buildroot(2)
Configurationtakesplacethrough a*configinterfacesimilartothe kernel makemenuconfig Allowstodefine
ArchitectureandspecificCPU Toolchainconfiguration Setofapplicationsandlibraries tointegrate Filesystemimagestogenerate Kernelandbootloader configuration
Buildbyjustrunning make
92
Buildroot:addinganewpackage(1)
Apackageallowstointegrateauserapplicationorlibraryto Buildroot Eachpackagehasitsowndirectory(suchas package/gqview).Thisdirectorycontains:
AConfig.infile(mandatory),describingtheconfigurationoptions forthepackage.Atleastoneisneededtoenablethepackage.This filemustbesourcedfrompackage/Config.in Agqview.mkfile(mandatory),describinghowthepackageisbuilt. Patches(optional).Eachfileoftheformgqview*.patchwillbe appliedasapatch.
93
Buildroot:addinganewpackage(2)
Forasimplepackagewithasingleconfigurationoptionto enable/disableit,theConfig.infilelookslike:
configBR2_PACKAGE_GQVIEW bool"gqview" dependsonBR2_PACKAGE_LIBGTK2 help GQviewisanimageviewerforUnixoperatingsystems http://prdownloads.sourceforge.net/gqview
Itmustbesourcedfrompackage/Config.in: source"package/gqview/Config.in"
94
Buildroot:addingnewpackage(3)
Createthegqview.mkfiletodescribethebuildsteps
GQVIEW_VERSION=2.1.5 GQVIEW_SOURCE=gqview$(GQVIEW_VERSION).tar.gz GQVIEW_SITE=http://prdownloads.sourceforge.net/gqview GQVIEW_AUTORECONF=NO GQVIEW_INSTALL_STAGING=NO GQVIEW_INSTALL_TARGET=YES GQVIEW_DEPENDENCIES=hostpkgconfiglibgtk2 $(eval$(callAUTOTARGETS,package,gqview))
ThelastargumentoftheAUTOTARGETSmacro,theprefixofall variablesmustbeidenticaltothesuffixofthemainconfiguration optionBR2_PACKAGE_GQVIEW TheAUTOTARGETSinfrastructureknowshowtobuildautotools packages.AmoregenericGENTARGETSinfrastructureis availableforpackagesnotusingtheautotoolsastheirbuild system.
95
OpenEmbedded
ThemostversatileandpowerfulembeddedLinuxbuildsystem
Acollectionofrecipes(.bbfiles) Atoolthatprocessestherecipes:bitbake
Integrates2000+applicationandlibraries,ishighlyconfigurable, cangeneratepackages(.ipk)tomakethesystem customizable,supportsmultipleversions/variantsofthesame package,noneedforfullrebuildwhentheconfigurationis changed. Configurationtakesplacebyeditingvariousconfigurationfiles GoodforlargerembeddedLinuxsystems,orpeoplelookingfor moreconfigurabilityandextensibility Drawbacks:nostablereleases,verysteeplearningcurve,very longfirstbuild.
96
Distributions(1)
DebianGNU/Linux,http://www.debian.org AvailableonARM,MIPSandPowerPCarchitectures Providesareadytousefilesystemwithallthesoftwareyouneed. Hugeflexibilitythankstothepackagemanagementsystem,butonly worksonlysystemswithenoughstoragesize(>300MB) andRAM(>64MB). Softwareiscompilednativelybydefault. Youcanbuildyourownrootfilesystemimagesonx86 byusingthedebootstrapcommand. EmdebianisaprojecttomakeDebianbetterforembeddedsystems: leverageDebianpackagedescriptions,butreducesdependencies, smallerconfiguration,removesdocumentation,supportsuClibc...See http://emdebian.org.
97
Distributions(2)
UbuntuGNU/Linux BasedonDebian,samebenefits Newreleaseevery6months, supportedfor18monthsoreven3years. SupportedonARM,butonlyonCortexA8andbeyond. SuppliesThumb2binaries.Neonnotsupported. Goodsolutionformobilemultimediadevices. Others FedoraalsohassupportforARM,butnotaskactively maintained.
98
Embeddeddistributions
Distributionsdesignedforspecifictypesofdevices Meego:http://meego.com/ Distributiontargetingphones,mediaplayers, netbooks,TVsandInVehicleInfotainment. BackedbyIntelandNokia. Android:http://www.android.com/ Google'sdistributionforphonesandtabletPCs. ExcepttheLinuxkernel,verydifferentuserspace thanotherLinuxdistributions.Verysuccessful, lotsofapplicationsavailable(manyproprietary). ngstrm:http://www.angstromdistribution.org/ TargetsPDAsandwebpads(SiemensSimpad...) Binariesavailableforarmlittleendian.
99
PracticallabBuildroot
Rebuildthesamesystem, thistimewithBuildroot. Seehoweasieritgets! AddingyourownDirectFBbased application.
100
GNU/Linuxworkstation
Emulators
101
qemu
http://fabrice.bellard.free.fr/qemu/ Fastprocessoremulator usingaportabledynamictranslator. Fullsystememulation Emulatestheprocessorandvariousperipherals Supported:x86,x86_64,ppc,arm,sparc,mips,m68k Toknowwhichmachinetypesaresupported: qemusystemarmM? i386,x86_64systememulation:nowclosetonativespeeds thankstothekqemukernelmodule(nowGPLv2!).
102
Otheremulators
ARMplatform SkyEye:http://skyeye.sourceforge.net
EmulatesseveralARMplatforms(AT91,Xscale...)andcan bootseveraloperatingsystems(Linux,uClinux,andothers)
Softgun:http://softgun.sourceforge.net
VirtualARMsystemwithmanyvirtualonboardperipherals. BootsLinux.
SWARMSoftwareARMarm7emulator http://www.cl.cam.ac.uk/~mwd24/phd/swarm.html
CanrunuClinux
ColdFireemulator http://www.slicer.ca/coldfire/
CanbootuClinux
103
GNU/Linuxworkstation
Commercialtoolsets
104
Commercialtoolsets
Caution:commercialdoesn'tmeanproprietary! VendorsplayfairwiththeGPLanddomaketheirsourcecode availabletotheirusers,andmostofthetime,tothecommunity. Aslongastheydistributethesourcestotheirusers,theGPL doesn'trequirevendorstosharetheirsourceswithanythird party. NoissuewithalltheGPLsourcesdevelopedbyorwiththe community. Graphicaltoolkitsdevelopedbythevendorslookproprietary. Theirlicensesarenotadvertisedontheirwebsites!Youhaveto beacustomertoknoworgetafreepreviewkittoknow.
105
Commercialtoolsetstrengths
Technicaladvantages Welltestedandsupported kernelandtoolversions Includingearlypatchesnot supportedbythemainstream kernelyet Completedevelopmenttoolsets: kernels,toolchains,utilities,binaries forimpressivelistsoftarget platforms Integratedutilitiesforautomatic kernelimage,initrdandfilesystem generation. Graphicaldevelopmentstools Developmenttoolsavailableon multipleplatforms:GNU/Linux, Solaris,Windows... Supportservices Usefulifyoudon'thaveyour ownsupportresources Longtermsupport commitment,evenfor versionsconsideredas obsoletebythecommunity, butnotbyyourusers!
106
WindRiver
WindRiverLinux: http://www.windriver.com/products/linux/ Newmarketleader,recentlyacquiredbyIntel AlotofembeddedandrealtimeexperiencefromVxWorks. Nowsaytheyintegrate,testandsupportLinux asrigorouslyastheydowithVxWorks. LinuxdevelopmentsupportedwiththeirWorkbenchintegrated developmentenvironment,alreadyusedforVxWorks. SupportstandardandrecentLinuxkernelsources,includingreal timepreemptpatches(Linux2.6.27).Alsoofferhardrealtime Linux(RealTimeCore:formerlyRTLinux).
107
Montavista
http://www.mvista.com/
Thesecondmarketleader Employssomeofthemostactivekernelhackers, inparticularonthearmplatform. Kerneldevelopmenteventuallysharedwiththe community.kernel.Manydriversmergedinmainstream Linux. Graphicaldevelopmenttoolsareproprietary.
108
TimeSys
http://timesys.com Similartoolsetofferingasothervendors. GreatflexibilityavailabletotheirLinuxLinkTM subscribers Communityfriendly:theyshareveryinterestingand generictechnicalwhitepapersandarticles.Theyalso employkeycommunityhackers(ThomasGleixner,Rob Landley...). FreeSoftwareBSPs(BoardSupportPackages) available. LinuxsoftandhardrealtimeOSproduct. Developmenttoolsseemtobeproprietary.
109
SysgoKoanSoftware
http://sysgo.com ELinOSdevelopmenttoolset, inparticularbasedonEclipse andtheLinuxTraceToolkit. IncludesFreeToolBox,afreely downloadablecompilingandrootfs creatingtoolchain. Supportsi386,armandppc. Hardrealtimesupportwiththeirown microkernel(PikeOS), anapproachsimilartoRTAI. http://koansoftware.com MakersofKaeilOS (http://koansoftware.com/kaeilos/index_en.htm), aGPLembeddedLinuxdistributionfor industrialapplications. KaeilOSsupportsi386andpopulararm platforms.Otherplatformssupportedupon request. Includesseveralgraphicaltoolkitsand supportshardrealtime(RTAI,Xenomai, preemptionpatches). Unfortunately,KaeilOSisGPLbutnot availableforpublicdownload.
110
DenxSoftwareEngineering
http://denx.de CreatedbyWolfgangDenk, theauthoroftheUBootbootloader. CreateandsupporttheEmbeddedLinuxDevelopmentKit (ELDK),acompleteandwelldocumenteddevelopment environment. ThiskitisnotonlyFreeSoftware, itcanbedownloadedfreelybyanyone. Agreatcommunitymemberandcontributor!
111
CommercialtoolsetsSummary
Majorvendors:MontaVista,WindRiver,TimeSys InvolvedinLinuxdevelopment. Smallervendors:Koan,Sysgo,Denx... Tryingtodifferentiatetheirproducts. Communitybasedcompanies:Denx,CodeSourcery Contributetocommunitytools. Mainlyoffersupportanddevelopment.
112
Commercialorcommunitysolutions?
Commercialdistributionsandtoolsets Bestifyoudon'thaveyourownsupportresourcesandhavea sufficientbudget Reallyhelpfocusingonyourrealjob:makinganembedded device. Youcanevensubcontractdriverdevelopmenttothevendor Communitydistributionsandtools Bestifyouareonatightbudget BestifyouarewillingtobuildyourownembeddedLinuxexpertise andtrainyourownsupportresources. Inanycase,yourproductsarebasedonFreeSoftware!
113
References
114
Usefulreading(1)
EmbeddedLinuxSystemDesignandDevelopment P.Raghavan,A.Lad,S.Neelakandan,Auerbach,Dec.2005. http://freeelectrons.com/redirect/elsddbook.html UsefulbookcoveringmostaspectsofembeddedLinux systemdevelopment(kernelandtools). BuildingEmbeddedLinuxSystems,O'Reilly ByKarimYaghmour,JonMasters,GiladBenYossefand PhilippeGerum,andothers(includingMichaelOpdenacker), August2008 http://oreilly.com/catalog/9780596529680/ EmbeddedLinuxPrimer,PrenticeHall ByChristopherHallinan,September2006 Coversaverywiderangeofinterestingtopics.
115
Usefulreading(2)
http://www.denx.de/wiki/DULG/Manual Lotsofusefulcommandexamples,generichelpandadvice forembeddedLinuxsystems. Seehttp://www.linuxdevices.com/articles/AT2969812114.html formorebooksonLinuxforembeddedsystems.
116
Usefulwebsites
LinuxDevices.com:http://linuxdevices.com Weeklynewsletterwithnewsandannouncementsabout embeddeddevicesrunningLinux. Articles,whitepapers,andLinuxembeddeddevicescatalog. Anexcellentsitetofollowindustrynews!
117
Internationalconferences
UsefulconferencesfeaturingembeddedLinuxandkerneltopics EmbeddedLinuxConference:http://embeddedlinuxconference.com/ OrganizedbytheCELinuxForum:California (SanFrancisco,April),inEurope(OctoberNovember). Veryinterestingkernelanduserspacetopicsforembeddedsystems developers.Presentationslidesfreelyavailable LinuxPlumbers http://linuxplumbersconf.org ConferenceonthelowlevelplumbingofLinux:kernel,audio,power management,devicemanagement,multimedia,etc. Fosdem:http://fosdem.org(Brussels,February) Fordevelopers.Presentationsaboutsystemdevelopment. Don'tmissourfreeconferencevideoson http://freeelectrons.com/community/videos/conferences/!
118
EmbeddedLinux application development
.S
make
.h
.c++
.c
Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/appdev Corrections,suggestions,contributionsandtranslationsarewelcome!
1
Contents
Applicationdevelopment DevelopingapplicationsonembeddedLinux Buildingyourapplications Sourcemanagement Integrateddevelopmentenvironments(IDEs) Versioncontrolsystems Debuggingandanalysistools Debuggers Memorycheckers Systemanalysis Developmentenvironments DevelopingonWindows
2
Applicationdevelopment
DevelopingapplicationsonembeddedLinux
3
AnembeddedLinuxsystemisjustanormalLinuxsystem,with usuallyasmallerselectionofcomponents Intermsofapplicationdevelopment,developingonembedded LinuxisexactlythesameasdevelopingonadesktopLinux system Allexistingskillscanbereused,withoutanyparticular adaptation Allexistinglibraries,eitherthirdpartyorinhouse,canbe integratedintotheembeddedLinuxsystem
Takingintoaccount,ofcourse,thelimitationoftheembedded systemsintermsofperformance,storageandmemory
4
Programminglanguage
Thedefaultprogramminglanguageforsystemlevelapplicationin LinuxisusuallyC
TheClibraryisalreadypresentonyoursystem,nothingtoadd
C++canbeusedforlargerapplications
TheC++librarymustbeaddedtothesystem Somelibraries,includingQt,aredevelopedinC++sotheyneedthe C++libraryonthesystemanyway
Scriptinglanguagescanalsobeusefulforquickapplication development,webapplicationsorscripts
Buttheyrequireaninterpreterontheembeddedsystemandhave usuallyhighermemoryconsumptionandslightlylower performances
Languages:Python,Perl,Lua,Ada,Fortran,etc.
5
Clibraryorhigherlevellibraries?
Formanyapplications,theClibraryalreadyprovidesarelatively largesetoffeatures
fileanddeviceI/O,networking,threadsandsynchronization,inter processcommunication Thoroughlydescribedintheglibcmanual,orinanyLinuxsystem programmingbook However,theAPIcarriesalotofhistoryandisnotnecessarilyeasy tograspfornewcomers
Therefore,usingahigherlevelframework,suchasQtortheGtk stack,mightbeagoodidea
Theseframeworksarenotonlygraphicallibraries,theircoreis separatefromthegraphicalpart Butofcourse,theselibrarieshavesomememoryandstorage footprint,intheorderofafewmegabytes
6
Buildingyourapplications
Forsimpleapplicationsthatdonotneedtobereallyportableor providecompiletimeconfigurationoptions,asimpleMakefilewill besufficient Formorecomplicatedapplications,orifyouwanttobeableto runyourapplicationonadesktopLinuxPCandonthetarget device,usingabuildsystemisrecommended
Lookattheautotools(ancient,complicatedbutverywidelyused)or CMake(modern,simpler,smallerbutgrowinguserbase)
TheQTlibraryisaspecialcase,sinceitcomeswithitsownbuild systemforapplications,calledqmake
7
SimpleMakefile(1)
CaseofanapplicationthatonlyusestheClibrary,containstwo sourcefilesandgeneratesasinglebinary CROSS_COMPILE?=armlinux CC=$(CROSS_COMPILE)gcc OBJS=foo.obar.o all:foobar foobar:$(OBJS) $(CC)o$@$^ clean: $(RM)ffoobar$(OBJS)
8
SimpleMakefile(2)
CaseofanapplicationthatusestheGlibandtheGPSlibraries CROSS_COMPILE?=armlinux LIBS=libgpsglib2.0 OBJS=foo.obar.o CC=$(CROSS_COMPILE)gcc CFLAGS=$(shellpkgconfigcflags$(LIBS)) LDFLAGS=$(shellpkgconfiglibs$(LIBS)) all:foobar foobar:$(OBJS) $(CC)o$@$^$(LDFLAGS) clean: $(RM)ffoobar$(OBJS)
Sourcemanagement
IntegratedDevelopmentEnvironments(IDE)
10
Sourcebrowsers
LXR:LinuxCrossReference Allowstobrowsecodethroughawebbrowser. cscope Consolemodesourcebrowsingtool. Integrateswitheditorslikeviandemacs. KScope Agraphicalinterfacetocscope. SeeourLinuxkernelanddriverdevelopmenttrainingmaterials (http://freeelectrons.com/docs/kernel)formoredetails.
11
KDevelop
http://kdevelop.org AfullfeaturedIDE! License:GPL Supportsmanylanguages: Ada,C,C++,Database,Java,Perl,PHP,Python,Ruby,Shell Supportsmanykindsofprojects: KDE,butalsoGTK,Gnome,kerneldrivers,embedded(Opie)... Manyfeatures:editor,syntaxhighlighting,codecompletion, compilerinterface,debuggerinterface,filemanager,class browser... Niceoverview:http://en.wikipedia.org/wiki/Kdevelop
12
KDevelopscreenshot
Ruby debugger
13
Eclipse(1)
http://www.eclipse.org/ Anextensible,pluginbased softwaredevelopmentkit,typicallyusedforcreatingIDEs. SupportedbytheEclipsefoundation,anonprofit consortiumofmajorsoftwareindustryvendors(IBM,Intel, Borland,Nokia,WindRiver,Zend,ComputerAssociates...). FreeSoftwarelicense(EclipsePublicLicense). IncompatiblewiththeGPL. Supportedplatforms:GNU/Linux,Unix,Windows Extremelypopular:createdalotofattraction.
14
Eclipse(2)
Eclipseisactuallyaplatformcomposedofmanyprojects: http://www.eclipse.org/projects/
SomeprojectsarededicatedatintegratingintoEclipsefeatures usefulforembeddeddevelopers(crosscompilation,remote development,remotedebugging,etc.)
TheplatformisusedbymajorembeddedLinuxsoftwarevendors fortheir(proprietary)systemdevelopmentkits: MontaVistaDevRocket,TimeSysTimeStorm,Windriver Workbench,SysgoELinOS. Eclipseisahugeproject. Itwouldrequireanentiretrainingsession!
15
Sourcemanagement
Versioncontrolsystems
16
Versioncontrolsystems
Realprojectscan'tdowithoutthem Allowmultipledeveloperstocontributeonthesameproject.Each developercanseethelatestchangesfromtheothers,orchoose tostickwitholderversionsofsomecomponents. Allowtokeeptrackofchanges,andrevertthemifneeded. Allowdeveloperstohavetheirowndevelopmentbranch (branching) Supposedtohelpdevelopersresolvingconflictswithdifferent branches(merging)
17
Traditionalversioncontrolsystems
Relyonacentralrepository.Themostpopularopensourceones: CVSConcurrentVersionsSystem
Stillquitepopular Shouldnolongerbeusedfornewprojects http://en.wikipedia.org/wiki/Concurrent_Versions_System
Subversion
CreatedasareplacementofCVS,removingmanyofitslimitations. Commitsonseveralfiles,properrenamingsupport,better performances,etc. TheuserinterfaceisverysimilartoCVS http://en.wikipedia.org/wiki/Subversion_(software)
18
Distributedsourcecontrolsystems(1)
Nolongerhaveacentralrepository MoreadaptedtothewaytheFreeSoftwarecommunitydevelops softwareandorganizes Alloweachdevelopertohavealocalfullhistoryoftheproject,to createlocalbranches.Makeseachdeveloper'sworkeasier. Peoplegetworkingcopiesfromotherpeople'sworkingcopies, andexchangechangesbetweenthemselves.Branchingand mergingismadeeasier. Makeiteasierfornewdeveloperstojoin,makingtheirown experimentswithouthavingtoapplyforrepositoryaccess.
19
Distributedsourcecontrolsystems(2)
Git
InitiallydesignedanddevelopedbyLinusTorvalds forLinuxkerneldevelopment Extremelypopularinthecommunity,andusedbymoreandmore projects(kernel,UBoot,Barebox,uClibc,GNOME,X.org,etc.) Outstandingperformance,inparticularinbigprojects http://en.wikipedia.org/wiki/Git_(software)
Mercurial
Anothersystem,createdwiththesamegoalsasGit. Usedbysomebigprojectstoo http://en.wikipedia.org/wiki/Mercurial
http://en.wikipedia.org/wiki/Version_control_systems#Distributed_revision_control
20
Softwaredevelopmenttools
Debuggingandanalysistools
Debuggers
21
GDB
TheGNUProjectDebugger http://www.gnu.org/software/gdb/ ThedebuggeronGNU/Linux, availableformostembeddedarchitectures. Supportedlanguages:C,C++,Pascal,ObjectiveC,Fortran, Ada... Consoleinterface(usefulforremotedebugging). Graphicalfrontendsavailable. Canbeusedtocontroltheexecutionofaprogram,set breakpointsorchangeinternalvariables.Youcanalsouseitto seewhataprogramwasdoingwhenitcrashed(byloadingits memoryimage,dumpedintoacorefile). Seealsohttp://en.wikipedia.org/wiki/Gdb
22
GDBcrashcourse
AfewusefulGDBcommands
breakfoobar putsabreakpointattheentryoffunctionfoobar() breakfoobar.c:42 putsabreakpointinfoobar.c,line42 printvarorprinttask>files[0].fd printsthevariablevar,oramorecomplicatedreference.GDBcan alsonicelydisplaystructureswithalltheirmembers continue continuetheexecution next continuetothenextline,steppingoverfunctioncalls step continuetothenextline,enteringintosubfunctions
23
GDBgraphicalfrontends
DDDDataDisplayDebugger http://www.gnu.org/software/ddd/ Apopulargraphicalfrontend,withadvanceddataplotting capabilities. GDB/Insight http://sourceware.org/insight/ FromtheGDBmaintainers. KDbg http://www.kdbg.org/ Anotherfrontend,fortheKDisplayEnvironment. IntegrationwithotherIDEs:Eclipse,Emacs,KDevelop,etc.
24
Introductiontosoftwaredevelopmenttools
Remotedebugging
25
Remotedebugging
Inanonembeddedenvironment,debuggingtakesplaceusinggdbor oneofitsfrontend. gdbhasdirectaccesstothebinaryandlibrariescompiledwith debuggingsymbols. However,inanembeddedcontext,thetargetplatformenvironmentis oftentoolimitedtoallowdirectdebuggingwithgdb(2.4MBonx86). Remotedebuggingispreferred gdbisusedonthedevelopmentworkstation,offeringallitsfeatures. gdbserverisusedonthetargetsystem(only100KBonarm).
gdb
gdbserver
26
Remotedebugging:architecture
Host Target gdbserver Serialor Ethernet connection
Binariesandlibraries withdebugging symbolsnotstripped Runningprogram withbinariesand librariesthat canbestripped
ARCHlinuxgdb
27
Remotedebugging:usage
Onthetarget,runaprogramthroughgdbserver. Programexecutionwillnotstartimmediately. gdbserverlocalhost:<port><executable><args> gdbserver/dev/ttyS0<executable><args> Otherwise,attachgdbservertoanalreadyrunningprogram: gdbserverattachlocalhost:<port><pid> Then,onthehost,runARCHlinuxgdbprogram, andusethefollowinggdbcommands: Toconnecttothetarget: gdb>targetremote<target>:<port> gdb>targetremote/dev/ttyS0 Totellgdbwheresharedlibrariesare: gdb>setsysroot<librarypath> (networking) (seriallink) (withoutlib/)
28
Postmortemanalysis
Whenanapplicationcrashesduetoasegmentationfaultandthe applicationwasnotundercontrolofadebugger,wegetno informationsaboutthecrash Fortunately,Linuxcangenerateacorefilethatcontainsthe imageoftheapplicationmemoryatthemomentofthecrash,and gdbcanusethiscorefiletoletusanalyzethestateofthe crashedapplication Onthetarget
Useulimitcunlimitedtoenablethegenerationofacore filewhenacrashoccurs
Onthehost
Afterthecrash,transferthecorefilefromthetargettothehost,and runARCHlinuxgdbccorefileapplicationbinary
29
Debuggingandanalysistools Memorycheckers
30
memcheck
http://hald.dnsalias.net/projects/memcheck/ GNUGPLtoolfordynamicmemorychecking Worksbyreplacingglibc'smemorymanagementfunctions byitsown. SupportsmostusefulCPUarchitectures.
31
DUMA
DetectUnintendedMemoryAccess http://duma.sourceforge.net/ ForkandreplacementforElectricFence Stopsyourprogramontheexactinstructionthatoverrunsor underrunsamalloc()memorybuffer. GDBwillthendisplaythesourcecodelinethatcausesthe bug. Worksbyusingthevirtualmemoryhardwaretocreatea redzoneattheborderofeachbuffertouchthat,andyour programstops. WorksonanyplatformsupportedbyLinux,whateverthe CPU(providedvirtualmemorysupportisavailable).
32
Valgrind(1)
http://valgrind.org/ GNUGPLSoftwaresuitefordebugging andprofilingprograms. Supportedplatforms:Linuxonx86,x86_64,ppc32,ppc64 armportinprogress(fundedbyLinaro). Others:compileyourprogramtotheseplatformstouseValgrind. Candetectmanymemorymanagementandthreadingbugs. Profiler:providesinformationhelpfultospeedupyourprogram andreduceitsmemoryusage. Themostpopulartoolforthisusage. Evenusedbyprojectswithhundredsofprogrammers.
33
Valgrind(2)
Canbeusedtorunanyprogram,withouttheneedto recompileit. Exampleusage valgrindleakcheck=yeslsla Worksbyaddingitsowninstrumentationtoyourcodeand thenrunninginonitsownvirtualx86(orppc)core. Significantlyslowsdownexecution,butstillfinefortesting! Moredetailsonhttp://valgrind.org/info/and http://valgrind.org/docs/manual/coregrind_core.html#howworks
34
Valgrindonotherplatforms
TocheckprogramsonnotsupportedyetCPUarchitectures Writeportablecode(alwaysuseful),andcompileitforx86. Then,debugyourapplicationwithValgrind. Oncebugsarefoundandfixed, youcangetbacktoyouroriginalplatform. ThisisparticularlyusefulwithuClinux, inwhichmemoryerrorsareverydifficulttofind.
35
Systemanalysis
36
strace
Systemcalltracer http://sourceforge.net/projects/strace/ AvailableonallGNU/Linuxsystems Canbebuiltbyyourcrosscompilingtoolchaingenerator. Allowstoseewhatanyofyourprocessesisdoing: accessingfiles,allocatingmemory... Oftensufficienttofindsimplebugs. Usage: strace<command> stracep<pid> Seemanstracefordetails. (startinganewprocess) (tracinganexistingprocess)
37
straceexampleoutput
>stracecatMakefile execve("/bin/cat",["cat","Makefile"],[/*38vars*/])=0 brk(0)=0x98b4000 access("/etc/ld.so.nohwcap",F_OK)=1ENOENT(Nosuchfileordirectory) mmap2(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,1,0)= 0xb7f85000 access("/etc/ld.so.preload",R_OK)=1ENOENT(Nosuchfileordirectory) open("/etc/ld.so.cache",O_RDONLY)=3 fstat64(3,{st_mode=S_IFREG|0644,st_size=111585,...})=0 mmap2(NULL,111585,PROT_READ,MAP_PRIVATE,3,0)=0xb7f69000 close(3)=0 access("/etc/ld.so.nohwcap",F_OK)=1ENOENT(Nosuchfileordirectory) open("/lib/tls/i686/cmov/libc.so.6",O_RDONLY)=3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320h\1\0004\0\0\0\344"...,512) =512 fstat64(3,{st_mode=S_IFREG|0755,st_size=1442180,...})=0 mmap2(NULL,1451632,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_DENYWRITE,3,0)= 0xb7e06000 mprotect(0xb7f62000,4096,PROT_NONE)=0 mmap2(0xb7f63000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED| MAP_DENYWRITE,3,0x15c)=0xb7f63000 mmap2(0xb7f66000,9840,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED| MAP_ANONYMOUS,1,0)=0xb7f66000 close(3)=0
38
ltrace
Atooltotracelibrarycallsusedbyaprogram andallthesignalsitreceives Veryusefulcomplementtostrace, whichshowsonlysystemcalls. Ofcourse,worksevenifyoudon'thavethesources Allowstofilterlibrarycallswithregularexpressions, orjustbyalistoffunctionnames. Manualpage:http://linux.die.net/man/1/ltrace Seehttp://en.wikipedia.org/wiki/Ltracefordetails
39
ltraceexampleoutput
ltraceneditindex.html sscanf(0x8274af1,0x8132618,0x8248640,0xbfaadfe8,0)=1 sprintf("const0","const%d",0)=7 strcmp("startScan","const0")=1 strcmp("ScanDistance","const0")=1 strcmp("const200","const0")=1 strcmp("$list_dialog_button","const0")=1 strcmp("$shell_cmd_status","const0")=1 strcmp("$read_status","const0")=1 strcmp("$search_end","const0")=1 strcmp("$string_dialog_button","const0")=1 strcmp("$rangeset_list","const0")=1 strcmp("$calltip_ID","const0")=1 ...
40
ltracesummary
Examplesummaryattheendoftheltraceoutput(coption)
Process17019detached %timesecondsusecs/callcallserrorssyscall 100.000.000050501set_thread_area 0.000.000000048read 0.000.000000044write 0.000.00000008063open 0.000.000000019close 0.000.00000001execve 0.000.000000022access 0.000.00000003brk 0.000.00000001munmap 0.000.00000001uname 0.000.00000001mprotect 0.000.000000019mmap2 0.000.00000005046stat64 0.000.000000018fstat64 100.000.000050288111total
41
Oprofile
http://oprofile.sourceforge.net Asystemwideprofilingtool CancollectstatisticslikethetopusersoftheCPU. Workswithouthavingthesources. Requiresakernelpatchtoaccessallfeatures, butisalreadyavailableinastandardkernel. Requiresmoreinvestigationtoseehowitworks. Ubuntu/Debianpackages: oprofile,oprofilegui
42
Callgrind/KCachegrind
Cachegrind/Callgrind:partoftheValgrindtoolsuite Collectsfunctioncallstatisticsandcallgraphs. Usefultoknowinwhichfunctionsmosttimeisspent. KCachegrind:http://kcachegrind.sourceforge.net/ AnamazingvisualizerforCachegrind/Callgrinddata. KCachegrindcanalsoimportdatafromotherprofilers (suchasOProfile),andfromprofilingoutputfromPython,Perl andPHP.
43
KCachegrindscreenshot
44
PracticallabRemotedebugging
TimetostartLab! Setupremotedebuggingtools onthetarget:strace,ltrace andgdbserver. Debugasimpleapplication runningonthetargetusing remotedebugging
45
DevelopingonWindows
46
DevelopingonWindows!?
UsingaGNU/Linuxworkstationistheeasiestwaytocreate softwareforGNU/LinuxorembeddedLinux Youusethesametoolsandenvironmentasallcommunity developersdo.Muchfewerissuesyouaretheonlyoneto face. Yougetfamiliarwiththesystem. Essentialforunderstandingissues. However,somedevelopershavenochoice: WindowsistheonlydesktopOSallowedintheircompany.
47
Cygwin
http://cygwin.com/ Linux(POSIX)likeenvironmentforWindows 2components: LinuxAPIemulationlayer:cygwin1.dll AcollectionoftoolsoriginallyfoundinGNU/Linux AllowstocompileandrunmanyGNU/LinuxprogramsonWindows: shells,compiler,httpservers,XWindow,GTK... Veryeasytoinstall.Canchoosewhichtoolstodownloadandinstall. ForembeddedLinuxsystemdevelopers: makesitpossibletouseGNUtoolchains(compiledforWindows) requiredtobuildLinuxbinaries(kernel,librariesorapplications).
48
Cygwinlimitations
CygwinisnotacompletesubstituteforarealGNU/Linuxsystem. AlmostalldevelopersworkonGNU/LinuxoronanotherUnix platform(typicallyBSD).Don'texpectthemtotestthattheir toolsbuildonWindowswithCygwin. ThenumberofCygwinusersisquitesmall. Youmaybethefirsttofaceorreportbuildingissuesonthis platformforagivencompilerortoolversion. Cygwinisveryslow. So,thebestsolutionistorunLinuxinsideWindows!
49
VMware
http://en.wikipedia.org/wiki/VMware License:proprietary CanrunaGNU/LinuxPCfromWindows, almostatthehostspeed. VMwarePlayerisnowavailablefreeofcharge. ManyFreeSoftwaresystemimagesavailablefordownload. Themostpopularsolutioninthecorporateworld.
50
VirtualBox
http://virtualbox.orgfromSunMicrosystems PCemulationsolutionavailable onbothWindowsandGNU/Linux 2licenses:
Proprietary:freeofcostforpersonaluseandevaluation. BinariesavailableforWindows.Fullfeatures. OpenSourceEdition(OSE):GPLlicense. Mostfeatures(exceptinparticularUSBsupport). NobinariesreleasedforWindowssofar(butpossible).
BasedonQEMU'scoreengine. PerformancesimilartothatofVMware. Seehttp://en.wikipedia.org/wiki/VirtualBox

51
Realtimein embeddedLinux systems

MichaelOpdenacker ThomasPetazzoni GillesChanteperdrix FreeElectrons
Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/realtime Corrections,suggestions,contributionsandtranslationsarewelcome!
1
RealTimeinEmbeddedLinuxSystems
Introduction
2
EmbeddedLinuxandrealtime
Duetoitsadvantages,Linuxandtheopensourcesoftwaresare moreandmorecommonlyusedinembeddedapplications However,someapplicationsalsohaverealtimeconstraints They,atthesametime,wantto
GetalltheniceadvantagesofLinux:hardwaresupport, componentsreuse,lowcost,etc. Gettheirrealtimeconstraintsmet
?
3
EmbeddedLinuxandrealtime
LinuxisanoperatingsystempartofthelargeUnixfamily Itwasoriginallydesignedasatimesharingsystem
Themaingoalistogetthebestthroughputfromtheavailable hardware,bymakingthebestpossibleusageofresources(CPU, memory,I/O) Timedeterminismisnottakenintoaccount
Ontheopposite,realtimeconstraintsimplytimedeterminism, evenattheexpenseoflowerglobalthroughput Bestthroughputandtimedeterminismarecontradictory requirements
4
Linuxandrealtimeapproaches
Overtime,twomajorapproacheshavebeentakentobringreal timerequirementsintoLinux Approach1
ImprovetheLinuxkernelitselfsothatitmatchesrealtime requirements,byprovidingboundedlatencies,realtimeAPIs,etc. ApproachtakenbythemainlineLinuxkernelandthe PREEMPT_RTproject.
Approach2
AddalayerbelowtheLinuxkernelthatwillhandlealltherealtime requirements,sothatthebehaviourofLinuxdoesn'taffectrealtime tasks. ApproachtakenbyRTLinux,RTAIandXenomai
5
Approach1 ImprovingthemainLinuxkernelwith PREEMPT_RT
6
Understandinglatency
Whendevelopingrealtimeapplicationswithasystemsuchas Linux,thetypicalscenarioisthefollowing
Aneventfromthephysicalworldhappensandgetsnotifiedtothe CPUbymeansofaninterrupt Theinterrupthandlerrecognizesandhandlestheevent,andthen wakeuptheuserspacetaskthatwillreacttothisevent Sometimelater,theuserspacetaskwillrunandbeabletoreactto thephysicalworldevent
Realtimeisaboutprovidingguaranteedworstcaselatenciesfor thisreactiontime,calledlatency
Somethingnotveryimportant... Yourimportant realtimetask!
Interrupt!
Linuxkernellatencycomponents
Waiting task
Makesthe taskrunnable interrupt latency Interrupt handler duration scheduler latency scheduler duration Interrupt handler Scheduler
Runningtask
Process context
Interrupt context
Schedulinglatency
kernellatency=interruptlatency+handlerduration +schedulerlatency+schedulerduration
8
Interruptlatency
Waiting task Runningtask
Schedulinglatency
Timeelapsedbeforeexecutingtheinterrupthandler
9
Sourceofinterruptlatency
Oneoftheconcurrencypreventionmechanismusedinthekernel isthespinlock Ithasseveralvariants,butoneofthevariantcommonlyusedto preventconcurrentaccessesbetweenaprocesscontextandan interruptcontextworksbydisablinginterrupts Criticalsectionsprotectedbyspinlocks,orothersectioninwhich interruptsareexplictlydisabledwilldelaythebeginningofthe executionoftheinterrupthandler
Thedurationofthesecriticalsectionsisunbounded
Otherpossiblesource:sharedinterrupts
Kernel code Criticalsection protectedbyspinlock Interrupt Interrupt handler
10
Interrupthandlerduration
Schedulinglatency
Timetakentoexecutetheinterrupthandler
11
Interrupthandlerimplementation
InLinux,manyinterrupthandlersaresplitintwoparts
Atophalf,startedbytheCPUassoonasinterruptare enabled.Itrunswiththeinterruptlinedisabledandis supposedtocompleteasquicklyaspossible. Abottomhalf,scheduledbythetophalf,whichstartsafterall pendingtophalfhavecompletedtheirexecution.
Therefore,forrealtimecriticalinterrupts,bottomhalf shouldn'tbeused:theirexecutionisdelayedbyallother interruptsinthesystem.

Tophalf Interrupt ACK Schedule Exit bottom half Otherinterrupt handlers... Bottomhalf Handle device data... Wakeup waiting tasks
Userspace...
12
Schedulerlatency
Schedulinglatency
Timeelapsedbeforeexecutingthescheduler
13
Understandingpreemption(1)
TheLinuxkernelisapreemptiveoperatingsystem Whenataskrunsinuserspacemodeandgetsinterruptedbyan interruption,iftheinterrupthandlerwakesupanothertask,this taskcanbescheduledassoonaswereturnfromtheinterrupt handler.
Interrupthandler WakesupTaskB
TaskA (runninginusermode)
TaskB (runninginusermode)
Interrupt
14
Understandingpreemption(2)
However,whentheinterruptcomeswhilethetaskisexecutinga systemcall,thissystemcallhastofinishbeforeanothertaskcan bescheduled. Bydefault,theLinuxkerneldoesnotdokernelpreemption. Thismeansthatthetimebeforewhichtheschedulerwillbe calledtoscheduleanothertaskisunbounded.
TaskA TaskA Interrupthandler (kernelmode) WakesupTaskB (kernelmode)
TaskA (usermode)
Returnfromsyscall
TaskB (usermode)
Systemcall
Interrupt
15
Schedulerduration
Schedulinglatency
Timetakentoexecutethescheduler andswitchtothenewtask.
16
Othernondeterministicmechanisms
Outsideofthecriticalpathdetailedpreviously,othernon deterministicmechanismsofLinuxcanaffecttheexecutiontime ofrealtimetasks Linuxishighlybasedonvirtualmemory,asprovidedbyanMMU, sothatmemoryisallocatedondemand.Wheneveranapplication accessescodeordataforthefirsttime,itisloadedondemand, whichcancreateshugedelays. ManyClibraryservicesorkernelservicesarenotdesignedwith realtimeconstraintsinmind.
17
Priorityinversion
Aprocesswithalowprioritymightholdalockneededbyahigher priorityprocess,effectivelyreducingthepriorityofthisprocess. ThingscanbeevenworseifamiddlepriorityprocessusestheCPU.
Priority
Triestoget thesame lock
waits preempted
Acquires alock
Time
18
Interrupthandlerpriority
InLinux,interrupthandlersareexecuteddirectlybytheCPU interruptmechanisms,andnotundercontroloftheLinux scheduler.Therefore,allinterrupthandlershaveanhigher prioritythanalltasksrunningonthesystem.
Anyinterrupt Anyinterrupt...
topprioritytask
topprioritytask
19
ThePREEMPT_RTproject
LongtermprojectleadbyLinuxkerneldevelopersIngoMolnar, ThomasGleixnerandStevenRostedt
https://rt.wiki.kernel.org
ThegoalistograduallyimprovetheLinuxkernelregardingreal timerequirementsandtogettheseimprovementsmergedinto themainlinekernel

PREEMPT_RTdevelopmentworksverycloselywiththemainline development
Manyoftheimprovementsdesigned,developedanddebugged insidePREEMPT_RTovertheyearsarenowpartofthemainline Linuxkernel

TheprojectisalongtermbranchoftheLinuxkernelthatultimately shoulddisappearaseverythingwillhavebeenmerged
20
Improvementsinthemainlinekernel
Comingfromthe PREEMPT_RTproject Sincethebeginningof2.6
O(1)scheduler Kernelpreemption BetterPOSIXrealtimeAPI support
Since2.6.30
Threadedinterrupts
Since2.6.33
Spinlockannotations
Since2.6.18
Priorityinheritancesupport formutexes
Since2.6.21
Highresolutiontimers
21
NewpreemptionoptionsinLinux2.6
2newpreemptionmodelsofferedbystandardLinux2.6:
22
1 option:noforcedpreemption
CONFIG_PREEMPT_NONE Kernelcode(interrupts,exceptions,systemcalls)neverpreempted. Defaultbehaviorinstandardkernels. Bestforsystemsmakingintensecomputations, onwhichoverallthroughputiskey. BesttoreducetaskswitchingtomaximizeCPUandcacheusage (byreducingcontextswitching). StillbenefitsfromsomeLinux2.6improvements: O(1)scheduler,increasedmultiprocessorsafety(workonRT preemptionwasusefultoidentifyhardtofindSMPbugs). Canalsobenefitfromalowertimerfrequency (100Hzinsteadof250or1000).
st
23
2 option:voluntarykernelpreemption
CONFIG_PREEMPT_VOLUNTARY Kernelcodecanpreemptitself Typicallyfordesktopsystems,forquickerapplicationreactionto userinput. Addsexplicitreschedulingpointsthroughoutkernelcode. Minorimpactonthroughput.
nd
24
3 option:preemptiblekernel
CONFIG_PREEMPT Mostkernelcodecanbeinvoluntarilypreemptedatanytime. Whenaprocessbecomesrunnable,nomoreneedtowaitfor kernelcode(typicallyasystemcall)toreturnbeforerunningthe scheduler. Exception:kernelcriticalsections(holdingspinlocks),buta reschedulingpointoccurswhenexitingtheoutercriticalsection, incaseapreemptionopportunitywouldhavebeensignaledwhile inthecriticalsection. Typicallyfordesktoporembeddedsystemswithlatency requirementsinthemillisecondsrange. Stillarelativelyminorimpactonthroughput.
25
rd
Priorityinheritance
Oneclassicalsolutiontothepriorityinversionproblemiscalled priorityinheritance
Theideaisthatwhenataskofalowpriorityholdsalockrequested byanhigherprioritytask,thepriorityofthefirsttaskgetstemporarly raisedtothepriorityofthesecondtask:ithasinheriteditspriority.
InLinux,since2.6.18,mutexessupportpriorityinheritance Inuserspace,priorityinheritancemustbeexplictlyenabledona permutexbasis.
26
Highresolutiontimers
Theresolutionofthetimersusedtobeboundtotheresolutionof theregularsystemtick
Usually100Hzor250Hz,dependingonthearchitectureandthe configuration Aresolutionofonly10msor4ms. Increasingtheregularsystemtickfrequencyisnotanoptionasit wouldconsumetoomuchresources
Thehighresolutiontimersinfrastructure,mergedin2.6.21, allowstousetheavailablehardwaretimerstoprograminterrupts attherightmoment.

Hardwaretimersaremultiplexed,sothatasinglehardwaretimeris sufficienttohandlealargenumberofsoftwareprogrammedtimers. UsabledirectlyfromuserspaceusingtheusualtimerAPIs
27
Threadedinterrupts
Tosolvetheinterruptinversionproblem,PREEMPT_RThas introducedtheconceptofthreadedinterrupts Theinterrupthandlersruninnormalkernelthreads,sothatthe prioritiesofthedifferentinterrupthandlerscanbeconfigured Therealinterrupthandler,asexecutedbytheCPU,isonlyin chargeofmaskingtheinterruptandwakingupthecorresponding thread Theideaofthreadedinterruptsalsoallowstousesleeping spinlocks(seelater) Mergedsince2.6.30,theconversionofinterrupthandlersto threadedinterruptsisnotautomatic:driversmustbemodified InPREEMPT_RT,allinterrupthandlersareswitchedtothreaded interrupts
28
PREEMPT_RTspecifics
29
CONFIG_PREEMPT_RT(1)
ThePREEMPT_RTpatchaddsanewlevelofpreemption, calledCONFIG_PREEMPT_RT Thislevelofpreemptionreplacesallkernelspinlocksbymutexes (orsocalledsleepingspinlocks)
Insteadofprovidingmutualexclusionbydisablinginterruptsand preemption,theyarejustnormallocks:whencontentionhappens, theprocessisblockedandanotheroneisselectedbythescheduler Workswellwiththreadedinterrupts,sincethreadscanblock,while usualinterrupthandlerscouldnot Somecore,carefullycontrolled,kernelspinlocksremainasnormal spinlocks
30
CONFIG_PREEMPT_RT(2)
WithCONFIG_PREEMPT_RT,virtuallyallkernelcodebecomes preemptible
Aninterruptcanoccuratanytime,whenreturningfromtheinterrupt handler,thewokenupprocesscanstartimmediately
ThisisthelastbigpartofPREEMPT_RTthatisn'tfullyinthe mainlinekernelyet
Partofithasbeenmergedin2.6.33:thespinlockannotations.The spinlocksthatmustremainasspinningspinlocksarenow differentiatedfromspinlocksthatcanbeconvertedtosleeping spinlocks.ThishasreducedalotthePREEMPT_RTpatchsize!
31
Threadedinterrupts
ThemechanismofthreadedinterruptsinPREEMPT_RTisstill differentfromtheonemergedinmainline InPREEMPT_RT,allinterrupthandlersareunconditionally convertedtothreadedinterrupts. Thisisatemporarysolution,untilinterestingdriversinmainline getgraduallyconvertedtothenewthreadedinterruptAPIthat hasbeenmergedin2.6.30.
32
SettingupPREEMPT_RT
33
PREEMPT_RTsetup(1)
PREEMPT_RTisdeliveredasapatchagainstthemainline kernel
Besttohaveaboardsupportedbythemainlinekernel,otherwise thePREEMPT_RTpatchmaynotapplyandmayrequiresome adaptations
Manyofficialkernelreleasesaresupported,butnotall.For example,2.6.31and2.6.33aresupported,butnot2.6.32. Quicksetup

Downloadandextractmainlinekernel DownloadthecorrespondingPREEMPT_RTpatch Applyittothemainlinekerneltree
34
PREEMPT_RTsetup(2)
Inthekernelconfiguration,besuretoenable
CONFIG_PREEMPT_RT Highresolutiontimers
Compileyourkernel,andboot YouarenowrunningtherealtimeLinuxkernel Ofcourse,somesystemconfigurationremainstobedone,in particularsettingappropriateprioritiestotheinterruptthreads, whichdependonyourapplication.
35
Realtimeapplicationdevelopment
36
Developmentandcompilation
Nospeciallibraryisneeded,thePOSIXrealtimeAPIispartof thestandardClibrary TheglibcoreglibcClibrariesarerecommended,asthesupport ofsomerealtimefeaturesisnotavailableyetinuClibc
PriorityinheritancemutexesorNPTLonsomearchitectures,for example
Compileaprogram
ARCHlinuxgccomyprogmyprog.clrt
TogetthedocumentationofthePOSIXAPI
Installthemanpagesposixdevpackage Runmanfunctioname
37
Process,thread?
Confusionaboutthetermsprocess,threadandtask InUnix,aprocessiscreatedusingfork()andiscomposedof
Anaddressspace,whichcontainstheprogramcode,data,stack, sharedlibraries,etc. Onethread,thatstartsexecutingthemain()function. Uponcreation,aprocesscontainsonethread
Additionalthreadscanbecreatedinsideanexistingprocess, usingpthread_create()
Theyruninthesameaddressspaceastheinitialthreadofthe process Theystartexecutingafunctionpassedasargumentto pthread_create()
38
Process,thread:kernelpointofview
Thekernelrepresentseachthreadrunninginthesystembya structureoftypetask_struct Fromaschedulingpointofview,itmakesnodifferencebetween theinitialthreadofaprocessandalladditionalthreadscreated dynamicallyusingpthread_create()
Thread A Addressspace
Processafterfork()
Thread A
Thread B
Addressspace
Sameprocessafterpthread_create() 39
Creatingthreads
LinuxsupportthePOSIXthreadAPI Tocreateanewthread
pthread_create(pthread_t*thread, pthread_attr_t*attr, void*(*routine)(*void*), void*arg); Thenewthreadwillruninthesameaddressspace,butwillbe scheduledindependently
Exitingfromathread
pthread_exit(void*value_ptr);
Waitingforathreadtermination
pthread_join(pthread_t*thread,void**value_ptr);
40
Schedulingclasses(1)
TheLinuxkernelschedulersupportdifferentschedulingclasses Thedefaultclass,inwhichprocessesarestartedbydefaultisa timesharingclass
Allprocesses,regardlessoftheirpriority,getsomeCPUtime TheproportionofCPUtimetheygetisdynamicandaffectedbythe nicevalue,whichrangesfrom20(highest)to19(lowest).Canbe setusingtheniceorrenicecommands
TherealtimeclassesSCHED_FIFOandSCHED_RR
ThehighestpriorityprocessgetsalltheCPUtime,untilitblocks. InSCHED_RR,roundrobinschedulingbetweentheprocessesof thesamepriority.Allmustblockbeforelowerpriorityprocessesget CPUtime. Prioritiesrangingfrom0(lowest)to99(highest)
41
Anexistingprogramcanbestartedinaspecificschedulingclass withaspecificpriorityusingthechrtcommandlinetool
Example:chrtf99./myprog
Thesched_setscheduler()APIcanbeusedtochangethe schedulingclassandpriorityofaprocess
intsched_setscheduler(pid_tpid,intpolicy, conststructsched_param*param); policycanbeSCHED_OTHER,SCHED_FIFO,SCHED_RR,etc. paramisastructurecontainingthepriority
42
Theprioritycanbesetonaperthreadbasiswhenathreadis created:
structsched_paramparm; pthread_attr_tattr; pthread_attr_init(&attr); pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED); pthread_attr_setschedpolicy(&attr,SCHED_FIFO); parm.sched_priority=42; pthread_attr_setschedparam(&attr,&parm);
Thenthethreadcanbecreatedusingpthread_create(), passingtheattrstructure. Severalotherattributescanbedefinedthisway:stacksize,etc.

43
Memorylocking
Inordertosolvethenondeterminismintroducedbyvirtual memory,memorycanbelocked
Guaranteethatthesystemwillkeepitallocated Guaranteethatthesystemhaspreloadedeverythingintomemory
mlockall(MCL_CURRENT|MCL_FUTURE);
Locksallthememoryofthecurrentaddressspace,forcurrently mappedpagesandpagesmappedinthefuture
Other,lessusefulpartsoftheAPI:munlockall,mock, munlock. Watchoutfornoncurrentlymappedpages

Stackpages Dynamicallyallocatedmemory
44
Mutexes
Allowsmutualexclusionbetweentwothreadsinthesame addressspace
Initialization/destruction pthread_mutex_init(pthread_mutex_t*mutex,const pthread_mutexattr_t*mutexattr); pthread_mutex_destroy(pthread_mutex_t*mutex); Lock/unlock pthread_mutex_lock(pthread_mutex_t*mutex); pthread_mutex_unlock(pthread_mutex_t*mutex);
Priorityinheritancemustexplictlybeactivated
pthread_mutexattr_tattr; pthread_mutexattr_init(&attr); pthread_mutexattr_getprotocol (&attr,PTHREAD_PRIO_INHERIT);
45
Timers
timer_create(clockid_tclockid, structsigevent*evp, timer_t*timerid)
Createatimer.clockidisusuallyCLOCK_MONOTONIC. sigeventdefineswhathappensupontimerexpiration:senda signalorstartafunctioninanewthread.timeridisthereturned timeridentifier.
timer_settime(timer_ttimerid,intflags, structitimerspec*newvalue, structitimerspec*oldvalue)

Configuresthetimerforexpirationatagiventime.
timer_delete(timer_ttimerid),deleteatimer clock_getres(),gettheresolutionofaclock Otherfunctions:timer_getoverrun(),timer_gettime()

46
Signals
Signalsareanasynchronousnotificationmechanism Notificationoccurseither
Bythecallofasignalhandler.Becarefulwiththelimitationsof signalhandlers! Bybeingunblockedfromthesigwait(),sigtimedwait()or sigwaitinfo()functions.Usuallybetter.
Signalbehaviourcanbeconfiguredusingsigaction() Maskofblockedsignalscanbechangedwith pthread_sigmask() Deliveryofasignalusingpthread_kill()ortgkill() AllsignalsbetweenSIGRTMINandSIGRTMAX,32signalsunder Linux.

47
Interprocesscommunication
Semaphores
Usablebetweendifferentprocessesusingnamedsemaphores sem_open(),sem_close(),sem_unlink(),sem_init(), sem_destroy(),sem_wait(),sem_post(),etc.
Messagequeues
Allowsprocessestoexchangedataintheformofmessages. mq_open(),mq_close(),mq_unlink(),mq_send(), mq_receive(),etc.
Sharedmemory
Allowsprocessestocommunicatebysharingasegmentofmemory shm_open(),ftruncate(),mmap(),munmap(), close(),shm_unlink()
48
Debuggingrealtimelatencies
49
ftraceKernelfunctiontracer
Newinfrastructurethatcanbeusedfordebuggingoranalyzing latenciesandperformanceissuesinthekernel. DevelopedbyStevenRostedt.Mergedin2.6.27. Forearlierkernels,canbefoundfromthertpreemptpatches. VerywelldocumentedinDocumentation/ftrace.txt Negligibleoverheadwhentracingisnotenabledatruntime. Canbeusedtotraceanykernelfunction! SeeourvideoofSteven'stutorialatOLS2008: http://freeelectrons.com/community/videos/conferences/
50
Usingftrace
Tracinginformationavailablethroughthedebugfsvirtualfs (CONFIG_DEBUG_FSintheKernelHackingsection) Mountthisfilesystemasfollows: mounttdebugfsnodev/debug Whentracingisenabled(seethenextslides), tracinginformationisavailablein/debug/tracing. Checkavailabletracers in/debug/tracing/available_tracers
51
Schedulinglatencytracer
CONFIG_SCHED_TRACER(KernelHackingsection) Maximumrecordedtimebetweenwakingupatopprioritytask anditsschedulingonaCPU,expressedins. Checkthatwakeupislistedin /debug/tracing/available_tracers Toselect,resetandenablethistracer: echowakeup>/debug/tracing/current_tracer echo0>/debug/tracing/tracing_max_latency echo1>/debug/tracing/tracing_enabled Letyoursystemrun,inparticularrealtimetasks. Example:chrtf5sleep1 Disabletracing: echo0>/debug/tracing/tracing_enabled Readthemaximumrecordedlatencyandthecorrespondingtrace: cat/debug/tracing/tracing_max_latency
52
Usefulreading
AboutrealtimesupportinthestandardLinuxkernel InternalsoftheRTPatch,StevenRostedt,RedHat,June2007 http://www.kernel.org/doc/ols/2007/ols2007v2pages161172.pdf Definitelyworthreading. TheRealTimeLinuxWiki:http://rt.wiki.kernel.org TheWikiWebfortheCONFIG_PREEMPT_RTcommunity, andrealtimeLinuxingeneral. Containsniceandusefuldocuments! Seealsoourbookspage.
53
Approach2 RealtimeextensionstotheLinuxkernel
54
Linuxrealtimeextensions
Threegenerations RTLinux RTAI Xenomai
realtime realtime tasks tasks
Acommonprinciple Addaextralayerbetweenthe hardwareandtheLinuxkernel, tomanagerealtimetasks separately.

Linux kernel
Microkernel
Hardware
55
RTLinux
FirstrealtimeextensionforLinux,createdbyVictorYodaiken. Nice,buttheauthorfiledasoftwarepatentcoveringtheadditionofreal timesupporttogeneraloperatingsystemsasimplementedinRTLinux! ItsOpenPatentLicensedrewmanydevelopersawayandfrightened users.CommunityprojectslikeRTAIandXenomainowattractmost developersandusers. February,2007:RTLinuxrightssoldtoWindRiver. NowsupportedbyWindRiverasRealTimeCoreforWindRiverLinux. FreeversionstilladvertisedbyWindRiveronhttp://www.rtlinuxfree.com, butnolongeracommunityproject.
56
RTAI
http://www.rtai.org/RealTimeApplicationInterfaceforLinux Createdin1999,byProf.PaoloMontegazza(longtime contributortoRTLinux),DipartimentodiIngegneria AerospazialePolitecnicodiMilano(DIAPM). Communityproject.Significantuserbase. AttractedcontributorsfrustratedbytheRTLinuxlegalissues. Onlyreallyactivelymaintainedonx86 MayofferslightlybetterlatenciesthanXenomai,atthe expenseofalessmaintainableandlessportablecodebase SinceRTAIisnotreallymaintainedonARMandother embeddedarchitectures,ourpresentationisfocusedon Xenomai.
57
Xenomaiproject
http://www.xenomai.org/ Startedin2001asaprojectaimingatemulating traditionalRTOS. Initialgoals:facilitatetheportingofprogramstoGNU/Linux. InitiallyrelatedtotheRTAIproject(astheRTAI/fusion branch),nowindependent. SkinsmimickingtheAPIsoftraditional RTOSsuchasVxWorks,pSOS+,andVRTXsaaswellasthe POSIXAPI,andanativeAPI. Aimsatworkingbothasacokernelandontopof PREEMPT_RTintheupcoming3.0branch. Willneverbemergedinthemainlinekernel.
58
Xenomaiarchitecture
Linuxapplication glibc VxWorksapplication glibc Xenomai libvxworks POSIXapplication glibc Xenomai libpthread_rt
Systemcalls
VFS Memory
Network ...
XenomaiRTOS (nucleus)
Linux kernelspace
Piecesadded byXenomai Xenomai skins
AdeosIPipe
59
TheAdeosinterruptpipelineabstraction
FromAdeospointofview,guestOSesareprioritizeddomains. Foreachevent(interrupts,exceptions,syscalls,etc...),the variousdomainsmayhandletheeventorpassitdownthe pipeline.
60
Adeosvirtualizedinterruptsdisabling
Eachdomainmaybestalled,meaningthatitdoesnotaccept interrupts. Hardwareinterrupts arenotdisabled however(except forthedomain leadingthepipeline), insteadtheinterrupts receivedduringthat timeareloggedand replayedwhenthe domainisunstalled.
61
Adeosadditionalfeatures
TheAdeosIpipepatchimplementadditionalfeatures,essential fortheimplementationoftheXenomairealtimeextension:
Disablesondemandmappingofkernelspacevmalloc/ioremap areas. Disablescopyonwritewhenrealtimeprocessesareforking. AllowsubscribingtoeventallowingtofollowprogressoftheLinux kernel,suchasLinuxsystemcalls,contextswitches,process destructions,POSIXsignals,FPUfaults. OntheARMarchitectures,integratestheFCSEpatch,whichallows toreducethelatencyinducedbycacheflushesduringcontext switches.
62
Xenomaifeatures
Factoredrealtimecorewithskinsimplementingvariousrealtime APIs Seamlesssupportforhardrealtimeinuserspace Nosecondclasscitizen,allportsareequivalentfeaturewise Xenomaisupportisasmuchaspossibleindependentfromthe Linuxkernelversion(backwardandforwardcompatiblewhen reasonable) EachXenomaibranchhasastableuser/kernelABI Timersystembasedonhardwarehighresolutiontimers Perskintimebasewhichmaybeperiodic RTDMskinallowingtowriterealtimedrivers
63
Xenomaiuserspacerealtimesupport.
Xenomaisupportsrealtimeinuserspaceon5architectures, including32and64bitsvariants. Twomodesaredefinedforathread
theprimarymode,wherethethreadishandledbyXenomai scheduler thesecondarymode,whenitishandledbyLinuxscheduler.
ThankstotheservicesoftheAdeosIpipeservice,Xenomai systemcallsaredefined.
Athreadmigratesfromsecondarymodetoprimarymodewhen suchasystemcallisissued ItmigratesfromprimarymodetosecondarymodewhenaLinux systemcallisissued,ortohandlegracefullyexceptionalevents suchasexceptionsorLinuxsignals.
64
LifeofaXenomaiapplication
XenomaiapplicationsarestartedlikenormalLinuxprocesses, theyareinitiallyhandledbytheLinuxschedulerandhaveaccess toallLinuxservices Aftertheirinitialization,theydeclarethemselvesasrealtime application,whichmigratesthemtoprimarymode.Inthismode:
TheyarescheduleddirectlybytheXenomaischeduler,sothey havetherealtimepropertiesofferedbyXenomai Theydon'thaveaccesstoanyLinuxservice,otherwisetheyget migratedbacktosecondarymodeandloosesallrealtime properties TheycanonlyusedevicedriversthatareimplementedinXenomai, nottheonesoftheLinuxkernel
NeedtoimplementdevicedriversinXenomai,andtosplitreal timeandnonrealtimepartsofyourapplications.
65
RealTimeDriverModel(RTDM)
Anapproachtounifytheinterfacesfordevelopingdevicedrivers andassociatedapplicationsunderrealtimeLinux
AnAPIverysimilartothenativeLinuxkerneldriverAPI
Allowsthedevelopment,inkernelspace,of
Characterstyledevicedrivers Networkstyledevicedrivers
Seethewhitepaperon
http://www.xenomai.org/documentation/xenomai2.4/pdf/RTDMandApplications.pdf
CurrentnotableRTDMbaseddrivers:
Serialportcontrollers; RTnetUDP/IPstack; RTsocketCAN,driversforCANcontrollers; Analogy,forkoftheComedyproject,driversforacquisitioncards.
66
SettingupXenomai
67
HowtobuildXenomai
DownloadXenomaisourcesat http://download.gna.org/xenomai/stable/ DownloadoneoftheLinuxversionssupportedbythisrelease (seeksrc/arch/<arch>/patches/) Sinceversion2.0,splitkernel/userbuildingmodel. Kernelusesascriptcalledscript/preparekernel.shwhich integratesXenomaikernelspacesupportintheLinuxsources. Runthekernelconfigurationmenu.
68
LinuxoptionsforXenomaiconfiguration
69
Xenomaiuserspacesupport
Userspacelibrariesarecompiledusingthetraditionalautotools ./configuretarget=armlinux&&make&& makeDESTDIR=/your/rootfs/install Thexenoconfigscript,installedwheninstallingXenomaiuser spacesupporthelpsyoucompilingyourownprograms. SeeXenomai'sexamplesdirectory. InstallationdetailsmaybefoundintheREADME.INSTALLguide. ForanintroductiononprogrammingwiththenativeAPI,see:
http://www.xenomai.org/documentation/branches/v2.3.x/pdf/NativeAPITourrevC.pdf
ForanintroductiononprogrammingwiththePOSIXAPI,see:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai
70
DevelopingapplicationsonXenomai
71
ThePOSIXskin
ThePOSIXskinallowstorecompilewithoutchangesatraditional POSIXapplicationsothatinsteadofusingLinuxrealtime services,itusesXenomaiservices
Clocksandtimers,conditionvariables,messagequeues,mutexes, semaphores,sharedmemory,signals,threadmanagement GoodforexistingcodeorprogrammersfamiliarwiththePOSIXAPI
Ofcourse,iftheapplicationusesanyLinuxservicethatisn't availableinXenomai,itwillswitchbacktosecondarymode TolinkanapplicationagainstthePOSIXskin

DESTDIR=/path/to/xenomai/ exportDESTDIR CFL=`$DESTDIR/bin/xenoconfigposixcflags` LDF=`$DESTDIR/bin/xenoconfigposixldflags` ARCHgcc$CFLorttestrttest.c$LDF
72
Communicationwithanormaltask
IfaXenomairealtimeapplicationusingthePOSIXskinwishesto communicatewithaseparatenonrealtimeapplication,itmust usethertipcmechanism IntheXenomaiapplication,createanIPCPROTO_XDDPsocket
socket(AF_RTIPC,SOCK_DGRAM,IPCPROTO_XDDP); setsockopt(s,SOL_RTIPC,XDDP_SETLOCALPOOL,&poolsz, sizeof(poolsz)); memset(&saddr,0,sizeof(saddr)); saddr.sipc_family=AF_RTIPC; saddr.sipc_port=MYAPPIDENTIFIER; ret=bind(s,(structsockaddr*)&saddr,sizeof(saddr));
AndthenthenormalsocketAPIsendto()/recvfrom()
IntheLinuxapplication
Open/dev/rtpX,whereXistheXDDPport Useread()andwrite()
73
ThenativeAPI(1)
AXenomaispecificAPIfordevelopingrealtimetasks
Usablebothinuserspaceandkernelspace.Developmentoftasks inuserspaceisthepreferredway. MorecoherentandmoreflexibleAPIthanthePOSIXAPI.Easierto learnandunderstand.Certainlythewaytogofornewapplications.
Applicationsshouldinclude<native/service.h>,where servicecanbealarm,buffer,cond,event,heap, intr,misc,mutex,pipe,queue,sem,task,timer Tocompileapplications:

DESTDIR=/path/to/xenomai/ exportDESTDIR CFL=`$DESTDIR/bin/xenoconfigxenocflags` LDF=`$DESTDIR/bin/xenoconfigxenoldflags` ARCHgcc$CFLorttestrttest.c$LDFlnative
74
ThenativeAPI(2)
Taskmanagementservices
rt_task_create(),rt_task_start(), rt_task_suspend(),rt_task_resume(), rt_task_delete(),rt_task_join(),etc.
Countingsemaphoreservices
rt_sem_create(),rt_sem_delete(),rt_sem_p(), rt_sem_v(),etc.
Messagequeueservices
rt_queue_create(),rt_queue_delete(), rt_queue_alloc(),rt_queue_free(), rt_queue_send(),rt_queue_receive(),etc.
Mutexservices
rt_mutex_create(),rt_mutex_delete(), rt_mutex_acquire(),rt_mutex_release(),etc.
75
ThenativeAPI(3)
Alarmservices
rt_alarm_create(),rt_alarm_delete(), rt_alarm_start(),rt_alarm_stop(), rt_alarm_wait(),etc.
Memoryheapservices
Allowstosharememorybetweenprocessesand/ortopreallocate apoolofmemory rt_heap_create(),rt_heap_delete(), rt_heap_alloc(),rt_heap_bind()
Conditionvariableservices
rt_cond_create(),rt_cond_delete(), rt_cond_signal(),rt_cond_broadcast(), rt_cond_wait(),etc.
76
Xenomaiandnormaltaskcommunication
Usingrt_pipes InthenativeXenomaiapplication,usethePipeAPI
rt_pipe_create(),rt_pipe_delete(), rt_pipe_receive(),rt_pipe_send(), rt_pipe_alloc(),rt_pipe_free()
InthenormalLinuxapplication
Openthecorresponding/dev/rtpXfile,theminorisspecifiedat rt_pipe_create()time Then,justread()andwrite()totheopenedfile
Xenomaiapplication
Usesthert_pipe_*()API
Linuxapplication
open(/dev/rtpX)
77
Realtimeapproaches
ThefollowingtableisPaulMacKenney'ssummaryofhisown articledescribingthevariousapproachesforrealtimeonLinux:
Approach VanillaLinux PREEMPT NestedOS (cokernel) DualOS/DualCore (ASMP) Quality 10sofms allservices 100sofus Schd,Int ~10us RTOSsvcs <1us RTOSsvcs Inspection All preemptor irqdisable RTOS, hwirqdisable API POSIX+RT Complexity N/A Fault isolation None None Good HW/SW Configs All All All
RTOS preemptandirq disable(most intsinprocessctx), 10sofus (mostlydrivers) PREEMPT_RT Schd,Int POSIX+RT MigrationbetweenOSes ?us RTOS, RTOS(can RTOSsvcs hwirqdisable bePOSIXRT) ?us Sched, RTOSsvcs RTOSsvcs MigrationwithinOS POSIX+RT (additionsinblue)
POSIX+RT N/A RTOS(can bePOSIXRT) Dualenv. RTOS(can bePOSIXRT) Dualenv.
Excellent Specialized All(except some drivers) All All?
"Modest"patch (carefultuning) None Dualenv.(easy mix) OK Smallpatch None
Fullstoryathttp://lwn.net/Articles/143323
78
Books
BuildingEmbeddedLinuxSystems,O'Reilly ByKarimYaghmour,JonMasters, GiladBenYossef,PhilippeGerumandothers (includingMichaelOpdenacker),August2008 AnicecoverageofXenomai(PhilippeGerum) andtheRTpatch(StevenRostedt) http://oreilly.com/catalog/9780596529680/
79
Organizations
http://www.realtimelinuxfoundation.org/ CommunityportalforrealtimeLinux. Organizesayearlyworkshop. http://www.osadl.org OpenSourceAutomationDevelopmentLab(OSADL) CreatedasanequivalentofOSDLformachineandplantcontrol systems.MembercompaniesareGermansofar(ThomasGleixner isonboard).Oneoftheirgoalsistosupportsthedevelopmentof RTpreemptpatchesinthemainlineLinuxkernel(HOWTOs,live CD,patches).
80
Hotpluggingwithudev
Hotplugging withudev
MichaelOpdenacker FreeElectrons
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/udev Corrections,suggestions,contributionsandtranslationsarewelcome!
/devissuesandlimitations
OnRedHat9,18000entriesin/dev! Allentriesforallpossibledevices hadtobecreatedatsysteminstallation. Neededanauthoritytoassignmajornumbers http://lanana.org/:LinuxAssignedNamesandNumbers Authority Notenoughnumbersin2.4,limitsextendedin2.6. Userspaceneitherknewwhatdeviceswerepresentinthe system,norwhichrealdevicecorrespondedtoeach/dev entry.
2
Theudevsolution
TakesadvantageofsysfsintroducedbyLinux2.6. CreatedbyGregKroahHartman,ahugecontributor. Otherkeycontributors:KaySievers,DanStekloff. Entirelyinuserspace. Automaticallycreates/removesdeviceentries in/dev/accordingtoinserted/removeddevices. Majorandminordevicetransmittedbythekernel. Requiresnochangetodrivercode. Fast:writteninC Smallsize:udevdversion108:61KBinUbuntu7.04
3
Startingudev(1)
Attheverybeginningofuserspacestartup, mountthe/dev/directoryasatmpfsfilesystem: sudomountttmpfsudev/dev /dev/ispopulatedwithstaticdevicesavailablein /lib/udev/devices/:
Ubuntu6.10example:
crw1rootroot5,12007013104:18console lrwxrwxrwx1rootroot112007013104:18core>/proc/kcore lrwxrwxrwx1rootroot132007013104:18fd>/proc/self/fd crwr1rootkmem1,22007013104:18kmem brw1rootroot7,02007013104:18loop0 lrwxrwxrwx1rootroot132007013104:18MAKEDEV>/sbin/MAKEDEV drwxrxrx2rootroot40962007013104:18net crw1rootroot1,32007013104:18null crw1rootroot108,02007013104:18ppp drwxrxrx2rootroot40962006101614:39pts drwxrxrx2rootroot40962006101614:39shm lrwxrwxrwx1rootroot242007013104:18sndstat>/proc/asound/oss/sndstat lrwxrwxrwx1rootroot152007013104:18stderr>/proc/self/fd/2 lrwxrwxrwx1rootroot152007013104:18stdin>/proc/self/fd/0 lrwxrwxrwx1rootroot152007013104:18stdout>/proc/self/fd/1
4
Startingudev(2)
Theudevddaemonisstarted. Itlistenstoueventsfromthedrivercore, whicharesentwheneverdevicesareinsertedorremoved. Theudevddaemonreadsandparsesalltherulesfoundin /etc/udev/rules.d/ andkeepstheminmemory. Wheneverrulesareadded,removedormodified, udevdreceivesaninotifyeventandupdatesits rulesetinmemory. Whenaneventisreceived,udevdstartsaprocessto: trytomatchtheeventagainstudevrules, create/removedevicefiles, andrunprograms(toload/removeadriver,tonotifyuserspace...)
Theinotifymechanismlets userspaceprogramssubscribe tonotificationsoffilesystem changes.Possibilitytowatch individualfilesordirectories.
5
ueventmessageexample
ExampleinsertingaUSBmouse
recv(4, //socketid "add@/class/input/input9/mouse2\0 //message ACTION=add\0 //actiontype DEVPATH=/class/input/input9/mouse2\0 //pathin/sys SUBSYSTEM=input\0 //subsystem(class) SEQNUM=1064\0 //sequencenumber PHYSDEVPATH=/devices/pci0000:00/0000:00:1d.1/usb2/22/22:1.0\0 //devicepathin/sys PHYSDEVBUS=usb\0 //bus PHYSDEVDRIVER=usbhid\0 //driver MAJOR=13\0 //majornumber MINOR=34\0", //minornumber 2048, //messagebuffersize 0) //flags =221 //actualmessagesize
6
udevrules
Whenaudevrulematchingeventinformationisfound, itcanbeused: Todefinethenameandpathofadevicefile. Todefinetheowner,groupandpermissionsofadevicefile. Toexecuteaspecifiedprogram. Rulefilesareprocessedinlexicalorder.
7
udevnamingcapabilities
Devicenamescanbedefined fromalabelorserialnumber, fromabusdevicenumber, fromalocationonthebustopology, fromakernelname, fromtheoutputofaprogram. Seehttp://www.reactivated.net/writing_udev_rules.html foraverycompletedescription.Seealsomanudev.
8
udevnamingruleexamples
#Namingtestingtheoutputofaprogram BUS=="scsi",PROGRAM="/sbin/scsi_id",RESULT=="OEM0815",NAME="disk1" #USBprintertobecalledlp_color BUS=="usb",SYSFS{serial}=="W09090207101241330",NAME="lp_color" #SCSIdiskwithaspecificvendorandmodelnumberwillbecalledboot BUS=="scsi",SYSFS{vendor}=="IBM",SYSFS{model}=="ST336",NAME="boot%n" #soundcardwithPCIbusid00:0b.0tobecalleddsp BUS=="pci",ID=="00:0b.0",NAME="dsp" #USBmouseatthirdportofthesecondhubtobecalledmouse1 BUS=="usb",PLACE=="2.3",NAME="mouse1" #ttyUSB1shouldalwaysbecalledpdawithtwoadditionalsymlinks KERNEL=="ttyUSB1",NAME="pda",SYMLINK="palmtophandheld" #multipleUSBwebcamswithsymlinkstobecalledwebcam0,webcam1,... BUS=="usb",SYSFS{model}=="XV3",NAME="video%n",SYMLINK="webcam%n"
9
udevpermissionruleexamples
Excerptsfrom/etc/udev/rules.d/40permissions.rules
#Blockdevices SUBSYSTEM!="block",GOTO="block_end" SYSFS{removable}!="1",GROUP="disk" SYSFS{removable}=="1",GROUP="floppy" BUS=="usb",GROUP="plugdev" BUS=="ieee1394",GROUP="plugdev" LABEL="block_end" #Otherdevices,byname KERNEL=="null",MODE="0666" KERNEL=="zero",MODE="0666" KERNEL=="full",MODE="0666"
10
Identifyingdevicedrivermodules
Kernel/modulecompiling
Eachdriverannounceswhichdeviceandvendor idsitsupports.Informationstoredinmodulefiles. Thedepmodacommandprocesses modulefilesandgenerates /lib/modules/<version>/modules.alias
Systemeverydaylife
Thedrivercore(usb,pci...)readsthedeviceid, vendoridandotherdeviceattributes.
Thekernelsendsaneventtoudevd,settingthe MODALIASenvironmentvariable,encodingthesedata.
Audeveventprocessruns modprobe$MODALIAS
modprobefindsthemoduletoload inthemodules.aliasfile.
11
Modulealiases
MODALIASenvironmentvariableexample(USBmouse): MODALIAS=usb:v046DpC03Ed2000dc00dsc00dp00ic03isc01ip02 Matchinglinein/lib/modules/<version>/modules.alias: aliasusb:v*p*d*dc*dsc*dp*ic03isc01ip02*usbmouse
12
udevmodproberuleexamples
Evenmoduleloadingisdonewithudev! Excerptsfrom/etc/udev/rules.d/90modprobe.rules
ACTION!="add",GOTO="modprobe_end" SUBSYSTEM!="ide",GOTO="ide_end" IMPORT{program}="ide_mediaexport$devpath" ENV{IDE_MEDIA}=="cdrom", RUN+="/sbin/modprobeQbaidecd" ENV{IDE_MEDIA}=="disk", RUN+="/sbin/modprobeQbaidedisk" ENV{IDE_MEDIA}=="floppy",RUN+="/sbin/modprobeQbaidefloppy" ENV{IDE_MEDIA}=="tape",RUN+="/sbin/modprobeQbaidetape" LABEL="ide_end" SUBSYSTEM=="input",PROGRAM="/sbin/grepmapudev",\ RUN+="/sbin/modprobeQba$result" #Loaddriversthatmatchkernelsuppliedalias ENV{MODALIAS}=="?*",RUN+="/sbin/modprobeQ$env{MODALIAS}"
13
Coldplugging
Issue:loosingalldeviceeventshappeningduringkernel initialization,becauseudevisnotreadyyet. Solution:afterstartingudevd,havethekernelemitueventsfor alldevicespresentin/sys. Thiscanbedonebytheudevtriggerutility. Strongbenefit:completelytransparentforuserspace. Legacyandremovabledeviceshandledandnamedinexactlythe sameway.
14
Debuggingeventsudevmonitor(1)
udevadmmonitorvisualizesthedrivercoreevents andtheudeveventprocesses. ExampleeventsequenceconnectingaUSBmouse:
UEVENT[1170452995.094476]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2 UEVENT[1170452995.094569]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UEVENT[1170452995.098337]add@/class/input/input28 UEVENT[1170452995.098618]add@/class/input/input28/mouse2 UEVENT[1170452995.098868]add@/class/input/input28/event4 UEVENT[1170452995.099110]add@/class/input/input28/ts2 UEVENT[1170452995.099353]add@/class/usb_device/usbdev4.30 UDEV[1170452995.165185]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2 UDEV[1170452995.274128]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UDEV[1170452995.375726]add@/class/usb_device/usbdev4.30 UDEV[1170452995.415638]add@/class/input/input28 UDEV[1170452995.504164]add@/class/input/input28/mouse2 UDEV[1170452995.525087]add@/class/input/input28/event4 UDEV[1170452995.568758]add@/class/input/input28/ts2
Itgivestimeinformationmeasuredinmicroseconds. Youcanmeasuretimeelapsedbetweentheuevent(UEVENTline),andthe completionofthecorrespondingudevprocess(matchingUDEVline).

15
Debuggingeventsudevmonitor(2)
udevadmmonitorenv showsthecompleteeventenvironmentforeachline.
UDEV[1170453642.595297]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UDEV_LOG=3 ACTION=add DEVPATH=/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 SUBSYSTEM=usb SEQNUM=3417 PHYSDEVBUS=usb DEVICE=/proc/bus/usb/004/031 PRODUCT=46d/c03d/2000 TYPE=0/0/0 INTERFACE=3/1/2 MODALIAS=usb:v046DpC03Dd2000dc00dsc00dp00ic03isc01ip02 UDEVD_EVENT=1
16
Miscudevutilities
udevinfo Letsusersquerytheudevdatabase. udevtest<sysfs_device_path> Simulatesaudevruntotesttheconfiguredrules.
17
Firmwarehotplugging
Alsoimplementedwithudev! Firmwaredataarekeptoutsidedevicedrivers Maynotbelegalorfreeenoughtodistribute Firmwareinkernelcodewouldoccupymemory permanently,evenifjustusedonce. Kernelconfiguration:needstobesetin CONFIG_FW_LOADER (DeviceDrivers>GenericDriverOptions>hotplug firmwareloadingsupport)
18
Firmwarehotpluggingimplementation
Kernelspace
Driver
callsrequest_firmware() Sleeps
Userspace
/sys/class/firmware/xxx/{loading,data} appear
firmwaresubsystemeventsenttoudev Calling/lib/udev/firmware_helper Kernel

Getreadytoloadfirmwaredata Growsabuffertoaccommodateincomingdata
/lib/udev/firmware_helper
echo1>/sys/class/firmware/xxx/loading catfw_image>/sys/class/firmware/xxx/data echo0>/sys/class/firmware/xxx/loading
Driver
wakesupafterrequest_firmware() Copiesthebuffertothehardware Callsrelease_firmware()
SeeDocumentation/firmware_class/foraniceoverview
19
udevfiles
/etc/udev/udev.conf udevconfigurationfile. Mainlyusedtoconfiguresyslogreportingpriorities. Examplesetting:udev_log="err" /lib/udev/rules.d/ Standardudeveventmatchingrules,installedbythedistribution. /etc/udev/rules.d/*.rules Local(custom)udeveventmatchingrules.Besttomodifythese. /lib/udev/devices/* static/devcontent(suchas/dev/console,/dev/null...). /lib/udev/* helperprogramscalledfromudevrules. /dev/* Createddevicefiles.
20
Kernelconfigurationforudev
Createdfor2.6.19 Caution:nodocumentationfound,andnottestedyetonaminimalisticsystem. Somesettingsmaystillbemissing. Subsystemsanddevicedrivers(USB,PCI,PCMCIA...)shouldbeaddedtoo! #Generalsetup CONFIG_HOTPLUG=y #Networking,networkingoptions CONFIG_NET=y CONFIG_UNIX=y CONFIG_NETFILTER_NETLINK=y CONFIG_NETFILTER_NETLINK_QUEUE=y #Pseudofilesystems CONFIG_PROC_FS=y CONFIG_SYSFS=y CONFIG_TMPFS=y CONFIG_RAMFS=y
Unixdomainsockets
Neededtomanage/dev
21
udevsummarytypicaloperation
Kerneldrivercore (usb,pci...) uevent
udevd
udeveventprocess Matcheseventtorules Creates/removes devicefiles /lib/udev/programsorothers Loadtherightmodule Notifyuserspace programs(GUI...)
22
udevresources
Homepage http://kernel.org/pub/linux/utils/kernel/hotplug/udev.html Sources http://kernel.org/pub/linux/utils/kernel/hotplug/ Theudevmanualpage: manudev
23
mdev,theudevforembeddedsystems
udevmightbetooheavyweightforsomeembeddedsystems, theudevddaemonstayinginthebackgroundwaitingforevents. BusyBoxprovidesasimpleralternativecalledmdev,availableby enablingtheMDEVconfigurationoption. mdev'susageisdocumentedindoc/mdev.txtintheBusyBox sourcecode. mdevisalsoabletoloadfirmwaretothekernellikeudev
24
mdevusage
Tousemdev,theprocandsysfsfilesystemsmustbemounted mdevmustbeenabledasthehotplugeventmanager
echo/sbin/mdev>/proc/sys/kernel/hotplug
Needtomount/devasatmpfs: mountttmpfsmdev/dev Tellmdevtocreatethe/deventriescorrespondingtothe devicesdetectedduringbootwhenmdevwasnotrunning:

mdevs
Thebehaviorisspecifiedbythe/etc/mdev.conf configurationfile,withthefollowingformat
<deviceregex><uid>:<gid><octalpermissions> [=path][@|$|*<command>]
Example
hd[az][09]*0:3660
25
EmbeddedLinuxoptimizations
EmbeddedLinux optimizations
Size,RAM,speed, power,cost
Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/optimizations Corrections,suggestions,contributionsandtranslationsarewelcome!
Penguinweightwatchers
Makeyourpenguinslimmer,faster,andreduceitsconsumptionoffish! Before 2weeksafter
2
CELinuxForum
http://celinuxforum.org/ Nonprofitorganization,whosemembersareembeddedLinux companiesandConsumerElectronics(CE)devicesmakers. Mission:developtheuseofLinuxinCEdevices HostsmanyprojectstoimprovethesuitabilityofLinuxforCE devicesandembeddedsystems.Allpatchesaremeanttobe includedinthemainlineLinuxkernel. Mostoftheideasintroducedinthispresentationhavebeen gatheredorevenimplementedbyCELinuxForumprojects!
3
Contents
IdeasforoptimizingtheLinuxkernelandexecutables Increasingspeed Reducingsize:diskfootprintandRAM Reducingpowerconsumption Globalperspective:costandcombinedoptimizationeffects Theultimateoptimizationtool!
4
EmbeddedLinuxOptimizations
Increasingspeed
Reducingkernelboottime
5
Measuringkernelboottime
CONFIG_PRINTK_TIME ConfigureitintheKernelHackingsection. Addstiminginformationtokernelmessages.Simpleandrobust. Notaccurateenoughonsomeplatforms(1jiffy=10msonarm!) Seehttp://elinux.org/Printk_Times
... [42949372.970000]Memory:64MB=64MBtotal [42949372.970000]Memory:54784KBavailable(1404Kcode,296Kdata,72Kinit) [42949373.180000]Mountcachehashtableentries:512 [42949373.180000]CPU:Testingwritebuffercoherency:ok [42949373.180000]checkingifimageisinitramfs...itisn't(badgzipmagicnumb ers);lookslikeaninitrd [42949373.200000]Freeinginitrdmemory:8192K [42949373.210000]NET:Registeredprotocolfamily16 ...
6
Boottracer
CONFIG_BOOT_TRACERinkernelconfiguration IntroducedinLinux2.6.28 Basedontheftracetracinginfrastructure Allowstorecordthetimingsofinitcalls Bootwiththeinitcall_debugandprintk.time=1parameters, rundmesg>boot.logandonyourworkstation,run catboot.log|perlscripts/bootgraph.pl>boot.svg togenerateagraphicalrepresentation ExampleonaboardwithatAtmelAT91CPU:
5s
tty_init
pty_init
atmel_nand_init
ip_auto_config
Grabserial
FromTimBird http://elinux.org/Grabserial Asimplescripttoaddtimestampstomessagescomingfroma serialconsole. Keyadvantage:startscountingveryearly(bootloader), anddoesn'tjuststartwhenthekernelinitializes. Anotheradvantage:nooverheadonthetarget,becauserunon thehostmachine.
8
DisableIPautoconfig
StoppedinitializingtheIPaddressonthekernelcommandline (oldremainsfromNFSbooting,wasconvenientnottohardcode theIPaddressintherootfilesystem.) Instead,diditinthe/etc/init.d/rcSscript. Thissaved1.56sonourAT91board. Youwillsaveevenmoreifyouhadotherrelatedoptionsinyour kernel(DHCP,BOOP,RARP)
9
ReducingthenumberofPTYs
PTYsareneededforremoteterminals(throughSSH) Theyarenotneededinourdedicatedsystem! ThenumberofPTYscanbereducedthroughthe CONFIG_LEGACY_PTY_COUNTkernelparameter. Ifthisnumberissetto4,wesave0.63sonourAtmelboard. Aswe'renotusingPTYsatallinourproductionsystem, wedisabledthemwithcompletelywithCONFIG_LEGACY_PTYS. Wesaved0.64s. Notethatthiscanalsobeachievedwithoutrecompilingthe kernel,usingthepty.legacy_countkernelparameter.
10
Disableconsoleoutput
Theoutputofkernelbootupmessagestotheconsoletakes time!Evenworse:scrollingupinframebufferconsoles! Consoleoutputnotneededinproductionsystems. Consoleoutputcanbedisabledwiththequiet argumentintheLinuxkernelcommandline(bootloadersettings) Example: root=/dev/ram0rwinit=/startup.shquiet Benchmarks:canreduceboottimeby30oreven50%! Seehttp://elinux.org/Disable_Console
11
Presetloops_per_jiffy
Ateachboot,theLinuxkernelcalibratesadelayloop(forthe udelayfunction).Thismeasuresaloops_per_jiffy(lpj) value.Thistakesabout25jiffies(1jiffy=timebetween2timer interrupts). Inembeddedsystems,itcanbeabout250ms! Youjustneedtomeasurethisonce!Findthelpjvalueinkernel bootmessages(ifyoudon'tgetitintheconsole,bootLinuxwith theloglevel=8parameter).Example:
Calibratingusingtimerspecificroutine...187.59 BogoMIPS(lpj=937984)
Atthenextboots,startLinuxwiththebelowoption: lpj=<value>
12
LZOkerneldecompression
LZOisacompressionalgorithmthatismuchfasterthangzip, atthecostofaslightlydegradecompressionratio(+10%). Itwasalreadyinuseinthekernelcode(JFFS2,UBIFS...) AlbinTonnerrefromFreeElectronsaddedsupportforLZO compressedkernels.Hispatchesarewaitingforinclusionin mainstreamLinux.Getthemfromhttp://lwn.net/Articles/350985/
13
LZOdecompressionresults
Savesapproximately0.25sofboottime Seehttp://freeelectrons.com/blog/lzokernelcompression/ OurpatchalsoallowsLZOtobeusedforinitramfs decompression(CONFIG_INITRAMFS_COMPRESSION_LZO=y) Anothersolutionistouseanuncompressedkernel (anotherpatchwillbesent),inwhichcasekernelexecutionisjust marginallyfasterthanwithLZO,attheexpenseofadoublesize.
14
DirectlybootLinuxfrombootstrapcode
Idea:makeaslightchangetoat91bootstraptodirectlyloadand executetheLinuxkernelimageinsteadoftheUbootone. RatherstraightforwardwhenbootUbootandthekernelare loadedfromNANDflash. Requirestohardcodethekernelcommandlineinthekernel image(CONFIG_CMDLINE) RequiresmoredevelopmentworkwhenUbootisloadedfroma differenttypeofstorage(SPIdataflash,forexample). Inthiscase,youcankeepUboot,butremoveallthefeaturesnot neededinproduction(USB,Ethernet,tftp...) Timesavings:about2s Seehttp://freeelectrons.com/blog/at91bootstraplinux/
15
Reducethekernelsize
ThroughtheCONFIG_EMBEDDEDoption Removethingsthatarenotneededinyourdedicatedsystem (features,debuggingfacilitiesandmessages) Makesureyouhavenounusedkerneldrivers Disablesupportforloadablekernelmodulesandmakeallyour driversstatic(unlesstherearemultipledriversthancanbe loadedlater). Asmallerkernelisfastertoload Asimplerkernelexecutesfaster Atleast,compiledriversasmodulesfordevicesnotusedatboot time.Thisreducestimespentinitializingdrivers.
16
Fasterrebooting(1)
kexecsystemcall:executesanewkernelfromarunningone. Mustfasterrebooting:doesn'tgothroughbootstrap/bootloader code. Greatsolutionforrebootingaftersystem(firmware)upgrades. Usefulforautomaticrebootingafterkernelpanics. Seehttp://developer.osdl.org/andyp/kexec/whitepaper/kexec.pdf andDocumentation/kdump/kdump.txtinkernelsources.
17
Fasterrebooting(2)
Anotheroption:usereboot=softinthekernelcommandline Whenyoureboot,thefirmwarewillbeskipped. Drawback:unlikekexec,cannotbechosenfromuserspace. Supportedplatforms:i386,x86_64,arm,arm26(Aug.2006) SeeDocumentation/kernelparameters.txt inthekernelsourcesfordetails.Notsupportedonallplatforms.
18
Skipmemoryallocation
Idea:sparememoryatboottimeandmanageitbyyourself! Assumeyouhave32MBofRAM Bootyourkernelwithmem=30 Thekernelwilljustmanagethefirst30MBofRAM. Drivercodecannowreclaimthe2MBleft: buf=ioremap( 0x1e00000, /*Start:30MB*/ 0x200000 /*Size:2MB*/ );
Thissavestimeallocatingmemory. CriticaldriversarealsosuretoalwayshavetheRAMtheyneed.
19
KernelboottimeOtherideas
CopykernelandinitramfsfromflashtoRAMusingDMA (UsedbyMontaVistainDellLatitudeON) Fastboot,asynchronousinitcalls:http://lwn.net/Articles/314808/ Mainlined,butAPIstillusedbyveryfewdrivers. MostlyusefulwhenyourCPUhasidletimeinthebootprocess. Usedeferredinitcalls Seehttp://elinux.org/Deferred_Initcalls NAND:justcheckforbadblocksonce Atmel:seehttp://patchwork.ozlabs.org/patch/27652/ Seehttp://elinux.org/Boot_Timeformoreresources
20
Increasingspeed
Systemstartuptimeandapplicationspeed
21
Startingsystemservices
SysVinit: Startsservicessequentially.Waitsforthecurrentstartupscriptto becompletetostartthenextone!Whiledependenciesexist, sometaskscanberuninparallel! Initng:http://initng.org NewalternativetoSysVinit,whichcanstartservicesinparallel, assoonastheirpreconditionsaremet.
22
Initngvs.SysVinit:bootcharts
Initngwins! Systemutilizationismuch better. YoucanhuntsystemstartuptroublebyusingtheBootchartprogram (http://www.bootchart.org/). Bootchartisslow(Java)andnotveryaccurate. Seehttp://elinux.org/Bootchartforsolutionsforembeddedsystems.

23
Readingahead
LinuxkeepsthecontentsofallthefilesitreadsinRAM(inthe pagecache),aslongasitdoesn'tneedtheRAMpagesfor somethingelse. Idea:loadfiles(programsandlibrariesinparticular)inRAM cachebeforeusingthem.Bestdonewhenthesystemisnot doinganyI/O. Thankstothis,programsarenotstuckwaitingforI/O. UsedtheKnoppixdistributiontoachieveverynicebootspeed ups. AlsoplannedtobeusedbyInitng. NotveryusefulforsystemswithverylittleRAM: cachedpagesarerecycledbeforethefilesareaccessed.
24
Implementingreadahead
Youcanusethesys_readahead()systemcall inyourCprograms.Seemanreadaheadfordetails. Youcanalsousethereadaheadlistutility,whichreadsafile containingthelistoffilestoloadincache. Availableon:http://freshmeat.net/projects/readaheadlist/. InembeddedsystemsusingBusybox,youcanusethe readaheadcommand(implementedbyFreeElectrons).
25
Compilerspeedoptimizations
Bydefault,mosttoolsarecompiledwithcompileroptimizations. Makesureyouusethemforyourownprograms! O2isthemostcommonoptimizationswitchofgcc. Lotsofoptimizationtechniquesareavailable. Seehttp://en.wikipedia.org/wiki/Compiler_optimization O3canbealsobeusedforspeedcriticalexecutables. However,thereisdoneattheexpenseofcodesize(forexample inlining:replacingfunctioncallsbythefunctioncodeitself).
26
Usingprocessoraccelerationinstructions
liboilhttp://liboil.freedesktop.org/ Libraryoffunctionsoptimizedforspecialinstructions fromseveralprocessors(Altivec,MMX,SSE,etc.) Mainlyfunctionsimplementingloopsondataarrays: typeconversion,copying,simplearithmetics,directcosine transform,randomnumbergeneration... Transparent:keepsyourapplicationportable! Sofarmainlysupportsdesktopprocessors License:BSDtype
27
Prelinking(1)
Appliestoexecutablesusingsharedlibraries Toloadandstartanexecutable,thedynamiclinkerhasa significantamountofworktodo(mainlyaddressrelocation) Itcantakealotoftimeforexecutablesusingmanyshared libraries! Inmanysystemsinwhichexecutablesandsharedlibrariesnever change,thesamejobisdoneeverytimetheexecutableis started.
28
Prelinking(2)
prelink http://people.redhat.com/jakub/prelink/ prelinkmodifiesexecutablesandsharedlibrariestosimplify thedynamiclinkerrelocationwork. Thiscangreatlyreducestartuptimeforbigapplications(50% lessforKDE!).Thisalsosavesmemoryconsumedby relocations. CanbeusedtoreducethestartuptimeofaLinuxsystem. Justneedstoberunagainwhenlibrariesorexecutablesare updated. Detailsonhttp://elinux.org/Pre_Linking
29
UsesimplerUnixexecutables
Big,featurerichexecutablestaketimetoload. Particularlytrueforshellscriptscallingthebashshell! Idea:replacestandardUnix/GNUexecutablesbylightweight,simplified implementationsbybusybox(http://busybox.net). ImplementedbyUbuntu6.10toreduceboottime,replacingbash(649K) bydash(79K,seehttp://en.wikipedia.org/wiki/Debian_Almquist_shell). Thisbrokevariousshellscriptswhichusedbashspecificfeatures (bashisms). InnonembeddedLinuxsystems wherefeaturerichexecutablesarestillneeded, shouldatleastusebusyboxashforsystemscripts.
30
Shells:reducingforking
fork/execsystemcallsareveryheavy. Becauseofthis,callstoexecutablesfromshellsareslow. Evenexecutingechoinbusyboxshellsresultsinaforksyscall! SelectShells>Standaloneshellinbusybox configurationtomakethebusyboxshellcallappletswhenever possible. Pipesandbackquotesarealsoimplementedbyfork/exec. Youcanreducetheirusageinscripts.Example: cat/proc/cpuinfo|grepmodel Replaceitwith:grepmodel/proc/cpuinfo Seehttp://elinux.org/Optimize_RC_Scripts
31
Usefasterfilesystems
Runfasterbyusingthemostappropriatefilesystems! Compressedreadonlyfilesystem(blockdevice): useSquashFS(http://squashfs.sourceforge.net) insteadofCramFS(muchslower,gettingobsolete). NANDflashstorage:youshouldtryUBIFS (http://www.linuxmtd.infradead.org/doc/ubifs.html),the successorofJFFS2.Itismuchfaster.Youcouldalsouse SquashFS.SeeourChoosingfilesystemspresentation (http://freeelectrons.com/docs/filesystems).
32
Usefasterfilesystems(2)
UseRAMfilesystemsfortemporary,speedcriticalfiles withnoneedforpermanentstorage.Detailsinthekernel sources:Documentation/filesystems/tmpfs.txt Benchmarkyoursystemandapplicationoncompeting filesystems!Reiser4ismoreinnovativeandbenchmarks founditfasterthanext3. GoodtobenchmarkyoursystemwithJFSorXFStoo. XFSisreportedtobethefastesttomount(goodfor startuptime),andJFStohavethelowestCPUutilization. Seehttp://www.debianadministration.org/articles/388
ext4isalsoreadytobeusednow.
33
Speedupapplicationswithtmpfs
WhenenoughRAMisavailable,theOSkeepsrecentlyaccessedfilesand applicationsinRAM(pagecache).Thissignificantlyspeedsupanynewusage. However,dependingonsystemactivity,thismaynotlastlong. Forprogramsthatneedfaststartupeveniftheyhaven'tbeenrunforalongtime: copythemtoatmpfsfilesystematsystemstartup!Thismakessuretheyare alwaysaccessedfromthefilecacheinRAM(providedyoudonothaveaswap partition). SeeDocumentation/filesystems/tmpfs.txtinkernelsourcesfordetailsabout tmpfs. Caution:don'tuseramdisksinstead! RamdisksduplicatefilesinRAMandunusedspacecannotbereclaimed. Caution:usewithcare.Mayimpactoverallperformance. Notneededifthere'senoughRAMtocacheallfilesandprograms.
34
Bootfromahibernateimage
Theultimatetechniqueforinstantboot! Indevelopment:startthesystem,requiredapplicationsandthe userinterface.Hibernatethesystemtodisk/flashinthisstate. Inproduction:bootthekernelandrestorethesystemstatefrom withthispredefinedhibernationimage. Thisway,youdon'thavetoinitializetheprogramsonebyone. Youjustgetthebacktoavalidstate. UsedinSonycamerastoachieveinstantpowerontime. UnlikeSuspendtoRAM,stillallowstoremovebatteries!
35
Useaprofiler
Usingaprofilercanhelptoidentifyunexpectedbehaviordegrading applicationperformance. Forexample,aprofilercantellyouinwhichfunctionsmostofthe timeisspent. Possibletostartwithstraceandltrace AdvancedprofilingwithValgrind:http://valgrind.org/
Compileyourapplicationforx86architecture YoucanthenprofileitwiththewholeValgrindtoolsuite: Cachegrind:sourcesofcachemissesandfunctionstatistics. Massif:sourcesofmemoryallocation.
SeeourSoftwareDevelopmentpresentationfordetails: http://freeelectrons.com/docs/swdev/
36
Reducingsize
KernelsizeandRAMusage
37
LinuxTiny
Goal:reducethediskfootprintandRAMsizeoftheLinuxkernel http://elinux.org/Linux_Tiny SetofpatchesagainstthemainstreamLinuxkernel. Mergeabilityinmainstreamisapriority. Manychangeshavealreadybeenmergedinrecentkernels. Allfeaturescanbeselectedinkernelconfiguration (CONFIG_EMBEDDED). Alsoshipsutilitiesorpatchesfortrackingsources ofmemoryusageorcodesize.
38
LinuxTinyideas(1)
Removekernelmessages(printk,BUG,panic...) Huntexcessinlining(speedvs.sizetradeoff) 2.6.26:canallowgcctouninlinefunctionsmarkedasinline: (CONFIG_OPTIMIZE_INLINING=y).Onlyusedbyx86sofar. Huntexcessmemoryallocations Memory(slobinsteadofslab)allocatormorespaceefficientfor smallsystems. Reducethesizeofkerneldatastructures(mayimpact performance) Simpleralternativeimplementationsofkernelfunctionalities withlessfeatures,ornotsupportingspecialcases.
39
LinuxTinyideas(2)
Removesomefeatureswhichmaynotbeneeded insomesystems. Compilingoptimizationsforsize. AsmallerkernelexecutablealsosavesRAM (unlessexecutedinplacefromstorage).
40
LinuxTiny:kernelconfigurationscreenshot
Manyfeatures configuredout
41
WithandwithoutCONFIG_EMBEDDED
TestsonLinux2.6.29,onaminimalisticbutworkingx86kernel
1800
1600
1400
1200
1000
Raw Compressed
800
600
400
200
0 CONFIG_EMBEDDED=n CONFIG_EMBEDDED=y
Raw:272KB(17%),Compressed:136KB(20%)
42
Replaceinitrdbyinitramfs
Replaceinitramdisks(initrd)withinitramfs: muchlessoverheadandramwaste!
Access tofile VirtualFile System Filesystem driver Block driver Copy Blockstorage File cache
Regular blockdevice
Access tofile VirtualFile System Filesystem driver Block driver
Ramdisk blockdevice
Access tofile VirtualFile System
ramfs
File cache Copy Blockstorage RAM
File cache
RAM
RAM
43
ramfsadvantagesoverramdisks
Noblockandfilesystemoverhead. NoduplicationinRAM. Filescanberemoved(reclaimingRAM)afteruse. Initramfs:ramfsarchiveembeddedintheLinuxkernelfile.
44
Reducingsize
ApplicationsizeandRAMusage
45
Staticordynamiclinking?(1)
Staticlinking Allsharedlibrarycodeduplicatedintheexecutables AllowsnottocopytheClibraryinthefilesystem. Simplerandsmallerwhenveryfewexecutables(busybox) Librarycodeduplication:badforsystemswithmoreexecutables (codesizeandRAM) Bestforsmallsystems(<12MB)withfewexecutables!
46
Staticordynamiclinking?(2)
Dynamiclinking Sharedlibrarycodenotduplicatedintheexecutables Makesmuchsmallerexecutables SavesspaceinRAM(biggerexecutablestakemoreRAM) Requiresthelibrarytothecopiedtothefilesystem Bestformediumtobigsystems(>500KB1MB)
47
UsingalighterClibrary
glibc(GNUClibrary):http://www.gnu.org/software/libc/ FoundonmostcomputertypeGNU/Linuxmachines Sizeonarm:approx1.7MB uClibc:http://www.uclibc.org/ FoundinmoreandmoreembeddedLinuxsystems! Sizeonarm:approx400KB(yousave1.2MB!) Executablesareslightlysmallertoo:
Cprogram Plainhelloworld Busybox Compiledwithsharedlibraries glibc 4.6K 245K uClibc 4.4K 231K Compiledstatically glibc 475K 843K uClibc 25K 311K
48
HowtouseuClibc?
NeedtocompileallyourexecutableswithauClibctoolchain. Readytousetoolchainscanbefoundon http://freeelectrons.com/community/tools/uclibc Youcanveryeasilybuildyourownwithbuildroot: http://buildroot.uclibc.org/ YoualsohavetocopytheuClibcfilesfromthetoolchaintothe /libdirectoryinthetargetrootfilesystem. Readytousefilesystemscanalsobegeneratedbybuildroot. Youjustneedtoaddyourspecificstuffthen.
49
Needforstripping
Compiledexecutablesandlibrariescontainextrainformation whichcanbeusedtoinvestigateproblemsinadebugger. Thiswasusefulforthetooldeveloper,butnotforthefinaluser. Toremovedebugginginformation,usethestripcommand. Thiscansaveaverysignificantamountofspace! gccohellohello.c (outputsize:4635bytes) striphello (outputsize:2852bytes,38.5%) Don'tforgettostriplibrariestoo!
50
Aremyexecutablesstripped?
Youcanusethefilecommandtogettheanswer gccohellohello.c filehello hello:ELF32bitLSBexecutable,Intel80386,version1 (SYSV),forGNU/Linux2.2.5,dynamicallylinked(uses sharedlibs),notstripped striphello hello:ELF32bitLSBexecutable,Intel80386,version1 (SYSV),forGNU/Linux2.2.5,dynamicallylinked(uses sharedlibs),stripped Youcanusefindstrip(http://packages.debian.org/stable/source/perforate) tofindallexecutablesandlibrariesthatneedstrippinginyoursystem.
51
Howtostrip
Somelightweighttools,likebusybox,areautomaticallystripped whenyoubuildthem. Makefilesformanystandardtoolsofferaspecialcommand: makeinstallstrip Caution:strippingisarchitecturedependent. Usethestripcommandfromyourcrosscompilingtoolchain: armlinuxstrippotato
52
sstrip:superstrip
http://muppetlabs.com/~breadbox/software/elfkickers.html Goesbeyondstripandcanstripoutafewmorebitsthatare notusedbyLinuxtostartanexecutable. Canbeusedonlibrariestoo.Minorlimitation:processed librariescannolongerbeusedtocompilenewexecutables. CanalsobefoundintoolchainsmadebyBuildroot(optional)
Hello World Regular stripped sstripped 4691 B 2904 B (-38 %) 1392 B (-70 %)
Busybox 287783 B
Inkscape 11397 KB
230408 B (-19.9 %) 9467 KB (-16.9 %) 229701 B (-20.2 %) 9436 KB (-17.2 %)
Bestfortiny executables!
53
LibraryOptimizer
http://libraryopt.sourceforge.net/ ContributedbyMontaVista Examinesthecompletetargetfilesystem,resolvesallshared librarysymbolreferences,andrebuildsthesharedlibrarieswith onlytheobjectfilesrequiredtosatisfythesymbolreferences. Canalsotakecareofstrippingexecutablesandlibraries. However,requirestorebuildallthecomponentsfromsource. WouldbenicertoachievethisonlywithELFmanipulations.
54
Compilerspaceoptimizations
Regularcompileroptimizationssimplifyingcodealsoreducesize Youcanalsoreducethesizeofexecutablesbyaskinggccto optimizegeneratedcodesize: gccOsohusbandhusband.c OscorrespondstoO2optimizationsexcepttheonesincreasing size,plusextrasizespecificones. Osisalreadyusedbydefaulttobuildbusybox. Possibletofurtherreducethesizebycompilingandoptimizingall sourcesatonce,withthefwholeprogramcombinegcc options. Seehttp://gcc.gnu.org/onlinedocs/gcc/OptimizeOptions.html forallgccoptimizationoptions.
55
Simplegccoptimizationbenchmark
Executable size
250000
200000
150000
None O2 (generic) O3 (speed) Os (size)
100000
50000
Busybox
Dropbear
56
Restartableapplications
WhenRAMisscarce,canbeusefultoabortapplicationsthatare notinuse(forexamplehiddengraphicalinterfaces). BettertodoitbeforetheLinuxKernelOOM (OutOfMemory)killercomesandmakesbaddecisions. YoucanusetheLinuxCheckpoint/Restartprojecttohavethe Linuxkernelsavethestateofarunningapplicationsothatitcan laterresumeitsexecutionfromthetimeatwhichitwas checkpointed. Seehttp://www.linuxcr.org/fordetails.
57
Compressingfilesystems
Cansignificantlyincreaseyourstoragecapacity MTD(flashorROM)storage:useUBIFS orJFFS2forsmallpartitions. Blockstorage:useSquashFS(http://squashfs.sourceforge.net) insteadofCramFSforreadonlypartitions.Itcompressesmuch betterandismuchfastertoo.
58
Mergingduplicatefiles
Softwarecompilingandinstallingoftencreateduplicatefiles... Checkthatyourrootfilesystemdoesntcontainany! dupmerge2:http://sourceforge.net/projects/dupmerge Replacesduplicatefilesbyhardlinks. clink:http://freeelectrons.com/community/tools/utils/clink Replacesduplicatefilesbysymboliclinks. Example:saves4%oftotalspaceinFedoraCore5. finddup:http://www.shelldorado.com/scripts/cmds/finddup Findsduplicatefiles.
59
Reducingpowerconsumption
60
Ticklesskernel
Kernelconfiguration:NO_HZsettinginProcessortypeandfeatures Toimplementmultitasking,theprocessorreceivesatimerinterrupt atagivenfrequency(every4msbydefaultonLinux2.6).Onidlesystems, thiswakesuptheprocessorallthetime,justtorealizethereisnothingtodo! Idea:whenallprocessorsareidle,disablethetimerinterrupt,andreenableit whensomethinghappens(arealinterrupt).Thissavespowerinlaptops,in embeddedsystemsandwithvirtualservers! 2.6.24:supportsx86,arm,mipsandpowerpc
61
PowerTOP
http://www.lesswatts.org/projects/powertop/ Averyniceutilityshowingthetop10sourcesofpowerconsumption. Requirements:aticklesskernel(>2.6.21),andamobile(x86)CPU. Bestrunonlaptopsrunningonbattery(toestimatebatterysavings). PowerTOPmeasuresthenumberofwakeups, andcountsthetimespentinlowpowermodes. Itdetectsissuesinbothkernelspace(drivers)anduserspace. Italreadyuncoveredseveralbugsintoday'sdistributions. Embeddedsystems:nonIntelprocessorsprobablynotsupportedyet. YoucanstillrunyourappsonaPCtochecktheirbehavior. Seealsohttp://www.lesswatts.orgforotherusefulresources.
62
PowerTOPinaction
63
cpufreq
Configuration:CPU_FREQinPowermanagementoptions AllowstochangetheCPUfrequencyonthefly Supportedarchitectures(2.6.20): i386,sh,ia64,sparc64,x86_64,powerpc,arm(i.MXonly). Usuallycontrolledfromuserspacethrough/sysbyauser configurablegovernorprocess,accordingtoCPUload,heat,battery status...Themostcommoniscpuspeed: http://carlthompson.net/software/cpuspeed/ Savesasignificantamountofbatterylifeinnotebooks.
64
SuspendhiddenGUIs
Idea:suspendhiddenuserinterfacestosaveCPUandpower. Sendasuspend(stop)signal: killSIGTSTP<pid> Sendacontinuesignal: killSIGCONT<pid>
65
Softwaresuspend
http://www.suspend2.net/ Lotsofgreatfeaturesfornotebookusers,suchasRAMsuspend orhibernatetodisk. Unfortunately,restrictedonsomeIntelcompatibleprocessors andtargetingonlymachineswithAPMorACPI(rarelyfoundin nonPCembeddedsystems!). Notaddressingtherequirementsofembeddedsystems(support forotherCPUs,voltagereduction...).
66
Globalperspective
Costandcombinedoptimizationeffects
67
Combinedbenefits
Speedincrease Morespeed LessRAM
Fasterallocations Lessswapping Sometimesless cacheflushing Fasterapplication loadingfromstorage andinRAM. Sometimes,simpler, fastercode. LessRAMusage
RAMreduction Powerreduction
CPUcanrunslowerorstaylonger inpowersavingmode Fewer/smallerRAMchips:less dynamicandstandbypower. CPUwithlesscache:lesspower Fewer/smallerstoragechips:less power
Costreduction
Slower,cheaper CPU Fewer/cheaper RAMchips CPUwithless cache:cheaper Fewer/cheaper storage
Lessspace
Lesspower
Cheaperbatteries orcheaperAC/DC converter
68
Theultimateoptimizationtool!
Wehaveseenmanywaystooptimize anexistingsystem. However,nothingreplacesagooddesign! So,firstcarefullydesignandimplement yoursystemandapplicationswiththeirrequirements inmind. Then,usetheoptimizationtechniquestofurtherimprove yoursystemandthepartsthatyoureused(kerneland applications).
69
Archiveyourlabdirectory
Cleanupfilesthatareeasyto retrieve,removedownloads. Generateanarchiveofyourlab directory.
1
Relateddocuments
Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

2
Lifeaftertraining
HerearethingswecoulddotosupportyouinyourembeddedLinux andkernelprojects: BSPdevelopmentforyourhardware (drivers,bootloader,toolchain) MaketheofficialLinuxsourcessupportyourhardware Systemdevelopmentandintegration Systemoptimization Huntingandfixingnastybugs Moretraining:seehttp://freeelectrons.com/training/.Your colleagueswhomissedthisclasscouldgotoourpublicsessions. Seehttp://freeelectrons.com/development andhttp://freeelectrons.com/services
3
Lastslides
Thankyou!
AndmaytheSourcebewithyou
4

E Linu Slides

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

E Linu Slides

Загружено:

Авторское право:

Доступные форматы

FreeElectrons

AttributionShareAlike3.0 Youarefree tocopy,distribute,display,andperformthework tomakederivativeworks tomakecommercialuseofthework Underthefollowingconditions Attribution.Youmustgivetheoriginalauthorcredit.

Introduction toembedded Linux

Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!

StandardClibrary Linuxkernel Bootloader Hardware

AfewMBofRAM,from4MB. 8MBareneededtodoreallydosomething. AfewMBofstorage,from2MB. 4MBtoreallydosomething. Linuxisn'tdesignedforsmallmicrocontrollersthatjusthavea fewtensorhundredsofKBofflashandRAM.

System administration basics

Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/commandline Corrections,suggestions,contributionsandtranslationsarewelcome!

routedeldefault orroutedel<IP> Deletesthegivenroute Usefultoredefineanewroute.

First,trytopingtheIPaddressofyourgateway. Thiswillconfirmthatyournetworkadapterworksfine. Then,makesureyoucanpingthenameserverIPaddress, whichwillconfirmthatyourgatewayisconfiguredproperly. Finally,makesureyoucanpinganyhostusingitsname,which willconfirmthatthenameserverconfigurationiscorrect.

mkfs.vfatvF32/dev/sda1(v:verbose) FormatsyourUSBkeybacktoFAT32format. mkfs.vfatvF32disk.img FormatsadiskimagefileinFAT32format. Blankdiskimagescanbecreatedasinthebelowexample(64MBfile): ddif=/dev/zeroof=disk.imgbs=1Mcount=64

Packagesarestoredinrepositories,usuallyonHTTPorFTP servers Oneshouldonlyusepackagesfromofficialrepositoriesofits distribution,unlessstrictlyrequired.

Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/toolchains Corrections,suggestions,contributionsandtranslationsarewelcome!

Kernel Kernelheaders CLibrary Applications

Therefore,compilingtheClibrary requireskernelheaders,andmany applicationsalsorequirethem.

Availablein<linux/...>and<asm/...>andafewother directoriescorrespondingtotheonesvisibleininclude/inthe kernelsources

SeveralClibrariesareavailable: glibc,uClibc,eglibc,dietlibc,newlib,etc. ThechoiceoftheClibrarymustbemadeat thetimeofthecrosscompilingtoolchain generation,astheGCCcompileriscompiled againstaspecificClibrary.

Cprogram Plainhelloworld (stripped) Busybox (stripped)

usedtobuildatoolchainthatrunsonyour workstationbutgeneratesbinariesforthe target Themostcommonsolutioninembedded

Timetobuildyourtoolchain ConfigureCrosstoolNG Runittobuildyourowncross compilingtoolchain

Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/bootloaders Corrections,suggestions,contributionsandtranslationsarewelcome!

Thispieceofcodeisusuallya1ststagebootloader,whichwill loadthefullbootloaderitself. Thebootloadercanthenofferallitsfeatures.Ittypically understandsfilesystemformatssothatthekernelfilecanbe loadeddirectlyfromanormalfilesystem.

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/uboot Corrections,suggestions,contributionsandtranslationsarewelcome!

AssumingthatyourboardisalreadysupportedbyUBoot,there shouldbeonefilecorrespondingtoyourboard,forexample include/configs/omap2420h4.h.

CompileUBoot,byspecifyingthecrosscompilerprefix. Example,ifyourcrosscompilerexecutableisarmlinuxgcc: makeCROSS_COMPILE=armlinux

NANDFlash information UBoot information

Target UBootTFTP client Ethernetconnection

Host TFTP server

Timetostartthepracticallab! ConfigureandcompileUboot foryourARMboard. Flashitonyourboard

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelintro Corrections,suggestions,contributionsandtranslationsarewelcome!

LinusTorvaldshasbeenabletocreatealargeanddynamic developerandusercommunityaroundLinux. Nowadays,hundredsofpeoplecontributetoeachkernelrelease, individualsorcompaniesbigandsmall.

Thisinterfaceisstableovertime:onlynewsystemcallscanbe addedbythekerneldevelopers ThissystemcallinterfaceiswrappedbytheClibrary,and userspaceapplicationsusuallynevermakeasystemcalldirectly butratherusethecorrespondingClibraryfunction

Note:inreality,manymoreminor versionsexistinsidethestableand developmentbranches

TheofficiallistofchangesforeachLinuxreleaseisjusta hugelistofindividualpatches! Verydifficulttofindoutthekeychangesandtogetthe globalpictureoutofindividualchanges. Fortunately,asummaryofkeychanges withenoughdetailsisavailableon http://wiki.kernelnewbies.org/LinuxChanges

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelusage Corrections,suggestions,contributionsandtranslationsarewelcome!

Linuxpatchcommandlineexample: cdlinux2.6.13 bzcat../patch2.6.14.bz2|patchp1 bzcat../patch2.6.14.7.bz2|patchp1 cd..;mvlinux2.6.13linux2.6.14.7

Keeppatchfilescompressed:usefultochecktheirsignaturelater. Youcanstillview(orevenedit)theuncompresseddatawithvim: vimpatch2.6.14.bz2(onthefly(un)compression)

Nowgettingbacktoanolderversion(fromthesamedirectory) >ketchupG2.6.30 2.6.31.6>2.6.30 Downloadingpatch2.6.31.6.bz2 Applyingpatch2.6.31.bz2R Downloadingpatch2.6.30.bz2 Applyingpatch2.6.30.bz2R

TomodifyakernelinaGNU/Linuxdistribution: theconfigurationfilesareusuallyreleasedin/boot/,togetherwith kernelimages:/boot/config2.6.1711generic

Driveroptions CONFIG_JOLIET=y CONFIG_ZISOFS=y

# #DOS/FAT/NTFilesystems # #CONFIG_MSDOS_FSisnotset #CONFIG_VFAT_FSisnotset CONFIG_NTFS_FS=m #CONFIG_NTFS_DEBUGisnotset CONFIG_NTFS_RW=y

Bydefault,thekernelbuildsystemassumesthatthekernelis beingbuiltforthehostarchitecture,i.enativecompilation Thearchitectureisnotdefinedinsidetheconfiguration,butatan higherlevel Wewillseelaterhowtooverridethisbehaviour,toallowthe configurationofkernelsforadifferentarchitecture

I2C,SPI,1wire,supportforthepopularembeddedbuses Hardwaremonitoringsupport,infrastructureanddriversforthermal sensors

MMC/SD/SDIOsupport LEDsupport RealTimeClockdrivers Voltageandcurrentregulators Stagingdrivers,crappydriversbeingcleanedup

Configureyourkernel Compileit BootitonavirtualPC Modifyarootfilesystemimageby addingentriestothe/dev/ directory

Setupacrosscompiling environment ConfigurethekernelMakefile accordingly Crosscompilethekernelforanarm targetplatform Onthisplatform,interactwiththe bootloaderandbootyourkernel.

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/busybox Corrections,suggestions,contributionsandtranslationsarewelcome!

Copyright20042009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/blockfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!

Blockdevicescanbereadandwrittenonaperblockbasis, withouterasing,anddonotwearoutwhenbeingusedforalong time

Writeanentry inthejournal Write tofile Clear journalentry

No Discard Discard incomplete incomplete journalentries journalentries

Journal Journal empty? empty?

Thankstothejournal, thefilesystemisnever leftinacorrupted state Recentlysaveddata couldstillbelost

Yes Execute Execute journal journal FilesystemOK FilesystemOK

Installationoftheimage Let'sassumeyourpartitiononthetargetisin/dev/sdc1 Onyourtarget,copythefilesystemimageonthedevice (CAUTION:don'trunthisonyourworkstation! Youcoulddestroycriticalsystempartitions.) ddif=rootfs.sqfsof=/dev/sdc1 Mountyourfilesystem: mounttsquashfs/dev/sdc1/mnt/root

Areadwritepartitionwithajournaledfilesystem(likeext3) Usedtostoreuserorconfigurationdata. Guaranteesfilesystemintegrityafterpowerofforcrashes. Ramstoragefortemporaryfiles(tmpfs)

readwrite userand configuration data

readonly Acompressedreadonlypartition(Squashfs) compressed Typicallyusedfortherootfilesystem(binaries,kernel...). root Compressionsavesspace.Readonlyaccessprotectsyour filesystem systemfrommistakesanddatacorruption.

Copyright20042010,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Dec7,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/flashfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!

FlashTranslationLayers forblockdeviceemulation Caution:patentedalgorithms!

RAMchips Blockdevice ROMchips Virtualmemory Virtualdevicesappearing asMTDdevices

Partitionsizesmustbemultipleoftheeraseblocksize. Youcanusesizesinhexadecimaltoo.Rememberthebelowsizes: 0x20000=128k,0x100000=1m,0x1000000=16m roliststhepartitionasreadonly isusedtousealltheremainingspace.

Bydefault,pkgconfiglooksin/usr/lib/pkgconfigforthe .pcfiles,andassumesthatthepathsinthesefilesarecorrect. PKG_CONFIG_PATHallowstosetanotherlocationforthe.pc filesandPKG_CONFIG_SYSROOT_DIRtoprependaprefixtothe pathsmentionedinthe.pcfiles.