Flash File Systems

JM2L
Updateon filesystems forflashstorage

MichaelOpdenacker. FreeElectrons http://freeelectrons.com/
1
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents
Introduction Availableflashfilesystems Ourbenchmarks Bestchoices Experimentalfilesystems Adviceforflashbasedblockdevices
2
Updateonfilesystemsforflashstorage
Introduction
3
Flashstorage
Wearetalkingaboutflashchips,accessedbytheLinux kernelasMemoryTechnologydevices. CompactFlash,MMC/SD,MemoryStickcards,together withUSBflashdrivesandSolidStateDrives(SSD), areinterfacedasblockstorage,likeregularharddisks. Attheend,wewillsayafewwords aboutdealingwiththesecondcategory.
4
Existingsolutions
Forthelastyears,only2filesystemchoicesforflashstorage jffs2 Wearleveling,ECC Powerdownresistant Compression Hugemounttimes Ratherbigmemoryusage Mainstreamsupport yaffs2 Wearleveling,ECC Powerdownresistant Nocompression Veryquickmounttime ProgrammedbyWookies (atleast1) AvailableasaLinuxpatch. 2solutions,butfarfrombeingperfect!
5
Electiontime!
Atlast,newchoiceshavebeendeveloped. LogFS NewfilesystemforMTD storage UBI Newlayermanagingerase blocksandwearleveling UBIFS Newfilesystemtaking advantageofUBI's capabilities AXFS AdvancedXIPFileSystem
6
Howdotheycompare toexistingsolutions? Mountingtime Accessspeed Memoryusage CPUusage Size?
Testhardware
CalaoSystemsUSBA9263 AT91SAM9263ARMCPU 64MBRAM 256MBflash 2USB2.0host 1USBdevice 100MbitEthernetport PoweredbyUSB! SerialandJTAGthrough thisUSBport. Multipleextensionboards. 162EUR SupportedbyLinux2.6.27!
7
Flashchips
NANDdevice:ManufacturerID:0xec,ChipID:0xda (SamsungNAND256MiB3,3V8bit)
Samsung'sreference:K4S561632HUC75
Availableflashfilesystems
9
TheMTDAPI
ALinuxkernelAPItoaccessMemoryTechnologyDevices AbstractsthespecificsofMTDdevices:eraseblocks,pagesize... Linuxfilesysteminterface MTDUsermodules
jffs2 yaffs2 Chardevice Readonlyblockdevice Blockdevice
MTDChipdrivers
CFIflash NANDflash DiskOnChipflash
RAMchips ROMchips
Memorydeviceshardware
10
MTDHowtouse
Creatingthedevicenodes Chardevicefiles mknod/dev/mtd0c900(badidea!!!) mknod/dev/mtd1c902(Caution!) mknod/dev/mtd2c904(Caution) Blockdevicefiles mknod/dev/mtdblock0b310(badidea!!!) mknod/dev/mtdblock0b312 mknod/dev/mtdblock0b312
11
jffs2
Today'sstandardfilesystemforMTDflash Nicefeatures:
Ontheflycompression.Savesstoragespace andreducesI/O. Powerdownreliable. Implementswearleveling
Standardfile API JFFS2 filesystem MTDdriver
Drawbacks:doesn'tscalewell
Mounttimedependingonfilesystemsize: thekernelhastoscanthewholefilesystemat mounttime,toreadwhichblockbelongsto eachfile. KeepingthisinformationinRAMismemory hungrytoo.
Flashchip
12
Newjffs2features
CONFIG_JFFS2_SUMMARY Reducesboottimebystoringsummaryinformation. Newjffs2compressionoptions:
Nowsupportslzocompression,andnotonlyzlib (andalsothertimeandrubincompressors) Cantryallcompressorsandkeep theonegivingthebestresults Canalsogivepreferencetolzo,totheexpenseofsize, becauselzohasthefastestdecompressiontimes.
13
jffs2Howtouse
Compilemtdtoolsifneeded: gitclonegit://git.infradead.org/mtdutils.git Eraseandformatapartitionwithjffs2: flash_eraseallj/dev/mtd2 Mountthepartition: mounttjffs2/dev/mtdblock2/mnt/flash Fillthecontentsbywriting Or,useanimage: nandwritep/dev/mtd2rootfs.jffs2
14
yaffs2
http://www.yaffs.net/ SupportsbothNANDandNORflash Nocompression Wearleveling,ECC,powerfailureresistant Fastboottime CodeavailableseparatelythroughCVS (DualGPL/Proprietarylicense fornonLinuxoperatingsystems) YAFFS2 filesystem MTDdriver Standardfile API
Flashchip
15
yaffs2Howtouse
Eraseapartition: flash_eraseall/dev/mtd2 Formatthepartition: sleep(any command can do!) Mountthepartition: mounttyaffs2/dev/mtdblock2/mnt/flash
16
UBI
UnsortedBlockImages http://www.linuxmtd.infradead.org/doc/ubi.html VolumemanagementsystemontopofMTDdevices. Allowstocreatemultiplelogicalvolumes andspreadwritesacrossallphysicalblocks. Takescareofmanagingtheeraseblocksandwearleveling. Makesfilesystemeasiertoimplement.
UBI Logical EraseBlocks MTD Physical EraseBlocks Volume1 Volume2
LEB
LEB
LEB
LEB
LEB
LEB
LEB
PEB PEB PEB PEB PEB PEB PEB PEB PEB

Freeblock Freeblock 17
UBIHowtouse(1)
First,eraseyourpartition(NEVERFORGET!) flash_eraseall/dev/mtd1 First,formatyourpartition: ubiformat/dev/mtd1s512 (possibletosetaninitialerasecountervalue) Seehttp://www.linuxmtd.infradead.org/faq/ubi.htmlifyoufaceproblems Needtocreatea/dev/ubi_crtldevice(ifyoudon'thaveudev) Majorandminornumberallocatedinthekernel.Findthesenumbersin /sys/class/misc/ubi_ctrl/dev/(e.g.:10:63) Orrunubinfo: UBIversion:1 CountofUBIdevices:1 UBIcontroldevicemajor/minor:10:63 PresentUBIdevices:ubi0
18
UBIHowtouse(2)
AttachUBItoone(ofseveral)oftheMTDpartitions: ubiattach/dev/ubi_ctrlm1 FindthemajorandminornumbersusedbyUBI: cat/sys/class/ubi/ubi0/dev(e.g.253:0) CreatetheUBIdevicefile: mknod/dev/ubi0c2530
19
UBIFS
http://www.linuxmtd.infradead.org/doc/ubifs.html Thenextgenerationofthejffs2filesystem, fromthesamelinuxmtddevelopers. AvailableinLinux2.6.27 WorksontopofUBIvolumes UBI MTDdriver Standardfile API UBIFS
Flashchip
20
UBIFSHowtouse
Creating ubimkvol/dev/ubi0Ntests116MiB mounttubifsubi0:test/mnt/flash Deleting umount/mnt/flash ubirmvol/dev/ubi0Ntest DetachtheMTDpartition: ubidetach/dev/ubi_ctrlm1
21
LogFS
http://logfs.org/logfs/ Alsodevelopedasareplacementforjffs2 Weannouncedwewouldcoverit,butitslatestversiononly supports2.6.25.Ourboardonlysupports2.6.21,2.6.27and beyond,andthe2.6.25LogFSpatchdoesn'tcompilein 2.6.27! Anyway,LogFSisnotreadyyetforproduction. Williteverbe,nowthatjffs2hasavaluablereplacement? Competitionisusefulthough.
22
AXFS
AdvancedXIPFileSystemforLinux http://axfs.sourceforge.net/ Allowstoexecutecodedirectlyfromflash, insteadofcopyingittomemory. AsXIPisnotpossiblewithNANDflash,worksbestwhen thereisamixofNORflash(forcode)andNAND(fornon XIPsections). Currentlypostedforreview/inclusioninthemainstream Linuxkernel.Tobeacceptedin2.6.29orlater? Notbenchmarkedhere.WeonlyhaveNANDflashanyway.
23
SquashFS
http://squashfs.sourceforge.net/ Filesystemforblockstorage!? Butreadonly!Noproblemwithmanagingeraseblocksand wearleveling.Finetousewiththemtdblockdriver. Youcanuseitforthereadonlysectionsinyourfilesystem. Activelymaintained.Releasesformanykernelversions (recentandold).
24
SquashFSHowtouse
Verysimple! Onyourworkstation,createyourfilesystemimage (example:120m/directoryinourbenchmarks) mfsquashfs120m120m.sqfs Eraseyourflashpartition: flash_eraseall/dev/mtd2 Makeyourfilesystemimageavailabletoyourdevice(NFS, copy,etc.)andflashyourpartition: ddif=120m.sqfsof=/dev/mtdblock2 Mountyourfilesystem: mounttsquashfs/dev/mtdblock2/mnt/flash
25
Benchmarks
26
Benchmarkoverview
Comparedfilesystems: jffs2,defaultoptions jffs2,lzocompressiononly yaffs2 ubifs,defaultoptions ubifs,nocompression squashfs DifferentMTDpartitions 8M 32M 120M Correspondingtomost embeddeddevicescenarios. Partitionsfilledatabout85%
AlltestedwithLinux2.6.27.
27
Readandmountingexperiments
MountinganarmLinuxrootfilesystem, takenfromtheOpenMokoproject. Advantages:mainlycontainscompressiblefiles (executablesandsharedlibraries). Representsaveryimportantscenario:bootingona filesysteminflash.Mountingandfileaccesstimearemajor componentsofsystemboottime.
28
Mounttime(seconds)
ubifsnoz/8M:doesn'tfit
18 16
14
12
10 jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs
0 8M 32M 120M
29
ZoomMounttime(seconds)8M
ubifsnoz/8M:doesn'tfit
1.4
1.2
0.8 jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs
0.6
0.4
0.2
0 8M
30
Memoryconsumptionaftermounting(KB)
Freememorymeasuredwith/proc/meminfo: MemFree+Buffers+Cached
1400
Nomistake. Proportional tofssize?
1200
1000
800
600
jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs
400
200
0 8M 32M 120M
31
Usedspace(MB)
Measuredwithdf AddsomespaceforUBIFS! 1MBfor8MB
120
100
80
60
jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs
40
20
0 8M 32M 120M
32
Readtime(seconds)
140
120
100
80 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs
60
40
20
0 8M 32M 120M
33
ZoomReadtime(seconds)8M
8
0 8M
34
CPUusageduringread(seconds)
Duringtheexperimentsinthepreviousslide (usingthesysmeasurefromthetimecommand)
120
100
80
60
40
20
0 8M 32M 120M
35
Fileremovaltime(seconds)
Removingallthefilesinthepartition (afterthereadexperiment)
45
40
35
30
25 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz
20
15
10
0 8M 32M 120M
36
Writeexperiment
Writing8Mdirectorycontentsmultipletimes (lessinthe8Mcase) Datacopiedfromatmpfsfilesystem, fornooverheadreadingthefiles. Contents:armLinuxrootfilesystem. Smalltomediumsizefiles,mainlyexecutablesandshared libraries. Notmanyfilesthatcan'tbecompressed.
37
Writetime(seconds)
yaffs2/8M32M120M:doesn'tfit ubifsnoz/8M:doesn'tfit
300
250
200
150
jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz
100
50
0 8M 32M 120M
38
ZoomWritetime(seconds)8M
yaffs2/8M:doesn'tfit ubifsnoz/8M:doesn'tfit
18
16
14
12
10 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz
0 8M
39
CPUusageduringwrite(seconds)
Duringtheexperimentsinthepreviousslide (usingthesysmeasurefromthetimecommand)
300
250
200
150
100
50
0 8M 32M 120M
40
Randomwriteexperiment
Writing1MBchunksofrandomdata (copiedfrom/dev/urandom). Tryingtomimicthebehaviorofdigitalcamerasand camcorders,recordingalreadycompresseddata.
41
Randomwritetime(seconds)
Caution:includesCPUtimegeneratingrandomnumbers!
1000
900
800
700
600
500
400
300
200
100
0 5M 27M 105M
42
ZoomRandomwritetime(seconds)8M
Caution:includesCPUtimegeneratingrandomnumbers!
40
35
30
25
20
15
10
0 5M
43
Otherexperiments
UBIFSwithonlylzosupport UBIFSsupportsbothlzo(fastertocompressand uncompress)andzlib(slower,butcompressesbetter),and triestofindthebestspeed/sizecompromise. WetriedUBIFSwithonlylzosupport,hopingthathaving onlyonecompressorwouldreduceruntime. Results:tinydifferencesinallbenchmarks,eveninCPU usage.(roughlybetween0.1and1%). Conclusion:don'ttrytobetoosmart. Thefilesystemisalreadyfinetuned toworkgreatinmostcases.
44
Suitabilityforverysmallpartitions
8MMTDpartition jffs2fits13MBoffiles Butprobablydoesn'tleave enoughfreeblocks UBIconsumes0.9MB ubifsfits6.6MBoffiles 4MMTDpartition jffs2fits5.1MBoffiles UBIconsumes0.8MB ubifsfitsonly1.6MBof files!
Biggersizes:UBIoverheadcanbeneglected: 32MB:consumes1.2MB 128MB:consumes3.6MB
45
Whatweobserved
jffs2 Dramaticallyoutperformedby ubifsinmostaspects. Hugemount/boottime. yaffs2 Alsooutperformedbyubifs. Maynotfitallyourdata Uglyfileremovaltime (poordirectoryupdate performance?) Memoryusagenotscaling ubifsleavesnoreason tosticktoyaffs2.
46
ubifs Greatperformanceinall cornercases. SquashFS Bestornearbest performance inallreadonlyscenarios.
Conclusions
Convertyourjffs2partitionstoubifs! Itmayonlymakesensetokeepjffs2 forMTDpartitionssmallerthan10MB, incasesizeiscritical. Noreasonlefttouseyaffs2insteadofjffs2? YoumayalsouseSquashFStosqueezemore stuffonyourflashstorage.Advisabletouseiton topofUBI,toletallflashsectorsparticipateto wearleveling.
SquashFS MTDblock MTDAPI UBI MTDdriver Flashchip

47
Experimentalfilesystems(1)
Alookatpossiblefuturesolutions? wikifs ACELFsponsoredproject. AWikistructuredfilesystem (today'sflashfilesystems arelogstructured). AlreadyusedinSonydigital camerasandcamcorders. Pros:direct/easyexportof devicefunctionality descriptiontoelinux.org. linuxtinyfs Targetssmallembedded systems. Negativememory consumption:achievedby compilingoutthekernelfile cache. Pros:veryfastmounttime Cons:amountonly filesystem.Wayto implementreadandwrite notfoundyet.
48
Experimentalfilesystems(2)
fsckfs Aninnovativefilesystemrebuildingitselfateachreboot. Pros:nouserspacetoolsareneeded. Nofsck.fsckfsutilityneeded. Cons:mounttimestillneedsimproving.
49
Adviceforflashbasedblockstorage
50
Issueswithflashbasedblockstorage
Flashstoragemadeavailableonlythroughablockinterface. Hence,nowaytoaccessalowlevelflashinterface andusetheLinuxfilesystemsdoingwearleveling. Nodetailsaboutthelayer(FlashTranslationLayer)they use.Detailsarekeptastradesecrets,andmayhidepoor implementations. Hence,itishighlyrecommendedtolimitthenumberof writestothesedevices.
51
Reducingthenumberofwrites
Mountyourfilesystemsasreadonly,orusereadonly filesystems(SquashFS),wheneverpossible. KeepvolatilefilesinRAM(tmpfs) Usethenoatimemountoption,toavoidupdatingthe filesystemeverytimeyouaccessafile.Oratleast,ifyou needtoknowwhetherfileswerereadaftertheirlastchange, usetherelatimeoption. Don'tusethesyncmountoption (commitswritesimmediately).Nooptimizationspossible. Youmaydecidetodowithoutjournaledfilesystems. Theycausemorewrites,butarealsomuchmorepower downresistant.
52
Usefulreading
IntroductiontoJFFS2andLogFS: http://lwn.net/Articles/234441/ NiceUBIpresentationfromToshiba: http://freeelectrons.com/redirect/celfubi.html Documentationonthelinuxmtdwebsite: http://www.linuxmtd.infradead.org/
53
Documentonhttp://freeelectrons.com/pub/conferences CreativeCommonsBYSAlicense
Thankyou!
Questions? Newfilesystem suggestions?
54

Flash File Systems

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Flash File Systems

Загружено:

Авторское право:

Доступные форматы

JM2L

Updateon filesystems forflashstorage

Howdotheycompare toexistingsolutions? Mountingtime Accessspeed Memoryusage CPUusage Size?

Standardfile API JFFS2 filesystem MTDdriver

PEB PEB PEB PEB PEB PEB PEB PEB PEB

10 jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs

0.8 jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs

Nomistake. Proportional tofssize?

jffs2 jffs2-lzo yaffs2 ubifs ubi-noz squashfs

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs

80 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz squashfs

25 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

10 jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

jffs2 jffs2-lzo yaffs2 ubifs ubifs-noz

Biggersizes:UBIoverheadcanbeneglected: 32MB:consumes1.2MB 128MB:consumes3.6MB

ubifs Greatperformanceinall cornercases. SquashFS Bestornearbest performance inallreadonlyscenarios.

SquashFS MTDblock MTDAPI UBI MTDdriver Flashchip

Вам также может понравиться