Android

(Chin-Feng Lai)
Assistant Professor, institute of CSIE, National Ilan University Sep 29th 2012
2012 MMN Lab. All Rights Reserved

2012

Outline
File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

File system Overview

Android system and Embedded system

File System

Linux Kernel Hardware Bootloader

File system Overview

File system
a method for easy managing and access the data stored and adhering to a specific data structure.

File system is a specialpurpose database for

Storage Organization Manipulation Retrieval of data and metadata

File system Overview

File system
You may be familiar with windows file system

File system Overview

Linux supports a wide range of filesystems

File system Overview

Unix is based on just a few judiciously selected paradigms.

Everything is a file.

File system Overview

What is File
Set of Data . Linux supported file type File Directories Symbol link Device Pipeline

In linux
/dev/hda /dev/hdb /dev/hdf /dev/sda /dev/sdb /dev/sdf

With data storage device

9

CDROM Hard disk SD . Flash Network

File system Overview

Dir /bin/ /boot/

Description Essential user command binaries Static files of the boot loader

/dev/ /etc/
/home/ /lib/

Device files Host-specific system configuration

User home directories Essential shared libraries and kernel modules

10

File system Overview

Root File System
That is contained on the same partition on which the root directory is located The File System on which all the other filesystems are mounted Minimum set of directories /boot, /dev, /etc, /bin, /sbin
Partition 1

dev \
bin Partition 2 home
File System Hierarchy Standard
11

File system Overview

File allocation
Contiguous allocation Linked allocation Indexed allocation

Indexed allocation

12

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

13

File System Abstraction Layer

Kernel implements an abstraction layer around its lowlevel File System interface. enables the kernel to support many types of filesystem

Common features

14

File System Abstraction Layer

Virtual File system (VFS)
A virtual file system (VFS) is an abstraction layer on top of a more concrete file system to allow client applications to access different types of concrete file systems in a uniform way. to access local and network storage devices transparently without the client application noticing the difference. a kernel software layer that handles all system calls related file systems. Providing a common interface to several kinds of file systems Not every File System supports all abstraction types

15

File System Abstraction Layer

application application application
User Space

C library (Libc)

Virtual File System (VFS) ext2 . ext3 FTL or NFTL MTD => Flash memory JFFS2 . YAFFS2

Abstraction Layer

Kernel Space

Hardware

16

File System Abstraction Layer

Interplay of VFS components

fb

Process

File object

f_dentry

Dentry object

d_inode i_sb

Disk File

Super block Object

Inode object

Common File Model

17

File System Abstraction Layer

Common File Model Consists of the following object types :
Superblock object which represents a specific mounted file system. Inode object general information about a specific file File object information about the interaction between an open file and a process Dentry object represents a directory entry, a single component of a path.

18

File System Abstraction Layer

Consists of the following object types :
Operations object

contained within each of these primary objects.

describe the methods that the kernel invokes against the primary objects.

super_operations object

19

kernel can invoke on a specific File System .

such as read_inode() and sync_fs(). kernel can invoke on a specific file. such as create() and link(). kernel can invoke on a specific directory entry. such as d_compare() and d_delete().

inode_operations object

dentry_operations object

File System Abstraction Layer

Special filesystem
File system Bdev Devpts Eventpollfs Futexfs Pipefs Proc Rootfs /proc /dev/pts mount point futex

Shm
Sysfs usbfs /sys /proc/bus/usb

IPC
USB

20

File System Abstraction Layer

/proc
allows the kernel to communicate information about each running process on a Linux system

/sys
models specific kernel objects such as physical devices and provides a way to associate devices with device drivers

21

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

22

Introduction to BusyBox
BusyBox combines tiny versions of many common UNIX utilities into a single small executable It has been called "The Swiss Army Knife of Embedded Linux Easy to use Releasing as Free software
under the GNU General Public License

Providing many standard Unix tools Designed to be a small executable file

For use with the Linux kernel For embedded system

23

Introduction to BusyBox
Size-optimization and limited resources Easily include or exclude features Provides a fairly complete POSIX environment Easy to customize your small or embedded systems BusyBox is a multi-call binary
an executable program that performs the same job as more than one utility program.

multi-call binary acts like a large number of utilities.

24

Introduction to BusyBox
BusyBox source tree
. |-- applets |-- arch |-- archival |-- AUTHORS |-- busybox |-- busybox.links |-- busybox_unstripped |-- busybox_unstripped.map |-- busybox_unstripped.out |-- Config.in |-- console-tools |-- coreutils |-- debianutils |-- docs |-- e2fsprogs |-- editors |-- examples |-- findutils |-- include |-- init |-- _install |-- INSTALL |-- libbb |-- libpwdgrp |-- LICENSE |-- loginutils |-- Makefile |-- Makefile.custom |-- Makefile.flags |-- Makefile.help |-- miscutils |-- modutils |-- networking |-- printutils |-- procps |-- README |-- runit |-- scripts |-- selinux |-- shell |-- sysklogd |-- testsuite |-- TODO |-- TODO_config_nommu `-- util-linux

25
25

Introduction to BusyBox
Currently available applets include:
[, [[, acpid, addgroup, adduser, adjtimex, ar, arp, arping, ash, awk, basename, beep, blkid, brctl,bunzip2,bzcat, bzip2, cal, cat,catv, chat, chattr, chgrp, chmod, chown, chpasswd, chpst, chroot,chrt, chvt, cksum, clear, cmp, comm, cp, cpio, crond, crontab,cryptpw, cut, date, dc, dd, deallocvt, delgroup, deluser, depmod,devmem, df, dhcprelay, diff, dirname, dmesg, dnsd, dnsdomainname,dos2unix, dpkg, du, dumpkmap, dumpleases, echo, ed, egrep, eject,env, envdir, envuidgid, expand, expr, fakeidentd, false, fbset,fbsplash, fdflush, fdformat, fdisk, fgrep, find, findfs, flash_lock,flash_unlock, fold, free, freeramdisk, fsck, fsck.minix, fsync,ftpd, ftpget, ftpput, fuser, getopt, getty, grep, gunzip, gzip, hd,hdparm, head, hexdump, hostid, hostname, httpd, hush, hwclock, id,ifconfig, ifdown, ifenslave, ifplugd, ifup, inetd, init, inotifyd,insmod, install, ionice, ip, ipaddr, ipcalc, ipcrm, ipcs, iplink,iproute, iprule, iptunnel, kbd_mode, kill, killall, killall5, klogd,last, length, less, linux32, linux64, linuxrc, ln, loadfont,loadkmap, logger, login, logname, logread, losetup, lpd, lpq, lpr,ls, lsattr, lsmod, lzmacat, lzop, lzopcat, makemime, man, md5sum,mdev, mesg, microcom, mkdir, mkdosfs, mkfifo, mkfs.minix, mkfs.vfat,mknod, mkpasswd, mkswap, mktemp, modprobe, more, mount, mountpoint,mt, mv, nameif, nc, netstat, nice, nmeter, nohup, nslookup, od,openvt, passwd, patch, pgrep, pidof, ping, ping6, pipe_progress, pivot_root, pkill, popmaildir, printenv, printf, ps, pscan, pwd,raidautorun, rdate, rdev, readlink, readprofile, realpath,reformime, renice, reset, resize, rm, rmdir, rmmod, route, rpm,rpm2cpio, rtcwake, run-parts, runlevel, runsv, runsvdir, rx, script,scriptreplay, sed, sendmail, seq, setarch, setconsole, setfont, setkeycodes, setlogcons, setsid, setuidgid, sh, sha1sum, sha256sum,sha512sum, showkey, slattach, sleep, softlimit, sort, split,start-stop-daemon, stat, strings, stty, su, sulogin, sum, sv,svlogd, swapoff, swapon, switch_root, sync, sysctl, syslogd, tac,tail, tar, taskset, tcpsvd, tee, telnet, telnetd, test, tftp, tftpd,time, timeout, top, touch, tr, traceroute, true, tty, ttysize,udhcpc, udhcpd, udpsvd, umount, uname, uncompress, unexpand, uniq,unix2dos, unlzma, unlzop, unzip, uptime, usleep, uudecode, uuencode,vconfig, vi, vlock, volname, watch, watchdog, wc, wget, which, who,whoami, xargs, yes, zcat, zcip

26

Introduction to BusyBox
Three steps:
$ make menuconfig (configure) $ make (to compile BusyBox using your .config) $ make install (install the 'bin/busybox' binary)

After make install, will create a dir named _install containing BusyBox utilities and basic root File System make menuconfig
BusyBox is extremely configurable. allows you to include only the components you need, thereby reducing binary size. Run xxxxconfig' to select the functionality that you wish to enable.

27

Introduction to BusyBox
make menuconfig

28

Introduction to BusyBox
Some of the make options available for BusyBox
Make target
help defconfig allnoconfig allyesconfig config menuconfig

Description
Show the complete list of make options Enable a default (generic) configuration Disable all applications (empty configuration) Enable all applications (complete configuration) Text-based configurator N-curses (menu-based) configurator

all
busybox clean distclean sizes

Build the BusyBox binary and documentation (./docs)

Build the BusyBox binary Clean the source tree Completely clean the source tree Emit the text/data sizes of the enabled applications

29

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

30

File system format

File System Class
Grouped into three general classes: Disk-based file systems Network file systems Special file systems

File System format:

Ext2 Ext3 JFFS2 YAFFS2 Cramfs UBIFS

31

File system format

Ext2 (second extended File System )
The most popular does not provide any power-down reliability or compression very fast and supports up to 16 or 32 TB.

Used with any of the following:

RAM disks in read/write mode Compact Flash devices in IDE mode NAND or NOR flash in read-only mode

Use mkfs.ext2 to format

32

File system format

Ext3 (third extended File System)
Enhanced ext2 by RedHat Its similar with Ext2 More fast and stable

33

File system format

JFFS (Journalling Flash File System)
a log-structured file system for use on NOR flash memory devices on the Linux operating system Flash memory (specifically NOR flash) must be erased prior to writing. The erase process has several limitations:
Erasing is very slow It is only possible to erase flash in large segment Flash memory can only be erased a limited number of times

It has been superseded by JFFS2.

JFFS2 (Journalling Flash File System Version 2)

NOR flash memory Develop by Axis Communications AB Version 2 is developed by Redhat
GC(garbage collection) Avoiding data lose when power off Open source
34

File system format

YAFFS (Yet Another Flash File System)
Develop by Aleph One The same as JFFS2 use for flash memory Designed for NAND flash memory Open source

YAFFS2 (Yet Another Flash File System Version 2)

YAFFS2 was designed to accommodate these newer chips YAFFS2 is based on the YAFFS1 source code, with the major difference being that internal structures are not fixed to assume 512 byte sizing YAFFS2 uses a more abstract definition of the NAND flash allowing it to be used with a wider variety of flash parts with different geometries, bad block handling rules etc.

35
35

File system format

UBIFS (Unsorted Block Images FS)
a new flash file system. Develop by Nokia Next generation of the JFFS2 file-system. UBIFS works on top of UBI, which are themselves on top of MTD devices UBIFS is the default root filesystem of the Nokia N900 Smartphone. UBIFS supports on-the-fly data compression with either zlib (Deflate algorithm) or LZO.

36

File system format

MTD (Memory Technology Devices)
The aim of the system is to make it simple to Provides an abstraction layer for flash device, , by providing a generic interface between the hardware drivers and the upper layers of the system. to use the same API when working with different flash types and technologies. Do not be confused by USB sticks, MMCs, SDs, CompactFlashes. Although they are also called "flash", they are not Memory Technology Devices. Hardware drivers need to know nothing about the storage formats used, such as FTL, FFS2, etc., but will only need to provide simple routines for read, write and erase. Presentation of the device's contents to the user in an appropriate form will be handled by the upper layers of the system.

37

File system format

MTD (Memory Technology Devices)
Chip drivers Memory driver Such as ROM . Flash . NOR .NAND Core Library and struct Modules User modules
UBI JFFS2

MTD modules

YAFFS
MTD Chip drivers

NOR flash

NAND flash

RAM

ROM
Memory devices

Hardware

38

File system format

MTD subsystem
which provides uniform interface to access flash chips.

UBI subsystem
which is a wear-leveling and volume management.

UBI volumes are higher level entities than MTD devices

Volume1 Volume2

UBI Block Block Block Block Block Block

MTD
Block Block Block Block Block Block

39

File system format

Network filesystem
Export a directory on an NFS server and mount that directory on a remote client machine as if it were a local FS An embedded developer can have access to files, libraries, tools, & utilities during development and debugging.

Will be introduced in next chapter

40

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

41

File system image tool

fdisk
Partition table manipulator for Linux divided Hard Disk in to one(or more) logical disks

42

File system image tool

mkfs.FS_FORMAT
Used to build a Linux file system Usually a hard disk partition

mount
Mount a file system Serves to attach the file system found unmount will detach

43

File system image tool

The third part of system for devkit8000.

Bootloader

Kernel

File System

44

File system image tool

Use mkfs.ubifs executable file to generate UBIFS image
1. Copy rootfs to work_dir
$ tar xvf [src_tar_path]

2. Make UBIFS image by

$ sudo [tool_path]/mkfs.ubifs -r rootfs -m 2048 -e 129024 -c 512 -o ubifs.img

3. Make UBI.img by
$ sudo [tool_path]/ubinize -o ubi.img -m 2048 -p 128KiB -s 512 [tool_path]/ubinize.cfg

45

File system image tool

UBIFS

MTD modules

46

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Network File System Introduction LAB

47

Network File System Introduction

Ways to Load Kernel Image
Device UART : Universal Asynchronous Receiver/Transmitter MMC : Multi Media Card Protocol TFTP :Trivial File Transfer Protocol Use Network cable to transfer data

48

Network File System Introduction

Trivial File Transfer Protocol
a technology for transferring files between networkdevices. It is a simplified version of FTP (File Transfer Protocol) TFTP typically uses UDP as its transport protocol Data transfer is initiated on port 69, but the data transfer ports are chosen independently by the sender and receiver during initialization of the connection. In transmission, the data less than a full-sized block(512 bytes) to signal that it is the last packet

49

Network File System Introduction

Network File System
Among many different file systems in many PCs , its not easy to share data or file Network File System allow various systems to share their directories and files just via the network

Whats the benefit NFS provide ?

No need for users to have separate home directories or disk partition on every network machine for sharing Storage devices such as floppy disks and CDROM drives can be used by other machines on the network. This may reduce the number of removable media drives throughout the network.

50

Network File System Introduction

/home /user1 /share_file /etc /opt /mnt /media /dev

/home /user2 /etc /opt /share_file /mnt /media /dev

NFS Mapping

51

MMN Lab.

Network File System Introduction

What is the purpose of NFS ?
Share the File System between different machine

Need not to compress the file system as image

Easy to mount file system from Host PC to target Board It consists of two components

NFS server
Share the file directory to clients as file system

NFS Client
Connect to server and mount NFS on local machine

52

Network File System Introduction

Through the network, you can reach to share individual files (share files). NFS will host a remote network share in a folder directory, and mount it to the local host.

NFS server share the /root/nfs directory

NFS client 2 NFS client 1 mount /root/nfs to /home/data/sharefile mount /root/nfs to /mnt/nfs/sharefile

53

Network File System Introduction

On x86 PC
Just using the mount command #mount -t nfs IP_addr:File_Path(To be shared) Usually used for mounting root file system
/home /user1 /rootfs /etc /opt /mnt /media /dev

In Cross-development environment

54

File system Overview File Abstraction Layer Introduction to BusyBox File system format File system image tool Trivial File Transfer Protocol Introduction Network File System Introduction LAB

55

LABLearn
Learn how to use File System
Understanding BusyBox How to create BusyBox File System

56

FS LAB
Step1 : Get the source code & tool
Host$ cd ~/embed/src Host$ tar -xvf busybox-1.18.5.tar.bz2

Step2 : Creat NFS directory

Host$ cd ~/embed/ Host$ tar -xvf rootfs.tar

57

FS LAB
Step3 : Use make utility to build
Host$ cd ~/embed/src/busybox-1.18.5 Host$ make menuconfig

Busybox Settings ---> Build Options ---> [*] Build BusyBox as a static binary (no shared libs)

Host$ make CROSS_COMPILE=arm-none-linux-gnueabiARCH=arm Host$ make install CROSS_COMPILE=arm-none-linux-gnueabiARCH=arm

58

FS LAB
All binary file link to busybox

59

FS LAB
Compile busybox and output _install folder

Step4 : Copy _install to SD card

Host$ cp -r _inatall/* ~/embed/rootfs/ Host$ cd ~/embed/rootfs/

Step5 : Configure

Host$ mkdir etc/init.d

Host$ ln s bin/busybox init Host$ chmod 777 etc/rcS

60

FS LAB
Step6 : Libary configure
Host$ cd ~/embed/rootfs

Host$ mkdir lib

Host$ cp ~/embed/Tool/arm-2007q3/arm-none-linuxgnueabi/libc/lib/* ~/embed/rootfs/lib

Step7 : Copy filesystem to SDcard

Host$ sudo cp ~/embed/rootfs/* /media/Filesystem/

61

FS LAB
Step8 : Connect Devkit8000
Host$ kermit -c

62

FS LAB
Step9 : Set the target boot argument

use a space to separate each

Devkit8000# setenv bootargs console=ttyS2,115200n8 rootdelay=1 root=/dev/mmcblk0p2 rw init/init Devkit8000# setenv bootcmd mmcinit;fatload mmc 1 80300000 uImage;bootm 80300000 Devkit8000# save

63

FS LAB
Devkit8000# pri
See the arguments of environment

64

FS LAB
Step10: Run boot
Devkit8000# boot

65

LAB (Coding)
Write a hello world program use cross compiler to compile
Host$ arm-none-linux-gnueabi-gcc hello.c -o hello Host$ cp hello ~/embed/rootfs

Open the terminal connected to devkit8000 and Execute!

66