Filesytem Fragmentation Lab

---------------------------------

OVERVIEW:

There are many myths and theories on how UFS deals withJfiles and fragments.

The purpose of this lab is to demonsstrate how the UFS

filesystem handles fragments on saturated filesystems.

SYSTEM TEST CONFIGURATION:

Ultra 10 440mhz ULTRASPARC IIi with 9 x2 GB PCI/IDE Drive

400mb UFS created on /dev/dsk/c0t1d0s0 and
mounted on /mnt

Section I - Filling the filesystem

----------------------------------

The following set of commands will fill the FS with 2k

files. This will put 4 files that are 2k in size per
logical data block.

#! We increased the amount of inodes to ensure that

#! we would not run out of them before we ran out
#! of disk space.

1) # exec ksh
# newfs -i 1024 /dev/rdsk/c0t1d0s0
# mount /dev/rdsk/c0t1d0s0 /mnt
# dd if=/dev/zero of=/2kfile bs=2k count=1
# x=0
# while :
> do cp /2kfile /mnt/file.$x
> let x=x+1
> done

Section II - Removing Files

---------------------------

The following set of commands removes every third 2k file

from /mnt. This will *effectively* leave 6k worth of data
on every logical data block.
1) # for i in 1 3 5 7 9
> do for x in 0 2 4 6 8
> do for y in 1 3 5 7 9
> do rm /mnt/file.1${i}${n}${x}${y}5
> done
> done
> done

Section III - New File Creation

-------------------------------

It has been the belief that the UFS filesystem is "self-policing."

UFS should shuffle data blocks (de-fragment on the fly) to
accomodate new files.

In the following example, we see that the UFS has plenty of free space.
Currently, each data block has 6 out of 8k occupying the data block.
We will see that the shuffling DOES NOT happen when we try to create
a file larger than 2k.

1) # df -k
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c0t0d0s0 96455 52430 34380 61% /
/dev/dsk/c0t0d0s6 8052181 703366 7268294 9% /usr
/proc 0 0 0 0% /proc
fd 0 0 0 0% /dev/fd
mnttab 0 0 0 0% /etc/mnttab
swap 520496 0 520496 0% /var/run
swap 521296 800 520496 1% /tmp
/dev/dsk/c0t0d0s5 288391 30168 229384 12% /opt
/dev/dsk/c0t0d0s7 96455 68844 17966 80% /export/home
/dev/dsk/c0t1d0s0 360775 359870 905 100% /mnt

2) # cd /mnt
# mkfile 2k file.a

#! The UFS had no problem creating a 2k file. The file was written to the remaining
#! 2k of the logical data block that was 6/8 full.

3) # mkfile 4k file.b
Could not set length of file.b: No space left on device

#! The output of df -k showed 902k still free in the UFS. However,

#! a file will not fragment until it has filled an entire 8k
#! data block. All the files in the filesystem are 2k. When
#! we try to create a 4k file, UFS will not allow us to
#! write 2k of file.b to one logical data block and 2 k to
#! another.
#!
#! Conclusions:
#!
#! - The UFS does not shuffle data blocks to accomodate new
#! files. It would not move files off of one logical data block
#! to create a contiguos 4k space to accomodate file.b.
#!
#! - Even with 905k free on the disk, the the largest file that
#! could be created is 2k.

Section IV - File Growth

------------------------

It is the belief that when a file needs to grow beyond its

current logical data block, it will take the fragmented portion
of the file, transfer it to a new space and write the new contents
to the end of the file.

1) # dd if=/dev/zero of=/2kfile bs=2k count=1

# ls -l /2kfile
-rw-r--r-- 1 root other 2048 Mar 9 11:19 /2kfile

#! Create a 2k file full of 0's.

2) # mkfile 2k file.c
# cat /2kfile >> /mnt/file.c
cat: write error: No space left on device

#! We created file.c in /mnt. This file occupied the 2k max

#! left on the 6/8 full data block. From there we tried to
#! append /2kfile to the end of file.c and it failed.
#!
#! Conlusions:
#!
#! - A file first has to occupy an 8k logical data block to be
#! able to grow a fragment on another block.
#!
#! - UFS will not move files off of data blocks to create a contiguos
#! space to grow a file. However, UFS will move data blocks if
#! the contiguos space to move files exists. See Section V.
#!

Section V - File System Growth with Contiguos Space

---------------------------------------------------

There is the belief that UFS will move files to make

contiguos space to make room for file growth. We have
proven that this is not the case. However, the UFS
does indeed move files from data blocks. The contiguos
space MUST exist first.

1) # rm file.[0-9][0-9]

#! Create contiguos spaces on the filesystem by removing

#! a series of 2k files

2) # mkfile 2k /mnt/file.d

#! We filled a data block with a 2k file.

# fsdb /dev/rdsk/c0t1d0s0

*SNIP OUTPUT*

/dev/rdsk/c0t1d0s0 > :ls -l file.d

/:
i#: bbc file.d
/dev/rdsk/c0t1d0s0 > 0xbbc:inode
/dev/rdsk/c0t1d0s0 > :inode:?i
i#: bbc md: ----rw-r--r-- uid: 0 gid: 1
ln: 1 bs: 4 sz : 800
db#0: 1ba6

*SNIP OUTPUT*

#! The fils system debugger (fsdb) lists bbc (hex) as the

#! inode number for file.d. From there, we find that the
#! first direct block is located at 1ba6.

3) # cat /2kfile >> /mnt/file.d

#! Append 2k to the end of file.d. Since we have

#! created contiguos space on the filesystem, we
#! did not get a filesystem full error.

4) # fsdb /dev/rdsk/c0t1d0s0

*SNIP OUTPUT*

/dev/rdsk/c0t1d0s0 > :ls -l file.d

/:
i#: bbc file.d
/dev/rdsk/c0t1d0s0 > 0xbbc:inode
/dev/rdsk/c0t1d0s0 > :inode:?i
i#: bbc md: ----rw-r--r-- uid: 0 gid: 1
ln: 1 bs: 8 sz : 1000
db#0: 3f0

*SNIP OUTPUT*

#! We now see that inode bbc's first block is now located at

#! block 3f0.
#!
#! Conclusions:
#!
#! - UFS does not manage or create contiguos space to accomodate
#! files. It will move data blocks to accomodate files if the
#! contiguos space ALREADY exists.

SECTION VI - Notes
------------------

This lab is created and tested by two Sun Certified Technical

Instructors and System Administrators. We are definitely not
programmers or operating systems architects. If any of the
previous examples prove to be flat out WRONG, feel free to
contact us