d1 02 Carrier Slides

An Investigator’s Guide to
File System Internals
Brian Carrier
carrier@atstake.com
@stake
Overview
• Motivation
• Disk Layout (MBR and Partitions)
• Basic File System Model
• Classical File Systems

– FAT (16-bit and 32-bit)
– UNIX (FFS and EXT2FS)
• Journaling File Systems

– NTFS
– EXT3FS
• Other File Systems
? 2002 @stake, Inc. 2 FIRST File System Tutorial
1
@stake
Motivation
• Currently, most evidence is found in the file system
• A thorough knowledge of the file systems allow one to:

– Make better use of forensics tools
– Recover deleted content when tools do not exist
– Allow one to describe what your automated tools are doing
• This talk is going to cover basic & advanced topics of file

systems for:
– Manual file recovery
– Finding evidence of “wiping” tools
– Knowing how to most efficiently use tools
@stake
What is a File System?
• File Systems manage data storage.
• Organized into files, unlike memory.
• Provide an addressing scheme that is easy for humans to

understand, unlike memory.
• Examples: FAT, EXT2FS, FFS, NTFS, Reiser FS,
2
@stake
Tools
• The @stake Sleuth Kit (TASK) will be used throughout this

to show examples on real images.
• The Coroner’s Toolkit (TCT) and TCTUTILs could also be

used to collect some of the UNIX file system data.
• TASK is a collection of forensics command line tools for file

system analysis
• The tools parse file system structures and do not rely on

kernel support
@stake
Disk Layout (MBR and Partitions)
3
@stake
The Disk
• A disk can be imagined as a long stream of millions bytes
• A disk is typically broken up into one or more groups (or

partitions)
• Logical volume management systems do not directly follow

this model because they can have several physical disks
appear as one volume, which is then broken up into
partitions.
• We will cover x86 (DOS) partitions and disk labels
@stake
x86: Disk Layout
• The bytes are grouped into sectors, usually 512-bytes each
• Each sector is addressed, using Logical Block

Addressing (LBA) or Cylinder / Head / Sector (CHS)
• The first sector contains the Master Boot Record (MBR)

and the Partition Table
• The remaining sectors are organized into Partitions,

which contain file systems
4
@stake
x86: Master Boot Record (MBR)
• Located in first 512-bytes of disk
• The first 446 bytes are x86 assembly instructions that read
the partition table to identify a bootable partition
• This is the sector that the systems reads when booting
• The Partition Table starts at offset 446 and contains four 16-
byte entries (48 bytes total)
• The final 2-bytes contain a signature (0x55AA)
@stake
x86: Partition Table
• Contains 4 entries, each can describe a partition
• Entries are each 16-bytes long
• Entries contain:
– Starting address of partition in CHS and LBA format
– Ending address of partition in CHS format
– Size of partition in sectors
– Flags (is it bootable?)
– Type of Partition
5
@stake
x86: Partition Types
• There are more than one partition type because only four
entries exist in the MBR Partition Table.
• Primary Partition: Contains 1 file system and can only

be described by one the four MBR partition table entries.
• Extended Partition: Contains 0 or 1 Secondary

Partitions and 0 or 1 additional Extended Partitions (a linked
list). Used when more than 4 partitions are needed.
• Secondary Partition: Contains 1 file system and

occupies either all or part of an Extended Partition.
@stake
x86: Partition Example

Disk
MBR Primary Primary Primary Extended

(NTFS) (EXT2FS) (EXT2FS)
Sec Extended
(FAT)
Secondary
(NTFS)
6
@stake
x86: Partition Tools
• Displaying:
– fdisk
– Partition Dump (pd) <http://heat.ca.sandia.gov/projects/software.html>

– GNU Parted <http://www.gnu.org/software/parted/>
• Finding partitions when the table is gone

– gpart <http://www.stud.uni-hannover.de/user/76201/gpart/>
@stake
disk labels: Overview
• BSD systems use disk labels to describe a disk
• The disk label contains the disk layout (partitions), disk

geometry, label, model etc. (but no boot code)
• It is located somewhere in the “beginning” of the disk
• There is not a standard format, but the contents include:

– Offset (in sectors or cylinders)
– Size
– Type
– Flags
7
@stake
disk labels: x86-based BSD flavors
• x86-based BSD systems (i.e. OpenBSD, FreeBSD) use both

x86 partitions and disk labels
• An x86 partition is created as the primary partition for the

BSD system (so that it can be booted)
• The disk label is located in the beginning of the BSD

primary partition and is used by the kernel to identify where
the partitions exist
• The disk label can describe partitions that do not exist in the
x86 partition tables
• The labels can be viewed with the ‘disklabel’ command.
@stake
disk labels: x86-based BSD flavors example

Windows XP OpenBSD
f a b d e
• Only two x86 partitions are defined
• • BSD disk label is located at the beginning of the OpenBSD

The
partition (a)
• The disk label describes 5 partitions (partition f has an offset

earlier than it does)
8
@stake
disk labels: Sun
• Sector 0 of the disk (slice 2) contains the Virtual Table Of Contents

(VTOC) and partition descriptors:
– ID (boot, root, unassigned, swap, usr, backup, stand, var, home etc.)
– Permission Flag (not -mountable, Read Only)
– Starting Sector (or cylinder in some versions)
– Size of partition
• Maximum of 8 or 16 partitions
• Sectors 1-15 contain the bootblock code
• The contents can be listed with:
prtvtoc /dev/rdsk/c0t0d0s2
• The format command can also display the partition details
@stake
Basic File System Model
9
@stake
Very Basic File System Model
• In general, data within a file system can fall into 1 of 4

categories
• File System Layer

– Descriptive data about the file system (i.e. block sizes, offsets, dirty status)
• Content Layer
– File content (i.e. blocks, clusters). Most data falls into this category.
• Meta Data Layer

– Descriptive data about files (i.e. file size, permissions, time of last access)
• Name Layer
– Human addressing format (i.e. file and directory names)
@stake
TASK tool names
• Each tool in TASK is named based on the file system layers:
• File System Layer:

– fsstat
• Content Layer (data):

– dls, dcalc, dstat
• Meta Data Layer (inode):

– ils, istat, icat, ifind
• Name Layer (file name)

– fls, ffind
10
@stake
FAT
@stake
FAT12, FAT16, and FAT32 File Systems
• Primitive File System – has been used since early DOS

versions
• Still used in Windows 9x and on floppies
• Size limits of file system:

– FAT12: 32 megabytes (used in floppy disks)
– FAT16: 4 gigabytes
– FAT32: 8 terabytes
11
@stake
FAT: File System Layout
• The file system is organized into 3 areas:
• Boot Sector
– Contains file system details such as sizes and offsets
• Data Area
– Contains file and directory contents. Consecutive sectors in it are grouped

into clusters.
• Primary and Backup File Allocation Tables (FAT)

– Contains tables that manage the clusters in the Data Area
@stake
FAT: File System Layout Diagram
Boot Sector Data Area
File Allocation Tables Cluster
12
@stake
FAT: Data Area
• All file and directory content is stored in clusters within

the Data Area
• Each cluster is 1 or more (power of 2) consecutive 512-byte

sectors. The size of which is saved in the Super Block.
• Each cluster has an address and the first usable one is

number 2
• The File Allocation Table (FAT) manages the clusters
@stake
FAT: File Allocation Table
• The File Allocation Table (FAT) contains as many entries as

there are clusters in the Data Area
• The table is indexed by the cluster number and the contents

describe that cluster
• Table entries can be:

– An unallocated marker, used when we need to allocate new clusters
– The address of the next cluster in a file or directory
– End of File (EOF) marker if it is the last cluster in the file or directory
– Bad sector marker, so it is not reallocated
• The size of each table entry depends on the version of FAT

(12-bits, 16-bits or 32-bits)
13
@stake
FAT: File Allocation Table Example
0 0 3 4 5 E 0 8 9 E
0 1 2 3 4 5 6 7 8 9
• Files are located in Clusters 2, 3, 4, 5 and 7, 8, 9

• Cluster 6 is unallocated
• E is our End-Of-File marker
@stake
FAT: Directory Entries
• File and directory meta data are stored in Directory

Entry structures
• One directory entry is allocated for each file and directory
• Directory entries have a static size (32-bytes) and form a list

within the clusters allocated by the parent directory
• The clusters for the Root Directory (C:\) are statically

located in FAT12 and FAT16 at the first cluster (#2) and are
dynamically located in FAT32
• The Root Directory does not have a directory entry (i.e. it

has no meta data associated with it)
14
@stake
FAT: Directory Entry Structure
• name in ASCII (8 bytes)
• extension in ASCII (3 bytes)

• attribute flags (1 byte) [Directory, Read Only, Hidden, System,
Volume ID, Archive, Long Name Entry]
• lowercase flags (1 byte)

• Create time & date (5 bytes) (optional, and is 0 on some Windows
9X directories)
• Access date (2 bytes) (no time, but is required by spec)
• Written time & date (4 bytes) (optional)
• Address of the first allocated cluster (4 bytes total)

• Size of file (4 bytes)
@stake
FAT: Date and Time structures
• FAT stores the actual time, not the number of seconds from
a known date
• Lower 5-bits (0x001f) of time are seconds (0-29) divided by 2
• Next 6-bits (0x07e0) of time are minutes (0-59)
• Upper 5-bits (0xf800) of time are hours (0-23)
• Lower 5-bits (0x001f) of date are day (1-31)
• Next 4-bits (0x01e0) of date are month (1-12)
• Upper 7-bits (0xfe00) of date are year from 1980 (0-127)
15
@stake
FAT: Long File Name Directory Entries
• The Directory Entry contains the file name, but is

constrained to the 8.3 format.
• Long File Name (LFN) Directory Entries are the same size as
normal directory entries, but have a special value in the
attribute flag (all bits are set).
• Older versions of DOS happen to ignore directory entries

with this flag.
• Files with long names also have a short name directory

entry after the LFN directory entries
• Each LFN can hold 13 UNICODE letters, multiple ones exist

for long names.
@stake
FAT: LFN Structure
• sequence (1 byte) [EOF or linear value]
• 5 UNICODE letters of name (10 bytes)
• attributes flag (1 byte)
• reserved (1 byte)
• checksum (1 byte) (equal for all LFN of a file)
• reserved (2 bytes)
16
@stake
FAT: LFN Example
File is: abcdefghijklmnopqrstuvwxyz1234.txt
Name Attribute
1234.txt LFN
nopqrstuvwxyz LFN
abcdefghijklm LFN
abcdef~1.txt FILE
NOTES: Stored in “reverse” order and the SFN entry is the

only one with the starting sector and other meta data
@stake
FAT: Example
cluster 5 cluster 6
File File
Root Dir Content
Content
file1 - 5
dir1 - 7 cluster 7 cluster 8
foo.c - 8
File
bar.h - 13 Content
FAT
cluster 13
File
6 E E 9 10 E 0 0 14 E Content
5 6 7 8 9 10 11 12 13 14
17
@stake
FAT: File Lookup Example
• Examine contents of c:\dir1\file1.dat
1. Identify location of the Root Directory sectors (from the

super block)
2. Read the directory entries in the root directory sectors,

looking for one with the name “dir1” and a directory
attribute flag
3. When the entry is found, identify the starting cluster (16 for
example)
4. Read cluster 16 and process the data as directory entries,

looking for the entry with “file1” as the name and “dat” as
the extension.
@stake
FAT: File Lookup Example
5. If an entry is not found, look in entry 16 in the FAT to

identify the next cluster.
6. Repeat the lookups until the entry is found. Identify the

starting cluster from the directory entry (23 for example).
7. Read the contents of cluster 23 and then lookup entry 23 in

the FAT for the next cluster, 24 for example.
8. Read the contents of 24 and repeat the FAT lookups until

<EOF> is reached
18
@stake
FAT: File System Layer
• Two structures describe the file system
• The Boot Sector structure describes file system attributes.

The structure is different in FAT16 and FAT32.
– Cluster size
– Location and size of the FAT
– Start and end of the Data Area
– Magic value (0xAA55)
• The File Allocation Table (FAT) manages the clusters

within the Data Area
@stake
FAT: TASK & sectors
• TASK uses sectors for addressing instead of clusters
• The first cluster is hundreds of sectors into the file system,

after the Super Block and FATs
• Therefore, sectors are used as addresses so that we can

access the contents of the FAT and Super Block
19
@stake
FAT: fsstat
• The fsstat tool lists the important Boot Sector values and a
summary of the FAT.
# fsstat –f fat fat.dd
• Boot Sector Values:

– Volume ID
– Sector and Cluster size
– Sector of first Cluster
– Sector Range
– Locations of primary and backup FAT
• FAT Data
– Sequence of consecutive sectors that are allocated and either the next nonconsecutive
sector or EOF.
@stake
FAT: Content Layer
• The clusters in the Data Area contain the file and directory
content data.
• We can view the contents of the clusters using the ‘dcat’

command in TASK, or using ‘dd’.
• NOTE: TASK will require the sector address and not the
cluster.
# dcat –f fat fat.dd 390
• We can get the statistics of a sector using the ‘dstat’

command:
# dstat –f fat fat.dd 390
20
@stake
FAT: Meta Data Layer
• The Directory Entries describe the files and directories in

the file system.
• We can list of details of which using the ‘ils’ command from

TASK:
# ils –e –f fat fat.dd
• We can get more details of a specific one using ‘istat’:
# istat –f fat fat.dd 5
• We can view the contents of the file using that directory

entry, use ‘icat’:
# icat –f fat fat.dd 5
@stake
FAT: TASK & Directory Entries
• Directory entries in FAT are not given addresses
• TASK treats every 32-bytes as a possible directory entry
• The Root Directory has no directory entry, so it is given an

address of 2 and the first 32-bytes in the first cluster are
given the address 3. The second 32-bytes are given the
address 4 etc.
• Not every address will be a directory entry
21
@stake
FAT: Name Layer
• All names are stored in the Directory Entry structure.
• We can get a listing of the file and directory names using

the ‘fls’ command:
# fls –rp –f fat fat.dd
@stake
FAT: File Allocation
• Results from testing a Windows 2000 system
• The new directory entry is placed at the end of the existing

list. Entries from deleted files are not reallocated, this
results in a LONG list of deleted files.
• The clusters are allocated by scanning the FAT for clusters

that are free. I’m not 100% sure of the algorithm yet:
– If the size is known, it will look for enough consecutive clusters (i.e. it will
skip over single free ones if it needs several of them)
– If the size is unknown or it is from a redirection, it allocates single free
clusters (but sometimes it skips over single ones and allocates several
consecutive ones - maybe due to buffering)
• In the last cluster, only the sectors that contain data are
overwritten (i.e. slack space)
22
@stake
FAT: File Deletion
• Directory entries are unallocated by setting the first letter

(which is also the first byte of the structure) to ‘0xEF’
• No other directory entry values are changed:

– The name will still exist
– The times do not reflect the deletion time
• The clusters are unallocated by setting the corresponding

FAT entries to 0
@stake
FAT: Allocation & Deletion Tools
• We can see how files are allocated by using the following

tools:
– ‘ils –e’ to list the status of all ‘inodes’
– ‘fls –rp’ to list all known directory entries

– ‘fsstat’ to get the contents of the FAT
– ‘dls -le’ to list the status of sectors
• Save the output of these tools, create or delete files, and

then compare the new tool output
23
@stake
Forensics Notes
• Because meta data is not in a static location, it can exist for a long
time before it is overwritten
• Deleted directory entries still contain names and starting cluster -

but not a deleted time
• Directory entry allocation algorithm is such that deleted entries are

not overwritten
• Only the Access time is required to be updated and it is only

accurate to the day. Created and written times are not consistently
written
• Deleted data can be found in the slack space of files
• Wiping tools will sometimes leave long file names such as

“aaaaaaaaaaaaaaaaaaaaaaaaaaaaa”
@stake
UNIX File Systems (FFS and EXT2FS)
24
@stake
UNIX File Systems
• Solaris, HP, and other BSD flavors use the Berkley Fast File
System (FFS)
• Linux uses the Extended 2 File System (EXT2FS) and the

Extended 3 File System (EXT3FS)
• EXT2FS is based on FFS
• We will present FFS first and then show the differences with
EXT2FS
@stake
UNIX: File System Layout
• The File System is divided into groups, called Cylinder

Groups in FFS (Block Groups in EXT2FS)
• All groups are the same size
• Groups contain both meta data structures and file content
• Each group manages its own resources
25
@stake
UNIX: File System Layout Diagram
Group
@stake
UNIX FFS: Cylinder Groups
• All Cylinder Groups in a given file system are the same size,
which is identified in the Super Block
• The first group starts at offset 0 (but the first 16 are

reserved)
• Each group contains a copy of the Super Block, but it

increments its offset (it has an offset of 16 for the first group
due to the disk labels)
• Each group contains a Group Descriptor structure that

describes the group. The inode and block bitmaps are
stored in the group descriptor.
26
@stake
UNIX FFS: Blocks & Fragments
• Sequential disk sectors (within the same group) are

organized into Blocks
• Blocks may also be broken into Fragments (which are

usually larger than a sector)
• Each Fragment is addressable
• Example:
– Sector Size: 512-bytes
– Block Size: 4096-bytes (8 sectors)
– Fragment Size: 1024-bytes (2 sectors)
• Each group has a bit-map that specifies the allocation

status of each fragment
@stake
UNIX FFS: Blocks & Fragments Diagram
0 1 2 3 4 5 6 7
512-byte 4096-byte
1024-byte
sector block
fragment
27
@stake
UNIX FFS: inodes
• File and directory meta data are stored in inode structures
• A static number of inode structures are allocated at file

system creation time
• Each group contains the same number of inode structures,

which are stored in an inode table.
• Multiple files may use the same inode (hard-links)
• Inodes have unique addresses, starting with the number 2
• The Root directory (/) is always described by inode 2
• Each group contains a bit-map that describes the allocation

status of inodes in that group
@stake
UNIX FFS: Summary of the inode Structure
• mode (2-bytes) [ permissions & type]
• number of links (2-bytes)
• size (8-bytes)
• Last access date & time (8-bytes)
• Last modified date & time (8-bytes) – file data modifications
• Last change date & time (8-bytes) - file status (inode)

change
• User ID (4-bytes) and Group ID (4-bytes)
• Partial list of allocated blocks (direct pointers) and list of

blocks with additional pointers (indirect pointers)
28
@stake
UNIX FFS: Block Pointers
• FFS was designed on research that showed that small files were
more common the large files
• An inode contains 12 direct, 1 single-indirect, 1 double-indirect,

and 1 triple-indirect pointer
• Direct Pointer: Points directly to a block that contains the file

or directory content
• Single-indirect Pointers: Points to a block that contains a

list of direct pointers, which point to the data
• Double-indirect Pointers: Points to a block that contains a

list of single-indirect pointers
• Triple-indirect Pointers: Points to a block that contains a

list of double-indirect pointers
@stake
UNIX FFS: Block Pointer Diagram

single double
direct
19 20 21 22 23 24 25 26 27 28 29 30 31 99
direct single
19 - file
32 33 34 35 ... 100 101 ...
content
direct
32 - file 200 201 ...
content
200 - file
content
29
@stake
UNIX FFS: Time & Date Format
• UNIX saves the time & date as the number of seconds since
1/1/1970 UTC.
@stake
UNIX FFS: Directory Entries
• File and directory names are stored in Directory Entry

structures that are located in blocks allocated by the parent
directory (similar to FAT, but different contents)
• Each structure contains a name and points to an inode

structure
• Directory Entry structures have a dynamic size and form a

linked list
• Multiple directory entries can point to the same inode

(symbolic links)
• The ‘.’ and ‘..’ directory entries are the first two in all
directories
30
@stake
UNIX FFS: Directory Entry Structure
• inode (4-bytes)
• record length (2-bytes)
• type (1-byte) {This does not exist in all versions}
• name length (1-byte) {This is 2-bytes if type does not

exist}
• name (less than 255, NULL terminated)
• The record length is rounded up to a multiple of 4
@stake
UNIX FFS: Directory Entry Example
rec len inode name len name
16 8357 5 file1
16 8358 5 file2
16 8359 5 file3
16 8357 file1 16 8358 file2 16 8359 file3
NOTE: The ‘.’ and ‘..’ entries are not shown
31
@stake
UNIX FFS: File Lookup Example
• Lookup /dir1/file1.dat
1. Locate inode 2 in the Inode Table of Group 1.
2. Read the blocks allocated to inode 2 and process them as

directory entries.
3. Analyze each directory entry for the one with the name
“dir1”. Save the inode value in the directory entry, 1810 for
example.
4. Calculate which group inode 1810 is part of by dividing it

by the number of inodes per group, group 2 for example.
5. Locate inode 1810 in the Inode Table of Group 2.
@stake
UNIX FFS: File Lookup Example
6. Read the blocks allocated to inode 1810 and process as

directory entries.
7. Look for an entry with the name “file1.dat” and save the
associated inode value, 2020 for example.
8. Calculate which group inode 2020 is part of by dividing it

by the number of inodes per group, group 3 for example.
9. Read the inode 2020 structure from the Group 3 Inode

Table and the blocks allocated by it.
32
@stake
UNIX FFS: File System Layer
• Two types of structures describe the file system
• The Super Block structure describes file system attributes.

– Size of Blocks & Fragments
– Size of groups
– Number of blocks and inodes per group
– Offset of group descriptor within each group
• The Group Descriptor structure within each Cylinder Group

describes the group attributes
– Location of inode bitmap
– Location of block bitmap
@stake
UNIX FFS: fsstat
• The fsstat tool lists important Super Block values and the
groups layout
# fsstat –f solaris solaris.dd
• Super Block Values:

– Block and fragment sizes
– Range of block addresses
– Range of inode addresses
• Group Info:
– Block and inode range of each group
– Location of inode table, group descriptor, and Super Block in each group
33
@stake
UNIX FFS: Content Layer
• The blocks and fragments contain the file and directory

content data.
• We can view the contents of the clusters using the ‘dcat’

command in TASK (& TCT/TCTUTILs), or using ‘dd’.
# dcat –f solaris solaris.dd 390
• We can get the statistics of a sector using the ‘dstat’

command:
# dstat –f solaris solaris.dd 390
@stake
UNIX FFS: Meta Data Layer
• The inode structures describe the files and directories in the

file system.
• We can list of details of which using the ‘ils’ command:
# ils –e –f solaris solaris.dd
• We can get more details of a specific one using ‘istat’:
# istat –f solaris solaris.dd 200
• We can view the contents of the file using that inode with
‘icat’:
# icat –f solaris solaris.dd 200
34
@stake
UNIX FFS: Name Layer
• All names are stored in Directory Entry structures.
• We can get a listing of the file and directory names using

the ‘fls’ command:
# fls –rp –f solaris solaris.dd
@stake
UNIX FFS: Block Allocation
• Blocks are allocated to the same Cylinder Group as the

inode. If the Cylinder Group is full or near full, a new one is
chosen based on disk rotation and percentage of free
blocks.
• Solaris will not fill up a group with a large file. It will fill up
half of the available blocks, then move on to the next group.
OpenBSD will fill up the group before moving on.
• The remainder of the fragment is wiped (i.e. no slack space)
35
@stake
UNIX FFS: Inode & Directory Entry Allocation
• File inodes are allocated in the same Cylinder Group as the

parent directory. If it is full, a quadratic hash search is
performed.
• Directory inodes are allocated in the next Cylinder Group

with a greater than average number of free inodes
• The first available inode (lowest address) in the group is

used
• Directory Entries are inserted into the list at the first

available location
@stake
UNIX FFS: File Deletion
• Blocks and Fragments: Blocks are unallocated in the bitmap

(including indirect blocks).
• Inode: The inode is set to unallocated in the group’s inode

bitmap. The size and block pointer fields are reset to 0. The
Modified & Changed times are set to the current time.
• Directory Entry: The record length of the previous directory

entry is increased by the length of the deleted entry. Solaris
will also reset the inode value to 0.
36
@stake
UNIX FFS: Deleted File Example

16 8357 5 file1
16 8358 5 file2
16 8359 5 file3
file2 is deleted
32 8357 5 file1
16 8358 5 file2
16 8359 5 file3
32 8357 file1 16 8358 file2 16 8359 file3

@stake
UNIX FFS: Allocation & Deletion Tools
• We can see how files are allocated by using the following

tools:
– ‘ils –e’ to list the status of all inodes
– ‘istat’ to get a list of all blocks and fragments that were allocated to a given
inode
– ‘fls –rp’ to list all known directory entries
– ‘dls -le’ to list the status of blocks and fragments
37
@stake
EXT2FS
@stake
UNIX EXT2FS: Block Groups
• EXT2FS calls the groups Block Groups - They are not

organized by disk geometry
• Each group manages itself and contains a duplicate copy of

the Super Block (which is always at the same offset)
• Each group has a table of Group Descriptors (FFS only

had the local one)
• Entire blocks are allocated to the inode and block bitmaps
• The Sparse Super Block option prevents the Super Block

from existing in every group and the space is used for data
38
@stake
UNIX EXT2FS: inode
• The EXT2FS inode contains much of the same data as FFS
• They have a different layout though
• The EXT2FS inode has a deletion time
@stake
UNIX EXT2FS: Block Allocation
• Blocks are allocated in the same Block Group as the inode

(if there is room). It tries to allocate a block where the next 7
are also free.
• Attention is not paid to one file filling up a Block Group.
39
@stake
UNIX EXT2FS: Inode & Dir Entry Allocation
• File inodes are allocated to the same block group as parent

directory, unless there is no room. Then they are allocated
to another based on a Quadratic Hash or linear search.
• Directory inodes are allocated to the block group with the

most free blocks and greater than average free inodes. The
search begins at group 0, so it could be the same one as the
parent directory.
• In both cases, the first (lowest address) inode is used.
• Directory entries are placed in the first available location in

the list.
@stake
EXT2FS: File Deletion
• Linux changed how they deleted files from the 2.2 kernel to
2.4
• Blocks & Fragments: Unallocated in the group bitmap.
• Inode: The inode is set to unallocated, but the block

pointers and size are not reset (the size is set to 0 when a
directory is deleted though).
• The Modified, Changed, and Deleted times of the inode are

set to the current time.
• Directory Entries: The length of the previous entry is

increased. In the 2.2 kernel the inode value was not cleared.
As of 2.4 it is set to 0, as FFS does.
40
@stake
EXT2FS: Other Tools
• debugfs runs on Linux and will let you view and change any
structure
@stake
UNIX: Forensics Notes
• Times still exist in unallocated inodes, but usually not the

directory entry pointer (except Open & FreeBSD FFS).
• With EXT2FS we still have access to the block pointers
• We can identify meta data wiping tools because there will be

zeroed inodes in between allocated inodes in a CG
• Blank directory entries in the middle of a directory are also

suspect
• To find the blocks for an unallocated inode, start looking in

the CG, not the beginning of the disk (use ‘fsstat’ and ‘dls’)
• By manually parsing the directory entry structure, you can

sometimes tell some order of file deletions
41
@stake
Journaling File Systems
@stake
Journaling File Systems: Motivation
• When a system crashes or is not shutdown properly, the file

system must be validated
• The ‘fsck’ tool scans inodes, directory entries, and bitmaps

to ensure that all structures agree
• This can take a LONG time on large file systems
42
@stake
Journaling File Systems: Overview
• A log is kept that contains meta data (and sometimes

content) changes, atomic transactions are used.
• Before the data is written, the intended updates are

recorded
• If the system crashes, the log is consulted and it can easily

identify which transactions did not complete. It then either
retries the update or turns back to the previous state.
• No public forensic tools currently take advantage of these

logs.
• Examples: NTFS, EXT3FS, IBM JFS
@stake
NTFS
43
@stake
NTFS
• Based on HPFS (designed by Microsoft and IBM)
• Used in Microsoft Windows NT, 2000, & XP
• Design is not widely published by Microsoft, but a

sourceforge project exists to write a Linux driver and
document it
• NTFS is object oriented and very flexible
• There are too many data structures to address in detail in

this talk (refer to TASK or Linux source)
@stake
NTFS: File System Layout
• 3 Major Areas (similar to FAT)
• Boot Sector
– Contains file system details such as sizes and offsets
• Master File Table (MFT)

– Manages the files and other objects
• Data Area
– Content area for files and objects
44
@stake
NTFS: Layout Example
Boot Sector Data Area
Master File Table
@stake
NTFS: Master File Table
• The Master File Table (MFT) contains an entry for each

user and system file
• MFT entries are called File structures and have a static size
of 1024 bytes
• MFT entries store file and directory meta data
• Data is stored in attribute structures (objects)
• Some files will require multiple MFT entries (due to static size)
• Multiple files may use the same MFT entry (hard-links)
• MFT entries are addressed (File Reference) and have a

sequence number
45
@stake
NTFS: MFT Entry Examples
• The MFT itself and mirror copy (0 & 1)
• LogFile (2)
• Root Directory (5)
• Volume Bitmap (3)
• Bad Clusters (8)
• Quota data (9)
• All files and directories (25+)
@stake
NTFS: Attributes
• An attribute describes a specific property of the file or

directory (i.e. the name or dates)
• Attributes have a header and a value.
• The header is always stored in the MFT entry
• The value is stored in either the MFT entry (resident) or in

the Data Area (non-resident)
46
@stake
NTFS: Attribute Header
• The attribute header is always located in the MFT
• It has the following fields:

– type (4-bytes) & id (2-bytes)
– length (4-bytes)
– resident flag
– name length and name (all have a name except the primary data one)
– If resident: A pointer to attribute stream
– If non-resident: A run-list of the clusters that contain the attribute data
– Virtual Cluster Number (VCN): The address of the cluster in a run with
respect to the beginning of the attribute (0 - n), not the beginning of the file
system (Logial Cluster Number - LCN)
@stake
NTFS: Attribute Types
• Standard Information: Read Only, Archive, MAC Times, number of

links (type 16)
• File Name(s) (type 48)
• Data (type 128)
• Index Root (type 144) and Index Allocation (type 160) (for directory
contents)
• Security Descriptor (type 80)
• Anything: For example, the icon on a Macintosh file is stored in an

attribute so it gets moved with the file (more type 128)
• Each attribute type has its own data structure
47
@stake
NTFS: Standard Information Attribute
• Attribute Type 16
• Dates and Times:

– Creation
– Last MFT change
– Last write
– Last access
• Permissions in DOS format
• version
• Owner Id
• Security Id
@stake
NTFS: File Name Attribute
• MFT Reference Number
• MFT Sequence Number
• Dates and Times:

– Creation
– Last MFT change
– Last write
– Last access
• Character set
• Name length and name
48
@stake
NTFS: Data Attribute
• Data attributes just contain the raw data
• The name of the data is given in the attribute header
• The only attribute that can not have a name is the primary
data attribute, which is typically given the name $Data by
default.
• Directories can have data attributes in addition to their B-

Tree attributes
@stake
TASK & NTFS Attributes
• MFT entries are addressed the same way in NTFS as inodes

are with UNIX file systems
• One can give the Type and Id (if there are more than one of a
given type) for a specific instance in the form of:
MFT-TYPE-ID
• For example:
# icat ntfs.dd 300
# icat ntfs.dd 300-128 (both are the same)
# icat ntfs.dd 300-128-3 (if an ADS exists)
49
@stake
NTFS: Data Area
• Data that can not fit in the MFT entry is stored in the Data
Area
• The Data Area is organized into Clusters (groups of

consecutive sectors)
• Cluster numbering starts with the first partition sector

(unlike FAT)
• The clusters are managed using the Volume Bitmap MFT

Entry (#6)
@stake
NTFS: Directories
• Directories store the file descriptors in B-trees, which are

sorted by any descriptor (file name by default)
• The B-tree nodes contain the:

– MFT reference number
– The structure that is being sorted by (file name structure for example)
• The MFT entry also stores the file name in an attribute
• Names are stored in UNICODE
• The Root Directory (\) is pointed to by a static MFT entry

(number 5)
50
@stake
NTFS: File Lookup Example
• Lookup \dir1\file1.dat
1. Read the Boot Sector to locate the beginning of the MFT
2. Read the MFT Entry for the Root Directory (#5)
3. Find & read the Index Root and Index Allocation attributes
4. Process the B-Tree and analyze the file name structures for
the one with the name “dir1”. Save the file reference value,
180 for example.
5. Read MFT entry 180
6. Find & read the Index Root and Index Allocation attributes
@stake
NTFS: File Lookup Example
7. Process the B-Tree and find the file name structure with
“file1.dat”. Save the reference value, 181 for example.
8. Read MFT entry 181.
9. Find & read the “$Data” attribute
51
@stake
NTFS: File System Layer
• Several structures describe the file system
• The Super Block contains the location of the MFT and the
cluster sizes
• The MFT entry in the MFT describes the location and size of
itself
• The $Log MFT entry contains the atomic transaction logs
• All of these can be viewed with the ‘fsstat’ command
# fsstat –f ntfs ntfs.dd
@stake
NTFS: Content Layer
• Data is stored in the MFT itself if it is small enough
• Larger data is stored in clusters in the Data Area
• Clusters are numbered from the beginning of the file system
• The dcat tool will allow us to display the contents of any

cluster
# dcat –f ntfs ntfs.dd 8016
• The icat tool will allow us to display the contents of

attributes that are stored in the MFT:
# icat -f ntfs ntfs.dd 76-160
52
@stake
NTFS: Meta Data Layer
• All file and directory meta data is generally located in the MFT
• If the value of the meta data is too large, it will be located in the
Data Area
• Each MFT is addressed using a File Reference Number
• We can view the details of an MFT entry using istat:
# istat –f ntfs ntfs.dd 5
• We can list them all using ils:
# ils –f ntfs –e ntfs.dd
• The contents of the file can be viewed using icat (with no type or id
value):
# icat -f ntfs ntfs.dd 9901
@stake
NTFS: Naming Layer
• File names are stored in a node of the B-Tree in the

directory’s MFT entry
• The file’s MFT entry also has all names that it uses (multiple
ones if it has hard-links)
• The list of known file names can be shown with fls:
# fls –f ntfs –rp ntfs.dd
53
@stake
NTFS: File Allocation
• An area following the MFT is reserved for its expansion
• MFT entries are allocated linearly
• Names are allocated into directories based on their name

and location in the B-tree
• Only the sectors in the last cluster that are used are
overwritten (slack space)
@stake
NTFS: File Deletion
• Content Blocks are unallocated by clearing the bits in the

bitmap
• MFT entry is freed by setting the in-use flag. The attributes

are unchanged. (link count never goes to 0).
• No times are set to the current time
• The name still exists in the MFT entry
• Parent directory Index Tree is resorted (the data may still

exist in the tree depending on the other files in the
directory)
54
@stake
Forensics Notes
• Because of MFT allocation, it is likely that unallocated entries will

be reused faster than with UNIX inodes
• No times are set when an MFT is unallocated, so time lines of file

activity are not as revealing
• Deleted file names will not exist as frequently as with FAT or UNIX
because they will get overwritten when the tree is sorted
• We can collect slack space data
• Clean MFT entries in the middle of the MFT is likely a result of

wiping tools
• If times in Standard Info are different than File Name, wiping tools
may have been used
@stake
NTFS: Other Tools
• NTFS File Sector Information Utility (nfi) (Microsoft)
• NTFS Info (www.sysinternals.com)
55
@stake
EXT3FS
@stake
EXT3FS: Overview
• EXT3FS is identical to EXT2FS except it interfaces with the

Journal layer
• The journal is kept in a special file or a specific journaling

device
• Currently, no tools publicly exist to utilize the journal for

forensics
56
@stake
Summary of Structures
FAT NTFS UNIX
File System Boo Sector, Boot Sector, Super Block,

FAT MFT Group Desc
Content Clusters in the Clusters in the Fragments &

Data Area Data Area Blocks in
groups
Meta Data Directory MFT Entries Inodes

Entries
Name Layer Directory B-tree Index Directory

Entries trees Entries
@stake
Log-based File Systems
57
@stake
Basics of Log-structured File Systems
• “Classical” File Systems waste time by seeking to allocated blocks

and overwriting the contents (for data or inodes)
• Log-structured File Systems work like a log file and append new
data to the “end” of the data
• All data and meta data are written to the current pointer.
• An inode index file points to the current location of the inode
• The inode points to the current location of its data blocks
• Deleted Content is not overwritten for a while...
• No Forensic Tools currently take advantage of Log FS...
• Example: BSD Log File System, Linux Log-Structured FS
@stake
PALM OS
• Everything is a database (even applications)
• Each database stores the dates of Creation, last

modification, and last backup
• When a database is deleted, it is flagged as not in use, but

not deleted (until a hotsync is done)
• See Joe Grand’s talk on the pdd Palm Acquisition tool

(Wednesday)
58
@stake
Summary
• Knowledge of file system structures will help interpreting

the results of tools
• Knowledge will also help write tools when ones do not exist
• Evidence of wiping tools can be found by examining the

structures and their location
• Using allocation algorithms, one can sometimes recover

deleted data more quickly
@stake
Software References
• TASK can be found at:
http://www.atstake.com/research/tools/task
• TCT & TCTUTILs (for UNIX only) can be found at:
www.fish.com
www.porcupine.org
www.cerias.purdue.edu/homes/carrier
59
@stake
References
• FAT
– Microsoft FAT32 File System Specification
– Microsoft Long Filename Specification
• FFS
– The Design and Implementation of the 4.4 BSD Operating System
(McKusick, Bostic , Karels , Quarterman)
• EXT2FS
– The extended-2 filesystem overview (Oxman)
– Design and Implementation of the Second Extended Filesystem (Card, Ts’o,

Tweedie)
@stake
References
• EXT3FS
– Transcript of Stephen Tweedie talk
(olstrans .sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html)
• NTFS
– Inside Windows 2000 - 3rd Edition (Solomon, Russinovich)
– Linux NTFS Project (linux-ntfs.sourceforge.net)
60

d1 02 Carrier Slides

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

d1 02 Carrier Slides

Загружено:

Авторское право:

Доступные форматы

An Investigator’s Guide to

File System Internals

• Disk Layout (MBR and Partitions)

• Basic File System Model

• Classical File Systems

– UNIX (FFS and EXT2FS)

• Journaling File Systems

• Other File Systems

? 2002 @stake, Inc. 2 FIRST File System Tutorial

• Currently, most evidence is found in the file system

• A thorough knowledge of the file systems allow one to:

– Allow one to describe what your automated tools are doing

• This talk is going to cover basic & advanced topics of file

– Finding evidence of “wiping” tools

– Knowing how to most efficiently use tools

? 2002 @stake, Inc. 3 FIRST File System Tutorial

What is a File System?

• File Systems manage data storage.

• Organized into files, unlike memory.

• Provide an addressing scheme that is easy for humans to

• Examples: FAT, EXT2FS, FFS, NTFS, Reiser FS,

? 2002 @stake, Inc. 4 FIRST File System Tutorial

• The @stake Sleuth Kit (TASK) will be used throughout this

• The Coroner’s Toolkit (TCT) and TCTUTILs could also be

• TASK is a collection of forensics command line tools for file

• The tools parse file system structures and do not rely on

? 2002 @stake, Inc. 5 FIRST File System Tutorial

Disk Layout (MBR and Partitions)

? 2002 @stake, Inc. 6 FIRST File System Tutorial

• A disk can be imagined as a long stream of millions bytes

• A disk is typically broken up into one or more groups (or

• Logical volume management systems do not directly follow

• We will cover x86 (DOS) partitions and disk labels

? 2002 @stake, Inc. 7 FIRST File System Tutorial

x86: Disk Layout

• The bytes are grouped into sectors, usually 512-bytes each

• Each sector is addressed, using Logical Block

• The first sector contains the Master Boot Record (MBR)

• The remaining sectors are organized into Partitions,

? 2002 @stake, Inc. 8 FIRST File System Tutorial

x86: Master Boot Record (MBR)

• Located in first 512-bytes of disk

• This is the sector that the systems reads when booting

• The final 2-bytes contain a signature (0x55AA)

? 2002 @stake, Inc. 9 FIRST File System Tutorial

x86: Partition Table

• Contains 4 entries, each can describe a partition

• Entries are each 16-bytes long

– Ending address of partition in CHS format

– Size of partition in sectors

– Flags (is it bootable?)

? 2002 @stake, Inc. 10 FIRST File System Tutorial

x86: Partition Types

• Primary Partition: Contains 1 file system and can only

• Extended Partition: Contains 0 or 1 Secondary

• Secondary Partition: Contains 1 file system and

? 2002 @stake, Inc. 11 FIRST File System Tutorial

x86: Partition Example

MBR Primary Primary Primary Extended

x86: Partition Tools

– Partition Dump (pd) <http://heat.ca.sandia.gov/projects/software.html>

• Finding partitions when the table is gone

? 2002 @stake, Inc. 13 FIRST File System Tutorial