Oracle Cluster File System 2

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Objectives

After completing this lesson, you should be able to:

• Describe the purpose of Oracle Cluster File System 2
(OCFS2)
• Describe the features of OCFS2
• Prepare for an OCFS2 configuration
• Install the OCFS2 software packages
• Configure kernel settings for OCFS2
• Configure the cluster layout
• Describe the OCFS2 heartbeat
• Configure and start the O2CB cluster service
• Create an OCFS2 volume
• Mount an OCFS2 volume
• Use OCFS2 tuning and debugging utilities

6-2 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 OCFS2: Introduction

• OCFS2 is a shared disk cluster file system, which allows

multiple nodes to access the same disk at the same time.
– You can also use OCFS2 on a non-clustered system.
• OCFS (Release 1) was created specifically for use by
Oracle Real Application Clusters (RAC).
• OCFS2 is a general-purpose cluster file system.
– You can store any files on OCFS2.
– You interact with OCFS2 the same way you do on a regular
file system.
– OCFS2 file systems can be mounted and used across
multiple architectures at the same time.
• The latest release of OCFS2 requires the Oracle Linux
Unbreakable Enterprise Kernel.

6-3 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 OCFS2 Features

• Variable block and cluster sizes for different types of data

• Extent-based allocations for efficient storage of large files
• Sparse files, inline-data, unwritten extents, hole punching,
REFLINKS, and allocation reservation
• Extended attributes by attaching name:value pairs to file
system objects
• POSIX ACLs and SELinux for additional security
• Metadata checksums to detect silent corruption in inodes
and directories
• Indexed directories to allow quick lookups of a directory
entry in a very large directory
• User and group quotas to limit file system usage
• JBD2 journaling to provide file system consistency
6-5 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Using OCFS2

1. Configure shared storage.

2. Configure a private network for nodes (recommended).
3. Configure the firewall.
4. Install the Unbreakable Enterprise Kernel (UEK).
5. Install the ocfs2-tools RPM package.
6. Configure the panic_on_oops and panic kernel settings.
7. Configure the cluster layout configuration.
– Use the /sbin/o2cb utility.
8. Configure and start the O2CB cluster service.
– Use the /etc/init.d/o2cb initialization script.
9. Create OCFS2 volumes.
– Use the mkfs.ocfs2 utility.
10. Mount the OCFS2 volumes.

6-7 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Preparing for OCFS2

Node1 Node2 Node3

Allow
TCP/UDP 7777

Internet

Public network
Shared Storage
Private network

6-8 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 OCFS2 Software
• OCFS2 software has a kernel component and a user-
space component.
• The kernel component is bundled with the Unbreakable
Enterprise Kernel beginning with Oracle Linux 5.
– The kernel component includes the core file system and the
O2CB cluster stack.
• The user-space component is provided by the ocfs2-
tools RPM.
– This RPM provides the command-line interface utilities that
are used to format, tune, mount, and check the file system.
• Use the following commands to upgrade the kernel and
install the user-space RPM:
# yum install kernel-uek
# yum install ocfs2-tools

6 - 10 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Kernel Configuration

• Two kernel settings must be configured for O2CB to

function properly:
– panic_on_oops – Enable this to change a kernel oops into
a panic.
– panic – Specify the number of seconds after a panic that
the system is auto-reset.
• To manually enable panic_on_oops and set a 30-
second timeout for reboot on panic:
# echo 1 > /proc/sys/kernel/panic_on_oops
# echo 30 > /proc/sys/kernel/panic

• Configure persistent settings in /etc/sysctl.conf:

kernel.panic_on_oops = 1
Kernel.panic = 30

6 - 12 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Configuring Cluster Layout
# cat /etc/ocfs2/cluster.conf
cluster:
name = mycluster
heartbeat_mode = local
node_count = 2

node:
name = host01
cluster = mycluster
number = 0
ip_address = 192.168.1.101
ip_port = 7777

node:
name = host02
cluster = mycluster
number = 1
ip_address = 192.168.1.103
ip_port = 7777

6 - 13 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 o2cb Utility

• Use the /sbin/o2cb command to add, remove, and list

information in the cluster configuration file.
• The syntax is:
o2cb [--config-file=path] [-h|--help] [-v|--verbose] [-
V|--version] COMMAND [ARGS]
• Some of the available COMMAND parameters:
– add-cluster <cluster-name>
– add-node <cluster-name> <node-name> [--ip
<ip-address>] [--port <port>] [--number
<node-number>]
– heartbeat-mode <cluster-name> [local|global]
– list-cluster <cluster-name> [--oneline]
– list-nodes <cluster-name> [--oneline]

6 - 15 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 OCFS2 Heartbeat

• The O2CB cluster stack uses a heartbeat to determine

whether a node is dead or alive.
• If a node loses network connectivity, it fences itself off from
the cluster.
– OCFS2 panics the node that is fenced off.
• There are two heartbeat modes: local and global.
– Local – The heartbeat is started when a volume is mounted.
– Global – The heartbeat is started when the O2CB cluster
stack is started.
• The default heartbeat mode is local and is recommended
for small clusters (fewer than five mounts).
• Heartbeat mode and heartbeat regions are configured in
/etc/ocfs2/cluster.conf.

6 - 17 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 O2CB Cluster Timeouts

• An O2CB cluster has four configurable cluster timeouts:

– Heartbeat Dead Threshold – Number of two-second
iterations before a node is considered dead
– Network Idle Timeout – Time in milliseconds before a
network connection is considered dead
– Network Keepalive Delay – Maximum delay in milliseconds
before a keepalive packet is sent to another node
– Network Reconnect Delay – Minimum delay in milliseconds
between connection attempts
• Cluster timeout values are configured using the following
command:
# service o2cb configure
• Timeout values are stored in /etc/sysconfig/o2cb.

6 - 19 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 o2cb Initialization Script

• Use /etc/init.d/o2cb to configure the cluster stack

and cluster timeout values:
# service o2cb configure
• Use o2cb to check the status of the cluster stack:
# service o2cb status
• Use o2cb to start the cluster stack:
# service o2cb online

• Use o2cb to stop the cluster stack:

# service o2cb offline
• Use o2cb to unload the cluster stack:
# service o2cb unload
• Additional o2cb commands exist to manage the cluster.

6 - 20 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 mkfs.ocfs2 Utility

• Use the mkfs.ocfs2 command to create an OCFS2

volume on a device partition.
– The utility requires the O2CB cluster service to be online.
• Some of the options:
– -b <block-size> – Specifies the smallest unit of I/O
performed by the file system. The default size is 4 KB.
– -c <cluster-size> – Specifies the smallest unit of space
allocated for file data. The default size is 4 KB.
– -L <volume-label> – Specifies a name for the volume
– -T <filesystem-type> – Specifies usage of the file
system. Options are datafiles, mail, and vmstore.
• Example of creating an OCFS2 volume with a label:
# mkfs.ocfs2 -L "myvolume" /dev/xvdb1

6 - 22 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Mounting OCFS2 Volumes

• Use the mount and umount commands to mount and

unmount OCFS2 volumes, as in the following example:
# mount –L myvolume /u01

• Expect a delay when mounting and unmounting a volume.

• The O2CB cluster stack must be online before mounting.
• Use the _netdev option in /etc/fstab:
/dev/xvdb1 /u01 ocfs2 _netdev,defaults 0 0
• Start the ocfs2 initialization service to mount at boot time.
• Use the following mount options when mounting OCFS2
volumes that contain Oracle database files:
– noatime
– nointr

6 - 25 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 OCFS2 Tuning and Debugging

• The following OCFS2 utilities exist for tuning and

debugging:
– tunefs.ocfs2 – Change file system parameters.
– fsck.ocfs2 – Detect and fix on-disk errors.
– mounted.ocfs2 – Detect and list all OCFS2 volumes.
– debugfs.ocfs2 – Display file system structures.
– o2image – Back up the OCFS2 file system metadata from a
device to a specified image file.
• OCFS2 uses the virtual file system, debugfs, to expose
in-kernel information to user space.
– This allows you to list the file system cluster locks, dlm locks,
dlm state, and other states.
– Mount the file system at /sys/kernel/debug.

6 - 27 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following statements are true?

a. OCFS2 allows multiple nodes to read from and write to
files on the same disk at the same time.
b. OCFS2 is the first native cluster file system to be included
in the Linux kernel.
c. OCFS2 cannot be used in a non-clustered environment.
d. OCFS2 file systems can be mounted and used across
multiple architectures at the same time.

6 - 30 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following statements are true?

a. The cluster layout configuration file for OCFS2 is
/etc/ocfs2/cluster.conf.
b. This cluster layout configuration file needs to be present
only on the master node.
c. Use the /sbin/o2cb command to add, remove, and list
information in the cluster configuration file.
d. The configuration file includes only two sections (or
stanzas): cluster and node.

6 - 31 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following statements are true?

a. OCFS2 sends regular keepalive packets (heartbeat) to
ensure that nodes are alive.
b. There are two types of heartbeat modes: local and global.
c. In local heartbeat mode, heartbeat is on a per-mount basis
on an area on disk. The heartbeat is established when a
volume is mounted by a node.
d. With global heartbeat, you specify the devices on which
you want a heartbeat thread. The heartbeat is started
when the O2CB cluster stack is started.

6 - 32 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following are valid OCFS2 commands?

a. service o2cb configure
b. mkfs.ocfs2
c. mounted.ocfs2
d. tunefs.ocfs2

6 - 33 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Summary

In this lesson, you should have learned how to:

• Prepare for an OCFS2 configuration
• Install the OCFS2 software packages
• Configure kernel settings for OCFS2
• Configure the cluster layout
• Configure and start the O2CB cluster service
• Create an OCFS2 volume
• Mount an OCFS2 volume
• Use OCFS2 tuning and debugging utilities

6 - 34 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

 Practice 6: Overview

This practice covers the following topics:

• Preparing your environment for OCFS2
• Installing and upgrading the required software packages
• Configuring the cluster layout
• Configuring and starting the O2CB cluster stack
• Creating an OCFS2 volume
• Mounting an OCFS2 volume
• Modifying the parameters of an existing OCFS2 volume
• Using OCFS2 debugging utilities

6 - 35 Copyright © 2013, Oracle and/or its affiliates. All rights reserved.