OCFS Oracle Cluster FileSystem for Linux

Users Guide

Table of Contents
Q. Can I install the Oracle Distribution in an OCFS partition?....................................19 Q. My partitions don't mount automatically durin boot. !hat's "ron ?....................19 Q. !hen runnin #sc$.oc#s% it returns the error &!'()I)*+ non,ero bytes a#ter the dis$ header structure-. !hat does it mean?..................................................................19 Q. !hat are the most appropriate .recommended/ tool IO and capacity analysis o# OCFS?...........................................................................................................................19 Q. Can I use oc#s on a )'S .)et"or$ 'ttached Stora e/ de0ice li$e )et'pp?..........12 Q. Can I use 34M or MD to create my OCFS #ilesystem on top o# it?.......................12 Q.I "ant to per#orm some tests usin ('C5OCFS% but I don't "ant to spend too much money on hard"are. Do I ha0e any other solution?......................................................12 Q.!hy do I ha0e to include the option 6netde0 in the #stab on (ed7at?.....................12 Q.!hat is the best "ay to archi0e the lo s to an OCFS directory?..............................12 Q.Is OCFS supported on 89 bit plat#orm li$e Itanium?................................................11 Q. Can I run oc#s on non enterprise 3inu: distributions li$e (ed7at 9?......................11 Q.Do I need any speci#ic 0ersion o# ;nited 3inu: to run OCFS?................................11 Q. Can I run the latest OCFS on a plain (ed7at 'S 1.1% "ithout any errata applied?.11 Q.!hat is the ad0anta e o# runnin oc#s a ainst ra" de0ices?....................................11 Q.!hat happen i# I ha0e to chan e the I< 'ddress o# my systems?............................11 Q.My )et"or$ Inter#ace Card had to be replaced. Do I need to do somethin ?..........11 Q.I ha0e a partition that is not mounted. 7o" do I $no" i# it is an oc#s partition or not?................................................................................................................................11 Q.Can I use my OCFS partition to store re ular #iles?.................................................11 Q.7o" much do I lose in terms o# per#ormance compared to ra" de0ices?................11 Q.7o" do I enable async I5O on Oracle usin OCFS?.................................................11 Q.7o" do I bac$up my OCFS #iles? Can I use tar or other OS command?.................11 Q.Is it possible to resi,e an e:istin OCFS partition?..................................................1= Q.I'm ha0in problems "ith OCFS. 7o" can I debu OCFS?.....................................1= Q.Can I run OCFS in a stand>alone system? !hat are the ad0anta es o# runnin it?..1= Q.I ha0e a database runnin on OCFS in a stand>alone node. !hy it is so slo" compared to other stand>alone systems runnin on e:t=?............................................1= Q.7o" can I obtain more in#ormation about OCFS?...................................................1= Q.I ha0e a customi,ed (ed7at 'S $ernel on my system. Does Oracle support OCFS and the (D?MS on it?..................................................................................................19 Q.7o" do I $no" i# my OCFS 0ersion is o##icially supported by Oracle?..................19 Q.Is OCFS is mountable on 3inu: and !indo"s 1222 simultaneously?.....................19

Introduction.
OCFS is a shared disk cluster filesystem. The current version (version 1) released for Linux is specifically desi ned to alleviate the need for mana in ra! devices. "t can contain all the oracle datafiles# archive lo files and controlfiles. "t is ho!ever not desi ned as a eneral purpose filesystem. This document descri$es the steps re%uired to install OCFS on Linux and !ill also ive uidelines for optimi&ations and some more in depth understandin of ho! the filesystem !orks.

Downloading OCFS.
OCFS can $e do!nloaded from 'http())oss.oracle.com)pro*ects)ocfs+ for the follo!in distri$utions ( ,ed-at .dvanced Server /.1 and 0nited Linux 1.1 (Conectiva# SuSe# Tu$oLinux and SCO). Oracle officially supports the Oracle data$ase on OCFS if it is installed from the $inary packa es that are availa$le for do!nload. "f the user decides to do!nload the source code and compile it# then there !ill $e no formal support provided $y Oracle. "n addition to the OCFS $inaries# !e also provide a collection of utilities (cp# dd# tar and textutils) that ena$le O23",4CT. The updated tools are recommended to $e used as they make more efficient use of the operatin system in con*unction !ith OCFS. 5inary distri$utions for ia6/ and ia78 can $e found under each one of the supported platforms. There are $asically three rpm packa es to do!nload in order to install OCFS. Those packa es are( OCFS>Support OCFS>@ools OCFS Module 5efore do!nloadin the OCFS 9odule# make sure it is compati$le !ith the kernel version in use (uname :a).

Installing OCFS
"nstallin OCFS is an easy process. .fter do!nloadin the packa es# issue the follo!in command on the directory !here the packa es !ere

do!nloaded(
# rpm -Uhv ocfs*.rpm

This !ill install the support tools# the actual kernel module for the filesystem and a raphical confi uration tool.

Automatically mount OCFS during boot.

.fter installin the OCFS packa es# verify that the module !ill $e properly initiali&ed on startup usin the command(
# chkconfig list !gr"p -i ocfs

"f the output looks like (

ocfs #$off %$off &$off '$on ($on )$on *$off

Then# no action is re%uired# $ut if the output doesn;t sho! 'on+ on 6#8 and < (rc levels)# issue the follo!in command to ena$le automatic startup of the ocfs durin $oot(
# chkconfig ocfs on

Configuring OCFS.
OCFS depends on a node specific confi uration file. This file is named ocfs.conf and it is located in the /etc directory= it can $e enerated automatically or manually usin the ocfstool. 3urin the next sections# the )etc)ocfs.conf file and $oth methods of confi urin !ill $e descri$ed in detail. This file is needed on every node in the cluster and it is hi hly recommended to use ocfstool to confi ure each node.

1.1.1 /etc/ocfs.conf file.

The )etc)ocfs.conf file can have the follo!in parameters( ip2address > Specify the "? .ddress to $e used $y the OCFS 3L9. The server must $e a$le to reach all nodes participatin on the cluster throu h the interface related to the "? .ddress specified in this field. ip2port > Specify the port to $e used $y the OCFS 3L9 to communicate !ith the other nodes in the cluster. The port must $e the same on all nodes in the cluster. @ode2name > Specify the server hostname associated to the "? .ddress specified in the ip2address parameter. comm2votin > Specify !hich method for votin is oin to $e used $y OCFS. "f set to 1 (default)# it means that OCFS !ill $e votin usin the disk# if set to 1# it !ill $e usin the net!ork to vote. "f the OCFS is set to use net!ork to vote and it $ecomes unavaila$le for some reason# it !ill

automatically (and transparently) fall $ack to disk. 4na$lin comm2votin !ill drastically increase performance for re ular filesystem operations such as rm# mv# mkdir etc. uid > This parameter is automatically filled $y the ocfs+uid+g"n utility and should never $e manually chan ed.

1.1.! "utomatic Configuration using ocfstool.

"n order to perform the automatic confi uration !ith ocfstool# it is necessary that the A0" environment is properly set and ena$le. Open a A0" session as root and execute the command 'ocfstool+. 9ake sure the 3"S?L.B varia$le is set $efore startin the tool. Chen the ocfstool !indo!(See Illustration 1) open# invoke the Aenerate Confi (See "llustration D) task $y either pressin the key se%uence ECT,L:AF or usin the menu# clickin on TasksFAenerate Confi . For more information on the parameters# check the item Aenerate Confi in the ocfstool section.

1.1.# $anual configuration

.lthou h the manual confi uration is provided# Oracle stron ly recommends the usa e of the ocfstool since it does provide a relia$le# consistent and easy !ay to properly confi ure OCFS. To manually confi ure the OCFS# create the file /etc/ocfs.conf $ased on the sample $elo! on each one of the nodes participant of the cluster. 9ake sure the parameters are consistent amon nodes.
# ocfs config # Ensure this file exists in /etc directory # node_name = ca-test2.us.oracle.com ip_address = 10.0.0.1 ip_port = 7000 comm_ oting = 1

.fter the file is created# execute the utility ocfs%uid%gen !ith the c ar ument as root in order to enerate the uni%ue identification key necessary for the OCFS to identify itself in the cluster. .fter the eneration of the uid key# the )etc)ocfs.conf file should looks like(
# # ocfs config # Ensure this file exists in /etc# node_name = ca-testt2.us.oracle.com ip_address = 10.0.0.1 ip_port = 7000

comm_ oting = 1 guid = !"2!!#!!1"$"2%&'($""000)(70$'E7%

,oading ocfs.
The )etc)init.d)ocfs startup script is provided in the packa e and automatically loads the OCFS module if there is an entry for it in )etc)fsta$. 0sin this startup script is the preferred method to load OCFS as it does all the verification $efore loadin the module and mountin the partitions. "f for some reason there still is a need of manually load the OCFS module# *ust issue the command load2ocfs as root. "f the process is successfully executed# it should sho! a messa e like(
# load+ocfs -sbin-insmod ocfs nod"+nam".ca-t"st&.us.oracl".com ip+addr"ss.%#.#.#.% ip+port./### cs.%0)1 guid.12&11*11%2C2&)3F(C22###'(/#CF4/) Using -lib-modul"s-&.(.1-"-"nt"rpris"-A25-ocfs-ocfs.o

Formatting an OCFS 6artition.

Similar to confi urin the OCFS# there are t!o !ays to format an OCFS partition. One is usin the 'ocfstool+ command and the A0" environment# and the other is usin the 'mkfs.ocfs+ command from the shell prompt. 5oth commands needs to $e executed as root. @one of the ocfs utilities !ill partition the disk at any time. So# $efore formattin # choose the utility of your preference and partition the disk accordin to the needs of your implementation# and make sure the disk)partition is not $ein used $y anythin else to avoid data loss. Cith OCFS you have to format a partition only once on one node# after that every node !ill $e a$le to mount this filesystem. (of course the device needs to $e visi$le on every node in the cluster)

1.1.& Format using ocfstool.

Start $y invokin the ocfstool command like descri$ed in the section '6././ .utomatic confi uration usin ocfstool+. .fter ettin the ocfstool !indo!(See illustration 7)# press either the se%uence key ECT,L:FF or from the menu# choose TasksFFormat. Fill in all #ields and clic$ the &OAbutton. For more in#ormation on the parameters% chec$ the format item under ocfstool section.

1.1.' Format using m(fs.ocfs.

For people !ho do not have a A0" availa$le# use mkfs.ocfs. "f a A0" is availa$le# the preferred method to format an OCFS partition is to use the ocfstool. The mkfs.ocfs command has the follo!in syntax(
# mkfs.ocfs usag"$ mkfs.ocfs -b block-si7" 8-C9 8-F9 8-g gid9 8-h9 -, volum"-lab"l -m mount-path 8-n9 8-p p"rmissions9 8-:9 8-u uid9 8-;9 d"vic"

-b 2lock si7" in kilo byt"s -C Cl"ar all data blocks -F Forc" format "<isting OCFS volum" -g =53 for th" root dir"ctory -h >"lp -, ;olum" lab"l -m 6ath ?h"r" this d"vic" ?ill b" mount"d -n @u"ry only -p 6"rmissions for th" root dir"ctory -: @ui"t "<"cution -u U53 for th" root dir"ctory -; 6rint v"rsion and "<it

Chen usin the mkfs.ocfs command# the user has to provide all the information that is prompt $y the ocfstool utility. The usa e of the ':C+ ar ument !ill force the mkfs.ocfs to clear all $locks. 3ependin on the si&e of the partition# it may $e a lon process. The ':F+ ar ument should $e used only if the partition !as previously formatted as an OCFS. The ':$+ ar ument specify the $locksi&e that the partition !ill $e formatted. The $locksi&e specifies the maximum si&e of the partition that can $e mounted. "t oes from 8k to 19 and allo!s volumes from 6/A$ up to GT$. Format !ith 1/Gk$ $locksi&e is optimal si&e. Si&es $et!een 8k$ and 1m$ are supported. The smaller $locksi&es !ill have a performance penalty# $ut !ill $e useful for the future !hen !e !ill support re ular files. 1/Gk$ $locksi&e means that every file created !ith content uses up a minimum of 1/Gk$ space on disk. 4ven if there is only 1 $yte of data in the file. The filesystem !ill allocate chunks of space in E$locksi&eF chunks.

The example $elo! sho! an ordinary partition $ein formatted and its output.
# mkfs.ocfs -F -b %&0 -g dba -u oracl" -, -u#% -m -u#% - p //) -d"v-sdb% Ch"cking h"art b"at on volum" ..........

Cl"aring volum" h"ad"r s"ctors...Cl"ar"d volum" h"ad"r s"ctors Cl"aring nod" config s"ctors...Cl"ar"d nod" config s"ctors Cl"aring publish s"ctors...Cl"ar"d publish s"ctors Cl"aring vot" s"ctors...Cl"ar"d vot" s"ctors Cl"aring bitmap s"ctors...Cl"ar"d bitmap s"ctors Cl"aring data block...Cl"ar"d data block Ariting volum" h"ad"r...Arot" volum" h"ad"r #

Bount OCFS partitions. 1.1.) $ounting manually.

.t least for the first time# it is ood to have the partitions mounted manually instead of automatically. That;s $ecause the user has control over all the process and can check if everythin !orks fine. To mount the ne! OCFS partition# use the mount command !ith the ':t+ ar ument specifyin 'ocfs+ in front of it. The example $elo! sho!s ho! an OCFS partition is mounted(
# mount -t ocfs -d"v-sdb% -u#%

1.1.* $ounting automatically

To mount the OCFS partitions automatically# *ust add the partition information to the /etc/fstab file. The example $elo! sho! an entry for ,ed-at .S /.1.
-d"v-sdb% -d"v-sdd% -u#% -u#& ocfs ocfs +n"td"v +n"td"v

Tools
ocfstool
is a A0" frontend for mana in and de$u in OCFS volumes on the system= and also the preferred method for mana in OCFS. One can
ocfstool

mount and unmount volumes# format partitions# vie! information and individual files# see the current node map% and bloc$ bitmap. '#ter startin the tool you are presented "ith a "indo" consistin sections or se ments. o# 1

@he top portion maps all $no"n partitions that are OCFS #ormatted and allo" users to mount and unmount these partitions. @he mount operation "ill try to mount the #ilesystem to the mountpoint speci#ied durin the #ormat operation. 3i$e any other #ilesystem the umount operation "ill only succeed i# there is no process usin it. @he bottom portion has a series o# #olders that are di0ided by areas o# mana ement and bro"sin . It "ill sho" the in#ormation related to the de0ice selected in the top portion o# the screen. @he in#ormation that can be obtained is di0ided in + Aeneral B Contain eneral in#ormation about the #ilesystem .See "llustration 1/. 3i$e oc#s 0ersion that #ormatted the de0ice% mountpoint% si,e o# the #ilesystem% number o# e:tents% userid and roupid "ith pri0ile es on the #ilesystem and the appropriate permission. File Listin B !ill sho" #ile in#ormation about the #ilesystem . See "llustration //. Selectin a #ile or directory% one can see its in#ormation% li$e si,e% allocation unit% o"nership and protection. One can also see "hich nodes ha0e the partition mounted at that speci#ic point in time. Confi ured @odes B On this #older% one can see "hich nodes ha0e the selected partition .See "llustration 6/. 5itmap Hie! B On this #older% one can see the bitmap allocation #or the selected partition .See "llustration 8/. Free Space B On this #older% one can see a list o# #ree space #or the selected partition .See Illustration C). This list !ill sho! the si&e and the $itI of the free space.

Illustration 1

Illustration 2

Illustration 3

Illustration 4

Illustration 5

"n addition to the t!o portions sho!ed# there are the tasks that can $e reached $y the menu. The availa$le tasks on the menu are(

1.1.+ Format.
@here are t"o "ays to in0o$e the #ormat "indo"% one is pressin the ECT,LJFF $ey seDuence and the other is in0o$in by selectin the menu TasksFFormat. !hen the #ormat "indo" is in0o$ed% the #ollo"in options "ill be a0ailable+ 3evice B From the pull>do"n menu% select the de0ice that is oin to be OCFS #ormatted. Ma$e sure the de0ice is not in use by any other application or #ilesystem be#ore proceedin . 5locksi&e B Select the bloc$si,e on "hich the partition is oin to be #ormatted. 4alid 0alues are 9$% E$% 18$ =1$% 89$% 11E$% 1C8$% C11$ and 1219$. @he optimal si,e su ested by Oracle is 11E$. Smaller bloc$si,e can be selected but they "ill carry some per#ormance penalty. @his is also oin to limit the si,e o# the partition that can be used "hen #ormattin "ith OCFS .=1*b to E@b/. Holume La$el B @his "ill speci#y the 0olume label. @his is use#ul i# the user "ants to use the 0olume label to mount the #ilesystem. 9ountpoint B Speci#y the location "here the partition is oin to be mounted. @he mount point must e:ist on all nodes in the cluster that "ill share

the de0ice. 0ser B Speci#y the user that "ill o"n the #ilesystem. !hen the #ilesystem is mounted% it is automatically o"ned by the user speci#ied in this #ield. Aroup B Speci#y the roup that the #ilesystem "ill belon to. Similar to the ;ser #ield. ?rotection B Sets the de#ault permission o# the #ilesystem "hen mounted. ;sually set to 2FCC. Clear all 3ata 5locks B !hen chec$ed% this option "ill ma$e oc#s #ormat bloc$ by bloc$% ,eroin all the #ilesystem. @his option "ill increase the time necessary to #ormat the partition considerably. .S3O!/ Force > This options needs to $e checked if the partition to $e formatted !as previously formatted $y OCFS.

Illustration 6

1.1., -enerate Config.

"nvokin this task !ill promptly enerate the )etc)ocfs.conf file accordin to the server confi uration. 9andatory fields !ill $e automatically filled !ith the server information. There is no need to do any further steps as the process !ill automatically enerate the uid in the confi uration file.. !hen the *enerate Con#i "indo" is in0o$ed% the #ollo"in options "ill be a0ailable+ "nterface B Select the inter#ace that is oin to be used by the OCFS to enerate uniDue identi#ication. @his inter#ace "ill also be used by the OCFS Distributed 3oc$ Mana er .D3M/ in #uture implementation. Ma$e sure that all nodes that "ill be sharin the OCFS can be reached throu h the selected inter#ace. ?ort B Select the port number that is oin to be used to communicate to the

other nodes. 'ny non>used port can be assi ned to OCFS. Ma$e sure all nodes ha0e the same port con#i ured. @ode:@ame B Select the node name associated to the inter#ace selected. ?referred @ode @um$er B @his is an optional #ield. I# there is a need to ha0e an speci#ic node assi ned to a number% #ill this #ield% other"ise% lea0e it to the OCFS.

Illustration 7

4<tfind"r
is an utility to report a list of conti uous free extents in the ocfs filesystem. "t !ill sho! ho! fra mented the filesystem is after usa e.
4<tfind"r

3"bugocfs
The d"bugocfs utility is used to extract metadata information from the OCFS partition. This information is useful to help dia nose eventual pro$lems that may arise. The de$u ocfs syntax is (
d"bugocfs$ Usag"$ d"bugocfs 8-C9 8-h9 8-g9 8-l9 8-v rang"9 8-p rang"9 8-d -dir-nam"9 8-f -fil"-nam" 8-s -path-to-fil"99 8-a rang"9 8-A rang"9 8-b rang"9 8-2 rang"9 8-r rang"9 8-c rang"9 8-, rang"9 8-B rang"9 8-n nod"num9 -d"v-nam"

-h$ volum" h"ad"r -g$ global bitmap -l$ full listing of all fil" "ntri"s -v$ vot" s"ctor -&$ print 0-byt" numb"r as & (-byt" numb"rs -p$ publish s"ctor -d$ ocfs+dir+nod" structur" for a giv"n path -f$ ocfs+fil"+"ntry structur" for a giv"n fil" -F$ ocfs+fil"+"ntry and ocfs+"<t"nt+group structur"s for a giv"n fil" -s$ suck fil" out to a giv"n location -a$ fil" allocation syst"m fil"

-A$ dir allocation syst"m fil" -b$ fil" allocation bitmap syst"m fil" -2$ dir allocation bitmap syst"m fil" -r$ r"cov"r log fil" syst"m fil" -c$ cl"anup log syst"m fil" -,$ vol m"tadata log syst"m fil" -B$ vol m"tadata syst"m fil" -n$ p"rform action as nod" numb"r giv"n -d"v-nam"$ r"adabl" d"vic" rang"$ nod" numb"rs to insp"ct D#-'%EF commas and dash"s ok "<. #-'F)F%(-%/

5elo!# some examples of the de$u ocfs output Holume -eader.

*root+ca-test2 root,# de-ugocfs -h /de /sd-12 dis.header/ ersion = 1.2 signature = 0racle$'1 mount_point = /u01 serial_num = 0 de ice_si2e = (1!(3#0%(( start_off = 0 -itmap_off = %#)20 pu-l_off = 2)%%2 ote_off = )!!)# root_-itmap_off = 0 data_start_off = 1)#30#( root_-itmap_si2e = 0 root_off = 2(1##(0 root_si2e = 0 cluster_si2e = 1)1072 num_nodes = )2 num_clusters = )1!3! dir_node_si2e = 0 file_node_si2e = 0 internal_off = 1)#30#( prot_-its = (!) uid = 1011 gid = 1011 excl_mount = -1 olumela-el/ curr_master = 0 file_loc. = 0$'1_&45_60_40$7 oin_node_map = 00000000000000000000000000000000 se8_num = 0 la-el = /u01 la-el_len = ( *root+ca-test2 root,#

Hote Sector.
*root+ca-test2 root,# de-ugocfs - 0-2 /de /sd-12 ote0/ se8_num = 0 dir_ent = 0 open_handle = 6o 9ote0 = :0x00000000; 9ote1 = :0x00000000; 9ote2 = :0x00000000; ote1/ se8_num = 0 dir_ent = 0 open_handle = 6o 9ote0 = :0x00000000; 9ote1 = :0x00000000; 9ote2 = :0x00000000; ote2/ se8_num = 0 dir_ent = 0 open_handle = 6o 9ote0 = :0x00000000; 9ote1 = :0x00000000; 9ote2 = :0x00000000;

*root+ca-test2 root,#

ocfs2file2entry structure
*root+ca-test2 root,# ls /u01/oradata/smd-/d-files/control01.ctl /u01/oradata/smd-/d-files/control01.ctl *root+ca-test2 root,# de-ugocfs -f /oradata/smd-/d-files/control01.ctl /de /sd-12 fileinfo/ 6ame = /oradata/smd-/d-files/control01.ctl curr_master = 0 file_loc. = 0$'1_&45_E6<"4E_$<$=E_40$7 oin_node_map = 10000000000000000000000000000000 se8_num = 0 local_ext = true granularity = -1 filename = control01.ctl filename_len = 1) file_si2e = 12%0!13( alloc_si2e = 12%32!12 attri-s = 0$'1_<>>?@"_?EA prot_-its = 1_@?B1? 1_@CB1? 1_@DB1? 1_@?A?E 1_@CA?E 1_@DA?E 1_@?0>= 1_@D0>= uid = 1011 gid = 1011 create_time = 'ri <ug 2! 1#/(!/10 200)

modify_time = 'ri <ug 2! 1#/(!/10 200) dir_node_ptr = 2!(0!23 this_sector = 2!(1((0 last_ext_ptr = 0 sync_flags = 0$'1_1F6$_'4<A_9<4@& lin._cnt = 0 next_del = 0 next_free_ext = 1 extent*0,.file_off = 0 extent*0,.num_-ytes = 12%32!12 extent*0,.dis._off = (#((3#( extent*1,.file_off = 0 extent*1,.num_-ytes = 0 extent*1,.dis._off = 0 extent*2,.file_off = 0 extent*2,.num_-ytes = 0 extent*2,.dis._off = 0 *root+ca-test2 root,#

Su..ort /tilities
"n this section# a description of each one of the utilities $elon in to the ocfs:support packa e. These packa es are the minimum necessary to make the OCFS !ork properly.

-"tc-init.d-ocfs
This script is responsi$le to automatically load the ocfs module# perform a sanity check on the environment and mount all the ocfs partitions listed in /etc/fstab.

-sbin-fsck.ocfs
?erform a filesystem check on the partition. .ll nodes of a cluster that share a specific device need to have them unmounted $efore runnin fsck.ocfs# if one of the nodes has the partition mounted# the fsck.ocfs !ill fail. The command syntax is(
usag"$ fsck.ocfs 8-G9 8-v9 d"vic"

-G Go ?rit" -; ;"rsion -v ;"rbos" -: @ui"t

-sbin-load+ocfs
The load+ocfs script !ill call the insmod command and load the ocfs module !ith all the correct parameters.

-sbin-mkfs.ocfs
This command is used to $uild a the filesystem structure in a partition. The syntax of the mkfs.ocfs command is (
usag"$ mkfs.ocfs -b block-si7" 8-C9 8-F9 8-g gid9 8-h9 -, volum"-lab"l -m mount-path 8-n9 8-p p"rmissions9 8-:9 8-u uid9 8-;9 d"vic"

-b 2lock si7" in kilo byt"s -C Cl"ar all data blocks -F Forc" format "<isting OCFS volum" -g =53 for th" root dir"ctory -h >"lp -, ;olum" lab"l -m 6ath ?h"r" this d"vic" ?ill b" mount"d -n @u"ry only -p 6"rmissions for th" root dir"ctory -: @ui"t "<"cution -u U53 for th" root dir"ctory -; 6rint v"rsion and "<it

-sbin-mount"d.ocfs
This command mount"d.ocfs is used $asically to check !hich nodes have a specific device mounted. The command syntax is (
usag"$ mount"d.ocfs Hd"vic"I

-sbin-ocfs+uid+g"n
The ocfs+uid+g"n command is usually used once durin the confi uration of the OCFS after it is installed. "ts function is to enerate the uni%ue key# in the /etc/ocfs.conf file# that identify the node in the OCFS node mana er. The command syntax is (
Usag"$ ocfs+uid+g"n -c ocfs+uid+g"n -r

-c

Cr"at" a n"? =U53 for a n"? nod" b"for" it "nt"rs th"

clust"r. -r J"cov"r th" =U53F ins"rting th" n"? BAC addr"ss. Khis op"ration is only for "<isting nod"s ?hos" BAC addr"ss has chang"d.

0est 1ractices
For ,ed-at .S /.1 o to http())oss.oracle.com)pro*ects)ocfs)dist)documentation),-.S2$est2practices.html. For0nited Linux# o to http())oss.oracle.com)pro*ects)ocfs)dist)documentation)0L2$est2practices. txt.

Fre2uently as(ed 3uestions 4 "nswers

Q. Can I install the Oracle Distribution in an OCFS partition?
A. No. OCFS version 1 supports only database files (datafiles, logfiles, controlfiles, archivelog files, srvm configuration file and Oracle Cluster anager !uorum file".

Q. My partitions don't during boot. What's rong?

mount

automatically

A. #he ocfs script on $etc$init.d directory is not enable. %se the ch&config command to verify that and enable it. 'nstructions on ho( to enable it are listed under item ).1 * Automatically mount OCFS during boot.

Q. When running !sc".oc!s# it returns the error $!'()I)*+ non,ero bytes a#ter the dis$ header structure-. !hat does it mean?
A. +arlier releases of m&fs.ocfs did not clear up 1st bloc& entirely. ,e-ve since fi.ed the issue in the format utility. As far as functioning goes, one can safely ignore this message.

Q. What are the most appropriate %recommended& tool IO and capacity analysis o! OCFS? A. /our usual tools (sar, vmstat, iostat" (ill

(or&, it filesystem.

handles

io

0ust

li&e

any

other

Q. Can I use oc!s on a '(S %'et or" (ttached Storage& de)ice li"e 'et(pp?
A. No. NAS (or&s through net(or& and the physical volume and filesystem is under the fileserver control. OCFS needs to use a direct attached storage device or SAN (Storage Area Net(or&" (here the local server has total control over the physical volumes.

Q. Can I use *+M or !ilesystem on top o! it?

MD

to

create

my

OCFS

A. 12 and 3 are not cluster a(are and it ma&es it very difficult to manage in a cluster environment. No tests have been performed using 12 or 3. 't may (or&, but it also may not. Corruptions are li&ely to happen since they are not cluster a(are soft(are.

Q.I ant to per!orm some tests using ,(C-OCFS# but I don't ant to spend too much money on hard are. Do I ha)e any other solution?
A./es. %sers can run on fire(ire. Oracle does provide a special &ernel and ocfs modules to run on fire(ire. ore information can be found at http4$$oss.oracle.com$pro0ects$fire(ire$.

Q.Why do I ha)e to include the option .netde) in the !stab on ,ed/at?

A.#he

5netdev parameter is used (hen the filesystem resides on a device that re!uires net(or& access, it is used to prevent the system from attempting to mount these filesystems until the net(or& has been enabled on the system. Q.What is the best OCFS directory? ay to archi)e the logs to an

A.#he ideal is to create a directory for each one of the nodes that (ill be archiving on OCFS. #his (ay, the logs (ill be &ept organi6ed and there (ill be no concurrency (hen t(o or more nodes archive at the same time. #his doesn-t mean there (ill be only archivelogs of the node on its directory, because Oracle does allo( a node to

perform an archive operation if a specific node is do(n for long time to reduce recovery time. Q.Is OCFS Itanium? supported on 01 bit plat!orm li"e

A./es, you can find specific pac&ages for ia)7 and ia89. Q. Can I run oc!s on non distributions li"e ,ed/at 3? enterprise *inu2

A.OCFS is primarily developed for +nterprise systems and therefore not officially supported on platforms other than :ed;at Advanced Server 7.1 or %1. #he binaries can be compiled from source at your o(n ris&. Q.Do I need any speci!ic )ersion o! 4nited *inu2 to run OCFS? A. /es. #he latest pac&age of OCFS (ill (or& (ith %1 S<1, but Oracle strongly recommends to run on S<7 or S<7a. Q. Can I run the latest OCFS on a plain ,ed/at (S 5.6# ithout any errata applied? A.No. =efore the errata e.17, there (as a specific module for each one of the errata releases. After e.17, the module is generic, (or&ing (ith any of the erratas released. Oracle strongly recommends the usage of the latest errata available. Q.What is the ad)antage o! running oc!s against ra de)ices? A.#here are many advantages on running OCFS. #he biggest and more important is that it loo&s and feels li&e a regular filesystem, so users don-t have to (orry (ith all the ra( devices administration difficulties. Also, users don-t have to create a partition for each datafile, a fe( partitions or even a single partition, depending on the needs and storage configuration, may be able to handle big databases. 't also eliminate the restriction of 7>> ra( partitions limit on 1inu., giving almost endless number of datafiles that can be created.

Q.What happen i! I ha)e to change the I7 (ddress o! my systems? A.Changing '< address on the same interface used by the ocfs (ill re!uire you to change the '< Address specified in the $etc$ocfs.conf file only. Q.My 'et or" Inter!ace Card had to be replaced. Do I need to do something? A./es. :un the utility ?ocfs5uid5gen *r@ to update
the $etc$ocfs.conf file (ith the ne( uid.

Q.I ha)e a partition that is not mounted. /o I "no i! it is an oc!s partition or not?

do

A. :un the command ?debugocfs *h $dev$....@ as root. 'f it is an ocfs partition, the signature (ill sho( ?OracleCFS@. For all other type of filesystem, it (ill 0ust return something else, not necessarily the fstype.

Q.Can I use my OCFS partition to store regular !iles?

A.1i&e Oracle 3istribution, regular files are not supported on OCFS at the moment. #his is planned for OCFS 7.

Q./o much do I lose in compared to ra de)ices?

terms

o!

per!ormance

A.Not too much. ,ithout async '$O, performance have been fluctuating from A to >B of ra( devices. Considering all the benefits of a filesystem against ra(, the loss is not that big. %sing async '$O, ocfs has sho(n to be about >B faster than ra( devices.

Q./o

do I enable async I-O on Oracle using OCFS?

A.First of all, it is necessary to relin& the Oracle :3= S to enable asynchronous '$O (cd CO:AC1+5;O +$rdbms$libD ma&e *f ins5rdbms.m& async5on". #hen add to the init.ora file, the parameters 4
filesystemio_options=setall _d-Gr_async_io=>?BE tape_asynch_io=true dis._asynch_io=true

Q./o do I bac"up my OCFS !iles? Can I use tar or other OS command?

A.#he best (ay to perform an Oracle bac&up is to use : AN. Since a lot of users have their o(n scripts that usually uses tar or other OS commands, (e have made available a set of commands that support direct5io, so that users can ta&e hot bac&up using tar, cp, dd. Eo to

http4$$oss.oracle.com$pro0ects$coreutils$files$ to get the fileutils$coreutils appropriate to your environment. Note that for :;AS 7.1 and S1+SF, the pac&age is called fileutils and for :;+1), it is called coreutils.

Q.Is it possible partition?

to

resi8e

an

e2isting

OCFS

A./es, tuneocfs command (ill do that. <lease refer to the tuneocfs manpages for additional information on synta..

Q.I'm ha)ing problems OCFS?

ith OCFS. /o

can I debug

A.One needs to be careful enabling debug on ocfs because it usually fills the $var$log$messages file very !uic&ly. #o enable OCFS tracing4
"cho -% I -proc-sys-k"rn"l-ocfs-d"bug+l"v"l "cho -% I -proc-sys-k"rn"l-ocfs-d"bug+cont"<t

#o disable OCFS tracing4

"cho # I -proc-sys-k"rn"l-ocfs-d"bug+l"v"l "cho # I -proc-sys-k"rn"l-ocfs-d"bug+cont"<t

Q.Can I run OCFS in a stand9alone system? What are the ad)antages o! running it?
A./es. #here is no problems on running OCFS in a stand* alone system since you use it for database files only (3atafile, logfiles, controlfiles, srvm configuration file and Oracle Cluster anager !uorum file". #he advantage of using it on stand*alone is that if one needs to upgrade to a :AC environment, it (ill be a smooth transition, basically adding a ne( node, recompiling the Oracle :3= S &ernel to enable :AC mode and add a ne( online log thread.

Q.I ha)e a database running on OCFS in a stand9 alone node. Why it is so slo compared to other stand9alone systems running on e2t:?
A.OCFS does direct5io, (hich bypass the 1inu. cache. +.t) in other hands ma&es full usage of the 1inu. cache. 3epending on the memory available, it is possible that one (ill mostly find the data in cache, not having to fetch from the dis&. =ut this condition happens only if there are enough free memory for cache. As you use more and more memory, performance drops significantly.

Q./o

can I obtain more in!ormation about OCFS?

A.Eo to http4$$oss.oracle.com$ocfs. /ou (ill be able to find documents, sources, binaries and all sort of information.

Q.I ha)e a customi8ed ,ed/at (S "ernel on my system. Does Oracle support OCFS and the ,D;MS on it?
A.No. Oracle only supports standard &ernels for OCFS and Oracle :3= S. 't (ould be impossible to trac& and reproduce any problem (ithout &no(ing all the customi6ations that (ere performed in a specific installation.

Q./o do I "no i! my OCFS )ersion is o!!icially supported by Oracle?

A.Chec& the file $var$log$messages for a message that loo&s li&e ?Oracle Cluster FileSystem 1.A.G*<:O3 ,ed Hul )A 1849F4AA <3# 7AA) (build 8F8e>97IG7ebee99eaa1f" Q.Is OCFS is mountable on *inu2 and Windo s 5<<< simultaneously? A.No. #he on*dis& format is not the same. 's similar... but not the same.

"..endix "
Ocfs:support packa e "nformation(
6ame / ocfs-support ?elocations/ :not relocata-le; 9ersion / 1.0.! 9endor/ 0racle $orporation ?elease / % "uild &ate/ Ced )0 Hul 200) 07/(3/(0 E5 E&> @nstall date/ :not installed; "uild =ost/ ca--uild1.us.oracle.com Aroup / 1ystem En ironment/7ernel 1ource ?E5/ ocfs-2.(.!-e-1.0.!-%.src.rpm 1i2e / #!30!! 4icense/ AE4 Eac.ager / no-ody Ino-ody+oracle.comJ B?4 / http///ocfs.otncast.otnxchange.oracle.com/ser lets/EroKect=ome 1ummary / 1upport programs for the 0racle $luster 'ilesystem &escription / 1upport programs for using the 0racle $luster 'ilesystem. /etc/init.d/ocfs /s-in/fsc..ocfs /s-in/load_ocfs /s-in/m.fs.ocfs /s-in/mounted.ocfs /s-in/ocfs_uid_gen

Ocfs:tools packa e information(

6ame / ocfs-tools ?elocations/ :not relocata-le; 9ersion / 1.0.! 9endor/ 0racle $orporation ?elease / % "uild &ate/ Ced )0 Hul 200) 07/(3/(0 E5 E&> @nstall date/ :not installed; "uild =ost/ ca--uild1.us.oracle.com Aroup / 1ystem En ironment/7ernel 1ource ?E5/ ocfs-2.(.!-e-1.0.!-%.src.rpm 1i2e / 1#!)31 4icense/ AE4 Eac.ager / no-ody Ino-ody+oracle.comJ B?4 / http///ocfs.otncast.otnxchange.oracle.com/ser lets/EroKect=ome

1ummary / >ools for managing the 0racle $luster 'ilesystem &escription / >ools to manage the 0racle $luster 'ilesystem /usr/-in /usr/-in/cdslctl /usr/-in/de-ugocfs /usr/-in/extfinder /usr/-in/ocfstool /usr/share /usr/share/man /usr/share/man/man1 /usr/share/man/man1/cdslctl.1.g2 /usr/share/man/man1/ocfstool.1.g2

Ocfs 9odule ?acka e "nformation(

6ame / ocfs-2.(.!-e-enterprise ?elocations/ :not relocatea-le; 9ersion / 1.0.! 9endor/ 0racle $orporation ?elease / % "uild &ate/ Ced )0 Hul 200) 07/(3/(0 E5 E&> @nstall date/ :not installed; "uild =ost/ ca--uild1.us.oracle.com Aroup / 1ystem En ironment/7ernel 1ource ?E5/ ocfs-2.(.!-e-1.0.!-%.src.rpm 1i2e / 110(0#! 4icense/ AE4 Eac.ager / no-ody Ino-ody+oracle.comJ B?4 / http///ocfs.otncast.otnxchange.oracle.com/ser lets/EroKect=ome 1ummary / >he 0racle $luster 'ilesystem for enterprise systems. &escription / 0$'1 is the 0racle $luster 'ilesystem. >his pac.age is compiled for symmetric processor .ernels on machines Gith more than (A" of ?<5. /li-/modules/2.(.!-e-enterprise-<"@/ocfs /li-/modules/2.(.!-e-enterprise-<"@/ocfs-noaio /li-/modules/2.(.!-e-enterprise-<"@/ocfs-noaio/ocfs.o /li-/modules/2.(.!-e-enterprise-<"@/ocfs/ocfs.o