<Insert Picture Here>

Top 10 Things You Always Wanted to Know About Automatic Storage Management Nitin Vengurlekar But Were Afraid to Ask RAC/ASM Engineering Grid Product Strategy

Agenda
ASM Overview 2 minute tour Top Ten Questions
ASM Architecture ASM Configuration & Performance Backups ACFS Migration 3rd Party Software
<Insert Picture Here>

Summary

ASM Architecture - Review

Automatic Storage Management (ASM) instance
Instance that manages the diskgroup metadata

Disk Groups
Logical grouping of disks Determines file mirroring options

ASM Disks
LUNs presented to ASM

ASM Files
Files that are stored in ASM disk groups are called ASM files, this includes database files

ASM Architecture
A Brief Overview
Oracle DB Instance

NonRAC Database Server

ASM Instance

Pool of Storage

Disk Group

Disk Group

ASM Process Architecture

A Brief Overview
RAC or NonRAC Databases
Oracle DB Instance ASM Instance Oracle DB Instance ASM Instance Oracle DB Instance ASM Instance Oracle DB Instance ASM Instance Oracle DB Instance ASM Instance

Clustered Servers Clustered Pool of Storage

Database

Database

Database

Disk Group

Disk Group

<Insert Picture Here>

Top 10 ASM Questions ASM Architecture

Top 10 ASM Questions

Q. What init.ora parameters does a user need to configure for ASM instances?

A. The default parameter settings work perfectly for ASM. The only parameters needed for 11g ASM:
PROCESSES ASM_DISKSTRING* ASM_DISKGROUPS* INSTANCE_TYPE*

Top 10 ASM Questions

Q. How does the database interact with the ASM instance and how do I make ASM go faster?

A. ASM is not in the I/O path so ASM does not impede the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.

RDBMS and ASM Instance Interaction

Server
DATABASE
(1) Database opens file
(2B) I/O Completes (3D) Initializes file

(1A) OPEN
(1B) Extent Map

ASM

(3B) Allocates file

(3A) CREATE
(3C) Extent Map

(2A) READ

(2) Database Reads file

(3) Database Creates file

(3D) Commit

Operating System

Top 10 ASM Questions

Q. Do I need to define the RDBMS FILESYSTEMIO_OPTIONS parameter when I use ASM?

A. No. The RDBMS does I/O directly to the raw disk devices, the FILESYSTEMIO_OPTIONS parameter is only for filesystems.

Top 10 ASM Questions

Q. Dont I lose all the advanced filesystem features when I move to ASM; e.g., directIO, write coalescing, and prefetch? A. Yes, but thats okay. Most of the filesystem features mentioned, though good for general file data performance, interfere and fractionalize the benefits inherently provided by the database; e.g.,
DBWR & LGWR does write coalescing, and user processes do sequential pre-fetches All IO capable processes do un-buffered IO (raw IO) because of ASM

Top 10 ASM Questions

Q. This is cool that ASM can now store Vote and OCR files. But how does CSS and CRS startup in this configuration? A. It just does There are two keys processes and a lot of crafty coding to get this to work in the correct startup sequence.

Top 10 ASM Questions

ASMCMD> pwd +DATA/rst-cluster/OCRFILE ASMCMD> ls -l Type Redund OCRFILE UNPROT

Striped COARSE

Time JUN 25 11:00:00

Sys Y

Name REGISTRY.255.718984285

ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 1048576 203824 193028 0 193028 0 Y DATA/

CSS-CRS-ASM Instance Architecture

ASM
2. Do ASM disk discovery to locate vote disk

DATA DG SPFILE Vote OCR

ASM Disk ASM Disk

CSS
1. Get ASM disk string

3. Once Vote disk located, startup CSS

4. ASM mounts DATA dg

GPnP

5. CRS open OCR in well known ASM dg location

OHASD
Agents starts GPnP, CSS and ASM

CRS

<Insert Picture Here>

Top 10 ASM Questions ASM Configuration

Top 10 ASM Questions

Q. Do I need 11gR2 Grid Infrastructure to use ASM

A. Yes. ASM is now part of Grid Infrastructure, which includes, Clusterware, ASM and ACFS. So youll to Install GI to use ASM
In 11gR2 there are two options for install GI for Standalone Server (aka Oracle Restart) and GI for Clusterware

Top 10 ASM Questions

Q. We have a 16 TB database. Im curious about the number of disk groups we should use; e.g. 1 large disk group, a couple of disk groups, or otherwise? What about a database consolidation scenario

A. For VLDBs you will probably end up with different storage tiers; e.g with some of our large customers they have Tier1 (RAID10 FC), Tier2 (RAID5 FC), Tier3 (SATA), etc. Each one of these is mapped to a diskgroup.

Top 10 ASM Questions

Q. What is the best LUN size for ASM A. There is no best size! In most cases the storage team will dictate based on their standardized LUN size. The ASM admin merely has to communicate the ASM Best Practices and application characteristics to storage folks :
Need equally sized / performance LUNs Minimum of 4 LUNs The capacity requirement The workload characteristic (random r/w, sequential r/w) & any response time SLA Using this info, and their standards, the storage folks should build a nice LUN group set for you.

Top 10 ASM Questions

Q. In 11gR2 can my RDBMS and ASM instances run different versions?

A. Yes. But since ASM is now part of GI stack, it must be at the highest version. Keep in mind, theres two components of compatibility:
Software compatibility Diskgroup compatibility attributes: compatible.asm compatible.rdbms

Disk Group Compatibility Example

Need to have compatible.asm set to 11.2.0.1 for OCR/Vote files and SPFILE in ASM ACFS also needs 11.2.0.1 compatible.asm Advance compatible.asm
ALTER DISKGROUP data SET ATTRIBUTE compatible.asm = 11.2.0.1.0

Top 10 ASM Questions

Q. Where do I run my database listener from; i.e., ASM HOME or DB HOME? A. For 11gR2, the SCAN listener is run from GI Home, and database listener from DB HOME. For pre-11gR2, it is recommended to run the listener from the ASM HOME. This is particularly important for RAC env, since the listener is a node-level resource. In this config, you can create additional [user] listeners from the database homes as needed.

<Insert Picture Here>

Top 10 ASM Questions Backups

Top 10 ASM Questions

Q. How do I backup my ASM instance?

A. Not applicable! ASM has no files to backup

Top 10 ASM Questions

Q. When should I use RMAN and when should I use ASMCMD copy? A. RMAN is the recommended and most complete and flexible method to backup and transport database files in ASM. ASMCMD copy is good for copying single files
Supports all Oracle file types In some cases. can be used to instantiate a Data Guard environment Does not update the controlfile Does not create OMF files

ASMCMD Copy
ASMCMD> ls +fra/dumpsets/expdp_5_5.dat ASMCMD> cp expdp_5_5.dat sys@rac1.orcl1:+DATA/dumpsets/ex pdp_5_5.dat source +fra/dumpsets/expdp_5_5.dat target +DATA/dumpsets/expdp_5_5.dat copying file(s)... file, +DATA/dumpsets/expdp_5_5.dat, copy committed.

<Insert Picture Here>

Top 10 ASM Questions Migration

Top 10 ASM Questions

Q. We are migrating to a new storage array. How do I move my ASM database from storage A to storage B?

A. Given that the new and old storage are both visible to ASM, simply add the new disks to the ASM disk group and drop the old disks. ASM rebalance will migrate data online. For pre-11gR2, See Note 428681.1, which covers how to move OCR/Voting disks to the new storage array

Top 10 ASM Questions

ASM_SQL> alter diskgroup DATA drop disk data_legacy1, data_legacy2, data_legacy3 add disk /dev/sddb1, /dev/sddc1, /dev/sddd1;

ASM Rebalancing
Automatic online rebalance whenever storage configuration changes Only move data proportional to storage added No need for manual I/O tuning

Disk Group DATA (legacy disks)

ASM Rebalancing
Automatic online rebalance whenever storage configuration changes Online migration to new storage

Disk Group DATA

ASM Rebalancing
Automatic online rebalance whenever storage configuration changes Online migration to new storage

Disk Group DATA

ASM Rebalancing
Automatic online rebalance whenever storage configuration changes Online migration to new storage

Disk Group DATA

ASM Rebalancing
Automatic online rebalance whenever storage configuration changes Online migration to new storage

Disk Group DATA (new disks)

Top 10 ASM Questions

Q. Is it possible to unplug an ASM disk group from one platform and plug into a server on another platform (for example, from Solaris to Linux)?

A. No. Cross-platform disk group migration not supported. To move datafiles between endian-ness platforms, you need to use XTTS, Datapump or Streams.

<Insert Picture Here>

Top 10 ASM Questions ACFS

Extending ASM to Manage ALL Data

Database RAC Application
3rd Party FS

Automatic Storage Management (ASM)

ACFS Snapshot ASM Cluster & Single Node File System (ACFS) Dynamic Volume Manager

ASM Instance Managing Oracle DB Files

ASM Files

Dynamic Volumes

ASM Disk Group

DB Datafiles DB Datafiles DB Datafiles DB Datafiles DB Datafiles DB Datafiles OCR & Voting files OCR & Voting files
ACFS FS Oracle Bin 3rd Party FS

46

What is ASM Cluster File System (ACFS)

General purpose scalable file system
Journaling, extent based Single node and cluster POSIX, X/OPEN file system solution for UNIX/Linux Windows file system solution for Windows platforms

Accessible through NAS protocols (NFS, CIFS) Leverages ASM technology Integrated with Oracle Clusterware for cluster support Multi OS platform (Linux and Windows at initial release) Integrated with Oracle system mgt tools
Oracle installation and configuration Enterprise Manager and ASM Storage mgt tools Native OS File System Management tools

47

ACFS Features
Provides filesystem snapshots (FCOW) File system integrity and fast recovery via ACFS metadata checksums and journaling. ACFS designed as a peer to peer, multi-node, shared file system model and delivers coherent data access ACFS file system is installed as a dynamically loadable OS VFS driver
Starting with RHEL5, Redhat now supports a 'white list' kernel APIs which they commit they will not change in updates or patches. APIs used by ACFS-ADVM were added to their 'white list'. Customers should be able to install an update or patch to the kernel and our drivers should not be impacted

48

Top 10 ASM Questions

Q. Is ACFS supported on other platforms besides Linux

A. Yes. Other platforms are forthcoming

Top 10 ASM Questions

Q. Can ACFS be used to store database datafiles? What about archive logs?

A. No. Currently we will not support database file to be stored in ACFS. This is due to performance reasons. Though you can do this in test/Q&A environments where performance is not essential

Top 10 ASM Questions

Q. Can I use ACFS to store BFILE data or other nondatabase related data

A. Yes. ACFS is POSIX compliant filesystem, and thus can store any file data type (besides database files )

Top 10 ASM Questions

Q. Will ACFS support other Data services, such advanced cloning, replication, de-dupe, etc..

A. Yes. ACFS Replication will be introduced in the next patchset release. Other advanced features are part of the roadmap.

<Insert Picture Here>

Top 10 ASM Questions 3rd Party Software

Top 10 ASM Questions

Q. How does ASM work with multipathing software?

A. It works great! Multipathing software is at a layer lower than ASM, and thus is transparent. You may need to adjust ASM_DISKSTRING to specify only the path to the multipathing pseudo devices.

ASM/RDBMS /dev/<pseudo> Multipath driver

/dev/sda1

/dev/sdb1

IO cloud

Controller Cache Disk

Controller

Top 10 ASM Questions

Q. Is ASM constantly rebalancing to manage hot spots?

A. NoNoNope!!

I/O Distribution
Average IOPS per disk during OLTP w orkload FG1: - cciss/c0d2 FG1: - cciss/c0d3 300 FG1: - cciss/c0d4 FG1: - cciss/c0d5 FG1: - cciss/c0d6 250 FG2: - cciss/c0d2 FG2: - cciss/c0d3 200 IOPS FG2: - cciss/c0d4 FG2: - cciss/c0d5 150 FG2: - cciss/c0d6 FG3: - cciss/c0d2 100 FG3: - cciss/c0d3 FG3: - cciss/c0d4 50 FG3: - cciss/c0d5 FG3: - cciss/c0d6 FG4: - cciss/c0d2 0 Total Disk FG4: - cciss/c0d3 FG4: - cciss/c0d4 FG4: - cciss/c0d5 FG4: - cciss/c0d6

ASM spreads file extents evenly across all disks in disk group
Since ASM distributes extents evenly, there are no hot spots

Summary:
ASM requires very few parameters to run ASM based databases inherently leverage raw disk performance No additional database parameters needed to support ASM Mixed ASM-database version support Facilitates online storage changes RMAN recommended for backing up ASM based databases Spreads I/O evenly across all disks to maximize performance and eliminates hot spots

ASM Collateral and Content

http://www.oracle.com/technology/asm

ASM 11g New Features ASM Best Practices ASM vendor papers ASM-RAC Customer Case Studies

<Insert Picture Here>

Top 10 ASM Questions Extra credit questions

Top 10 ASM Questions

Q. Is ASMLIB required on Linux systems and are there any benefits to using it?

A. ASMLIB is not required to run ASM, but it is certainly recommended. ASMLIB has following benefits:
Simplified disk discovery Persistent disk names Efficient use of system resources

Top 10 ASM Questions

Q. Is it possible to do rolling upgrades on ASMLIB in a RAC configuration

A. ASMLIB is independent of Oracle Clusterware and Oracle Database, and thus can be upgraded on its own