Вы находитесь на странице: 1из 134

THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY.

COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Activity Guide
D74942GC40
Administration

Edition 4.0 | February 2016 | D94670


Learn more from Oracle University at oracle.com/education/
Oracle Solaris Cluster 4.x

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Disclaimer

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
This document contains proprietary information and is protected by copyright and other intellectual property laws. You may copy and
print this document solely for your own use in an Oracle training course. The document may not be modified or altered in any way.
Except where your use constitutes "fair use" under copyright law, you may not use, share, download, upload, copy, print, display,
perform, reproduce, publish, license, post, transmit, or distribute this document in whole or in part without the express authorization
of Oracle.

The information contained in this document is subject to change without notice. If you find any problems in the document, please
report them in writing to: Oracle University, 500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
warranted to be error-free.

Restricted Rights Notice

If this documentation is delivered to the United States Government or anyone using the documentation on behalf of the United
States Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS


The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or disclose these training materials are restricted
by the terms of the applicable Oracle license agreement and/or the applicable U.S. Government contract.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective
owners.

Author
Raghavendra J.S.

Technical Contributors and Reviewers


Hemachandran Namachivayam, Steve McKinty, Lisa Shepherd, Vidhya Cidambi, Uma Sannasi,
Venu Poddar, Nancy Chow, Sambit Nayak, Harish Mallya, Thorsten Fruauf, Sidney Jannatpour,
Stephen Baeten, Bikramjit Saikia

Editor
Chandrika Kennedy

Graphic Designers
Seema Bopaiah, Kavya Bellur

Publishers
Sumesh Koshy, Raghunath M, Asief Baig, Jayanthy Keshavamurthy, Joseph Fernandez

This book was published using: Oracle Tutor


THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Table of Contents

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 1: Introduction ........................................................................................................ 1-1
Practices for Lesson 1: Overview............................................................................................................. 1-2
Practices for Lesson 2: Introducing Oracle Solaris Cluster Hardware and Software.............................. 2-1
Practices for Lesson 2............................................................................................................................. 2-2
Practices for Lesson 3: Establishing Cluster Node Console Connectivity.............................................. 3-1
Practice for Lesson 3: Overview .............................................................................................................. 3-2
Practice 3-1: Connecting to the Cluster Node Console ............................................................................. 3-3
Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation ......................................... 4-1
Practices for Lesson 4: Overview............................................................................................................. 4-2
Practice 4-1: Preparing for Installation ..................................................................................................... 4-3
Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software............................................... 5-1
Practices for Lesson 5: Overview............................................................................................................. 5-2
Practice 5-1: Installing and Configuring the Oracle Solaris Cluster Software.............................................. 5-3
Practices for Lesson 6: Administering Oracle Solaris Cluster ................................................................ 6-1
Practices for Lesson 6: Overview............................................................................................................. 6-2
Practice 6-1: Performing Basic Cluster Administration .............................................................................. 6-3
Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS .............................. 7-1
Practices for Lesson 7: Overview............................................................................................................. 7-2
Practice 7-1: Configuring Volume Management by Using Oracle Solaris ZFS............................................ 7-3
Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager ..................... 8-1
Practices for Lesson 8: Overview............................................................................................................. 8-2
Practice 8-1: Configuring Volume Management by Using Solaris Volume Manager ................................... 8-3
Practices for Lesson 9: Managing the Public Network with IPMP ........................................................... 9-1
Practices for Lesson 9: Overview............................................................................................................. 9-2
Practice 9-1: Configuring and Testing IPMP ............................................................................................. 9-3
Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS .............................. 10-1
Practices for Lesson 10: Overview ........................................................................................................... 10-2
Practice 10-1: Installing and Configuring HA for NFS................................................................................ 10-3
Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group
Relationships ..................................................................................................................................... 11-1
Practices for Lesson 11: Overview ........................................................................................................... 11-2
Practice 11-1: Installing and Configuring Apache as a Scalable Service on Oracle Solaris Cluster ............. 11-3
Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster ........................... 12-1
Practices for Lesson 12: Overview ........................................................................................................... 12-2
Practice 12-1: Building a Zone Cluster ..................................................................................................... 12-3
Practice 12-2: Importing an Installed Zone into an Existing Zone Cluster................................................... 12-7
Practice 12-3: Configuring a Scalable Application in a Zone Cluster .......................................................... 12-13
Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional) .................................. 13-1
Practices for Lesson 13: Overview (Optional) ........................................................................................... 13-2
Practice 13-1: Integrating Oracle Database 12c into Oracle Solaris Cluster Software as a Failover
Application ......................................................................................................................................... 13-3
Practice 13-2: Install and Configure Oracle Solaris and Oracle Solaris Cluster Software Using Unified
Archives (Demonstration) ................................................................................................................... 13-18

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Oracle Solaris Cluster 4.x Administration Table of Contents


i
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle Solaris Cluster 4.x Administration Table of Contents


ii
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 1: Introduction


Chapter 1 - Page 1
Chapter 1
Introduction

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.


Practices for Lesson 1:

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 1: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices Overview
This practice introduces you to your course infrastructure, which you will use for performing the
practices. The practices map to the respective lessons.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices Infrastructure
Your practice environment is based on the Oracle VirtualBox virtualization software. VirtualBox

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
is a cross-platform virtualization application. Figure 1 shows the configured virtual machines
(VM).

Figure 1: Oracle VirtualBox Virtual Machines

The VirtualBox environment consists of the following VMs:

Name of the VM Description


This VM is installed with Oracle Solaris 11.3
LiveMedia for x86.This VM runs in GUI mode and
enables students to use the pconsole utility to
install, configure, and administer Oracle Solaris
Administrative Workstation Cluster 4.3 software.
This VM also contains the IPS repository for
Oracle Solaris 11.3 OS and Oracle Solaris Cluster
4.3 software, required for running the course
practices.
This VM is installed with Oracle Solaris 11.3 text
Cluster_Node_1 installer for x86 and acts as a regular node in a
two-node cluster.
This VM is installed with Oracle Solaris 11.3 text
Cluster_Node_2 installer for x86 and acts as a regular node in a
two-node cluster.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Name of the VM Description


This is the Sun ZFS Storage 7000 appliance

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Sun_ZFS_Storage_700 simulator and provides shared storage to the
cluster nodes.
This VM is the Sun ZFS Storage 7000 appliance
simulator and provides shared storage to the
Sun_ZFS_Storage_7000_DB
cluster nodes to perform the Oracle Database 12c
practice.
This VM is preinstalled with Oracle Solaris 11.3
Cluster_Node_1_UA OS and Oracle Solaris Cluster 4.3 and used to
perform the Oracle Database 12c practice.
This VM is preinstalled with Oracle Solaris 11.3
Cluster_Node_2_UA OS and Oracle Solaris Cluster 4.23 and used to
perform the Oracle Database 12c practice.
The VMs are further configured to communicate with the host machine through the shared
directory. The shared directories are as listed in the following table:
Resource Name Location Description
Host share directory /opt/ora Contains various course files
Demo files /opt/ora/demo Contains the demonstration files that
are used in the course
OVA files /opt/ora/images Contains a backup of all virtual
machines used in the course.
If any virtual machine gets corrupted
while performing a practice, you can
delete the virtual machine and import
the respective VM from this folder.
However, after reimporting, you will
have to reperform previous practices
to bring the VM to the required start
state of the practice where the VM
got corrupted.
Lab files /opt/ora/lab Contains practice files that are
required to perform certain practices
The details of the shared directories can be verified in the respective VM settings. Each VM has
an entry in the /etc/vfstab file, which stores information about the mount points and related
directories on the system.

Logging In to the Practice Environment


When you first log in to the practice environment, you are prompted to provide a login and
password for the host system:
• Userid: root
• Password: 0racle
After you have gained access to the host system, the user account and password for each
virtual machine is as follows:

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

For the Administrative Workstation VM:


User account: oracle1

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Password: oracle1
Administrator privileges: root/oracle1
For other VMs:
• User account: oracle1
• Password: oracle1
• Administrator privileges: As the oracle1 user, use su - to switch to the primary
administrator (root) role. The password is oracle1. The oracle1 user switches to
root because root is configured as a role by default. The first username created on the
system (during the OS installation) is the initial privileged user who can assume the
administrator role. This can be verified in the /etc/user_attr database file.
The following table lists the login credentials for the cluster nodes and the administration
workstation:
Virtual Machines Regular User/Password Superuser/Password
Cluster_Node_1 oracle1/oracle1 root/oracle1
Cluster_Node_2 oracle1/oracle1 root/oracle1
Administration Workstation oracle1/oracle1 root/oracle1
Cluster_Node_1_DB oracle1/oracle1 root/oracle1
Cluster_Node_2_DB oracle1/oracle1 root/oracle1

Task: Becoming Familiar with Your Practice Environment (Example)


1. On your host system, start the Oracle VM VirtualBox Manager by double-clicking its icon on
your desktop.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

How to Start the VM?


2. In the Oracle VM VirtualBox Manager window, double-click the Administrative

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Workstation VM to start it. Alternatively, you can simply select the Administrative
Workstation VM and click the Start button.

How to Log In to the VM?


3. After the Administrative Workstation VM is powered ON, the Username login screen
appears. Enter oracle1 as the username and click the Log In button.
Note: It might take a few minutes for the Username login screen to appear.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. When the password login screen appears, enter the password oracle1 and click the Log
In button.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
5. Open a terminal window by right-clicking the desktop and selecting Open Terminal. In the
terminal window, run the su - command to assume the administrator privileges. The
password is oracle1.
oracle1@adminws:~$ su –
Password: oracle1
Oracle Corporation SunOS 5.11 11.3 September 2015
root@adminws:~#

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

How to Power Off the VM?


6. At times, you may need to power off a VM and close its window. You may also need to shut

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
down a VM to comply with the maximum recommended number of VMs running
simultaneously, which is currently limited to three VMs.
Now, practice shutting down a VM by using the Administrative Workstation VM. To shut
down the VM, click the “close” button (x) in the top-right corner of the VM window.

7. When the Close Virtual Machine dialog box appears, select “Power off the machine” and
click OK.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Note: You can verify that the VM is shut down by checking the status that appears under
the VM’s name in the Oracle VM VirtualBox Manager window. The status for the
Administrative Workstation VM must be “Powered Off.”

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 1: Introduction


Chapter 1 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 1: Introduction


Chapter 1 - Page 10
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 2:
Introducing Oracle Solaris
Cluster Hardware and
Software
Chapter 2

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Introducing Oracle Solaris Cluster Hardware and Software
Chapter 2 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 2

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
This lesson has no practices.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 2: Introducing Oracle Solaris Cluster Hardware and Software
Chapter 2 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 3:
Establishing Cluster Node
Console Connectivity
Chapter 3

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice for Lesson 3: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
The Oracle Solaris pconsole utility is used to connect to the cluster nodes. The pconsole
utility creates a host terminal window for each remote host that you specify on the command
line. The utility also opens a central, or master, console window that you can use to send input
to all the nodes at the same time.
Note
• Command output or values may vary across systems.
• To accommodate the complete command output, the font size of the output is reduced in
a few places.
• Your system performance depends on the network speed and network load. If you find
your VM is too slow to proceed with, it is suggested that you restart the VM.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 3-1: Connecting to the Cluster Node Console

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you install and configure the pconsole utility on an administrative workstation
to manage a cluster environment.
In this practice, you perform the following tasks:
• Update host name resolution.
• Install the pconsole utility.
• Configure the pconsole utility.
• Use the pconsole utility.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs in that
order. Wait for the VMs to boot properly until you see the login prompt.
This practice assumes that the Oracle Solaris 11.3 OS is already installed on the two cluster
nodes and the administrative workstation.
For the purpose of this practice, use the Administrative Workstation (adminws) system
to install, configure, and use the pconsole utility.

Task 1: Update Host Name Resolution


1. Log in to your Administrative Workstation VM as the oracle1 user.
2. Verify that the Oracle Solaris 11.3 OS is installed on the administrative workstation.
$ cat /etc/release
Oracle Solaris 11.3 X86
Copyright (c) 1983, 2015, Oracle and/or its affiliates. All rights reserved.
Assembled 22 June 2015

3. Ensure that the /etc/inet/hosts file has entries for the IP addresses and host names of
the clnode1 and clnode2 cluster nodes.
$ cat /etc/inet/hosts

192.168.1.101 cluster-1 clnode1
192.168.1.102 cluster-2 clnode2

4. Verify that the config/host property of the system/name-service/switch service is
set to “files nis” to ensure that the name resolution first happens by using the
/etc/inet/hosts file before trying any other name service.
$ svcprop system/name-service/switch | grep config/host
config/host astring files\ nis

If the config/host property is not set, use the svccfg command as the root role to set
the config/host property of the system/name-service/switch service to “files
nis”.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

# svccfg -s system/name-service/switch
svc:/system/name-service/switch> setprop config/host = astring: "files nis"

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
svc:/system/name-service/switch> quit
# svcadm refresh name-service/switch

# svcprop system/name-service/switch | grep config/host


config/host astring files\ nis
sysconfig/config_properties astring sc_ns_switch_default:config/default
sc_ns_switch_host:config/host sc_ns_switch_netgroup:config/netgroup
root@adminws:~#
Note: Ensure that you exit the root role before proceeding to the next task.

Task 2: Install the pconsole Utility


1. Ensure that you are logged in as the oracle1 user.
2. Verify that the solaris and ha-cluster publishers correspond to the Oracle Solaris 11.3
OS and Oracle Solaris Cluster 4.3 repositories, respectively.
$ pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://adminws:10001/
ha-cluster origin online F http://adminws:10002/
Note: By default, the pkg publisher command must list the solaris publisher having
https://pkg.oracle.com as the URI. If not set, type the following commands to set the origin
for the solaris publisher to get the Oracle Solaris 11.3 OS software packages and the
ha-cluster publisher to get the Oracle Solaris Cluster 4.3 software packages from a local
repository:
$ pfexec pkg set-publisher –G ‘*’ –g http://adminws:10001/ solaris
$ pfexec pkg set-publisher –g http://adminws:10002/ ha-cluster
$ pkg publisher
3. Install the terminal/pconsole package.
$ pkg info terminal/pconsole
pkg: info: no packages matching the following patterns you specified are
installed on the system. Try specifying -r to query remotely:
terminal/pconsole
$ pfexec pkg install terminal/pconsole
Packages to install: 1
Services to change: 1
Create boot environment: No
Create backup boot environment: No

DOWNLOAD PKGS FILES XFER (MB) SPEED


Completed 1/1 10/10 0.0/0.0 49.2k/s

PHASE ITEMS
Installing new actions 25/25
Updating package state database Done
Updating package cache 0/0
Updating image state Done
Creating fast lookup database Done
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Updating package cache 2/2

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
$ pkg info terminal/pconsole
Name: terminal/pconsole
Summary: Parallel console
Description: pconsole is an administrative tool for working with clusters
of machines: it distributes the commands you type to each node
in the cluster.
Category: Applications/System Utilities
State: Installed
Publisher: solaris
Version: 1.0
Build Release: 5.11
Branch: 0.175.3.0.0.25.3
Packaging Date: June 22, 2015 12:47:25 AM
Size: 73.89 kB
FMRI: pkg://solaris/terminal/pconsole@1.0,5.11-
0.175.3.0.0.25.3:20150622T004725Z
4. Install the Oracle Solaris Cluster man page packages:
$ pkg info ha-cluster/system/manual
pkg: info: no packages matching the following patterns you specified are
installed on the system. Try specifying -r to query remotely:

ha-cluster/system/manual

$ pfexec pkg install ha-cluster/system/manual


Packages to install: 2
Create boot environment: No
Create backup boot environment: No

DOWNLOAD PKGS FILES XFER (MB) SPEED


Completed 2/2 227/227 0.5/0.5 576k/s

PHASE ITEMS
Installing new actions 321/321
Updating package state database Done
Updating package cache 0/0
Updating image state Done
Creating fast lookup database Done
Updating package cache 2/2

$ pkg info ha-cluster/system/manual


Name: ha-cluster/system/manual
Summary: Oracle Solaris Cluster Manual Pages
Description: Oracle Solaris Cluster On-Line Manual Pages
Category: System/HA Cluster
State: Installed
Publisher: ha-cluster
Version: 4.3 (Oracle Solaris Cluster 4.3.0.22.0)

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Build Release: 5.11


Branch: 0.22.0

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Packaging Date: August 26, 2015 08:13:57 PM
Size: 1.94 MB
FMRI: pkg://ha-cluster/ha-cluster/system/manual@4.3,5.11-
0.22.0:20150826T233558Z

$ pkg info ha-cluster/system/manual/data-services


$ pfexec pkg install ha-cluster/system/manual/data-services
Packages to install: 1
Create boot environment: No
Create backup boot environment: No

DOWNLOAD PKGS FILES XFER (MB) SPEED


Completed 1/1 74/74 0.1/0.1 255k/s

PHASE ITEMS
Installing new actions 89/89
Updating package state database Done
Updating package cache 0/0
Updating image state Done
Creating fast lookup database Done
Updating package cache 2/2

$ pkg info ha-cluster/system/manual/data-services


Name: ha-cluster/system/manual/data-services
Summary: Oracle Solaris Cluster Data Services Manual Pages
Description: Oracle Solaris Cluster Data Services On-Line Manual Pages
Category: System/HA Cluster
State: Installed
Publisher: ha-cluster
Version: 4.3 (Oracle Solaris Cluster 4.3.0.22.0)
Build Release: 5.11
Branch: 0.22.0
Packaging Date: August 26, 2015 07:35:54 PM
Size: 338.77 kB
FMRI: pkg://ha-cluster/ha-cluster/system/manual/data-
services@4.3,5.11-0.22.0: 20150826T233558Z
Installing the Oracle Solaris Cluster man page packages on the administrative workstation
enables you to view the various man pages before you actually install the software on the
cluster nodes.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 3: Configure the pconsole Utility


1. Ensure that you are logged in as the oracle1 user.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
2. Assign the /usr/cluster/man directory path to the PATH environment variable in the
.profile file of the oracle1 user.
$ vi ~/.profile
...
export PATH=/usr/cluster/man:/usr/bin:/usr/sbin
...
3. Set the environment variables by running the .profile login script.
$ echo $PATH
/usr/bin:/usr/sbin
$ . ~/.profile
$ echo $PATH
/usr/bin:/usr/sbin:/usr/cluster/man

4. Assume the root role.


5. Grant the Parallel Console Access rights profile to the oracle1 user.
# usermod –P +“Parallel Console Access” oracle1

Note: Ignore the following message:


UX: usermod: oracle1 is currently logged in, some changes may not take effect
until next login.
Note: Throughout the practices of this course, you use the pconsole utility as the
oracle1 user with the Parallel Console Access rights profile.
6. Exit the root role.
7. Verify that the Parallel Console Access rights profile is granted to the oracle1 user.
$ profiles | grep –i parallel
Parallel Console Access

Task 4: Use the pconsole Utility


1. Ensure that the Cluster_Node_1 and Cluster_Node_2 VMs acting as the two cluster
nodes are powered on or booted.
2. Ensure that you are logged in to the Administrative Workstation as the oracle1
user.
3. Start the pconsole utility to manage the cluster nodes.
$ pconsole clnode1 clnode2 &
This command opens a host terminal window for each cluster node. It also opens a central
or master console window that propagates what you input there to each of the cluster node
terminals.
The pconsole utility uses the Secure Shell (SSH) protocol to establish a connection with
the cluster nodes. Therefore, on the host terminal windows of the cluster nodes, you should
see a message asking you to confirm the connection to the cluster nodes.
4. Specify oracle1 as the password in the master console window to accept an SSH
connection to the cluster nodes.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

5. Place the cursor in the master console window, and press the Enter key a couple of times.
You should see a response in the host terminal window of each cluster node. The

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
pconsole utility is now ready for use.
6. Adjust the two host terminal windows for each cluster node and the master console window
on the desktop of the administrative workstation, as shown in the following screenshot:

Summary: Now, you have successfully established console access to the cluster nodes
using the pconsole utility. Adjusting the two host terminal windows and the master
console window is just for your convenience and is not a mandatory step.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 3: Establishing Cluster Node Console Connectivity


Chapter 3 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 4:
Preparing for the Oracle
Solaris Cluster Installation
Chapter 4

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 4: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you verify the Oracle Solaris 11.3 OS installed on your practice systems, identify
a cluster topology, select quorum devices, verify the cluster interconnect configuration, and
select public network interfaces.
Note
• Command output or values may vary across systems.
• To accommodate the complete command output, the font size of the output is reduced in
a few places.
• Your system performance depends on the network speed and network load. If you find
your VM too slow to proceed with, it is suggested that you restart the VM.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 4-1: Preparing for Installation

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you prepare your assigned cluster nodes for installing the Oracle Solaris Cluster
4.3 software.
In this practice, you perform the following tasks:
• Verify the operating system environment.
• Identify the cluster topology.
• Select quorum devices.
• Verify the cluster-private interconnect configuration.
• Select public network interfaces.

Preparation
To begin this practice, you must be connected to the cluster nodes through the pconsole utility
from the administrative workstation and you must be logged in as the oracle1 user.

Task 1: Verify the Operating System Environment


1. Verify that the Oracle Solaris 11.3 operating system is installed on the cluster nodes.
$ cat /etc/release
Oracle Solaris 11.3 X86
Copyright (c) 1983, 2015, Oracle and/or its affiliates. All rights reserved.
Assembled 06 October 2015
2. Record the size of (physical) memory on the designated cluster nodes by using the
/usr/sbin/prtconf command.
$ /usr/sbin/prtconf | grep –i memory

Example
$ /usr/sbin/prtconf | grep -i mem
Memory size: 4096 Megabytes
Node 1 memory: _______________
Node 2 memory: _______________
3. Determine whether ZFS is the root file system on the cluster nodes by using the following
commands:
$ df –kh
Filesystem Size Used Available Capacity Mounted on
rpool/ROOT/solaris 49G 2.9G 42G 7% /
....

$ zpool status
pool: rpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM


rpool ONLINE 0 0 0
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

c1t0d0s0 ONLINE 0 0 0

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
errors: No known data errors

$ zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool 49.8G 7.25G 42.5G 14% 1.00x ONLINE -

4. Verify that at least 750 MB of swap space is given to each cluster node by using the swap
–l command. The swap space must be in zvol.
$ swap -l

Task 2: Identify the Cluster Topology


1. Record the desired topology configuration of your cluster in the following table:
Number of nodes
Number of storage arrays
Types of storage arrays
2. Verify that the storage arrays in your cluster are properly connected to your target topology.
Note: In this course, you will be creating a two-node cluster by using a preconfigured
Sun_ZFS_Storage_7000 appliance simulator as the storage device.

Task 3: Select Quorum Devices


1. Record the number of quorum devices that you must configure after the cluster host
software installation.
Number of quorum devices: __________
Note: Consult with your instructor if you are not sure about your quorum device
configuration.
2. Decide whether the scinstall utility will automatically be able to choose a quorum device
(it can for a single quorum device for a two-node cluster).
Automatic quorum configuration: (yes/no)?
3. If there can be no automatic quorum configuration (for a three-node cluster, for example),
enter the format command and record the logical path to the disks that you want to use as
quorum disk drives in your storage arrays.
Quorum disks: ________________________________

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. Press Ctrl + D to exit the format utility.


Note: You will be configuring only a two-node cluster as you perform the practices inside

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
the VirtualBox environment, so ignore steps 3 and 4. Specify yes for automatic quorum
device selection and configuration by the scinstall utility in step 2.

Task 4: Verify the Cluster-Private Interconnect Configuration


This task describes how to verify the cluster-private interconnect configuration.
Perform the following steps to configure an Ethernet-based, point-to-point interconnect:
1. Determine the names of your cluster-private interconnect adapters by using the dladm
show-phys command, followed by the ipadm show-addr command.
$ dladm show-phys
$ ipadm show-addr
Refrain from assigning IP addresses to the existing public network interfaces.
2. Determine the cluster-private interconnect interfaces for as many interfaces as are
applicable in your practice environment.
The following figure is a schematic representation of a cluster that uses an Ethernet-based,
point-to-point cluster-private interconnect configuration:

Note: In your VirtualBox practice environment, cluster nodes are configured with Ethernet-
based, point-to-point interconnects. Skip the steps listed under the switch-based Ethernet
interconnect section.
Perform the following steps to configure a switch-based Ethernet interconnect:
1. Record the logical names of the cluster interconnect interfaces.
Note: You can use the strategy of determining the cluster transport interfaces by using the
dladm show-phys command, followed by the ipadm show-addr command. Do not use
IP addresses in the existing public network space.
$ dladm show-phys
$ ipadm show-addr

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. Determine the interconnect interfaces for as many interfaces as are applicable in your
practice environment.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
The following figure is a schematic representation of a cluster that uses Ethernet-based,
cluster-private interconnects with switches:

3. Verify that each Ethernet interconnect interface is connected to the correct switch by
pinging each other across the private switches.
Note: If you have any doubt about the private interconnect cabling, consult with your
instructor now. Do not continue this practice until you are confident that your cluster-private
interconnects are cabled correctly, and that you know the names of the cluster-private
transport adapters.
Note: In your given cluster setup environment inside VirtualBox, net1 and net3 are the
designated switch-based cluster-private interconnects.

Task 5: Select Public Network Interfaces


Ask your instructor to help you identify the public network interfaces on each cluster node that
can be used in IPMP groups.
Record the logical names of the potential IPMP Ethernet interfaces on each cluster node, in the
following table:
System Primary IPMP interface Backup IPMP interface
Node 1
Node 2
Note: It is important that you are sure about the logical name of each public network interface.
In your given cluster setup environment inside VirtualBox, net0 is the primary IPMP interface,
while net2 is the backup IPMP interface.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 4: Preparing for the Oracle Solaris Cluster Installation
Chapter 4 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 5:
Configuring the Oracle
Solaris Cluster Software
Chapter 5

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 5: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you install and configure the Oracle Solaris Cluster software.
Note
• Command output or values may vary across systems.
• To accommodate the complete command output, the font size of the output is reduced
in a few places.
• Your system performance depends on the network speed and network load. If you find
your VM too slow to proceed with, it is suggested that you restart the VM.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 5-1: Installing and Configuring the Oracle Solaris Cluster


Software

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you install and configure the Oracle Solaris Cluster software. You verify the
practice environment, update local name resolution, install the Oracle Solaris Cluster packages,
configure a new cluster, verify an automatically selected quorum device, configure a quorum
device, and verify the cluster configuration and status.
In this practice, you perform the following tasks:
• Establish connection with the cluster nodes.
• Prepare the cluster node environment.
• Install the Oracle Solaris Cluster packages.
• Authorize the acceptance of cluster configuration commands from the sponsor node.
• Configure a new cluster by using the all-nodes-at-once method.
• Configure a new cluster by using the one-node-at-a-time method.
• Verify the cluster configuration.

Preparation
Before proceeding with this practice, start the Sun_ZFS_Storage_7000, Cluster_Node_1,
Cluster_Node_2, and Administrative Workstation VMs, in that order. Wait for the
VMs to boot properly until you see the login prompt.
For the purpose of this practice, use the Administrative Workstation (adminws) system
to install and configure the Oracle Solaris Cluster software by using the pconsole utility.

Task 1: Establish Connection with the Cluster Nodes


1. Ensure that the Cluster_Node_1 and Cluster_Node_2 VMs acting as the two cluster
nodes are powered on.
2. Log in to the Administrative Workstation VM as the oracle1 user and open a
terminal window.
3. Start the pconsole utility to open the host terminal window for each cluster node and a
master console window.
$ pconsole clnode1 clnode2 &
4. Use the master console window to log in to the two cluster nodes with oracle1 as the
username and password.
Note: You perform the remaining tasks of this practice in the master console window,
unless instructed otherwise.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 2: Prepare the Cluster Node Environment


Perform the following steps in the master console window on the administrative workstation:

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
1. Verify that the Oracle Solaris 11.3 OS is installed on the cluster nodes.
$ cat /etc/release
Oracle Solaris 11.3 X86
Copyright (c) 1983, 2015, Oracle and/or its affiliates. All rights reserved.
Assembled 22 June 2015
2. Ensure that the CONSOLE=/dev/console line in the /etc/default/login file is
commented out on the cluster nodes.
$ cat /etc/default/login | grep CONSOLE
# If CONSOLE is set, root can only login on that device.
#CONSOLE=/dev/console

3. Choose the primary group package of the Oracle Solaris Cluster 4.3 software to install.
Primary group package: _________________________________________________
For the purpose of this practice, you select the ha-cluster-full primary group package
of the Oracle Solaris Cluster 4.3 software to install on the cluster nodes.
4. If necessary, edit the /etc/inet/hosts file of the cluster nodes and add the IP
addresses of your administrative workstation and of all your cluster nodes.

192.168.1.100 adminws
192.168.1.101 cluster-1 clnode1
192.168.1.102 cluster-2 clnode2

5. Verify that the Network Auto-Magic (NWAM) feature is disabled on the cluster nodes.
$ netadm list
TYPE PROFILE STATE
ncp Automatic disabled
ncp DefaultFixed online
loc Automatic offline
loc NoNet offline
loc DefaultFixed online

Note: NWAM activates a single network interface and disables all others. For this reason,
NWAM cannot coexist with the Oracle Solaris Cluster software and you must disable it
before you configure or run your cluster. To disable NWAM, you enable the
defaultfixed profile.
# netadm enable -p ncp defaultfixed
# netadm list -p ncp defaultfixed

6. Grant the Service Management and Service Operator rights profiles to the oracle1
user on the cluster nodes.
a. While logged in as the oracle1 user on the cluster nodes, assume the root role.
b. Grant the Service Management and Service Operator rights profiles to the
oracle1 user.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

# usermod –P +”Service Management”,”Service Operator” oracle1

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Ignore the following error.
UX: usermod: oracle1 is currently logged in, some changes may not take effect
until next login.
c. Exit the root role to return to the prompt of the oracle1 user on the cluster nodes.
7. Verify that the local_only property of the network/rpc/bind service is set to false
to enable external access to remote procedure call (RPC) communication.
$ svcprop network/rpc/bind:default | grep local_only
config/local_only boolean false
If the local_only property is set to true, use the svccfg command as follows to set the
local_only property of network/rpc/bind service to false.
Note: During the installation of the Oracle Solaris OS, a restricted network profile is used
that disables external access for certain network services. The restricted services include
the RPC communication service, which is required for cluster communication.
Use the following commands to restore external access to RPC communication:
$ svccfg
svc:> select network/rpc/bind
svc:/network/rpc/bind> setprop config/local_only=false
svc:/network/rpc/bind> quit
$ svcadm refresh network/rpc/bind:default
$ svcprop network/rpc/bind:default | grep local_only
config/local_only boolean false

Task 3: Install the Oracle Solaris Cluster Packages


Perform the following steps in the master console window on the administrative workstation as
the oracle1 user:
1. Verify that the solaris and ha-cluster publishers are set.
$ pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://adminws:10001/
ha-cluster origin online F http://adminws:10002/
2. If the publishers are not, set the solaris and ha-cluster publishers for the Oracle
Solaris 11.3 OS and Oracle Solaris Cluster 4.3 software packages.
$ pfexec pkg set-publisher –G ‘*’ –g http://adminws:10001/ solaris
$ pfexec pkg set-publisher –g http://adminws:10002/ ha-cluster
$ pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://adminws:10001/
ha-cluster origin online F http://adminws:10002/
3. Install the primary group package of the Oracle Solaris Cluster 4.3 software that you
selected in Step 3 of Task 2.
$ pfexec pkg install –-accept ha-cluster-full
4. Verify the Oracle Solaris Cluster software package installation.
$ pkg info ha-cluster-full
Name: ha-cluster/group-package/ha-cluster-full
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Summary: Oracle Solaris Cluster full installation group package


Description: Oracle Solaris Cluster full installation group package

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Category: Meta Packages/Group Packages
State: Installed
Publisher: ha-cluster
Version: 4.3 (Oracle Solaris Cluster 4.3.0.24.0)
Build Release: 5.11
Branch: 0.24.0
Packaging Date: August 26, 2015 03:29:30 AM
Size: 434.00 B
FMRI: pkg://ha-cluster/ha-cluster/group-package/ha-cluster-
full@4.3,5.11-0.20.0:20150826T233336Z
5. Assign the /usr/cluster/bin and /usr/cluster/man directory paths to the PATH
environment variable in the .profile login script of the oracle1 and root user.
$/# vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
...
6. Set the environment variables by running the .profile login script.
$/# . .profile
$/# echo $PATH
/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin

Task 4: Authorize the Acceptance of Cluster Configuration Commands from the


Sponsor Node
1. Determine the sponsor (primary) node that will run the cluster creation command.
The sponsor node is clnode1.
2. On all the nodes that you will configure the cluster, other than the sponsor node, authorize
acceptance of commands from the sponsor node as root role.
clnode2# /usr/cluster/bin/clauth enable -n clnode1
clnode2# /usr/cluster/bin/clauth show

===== Cluster Installation Service =====

svc:/network/rpc/scrinstd:default online
Control node clnode1
Authentication protocol sys

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 5: Configure a New Cluster by Using the All-Nodes-at-Once Method

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
If you prefer to use the one-node-at-a-time method, skip this task and complete Task 6 instead.
Perform the following steps to configure the entire cluster by using the all-nodes-at-once
method.
Complete the following steps in the host terminal window of the clnode1 cluster node as
identified in Task 4:
1. While logged in as the oracle1 user on the clnode1 cluster node, assume the root role.
2. Verify that the /usr/cluster/bin and /usr/cluster/man directory paths are set the
.profile login script of the root role
# echo $PATH
/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
If the path variables are not set, assign the /usr/cluster/bin and /usr/cluster/man
directory paths to the PATH environment variable in the .profile login script of the root
role.
# vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
...
# . .profile
# echo $PATH
/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
3. Start the scinstall utility.
# /usr/cluster/bin/scinstall
4. Make the following choices, as the installation proceeds:
a. From the Main menu, select option 1, Create a new cluster or add a
cluster node.
b. From the New Cluster and Cluster Node menu, select option 1, Create a new
cluster.
c. Specify Yes when asked whether you want to continue.
The scinstall utility now checks whether the value of the local_only property of
the network/rpc/bind service is correctly set to false.
d. Press the Enter key to continue.
e. From the Typical or Custom Mode menu, select option 1, Typical.
f. Specify a cluster name, such as cluster1.
g. Specify the name of the other cluster node, such as clnode2.
h. Press Ctrl + D to finish adding the cluster nodes, to display the complete list of cluster
nodes added.
i. Specify Yes to confirm the listing of the cluster nodes.
j. Select the cluster transport adapters that will be used as the cluster-private
interconnect.
For this practice, select net1 and net3, one after the other.
k. Specify No when asked whether you want to disable the automatic quorum device
selection.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

l. Specify Yes when asked whether it is okay to create the new cluster.
m. Specify No when asked to interrupt cluster creation for cluster check errors.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
The cluster configuration begins. Somewhere towards the end of the cluster configuration,
the clnode2 cluster node reboots and you lose the pconsole connection to the node.
After the clnode2 cluster node successfully reboots and joins the cluster, the scinstall
utility then proceeds to configure the clnode1 cluster node. After the configuration, the
clnode1 cluster node also reboots to join the cluster. At this point, you now lose the
pconsole connection to the clnode1 cluster node too and only the master console
window remains open. Press Ctrl + D to close the master console window.
After the clnode1 cluster node also successfully reboots and joins the cluster, reestablish
the pconsole connection to the cluster nodes to continue with cluster configuration and
administration.
Note: Refer to the various messages displayed in the console window of the cluster nodes
when they reboot.

Task 6: Configure a New Cluster by Using the One-Node-at-a-Time Method


Perform the following steps to configure the first node in your cluster (the one that will be
assigned node ID 1). You must wait for this node to complete and reboot into the cluster before
configuring the other node.
Note: If you chose to install your cluster by using the all-nodes-at-once method, do not
perform this task.
Complete the following steps in the host terminal window of the clnode1 cluster node:
1. Assume the root role while logged in as the oracle1 user on the clnode1 cluster node.
2. Assign the /usr/cluster/bin and /usr/cluster/man directory paths to the PATH
environment variable in the .profile login script of the root role.
# vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
...
3. Set the environment variables by running the .profile login script.
# . .profile
# echo $PATH

4. Start the scinstall utility.


# /usr/cluster/bin/scinstall
5. As the installation proceeds, make the following choices:
a. From the Main menu, select option 1, Create a new cluster or add a
cluster node.
b. From the Install menu, select option 2, Create just the first node of a new
cluster on this machine.
c. Answer Yes to confirm the creation of just the first node of a new cluster.
d. Press Enter to continue after the local_only property value is checked to be false.
e. From the Type of Installation menu, select option 1, Typical.
f. Furnish your assigned cluster name.
g. Do not allow the cluster check to run.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

h. Furnish the names of the other nodes to be added later.


i. Verify the list of node names.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
j. Select the transport adapters. If you are asked (with a tagged VLAN-capable adapter)
whether it is a dedicated cluster transport adapter, answer Yes.
k. For a two-node cluster, do not disable the automatic quorum selection (answer No).
l. Reply Yes to the automatic reboot question.
m. Examine the scinstall command options for correctness. Accept them if they seem
appropriate.
You must wait for this node to complete rebooting to proceed to the second node.
Note: After cluster configuration completes, the node reboots, causing the host terminal
window to close. You have to reestablish the connection to the clnode1 cluster node by
using the pconsole utility. Refer to the messages displayed in the console window of the
cluster nodes.
Complete the following steps in the host terminal window of the clnode2 cluster node:
1. Assume the root role, while logged in as the oracle1 user on the clnode1 cluster node.
2. Assign the /usr/cluster/bin and /usr/cluster/man directory paths to the PATH
environment variable in the .profile login script of the root role.
# vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
...
3. Set the environment variables by running the .profile login script.
# . .profile
# echo $PATH

4. Start the scinstall utility.


# /usr/cluster/bin/scinstall
5. As the installation proceeds, make the following choices:
a. From the Main menu, select option 1, Create a new cluster or add a
cluster node.
b. From the Install menu, select option 3, Add this machine as a node in an
existing cluster.
c. Answer Yes to confirm the addition of a node to an existing cluster.
d. Press Enter to continue after the local_only property value is checked to be false.
e. From the Type of Installation menu, select option 1, Typical.
f. Provide the name of a sponsoring node, clnode1.
g. Provide the name of the cluster that you want to join.
Note: Use cluster show -t global on the first cluster node (the sponsoring
node) if you have forgotten the name of the cluster.
h. Press Enter to continue.
i. Do not allow the cluster check to run.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

j. Use auto-discovery for the transport adapters.


k. Reply Yes to the automatic reboot question.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
l. Examine and approve the scinstall command-line options.
You must wait for this node to complete rebooting to proceed to the next step.
6. Exit the master console window on the administrative workstation.
Note: After the cluster configuration completes, the node reboots, causing the host terminal
window to close. You have to reestablish the connection to the clnode2 cluster node by
using the pconsole utility. Refer to the messages displayed in the console window of the
cluster nodes.

Task 7: Verify the Cluster Configuration


Perform the following steps on any one of the cluster nodes as the oracle1 user to verify the
cluster configuration:
1. Run the clquorum status command.
$ clquorum status
=== Cluster Quorum ===

--- Quorum Votes Summary from (latest node reconfiguration) ---

Needed Present Possible


------ ------- --------
2 3 3

--- Quorum Votes by Node (current status) ---

Node Name Present Possible Status


--------- ------- -------- ------
clnode2 1 1 Online
clnode1 1 1 Online

--- Quorum Votes by Device (current status) ---

Device Name Present Possible Status


----------- ------- -------- ------
d1 1 1 Online
You must see the correct node and disk quorum votes.
2. Run the clinterconnect status command.
$ clinterconnect status
=== Cluster Transport Paths ===

Endpoint1 Endpoint2 Status


--------- --------- ------
clnode2:net3 clnode1:net3 Path online
clnode2:net1 clnode1:net1 Path online
You must see active redundant interconnect paths between all pairs of nodes.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Run the cldevice list -v command.


$ cldevice list –v

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
DID Device Full Device Path
---------- ----------------
d1 clnode2:/dev/rdsk/c0t600144F0C96CB51800005382CD780002d0
d1 clnode1:/dev/rdsk/c0t600144F0C96CB51800005382CD780002d0
d2 clnode2:/dev/rdsk/c0t600144F0C96CB51800005382CD820003d0
d2 clnode1:/dev/rdsk/c0t600144F0C96CB51800005382CD820003d0
… …
Each shared (dual-ported) DID device must show a logical path from each cluster node.
4. Run the cluster show and cluster status commands.
$ cluster show
$ cluster status
The cluster status, node names, transport configuration, and quorum device information
must be complete.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software


Chapter 5 - Page 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 5 - Page 12
Practices for Lesson 5: Configuring the Oracle Solaris Cluster Software
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 6:
Administering Oracle Solaris
Cluster
Chapter 6

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 6: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you perform basic cluster administration tasks such as verifying basic cluster
configuration and status, reassigning a quorum device, adding a quorum server quorum device,
preventing cluster amnesia, and changing the cluster private IP address range.
In this practice, you perform the following tasks:
• Verify basic cluster configuration and status.
• Reassign a quorum device.
• Configure the Oracle Solaris Cluster quorum server software.
• Add a quorum server quorum device.
• Prevent cluster amnesia.

Note
• Command output or values may vary across systems.
• To accommodate the complete command output, the font size of the output is reduced in
a few places.
• Your system performance depends on the network speed and network load. If you find
your VM too slow to proceed with, it is suggested that you restart the VM.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 6-1: Performing Basic Cluster Administration

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you perform basic cluster administration tasks such as verifying basic cluster
configuration and status, reassigning a quorum device, adding a quorum server quorum device,
preventing cluster amnesia, and changing the cluster private IP address range.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.

Task 1: Verify Basic Cluster Configuration and Status


Perform the following steps as the oracle1 user in the host terminal window of the clnode1
cluster node:
1. Verify the cluster global properties and node properties.
$ cluster show -t global
=== Cluster ===

Cluster Name: cluster1


clusterid: 0x53955907
installmode: disabled
heartbeat_timeout: 10000
heartbeat_quantum: 1000
private_netaddr: 172.16.0.0
private_netmask: 255.255.240.0
max_nodes: 62
max_privatenets: 10
num_zoneclusters: 12
num_xip_zoneclusters: 3
udp_session_timeout: 480
concentrate_load: False
resource_security: SECURE
global_fencing: prefer3
Node List: clnode2, clnode1

$ clnode show
=== Cluster Nodes ===

Node Name: clnode2


Node ID: 1
Enabled: yes
privatehostname: clusternode1-priv

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

reboot_on_path_failure: disabled
globalzoneshares: 1

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
defaultpsetmin: 1
quorum_vote: 1
quorum_defaultvote: 1
quorum_resv_key: 0x5395590700000001
Transport Adapter List: net1, net3

Node Name: clnode1


Node ID: 2
Enabled: yes
privatehostname: clusternode2-priv
reboot_on_path_failure: disabled
globalzoneshares: 1
defaultpsetmin: 1
quorum_vote: 1
quorum_defaultvote: 1
quorum_resv_key: 0x5395590700000002
Transport Adapter List: net1, net3
2. Use the clquorum (clq) command to verify the current cluster membership and the
quorum status.
$ clq status
=== Cluster Quorum ===

--- Quorum Votes Summary from (latest node reconfiguration) ---

Needed Present Possible


------ ------- --------
2 3 3

--- Quorum Votes by Node (current status) ---

Node Name Present Possible Status


--------- ------- -------- ------
clnode2 1 1 Online
clnode1 1 1 Online

--- Quorum Votes by Device (current status) ---

Device Name Present Possible Status


----------- ------- -------- ------
d1 1 1 Online

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Record the quorum configuration from the previous step.


Quorum votes needed: _____

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Quorum votes present: _____
Quorum votes possible: _____
4. Verify all cluster disk paths.
$ cldev status

=== Cluster DID Devices ===

Device Instance Node Status


--------------- ---- ------
/dev/did/rdsk/d1 clnode1 Ok
clnode2 Ok

/dev/did/rdsk/d10 clnode1 Ok
clnode2 Ok
<… output omitted…>
5. Verify the interconnect status.
$ clintr status

=== Cluster Transport Paths ===

Endpoint1 Endpoint2 Status


--------- --------- ------
clnode2:net3 clnode1:net3 Path online
clnode2:net1 clnode1:net1 Path online
6. Verify the revision of the currently installed Oracle Solaris Cluster software on each node.
$ clnode show-rev –v
Oracle Solaris Cluster 4.3.0.20.0 for Solaris 11 i386
ha-cluster/data-service/apache: 4.3-0.20.0
ha-cluster/data-service/dhcp: 4.3-0.20.0
ha-cluster/data-service/dns: 4.3-0.20.0
<… output omitted…>

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 2: Reassign a Quorum Device


Assume that your quorum device (or one of your quorum devices) has failed. Perform the

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
following tasks from any one node in the cluster to reassign the quorum device:
1. Verify the status of your current quorum device(s).
$ clq status
$ cldev status d#
2. Show all disk paths.
$ cldev list -v
3. Choose a different shared disk to be the new quorum device, and add your new quorum
device.
$ clq add new-d-#
$ clq status
$ clq status new-d-#
4. Remove your old broken quorum device.
$ clq remove old-d-#
5. Verify the quorum status again.
$ clq status

Task 3: Configure Oracle Solaris Cluster Quorum Server Software


Perform the following steps to configure your administrative workstation as the quorum server:
1. Ensure that the quorum server machine is connected to a public network that is accessible
to the cluster nodes.
2. Ensure that you are logged in to the selected quorum server machine (admin workstation)
as the oracle1 user.
3. Ensure that the solaris and ha-cluster publishers are valid.
adminws$ pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://adminws:10001/
ha-cluster origin online F http://adminws:10002/
4. Install the Quorum Server group package.
adminws$ pfexec pkg install ha-cluster-quorum-server-full
adminws$ pkg info ha-cluster-quorum-server-full
Name: ha-cluster/group-package/ha-cluster-quorum-server-full
Summary: Oracle Solaris Cluster Quorum Server full group package
Description: Oracle Solaris Cluster Quorum Server full group package
Category: Meta Packages/Group Packages
State: Installed
Publisher: ha-cluster
Version: 4.3 (Oracle Solaris Cluster 4.3.0.20.0)
Build Release: 5.11
Branch: 0.20.0
Packaging Date: June 24, 2015 03:29:57 AM
Size: 434.00 B
FMRI: pkg://ha-cluster/ha-cluster/group-package/ha-cluster-quorum-
server-full@4.3,5.11-0.20.0:20150624T032957Z
oracle1@adminws:~$

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

5. Assume the root role.


6. Add the Oracle Solaris Cluster Quorum Server binary location to your PATH environment

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
variable by adding the following entry to the .profile file:
# vi .profile
...
export PATH=/usr/cluster/bin:/usr/cluster/man:/usr/bin:/usr/sbin
adminws# . .profile
7. Specify quorum configuration information in the /etc/scqsd/scqsd.conf file as follows:
# vi /etc/scqsd/scqsd.conf

/usr/cluster/lib/sc/scqsd -d /var/scqsd -i quorumserver -p 9000
8. Save and close the /etc/scqsd/scqsd.conf file.
9. Start the newly configured quorum server.
adminws# /usr/cluster/bin/clquorumserver start quorumserver
10. Exit the root role.

Task 4: Add a Quorum Server Quorum Device


To get the IP address of the quorum server that you configured in Task 3, either use the ipadm
command or refer to the /etc/inet/hosts file. In your cluster practice environment, the IP
address of the administrative workstation, now acting as a quorum server, is 192.168.1.100.
You can add the quorum device just by using the IP address, or you can use the host name, if it
is resolvable on your cluster nodes.
Perform the following steps on any one of the cluster nodes as the oracle1 user:
1. Add the quorum server quorum device.
$ clq add -t quorum_server -p qshost=qserver_name_or_IP -p port=9000 myqserver
$ clq status
The name that you give to the device on the cluster side as the last argument to the
command is not important. You can use any name.
2. Delete your disk quorum device(s).
$ clq remove old-d#
$ clq status
3. Add back the disk quorum devices and remove the quorum server quorum device.
$ clq add d#
$ clq status
$ clq remove myqserver
$ clq status
If you have a cluster with nonstorage nodes, having a quorum server quorum device is not
necessarily exactly what you want, because it enables the nonstorage node to stay alive
even if both the storage nodes have failed.

Task 5: Prevent Cluster Amnesia


Perform the following steps to demonstrate how the cluster prevents cluster amnesia by using
persistent reservations on the quorum device:
1. Verify that all nodes are active cluster members.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. As the root role, shut down the clnode1 and clnode2 cluster nodes with init 0, one
after the other.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# init 0
Note: Wait until you see the message on the console screen.

syncing file systems... done
WARNING: CMM: Node being shut down.

System is halted. It is now safe to power it off.


3. Boot the clnode1 cluster node.
The clnode1 cluster node must hang waiting for operational quorum. This is because the
reservation key for the clnode1 cluster node has been removed from the quorum disk(s).
Note: You may see output like the following on the clnode1 console screen:

Aug 21 08:33:00 clnode1 genunix: NOTICE: CMM: Node clnode1: attempting to join cluster.
Aug 21 08:34:02 clnode1 genunix: NOTICE: CMM: Cluster doesn't have operational quorum
yet; waiting for quorum.

4. Now boot the clnode2 cluster node.
Note: Both cluster nodes must complete the cluster software startup sequence and join the
clustered operations.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 6: Administering Oracle Solaris Cluster


Chapter 6 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 7:
Configuring Volume
Management with Oracle
Solaris ZFS
Chapter 7

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 7: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you configure volume management by using ZFS.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 7-1: Configuring Volume Management by Using Oracle


Solaris ZFS

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you create a ZFS pool and file system for cluster data, create a snapshot and
then modify your data, and manually migrate your ZFS pool to another node.
In this practice, you perform the following tasks:
• Create a ZFS pool and file system for cluster data.
• Create a snapshot and modify your data.
• Migrate your ZFS pool manually to another node.
Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.
Task 1: Create a ZFS Pool and File System for Cluster Data
Perform the following steps as the oracle1 user in the host terminal window of the clnode1
cluster node:
1. Select disk drives from the shared storage to use in a new ZFS pool for data.
$ zpool status
$ cldev list –v
$ clq status
Note: Ensure that you do not use any disks already in use in the existing device group.
Note the logical device name of the identified disk, referred to as c#t#d# in the next step.
Consult with your instructor if you are unsure.
2. Assume the root role.
3. Create a ZFS pool that provides mirroring for the selected disks.
# zpool create nfspool mirror c#t#d# c#t#d#

# zpool status
Note: Use d3 and d4 to create the mirrored pool.
4. Create a ZFS file system within the pool.
# zfs create nfspool/nfs
# zfs list -r nfspool
NAME USED AVAIL REFER MOUNTPOINT
nfspool 124K 1.95G 32K /nfspool
nfspool/nfs 31K 1.95G 31K /nfspool/nfs

# df –kh
Filesystem Size Used Available Capacity Mounted on
...
nfspool 2.0G 32K 2.0G 1% /nfspool
nfspool/nfs 2.0G 31K 2.0G 1% /nfspool/nfs
...

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

5. Change the mount point.


# zfs set mountpoint=/testnfs nfspool/nfs

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# df –kh
Filesystem Size Used Available Capacity Mounted on
...
nfspool 2.0G 31K 2.0G 1% /nfspool
nfspool/nfs 2.0G 31K 2.0G 1% /testnfs
...
6. Populate your file system with some simple files.
# cd /testnfs
# touch file1 file2 file3
# vi my_letter
ZFS is much better than that legacy stuff
# ls
file1 file2 file3 my_letter
# cd

Task 2: Create a Snapshot and Modify Your Data


Perform the following steps as the root role in the host terminal window of the clnode1 cluster
node:
1. Snapshot your ZFS file system.
# zfs snapshot nfspool/nfs@SAVEMYWORK
# zfs list -r nfspool
NAME USED AVAIL REFER MOUNTPOINT
nfspool 136K 1.95G 31K /nfspool
nfspool/nfs 31.5K 1.95G 31.5K /testnfs

Now, verify the value of the listsnapshots pool property.

# zpool get listsnapshots nfspool


NAME PROPERTY VALUE SOURCE
nfspool listsnapshots off default

As displayed here, the listsnapshots property is off by default. Set it on and verify
its status.
# zpool set listsnapshots=on nfspool
# zpool get listsnapshots nfspool
NAME PROPERTY VALUE SOURCE
nfspool listsnapshots on local

Now, when you display the descendent file systems of the nfspool pool, the
snapshots are included.
# zfs list -r nfspool
NAME USED AVAIL REFER MOUNTPOINT
nfspool 139K 1.95G 31K /nfspool
nfspool/nfs 31.5K 1.95G 31.5K /testnfs
nfspool/nfs@SAVEMYWORK 0 - 31.5K -

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. Modify the content of your file system.


# cd /testnfs

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# touch file4
# ls
file1 file2 file3 file4 my_letter
# rm file2
# ls
file1 file3 file4 my_letter
# vi my_letter
ZFS is much better than that legacy stuff
Here is more information about my class
# cd

Task 3: Migrate Your ZFS Pool Manually to Another Node


Perform the following steps as the root role.
On the host terminal window of the clnode1 cluster node:
1. Export your nfspool pool.
# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
nfspool 1.98G 177K 1.98G 0% 1.00x ONLINE -
rpool 49.8G 10.4G 39.4G 20% 1.00x ONLINE -

# zfs list | grep nfs


nfspool 173K 1.95G 31K /nfspool
nfspool/nfs 52K 1.95G 32.5K /testnfs
nfspool/nfs@SAVEMYWORK 19.5K - 31.5K -

# zpool export nfspool

# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool 49.8G 10.4G 39.4G 20% 1.00x ONLINE -

# zpool status
On the host terminal window of the clnode2 cluster node:
2. Assume the root role.
3. Import the nfspool pool.
# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool 49.8G 9.43G 40.3G 18% 1.00x ONLINE -

# zpool import nfspool


# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
nfspool 1.98G 214K 1.98G 0% 1.00x ONLINE -
rpool 49.8G 9.43G 40.3G 18% 1.00x ONLINE -

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

# zpool status
pool: nfspool

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM


nfspool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c0t600144F0C96CB51800005382CDB50004d0 ONLINE 0 0 0
c0t600144F0C96CB51800005382CDCC0005d0 ONLINE 0 0 0

errors: No known data errors

pool: rpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM


rpool ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0

errors: No known data errors

# zfs list -r nfspool


NAME USED AVAIL REFER MOUNTPOINT
nfspool 174K 1.95G 31K /nfspool
nfspool/nfs 52K 1.95G 32.5K /testnfs
nfspool/nfs@SAVEMYWORK 19.5K - 31.5K -

# df –kh
...
nfspool 2.0G 31K 2.0G 1% /nfspool
nfspool/nfs 2.0G 32K 2.0G 1% /testnfs
...

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. Recover the snapshot that migrated along with the pool.


# cd /testnfs

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# ls
file1 file3 file4 my_letter
# zfs rollback nfspool/nfs@SAVEMYWORK
# ls
file1 file2 file3 my_letter
5. Exit the root role.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Chapter 7 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 7 - Page 8
Practices for Lesson 7: Configuring Volume Management with Oracle Solaris ZFS
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 8:
Configuring Volume
Management with Solaris
Volume Manager
Chapter 8

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 8: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you configure Solaris Volume Manager for use with Oracle Solaris Cluster.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 8-1: Configuring Volume Management by Using Solaris


Volume Manager

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you configure Solaris Volume Manager by performing the following tasks:
• Initialize the Solaris Volume Manager local metadb replicas.
• Select the Solaris Volume Manager demo volume disk drives.
• Configure Solaris Volume Manager disk sets and volumes for a web server.
• Create a cluster file system.
• Test cluster file systems.
• Manage disk sets.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_ Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.
At least one local disk on each cluster node must have a small unused slice that you can use for
the local metadb replicas. The examples in this practice use slice 6 of the boot disk, which must
be reserved ahead of time. During this practice, you create two data service disk sets, each of
which contains a single mirrored volume.

Task 1: Initialize the Solaris Volume Manager Local metadb Replicas


Before you can use Solaris Volume Manager to create disk sets and volumes, you must
initialize the state database and create one or more replicas.
Perform the following steps in the master console window on the Administrative Workstation
VM:
1. Assume the root role, while logged in as the oracle1 user on the cluster nodes.
2. Verify that the boot disk has a small slice, s6, reserved for creating state database replica.
# zpool status
pool: rpool
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM


rpool ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0

errors: No known data errors

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

# prtvtoc /dev/rdsk/c1t0d0s0
* /dev/rdsk/c1t0d0s0 partition map

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
*
* Dimensions:
* 512 bytes/sector
* 63 sectors/track
* 255 tracks/cylinder
* 16065 sectors/cylinder
* 6526 cylinders
* 6524 accessible cylinders
*
* Flags:
* 1: unmountable
* 10: read-only
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 0 00 16065 104679540 104695604
2 5 01 0 104808060 104808059
6 0 00 104695605 112455 104808059
8 1 01 0 16065 16064
root@clnode1:~#

Note: Output varies from system to system.


3. Create three database replicas on the unused boot disk slice by using the metadb
command.
# metadb -a -f -c 3 c#t#d#s6
4. Verify that the replicas are configured and operational.
# metadb
flags first blk block count
a u 16 8192 /dev/dsk/c1t0d0s6
a u 8208 8192 /dev/dsk/c1t0d0s6
a u 16400 8192 /dev/dsk/c1t0d0s6

Task 2: Select the Solaris Volume Manager Demo Volume Disk Drives
Perform the following steps in the host terminal window of the clnode1 cluster node to select
the Solaris Volume Manager demo volume disk drives:
1. As the oracle1 user or root role, use the cldev list -v command to list all of the
available DID drives.
$ cldev list –v
$ clq status
Make note of the DID numbers of shared disks that are already in use. Make sure that you
do not use those shared disks in this practice.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. Record the logical path and DID path numbers of the two disks that you will use to create
the demonstration disk set, webds, and volumes in the following table:

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Disk Set Volumes Primary Disk Mirror Disk
Webds d100
Note
− You need to record only the last portion of the DID path. The first part is the same
for all DID devices: /dev/did/rdsk.
− Make sure that the disks you select are not local devices. They must be dual-hosted
and available to more than one cluster node.
− For the purpose of this practice, select the d5 and d6 shared disks for the webds
disk set.

Task 3: Configure Solaris Volume Manager Disk Sets and Volumes for a Web
Server
Perform the following steps to create demonstration disk sets and volumes to use in later
practices:
1. Log in as the oracle1 user on the clnode1 cluster node, and assume the root role.
2. Create the webds disk set, and configure the nodes that are physically connected to it.
# metaset -s webds -a -h clnode1 clnode2
3. Add the same nodes as disk set mediators to each disk set.
# metaset -s webds -a -m clnode1 clnode2
4. Add the two disks chosen in Task 2 to the webds disk set.
# metaset -s webds -a /dev/did/rdsk/d5 /dev/did/rdsk/d6
Note: Here, add d5 and d6 to the webds disk set.
5. Verify the status of the new disk set.
# metaset -s webds
Set name = webds, Set number = 1

Host Owner
clnode1 Yes
clnode2

Mediator Host(s) Aliases


clnode1
clnode2

Driv Dbase

d5 Yes

d6 Yes

# medstat -s webds
Mediator Status Golden
clnode1 Ok No
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

clnode2 Ok No

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# cldg status
=== Cluster Device Groups ===

--- Device Group Status ---

Device Group Name Primary Secondary Status


----------------- ------- --------- ------
webds clnode1 clnode2 Online
6. Create a submirror on each of your disks in the webds disk set.
# metainit -s webds d0 1 1 /dev/did/rdsk/d5s0
webds/d0: Concat/Stripe is setup
# metainit -s webds d1 1 1 /dev/did/rdsk/d6s0
webds/d1: Concat/Stripe is setup
7. Create a mirror volume, d99, by using the d0 submirror.
# metainit -s webds d99 -m d0
webds/d99: Mirror is setup
8. Attach the second submirror, d1, to the mirror volume d99.
# metattach -s webds d99 d1
webds/d99: submirror webds/d1 is attached
9. Create a 500 MB soft partition, d100, on top of your mirror. This is the volume that you
actually use for your file system data.
# metainit -s webds d100 -p d99 500m
d100: Soft Partition is setup
10. Verify the status of the new volume.
# metastat -s webds
webds/d100: Soft Partition
Device: webds/d99
State: Okay
Size: 1024000 blocks (500 MB)
Extent Start Block Block count
0 1024 1024000

webds/d99: Mirror
Submirror 0: webds/d0
State: Okay
Submirror 1: webds/d1
State: Resyncing
Resync in progress: 12 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4173824 blocks (2.0 GB)

webds/d0: Submirror of webds/d99


State: Okay

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Size: 4173824 blocks (2.0 GB)


Stripe 0:

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Device Start Block Dbase State Reloc Hot Spare
d5s0 0 No Okay Yes

webds/d1: Submirror of webds/d99


State: Resyncing
Size: 4173824 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
d6s0 0 No Okay Yes

Device Relocation Information:


Device Reloc Device ID
d6 Yes id1,did@n600144f0c96cb51800005382cdf50007
d5 Yes id1,did@n600144f0c96cb51800005382cde70006

Task 4: Create a Cluster File System


Perform the following steps to create a cluster file system in the webds disk set:
1. Ensure that you are logged in as the oracle1 user on the clnode1 cluster node and have
assumed the root role.
2. Create a file system on d100 in the webds disk set by using the following command, and
type y to construct a new file system:
# newfs /dev/md/webds/rdsk/d100
3. On all nodes, create a global mount point for the new file system.
# mkdir /global/web
4. On all nodes, add a mount entry in the /etc/vfstab file for the new file system with the
global mount option.
# vi /etc/vfstab
...
/dev/md/webds/dsk/d100 /dev/md/webds/rdsk/d100 \
/global/web ufs 2 yes global
Note: Do not use the line continuation character (\) in the vfstab file.
5. On the clnode1 cluster node, mount the /global/web file system.
# mount /global/web
6. Verify that the file system is mounted and available on all nodes.
# mount | grep global/web
# ls /global/web
lost+found
# df –kh | grep global/web

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 5: Test Cluster File Systems

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Perform the following steps to test the accessibility of the webds cluster file system from
different cluster nodes:
1. Ensure that you are logged in as the oracle1 user on both the cluster nodes and have
assumed the root role.
2. On the clnode2 cluster node, move into the /global/web file system.
# cd /global/web
3. On the clnode1 cluster node, try to unmount the /global/web file system.
# umount /global/web
You should get an error that the file system is busy.
4. On the clnode2 cluster node, move out of the /global/web file system (cd /) and try to
unmount it again on the clnode1 cluster node.
5. On the clnode2 cluster node, mount the /global/web file system.
6. Try unmounting and mounting /global/web from all nodes.
Ensure that the /global/web cluster file system remains mounted on the clnode1
cluster node before you proceed to the next task.

Task 6: Manage Disk Sets


Perform the following steps to migrate a disk set between cluster nodes:
1. Ensure that you are logged in as the oracle1 user on the clnode1 cluster.
2. Make sure that the webds disk set is online.
$ cldg status

=== Cluster Device Groups ===

--- Device Group Status ---

Device Group Name Primary Secondary Status


----------------- ------- --------- ------
webds clnode1 clnode2 Online

Note: You can bring a device group online to a selected node as follows:
# cldg switch -n node_to_switch_to devgrpname

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Verify the current demonstration disk set configuration.


$ cldg show

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
=== Device Groups ===

Device Group Name: webds


Type: SVM
failback: false
Node List: clnode1, clnode2
preferenced: true
numsecondaries: 1
diskset name: webds
4. Reboot the clnode1 cluster node.
5. On the clnode2 cluster node, verify that the webds disk set has automatically migrated to
the clnode2 cluster node by using the cldg status command.
$ cldg status

=== Cluster Device Groups ===

--- Device Group Status ---

Device Group Name Primary Secondary Status


----------------- ------- --------- ------
webds clnode2 clnode1 Online

6. After clnode1 reboots, the webds disk set should remain mastered by the clnode2
cluster node.
7. Use the cldg switch command from either node to migrate the webds disk set to the
clnode1 cluster node.
# cldg switch -n clnode1 webds
# cldg status

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Chapter 8 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 8 - Page 10
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 8: Configuring Volume Management with Solaris Volume Manager
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 9:
Managing the Public Network
with IPMP
Chapter 9

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 9: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you configure and test IPMP.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 9-1: Configuring and Testing IPMP

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you verify the local-mac-address? variable, verify the adapters for the IPMP
group, and enter test addresses in the /etc/inet/hosts file. Next, you verify that IPMP is
configured and test IPMP failover and failback.
In this practice, you perform the following tasks:
• Verify the local-mac-address? variable.
• Verify the adapters for the IPMP group.
• Verify or enter test addresses in the /etc/inet/hosts file.
• Configure IPMP with public network adapters and test addresses.
• Verify that IPMP is configured.
• Verify IPMP failover and failback.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.

Task 1: Verify the local-mac-address? Variable


Perform the following steps in the master console window on the administrative workstation:
1. Log in as the oracle1 user and assume the root role.
2. Configure the cluster nodes to use unique MAC addresses.
Note: All public-network adapters must use network interface cards (NICs) that support
local MAC address assignment. The local-mac-address? variable must use the default
value true for Ethernet adapters. The Oracle Solaris Cluster software does not support a
local-mac-address? value of false for Ethernet adapters.
# eeprom local-mac-address?=true
# eeprom "local-mac-address?"
local-mac-address?=true
3. Reboot the cluster nodes to enable unique MAC address assignment.
# init 6

Task 2: Verify the Adapters for the IPMP Group


Perform the following steps from the host terminal window of each cluster node, one at a time:
1. Ensure that you are logged in as the oracle1 user on the cluster nodes.
2. Identify the redundant network adapters on the public network.
You might have already made a note of this during an earlier practice.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Your primary public network adapter must be the only one currently configured on the
public network.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
$ ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
sc_ipmp0 ipmp ok yes net0
net0 ip ok yes --
net1 ip ok yes --
net3 ip ok yes --
clprivnet0 ip ok yes --
$ ipadm show-addr
4. You can verify your secondary public network adapter by:
• Making sure that it is not configured as a private transport
• Making sure that it can snoop public network broadcast traffic
Assume the root role and use the following commands:
# ipadm create-ip net2
# snoop -d net2
In another terminal window of the same cluster node or another cluster node, use the
following command:
$ ping -s pubnet_broadcast_addr
For example, ping the broadcast address, 192.168.1.255:
$ ping -s 192.168.1.255

Task 3: Verify or Enter Test Addresses in the /etc/inet/hosts File


It is a good idea, although not required, to have test IP addresses listed in the hosts file.
Although one cluster node does not need to know anything about another cluster node’s test
addresses, it is advisable to have all test addresses in the /etc/inet/hosts file for all nodes
to indicate that these addresses are reserved and what they are reserved for.
Perform the following steps from the host terminal window of each cluster node:
1. Log in as the oracle1 user on the cluster nodes and assume the root role.
2. Verify the IP addresses to be used for each network adapter on each cluster node.
3. Type the IP addresses in /etc/inet/hosts on each node, if they are not already there.
The following is a sample entry of test addresses in the /etc/inet/hosts file of a cluster
node. It really does not matter whether you use the same names as another group working
on another cluster as long as the IP addresses are different:
# IPMP Test Addresses
192.168.1.150 clnode1-local-net2
192.168.1.151 clnode1-net0-ipmp-test
192.168.1.152 clnode1-net2-ipmp-test
192.168.1.153 clnode2-local-net2
192.168.1.154 clnode2-net0-ipmp-test
192.168.1.155 clnode2-net2-ipmp-test

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 4: Configure IPMP with Public Network Adapters and Test Addresses

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Perform the following steps in the host terminal window of each cluster node, one at a time:
1. Assume the root role.
2. View the existing IPMP group information.
# ipmpstat –g
GROUP GROUPNAME STATE FDT INTERFACES
sc_ipmp0 sc_ipmp0 ok -- net0
Note: During cluster configuration, the scinstall utility creates an IPMP group called
sc_ipmp0.
3. Add the second public network adapter to the IPMP group:
# ipadm add-ipmp -i net2 sc_ipmp0
# ipmpstat –g
GROUP GROUPNAME STATE FDT INTERFACES
sc_ipmp0 sc_ipmp0 ok -- net2 net0
4. Add a data address to the second public network adapter, net2.
# ipadm create-addr -T static -a 192.168.1.150/24 sc_ipmp0/v4add2
5. Add test addresses to the first and second public network adapters:
# ipadm create-addr -T static -a 192.168.1.151/24 net0/test
# ipadm create-addr -T static -a 192.168.1.152/24 net2/test
Similarly, perform steps 3 through 5 on the other cluster node. Ensure that you use the
correct second public network adapter, data address, and test addresses.
For example, run the following commands on the other cluster node:
# ipadm add-ipmp -i net2 sc_ipmp0
# ipadm create-addr -T static -a 192.168.1.153/24 sc_ipmp0/v4add2
# ipadm create-addr -T static -a 192.168.1.154/24 net0/test
# ipadm create-addr -T static -a 192.168.1.155/24 net2/test

Task 5: Verify That IPMP Is Configured


Perform the following steps in the host terminal window of each cluster, one at a time:
1. Ensure that you are logged in as the oracle1 user on the cluster node.
2. Verify the IPMP configuration.
$ ipmpstat -a
$ ipmpstat -t
$ ipmpstat –i
3. Verify IPMP cluster-wide status with the clnode status –m command.
$ clnode status –m
--- Node IPMP Group Status ---

Node Name Group Name Status Adapter Status


--------- ---------- ------ ------- ------
clnode2 sc_ipmp0 Online net2 Online
clnode2 sc_ipmp0 Online net0 Online
clnode1 sc_ipmp0 Online net2 Online
clnode1 sc_ipmp0 Online net0 Online

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 6: Verify IPMP Failover and Failback

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Perform the following steps from the host terminal window of a cluster node:
1. As the oracle1 user in the Administrative Workstation system, launch ping -s
nodename, and keep it running.
adminws$ ping –s nodename
2. Simulate the unplugging of the Ethernet cable from the network adapter that currently has
the physical interface of the cluster node on it.
Note: If you do not know to simulate the unplugging of ethernet cable, check with your
instructor.
3. Observe the cluster node messages in the console or in the /var/adm/messages file.
4. Observe the output of the clnode status -m command.
$ clnode status –m

--- Node IPMP Group Status ---

Node Name Group Name Status Adapter Status


--------- ---------- ------ ------- ------
clnode2 sc_ipmp0 Online net2 Online
clnode2 sc_ipmp0 Online net0 Online
clnode1 sc_ipmp0 Online net2 Online
clnode1 sc_ipmp0 Online net0 Offline
5. Observe the behavior of your command from step 1 (keep it running).
6. Simulate the reattachment of the broken Ethernet cable.
7. Observe the messages and the behavior of your command from step 1.
8. Observe the output of the clnode status -m command.
$ clnode status –m

--- Node IPMP Group Status ---

Node Name Group Name Status Adapter Status


--------- ---------- ------ ------- ------
clnode2 sc_ipmp0 Online net2 Online
clnode2 sc_ipmp0 Online net0 Online
clnode1 sc_ipmp0 Online net2 Online
clnode1 sc_ipmp0 Online net0 Online

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 9: Managing the Public Network with IPMP


Chapter 9 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 10:
Managing Data Services,
Resource Groups, and HA-
NFS
Chapter 10

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 10: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you install and configure HA for NFS.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 10-1: Installing and Configuring HA for NFS

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you install and configure HA for NFS.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open the host terminal window for each cluster node and a
master console window.

Task 1: Install and Configure the HA for NFS Agent and Server
In earlier practices, you created the cluster file system for NFS. Confirm that this file system is
available and ready to configure for Oracle Solaris Cluster HA for NFS.
Perform the following steps on the specified cluster nodes as the oracle1 user, unless
otherwise specified:
In the master console window:
1. Verify that the HA for NFS software package is already installed on both cluster nodes.
$ pkg info ha-cluster/data-service/nfs
2. Verify the grace_period property of the NFS protocol.
$ sharectl get -p grace_period nfs
grace_period=90
3. Lower the value of the grace_period property of the NFS protocol.
$ sharectl set -p grace_period=10 nfs
$ sharectl get -p grace_period nfs
grace_period=10
Note: Lowering the grace period speeds up switchovers and failovers of the HA for NFS
resource. Oracle Solaris 11.3 OS uses NFS Version 4 by default. This is a stateful protocol,
which intentionally delays resumption of NFS activity so that clients have a chance to
reclaim their state any time that the server is recovering, which includes any cluster
switchover or failover. The grace_period property controls the length of the delay.
4. Ensure that the /etc/inet/hosts file on the two cluster nodes and the administrative
workstation already contains an entry for the logical host name resource cluster1-nfs.
$ cat /etc/inet/hosts | grep cluster1-nfs
192.168.1.120 cluster1-nfs
5. Identify the node that is currently importing the nfspool ZFS storage pool, which you
created in the lesson titled “Configuring Volume Management with ZFS.”
$ zpool status

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

6. On the node identified in step 5, assume the root role and mount the nfspool/nfs ZFS
file system under /global/nfs.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# zfs set mountpoint=/global/nfs nfspool/nfs
# df –kh | grep nfspool/nfs
nfspool/nfs 2.0G 31K 2.0G 1% /global/nfs

In the host terminal window of the cluster node identified in step 5


7. Create a directory within the /global/nfs file system for the HA for NFS resource to
maintain administrative information.
# cd /global/nfs
# mkdir SUNW.nfs
8. Create the dfstab.nfs-res file in the /global/nfs/SUNW.nfs directory and add the
entry to share the /global/nfs/data directory.
# cd SUNW.nfs
# vi dfstab.nfs-res
share -F nfs -o rw /global/nfs/data
9. Create the directory specified in the dfstab.nfs-res file.
# cd /global/nfs
# mkdir data
# touch data/sample.file
# chmod 777 data
# cd
10. Exit the root role.
Note: You change the mode of the data directory only for the purpose of demonstration. In
this practice, you would be more specific about the share options in the dfstab.nfs-res
file.

Task 2: Register and Configure the Oracle Solaris Cluster HA for NFS Data
Service
Perform the following steps on the specified cluster nodes as the oracle1 user, unless
otherwise specified:
In the host terminal window of any one cluster node:
1. Register the SUNW.nfs and SUNW.HAStoragePlus resource types.
$ clrt register SUNW.nfs
$ clrt register SUNW.HAStoragePlus
$ clrt list –v
Resource Type Node List
------------- ---------
SUNW.LogicalHostname:5 <All>
SUNW.SharedAddress:3 <All>
SUNW.nfs:3.3 <All>
SUNW.HAStoragePlus:11 <All>

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. Create the NFS failover resource group.


$ clrg create –n clnode1,clnode2 -p Pathprefix=/global/nfs nfs-rg

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
3. Create the logical host name resource for the NFS failover resource group.
$ clrslh create -g nfs-rg cluster1-nfs

Note: It is safe to ignore the following warning message:


clrslh: Failed to retrieve netmask for the given hostname(s)/IP(s). Will try
again when the resource being brought online.
4. Create the SUNW.HAStoragePlus resource and set the value of Zpools to nfspool.
$ clrs create -t SUNW.HAStoragePlus -g nfs-rg -p Zpools=nfspool nfs-stor
5. Bring the resource group to a managed state and then bring it online.
$ clrg online -M nfs-rg
6. Create the SUNW.nfs resource and set the value of Resource_dependencies to nfs-
stor resource.
$ clrs create -t SUNW.nfs -g nfs-rg -p Resource_dependencies=nfs-stor nfs-res
7. Verify that the data service is online.
$ clrs list –v
Resource Name Resource Type Resource Group
------------- ------------- --------------
nfs-res SUNW.nfs:3.3 nfs-rg
nfs-stor SUNW.HAStoragePlus:11 nfs-rg
cluster1-nfs SUNW.LogicalHostname:5 nfs-rg

$ clrs status

=== Cluster Resources ===

Resource Name Node Name State Status Message


------------- --------- ----- --------------
nfs-res clnode1 Online Online - Successfully started NFS service.
clnode2 Offline Offline

nfs-stor clnode1 Online Online


clnode2 Offline Offline

cluster1-nfs clnode1 Online Online - LogicalHostname online.


clnode2 Offline Offline
$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Online
clnode2 No Offline

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 3: Verify Access by NFS Clients


In this task, you verify that NFS clients can access the HA for NFS cluster file system.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Perform the following steps on the specified cluster nodes as the oracle1 user, unless
otherwise specified:
On the Administrative Workstation:
1. Verify that you can access the cluster file system.
oracle1@adminws:~$ ls /net/cluster1-nfs/global/nfs/data
sample.file
2. Copy the test.nfs file from the /opt/ora/lab directory into your home directory.
oracle1@adminws:~$ cp /opt/ora/lab/test.nfs $HOME
3. View the $HOME/test.nfs script content.
oracle1@adminws:~$ cat test.nfs
#!/bin/ksh

cd /net/cluster1-nfs/global/nfs/data
while :
do
date > date.out
cat date.out
rm date.out
sleep 1
done
oracle1@adminws:~$
Note: When you run the test.nfs script in the next task, it creates and writes a file
containing a timestamp to the NFS-mounted cluster file system. The script also displays the
file to standard output (stdout). This script helps you determine the duration for which the
NFS data service is interrupted during switchovers and takeovers.

Task 4: Observe Oracle Solaris Cluster HA for NFS Failover Behavior


Perform the following steps on the specified cluster nodes as the oracle1 user, unless
otherwise specified:
In the host terminal window of any one cluster node:
1. Determine the name of the node that is currently primary for the nfs-rg resource group.
$ clrg status nfs-rg

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Online
clnode2 No Offline

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

On the Administrative Workstation:


2. Run the test.nfs script.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
oracle1@adminws:~$ chmod u+x test.nfs
oracle1@adminws:~$ ./test.nfs
In the host terminal window of any one cluster node:
3. Transfer control of the NFS resource to the other node.
$ clrg switch -n dest-node nfs-rg
Substitute the name of your offline node for dest-node.
On the Administrative Workstation:
4. Observe the messages displayed by the test.nfs script.
For example:
oracle1@adminws:~$ ./test.nfs
Thursday, August 19, 2015 07:00:56 AM UTC
Thursday, August 19, 2015 07:00:57 AM UTC
Thursday, August 19, 2015 07:00:58 AM UTC
Thursday, August 19, 2015 07:00:59 AM UTC
Thursday, August 19, 2015 07:01:00 AM UTC
Thursday, August 19, 2015 07:01:01 AM UTC
Thursday, August 19, 2015 07:01:02 AM UTC
Thursday, August 19, 2015 07:01:03 AM UTC
Thursday, August 19, 2015 07:01:04 AM UTC

Thursday, August 19, 2015 07:01:38 AM UTC


Thursday, August 19, 2015 07:01:39 AM UTC
Thursday, August 19, 2015 07:01:40 AM UTC
Thursday, August 19, 2015 07:01:41 AM UTC
Thursday, August 19, 2015 07:01:42 AM UTC
...
5. How long was the data service interrupted during the switchover from one physical host to
another?
__________________________________________________
6. Stop the test.nfs script.
In the master console window:
7. Verify which file systems the nodes are now mounting and exporting by using the share
command.
__________________________________________________
__________________________________________________
__________________________________________________
8. Observe the additional IP address associated with the Logical Hostname resource
configured as a virtual interface on one of the adapters in your IPMP group by using the
ipadm show-addr command.
$ ipadm show-addr

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

In the host terminal window of any one cluster node:


9. On one node of the cluster, transfer control of the NFS service back to its preferred host by

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
using the clrg switch command.
$ clrg switch -n dest-node nfs-rg

Task 5: Generate Cluster Failures and Observe the Behavior of the NFS Failover
Generate failures in your cluster to observe the recovery features.
Try to generate the following failures:
• Power down a cluster node.
• Simulate the unplugging of network cable of one public network interface.
• Simulate the multiple public network failure on a single node.
Try your tests while the resource group is in its normal, nonsuspended state. Repeat some tests
after suspending the resource group:
$ clrg suspend nfs-rg
Note: By suspending a resource group, you will temporarily be suspending the automatic
recovery actions of the resource group. You might need to suspend the automatic recovery
of a resource group to investigate and fix a problem in the cluster.
When you are satisfied with your results, remove the suspension.
$ clrg resume nfs-rg

Task 6: Make a Customized Application Fail Over with a Generic Data Service
Resource
In this task, you can see how easy it is to get any daemon to fail over in the cluster, by using the
Generic Data Service.
Perform the following steps on the specified cluster nodes as the root role:
1. On each cluster node, create a daemon that represents your customized application.
# vi /var/tmp/myappdaemon
#!/bin/ksh
while :
do
sleep 10
done
2. Make sure that the file is executable on all nodes.
3. From any one cluster node, create a new failover resource group for your application.
# clrg create –n clnode1,clnode2 myapp-rg
4. From one cluster node, register the Generic Data Service resource type.
# clrt register SUNW.gds
5. From one cluster node, create the new resource and enable the resource group and
resources.
# clrs create -g myapp-rg -t SUNW.gds -p Start_Command=/var/tmp/myappdaemon \
-p Probe_Command=/bin/true -p Network_aware=false myapp-res
# clrg online -M myapp-rg
# clrg status myapp-rg
# clrs status myapp-res
# ps –ef | grep myappdaemon

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

6. Verify the behavior of your customized application.


a. Verify that you can manually switch the group from the other cluster node.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# clrg switch –n dest_node myapp-rg
# clrg status myapp-rg

b. On the destination cluster node, kill the myappdaemon process.


# ps –ef | grep myappdaemon
# pfexec kill -9 pid
# ps –ef | grep myappdaemon
$ clrs status

Wait a little while and note that the myappdaemon process restarts on the same node.
Also, wait until clrs status shows that the resource is fully online again.
c. Repeat step 6b a few times.
Eventually, the resource group switches over to the other node.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Chapter 10 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 10 - Page 10
Practices for Lesson 10: Managing Data Services, Resource Groups, and HA-NFS
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 11:
Configuring Scalable
Services and Advanced
Resource Group
Relationships
Chapter 11

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 11: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you install and configure Apache web server as a scalable data service on the
Oracle Solaris Cluster software.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 11-1: Installing and Configuring Apache as a Scalable


Service on Oracle Solaris Cluster

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you install and configure Apache as a scalable service on the Oracle Solaris
Cluster software.
In this practice, you perform the following tasks:
• Prepare for Apache data service configuration.
• Configure the Apache environment.
• Test the server on each node before configuring the data service resources.
• Register and configure the Apache data service.
• Verify the Apache web server access.
• Observe cluster failures.
• Configure advanced resource group relationships.

Preparation
Before proceeding with the tasks in this practice, start the Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, in that
order. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open a master console window and the host terminal
window for each cluster node.

Task 1: Prepare for Apache Data Service Configuration


Perform the following steps on each node of the cluster:
1. Ensure that you are logged in as the oracle1 user on the cluster node.
2. Verify that the Solaris Cluster Apache data service software package is installed.
$ pkg info apache
3. Ensure that the /etc/inet/hosts file on the two cluster nodes and the administrative
workstation contains an entry for the logical host name resource cluster1-web.
$ cat /etc/inet/hosts | grep cluster1-web
192.168.1.121 cluster1-web

Task 2: Configure the Apache Environment


Perform the following steps on any one of the cluster node:
1. While logged in as the oracle1 user on the cluster node, assume the root role.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

2. Make a resource-specific copy of the /usr/apache2/bin/apachectl script and edit it.


# mkdir /global/web/bin

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# cp /opt/ora/lab/apachectl /global/web/bin/
# chmod u+w /global/web/bin/apachectl
# vi /global/web/bin/apachectl
a. Add a line to make an Apache runtime directory. This directory is deleted every time
you reboot. Letting this script create it resolves the problem. Just add the line in bold as
the second line of the file.
#!/bin/sh
mkdir -p /var/run/apache2
b. Locate the line:
HTTPD='/usr/apache2/2.2/bin/httpd'
And change it to:
HTTPD='/usr/apache2/2.2/bin/httpd -f /global/web/conf/httpd.conf'
3. Copy the sample /etc/apache2/2.2/httpd.conf example file to the
/global/web/conf/httpd.conf file.
# mkdir /global/web/conf
# cp /etc/apache2/2.2/httpd.conf /global/web/conf/httpd.conf
4. Edit the /global/web/conf/httpd.conf file, and change the entries as shown in the
following table:
Old Entry New Entry
Listen 80 Listen cluster1-web:80
ServerName 127.0.0.1 ServerName cluster1-web
DocumentRoot "/var/apache2/2.2/htdocs" DocumentRoot "/global/web/htdocs"
<Directory "/var/apache2/2.2/htdocs"> <Directory "/global/web/htdocs">
ScriptAlias /cgi-bin/ ScriptAlias /cgi-bin/ "/global/web/cgi-
"/var/apache2/2.2/cgi-bin/" bin/"
<Directory "/var/apache2/2.2/cgi-bin"> <Directory "/global/web/cgi-bin">

Note: The changes mentioned in the table are shown in the order of their appearance in the
httpd.conf file, so you can search for the first place to change, change it, then search for
the next, and so on.
5. Create directories for the HTML and CGI files and populate with the sample files.
# cp -rp /var/apache2/2.2/htdocs /global/web
# cp -rp /var/apache2/2.2/cgi-bin /global/web
6. Copy the test-apache.cgi file from the classroom server to /global/web/cgi-bin.
# cp /opt/ora/lab/test-apache.cgi /global/web/cgi-bin/
You use the test-apache.cgi file to test the scalable service.
7. Make sure that test-apache.cgi can be executed by all users.
# chmod 755 /global/web/cgi-bin/test-apache.cgi
8. Exit the root role.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Task 3: Test the Server on Each Node Before Configuring the Data Service
Resources

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Perform the following steps on both the cluster nodes, one node at a time:
1. While logged in as the oracle1 user on a cluster node, assume the root role.
2. Temporarily configure the logical shared address (on one node).
# ipadm create-addr -T static -a 192.168.1.121/24 net0/web
3. Start the server (on that node).
# /global/web/bin/apachectl start
4. Verify that the server is running.
clnode1:/# ps –ef | grep apache2
webservd 4604 4601 0 10:20:05 ? 0:00 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
webservd 4603 4601 0 10:20:05 ? 0:00 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
webservd 4605 4601 0 10:20:05 ? 0:00 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
root 4601 1 0 10:20:04 ? 0:01 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
webservd 4606 4601 0 10:20:05 ? 0:00 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
webservd 4602 4601 0 10:20:05 ? 0:00 /usr/apache2/2.2/bin/httpd
-f /global/web/conf/httpd.conf -k start
5. Connect to the Apache web server from the web browser on your Administrative
Workstation by using the following URL:
http://cluster1-web/

6. Stop the Apache web server.


# /global/web/bin/apachectl stop
7. Verify that the server has stopped.
# ps -ef | grep apache2
8. Delete the logical IP address.
# ipadm delete-addr net0/web

Task 4: Register and Configure the Apache Data Service


Perform the following steps on any one of the cluster nodes:
1. Ensure that you are logged in as the oracle1 user on the cluster node.
2. Register the resource type required for the Apache data service.
$ clrt register SUNW.apache
3. Create a failover resource group for the shared address resource. If you have more than
two nodes, you can include more than two nodes after the -n.
$ clrg create -n clnode1,clnode2 sa-rg

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. Add the SharedAddress logical host name resource to the resource group.
$ clrssa create -g sa-rg cluster1-web

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Note: It is safe to ignore the following warning message:
clrssa: Failed to retrieve netmask for the given hostname(s)/IP(s). Will try
again when the resource being brought online.
5. Bring the failover resource group online.
$ clrg online -M sa-rg
6. Create a scalable resource group to run on all nodes of the cluster. (The example assumes
two nodes.)
$ clrg create -S -n clnode1,clnode2 web-rg
7. Add a storage resource to the scalable group.
$ clrs create -g web-rg -t SUNW.HAStoragePlus \
-p FilesystemMountPoints=/global/web -p AffinityOn=true web-stor
8. Add an application resource to the scalable resource group.
$ clrs create -g web-rg -t SUNW.apache -p Bin_dir=/global/web/bin \
-p Scalable=true -p Resource_dependencies=cluster1-web,web-stor \
apache-res
9. Bring the scalable resource group online.
$ clrg online -M web-rg
10. Verify that the data service is online.
$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Offline
clnode2 No Online

myapp-rg clnode1 No Online


clnode2 No Offline

sa-rg clnode1 No Online


clnode2 No Offline

web-rg clnode1 No Online


clnode2 No Online
$ clrs status

=== Cluster Resources ===

Resource Name Node Name State Status Message


------------- --------- ----- --------------
nfs-res clnode1 Offline Offline
clnode2 Online Online - Service is online.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

nfs-stor clnode1 Offline Offline


clnode2 Online Online

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
cluster1-nfs clnode1 Offline Offline
clnode2 Online Online - LogicalHostname online.

myapp-res clnode1 Online Online - Service is online.


clnode2 Offline Offline

cluster1-web clnode1 Online Online - SharedAddress online.


clnode2 Offline Offline

apache-res clnode1 Online Online – Service is online.


clnode2 Online Online - Service is online.

web-stor clnode1 Online Online


clnode2 Online Online

Task 5: Verify the Apache Web Server Access


Perform the following steps on the Administrative Workstation:
1. Open a web browser, such as Firefox.
2. Connect to the web server from a browser anywhere other than from a cluster node by
using http://cluster1-web/cgi-bin/test-apache.cgi.
3. Click the Refresh or Reload button repeatedly on the browser.
The test-apache.cgi script shows the name of the cluster node that is currently
servicing the request. The load balancing is not performed on a round-robin basis, so you
might see several consecutive requests serviced by the same node. Over time, however,
you should see the load balancing to be about 50-50.

Task 6: Observe Cluster Failures


Cause as many of the following failures (one at a time, and fix each one before going on to the
next one) as you can, given your physical access to the cluster.
Induce the following failure scenarios to observe the behavior of the scalable service:
• Fail a single public network interface on the node where the SharedAddress resource
is online.
• Fail all public network interfaces on that node.
• Reboot one of the nodes running the scalable service.
• Shut down one of the cluster nodes running the scalable service.

Task 7: Configure Advanced Resource Group Relationships


In this task, you observe the effects of configuring variations of the RG_affinities property.
Perform the following steps on any one of the cluster nodes:
1. Ensure that you are logged as the oracle1 user on the cluster node.
2. If you have not yet done so, complete Task 6 titled “Make a Customized Application Fail
Over with a Generic Data Service Resource” of Practice 10-1.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Put your myapp-rg resource group offline.


$ clrg offline myapp-rg

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
4. Create a strong negative affinity with the Apache resource group as the source and
myapp-rg as the target. This means that Apache categorically refuses to run on any node
where myapp-rg is running.
$ clrg set -p RG_affinities=--myapp-rg web-rg
5. Switch myapp-rg to any cluster node where Apache is running. Observe what happens to
the Apache data service. Also, observe the console messages.
$ clrg status web-rg

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
web-rg clnode1 No Online
clnode2 No Online
$ clrg status myapp-rg

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
myapp-rg clnode1 No Offline
clnode2 No Offline

$ clrg switch -n apacheclusternode myapp-rg


(C258827) NOTICE: Resource group(s) {web-rg on node clnode1} were either
switched to different nodes or brought offline as a result of their strong
resource group affinities for resource group myapp-rg.

$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------

myapp-rg clnode1 No Online


clnode2 No Offline

web-rg clnode1 No Offline


clnode2 No Online

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

6. Switch myapp-rg to another node where Apache is running. Observe what happens to the
Apache data service. Does it come back online on the first node?

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
$ clrg switch -n apacheclusternode myapp-rg
(C258827) NOTICE: Resource group(s) {web-rg on node clnode2} were either
switched to different nodes or brought offline as a result of their strong
resource group affinities for resource group myapp-rg.
$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------

myapp-rg clnode1 No Offline
clnode2 No Online

web-rg clnode1 No Online
clnode2 No Offline

7. Switch myapp-rg offline. Can Apache come back online on both the cluster nodes? Now
remove the relationship.
$ clrg offline myapp-rg
$ clrg status
$ clrg set -p RG_affinities="" web-rg
8. Set a weak positive affinity so that myapp-rg (source) always prefers to run on the same
node as nfs-rg (target).
$ clrg set -p RG_affinities=+nfs-rg myapp-rg
9. Print out the value of Nodelist for myapp-rg.
$ clrg show -p Nodelist myapp-rg

=== Resource Groups and Resources ===

Resource Group: myapp-rg


Nodelist: clnode1 clnode2
10. Switch your nfs-rg so that it is not on the preferred node of myapp-rg.
$ clrg status
$ clrg switch -n nonpreferrednode nfs-rg

11. Bring myapp-rg online without specifying the node. Where does it end up? Why does it not
end up on the nonpreferred node of its own node list?
$ clrg online myapp-rg
$ clrg status
12. Switch myapp-rg so it is no longer on the same node as nfs-rg. Check whether this can
be done. Notice that a weak affinity is a preference and not a requirement.
$ clrg switch -n othernode myapp-rg
$ clrg status

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

13. Switch myapp-rg offline and now change the affinity to a strong positive affinity with
delegation. What is the difference between ++ and +++ (use +++)? The answer is a few
steps further on.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
$ clrg offline myapp-rg
$ clrg set -p RG_affinities=+++nfs-rg myapp-rg
14. Bring myapp-rg online without specifying the node. Identify the node on which it becomes
online.
$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Offline
clnode2 No Online

myapp-rg clnode1 No Offline


clnode2 No Offline

$ clrg online myapp-rg


(C254375) NOTICE: Resource group nfs-rg was also included in the request, due
to a +++ affinity of resource group myapp-rg (possibly transitively).

$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Offline
clnode2 No Online

myapp-rg clnode1 No Offline


clnode2 No Online

Note: The above output is given only for your reference.


15. Switch myapp-rg so that it is no longer on the same node as nfs-rg. Note what happens.
$ clrg switch -n othernode myapp-rg
clrg: (C406107) Cannot switch resource group myapp-rg online on node clnode1
because it has strong positive affinities for resource group(s) {nfs-rg},
which are not online on that node.

$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

---------- --------- --------- ------


nfs-rg clnode1 No Offline

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
clnode2 No Online

myapp-rg clnode1 No Offline


clnode2 No Online

16. Switch the target group nfs-rg. Note what happens.
$ clrg switch -n othernode nfs-rg
(C258827) NOTICE: Resource group(s) {myapp-rg on node clnode2} were either
switched to different nodes or brought offline as a result of their strong
resource group affinities for resource group nfs-rg.

$ clrg status

=== Cluster Resource Groups ===

Group Name Node Name Suspended Status


---------- --------- --------- ------
nfs-rg clnode1 No Online
clnode2 No Offline

myapp-rg clnode1 No Online


clnode2 No Offline

17. What happens if Resource Group Manager (RGM) wants to fail over the source myapp-rg
because its fault monitor indicates application failure?
Kill myappdaemon on whichever node it is running a few times (be patient with restarts) and
observe the console messages.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Chapter 11 - Page 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 11 - Page 12
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 11: Configuring Scalable Services and Advanced Resource Group Relationships
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 12:
Configuring Oracle Solaris
Zones in Oracle Solaris
Cluster
Chapter 12

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 12: Overview

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you perform the following tasks:
• Create a zone cluster and recreate your scalable web application within the zone
cluster.
• Import an installed zone into an existing zone cluster.
• Configure the Apache scalable application, running in the global cluster, as a scalable
application in the zone cluster by reusing the Apache binaries of the global cluster.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 12-1: Building a Zone Cluster

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you create a zone cluster and recreate your scalable web application within the
zone cluster.

Preparation
Before proceeding with the tasks in this practice, start Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, one after
the other. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open the host terminal window for each cluster node and
the master console window.

Task 1: Create Static Addresses for Zone Cluster Nodes


On all cluster nodes, edit the /etc/inet/hosts file as the root role and provide static IP
addresses on your public subnet for the two nodes (zones) of the zone cluster.
# vi /etc/inet/hosts
192.168.1.140 zcnode1
192.168.1.141 zcnode2

Task 2: Configure a Zone Cluster


Perform the following steps on the specified cluster node to configure a zone cluster:
1. Assume the root role on a cluster node, such as clnode1.
2. Create a zone cluster, myzc.
# clzc configure myzc
myzc: No such zone cluster is configured.
Use 'create' to begin configuring a new cluster zone
clzc:myzc> create
clzc:myzc> set zonepath=/zones/myzc
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode1
clzc:myzc:node> set hostname=zcnode1
clzc:myzc:node> add net
clzc:myzc:node:net> set address=zcnode1
clzc:myzc:node:net> set physical=sc_ipmp0
clzc:myzc:node:net> end
clzc:myzc:node> end
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode2
clzc:myzc:node> set hostname=zcnode2
clzc:myzc:node> add net
clzc:myzc:node:net> set address=zcnode2
clzc:myzc:node:net> set physical=sc_ipmp0
clzc:myzc:node:net> end

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

clzc:myzc:node> end
clzc:myzc> commit

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
clzc:myzc> exit
3. Verify the zone cluster configuration.
# clzc verify myzc

Task 3: Install the Zone Cluster (On any one of the cluster nodes)
Perform the following steps on the specified cluster node to install a zone cluster:
1. Log in to any one cluster node as the root role.
2. Install the zone cluster myzc by using the clzc install command.
The -v option shows the output of the underlying zoneadm command on the console of
each cluster node:
# clzc status myzc
=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Offline Configured
clnode2 zcnode2 Offline Configured

# clzc install -v myzc


Waiting for zone install commands to complete on all the nodes of the zone
cluster "myzc"...

// zoneadm output for installation appears on the physical node consoles only

# clzc status myzc


=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Offline Installed
clnode2 zcnode2 Offline Installed
Note: The zone cluster installation could take anywhere from 30 to 45 minutes.

Task 4: Boot the Zone Cluster


From any global cluster node, as the root role, boot all zones in the zone cluster.
Note: The zone name (from the point of view of the zone commands in the global zone) will
be the same as the name of the zone cluster (myzc) on all physical nodes.
# clzc boot myzc
Waiting for zone boot commands to complete on all the nodes of the zone
cluster "myzc"...
# clzc status myzc

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

=== Zone Clusters ===

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Offline Running
clnode2 zcnode2 Offline Running

Task 5: Access the Zone Consoles


In this task, you will access the zone console and customize your zone.
Perform the following steps on each global cluster node, one at a time:
1. Ensure that you have assumed the root role on each global cluster node.
2. Connect to the zone console on each global cluster node.
# zlogin -C myzc
You still see the zone doing its first boot. Navigate through the System Configuration Tool
screens and specify the following values, when asked for:
• Computer name: zcnode1/zcnode2
• DNS Name Service: Do not configure DNS
• Name Service: None
• Time Zone: timezone
• Root password: 2secure
• Username: oracle1
• User password: oracle1
When you are finished with the system configuration of each zone cluster node, it will
display the console login prompt.
zcnode console login:
3. Log in to each zone cluster node as the oracle1 user and assume the root role.
4. Set your zone cluster environment on both the cluster nodes:
a. Create a better login environment.
# vi ~/.profile
...
export PATH=/usr/cluster/bin:/usr/bin:/usr/sbin
...
# . .profile
b. Make any other desired customizations (for example, change your shell if you want, but
do it carefully).
5. While still logged in to each zone cluster node, explore the zone cluster environment from
the point of view of the zone:
# uname -a
# cat /etc/inet/hosts
# clnode status
# clq status // see an error: You cannot run this command from zone cluster
# clrg status

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

# clrs status
# clrt list –v

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
6. Exit from the zone console by using the ~. or ~~. key combinations.

Task 6: Explore the Zone Cluster from the Global Zone Point of View
Perform the following steps on the global cluster node:
1. Ensure that you are logged as the oracle1 user on the global cluster node, not the zone
cluster node.
2. Explore various cluster resource and resource group status options:
$ clnode status
$ clnode status -Z myzc
$ clnode status -Z all

$ clrg status
$ clrg status -Z myzc
$ clrg status -Z all

$ clrs status
$ clrs status -Z myzc
$ clrs status -Z all

$ clrs list -v
$ clrs list -v -Z myzc
$ clrs list -v -Z all

$ clrt list -v
$ clrt list -v -Z myzc
$ clrt list -v -Z all

Task 7: Experiment with Zone Cluster Failures


Cause as many of the following failures as you can, given access to the cluster. Ensure that you
induce the failure only one at a time, and fix each one before going on to induce the next failure.
Induce the following failure scenarios to observe the behavior of the zone cluster:
• Halt a zone in a zone cluster.
• Reboot a zone in a zone cluster.
• Reboot a cluster node hosting a zone in a zone cluster.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 12-2: Importing an Installed Zone into an Existing Zone


Cluster

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you import the installed zone (zone1) into an existing zone cluster (myzc).

Task 1: Removing a Zone from a Zone Cluster


Perform the following steps to remove the clnode1 cluster node from a zone cluster in an
organized fashion:
Perform the following steps on the specified cluster node to remove the zone cluster:
1. Assume the root role on the clnode1 cluster node.
2. Verify the status of the myzc zone cluster.
root@clnode1:~# clzc status myzc
=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Online Running
clnode2 zcnode2 Online Running
3. Shut down the zone-cluster node you want to remove by specifying the node and its zone
cluster.
root@clnode1:~# clzc halt -n clnode1 myzc
Waiting for zone halt commands to complete on all the nodes of the zone
cluster "myzc"...
root@clnode1:~# clzc status myzc

=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Offline Installed
clnode2 zcnode2 Online Running

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 7
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. Uninstall the zone-cluster node.


root@clnode1:~# clzc uninstall -n clnode1 myzc

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Are you sure you want to uninstall zone cluster myzc (y/[n])?y
Waiting for zone uninstall commands to complete on all the nodes of the zone
cluster "myzc"...
root@clnode1:~# clzc status myzc
=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode1 zcnode1 Offline Configured
clnode2 zcnode2 Online Running
5. Remove the zone-cluster node from configuration and verify the status of the myzc zone
cluster.
root@clnode1:~# clzc configure myzc
clzc:myzc> remove node physical-host=clnode1
clzc:myzc> info
zonename: myzc
zonepath: /zones/myzc
autoboot: true
hostid:
brand: solaris
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
enable_priv_net: true
attr:
name: cluster
type: boolean
value: true
node:
physical-host: clnode2
hostname: zcnode2
net:
address: zcnode2
physical: sc_ipmp0
defrouter not specified
allowed-address not specified
configure-allowed-address not specified
clzc:myzc> exit
root@clnode1:~#

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 8
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

6. Verify that the node was removed from the zone cluster.
root@clnode1:~# clzc status myzc

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode2 zcnode2 Online Running
root@clnode1:~#

Task 2: Importing a New Zone into a Zone Cluster


Perform the following steps on the specified cluster node to import the zone into the zone
cluster:
1. Assume the root role on a cluster node, such as clnode1.
2. Verify that the zone that you want to import is in an installed state.
root@clnode1:~# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
1 zone1 running /zones/zone1 solaris Shared
root@clnode1:~#
3. Halt the zone1 zone.
The zone to be imported must be in an installed state.
root@clnode1:~# zoneadm -z zone1 halt
root@clnode1:~# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
- zone1 installed /zones/zone1 solaris shared
root@clnode1:~#
4. Run the clzonecluster configure command on a configured zone cluster to set the
necessary properties and import the zone.
root@clnode1:~# clzc configure myzc
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode1
clzc:myzc:node> import-zone zonename=zone1
This operation renames the zone: zone1 to the zone-cluster's name: myzc. Do
you want to proceed ([y]/n)? y
Validating the resources and properties...
Import Operation Failed: Error while validating zone properties.
Zone:'zone1' is not in the 'installed' state.
Zonepath of zone cluster does not match zonepath of the zone:'zone1'
clzc:myzc:node> exit
Are you able to import the zone? Why? Zone paths are different.
When you import an installed zone into a new zone cluster, zone properties such as
zonepath, ip-type, and brand must be defined in the zone cluster configuration before
you run the import-zone command.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 9
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

5. Verify that the zonepath of the zone and the zone cluster are the same.
a. Verify the zonepath of the zone1 zone.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
clnode1:~# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
- zone1 installed /zones/zone1 solaris shared
b. Verify the zonepath of the myzc zone cluster.
# clzc show -v myzc | grep zonepath
zonepath: /zones/myzc
or
# zonecfg –z myzc info | grep zonepath
zonepath: /zones/myzc
c. Move the zonepath of the zone1 zone if it is different from the myzc zone cluster.
From the above output, you can observe that the zonepath of zone1 and myzc are
different.
root@clnode1:~# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
- zone1 installed /zones/zone1 solaris shared
root@clnode1:~# zoneadm -z zone1 move /zones/myzc
root@clnode1:~# zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
- zone1 installed /zones/myzc solaris shared
root@clnode1:~#
6. Run the clzonecluster configure command on a configured zone cluster to import
the zone1 zone.
root@clnode1:~# clzc configure myzc
clzc:myzc> add node
clzc:myzc:node> set physical-host=clnode1
clzc:myzc:node> import-zone zonename=zone1
This operation renames the zone: zone1 to the zone-cluster's name: myzc. Do
you want to proceed ([y]/n)? y
Validating the resources and properties...
Extracting Zone Configuration data ...
Importing zone configuration into the zone cluster....
clzc:myzc:node> set hostname=zcnode1
clzc:myzc:node> add net
clzc:myzc:node:net> set address=zcnode1
clzc:myzc:node:net> set physical=sc_ipmp0
clzc:myzc:node:net> end
clzc:myzc:node> end
clzc:myzc> commit
Renaming the zone:zone1 to myzc...
clzc:myzc> exit
root@clnode1:~#

root@clnode1:~# clzc status myzc


Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 10
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

=== Zone Clusters ===

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode2 zcnode2 Online Running
clnode1 zcnode1 Offline Installed

Boot the myzc zone cluster.


root@clnode1:~# clzc boot -n clnode1 myzc

root@clnode1:~# clzc status myzc

=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode2 zcnode2 online Running
clnode1 zcnode1 Offline Running
root@clnode1:~#
7. Install the cluster packages on zcnode1.
root@clnode1:~# clzc install-cluster -n clnode1 myzc
Installing the cluster packages in zone cluster "myzc" ...
root@clnode1:~# clzc status myzc

=== Zone Clusters ===

--- Zone Cluster Status ---

Name Brand Node Name Zone Host Name Status Zone Status
---- ----- --------- -------------- ------ -----------
myzc solaris clnode2 zcnode2 Online Running
clnode1 zcnode1 Online Running

root@clnode1:~#

Now, you have successfully imported the zone1 zone into an existing zone cluster
(myzc).
8. Change the hostname from zone1 to zcnode1.
Note: It is not required to change the hostname of zone1. However, you must change the
hostname so that other practices are functional.
root@clnode1# zlogin myzc
root@zone1# svccfg -s system/identity:node setprop config/nodename="zcnode1"
root@zone1# svccfg -s system/identity:node setprop config/loopback="zcnode1"
root@zone1# svccfg -s system/identity:node refresh
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 11
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

root@zone1# svcadm restart system/identity:node


root@zone1# svccfg -s system/identity:node listprop config

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
config application
config/enable_mapping boolean true
config/ignore_dhcp_hostname boolean false
config/nodename astring zcnode1
config/loopback astring zcnode1
root@zone1# exit

Login and verify the changes.

root@clnode1# zlogin myzc


[Connected to zone 'myzc' pts/2]
Last login: Fri Jan 8 12:59:08 2016 on pts/2
Oracle Corporation SunOS 5.11 11.3 September 2015
root@zcnode1# hostname
zcnode1

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 12
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 12-3: Configuring a Scalable Application in a Zone Cluster

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you configure the Apache scalable application, running in the global cluster, as
a scalable application in the zone cluster by reusing the Apache binaries of the global cluster.

Preparation
Before proceeding with the tasks in this practice, start Sun_ZFS_Storage_7000,
Cluster_Node_1, Cluster_Node_2, and Administrative Workstation VMs, one after
the other. Wait for the VMs to boot properly until you see the login prompt.
Before you begin this practice, ensure that you have completed the tasks covered in the
previous practices.
For the purpose of this practice, run the pconsole utility on the Administrative
Workstation (adminws) system to open the host terminal window for each cluster node and
the master console window.

Task 1: Delete the Scalable Apache Data Service from the Global Cluster
Perform the following steps on a global cluster node:
1. Ensure that you are logged in as the oracle1 user on the global cluster node.
2. From any node of the global cluster (logged into the physical node, not the zone), delete the
Apache resource groups and all their contained resources.
$ clrg delete -F web-rg
$ clrg delete -F sa-rg

Task 2: Adding the Cluster File System to a Zone Cluster


Perform the following steps to add a cluster file system for the Apache web server data service
to the zone cluster:
1. Assume the root role while logged in as the oracle1 user on any one cluster node.
2. Add the /zonecfs/web-lofs highly available local file system for use by the myzc zone
cluster. Ensure that Oracle Solaris Cluster does not manage the loopback file system
during the initial zone cluster configuration by setting the value of the cluster-control
property to false.
# clzc configure myzc
clzc:myzc> add net
clzc:myzc:net> set address=web-lofs-lh
clzc:myzc:net> end
clzc:myzc> add fs
clzc:myzc:fs> set dir=/zonecfs/web-lofs
clzc:myzc:fs> set special=/global/web
clzc:myzc:fs> set type=lofs
clzc:myzc:fs> set cluster-control=false
clzc:myzc:fs> end
clzc:myzc> verify
clzc:myzc> commit
clzc:myzc> exit

# clzc show –v myzc

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 13
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

=== Zone Clusters ===

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Zone Cluster Name: myzc
zonename: myzc
zonepath: /zones/myzc
autoboot: TRUE
brand: solaris
hostid: <NULL>
bootargs: <NULL>
pool: <NULL>
limitpriv: <NULL>
scheduling-class: <NULL>
ip-type: shared
enable_priv_net: TRUE
resource_security: SECURE

--- Solaris Resources for myzc ---

Resource Name: net


address: web-lofs-lh
physical: auto

Resource Name: fs
dir: /zonecfs/web-lofs
special: /global/web
raw:
type: lofs
options: []
cluster-control: false

--- Zone Cluster Nodes for myzc ---

Node Name: clnode1


physical-host: clnode1
hostname: zcnode1

--- Solaris Resources for clnode1 ---

Resource Name: net


address: zcnode1
physical: sc_ipmp0
defrouter: <NULL>
allowed-address: <NULL>

Node Name: clnode2


physical-host: clnode2
hostname: zcnode2

--- Solaris Resources for clnode2 ---

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 14
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Resource Name: net


address: zcnode2

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
physical: sc_ipmp0
defrouter: <NULL>
allowed-address: <NULL>

Note: Output might differ.


3. Assume the root role and log in to the myzc zone.
# zlogin myzc
4. Create the /zonecfs/web-lofs directory on both the zone cluster nodes.
# mkdir –p /zonecfs/web-lofs
5. Ensure that the /etc/inet/hosts file on both the zone cluster nodes include entries for
the two zone clusters and for web-lofs-lh.
# cat /etc/inet/hosts
...
192.168.1.140 zcnode1
192.168.1.141 zcnode2
...
192.168.1.124 web-lofs-lh
...
6. Exit the root role and then reboot the myzc zone cluster from any one global cluster node.
# clzc reboot myzc

Task 3: Configure the Apache Web Server Environment in the Zone Cluster
Perform the following steps on any one zone cluster node:
1. Ensure that you are logged in as the root role on a zone cluster node.
# zlogin myzc
2. Edit the HTTPD entry in the /zonecfs/web-lofs/bin/apachectl script:
HTTPD='/usr/apache2/2.2/bin/httpd -f /global/web/conf/httpd.conf'
And change it to:
HTTPD='/usr/apache2/2.2/bin/httpd -f /zonecfs/web-lofs/conf/httpd.conf'
3. Edit the /zonecfs/web-lofs/conf/httpd.conf file, and change the entries as shown
in the following table:
Old Entry New Entry
Listen cluster1-web:80 Listen web-lofs-lh:80
ServerName cluster1-web ServerName web-lofs-lh
DocumentRoot "/global/web/htdocs" DocumentRoot "/zonecfs/web-lofs/htdocs"
<Directory "/global/web/htdocs"> <Directory "/zonecfs/web-lofs/htdocs">
ScriptAlias /cgi-bin/ /global/web/cgi- ScriptAlias /cgi-bin/ "/zonecfs/web-
bin/" [single line] lofs/cgi-bin/" [single line]
<Directory "/global/web/cgi-bin"> <Directory "/zonecfs/web-lofs/cgi-bin">

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 15
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Note: The changes mentioned in the preceding table are shown in the order of their
appearance in the httpd.conf file, so you can search for the first place to change,
change it, then search for the next, and so on.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
4. Exit the zone cluster node to return to the global cluster node.
5. Change the myzc zone cluster configuration to ensure that Oracle Solaris Cluster now
manages the loopback file system by setting the value of the cluster-control property
to true.
# clzc configure myzc
clzc:myzc> select fs dir=/zonecfs/web-lofs
clzc:myzc:fs> set cluster-control=true
clzc:myzc:fs> end
clzc:myzc> verify
clzc:myzc> commit
clzc:myzc> exit
# clzc show –v myzc
6. Reboot the myzc zone cluster.
# clzc reboot myzc

Task 4: Create a Scalable Apache Data Service in the Zone Cluster


Perform the following steps on any one zone cluster node:
1. Log in as the root role on a zone cluster node.
# zlogin myzc
2. Register resource types in the zone cluster.
# clrt list -v
# clrt register SUNW.apache
# clrt register SUNW.HAStoragePlus
Note: It is safe to ignore warning messages, if any.

# clrt list -v
3. Create a scalable resource group containing the shared address.
# clrg create -n zcnode1,zcnode2 sa-rg
# clrssa create -g sa-rg web-lofs-lh

Note: It is safe to ignore the following warning message:


clrssa: Failed to retrieve netmask for the given hostname(s)/IP(s). Will try
again when the resource being brought online.

# clrg online -M sa-rg


# clrg status
# clrs status

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 16
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

4. Create a scalable group containing the Apache resource.


# clrg create -S -n zcnode1,zcnode2 web-rg

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# clrs create -g web-rg -t SUNW.HAStoragePlus \
-p FilesystemMountPoints=/zonecfs/web-lofs -p AffinityOn=false web-stor
# clrg online -M web-rg
# clrg status
# clrs status
# clrs create -g web-rg -t SUNW.apache -p Bin_dir=/zonecfs/web-lofs/bin \
-p Scalable=true -p Resource_dependencies=web-lofs-lh,web-stor apache-res
# clrg status
# clrs status

Task 5: Verify That the Web Server Works


Perform the following steps on the Administrative Workstation:
1. Open a web browser, such as Firefox.
2. Navigate to http://web-lofs-lh/cgi-bin/test-apache.cgi.
3. Click the reload or refresh button several times to verify the behavior of the scalable
application.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Chapter 12 - Page 17
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Chapter 12 - Page 18
Practices for Lesson 12: Configuring Oracle Solaris Zones in Oracle Solaris Cluster
Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practices for Lesson 13:
Exploring Oracle Solaris
Cluster Use Cases (Optional)
Chapter 13

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 1
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practices for Lesson 13: Overview (Optional)

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Practice Overview
In this practice, you perform the following tasks:
• Integrate Oracle Database 12c into Oracle Solaris Cluster software as a failover
application.
• Install and configure Oracle Solaris and Oracle Solaris Cluster software by using
Unified Archives (demonstration).

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 2
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

Practice 13-1: Integrating Oracle Database 12c into Oracle Solaris


Cluster Software as a Failover Application

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
Overview
In this practice, you integrate Oracle Database 12c into Oracle Solaris Cluster software as a
failover application.
In this practice, you complete the following tasks:
• Task 1: Create a logical IP entry in the /etc/inet/hosts file.
• Task 2: Create the oracle user and the required group accounts.
• Task 3: Set system parameters (on both nodes as the root role).
• Task 4: Create a shared storage file system (Solaris Volume Manager or ZFS pool).
• Task 5: Prepare the oracle user environment.
• Task 6: Copy the Oracle database files.
• Task 7: Install the required packages.
• Task 8: Disable Access Control of X Server on the Admin Workstation.
• Task 9: Run the runInstaller, netca, and dbca installation scripts.
• Task 10: Prepare an Oracle Instance for Cluster integration.
• Task 11: Register the resource types.
• Task 12: Create resources and resource groups for Oracle.
• Task 13: Verify that Oracle runs properly in the cluster.

Preparation
Oracle database installation is very time consuming and could take anywhere between 120 and
150 minutes. Therefore, to save time, you are given access to a new set of virtual machines
with preinstalled Oracle database using Oracle Solaris ZFS only (refer to Task 4 of this
practice).
Before proceeding with the tasks in this practice, perform the following steps:
1. Power off the following virtual machines:
• Administrative Workstation
• Sun_ZFS_Storage_7000
• Cluster_Node_1
• Cluster_Node_2
2. Reduce the memory of the Administrative Workstation VM.
a. Select the Administrative Workstation VM in VirtualBox Manager.
b. Click Settings->System.
c. Change the Base Memory to 1584 from 2560 MB.
d. Click OK.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 3
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

3. Boot the following virtual machines in the given order, one after the other. Wait for the
virtual machines to boot properly until you see the login prompt.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
• Administrative Workstation
• Sun_ZFS_Storage_7000_DB
• Cluster_Node_1_DB
• Cluster_Node_2_DB

Task 1 to Task 9 of this practice are already performed for you to save time and to speed
up the practice. However, these tasks are documented here for your reference and you
just need to verify them before proceeding to the next step.
Note: You must verify the following steps:
• Task 4, Step 2c
• Task 8
You start integrating the Oracle database instance with the cluster from Task 10
onwards.

Task 1: Create a Logical IP Entry in the /etc/inet/hosts File


Note: This task is already performed for you.
Assume the root role and create an entry for a logical IP address for Oracle in the
/etc/inet/hosts file on all the cluster nodes as follows:
# vi /etc/inet/hosts
...
192.168.1.126 ora-lh

Task 2: Create the oracle user and the required group accounts
To create the oracle user and the dba and oinstall group accounts, perform the following
steps from within the master console window on the administrative workstation:
Note: This task is already performed for you.
4. Create the dba and oinstall accounts on both the cluster nodes.
# groupadd dba
# groupadd oinstall
5. Create the oracle user on both the cluster nodes.
# useradd -d /export/home/oracle -m -s /bin/bash -g dba \
-G oinstall oracle
80 blocks
6. Create the oracle user password on both the cluster nodes.
# passwd oracle
New Password: oracle1
Re-enter new Password: oracle1
passwd: password successfully changed for oracle

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 4
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

7. Provide administrative access to the oracle user by assuming the root role on both the
cluster nodes.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# usermod -R root oracle
# usermod -P +"System Administrator" oracle
8. Verify the administrative access provided for the oracle user on both the cluster nodes.
# su - oracle
Oracle Corporation SunOS 5.11 11.3 September 2015
oracle@clnode1:~$ roles
root
oracle@clnode1:~$ exit

Task 3: Set the system parameters (on both nodes as root role)
Perform the following steps from within the master console window on the administrative
workstation as the root role:
Note: This task is already performed for you.
1. Configure system kernel parameters for the oracle user.
# projmod -U oracle -K "project.max-shm-memory=(priv,6g,deny)" default
# projmod -sK "project.max-sem-nsems=(priv,512,deny)" default
# projmod -sK "project.max-sem-ids=(priv,128,deny)" default
# projmod -sK "project.max-shm-ids=(priv,128,deny)" default
2. Set the system swap space.
Oracle database installation requires swap space of up to 4 to 5 GB.
Increase the swap space to meet the Oracle database installation requirement:
# swap -lh
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 274,2 4K 1.0G 1.0G
# swap -d /dev/zvol/dsk/rpool/swap
# zfs volsize=5G rpool/swap
# swap -a /dev/zvol/dsk/rpool/swap
# swap -lh
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 274,2 4K 5.0G 5.0G

Task 4: Create a Shared Storage File System


You can use either Solaris Volume Manager or ZFS pool to create a shared storage file system
for the Oracle database software.
Note: This task is already performed for you using Oracle Solaris ZFS.
Perform any one of the following two steps of this task:
1. Creating a shared storage file system for Oracle Database software (Solaris Volume
Manager)
Perform the following steps on any node physically connected to the storage as root role.
This practice assumes that local metadbs have been already created.
Just to speed up the practice, you will create unmirrored volumes. You can always add
another disk and mirror the parent volume later at your leisure, or pretend that you were
going to.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 5
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

a. Verify that the metadb replicas are configured and operational on both the cluster
nodes.

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
root@clnode1:~# metadb
flags first blk block count
a u 16 8192 /dev/dsk/c1t0d0s6
a u 8208 8192 /dev/dsk/c1t0d0s6
a u 16400 8192 /dev/dsk/c1t0d0s6
root@clnode1:~#
b. Select a disk from shared storage. Make sure that you do not use any disks already in
use in existing device groups.
root@clnode1:~# cldev list -v
. . .
d7 clnode2:/dev/rdsk/c0t600144F0AE8319BA0000508E382B0010d0
d7 clnode1:/dev/rdsk/c0t600144F0AE8319BA0000508E382B0010d0
. . .
root@clnode1:~#
Note: For the purpose of this practice, select the d7 shared disk for the orads disk
set.
c. Create the orads disk set by using the shared disk identified in the previous step.
root@clnode1:~# metaset -s orads -a -h clnode1 clnode2
root@clnode1:~# metaset -s orads -a /dev/did/rdsk/d7
d. Create a soft partition (d200) of a mirror volume for Oracle. (Solaris Volume Manager
still calls it a mirror, although we only have one submirror. This gives us flexibility to
add the other half of the mirror at our leisure later.)
root@clnode1:~# metainit -s orads d21 1 1 /dev/did/rdsk/d7s0
orads/d21: Concat/Stripe is setup
root@clnode1:~# metainit -s orads d20 -m d21
orads/d20: Mirror is setup
root@clnode1:~# metainit -s orads d200 -p d20 9g
d200: Soft Partition is setup
e. Create a cluster file system.
root@clnode1:~# newfs /dev/md/orads/rdsk/d200
newfs: construct a new file system /dev/md/orads/rdsk/d100: (y/n)? y

f. Create a mount point and an entry in /etc/vfstab on all the nodes.
root@clnode1:~# mkdir /oracledb
root@clnode1:~# vi /etc/vfstab
. . .
/dev/md/orads/dsk/d200 /dev/md/orads/rdsk/d200 /oracledb ufs \
2 yes global
root@clnode1:~#
g. On the node on which you created the storage, mount the file system and modify its
ownership.
root@clnode1:~# mount /oracledb
root@clnode1:~# chown oracle:dba /oracledb

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Practices for Lesson 13: Exploring Oracle Solaris Cluster Use Cases (Optional)
Chapter 13 - Page 6
THESE eKIT MATERIALS ARE FOR YOUR USE IN THIS CLASSROOM ONLY. COPYING eKIT MATERIALS FROM THIS COMPUTER IS STRICTLY PROHIBITED

h. Verify that the file system is mounted and available on both the cluster nodes.
# mount | grep /oracledb

Oracle University and XTENTICS CONSULTANCY SERVICES PRIVATE LIMITED use only
# ls /oracledb
lost+found
# df -kh | grep /oracledb
2. Create a shared storage file system for Oracle database files (ZFS).
Perform the following steps on any one node physically connected to the storage as root
role.
To simplify, we will create a ZFS storage pool with one disk. You can always add another