Summary

What is high availability and cluster? VCS Communication Mechanism VCS Architecture and Agent Monitoring Fail Over Configurations Installation of Veritas Cluster Server in Unix platforms

High Availability & VCS Features

Combination of 2 or more servers with appropriate middleware and interconnects to achieve some or all of the following: High levels of system reliability (availability)
Monitor, restart and failover

Simplified administration of IT assets (manageability)

Java, web interface, Manage multiple clusters

Increased processing power (scalability)

Distributed services, Add systems, storage to running clusters

Sample Cluster

High Availability Design

HA-aware applications
Restart capability Crash-tolerance

HA Management software
Site replication Fault detection, notification, and failover Storage management Backup and recovery

Redundant Hardware
Power supplies Network interface cards, hubs, switches storage

VCS Terms & Concepts

Cluster A VCS cluster consists of multiple systems connected in various combinations to shared storage devices VCS engines (had) runs on each system in the cluster that controls applications running in the cluster Systems Member of a cluster, also referred as node Has its own copy of VCS software Share a common view of cluster configuration which is dynamically communicated whenever changes are made within the cluster Connected to public network to provide application services to users Service groups Related collection of resources All resources within the service group must be available and this cant be spitted across systems Resources have interdependencies which must be considered during configuration and management Service groups also have interdependencies Service group types Failover online on one server at a time. Most applications are configured this way Parallel Online on multiple services at a time. Application must be specifically designed to run in parallel. E.g., FTP, web servers, Oracle parallel servers

VCS Terms & Concepts contd..

Resources
VCS objects that correspond to hardware of software components disks, interface cards, IPs, etc Monitored and controlled by VCS online, offline, monitor Classified by type Identified by unique names and attributes Can depend on other resources within the same service group ( correct order)

Resource Types
General description of the attributes of a resource such as disks, IP, applications, databases Predefined resource types: Mount, Share, IP, NIC Resource Types have inherent interdependencies IP resource type depends on the NIC resource type New resource types can be created as required

Agents
Processes that control resources One agent per resource type Agent controls all resources of that type New agents can be added apart from predefined agents

Dependencies
The order in which VCS brings the resources and service groups online or offline Parent resource depends on child resource for example Mount depends on Disk Dependencies are homogenous, for example Service group cant depend on a resource

VCS Communication Mechanisms

Private Network
Minimum of two dedicated communication channels between each system in the cluster Heartbeat communication Determines cluster membership Configuration distribution Configuration & status information for every resource and service groups are dynamically distributed dynamically to all systems

LLT (Low Latency Transport)

Proprietary networking protocol used by VCS for heartbeat communication and Traffic distribution Very fast, kernel-to-kernel communication, not routable Uses DLPI over Ethernet

GAB (Group Membership Services/Atomic Broadcast)

Manages cluster membership and maintain cluster state Runs over LLT Uses broadcasts to distribute cluster configuration information

VCS Engine (had)

Maintains configuration and state information for all cluster resources Uses GAB to communicate among systems Monitored by hashadow process VCS Architecture overview

VCS components

VCS: A Replicated State Machine

Monitoring and Communication

VCS HAD
process agent web process SGweb file system agent web fs web ip nic agent ip agent

web dg disk group agent

nic

VCS Architecture Overview

Each cluster system has VCS agents that monitor the status of resources Agents communicate the status of resources to the VCS engine The VCS engine communicates the status of all local resources to other systems using GAB GAB uses LLT to communicate between systems over private network These mechanisms work together to ensure all members of the cluster maintain identical copy in local memory

Fail Over Configurations

Asymmetric
Before After

192.1.1.1

192.1.1.1

Asymmetric Failover

Fail Over Configurations

Symmetric
Before After

192.1.1.1

192.1.1.2

192.1.1.2

192.1.1.1

Symmetric Failover

Fail Over Configurations

N to 1

N to 1 failover

Application Service Management

SG
Web

SG Web SG Oracle

NBU

SG

SG Oracle

Multi Node / Multi Group Clusters Unlimited numbers of resource (service) groups Migrate RG without affecting other RGs Cascading failover Policy based failover - load distribution Attribute based RGs Easy start, stop and migration of resource groups - GUI

VCS Installation on Solaris

Prerequisites:
Sun Solaris with latest patches, VxVM, VCS and VxFS Veritas license keys for all Veritas products (http://vlicense.veritas.com) A minimum of two physically independent private networks Crossover cable or hub based on number of nodes Atleast 3 ethernet cards Shared storage with separate controller

Environment variables
PATH=$PATH:/sbin:/usr/sbin:/opt/VRTSvcs/bin MANPATH=$MANPATH:/opt/VRTS/man export $PATH, $MANPATH

VCS 5.0 Installation

Disk space required:

Operating System Support:

Cluster Manager (Java Console) AIX 5.2 and 5.3 HPUX 11.23 IA and PA - RISC RHEL 4 Update 3 and SLES9 SP3 Solaris SPARC 2.8, 2.9, and 2.10 Windows 2000, Windows XP & Windows 2003

The installvcs utility

VCS packages: VRTSgab, VRTSllt, VRTSperl, VRTSvcs, VRTSvcsag, VRTSvcsmg, VRTSvcsmn, VRTSvlic, VRTSvcsdc, VRTSweb, VRTSvcsw It creates VCS communication files:
/etc/llttab /etc/gabtab /etc/llthosts

The following configuration files are added:

/etc/VRTSvcs/conf/config/main.cf /etc/VRTSvcs/conf/config/types.cf

Installvcs options
-installonly -> Only licensing & installation of VCS software -configonly -> creates configuration files without performing installation -configfile -> unattended installation using VCSvars.pm file

Installation parameters
Cluster Name Cluster Number : 0 to 255, Avoid 0 as its default System Names (host name) Network interfaces identified for VCS private network Web console configuration Virtual IP, Subnet mask and Network interface. ClusterService servicegroup is created SMTP/SNMP notification configuration

Managing Cluster Services

Cluster Configuration
File: /etc/VRTSvcs/conf/config/main.cf on each system in the cluster The cluster configuration in memory is created from main.cf had daemon maintains configuration in memory and state information of all cluster resources

Starting VCS
hastart command starts the had and hashadow processes had checks for active configuration in private network otherwise it loads the configuration from disk. Node-1 starts had & hashadow processes on second node and process is repeated until all members have identical copies.

Managing Cluster Services

Modifying Cluster Configuration
haconf makerw -> Cluster configuration opened and .stale file is created in config directory haconf dump makero Changes written to disk and .stale file removed otherwise it keeps the changes only in memory In GUI, it prompts to open the configuration when attempting to modify the configuration

Starting VCS with a stale configuration

If .stale file found during startup, it requests the configuration from another server via private network .stale file is removed after saving valid configuration to main.cf

Managing Cluster Services

Modifying Cluster Configuration
haconf makerw -> Cluster configuration opened and .stale file is created in config directory haconf dump makero Changes written to disk and .stale file removed otherwise it keeps the changes only in memory In GUI, it prompts to open the configuration when attempting to modify the configuration

Starting VCS with a stale configuration

If .stale file found during startup, it requests the configuration from another server via private network .stale file is removed after saving valid configuration to main.cf

References : Symantec Tech notes & VCS Admin Guide 5.0 http://www.symantec.com/connect/forums/top-technotes-veritascluster-server-updated-march-2012 http://www.symantec.com/business/support/index?page=content &id=DOC5492 https://sort.symantec.com/public/documents/sf/5.0/solaris/pdf/vcs _users.pdf

Thanks & Regards Kiran Jangam Fujitsu India Pvt Ltd