TAF & FCF

Copyright © Oracle Corporation, 2010. All rights reserved.

 Failover Types
1. Server Side TAF  RAC
2. Client Side TAF
Load Balance=Yes
Failover = On
Type = Select
Type = Preconnect
3.Server Side Listener Connection Load Balancing.
Load Based
Session Based
4.Fast Connection Failover (FCF)

Copyright © Oracle Corporation, 2010. All rights reserved.

 Server Side Load Balancing
For high availability and scalability, Oracle provides the
Transparent Application Failover feature part of RAC.
LBA (Load balance advisory)
1. Short Goal its used for short connections
2. Long Goal  Inverse of short (pmon will have
information about load, & it updates scan listeners)
This is typical for connection pools and SQL*Forms sessions
It can be done by
EXECUTE DBMS_SERVICE.MODIFY_SERVICE
(service_name => ‘RAC_TEST’, clb_goal =>
DBMS_SERVICE.CLB_GOAL_LONG);
Or by srvctl command

Copyright © Oracle Corporation, 2010. All rights reserved.

 Client Side TAF
The client load balancing feature enables clients to
randomize connection requests among the listeners.
This is done by client Tnsnames Parameter:
LOAD_BALANCE.

The (load_balance=ON) instructs SQLNet to progress

through the list of listener addresses in the address_list
section of the net service name in a random sequence.

Copyright © Oracle Corporation, 2010. All rights reserved.

 Client Side Connect-Time failover

This is done by client Tnsnames Parameter: FAILOVER

The (failover=on) enables clients to connect to another
listener if the initial connection to the first listener fails.
Without connect-time failover, Oracle Net attempts a
connection with only one listener

Copyright © Oracle Corporation, 2010. All rights reserved.

 Under which condition TAF works

Client Action Failure

Establish Connection NodeA down

" ListenerA down

" InstanceA down

" InstanceA "in an indeterminate state"

No transaction But Select, 2nd Select NodeA dies between the 2 selects

" ListenerA dies "

" InstanceA dies "

" Session dies "

Copyright © Oracle Corporation, 2010. All rights reserved.

 DML Operations wont be failover
Till 11g
PL/SQL package state is lost after failover
ALTER SESSION statements are lost
Continuing work on existing cursors may raise an error (eg: ORA-25401 "cannot
continue fetches")
Failed over selects may take time to re-position (when FAILOVER_TYPE=SELECT)
Failed over selects may raise an error.

In 12.2 new feature, Even update statement can be failover.

Transaction Guard ( for DB side) -->dbms.app.get_commit_outcome
 Application Continuity ( for application side)
*** Condition apply(its available for only for JDBC .NET,OCI)

Need to add one more service

srvctl add service -commit_outcome true -failovertype transacion

Copyright © Oracle Corporation, 2010. All rights reserved.

 Failover Mode Parameters

METHOD: This parameters determines how failover occurs from

the primary node to the backup node

BASIC: Use this mode to establish connections at failover time, no

work on the backup server until failover time.

Copyright © Oracle Corporation, 2010. All rights reserved.

SELECT
When the type=select specified, it will failover to the available instance in the cluster
database and the failover process also replays the queries that is in the progress.

PRECONNECT: Use this mode to pre-established connections. This PRECONNECT

mode provides faster failover but requires that the backup instance be capable of
supporting all connections from every supported instance.

RETRIES: Use this parameter to specify number of times to attempt to connect after a
failover. If DELAY is specified but RETRIES is not specified, RETRIES default to five
retry attempts.

DELAY: Use this parameter to Specify the amount of time in seconds to wait between
connect attempts. If RETRIES is specified but DELAY is not specified, DELAY default
to one second

Copyright © Oracle Corporation, 2010. All rights reserved.

 Tnsentries for Client failover with Method -
Basic
CLIENT_FAILOVER=
(DESCRIPTION =
(ADDRESS_LIST =
(FAILOVER = on )
(LOAD_BALANCE = on)
(ADDRESS = (PROTOCOL = TCP)(HOST = saraswati-vip.localdomain)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = ganga-vip.localdomain)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = Clienttaf)
(FAILOVER_MODE = (TYPE = SELECT)(METHOD = BASIC)(RETRIES = 10)(DELAY = 5))
)
)

Copyright © Oracle Corporation, 2010. All rights reserved.

 Tnsentries for Client failover with
Method=Preconnect
CLIENT_FAILOVER=
(DESCRIPTION =
(ADDRESS_LIST =
(FAILOVER = on )
(LOAD_BALANCE = on)
(ADDRESS = (PROTOCOL = TCP)(HOST = saraswati-vip.localdomain)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = ganga-vip.localdomain)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = Clienttaf)
(FAILOVER_MODE = (TYPE = SELECT)(METHOD = PRECONNECT)(RETRIES = 10)(DELAY = 5))
)
)

Copyright © Oracle Corporation, 2010. All rights reserved.

 Tnsentries for Client failover with SCAN &
Method=Preconnect
CLIENT_FAILOVER_PRE_SCAN =
(DESCRIPTION =
(ADDRESS_LIST =
(FAILOVER = on )
(LOAD_BALANCE = ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = scan.localdomain)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = Clienttafpre)
(FAILOVER_MODE = (Backup= mysrv_preconnect) (TYPE = SELECT)(METHOD = PRECONNECT)(RETRIES = 10)(DELAY = 5))
)
)
mysrv_preconnect=
(description =
(ADDRESS_LIST =
(LOAD_BALANCE = ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = saraswati-vip.localdomain)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = ganga-vip.localdomain)(PORT = 1521))
)
(connect_data=(service_name=Clienttaf))
)

Copyright © Oracle Corporation, 2010. All rights reserved.

SQL> select INST_ID,SID,USERNAME from gv$session where USERNAME='SYSTEM';

Copyright © Oracle Corporation, 2010. All rights reserved.

 Server Side Listener Connection Load
Balancing
With server-side load balancing, the listener directs a connection request to the best
instance.
Init parameter remote_listener should be set.
When set, each instance registers with the TNS listeners running on all nodes within
the cluster.

There are two types of server-side load balancing:

Load Based — Server side load balancing redirects connections by default depending
on node load. This is default.

Session Based — Session based load balancing takes into account the number of
sessions connected to each node and then distributes the connections to balance the
number of sessions across the different nodes.

Session count balancing is used when you set a listener parameter,

PREFER_LEAST_LOADED_NODE_<listener_name>Please be aware that connections
bursts (large number of connections initiated very quickly one after another) may not
be properly balanced due to statistics update latency.
PMON registers itself with listener.
PMON computes metrics every 2-3 seconds
PMON checks the listener status every 1 second

Copyright © Oracle Corporation, 2010. All rights reserved.

 Server Side TAF

For high availability and scalability, Oracle provides the Transparent

Application Failover feature part of RAC.

Transparent Application Failover (TAF) is a client-side feature that

allows for clients to reconnect to surviving nodes in the event of a
failure of an instance.

Any uncommitted transactions are rolled back and server side

program variables and session properties will be lost. In some case
the select statements automatically re-executed on the new
connection with the cursor positioned on the row on which it was
positioned prior to the failover.

Copyright © Oracle Corporation, 2010. All rights reserved.

 TAF
SYNTAX
srvctl add service -d <db_unique_name> -s <service_name>
-r "<preferred_list>" [-a "<available_list>"] [-P {BASIC | NONE | PRECONNECT}]
-g <server_pool> [-c {UNIFORM | SINGLETON}]
[-k <net_num>]
[-l [PRIMARY][,PHYSICAL_STANDBY][,LOGICAL_STANDBY][,SNAPSHOT_STANDBY]]
[-y {AUTOMATIC | MANUAL}]
[-q {TRUE|FALSE}]
[-x {TRUE|FALSE}]
[-j {SHORT|LONG}]
[-B {NONE|SERVICE_TIME|THROUGHPUT}]
[-e {NONE|SESSION|SELECT}]
[-m {NONE|BASIC}]
[-z <failover_retries>]
[-w <failover_delay>]

Copyright © Oracle Corporation, 2010. All rights reserved.

-r "<preferred_list>" [-a "<available_list>"] [-P {BASIC | NONE | PRECONNECT}]
This clause is valid only for Administrator-managed database which support
PRECONNECT method.
-g <server_pool> [-c {UNIFORM | SINGLETON}]
This clause is valid only for Policy-managed database where in PRECONNECT
method is not available.

The above two options instruct clusterware on how to handle this service.

[-e {NONE|SESSION|SELECT}]
This defines the type of TAF whether SESSION or SELECT.
[-m {NONE|BASIC}]
This defines the method of TAF.
[-z <failover_retries>]
This defines the the number of times to attempt to connect after a failover.
[-w <failover_delay>]
This defines the amount of time in seconds to wait between connect attempts.

Copyright © Oracle Corporation, 2010. All rights reserved.

 Service Creation

Grid User
$srvctl add service -d windy -s Clienttafservpre -r windy1 -a windy2 -P preconnect

Start the Service.

$ srvctl start service -d windy -s Clienttafservpre

Preconnect creates an additional shadow service.

$ crsctl status resource –t

...
...
ora.Clienttafservpre.svc
1 ONLINE ONLINE windy1
ora.Clienttafservpre.svc
1 ONLINE ONLINE windy2

Copyright © Oracle Corporation, 2010. All rights reserved.

Check if the service is correctly registered with the listener.

$ lsnrctl status listener_scan1

...
...
Service "Clienttafservpre" has 1 instance(s).
Instance “windy1", status READY, has 1 handler(s) for this service...
Service "Clienttafservpre" has 1 instance(s).
Instance “windy2", status READY, has 1 handler(s) for this service...
...
...

Copyright © Oracle Corporation, 2010. All rights reserved.

 Fast Connection Failover

The Fast Connection Failover (FCF) feature is a Fast Application Notification (FAN)
client implemented through the connection pool.

The feature requires the use of an Oracle JDBC driver and an Oracle RAC database
FCF will support from 10.1.X & later versions

FCF provides is very fast notification of the failure and the ability to reconnect
immediately using the same URL. When a RAC node fails the application will receive
an exception. The application has to handle the exception and reconnect.

Reason its depends on ONS reason its doesn't required negative acknowledge.

Copyright © Oracle Corporation, 2010. All rights reserved.

 Fast Connection Failover Configuration
Example
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();

pds.setConnectionPoolName("FCFSamplePool");
pds.setFastConnectionFailoverEnabled(true);
pds.setONSConfiguration("nodes=racnode1:4200,racnode2:4200\nwalletfile= /oracle11/onswalletfile");
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");pds.setURL("jdbc:oracle:thin@(DESCRIPTION= "+
"(LOAD_BALANCE=on)"+
"(ADDRESS=(PROTOCOL=TCP)(HOST=racnode1) (PORT=1521))"+
"(ADDRESS=(PROTOCOL=TCP)(HOST=racnode2) (PORT=1521))"+
"(CONNECT_DATA=(SERVICE_NAME=service_name)))");
...

Copyright © Oracle Corporation, 2010. All rights reserved.

 Enabling Fast Connection Failover

The FCF pool property is used to enable and

disable FCF. FCF is disabled by default.

pds.setFastConnectionFailoverEnabled(true);

Copyright © Oracle Corporation, 2010. All rights reserved.

 FCF manages pooled connections for high availability and
provides the following features:

FCF supports unplanned outages. Dead connections are rapidly detected and then
the connections are aborted and removed from the pool. Connection removal relies
on abort to rapidly sever socket connections in order to prevent hangs. Borrowed
and in-use connections are interrupted only for unplanned outages.

FCF supports planned outages. Borrowed or in-use connections are not interrupted
and closed until work is done and control of the connection is returned to the pool.

FCF recognizes new nodes that join an Oracle RAC cluster and places new
connections on that node appropriately in order to deliver maximum quality of
service to applications at run-time. This facilitates middle-tier integration of Oracle
RAC node joins and work-request routing from the application tier.

FCF distributes runtime work requests to all active Oracle RAC instances.

Copyright © Oracle Corporation, 2010. All rights reserved.

 What exactly is the use of FCF?

FCF provides is very fast notification of the failure and the ability to reconnect immediately
using the same URL. When a RAC node fails the application will receive an exception. The
application has to handle the exception and reconnect.
The JDBC driver does not re-target existing connections. If a node fails the application must
close the existing connection and get a new one. The way the application knows that the
node failed is by getting an exception. There is more than one ORA error that can be thrown
when a node fails,the application must be able to deal with them all.
An application may call isFatalConnectionError() API on the
OracleConnectionCacheManager to determine if the SQLException caught is fatal.

If the return value of this API is true, we need to retry the getConnection on the
DataSource.xxxxx

Copyright © Oracle Corporation, 2010. All rights reserved.

 Difference between TAF and FCF
TAF is a database session-level connection failover
mechanism: it works via the OCI interface only so
applies to applications using thick JDBC drivers and
those using the OCI libraries (e.g. written in C, like
OID). During failover a new session will be started on
an alternative node (though it can be pre-connected
ahead of the failure) and this can optionally re-open
a cursor and advance it to the point it was at prior to
failure (providing underlying data hasn't changed).
Copyright © Oracle Corporation, 2010. All rights reserved.
FCF is an application-level failover mechanism: the
database-tier notifies the application-tier by means of
a FAN (fast application notification) message,
distributed via the ONS (Oracle Notification Service)
daemons. Where database connections are pooled,
as offered by Oracle JDBC, the driver then has the
opportunity to clean up stale connections if a node
fails. One very nice feature is that when a node
comes back online (i.e. a FAN message is sent out)
the JDBC driver will automatically make new
connections to the node to re-balance the pool.
 Reference:
How to configure Connection Manager (CMAN) with Transparent
Application Failover (TAF)? (Doc ID 397021.1)
SYSDBA Sessions Do Not Failover with SRVCTL TAF Configured (Doc ID
1342992.1)
11.2 : ORA-12518 Listener Hangs and Reports "Too Many Open Files" (Doc
ID 1527483.1)
How to Configure Client Side Transparent Application Failover with
Preconnect Option (Doc ID 802434.1)
11gR2(11.2) RAC TAF Configuration for Admin and Policy Managed
Databases (Doc ID 1312749.1)
Failover Issues and Limitations [Connect-time failover and TAF] (Doc ID
97926.1)
Understanding Transparent Application Failover (TAF) and Fast Connection
Failover (FCF) (Doc ID 334471.1)
How to Verify Universal Connection Pool (UCP) / Fast Connection Failover
(FCF) Setup (Doc ID 1064652.1)
Fast Connection Failover (FCF) Test Client Using 11g JDBC Driver and 11g
RAC Cluster (Doc ID 566573.1)

Copyright © Oracle Corporation, 2010. All rights reserved.

Copyright © Oracle Corporation, 2010. All rights reserved.
 References
Database Link Fails with ORA-2019 (Doc ID 115857.1)

ORA-2019 Error Encountered When Accessing a

Private dblink in a View After Set current_schema (Doc
ID 428600.1)

Database Links: Troubleshooting ORA-2085: database

link %s connects to %s (Doc ID 210630.1)

Getting ORA-2019 After Upgrading Database to

11.2.0.4 (Doc ID 1967705.1)

Copyright © Oracle Corporation, 2010. All rights reserved.