TABLE OF CONTENTS

2 Document control............................................................................................................................................3

2.1 Approval signatures................................................................................................................................3

2.2 Revision History......................................................................................................................................3

3 Goal..................................................................................................................................................................3

4 Validity..............................................................................................................................................................3

5 Results..............................................................................................................................................................4

6 Definitions used................................................................................................................................................4

7 Responsibility list..............................................................................................................................................4

8 Reference documents and procedures............................................................................................................4

9 Pre-requisites...................................................................................................................................................4

10 Preparations................................................................................................................................................6

11 DDBA INStallation procedure......................................................................................................................6

11.1 Installation of ddba software on linux....................................................................................................6

11.1.1 Preparation....................................................................................................................................6

11.1.2 Procedure.......................................................................................................................................6

lockbox configuration procedure.............................................................................................................................7

11.2 configuration of ddba software on linux................................................................................................8

12 configuration of ddba in sap and hana studio............................................................................................9

13 Create a backup intake form.....................................................................................................................11

14 Create BACKUP..........................................................................................................................................11

14.1 offline HANA database backup (optional)............................................................................................12

15 Backup Methods.......................................................................................................................................12

15.1 Method 1: Initiate backup via hana studio (optional)..........................................................................12

15.2 Method 2: Initiate backup via dbacockpit............................................................................................14

16 doS AND dontS........................................................................................................................................14

17 hana recovery scenarios...........................................................................................................................14

18 Troubleshooting........................................................................................................................................15

Page 1 of 15-2016)
 18.1 Configure DEBUG Logging....................................................................................................................15

1 DOCUMENT CONTROL

1.1 APPROVAL SIGNATURES

Authored by
Name, title, company Signature Date
Rohit S Nirkhe
SAP BASIS
Capgemini India Pvt Ltd.

Technical Review by
Name, title, company Signature Date
Senior SAP Technology Consultant
Capgemini Outsourcing B. V.

Quality Approval by
Name, title, company Signature Date
Validation Manager
Capgemini Outsourcing B. V.

1.2 REVISION HISTORY

Version Date Author(s) Revision Description
0.1 18-08-2016 Rohit S Nirkhe Initial
0.2 22-08-2016 Subramani, Updated with additional comments from Backup team (Karthik)
Karthikeyan
0.3 07-09-2016 Rohit S Nirkhe Updated with scheduling HANA backup Jobs via DBACOCKPIT
(SAP)
0.4 07-09-2016 Jeroen Additional comments and clarifications
Boekkamp
0.5 08-09-2016 Rohit S Nirkhe Updated with 2 options in the Step 12 - configuration of ddba in
sap and hana studio

2 GOAL

Purpose of this document is to give a high level explanation of the combination uDPS (Utility Dataprotection
Storage) and DDBA (DataDomain Boost for enterprise Applications) enabling the DBA to use native application
tooling for database backup and recovery (enabling the DBA to be in the driver seat for its own backups).

Secondly the document provides information and examples on how to configure HANA backups using EMC
DDBA using the backint interface (tape library).

This document is the initial document to configuring DDBA and other aspects related to this product.

3 VALIDITY

Page 2 of 15-2016)
 This working method is valid for all HANA systems running on Linux OS

4 RESULTS

Result is backup configuration of HANA database and logs to the backint interface (tape library).

5 DEFINITIONS USED

None

6 RESPONSIBILITY LIST

Since a successful implementation and configuration of DDBA depends on several teams a responsibility list has
been created.

Phase Activity System DBA Storage Comment

owner team
Prepare Request uDPS storage tenant unit R I Neo form?
Prepare Deliver uDPS tenant unit information and password I R
Prepare Secure password for future use R In case a new SAP HANA host is
deployed
Prepare Request system owner to install DDBA software I R
Install Configure network and name resolution SAP HANA host R Able to resolve and ping the uDPS
DataDomain
Install Install DDBA software R
Install Provide DBA group read/write access to R I DBA should be able to update the
/opt/ddbda/config/lockbox config file file to (re)configure the lockbox
Configure Configure lockbox R
Configure Configure SAP perquisites for backup R
Configure Configure backup jobs in SAP cockpit R
Configure Configure retention rules for backup using SAP cockpit R
and catalog
Execute Monitor for successful backup R
Execute Monitor expired backup job cleanup R
Execute Create and manage incidents for failed backups R SLA reporting needs to be defined
by the SAP team
Execute Perform restores when needed R

7 REFERENCE DOCUMENTS AND PROCEDURES

Boost for enterprise Applications documentation on IS-Portal

8 PRE-REQUISITES

To use DDBA uDPS capacity is needed which is delivered using OLAxxxx. Next to this software and licensing
needs to be in place for the DDBA software itself

The section below describes the information which is in the uDPS ola.

Utility Data Protection Storage (uDPS) provides RAW EMC DataDomain disk storage to hosts which are
connected to the Capgemini network environment and provides (native) backup tooling capabilities to perform
backup and restore over the local area network.

The following tooling is validated to be used in combination with uDPS:

Page 3 of 15-2016)
Application Protocol

EMC Networker Boost

EMC Database application agent for DD Boost for Enterprise Applications and Boost
ProtectPoint

EMC Microsoft application agent for DD Boost for Enterprise Applications Boost

Symantec Enterprise Vault (Primary Vault Store) CIFS

The new uDPS solution by GS will offer DELL EMC DataDomain RAW storage on per TB / per month basis with
the following points:

Backup application or tool validated by GS team

Consumption of storage is the responsibility of the tenant
Application compression should not be used for optimal (de-duplication) storage usage
Application encryption should not be used for optimal (de-duplication) storage usage
Daily capacity and storage performance report e-mail based on native DataDomain reports
Optional managed replication to second uDPS system (dual datacenter data protection)

The uDPS solution is delivered at datacenters with Capgemini LAN connectivity (currently only at DC3) and is
ideally used to protect data cross site (datacenter A -> datacenter B).

Based on reserved TB RAW capacity a maximum number of streams is assigned to the tenant in a 1:1 ratio.
RAW capacity used above the reserved volume will not provide additional streams. Charging is based on
reserved + additional used RAW capacity. Usage below the reserved capacity does not result in concession
because of stream reservations made for the tenant.

A uDPS storage system delivers 50 TB/hr throughputs and delivers 1,000 concurrent streams. This means the
minimum throughput per stream is calculated to 50 GB/hr. When reserving 40 TB of RAW storage capacities
the lowest expected performance would be 2 TB/hr as long as the combined systems sending the data and the
network infrastructure used can deliver at this rate.

The service includes the provisioning, modification and cancellation of storage, and solving storage incidents
which might occur. Furthermore the service includes 7x24 monitoring of the EMC DataDomain systems
delivering the uDPS solution in order to guarantee the availability, performance, scalability, reliability and
required security level.

This OLA does not cover consultancy, application support or optional backup & archiving software licensing.
A tenant agreement is based on a single customer or application usage because of secure isolation of users and
workloads

Guaranteed storage resources are only available when properly forecasted

Storage allocation 50% above the reserved RAW capacity requires adjustment of the reservation.
Minimal reservation is 20 TB RAW per tenant per datacenter
Reduction of the service may not exceed 10% of the reserved capacity per month per tenant.

Page 4 of 15-2016)
 Backup and/or consultancy services are optional.
The tenant must meet the acceptance criteria.

A full description of the uDPS and DDBA OLAs will be available in the product catalog

9 PREPARATIONS

Before DDBA software can be installed and configured some network preparations need to be done

1. It needs to be clear which VLAN will be used for the data path (backup subnet for regular filesystem
backups will be used) and connectivity needs to be checked

2. Backup host configuration

At the backup host side the DataDomain network information needs to be known and added to the local hosts
file (/etc/hosts) so it can be resolved.

For example:
172.16.77.125 ddve9001.bnl.cos.lan

3. The DataDomain system needs to be configured so it can resolve the client name (done by storage team
after requesting tenant resources).

cgkarthi@ddve9001# net hosts add 172.16.77.89 psthanbutest.os.capgemini.nl

Added "psthanbutest.os.capgemini.nl" -> "172.16.77.89" mapping to hosts list.

It can be verified using the below command:

cgkarthi@ddve9001# net hosts show

Hostname Mappings:
10.212.48.17 -> ddmc1001.bnl.cos.lan
172.16.77.89 -> psthanbutest.os.capgemini.nl
cgkarthi@ddve9001#

10 DDBA INSTALLATION PROCEDURE

10.1 INSTALLATION OF DDBA SOFTWARE ON LINUX

You can install the database application agent software on Linux by running the rpm command. In a supported
cluster, you must install the software on each node that will perform backups and recovery.

10.1.1 PREPARATION
Log in as root or equivalent user on the client system
Make sure the client can resolve the target DataDomain system using the FQDN
Make sure the client can ping the target DataDomain system to check connectivity
Copy the installation files from http://narnia.capgemini.nl/ under downloads -> DDBA to the host and
extract the tar.gz file
Follow the procedure below to install the software

10.1.2 PROCEDURE
1. Complete the preparation tasks in Road map to install or update the software on page
42. Ensure that you log in as the root user and you are in the correct directory, which
contains the downloaded software installation files.

Page 5 of 15-2016)
Note If you do not start the installation from the correct directory, the installation might fail.
2. On a RHEL 6 Linux platform, ensure that you have downloaded and installed the
compat-libstdc++-33 package.
3. Type the rpm command to install the software on Linux x64:
rpm -i emcdbappagent-3.0.0-1.x86_64.rpm
4. To verify that the installation was successful, type the rpm -aq command:
rpm -aq | grep -i emc
The command output must include the following: emcdbappagent-3.0.0-1

The other command to check the correct version of the software is

strings /opt/ddbda/bin/hdbbackint | grep "@(#)"
The command output must include the following:

The installation on Linux stores the different types of software files in the directories shown in the following
table.

Table 5 Software installation directories on Linux:

Types of installed files or directories Installation directory

Executable files /opt/ddbda/bin
Configuration file templates /opt/ddbda/config
Debug log files directory /opt/ddbda/logs (linked to /var/opt/ddbda/logs)
Library files /usr/lib/ddbda/lib64

On UNIX, the root user can assign the lockbox group ownership to a different group, such as a DBA group,
which enables the group users to perform specific lockbox operations

LOCKBOX CONFIGURATION PROCEDURE

1. Configuration of Lockbox for Backup and Restore

A lockbox is an encrypted file that DDBA uses to store and protect confidential information from unauthorized
access. The lockbox stores the Data Domain system information, including credentials for the DD Boost user.

The DBA group must have read and write permissions to the lockbox and at least the read permission to the
debug log files directory (this is arranged by the system owner).

Below is the step to update the lockbox

Logon as user root on the HANA DB server

Go to the directory /opt/ddbda/bin
Enter the below command:

<hostname>:/opt/ddbda/bin # ./ddbmadmin -L

Page 6 of 15-2016)
Provide full pathname for the lockbox, or press Enter to accept the default directory (/var/opt/ddbda/lockbox):
/opt/ddbda/config/lockbox

Provide a group ID for lockbox ownership, or type 0 to accept the 'root user' group as the lockbox owner: 1200
Lockbox has been successfully created in the directory '/opt/ddbda/config/lockbox' with group ownership 1200.

10.2 CONFIGURATION OF DDBA SOFTWARE ON LINUX

After the DDBA backup software is installed the below files must exists

/opt/ddbda/bin/hdbbackint

The following directories must exist:

/nsr/applogs
/usr/sap/<SID>/SYS/global/hdb/backint
/usr/sap/<SID>/SYS/global/hdb/opt

Create a softlink in the /usr/sap/<SID>/SYS/global/hdb/opt/hdbbackint pointing to /opt/ddbda/bin/hdbbackint

as user <sid>adm on the HANA server

cd /usr/sap/<SID>/SYS/global/hdb/opt
ln -s /opt/ddbda/bin/hdbbackint /usr/sap/<SID>/SYS/global/hdb/opt/hdbbackint

Go to the below directory as user root

cd /opt/ddbda/config/

Make a backup of the below configuration file

cp sap_hana_ddbda.utl sap_hana_ddbda.utl.ORIG

Via Unix vi editor open the file sap_hana_ddbda.utl and make the below changes:
DDBOOST_USER= DDBA_POC
DEVICE_PATH= /SU_DDBA_POC
DEVICE_HOST= ddve9001.bnl.cos.lan
CLIENT = psthanbutest.os.capgemini.nl
LOCKBOX_PATH = /opt/ddbda/config/lockbox

Adding Data Domain systems to the lockbox

To enable backups to and restores from a Data Domain system, you must run the
ddbmadmin -P -z configuration_file command to register the Data Domain system to the
host. This command creates the required lockbox (if it does not exist) or updates an
existing lockbox.

<hostname>:/opt/ddbda/bin # ./ddbmadmin -P -z /opt/ddbda/config/sap_hana_ddbda.utl

Performing the registration of the device host 'ddve9001.bnl.cos.lan' for DD Boost user 'DDBA_POC'.

Page 7 of 15-2016)
Enter password:
Confirm password:

Logging in to the device host 'ddve9001.bnl.cos.lan' with DD Boost credentials.

Logging in to the device host 'ddve9001.bnl.cos.lan' with DD Boost credentials was successful.

Lockbox directory is '/opt/ddbda/config/lockbox'.

Device host 'ddve9001.bnl.cos.lan' for DD Boost user 'DDBA_POC' has been registered in the lockbox.
psthanbutest:/opt/ddbda/bin #

11 CONFIGURATION OF DDBA IN SAP AND HANA STUDIO

Below is an example on how the backup was configured in HANA studio during the POC of DDBA.
Production backups should be scheduled and managed using DBA cockpit (see below option -2)

Option 1 : Configuration in HANA Studio.

Change the below parameters in the HANA Studio for the HANA backup

Backint Agent = /opt/ddbda/bin/hdbbackint

Data Backup
Backint Parameter File = /opt/ddbda/config/sap_hana_ddbda.utl

Log Backup
Backint Parameter File = /opt/ddbda/config/sap_hana_ddbda.utl

Select Use the same parameter file for data backup and log backup.

Log Backup Settings:

Page 8 of 15-2016)
Log Backup Settings
Destination Type = Backint
Destination = /usr/sap/<SID>/SYS/global/hdb/backint

Tick Enable Automatic Log Backup

Click Save

Option 2 : Configuration in DBA Cockpit

1. Logon to the SAP Application server as user DDIC in client 000 having SAP_ALL authorization.
2. Execute t-code DBACOCKPIT

3. Go to Configuration Backup Configuration

4. Enter the below details in the Backup Configuration

Data Backup = /opt/ddbda/config/sap_hana_ddbda.utl

Tick Use same parameter file for data backup and log backup

Log Backup = /opt/ddbda/config/sap_hana_ddbda.utl

Page 9 of 15-2016)
 Log Backup Settings
Destination Type = Backint
Backup Interval = 15 minutes

Click

to save the configuration.

Example:

Please Note: In the above screenshot the destination for Log Backup settings still shows wrong as local
File-system rather than the backint destination which is correctly shown in the HANA Studio

12 CREATE A BACKUP INTAKE FORM

https//:neo.capgemini.com
Create the backup intake form!!!!!!

13 CREATE BACKUP

Page 10 of 15-2016)
13.1 OFFLINE HANA DATABASE BACKUP (OPTIONAL)
Start an offline HANA database backup when SAP is completely down from HANA Studio.
Ask the backup team to start HANA database backup immediately.

14 BACKUP METHODS

14.1 METHOD 1: INITIATE BACKUP VIA HANA STUDIO (OPTIONAL)

To DDBA:
a) Open HANA Studio
b) Right-click the HANA database
c) Select Backup and Recovery -> Backup system

d) Backup Type Complete Data Backup

e) Select Destination Type Backint
f) Backup Destination - /usr/sap/<SID>/SYS/global/hdb/backint
g) Backup Prefix - <<Any Name to the backup for easy identification>>
h) Click Next
i) Review the backup settings

Page 11 of 15-2016)
j) Click Finish.

Page 12 of 15-2016)
 k) Click Close.

14.2 METHOD 2: INITIATE BACKUP VIA DBACOCKPIT

See the attached WI to schedule HANA Complete and Differential backups via SAP DBACOCKPIT for
production environments.

Workinstruction
HANA Backup Schedule from SAP DBACOCKPIT v0.1.doc

15 DOS AND DONTS

1. Application compression should not be used for optimal (de-duplication) storage usage.
2. Application encryption should not be used for optimal (de-duplication) storage usage.
3. Depending on the size of the database activation of concurrent stream can be increased and by
default it is 1 .It is recommended to use default stream values for smaller database and can only be
increased if in case we see long running jobs (online backups ) for larger Database.

16 HANA RECOVERY SCENARIOS

Please find the attached documents for the HANA recovery scenarios

1. Recover the database to as close as possible.

__Workinstruction to
perform HANA DATABASE RESTORE - Recover the database to as close as possible v0.1.doc

2. Recover the database to point-in-time using Differential Backup.

__Workinstruction to
perform HANA DATABASE RESTORE - Recover the database to point-in-time using Differential Backup v0.1.doc

3. Recover the database to point-in-time using Incremental Backup.

Page 13 of 15-2016)
 __Workinstruction to
perform HANA DATABASE RESTORE - Recover the database to point-in-time using Incremental Backups v0.1.doc

4. Recover the database to specific full database backup.

__Workinstruction to
perform HANA DATABASE RESTORE - Recover the database to specific full data backup v0.1.doc

17 TROUBLESHOOTING

17.1 CONFIGURE DEBUG LOGGING

By default there is no separate services that is responsible and running to handle the backup and every time a
backup is initiated from the HANA studio there would be a hdbbackint process that will give the status of the
backups on live .DEBUG_LEVEL on sap_hana_ddbda.utl can be set to 9 during the time of issues and review the
additional logging information which can be sent to SAP & EMC to sort out the issue.

Debug logs created for SAP HANA operations

l Operational log: hdbbackintSID.log

For example: hdbbackintD27.log

Also before setting the debug log level it is must to test the connection from backup system and to DD by the
following options:

1. Ping from backup client to DD :

psthanbutest:/opt/ddbda/logs # ping ddve9001.bnl.cos.lan

PING ddve9001.bnl.cos.lan (172.16.77.125) 56(84) bytes of data.
64 bytes from ddve9001.bnl.cos.lan (172.16.77.125): icmp_seq=1 ttl=64 time=0.228 ms
64 bytes from ddve9001.bnl.cos.lan (172.16.77.125): icmp_seq=2 ttl=64 time=0.171 ms
64 bytes from ddve9001.bnl.cos.lan (172.16.77.125): icmp_seq=3 ttl=64 time=0.217 ms
64 bytes from ddve9001.bnl.cos.lan (172.16.77.125): icmp_seq=4 ttl=64 time=0.233 ms
^C
--- ddve9001.bnl.cos.lan ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 0.171/0.212/0.233/0.026 ms

2. Ping from DD to hana system :

cgkarthi@ddve9001# net ping psthanbutest.os.capgemini.nl

PING psthanbutest.os.capgemini.nl (172.16.77.89) 56(84) bytes of data.
64 bytes from psthanbutest.os.capgemini.nl (172.16.77.89): icmp_seq=0 ttl=64 time=0.247 ms
64 bytes from psthanbutest.os.capgemini.nl (172.16.77.89): icmp_seq=1 ttl=64 time=0.203 ms
64 bytes from psthanbutest.os.capgemini.nl (172.16.77.89): icmp_seq=2 ttl=64 time=0.213 ms

Page 14 of 15-2016)
 64 bytes from psthanbutest.os.capgemini.nl (172.16.77.89): icmp_seq=3 ttl=64 time=0.200 ms
^C
--- psthanbutest.os.capgemini.nl ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.200/0.215/0.247/0.025 ms, pipe 2

3. rpcinfo test from hana system to make sure it is listening nfs and mountd along with portmapper

psthanbutest:/opt/ddbda/logs # rpcinfo -p ddve9001.bnl.cos.lan

program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
537220272 2 tcp 3006
100005 1 tcp 2052 mountd
100005 1 udp 2052 mountd
100005 2 tcp 2052 mountd
100005 2 udp 2052 mountd
100005 3 tcp 2052 mountd
100005 3 udp 2052 mountd
100003 3 tcp 2049 nfs
100003 3 udp 2049 nfs
537220001 2 tcp 2051
537220001 3 tcp 2051
285824256 1 udp 638
537329792 1 tcp 3007

4. To confirm that the client is able to talk to DD using the DDBOOST user credetnials

psthanbutest:/opt/ddbda/bin # ./ddbmadmin -P -z /opt/ddbda/config/sap_hana_ddbda.utl

Performing the registration of the device host 'ddve9001.bnl.cos.lan' for DD Boost user 'DDBA_POC'

Enter password:
Confirm password:

Logging in to the device host 'ddve9001.bnl.cos.lan' with DD Boost credentials.

Logging in to the device host 'ddve9001.bnl.cos.lan' with DD Boost credentials was successful.

Lockbox directory is '/opt/ddbda/config/lockbox'.

Device host 'ddve9001.bnl.cos.lan' for DD Boost user 'DDBA_POC' has been registered in the lockbox.

If the entire above works fine as expected then a SAP restart should be kept as last option before
raising a call.

Page 15 of 15-2016)