Contents Overview..................................................................................................................32 Design Principles....................................................................................................32 Backup .....................................................................................................................33 Offline Versus Online Backup...................................................................................33 Performance and Processing Considerations ..........................................................35 Scheduled or On-Demand ........................................................................................37 Restore and Recovery..........................................................................................310 Performance............................................................................................................310 Strategy...................................................................................................................310 Performance ..........................................................................................................311 Overview.................................................................................................................311 Backup ....................................................................................................................311 Recovery.................................................................................................................312 Backup Options.......................................................................................................312 Database Restore Options......................................................................................315 Recommendations................................................................................................315 Database.................................................................................................................315 Online and Offline Redo Logs.................................................................................315 Operating System Level Files.................................................................................316 Tape Management, Rotation, and Storage.........................................................316 Tracking and Documenting Tapes..........................................................................316 Storage....................................................................................................................319 Retention Requirements .........................................................................................320 Choosing and Implementing a Backup Strategy...............................................321 Backup and Archive Procedures and Policies ..................................................322 Backup ....................................................................................................................322 Policies....................................................................................................................323 Procedures..............................................................................................................326 Recovery.................................................................................................................327 SAPDBA Backup Tasks........................................................................................328 Setting the Parameter File ......................................................................................329 Start SAPDBA.........................................................................................................329 Initializing the Backup Tapes ..................................................................................331 Back Up...................................................................................................................338 Backing Up the Database .......................................................................................339 Back Up the Archive Logs.......................................................................................342 Useful Online Service System Notes ..................................................................344
Chapter 3: Backup and Recovery
Overview Release 4.0B 32 Overview This chapter discusses the backup, restore, and recovery functions of an R/3 System using an Oracle database. With the information presented in this chapter, you should be able to better understand the concepts that can enhance your operating environment and access which methods best suit your needs based on reliability, performance, and budget considerations. Design Principles Possibly the single most important aspect of the technical implementation is establishing an effective backup and recovery strategy. This entails a restore of all or part of the database for any reason due to either hardware or software errors, then a recovery in which the system is updated with data up to just before the failure. There are many situations other than disk failures that may require a restore and recovery. Your backup strategy should be designed based on business recovery needs. It is important to understand that a system failure will negatively impact productivity of your business and its associated costs. While your business defines Service Level Agreements (SLA), windows for backup time and recovery time should be established. You should set expectations early before going live and review them over time as the growth of your business, user base, or database size changes. Your backup strategy should be manageable and as uncomplicated as possible. Complications in backup design can create difficult situations during restoration and recovery. Procedures and problem identification and handling must be well documented so all individuals clearly understand their roles and required tasks. The backup strategy should not have an adverse impact on daily business. SAP provides tools such as CCMS-DBA within R/3, and SAPDBA at the operating system level. CCMS functions within the system are designed for processing backups in a scheduled method. CCMS provides historical information to review backup statistics and tape management information. The R/3 System integrates with Oracles capacity to provide both online and offline backups. In summary, an online backup is performed while the database is available; an offline backup is performed while the database is not available. Both types of backups can be performed at any of the following levels: < Database (a full backup in which all data files are saved) < Tablespace (a partial backup in which data files of one or more tablespaces are saved) < Data file (one or more data files are saved) Whenever downtime has to be minimized, we recommend that you perform an online backup (offline backups cause system downtime). In all other situations, perform a periodic, full offline backup. Your backup should be part of the 28-day cycle that SAP recommends in conjunction with other database administration tasks. Chapter 3: Backup and Recovery Backup R/3 System Administration Made Easy 33 To design your backup procedures, follow these steps: 1. Determine the recovery requirements based on an acceptable outage. It is difficult to define the concept of acceptable outage since acceptable means different things to different people. An outage has a cost due to the loss of productivity and cost (time, money, etc.) spent on recovery. This should be evaluated in much the same way as insurance. The more coverage you want, the more the insurance will cost. Therefore, the faster the recovery time requirements, the more expensive the solution. 2. Determine what hardware/software combinations can deliver the solution. Review the section on performance to help decide on the method of backup that is best suited to your particular situation. Follow the Keep It Simple (KISS) rule, but make sure your method is reliable. 3. Test your backup procedures by implementing the hardware and reviewing the actual run times and results of your tests. Make certain you get results from all types of backup that could be used in your environment, not just the ones you think might be used. This information will aid further evaluation and capacity planning decisions and provide useful comparison information as needed. 4. Test your recovery procedures by creating various situations of failure. Document all aspects of the recovery including the process of what has to be done, who should perform various tasks, who should be notified, etc. Keep in mind, a recovery will be needed when you least expect it, so be prepared. Testing is not a one-time event, but should occur on a regular basis, with additional tests should any of the hardware/software components be altered or changed. Backup Offline Versus Online Backup Overview Although we discuss the two primary types of backups, there are variations within each that may be selected based on your organizational needs. In general, you should achieve three main goals: < Provide a reliable backup that can be restored. < Keep the backup simple and reduce the number of dependencies required for operation. < Provide the above two items with little or no impact to business units. New features in SAP R/3 Release 4.0 provide increased safety (for example, the online consistent backup) and flexibility (for example, integration of the concept of splitting a mirror backup). Chapter 3: Backup and Recovery Backup Release 4.0B 34 Offline An offline backup is taken with the Oracle database and R/3 System down (that is, not running). Benefits: < An offline backup is faster than an online backup. < There is no issue with data changing in the database during the backup. < Related operating system level files will be in sync with the R/3 database, if they are backed up at the same time. Disadvantages: < R/3 is normally not available for use during an offline backup. If the split mirror technique is used, the database will be shutdown for a short time, while R/3 continues to run. < Buffers for R/3 and database are flushed. This will impact performance until the buffers are populated. Online An online backup is taken with both the Oracle database and R/3 running. Benefits: < R/3 is available to users during backup. Having a system that is available 24-hours a day may be critical in manufacturing or other areas where the system provides information to global users. < Buffers are not flushed. Since buffers are not flushed, there is no impact to performance once the backup is complete. Disadvantages: < An online backup is somewhat slower than an offline backup, resulting in a longer backup time. Backup time is increased because processes such as R/3 are running and competing for system resources. < Online performance is degraded while the backup is running. < Data may change in the database while it is being backed up. Because of this, the redo logs become critical to a successful recovery. < Related operating system level files may be out of sync with the R/3 database. You need the redo logs to successfully recover using online backups. Chapter 3: Backup and Recovery Backup R/3 System Administration Made Easy 35 Performance and Processing Considerations Many options can be used to process a backup. Some methods to reduce the backup demands, for situations where a daily backup may be difficult, are stated in the scenarios below. Business recovery time = time to find the problem + time to repair the damage + time to recover the database. Ad]usting Backup Frequency by Carrying Out Tests No rule of thumb exists to determine backup frequency. Your goal should be to provide backup time each day, or at the end of each processing day. Some customers perform a full backup once a week without considering how long a recovery might take. For example, recovery from a three-day-old full backup: Your testing shows that applying redo log files to a restored full backup takes three minutes per log. Assuming on average that 50 redo log files are archived a day Recovery would take 450 minutes (or 7.5 hours). 3 (days) x 50 (logs per day) x 3 (minutes per log). If the same full backup were performed each night: Recovery would require about 150 minutes (or 2.5 hours). 1 (day) x 50 (logs per day) x 3 (minutes per log). Back Up Using Parallel Options Offline and online backups can be done in parallel to increase throughput. In the case of a full database backup where multiple tape devices are available, data can be distributed between devices. Many installations utilize tape libraries. This allows for reduced tape handling and permits sharing by other systems. When reviewing alternatives, consider that units such as stack loaders provide a serial process (the a backup is done one tape at a time), while multiple single tape units provide a parallel process. If the online backup must go directly to tape, use multiple tape drives for parallel backups. Using Hardware Compression with Backups Using hardware compression with backups can reduce backup time by as much as 50%, or more. The actual compression depends on how much the data files can be compressed. 8kipping ndex Tablespaces to Reduce Backup Downtime To improve backup performance, consider skipping index tablespaces from your backups. An index tablespace contains only indices, which can always be rebuilt. If indices need to be recreated as part of a restorationor for any other reasonthe system would not be available to users during this rebuilding process. Indices are used in the system not only for performance, but also for data integrity. Chapter 3: Backup and Recovery Backup Release 4.0B 36 If index tablespaces are not backed up, the reorganization functions within SAPDBA will generate the scripts to create the tablespaces, and indices if they have to be rebuilt. This function can be stopped after the scripts are generated, before the actual reorganization takes place. The scripts should be recreated every time index structures or index tablespace structures have changed. Monitor the backup to make sure that no tables are accidentally residing in an index tablespace. If a table resides in an index tablespace, it should be reported by SAPDBA during the creation of the reorganization scripts. If a table has been misplaced, move it back to the appropriate data tablespace. The recommendation to omit index tablespaces from backups is valid only if no data tables reside in these tablespaces. The recovery time will be longer because of the time required to rebuild the indices. This must be balanced against the time saved taking the backups. Back Up to Disk First If sufficient disk space exists, consider first backing up to a disk. A disk backup is usually faster than a tape backup because disk devices are generally faster. You can then copy your backups from disk to tape without incurring R/3 downtime. If possible, retain the disk backup copy since a restore from disk is faster than from the tape. Note that this assumes the disks are not mirrored because with mirrored disks other options are available (see below). Backups with Mirrored Disks Backups with mirrored disks can be made offline. In this case, R/3 is online and available to users however the database is offline for a short time. This method is not recommended in a single mirror environment since your system would be unmirrored during the backup and mirror resync time. Many hardware vendors have worked with customers of large systems to implement this. SAP R/3 Release 4.0 now supports integration of these techniques into BRBACKUP. If your system is double-mirrored (three-way mirror), and you want to perform offline backups in a mirrored disk environment: 1. Shut down the database and split the disk mirror. 2. Restart the database on the remaining disks. 3. Mount the second disk set onto another system in which you will perform the offline backup. After finishing the backup, the two disk sets have to be resynchronized. The downtime incurred with this method is the time it takes to shut down the DBMS, split the mirror, and restart the DBMS. Chapter 3: Backup and Recovery Backup R/3 System Administration Made Easy 37 It may take a substantial amount of time to resynchronize the two disk sets, and this leads to severe performance degradation. Performance of Tape Devices If you are performing an offline backup, it is important for you to consider the type of tape devices you will use since this determines the downtime. Tape capacity currently ranges from two to 30+ GB and data transfer speed ranges from one to 10 GB per hour. See the performance table later in this chapter. 8cheduled or On-Demand After gaining familiarity with the types of backup methods available, you need to consider how they will be processed and monitored. The processing types are scheduled and on- demand. < Scheduled backups are processed via R/3s scheduler with CCMS or a scheduler of your choice. < On-demand backups are processed manually. The tool that performs a backup of the R/3 database is BRBACKUP. The tool that performs a backup of the Oracle Archive logs is BRARCHIVE. SAPDBA and BRBACKUP must be configured prior to use by modifying the appropriate configuration files. 8cheduled For normal operations you should configure a scheduled backup. Backups that are automated should use the DBA Planning Calendar (transaction DB13). This planning calendar provides the ability to set up and review backup cycles. It also has the ability to process essential database checks and update cost based optimizer statistics. You can set up CCMS to process the backup of archive logs via BRARCHIVE. Backups and other processes configured here can also be viewed in the Batch Processing Monitors (for example, transaction SM37). The status of the backups can be viewed using transaction DB12 (Backup Logs Overview). You may consider other scheduling packages or something as basic as an operating system scheduler such as CRON. Make certain your choice of scheduler can integrate well with R/3s CCMS for more complete integration. On-demand For an upgrade of R/3 or when other major changes are to be made, you would need an on- demand backup. Backups that are controlled directly by an operator, or on-demand, can be performed either by the DBA Planning Calendar (transaction DB13), SAPDBA, or via command line. Although the DBA Planning Calendar can schedule backups for periodic use starting on a given time, it can also be used to perform an immediate backup. In the case of on-demand needs it is more common to use SAPDBA which then calls either BRBACKUP or BRARCHIVE, depending on need. Chapter 3: Backup and Recovery Backup Release 4.0B 38 Backup Frequency Risks A consideration for recovery involves the frequency of backups. There are three backup frequency items to consider: < Full database < Log files < Operating system files Database The frequency of a full database backup determines how many days back in time you must go to begin the restore: < If a daily full backup is done, you will need yesterdays full backup. Only logs since yesterdays backup need to be applied to bring the system current. < If a weekly full backup is done, you will need last weeks full backup. Then, all the logs for everyday since the full backup must be applied to bring the system current. A daily full backup reduces the number of logs that need to be applied to bring the database current. This reduces the risk of not being able to get current because of a bad (unusable) log file. If a daily full backup is not done, the more logs there are to apply, thus lengthening the recovery process time and increasing the risk of not being able to recover to the current time. A point may be reached when it would take too long to restore the logs, because so many logs need to be applied. Example 1 Weekly Backup Restore from last weeks full backup that was done four days ago. There are 40 logs a day. A total of 160 logs (40 logs per day 4 days) need to be restored. If it takes 2 minutes to restore a log (from tape to disk), the total time to do the restore is 320 minutes (5.3 hrs). Recovery time at 3 minutes per log (160 3) is 480 minutes (8 hours). Total log restore and recovery is 13.3 hours (database files excluded) Example 2 Daily Backup Restore from last nights full backup There are 40 logs a day. Since there is ample disk space (40 logs per day 2 days) there is no need to restore from tape. Since logs were on disk, recovery started at 3 minutes per log (40 3) is 120 minutes (2 hours). Total log recovery time is 2 hours (database files excluded) Chapter 3: Backup and Recovery Backup R/3 System Administration Made Easy 39 These examples show that the length of time it takes to do a log restore depends on how far back (days) you have to go back to get to the last full backup. Increasing the frequency of the full backup (with less days between full backups) reduces the recovery time. In addition, you should consider maintaining two backup cycles of archive logs on disk to reduce the need for a restore of these logs from tape. Archive Logs The Oracle redo logs are the mechanism used to recover the database. The online redo logs, which then become offline or archive logs, are Oracles transaction logs of database updates. Oracle has several online redo logs that are used in sequence and then reused in sequence once archived. The closed online redo logs are copied to the offline or archive area known as SAPARCH. In /oracle/<SID>/saparch. The offline (archive) redo logs must then be backed up. The offline or archive logs are created when the database is in ARCHIVELOG mode. If the system is not in this mode online redo logs can be overwritten as needed. In normal operation, the system should be in ARCHIVELOG mode which assures this will not happen. You must save the archive logs to tape or other media via BRARCHIVE. Once this is done, the archive can be safely removed from the system. The preferred method is to back up to tape twice and only delete archives as space requirements demand. If the archived redo logs are not backed up and cleared, the archive directory can fill up and causes an archiver stuck situation. This halts activity in the database, and in turn halts processing in the R/3 System. NEVER delete archived redo logs without first making certain you have a good backup of them. If your transaction volume is high, it would be prudent to decrease the time interval between log backups. This decreases the amount of data that could be lost in a potential data center disaster. < The frequency of the log backups is a business decision based on: Transaction volume Critical period(s) for the system Amount of data senior management is willing to lose Resources to perform the backups and take them offsite Operating 8ystem Level Files Operating system level files must also be backed up. These files are for: < Operating environment (for example, system and network configuration) < R/3 files Chapter 3: Backup and Recovery Restore and Recovery Release 4.0B 310 Spool files, if stored at the operating system level (system profile: rspo/store_location = G) Change management transport files located in /usr/sap/trans < Other R/3 related applications Interface or add-on products, such as those used for EDI or taxes that store their data or configuration outside the R/3 database. The amount of data described is small in relation to the R/3 database. In general, if you are using large capacity tapes for the database and smaller tapes for archives, the smaller tapes can also be used for these operating system files. Depending on how your system is used, the above list should only require several hundred megabytes to a few gigabytes of storage. In addition, some of the data could be static (that is, it would not change). The frequency of the operating system level backup depends on the specific application. If these files must be kept in sync with the R/3 System, then they must be backed up at the same frequency as the log backup files. An example of this is a tax program that stores its sales tax data in files external to the R/3 database. These files must be in sync with the sales orders within the system. A simple and fast method to back up operating system files is to copy all data file directories to disk on another server. Then on the other server, back up those files to tape. This process minimizes file downtime. Restore and Recovery There are normally only three reasons to do a restore: < A disaster has occurred A hardware failure has occurred A software problem requires resetting the database to a previous state (that is, restore the database as of a certain date/time) < To test your disaster recovery plan < To copy your database to another system For additional details on the first two items, see chapter 2, Disaster Recovery. For details on the last item, see chapter 11, Nonscheduled Special MaintenanceProduction Refresh Strategies. Performance The business requirement for speed in a restore is primarily driven by the need to get the system operational as soon as possible after a disaster, so the company can continue to do business. 8trategy Factors that affect the chosen strategy include: < Business cost of downtime to recover < Operational schedule Chapter 3: Backup and Recovery Performance R/3 System Administration Made Easy 311 < Global or local users < Number of transactions an hour < Budget Note: The actual process to restore R/3 and database will not be covered in this book. This is a task so critical and has specific system dependencies that it is left to a specialist to teach. If the restore is not done properly, the recovery could fail and the restore must be restarted. If a restore must be done, contact a specialist. You should work with your DBA or consultant to test and document the restore process for your system. With proper training, you should be able to do the restore. There may be special data that you must record about your database in order to recover it. Work with your specialist to identify and document this special data. Performance Overview Increasing performance is an iterative investigative process. 1. Identify the bottleneck or device that is limiting the throughput. 2. Eliminate the bottleneck. 3. Repeat steps 1 and 2 until the performance is adequate or the additional cost is no longer justified. This entire process is subject to cost considerations. Additional performance can always be purchased, which is almost always a business cost justification exercise. Backup There are three major variables which affect performance: < Database size The larger the database, the longer it will take to back up. Chapter 3: Backup and Recovery Performance Release 4.0B 312 < Backup window size For an online backup, the backup window is defined where there are few, or no, users on the system, to minimize the impact the backup has on the users. This backup typically happens in the early morning. An offline backup is defined as when and for how long R/3 can be brought down to complete this backup. This backup typically happens during the weekend. < Hardware throughput This variable limits how fast the backup can run and is defined by the slowest link in the backup chain such as: Database drive array I/O channel that is used Tape drive Backup performance can be improved by increasing the tape device blocking factor. CPIO, a UNIX tool, is delivered with all UNIX Systems. We recommend increasing the blocking factor to increase the throughput. The blocking factor can be changed by modifying the CPIO parameters in init<SAPSID>.sap. Third party backup products use high blocking factors as well as vendor-specific tape writing methods to achieve higher performance. Recovery The performance requirement for a recovery is more critical than for backup. Recovery performance determines how quickly the system will be available for use and how soon business can continue. The goal is to be able to restore the database and related files to make the system available for general use as quickly as possible. The longer this takes, the greater the impact to your business. Backup Options Our backup options assume that the backup device is local to the database server, not on another computer on the network. A backup performed over a network will be effected by the topology, overhead, and traffic on the network. Rarely is the full capacity of the network available. If a backup is done over the network, it will decrease performance for anyone else working on that network. Although technically possible, performing a backup over a network is beyond the scope of this guidebook. Back Up to Faster Devices All of these backup options attempt to eliminate the bottleneck at the backup device. The backup device, usually a tape drive, is the throughput-limiting device. Chapter 3: Backup and Recovery Performance R/3 System Administration Made Easy 313 Here are some capacity and throughput values to help you plan tape drive selection: Type Capacity (GB) (native/compressed) Rate (GB/hr) (native/compressed) DAT (DDS-2) 4 / 6.8 1.8 / 3.1 DAT (DDS-3) 12 / 20.4 3.6 / 6.1 DLT 4000 20 / 34 5.4 / 9.2 DLT 7000 35 / 60 18 / 30.6 Compressed capacity values here assume the use of hardware compression and use a more conservative 1.7x compression ratio; not the typical 2x compression ratio. The actual compression ratio and rate depends on the nature of the file and how much it can be compressed. Example, the backup of the R/3 Oracle database files has shown a hardware compression ratio of 3x-4x. However as the fill level of the database increases, the amount of compression will decrease. As technology advances, and the capacity and throughput of tape drives increases, the values stated here will become dated. We advise you to investigate what is currently available at the time of your purchase. Benefits: Faster and larger capacity tape drives allow you to back up an entire database on a single tape cartridge within a reasonable time period (for example, a two-hour backup of a 60GB database to a DLT7000). Disadvantages: A backup to a single tape drive is slowest (if compared to the other two options). Unless an automated changer or library is used, you are limited to the maximum capacity of the tape cartridge, without manually changing the cartridge. Not all databases and backup tools support tape changers or libraries; make certain they are compatible before purchasing them. For example, SAPDBA supports tape changers, but NTBackup does not. Parallel Backup There are two options for backing up to multiple tape drives: < A RAID-0 (stripe) array, in which several tape drives are written to in parallel < Individual tablespaces or files (Oracle), which are simultaneously backed up to separate tape drives Because you are writing to multiple tape drives in parallel, the total performance is significantly faster than if you were using a single tape drive. Chapter 3: Backup and Recovery Performance Release 4.0B 314 Unlike RAID or tape methods, BRBACKUP (used by SAPDBA) will not stripe data files across multiple tapes. BRBACKUP distributes data at the data file level. If a tablespace consists of several data files, these data files can be distributed on different tapes. With enough tape drives in parallel you can shift the bottleneck from the tape drives to another component. You must consider the performance of each subsystem when using tape drives in parallel. This subsystem includes the tape drive(s), controller(s), CPU, and I/O buss. In many configurations a controller or buss is the limiting factor. To restore a parallel backup, all the tapes in the set must be readable. If one tape is bad, the entire backup set will not be usable. The more tapes you have in a set, the greater the chance that one tape will be bad. Backing Up to Disks, then Tapes 1. From R/3, back up to disk first. 2. Then from these disks, back up to tape. Benefits: < Backup to disk is the fastest option (for the database). Under most situations, you can back up to disk faster than to tape. < This option allows you to make several identical backup copies (for example, one for onsite storage and one for offsite storage). < R/3 System performance is minimally affected, once the backup has been made to disk. Because the tape backup is made from the disk copy, not the live database, the backup to tape is not competing with database activity for significant system resources. < In an onsite disaster recovery to the same equipment, the recovery can be done from the on-disc backup. Disadvantages: < Significant additional disk space, up to the same amount of space as the database, is required. This makes the option the most expensive, especially for a large database. < Until the backup to tape is made, you are vulnerable to a data center disaster. < In a major disaster recovery, you have to first restore the files to disk, then execute the database recovery from the files on disk. This process increases the time to recover the system. There are other options available for a faster backup, such as the various High Availability options, but these options are beyond the scope of this guidebook. Chapter 3: Backup and Recovery Recommendations R/3 System Administration Made Easy 315 Database Restore Options If you want to increase database restore performance, all of the database backup options above are valid. In addition, you may restore to a faster disk array, a disk array with a higher data-write throughput. There are different ways to restore to a faster disk array: < Dedicated drives Restoring tablespaces to individually dedicated disc drives makes the process faster. Because only one tablespace or file is written to the drive, at any one time, you do not have head contention writing another tablespace to the same drive. < RAID type Mirrored stripe (RAID 0+1) is faster than RAID5, but this depends on the specific hardware. In some cases, the task of computing the parity data for the parity drive (RAID5) takes more time than it would to write all the data twice (RAID 0+1). This is an expensive option because the usable capacity is 50 percent of the total raw capacitysignificantly less than RAID5. RAID 0+1 = [single_drive_capacity (number_of_drives/2)] RAID5 = [single_drive_capacity (number of drives 1)] < Drives with faster write performance < Drive array system with faster write performance Recommendations Database We recommend a full database backup be taken every day, assuming the size of your database and backup window permits this. For databases that are too large for daily full database backup, a full backup should be taken weekly. Online and Offline Redo Logs Managing the offline redo logs is critical. If the filespace is used up, the Oracle database will stop, and that stops R/3. See the section Archive Logs on page 39. 1. A suggestion is to back up the offline redo logs every three hours during the business day, from 6:00 AM to 9:00 PM. A company with high transaction volume carries higher risk and would increase the frequency accordingly, perhaps to every hour. Similarly, if you have a shipping department that opens at 3:00 AM and a Finance department that closes at 10:00 PM, you would need to adjust the start and end times. Chapter 3: Backup and Recovery Tape Management, Rotation, and Storage Release 4.0B 316 2. Immediately copy the redo log backups to an offsite backup file server. This backup file server should ideally be in another building or in another city. A separate location increases the chance that the log files will be preserved if the primary data center (containing the R/3 servers) is destroyed. 3. Back up the redo logs backups on both servers (the R/3 server and the offsite backup file server) to tape each day with the other operating-system-level files. If you do not have an offsite backup server, back up the redo log backups to tape after each log backup and immediately send the tape(s) offsite, or at least outside of the data center. Do not back up the logs to the tape drive in append mode and append multiple backups on the same tape. If a data center disaster occurs, the tape with all these logs will be lost. Operating 8ystem Level Files The frequency of the operating system level backup depends on the specific application. If these files must be kept in sync with R/3, then they must be backed up with the same frequency, and at the same time as the database and log backups. An option for a non-sync-critical situation is to back up these operating system level files once a day. Tape Management, Rotation, and 8torage Tracking and Documenting Tapes Tapes must be tracked and documented because if they need to be retrieved from storage, you will need to know which tapes to retrieve and where they are stored. Labeling Tapes should be clearly labeled using one of many labeling methods. For a manual system, two simple methods are described in the examples that follow: Example 1: This method uses the six-character naming convention that is used by SAPDBA and BRBACKUP. It can also be used for other programs having a short (six-character) limit on the tape name. It uses the following logic: < Each label has the following data: System ID <SID> What is backed up (database, operating system, log, etc) Sequence number Chapter 3: Backup and Recovery Tape Management, Rotation, and Storage R/3 System Administration Made Easy 317 Sample Label: PRDB01 PRD + B + 01 = PRD Production, B Brbackup (Database), 01 - tape number 1. Example 2: < Each label has the following data: Database ID What is backed up (database, operating system, log, etc.) Day of the Month Multiple tape indicator for a single day Sample Label: PRD-db-11a The PRD database on the 11 th of the month, tape a #1. In addition to the naming schemes use a different color label for each system: < PRD = yellow < QAS = green < DEV = white Third party backup management software may assign their own tracking number to the labels. Tracking Tapes should be logged to track where they are stored, so you can locate them when you need them. Tapes should be tracked and documented when they are: < Used < Sent to offsite storage < Returned from offsite storage < Any other time when the location of the tape changes To help you track and retrieve the offsite backup, log the following information: < Date of backup < Database < Tape number Chapter 3: Backup and Recovery Tape Management, Rotation, and Storage Release 4.0B 318 < Tape storage companys number Some storage companies label the cartridges with their own tracking label, so that they can track them internally to their system and facility. < OS level backup tape number < Date sent offsite < Date returned For example: Date Volume Label Purpose Notes Out Back 7/15/98 PRDB01 Data Base 7/15/98 7/30/98 7/15/98 PRDO23 Operating Sys 7/15/98 8/15/98 Handling Any time you transport tape cartridges, carry them in a protected box to minimize damage and potential data loss if they are accidentally dropped. The box should have foam cutouts for each tape cartridge you use. For a small company, an ideal tape collection device is a small or medium plastic (plastic is nonmagnetic) tool box with a foam insert that has cutouts for each tape cartridge. A recommended procedure is to use two boxes: < One to collect the tapes to be sent offsite < One to contain the new backup tapes This second box should be empty when you finish changing tapes. When changing tapes, handle only one tape cartridge at a time, to avoid confusion. Remove the tape cartridge from the tape drive, then insert it in the collection box. Then remove the next tape. After all tapes have been removed, insert the new tapes in the drive in the same manner, one at a time. If you are using preinitialized tapes, as with SAPDBA, you must insert the correct tape for that day, or the backup program will reject the tape. The backup program reads the tape header for the initialization info, which includes the tape label name, and compares it to the next label that is in sequence to be used. Keep track of which tape cartridges have been used and are to be sent offsite, and which are to be loaded in the drives. It is all too easy to accidentally put the wrong tape cartridge in a drive, and thus, destroy the backup just done, or cause the next backup to fail. Chapter 3: Backup and Recovery Tape Management, Rotation, and Storage R/3 System Administration Made Easy 319 When you initialize a tape, BRBACKUP and BRARCHIVE write an expiration day onto the tape. The tape cannot be overwritten by SAPDBA, BRBACKUP, and BRARCHIVE before that number of days has elapsed since it was written to. It might however be overwritten by another backup application that ignores the tape header. Other backup software that interface with BRBACKUP and BRARCHIVE initialize tapes with their own header. 8torage Offsite What Offsite storage is a storage location in a separate facility (building or campus) other than the R/3 data center. Why The purpose of offsite storage is to safeguard company data in the event that your facility is destroyed. Where The magnitude of the disaster will effect what is considered adequate protection. Sending tapes to a separate location will be sufficient if the disaster is confined to the building where the data center is located. If the disaster is local or regional (for example, a flood or earthquake) adequate protection means sending tapes to a distant location several hundred miles away. Offsite data storage can be at a separate company facility or a commercial data storage company. The offsite data storage vendor should have a certified data storage site. Data tapes have different handling and storage requirements than paper. Backup tapes should be sent offsite as soon as possible to prevent potential loss in the event of disaster! Onsite What Onsite storage is the storage of your data in the same facility as your data center. Chapter 3: Backup and Recovery Tape Management, Rotation, and Storage Release 4.0B 320 How Tape cartridges should be properly stored, following the tape manufacturers storage requirements. The most difficult requirement to comply with is magnetic fields. The problem is determining if there is a strong magnetic field near the tape storage location (for example, a vacuum cleaner motor could generate enough of a magnetic field to damage tapes, as could a large electric motor located on the opposite side of the wall where the data tapes are stored). When storing tape cartridges, keep all related tape cartridges together. All the tapes used in a daily backup should be considered as a set, comprising backups for: < Database < Logs < Operating system files Tapes and files taken in a set need to be restored as a set. For example, if operating system files are not restored with database and log files, the operating system files will not be in sync with the database and critical information will be missing. Retention Requirements There are legal requirements that determine data retention. Check with your companys legal department on complying with federal, state, and local data retention requirements. How to comply with these requirements should be discussed with your legal and finance departments, external auditors, and consultants. The practical side of data retention is that you may be unable to realistically restore an old backup. < If the operating system, database, and the R/3 System have each been upgraded twice since the backup, it is unlikely that the backup can be restored without excessive costif at all. Retention is related to your backup cycle. It is important to have several generations of full backups and all their logs because: < If the database is corrupted, you will have to return to the last full backup before the database corruption. < If the last full backup is corrupted, you will have to return to the prior full backup and roll forward using the backup of all the logs since the corrupted backup. Depending on the level of corruption, you may have to go back further still. < Since R/3 is an online real-time system, to recover the database from a full database backup, you must apply all the logs since that backup. If this is a significant amount of time, the number of logs could be tremendous. Therefore, the number of logs you may need to apply creates a practical limit to how far back you can recover. Recommendation < If a full database backup is taken daily, we recommend that you keep at least two weeks of backups, and all the redo logs for these weeks. Chapter 3: Backup and Recovery Choosing and Implementing a Backup Strategy R/3 System Administration Made Easy 321 < If a full database backup is taken weekly, you should be able to go back at least three generations. The traditional three generations of backup are: grandfather father son < Store selected backup sets (month-end, quarter-end, year-end, etc) for extended periods, as defined by your legal department and auditors. Choosing and mplementing a Backup 8trategy It is very important to set up a proper procedure to back up the valuable information of your system. Procedures should be defined as early as possible to prevent possible data loss. The following list presents a list of backup issues that should be resolved before going live: < Decide how often to perform complete database backups < Decide whether partial backups are necessary < Decide when to perform redo log archives < Set as a goal the ability to hold redo logs on the archive directory returning to latest complete backup < Provide ample disk space for the archive directory < Consider using DBA Planning Calendar (DB13) to schedule redo log archives < Create a volume labeling scheme to ensure smooth operations < Decide on backup retention period < Determine tape pool size (that is, tapes needed per day retention + 20%) Make certain to allow for growth and special needs < Initialize tapes < Determine physical tape storage strategy < Decide whether to use unattended operations < If unattended operations, decide whether in CCMS or elsewhere < Document backup procedures in operations manual < Train operators in backup procedures < Implement a backup strategy < Perform a test restore and recovery Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies Release 4.0B 322 Backup and Archive Procedures and Policies Backup policies and procedures should be defined as early as possible in preparation for potential data loss. The following is an example of backup and archive procedures and policies for a three-system landscape where DEV is a development system, QAS is a consolidation/quality assurance system, and PRD is a production systems. Backup 8ystem Environment 8oftware Components The following tools perform the backup/restore tasks: System Name Backup Software DEV CCMS (BRBACKUP + ) QAS CCMS (BRBACKUP + ) PRD CCMS (BRBACKUP + ) Hardware Components The hardware listed in the table below is for backup and restore of the database and archive logs: System Name Backup Hardware DEV 1 x DLT 7000 35/70 GB, 1 DDS-3 12/24 QAS 1 x DLT 7000 35/70 GB, 1 DDS-3 12/24 PRD 2 x DLT 7000 35/70 GB, 2 DDS-3 12/24 Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies R/3 System Administration Made Easy 323 Policies Tape Retention Period Even if one tape (backup/archive) gets damaged or lost, the tape retention period assures the ability to recover the database. System Name Regular Backup Month-End Backup Quarter- End Backup Year-End Backup Archives DEV 14 days 7 days QAS 14 days 28 days PRD 28 days 24 months 2 years 4 years 28 days Check with your legal and finance departments and others as applicable to determine what the retention period should be for your company. Backup Frequency {DB + Archives} Use a schedule similar to the ones below to ensure that you will be able to quickly and easily restore the database. Database Schedule System Name Mon Tue Wed Thu Fri Sat Sun DEV Full Online Full Online Full Online Full Online Full Offline QAS Full Online Full Online Full Online Full Online Full Offline PRD Full Online Full Online Full Online Full Online Full Online Full Online Full Offline Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies Release 4.0B 324 Archives Backing up the archives (former online redo logs) is necessary to perform either a recovery of the database from an online backup or to perform point-in-time recovery. The CDS backup option of the BRARCHIVE program ensures that two copies (from each archive) will be stored on two different tapes before the archive is automatically deleted. System Name Mon Tue Wed Thu Fri Sat Sun DEV CDS CDS CDS CDS CDS QAS CDS CDS CDS CDS CDS PRD CDS CDS CDS CDS CDS CDS S: Save SD: Save and Delete DS: Delete Saved CDS: Copy, Delete, Save 8upplementary Backups Supplementary backups are made at special dates (month-end, year-end) so that, if needed, you can restore the database to a previous state. System Name Month-End Backup Year-End Backup DEV None None QAS None None PRD Full offline with verification Full offline with verification 8torage Location For safety reasons, the backup media must be stored in a safe place. The production system copies of the tapes should be stored in a remote (external) location. System Name Offsite Additional Location DEV Yes No QAS Yes No PRD Yes Yes Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies R/3 System Administration Made Easy 325 Tape Labeling Choose self-explanatory names to indicate the backup type and source. For further differentiation, use sequential numbers. System Name Data Base Archives Operating System Special R/3 DEV DEVB<NN> DEVA<NN> DEVO<NN> DEVS<NN> QAS QASB<NN> QASA<NN> QASO<NN> QASS<NN> PRD PRDB<NN> PRDA<NN> PRDO<NN> PRDS<NN> <NN>: Sequential Number Verifying Backups To guarantee the integrity of the backups, perform checks on the tapes based on the following schedule. System Name Frequency of Backup Verification DEV Every 2 weeks QAS Every 2 weeks PRD Every 2 weeks Checking Database ntegrity To avoid backing up a hidden, inconsistent database, the database must be checked at least once during a retention period. System Frequency of DB-checks DEV Every 2 weeks QAS Every 2 weeks PRD Every week Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies Release 4.0B 326 Procedures Backup The unattended backup is performed based on the backup frequency table. The scheduling functionality of the R/3 CCMS is used to schedule the backup. For systems running plain SAPDBA, the required tapes can be listed with the VOLUMES Needed button on the backup scheduling screen within CCMS. Extra backups, such as the monthly and yearly backup, have to be performed offline and will be done either with SAPDBA interactive or prepared BRBACKUP script files with a special tape pool. Archiving If archiving is performed during normal operation of the system, there is no performance impact. With plain SAPDBA, the necessary volumes can be found using the query only option in the backup archive menu of SAPDBA. For the extra backups, no special archiving is required. Because, since the backup is performed offline, the database remains in a consistent state. Verifying of Backups Backups must be verified following the schedule. The verify and list tape contents option of SAPDBA will be used to perform this task. On systems running with other backup tools, those tools are used to verify the tapes. Monitoring/Controlling After backing up the database and finishing the archives, all logs must be printed and placed in the folder for each system. Database Check An integrity check of the database must be performed within one retention period, in order to ensure that no corrupted blocks exist in the database, something that would go unrecognized during backup. The SAPDBA DB Verification can perform this check. Roles and Responsibilities Task Role Backup Database Operator Backup Archives Operator Verifying Backups Operator/DBA Monitoring/Controlling Operator/DBA Database check DBA Chapter 3: Backup and Recovery Backup and Archive Procedures and Policies R/3 System Administration Made Easy 327 Recovery 8ystem Environment 8oftware Components For a database recovery, use the same software tools that you used for the backup. System Name Backup Software DEV SAPDBA 4.0B + Vendor Add-on QAS SAPDBA 4.0B + Vendor Add-on PRD SAPDBA 4.0B + Vendor Add-on Hardware Components System Name Backup Hardware DEV 1 x DLT 7000 35/70 GB, 1 DDS-3 QAS 1 x DLT 7000 35/70 GB, 1 DDS-3 PRD 2 x DLT 7000 35/70 GB, 2 DDS-3 Policies Testing Recovery The restore procedure is one of the key issues of the R/3 System. Therefore, the procedures to recover a database must be regularly maintained and tested. System Name Regularly Tests Supplemental Tests DEV Every 2 months When any of the involved components changes (for example, SAPDBA) QAS Every 2 months When any of the involved components changes (for example, SAPDBA) PRD Every 3 months When any of the involved components changes (for example, SAPDBA) Procedures Recovery procedures are based on SAPDBA or on SAPDBA plus vendor software. Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 328 Roles and Responsibilities Task Role Developing Restore Procedures DBA Testing Restore Procedures DBA Training Operators DBA Perform Restores Operator/DBA 8APDBA Backup Tasks SAPDBA is a tool provided by SAP to perform Oracle database administration tasks. Tasks include: < Setting the parameter file < Initializing the backup tapes for Database Archive logs < Performing backups for Database Archive logs Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 329 8etting the Parameter File Guided Tour 1. Open the file init<SID>.sap with a UNIX text editor. NT: using sappad, <drive>:\orant\database. UNIX: using vi, /oracle/<SID>/dbs 2. Edit the parameters as appropriate. Some of the parameters that you should check and set as needed are: < Compress = hardware < Tape_size Value should be set to match your tape drive < Volume_backup for database tape labels names < Volume_archive for archive log tape labels names Note: This file should have been configured as part of the installation of your system. 8tart 8APDBA 1. At the Command prompt, enter SAPDBA, press Enter. Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 330 This is the main SAPDBA screen. All other SAPDBA tasks begin from this screen. Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 331 nitializing the Backup Tapes nitializing the Database Backup {BRBACKUP} Tape This can be done in one of two ways: < Using SAPDBA < Using BRBACKUP Using 8APDBA to nitialize a Database Backup Tape Guided Tour 1. At the Command prompt, enter SAPDBA, press Enter. This is the main SAPDBA screen. 2. Enter h (Backup database) at the Please select prompt. 3. Press Enter. 4. Enter a (Backup function) at the Please select prompt. 5. Press Enter. 2 4 Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 332 6. Enter b (Initialize BRBACKUP tape) at the Please select prompt. 7. Press Enter. 8. Enter q (Return) at the Please select prompt. 9. Press Enter. 10. Note the description on the line Backup function showing Initialize BRBACKUP tape. 11. If you only have one tape to initialize, go to step 16. If you have more than one tape to initialize, enter d (Number of tapes) at the Please select prompt. 12. Press Enter. 13. Enter the number of tapes to initialize. 14. Press Enter. 15. The number of tapes to initialize should appear on the line d Number of tapes. 16. Enter s (Start BRBACKUP) at the Please select prompt. 17. Press Enter. 6 10 11 13 15 16 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 333 This screen shows: A. Volume name that BRBACKUP will use to initialize the tape (from the init<SID>.sap file) B. Completed initialization (indicated by the message BRBACKUP terminated successfully) 18. Press Enter. B A Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 334 Using BRBACKUP to nitialize a Database Backup Tape Guided Tour 1. To force initialization of a tape, at the Command prompt, enter brbackup i force n 1 v <volname> < n (Number of tapes parameter) is required to initialize a pool of tapes. < v <volname> (Volume name parameter) is optional. Use this option only if you need to initialize a tape with a specific volume name (for example, when replacing a damaged tape). If v <volname> is omitted, the command will use the name table in the init<SID>.sap file. 2. When the program prompts Your reply, enter cont to continue. The entry cont is case-sensitive. 3. Press Enter. 4. When initialization has finished successfully, the message BRBACKUP terminated successfully displays. 5. Remove the tape from the drive and label it matching the specified name. 2 1 5 2 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 335 nitializing the Archive Tape This can be done in one of two ways: < Using SAPDBA < Using BRARCHIVE Using 8APDBA to nitialize an Archive Tape Guided Tour 1. At the Command prompt, enter SAPDBA, press Enter. 2. From the main SAPDBA screen, enter i (Backup offline redo logs) at the Please select prompt. 3. Press Enter. 4. Enter a (Archive function) at the prompt. 5. Press Enter. 1 4 Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 336 6. Enter k (Initialize BRARCHIVE tape) at the Please select prompt. 7. Press Enter. 8. Enter q at the prompt to exit (not shown). 9. Press Enter. 9. Note the message to the right of Archive function, showing Initialize BRARCHIVE tape. The number of tape to be initialized can be changed in the same way as BRBACKUP. Enter d (Number of tapes) Press Enter. Enter the number of tapes to initialize. Press Enter. 10. Enter s at the prompt to start BRARCHIVE (not shown) at the Please select prompt. 11. Press Enter. 12. When the initialization finishes, the message BRARCHIVE executed successfully displays. 13. Remove the tape and label it to match the label name. 14. Press Enter. 5 10 9 13 12 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 337 Using BRARCHVE to nitialize an Archive Tape 1. At the Command prompt enter brarchive i force n 1 v <volname> Initializing parameters are the same as for BRBACKUP. 2. When the program prompts Your reply, enter cont to continue. 3. Press Enter. 4. When initialization has finished successfully, the message BRBACKUP terminated successfully displays. 5. Remove the tape from the drive and label it matching the initialized label. 1 2 4 5 Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 338 Back Up To back up both the database and archive logs: 1. Determine the tapes required to do the backup. 2. Gather the required tapes. 3. Load the tape drive with the tapes. 4. Execute the appropriate backup process. Determining the Tapes Required to Back Up The following process applies to both database and archive log backups using SAPDBA, BRBACKUP, or BRARCHIVE. Guided Tour 1. At the command prompt: < For database enter brbackup q < For archive logs enter brarchive q 2. Press Enter. 3. The volume label(s) that will be used is displayed (in this example, SAOB04 and SAOB01). 1 3 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 339 Backing Up the Database Prerequisites < For an offline backup: Stop R/3 < Have the required, initialized and labeled, tape(s). As specified by brbackup -q We will use SAPDBA to perform the backup. SAPDBA executes BRBACKUP to perform the backup. BRBACKUP can be executed directly, which allows it to be executed by a chron job and scheduled. Guided Tour 1. At the Command prompt, enter SAPDBA, press Enter. This is the main SAPDBA screen. 2. Enter h (Backup database) at the Please select prompt. 3. Press Enter. 4. Verify that Normal backup appears on Backup function. 2 4 Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 340 5. Review line e (Backup type) to determine what type of backup is configured, online or offline. 6. If the type of backup needs to be changed, enter e (Backup type) at the Please select prompt. 7. Press Enter. From this screen you have two options: < a (online backup) < b (offline backup) 8. Enter your option choice at the Please select prompt (for example, a). 9. Press Enter. 10. Enter q (Return) at the Please select prompt to exit (not shown). 11. Press Enter. 12. Enter S (Start BRBACKUP) at the Please select prompt. 13. Press Enter. 6 8 12 5 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 341 14. The program prompts you to replace the tape when it needs changing. Replace the tape with the new tape and volume name specified. 15. Enter cont, when the program prompts you to enter cont to continue. 16. Press Enter. 17. When the backup has finished successfully, the message BRBACKUP terminated successfully appears. 18. Remove the tape and store properly. 15 17 14 Chapter 3: Backup and Recovery SAPDBA Backup Tasks Release 4.0B 342 Back Up the Archive Logs Prerequisites < Initialize and label sufficient tapes < Have the required, initialized and labeled, tape(s). As specified by brarchive -q We will use SAPDBA to perform the backup. SAPDBA executes BRARCHIVE to perform the backup. BRARCHIVE can be executed directly, which allows it to be executed by a chron job and scheduled. Guided Tour 1. At the Command prompt, enter SAPDBA, press Enter. This is the main SAPDBA screen. 2. Enter i (Backup offline redo logs) at the Please select prompt. 3. Press Enter. 4. To enter some of the parameters of archiving, enter a (Archive function) at the prompt. 5. Press Enter. 2 4 Chapter 3: Backup and Recovery SAPDBA Backup Tasks R/3 System Administration Made Easy 343 6. Enter the letter for the type of archive log backup you want to do. 7. Press Enter. We recommend that you make two (2) copies of the Oracle Archive Logs. 8. Enter s (Start BRARCHIVE) at the prompt. 9. Press Enter. 10. When the archive logs have been backed up successfully, the message BRARCHIVE executed successfully appears. 11. Press Enter. 6 8 10 Chapter 3: Backup and Recovery Useful Online Service System Notes Release 4.0B 344 Useful Online 8ervice 8ystem Notes Online Service System Note # Description 68059 SAPDBA - option -next with tablespace list 43499 All collective notes concerning DBA Tools 43491 Collective note: SAPDBA - Command line options 43486 Collective note: General SAPDBA 43484 Collective note: General DBA 42293 SAPDBA - new command line option analyze 34432 ORA-00020: max number of processes exceeded 33307 Cost-Based - Rule Based Optimizer (ORACLE) 31073 SAPDBA - new command lines -next, -analyze 21568 SAPDBA: Warning: only one member of online redo 16513 File system is fullwhat do I do? 15465 SAPDBA - shrinking a tablespace 04754 Buffer synchronization in centralized systems 03807 Tablespace PSAPROLL, rollback segments too small 02425 Function of tablespaces/DBspaces on the database 01042 ORACLE TWO_TASK connect failed 01039 Problems with ORACLE TWO_TASK linking
Kali Linux - An Ethical Hacker's Cookbook - Second Edition: Practical recipes that combine strategies, attacks, and tools for advanced penetration testing, 2nd Edition
Mastering Linux Security and Hardening - Second Edition: Protect your Linux systems from intruders, malware attacks, and other cyber threats, 2nd Edition
React.js for A Beginners Guide : From Basics to Advanced - A Comprehensive Guide to Effortless Web Development for Beginners, Intermediates, and Experts