Repunix26 Install Reference r16

IBM Data Mobility Services Softek Replicator for UNIX
Version 2.6
Installation and Reference Guide for AIX, HP-UX, Linux, and Solaris
REP-U26IR-016
IBM Data Mobility Services Softek Replicator for UNIX

Version 2.6
Installation and Reference Guide for AIX, HP-UX, Linux, and Solaris
REP-U26IR-016
REVISION NOTICE
This is the sixteenth release of this manual for Softek Replicator for UNIX, version 2.6. This manual provides technical and editorial updates and replaces all previous editions. Please refer to the Summary of Changes for a complete revision history.
ABSTRACT
The Softek Replicator for UNIX 2.6 Installation and Reference Guide for AIX, HP-UX, Linux, and Solaris (REP-U26IR) contains installation instructions, planning considerations, and configuration and usage instructions.
FOR FURTHER INFORMATION RESTRICTIONS ON USE
If you wish to obtain further information about Softek Replicator, please contact your Account Executive. The following terms are registered or common law trademarks of International Business Machines Corporation in the United States, other countries, or both: AIX, IBM, the IBM logo, ibm.com, OS/390,OS/400, Softek, Softek Replicator for UNIX, z/OS, and zSeries Intel and Itanium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product and service names may be trademarks or registered trademarks or service marks of others. The information contained in this documentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this documentation, it is provided as is without warranty of any kind, express or implied. In addition, this information is based on IBMs current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this documentation or any other documentation. Nothing contained in this documentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM (or its suppliers or licensors), or altering the terms and conditions of the applicable license agreement governing the use of IBM software. This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the users responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: IBM World Trade Asia Corporation Licensing 2-31 Roppongi 3-chome, Minato-ku Tokyo 106-0032, Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged should contact: IBM Global Services IBM Corporation Route 100 Somers, NY 10589 U.S.A. Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this information and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement, or any equivalent agreement between us. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. Copyright IBM Corporation 2010 All rights reserved. Printed in Canada. All specifications are subject to change without notice.
BIND Notice
Copyright 1986, 1989, 1990 The Regents of the University of California. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgement: 4. This product includes software developed by the University of California, Berkeley and its contributors. 5. Neither the name of the University nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS `ÀS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF UBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Portions Copyright (c) 1993 by Digital Equipment Corporation. Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies, and that the name of Digital Equipment Corporation not be used in advertising or publicity pertaining to distribution of the document or software without specific, written prior permission. THE SOFTWARE IS PROVIDED "AS IS" AND DIGITAL EQUIPMENT CORP. DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL DIGITAL EQUIPMENT CORPORATION BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. Copyright Copyright (c) 1986, 1989, 1990 The Regents of the University of California. portions Copyright (c) 1993 Digital Equipment Corporation portions Copyright (c) 1995 Internet Software Consortium All rights reserved.
Klog.c Notice
Copyright 1992, 1993 The Regents of the University of California. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the University of California, Berkeley and its contributors. 4. Neither the name of the University nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS `ÀS IS''AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Tcl/Tk License Terms

This software is copyrighted by the Regents of the University of California, Sun Microsystems, Inc., Scriptics Corporation, and other parties. The following terms apply to all files associated with the software unless explicitly disclaimed in individual files. The authors hereby grant permission to use, copy, modify, distribute, and license this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required for any of the authorized uses. Modifications to this software may be copyrighted by their authors and need not follow the licensing terms described here, provided that the new terms are clearly indicated on the first page of each file where they apply. IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. GOVERNMENT USE: If you are acquiring this software on behalf of the U.S. government, the Government shall have only "Restricted Rights" in the software and related documentation as defined in the Federal Acquisition Regulations (FARs) in Clause 52.227.19 (c) (2). If you are acquiring the software on behalf of the Department of Defense, the software shall be classified as "Commercial Computer Software" and the Government shall have only "Restricted Rights" as defined in Clause 252.227-7013 (c) (1) of DFARs. Notwithstanding the foregoing, the authors grant the U.S. Government and others acting in its behalf permission to use and distribute the software in accordance with the terms specified in this license.
Summary of Changes
Summary of Changes
This section lists technical updates and new material added to the Softek Replicator Installation and Reference Guide for AIX, HP-UX, Linux, and Solaris, for each release of Softek Replicator.
Softek Replicator for UNIX 2.6.x

REP-U26IR-016 - March 2010
The former Installation Guide has been merged with the former Administration and User Guide and the former Planning and Implementation Guide, to create a new single manual: Installation and Reference Guide. This manual combines all contents for Softek Replicator versions 2.6.2, 2.6.3, and 2.6.4, and adds new content for version 2.6.5.
New Features for 2.6.5

D
Support for the SuSE 11 Enterprise Server; for more information, refer to SuSE Linux System Requirements on page 57.
Technical Updates for 2.6.5

D D
The Courier Transport Method has been updated; refer to Using the Courier Transport Method on page 126. A new appendix has been added; refer to Appendix F: Implementing Softek Replicator With Solaris Zones.

D
Support for the following platforms; for more information, refer to Chapter 6: Installing Softek Replicator on Linux:
H
Red Hat Enterprise Linux AP, Version 5.0 (2.6.18 kernel), on 64-bit systems (x86, Intel64 Xeon, Pentium 4, AMD Athlon, and Itanium), Updates 1, 2, and 3. This also includes support of hyper-threaded Intel/AMD (dual core) 64-bit servers. Support for SuSE Linux Enterprise Server 10 through 10.2 with 32-bit and 64-bit kernels.
H D
Support for the Autostart feature, which automatically starts Mobility Groups after a server reboot. For details, refer to Step 1: Creating Mobility Groups on page 104.
www.ibm.com
Licensed Material Property of IBM Corporation
vii
Summary of Changes
Technical Updates for 2.6.4

D D
The Courier Transport Method has been updated; refer to Step 4: Starting the Replication on page 126. A new appendix has been added; refer to Appendix F: Implementing Softek Replicator With Solaris Zones.

D D
This release of Softek Replicator supports Dynamic Driver Loading (non-disruptive install) on Solaris. For details, refer to Chapter 10: Getting Started With Replication. Support for zFS File Systems on Solaris; refer to Chapter 12: Replicating Data on ZFS File Systems.

D D
Support for AIX 6.1. Support for SuSE 9 Linux.
viii
www.ibm.com
About This Guide
About This Guide

The Softek Replicator for UNIX 2.6 Installation and Reference Guide for AIX, HP-UX, Linux, and Solaris (REP-U26IR) provides instructions on how to install, configure, administer, and use Softek Replicator on AIX, HP-UX, Linux, and Solaris systems.
Contacting Technical Support

If you have a technical issue that you cannot answer with the provided resources, please contact the Global Support Center by Email, on the Web, or by Phone.
" D
By Email: Send email requests to DMSOpen@us.ibm.com for:

H H H H
Requesting the creation of Severity 2, 3, or 4 problems; Tracking the status of open problem cases; Asking questions; Submitting enhancements requests.
" D
On the Web: Visit http://www-950.ibm.com/services/dms/en/support/ for product-specific information such as FAQs, user libraries, patches, discussion forums, and to download documentation. For license requests, please contact your Account Executive. By Phone: For severity 1 problems and around-the-clock mission-critical support, contact the Global Support Center using the following toll-free phone numbers:
H H
" D
North America: 1-800-667-6383

Europe:
D D
Germany, Ireland, Italy, and UK: 00-800-6676-3835 All Other European Countries: +1 918-281-2623 Australia: 0011-800-6676-3835 Hong Kong: 001-800-6676-3835 Japan: 010-800-667-63835 South Korea: 002-800-6676-3835 Philippines: 00-800-6676-3835 China: 800 810 1818-5174
Asia-Pacific Region:
D D D D D D
H H
Other Countries: +800-800-667-6383 Alternate number : +1 918-281-2623

www.ibm.com
ix
About This Guide
NOTE: During weekends and holidays, you will be asked if you require assistance before the next business day. If so, Softek will respond within stated response time commitments. Other phone numbers may be used within European countries where local language capability exists. Contact your local Softek Sales Representative. Major issues can be addressed to Softek management by a request through the Alert Centres.
Related Publications
The following publications contain related information:
D D D D D D
Softek Replicator for UNIX 2.6 Messages Guide for AIX, HP-UX, Linux, and Solaris (REPU26MG) Softek Replicator for UNIX 2.6 Release Notes for AIX, HP-UX, Linux, and Solaris (REPU26RN) Softek Replicator for UNIX 2.6 Quick Reference Card for AIX, HP-UX, Linux, and Solaris (REP-U26QC) Softek Data Mobility Console 1.1.x Installation and User Guide (DMC-W11IU) Softek Data Mobility Console 1.1.x Quick Reference Card (DMC-W11QC) Softek Data Mobility Console 1.1.x Release Notes (DMC-W11RN)
Registering for Email Notifications

You can register to receive an email notification when new support information is available for Softek Replicator, including new hotfixes/patches, documentation updates, and other support related updates.
"
To register for email notifications:
1. Visit http://www-950.ibm.com/services/dms/en/support/ 2. Select Softek Replicator from the drop-down list box to open its support homepage. 3. Select the UNIX platform. 4. Click the link under Notification Registry.
NOTE: You must have a User ID and Password to access the Notification Registry page. To request access, go to the Call Tracking web page: http://www-950.ibm.com/services/dms/en/support/tracking/newaccount.php
5. Follow the on-screen instructions to complete registration.
www.ibm.com
Contents
Contents
PART I: Introducing and Installing Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . 1 Chapter 1: Introducing Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disaster Recovery and the Role of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using Softek Replicator for Data Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 6 7 8
Chapter 2: Understanding Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . 9

How Softek Replicator Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dynamic Activation on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Softek Replicator Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Primary Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 12 13 14
Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Softek Replicator Master Daemon: in.dtc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 dtc Device Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 Primary Mirror Daemon (PMD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 Big Asynchronous Buffer (BAB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 Pstore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Secondary Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Remote Mirror Daemon (RMD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Target Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Journal File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Softek Replicator Parameters and Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Operating Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Softek Replicator Operating States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Throttles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Internet Protocol Version 6 (IPv6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Data Replication from the Softek Data Mobility Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20 20 21 22 25 25 26
Chapter 3: Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

System Requirements for the Mobility Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Supported Operating Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Preparing the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sizing the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sizing the BAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 30 30 31 31 32
Minimum and Maximum BAB Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

www.ibm.com Licensed Material Property of IBM Corporation
xi
Contents Other Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Sizing the Journal File Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 UNIX Compatibility Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Installing Softek Replicator for UNIX and Softek TDMF for UNIX on the Same Server . . . . . . . . . . . . . 35
Chapter 4: Installing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

AIX System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Removing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 40 41 42 43
Chapter 5: Installing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

HP-UX System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Minimum Patch Requirement for HP-UX 11.11 (11i v1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Removing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
49 49 50 52
Chapter 6: Installing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Red Hat Linux System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Opens Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Required Libraries for Red Hat Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SuSE Linux System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Open Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Required Libraries for SuSE Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Removing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 55 56 57 57 58 59 60 61 62
Chapter 7: Installing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Solaris System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Upgrading Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Removing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 66 66 67 68
PART II: Using Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Chapter 8: Planning for Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Preparing the Mobility Server for Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
xii
Licensed Material Property of IBM Corporation www.ibm.com
Contents
Considerations for Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Downtime Required . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Planning Considerations for Using the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Replicating Data with FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Actual Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sizing the Journal File Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Security Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . User Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . P-S Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R-C and C-C Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Port Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
73 74 74 74 75 75 77 78 79 80 80 80 82 82 83
Network Speeds and Elapsed times for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76
Softek Replicator Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83 MTU (Maximum Transmission Unit) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83 TCP Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83 RFC1323 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84 SACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84 Chunksize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84
Determining the Quality of the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Chapter 9: Complex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Symmetric Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Defining a Symmetric Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
On System B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90 On System A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Running B as Stand-Alone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inverting the A-B Topology to B-A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Switching Back from B to A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One-to-many Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chaining Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B to C Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A to B Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Loopback Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91 92 92 93 95 96 96 97
Chapter 10: Getting Started With Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Entering the License Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Obtaining the Host ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Entering License Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General Configuration Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General Replication Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.ibm.com
101 101 102 102 102 103

xiii
Contents
Replication Steps Breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step 1: Creating Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a Mobility Group Using Softek Data Mobility Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the Data Mobility Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step 2: Distributing the Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the PStore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding Other Devices for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step 3: Configuring the File Systems for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring File Systems on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
103 104 108 109 110 112 112 113 113
Sample Configuration File of a Primary Mobility Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111
File Systems Replication on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113 File System Expansion Device Naming on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113 Replicating JFS/JFS2 File Systems with External Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114 Matching Device Minor Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115 Target Minor Number Collisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115 JFS/JFS2 Logs for Data Replications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116 Creating and Mounting New Journaled File Systems on dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116 Creating and Mounting New JFS2 File Systems on dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117 Determining the Logical Volume Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117 Mounting Existing Journaled File Systems (JFS and JFS2) with dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . .121 JFS2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121 Starting Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .122
Configuring File Systems on HP-UX, Linux, and Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Starting Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123 Modifying the system mounting file (/etc/fstab or /etc/vfstab) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123
Step 4: Starting the Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Using the launchrefresh Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Using the Courier Transport Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Option 1 - Using dd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 Option 2 - Loopback Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .128
File Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS) . . Creating Symbolic Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moving a Non-Symbolic Linked Database to Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
132 132 133 134
Chapter 11: Online Volume Expansion of dtc Devices . . . . . . . . . . . . . . . . . . . . . . . 135

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Examples of Volume Expansion on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of Volume Expansion on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Examples of Volume Expansion on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of Volume Expansion in an HA/CMP Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 138 139 139 140
Chapter 12: Replicating Data on ZFS File Systems . . . . . . . . . . . . . . . . . . . . . . . . . 141

Replicating Data Using a One-to-One Configuration for Zpools Based on LUNs . . . . . . . . . . . . . . . . . . . Replicating Data Using a One-to-One Configuration for Zpools Based on Slices . . . . . . . . . . . . . . . . . . Replicating Data Using a Loopback Configuration for Zpools Based on LUNs . . . . . . . . . . . . . . . . . . . . . Replicating Data Using a Loopback Configuration for Zpools Based on Slices . . . . . . . . . . . . . . . . . . . . .
xiv
143 144 146 147
www.ibm.com
Contents
Chapter 13: Using Softek Replicator in a Cluster Environment . . . . . . . . . . . . . . . . 149

Configuring Softek Replicator in a Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setting the Initial Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Failover Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Restarting the Data Replication in Case of Cluster Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Veritas Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setup and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 151 152 152 153 153
Environment Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Softek Replicator Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Veritas Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154 Softek Replicator / Veritas Cluster Integration Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154
Veritas Cluster Replication Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Scenario 1 - Replicating Data From Active Cluster Node to a Disaster Recovery System . . . . . . . . . . . . . . . . .154 Scenario 2 - Failover of Active Node to Passive Node With Replication Through Single IP Address . . . . . . . . .155 Scenario 3 - Failover of Active Node to Passive Node without the IP address as Part of Cluster Service Group 156 Scenario 4 - Failover with Veritas Cluster Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .157 Scenario 5 - Failover with VCS and Network Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .157
HP-UX MC/Serviceguard Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 AIX HACMP Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Configuring Softek Replicator in HACMP Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Pre-configuration Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .161 Building the Softek Replicator Configurations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .161 Disabling Softek Replicator in the HACMP Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 Adding Resource Groups to a Running Softek Replicator Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164
Chapter 14: Replication in a Virtual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Running Softek Replicator on a VMware System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintaining and Changing the MAC Address of a Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Avoiding MAC Address Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manually Assigning a MAC Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 167 167 168
Chapter 15: Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

dtcagentset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcautomount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtccheckpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcconfigtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcdebugcapture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcexpand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcgenmklv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtchostinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcinit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcjfspremount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcjfspostmount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtckillbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.ibm.com
171 172 173 173 174 175 176 176 177 177 179 179 180 181
xv
Contents
dtckillpmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtckillrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtckillrmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtclicinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtclimitsize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcmklv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcmodfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcmonitortool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcmonitortty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcoverride . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcpanalyze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcperftool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcpmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcpsreplicate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcrmdreco . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcstart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcstop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . dtcsyncfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killdtcmaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killpmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . killrmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . launchagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . launchbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . launchdtcmaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . launchpmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . launchrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
181 182 182 182 183 183 184 184 185 186 187 187 188 189 190 191 192 193 193 194 194 195 195 196 196 197 198 198 199 199 200
Chapter 16: Using the Monitoring Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Using dtcperftool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step I: Setting up the Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step II: Specifying Measurements Shown in the Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step III: Displaying the Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 204 204 205
Modifying a Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .205 Other Chart Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206
Performance Monitoring Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Related Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206
Using dtcmonitortool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Status Message Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Error and Warning Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mobility Group / dtc Device Status Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Notification and Update Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xvi
207 207 208 208 209
www.ibm.com
Contents
dtcinfo ASCII Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Chapter 17: Administrative Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Working with Throttles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Throttle Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Measurement Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 213 215 216
Action Directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .216 Action Message Substitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .217
Creating Throttle Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Throttle Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Throttle to Regulate Maximum Network Bandwidth during Peak Business Hours . . . . . . . . . . . . . . . . . . . . . . .222
Working with Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Changing Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Changing Port Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Changing the Primary Server Master Daemon Port Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .227 Changing the Secondary Server Master Daemon Port Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228
Managing dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding/Modifying dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding a Mobility Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deleting Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Relocating the Pstore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modifying the BAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modifying the BAB Using dtcconfigtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modifying the BAB Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Working with the Journal File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resizing the Journal File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Turning Journal Operations On or Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Journal-less Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
228 228 230 231 231 232 233 233 233 234 234 235 235 236
Smart Refresh / Checksum Refresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .236 Checkpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .236
Using the Checkpoint Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Examples of Checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Checkpointing Procedure Example on AIX JFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .237
Checkpoint Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Checkpoint ON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .238 Checkpoint OFF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .239 Checkpoint Command/Action Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .239
Considerations and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Checkpoint Shell Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Primary Server Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .241 Secondary Server Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .242
Checkpoint Scripts for File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Primary Checkpoint Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .244 Secondary Checkpoint Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .244 Tips for Checkpoint Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .245
xvii
Contents
Mounting Mirror Devices on HP-UX and AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rebooting the Primary Server on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing Online Replication on AIX JFS2 File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Preparing JFS2 Volumes for Checkpoint Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Accessing a JFS2 Target Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
245 246 246 246 247
Accessing a JFS2 Target Drive in Case of Primary Server Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .247
User-Defined Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Changing the Primary Server into a Secondary Server, and Vice Versa . . . . . . . . . . . . . . . . . . . . . . . . . . 248
PART III: Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Appendix A: Recovering Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix B: HP-UX MC ServiceGuard Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix C: Sample Scripts for Veritas Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix D: Checkpoint Scripts for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix E: Sample Script for Changing Symbolic Links . . . . . . . . . . . . . . . . . . . . Appendix F: Implementing Softek Replicator With Solaris Zones . . . . . . . . . . . . . . . 253 261 285 291 307 311
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
xviii
www.ibm.com
Contents
Figures
Figure 2.1: Softek Replicator Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Figure 2.2: Relationship Between Softek Replicator Components . . . . . . . . . . . . . . . . . . . . . . 14 Figure 2.3: dtc Device Driver in the UNIX Kernel and its Relationship to Data Devices . . . . . . . 16 Figure 2.4: The PMD/RMD Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Figure 2.5: Operating Modes and State Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 2.6: Replications Managed by the Softek Data Mobility Console . . . . . . . . . . . . . . . . . . 26 Figure 8.1: Network Compression Throttled at 1 MB per second . . . . . . . . . . . . . . . . . . . . . . . 76 Figure 8.2: Link Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Figure 9.3: Sample Symmetric Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Figure 9.4: Sample One-To-Many Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Figure 9.5: Sample Chaining Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Figure 10.6: Defining the BAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Figure 10.7: dtcconfigtool - Systems Tab (AIX, HP-UX, Linux) . . . . . . . . . . . . . . . . . . . . . . . 105 Figure 10.8: dtcconfigtool - Systems Tab (Solaris Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Figure 10.9: dtcconfigtool - dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Figure 10.10: Courier Transport Method Using Loopback Configuration 1 . . . . . . . . . . . . . . . 128 Figure 10.11: Courier Transport Method Using Loopback Configuration 2 . . . . . . . . . . . . . . . 129 Figure 10.12: Courier Transport Method Using Loopback Configuration 3 . . . . . . . . . . . . . . . 130 Figure 10.13: Courier Transport Method Using Loopback Configuration 4 . . . . . . . . . . . . . . . 131 Figure 10.14: Courier Transport Method Using Loopback Configuration 5 . . . . . . . . . . . . . . . 131 Figure 13.1: Sample Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Figure 16.1: Softek Replicator Performance Monitor Window . . . . . . . . . . . . . . . . . . . . . . . . 203 Figure 16.2: dtcmonitortool Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Figure 16.3: Mobility Group and Device Status Area of dtcmonitortool . . . . . . . . . . . . . . . . . 208 Figure 16.4: dtcinfo Output Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Figure 17.5: dtcconfigtool - Throttles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Figure 17.6: Setting the Evaluation Time through the Throttle Builder . . . . . . . . . . . . . . . . . . 219 Figure 17.7: Throttle Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 Figure 17.8: Building Throttle Actions with the Throttle Builder . . . . . . . . . . . . . . . . . . . . . . . 221 Figure 17.9: Tunable Parameters Tab in dtcconfigtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Figure 17.10: Configuring AIX. HP-UX, Linux, and Solaris - TCP Settings . . . . . . . . . . . . . . 227 Figure 17.11: Script Sample of /etc/opt/SFTKdtc/cp_pre_on_p012.sh for Primary Server . . . 241 Figure 17.12: Script Sample of /etc/opt/SFTKdtc/cp_post_on_p012.sh for Primary Server . . 242 Figure 17.13: Script Sample of /etc/opt/SFTKdtc/cp_post_on_s012.sh for Secondary Server 243 Figure 17.14: Script Sample of /etc/opt/SFTKdtc/cp_pre_off_s012.sh for Secondary Server 243 Figure 17.15: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_post_on_s001.sh . . . . 244 Figure 17.16: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_pre_off_s001.sh . . . . . 245
www.ibm.com
xix
Contents
Tables
Table 3.1: Softek Replicator Configuration Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Table 3.2: UNIX Compatibility Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Table 4.1: Softek Replicator Directory Structure on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Table 5.1: Softek Replicator Directory Structure on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Table 6.1: Softek Replicator Directory Structure on Red Hat Linux . . . . . . . . . . . . . . . . . . . . . 59 Table 6.2: Softek Replicator Directory Structure on SuSE Linux . . . . . . . . . . . . . . . . . . . . . . . 59 Table 7.1: Softek Replicator Directory Structure on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Table 8.1: Network Numbers for Sending 1 GB of Oracle Data . . . . . . . . . . . . . . . . . . . . . . . . 75 Table 8.2: Network Speed for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Table 9.3: Sample Symmetric Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Table 9.4: Sample One-To-Many Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Table 9.5: Sample Chaining Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Table 17.1: Throttle Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Table 17.2: Action Directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Table 17.3: Action Message Substitutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Table 17.4: Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Table 17.5: Checkpoint Command/Action Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Table E.1: Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
xx
www.ibm.com
Part I
Introducing and Installing Softek Replicator for UNIX
Chapter 1: Introducing Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Chapter 2: Understanding Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . . .9 Chapter 3: Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 Chapter 4: Installing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37 Chapter 5: Installing Softek Replicator on HP-UX. . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Chapter 6: Installing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 Chapter 7: Installing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Chapter 1
Introducing Softek Replicator for UNIX
Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Disaster Recovery and the Role of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Using Softek Replicator for Data Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
To an enterprise, a disaster is any event, large or small, that causes applications to be no longer available at a primary location, and requires these applications to be run at a remote (secondary) location. This could include natural events, such as floods, fires or tornados; long power failures; deliberate acts of destruction; accidental critical failure at site; or even a simple network issue. A disaster is a situation that cannot be handled locally. Data plays an important but very specific role in the areas of Disaster Recovery, Business Continuity, and High Availability. It is important to understand the full scope of these areas to appreciate the full importance of .
Disaster Recovery
The most common definition of Disaster Recovery is the provision of an alternate production setup to be available in the event of a major disaster at the primary location. This typically occurs when the entire primary facility is not available, making it necessary to provide an alternate production setup that is remote from the primary production site. Today, the type of that keeps data most up-to-date is synchronous mirroring, in which data is written to the secondary system at the time it is written to the primary system. Synchronous mirrors, across large distances are generally not practical for most companies. They are practical only over very limited distances but are very expensive due to network and hardware requirements. Synchronous mirroring has been implemented by a limited number of enterprises as a method of securing data remotely in the event of disaster. This is an extremely expensive solution and is only cost-effective for the most critical applications. The Disaster Recovery solution attracting most enterprises is asynchronous volume-based , a solution that can be applied in a cost-effective manner to a much wider range of business applications because of its low cost. In addition, asynchronous leverages existing infrastructure and does not require specialized hardware to implement. However, it does not affect application performance as synchronous does. Although securing data is the most difficult and most critical part of providing a remote Disaster Recovery solution, there are other important considerations. Disaster Recovery is always an expensive exercise due to its goal to keep applications available when an unexpected event occurs. As a result, there is little or no purpose in going to the expense of providing a Disaster Recovery solution that is not always reliableit must be all or nothing. Another difficulty in providing a Disaster Recovery facility is the ability to test the solution. With Disaster Recovery you have to assume that there are no resources available at the primary site, such as people, documentation, procedures, equipment, and data. Applications must keep running, or be recovered at the secondary site. In Disaster Recovery situations, there are two main measures that determine the required service:
D
Recovery Point Objective (RPO). This is the point in time to which systems and data must be recovered in the Disaster Recovery facility. With asynchronous , the objective is usually to recover the data to within seconds of the interruption at the primary site. Without asynchronous , the RPO may be to a backup or some other Point-in-Time (PIT) copy of the data containing significantly older data. Recovery Time Objective (RTO). This is the period of time within which systems and applications must be recovered after an outage. This includes application recovery time.
Applications often have differing RPOs and RTOs depending on the nature of the application and its significance to the operation of the business. For example, an airline reservation system
Business Continuity
or a banks financial tracking system will have a much different RPO and RTO than a retailers stock ordering system. Although can play a role in business continuity, by providing PIT copies for otherwise disruptive activities, it is the main enabler of Disaster Recovery for midrange systems. It is seldom used in High Availability solutions.
Business Continuity
The importance of the role of Information Technology (IT) in business continuity depends somewhat on the nature of the business. For some companies, IT is a fundamental part of the business; they cannot operate without it. Other companies can survive for varying periods of time without IT. Full-scale business continuity initiatives can cost a huge amount of money so very few companies have full time Business Continuity managers. Almost all companies look at providing some kind of Business Continuity for the most critical aspects of their business, and one of the main areas is IT. It is much easier to sell High Availability and Disaster Recovery solutions to a company that has at least some background in Business Continuity initiatives. These are often enterprises that have either suffered some kind of disruption to their business; worked with companies that have suffered a disruption; or have in some other way been affected by a disruption.
High Availability
High Availability can be defined as any initiative that keeps the business running normally without disruption. In general it encompasses anything just short of a full-scale disaster. The essential difference between High Availability and Disaster Recovery is that High Availability does not involve any loss of data, does not require application recovery, and the service disruption can be measured (usually) in seconds rather than minutes, hours or days. A very important aspect of High Availability is that it must be designed into the system from the top down (from the application down to the disk). High Availability needs to be designed into every layer of the technology stack with the goal of avoiding single points of failure. Examples of High Availability products include application clusters and redundant hardware. Softek Replicator is a High Availability tool because it allows applications to be kept online while changing storage. Softek Replicator is also a Disaster Recovery tool because it provides recovery in the event of a major disaster or disruption.
www.ibm.com
Disaster Recovery and the Role of Data

Disaster Recovery contingency planning is a large topic and must start with an understanding of the business need for the availability of an application in various disaster scenarios. Every application has different characteristics and these characteristics may vary over time. For example, a payroll system is not critical except for 2 or 3 days at the end of the payroll period. Similarly, an application that once made a small contribution to the business may become very critical to the business over quite a short period of time, because of changes in business focus. What this means is that Disaster Recovery requirements are dynamic and need to be reassessed on a frequent basis. A business analysis of each application must answer two basic questions: 1. How long can the business continue to function without the application, and in essence, how much will it cost the business to operate without the application? 2. How much loss of data can the application tolerate, and what is the cost to the business of not having the data? This second question is actually quite complex and has changed significantly in recent years. Until recently, most systems, including online systems, operated in such a way that data could be recreated. In other words, if a system failed during the day, administrators could exactly recreate the situation by restoring the last backup and re-running the input data. Typically, transactions were captured on other media or backed up at various stages of processing before being applied to the main system. Disaster Recovery in these cases was lengthy and complicated, but in general, a major disaster at any location or even just a disk failure would not result in any data loss if the right backups were applied at the right points. Today, many transactions are entered directly into online 24/7 systems. The most cost-effective and flexible solution for Disaster Recovery is host-based . There are two types of host-based , file-level and volume- or block-level. File-level is specific to a particular file system and is really only practical for small systems. Volume-level is much more suitable for mid-range systems and database servers. Asynchronous data for Disaster Recovery is a compromise between cost and risk. In all cases there will be some data loss with asynchronous , but the cost is a fraction of a synchronous fiberbased solution. In most cases the data loss for these systems is permanent and cannot be recovered. Although the loss may only be a few seconds worth of data, it will still represent a loss to the business. So for banking systems where each transaction could be millions of dollars, a synchronous Disaster Recovery solution is probably required. For an online system where the cost of a transaction is relatively small, an asynchronous solution is almost certainly the best solution. Often in these cases the RTO (Recovery Time Objective) is more critical than the RPO (Recovery Point objective). It is more urgent to get the system back online with some small data loss than to be offline for hours or days but have no data loss. Asynchronous is a perfect solution in these types of situations.
www.ibm.com
Using Softek Replicator for Data Replication
Using Softek Replicator for Data Replication

Softek Replicator is a host-based server-to-server data product designed for use in a Disaster Recovery solution. It es logical volumes in real time from a Primary System to a Secondary System with minimal impact on production applications while ensuring data consistency at the Secondary site. Unlike other technologies, Softek Replicator is independent of application, file system, volume manager, and storage vendor. Softek Replicator provides a cost-effective means of securing data for Disaster Recovery use. Cost-effectiveness is the key criteria that gives Softek Replicator a distinct advantage for Disaster Recovery since alternate hardware- or network-based solutions, although possible, are too expensive to address the needs of these mid-range servers. In many cases, the servers are geographically dispersed with relatively small amounts of storage attached to them, so hardware-based solutions are extremely expensive. Softek Replicator also covers a very wide range of operating systems and operating system levels, which provides a single solution across a very broad set of servers. The resources that are required to perform the functions can also be tuned to align with the size of the task to be performed. Softek Replicator synchronizes disk-based data from partitions (Source devices) on a Primary System to partitions (Target Devices) on a Secondary System, across any available TCP/IP network connection. Softek Replicator duplicates data in near real-time to ensure integrity between the two systems in the event of hardware failure, natural disaster, or human intervention. Softek Replicator accomplishes this through time-sequenced transfers of data from the Primary System to the Secondary System over the network. Softek Replicator incorporates multi-level recovery software to ensure that you have uninterrupted access to a usable dataset. It does not manipulate or corrupt data in any way. The ed data can be frozen at a specific point in time in the Primary Servers production schedule and made available for reading to applications on the Secondary System. This provides an application-consistent, point-in-time copy of the ed data to be used for backup, data analysis, or for seeding other applications, such as Web servers. This process does not in any way affect or compromise the Disaster Recovery requirement. Should a failure occur on the primary System, the Secondary System can provide immediate access to business-critical data. Softek Replicator ensures that you have uninterrupted access to usable data. In case of an outage, Softek Replicator provides a recoverable copy of coherent data on a Secondary Server. Softek Replicator allows you to begin synchronizing by selecting the partitions on which this data is stored into the Softek Replicator configuration. Disks do not need to be repartitioned or reformatted, and data does not need to be imported or exported. In addition, Softek Replicator does not manipulate or corrupt data in any way. Unlike other programs, Softek Replicator performs continuous transfers of updated data to the remote site, minimizing data loss and recovery time should a failure occur. Softek Replicator is a stand-alone product that does not require a volume manager or file system to be present, and can fit any system the customer has installed with no additional requirements.
www.ibm.com
Chapter 2
Understanding Softek Replicator for UNIX

How Softek Replicator Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Softek Replicator Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Softek Replicator Parameters and Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 Internet Protocol Version 6 (IPv6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Managing Data Replication from the Softek Data Mobility Console . . . . . . . . . . . . . . . .26
2
Softek Replicator is a comprehensive and powerful data synchronization solution for both planned and unplanned events. This section provides information on the different aspects of Softek Replicator and how they all fit together.
How Softek Replicator Works

Softek Replicator must be installed on all systems in the replication environment. Softek Replicator installs a special dtc device driver architecturally above the actual disk device drivers or volume management device drivers, but below file systems or applications. As a result, any disk-based file system supported by the operating system is compatible with Softek Replicator, as are applications that work directly with raw disk devices. Softek Replicator uses a kernel memory-based buffer cache - called the BAB (Big Asynchronous Buffer) - on the Primary Server to track all updates made to the Source Volume. As data is written to the source volume, the same information is stored in the BAB. Entries are then read from the BAB by the Primary Mirror Daemon (PMD) and transferred across the network connection to the Remote Mirror Daemon (RMD) on the Secondary Server. The PMD is a background daemon program running on the Primary Server; Softek Replicator creates one PMD for each Mobility Group. The RMD writes data received from the Primary Server to the target volumes. For each source volume on the Primary Server there must be a corresponding target volume on the Secondary Server. The dtc device provides the mapping to and management of a specific source volume, as well as the corresponding target volume. You can define a collection of dtc devices as a coherent unit, called a Mobility Group. Each Mobility Group operates with its own independent PMD/RMD daemon pair. During normal operation, updates are written directly to the target volume on the Secondary Server. You configure a journal file directory on the Secondary Server to handle situations when the data being transferred is not chronologically ordered. If this data were to be applied to the target volume directly, it would make the data on the target volume incoherent. The journal file directory is also actively receiving updates during checkpoint of the data set on the target volume. Checkpoint allows you to take the secondary mirror off-line from Softek Replicator, such as maintenance, and make it available for read-only access by other applications. In the standard configuration, Softek Replicator accumulates disk updates in the BAB independently of transmissions made to the Secondary Server. This dissociation is termed Async mode. The advantage of this configuration is that applications have near normal I/O performance, while allowing the daemon processes to exploit network bandwidth in an optimal way. However, if you need an exact mirror of the data on the Primary Server available at all times, you can also configure Softek Replicator to operate in Sync and Near Sync modes. For more information on operating modes, refer to Operating Modes on page 21.
www.ibm.com
11
When using Softek Replicator, there are two ways in which to interface with the product: 1. Using the Softek Data Mobility Console The Softek Data Mobility Console is the recommended interface for most replication environments and provides significantly improved productivity, through centralized management, enhanced ease of use, security and greater protection from human error. The Softek Data Mobility Console is essential in most disaster recovery scenarios and complex server and storage replication environments, where it is preferable to see all replication activities from a single point of control. The Softek Data Mobility Console provides a single user interface, through which all commands and configurations are completed. All events, logs, and performance displays, as well as overall Softek Replicator system views, are shown in the Softek Data Mobility Console. 2. Using the Configuration Tool from the UNIX Shell From the Primary Server, these tools are best utilized for disaster recovery or replications involving no more than two Mobility Servers (one Primary and one Secondary).
Dynamic Activation on Solaris

The new Dynamic Activation behaviour allows non-disruptive installation, configuration, and replication support, thus does not require you to unmount the source file system, modify the mount table to point to the dtc device, and then remount the target file system like previous releases of Softek Replicator. The initial phase of implementation involves determining which storage devices and/or volumes (source volumes) are eligible for replication, and then creating an association between the source volume(s) and a dtc device provided by the Softek Replicator driver. The next phase involves performing the actual replication by copying the data to the target volume. I/O operations are transparently intercepted by the Softek Replicator driver and applied to source and target devices while allowing applications to operate normally.
12
www.ibm.com
2
Softek Replicator Components
The basic replication environment consists of Mobility Servers. Mobility Servers are the servers from which, and to which, data is transferred. In a basic replication environment, there are two types of Mobility Servers: Primary Servers and Secondary Servers.
D
Primary Servers: The original servers containing the source volumes, from which data is
mirrored.
Secondary Servers: The backup servers containing the target volumes, to which data is
mirrored.
Primary Servers and Secondary Servers need not be exclusionary. One server can serve as both a Primary Server on one volume, and a Secondary Server on another volume. UNIX Mobility Servers can also receive commands from the Softek Data Mobility Console through the DMC Collector. They also send statistical records, performance data, and event logs to the DMC Collector, and keep the DMC Collector informed of any changes or problems. For more information on Softek Data Mobility Console and the DMC Collector, refer to Softek Data Mobility Console 1.1.x Installation and User Guide (DMC-W11IU).
Figure 2.1: Softek Replicator Components
www.ibm.com
13
Primary Server
The Primary Server provides application access and data storage services to users. The Primary Server must be running a supported operating system, have active TCP/IP connections to the Secondary Server, and have enough disk space and memory available to run Softek Replicator. After installing Softek Replicator on the Primary Server, you can configure the following components:
D D D D D D D
Mobility Groups Configuration Files Softek Replicator Master Daemon: in.dtc dtc Device Driver Primary Mirror Daemon (PMD) Big Asynchronous Buffer (BAB) Pstore
Mobility Groups
A Mobility Group is a collection of source partitions or source volumes treated as a single unit by Softek Replicator. Each Mobility Group operates with its own independent PMD/RMD daemon pair. Mobility Groups allow time-ordered writing coherence between member dtc devices, and complete operational and state independence.
Figure 2.2: Relationship Between Softek Replicator Components
14
www.ibm.com
2
You may want to define multiple Mobility Groups for the following reasons:
D
Some applications, especially databases, can work with a number of disk partitions at the same time. Organizing source volumes into a Mobility Group ensures chronological coherence. Chronological coherence must be maintained, not only within a single source volume, but also between source volumes, so that all partitions are in the same state. Mobility Groups maintain chronological coherence of I/O transfers between source volumes. A single Mobility Group can target multiple Secondary Servers, creating a one-to-many configuration where a portion of the original data set resides on multiple Secondary Servers. For more information on this and other configuration options, refer to Chapter 9: Complex Configurations. Mobility Groups can use independent network connections to the Secondary Server, creating an aggregated throughput greater than that of a single network connection. The control of one Mobility Group does not affect the operations of any other Mobility Groups (with the exception of BAB overflows). There is time-ordered write coherence between the local partitions in a Mobility Group, and complete operational and state independence between Mobility Groups.
D D D
Source Volume
The Source Volume is a disk or a managed volume associated with the Primary Server. A Source Volume is specified as a special character device and acts as the interface to the block mode device. Data stored on a Source Volume is duplicated onto the Secondary Server, so the Source Volume must have a correlating target volume of equal or greater size on the Secondary Server. You specify the Source Volume to be included in the Softek Replicator profile during configuration.
Configuration Files
For each Mobility Group on the Primary Server, Softek Replicator creates a configuration file. These configuration files contain Mobility Group definitions for the source and target volumes. The naming convention for the configuration files corresponds to the Mobility Groups number and prefixed with the letter p on the Primary Server. For example: p000.cfg corresponds to Mobility Group 0 on the Primary Server. Once these configuration files are created, they must be copied to the Secondary Server and the prefix must be changed to s from p. For more information, refer to Step 2: Distributing the Configuration Files on page 110.
Softek Replicator Master Daemon: in.dtc

When installing Softek Replicator, a user-mode background program is installed. The in.dtc establishes and manages both the PMD (Primary Mirror Daemon) and RMD (Remote Mirror Daemon) processes for each defined Mobility Group. This daemon also launches the process that manages and evaluates throttles for each Mobility Group of devices. For information, see Throttles on page 25.
www.ibm.com
15
dtc Device Driver

The dtc device is the means by which applications or file systems interact, access, or store information within Softek Replicator. The dtc device provides the mapping to and management of a specific source volume and the corresponding target volume. During configuration, each dtc device instance must have a unique name within a Mobility Group, for example, dtc0, dtc12, dtc93. Device names usually begin with 0 and are incremented by 1. Dtc devices appear as volumes to the kernel, so a dtc device accepts and handles any request that can be made to a normal volume or fixed-size volume, such as create and mount a file system, or allocate DBMS tablespace. Dtc devices are not shared by the Primary and Secondary Servers, they only exist on the Primary Server. If you want the Secondary Server to assume all activities if the Primary Server fails, then you must install the same application software on both systems. In a standard Softek Replicator configuration, you should run the applications on the Secondary Server only when the Server is ready to act as the main application server because the target volume is always being updated. The exception is when a snapshot of data on the Primary Server at a specific point-in-time is active.
Figure 2.3: dtc Device Driver in the UNIX Kernel and its Relationship to Data Devices
16
www.ibm.com
2
Primary Mirror Daemon (PMD)
The Primary Mirror Daemon (PMD) is a background process running on the Primary Server. Each Mobility Group has one PMD that is started automatically by the master daemon as part of the system bootscript or using the launchpmd or launchrefresh command. For more information on Softek Replicator commands, refer to Chapter 15: Commands. The PMD removes entries out of the BAB and sends them to the corresponding RMD on the Secondary Server. For information on the RMD, refer to Remote Mirror Daemon (RMD) on page 19. There is an independent PMD process for each Mobility Group defined on the Primary Server, so each Mobility Group can connect to a unique Secondary Server.
NOTE: If you experience a file table overflow error when attempting to start Softek Replicator or launch PMD processes on HP-UX servers, you may need to increase the value of the maxfiles_lim and nfile tunable parameters. These two HP-UX kernel tunable parameters determine the maximum number of files that can be opened per-process and system-wide, respectively.
Figure 2.4: The PMD/RMD Relationship
Big Asynchronous Buffer (BAB)

Softek Replicator uses a large kernel buffer called the Big Asynchronous Buffer (BAB) to journal updates as they are written to the source volume. This buffer resides in the physical kernel memory owned by the operating system. You must allocate this kernel memory to Softek Replicator during initial configuration. Writes to a dtc device are simultaneously logged in the BAB and written to source volume. Chronological coherence is maintained on a Mobility Group basis, therefore each Mobility Group assumes a portion of the BAB and updates to the devices in that Mobility Group are journaled sequentially within the area allocated. The entire memory designated for the BAB is allocated dynamically (on an as needed basis) among Mobility Groups. During configuration, you specify the memory allocated for the BAB.
www.ibm.com
17
Pstore
A Persistent Store (Pstore) is a volume on the Primary Server that stores state information, tunable parameter definitions, and tracking data about the Mobility Groups, as well bitmaps which track updates to the source volumes. When a block is updated on the source volume, the bit is turned on; when that block has been successfully transferred to the target volume, the bit is turned off. During configuration, you can define a single Pstore for all Mobility Groups or one Pstore for each Mobility Group. The default setting is one Pstore device, but multiple Pstores are recommended for a high availability environment.
Components of the Pstore

The Pstore consists of two components: a memory resident component, HRT, and a disk-based component, LRT. Both the HRT and LRT have two sections: a header section that keeps track of current Mobility Group and device states and a bitmap section.
D
HRT High Resolution Tracking. This is a memory-based bitmap. It is 128 kilobits for each
device and is used in normal update tracking of changed blocks and in Smart Refresh processing.
LRT Low Resolution Tracking. This is a disk-based bitmap. It is eight kilobits for each
device, used only in recovery mode when the Primary Server has failed.
The bitmaps are of fixed size and the granularity varies depending on the size of the volume the larger the volume the more data each bit represents. There are only two situations where the bitmaps will not be valid:
D D
If a Mobility Group has been in Passthru state, no tracking is done on updates to the source volume. If the Mobility Group is STOPPED, then tracking in the bitmaps is stopped as well.
In both cases, the only way to get back to a consistent state is to run either a Full or Checksum Refresh. Also, I/O is not tracked on the target volumes, so if the target volumes are updated in Checkpoint or Tracking state, a Full or Checksum Refresh must be run to get back into a consistent state. For information on operating states, refer to Softek Replicator Operating States on page 22. If you are replicating large volumes and will be using large bitmaps, ensure that you allocate 12 MB per device for the Pstore. If you intend to use the standard sized bitmaps, allow 140 KB per device for the Pstore.
18
www.ibm.com
2
Secondary Server
The Secondary Server is the server on which Softek Replicator replicates data from the Primary Server. During normal operation, this server provides a mirror of the data. The functionality of each system is defined in the configuration files, which you define on the Primary Server and copy to the Secondary. This allows you to switch the roles of Mobility Servers in Softek Replicator more easily. All applications used on the Primary Server should also be on the Secondary Server to allow you to switch operations over to the Secondary Server if needed. Target volumes and journal file directories are set up on the Secondary Server. The master daemon process (in.dtc) running on the Secondary Server launches an RMD for each Mobility Group. You can have more than one Secondary Server in the configuration, each representing the entire original data set or a portion of the original data set. For more information on advanced configurations, refer to Chapter 9: Complex Configurations. After installing Softek Replicator on the Secondary Server, you can configure the following components:
D D D
Remote Mirror Daemon (RMD) Target Volume Journal File Directory
Remote Mirror Daemon (RMD)

The Remote Mirror Daemon (PMD) is a background process running on the Secondary Server. Each Mobility Group has one PMD that is started automatically by the master daemon as part of the system bootscript or using the launchpmd or launchrefresh command. The RMD receives entries from the corresponding PMD on the Primary Server and writes the data to the target volumes. If you stop a PMD on the Primary Server, its corresponding RMD is is automatically brought down.
Target Volume
A target volume is specified as a special character device (but pertains to both the special character and corresponding block mode devices) on the Secondary Server. You must have a target volume on the Secondary Server for each source volume on the Primary Server. The target volume must be the same size or larger than the corresponding source volume. During normal operation, the target volume contain a coherentthat is, usablecopy of the data stored on the Primary Server. You can checkpoint the data on these devices, and applications other than Softek Replicator can open target volumes in read-only mode. For more information on checkpoint, see Using the Checkpoint Feature on page 237.
www.ibm.com
19
Journal File Directory

The journal file directory is used during a Smart Refresh operation, to store updates which, if applied to the target volume, would place it in an incoherent state. The journal file directory is also used to store updates while the checkpoint is active on the target volumes. These changes are eventually applied to the target volume to maintain coherence. These entries are saved to the target volume only after the PMD indicates that all updates have been transferred.You can set the journal file directories to any writable directory during configuration. A new journal file is created each time there is a state change that causes the data being transferred to go from coherent to incoherent. Softek Replicator uses the following naming convention for these files:
j###.###.c j###.###.i
(for coherent state) (for incoherent state)
The j indicates that this is a journal file, the first three numbers represent the Mobility Group to which the journal file belongs, and the second set of numbers are sequence numbers of journal files for each Mobility Group. For example:
j001.002.c
(second journal file for Mobility Group 1 in a coherent state)
Softek Replicator allows a maximum of 999 journal files, at approximately 500 MB per file, for each Mobility Group.
Softek Replicator Parameters and Modes

Softek Replicator uses different parameters and modes while in normal operation. Understanding these parameters and modes is essential to maximizing the benefits of the replication environment.
Tunable Parameters
Tunable parameters allow you to control certain Mobility Group function, such as the maximum size of a data packet sent from the Primary Server to the Secondary Server. Most of these parameters are saved in the Pstore, and are frequently evaluated by the PMD. Some tunable parameters are only processed by the throttle, since they are part of a throttle definition. Tunable parameters are an optional part of the configuration. If you choose not to define these parameters, the default values are used. For more information on tunable parameters, refer to Working with Tunable Parameters on page 223.
20
www.ibm.com
2
Operating Modes
Softek Replicator can operate in one of three modes: Asynchronous, Synchronous, or Near Synchronous.
D
Asynchronous Mode: By default, Softek Replicator operates in Asynchronous Mode (Async

mode), accumulating volume updates in the BAB that occur independently of the transmission of these entries to the Secondary Server. In Async mode, applications function with near-normal I/O performance while data is transferred using optimal network bandwidth.
Synchronous Mode: You can also configure Softek Replicator to operate in Synchronous mode (Sync mode) with the SYNCHMODE tunable parameter. SYNCHMODE does not return control to an application until after a disk update has been committed to both the Primary Servers source volume and the Secondary Servers target volume. In Sync mode, the target volumes are an exact copy of the source volumes at all times, except when a Sync mode timeout occurs. In the case of Sync mode timeout, the target volume will have an exact copy when the Mobility Group status reverts to Normal state after the Smart Refresh. For more information on states, refer to Softek Replicator Operating States on page 22.
NOTE: UFS files systems mounted on top of a Softek Replicator device in Sync mode continue to perform updates of I/Os asynchronously. VxFS file systems honour Softek Replicators Sync mode if installed with -o mincache=dsync.
D
Near Synch Mode: This mode is a middle ground between Async and Sync modes. Like Sync mode, you can configure Softek Replicator to operate in Near Sync mode with the SYNCHMODE tunable parameter. In Near Sync mode, disk updates accumulate in the BAB asynchronously until they reach the allowed limit. When the limit is reached, Softek Replicator blocks I/O operations to allow updates to be written to the Secondary Server until the entries in the BAB fall below the tunable limit. Near Sync mode improves on Sync mode performance, yet ensures that the data on the Secondary Servers target volume is no more than a fixed number of disk updates behind the Primary Servers source volume.
www.ibm.com
21
Softek Replicator Operating States

Softek Replicator operates in five different states:
D D D D D
Backfresh State Normal State PassThru State Refresh State Tracking State
With the Monitoring tool or the dtcinfo command, you can see what operating mode Softek Replicator is in.
Figure 2.5: Operating Modes and State Transitions
Backfresh State
Backfresh synchronizes the Primary Server back from the Secondary Server. Backfresh is useful when you are performing maintenance on the Primary Server. A Backfresh operation moves all blocks of data on the secondary target volumes that differ from those on the source volume to their corresponding locations on the Primary Server. Softek Replicator compares the blocks on the Primary Server with those on the Secondary Server to detect changes, using a checksum method. While the Backfresh operation is running, the applications should not access the source volumes and secondary target volumes because the Backfresh operation requires exclusive access to them. For more information on re-synchronizing data using Backfresh, see Appendix A: Recovering Data.
22
www.ibm.com
2
Normal State
Normal indicates that Softek Replicator is installed; dtc devices handle read/write requests; updates are applied to source volumes and written to the BAB for transfer to the target volumes on the Secondary Server, through the PMD/RMD daemons. In this mode, Softek Replicator performs continuous mirroring to have a coherent, recoverable copy of data on the Secondary Server.
PassThru State
In PassThru mode, read/write requests are fulfilled by Softek Replicator devices, but the dtc device driver writes only to the source volume. Updates are not transmitted to the BAB and no data is mirrored to the Secondary Server. Softek Replicator is in PassThru mode by default when installed. After configuring the product, perform a Full Refresh to create the initial mirror and move Softek Replicator into Normal state. Refreshing the set of data is the standard method to move a Mobility Group from PassThru mode to Normal state.
Refresh State
You use the Refresh to create an initial mirror or to synchronize the Secondary Server with the primary one. The Refresh state is a background operation, which means that it does not interfere with data mirroring. You place Softek Replicator in Refresh state with the launchrefresh command. A Refresh state transition also occurs automatically if the BAB becomes full while in Normal state. In this case, the system automatically goes into Tracking state, then to Refresh state. When the Refresh operation is complete, the system goes back to Normal state. You can specify a Refresh of single or multiple Mobility Groups, depending on the option you specify on the Refresh command. If you specify no options, a Smart Refresh is performed on all Mobility Groups. A Refresh is stopped automatically when the Refresh process is complete and Softek Replicator transitions to Normal state. You can also explicitly kill a Refresh operation with the killrefresh command. However, killing a Refresh operation before it is complete causes a loss of synchronization of the primary and Secondary Servers. The following Refresh states are available:
Full Refresh: A Full Refresh mirrors every block on the designated source volume(s) to the Secondary Server. The data is coherent only when the Full Refresh is complete. You should perform a Full Refresh to create the initial mirror.
Full Refresh is performed in two phases: 1. During Phase 1, new I/O information is stored in the Pstore and not sent to the target volume via the BAB (to avoid BAB overflows). 2. In Phase 2, a Smart Refresh-like operation updates the target volume by writing data directly to the disk, without using the Journals on the Secondary Server. Once Phase 2 completes, the corresponding Mobility Group returns to Normal state.
www.ibm.com
23
Smart Refresh: A Smart Refresh (the default Refresh method) mirrors only those blocks on the
source volume that have changed. Softek Replicator uses the information stored in the Pstore to track changes made to the source volume(s). When the BAB becomes full, the system automatically transfers to Tracking state and then to Refresh state. When the Smart Refresh completes, the system goes back to Normal state. During a Smart Refresh, the data is written to journal files on the secondary machine. These entries are saved on the target volume after all updates have been transferred. Data on the target volume is incoherent while the journal entries are being applied. The data is coherent only when all entries have been saved to the target volume. If the Journal feature is turned off, disk data transferred from the source device will be written directly to the target device, without storing it in Journals.
Checksum Refresh: A Checksum Refresh compares all blocks on the source volume with the
target volume, using a checksum method to identify deltas. Only blocks that have been modified (that is, those for which the checksum varies) are sent to the Secondary Server. A Checksum Refresh writes mirror data to the journal file system, if the Mobility Group has the Journal feature turned ON. If the Journal feature is turned off, disk data transferred from the source device will be written directly to the target device, without storing it in Journals.
Tracking State
Tracking state reduces the need for a Full Refresh in the case of network outage or loss of communication with the Secondary Servers target volumes. In Tracking state, Softek Replicator directs reads and writes to the dtc devices. Modifications to source volumes are also tracked. The Pstore is updated on every appropriate I/O operation. The Pstore consistently tracks all changes while Softek Replicator is in Normal, Refresh, and Tracking states. Entries are not written to the BAB, although the BAB may contain entries that have not been replicated from an earlier Normal state environment. Updates to the dtc device received while in Tracking state are not mirrored to the Secondary Server, but are written to the source volumes. When Softek Replicator leaves this mode, the PMD uses the changes in the Pstore to perform a Smart Refresh. A Smart Refresh is the standard method of transition from Tracking state to Normal state.
CAUTION: Leaving the Tracking state without performing a Refresh operationeither Smart or Fullcauses the Primary and Secondary Servers to become out of sync.
24
www.ibm.com
2
Throttles
Throttles help you to automate the administration of Softek Replicator. During configuration, you can define a throttle to alert you to system trends or problems. Throttles help keep a machine operating within a defined range. Throttle definitions are defined for each Mobility Group. Softek Replicator evaluates throttles periodically, based on a tunable parameter, and performs the specified actions. You can define an unlimited number of throttles for each Mobility Group. Each throttle can run up to 16 actions. In addition, there can be up to 16 clauses, or linked tests, in the throttle definition. For details on formatting throttles, refer to Working with Throttles on page 213.
Internet Protocol Version 6 (IPv6)

Softek Replicator now supports the specifications of Internet Protocol Version 6 (IPv6) addresses in the configuration files and the agent on the following operating systems:
D D D D
AIX 5.2 and higher HP-UX 11.23 and higher Red Hat Linux 4.0 with kernel 2.4.0 and higher Solaris 8 and higher dtcconfigtool does not support IPv6 addressing. When IPv6 addresses are used for the Primary and/or Secondary Server address, or when the hostname specified resolves to IPv6 address, dtcconfigtool will not be able to connect to the servers to obtain device list information. Device names can be entered manually, or, alternatively IPv4 addresses can be used temporarily to allow device configuration. After the configuration is complete, the host can be re-configured to use the IPv6 address. dtcmonitortool does not support IPv6 addressing, so it will not be able to obtain status information from the Secondary Server. The Softek Data Mobility Console does not support IPv6. Softek recommends using host name resolution instead of static IP addresses when configuring Softek Replicator for IPv6 to prevent configuration errors.
For complete information on IPV6, refer to your operating system manuals.

D
D D D
www.ibm.com
25
Managing Data Replication from the Softek Data Mobility Console

If you need to replicate data that is on both Windows and UNIX servers, or if you want to manage data replication from a centralized interface, you can use Softek Data Mobility Console. Softek Data Mobility Console is installed on a Windows server, and enables you to view information on every Mobility Server from a single point of reference, regardless of the operating system. This reduces the need for redundant monitoring and administration over separate systems and servers. The Softek Data Mobility Console enables you to do the following:
D D D
Manage a single Windows-to-Windows replication Manage a single UNIX-to-UNIX replication Manage two replication environments at the same time:
H H
Windows-to-Windows replication, and UNIX-to-UNIX replication.
In the UNIX-to-UNIX replication scenario, the Data Mobility Agent, installed automatically with Softek Replicator, will communicate with the Softek Data Mobility Console through the Data Collector and Database. You must configure and activate the Data Mobility Agent using the dtcagentset command.
Figure 2.6: Replications Managed by the Softek Data Mobility Console
26
www.ibm.com
Chapter 3
Installation Requirements
System Requirements for the Mobility Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Preparing the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 UNIX Compatibility Matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Installing Softek Replicator for UNIX and Softek TDMF for UNIX on the Same Server . 35
This chapter contains instructions for installing Softek Replicator 2.6, on AIX, HP-UX, Linux, and Solaris systems. Before installing Softek Replicator, verify the following conditions on all Primary and Secondary Servers.
System Requirements for the Mobility Servers

The following is a list of components required for the installation and functioning of Softek Replicator on UNIX platforms:
D D D D D
Primary Server with the source volume mounted with Read/Write enabled. Secondary Server LAN or WAN IP network(s) between Primary and Secondary Servers Local data devices (on primary) Disk partition or volume for the PStore on the Primary Server.
CAUTION: The Pstore disk partition/logical volume cannot be shared by the operating system or an application for any other usage (that is, the file system created on the Pstore volume). The Pstore device is used as a raw volume device to write binary data and can be corrupted if any other process writes to the Pstore partition/logical volume.
D D D
Mirror devices on the Secondary Server. Journal file directory on the Secondary Server. Softek Replicator Server Resources:
H H
10% CPU available at all times Physical memory allocated exclusively to Softek Replicator for the Big Asynchronous Buffer (BAB) on the Primary System. Maximum BAB size will vary by Operating System. For more information, refer to Supported Operating Parameters on page 30 and Sizing the BAB on page 32. In order to assign enough memory for Softek Replicator to work properly, use the following formula to set the system SWAP space: System SWAP Space = 1.5 System Physical Memory
NOTE: This is only a minimum. The amount of SWAP space to allocate depends on the configuration, such as the number of dtc devices, and the size of BAB.
H H
Network bandwidth to support peak update rates on the Primary Server. For large servers with unpredictable I/O loads; An environment where there is frequent activity in creating large files; or unreliable networks:
D D D
Identify and isolate volumes that contain only temporary data and that do not need to be replicated. Phase the Full Refresh by creating multiple Softek Replicators. Use compression on slow networks.
www.ibm.com
29
System Requirements for the Mobility Servers
Performance
The following performance guidelines can be used for planning purposes:
D
The actual performance depends on:

H H H
the speed or the source disks (Refresh state), the speed of the network, including the network quality and latency, and to some extent the server resources available (memory, CPU). the production application activity, which always takes precedence.
In Full Refresh state, Softek Replicator can achieve up to about 20 MB/s per Mobility Group over a network with low latency. Multiple Mobility Groups will scale in a linear fashion until the network reaches 70% utilization or more (two Mobility Groups will achieve 40 MB/s if the network can support it). In Normal state, Softek Replicator can achieve up to 15 MB/s across a network per Mobility Group. In a single server loopback configuration, Softek Replicator can achieve up to 50 MB/s transfer rates, and is usually limited by the performance of the disk subsystem on either the source or target disk.
D D
Supported Operating Parameters

This section describes guidelines for configuring Softek Replicator to run optimally.
Table 3.1: Softek Replicator Configuration Limits
Component BAB Memory Number of Devices
Minimum 32 MB 1
Maximum 1,547 MB
D D
Per Mobility Group: 500 For entire system: 500
NOTE: For Linux, up to 127 devices on Primary Server.
Number of Mobility Groups Pstore Size Number of Pstores Number of Journal files per Mobility Group
1 1 MB for 5 devices 1 0
100 100 MB 100 999

NOTE: You must perform a Full Refresh to empty the Journal files before Softek Replicator reaches the 1,000 file.
Size of a Journal file
0 MB
500 MB
30
www.ibm.com
Installation Requirements Table 3.1: Softek Replicator Configuration Limits (Continued)
Component Volume Size
Minimum
Maximum 5 TB
NOTE: If supported by the operating system and underlying device.
Preparing the System

This section describes the steps in preparing the system for Softek Replicator.
Sizing the System

Sizing of certain components such as the BAB and Pstore should be thoroughly thought out before starting the installation. The following rules can be helpful when sizing and allocating resources:
D D
Mirror devices must be at least as large as local data devices. In symmetric environments, they must be identically sized. Use tools like iostat (or volstat and sar on Solaris) or commercial systems management performance tools to profile the I/O characteristics and demands of your system. Peak I/O update activity over time. Do not ignore quarter-end or year-end processing requirements. Ratio of reads to writes. Number of disk blocks updated over a measurable amount of time. This data is especially useful when determining the appropriate size for the journaling file system on the secondary system(s), and the amount of memory for the BAB.
Look for:
D D D
Factor in the maximum rate for data transfer across the network and how long you can tolerate a network outage without a BAB overflow. A BAB overflow causes Softek Replicator to move into Tracking state and requires a Refresh to synchronize the primary and secondary systems. If possible, perform an iterative implementation of Softek Replicator and monitor operations. Pay attention to incidents when network connection is lost, when the secondary system goes down, or during unanticipated bursts in I/O activity.
www.ibm.com
31
Preparing the System
Sizing the BAB

It is recommended that additional physical memory be installed for the BAB on any Primary Server where Softek Replicator is installed. The BAB is in the physical kernel memory and not the virtual memory.
Minimum and Maximum BAB Sizes

on all operating systems, the size of the BAB is determined by the update rate and by the size of the network. The BAB is a buffer built to absorb temporary I/O peak rates.
D D D
Small systems should have a BAB between 128 MB-256 MB Medium sized systems should have a BAB between 256 MB and 512 MB Large sized systems should have a BAB larger than 512 MB; preferably over 1 GB.
NOTE: Sizing the BAB on AIX system with 32-bit kernel On systems running the AIX 32-bit kernel, the maximum kernel memory size is 512 MB, out of which a maximum of 256 MB may be allocated to the BAB. If you allocate more than 256 MB of kernel memory to the BAB, and depending on the processes and applications that are running on the system at a given time, you may encounter insufficient memory errors when you attempt to start a new process. If you encounter issues with kernel memory, you should lower the BAB size. CAUTION: Softek Replicator will fail if the BAB size is set to zero on the Primary Server or the system will fail if you exceed the the physical memory limit on AIX, HP-UX, Linux or Solaris.
Other Considerations
Since the BAB stores changes made to the primary data set, the following variables are important in determining the appropriate size:
D D D
Amount of data changed during a measurable period of time (burst of data); Speed of network and how fast entries are removed from the BAB; How long you can tolerate a network outage.
During configuration with dtcconfigtool, if the dtc device driver is unable to allocate all of the requested memory, a message prompts you to reboot the system. The reboot allows the dtc device driver to obtain the requested memory. The amount of memory actually allocated to the BAB is shown when the dtc device driver is added to the system. You can also determine this value with the dtcinfo -a command after the driver has been added.
32
www.ibm.com
Sizing the Journal File Directories

Journaling on the Secondary Server allows incoherent data to be transferred over the network without affecting the coherence or recoverability of the data on the mirror devices. Journaling also allows you to checkpoint (temporarily stop) mirror devices on the Secondary Server, while safely accumulating new updates from the Primary Server until after the Checkpoint has been released. At that time, the journaled updates are applied to the appropriate mirror devices. This process ensures that the data on the mirror devices is known during the Checkpoint operation, and allows other applications access to the mirror devices without interrupting operations on the Primary Server or requiring a Refresh.
NOTE: The default setting for the Journal feature is off.
The secondary journals are not used during a Full Refresh, but only when data is written directly to the mirror device(s)during a Smart Refresh, when blocks of changed data are written. The journal file directory should be large enough to handle all changed data from a Smart Refresh of one or more Mobility Groups, and to accommodate the updates that occur during the Checkpoint. Knowing the rate of changes made to data on the Primary Server is also useful in estimating the amount of data written to journal files during a Checkpoint. For example, the Checkpoint allows read-only access to the mirror devices so that you can do a backup. You can determine the time to complete a backup by simply dividing the total amount of data to be backed up by the transfer speed of the selected backup device. Then, you can determine the average amount of data changed during that time period and make accommodations for data bursts. The journal file directory should be large enough to accommodate this amount of data. The size of the journal file directory should be 25% of the entire amount of data stored on all local data devices in a Mobility Group. (A Mobility Group is a coherent set of devices that ensures time-ordered writing coherence between member devices.) Thus, if a Mobility Group has three local data devices totalling 12 GB, then the size of the journal file directory should be 3 GB (25% of 12 GB) for this Mobility Group. During configuration, you can define a journal file directory for all Mobility Groups or for each one.
www.ibm.com
33
UNIX Compatibility Matrix
UNIX Compatibility Matrix

Replication can be carried out on operating systems of the same type (for example, AIX to AIX), and of the same version, or from an older version to a newer version. This assumes that the Primary and Secondary Servers have the same configuration in terms of operating systems compatibility, and applications should have the same data structure definitions for the data device used, that is, identical file systems and database management systems. The following table describes the supported replications between the various UNIX platforms.
NOTE: Replication from a 32-bit kernel system to a 64-bit kernel system is supported.
Table 3.2: UNIX Compatibility Matrix
Platform AIX on IBM POWER (32-bit or 64-bit)
Primary Server Version 4.3 Version 5.1 Version 5.2 Version 5.3 Version 6.1
a
Secondary Server Version 4.3 Version 5.1, 5.2, 5.3 or 6.1 Version 5.1 Version 5.2, 5.3, or 6.1 Version 5.2 Version 5.3, or 6.1 Version 5.3 Version 6.1 Version 6.1 Version 11.11 Version 11.23 Version 11.31 Version 11.23 Version 11.31 Version 11.31 Version 4.0
HP-UX on PA-RISC (32-bit/64-bit)
Version 11.11
Version 11.23 Version 11.31 Red Hat AS/ES Linux on i386-based (32-bit/64-bit) Red Hat AS/ES Linux on 64-bit Itanium Red Hat AS/ES Linux on 64-bit Itanium SuSE Linux Enterprise Server (32-bit/64-bit) on Intel, AMD Version 4.0
Version 4.0
Version 4.0
Version 5.0
Version 5.0
Version 9
Version 9
34
www.ibm.com
Installation Requirements Table 3.2: UNIX Compatibility Matrix (Continued)
Platform SuSE Linux Enterprise Server (64-bit) on Itanium SuSE Linux Enterprise Server (32-bit/64-bit) on Intel, AMD SuSE Linux Enterprise Server (64-bit) on Intel, AMD
Primary Server Version 9
Secondary Server Version 9
Version 10
Version 10
Version 11
Version 11
a. AIX 4.3 only works on a 32-bit kernel.
Installing Softek Replicator for UNIX and Softek TDMF for UNIX on the Same Server
With the current release of Softek TDMF for UNIX and Softek Replicator, you can install both products on the same server. Both Data Mobility Agents can co-exist with each other, but you must create a *lparid.cfg configuration file and use different IP addresses for each product in order for Softek Data Mobility Console to differentiate between the two.
" D D
To create the *lparid.cfg file: For Softek TDMF for UNIX, create tdmflparid.cfg in /etc/opt/SFTKtdmf/ For Softek Replicator, create dtclparid.cfg in /etc/opt/SFTKdtc/
NOTE: Both files must contain a number between 1 and 512 and must not be identical to each other.
In the Hierarchy tree, all Softek TDMF for UNIX Replication Servers icons will have the letter T appended and when selecting a Secondary Server, the IP addresses of all Replication Servers will have the letter T appended.
www.ibm.com
35
Chapter 4
Installing Softek Replicator on AIX

AIX System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39 Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 Installing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41 Upgrading Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42 Removing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
This section provides installation instructions for each operating system. Softek Replicator is provided on a single CD that contains a software version for each UNIX operating system.
NOTE: You must install Softek Replicator on the Primary Server before installing it on the Secondary Server.
As with any software and hardware installation, it is strongly recommended that you back up the system fully before connecting new devices or installing Softek Replicator. A backup enables you to recover the original files if needed. If there are irreplacable files on the system, it is imperative that you perform a backup before proceeding to the installation.
AIX System Requirements

NOTE: It is recommended that you install the same operating system version on both Primary and Secondary Servers.
D
Supported versions installed on IBM POWER or POWER5 platforms:

H H H H H
4.3.3 with 32-bit kernel 5.1 with 32-bit or 64-bit kernel 5.2 with 32-bit or 64-bit kernel 5.3 with 32-bit or 64-bit kernel 6.1 with 64-bit kernel
D D D D
Minimum of 14.5 MB of disk space must be available on both Primary and Secondary Servers. Minimum of 32 MB physical memory to be allocated exclusively for the BAB on the Primary Server. An X-windows environment to use dtcperftool, dtcconfigtool, or dtcmonitortool. Supported File Systems:
H H
JFS JFS2
www.ibm.com
39
Softek Replicator Directory Structure

The tables outlines the directory structure that Softek Replicator uses on each operating system.
Table 4.1: Softek Replicator Directory Structure on AIX
Directory
/usr/dtc/bin/ /usr/sbin/dtcstop /usr/sbin/dtcstart /etc/dtc/lib /var/dtc/ /etc/dtc/init.d/dtc-chkpt_boot /etc/dtc/r3.d/S25dtc-startdaemons /lpp/dtc.rte /usr/lpp/dtc.rte* /usr/sys/inst.images/dtc.rte* /usr/lib/drivers/dtc* /usr/dtc/lib/ /usr/dtc/samples/chlink
Contains... Executable programs
Configuration and license files Performance and error logs Boot Scripts Installation files used by installp and
lpp
Drivers Libraries Sample script for changing symbolic links. For more information, refer to Appendix E: Sample Script for Changing Symbolic Links. Methods
/usr/lib/methods/dtc*
40
www.ibm.com

Use the following procedure to install Softek Replicator on an AIX system. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To install Softek Replicator on AIX:
1. Login as root user. 2. Load and mount the Softek Replicator CD-ROM:
mount -r -rv cdrfs /dev/cd0 /cdrom
3. Type the correct path for operating system, as follows:

cd /cdrom/Softek/Replicator/AIX/[4.3.3 | 5.1 | 5.2 | 5.3]
4. Type the following commands:

mkdir /var/dtc installp -a -V 4 -e /var/dtc/dtc_install.log -d . dtc.rte
5. To verify the installation, type the following:

lslpp -l | grep dtc
6. Type the license key into the /etc/dtc/lib/DTC.lic file on each Mobility Server to activate Softek Replicator. For more information, refer to Entering the License Keys on page 101.
NOTE: The installation modifies /usr/sbin/extendlv, and also replaces the default AIX /usr/sbin/ chfs command with a symbolic link to /usr/dtc/bin/dtcchfs. These changes provide additional safeguards for doing volume and file system expansion with Softek Replicator devices. If you upgrade or install a patch that overwrite these commands will remove these safeguards. To restore them, type: /usr/dtc/libexec/chfs_install.
www.ibm.com
41
Upgrading Softek Replicator on AIX
Upgrading Softek Replicator on AIX

The chapter provides upgrade information for Softek Replicator. Softek Replicator is backward compatible with previous releases that do not support replication of volumes larger than 1 TB. This allows you to perform an incremental upgrade of the Softek Replicator so that replication can continue when some sites have been upgraded to the newer version but others have not. You must first uninstall the current version of Softek Replicator from the Mobility Server before upgrading to Softek Replicator 2.6.5.
NOTE: A copy of the configuration files, license files and checkpoint scripts is automatically saved in the directory /var/dtc when uninstalling Softek Replicator. The configuration information can then be imported into the new version of Softek Replicator by running the dtcconfigtool command after installation.
If you are upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To upgrade Softek Replicator on AIX:
1. Unmount all dtc devices. 2. Wait for the BAB to release the memory and Softek Replicator to go into Normal state. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 4. On the Primary Servers, type: dtcstop -a to stop all Mobility Groups.
NOTE: To preserve tunable parameter settings, copy the files from /etc/dtc/lib into a temporary directory.
5. Type: installp -u dtc.rte to uninstall Softek Replicator. 6. Install the new version of Softek Replicator as described in Installing Softek Replicator on AIX on page 41. 7. Type dtcconfigtool to run the Configuration Tool after upgrading to replicate the configuration files from /var/dtc directory. To ensure that you replicate the correct files, you should remove unwanted directories from /var/dtc before upgrading.
NOTE: When run for the first time, the dtcagentset command also prompts you to copy old configuration files.
8. Type dtcpsreplicate to upgrade the Pstore volumes to the current version while preserving the existing tunable parameters.
NOTE: If you are upgrading to the latest release of Softek Replicator and have volumes that are larger than 300 GB, you may want to consider reinitializing the Pstore to support large high-resolution tracking (HRT) bitmaps. This functionality provides greater bit resolution and better performance and efficiency of Smart Refresh for larger volumes. However, reinitializing the Pstore will cause you to lose previous tunable settings and bit tracking information. Therefore a Full Refresh or Checksum Refresh will be required after reinitializing the Pstore. Also, you may need to increase the size of the Pstore volume to handle the larger bitmap sizes (approximately 12 MBs per DTC device is required). After resizing the underlying Pstore volumes, type dtcinit -l to enable support for large bitmaps.
42
www.ibm.com
9. Repeat this procedure on each Mobility Server in the environment.
Removing Softek Replicator on AIX

Use the following procedure to uninstall Softek Replicator on a AIX system using the installp command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To remove Softek Replicator on AIX:
1. Unmount all dtc devices and remove relevant entries from /etc/filesystems. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 4. Type: installp -u dtc.rte to uninstall Softek Replicator. 5. Type: lsvg -o | lsvg -i -l 6. Use the appropriate rmlv command to remove unused volumes.
NOTE: You need to reboot the system, before you can remove the Pstore using the rmlv command.
7. Type: rm -r /etc/dtc /usr/dtc /var/dtc to completely delete cached configuration files, checkpoint shell scripts, license files, and empty directories.
CAUTION: Do NOT type this command if you are upgrading to the latest version of Softek Replicator.
www.ibm.com
43
Chapter 5
Installing Softek Replicator on HP-UX

HP-UX System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Installing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Upgrading Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 Removing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
HP-UX System Requirements

D
PA-RISC systems:
H H H H
11.11 (11i v1) with 32-bit and s700_800 11.11 som2elf cumulative patch 11.11 (11i v1) with 64-bit kernel 11.23 (11i v2) with 64-bit kernel 11.31 (11i v3) with 64-bit kernel 11.23 (11i v2) with 64-bit kernel
Itanium systems:
H
NOTE: HP-UX only supports greater than 2 TB volumes on HP-UX 11.23, with VxFS version 5.
H D D D D
11.31 (11i v3) with 64-bit kernel
Minimum of 14.5 MB of disk space must be available on both Primary and Secondary Servers. An X-windows environment to use dtcperftool, dtcconfigtool, or dtcmonitortool. Minimum of 32 MB physical memory to be allocated exclusively for the BAB on the Primary Server. Supported File Systems:
H H
hfs VxFS
Supported volume managers: VERITAS Volume Manager 4.0 (VxVM)
CAUTION: Softek Replicator supports replicating a VxFS file system from HP-UX version 11.11 to version 11.23. However, Softek Replicator does NOT support replicating from version 11.23 to version 11.11. This is due to a change in the VxFS file system in version 11.23.
www.ibm.com
47
HP-UX System Requirements
Minimum Patch Requirement for HP-UX 11.11 (11i v1)

HP-UX 11.11 (11i v1) servers with 32-bit kernel must have the s700_800 11.11 som2elf cumulative patch be installed to ensure that the Softek Replicators loadable driver installs correctly. Without this patch, the following error message is displayed when trying to start a Mobility Group, or typing the dtcinfo -a command:
kmadmin: dtc: Object file error in loading kernel module. mksf: Couldn't find driver matching arguments DTC: Object file error in loading kernel module
You can download the patch from the following URL: http://www2.itrc.hp.com/service/patch/ mainPage.do
"
To determine whether the HP-UX system uses 32-bit or 64-bit hardware:
1. Type: getconf HW_CPU_SUPP_BITS

"
To determine whether the HP-UX system uses a 32-bit or 64-bit kernel:
1. Type: getconf KERNEL_BITS

" D
To check if the patch is installed: On HP-UX 11.11 (11i v1), type: swlist -v |grep 29436 The following should be displayed: contents PHCO_29436,r=1.0,a=HP-UX_B.11.11_32/
64,v=HP
NOTE: The following is an extract from the HP-UX PHCO_29436 README file for the s700_800 11.11 som2elf cumulative patch:
s700_800 11.11 som2elf cumulative patch For a 32-bit DLKM module that is linked with milli.a library,'kmadmin -L' fails with the following error message: "kmadmin: <module name>: Object file error in loading kernel module."
48
www.ibm.com

Table 5.1: Softek Replicator Directory Structure on HP-UX
Directory
/etc/opt/SFTKdtc /opt/SFTKdtc/bin /opt/SFTKdtc/ /opt/SFTKdtc/samples/chlink /sbin /sbin/init.d /sbin/rc3.d /usr/conf /usr/local/bin /opt/SFTKdtc/doc /var/opt/SFTKdtc
Contains... Licensing files, configuration files, template rc scripts User Executables Supporting libraries Sample script for changing symbolic links Startup scripts
Driver Files Symbolic links to User Executables User Manuals

dtcerror.log,
performance logs, configuration files from previous version

Use the following procedure to install Softek Replicator on a HP-UX system using the swinstall command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To install Softek Replicator on HP-UX:
1. Login as root user. 2. Load and mount the Softek Replicator CD-ROM:
mount /dev/dsk/"cd device" /<CD-ROM_mount_point>
Where <CD-ROM_mount_point> is the location of the CD-ROM. 3. Type the following command to display the SD Install - Software Selection dialog box:
swinstall -s /<CD-ROM_mount_point>/Softek/Replicator/HPUX/<HP-UX_Version> DTC
Where <HP-UX_Version> is one of the following:

D D D D D 11i/11i.depot 1123pa/1123pa.depot 1123ipf/1123ipf.depot 1131pa/1131pa.depot 1131ipf/1131ipf.depot
4. Highlight the DTC package from the list. 5. From the Action menu, select Mark for Install.
49
Upgrading Softek Replicator on HP-UX
6. From the Actions menu, select Install Analysis to display the analysis dialog box. 7. Click OK after the analysis of the system is complete. 8. In the Confirmation dialog box, Click Yes to install the product. 9. In the Install Window dialog box, wait until the Status indicates Completed.
H H
Select Logfile to see more information about the installation process. Select Done to return to the swinstall main dialog box.
10. Repeat these steps for each Primary and Secondary Server. 11. Add /opt/SFTKdtc/bin to the PATH environment variable. 12. Type the license key into the /etc/opt/SFTKdtc/DTC.lic file on each Mobility Server to activate Softek Replicator. For more information, refer to Entering the License Keys on page 101.
Upgrading Softek Replicator on HP-UX

NOTE: A copy of the configuration files, license files and checkpoint scripts is automatically saved in the directory /var/opt/SFTKdtc when uninstalling Softek Replicator. The configuration information can then be imported into the new version of Softek Replicator by running the dtcconfigtool command after installation.
"
To upgrade Softek Replicator on HP-UX:
1. Unmount all dtc devices and remove relevant entries from /etc/fstab. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes.
NOTE: To preserve tunable parameter settings, copy the files from /etc/opt/SFTKdtc into a temporary directory.
50
www.ibm.com
4. Type swremove -x mount_all_filesystems=false DTC to uninstall Softek Replicator. The swremove command preserves configuration files, license files and checkpoint scripts in the following directory:
H H /var/opt/SFTKdtc/SFTKdtc<Version>-1 (for HP-UX 11.11) /var/opt/SFTKdtc/SFTKdtc<Version>-2 (for HP-UX 11.23)
Where <Version> is the previous version of Softek Replicator in the format x.x.x.
NOTE: If you are upgrading from Softek Replicator version 2.1, remove the SFTKdua Agent package prior to uninstalling Softek Replicator 2.1. To remove the Agent package, type swremove -x mount_all_filesystems=false DUA
5. Install the new version of Softek Replicator as described in Installing Softek Replicator on HPUX on page 49. 6. Type: dtcconfigtool to run the Configuration Tool after upgrading to replicate the configuration files from /var/dtc directory. 7. To ensure that you replicate the correct files, you should remove unwanted directories from /var/dtc before upgrading.
8. Repeat the procedure on each Mobility Server in the environment. 9. Configure the environment as outlined in Chapter 10: Getting Started With Replication.
NOTE: If you have changed any configuration files or replicated configuration files on the Primary Servers, you must copy the updated file to the Secondary Servers. Copy the file, then stop and restart the Mobility Group. The dtcstart command processes the .cfg files, at which point changes to the file become effective. Tunable parameters are not stored in the .cfg files, so you need not copy the files after changing these parameters.
www.ibm.com
51
Removing Softek Replicator on HP-UX
Removing Softek Replicator on HP-UX

Use the following procedure to uninstall Softek Replicator on a HP-UX system using the swremove command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To remove Softek Replicator on the primary system:
1. Unmount all dtc devices and remove relevant entries from /etc/fstab. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 4. Type: swremove -x mount_all_filesystems=false DTC to uninstall Softek Replicator.
CAUTION: You must follow the above procedure exactly. Otherwise, you will have dtc remnants in the kernel. To remove the remants, type: kmmodreg -U dtc
5. Type the following commands to completely delete cached configuration files, checkpoint shell scripts, license files, and empty directories:
a. rm -rf /var/opt/SFTKdtc b. rm -rf /etc/opt/SFTKdtc c. rm -rf /opt/SFTKdtc
CAUTION: Do NOT type these commands if you are upgrading to the latest version of Softek Replicator.
52
www.ibm.com
Chapter 6
Installing Softek Replicator on Linux

Red Hat Linux System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 SuSE Linux System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57 Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59 Installing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Upgrading Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61 Removing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62
Red Hat Linux System Requirements

Opens Systems
H
For Softek Replicator version 2.6.4 only: Red Hat Enterprise Linux AP, Version 5.0 (2.6.18 kernel):
D
On 64-bit systems (x86, Intel-64 Xeon, Pentium 4, AMD Athlon, and Itanium), Updates 1, 2, and 3. This also includes support of hyper-threaded Intel/AMD (dual core) 64-bit servers. On 32-bit systems (x86, Intel-32 Xeon, Pentium 4 and AMD Athlon), Updates 4, 5, and 6. This also includes support of hyper-threaded x86 (dual core) 32-bit servers. On 64-bit systems (x86, Intel-64 Xeon, Pentium 4, AMD Athlon, and Itanium), Updates 4, 5, and 6. This also includes support of hyper-threaded Intel/AMD (dual core) 64-bit servers. ext2 ext3 IDE (hdx) SCSI (sdxx) via parallel SCSI, FCP, and iSCSI LVM, RAID
Red Hat Enterprise Linux AS/ES, Version 4.0 (2.6 kernel):

D D
Supported file systems:

D D
Supported volumes disks:

D D D D
H H H
At least 7.5 MB of disk space in /opt on both Primary and Secondary Servers, with 500 KB available in /etc/opt and at least 500 KB available in /var/opt. An X-windows environment to use dtcperftool, dtcconfigtool, or dtcmonitortool. Minimum of 32 MB physical memory to be allocated exclusively for the BAB on the Primary Server.
www.ibm.com
55
Red Hat Linux System Requirements
Minimum of 512MB of physical RAM for installation.

Installing Softek Replicator loadable kernel module insmod: error inserting '/opt/SFTKdtc/driver/linux-2.6.9-22.ELsmp.i686/sftkdtc.ko': -1 Operation not permitted
NOTE: If you do not have enough memory for installation, you will receive the following message:
Required Libraries for Red Hat Linux

Prior to installing Softek Replicator, install the required third-party libraries to ensure that Softek Replicator will function properly. This is accomplished by copying the appropriate Softek Replicator installation RPM file to the disk and double-clicking the file to install it. This will prompt you for requisite Red Hat installation CD.
D
For Red Hat Enterprise Linux AS/ES, 4.0:

H H H tcl-8.4.7-2.<platform>.rpm tk-8.4.7-2.<platform>.rpm tix-8.1.4-98.<platform>.rpm
Where <platform> is the system you will install the rpm.

H
BLT libraries The dtcperftool command requires the BLT libraries, which can be found on the Softek Replicator installation CD in the Redist/Red Hat/RPM directory.
For Red Hat Enterprise Linux Server 5 AP, versions 5.0 and 5.1:
H H H H otcl-8.4.13-3.fc6.<platform>.rpm otclx-8.4.0-5.fc6.<platform>.rpm otix-8.4.0-11.fc6.<platform>.rpm otk-8.4.13-3.fc6.<platform>.rpm
Where <platform> is the system you will install the rpm on.
D
For Red Hat Enterprise Linux Server 5 AP, versions 5.2 and 5.3:
H H H H otcl-8.4.13-3.fc6.<platform>.rpm otclx-8.4.0-5.fc6.<platform>.rpm otix-8.4.0-11.fc6.<platform>.rpm otk-8.4.13-5.el5_1.1.<platform>.rpm
Where <platform> is the system you will install the rpm on.
H
BLT Library The dtcperftool command requires the BLT library. Please contact support about obtaining the BLT library if you need to run dtcperftool.
" D
To discover installed packages: Type: rpm -q -a |egrep "tcl|tix|tk"
56
www.ibm.com
" D
To discover the CDs required for installation: Type /usr/share/comps-extras/whichcd.py <rpm_short_name> Where <rpm_short_name> can be itcl, tcl, tix, or tk
SuSE Linux System Requirements

Open Systems
D
Intel x86 systems:

H H H
SuSE Linux Enterprise Server 9 through 9.4 with 32-bit and 64-bit kernels SuSE Linux Enterprise Server 10 through 10.2 with 32-bit and 64-bit kernels (supported by Softek Replicator version 2.6.4 only) SuSE Linux Enterprise Server 11 with 64-bit kernels (supported by Softek Replicator version 2.6.5 only) SuSE Linux Enterprise Server 9 through 9.4 with 64-bit kernel SuSE Linux Enterprise Server 10 through 10.2 with 64-bit kernel (supported by Softek Replicator version 2.6.4 only)
Itanium systems:
H H
D D
AMD64 systems: SuSE Linux Enterprise Server 11 with 64-bit kernels (supported by Softek Replicator version 2.6.5 only) Supported file systems:
D D D D D D D
EXT2 EXT3 Reiser VxFS JFS XFS OCFS2
Supported volumes disks:

H H H H H
IDE (hdx) SCSI (sdxx) via parallel SCSI, FCP, and iSCSI LVM, RAID EVMS
www.ibm.com
57
SuSE Linux System Requirements
Supported Volume Managers:

H H
SuSE-VM VVM
D D D D
At least 7.5 MB of disk space in /opt on both Primary and Secondary Servers, with 500 KB available in /etc/opt and at least 500 KB available in /var/opt. An X-windows environment to use dtcperftool, dtcconfigtool, or dtcmonitortool. Minimum of 32 MB physical memory to be allocated exclusively for the BAB on the Primary Server. Minimum of 1 GB of physical RAM for installation.
Required Libraries for SuSE Linux

Prior to installing Softek Replicator, install the required third-party libraries to ensure that Softek Replicator will function properly. This is accomplished by copying the appropriate Softek Replicator installation RPM file to the disk and double-clicking the file to install it. This will prompt you for the requisite SuSE installation CD.
D
SuSE Linux Enterprise Server 9:

H H H H H H H tcl-8.4.6-26.3.<platform>.rpm, for otcl-8.4.6-26.7.<platform>.rpm, otclx-8.4-326.1.<platform>.rpm otix-8.1.4-76.1.<platform>.rpm otk-8.4.6-41.1.<platform>.rpm, otk-8.4.6-41.5.<platform>.rpm, oblt-2.4z-204.1.<platform>.rpm
versions 9.0 through 9.3
for version 9.4
for versions 9.0 through 9.3 for version 9.4

H H H H H H otcl-8.4.12-16.2.<platform>.rpm otk-8.4.12-14.2.<platform>.rpm, otk-8.4.12-14.12.<platform>.rpm, otix-8.1.4-91.2.<platform>.rpm otclx-8.4-345.2.<platform>.rpm oblt-2.4z-222.2.<platform>.rpm
for version 10.0 and 10.1 for version 10.2
Where <platform> is the system on which you will install the rpm.
D

H H H H H H blt-2.4z-343.13.x86_64.rpm tclx-8.4-470.22.x86_64.rpm tix-8.4.3-1.17.x86_64.rpm tcl-32bit-8.5.5-2.81.x86_64.rpm tcl-8.5.5-2.81.x86_64.rpm tk-32bit-8.5.5-3.12.x86_64.rpm
58
www.ibm.com

H " D tk-8.5.5-3.12.x86_64.rpm
To discover installed packages: Type: rpm -q -a |egrep "tcl|tix|tk"

Table 6.1: Softek Replicator Directory Structure on Red Hat Linux
Directory
/etc/opt/SFTKdtc /opt/SFTKdtc/bin /opt/SFTKdtc/ /etc/init.d /etc/rc0.d /etc/rc1.d /etc/rc2.d /etc/rc3.d /etc/rc4.d /etc/rc5.d /etc/rc6.d /usr/sbin /usr/local/bin /etc/opt/SFTKdtc/driver /var/opt/SFTKdtc
Contains... Licensing files, configuration files, template rc scripts User Executables Supporting libraries Startup/Shutdown scripts
Symbolic links to User Executables Softek Replicator modules

dtcerror.log,
Table 6.2: Softek Replicator Directory Structure on SuSE Linux
Directory
/etc/opt/SFTKdtc /opt/SFTKdtc/bin /opt/SFTKdtc/ /etc/init.d /etc/rc0.d /etc/rc1.d /etc/rc2.d /etc/rc3.d /etc/rc4.d /etc/rc5.d /etc/rc6.d /usr/sbin
Contains... Licensing files, configuration files, template rc scripts User Executables Supporting libraries Startup/Shutdown scripts
www.ibm.com
59
Installing Softek Replicator on Linux Table 6.2: Softek Replicator Directory Structure on SuSE Linux (Continued)
Directory
/usr/local/bin /etc/opt/SFTKdtc/driver /var/opt/SFTKdtc
Contains... Symbolic links to User Executables Softek Replicator modules

dtcerror.log,

Use the following procedure to install Softek Replicator on Red Hat Linux systems (open systems and mainframe) or SuSE Systems using the rpm command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To install Softek Replicator on Linux:
1. Login as root user. 2. Load and mount the Softek Replicator CD-ROM: 3. Change to the following directory:
H H /32-bit /64-bit D
for 32-bit platforms for 64-bit platforms
On AS4: /<location>/ Where <location> can be one of the following:

H H H H /media/cdrom /media/cdrecorder /media/dvd /media/dvdrecorder
4. Type the following command:

rpm -ivh Replicator*.rpm
5. Add /opt/SFTKdtc/bin to the PATH environment variable. 6. Type the license key into the /etc/opt/SFTKdtc/DTC.lic file on each Mobility Server to activate Softek Replicator. For more information, refer to Entering the License Keys on page 101.
60
www.ibm.com
Upgrading Softek Replicator on Linux

NOTE: A copy of the configuration files, license files and checkpoint scripts is automatically saved in the directory /var/opt/SFTKdtc when uninstalling Softek Replicator. The configuration information can then be imported into the new version of Softek Replicator by running the dtcconfigtool command after installation.
"
To upgrade Softek Replicator on Linux:
1. Use the /bin/mount command to display all devices that are installed. Make sure that no dtc device is currently installed or in use. 2. Uninstall all dtc devices and remove any relevant entries from /etc/fstab. 3. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 4. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 5. Type: /sbin/rmmod sftkdtc to remove the driver.
NOTE: If the driver is in use, the rmmod command will fail.
6. Type rpm -e Replicator* to uninstall Softek Replicator.

NOTE: This command preserves configuration and license files in a directory under /var/opt/SFTKdtc.
7. Install the new version of Softek Replicator as described in Installing Softek Replicator on Linux on page 60. 8. Repeat the procedure on each Mobility Server in the environment.
www.ibm.com
61
Removing Softek Replicator on Linux
Removing Softek Replicator on Linux

Use the following procedure to uninstall Softek Replicator on a Linux system using the rpm command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To remove Softek Replicator on Linux:
1. Uninstall all dtc devices and remove any relevant entries from /etc/fstab. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 4. Type rpm -e Replicator* to uninstall Softek Replicator. 5. Type the following commands to completely delete cached configuration files, checkpoint shell scripts, license files, and empty directories:
62
www.ibm.com
Chapter 7
Installing Softek Replicator on Solaris

Solaris System Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65 Softek Replicator Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66 Installing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66 Upgrading Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67 Removing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
Solaris System Requirements

D
Supported versions on UltraSPARC systems:

H H H H
Version 7 32-bit and 64-bit Version 8 32-bit and 64-bit Version 9 32-bit and 64-bit Version 10 64-bit
NOTE: Softek Replicator does NOT support Solaris 2/x86 (Intel platform).
D
Supported file systems:

H H H
UFS VxFS ZFS
D D D
Minimum of t 14.5 MB of disk space must be available in /opt on both Primary and Secondary Servers, with 500 KB available in /etc/opt and at least 1 MB available in /var/opt. An X-windows environment to use dtcperftool, dtcconfigtool, or dtcmonitortool. Minimum of 32 MB physical memory to be allocated exclusively for the BAB on the Primary Server.
H
Supported volume managers: VERITAS Volume Manager 4.0 (VxVM)
www.ibm.com
65

Table 7.1: Softek Replicator Directory Structure on Solaris
Directory
/etc/opt/SFTKdtc /opt/SFTKdtc/bin /opt/SFTKdtc/lib /opt/SFTKdtc/samples/chlink /etc/init.d /etc/rc3.d /etc/rcS.d /usr/sbin /usr/local/bin /opt/SFTKdtc/doc /var/opt/SFTKdtc
Contains... Licensing files, configuration files, template rc scripts User Executables Support libraries Sample script for changing symbolic links Startup/Shutdown scripts
Symbolic links to User Executables User Manuals

dtcerror.log,

All Softek Replicator components are installed on both primary and secondary systems - in the directories designated by the package. Softek Replicator is installed with the Solaris pkgadd command.
"
To install Softek Replicator on Solaris:
1. Login as root user. 2. Load and mount the Softek Replicator CD-ROM.
NOTE: If the /usr/sbin/VOLD process is running, the CD-ROM will mount automatically.
3. Type the following command:

pkgadd -d /<CD-ROM_mount_point>/Softek/Replicator/solaris/ <Solaris_Version>/SFTKdtc.2.6.5.0.pkg
Where <Solaris_Version> is one of the following:

D D D D 7 8 9 10
4. From the list of packages, type 1 to install the Softek Replicator package (SFTKdtc Replicator).
66
5. Type y to accept the license agreement and continue with installation. 6. Accept the default port number, 575; Otherwise, type another available port number. 7. After the system processes the package, type y to install <SFTKdtc>. 8. Type q to exit the installation. 9. Add /opt/SFTKdtc/bin to the PATH environment variable. 10. Type the license key into the /etc/opt/SFTKdtc/DTC.lic file on each Mobility Server to activate Softek Replicator. For more information, refer to Entering the License Keys on page 101. 11. Repeat these steps on each system with Softek Replicator.
Upgrading Softek Replicator on Solaris

NOTE: A copy of the configuration files, license files and checkpoint scripts is automatically saved in the directory /var/opt/SFTKdtc/ when uninstalling Softek Replicator. The configuration information can then be imported into the new version of Softek Replicator by running the dtcconfigtool command after installation.
"
To upgrade Softek Replicator on Solaris:
1. Unmount all dtc devices and remove any relevant entries from /etc/vfstab. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes.
NOTE: To preserve tunable parameter settings, copy the files from /etc/opt/SFTKdtc to a temporary directory.
4. Type: /usr/sbin/rem_drv dtc to remove the driver.

NOTE: If the driver is in use, the rem_drv command fails.
5. Type pkgrm SFTKdtc to uninstall Softek Replicator.

NOTE: If you are upgrading from Softek Replicator version 2.1, remove the SFTKdua Agent package prior to uninstalling Softek Replicator 2.1. To remove the Agent package, type pkgrm SFTKdua
67
www.ibm.com
Removing Softek Replicator on Solaris
All configuration and license files are saved in:

/var/opt/SFTKdtc/SFTKdtc<Version>
Where <Version> is the previous version of Softek Replicator in the format x.x.x. 6. To install the upgrade, follow the steps described in Installing Softek Replicator on Solaris on page 66. 7. Type dtcconfigtool to run the Configuration Tool after upgrading to replicate the configuration files from /var/opt/SFTKdtc directory. To ensure that you replicate the correct files, you should remove unwanted directories from /var/opt/SFTKdtc before upgrading. 8. When the Configuration Tool prompts you to replicate your configuration files, select Yes.
9. Repeat the procedure on each Mobility Server in the environment. 10. Add /opt/SFTKdtc/bin to the PATH environment variable.
Removing Softek Replicator on Solaris

Use the following procedure to uninstall Softek Replicator on a Linux system using the pkgrm command. If you plan on upgrading the operating system, you must uninstall Softek Replicator and then reinstall Softek Replicator after performing the upgrade.
"
To remove Softek Replicator:
1. Unmount all dtc devices and remove any relevant entries from /etc/vfstab. 2. On the Primary Server, type the following commands: a. killpmds -a to stop the PMDs. b. dtcstop -a to stop all the Mobility Groups. 3. On all Mobility Servers, type killdtcmaster to shut down all network daemon processes. 4. Type: /usr/sbin/rem_drv dtc to remove the dtc driver.
NOTE: If the driver is in use, the rem_drv command will fail.
5. Type pkgrm SFTKdtc to uninstall Softek Replicator. 6. Type the following commands to completely delete cached configuration files, checkpoint shell scripts, license files, and empty directories:
68
www.ibm.com
Part II
Using Softek Replicator for UNIX
Chapter 8: Planning for Softek Replicator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71 Chapter 9: Complex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87 Chapter 10: Getting Started With Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99 Chapter 11: Online Volume Expansion of dtc Devices. . . . . . . . . . . . . . . . . . . . . . . . .135 Chapter 12: Replicating Data on ZFS File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . .141 Chapter 13: Using Softek Replicator in a Cluster Environment . . . . . . . . . . . . . . . . . .149 Chapter 14: Replication in a Virtual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . .165 Chapter 15: Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .169 Chapter 16: Using the Monitoring Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .201 Chapter 17: Administrative Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211
Chapter 8
Planning for Softek Replicator

Preparing the Mobility Server for Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . .73 Considerations for Softek Replicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73 Downtime Required . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 Planning Considerations for Using the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 Sizing the Journal File Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77 Security Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78 Network Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80
This section describes the steps in preparing the system for Softek Replicator.
Preparing the Mobility Server for Softek Replicator

You should fully back up the system before creating any new partitions or installing Softek Replicator. A backup enables you to recover the original files if needed. If there are irreplacable files on the system, it is imperative that you perform a backup before proceeding to the installation. Be sure to verify the target volume, where the disk sectors will be overwritten.
Considerations for Softek Replicator

Before you install Softek Replicator, you should take into account the following items when sizing and allocating resources:
D D
Target volumes must be at least as large as source volumes. In symmetric environments, they must be identically sized. Use tools like iostat (or volstat and sar on Solaris) or commercial systems management performance tools to profile the I/O characteristics and demands of the system. Peak I/O update activity over time. Do not ignore quarter-end or year-end processing requirements. Ratio of reads to writes. Number of disk blocks updated over a measurable amount of time. This data is especially useful when determining the appropriate size for the journaling file system on the secondary system(s), and the amount of memory for the BAB. For large servers with unpredictable I/O loads; An environment where there is frequent activity in creating large files; or unreliable networks:
H H
Look for:
D D D
Identify and isolate volumes that contain only temporary data and that do not need to be replicated. Use compression on slow networks.
Factor in the maximum rate for data transfer across the network and how long you can tolerate a network outage without a BAB overflow. A BAB overflow causes Softek Replicator to move into Tracking state and requires a Refresh to synchronize the Primary and Secondary systems. If possible, perform an iterative implementation of Softek Replicator and monitor operations. Pay attention to incidents when network connection is lost, when the secondary system goes down, or during unanticipated bursts in I/O activity.
www.ibm.com
73
Downtime Required
Downtime Required
If you plan carefully, a single reboot will only be required to allocate memory for the BAB, unless there is sufficient memory. In addition, an application re-cycle is required to change the mount points to the Softek Replicator virtual devices. If the replication is planned carefully, then only a single outage is required. A second outage is required to switch over to new devices and to remove Softek Replicator from the system. To achieve this in a single outage, careful planning is required. The mount table must be reset to the new devices and Softek Replicator must be uninstalled before the reboot.
Planning Considerations for Using the Network

This section gives you some sizing numbers and approaches to figure out how much of a network link you will need in order to replicate data in a period of time. There are other factors to consider, aside from the network. The end-to-end response time is a factor of the following:
D D D
Speed of the disk at each end The speed of the entry and exit points from the server to the network The network in between, as well as the workloads running as both servers
Bottlenecks at any point will affect the end-to-end response time. Another thing that can slow down the process is the amount of updates that occur on the source system.
Configuration Example
The following configuration describes the replication of an Oracle database in an HP-UX environment. The slice of data to replicate is 1GB, and is fully populated with Oracle databases and control files. This scenario assumes that there is no activity against this disk. The data is replicated from an HP-UX 11.31 system to another HP-UX 11.31 system. To get network sizing numbers, you can simulate the bandwidth by using the Softek Replicator network control parameter, NETMAXKBPS. This parameter limits the amount of bandwidth that can be used. You may vary the tests by 1024k increments to give a measure of what one could expect for the initial copies with networks of various sizes. Server 1
D D D D
HP L1000B Server 2 PAs 8500 440 Mhz CPU, 1.5 GB RAM 2 disks of 18 GB each, Internal DVD, Dual port Ultra 2 SCSI Interface PCI 100 BaseT LAN Adapter, HP UX 11.31 running 64-bit mode HP L1000B Server 1 PA 8500 440 Mhz CPU (NOTE 1 CPU), 1GB RAM
www.ibm.com
Server 2
D D
74
D D
2 disks of 18 GB each, Internal DVD, Dual port Ultra 2 SCSI Interface PCI 100 BaseT LAN Adapter, HP UX 11.31 running 64-bit mode
Table 8.1: Network Numbers for Sending 1 GB of Oracle Data
Network Limit Per Second 700 k 1024 k 2048 k 3072 k 4096 k no limit
Elapsed Time With Compression 5 min. 50 sec 5 min. 30 sec 3 min. 20 sec 2 min. 50 sec 2 min. 40 sec 2 min. 30 sec
Elapsed Time Without Compression 6 min. 30 sec 6 min. 30 sec 5 min. 40 sec 6 min. 41sec 6 min. 29 sec 1 min. 50 sec
Replicating Data with FTP

The following lists what is possible with FTP over the same network:
D D D D D D D D
83,888,128 bytes received in 7.69 seconds (10,655.09 Kilobytes/second) 10,487,808 bytes received in 0.90 seconds (11,373.64 Kilobytes/second) 5,244,928 bytes received in 0.47 seconds (10,857.40 Kilobytes/second) 26,935,296 bytes received in 2.37 seconds (11,112.51 Kilobytes/second) 20,973,568 bytes received in 1.87 seconds (10,969.46 Kilobytes/second) 183,502,848 bytes received in 16.42 seconds (10,910.95 Kilobytes/second) 10,487,808 bytes received in 0.93 seconds (10,987.97 Kilobytes/second) 10,487,808 bytes received in 0.90 seconds (11,357.64 Kilobytes/second)
Actual Example
Configuration
D D D
WAN Speed: 52K-67K Shared T1 LAN connection: 100T Ethernet Servers: HP-UX 11.31
The Primary Server is a HP K450 running HP-UX 11.31 with internal storage that consists of mirrored SCSI Quantum XP34361WD 4095 and SEAGATE ST34572WC 4095 drives. Data is replicated from Server 1 on HP-UX, through a 100t NIC, over a 52k-67k WAN, through another 100t NIC to Server 2 on HP-UX. The amount of data transferred is 4.91 GB (including application data Sybase databases and control files):
D
Compression ON Elapsed time: 2 hours and 40 minutes

www.ibm.com
75
Planning Considerations for Using the Network
Compression OFF Elapsed time: 8.5 hours

Figure 8.1: Network Compression Throttled at 1 MB per second
Network Speeds and Elapsed times for Replication

A single replication can use about 70 percent of the bandwidth. If a Mobility Server has multiple Mobility Groups copying data at the same time, Softek Replicator will use 100 percent of the network that is available. If there is sufficient CPU capacity, you can enable compression to decrease the time of the replication for slower network links. If the network is fully utilized, it will take a long time to copy the data. You can workarounds this by performing the replication at off-peak times. Due to protocol overhead, the speed that Softek Replicator sends the data will be slow. This number is about 80 percent of the theoretical maximum. The following table shows theoretical maximum speeds of various network types:
Table 8.2: Network Speed for Replication
Connection Type 384K DSL T1 10 base T Ethernet T3 100 Base T Ethernet OC3 OC12 Gigabit Ethernet
Bytes Per Minute 2,949,120 11,520,000 75,000,000 345,600,000 750,000,000 1,162,500,000 4,500,000,000 7,500,000,000
MB Per Minute 2.8125 10.98 71.52 329.589 715.255 1,108.64 4,291.53 7,152.55
MB per Hour 168.75 658.18 4,291.53 19,775 42,915 66,518 257,492 429,153
GB per Hour 0.1647 0.6437 4.19 19.31 41.90 65.95 251.45 419.09
Replication Speed (GB/h) 0.1318 0.5149 3.35 15.45 33.52 51.96 201.16 335.27
76
www.ibm.com
Planning for Softek Replicator Table 8.2: Network Speed for Replication
Connection Type OC48 OC192
Bytes Per Minute 18,000,000,000 72,000,000,000
MB Per Minute 17,166.13 68,664.52
MB per Hour 1,029,968 4,119,872
GB per Hour 1,005.82 4,023.28
Replication Speed (GB/h) 804.66 3,218.64
Sizing the Journal File Directories

Journaling on the Secondary Server allows incoherent data to be transferred over the network without affecting the coherence or recoverability of the data on the target volumes. Journaling also allows you to checkpoint (temporarily stop) target volumes on the Secondary Server, while safely accumulating new updates from the Primary Server until after the Checkpoint has been released. At that time, the journaled updates are applied to the appropriate target volumes. This process ensures that the data on the target volumes is known during the Checkpoint operation, and allows other applications access to the target volumes without interrupting operations on the Primary Server or requiring a Refresh. The secondary journals are used only when only when data is written directly to the target volume(s)during a Smart Refresh, when blocks of changed data are written. The journal file directory should be large enough to handle all changed data from a Smart Refresh of one or more Mobility Groups, and to accommodate the updates that occur during the Checkpoint. Knowing the rate of changes made to data on the Primary Server is also useful in estimating the amount of data written to journal files during a Checkpoint. For example, the Checkpoint allows read-only access to the target volumes so that you can do a backup. You can determine the time to complete a backup by simply dividing the total amount of data to be backed up by the transfer speed of the selected backup device. Then, you can determine the average amount of data changed during that time period and make accommodations for data bursts. The journal file directory should be large enough to accommodate this amount of data. The size of the journal file directory should be 25% of the entire amount of data stored on all source volumes in a Mobility Group. For example, if a Mobility Group has three dtc devices totalling 12 GB, then the size of the journal file directory should be 3 GB (25% of 12 GB) for this Mobility Group. During configuration, you can define a journal file directory for all Mobility Groups or for each one.
www.ibm.com
77
Security Overview
Security Overview
Softek Replicator is a network-based volume replication product for AIX, HP-UX, Linux, and Solaris operating systems that enables data exchange and synchronization in support of local applications, data sharing among remote sites, and disaster recovery. Softek Replicator is designed to replicate live disk-based data from devices on a Primary Server to disk devices on a Secondary Server, across any available network connection supporting TCP/ IP. Data is duplicated in near-real-time to ensure integrity between the two systems in the event of hardware failure, natural disaster, or human intervention. Softek Replicator accomplishes this through time-sequenced transfers of data from the Primary Server to the Secondary Server over the network. The replicated copy can be made available to applications on the secondary at any point in time by means of the checkpointing mechanism. For example, this allows replicated data to be used for creating backups, doing data analysis, or for seeding other applications such as Web servers. In no circumstances is it possible to corrupt or alter in any way the source data using Softek Replicator components or commands. Softek Replicator can be set up in a secure environment if the correct components are controlled. This document deals with the security aspects of setting up replication configurations using Softek Replicator on all platforms.
D D D
P-S link: The TCP/IP connection between the Primary and Secondary Servers. Carries data and commands. R-C link: The TCP/IP connection between the Mobility Server and the DMC Collector. C-C link: The Softek Data Mobility Console connection to the DMC Collector.
Figure 8.2: Link Architecture
78
www.ibm.com
Typically the Primary and Secondary Servers are different and separate servers connected over a TCP/IP network, either a Local Area Network (LAN) or Wide Area Network (WAN). The DMC Collector and Softek Data Mobility Console are typically on the same Windows server, and are connected to all Primary and Secondary Servers by means of a TCP/IP network. In the case where a UNIX configuration is managed by Softek Replicator, a local simplified GUI is provided on each Mobility Server, both Primary and Secondary. In addition, Softek Replicator may be operated and managed through a command line interface on all platforms. For extremely sensitive data, disk encryption or network encryption methods should be employed using external hardware encryption devices.
User Access
The security access for installation, configuration, and the ability to enter command on any Mobility Server requires root access on Unix, and Administrator privileges on Windows. If the DMC Collector and Softek Data Mobility Console is used, the user of the Softek Data Mobility Console effectively has root access to all associated Mobility Servers for the purposes of controlling and managing Softek Replicator. There is no general ability to do anything other than manage and control Softek Replicator, including the ability to deploy Softek Replicatorspecific scripts. The IP address of the DMC Collector has to be specified in the Mobility Server by a user with root access. The Mobility Server therefore grants permission to the DMC Collector to connect. User access to the DMC Collector is protected in several ways: 1. In general the security control mechanism for the Softek Data Mobility Console is secured by Windows Domain Security. Any user who has the ability to connect to the DMC Collector must have Windows administrator privileges. 2. In addition, a logon is required in the Softek Data Mobility Console for a user to connect to the DMC Collector. The user ID and password requires that a valid user ID and Password exists on the DMC Collector system's SQL Server 2005 Express Database. This user ID must have administrator privileges. The SQL Server 2005 Express option has some benefits in that it allows for three levels of users:
H H H
Superuser: Can create other users Administrator: Can manage Softek Replicator Users: Can view Softek Replicator status and performance
3. Permission for the DMC Collector to be configured to the Mobility Server requires root access to the Mobility Server. This can be configured to only allow Softek Replicator-specific commands. Access to the DMC Collector must be carefully managed. For further details on user access, refer to the Softek Data Mobility Console 1.1.x Installation and User Guide (DMC-W11IU).
www.ibm.com
79
Network Environment
Network Environment
In general, Softek Replicator should be operated in a private secure network. If you install Softek Replicator across public networks, including the Internet, additional precautions should be taken to ensure security of protected information. There are several key network connections that will be dealt with from a security aspect. These links carry different types of data and information as follows:
D
Primary to Secondary Mobility Server (P-S link): Sends blocks of disk data, as well as command and control information. Primary or Secondary Mobility Server to the Collector (R-C link): Sends command and control information, and provides server state and performance statistics. Collector to the Softek Data Mobility Console (C-C link): Sends information, server state and performance statistics, and command and control information.
None of the information in any of these links is currently encrypted. The command, control, and state and command information is sent across the network in encoded binary form. The format of these blocks is not published and not generally available and so would not be easily decoded by the casual user. A determined user who has access to a Softek Data Mobility Console and DMC Collector and is able to sniff the network link between either the Softek Data Mobility Console and DMC Collector, or the DMC Collector and a Mobility Server could potentially decode some of the messages by repeatedly entering commands and analysing the control blocks. This unlikely scenario can be prevented in various ways (see User Access on page 79 and Risk Assessment on page 80).
Risk Assessment
The risks associated with unauthorized access to information on the previous links can be summarized as follows:
D
P-S link: Access to private information held on disk, and the ability to cause corruption of
data on the secondary system.
R-C link, and C-C link: Ability to enter commands to control the replication process, and reconfigure Mobility Servers.
P-S Link
The primary-secondary server link carries data from the source volumes to the target volumes as well as certain control information to start and stop daemons, divert writes to the Journal files and other process commands. Creating a replication configuration on a Primary Server in effect is allowing the secondary server to overwrite a logical volume on the secondary server. For this reason a manual distribution of certain information to the secondary server is requiredthis information acts as a private key. The creation of a configuration is controlled by the following required process:
80
www.ibm.com
A configuration is created and saved on the Primary Serverthis requires root access. A p###.cfg file is created which contains details of Primary Server, Secondary Server, and all associated replication source and target devices. This configuration is identified by the Mobility Group number (### portion of the configuration file). Before any replication can take place to the Secondary Server, this p###.cfg file must be copied over to the Secondary Server, placed in the /etc/SFTKdtc directory, and renamed to s###.cfg. This process requires root access on the secondary server and is equivalent to the distribution of a private key. A connection between the Primary and Secondary Server cannot take place without an identical s###.cfg file existing on the Secondary Server. Any modification to either the p###.cfg file or the s###.cfg file will prevent the connection from being established. The p### and s### files represent an agreement between the Primary and Secondary Server as to which source volumes are to be replicated, to where they are to be replicated, and what network addresses are to be used to communicate between the two servers. In the configuration files (p###, s###), an IP address and port number are specified. These are used to communicate between the two servers. The port number defaults to 575, but can be changed at the time the configuration is created on the primary server. The IP address relates to a specific communication interface. Both of these are specified at the Mobility Group level and may be different for different Mobility Groups on the same servers.
Data Flow Encryption

The blocks of data (off the source volumes) that are sent across the network are not encrypted. Softek investigated the possibility of encryption and the impact of doing so was too severe. If the data is absolutely required to be encrypted, then it should be encrypted on the source volume, or by using a hardware encryption network solution. Softek Replicator does provide a compression mechanism, which, although using standard compression mechanisms, would render the data unreadable. This has an impact on CPU usage. In any event, the nature of the data flow between primary and secondary would make it virtually impossible to reconstruct the data in any meaningful manner. The blocks of data that are sent across the network are not related to any actual disk construct (block, sector, cylinder), but are accumulated into a user-specifiable size. The only occasion that data is sent across the network in any kind of ordered manner is during a Full Refresh (initial synchronization of the source and target disks). If there is any I/O to the Primary System during this process, the disk image received at the secondary server after the Full Refresh will be corrupt and unreadable. It will only reach a synchronized state after the twophase process of Full Refresh and re-synchronization. Also, many disks are usually interleaved in multiple Mobility Groups, so re-constructing a full disk image is virtually impossible. The Full Refresh may be performed without using the network if necessary (courier transport method); thus, a full disk image could never be constructed from the network traffic. After the Full Refresh, the chunks that are sent across the network contain essentially a random ordering of blocks on the source volume, based on not just data updates, but also on updates to logical volume structures and file system structures. The format of the chunks are proprietary and cannot easily be decoded. They also contain embedded control information and CRC checks.
www.ibm.com
81
Network Environment
R-C and C-C Links

These links carry configuration files, scripts, as well as Mobility Server commands and performance statistics. In general, commands and statistics are carried in encoded format, and scripts and configuration files are transmitted in clear text. These links should be protected and not configured through firewalls or over public networks because they are not encrypted. The connection is established by specifying the IP address and port; refer to Port Usage on page 82. The Mobility Server gives the authority for the Collector to connect to it. Specifying an IP address and port on the Mobility Server requires root access. It also means that a rogue Collector cannot be connected to any of the Mobility Servers without root access to the Mobility Server. Once this link is activated, any users on the Softek Data Mobility Console or the connection between the Collector and Mobility Server effectively has root access to the Mobility Server. This is required because many of the Softek Replicator commands require root authority. In addition, new configuration files that are created through the Softek Data Mobility Console are automatically distributed to both the primary Mobility Server (p###.cfg) and the secondary Mobility Server (s###.cfg). For this reason, this link should be behind a firewall, and user access to the Softek Data Mobility Console and DMC Collector should be secured. If this link is opened through a firewall it should be encrypted using network encryption hardware.
Port Usage
By default, Softek Replicator listens on the following ports:
D D D D
Port 575 for data flow between UNIX Mobility Servers. Port 16386 to communicate with the Collector (instead of Port 576 on Windows). The Collector uses uses Port 16642 to communicate with Softek Replicator on UNIX machines (as opposed to 577 to communicate with Windows machines). Port 16130 is used to communicate with Windows Mobility Servers.
The connection between the Softek Data Mobility Console and the DMC Collector uses port 576 by default on Windows, as well as an ODBC connection on port 1035 on which Microsoft SQL Server 2005 Express listens. For replication through a firewall, these ports must be opened. The Secondary Server never initiates a connection on its own, but always responds to Primary Server connections. Replication through a firewall is not normally recommended since it creates another point of access for data, from either a Mobility Server outside the firewall to get data behind the firewall, or it transports potentially private data from behind a firewall to a Mobility Server outside the secure network.
82
www.ibm.com
Data Flow
Many applications use TCP/IP sockets programming to connect and send data across a TCP/IP network. The sockets programming construct allows for two programs to bind to each other across a network and communicate. A negotiation process is completed during the initiation of communication that sets up the conversation tone. In general, a sockets program binds to a socket, reads data from a source disk or memory location, and sends this information to a remote location via the local socket. The remote program reads this information from its end of the socket connection. When the Read is completed, this causes an acknowledgement to be sent to the Send via TCP/IP.
Softek Replicator Data Flow

The steps that Softek Replicator follows in its data flow are described here. The sockets programs in Softek Replicator are the PMD (Primary Mirror Daemon) and RMD (Remote Mirror Daemon): 1. The PMD reads a block of data from memory or disk depending on the mode. 2. The PMD writes the block of data read in the previous step to the local socket file descriptor. 3. On the RMD, the remote socket reads the block of data. 4. The RMD writes the data and sends acknowledgement that the data is committed on the remote site.
MTU (Maximum Transmission Unit)

TCP/IP segments data into packets to be sent across the network. This segmentation is done based on the MTU (Maximum Transmission Unit). This MTU is controlled by the physical network interface. Conventional 100T Ethernet has an MTU of 1500 bytes. It is especially important in long-distance replication to ensure that the largest possible MTU is set, because this makes it possible to send more data in a single packet by increasing the possible throughput of the network connection. The MTU will negotiate down to the smallest MTU in the total path. It is recommended to do a trace route command and ensure that the network path that is being taken is the optimal path, and that the maximum MTU for each network transport type is set.
TCP Window
Also referred to as windowing, TCP Window is a method of flow control for transferring data over networks. TCP sends data across a network in packets and requires the receiving device to send an acknowledgement, or ACK, when it receives the packet, which signals the sending device that the packet was successfully transmitted. The window size is the total size of data packets that can be sent without waiting for an ACK. With the sliding window method, the receiving device can send a single acknowledgement message for multiple packets of data sent in one window. Within that acknowledgement message is information about the receiving device's buffer size, which tells the sending device to increase or decrease the number of packets in the next transmission (this is where the sliding in the name comes in). If the application reading the data processes the packets at a slower rate than the sending device is transmitting them, it will tell the sending device to decrease the number of packets or temporarily cease transmission altogether, in order to free up room in the buffer. If the receiving application can process the packets faster than the sending device is transmitting them, it will tell the sending device to
83
Network Environment
increase the number of packets in the next window as the application's buffer can handle more data. The window size is a very important variable because it limits the amount of data that is buffered from the application level to the TCP protocol for a particular transfer. This involves both the sender and the receiver buffer. The TCP Window is the network buffer between the application and the network stack. This should be set large enough to hold enough application data so that TCP/IP is never waiting for more data to be sent. This varies by application and requires detailed information about the application to set properly. Most operating systems have system-wide settings that default to 32 K. This default setting is too low for long-distance replication. More reasonable sizes are 128 K or 256 K. If this is set too big, there will be no throughput or performance payback and will only consume more memory.
NOTE: For optimal performance, both sides of the TCP connection should have the same window size.
RFC1323
RFC1323 is the TCP/IP Request For Comment that supports window scaling or TCP Window Sizes above 64 k. This is normally set to OFF on most operating systems. This option should be set to ON for long distance replication. Having larger windows allows for more buffering, and potentially more data to be sent over a long-distance link.
SACK
SACK is Selective ACKnowledgement as defined in RFC2018. This should be set to ON for longdistance replication. Most operating systems have this set to OFF by default. This setting helps during long-distance data transfer by only acknowledging packets that needed to be retransmitted instead of the resending and acknowledging the complete segment. As an example, if you send a shipment with 20 items in it through UPS and only 18 items showed up with selective acknowledgement, you would only have to resend the missing 2.
Compression
In general, compression will help with long-distance replication by reducing the number of packets sent because of the compression. Softek Replicator has compression included that can be dynamically toggled on and off. The penalty for using compression is increased CPU consumption. Compression doubles the processor usage.
Chunksize
Softek Replicator has a tunable parameter called CHUNKSIZE that affects many things within the product. The default setting is 2,048 K. The CHUNKSIZE setting determines how big the read is during the Full Refresh phase. It is also used to size the TCP application buffer.
84
www.ibm.com
Determining the Quality of the Network

You can have the largest bandwidth network in the world but it will be worthless if it has poor quality. So how does one gauge the quality of a network? Normally the retransmission or error rate of network shows the overall quality. A long-distance network can expect to drop 10 percent of the transmission. If the network performs better than this, the quality is good. One can use the netstat -in command at intervals to determine the network quality. The way to maximize the throughput with Softek Replicator is to always keep the data flowing at all stages in the process. The data should be read from disk with the smallest CHUNKSIZE possible. This keeps the PMD continually reading data. This data flows to the TCP socket via the write from the PMD. The TCP window size should be between 128k and 256k to allow continuous data flow from the sender process (PMD). The receiver process (RMD) should have a matching TCP Receive Window between 128-256k. There is nothing currently in the Softek Replicator process to optimize the write process. The TCP/IP parameters SACK should be turned on and the RFC1323 for large window size should be turned on. For bandwidthconstrained environments, network throttling and compression should be evaluated. These options can be dynamically changed in the UNIX environment to evaluate their effect. We recommend that you follow the guidelines mentioned below for optimal network performance:
D D D D
Ensure that the NIC cards are set in full duplex mode. Validate your MTU from start point to end point throughout your network route. You can only go as fast as your smallest MTU. Ensure that you are taking the best route in the network possible; use the tracetroute command to verify this. Set the Softek Replicator CHUNKSIZE parameter as small as possible. For complete details, refer to dtcset on page 192. Use Iperf to see which TCPWINDOWSIZE is best for your network. Use this setting with Softek Replicator. For more information, refer to the following Internet resource: http://dast.nlanr.net/Projects/Iperf/ Turn on the COMPRESSION with Softek Replicator on slow links. When testing, make sure that your test runs send data for at least 15 to 30 minutes. Longrunning data transfers will experience network problems in the infrastructure. Break your long-distance replication into many smaller Mobility Groups for better operational control. Use hardware snapshot technologies for application testing or remote copies of data. Turn on SACK (use the ndd command for HP-UX and Solaris, and no for AIX). Turn on RFC1323 large windows (use the ndd command for HP-UX and Solaris, and no for AIX). Become familiar with the platform packet sniffing software to look at the data streams to ensure that your window sizes are being honored. This also gives you a picture as to how clean the network is. Look at the network error statistics using the netstat -in / or -en options. Run ping commands with a packet size just under the maximum MTU, and check the network latency at different times of the day. Try to segregate the long-distance replication activity away from other traffic if possible.
D D D D D D D
D D D
www.ibm.com
85
Chapter 9
Complex Configurations
Symmetric Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89 One-to-many Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93 Chaining Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95 Loopback Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97
This chapter explains the processes involved to configure Softek Replicator in environments that are slightly more complex than a simple Primary Server to a Secondary Server configuration.
Symmetric Configuration
A symmetric configuration is appropriate for an enterprise in which an automated switch-over is desired for load-balancing or other planned events. Two-way mirroring is not achieved in this manner. Rather, data is being mirrored from the Primary Server (A) to the Secondary Server (B). The difference being that the Secondary Server is symmetrically configured and ready to assume control of operations if application focus is deliberately moved to it. After this type of switchover, the Secondary Server assumes the role of Primary Server, and the previous Primary Server becomes the Secondary Server.
CAUTION: If you are using Dynamic Activation on Solaris or if you want replicate a ZFS file system, you cannot use a symmetric configuration.
When creating this type of configuration, it is best to plan out the Mobility Groups and dtc devices for both systems prior to installation. The key to this configuration is having the mirror device on the Secondary Server be a dtc device rather than a raw disk or volume. There is, of course, a mirror device for each local data device on the Primary Server, and each of these mirror devices should be a dtc device on the Secondary Server. Basically that means establishing a BAB and Pstore on both the Primary and Secondary Servers in a symmetric configuration and using dtc device names for the mirror devices rather than dsk or rdsk conventions. You should also establish the Secondary Servers mirror devices so they are the same as the Primary Servers local data device. Thus, when a switch-over occurs, Secondary Server is mirroring to the appropriate device and the data on both systems remains coherent and up to date. The following figure describes the relationship between devices in a symmetric configuration. If you are managing data replication using the Softek Data Mobility Console, for more information, refer to the Softek Data Mobility Console 1.1.x Installation and User Guide (DMCW11IU).
Figure 9.3: Sample Symmetric Configuration
www.ibm.com
89
As you can see in figure: 9.3: Sample Symmetric Configuration on page 89, System A (Primary Server) mirrors to System B (Secondary Server) and System B mirrors to System A.
NOTE: The partitions used for the dtc devices must be identical in size.
Defining a Symmetric Configuration

When defining a symmetric configuration, you need to configure System B before configuring System A.
On System B
1. Determine the system configuration. 2. Establish the Mobility Group(s) and dtc device numbers for System B. An effective methodology for defining this kind of symmetric configuration is to bias the Mobility Group numbering by 500. The following table gives a sample configuration.
Table 9.3: Sample Symmetric Definitions
Primary Server (A) Mobility Group 1 dtc device 0 (dtc0) local data device:
/dev/rdsk/c1t2d0s1
Secondary Server (B) Mobility Group 501 dtc device 0 (dtc0) local data device: /dev/rdsk/c1t3d0s1 mirror data device (same as local device on System A):
/dev/rdsk/c1t2d0s1
mirror data device (same as the dtc device path for Mobility Group 501 on System B):
/dev/dtc/lg501/rdsk/dtc0
3. Run dtcconfigtool on System B to configure Mobility Group 501 as described in the Sample Symmetric Definitions table. Allocate the BAB and Pstore, and set up the Mobility Group(s) and dtc devices. Keep in mind that devices defined as mirror devices on System B need to be the same device name as the local data device on System A. For clarification, refer to figure: 9.3: Sample Symmetric Configuration on page 89. 4. Commit devices, save the Mobility Group, then exit dtcconfigtool. 5. Copy configuration files over to System A and rename. 6. Verify that dtc devices are not mounted automatically when the system is booted. 7. Type dtcstart -g 501 to start the Mobility Group. 8. Run dtcinfo -g 501 to verify that the Mobility Group is in Passthru mode.
NOTE: The Mobility Group(s) on System B MUST be running in Passthru mode to allow A to B mirroring.
90
www.ibm.com
On System A
1. Run dtcconfigtool to configure Mobility Group 1 as described in Sample Symmetric Definitions on page 90. 2. Allocate the BAB and Pstore. 3. Define Mobility Groups and dtc devices for the primary on A as in Sample Symmetric Definitions on page 90.
NOTE: Remember that mirror devices for Mobility Group 1 should be dtc device names for Mobility Group 501. All started dtc devices are displayed in the drop-down menu in dtcconfigtool, and can be selected appropriately as mirror devices.
4. Commit Devices and exit. 5. Copy configuration files over to System B and rename. 6. Verify that dtc devices are not mounted automatically when the system is booted. 7. Type dtcstart -g 1 on the Primary Server. 8. Softek Replicator is operating in Passthru mode at this point. A refresh operation or a manual operation is required to transition into Normal state. This initializes the mirror for Mobility Group 1.
Running B as Stand-Alone
1. On System B, type: dtcrmdreco -g 1 To verify that Mobility Group 1 is in recovery mode, check that the s001.off file is in the /etc/opt/SFTKdtc on B.
NOTE: This step is necessary to prepare the mirror devices to be accessed as data devices. The dtcrmdreco command flushes all outstanding entries to disk and prohibits further mirroring to these devices. For AIX Only Replace /etc/opt/SFTKdtc with /etc/dtc/lib.
2. On System A, type: dtcstop -a 3. On System B, place Mobility Group 501 into Tracking state by entering:
dtcoverride -g 501 state tracking
4. fsck the Mobility Group 501 dtc devices that had file systems mounted. 5. mount your file systems and start applications on System Bs dtc devices.
NOTE: Placing B into Tracking state reduces the length of time required for a refresh operation when A returns to service.
www.ibm.com
91
Inverting the A-B Topology to B-A

1. On System A: When System A boots, Mobility Group 1 (and any other Mobility Group defined on this system) is started in Passthru mode. Type a dtcstop -a to stop all Mobility Groups on System A. The -a option inhibits these Mobility Groups from being restarted on System A after each reboot. 2. On System B: launchrefresh -g 501. If System B was put into Tracking state as illustrated above, this refresh should be very efficient since it requires only changed data be transferred to System A.
NOTE: Applications may continue to update dtc devices in Mobility Group 501 during the refresh.
3. Continue running the applications on System B until it is desirable to make System A the primary once again.
Switching Back from B to A

1. Halt the applications on System B and force all pending updates to disk (for example, with the sync command). Unmount any file systems on the dtc devices in Mobility Group 501. 2. Verify that all updates have been mirrored to System A. 3. On System A, type dtcrmdreco -g 501. Then, verify that the s501.off file is in the /etc/opt/SFTKdtc directory.
NOTE: For AIX ONLY Replace /etc/opt/SFTKdtc with /etc/dtc/lib.
4. On System B, type: dtcoverride -g 501 state passthru 5. Type the dtcrmdreco -g 1 -d command on B to enable mirroring to the dtc devices in Mobility Group 501 on System B in an A-B topology. 6. Verify that the s001.off file is NOT in the /etc/opt/SFTKdtc directory on System B. 7. On System A, start Mobility Group 1 with dtcstart -g 1 8. On System A, type: dtcoverride -g 1 state normal 9. On System A, mount file systems and start applications.
92
www.ibm.com
One-to-many Configuration
This configuration allows the mirroring of the same data set to two or more Secondary Servers. The following figure shows the process for establishing a configuration where System A (Primary Server) mirrors to System B (Secondary Server) while simultaneously mirroring the same data to System C (Secondary Server).
CAUTION: If you are using Dynamic Activation on Solaris or if you want replicate a ZFS file system, you cannot use a One-to-many configuration.
Figure 9.4: Sample One-To-Many Configuration
The figure: 9.4: Sample One-To-Many Configuration shows that System A is simultaneously mirroring the same data set to System B and System C.
Table 9.4: Sample One-To-Many Definitions
Primary Server (A) Mobility Group 0 dtc device 0 (dtc0) Mobility Group 1 dtc device 1 (dtc1) This dtc device maps to dtc0 as the local data device and to the raw device on System C. local data device: mirror data device (on System C)
local data device: mirror data device (on System B)
www.ibm.com
93
One-to-many Configuration
"
To set up a One-to-Many Configuration:
1. Plan the devices to be used on all systems. 2. Run dtcconfigtool to allocate BAB on the System A (Primary Server). Create Mobility Group 0 which specifies System A as the Primary Server and System B as the Secondary Server. 3. Define dtc 0. 4. Commit devices, and Exit from the File menu. 5. Copy the p000.cfg file to B and rename to s000.cfg. 6. Start Mobility Group 0 on System A with the dtcstart -g 0 command. 7. Use dtcconfigtool to create a new Mobility Group 1, with System A as the primary and System C as the secondary). 8. From the drop-down list, select Mobility Group 0s dtc 0 as the local data device.
NOTE: All dtc devices that have been started are visible and can be selected from the drop down menu in dtcconfigtool.
9. Commit the device, and Exit from the File menu. Save changes when prompted. 10. Copy the p001.cfg file to C and rename to s001.cfg. 11. Type dtcstart -g 1. 12. Launch a refresh on A using the launchrefresh command. 13. Mount file systems and have applications only interact with dtc devices defined in Mobility Group 1. This provides mirroring to both B and C.
94
www.ibm.com
Chaining Configuration
Chaining refers to a configuration where systems appear to be connected in a sequential manner, such as A, B, and C where Systems A and C have no connection or awareness of the other. The following figure shows the process for setting up a configuration in which System A mirrors to System B, which, in turn, mirrors to System C.
Figure 9.5: Sample Chaining Configuration
The figure: 9.5: Sample Chaining Configuration shows a configuration where System A is the Primary Server, mirroring to System B, which is the Secondary Server, and System B is the Primary Server, mirroring to System C which is the Secondary Server. The B to C component of this configuration is similar to a simple A to B setup with two minor variations. You should do the B to C part of the setup first.
NOTE: When you define Sync mode in chain structure, please set the tunable parameter, SYNCMODETIMEOUT, bigger for A to B than for B to C.
Table 9.5: Sample Chaining Definitions
Primary Server (A) Mobility Group 0 dtc device 0 (dtc0, dtc1...) local data device: mirror data device (on System B):
Secondary Server (B) Mobility Group 100 dtc device 0 (dtc0) local data device: mirror data device (on System C):
www.ibm.com
95
Chaining Configuration Table 9.5: Sample Chaining Definitions (Continued)
Primary Server (A) Mobility Group 0 The mirror data device maps to the dtc device(s) defined for Mobility Group 100.
Secondary Server (B) Mobility Group 100
B to C Setup
1. Run dtcconfigtool and define Mobility Group 100 (instead of defaulting to Mobility Group 0) with System B as primary and System C as secondary. 2. Define dtc devices (dtc0, dtc1, etc.) as normal. 3. Copy the configuration file /etc/opt/SFTKdtc/p100.cfg to /etc/opt/SFTKdtc/ s100.cfg on System C.
NOTE: Replace /etc/opt/SFTKdtc with /etc/dtc/lib.
4. Use the dtcstart -g 100 command to start the Mobility Group. 5. Use the dtcoverride -g 100 state normal command to bypass the refresh operation and place the Mobility Group into Normal state. 6. Execute launchpmds to establish a network connection between B and C.
CAUTION: Do not mount any file systems or have applications interact with the dtc devices defined in Mobility Group 100 on System B.
A to B Setup
1. Use dtcconfigtool to create a configuration for Mobility Group 0 that has System A as primary and System B as secondary. 2. Note that in the Secondary area, there is a box for turning chaining on or off. Select on (the default is off). 3. Define the dtc devices (dtc0, dtc1) but in the drop-down menu for the mirror devices, select the dtc device as defined for Mobility Group 100 on System B. For example:
/dev/dtc/lg100/rdsk/dtc0
Repeat steps 1 to 3 for each dtc device in the Mobility Group. 4. Commit devices. 5. From the File menu, Save changes and Exit the Configuration tool. 6. Copy the configuration file /etc/opt/SFTKdtc/p000.cfg to /etc/opt/SFTKdtc/ s000.cfg on System B.
NOTE: For AIX Only Replace /etc/opt/SFTKdtc with /etc/dtc/lib.
96
7. Run dtcstart -g 0 on System A to start the Mobility Group. 8. Type a launchrefresh -g 0 -f on System A. At this point the chaining topology is active and in effect. To verify this, during the refresh operation, run dtcmonitortool on System B to watch the refresh updates from System A arrive as input to the dtc devices defined in Mobility Group 100.
Loopback Configuration
Softek Replicator supports a loopback configuration, that is, a configuration in which both Primary and Secondary Servers are on the same Server. This can be useful for a variety of needs, such as testing or employing checkpointing to create point-in-time snapshots of data sets.
"
To define a loopback environment:
1. Launch dtcconfigtool. 2. Under the Systems tab, specify the same host name or IP address for both Primary and Secondary Servers. 3. Define dtc devices as usual.
www.ibm.com
97
Chapter 10
Getting Started With Replication

Entering the License Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101 Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102 Step 1: Creating Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104 Step 2: Distributing the Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110 Step 3: Configuring the File Systems for Replication . . . . . . . . . . . . . . . . . . . . . . . . .113 Step 4: Starting the Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126 Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .132
10
This chapter provides information on how to configure and start Softek Replicator.
Entering the License Keys

To run Softek Replicator, you must obtain a license key24 alphanumeric charactersfor each Server using Softek Replicator. In order to obtain a new license for Softek Replicator, you will need to acquire the Host ID of the system on which you have installed Softek Replicator. The Host ID is an 8-digit hexadecimal number that is tied to the physical server hardware. The Host ID is calculated using the Media Access Control (MAC) address of the primary LAN card. You can obtain licenses from a Softek Replicator reseller or support organization.
Obtaining the Host ID

Use the following procedure to obtain the Host ID of a Server.
"
To obtain the Host ID on UNIX: Type dtchostinfo
1. From the shell prompt:

H
NOTE: The dtclicinfo command will also provide detailed information about existing Softek Replicator license on the system.
or
H
Use one of the following OS specific command:

D
On AIX: uname
-m
CAUTION: Do not use the AIX hostid command.

D D
On HP-UX: uname
-i
On Linux and Solaris: hostid
2. Copy the returned value into the license application form available from the main Softek Replicator Support page at:
http://www-950.ibm.com/services/dms/cgi-bin/tdmf_RNewOse.pl?C01&prod=tdmf_unixip
We will send a new license to you within one business day.
www.ibm.com
101
Process Overview
Entering License Information

After you have installed Softek Replicator, follow these steps to establish key files and enter the appropriate license numbers. A single key typed in the DTC.lic file on each server allows the operation of all Softek Replicator daemons. 1. On all Mobility Servers, create the DTC.lic file in the /etc/opt/SFTKdtc directory. For AIX create the DTC.lic file in the /etc/dtc/lib directory. 2. Type the 24 alphanumeric characters that make up the license key associated with the Mobility Server Host ID into the DTC.lic file.
NOTE: The product license is tied to the host ID of the Mobility Server. Therefore, you need a license key for each Mobility Server on which you installed Softek Replicator. You do not need a new license key when you upgrade.
Process Overview
This section describes the general steps for any replication on UNIX systems. The following scenario involves a simple server-to-server replication in a local area network (LAN). For wide area networks (WAN), the steps will be similar, except that you must take network details into account. Long distance replication is sensitive to network latency and quality of the network. When planning for long distance data replication or permanent disaster recovery replication, care should be taken when sizing the network pipe to ensure that the bandwidth that is procured is as high a quality as possible.
General Configuration Steps

1. Size the BAB using the dtcagentset command if you are using the Softek Data Mobility Console, or the X-based dtcconfigtool. UNIX environments start at 128 MB for small servers, and 256 MB for larger servers. 2. Create one Pstore volume, allowing 200 KB per Mobility Group. This is a raw volume with no file system, with a size of a few hundred megabytes. 3. Install, configure, and format new target storage. 4. Create the Mobility Groups. Group the volumes associated with the application to be replicated.
102
www.ibm.com
10
General Replication Steps

1. For online replication, change the mount tables with the dtcautomount command, and then mount the dtc devices. For offline replication, applications and file systems are unmounted. 2. Bring down applications that need to be replicated. Unmount file systems. 3. Start the Mobility Groups. 4. For online replication, mount the file systems to the dtc devices. 5. Schedule the Full Refresh processes. 6. When the Full Refresh processes finish, schedule another application downtime to cut to the new disk. 7. Change the mount tables to point to the target disk. 8. Bring up the applications using the new storage. 9. Uninstall Softek Replicator.
Replication Steps Breakdown

To prepare for data replication, you must perform the following procedures, in the specified order:
D
Step 1: Creating Mobility Groups :

H H
If you are using the UNIX Command Prompt, go to Step 1: Creating Mobility Groups on
page 104
i)
If you are using Softek Data Mobility Console, you must perform the following tasks: Set up the Data Mobility Agent on the UNIX Mobility Servers. Refer to Configuring the Data Mobility Agent.
ii) Create the Mobility Groups. Refer to the Softek Data Mobility Console 1.1.x Installation and User Guide (DMC-W11IU). iii) Go to Step 3: Configuring the File Systems for Replication.
D
Step 2: Distributing the Configuration Files

H
Perform this step if you are using the UNIX Command Prompt, and if you are replicating data on AIX, HP-UX, or Linux systems. If you are using the Softek Data Mobility Console, the configuration files will automatically be distributed.
Step 3: Configuring the File Systems for Replication

H H
For AIX, refer to Configuring File Systems on AIX on page 113. For HP-UX, Linux, and Solaris, refer to Configuring File Systems on HP-UX, Linux, and Solaris on page 123.
Step 4: Starting the Replication.
www.ibm.com
103
Step 1: Creating Mobility Groups

On the Primary Server, use the dtcconfigtool command to create and manage the Mobility Groups. A Mobility Group is a collection of local partitions or devices treated as a single unit by Softek Replicator. Each Mobility Group operates with its own independent PMD/RMD daemon pair. Mobility Groups allow time-ordered writing coherence between member dtc devices, and complete operational and state independence. Once you have started the dtcconfigtool, Softek Replicator automatically creates a Mobility Group 0 and selects it as the initial group to edit.
NOTE: When starting the dtcconfigtool for the first time, you need to allocate memory for the BAB.
"
To create a Mobility Group:
1. Type: dtcconfigtool 2. Click Ok to dismiss the informational message about defining the BAB and display the Configuration Tool dialog box. 3. Type a value to allocate memory for the BAB.
Figure 10.6: Defining the BAB
NOTE: AIX 32-bit kernel The maximum kernel memory size is 512 MB, out of which a maximum of 256 MB may be allocated to the BAB. Allocating more than 256 MB may cause you to encounter insufficient memory errors when you attempt to start a new process. If you encounter issues with kernel memory, You should lower the BAB size.
4. From the File menu, select New Mobility Group. 5. Use the arrows to select the Mobility Group number for editing.
104
www.ibm.com
Getting Started With Replication Figure 10.7: dtcconfigtool - Systems Tab (AIX, HP-UX, Linux)
10
6. Select the Systems Tab.

H
In the Primary System section, the primary system Hostname or IP Address is filled in automatically with the name or IP Address of the Primary Server.
7. Type or select the name of the Persistent Store Device (Pstore) for the Mobility Group.
NOTE: You can define a unique Pstore for each Mobility Group. Make sure that the Pstore partition is clean and available.
8. On AIX, HP-UX, or Linux systems, set Autostart to Yes to automatically start the Mobility Group after rebooting the server. If this option is enabled, all I/Os are tracked at boot time, when you start a Mobility Group.
NOTE: dtc devices with Autostart set to off (No) will NOT be automatically started after a system reboot. The default setting for Autostart is Yes.
9. On Solaris, select Yes to turn on Dynamic Activation.
www.ibm.com
105
Step 1: Creating Mobility Groups Figure 10.8: dtcconfigtool - Systems Tab (Solaris Only)
10. In the Secondary Server section, type the resolvable system name or IP address for the Secondary Server.
TIP: Keep in mind that each Mobility Group has an independent connection; therefore, you can define a different Secondary Server for each Mobility Group.
11. In Journal Directory, specify any writable directory on the Secondary Server for journal files.
NOTE: The default setting for the Journal feature is .
After you complete the configuration, make sure you specify this directory in the system so that the configuration file is installed automatically at boot up. For AIX, see Creating and Mounting New Journaled File Systems on dtc Devices on page 116. For HP-UX, Linux, and Solaris, see Modifying the system mounting file (/etc/fstab or /etc/vfstab) on page 123. 12. Type the Port number to use in the Port field or use the default of 575 on the Secondary Server. This is the port you are connecting to on the Secondary Server. To change this port number, see Changing Port Numbers on page 227. You can also change the port that the Softek Replicator master daemon listens on as described in Changing Port Numbers on page 227.
NOTE: You can define Mobility Groups that connect to different Secondary Servers with different port numbers. Use this field to specify those secondary port numbers.
106
www.ibm.com
10
13. If needed, change the Allow Chaining option (No is the default) for each Secondary Server defined. For more information on chaining, refer to Chaining Configuration on page 95.
CAUTION: For Solaris Only Softek Replicator will generate errors during the replication process if you are using Solaris Solstice Disk Suite (SDS) on the Secondary Server, with partitions starting at block 0 of cylinder 0. The SDS partitions must start at cylinder 1, or greater. This only affects the target devices and is a standard Solstice Disk Suite recommendation. Alternatively, with Solaris 9 or higher, you could use the soft partition from Solstice, which resolves this problem and has additional features to overcome constraints with hard partitioning.
14. Select the dtc Devices tab.

Figure 10.9: dtcconfigtool - dtc Devices
15. In the dtc Device text box, select a dtc name or type in a name for the dtc device using the dtc# naming convention. Where # is the device number.
NOTE: Device naming starts with dtc0 by default. Dtc device names are automatically incremented by one when creating a new device, but you can change these names. You are not required to name dtc devices sequentially or start device names with 0. Dtc device names must be unique only within a specific Mobility Group. Multiple Mobility Groups can contain identical dtc device names.
16. (AIX Only) Set the Match Minor Numbers to ON for AIX JFS2 file system or a JFS2 external log device. For more information, refer to the Configuring File Systems on AIX on page 113. 17. In the On Primary System section, select a data device from the Data Device drop-down list box. The Data Device drop-down list box displays all volumes and started dtc devices. 18. In the On Secondary System section, select a data device from the Mirror Device dropdown list box.
NOTE: Each mirror device must be at least as large as its corresponding local data device.
www.ibm.com
107
19. Click Commit Device once you have defined the local data and mirror devices. 20. Click Create New Device to add more dtc devices to the current Mobility Group. 21. (Optional) Select the Throttles tab. For more information, refer to Working with Throttles on
page 213
22. (Optional) Select the Tunable Parameters tab. For more information, refer to Working with Tunable Parameters on page 223 23. Repeat these steps to create all Mobility Groups and dtc devices 24. From the File menu, click Save Changes to save the Mobility Group definition. 25. From the File menu, click Exit to exit dtcconfigtool. 26. Remount the file system and restart applications.
NOTE: Softek Replicator does not replicate a devices physical sector 0 because doing so may overwrite the devices volume table of contents, which contains critical disk label information.
Creating a Mobility Group Using Softek Data Mobility Console

If you plan on using Softek Data Mobility Console to perform UNIX replication, you must first configure the Data Mobility Agent on the UNIX Mobility Servers before you create the UNIX Mobility Groups. After configuring the Data Mobility Agent, refer to the Softek Data Mobility Console 1.1.x Installation and User Guide (DMC-W11IU) on creating the UNIX Mobility Groups and performing replications. Once you have created the UNIX Mobility Group through Softek Data Mobility Console, you must initialize the BAB using the dtcagentset -b <bab_size_MB> command and then load the dtc driver using the launchagent command, in order to start the Mobility Group for the first time. For more information, refer to dtcagentset on page 171 andlaunchagent on page 198.
NOTE: HP-UX Servers If you experience a file table overflow error when attempting to start a Mobility Group or launch a PMD process, you may need to increase the value of the maxfiles_lim and nfile tunable parameters. These two HP-UX kernel tunable parameters determine the maximum number of files that can be opened per-process and system-wide, respectively.
108
www.ibm.com
10
Configuring the Data Mobility Agent

The Data Mobility Agent provides the bridge between the Softek Data Mobility Console and Softek Replicator Mobility Servers. The Data Mobility Agent allows you to perform UNIX replication using Softek Data Mobility Console. Although the Data Mobility Agent automatically starts at system boot time, you must activate it manually if you need to use it. The Data Mobility Agent can be managed using the following commands:
D D D dtcagentset command activates the Data launchagent command starts the Data killagent command stops the Data
Mobility Agent refer to dtcagentset on page 171.
Mobility Agent refer to launchagent on page 198.
Mobility Agent refer to killagent on page 194.
The Data Mobility Agent settings are saved in the dtcAgent.cfg configuration file. For details on its contents, refer to the -I option of the dtcagentset command on page 171.
"
To configure the Data Mobility Agent:

[-b BabSize] | [-l] [-t <TransmitInterval>]]
1. Type: dtcagentset [[-e <CollectorIP> [:CollectorPort]] [-i <AgentIP>] Where:

D D D -e CollectorIP [:CollectorPort]: -i<AgentIP>:
Activates the Data Mobility Agent and the IP address and port number of the DMC Collector. The Agent IP address that the Collector will use for communication.
-b <bab_size_MB>:
Initializes the BAB and loads the dtc driver. The bab_size is specified in Megabytes as an integer between 1 and 1547. The -b parameter takes effect for the subsequent time the dtc driver is loaded.
Displays the state of the Data Mobility Agent setup this is the default option. If no options are specified, the dtcagentset command considers that the -l option is specified. If the agent is active, the CollectorIP and the CollectorPort are displayed. If the Data Mobility Agent is not active, this command displays: Collector
-l: connection information is not set up. -t <TransmitInterval>:
Use the -t option to set up the frequency with which the agent process sends information to the collector, that is, the transmit interval between two transmissions to the Softek Data Mobility Console. For example, during a period of great activity, such as a full refresh or heavy I/O, you can increase the transmit interval. This value must equal or be greater than 4.
www.ibm.com
109

For each Mobility Group on the Primary Server, the dtcconfigtool command creates a configuration file on the Primary Server. Since each configuration files contains Mobility Group definitions for local and mirror partitions, these files must also be copied to the Secondary Server. The configuration files use the following naming convention: [p|s]<###>.cfg. Where:
H H H p s
indicate the Primary Server indicate Secondary Server indicates the Mobility Group number
<###>
The configuration file must reside on both the Primary and Secondary Servers before you begin replicating data, since each Mobility Group contains definitions for dtc devices, including mirror devices. When changing a configuration file on the Primary Server, you must copy the updated file to the Secondary Server, then stop and restart the Mobility Group. The dtcstart command processes the new configuration file, at which point changes to the file become effective. Tunable parameters are not stored in the configuration files, so you do not need to copy the files after changing these parameters. For more information on advanced configuration options, refer to Chapter 9: Complex Configurations. Softek Replicator references the Primary Server configuration files of started Mobility Groups many times during normal system operation. To ensure that all components of Softek Replicator refer to the same configuration as the dtc device driver, Softek Replicator creates a copy of the .cfg file when the Mobility Group is started. This copy is given a .cur extension since it reflects the current configuration. The .cfg files are only processed by the dtcstart command. All other commands reference the .cur files. The .cur files are deleted when a Mobility Group is stopped and restarted, at which point they reflect any modifications made to the configuration. This allows you to make changes to the configuration of any Mobility Group and then allow the changes to take effect at a specific time.
110
www.ibm.com
10
"
To copy the configuration files from the Primary Server to the Secondary Server:
1. Using ftp or rcp, copy the configuration files (p<###>.cfg), from the Primary Server to the same directory on the Secondary Server. The configuration files are located in the following directories:
H H
On AIX: /etc/dtc/lib On HP-UX, Linux, and Solaris: /etc/opt/SFTKdtc
2. Change the name of the configuration files on the Secondary Server to s<###>.cfg.
NOTE: Since all Mobility Group configuration files are located in the same directory, renaming allows each system in Softek Replicator environment to operate as both a Primary and Secondary Server without conflict.
Sample Configuration File of a Primary Mobility Group

#======================================================= # # # # Last Updated: Sat Feb 9 17:01:56 2008 Softek Replicator Configuration File: Softek Replicator Version 2.6.x.0 Build p000.cfg
#======================================================= # NOTES: # # # Primary Server Definition: # SYSTEM-TAG: HOST: PSTORE: # # Secondary Server Definition: # SYSTEM-TAG: HOST: JOURNAL: SECONDARY-PORT: CHAINING: # # Device Definitions: #
SYSTEM-A 127.0.0.1 /dev/dsk/c2t0d0s4
PRIMARY
SYSTEM-B 127.0.0.1 /export/home/journal 575 off
SECONDARY
111

PROFILE: 1 REMARK: PRIMARY: DTC-DEVICE: DATA-DISK: SECONDARY: MIRROR-DISK: # # -- End of Softek Replicator Configuration File: p000.cfg loop back 1 SYSTEM-A /dev/dtc/lg0/rdsk/dtc0 /dev/rdsk/c2t0d0s0 SYSTEM-B /dev/rdsk/c2t0d0s1
Configuring the PStore

If you plan to replicate volumes that are larger than 300 GB, you need to configure the Persistent Store (Pstore) to support large high-resolution tracking (HRT) bitmaps. This functionality provides greater bit resolution and better performance and efficiency of smart refreshes for larger volumes. The PStore volumes require approximately 12 MB per dtc device so the Pstore volume should be sized appropriately.
" D
To configure the Pstore: Type: dtcinit -p <Pstore_device> The dtcinit command will write 0 to the specified device.
NOTE: If the specified Pstore device is used by a operating Mobility Groups, or the specified Pstore device is not defined in the configuration file, the initialization process for the Pstore will not be processed and the dtcinit command will terminate with an error.
" D
To enable large bitmap support: Type: dtcinit -l.
NOTE: Small bitmap support can be re-enabled by typing: dtcinit -s. The dtcinit command re-initializes the Pstore and cannot be used while Mobility Groups are started. Using the command will require a subsequent full refresh (or checksum refresh). For more information on dtcinit, refer to dtcinit on page 179.
Adding Other Devices for Replication

You can use the devlist.conf configuration file to add other (uncommon) devices for replication. You can add directory paths for devices that you want to replicate. For complete details on this configuration file, refer to User-Defined Devices on page 247.
112
www.ibm.com
10

This section describes how to initially synchronize the Primary and Secondary Servers in a file system environment. Databases and other applications can work directly with raw disk devices rather than using file systems for data storage. These application configurations must be changed to the new Softek Replicator devices rather than the local data devices. Creating devices does not modify the original contents of the local data devices. It is appropriate to define dtc devices on top of preexisting data. No export or import of data is required.
Configuring File Systems on AIX

NOTE: To set up the AIX environment for Disaster Recovery purposes, please refer to Using Softek Replicator in a Disaster Recovery AIX Environment on page 293.
Journal File Systems (JFS) require a log device for journaling changes to the file system metadata. Journal file systems can either share a single log device, or each can use a separate log device for optimal performance. For more information on organizing JFS logs for optimal performance, please refer to the AIX Performance Management Guide.
NOTE: If the shared log device is used for replication, the file systems that are NOT replicated are unable to use that log device.
"
To determine if the file systems on LVM volumes are JFS or JFS2: Where <path> is the path to the device.
1. Type: /usr/sbin/lslv <path>
File Systems Replication on AIX

The AIX operating system provides the JFS and JFS2 journaled file systems for storage of file data on block devices. These file systems provide fast recovery after a system crash by journaling meta-data operations associated with file system transactions, which can later be replayed and applied to the file systems in order to restore them to a consistent state. By default, JFS and JFS2 rely on external log devices. JFS2 also provides an option to use an internal log device; this allows both a file system and its log to reside within the same logical device.
File System Expansion Device Naming on AIX

To start a Mobility Group with the dtcstart command, two forms of device nodes are created that refer to the same devices. The standard dtc device path format is:
D D /dev/dtc/lg<X>/dsk/dtc<Y> for a block device /dev/dtc/lg<X>/rdsk/dtc<Y> for a character device
where <X> and <Y> are any numeric values. The short form device name appears under /dev and appears in a form similar to AIX LVM volume names:
113

/dev/lg<X>dtc<Y> for a block device /dev/rlg<X<dtc<Y> for a character device
D D
where <X> and <Y> are any numeric values.

NOTE: In order to support online JFS expansion of dtc devices, the short form device name must be mounted and must also appear in /etc/filesystems.
Replicating JFS/JFS2 File Systems with External Logs

When replicating a JFS or JFS2 file system with external log volumes, the volume containing the file system and its associated log volume must both be replicated to the Secondary Server. This allows the logs to be replayed on the Secondary Server in the event of a disaster or fail over. All log volumes that are replicated must only contain transactions that apply to replicated volumes.
NOTE: Log devices cannot be shared between replicated and non-replicated volumes.
If multiple file systems are associated with a particular log volume then they all must be replicated. If you do not want to replicate all file systems associated with a particular log, you need to create a new log volume and assign it the volumes to be replicated, leaving the original log for the volumes that are not being replicated. We recommend that you have one log per Mobility Group. This will maintain chronological write consistency between the log and its associated file system volumes, which is necessary in a disaster recovery environment. Once the new logs are created, use the logform command to format the new logs, and the chfs command to associate the various logs with the appropriate file systems in /etc/filesystems. The dtcautomount command will then further modify the /etc/filesystems entries to properly insert the Softek Replicator devices as data and log devices. Lastly, unmount any file systems whose log specification has been changed, as well as any volumes to be replicated, and then remount all file systems after the Mobility Groups have been started.
114
www.ibm.com
10
Matching Device Minor Numbers

JFS and JFS2 logs contain references to device minor numbers for each file system volume associated with the log device. These device numbers allow meta-data transactions to be replayed back to the proper device in the event of a system crash. In a disaster recovery scenario, replicated log volumes on the Secondary Server will be used to replay transactions for application to secondary data devices. This process is necessary to return these volumes to a consistent state prior to mounting the file systems. Because these minor numbers are critical to the recovery process after a fail over to the secondary, the minor numbers of both the primary dtc device, and the secondary target device must match. In many cases this matching can be taken care of automatically by simply selecting 'Match Minor Numbers' in the GUI for a particular dtc device. This will cause a dtc device to be created with a device minor number that matches the target. If you are using the Softek Data Mobility Console this option is chosen through selection of Send Device Numbers. The device number specification will appear similar to the following in the Mobility Group configuration files:
MIRROR-DISK: /dev/rdest_log MIRROR-DEVNO: 1 40
Target Minor Number Collisions

In some cases, targets may be spread out over multiple volume groups that use different major numbers, allowing more than one target to have the same minor number. AIX does not support a native command that will allow the user to specify a minor number at volume creation time, so Softek Replicator includes an auxiliary command, dtcmklv, that allows volumes to be created with a particular minor number using the -p option. The following is an example of two target volume groups with a single volume in each group:
VG Name = oracle Major Number 45 Logical volume Name oracledb Minor Number 1 VG Name = sybase Major Number 46 Logical volume Name sybasedb Minor Number 1 # ls -l /dev/roracledb crw-rw---- 1 root system 45, 1 Dec 28 09:24 /dev/roracledb # ls -l /dev/rsybasedb crwxr-xr-x 1 root system 46, 1 Dec 28 08:33 /dev/rsybasedb
As you can see, since both volume groups in the previous example have a logical volume with a minor number of 1. The above problem can be corrected by using the dtcmklv command, for modifying the target logical volume. To correct this configuration, you would need to change the logical volume sybasedb to have a minor number other than 1. It is recommended to assign a range of minor numbers to each volume group so that there are no minor number collisions. The following is an example of using the dtcmklv command to set the minor number to 100.
# dtcmklv -t'jfs' -y'sybasedb' -p 100 rootvg 16 sybasedb # ls -l /dev/sybasedb brw-rw---- 1 root system 10,100 Dec 29 08:53 /dev/sybasedb
The following message during the dtcstart command processing indicates that you have minor device number collisions:
115

# dtcstart -g 1 DTC: [WARNING / MINORMISMATCH]: Minor numbers between dtc device /dev/dtc/ lg1/rdsk/dtc0 [17] and target device /dev/rlv01 [16] do not match
JFS/JFS2 Logs for Data Replications

It is not necessary to replicate the log volumes if you are doing a data replication, as long as the file systems are cleanly unmounted at the end of the data replication, before cutover. If the JFS log is not replicated, it will be necessary to ensure that the new JFS log for the target volume is formatted for use. Use AIX logform command to format a new log volume. Also, ensure that the new /etc/filesystems entry is updated appropriately to point to the new JFS Log on both the source and target systems. This can be done by using the AIX chfs command. For Data Replications, a single Mobility Group can be created, containing all JFS logs. This Mobility Group is started and left in PassThru mode since there is no need to copy the JFS logs during a replication, if the JFS file systems are cleanly unmounted at the end of the replication, before cutover. The logs must be in a Mobility Group and used as dtc devices, even though they are not being replicated, to satisfy the AIX requirement that log files must be in the same volume group as their data volumes. All dtc devices appear to be a single AIX volume group.
Creating and Mounting New Journaled File Systems on dtc Devices

The device that contains the file system and the associated log device must be replicated in the same Mobility Group. For this reason, it is expected that a new journal log will be created and associated with the file system to be replicated as described in the procedure that follows. Before configuring the dtc Mobility Group, create a new JFS log volume. The default size is 4 MB or 1 PP. The logform command must be applied to the native device. The chfs command should be performed so that the file system is correctly associated with the new JFS log for use if Softek Replicator is not started.
"
To mount a new JFS file system on a dtc device:

/dev/source_log
1. Type: mklv -y'source_log' -t jfslog -L'source_log' <rootvg> 1 2. Type: logform 4. Type: chfs -a 3. Type Y at the following prompt: logform: destroy /dev/source_log (y)?
log=/dev/source_log /source_data
Configure a new Mobility Group, specifying the newly created log device as the local device for dtc0, and the existing JFS file system as the local device for dtc1.
CAUTION: When you use JFS journal file system to comprise dtc devices, you need to be aware of the following: In AIX, JFS volumes and their log device must be on the same volume group. If JFS volumes and their log device for dtc devices are created on different volume groups, you will be unable to mount the file system on the secondary server even when Checkpoint is on. For more information on AIX file system replication, refer to File Systems Replication on AIX on page 113.
116
www.ibm.com
10
Creating and Mounting New JFS2 File Systems on dtc Devices

The log device is used to maintain a consistent file system structure, and must be replicated using a dtc device. The device that contains the file system and the log device must be replicated in the same Mobility Group. JFS2 also stores the LVM device numbers in their internal disk structures. You need to make sure the mirror device and the dtc device use the same minor device numbers. Otherwise, there could be data loss because the metadata on the mirrored log may not be applied on the mirrored data disk. The dtc device driver looks like a single AIX volume group. You need to make sure that your minor device numbers do not overlap between AIX volume groups. The dtcmklv command allows you to set the minor device numbers for AIX logical volumes. The minor device numbers for an AIX volume group can be either between 0 and 255, or 0 and 511, based upon the setting of the large volume support option. It is recommended you track the minor device numbers for all your mirror devices to make sure they do not overlap. If you have three AIX volume groups, each with four devices, then one scenario would be that the first group would use minor device numbers 0-3, the next group would use 4-7 and the last group would use 8-11.
"
To mount a new JFS2 file system on a dtc device:

<rootvg> 1
1. Type: dtcmklv -p minor_number -y'source_log' -t jfs2log -L'source_log' 2. Type: logform /dev/source_log 3. Type Y at the following prompt: logform: destroy /dev/source_log (y)? 4. Type: chfs -a log=/dev/source_log /source_data
Determining the Logical Volume Type

You can either edit the /etc/filesystems file or use the lsfs command to list the contents of the /etc/filesystems file. Refer to the possible Logical Volume combinations on these pages:
D D D D
defaultvfs is JFS and JFS Will Be Used on page 118 defaultvfs is JFS and JFS2 Will Be Used on page 118 defaultvfs is JFS2 and JFS Will Be Used on page 119 defaultvfs is JFS2 and JFS2 Will Be Used on page 120
www.ibm.com
117
defaultvfs is JFS and JFS Will Be Used

When defaultvfs is JFS, and JFS will be used: 1. To format a device for use as a JFS log, type:logform /dev/dtc/lg0/dsk/<dtc0> 2. Type: mkfs -V jfs -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/rdsk/<dtc1> 3. When mkfs has completed, create a mount point if one does not already exist:
mkdir /dtctest
4. Mount the new file system:

mount -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/dsk/<dtc1> /dtctest
5. To configure the file system to mount automatically at boot time, simply add the following entry to the /etc/filesystems file:
/dtctest: dev=/dev/dtc/lg0/dsk/<dtc1> vfs=jfs log=/dev/dtc/lg0/dsk/<dtc0> mount=true options=rw account=false
defaultvfs is JFS and JFS2 Will Be Used

You must record the mirror device minor device number. If you need to recreate the mirror device using dtcmklv to set the minor device number you want to use. Use the ls -l command to list the minor device number.
D
When defaultvfs is JFS, but JFS2 will be used, and the log device will be used as an EXTERNAL log:
logform -V jfs2 /dev/dtc/lg0/dsk/<dtc0>
1. To format a device for use as an external JFS2 log, type: 2. Type:

mkfs -V jfs2 -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/rdsk/<dtc1>
3. When mkfs has completed, create a mount point if one does not already exist:
mkdir /dtctest
4. Mount the new file system as follows:

mount -v jfs2 -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/dsk/<dtc1> / dtctest
118
www.ibm.com
10
/dtctest: dev =/dev/dtc/lg0/dsk/<dtc1> vfs =jfs2 log =/dev/dtc/lg0/dsk/<dtc0> mount =true options =rw account =false D
When defaultvfs is JFS but JFS2 will be used, and the log device will be used as an INTERNAL log:
mkfs -V jfs2 -o log=INLINE /dev/dtc/lg0/rdsk/<dtc0>
1. Type for use as an internal JFS2 log: 2. When mkfs has completed, create a mount point if one does not already exist as follows:
mkdir /dtctest

mount -v jfs2 -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/dsk/<dtc0> / dtctest
/dtctest: dev =/dev/dtc/lg0/dsk/<dtc0> vfs =jfs2 log =/dev/dtc/lg0/dsk/<dtc0> mount =true options =rw account =false
defaultvfs is JFS2 and JFS Will Be Used

When defaultvfs is JFS2 but JFS will be used: 1. To format a device for use as an external JFS2 log, type:
logform -V jfs /dev/dtc/lg0/dsk/<dtc0>
2. Type:
mkfs -V jfs -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/rdsk/<dtc1>
mkdir /dtctest

mount -v jfs -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/dsk/<dtc1> /dtctest
www.ibm.com
119
/dtctest: dev =/dev/dtc/lg0/dsk/<dtc1> vfs =jfs log =/dev/dtc/lg0/dsk/<dtc0> mount =true options =rw account =false
defaultvfs is JFS2 and JFS2 Will Be Used

Record the mirror device minor device number. If you need to recreate the mirror device using dtcmklv to set the minor device number you want to use. Use the ls -l command to list the minor device number
D
When defaultvfs is JFS2 and JFS2 will be used, and the log device will be used as an EXTERNAL log:
logform /dev/dtc/lg0/dsk/<dtc0>
1. To format a device for use as a external JFS2 log, type: 2. Type:

mkfs -V jfs2 -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/rdsk/<dtc1>
mkdir /dtctest

/dtctest: dev =/dev/dtc/lg0/dsk/<dtc1> vfs =jfs2 log =/dev/dtc/lg0/dsk/<dtc0> mount =true options =rw account =false D
When defaultvfs is JFS2 and JFS2 will be used, and the log device will be used as an INTERNAL log:
mkfs -V jfs2 -o log=INLINE /dev/dtc/lg0/rdsk/<dtc0>
1. Type for use as an internal JFS2 log: 2. When mkfs has completed, create a mount point if one does not already exist:
mkdir /dtctest

120
www.ibm.com
10
/dtctest: dev =/dev/dtc/lg0/dsk/<dtc0> vfs =jfs2 log =/dev/dtc/lg0/dsk/<dtc0> mount =true options =rw account =false
Alternatively, you may want to create dtc devices with previously existing logs and file systems as local data devices. In this case, you would simply need to mount the file system and modify /etc/filesystems as described in Step 4 on page 120 and Step 5 on page 120.
Mounting Existing Journaled File Systems (JFS and JFS2) with dtc Devices
If the log device will be used as an EXTERNAL log, mount the file system without modifying /etc/filesystems, as follows:
mount -v jfs|jfs2 -o log=/dev/dtc/lg0/dsk/<dtc0> /dev/dtc/lg0/dsk/<dtc1> / dtctest
If you have existing file systems, you only need to update the entries in the /etc/filesystems file.
"
To update the /etc/filesystems file:
1. Make a backup copy of the /etc/filesystems file. 2. Change the dev lines from the file system device to the associated dtc device. 3. Change all log lines that contain the name of the log device to its associated dtc device.
NOTE: You may also use the dtcautomount command to modify /etc/filesystems if the Mobility Groups are running and the old file system auto-mounts at every reboot.
JFS2 Limitations
An AIX volume group can have either 256 or 512 devices. These devices are numbered from 0 to N-1 for each AIX volume group. A volume group is defined by its major number. Softek Replicator dtc devices appear as a single volume group to the AIX system because they only have one major number. This implies that the dtc device can only handle 512 matching device numbers across all volume groups. To create a volume group that supports 512 devices, you need to use the -B parameter on the AIX mkvg command.
www.ibm.com
121
Starting Softek Replicator on AIX

Type: /usr/dtc/bin/dtcstart -a
"
To associate the dtc devices with the mountable file systems and log devices: Previous entry, without TDMFIP, in /etc/filesystems:
/source_data: dev vfs log mount options account = /dev/source_data = jfs = /dev/hd8 = true = rw = false
1. Modify /etc/filesystems:
The modified entry, specifying dtc devices for the file system and log, in /etc/filesystems:
/source_data: dev vfs log mount options account = /dev/dtc/lg0/dsk/<dtc1> = jfs = /dev/dtc/lg0/dsk/<dtc0> = true = rw = false
NOTE: You may also use the dtcautomount command to modify /etc/filesystems.
2. Mount the dtc device as your source data:

mount /source_data mount node mounted mounted over vfs date options -----------------------------------------------------------------------/dev/hd4 / jfs Dec 05 12:54 rw,log=/dev/hd8 ... /dev/dtc/lg0/dsk/<dtc1> /source_data jfs Jan 07 15:03 rw,log=/dev/dtc/lg0/dsk/ <dtc0>
To start replicating data, refer to Step 4: Starting the Replication on page 126.
122
www.ibm.com
10
Configuring File Systems on HP-UX, Linux, and Solaris

NOTE: For data replication on Solaris, this step is not required if you are using the Dynamic Activation feature. You can skip this step and go to Step 4: Starting the Replication on page 126. CAUTION: HP-UX Replication of a VxFS file system from HP-UX versions 11.00 and 11.11 to version 11.23 is supported. However, backward replication from 11.23 to 11.00 or 11.11 does NOT work. This is due to a change in the VxFS file system in version 11.23. LINUX - RAID Devices If the administrator is using mdadm.conf (multi-disk administration configuration file) to start up the RAID, or MD, devices, the administrator must start these devices BEFORE Softek Replicator is initialized. If the RAID/MD devices are not already started in the /etc/rc.d/rc.sysinit file, then the administrator must uncomment and adjust the RAID/MD startup lines, if necessary.
Starting Softek Replicator

Type: /opt/SFTKdtc/bin/dtcstart -a after installing the package, adding the license files, running the dtcconfigtool command, and distributing configuration files to the Secondary Servers. The dtcinfo -a command indicates that the dtc devices are running in PassThru mode, which is the default operating mode when Softek Replicator is first installed. After creating the initial mirror, Softek Replicator goes into Normal state.
Modifying the system mounting file (/etc/fstab or /etc/vfstab)

You must modify the /etc/fstab (for HP-UX and Linux), or /etc/vfstab (for Solaris) system mounting file so that dtc devices are mounted rather than the original local data devices if you are mirroring file systems across the network.
NOTE: You must also add an entry to this file so that your journal file directory is mounted automatically when you boot the system.
"
To modify the system mounting file:
1. Make a backup copy of the system mounting file (/etc/fstab for HP-UX and Linux /etc/vfstab for Solaris). 2. If the file system to be mounted on the dtc device was just created, simply add it to the fstab or vfstab file.
H
For example, on HP-UX and Solaris:

/dev/dtc/lg0/dsk/<dtc0> /dtctest hfs defaults 0 2
For example, on Linux, using an ext2 file system:

/dev/dtc/lg0/dsk/<dtc0> /dtctest ext2 defaults 0 2
www.ibm.com
123
3. If the file system already exists, modify the file system mounting configuration file /etc/ fstab (for HP-UX and Linux), or /etc/vfstab (for Solaris), to refer to the new dtc devices rather than the original local data devices. Example for Linux: /dev/hdc3 /datadev ext defaults 1 2 to: /dev/dtc/lg0/dsk/<dtc0> /datadev ext defaults 1 2
NOTE: You can also use the dtcautomount command to modify the fstab or vfstab file if the Mobility Groups are running and the old file system auto-mounts at every reboot.
If you need to create or overwrite a file system, follow the next three steps, otherwise skip to Step 7.
CAUTION: File systems can be created or overwritten before being mounted on dtc devices. This operation is straightforward when done on a dtc device in Async mode. Performance may be noticeably degraded when running in Sync mode.
4. For HP-UX and Solaris: Type newfs /dev/dtc/lg0/rdsk/<dtc0>

NOTE: If you are using Solaris 8, type newfs using the actual physical raw device:
newfs /dev/rdsk/c0t0d0s4.
For Linux: To write a VxFS file system through a dtc device on Linux, use the following: a. Type: blockdev --getsize /dev/dtc/lg0/dsk/dtc1 to get the number of blocks in the dtc device. b. Type: mkfs -t vxfs /dev/dtc/lg0/dsk/dtc1 <Number_blocks>,where <Number_blocks> is the number of blocks in the dtc device, to pass the number to mkfs. 5. Type: mkdir /dtctest to create a mount point if one does not already exist, after newfs| mkfs has completed. 6. Type mount /dev/dtc/lg0/dsk/<dtc0> /dtctest to mount the new file system.
NOTE: Update the /etc/fstab file (for HP-UX and Linux), or /etc/vfstab file (for Solaris) if you want the file system mounted automatically after every Primary Server reboot.
For HP-UX and Solaris: With VxFS, use the mkfs command as follows: a. Type: mkfs -F vxfs /dev/dtc/lg0/rdsk/<dtc0> b. Type: mount -F vxfs /dev/dtc/lg0/dsk/<dtc0> /dtctest
NOTE: For Solaris Only If you are using Solaris 8, type mkfs using the actual physical raw device:
mkfs -F vxfs /dev/rdsk/c0t0d0s4
124
www.ibm.com
10
Running a Mobility Group in sync mode with a UFS file system does not provide true synchronous updates of the Softek Replicator devices due to the asynchronous ordering of the disk writes by the UFS file system.
CAUTION:
CAUTION: SOLARIS 10 - UFS File System With 100% Usage Solaris version 10 mounts UFS file systems with the Logging feature enabled by default, which results in the following situations when you replicate UFS file systems with 100% usage: Replication from versions 7, 8, and 9, to version 10: When mounting the UFS filesystem on Solaris 10, you get a message stating that it cannot mount the file system in logging mode as there is no space available, and mounts it in No Logging mode instead. This is because versions 7, 8, and 9, mount the UFS file system in No Logging mode by default. Replication from version 10 to version 2.6: You must run an fsck before mounting the file system, because version 2.6 does not have the Logging feature. This issue does not cause any data loss, and applies only to Softek Replicator for Solaris 10.
When running a Mobility Group in sync mode with VxFS mounted use the mincache=dsync option to ensure that the underlying VxFS does not cache the writes but instead writes them directly to disk:
mount -F vxfs -o mincache=dsync /dev/dtc/lg0/dsk/<dtc0> /dtctest
NOTE: Performance may be noticeably degraded when running in Sync mode.
7. Type umount to unmount the file system. 8. Type mount or mountall to remount the affected file systems before changing their contents. Databases and other applications can work directly with raw disk devices rather than using file systems for data storage. These application configurations must be changed to the new Softek Replicator devices rather than the local data devices. Creating devices does not modify the original contents of the local data devices. It is appropriate to define dtc devices on top of pre-existing data. No export or import of data is required.
NOTE: Error Messages When dtc devices are described in /etc/fstab file, a mount device error will occur at system restart. However, an auto-mount process will be launched for devices that could not be mounted.
In HP-UX, if an application using a dtc device is set to auto-start at system restart, you must confirm the starting time for the application. If the application auto-start is registered in rc script before rc2.d, you must re-register it in rc script after rc2.d.
www.ibm.com
125
Step 4: Starting the Replication

Before using the dtc devices, you must make an initial copy of the local data devices to the mirror devices on the Secondary Server. Start Softek Replicator with the dtcstart command and create a new file system or start writing data to the data devices. To create the initial mirror, you can use one of two methods, using the launchrefresh command or the Courier Transport Method.
Using the launchrefresh Command

"
To create the initial mirror using the launchrefresh command: This command synchronizes all mirror devices by transferring every data block on the Primary Servers local data devices.
H
1. On the Primary Server, type: launchrefresh -f a
You can start using the dtc devices for active I/O while the Refresh synchronization is taking place, without worrying about data integrity. If either system halts during a Refresh operation, the process restarts at reboot. You can monitor the Refresh progress with the dtcinfo command, the dtcmonitortool command, or the dtcperftool performance monitoring utility.
NOTE: Data on the mirror devices is in an incoherent, non-usable state until the Refresh operation completes.
H
Once the Refresh operation is complete, Softek Replicator goes to Normal state and assumes normal operations.
CAUTION: If the system experiences significant I/O during a Full Refresh, there may be an infinite number of BAB overflows when Softek Replicator tries to go back into Normal state.
For details on the Full Refresh process, please refer to Refresh State on page 23.
Using the Courier Transport Method

The Courier Transport Method or Limited Network Initial Synchronization is a way to synchronize data remotely without sending all the data over the network. The network is used to send changes only and not all the data. Softek Replicator tracks all changes when the data is backed up to a transportable media disk or tape. This method is most effective when there is a tremendous amount of data to be synchronized and a Full Refresh operation would take too long.
NOTE: This backup must consist of physical blocks and not files.
Once the data has been backed up, you can ship the removable media to a remote site and restore the data. You can then initiate a Smart Refresh that mirrors only those blocks that have changed.
126
www.ibm.com
10
With the Panalyze tool, you can view the amount of data that has changed when Softek Replicator is in Tracking state. This is useful for calculating how long the Smart Refresh will take to complete with a particular network link.
Option 1 - Using dd
CAUTION: When using the Courier Transport Method with non-Volume Manager partitions on slice 0, if dd is used to move the data and block 0 is copied and then restored on the target disk, it will corrupt the block 0 that holds the volume information. Block 1 through the end of slice should be copied using dd or other physical copy mechanisms when using non volume manager partitions.
"
To create the initial mirror using the Courier Transport Method with dd:
The dd command is a UNIX command that copies disk blocks to tape or other disks. The dd command is a device to device copy; it could be tape to disk, disk to disk, or any combination. dd example
dd if=/dev/vg01/volume of=/dev/vg01new/volume bs=8096 dd if=/dev/dsk/c0t0d0s2 of=/dev/tape1 bs=8000
1. Clear the Pstore before creating the initial mirror: a. Type: dtcstart -a b. Type: dtcoverride -a state normal
NOTE: Ignore the Can't get Pstore... warning for non-existent Mobility Groups.
c. Type: dtcoverride -a clear LRT d. Type: dtcoverride -a clear HRT e. Type: dtckillpmd -a f.
g.
Type: dtcstop -a Type: dtcstart -a
NOTE: Make sure that the dtc device is used when mounting the file system.
2. Type: dtcoverride -a state tracking on the Primary Server to place Softek Replicator into Tracking state. At this time, you can start applications. All changes are being tracked and will eventually be copied to the Secondary Server. 3. On the Secondary Server, type: dtcrmdreco -a 4. Make a disk image backup to a transportable media (disk or tape) of the local data devices on the Primary Server. 5. Transport the media to the Secondary Server location, and restore the backup on the corresponding mirror devices. 6. On the Secondary Server, type: dtcrmdreco -a d. 7. Type: launchrefresh -a to initiate a Smart Refresh operation that goes out of Tracking state and transfers only the changed data since the time when the backup to tape was made and applied to the mirror devices.
www.ibm.com
127
Softek Replicator goes to Normal state and assumes typical operations after the data is transferred to the mirror devices.
Option 2 - Loopback Configuration

In a loopback configuration, you must change the IP address to point to the new Secondary Server. Changing the IP address when Softek Replicator is in Tracking state provides you with the opportunity to move the data transfer to a new LAN segment or to use another NIC card in case of hardware failure. The transfer operation can then continue without having to take an outage for re-configuration.
"
To synchronize data using the Courier Transport Method: a. Enable the Locking Protection by checking the option. b. Leave all settings to default, except for the Mirror Group option, which should not be used (be unchecked). c. In our example, we define the source disk as being E: and the target disk as being F: using the Softek Data Mobility Console. The drive letter F: should be the drive letter to be used as the eventual target volume letter on the Secondary Server.
Figure 10.10: Courier Transport Method Using Loopback Configuration 1
1. On the Primary Server, create a Mobility Group in a loopback configuration as follows:
2. Start the group. 3. Clear the Pstore before creating the initial mirror: a. Type: dtcoverride -g <group#> state normal
128
www.ibm.com
10
NOTE: Ignore the Can't get Pstore... warning for non-existent Mobility Groups.
b. Type: dtcoverride -g <group#> clear LRT c. Type: dtcoverride -g <group#> clear HRT 4. Launch a Full Refresh. This will copy the data to the Courier Volume locally, which will later be transported to the remote site following the local synchronization. 5. Once the Mobility Group reaches Normal mode, kill the PMD using the following command:
dtcKillpmd -u -g <group#>
6. Shutdown the Primary Server and remove the Courier drive. 7. Transport the Courier drive and install it on the Secondary Server. 8. Assign the Courier Volume a drive letter other than F: (the primary server's target drive letter).
www.ibm.com
129
In this example, we are choosing E: as the temporary drive letter to copy the transported disk to the final target disk location.
9. On the Secondary Server, create the same Mobility Group as the one that is on the Primary Server in a loopback configuration: a. Use the same group number as the group that was initially used to copy the data at the primary location. b. Enable the Locking Protection by checking the option. c. Leave all settings to default, except for the Mirror Group option, which should not be used (be unchecked). d. The source is the E: drive (on the transported disk) and the target drive is F: (on the Secondary Server disk, which will be the final location.)
10. Start the Group and launch a Full Refresh. 11. Once the Mobility Group reaches Normal mode, kill the PMD without the -u option, and stop the group:
dtcKillpmd -g <group#>
130
www.ibm.com
Getting Started With Replication Figure 10.13: Courier Transport Method Using Loopback Configuration 4
10
12. Delete the Mobility Group that was created on the Secondary Server. The lock gets preserved on the target drive so that when you refresh the group, the F: drive that is now located in the Secondary Server will be refreshed. 13. From the Softek Data Mobility Console, select the option to dynamically change the IP address on the Mobility Group to reflect the new target's IP address, and click Save. You can also change the IP address from the dtcconfigtool or by manually editing the configuration files.
A Smart Refresh is automatically launched.
www.ibm.com
131
Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS)
File Ownership
Because file ownership changes after replication, make sure that /etc/passwd and /etc/groups match between the Primary and Secondary Servers. Softek Replicator copies the file structures between the systems at block level. It copies the inode or directory information. This directory information contains a user id number and Mobility Group number. These numbers are put into readable group and user id names by the /etc/groups and /etc/passwd files. If the passwd and groups files do not match when you attempt to use the replicated copy, you will have security problems when trying to access these files. This can solved by issuing the chown command.
Many production database environments install the database on top of raw disk partitions rather than on top of file systems. These raw disk partitions may be volumes created by either VERITAS Volume Manager (VxVM) or other volume managers. Veritas When the database is created, it stores the paths to these raw disk devices as part of the database itself. This is a challenge for Softek Replicator, or for any disaster recovery scheme that needs to restore the database on another computer whose disk layout may not be identical to the disk layout of the original server. The solution is to specify a symbolic link path rather than the actual path to the raw disk or volume when creating the database. The method is illustrated in the following example. Suppose you have created a Sybase database on three raw disk devices that are actually striped volumes. The normal specification to the RDBMS software is:
TABLESPACE A: /dev/vx/rdsk/sybasedg/vol01 TABLESPACE B: /dev/vx/rdsk/sybasedg/vol02 TABLESPACE C: /dev/vx/rdsk/sybasedg/vol03
NOTE: This example can also apply to Oracle or other RDBMS packages.
132
www.ibm.com
10
Creating Symbolic Links

Use the following procedure to create a symbolic link rather than the actual path to the raw disk or volume when creating the database.
"
To create symbolic links:

mkdir /dev/devlinks ln -s /dev/vx/rdsk/sybasedg/vol01 /dev/devlinks/tblspA ln -s /dev/vx/rdsk/sybasedg/vol02 /dev/devlinks/tblspB ln -s /dev/vx/rdsk/sybasedg/vol03 /dev/devlinks/tblspC TABLESPACE A: /dev/devlinks/tblspA TABLESPACE B: /dev/devlinks/tblspB TABLESPACE C: /dev/devlinks/tblspC
1. Create symbolic link definitions to these volumes and give those paths to the database.
NOTE: This example can also apply to Oracle or other RDBMS packages.
2. Create and populate the database as you would normally. 3. Shut down the database. 4. Install Softek Replicator and configure it to mirror these volumes; change the symbolic links to point to the created dtc devices instead of the original raw disks or volumes. 5. Create dtc devices, but do not start the PMD. 6. Create the symbolic links:
rm ln rm ln rm ln /dev/devlinks/tblspA -s /dev/dtc/lg0/rdsk/<dtc0> /dev/devlinks/tblspA /dev/devlinks/tblspB -s /dev/dtc/lg0/rdsk/<dtc1> /dev/devlinks/tblspB /dev/devlinks/tblspC -s /dev/dtc/lg0/rdsk/<dtc2> /dev/devlinks/tblspC
NOTE: The sample script is provided in order to change symbolic links. Please refer to Sample Script for Changing Symbolic Links on page 307 for a detailed explanation.
7. Type: launchrefresh -af 8. Type dtcmonitortool or dtcperftool to monitor for the completion of the refresh operation. 9. Change the ownership of the actual dtc devices involved as follows:
killpmds chown -R sybase:sybase /dev/dtc/lg0 launchpmds
NOTE: Most RDBMS must own the raw device against which they perform I/O. In Oracle, the command would be: chown -R oracle:dba /dev/dtc/lg0
10. Bring up the database so that it accesses its data through the dtc devices, and all updates are mirrored to the Secondary Server.
www.ibm.com
133
Moving a Non-Symbolic Linked Database to Softek Replicator

You may encounter a situation in which a legacy database uses the absolute path to the raw disk devices. The following procedure gives an example for an Oracle database. You may deal with this situation by redirecting the database through symbolic links as follows: 1. Stop the Database Instance. 2. Rename the current volumes:
mv /dev/vg01/oracle_vol1 /dev/vg01/oracle_vol1_t mv /dev/vg01/roracle_vol1 /dev/vg01/roracle_vol1_t
3. Create the Softek Replicator configurations using the new renamed volumes as source volumes:
source=/dev/vg01/oracle_vol1_t target=/dev/vg01/oracle_vol1
4. Create symbolic links using the old name as the symbolic link name and the dtc devices as the linked object:
ln -s /dev/dtc/lg0/dsk/dtc0 /dev/vg01/oracle_vol1 ln -s /dev/dtc/lg0/rdsk/dtc0 /dev/vg01/oracle_vol1
5. Apply the required permisions and ownership to the dtc devices:

chown oracle:dba /dev/dtc/lg0/dsk/dtc0 chown oracle:dba /dev/dtc/lg0/rdsk/dtc0
134
www.ibm.com
Chapter 11
Online Volume Expansion of dtc Devices

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137 Examples of Volume Expansion on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .138 Example of Volume Expansion on HP-UX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139 Examples of Volume Expansion on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139 Example of Volume Expansion in an HA/CMP Cluster . . . . . . . . . . . . . . . . . . . . . . . . .140
11
Overview
By default, dtc devices can be expanded while Mobility Groups are running by as much as three times the original size of the device at dtcstart time. After this limit is reached, a Mobility Group must be stopped and restarted if the device is to be expanded further. Once Mobility Groups have been restarted (which would normally happen during a machine reboot), an additional 3x expansion is permitted for the device. For situations in which the 3x limitation would be too restrictive, or for situations in which a 3x expansion will likely not occur in between reboots, the default may be changed using the dtclimitsize command. For information on using the dtclimitsize command, refer to the dtclimitsize on page 183.
CAUTION: The current version Softek Replicator does NOT support dynamic volume expansion on Linux.
dtc devices may be expanded while Mobility Groups are started and dtc devices are mounted through the use of the volume expansion commands for the underlying local data device, followed by running the dtcexpand command to expand the dtc device.
NOTE: The dtcexpand command kills the PMD for the expanded device's Mobility Group. The launchrefresh -g <group#> command must be run to bring the Mobility Group to Normal state following a device expansion.
"
To expand a dtc device:
1. Expand the secondary target for the dtc device using the native volume expansion command or user interface for the volume. 2. Expand the local data device for the dtc device using the native volume expansion command or user interface for the volume.
CAUTION: Do not attempt to expand the file system without first completing the underlying volume expansion and dtc device expansion. Some administrative commands and user interfaces will attempt to automatically expand underlying volumes when a file system expansion is executed. This functionality is not compatible with Softek Replicator devices.
3. Check the device size and note the size of the device by typing the following command:
dtcinfo -g <group#>
where <group#> is the group number. 4. Type: dtcexpand -g <group#> -d <dtc_device_path> to perform the volume expansion. You can optionally add the -d <dtc_device_name> option
CAUTION: On AIX, the <dtc_device_name> passed to dtcexpand must be in the form /dev/dtc/ lg<X>/rdsk/dtc<Y>, rather than in the short form /dev/lg<X>dtc<Y>. Where <X> and <Y> are numeric values.
5. Type: dtcinfo g <group#> to verify the new size of the dtc device. 6. Run the launchrefresh command for the group by typing: launchrefresh -g <group#> After the volume has been expanded, the mounted file system may be extended using the file system specific commands or user interface that is designated for this purpose.
www.ibm.com
137
Examples of Volume Expansion on AIX
Examples of Volume Expansion on AIX

The following are some examples procedures on how to perform a volume expansion on AIX.
"
Growing a dtc device mounted as jfs file system based on lvm volumes:
1. Determine the physical partition size by running lslv <target_volume>, and noting the PP Size parameter. 2. Determine number of additional physical partitions needed by dividing total additional size needed by physical partition size. 3. Run extendlv <targetvol> <# of physical partitions> to extend the target volume. 4. Run extendlv <sourcevol> <# of physical partitions> to extend the source volume. 5. Check the device size and note the size of the device by typing the following command:
dtcinfo -g <group#>
where <group#> is the group number. 6. Type: dtcexpand g x to expand the dtc device. 7. Type: dtcinfo g x to verify the new size of the dtc device. 8. Type one of the following:
chfs -a size=<new_size_in_512_byte_blocks> <new_size_in_512_byte_blocks>. <mount_point> to set the size to the value
or
chfs -a size =+<new_size_in_512_byte_blocks> <mount_point> <new_size_in_512_byte_blocks> to the current size.
to add the value
9. Type: df -k
"
Growing a dtc device mounted as vxfs file system and based on vxvm volumes:
<new_size_in_512_byte_blocks>
1. Type: vxassist -g <group_name> <target_volume_name> 2. Type: vxassist -g <group_name> <source_volume_name>

3. Type: vxprint to verify the new size. 4. Check the device size and note the size of the device by typing the following command:
dtcinfo -g <group#>
where <group#> is the group number. 5. Type: dtcexpand g x to expand the dtc device. 6. Type: dtcinfo g x to verify the new size of the dtc device. 7. Type: fsadm b <newsizem> /mountpoint 8. Type: df -k
138
www.ibm.com
11
Example of Volume Expansion on HP-UX

The following are some examples procedures on how to perform a volume expansion on HPUX.
"

dtcinfo -g <group#>
where <group#> is the group number. 5. Type: dtcexpand g x to expand the dtc device. 6. Type: dtcinfo g x to verify the new size in 512-byte sectors of the dtc device. 7. Type: fsadm b <newsize_in_512-byte_sectors> /mountpoint 8. Type: bdf
NOTE: fsadm b requires HP OnLineJFS to grow a vxfs filesystem while mounted. extendfs can be used but requires unmount/mount
Examples of Volume Expansion on Solaris

The following are some examples procedures on how to perform a volume expansion on Solaris.
"
<new_size_in_512-byte_blocks>

<new_size_in_512-byte_blocks>
dtcinfo -g <group#>
where <group#> is the group number. 5. Type: dtcexpand g x to expand the dtc device. 6. Type: dtcinfo g x to verify the new size of the dtc device. 7. Type: fsadm b <newsizem> /mountpoint
www.ibm.com
139
Example of Volume Expansion in an HA/CMP Cluster
8. Type: bdf
NOTE: fsadm b requires HP OnLineJFS to grow a vxfs filesystem while mounted. extendfs can be used but requires unmount/mount
"
Growing a dtc device mounted as ufs file system and based on disksuite/md volumes (softpartitions example)
1. Type: metattach d1 <newsizeM> 2. Type: metattach d2 <newsizeM> 3. Type: metastat t to verify the new size. 4. Check the device size and note the size of the device by typing the following command:
dtcinfo -g <group#>
where <group#> is the group number. 5. Type: dtcexpand g x to expand the dtc device. 6. Type: dtcinfo g x to verify the new size of the dtc device. 7. Type: growfs M /mountpoint 8. Type: df -k
Example of Volume Expansion in an HA/CMP Cluster

Use the following procedure to perform a volume expansion in a HA/CMP Cluster.
"
Growing a dtc device in a HA/CMP Cluster:
NOTE: Verify that the Mobility Group is in Normal state.
1. Type: extendlv <targetvol> <# of physical partitions> to extend the target volume. 2. Type: extendlv <sourcevol> <# of physical partitions> to extend the source volume. 3. Check the device size and note the size of the device by typing the following command:
dtcinfo -g <group#>
where <group#> is the group number. 4. Type: dtcexpand g x to expand the dtc device. 5. Type: dtcinfo g x to verify the new size of the dtc device. 6. Type: chfs -a size=+'<new_size_in_512_byte_blocks>' '<mount_point>'. 7. Type: smitty hacmp -> System Management (C-SPOC) -> HACMP Logical Volume
Management -> Synchronize a Shared Volume Group Definition
8. Failover to other node and launch the launchrefresh command for the Mobility Group by typing: launchrefresh -g <group#>. 9. Type: launchrefresh -g <group#> to initiate a smart refresh command for the Mobility Group. This is necessary because the dtcexpand command used in Step 4 put the Mobility Group in Tracking state.
140
Chapter 12
Replicating Data on ZFS File Systems

Replicating Data Using a One-to-One Configuration for Zpools Based on LUNs. . . . .143 Replicating Data Using a One-to-One Configuration for Zpools Based on Slices . . . .144 Replicating Data Using a Loopback Configuration for Zpools Based on LUNs . . . . . . .146 Replicating Data Using a Loopback Configuration for Zpools Based on Slices. . . . . . .147
12
The following procedures demonstrate how you can use Softek Replicator UNIX to replicate data on a Solaris ZFS file system; the following examples are provided:
D D D D
Replicating data using a one-to-one configuration for zpools based on LUNs Replicating data using a one-to-one configuration for zpools based on slices Replicating data using a loopback configuration for zpools based on LUNs Replicating data using a loopback configuration for zpools based on slices
The procedures below are provided as examples; steps and parameters will vary depending on your configuration/implementation.
Replicating Data Using a One-to-One Configuration for Zpools Based on LUNs

The following example uses two disk volumes, where the target volume is the same size as the source:
D D " c1t3d0 c1t3d1
To replicate data between the source and target volumes:
1. On the source volume, type zpool create pool c1t3d0 to create a pool. 2. Prepare the target volume by creating and then destroying a dummy pool to ensure the same EFI layout as the source disk:
zpool create pool c1t3d1 zpool destroy pool
3. Create a one-to-one, Dynamic Activation Mobility Group, selecting c1t3d0s0 as the source volume and c1t3d1s0 as target.
CAUTION: When a pool is created on a LUN the data is stored on slice 0 of the disk; this is what should be selected as source and target. Dynamic Activation needs to be ON for ZFS pool replication. All LUNs associated with a pool must be in the same group, and must have a target LUN of at least the same size prepared prior to replication, as demonstrated above. The ZFS version on both the source and target volumes must match.
4. On the source volume, start the Mobility Group:

dtcstart -g 0
5. Type launchrefresh -fg 0 to force a Full Refresh of all data blocks from the source volume to the target volume, and to put both the source and target volumes into Refresh state. Softek Replicator UNIX goes to Normal state and assumes typical operations after the data is transferred to the target volume. 6. Type zpool export pool to export the pool:
www.ibm.com
143
Replicating Data Using a One-to-One Configuration for Zpools Based on Slices
7. Wait for I/O caused by the export function to quiesce, then issue the killpmds and dtcstop commands to stop the group:
killpmds -g 0 dtcstop -g 0
8. Type zpool import pool to import the pool. 9. Type ls -la /pool and df -k to verify that the target volume pool contains the same files and space usage data as the source.
Replicating Data Using a One-to-One Configuration for Zpools Based on Slices

c1t3d0s3: 100 c1t3d0s4: 100 " 6282 6282 30.00GB 30.00GB (6183/0/0) (6183/0/0) 62918208 62918208
To replicate data between the source and target volume slices:
1. On the source volume, type zpool create pool c1t3d0s3 to create a pool. 2. On the source volume, verify the pool using the df -k, status and list commands as follows:
df -k pool 30707712 24 30707627 1% /pool
zpool status pool pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 c1t3d0s3 ONLINE 0 0 0 errors: No known data errors zpool list NAME SIZE pool 29.8G
USED 88K
AVAIL 29.7G
CAP 0%
HEALTH ONLINE
ALTROOT -
144
www.ibm.com
12
3. Create a one-to-one, Dynamic Activation Mobility Group, making sure that the slice on the target volume is at least the same size as the source volume slice.
TIP: Use the Softek Data Mobility Console and/or the dtcconfigtool to verify that the device lists correctly show the pool against the corresponding slice. CAUTION: Dynamic Activation needs to be ON for ZFS pool replication. All slices associated with a pool must be in the same group, and must have a target slice of at least the same size prepared prior to replication, as demonstrated above. The ZFS version on both the source and target volumes must match.

dtcstart -g 0
5. On the source volume, type launchrefresh -fg 0 to force a Full Refresh of all data blocks from the source volume to the target volume, and to put both the source and target volumes into Refresh state. Softek Replicator UNIX goes to Normal state and assumes typical operations after the data is transferred to the target volume. 6. On the source volume, type zpool export pool to export the pool: 7. Agian on the source volume, wait for I/O caused by the export function to quiesce, then issue the killpmds and dtcstop commands to stop the group:
8. On the target volume ,type zpool import pool to import the pool. 9. Again on the target volume, type ls -la /pool and df -k to verify that the target volume pool contains the same files and space usage data as the source.
www.ibm.com
145
Replicating Data Using a Loopback Configuration for Zpools Based on LUNs
Replicating Data Using a Loopback Configuration for Zpools Based on LUNs

D D " c1t3d0 c1t3d1
To replicate data between the source and target volumes:
1. On the source volume, type zpool create pool c1t3d0 to create a pool. 2. Prepare the target volume by creating and then destroying a dummy pool to ensure the same EFI layout as the source disk:
zpool create pool c1t3d1 zpool destroy pool
3. Create a loopback, Dynamic Activation Mobility Group, selecting c1t3d0s0 as the source volume and c1t3d1s0 as target. For information on creating a loopback configuration, refer to Loopback Configuration on page 97.
CAUTION: When a pool is created on a LUN the data is stored on slice 0 of the disk; this is what should be selected as source and target. Dynamic Activation needs to be ON for ZFS pool replication. All LUNs associated with a pool must be in the same group, and must have a target LUN of at least the same size prepared prior to replication, as demonstrated above. The ZFS version on both the source and target volumes must match.

dtcstart -g 0
5. On the source volume, type launchrefresh -fg 0 to force a Full Refresh of all data blocks from the source volume to the target volume, and to put both the source and target volumes into Refresh state. Softek Replicator UNIX goes to Normal state and assumes typical operations after the data is transferred to the target volume. 6. On the source volume, type zpool export pool to export the pool: 7. Again on the source volume, wait for I/O caused by the export function to quiesce, then issue the killpmds and dtcstop commands to stop the group:
8. Zone out or disconnect the source volume LUN. 9. On the target volume ,type zpool import pool to import the pool. 10. Again on the target volume, type ls -la /pool and df -k to verify that the target volume pool contains the same files and space usage data as the source.
146
www.ibm.com
12
Replicating Data Using a Loopback Configuration for Zpools Based on Slices

c1t3d0s3: 100 c1t3d0s4: 100 " 6282 6282 30.00GB 30.00GB (6183/0/0) (6183/0/0) 62918208 62918208
To replicate data between the source and target volume slices:
1. On the source volume, type zpool create pool c1t3d0s3 to create a pool. 2. On the source volume, verify the pool using the df -k, status and list commands as follows:
df -k pool 30707712 24 30707627 1% /pool
zpool status pool pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 c1t3d0s3 ONLINE 0 0 0 errors: No known data errors zpool list NAME SIZE pool 29.8G
USED 88K
AVAIL 29.7G
CAP 0%
HEALTH ONLINE
ALTROOT -
3. Create a loopback, Dynamic Activation Mobility Group, making sure that the slice on the target volume is at least the same size as the source volume slice. For information on creating a loopback configuration, refer to Loopback Configuration on page 97.
TIP: Use the Softek Data Mobility Console and/or the dtcconfigtool to verify that the device lists correctly show the pool against the corresponding slice. CAUTION: Dynamic Activation needs to be ON for ZFS pool replication.Replication All slices associated with a pool must be in the same group, and must have a target slice of at least the same size prepared prior to replication, as demonstrated above. The ZFS version on both the source and target volumes must match. Since the source volume will need to be disconnected after replication to allow the target volume pool to be imported, make sure no other data is needed on the source volume on another slice.

dtcstart -g 0
5. On the source volume, type launchrefresh -fg 0 to force a Full Refresh of all data blocks from the source volume to the target volume, and to put both the source and target volumes into Refresh state.
147
Replicating Data Using a Loopback Configuration for Zpools Based on Slices
Softek Replicator UNIX goes to Normal state and assumes typical operations after the data is transferred to the target volume. 6. On the source volume, type zpool export pool to export the pool: 7. Agian on the source volume, wait for I/O caused by the export function to quiesce, then issue the killpmds and dtcstop commands to stop the group:
8. Zone out or disconnect the source volume LUN. 9. On the target volume ,type zpool import pool to import the pool. 10. Again on the target volume, type ls -la /pool and df -k to verify that the target volume pool contains the same files and space usage data as the source.
148
www.ibm.com
Chapter 13
Using Softek Replicator in a Cluster Environment

Configuring Softek Replicator in a Cluster Environment . . . . . . . . . . . . . . . . . . . . . . .151 Veritas Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 HP-UX MC/Serviceguard Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159 AIX HACMP Cluster Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .160
13
This section describes the general procedure on how to successfully set up Softek Replicator in a locally clustered system, and to replicate data to a remote system. The setup of the specific cluster software is release- and platform-dependent. For specific environments, contact Technical Support to assist you in the setup of Softek Replicator.
NOTE: This section is specifically written for the Solaris environment, but it applies to all UNIX platforms.
Configuring Softek Replicator in a Cluster Environment

Use the following procedure to set up Softek Replicator in a cluster environment.
"
To configure Softek Replicator for cluster environments:
1. Define the Softek Replicator configurations using dtcconfigtool or the Softek Data Mobility Console, on the A side of the cluster. 2. Use the IP address or DNS associated with the cluster application virtual server as the source IP/hostname. 3. Copy the /etc/opt/SFTKdtc/pxxx.cfg file to /etc/opt/SFTKdtc on the remote server and change the prefix from p to s. 4. Copy the /etc/opt/SFTKdtc/pxxx.cfg file from the A side of the cluster to /etc/opt/ SFTKdtc/pxxx.cfg on the B side of the local cluster. 5. Create a Pstore volume on a raw logical volume that is on a shared disk. 6. Add entries in the cluster fstab for all file systems that will be replicated. These entries need to be the same dtc devices that exist on the current non-clustered fstab.
Setting the Initial Startup

The script that starts the application must have the dtcstart -a command added to it in order to activate the dtc devices. If this is a real clustered systemthere are no local applications the initialization scripts in /sbin/init.d must be modified to comment out the dtcstart -ba command. This is found in /etc/init.d/SFTKdtc-scan. The following activation procedure describes how to start data replication and a Sybase instance.
"
To set the initial startup:
1. Issue a dtcstart -g x command for the Mobility Group that is being replicated, or dtcstart -a (for all Mobility Groups). 2. Change the ownership of the raw volumes to the proper group for Sybase. For example, sybase:dba, as follows: Chown -R sybase:dba /dev/dtc/lgx/* This should be added to the runserver script for the Sybase instance. 3. Start the Sybase instance.
www.ibm.com
151
Configuring Softek Replicator in a Cluster Environment
Failover Procedure
Use the following procedure on the current node when you move an application to a new node.
"
To fail over to another node:
1. Stop the applications that is to be moved to the new node. 2. Type: killpmds -g x to stop the PMD on the current node. 3. Type: dtcstop -g x to stop the Mobility Group on the current node. 4. Move the IP address and other resources to the new node. These resources should include the Pstore, which is on shared storage. The fail-over script running on the new node will automatically perform the following actions (In Sun clusters, this would be the high-availability agent): 1. Start the Mobility Groups with dtcstart -g x command. 2. Run a Smart Refresh with launchrefresh -g x command. 3. Start the applications that was moved to the new node.
Restarting the Data Replication in Case of Cluster Failure

If there is any cluster failures during the initial Full Refresh, it will be necessary to restart the Full Refresh process from the beginning. If the B side of the cluster fails while the Full Refresh occurs on the A side of the cluster, there is no need to restart the refresh process as long as the Softek Replicator driver on the B side is brought up in Tracking mode after the restart. This should be done with the standard cluster scripts that are invoked during restart.
152
www.ibm.com
13
Veritas Cluster Environment

In many cases, the Primary Server is part of a cluster environment that is replicating to a remote disaster recovery site. This section describes how to configure Softek Replicator where the Primary Server is part of a Veritas Cluster configuration.
Setup and Configuration

The environment consists of a two nodes active/passive Veritas Cluster Server (VCS) and a disaster recovery system, all running the Solaris operating system (other UNIX OS environments supported by both Softek Replicator and VCS will be similar). The environment is designed to represent a scenario where the cluster environment located at the main data processing site replicates data to a disaster recovery location.
Environment Example
D
Primary location:
H
Systems:
D D D D D
Cluster Node1 (Active): SUN1 Cluster Node2 (Passive): SUN2 Solaris 8 with 12/15/03 patch cluster 108528-27 Softek Replicator 3.0 VCS 3.5 MP2 QLA2200F, driver v4.08, FW v2.00.05 18 GB Shared FC disk w/ 500 MB slice as c1t2d0s0 on both systems
Storage:
D D
Secondary location:
H
Systems:
D D D
DR Node SUN3 Solaris 8 with 12/15/03 patch cluster Softek Replicator 3.0 18 GB DA disk w/ 500 MB slice as c0t0d0s3
Storage:
D
Softek Replicator Configuration Example

Softek Replicator is set up on all systems (cluster nodes and disaster recovery system) with default settings and a 32-MB BAB for all primary systems. There is one Mobility Group, lg0, with one dtc device, dtc0, created for replication between the cluster nodes and the disaster recovery system. A second Mobility Group, lg1, is added later for more specialized testing.
153
NOTE: All systems are referenced by IP address, as opposed to hostname, to avoid any issues with name resolution.
Veritas Configuration
Veritas Cluster Server (VCS) was set up on Sun1 and Sun2 with defaults in active/passive mode. For each replication involving both Softek Replicator and VCS, one service group containing Softek Replicator, a cluster IP, and an optional application, is created using the standard Veritas commands: haconf, hagrp, and hares. The pertinent portion of the VCS configuration file and resource dependency tree are listed under each section.
Softek Replicator / Veritas Cluster Integration Considerations

The following guidelines are to be followed: 1. Each cluster service group should generally use only one Mobility Group. This is not an absolute necessity but adhering to this one-service-group-to-one-Mobility Group principle eases administration of the cluster service groups. 2. The shared cluster volume that will serve as the replication source volume should be mapped (via persistent binding or SCSI ID) to the same device on each cluster node. That is, the volume should be c1t2d0s0 on the active cluster node and the passive cluster node. Once again, this is not an absolute requirement but administration is greatly simplified if the volumes are mapped consistently amongst cluster nodes.
Veritas Cluster Replication Scenarios

The following replication scenarios are given as examples.
Scenario 1 - Replicating Data From Active Cluster Node to a Disaster Recovery System
This scenario verifies that basic replication is occurring between the active cluster node, Sun1, and the disaster recovery system, Sun3, prior to attempting any Softek Replicator failovers. 1. Start the dtc device for lg0 on Sun1. 2. Prepare and mount dtc0 on Sun1 over /replvol1. 3. With lg0 in PassThru mode, send 100 files of 1 KB through FTP into /replvol1. 4. Start PMD for lg0 on Sun1. 5. Launch a Full Refresh of lg0. 6. Verify that the refresh completes normally. 7. Send more files through FTP into /replvol1. 8. Verify that the replication occurs properly. 9. Place lg0 into Checkpoint mode. 10. Mount the 500-MB target slice on Sun3.
154
www.ibm.com
13
11. Verify that the data matches what was placed in /replvol1 on Sun1. 12. Launch a Smart Refresh to place everything in a consistent state. The replication should occur as expected between the Primary and Secondary systems.
Scenario 2 - Failover of Active Node to Passive Node With Replication Through Single IP Address
This scenario describes the procedure to follow if the failover of a service group happens. The service group containing Softek Replicator fails from the active cluster node, Sun1, to the passive cluster node, Sun2, with replication occurring through a single IP address. This scenario is typical of cluster configurations that are designed to support a single service group, and thus only seek to offer server level failover granularity. It is also typical of more complex cluster configurations that assign a virtual IP to each service group and thus offer application-level failover granularity. The Disaster Recovery system, Sun3, will see no change in the replication process other than a brief outage of the PMD.
Sun1 and Sun3 are in the state of the end of Replication Scenario 1 on page 154 (Sun1 and Sun3 replicating in Normal mode, dtc device mounted over /replvol1 on Sun1).
1. Copy /etc/opt/SFTKdtc/p000.cfg from Sun1 to Sun2. 2. The failure of Sun1 occurs. 3. Enable the IP address of Sun1 on Sun2 with ifconfig:ifconfig hme<n> addif <Sun1 IP>
netmask <Sun1 netmask> up
4. Start DTC for lg0 on Sun2. 5. Run fsck dtc0 on Sun2 to ensure data integrity. 6. Mount dtc0 on Sun2 over /replvol1. 7. Start PMD for lg0 on Sun2. 8. Verify that RMD detects return of PMD and that replication resumes normally. 9. Place lg0 into Checkpoint mode from Sun2. 10. Mount the 500-MB target slice on Sun3. 11. Verify that the data matches what was in /replvol1 on Sun2.
NOTE: A Full Refresh may be required at this point depending on the state of the data following the failover.
The replication should occur as expected between the primary and secondary systems following the failover.
www.ibm.com
155
Scenario 3 - Failover of Active Node to Passive Node without the IP address as Part of Cluster Service Group
This scenario describes the failover of a service group containing Softek Replicator from the active cluster node, Sun1, to the passive cluster node, Sun2, with replication occurring through multiple IP addresses. The failover service group includes Softek Replicator but does not include the IP address used for replication from the cluster nodes to the Disaster Recovery system. This scenario is typical of cluster configurations that are designed to support multiple service groups, and seek to offer application-level failover granularity, but use a single cluster IP for all service groups. In this type of scenario, failing over the cluster IP along with Softek Replicator to the passive node will render service groups that have not been failed unusable, reducing failover granularity to the server level. The Disaster Recovery system, Sun3, will see a change in replication process reflecting a brief outage of the PMD and a subsequent return of replication via a different PMD / Mobility Group.
NOTE: This scenario is also applicable to active/active clusters.
1. Sun1 and Sun3 are in the state of the end Replication Scenario 1 on page 154. (Sun1 and Sun3 replicating in Normal mode, dtc device mounted over /replvol1 on Sun1). 2. Copy /etc/opt/SFTKdtc/p001.cfg from Sun1 to /etc/opt/SFTKdtc/p001.cfg on Sun2. 3. Copy /etc/opt/SFTKdtc/p001.cfg from Sun2 to /etc/opt/SFTKdtc/s001.cfg on Sun3. 4. The failure of Sun1 occurs. 5. Start the dtc device for lg0 on Sun2. 6. Run fsck on the 500-MB source slice on Sun2 to ensure data integrity. 7. Mount dtc0 on Sun2 over /replvol1. 8. Override mode for lg0 to NORMAL. 9. Start PMD for lg0 on Sun2. 10. Verify that RMD detects return of PMD and that replication resumes normally. 11. Place lg0 into Checkpoint mode from Sun2. 12. Mount the 500-MB target slice on Sun3. 13. Verify that the data matches what was in /replvol1 on Sun2.
NOTE: A Full Refresh may be required at this point depending on the state of the data following the failover.
Replication should occur as expected between the primary and secondary systems following the failover.
156
www.ibm.com
13
Scenario 4 - Failover with Veritas Cluster Server

This environment is an actual active/passive Veritas Cluster. The system status is the same as the one at the end of Scenario 1 (on page 154); however, lg0 is stopped so that it can be started by the cluster service. 1. Sun1 and Sun2 are configured with the following cluster service group and resources: Softek Replicator starts Mobility Group lg0 and maintains resource dependencies, as described in Softek Replicator Configuration Example on page 153. You may create a script to perform this automatically; refer to Veritas Script - Sample 1 on page 287. 2. Copy /etc/opt/SFTKdtc/p000.cfg from Sun1 to Sun2, as created in Scenario 1 on page 154. 3. Bring the Softek Replicator service group online on Sun1 with the cluster service manager. This will enable the replication cluster IP address and start Softek Replicator with lg0 on the active node. 4. Launch a Full Refresh of lg0 on the active node from the command line. 5. Prepare and mount the device representing the Mobility Group/Volume Pair started by the cluster service group (dtc0) over replvol1 on the active node. 6. Write some files to the mounted device to force replication to occur. 7. The failure of Sun1 occurs. 8. The failover of Softek Replicator from Sun1 to Sun2 occurs, as illustrated in the following image. 9. Place lg0 into Checkpoint mode from the command line on Sun2. 10. Mount the 500-MB target slice on Sun3. 11. Verify that the data matches what was in /replvol1 on Sun2.
Scenario 5 - Failover with VCS and Network Applications

This environment is an active/passive Veritas Cluster that includes a service group monitoring a network application. Samba is the network application because of its known integration with VCS and its use of the ubiquitous CIFS standard for data transfer. The system status is the same as the one at the end of Scenario 1 (on page 154); however, a new Mobility Group with one 500-MB Volume Pair, lg1 / dtc0, was created for use by Samba described as follows.
www.ibm.com
157
Veritas Cluster Environment Figure 13.1: Sample Configuration File

#================================================================= # Softek Replicator Configuration File: /etc/opt/SFTKdtc/p001.cfg # Softek Replicator Version 3.0.0.0 # # Last Updated: Tue Mar 20 13:00:31 PST 2007 #================================================================= # # Primary System Definition: # SYSTEM-TAG: SYSTEM-A PRIMARY HOST: 129.212.188.11 PSTORE: /dev/dsk/c0t2d0s3 # # Secondary System Definition: # SYSTEM-TAG: SYSTEM-B SECONDARY HOST: 129.212.188.122 JOURNAL: /var/spool/journal2 # # Device Definitions: # PROFILE: 1 REMARK: FTP PRIMARY: SYSTEM-A DTC-DEVICE: /dev/dtc/lg1/rdsk/dtc0 DATA-DISK: /dev/rdsk/c1t2d0s1 SECONDARY: SYSTEM-B MIRROR-DISK: /dev/rdsk/c0t0d0s4 # # # -- End of dtc Configuration File: /etc/opt/SFTKdtc/p001.cfg
1. Sun1 and Sun2 are configured with the following cluster service group and resources: The group starts Softek Replicator with lg1, Samba serving lg1/dtc0 mounted over /replvol2 as \\129.212.188.11\test, and maintains resource dependencies, as described in Veritas Script - Sample 2 on page 288. lg1/dtc0 is prepared and ready for mounting by VCS and sharing by Samba. 2. Copy /etc/opt/SFTKdtc/p001.cfg from Sun1 to Sun2. 3. Bring the Softek Replicator service group online on Sun1 with the cluster service manager. This will enable the replication cluster IP address; start Softek Replicator with lg1; and start Samba on the active node. 4. Launch a Full Refresh of lg1 on the active node from the command line. 5. Start data replication. 6. The failure of Sun1 occurs. 7. The failover of Softek Replicator and Samba from Sun1 to Sun2 occurs. 8. Place lg1 into Checkpoint mode from the command line on Sun2. 9. Mount the 500-MB target slice on Sun3. 10. Verify that data matches what was in /replvol2 on Sun2.
158
www.ibm.com
13
HP-UX MC/Serviceguard Cluster Environment

This section explains how to make Softek Replicator work in an HP-UX MC/Serviceguard cluster environment. HP MC/Serviceguard is specialized in protecting applications from hardware and software failures. With Serviceguard, multiple servers (nodes) and/or server partitions are organized into a cluster that delivers highly available application services to LAN-attached clients. HP MC/Serviceguard monitors the health of each node and responds to failures in a way that minimizes or eliminates application downtime. HP MC/Serviceguard provides data protection; application availability; ease of management; and quick deployment of applications. HP MC/Serviceguard provides a mechanism to automatically move applications between nodes in an HP-UX cluster. An application is set up as a package within HP MC/Serviceguard. A package consists of shared disk, network address, and executable files that run on a server. HP MC/Serviceguard uses a parameter file and scripts to automatically move an application, including an application that has failed. The first step in a reconfiguration process is to bring down the application, deactivate the disk from the logical volume group, and then deactivate the network resources. When this is completed, the disk resources are reactivated, the network address is activated and the application is restarted. The key is the order of execution. The default order of execution is as follows: 1. Activate or make online the volume group. 2. Mount the file systems. 3. Start the applications. In order for Softek Replicator to work properly, you must modify the following:
D
The current logical volume that gets mounted to the file system in the Parameter section of the cluster script, for the Softek Replicator special devices or dtc device. For example:
LV[3]=/dev/dtc/lg2/dsk/dtc0 FS[3]=/opt/app/xp01/oracle FS_MOUNT_OPT[3]="-o
The Custom Script section in the Parameter section of the cluster control script. In the Custom Script section, the dtc device must be started with the dtcstart command for the Mobility Group that is being reconfigured. Also, the Pstore associated with the Mobility Group must be on shared disk that is reconfigured to the surviving node in the cluster.
For a sample MC/Serviceguard Script, please refer to Appendix B: HP-UX MC ServiceGuard Script.
www.ibm.com
159
AIX HACMP Cluster Environment

HACMP (High Availability Cluster MultiProcessing for AIX) provides monitoring capabilities and detection of application failures. HACMP manages recovery application environments to back up resources. HACMP achieves availability through automation, using the redundant hardware configured in a cluster to keep applications running, and restarting them on a backup server if necessary. This enables you to minimize outage downtime. You can set up to 32 servers in an HACMP cluster environment. HACMP can also detect application or system software problems that are not severe enough to interrupt proper operation of the system, such as process failure or exhaustion of system resources. HACMP monitors, detects, and reacts to such conditions, allowing the system to stay available during random, unexpected software problems. HACMP can be configured to react to hundreds of system events. Using HACMP can eliminate planned outages, since users, applications and data can be moved to backup systems during scheduled system maintenance. HACMP clusters can be configured to meet complex and varied application availability and recovery needs.
CAUTION: When using Softek Replicator HACMP automation DO NOT modify /etc/filesystems or run the dtcautomount command. This will cause problems. The Softek Replicator HACMP automation uses symbolic links which does not require ANY changes to /etc/filesystems. The softlinks are created in the modified HACMP script upon activation of the resource group.
Configuring Softek Replicator in HACMP Cluster Environment

The procedure described below will successfully configure Softek Replicator in a HACMP clustered system. The automation will require that the administrator know the following information.
D D D D
Name of the HACMP Resource Group which requires data replication. The resource group IP address to be used for the source IP address. The remote system IP address which is the target of the data replication. The directory name for the Softek Replicator Journal on the remote system.
The automation assumes that all the volumes in a volume group associated with a particular resource group are to be replicated. If this is not the case then the configuration files will have to be manually edited.
160
www.ibm.com
13
Pre-configuration Steps
Define the logical volume groups and logical volumes on the remote system to be the same size as the source volumes. These logical volumes should not contain file systems. Create a Pstore in the same volume group that is to be replicated. The Pstore must be a logical volume without a file system and cannot be mounted. This volume group must be a shared volume group in the same resource group that you are setting up for replication. This is so that you can replicate on any node in the cluster. The logical volume name for the Pstore device must have the word Pstore in the name. For example: /dev/Pstore or /dev/Pstore-1 Create a Journal on the remote system which is the target for the replication. Insure that the cluster is operating properly before you do anything. Fail the cluster back and forth. Optionally, you can reboot each node to insure that the cluster is healthy.
Building the Softek Replicator Configurations:

Using the Softek Data Mobility Console, build the Softek Replicator configurations using the following procedure. To properly protect the data in the resource group you must configure every logical volume individually.
"
To build the Softek Replicator configuration:
1. Start with the Node in the cluster that has the resource currently running. 2. Configure the source and target volumes. When selecting the primary replication link address, clear the default IP address and type in the Resource Group IP address instead.
NOTE: This is very important since the resource group IP address follows the application and this is what Softek Replicator uses to continue replication properly.Save the configuration.
3. Copy the Softek Replicator configuration files (pxxx.cfg and sxxx.cfg)to the other nodes in the cluster. 4. Using the Softek Data Mobility Console Import Mobility Group function, import the configurations file from other nodes in the cluster that do not have the groups already displayed in the Softek Data Mobility Console. 5. Right click on the server and select Import Mobility Group. 6. Repeat this procedure on all other nodes in the cluster.
www.ibm.com
161
"
To activate Softek Replicator with a HACMP Cluster:
NOTE: Softek Replicator HACMP automation is triggered by taking resource groups offline or online on a node with Softek Replicator HACMP automation enabled.
1. Type dtcmodhacmp resource_group1:<group#> resource_group2:<group#> .... on each node in the cluster.

NOTE: Multiple resource groups must be specified one after the other.
Where:
H H resource_group <group#>
is the HACMP resource groups that needs to be replicated.
is the Mobility Group which has all of the volumes associated with the resource groups.
This script deploys two scripts within the HACMP cluster framework that call Softek Replicator to perform activation and deactivation cluster methods. Example of replicating one resource group: dtcmodhacmp rsgrp1:0 Example of replicating multiple resource groups: dtcmodhacmp rsgrp1:5 rsgrp2:6 2. Move the resource groups away from the node to be Softek Replicator HACMP enabled. 3. Activate the Softek Replicator HACMP scripts.
NOTE: Insure that the resources groups are not running on the nodes where the cluster scripts are being modified.
a. Type: cd /usr/es/sbin/cluster/events/utils b. Backup the original cl_activate_vgs and cl_deactivate_vgs scripts using the following commands: i) Type: cp -p
cl_activate_vgs cl_activate_vgs_original
ii) Type: cp -p cl_deactivate_vgs cl_deactivate_vgs_original c. Move in the new cl_activate_vgs and cl_deactivate_vgs scripts using the following commands: i) Type: mv cl_activate_vgs.dtc cl_activate_vgs ii) Type: mv cl_deactivate_vgs.dtc cl_deactivate_vgs d. Bring the resource groups back online to the Softek Replicator HACMP enabled node. 4. Type dtcinfo to validate that the dtc devices have started. 5. Repeat Step 2 and Step 4 for the remaining nodes in the cluster. 6. Test moving the resource group between nodes. When the resource group is moved from node to node, type dtcinfo on the new node to insure that the dtc devices are present and operational. 7. Issue a full refresh to synchronize the data.
162
www.ibm.com
13
Disabling Softek Replicator in the HACMP Cluster

Softek Replicator uses the following files to store configuration information: /etc/dtc/hacmp/cl_dtc_flags and /etc/dtc/hacmp/cl_dtc_rsgrps. If you need to disable Softek Replicator in the cluster, you can edit these files.
NOTE: Insure that the resource groups are not running on the node(s) in which these configuration files are being edited.
"
To stop HACMP from using Softek Replicator:
1. Stop the resource groups from running on the node in which the changes are to be made.
NOTE: This could be moving to another node or stopping the resource group in the cluster.
2. Open cl_dtc_flags located in /etc/dtc/hacmp/. 3. Change the value of usingSFTKdtc to 0.

H H
0: do not use Softek Replicator. 1: use Softek Replicator.
4. Start the resource groups on this node.

"
To disable selected HACMP resource groups from using Softek Replicator:
1. Stop the selected resource groups from running on the node in which the changes are to be made.
NOTE: This could be moving to another node or stopping the resource groups in the cluster.
2. Open cl_dtc_rsgrps located in /etc/dtc/hacmp/. 3. Remove the lines for the resource groups in which you would like to disable the usage of Softek Replicator. 4. Start the resource groups on this node.
NOTE: When disabling or enabling resource groups by deleting or adding lines to /etc/dtc/hacmp/ cl_dtc_rsgrps, the changes will not be effective while the resource groups are in an offline state on the node.
www.ibm.com
163
Adding Resource Groups to a Running Softek Replicator Configuration

CAUTION: If Softek Replicator HACMP automation is running on a HACMP node then do NOT remove Resource Groups from this file until these Resource Groups are offline or moved over to another node.
"
To add new resource groups to a currently running HACMP Softek Replicator configuration:
1. Create the new configurations following the procedure above Building the Softek Replicator Configurations: on page 161. 2. Add the resource group name and the corresponding Mobility Group number (only one resource group name and number allowed per line) to the following file on one of the cluster nodes that is not currently running the resource group: /etc/dtc/hacmp/
cl_dtc_rsgrps
3. Move the resource group to the node where the above change was made. Check to see that the dtc devices are started using the dtcinfo command. 4. Update the other nodes as done in step 2 above so that they are ready to run Softek Replicator for this resource group.
164
www.ibm.com
Chapter 14
Replication in a Virtual Environment
Running Softek Replicator on a VMware System . . . . . . . . . . . . . . . . . . . . . . . . . . . .167 Maintaining and Changing the MAC Address of a Virtual Machine . . . . . . . . . . . . . . .167
14
Running Softek Replicator on a VMware System
If you want to install Softek Replicator and perform data replication on a virtual machine, for simulation and testing purposes, you may encounter licensing problems because of the host ID. The license requires the Host ID for Softek Replicator to work. A Host ID is an 8-digit hexadecimal number that is tied to the physical server hardware. The Host ID is calculated from the Media Access Control (MAC) address of the primary LAN card. A change in the MAC address of the virtual machine will cause licensing problems. You have to maintain the MAC address and ensure that it is the same no matter how the virtual machine is booted. The following section explains how to keep the same MAC address in order to have a non-changing host ID on a VMware system. The following section, Maintaining and Changing the MAC Address of a Virtual Machine, is available from VMware at: http://www.vmware.com/support/ws4/doc/network_macaddr_ws.html
Maintaining and Changing the MAC Address of a Virtual Machine

When a virtual machine is powered on, VMware Workstation automatically assigns each of its virtual network adapters an Ethernet MAC address. A MAC address is the unique address assigned to each Ethernet network device. The software guarantees that virtual machines are assigned unique MAC addresses within a given host system. In most cases, the virtual machine is assigned the same MAC address every time it is powered on, so long as the virtual machine is not moved (the path and filename for the virtual machine's configuration file must remain the same) and no changes are made to certain settings in that file. In addition, VMware Workstation does its best, but cannot guarantee, to automatically assign unique MAC addresses for virtual machines running on multiple host systems.
Avoiding MAC Address Changes

To avoid changes in the MAC address automatically assigned to a virtual machine, you must not move the virtual machine's configuration file. Moving it to a different host computer, or even moving it to a different location on the same host computer, changes the MAC address. You also need to be sure not to change certain settings in the virtual machine's configuration files. If you never edit the configuration file and do not remove the virtual Ethernet adapter, these settings remain untouched. If you do edit the configuration file, do not remove or change the following options:
ethernet[n].generatedAddress ethernet[n].addressType ethernet[n].generatedAddressOffset uuid.location uuid.bios
167
ethernet[n].present
In these options, [n] is the number of the virtual Ethernet adapter, for example ethernet0.
NOTE: To preserve a virtual Ethernet adapter's MAC address, you must be careful not to remove it. If you remove the adapter, and then recreate it, it may receive a different MAC address.
Manually Assigning a MAC Address

If you want to guarantee that the same MAC address is assigned to a given virtual machine every time, even if the virtual machine is moved, or if you want to guarantee a unique MAC address for each virtual machine within a network environment, you can assign the address manually instead of allowing VMware Workstation to assign it automatically. To assign the same, unique MAC address to any virtual machine manually, use a text editor to remove three lines from the configuration file and add one line. The configuration file has a .vmx extension at the end of the filename. On a Linux host, a virtual machine created with an earlier VMware product may have a configuration file with a .cfg extension. Remove the three lines that begin with the following from the configuration file:
ethernet[n].generatedAddress ethernet[n].addressType ethernet[n].generatedAddressOffset
In these options, [n] is the number of the virtual Ethernet adapterfor example, ethernet0. Add the following line to the configuration file:
ethernet0.address = 00:50:56:XX:YY:ZZ
In this line, XX must be a valid hexadecimal number between 00 and 3F, and YY and ZZ must be valid hexadecimal numbers between 00 and FF. Because VMware Workstation virtual machines do not support arbitrary MAC addresses, you must use the above format. So long as you choose a value for XX:YY:ZZ that is unique among your hard-coded addresses (where XX is a valid hexadecimal number between 00 and 3F, and YY and ZZ are valid hexadecimal numbers between 00 and FF), conflicts between the automatically assigned MAC addresses and the manually assigned ones should never occur.
168
www.ibm.com
Chapter 15
Commands
dtcagentset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171 dtcautomount. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .172 dtcbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 dtccheckpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .173 dtcconfigtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .174 dtcdebugcapture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175 dtcexpand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176 dtcgenmklv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176 dtchostinfo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177 dtcinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177 dtcinit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179 dtcjfspremount. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179 dtcjfspostmount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .180 dtckillbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181 dtckillpmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181 dtckillrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182 dtckillrmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182 dtclicinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182 dtclimitsize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .183 dtcmklv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .183 dtcmodfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .184 dtcmonitortool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .184 dtcmonitortty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185 dtcoverride . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .186 dtcpanalyze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .187 dtcperftool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .187 dtcpmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .188
dtcpsreplicate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 dtcrefresh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 dtcrmdreco. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 dtcset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 dtcstart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 dtcstop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 dtcsyncfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 killagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 killbackfresh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 killdtcmaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 killpmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 killrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 killrmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 launchagent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 launchbackfresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 launchdtcmaster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 launchpmds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 launchrefresh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
Commands
15
This chapter describes all commands of Softek Replicator. In order to run the Softek Replicator commands, you must be a super-user.
dtcagentset
Description: This command activates or deactivates the Data Mobility Agent. This agent is used
to communicate with the Softek Data Mobility Console if you are managing the replication from a Windows server. The Data Mobility Agent is automatically started at system boot time but you must activate it manually if you need to use it.
Syntax: dtcagentset
[[-e <CollectorIP> [:CollectorPort]] [-i <AgentIP>] [-b BabSize] | [-d] | [-l] [-t <TransmitInterval>]] [-p <Listener_Port>]]
Options: The following options are available for the dtcagentset command:
H
-e CollectorIP [:CollectorPort]: Activates the Data Mobility Agent to communicate with the Data Collector. Specify an IP (Internet Protocol) address and a port number, which is used by the Data Collector. If CollectorPort is omitted, the default value of 576 is used. The default value is used when the agent is inactive, and the current value is used when the Data Mobility Agent is already active. -e 193.91.100.15
Example: dtcagentset
H
-i<AgentIP>: The Data Mobility Agent IP address that the Collector will use for
communication.
Example: myserver#
H
dtcagentset -e 193.91.100.15 -i 192.168.111.2
-b <bab_size_MB>: Initializes the BAB and loads the dtc driver. The bab_size is specified in Megabytes as an integer between 1 and 1547. The -b parameter takes effect for the subsequent time the dtc driver is loaded. dtcagentset -e 193.91.100.15 -i 192.168.111.2 -b 64
Example: myserver#
H
-d: The agent becomes inactive. The values of CollectorIP and CollectorPort will
be deleted when the agent is deactivated. The Data Mobility Agent cannot be deactivated when it is running. You should deactivate the Data Mobility Agent after stopping it.
H
-l: Displays the state of the Data Mobility Agent setup this is the default option. If no options are specified, the dtcagentset command considers that the -l option is specified. If the agent is active, the CollectorIP and the CollectorPort are displayed. If the Data Mobility Agent is not active, this command displays: Collector connection information is not set up.
-t <TransmitInterval>: Use the -t option to set up the frequency with which the agent process sends information to the collector, that is, the transmit interval between two transmissions to the Softek Data Mobility Console. For example, during a period of great activity, such as a full refresh or heavy I/O, you can increase the transmit interval. This value must equal or be greater than 4. Default value: 4.
Example: myserver# dtcagentset -e 193.91.100.15 -i 192.168.111.2 -b 32 -t 10
www.ibm.com
171
dtcautomount
-p <Listener_Port>: Use the -p option to change the listener port from the default
value of 57.
Example: myserver#
dtcagentset -p 578
dtcautomount
Description: When the local data device is registered in the filesystems file as auto-mount, this
command modifies the filesystems file and registers the dtc device mount information for the local data device so that the primary devices are replaced by the dtc devices. The dtcautomount command now allows you to modify file system mount tables for individual Mobility Groups, or all Mobility Groups.
Syntax: dtcautomount
H H H H H H
[-c] {-g <group#>]...|-a|-r<group#>..|-ra} [-d]
Options: The following options are available for the dtcautomount command:
-a: Changes the entries for all defined dtc devices. -c: Checks if the file system mount table needs to be updated. -g <group#>: Changes entries for dtc devices in the specified Mobility Group. -r <group#|a>: Changes the mount points from dtc devices to the data devices. -c: Checks if the file system mount table needs to be updated.
-d: Deletes all file systems file backup copies.

dtcautomount -g 101fstab was updated
Example: myserver#
NOTE: FSTAB is only updated if the following conditions are true: AIX: When the data device of the specified Mobility Group exists; the mount stanza of that record is defined to [true] [automatic] ; and the dev stanza or the log stanza agrees with the device name. HP-UX: When the data device of the specified Mobility Group exists, and the file system type of that record is other than [swap] [swapfs] [dump] [ignore] [nfs] . Linux: When the data device of the specified Mobility Group exists, and the file system type of that record is other than [swap] [proc] [tmpfs] [devpts] [dump] [ignore] [auto] [nfs] . Solaris: When the data device of the specified Mobility Group exists, and the automount is specified to yes. During the update process, any data that is changed is first converted into a comment line and (-specific) character string, and the date of the update is added to the end of the record.
172
www.ibm.com
Commands
15
dtcbackfresh
Description: This command initiates a Backfresh operation as a means to synchronize data on
the Primary Server with more up-to-date data on the secondary one.
NOTE: The Backfresh state is only for maintenance. Do not access or update local or mirror data devices until the Backfresh operation is complete.
Syntax: dtcbackfresh
H H
[-a][-g <group#>]
Options: The following options are available for the dtcbackfresh command:
-a: Indicates all Mobility Groups. -g <group#>: Selects one or more Mobility Groups to place into Backfresh state. dtcbackfresh -g 101
Example: myserver#
dtccheckpoint
Description: This command turns checkpointing of the mirror devices on or off. When
checkpointing is on, Softek Replicator relinquishes its exclusive lock on the mirror device(s) and allows other applications to access the device(s) in read-only mode. The dtccheckpoint command looks in the following directories for optional, user-defined checkpoint scripts. Primary scripts:
D D
/etc/opt/SFTKdtc/cp_pre_on_p###.sh /etc/opt/SFTKdtc/cp_post_on_p###.sh /etc/opt/SFTKdtc/cp_post_on_s###.sh /etc/opt/SFTKdtc/cp_pre_off_s###.sh
Secondary scripts:
D D
NOTE: The SFTKdtc directory does not exist on AIX systems; the dtccheckpoint command will look for optional, user-defined checkpoint scripts in the dtc directory instead. For example: /etc/dtc/
lib/cp_pre_on_p###.sh
If these scripts are implemented on the primary and Secondary Servers, they are executed when the command is entered. For more information on checkpointing, refer to Using the Checkpoint Feature on page 237. If the Journal feature is turned off, dtccheckpoint operates in a different way. For complete details, refer to Turning Journal Operations On or Off on page 235.
Syntax: dtccheckpoint
[-g ###|-a] [-on|-off] [-p] [-s]
www.ibm.com
173
dtcconfigtool
Options: The following options are available for the dtccheckpoint command:
H H H H
-a: Indicates all Mobility Groups. -g <group#>: Selects one or more Mobility Groups. -p: Causes the system on which the dtccheckpoint command is entered to act like a
Primary Server and run the appropriate scripts.

-s: Causes the system on which the dtccheckpoint command is entered to act like a
Secondary Server and run the appropriate scripts.

NOTE: The -p and -s options for the dtccheckpoint command are only valid in a complex system configuration: when the system is set up to act as both primary and Secondary Servers.
H
-on/off: Turns checkpointing on or off. dtccheckpoint -g 101 -on
Example: myserver#
dtcconfigtool
Description: This command starts the utility for viewing, editing, or defining Mobility Group
configuration files, including primary and Secondary Servers, tunable PMD parameters, dtc devices, and throttles.
Syntax: dtcconfigtool
H
[-rmdtimeout t] [-noconfigchk] [-nohelp] [-syncminor]
Options: The following options are available for the dtcconfigtool command:
-rmdtimeout t: Sets the time (in seconds) during which the dtcconfigtool should wait for the RMD to return a list of disk partition devices and volumes defined on each Secondary Server. The default is 15. -noconfigchk: Bypasses consistence and collision checking of component devices for dtc device definitions in each of the Mobility Group configuration files when dtcconfigtool is brought up. -nohelp: Suppresses the help messages that pop up after 2 seconds when the pointer
H H
is placed on a control or entry.

-syncminor: Defines the value for the minor device number that will be allocated for a specific volume.
174
www.ibm.com
Commands
15
Example: myserver#
dtcconfigtool
dtcdebugcapture
Description: This command collects system and software information that can be used for
diagnosing problems with the Softek Replicator environment. The information is saved in a file named nodename1.tar.Z.uufor example, onefish1.tar.Z.uuin directory /tmp. Save this file and send it to Technical Support personnel, if necessary.
Syntax: dtcdbugcapture Options: None. Example: dtcdebugcapture
www.ibm.com
175
dtcexpand
dtcexpand
Description: To expand a dtc device, run this command after the dtc devices are mounted
through the use of the volume expansion commands for the underlying local data device.
NOTE: The dtcexpand command will stop the PMD for the expanded device's Mobility Group. The dtcexpand command allows you to expand the device volume up to three times without using the dtclimitsize command. If you need to expand the volume by more than three times the original size, then dtclimitsize must be used and the Mobility Group must be stopped and restarted for it to take effect.
Syntax: dtcexpand
-g <group#> -d <dtc_raw_device_name>
CAUTION: On AIX, the <dtc_raw_device_name> passed to dtcexpand must be in the form /dev/dtc/lg<X>/rdsk/dtc<Y> , rather than in the short form /dev/lg<X>dtc<Y> .
Options: The following options are available for the dtcexpand command:
H H
-g <group#>: Is the name of the Mobility Group. -d <device_path>: The path of the dtc device. -g 0 -d /dev/dtc/lg1/rdsk/dtc2
Example: dtcexpand
dtcgenmklv
Description: The dtcgenmklv command creates a new file called dtcmklv in /usr/dtc/bin.
It takes the current AIX mklv script and modifies it to add the -p option to set the minor device number for AIX devices. The minor device number range is either 0 to 255, or 0 to 511 (for large file system support). For more information on the dtcmklv command, please refer to dtcmklv on page 183. The dtcgenmklv command should be run when new AIX updates come out in order to recreate the dtcmklv file.
Syntax: dtcgenmklv Options: None. Example: dtcgenmklv
176
www.ibm.com
Commands
15
dtchostinfo
Description: This command runs a utility that returns the networked hostnames and IP
addresses for a system, and the host id of the system where the command is run. You can run this command on either primary or Secondary Server to obtain local host information. From the Primary Server, you can also run the command with the secondary host name to obtain Secondary Server information.
Syntax: dtchostinfo Options: None Example: myserver#
<system_name>
Where <system_name> is an optional hostname. Defaults to current hostname.
dtchostinfo onefish myserver# dtchostinfo
(from the Primary Server only)
Example: 193.91.182.100
onefish hostid:0x82425e9b
dtcinfo
Description: This command displays state information for one or more Mobility Groups and
their dtc devices. It must be run on the Primary Server. The following state information can be displayed:
D D D D D
BAB memory size requested and BAB memory actually allocated Mobility Group operating state (for example, Normal or PassThru) dtc device information Major and minor device numbers for the dtc and target device Version of the product installed on a system
Syntax: dtcinfo Options: The following options are available for the dtcinfo command:
H H H H
-g <group#>: Displays information for all devices within the specified Mobility Group. This option may be repeated to include more than one Mobility Group. -a: Displays information pertaining to all dtc devices for all started Mobility Groups. -v: Displays the version of the product installed on the system. This option can be used on the Secondary Server. -h: Displays help.
www.ibm.com
177
dtcinfo
Example: myserver#
dtcinfo -g 0
NOTE: For Mobility Groups in Backfresh state, Shutdown state 0 means that the Mobility Group is up and running. 1 means that the Mobility Group is NOT. Run dtcstart if you need to use the Mobility Group and the Shutdown state is 1.
Requested BAB size ................ 67108864 (~ 64 MB) Actual BAB size ................... 67108864 (~ 64 MB) Free BAB size ..................... 67108864 (~ 64 MB) Mobility Group 0 (bmaix52 -> 127.0.0.1) Mode of operations.............. Entries in the BAB.............. Sectors in the BAB.............. Sync/Async mode................. I/O delay....................... Persistent Store................ Device /dev/dtc/lg0/rdsk/dtc0: dtc device number............... Local disk device number........ Local disk size (sectors)....... Local disk name................. Remote mirror disk.............. Remote mirror device number..... Read I/O count.................. Total # of sectors read......... Write I/O count................. Total # of sectors written...... Entries in the BAB.............. Sectors in the BAB.............. Device /dev/dtc/lg0/rdsk/dtc1: dtc device number............... Local disk device number........ Local disk size (sectors)....... Local disk name................. Remote mirror disk.............. rtargetlog Remote mirror device number..... Read I/O count.................. Total # of sectors read......... Write I/O count................. Total # of sectors written...... Entries in the BAB.............. Sectors in the BAB.............. 0x2b0001 0x2c0003 81920 /dev/rsourcelog 127.0.0.1:/dev/ 0x2d0001 0 0 0 0 0 0 0x2b0002 0x2c0001 983040 /dev/rsource1 127.0.0.1:/dev/rtarget1 0x2d0002 0 0 0 0 0 0 Tracking 0 0 Async 0 /dev/Pstore
178
www.ibm.com
Commands
15
dtcinit
Description: This command resizes the BAB, or initializes the Pstore device. Using this
command is the only way to specify a size other than the standard size available in the Configuration Tool dialog box.
Syntax: dtcinit
H
[-b<bab_size_MB>] [-p <Pstore_device>] [-s] [-l]
Options: The following options are available for the dtcinit command:
-b <bab_size_MB>: Resizes the BAB and changes the .cfg file to reflect the designated size. The bab_size should be specified in Megabytes as an integer between
1 and 1547.
H
-p <Pstore_device>: Initializes the designated device to 0. If the specified device is
used as a Pstore for an operating Mobility Group, or the specified device is not defined in any .cfg files, the specified device is not initialized and the dtcinit command terminates.
H H
-s: Initializes all Pstores with small HRT (128KB per device). This option should be
used for small devices.

-l: Initialize all Pstores with large HRT (12MB per device). This option should be used
for large devices, that is, volumes in excess of 500 gigabytes.

NOTE: To determine the bitmap size in use, run the dtcpanalyze command with the -v option.
Example: myserver#
dtcinit -p 512
dtcjfspremount
Description: The dtcjfspremount command integrates mirrored data device and log device
on a secondary JFS system. The command runs on the Secondary Server only. When you attempt to mount the AIX device that is mirrored by Softek Replicator as JFS, dtcjfspremount needs to be executed against the mirrored device before the fsck and mount commands. There are two arguments required for this command:
i) Mirrored JFS data device: data-device ii)Mirrored JFS log device
NOTE: The dtcjfspremount command is only for JFS and does not work for JFS2.
Syntax: dtcjfspremount Options: None Example: If

data-device = /dev/s_jfs000 and Log-device = /dev/s_jfslog000,
then: dtcjfspremount /dev/s_jfs000 /dev/s_jfslog000
If the operation is successful, no output is produced. If the operation fails or an error is encountered, an error message is displayed.
179
dtcjfspostmount
Procedure Example with Checkpointing: The following procedure lists the steps in the mirroring mode to mount the secondary device: 1.) On the Primary Server, type: dtccheckpoint -g ### -on 2.) On the Secondary Server, perform the following steps:
dtcjfspremount fsck mount
NOTE: You must run a Full Refresh to resynchronize the Primary and Secondary Servers after running fsck on the Secondary Server.
dtcjfspostmount
Description: The dtcjfspostmount command sets data-device and log-device integrity back
to where it was before dtcjfspremount. If you executed dtcjfspremount and used a mirror device, you need to execute dtcjfspostmount. This command only runs on the Secondary Server. The JFS data-device is a mandatory argument for this command.
NOTE: The dtcjfspostmount command is only for JFS, and does not work on JFS2.
Syntax: dtcjfspostmount Options: None. Example: If

data-device is /dev/s_jfs000, then: # dtcjfspostmount /dev/s_jfs000
If the operation is successful, no output is produced. If the operation fails or an error is encountered, an error message is displayed. Procedure Example with Checkpointing: The following procedure describes the steps for checkpointing; from mounting the secondary device to going back to the mirroring mode after unmounting: 1) On the Primary Server, type: dtccheckpoint -g ### -on 2) On the Secondary Server, perform the following steps:
dtcjfspremount fsck mount umount dtcjfspostmount
3) On the Primary Server, type: dtccheckpoint -g ### -off
180
www.ibm.com
Commands
15
For more information on Checkpoint, refer to dtccheckpoint on page 173 and Using the Checkpoint Feature on page 237.
NOTE: You must run a Full Refresh to rnesynchronize the Primary and Secondary Servers after running fsck on the Secondary Server.
dtckillbackfresh
Description: This command terminates the Backfresh state for one or more Mobility Groups.
dtckillbackfresh moves the secondary Mobility Groups out of Backfresh state and into
Normal state. This action takes place whether the PMD is running or not. If the secondary PMD is not running but is in Backfresh state (that is, if the PMD was in Backfresh state and the killpmds command was entered for it), then when it is restarted with the launchpmds command, it is restarted in Normal state.
Syntax: dtckillbackfresh
H H H
[-a] [-g<group#>] [-h]
Options: The following options are available for the dtckillbackfresh command:
-g <group#>: Terminates backfresh daemon for Mobility Group <group#>. Can be
repeated to include multiple Mobility Groups.

-a: Terminates all Backfresh daemons. -h: Displays help. dtckillbackfresh -a
Example: myserver#
dtckillpmd
Description: This command terminates one or more active PMD daemons.
NOTE: If any secondary PMD is in Backfresh state when dtckillpmd is entered for it, Softek Replicator will remember it. When the PMD is restarted, Softek Replicator restarts it in Backfresh state where it left off. The only way to take the PMD out of Backfresh state is with the killbackfresh command.
Syntax: dtckillpmd
H H H
[-a] [-g<group#>] [-h]
Options: The following options are available for the dtckillpmd command:
-g <group#>: Terminates PMD daemon for Mobility Group <group#>. This option
can be repeated to affect multiple Mobility Groups.

-a: Terminates all PMD daemons. -h: Displays help.
dtckillpmd -a
Example: myserver#
www.ibm.com
181
dtckillrefresh
dtckillrefresh
Description: This command terminates Refresh state in one or more Mobility Groups. Syntax: dtckillrefresh
H H H -a] [-g<group#>] [-h]
Options: The following options are available for the dtckillrefresh command:
-g <group#>: Terminates Refresh state for Mobility Group <group#>. Can be
repeated to include multiple Mobility Groups.

-a: Terminates Refresh state for all Mobility Groups. -h: Displays help.
dtckillrefresh -a
Example: myserver#
dtckillrmd
Description: This command terminates one or more active RMD daemons. Syntax: dtckillrmd
H H H [-a] [-g<group#>] [-h]
Options: The following options are available for the dtckillrmd command:
-g <group#>: Terminates RMD daemon for Mobility Group <group#>. This option can be repeated for multiple Mobility Groups. -a: Terminates all RMD daemons. -h: Displays help.
dtckillrmd -a
Example: myserver#
dtclicinfo
Description: This command reports the state of Softek Replicator licenses on this system (valid,
missing or expired).
Syntax: dtclicinfo Options: None Example: myserver#

dtclicinfo Valid DTC Demo License -- expires: Mon May 25 15:30:40 2009
182
www.ibm.com
Commands
15
dtclimitsize
Description: This command allows the device expansion limit to be set on a per-device, or perMobility Group basis.
NOTE: The Mobility Group must be stopped before running dtclimitsize and must be restarted after the command is run.
Syntax: dtclimitsize
[-g<group#>] [-d <device_path>] [-s <device_size_multiplier>]
Options: The following options are available for the dtclimitsize command:
H H H
-g <group#>: Is the name of the Mobility Group. -d <device_path>: The path of the dtc device. -s <device_size_multiplier>: A multiplier indicating how large the volume can
be expanded from its original size. To modify the expansion limit for a single device:
dtclimitsize -g <group#> -d <device_path> -s <device_size_multiplier>
To modify the expansion limit for an entire Mobility Group:

dtclimitsize -g <group#> -s <device size multiplier>
Example: myserver#
dtclimitisize -g 101 -d /dev/dtc/lg1/rdsk/dtc2 -s 2
dtcmklv
Description: This command creates a logical volume, on AIX only. It is a superset of the standard
AIX mklv command, adding a -p option to control what minor device number is given to the new logical volume. For usage, refer to Creating and Mounting New JFS2 File Systems on dtc Devices on page 117. Other options are the same as the AIX-supplied mklv command. This command is created by the dtcgenmklv command at installation time, and needs to be recreated after certain operating system upgrades. For more information, please refer to dtcgenmklv on page 176.
Syntax: dtcmklv
H
[-p]
Options: The following options are available for the dtcmklv command:
-p [MinorNumber]: Defines the minor device number on AIX.
NOTE: For other options, refer to the proper AIX documentation.
Example: dtcmklv
-p 44 -y'target_log' -t jfs2log -L'target_log' rootvg logform /dev/target_log logform: destroy /dev/target_log (y)? y chfs -a log=/dev/target_log /target_data
www.ibm.com
183
dtcmodfs
dtcmodfs
Description: When the local data device is registered in the filesystems file as auto-mount, this
command modifies the filesystems file and registers the dtc device mount information for the local data device so that the primary devices are replaced by the dtc devices. The dtcmodfs command now allows you to modify file system mount tables for individual Mobility Groups, or all Mobility Groups.
Syntax: dtcmodfs
H H H H H H
[-c] [-g <group#>...|-a] [-r<group#>...|a] [-d]
Options: The following options are available for the dtcmodfs command:
-a: Change the entries for all defined dtc devices. -c: Check if the file system mount table needs to be updated. -g <group#>: Change entries for dtc devices in the specified Mobility Group. -r <group#|a>: Change the mount points from dtc devices to the data devices. -c: Check if the file system mount table needs to be updated.
-d: Delete all file systems file backup copies.

-g 123
Example: dtcmodfs
dtcmonitortool
Description: This command starts a tool for displaying Softek Replicator performance statistics
in real time. dtcmonitortool must be run on the primary, and only monitors Mobility Groups that have been started. It displays current values for a variety of parameters, error messages from both primary and Secondary Servers, and alerts the operator of potentially fatal errors.
Syntax: dtcmonitortool
[--help]
[-timeout <value_in_seconds>] [-messages<count>]
Options: The following options are available for the dtcmonitortool command:
H
-timeout <value_in_seconds>: Sets the number of seconds during which dtcmonitortool waits for a connection to be established to a primary or Secondary
Server to get the latest error messages and daemon status. Default: 30.
H
-messages <count>: Sets the maximum number of retained error and warning
messages. Default: 200.
184
www.ibm.com
Commands
--help: Displays the command line argument help message and exits.
dtcmonitortool
15
Example: myserver#
dtcmonitortty
Description: This command starts a tool for displaying Softek Replicator performance statistics
in real time. dtcmonitortty can be run on the primary or secondary servers, and monitors any Mobility Groups that you specify. It displays current values for a variety of parameters, error messages from both primary and Secondary Servers, and alerts the operator of potentially fatal errors.
Syntax: dtcmonitortty
H H H H H H
[-p|-s] [-l <logfile>] [-i <interval>] [-g <group#>]
Options: The following options are available for the dtcmonitortty command:
-p: Displays data for the primary server. -s: Displays data for the secondary server. -l <logfile>: Redirects the output to a log file that you specify. -i <interval>: Refreshes the display in seconds that you specify. If an interval is not specified then it defaults to five seconds. -g <group#>: Displays information for the Mobility Group specified. If no Mobility Groups are specified then data for all the groups are displayed. -h|-?: Displays this usage help.
-p
Example: dtcmonitortty
www.ibm.com
185
dtcoverride
dtcoverride
Description: The dtcoverride command clears the BAB or Pstore or forces a transition
between Softek Replicator operating modes.
CAUTION: Use caution when issuing this command, since it may cause a loss of synchronization between the systems in Softek Replicator.
Syntax: dtcoverride
[-a|-g<group#>] {clear [BAB|LRT|HRT]} {state [normal|passthru|tracking]}
Options: The following options are available for the dtcoverride command:
H H H
-a: Validates all Mobility Groups to be affected by the forced change of state. -g <group#>: Selects one or more Mobility Groups to be affected by the forced
change of state.
clear [BAB|LRT|HRT]: The BAB option clears the BAB. The LRT and HRT options
page 18.
clear the tracking bitmaps. For more information on LRT and HRT, refer to Pstore on
NOTE: In AIX and Solaris, if the dtcoverride -g 0 clear bab command is issued while I/O is running on dtc devices for a particular Mobility Group, the following message is displayed: Driver replicate failure for device. Despite this message, the Mobility Group transitions into Refresh state and eventually returns to Normal.
H
state [normal|passthru|tracking]: Forces a change of state for the designated
Mobility Groups.
NOTE: If you specify a state of tracking or passthru on the dtcoverride command, the PMD daemon associated with the Mobility Group is killed (if it is running) before the change of state takes effect.
Example: myserver#
dtcoverride -g 101 state normal
186
www.ibm.com
Commands
15
dtcpanalyze
Description: The panalyze command is based on Pstore information, and returns the amount
of journal space needed for a Smart Refresh of a Mobility Group.
Syntax: panalyze
H H H H
[-a|-g<group#>][-h ][-v]
Options: The following options are available for the dtcpanalyzer command:
-a: analyze all Mobility Groups that are started. -g <group#>: Analyze the selected the Mobility Group. -h: Displays help for the command. -v : Select the verbose mode. This will return more detailed info about the state of LRDB and HRDB.
-g 101
Example: panalyze
dtcperftool
Description: This command starts the graphical charting tool for displaying Softek Replicator
performance data. If the primary and secondary configuration files are present on the system when it is run, using one of the following arguments determines which files are queried.
Syntax: dtcperftool
H H
[-p] [-s]
Options: The following options are available for the dtcperftool command :
-p: Specifies that dtcperftool should look for primary configuration files only. -s: Specifies that dtcperftool should look for secondary configuration files only.
www.ibm.com
187
dtcpmd
Example: myserver#
dtcperftool
dtcpmd
Description: This command starts PMD daemons for one or more Mobility Groups. Syntax: dtcpmd
H H H [-a|-g<group#>][-h ]
Options: The following options are available for the dtcpmd command:
-a: Starts all PMD daemons for all Mobility Groups on this system. -g <group#>: Starts PMD daemon for Mobility Group <group#>. This option can be
repeated for multiple Mobility Groups.

-h: Displays help.
-a
Example: dtcpmd
188
www.ibm.com
Commands
15
dtcpsreplicate
Description: This command allows older version of the Pstores to be converted to the new format
when a new version of Softek Replicator is installed. It also allows the newer version of the Pstores to be converted to the previous version. This command might be necessary if the user installed a new product version, converted the Pstore, then decided to un-install the product to go back to a previous revision. This command would be run prior to the product de-installation, to convert the Pstore to the previous version.
NOTE: You will be notified if you need to use this command when dtcstart is run to start the Mobility Groups.
Syntax: dtcpsreplicate
D
-r<version>
Options: The following options are available for the dtcpsreplicate command:
-r<version>: Reverts a Pstore back to a previous version. This command must be prior to uninstalling the current version of Softek Replicator inorder to convert the Pstore.
NOTE: The version string must be typed in the form x.x.x.x. For example, -r2.3.0.0 to revert to a 2.3.0 Pstore version.
Example: myserver#
dtcpreplicate -r2.3.0.0
www.ibm.com
189
dtcrefresh
dtcrefresh
Description: This command places one or more Mobility Groups in Refresh state to synchronize
the Secondary Servers mirror devices with the contents of the Primary Servers local data devices. Use dtcrefresh to:
D D
Establish an initial remote mirror during a new Softek Replicator installation. Re-establish a remote mirror if the BAB fills and automatically moves the Mobility Groups into Tracking state on the Primary Server by transferring only the changed data (Smart Refresh). Re-establish the mirror after a remote mirror device is replaced due to hardware failure.
NOTE: Use the -f option to perform a full, sector by sector synchronization of the systems.
Syntax: dtcrefresh
H H H
[-a] [-g <group#>] [-c] [-f] [-r]
Options: The following options are available for the dtcrefresh command:
-g <group#>: Places all dtc devices in Mobility Group <group#> in Refresh state. This
option can be repeated for multiple Mobility Groups.

-a: Places all dtc devices for all Mobility Groups in Refresh state. -c: Initiates a Checksum Refresh in which all blocks on the local data device and mirror device are compared using a checksum method to identify deltas. Only blocks that have been modified (that is, those for which the checksum varies) are sent to the Secondary Server. A Checksum Refresh writes mirror data to the journal file system, if the Mobility Group has the Journal feature turned on. If the Journal feature is turned off, disk data transferred from the source device will be written directly to the target device, without storing it in Journals. -f: Forces a Full Refresh of all data blocks from the local data devices to the mirror devices on the Secondary Server. -r: Continues a Full Refresh that was previously halted due to a NETBROKE error.
dtcrefresh -a
H H
Example: myserver#
190
www.ibm.com
Commands
15
dtcrmdreco
Description: This command should be used only when you are switching data ownership over
to the Secondary Server. It is a recovery utility that ensures the mirror devices are in a coherent and recoverable state by flushing data from the journal file(s) to the corresponding mirror device(s). The dtcrmdreco command creates a /etc/opt/SFTKdtc/s###.off file for each Mobility Group. When the Primary Server comes back online or when a new system is added to the configuration, the Mobility Groups RMD daemon detects the s###.off file and does not start mirroring operations. This keeps the mirror devices from being corrupted before you perform a Backfresh/Refresh.
NOTE: On AIX, the dtcrmdreco command will create a s###.off file in the /etc/dtc/lib/ directory.
See Appendix A: Recovering Data for detailed information on using this command for data recovery.
Syntax: dtcrmdreco
H H
[-a] [-g<group#>] [-d]
Options: The following options are available for the dtcrmdreco command:
-g <group#>: Recovers data for Mobility Group <group#>. -a: Recovers data from all Mobility Groups.
NOTE: One of the above-mentioned options MUST be used in the dtcrmdreco command.
H
-d: Deactivates the recovery mode.
NOTE: The -d option must only be used AFTER the recovery has been performed for the Mobility Group. Using the -d option before will generate the following error:
Couldn't unlink file /etc/opt/SFTKdtc/s###.off: The system cannot find the file specified.
Example:
1) On the secondary server:
D D
To activate the recovery mode and create the s###.off file, type:
dtcrmdreco -g 0
To deactivate the recovery mode and remove the s###.off file, type:
dtcrmdreco -d -g 0
2) On the primary server type: launchbackfresh -g 0
www.ibm.com
191
dtcset
dtcset
Description: This command sets tunable parameters for each designated Mobility Group. You
can also use it to view the current setting of a specific tunable parameter, or all tunable parameters for a Mobility Group. Run it on the Primary Server only.
NOTE: You must run killpmds before and launchpmds after changing the following tunable parameters:
CHUNKSIZE, COMPRESSION, NETMAXKBPS
You must run dtcstop before and dtcstart after changing the following tunable parameters:
SYNCMODE, SYNCMODEDEPTH, SYNCMODETIMEOUT
Syntax: dtcset [-a|-g <group#> <keyword>=<value>] [JOURNAL=on|off] [LRT on|off] Options: The following options are available for the dtcset command:
H
-g <group#> <keyword>=<value>: Sets value of designated tunable parameter for each Mobility Group. If you do not specify a keyword or value, Softek Replicator shows you all tunable parameter values for the Mobility Group. For the complete list of parameters, refer to Working with Tunable Parameters on page 223. -g <group#>: Displays the tunable parameters settings for the specified Mobility
H H
Group.
[LRT on|off]: Allows you to turn on or off low resolution tracking. The Pstore is not
used during normal replication and all bit maps will be used from memory only. A complete resynchronizatoin will be required if a system panic occurs. This will improve performance since the disk copy of the Pstore will not have to be written to during replication. The default state is on.
Example: myserver#
dtcset -g 6 netmaxkbps=500
Sample Output for myserver# dtcset -g 6

CHUNKSIZE: 256 CHUNKDELAY: 0 SYNCMODE: off SYNCMODEDEPTH: 1 SYNCMODETIMEOUT: 30 NETMAXKBPS: 500 STATINTERVAL: 10 MAXSTATFILESIZE: 64 TRACETHROTTLE: off COMPRESSION: off
192
www.ibm.com
Commands
15
dtcstart
Description: The dtcstart command processes the configuration file (.cfg) for the Mobility
Group(s) specified and activates the dtc devices defined within the Mobility Group. As the command processes each file, it creates a copy that it renames with a .cur extension. The .cur file is an exact copy of the Mobility Groups configuration file when it was started and is referenced by all Softek Replicator commands during operations. Configuration(.cfg) files are only processed by dtcstart, so when a Mobility Group is stopped and restarted a new .cur file is created. This allows you to make modifications to the .cfg files (for example, edit throttle definitions) and have precise control over those changes when they are implemented. Also, the .cur file ensures that all components of Softek Replicator have a consistent view of the configuration. Starting a Mobility Group make its dtc devices become available in the drop-down menus in the Configuration Tool.
NOTE: Running dtcstart on a Mobility Group with a large number of dtc devices can be slow.
Syntax: dtcstart
H H H
[-a|-g <group#>] [-b]
Options: The following options are available for the dtcstart command:
-g <group#>: Starts a specific Mobility Group and its dtc devices. -a: Starts all Mobility Groups and their dtc devices. -b: For boot scripts only - Restarts previously started Mobility Groups and their dtc devices that were active prior to a system crash or that were stopped with the dtcstop -s option. -g 1 dtcstart -g 2 dtcstart -a
Example: dtcstart
dtcstop
Description: This command stops one or all Mobility Groups and corresponding dtc device
definitions. The tracking bitmaps are written to the Pstore and the dtc device definitions associated with the Mobility Group(s) are removed from the /dev/dtc device tree, making them no longer available for active use.
Syntax: dtcstop
H H
[-a|-g <group#>] [-s]
Options: The following options are available for the dtcstop command:
-g <group#>: Stops a specific Mobility Group and removes its dtc devices. -a: Stops all Mobility Groups and removes their dtc devices.
www.ibm.com
193
dtcsyncfs
-s: For boot scripts only - Stops the previously started Mobility Groups but marks them enabled for restart when the system is next booted. -g 1 dtcstop -g 2 dtcstop -a
Example: dtcstop
dtcsyncfs
Description: This command enables you to force JFS2 journals to be applied to the data disk,
on AIX only, as follows:
D D D
Forces the log meta data to be applied to all file system disks within the Mobility Group. Forces the log device and file system superblocks to be rewritten back to the target disk. Verifies for all log devices in the Mobility Group that the dtc log device is being used even for the non-replicated disks. The command only checks the /etc/filesystems file.
-f <mount_points>
This command must be run from the source side only.
Syntax: dtcsyncfs
H
Options: The following options are available for the dtcsyncfs command:
-f <mount_points>: Specifies a list of mount points that need to be flushed.
-f mnt/rep1
Example: dtcsyncfs
killagent
Description: This command stops the Agent daemon. When the Agent daemon is stopped, the
communication with the Data Collector is closed.
Syntax: killagent Options: None Example: myserver#

killagent Agent has been shutdown
194
www.ibm.com
Commands
15
killbackfresh
Description: Shell script that runs dtckillbackfresh; terminates any Backfresh operation
currently running on this system.
killbackfresh provides the following additional functionality beyond that provided by dtckillbackfresh.
D D
The script checks to see if any Mobility Group PMDs are running. If a PMD daemon is running, then dtckillbackfresh is started. If command line arguments are specified, they are passed to dtckillbackfresh. If no command line arguments are specified, then dtckillbackfresh is run with the -a argument, which terminates all Mobility Group PMDs that are currently in Backfresh state. A message is written to the Softek Data Mobility Console stating that the Backfresh operations have been terminated. If there is no PMD daemon running, then a message is written to the Softek Data Mobility Console saying that there is no PMD daemon running, and the script exits.
[-a|-g <group#>]
Syntax: killbackfresh
H H
Options: The following options are available for the killbackfresh command:
-g <group#>: Terminates Backfresh daemons for the Mobility Group <group#>.
This option can be repeated for multiple Mobility Groups.

-a: Terminates all Backfresh daemons. -g 12 -g 0
Example: killbackfresh
If the script completes and the command is executed, there is no output. If there is no PMD daemon running, a message indicating this is written to the Softek Data Mobility Console and the script exits.
killdtcmaster
Description: Shell script that terminates any PMD daemon, RMD daemon, in.dtc, or throtd
processes currently running on the system. Use this script for maintenance activities, or for removing Softek Replicator from a system prior to a software upgrade.
Syntax: killdtcmaster Options: None Example: myserver#

killdtcmaster
in.dtc master Softek Replicator daemon has been shutdown throtd Softek Replicator throttle daemon has been shutdown
www.ibm.com
195
killpmds
killpmds
Description: This is a shell script that runs dtckillpmd, and terminates PMD daemons currently
running on this system. When PMD daemons are terminated, their corresponding RMD daemons on the Secondary Servers are terminated as well. killpmds provides the following additional functionality beyond that provided by dtckillpmd:
D D D D D
The script checks to see if any Mobility Group PMD daemons are running. If PMD daemons are running, then dtckillpmd is started: If command line arguments are specified, they are passed to dtckillpmd. If no command line arguments are specified, then dtckillpmd is started with the -a argument, which terminates all PMD daemons. If there is no PMD daemon running, then a message is written to the Softek Data Mobility Console saying that there is no PMD daemon running, and the script exits.
[-a|-g <group#>]
Syntax: killpmds
H H
Options: The following options are available for the killpmds command:
-g <group#>: Terminates PMD daemons for Mobility Group <group#>. This option
can be repeated for multiple Mobility Groups.

-a: Terminates all PMD daemons. killpmds -g 12 -g 0
Example: myserver#
If the command is executed, there is no output. If there are no PMD daemons running, a message indicating this is written to the Softek Data Mobility Console and the script exits.
killrefresh
Description: Shell script that runs dtckillrefresh; terminates any Refresh operations
currently running on this system. killrefresh provides the following additional functionality beyond that provided by dtckillrefresh:
D D D D D
The script determines whether any Mobility Group Refresh operations are running. If Refresh operations are running, then dtckillrefresh is started. If command line arguments are specified, they are passed to dtckillrefresh. If no command line arguments are specified, then dtckillrefresh is run with the -a argument, which terminates all Mobility Group Refresh operations. If there are no Refresh operations running, then a message is displayed that there are no Refresh operations running, and the script exits.
[-a|-g <group#>]
Syntax: killrefresh
Options: The following options are available for the killrefresh command:
196
www.ibm.com
Commands
-g <group#>: Terminates Refresh state for Mobility Group <group#>. This option
15
H H

-a: Terminates Refresh state for all Mobility Groups. killrefresh -g 2 -g 10
Example: myserver#
If the script completes and the command is executed, there is no output. If no Refresh operations are running, a message indicating this is written to the Softek Data Mobility Console and the script exits.
killrmds
Description: Shell script that terminates all RMD daemon currently running on this system.
Enter the command on the Secondary Server. Once the RMD daemon has been killed, you can restart it by either killing (killpmds) and relaunching (launchpmds) the associated PMD daemon on the Primary Server. killrmds provides the following additional functionality beyond that provided by dtckillrmd:
D D D D D
The script checks to see if any Mobility Group RMD daemons are running. If RMD daemons are running, then dtckillrmd is started. If command line arguments are specified, they are passed to dtckillrmd. If no command line arguments are specified, then dtckillrmd is started with the a argument, which terminates all RMD daemons.
If there are no RMD daemons running, then a message is written to the Softek Data Mobility Console saying that there are no RMD daemons running, and the script exits.
[-a|-g <group#>]
Syntax: killrmds
H H
Options: The following options are available for the killrmds command:
-g <group#>: Terminates the RMD daemon for Mobility Group <group#>. This option can be repeated for multiple Mobility Groups. -a: Terminates the RMD daemon for all Mobility Groups on the Secondary Server. -g 2 -g 10
Example: killrmds
NOTE: If there is no RMD daemon running, then a message indicating this is written to the Softek Data Mobility Console, and the script exits.
www.ibm.com
197
launchagent
launchagent
Description: This command starts the Agent daemon (in.dua). As the agent is automatically
started at system boot time, you must use this command AFTER the agent is activated or manually stopped. If you use the server on which the Agent is installed as a Primary Server, set the BAB size using the dtcconfigtool or the dtcinit command.
NOTE: To activate the agent, refer to dtcagentset on page 171; to stop the agent, refer to killagent on page 194.
Syntax: launchagent Options: None Example: launchagent
launchbackfresh
Description: Shell script that runs dtcbackfresh to start Backfresh operations. Recall that
Backfresh synchronizes the Primary Server with the Secondary Server using the mirrored data.
NOTE: Backfresh state is a maintenance only mode. Do not access or update local or mirror data devices until the Backfresh is complete.
launchbackfresh provides the following functionality beyond that provided by the dtcbackfresh command:
D D
The script launches the PMD daemons if they are not already running. It validates system license information and then starts dtcbackfresh. If no command line arguments are specified, then dtcbackfresh is run with the a argument, which puts all Mobility Groups and all devices into Backfresh state. See Appendix A: Recovering Data for more detailed information on using this command for data recovery.
[-a|-g <group#>]
Syntax: launchbackfresh
H H
Options: The following options are available for the launchbackfresh command:
-a: Indicates all Mobility Groups. -g <group#>: Selects one or more Mobility Groups designated by the <group#>. -a
Example: launchbackfresh
198
www.ibm.com
Commands
15
launchdtcmaster
Description: Shell script that starts the master Softek Replicator daemon (in.dtc) and Softek
Replicator throttle daemon (throtd). This is a convenience utility to re-establish the Softek Replicator daemon environment after system maintenance when you entered the killdtcmaster command. The Softek Replicator master daemon and throttle daemon are launched automatically during installation and after reboot. You do not normally use this command except for system maintenance activities.
Syntax: launchdtcmaster Options: None Example: launchdtcmaster
launchpmds
Description: Shell script that starts PMD daemons for the designated Mobility Groups. This is
the preferred method of starting the daemons. launchpmds provides the following additional functionality beyond that provided by dtcpmd:
D D D D D D
The script checks that a valid license file (DTC.lic) is present on the system. The script checks to see if the PMD daemons are already running. If the PMD daemon is not running, then dtcpmd is started. If command line arguments are specified, they are passed to dtcpmd. If no command line arguments are specified, then dtcpmd is started with the -a argument, which starts all PMD daemons. If there are PMD daemons running, then a message is written to the Softek Data Mobility Console saying that there is a PMD daemon running, and the script exits.
[-a|-g <group#>]
Syntax: launchpmds
H H
Options: The following options are available for the launchpmds command:
-g <group#>: Starts the PMD daemon for Mobility Group <group#>. This option

-a: Starts PMD daemons for all Mobility Groups on this system. -g 1 -g 2
Example: launchpmds
NOTE: If you experience a file table overflow error when attempting to start Mobility Groups or launch PMD processes on HP-UX machines, you may need to increase the value of the maxfiles_lim and nfile tunable parameters. These two HP-UX kernel tunable parameters determine the maximum number of files that can be opened per-process and system-wide, respectively.
www.ibm.com
199
launchrefresh
launchrefresh
Description: Shell script that starts dtcrefresh, which causes the product to transition to
Refresh state and synchronize the local data device contents with the mirror devices. This is the preferred method for starting dtcrefresh. launchrefresh provides the following additional functionality beyond that provided by dtcrefresh:
D D
If no command line arguments are specified, then dtcrefresh is run with the -a argument, which starts a Smart Refresh operation on all Mobility Groups. If a Full Refresh is required, use the -f option.
[-a|-g <group#>] [-c] [-f] [-r]
Syntax: launchrefresh
H H H
Options: The following options are available for the launchrefresh command:
-g <group#>: Puts all dtc devices in Mobility Group <group#> into Refresh state.
This option can be repeated for multiple Mobility Groups.

-a: Puts all dtc devices for all Mobility Groups into Refresh state. -c: Initiates a Checksum Refresh in which all blocks on the local data device and mirror device are compared using a Checksum method to identify deltas. Only blocks that have been modified (that is, those for which the checksum varies) are sent to the Secondary Server. A Checksum Refresh writes mirror data to the journal file system, if the Mobility Group has the Journal feature turned on. If the Journal feature is turned off, disk data transferred from the source device will be written directly to the target device, without storing it in Journals. -f: Forces a Full Refresh of all data blocks from the data devices to the mirror devices on the Secondary Server. -r: Continues a Full Refresh that was previously halted due to a NETBROKE error. -g 1 -g 21
H H
Example: launchrefresh
200
www.ibm.com
Chapter 16
Using the Monitoring Tools

Using dtcperftool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203 Using dtcmonitortool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .207 dtcinfo ASCII Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .210
16
Softek Replicator provides three utilities that let you monitor the system as it is running. These tools allow you to observe the effect that current tunable parameters and throttles have on the system. Monitoring can also alert you to problems as they occur in initial configurations or due to the expansion of existing Softek Replicator environments.
Using dtcperftool
The dtcperftool displays charts of various Softek Replicator performance metrics. You can view multiple charts at one time, and modify, delete or print them. dtcperftool lets you observe Softek Replicator performance and trends over time, and create printouts. You can run dtcperftool on the Primary Server, Secondary Server, or both. Thus, you can monitor data being sent and received. If primary and secondary configuration files exist on the system when dtcperftool is run, use the optional -p or -s arguments to determine whether the primary or secondary configuration files are queried. The only difference in using the tool on the systems is what performance metrics you can observe. To start the performance monitoring tool, type dtcperftool in the command line. The dtcperftool screen is divided into three steps, corresponding to the steps you follow to generate a performance monitoring chart.
Figure 16.1: Softek Replicator Performance Monitor Window
www.ibm.com
203
Using dtcperftool
Step I: Setting up the Chart

The first step is to define a chart and the way you want to view the information.
"
To setup the chart:
1. Click Define New Chart to clear all the fields. 2. In the Title text box, type a title for the chart. The name can be up to 60 characters. 3. In the Chart Type list box, select one of the following chart types:
H H H
Line Chart Bar Chart Stacked Bar Chart
4. In the Number of Samples text box, select the number of samples to be shown in a chart. The PMD writes performance statistics to an ASCII file. By default, this file is updated every 10 seconds. Thus a 21-sample analysis reflects 210 seconds of operation.
Step II: Specifying Measurements Shown in the Chart

The next step is to define which Mobility Group, dtc devices, and its measurements that you want to display in the chart.
"
To specify the measurements to show in the chart:
1. From the Mobility Group list box, select the Mobility Group you want monitor. 2. From the DTC Device list box, select the dtc devices you want to monitor. 3. From the Measurement list box, select the measurements you want to add to the chart. The measurements you can select for the Primary Server are:
D D
Actual KBps The actual number of kilobytes of data sent over the network. Effective KBps If you have compression activated with the COMPRESSION tunable parameter, the Effective KBps is the pre-compression data rate being transferred over the network. Entries The current total number of entries in the BAB awaiting transmission. Percent BAB full The percentage of BAB in use by pending entries awaiting transfer. Percent done The percentage of data from the dtc devices transmitted over the network during a refresh or backfresh operation. Read KBps The local dtc device read I/O rate for applications accessing the dtc device. Write KBps The local dtc device write I/O rate for applications updating the dtc device.
D D D D D
204
www.ibm.com
16
The measurements you can select for the Secondary Server are:
D D
Actual KBps The actual number of kilobytes of data sent over the network. Effective KBps If you have compression activated with the COMPRESSION tunable parameter, the Effective KBps is the pre-compression data rate being transferred over the network. Entry age recvd The age of the oldest entry received in seconds.
4. Click Add To Chart to add the measurement to the chart. or Click Modify Measurement to make changes. or Click Remove From Chart to remove the measurement from the chart.
Step III: Displaying the Chart

When you have made all the necessary choices to define the chart you want, you can now display the chart.
"
To display the chart:
1. Click Display Chart to display the chart on the screen.

NOTE: Because of the fixed width of the chart label at the bottom of the display, if you size the chart and make it too narrow, the display becomes corrupted. Click the border of the chart window and drag the window larger to correct the problem.
Modifying a Chart
After displaying the chart, you can still change what is displayed. There are two ways to edit a chart: 1. Left-click anywhere in the chart. 2. The background colour of the chart flashes briefly to yellow. or 1. Right-click anywhere in the chart to display the Chart Options Menu. 2. Select Edit Chart. 3. Modify the chart definition. 4. Click Display Chart to view the new version.
www.ibm.com
205
Using dtcperftool
Other Chart Functions

You can display multiple charts at one time. Each chart is defined separately. To delete a chart, print a chart, or save it in a file, use the Delete Chart, Print Chart, or Print Chart to File option from the Chart Options Menu. You can also delete a chart by selecting it to edit and then selecting Delete Chart in dtcperftool. The Performance Tool displays error messages in red in the upper left area.
Performance Monitoring Files

obtains current performance information from a set of ASCII files created by throtd (RMDs). These files are called performance monitoring files or performance tracking files. These files are written in row/column format and are suitable for use by other applications such as spreadsheets or databases. They are located at:
dtcperftool D D D /var/opt/SFTKdtc/p###.prf /var/opt/SFTKdtc/p###.prf.1 /var/opt/SFTKdtc/p###.phd
(previous generation file for this Mobility Group)
(column descriptions for .prf files)
NOTE: Replace /var/opt/SFTKdtc with /var/dtc.
Related Tunable Parameters

The following tunable parameters for throtd affect the performance monitoring files:
D D STATINTERVAL MAXSTATFILESIZE
As throtd takes new performance snapshots, the latter are appended to the end of the appropriate performance monitoring file. Thus, a performance monitoring file records observations moving from the oldest at the top of the file to the most recent at the bottom of the file. For more information, refer to Working with Tunable Parameters on page 223.
206
www.ibm.com
16
Using dtcmonitortool
dtcmonitortool provides a comprehensive picture of Softek Replicator activity and state information from the Primary Server.
CAUTION: dtcmonitortool may become unusable if there are more than 250 devices per Mobility Group.
Figure 16.2: dtcmonitortool Window
Status Message Area
Error and Warning Messages Area
Mobility Group / dtc Device Status Area
Notification and Update Controls
Status Message Area

This area is at the top of the window. If no Softek Replicator devices have been defined or initialized on the Primary Server, this status message area displays a message to that effect. The status message area displays Updating... while the status update information is being obtained.
www.ibm.com
207
Using dtcmonitortool
Error and Warning Messages

Under the Status Message Area is a window that contains Softek Replicator error and warning messages listed from oldest at the top to the newest at the bottom. Each update cycle obtains new error or warning messages from the Primary Server and the Secondary Servers associated with it. Each message shows:
D D D
The date and time that the message was generated Function name where the message originated Message level, message ID, message text Fatal error messages red Warning error messages black
Messages are shown as follows:

D D
The size of this scrolling list is 200 messages by default; you can change it with the dtcmonitortool command line argument:
-messages count
NOTE: Dtcmonitortool displays the same message multiple times under certain circumstances; these messages are non-critical and may be safely ignored.
Mobility Group / dtc Device Status Area

Below the Error and Warning Messages area is the Mobility Group / dtc Device Status area. If this area of the window is not big enough to hold the status groups and status bars, then it automatically becomes a scrollable subwindow. Each Mobility Group defined on the Primary Server is shown in a bordered box as shown in the following figure. The top line indicates the Mobility Group number (for example, Group 0) followed by the field descriptor.
Figure 16.3: Mobility Group and Device Status Area of dtcmonitortool
The fields shown for the Mobility Group display the following values:
D
Connection The state of the Mobility Groups connection.

H H H
CONNECTED: (green) PMD and RMD daemons for this Mobility Group are connected and active and will transfer any entries in the BAB. ACCUMULATE: (red) Neither the PMD or RMD daemons for this Mobility Group are present. Entries added to the local device are accumulating in the BAB. PMD ONLY: (yellow) The PMD daemon alone is active and is attempting to create a connection with the RMD on the Secondary Server. Entries added to the local device are accumulating in the BAB.
www.ibm.com
208
16
RMD ONLY: (yellow) The RMD daemon for the Mobility Group on the Secondary Server is active without a PMD on the Primary Server and should timeout and die within 30 seconds from the appearance of this indicator.
Mode % Done Softek Replicators current operating state and percentage complete for refresh operations.
H H H H H
REFRESH % Done TRACKING NORMAL BACKFRESH % Done PASSTHRU
D D
Local Read, Loc. Write Local reads and writes in kilobytes per second. Net Actual, Net Effect The actual rate of data flow (amount of data flow without compression) and the effective data rate (amount of data flow with compression and smart refresh taken into consideration) over the network. Entries, % BAB used The number of entries in the BAB for each device and the percentage of BAB the device entries are consuming.
H H H H
Grey No entries in the BAB. Green Percentage in use is between 1 and 50. Yellow Percentage is between 51 and 80. Red Percentage is 80 or more.
Under the Group status is a status line for each dtc device defined for the Mobility Group. On figure: 16.3: Mobility Group and Device Status Area of dtcmonitortool on page 208, the shown status line is for dtc0. The items displayed are the same as those displayed for the Mobility Group, but the values may vary.
Notification and Update Controls

At the bottom of dtcmonitortool are the Notification controls:
You can set these controls to occur if any indicators go into a red state, including receiving Fatal error messages.
www.ibm.com
209
dtcinfo ASCII Report
dtcinfo ASCII Report

On the Primary Server, the dtcinfo command generates an ASCII report for one or more dtc devices indicating the operating mode and performance metrics specific to the BAB.
Figure 16.4: dtcinfo Output Example
Requested BAB size ................ 67108864 (~ 64 MB) Actual BAB size ................... 67108864 (~ 64 MB) Free BAB size ..................... 67108864 (~ 64 MB) Mobility Group 0 (bmaix52 -> 127.0.0.1) Mode of operations.............. Entries in the BAB.............. Sectors in the BAB.............. Sync/Async mode................. I/O delay....................... Persistent Store................ Device /dev/dtc/lg0/rdsk/dtc0: dtc device number............... Local disk device number........ Local disk size (sectors)....... Local disk name................. Remote mirror disk.............. Remote mirror device number..... Read I/O count.................. Total # of sectors read......... Write I/O count................. Total # of sectors written...... Entries in the BAB.............. Sectors in the BAB.............. Device /dev/dtc/lg0/rdsk/dtc1: dtc device number............... Local disk device number........ Local disk size (sectors)....... Local disk name................. Remote mirror disk.............. rtargetlog Remote mirror device number..... Read I/O count.................. Total # of sectors read......... Write I/O count................. Total # of sectors written...... Entries in the BAB.............. Sectors in the BAB.............. 0x2b0001 0x2c0003 81920 /dev/rsourcelog 127.0.0.1:/dev/ 0x2d0001 0 0 0 0 0 0
www.ibm.com
Tracking 0 0 Async 0 /dev/Pstore
0x2b0002 0x2c0001 983040 /dev/rsource1 127.0.0.1:/dev/rtarget1 0x2d0002 0 0 0 0 0 0
210
Chapter 17
Administrative Functions
Working with Throttles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213 Working with Tunable Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .223 Changing Port Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .227 Managing dtc Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228 Managing Mobility Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231 Relocating the Pstore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233 Modifying the BAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233 Working with the Journal File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .234 Using the Checkpoint Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .237 Mounting Mirror Devices on HP-UX and AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .245 Rebooting the Primary Server on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .246 Managing Online Replication on AIX JFS2 File Systems . . . . . . . . . . . . . . . . . . . . . . .246 User-Defined Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .247 Changing the Primary Server into a Secondary Server, and Vice Versa . . . . . . . . . . .248
17
This chapter covers all Softek Replicator administration tasks that are performed after the initial installation and configuration of the software.
Working with Throttles

A throttle is a set of statements you can define to optimize Softek Replicator by monitoring elements, measuring variables, and performing actions based on evaluations. Throttles can be defined during the initial configuration of Softek Replicator and they can be edited at any point during operations. Throttles are defined on a Mobility Group basis and the definitions are stored in the .cfg files. Softek Replicator evaluates throttles periodically, based on a tunable parameter, and performs the specified actions. You can define an unlimited number of throttles for each Mobility Group. Each throttle can run up to 16 ACTIONS when the throttle evaluates TRUE. In addition, there can be up to 16 clauses, or linked tests, in the throttle definition.
D D D D
Determine a timeframe for the throttles to be active. Construct an expression that identifies the variable to be monitored by the throttle and establishes the parameters for that variable. The Configuration Tool provides drop-down menus with lists of options for these sections. Define actions for the throttle to perform when variable measurements fall outside the defined parameters.
Throttle Format
The general throttle format for Softek Replicator is:
THROTTLE[:] <dow-dom> <from-time> <to-time> <throttle-tests> ACTIONLIST[:] <actions> = <directive> ENDACTIONLIST[:]
NOTE: The colon : is optional.
You can break up long lines in throttles into multiple, more readable lines by placing the \ character at the end of a line that is continued on the next line. For example:
THROTTLE: - - - PCTCPU > 30 AND \ ACTUALKBPS > 1000
www.ibm.com
213
The > character means that the <measurement> parameter is greater than the <Value> at the time of evaluation. It is true for every time <measurement> is greater than <Value>. However, if <Value> becomes greater than <Measurement>, this transition is expressed with a T, as follows:
<Measurement> T > 100
It is true when y transitions to be greater than x for the first time only.
Table 17.1: Throttle Fields
Field
<dow-dom> = <Day_Date>
Description Where <Day_Date> can be: D the day of the month or month in numeric values. D the days of the week as a two-character value (mo,tu,we,th,fr,sa,su) D em: maps to the last day of the current month. D -: no day-of-week or day-of-month sensitivity.
NOTE: These values are case-insensitive and duplicates are allowed.
<from-time> = <Time_Value>
Where <Time_Value> can be: D -: no time sensitivity. D a time specified in HH:MM:SS format. For example:
01:00:00 12:59:00 18:15:00
<to-time> = <Time_Value>
Where <Time_Value> can be: D -: no time sensitivity. D a time specified in HH:MM:SS format. For example:
01:00:00 12:59:00 18:15:00
<throttle-tests> = <Expression>
Where <Expression> is: D one or more expression using the following syntax:
<Measurement_Parameter> <Relational_Operator> <Value>
NOTE: If more than one test clause is given, a logical operator (AND or OR) must be specified between each test clauses.
When an expression evaluates to TRUE, it executes the subsequent ACTIONLIST.

NOTE: Expressions are evaluated from left to right and you are allowed up to 16 expressions linked by logical operators in a throttle test.
214
www.ibm.com
17
Measurement Keywords
The following are throttle measurement keywords used by Softek Replicator:
D D
ACTUALKBPS: The number of actual physical kilobytes sent from the PMD to the RMD per second over the network. CHUNKDELAY: The number of milliseconds the PMD sleeps after sending a packet of data to the RMD. This is a tunable parameter for moderating CPU utilization and is separate from the NETMAXKBPS regulating mechanisms in the PMD, but affects the network throughput realized. The minimum and maximum values are 0 and 999, respectively, with the default set to 0 milliseconds. CHUNKSIZE: Maximum size of a single data packet that the PMD sends to the RMD. The default value is 256 KB, with 64 KB as the minimum and 16384 KB as the maximum. COMPRESSION: If set to 1 (ON), moderately effective compression is applied to updates sent by the PMD to the RMD. If set to 0 (OFF), only trivial zero-filled block discard compression is employed. The default is 0. DRIVERMODE: May assume one of the following values:
H H H H H 0 1 2 3 4
D D
= Normal state -- coherent update collection/transmission = Tracking state -- collecting changed block pointers for a Smart Refresh = PassThru mode -- all operations directed to local data devices only
= Refresh state -- performing a refresh on Tracking state data and collecting new updates coherently
= Backfresh state -- updating the local data devices from the mirror devices on the secondary
D D D D D D
EFFECTKBPS: The effective kilobytes per second sent by the PMD to the RMD over the network (decompressed). ENTRIES: The number of I/O updates received by the dtc device driver awaiting transmission to the Secondary Server. MAXSTATFILESIZE: Number of kilobytes that a performance file is allowed to grow to. The minimum and maximum values are 1 and 32000, respectively, with the default set to 64 KB. NETCONNECTFLAG: Set to 1 if the PMD is connected to the RMD for this Mobility Group, 0 otherwise. NETKBPS: Network throughput in kilobytes per second. This is the same as EFFECTKBPS. NETMAXKBPS: Limit imposed on the PMD as to how many kilobytes per second (on average) will be allowed to occupy the network bandwidth between the primary and Secondary Servers. The minimum and maximum values are 1 and 2147483647, respectively, with the default set to -1 (OFF). PCTBAB: Percentage of the BAB that this Mobility Group has in use. This is the same as PERCENTBABINUSE. PCTCPU: Percentage of system CPU that the PMD for this Mobility Group is using. PERCENTBABINUSE: The percentage of occupation of the BAB by entries awaiting transmission to the Secondary Server.
D D D
www.ibm.com
215
D D D
PERCENTDONE: The percentage that a refresh or Backfresh operation has completed for a Mobility Group. PID: Process id (as returned by the ps command) for the PMD for this Mobility Group (or -1 if PMD is not running). STATINTERVAL: Number of seconds between each performance update calculation and throttle evaluation. The minimum and maximum values are 1 and 86400, respectively, with the default set to 10 seconds. SYNCMODE: If set to 0 (OFF), this Mobility Group is in asynchronous transfer mode. If set to 1 (ON), this Mobility Group is in synchronous, or semi-synchronous transfer mode. The default is 0. SYNCMODEDEPTH: The depth of I/O updates that may accumulate in Sync mode before the dtc device driver will block the application. If this is set to 1, then this Mobility Group is in Sync mode. If this value is greater than 1, then this Mobility Group is in semi-synchronous mode. The minimum and maximum values are 1 and 2147483647, respectively, with the default set to 1. SYNCMODETIMEOUT: The number of seconds that the dtc device driver will allow a synchronous or semi-synchronous update to be sent to the Secondary Server, and wait for an acknowledgment of committed update by the RMD before moving on to the next update pending. The minimum and maximum values are 1 and 86400, respectively, with the default set to 30 seconds. TRACETHROTTLE: If set to 1 (ON), every throttle that evaluates to TRUE and the corresponding ACTION executed is logged to syslog and the Softek Replicator UNIX error log. If set to 0 (OFF), these are not logged. The default is 0.
Actions
Throttle actions are written in the following format: <action> = <directive> Where <directive> tell Softek Replicator to do something with optional values or messages.
NOTE: A maximum of 16 actions can be run in an ACTIONLIST.
Action Directives
The following is a list of possible action directives you can use in throttles:
Table 17.2: Action Directives
Action
do console <message> do mail <to-whom> <message> do exec <program-or-script> <arguments> do log <message>
Description Send a message to the Primary Server's console. Send a mail message to a specific person. Run an external program or script. Log a message in the syslog file and in the dtcerror.log file.
216
www.ibm.com
Administrative Functions Table 17.2: Action Directives
17
Action
set <tunable> <value> D D D D D D D D D incr <tunable> <amount-value> decr <tunable> <amount-value>
Description Set a tunable parameter to a specific value.

CHUNKDELAY CHUNKSIZE COMPRESSION MAXSTATFILESIZE STATINTERVAL SYNCMODE SYNCMODEDEPTH SYNCMODETIMEOUT TRACETHROTTLE
Increment a tunable parameter value by a specific amount. Decrement a tunable parameter value by a specific amount.
Action Message Substitutions

Action message substitutions must be in uppercase within ACTION statements and must be specified with a leading %% and trailing %%. They must not be imbedded in another word. An example of an action message substitution is: do mail root effective kbps = %%EFFECTKBPS%%.
Table 17.3: Action Message Substitutions
Action
ACTUALKBPS CFGFILE CHUNKDELAY CHUNKSIZE COMPRESSION CPU DATE DRIVERMODE EFFECTKBPS GROUPNO KBPS MAXSTATFILESIZE NETCONNECTFLAG
Description Physical kilobytes sent over the network per second. Path to the configuration file for Mobility Group. Milliseconds slept between packets sent over net. Maximum size of packets sent to Secondary Server.
1
means best compression employed, 0 means trivial compression employed.
CPU percentage consumed by PMD process. Current date as mm dd, yyyy. Normal, Tracking, Passthru, Refresh, or Backfresh. Decompressed kilobytes received by the RMD. Current Mobility Group number. Effective kilobytes per second transmitted. Maximum size performance files will grow in KBs. 1=PMD is actively in communication with RMD.
www.ibm.com
217
Working with Throttles Table 17.3: Action Message Substitutions (Continued)
Action
NETMAXKBPS PCTWL PERCENTBABINUSE PERCENTDONE PID SLEEP STATINTERVAL SYNCMODE SYNCMODEDEPTH SYNCMODETIMEOUT TIME TRACETHROTTLE
Description Maximum kilobytes per second (average) allowed over net. Percent of the BAB currently in use. Percentage of BAB with data awaiting transfer. Percentage the Refresh or Backfresh has completed. PMDs process id (as from ps command). Current setting for CHUNKDELAY. Number of seconds between throttle evaluations.
1
means Sync or Semi-sync mode, 0 means Async mode.
1 means Sync mode, >1 means Semi-sync mode maximum growth.
Seconds in which Sync mode is honored for each update sent. Current time as hh:mm:ss. Write message to syslog/dtcerror.log for executing actions.
Creating Throttle Definitions

Use the following procedure to create a throttle definition for a Mobility Group.
"
To create a throttle definition:
1. Type: dtcconfigtool to start the Configuration Tool. 2. From the File menu, click Select Mobility Group to display the Select Mobility Group dialog box. 3. Type the Mobility Group number to which the throttle definition will apply. 4. Click OK. 5. Select the Throttles tab.
218
www.ibm.com
Administrative Functions Figure 17.5: dtcconfigtool - Throttles
17
6. Click Throttle Builder to begin creating the throttle definition. 7. From the Evalution Time dialog box, select Only On if you only want the throttle to be active on selected days and time.
NOTE: The default option is set to Always.
8. From the calendar, select the days on which you want the throttle to be active.
Figure 17.6: Setting the Evaluation Time through the Throttle Builder
www.ibm.com
219
9. In the From and To text boxes, time of day for the throttle to be active using the HH:MM:SS format.
NOTE: Accepting default value (- symbol) makes the throttle active continuously during the days that are selected in Step 8.
10. Click Next to view the Build Expressions screen.

Figure 17.7: Throttle Expressions
11. From the Variable drop-down menu, select a variable to measure or monitor. 12. From the Operand drop-down menu, select an operand to defines the relationship between the Variable and the Value. The following operands are available:
H H H H H H H H H H H
>: Greater than >=: Greater than or equal to ==: Equal to !=: Not equal to <: Less than <=: Less than or equal to T>=: Transition is greater than or equal to T==: Transition is equal to T!=: Transition is not equal to T<: Transition is less than T<=: Transition is less than or equal to
NOTE: However, if <Value> becomes greater than <Measurement>, this transition is expressed with a T.
13. In the Value field, type an appropriate value. 14. Click Add to Expression.
NOTE: Use the Clear Expression button to reset the throttle editor.
15. (Optional) From the Boolean Operator list box, select either AND or OR. 16. Click Add to Expression.
220
www.ibm.com
17
17. Repeat Step 11 to Step 14 to add additional expression.

NOTE: You can have up to a maximum of 16 additional expressions.
18. Click Next to access the Build Actions screen.

Figure 17.8: Building Throttle Actions with the Throttle Builder
19. Select an Action from the drop-down menu. 20. Select a Variable from the drop-down menu. 21. Type an Argument in the field. 22. Click Add to Action List.
NOTE: Each throttle can have up to 16 defined actions.
23. Repeat Step 19 to Step 22 to add additional Actions. 24. Click Done.
"
To initiate the new throttle definition:
1. Stop all applications and unmount any file systems accessing the dtc devices in the affected Mobility Group. 2. Type: killpmds -g <group#> to stop the PMD. Where <group#> is the Mobility Group whose throttle you have defined. 3. Type: dtcstop -g <group#>. 4. Type: dtcstart -g <group#>. 5. Type: launchpmds -g <group#> to start the PMD. 6. Start any user applications or mount file systems using devices in the Mobility Group.
www.ibm.com
221
Throttle Examples
This section includes throttle examples to illustrate the value of throttles in a Softek Replicator production environment. These throttle examples are provided for illustration purposes only.
CAUTION: When defining throttles, you must balance all critical business factors that may affect applications and apply appropriate throttle conditional tests and actions to cover all situations that might arise. Carefully research and test throttles before implementing them in a production environment.
Throttle to Regulate Maximum Network Bandwidth during Peak Business Hours

The following throttle restricts the maximum network utilization to 300 KBps during the standard business day, but lift the restriction outside those hours.
THROTTLE: MO,TU,WE,TH,FR 08:00:00 17:00:00 \ NETMAXKBPS != 300 ACTIONLIST: ACTION: set NETMAXKBPS 300 ENDACTIONLIST: THROTTLE: MO,TU,WE,TH,FR 17:01:00 23:59:59 \ NETMAXKBPS != 1000000 ACTIONLIST: ACTION: set NETMAXKBPS 1000000 ENDACTIONLIST: THROTTLE: MO,TU,WE,TH,FR 00:00:00 07:59:59 \ NETMAXKBPS != 1000000 ACTIONLIST: ACTION: set NETMAXKBPS 1000000 ENDACTIONLIST: THROTTLE: SA,SU - - NETMAXKBPS != 1000000 ACTIONLIST: ACTION: set NETMAXKBPS 1000000 ENDACTIONLIST:
222
www.ibm.com
17
Working with Tunable Parameters

Tunable parameters allow you to control aspects of Softek Replicators function, such as the maximum size of a data packet sent from the Primary Server to the Secondary Server. Most of these parameters are saved in the Pstore, and are frequently evaluated by the PMD. Some tunable parameters are only processed by the throttle, since they are part of a throttle definition. Tunable parameters are an optional part of the configuration. If you choose not to define these parameters, the default values are used. Tunable parameters can be set during initial configuration or later using the dtcset command. Tunable parameter changes become effective within STATINTERVAL seconds (default is 10).
Table 17.4: Tunable Parameters
Tunable Parameter CHUNKDELAY
Description Sets the amount of time, in milliseconds, that the PMD waits before attempting to send a chunk of entries from the BAB across the network. CHUNKDELAY regulates the performance of a PMD. Use this parameter to regulate the amount of network bandwidth consumed by Softek Replicator. Setting CHUNKDELAY to a value greater than 0 ms causes the PMD to wait the designated amount of time before attempting to transfer data. The default value is 0 ms. Sets the amount of information, in kilobytes, that the PMD reads from the BAB at a time and sends to the RMD. CHUNKSIZE helps optimize disk access while Softek Replicator is performing a Refresh. Each operation reads the amount of data defined by CHUNKSIZE. The default value is 256 KB.
NOTE: You must run killpmds before and launchpmds after changing the value. CAUTION: When setting this parameter, any I/O operation that reads or writes a data amount greater than the value of CHUNKSIZE may cause the BAB to overflow.
CHUNKSIZE
COMPRESSION
Sets whether to use data compression or not. The default is OFF, which indicates that data is transferred in its original, non-compressed state. If set to ON, then data is compressed before being transferred and decompressed before being written to the journal or mirror device on the Secondary Server. Compression is a means to optimize network throughput.
NOTE: You must run killpmds before and launchpmds after changing the value.
JOURNAL
Sets the Journal feature ON or OFF. For complete details on the Journal feature, refer to Journal-less Processing on page 236.
www.ibm.com
223
Working with Tunable Parameters Table 17.4: Tunable Parameters (Continued)
Tunable Parameter MAXSTATFILESIZE
Description Governs the size, in kilobytes, for the performance file when a LOGSTATS is set to Y. Once a performance file reaches the size set by this parameter, the file is closed and renamed with a .1 suffix. Only one generation of files is saved and the original file is emptied and prepared to collect new performance metrics.
NOTE: This parameter only takes effect after stopping and restarting the Mobility Group.
MAXSTATFILESIZE allows you to establish the amount of disk space for performance statistics. The frequency at which these files are updated is determined by STATINTERVAL. MAXSTATFILESIZE also allows you to save a larger number of statistics, spread out over a long period of time (by increasing the value of this parameter) if the goal is to decipher trends over a long operating period. Or the operator can choose to limit the statistics (by reducing the value of this parameter) to only a short period of time. The default value is 64 KB. NETMAXKBPS Regulates the network bandwidth (in kilobytes per second) by inserting incremental 0.5 second delays between CHUNKSIZE packets of data, until the proper delay is achieved.The default is -1 (OFF).
NOTE: You must run killpmds before and launchpmds after changing the value.
STATINTERVAL
Defines the frequency, in seconds, at which Softek Replicator evaluates throttles, tunable parameters, and performance metrics. The default value is 10 seconds. Determines whether dtc devices in a Mobility Group require a synchronous and acknowledged update from the Secondary mirror device with each I/O update to the local data device. When set to 1 (ON), all I/O operations are performed on both the local data and mirror devices disks at all times. Setting this parameter to 1 (ON) affects application performance because of round trip network time added to each I/O. Setting this parameter to 0 (OFF) allows transfers to be made asynchronously. The default value is 0 (OFF).
NOTE: You must run dtcstop before and dtcstart after changing the value.
SYNCMODE
SYNCMODEDEPTH
Sets the number of I/Os that can accumulate in the BAB before Sync mode is triggered. If SYNCMODEDEPTH is set to 1 and SYNCMODE is set to 1 (ON), then dtc devices in the Mobility Group are in full Sync mode. If SYNCMODEDEPTH is set to a value larger than 1, then the Mobility Group is in Async mode, but the mirror devices on the Secondary Server are no more than the specified number of entries behind the local data devices. The default value is 1.
224
www.ibm.com
Administrative Functions Table 17.4: Tunable Parameters (Continued)
17
Tunable Parameter SYNCMODETIMEOUT
Description Establishes the amount of time, in seconds, that the dtc device driver waits for a Sync mode update to complete before returning control. If the PMD does not receive an acknowledgment, the status of its Mobility Group changes to Tracking state automatically. SYNCMODETIMEOUT keeps the application from freezing up if there is a high load burst on either the network or the Secondary Server. The default value is 30 seconds.
TRACETHROTTLE
syslog (/var/adm/messages in syslog/syslog.log in HP-UX,
Determines whether throttle information is written to the AIX and Solaris, /var/adm/ and /var/log/messages in Linux) and to the error log file (/var/opt/SFTKdtc/ dtcerror.log) when a throttle expression evaluates TRUE. If this parameter is set to either on, ON, or 1, then each ACTION in the ACTIONLIST for the throttle is written to these files along with the current values of variables included in the throttle definition. This provides you with a method of verifying that a throttle executed properly. The default value is OFF.
a. LOGSTAT is a parameter indicating whether performance statistics should be logged in the performance files. It is always set to Y and cannot be changed.
Changing Tunable Parameters

Tunable Parameters can be set at any time using the dtcconfigtool or the dtcset commands.
"
Using dtcset to change the tunable parameters: Where:

H H <group#>
1. Type: dtcset -g <group#> <tunable_parameter>=<value> is the Mobility Group number. is the tunable parameter that you want to set and the
<tunable_parameter>=<value>
value that it should take.

"
Using dtcconfigtool to change the tunable parameters:
1. Start dtcconfigtool. 2. From the File menu, click Select Mobility Group to display the Select Mobility Group dialog box. 3. Type the Mobility Group number to which the tunable parameter will apply. 4. Click OK. 5. Select the Tunable Parameters tab.
www.ibm.com
225
Working with Tunable Parameters Figure 17.9: Tunable Parameters Tab in dtcconfigtool
6. Edit the definitions. Refer to table: 17.4: Tunable Parameters on page 223 on details of each tunable parameter. 7. From the File menu, select Save Changes 8. From the File menu, select Exit. 9. Type: killpmds -g <group#>. Where <group#> is the Mobility Group whose tunable parameters you have changed. 10. Unmount any file systems and stop any applications accessing the dtc devices in the Mobility Group. 11. Type: dtcstop -g <group#>. 12. Type: dtcstart -g <group#>. 13. Type: launchpmds to restart the PMDs. 14. Remount the file system and restart applications.
226
www.ibm.com
17
Changing Port Numbers

Softek Replicator master daemon (in.dtc) uses Port 575 by default, to communicate with another Softek Replicator server. Softek Replicator uses Port 16386 (instead of Port 576) to communicate with the DMC Collector. The DMC Collector uses Port 16642 (instead of 577) to communicate with Softek Replicator. This may cause issues when configuring firewalls, or enabling Server-to-Server and DMC Collector-to-Server communication.
Changing the Primary Server Master Daemon Port Number

Use the following procedure to change the Master Daemon Port Number on the Primary Server.
NOTE: The port number on the Secondary Server needs to match the one on the Primary Server. For more information, refer to Changing the Secondary Server Master Daemon Port Number on page 228.
"
To change the port number on the Primary Server:
1. From the System menu, select TCP Settings to display the TCP Setting dialog box.
Figure 17.10: Configuring AIX. HP-UX, Linux, and Solaris - TCP Settings
2. In Listen Port, type the port number to use. 3. (Optional) In the Socket Buffer Size, type a value (in bytes) to set the TCP/IP send and receive buffer size. 4. Click OK to exit the Configuration Tool. 5. Copy the configuration file (p000.cfg) from the Primary to the Secondary Server as described in Step 2: Distributing the Configuration Files on page 110. 6. Type dtcstop. 7. Type dtcstart so that the new configuration file takes effect. 8. Type killdtcmaster. 9. Type launchdtcmaster to restart the master daemon.
www.ibm.com
227
Managing dtc Devices
Changing the Secondary Server Master Daemon Port Number

If you change the port number on the Primary Server, you must also change it on the Secondary Server.
"
To change the port number on the Secondary Server:
1. In the /etc/services file, locate the line with in.dtc. 2. Change the port number so that it matches the port number on the Primary Server. 3. Save the file. 4. Type killdtcmaster. 5. Type launchdtcmaster to restart the master daemon.

The dtc device is the means by which applications or file systems interact, access, or store information within Softek Replicator. A dtc device provides the mapping to and management of a specific local data device and the corresponding mirror device(s). During configuration, you give each dtc device instance a unique name within a Mobility Groupfor example, dtc0, dtc12, dtc93. Device names usually begin with 0 and are incremented by 1. Dtc devices appear as volumes to the kernel, so a dtc device accepts and handles any request that can be made to a normal volume or fixed-size volume, such as create and mount a file system, or allocate DBMS tablespace. Dtc devices are not shared by the Primary and Secondary Servers. Rather, data is mirrored across the network from the local data devices to the mirror devices. If you want the Secondary Server to assume all activities if the Primary Server fails, you must install the application software on both systems. In a standard Softek Replicator configuration, applications on the Secondary Server must not be executed until the system is prepared to act as the application server. The exception is when Checkpoint is active. For more details on preparing for a changeover to the Secondary Server, see Recovering Data on page 253.
Adding/Modifying dtc Devices

"
To add or modify dtc devices while Softek Replicator is running:
1. Type: dtcconfigtool to start the Configuration Tool. 2. From the File menu, select the Mobility Group for which you want to add or modify a dtc device. 3. Select the dtc Devices tab. 4. Choose the dtc device to be modified from the device list on the left side of the screen. The selected device is highlighted and the device information is displayed in the fields to the right. 5. Select a new device from the drop-down menus or enter a new value.
228
www.ibm.com
17
6. To add a new dtc device, select the Create new device button and enter the local data device and mirror device definitions. 7. From the File menu, select Save Changes. 8. If the local data device is already mounted, a dialog box is displayed to confirm the unmount for the device. Select Yes to unmount.
NOTE: : Steps 8 and 9 Softek Replicator does not perform Steps 8 and 9, if ONE of the following is true: The selected Mobility Group is started The data device is already registered to the selected Mobility Group
9. If the local data device is registered for auto-mount in system mounting file, a dialog box is displayed to confirm the change of mount information. To confirm the change, select Yes.
NOTE: Step 9 is only performed under the following conditions: AIX When the data device of the specified Mobility Group exists; the mount stanza of that record is defined to [true] [automatic]; and the dev stanza or the log stanza agrees with the device name. HP-UX When the data device of the specified Mobility Group exists, and the file system type of that record is other than [swap] [swapfs] [dump] [ignore] [nfs]. Linux When the data device of the specified Mobility Group exists, and the file system type of that record is other than [swap] [proc] [tmpfs] [devpts] [dump] [ignore] [auto] [nfs]. Solaris When the data device of the specified Mobility Group exists, and the automount is specified to yes. During the update process, any data that is changed is first converted into a comment line and (Softek Replicator-specific) character string, and the date of the update is added to the end of the record.
10. Copy the configuration files over to the Secondary Server and rename them. 11. Type: killpmds -g <group#> 12. Unmount any file systems and stop any application from accessing the dtc devices in the Mobility Group. 13. Type: dtcstop -g <group#> 14. Type: dtcstart -g <group#>
NOTE: Remember that the configuration files are only processed by the dtcstart command. Changes to the configuration do not take effect until the Mobility Group is stopped and restarted.
15. Mount file systems or start applications. 16. Type: launchpmds -g <group#> 17. If new devices have been added to the Mobility Group, type: launchrefresh -g <group#>
-f
www.ibm.com
229
Deleting dtc Devices

Use the following procedure to remove dtc Devices.
"
To remove dtc Devices:
1. From the Primary Server, type: dtcconfigtool. 2. From the File menu, select the Mobility Group to modify. 3. Select the dtc Devices tab. 4. Choose the dtc device to be deleted from the device list window. 5. Select Delete Device. 6. From the File menu, select Save Changes. 7. Copy the updated p###.cfg configuration file from the primary to the Secondary Server, and rename it s###.cfg. 8. Type: killpmds -g <group#> 9. Unmount any file systems and stop any application from accessing the dtc devices in the Mobility Group. 10. Type: dtcstop -g <group#> 11. Type: dtcstart -g <group#>
NOTE: The changes to the Mobility Group do not take effect until the Mobility Group has been stopped and restarted - when dtcstart creates a new .cur file.
12. Mount file systems or start applications. 13. Type: launchpmds -g <group#>
230
www.ibm.com
17
Managing Mobility Groups

A Mobility Group is a collection of local partitions or devices treated as a single unit by Softek Replicator. Each Mobility Group operates with its own independent PMD/RMD daemon pair. Mobility Groups allow time-ordered writing coherence between member dtc devices, and complete operational and state independence between Mobility Groups.
Adding a Mobility Group

If you are using the Softek Data Mobility Console to create and configure Mobility Groups, you must initialize the BAB using the dtcagentset -b command and then load the dtc driver using the launchagent command, in order to start the Mobility Group for the first time; refer to dtcagentset on page 171.
CAUTION: If you experience a file table overflow error when attempting to start a Mobility Group or launch a PMD process on HP-UX servers, you may need to increase the value of the maxfiles_lim and nfile tunable parameters. These two HP-UX kernel tunable parameters determine the maximum number of files that can be opened per-process and system-wide, respectively.
"
To add a Mobility Group:
1. From the Primary Server, type: dtcconfigtool. 2. From the File menu, select New Mobility Group. 3. Use the arrows to select the Mobility Group number for editing. 4. Select the Systems Tab. 5. Under Primary System, the primary system Hostname or IP Address is filled in automatically with the name of the server you are running on. Accept this, or enter a valid system name or IP address in this field. 6. Define the Persistent Store Device (Pstore) for the Mobility Group. You can define a unique Pstore for each Mobility Group. Make sure that the Pstore partition is clean and available. 7. Under Secondary Server, specify either a resolvable system name or IP address for the secondary or target system. 8. To configure a local loopback configuration (that is, where both the primary and secondary are on the same system), specify localhost or 127.0.0.1 for the primary and Secondary Servers.
TIP: Keep in mind that each Mobility Group has an independent connection; therefore, you can define a different Secondary Server for each Mobility Group.
9. If you specify a port number other than the default of 575 on the Secondary Server, you must enter that number in the Port field. This is the port you are connecting to on the Secondary Server. To change this port number, see Changing Port Numbers on page 227. You can also change the port that the Softek Replicator master daemon listens on as described in Changing Port Numbers on page 227.
www.ibm.com
231
Managing Mobility Groups
NOTE: You can define Mobility Groups that connect to different Secondary Servers with different port numbers. Use this field to specify those secondary port numbers.
10. In Journal Directory, specify any writable directory on the Secondary Server for journal files.
11. After you complete the configuration, make sure you specify this directory in the system so that the configuration file is installed automatically at boot up. For AIX, see Creating and Mounting New Journaled File Systems on dtc Devices on page 116. For HP-UX, Linux, and Solaris, see Modifying the system mounting file (/etc/fstab or /etc/vfstab) on page 123. 12. If needed, change the Allow Chaining option (No is the default) for each Secondary Server defined. For more information on chaining, refer to Chaining Configuration on page 95.
CAUTION: For Solaris Only Softek Replicator will generate errors during the replication process if you are using Solaris Solstice Disk Suite (SDS) on the Secondary Server, with partitions starting at block 0 of cylinder 0. The SDS partitions must start at cylinder 1, or greater. This only affects the target devices and is a standard Solstice Disk Suite recommendation. Alternatively, with Solaris 9 or higher, you could use the soft partition from Solstice, which resolves this problem and has additional features to overcome constraints with hard partitioning.
Deleting Mobility Groups

Use the following procedure to delete a Mobility Group.
"
To delete a Mobility Group:
1. Type: killpmds -g <group#> to stop the Mobility Group you want to delete. 2. Unmount the file systems that are mounted on any dtc devices belonging to the Mobility Group. 3. Type: dtcstop -g <group#>. 4. Remove the appropriate p###.cfg configuration file (where ### is the number of the Mobility Group to be deleted). For example: On AIX: rm /etc/dtc/lib/p001.cfg On HP-UX, Linux, and Solaris: rm /etc/opt/SFTKdtc/p001.cfg The next time you reboot the system, the blocks used by the deleted Mobility Group will be available.
232
www.ibm.com
17
Relocating the Pstore

To relocate the Pstore volume, run dtcconfigtool and select a new Pstore volume from the drop-down menu for each Mobility Group, for which the Pstore is being modified. 1. Shut down any processes and unmount any file systems accessing the dtc devices in the affected Mobility Groups. 2. Type: killpmds -g <group#> for each Mobility Group affected by the Pstore modification. 3. Type: dtcstop -g< group#> for each Mobility Group affected by the Pstore modification. 4. Type: dtcconfigtool to start the Configuration Tool. 5. Select the Systems tab, and choose a new Pstore volume from the drop-down menu. 6. From the File menu, select Save Changes. 7. Type: dtcstart -g <group#> for each modified Mobility Group. 8. Type: launchpmds -g <group#> to restart the Mobility Groups. 9. Remount file systems and restart applications.
Modifying the BAB

To resize the BAB, you must reload the dtc driver. Before doing this, shut down any processes accessing the dtc devices, unmount any file systems residing on the dtc devices, and stop all Mobility Groups using the dtcstop command. All Softek Replicator daemons are temporarily stopped while this reload takes place.
Modifying the BAB Using dtcconfigtool

"
To modify the BAB using dtcconfigtool:
1. Type: dtcconfigtool to start the Configuration Tool. 2. From the Systems menu, select BAB. 3. Use arrows to select designated amount of memory or type enter a new amount. 4. Click OK. 5. If the newly requested memory amount cannot be allocated, reboot the system. 6. Restart applications.
www.ibm.com
233
Working with the Journal File Directory
Modifying the BAB Manually

"
To modify the BAB manually:
1. Unmount file systems and kill applications accessing dtc devices. 2. Type: killdtcmaster.
NOTE: Unload the dtc driver by typing: /sbin/rmmod/sftkdtc
3. Type: dtcinit -b <memory in MB> Where <memory in MB> is size of the memory you want to allocate in MB. In Linux, this command modifies /etc/modules.conf so that after the module is reloaded, this system reads new values and initializes the BAB.
NOTE: The only way to specify a BAB size outside the standard selections in the Configuration Tool dialog is to type dtcinit -b.
4. Type: dtcinfo to verify that memory was acquired. 5. Type: launchdtcmaster. 6. Remount filesystems and restart applications. If the BAB overflows during a Smart Refresh, Softek Replicator is placed in Tracking state. You must run the launchrefresh command to restart the Mobility Group back to Normal state. If the BAB overflows frequently, consider resizing the BAB and adding network resources.

The journal file directory is used during a Smart Refresh operation, to store updates which, if applied to the mirror device, would place it in an incoherent state. The journal file directory is also used to store updates while the checkpoint is active on the mirror devices. These changes are eventually applied to the mirror device to maintain coherence. These entries are saved to the mirror device only after the PMD indicates that all updates have been transferred.You can set the journal file directories to any writable directory during configuration.
A new journal file is created each time there is a state change that causes the data being transferred to go from coherent to incoherent. Softek Replicator uses the following naming convention for these files:
j###.###.c j###.###.i
(for coherent state) (for incoherent state)
The j indicates that this is a journal file, the first three numbers represent the Mobility Group to which the journal file belongs, and the second set of numbers are sequence numbers of journal files for each Mobility Group. For example:
j001.002.c
(second journal file for Mobility Group 1 in a coherent state)
234
www.ibm.com
17
Softek Replicator allows a maximum of 999 journal files, at approximately 500 MB per file, for each Mobility Group.
Resizing the Journal File Directory

Use the following procedure to resize the Journal File directory.
"
To resize the Journal File directory:
1. From the Primary Server, type: killpmds 2. Type: killdtcmaster. 3. On the Secondary Server, Type: killdtcmaster. 4. Verify that the Journal File directory is empty. If there are journal file(s); move them to another directory. 5. Mount a larger journal file directory. 6. If you moved existing journal files in Step 4, and the Journal file directory has the same name, copy these files back into this directory to preserve data consistency. 7. From the Primary and Secondary Servers, type: launchdtcmaster. 8. From the Primary Server, type: launchpmds. Softek Replicator automatically launches a Smart Refresh to synchronize the primary and Secondary Servers, applies the new journal file to the mirror device, updates journals during the Smart Refresh, and then goes to Normal state.
Turning Journal Operations On or Off

Use the following procedure to turn the Journal feature on or off.
" D
To turn the Journal feature on or off: The Journal feature can be turned off for each Mobility Group in the following ways:
H
In the dtcconfigtool, Tunable Parameters tab, by selecting the ON or OFF option button. In the Windows Softek Data Mobility Console, in the Group properties dialog box, Tunables tab, by selecting or deselecting the Do not use Journals checkbox, if you are managing your data replication using the Softek Data Mobility Console. By using the dtcset -g # -journals ON|OFF command.
You must specify the Journal directory for each Mobility Group.
NOTE: Turning the Journal feature on and off requires running the killpmds command, followed by the launchpmds command after the change is made. A prompt appears in the command line or dtcconfigtool if PMDs and RMDs are running when this is changed.
www.ibm.com
235
Journal-less Processing
The following operations are affected by the Journal feature:
D D D
Smart Refresh Checksum Refresh Checkpoint
Smart Refresh / Checksum Refresh

When Journal-less option is in effect, disk data transferred from the source device will be written directly to the target device, without storing it in Journals.
Checkpoint
When Journal-less option is in effect, CHECKPOINT operates in the following way:
D
Checkpoint ON (by typing: CHECKPOINT -g ### | a ON):

H H H H H H H cp_pre_on_p###
will run will run
A token will be put in the BAB

cp_post_on_p###
All subsequent I/O to the primary disks will be TRACKED in the Pstore The Softek Data Mobility Console will display the mode as CHECKPOINT When the token reaches the RMD, the J###001.p file will be placed in the Journal directory The cp_post_on_s### script will run
NOTE: You will not be able to launch a Full Refresh, Smart Refresh, or Checksum Refresh for this Mobility Group. An error message will display to indicate that the refresh operation is rejected. This is to prevent any loss of changes since no journals are used to hold refreshed data on RMD side. Mobility Groups that are in Checkpoint mode and Journal-less mode do not use the BAB.
D
Checkpoint OFF (by typing: CHECKPOINT -OFF):

H H H H H cp_pre_off_s###
will run
The j###001.p file will be deleted A Smart Refresh will be issued and this will apply changes directly to the secondary (mirror) devices since the Journal state is still OFF. The Mobility Group will transition to Normal Connected mode. A message will be logged in the event log to indicate that journaling has been turned off for a Mobility Group. Similarly, a message will indicate the result of the command and any error that occurred as a result of the command. The Softek Data Mobility Console will indicate in the Group Details tab that the Mobility Group has returned to Normal state.
236
www.ibm.com
17
Using the Checkpoint Feature

A checkpoint is a frozen system snapshot of a data set at a specific point in time. A checkpoint allows you to take the secondary mirror off-line from Softek Replicator and make it available for read-only access by other applications. Checkpointing ensures that the data on the mirror device is in a known, usable state and that no updates are lost while the mirror is not available to Softek Replicator. While the checkpoint is in effect, all transactions targeted at the mirror devices are diverted to the journal on the Secondary Server.
TIP: You can use the dtccheckpoint command on either the primary or Secondary Servers with the same result. NOTE: Mount file systems on checkpointed mirror devices in read-only mode. Likewise, if you run a file system check (fsck) on the mirror devices, it must be a report-only, non-modifying fsck.
Examples of Checkpoints
For AIX:
aixserv# fsck -n /dev/lv12 aixserv# mount -o ro /dev/lv12 /mirror_mount
For HP-UX:
hpserv# fsck -n /dev/vg00/lv12 hpserv# mount -o ro /dev/vg00/lv12 /mirror_mount
For Linux:
linuxserv# linuxserv# /sbin/fsck /dev/hda5 mount -o ro /dev/hda5 /mirror_mount
For Solaris:
sunserv# fsck -n /dev/rdsk/cotod0s# sunserv# mount -o ro /dev/rdsk/c0t0d0s# /mirror_mount
When the checkpoint has been turned off, the RMD daemons re-acquire their exclusive open lock on the mirror devices and immediately apply the accumulated transactions from the secondary journals, bringing the mirror devices up-to-date with the primary, local data devices.
Checkpointing Procedure Example on AIX JFS

1. Type: dtccheckpoint -g <group#> -on to turn on Checkpoint on the Primary Server. 2. On the Secondary Server, perform the following steps: a. Type: dtcjfspremount b. Type: fsck c. Type: mount d. Type: umount e. Type: dtcjfspostmount 3. Type: dtccheckpoint -g <group#> -off to turn off Checkpoint on the Primary Server.
237
NOTE: If you do not execute dtcjfspostmount and execute dtccheckpoint -off, checksums for the primary and Secondary Servers will not match even after a Smart Refresh. Also, running fsck on the Secondary Server will cause a checksum mismatch. You must execute a Full Refresh to synchronize. For further details on the dtcjfspremount and dtcjfspostmount commands, refer to Chapter 15: Commands.
Checkpoint Processing
The dtccheckpoint command is used to activate the checkpoint feature for Softek Replicator. This command can be run on either the primary or Secondary Server with the same result. Before typing the dtccheckpoint command with the appropriate options, applications on the primary must be quit so that all data can be flushed to the disk. You can do this either manually or via a checkpoint shell script that the checkpoint command calls automatically. For more information on preparing both systems for a checkpoint operation, refer to Checkpoint Shell Scripts on page 240.
Checkpoint ON
1. You can run the dtccheckpoint -on command from either the primary or Secondary Server. If you have defined checkpoint scripts, the /etc/opt/SFTKdtc/ cp_pre_on_p###.sh script is run at this time. If this script file is not defined, you must manually quit the applications on the primary and force all pending updates to the disk.
NOTE: Before entering the dtccheckpoint -on command on the Primary Server, you must stop the database, and unmount and remount all applications accessing dtc devices in the Mobility Groups being placed into checkpoint. This ensures that application data is flushed to disk and that the mirror device is in a known state.
2. Once all data has been flushed to disk and to the BAB on the primary, a flag is set in the BAB. This flag alerts the RMD to start journaling all subsequent entries received by the secondary. 3. Operations that were interrupted on the Primary Server in order to flush the data should be brought back into a normal operating state. You can create an /etc/opt/SFTKdtc/ cp_post_on_p###.sh script to automate this process. Journal feature turned OFF: If checkpoint is ON and Journals are NOT used, you will not be able to launch a Full Refresh, Smart Refresh, or Checksum Refresh for this Mobility Group. An error message will display to indicate that the refresh operation is rejected. This is to prevent any loss of changes since no journals are used to hold refreshed data on the RMD side. Journal feature turned ON: On the Secondary Server, updates are journaled if Journals are used, and the RMD has released its exclusive lock on the mirror devices, which can now be mounted and accessed in a read-only fashion by applications. You can create an /etc/opt/SFTKdtc/cp_post_on_s###.sh script to automate mounting or accessing these mirror devices on the Secondary Server. This is a useful way to back up mirror devices without interrupting the Primary Server for a prolonged period. You will not be able to launch a Full Refresh for this Mobility Group due to possible space shortage on the journal partition. An error message will be displayed to indicate that the Full Refresh operation is rejected. However, Smart Refresh and Checksum Refresh operations are allowed.
NOTE: Mobility Groups that are in Checkpoint mode and Journal-less mode do not use the BAB.
238
17
Checkpoint OFF
1. Before typing the dtccheckpoint -off command, file systems or applications accessing the mirror devices on the Secondary Server must be stopped and the file systems unmounted. (If the mirror device(s) is mounted, checkpoint is NOT turned OFF). The /etc/opt/SFTKdtc/cp_pre_off_s###.sh script can be used to automate this process. 2. The mirror devices are re-acquired with an exclusive lock by the RMDs and the journal entries that have been accumulating in the secondary journal file directory are now applied to the mirror devices as appropriate, if the Journal feature was turned ON.
NOTE: A mirror device is in an incoherent but recoverable state while these journal entries are being applied.
Checkpoint Command/Action Matrix

Table 17.5: Checkpoint Command/Action Matrix
Checkpoint ON Commands
launchrefresh -f
Checkpoint OFF PMD Full Refresh RMD Full Refresh Journal applied Mirroring Full Refresh Mirroring Smart Refresh Journal applied Mirroring Smart Refresh Mirroring Check. Refresh Journal applied Mirroring Check. Refresh Mirroring Smart Refresh Journal applied Smart Refresh Mirroring
239
Journal PMD ON No action RMD Not reachable
OFF
launchpmds launchrefresh
No action Smart Refresh
Not reachable Smart Refresh Journal applied Not reachable Check. Refresh Journal applied Not reachable Smart Refresh Journal applied Not reachable
Full Refresh Smart Refresh
ON
OFF
No action
Smart Refresh Check. Refresh
launchrefresh -c
ON
Check. Refresh
OFF
No action
Smart Refresh Smart Refresh
BAB Overflow Recovery (Smart Refresh)
ON
Smart Refresh
OFF
Not reachable
Smart Refresh
www.ibm.com
Considerations and Limitations

D
Checkpoint operations are not immediate. The checkpoint -on token is queued in the BAB on the primary with data update transactions. Only after the RMD receives this token does the checkpoint go into effect. When the amount of data in the BAB awaiting transfer is large, there may be a noticeable delay before the Checkpoint takes effect. If a failure occurs on the Primary Server or an explicit dtcstop is entered to a Mobility Group with a pending Checkpoint token in the BAB, the token is lost when the Mobility Group is rebooted/restarted and the Checkpoint operation is cancelled. After the dtccheckpoint command has been entered, but before the dtccheckpoint token is transmitted to the Secondary Server, there is no graceful way to cancel the requested Checkpoint operation. The operation is irrevocable at this point without killing the PMD/ RMD daemons, deactivating the Mobility Group by typing an dtcstop command, clearing the BAB with the dtcoverride command, or re-initializing the Primary Server. Any one of these actions may require refresh operations to bring the systems back into synchronization. Activating or deactivating Checkpoint can only occur if the Mobility Group is in Normal state. The Mobility Group must remain in Normal state until the Checkpoint token is received by the RMD on the secondary machine at which time it is acceptable for the system to go to a new operating mode. If the BAB fills up and forces the Mobility Group into Tracking or Refresh state while the checkpoint token is queued for transfer, the operation is cancelled. Note that, if defined, the cp_pre_on_p###.sh script has been run at this point. Therefore, you must manually execute the cp_post_on_p###.sh script on the primary to place applications in a pre-checkpoint state. You should not manually create or remove the .p file. The Checkpoint state may be lost by deleting the .p file mistakenly, and new updates to the host will be incorrectly handled. The RMD will not delete the .p files unless you turn the Checkpoint operation OFF. Should your application modify the mirror devices while in Checkpoint mode, a Full Refresh is required. There is no way around this.
D D D
Checkpoint Shell Scripts

NOTE: Replace /etc/opt/SFTKdtc with /etc/dtc/lib in the shell script. For example: /etc/dtc/
lib/cp_pre_on_p###.sh
To automate the process that ensures data set consistency at the time of checkpointing, you may choose to implement one or more optional checkpoint shell scripts. For a data set to be usable on the Secondary Server in checkpoint mode, the data must be complete and in a consistent state. This often means that applications such as databases must be momentarily shut down to bring the data set to a known, good state. The checkpoint facility does this automatically for file systems on the Primary Server by flushing any uncommitted blocks in the buffer cache to disk before entering into checkpoint mode. Once the dtccheckpoint -on command adds the checkpoint -on token to the BAB, then applications on the Primary Server can resume data updating activities.
240
www.ibm.com
17
Primary Server Scripts

The process of preparing the data set for checkpointing, placing the token in the BAB and resuming normal application operations on the Primary Server should take only a few seconds (or a few minutes for a highly complex database environment). This process can be automated using optional shell scripts for the Primary Server. The location and names of the following shell scripts on the Primary Server are predefined and cannot be changed:
D /etc/opt/SFTKdtc/cp_pre_on_p###.sh is run on the Primary Server (denoted by the p### naming convention) before activating the checkpoint to prepare the specified Mobility Group. It is automatically called when the dtccheckpoint -on command is run. /etc/opt/SFTKdtc/cp_post_on_p###.sh is run on the Primary Server after the dtccheckpoint -on command has placed a marker in the BAB. This marker notifies the
RMD of the checkpoint request. At this point, all future updates to the RMD are sent to the journal file area on the Secondary Server, until the dtccheckpoint -off token is received. Use this script to bring applications back online on the Primary Server.
NOTE: The # symbol corresponds to the Mobility Group number to which this shell script applies. For instance, for Mobility Group 12, the cp_pre_on_p012.sh shell script would be run before creating the checkpoint.
The figure: 17.11: Script Sample of /etc/opt/SFTKdtc/cp_pre_on_p012.sh for Primary Server and the figure: 17.12: Script Sample of /etc/opt/SFTKdtc/cp_post_on_p012.sh for Primary Server show sample Primary Server checkpoint shell scripts.
Figure 17.11: Script Sample of /etc/opt/SFTKdtc/cp_pre_on_p012.sh for Primary Server
#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_on_p012.sh ***** echo ---------- date ---------- >> /usr/tmp/ xpoint.log echo starting cp_pre_on_p012.sh >> /usr/tmp/ xpoint.log /oracle/script/shutdown_database.sh echo database has been shut down >> /usr/tmp/ xpoint.log echo >> /usr/tmp/xpoint.log exit 0
www.ibm.com
241
Using the Checkpoint Feature Figure 17.12: Script Sample of /etc/opt/SFTKdtc/cp_post_on_p012.sh for Primary Server
#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_p012.sh ***** echo ---------- date ---------- >> /usr/tmp/ xpoint.log echo starting cp_post_on_p012.sh >> /usr/tmp/ xpoint.log /oracle/script/startup_database.sh echo database has been restarted >> /usr/tmp/ xpoint.log echo >> /usr/tmp/xpoint.log exit 0
NOTE: These scripts must be executable (have an x permission attribute) and return an exit code of 0 to be properly executed by the dtccheckpoint command. An error message is issued if there is any problem with the execution of these shell scripts.
Secondary Server Scripts

To automate the process of checkpointing information as it becomes available on the Secondary Server, you may choose to implement one or both optional shell scripts that are run on the Secondary Server:
D /etc/opt/SFTKdtc/cp_post_on_s###.sh
is run immediately after the checkpoint -on token is received by the RMD. At this point, all updates received by the RMD are directed to the secondary journal area. You can use this script to launch a tape backup of the mirror, to mount file systems (in read-only mode) on the mirror devices, or to start a database in readonly mode for report generation or decision support applications.
/etc/opt/SFTKdtc/cp_pre_off_s###.sh is run immediately before the RMD regains exclusive control of the mirror devices after the dtccheckpoint -off command is issued. You can use this script to unmount the read-only file systems on the mirror devices, to stop database applications running for report generation or decision support applications, or to terminate applications on the secondary so that the RMDs can regain control of the mirror devices.
The figure: 17.13: Script Sample of /etc/opt/SFTKdtc/cp_post_on_s012.sh for Secondary Server and the figure: 17.14: Script Sample of /etc/opt/SFTKdtc/cp_pre_off_s012.sh for Secondary Server show an example set of secondary scripts that fit the context of the previous primary script examples. In the following example, a level 0 tape backup is performed on the checkpointed data set. Note that this procedure has a minimal impact on the Primary Server (momentary shutdown for synchronization).
242
www.ibm.com
Administrative Functions Figure 17.13: Script Sample of /etc/opt/SFTKdtc/cp_post_on_s012.sh for Secondary Server
#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s012.sh ***** echo ---------- date ---------- >> /usr/tmp/ xpoint.log echo starting cp_post_on_s012.sh >> /usr/tmp/ xpoint.log echo starting full level-0 cold back-up of database \ >> /usr/tmp/xpoint.log /usr/local/bin/db_backup.sh echo database backup completed, exiting checkpoint \ >> /usr/tmp/xpoint.log echo >> /usr/tmp/xpoint.log /opt/SFTKdtc/bin/dtccheckpoint -g 12 -off exit 0
17
Figure 17.14: Script Sample of /etc/opt/SFTKdtc/cp_pre_off_s012.sh for Secondary Server

#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s012.sh ***** echo ---------- date ---------- >> /usr/tmp/ xpoint.log echo starting cp_pre_off_s012.sh >> /usr/tmp/ xpoint.log if [ -f /tmp/db_backup_PID ]; then echo database back-up error, killing it \ >> /usr/tmp/xpoint.log /bin/kill -9 </tmp/db_backup_PID fi echo >> /usr/tmp/xpoint.log exit 0
www.ibm.com
243
Checkpoint Scripts for File Systems

The figure: 17.15: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_post_on_s001.sh and the figure: 17.16: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_pre_off_s001.sh show checkpointing scripts written to make a file system available for examination, reading, copying or for other applications. These scripts are not mandatoryyou can do these steps manually.
Primary Checkpoint Scripts

No scripts are required on the primary. The checkpoint command issues a UNIX sync command to flush uncommitted updates from the buffer cache to disk prior to inserting the checkpoint token into the BAB. If an active application, such as a database, requires additional steps for quiescence, then you can write primary scripts to automate the process.
Secondary Checkpoint Scripts

The following figures show examples of the scripts that can be defined on the secondary to automate the procedures for checkpointing file systems.
Figure 17.15: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_post_on_s001.sh
#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s001.sh ***** echo ---------- date ---------- >> /usr/tmp/xpoint.log echo starting cp_post_on_s001.sh >> /usr/tmp/xpoint.log # # -- mount the mirror devices read-only mount -o ro /dev/dsk/c1t4d0s5 /mnt_rmt echo /mnt_rmt file system mounted read-only >> \ /usr/tmp/xpoint.log echo >> /usr/tmp/xpoint.log # # -- this is a good spot to start any application that # -- needs to work with the checkpointed file system data # -- on the secondary or to notify users that this data # -- is now available exit 0
244
www.ibm.com
Administrative Functions Figure 17.16: File System Checkpoint Script - /etc/opt/SFTKdtc/cp_pre_off_s001.sh

#!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s001.sh ***** echo ---------- date ---------- >> /usr/tmp/xpoint.log echo starting cp_pre_off_s001.sh >> /usr/tmp/xpoint.log # # -- this is a good spot to stop any application using the # -- checkpointed file system and to shoo users away from it # -- (with force, if necessary). # umount /mnt_rmt echo /mnt_rmt file system un-mounted >> /usr/tmp/xpoint.log echo >> /usr/tmp/xpoint.log exit 0
17
cp_reboot_s###.sh
In addition to the four shell scripts described, you can create an etc/opt/SFTKdtc/ script to automate actions when a reboot occurs while checkpoint is on. This script can simply enter a dtccheckpoint -off command or verify that a backup completed before doing a reboot. If it detects that the operation did not complete, it may relaunch the application.
NOTE: If data on the mirror device(s) is altered by applications accessing the devices while the checkpoint is on, then the mirror is broken and manual intervention is required to synchronize the systems. The actual process may require a full refresh, a backfresh or a backup/restore operation depending on the how the data is prioritized on each system.
Tips for Checkpoint Scripts

D D D D
Scripts intended to run on the primary have a p before the Mobility Group number and scripts intended to run on the secondary have an s before the Mobility Group number. Script files must have the x executable attribute set. All scripts must end with exit 0 or the dtccheckpoint command fails and the operation is aborted. Define only the checkpoint scripts that you need. You need not define all four scripts.
Mounting Mirror Devices on HP-UX and AIX

On HP-UX and AIX, mirror devices for a particular Mobility Group are allowed to be mounted with read/write permission even though the Mobility Group is in Normal state; whereas, Softek Replicator for Solaris prevents the same mirror devices from being mounted. Should this mount happen and some data is written to the mounted file system, it will cause the data to be inconsistent between the primary and secondary devices. Any mirror device that is defined by the Mobility Group in Normal state must not be mounted as read/write.
www.ibm.com
245
Rebooting the Primary Server on HP-UX
Rebooting the Primary Server on HP-UX

"
To reboot the Primary Server, enter:

shutdown -r
Do not use the reboot command to restart or reboot the Primary Server. This applies to all supported HP-UX versions (11.00, 11.11, and 11.23).
NOTE: Stopping Applications That Use the dtc Device Applications (or processes) that use the dtc device must be stopped, either manually or automatically via the rc script, before the Primary Server is rebooted. To auto-stop a specific application or process via the rc script, set the applications stopping procedures before /etc/rc0.d/k800SFTKdtc-scan.
Managing Online Replication on AIX JFS2 File Systems

The following are the recommended procedures for preparing JFS2 volumes for Checkpoint mode, for accessing a JFS2 target disk volume, and for getting it back into Normal state after completing target drive use.
Preparing JFS2 Volumes for Checkpoint Mode

Before turning Checkpoint on JFS2 volumes, you must perform the following steps: 1. Match the minor device numbers between the dtc device and the target device, as described in Managing dtc Devices on page 228. 2. Update the cp_pre_on_pxxx script with the dtcsyncfs command; refer to dtcsyncfs on page 194. This forces the log on the primary device to be applied. 3. The Mobility Group should have all its volumes in a log, or each volume should have its own log.
246
www.ibm.com
17
Accessing a JFS2 Target Volume

Checkpointing and Accessing a JFS2 Target Volume When the Mobility Group is in Normal State 1. Turn the Checkpoint mode ON; refer to Checkpoint ON on page 238. 2. Attempt to mount the target volume as read-only. 3. If the read-only mount fails, perform a file system check (fsck) on the target device. 4. If the target device is not already mounted, mount it. 5. Unmount the target device. 6. Turn the Checkpoint mode OFF; refer to Checkpoint OFF on page 239. 7. If fsck was run, launch a Checksum Refresh.
Accessing a JFS2 Target Drive in Case of Primary Server Failure

If the Primary server has failed or is having a disk failure, run a file system check (fsck). In this case, the log will be applied when you run fsck and whatever state the log is in, the log will be applied.
User-Defined Devices
Softek Replicator allows you to create a configuration file named devlist.conf which enables you to add uncommon devices for replication. This feature supports uncommon Logical Volume managers, including some of those found in Linux environments. You can specify a directory path where uncommon devices that are eligible for replication can be found. The devlist.conf file is located in the same directory as the pxxx.cfg and sxxx.cfg files, and has the following characteristics:
D
You must be a root-privilege user to create the devlist.conf configuration file. This configuration file does not exist by default. The devlist.conf configuration file is a simple ASCII file that you can create using an editor. This file can contain one or more paths to directories that contain devices. All devices found in these directories will be combined with other normally found devices, which you will be able to select from dtcconfigtool and the Softek Data Mobility Console, for both source and target devices. You open and edit the file directly. Then, you type the new device path in the file in a separate line. Each line of the devlist.conf configuration file is treated as a directory for device search path, as displayed in the following example:
/dev/vx/rdsk /dev/xd /dev/TDMFIP /dev/disabled_directory_path(commented line) /dev/invalid_directory_path(ignored line) TDMFIP
Every leaf filename (that is, not a directory) under /dev/vx/rdsk, /dev/xd, and /dev/ is treated as an additional available device for Softek Replicator to use. During the configuration process, the properties, such as size, of the candidate device are further
www.ibm.com
247
Changing the Primary Server into a Secondary Server, and Vice Versa
validated. Lines starting with characters other than /, space, and #, will be ignored. If the directory is not found, the entry is ignored.
NOTE: On the Linux version, do not add /dev/vx/.. to devlist.conf. Softek Replicator can find VERITAS devices on Linux without the need to create or edit devlist.conf. Other platforms still require that you add information to devlist.conf to see VERITAS devices.
D
There is at most one devlist.conf file on the source system, and one on the target system. The devlist.conf file on the PMD side need not be identical to the devlist.conf file on the RMD for data replication. Each devlist.conf file in a Softek Replicator system delineates only the user-defined devices available to Softek Replicator as either primary or mirror devices. The devlist.conf configuration file is automatically saved at Softek Replicator uninstallation. Also, Softek Replicator will automatically use the saved devlist.conf file at installation if there is any saved file.
Changing the Primary Server into a Secondary Server, and Vice Versa
In order to switch the Primary Server to a Secondary Server, you must remove the BAB on one server and set it on the other. If you are using the Softek Data Mobility Console to manage the replication, you have to run the dtcagentset command on the UNIX server to load the driver and set the BAB size. The following procedures describe how to switch from Primary Server (with BAB) to Secondary Server (without BAB) and vice versa.
"
To convert the Primary Server to Secondary Server: a. Load the driver and set the BAB to 64 MB, as shown in the following example:
dtcagentset e 129.212.239.3 b 64
1. On the Primary Server (PMD) machine:
b. List the configuration information by running dtcagentset l. c. Run launchagent. d. On the Softek Data Mobility Console, verify the BAB size once you see the server, by running dtcinfo a. e. If you must change the BAB size, you can do so in the Server Properties dialog box. Run dtcinfo a to see the new BAB settings. 2. To convert the Primary Server to Secondary Server, you must remove the driver and not use the BAB, using one of the following methods:
H
Method 1: i) Issue the following commands:

Killpmds a dtcstop a killdtcmaster
248
www.ibm.com
17
ii) Remove the driver. The command will change depending on your operating system:
H H H
On AIX: rmdev -l dtc0 On HP-UX: kmadmin -U dtc On Solaris: rem_drv dtc
iii) Re-launch in.dtc by running launchdtcmaster. iv) Run the dtcagentset command with no -b option, as follows:
dtcagentset e 129.212.239.3 H
Method 2: i) Uninstall and re-install Softek Replicator. ii) Run the dtcagentset command with no -b option, as shown in the following example:
dtcagentset e 129.212.239.3
"
To convert the Secondary Server to Primary Server: a. Run the dtcagentset e 129.212.239.3 command. The b option is not present, which indicates that there is no driver. b. Run launchagent. The dtc driver is not loaded and no BAB memory is consumed. Running dtcinfo a should show that the driver is not loaded.
1. On the Secondary Server (RMD) machine:
NOTE: Changing the BAB size from the Softek Data Mobility Console will have no effect. The command is ignored by the UNIX agent because the driver is not loaded.
c. Run Killagent and launchagent to display the information. 2. To convert from Secondary Server (RMD) to Primary Server (PMD), you must load the dtc driver and set the BAB size, as follows: a. killagent b. dtcagentset e 129.212.239.3 b 64 c. launchagent 3. Run dtcinfo a from the Server or from the Softek Data Mobility Console to display the information.
www.ibm.com
249
Part III
Appendices
Appendix A: Recovering Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .253 Appendix B: HP-UX MC ServiceGuard Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .261 Appendix E: Sample Script for Changing Symbolic Links . . . . . . . . . . . . . . . . . . . . . .307 Appendix D: Checkpoint Scripts for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .291 Appendix C: Sample Scripts for Veritas Clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . .285 Appendix F: Implementing Softek Replicator With Solaris Zones. . . . . . . . . . . . . . . . .311
Appendix A
Recovering Data
Scenario 1: Secondary Server/Network Failure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .255 Scenario 2: Primary Server Failure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .256 Scenario 3: Failure to Transition Out of Tracking state. . . . . . . . . . . . . . . . . . . . . . . .257 Scenario 4: Failing Over to the Secondary Server, Restoring the Primary Server . . . .258 Scenario 5: Device Failure on the Primary or Secondary Server . . . . . . . . . . . . . . . . .259
Recovering Data
This appendix describes how to recover data in case:

D D D D
the Primary or Secondary Server fails; a device on either system fails; the network between systems is unavailable; you need to restore the Primary Server after a changeover to the Secondary Server.
Scenario 1: Secondary Server/Network Failure

If the Secondary Server or network has failed, the Connection field in dtcmonitortool turns red and indicates Accumulate mode.
If the Secondary Server or network becomes unavailable for a prolonged period of time, do one of the following:
D
If the Mode field indicates that the product is still in Normal state: a. When the Secondary Server comes back into service, use the launchpmds command from the Primary Server to start the PMD and RMD. b. Softek Replicator will continue to operate in Normal state and will begin mirroring the data again.
If the Mode field indicates that the product is in Tracking state: When the Secondary Server comes back into service, use the launchrefresh command from the Primary Server to perform a smart refresh to resynchronize the systems.
www.ibm.com
255
Scenario 2: Primary Server Failure
If the Secondary Server must be replaced, create a new Secondary Server and network connection to receive the mirrored data as follows: a. Install Softek Replicator on the new Secondary Server. b. Change the configuration of each defined Mobility Group with dtcconfigtool (on the Primary Server) so that they refer to the new Secondary Server. Follow the process outlined in Chapter 10: Getting Started With Replication as it pertains to setting up the new Secondary Server. Remember to copy the updated configuration file to the new Secondary Server. c. Follow the synchronization procedure described in Chapter 10: Getting Started With Replication to create the initial mirror from the primary to the new Secondary Server.
Scenario 2: Primary Server Failure

If a failure occurs on the Primary Server that places it out of operation for hours, days, or longer, data can be accessed from the Secondary Server.
"
To allow data to be accessed from the Secondary Server:
1. Type killpmds to stop the PMD on the Primary Server. 2. From the Secondary Server, type dtcrmdreco -g <group#> to commit any buffered updates to the mirror devices.
NOTE: The dtcrmdreco command ensures data integrity on the mirror devices and relinquishes the exclusive hold that Softek Replicator has on these devices.
3. If the mirror devices contain a file system, fsck those devices and mount <filesystems>. 4. Start application(s) on the Secondary Server. 5. See Scenario 4: Failing Over to the Secondary Server, Restoring the Primary Server on page 258 for instructions on restoring the Primary Server to primary status.
Switching to the Secondary Server with an RDBMS

If you are using Softek Replicator to mirror data from a database (as described in Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS) on page 132) follow these steps to change over to the Secondary Server. 1. On the Secondary Server, type killrmds to ensure that no Softek Replicator process has mirror devices locked. 2. Type dtcrmdreco -a to ensure that all uncommitted Softek Replicator data actually gets written to the mirror devices.
256
www.ibm.com
Recovering Data
3. Create the necessary symbolic links to now point to the mirror devices. For example:
mkdir ln -s ln -s ln -s /dev/devlinks /dev/rdsk/c1t1d0s1 /dev/devlinks/tblspA /dev/rdsk/c1t2d0s1 /dev/devlinks/tblspB /dev/rdsk/c1t3d0s1 /dev/devlinks/tblspC
Change the ownership of the mirror devices to the RDBMS.

/dev/rdsk/c1t1d0s1 is actually a symbolic link itself. Use the command ls -la /dev/rdsk to see where the device instance actually is, cd to that directory and change the device ownership appropriately. Now you can bring up the database on the Secondary Server. The database performs cleanup and rollback of any uncommitted partial transactions that it finds as part of its standard recovery process.
Scenario 3: Failure to Transition Out of Tracking state

During normal operation, if the BAB overflows for a Mobility Group on the Primary Server, the following message is displayed in the dtcmonitortool window or in the dtcerror.log:
BABOFLOW BAB exhausted during normal operation <processname>.
The Mobility Group is temporarily placed in Tracking state. Then, the PMDs drain the BAB entries, the Mobility Group transitions to a Smart Refresh, and transitions back into Normal state after the Refresh is complete. No user intervention is required. In the case of a double BAB overflow a second BAB overflow occurs while the Mobility Group is still in Tracking or Refresh state. With a double BAB overflow, the Mobility Group is placed in Tracking state, in which it remains until you type: launchrefresh -g <group#> The Mobility Group goes to a Smart Refresh and then back to Normal state. After all journal files are applied to the mirror devices, the Primary and Secondary Servers are in a coherent state. Note that if you experience double BAB overflows, you may need to increase the size of your BAB. For more information, see Modifying the BAB on page 233. Be aware that these messages may have scrolled off the dtcmonitortool screen.
www.ibm.com
257
Scenario 4: Failing Over to the Secondary Server, Restoring the Primary Server
Scenario 4: Failing Over to the Secondary Server, Restoring the Primary Server
Assume that you had a disaster on the Primary Server and want to relocate operations to a Secondary Server using the mirrored data. Follow these steps: 1. Type: dtcrmdreco -g <group#> on the Secondary Server. 2. If necessary, mount mirror devices. Then start applications on the Secondary Server. Now your Primary Server is back in order, and you want to move operations back to the Primary Server. Synchronize the Primary Server with the more recent data on the secondary data devices as follows: 1. Stop all applications and unmount the devices on the Secondary Server in preparation for restoring the primary. 2. Type: dtcrmdreco -g <group#> -d on the Secondary Server. 3. On the Primary Server, type: /opt/SFTKdtc/bin/launchbackfresh -g <group#> When the Backfresh completes, Softek Replicator shifts into Normal state, the PMDs and RMDs are connected, and you can resume mirroring data to the Secondary Server.
NOTE: Backfresh is an operation in maintenance mode only. Do not access or update local or mirror data devices until the Backfresh is complete.
Alternately, you can bring the Primary Server back into synchronization with the more recent data on the Secondary Server by switching the Primary and Secondary Servers roles and performing a refresh operation from the original Secondary Server back to the original primary one. One advantage of this approach is that applications can continue to update the data on the original Secondary Server during the refresh operation. For more information, refer to Complex Configurations on page 87.
258
www.ibm.com
Recovering Data
Scenario 5: Device Failure on the Primary or Secondary Server

You can detect a device failure on one of your systems using a tool such as Volume Manager. The device appears greyed if it has failed. You may also see a Softek Replicator message to this effect in dtcerror.log. In this case, take the following steps: 1. If you are using file systems, unmount the dtc devices in the affected Mobility Group. 2. Type killmpds -g <group#>. 3. On the Primary Server, type: dtcstop -g <group#> to stop the Mobility Groups affected by the device failure. 4. Add or replace the device on the system. 5. Type: dtcconfigtool to modify the affected Mobility Group definitions so that they refer to the new device. Remember to copy the new .cfg file to the Secondary Server. 6. Type: dtcstart -g <group#> to restart the Mobility Groups. 7. If the device failed on the Primary Server, type: launchbackfresh -g <group#> If the device failed on the Secondary Server, type: launchrefresh -f -g <group#>
www.ibm.com
259
Appendix B
HP-UX MC ServiceGuard Script

Sample ServiceGuard Script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .263
Sample ServiceGuard Script

The appendix provides an example of an HP-UX ServiceGuard script to enable you to set up Softek Replicator in an HP-UX MC ServiceGuard cluster environment.
********************************************************************** # * * # * HIGH AVAILABILITY PACKAGE CONTROL SCRIPT (template) * # * * # * Note: This file MUST be edited before it can be used. * # * * # ********************************************************************** # UNCOMMENT the variables as you set them. # Set PATH to reference the appropriate directories. PATH=/usr/bin:/usr/sbin:/etc:/bin # VOLUME GROUP ACTIVATION: # Specify the method of activation for volume groups. # Leave the default ("VGCHANGE="vgchange -a e") if you want volume # groups activated in exclusive mode. This assumes the volume groups have # been initialized with 'vgchange -c y' at the time of creation. # # Uncomment the first line (VGCHANGE="vgchange -a e -q n"), and comment # out the default, if your disks are mirrored on separate physical paths, # # Uncomment the second line (VGCHANGE="vgchange -a e -q n -s"), and comment # out the default, if your disks are mirrored on separate physical paths, # and you want the mirror resynchronization to occur in parallel with # the package startup. # # Uncomment the third line (VGCHANGE="vgchange -a y") if you wish to # use non-exclusive activation mode. Single node cluster configurations # must use non-exclusive activation. # # VGCHANGE="vgchange -a e -q n" # VGCHANGE="vgchange -a e -q n -s" # VGCHANGE="vgchange -a y" # VGCHANGE="vgchange -a e" # Default VGCHANGE="vgchange -a e" # Default VGCHANGE_EXCL="vgchange -a e" # Default # VOLUME GROUPS # Specify which volume groups are used by this package. Uncomment VG[0]="" # and fill in the name of your first volume group. You must begin with # VG[0], and increment the list in sequence. # # For example, if this package uses your volume groups vg01 and vg02, enter: # VG[0]=vg01 # VG[1]=vg02 #
263

# The volume group activation method is defined previously. The filesystems # associated with these volume groups are specified as follows: # # Archive & App FS Volume Groups VG_FS[1]="/dev/vg_app1" VG_FS[2]="/dev/vg_app2" VG_FS[3]="/dev/vg_app3" #Mobility Groups Replicator_LG[1]=1 Replicator_LG[2]=2 Replicator_LG[3]=3 # FILESYSTEMS # Specify the file systems that are used by this package. Uncomment # LV[0]=""; FS[0]=""; FS_MOUNT_OPT[0]="" and fill in the name of your first # logical volume, filesystem and mount option for the file system. You must # begin with LV[0], FS[0] and FS_MOUNT_OPT[0], and increment the list in # sequence. # # For the LVM example, if this package uses the file systems pkg1a and # pkg1b, which are mounted on the logical volumes lvol1 and lvol2 with # read and write options, enter: # LV[0]=/dev/vg01/lvol1; FS[0]=/pkg1a; FS_MOUNT_OPT[0]="-o rw" # LV[1]=/dev/vg01/lvol2; FS[1]=/pkg1b; FS_MOUNT_OPT[1]="-o rw" # # For the CVM or VxVM example, if this package uses the file systems # pkg1a and pkg1b, which are mounted on the volumes lvol1 and lvol2 # with read and write options, enter: # LV[0]="/dev/vx/dsk/dg01/vol01"; FS[0]="/pkg1a"; FS_MOUNT_OPT[0]="-o rw" # LV[1]="/dev/vx/dsk/dg01/vol02"; FS[1]="/pkg1b"; FS_MOUNT_OPT[1]="-o rw" # # The file systems are defined as triplets of entries specifying the logical # volume, the mount point, and the mount options for the file system. Each # file system will be fsck'd prior to being mounted. The file systems will be # mounted in the order specified during package startup and will be unmounted # in reverse order during package shutdown. Ensure that volume groups # referenced by the following logical volume definitions are included in # volume group definitions described previously. # LV[0]=/dev/dtc/lg1/dsk/dtc0 FS[0]=/opt/app FS_MOUNT_OPT[0]="-o rw" LV[1]=/dev/dtc/lg1/dsk/dtc1 FS[1]=/opt/app /xp01 FS_MOUNT_OPT[1]="-o rw" LV[2]=/dev/dtc/lg1/dsk/dtc2 FS[2]=/opt/app/xp01/app FS_MOUNT_OPT[2]="-o rw" LV[3]=/dev/dtc/lg2/dsk/dtc0 FS[3]=/opt/app/xp01/oracle FS_MOUNT_OPT[3]="-o rw" LV[4]=/dev/dtc/lg2/dsk/dtc1 FS[4]=/opt/app/xp01/oracle/data FS_MOUNT_OPT[4]="-o rw" LV[5]=/dev/dtc/lg2/dsk/dtc0 FS[5]=/opt/app/xp01/data/cl FS_MOUNT_OPT[5]="-o rw"
264
www.ibm.com

LV[6]=/dev/dtc/lg3/dsk/dtc1 FS[6]=/opt/app/xp01/bdata/server FS_MOUNT_OPT[6]="-o rw" LV[7]=/dev/dtc/lg3/dsk/dtc2 FS[7]=/opt/app/xp01/data/all_archives FS_MOUNT_OPT[7]="-o rw" # Bring up, if recovery is required the default behavior is for # the package control script to wait until recovery has been # completed. # To allow mirror resynchronization to occur in parallel with # the package startup, uncomment the line # VXVOL="vxvol -g \$DiskGroup -o bg startall" and comment out the default. # # VXVOL="vxvol -g \$DiskGroup -o bg startall" # VXVOL="vxvol -g \$DiskGroup startall" # Default # FILESYSTEM UNMOUNT COUNT # Specify the number of unmount attempts for each file system during package # shutdown. The default is set to 1. FS_UMOUNT_COUNT=4 # FILESYSTEM MOUNT RETRY COUNT. # Specify the number of mount retrys for each file system. # The default is 0. During startup, if a mount point is busy # and FS_MOUNT_RETRY_COUNT is 0, package startup will fail and # the script will exit with 1. If a mount point is busy and # FS_MOUNT_RETRY_COUNT is greater than 0, the script will attempt # to kill the user responsible for the busy mount point # and then mount the file system. It will attempt to kill user and # retry mount, for the number of times specified in FS_MOUNT_RETRY_COUNT. # If the mount still fails after this number of attempts, the script # will exit with 1. # NOTE: If the FS_MOUNT_RETRY_COUNT > 0, the script will execute # "fuser -ku" to free up busy mount point. FS_MOUNT_RETRY_COUNT=2 # # # # # # # # # # # # # # # #
www.ibm.com
IP ADDRESSES Specify the IP and Subnet address pairs that are used by this package. Uncomment IP[0]="" and SUBNET[0]="" and fill in the name of your first IP and subnet address. You must begin with IP[0] and SUBNET[0] and increment the list in sequence. For example, if this package uses an IP of 192.10.25.12 and a subnet of 192.10.25.0, enter: IP[0]=192.10.25.12 SUBNET[0]=192.10.25.0 # (netmask=255.255.255.0) Hint: Run "netstat -i" to see the available subnets in the Network field. IP/Subnet address pairs for each IP address you want to add to a subnet interface card. Must be set in pairs, even for IP addresses on the same subnet.
265

# IP[0]=15.145.34.184 SUBNET[0]=15.145.32.0 # SERVICE NAMES AND COMMANDS. # Specify the service name, command, and restart parameters that are # used by this package. Uncomment SERVICE_NAME[0]="", SERVICE_CMD[0]="", # SERVICE_RESTART[0]="", and fill in the name of the first service, command, # and restart parameters. You must begin with SERVICE_NAME[0], SERVICE_CMD[0], # and SERVICE_RESTART[0] and increment the list in sequence. # # For example: # SERVICE_NAME[0]=demo # SERVICE_CMD[0]="/usr/bin/X11/xclock -display 192.10.25.54:0" # SERVICE_RESTART[0]="" # Will not restart the service. # # SERVICE_NAME[1]=pkg1b # SERVICE_CMD[1]="/usr/bin/X11/xload -display 192.10.25.54:0" # SERVICE_RESTART[1]="-r 2" # Will restart the service twice. # # SERVICE_NAME[2]=pkg1c # SERVICE_CMD[2]="/usr/sbin/ping" # SERVICE_RESTART[2]="-R" # Will restart service an infinite number of times # # Note: No environment variables will be passed to the command, this # includes the PATH variable. Absolute path names are required for the # service command definition. Default shell is /usr/bin/sh. # SERVICE_NAME[0]= SERVICE_CMD[0]= SERVICE_RESTART[0]= # DEFERRED_RESOURCE NAME # Specify the full path name of the 'DEFERRED' resources configured for # this package. Uncomment DEFERRED_RESOURCE_NAME[0]="" and fill in the # full path name of the resource. # #DEFERRED_RESOURCE_NAME[0]="" # DTC manager information for each DTC. # Example: DTC[0]=dtc_20 #DTC_NAME[0]= # # # # START OF CUSTOMER-DEFINED FUNCTIONS This function is a placeholder for customer-defined functions. You should define all actions you want to happen here, before the service is started. You can create as many functions as you need.
function customer_defined_run_cmds
266
www.ibm.com

{ } # This function is a placeholder for customer-defined functions. # You should define all actions you want to happen here, before the service is # halted. function customer_defined_halt_cmds { } # END OF CUSTOMER-DEFINED FUNCTIONS # START OF RUN FUNCTIONS ############################################################### # This function checks for the existence of MetroCluster or # ContinentalClusters packages that use physical data # Replication via Continuous Access XP on HP SureStore XP # series disk arrays or SRDF on EMC Symmetrix disk arrays. # # If the /usr/sbin/DRCheckDiskStatus file exists in the system, # then the cluster has at least one package that will be # configured for remote data mirroring in a metropolitan or # continental cluster. # # The function is called before attempting to activate the # volume group. If no /usr/sbin/DRCheckDiskStatus file exists, # the function does nothing. # ############################################################### # ############################################################## # This function tests whether the package is using HA NFS or # not. If the HA NFS script file, hanfs.sh, exists in the # package directory, then the package will be configured for # use with HA NFS, and the script hanfs.sh will be executed. # # This function has one parameter passed to it, which will then # be passed to the hanfs.sh script: # # - start - to indicate the package is starting up # - stop - to indicate the package is shutting down # ############################################################### function activate_volume_group { for I in ${VG[@]} do if [[ "${VGCHANGE}" = "vgchange -a y" ]] then print "$(date '+%b %e %X') - Node \"$(hostname)\": Activating volume group $I with non-exclusive option."
267

else print "$(date '+%b %e %X') - \"$(hostname)\": Activating volume group $I with shared option." fi $VGCHANGE $I test_return 1 # If the -s option has been specified, then we perform # the resynchronization as a background task # if [[ ${VGCHANGE#*-s} != ${VGCHANGE} ]] then { if /sbin/vgsync $I then print "$(date '+%b %e %X') - Node \"$(hostname)\": Resynchronized volume group $I" else print "$(date '+%b %e %X') - Node \"$(hostname)\": Resynchronization of volume group $I encountered an error" fi } & fi done $VGCHANGE_EXCL ${VG_FS[0]} $VGCHANGE_EXCL ${VG_FS[1]} $VGCHANGE_EXCL ${VG_FS[2]} $VGCHANGE_EXCL ${VG_FS[3]} $VGCHANGE_EXCL ${VG_FS[4]} } #This function is used to kill the user to free up a mountpoint #that could be busy, and then do the mount operation. #freeup_busy_mountpoint_and_mount_fs(x, y, z) # x = Logical volume group to be mounted. # y = File System where the logical volume is to be mounted. # z = Mount Options to be used for mount operation # function freeup_busy_mountpoint_and_mount_fs { typeset vol_to_mount typeset mount_pt typeset fs_mount_opt vol_to_mount=$1 mount_pt=$2 shift 2 fs_mount_opt=$* print "\tWARNING: Running fuser on ${mount_pt} to remove anyone using the busy mount point directly." UM_COUNT=0 RET=1
268
www.ibm.com

# The control script exits, if the mount failed after # retrying FS_MOUNT_RETRY_COUNT times. while (( $UM_COUNT < $FS_MOUNT_RETRY_COUNT && $RET != 0 )) do (( UM_COUNT = $UM_COUNT + 1 )) fuser -ku ${mount_pt} if (($UM_COUNT == $FS_MOUNT_RETRY_COUNT)) then mount ${fs_mount_opt} ${vol_to_mount} ${mount_pt} test_return 17 else mount ${fs_mount_opt} ${vol_to_mount} ${mount_pt} (( RET = $? )) sleep 1 fi done } # For each {file system/logical volume} pair, fsck the file system # and mount it. # If the mount point is busy and if FS_MOUNT_RETRY_COUNT = 0, # mounting of the file system will fail and the control script # will exit with an error. # function check_and_mount { integer NOT_READY=1 while (( $NOT_READY != 0 )) do NOT_READY=0 integer R=0 for I in ${LV[@]} do if [[ $(mount -p | awk '$1 == "'$I'"') = "" ]] then case $I in */dev/vx/dsk*) check_vxvm_vol_available $I if(( $? != 0 )) then NOT_READY=1 else RLV[$R]=$(print $I | sed -e 's/dsk/rdsk/') fi ;; *) #Softek Replicator UNIX Modification #RLV[$R]="${I%/*}/r${I##*/}" RLV[$R]=ècho $I | sed 's/dsk/rdsk/g'` ;;
269

esac if [ -x /usr/sbin/fstyp ] then fstype[$R]=$(fstyp $I) fi (( R = $R + 1 )) fi done if(( $NOT_READY != 0 )) then sleep 5 fi done # Verify that there is at least one file system to check and what type. if [[ "$exit_value" != 1 && ${RLV[@]} != "" ]] then print -n "$(date '+%b %e %X') - Node \"$(hostname)\": " print "Checking filesystems:" print ${LV[@]} | tr ' ' '\012' | sed -e 's/^/ /' # If there is more than one filesystem type being checked # then each file system is checked individually. # R=$(print ${fstype[*]} | tr ' ' '\012' | sort -u | wc -l) if (( R > 1 )) then R=0 while (( R < ${#RLV[*]} )) do case ${fstype[$R]} in hfs) fsck -F hfs -P ${RLV[$R]} test_return 2 ;; vxfs) fsck -F vxfs -y ${RLV[$R]} test_return 2 ;; unk*) fsck ${RLV[$R]} test_return 2 ;; *) if [[ ${fstype[$R]} = "" ]] then fsck ${RLV[$R]} else fsck -F ${fstype[$R]} ${RLV[$R]} fi test_return 2 ;; esac (( R = R + 1 )) done # If there is only one file system type being checked, then
270
www.ibm.com

# multiple invocations of fsck can be avoided. All file systems # are specified on the command line to one fsck invocation. # else case ${fstype} in hfs) fsck -F hfs -P ${RLV[@]} test_return 2 ;; vxfs) fsck -F vxfs -y ${RLV[@]} test_return 2 ;; unk*) fsck ${RLV[@]} test_return 2 ;; *) if [[ ${fstype} = "" ]] then fsck ${RLV[@]} else fsck -F ${fstype} ${RLV[@]} fi test_return 2 ;; esac fi fi # Check exit value (set if any proceeding fsck calls failed) if (( $exit_value == 1 )) then deactivate_volume_group print "\n\t########### Node \"$(hostname)\": Package start failed at $(date) ###########" exit 1 fi integer F=0 for I in ${LV[@]} do if [[ $(mount | grep -e $I" ") = "" ]] then print "$(date '+%b %e %X') - Node \"$(hostname)\": Mounting $I at ${FS[$F]}" #if there is permission to kill the user, we can #run fuser to kill the user, on the mount point. #This would freeup the mount point, if it is busy if (( $FS_MOUNT_RETRY_COUNT > 0 )) then mount ${FS_MOUNT_OPT[$F]} $I ${FS[$F]} if (( $? != 0 )) then
271

freeup_busy_mountpoint_and_mount_fs $I ${FS[$F]} ${FS_MOUNT_OPT[$F]} fi else mount ${FS_MOUNT_OPT[$F]} $I ${FS[$F]} test_return 3 fi else print "$(date '+%b %e %X') - Node \"$(hostname)\": WARNING: system \"${FS[$F]}\" was already mounted." fi (( F = $F + 1 )) done } function retry_print { if [[ $(echo $1 | grep "Retrying") != "" ]] then print "$1" >> $0.log fi }
File
# For each {IP address/subnet} pair, add the IP address to the subnet # using cmmodnet(1m). function add_ip_address { integer S=0 integer error=0 for I in ${IP[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Adding IP address $I to subnet ${SUBNET[$S]}" XX=$( cmmodnet -a -i $I ${SUBNET[$S]} 2>&1 ) if (( $? != 0 )) then if [[ $(echo $XX | grep "Retried adding IP") != "" ]] then print "$XX" >> $0.log (( error = 1 )) elif [[ $(echo $XX | grep "heartbeat IP") != "" ]] then # IP has been configured as a heartbeat IP address. print "$XX" >> $0.log (( error = 1 )) else YY=$( netstat -in | awk '$4 == "'${I}'"') if [[ -z $YY ]] then print "$XX" >> $0.log
272
www.ibm.com

print "\tERROR: ${SUBNET[$S]}" (( error = 1 )) else retry_print "$XX" print "\tWARNING: IP $I is already configured on the subnet ${SUBNET[$S]}" fi fi else retry_print "$XX" fi (( S = $S + 1 )) done if (( error != 0 )) then # `let 0` is used to set the value of $? to 1. The function test_return # requires $? to be set to 1 if it has to print error message. let 0 test_return 4 fi } # Own and reset the DTC connections function get_ownership_dtc { for I in ${DTC_NAME[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Assigning Ownership of the DTC $I" dtcmodifyconfs -o $I test_return 5 for J in ${IP[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Resetting the DTC connections to IP address $J" dtcdiag -Q $J -q -f $I test_return 6 done done } # For each {service name/service command string} pair, start the # service command string at the service name using cmrunserv(1m). function start_services { integer C=0 for I in ${SERVICE_NAME[@]}
Failed to add IP $I to subnet
273

do print "$(date '+%b %e %X') - Node \"$(hostname)\": Starting service $I using" print " \"${SERVICE_CMD[$C]}\"" # # Check if cmrunserv should be called the old # way without a restart count. # if [[ "${SERVICE_RESTART[$C]}" = "" ]] then cmrunserv $I ">> $0.log 2>&1 ${SERVICE_CMD[$C]}" else cmrunserv ${SERVICE_RESTART[$C]} $I ">> $0.log 2>&1 ${SERVICE_CMD[$C]}" fi test_return 8 (( C = $C + 1 )) done } # For each {deferred resource name}, start resource monitoring for this # resource using cmstartres(1m). function start_resources { for I in ${DEFERRED_RESOURCE_NAME[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Starting resource monitoring for $I" cmstartres -u -p $PACKAGE $I >> $0.log 2>&1 test_return 15 done } # END OF RUN FUNCTIONS. # START OF HALT FUNCTIONS # For each {deferred resource name}, stop resource monitoring for this # resource using cmstopres(1m). function stop_resources { for I in ${DEFERRED_RESOURCE_NAME[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Stopping resource monitoring for $I" cmstopres -p $PACKAGE $I >> $0.log 2>&1 test_return 16 done } # Halt each service using cmhaltserv(1m). function halt_services {
274
www.ibm.com

for I in ${SERVICE_NAME[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Halting service $I" cmhaltserv $I test_return 9 done } # Disown the DTC. function disown_dtc { for I in ${DTC_NAME[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Disowning the DTC $I" dtcmodifyconfs -d $I test_return 11 done } # For each IP address/subnet pair, remove the IP address from the subnet # using cmmodnet(1m). function remove_ip_address { integer S=0 integer error=0 for I in ${IP[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Remove IP address $I from subnet ${SUBNET[$S]}" XX=$( cmmodnet -r -i $I ${SUBNET[$S]} 2>&1 ) if (( $? != 0 )) then echo $XX | grep "is not configured on the subnet" if (( $? != 0 )) then print "$XX" >> $0.log (( error = 1 )) fi else retry_print "$XX" fi (( S = $S + 1 )) done if (( $error != 0 )) then # `let 0` is used to set the value of $? to 1. The function test_return # requires $? to be set to 1 if it has to print error message. let 0 test_return 12 fi } # Unmount each logical volume.
275

function umount_fs { integer UM_CNT=${FS_UMOUNT_COUNT:-1} integer ret set -A LogicalVolumes ${LV[@]} if [[ $UM_CNT < 1 ]] then UM_CNT=1 fi integer L=${#LogicalVolumes[*]} while (( L > 0 )) do (( L = L - 1 )) I=${LogicalVolumes[$L]} mount | grep -e $I" " > /dev/null 2>&1 if (( $? == 0 )) then print "$(date '+%b %e %X') - Node \"$(hostname)\": Unmounting filesystem on $I" umount $I; ret=$? if (( ret != 0 )) then print "\tWARNING: Running fuser to remove anyone using the file system directly." fi UM_COUNT=$UM_CNT while (( ret != 0 && UM_COUNT > 0 )) do fuser -ku $I umount $I; ret=$? if (( ret != 0 )) then if (( $UM_COUNT == 1 )) then let 0 test_return 13 fi (( UM_COUNT = $UM_COUNT - 1 )) sleep 1 if (( $UM_COUNT > 0 )) then print "\t$(date '+%b %e %X') - Unmount failed, trying again." fi fi done fi done }
276
www.ibm.com

function deactivate_volume_group { #Softek Replicator Modification stop_Replicator for I in ${VG[@]} do print "$(date '+%b %e %X') - Node \"$(hostname)\": Deactivating volume group $I" vgchange -a n $I test_return 14 done vgchange -a n ${VG_FS[0]} vgchange -c y ${VG_FS[0]} vgchange -a n ${VG_FS[1]} vgchange -c y ${VG_FS[1]} vgchange -a n ${VG_FS[2]} vgchange -c y ${VG_FS[2]} vgchange -a n ${VG_FS[3]} vgchange -c y ${VG_FS[3]} vgchange -a n ${VG_FS[4]} vgchange -c y ${VG_FS[4]} } #Replicator Modification function start_Replicator { ## remove /dev/dtc/ctl and /dev/dtc/lg* and recreate CTL device if [ -c /dev/dtc/ctl ] then echo "Removing SFTK CTL file ...." rm -f /dev/dtc/ctl cmaj=`/usr/sbin/lsdev -h -d dtc | /usr/bin/awk '{print $1}'` /usr/sbin/mknod /dev/dtc/ctl c $cmaj 0xffffff /usr/bin/chmod 644 /dev/dtc/ctl /usr/bin/chown bin /dev/dtc/ctl /usr/bin/chgrp bin /dev/dtc/ctl fi ##Remove any group device files if they exist for LG in /dev/dtc/lg* do echo "Removing SFTK $LG files ...." rm -fr $LG done # Startup Replicator processes if ps -ef | grep "in.dtc" | grep -v "grep" then echo "Master Softek Replicator process already running" else echo "Starting Master Softek Replicator process" /usr/bin/nohup /opt/SFTKdtc/bin/in.dtc >/tmp/in.dtc.log 2>&1
277

sleep 5 fi for I in ${Replicator_LG[@]} do echo "/opt/SFTKdtc/bin/dtcstart -g $I" /opt/SFTKdtc/bin/dtcstart -g $I 2>&1 sleep 2 echo "/opt/SFTKdtc/bin/launchpmds -g $I" /opt/SFTKdtc/bin/launchpmds -g $I 2>&1 sleep 2 echo "Softek Replicator Process Output" REPGR=`printf "%.3d" ${I}` ps -ef | grep "CTL_${REPGR}" | grep -v grep echo done } #Softek Replicator Modification function stop_Replicator { # Shutdown Softek Replicator processes echo "Wait for Softek Replicator processes to flush data" sleep 10 for I in ${Replicator_LG[@]} do BAB_COUNTER=1 BAB_VAL=`/opt/SFTKdtc/bin/dtcinfo -g $I| grep 'Entries in the BAB' |\ awk 'BEGIN{x=0;}{x=x+ $5;}END{print x;}'` if [ $BAB_VAL -eq 0 ] then echo "The BAB is Clear. Killing PMD for GROUP $I.." echo "/opt/SFTKdtc/bin/killpmds -g $I" /opt/SFTKdtc/bin/killpmds -g $I 2>&1 let BAB_COUNTER=30 sleep 1 else while [[ BAB_COUNTER -le 30 ]] do if [ $BAB_VAL -eq 0 ] then echo "The BAB is Clear. Killing PMD for GROUP $I.." echo "/opt/SFTKdtc/bin/killpmds -g $I" /opt/SFTKdtc/bin/killpmds -g $I 2>&1 let BAB_COUNTER=30 sleep 1 else echo "There are $BAB_VAL entries in the BAB for group $I ..wait.." BAB_VAL=`/opt/SFTKdtc/bin/dtcinfo -g $I| grep 'Entries in the BAB' |\ awk 'BEGIN{x=0;}{x=x+ $5;}END{print x;}'`
278
www.ibm.com

sleep 1 if [ $BAB_COUNTER -eq 30 ] then print "Group $I still had $BAB_VAL BAB entries after $BAB_COUNTER" print "seconds. Package halted without further deactivation of
volume groups" exit 1 fi let BAB_COUNTER=BAB_COUNTER+1 fi done fi echo "Placing all Mobility Groups in PASSTRHU mode ..." echo "/opt/SFTKdtc/bin/dtcoverride -g $I state passthru" /opt/SFTKdtc/bin/dtcoverride -g $I state passthru sleep 2 echo "/opt/SFTKdtc/bin/dtcstop -g $I" /opt/SFTKdtc/bin/dtcstop -g $I 2>&1 sleep 2 REPGR=`printf "%.3d" ${I}` ps -ef | grep "CTL_${REPGR}" | grep -v grep echo done } # # # # #
END OF HALT FUNCTIONS. FUNCTIONS COMMON TO BOTH RUN AND HALT. Test return value of functions and exit with NO RESTART if bad. Return value of 0 - 50 are reserved for use by Hewlett-Packard. System administrators can use numbers greater than 50 for return values. function test_return { if (( $? != 0 )) then case $1 in 1) print "\tERROR: Function activate_volume_group" print "\tERROR: Failed to activate $I" deactivate_volume_group exit 1 ;; 2) print "\tERROR: Function check_and_mount" print "\tERROR: Failed to fsck one of the logical volumes." exit_value=1 ;; 3) print "\tERROR: Function check_and_mount" print "\tERROR: Failed to mount $I to ${FS[$F]}"
www.ibm.com
279

umount_fs deactivate_volume_group exit 1 ;; 4) print "\tERROR: Function add_ip_address" print "\tERROR: Failed to add IP address to subnet" remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 5) print "\tERROR: Function get_ownership_dtc" print "\tERROR: Failed to own $I" disown_dtc remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 6) print "\tERROR: Function get_ownership_dtc" print "\tERROR: Failed to switch $I" disown_dtc remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 8) print "\tERROR: Function start_services" print "\tERROR: Failed to start service ${SERVICE_NAME[$C]}" halt_services customer_defined_halt_cmds disown_dtc remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 9) print "\tFunction halt_services" print "\tWARNING: Failed to halt service $I" ;; 11) print "\tERROR: Function disown_dtc" print "\tERROR: Failed to disown $I from ${SUBNET[$S]}" exit_value=1
280
www.ibm.com

;; 12) print "\tERROR: Function remove_ip_address" print "\tERROR: Failed to remove $I" exit_value=1 ;; 13) print "\tERROR: Function umount_fs" print "\tERROR: Failed to unmount $I" exit_value=1 ;; 14) print "\tERROR: Function deactivate_volume_group" print "\tERROR: Failed to deactivate $I" exit_value=1 ;; 15) print "\tERROR: Function start_resources" print "\tERROR: Failed to start resource $I" stop_resources halt_services customer_defined_halt_cmds disown_dtc remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 16) print "\tERROR: Function stop_resources" print "\tERROR: Failed to stop resource $I" exit_value=1 ;; 17) print "\tERROR: Function freeup_busy_mountpoint_and_mount_fs" print "\tERROR: Failed to mount $I to ${FS[$F]}" umount_fs deactivate_volume_group exit 1 ;; 21) print "\tERROR: Function activate_disk_group" print "\tERROR: Failed to activate $I" deactivate_volume_group exit 1 ;; 22) print "\tERROR: Function check_dg failed" deactivate_volume_group
281

exit 1 ;; 23) print "\tERROR: Function activate_disk_group" print "\tERROR: Failed to import $I" deactivate_volume_group exit 1 ;; 24) print "\tERROR: Function activate_disk_group" print "\tERROR: Failed to vxvol -g $I startall" deactivate_volume_group exit 1 ;; 25) print "\tERROR: Function deactivate_disk_group" print "\tERROR: Failed to deactivate $I" exit_value=1 ;; 26) print "\tERROR: Function deactivate_disk_group" print "\tERROR: Failed to deport $I" exit_value=1 ;; 50) print "\tERROR: Function verify_ha_nfs" print "\tERROR: Failed to start/stop NFS" umount_fs deactivate_volume_group exit 1 ;; 51) print "\tERROR: Function customer_defined_run_cmds" print "\tERROR: Failed to RUN customer commands" halt_services customer_defined_halt_cmds disown_dtc remove_ip_address umount_fs deactivate_volume_group exit 1 ;; 52) print "\tERROR: Function customer_defined_halt_cmds" print "\tERROR: Failed to HALT customer commands" exit_value=1 ;; *) print "\tERROR: Failed, unknown error." ;;
282
www.ibm.com

esac fi } # END OF FUNCTIONS COMMON TO BOTH RUN AND HALT #-------------------MAINLINE Control Script Code Starts Here----------------# # FUNCTION STARTUP SECTION. typeset MIN_VERSION="A.11.13" # Minimum version this control script works on integer exit_value=0 typeset CUR_VERSION # # Check that this control script is being run on a A.10.03 or later release # of MC/ServiceGuard or ServiceGuard OPS Edition. The control scripts are forward # compatible but are not backward compatible because newer control # scripts use commands and options not available on older releases. CUR_VERSION="$(/usr/bin/what /usr/lbin/cmcld | /usr/bin/grep "Date" | \ /usr/bin/egrep '[AB]\...\...|NTT\...\...' | \ cut -f2 -d" ")" if [[ "${CUR_VERSION}" = "" ]] || \ [[ "${CUR_VERSION#*.}" < "${MIN_VERSION#*.}" ]] then print "ERROR: Mismatched control script version ($MIN_VERSION). You cannot run" print "\ta version ${MIN_VERSION} control_script on a node running pre" print "\t${MIN_VERSION} MC/ServiceGuard or ServiceGuard OPS Edition software" exit 1 fi # Test to see if we are being called to run the package, or halt the package. if [[ $1 = "start" ]] then print "\n\t########### Node \"$(hostname)\": Starting package at $(date) ###########" activate_volume_group start_Replicator check_and_mount add_ip_address get_ownership_dtc customer_defined_run_cmds start_services start_resources # Check exit value if (( $exit_value == 1 ))
283

then print "\n\t########### Node \"$(hostname)\": Package start failed at $(date) ###########" exit 1 else print "\n\t########### Node \"$(hostname)\": Package start completed at $(date) ###########" exit 0 fi elif [[ $1 = "stop" ]] then print "\n\t########### Node \"$(hostname)\": Halting package at $(date) ###########" stop_resources halt_services customer_defined_halt_cmds disown_dtc remove_ip_address umount_fs deactivate_volume_group # Check exit value if (( $exit_value == 1 )) then print "\n\t########### Node \"$(hostname)\": Package halt failed at $(date) ###########" exit 1 else print "\n\t########### Node \"$(hostname)\": Package halt completed at $(date) ###########" exit 0 fi fi
284
www.ibm.com
Appendix C
Sample Scripts for Veritas Clusters

Veritas Script - Sample 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .287 Veritas Script - Sample 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .288
Veritas Script - Sample 1

The section provides an example of Veritas script to enable you to start Softek Replicator with lg0 and maintain resource dependencies in a Veritas cluster environment. For details, please refer to Scenario 4 - Failover with Veritas Cluster Server on page 157.
group Replicator_LG0 ( SystemList = { ssgsun6 = 1, ssgsun7 = 2 } AutoStartList = { ssgsun6 } ) Application Replicator_LG0 ( StartProgram = \ "/opt/SFTKdtc/bin/dtcstart -g 0;/opt/SFTKdtc/bin/launchpmds -g 0" StopProgram = \ "/opt/SFTKdtc/bin/killpmds -g 0;/opt/SFTKdtc/bin/dtcstop -g 0" MonitorProcesses = { PMD_000 } ) Disk c1t2d0s0 ( Partition = c1t2d0s0 ) IP Replicator_LG0_ip ( Device @ssgsun6 = hme0 Device @ssgsun7 = hme1 Address = "129.212.188.10" NetMask = "255.255.255.0" ) NIC Replicator_LG0_nic ( Device @ssgsun6 = hme0 Device @ssgsun7 = hme1 NetworkHosts = { "129.212.188.9" } ) Replicator_LG0 requires c1t2d0s0 Replicator_LG0 requires Replicator_LG0_ip Replicator_LG0_ip requires Replicator_LG0_nic // resource dependency tree // // group Replicator_LG0 // { // Application Replicator_LG0 // { // IP Replicator_LG0_ip // { // NIC Replicator_LG0_nic // } // Disk c1t2d0s0 // } // }
www.ibm.com
287

The section provides an example of Veritas script that starts Mobility Group lg1, with Samba serving lg1 / dtc0 mounted over /replvol2 as \\129.212.188.11\test, and maintains resource dependencies. For details, refer to Scenario 5 - Failover with VCS and Network Applications on page 157.
group Replicator_LG1 ( SystemList = { ssgsun6 = 1, ssgsun7 = 2 } AutoStartList = { ssgsun6 } ) Application Replicator_LG1 ( StartProgram = "/opt/SFTKdtc/bin/dtcstart -g 1;/opt/SFTKdtc/bin/launchpmds -g 1" StopProgram = "/opt/SFTKdtc/bin/killpmds -g 1;/opt/SFTKdtc/bin/dtcstop -g 1" MonitorProcesses = { PMD_001 } ) Application Replicator_SMB ( StartProgram = "/etc/init.d/samba.server start" StopProgram = "/etc/init.d/samba.server stop" PidFiles = { "/usr/local/samba/var/locks/smbd.pid" } MonitorProcesses = { "/usr/local/samba/bin/smbd -D -s/usr/local/samba/lib/smb.conf","/usr/local/samba/bin/nmbd -D -s/usr/ local/samba/lib/smb.conf" } ) Disk c1t2d0s1 ( Partition = c1t2d0s1 ) IP Replicator_LG1_ip ( Device @ssgsun6 = hme0 Device @ssgsun7 = hme1 Address = "129.212.188.11" NetMask = "255.255.255.0" ) Mount replvol2 ( MountPoint = "/replvol2" BlockDevice = "/dev/dtc/lg1/dsk/dtc0" FSType = ufs FsckOpt = "-y" ) NIC Replicator_LG1_nic ( Device @ssgsun6 = hme0 Device @ssgsun7 = hme1 NetworkHosts = { "129.212.188.9" } ) Replicator_LG1 requires c1t2d0s1 Replicator_LG1 requires Replicator_LG1_ip Replicator_LG1_ip requires Replicator_LG1_nic
288
www.ibm.com

Replicator_SMB requires replvol2 replvol2 requires Replicator_LG1
// resource dependency tree // //group Replicator_LG1 //{ //Application Replicator_SMB // { // Mount replvol2 // { // Application Replicator_LG1 // { // IP Replicator_LG1_ip // { // NIC Replicator_LG1_nic // } // Disk c1t2d0s1 // } // } // } //}
www.ibm.com
289
Appendix D
Checkpoint Scripts for AIX

Using Softek Replicator in a Disaster Recovery AIX Environment . . . . . . . . . . . . . . . .293 Checkpoint Process JFS with Target Volume Enabled for Read-only Access . . . . . . .299 Checkpoint Process JFS with Target Volume Enabled for Read/Write Access . . . . . . .300 Checkpoint Process JFS2 with Target Volume Enabled for Read-only Access . . . . . .301 Checkpoint Process JFS2 with Target Volume Enabled for Read/Write Access . . . . . .302 Sample Script to Generate Checkpoint Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .303
This appendix provides sample scripts for initiating a Checkpoint process and a Checksum Refresh when replicating data in an AIX JFS or JFS2 disaster recovery environment.
Using Softek Replicator in a Disaster Recovery AIX Environment

The architecture that IBM employs for their Journal File Systems (JFS and JFS2) uses an exposed log file versus hiding the log file, such as Veritas. This log is referenced by the minor device number that is embedded in the file system during mount processing. The log file has minor device numbers of the file system embedded in the generated log records so the proper records can be applied to the appropriate file system. To have remote copies of these file systems that will properly be fully available on the target systems the source minor device numbers must match the target minor device numbers. It is important to have minor device numbers match to ensure that the JFS log is available, and properly replayed to ensure that any intermediate data commits are properly committed on the target JFS file system. This fact is critical when setting up highly available disaster recovery configurations.
CAUTION: All the considerations mentioned in this section do not apply to data replication as long as the file systems are cleanly unmounted on the source system and all updates are emptied from the BAB to the target system. The log file records are applied when the file system is umounted because IBM's Journal File System architecture allows log files to be replaced when the file system is mounted.
Setting Softek Replicator JFS Disaster Recovery Configurations

When setting up AIX JFS or JFS2 configurations you must enable the Match Minor Numbers option when using the dtc Devices tab of dtcconfigtool, or check the Send Device Numbers to Configuration File box from Select Devices dialog box of the Softek Data Mobility Console. When these options are set, additional parameters are included automatically into the Primary and Secondary configuration files. The following is a sample configuration file that has the new AIX JFS-specific parameters added to it. The new parameter is MIRROR-DEVNO; it is automatically configured based on the target logical volume minor device number. When the source Softek Replicator virtual device (dtc device) is created, its minor device number is created with the same minor number as what is in the MIRROR-DEVNO parameter. The first parameter is the minor device number and the second parameter is the major number. The major number is not used in this release, but is present for a future release.
PROFILE: REMARK: PRIMARY:SYSTEM-A DTC-DEVICE:/dev/dtc/lg0/rdsk/dtc0 DATA-DISK:/dev/rs1024 SECONDARY: SYSTEM-B MIRROR-DISK:/dev/rt1024 MIRROR-DEVNO:22 10
293
To validate that the minor numbers do, in fact, match, you can do a listing of the dtc device that is associated with the mirror disk and also do a listing of the mirror disk as shown in the following example. The minor number in this case is 22, which matches on both the source (dtc device) and the target mirror device.
# ls -l /dev/rt1024 crw-rw---1 root system system 10, 22 Dec 28 09:24 /dev/rt1024 45, 22 Dec 28 08:33 /dev/dtc/lg0/rdsk/dtc0 # ls -l /dev/dtc/lg0/rdsk/dtc0 crwxr-xr-x 1 root
Matching Device Minor Numbers

JFS and JFS2 logs contain references to device minor numbers for each file system volume associated with the log device. These device numbers allow meta-data transactions to be replayed back to the proper device in the event of a system crash. In a disaster recovery scenario, replicated log volumes on the Secondary Server will be used to replay transactions for application to secondary data devices. This process is necessary to return these volumes to a consistent state prior to mounting the file systems. Because these minor numbers are critical to the recovery process after a fail over to the secondary, the minor numbers of both the primary dtc device, and the secondary target device must match. In many cases this matching can be taken care of automatically by simply selecting 'Match Minor Numbers' in the GUI for a particular dtc device. This will cause a dtc device to be created with a device minor number that matches the target. If you are using the Softek Data Mobility Console this option is chosen through selection of Send Device Numbers. The device number specification will appear similar to the following in the Mobility Group configuration files:
MIRROR-DISK: /dev/rdest_log MIRROR-DEVNO: 1 40
Target Minor Number Collisions

In some cases, targets may be spread out over multiple volume groups that use different major numbers, allowing more than one target to have the same minor number. AIX does not support a native command that will allow the user to specify a minor number at volume creation time, so Softek Replicator includes an auxiliary command, dtcmklv, that allows volumes to be created with a particular minor number using the -p option. The following is an example of two target volume groups with a single volume in each Mobility Group:
VG Name = oracle Major Number 45 Logical volume Name oracledb Minor Number 1 VG Name = sybase Major Number 46 Logical volume Name sybasedb Minor Number 1 # ls -l /dev/roracledb crw-rw---- 1 root system 45, 1 Dec 28 09:24 /dev/roracledb # ls -l /dev/rsybasedb crwxr-xr-x 1 root system 46, 1 Dec 28 08:33 /dev/rsybasedb
As you can see, since both volume groups in the previous example have a logical volume with a minor number of 1. The above problem can be corrected by using the dtcmklv command, for modifying the target logical volume. To correct this configuration, you would need to change the logical volume
294
sybasedb to have a minor number other than 1. It is recommended to assign a range of minor numbers to each volume group so that there are no minor number collisions. The following is an example of using the dtcmklv command to set the minor number to 100.
# dtcmklv -t'jfs' -y'sybasedb' -p 100 rootvg 16 sybasedb # ls -l /dev/sybasedb brw-rw---- 1 root system 10,100 Dec 29 08:53 /dev/sybasedb
The following message during the dtcstart command processing indicates that you have minor device number collisions:
# dtcstart -g 1 DTC: [WARNING / MINORMISMATCH]: Minor numbers between dtc device /dev/dtc/ lg1/rdsk/dtc0 [17] and target device /dev/rlv01 [16] do not match
JFS/JFS2 Logs
It is not necessary to replicate the log volumes if you are doing a data replication, as long as the file systems are cleanly unmounted at the end of the data replication, before cutover. If the JFS log is not replicated, it will be necessary to ensure that the new JFS log for the target volume is formatted for use. The AIX logform command is used to format a new log volume. Also, ensure that the new /etc/filesystems entry is updated appropriately to point to the new JFS Log on both the source and target systems. This can be done by using the AIX chfs command. For data replications, a single Mobility Group can be created, containing all JFS logs. This Mobility Group is started and left in PassThru mode since there is no need to copy the JFS logs during a replication, if the JFS file systems are cleanly unmounted at the end of the replication, before cutover. The logs must be in a Mobility Group and used as dtc devices, even though they are not being replicated, to satisfy the AIX requirement that log files must be in the same volume group as their data volumes. All dtc devices appear to be a single AIX volume group.
Dtcinfo Command Extension

The dtcinfo command has been extended to show that minor device matching has been specified. The following is an example of the new dtcinfo output. The difference is Remote mirror device number shown in the following dtcinfo output. The format of the devices number is:
0xM00mm
where
D D D D 0x is an hexadecimal value M is the major number in hexadecimal format 00 is always 00 place holder mm is the minor number is hexadecimal format
Mobility Group 0 (127.0.0.1 -> 127.0.0.1)

Mode of operations.............. Normal Entries in the BAB.............. 0
www.ibm.com
295

Sectors in the BAB.............. 0 Sync/Async mode................. Async I/O delay....................... 0 Persistent Store................ /dev/Pstore Device /dev/dtc/lg0/rdsk/dtc0: dtc device number............... 0x2d0016 Local disk device number........ 0xa0015 Local disk size (sectors)....... 524288 Local disk name................. /dev/rs1024 Remote mirror disk.............. 127.0.0.1:/dev/rt1024 Remote mirror device number..... 0xa0016 Read I/O count.................. 0 Total # of sectors read......... 0 Write I/O count................. 0 Total # of sectors written...... 0 Entries in the BAB.............. 0 Sectors in the BAB.............. 0
Checkpoint Scenarios on Journaled File Systems

The Checkpoint process allows the target volume(s) to be accessed in Read-only or a Read/Write mode. This section will discuss how to do this properly in an AIX environment to insure that the data from the Primary Server is flushed out to the disk so that the most up-to-date version of the data is available on the Secondary Server. The key differences for the following scenarios are that JFS2 file systems require a dtcsyncfs command to occur to properly flush the data to disk, and the second difference is that any Read/Write access that occurs on the Secondary Server must be followed up with a Checksum Refresh.
Checkpoint Process JFS with Target Volume Enabled for Read -only Access
1. On the Primary Server, run the sync command to flush file system cache:
sync;sync
2. On the Primary Server, turn on the checkpoint as follows:

dtccheckpoint -g x -on
3. On the Secondary Server, mount the volume as read-only:

Mount -v'jfs' -r' ' /dev/targetlv /targetlv
4. When data processing is complete, proceed as follows: a. On the Secondary Server, unmount the volumes:
296
www.ibm.com

umount /targetlv
b. On the Primary Server, turn off the checkpoint:

dtcheckpoint -g x -off
For a sample script to automate these steps, refer to Checkpoint Process JFS with Target Volume Enabled for Read-only Access on page 299.
Checkpoint Process JFS with Target Volume Enabled for Read/Write Access
1. On the Primary Server, run the sync command to flush the file system cache:
sync;sync
2. On the Primary Server, turn on the checkpoint as follows:

3. On the Secondary Server, mount the volume with Read/Write access:

Mount -v'jfs' /dev/targetlv /targetlv
umount /targetlv

c. Issue a Checksum Refresh since the Secondary Server updated the data:
launchrefresh -g x -c
For a sample script to automate these steps, refer to Checkpoint Process JFS with Target Volume Enabled for Read/Write Access on page 300.
Checkpoint Process JFS2 with Target Volume Enabled for Read-only Access
sync;sync
2. On the Primary Server, issue the dtcsyncfs command to flush the JFS2 cache:
dtcsyncfs -f /targetlv
3. On the Primary Server, turn on the checkpoint:

4. On the Secondary Server, mount the volume as Read-only:

Mount -v'jfs2' -r' ' /dev/targetlv /targetlv
umount /targetlv
www.ibm.com
297

For a sample script to automate these steps, refer to Checkpoint Process JFS2 with Target Volume Enabled for Read-only Access on page 301.
Checkpoint Process JFS2 with Target Volume Enabled for Read/Write Access
sync;sync
2. On the Primary Server, issue the dtcsyncfs command to flush the JFS2 cache:
dtcsyncfs -f /targetlv
3. On the Primary Server, turn on the checkpoint:

4. On the Secondary Server, mount the volume with Read/Write access:

Mount -v'jfs2' /dev/targetlv /targetlv
umount /targetlv

c. Issue a Checksum Refresh since the Secondary Server updated the data:
For a sample script to automate these steps, refer to Checkpoint Process JFS2 with Target Volume Enabled for Read/Write Access on page 302.
JFS2 Specific Notes

JFS2 file systems are the newer versions of JFS file systems, which have been available since AIX Release 5. The added features seem to hold data longer in the JFS2 logs where previously in basic JFS file systems, the logs were emptied out quicker. Softek has added a command that flushes the JFS2 log buffer out to the file system. This command is necessary, in conjunction with a Checkpoint, to ensure that the data is flushed to the disk so that it is committed in a timely manner to the remote volumes. The following is an example of the dtcsyncfs command, which only currently supports JFS2 file systems:
dtcsyncfs -f /oracle
298
www.ibm.com
Scripting AIX Disaster Recovery Configuration Files

Configuration files can be generated via scripts for large configurations. If configuration files are generated by scripts, script code should be added to capture the target device numbers and to add the MIRROR-DEVNO parameter, as described above. We recommend that you create a script that automatically creates the Checkpoint scripts and a shell script to initiate the Checksum Refresh, as explained in the following sections.
Checkpoint Process JFS with Target Volume Enabled for Read-only Access
cp_pre_on_p000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_on_p000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_on_p000.sh >> /tmp/xpoint.log sync;sync exit 0 cp_post_on_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_post_on_s000.sh >> /tmp/xpoint.log mount -v'jfs' -r'' /dev/t1024 /t1024 exit 0 cp_pre_off_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_off_s000.sh >> /tmp/xpoint.log umount /t1024 exit 0
www.ibm.com
299
cp_pre_on_p000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_on_p000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_on_p000.sh >> /tmp/xpoint.log sync;sync exit 0 cp_post_on_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_post_on_s000.sh >> /tmp/xpoint.log mount -v'jfs' /dev/t1024 /t1024 exit 0 cp_pre_off_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_off_s000.sh >> /tmp/xpoint.log umount /t1024 exit 0
NOTE: Remember to issue a Checksum Refresh after the group has transitioned out of Checkpoint mode:
300
www.ibm.com
Checkpoint Process JFS2 with Target Volume Enabled for Read-only Access
cp_pre_on_p000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_on_p000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_on_p000.sh >> /tmp/xpoint.log sync;sync /usr/dtc/bin/dtcsyncfs -f /s1024 /usr/dtc/bin/dtcsyncfs -f /s1024 sleep 5 exit 0 cp_post_on_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_post_on_s000.sh >> /tmp/xpoint.log mount -v'jfs2' -r'' /dev/t1024 /t1024 exit 0 cp_pre_off_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_off_s000.sh >> /tmp/xpoint.log umount /t1024 exit 0
www.ibm.com
301
cp_pre_on_p000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_on_p000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_on_p000.sh >> /tmp/xpoint.log sync;sync /usr/dtc/bin/dtcsyncfs -f /s1024 /usr/dtc/bin/dtcsyncfs -f /s1024 sleep 5 exit 0 cp_post_on_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_post_on_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_post_on_s000.sh >> /tmp/xpoint.log mount -v'jfs2' /dev/t1024 /t1024 exit 0 cp_pre_off_s000.sh #!/bin/sh # ***** /etc/opt/SFTKdtc/cp_pre_off_s000.sh ***** echo ----------'date'---------- >> /tmp/xpoint.log echo starting cp_pre_off_s000.sh >> /tmp/xpoint.log umount /t1024 exit 0
NOTE: Remember to issue a Checksum Refresh after the group has transitioned out of Checkpoint mode:
302
www.ibm.com
Sample Script to Generate Checkpoint Scripts

#! /usr/bin/awk -f BEGIN {n=0;s=0; tprefix=""; aixcfgprefix="/etc/dtc/lib/"; aixbinpath="/usr/dtc/bin/"; # outf1=workdir "checkpt_on_group_"; outf2=workdir "checkpt_off_group_"; outf3=workdir "cp_pre_on_p"; outf4=workdir "cp_post_on_s"; outf5=workdir "cp_pre_off_s"; # # # printf "Which group number do you want chkpt scripts built for? "; getline ; if (length($1)!=0) {grpno=$1}; # deflog="/tmp/xpoint_g" grpno ".log"; printf "Where do you want your checkpoint log? <default press enter> " deflog ; getline ; if (length($1)!=0) {deflog=$1}; # # printf "Will you target volume be accessed in Read Only mode<y or n> "; getline ; if (length($1)!=0) {readonly=$1}; # if (length(grpno)==1){grpno="00" grpno;} if (length(grpno)==2){grpno="0" grpno;} outmp0=workdir "pwd"; a=system("pwd>"outmp0); getline<outmp0; workdir=$1; printf "specify working directory default<press enter>:"workdir " "; getline ; if (length($1)!=0) {workdir=$1}; workdir=workdir "/"; # # set final script names outf1= outf1 grpno ".sh" outf2= outf2 grpno ".sh" outf3= outf3 grpno ".sh" outf4= outf4 grpno ".sh" outf5= outf5 grpno ".sh" #
303

# # outmp1=aixcfgprefix"p" grpno ".cfg"; xx=0; while ((getline < outmp1) > 0) { if ($1=="DTC-DEVICE:") { a=split($2,sba,"/"); dtcdf="/" sba[2] "/" sba[3] "/"sba[4] "/" substr(sba[5],2) "/" sba[6]; dtcdev[xx]=dtcdf; } if ($1=="DATA-DISK:") { b=split($2,sbb,"/"); datadf="/" sbb[2] "/" substr(sbb[3],2); datadisk[xx]=datadf; } if ($1=="MIRROR-DISK:") { c=split($2,scc,"/"); mirrdf="/" sbb[2] "/" substr(scc[3],2); mirrdisk[xx]=mirrdf; xx=xx+1; } } # end of while loop outmp2=workdir "dtcdf"; outmp3=workdir "datadf"; for (n=0;n<xx;n++1) { #print dtcdev[n] " " mirrdisk[n]; a=system("lsfs|grep "dtcdev[n]">"outmp2); a=system("lsfs|grep "datadisk[n]">"outmp3); close(outmp2); close(outmp3); getline<outmp2; if (length($0)!=0) {fs[n]=$3;fstyp[n]=$4;} getline<outmp3; if (length($0)!=0) {fs[n]=$3;fstyp[n]=$4;} } # end for loop # # print "#!/bin/sh">outf3; print "# ***** " outf3 "*****">outf3 print "echo ----------`date`---------- >> " deflog>outf3; print "echo starting " outf3 ">> " deflog>outf3; print "sync;sync">outf3; # # print "#!/bin/sh">outf4; print "echo ----------`date`---------- >> " deflog>outf4; print "echo starting " outf4 ">> " deflog>outf4; # #
304
www.ibm.com

print "#!/bin/sh">outf5; print "echo ----------`date`---------- >> " deflog>outf5; print "echo starting " outf5 ">> " deflog>outf5; for (m=0;m<xx;m++) { #print datadisk[m] " " mirrdisk[m] " " dtcdev[m] " " fs[m] " " fstyp[m]; if (fstyp[m]=="jfs2") { print aixbinpath "dtcsyncfs -f " fs[m]>outf3; print aixbinpath "dtcsyncfs -f " fs[m]>outf3; } # if (readonly=="y") { # if (fstyp[m]=="jfs") { print "mount -v'jfs' -r'' " mirrdisk[m] " " fs[m]>outf4; } # if (fstyp[m]=="jfs2") { print "mount -v'jfs2' -r'' " mirrdisk[m] " " fs[m]>outf4; } } #endif readonly # if (readonly=="n") { if (fstyp[m]=="jfs") { print "mount -v'jfs' " mirrdisk[m] " " fs[m]>outf4; } # if (fstyp[m]=="jfs2") { print "mount -v'jfs2' " mirrdisk[m] " " fs[m]>outf4; } } #endif readonly print "umount " fs[m]>outf5; } #end of for loop print "exit 0">outf5; # # build checkpoint on/off scripts # # build checkpoint script print "exit 0">outf3; close(outf3); # print "exit 0">outf4; close(outf4); # print "echo turning checkpoint on for group " grpno>outf1; print aixbinpath "dtccheckpoint -g " grpno " -on">outf1; close(outf1);
305
if (readonly=="y") { print "echo turning checkpoint off for group " grpno" target READONLY">outf2; print aixbinpath "dtccheckpoint -g " grpno " -off">outf2; close(outf1); } if (readonly=="n") { print "echo turning checkpoint off for group " grpno " target READ/ WRITE">outf2; print aixbinpath "dtccheckpoint -g " grpno " -off">outf2; print "echo starting checksum refresh because target disk was changed ">outf2; print aixbinpath "launchrefresh -g " grpno " -c">outf2; close(outf2); } # # # # a=system("chmod +x " outf1); a=system("chmod +x " outf2); a=system("chmod +x " outf3); a=system("chmod +x " outf4); a=system("chmod +x " outf5); # # # } # last bracket # #
306
www.ibm.com
Appendix E
Sample Script for Changing Symbolic Links

Using the Sample Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .309
Sample Script for Changing Symbolic Links
This appendix describes the sample scripts that can be used for changing symbolic links, as described in Step 6 of Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS) on page 132. The file names of the sample scripts are as follows:
D D D D
AIX sample script: /usr/dtc/samples/chlink HP-UX sample script: /opt/STFKdtc/samples/chlink Linux sample script: /opt/SFTKdtc/samples/chlink Solaris sample script: /opt/STFKdtc/samples/chlink
Using the Sample Script

"
To use the sample script:
1. Make a copy of the sample script and study it. 2. Modify the script to meet your settings. 3. Change the execution permission and run the script.
Table E.1: Options
Option
-g <group#> SYMBOLIC_LINK_name
Comment Mobility Group number Symbolic link name that is targeted to change, in absolute path form.
CAUTION: Before running the sample script, you must perform Steps 1 through 5, under Optional Step: Configuring Softek Replicator for Relational Database Management Systems (RDBMS) on page 132. If you use this script for Oracle, the Oracle instance which is targeted to change, must be stopped. You must provide a valid definition for SYMBOLIC_LINK_name and -g <group#> (Mobility Group). The sample script is provided as is and is not guaranteed to function in all environments. You must modify the script to reflect your specific environment. If you run the sample script without modifying it, you will encounter errors. The sample script changes the specified symbolic link, for one dtc device at a time. If the specified Mobility Group includes multiple dtc devices, you must run the script multiple times against the Mobility Group. Uninstalling Softek Replicator will delete the original sample script.
www.ibm.com
309
Using the Sample Script
Sample Input
myserver# chlink -g 3 /oracle/space3
Output
The following message is displayed:
**Warning** This script needs to be modified to meet your Oracle settings, and please run this script on your own responsibility. Change Symbolic link /oracle/space3 Original Link destination is: /dev/rdsk/c0t1d3s1 New Link destination is: /dev/dtc/lg3/rdsk/dtc1 Do you want to change it now? (Y or N) > Completed
310
www.ibm.com
Appendix F
Implementing Softek Replicator With Solaris Zones

Replicating Data Associated with Solaris Containers . . . . . . . . . . . . . . . . . . . . . . . . .313
This appendix provides a sample procedure to replicate or clone a Solaris 10 Container from one location to another. It is assumed that the reader has some familiarity with Solaris Containers and Softek Replicator for Unix.
NOTE: Softek Replicator does not support the replication of data from a non ZFS file system (for example, VxFS or UFS) to a ZFS file system.
Replicating Data Associated with Solaris Containers

To prepare for replicating data, ensure that Softek Replicator is installed in the Solaris Global Zone. This is the only supported method for copying data within a Solaris zone environment.
Generic Approach
1. Verifying the prerequisites. To ensure proper operation, it is recommended to have the target system patched to the same level as the source system. Verify that the target volume matches the size of the source volume. 2. Set up Softek Replicator as explained in Chapter 7: Installing Softek Replicator on Solaris.. Install Softek Replicator version 2.6.3 or higher in the Solaris Global Zone ONLY on both the source and target systems. It is important to use the 2.6.3 version of the software because the Dynamic Driver Activation feature is included in this release. Build the appropriate configurations to support the replication or replication. The devices that should be replicated are the devices that are referenced in the zone configurations. To display these devices, use the zoneadm export command. 3. Saving the zone configuration. This step is necessary so that the target zone can be built, attached and booted. It is recommended to export the configuration for the zone on the same storage that is being replicated so the target zone can be started with the same configuration as the source zone. Use the zoneadm -z zonename export command to save the source zone configuration to the replicated storage. 4. Synchronize the data. Issue the Full Refresh command to synchronize the data.
NOTE: Ensure that the target system does not have the target zone running during a synchronization or during normal replication.
5. Test the remote copy of the data. There are two options for doing testing:
H
Option 1 is to gracefully stop the source zone (normally used for a data replication or for system cloning); A) Shutdown the source zone; zlogin zone1 shutdown -i 0 B) Detach the source zone; zoneadm -z zonename detach C) Ensure that the data has been sent to the secondary system by monitoring that the BAB is empty. Use the dtcinfo command. D) Create the target zone using the zone configuration that was exported in Step 3 on page 313: zonecfg -z zonename -f exported.config
www.ibm.com
313
It is a recommended to change the IP address in the remote copy of the configuration file before booting so that duplicate IP addresses aren't encountered on the network. E) Attach the container to the remote system and boot the zone.
zonecfg -z zonename attach -u
The -u option will not be necessary if the source and target systems are patched to the same levels. F) Boot the target zone using the following command: zoneadm -z zonename boot
H
Option 2 is used when a failure of the primary system requires the target system to be used. This is typically used for a disaster recovery scenario. A) Mount the volume in order to access the zone configuration that you have replicated. B) Issue the dtcrmdreco command on the target server to stop any new inbound replication from the production source to the Disaster Recovery target server. Example: dtcrmdreco -g 0 C) Create the target zone using the zone configuration that was exported in Step 3 on page 313: zonecfg -z zonename -f exported.config This configuration file should be on the target system because of the ongoing replication. It is recommended to change the IP address in the remote copy of the configuration file before booting so that duplicate IP addresses aren't encountered on the network. D) Attach the container to the remote system and boot:
zonecfg -z zonename attach -u The -u option will not be necessary if the source and target systems are patched
to the same levels. E) Boot the target zone using the following command: zoneadm -z zonename boot
Example
Below is a complete example showing the Solaris commands. This example includes creating the zones which would already be in place in a production configuration.
Set up zone1 with a file system on source Solaris 10 server

1. Create a partition, file system and mount the file system. Create a partition on an available disk using the format command; for example, c1t0d0s0.
#newfs /dev/rdsk/c1t0d0s0 #mkdir -p /export/zone1 #mount -F ufs /dev/rdsk/c1t0d0s0 /export/zone1
314
www.ibm.com
2. Create a zone config file to create a simple zone named zone1.cfg, using a text editor.
create -b set zonepath=/export/zone1 set autoboot=true add net set address=10.10.10.2 set physical=bge0 end
3. Create a zone named zone1 using the zonecfg command and configuration file that was created in the previous step.
#zonecfg -z zone1 -f ./zone1.cfg #zoneadm list -cv ID NAME STATUS PATH 0 global running / - zone1 configured /export/zone1
BRAND native native
IP shared shared
4. Adjust the permissions for the new file system.

#chmod 700 /export/zone1
5. Install the zone named zone1.

#zoneadm -z zone1 install Preparing to install zone <zone1>. Creating list of files to copy from the global zone. Copying <137879> files to the zone. Initializing zone product registry. Determining zone package initialization order. Preparing to initialize <1141> packages on the zone. Initialized <1141> packages on zone. Zone <zone1> is initialized. Installation of <4> packages was skipped. The file </export/zone1/root/var/sadm/system/logs/install_log> contains a log of the zone installation. #zoneadm list -cv ID NAME STATUS 0 global running - zone1 installed #zoneadm -z zone1 ready #zoneadm list -cv ID NAME STATUS 0 global running 1 zone1 ready
PATH / /export/zone1
BRAND native native
IP shared shared
PATH / /export/zone1
BRAND native native
IP shared shared
6.
Boot the new zone named zone1 and login:

#zoneadm -z zone1 boot #zlogin -C zone1 .. login to the zone1 console.
7. zone1 is now running on source server, so you can set up a 1-1 group in Softek Replicator. Install Softek Replicator version 2.6.3 GA on source and target servers in the global zone only.
www.ibm.com
315
8. Use the Softek Data Mobility Console or dtcconfigtool to create a 1-1 group 0 that replicates c1t0d0s0 to the target server slice. 9. Ensure that Dynamic activation=on is configured by looking at the configuration file(p000.cfg) and finding the DYNAMIC-ACTIVATION: On section. 10. Start the Mobility Group using the following command:
#dtcstart -g 0
11. Synchronize the data using the following Full Refresh command:
# launchrefresh -f -g0
12. Once the Normal state is reached, proceed to either Procedure 1 or 2. Use procedure 1 or procedure 2 below to be able to bring up zone1 on the target server.
Procedure 1 With Graceful Shutdown

Source Server 1. Gracefully shut down the source zone:
login into zone1 and shutdown the zone #zlogin zone1 shutdown -i 0
2. Save the zone configuration to the replicated volume:

#zonecfg -z zone1 export > /export/zone1/zone1.cfg
3. Detach the zone to create a clean image for the replication:

#zoneadm -z zone1 detach #unmount /export/zone1
4. Once the I/O is quiesced and BAB is empty, kill the PMD, use the dtcinfo command to check that the BAB is empty. Use the following commands to accomplish this task.
#dtcinfo -g 0 #killpmds -g 0
** Ensure that there are no entries in the BAB before proceeding. **
Target Server 5. If the target volume is clean, make the target file system available for the zone:
mount the target slice as /export/zone1 #mount /dev/dsk/c0t0d0s0 /export/zone1
NOTE: If the volume doesn't mount without a fsck, this means the graceful shutdown was not successfull.
6. Create the target zone using the zone configuration that was exported on the source system.
#zonecfg -z zone1 -f /export/zone1/zone1.cfg
7. Validate that the configuration succeeded as expected:

#zoneadm list -cv
316
www.ibm.com
8. Attach the zone using the replicated storage:

#zoneadm -z zone1 attach (-u)
NOTE: The -u option is required if the target and source servers pkgs/patch levels differ. -u will remove and install patch levels consistent with target global zone. Normally if the source and target systems are at the same maintenance levels, the -u option is not necessary. This may affect any application within the zone1 we are replicating or replicating. The -F parameter may be used when you desire not to update the target zone, but this may lead to system instability.
9. Boot the target zone:

#zoneadm -z zone1 boot
10. Verify that zone1 can be accessed and is ok:

#zlogin -z zone1
Procedure 2
This procedure does not detach zone1 on source server and attaches a "fuzzy" zone1 on the target server. This "fuzzy" zone1 means that zone1 was not cleanly shutdown. Source Server When the group is in Normal state after a Full Refresh, and zone1 is active on the source server with I/O, perform the following steps:
NOTE: If the group is not in Normal state at failure time, the viability of the target depends on the setting of "Use journals", and may be a valid point in time copy that is not immediately current. It should be recoverable with an fsck.
1. Create a current configuration that will be replicated to the remote system. This must be refreshed if the configurations change frequently. The replication happens automatically if the configuration file is placed on the disk that is being replicated. In this example, /export/zone1/zone1.cfg is automatically replicated to the remote system because /export/zone1 is being replicated.
#zonecfg -z zone1 export > /export/zone1/zone1.cfg
Wait for this I/O to complete and to be replicated to the remote site. 2. To simulate a network outage or source server crash, kill the PMD using the following command:
#killpmds -g 0
Target Server 3. Make the target storage and container available for use. The following command sequence does a mount of the file system, configures the zone and attaches it.
mount target slice as /export/zone1;# mount /dev/dsk/c1t0d2s0 /export/zone1
The configuration file below was moved to the target machine by the replication process:
#zonecfg -z zone1 -f /export/zone1/zone1.cfg #zoneadm list -cv #zoneadm -z zone1 attach (-u)
www.ibm.com
317
NOTE: You will get the following message showing that zone1 was not detached:
#zoneadm: zone 'zone1': The zone was not properly detached. Attempting to attach anyway.
NOTE: The -u option is required if the target and source servers pkgs/patch levels differ. -u will remove and install patch levels consistent with target global zone. Normally if the source and target systems are at the same maintenance levels, the -u option is not necessary. This may affect any application within the zone1 we are replicating or replicating. The -F parameter may be used when you desire not to update the target zone, but this may lead to system instability.
4. Before booting zone1 on the target server, you need to decide what to do with zone1 on the source server. There will be an IP address conflict if both source and target zones are running at the same time. It is up to you to change the IP address on the new zone1 on the target server or shut down zone1 on the source server.
#zoneadm -z zone1 boot
5. Verify that zone1 can be accessed and is ok by logging in with the zlogin command
#zlogin zone1
6. To remove a zone, run the following command to delete and clean up any zone1 at anytime:
zoneadm -z zone1 uninstall
At this point, the zone is in the configured state. To remove it completely from the system, use this command:
zonecfg -z zone1 delete
NOTE: You need to unmount the target file system after a test, before resuming a replication or disaster recovery operation: - Shut down the target zone. - Launch a Full or Checksum Refresh.
318
www.ibm.com
Index
Index
Symbols
.cfg 82 .Cfg Files 51 .cfg files /Var/Dtc/ 40 /Var/Opt/Sftkdtc 49, 59, 60, 66
action message 217 distributing 110 overview 110 throttle definitions in 213 used by start command 193
.conf 112 .cur files 110, 193 .off files 191 .phd 206 .prf 206 .tar.Z.uu 175 /Etc/Dtc/Init.D/Dtc-Chkpt_Boot 40 /Etc/Dtc/Lib 40, 102 /Etc/Dtc/R3.D/S25dtc-Startdaemon 40 /etc/filesystems file 117 /etc/fstab|vfstab 123 /Etc/Init.D 59, 66 /Etc/Opt/Sftkdtc 49, 59, 66 /Etc/Rc0.D 59 /Etc/Rcs.D 66 /etc/services file 228 /Lpp/Dtc.Rte 40 /Opt/Sftkdtc/ 49, 59 /Opt/Sftkdtc/Bin 49, 59, 66 /Opt/Sftkdtc/Doc 49, 66 /Opt/Sftkdtc/Driver 59, 60 /Opt/Sftkdtc/Lib 66 /Opt/Sftkdtc/Samples/Chlink 49, 66 /Sbin 49 /Sbin/Init.D 49 /Sbin/Rc3.D 49 /Usr/Conf 49 /Usr/Dtc/Bin/ 40 /Usr/Dtc/Lib/ 40 /Usr/Dtc/Samples/Chlink 40 /Usr/Lib/Drivers/Dtc* 40 /Usr/Lib/Methods/Dtc* 40 /Usr/Local/Bin 49, 59, 60, 66 /Usr/Lpp/Dtc.Rte* 40 /Usr/Sbin 59, 66 /Usr/Sbin/Dtcstart 40 /Usr/Sbin/Dtcstop 40 /Usr/Sys/Inst.Images/Dtc.Rte* 40
www.ibm.com
Numerics
127.0.0.1 231 25% Of The Entire Amount Of Data 33, 77 32-Bit Kernel BAB on AIX 32 HP-UX 48 migration to 64-bit kernel 34 8-Digit Hexadecimal Number 101
A
Absolute path 134, 309 Access immediate 8 root 79 Access Mirror Devices 33, 77 Accumulate mode 255 Accumulating New Updates 33, 77 ACK 83 acknowledgment 216 Action Directives 216 Action Parameters CHUNKDELAY 217 CHUNKSIZE 217 COMPRESSION 217, 223 MAXSTATFILESIZE 217 STATINTERVAL 217 SYNCMODE 217 SYNCMODEDEPTH 217 SYNCMODETIMEOUT 217 TRACETHROTTLE 217 ACTIONLIST 213 ACTIONS 213 Actions Menu 50 Activity Of Production Applications 30 ACTUALKBPS 204, 213, 215, 217 Add To Chart 205 Adding Mobility Group Autostart Option 105 Additional Physical Memory 32 Administrator 79 Agent daemon 198
319
Index
introduction 109 kill 194 launch 198 Removing on HP-UX 51, 67 set 171
AIX
32-bit kernel 104 dtcjfspostmount 180 dtcjfspremount command 179 HACMP Clusters 160 Building Configuration 161 Configuring 160 Disabling Softek Replicator Unix 163 Pre-configuration steps 161 JFS/JFS2 Logs for Data Replications 116, 295 JFS2 external logs 107 JFS2 journals applied 194 Matching Device Minor Numbers 294 minor device number 183 Minor Number Collisions 294 Replicating JFS/JFS2 114 set minor device number 176
Aix
At Least As Large As 31, 73 At least as large as 107 Automated switch-over 89 Autostart Option Adding Mobility Group 105 Average Amount Of Data Changed 33, 77 Avoid BAB overflows 23
B
BAB 29, 30 clear 186
Alerts 184, 185 Allocation, tablespace 228 Allow chaining 107, 232 Allow data to be accessed from the secondary system 256 Amount Of Data 32, 33, 77 Amount of information 223 Amount of time the dtc device driver waits 225 Application server 228 Applied to the mirror device 234 Architecture Primary System 14 ASCII file 204 report 210 Assign Enough Memory 29 Assigning a MAC address 168 Associate dtc devices on AIX 122 Async mode 124, 224 Asynchronous mirroring 5 Replication 7 throttles 216 transfers 224 Asynchronous Mode 21 320
creating the license file 102 Directory Structure 40 Installation 41 requirements 39 Sizing the BAB 32 supported migrations 34 Uninstall 43
Components 17 Primary Server 17 dtcinfo 177 may cause overflow 223 modifying 233 number of I/Os 224 overflow 126 overflow during refresh 234 percentage 215 Primary Server Components 17 recovering from overflow 257 resize with dtcinit 179 tracking overflow 209
Bab
amount of memory 31, 73 Back Up The System Fully 39, 47, 55, 65 Backfresh Operating State 22 Backfresh operation after a recovery 258 Mode 22 Background operation 23 Backup 33, 77 Bandwidth 29, 85, 215 regulate 223, 224 Before Connecting New Devices 39, 47, 55, 65 Before Installing Softek TDMF (IP) 39, 47, 55, 65 Before Starting The Installation 31 Bias the Group 90 Big Asynchronous Buffer 17 Binary form 80 block 0 127 Blocks Of Changed Data 33, 77 Boot Scripts 40 Buffer size 83 Business analysis 7 Business Continuity 6 Bytes Per Minute 76
www.ibm.com
Index Collector
C
Cached Configuration Files 43, 52, 62, 68 Calculate The Average Amount Of Data Changed 33, 77 Card, Lan 101 C-C Link 82 CFGFILE 217 Chaining A to B Setup 96 B to C Setup 96 Configuration 95 defining 107, 232 Change over to the secondary system 228, 256 Changeover 255 Changing the MAC Address 167 Chart 203 Displaying 205 Number of samples 204 Setting up 204 Specifying measurements 204 Checkpoint 33, 77 Command/Action Matrix 239 loopback configuration 97 checkpointing activating 237 considerations and limitations 240 file systems 244 Normal Mode 240 primary system scripts 241 secondary system scripts 242 shell scripts 240 stopping 239 token in BAB 244 Checksum method 190 refresh 24, 190, 200 chfs 116 chlink 309 chown 133 CHUNKDELAY 192, 215, 217, 223 CHUNKSIZE 84, 192, 215, 217, 223 Clauses 213 Clusters HP-UX 159 restarting data Replication 152 setting up in UNIX 151 Veritas 153 Coexistance Softek TDMF (IP) and Softek TDMF for UNIX 35 Coherence Or Recoverability Of The Data 33, 77 Coherent and recoverable state 191 Coherent Set Of Devices 33 Coherent State 234 Journal File Directory 20
www.ibm.com
Security 79
Commands
panalyze 187 Commit any buffered updates 256 Commit Device 90, 108 Compatibily Matrix UNIX migration 34 Compatible Devices Disk-based File System 11 Raw Disks 11 complex configurations chaining configuration 95 loopback configuration 97 one-to-many configuration 93 symmetic configuration 89 Components 13 Mobility Server 13 Primary Server 13, 14 Secondary Server 13, 19
COMPRESSION 192, 215, 217, 223 Net Actual, Net Effect 209 Compression 29, 81, 84 Configuration loopback 30 Steps on UNIX 102 symmetric 31, 73 Veritas clusters 153 configuration chaining 95 current 110 file sample 111 logical group 110
Primary Server BAB 17 Configuration File 15 Daemon PMD 17 dtc device driver 16 Master Daemon in.dtc 15 Mobility Group 14 Persistent Store (Pstore) 18 HRT Memory-based Bitmap 18 LRT disk-based Bitmap 18 relationship 14 Secondary Server Daemon RMD 19 Journal File Directory 20 Coherent State 20 Incoherent State 20
321
Index
loopback 97 one to many 93 symmetric configuration 89

Configuration And License Files 40 Configuration File Components 15 Configuration Files 15, 49, 59, 66 Configuration files devlist.conf 247 distributing 110 p###.cfg 110 processing 193 s###.cfg 110 Configuration Limits 30 Configuration Tool Interfacing with 12 Configurations chaining 95 loopback 97 one-to-many 93 symmetric 89 Configuring File Systems 113 Confirmation Dialog 50 Connection 208 Connection field 255 Connection Type 76 Connection, independent 106, 231 Considerations on UNIX 74 Control blocks 80 Copying configuration files 111 Cost-effective 8 Courier transport method 81 courier transport method 126 cp_post_on_p###.sh 240 cp_pre_on_p###.sh 240 CPU 215, 217 Cpu 30 Cpu Available 29 CRC checks 81 Create new device 108 Creating symbolic links 133 Cylinder 81
Primary Server Components 15
Components 17 port number 227 Primary Mirror 11 Remote Mirror 11 RMD Components 19 Secondary Server Components 19
Data
Data Coherence 33, 77 Data Flow 81, 83 Data Replication Data Replications
accessed from the secondary system 256 compression 223 flow 209 monitoring 203 ownership switching 191 recovery 253, 255 Replication 126
Restarting in case of failure 152 JFS/JFS2 Logs 116, 295 Data Structure Definitions 34 Data Transfer 31, 73 Database Management Systems 34 Databases 125 configuring product with 132 non-symbolic linked devices 134 symbolic links 133 DATE 217 day-of-month 214 day-of-week 214 DBMS tablespace allocation 228 decipher trends 224 decompressed 215, 223 Decrease The Bab Size 32 defaultvfs is JFS and JFS Will Be Used 118 defaultvfs is JFS and JFS2 Will Be Used 118 defaultvfs is JFS2 and JFS Will Be Used 119 defaultvfs is JFS2 and JFS2 Will Be Used 120 Defining a Group 104 dev stanza 172 device failure 259 mirror 107 naming 107 status area 208 user-defined 247 Device Naming AIX File System Expansion 113 devlist.conf 247 Directives, action 216
www.ibm.com
D
Daemon
kill 181, 182, 196, 197 PMD Components 17 Primary Server
322
Index Directory Dtc Device Driver 32 dtc device driver Components 16
Journal File 20 Coherent State 20 Incoherent State 20

Directory Structure 40 on AIX 40 on HP-UX 49
Primary Server Components 16
Disaster 5, 258 Disaster Recovery 5 Long distance 102 tool 6 Veritas Clusters 153 disaster recovery 255 steps 255 Disk encryption 79 Disk access, optimize 223 Disk label information 108 Disk Partition 29 Disk Space on AIX 39, 47 on Linux 55, 58 on Solaris 65 Disk Subsystem 30 disk-based Bitmap
on Red Hat Linux 59 on Solaris 66 on SuSE Linux 59
Dtc Remnants 52 DTC.lic 199 Dtc.Lic 102 dtcAgent.cfg 109 dtcagentset 109, 171 dtcautomount 172 dtcbackfresh 173 dtccheckpoint 173 Dtcconfigtool 32, 42, 50, 61, 67 dtcconfigtool 174, 256 Defining a Group 104 Defining the BAB 104
LRT
Pstore Components 18
Disk-based File System
Compatible Devices 11 disruption 6 Distributing Configuration Files 110 Double BAB overflow 257 Downtime Required 74 Driver 48 AIX 40 Files on HP-UX 49 DRIVERMODE 215, 217 dtc device 228 changing symbolic links 309 for chaining 97 for symmetric configuration 89 information 177 mount information 172, 184
mounting existing JFS file systems 121 mounting new JFS file systems 116 mounting new JFS2 file systems 117 put in refresh mode 190 syncmode 224 Volumes Expansion 135 Volumes Expansion in HA/CMP Cluster 140
Defining Tunable parameters 223 dtcdebugcapture 175 Dtcerror.Log 49, 59, 60, 66 dtcerror.log 216, 259 dtcexpand 176 dtcgenmklv 176 dtchostinfo 177 Dtcinfo 32, 48 dtcinfo 123, 177, 210 dtcinit 112, 179 dtcjfspremount 179 dtckillbackfresh 181 dtckillpmd 181 dtckillrefresh 182 dtckillrmd 182 dtclicinfo 182 dtclimitsize 183 dtcmklv 117, 183 dtcmodfs 184 dtcmonitortool 126, 184, 255 dtcmonitortty 185 dtcoverride 186 dtcperftool 126, 187, 203 dtcpmd 188 dtcrefresh 190 dtcrmdreco 191 dtcset 192, 223 dtcstart 110, 193, 227 Dtcstop 42, 43, 50, 52, 61, 62, 67, 68 dtcstop 193, 227 dtcsyncfs 194 Dynamic Activation Solaris 12
www.ibm.com
323
Index
E
EFFECTKBPS 204, 205, 215, 217 Elapsed times for Replications 76 Encoded binary form 80 Encryption 79, 81 ENDACTIONLIST 213 Entering License Information 102 ENTRIES 215 Entries 204 Entry age recvd 205 Error and Warning Messages 208 File Table Overflow 17 Messages 125, 184, 185 Error Insufficient Memory 32 Establish an initial remote mirror 190 Ethernet 83, 167 Ethernet Media Access Control 167 Events Disaster definition 5 Exact copy of the configuration file 193 Exceeding Bab Size Limitations 32 Executable Programs 40 Ext2 55, 57 Ext3 55, 57 External log 107, 118, 120, 121 External Logs
configuring on AIX 113 configuring on HP-UX, Linux, and Solaris 123 ext2, ext3 55, 57 identical 34 JFS/JFS2 39, 47 UFS 65 VxFS 47, 65
File Table Overflow Error 17 File Table Overflow Error HP-UX 108, 199, 231 Files Configuration 15 Ownership 132 Files Systems Compatible Devices 11 Firewall 82 Flush data from the journal file 191 Forces a transition 186 Frequency, of throttle evaluation 224 From An Older Version To A Newer Version 34 from-time 214 fsck command for checkpointing 237 in data recovery 256 FSTAB file 172 FTP 75, 154 Full Refresh 30 Chunksize 84 in migrations 29 Initial Mirror 81 journals 33 full refresh 23, 200
AIX
Replicating JFS/JFS2 114
F
failover
of Active Node to Passive Node With Replication Through Single IP Address 155 of Active Node to Passive Node without the IP address as Part of Cluster Service Group 156 UNIX clusters 152 with VCS and Network Applications 157 with Veritas Cluster Server 157
Failure
G
General
Planning Steps on UNIX 102 Setup for UNIX clusters 151

General Requirements 29 Getconf 48 Graphical charting tool, start 187 Graphical user interface 12 GROUPNO 217
Fcp 55, 57 File System Expansion Device Naming 113 File system superblocks 194 File Systems 324
Network 255 of device on the primary or secondary systems 259 over to the secondary system 258 Primary system 256 Secondary System 255 to transition out of tracking mode 257
H
HACMP 160
Building Configuration 161 Disabling Softek Replicator Unix 163 in Cluster Environment 160 Pre-configuration steps 161
www.ibm.com
Index Hardware Incoherent State
IBM POWER 39 UltraSPARC 65

Hexadecimal Number 101 Hexadecimal number 167, 168 High Availability 6, 152 AIX HACMP 160 HP-UX sample script 263 High load burst 225 High Resolution Tracking
Host ID 167 Host Id 101 Host-based Replication 7 Hostname or IP Address 105, 231 hostnames 177 HP-UX Configuration Example 74 File Overflow Error 17 File Table Overflow Error 108, 199, 231 MC/Serviceguard Clusters 159 Script example 261 Hp-Ux Directory Structure 49 Installation 49 license key 41, 50, 60, 67 patch requirement 48 requirements 47 supported migrations 34 Uninstall 52 HRT 186
Memory-based Bitmap Pstore Components 18

Hw_Cpu_Supp_Bits 48
Journal File Directory 20 Incoherent state 234 Independent connection 106, 231 Information, amount 223 Initial copy 126 Initial Startup, UNIX clusters 151 Initialize the BAB 109, 171 Initialize the Pstore 179 Install (Analysis) 50 Installation Files 40 Installation Instructions 39, 47, 55, 65 Installing Softek TDMF (IP) On AIX 41 on HP-UX 49 On Linux 60 on Solaris 66 Instructions on installing 39, 47, 55, 65 Insufficient Memory Errors 32 Interfacing Configuration Tool 12 Softek Data Mobility Console 12 Internal disk structures 117 Internal log 119, 120 Internet Protocol Version 6 25 Inverting the A-B Topology to B-A 92 Iostat 31, 73 IP address 82, 151 Security 81 Veritas 154 IP addresses 177 Ip Network 29 Irreplacable Files 39, 47, 55, 65 Iscsi 55, 57 Iterative Implementation 31, 73
I
I/O Characteristics 31, 73 I/O updates 216 I386-Based 34 Ibm Power 34, 39 Ide 55, 57 Identical Database Management Systems 34 Identical File Systems 34 Identical partitions 90 Identify deltas 190 Immediate access 8 Imported By The New Version 42, 50, 61, 67 in.dtc 195, 227 in.dua 198 Incoherent Data 33, 77
www.ibm.com
J
j###.###.c 234 j###.###.i 234 JFS 113 Jfs 39 JFS data-device 179, 180 JFS log volume 116 JFS log-device 179 JFS/JFS2 Logs for Data Replications 116, 295 JFS2 246 journals applied 194 limitations 121 Jfs2 39 Journal File 29
325
Index
checkpointing 33, 77 per group 30 size 30 sizing 31, 73 sizing the directory 33, 77
Journal File Directory
Secondary Server Components 20 Coherent State 20 Incoherent State 20

journal file directory
adding to /etc/vfstab 123 creating 106, 232 definition 234 Journal Operations 235 resizing 235
Journal files 257 Journal-less Processing 236 JOURNALVAL 223
K
KBPS 217 Keep the application from freezing up 225 Kernel memory size 104 Kernel Issues Lowering BAB Size 104 Kernel Memory 32 BAB 17 Kernel_Bits 48 Key, private 80 killagent 109, 194 killbackfresh 195 Killdtcmaster 42, 43, 50, 52, 61, 62, 67, 68 killdtcmaster 195, 227, 228 Killpmds 43, 50, 52, 61, 62, 67, 68 killpmds 196 killrefresh 196 killrmds 197 Kmmodreg 52
launchrefresh 200 Libraries 40 Libraries For Red Hat Linux 56 Libraries For SuSE Linux 58 License File removing on AIX 43, 52, 62, 68 license file 199 License Information 102 License Key 101 on HP-UX 41, 50, 60, 67 Licensing 182 on VMware 167 Licensing Files 49, 59, 66 licinfo 182 Limit the statistics 224 Link Architecture 78 Link destination 310 Linked tests 213 Linux Installation 60 supported migrations 34 Uninstall 62 Upgrade 61 Listen Port 227 Loadable Driver 48 Load-balancing 89 Local Data Device 29, 31, 33, 73, 77 local data device initial copy of 126 Local host information 177 Local Read, Loc. Write 209 localhost 231 Location, remote 5 log device 113, 179, 180, 194
device on existing JFS file systems 121 device on new JFS file systems 116 device on new JFS2 file systems 117 meta data 194 stanza 172
L
LAN 79
card 167 Lan Card 101 LAN-attached clients 159 Large Servers 29 launchagent 109, 198 launchbackfresh 198 launchdtcmaster 199, 227, 228 launchpmds 199
326
Logfile 50 logform 116 Logging feature 125 logical group deleting 232 Logical Volume Type 117 LOGSTATS 224 Loopback Configuration 30 loopback configuration 97, 231 Low Latency 30 Low Resolution Tracking
Lparid.Cfg
www.ibm.com
Index
Using 35
LRT 186
disk-based Bitmap Pstore Components 18

lsfs 117 Lsvg 43 LUNs
metadata 117 Methods 40 Metrics, Performance 203 Migration sizing 31, 73 Migration Group 33 Migration of storage 12 mincache=dsync 125 minor device number 176, 177, 183 Minor Device Numbers Matching 294 minor number 117, 120 Minor Number Collisions 294 Mirror Daemon 83 Mirror data from a database 256 Mirror device data applied to 234 defining 107 minor device number 120 Mirror Devices 29 Accessing 33, 77 Mirroring Asynchronous 5 synchronous 5 mkfs 124 mklv 183 mkvg 121 Mobililty Group Autostart Option 105 Mobility Group Components 14
Replicating Data on ZFS Using Loopback Configuration 146 Using One-to-OneConfiguration 143
LVM 117 Lvm 55, 57
M
MAC Address Maintenance mode 173, 198 Maintenance on Primary Server Using Backfresh 22 Major device numbers 177 Managing Online Replication 246 mapping local and mirror devices 228 Mark For Install 49 Master daemon
avoiding changes 167 manually assigning 168
in.dtc Components 15 Primary Server Components 15 port 106, 231 port changing 227
Match Minor Numbers 107 Matching Device Minor Numbers 294 Matrix UNIX migration compatibility 34 Matrix, Checkpoint 239 maxfiles_lim
Primary Server Components 14 Throttles 25
Mobility Server
Components 13 Primary Server 13, 14 Secondary Server 13, 19

Mode % Done 209 Modes 22
Tunable Parameters File Table Overflow Error 17

Maximum number of errors/messages 184 Maximum Rate For Data Transfer 31, 73 Maximum Transmission Unit (MTU) 83 MAXSTATFILESIZE 192, 206, 215, 217, 224 Measurement keywords 215 Media Access Control (Mac) 101 Memory 30, 31, 73 Additional BAB 32 Memory-based Bitmap
Asynchronous 21 field 255 Near Synch 21 passthru 23 refresh checksum 24 full 23 smart 24 Synchronous 21
modfs 184 Modify Measurement 205 Modifying the fstab/vfstab file 124
HRT
www.ibm.com
327
Index monitoring BAB 209 data 203
data flow 209 performance files 206 tunable parameters 206
Tunable Parameters File Table Overflow Error 17

NIC cards 85 No connection or awareness of the other 95 No Logging mode 125 Non-symbolic linked databases 134 Not shared by the primary and secondary systems 228 Notification and Update Controls 209 Number Of Devices 30 disk blocks 31, 73 Journal files per group 30 Migration Groups 30 pstores 30 Number of Samples 204
monitortool 184 monitortty 185 mount command 256 mountall 125 Mounting
existing JFS file systems 121 mirror devices 245 new JFS file systems 116 new JFS2 file systems 117
MSDE
User access 79 MTU 83 Multi-level recovery software 8 Must Be Identically Sized 31, 73 Must be unique 107
O
ODBC 82 Offline Replication 103 One-to-Many Configuration 93 Online Replication 103 Online Replication on AIX JFS2 246 Open Systems Red Hat Linux Requirements 55 Operating Mode Asynchronous 21 Near Synch 21 Synchronous 21 Operating Modes 21 Operating modes 22 Backfresh Mode 22 dtcinfo 177 Forces a transition 186 Normal mode 23 Passthru Mode 23 Refresh Mode 23 Tracking Mode 24 Operating Parameters 30 Operating State Backfresh 22 Operating System Upgrading 36 Optimize disk access 223 Optimize network throughput 223 Outage 31, 32, 73 outage 74 override 186 Ownership files 132 of data 191 of mirror devices 257
www.ibm.com
N
Naming convention 110 journal 234 Near Synch Mode 21 Near-real-time 78 Net Actual 209 NETCONNECTFLAG 215, 217 NETKBPS 215 NETMAXKBPS 74, 192, 215, 218, 224 netstat 85 Network Bandwidth 85 bandwidth 29, 215 regulate 223, 224 encryption 79 Failure 255 Measuring for long distance 85 numbers for sending 1 GB of data 75 outage 31, 32, 73 slow 29 Speed 76 speed 30 TCP/IP 79 Network throughput 215 optimize 223 Network-based volume Replication 78 Networked hostnames and IP addresses 177 New Replication group 104, 231 newfs 124 nfile 328
Index
P
p###.cfg 81, 82, 110 p###.off 191 p###.prf,Performance Monitoring Files 206 p000.cfg 227 Packet of data 215 Packets 83 panalyze command 187 Parallel Scsi 55, 57 Parameters and Modes
Tunable Parameters 20 Parameters For Operation 30 Pa-Risc 34 Partitions identical 90 PassThru Mode 154 Passthru Mode 23, 90, 123 password 79 Patch Requirement 48 Path Environment Variable 50, 60, 67, 68 Path, absolute 134 PCTBAB 215 PCTCPU 213, 215 PCTWL 218 Peak I/O Update Activity 31, 73 Peak Update Rates 29 Percent BAB full 204 Percent done 204 PERCENTBABINUSE 215, 218 PERCENTDONE 216, 218 Performance 30, 113, 125 and error logs on AIX 40 data 187 Displaying the Chart 205 file size 215, 224 logs on HP-UX 49 logs on Linux 59, 60 logs on Solaris 66 Monitor Window 203 Monitoring Files 206 of applications 224 regulate PMD 223 Setting up the Chart 204 Specifying Measurements 204 statistics 184, 185, 224 Tool 203 update calculation 216 Persistent Store (Pstore) Primary Server Components 18 HRT Memory-based Bitmap 18 LRT
www.ibm.com
disk-based Bitmap 18 persistent store (pstore) 231 Phase The Full Refresh 29 Physical Memory 29 Physical Memory Available 39, 47, 55, 58, 65 Physical raw device 124 Physical sector 0 108 PID 216, 218 Pkgadd 66 Pkgrm 67, 68 Planning the installation On UNIX 74 PMD 11, 23, 85, 223 Data flow 83 relationship with RMD 17 PMD Daemon kill 181, 196 start 188, 199 Point to the mirror devices 257 Point-in-time 5, 8 Port 67 port number agent 171 changing connecting port 106, 231 changing on secondary system 228 Ports 1035 82 575 81, 82 576 82 Power 39 Preparing Pstore 112 Preserve Tunable Parameter Settings on AIX 42, 50 on Solaris 67 Primary Data Set 32 Primary Lan Card 101 Primary Mirror Daemon 11, 83 Components Primary Server 17 BAB
Primary Replication Group file sample 111 Primary Server
Component 17 Components Mobility Server 13, 14 Configuration File Component 15 Daemon PMD Component 17 dtc device driver Component 16 maintenance on Using Backfresh 22
329
Index
Master daemon in.dtc Component 15 Mobility Group Component 14 Persistent Store (Pstore) Component 18 HRT Memory-based Bitmap 18 LRT disk-based Bitmap 18 Server Primary 29
Primary Server Server rate of changes 33, 77 Primary System memory 32 Source Volume 15 Primary system changing port number 227 device failure 259 failure 256 restoring after failover 258 primary system
R
Raid 55, 57 Random ordering of blocks 81 Rate For Data Transfer 31, 73 Rate Of Changes 33, 77 Ratio Of Reads To Writes 31, 73 Raw device 124 Raw disk devices 125, 134 Raw disk partitions 132 Raw Disks Compatible Devices 11 Raw Volume Device 29 R-C Link 82 RDBMS environments 132
script cp_post_on_p###.sh 241 cp_pre_on_p###.sh 241

Private key 80 Processor 84 Production Application Activity 30 Protocol
Internet Version 6 25
P-S Link 80 PStore
Components 18
Pstore
Components HRT Memory-based Bitmap 18 LRT disk-based Bitmap 18 disk partition cannot be shared 29 number of 30 Preparation 112 size 30 sizing 31
pstore
switching to secondary system 256 Read KBps 204 Read/Write 29 Read-Only Access 33, 77 Ready to assume control of operations 89 Reboot The System 32 Rebooting primary system on HP-UX 246 Recover The Original Files 39, 47, 55, 65 Recover your data 255 Recovery 253 Point Objective 5 Time Objective 5 utility 191 Veritas clusters 153 Red Hat Linux Directory Structure 59 Requirements 55 Open Systems 55 Reestablish a remote mirror 190 Refresh 31, 73 Refresh mode 23 after a recovery 258 checksum 24 command 190 full 23 launch 200 smart 24 Refreshing the set of data 23 Regulate network bandwidth 224 Regulate the performance of PMD 223 ReHat Linux Libraries Required 56
Relationship
defining for Replication group 231 preparing 112 relocating 233
between all components 14 Relocate operations to a secondary system 258 Rem_Drv 67 Remote Location 5
www.ibm.com
330
Index Remote Mirror Daemon 11, 83 Remove From Chart 205 Remove The Core Software 51 Remove The Driver on Linux 61, 67, 68 Removing Agent HP-UX 51, 67 Removing Softek TDMF (IP) On HP-UX 52 On Linux 62 On Solaris 68 Replicating
Components Secondary Server 19
Red Hat Linux 55 Opens Systems 55 Solaris 57 SuSE Linux 65

Resizing the journal file directory 235 Restarting data Replication 152 Restarting in case of failure, Data Replication, See also Data Replication 152 Restore the database 132 Restoring the primary system 258 RFC1323 84 RFC2018 84 Risk Assessment 80 Rm 43 RMD 11, 85, 155, 156 Data flow 83 relationship with PMD 17 RMD Daemon kill 182, 197 rmdreco 191 Rmlv 43 Rmmod 61 rollback 257 Root 41, 49, 60, 66 Root access 79, 81 Round trip network time 224 Rpm 60 RPO 5, 7 RTO 5, 7 Running B as Stand-Alone 91
data from Cluster Node to a Disaster Recovery System

154
data in VMware 167 data with FTP 75

Replicating Data
ZFS
Using Loopback Configuration based on LUNs 146 Using Loopback Configuration based on Slices
147
Using One-to-One Configuration based on LUNs

143
Using One-to-One Configuration based on Slices

144 Replicating Data based on LUNs
Using Loopback Configuration 146 Using One-to-OneConfiguration 143

Replicating Data based on Slices
Using Loopback Configuration 147 Using One-to-OneConfiguration 144

Replicating Data on ZFS 141 Replicating JFS/JFS2 114 Replication
S
s###.cfg 81, 82 S700_800 11.11 Som2elf 47, 48 SACK 84 Same Configuration 34 Same host name or IP address 97 Same minor device numbers 117 Same window size 84 Sample Chaining Definitions 95 One-To-Many Definitions 93 script for Oracle database 309 Symmetric Configuration 89 Sample Script 40, 49, 66 Sar 31, 73 Save the group definition 108 Scenarios Replicating Veritas data 154 Script cp_post_on_p###.sh 236, 238 cp_post_on_s###.sh 238
distributing configuration files 110 Host-based 7 in a Virtual Environment 165 in RDBMS environments 132 managed by the Softek Data Mobility Console 26 starting 126 starting the agent 109
Replication Group
Definition of 104, 231 start 193 stop 193
Request For Comment 84 Requested Memory 32 Requirements AIX 39 General 29 HP-UX 47 HP-UX patch 48
www.ibm.com
331
Index
cp_pre_off_s###.sh 236, 239 cp_pre_on_p###.sh 236, 238 for maintenance activities 195 primary system sample 241 rc 125 sample for Oracle databases 309
Script Samples
Scsi 55, 57 Second BAB overflow 257 Secondary Server
for HP-UX MC ServiceGuard 263 For MS Exchange 291 for Veritas clusters 285
Components Mobility Server 13, 19 Daemon RMD Component 19 Journal File Directory Component 20 Coherent State 20 Incoherent State 20 journaling 33, 77
Secondary System
Shell Prompt 101 Significant I/O 126 Single Cd 39, 47, 55, 65 Single Server Loopback Configuration 30 Size Of A Journal File 30 Sizing BAB 31 journal file directory 33, 77 mirror devices 31, 73 on AIX 32 the BAB 104 the journal file directory 235 the network 74 the system 31, 73 SLEEP 218 Slices
Replicating Data on ZFS Using Loopback Configuration 147 Using One-to-OneConfiguration 144
Slow Networks 29 Smart Refresh 33, 77 smart refresh 190, 200 Socket Buffer Size 227 Sockets 83 Softek Data Mobility Console Interfacing with 12
assumes the role of primary 89 changing port number 228 copying configuration files on 111 defining IP address 106, 231 device failure 259 failing over to 258 failure 255 journaling 20 switching to 256
Sector 81 Sector 0 108 Sector by sector synchronization 190 Security overview 78 Risk Assessment 80 User access 79 users 79 Selective ACKnowledgement 84 Semi-synchronous throttles 216 Server downtime on UNIX 74 Secondary 29 Server and storage migration 12 Server Resources 29 Server Resources Available 30 Set Of Devices 33 Set The Bab Size To Zero 32 Sets the Journal feature ON or OFF 223 Sftkdua 51, 67 332
Managing Replication from 26 Security 79
Softek Replicator Using 11 Softek TDMF (IP) Module 59, 60 Solaris Directory Structure 66 Dynamic Activation 12 Installation 66 requirements 57 Uninstall 68 Upgrade 67 Solaris 8 124 Source Volume
Volume Source 15
Speed
network 30, 32 source disks 30
Starting Replication 126 Starting Softek TDMF (IP) on AIX 122
on HP-UX, Linux, and Solaris 123 Startup Scripts 49 Startup/Shutdown Scripts 59, 66 State change 234 State independence 104, 231 STATINTERVAL 192, 206, 216, 217, 218, 224
www.ibm.com
Index Status Message Area 207 Store updates 234 Stores the paths to raw disk devices 132 Striped volumes 132 Structure Directory 40 superblocks 194 Superuser 79 Support Libraries 66 Supported Operating Parameters 30 Supporting Libraries 49, 59 SuSE Linux Directory Structure 59
T
Tablespace allocation 228 Target Server 29 TCP Settings 227 TCP Window 84 TCPWINDOWSIZE 85 TCP/IP 79 sockets 83 Template Rc Scripts 49, 59, 66 Temporary Data 29 Terminates any Backfresh operation 195 Terminates any Refresh operation 196 Terminates Refresh mode 182 throtd 195 throttles action directives 216 action messages 217 defining 219 editing 213, 255 examples 222 keywords 215 overview 25 tests 214 Throughput 85 network 215 optimize 223 TIME 218 Time-Ordered Writing Coherence 33 Title, Chart 204 to-time 214 TRACETHROTTLE 192, 216, 217, 218, 225 Tracking bitmaps 193 failure to transition out of 257 Mode 24 Tracking Mode 31, 73 Transfer Rates 30 Transfer Speed 33, 77 Treated as a single unit 104, 231 Trends, Performance 203 TRUE 213 Tunable Parameters 20 action 217 CHUNKDELAY 223 CHUNKSIZE 223 COMPRESSION 223 JOURNALVAL 223 maxfiles_lim 17 MAXSTATFILESIZE 224 monitoring 206 NETMAXKBPS 224 nfile 17 overview 223
Libraries Required 58 Requirements 65
Swap Space 29 Swinstall 50 Switching Back from A to B 92 Switching data ownership over to the secondary system 191 Switching to the Secondary System 256 Switch-over 89 Swremove 51 Sybase starting an instance 151 Symbolic link 132, 133, 257, 309 Symbolic Links 40, 49, 66 to user executables 49, 59, 60, 66 Symmetric Configuration 31, 73 Symmetric configuration 89 defining 90 Inverting the A-B Topology to B-A 92 Running B as Stand-Alone 91 Switching Back from A to B 92 sync 92 Sync mode 95, 124, 224 Synchroning Primary Server from Secondary Server Using Backfresh 22 Synchronization 78 Synchronize 126 Synchronize The Primary And Secondary Servers 31, 73 synchronizing after a device failure 258 after a secondary system failure 256 after installation 113 Synchronous mirroring 5 Synchronous Mode 21 Synchronous, throttles 216 SYNCMODE 192, 216, 217, 218, 224 SYNCMODEDEPTH 192, 216, 217, 218, 224 SYNCMODETIMEOUT 95, 192, 216, 217, 218, 225 System sizing 31, 73
www.ibm.com
333
Index
set command 192 STATINTERVAL 224 SYNCMODE 224 SYNCMODEDEPTH 224 SYNCMODETIMEOUT 225 TRACETHROTTLE 225
Turning Journal Operations Off 235 Type, Chart 204
VMware 167 Vold Process 66 Volstat 31, 73 Volume 29 Volume Expansion dtc device 135
in HA/CMP Cluster 140
U
UFS 125
Caution when replicating UFS on Solaris 125

Ufs 65 UFS File Systems
Using Synch Mode 21 Ultrasparc 65 umount 125 Uname 101 uncommitted partial transactions 257 Understanding business need 7 Unpredictable I/O Loads 29 Updates Accumulating New 33, 77 Upgrading Softek TDMF (IP) On AIX 42 On Linux 61 On Solaris 67 Upgrading The Operating System 36 User Access 79 User Executables 49, 59, 66 user ID 79 User interface 12 User Manuals 49, 66 User-defined Devices 247 Using Backfresh Maintenance on Primary Server 22
Volume Manager 259 volume managers 132 Volume table of contents 108 Volumes Disks 55, 57, 58 VxFS 123, 125 Vxfs 47, 65 VxFS File Systems Using Synch Mode 21 VxVM 132
W
WAN 79 Windows Domain Security 79 Windows Replication Console
symmetric configuration 89 Windows Replication Replication Console 108, 231 Writable directory 106, 232 Write KBps 204 Writing Coherence 33 Writing coherence 104, 231
X
X-Windows 39, 47, 55, 58, 65
Z
ZFS
Synchroning Primary Server from Secondary Server 22
Using Lparid.Cfg 35 Using Softek Replicator 11 Using Synch Mode UFS File System 21 VxFS File System 21 Utility to reestablish the daemon environment 199
Replicating Data 141 Using Loopback Configuration based on LUNs 146 Using Loopback Configuration based on Slices
147
Using One-to-One Configuration based on LUNs

143
Using One-to-One Configuration based on Slices

144
V
VCS 153 Veritas 153
Cluster Server 154 Script examples 285

Virtual machine 167 Virtual Memory 32 334

Printed in Canada
For more information, please visit us at www.ibm.com 2010 IBM Corporation. IBM, the IBM logo, AIX, z/OS, Softek, and Softek Replicator for UNIX are registered or common law trademarks of IBM Corporation in the United States, other countries, or both. All other trademarks and product names are the property of their respective owners. The information in this document may be superseded by subsequent documents.
REP-U26IR-016

Repunix26 Install Reference r16

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Repunix26 Install Reference r16

Загружено:

Авторское право:

Доступные форматы

IBM Data Mobility Services Softek Replicator for UNIX

IBM Data Mobility Services Softek Replicator for UNIX

FOR FURTHER INFORMATION RESTRICTIONS ON USE

Tcl/Tk License Terms

Softek Replicator for UNIX 2.6.x

New Features for 2.6.5

Technical Updates for 2.6.5

New Features for 2.6.4

Licensed Material Property of IBM Corporation

Technical Updates for 2.6.4

New Features for 2.6.3

New Features for 2.6.2

Support for AIX 6.1. Support for SuSE 9 Linux.

Licensed Material Property of IBM Corporation

About This Guide

About This Guide

Contacting Technical Support

By Email: Send email requests to DMSOpen@us.ibm.com for:

North America: 1-800-667-6383

Other Countries: +800-800-667-6383 Alternate number : +1 918-281-2623

About This Guide

Registering for Email Notifications

To register for email notifications:

5. Follow the on-screen instructions to complete registration.

Licensed Material Property of IBM Corporation

Chapter 2: Understanding Softek Replicator for UNIX . . . . . . . . . . . . . . . . . . . . . . . 9

Chapter 3: Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Minimum and Maximum BAB Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Contents Other Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Chapter 4: Installing Softek Replicator on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Chapter 5: Installing Softek Replicator on HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Chapter 6: Installing Softek Replicator on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Chapter 7: Installing Softek Replicator on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Network Speeds and Elapsed times for Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

Determining the Quality of the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Chapter 9: Complex Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Chapter 10: Getting Started With Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

101 101 102 102 102 103

Licensed Material Property of IBM Corporation

103 104 108 109 110 112 112 113 113

Sample Configuration File of a Primary Mobility Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

Configuring File Systems on HP-UX, Linux, and Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

132 132 133 134

Chapter 11: Online Volume Expansion of dtc Devices . . . . . . . . . . . . . . . . . . . . . . . 135

Chapter 12: Replicating Data on ZFS File Systems . . . . . . . . . . . . . . . . . . . . . . . . . 141

143 144 146 147

Chapter 13: Using Softek Replicator in a Cluster Environment . . . . . . . . . . . . . . . . 149

Veritas Cluster Replication Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Chapter 14: Replication in a Virtual Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Chapter 15: Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Licensed Material Property of IBM Corporation

Chapter 16: Using the Monitoring Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Performance Monitoring Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

207 207 208 208 209

dtcinfo ASCII Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Chapter 17: Administrative Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Creating Throttle Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Throttle Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Using the Checkpoint Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Examples of Checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Checkpoint Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Considerations and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Checkpoint Shell Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Checkpoint Scripts for File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

245 246 246 246 247

Accessing a JFS2 Target Drive in Case of Primary Server Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .247