Академический Документы
Профессиональный Документы
Культура Документы
Authors: Seewah Chan Matthias Hein Paul Lekkas Rose L. Manz Howard E. Poole Michael R. Sheets
Document Owner: Michael R. Sheets IBM Americas Solutions Advanced Technical Support
TABLE OF CONTENTS 1 2 3 4 5 6
6.1 6.2
INTRODUCTION............................................................................................................................. 4 REMINDERS .................................................................................................................................... 4 TRADEMARKS................................................................................................................................ 5 FEEDBACK ...................................................................................................................................... 6 ACKNOWLEDGEMENTS ............................................................................................................. 6 UNICODE MIGRATION PROCESS............................................................................................. 6
Standard Import and Fastload Processes..................................................................................................................7 Import Measurements.................................................................................................................................................7
6.3 Details ...........................................................................................................................................................................8 Creating the Export Files ........................................................................................................................................................8 Tuning.....................................................................................................................................................................................8 Balancing the Threads.............................................................................................................................................................9 Fastload and Partitioned Tablespaces .....................................................................................................................................9 Index Considerations ..............................................................................................................................................................9
7
7.1 7.2
CONFIGURATIONS ..................................................................................................................... 10
Hardware Environment............................................................................................................................................10 Software Environment ..............................................................................................................................................12
8 9 10 11
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 2 of 15
TABLES
Table 1: Import Measurement Results ..................................................................................................................................12
FIGURES
Figure 1: SAP Unicode Conversion Overview ........................................................................................................................6 Figure 2: Conceptual View of Hardware Configuration......................................................................................................11 Figure 3: Import Throughput - GB/Hour..............................................................................................................................13 Figure 4: Import CPU Efficiency ...........................................................................................................................................14
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 3 of 15
1 Introduction
The Unicode standard provides a unique numeric representation for every character in every language. It is platform and program independent. This is particularly important for ideographic characters such as used in Chinese, Japanese, and Korean. Unicode currently defines over 90,000 characters with room for over one million unique characters. Prior to Unicode, on the same computer platform, several languages could, and would, use the same representation for different characters. Further, different platforms and programs sometimes used conflicting representations. More about Unicode may be found at [1]. Unicode is not used to represent numeric data. To provide support for multinational companies and companies throughout the world, SAP AGs products are embracing Unicode. Most of their NetWeaver offerings support Unicode. Some require Unicode. The same is true of their SAP Business Suite and ECC 5.0. This, in turn, places Unicode requirements on the relational database SAP uses for its DB Server. Further, there are several encoding forms within Unicode (e.g., UTF-8, UTF-16, and UTF-32). IBMs DB2 for z/OS Version 8 and its associated prerequisite products [2, 3] provide this support for SAP on IBMs System z using the UTF16 encoding form. SAP customers who want to migrate their SAP systems to Unicode are facing a challenge. They need to copy their systems using SAPs heterogeneous system copy function. Heterogeneous system copy unloads (i.e., exports) all the data in their database to the application server. There it is converted to Unicode. Finally it is loaded back (i.e., imported) into an empty database system. The normal process for the import is that one or more Standard Import processes are started on the application server that owns the files with the exported data. These processes load the data into the target database using INSERT SQL-statements. In the common case for most large SAP installations, a three-tier system, the INSERT statements are sent over the network to the target database. This procedure requires CPU resources on the DB Server and, because of the network latency, long elapsed times for single threaded processing. For SAP installations using DB2 for z/OS, these issues can be minimized by using a Fastload process that facilitates larger amounts of data to be sent at once over the network. This data are subsequently loaded into the database using the DB2 LOAD utility, instead of INSERT statements. This paper describes a set of measurements of the Standard Import and the Fastload processes. We concentrated on this portion of the Unicode migration because it is lengthy and requires the SAP system be shut down. Not only do we compare the two options, we also investigated the scalability of the Fastload process.
2 Reminders
Neither this document nor any part of it may be copied or reproduced in any form or by any means or translated into another language, without the prior consent of the IBM Corporation. IBM makes no warranties or representations with respect to the content hereof and specifically disclaims any implied Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 4 of 15
IBM Americas Advanced Technical Support warranties of merchantability or fitness of any particular purpose. IBM assumes no responsibility for any errors that may appear in this document. The information contained in this document is subject to change without any notice. IBM reserves the right to make any such changes without obligation to notify any person of such revision or changes. IBM makes no commitment to keep the information contained herein up to date. Performance is sometimes stated in Internal Throughput Rate (ITR) ratios based on measurements and projections using standard benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput equivalent to the performance stated here. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of nonIBM products should be addressed to the suppliers of those products.
3 Trademarks
SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and all SAP product and service names mentioned herein are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. AnyNet, CICS, DB2, DFSMSdfp, DFSMShsm, DFSMSrmm, DFSORT, DS6000, DS8000, ESCON, FICON, GDPS, HiperSockets, HyperSwap, IBM, IBM eServer, IBM logo, IMS, InfoPrint, Language Environment, Multiprise, NetView, OMEGAMON, Open Class, OS/390, Parallel Sysplex, PrintWay, RACF, RMF, S/390, Sysplex Timer, System Storage, System z, System z9, SystemPac, Tivoli, TotalStorage, VSE/ESA, VTAM, WebSphere, z/Architecture, z/OS, z/VM, z/VSE, and zSeries are trademarks or registered trademarks of the International Business Machines Corporation in the United States and other countries. Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries Linux is a trademark of Linus Torvalds in the United States and other countries. Unicode is a trademark of Unicode, Inc. Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 5 of 15
IBM Americas Advanced Technical Support UNIX is a registered trademark of The Open Group in the United States and other countries. Other products may be trademarks or registered trademarks of their respective companies.
4 Feedback
Please send comments or suggestions for changes to msheets@us.ibm.com
5 Acknowledgements
The authors would like to recognize the contributions of Manfred Olschanowsky, Joachim Rese, Wolfgang Reichert, and Andrea Fuga.
Conversion
Conversion preparation in non-Unicode system (SPUMG)
Several database scans Generate vocabulary (if using MDMP)
SAPinst (R3load)
Database export Conversion using vocabulary and control information
Database Import
Standard Import (Insert-SQL) DB2 Load (Fastload using trusted row format)
Post-Conversion (SUMG)
Adjustment of incorrectly converted data Automated conversion Usage of repair hints
The SAP system must be shut down and is unavailable to the business from the start of the Export until the end of the Import. Often SAP is crucial to customers business operations. The unavailability of an Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 6 of 15
IBM Americas Advanced Technical Support SAP system can, for example, stop manufacturing, product shipments, processing of orders, accounts receivable, accounts payable, etc. Thus, especially for large SAP systems, the time needed for Export and Import of the data is very critical. This study concentrates only on the Import part of the conversion. We didnt look at the Export because there were so many possible source systems to consider. For example, just considering the database there are at least five options. Operating systems and hardware options possibilities further compound the options. Having said that, our feeling is that Export by itself is often faster than import. It is possible to run Export and Import in parallel. That is, the Export is started, and when the first packages are created, Import can start processing them. One of the main reasons we looked at the Import, as discussed below, was that the Fastload was being developed and we had the opportunity to test early versions of the code. Finally, the Import step is considered by many customers to be one of the most crucial in the entire process.
6.1
In preparation for the import, the SAP data were exported from the source database into flat file packages on the Application Server. The unloaded data are then imported into the target system by either the SAP Standard Import process for heterogeneous system copies, or by the new Fastload process. For the Standard Import, the SAP R3load processes read the exported data one package at a time and create an SQL INSERT statement for each row of data to be imported and send it to the database. However, this can be expensive in terms of resources consumed and especially elapsed time. This is due to not only shipping one row at a time, but also because of the extensive checking done by DB2. There is also a new option - the Fastload import [5]. Fastload is intended to be less costly in terms of resources consumed, and especially in elapsed time. Fastloads data format is especially suited to DB2 Load. Another advantage of Fastload is it sends larger blocks of data at one time, reducing network resources. For the Fastload import, the R3load processes spawn two new processes. One reads a package with converted data, converts it into a format suitable for the DB2 for z/OS LOAD utility, and sends the complete package via file transfer program (FTP) to the z/OS database server. The second process spawned by the R3load process kicks off the DB2 LOAD utility using a call to a DB2 stored procedure. The target of the FTP transfer can be either z/OS Batch Pipes or z/OS USS pipes. At the time we made these measurements, USS Pipes was not yet available so we used Batch Pipes. Most likely in the future only USS Pipes will be supported. We expect USS Pipes to have similar performance to Batch Pipes.
6.2
Import Measurements
As part of IBMs continuous testing and measuring of its products, we have been testing and measuring various aspects of SAP several years. We do this to gain experience with SAP as it is used by customers and as it evolves. Over the years, we have published several papers like this one through IBMs Techdocs Website (http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs). In fact, this is not the first paper we have published related to Unicode. In 2005, we published [6], which showed Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 7 of 15
IBM Americas Advanced Technical Support some of the performance effects of running SAP Unicode systems with z/OS. However, these measurements did not address the SAP Unicode migration process much less the Fastload import which is the focus of this paper. The primary goal of these measurements was to test early versions of the Fastload to provide an assessment of the performance. Because we were interested only in the Import, we skipped as many steps in the Unicode Migration process as possible to improve our productivity. We exported an SAP Banking database from some previous work. As this database was already Unicode, we could skip the Unicode conversion which is normally done during Export and doesnt affect Import. All the tests shown here were performed in a 3-tier setup with a 16w p5-570 IBM System p server running the application server functions as well as the migration tools. Attached to it via two Giga-bit Ethernet lines was a System z9 z/OS LPAR running DB2 for z/OS hosting the SAP database system. Measurement of the Standard Import process was done to get a baseline, as comparison for the Fastload. The Standard Import measurement used a dedicated 8-way System z9 LPAR The Fastload measurements were performed on 8-way, 12-way and 16-way z/OS LPARs to also investigate the scalability of Fastload. We encountered no apparent constraints in our measurements.
6.3
Details
Tuning
Here are some other tuning techniques that we used. For the Fastload process, the LOAD utility is automatically invoked with the LOG NO parameter. However, even in this case, the creation of indexes is still logged. Therefore, it is advisable to disable DB2 subsystem logging for inserts at the subsystem level. This is often done for time critical migrations of large databases. Use fully qualified CCSID in LOAD statements or apply PTF PK55077 to minimize Unicode conversion (DSNURGVC) overheads. Reduce LOAD utility Inline statistics to use only 5% sampling or omit creating inline statistics altogether. In the published runs, no inline statistics were created. Use VLF/CAS - for ICF usercats to improve access to DB2 data sets. Disable AUTOTUNING for ICF usercats and increase usercat catalogue parameters STRNO, BUFND, and BUFNI as below o ALTER 'UCAT.UAP' BUFND(70) STRNO(69) o ALTER 'UCAT.UAP.CATINDEX' BUFNI(70) Implement TCP/IP multipath static routing [7, 8] to increase network bandwidth. Provide sufficient DFSORT workfiles used by index build phases during Fastload.
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 8 of 15
IBM Americas Advanced Technical Support Drop NPI indexes on two of the largest partitioned tables (BCA_CN_EVENT and BCA_CN_LINK).
Index Considerations
Because indexes are necessary for the performance of any productive SAP system, we created them as part of the import process. The only exceptions were two non-partitioned indexes (NPI) on very large partitioned tables. Creating those during the load of the data would have caused too much contention. This is because there is a separate LOAD process for each partition, and all of these would need to Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 9 of 15
IBM Americas Advanced Technical Support create entries in the same NPI. We expect customers would run into the same situation with production systems as well. They should find the very largest partitioned tables using NPI and create these indexes after the system went live again. While index creation was part of the import, we decided not to include the size of the indexes when calculating the throughput numbers as discussed in Section 9, Analysis on page 12. The reason for this is that the size and number of indexes may vary from installation to installation. Further, the data contained in the indexes are not moved in the migration, but are recreated from the tablespace data.
7 Configurations
7.1 Hardware Environment
System z DB Server: Measurements were done on a single z9 Enterprise Class Model S18 with a total of 128 GB installed. The runs utilized one dedicated LPAR with 8, 12, and 16 processors corresponding to 2094-708, -712, and -716 systems respectively. The amount of central memory assigned to this dedicated LPAR was 32GB. Database DASD: The target database resided on a single frame IBM TotalStorage Model DS8300 (2421-932) server with 15K RPM disks configured as RAID 5. The effective total capacity was 520 3390-mod54 volumes about 27 TB. The unit has 128 GB regular cache, 4 GB non-volatile storage (NVS), and 32 long wave FICON ports. However, only eight FICON ports were used for these tests. For the SAP database we chose an existing SAP Transaction Banking system (TRBK) with 5 million accounts. The database size was about 1 TB of allocated space, which contained 370GB of active pages in table spaces and 120GB of active pages in index spaces. However, as discussed in Section 6.3 Index Considerations on page 9, we did not count the index spaces in our throughput calculations. Application Server: We used a System p p5-570 with sixteen 2.2 GHz processor cores using SMT and 128 GB of memory. Network: Gigabit Ethernet networks were used for all connections. The p570 application server was connected to the z9 via an OSA-Express2 1 Gb adapter using two ports. Below is a conceptual view of the configuration.
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 10 of 15
8 FICON connections
Hardware: SAP Database Server : z9 Enterprise Class S18; One z/OS LPAR w/ 8, 12, or 16 CPs and 32 GB of central memory ; Software: zOS 01.08.00 DB2 9.1 GA driver
System z9 EC 2094-S18
LPAR 1 8 CPs or 12 CPs or 16 CPs
32 GB memory DB2
OSA-Express2 adapters
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 11 of 15
7.2
Software Environment
z/OS z/OS release 01.08.00 DB2 9.1 level dated 3/20/2008 with fixes DB2 Connect Informational tokens are: DB2 v9.1.0.0 Fix Pack 0. AIX AIX 5.3 release level 06 SAP Application Levels SAP NetWeaver 2004s, SP12 for basis, SP13 for BW, kernel release 700 patch no 102 with Fastload extension.
8 Measurement Results
During the course of this effort many measurement runs were done. Some were to get familiar with the environment and the workload. Some were for debugging. Several were to test changes to the Fastload. It is beyond the scope of this paper to show them all. In general, these measurements were not tuned as one might do with a real benchmark. Listed below are the measurement results selected as being the most useful, given the time constraints of this effort.
Standard Import 40 S80318U1 2094-708 8 Standard Import 40 370 8,156 0.5957 38,868 163 1 1
Fastload - 46 S80429U2 2094-708 8 Fastload 46 370 3,353 0.7454 19,995 397 2.43 1.94
Fastload - 48 S80602U1 2094-712 12 Fastload 48 370 2,415 0.7639 22,138 552 3.38 1.76
Fastload - 64 S80609U1 2094-716 16 Fastload 64 370 2,152 0.6733 23,183 619 3.79 1.68
Runid Processor Model # of processors Method # of streams Database size (GB) Elapsed time (sec) z/OS CPU utilization CPU busy time (sec) Throughput (GB/hour) Throughput (ETRR) CPU Efficiency Ratio
9 Analysis
Analysis of various aspects of these measurements was done. In some cases this analysis used standard Large Systems Performance Reference (LSPR) techniques and methodology [9]. It is assumed the reader is familiar with the terms therein, especially External Throughput Rate (ETR), Internal Throughput Rate (ITR) and the corresponding ratios (ETRR and ITRR). All the ITR and ITRR based Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 12 of 15
IBM Americas Advanced Technical Support results were considering the DB Server utilizations only. Thus, they should be considered as DB ITRs or DB ITRRs. In general, we found the Fastload process to be significantly faster and more efficient than the Standard Import. The system utilizations of Fastload ranged from 67% to 76%. We did not see any indications of severe constraints. We feel that we could have increased utilization and perhaps reduced elapsed time further by fine-tuning the creation of the export tables and balancing the threads. From a pragmatic business perspective, probably the best figure of merit for the Import is the GB/hour of the different measurements. This metric can be used to most directly to estimate how much time could be required for an import. However, as discussed in Section 6.3 Index Considerations on page 9, we did not count the index spaces in our throughput calculations. We counted only the active pages in the table spaces. The results (see Figure 3: Import Throughput - GB/Hour,below) show that the Fastload provides significantly more throughput than the Standard Import. Further, it shows that the Fastload throughput can be increased by adding more parallel streams and processors. The best Fastload measurement is 3.79 times the GB/hour as the Standard Import.
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 13 of 15
IBM Americas Advanced Technical Support Dividing the above throughput by the CPU seconds of the Import can give a view of the CPU efficiency of the different measurements. In Figure 4: Import CPU Efficiency, below, we normalized this with respect to the Standard Import. All the Fastload measurements were significantly more CPU efficient in moving the same amount of data as the Standard Import. However, as the number of streams and processors was increased, the efficiency of the Fastload fell off somewhat. The peak efficiency of all the Fastloads was almost twice as efficient as the Standard Import.
1.5
0.5
0 Std. Import - 40
Figure 4: Import CPU Efficiency
Fastload - 46
Fastload - 48
Fastload - 64
10 Conclusions
More and more businesses are gaining or increasing their international perspective. Put differently, businesses are more often participating and competing in an international arena. That is, dealing with other geographies in terms of suppliers, partners, and customers. This can require Unicode support for their computer applications. Even in the rare cases where this is not happening, application providers like SAP are going down a path where new versions of their applications require Unicode. It is only a matter of time before virtually every SAP customer will have to face Unicode. Some portion of Unicode SAP systems will be new installs. However, most customers will have to migrate currently existing SAP systems to Unicode. For the Import portion, we have found the Fastload import significantly better than the Standard Import. It takes significantly less elapsed time per database GB. It also requires less processor resources per database GB. However, the import data rates and CPU costs we documented in this paper should not be used for sizing purposes. Import times can by influenced by many situation unique factors such as the use of data compression, the types and use of indexes, and the specific import configuration should also
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 14 of 15
IBM Americas Advanced Technical Support be considered. Customers should carefully study the SAP migration materials and notes and work with their local IBM representatives to make sure they use the best current advice and tips.
11 References
[1] Unicode, Inc., http://www.unicode.org/ [2] IBM Corp. 2003. DB2 UDB for z/OS V8: Through the Looking Glass and What SAP Found There (SG24-7088) or from http://www.ibm.com/redbooks [3] IBM Corp. 2004. DB2 UDB for z/OS Version 8: Everything You Ever Wanted to Know, ... and More (SG24-6079) or from http://www.ibm.com/redbooks [4] SAP AG 2006. System Copy for SAP Systems Based on SAP NetWeaver 2004s SR2 ABAP+Java from https://service.sap.com/instguides [5] SAP AG 2008. SAP Unicode Conversion on DB2 for z/OS - Cookbook from SAP on DB2 for z/OS forum in the SAP Community Network: https://www.sdn.sap.com/irj/sdn/db2 [6] IBM Corp. 2005. Measurements of SAP ECC 5.0 with Unicode on IBM eServer zSeries Using DB2 for z/OS V8 from http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100635 or msheets@us.ibm.com [7] IBM Corp. 2004. AIX 5L Differences Guide Version 5.3 Edition, Section 7.2, (SG24-7463) or from http://www.redbooks.ibm.com/redbooks/pdfs/sg247463.pdf [8] IBM Corp. 2008. Communications Server for z/OS V1R9 TCP/IP Implementation Volume 3: High Availability, Scalability, and Performance, Section 3.2 (SG24-7534) or from http://www.redbooks.ibm.com/redbooks/pdfs/sg247534.pdf [9] IBM Corp. 2006. Large Systems Performance Reference (SC28-1187) or http://www.ibm.com/servers/eserver/zseries/lspr/
Unicode Migration of SAP on IBM System z: Evaluation of Fastload for Import Copyright IBM Corporation 2008 All Rights Reserved Page 15 of 15