Oracle E-Business Suite Release 12.

2: Upgrade Sizing and Best Practices

This document contains database schema sizing information, application tier sizing information, upgrade timing, and other best practices, all designed to make your upgrade to Oracle E-Business Suite Release 12.2 as smooth as possible. The sizing data is based on the upgrade of a 456 GB Release 12.1.3 database to Release 12.2. There is a change log at the end of this document.

In This Document

Application Tier Size Tablespace Size Schema Size R12.2 Best Practices Required Post upgrade Jobs Sample Upgrade Timings

Database Size: Before Upgrade DB Size (GB)

456

After Upgrade DB Size (GB)

481

Delta (GB)
25

% Growth
5.5

The 25 GB greater database size after the upgrade reflects the additional system tablespace needed as a prerequisite for enablement of Edition-Based Redefinition (EBR). Environment details
Operating system: Server Memory: Number of CPUs: Database Release: Oracle E-Business Suite: Oracle Linux Enterprise Edition Server Release 5.8 34 GB 32 11.2.0.3 12.1.3

Note: The database and application tiers are on the same machine. Database Configuration SGA: Shared Pool Size: PGA: Log Buffer: job_queue_processes: 5 GB 1 GB 3 GB 30 MB 32

During the upgrade of the Admin Tier, batchsize was 1000 and number of workers used was 32.

Application Tier Size

Oracle E-Business Suite Release 12.2 introduces Online Patching. This requires three application tier file systems: fs1 (Production file system) Used by the current users of the system.

fs2 (Copy of production file system) Used by the patching tools. fs_ne (Non-editioned file system) Stores files containing data that is needed across all file systems (for example, data import and export files, report output files, and log files). Inst (INST_TOP) - Oracle E-Business Suite Release 12 Instance Home, contains all the config files, log files, SSL certificates etc.

All three file systems serve a single database. The file system that is currently being used by the running application is never patched: all patches are applied to the file system that is not currently in use.

Component
ORACLE_HOME APPL_TOP INST_TOP fs1 (APPL_TOP+ INST_TOP) fs2 (APPL_TOP+ INST_TOP) fs_ne

Before Upgrade Size

3.6 GB 28 GB 20 MB Not applicable Not applicable Not applicable

After Upgrade Size

3.6 GB Not applicable Not applicable 30 GB 29 GB 660 KB

Tablespace Size
Tablespace size was calculated as Total TableSpace Size Free Space.

Tablespace Name
SYSTEM CTXSYS OWAPUB SYNCSERVER DCM CWMLITE SYSAUX HTMLDB B2B_IDX MTR DSGATEWAY_TAB DISCO_PTM5_META B2B_LOB OLTS_DEFAULT OPMOR PORTAL PORTAL_DOC B2B_DT OLTS_SVRMGSTORE RE IAS_META WCRSYS_TS OCATS UDDISYS_TS PORTAL_LOG OLTS_ATTRSTORE ODM_DATA

Before Upgrade Size (MB)

16489 475 0 25 178 0 1594 311 14 2 5 2 11 2 5 1 0 61 2 7 201 2 2 19 0 0 30

After Upgrade Size Difference (MB) (MB) 16819 330

475 0 25 178 0 1662 311 14 2 5 2 11 2 5 1 0 61 2 7 201 2 2 19 0 0 30 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

% Growth
2% 00% 0.06% 0% 0% 0% 4.26% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%

APPS_TS_DISCO_OLAP APPS_TS_QUEUES APPS_TS_TOOLS APPS_TS_INTERFACE APPS_TS_SEED XDB APPS_TS_TX_IDX GEOR_TBS NOETIX APPS_TS_DISCO TEMP2 APPS_TS_MEDIA DISCO_PTM5_CACHE B2B_RT APPS_TS_SUMMARY APPS_TS_ARCHIVE APPS_TS_TX_DATA APPS_CALCLIP PORTAL_IDX ORABPEL OLTS_CT_STORE APPS_OMO TEMP1 BAM OLTS_BATTRSTORE APPS_TS_NOLOGGING NON_APPS_TS_DATA Total

10 7201 14 17456 34184 151 98809 2021 0 766 9192 2536 97 38 11694 938 94631 4676 0 11 7 638 10740 6 0 94 66 315414

10 7201 14 17303 34317 151 99286 2021 0 766 9192 2536 97 38 10580 937 94462 4676 0 11 7 638 10740 6 0 267 66 315159

0 0 0 -153 133 0 477 0 0 0 0 0 0 0 -1114 1 -169 0 0 0 0 0 0 0 0 173 0 -254

0% 0% 0% -0.89% 0.39% 0% 0.48% 0% 0% 0% 0% 0% 0% 0.00% -0.97% 0.10% -0.18% 0% 0% 0% 0% 0% 0.00% 0.00% 0% 185% 0%

Schema Size
Two new schemas have been created for Release 12.2: 1. APPS_NE: For the Non-Editioned file system 2. GHG: For the Oracle Environmental Accounting and Reporting. Only a few of the schemas are listed below show significant change in the size after the upgrade.

Schema Name
AP APPLSYS APPS APPS_NE AR BSC CSC CSI EGO INV

12.1.3 Size (MB)

4360.63 38724.44 2395.63
Not applicable

12.2 Size (MB)

4379.5 39377.81 1765.38 95.25 90567.63 285.69 183.5 6036.94 2690.44 11658.13

Difference
18.88 653.37 -630.3 95.25 -141.8 -171.8 -49.25 72.5 34.5 175.57

% Difference
0.43 1.69 -26.31
Not applicable

90709.44 457.44 232.75 5964.44 2655.94 11482.56

-0.16 -37.55 -21.16 1.22 1.3 1.53

JTF MSC OKC PJI SYS XDO XLA

30272.75 5120.44 2969.19 936.25 19893.52 1477.13 3059.06

30340.38 5153.94 3037.69 969.38 20819.65 1626.38 3134

67.63 33.5 68.5 33.13 926.13 149.26 74.94

0.22 0.65 2.31 3.54 4.66 10.1 2.45

Release 12.2 Upgrade Best Practices

Recommended initialization (init.ora) parameter settings for the upgrade.

It is recommended to follow AWR pool advisories for optimizing the SGA and PGA size in the Release 12.2 upgrade. An upgrade involves tens of concurrent sessions. Suitable starting values are: 1. log buffer = 30 to 100 Mb 2. shared pool = 1 to 4 GB 3. pga target = 3 to 20 GB 4. buffer cache = 5-50 GB (multi-GB) You should ensure that SGA and PGA are allocated appropriately within the available memory of the system. If set higher it may cause excessive paging, or even swapping. Set job_queue_processes = No. of CPUs adobjcmp.sql (Phases: plb+90 and last+63) This parameter is set at the beginning of the upgrade; it will be most useful in the phases mentioned above. Setting parallel_max_servers to 2 x no. of CPUs will help with large index creation, statistics gathering, and some large upg+ phase jobs. In order to maximize multiblock I/O sizes, you should remove "db_file_multiblock_read_count" (if specified). Ensure you reset the following init.ora parameters after completion of the Release 12.2 upgrade. o recyclebin o parallel_max_servers o job_queue_processes

Isolate post-upgrade concurrent programs to a separate manager queue. The downtime portion of the upgrade involves the automatic submission of 95 distinct concurrent programs. For a list of the programs, refer to the "Required Post-Upgrade Jobs" section of this document. In several cases, these programs will run in multiple threads, so the total number of concurrent requests that form part of the post-upgrade step is much higher. These programs will be picked up and executed by the concurrent manager once the system is up, and as such their execution will be mixed with the execution of ongoing concurrent jobs in the system.

To improve manageability of this situation, you can define a separate concurrent manager queue to process only these programs. This can be achieved by using inclusion and exclusion rules to prevent other manager queues (such as the standard manager) from picking up these requests, and cause this new manager queue to only process these requests. Doing this allows you to control the number of target processes allocated to these post-upgrade concurrent programs, including dynamically with the use of work shifts. For additional details about configuring new manager queues, target processes, inclusion/exclusion rules, and work shifts, refer to Oracle E-Business Suite Setup Guide from the Onine Documentation Library. Before you shut down all services as part of the Perform the Upgrade tasks mentioned in the Upgrade Guide, you need to deactivate all Request Processing managers. This is needed so that when the services are brought up at the end of the upgrade, the existing Request Processing Managers do not start picking up and executing requests. After the upgrade is complete, you need to perform the following steps to create new program type and managers to run the post-upgrade requests. To create a new program type, and include a program in that program type: 1. Log in to Oracle E-Business Suite as SYSADMIN, and navigate to System Administrator > Concurrent : Program > Type Name: R12PUPT Application: System Administration Description: Program Type for Post Upgrade Request 2. Include programs from the above list into this new program type. 3. Navigate to System Administrator >Concurrent: Program >Define 4. Search for Program "Refresh Materialized Views"Type (choose):R12PUPT 5. Do the same for other programs in the list. To exclude a program type from the Standard Manager: 1. Log in to Oracle E-Business Suite as SYSADMIN and navigate to System Administrator > Concurrent : Manager >Def ine 2. Search for Standard Manager. 3. Click on Specialization Rules. 4. Go at the bottom of the list to add a new record. 5. Include/Exclude (choose): Exclude Type(choose): Request Type Application(choose): System Administration Name(choose): R12PUPT 6. Do the same for all other Request Processing managers. To create a new concurrent manager and include a program type: 1. Log in to Oracle E-Business Suite as SYSADMIN and navigate to System Administrator > Concurrent: Manager > Define 2. Check the Enabled box. 3. Enter the following to create the new concurrent manager: Manager: R12_Post_Upgrade Short Name: R12PU Application(choose): Application Object Library Description: New manager queue for R12 post upgrade requests Type(choose): Concurrent Manager Cache Size: 1 Program Library Name(choose): FNDLIBR Specialization Rules: Include/Exclude(choose): Include Type(choose): Request Type

Application(choose): System Administration Name(choose): R12PUPT Work Shifts: Work Shift (choose): Standard Processes: Enter a Value, for example 4 Sleep Second: 30 After completing the above steps, activate all "Request Processing Managers" that you previously deactivated. Note that after all these post-upgrade requests run, you should revert the exclusion/inclusion of managers and programs to the original state, removing the new program type and new manager. This is necessary as some of the programs in the list may need to run in the future, as part of the regular system batch processing requirements rather than part of the upgrade. Determine optimal number of workers for upgrade. While monitoring CPU usage and I/O response times, you should test a range of values between 1 x No. of CPUs to 1.5 x No. of CPUs, . The goal is to maximize the number of workers, while keeping CPU usage below 100% (ignoring occasional spikes) and I/O response times below 10-15 milliseconds (again, ignoring occasional spikes).

Required Post-Upgrade Jobs

Below is a list of of the concurrent programs that form part of the required post-upgrade steps. These concurrent programs are automatically submitted during the regular downtime upgrade, so no manual submission is required. These jobs took 7 hours for the 456 GB database upgrade, using 30 target processes in the concurrent manager.

Application Name

Concurrent Program Name

User Concurrent Program Name

Elapsed Time (min.)

0.10 0.57

Total Requests

Advanced Benefits (BEN) Advanced Product Catalog (EGO) Advanced Supply Chain Planning (MSC) Application Implementatio n (AZ) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND)

BENCWBMU EGOCPAGV

Compensation Workbench Model Upgrade Process User-Defined Attribute Group Views (AGV) Compilation Program Refresh Materialized Views

1 1

MSCREFMV

0.70

AZR12UPGRADE

iSetup R12 Upgrade Selection Sets Column Fix FND role names

0.47

FNDFIXRN

0.10

DIAGPATCHINGCP

Diagnostics patching CP

12.60

FDFVGN

Flexfield View Generator

135.10

985

Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Application Object Library (FND) Applications DBA (AD) Asia/Pacific Localizations (JA) Collections

AFLOBBLD

Rebuild Help Search Index

16.72

FNDIRLPP

iRep Loading Post Processor

20.72

FNDRSSTG

Request Set Stage

0.87

FNDSCMPI

Compile Security

0.40

FNDLOAD

Generic Loader

13.05

FNDIRLOAD

FNDIRLOAD

24.70

FNDRSSUB

Report Set

1.00

FNDWFAASTATCC

Workflow Agent Activity Statistics Concurrent Program Workflow Work Items Statistics Concurrent Program Workflow Directory Services User/Role Validation Workflow Mailer Statistics Concurrent Program Workflow Update Notification Action Comments Workflow Role Hierarchy Propagation Compile Non-Compiled Flexfields

1.98

10

FNDWFWITSTATCC

2.75

10

FNDWFDSURV

0.88

FNDWFMLRSTATCC

0.70

10

FNDWFNTFCU

1.50

FNDWFDSRHP

0.27

FDFCMPN

56.33

FNDWFBES_CONTROL_QUEUE_CLE ANUP ADDRPOBS JAINDFOBS

Workflow Control Queue Cleanup

0.02

Drop obsolete products schema India - Remove India Localization contexts from DFFs IEX: AR Collectors To Resource

51.62 0.07

204 3

IEX_COLLECTORS_TO_RESOURCE

0.88

(IEX) Contracts Core (OKC) Contracts Core (OKC) Human Resources (PER) Human Resources (PER) Human Resources (PER) Human Resources (PER) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Incentive Compensation (CN) Inventory (INV) Labor Distribution (PSP) Labor Distribution

OKCARTCRTCTX OKCCHRCRCTX PAY_GEN_UPG

Collectors Create Clause Text Index Create Contract Header Text Index Generic Upgrade Mechanism

0.18 0.27 2.03

1 1 1

PER_FND_SESSIONS_CLEANUP

Remove obsolete sessions from fnd_sessions Unmerge contacts from TCA (TCA purge only) iRecruitment Create Notification Preferences CN_R1212_CNCMAUPD

0.53

12

HRTCAUNMP

3.62

IRC_CREATE_NOTIFICATION

0.07

CN_R1212_CNCMAUPD

1.35

48

CN_R1212_CNCMHUPD

CN_R1212_CNCMHUPD

0.30

48

CN_R1212_CNSRPTXN

CN_R1212_CNSRPTXN

0.35

48

CN_FORMULA_GEN

Generate Formula Packages

7.55

CN_COLLECT_GEN

Generate Collection Packages

1.90

CNUPGPMTRXWRKER

Upgrade payment transactions Worker Upgrade payment commission lines Worker Upgrade transactions for AutoUpgrade or AutoPatch Upgrade commissions and transactions for AutoUpgrade or AutoPatch - Manager Items Data Scripts Execution PSP: Migrate to OAFramework Effort Reporting PSP: Post Upgrade Bad Data Report

0.05

10

CNUPGPMTCLWRKER

0.05

10

CNUPGPMTRXMGR

0.08

CNUPGPMTCOMMTRXMGR

0.12

INVIDSEP PSPERUPB

4.45 0.03

6 1

PSPUPREP

0.82

(PSP) Learning Management (OTA) Learning Management (OTA) Lease and Finance Management (OKL) Marketing (AMS) Marketing (AMS) Oracle iProcurement (ICX) Payables (SQLAP) Payables (SQLAP) Payments (IBY) Payroll (PAY) Purchasing (PO) Purchasing (PO) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables (AR) Receivables

OTARPUPG

Upgrade Log Report

1.30

OTATZUPG

Update Time Zone

0.23

OKLSVPTH

Service Passthrough Report

0.80

AMSUPGHISTORYRECSWKR AMSUPGHISTORYRECSMGR ICXCUCDER

Upgrade metrics history records process - Worker Upgrade metrics history records process - Manager Catalog Child Data Exceptions Report

1.05 1.33 3.93

64 8 1

APWUPDST APWUPRD IBYUPGCP PYUSLV POXUPMGR POXUPWKR HZ_THIRD_PARTY_UPDATE ARHGNRW HZ_GNR_DYNMK_PKG_GEN ARHDCH ARHSUFXS ARHDQSYN ARHMAILS ARHDQCMAL ARHDNBCH ARHDQM ARHDQMPP

Update Expense Status Code in Expense Report Headers Upgrade Report Distribution iPayment FP.G Upgrade Program Payroll Worker Process Purchasing Post-Upgrade Process Purchasing Post-Upgrade Sub Process Third Party Data Integration Update Geography Name Referencing Worker GNR Dynamic packages Generation DQM Sync Import Child Program Data Fix for Party Site Use DQM Synchronization Program Mailstop Migration Program DQM Compile All Rules Create DNB Hierarchy DQM Staging Program DQM Parallel Sync Index Parent

0.07 0.07 0.00 1.47 0.05 0.00 0.47 1.75 0.75 0.23 0.07 0.32 0.03 17.60 0.07 0.42 0.10

1 8 2 4 1 3 2 20 6 4 1 1 1 3 1 3 1

(AR) Receivables (AR) Regional Localizations (JG) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service Contracts (OKS) Service (CS) Service (CS) Service (CS)

ARHDQMPC JGZZVATTRXUPG

Program DQM Parallel Sync Index Child Program EMEA VAT: Transaction Upgrade

0.02 0.95

4 6

OKSUHAFMGR

Service Contracts Update History Annualized Factor Manager Service Contracts eBTax Migration

0.28

OKS_TAX_MIGRATION

0.05

OKSAQUMG

Service Contracts ASO Queue Migration Service Contracts Update History Annualized Factor Worker Service Contracts Update Base Annualized Factor Manager Service Contracts Subrequest for Reprocessing Rule Data Service Contracts Update Base Annualized Factor Worker Launch Process Workflow for existing Service Contracts Service Contracts Update History Tax Columns Manager Service Contracts Update History Tax Columns Worker Service Contracts Reprocess/Synchronize Rule Data Service Contracts Update Base Tax Columns Manager Service Contracts Update Base Tax Columns Worker Service: Upgrade for Audit Creation Record - Manager Service: General Upgrade for Audit Records - Worker Service: Upgrade for Audit Creation Record - Worker

0.58

OKSUHAFWKR

0.02

OKSUBAFMGR

0.20

OKS_REPROCESS_SUBREQUEST

0.02

OKSUBAFWKR

0.02

OKSLAUNCHKWF

0.10

OKSTAXHMGR

0.23

OKSTAXHWKR

0.00

REPROCESS_SYNC_RULE_DATA

0.23

OKSTAXBMGR

0.20

OKSTAXBWKR

0.00

CSSRAUDCMAN CSSRAWGEN CSSRAUDCWKR

0.03 0.00 0.02

1 3 3

Service (CS) Service (CS) Service (CS) Service (CS) Service (CS) Service (CS) Service (CS) Trade Management (OZF) Trade Management (OZF) Warehouse Management (WMS) XML Publisher (XDO) Total

CS_KB_SYNC_SOLUTIONS_INDEX CS_KB_SYNC_STATEMENTS_INDEX CSSRAMGEN CSCHGLINENOMGR CSCHGLINENOWKR CS_SR_SYNC_TEXT_INDEX CS_KB_SYNC_CATEGORIES_INDEX OZFEARNMV

Knowledge Management Solution Index Synchronization Knowledge Management Statement Index Synchronization Service: General Upgrade for Audit Records - Manager Charges: Upgrade for Line Number Manager Charges: Upgrade for Line Number Worker Service Request: Synchronize Text Index Program Knowledge Management Category Index Synchronization Refresh Materialized View

0.73 0.12 0.03 0.03 0.02 1.08 0.05 0.95

2 1 1 1 3 2 1 3

OZFMIGUTLREC

Migrate Accruals for Multi Currency Changes Generate All Rules

0.55

WMSGRULE

1.05

XDOTMGEN

XML Publisher Template ReGenerator

23.53 431.05

1 1692

Sample Upgrade Timings

Before applying the main 12.2 upgrade driver (u10124646.drv), AD patch 10117518 must be applied on both file systems (fs1 and fs2). As a result of applying this patch, two new schemas (APPS_NE and GHG) will be created for use in Release 12.2. The overall time taken to apply the patch was about two hours. 1. Time taken for creating the users and granting privileges to them was 20 minutes. 2. Compiling JSPs (using perl) took 1 hour 30 minutes. 3. The following database activity timing information is for the Release 12.2 AD patch (10117518), which took 5.5 minutes.

Phase
first con seq tab tbm pls plb dat dfr en

Time (sec.)
24 55 2 22 18 5 19 11 1 0

last Invoker Total

89 90 336 (5.5 min.)

The following timing information is for the Release 12.2 main upgrade driver (u10124646.drv) only Phase-wise breakdown of time for upgrade:

Phase
first con seq tab tbm os pls vw plb pda daa upa pdt dat upg dfr en last Total

Time (in min.)

0.31 2.75 1.15 4.83 16.98 0.03 7.11 3.14 28.04 2.75 21.96 0.08 2.57 34.4 102.2 1.06 8.00 98.04 335.4 min. (5.59 hours)

The upgrade is a very I/O intensive process, moving, updating, and indexing large amounts of data. As such, I/O subsystem performance and bandwidth are critical to minimizing the amount of downtime. During many Release 11i to Release 12 customer upgrades, poor I/O sub-system performance was found to be one of the major sources of upgrade performance service requests. Therefore, during the upgrade testing process you should monitor I/O performance and correct any issues found. As a reference, shown below are the IOPS rates and bandwidth measurements that were obtained while testing the upgrade with the 456 GB database. The numbers were calculated based on data from AWR and OS monitoring utilities. IOPS: Maximum IOPS value is 478.9. During upgrade, IOPS range is from 142 to 478.9. Max value is reported during large indexes creation phase. High IOPS value (140 to 478) is seen during UPG phases.

I/O Read bandwidth: Maximum I/O Read bandwidth (in MB): 10.23 During upgrade I/O Reads ( in MB) per second range is from 1.44 to 10.23 Max value is reported during dat and upg phase. I/O Write bandwidth: Maximum I/O Write bandwidth (in MB): 3.09

During upgrade I/O Writes per second range is from 0.9 to 3.09 MB. Max value is reported during dat phase.

I/O Response Times ( in milliseconds): Below 20ms for the vast majority of AWR snapshot periods. Higher values were present with spikes above 50ms during the Large indexes create phase and some upgrade phases. Timing information for Release 12.2 Edition-Based Redefinition (EBR) enablement (Patch 13543062) Timing reported here is only for application of the EBR enablement patch. Apart from patch application, many other steps are performed in enabling EBR. The time these take will depend upon the customizations you have. Refer the the following document for the steps needed to enable EBR: Oracle E-Business Suite Upgrade Guide Release 12.0 and 12.1 to 12.2, Part No. E48839 (Section 4, Oracle EBusiness Suite Online Patching) You need to register all your custom schemas, following the procedures in the above document. Phase-wise breakdown of time needed for the upgrade:

Phase
first con seq tab tbm pls last Total

Time (min.)
57.13 0.46 0.8 0.25 35.13 2.65 3.53 100 min. (1.6 hours)

References:
Oracle E-Business Suite Release 12.2 Information Center (My Oracle Support Knowledge Document 1581299.1) Database Initialization Parameters for Oracle E-Business Suite Release 12 (My Oracle Support Knowledge Document 396009.1) Oracle E-Business Suite Recommended Performance Patches (My Oracle Support Knowledge Document 244040.1)

Change Log Date

04-Nov-2013

Description
Initial publication.