HCM

Data Loader (HDL)

Advanced Topics

Prasanna Borse
Oracle HCM Cloud Center of Excellence
June, 2017

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 2

HDL Architecture

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 3

Recap: Logical Objects HCM Data Loader loads Logical Objects,
not distinct file lines
Worker123.zip

Worker.dat

Physical Record
Top Level Business Worker: 2724
Object
34234|2011/12/03|4712/12/31|2011/12/03|UK|1974/03/03|2724

Logical Object Person Name: Richards Physical Record

34234|2011/12/03|4712/12/31|John|Richards|Peter|MRM
Person E-Mail: Work
34234|2011/12/03|W|john.richards@vision.com
Person E-Mail: Home
34234|2011/12/03|H|john.richards@me.com
Worker: 3424 Physical Record

Logical Object 73427|2010/02/13|4712/12/31|2010/02/13|UK|1969/08/07|3424

Person Name: Thompson

73427|2010/02/13|2010/08/04|Jane|Wood||MISS
Logical Record
73427|2010/08/05|4712/12/31|Jane|Thompson||MRS
Person E-Mail: Work Physical Record
73427|2010/02/13|W|jane.thompson@vision.com

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Recap: Logical Objects
Grouping Related File Lines
Worker|2012/02/03||2342|A
PersonName|2012/02/03|2014/08/24|2342|GLOBAL|Louise|Morrison|MISS
PersonName|2012/02/03|2014/08/24|2342|GLOBAL|Louise|Morrison|MISS
PersonName|2014/08/25|4712/12/31|2342|GLOBAL|Louise|Jehan|MRS
PersonName|2014/08/25|4712/12/31|2342|GLOBAL|Louise|Jehan|MRS
PersonEmail|2014/08/25||2342|louise.jehan@vision.com|Y|WORK
PersonEmail|2014/08/25||2342|louise.jehan@vision.com|Y|WORK
PersonLegislativeData|2012/02/03|2014/08/24|2342|US|F|S
PersonLegislativeData|2012/02/03|2014/08/24|2342|US|F|S
PersonLegislativeData|2014/08/25|4712/12/31|2342|US|F|M
PersonLegislativeData|2014/08/25|4712/12/31|2342|US|F|M
WorkRelationship|2012/02/03|2342|Vision|E|2012/02/03|Y|HIRE
WorkRelationship|2012/02/03|2342|Vision|E|2012/02/03|Y|HIRE
WorkTerms|2012/02/03|2014/10/07|2830|2342|Vision|E|2012/02/03|ET|SNBUYER|B1
WorkTerms|2012/02/03|2014/10/07|2830|2342|Vision|E|2012/02/03|ET|SNBUYER|B1
WorkTerms|2014/10/18|4712/12/31|2830|2342|Vision|E|2012/02/03|ET|SNBUYER|B2
WorkTerms|2014/10/18|4712/12/31|2830|2342|Vision|E|2012/02/03|ET|SNBUYER|B2
Assignment|2012/02/03|2014/10/07|2830|ET|SNBUYER|B1|2830-1
Assignment|2012/02/03|2014/10/07|2830|ET|SNBUYER|B1|2830-1
Assignment|2014/10/18|4712/12/31|2830|ET|SNBUYER|B2|2830-1
Assignment|2014/10/18|4712/12/31|2830|ET|SNBUYER|B2|2830-1
AssignmentExtraInfo|2012/02/03|4712/12/31|US|2830-1|Y|C1
AssignmentExtraInfo|2012/02/03|4712/12/31|US|2830-1|Y|C1
Contract|2012/02/03|4712/12/31|2830|24|M
Contract|2012/02/03|4712/12/31|2830|24|M

Logical records are created by relating all records for a component with the same key value
Logical objects are formed by resolving parent references

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 5

HDL Architecture

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 6

What does HDL do with my data
• File at rest in UCM
• File streamed to HDL
• Transfer process
• Import process
• Load process

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 7

Data Loader
Process Flow

Get Fusion Objects File Templates

Transform & Generate files in Fusion Objects Shape
Generate Zip file
Upload to Fusion UCM

Fusion

Source System

Load data to Interface tables

Validate & Load Data

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Data Loader
Architecture

Import Load

File Staging Tables Logical Staging Tables

Import to Logical Staging Tables Invoke Product Services

Transfer data to File Staging Tables Interface to LOI

Key Resolutions Services with Validations

Key Validations
Unzip from UCM to Staging Tables Transform to Logical Object Format
Validate the file structure and formats

Import Errors and Load Errors

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Data Loader
Architecture – Import (Transfer and Import)
Import

File Staging Tables Logical Staging Tables

Transfer data to File Staging Tables Import to Logical Staging Tables

Transfer data to File Staging Tables Import to Logical Staging Tables

•Validate the file structure and formats • Resolves Parent-Child relationships

• Key Validations • Groups into Logical and Physical Rows
• Perform Foreign key Resolutions
• Build Logical Object Hierarchy

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Transfer
FILE_CLOBS

DATA_SETS
FILE_BLOBS

DATA_SETS_BUS_OBJS FILE_LINES

SOURCE_KEYS

BUSINESS_OBJECTS

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11

Transfer

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 12
Transfer

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 13
Transfer
FILE_UK_ROWS FILE_ROWS

FILE_KEYS_INFO FILE_HEADERS FILE_LINES

VO_ATTR_MAPS VO_MAPS

BUSINESS_OBJECTS

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 14

Import
DATA_SETS_BUS_OBJS

Logical Objects – File Line Grouping

VO_MAPPING LOGICAL_LINES

PHYSICAL_LINES

VO_ATTR_MAPPING

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 15

Import Complete

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 16

Data Loader Load
Architecture - Load

Payroll

Interface to LOI Invoke

Talent Benefits
Product Services
CoreSetup Compensation

Scaled out servers

Load Data

threads • Perform data load using Internal services

Batch Size

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Load Complete

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 18

Error Reporting DATA_SETS

DATA_SETS_BUS_OBJS

FILE_LINES
MESSAGE_LINES
FILE_HEADERS

FILE_ROWS

LOGICAL_LINES

PHYSICAL_LINES

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 19

Error Messages

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20

Number of Threads
• Quick recap
• Import : PL\SQL program and hence dependent on database
configuration
• Load: Java programs and hence dependent on scaled out servers and
JRE
• Irrespective of the program ‘type’ , ESS will control maximum number
of concurrent programs one can run in the application.

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 21
Number of Threads
CUSTABC-TEST COE-TEST • Thread count for Import- Since IMPORT is sql
CoreSetupServer 10 with 8GB Heap 2 with 2GB Heap
job, the parameter will be dependent on
Worker Load Threads 80 16 job_queue_processes and not on #of
servers*8 rule.
PayrollServer 6 with 8GB Heap 1with 1GB Heap

Elements Load Threads 48 8 • Thread count for Load – as shown in the table.
CoreProcessesServer 2 with 8 GB Heap 2 with 2 GB Heap Since load is coded in java, the parameter is
DoR Load Threads 16 16 depended on #of Servers*8 rule
CompensationServer 6 with 8GB Heap 1 with 2GB Heap
• Irrespective of the type, # of ess_servers will
Salary Load Threads 48 8 control max concurrent programs per POD.
WorkforceMgmt 1 with 8GB Heap 1 with 2GB Heap
Default calculation goes like 1ess_server to
Absences Load Threads 8 8 handle 25 concurrent programs.
TalentManagementServer 2 with 8GB Heap 2 with 2GB Heap

UCM 1 with 8GB Heap 1 with 2GB Heap

• HDL batch size:- If interested review the blog.
bi_server 1 with 7 GB Heap 1 with 2GB Heap • HDL Thread count – the blog
HCM Domain ess_server 4 with 8GB Heap 1 with 2GB Heap

Concurrent Programs# 100 25

job_queue_processes# 100 10

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 22
HRC_DL_BUSINESS_OBJECTS
Sample Data

BUSINESS_OBJECT_ID BUS_OBJ_FILE_DISCRIMINATOR BUS_OBJ_TOP_DISCRIMINATOR PARENT_BUSINESS_OBJECT_ID PROCESSING_ORDER

300000005908638 ElementEntry ElementEntry 105000
300000005908639 ElementEntryValue ElementEntry 300000005908638
300000005922813 CountrySecurityProfile CountrySecurityProfile 4400
300000005920811 ContentItem ContentItem 10700
300000005916253 GradeLadderTranslation GradeLadderTranslation 2710
300000005916254 GradeRate GradeRate 2650
300000005916255 GradeRateTranslation GradeRateTranslation 2660
300000005916256 GradeStepTranslation GradeStepTranslation 2620

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 24

HRC_DL_VO_ATTR_MAPS
Sample Data

Person
VO_ATTRIBUTE_NAME STAGE_COLUMN_NAME
PersonId PVAL001
EffectiveStartDate PDATE001
EffectiveEndDate PDATE002
PersonNumber UK001
BloodType PVAL002
CorrespondenceLanguage PVAL003
ApplicantNumber PVAL007
WaiveDataProtectFlag PVAL008

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 25