Oracle Data Integrators for Beginners

Presented by: Dip Jadawala

Company: BizTech
Session ID: 9950
Please silence your
cell phones
Overview
• Introductions
• Architecture of ODI
• Topology Manager
• Designer
• Operator
• Security Manager
• Q&A
About Dip Jadawala
• Senior Technical Consultant with BizTech
• 6 Years of Data warehousing , ETL , Reporting experience
• Experience with different ETL tools like ODI, Informatica,
Pentaho and OWB.
• Ad hoc Reporting using OBIEE
• experience in all phases of the system development and
software development lifecycle.
About BizTech
• Leading Mid-Atlantic Oracle Platinum Partner and IT Services
firm focused on Oracle Applications and Technology solutions
• Over 400 successful Oracle implementations over the past 15
years
• Based in King of Prussia, PA with offices in New Jersey, New
York City and Washington DC
• Service Fortune 500 companies, organizations and
government agencies
• 100 + Oracle certified and experienced consultants
Client-Centric Practice Areas
Oracle Technology and Business Intelligence
Oracle Applications
- End to end service offering in BI and EPM
- Full Portfolio of Oracle Applications
- Fully staffed team of Data Architects and
Solutions
DBAs
- Implementation, Upgrade, Migration
- Solid experience in RAC, HA and HS
- Since 1990 MPL6 to R12 Experience
designs
- Over 400 successful implementations to Oracle
Oracle - Understand full Oracle technology stack
date Technology
Applications
BI/EPM

Clients
Managed Oracle
Services Software
ITO Provider

Managed Services and IT Outsource Oracle Software Provider

- Remote or Onsite services - Full Portfolio of Oracle License Resell
- Full portfolio of Oracle Applications and - Help Clients Optimize License models
Technologies - RapidApp BI Software for the agile
- World-Class Data Center with 24x7 Support enterprise
- Instant capacity, operational focused - RapidApp Auditor to manage change and
business model GRC
Prerequisites
1. The system should include the following installed
products:
• Oracle Database 11g
• Oracle Data Integrator 11gR1

2. Creation of ODI Master Repository and Work

Repository has been done.
Architecture of ODI
Architecture of ODI
• Oracle Data Integrator is built around a modular
repository architecture and accessed in client/server or
thin client mode by components that are developed
entirely in Java.
• Repositories.
• ODI Studio.
• Weblogic/Application Server.
Architecture of ODI
Repositories
• The central component of the architecture is the Oracle Data
Integrator Repository. It stores configuration information about the
IT infrastructure, metadata of all applications, projects, scenarios,
and the execution logs.
There are 2 types of repositories in ODI.
1. Master repository.
2. Work repository.
• Work repositories can be 1 or more. There should be only one
master repository talking to all work repositories. According to best
practices we should have only 1 master repository and multiple work
repositories like DEV, TEST and PROD.
Architecture of ODI
Master Repository

Master Repository consists of

1. Security information including users, profiles and rights for
the ODI platform.
2. Topology information including technologies, server
definitions, schemas, contexts, languages and so forth.
3. Versioned and archived.
Architecture of ODI
Work Repository

A Work Repository consists of

1. Models, including schema definition, data stores structures
and metadata, fields and columns definitions, data quality
constraints, cross references, data lineage etc.
2. Projects, including business rules, packages, procedures,
folders, Knowledge Modules, variables etc.
3. Scenario execution, including scenarios, scheduling
information and logs.
Architecture of ODI
ODI Studio

• Administrators, Developers and Operators use the Oracle Data

Integrator Studio to access the repositories.
• This Fusion Client Platform (FCP) based UI is used for
administering the infrastructure (security and topology), reverse-
engineering the metadata, developing projects, scheduling,
operating and monitoring executions.
• ODI Studio provides four Navigators for managing the different
aspects and steps of an ODI integration project
Architecture of ODI
Architecture of ODI
Run-Time Agent
• At design time, developers generate scenarios from the business
rules that they have designed. The code of these scenarios is then
retrieved from the repository by the Run-Time Agent.
• This agent then connects to the data servers and orchestrates the
code execution on these servers. 2 types of agents comes with ODI.

1. The Java EE.

2. Standalone Agent.
Topology Manager
• Topology Navigator is used to manage the data describing the
information system's physical and logical architecture.
• Through Topology Navigator you can manage the topology of your
information system, the technologies and their data types, the
data servers linked to these technologies and the schemas they
contain, the contexts, the languages and the agents, as well as the
repositories.
• The site, machine, and data server descriptions will enable Oracle
Data Integrator to execute the same integration interfaces in
different physical environments.
Topology Manager
Connection to Master Repository and Topology Manager
• User snmp1 has been created on database already.
• jdbc:oracle:thin:@<host>:<port>:<sid>
• You can use any Driver depending on target Database.
Topology Manager
Work repository connection to Master Repository
Topology Manager
Work repository connection to Master Repository
Topology Manager
Work repository connection to Master Repository
Topology Manager
Agent Setup
Topology Manager
Agent Setup
Topology Manager
Start Agent Services
Command Prompt – agent -NAME=localagent for windows machine

Command Prompt – ./agent.sh –NAME=localagent for linux/unix machine

Topology Manager
SCHEMA – Schema where ODI inserts and updates the target tables.
WORK_SCHEMA – Schema where all the ODI Temporary table ( C$_,I$_ etc ) are created
and dropped by ODI.
Topology Manager
• Contexts are bridge between Physical and Logical Architecture.
• The contexts in ODI are basically labels that you can create as needed. ODI comes with
only one context called Global. Typical implementations would actually have three
contexts: Development, QA, Production.
Designer
• Designer Navigator is used to design data integrity checks and to build transformations
such as for example:
• Automatic reverse-engineering of existing applications or databases
• Graphical development and maintenance of transformation and integration
interfaces
• Visualization of data flows in the interfaces
• Automatic documentation generation
• Customization of the generated code
• 3 things in Designer where development is required are
• Interface
• Package
• Scenario
Designer
Main Tabs on Designer
1. Projects
The Projects accordion contains the developments
made with Designer Navigator.

2. Models
The Models accordion contains the descriptions of
the data and applications structures.

3. Others
The Others accordion contains the Global User
Functions, Variables, Markers, and Sequences.

4. Solutions
The Solutions accordion contains the Solutions
that have been created when working with version
management.
 Designer
Knowledge Module Description Usage
Reverse-engineering Retrieves metadata to the Oracle Used in models to perform a
KM Data Integrator work repository customized reverse-engineering
Check KM Checks consistency of data against •Used in models, sub models and
constraints datastores for data integrity audit
•Used in interfaces for flow control
or static control
Loading KM Loads heterogeneous data to a Used in interfaces with
staging area heterogeneous sources
Integration KM Integrates data from the staging area Used in interfaces
to a target
Journalizing KM Creates the Change Data Capture Used in models, sub models and
framework objects in the source datastores to create, start and stop
staging area journals and to register subscribers.
Service KM Generates data manipulation web Used in models and datastores
services
Designer
Connecting to Designer
Designer
Data store creation in Model
Designer
You can see Flat File is also reverse engineered in right most snapshot.
Designer
Interface creation
Designer
Designer
Designer
Designer
Execution of Interface
• Right click on the interface and click Execute
• You can see interface running in Operator.
Designer
2nd example
Designer
CKM Use
Designer
Package Creation
Designer
Scenario Creation
Designer
Parallel Processing of Interfaces
Operator
The Operator Navigator has the following accordions:
Session List
The Session List accordion displays all sessions organized
per date, physical agent, status, keywords, and so forth.
Hierarchical Sessions
The Hierarchical Sessions accordion displays the execution
sessions organized in a hierarchy with their child sessions.
Scheduling
The Scheduling accordion displays the list of physical
agents and schedules.
Scenarios
The Scenarios accordion displays the list of scenarios
available
Solutions
The Solutions accordion contains the Solutions that have
been created when working with version management.
Operator
Operator
• In Designer Navigator Select the TRG_CUSTOMER datastore. Right-click and
select Control > Errors...
• 2 records that did not satisfy the FK_CUST_CITY constraint (for example, the CITY_ID
value does not exist in the table of cities SRC_CITY table).
• 7 records that did not satisfy the business rule acting on customers under 21 (AGE > 21
constraint).
• The invalid records were saved into an error table and were not integrated into the
target table.
• Correct the data and Re run it.
Operator
Scheduling the Scenario
• You can schedule the scenario either in operator or in Designer
Operator
Scheduling the Scenario
Operator
Scheduling the Scenario
Operator
Update Scheduling in Topology
Operator
Update Scheduling in Topology
Operator
View the scheduler in Topology
Security Manager
• Security Navigator is the tool for managing the security
information in Oracle Data Integrator.
• Through Security Navigator you can create users and profiles and
assign user rights for methods (edit, delete, etc) on generic objects
(data server, data types, etc), and fine-tune these rights on the
object instances (Server 1, Server 2, and so forth)
Security Manager
User Creation
Security Manager
Manage Profiles
Security Manager
Manage Profiles
Security Manager
Open Designer and drag /drop folder.
Security Manager
Allow/Deny to appropriate Methods that User can access
Security Manager
Summary
• Architecture of ODI is made of Java Programming. Core part is
Repository.
• Topology Manager has source and Target connection details.
• Designer has interfaces, packages, scenarios, variables, KM,
procedures and so on. All the development components.
• Operator is monitoring tool which keeps the logs of interfaces,
packages and scenarios.
• Security Manager keep the Users and their respective profiles to
access different modules of ODI Studio.
References
•http://docs.oracle.com
•http://www.oracle.com/technetwork/tutorials
•http://odiexperts.com
•http://blogs.oracle.com
Any Questions?

Dip Jadawala
Senior Technical Consultant
DJadawala@Biztech.com
Twitter: @BizTechOracle
Booth #1252 on Main
Street