Академический Документы
Профессиональный Документы
Культура Документы
1
Basics
Education Services
Version PC7B-20041208
Introduction
Course Objectives
By the end of this course you will:
Understand how to use the major PowerCenter
components for development
Be able to build basic ETL mappings and mapplets*
Be able to create, run and monitor workflows
Understand available options for loading target data
Be able to troubleshoot most problems
Note: The course does not cover PowerCenter optional
features or XML support.
* A mapplet is a subset of a mapping
3
About Informatica
Founded in 1993
Leader in enterprise solution products
Headquarters in Redwood City, CA
Public company since April 1999 (INFA)
Worldwide distributorship
Informatica Products
PowerCenter
PowerAnalyzer
SuperGlue*
PowerExchange
PowerCenter
Data access to transactional applications and
Connect products real-time services
Informatica Resources
www.informatica.com provides information (under Services) on:
Professional Services
Education Services
my.informatica.com sign up to access:
Technical Support
Product documentation (under Tools online documentation)
Velocity Methodology (under Services)
Knowledgebase
Webzine
Mapping templates
devnet.informatica.com sign up for Informatica Developers Network
Discussion forums
Web seminars
Technical papers
6
Operational Systems
RDBMS
Mainframe
Extract
8
Data
Other
Warehouse
Aggregate data
Cleanse data
Consolidate data
Apply business rules
De-normalize data
Aggregated data
Historical data
Transform
ETL
Load
Manage repository:
Connections
Folders
Objects
Users and groups
Build ETL
mappings
Monitor
and start
workflows
Administer repositories on a
Repository Server:
Create/upgrade/delete
Configuration
Start/stop
Backup/restore
PowerCenter 7 Architecture
Native
Informatica Server
Native
Targets
Sources
TCP/IP
Heterogeneous
Sources
Repository
Server
TCP/IP
Heterogeneous
Targets
Repository
Agent
Native
Repository Designer Workflow Workflow Rep Server
Manager
Manager Monitor Administrative
Console
Repository
Not Shown: Client ODBC connections from Designer to sources and targets for metadata
10
Repository Servers
Repository Databases
Sources and Targets
Platforms:
Client tools run on Windows
Servers run on AIX, HP-UX, Solaris, Redhat Linux, Windows
Repositories on any major RDBMS
11
12
Demonstration
13
15
Import from:
Relational database
Flat file
XML object
Create manually
Repository
Server
TCP/IP
Repository Agent
Native
DEF
16
Repository
Relational DB Source
ODBC
Table
View
Synonym
DEF
Repository
Server
TCP/IP
Repository Agent
Native
DEF
17
Repository
18
Mapped Drive
NFS Mount
Local Directory
Flat File
DEF
Fixed Width
Delimited
Repository
Server
TCP/IP
Repository Agent
Native
DEF
19
Repository
21
Mapped Drive
NFS Mounting
Local Directory
Repository
Server
DATA
TCP/IP
Repository Agent
Native
DEF
22
Repository
Data Previewer
Preview data in
Relational database sources
Flat file sources
Relational database targets
Flat file targets
23
24
Data
Display
View up
to 500
rows
25
Metadata Extensions
Allows developers and partners to extend the
metadata stored in the Repository
Metadata extensions can be:
User-defined PowerCenter users can define and create
their own metadata
Vendor-defined Third-party application vendor-created
metadata lists
For example, applications such as Ariba or PowerCenter Connect
for Siebel can add information such as contacts, version, etc.
26
Metadata Extensions
Can be reusable or non-reusable
28
30
31
DEF
TCP/IP
Repository Agent
Native
DEF
32
Repository
View
Synonym
Warehouse
Designer
Repository
Server
DEF
DATA
TCP/IP
Repository Agent
Native
DEF
33
Repository
34
35
Heterogeneous Targets
36
Heterogeneous Targets
By the end of this section you will be familiar with:
37
Relational database
Flat file
XML
Targets supported by PowerCenter Connects
Heterogeneous targets are targets within a single
Session Task that have different types or have different
database connections
38
Oracle table
39
Oracle table
Flat file
40
42
43
Midstream XML Parser: reads XML from database table or message queue
Midstream XML Generator: writes XML to database table or message queue
More Source Qualifiers: read from XML, message queues and
applications
45
Transformation Views
A transformation has
three views:
Iconized shows the
transformation in relation
to the rest of the
mapping
Normal shows the flow
of data through the
transformation
Edit shows
transformation ports
(= table columns)
and properties;
allows editing
46
Expression Transformation
Perform calculations using non-aggregate functions
(row level)
Ports
Mixed
Variables allowed
Create expression in an
output or variable port
Usage
Perform majority of
data manipulation
47
Expression Editor
An expression formula is a calculation or conditional statement for a
specific port in a transformation
48
Expression Validation
The Validate or OK button in the Expression Editor will:
Parse the current expression
Remote port searching (resolves references to ports in
other transformations)
Parse default values
Check spelling, correct number of arguments in functions,
other syntactical errors
49
50
Character Functions
Used to manipulate character data
CHRCODE returns the numeric value
(ASCII or Unicode) of the first character of
the string passed to this function
CONCAT is for backward compatibility only.
Use || instead
TO_CHAR (numeric)
TO_DATE
TO_DECIMAL
TO_FLOAT
TO_INTEGER
51
Conversion Functions
Used to convert datatypes
52
Date Functions
Used to round, truncate, or
compare dates; extract one part
of a date; or perform arithmetic
on a date
To pass a string to a date
function, first use the TO_DATE
function to convert it to an
date/time datatype
53
54
Numerical Functions
Used to perform mathematical
operations on numeric data
Scientific Functions
Used to calculate
geometric values
of numeric data
COS
COSH
SIN
SINH
TAN
TANH
ABORT
DECODE
ERROR
IIF
LOOKUP
IS_DATE
IS_NUMBER
IS_SPACES
ISNULL
55
IIF(Condition,True,False)
Test Functions
Used to test if a lookup result is null
Used to validate data
Variable Ports
56
57
Selected
port
Default
value for the
selected
port
58
Validate the
default
value
expression
ISNULL function
is not required
Informatica Datatypes
NATIVE DATATYPES
Native
59
TRANSFORMATION DATATYPES
Transformation
Native
Transformation datatypes allow mix and match of source and target database types
When connecting ports, native and transformation datatypes must be compatible
(or must be explicitly converted)
For further information, see the PowerCenter Client Help > Index
> port-to-port data conversion
60
Mappings
Mappings
By the end of this section you will be familiar with:
62
Mapping Designer
Transformation Toolbar
Mapping List
Iconized Mapping
63
All input/output
Usage
Convert datatypes
For relational sources:
64
65
66
Mapping Validation
67
Connection Validation
Examples of invalid connections in a Mapping:
Connecting ports with incompatible datatypes
Connecting output ports to a Source
Connecting a Source to anything but a Source
68
Mapping Validation
Mappings must:
Be valid for a Session to run
Be end-to-end complete and contain valid expressions
Pass all data flow rules
Mappings are always validated when saved; can be validated
without being saved
69
70
Workflows
Workflows
By the end of this section, you will be familiar with:
The Workflow Manager GUI interface
Creating and configuring Workflows
Workflow properties
Workflow components
Workflow tasks
72
Task
Tool Bar
Navigator
Window
Workflow
Designer
Tools
Workspace
Status
Bar
73
Output
Window
Task Developer
Create Session, Shell Command and Email tasks
Tasks created in the Task Developer are reusable
Worklet Designer
Creates objects that represent a set of tasks
Worklet objects are reusable
74
Workflow Structure
A Workflow is set of instructions for the Informatica
Server to perform data transformation and load
Start
Task
75
Session
Task
Reusable Tasks
Reusable Tasks
Three types of reusable Tasks
Session Set of instructions
to execute a specific
Mapping
Command Specific shell
commands to run during
any Workflow
Email Sends email during
the Workflow
77
Reusable Tasks
Use the Task Developer to
create reusable tasks
These tasks will then appear
in the Navigator and can be
dragged and dropped into
any workflow
78
Reusable
79
Non-reusable
Command Task
Specify one or more Unix shell or DOS commands to
run during the Workflow
Runs in the Informatica Server (UNIX or Windows)
environment
80
Command Task
Specify one (or more) Unix shell or DOS (NT, Win2000)
commands to run at a specific point in the workflow
Becomes a component of a workflow (or worklet)
If created in the Task Developer, the Command task is
reusable
81
82
Add Cmd
Remove Cmd
83
Email Task
Configure to have the Informatica Server to send email
at any point in the Workflow
84
85
Non-Reusable Tasks
Non-Reusable Tasks
Six additional Tasks are available in the
Workflow Designer
Decision
Assignment
Timer
Control
Event Wait
Event Raise
87
Decision Task
Specifies a condition to be evaluated in the Workflow
Use the Decision Task in branches of a Workflow
Use link conditions downstream to control execution flow by
testing the Decision result
88
Assignment Task
Assigns a value to a Workflow Variable
Variables are defined in the Workflow object
General Tab
Expressions Tab
89
Timer Task
Waits for a specified period of time to execute the
next Task
General Tab
Timer Tab
Absolute Time
Datetime Variable
Relative Time
90
Control Task
Stop or ABORT the Workflow
Properties Tab
General
Tab
91
the workflow
92
93
Events Tab
94
General Tab
Properties Tab
95
Session Task
Server instructions to run the logic of ONE specific mapping
e.g. source and target data location specifications, memory
allocation, optional Mapping overrides, scheduling, processing and
load instructions
Becomes a component of a
96
Command
Email
Decision
Assignment
Timer
Control
Event Wait
Event Raise
97
Sample Workflow
Session 1
Command
Task
Start Task
(required)
98
Session 2
Concurrent
Combined
Note: Although only session tasks are shown, can be any tasks
99
Creating a Workflow
Customize
Workflow name
Select a
Server
100
Workflow Properties
Customize Workflow
Properties
Workflow log displays
May be reusable or
non-reusable
Select a Workflow
Schedule (optional)
101
Workflow Scheduler
102
103
Workflow Links
Required to connect Workflow Tasks
Can be used to create branches in a Workflow
All links are executed unless a link condition is used which
makes a link false
Link 1
Link 2
104
Link 3
Conditional Links
Optional link
condition
$taskname.STATUS
is a pre-defined
task variable
105
Workflow Variables 1
Used in decision tasks and conditional links edit task or link:
Pre-defined variables
Task-specific
variables
Built-in system
variables
106
User-defined variables
(see separate slide)
Workflow Variables 2
User-defined variables are set in Workflow properties,
Variables tab can persist across sessions
Can be reset in
an Assignment task
107
Workflow Summary
1.
2.
3.
4.
108
Session Tasks
Session Tasks
After this section, you will be familiar with:
110
111
112
113
Select target
instance
Set connection
Set properties
114
Monitoring Workflows
Monitoring Workflows
By the end of this section you will be familiar with:
The Workflow Monitor GUI interface
Monitoring views
Server monitoring modes
116
Workflow Monitor
The Workflow Monitor is the tool for monitoring
Workflows and Tasks
Choose between two views:
Gantt chart
Task view
117
Task view
118
Monitoring Operations
Perform operations in the Workflow Monitor
Stop, Abort, or Restart a Task, Workflow or Worklet
Resume a suspended Workflow after a failed Task is
corrected
Reschedule or Unschedule a Workflow
119
Status Bar
120
Server
Workflow
Worklet
Start
Time
Completion
Time
Monitoring filters
can be set using
drop down menus.
Minimizes items
displayed in
Task View
121
Filter Toolbar
122
Repository Manager
Repository Managers
Truncate Log option
clears the Workflow
Monitor logs
123
124
125
Debugger
Debugger
By the end of this section you will be familiar with:
127
Debugger Features
Wizard driven tool that runs a test session
Initialize variables
Manually change variable values
Data can be loaded or discarded
Debug environment can be saved for later use
128
Debugger Interface
Edit
Breakpoints
Debugger Mode
indicator
Solid yellow
arrow is current
transformation
indicator
Flashing
yellow
SQL
indicator
Output Window
Debugger Log
129
Transformation
Instance
Data window
Target Instance
window
Set Breakpoints
1. Edit breakpoint
2. Choose global or
specific transformation
3. Choose to break on
data condition or error.
Optionally skip rows.
4. Add breakpoint(s)
5. Add data conditions
Debugger Tips
Server must be running before starting a Debug Session
When the Debugger is started, a spinning icon displays.
Spinning stops when the Debugger Server is ready
The flashing yellow/green arrow points to the current active
Source Qualifier. The solid yellow arrow points to the current
Transformation instance
Next Instance proceeds a single step at a time; one
row moves from transformation to transformation
Step to Instance examines one transformation at a
time, following successive rows through the same
transformation
131
132
Filter Transformation
Filter Transformation
Drops rows conditionally
Ports
All input / output
134
135
Sorter Transformation
Sorter Transformation
Can sort data from relational tables or flat files
137
Sorter Transformation
Sorts data from any source, at any point in a data flow
Sort Keys
Ports
Input/Output
Define one or more
sort keys
Define sort order for
each key
Example of Usage
Sort data before
Aggregator to improve
performance
Sort Order
138
Sorter Properties
139
Aggregator Transformation
Aggregator Transformation
By the end of this section you will be familiar with:
Aggregator properties
Using sorted data
141
Aggregator Transformation
Performs aggregate calculations
Ports
Mixed I/O ports allowed
Variable ports allowed
Group By allowed
Create expressions in
variable and output ports
Usage
Standard aggregations
142
Aggregate Expressions
Aggregate
functions are
supported only in
the Aggregator
Transformation
Conditional Aggregate
expressions are supported: Conditional SUM format: SUM(value, condition)
143
Aggregator Functions
AVG
COUNT
FIRST
LAST
MAX
MEDIAN
MIN
PERCENTILE
STDDEV
SUM
VARIANCE
144
Aggregator Properties
Sorted Input Property
Instructs the
Aggregator to
expect the data
to be sorted
Set Aggregator
cache sizes for
Informatica Server
machine
145
Sorted Data
The Aggregator can handle sorted or unsorted data
Sorted data can be aggregated more efficiently, decreasing total
processing time
146
Group By:
- store
- department
- date
147
Group By:
- store
- department
- date
Active transformation
Can operate on groups of data rows AND/OR
149
DISALLOWED
Active
Passive
T
150
Joiner Transformation
Joiner Transformation
By the end of this section you will be familiar with:
Nested joins
152
153
154
Joiner Transformation
Performs heterogeneous joins on different data
flows
Active Transformation
Ports
All input or input / output
M denotes port comes
from master source
Examples
Join two flat files
Join two tables from
different databases
Join a flat file with a
relational table
155
Joiner Conditions
156
Joiner Properties
Join types:
Normal (inner)
Master outer
Detail outer
Full outer
Set Joiner
Caches
Joiner can accept sorted data (configure the join condition to use the
sort origin ports)
157
Nested Joins
Used to join three or more heterogeneous sources
158
159
160
Lookup Transformation
Lookup Transformation
By the end of this section you will be familiar with:
Lookup principles
Lookup properties
Lookup conditions
Lookup techniques
Caching considerations
Persistent caches
162
Return value(s)
163
Lookup Transformation
Looks up values in a database table or flat file and
provides data to other components in a mapping
Ports
Mixed
L denotes Lookup port
R denotes port used as a
return value (unconnected
Lookup only see later)
Specify the Lookup Condition
Usage
Get related values
Verify if records exists or if
data has changed
164
Lookup Conditions
165
Lookup Properties
Lookup
table name
Lookup condition
Native database
connection object name
Source type:
Database or Flat File
166
Policy on multiple
match:
Use first value
Use last value
Report error
167
Lookup Caching
Caching can significantly impact performance
Cached
Lookup table data is cached locally on the Server
Mapping rows are looked up against the cache
Uncached
Each Mapping row needs one SQL SELECT
Persistent Caches
By default, Lookup caches are not persistent; when the
session completes, the cache is erased
169
Toggle
caching
Cache
directory
170
Reload
persistent
cache
171
172
Target Options
Target Options
By the end of this section you will be familiar with:
Constraint-based loading
174
175
Target Properties
Edit Tasks: Mappings Tab
Session Task
Select target
instance
Target load type
Row loading
operations
Error handling
176
Delete SQL
DELETE from <target> WHERE <primary key> = <pkvalue>
Constraint-based Loading
PK1
FK1 PK2
FK2
179
Active source
Active transformation that generates rows
Cannot match an output row with a distinct input row
Examples: Source Qualifier, Aggregator, Joiner, Sorter
(The Filter is NOT an active source)
Active group
Group of targets in a mapping being fed by the same active
source
180
Example 1
FK1
PK2
FK2
PK1
FK1
PK2
FK2
181
Example 2
With two Active sources, it is not
possible to control whether rows
for Target3 will be loaded before
or after those for Target2
182
Update Strategy
Transformation
Ports
All input / output
Specify the Update
Strategy Expression
IIF or DECODE logic
determines how to
handle the record
Example
Updating Slowly
Changing Dimensions
184
186
187
188
Router Transformation
Router Transformation
Rows sent to multiple filter conditions
Ports
All input/output
Specify filter conditions
for each Group
Usage
Link source data in
one pass to multiple
filter conditions
190
Router Groups
Input group (always one)
User-defined groups
192
Lab 13 Router
193
Sequence Generator
Transformation
Ports
Two predefined output
ports, NEXTVAL and
CURRVAL
No input ports allowed
Usage
Generate sequence
numbers
Shareable across mappings
195
Number of
cached values
196
System variables
Mapping parameters and variables
Parameter files
198
System Variables
SYSDATE
SESSSTARTTIME
$$$SessStartTime
199
200
Set datatype
User-defined
names
Set
aggregation
type
Set optional
initial value
SETMINVARIABLE($$Variable,value)
Sets the specified variable to the lower of of the
current value or the specified value
SETVARIABLE($$Variable,value)
Sets the specified variable to the specified value
SETCOUNTVARIABLE($$Variable)
Increases or decreases the specified variable by the
number of rows leaving the function(+1 for each
inserted row, -1 for each deleted row, no change for
updated or rejected rows)
203
Parameter Files
205
Unconnected Lookups
Unconnected Lookups
By the end of this section you will know:
207
Unconnected Lookup
Physically unconnected from other transformations NO data flow
arrows leading to or from an unconnected Lookup
Lookup data is called from the point in the Mapping that needs it
Lookup function can be set within any transformation that supports
expressions
Function in the Aggregator
calls the unconnected Lookup
208
Row keys
(passed to Lookup)
IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no))
Lookup function
209
Condition
(true for 2 percent of all rows)
Lookup
(called only when condition is true)
Must check a
Return port in the
Ports tab, else
fails at runtime
211
212
UNCONNECTED LOOKUP
213
214
Mapplets
Mapplets
By the end of this section you will be familiar with:
Mapplet Designer
Mapplet advantages
Mapplet types
Mapplet rules
Active and Passive Mapplets
Mapplet Parameters and Variables
216
Mapplet Designer
217
Mapplet
Input and Output
Transformation
Icons
Mapplet Advantages
Useful for repetitive tasks / logic
218
219
220
Unsupported Transformations
You cannot not use the following in a mapplet:
Normalizer Transformation
XML source definitions
Target definitions
Other mapplets
221
External Sources
Mapplet contains a Mapplet Input transformation
Mixed Sources
Mapplet contains one or more of either of a Mapplet
Input transformation AND one or more Source Qualifiers
Receives data from the Mapping it is used in, AND from
the Mapplet
222
Passive Transformation
Connected
Ports
Output ports only
Usage
Only those ports
connected from an
Input transformation
to another
transformation
will display in the
resulting Mapplet
223
Transformation
Transformation
Connecting the
same port to more
than one
transformation is
disallowed
Pass to an
Expression
transformation
first
Mapplet
Mapplet
Usage
226
Warning: An unlinked
Mapplet Output Group
may invalidate the
mapping
228
229
Passive
Active
230
Multiple Passive
Mapplets can populate
the same target
instance
Lab 17 Mapplets
232
Reusable Transformations
Reusable Transformations
By the end of this section you will be familiar with:
Transformation Developer
Reusable transformation rules
Promoting transformations to reusable
Copying reusable transformations
234
Transformation Developer
Reusable
transformations
235
Make a
transformation
reusable from
the outset,
or
test it in a
mapping first
Reusable Transformations
Define once, reuse many times
Reusable Transformations
Can be a copy or a shortcut
Edit Ports only in Transformation Developer
Can edit Properties in the mapping
236
Check the
Make reusable box
(irreversible)
237
238
239
241
Error Types
Transformation error
Data row has only passed partway through the mapping
transformation logic
An error occurs within a transformation
Data reject
Data row is fully transformed according to the mapping
logic
Due to a data issue, it cannot be written to the target
A data reject can be forced by an Update Strategy
242
243
Error Type
Logging ON
Transformation
errors
Data rejects
244
245
246
First column:
0=INSERT 0,D,1313,D,Regulator System,D,Air Regulators,D,250.00,D,150.00,D
1=UPDATE 1,D,1314,D,Second Stage Regulator,D,Air Regulators,D,365.00,D,265.00,D
2=DELETE 2,D,1390,D,First Stage Regulator,D,Air Regulators,D,170.00,D,70.00,D
3=REJECT 3,D,2341,D,Depth/Pressure Gauge,D,Small Instruments,D,105.00,D,5.00,D
Session metadata
Reader, transformation, writer and user-defined errors
For errors on input, logs row data for I and I/O ports
For errors on output, logs row data for I/O and O ports
248
Relational Database
Log Settings
249
250
251
252
253
254
Workflow Configuration
256
257
(Native Databases)
(MQ Series)
(File Transfer Protocol file)
(Custom)
(External Database Loaders)
258
259
260
FTP Connection
Create an FTP connection
Instructions to the Server to ftp flat files
Used in Session Tasks
261
262
263
264
265
266
Session Configuration
Define properties to be reusable across different
sessions
Defined at folder level
Must have one of these tools
open in order to access
267
268
269
270
Attributes
may be
overridden
within the
Session task
271
Worklets
Worklets
An object representing a set or grouping of Tasks
Can contain any Task available in the Workflow
Manager
Worklets expand and execute inside a Workflow
A Workflow which contains a Worklet is called the
parent Workflow
Worklets CAN be nested
Reusable Worklets create in the Worklet Designer
Non-reusable Worklets create in the Workflow
Designer
273
Re-usable Worklet
In the Worklet Designer, select Worklets | Create
Worklets
Node
Tasks in a Worklet
274
Worklet
used in a
Workflow
275
Non-Reusable Worklet
NOTE: Worklet
shows only under
Workflows node
276
1.
2.
3.
Workspace switches to
Worklet Designer
277
278
279
280
Mappings
& Mapplets
Workflows
& Worklets
281
TYPE
HOW DEFINED
WHERE USED
EXAMPLES
Mapping/
Mapplet
Variables
Mapping/mapplet
properties. Reset by
variable functions.
Transformation port
expressions
$$LastUpdateTime
$$MaxValue
Mapping/
Mapplet
Parameters
Mapping/mapplet
properties. Constant
for session.
Transformation port
expressions
$$FixedCosts
$$DiscountRate
System
Variables
Built-in, pre-defined.
Transformation port
expressions,
Workflow decision
tasks and
conditional links.
SYSDATE
SESSSTARTIME
WORKFLOWSTARTTIME
Task
Variables
Built-in, pre-defined.
Workflow decision
tasks and
conditional links
$session1. Status
$session1.ErrorCode
Workflow/
Worklet
Variables
Workflow or worklet
properties. Reset in
Assignment tasks.
Workflow decision
tasks and
conditional links
$$NewStartTime
Session
Parameters
Parameter file.
Constant for session.
Session properties
$DBConnectionORCL
$InputFile1
282
Data Cleansing
Server Grid
Real-Time/Web Services
Partitioning
Team-Based Development
PowerCenter
283
Virtual Classes
Watch for short web-based virtual
classes on most PowerCenter
options and XML support
284
285
Real-time Services
HTTP
JMS
MSMQ
MQSeries
TIBCO
WebMethods
Web Services
286
287