1/25/2011 DataStage tip for beginners: developer …

Featured Webcast: Windows 7 Deployment - "An Insider's Guide" Register Today

My Home Topics People Companies Jobs White Paper Library Search Toolbox.com

Home Blogs Groups Wiki Communities White Papers Q&A and Docs Directory Events Subscriptions

Toolbox for IT Topics Business Intelligence Blogs

Receive the latest blog posts in your
Tooling Around in the IBM InfoSphere inbox
by Vincent McBurney IBM Information Champion
Your E-mail address FOLLOW THIS BLOG
The blog dedicated to a tool based approach to data integration with news and tips on
IBM InfoSphere, Informatica, Oracle, ... more
Blog Main | Archive

Previous | Next

DataStage tip for beginners: developer short cuts 0

Vincent McBurney | Feb 21, 2006 | Comments (21)

People who have been using a tool for a long time learn some shortcuts. I've done a brain
dump of any DataStage short cuts I can remember.

Feel free to add comments with your own shortcuts.

Import Export
* When you do an export cut and paste the export file name. When you go to your
project and run an import paste the file name instead of having to browse for it. While
export and import independently remember the last file name used they do not share that
name between each other. Share Your Perspective
* When you switch export type between category and individual job it is quick to switch Share your professional knowledge and
the type, close the export form and open it again. That way the job name or category you experience with peers. Start a blog on Toolbox for
have highlighted will be automatically picked. IT today!
There is an Export option to export by individual job name or export by category name.
BEGIN NOW
This is on the second tab in the export form. Often when you go to export something it is
on the wrong option, eg. you want a job but it is showing the category. You switch from
category export to individual job export but back on tab 1 your job is still not highlighted. Work With Me
* When you do an export there is a "View" button, click this to open the export file and
run any type of search and replace on job parameter values when moving between dev Want to work for the IBM Information Management
Australian partner of the year 2010? I am building a team of
and test.
great Information Server and DataStage consultants in
* If you want to export several jobs that are not in the same category use the append Melbourne, Sydney, Canberra, Adelaide and Perth. Send an
option. Highlight and export the first job. Close the export window, find and highlight the email to vmcburney at focuss.com.au.
second job, in the export form click the "Append" option to add to the file. Continue until
all jobs have been selected and exported.
* On the Options tab is a check box to include in the export "Referenced shared
containers" to also export those.
Links
Things you could easily miss
You could use DataStage for months and not see some of these time savers, done this Steal This IM Methodology
myself. Informatica Data Quality Blog
* There is an automap button in a lot of stages, especially the transformers, maps fields DataFlux Community of Experts
with the same names. Group 1 Data Quality Blog
* When you add a shared container into your job you need to map the columns of the IBM LeverageInformation Community Blog
container to your job link. What you might miss is the extra option you get on the Data Governance Blog
Columns tab "Load" button. In addition to the normal column load you get "Load from dq:view - Steve Tuck on Data Quality
Container" which is a quick way to load the container metadata into your job.
* Don't create a job from an empty canvas. Always copy and use an existing job. Don't
Categories
create shared containers from a blank canvas, always build and test a full job and then
turn part of it into a container. Blogging GO
* If you want to copy and paste settings between jobs, for example database login values
or transformer functions, open each job in a seperate Designer session. Most property
windows in DataStage are modal and you can only have one property window open per
Designer session, by opening two Designers you can have two property windows open at
the same time and copy or compare them more easily.
* You can load metadata into a stage by using the "Load" button on the column tab or by
dragging and dropping a table definition from the Designer repository window onto a link
in your job. For sequential file stages the drag and drop is faster as it loads both the
column names and the format values in one go. If you used the load button you would
needNewtotoload the column Ask
Toolbox? names and then theJoin
A Question format details seperately.

…toolbox.com/…/datastage-tip-for-begi… 1/6
1/25/2011 DataStage tip for beginners: developer …
* Can't get a Modify function or Transformer function working correctly? Trial and error is
often the only way to work out the syntax of a function. If you do this in a large and
complex job it can be time consuming to debug due to job startup times. Consider have
a couple test jobs in your dev project with a row generator, a modify or transformer stage
and a peek stage. Have a column of each type in this test job. Use this throughout your
project as a quick way to test a function or conversion.
* You can put job parameters into stage properties text boxes, eg. #filedir#/#filename#
but you may not know that you can put macros into property text boxes.
#filedir#/#filename#_#DSJobName#. In this example the first two are job parameters and
the third value is not, it's a DataStage macro.

Sequence Jobs
My most annoying Sequence job "feature" is the constant need to enter job parameters
over and over and over again. If you have 10-20 parameters per job (as I normally do) it
becomes very repetitive and is open to manual coding errors.
Under version 7.1 and earlier you could copy and paste a job activity stage, change the
job name and retain most of the parameter settings. Under 7.5.x when you change the
job name all the parameter settings get wiped.

You need to set the parameters for every flippin job activity stage, even though they are
likely to have the same or similar parameter lists and settings. A faster way is to do the
job renaming in an export file.

* In an empty sequence job add your first job activity stage and set all parameter values
or copy one in from an existing job.
* Copy and paste as many copies of this job activity as you need for your sequence.
* Close the sequence job and export the job and click the View button.
* Open the sequence job, you need it to retrieve the stage names.
* Copy the name of the last job activity stage name and search for it in the export file.
When the cursor is on that part of the export file search and replace the old job name
with the new job name. Make sure you only replace to the bottom of the file, most text
editors should have this option. This will rename the job of the last activity stage.
* Repeat this for the second last job activity, then the third last etc until you have
replaced all job names back to the second job activity stage.
* Import the job into your project.

This should give you the same set of job activity stages but with each one pointing at a
different job and with the full set of job parameters set.

Debugging
* In parallel jobs I use the copy stage a lot to debug errors that do not indicate which
stage caused the error. I create a copy of the job and start removing output stages,
replacing them with a copy stage. I progressively remove stages until I locate the one
with the error.

Disclaimer: The opinions expressed herein are my own personal opinions and do not
represent my employer's view in any way.

Related White Papers

Cloud Success in Seven Steps with External Monitoring
Then what happens?
Six Data Protection Tips for SMBs

21 Comments
Feb 21, 2006
Bunch of Thanks for Valuable info .... As All these help will save Time. Thanks
Again. Himanshu

Mar 6, 2006
I have bookmarked this page. I am still learning DS extended. I used to use
Informatica, so understanding these shortcuts will help my learning curve and
development time. Thanks for taking the time to put this together, it is
appreciated. Charlie

Mar 24, 2006

Hi Vincent, I've been reading through the Datastage documentation and finally
figured out why your blog has the name it does. ;-) I couldn't figure out what
websphere had to do with datastage before that. If I could make a request: do
some more datastage newbie posts. Getting into it, short cuts (like this one),
gotchas, etc. Thanks, LewisC http://blogs.ittoolbox.com/oracle/guide/

Vincent McBurney | Mar 24, 2006

…toolbox.com/…/datastage-tip-for-begi… 2/6
1/25/2011 DataStage tip for beginners: developer …
I think I need to explain my blog title in my blog description! = Now that I
am using qumana to create blog entries it is a LOT= easier to incorporate
images, tables and formatting, so I plan= on doing some DataStage tip
blogs that have been banging around= in my head=2E

Jan 18, 2007

very helpful for beginners.gathered lot of valuable informations.... Thanks,
Rajeswari.

May 28, 2007

For debugging, I have some tips i have discovered as well. For server jobs, you
can run a trace (on the run dialog box, there is a trace tab and you can trace in
different levels) This usually spits out in-detail error messages in director log,
which definitely help in the debug process. Also, if its a parallel job, i use a new
job parameter $APT_DUMP_SCORE and it gives me step by step execution
process of the parallel job and 95% of time i can see the error causing the job
abort

May 29, 2007

Hi

I need DATASTAGE PDF file .Right now iam learning the datastage please
anyone having any pdf files please forward my mail id..

Vasantha.Venkat

Jun 29, 2007

i want to know about routines and how to use it in a job

mahadev vasireddy | Aug 2, 2007

If you have director open and want to open designer and then open a job,
you can first select the job in the director and then open the designer from
the "Tool" option in director. Designer opens with the job selected in
director.

mahadev vasireddy | Aug 2, 2007

Another shortcut. If you have director open and want to open designer and
then open a job, you can first select the job in the director and then open
the designer from the "Tool" option in director. Designer opens with the job
selected in director.

Jan 15, 2008

Do you guys know how to rename tables in datastage for db2?

vanayo | Jan 16, 2008

Anyone knows what should be entered as the first parameter of the routine
UtilityGetRunJobInfo? The description of the routine is the following:

The routine takes the output from UtilityRunJob, and extracts information
based on that output.

Parameters are:

Arg1 - Output from UtilityRunJob

Arg2 - Action required (see list below)
Arg3 - Link name (for link-related actions), otherwise ''

The problem is that I have no idea what should Output from UtilityRunJob
refer to. I have a job sequence in which I´ve put 2 Rooutine activity,
connected one with another, the first to execute the job, e.g. to execute
UtilityRunJob, and the second to execute UtilityGetRunJobInfo, but the
director gives me the warning:
Unable to locate StageName.LinkName in JobOutput:

Could anyone help?

Btw, I´ve used "LinkCount" as Arg2 and "StageName.LinkName" as Arg3

Thanx

Jan 20, 2008

Hi
very happy to join the community

Feb 1, 2008
hi all,

…toolbox.com/…/datastage-tip-for-begi… 3/6
1/25/2011 DataStage tip for beginners: developer …
iam taking datastage training.can somebody help me out with pdf files for notes
& refernces.
thanks.
vasanth

Mar 4, 2008
Vasanth,

i have just for the version 6.0 the student guide.

Jul 8, 2008
A capital tip! Been unaware of it for months!
=> the contextual menu over the column names/derivation in the properties of a
tranformer.
Specially "Derivation substitution" and "Find/Replace". E.g. can save a lots of
typing when you realize you have to add a Trim on those 130 input fields :) .. or
e.g. when copy/pasting stages and you need to rename links in the derivation.

Otherwise, I noticed some people were unaware of the "Debug>View job log"
command from the Designer, much quicker than launching Director, logging in,
selecting the job, etc...
Also, in the Manager, the "Usuage analysis" item from the contextual menu
doesn't seem to be widely known.

Oct 6, 2009
I LOVE JOB TEMPLATES : Fill it with all parameter and parameter sets that will
be used, Develop the job and delete any parameters that will definitely not be
used in the job..

Create any job using a copy or save as of the template job .. and you will find
yourself saving time to sneak out of the office for a quick lunch at a fancy
restaurant ..

HOP TRIEUSUNG | Oct 7, 2009

Power Management in Windows 7 Overview
Microsoft Windows Family of Operating Systems
Microsoft Corporation
Published: May 2009
Abstract
This white paper provides an overview of the power management features in
Windows 7. Discover how Windows 7 reduces power consumption and
makes power management easier.

This have many useful templates for practisting

USER_2043147 | Jun 4, 2010

hi, i am training datastage ,i need help on pdf files links,
documentations,tutorials............help plz.......

venkatsandu123 rao | Jul 13, 2010

Hi, this is very good to reduce the development time and reduce the time
consumin.....and i need some enhancements from datastage please send
my email

Santhosh Sasanapuri | Dec 8, 2010

Hi, I am taking DS training.
Can you please send me some important & real time scenarios or kind of
projects you people are working on.
And what are the other supporting tools which would help them in realtime.
So that it will be helpfull in preparing for interviews to me.

Leave a Comment

…toolbox.com/…/datastage-tip-for-begi… 4/6
1/25/2011 DataStage tip for beginners: developer …

Connect to this blog to be notified of new entries.

Name E-mail
PREVIEW SUBMIT

You are not logged in.

Sign In to post unmoderated comments.
Join the community to create your free profile today.

Archive Category: DataStage

Keyword Tags: DataStage tips shortcuts

Disclaimer: Blog contents express the view points of their independent authors and are not
review ed for correctness or accuracy by Toolbox for IT. Any opinions, comments, solutions or
other commentary expressed by blog authors are not endorsed or recommended by Toolbox for IT
or any vendor. If you feel a blog entry is inappropriate, click here to notify Toolbox for IT.

Browse all IT Blogs

Suggested Links
Spiceworks Free IT Management Software
Download Spiceworks 100% Free Network Management & Helpdesk Software for IT
Pros
www.spiceworks.com

Unified Storage Leader

Learn How to Store 10X More Data. Download White Paper from NetApp.
www.netapp.com

Systems Management Software

Software Deployment, Patch Mgmt, Asset Mgmt, Remote Control & more. Starts@
$545
www.DesktopCentral.com

Customer Support Software - Free Trial!

Exceed Your Customers Expectations By Providing Exceptional Support. Sign Up
Now
www.TeamSupport.com
Buy a Link Now

Toolbox for IT Topics on Toolbox for IT Toolbox.com

My Home Data Center Enterprise Architecture & EAI Networking & Infrastructure About
Topics Data Center Enterprise Architecture & EAI Hardware News
People Mobile & Wireless Privacy
Companies Development Information Management Networking Terms of Use
Jobs C Languages Business Intelligence Telephony Work at Toolbox.com
White Paper Library Java Database Advertise
Visual Basic Data Warehouse Operating Systems Contact us
Collaboration Tools Web Design & Development Knowledge Management Linux Provide Feedback
Discussion Groups Oracle UNIX
Blogs Windows Help Topics
Wiki Enterprise Applications Technical Support
CRM IT Management & Strategy
Follow Toolbox.com ERP Emerging Technology & Trends Security Other Communities
Toolbox for IT on Infor IT Management & Strategy Security
Twitter PeopleSoft Project & Portfolio Management Toolbox for HR
Toolbox.com on Twitter SAP Storage Toolbox for Finance
Toolbox.com on SCM Cloud Computing Storage
Facebook Siebel Cloud Computing

Copyright 1998-2011 Toolbox.com. All rights reserved. All product names are trademarks of their respective companies. Toolbox.com is not
affiliated with or endorsed by any company listed at this site. Toolbox.com is a subsidiary of the Corporate Executive Board.

…toolbox.com/…/datastage-tip-for-begi… 5/6
1/25/2011 DataStage tip for beginners: developer …

…toolbox.com/…/datastage-tip-for-begi… 6/6