How i can do incremental aggregation in real time?

For incremental Aggregation.. We need to use Aggregations + Look up on Target + Expression to SUM up Count obtained rom !e" Aggregations and Lookup on target. For one record alread# present in Aggregations table.. count is also t$ere.. %t "ill be a&ailable in lookup.. ne" count "ill be a&ailable t$roug$ A''.. Sum t$en up and update t$at record in target.. This is a scenario in which the source has 2 cols 10 A 10 A 20 C 30 D 40 E 20 C And there should be 2 targets one to show the duplicate values and another target for distinct rows. T1 T2 10 A 10 A 20 C 20 C 30 D hich transfor!ation can be used to load data into target" 40 E Ans# (.Sorce ) Source *uali ier ) Target C$eck) Select distinct option +.Source ))Source *uali ier) aggregator ) Target group b#)empno ,.Source ))Source *uali ier) sorter ) Target C$eck) Select distinct option -.Source )Source *uali ier )Expression )Target )Source *uali ier. c$eck sotred b# empno Expression.) port expression in/empno &ar)empno0&ar/duplicate &ar/duplicate0in/emp lag0 %%F1&ar/duplicate0&ar/emp234323!35 6.Source )Source *uali ier )7ank)Expression 8Target

hat is the !ethod of loading $ flat files of having sa!e structure to a single target and which transfor!ations % can use" Ans# union trans ormation2 ot$er"ise "rite all ile pat$s o i&e iles in one ile and use t$is ile in session properties as indirect %n the source& if we also have duplicate records and we have 2 targets& T1' for uni(ue values and T2' onl) for duplicate values. *ow do we pass the uni(ue values to T1 and duplicate values to T2 fro! the source to these 2 different targets in a single !apping" source)))9s*)))9exp))9sorter1"it$ enable select distinct c$eck box5)))9t( )))9aggregator1"it$ enabling group b# and "rite count unction5)))9t+ % u "ant onl# duplicates to t+ u can ollo" t$is se*uence )))9agg1"it$ enable group b# "rite t$is code decode+count+col,&1&1&0,,''' -.ilter+condition is 0,'''-t2. in the concept of !apping para!eters and variables& the variable value will be saved to the repositor) after the co!pletion of the session and the ne/t ti!e when u run the session& the server ta0es the saved variable value in the repositor) and starts assigning the ne/t value of the saved value. for e/a!ple i ran a session and in the end it stored a value of $0 to the repositor).ne/t ti!e when i run the session& it should start with the value of 10. not with the value of $1. u can do onet$ing a ter running t$e mapping22 in "ork lo" manager start))))))))9session. rig$t clickon t$e session u "ill get a menu2 in t$at go or persistant &alues2 t$ere u "ill ind t$e last &alue stored in t$e repositor# regarding to mapping &ariable. t$en remo&e it and put ur desired one2 run t$e session... i $ope ur task "ill be done

W$en "e are $a&ing data in t$e target table and t$ere are no primar# ke# or oreign ke# mentioned in t$e target table t$en $o" "ill #ou update as "ell as insert t$e data: Simple Mr Mad$u Take a d#namic lookup on t$e target table and compare t$e &alues o target and source in an expression and lag t$e &alues so t$at #ou can update or insert. !o" in %n ormatica ;.( t$ere is a ne"l# added unction is t$ere to a&oid to do all t$ese stu . For ex. i u $a&e some &alues like (<( (<+ (<, in target look up on target "ill get t$e &alues and connect to an expression and connect t$e ports comming rom source also and t$en compare t$ose &alues... i t$ere is an# c$ange in an# column t$en lag t$at &alue "it$ 4 or !. and ilter accordingl#.

For sending t$e irst and last ro" into target. $ead )( EM=/TEST.txt 99 testtail )( EM=/TEST.txt 99 test2apping variable#
In the Designer, you can create mapping variables in a mapping or mapplet. After you create a mapping variable, it appears in the Expression Editor. You can then use it in any expression in the mapping or mapplet. You can also use mapping variables in a source qualifier filter, user-defined oin, or extract override, and in the Expression Editor of reusable transformations. !nli"e mapping parameters, mapping variables are values that can change bet#een sessions. $he Integration %ervice saves the latest value of a mapping variable to the repository at the end of each successful session. During the next session run, it evaluates all references to the mapping variable to the saved value. You can override a saved value #ith the parameter file. You can also clear all saved values for the session in the &or"flo# 'anager. You might use a mapping variable to perform an incremental read of the source. (or example, you have a source table containing timestamped transactions and you #ant to evaluate the transactions on a daily basis. Instead of manually entering a session override to filter source data each time you run the session, you can create a mapping variable, ))IncludeDate$ime. In the source qualifier, create a filter to read only ro#s #hose transaction date equals ))IncludeDate$ime, such as* $I'E%$A'+ , ))IncludeDate$ime In the mapping, use a variable function to set the variable value to increment one day each time the session runs. If you set the initial value of ))IncludeDate$ime to -./.0112, the first time the Integration %ervice runs the session, it reads only ro#s dated -./.0112. During the session, the Integration %ervice sets )

)IncludeDate$ime to -.0.0112. It saves -.0.0112 to the repository at the end of the session. $he next time it runs the session, it reads only ro#s from August 0, 0112.

DIFFERENCE BETWEEN INFORMATICA 7.1 AND 8.1 =========================================== Object Permissions E!!ecti"e in "ersion 8.1.1# $o% c&n &ssi'n object (ermissions to %sers )*en $o% &++ & %ser &cco%nt# set %ser (ermissions# or e+it &n object. ,&te)&$ &n+ -o''in' Con!i'%r&tion E!!ecti"e in "ersion 8.1# $o% con!i'%re t*e '&te)&$ no+e &n+ .oc&tion !or .o' e"ent !i.es on t*e Pro(erties t&b !or t*e +om&in. -o' e"ents +escribe o(er&tions &n+ error mess&'es !or core &n+ &((.ic&tion ser"ices# )or/!.o)s &n+ sessions. -o' M&n&'er r%ns on t*e m&ster '&te)&$ no+e in & +om&in. We c&n con!i'%re t*e m&0im%m si1e o! .o's !or &%tom&tic (%r'e in me'&b$tes. Po)ercenter 8.1 &.so (ro"i+es en*&ncements to t*e -o' 2ie)er &n+ .o' e"ent !orm&ttin'. 3nico+e com(.i&nce E!!ecti"e in "ersion 8.1# &.. !ie.+s in t*e A+ministr&tion Conso.e &cce(t 3nico+e c*&r&cters. One c&n c*oose 3TF48 c*&r&cter set &s t*e re(ositor$ co+e (&'e to store m%.ti(.e .&n'%&'es. Memor$ &n+ CP3 Reso%rce 3s&'e 5o% m&$ notice &n incre&se in memor$ &n+ CP3 reso%rce %s&'e on m&c*ines r%nnin' Po)erCenter 6er"ices. Dom&in Con!i'%r&tion D&t&b&se Po)erCenter stores t*e +om&in con!i'%r&tion in & +&t&b&se. -icense 3s&'e

E!!ecti"e in "ersion 8.7# t*e 6er"ice M&n&'er re'isters .icense in!orm&tion. 8i'* A"&i.&bi.it$ 8i'* &"&i.&bi.it$ is t*e Po)erCenter o(tion t*&t e.imin&tes & sin'.e (oint o! !&i.%re in t*e Po)erCenter en"ironment &n+ (ro"i+es minim&. ser"ice interr%(tion in t*e e"ent o! !&i.%re. 8i'* &"&i.&bi.it$ (ro"i+es t*e !o..o)in' !%nction&.it$9 Resi.ience. Resi.ience is t*e &bi.it$ !or ser"ices to to.er&te tr&nsient !&i.%res# s%c* &s .oss o! connecti"it$ to t*e +&t&b&se or net)or/ !&i.%res. F&i.o"er. F&i.o"er is t*e mi'r&tion o! & ser"ice (rocess or t&s/ to &not*er no+e )*en t*e no+e r%nnin' t*e ser"ice (rocess becomes %n&"&i.&b.e. Reco"er$. Reco"er$ is t*e &%tom&tic or m&n%&. com(.etion o! t&s/s &!ter &n &((.ic&tion ser"ice is interr%(te+.

Web ser"ices !or A+ministr&tion T*e A+ministr&tion Conso.e is & bro)ser4b&se+ %ti.it$ t*&t en&b.es $o% to "ie) +om&in (ro(erties &n+ (er!orm b&sic +om&in &+ministr&tion t&s/s# s%c* &s &++in' +om&in %sers# +e.etin' no+es# &n+ cre&tin' ser"ices. Re(ositor$ 6ec%rit$ E!!ecti"e in "ersion 8.1# Po)erCenter %ses & more rob%st encr$(tion &.'orit*m. A.so (ro"i+es &+"&nce+ (%r'es !or (%r'in' obso.ete "ersions o! re(ositor$ objects. P&rtitionin' T*ere is +&t&b&se (&rtitionin' &"&i.&b.e. A(&rt !rom t*is )e c&n con!i'%re +$n&mic (&rtitionin' b&se+ on no+es in & 'ri+# t*e n%mber o! (&rtitions in t*e so%rce t&b.e &n+ t*e n%mber o! (&rtitions o(tion. T*e session cre&tes &++ition&. (&rtitions &n+ &ttrib%tes &t t*e r%n time.

Reco"er$ T*e reco"er$ o! )or/!.o)# session &n+ t&s/ &re more rob%st no). T*e st&te o! t*e )or/!.o):session is no) store+ in t*e s*&re+ !i.e s$stem &n+ not in memor$. FTP We *&"e o(tions to *&"e (&rtitione+ FTP t&r'ets &n+ In+irect FTP !i.e so%rce;)it* !i.e .ist<. Per!orm&nce P%s*+o)n o(timi1&tion 3ses incre&se+ (er!orm&nce b$ (%s*in' tr&ns!orm&tion .o'ic to t*e +&t&b&se b$ &n&.$1in' t*e tr&ns!orm&tions &n+ iss%in' 6=- st&tements to so%rces &n+ t&r'ets. On.$ (rocesses &n$ tr&ns!orm&tion .o'ic t*&t it c&nnot (%s* to t*e +&t&b&se. F.&t !i.e (er!orm&nce We c&n cre&te more e!!icient +&t& con"ersion %sin' t*e ne) "ersion. We c&n conc%rrent.$ )rite to m%.ti(.e !i.es in & session )it* (&rtitione+ t&r'ets. One c&n s(eci!$ & comm&n+ !or so%rce or t&r'et !i.e in & session. T*e comm&n+ c&n be to cre&te & so%rce !i.e .i/e >c&t & !i.e>. A((en+ to !.&t !i.e t&r'et o(tion is &"&i.&b.e no). Pmcm+:in!&cm+ Ne) !e&t%res &++e+ to (mcm+. We c&n %se in!&cm+ to &+minister Po)ercenter +om&ins. In!&set%( (ro'r&m to set%( +om&in &n+ no+e (ro(erties. M&((in's We c&n no) b%i.+ c%stom tr&ns!orm&tion en*&ncements in API %sin' c?? &n+ @&"& co+e. We c&n %se (&rtition t*re&+s to s(eci!ic c%stom t

2) caches or automatic u dont need to allocate at transformation level, push down optimization techniques,some new java transformations,changes in parameter file and intigration services rather than powercentre server 3) 1)powercenter connect for S ! "et#eaver $# %ption 2)S&' (ransformation is added 3)Service %riented rchitecture ))*rid concept is additional feature +) ,andom file name can genaratation in target -) .ommand line programms/ 0nfacmd and infasetup new commands were added1 2) 3ava (ransformation is added feature 4)concurrent cache creation and faster inde5 6uilding are additi

3erfor!ance tuning
11 21 31 )1 +1 -1 21 41 ;1 0ncrease loo7up,aggregator,joiner cache size1 (urning off the ver6ose mode1 (urn off 8collect performance statistics91 ,emoving unnecessar: ports1 dropping inde5es1 3oining the sources in the source qualifier if coming from the same d61 partitioning .reate parallel sessions1 !assing sorted input for ggregator,3oiner

4or!al .or!# (. !ormali>ation is process or assigning attributes to entities: 7educes data redundancies: ?elps eliminate data anomalies: =roduces controlled redundancies to link tables +.!ormali>ation is t$e anal#sis o unctional dependenc# bet"een attributes @ data items o user&ie"sA%t reduces a complex user &ie" to a set o small and stable subgroups o ields @ relations (!F.7epeating groups must be eliminated2 Bependencies can be identi ied2 All ke# attributes de ined2 !o repeating groups in table +!F. T$e Table is alread# in(!F2%ncludes no partial dependencies: !o attribute dependent on a portion o primar# ke#2 Still possible to ex$ibit transiti&e dependenc#2

Attributes ma# be unctionall# dependent on non)ke# attributes ,!F. T$e Table is alread# in +!F2 Contains no transiti&e dependencies

Junk dimensions and degenerate dimensions

Chuc0 5elle)6s Answer# Cunk dimensions are dimensions t$at contain miscellaneous data 1like lags and indicators5 t$at do not it in t$e base dimension table. ?ere is a link to 7alp$ Dimball and Marg# 7oss on Eunk dimensions. A degenerate dimension is data t$at is dimensional in nature but stored in a act table. For example2 i #ou $a&e a dimension t$at onl# $as Frder !umber and Frder Line !umber2 #ou "ould $a&e a (.( relations$ip "it$ t$e Fact table. Bo #ou "ant to $a&e t"o tables "it$ a billion ro"s or one table "it$ a billion ro"s. T$ere ore2 t$is "ould be a degenerate dimension and Frder !umber and Frder Line !umber "ould be stored in t$e Fact table.

2) caches or automatic u dont need to allocate at transformation level, push down optimization techniques,some new java transformations,changes in parameter file and intigration services rather than powercentre server 3) 1)powercenter connect for S ! "et#eaver $# %ption 2)S&' (ransformation is added 3)Service %riented rchitecture ))*rid concept is additional feature +) ,andom file name can genaratation in target -) .ommand line programms/ 0nfacmd and infasetup new commands were added1 2) 3ava (ransformation is added feature 4)concurrent cache creation and faster inde5 6uilding are dditi

T$e arc$itecture o =o"er Center ; $as c$anged a lotG (. =C; is ser&ice)oriented or modularit#2 scalabilit# and lexibilit#. +. T$e 7epositor# Ser&ice and %ntegration Ser&ice 1as replacement or 7ep Ser&er and %n ormatica Ser&er5 can be run on di erent computers in a net"ork 1so called nodes52 e&en redundantl#