Вы находитесь на странице: 1из 42

Today’s Agenda

 Theory of Fives
 A Bit About Methodology
 Star & Snowflake Schemas
 Populating a Star Schema
 Slowly Changing Dimensions (the real
reason to warehouse data)
Presenter(s):
 ResiliEnt Overview Roger Panfil, President,
ResiliEnt Business Solutions

www.resilientbiz.com
“Helping organizations leverage their investment in Information Builders technology ”
Presentation Information

 Author: Roger Panfil


 Company: ResiliEnt Business Solutions
 Presentation Title: Data Warehousing Best Practices
 Presentation Abstract: Do you know the difference between a
star schema and snowflake design? What are type 1, 2, and
3 dimensions and when should they be important in providing
the business intelligence you need? Do you know the telltale
signs for pre-aggregating/summarizing data in the data mart
for efficient reporting? These questions and more will be
answered with tips for handling each in iWay's DataMigrator
product. A customer success story will accompany the
presentation.

“Helping organizations leverage their investment in Information Builders technology ”


Before We Get Started . . .

1) How many of you are building your first DW?


2) Are you going to use an ETL tool?
3) Are you going it alone, or are you partnering with a
consulting firm?
4) What is the expected time frame to deliver the project?
5) What is the expected time frame to deliver the first
report?

“Helping organizations leverage their investment in Information Builders technology ”


The Theory of Five’s

“Helping organizations leverage their investment in Information Builders technology


“Better Service
” at a Better Price”
The Theory of Five’s

1) Pick a business line VP with an information problem . . .


2) Gather his/her Five top key performance indicators
3) Deliver Five answers:
 Requires Five things: Web interface, Security, Supporting tables
and ETL process, Content (Reports, Graphs, Etc).
 Five reports
 Five drills
 Five more drills
4) Solicit VP #2
 Repeat Step #3
5) Repeat steps 4-5 until you have satisfied Five VP’s.

You have just delivered and enterprise solution across Five


business lines (subject areas) with BIG value.
“Helping organizations leverage their investment in Information Builders technology ”
The Theory of Five’s

1) It adheres to a time-box methodology


2) It’s a repeatable model for success
3) It delivers real value to the organization
4) It’s easily expanded
5) It works!

“Helping organizations leverage their investment in Information Builders technology ”


A Bit About Methodology

“Helping organizations leverage their investment in Information Builders technology


“Better Service
” at a Better Price”
Methodology
Our’s Looks Like This

We Work With You: We Deliver:


Project Requirements…………………………. Findings Document
Initial Proposal
Project Charter
JAD Session……………………………………..
Functional Specifications
Gap Analysis
Statement Of Work
Technical Requirements
Investment Analysis

Project Management…………………………… Project Kickoff Meeting


Detailed Project Plan
Technical Specifications
Weekly Status Reporting
Measurable Milestones………………………...
Frequent, Tangible Results

Client Satisfaction ……………………………... An End Result You Are Happy With!
Unparalleled Client Care

“Helping organizations leverage their investment in Information Builders technology ”


Apply Proven App Dev Concepts to
Data Warehousing

 Define the problem


 Define the project and team JAD: Joint
Application Design
 Define the solution
 Define the acceptance criteria
 Define the time-line Time-Boxing
 Define the deliverables
 Deliver RAD: Rapid Application
 Get Feedback Development

 Revise
Client Satisfaction
 Achieve signoff
 Promote Success

“Helping organizations leverage their investment in Information Builders technology ”


Resilient Business Intelligence
A Matter of Topology, Technology and
Methodology

Cover all of your


platform, Ensure you have
connectivity and all of the right
information asset tools across as
needs . . . Past, few vendors as
present & future. possible.
Topology Technology

Where the circles


Methodology converge marks the
Spend the money on
sweet spot for resilient
services, training, and business intelligence . . .
mentoring to ensure your
technology expenditure
is safe!
“Helping organizations leverage their investment in Information Builders technology ”
The Star Schema

“Helping organizations leverage their investment in Information Builders technology


“Better Service
” at a Better Price”
Some Questions We’ll Answer . . .

1) What are some of biggest reasons to report from


staged data rather than live data?
2) What is a Star Schema?
3) What’s the difference between a Star Schema and a
snowflake schema?
4) What are Slowly Changing Dimensions?

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema

The star schema is a favorite


tool of warehouse designers for
providing high performance
query structures in a relational
environment.

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema

The Star Schema is query fast because . . .


 Pre-defined paths based on the way users
want to see the data that is important to them.
 Pre-defined aggregates and calculations turn
raw data into information.
 Data Keys replaced with high performance
Surrogate Keys

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema

Dimension Tables:
•Product
•Customer
•Time

Facts Table:
•Sales

Fact Measures:
•Units Sold
•Unit Price
•Unit Cost

Example taken from Oracle 8i manual Chapter 7

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema

 Dimension Tables are short, but wide (low # rows,


high # fields)
 Fact Table is long, but narrow (high # rows, low #
fields)
 Theory of Fives: Five or less dimensions per Star
keeps it manageable.
 ETL tool is absolutely required to assist with the
build and maintenance of these structures.

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema

Dimension tables
contain both Why?
original data and
surrogate keys

Dates and flag for


tracking of slowly
changing dimensions

Time largely controls


granularity . . . And
Fact table only granularity largely
contains controls the size of the
generated Fact data store
surrogate keys

“Helping organizations leverage their investment in Information Builders technology ”


The Snowflake Schema =
Star Schema + Normalization

...

Shorten the width of the


Dimensions thru
normalization

“Helping organizations leverage their investment in Information Builders technology ”


The Snowflake Schema

While this design can save some storage space . . .

 More complex to build and maintain


 Negative effect on response time

“Helping organizations leverage their investment in Information Builders technology ”


Loading a Star Schema
with DataMigrator

“Helping organizations leverage their investment in Information Builders technology


“Better Service
” at a Better Price”
The Star Schema
Data Flow at Work – Create Dimension Tables

Draw an ETL request


like this one to load
each dimension table:
Customer, Product,
Time

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Data Flow at Work – Create Dimension Tables

Simple source
transform to create
surrogate key

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Data Flow at Work – Create Dimension Tables

Simple target
transforms to populate
fields for tracking
slowly changing
dimensions (SCD)

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Data Flow at Work – Create Staging Table
Source tables are joined to
extract the fact data, then the
measures are aggregated by
dimension, the Time dimension
typically controlling granularity

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Data Flow at Work – Create Staging Table

Another transform,
this time in the column
object, to derive Line
Cost of Goods
(LINECOGS)

Dimension tables and the staging


table are created in the same RDBMS
for optimal join performance . . .

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Data Flow at Work – Create Fact Table

Join all dimension


tables to the staging
table by data-key
fields, but selecting
the surrogate key
fields, to create the
fact table

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
Process Flow at Work
CRE8ECNTFACTS waits
until all jobs in group
are finished

eMail DW
Admin that ETL
process is
complete

Concurrent execution of
jobs to load dimension
tables and staging table

“Helping organizations leverage their investment in Information Builders technology ”


The Star Schema
70k rows, 5 tables, 57 seconds!

“Helping organizations leverage their investment in Information Builders technology ”


Slowly Changing
Dimensions

“Helping organizations leverage their investment in Information Builders technology


“Better Service
” at a Better Price”
Slowly Changing Dimensions

Source: www.1keydata.com/datawarehousing

“Helping organizations leverage their investment in Information Builders technology ”


Slowly Changing Dimensions
Type 1

Type 1
slowly changing
dimension should be
applied to changes
in source data fields
when it is not
necessary for the
data warehouse to
keep track of
historical changes.

No need for
active record
flag and date-
added stamp.

Source: www.1keydata.com/datawarehousing

“Helping organizations leverage their investment in Information Builders technology ”


Slowly Changing Dimensions
Type 2

Type 2
slowly changing
dimension should
be used for when
it is necessary for
the data
warehouse to
track historical
changes in source
data fields.

Active record
flag and date-
added stamp
are critical.
Source: www.1keydata.com/datawarehousing

“Helping organizations leverage their investment in Information Builders technology ”


Slowly Changing Dimensions
Type 3

Type 3
slowly changing
dimension can be
used for when it is
necessary for the
data warehouse to
track a limited and
finite number of
historical changes
in source data
fields.

Requires prior and


current fields and
effective date of
change fields.

Source: www.1keydata.com/datawarehousing

“Helping organizations leverage their investment in Information Builders technology ”


Slowly Changing Dimensions
Quiz

 A top salesman for Information Builders gets


married and changes his name . . .
Type 1 or Type 2 . . . And Why?
 Dave Kemler decides to make Pittsburgh part of
the Mid-West region, and does this on June 1 . . .
Type 1 or Type 2 . . . And Why?
 Jerry Duci changes the product code for
WebFOCUS on the IBI Fee Schedule . . .
Type 2 or Type 3 . . . And Why?
“Helping organizations leverage their investment in Information Builders technology ”
Wrap Up

“Helping organizations leverage their investment in Information Builders technology ”


Who We Are . . .

 Founded September 2004.


 Co-Founders: Roger Panfil (formerly with IBI/iWay Software) & Laila
Samawi-Utley (formerly with SunTrust Bank), Penny Lester (formerly with
IBI).
 12 full time staff, with the key positions being:
 SVP Delivery
 DW/ETL Specialist
 BI Specialist
 Technical Recruiter
 Over 350 screened consultants in our DB promotes “on demand” staffing.
 State of the art back office system (WebPAS) for managing a consulting
practice.
 Administaff runs our payroll & benefits keeping our internal headcount
light.

“Helping organizations leverage their investment in Information Builders technology ”


ResiliEnt’s Six Drivers of
Customer Satisfaction . . .

“Helping organizations leverage their investment in Information Builders technology ”


What We Do . . .

 Practice Expertise:
 Business Intelligence
 Data Warehousing
 Healthcare and Financial Services verticals

 We can help with staffing or full project outsourcing for many


popular technologies:
 DW: IBI DataMigrator, DataStage, Informatica, MS DTS
 BI: IBI WebFOCUS, BO/Crystal, Cognos,
 DB: IBI FOCUS, Oracle, DB2, MS SQL Server
 Technologies: IBI Maintain, Java, .Net

“Helping organizations leverage their investment in Information Builders technology ”


Full Service Consulting . . .

Information System Delivery Business Planning


 Development of New Business Intelligence  Vendor and Asset Management
Applications  Organizational/Infrastructure Design
 Operational/Production Report Writing  Tactical and Strategic Solutions Design
 Legacy Report Conversion to New  Global IT Infrastructure Operations
Technologies Assessment
 Development of Data Warehouses, Data  Business Impact Analysis
Marts and Operational Data Stores  Cost Reduction Assessment
 Enterprise Application Integration  ROI & TCO (Total Cost of Ownership)
 System/Application/Product Configuration Analysis, Quantification and Optimization
Review and Tuning

IT Project Assistance “We excel at evaluating business needs, then leveraging existing
 Full Project Outsourcing resources and combining them with new technologies to create
 Project Management efficient, effective and resilient solutions. We ensure success by
 Staff Supplementation providing the resources and knowledge that will help your team
 Staff Mentoring effectively execute your company's strategic plan.”
 Training and Education Laila Utley, CEO, ResiliEnt Business Solutions

“Helping organizations leverage their investment in Information Builders technology ”


Creative Solutions . . .

 We excel at “expense free” consulting . . . We run our


company as a virtual organization, that’s how good we are
at it!
 Student labor program with 17 universities that make up the
North Carolina University System – Great for conversion
work and contained projects.
 Deep list of consultants that will take on part-time
assignments at reduced rates.
 Offshore capacity established for large contained projects
where the rate must be brought down (e.g. conversion
work).

“Helping organizations leverage their investment in Information Builders technology ”


Testimonials

“ResiliEnt Business Solutions can be counted on to deliver. They have quickly become
a trusted partner to my IT staff. I consider them an essential part of my team.”
Suresh Ramakrishnan, Vice President of Information Technology, XLHealth

“ResiliEnt came into a project that was deemed un-doable by the ‘experts’ . . . un-
doable because of time and technology. ResiliEnt parachuted 3 developers into the
account and on time and on budget what others said couldn't be done. Actually, the
project team was done 3 days ahead of schedule!”
Craig Kozlow, Project Manager, Lead Developer and Independent Consultant

“ResiliEnt Business Solutions has proven to be worthy of our confidence and high
regard.”
Ken Heller, President, Thrift Credit Union

“Helping organizations leverage their investment in Information Builders technology ”


www.resilientbiz.com

770-625-5090
“Helping organizations leverage their investment in Information Builders technology ”

Вам также может понравиться