DATA WAREHOUSING

Submitted By: Bhawna Rai (12215) Shameel javed (12259) Shruti Pandey (12206) Shweta Kapoor(12248) Sabina kauser (12243)

Data Warehouse
A collection of corporate information, derived directly from operational systems and some external data sources. Its specific purpose is to support business decisions, not business operations

DATA WAREHOUSING :- BRIEF

Data warehousing is subject-oriented, integrated, time-variant, and non-volatile collection of data in support of managements decision-making process Goal: is to integrate enterprise wide corporate data into a single repository from which users can easily run queries

Data Warehouse Components

Staging Area A preparatory repository where transaction data can be transformed for use in the data warehouse Data Mart Traditional dimensionally modeled set of dimension and fact tables Per Kimball, a data warehouse is the union of a set of data marts Operational Data Store (ODS) Modeled to support near real-time reporting needs.

Data mart
data mart a subset of a data warehouse that supports the requirements of particular department or business function The characteristics that differentiate data marts and data warehouses include:
a data mart focuses on only the requirements of users associated with one department or business function data marts do not normally contain detailed operational data, unlike data warehouses data marts contain less data compared with data warehouses, data marts are more easily understood and navigated

Reasons for creating a data mart

To give users access to the data they need to analyze most often To provide data in a form that matches the collective view of the data by a group of users in a department or business function To improve end-user response time due to the reduction in the volume of data to be accessed To provide appropriately structured data as dictated by the requirements of end-user access tools Normally use less data so tasks such as data cleansing, loading, transformation, and integration are far easier, and hence implementing and setting up a data mart is simpler than establishing a corporate data warehouse

Tools And Technologies

The critical steps in the construction of a data warehouse: a. Extraction b. Cleansing c. Transformation after the critical steps, loading the results into target system can be carried out either by separate products, or by a single, categories: code generators database data replication tools dynamic transformation engines

Data flows
Inflow- The processes associated with the extraction, cleansing, and loading of the data from the source systems into the data warehouse. Up-flow- The process associated with adding value to the data in the warehouse through summarizing, packaging , packaging, and distribution of the data Down-flow- The processes associated with archiving and backing-up of data in the warehouse Outflow- The process associated with making the data available to the end-users Meta-flow- The processes associated with the management of the meta-data

Evolution architecture of data warehouse

Top-Down Architecture Bottom-Up Architecture Enterprise Data Mart Architecture Data Stage/Data Mart Architecture

Top-Down Architecture

Bottom-Up Architecture

Enterprise Data Mart Architecture

Data Stage/Data Mart Architecture

Very Large Data Bases

WAREHOUSES ARE VERY LARGE DATABASES
Terabytes -- 10^12 bytes: Wal-Mart -- 24 Terabytes

Petabytes -- 10^15 bytes:

Exabytes -- 10^18 bytes: Zettabytes -- 10^21 bytes: Zottabytes -- 10^24 bytes:

Geographic Information Systems National Medical Records

Weather images Intelligence Agency Videos

Benefits
Extract Information from data to use as the basis for decision making Used at all levels of the Organization Combines historical operation data with business activities Allows end users to perform extensive analysis Allows a consolidated view of corporate data Better and more timely information A Enhanced system performance Simplification of data access

Contd..
Created to facilitate the decision making process So much information that it is difficult to extract it all from a traditional database Need for a more comprehensive data storage facility

Data Warehouse Pitfalls

You are going to spend much time extracting, cleaning, and loading data You are going to find problems with systems feeding the data warehouse You will find the need to store/validate data not being captured/validated by any existing system Large scale data warehousing can become an exercise in data homogenizing

Success & Future Of Data Warehouse

The Data Warehouse has successfully supported the increased

needs of the State over the past eight years.

The need for growth continues however, as the desire for more integrated data increases. The Data Warehouse has software and tools in place to provide the functionality needed to support new enterprise Data Warehouse projects. The future capabilities of the Data Warehouse can be expanded to include other programs and agencies

THANK YOU