Академический Документы
Профессиональный Документы
Культура Документы
What you need to know to get the most out of using SAP NetWeaver BW as your enterprise data warehouse
Dr. Bjarne Berg
20010 Wellesley Information Services. All rights reserved.
Designing for Flexibility The Support Organization The top 10 EDW pitfalls
In th is se ssio n .
We will take a detailed look at the pros and cons of your EDW architectural options, including federated, centralized, and distributed EDW models, and explore when each approach is appropriate. Learn how to interface The Support Organization and how to consolidate different master and transactional data. Weigh your options for building a centralized or a decentralized EDW support organization. Examine the top 10 pitfalls companies face when implementing SAP NetWeaver BW as their EDW and how to overcome them.
4
BI applications are not required to run BI applications are not required to run on top of data warehouses, but the on top of data warehouses, but the majority does majority does
5
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
Source Data
Invoicing Systems Purchasin g Systems General Ledger Other Internal Systems External Data Sources
Extract
Transform
Data Warehouse
BI Applications
Custom Developed Applicati ons Data Mining Statistic al Programs Query Access Tools
Metadata
IT Driven Data Warehouses
SAP BW(s) SAP BW(s)
SAP BW InfoCubes SAP DSOs
Enterprise Portal
Financial Report center Sales Report center Manufactu ring Report center HR Report center Partner facing Report center Customer facing Report center Ad-Hoc Report center
Users
Employees
SAP BWA
Customers
Partners
Federated Data Warehouses are best in very large organization where development is separated by geography, organizational boundaries, or where multiple data warehouses exists due to mergers & acquisitions. To make FDWs successful, there needs to be a rapid convergence to standardized technologies. This include:
Same type of databases and support pack levels (costs and compatibility) Same technical platforms Hardware, Backups and Archiving (costs) If the data is federated you and faster response time to business needs, Shared Portal gain user interface strategy (reduced can execute multiple projects in parallel, and work 24/7 across the training and support) globe. But without any standardization, it can also be very costly. Shared security design and centralized 9 administration (risk management)
Metadata
IT Driven Data Warehouses
OLTP sources OLTP sources
SAP ECC Siebel, JDE Oracle Others
Enterprise Portal
Financial Report center Sales Report center Manufactu ring Report center HR Report center Partner facing Report center Customer facing Report center Ad-Hoc Report center
Users
Employees
SAP BW SAP BW
Customers
Partners
SAP BWA
10
Centralized Data Warehouses are great for small and mid-size data warehouses (less than 1540Tb). There are great benefits in terms of the ease to mange upgrades, support packs, enforcing development standards, transport control, master data management and the overall total cost of ownership To make CDWs successful, there needs to be:
Adequate funding of hardware, application servers, database servers Serious consideration should be made to move BI and reporting to BWA Focus on using the database capacity on storage and data loads-- not queries If No data is centralized it isfrom to develop new solutions for the the direct reporting faster DSOs (takes too much system resources) business and merging from different data sources are easier 11 Broadcasting , caching and performance tuning is a
Metadata
IT Driven Data Warehouses
SAP BW(s) SAP BW(s)
SAP BW InfoCubes SAP DSOs
Enterprise Portal
Financial Report center Sales Report center Manufactu ring Report center HR Report center Partner facing Report center Customer facing Report center Ad-Hoc Report center
Users
Employees
SAP BWA
Customers
Partners
12
A Decentralized Data Warehouses makes sense if there are logical division between business units, geographies and little shared reporting I.e. in a conglomerate organization with diverse business units. The benefits of DDWs include the flexibility of the FDW with the technology standardization and lower cost of ownership of the CDW. To make DDWs successful, there needs to be:
A formal Masterdata Management (MDM) strategy with clearly defined standards A rule based data cleaning and data integration plan for centralized reporting
13
With DDWs there is a risk of creating stove-pipe data marts that A be integrated at the corporate level without very high costs. cannot shared hardware location to keep costs lower
Fe
Be wh Organization by
14
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
15
For all data warehouses 60-80% of the effort is to move, store, retrieve and integrate data from various source systems.
From a SAP perspective, Information management is six distinct efforts. Therefore, several SAP BI tools exists with different capabilities
1616
Target
17
Extraction and data movement may take 30-50% of the time in a process chain. Therefore, do not plan to build an EDW with slow non-native connectivity to the source systems.
You have to prove that the data in the warehouse is equal to the data you extracted, or your financial reporting systems will have no credibility.
You are also legally required to have a reconciliation process that can be tracked, if you use the warehouse for financial reporting to external entities.
Many companies invest in developing manual control queries, while others use reconciliation products that are powered by SAP NetWeaver
An example of a reconciliation Dashboard built on SAP BW. In this example: 1.A reconciliation memo was written on Feb. 1st 2.PCA reconciliation between BW and R/3 failed on Feb. 16th
Using BOBJ Data Services you can consolidate data from many source systems, cleanse and integrate them before you send it to SAP BI. This avoids multi-nested DSOs and complex load logic.
Source systems - Oracle - JDE - Peoplesoft - Baan - Siebel - Custom - Hyperion - Other.
2121
Interesting use BOBJ Data Services Using BOBJ Data Services you integrate, cleanse and merge data from source systems during
The Data Profile Tab in BOBJ Data Services This tab in the view data screen contains data profile statistics on each column that can help you decide on the quality of the input data.
1. Column Name 2. Number of distinct values in a column 3. Number of records with a NULL value in this column 4. Maximum & Minimum value of the column
23
The Validation Validation allows you to create rules for cleaning data prior to loading it to the system. You can have a pass rule and an 'Action on Failure' that can provide complex logic.
24
The Audit The Auditing selection allows you to take complex actions when the data quality is poor. You can:
1.Send an email to an administrator 2.Load the data to a table for later correction 3.Modify the data through scripts 4.Create custom functions for your own processing logic
25
rsa l D a ta C le a n sin g : E x a m p le o f E n h a n ce d P a rty M a ste You can also add new items such as geocodes for visualization in SAP BI I . e . maps You can add new characteristics to the data such as :
1)Legal tax jurisdictions 2)Census track ID 3)Block group ID 4)Insurance rating territories 5)Tax authority name 6)Tax authority FIPS codes 7)Longitude & Latitude 8)City type 9)...
GREAT FEATURE : The Census track ID allows you to analyze your customers and partners using government census information Source: SAP AG, 2009
A common way to look at customer data is by Households instead of single records . BOBJ DQ allows you to look at customer's addresses and create shared master records , customer mapping keys , aggregating data ( i . e . aggregated sales data for the household ), check " no - call " lists , examining churn ( apparent customer turn - over ). You can also integrating all master data from many records into a single " super record " that contains all the unique master data you have about a single customer or partner .
SAP Data Quality Management has pre-delivered content for many solutions including CRM -> ECC integration, including:
1)Across platform search capabilities 2)Automated address correction 3)De-Duplication of records 4)Direct system connection (no file extraction) 5)Supported for all major releases: R/3 4.6c; ECC 5 and 6; CRM 4 and 5
" Data Quality Management for SAP provides a prepackaged native integration of data quality best practices within the SAP environment using the BOBJ Data Services platform "
SAP AG, 2009
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
30
In BW 7.2 a new feature called "Semantic partitioned object" (SPO) is introduced to help partition InfoCubes for query performance, and DSOs for load performance. B W 7 . 2 p ro v id e s W iza rd s to h e lp y o u p a rtitio n o b je cts b y y e a r, b u sin e ss u n its o r p ro d u cts . B W a lso g e n e ra te a u to m a tica lly a ll n e e d e d D T P su ch a s tra n sfo rm a tio n ru le s Maintenance to loeasier since ct a n d filte rs is a d th e co rre any remodeling only P need to. change the reference structure . in fo ro v id e r
SPOs can be added to MultiProviders for easy query administration and to mask complexity
31
32
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
34
If there is no support organization, the BI system quickly becomes an orphan when the project ends
Without a support org. there is a risk that future BI projects are delayed since the project team has to support previous projects
The BI Help Desk Level 1 Support The first level support should be done by Power Users in the organization
You will have to train these resources, empower them to make changes, and leverage them
Query related support tickets from a central location/Web site should be routed to the power users in each department. The power user can escalate the ticket to Level2 support if he/she is unable to resolve it.
The BI Help Desk Level 2 Support The second level support is used for issues that are not related to queries, presentations, reports, and formatting
Some support ticket types are always routed to Level 2 support. It is important to have a generic email address for Level 2 support that is not related to an individual. Emails to this address should not be deleted.
BWB
Project Stack
BWQ
BWP
Training
The Break-Fix and production stack as well as the training environment is owned by the support team. The project teams own the development and Sandbox environments (BWS and BWD).
BWD BWS
BWT
By Introducing a Break-Fix (BWB) environment, the support team can correct break-fixes and move code into the Testing environment (BWQ) and Production environment (BWP) without impacting the project team
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
39
1.90% of all queries run under 20 seconds 2.System is available 98% of the time 3.Data loads are available at 8am 99% of the time 4.User support tickets are answered within 30 minutes (first response) 5.User support tickets are closed within 48 hours 95% of the time. 6.System is never unavailable for more than 72 hrs including upgrades, service packs,
BI is complex with many different tools and technologies. Dont rely on a single person with no specialized skills. Make each person responsible for a focused technology/task.
Have one architect quality is more important than quantity Architecture is technical by nature. PowerPoints only gets you a small part of
Pitfall #5: A Firm Belief in Monolithic Data Warehouses Google runs on over 500,000 servers, why must your data warehouse run on one? Divide and concur when the performance becomes a too-large problem. Separate BI onto SAP BWA and use the data warehouse for data movement and data
Pitfall #6: Analysis Paralysis. You will never have perfect EDW requirements get over it. The business will change and so will the BI system. Change is a sign of success not failures (people who cares wants to make it better). Not moving forward and keep analyzing is a costly decision
Pitfall #7: A Single User Interface will solve all my EDW problems.. There are no magic bullets. Most companies need 2-3 end user tools. Start with OLAP (Pioneer) web, then continue
Pitfall #8: Enforce EDW Standards Standards are not a word document buried in a file cabinet If you allow exceptions the standards quickly become meaningless. It costs to keep your house clean, but data management and data integration will benefit greatly from it. Remember: the road to
hell is paved with good intentions - unknown.
Pitfall #9: Keep Your EDW Support Team motivated The average application developer stays on the job for 47 months, the average support person is only there for 25 months!
Use ad-hoc best practice advise from external experts on an periodic basis. If you are struggling with something, there are many others who have cracked the nut already leverage their experiences. Attend BI conferences, take good notes and leverage the many experts at the booths, the speakers and the forums. You are not alone, but your team needs to get plugged into the many ASUG, BI Expert, SDN and SAP BI communities.
Federated Data Warehouse Centralized Data Warehouse Distributed Data Warehouse Masterdata Transaction data conversion Data cleansing
Designing for Flexibility The Support Organization The top 10 EDW pitfalls Wrap-up
46
Resources
http://www.comeritinc.com/Downloads. htm
by Alan Schlukbier
47
There are more than one way to architect an EDW. However, you need to make sure your BI solution is designed, not evolutionary Consider FDW and DDWs when data volumes are extremely high or your company just underwent a merger or acquisition Make the front-end independent from the backend Formalize a data integration strategy with MDM and Reconsolidation as key focus areas Invest in people, not just technology Great support staff is key to EDW success SAP BWA should be part of your EDW strategy
48
Your Turn!
H o w to co n ta ct m e : D r. B ja rn e B e rg b b e rg @ C o m e ritIn c . co m
49
Disclaimer
SAP, R/3, mySAP, mySAP.com, SAP NetWeaver, Duet, PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.