Академический Документы
Профессиональный Документы
Культура Документы
Deb Bhattacharjee
Director
Lionel Hsu
Consulting Member of Technical Staff
Saugata Chowdhury
Consulting Member of Technical Staff
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.
Agenda
What is Exalytics? Exalytics Performance Tuning Exalytics Considerations
WHAT IS EXALYTICS?
5 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Benefits
Speed-of-Thought Interactive Visual Analysis Faster Planning Cycles with Richer Models Quick to Deploy, Supports More Users
Unique Features
In-memory Analytics Accelerates BI and EPM Apps Fits with existing data sources, infrastructure Full-stack optimizations hardware & software Optimized with Exadata InfiniBand
AVAILABLE TODAY
Exalytics Components
In-Memory Software
(TimesTen for Exalytics and Optimized Essbase)
RAID 1
/boot (99MB) boot partition / (447GB) Operating system, logs, temp space
600 GB SAS - 2
600 GB SAS - 3 600 GB SAS - 4 600 GB SAS - 5 RAID 5 /u01(1.7TB) software, TT data files
OBIPS (sawserver)
OBIS (nqserver)
Webcat
Repository Essbase
Exalytics
Data Warehouse
10
2.7x throughput gain for on Exalytics when compared to 11.1.1.5 on same hardware (comparing peak throughput of 637 vs 235) Baselines throughput actually dropped under load due to lock contentions
11
Throughput
BI Query reporting w/ Exalytics on Oracle DB In-memory aggregates (Adaptive In-Memory Cache) vs. the baseline DW: 18X faster on the average Individual dashboard response time gain upto 45x (Filter Department)
12
BI Query reporting w/ Exalytics on Exadata 23X average response time improvements thanks to in-memory aggregates, fast interconnect and SQL optimizations for Exadata as compared to the base system Individual transaction gains upto 60X (Filter-Department)
13 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
About 5x response time improvement under load 7x improvement in CPU Utilization by Essbase
14
Up to 93x response time improvement with individual MDX Queries Up to 15x response time improvement for individual interactive calc scripts About 10x overall throughput improvement in mixed load scenarios accompanied by 1.8x CPU usage improvement
15
Exalytics
Metrices
Comments
Avg response time (sec) 0.18 Total CPU % (BI + TT) 91%
0.18 93%
16
Exalytics
Essbase BSO Interactive Calc and Query
Setup
Bare metal : 40 cores , 1 TB RAM OVM ver 3.0.3: Dom 0 pinned to 10 cores
Throughput Txn/sec
30 25 20 15 10 5 0
22.9
17
Browser/Mobile
Timesten DB
OBIPS (sawserver) OBIS (nqserver) Hardware Acceleration flag Logical SQLs/Physical SQLs Init block Server Thread pool DB Gateway connection pool OBIS Cache Temp/log folder location Log level
Hardware Acceleration flag Chart Thread Pool OBI PS Cache Session timeouts Cursor Wait Time Memory Quota Temp/log folder location
Webcat
Repository Essbase
Exalytics
Data Warehouse
19
Exalytics
Set the default log level to Error
Log levels should be the first place to check while debugging performance issues
20
Exalytics
Move tmp/log files to RAMDisks
Move temp files and log files to ramdisk for better scalability
OBIPS cd $BI_HOME/instances/instance1/tmp/OracleBIPresentationServicesComponent mv coreapplication_obips1 coreapplication_obips1.localDisk ln -s /dev/shm/coreapplication_obips1 .\ Javahost mkdir /dev/shm/coreapplication_obijh1 cd $BI_HOME/instances/instance1/tmp/OracleBIJavaHostComponent mv coreapplication_obijh1 coreapplication_obijh1.localDisk ln -s /dev/shm/coreapplication_obijh1 .\ OBIS tmp folder mkdir /dev/shm/coreapplication_obis1 cd $BI_HOME/instances/instance1/tmp/OracleBIServerComponent mv coreapplication_obis1 coreapplication_obis1.orig ln -s /dev/shm/coreapplication_obis1 . OBIS Sorting/Working directory: since temp folder is pointing the RAMDISK from above settings, use default for sorting/working directory so it will be using RAMDISK as well WORK_DIRECTORY_PATHS = ""; (blank means using default path)
21
Exalytics
I/O buffer size
OBIS file buffer size
Increase file buffer size to 128KB to ensure good write to disk performance of sorting and cache Population (28.5x improvement in 11.1.1.7 )
22
Exalytics
Caching
OBIPS Caches (instanceconfig.xml)
23
Exalytics
Configure Right Threadpool Size
OBIPS Chart Threadpool size(instanceconfig.xml)
OBIPS has a thread pool for Javahosts. Its size needs to be the total number of threads allowed in all Javahosts.
For example, there are two Javahost instances . Each has 300 threads defined in its own configuration file, and then the Max size for OBIPS threadpool size needs to be 600 (300 + 300).
<ThreadPoolDefaults> <ChartThreadPool> <MinThreads>300</MinThreads> <MaxThreads>600</MaxThreads> <MaxQueue>4096</MaxQueue> </ChartThreadPool> </ThreadPoolDefaults>
24
Exalytics
Configure Right connection pool size
OBIS RPD
AdminTool: Connection Pool -> General Tab -> Maximum connections
WLS
LDAP connection pool size in WLS, needs to meet the concurrent login rate
25
Exalytics
Configure Max connections/Sessions
WLS /analytics plug-in Max connections
26
Exalytics
Check Session Timeouts
WLS Analytics plug-in session timeout
Set the Session timeout(in seconds) to control the timeout of the sessions in Analytics plug-in in WLS The timeout should be in sync with the session timeout in OBI PS.
<ClientSessionExpireMinutes>5</ClientSessionExpireMinutes>
27
Exalytics
Exalytics Memory Tuning
Set the following shell variable for OBIPS in opmn.xml SIEBEL_MEMORY_THREADALLOC_QUOTA=8388608 (e.g to set thread quota to 8MB) UI will throw an error when a users request consumes more than this memory
For very large TimesTen Datastore (>256GB), The following Linux kernel and Timesten settings need to be changed. Please refer to Exalytics Best Practices doc for details.
kernel.shmmax/kernel.shmall/kernel.sem
/proc/sys/vm/nr_hugepages
/proc/meminfo: Hugepagesize ttendaemon.options: linuxLargePageAlignment
28
Exalytics
JVM heap size
This needs to be modified based on the number of concurrent sessions and BIP reports.
SERVER_MEM_ARGS="-Xms256m -Xmx2048m -XX:MaxPermSize=512m (setDomainEnv.sh) export SERVER_MEM_ARGS
For scenarios with Trellis charts , increase JVM heap size to 8GB (-xmx8192M ) <ias-component id="coreapplication_obijh1"> <data id="start-args" value="-server -Xms2048M -Xmx8192M "
29
Exalytics
Javahost clustering
Clustering Javahosts can help scalability
30
Exalytics
OBIS - Init Blocks
Repository init-blocks : Called once during
31
Exalytics
Currently BI does not support issuing SQLs with bind variables to the database
May result in high parse time in the database specially for very complex Queries
Work around: Add inititalization code in connection setting to change session parameter for
32
Exalytics
OBIS Log Analysis for Performance Tuning
OBIS Query Loglevel
Loglevel 2 provided the logical query , physical
overhead
Needs rpd access to change
Work-around: Have a session init-block defined
33
Exalytics
OBIS Log Analysis for Performance Tuning (contd)
Inspecting query log
Query Status
Query Status: Successful Completion
BI physical query will add a hash as a comment - /* 2c16620a */ select T2005.Country as c BI Query log will have the corresponding hash
-------------------- SQL Request, logical request hash: fdc49e9c Sending query to database named FSCM_OLTP (id: <<1712462>>), connection pool named Connection Pool, logical request hash fdc49e9c, physical request hash 2c16620a: [[
34
Exalytics
Monitoring
WLS Console
# of sessions (/analytics) in WLS
EM (Enterprise manager)
Sessions, requests/setc for WLS, OBIPS, OBIS & Javahost
BIEE AdminTool
Manage-Sessions: # of sessions, current request and session variables
35
Exalytics
36
Exalytics
37
Exalytics
38
Exalytics
39
Exalytics
Configuration changes for Load testing scripts
OBIPS Cursor Wait Time
There are two parameters that control how long OBI PS should wait before replying refreshing page to the browser. NewCursorWaitSeconds is the time to wait on the initial callback from the UI for a query to finish. OldCursor is for subsequent call backs for the query to finish. It is helpful to increase the NewCursorWaitSecons for performance load testing, so the load test scripts wont error out because of not seeing the expected results
<Cursors> <NewCursorWaitSeconds>1000</NewCursorWaitSeconds> <OldCursorWaitSeconds> 1000</OldCursorWaitSeconds> </Cursors>
40
EXALYTICS CONSIDERATIONS
41 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
42
Exalytics
OBIEE Performance acceleration for Analytic Queries
Technique Expected Performance Improvement Considerations
Consistently sub-second
For relatively static dashboard Pre-seeding cache, use IBOTs Purge cache on source data changes Use RAMDisk for storing the cache Aggregate Persistence for aggregate creation Summary Advisor for aggregate recommendation Build high-level KPIs with summary level performance and exception monitoring to tell a story Just raw data reports have no competitive advantage. OBIEE advantage is in multi-pass calculations, indexing, comparisons Drill-down reports via Action Links to root cause analysis Sizing and future data growth. Freeing up DB resources, in turn benefits other queries
Moving data into TimesTen Orders of magnitude faster depending on the queries and the reference database
43
Exalytics
BI Server TimesTen Connection Architecture
BI Server
TimesTen client driver
client/server connections within the same host (shared memory IPC)
Server daemon
Server
proxies
Admin/Utility programs
data source BI Server connects to TimesTen v11 database using TT specific ODBC 3.5
44
OBIEE
Modeling Function Shipping
Make sure the DB type is set
accurately.
Typically more recent DB version will
source
Common Performance issue: Using such function in filters derived columns used in filters
45
OBIEE
Physical and Logical Joins
Set right cardinality ({0..1, 1, N} to {0..1, 1, N, Unknown}) important for join trimming
Outer joins
Logical level
Add dimension table row for N/a or Unknown records during ETL Replace Null foreign keys with the N/A value either during ETL or via IfNull expressions in the RPD
46
OBIEE
Level Keys
defines the unique elements in each
logical level
Portland (ME), Portland (OR) are distinct Basis of dimensional conformance across
sources
Use unique, single column level keys
whenever possible
Do not use derived columns Consider LTS expression
47
OBIEE
OBIEE - Descriptor ID columns
Use Descriptor ID Columns,
Answers Filters / Dashboard Prompts on
description
Database filters in terms of Key Columns Index / Partition pruning friendly SQL
SELECT
0 s_0, "Airlines Delay"."Carrier"."Carrier" s_1, DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier") s_2 FROM "Airlines Delay WHERE (DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier")='Q5') ORDER BY 1, 2 ASC NULLS LAST FETCH FIRST 100001 ROWS ONLY
48
OBIEE
Model Checker
Runs in Online mode Detects artifacts that can cause correctness issues Uniqueness of level keys Uniqueness of physical table keys that prevent BI to recommend/create
aggregates
Session Variables in LTS expression Session Variables in LTS content filter Generates warning Attributes not mapped to levels (impact on
aggregate navigation)
49
AGGREGATES
50 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
OBIEE
Aggregate Persistence
Ability to generate aggregate tables based on logical star and map back to the metadata repository Wizard to allow the administrator to identify measures and grains for aggregate tables
Output is a set of scripts to create and populate aggregate tables, as well as automatically map sources to the rpd
51
OBIEE
Aggregate Generation Script
create aggregates "ag_4183394923 (aggregate name) for "Airlines"."Flight Facts (logical fact table) ("Actual Elapsed Time (Min)","Air Time (Min)","Arr Lag (Min)",...) (measures)
at levels
("Airlines"."00 Time"."Time Year", "Airlines"."18 Route"."18 Route Detail", "Airlines"."11 Orig Airport Map"."11 Orig Division Map") (grain level combination)
52
Exalytics
Summary Advisor
Analyzes query workload over time Usage Tracking Summary Statistics table Cache hits are ignored. Recommends minimal set of aggregates to
53
Exalytics
OBIEE - Summary Advisor
Analyze facts
Eliminate the ones not suitable for
aggregation
Few facts at a time
Accelerating Dashboard ?
Select logical facts accessed
54
Exalytics
OBIEE - Summary Advisor
Analyze facts frequently queried
Workload sensitivity
Cumulative time because of lots of small queries or small number of time consuming queries May need to modify usage tracking tables manually to feed summary advisor right input
55 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Exalytics
Summary Advisor Stopping criteria
Default stopping criteria 0% Longer recommendation time Longer Aggregate build/refresh time
56
Exalytics
Summary Advisor Execute Aggregate Advisor
57
Exalytics
Summary Advisor
Update database statistics (Oracle) before running Summary Advisor Aggregate Recommendation depends on
Workload
Subject area, dashboard
No recommendation
58
OBIEE
Aggregate Navigation - Chronological keys
AGO/TODATE functions will be navigated to the
detail source for lack of ordering information, even when the aggregates are available
(Quarter, Sales) will hit the aggregates at
Time dimensions
59
OBIEE
Aggregate Navigation Measures w/ Dimension Specific Aggregation
Country<-Region<-State<-City<-Store Year<-Quarter<-Month<-Day
60
OBIEE
Aggregate Navigation Measures w/ Distinct Count aggregation
Count distinct aggregation
BI Server uses the most detailed grain possible Unless you register additional LTSes explicitly
Aggregate Persistence
Creates aggregates with the counts Not reusable except for Exact match with query grain
61
Exalytics
Aggregates other considerations
Aggregates on a different target ?
Consider expressing security in terms of BI logical columns, replace with BI security filter. Consider Role based security
LTS expressions:
More columns in aggregate table compared to base fact [ physical tables in the database]
Aggregate build/refresh longer, query performance faster
62
TIMESTEN
63 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Exalytics
TimesTen Data mart
Move Select Detail Data into TimesTen
Use ODI to replicate from warehouse tables Use GoldenGate to replicate for Operational reporting Use ttBulkCp utility to load from text file Data volume considerations
TimesTen in-memory database
Space for data (PermSize) 400 GB Space for query processing (TempSize) 400 GB
64
Exalytics
TimesTen Optimizing using native Data types
TimesTen implements native numeric types
more efficient in query processing and improved space usage
BI Server automatically maps the BI NQS data types to TimesTen native data
types
INT maps to TT_INTEGER, DOUBLE maps to BINARY_DOUBLE etc.
NUMBER (Max < 28) should be of type TT_TINYINT (1 Byte) NUMBER (Max < 215) should be of type TT_SMALLINT (2 Bytes)
65
Exalytics
TimesTen database tuning
Data load:
Disable Redo log and use fuzzy checkpoints for faster load
Exact statistics: Use the ttOptUpdateStats built-in procedure Statistical sampling: Use the ttOptEstimateStats built-in procedure
Indexing
Necessary for good query performance BI Server currently creates single-column indexes for aggregates stored in TimesTen Multi-column composite indexes can provide significant improvements for complex queries In case of replication, use the indexes on source table as reference Use TimesTen Index Advisor for recommendation on indexes, for specific SQL queries/workload (nqquery.log)
66
Exalytics
TimesTen database maintenance
ttBackup creates a backup copy of a database that can be restored using the
ttRestore utility
Full backup and incremental backup
for HA/DR
67
ESSBASE
68 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Exalytics
ESSBASE Block Storage - all things Parallel
Parallel Data Load
import database Sample.Basic using max_threads 12
Parallel Export
PAREXPORT -threads 4 -in e:\data\input.txt 1 1;
Default number of threads is specified using EXPORTTHREADS in essbase.cfg
Set CALCPARALLEL in essbase.cfg for parallelism and also set CALCTASKDIMS for the sparse dimensions to be included in parallel tasks
69
Exalytics
ESSBASE Block Storage - considerations
Multiple user concurrent queries and calculation script execution
Design the application to allow for large degree of parallelism in queries and calculation script
execution.
number of Essbase applications in different machines into a single box leading to efficient management.
Increase the memory foot print of index and block cache to take advantage of
Exalytics
ESSBASE Aggregate Storage - considerations
ASO scales linearly with the number of cores and memory
Configure the ASO application to run on the entire box and assign as much memory and threads as possible using essbase.cfg settings
Exalytics has 4 sockets, 40 cores and large amount of memory and this allows us to consolidate number of Essbase ASO applications in different machines into a single box leading to efficient management.
During consolidation, explicitly set the affinity to group of applications to a single socket so as to efficiently utilize
the memory associated with that socket.
A large single cube is possible with the memory and CPU power of Exalytics and this avoids having to consolidate the results from multiple cubes and re-aggregating and collating externally.
71
promotional pricing
Contact your Sales Consultant today to learn more.
72
73
74
75
Exalytics
Tip 3: HardwareAccelarator flag
<OracleHardwareAcceleration>true</OracleHardwareAcceleration>
OBIS (NQConfig.INI)
ORACLEHARDWAREACCELERATION = YES
76
OBIEE Architecture
77