0 оценок0% нашли этот документ полезным (0 голосов)
18 просмотров27 страниц
SAP Sybase IQ - Tips and Tricks for Optimizing Performance. This session will provide novices and experienced DBAs with information they may not know. We strongly suggest taking advantage of the education course offerings available to enhance your knowledge of IQ.
SAP Sybase IQ - Tips and Tricks for Optimizing Performance. This session will provide novices and experienced DBAs with information they may not know. We strongly suggest taking advantage of the education course offerings available to enhance your knowledge of IQ.
SAP Sybase IQ - Tips and Tricks for Optimizing Performance. This session will provide novices and experienced DBAs with information they may not know. We strongly suggest taking advantage of the education course offerings available to enhance your knowledge of IQ.
Mark Mumy IQ Technology Specialist SAP mark.mumy@sap.com 2 November 8, 2012 Disclaimer In this session we will be giving tips and guidelines for IQ to provide novices and experienced DBAs with information they may not know This is by no means exhaustive. We strongly suggest taking advantage of the education course offerings available to enhance your in-depth knowledge of IQ 3 November 8, 2012 Agenda Sizing Configuration Indexes Monitoring
4 November 8, 2012 Sizing 5 November 8, 2012 Memory Summary Operating System .5 to 1 GB RAM Filesystem Cache 20% of RAM All Other Applications IQ Catalog Memory -c/-cl/-ch parameters IQ Thread Memory stack size * thread count Load Memory (pre 15.2) per concurrent load Bitmap Memory per concurrent load IQ Main Cache 40% of remaining RAM IQ Temporary Cache 60% of remaining RAM Backup Memory per backup instance Monitor the IQ Dynamic Memory in sp_iqstatus Lists the current, and maximum RAM that IQ has allocated for all memory structures Swap recommendation is still 1x virtual memory Some customers run with 4- 8 GB I prefer to run with 25-100 GB depending on total RAM May seem excessive, but it can help avoid the dreaded all available virtual memory exhausted error! 6 November 8, 2012 Quick Sizing Reference RAM: 4-8 GB per core (prefer 8, settle for 4) RAM: Give IQ 75% of available RAM (assumes there are no other major consumers of RAM on the host)
Storage: Prefer RAID 10 for write intensive systems and temp store IQ can drive 50-500 MB/sec on and off disk per core in the multiplex so size disks accordingly, but ground the sizing in reality and service levels
MAIN Store disk: 2-5 drives per core on the host or in the entire multiplex TEMP Store disk: 2-5 drives per core on the host The number of drives does NOT include any overhead needed by RAID
MAIN Store Fiber Controllers/HBAs: 1 per 5-10 cores TEMP Store Fiber Controllers/HBAs: 1 per 5-10 cores
7 November 8, 2012 Sizing CPUs for Load For systems with 4 or fewer CPUs, expect to load roughly 10-20 GB of data per hour per CPU A 4 CPU system should be able to load about 40 GB of raw data per hour For systems with 8 or more CPUs, expect a load rate of 20-50 GB per hour per CPU An 8 CPU system should be able to load between 160 and 400 GB of raw data per hour Load times with this approach will vary greatly based on CPU count / speed and the number and types of indexes on the table being loaded For each BLOB or CLOB being loaded into IQ a single CPU will be necessary for maximum performance IQ 16 will change these ratios as we move to a 100% parallel load engine
8 November 8, 2012 Sizing CPUs for Queries On average, 1-2 CPU per active query more if queries are complex and can be run in parallel As IQ matures, we constantly push the engine to run more queries fully parallel This changes the sizing from how many cores per query to what is the service level we need to achieve and how many cores will that take Most queries are now run in parallel and can consume all CPU resources on the host IQ will blend single and multi-user query performance and rebalance resources as the workload changes As more queries appear on the run queue, the available resources will change and IQ will adjust accordingly
9 November 8, 2012 Sizing Memory Total system RAM should be based on 4-8 GB RAM per core Threads do not matter Allocate no more than 75% of total RAM to the main and temp caches Shared memory is not used by Sybase IQ except in small amounts for client/server communication where both are on the same host Typically, we want more temp cache for loading as the HG indexes are built in temp cache For queries the mix of main to temp will vary depending on a few factors: If there are temp tables, we want more temp cache If the queries contain a lot of ordering and grouping, this will use work tables based in temp cache Typically, we weigh main and temp cache at 50/50 or 40/60 split 10 November 8, 2012 Configuration 11 November 8, 2012 Creation Options Uppercase and lowercase? Can force IQ to treat all text data as CASE IGNORE or CASE RESPECT CASE RESPECT has slightly better performance but puts a burden on applications to compare data with the appropriate case Page size Default is 128k which is acceptable for most sites Increase the page size as the database and table sizes (rowcounts) increase Increasing the page size should be accompanied by an increase in RAM Collation and sort orders Can only be set at database creation If it needs to be changed the database must be rebuilt 12 November 8, 2012 Runtime Options (config file) -iqnumbercpus Should be set to match the physical core count, not the total processing threads that are on the host Caveat is IBM p-series where this can be tested and adjusted to a value between physical cores and total SMT threads -gm Total number of users that can connect Keep this setting low as memory is carved based on this value. This can impact query performance. -gn Should be set to 1.5x the gm setting (default is gm + 5) 13 November 8, 2012 Database Options (set options) FORCE_NO_SCROLL_CURSORS Should always be set to ON Very few applications require this to be OFF Can improve query performance Append_Load Can be used to improve load performance Will not reuse Row IDs or the space occupied by those Row IDs Great for systems where large, contiguous chunks of data are deleted
14 November 8, 2012 Indexes 15 November 8, 2012 Key Characteristics
Most columns will have at least one index
Index selection decisions based on column cardinality (number of unique values)
Multiple indexes used to resolve a query
Indexes are self maintaining
No optimizer statistics to update
Indexes are compressed
Index advisor demystifies index selection
TYPE USAGE Fast Projection Compressed raw data for result sets (Default) Low Fast Low cardinality data (up to 1000 unique values) High Non-Group Aggregation on the fly and range searches High Group Key fields and groupings for cross- tabular Date, Time, DT Date ranges, date part operations Multi-Column Concatenated indexes Word, Text Sophisticated key word or phrase string searches with boolean, ranking, proximity, fuzzy features Compare Column comparisons SAP Sybase IQ Index Types 16 November 8, 2012 Should We Index? Is the column searchable? Is the column a join column? Is the column a date/time/datetime datatype? Will the column be part of an aggregation? Will the column be part of a group by? 17 November 8, 2012 When To Use Indexes TYPE When to use it Fast Projection This is the default storage for a column. You can only influence the type of Fast Project via the IQ UNIQUE() syntax or Minimize_Storage option High Group Low cardinality data (over 2000 unique values) where the columns are used in a WHERE clause for searching or joins. Also include columned used in a GROUP BY. Low Fast Low cardinality data (up to 2000 unique values) where the columns are used in a WHERE clause for searching or joins High Non-Group Not used that much as the functionality has been implemented in other indexes (date, hg, lf) Date, Time, DTTM All date fields should have these indexes Word, Text Use only if you will be using the word or text search capabilities in IQ Compare Use when comparing 2 columns in the same table. Multi-Column Usually used for primary keys and for columns in a GROUP BY. Make sure that every column has an LF or HG as well! 18 November 8, 2012 Monitoring 19 November 8, 2012 Monitoring Hardware and storage utilities iostat, vmstat, top, glance, gpm, sar, etc. Use tools to monitor the hardware and storage for any potential bottlenecks outside of IQ IQ utilities sp_iqsysmon Sybase Control Center 3 rd party tools like Bradmark Surveillance DBA and White Sands ProActive DBA
20 November 8, 2012 Monitoring via sp_iqsysmon sp_iqsysmon is a great tool to consolidate the individual monitoring!! SAP Sybase IQ Monitor offers a series of views of the counters to showing differing aspects of the server and buffer cache workload A consolidated interface for the iq utilities command that is much easier to use and provides a way to run multiple reports with a single command
21 November 8, 2012 Monitoring via Sybase Control Center Sybase Control Center Sybase Central is being deprecated in version 16 Is a separate download via the product download center Leverages a lightweight client (browser) with a server component so that users need not install software Graphical interface that presents the data in a way that highlights issues visually Most counters that are in sp_iqsysmon are available in Sybase Control Center
22 November 8, 2012 Monitoring Guidance Use sp_iqsysmon or Sybase Control Center when issues arise and active monitoring is needed Frequency and key items Run at 10-15 second intervals for the duration of the issue (sp_iqsysmon), or actively via Sybase Control Center Thread use/starvation Cache usage Cache hit rates Dirty page grabs Page rereads Physical and logical i/os
23 November 8, 2012 Query Monitoring Leverage HTML query plans! Query plan settings (off by default) to provide optimal query information to DBAs and engineering set temporary option query_plan='off'; set temporary option query_plan_as_html='on'; set temporary option query_plan_as_html_directory=qplans'; set temporary option query_plan_after_run='on'; set temporary option query_timing='on'; set temporary option query_detail='on'; set temporary option DML_options10='on'; set temporary option Query_Name = Query Name Should not be set globally as the Sybase IQ MSG file or query plan directory will grow rapidly No need to set both Query_Plan and Query_Plan_As_HTML Query_Plan is on by default, but should be turned off immediately to keep the IQ message file output to a minimum This is much more difficult to read than the HTML plans and is note widely used 24 November 8, 2012 Index Monitoring Index advice and monitoring is off by default Turn on the index advisor globally and keep 1000 rows of advice set option public.index_advisor='on'; set option public.index_advisor_max_rows=1000; Will output data to HTML and text based query plans, if on Will output advice to an in-memory structure that can be accessed via sp_iqindexadvice() Captures the advised index type, where it was used, the last time the advice was generated, and the number of times this piece of advice was made The advisor will recommend any index that will cost less This could be 1 ms or 1 i/o less Be smart about the advice and whether or not you want to implement it
25 November 8, 2012 Troubleshooting Advice Determine where the issue is External to IQ Monitor the OS, hardware, and storage for any bottlenecks or issues Look for high CPU use, high CPU system time, low CPU user time, high wait time Look for I/O service times that are more than 10 ms Internal to IQ Enable the index advisor and look for missing indexes (this is usually the culprit) Enable HTML query plans and the index advisor if the issue is with a single query If the overall health is in question, leverage sp_iqsysmon at 30-60 intervals for a few days Sysmon output can then be parsed looking for the key items previously discussed 26 November 8, 2012 27 November 8, 2012 Useful links SAP Sybase IQ Users Group http://iqug.dssolutions.com/ Send an email to iqug-subscribe@dssolutions.com - follow the info in the reply email to finalize subscription SAP Sybase Education Curriculum https://training.sap.com/us/en/courses-and-curricula/sybase Sybooks IQ 15.4 Documentation http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infoc enter.help.iq.15.4/title.htm SAP Sybase IQ Sizing Guide http://www.sybase.com/detail?id=1093493