Академический Документы
Профессиональный Документы
Культура Документы
QueryProcessing&Optimization
TECHNICALWHITEPAPER
www.sybase.com
Contents
ExecutiveSummary...................................................................................................................................3
HistoricalOverview................................................................................................................................................3
What'sNewwiththeASE15OptimizerandQueryProcessingEngine.................................................................4
UnderstandingASE15'sQueryProcessing...............................................................................................5
Whatisanoptimizationgoal?...............................................................................................................................5
Optimizationcriteria..............................................................................................................................................6
Choosingthebestoptimizationgoalforyourapplication....................................................................................6
Experimentingwithoptimizationgoals.................................................................................................................7
Howtosetsessionleveloptimizationgoals..........................................................................................................8
ParallelqueryprocessinginASE15.......................................................................................................................8
Performancemonitoringtips................................................................................................................................9
Troubleshootingtips..............................................................................................................................................9
Using'CompatibilityMode'inASE15.0.3ESD#1................................................................................................12
Ifallelsefails.....................................................................................................................................................12
ObsoleteoptimizationcommandsinASE15.......................................................................................................13
ResourceRecommendationsforASE15.................................................................................................14
Procedurecache..................................................................................................................................................14
Procedurecacheusagelimitationin15.0.2ESD#2.............................................................................................14
OtherresourceusageaspectsofASE15.............................................................................................................15
StatisticsInASE15..................................................................................................................................16
WhystatisticsmatterespeciallyinASE15........................................................................................................16
Recommendation:runupdateindexstatistics....................................................................................................16
Using'sampling'withupdateindexstatistics......................................................................................................16
Speedingupupdatestatisticswithparallelism...................................................................................................17
Whentorunupdatestatistics?...........................................................................................................................17
Howmanyhistogramsteps?...............................................................................................................................18
Identifyingmissingstatistics................................................................................................................................20
OldstatisticsandupgradingtoASE15................................................................................................................20
Summaryofrecommendationsforstatistics.......................................................................................................21
RecommendedTestingBeforeUpgradingToASE15.............................................................................22
Whyshouldyoutest?..........................................................................................................................................22
Pleasenote.......................................................................................................................................................23
UsingCompatibilityMode...................................................................................................................................23
Identifyingqueriesthatrunslower.....................................................................................................................23
AnalyzingperformancedifferencesbetweenASE12.xandASE15....................................................................23
Analyzingshortdurationqueries........................................................................................................................26
Serverlevelperformanceaspects.......................................................................................................................26
ExampleOfASE15QueryPlanAndLavaTree........................................................................................27
ASE15QueryPlans..............................................................................................................................................27
Example:setshowplan........................................................................................................................................27
Example:plancostandLavaTree........................................................................................................................30
CapturingApplicationSQL.......................................................................................................................32
SybaseASE15BestPractices:QueryProcessing&Optimization
1
Version1.1May2009
Auditing...............................................................................................................................................................32
MDAtables(12.5.0.3)..........................................................................................................................................32
ApplicationTracing(15.0.2).................................................................................................................................33
AbstractPlanCapture/QueryMetricsCapture...................................................................................................34
InformationToCaptureBeforeContactingSybaseTechSupport...........................................................35
701Errors............................................................................................................................................................35
PerformanceProblemwithaLimitedNumberofQueries..................................................................................36
SystemWidePerformanceProblemsorHighCPUUsage:step1.......................................................................37
SystemWidePerformanceProblemsorHighCPUUsage:step2.......................................................................38
UploadingdiagnosticstoSybaseTechSupportthroughFTP...............................................................................39
ConclusionsAndRecommendations.......................................................................................................40
ReferenceDocuments.............................................................................................................................41
Principalauthor
RobVerschoor
Contributingauthors
SudiptoChowdhuri
BillCox
MarkKusma
NitinSadalgekar
PeterThawley
RaymundoTorres
NingzhenZhu
RevisionHistory
Version1.0 February2008 Firstversion
Version1.1 May2009 CoverCompatibilityModein15.0.3ESD#1;also,minorediting.
SybaseASE15BestPractices:QueryProcessing&Optimization
2
Version1.1May2009
ExecutiveSummary
HistoricalOverview
WhenSybasefirstbegantodesignitsrelationaldatabasemanagementsystemin1984,theworldofInformation
Technology(IT)wasradicallydifferentapplicationsweremonolithicwithdataaccesscodeandbusinesslogic
tightlybound;transactionprocessing(OLTP)waspredominatelyabatchprocess;anddecisionsupport(DSS)
reportingnever,ever,ranatthesametimeastransactionprocessing.Sybase'spioneeringconceptofclient/server
computingwasbasedontwosimpleideas.First,manybusinessescouldofferbetterserviceandvaluetotheir
customersiftransactionprocessingweredone"online".Second,businessescouldservicetheircustomersbetter,
andthereforegrowfaster,bydecentralizingapplicationdevelopmentandnotusingthemainframe.Toenablethis
radicalapproach,Sybasehadtodesignandbuildasubstantiallydifferenttypeofdatabasekerneltosupportthis
new"onlineenterprise".
SybaseAdaptiveServerEnterprise(i.e.,theproductformerlyknownastheSybaseSQLServer)becametheleading
RDBMSproductforhighperformancetransactionprocessingsupportinglargenumbersofclientapplication
connectionsonrelativelysmall,inexpensivehardware.Tobesuccessfulinthisdemandingarea,Sybasefocused
heavilytooptimizeandstreamlineallthelayersofourdatabaseenginefortransactionprocessingworkloads.
WhilethisfocusturnedouttobetheassetthatcatapultedSybasetoadominantpositioninhighendOLTP
applicationsduringthe1990's,itincreasinglybecamealiabilityinsomeenvironmentsasapplicationworkloads
havechangedandhardwareperformancehasincreased.
Withmoreandmoreenterprisesrequiringamixtureofbothtransactionprocessingandoperationaldecision
supportreportingonthesamedataatthesametime,itbecameclearthatSybaseneededtorearchitectitsquery
processinglayerforthefuture.Sybasebegantheprocessinthe11.9.2releasewhenwerearchitectedthe
"bottomhalf"ofthequeryoptimizerthatdealtwiththestatisticsandalgorithmsthatdrivequerycosting.We
thenembarkeduponasignificantefforttoreplacethetophalfoftheoptimizerandthequeryexecutionlayerwith
onethatcouldmeetthenextgenerationofapplicationrequirements.Equallyimportant,weneededittobemore
extensiblesothatinthefuture,wecouldaddsupportfornewindextypesornewjoinstrategiestomorequickly
respondtotheneedsofourcustomers.WithASE15,Sybasehasdeliveredthis.
Withthishistoricalcontextinmind,itisimportantforcustomersandpartnerswhoarelookingtoleveragethe
manybenefitsofASE15.0.xunderstandthesignificanceofthesechanges.Agoodanalogyforhowtoapproach
thiseffortisjustlikebreakingintheengineonyournewcar.Shouldyouwinduptheengine'sRPMstothatred
lineandcruiseyourfavoritewindingbackroadat100mph?Asmuchassomeofuswouldliketosayyes,Isuspect
mostofusknowtherightanswer.Well,usingthenewASE15queryoptimizerandqueryexecutionissimilargo
slowerthanyou(oryourboss)wants,learntheimportantcharacteristicsofthesystem,andknowwhattodoin
casetheredlightcomeson.Thisdocumentismeanttohelpyouwiththeselasttwo!
SybaseASE15BestPractices:QueryProcessing&Optimization
3
Version1.1May2009
What'sNewwiththeASE15OptimizerandQueryProcessingEngine
ThefirstandmostimportantthingtoknowaboutASE15'squeryoptimizerandqueryexecutionengineisthatit's
basicallytotallynew.Theshiftinrequirementsovertheyearstowardsmixedworkloads(i.e.,concurrentOLTPand
DSSapplicationprofiles)stretchedourtraditionalrelationalprocessingmethodsof"nestedloop"joinstotheir
breakingpoint.Ifyouconsiderthefactthatitisnowcommontoseequeriesjoiningten'softableswithmany
complexsearchargumentsand/oraggregation,itwasclearthatboththequeryoptimizerandthequeryexecution
engineneededamajoroverhaul.Moreimportantly,weneededtodesignasolutiontoensureboththeoptimizer
andexecutionenginewouldbeextensible.Thisallowsustoaddnewandinnovativealgorithms(forjoining,
grouping,union,etc)andstoragestructures(e.g.indextypes)moreeasilywithoutkludgingexistingcodeand
makingitunmanageableorerrorprone.Tofacilitatethemodularityandextensibilityofthequeryexecution
engine,wehaverebuilttheexecutionengineusingamoremodern,consumerdriven,iteratorbasedexecution
model.Thismodelimplementsrelationaloperatorsasprimitivebuildingblocksandprovidesboththeabilityto
easilysolvenagginglimitationsaswellasaddimportantnewjoinstrategiesandoptimizations.Beforewegetinto
thedetailsofhowtouseandtroubleshootthenewoptimizerandqueryprocessingengine,let'sfirstgainan
understandingofthemajorchangeswhichaffectbothapplicationsandoperations.
LongtimeASEcustomerswillappreciatetheeliminationoflimitationswhichallofushavesufferedthrough.Over
theyears,Sybasehassteadilybeenreducingtheoccurrenceswhendatatypemismatcheswouldpreventa
predicatefrombeingconsideredduringoptimization.Unfortunately,oureffortswere"hardwired"sowehadto
addspecialcasecodeforeachandeverycase.Althoughnot"rocketscience",itwasaresourceintensive
approachthatdidnotscaleasweaddednewdatatypes.Moreinterestingly,howmanytimeshaveyouwished
theoptimizerandQPenginecouldusemultipleindexesonatablethroughindex"union"or"intersection"
operations?Well,wishnomore!Now,querieswithANDand/orORpredicatescanpotentiallyusemultiple
differentindexesbasedondatadistributionstatistics'estimatedselectivityforeachpredicate.
Howaboutthedilemmawefacedinthepastwithoptimizationofqueriesthatjoinedlargenumbersoftables
resultedina"lessthanstellar"queryplanduetojoincostingthatdidnotuserealdatadistributionstatistics
(histograms)butrather"magicnumbers"fromourengineer'sheads?Now,statisticsforjoins,bothtoderivejoin
orderaswellastofeedmoreaccuraterowestimatesintocostingofinnertablesofthejoinorder,shouldbemore
accuratebecausewedynamicallycreatejoinhistogramsatoptimizationtime.
Whyisallthisimportant?Well,franklyspeaking,it'stoensurethatyoudon'ttreatASE15as"justanother
release".Asmuchaswewouldliketosaythatyoucouldsimplyupgradeandpointyourapplicationsatthe
upgradedservers,thedepthandbreadthofchangeinoneofthemostfundamentalareasofadatabase,query
execution,necessitatesamorefocusedtestingregimen.Thispaperismeanttoprovideyouwiththeclearfacts
andbestpracticestoreducethiseffortasmuchaspracticallypossible.
SybaseASE15BestPractices:QueryProcessing&Optimization
4
Version1.1May2009
UnderstandingASE15'sQueryProcessing
Whatisanoptimizationgoal?
AcentralconceptofASE15'snewqueryprocessingengineisthe'optimizationgoal'.Inessence,thiscanbeseen
asahinttotheASEqueryoptimizer,providinganindicationofthenatureofthequerybeingoptimized.To
illustratetheunderlyingidea,atypicalOLTPqueryandatypicalDSSquerywillnormallyendupwithverydifferent
queryplansduetothedifferentdataaccesspatternsusedbythesequeries.OLTPqueriestendtohitoneorjust
veryfewrows,andjoinonlyafewwellindexedtables;DSStypicallyhitsmanyrowsbutreturnsjustafew,andcan
joinalargenumberoftables.
Becauseoftheirdifferentaccesspatterns,OLTPquerieswilloftenperformbestwhentheyuseaclassicnested
loopjoinwhereasaDSSqueryismorelikelytorunfasterwithahashjoin.ByindicatingthataqueryisforOLTPor
DSSpurposes,theoptimizermaysaveitselfalotofwork(time,memory,CPU)whengeneratingaqueryplan.
ASE15providesthreeoptimizationgoals(orderedfrom'narrow'to'wide',correspondingtothenumberofoptions
andstrategiestheyallowtheoptimizertoconsider):
allrows_oltp (bestforOLTPqueries)
allrows_mix (defaultafterupgradingtoASE15)
allrows_dss (bestforDSSqueries)
ThedifferentoptimizationgoalshavethefollowingeffectontheASEqueryoptimizer:
allrows_oltpoffersthenarrowestselectionofjoinmethods:thequeryoptimizerisallowedtoconsider
nestedloopjoinsonly
allrows_mixallowsalsomergejoinstobeconsideredbytheoptimizer,aswellasparallelplans(iftheASE
serverisconfiguredforparallelism).
allrows_dssoffersthewidestselectionofjoinmethods:alsohashjoinsmaybeconsideredbythe
optimizer
Forallrows_mixandallrows_dss,additionallowlevelprocessingalgorithmsareenabledwhicharedisabledfor
allrows_oltp.
Itshouldbenotedthatwhentheoptimizationgoaliswidened,thequeryoptimizermightusesignificantlymore
resources(timeandprocedurecache)togenerateaqueryplan.Iftheoptimizergeneratesthesamequeryplan,
withonlynestedloopjoins,underallrows_dssandallrows_oltp,youshouldexpecttheoptimizationunder
allrows_dsstotakemoretimeandprocedurecachethanunderallrows_oltp.
TheoptimizationgoalcanbedefinedontheleveloftheASEserver,sessionorindividualquery:
-- server-wide default:
sp_configure 'optimization goal', 0, 'allrows_dss'
SybaseASE15BestPractices:QueryProcessing&Optimization
5
Version1.1May2009
plan '(use optgoal allrows_dss)'
Thesessionleveloptimizationgoalcanalsobesetinalogintrigger;seepage8formoreinformation.
Optimizationcriteria
Anoptimizationgoalisactuallyshorthandforacollectionofon/offsettingsforaseriesofpropertiesknownas
'optimizationcriteria'.Individually,anoptimizationcriterionallows(whenitisenabled)ordisallows(when
disabled)theoptimizertoconsideraparticularalgorithm(asforaccessmethods,forjoins,forgrouping,sorting,
etc.).Someexamplesofoptimizationcriteriaare:
NotethatSybasedoesnotrecommendusingexplicitsettingsforoptimizationcriteriaunlessadvisedbySybase
TechnicalSupport,oraspartofatroubleshootingprocess.Inproductioncode,Sybaserecommendsusing
optimizationgoalsinsteadunlessadviseddifferentlybyTechnicalSupport.
Choosingthebestoptimizationgoalforyourapplication
Thechoiceofoptimizationgoalcanhaveabigimpactontheperformanceofyourqueries.Therefore,two
importantquestionsmustbeansweredwhenupgradingtoASE15:
1. Whichoptimizationgoalshouldbesetserverwide?
2. Isitnecessarytospecifysessionorqueryleveloptimizationgoalsthataredifferentfromtheserverwide
setting?
Sinceitshouldbeconsideredunfeasibletoanalyzeallqueriesinyoursystem,apragmaticapproachshouldbe
takentowardstheseissues.Pleasenotethatblindlysettingtheserverwideoptimizationgoaltoallrows_dss
(sinceitisthewidestoptimizationgoal)isnotrecommendsincethismayconsumemoreresources,including
compilationtime,thanneeded.Itisworthtryingtodetermineifanarroweroptimizationgoalwilldo.
Asforthechoiceoftheserverwideoptimizationgoal,twoapproachescanbedistinguished:
A. Picktheoptimizationgoalthatprovidesthebestoverallperformanceforyoursystem.
Thisshouldbesetsothatitmatchesthetypeofworkloadofthemajorityofyourqueries.Ifyouhavea
pureOLTPorpureDSSsystem,thenitisobviouswhichoptimizationgoaltopick.
Ideally,youshouldexperimentwiththedifferentoptimizationgoalstoseewhichoneprovidesbest
resultsforyourmixofapplicationsandqueries.Seethesection"Experimentingwithoptimizationgoals"
forsomesuggestions.
B. Justsettheserverwideoptimizationgoaltoallrows_oltpforthesimplereasonthatthis
optimizationgoalresemblesthebehaviourofASE12.xmostclosely.Thiswillreducethepossibilityof
runningintounexpectedqueryplanissuesafterupgradingtoASE15,butmayalsonotletyoubenefit
fromthefullpotentialofASE15'scapabilities.
Thesecondissue(theneedforsessionorqueryspecificoptimizationgoalsettings)mayrequiremorework.
Whenyouknowthatcertainapplicationshavedifferentworkloadcharacteristicsthantherestofyoursystem,for
example,someOLTPworkloadinamostlyDSSsystem,thenitmaybeagoodideatosettheappropriatesession
leveloptimizationgoalforthatapplication.Inthisexample,thatwouldmeantheserverwideoptimizationgoal
wouldbesettoallrows_dsswhilethesessionleveloptimizationgoalwouldbesettoallrows_oltp.
SybaseASE15BestPractices:QueryProcessing&Optimization
6
Version1.1May2009
Notethatthiscanquicklybecomeanopenendedundertaking:thereisalmostnolimitontheamountofeffort
youcanspendonanalyzingyourqueriesandderivingthebestoptimizationgoalsettingforasessionoran
individualquery.Asalways,youhavetostrikeabalancebetweenperformancegainsandeffortspentwhen
engaginginperformancetuning.
Experimentingwithoptimizationgoals
Asexplainedabove,youneedtopickanoptimizationgoaltosetasyourserverwidedefault.Theeasiestapproach
isjusttopickone,seehowthingsbehaveoverall,andtryanotheronetoseeifthat'sanybetter.Whenusing
storedprocedures,makesuretorunsp_recompileonalltables,orrebootASE,afterchangingtheserverwide
optimizationgoalinordertomakeittakeeffectbyrecompilingexistingplans.Also,makesurethestatement
cacheisdisabledtogetagoodcomparison.
Whilethisapproachmaywork,itisperhapsagoodideatomakeamoreinformeddecisionabouttheoptimization
goaltouse.Fortunately,thisisnottoodifficult.First,youneedtocollectorcapturearepresentativesetofreal
lifeSQLqueries,forexample,throughtheMDAtables,orwithauditingorMonitorServer(forsomesuggestions,
seepage32).Putallthesequeriesinafileasseparatebatches,with'go'toterminateeverybatch,justasyou'ddo
whenusingisql.Then,putthefollowingatthestartofthefile:
Nowrunthisfilethroughisqlandcapturetheoutputinanotherfile:
Timehowlongittakesbeforetherunhascompleted.
Repeatthiswiththeoptimizationgoalabovechangedtoallrows_mixandallrows_dss,sothatyouhave
threeoutputfilesofqueryplanoutput.
Fromthecompletiontimesofthedifferentruns,itmaybeimmediatelyapparentwhichoptimizationgoalprovides
thebestresultoverall,andthatmightbeacandidatefortheserverwideoptimizationgoalsetting.
However,itwillbeinterestingtocounthowmanytimeseachjointypehasbeenchosen.Thiscanbedonequickly
withgrepforeachoftheoutputfiles:
Supposethereappeartobejustafewhashjoinsusedwhentheoptimizationgoalwassettoallrows_dss.For
thequeriesusingahashjoin,itwouldbeworthcheckingiftheyranmuchslowerwiththeotheroptimization
goals.Withtheapproachabove,thiswillberelativelyeasytocomparesinceset statistics timewas
enabled,theelapsedtimeforeachquerywillbeintheoutputfile.Ifitturnsoutthat,forthosespecificqueries,
SybaseASE15BestPractices:QueryProcessing&Optimization
7
Version1.1May2009
hashjoinsmakeabigdifferenceindeed,youcouldconsidersettingthesessionspecific,orevenqueryspecific,
optimizationgoalforthosequeriestoallrows_dss.If,ontheotherhand,thereappeartobemanyqueries
usingahashjoin,andallrows_dssturnsouttoprovideverygoodperformanceoverall,perhapsitisbettertoset
theserverwidegoaltoallrows_dssinstead.
Withthisapproach,itisrelativelyeasytogetanideaoftheimpactofusingdifferentoptimizationgoals.Ofcourse,
thequalityoftheresultsdependsonhowrepresentativethesampledqueriesare.
Whenrunningtestslikethese,makesurethestatementcacheisdisabled(eitherbysettingtheconfiguration
parameter"statement cache size" to0,orbyrunningset statement_cache offatthestartof
yoursession).Also,whenusingstoredproceduresbeforererunningwithadifferentoptimizationgoal,run
sp_recompileonalltables,orrebootASE,afterchangingtheserverwideoptimizationgoalsoastomakesure
thenewgoaltakeseffect.
Whencomparingperformanceofqueriesunderdifferentoptimizationgoals,especiallywhenarebootisinvolved,
makesuretotestundercomparablesituationswithrespecttodatacacheusage.Itdoesnotmakesenseto
compareaquerywheremostdataneedstobereadfromdisk,withaquerywheremostdataisalreadyinthedata
cache.Ingeneral,itisbesttocomparequerieswitha'warmcache',wherebythequeryisrunonceortwice
withoutmeasuringitsperformance,followedbythefinalmeasurementrun.
Howtosetsessionleveloptimizationgoals
Let'sassumeyouhavedeterminedthataparticularapplicationwouldbenefitfromusingallrows_mixwhile
theserverwidegoalissettoallrows_oltp.Howdoyouimplementthis?Initssimplestform,youmodifythe
applicationtomakeitexecutethestatementset plan optgoal allrows_mixbeforesubmittingany
otherstatements.However,itmaynotbepossible,orimpractical,tomodifytheSQLcodesubmittedbyan
existingapplication.
Fortunately,thereisaneasysolution:youcansetthesessionleveloptimizationgoalinalogintriggerwithset
plan optgoal.Thisway,theoptimizationgoalforasessioncanbesetasdesired,withoutchangingany
existingapplicationcode.PleaserefertotheASESystemAdministrationGuideformoreinformationaboutlogin
triggers:chapter"ManagingUserPermissions",section"Rowlevelaccesscontrol",subsection"Usinglogin
triggers"(thoughpleasenotethatlogintriggerscanbeandusuallyareusedindependentlyofrowlevelaccess
control,asisindeedthecaseforsettingsessionleveloptimizationgoalasdescribedabove).
ParallelqueryprocessinginASE15
Sinceversion11.5,ASEhassupportedintraqueryparallelism,wherebyasinglequeryisprocessedbymultiple
workerprocesses.ManySybasecustomershaveusedparallelism,usuallytoimproveresponsetimesforDSStype
queries.Forthistypeofquery,wherealargenumberofrowsareaccessedbutonlyasmallresultsetisreturned,
parallelismtendstobemostefficientinpre15.0.Parallelqueryprocessingtendstorequiresignificantlymore
resources(CPU,memory,diskI/O)thanserialprocessing,butwhensuchresourcesareavailableparallelismmay
deliverbetterperformanceforspecificqueries.
ASE15alsosupportsparallelism,butinadifferentwaythanpre15versions.Withoutgoingintotoomanydetails
thatwouldexceedthescopeofthisdocument,insummarywecansaythatparallelisminASE15canoccurat
morelevelsinsideaquerythaninpre15.Inaddition,parallelisminASE15isdesignedforuseinconjunctionwith
semanticallypartitionedtables,whichpre15didnotsupport.
SincethenewqueryprocessingfeaturesofASE15offerpotentialperformancebenefitsforDSStypequeriesin
particular,SybaserecommendstoinitiallynotuseparallelprocessingwhenupgradingtoASE15,evenwhen
parallelismwasusedinASE12.x.Sinceserialprocessingismoreresourceefficientthanparallelprocessing,
avoidingparallelismmayallowyoutodeliverbetteroverallperformancewiththesamehardware.Itmaywellbe,
andhasindeedbeenobservedbycustomers,thatASE15inserialmoderunsqueriesfasterthanASE12.xwith
SybaseASE15BestPractices:QueryProcessing&Optimization
8
Version1.1May2009
parallelism.Forthesereasons,bestexploreASE15'scapabilitiesinserialmodefirstbeforelookinginto
parallelism.ForDSStypequeries,whereparallelismwasoftendeployedinASE12.x,itisspecificallysuggestedto
lookintotheeffectsoftheoptimizationgoalallrows_dss.
Parallelismmaydeliverbetterresponsetimesthanserialprocessingforqueriesusingsemantictablepartitioningin
ASE15,orforDDLcommands(likecreate index).Forsuchcases,itmaybeworthwhiletoexploretheimpact
ofparallelisminASE15.
Performancemonitoringtips
Thissectioncontainssomesuggestionsformonitoringperformanceaspectsofyourqueries.Theserequirethe
MDAtablestobeinstalledandenabled.
TodetermineifanysessionsarecurrentlyexecutingqueriesthatconsumealotofdiskI/O,use
sp_monitor:
select top 20
SPID, CpuTime, LogicalReads, PhysicalReads,
ProcName=object_name(ProcedureID,DBID),
MemUsageKB, StartTime, EndTime,
DurationSecs = datediff(ss, StartTime, EndTime),
KPID, BatchID
from master..monSysStatement
order by LogicalReads desc
Tofindthetop20queriesforCPUconsumption,changetheorderbyclausetosortonCpuTimeinstead.
Tofindthetop20longrunningqueries,sortontheduration("order by 9 desc").
TofindtheSQLtextcorrespondingtothesestatements,usetheKPIDandBatchIDcolumnstoquery
themonSysSQLTexttable.NotethatmonSysSQLText,likemonSysStatement,isasocalled
"historical"MDAtable,meaningthatitcontainsonlyalimitedamountofhistory.Tocaptureinformation
fromthesetablesoverlongerperiodsoftime,youshouldfrequentlyselectfromtheseMDAtablesand
storetheresultsetinapermanenttable.
Troubleshootingtips
ASE15wasdesignedtoofferagreatlyimprovedqueryprocessingenvironment.Intherareeventthatqueryplans
orqueryperformancearenotwhatyouexpect,herearesomethingstotrytoisolatetheproblem.
Whenexperimentingwithdifferentoptimizationgoals,makesurenocachedplansareused:changingthe
sessionlevelorserverwideoptimizationgoalwillnotcausecachedplanstoberecompiled.
Forstoredprocedures,eitherexecutethemwith recompile,orrunsp_recompileononeofthe
SybaseASE15BestPractices:QueryProcessing&Optimization
9
Version1.1May2009
tablesbeingaccessed.Forbatches,makesurethestatementcacheisdisabledbyrunningset
statement_cache offfirst.
Whenusingdifferentsessionleveloptimizationgoalsinasinglebatchorstoredprocedure(i.e.set
plan optgoalstatements),ensuretoberunning15.0.2ESD#2orlater.Thisisrecommendedbecause
someaspectsofsessionleveloptimizationgoalswerechangedinthatrelease,makingthefeaturemore
intuitiveandeasiertouse.
Fulldetailsonthescopeandsemanticsofsessionleveloptimizationsettings,willbedescribedinanother
whitepaper,titles"ChangestoscopeandsemanticsofsessionleveloptimizationsettingsinASE15.0.2"
(seeURLonpage41).
Whenyouwanttoguaranteethatastoredprocedureisalwaysoptimizedwithaparticularoptimization
goal,irrespectiveofanyserverwideorsessionlevelsettings,putset plan optgoal
allrows_xxxasthefirststatementinthebodyofthestoredprocedure.ThisrequiresASEversion
15.0.2ESD#2orlater.
Ifitissuspectedthatmergejoinsarecausingsuboptimalperformanceinyoursystem,andaserverwide
optimizationgoalisallrows_dss,youcouldchangethissettingtoallrows_oltp.However,this
alsoimpliesthathashjoinscannolongerbeused.
In15.0.2orlater,ifyoudon'twanttolosethepossibilitytousehashjoins,youcanalsokeep
allrows_dssastheserverwidegoal,andsettheconfigurationparameter'enable merge join'
to0;thiswilldisablemergejoinsserverwideforalloptimizationgoals.
IfyourSQLcodefromASE12.xcontainsexplicitlyforcedjoinorders(withset forceplan),these
forcingsshouldideallybereevaluatedwhenupgradingtoASE15(asSybasehasalwaysrecommended
whentalkingaboutsuchforcings).Ofcourse,ifyoursystemrunstofullsatisfactioninASE15,theremay
benoneedtochangeexistingforcings.However,theymayalsostopyoufromtakingfullbenefitofthe
capabilitiesofASE15.Itisthereforerecommendedtoevaluatetheimpactofremovingtheseforcings.If
thisisimpractical,in15.0.1ESD#2orlater,traceflag15307canbeenabledtonullifytheeffectofanyset
forceplanstatementswhenqueryplansarecompiled.
Likewise,anyexplicitforcingsforindexesorforprefetch/parallelism/bufferreplacementstrategycanbe
nullifiedbyenablingtraceflag15308.
Traceflags15307and15308canbesetatboottime.Theycanalsobeenabledwithdbcc traceon,but
notethattheireffectisserverwide,soallsessionsareaffected.
Notethatbothtraceflagsdonotaffectanyqueryplanpropertiesdefinedbyabstractqueryplans.
Ifyoursystemseemstobeconsuminganunreasonableamountofspaceintempdb,youcanusethe
MDAtablestoseeifoneparticularsessionconsumesalotofspaceinaworktable.
AssumingtheMDAtablesareenabled,runthefollowingquery:
select SPID, DBName, ObjectName, PartitionSize
from master..monProcessObject
where DBID = tempdb_id(SPID)
order by SPID
LookforsessionsthathavealargevalueforPartitionSize.WorktableshaveanObjectNameof
'temp worktable'.Byqueryingmaster..monProcessSQLTextand/or
master..monProcessStatementyoucanfindthecorrespondingSQLstatementforthe
correspondingsessions.Tostopsessionsfromfillinguptempdb,andthusaffectingothersessionsalso
requiringtempdbspace,youcancreatearesourcelimitoftype'tempdb_space'.Anotherpossible
SybaseASE15BestPractices:QueryProcessing&Optimization
10
Version1.1May2009
remedyiscreatingmultipletemporarydatabasesandassigningthesetospecificusers.SeetheASE
documentationforfurtherdetailsonthesefeatures.
Ifyouencounteraslowqueryandyoususpectthatreformattingiscausingthesuboptimalperformance,
firstfollowtherecommendationswithrespecttostatistics(seepage21).Ifthereformattingpersistsand
performanceremainsunsatisfactory,youcanverifywhetherdisablingreformattingimprovesthingsas
follows:
set store_index off -- disables reformatting
go
select -- your query
go
Or:
select -- your query
plan '(use store_index off)'
Beforeusingindividualoptimizationcriteriasettingsinproductioncode,Sybaserecommendsverifying
thissolutionwithTechnicalSupport.
Underallrows_dss,ifyouencounteranunreasonablyslowquery,orifthequeryrunsintoa701error
duringoptimization,firstfollowtherecommendationswithrespecttostatistics(seepage21).Iftheissues
persist,tryifthefollowingsolvestheproblem:
set bushy_space_search off -- relevant only under allrows_dss!
go
select -- your query
go
Or:
select -- your query
plan '(use bushy_space_search off)'
Beforeusingindividualoptimizationcriteriasettingsinproductioncode,Sybaserecommendsverifying
thissolutionwithTechnicalSupport.
WhenrunninglargenumbersofidenticalorverysimilarclientgeneratedSQLqueries(i.e.notstored
procedures,andalsonotinexecuteimmediate)wherethedifferenceliesinthesearchargument(i.e.a
differentprimarykeyvalue),andyou'rerunning15.0.1orlater,considerenablingthestatementcacheas
wellasliteralautoparameterization.Thesemaysignificantlyreducethetimeandresourcesspenton
queryoptimizationandthereforeimproveoverallperformance.
LiteralautoparameterizationisanenhancementtothestatementcacheinASE15.0.1orlater.Whenthe
statementcacheisenabled,aquery'splanwillbecachedsothatanexactlyidenticalquerydoesnotneed
tobecompiledagain,butthecachedplancanbeusedinstead,savingtimeandresources.Withliteral
autoparameterizationenabled,thiscachingisextendedtoalmostidenticalqueriesthatdifferonlyina
constantvalue.Forexample,takethesetwoqueries:
SybaseASE15BestPractices:QueryProcessing&Optimization
11
Version1.1May2009
select CustName from Customers where CustID = 123
select CustName from Customers where CustID = 456
Thesequeriesarenotidentical,buttheyarelikelytoendupwiththesamequeryplan.Enablingliteral
autoparameterizationhastheeffectthatthestatementcachefactorsouttheconstantvalueinthe
whereclauseandcachesaplanforallquerieslookinglikethis:
select CustName from Customers where CustID= <integer-constant>
Thismeansbothqueriesabovewouldusethesamecachedqueryplan. Literalautoparameterizationcan
greatlyimprovetheeffectivenessofthestatementcache,andthereforeitisrecommendedtoexperiment
withthisfeatureifyourapplicationsusemanyalmostidenticalSQLqueriesthatarenotpartofstored
proceduresandarenotruninexecuteimmediate.
Thestatementcacheisenabledseverwidewiththeconfigurationparameter'statement cache
size'.Onsessionlevel,thestatementcachecanbedisabledwithset statement_cache off.
Literalautoparameterizationisenabledserverwidewiththeconfigurationparameter'enable
literal autoparam',andonsessionlevelwithset literal_autoparam on(oroff).It
appliesonlywhenthestatementcacheisenabledtoo.
Using'CompatibilityMode'inASE15.0.3ESD#1
ForcustomerswhoprefertolimittherequiredtestingeffortforupgradingtoASE15,Sybasehasintroducedanew
featurenamed'CompatibilityMode'inASE15.0.3ESD#1.
CompatibilitymodeisaqueryprocessingenhancementallowingqualifyingTSQLqueriestobeprocessedwitha
methodofqueryoptimizationandqueryexecutionverysimilartothatusedinASE12.5,insteadofusingtheASE
15queryprocessingalgorithms.
Compatibilitymodeisnotanoptimizationgoal,butaqueryprocessingoptiononahigherlevelwhichdetermines
whether12.5likeoptimizerandqueryexecutionlogicwillbeused(whichexcludeallASE15features),orwhether
theASE15featuresareused;inthelattercase,theconsiderationsintherestofthisdocumentapply.
Compatibilitymodehasbeenmadeavailabletocustomerstoprovidemorecontrolovertheprecisemomentwhen
ASE15specificqueryprocessingfeaturesbecomeactiveintheirqueriesandapplications,thusallowinggradual
introductionofASE15queryprocessingaspectsaftermigratingtoASE15.Consequently,therequiredtesting
effortcanbebetterfocusedandplannedthanwhenmigratingtoASE15withoutusingcompatibilitymode.
Itshouldhoweverbenotedthat,asaconsequenceofusingcompatibilitymode,applicationsmaynotbeableto
fullybenefitfromthequeryprocessingenhancementsofASE15.Thisisaconsciouschoicethatcustomerswill
needtomakewhenusingcompatibilitymode.
Itisexpectedthatcustomers,afterhavingmigratedtoASE15,willeventuallydisablecompatibilitymodesoasto
takefulladvantageoftheperformanceimprovementsinASE15.
Forfulldetailsoncompatibilitymode,pleaserefertothewhitepaper"Using'CompatibilityMode'InASE15.0.3
ESD#1ForASE15Migration"(seeURLonpage41).
Ifallelsefails
IncasenoneofthesuggestionsinthecurrentdocumenthelpyoutoresolvequeryperformanceissuesinASE15,
SybaserecommendsthatyouupgradeyourASEinstallationtothelatestESDavailableforyourplatform.As
always,Sybaseismakingcontinuousproductimprovementsandyourissuemayalreadyhavebeenfixedinalater
ESDthanyouarecurrentlyrunning.Ifyourproblemstillpersists,youshouldopenacasewithSybaseTechnical
SybaseASE15BestPractices:QueryProcessing&Optimization
12
Version1.1May2009
Support.Pleaserefertopage35forguidanceabouttheinformationtocollectbeforecontactingTechnical
Support.
ObsoleteoptimizationcommandsinASE15
Variousoptimizationrelatedsettingsfrom12.xarenolongerwhenusingtheASE15queryprocessingfeatures.
AlthoughthefollowingcommandsstillexistinASE15,theydonotperformanyfunctionanymore,andsimply
returnasuccessstatus:
set sort_merge:inASE15,replacedbyset merge_join,optimizationgoalsandthe
configurationparameter'enable merge join'(in15.0.2)
set jtc:JoinTransitiveClosureisalwaysenabledinASE15
set table count:nolongerappliesinASE15
Inaddition,thefollowingfeaturesnolongerapplyinASE15:
configurationparameter'enable sort-merge join and JTC':replacedbyoptimizationgoals
andbytheconfigurationparameter'enable merge join'(in15.0.2).
boottimetraceflags334and384,whichenabledmergejoinsandJTCindividually,nolongerhaveany
functioninASE15.
However,notethatwhenusingcompatibilitymode(seepage12),thenthesecommandsandfeatureswillstill
functionasinpre15.0forthosestatementsthatareprocessedinfullcompatibilitymodeorrestricted
compatibilitymode.ForstatementsthatareprocessedusingASE15mode,thesecommandsarenoopsas
describedabove.
SybaseASE15BestPractices:QueryProcessing&Optimization
13
Version1.1May2009
ResourceRecommendationsforASE15
Procedurecache
OneoftheconsequencesoftheredesignedqueryprocessingengineinASE15isthatASErequiresmoreprocedure
cache.Thisincreasedmemoryrequirementappliestooptimizationandexecutionofqueries.
OnereasonforthisincreasedrequirementisthatASEhasmanymorejoinmethodsanddataaccessalgorithmsto
considerthanpriortoversion15.Thisisespeciallytrueforallrows_dss,whichwillgenerallytendtowards
consumingmoreprocedurecacheforqueryoptimizationthanallrows_mix,whichinturnusuallyneedsmore
procedurecachethanallrows_oltp.
ItisdifficulttopredictexactlyhowmuchmorememoryASE15willneedcomparedwith12.5.Asaruleofthumb,
SybaserecommendstoplanforsizingoftheprocedurecacheinASE15between2to6timeslargerthaninASE
12.5.Whenallrows_dssisused,itislikelythatmoreprocedurecachemaybeneededthanwhen
allrows_oltpisused.Usethe'procedurecache'sectionofsp_sysmontomonitorprocedurecacheusage.
Procedurecacheusagelimitationin15.0.2ESD#2
InASE15.0.2ESD#2,alimitationwasimplementedontheamountofprocedurecachethatcanbeusedbytheASE
queryoptimizer.Thisisbasedonthesettingofconfigurationparameter'max resource granularity'
(settableonsessionlevelwithset resource_granularity).Thissettingisanintegerbetween1100,
reflectingapercentage.Thedefaultsettingis10.
Thisfeatureworksintwoways.First,whentheoptimizerhasconsumedmoreprocedurecachethan:
andwhenatleastonefullplanhasalreadybeengenerated,theoptimizerwilltimeoutandusethecurrentbest
planasthefinalplan.
Second,whentheoptimizerhasnotbeenabletogenerateafullplanyet,buthasalreadyconsumedanamountof
procedurecacheof:
max(50, <max_resource_granularity>)
100 <configured procedure cache size>
thentheoptimizerabortsthequeryplangenerationtoavoidconsumingalloftheprocedurecache.Sincethe
queryhasstillnoplan,itsexecutionwillthenfail.
Thisnewbehaviourwasintroducedin15.0.2ESD#2.Itcanbedisabledwithtraceflag15380.
SybaseASE15BestPractices:QueryProcessing&Optimization
14
Version1.1May2009
OtherresourceusageaspectsofASE15
OneofthenewaspectsofASE15isthefeatureoftablepartitions.Althoughsemanticdatapartitioningisan
optionallicensefeature,itmattersevenwhenyoudonotusesemanticdatapartitioning:inASE15,eachtableand
indexalwaysconsistsofatleastonepartition.Consequently,theconfigurationparameter'number of open
partitions'mustbesethighenoughtoavoidunnecessaryperformancepenalties.
Sybaserecommendsusingsp_countmetadatatoestimatetherequiredsettingforthisparameter,anduse
sp_monitorconfigtomonitoritsusage(seepage26).
SybaseASE15BestPractices:QueryProcessing&Optimization
15
Version1.1May2009
StatisticsInASE15
WhystatisticsmatterespeciallyinASE15
ASEusesacostbasedqueryoptimizertochoosethebestplanforaparticularquery.Toachievethatgoal,the
optimizerreliesonstatisticsaboutthetables,indexes,partitions,andcolumnsreferencedinaqueryforestimating
thecostintermsofI/OandCPUtimeofdifferentpossiblequeryplans.Theoptimizerthenchoosesthequery
planmethodthathasthelowestcost.However,thiscostestimatecannotbeaccurateifthestatisticsthemselves
arenotaccurate.Asaresult,inaccuratestatisticscouldleadtoasuboptimalchoiceofplansandresultinslower
performancethannecessary.
Somestatistics,suchasthenumberofpagesorrowsinatable,areupdatedautomaticallyduringqueryprocessing
(thesearestoredinsystabstats).Otherstatistics,notablythehistogramsoncolumns,aswellasdensity
information,(bothstoredinsysstatistics),areupdatedonlywhenupdate statisticsruns,orwhen
indexesarecreated.Inpractice,itiscriticalthatthehistogramsbeuptodate.ItisaDBAresponsibilityto
scheduleandrunupdate statistics.
InASE15,havinguptodatestatisticshasbecomeevenmoreimportantthanitalreadywas.
ASE15ismoresusceptibletostatisticsissuesthanpreviousASEreleasesduetonowhavingmultiplealgorithmsfor
sorting,grouping,unions,joiningandotheroperations,thusofferingtheoptimizermanymorepossiblechoices.In
addition,ASE15usesstatisticsinmorewaysthaninASE12.x,forexamplefordeterminingthejoinorderinmulti
tablequeries.Goodstatisticsarethereforeimportanttohelpavoidgeneratingasuboptimalqueryplan.
Recommendation:runupdateindexstatistics
ItisrecommendedtomaintainuptodatehistogramsforallcolumnsreferencedinWHEREclauses,eitherasjoin
predicatesorassearcharguments.Sincethesecolumnsaretypicallypartofanindex,inpracticethismeansthat
runningupdate index statisticsforallusertablesislikelytodothejob.Ifsuchcolumnsarenotpartof
anindex,youshouldatleastgeneratehistogramsforthesenonindexedcolumns,orconsiderindexingthem.
Comparedwithupdate statistics,runningupdate index statisticsalsogenerateshistogramsfor
thenonleadingindexcolumns,whichprovidestheoptimizerwithmoreinformationtogenerateoptimalquery
plans.However,update index statisticshassomepotentialdisadvantagescomparedwithupdate
statistics.Sinceitneedstoperformasortoperationforeverynonleadingindexcolumn(theleadingindex
columnisalreadysortedintheindextreebydefinition),update index statisticswilltakemoretimeand
resourcestorunthanupdate statistics.
Using'sampling'withupdateindexstatistics
Onepossibleconsequenceisthatupdate index statisticsmayrunintoa701errorwhenthetableistoo
largetoperformthesortforacolumnwiththeavailableamountofprocedurecache.
Whenthishappens,theremedyapartfrompossiblyincreasingthesizeoftheprocedurecacheistorun
update index statistics withthe"sampling"option.Insteadofreadingthecolumnvaluesfortheentire
table,onlyapercentageofthetablewillberead.Thismakessensesincethefinalhistogramforalargetableis
basedonjustafractionofthecolumnvaluesread.Apositivesideeffectisalsothatupdate index
statisticswithsamplingrunsfasterthanwithoutsampling,soitmayalsobeusedforlargetableswhen
update index statisticstakestoolongtorun.Notethatwhenusingsampling,thereisalwaysa
SybaseASE15BestPractices:QueryProcessing&Optimization
16
Version1.1May2009
possibilitythatthehistogramcontainslessaccurateinformationthanwhenthefulltablewasread.However,this
potentialdownsideisdifficulttoquantify.
Inthefollowingexample,onlyabout5%ofthetableisreadforcreatingthehistogramsfornonleadingindex
columns:
update index statistics big_table with sampling=5
Thelowestpossiblevalueforsamplingis1%.Settingthepercentageto0isequivalentto100%.
Intheoptdiagoutput,thesamplingpercentageusedisdisplayedas"SamplingPercent:"(0means:nosampling
used).
Insteadofspecifyingthesamplingpercentageexplicitly,adefaultsamplingpercentagecanalsobeconfigured
serverwide,bymeansoftheconfigurationparameter"sampling percent".However,thissettingwillapply
toallusertablesandthatmaynotalwaysbedesirable.
Speedingupupdatestatisticswithparallelism
Itmaybepossibletospeedupupdate statisticsfornonleadingindexcolumnsornonindexedcolumnsby
usingparallelism.Forsuchcolumns,histogramcreationrequiresthatallcolumnvaluesbesortedfirst.By
performingthissortoperationwithparallelworkerthreads,theruntimeofupdate statisticscanoftenbe
improved(assumingyoursystemhasenoughresourcestouseparallelism).
Touseparallelsorting,specifythewith consumersclause:
Exactlyhowmuchimprovementcanbeachievedwithparallelism,andwhatlevelofparallelismwouldbeoptimal,
highlydependsonthespecificsystemhardware,ASEconfigurationandworkload.Youshouldexperimentto
determinewhatworksbestforyoursituation.
Notethatparallelismforupdate statisticsdoesnotapplytothetablescanorindexscanwithwhichthe
columnvaluesareretrieved.
Whentorunupdatestatistics?
Statisticsshouldbeanaccuratereflectionofthedatainatable.Thisimpliesthatstatisticsmayneedtobeupdated
whendataischangedmorefrequently.Oneexampleisadatetimecolumn,holdingthedatethatarowwas
inserted.Anotherisanidentitycolumn,whichwillcontinuouslyinserthighervalues.Forthesetypesofcolumns,
histogramswillquicklybecomeoutofdate,sinceallnewrowsbeinginsertedwillhavevalueshigherthanthe
previousmaximumvalueinthehistogram.Thismaycontributetotheoptimizergeneratingsuboptimalquery
plans.
ASE15offersanewfeaturetoassistindecidingwhetherupdate [index] statisticsmaybenecessary
foratable.Thenewbuiltinfunctiondatachange(table_name, partition_name, column_name)
functionmeasurestheamountofchangeinthetable(orpartition)sincethelasttimeupdate [index]
statisticswasrun.Dependingonthispercentage,youmaydecidetorun update [index]
statisticsmore,orless,frequently.
Specifically, datachange()measuresthenumberofinserts,updates,anddeletesthathaveoccurred
onthegiventable,partition,orcolumn.Thisinformationhelpsyoudetermineifrunningupdate [index]
statisticsmaybeneeded.
SybaseASE15BestPractices:QueryProcessing&Optimization
17
Version1.1May2009
Anexampleofthedatachange()functionis:
Inthisoutput,thepercentageofchangestothetablemy_tableis243%,meaningthatthenumberofchangesis
higherthenthenumberofrowsinthetable.Thisindicatesthatitisagoodcandidatetorunupdate [index]
statisticsnow.
Thedatachange()builtinfunctioncanalsoberunonindividualcolumns:
---------------------------
0.017761
Theresultof0.017%indicatesthattheid_numbercolumnhashadarelativelylowamountofchange,and
thereforethestatisticsforthatcolumnarestilllikelytobeaccurate.Itisgoodpracticetorundatachangeona
regularbasisagainstcommonlyusedtablesandcolumns.Thisgivesabasisfordeterminingwhenitisusefultorun
update [index] statistics.Lastly,byspecifyingthesecondparameterofdatachange(),whichisa
partitionname,onlythechangesforthatpartitionarereported.Youmayneedtousesp_helptodeterminea
table'spartitionnames.
ASEhasthefunctionalitytoautomaticallycheckobjectsusingdatachange(),andscheduleupdate
[index] statisticsaccordinglyusingASE'sbuiltinjobschedulerfeature.Formoredetailsonsettingup
andusingthisfunctionality,pleaserefertothe15.xmanualtitled"PerformanceandTuningSeries:Query
ProcessingandAbstractPlans".Inchapter6,"UsingStatisticstoImprovePerformance",refertothesectiontitled
"Automaticallyupdatingstatistics".
Howmanyhistogramsteps?
Anotherissueishowmanystepsahistogramshouldhaveinordertoprovidetheoptimizerwithanaccurate
reflectionofthedatadistributioninacolumn.Thedefaultof20histogramstepsmaybeadequateforcolumns
withasmallnumberofdistinctvalues("lowcardinality"),orfortableswithasmallnumberofrows.However,for
tableswithalargenumberofrows,havingcolumnswithmanydistinctvalues20stepsmaybeinsufficient.Thisis
especiallytruewhenacolumnhasahighdegreeofdataskew(thatis,thevaluesforthecolumnarenotevenly
distributedovertherows).
Obviously,ifyoursystemrunstofullsatisfactionwith20histogramsteps,donotchangeanything.However,ifyou
suspectsuboptimalqueryplansarebeinggenerated,youmaywanttoconsiderincreasingthegranularityof
histogramsbyincreasingtheirnumberofstepsanddeterminewhetherthathasapositiveeffect.Asalways,there
canbetoomuchofagoodthing.Increasingthenumberofhistogramstepscanleadtohigherresource
consumption,especiallyprocedurecacheusage,andlongeroptimizationtimesbeforeaqueryplanisgenerated.
SybaseASE15BestPractices:QueryProcessing&Optimization
18
Version1.1May2009
DuetotheoptimizationtimeoutlimitfeatureinASE15,thiscaninturnalsoresultinsuboptimalplansbeing
generated.
Exactlyhowmanystepsahistogramshouldhaveishardtopredict.Inpractice,youneedtoseeifahighernumber
ofhistogramstepsresultsinbetterqueryplansandbetteroverallperformance.
Nevertheless,asapracticalguidelineitmaybeusefultostartbyincreasethestepcountto200.Ifthatdoesnot
havethedesiredpositiveeffect,tryahighernumber,like500.Analternativeguidelineistouseonehistogram
stepforevery10,000datapages.However,morethan10002000histogramstepsisgenerallyconsideredtoohigh
anumber:ifnopositiveeffecthasbeenachieved,thehistogramstepsmaynotcontainthesolutiontoyour
problem.
Thenumberofhistogramstepsisdeterminedwhenupdate [index] statisticsisexecuted.Thisisdone
asfollows:
Theconfigurationparameter'number of histogram steps'definesthedefaultforallcreate
indexandupdate [index] statisticscommandsfornewhistograms.Theoutofthebox
defaultis20.
Forupdate [index] statistics,thenumberofstepscanalsobesetexplicitlywiththeclause
using nnn values:
update index statistics big_table using 300 values
SybaseASE15BestPractices:QueryProcessing&Optimization
19
Version1.1May2009
Thisformulahastheeffectofassumingthat'histogram tuning factor'issetto1(aswaslikely
thecaseforthosecustomersbeforetheyupgradedto15).
Identifyingmissingstatistics
ASE15hasanewfeaturetoflagcaseswherenostatisticsexistforcolumnswheretheoptimizerwouldhave
expectedthem.Whenrunningthecommandset option show_missing_stats on,theoptimizerwill
printamessagewhenitencounterssuchacase.
Bydefault,theoutputiswrittentotheASEconsole(that'snotthesameastheASEerrorlog!);enabletraceflag
3604togettheoutputsenttotheclient.
Twoexamplesofthesemessagesare:
Whenseeingsuchmessages,itisrecommendedthattheDBArunupdatestatisticsforthecolumnsinvolved.The
correspondingcommandswouldbe:
Dependingonwhichindexesexistonthistable,thesameeffectmightbeachievedbyrunningupdate index
statistics t.Notethatcreatingthesestatisticsforthereportedcolumnsdoesnotguaranteethatadifferent
queryplanwillbegeneratedbytheoptimizer.However,sincetheoptimizersignalsitwouldhavelikedtoconsider
additionaloptions,itisbesttoensuretherequiredstatisticsareavailable.
Itisrecommendedtoenableset option show_missing_stats onatleastforsometimeintheASE15
testenvironment.Thisoptioncanbesetinalogintrigger,soapplicationcodechangescanbeavoided.Please
refertotheASEdocumentationformoreinformationaboutlogintriggers.
OldstatisticsandupgradingtoASE15
BeforeorafterupgradingtoASE15,itisrecommendedtodeterminewhetherveryoldstatisticsexist.Suchold
statisticsmayexistforvariousreasons:
SinceASE11.9,whenanindexisdropped,histogramsfortheindexcolumnsarenolongerdropped
automaticallyatthesametime.Therefore,ifthesehistogramsareneverexplicitlyupdatedandthat
mayneverhappenwhenthosecolumnsarenotpartofanotherindextheoldhistogramsremain.
Ifahistogramwasevercreatedforanonindexedcolumn,itwillstillbearoundifitwasneverupdatedor
deleted.
Irrespectiveofthereasonwhytheycametoexist,oldstatisticsmaynotprovideanaccuratereflectionofthe
currentdatadistributioninthecolumn.Inrarecases,thismayresultinneedlesslyinefficientqueryplansafter
upgrading.
Todetermineifoldstatisticsexist,querythecolumnsysstatistics.moddate(thisisthedate/timewhen
theexistinghistogramsanddensitieswerelastupdated).Ifthisindicatesthatveryoldstatisticsindeedexist,use
theoptdiagutilitytodeterminewhichstatisticsareinvolved.If,atalaterpoint,youareseeingsuboptimal
SybaseASE15BestPractices:QueryProcessing&Optimization
20
Version1.1May2009
performanceforquerieswheretablesareinvolvedthatyouknowhaveveryoldstatistics,considerupdating(the
preferredmethod)ordeleting(thefastermethod)thoseoldstatistics.
However,beforedeletinganystatistics,pleasemakeabackupcopywiththeoptdiagutility.Thisallowsyouto
restoretheoriginalstatistics(again,withoptdiag)inthecasethatthefreshlyupdatedstatisticsshouldresultin
unexpectedperformanceeffects.
Infact,itmaybeagoodideatomakeabackupcopyofthestatisticsforallyourtablesafterupgradingtoASE15,
butbeforerunningupdate [index] statisticstorefreshthestatistics.
Todeletestatistics,useoneofthesecommands:
Summaryofrecommendationsforstatistics
Pleaserefertotheprecedingpagesfordetailsoneachoftheserecommendations.
Keepstatisticsonusertablesuptodatebyrunningupdate [index] statistics regularly
Usethedatachange()functiontodeterminewhetherupdate [index] statisticsshouldberun
moreoften
Preferablyrunupdate index statisticsinsteadofupdate statistics.Forlargetables,use
samplingifnecessary.
Ifyoususpectsuboptimalqueryplansaregenerated,considerincreasingthenumberofhistogramstepsfor
thetablesinvolved.
Usethecommandset option show_missing_stats ontodeterminecaseswherestatisticsmaybe
missing;andthencreatethosestatistics.
ConsiderdeletingorupdatingoldstatisticsafterupgradingtoASE15whentheseareassociatedwithsub
optimalqueryplans;however,alwaysmakeabackupcopyfirstwithoptdiag.
SybaseASE15BestPractices:QueryProcessing&Optimization
21
Version1.1May2009
RecommendedTestingBeforeUpgradingToASE15
Whyshouldyoutest?
BeforeupgradingyourproductionsystemstoASE15,itisimportanttogatherdetailsabouttheperformance
characteristicsofyourapplicationsonthecurrent,pre15versionofASE,preferablyinaproductionenvironment.
Gatheringsuchdataisimportant,sinceanydiscussionsaboutperformancedifferencesafterupgradingtoASE15
willnotbeveryusefulifthesediscussionsarenotbasedonhardnumbers.Forthe12.xvs15performance
comparisontobevaluable,itisstronglyrecommendedthatthesetestsbeperformedasrealisticallyaspossible.
Thismeans:
Runtestsforasmanyapplicationfunctionsaspossible,oratleastthemostcriticalfunctions.Foreach
function,measuretheresponsetimeorthroughput;preferablyalsoperformthesemeasurementsfor
eachqueryexecutedbytheapplication(seepage32forsuggestions)
RuntheperformancemeasurementsinyourcurrentASE12.xproductionsystem
runthesametestsbeforeupgradingyourproductionsystemtoASE15,inafullyconfiguredASE15test
systemwithacopyofthefullASE12.xproductiondatabase,aswellasrealisticworkload(i.e.thesame
queriesasin12.x,andwiththesamelevelofconcurrentuseractivity)
Also,preferablyrunthesetestsbeforeupgradingyourproductionsystemtoASE15,sothatyoucanidentifyany
issuesbeforegoingliveonASE15.Capturingthe"performancefootprint"ofyourcurrentASE12.xproduction
environmentwillprovideagoodbaselineforanycomparisonswithASE15.Themeasurementsshouldprovide
specificnumberssuchasnumberoflogicalI/Os,elapsedtime,compilationtime,CPUutilizationetc.forapplesto
applescomparison.Differencesinthemeasuredvalueswillhelpidentifyindividualqueriesperformingpoorlyas
wellasanyserverlevelconfigurationissues.
Therearesomedifferentanglestolookingatperformance.ToenableasensiblecomparisonofperformanceinASE
12.xandASE15,performancedataneedstobegatheredontwolevels:
forindividualqueries,runinisolation
forindividualqueries,runwithfullworkloadbyotherusers
forASEasawhole,fromaserverwideresourceusageperspective
PleaseensurethattheconfigurationsoftheASE12.xandASE15serverareasidenticalaspossible.Havingsaid
that,theASE15servermayhavemoreresourcesallocatedtoitthan12.x,andinsomecases(seepage14)thatis
infactrequired.Also,makesurethetestsareexecutedinsimilarcircumstances.
Thesearesomeaspectsthatarecriticaltodoingafaircomparisonbetween12.xand15:
Ensurethecacheis'warmedup'thesamewayin12.xandin15duringthetests
Useidenticalcache/bufferpoolconfigurations
MakesuretheamountofprocedurecacheinASE15isamultipleofwhatisusedin12.x(seepage14)
Usesimilardatadevicelayout/placement,especiallyforlogdevicesandfortempdb?
Ifyourunteststhatmodifydata,considersettinguptestsystemswherebytheoriginaldatabasecanbe
restoredaftereachtestrun.
Theseconsiderationsarecriticalastheywillaffecttheperformancebeingmeasured.Differencesintheseareas
canthereforeleadtofalsealarmsandwastedtimefollowinguponnonexistentissues.
SybaseASE15BestPractices:QueryProcessing&Optimization
22
Version1.1May2009
Pleasenote
ItisimportanttoobservethattheapproachtowardstestingasadvocatedaboveisnotspecificforASE15,butisa
bestpracticethatappliestoanymajorupgradeofanysoftwareproduct.Thebestchanceofavoidingpotential
risksaroundupgradesistorunasrealisticanenvironmentaspossiblebeforeputtingyouractualproduction
systemsthroughtheupgrade.Shouldyoubeunabletofollowthisapproach,thenaturalimplicationisthatyou
simplywillnotdiscoveranyissuesuntilyourproductionsystemdoes.
Fromariskmanagementperspective,youshouldaskyourselfhowimportantitistoidentifyanyperformance
issuespriortotheupgrade,andhowmuchopportunityyouwillhavetoidentifyandfixsuchissuesafteryour
productionsystemhasbeenupgraded.Although,again,thisisnotspecificforASE,Sybaserecommends
performingthoroughtestingasoutlinedinthisdocumenttominimizetheriskofdisruptionsduetoanupgrade.
UsingCompatibilityMode
Asdescribedonpage12,ASE15.0.3ESD#1providesafeaturenamed'CompatibilityMode'tooffercustomersa
migrationpathtoASE15thatrequireslesstestingeffortatthemomentofmigration.
Nevertheless,SybasestillrecommendstoperformsufficienttestingonapplicationsthataremigratedtoASE15,
evenwhenusingcompatibilitymode.
Identifyingqueriesthatrunslower
Assumingyouareabletorunrealistic,representativetestsonbothASE12.xandASE15,yournextstepsdepend
onwhatyoufindinASE15.Alongthelinesofthesuggestionsonpage7,itisrecommendedtorunyourtestsin
ASE15atleastthreetimes,eachtimewithadifferentserverwideoptimizationgoal.
Intheidealsituation,youcanestablishaserverwideoptimizationgoalunderwhicheverythingrunsthesameor
fasterinASE15,inwhichcasethereisprobablynotmuchelselefttodo.Intheeventthatsomeapplications
wouldappeartorunslowerinASE15thaninASE12.x,youwillfirstneedtoidentifythequeriesinvolved,then
determinewhytheyrunslower,andfinallyfigureouthowtoresolvethis.Themainchallengeliesinidentifying
thosequeriesthatrunslower.
IfmaybeimmediatelyapparentwhichqueriesrunslowerinASE15,forexamplewhenaclearlydefined
applicationfunctionappearstoperformlessthanin12.x.Insuchacase,thecorrespondingSQLqueriescan
hopefullybeidentifiedrelativelyeasilybyusingtheMDAtablesorbyapplicationtracing(seepage32).Ifthe
slowerqueriescannotbeidentifiedeasily,thebestapproachistocaptureallSQLqueriessubmittedbythe
applicationinASE15(again,seepage32),andanalyzethesefurther(seebelow).
Ingeneral,itisparticularlyimportanttoensurethefollowingtypesofqueriesareincludedinyourtests,evenif
theyarenotexecutedfrequently(forexample,forperiodicreporting).Applicationdevelopersmaybeableto
providesomeofthesequeries.Thisconcernsqueriesinvolving:
multitablejoins,
multipleviews/unions,
starschemajoins
severalsubqueries
derivedtables
outerjoins
AnalyzingperformancedifferencesbetweenASE12.xandASE15
OnceyouhavecapturedtheSQLtextofactualproductionqueries,youwillneedtoanalyzethesetodetermine
whichqueriesperformworseinASE15thanin12.x.Thisprocesscanbeperformedwithsimplecommandline
SybaseASE15BestPractices:QueryProcessing&Optimization
23
Version1.1May2009
tools,forexampleasdescribedbelow.Notethatthisapproachiswellsuitedforidentifyingqueryplanproblem
withlongerexecutiontimes(i.e.multipleseconds),butlesssuitableforquerieswithveryshortexecutiontimes
(e.g.100milliseconds).
First,createafile,named,say,sql.in,withthefollowinglayout,containingtheSQLtextofyourcaptured
queries(inbatches,sincethisishowapplicationssubmitqueriestotheASEserver):
set plan optgoal allrows_xxx -- only for 15; this raises an error in
12.x
go
set statistics plancost on -- only for 15; this raises an error in
12.x
go
use your_db -- your default database
go
set statistics io on
set statistics time on
set showplan on
go
<batch 1> -- captured SQL query batches go here
go
<batch 2>
go
[...]
<batch n>
go
NowrunthisfilebothinASE12.xandin15andcapturetheoutput:
Nowyouneedtoanalyzethedifferencesinperformancebetweenexecutionin12.xandin15.
Thiscanbedonebysearchingforthestring"Serverelapsedtime:"inbothfiles(forexample,withgrep),and
comparingtheresults(forexample,byimportingthelistofexecutiontimesforbothcasesintoaspreadsheetand
lookingforthosecasesthataremorethan,say,50%slowerinASE15).
Hereisanexampleofsuchaline,indicatingthatthequeryraninlittlelessthan16minutesinASE15:
Adaptive Server cpu time: 27300 ms. Adaptive Server elapsed time: 945953 ms.
AnothercriteriontoidentifyqueriesistocomparetheamountofI/Othequeriesspentin12.xand15.0.Thiscan
bedonebyexaminingoutputlineslikethefollowing(notethateachquerymayproduceanumberofsuchlines,
oneforeachtablereferencedbythequery).Thenumbertolookforisthenumberofregularlogicalreads(shown
inboldbelow):
SybaseASE15BestPractices:QueryProcessing&Optimization
24
Version1.1May2009
Onceyouhaveidentifiedspecificqueriesthisway,youcanstudythoseparticularcasesinmoredetailbylookingat
theirqueryplan,andtheLavaTreeoutputresultingfromtheplancostoption(seepage30).
SybaseASE15BestPractices:QueryProcessing&Optimization
25
Version1.1May2009
Analyzingshortdurationqueries
Tocomparetheperformancedifferenceofveryshortqueries,youmaywanttorunsuchaqueryapredefined
numberoftimes,e.g.
Withthisapproach,youshouldcomparethenumberofmillisecondsfor1000executionsin12.xand15.Repeating
aquery1000timeswilleliminatejitterduetoexternalcircumstances,whichcouldaffecttheresponsetimeofone
suchindividual,shortexecutionquitedrastically.
Serverlevelperformanceaspects
Serverlevelconfigurationsettingscanhaveabigimpactonqueryperformance,thoughitmaynotbepossibleto
predictwhichquerieswillbeaffectedmost.
AspartofyourtestingaroundtheASE15migration,capturethefollowingserverleveldata:
Runsp_sysmon regularlyandcaptureitsoutput;recommendisrunningitevery1015minuteswitha
sampleintervaltimeof1minute.
Itiscriticalthatdataiscapturedduringallthesignificantperiodsoftheapplicationprocessing.For
example:dailypeakperiods,nightly/weekly/monthlybatchcycles,MondaymorningMarketOpensetc.
Whenanalyzingthe sp_sysmon output,alsopayattentiontothefollowing:
o DataCacheusage:Reviewthecachehits,spinlockcontentionandthebufferpoolusage.
Highcountsforcachehitsand/orhighspinlockcontentionmaybeanindicationofpoorlyperforming
queries.
o CPUutilization:ReviewtheCPUcharacteristicsoftheloadi.e.howbusyarethevariousengines,how
muchtimeisspentinI/OBusy,Idlev/sCPUBusyetc.
o DeviceIOcharacteristics:ReviewtheI/Osoutstanding,I/Oscompleted/sectogetaroughideaofthe
levelofI/Osgeneratedbytheapplication/module.ItwouldalsobeinterestingtonotehowtheI/Os
aredistributedacrossthevariousdatabasedevices.IfthelayoutinthetargetASE15.0isexpectedto
change,thisinformationmaycomeinhandy.
Runsp_monitorconfig 'all'regularly(every1015minutes),andlookforconfiguration
parametersshowinganonzero'reuse',inparticularprocedurecache.Youmayalsorun
sp_monitorconfigwithatablenameasanadditionalparameter:thiswillcausetheresultvalues
fromsp_monitorconfigtobestoredinatable(whichyouhavetocreatefirst;seetheASEReference
manualfordetails).
SybaseASE15BestPractices:QueryProcessing&Optimization
26
Version1.1May2009
ExampleOfASE15QueryPlanAndLavaTree
ASE15QueryPlans
EachSQLstatementiscompiledintoaqueryplan.InASE15,aqueryplanisbuiltasanupsidedowntreeof
operators:theoperatoratthetophasoneormorechildoperators.Eachofthese,inturn,canhaveoneormore
childoperatorsthemselves,andsoon,thusbuildingatreeofoperators.Theexactshapeofthetreeandthe
operatorsinitarechosenbytheASEqueryoptimizer.
Thequeryplancanberepresentedindifferentways:
set showplan onshowsthequeryplaninthewellknownformat,showingthejoinorder,indexes
usedandI/Ostrategy.
set statistics plancost on(anewcommandinASE15)showsthequeryplaninaformatmore
closelyresemblingtheinternal"LavaTree".
Lastly,theGUItoolDBISQLcandisplaythequeryplaninagraphicaltreeformat.Thisisnotdiscussed
furtherinthisdocument.
Example:setshowplan
TodisplaythequeryplanintheclassicASEformat,useset showplan on:
set showplan on
go
Thefollowingshowssomehighlightsfromtheresultingqueryplanoutput:
SybaseASE15BestPractices:QueryProcessing&Optimization
27
Version1.1May2009
| | | FROM TABLE
| | | titleauthor
| | | TA
| | | Index : auidind
| | | Forward Scan.
| | | Positioning by key.
| | | Index contains all needed columns. Base table will not be
read.
| | | Keys are:
| | | au_id ASC
[...]
EMIT
Aqueryplancanhaveanarbitrarilydeeptreeofqueryplanoperators.Eachoperatorreceivesrequestsforthe
nextrowfromitsparent(topdowncontrolflow),processestherequestandreturnstherequestedrowtoits
parent(bottomupdataflow).Theorderofexecutionofanoperatordependsthusonitsparent'salgorithmand,
recursively,ontheorderofexecutionofitsparent.
Forinstance,a"leftdeep"treeofnestedloopjoinoperatorswillstartbyscanningtheoutermosttable,whenthe
"nextrow"request,whichispasseddownontheoutersideofeachnestedloopjoin,reachestheoutermostscan.
Then,asanestedloopjoinreceivesarowfromitsouterchild,itwilldoacorrespondingscanofitsinnerside
basedonthatrow.Thenestedloopjoinwillreturn,eachtimeitsparentasksfor,itsownrowbasedonitscurrent
SybaseASE15BestPractices:QueryProcessing&Optimization
28
Version1.1May2009
outerandinnerchildrows.
Thefollowingexampleshowsamorecomplexqueryplan:
EMIT
MERGE JOIN
SORT
Inthiscase,EMITasksitsMERGEJOINchildforarow,whichinturnasksits2childrenforrowsattemptingto
satisfythemergeclause.
TheupperMERGEJOIN'souterchild,SORT,isablockingoperator.Thismeansitneedstoconsumeitsentireinput
beforereturningitsfirstrow(sincethefirstrowinthesortedresultcouldbelastrowreturnedbyitschild).SORT
willthusaskitschild,thelowerMERGEJOIN,onebyoneforallofitsrowsandwillsortthem(ifpossiblein
memory,overflowingifneededtoaworktable)andonlythenwillitreturnitsfirstrowtotheuppermostMERGE
JOIN.
ToproducetheserowsateachrequestofitsSORTparent,thelowerMERGEJOINwillaskfornewrowsfromits
outerand/orinnerchildren,dependingonthecurrentvaluesinthemergecolumns,untiltheequijoinclauseis
satisfiedandaMERGEJOINrowisreturned.
AsthesechildrenareINDEXSCANsusingindexesontheequijoincolumns,therowsarereturnedordered,as
requiredbytheMERGEJOINalgorithm,andnosortisdeeded.
TheupperMERGEJOINwillhavethesamebehavior,askingSORTandINDEXSCAN(titles)forrowsandreturnat
eachEMITrequestarowwheretheequijoinclauseissatisfied.
Finally,therootoperatorEMITsendseachfinalresultsetrowbacktotheclientapplicationthatinvokedthequery.
Notethatitisnotguaranteedthateachoperatorinthetreeisexecuted.Forexample,afilterconditionmay
evaluateinsuchawaythatnorowscaneverresultfromasubtree,whoseoperatorsmaythereforenotbe
executedatall.Formoreinformationabouttheoperatorsandtreestructure,seetheASEPerformanceand
TuningGuide,volume"QueryProcessingandAbstractPlans".
SybaseASE15BestPractices:QueryProcessing&Optimization
29
Version1.1May2009
Example:plancostandLavaTree
AusefulnewfeatureinASE15istheplancostoption,whichdisplaysthequeryplaninasemigraphicaltree
formatmorecloselyresemblingtheinternalrepresentation,knownasthesocalled"LavaTree"(whichgotits
namesincedatarowsflowupwardthroughthetreeoperatorstoasinglepoint,similartolavainavolcano).
Theplancostoptioncanbeturnedonusingthefollowingcommand:
Oneapplicationofthiscommandistocomparetheestimatedandactualcostsinaqueryplan.Itcanbeauseful
toolfordiagnosingqueryperformanceproblems.Foreachoperator,theLavaTreeoptionshowstheestimated
(el:)andactual(l:)numbersoflogicalI/O,estimated(ep:)andactual(p:)physicalI/O,estimated(er:)and
actual(r:)rowcountsandactualCPUticks(cpu:).Inaddition,executionoperatorsthatdoasortorahashbased
operationwillalsoreportthenumberofprivatebuffersusedforthatoperation(bufct:,notshowninthe
examplebelow).Sincenotallofthesequantitiesapplytoeachtypeofoperator,asubsetmaybeshown.Forsub
optimalqueryplans,thisinformationcanbeusedtoverifythattheoptimizer'sestimatesaresound.
Ifthejoinqueryfromthepreviousexampleisrun,yougetoutputthatlooksliketheoperatortreeshowninthe
previousdiagramwithadditionalannotationsshowingtheactualversusestimatednumbers.Theestimated
numbersarethosecomputedbythequeryoptimizerandtheactualnumbersaretheresultoftheactualquery
execution.Thisisagoodwaytovalidatehowclosetheoptimizer'scostmodeliswithrespecttotheactual
numbers.
Emit
(VA = 6)
r:25 er:342
cpu: 0
/
MergeJoin
Inner Join
(VA = 5)
r:25 er:342
/ \
Sort IndexScan
(VA = 3) titles_672002394 (T)
r:25 er:25 (VA = 4)
l:6 el:6 r:18 er:18
p:0 ep:0 l:2 el:3
cpu: 0 bufct: 24 p:0 ep:3
/
MergeJoin
Inner Join
(VA = 2)
r:25 er:25
/ \
SybaseASE15BestPractices:QueryProcessing&Optimization
30
Version1.1May2009
IndexScan IndexScan
auidind (TA) authors_57600205 (A)
(VA = 0) (VA = 1)
r:25 er:25 r:23 er:23
l:1 el:2 l:1 el:2
p:0 ep:2 p:0 ep:2
Fromtheaboveoutput,someinterestingobservationscanbemade.ThebottomleftIndexScanoperator(on
thetitleauthorstable)hasanestimatedrowcount(er:)of25andtheactualrowcount(r:)is25aswell;thismeans
thattheoptimizer'sestimatewascorrect.
However,therowcountestimatesforthetopMergeJoin(VA=5)arecompletelyoff,astheoptimizer's
estimateis342buttheactualrowcountis25.Itmaybepossibletorectifysomeoftheinaccuraciesinthe
optimizer'sestimatesbymakingsurethatthestatisticsareuptodate,orbyincreasingthenumberofstepsinthe
histogram.Ifnohistogramsexistonthejoincolumns,asindicatedbyset option show_missing_stats
on,creatingthosehistogramsmayimprovetheoptimizer'sestimates.
Notethatcomparingtheoptimizerestimatesandactualsisnotanexactscience:whentheestimatedrowcountis
25andtheactualrowcountis30,thatshouldnotnecessarilybetakenasanindicationthattheoptimizer's
estimatesareincorrect.Orderofmagnitudedifferences,suchas25vs.342intheexampleabove,arewhatyou
shouldlookfor.
Also,notethatthetablenameisnotdisplayedforIndexScanoperatornodes,butonlythenameoftheindex.If
itisunclearwhichtableisassociatedwithanoperatornode,therearevariouswaystofindthisout:
Thenameoftheindexmayuniquelyidentifythetable
Ifacorrelationnamewasusedinthequery,thisisshownintheoperatornode.Forthebottomleft
IndexScanoperator,"(TA)"refersto"titleauthor TA"fromtheoriginalSQLquery.
Theindication"(VA=n)",wheren=0,1,2,,isshownbothintheLavaTreeandintheshowplanoutput,and
uniquelyidentifieseachoperatornode.
SybaseASE15BestPractices:QueryProcessing&Optimization
31
Version1.1May2009
CapturingApplicationSQL
Inthecontextofmonitoringortroubleshootingperformanceissues,itisoftenusefultodeterminetheexactSQL
queriesanapplicationissubmittingtotheASEserver,alongwithsomeperformancemetricssuchaselapsedtime
orI/Oconsumption.ThisinformationcanbeusefulbothbeforeandaftermigratingtoASE15,forexampleinthe
contextofperformancetestingaroundtheASE15upgrade(seepage22).
FromthesideoftheASEserver,thiscanbedoneinanumberofways,dependingonyourASEversion:
Auditing
MDAtables(12.5.0.3orlater)
Applicationtracing(15.0.2orlater)
Abstractplancapture(12.0orlater)orquerymetricscapture(15.0orlater)
MonitorServer
SQLinterceptiontools,likeRibo(see$SYBASE/jutils-x_y)or3rdpartyproducts
Wewilllookatthefirstfourinalittlemoredetailbelow.
Auditing
Withauditing,youcanenablethe'cmdtext'auditingoption(withsp_audit).Thiswillcapturetheclient
submittedSQLtextinthesybsecuritydatabase.Youwillneedtoextractthecapturedtextfromthisdatabase.
NoperformancemetricsrelatedtotheSQLarecaptured.
SeetheASESystemAdministrationGuidefordetailsonsetupandconfigurationofauditing,andforinformation
aboutaccessingthecaptureddata.
MDAtables(12.5.0.3)
InASE12.5.0.3orlater,theMDAtables(alsoknownas'monitoringtables')canbeusedtocapture,amongother
things,theSQLtextsubmittedbyclientapplications.Forthis,usetheMDAtablemaster..monSysSQLText.
Associatedexecutionmetrics(executiontime,I/Ocounts,andmore)canbefoundin
master..monSysStatement.
Whenlookingforslowrunningorresourceintensivequeries,youcanfilterthecontentsofmonSysStatement
by,forexample,lookingfor:
queriesrunninglongerthan,say,10seconds(i.e.10000milliseconds):
filterondatediff(ms, StartTime, EndTime) > 10000
queriesusingmorethanacertainnumberoflogicalI/Os,say,20000:
filteronLogicalReads > 20000
NotethatmonSysSQLTextandmonSysStatementhaveaconfigurablebutlimitedsize,sotheyshouldbe
queriedregularlytoextractthecapturedinformationandstorethisinapermanent,regulartable.Ifthisisnot
donefrequentlyenough,orifthenumberofsubmittedqueriesisveryhigh,someofthesubmittedSQLmaynotbe
capturedthroughtheseMDAtables.
ForhelpwithinstallingandusingtheMDAtablespleaserefertoeitherofthefollowing:
theASEPerformanceandTuningGuide,volume"MonitoringandAnalyzing",chapter"MonitoringTables"
membersofISUGcanusethisinformationavailableattheISUGwebsite:
SybaseASE15BestPractices:QueryProcessing&Optimization
32
Version1.1May2009
http://my.isug.com/p/do/sd/sid=13&fid=9&req=direct
ApplicationTracing(15.0.2)
InASE15.0.2,youcanintercepttheSQLqueriessenttotheASEserverbytheclientapplicationbymeansofanew
featurenamedapplicationtracing.ThiscanbeusedtoidentifyquerieswithlongrunningtimesorhighI/O
consumption.
SupposeanapplicationhasconnectedtoASEassession#24,andyouwanttoseethequeriessubmittedthrough
thisconnection.Thiscanbedoneasfollows(assumingASErunsonUnix/Linux):
Asofthismoment,allSQLsubmittedbysession24willbewrittentothefile
'/tmp/my_dir/apptrace.24.out',accompaniedbytheoutputfromset showplan andset
statistics.Notethatthesetcommandsaboveapplytosession24,andnottothesessionexecutingthese
statements,aswouldbethenormalsemantics.
Whileyoursessionhasinitiatedapplicationtracingonanothersession,itisimportanttoobservethefollowing:
Donotrunanyothercommandsinthissessionuntilyouhavedisabledthetracingagain(seebelow).
Onceapplicationtracingisenabled,donotrunothersetcommandsthanthoseshownabove.
Donotdisconnectyoursessionuntilyouhavedisabledthetracingagain;thismaycause showplanoutput
tobesenttosession24,eventhoughthatsessiondidnotrunset showplan onitself.
Donotleavetracingenabledforlongperiods,toavoiddisconnectslikementionedinthepreviousbullet.
Todisableapplicationtracing,firstswitchallsetcommandsoffagainandthenstopthetracing:
Youcannowaccessthefile'/tmp/my_dir/apptrace.24.out' andanalyzeitscontents.
Inpractice,itmaybemostpracticaltouseapplicationtracingforshortintervalsratherthanprolongedperiodsof
time,sincetheamountofdatawrittentothetracefilecanbesignificant.Formoreinformationaboutapplication
tracing,pleaseseeaseparatewhitepaperavailableontheSybasewebsiteat
http://www.sybase.com/detail?id=1052835.
SybaseASE15BestPractices:QueryProcessing&Optimization
33
Version1.1May2009
AbstractPlanCapture/QueryMetricsCapture
SinceASE12.0,abstractplancapturecanbeusedtointerceptclientsubmittedSQLtext.Thistextisstoredinthe
sysqueryplanstablealongwithitsabstractqueryplan.Thecapturedquerytextcanberetrieveddirectlyfrom
thistable.InASE15,thenewfeatureofquerymetricscaptureusesthesameunderlyingmechanismtocapture
executionmetricssuchasI/Ocountsandexecutiontimes,alongwiththeSQLtext.Thesemetricsarealsostoredin
sysqueryplans,butshouldbeaccessedthroughaviewnamedsysquerymetrics.
Apotentialdisadvantageofthesefeaturesisthatthecaptureddataisstoredinthesysqueryplanstableinthe
session'scurrentdatabaseatthemomentthequerywasexecuted.Thismeansthatthecapturedquerytextcould
endupin,forexample,tempdb,ifthatwasthesession'scurrentdatabase,eventhoughallqueriedtablesmay
resideinotherdatabases.
Formoreinformationaboutquerymetricscapture,seetheASEPerformanceandTuningGuide,volume"Query
ProcessingandAbstractPlans".
SybaseASE15BestPractices:QueryProcessing&Optimization
34
Version1.1May2009
InformationToCaptureBeforeContactingSybase
TechSupport
BeforecontactingTechSupport,itisoftenusefultogatheradditionaldiagnostics.Thisisespeciallytruewhenthe
problemisreproducible.Thissectionwilldiscusssomepotentialproblemareasanddocumentthecollection
procedures.
701Errors
Whenaregularquery(i.e.notupdate index statistics)runsintoa701error,thisindicatesthatASE
exhaustedtheprocedurecachespace.Ifyouarerunningwiththedefaultprocedurecachesize,youshould
increasethisandtryagain.Thegeneralguidelineistoinitiallytake23timesthesizeofyour12.5.xprocedure
cache,thoughinsomecases,especiallywhenusingtheoptimizationgoalallrows_dss,yourprocedurecache
mayneedtobelarger.
Ifincreasingtheprocedurecachehasnotresolvedthe701errorandyoucannotisolatetheproblem,thenyou
shouldsetupaConfigurableSharedMemoryDump(CSMD).Thefollowinginstructionsareused.
Thesecondinstructionaddstheerror701conditiontoinitiateamemorydump.Thefourthparameterindicates
thenumberofmemorydumpstocapture,inthiscase1.ASEwillnotcaptureadditionalmemorydumpsonthis
conditionuntilASEisrecycledorthecounterismanuallyreset.Thelastparametershownisthenameofa
directorytoholdthememorydump.Notethatthefilesystemonwhichthedirectoryresidesshouldhaveenough
freespacetoholdthememorydumpfile,whichcanbelarge.
Youcanverifythedumpconditionscurrentlydefinedbyrunningsp_shmdumpconfigwithoutanyparameters.
Thiswillalsoshowanestimatedsizeofthememorydumptobecaptured.Youshouldverifythatthediskspace
availableinthedirectoryislargerthantheestimatedmemorydumpfilesize.
Afilenamewillautomaticallybegeneratedthatincludesthedateandtimeofthememorydump.Oncethe
memorydumphasbeencaptured,youcanremovethisconditionusing
IfyouwishtostopallCSMDs,configure'dump on conditions'to0.
Oncethememorydumphasbeencaptured,youshouldopenacasewithTechSupportanduploadthememory
dumptotheftpsite(forinstructions,seepage39).
YoushouldalsoincludetheoutputfromthefollowingSQLstatements.TheseusetheMDAtableswithinASE,
whichisautomaticallysetupwhenrunningtheinstallmasterscriptinASE15.0.2orlater.EarlierASEversions
needtoruntheinstallmontablesscript.SeeexistingdocumentationforconfiguringtheMDAtables.
SybaseASE15BestPractices:QueryProcessing&Optimization
35
Version1.1May2009
select * from master..monProcedureCacheModuleUsage
go
Note:thecontentsoftheMDAtablesshownabovedonotcontainanyusefulinformationforcustomers.
Bydefault,ASEwillsendthe701errormessagetotheclient.Youshouldconsidergettingthismessagealso
reportedintheerrorlog,sothatyoucantrackhowoftenitishappening.
Thiscanbedoneasfollows:
sp_altermessage 701,'with_log',true
PerformanceProblemwithaLimitedNumberofQueries
Ifalimitednumberofqueriesarenotperformingwellduetosuboptimalqueryplansorsuboptimalresource
consumption,youmaywanttoconsiderinstallingthelatestASE15.0.xreleaseonyourdevelopmentservertosee
iftheproblemstillexists.Iftheproblemstillexists,orifyoucannottestthelatestASErelease,thenareproduction
shouldbecollectedandsubmittedtoSybaseTechSupport.IfyoucannotsendareproductiontoTechSupport,you
shouldusethefollowingstepstogatherdiagnostics:
1. Createascriptfile(here,namedsql.txt)containingthesecommands:
select @@version
go
select @@optgoal
go
sp_cacheconfig
go
sp_configure 'nondefault' -- only if you're running 15.0.2 or later!
go
dbcc traceon(3604)
set showplan on
set statistics time, io, plancost on
set option show long
go
<your query text>
go
2. Runsql.outusingisqlandcapturetheoutputinafile(WARNING:theoutputfilecanpotentiallybe
verylargepotentiallyGigabytesforcomplexquerieswithmanytables,underallrows_dss):
ThefollowinginformationshouldalsobeincludedsothatTechSupportcanattemptareproduction:
Thefilessql.txtandsql.out.Ifapplicable,includeacasewiththe'fast'queryplan
(sql.fast.txt)andonewiththe'slow'queryplan(sql.slow.txt)andcorrespondingfiles
SybaseASE15BestPractices:QueryProcessing&Optimization
36
Version1.1May2009
sql.fast.out,sql.slow.out
DDLforthebasetable(s)andindex(es);thiscanbegeneratedwiththeddlgenutility
Simulatestatisticsoutputforthebasetable(s)viaoptdiag:
AcopyoftheASEconfigurationfile,or,in15.0.2,theoutputofsp_configure 'nondefault'
Ifthequeryisusingview(s)orstoredprocedure(s),thenalsoincludetheirSQLsourcecodeasobtainedby
defncopyorddlgen.
Theoutputofsp_monitorconfig 'all'.
SystemWidePerformanceProblemsorHighCPUUsage:step1
IftheperformanceofASE,ataserverwidelevel,isnotacceptable,andyouarerunning15.0.2ESD#3orlater,
pleasefollowthestepsbelow.ThisisalsorequiredwhenASEisexperiencingunusuallyhighlevelsofCPUusage
withoutaclearcause.
ThisremedyisrelevantmostlywhenrunningamultiengineASEserver;forasingleengineASEserverthis
proceduremaynotyieldasmucheffect.Pleaseenablethistraceflagonlyasdescribedbelow:
ShutdownASEandrestartitwithtraceflag757intheRUN_serverfile(i.e.addT757)
IfASEcannotberebooted,youcantryrunningthefollowingdbcccommandsinstead.Itshouldhowever
benotedthatthismayhavelesseffectthananASEreboot:
dbcc traceon(757)
go
dbcc proc_cache(free_unused)
go
Broadlyspeaking,theeffectofthistraceflagisthatallocation/deallocationalgorithmsintheprocedurecache
behavedifferentlyonsomepoints.
PleasereporttoTechSupport(byopeningacase)whetherthesestepsmakeadifferencetothesituationinyour
ASEserver.Whenreportingthisinformation,pleaserefertothiswhitepaperandrequestyourcasetobelinkedto
CR484362;pleasealsoindicatewhetheryourebootedASEornot.
NotethatthistraceflagshouldnotbeusedinanyASEversionsearlierthan15.0.2ESD#3.
SybaseASE15BestPractices:QueryProcessing&Optimization
37
Version1.1May2009
SystemWidePerformanceProblemsorHighCPUUsage:step2
IftheperformanceofASE,atasystemwidelevel,isstillnotacceptableafterthestepabove,SybaseTechSupport
needsmoreinformationabouthowASEisspendingitsprocessingtime.
Since,incaseslikethese,thereisnoerrorconditiontoconfigureamemorydumpfor,youwillneedtocapturea
ManualSharedMemoryDump.Todothis,youneedtouseSybMon,whichisanundocumenteddiagnostictool.
NOTE:THEINFORMATIONOBTAINEDBYTHEFOLLOWINGPROCEDUREISUSEFULFORSYBASETECHSUPPORT
ONLYANDDOESNOTCONTAINANYINFORMATIONTHATCANBEUSEDBYCUSTOMERS.STRICTLYNOOTHER
COMMANDSSHOULDBEUSEDBYCUSTOMERSOTHERTHANASINSTRUCTEDBYSYBASETECHSUPPORTAND
THISDOCUMENT.PUBLICDISCUSSIONSABOUTSYBMON,SUCHASINNEWSGROUPS,SHOULDBEAVOIDED
SINCEINCORRECTINFORMATIONMAYBECOMMUNICATEDTHATMAYBEHARMFULTOYOURASESERVER!
USINGSYBMONOTHERTHANASINSTRUCTEDBYSYBASETECHSUPPORTANDTHISDOCUMENT,ORFOR
DIFFERENTPURPOSES,ISENTIRELYATYOUROWNRISK.
SybMonisstartedasshownbelow.YoumustusethesameOSloginthatwasusedtostartASEinordertoavoid
permissionproblems.Notethatthepasswordisnotthe'sa'loginpassword,butaspecialpasswordthatcanonly
beobtainedfromSybaseTechSupport.PleasecontactTechSupportifyouneedtorunthisstep.
Pleasenote:itisessentialthatthefollowingprocedurebeperformedwhenperformanceproblemsareactually
occurring,sinceonlyatthatmomentcantherightinformationbegatheredthatisneededtodiagnosetheissue.
OnUnix:
dataserver X P password D<path to directory with .krg file>
OnWindows:
sqlsrvr.exe X P password -D<path to directory with .krg file>
Ifyouhaveonly1krgfileinthedirectory,ASEwillautomaticallyattachtothesharedmemoryregionand,ifthe
serverisnamed"PROD1",youwillseeapromptlikethis:
PROD1:active>
Ifyouhavemultiple.krgfiles,youwillneedtoattachtothecorrectserver'ssharedmemoryusingthecommand:
attach <servername>
Onceyouhaveattachedtothesharedmemoryregion,runthefollowingcommands:
log on <not-yet-existing file>
set display off
sample count=150 interval=200
set display on
log close
quit
SybaseASE15BestPractices:QueryProcessing&Optimization
38
Version1.1May2009
Theoutputwillbeinthefilespecifiedinthe"logon"command.
Inaddition,youshouldrunsp_sysmon(fora1or2minuteinterval).SubmittheoutputfrombothSybMonand
sp_sysmontoTechSupport.
UploadingdiagnosticstoSybaseTechSupportthroughFTP
Oncetherequireddiagnosticshavebeencaptured,firstopenacasewithSybaseTechSupport.Then,usethe
followinginstructionstouploaddiagnosticstotheSybaseftpsite:
ftp ftp.sybase.com
user: anonymous
pwd: <your email address>
cd /pub/incoming/wcss
mkdir <your case number>
cd <your case number>
bin
put <memory dump filename>
quit
SybaseASE15BestPractices:QueryProcessing&Optimization
39
Version1.1May2009
ConclusionsAndRecommendations
Themainconclusionsandrecommendationsfromthisdocumentarethefollowing.Pleaserefertothementioned
pagesfordetails:
ThequeryoptimizerinASE15issignificantlydifferentfromtheoptimizerinpriorASEreleases,offering
greatperformancebenefits,butalsointroducingnewtuningaspects(seepages321).
ForcustomerswhoprefertolimittherequiredtestingeffortforupgradingtoASE15,'Compatibility
Mode'maybeusedinASE15.0.3ESD#1.However,thismayresultinnotbeingabletobenefitfromthe
queryprocessingenhancementsinASE15(seepage12).
DonotinitiallyuseparallelprocessingwhenupgradingtoASE15,evenwhenparallelismwasusedinASE
12.x.ASE15maywelldeliverbetteroverallperformanceinserialmodethanpre15inparallelmode,
especiallyforDSStypequerieswithoptimizationgoalallrows_dss(seepage8).
MoreprocedurecachewillbeneededinASE15thanin12.5;expecttorequire2to3timesasmuch
procedurecacheasinpre15(seepage14).
UptodatestatisticsaremorecriticalforgoodperformanceinASE15thaneverbefore.Preferably,run
updateindexstatistics,andifneeded,configureadditionalhistogramsteps(seepage21).
Sybaserecommendstestingyourapplicationsandqueriesextensivelybeforeupgradingyourproduction
systemtoASE15.Forbestresults,testonarealisticproductiondatabasewithactualproductionqueries,
usingrealisticworkloadlevelsandreallifeconcurrency(seepage22).
Whenencounteringunexpectedqueryprocessingissues,upgradetothelatestESDavailable;Sybaseis
makingcontinuousimprovementsandyourissuemayalreadyhavebeenfixed(seepage12).
EnjoySybaseASE15!
SybaseASE15BestPractices:QueryProcessing&Optimization
40
Version1.1May2009
ReferenceDocuments
Technicalwhitepaper"ChangestoscopeandsemanticsofsessionleveloptimizationsettingsinASE15.0.2":
http://www.sybase.com/detail?id=1056206
Technicalwhitepaper"Using'CompatibilityMode'InASE15.0.3ESD#1ForASE15Migration":
http://www.sybase.com/detail?id=1063556
Technicalwhitepaper"RequiredSQLcodechangeswhenmigratingtoASE15":
http://www.sybase.com/detail?id=1063534
SybaseASE15BestPractices:QueryProcessing&Optimization
41
Version1.1May2009
CONTACTINFORMATION
ForEurope,MiddleEast,
orAfricainquiries:
+(31)346582999
ForAsiaPacificinquiries:
+85225068900(HongKong)
ForLatinAmericainquiries:
+7707773131(Atlanta,GA)
SYBASE,INC.
WORLDWIDEHEADQUARTERS
ONESYBASEDRIVE
DUBLIN,CA945687902USA
Tel:18008SYBASE
www.sybase.com
Copyright2009Sybase,Inc.Allrightsreserved.UnpublishedrightsreservedunderU.S.
copyrightlaws.Sybase,andtheSybaselogoaretrademarksofSybase,Inc.oritssubsidiaries.
Allothertrademarksarethepropertyoftheirrespectiveowners.indicatesregistrationinthe
UnitedStates.Specificationsaresubjecttochangewithoutnotice.2/09.