Академический Документы
Профессиональный Документы
Культура Документы
Databases
User Guide
6.8.1
Copyright
2012 Quest Software, Inc.
ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
Patents
Toad for Cloud Databases contains patent pending technology.
Trademarks
Copyright
Quest, Quest Software, the Quest Software logo, Benchmark Factory, Spotlight, LiteSpeed,
Simplicity at Work, Toad, TOAD, T.O.A.D., and TOADWORLD and vToadare trademarks and
registered trademarks of Quest Software, Inc. For a complete list of Quest Softwares trademarks,
see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered
trademarks used in this guide are property of their respective owners.
Table of Contents
Copyright
Getting Started
17
Getting Started
17
17
17
17
17
18
18
20
20
20
21
21
22
25
26
27
35
BenchmarkFactory Console
35
Agent(s)
36
Run Reports
37
38
38
39
40
Table of Contents
Repository Manager
41
Views
44
Script View
44
45
Properties View
47
Agents View
47
Output View
47
Properties
47
How Do I...
91
Overview
91
How do I...
91
92
92
Overview
92
Edit a Profile
94
94
Run Reports
96
97
97
98
99
100
101
103
103
105
BFScript Wizard
105
Table of Contents
Script Wizard
105
Script Assist
107
107
Overview
107
Overview Tab
109
109
Transactions
109
Real-Time
110
Summary
110
Messages
111
112
113
113
115
Overview
115
115
124
125
125
Overview
125
126
126
136
Schedule A Job
136
About Latency
137
138
140
Table of Contents
140
140
141
141
142
142
Overview
Settings
142
144
About Settings
144
General Settings
144
Benchmarks Settings
146
Timing Settings
147
148
LatencySettings
150
Repository Settings
152
Agent Settings
154
Profiles
156
Overview
156
Edit Profiles
156
Drivers
158
Profile Properties
163
Overview
163
General Tab
163
Options Tab
165
166
Clustering Tab
169
Table of Contents
Wizards
171
Overview
171
171
171
173
173
174
174
175
175
176
192
193
195
196
197
200
201
202
203
205
206
207
208
Mix Test
208
Replay Test
209
Goal Test
210
Scalability Test
212
Table of Contents
213
213
BFScript
BFScript Wizard
214
214
217
217
222
222
Transaction Wizard
224
Transaction Wizard
224
225
226
227
231
Import Transactions
232
233
233
233
234
235
235
236
237
237
237
238
239
239
10
Table of Contents
239
Summary Tab
242
242
244
Transaction Node
246
248
Run Reports
248
251
254
256
257
270
274
Benchmarks
276
Overview
276
276
276
Provided Benchmarks
276
AS3AP Benchmark
Best Practices
277
277
278
278
Scaling Factor
278
Best Practices
278
TPC-B Benchmark
279
Overview
279
279
Best Practices
279
11
Table of Contents
Scaling Factor
280
TPC-C Benchmark
280
Overview
281
TPC-C Tables
281
282
Best Practices
282
TPC-D Benchmark
283
Overview
283
283
Best Practices
283
Scaling Factor
284
TPC-E Benchmark
284
Overview
284
285
Best Practices
285
Scaling Factor
285
TPC-H Benchmark
285
Overview
285
286
Best Practices
286
Scaling Factor
286
BFScripts
288
About Scripts
288
290
290
$BFCreditCardExp
290
$BFCurrentDate
290
BFCurrentDateTime
291
12
Table of Contents
$BFDate
File Access
$BFFileArray
Global Variable
292
293
293
294
$BFGetGlobalVar
294
$BFSetGlobalVar
294
$BFSetGlobalVarRtn
295
295
$BFAddress
295
$BFAddress2
295
$BFCity
296
$BFCompany
296
$BFCountry
297
$BFEmail
297
$BFFirstName
297
$BFFullName
298
$BFLastName
298
$BFMiddleInitial
298
$BFPhone
299
$BFState
299
$BFZipCode
300
Numerical Manipulation
300
$BFFormat
300
$BFProd
301
$BFSum
301
Random Numbers
302
$BFRand
302
$BFRandRange
302
13
Table of Contents
$BFURandRange
303
$BFURand
303
Random String
304
$BFCreditCard
304
$BFRandList
304
$BFRandMultiList
305
$BFRandStr
306
$BFURandList
307
$BFList
307
String Manipulation
308
$BFAsc
308
$BFChr
308
$BFConcat
309
$BFLeft
309
$BFLen
310
$BFLower
310
$BFMid
311
$BFRight
311
$BFTrim
311
$BFTrimLeft
312
$BFTrimRight
312
$BFUpper
313
Test Info
313
$BFGetVar
313
$BFMaxNode
314
$BFNode
314
$BFNumberOfIterations
314
$BFProfile
315
14
Table of Contents
$BFRunID
315
$BFSetVar
316
$BFSetVarRtn
316
$BFUserCounter
317
$BFUserID
318
$BFUserLoad
318
$BFNextUserload
318
$BFPrevUserload
319
Quick Starts
320
320
320
Overview
320
321
321
331
331
339
Overview
339
339
346
Overview
346
346
353
Overview
353
354
Troubleshooting
363
Agent Connection
363
363
15
Table of Contents
363
364
364
364
Graph Legend
365
Toolbar
366
366
366
366
Load Configuration
366
Save Configuration
366
Set as Default
366
Clear Configuration
367
367
367
Create a SQL 2000 Trace Table Using the SQL Server Profiler
368
Create a SQL 2005 Trace Table Using the SQL Server Profiler
369
370
371
372
373
Support Bundle
375
376
System Requirements
Shortcut Keys
Copyright, Licensing, and Licensing Agreement
Copyright
376
378
380
380
16
Table of Contents
381
382
383
Legal Notice
383
384
384
Appendix:Contact Quest
Contact Quest/About Quest Software
396
396
About Quest
396
396
ContactingQuest Support
396
Index
398
1
Getting Started
Getting Started
Benchmark Factory is an intuitive, easy to use database load testing tool. The following links
provide you with information to quickly start the testing process and gather results.
l
Quick Starts
Quest Benchmark Factory Web Site (contains additional videos and white papers)
18
Getting Started
User Interface for SQLStatements-The user interface for SQLStatements has been
enhanced.
Reports- A performance comparison report has been added to Run History to compare
Oracle runs.
Wizards-The Oracle and SQL Server Capture wizards user interfaces have been streamline
for easier use.
Improved Run Reports User Interface-Benchmark Factory Run Reports has been
improved to provide easier access to test results tables and graphs.
19
Getting Started
Import and Export of Test Results-You can now import and export test results form
Run Reports.
2
Understanding Benchmark Factory
About Benchmark Factory
Benchmark Factory is a database performance and code scalability testing tool that simulates
users and transactions on the database and replays production workload in non-production
environments. This enables developers, DBAs, and QA teams to validate that their databases will
scale as user load increases, application changes are made, and platform changes are
implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and
other databases via ODBC connectivity. The following topics provides an overview of the
Benchmark Factory Environment:
l
Find applications that do not scale well with an increase in the number of users
All test results are collected and stored in the repository for data analysis and reporting.
Benchmark Factory collects a vast amount of statistics, including overall server throughput
(measured in transactions per second, bytes transferred, etc.) and detailed transaction statistics by
21
individual workstation producing a load. You use these statistics to measure, analyze, and
predict the capacity of a system.
22
A script is the user's workspace that allows you to create and save jobs.
l
Scripts are created in the Script View. See "Script View" (page 44) for more
information.
When ready to run, jobs are submitted to the Jobs Queue View. See "Jobs Queue
View" (page 45) for more information.
The Load Scenario Wizard is the starting point for the creation of scripts. See
"Load Scenario Wizard" (page 175) for more information.
Mixed Workload: A Mixed Workload test runs for a specified time at each
predetermined user load level. Each user will run a transaction mix based upon
the weights defined on the transactions tab. For example, if a test has two
transactions, A and B, with A having a weight of one and B having a weight of
four, on average B will run four times for every time A is run once. The run order
will be randomly generated for each user so they are not all running the same
transaction simultaneously. That run order is used for that user each time the test is
performed to ensure reproducible results.
Goal Test: A goal test is used to find maximum throughput or response time
goals. A transaction mix is executed at user load levels, determined by setting a
beginning, ending, and interval value. When run, the specified goal criterion is
evaluated at the end of each iteration and the test ends once the goal or maximum
user load has been reached.
Replay Test: A Replay Test runs multiple transactions with each one running
independently on a specified number of users. The test will run until the defined
number of executions for each transaction or a specified time limit is reached.
23
A transaction is a single unit of work in the testing process, used in load scenarios and
user scenarios. For example, a SQL statement. Click here for procedures to create a
transaction.
Execute a file allows you to execute a file during the running of a job. See "Execute File
Wizard" (page 174) for more information.
24
25
26
27
Note: The Benchmark Factory Quick Starts provide procedures that introduce you to the
Benchmark Factory environment. See "About the Quickstarts" (page 320) for more information.
Agent Settings
28
2. Create a Script
Using the Benchmark Factory graphical user interface, you can create scripts in the Script View.
You open one script at a time when using Benchmark Factory. See "BenchmarkFactory Console"
(page 35) for more information.
All tasks required to complete a Benchmark Factory script can be accomplished using wizards.
The first wizard you will use is the Load Scenario Wizard. The Load Scenario Wizard takes you
through the steps required to create a script and complete a database load test. See "Load
Scenario Wizard" (page 175) for more information.
29
The first step the Load Scenario wizard asks you to perform is to select a load scenario type from
the load scenario dialog. Load Scenarios are an assembled load test comprised of:
l
Transaction: A single unit of work in the testing process, such as retrieving a Web page,
executing a SQL statement, writing a file, or sending an email.
30
After selecting the desired task type, the Load Scenario Wizard will guide you through the steps
for script creation, depending on which type of task you select.
3. Create a Profile
Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test . This information includes, server name, IP address, and user name. You create
profiles using the:
l
Profile Creation Wizard: (See "Profile Creation Wizard" (page 213) for more information.
Edit Profiles (See "Edit a Profile" (page 94) for more information.
31
4. Create a Job
A job holds a load scenario, execution of a file, creation/deletion of benchmark objects, or a
combination of each. Jobs are created in Benchmark Factory when a load scenario is added to a
script. A script can hold as many jobs as you chose to create.
32
33
To change the User Load properties, select the User Load tab and select the desired
number of users.
34
Errors incurred
Transaction Distribution
Any user defined counters (Windows NT/2000 Performance Monitor) added using the
Benchmark Factory Console.
Real time statistics (See "Realtime Statistics Tab " (page 242) for more information.)
Run Reports (See "Run Reports " (page 248) for more information.)
Testing results exported to Excel (See "Export BenchmarkFactory Test Results to Excel"
(page 274) for more information.)
3
Benchmark Factory Components
BenchmarkFactory Console
The Benchmark Factory console implements the database workload testing process. This
interface is where:
l
Scripts are created and initiated. A script is the user workspace that allows you to design
and save jobs.
Jobs are created and submitted to the Jobs Queue View. See "Jobs Queue View" (page
45) for more information.
All script properties are set. See About the Script Node for more information.
Settings can be made. See "About Settings" (page 144) for more information.About
Scripts (page 288)
Users can access test results. See "Benchmark Factory TestingResults" (page 237) for
more information.
Agents are managed. See "Agent(s)" (page 36) for more information.
36
Agent(s)
Benchmark Factory Agent(s) reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent
generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control
hundreds of Agent machines. Each Agent can simulate up to 1500 virtual users, depending upon hardware and test
configuration. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting
independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own
statistics that includes transaction times and the number of times a transaction executes.
37
Run Reports
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark
Factory and provides a comprehensive and detailed collection of database load testing results.
With Benchmark Factory you can drill down into a database to view a wide array of information
and statistics that gives you accurate insight into database performance. Run Reports Viewer
allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page
254) for more information.
Note: Three instances of Run Reports can be viewed at one time.
38
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
39
40
41
Repository Manager
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark
Factory will not work against this repository.
The Repository is a database where all of the test results are stored. Benchmark Factory inserts
test results into the repository and provides an easy way to access the data. By default, the
Repository is a MySQL database that resides on the same machine as Benchmark Factory for
Databases. The Repository can reside on another database server if required. Tto change the
database select the Data Source Name of the ODBC connection for the new database.
Note: If the database structure does not exist on the selected database, a prompt to create the
structure will appear when OK is clicked.
The supported repository database types are:
l
MySQL
Oracle
42
Section
Connection
Parameters
Field/Icon
Description
Data Source
Name
ODBC Driver
User Name
Password
43
Section
Field/Icon
Description
Maintenance
44
Views
Script View
The Script View displays the currently open script and provides user workspace that allows you
to design and save jobs. When ready to run, jobs are submitted to the Jobs Queue. The Script
View Toolbar provides user functionality when creating or editing load testing scripts.
Note: You must be in the script view to create jobs and edit jobs.
Creates a blank job. You must click on the Script Node to active the icon.
45
Scheduled: All jobs currently waiting to run or are scheduled to run a future time.
46
Properties View
The Properties View allows you to monitor and managed scripts. Properties can be
viewed for the:
l
Jobs Node
Agents View
The Agent view displays the status of connected agents.
Output View
The Output window displays errors and messages that occur outside of running a job. For
example, a .dll error or re-registering a benchmark.
Properties
About the Properties View
Benchmark Factory properties allow you to:
l
47
48
Access Benchmark Factory Run Reports See "Run Reports " (page 248) for more
information.
View realtime statistics (See "Realtime Statistics Tab " (page 242) for more information.
Set and view latency See "About Latency " (page 137) for more information.
Latency (See "About Latency " (page 137) for more information.)
Benchmark Properties See "Benchmark Properties" (page 87) for more information.)
Jobs Node
Note: In the Jobs Queue view, not all properties are editable.
The Jobs node provides the following tabs:
49
General
Alerts
Job Schedule
Real-Time Counters
Transaction Tab
Timing Tab
Settings
Description
Name
Weight
Percentage
Total
Weight
Delete transactions
Adjust latency (See "About Latency " (page 137) for more information.)
50
Adjust latency (See "About Latency " (page 137) for more information.)
51
5. Under Session Details, click on the SQL statement you wish to edit. That statement
displays in the SQL Statement view.
52
6. From the Prepare SQL tab, you can click in the upper right-hand corner to:
l
Run SQL Preview See "Running SQL Preview" (page 113) for more
information.
Launch the BFScript Wizard See "BFScript Wizard" (page 214) for more
information.
7. After you finish making the required edits, click File| Save to save your edits.
53
Transaction Node
Note: In the Jobs Queue view, no properties are editable.
The transaction node displays properties based on transaction type.
To view the Transaction node
54
55
Section
Field/Icon
Description
Repository Settings
Error Handling
56
Section
Field/Icon
Description
reported.
Database Checkpoints
File Name
Note: This field accepts
BFScripts.
Enforce Timeout
File Name
Initiates a database
checkpoint at the beginning
of a test iteration.
Initiates a database
checkpoint during a test
iteration.
Number of checkpoints
57
Setting
Description
Repository Settings
Save Distribution
Error Handling
Run Reports
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark
Factory and provides a comprehensive and detailed collection of database load testing results.
58
With Benchmark Factory you can drill down into a database to view a wide array of information
and statistics that gives you accurate insight into database performance. Run Reports Viewer
allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page
254) for more information.
Note: Three instances of Run Reports can be viewed at one time.
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
59
60
61
62
Field
Description
Benchmark
Scale
63
Field
Description
Displays the Benchmark Properties page and allows you to edit the
benchmark scale and map database tables and indexes. See
"Benchmark Properties" (page 87) for more information.
Always delete
objects before
create
Distribute load
using Benchmark
Factory Agents
Number of virtual
users to perform
creation/load
64
Field
Description
Execute File
to open a file.
Enforce Timeout
Enter a number
between 30 and 10000
[n] Secs.
Scheduled
When a job is scheduled to run, it is placed in the Scheduled Jobs Queue.
Note: Not all of the tabs are editable in the schedule section.
The Job Queue provides the following tabs:
l
Running
As a job runs, clicking on the Run Job tab allows you to view realtime testing results.
65
66
Completed
Jobs in the Completed Jobs Queue section allow you to perform limited test edits and completed
testing results using the See Result Tab (page 69).
Note: Not all of the tabs are editable in the schedule section.
The following provides tab details in the Completed tests section:
l
General
Jobs Alerts
Real-Time Counters
67
68
Field
Description
Benchmark
Scale
Displays the Benchmark Properties page and allows you to edit the
benchmark scale and map database tables and indexes. See
"Benchmark Properties" (page 87) for more information.
Always delete
objects before
create
Distribute load
using Benchmark
Factory Agents
Number of virtual
users to perform
creation/load
Schedule
Sort test results by successful completion or those that completed with errors.
69
70
4. To delete a test, select the test and click Delete in the upper right section of the dialog.
71
5. To view Run Reports, select the required test and click the Show Test Results button in
the upper right section of the dialog.
Transaction Tab
Timing Tab
Delete transactions
Adjust latency (See "About Latency " (page 137) for more information.)
Transaction Node
Note: In the Jobs Queue view, no properties are editable.
The transaction node displays properties based on transaction type.
To view the Transaction node
72
3. Click Add.
The new global variable dislays.
73
74
Run Reports
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark
Factory and provides a comprehensive and detailed collection of database load testing results.
With Benchmark Factory you can drill down into a database to view a wide array of information
and statistics that gives you accurate insight into database performance. Run Reports Viewer
allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page
254) for more information.
Note: Three instances of Run Reports can be viewed at one time.
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
75
Total Errors
Total Rows
Response Time
76
77
Summary Tab
The Summary Tab properties displays information on the job when it is running. You can
monitor the status of the test, which userload its running, transactions per second (TPS), and a
variety of other datapoints that give you insight into the actual performance your database
provides under the userload being tested.
Iteration Overruns
Iteration overruns occur at the end of an iteration to allow time for all transactions submitted
within the test iteration cycle to complete, so that all transaction statistics can be collected. For
example, an agent may execute a transaction during the last five seconds of test iteration, if this
transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur.
78
2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.
79
80
3. Click OK. The graph displays with the selected datapoints. (To change graph views,
right-click. See "Change Graph Views" (page 364) for more information.)
2. Click the datapoints Vs. Userload drop-down and select the desired datapoint.
3. To change graph views, right-click. See Change Graph Views for more information.
81
82
2. Click the datapoints vs. Userload drop-down and select the required datapoint.
3. To change graph views, right-click. See "Change Graph Views" (page 364) for more
information.
Messages Node
The Messages node displays:
l
Job status
83
84
Latency
About Latency
Latency is the speed in which transactions are sent to a server or the period between receiving
results of one transaction and sending the next transaction to a serve. Benchmark Factory allows
you to select the following latency models and distribution models:
Latency Types
Distribution Models
No Delay
No Delay means that transactions execute as fast as possible. As soon as one transaction is
processed, the next transaction is issued against the server. In the case of a mixed workload test,
each virtual user issues transactions as fast as possible. The No Delay option is used when the
goal of the test is to stress the system to its limits, without concern for accurately simulating
85
users. With No Delay specified, a relatively low number of users can stress the system to its
limits. As such, there is no easy way to correlate N virtual users running with no delay to some
number of real users.
Interarrival Time
Interarrival Time derives its name from achieving a specific transaction rate as seen by a server
(the rate at which transactions are arriving at a server.) When specifying Interarrival Time,
Benchmark Factory is instructed, that regardless of how long a transaction actually takes to
execute, to ensure that the transactions arrive at a server at a specific interval.
Keying Time
Keying Time is used to simulate an amount of time performing data entry, before executing a
transaction (entering information). In many cases, Keying Time is used with Think Time to
provide a delay before and after a transaction execute. Selecting Keying Time inserts a fixed or
variable delay before executing a transaction.
Think Time
Think Time is used to simulate an amount of time "to think" about the results of a previous
transaction. This could be time spent performing analysis on the results of a database query.
Selecting Think Time inserts a fixed or variable delay after executing a transaction.
86
Absolute Distribution
Absolute is a fixed length delay. If a 2000 ms delay is specified for Keying Time (page 85) and a
3000 ms delay is specified forThink Time (page 85), each time the transaction should execute,
Benchmark Factory waits 2000 ms, then fires the transaction, then waits an additional 3000 ms
before deciding which transaction to execute next. If an Inter-Arrival model is chosen with a
delay type of Absolute set to 2000 ms, Benchmark Factory marks the time, executes the
transaction, and assuming the transaction finishes in less than two seconds, waits until two
seconds from the marked time has elapsed before determining which transaction to execute next.
Uniform Distribution
Selecting a Uniform delay instructs that random delay should be chosen, with an equal
probability of being the minimum value, the maximum value, or any value in between. Uniform
delays are chosen when it is suspected that the delay is highly random within a range or a
minimal amount of statistical analysis has been performed to determine how the actual users
react. Suppose a uniform Keying Time (page 85) is selected with a minimum value of 1000 ms
and a maximum value of 1500 ms. If the transaction is executed more than 500 times, there is a
high probability that each possible delay has been selected at least once. With the other delay
types, this is not the case. If the same 2000 ms to 2500 ms uniform delay is set for Interarrival
time, the tester essentially is setting the test so that a server sees the transaction every 2 to 2.5
seconds, instead of exactly two seconds as in the Absolute delay time.
Normal Distribution
Normal distributions differ from Uniform delays in that most of the delays chosen by Benchmark
Factory will be close to the average, but can vary by as much as 10% of the mean. While a
Uniform delay is used when users have latencies within equal likelihood of being anywhere
between two values, Normal distributions are chosen when all users fall within a range, but most
of the modeled users have a latency close to the average latency.
87
Poisson Distribution
A Poisson distribution is very similar to the Normal distribution and can be used most places
where a Normal distribution delay could be used. The biggest difference between a Normal
distribution and a Poisson distribution is that Poisson selects discreet values.
Benchmark Properties
Benchmark Properties
Benchmark scale factors are used to increase the size of the database testing tables during the
creation of benchmark objects, and to correctly execute the benchmark transactions as needed to
fully utilize the database objects created.
The Benchmark Properties page displays information on:
l
Scales
The scale dialog specifies the benchmark scale used to create table data
88
Field
Description
Benchmark Scale
Show Empty
Tables
Displays any tables that will be created but not populated with
data.
Estimated Size
The approximate size that the tables will use in the database
without indexes. Additional space is needed for indexes.
89
Advanced
Advancer mapping provides views of the tables and indexes that allows you to determine where
there tables and indexes will be stored in the database. This allows your database to run more
efficiently. See "Advanced Creation Objects (Object Mapping)" (page 100) for more information.
History Tables
History tables allows you to set the number of history tables to create. The TPC-B benchmark is
made up of only one transaction that updates three tables and inserts a record into a history table.
Inserting one record into one history table limits testing performance. The Benchmark Factory
properties page allows the user to set the number of history tables to create during a test. The
best ratio of history tables to virtual users is based on database configuration and hardware. The
number of history tables to use is determined by the tester.
Field
Description
Number of History
tables
90
4
How Do I...
Overview
The How Do I section points you to the procedures required to perform load testing.
How do I...
l
Create a Load Scenario - This wizard walks you through the steps to create and run
scripts and load scenarios.
Create a User Scenario - This wizard walks you through the steps to create a
user scenario.
Create a Profile - This wizard walks you through the steps to hook up to the data base
you are testing.
Edit a Profile
Submit a Job - This wizard walks you through the steps to submit a job to the Jobs
Queue for testing.
Create Benchmark Objects for Load Testing This wizard walks you through the steps to
create benchmark objects for load testing.
Execute a File - This wizard walks you through the steps to execute a file outside of
Benchmark Factory.
Use Benchmark Factory Scripting Capabilities - This wizard walks you through the steps
to use Benchmark Factory built-in functions for use in scripts and the load testing process.
92
How Do I...
Set Latency
93
How Do I...
94
How Do I...
Edit a Profile
To edit a profile
1. Click the Edit Profile icon
The Edit Profiles dialog displays. The following options are provided.
l
Adds a profile
Deletes a profile
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Actions
95
96
How Do I...
Section
Description
Comments
Alerts
Job Counters
Run Reports
Benchmark Factory Run Reports is a separate executable that opens outside of Benchmark
Factory and provides a comprehensive and detailed collection of database load testing results.
With Benchmark Factory you can drill down into a database to view a wide array of information
and statistics that gives you accurate insight into database performance. Run Reports Viewer
allows you to access Benchmark Factory load testing results. See "Run Reports Viewer" (page
254) for more information.
Note: Three instances of Run Reports can be viewed at one time.
97
How Do I...
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
98
How Do I...
99
How Do I...
100
101
How Do I...
102
How Do I...
3. Click Next.
4. From the drop-down, select the desired benchmark.
5. Click Next.
6. Select the desired benchmark scale in Benchmark Scale. When setting up a load test,
Benchmark Factory allows you to change the scaling factor of the standard benchmarks
provided. Scales factors increase the size of a database during the testing process allowing
realistic testing. By the scale factors, the number of rows added to the tables increases,
allowing for larger userloads to be used, placing a greater stress on the system under test.
During load testing, you can run various test with numerous scaling factors to ensure your
database performs at various performance levels.
7. The Advanced Creation Option allows you to set advanced properties for tablespaces and
datafiles, used for the creation of standard benchmark tables and indexes. To add mapping
options click Advanced Creation Option or proceed to step 18.
8. From the Database Type drop-down, select the desired database type.
9. Select the Tables or Indexes radio button.
10. Select the object(s) you wish to map.
Create Benchmark Objects: Creates and initializes benchmark objects (tables, indexes,
etc.) required to run the standard benchmark transactions.
103
How Do I...
Always delete objects before create: Deletes existing benchmark objects (tables,
indexes, etc.) before creating new ones.
Note: Checking the "Always delete objects before create" check box will delete all
benchmark objects in the selected database and then re-create the objects in the
databases. When the checkbox is unchecked, Benchmark Factory checks for the
benchmark tables. If they exist, then the creation will be skipped. The scale factor is
not checked.
Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
TPC-H
The Benchmark properties page allows you to change scale factors properties.
104
How Do I...
2. In the properties view for the Benchmark Object, select Edit. The Benchmark Properties
page displays.
3. Set the desired scale in Benchmark Scale.
4. Click OK.
105
How Do I...
BFScript Wizard
Note: Controls with yellow background can be scripted.
Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be
utilized by selecting the SQL Statement node and inserting the script in the properties view. A
field has scripting capabilities if its right-click menu contains the Script Wizard option.
The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard
consists of a script category and a script function. Each script function has a short description
included and if applicable, its parameters.
There are two features within Benchmark Factory that assist you when using scripting
capabilities:
l
Script Wizard
Script Assist
Script Wizard
1. Right-click the edit control. A drop-down display.
106
How Do I...
2. Select BFScript Wizard from the drop-down. The Script Wizard displays.
107
How Do I...
5. Click Finish. The script and parameters display in the Script Wizard.
Script Assist
1. Type in the desired SQL statement. When $BF is typed the Script Assist is activated
allowing you to insert the desired script.
Overview
The SQL Scalability test allows you to execute SQL statements, letting users spot potential
issues not seen with a single execution. Users can run variations of a SQL statement generated
by SQL Tuning in order to find the SQL that will perform the best under a load test.
Creating a SQL Scalability load scenario requires the following steps.
108
How Do I...
109
How Do I...
13. Click Finish. The SQL Scalability dialog displays. From this dialog, you can control,
monitoring, and view SQL testing results. This dialog provides the following tabs
and buttons:
Overview Tab
The Overview tabs provides transactions per second testing results for individual user loads and
iterations.
Transactions
The Transactions view provides the following statistics on the SQL statements being run:
l
Executions
Rows
Bytes
Errors
110
How Do I...
Real-Time
Real-Time Statistics provides real-time graphs and raw data. This data allows you to spot systemunder test issues that may be affecting server performance. Right-clicking inside the graph
displays a drop-down that allows you to change graph settings and view.
Summary
The Summary graph provides real-time and raw data summary graphs. The Summary graph
displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows
you to change graph settings and view.
Messages
The Output window displays messages about the job being run.
111
112
How Do I...
Replay test
A Replay Test runs multiple transactions with each one running independently on a specified number of users. The
test will run until the defined number of executions for each transaction or a specified time limit is reached.
Mixed test
A Mixed Workload test runs for a specified time at each predetermined user loads level. Each user will run a
transaction mix based upon the weights defined on the transactions tab. For example, if a test has two transactions,
A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every
time A is run once. The run order will be randomly generated for each user so they are not all running the same
transaction simultaneously. That run order is used for that user each time the test is performed to ensure
reproducible results.
Goal test
A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load
levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is
evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached.
Scalability
A SQL Scalability test executes each transaction individually for each userload and timing period. For example, a
test has two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute.
Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run
at userload 20, followed again by test B, for a total time of 4 minutes.
Copying a load scenario creates a new load scenario containing all compatible settings, user
scenarios, and transactions from the original test.
To copy a Load Scenario to a Replay Test, Mix Test, or Goal test
1. Right-click the desired Load Scenario in the Script View. A drop-down displays.
2. Select the desired type of test you want to copy to.
3. The test creates and displays in the Script View.
113
How Do I...
4. Type the appropriate maximum number of rows to preview into Maximum Returned, or
check the All box to preview all rows.
5. Click OK. The SQL Preview window displays.
6. Review the information.
7. Click Close.
4. Click OK. The latencies replace the previously set latencies for the children of the
selected Load Scenario or User Scenario.
Note: The latencies grandchildren of the selected node will not be changed.
114
115
How Do I...
Overview
This quick start presents a load testing methodology to run a goal test that finds the maximum
Transactions Per Second (TPS) of a database, by adding a SQL Statement Transactions to the
goal test, then viewing real-time and stored testing results to determine if your database lives up
to the required performance parameters.
A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process,
used in load scenariosand user scenarios. For example, a SQL statement. based on userload
intervals. The userloads are determined by setting a beginning, ending value, and interval value.
This value specifies an interval to increase the number of users tested for each iteration of the
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and
the test ends once the goal or maxim user load has been reached.
116
How Do I...
Find maximum TPS: Selecting this option will show how the maximum TPS can
be reached during a load test.
Find maximum userload where response time is less than [n]ms: Selecting this
option shows how a maximum userload can be reached when response time is less
than a keyed in value.
117
How Do I...
10. Select the desired option. For this quick start, the Find maximum TPS is selected.
11. Set the desired Start, Ending, and Interval in Userloads. For this test, the defaults values of
1, 5, and 1 are used. When run, User Loads of 1, 2, 3, 4, 5 are run until the TPS of one
run is less than the TPS of the previous run.
12. Click Next. The Define Transaction Mix Page displays.
13. Click Next. The Enter Script Name dialog displays and provides the following options:
2. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you
to add the load scenario you are building to the current script file you are
working with.
14. Click Create a new Benchmark Factory Script file.
15. Click Next. The wizard completion dialog displays.
16. Click Finish. The Scripts creates and displays in the Script View.
Setting Script Timing
1. Click the Max TPS (1-5 by 1) node.
118
How Do I...
119
How Do I...
120
How Do I...
121
How Do I...
4. Click Next.
5. From the drop-down, select the AS3AP standard benchmark.
6. Click Next. The Benchmark Scale displays.
7. Set the desired scale in Benchmark Scale.
8. Click Next. The Create/Delete benchmark objects dialog displays.
9. Click Next. The wizard completion dialog displays.
10. Click Finish.
11. You must create the benchmark objects before running the script. Right-click Create
Objects for AS3AP.
12. Select Move Up.
13. Move the Benchmark Object to the top node in the script view.
2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
122
123
How Do I...
Section
Description
Actions
Comments
Alerts
Job Counters
2. Expand the Run Status node. Run Status provides a real-time status on the test currently
running in the Job's Queue.
3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time
graphs and raw data. This data allows you to spot system-under test issues that may be
affecting server performance. The Real Time Statics node displays real-time counter data.
124
How Do I...
4. Click the Summary node. The Summary Graph displays. The Summary page displays user
selected statistics for the job timing period following completion of an iteration.
5. Click the Transaction Graph node. The Summary Graph displays. The transaction node
displays properties based on transaction type.
Viewing Run Reports Results
1. When the job completes, click the Run Reports node.
125
How Do I...
126
How Do I...
When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes
continue to run, providing uninterrupted service to users. Oracle RAC distributes database
demands across several nodes, performing the function of load balancing. When more capacity is
required, additional nodes can be added with minimal effort. Organizations spending time and
money setting up a RAC cluster must ensure their systems have the stability to withstand user
environment demands when adding nodes. An Oracle Real Application Cluster (RAC) combines
several independent servers/nodes into one database system. RAC is a clustered database solution
that provides two major functions: scalability and availability to a business continuum.
Scalability is a relative term, based on a simple rule that as the number of users accessing the
system increases, the RAC configuration should be able to handle increased activity. Availability
is the ability of the system to provide continuous service when one or more of the components in
the cluster fail.
Two important criteria used to achieve these goals focus on the cluster being able to load
balance across all nodes, and when one or more nodes in the cluster fails, the users failover.
When a node in an Oracle RAC cluster fails, an Oracle database and the remaining nodes
continue to run, providing uninterrupted service to users. Oracle RAC distributes database
demands across several nodes, performing the function of load balancing. When more capacity is
required, additional nodes can be added with minimal effort. Organizations spending time and
money setting up a RAC cluster must ensure their systems have the stability to withstand user
environment demands when adding nodes.
127
How Do I...
"MyTNS1" connections are duplicates and are required for Benchmark Factory to perform load
testing. MyTNS2 and MyTNS3 are for the other two machines in the cluster. Figure 1
provides an excerpt from a tnsnames.ora file set up for a three node cluster.
128
How Do I...
Note: Clicking the Advanced tab allows you to map benchmark objects. For these
procedures, please see Advanced Creation Objects (Object Mapping).
129
How Do I...
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
130
How Do I...
which in turn reduces downtime, development costs, and potential loss of revenue.
Benchmark Factory allows you to import a variety of user loads that include:
l
Imports from various tools such as Quest Performance Analysis SQL from the repository
for replay.
131
How Do I...
132
How Do I...
133
How Do I...
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Actions
Comments
134
How Do I...
Section
Description
Alerts
Job Counters
135
How Do I...
2. You can select a single test or multiple test to view testing results. In this example,
multiple tests are selected to view the RAC testing results.
4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.
5. To view other testing results, click the desired result from the User Load drop-down.
136
How Do I...
Schedule A Job
Benchmark Factory allows you to run a job immediately after creation or schedule a job to run at
a future time. Scheduling allows you to set predetermined dates, recurrences, daily frequencies,
and durations of jobs.
Note: When using the recurring option, use Ready Job in place of Run Job.
Click here for definitions on scheduling parameters.
To schedule a job
1. Click the Job node.
2. Click the Schedule tab.
3. Check the Enabling check box.
4. Enter the desired date you want to start the job in On date.
5. Enter the desired time in At time.
6. If you wish only to run the job once, submit the job. If you wish to run the job on a
recurring basis, proceed to step 7.
137
How Do I...
15. Submit the Job if it is in the Script View or Ready the job if it is in the Jobs Queue by
clicking the Ready Job button on the Jobs Toolbar.
About Latency
Latency is the speed in which transactions are sent to a server or the period between receiving
results of one transaction and sending the next transaction to a serve. Benchmark Factory allows
you to select the following latency models and distribution models:
Latency Types
Distribution Models
138
How Do I...
Latency Types
Distribution Models
85)
139
How Do I...
3. Click OK.
To set the number of users from the Agent View in the Benchmark Factory Console
1. Right-click on the Agent in the Agent view of Benchmark Factory. A drop-down
displays.
2. Select Settings. The Agent Settings dialog displays.
140
How Do I...
Start all users as quickly as possible: Starts all users immediately after a test begins.
Start all users at even intervals for: Sets the length of time to start the users in. The
value divided by the number of new users of new users to start in the iteration,
determines how often a user is started.
Start a new user every: Starts a new user, then waits the entered number of seconds
before starting the next user.
Load the proper client libraries (i.e. Oracle, SQL server, etc) on the Benchmark Factory
Console and Agent Machine.
Install Benchmark Factory for Databases on the console and agent machines.
141
How Do I...
5
Creating Load Scenarios
Load Scenario Wizard
Overview
Note: To create load scenarios, you must be in the Script View.
The Load Scenario Wizard is a starting point for creating Benchmark Factory scripts and walks
you through the steps to build a complete workload testing script.
Note: Only one Load Scenario can be run at a time.
From this wizard you can test:
l
Capture andReplay Oracle Workload See "Capture and Replay Oracle Workload" (page
176) for more information.
Replay load from an Oracle trace file (See "Replay Load from an Oracle Trace File" (page
197) for more information.)
Validate Oracle RAC Cluster Scalability (See "Validate Oracle RAC Cluster Scalability"
(page 193) for more information.)
Test Database Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test SQL for Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test Stored Procedures for Scalability (See "Test Stored Procedures for Scalability" (page
225) for more information.)
Capture and Replay SQL Server Workload (See Capture and Replay SQL Server
Workload for more information.
Replay load from SQL server trace table (See "Replay Load from SQL Server Trace Table"
(page 201) for more information.)
Import from Foglight Performance Analysis (See "Import from Foglight Performance
Analysis Repository" (page 202) for more information.
Import from a Delimited Text file (See "Import from a Delimited Text File" (page 203) for
more information.)
Replay load from an ODBC trace file (See "Replay Load From an ODBC Trace File"
(page 205) for more information.
143
Import from Oracle Dynamic Performance View (See "Import from Oracle Dynamic
Performance View" (page 206) for more information.)
Import from SQL files (See "Import from Quest Tools Export Files" (page 207) for more
information.)
Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario
(Scalability, Goal...)" (page 208) for more information.)
6
Settings
About Settings
The Settings window displays default values used when creating new scripts. Changes to these
settings affect only new scripts, not existing scripts.
To Display Benchmark Factory Settings
l
General
Benchmarks
Timing
User Load
Latency
Repository
Agent
General Settings
The General settings dialog specifies workplace settings and error handling options.
145
Settings
Section
Field
Description
Workplace
Settings
Error Handling
Directories
Script Files
Error Logs
146
Settings
Section
Retrieval
Options
Field
Description
Data Files
Benchmarks Settings
The Benchmarks dialog specifies benchmark properties and options.
Section
Benchmarks
Options/Icons
Description
Show Properties button.
Displays benchmark properties.
Benchmark Implementation
147
Settings
Timing Settings
The Timing dialog specifies timing phases associated with an item.
Note: Setting the timing phase sets the default value for a new load scenario.
Section
Field
Property Definition
Time Phase
Pre-Sampling
148
Settings
Section
Field
Property Definition
before statistics are collected.
User Startup
Sampling
149
Settings
Note: Setting the user load sets the default value for a new load scenario.
Section
Field/Icon
Description
Determines user load to add to the
selected list.
User Load(s)
Add a Range of
User Loads
150
Settings
Section
Field/Icon
Description
Start
End
Interval
LatencySettings
The Latency settings dialog specifies the speed in which transactions are sent to the server to model
real world user interactions.
Note:Setting the latency sets the default value for a new load scenario.
Latency Type:
Distribution Models
No Delay
Absolute
Interarrival Time
Uniform
Keying Time
Negative Exponential
Think Time
Normal
Poisson
151
152
Settings
Repository Settings
Note: If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark
Factory will not work against this repository.
The repository is a database where all of the test results are stored. Benchmark Factory inserts test
results into the repository and provides an easy way to access all test results data. By default, the
Repository is a MySQL database that resides on the same machine as Benchmark Factory for
Databases. The Repository can reside on another database server if required, to change the
database select the Data Source Name of the ODBC connection for the new database.
Repository settings allows you to edit DSN, perform ODBC administration, and test the
connection. Benchmark Factory also provides a Repository Manager and Data Repository
Migration wizard to assist you with repository management.
Note: If the database structure does not exist on the selected database, a prompt to create the
structure will appear when OK is clicked.
The supported database types are:
l
MySQL
SQL Server
Oracle
153
Settings
Section
Connection
Parameters
Field/Icon
Description
Data Source
Name
User Name
Password
154
Settings
Section
Field/Icon
Description
Agent Settings
The Agent Settings tab allows you to manage Agent startup, shutdown, and maximum virtual
user per agent settings.
Section
Field
Description
Agent
Management
Start agents
automatically
Shutdown agents
automatically
155
Settings
Section
Field
Description
Maximum virtual
users per agent:
7
Profiles
Overview
Profiles are sets of information that Benchmark Factory uses to communicate with the systemunder-test. This information may include, server name, IP address, and user name. You create
profiles using the Profile Creation Wizard.
Edit Profiles
Benchmark Factory Profiles can be created, deleted, and changed to meet script requirements.
Click here for procedures on editing profiles.
Buttons
Description
Creates a new profile using the Profile Creation Wizard.
Deletes the highlighted profile
157
8
Drivers
Driver Selection
Note: Controls with yellow background can be scripted.
Drivers connect Benchmark Factory to the respective database. Benchmark Factory provides the
following database drivers:
l
IBM DB2CLI
MySQL (ODBC)
ODBC
Oracle
Sybase
IBM DB2CLI
The IBM DB2CLI driver requires the following connection parameters:
159
160
MySQL (ODBC)
The MySQL ODBC Server driver requires the following connection parameters:
MySQL ODBC
The MySQL ODBC Server driver requires the following connection parameters:
Oracle
The Oracle driver requires the following connection parameters:
161
Connect Using: Provides a dropdown to select the data base type (for
example, Oracle XE or 11g) to
connect to.
Sybase
The Sybase driver requires the following connection parameters:
Note: BFScripts have been enabled on the
User Name and Password fields.
162
163
Profile Properties
Overview
Profile properties provides the following tabs:
l
General
Options
Real-Time Counters
Statistics Tab
General Tab
The General Tab allows you to enter database parameters, installed client connection parameters,
and test profiles for connection.
164
Section
Field/Icon
Description
Parameters
User/Schema
Password
TNS
Direct
Installed
Clients
Test Profile
Connect as:
Connect Using:
Test Profile
165
Options Tab
The Profile Options tab allows you to adjust timeout and reconnect.
Section
Field/Icon
Description
Timeout
Time
Infinite Timeout
Enable Reconnect
Reconnect
Section
166
Field/Icon
Description
Number of reconnect
attempts
AVG_TIME
BPS
DEADLOCKS
TOTAL_ERRORS
MAX_TIME
RPS
TOTAL_BYTES
TOTAL_ROWS
TPS
USERLOAD
The Real-Time Counters tab displays on the profile and job property pages. The combination of these
settings are used to create a complete list of real-time counters that are recorded during test
execution. From this tab you can:
167
168
169
Clustering Tab
Clustering is the process of using two or more computer systems that work together. Multiple
servers are linked to handle variable workloads or to provide continued operation in the event
one fails. Computers may be multiprocessor systems. A cluster of four computers with four CPUs
each, provides a total of sixteen CPUs processing at one time.
The Clustering tab in Benchmark Factory allows you to perform Oracle Real Application
(RAC)Testing.
170
9
Wizards
Overview
Benchmark Factory wizards streamline the load testing processing and provide the flexibility to
customize testing workloads.
The following wizards are provided:
l
Benchmark Objects
Execute File
Profile Creation
BFScript
Submit Job
Transaction
User Scenario
172
Wizards
173
Wizards
18. Click Next. The Create/Delete benchmark objects dialog displays and provides the
following options:
l
Create Benchmark Objects: Creates and initializes benchmark objects (tables, indexes,
etc.) required to run the standard benchmark transactions.
Always delete objects before create: Deletes existing benchmark objects (tables,
indexes, etc.) before creating new ones.
Note: Checking the "Always delete objects before create" check box will delete all
benchmark objects in the selected database and then re-create the objects in the
databases. When the checkbox is unchecked, Benchmark Factory checks for the
benchmark tables. If they exist, then the creation will be skipped. The scale factor is
not checked.
Delete Benchmark Objects: Deletes existing benchmark objects (tables, indexes, etc.).
174
Wizards
175
Wizards
To execute a file:
1. Select the Job or Load Scenario node.
2. Select Wizards | Execute File.
3. Click Next. The Execute File dialog displays.
4. Enter the path or navigate to the desired file.
5. Click the Enforce Timeout checkbox if desired. Enforce Timeout allows you to enter the
desired number of seconds Benchmark Factory waits for the execution to complete before
continuing the job.
6. Enter the desired number of seconds.
7. Click Next. The completion dialog displays.
8. Click Finish. The Execute File is created and displays in the Script View.
Capture andReplay Oracle Workload See "Capture and Replay Oracle Workload" (page
176) for more information.
Replay load from an Oracle trace file (See "Replay Load from an Oracle Trace File" (page
197) for more information.)
Validate Oracle RAC Cluster Scalability (See "Validate Oracle RAC Cluster Scalability"
(page 193) for more information.)
Test Database Scalability (See "Test Database Scalability" (page 195) for more
information.)
Test SQL for Scalability (See "Test Database Scalability" (page 195) for more
information.)
176
Wizards
Test Stored Procedures for Scalability (See "Test Stored Procedures for Scalability" (page
225) for more information.)
Capture and Replay SQL Server Workload (See Capture and Replay SQL Server
Workload for more information.
Replay load from SQL server trace table (See "Replay Load from SQL Server Trace Table"
(page 201) for more information.)
Import from Foglight Performance Analysis (See "Import from Foglight Performance
Analysis Repository" (page 202) for more information.
Import from a Delimited Text file (See "Import from a Delimited Text File" (page 203) for
more information.)
Replay load from an ODBC trace file (See "Replay Load From an ODBC Trace File"
(page 205) for more information.
Import from Oracle Dynamic Performance View (See "Import from Oracle Dynamic
Performance View" (page 206) for more information.)
Import from SQL files (See "Import from Quest Tools Export Files" (page 207) for more
information.)
Create a custom load scenario (Scalability, Goal...) (See "Create a Custom Load Scenario
(Scalability, Goal...)" (page 208) for more information.)
177
The Choose Database dialog provides the following fields and buttons:
l
Home-Sets the desired database home, for example, 10g. This field is
not required.
7. Click Next.
178
179
Wizards
TheSelect the Capture Method dialog displays. For details on which method to use, see
Pros and Cons of Using Trace or Fine Grain Analysis Files When Testing
withBenchmarkFactory.
Tablespace for capture table-Allows you select the required tablespace for
Fine Grained Auditing captures.
The directory that the Oracle database server uses for its input and output
during workload captures and replays. This directory can be in one of
three locations:
o
180
181
Wizards
On the network file server as a local directory, but that requires that
directory to be reachable by both the database server and the
BenchmarkFactory client PC (Windows network share, Samba, or
NFS mount). This is the recommended scenario for two reasons:
n
182
Diagnostics Pack.
l
183
184
Wizards
Include schemas that are related to the selected for export-Select this checkbox to
ensure any related schemas are included in the export.
Include any schemas owning objects relate to those selected for exportChecking this box includes objects owned by the schema during the
export process.
185
Wizards
To remove a filter:
a. Click on the required filter.
b. Click Remove.
19. Make the required selections and click Next.
The Capture Control dialog displays.
186
187
Wizards
Scheduled (via database schedule)-Allows you to set a date and time to begin the
job and enter a description.
Capture Automatically Stop After-Allows you to stop the data capture in hours
and minutes.
188
Wizards
189
Wizards
190
Wizards
Select a profile for the target server-Allows you to select a server to replay the
workload against.
191
192
Wizards
32. To view full test results, click For full test results, click here.
193
Wizards
5. Select the desired benchmark: AS3AP, Scalable Hardware, TPC-B, TPC-C, TPC-D,
and TPC-H.
6. Click Next. The Profile Selection dialog displays.
7. From the Profile drop-down, select the desired profile or click New display the Profile
Creation Wizard and create a profile.
8. Click Next. The Benchmark Scale dialog displays.
9. Select the desired Benchmark Scale.
10. Click the Advanced Creation Option button to map individual benchmark objects.
11. Click OK on the Advanced Creation Option page.
12. Click Next. The Measurement Intervals dialog displays.
13. Select the desired user load.
14. Click Next. The Enter Script Name dialog displays and provides the following options:
2. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
15. Select the desired option.
16. Click Next. The completion dialog displays.
17. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
18. Click Finish. The workload creates and displays in the Script View.
194
Wizards
5. From the Profile drop-down, select the desired profile or click New to display the Profile
Creation Wizard and create a new profile.
6. Click Next. The Edit Scale dialog displays. The Edit Scale dialog allow you to set the
scale and map the objects for the TPC-C and TPC-H benchmarks.
7. If you do not desired to edit either benchmark, click Next and proceed to step 23.
8. To edit the properties of a benchmark, click the Edit button for benchmark (TPC-C or
TPC-H) you wish to edit. The benchmark properties dialog displays.
9. Select the desired scale in Benchmark Scale.
10. To edit the advanced properties of the benchmark, select the Advanced tab. Here you
can map properties for tablespaces and datafiles, used for the creation of standard
benchmark tables and indexes.
11. Check the Use advanced options for creation of benchmarks objects check box.
12. From the Database Type drop-down, select the desired database type.
13. Select the Tables or Indexes radio button.
14. Select the object(s) you wish to map.
15. Click Edit Options. The Mapping Options dialog displays.
16. Select the desired profile from Profile drop-down.
17. Select the desired mapping options.
18. Click OK.
19. To save the mapping options, click Save. The Save Options File As dialog display.
20. Enter the desired name.
21. Click Save.
22. Click OK. The Edit Scale dialog displays.
23. Click Next. The Node Selection dialog displays.
24. Enter the desired number of nodes in Enter maximum number of nodes (2-100).
25. Select the desired load balancing method.
26. Click Next. The Measurement Intervals dialog displays.
27. Select the desired load user loads.
28. Click Next. The Enter Script Name dialog displays and provides the following options:
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
195
Wizards
31. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
32. Click Finish. The workload creates and displays in the Script View.
196
Wizards
Execution time per iteration-Executes each transaction for the specified length
of time.
197
Wizards
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
Note: To create an Oracle trace file which includes bind parameters, a level number of 4 must be
used. For example, to start an Oracle trace for a user session, use the following command:
ALTER SESSION SET EVENTS '10046 trace name context forever, level 4'
1. Select Wizards | Load Scenario.
or
Click New.
The Load Scenario Wizard displays.
2. Click Next.
The Select Load Scenario dialog displays.
3. Click Replay load from an Oracle Trace file.
198
199
Wizards
200
Wizards
If you click Yes, the trace import file import continues. Individual SQL will be
converted to XML files. You can then edited the XML files in the Benchmark Factory
Session Editor. Click the desired option.
b. If the trace import does not exceed the import limit for displaying individual transactions,
this dialog will not display.
15. The Enter Script Name dialog displays and provides the following options:
16. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
17. Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
16. Select the desired option.
17. Click Next. The wizard completion dialog displays.
18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
19. Click Finish. The workload creates and displays in the Script View.
201
Wizards
6. Click Next. The Select Stored Procedures dialog displays. (Click here for stored
procedure examples.)
7. Check the desired procedures.
8. Click Next. If the selected procedures contain bind parameters, the Bind Parameters
dialog displays.
9. From SQL Name, select the desired SQL statement.
10. To add a parameter, right-click on the parameter. A drop-down displays.
11. Click Add.
12. Edit the bind parameter as needed.
13. Click Next. The Measurement Intervals dialog displays.
14. Select the desired user loads.
15. Click Next. The Enter Script Name dialog displays and provides the following options:
2. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
16. Select the desired option.
17. Click Next. The wizard completion dialog displays.
18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
19. Click Finish. The workload creates and displays in the Script View.
Creating a SQL 2000 Trace Table Using the SQL Server Profiler
Creating a SQL 2005 Trace Table Using the SQL Server Profiler ,or
Creating a SQL 2008/2008 R2 Trace Table Using the SQL Server Profiler
202
Wizards
Note: To import a trace file, you will need to use SQL profiler to save the file as a trace table
and then import the table.
To replay a load from SQL Server:
1. Select Wizards | Load Scenario.
or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Select Load Scenario dialog displays.
3. Click Replay load from SQL Server trace table.
4. Click Next. The SQL Server Authentication window displays.
5. Select a host from the Select a host drop-down list or type in the host.
6. Select the desired connection method and fill in the required connection parameters.
7. Click Next. The SQL Server Trace Table dialog displays.
8. Select a database from the Select a Database drop-down list.
9. Select a table from the Select a Table drop-down list.
10. Click Next. The SQL Server Trace Processing window displays.
11. When the processing is complete, click Next. The Enter Script Name dialog displays and
provides the following options:
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
203
Wizards
204
Wizards
you to load test your database using workloads from an existing in-house generic database
applications.
To Import Delimited Text File
1. Select Wizards |Load Scenario.
or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Define load scenario dialog displays.
3. Click Import from a Delimited Text File.
4. Click Next. The Text File Input dialog displays.
5. Click Browse.
6. Navigate to and select the desired file.
7. Click Open.
8. Click Next. The Select file format dialog displays.
9. Click the desired format (Delimited or Fixed field).
1. If Fixed Field is selected, select the desired Row delimiter from the drop-down.
2. Select the desired rows to skip in Skip row.
or
1. If Delimited is selected, select the desired row delimiter, text qualifier, and rows to skip.
2. Click Next. The Specify Column Delimiter dialog displays.
3. Select the desired properties.
4. Click Next. The Fixed Field Column Positions dialog displays.
5. Select the desired properties.
10. Click Next. The Enter Script Name dialog displays and provides the following options:
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
create a and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
205
Wizards
206
Wizards
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
207
Wizards
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
208
Wizards
Mix Test-Runs a transaction mix based upon weights for a specified time at each
predetermined user load level.
Mix Test
A Mixed Workload test runs for a specified time at each predetermined user load level. Each
user will run a transaction mix based upon the weights defined on the transactions tab. For
example, if a test has two transactions, A and B, with A having a weight of one and B having a
weight of four, on average B will run four times for every time A is run once. The run order will
be randomly generated for each user so they are not all running the same transaction
simultaneously. That run order is used for that user each time the test is performed to ensure
reproducible results.
To create a mix test
1. Select Wizards |Load Scenario.
Or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Define load scenario dialog displays.
3. Click Create a custom load scenario (Scalability, Goal, ...).
4. Click Next. The Test Type dialog displays.
5. Click the Mix radio button.
6. Click Next. The Define Transaction Mix dialog displays. You have three testing options
at this point:
209
Wizards
1. You can click Next and create a blank mix test, and proceed to step 10.
2. Add a User Scenario using the User Scenario Wizard by clicking the Add User
Scenario button.
3. Add transaction(s) using the Transaction Wizard by clicking Add a SQL Statement.
7. Select the desired option and follow the wizard steps.
8. After completing the chosen wizard, you are taken back to the Define
Transaction Mix dialog.
9. Add another workload, or Click Next. The Measurements Interval dialog displays.
10. Select the desired virtual user load.
11. Click Next. The Enter Script Name dialog displays and provides the following options:
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
Replay Test
A Replay Test runs multiple transactions with each one running independently on a specified
number of users. The test will run until the defined number of executions for each transaction or
a specified time limit is reached.
To create a replay test
1. Select Wizard | Load Scenario.
Or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Define load scenario dialog displays.
3. Click Create a custom load scenario (Scalability, Goal, ...).
4. Click Next. The Test Type dialog displays.
5. Click the Replay radio button.
210
Wizards
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
Goal Test
A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process,
used in load scenariosand user scenarios. For example, a SQL statement. based on userload
intervals. The userloads are determined by setting a beginning, ending value, and interval value.
This value specifies an interval to increase the number of users tested for each iteration of the
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration and the
test ends once the goal or maxim user load has been reached.
To create a goal test
1. Select Wizards |Load Scenario.
Or
Click New. The Load Scenario Wizard displays.
211
Wizards
Find maximum TPS: Selecting this option will show how the maximum TPS can
be reached during a load test.
Find maximum userload where response time is less than [n] ms: Selecting this
option shows how a maximum userload can be reached when response time is less
than a keyed in value.
212
Wizards
Scalability Test
The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential
issues not seen with a single execution. Users can run variations of a SQL statement in order to
find the SQL that will perform the best under a load test.
To create a scalability test
1. Select Wizards | Load Scenario.
or
Click New. The Load Scenario Wizard displays.
2. Click Next. The Select Load Scenario dialog displays.
3. Click Create a custom load scenario (Scalability, Goal, ...).
4. Click Next.
5. Click the Scalability radio button.
6. Click Next. The Define Iteration Length dialog displays.
7. When running a scalability test, you have two options during the running of the
workload:
l
Execution time per iteration-Executes each transaction for the specified length
of time.
213
Wizards
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
214
Wizards
10. Click the Verify connection before proceeding check box to verify the connection to the
specified database.
BFScript
BFScript Wizard
Note: Controls with yellow background can be scripted.
Benchmark Factory scripting allows you to randomize the load testing process. Scripting can be
utilized by selecting the SQL Statement node and inserting the script in the properties view. A
field has scripting capabilities if its right-click menu contains the Script Wizard option.
The Script Wizard is a quick and easy way to use Benchmark Factory scripts. The Script Wizard
consists of a script category and a script function. Each script function has a short description
included and if applicable, its parameters.
There are two features within Benchmark Factory that assist you when using scripting
capabilities:
l
Script Wizard
Script Assist
2. Select BFScript Wizard from the drop-down. The Script Wizard displays.
215
216
Wizards
217
Wizards
218
Wizards
219
Wizards
Overview Tab
The Overview tabs provides transactions per second testing results for individual user loads and
iterations.
Transactions
The Transactions view provides the following statistics on the SQL statements being run:
l
Executions
Rows
Bytes
Errors
220
Wizards
Real-Time
Real-Time Statistics provides real-time graphs and raw data. This data allows you to spot systemunder test issues that may be affecting server performance. Right-clicking inside the graph
displays a drop-down that allows you to change graph settings and view.
Summary
The Summary graph provides real-time and raw data summary graphs. The Summary graph
displays real-time counter data. Right-clicking inside the graph displays a drop-down that allows
you to change graph settings and view.
Messages
The Output window displays messages about the job being run.
221
222
Wizards
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Actions
223
224
Wizards
Section
Description
Comments
Alerts
Job Counters
Transaction Wizard
Transaction Wizard
A transaction is a single unit of work in the testing process, used in load scenarios and user
scenarios. For example, a SQL statement. The Transaction Wizard walks you through the steps
required to create a transaction. From the Transaction Wizard you can:
225
Wizards
Import Transactions
226
Wizards
3. Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
16. Select the desired option.
17. Click Next. The wizard completion dialog displays.
18. If the Do you want the job to be automatically submitted to the jobs queue? checkbox
displays, then checking this will cause the Submit Job dialog to display when this
wizard completes.
19. Click Finish. The workload creates and displays in the Script View.
5. After enter the desired SQL statement or completing the BFScript Wizard, Click Next.
The Real World Latencies dialog displays.
6. Select the desired latency type.
7. Enter the desired mean and value time.
8. Click Next. The wizard completion dialog displays.
9. Click Finish. The SQL statement creates and displays in the Script View.
227
Wizards
When a Prepare SQL Statement transaction is contained within a User Scenario, it will
always prepare the SQL and, if Execute after prepare is checked (Figure 1) , it will
execute the SQL immediately after the preparation.
If the transaction is not within a User Scenario and Execute after prepare is not
checked, it will always prepare the SQL without executing.
If the transaction is not within a User Scenario and Execute after prepare is checked, the
statement will be prepared once for each user and immediately executed, and then
executed without a prepare each time after that point. This enables a Load Scenario to be
optimized for better real world simulation.
228
Wizards
7. After enter the desired SQL statement or completing the BFScript Wizard, Click Next.
The Real World Latencies dialog displays.
8. Select the desired latency type.
9. Select the desired latency type.
10. Enter the desired mean and value time.
11. Click Next. The Transaction Wizard completion dialog displays.
12. Click Finish. The transaction creates and displays in the Script View.
To Create a prepared SQL statement transaction with a bind variable
1. Click the Load Scenario, User Scenario, or Transaction node.
2. Select Wizards|Transaction. The Transaction Wizard displays.
3. Click Next. The Define Transaction dialog displays.
4. Click Create Prepare SQL Statement Transaction.
5. Click Next. The SQL Statement dialog displays.
6. Enter the desired SQL Statement
or
8. Select Add.
9. Right-click inside the type list control.
229
230
231
Wizards
232
Wizards
Import Transactions
Import transactions allows you to import a transaction(s) from the AS3AP, Scalable Hardware,
TPC-B, TPC-C, TPC-D or TPC-H benchmark standard benchmarks to build workloads.
To import a transaction
1. Select Wizards | Transaction Wizard. The Transaction Wizard displays.
2. Click Next. The Define Load Transaction dialog displays.
3. Click Import Transaction.
4. Click Next. The Import Transactions dialog displays.
5. From the Benchmark drop-down list select the desired benchmark.
6. Select the desired transactions by clicking on the gray square in the first column.
7. Click Next. The Real World Latencies dialog displays.
8. Select the desired latency type.
9. Enter the desired mean or value time.
10. Click Next. The Transaction Wizard completion dialog displays.
11. Click Finish. The transaction creates and displays in the Script View.
233
Wizards
234
Wizards
235
Wizards
236
Wizards
10
Testing Results (Run Reports)
Benchmark Factory TestingResults
Analysis is the ultimate goal of your load test. When you run a test, you get results both in realtime and in a saved format. Benchmark Factory provides you with testing results that are easy to
interpret and allows you to attribute individual results to individual tasks and users.
Whether you are viewing a real-time or historical graph, Benchmark Factory uses the same
graphing tool. In most cases you will find that the graphs are presented in the most meaningful
form. The Benchmark Factory graphing tool allows you to customize graphs to tailor your load
testing viewing requirements.
Benchmark Factory provides the following testing results types:
l
Definition
Bytes
238
Term
Definition
period in seconds.
Response Time
The time it takes from when the SQL is sent to the server
responds.
Retrieval Time
Rows
Transaction Time
(sometimes listed as just
Time)
239
Realtime Statistic
Summary
Transaction Graph
Graph Legend
Toggling to Graph Legend displays a legend on the side of the graph.
240
Toolbar
Toggling to Toolbar displays the graph toolbar.
Print
Choosing Print displays the Print Dialog.
Copy Data to Clipboard
Copies the data to the clipboard.
Copy Graph to Clipboard
Copies the graph to the clipboard.
Load Configuration
Benchmark factories graphs allows you to save graph configurations.
Save Configuration
Saves a graph configuration.
Set as Default
Sets a configured graph as default.
241
242
Clear Configuration
Clears a graph configuration.
Summary Tab
The Summary Tab properties displays information on the job when it is running. You can
monitor the status of the test, which userload its running, transactions per second (TPS), and a
variety of other datapoints that give you insight into the actual performance your database
provides under the userload being tested.
Iteration Overruns
Iteration overruns occur at the end of an iteration to allow time for all transactions submitted
within the test iteration cycle to complete, so that all transaction statistics can be collected. For
example, an agent may execute a transaction during the last five seconds of test iteration, if this
transaction takes 15 seconds to complete, an iteration overrun of 10 seconds will occur.
243
Statistics node displays real-time counter data. Right-clicking inside the graph displays a dropdown that allows you to change graph settings and view.
To view Real-Time Statistic
1. In the Jobs Queue, click the job running.
2. Select Click here to Add Datapoints. The Add Datapoints dialog displays.
244
3. Click OK. The graph displays with the selected datapoints. (To change graph views,
right-click. See "Change Graph Views" (page 364) for more information.)
2. Click the datapoints Vs. Userload drop-down and select the desired datapoint.
3. To change graph views, right-click. See Change Graph Views for more information.
245
Transaction Node
The Transaction node displays user selected transactions statistics for the job timing period
following completion of an iteration.
To view the Real-time statistics node
1. Click the Summary node in the Jobs Queue. The Summary graph displays in the
Properties view.
246
247
248
Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
249
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time
250
251
Graph Legend
Toggling to Graph Legend displays a legend on the side of the graph.
252
Toolbar
Toggling to Toolbar displays the graph toolbar.
Print
Choosing Print displays the Print Dialog.
253
254
255
See "Using Benchmark Factory Run Reports" (page 257) for more information.
Accessing a List of All Benchmark Factory Jobs from Inside of
Benchmark Factory
To access all Benchmark Factory Jobs from inside of Benchmark Factory
1. From the Benchmark Factory console click Tools | Run Reports. Run Reports displays.
Note: For procedures on viewing individual reports, please see Viewing Load Testing Results.
Importing and Exporting Benchmark Factory Test Results from Run
Reports
From Run Reports, you can import or export run results.
To import or export testing results:
1. Open Run Reports.
2. Click on the required Import or Export button.
256
257
258
When Benchmark Factory Run Reports opens, the Results Summary (page 260) displays.
Breadcrumbs at the top of view allow you to navigate to different views.
259
260
2. Click Details.
The Testbed view displays.
3. To navigate back to Results Summary, click the Results Summary breadcrumb in the
upper left corner.
Results Summary
Results Summary view provides graphs and user load statistics for the selected test. The
following graphs and tables are provided:
l
(page 263)
ResultsSummaryGraph
Note: Not all test provide the Results Summary graph.
This is a customizable graph that allows you to view selected datapoints.
261
262
3. Click Ok.
The selected datapoints display.
263
264
Userload
Understanding how user loads affect the performance of a database is essential to end user
satisfaction. The Userload graph plots user load against:
l
Response Time
TotalBytes
Total Errors
Total Rows
Response Time
Reviewing these datapoints allows you to fully understand how "real-world" userloads affect
database performance.
265
Realtime Summary
The Realtime Statistic graph allow you to view what "actually" happened during a load test.
You can plot userload against:
l
Transaction/Second
Total Rows
Errors
Total Bytes
Rows/Second
Minimum Time
Deadlocks
Bytes/Second
Average Time
Bytes/Second
266
Realtime Detail
The Realtime Detail graph allows you to view what actually happened during the running of a
load test. This allows you to view the actual timing events. From the Realtime Detail graph
you can view:
l
Average Time
Bytes/ Second
Deadlocks
Maximum Time
Minimum Time
Rows/Second
TotalBytes
Errors
TotalRows
Transactions/Second
User Load
267
From this table you can drill down to view userload stastics.
To drill down from the UserloadStatistics table:
1. Click on the required Userload.
2. From this view, you drill down further by clicking on required User Scenario name.
Workload
The Workload view provides testing details.
268
269
Clicking Details displays the transactions mix, timing, userload, and testing options used
for the test.
Testbed
The Testbed view shows data on agent configuration and processes during the running of a job.
Clicking Details displays a table with machine and operating system details.
270
271
4. Click on Details for Workload, Testbed, or Database Under Test to drill down on
testing results.
272
Note: To change Run Reports graph views, refer to Changing Graph Views.
View other graphs as required.
273
274
275
11
Benchmarks
Overview
A benchmark is a performance test of hardware or software. A benchmark simulates real-world
application work loads. These models of real applications can then be run against the system
being evaluated. Models work better than the actual application and offers reproducible results.
The real application, on the other hand, has too many variables that can change the results over
several test sessions.
Provided Benchmarks
Benchmark Factory provides the following standard benchmarks:
l
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
277
Benchmarks
TPC-E
TPC-H
AS3AP Benchmark
The AS3AP benchmark is an American National Standards Institute (ANSI) Structured Query
Language (SQL) relational database benchmark. The AS3AP benchmark provides the
following features:
l
Built-in scalability and portability that tests a broad range of database systems
Systems tested with the AS3AP benchmark must support common data types and provide a
complete relational interface with basic integrity, consistency, and recovery mechanisms. The
AS3AP tests systems ranging from a single-user microcomputer Database Management System
(DBMS) to a high-performance parallel or distributed database.
Best Practices
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Run the Benchmark Object Wizard to add the Delete Benchmark objects for AS3AP node
to a job as needed. See "Benchmark Objects Wizard" (page 171) for more information.
278
Benchmarks
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'AS3AP' job will clean the environment.
Scaling Factor
The AS3AP benchmark scales by factor of 10.
CPU
Disk
Network
Scaling Factor
The Scalable Hardware benchmark has a scaling factor of one.
Best Practices
Load-testing against production servers
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
279
Benchmarks
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Run the Benchmark Object Wizard to add the Delete Benchmark objects for the Scalable
Hardware node to a job as needed.
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'Scalable Hardware" job will clean the
environment.
TPC-B Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
The Transaction Processing Council is an organization that establishes transaction processing and
database benchmark standards. For a complete overview and detailed explanation of the TPC-B
Benchmark, please refer to:
TPC-B Benchmark
Best Practices
The following provides best practices for the TPC-B Benchmark.
280
Benchmarks
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-B node
to a job as needed.
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'TPC-B' job will clean the environment.
History Tables
The TPC-B benchmark is made up of only one transaction that updates three tables and inserts a
record into a history table. Inserting one record into one history table limits testing performance.
The Benchmark Factory properties page allows the user to set the number of history tables to
create during a test. The best ratio of history tables to virtual users is based on database
configuration and hardware. The number of history tables to use is determined by the tester.
Scaling Factor
The TPC-B benchmark scales by a factor of one.
TPC-C Benchmark
l
Overview
TPC-C Tables
281
Benchmarks
Best Practices
Overview
Note: For a detailed overview of the TPC-C Benchmark, please refer to the Transaction
Processing Council - TPC-C Benchmark.
The TPC-C benchmark is an online transaction processing benchmark that simulates
environments that have a number of terminal operators that send transactions to a database. This
benchmark is focused on the concept of an order-entry type environment with transaction that
include orders, payment recording, order status, and stock level monitoring. This benchmark
portrays the activities of a wholesale supplier. However, the TPC-C is not limited to one
particular business segment. It can represent numerous categories of a business that sell or
distribute products and services.
The TPC-C benchmark simulates a wholesale parts dealer operating out of warehouses. This
Benchmark scales as a company, in theory, expands their business or number of facilities. As the
TPC-C benchmark scales, so do the number components for the benchmark, for example, the sales
districts and customers.
TPC-C Tables
The scale factor determines the amount of information initially loaded into the benchmark tables.
For the TPC-C benchmark, each scale factor represents one warehouse as per TPC-C specification.
The TPC-C benchmark involves a mix of five concurrent transactions of different types and
complexity. The database is comprised of nine tables with a wide range of records.
A maximum of 10 users should be run against each warehouse. For example, user loads of 1, 5,
and 10, set the scale to 1. If using other user load values, change the scale factor accordingly.
The TPC-C database consists of the following tables:
Warehouse
District
Customer
History
New_Order
Order
282
Benchmarks
Order_Line
Item
Stock
Best Practices
The following provides best practices for the TPC-C Benchmark.
Load-testing against production servers
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-C node
to a job as needed.
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'TPC-C' job will clean the environment.
283
Benchmarks
TPC-D Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
The Transaction Processing Council is an organization that establishes transaction processing and
database benchmark standards. For a complete overview and detailed explanation of the TPC-D
Benchmark, please refer to:
TPC-D Benchmark
Best Practices
The following provides best practices for the TPC-D Benchmark.
Load-testing against production servers
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
284
Benchmarks
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-D node
to a job as needed.
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'TPC-D' job will clean the environment.
Scaling Factor
The TPC-D benchmark scales by the following factors:
l
0.10
1.00
10.00
30.00
100.00
300.00
TPC-E Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
The Transaction Processing Council is an organization that establishes transaction processing and
database benchmark standards. For a complete overview and detailed explanation of the TPC-E
Benchmark, refer to:
TPC-E Benchmark
285
Benchmarks
Best Practices
The following provides best practices for the TPC-E Benchmark.
Load-testing against production servers
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Scaling Factor
The TPC-E benchmark scales by factor of 500.
TPC-H Benchmark
l
Overview
Best Practices
Scaling Factor
Overview
The Transaction Processing Council is an organization that establishes transaction processing and
database benchmark standards. For a complete overview and detailed explanation of the TPC-H
286
Benchmarks
Best Practices
The following provides best practices for the TPC-H Benchmark.
Load-testing against production servers
Do not load-test against a production server if possible. Load-testing and benchmarking on a
production server significantly degrades performance. In some cases, load-testing can cause a
server to fail. However, if testing against a production server, take the following precautions:
l
Perform the testing when no other users are on the system and no automated processes are
running. Users and automated processes can adversely affect testing results
Determine how long it will take to restore a production server if it went down during
load-testing
Perform manual testing. Manual testing ensures that no unexpected outside activity takes
place during the testing process
Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-H node
to a job as needed.
Run the Load Scenario Wizard to create a new script containing the Create Objects for
the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the
Benchmark. Running the delete objects for 'TPC-H' job will clean the environment.
Scaling Factor
The TPC-H benchmark scales by the following factors:
l
1.00
10.00
30.00
100.00
300.00
1000.00
3000.00
10000.00
30000.00
100000.00
287
12
BFScripts
About Scripts
Note: All edit field with a yellow background are scriptable.
Benchmark Factory provides a scripting capability as a means of inserting built-in functions for
use in scripts. A function is a formula that takes one or more values (arguments), performs an
operation, and returns a value that simulates real-world user activity. Functions are used alone or
as building blocks in creating complex user activity. Randomized data is important when
attempting to simulate real-world user activity because data that is random, prevents a server
from using data stored in its cache. Click here for procedures on how to use scripts.
The following provides a list of available scripts:
Category
Script Function
$BFCreditCardExp
$BFCurrentDate
$BFCurrentDateTime
$BFDate
File Access
$BFFileArray
Global Variables
$BFAddress
$BFAddress2
$BFCity
$BFCompany
$BFCountry
$BFEMail
$BFFirstName
$BFFullName
$BFLastName
$BFMiddleInitial
$BFPhone
$BFState
$BFZipCode
Numerical Manipulation
$BFFormat
$BFProd
$BFSum
Random Numbers
$BFRand
$BFRandRange
$BFURand (page 303)
$BFURandRange
Random Strings
$BFCreditCard
$BFRandList
$BFRandMultiList
$BFRandStr
$BFURandList (page 307)
$BFList
String Manipulation
$BFAsc
$BFChr
$BFConcat
$BFLeft
$BFLen
$BFLower
$BFMid (page 311)
$BFRight
$BFTrim
$BFTrimLeft
$BFTrimRight
$BFUpper
Test Info
$BFGetVar
$BFMaxNode
$BFNode
$BFNumberOfIterations
$BFProfile
$BFRunID
$BFSetVar
$BFSetVarRtn (page 316)
$BFUserCounter
$BFUserID
$BFUserLoad
$BFNextUserload
$BFPrevUserload
289
290
BFScripts
This example can be useful to log in each virtual user with a unique username and password
designated for this use. In this case users should be created in the test system with a username
and password containing the virtual user number that will be used.
Parameters:
N/A
Syntax:
$BFGetGlobalVar('myvar')
$BFCurrentDate
Description:
Allows you to change the date format used to populate date fields.
Parameters:
Variables
%a
%A
Definition
Abbreviated weekday name
%b
%B
%c
291
BFScripts
%d
%H
%I
%j
%m
%M
%p
%S
%U
%w
%W
%x
%X
%y
%Y
%z, %Z
%%
Percent sign
Syntax:
Example:
BFCurrentDateTime
Description:
Allows you to change the date/time format used to populate date fields.
Parameters:
Variables
%a
%A
Definition
Abbreviated weekday name
%b
%B
%c
%d
%H
292
BFScripts
%I
%j
%m
%M
%p
%S
%U
%w
%W
%x
%X
%y
%Y
%z, %Z
%%
Percent sign
Syntax:
Example:
$BFDate
Description:
Parameters:
mm/dd/yyyy
Syntax:
$BFDate(nStart,nEnd)
Example:
293
BFScripts
File Access
$BFFileArray
Description:
Selects an item from a list. Returns a single item from a comma-delimited file.
The item returned depends on the mode selected. The syntax of the statement is
also slightly different for each mode. Each virtual user gets a different seed
value to generate unique sequences. Each agent machine must have a file with
the name and path that is specified in the script function. If $BFFileArray is to
return strings, the items in the file must be in double-quotes
Parameters:
Syntax:
Example:
Global Variable
$BFGetGlobalVar
Description:
Parameters:
N/A
Syntax:
$BFGetGlobalVar('myvar')
$BFSetGlobalVar
Description:
Parameters:
N/A
Syntax:
294
$BFSetGlobalVarRtn
Description:
Parameters:
N/A
Syntax:
Parameters:
N/A
Syntax:
BFAddress()
Example:
$BFAddress2
Description:
Parameters:
N/A
295
296
BFScripts
Syntax:
$BFAddress2()
Example:
$BFAddress2() ; returns
"Apt 5442"
$BFCity
Description:
Parameters:
N/A
Syntax:
$BFCity()
Example:
$BFCompany
Description:
Parameters:
N/A
Syntax:
$BFCompany()
Example:
$BFCountry
Description:
Parameters:
N/A
Syntax:
$BFCountry()
Example:
$BFEmail
Description:
Parameters:
N/A
Syntax:
$BFEmail()
Example:
$BFFirstName
Description:
Parameters:
N/A
Syntax:
$BFFirstName()
Example:
297
$BFFullName
Description:
Parameters:
N/A
Syntax:
$BFFullName()
Example:
$BFLastName
Description:
Parameters:
N/A
Syntax:
$BFLastName()
Example:
$BFMiddleInitial
Description:
Parameters:
N/A
Syntax:
$BFMiddleInitial()
298
299
BFScripts
Example:
$BFPhone
Description:
Parameters:
N/A
Syntax:
$BFPhone()
Example:
$BFPhone() ; returns
"(348)889-6599"
$BFState
Description:
Parameters:
N/A
Syntax:
$BFState()
Example:
300
BFScripts
$BFZipCode
Description:
Parameters:
N/A
Syntax:
$BFZipCode()
Example:
Numerical Manipulation
$BFFormat
Description:
Parameters:
Syntax:
Example:
$BFProd
Description:
Parameters:
Syntax:
Example:
$BFSum
Description:
Parameters:
Syntax:
Example:
301
302
BFScripts
Random Numbers
$BFRand
Description:
Returns a random
integer between 0
and nMax. Each
virtual user gets a
different seed
value to generate
the same unique
sequences for each
run.
Parameters:
nMax-The
maximum integer
to be returned by
the function.
Syntax:
$BFRand(nMax)
Example:
$BFRand(100) ;
returns "45"
$BFRandRange
Description:
Parameters:
Syntax:
$BFRandRange(nMin,nMax)
Example:
303
BFScripts
$BFURandRange
Description:
Parameters:
Syntax:
$BFURandRange(nMin,nMax)
Example:
$BFRandRange(1,100) ;
returns 100 the first time this function executes and 95,
and 85 for subsequent executions of this function.
$BFURand
Description:
Parameters:
Syntax:
$BFURand(nMax)
Example:
$BFURand(100) ;
returns 78 the first time this function executes, and 50, and 19 for
subsequent executions of this function.
304
BFScripts
Random String
$BFCreditCard
Description:
Parameters:
N/A
Syntax:
$BFCreditCard()
Example:
$BFRandList
Description:
Parameters:
305
BFScripts
$BFRandList(string1[:nWeight1], string2[:nWeight2], ,
stringN[:nWeightN])
Example:
$BFRandMultiList
Description:
Parameters:
306
BFScripts
Syntax:
Example:
$BFRandStr
Description:
Parameters:
Syntax:
$BFRandStr(n,[constant],[1..255],[1..255])
Example:
307
BFScripts
$BFURandList
Description:
Parameters:
Syntax:
$BFURandList(string1,string2, stringN)
Example:
$BFList
Description:
Parameters:
Retrieval Mode:
RANDOM: Select a random item from the list.
SEQUENTIAL: Select each item sequentially.
UNIQUE: Select a non-repeating item from the list.
string1-The first string to return from a list.
string2-The second string to return from a list.
stringN-The last string to return from a list.
Syntax:
Example:
$BFList (Random 1, 2, 3, 4)
Returns 2
3
3
1
2
4
etc.
$BFList (Unique, "1", "2", "3","4")
Returns 2
4
3
1
2
etc.
String Manipulation
$BFAsc
Description:
Parameters:
Syntax:
$BFAsc(string)
Example:
$BFChr
Description:
308
Parameters:
Syntax:
$BFChr(n)
Example:
$BFConcat
Description:
Parameters:
Syntax:
Example:
$BFLeft
Description:
Parameters:
309
310
BFScripts
Syntax:
$BFLeft(string,length)
Example:
"Bench"
$BFLen
Description:
Parameters:
Syntax:
$BFLen(string)
Example:
$BFLower
Description:
Parameters:
Syntax:
$BFLowerstring)
Example:
$BFMid
Description:
Parameters:
Syntax:
$BFMid(string, nStart,nLength)
Example:
$BFRight
Description:
Parameters:
Syntax:
$BFRight(string,nLength)
Example:
$BFTrim
Description:
311
312
BFScripts
Parameters:
Syntax:
$BFTrim(string)
Example:
$BFTrim(" happy days are here to stay. ") ; returns "happy days
are here to stay"
$BFTrimLeft
Description:
Parameters:
Syntax:
$BFTrimLeft(string)
Example:
$BFTrimRight
Description:
Parameters:
Syntax:
$BFTrimRight(string)
Example:
313
BFScripts
$BFUpper
Description:
Parameters:
Syntax:
$BFUpper(string)
Example:
Test Info
$BFGetVar
Description:
Parameters:
Syntax:
$BFGetVar("VarName")
Example:
314
BFScripts
$BFMaxNode
Description:
Returns the total number of nodes for all users. This function
is intended only for Oracle clustering.
Parameters:
N/A
Syntax:
$BFMaxNode()
Example:
$BFNode
Description:
Parameters:
N/A
Syntax:
$BFNode()
Example:
$BFNumberOfIterations
Description:
Parameters:
N/A
Syntax:
$BFNumberOfIterations()
Example:
315
BFScripts
$BFProfile
Description:
Parameters:
SERVERNAME
DATABASE
USERNAME
ORACLE:
l
NETSERVICENAME
USERNAME
DATASOURCENAME
USERNAME
DB2:
l
DATABASEALIAS
USERNAME
Syntax:
$BFProfile(constant)
Example:
$BFRunID
Description:
316
BFScripts
Parameters:
N/A
Syntax:
$BFRunID()
Example:
$BFSetVar
Description:
Parameters:
Syntax:
Syntax:$BFSetVar("VarName", "Value")
Example:
$BFSetVarRtn
Description:
Parameters:
Syntax:
$BFSetVarRtn("Varkey","Value")
Example:
$BFUserCounter
Description:
Returns the
user
counter.
Parameters:
N/A
Syntax:
$BFUserCounter()
Example:
$BFUserCounter() ;
returns "1"
317
$BFUserID
Description:
Parameters:
N/A
Syntax:
$BFUserID()
Example:
$BFUserLoad
Description:
Parameters:
N/A
Syntax:
$BFUserload()
Example:
$BFNextUserload
Description:
Returns the next user load for the load scenario running.
Parameters:
N/A
Syntax:
$BFNextUserload()
318
319
BFScripts
Example:
$BFNextUserload()
When running with userloads 1, 4, 6, 10 this will return "6"
when running at userload 4.
$BFPrevUserload
Description:
Parameters:
N/A
Syntax:
$BFPrevUserload()
Example:
$BFPrevUserload()
When running with userloads 1, 4, 6, 10 this will
return "4" when running at userload 6.
13
Quick Starts
About the Quickstarts
These quick start procedures introduce you to the Benchmark Factory environment. The intent of
these procedures is not to show every aspect of the load testing process or every feature of
Benchmark Factory. They provide the necessary background to quickly explore and begin using
Benchmark Factory. The following quick starts are provided:
l
Standard Benchmark
Goal Testing
321
Quick Starts
322
Quick Starts
323
Quick Starts
Note: Clicking the Advanced tab allows you to map benchmark objects. For these
procedures, please see Advanced Creation Objects (Object Mapping).
19. Click OK.
20. Click the TPC-H Benchmark Edit button.
21. Select the desired benchmark scale. The TPC-H benchmark creates large testing databases. A scale of 1
is suggested.
22. Click OK.
23. Click Next. The Node Selection dialog displays.
24. Enter the desired number of nodes in Nodes. In this example, 10 nodes are selected.
25. Under Method, click the Benchmark Factory radio button.
26. Click Next. The Measurement Intervals dialog displays.
324
Quick Starts
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
Imports from various tools such as Quest Performance Analysis SQL from the repository
for replay.
325
Quick Starts
326
Quick Starts
12. If appropriate, select the "From" and "To" dates and times.
13. Uncheck the Include system activity (SYS USER) box to include system activity.
Note: It is recommended to not check the Include system activity (SYS USER) check box.
This imports Oracle SQL statements routinely run by Oracle for maintenance reasons and
is not required for the load testing process.
14. Click Next. The Enter Script Name dialog displays and provides the following options:
15. Create a new Benchmark Factory Script file-Selecting this option allows you to
create a and save a completely new script.
16. Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
15. Check the Add to existing Benchmark Factory Script file radio button.
16. Click Next. The wizard completion dialog displays.
17. Click Finish. The trace files create and display in the Script View.
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
327
328
Quick Starts
Section
Description
Profile
Actions
Comments
Alerts
Job Counters
2. You can select a single test or multiple test to view testing results. In this example,
multiple tests are selected to view the RAC testing results.
329
330
Quick Starts
4. To view RAC testing results for the nodes selected, click Cluster Performance Graph.
5. To view other testing results, click the desired result from the User Load drop-down.
331
Quick Starts
Are you better able to meet important business requirements such as higher availability,
better scalability, or better performance?
Can you extract greater value out of the system by running it on the newest technology?
Are you able to maintain the old application more efficiently than before?
This quick start presents a load testing methodology to compare your current database servers
with newer versions to determine if upgrading helps you obtain better performance.
Creating a SQL Server 2000 Trace File using SQL Server 2000 SQL Profiler
Creating the SQL Server 2000 Trace Files Using SQL Profiler
Trace files obtain data in which events are logged during the running of a database application.
You can then use the trace table to play back the logged events. You may have a trace table
readily available, or can use Microsoft SQL Profiler to create a trace table.
For procedures on creating SQL Server 2000 trace files, please refer to:
l
Creating a SQL 2000 Trace Table Using the SQL Server Profiler
332
Quick Starts
333
Quick Starts
5. Right-click.
6. Select Remove.
7. Click Yes when asked, "Are you sure you want to delete the selected item(s)?
8. Select the newly created Load Scenario.
9. In the properties view, select the Timing Tab. For this example, a 2:00 sampling length
is selected.
334
Quick Starts
10. In the properties view, select the User Load tab. For this example, user loads of 1 to 3000
are selected, with intervals of 250 Users.
335
Quick Starts
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Actions
Comments
336
Quick Starts
Section
Description
Alerts
Job Counters
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
337
Quick Starts
Section
Description
Profile
Actions
Comments
Alerts
Job Counters
338
Quick Starts
represents the SQL Server 2005. Note that in this example, SQL Server 2005 dramatically
outperformed SQL Server 2000.
339
Quick Starts
Creating a profile
340
Quick Starts
Accessing the Benchmark Factory script and Oracle Trace Files for this
Quick Start
Industry standard benchmarks have been developed to measure system performance. Each
benchmark has specific system configuration requirements that include tables, indexes, and data.
For this Quick Start, we will use the AS3AP Benchmark to create the environment to run the test
on. The Oracle Trace File Setup.bfp, OraDemo_2.trc, and OraDemo_2.trc files are provided to
create the AS3AP tables and run the load test. The following files are located in C:\Program
Files\Quest Software\Benchmark Factory for Databases\Quick Starts:
l
OraDemo_1.trc
OraDemo_2.trc
341
Quick Starts
The Submit Job dialog allows you to tailor your job needs and provides the following options:
Section
Description
Profile
342
Quick Starts
Alerts
Job Counters
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
343
344
Quick Starts
Section
Description
Displays the Profile Creation Wizard.
Actions
Comments
Alerts
Job Counters
345
Quick Starts
3. Double-click. Run Reports opens to the Run ID node. This view contains a summary of
job statistics.
4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that
allow you to view in detail your testing results.
5. Expand the Results node. Results provides information on individual user load and agent
load testing activity.
6. Close Run Reports when finished reviewing the job.
Saving the Script
1. Click the script tab in the Script View.
2. Select File | Save. The Save As dialog displays.
3. Enter the desired script name in File Name.
4. Click Save.
346
Quick Starts
Create a profile
Create a profile
347
Quick Starts
3. Select Wizards|Load Scenario. The Load Scenario Wizard displays. The Load Scenario
Wizard is the starting point for all Benchmark Factory scripts. The New Script Wizard
walks you through the steps to build a complete load testing script.
4. Click Next. The Select Load Scenario dialog displays.
5. Click Industry standard benchmarks (TPC-C, TPC-H...). Benchmark Factory
provides industry Standard Benchmarks that allows you to populate test databases
according to a selected industry standard benchmark and measures the performance of
a system-under-test.
Benchmark Factory provides the following standard benchmarks:
l
AS3AP
Scalable Hardware
TPC-B
TPC-C
TPC-D
TPC-H
6. Click Next. The Select Benchmark dialog displays. This dialog displays the standard
benchmarks that come with Benchmark Factory. For a complete overview of all
Benchmarks, please refer to Benchmark Overview.
7. Click the AS3AP benchmark.
348
Quick Starts
Creating a Profile
1. If no profile has been created, click New. The Profile Creation Wizard displays. If a
profile has been created, select the desired profile from the Profile drop-down.
2. Click Next. The Benchmark Scale dialog displays.
Setting the Benchmark Scale and User Load
1. Benchmark Factory allows you to set the desired benchmark scale to perform realistic
load tests. Select the desired benchmark scale in Benchmark Scale.
2. Click Next. The Measurement Intervals dialog displays.
3. Add the desired User Load.
4. Click Next. The Enter Script Name dialog displays and provides the following options:
l
Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
Add to existing Benchmark Factory Script file-Selecting this option allows you to
add the load scenario you are building to the current script file you are working with.
349
Quick Starts
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
Comments
Alerts
350
Quick Starts
Job Counters
2. Expand the Run Status node. Run Status provides a real-time status on the test currently
running in the Jobs Queue.
3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time
graphs and raw data. This data allows you to spot system-under-test issues that may be
affecting server performance. The Real Time Statics node displays real-time counter data.
Right-clicking inside the graph displays a drop-down that allows you to change graph
settings and view.
4. Click Click here to Add Datapoints.
5. Select the desired datapoints.
6. Click OK.
7. Right-click on the graph.
8. Select Graph Legend.
351
Quick Starts
9. Click the Summary node. The Summary Graph displays. The Summary page displays user
selected statistics for the job timing period following completion of an iteration.
10. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the
desired datapoint.
11. Click the Transaction Graph node. The transaction node displays properties based on
transaction type.
12. From the Vs. Userload drop-down in the upper left-hand corner of the graph, select the
desired datapoint.
352
Quick Starts
353
Quick Starts
3. Double-click. Run Reports opens to the Run ID node. This view contains an summary of
load scenario statistics.
4. Expand Summary Graphs node. Summary graphs provide a wide array of graphs that
allow you to view in detail your testing results.
5. Expand the Results node. Results provides information on individual user load and agent
load testing activity.
6. Close Run Reports when finished reviewing the load scenario.
Saving the Script
After running a job, you can save a job to run again, or edit to run using different testing
parameters.
1. Click the Script tab in the Script View.
2. Select File | Save. The Save As dialog displays.
3. Enter the desired script name in File Name.
4. Click Save.
Overview
This quick start presents a load testing methodology to run a goal test that finds the maximum
Transactions Per Second (TPS) of a database, by adding a SQL Statement Transactions to the
goal test, then viewing real-time and stored testing results to determine if your database lives up
to the required performance parameters.
354
Quick Starts
A Goal Test uses a transaction mix A transaction is a single unit of work in the testing process,
used in load scenariosand user scenarios. For example, a SQL statement. based on userload
intervals. The userloads are determined by setting a beginning, ending value, and interval value.
This value specifies an interval to increase the number of users tested for each iteration of the
Goal Test. When run, the specified goal criterion is evaluated at the end of each iteration, and
the test ends once the goal or maxim user load has been reached.
Find maximum TPS: Selecting this option will show how the maximum TPS can
be reached during a load test.
Find maximum userload where response time is less than [n]ms: Selecting this
option shows how a maximum userload can be reached when response time is less
than a keyed in value.
355
Quick Starts
10. Select the desired option. For this quick start, the Find maximum TPS is selected.
11. Set the desired Start, Ending, and Interval in Userloads. For this test, the defaults values of
1, 5, and 1 are used. When run, User Loads of 1, 2, 3, 4, 5 are run until the TPS of one
run is less than the TPS of the previous run.
12. Click Next. The Define Transaction Mix Page displays.
13. Click Next. The Enter Script Name dialog displays and provides the following options:
2. Create a new Benchmark Factory Script file-Selecting this option allows you to
create and save a completely new script.
3. Add to existing Benchmark Factory Script file-Selecting this option allows you
to add the load scenario you are building to the current script file you are
working with.
14. Click Create a new Benchmark Factory Script file.
15. Click Next. The wizard completion dialog displays.
16. Click Finish. The Scripts creates and displays in the Script View.
Setting Script Timing
1. Click the Max TPS (1-5 by 1) node.
356
Quick Starts
357
Quick Starts
358
Quick Starts
359
Quick Starts
4. Click Next.
5. From the drop-down, select the AS3AP standard benchmark.
6. Click Next. The Benchmark Scale displays.
7. Set the desired scale in Benchmark Scale.
8. Click Next. The Create/Delete benchmark objects dialog displays.
9. Click Next. The wizard completion dialog displays.
10. Click Finish.
11. You must create the benchmark objects before running the script. Right-click Create
Objects for AS3AP.
12. Select Move Up.
13. Move the Benchmark Object to the top node in the script view.
2. Select Wizards | Submit Job Wizard. The Submit Job dialog displays.
The Submit Job dialog allows you to tailor your job needs and provides the
following options:
Section
Description
Profile
360
361
Quick Starts
Section
Description
Actions
Comments
Alerts
Job Counters
2. Expand the Run Status node. Run Status provides a real-time status on the test currently
running in the Job's Queue.
3. Click the Realtime Statistics node. The Real Time Statistics provides you with real-time
graphs and raw data. This data allows you to spot system-under test issues that may be
affecting server performance. The Real Time Statics node displays real-time counter data.
362
Quick Starts
4. Click the Summary node. The Summary Graph displays. The Summary page displays user
selected statistics for the job timing period following completion of an iteration.
5. Click the Transaction Graph node. The Summary Graph displays. The transaction node
displays properties based on transaction type.
Viewing Run Reports Results
1. When the job completes, click the Run Reports node.
14
Troubleshooting
Agent Connection
If the agent is having a problem connecting to the Benchmark Factory console, please check
the following
l
Verify that the agent is pointed to the console machine. Go to Options > Settings and
check the Machine Name/IP Setting.
Verify TCP connectivity between the Agent and Console. If there is a firewall in place,
allow for communication on Port #4568.
15
Appendix
Change Graph Views
Right-clicking on a Benchmark Factory graph displays a drop-down that allows you to
customize graph settings. Whether you are viewing a real-time graph or one in Benchmark
Factory Run Reports, Benchmark Factory uses the same graphing tool. In most cases you will
find that the graphs are presented in the most meaningful form (Line Graphs or Bar Graphs based
on the data being presented.)
Displays Benchmark Factory Run Reports
Shows in a data form or graph form
Displays the Graph Legend
Displays the graph toolbar
Prints the graph
Copies Data to Clipboard
Copies Graph to Clipboard
Loads a saved graph configuration
Saves a graph configuration
Sets the current configuration as default
Clears the current chart configuration
Graph Legend
Toggling to Graph Legend displays a legend on the side of the graph.
365
Toolbar
Toggling to Toolbar displays the graph toolbar.
Print
Choosing Print displays the Print Dialog.
Load Configuration
Benchmark factories graphs allows you to save graph configurations.
Save Configuration
Saves a graph configuration.
Set as Default
Sets a configured graph as default.
366
367
Appendix
Clear Configuration
Clears a graph configuration.
368
Appendix
369
Appendix
10. Select the appropriate database from the Database drop-down list.
11. Select the appropriate table from the Table drop-down list.
12. Click OK. The Trace Properties window displays.
13. Click the Events tab.
14. Select TSQL from the Available events classes list.
15. Click Add.
16. Click the Data Columns tab.
17. Add the following columns (minimal):
15. EventClass
16. Login Name
17. ClientProcessID
18. NTUserName
19. Duration
20. StartTime
21. TextData
18. Click Run.
19. Run SQL statements.
20. When finished running SQL statements, click the Stop selected trace icon. The
table creates.
370
Appendix
9. Select the appropriate authentication from the drop-down. Type the login name and
password if required.
10. Click Connect. The Destination Table window displays.
11. Select the appropriate database from the Database drop-down list.
12. Select the appropriate owner from Own drop-down.
13. Select the appropriate table from the Table drop-down list.
14. Click OK. The Trace Properties window displays.
15. Click the Events Selection tab.
16. Verify that all check boxes are checked in the TSQL-SQL:BatchStarting Events, and
TSQL-SQL:BatchComplete Events.
17. Click Run.
18. Run SQL statements.
19. When finished running SQL statements, click the Stop selected trace icon. The table
creates.
371
Appendix
Example:
BMFDataMigrationWizard -s [MyDatabase,root,yourpassword] -d
[LocalServer,sa,sa]
372
Appendix
Examples
Example with parameters:
BEGIN
YOUR_PROC(:VAR1,:VAR2);
END;
Bind Parameter Settings
Parameter
:VAR1
:VAR2
Type
INTEGER
INTEGER
Value
5
8
Type
INTEGER
INTEGER
Value
55
77
373
Appendix
and MySQL
call procedure-name (?,?,?,?);
Bind Parameter Settings
Parameter
1
2
3
4
Type
STRING
INTEGER
INTEGER
INTEGER
Value
A
5
7
0
374
Appendix
Note: If using a release before Oracle 8, release 8.1.6, these parameters can be changed with
ALTER SYSTEM commands.
3. Activate the trace.
dbms_support.start_trace_in_session(
sid => 55,
serial => 1222,
waits => true,
binds => true)
375
Appendix
Support Bundle
The Benchmark Factory Support Bundle allows you to zip up a .bfp file and send it to support
for review. The bundle contains the following information:
l
Settings
Files located in the data directory. This will be not only the xml files for imported users
scenarios, but also dump files.
A file that contains hardware information about the system running on it as well as the
output of the agents configurations from the repository
376
Appendix
System Requirements/Upgrade
Requirements/Supported Databases
System Requirements
Before installing Benchmark for Databases , ensure your system meets the following minimum
hardware and software requirements.
Repository
Benchmark Factory cannot run without a repository, and by default a MySQL repository is
installed during the install process. After installation, you can remove the MySQL service and
delete the MySQL repository, then select a repository of your choice. Benchmark Factory
supports the following databases:
l
Oracle
SQL Server
SQLite
MySQL
Memory
512 MB of RAM recommended
Note: The memory required may vary based on the following:
l
Platform
1.0 GHz x86 or x64 compatible CPU
377
Appendix
Windows 2000
Window 2003
Windows 2008
XP Professional
Vista
Windows 7
Database Server
Oracle: 8, 8i, 9i, 9.2, 10g, and 11g (Requires Oracle 8.1.7 client libraries or higher.)
Note: Oracle Capture/Replay of workloads must use Oracle 9i or higher.
SQL Server: 7, 2000, 2005, and 2008
Sybase: 12.5 and 15
DB2 LUW: 8.1.5 through 9
MySQL: 4.5 and 5.x
Note: All client libraries require the 32-bit versions.
Upgrade Requirements
l
Client libraries for database types used during the workload testing process must be
installed on all testing machines (Benchmark Factory and Agents).
The Benchmark Factory 3.3 and below files can be converted to Benchmark Factory for
Databases 4.x files. A script conversion program is available, and can be accessed by
logging on to Quest Software, then select Downloads and Upgrades | Benchmark
Factory for Databases| Special Patches| Benchmark Factory Project Converter.zip.
There is no upgrade path for the Benchmark Factory Repository version 3.3 or below.
If you create a new Benchmark Factory 5.5 repository, earlier versions of Benchmark
Factory will not work against this repository.
Shortcut Keys
The following provides a list of shortcut keys used in Benchmark Factory.
Key
Action
ALT+1
ALT+2
ALT+M
ALT+R
Runs a job
ALT+S
Stops a job
CTRL+B
CTRL+C
CRTL+D
CTRL+E
Readies a job
CTRL+F
CTRL+J
CTRL+L
CTRL+N
CTRL+O
Displays the Select Script File to Open dialog (Script view only)
CTRL+P
CTRL+R
CTRL+S
CTRL+T
CTRL+U
CTRL+V
CTRL+W
CTRL+X
378
Key
Action
CTRL+ ALT+E
CTRL+H
Holds a job
CTRL+ALT+H
Delete
END
HOME
LEFT ARROW
RIGHT
ARROW
F1
F8
F12
PLUS SIGN on
numeric keypad
(+)
LOCK+MINUS
SIGN on
numeric keypad
(-)
379
16
Copyright, Licensing, and Licensing
Agreement
Copyright
2012 Quest Software, Inc.
ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in
this guide is furnished under a software license or nondisclosure agreement. This software may be
used or copied only in accordance with the terms of the applicable agreement. No part of this
guide may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording for any purpose other than the purchasers personal use
without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license,
express or implied, by estoppel or otherwise, to any intellectual property right is granted by this
document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN
QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,
INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES
(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS
INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR
INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect
to the accuracy or completeness of the contents of this document and reserves the right to make
changes to specifications and product descriptions at any time without notice. Quest does not
make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
381
Patents
Toad for Cloud Databases contains patent pending technology.
Trademarks
Quest, Quest Software, the Quest Software logo, Benchmark Factory, Spotlight, LiteSpeed,
Simplicity at Work, Toad, TOAD, T.O.A.D., and TOADWORLD and vToadare trademarks and
registered trademarks of Quest Software, Inc. For a complete list of Quest Softwares trademarks,
see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered
trademarks used in this guide are property of their respective owners.
382
383
Legal Notice
Privacy Policy for Optional User Registration
How to opt out
If you registered at the time of installation and you no longer wish to receive monthly
newsletters, you may opt out of receiving them by following the instructions included in each
newsletter or communication or by updating your preferences here: www.quest.com/unsubscribe.
What data is being collected?
384
When you register you must supply your email address and job role. You may also provide your
name and phone number but this is not required. All this information will be stored locally and
also be sent to Quest Software in Aliso Viejo, California. Whether or not you choose to register,
we will collect your license number for tracking purposes.
Where the data will be stored?
All the submitted data will be stored locally on the computer where you have installed the
software and in servers located at the Aliso Viejo, California facility in the United States.
How the data will be used?
The information that we collect will help Quest Software to deliver a more consistent and
personalized customer support and marketing experience. For example, we may use your
information to:
l
Contact you regarding Quest products or services that we think will be useful to you
385
386
terms of this Agreement and the applicable Order. Orders placed with Quest by Customer
purchase order only and all Orders placed through a Partner shall be governed solely by the
terms of this Agreement. Any conflicting or additional terms in or accompanying an Order will
not be binding on Quest unless Quest accepts such terms in writing. Each Order shall be the
Customers irrevocable commitment to purchase and pay for the Products and/or Maintenance
Services stated in the Order.
(j) Software means the object code version of the software that is delivered pursuant to an
Order as well as any corrections, enhancements, and upgrades to such software that Quest may
provide to Customer pursuant to this Agreement, and all copies of the foregoing.
2. Software License.
(a) Internal Use License. Subject to the terms of this Agreement, Quest grants to Customer, and
Customer accepts from Quest, a perpetual (unless otherwise set forth in an Order), non-exclusive,
non-transferable (except as otherwise set forth herein) and non-sublicensable license to (i) install,
execute, access, run, or otherwise use the quantities of each item of Software identified in the
applicable Order within the parameters of the Product Terms associated with the applicable
Product and License Type, (ii) make a reasonable number of additional copies of the Software to
be used solely for non-productive archival or passive disaster recovery purposes, so long as
neither the original and a copy nor two copies of the same Software are used at the same time,
and (iii) make and use copies of the Documentation as reasonably necessary to support
Customers authorized users in their use of the Software (collectively, License). Except for MSP
Licenses (as defined below), each License shall be used by Customer solely to manage its own
internal business operations as well as the business operations of its Affiliates.
(b) MSP License. If an MSP License is specifically identified in an Order, Customer shall be
granted a License to use the Software identified in the Order and the associated Documentation
as a managed service provider (MSP) to provide software and systems management services,
including, without limitation, application, operating system, and database implementation,
performance tuning, and maintenance services (Management Services), for the benefit of a
single named client (Client), pursuant to the terms of this Agreement and the MSP Use Terms
in the Product Guide.
(c) Evaluation License. If an Order indicates that Software is to be used for evaluation purposes
or if the Software is otherwise obtained from Quest for evaluation purposes, Customer shall be
granted a non-production License to use such Software and the associated Documentation solely
for Customers own internal evaluation purposes for an evaluation period of up to thirty (30)
days from the date of delivery of the Software, plus any extensions granted by Quest in writing
(the "Evaluation Period"). There is no fee for Customers use of the Software for non-production
evaluation purposes during the Evaluation Period, however, Customer is responsible for any
applicable shipping charges or taxes which may be incurred, and any fees which may be
associated with usage beyond the scope permitted herein. Customers opportunity for a free
evaluation of the Software is limited to one Evaluation Period per release of the Software.
Notwithstanding anything otherwise set forth in this Agreement, Customer understands and
agrees that evaluation Software is provided AS IS and that Quest does not provide a Warranty
or Maintenance Services for evaluation Licenses.
387
(d) Third Party Use. If Customer contracts with a third party who performs Software
implementation, configuration, consulting or outsourcing services (Service Provider), the
Service Provider may use the Software and Documentation Licensed by Customer hereunder
solely for purposes of providing such services to Customer, provided that (i) Customer ensures
that the Service Provider uses the Software and Documentation in accordance with the terms of
this Agreement, (ii) the use of the Software and Documentation by the Service Provider will not
violate the terms of the export restrictions set forth herein, and (iii) the Service Provider is not a
Quest competitor. Customer shall be jointly and severally liable to Quest for the acts and
omissions of its Service Providers in connection with their permitted use of the Software and
Documentation.
(e) Freeware. If a freeware version of Quest software (Freeware) is downloaded by Customer
from http:///a Quest website, the terms of such use shall be governed by the applicable Freeware
definition provided at: www.quest.com/productguide.
3. Restrictions. Except to the extent expressly permitted by applicable law, and to the extent
that Quest is not permitted by such applicable law to exclude or limit the following rights,
Customer may not reverse engineer, decompile, disassemble, or attempt to discover or modify in
any way the underlying source code of the Products, Documentation or any part thereof. In
addition, Customer may not (i) modify, translate, localize, adapt, rent, lease, loan, create or
prepare derivative works of, or create a patent based on the Products, Documentation or any part
thereof, or (ii) resell the Products or Documentation or use the Products or Documentation in any
commercial time share arrangement, in connection with the operation of any nuclear facilities, or
for purposes which are competitive to Quest. Each permitted copy of the Software and
Documentation made by Customer hereunder must contain all titles, trademarks, copyrights and
restricted rights notices as in the original. Customer understands and agrees that the Products may
work in conjunction with third party products and Customer agrees to be responsible for
ensuring that it is properly licensed to use such third party products. Notwithstanding anything
otherwise set forth in this Agreement, the terms and restrictions set forth herein shall not prevent
or restrict Customer from exercising additional or different rights to any open source software
that may be contained in or provided with the Products in accordance with the applicable open
source licenses.
4. Reservation of Rights and Ownership. Quest reserves any and all rights, implied or
otherwise, which are not expressly granted to Customer in this Agreement. Customer understands
and agrees that (i) the Products are protected by copyright and other intellectual property laws
and treaties, (ii) Quest and/or its suppliers own the title, copyright, and other intellectual
property rights in the Products, (iii) the Software is licensed, and not sold, and (iv) this
Agreement does not grant Customer any rights to Quests trademarks or service marks.
5. Hardware. In the event Customer acquires Hardware under this Agreement, title to such
Hardware shall pass to Customer upon shipment (unless such Hardware is rented, leased or
loaned to Customer).
6. Payment. Customer agrees to pay to Quest (or, if applicable, the Partner) the fees specified in
each Order, including any applicable shipping fees. Customer will be invoiced promptly
following delivery of the Products or prior to the commencement of any Renewal Maintenance
Period and Customer shall make all payments due to Quest in full within thirty (30) days from
the date of each invoice or such other period (if any) stated in an Order signed by Quest. Any
388
amounts payable to Quest by Customer that remain unpaid after the due date shall be subject to
a late charge of 1.5% of the invoice amount per month from the due date until such amount is
paid, or the maximum rate permitted by law if less.
7. Taxes. The fees stated in an Order may not include taxes. If Quest is required to pay sales,
use, property, value-added or other taxes based on the Products or Maintenance Services
provided under this Agreement or on Customers use of Products or Maintenance Services, then
such taxes shall be billed to and paid by Customer. This Section does not apply to taxes based
on Quests income.
8. Termination. This Agreement and/or the License(s) granted hereunder may be terminated (i)
by mutual agreement of Quest and Customer, (ii) by Quest, if Customer or a Service Provider
commits a material breach of this Agreement and fails to cure such breach to Quests reasonable
satisfaction within thirty (30) days following receipt of Quests notice thereof, or (iii) by
Customer for any reason upon thirty (30) days written notice to Quest. Upon termination of this
Agreement or expiration or termination of a License for any reason, all rights granted to
Customer for the applicable License(s) shall immediately cease and Customer shall immediately:
(i) cease using the applicable Software and Documentation, (ii) return the applicable Software to
Quest together with all Documentation and other materials associated with the Software and all
copies of any of the foregoing, or destroy such items, (iii) cease using the Maintenance Services
associated with the applicable License(s), (iv) pay Quest or the applicable Partner all amounts
due and payable up to the date of termination, and (v) give Quest a written certification that
Customer has complied with all of the foregoing obligations. Termination of this Agreement or a
License shall be without prejudice to any other remedies that the terminating party may have
under law, subject to the limitations and exclusions set forth in this Agreement. Any provision of
this Agreement that requires or contemplates execution after termination of this Agreement or
expiration of a License is enforceable against the other party and their respective successors and
assignees notwithstanding termination or expiration, including, without limitation, the
Payment, Taxes, Termination, Warranty Disclaimer, Infringement, "Limitation of
Liability," "Nondisclosure," Usage Verification, and General Sections of this Agreement.
9. Export. Customer acknowledges and agrees that the Products are subject to the export control
laws, rules, regulations, restrictions and national security controls of the United States and other
applicable foreign agencies (the "Export Controls"), and agrees not to export or re-export, or
allow the export or re-export of the Products or any copy, portion or direct product of the
foregoing in violation of the Export Controls. Customer hereby represents that (i) Customer is not
an entity or person to which shipment of Products is prohibited by the Export Controls; and (ii)
Customer will not export, re-export or otherwise transfer the Products to (a) any country subject
to a United States trade embargo, (b) a national or resident of any country subject to a United
States trade embargo, (c) any person or entity to which shipment of Products is prohibited by the
Export Controls, or (d) anyone who is engaged in activities related to the design, development,
production, or use of nuclear materials, nuclear facilities, nuclear weapons, missiles or chemical
or biological weapons.
10. Maintenance. During any Maintenance Period and for the applicable fees, Quest shall make
available to Customer the Maintenance Services for the Software as defined in this Section. The
first Maintenance Period begins on the date of delivery of the Software following an Order and
ends twelve (12) months thereafter unless otherwise set forth in the applicable Order (the Initial
Maintenance Period). Following the Initial Maintenance Period, Maintenance Services shall
389
automatically renew for additional terms of twelve (12) months (each, a Renewal Maintenance
Period) unless the renewal has been cancelled by either party giving written notice to the other
at least sixty (60) days prior to the first day of the applicable Renewal Maintenance Period.
Cancellation of Maintenance Services will not terminate Customers rights to continue to use the
Software. Maintenance fees shall be due in advance of a Renewal Maintenance Period and shall
be subject to the payment requirements set forth in this Agreement. The procedure for reinstating
Maintenance Services after it has lapsed is posted at http://support.quest.com/Maintenance_
Service.asp. Except as otherwise stated in the Product Guide, Maintenance Services shall be
available via the Internet, e-mail, or telephone and shall mean the following:
(a) Quest shall make available to Customer new versions and releases of the Software, including
Software corrections, enhancements and upgrades, if and when Quest makes them generally
available without charge as part of Maintenance Services.
(b) Quest shall respond to unlimited communications from Customer that report Software failures
not previously reported to Quest by Customer. Nothing in the foregoing shall operate to limit or
restrict follow up communication by Customer regarding Software failures.
(c) Quest shall respond to requests from Customers technical coordinators for assistance with the
operational/technical aspects of the Software; provided that Quest shall have the right to limit
such responses if Quest determines, in its sole reasonable discretion, that on-site consulting
services would be more appropriate to address the scope and nature of the requests. Any such
onsite consultation would be pursuant to a services agreement as agreed upon by the parties.
(d) Customer shall have access to Quests Support Web site at http://support.quest.com
(SupportLink).
(e) Maintenance Services are available during standard support hours (Business Hours) as
indicated on SupportLink. In addition, Customer may purchase Business Critical Support (i.e.
24x7 Severity Level 1 support) for certain Software. The list of Software for which Business
Critical Support is available and/or required is set forth on SupportLink.
(f) During Business Hours, Quest will respond within one (1) hour to a call from Customer which
reports a critical Software condition (a Severity Level 1 Problem). Customer must use
commercially reasonable efforts to provide Quest with the necessary remote access to facilitate
the identification and resolution of a Severity Level 1 Problem. Quests ability to identify and
resolve a Severity Level 1 Problem may be delayed without such remote access.
(g) The Maintenance Services for those Software products that Quest has obtained through an
acquisition or merger may, for a period of time following the effective date of the acquisition or
merger, be governed by terms other than those in this Section 10. The applicable different terms,
if any, shall be stated on SupportLink.
11. Warranties.
(a) Software Warranty. Quest warrants that, for a period of thirty (30) days following the initial
delivery of Software pursuant to an Order (the Warranty Period), (i) the media provided by
Quest, if any, on which the Software is recorded will be free from material defects in materials
and workmanship under normal use, (ii) the operation of the Software, as provided by Quest, will
substantially conform to the Documentation applicable to such Software, and (iii) the Software as
390
delivered by Quest does not contain any viruses, worms, Trojan Horses, or other malicious or
destructive code designed by Quest to allow unauthorized intrusion upon, disabling of, or
erasure of the Software (however, the Software may contain a key limiting use of the Software to
within the scope of License granted, and license keys issued by Quest for temporary use are timesensitive) (the Warranties). Customer must give written notice to Quest of any breach of the
Warranties no later than five days following the expiration of the Warranty Period.
Customers exclusive remedies, and Quests sole obligations, for any such breach of these
Warranties shall be as follows: (a) for the warranty in subsection (i), Quest shall, at its
expense, replace any defective media; (b) for the warranty in subsection (ii), Quest shall
correct or provide a workaround for reproducible errors in the Software that cause a breach of
the warranty within a reasonable time considering the severity of the error and its effect on
Customer, or, at Quests option, refund the license fees paid for the nonconforming Software
upon return of such Software to Quest and termination of the related License(s) hereunder; and
(c) for the warranty in subsection (iii), Quest shall provide a copy of the Software that is in
conformance with such warranty.
The foregoing Warranties shall not apply to any non-conformance (i) that Quest cannot
recreate after exercising commercially reasonable efforts to attempt to do so; (ii) caused by
misuse of the Software or by using the Software in a manner that is inconsistent with this
Agreement or the Documentation; or (iii) arising from the modification of the Software by
anyone other than Quest.
(b) Hardware Warranty. Hardware shall be warranted in accordance with the warranty
document delivered with the Hardware and/or included on the hardware manufacturers website.
In the event Customer acquires Hardware that is delivered with a third party warranty (Third
Party Warranty), Customer will rely solely on the applicable third party for all Third Party
Warranty obligations.
(c) Warranty Disclaimer. THE EXPRESS WARRANTIES AND REMEDIES SET FORTH IN
THIS SECTION ARE THE ONLY WARRANTIES AND REMEDIES PROVIDED BY QUEST
HEREUNDER. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, ALL
OTHER WARRANTIES OR REMEDIES ARE EXCLUDED, WHETHER EXPRESS OR
IMPLIED, ORAL OR WRITTEN, INCLUDING ANY IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, NONINFRINGEMENT, SATISFACTORY QUALITY, AND ANY WARRANTIES ARISING FROM
USAGE OF TRADE OR COURSE OF DEALING OR PERFORMANCE. QUEST DOES NOT
WARRANT UNINTERRUPTED OR ERROR-FREE OPERATION OF THE PRODUCTS.
12. Infringement. Quest will at its own expense defend or settle any claim, suit, action, or
proceeding brought against Customer by a third party to the extent it is based on an allegation
that the Software directly infringes any patent, copyright, trademark, or other proprietary right
enforceable in the country in which the Software is delivered to Customer, or misappropriates a
trade secret in such country (a Claim). Additionally, Quest shall pay any judgments finally
awarded against Customer under a Claim or any amounts assessed against Customer in any
settlements of a Claim, and reasonable administrative costs or expenses, including without
limitation reasonable attorneys fees, necessarily incurred by Customer in responding to the
Claim. Quests obligations under this Section are conditioned upon Customer (i) giving prompt
written notice of the Claim to Quest; (ii) permitting Quest to retain sole control of the
391
investigation, defense or settlement of the Claim, and (iii) providing Quest with such cooperation
and assistance as Quest may reasonably request from time to time in connection with the
investigation, defense or settlement of the Claim. Quest shall have no obligation hereunder to
defend Customer against any Claim (a) resulting from use of the Software other than as
authorized in this Agreement, (b) resulting from a modification of the Software other than by
Quest, or (c) based on Customers use of the Software after Quest recommends discontinuation
because of possible or actual infringement, (d) based on Customers use of a superseded or altered
release of Software if the infringement would have been avoided by use of a current or unaltered
release of the Software made available to Customer, or (e) to the extent the Claim arises from or
is based on the use of the Software with other products, services, or data not supplied by Quest if
the infringement would not have occurred but for such use. If Customers use of the Software is
enjoined as a result of a Claim, Quest shall, at its expense and option either (i) obtain for
Customer the right to continue using the Software, (ii) replace the Software with a functionally
equivalent non-infringing product, (iii) modify the Software so that it is non-infringing, or (iv)
accept the return of the infringing Software and refund the license fee paid for the infringing
Software, pro-rated over a sixty (60) month period from the date of delivery of the Software
following an Order . This Section states the entire liability of Quest, and Customers sole and
exclusive remedy, with respect to a Claim.
13. Limitation of Liability. EXCEPT FOR (A) ANY BREACH OF THE "RESTRICTIONS" OR
"NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, (B) AMOUNTS CONTAINED IN
JUDGMENTS OR SETTLEMENTS WHICH QUEST IS LIABLE TO PAY ON BEHALF OF
CUSTOMER UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT, OR (C)
ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED
AS A MATTER OF LAW, IN NO EVENT SHALL QUEST, ITS AFFILIATES, OR
SUPPLIERS, OR CUSTOMER BE LIABLE FOR ANY LOSS OF REVENUE, LOSS OF
ACTUAL OR ANTICIPATED PROFITS, LOSS OF BUSINESS, LOSS OF CONTRACTS, LOSS
OF GOODWILL OR REPUTATION, LOSS OF ANTICIPATED SAVINGS, LOSS OF,
DAMAGE TO OR CORRUPTION OF DATA, OR FOR ANY INDIRECT, INCIDENTAL,
SPECIAL OR CONSEQUENTIAL LOSS OR DAMAGE OF ANY KIND, IN EACH CASE
HOWSOEVER ARISING, WHETHER SUCH LOSS OR DAMAGE WAS FORESEEABLE OR
IN THE CONTEMPLATION OF THE PARTIES AND WHETHER ARISING IN OR FOR
BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), BREACH OF
STATUTORY DUTY, OR OTHERWISE.
EXCEPT FOR (A) ANY BREACH OF CUSTOMERS PAYMENT OBLIGATIONS; (B) ANY
BREACH OF THE "SOFTWARE LICENSE," RESTRICTIONS," EXPORT OR
"NONDISCLOSURE" SECTIONS OF THIS AGREEMENT, OR ANY OTHER VIOLATION OF
THE OTHER PARTYS INTELLECTUAL PROPERTY RIGHTS; (C) QUESTS EXPRESS
OBLIGATIONS UNDER THE INFRINGEMENT SECTION OF THIS AGREEMENT; OR (D)
ANY LIABILITY TO THE EXTENT LIABILITY MAY NOT BE EXCLUDED OR LIMITED
AS A MATTER OF LAW, THE MAXIMUM AGGREGATE AND CUMULATIVE LIABILITY
OF QUEST, ITS AFFILIATES AND SUPPLIERS, AND CUSTOMER UNDER THIS
AGREEMENT, WHETHER ARISING IN OR FOR BREACH OF CONTRACT, TORT
(INCLUDING NEGLIGENCE), BREACH OF STATUTORY DUTY, OR OTHERWISE, SHALL
NOT EXCEED THE FEES PAID AND/OR OWED (AS APPLICABLE) BY CUSTOMER FOR
THE PRODUCTS OR MAINTENANCE SERVICES THAT ARE THE SUBJECT OF THE
CLAIM. FOR MAINTENANCE SERVICES OR A PRODUCT SUBJECT TO RECURRING
392
FEES, THE LIABILITY SHALL NOT EXCEED THE AMOUNT PAID AND/OR OWED (AS
APPLICABLE) FOR SUCH MAINTENANCE SERVICE OR PRODUCT DURING THE
TWELVE (12) MONTHS PRECEDING THE CLAIM.
Quests Affiliates and suppliers shall be beneficiaries of this Limitation of Liability" section and
Customers Service Providers are entitled to the rights granted under the "Third Party Use"
section of this Agreement; otherwise, no third party beneficiaries exist under this Agreement.
Quest expressly excludes any and all liability to Customers Service Providers, Clients and to
any other third party.
14. Nondisclosure. Confidential Information means information or materials disclosed by one
party (the Disclosing Party) to the other party (the Receiving Party) that are not generally
available to the public and which, due to their character and nature, a reasonable person under
like circumstances would treat as confidential, including, without limitation, the Disclosing
Partys personal data, financial information, marketing information, trade secrets, know-how,
proprietary tools, proprietary knowledge and proprietary methodologies. Confidential Information
of Quest further includes this Agreement, the Products (in source code and/or object code form),
the pricing and discounting offered by Quest to Customer hereunder, information regarding the
functionality and performance of the Products, benchmark test results regarding the Products, and
any Software license keys provided to Customer. Additionally, Confidential Information shall
include Individually Identifiable Health Information (as that term is defined in 45 CFR
164.501) or Nonpublic Personal Information (as that term is defined in Title V of the GrammLeach-Bliley Act of 1999) that Quest may come into contact with under this Agreement.
Confidential Information shall not include information or materials that (a) were, on the date of
disclosure, generally known to the public; (b) become generally known to the public other than
as a result of the act or omission of the Receiving Party; (c) were known to the Receiving Party
without an obligation of confidentiality prior to that party receiving the same from the
Disclosing Party; (d) the Receiving Party lawfully received from a third party without that third
partys breach of agreement or obligation of trust; or (e) are or were independently developed by
the Receiving Party without access to or use of the Disclosing Partys Confidential Information.
Additionally, it shall not be a breach of this Section for the Receiving Party to disclose the
Disclosing Partys Confidential Information as may be required by operation of law or legal
process, provided that the Receiving Party provides prior notice of such disclosure to the
Disclosing Party unless expressly prohibited from doing so by a court, arbitration panel or other
legal authority of competent jurisdiction. The Receiving Party shall not (1) make the Disclosing
Partys Confidential Information available to any Affiliates, directors, officers, employees,
consultants or representatives (collectively, the Representatives) who do not have a need to
know in order to carry out the purposes of this Agreement; (2) otherwise disclose the Disclosing
Partys Confidential Information to any third party without the written consent of the Disclosing
Party; or (3) use the Disclosing Partys Confidential Information for any purpose other than as
contemplated by this Agreement. The Receiving Party shall inform its Representatives of the
confidential nature of the Disclosing Partys Confidential Information and the requirements
regarding restrictions on disclosure and use as set forth in this Section and shall disclose the
Disclosing Partys Confidential Information only to its Representatives who are legally bound to
protect the Confidential Information under terms at least as restrictive as those provided herein.
The Receiving Party agrees to protect the Disclosing Partys Confidential Information from
unauthorized use or disclosure by exercising at least the same degree of care it uses to protect its
own similar information, but in no event less than a reasonable degree of care. The Receiving
393
Party shall be liable to the Disclosing Party for any disclosure or other breach in violation of this
Agreement by any of its Representatives. The Receiving Party shall promptly notify the
Disclosing Party of any known unauthorized use or disclosure of the Disclosing Partys
Confidential Information and will cooperate with the Disclosing Party in any litigation brought
by the Disclosing Party against third parties to protect its proprietary rights.
15. Usage Verification. At Quest's request, but not more frequently than once per year, Customer
shall furnish Quest with a document signed by an authorized representative verifying Customer's
installations and usage of the Products. Customer will permit Quest to review Customer's
deployment and use of the Products for compliance with the terms and conditions of this
Agreement. Any such reviews shall be scheduled at least fifteen (15) days in advance, shall be
conducted during normal business hours at Customer's facilities, and shall not unreasonably
interfere with Customer's business activities. If Customer's use of the Products is found to be
greater than contracted for Customer will be invoiced for the additional use and the unpaid fees
shall be payable in accordance with this Agreement. Additionally, if the unpaid fees exceed five
percent (5%) of the fees paid for the subject Products, then Customer shall also pay Quest's
reasonable costs of conducting the audit. This Section shall not limit or restrict any other rights
or remedies of Quest that are otherwise set forth in this Agreement or available at law.
16. General.
(a) Governing Law and Venue. This Agreement shall be governed by and construed in
accordance with the laws of the State of California, without giving effect to any conflict of laws
principles that would require the application of laws of a different state. The parties agree that
neither the United Nations Convention on Contracts for the International Sale of Goods, nor the
Uniform Computer Information Transaction Act (UCITA) shall apply to this Agreement,
regardless of the states in which the parties do business or are incorporated. Any action seeking
enforcement of this Agreement or any provision hereof shall be brought exclusively in the state
or federal courts located in the County of Orange, State of California, United States of America.
Each party hereby agrees to submit to the jurisdiction of such courts.
(b) Assignment. Except as otherwise set forth herein, Customer shall not, in whole or part, assign
or transfer any part of this Agreement or any rights hereunder without the prior written consent
of Quest. Any attempted transfer or assignment by Customer that is not permitted by this
Agreement shall be null and void.
(c) Severability. If any provision of this Agreement shall be held by a court of competent
jurisdiction to be contrary to law, such provision will be enforced to the maximum extent
permissible and the remaining provisions of this Agreement will remain in full force and effect.
Notwithstanding the foregoing, the terms of this Agreement that limit, disclaim, or exclude
warranties, remedies or damages are intended by the parties to be independent and remain in
effect despite the failure or unenforceability of an agreed remedy. The parties have relied on the
limitations and exclusions set forth in this Agreement in determining whether to enter into it.
(d) Use by U.S. Government. The Software is a commercial item under FAR 12.201.
Consistent with FAR section 12.212 and DFARS section 227.7202, any use, modification,
reproduction, release, performance, display, disclosure or distribution of the Software or
Documentation by the U.S. government shall be governed solely by the terms of this Agreement
and shall be prohibited except to the extent expressly permitted herein.
394
(e) Personal Data. Customer hereby acknowledges and agrees that Quests performance of this
Agreement may require Quest to process or store personal data of Customer, its employees and
Affiliates and to transmit such data internally within Quest or to Quest Affiliates. Such
processing, storage, and transmission shall only be to the extent necessary for, and for the sole
purpose of, enabling Quest to perform its obligations under this Agreement and may take place
in any of the countries in which Quest and its Affiliates conduct business, which may include
countries outside of the European Economic Area. Quest hereby affirms to Customer that Quest
Software, Inc. currently abides by the safe harbor framework as set forth by the U.S. Department
of Commerce regarding the collection, use and retention of data from the European Union.
(f) Notices. All notices provided hereunder shall be in writing, delivered personally, sent by
facsimile or e-mail, or mailed by first class mail, postage prepaid, addressed to the legal
department of the respective party or to such other address as may be specified in an Order or in
writing by either of the parties to the other in accordance with this Section. All notices, requests,
demands or communications shall be deemed effective upon personal delivery or four (4) days
following deposit in the mail in accordance with this paragraph.
(g) Disclosure of Customer Status. Quest may include Customer in its listing of customers and,
upon written consent by Customer, announce Customer's selection of Quest in its marketing
communications.
(h) Waiver. Performance of any obligation required by a party hereunder may be waived only by
a written waiver signed by an authorized representative of the other party, which waiver shall be
effective only with respect to the specific obligation described therein. Any waiver or failure to
enforce any provision of this Agreement on one occasion will not be deemed a waiver of any
other provision or of such provision on any other occasion.
(i) Injunctive Relief. Each party acknowledges and agrees that in the event of a material breach
of this Agreement, including but not limited to a breach of the Software License," Restrictions
or "Nondisclosure" Sections of this Agreement, the non-breaching party shall be entitled to seek
immediate injunctive relief, without limiting its other rights and remedies.
(j) Force Majeure. Each party will be excused from performance for any period during which,
and to the extent that, it is prevented from performing any obligation or service as a result of
causes beyond its reasonable control, and without its fault or negligence, including without
limitation, acts of God, strikes, lockouts, riots, acts of war, epidemics, communication line
failures, and power failures. Nothing in the foregoing shall be deemed to relieve Customer or its
Affiliates of its obligation to pay fees owed under this Agreement.
(k) Equal Opportunity. Quest Software Inc. is a federal contractor and Affirmative Action
employer (M/F/D/V) as required by the Equal Opportunity clause C.F.R. 60-741.5(a).
(l) Headings. Headings in this Agreement are for convenience only and do not affect the
meaning or interpretation of this Agreement. This Agreement will not be construed either in
favor of or against one party or the other, but rather in accordance with its fair meaning. When
the term including is used in this Agreement it will be construed in each case to mean
including, but not limited to.
(m) Entire Agreement. This Agreement is intended by the parties as a final expression of their
agreement with respect to the subject matter hereof and may not be contradicted by evidence of
395
any prior or contemporaneous agreement unless such agreement is signed by both parties. In the
absence of such an agreement, this Agreement shall constitute the complete and exclusive
statement of the terms and conditions and no extrinsic evidence whatsoever may be introduced
in any judicial proceeding that may involve the Agreement. In the event of a conflict between
the terms of this Agreement and the terms contained in an Order, the terms in the Order shall
only control if the Order is signed by both Quest and Customer; otherwise, the terms of this
Agreement shall control. Neither this Agreement, nor an Order, may be modified or amended
except by a writing executed by a duly authorized representative of each party. No other act,
document, usage or custom shall be deemed to amend or modify this Agreement or an Order.
Delivery of Products shall be FOB Shipping Point.
Appendix:Contact Quest
Contact Quest/About Quest Software
About Quest
Quest Software simplifies and reduces the cost of managing IT for more than 100,000 customers
worldwide. Our innovative solutions make solving the toughest IT management problems easier,
enabling customers to save time and money across physical, virtual and cloud environments. For
more information about Quest go to www.quest.com.
Established in 1987, Quest Software (Nasdaq: QSFT) provides simple and innovative IT
management solutions that enable more than 100,000 global customers to save time and money
across physical and virtual environments. Quest products solve complex IT challenges ranging
from database management, data protection, identity and access management, monitoring, user
workspace management to Windows management. For more information, visit www.quest.com.
info@quest.com
Quest Software, Inc.
World Headquarters
5 Polaris Way
Aliso Viejo, CA 92656
USA
Web site
www.quest.com
See our web site for regional and international office information.
ContactingQuest Support
Quest Support is available to customers who have a trial version of a Quest product or who have
purchased a Quest product and have a valid maintenance contract. Quest Support provides
unlimited 24x7 access to our Support Portal at www.quest.com/support.
From our Support Portal, you can do the following:
l
397
Contacting Quest
View the Global Support Guide for a detailed explanation of support programs, online services,
contact information, policies and procedures. The guide is available at: www.quest.com/support.
Create_Delete_Benchmark 62, 67
Index
Creation 100
A
Advanced 100
Advanced Creation Objects 100
Data 371
Agent 154
Agents Overview 36
AS3AP Benchmark 277
B
Environment 20
Environment#Understanding_
Benchmarks 21
Excel 274
Exponential 86
BFCreditCardExp 290
Exporting 274
BFFileArray 293
BFFormat 300
Factory 20-21, 363
BFMaxNode 314
BFNumberOfIterations 314
BFState 299
Getting Started 17
BFTrimRight 312
BFURandRange 303
BFUserID 318
Clustering 169
Command Line Wizard 371
Create 227
Create Execute prepared SQL Statement
Transaction 231
How Do I...
Create a Load Scenario 142, 175
I
Import Stored Procedure 200, 225
399
Index
Project Node 48
Project View 44
L
Latency 86
Properties Overview 47
Properties View 47
Latency_Keying 85
Latency_Uniform 86
Queue#Jobs_Queue_Toolbar 45
Load Scenario Wizard 176, 192-193, 195196, 200-202, 204-210, 212, 225
Migrating 371
Mix 208
Repository Manager 41
Negative 86
New 17
Objects 100
Oracle's Instant Client 370
Output View 47
Script 48
Overview 237
P
Prepare 227
Preview 113
Server 363
400
Index
What's 17