GG 244131

Using Resource Measurement Facility Monitor III Efficiently
Document Number GG24-4131-00
November 1993
International Technical Support Organization

Poughkeepsie
Take Note!
Before using this information and the product it supports, be sure to read the general information under
Special Notices on page vii.
First Edition (November 1993)

This edition applies to Version 4 Release 3.0 of Resource Measurement Facility for use with the MVS/ESA Version
4 Release 3.0.
Order publications through your IBM representative or the IBM branch office serving your locality. Publications
are not stocked at the address given below.
An ITSC Technical Bulletin Evaluation Form for readers feedback appears facing Chapter 1. If the form has been
removed, comments may be addressed to:
IBM Corporation, International Technical Support Center
Dept. H52 Mail Station P402
522 South Road
Poughkeepsie, New York 12601-5400
When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any
way it believes appropriate without incurring any obligation to you.
Copyright International Business Machines Corporation 1993. All rights reserved.
Note to U.S. Government Users Documentation related to restricted rights Use, duplication or disclosure is
subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.
Abstract
Resource Measurement Facility (RMF) Monitor III is an interactive monitor that
collects data and reports contention for resources and their users. The data
allows identification of system bottlenecks and determination of the reasons for
possible system performance degradations.
This document, along with its corresponding on-line demonstration, which can be
set up using the diskette included in this book, illustrates how RMF Monitor III
Version 4 Release 3.0 can be used to monitor the system and to identify the
reasons for job and resource delays.
LS
Copyright IBM Corp. 1993
(140 pages)
iii
iv
RMF MonIII Demonstration
Contents
Abstract
Special Notices
Part 1. How to Use the Diskette

Part 2. Text for Demonstration
. . . . . . . . . . . . . . . . . . . . . . . . . . .
ix
ix
ix
ix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 1. RMF Monitor III Basics
. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 2. Demonstration 1 - Analyzing System Workloads
. . . . . . . . . . .
Chapter 3. Demonstration 2 - Monitoring System Performance

Chapter 4. Additional Features
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
. . . . . . . . . . . . . . . .
71
. . . . . . . . . . . . . . . . .
81
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
43
Chapter 6. Installation Performance Specification
Index
11
. . . . . . . . .
Chapter 5. Specifications for System Trend Report
Part 4. Foils
vii
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Preface
. . . . . . . . . . . . . . .
How This Document is Organized
. . . . . . .
Related Publications
Acknowledgments . . . . . . . . .
Part 3. Specifications
iii
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
151
vi
Special Notices
This publication, and the demonstrations that can be set up using the diskette
included in the book is intended to help IBM System Engineers and customers
understand how to use the Resource Measurement Facility Monitor III efficiently.
The information in this publication is not intended as the specification of any
programming interfaces that are provided by Resource Measurement Facility.
See the PUBLICATIONS section of the IBM Programming Announcement for
Resource Measurement Facility for more information about what publications are
considered to be product documentation.
References in this publication to IBM products, programs or services do not
imply that IBM intends to make these available in all countries in which IBM
operates. Any reference to an IBM product, program, or service is not intended
to state or imply that only IBMs product, program, or service may be used. Any
functionally equivalent program that does not infringe any of IBMs intellectual
property rights may be used instead of the IBM product, program or service.
Information in this book was developed in conjunction with use of the equipment
specified, and is limited in application to those specific hardware and software
products and levels.
IBM may have patents or pending patent applications covering subject matter in
this document. The furnishing of this document does not give you any license to
these patents. You can send license inquiries, in writing, to the IBM Director of
Commercial Relations, IBM Corporation, Purchase, NY 10577.
The information contained in this document has not been submitted to any
formal IBM test and is distributed AS IS. The information about non-IBM
(VENDOR) products in this manual has been supplied by the vendor and IBM
assumes no responsibility for its accuracy or completeness. The use of this
information or the implementation of any of these techniques is a customer
responsibility and depends on the customers ability to evaluate and integrate
them into the customers operational environment. While each item may have
been reviewed by IBM for accuracy in a specific situation, there is no guarantee
that the same or similar results will be obtained elsewhere. Customers
attempting to adapt these techniques to their own environments do so at their
own risk.
Any performance data contained in this document was determined in a
controlled environment, and therefore, the results that may be obtained in other
operating environments may vary significantly. Users of this document should
verify the applicable data for their specific environment.
You can reproduce a page in this document as a transparency, if that page has
the copyright notice on it. The copyright notice must appear on each page being
reproduced.
The following terms, which are denoted by an asterisk (*) in this publication, are
trademarks of the International Business Machines Corporation in the United
States and/or other countries:
IBM
OS/2
MVS/ESA
vii
viii
Preface
This material is intended to help IBM System Engineers and customers make the
most of RMF Monitor III. The material includes this book and an included
diskette. The book contains, after a basic presentation of RMF Monitor III, a
support for the two demonstrations that can be set up easily on an MVS system
from the diskette. The diskette contains the files needed for the demonstrations;
namely, two RMF Monitor III data sets and the JCL to unload them. Foils are
provided that can help make up an RMF Monitor III presentation.
How This Document is Organized

The document is organized as follows:
Part 1, How to Use the Diskette describes how to use the diskette included
in this book.
Part 2, Text for Demonstration provides an overview of RMF Monitor III

and provides an explanation of the data you see during the demonstration.
Part 3, Specifications provides additionnal and more technical

specifications related to installation performance and a system trend report.
Part 4, Foils provides the foils to use for a presentation, whether or not
you use the demonstrations.
Related Publications
Resource Measurement Facility Version 4 Users Guide, GC28-1058
Resource Measurement Facility Version 4 Release 3.0 Technical Overview,

GG24-3898
Resource Measurement Facility Getting Started on Performance

Management, LY33-9174 (available December 1993)
Acknowledgments
The author of this document is:
Robert Vaupel IBM Bblingen
The conceptor of this document is:
Christian Matthys IBM International Technical Support Center, Poughkeepsie.
Thanks to the following people for the invaluable advice and guidance provided
in the production of this document:
Peter Enrico IBM Poughkeepsie
Hans Osterod IBM Bblingen
Gnther Pfuderer IBM Bblingen
Richard Rothermel IBM Bblingen
Michael Teuffel IBM Bblingen
ix

This part describes how to download the demonstrations files from the diskette
to an MVS system.
Warning
Though the following steps have been tested in a real environment, some
installations (MVS environment or PC environment) may have different
defaults options. That means some minor changes may have to be made to
use the provided files exactly as described.
Prerequisites
To set up the demonstrations, you must have the following:
MVS/ESA* Version 4 Release 3.0 with RMF Version 4 Release 3.0 installed on
the system.
Note: If you do not have this MVS Version/Release installed, an internal IBM
center with this release installed can help you use this demonstration.
About four available cylinders on a diskpack.
A PS/2, a PC, or the equivalent with DOS or OS/2 to transfer the

demonstration files from the diskette.
Diskette Contents
The diskette contains:
Two files, RMFCC98.SEQ and RMFCC99.SEQ, which are sequential data sets.
They contain data that will be unloaded in two VSAM Monitor III data sets.
File VSDEF.JCL, which is the MVS JCL to allocate two VSAM data sets.
File SQTOVS.JCL, which is the MVS JCL to import the sequential data sets to
the defined VSAM data sets.
Two files ERB3PRM.PAN, and ERB3USR.PAN, which are, respectively, a

changed RMF Primary Menu and a selection menu for your own reports.
File USGTREND.CLI, which is a REXX CLIST to create the System Trend User
report.
File IEAIPS00.PLB which is an IEAIPS parmlib member that should be used

in your demonstartion by RMF to setup the best RMF system options for
these demonstrations.
Unloading the Files

This chapter describes the steps you use to set up the demonstration:
1. Receive the diskette files on your MVS system from a DOS or an OS/2
window, using the following commands:
SEND a:\RMFCC98.SEQ x:RMFCC98.SEQ LRECL (32752)

(1080816 bytes to transfer)
SEND a:\RMFCC99.SEQ x:RMFCC99.SEQ LRECL (32752)

SEND a:\SQTOVS.JCL x:dsname(SQTOVS) ASCII CRLF

SEND a:\VSDEF.JCL x:dsname(VSDEF) ASCII CRLF

SEND a:\IEAIPS00.PLB x:parname(IEAIPS00) ASCII CRLF

SEND a:\ERB3PRM.PAN x:panname(ERB3PRM) ASCII CRLF

SEND a:\ERB3USR.PAN x:panname(ERB3USR) ASCII CRLF

SEND a:\USGTREND.CLI x:cdsname(USGTREND) ASCII CRLF

where:
a: is the diskette reader of your PC or PS.
x: is the window of your MVS system on your PC or PS.
dsname is the name of an existing partitionned data set to receive the

JCL.
parname is the name of an existing partionned data set to receive the

IPS member.
panname is the name of an existing partitionned data set to receive

the panel.
cdsname is the name of an existing partitionned data set to receive a

clist.
RMFCC98.SEQ (idem for RMFCC99.SEQ) is the name of a sequential

data set that will be dynamically allocated. On your system, this data
may be prefixed by the userid.
2. On your MVS system, allocate the VSAM data sets, using the VSDEF JCL
member, as shown in the following example, and customized for your
environment:
//USERX
JOB 111111,Y25A,X,USERX,MSGCLASS=H,CLASS=J,
//
NOTIFY=USERX,
//
MSGLEVEL=(1,1)
//**************************************************************
//* DEFINES A VSAM DATASET FOR MONITOR III
//**************************************************************
//*
//DEFINE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN
DD *
DEFINE CLUSTER
(NAME(userx.RMFCC99) CYLINDERS(2 2)
VOLUME(volnam)
NUMBERED
REUSE
SHAREOPTIONS( 2 3 )
CONTROLINTERVALSIZE(32768) BUFFERSPACE (65536)
RECORDSIZE( 32752 32752) )
DEFINE CLUSTER
(NAME(userx.RMFCC98) CYLINDERS(2 2)
VOLUME(volnam)
NUMBERED
REUSE
SHAREOPTIONS( 2 3 )
CONTROLINTERVALSIZE(32768) BUFFERSPACE (65536)
RECORDSIZE( 32752 32752) )
/*
3. Import the sequential data sets into the VSAM data sets using the SQTOVS
JCL member, as shown in the following example:
//USERX
JOB 111111,Y25A,X,USERX,MSGCLASS=H,CLASS=J,
//
NOTIFY=USERX,
//
MSGLEVEL=(1,1)
//CSQTOVS EXEC PGM=IDCAMS
//INDD
DD DISP=SHR,DSN=userx.RMFCC98.SEQ
//OUTDD
DD DISP=SHR,DSN=userx.RMFCC98
//IND2
DD DISP=SHR,DSN=userx.RMFCC99.SEQ
//OUTD2
DD DISP=SHR,DSN=userx.RMFCC99
//SYSPRINT DD SYSOUT=*
//SYSIN
DD *
REPRO INFILE(INDD) OUTFILE(OUTDD)
REPRO INFILE(IND2) OUTFILE(OUTD2)
//
(33 records for RMFCC98 and 8 records for RMFCC99 have to be recorded).
4. Make sure that:
The cdsname is put into your SYSEXEC or your SYSPROC concatenation.

We suggest that you copy the standard logon clist from the system
library to your own library; then you add your own library in front of the
system library. Hence, you can log on from your own clist library in the
log-on panel.
The panname is either your RMF panel library or a library that will be
concatenated in front of your main RMF panel library (ERBPLIB ddname).
We suggest that you copy the ERBRMF3X clist from your standard
system libraries to your user clist library; then change the ALLOCATION
ERBPLIB so that your own panel library is accessed first.
5. Under TSO, allocate the VSAM data sets to the Monitor III using the following
command:
ALLOC FI(RMFDS00) DA(userx.RMFCC99) SHR

ALLOC FI(RMFDS01) DA(userx.RMFCC98) SHR
6. Check the validity of your unloading; try the following:
Note: Starting RMF is not mandatory because the demonstrations do not
use the data gatherer.
On your MVS system, log on to a Monitor III session reporting; in TSO:
RMF
You must have the menu as displayed in Figure 1.
RMF Monitor III Primary Menu -- Demo

Version 4, Release 3.0
Enter selection number or command on selection line.
Basic Reports
1 WFEX
Workflow/Exceptions (WE)
2 SYSINFO System information (SI)
3 GROUP
Group response time (RT)
Group =>
4 GROUPS Group reports selection
5 JOB
Jobs primary delay
Job =>
6 JOBS
Job report selection
Detail Reports
7 DELAY
Delays (DLY)
8 DEV
Device delays (DD)
9 DEVR
Device resource (DR)
10 ENQ
ENQ delays (ED)
11 ENQR
ENQ resource (ER)
12 PROC
Processor delays (PD)
13 STORAGE Storage report selection
14 SUBS
Subsystem report selection
Other Selections
DI DSINDEX Dataset index
T TUTORIAL Tutorial aid
OP OPTIONS Option selection
X EXIT
End display session
US USER
Private Reports
5685-029 (C) Copyright IBM Corp. 1986, 1993. All Rights Reserved
Selection ===>
F1=HELP
F2=SPLIT
F3=END
F4=RETURN
F5=RFIND
F6=TOGGLE
F7=UP
F8=DOWN
F9=SWAP
F10=LEFT
F11=RIGHT
F12=RETRIEVE
Figure 1. Demonstration Primary Menu
Choose the DI option, typing DI on the command line. An equivalent

menu must appear, showing clearly the Monitor III used data set, as
shown in Figure 2.
Samples:
RMF 4.3.0 Data Set Index -- AQTS

100 Time: 03/31/93 at 15.10.00
Line 1 of 2
Range +
100 Sec
--Date-- --Time--
-DDNAME-
---------------Data Set Name--------
03/30/93 14.20.00
14.28.20
RMFDS01
userx.RMFCC98
03/31/93 15.10.00
15.11.40
RMFDS00
userx.RMFCC99
Graphic version of this report is not available.

Command ===>
F1=HELP
F2=SPLIT
F3=END
F4=RETURN
F7=UP
F8=DOWN
F9=SWAP
F10=BREF
F5=RFIND
F11=FREF
Scroll ===>CSR
F6=TOGGLE
F12=RETRIEVE
Figure 2. Monitor III Used Data Set
7. Set up the options.

In order to be able to compare to the reports described in the book, the
IEAIPS00 Parmlib member must be included into a partitioned data set and
this data set must be specified to RMF. For that, from the RMF Primary
Panel, invoke the SESSION option panel, typing SESSION on the command
line. The panel shown in Figure 3 is displayed. Enter YES in the
Customization line and the parname in the Input Data Set line as in the
figure.
RMF Session Options

Current option set: IEAIPS00 on AQTS
Change or verify parameters. Press END to save and end.
Mode
First Screen
Refresh
Range
===>
===>
===>
===>
GO
PRIMARY
100
100S
Time Limit
===> NONE
DSINDEX Order
Hardcopy
SYSOUT
Output Data Set
===>
===> OFF
===> A
===>
Initial mode (STOP GO)

Initial screen selection (ex: PRIMARY)
Refresh period (in seconds)
Time range 10-9999 sec (ex: 100S, 100)
1-166 min
(ex: 2M)
Time limit or NONE
1-999 min
(ex: 10M)
1-128 hours (ex: 1H)
(ASCEND DESCEND)
(ex: ASCEND)
Hardcopy mode (ON OFF) (ex: ON)
Class for printed output (ex: A)
Data set for hardcopy (Overrides SYSOUT)

Report Format ===> GRAPHIC
(GRAPHIC TABULAR)
(ex: GRAPHIC
Customization ===> YES
Automatically tailor WFEX re port (YES NO)
Input Data Set ===> parname
Command ===>
Data set with information for customization
F1=HELP
F7=UP
F2=SPLIT
F8=DOWN
F3=END
F9=SWAP
F4=RETURN
F10=BREF
F5=RFIND
F11=FREF
F6=TOGGLE
F12=RETRIEVE
Figure 3. Setting the Option Panel
8. Check IEAIPS00.
To be sure you use the IEAIPS00 provided member, which you need for the
second demonstration, please perform follow the following steps:
a. Step 1:
1) Start RMF as explained in 6 on page 4.
2) From the command line, type SESSION.
3) Change the Customization and Input Data Set fields to reflect your
own library.
4) Quit this panel, using PF3.
5) On the Primary panel, type WFEX on the command line.
6) The top part of the panel must show the Btch user name as shown
in Figure 4.
Note: Depending on your installation, an ALL OMVS user or a TSO
extra user can also be shown.
RMF 4.3.0 Workflow/Exceptions -- AQTS

Line 1 of 5
HARDCOPY
Samples:
100 Time: 03/31/93 at 15.10.00 Range +
100 Sec
--------------------------- Speed (Workflow) --------------------------------Speed of 100 = Maximum, 0 = Stopped
Average CPU Util: 94 %
Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
577
17
49
TSO short
467
0
73
ALL TSO
479
6
66
TSO longer
5
4
68
ALL BATCH
11
8
30
TSO medium
1
1
40
ALL STC
87
2
70
TSO long
2
1
68
ALL ASCH
Not avail
Btch short
2
2
23
*DMN000
35
1
78
Btch long
9
6
32
STCs
53
1
67
TSO extra
1
0
No work
Figure 4. Sample of WFEX Report when using the IEAIPS00 M e m b e r User
7) if you do not get the Btch user, please go to Step 2; you have to get
rid of an IEAIPS00 member that was installed before in your
installation. If you got this panel with the Btch user, your installation
is well done; please go to Part 2, Text for Demonstration on
page 9.
b. Step 2:
1)
2)
3)
4)
5)
6)
7)
8)
From the Primary panel, type OPTSET on the command line.

Type an S (as Select) in front of INITIAL.
Leave RMF.
Restart RMF.
From the Primary panel, type OPTSET.
Type a D (as Delete) in front of IEAIPS00.
Press PF3.
Restart at Step 1.
Part 2. Text for Demonstration

This part contains the foils in small format and text that you need to understand
the demonstration and to make a presentation.
The presentation itself is divided in four chapters:
Chapter 1, RMF Monitor III Basics on page 11 presents a technical

overview of RMF Monitor III. If you are familiar with the RMF Monitor III
concepts, you can bypass this part.
Chapter 2, Demonstration 1 - Analyzing System Workloads on page 31

presents the first demonstration. The data used in this part is included in the
diskette. It must be downloaded to an MVS system, as described in Part 1,
How to Use the Diskette on page 1.
Chapter 3, Demonstration 2 - Monitoring System Performance on page 43

presents the second demonstration. The data used in this part is included in
the diskette. It must be downloaded to an MVS system, as described in
Part 1, How to Use the Diskette on page 1.
Chapter 4, Additional Features on page 61 explains some additionnal

features that are useful when writing your own reports.
RMF Monitor III is an interactive monitor that collects data and reports about
contention for resources and their users. The data allows you to identify system
bottlenecks and to determine the reasons for possible system performance
degradation.
This presentation, along with its corresponding on-line demonstrations,
illustrates how RMF Monitor III Version 4 Release 3.0 can be used to monitor the
system and to identify the reasons for job and resource delays. The introductory
part outlines the concept of RMF Monitor III, its terminology, and report types.
The demonstrations are based on two case studies with data from an MVS/ESA
Version 4 Release 3.0 system. They illustrate different techniques to identify
problems and analyze them.
The first demonstration highlights the decomposition of response time for system
workloads in order to analyze reasons for delays. The second demonstration
stresses how RMF Monitor III can be used for immediate problem determination
and for identification of jobs and resources with exceptional use.
Agenda
This presentation provides an overview of the RMF Monitor III functions and their
intended usage.
First there will be a brief description of what RMF Monitor III is and an
introduction to the basic concepts of the performance monitor.
The second part gives an overview of the RMF Monitor III reporter, its reports,
and how they can be used to monitor system activity. Selected data used in two
case studies during an on-line demonstration will provide an idea of how RMF
Monitor III detects performance problems and how it can be used to determine
the reason for such problems.
The last section presents an overview of additional features that enhance the
capability and usability of the monitor and make it more valuable for the system
analyst.
10
Using RMF Monitor III Efficiently

What is RMF Monitor III?
Use Monitor III to:
Determine whether your MVS system runs without

performance problems
Identify jobs that are delayed and the reason for

the delay
Identify the resources associated with delays
Identify resources with exceptional utilization
Monitor the availability of system resources
RMF Monitor III Basics

Monitor III Concepts
Monitor III Usage
GG24-4131
Copyright IBM CORP. 1993
Basics-1
GG24-4131
Basics-2

Every MVS installation has the goal of having all MVS systems running without
problems. Using Monitor III helps you to monitor MVS system performance.
If performance problems occur, use Monitor III to identify their cause and to
determine actions for their correction. RMF Monitor III assists you in pinpointing
performance problems before they become critical.
RMF Monitor III reports provide information about the activity of individual jobs,
specific groups of jobs, and all jobs in the system. For each job or group of jobs,
Monitor III reports the delay experienced over the report interval. It also shows
the primary cause and possible reason for the delay.
RMF Monitor III measures delay associated with I/O devices, serially reusable
resources, subsystems (JES, HSM, XCF), processors, paging and swapping, and
operator requests (mounts and message). The provided reports enable your
installation to determine how the delays affect the jobs in the system and
furthermore, give information regarding the contention for the resources
associated with the delay.
11
RMF Monitor III allows the performance analyst to monitor the availability of
hardware (processor and devices) and software (jobs) system resources.
12

What is RMF Monitor III? ...

Data Reporter
Data Gatherer
Multiple sessions per MVS system possible that:
One session per MVS system which:
Collects data for short-term problem solving
Collects data about system delays
Collects data about system resource utilization
Keeps data available in an in-storage buffer for

immediate problem determination
Run under TSO/E and ISPF
Report on current data,

or report on earlier collected data,
or report on data from another system
Records compressed data in VSAM data sets for

later use
Provide powerful features to monitor the system

and identify performance problems:
Creates SMF records
GG24-4131
Basics-3
GG24-4131
Continous Monitoring (GO Mode)

Exception Processing
Automatic Customization
Cursor Sensitive Control
Basics-4
RMF Monitor III Data Gatherer

The RMF Monitor III data gatherer collects data for the Monitor III reporter. The
operator or system programmer starts the data gatherer and can issue
commands to control its processing. This data is available to all RMF Monitor III
reporter sessions running on that system.
All the data collected by the gatherer over time can be stored in installation
defined VSAM data sets by means of the data set support function. This feature
allows a user to analyze system failures after they have happened without
having to recreate the situation. Selected data from the RMF Monitor III data
gatherer is written into SMF records. See Chapter 4, Additional Features on
page 61 for more details.
RMF Monitor III Data Reporter

The RMF Monitor III reporter session runs under TSO and ISPF. All the features
available under ISPF (such as program function keys and ISPF commands) are
available to the user.
The RMF Monitor III reporter reports on current data. Using pre-allocated data
created by the data set support function, it is also able to report on previously
collected data and data from other systems.
Monitor III provides powerful functions that enable continuous monitoring of the
latest data (GO Mode), exception processing to identify system problems,
13
automatic customizing based on the installation performance parameters, and

cursor-sensitive control to navigate through the current data and identify the
reason for system delays.
14

RMF Monitor III Concepts
How does RMF Monitor III gather data?

Cycle = {50ms, ..., 1000ms, ..., 9999ms}
Mintime = {10s, ..., 100s, ..., 999s}
Exact Count/Sampling Technique
Formats the collected data into samples (at the end

of each cycle)
States of a job
Stores the Set-of-Samples into the in-storage buffer

and VSAM data sets (at the end of each Mintime)
Monitor III Metrics
Data gathering techniques
Cycle/Mintime/Range
GG24-4131
Basics-5
Exact count technique
Sampling technique
GG24-4131
Basics-6
Gathering Data with RMF Monitor III

The Monitor III data gatherer collects system data, formats it into samples and
stores the set-of-samples into the in-storage buffer (wrap-around storage). The
time after which RMF Monitor III takes one sample from the system (snap shot)
is called the cycle time. The default of 1000ms (1 sec) can be modified with data
gatherer options to any value between 50 and 9999ms. The time that RMF
Monitor III uses to combine all gathered samples into a set-of-samples is called
the mintime. The default of 100s can be modified with data gatherer options to
any value between 10 and 999s. The mintime is the shortest time available for
reporting by a reporter session.
To understand the data, it is helpful to separate it into two categories: that which
RMF obtains from system counters, and that which RMF obtains by sampling.
The system can accurately count certain system events, such as the number of
start subchannel (SSCH) instructions to a device. RMF Monitor III collects this
data from the system counters at the beginning and the end of each mintime and
calculates the difference in the counters for each event. This technique is
referred as the exact count technique.
Data that is not available from system counts, such as the time a certain job
uses a device, is obtained by RMF by sampling at the end of each cycle. RMF
Monitor III acquires this data by checking the resources and then uses statistical
techniques to calculate the percent of time the job was found using the device
15
during the mintime. For example, if a job was found using a device ten times
within a mintime of 100 cycles, the job is reported as 10% using the device.
16

RMF Monitor III Concepts ...
Job States
Using
Processor
Device
Delayed
Processor
Device
Storage
Subsystems
Operator
Enqueue
Idle
-TCB, SRB
-Connect
-IOS queue
-Disconnect
-Pending
-Common Storage Paging
-Local Storage Paging
-Virtual I/O
-Swap in
-Swapped out and ready
-HSM
-JES
-XCF
-Mount
-Message
Terminal Wait
Timer Wait
JES Queue
(Unknown)
GG24-4131
Basics-7
Job States
At the end of each cycle, RMF examines the system resources to determine the
actual states of the jobs running in the system. When jobs receive service from
hardware resources, they are using these resources. The delay in an address
space indicates that a job is not getting the service it requires. For example, a
job that needs one or more resources must wait while others use these
resources. The delayed job is contending for the resource(s) with other users in
the system. In calculating Delay and Using values, Monitor III counts an address
space as using a resource even if the address space is also delayed for the
identical resource (single state case).
Using(%) = (Using Samples / Number of Samples) * 100
Delay(%) = (Delay Samples / Number of Samples) * 100
Because there are two resources an address space can use and six reasons
why an address space can be delayed, the individual Using percentage can add
up to 200% and the individual Delay percentage up to 600%.
A TSO address space in TSO terminal input wait, a batch address space waiting
for job selection, or any address space in a not-using, not-delayed state, with at
least one task waiting for a timer event is referred as being in Idle state.
Unknown is a state assigned to an address space when it cannot be classified as
using, delayed, or idle. The reasons for unknown states can be divided into two
17
groups. One is lack or use of non-tracked resources (for example, non-tape,

non-DASD devices), the other is lack of work (for example, a DB/DC driver
without transactions).
18

How does RMF Monitor III report data?
Common Metrics
Reporting Range
Workflow (%) for jobs (address spaces) and

resources (device and processor)
Using (%) for jobs (address spaces)
Delay (%) for jobs (address spaces)
Average Active Users using or delayed for resources
Multiple of Mintimes
Reports
GG24-4131
Job/Group oriented (DELAY, GROUP, SYSINFO, DEV,

ENQ, PROC, STOR, STORF, STORS, JOB)
Resource oriented (DEVR, ENQR, HSM, JES, XCF,
STORR, STORC, STORCR)
Workflow/Exception oriented (WFEX)
Availability of resources (WFEX)
Basics-8
Common Metrics
The workflow of a job (address space) represents the speed at which a job
moves through the system in relation to the maximum speed at which the job
could move through the system, as follows:
Workflow(%) = (Using Samples / (Using Samples + Delay Samples)) * 100
The workflow of a job is a measure of whether a job has all its resources (high
value) or has only a few of the resources it needs (low value). The workflow for
a group of jobs is calculated by summarizing the using and delay samples for all
jobs in the group.
The workflow of a resource indicates how efficiently users are being served. The
using samples are the number of all jobs using the resource (processors or
devices) and the delay samples are the number of all jobs being delayed for the
resource.
The average number of active users in a group using a resource (processor or
device) expresses the utilization of the resource in terms of users being served
by the resource. The average number of active users of a group delayed for a
resource shows the average waiting queue length of users waiting for service
(except for device, where the average active users (AAU) equals the number of
users on the IOS queue plus the users started on the queue but not connected to
the device).
19
Reporting
The reporting range of the Monitor III reporter is a multiple of the data gatherer
mintime. The default reporting range is one mintime. The reports can be
classified as job- or group-oriented reports, which show the reason for job
delays, and resource-oriented reports, which show the resource contention. In
addition, a Workflow/Exception report shows the workflow for specified groups of
jobs or resources, exceptions based on the specified exception criteria, and
exceptions if critical resources are no longer available.
20

Example: Data Gathering and Reporting
GG24-4131
Basics-9

This example summarizes the basic concepts of RMF Monitor III data gathering
and reporting. It is a simplified example and assumes a system with one
resource (RES) and four jobs (A, B, C, and D). The jobs may either use the
resource, wait for the resource, or be in an idle state, which means that they
have no work.
RMF takes a snapshot every second (cycle time), formats the current situation
internally into a sample and ends the reporting interval (end of mintime) after
four samples.
The reporter sessions obtain this set-of-samples and create either a job-oriented
or resource-oriented view of the gathered situation.
The job-oriented view lists all jobs sorted in descending order by their delay
percentage and calculates for all jobs the percent of the reporting time range
that RMF Monitor III found the job delayed (DLY%) and using (USG%) the
resource, the time the job was in idle (Idle%) state, and the workflow of the job
(WFL%). In addition, if the job has been delayed, the job that was found using
most often is listed as the Primary Reason for the delay. For job A this means:
The job was found 3x waiting for RES => DLY% = (3/4) x 100 = 75%
The job was found 1x using RES => USG% = (1/4) x 100 = 25%
21
The workflow (WFL%) = Using Samples / (Using + Delay) Samples = (1/4)

x 100 = 25%
Job C was found 2x and job B 1x using RES while job A was delayed for RES
= > J o b C i s t h e Primary Reason for the delay of job A.
The resource-oriented view provides the workflow for the resource, the average
number of users delayed for the resource, and a list of jobs that were delayed
for the resource, sorted in descending order by their delay percentage.
The average number of users delayed for RES (ADU) = The total number of
jobs found waiting for RES divided by the number of samples for the Mintime
= 6/4 = 1.5 .
WFL% = Number of times jobs were found using RES divided by the sum of
jobs were found using and delayed for RES during the Mintime = (4 / (4 +
6)) * 100 = 40% .
22

RMF Monitor III Primary Menu
ERB3PRM

Enter selection number or command on Selection line.
Usage of Monitor III
Basic Reports
Detail Reports
1 WFEX
7 DELAY
Delays (DLY)
8 DEV
Device delays (DD)
3 GROUP
9 DEVR
Group => DMN001__
10 ENQ
ENQ delays (ED)
4 GROUPS
Group reports selection
11 ENQR
ENQ resource (ER)
5 JOB
Jobs primary delay
12 PROC
Job
=> ROBERTV_
6 JOBS
14 SUBS
Startup
Tabular/Graphics Reports
DI DSINDEX
OP OPTIONS
Other Selections
Dataset index
Option selection
X EXIT
End display session
Report Options/Commands
Licensed Materials - Property of IBM
Sequence to use RMF Monitor III Reports

Cursor Sensitivity
Selection ===> _
GG24-4131
Basics-10
GG24-4131
Basics-11
Starting Monitor III Reporter

RMF Monitor III reporter can be started from a TSO/E session or ISPF session by
just typing RMF, or RMFWDM, in the TSO Ready line (or TSO RMF (RMFWDM) in
the ISPF command line). The RMF Monitor III reporter starts with its Primary
Menu. The menu provides, in the upper half, an overview of the available
reports that can be selected and, in the lower half, additional selections, such as
the Data Set Index screen, a tutorial, and selection menus to customize the
reporter session.
The reports are categorized in two groups: basic reports such as the WFEX or
SYSINFO reports are intended as entry points in monitoring the system, and
reports that provide comprehensive information about single jobs or group of
jobs (domains or performance groups). Detail reports allow you to determine
possible resource contentions from a job, group, or resource view. Some
selections, such as STORAGE, SUBS, GROUPS, and JOBS lead to an additional
menu that offers further report capability.
A report can be selected by entering the selector, the command, or an
abbreviation of the command. The commands are available from all menus and
report panels and allow fast-path selection of the associated report.
For example, the Workflow/Exception Report can be selected by entering 1 on
the Primary Menu or by entering the command WFEX, or its abbreviation WF
from any menu or report panel.
23

RMF Monitor III Tabular Reports
RMF 4.3.0
Samples:
100
System Information -- AQTS

Time: 03/31/93 at 15.15.00

RMF Monitor III Graphic Reports
Line 1 of 30
Range +
100 Sec
RMF 4.3.0
Samples:
100

Time: 03/31/93 at 15.15.00
Line 1 of 30
Range +
100 Sec
--------------------- 9021 Version A6 ----------------------- IPS = IEAIPS00

Processor(s) Online:
6
Vector Processors: 0
OPT = IEAOPT00
Average CPU Util:
74%
Average TCB+SRB:
66%
ICS = IEAICS00
Name
WFL --Users-- RESP TRANS VEC -AVG USG%

TOT ACT Time /SEC UTL PROC DEV
*SYSTEM
*TSO
*BATCH
*STC
*ASCH
*OMVS
DMN000
DMN001
DMN002
DMN003
DMN004
DMN005
DMN006
DMN007
DMN009
DMN012
PG000
PG001
PG002
PG003
Command
50 576
14
11.76
0 4.7 3.1
1.6 3.0 0.0 1.3 0.7 1.0
54 478
8
11.76
0 2.8 1.7
0.6 2.0 0.0 1.2 0.0 0.0
45
7
4
0.00
0 1.4 0.7
0.9 0.7 0.0 0.0 0.0 1.0
49
91
2
0.00
0 0.5 0.7
0.0 0.3 0.0 0.1 0.7 0.0
0
0
0.00
0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
0
0
0.00
0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
70
39
1 .000 0.00
0 0.2 0.3
0.0 0.2 0.0 0.0 0.0 0.0
30
2
2 15.2 0.08
0 0.2 0.5
0.1 0.6 0.0 0.0 0.0 1.0
59
5
2 12.6 0.04
0 1.2 0.2
0.8 0.1 0.0 0.0 0.0 0.0
90 465
0 .033 11.75
0 0.2 0.1
0.0 0.0 0.0 0.0 0.0 0.0
31
1
0 .178 3.50
0 0.1 0.1
0.0 0.1 0.0 0.2 0.0 0.0
41
3
2 .493 1.91
0 0.6 0.6
0.1 0.5 0.0 1.0 0.0 0.0
60
5
4 6.43 0.64
0 2.0 0.9
0.5 1.4 0.0 0.0 0.0 0.0
39
53
1 .000 0.00
0 0.3 0.4
0.0 0.1 0.0 0.1 0.7 0.0
67
2
0 .000 0.00
0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
1
0 .000 0.00
0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
70
39
1 .000 0.00
0 0.2 0.3
0.0 0.2 0.0 0.0 0.0 0.0
45
7
4 21.5 0.08
0 1.4 0.7
0.9 0.7 0.0 0.0 0.0 1.0
54 474
8 .516 11.75
0 2.8 1.7
0.6 2.0 0.0 1.2 0.0 0.0
63
39
0 .000 0.00
0 0.1 0.1
0.0 0.0 0.0 0.0 0.1 0.0
===> ________________________________________________ Scroll ===> CSR
GG24-4131
-Average Number Delayed For PROC DEV STOR SUBS OPER ENQ
Basics-12
Command ===> ________________________________________________ Scroll ===> CSR
GG24-4131
Basics-13
System Information Report

The System Information (SYSINFO) report provides an overview of workload and
delay on the measured system in either tabular or graphic form. It summarizes
workflow and delay information by users, domain, and performance group. This
report also lets you examine how effective the MVS system controls are for an
installation.
The SYSINFO report will be used to demonstrate some general reporting
functions of RMF Monitor III throughout the next foils.
Tabular Reports
Tabular reports provide detailed information about jobs and resources. They
exist for all type of reports. For the System Information report, the tabular report
has two sections. The upper section provides you with an overview of the
system. The bottom section summarizes information for each domain and
performance group for the measured system.
24
Graphic Reports
Graphic reports provide quick information about potential problem areas. For
the SYSINFO report, the graphic form of the report shows the average number of
active users for each type of delay and using category.
You can switch between graphic and tabular versions of a report by entering the
TOGGLE command or pressing the Toggle key (PF6).
Graphic report versions exist for all reports except DSINDEX, Common Storage
(STORC), Common Storage Remaining (STORCR), and Group reports.
25

RMF Monitor III Report Options

RMF Monitor III Report Options ...
RMF SYSINFO Report Options
Line 1 of 5
Select (S), exclude (X) or fill-in groups for the SYSINFO report. Press END.
Selections made here also affect the Storage Summary (STORS) report.
Date ===> 03/31/93
Time ===> 15.15.00
Summary ===> NO
RPG data ===> NO
Sel
_
_
_
_
_
_
_
Group
______
______
*ALL
DMN003
DMN012
PG005
PG020
Sel
_
_
S
_
_
_
_
Group
______
______
*DMN
DMN004
PG000
PG008
PG021
Range ===> 100
Summary lines on SYSINFO report (YES NO)

RPG data lines on SYSINFO report (YES NO)
Sel
_
_
_
_
_
_
_
Group
______
______
*PG
DMN005
PG001
PG013
PG022
Sel
_
_
_
_
_
_
_
Group
______
______
DMN000
DMN006
PG002
PG014
Sel
_
_
_
_
_
_
_
Group
______
______
DMN001
DMN007
PG003
PG017
Sel
_
_
_
_
_
_
_
Group
______
______
DMN002
DMN009
PG004
PG019
Basics-14

Time: 03/31/93 at 15.15.00
Line 1 of 30
Range +
100 Sec

6
OPT = IEAOPT00
Average CPU Util:
74%
Average TCB+SRB:
66%
ICS = IEAICS00
Name
GG24-4131
RMF 4.3.0
Samples:
100
DMN000
DMN001
DMN002
DMN003
DMN004
DMN005
DMN006
DMN007
DMN009
DMN012

70
39
30
2
59
5
90 465
31
1
41
3
60
5
39
53
67
2
1
1 .000 0.00
2 15.2 0.08
2 12.6 0.04
0 .033 11.75
0 .178 3.50
2 .493 1.91
4 6.43 0.64
1 .000 0.00
0 .000 0.00
0 .000 0.00
0
0
0
0
0
0
0
0
0
0
0.2 0.3
0.2 0.5
1.2 0.2
0.2 0.1
0.1 0.1
0.6 0.6
2.0 0.9
0.3 0.4
0.0 0.0
0.0 0.0
0.0
0.1
0.8
0.0
0.0
0.1
0.5
0.0
0.0
0.0
0.2
0.6
0.1
0.0
0.1
0.5
1.4
0.1
0.0
0.0
0.0 0.0 0.0 0.0

0.0 0.0 0.0 1.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.2 0.0 0.0
0.0 1.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.1 0.7 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
Changed option(s) now in effect.

GG24-4131
Basics-15
Report Options
All reports are provided with a set of report options. You control the scope of
the reported data by altering these options.
To display the Report Options panel for a report, enter ROPTIONS (or just RO)
on the command line of the report. Some reports have common report options;
for example, all reports that show delays for single jobs (DELAY, DEV, ENQ,
HSM, JES, PROC, STOR, STORC, STORF, and XCF) have the same report option
panel. The header of the Report Options panel displays the reports that are
affected by a report option change. For the SYSINFO report, the storage
summary report (STORS) is also affected.
All Report Options panels allow you to change the date, time, and range for the
reporting interval. The Report Options panel for the SYSINFO report allows you
to choose whether the summary lines (*SYSTEM, *TSO, ...), report performance
groups, and whether all or only a subset of groups should be included in the
report. By altering the Summary to NO and changing the Groups from *ALL to
*DMN all domains, no performance groups and no summary lines will be
included in future invocations of the SYSINFO report.
26
Commands
You can also control the scope of the reported data with commands; for
example, the command SY DMN provides a system information report that
includes domains and summary lines. The command does not change the
options specified on the option panel.
To change the date, time, and range of the reporting interval, the commands
FREF to go forward and BREF to go backward are available.
27

Using RMF Monitor III Reports
GG24-4131

Cursor Sensitivity
Basics-16
GG24-4131
Basics-17
Suggested Sequence for Using RMF Monitor III Reports

When you use a Monitor III reporter session to monitor system activity, it is
helpful to start with the Workflow/Exception (WFEX) report. The WFEX report
identifies jobs and resources with low workflow values or jobs that have met
exceptional conditions.
You can start with the DSINDEX screen to get an overall picture of the available
data for this Monitor III reporter session. The DSINDEX report shows the
date/time range of all available data, all data sets used during data set
recording, all data contained in the in-storage buffer during real-time monitoring,
or all pre-allocated data sets for the display session.
For a summary of how workflow and delay of the measured system affect
performance, the SYSINFO report is the best starting point.
The DELAY report is the right starting point if you are interested in jobs with the
most delays in the system.
28
Cursor Sensitivity
Cursor sensitivity control lets you place the cursor on a field in a tabular report,
press the Enter key, and see another report containing additional information
about the field. This is a significant function in analyzing system performance
and determining delay reasons. RMF always keeps track of your path. Pressing
the END (PF3) key returns you to the previous report. If you issue any command
while using cursor sensitivity control, RMF will erase the return path and the
END (PF3) command returns you to the Primary Menu.
29
30
Demonstration-1
Analyzing System Workloads
Analyze System Workloads

Determine Components of Group Response Time
Determine Delay Reasons
Reports
SYSINFO
GROUP
PROC
JOB
GG24-4131
D e m o n s t r a t i o n 1-1
Objective
The first demonstration shows how RMF Monitor III can be used to analyze the
system workloads, to identify the components of transaction response time, and
to determine the reason for delays. The reports used throughout this
demonstration are the System Information report, the Group Response Time
report, the Processor Delay report, and the Job report.
31
Demo
At this point you must have on your screen the RMF Monitor III Primary menu
as explained in step 6 on page 4. This box, at the beginning of each new
description, tells you which commands you have to use to get the same
panel as presented in the book.
Setup: To ensure the right option settings for the demonstration invoke the
Session Option panel and verify or modify the following entries:
SESSION
To invoke the Session Option panel
Mode
Stop
Range
100S
Report Format
Graphic
To return to the RMF Monitor III Primary Menu

Demonstration 1
GG24-4131
Analysis Sequence
By starting with the System Information report, we will get an overall picture of
the system, its current processor utilization, its overall contention situation, and
information about the groups affected by delays. The next step is to examine
more closely the groups for which we see problems on the System Information
report. This is done with the Group Response Time report, which gives a
32
detailed picture of the components of the transaction response time. To identify

the real contention situation, we will finish our analysis with the Processor Delay
and Job reports.
33
Demo
SYSINFO
To invoke the SYSINFO report
BREF DATE=03/31/93,TIME=15.10
this demonstration
To obtain the reporting range for
to change from the graphic report layout to the tabular

report layout during the session

Demonstration 1 ...
RMF 4.3.0
Samples:
100

Time: 03/31/93 at 15.10.00

Demonstration 1 ...
RMF 4.3.0 System Information -- AQTS

Samples:
100
Time: 03/31/93 at 15.10.00
Line 1 of 30
Range +
100 Sec
Line 1 of 30
Range +
100 Sec

6
OPT = IEAOPT00
Average CPU Util:
94%
Average TCB+SRB:
85%
ICS = IEAICS00
GG24-4131
Name

*SYSTEM
*TSO
*BATCH
*STC
*ASCH
*OMVS
DMN000
DMN001
DMN002
DMN003
DMN004
DMN005
DMN006
DMN007
DMN009
DMN012
PG000
PG001
PG002
PG003
Command
49 577
66 479
30
11
70
87
0
0
78
35
23
2
32
9
73 467
40
1
68
2
68
5
67
53
100
2
1
78
35
30
11
66 474
49
39
===>
GG24-4131
17
6
8
2
0
0
1
2
6
0
1
1
4
1
0
0
1
8
6
1
.000
27.8
37.7
.033
.121
1.45
11.1
.000
.000
.000
.000
58.7
1.06
.000
14.25
14.25
0.00
0.00
0.00
0.00
0.00
0.11
0.09
14.24
4.28
2.45
0.95
0.00
0.00
0.00
0.00
0.11
14.24
0.00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.9
3.3
2.0
0.6
0.0
0.0
0.2
0.2
1.8
0.2
0.1
0.6
2.3
0.4
0.0
0.0
0.2
2.0
3.2
0.2
3.0
1.5
0.6
1.0
0.0
0.0
0.3
0.2
0.4
0.1
0.1
0.3
1.0
0.7
0.0
0.0
0.3
0.6
1.5
0.1
5.3
0.9
4.3
0.1
0.0
0.0
0.0
0.3
4.1
0.1
0.0
0.1
0.6
0.1
0.0
0.0
0.0
4.3
0.9
0.0
2.4
1.3
0.8
0.3
0.0
0.0
0.2
0.1
0.7
0.1
0.1
0.3
0.9
0.1
0.0
0.0
0.2
0.8
1.3
0.1
0.0 0.3
0.0 0.2
0.0 0.0
0.0 0.1
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.2
0.0 0.0
0.0 0.0
0.0 0.1
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.2
0.0 0.0
Scroll
0.2
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.2
===>
1.1
0.0
1.0
0.0
0.0
0.0
0.0
1.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1.0
0.0
0.0
CSR
System Information Report

The SYSINFO report provides an overall picture of the workloads in the system.
The tabular SYSINFO report has two sections; the top section provides you with
an overview of the system and identifies the measured system, the number of
processors and vector processors on-line, the parmlib members used for the
installation performance specification (IPS), the option member (OPT), and the
installation control specification (ICS). The top section also includes information
about processor usage during the report interval. The bottom section
summarizes information about the total system, job classes ( *SYSTEM, *TSO,
*BATCH, *STC, or *ASCH), domains ( DMN), performance groups ( PG), and report
performance groups ( RPG).
The SYSINFO report shows the average response time for all transactions that
ended during the range. The response time value is the sum of the queued time
and the active time for an average ended transaction. If the RESP Time field is
34
shown in dark blue, the data reported can be statistically insignificant. The field
will always be shown in dark blue if less than 100 transactions ended during the
range, or the average response time is longer than the range value.
The Group report provides more detailed information on whether the response
time is significant. The proportion of the active users in each using or delay
category indicates the proportion of the average response time that is spent in
that category. The graphic form of this report shows the average number of
active users for each type of using and delay.
Example
Starting with the graphic version of the report, we see about 17 users active on
average for the overall system and we can identify the processor as the major
delay and using component. By examine the groups, we find that obviously
BATCH groups (DMN002 and PG001) are mainly delayed for the processor while
the processor using is distributed between BATCH and TSO groups.
The tabular version completes the overall picture. In the top section we see that
the processors are heavily utilized (94%). We will now examine response time
for the BATCH performance group (DMN002 and PG001) and will continue with
the Group Response Time report.
35
Demo
Place the cursor on
column RESP Time and
line PG001 and press
...
Name
...
PG000
PG001
PG002
...

78
35
1 .000 0.00
0 0.2 0.3
0.0 0.2 0.0 0.0 0.0 0.0
30
11
66 474
8 58.7 0.11
6 1.06 14.24
0 2.0 0.6
0 3.2 1.5
4.3 0.8 0.0 0.0 0.0

0.9 1.3 0.0 0.2 0.0 0.0
to invoke the
Group Response Time
report for the batch
performance group.

Demonstration 1 ...
RMF 4.3.0 Group Response Time -- AQTS

Samples:
100
Time: 03/31/93 at 15.10.00
Range +
Group: PG001
Description:
Primary Response Time Component:
WFL
%
30
Users
Frames
TOT ACT
%ACT
11
8
1
Average Users
Response Time ACT
Average Users
Response Time ACT
Statistical error:
Total
8.200
49.65
Vector
UTIL
0
100 Sec
Batch classes
Processor delay
EXCP
Rate
3,714
-AVG USGPROC DEV

2.04 0.58
12.3 3.50
---STOR Delay--Page Swap OUTR

0.00 0.00 0.00
0.00 0.00 0.00
PgIn
Rate
0.0
TRANS
Ended
Rate
0.1
--- Response Time ----- Ended TRANS-(Sec) Queued Active

Total
9.000
49.65
58.65
-------------Average Delay-------------PROC
DEV STOR SUBS OPER
ENQ OTHER
4.33 0.78 0.00 0.00 0.00 1.04 0.00
26.1 4.71 0.00 0.00 0.00 6.27 0.00
---OUTR Swap Reason--TI
TO
LW
XS
0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
---SUBS Delay--JES
HSM
XCF
0.00 0.00 0.00
0.00 0.00 0.00
33%
Command ===>
GG24-4131
Group Response Time Report

The Group Response Time report, which is new for RMF Version 4 Release 3.0,
provides SRM response time in seconds and a breakdown of where the
components of response time were spent. Most of the data in this report is
sampled data representing all transactions that went through the requested
performance group (PG) or domain. The three new items collected from SRM,
TRANS Ended Rate, and the Queued and Active times for Ended TRANSactions,
represent transactions that ended in the requested domain or PG. The sum of
queued time plus active time gives the total time the average transaction for this
group took at the host.
The calculations used in this report make assumptions that must hold true if the
report is reasonably accurate. The most important assumption is that the SRM
data for transactions ended during the interval is representative of what
36
happened during the range. This will usually be true if the response time for the
transactions is short compared to the range, and the number of transactions that
ended is large enough to be a statistically significant sample. Since the
assumptions will not always be applicable, a sanity check is made and warning
information is provided. The check used is to compute Average Active Users for
the group in two ways, one based on Monitor III sampled data and the other on
SRM measured data. If the smaller of the two differs from the larger by more
than 25%, a statistical error field is displayed.
Example
The example shows a low TRANS Ended Rate (0.1) and a long active response time
(49.65 seconds), which explains that the data is not very accurate (statistical
e r r o r = 3 3 % ) . The decomposition of the response time, relative to the number
of Average Active Users, gives information on how long the average transaction
spent in each category. We find that the users wait most of the time for the
processors (26.1 seconds) which is also described in the header as Primary
Response Time Component. We will continue with the Processor Delay report to
analyze the reason for the delay.
37
Demo
...
Average Users
Response Time ACT
...
Total
8.200
-AVG USGPROC DEV

2.04 0.58
-------------Average Delay-------------PROC
DEV STOR SUBS OPER
ENQ OTHER
4.33 0.78 0.00 0.00 0.00 1.04 0.00
49.65
12.3 3.50
26.1
4.71 0.00
0.00
0.00
Place the cursor on

column Average Delay
PROC and press

to invoke the Processor
Delay report in order to
determine the reason
for the delay.
6.27

Demonstration 1 ...
Samples:
Jobname
C DMN
IARHI
SCHIFFU
#SIMMONS
IARHD
IARHC
RBAKERAC
SCHUMAC1
IARHA
IARHT
TCHAND
UY81623
WASINATS
ALAN1@
##903010
##903007
KIRKIEC7
KIRKIEC6
##903011
WASINPM
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
2
2
2
2
2
***
***
2
***
***
***
***
***
***
1
2
2
***
1
PG
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
RMF 4.3.0 Processor Delays -- AQTS

100
Time: 03/31/93 at 15.10.00
DLY USG TCB+ VEC

----------- Holding Job(s) ----------%
%
SRB% Ratio % Name
% Name
% Name
63
58
53
51
41
35
29
25
23
12
11
8
7
4
4
3
3
2
1
23
30
47
16
17
14
12
12
5
3
5
5
1
4
3
1
1
5
0
25.2
30.6
52.1
19.0
15.3
11.2
11.6
9.9
9.0
3.3
6.5
4.7
1.9
2.5
1.1
1.0
1.0
2.9
0.0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
56
54
46
45
36
28
25
20
20
10
10
7
5
3
4
3
3
2
1
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
IARHT
Report is for performance group 1 only.

Command ===>
GG24-4131
Line 1 of 19
Range +
100 Sec
23
22
16
18
15
15
12
13
11
6
8
5
5
3
2
2
2
1
1
#SIMMONS
#SIMMONS
MINYANG
JEAHA
MINYANG
MINYANG
SCHIFFU
JEAHA
#SIMMONS
RHOWARD
MINYANG
ANN
ANN
CLIANG
KIM1
JEAHA
KIM1
RBAKERAC
SCHIFFU
22
18
16
17
14
15
12
9
9
5
7
3
3
3
2
2
1
1
1
MINYANG
MINYANG
SCHIFFU
MINYANG
#SIMMONS
SCHIFFU
MINYANG
CLIANG
ANN
WEISS
#SIMMONS
D10MRM1
JES2
H65JHF1
RHOWARD
D10MRM1
VTAM
H65JHF1
ANN
Scroll ===> CSR
Processor Delay Report

The Processor Delay report (PROC) displays all jobs that were waiting for or
using the processors during the range. RMF reports the jobs by descending
overall delay percentages. Because use of the processor by many jobs might
contribute to the delay of other jobs, RMF reports up to three jobs in the Holding
Job(s) field. The jobs in this field are those that were most often found using the
processor while the job was delayed.
38
Example
In our example, the report lists only jobs delayed for the processors for
performance group one, because RMF Monitor III maintains the scope of the
reported data. By examine the list of holding jobs we can easily identify the job
ROBK as the primary reason for the processor delay. We can now examine the
holding jobs or the delayed jobs by calling the job reports. To examine the
probable causes of the delay we will continue with the Job report for JOB IARHI.
39
Demo
...
Jobname
C DMN
IARHI
PG
DLY USG TCB+ VEC

----------- Holding Job(s) ----------%
%
SRB% Ratio % Name
% Name
% Name
63
23
25.2
56 ROBK
23 #SIMMONS
22 MI
...
Place the cursor on

Jobname IARHI and
press
to
invoke the Job report
for Job IARHI to
determine the probable
causes for the delay.

Demonstration 1 ...
Samples:
Job: IARHI
RMF 4.3.0 Job Delays -- AQTS

100
Time: 03/31/93 at 15.10.00
Line 1 of 1
Range +
100 Sec
Primary delay: Job is waiting to use the processor.
Probable causes: 1) Job ROBK may be looping.

2) Higher priority work is using the system.
3) Improperly tuned dispatching priorities.
------------------------- Jobs Holding the Processor -------------------------Job:

ROBK
Job:
#SIMMONS
Job:
MINYANG
Holding:
56%
Holding:
23%
Holding:
22%
PROC Using:
88%
PROC Using:
47%
PROC Using:
34%
DEV Using:
0%
DEV Using:
0%
DEV Using:
1%
--------------------------- Job Performance Summary --------------------------WFL -Using%- DLY IDL UKN ------- % Delayed for ------- Primary
C ASID DMN PG % PROC DEV %
%
% PROC DEV STOR SUBS OPER ENQ Reason
B 0579
2
1 31
23
7 65
0
6
63
4
0
0
0
0 ROBK
Command ===>
GG24-4131
Scroll ===> CSR
Job Report
The job report has four sections. On the top it lists the jobname and primary
delay component, the second section provides a list of probable causes for the
delay, the third section provides data for the resource discovered with contention
and the bottom section provides a job summary.
Example
In our example, we will focus on the list of probable causes and how they are
derived from the data listed in the third section. The first provided cause: Job
ROBK may be looping has been assumed because the sum of PROC Using and PROC
Delay equals 100 percent and ROBK does not use any other resource (not shown
on this screen: you have to go to the ROBK Job report). In addition, the Proc
Using and Dev Using values are depicted in red to provide a reference between
the probable cause for the delay and the way it is calculated.
40
The Help panel for the job report always provides a more comprehensive
explanation for the causes and an extended list of additional possibilities.
Enter HELP in the command line or press
select 2 to obtain the delay explanation.
to invoke the help facility and
Further Investigations
When we look at the Job report for ROBK we see that ROBK is a TSO user. On
the Sysinfo report, we saw that the BATCH groups suffer the most processor
delays. This is due to the setup of the dispatching priorities for BATCH and TSO
(see IPS member in Chapter 6, Installation Performance Specification on
page 81).
41
42
Demonstration-2
Monitoring System Perfomance
Monitor System Performance

Customize Exceptions Criteria Sets
Identify Jobs and Resources with exceptional
conditions
Reports
WFEX
STORC/STORCR
DELAY (Device/Enqueue)
GG24-4131
Objective
The second demonstration shows how RMF Monitor III can be used in monitoring
MVS system performance and identifying exceptional system conditions. For this
reason, we have to see how RMF Monitor III sets up default exceptions and how
the exception criteria can be customized. In the second half of the
demonstration we will use the RMF Monitor III to visually analyze system
performance degradations. The reports used throughout the demonstration are
the Workflow/Exception report and its report option panels, the Common Storage
report, and the Device and Enqueue Delay report.
43
Demo
At this point, you must have on your screen the RMF Monitor III Primary
Menu This box, at the beginning of each new description, tells you which
commands you have to use to get the same panel as presented in the book.
Setup: To ensure the right option settings for the demo, invoke the Session
Option panel and verify or modify the following entries:
SESSION
To invoke the Session Option panel
Mode
Stop
Range
100S
Report Format
Graphic
Customization
Yes
Input Data Set
Specify the data set that contains the parmlib member

for the demonstration
To return to the RMF Monitor III Primary Menu
44
Demo
WFEX
To invoke the Workflow/Exceptions report
BREF DATE=03/30/93,TIME=14.20
this demonstration
To obtain the reporting range for
To change from the graphic report layout to the tabular

report layout during the session

Demonstration 2
RMF 4.3.0
Samples:
100
Workflow/Exceptions -- AQTS
Time: 03/30/93 at 14.20.00
Line 1 of 23
Range +
100 Sec

Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
10
51
STCs
53
2
42
ALL TSO
520
5
57
TSO short
510
0
81
ALL BATCH
3
3
42
TSO longer
3
2
63
ALL STC
97
3
49
TSO medium
0
0
67
ALL ASCH
Not avail
TSO long
3
2
44
ALL OMVS
Not avail
Btch short
2
2
37
*DMN000
45
1
85
Btch long
1
1
48
------------------------------ Exceptions ------------------------------------Name
Reason
Critical val. Possible cause or action
D11DFF1T
SUBS-HSM
52.0 % delay Dataset recall from auxiliary storage.
JWHEEL
SUBS-HSM
CONSOLE
OPER-Message
43.0 % delay Awaiting reply to operator request 90.
CORVUS
DEV -HSML15
82.0 % delay Short block size or excessive seeks.
DRV021
Not avail
Volume DRV021 is not mounted.
D54ECA
Not avail
Volume D54ECA is not mounted.
D54ECB
Not avail
Volume D54ECB is not mounted.
D54ECC
Not avail
Volume D54ECC is not mounted.
D54ECD
Not avail
Volume D54ECD is not mounted.
D54ECE
Not avail
Volume D54ECE is not mounted.
D54ECF
Not avail
Volume D54ECF is not mounted.
D54EC0
Not avail
Volume D54EC0 is not mounted.
D54EC1
Not avail
D54EC2
Not avail
D54EC3
Not avail
D54EC4
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
Workflow/Exceptions Report
The Workflow/Exceptions (WFEX) report presents information about system
activity and user defined exceptions. The report has two parts; on the top, the
Speed section, RMF reports the workflow of jobs and resources as speed. In
the bottom, Exceptions section, RMF lists jobs, job groups, or system resources
that meet the users exception criteria.
The workflow and exception lines are color coded according to severity. Usually,
red indicates a problem, yellow indicates caution, and turquoise indicates that a
job is not started or a volume is missing from the system configuration. You can
specify exception criteria on the Workflow/Exceptions report options panels, or
you can use automatic customization.
Note: In the top part of the report, you may or not see the Name ALL OMVS
depending on the installation of the APAR OY57005 on your system. APAR
45
OY57005 is the support by RMF of the Open function, that means the OMVS
address spaces. If this AS is not shown in your system, another user, TSO extra
is shown after the Btch long still in the top part of the report.
46
Demo
RO
To invoke the report options panel for the WFEX report

Demonstration 2 ...

Demonstration 2 ...
RMF WFEX Report Options: Action Panel

Date ===> 03/30/93
Time ===> 14.20.00
Line 1 of 37
Range ===> 100
Enter Action Code in the Action Column. To exit press END.

Action Codes: Select
(S)
Copy
(C)
Move
(M)
Add
(AD)
Delete (D)
Move Block (MM)
Before
After
(B)
(A)
Action
Class
Qualifier Indicator Label
Row Position
__
Only Add (AD) and After (A) are valid on this line.
__
SYSTEM
WF
*SYSTEM
1
1
__
TSO
WF
ALL TSO
1
2
__
BATCH
ALL
WF
ALL BATCH
1
3
__
STC
WF
ALL STC
1
4
__
ASCH
WF
ALL ASCH
1
5
__
OMVS
WF
ALL OMVS
1
6
__
DMN
0
WF
*DMN000
1
7
__
DMN
7
WF
STCs
2
1
__
DMN
3
WF
TSO short
2
2
__
DMN
6
WF
TSO longer
2
3
__
DMN
4
WF
TSO medium
2
4
__
DMN
5
WF
TSO long
2
5
__
DMN
1
WF
Btch short
2
6
__
DMN
2
WF
Btch long
2
7
__
PROC
EX-AVG
*PROC
__
STOR
EX-AVG
*STOR
__
STOR
EX-AVG
*STOR
__
STOR
EX-AVG
*STOR
__
STOR
EX-AVG
*STOR
__
STOR
EX-AVG
__
JOB
EX-UNAVAIL
GG24-4131
Class
Indicator
Setting for Yellow
Setting for Red
(1)
WF
WFL%<20 & ADU>2
WFL%<12 & ADU>2
(2)
WF
WFL%<30 & ADU>.2
WFL%<20 & ADU>.1
PROC
EX-AVG
STOR
EX-AVG
SQA%>5
STOR
EX-AVG
CSA%>2
STOR
EX-AVG
O N L F < (3)
STOR
EX-AVG
O N L X F < (4)
STOR
EX-AVG
STOR
EX-AVG
SCSA%>70
SCSA%>85
STOR
EX-AVG
SECS%>70
SECS%>85
J O B (5)
EX-UNAVAIL
Job not found
DEV (5)
EX-UNAVAIL
Device not found
DMN_w/_1
EX-ANY
WFL%<30 &DLY%>5
WFL%<20 & DLY%>5
DMN_w/_2
EX-ANY
WFL%<20 &DLY%>5
WFL%<10 & DLY%>5
DMN_1
EX-ANY
WFL%<30 &DLY%>5
DMN_2
EX-ANY
TSQAO>0
WFL%<10 &DLY%>5
WFL%<10 & DLY%>5
(1)
SYSTEM, BATCH, STC
(2)
TSO, ASCH, OMVS, DMNs
(3)
average number of frames online
(4)
average number of expanded frames online
(5)
line is displayed in turquoise
DMN_w/_1
Domain with DUR parameter in first half of list
DMN_w/_2
DMN_1
DMN_2
AVAIL<#_of_cpus
GG24-4131
- - in second half of list

Domain without DUR parameter (1st period only)
- - (all other periods)
Automatic WFEX customization tailors the WFEX report. Automatic customization
can be selected by specifying YES for Customization on the Session Options
panel. With automatic customization, RMF reads the Installation Performance
Specifications (IPS) that were in effect when the WFEX data was collected. RMF
then sets up the workflow and exception indicators related to the installation s
specifications for workload. The workflow indicators will always be SYSTEM,
TSO, BATCH, STC, ASCH, and OMVS (when available in your system), followed
by the eight most important domains as determined by the following sort order:
1. Number of m ax i mu m users (descending) (from CNSTR value)
2. Number of m i n i m u m users (descending)
3. DUR value (ascending)
4. Relative position in IPS
The labels for the workflow indicators for the domains are derived from the first
comment found in the IPS.
Exception criteria are created for every domain found in the IPS. If the domains
0 and 1 exist they are handled as all other domains; otherwise, RMF will create
47
exception criteria for them. The exceptions are of type ANY, which displays
exceptions when individual jobs meet the criteria. The result is that every job in
the system has one or more exception criteria associated with it. The exception
criteria will be determined by the DUR value of the domain specification as
shown in foil Demonstration 2-4.
The automatic customization uses the current data to set up exceptions for
applications and DASD volumes (type: UNAVAIL). The job exception checks for
the presence of all STCs found in the data and the DEV exception for the
presence of all DASD volumes found in the data.
Lastly, RMF sets up exceptions for classes PROC and STOR to identify resource
problems of the processor configuration and the available storage. In addition
an exception will always be shown whenever a SLIP trap has been established
on the system.
48
Demo
Please follow the steps described under Example in order to specify and
modify the exception criteria.

Demonstration 2 ...

Demonstration 2 ...
RMF WFEX Report Options: Definition and Criteria
RMF WFEX Criteria Names for Class: DMN
Enter or edit information below. To view a list of criteria name values,

place the cursor in a blank Name field and press ENTER.
Exception will be displayed if all criteria in a set 1, set 2 or set 3 are met
Class
Qualifier
Indicator
Label
Alert
Text
===> DMN___
For example: SYSTEM, BATCH, JOB, DEV, STC
===> 1_______
For example: Jobname, volume serial, job class
===> EX-AVG____
WF, EX-ANY, EX-AVG, EX-GROUP or EX-UNAVAIL
===> __________
Label for workflow monitor or exception line
===> NONE_
Alerting signal: BLINK, BEEP, BOTH, NONE
===> ______________________________________ Leave blank for defaul
Criteria set
Name
<>
Yel
AT___ >_
25__
_____ __
____
_____ __
____
_____ __
____
_____ __
____
_____ __
____
_____ __
____
1
Criteria set
Red
Name
<>
Yel
40__
_____ __
____
____ or _____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
2
Criteria set
Red
Name
<>
Yel
____
_____ __
____
____ or _____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
____
_____ __
____
3
Red
____
____
____
____
____
____
____
Command ===> _________________________________________________________________
GG24-4131
Select (S) a maximum of 7 items to use in a criteria set. Press END.

_
_
_
_
S
_
_
_
_
_
_
_
_
_
_
_
_
_
AAU
AAUS
ADU
ASTO%
AT
AUU
AVAIL
COMM%
CPU%
CPUS%
CSA%
DEV%
DLY%
ENQ%
ESQO%
HSM%
ISTO%
JCSA%
Avg active users

Total avg active
Avg delayed users
Active storage %
ACT response time
Avg using users
No. jobs in group
Common stor dly %
TCB+SRB %
CPU utilization %
CSA storage %
Device delay %
Overall delay %
ENQ delay %
ESQA overflow %
HSM delay %
Idle storage %
Job CSA use %
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
JECS%
JES%
JESQ%
JSQA%
LOCL%
LPA%
MNT%
MSG%
ONLF
ONLXF
OPER%
OUTR%
PROC%
QT
RATE
RT
SCSA%
SECS%
Job ECSA use %

JES delay %
Job ESQA use %
Job SQA use %
Local stor dly %
LPA storage %
MOUNT delay %
MSG delay %
Online RS frames
Online EXP frames
Operator delay %
Out/ready delay %
Processor delay %
Queue time (ASCH)
Transaction rate
Response time tot
System CSA use %
System ECSA use %
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
SESQ%
SQA%
SQAO%
SSQA%
STOR%
SUBS%
SWAP%
TET
TSQAO
USG%
USGD%
USGP%
VIO%
WFL%
WFLG%
XCF%
System ESQA use

SQA storage %
SQA overflow %
System SQA use %
Storage delay %
SUBS delay %
Swap delay %
TRAN ELAP time
Tot SQA overflow
Overall using %
Device using %
Processor using
VIO storage dly
Workflow/Speed %
Group workflow %
XCF delay %
Command ===> _________________________________________________________________
GG24-4131
Specification of Exceptions
We are now able to modify exceptions and to specify additional exceptions. To
specify a new exception, we enter AD in the Action column, after which we want
to see the new exception. RMF presents the Definition and Criteria panel, which
allows us to specify the scope of the exception (class, qualifier, and indicator),
the option for a label, an alert and its description that appears under Possible
cause or action, and the ability to specify up to three criteria. RMF determines
whether the exception is true by combining all criteria in one set with a logical
AND, once the criteria sets with a logical OR. By setting the cursor under the
name column and pressing Enter, RMF presents a menu of criteria that can be
used for the specified scope of the exception.
Example
To continue with our analysis, we specify two additional exceptions and two new
workflow speedometers:
1. Average active response time of a group, as shown in the example above.
Whenever this exception shows up in the exception section of the WFEX
report, it can be used to invoke the Group Response Time report to analyze
49
the transaction response for the specified group. In our example, we specify
this exception for DMN001, which describes the first and second period for
the batch classes.
Demo
AD
In the Action column for DMN 9 (last definition on the

screen; use the scrolling key, PF8, to find the last DMN 9)
of the report option screen to create a new exception for
DMN 1
Class
DMN
Qualifier
Indicator
EX-Avg
Place cursor in Name Column of Criteria Set 1

WFEX criteria names for class DMN
S at AT
To invoke the
To select the criteria for ACT response time

To return to the WFEX Report Options: Definition and
Criteria
>
In <> column of Criteria Set 1 to specify the limits for

the ACT response time for DMN 1 above which the
exception line should show up on the WFEX screen.
25
For warning alert (yellow)
40
For problem alert (red)

To finish the definition and to return to the WFEX Report
Options: Action Panel
2. Overall SQA usage of the system, similar to the default exception for the
overall CSA usage. The easiest way is to copy the exception for the ECSA
usage and modify the criteria.
Demo
C
In the Action column for the last STOR exception criteria

(without Enter)
In the Action column of the following exception criteria in

order to copy the STOR exception
In the Action column of the new STOR exception criteria

in order to modify it
SSQA%
Overtype SECS% to specify an exception for the system

SQA usage
80
for warning alert (yellow), overtyping the specified value
90
for problem alert (red), overtyping the specified value

50
3. Because we do not have ASCH and OMVS running on the system, we will
exchange these workflow indicators with workflow indicators for devices and
processors.
Demo
Copy the WF indicators for ASCH and OMVS (when available) at the end
of the list, keeping their definitions, and discarding them from our display:
C
In the Action column for the ASCH exception
In the Action column of the DMN 2 exception criteria

(Btch long), to execute the copy command
In the Action column for the OMVS exception
In the Action column of the DMN 2 exception criteria

(BTCH long), to execute the copy command
Modify the WF indicators for the original ASCH and OMVS (when
available) to create new WF indicators for processor and devices: create
the WF indicators for processor and devices:
S
In the Action column of the ASCH exception criteria
Class
Change to DEV for the device WF indicator
Label
Change to Devices
In the Action column of the OMVS exception criteria

(when available)
Class
Change to PROC for the processors WF indicator
Label
Change to Processors
To return to the WFEX Panel
51
Demo
Return to the WFEX report (press

Demonstration 2 ...
RMF 4.3.0
Samples:
100
Time: 03/30/93 at 14.20.00

Demonstration 2 ...
Line 1 of 25
Range +
100 Sec

Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
10
51
STCs
53
2
42
ALL TSO
520
5
57
TSO short
510
0
81
ALL BATCH
3
3
42
TSO longer
3
2
63
ALL STC
97
3
49
TSO medium
0
0
67
Processors
139
4
78
TSO long
3
2
44
Devices
56
6
55
Btch short
2
2
37
*DMN000
45
1
85
Btch long
1
1
48
------------------------------ Exceptions ------------------------------------Name
Reason
D11DFF1T
SUBS-HSM
JWHEEL
SUBS-HSM
*DMN001
AT > 25
25.52 sec
Active time is 25.52 s for DMN 001.
*SQA*
SSQA% > 80
88.2 %
System SQA usage 88 %.
CONSOLE
OPER-Message
CORVUS
DEV -HSML15
DRV021
Not avail
D54ECA
Not avail
D54ECB
Not avail
D54ECC
Not avail
D54ECD
Not avail
D54ECE
Not avail
D54ECF
Not avail
D54EC5
Not avail
D54EC6
Not avail
D54EC7
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
) and verify your changes.
Demonstration 2 Analysis
Common Storage Usage
Enqueue Delays
Device Delays
GG24-4131
WFEX Report after Customization

We now return to the WFEX report. We see in the top sections two new
speedometers for devices and processors in place of the speedometers for
ASCH and OMVS. In the bottom section, we see exceptions caused by HSM data
set recall, the two new exceptions for the active response time for DMN001 and
SQA usage in the system, an exception for the CONSOLE address space waiting
on an operator reply and a minor device exception.
GO Mode
RMF allows continues system monitoring, which means that the report screens
will be refreshed automatically after each mintime. Whenever critical situations
appear, indicated either by speedometers starting to become yellow and red, or
red and yellow exception lines in the bottom section, GO mode can be
interrupted by pressing the ATTN (PA1) key and the reason for the delays can be
analyzed. In our example, we use pre-allocated data sets and the command
FREF to advance in the time axis.
52
Relation Workflow - Active Users

Workflow indicators in this system might turn red, without having a real problem.
For example, a range with only three users active, one STC waiting for an
operator reply, one BATCH job, and one TSO user delayed in case of a data set
recall, all of them 100% delayed, can cause the SYSTEM, TSO, BATCH, STC,
TSO short, and Btch short workflow indicators to be displayed in red. This
situation looks much worse than it really is, because the workflow metric reflects
the proportion of delayed work to running work and not the actual load on the
system. To avoid a misinterpretation, RMF provides the average number of
users active during the range to indicate the real impact on the system.
53
Demo
To obtain the correct
reporting range
place the cursor in the
Reason column of the
*SQA* exception line
FREF
...
JWHEEL
*SQA*
CONSOLE
...
SUBS-HSM
SSQA% > 80
88.2 %
OPER-Message
and press
To
invoke the Common
Storage Summary
report
Invoking the Common Storage Remaining Report within the example:

To find job FATH on the
common storage report
Place the cursor in the
Act column of Job
FATH and press
F FATH
...
FATH
...
T ***
582
6.7H
106
1760
44
to invoke the
Common Storage
Remaining report

Demonstration 2 ...
Samples:
RMF 4.3.0 Common Storage -- AQTS

100
Time: 03/30/93 at 14.21.40
System Information
IPL Definitions
Peak Allocation Values
Average CSA to SQA Conversion
Average Use Summary
Available at End of Range
Job Information
Name
Act C DMN
%MVS
%REMAIN
JES2
S
7
*MASTER*
S
0
CATALOG
S
0
VTAM
S
7
TSO
S
0
DFHSM
S
7
DB2LMSTR
S
7
DB2LDIST
S
7
OMIIETE
S
7
RMF43
S
9
CRYPTO
S
7
DB2LIRLM
S
7
NETVIEW
S
7
NPM
S
0
AUTONET
S
7
FATH
H T ***
Command ===>
GG24-4131
54
PG ASID
4
0
0
5
0
17
13
13
3
8
3
13
20
0
3
2

Demonstration 2 ...
29
1
35
54
72
57
66
81
69
61
55
83
58
68
17
582
---- Percent ---CSA ECSA SQA ESQA

25
0
24
76
61
0
61
39
88
88
12
63
63
37
ELAP
Time
-- Percent Used CSA ECSA SQA ESQA

5
2 53
55
2
1
0
0
6.5D
1
29
0
0
6.5D
3
7 12
5
6.5D
2
0
9
0
6.5D
1
9
0
0
6.5D
3
0
0
0
6.5D
0
0
3
0
6.5D
1
2
0
0
6.5D
0
2
0
0
6.5D
0
0
1
0
6.5D
0
0
1
1
6.5D
1
0
0
0
6.5D
1
0
0
0
6.5D
0
0
1
0
6.5D
0
1
1
0
6.5D
0
1
1
0
6.7H
0
0
0
0
Line 1 of 654
Range +
100 Sec
------- Amount -------CSA ECSA

SQA ESQA
3832K
28M 1364K
49M
940K
17M 1204K
31M
0
0
936K
17M 1203K
31M
2898K
11M 162K
18M
----CSA
197K
80928
20592
119K
65152
30652
130K
8427
30152
128
120
0
44424
44320
4864
16840
3464
106
Amount Used ----ECSA

SQA ESQA
544K 722K
27M
371K 1168 67416
8101K
48
464
2131K 158K 2299K
67208 125K 198K
2500K
0
352
2120
0
224
704 38616
503
663K
528 4528
502K
48 2744
96048 18480 14496
2952 8216 635K
424
392
224
7992
128
288
75405 11240
480
200K 7968
480
198K 9560
368
1760
440
206
Scroll ===> CSR
RMF 4.3.0 Common Storage Remaining -- AQTS

Line 1 of 109
Samples:
100
Time: 03/30/93 at 14.21.40
Range +
100 Sec
Jobname
ID
FATH
TSU15833
TNWILLI TSU09636
LALITHA TSU26463
LALITHA TSU26356
LALITHA TSU20770
RONDA
TSU17398
BEVK
TSU15161
D24PAE1 TSU11662
BAISE
TSU12386
ALTENBE TSU06082
ALTENBE TSU05989
D81HMY1 TSU03619
FARRELL TSU03860
D10LLC1 TSU10469
WALDMARI JOB26251
WNC
TSU10641
LALITHA TSU23462
TCP
STC20125
##CSFPK1 JOB17489
##DB2LD0 JOB17466
##DB2LS1 JOB17470
##DB2MD1 JOB17478
RACFCMQT JOB20194
NETSADMX JOB20266
Command ===>
GG24-4131
Job Ended
Date
Time
03/30/93
03/29/93
03/29/93
03/29/93
03/29/93
03/29/93
03/28/93
03/28/93
03/28/93
03/27/93
03/27/93
03/27/93
03/26/93
03/26/93
03/25/93
03/25/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
14.23.05
20.02.09
11.58.32
11.54.38
11.47.47
08.25.49
21.06.03
15.26.12
14.53.01
16.00.32
15.56.16
11.39.24
14.09.28
10.46.52
17.43.32
08.26.51
08.58.43
02.48.17
03.48.07
03.44.50
03.43.19
03.41.35
03.03.32
03.03.31
Amount of Common Storage

Not Released at End of Job
CSA ECSA
SQA ESQA
128
288
288
288
288
288
288
288
288
288
288
288
288
288
288
288
288
0
0
0
0
0
0
0
2288
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1808
1424
1424
1424
1424
1424
1424
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Scroll ===> CSR
Common Storage Analysis

Starting with the SQA usage exception, we will take a look at the common
storage usage. We invoke the Common Storage report (STORC) by placing the
cursor on the SSQA% exception line and pressing Enter.
The STORC report provides information about the use of common storage (CSA,
ECSA, SQA, and ESQA) within the system. The top section of the report provides
overall system information about the use of common storage. All data is
provided in total number of bytes used and the percentage used relative to the
IPL definitions.
The bottom section of the report provides job-related information about the use
of common storage for jobs active during the specified range period. The jobs
are sorted by descending storage percentage; that is, for each job with the
maximum of the four common storage percentages, the job with the highest
maximum percentage is reported first. Two additional lines on top of the bottom
section show the common storage allocations on behalf of the MVS system
( %MVS) and the common storage that was not freed after job termination ( %REMAIN).
The ACT column indicates whether a job is still running (blank), ended and
released all its common storage (N), or ended and has not released all its
common storage (H). The ACT column with an H and the %REMAIN line provide
cursor sensitivity to invoke the Common Storage Remaining report (STORCR).
The STORCR report provides a list of all jobs that ended but have not released
all of their common storage since the last IPL, plus information about the JES
jobid and the job ended date and time. The sort order is the same as for the
STORC report.
In our example, we can see that the high SQA usage is caused by allocations on
behalf of the MVS system and MVS system address spaces.
55
Demo
Return to the WFEX report. For this part of the demonstration, the invocation
of the ENQ report is used as an example.
To obtain the correct
reporting range
Place the cursor in the
Reason column on ENQ
for Job D1CI2PVX and
FREF
...
CORVUS
D1CI2PVX
D1CL2FDF
...
DEV -HSM013
ENQ -SYSDSN
98.0 % delay SLM.ZRAALLEN.LOADLIB
ENQ -SYSDSN
press
to
invoke the Enqueue
Delay report

Demonstration 2 ...
RMF 4.3.0
Samples:
100
Time: 03/30/93 at 14.23.20

Demonstration 2 ...

Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
16
35
STCs
53
2
47
ALL TSO
520
5
51
TSO short
509
0
73
ALL BATCH
10
9
18
TSO longer
2
2
61
ALL STC
90
3
52
TSO medium
1
0
39
Processors
138
3
74
TSO long
3
2
41
Devices
59
8
49
Btch short
6
6
9
*DMN000
39
1
78
Btch long
4
3
33
------------------------------ Exceptions ------------------------------------Name
Reason
*DMN001
AT > 40
47.38 sec
Btch short ENQ -SYSDSN
4.0 users
SLM.ZRAALLEN.LOADLIB
CORVUS
DEV -HSM013
CORVUS
DEV -HSM013
D1CI2PVX
ENQ -SYSDSN
D1CL2FDF
ENQ -SYSDSN
D1LI1LPA
ENQ -SYSDSN
D1LI1LPX
ENQ -SYSDSN
D11DFF1T
SUBS-HSM
GREAGOR
DEV -HSM013
*SQA*
SSQA% > 80
88.0 %
ALL BATCH ENQ -SYSDSN
4.0 users
SLM.ZRAALLEN.LOADLIB
CONSOLE
OPER-Message
DRV021
Not avail
D54ECA
Not avail
D54ECB
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
WFEX field
Invoked Report
DMN001
G R O U P (analyze components of active response time)

Continue with
Line 1 of 32
Range +
100 Sec
ENQ
Btch short
ALL BATCH
ENQ
-SYSDSN
D E L A Y (overview of all delays related to this groups)

Continue with
JOB
via Jobname, % Delayed for
ENQR
via Primary Reason
ENQ (Relation: Job delayed - Resource - Job(s) holding)

Continue with
JOB
via Jobname, Holding
ENQR
via Resource Waiting
E N Q R (Relation: Resource - Job(s) delayed - Job(s) holding)

Continue with
JOB
D1CI2PVX
D1CL2FDF
...
GG24-4131
ENQ column
via Delayed, Holding
J O B (upto 3 Jobs found holding the resource)

Continue with
.JOB
via Job
ENQR
via Primary Reason
Enqueue Delay Analysis

We continue with our problem determination and look at jobs delayed for an ENQ
resource. For all jobs related to this delay we see the delay percentage during
the reporting range and the data set name to which the serial access has been
requested. The effect of this delay on the system is reflected in the top section
by a red workflow indicator for Btch short and a yellow indicator for ALL BATCH
jobs. In the bottom section, two exception lines are displayed for Btch short and
ALL BATCH, using the scrolling with the PF8 key, indicating the average number of
delayed users. The reason shows the exception criteria specification, and the
56
critical value column provides the actual active response time causing the
exception.
Foil Demonstration 2-12 shows the available possibilities for continuing the
analysis. Starting with the Group Response Time report provides an overview of
the actual impact to the active response time for DMN001. The DELAY reports
for Btch short or ALL BATCH provide an overview of all delays related to these
groups.
We will continue with the ENQ or ENQR reports to analyze the problem directly.
Both reports provide the relationship between the delayed jobs and the resource
that is the reason for the delay, and the jobs holding the resource and causing
the delay. In our case, the ENQR report is the best solution, because all jobs
wait for the same enqueue. You get this report by placing the cursor not under
ENQ as before, but under -SYSDSN. The ENQR report lists the resource, all jobs
delayed in ascending order of their delay percentage and, for each job, a list of
the jobs found holding the resource during the reporting range. If the enqueue
resource was held by a job from another system, the system id is provided as
well.
57
Demo
Place the cursor on Job
CORVUS and press
...
CORVUS
DEV -HSM013
59.0 % delay Short block size or excessive see
...
to invoke Job
report for CORVUS
Invoking the Device Resource Report within the example:

Place the cursor on
Volume HSM013 and
...
-------------------------- Volume
...
HSM013
Device Data -------------------

Demonstration 2 ...
Samples:
Job: CORVUS

100
Time: 03/30/93 at 14.23.20
press
to
invoke Device
Resource report for
volume HSM013

Demonstration 2 ...
Line 1 of 4
Range +
100 Sec
Primary delay: Excessive disconnect time on volume HSM013.
Probable causes: 1) Sequential access of data with short blocksize.

2) Active datasets spaced widely across volume.
3) Overloaded channel paths causing reconnect delays.
RMF 4.3.0 Device Resource Delays -- AQTS

Line 1 of 5
Samples:
100
Time: 03/30/93 at 14.23.20
Range +
100 Sec
Volume
/Num S
HSM013 S
06C
Act Resp ACT CON DSC PND %,

Rate Time
%
%
%
Reasons
9.1 .092
-------------------------- Volume HSM013 Device Data -------------------------Number:

06C
Active:
68%
Pending:
2%
Average Users
Device:
33903
Connect:
4%
Delay DB:
1%
Delayed
Shared:
Yes
Disconnect: 62%
Delay CU:
0%
0.8
Delay DP:
0%
C ASID DMN PG % PROC DEV %
%
% PROC DEV STOR SUBS OPER ENQ Reason
T 0137 ***
2
7
1
4 70 21
5
0
70
0
0
0
0 HSM013
4
2 15
1
1 11
0
0
0
11
0
0
0
0 HSM013
3
2
0
0
0 21
0
0
0
0
0
0
0
5
2
5
0
3 59
0
5
0
59
0
0
0
0 HSM013
Command ===>
GG24-4131
Scroll ===> CSR
68
62
PND
DB
2
1
DEV/CU
Type
33903
3990-3
Jobname C DMN
PG
CORVUS
T ***
GREAGOR T ***
DURAND
T ***
MJORDAN T ***
FIGURA
T ***
2
2
2
2
2
Command ===>
GG24-4131
61
9
4
3
0
Finally, we will take a look at the device delays shown as exceptions on the
WFEX screen. The overall system impact is relatively low as indicated in the
speed section of the WFEX report. Because there are just two jobs in the
exception section and only one with higher delays, we will examine the job
directly by placing the cursor on the jobname and pressing Enter.
3
0
0
0
1
Scroll ===> CSR
Device Delay Analysis
58
USG DLY
%
%
The top section of the Job report gives possible reasons for the delay; the
middle section provides a summary for the device causing the delay; the bottom
section gives a performance summary for all periods. The shown reasons are
derived from the device response time data. In our example, the high
disconnect time (>40%) combined with a low connect time (<30%) is taken as
a base for the possible explanations. By entering HELP, RMF Monitor III
provides more background information on why it chooses this explanation and
on other reasons that might cause the delay.
We invoke the resource oriented device report (DEVR) from the middle section to
get a final picture of the device. The low activity rate and the relatively high
response time are consistent with the high disconnect time. It looks like that the
active data sets are spaced widely across the volume for the user CORVUS.
59
60

Additional Features
Additional Features
Tutorial/Help
Print Reports
Modify Reports
GG24-4131
A d d . Feat.-1
Options
Session Initialization Options (including: output data

set for Hardcopy and input data set containing the IPS
member)
Color Options (Graphic colors and patterns)
Language Options (Date/Time format)
Report Options
Option Sets (customization results, color and language

options)
Tutorial and Help
Hardcopy and ICU
RMF Utility
SMF Records
GG24-4131
A d d . Feat.-2
Additional Features
We changed, modified, and discussed options several times throughout the
demonstration of RMF Monitor III. It should be mentioned that this was just a
portion of the full customization capability of RMF Monitor III. It is also important
to mention that RMF Monitor III is able to save different option sets and converts
option sets from prior releases to the current release.
The tutorial provides an online introduction into the concepts and usage of RMF
Monitor III. Help is available for all reports and option panels and sometimes
provides more detailed analysis information than shown in the reports (for
example, delay reasons and explanations for the Job report).
Hardcopy output of the Monitor III reports can be generated using the
HARDCOPY and ICU commands. ICU generates a plotfile from a graphic report
and requires GDDM. HARDCOPY ON saves all future invoked reports into the
output data set specified under session options until HARDCOPY OFF is issued.
It saves the whole report as it is contained in the ISPF table.
RMF Monitor III gatherer also writes the following SMF records, which contain
information for postprocessing the available Monitor III reports:
61
62
72(2)
Workload information, related to the group response time report.
74(2)
XCF data, related to the XCF subsystem report and which can be
processed by the RMF Postprocessor.
74(3)
OpenEdition Services MVS measurements, which can be processed by

the RMF Postprocessor.

Monitor III Report Modification

RMF Monitor III Phase Concept
Each report can be changed

Phase 1
ERB3RHP1
Content is determined by options and format tables

Build ISPF table
Layout can be modified via change/replacement of

panels
Phase 2
N/A
Modify tables
Phase 3
User exits available:
ERB3RDSP
Display tables
Reporter data exists in ISPF tables

Phase 4
Modification using ISPF services
Creation of own reports
Report format definition utility ERBRMFU for

modification of existing and creation of new reports
GG24-4131
Clean up
A d d . Feat.-3
GG24-4131
N/A
Input
Address of Set-of-Samples
Proc.
Updates phase driver table and

creates ISPF table for report
Output:
Name of table in variable

PHDRTAB1
Input
Variable PHDRTAB1
Proc.
Creates new ISPF table
Output
Name of new table in PHDRTAB2
Input
PHDRTAB2, ERBFMTS3,
ERBPTGS3
Proc.
Creates of new ISPF table
Output
Report defined by var. ERBREPC
Input
User defined variables (phase 2)
Proc.
Cleans up resources allocated by

phase 2
Output
None
A d d . Feat.-4

The RMF Utility is an additional tool that provides the capability to modify
existing reports and to create completly new reports from the ISPF data collected
by RMF Monitor III. The RMF Utility is an ISPF based application that operates
on the RMF control tables (ERBPHDS3, ERBFMTS3, ERBPTGS3, contained in data
set userid.RMF430.ISPTABLE). It provides a panel-driven dialog that leads the
user through all modification steps and can be invoked from the TSO READY line
or ISPF command line by typing ERBRMFU, or TSO ERBRMFU, and pressing
Enter. Before we can start to use the utility we have to understand the RMF
Monitor III reporter phase concept.
Phase Concept
Whenever a report is invoked, RMF Monitor III creates an ISPF table from the
gatherer data and displays this table. These steps are described by four phases
as shown in foil Add. Feat.-4. In addition to the creation of the ISPF table for the
selected report, RMF Monitor III maintains three control tables to define the
functions that must be performed (ERBPHDS3), to define the tabular report layout
(ERBFMST3), and to define the graphic report layout (ERBPTGS3).
63

Reporter Control Tables

RMF Utility
Phase driver table E R B P H D S 3

Dialog Step
defines function (process) to be performed in each phase
describes names of data tables created in phase 1 and 2
contains an entry for each report type
Action
Tabular report format table E R B F M T S 3
defines layout of each RMF report (display and hardcopy)
contains related panel and help panel name
describes in detail header,sub-header lines and scrollable data
Requested Information
RMF Utility Primary Panel

Action: Create, Modify, or Delete
Report Name
Prototype Name (for a new report)
Control Table Modifications

Phase Driver Definition
Select String (Program or Command File) for each

phase
Table Names for phase 1 and 2.
Report Format Definition
Tabular and/or Graphic Report

Name of Report Panel
Name of Help Panel
Variable names of line and sequence number to
access the data
Parameter table for graphic reports E R B P T G S 3
defines layout of each graphic RMF report
contains associated help panel name
Tabular Parameter Definition
Variable
Content
PHDREPNA
SYSINFO
Report
Report
Report
Report
PHDREPSE
11
Graphic Parameter Definition
PHDRPH1
PGM(ERB3RHP1) PARM(SYSINFO)
Layout
Title of Y-Axis, Minimum Axis Range, Selection Rule
PHDRPH2
null
Label D e f i n i t i o n s
Bar Labels for upto 2 bartypes
PHDRPH3
PGM(ERB3RDRS)
Column Specification
Assignement of ISPF variable substitution
PHDRPH4
null
Verification Step
PHDRTAB1
ERBSYST3
Confirmation
PHDRTAB2
ERBSYST3
GG24-4131
A d d . Feat.-5
Header layout
Subheader layout
Column layout
Cmmndline layout
GG24-4131
Definition of text and variable positions on screen

Assignment of ISPF variable substitutions
SAVE or CANCEL
Display modified report
A d d . Feat.-6
RMF Utility Dialog

Foil Add. Feat.-5 shows the steps of the RMF Utility dialog. Each described step
is related to one or more panels to enter the information. You can go forward by
pressing ENTER and backward by pressing PF3. The information will be saved in
the RMF control tables only after SAVE has been entered on the last panel.
New User Reports

To create a new report, the following steps must be executed:
1. Provide a program or command file for phase 1 or 2.
2. Use the RMF Utility to include the new report in Monitor III.
3. Change the RMF Primary Menu (ERB3PRM) to select the new report.
Why Modify Existing Reports

Some information is collected by the RMF Monitor III data gatherer and available
through the RMF ISPF tables, but is not displayed on the standard reports; for
example:
64
SYSINFO report, TCB and SRB time for groups
DEVR report, IOS queue time

Example of User Report: System Trend 1/2
RMF 4.3.0 System Trend

-- AQTS
Samples(AVG):
100 (Tot):
2000 Range: 00.33.20

Example of User Report: System Trend 2/2

-- AQTS
Samples(AVG):
100 (Tot):
2000 Range: 00.33.20
Line 1 of 20
/Line:
100 Sec
Startime: 03/30/93 at 14.30.00

Endtime: 03/30/93 at 13.58.20
Time
T143000
T142820
T142640
T142500
T142320
T142140
T142000
T141820
T141640
T141500
T141320
T141140
T141000
T140820
T140640
T140500
T140320
T140140
T140000
CPU
%
39
53
50
35
35
39
40
41
49
54
61
69
59
64
52
48
58
54
63
SRB
%
2.5
2.5
2.5
2.2
2.2
2.2
2.2
1.9
2.2
2.2
2.0
2.7
2.2
2.5
1.9
2.0
1.9
2.1
2.6
Range/Line:
Total Range:
TCB
%
WFL -Users-- -AVG USG% TOT ACT PROC DEV
31.0
41.9
39.8
26.5
26.3
30.7
32.1
33.8
39.9
45.4
52.2
57.2
50.7
54.1
44.2
39.7
50.6
44.4
50.8
45
52
52
38
35
39
51
52
53
54
48
59
57
62
62
67
67
57
50
623
623
623
621
620
620
620
620
620
623
623
623
623
623
622
619
618
620
619
11
13
14
15
16
16
10
9
11
12
12
12
11
10
8
9
9
12
14
2.5
3.6
3.8
2.0
2.2
2.6
2.9
2.5
3.2
3.4
3.8
4.3
3.8
4.0
3.4
3.2
3.6
3.5
4.0
2.8
4.4
4.6
4.2
4.2
4.5
3.6
3.0
3.4
3.6
2.7
3.9
3.4
3.4
2.5
3.6
3.3
4.4
4.5
100 Sec
2000 Sec
A d d . Feat.-7
GG24-4131
00.33.20
-Avg. Act. Users Delayed ForPROC DEV STOR SUBS OPER ENQ
1.5
1.0
1.8
1.1
0.8
0.8
0.8
0.6
1.0
0.8
1.0
1.8
1.1
1.1
1.0
0.8
0.7
0.7
2.3
Command ===>
GG24-4131
Line 1 of 20
/Line:
100 Sec
2.4
4.2
4.8
4.5
4.4
3.3
3.0
1.7
2.9
2.5
2.1
2.6
2.7
1.7
1.2
1.1
1.5
2.0
2.9
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1.4
0.9
0.1
1.1
1.5
1.6
0.9
1.7
0.8
1.2
1.6
0.7
0.7
0.4
0.1
0.2
0.7
2.1
2.0
1.3
1.2
0.7
0.6
1.4
0.2
1.5
1.1
0.7
0.4
1.5
0.3
0.9
1.1
1.3
1.2
0.5
1.2
1.2
0.0
0.0
0.3
3.0
4.0
5.0
0.0
0.0
0.5
0.9
1.0
0.2
0.0
0.2
0.0
0.0
0.0
0.0
0.0
Scroll ===> CSR
A d d . Feat.-8
Example of User Report: System Trend

An example of a System Trend report is provided to demonstrate the possibility
of enhancing the capabilities of RMF Monitor III with user defined reports. A
complete description of the command file, the necessary changes to the RMF
Primary Menu and the steps to include the report to the RMF tables is provided
in Part 3, Specifications on page 69.
The System Trend report shown extracts the *SYSTEM line from the system
information report (SYSINFO) respectively for the last 20 ranges, starting at the
current reporting range. It can be used:
To analyze how delay situations develop in the system
To analyze how long delay situations last in the system
To understand the change in system utilization
To identify peak utilization
As a system summary report
The header was changed to show the reporting range, the average number of
samples, and the total number of samples for the total range. The subheader
provides more detail information about the reporting range. The data lines show
the start time of the reporting range, the WFL, USER (TOT and ACT), USING, and
DELAY information from the *SYSTEM line of the SYSINFO report, and the
percent of CPU, TCB, and SRB utilization for the system. The TCB and SRB data
65
is also contained in the *SYSTEM line, but not displayed in the default SYSINFO
report.
The reporting interval of 20 ranges displays the last 30 minutes beginning from
the current range. By using the FREF command, the next reporting interval
starts exactly 30 minutes after the current interval and the last two ranges of the
current interval will be displayed again.
Note
The System Trend report shown in the example above cannot be generated
for 20 Mintimes from the provided data sets due to the limited number of
samples included in the data sets. The report will show a maximum of only 6
Mintimes for data set userid.RMFCC98 and a a maximum of only 2 Mintimes
for data set userid.RMFCC99.
To obtain the System Trend report, include the provided RMF Monitor III
Primary Menu in your ISPPLIB concatenation, the REXX Exec USGTREND in
your SYSPROC or ISPEXEC concatenation, and follow the steps described in
Part 3, Specifications on page 69 for the RMF Utility Dialog. The report
can then be invoked using USGTREND from the RMF Monitor III Primary
Menu and using =USGTREND from all other report panels.
66

Summary
RMF Monitor III to analyze performance

degradations
RMF Monitor III to monitor the system
Concepts (Delay - Reason - Cause)
Delayed Job(s) - for Resource - Holding Job(s)
Resource - Competing Jobs
Features:
Go Mode, Cursor Sensitivity, Automatic Customization,

Helps, Tutorials, ...
Enhancements:
GG24-4131
Use the RMF Utility to define own reports
A d d . Feat.-9
Summary
To summarize, this presentation has focused on RMF Monitor III and how it can
be used in different ways to improve performance management in an
installation.
The major concept of RMF is to identify the cause for delays in the system. It
shows the relationship between delayed job(s) and holding job(s) and the
contending resource. It provides different perspectives to analyze problem
situations (job oriented, resource oriented).
The two demonstrations showed how RMF Monitor III can be used to analyze
performance problems and to monitor the system. RMF Monitor III offers many
features to make performance management easier.
With the RMF Utility, RMF Monitor III offers you the capability to define your own
user reports and to enhance the default report set.
67
68
Part 3. Specifications
This part describes very specific technical items, that complement Chapter 4,
Additional Features on page 61.
Chapter 5, Specifications for System Trend Report on page 71 details the
CLIST, the new Primary Menu, and all the panels used to generate a new report
as explained in Monitor III Report Modification on page 63.
Chapter 6, Installation Performance Specification on page 81 presents an IPS
sample member as used in Further Investigations on page 41.
69
70

This chapter details the clist, the changed RMF Primary Menu, and all the
screens used to specify the System Trend Report.
The clist and the modified Primary Panel are provided with the diskette.
The following steps must be accomplished:
1. Include the command file USGTREND in RMF Monitor III Phase 1
2. Change the RMF Monitor III Primary Selection Menu to allow the selection of
the new report. (Changes are highlighted, dots identify unchanged part of
the panel)
3. Walkthrough of the RMF Utility Dialog, with the correct specifications for the
System Trend Report.
5.1 Command File to Generate the System Trend Report

Note: The numbers in reverse font point to the explanations at the end of this
section.
/*REXX*/
/* System Trend Report:
*/
/* - retrieves data for the last twenty Ranges from the
*/
/*
SYSINFO report, starting at the current Range
*/
/* - extracts the SYSTEM infromation line and saves the
*/
/*
data into a new user defined table USGTREND
*/
/* - calculate the reporting range and amount of samples
*/
/*
and save them into the shared variable pool to
*/
/*
generate the header information
*/
address ISPEXEC
/* Defines the logical line and sequence number key variables */
KEYVC = KEYS(USGLOGLN USGSEQUN)
1
/* Defines the table variables
*/
TVARs = USGNAMVC USGWFLVC USGTUSVC USGAUSVC USGAUPVC USGAUDVC,
USGADPVC USGADDVC USGADSVC USGADUVC USGADOVC USGADEVC,
USGCUVVC USGTCBVC USGSRBVC
NAMVC = NAMES(TVARS)
TBCREATE USGTREND KEYVC NAMVC NOWRITE REPLACE
/* Initializes the logical line and sequence number
*/
/* Note: must be set for graphic display
*/
USGLOGLN = 0
USGSEQUN = 0
/* Gets the current BEGIN and END date/time and the Mintime */
/*
option of the gatherer
*/
/* Note: When this EXEC gets control RMF has already data
*/
/*
retrieved
*/
/* Error Exit: If the data is not valid exit immediately
*/
VGET (ERBDTBEG ERBDTEND ERBMNTIM) SHARED; 2
VGET (ERBDATE ERBTIME ERBRANGE ERBSAMPL) SHARED;
IF DATATYPE(ERBSAMPL,N) = 1 THEN EXIT
USGDATE = ERBDATE; USGTIME = ERBTIME
USGRANGE = ERBRANGE
USGSMPLT = ERBSAMPL
/* Starting from the current date/time
*/
/*
---- gets data out of the SYSINFO report by invoking */
/*
RMF PHASE 1 module ERB3RPH1 with PARM(SYSINFO)
*/
IMax = 20 3
/* Number of Times to retrieve data */
ICur = 0
/* Number of times data was retrieved */
DO I = 1 TO IMax
SELECT PGM(ERB3RPH1) PARM(SYSINFO)
4
/* Get the CPU utilization from the shared pool
*/
VGET (SYSCUVVC) SHARED;
5
/* Read the system line from the SYSINFO table
*/
TBTOP ERBSYST3;
6
TBSKIP ERBSYST3;
USGWFLVC = SYSWFLVC; USGTUSVC = SYSTUSVC; USGAUSVC = SYSAUSVC;
USGRSPVC = SYSTRSVC; USGAUPVC = SYSAUPVC; USGAUDVC = SYSAUDVC;
USGADPVC = SYSADPVC; USGADDVC = SYSADDVC; USGADSVC = SYSADSVC;
USGADUVC = SYSADUVC; USGADOVC = SYSADOVC; USGADEVC = SYSADEVC;
71
USGSRBVC = SYSSRBVC; USGTCBVC = SYSTCBVC;

USGCUVVC = SYSCUVVC;
USGNAMVC = T||SUBSTR(ERBDTBEG,9)
USGLOGLN = USGLOGLN + 1
USGSEQUN = USGSEQUN + 1
/*
---- puts them into the table USGTREND via TBADD
*/
/*
gets the previous Mintime by invoking the
*/
/*
GET_DATA Function ERB3RGTS
*/
TBADD USGTREND ; 7
/*
---- prepares the parameter area for GET_DATA function */
/*
ERB3RGTS
*/
/*
GDPR3_FC
= 3 (GET_BRF) (BREF equivalent) */
/*
GDPR3_NUM
= 1 (not relevant for BREF )
*/
/*
GDPR3_END_DT = Begin date/time of prev. call */
/*
GDPR3_BEG_DT = same ( not used by BREF )
*/
ICur = ICur + 1;
IF I < IMax THEN DO;
8
GDPR3_FC = D2C(3,4);
GDPR3_NUM = D2C(1,4);
ENDDATI = ERBDTBEG;
BEGDATI = ERBDTBEG;
TIMVC = BEGDATI||ENDDATI;
PARM1 = PARM(||GDPR3_FC||GDPR3_NUM||TIMVC||);
/*
---- invokes function GET_DATA module ERB3RGTS
*/
SELECT PGM(ERB3RGTS) PARM1;
IF RC = 0 THEN LEAVE
9
/*
---- gets the Begin/End date/time and the gatherer
*/
/*
Mintime value
*/
VGET (ERBDTBEG ERBDTEND ERBMNTIM) SHARED;
VGET (ERBSAMPL) SHARED
USGSMPLT = USGSMPLT + ERBSAMPL
END;
END;
/*
---- calculates reported time range and put start
*/
/*
time, date, range and sample count into the
*/
/*
shared pool to generate the header information
*/
USGRANGE = strip(USGRANGE * ICur);
10
HH = RIGHT(USGRANGE % 3600,2,0); MINS = USGRANGE // 3600
MM = RIGHT(MINS % 60,2,0); SS = RIGHT(MINS // 60,2,0);
USGRGTIM = HH.MM.SS
USGRANGE = right(USGRANGE,6)
USGSAMPL = FORMAT((USGSMPLT/MAX(ICur,1)),6,0);
USGSMPLT = FORMAT(USGSMPLT,6);
VPUT (USGDATE USGTIME USGRANGE USGRGTIM USGSMPLT USGSAMPL) SHARED;
EXIT;
Defines the logical line number and sequence number to access the rows
in the ISPF table for the new report. Defines all data variables for the
table, creates the ISPF table and initializes the logical line and sequence
number.
In order to generate the header for the trend report, the date, time, range
and sample values for the current MinTime has to be retrieved from the
ISPF shared pool and the shared pool variables of the trend report, which
will be used to displayed the total available reporting range, must be
initialized.
Variable IMax limits the number of times the RMF Monitor III reporter
attempts to retrieve data to create the trend report. The current value of
20 together with the default reporting range of 100 seconds creates a
report for the last 33.20 minutes. By using the FREF command the next
report will start exactly 30 minutes after the actual first reporting interval
and the last two reporting ranges will overlap.
The Variable ICur is used to count the number of times data has been
retrieved and to calculate the total reporting interval (see 10 ).
72
The REXX Exec USGTREND has been designed to replace the RMF
provided phase I program. In order to generate a new report the data
from the basis report must be retrieved via the default RMF Monitor III
phase I program. The USGTREND report extracts the *SYSTEM line from
the SYSINFO report, so the SYSINFO report must be requested as
parameter for program ERB3RPH1.
The CPU utilization (TCB+SRB%) is part of the top section of the

SYSINFO report. All these variables are contained in the RMF Monitor III
reporter ISPF shared pool.
The *SYSTEM line is always the first line in the SYSINFI ISPF table. With
TBTOP and TBSKIP the pointer is set on the first line and the content of
the variables for the *SYSTEM line can be copied into the variables for the
System Trend report.
At this point, the new line is added to the table for the System Trend
report.
To retrieve the previous reporting range module ERB3RDRS must be

invoked.
Input Parameters
Format:
0 . . 3 4 . . 7 8 . . . . . . . . . . . . .22 . . . . . . . . . . . . . .35
FC
---
Begin_Date
End_Date
Parameter
Description
Example
FC
Functioncode, valid values are 3=BREF and

4=FREF. The functioncode is integer value,
therefore it must be converted to character
before it is passed to ERB3RDRS.
DC(3,4)
---
RMF internal parameter, set to 1
DC(1,4)
Begin_Date
The begin date is used by FREF (FC=4) to obtain

the next reporting range.
03301993142000
Format: MMDDYYYYHHMMSS
End_Date
The end date is used by BREF (FC=3) to obtain

the previous reporting range.
03301993142000
Format: MMDDYYYYHHMMSS
If ERB3RGTS is not able to obtain data (for example, at the end of the data
in the VSAM dataset), it returns with a non-zero return code. In this case,
the USGTREND Exec terminates the loop and continues with 10 .
10
After all data has been obtained and the ISPF table for the System Trend
report has been created, the total available reporting range and the
average number of samples is calculated and the variables are placed in
the ISPF shared pool to allow RMF to generate the header and subheader
for the report.
73
5.2 Changed RMF Primary Menu

This section details the user Primary Menu.
)attr default(!+_)
/* ************************************************************************** *
/*
PANEL NAME: ERB3PRM
@B1C
/*
/*
PROPRIETARY STATEMENT=
@B1A
/*
LICENSED MATERIALS - PROPERTY OF IBM
@B1A
/*
THIS PANEL IS RESTRICTED MATERIALS OF IBM
@B1A
/*
5685-029 (C) COPYRIGHT IBM CORP 1986 1993
@B1A
/*
SEE COPYRIGHT INSTRUCTIONS
@B1A
/*
5@B1D
/*
RMF Primary Selection Menu
@B1A
.
.
.
)body expand() cmd(zcmd)
+
!
+
!Selection ===>_zcmd +
<Enter selection number or command on selection line.
!
!
<Basic Reports
Detail Reports
! 1<WFEX
+Workflow/Exceptions (WE)! 7<DELAY +Delays (DLY)
! 2<SYSINFO +System information (SI) ! 8<DEV
+Device delays (DD)
! 3<GROUP +Group response time (RT)! 9<DEVR
+Device resource (DR)
!
+Group!=> z
+
! 10<ENQ
+ENQ delays (ED)
! 4<GROUPS +Group reports selection ! 11<ENQR
+ENQ resource (ER)
! 5<JOB
+Jobs primary delay
! 12<PROC
+Processor delays (PD)
!
+Job! => roxjjnu3 + ! 13<STORAGE +Storage report selection
! 6<JOBS
+Job report selection
! 14<SUBS
+Subsystem report selection
!
!
<Other Selections
! DI<DSINDEX +Dataset index
! T<TUTORIAL+Tutorial aid
! OP<OPTIONS +Option selection
! X<EXIT
+End display session
+
+
!
!
Licensed Materials - Property of IBM +
)init
.
.
.
)REINIT
.
.
.
)PROC
.
.
.
/* translate abbreviated or numbered selections to standard string */
&erbcmdc=trans(&erbcmdc
1,WFEX
2,SYSINFO
3,GROUP
4,GROUPS
5,JOB
6,JOBS
7,DELAY
8,DEV
9,DEVR
10,ENQ
11,ENQR
12,PROC
13,STORAGE
14,SUBS
DSINDEX,DSINDEX
DI,DSINDEX
OP,OPTIONS
TUTORIAL,TUTORIAL
TUT,TUTORIAL
T,TUTORIAL
X,EXIT
Z,EXIT
EXIT,EXIT
GR,GRAPHIC
GRAPH,GRAPHIC
GRAPHIC,GRAPHIC
GRAPHICS,GRAPHIC
74
CANCEL,CANCEL
F,FIND
FIND,FIND
ICU,ICU
HC,HARDCOPY
HARDCOPY,HARDCOPY
RESET,RESET
RFIND,RFIND
TAB,TABULAR
TABLE,TABULAR
TABULAR,TABULAR
TOG,TOGGLE
TOGGLE,TOGGLE
USGTREND,USGTREND
*,*)
/* USGTRND added
@RVC*/
/* 2@B1D*/
/* gives message if value entered neither a valid selection nor a */
/* valid RMF PASSTHROUGH command ************************************* */
if (&erbzcmd = )
ver(&erbcmdc,NB,MSG=ERB562I)
ver(&erbcmdc,LIST, DELAY, DEV, DEVR, ENQ, ENQR, JOB, JOBS,
PROC, STORAGE, SUBS, SYSINFO, WFEX, DSINDEX, OPTIONS,
TUTORIAL, EXIT, GRAPHIC, GROUP, GROUPS, CANCEL, FIND,
ICU, HARDCOPY, RESET, RFIND, TABULAR, TOGGLE,
USGTREND,
MSG=ERB562I)
/* @RVC*/
/* gives message if value entered neither a valid selection nor a */
/* valid RMF PASSTHROUGH command for this panel ******************* */
ver(&erbcmdc,LIST, DELAY, DEV, DEVR, ENQ, ENQR, JOB, JOBS,
PROC, STORAGE, SUBS, SYSINFO, WFEX, DSINDEX, OPTIONS, USER,
TUTORIAL, EXIT,
USGTREND,
MSG=ERB573I)
/* @RVC*/
&ERBZCMDT = &ERBCMDC &ERBPARC /* Saves translated command input*/
/* selects action according to input ****************************** */
/********************************************************************/
/* The following code should be modified when a private report
*/
/* has been defined through use of the utility ERB3RDUT. The par. */
/* passed to program ERB3RDPC must be the report name as it has
*/
/* been defined to the utility.
*/
/********************************************************************/
if (&trail2 = &z)
&prmerr = noerror
if (&ERBCMDC = OPTIONS)
if (&ERBCMDC = STORAGE)
/* @W1A*/
if (&ERBCMDC = JOBS)
/* @W1A*/
if (&ERBCMDC = SUBS)
/* @Y1A*/
if (&ERBCMDC = GROUPS)
/* @B1A*/
if (&ERBCMDC = USGTREND)
/* @RVA*/
&prmerr = &z
/* @RVC*/
ver(&prmerr,NONBLANK,MSG=ERB562I)
&zcmd = &erbcmdc..&trail2
if (&erbparc = &z)
/* @W1A*/
&zcmd = &zcmd&erbsepc&erbparc
/* @W1A*/
if (&trail2 = &z)
&zcmd = &erbcmdc
if (&ERBCMDC = JOB)
&ERBJOBT =
/* Resets job report type
@W1A*/
&zsel = trans(trunc(&zcmd,.)
DELAY,PGM(ERB3RDPC) PARM(DELAY)
DEV,PGM(ERB3RDPC) PARM(DEV)
DEVR,PGM(ERB3RDPC) PARM(DEVR)
ENQ,PGM(ERB3RDPC) PARM(ENQ)
ENQR,PGM(ERB3RDPC) PARM(ENQR)
JOB,PGM(ERB3RDPC) PARM(JOB)
JOBS,PANEL(ERB3JRS)
GROUP,PGM(ERB3RDPC) PARM(GROUP)
GROUPS,PANEL(ERB3GPS)
PROC,PGM(ERB3RDPC) PARM(PROC)
STORAGE,PANEL(ERB3STR)
SUBS,PANEL(ERB3SUB)
SYSINFO,PGM(ERB3RDPC) PARM(SYSINFO)
WFEX,PGM(ERB3RDPC) PARM(WFEX)
DSINDEX,PGM(ERB3RDPC) PARM(DSINDEX)
OPTIONS,PANEL(ERB3OPT)
EXIT,EXIT
TUTORIAL,PGM(ISPTUTOR) PARM(ERBC)
USGTREND,PGM(ERB3RDPC) PARM(USGTREND)
, )
/* USGTRND added
@RVC*/
)end
75
5.3 Panel Samples

This section presents all the panels you walk through when defining the user
panel.
Warning
In Step 1, you have to CREATE the panel (in the ACTION line). MODIFY can be
used once the panel is created. Some panels can show different variables,
depending on whether you are in CREATE mode or MODIFY mode, and some
may not exactly reflect the provided panels.

Using the RMF Utility
Invocation

Step 1: RMF Utility Startup Panel
ERBRMFU (from TSO READY line or

ISPF command line)
Dialog
Provides 14 panels to define the new

report to RMF
Steps
The panels are grouped into 6 logical

steps
RMF Report Format Definition

Enter the following information. To continue press ENTER.
To exit enter CANCEL or press the END key.
ACTION
===> MODIFY
MODIFY, CREATE or DELETE
or CREATE, see WARNING
1. Startup/Action panel
REPORT NAME
===> USGTREND
Name of report
2. Phase D r i v e r table information (saved in

ERBPHDS3)
PROTOTYPE NAME
===> SYSINFO
Name of existing report to be

used as a prototype (optional)
3. Specify the desired report types and the

variable names for the line and
sequence number (necessary to access
the table data)
4. Tabular r e p o r t layout (saved in
ERBFMTS3)
5. Graphic r e p o r t layout (saved in
ERBPTGS3)
6. Verification (Save to confirm
changes/Cancel to discard changes)
Navigation
ENTER to continue with the next panel

Command ===> ____________________________________________________
PF3 to go back to the last panel
GG24-4131
76
GG24-4131

Step 2: Phase Driver Definition

Step 3: Report Format Definition

Report Name: USGTREND
Section 1: Phase Driver Information

To quit enter CANCEL. To go backwards press END.
Section 2: Report Format Information

Select Strings format is:PGM(nnnnnnnn) PARM(mmm) or CMD(nnnnnnnn mmm)

REPORT MODE
===> BOTH
TABULAR, GRAPHIC or BOTH
PHASE 1 SELECT STRING ===> CMD(USGTREND)

TABLE NAME
===> USGTREND
Name of reporter phase 1 result table
PANEL NAME
===> ERB3SYS
Name of tabular report panel
PHASE 2 SELECT STRING ===>

TABLE NAME
===> USGTREND
HELP PANEL NAME
===> ERB3SYS0
Name of HELP panel
LOGICAL LINE NUMBER

SEQUENCE NUMBER
===> USGLOGLN
===> USGSEQUN
Name of table variable

SELECTION CHARACTERS
===> USGTREND
Selection on Primary Option Panel
Optional name of phase 2 result table
PHASE 3 SELECT STRING ===> PGM(ERB3RDSP)

Command ===> ____________________________________________________
Command ===> ____________________________________________________
GG24-4131
GG24-4131

Step 4: Tabular Report Parameters

Step 4: Tabular Report Parameters ...

Section 3: Report Header Layout
Enter or change the report header lines. To continue press ENTER.

The following report header lines have been specified:

&ERBRMFVD System Trend
-- Z1
&ERBHCTXT Samples(AVG): Z2
(Tot): Z3
Range: &USGRGTIM /Line: Z4
You may intermix: text, variables, and variable placeholders (&Z).

If you specify variable placeholders (&Z) the next panel will ask you
to specify the variable name that is to replace each
The following
&ERBSID
&ERBSAMPL
&ERBTIME
Sec
Specify the placeholder (Z) number next to the variable name to replace each Z
above.To continue press ENTER.To go backwards press END.To quit enter CANCEL.
variables are available for use in the header:

&ERBHCTXT
&ERBRMFVD
&ERBDATE
&ERBRANGE
&ERBSID
&ERBDATE
&ERBRMFVD
===> _________
===>
===>
===>
===>
1
__
__
__
&ERBHCTXT
&ERBTIME
&USGSAMPL
===> _________
===>
===>
===>
===>
__
__
2
__
&ERBSAMPL
&ERBRANGE
&USGSMPLT
===> _________
===>
===>
===>
===>
__
4
3
__
Enter or change up to two report heading lines:

&ERBRMFVD
&ERBHCTXT Samples(AVG): &Z_
System Trend
(Tot): &Z_
-- &Z_
Range: &USGRGTIM /Line:
Command ===> ____________________________________________________
GG24-4131
Sec
Command ===> ____________________________________________________
GG24-4131
77



Section 4: Report Subheader Layout
Enter or change the report subheader lines. To continue press ENTER.

The following report subheader lines have been specified:

Startime: &USGDATE at &USGTIME
Range/Line:
Z1
Sec
Endtime: &ERBDATE at &ERBTIME
Total Range: Z2
Sec
&USGRGTIM

The following variables are available for use in the subheader:
above. To continue press ENTER. To go backwards press END.To quit enter CANCEL.
===>
===>
===>
===>
===>
===>
===>
===>
1
__
__
__
__
__
__
__
===>
===>
===>
===>
===>
===>
===>
===>
2
__
__
__
__
__
__
__
===>
===>
===>
===>
===>
===>
===>
===>
_________
_________
_________
_________
_________
_________
_________
_________
Command ===> ____________________________________________________
Command ===> ____________________________________________________

Line 1 of 1
Enter or change the following information. To continue press ENTER.

Define meaning in attribute section
of associated table display (ERB3SYS).
Enter or change up to three column header lines:

CPU SRB
TCB
WFL -Users-- -AVG USG- -Avg. Act. Users Delayed ForTime
%
%
%
% TOT ACT PROC DEV
PROC DEV STOR SUBS OPER ENQ
_______________________________________________________________________________
Enter or change up to three model lines:
?Z
?Z ?Z
?Z
?Z ?Z
?Z
?Z
?Z
?Z
?Z
?Z
?Z
?Z
?Z
_______________________________________________________________________________
_______________________________________________________________________________
The following variables are available for use in the model lines:
Command ===> ____________________________________________________
GG24-4131
__
__
__
__
__
__
__
__

Section 5: Report Column Layout
DATA ATTRIBUTE CHARACTERS ===> _?|
===>
===>
===>
===>
===>
===>
===>
===>
GG24-4131

===>
===>
===>
===>
===>
===>
===>
&USGRANGE
_________
_________
_________
_________
_________
_________
_________
===>
===>
===>
===>
===>
===>
===>
GG24-4131
78
&ERBRANGE
_________
_________
_________
_________
_________
_________
_________
Enter or change up to five report subheading lines:

Startime: &USGDATE at &USGTIME
Range/Line:
&Z_
Sec
Endtime: &ERBDATE at &ERBTIME
Total Range: &Z_
Sec
&USGRGTIM
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________

Line 1 of 8
The following report column header and model lines have been specified:
CPU SRB
TCB
WFL -Users-- -AVG USG- -Avg. Act. Users Delayed ForTime
%
%
%
% TOT ACT PROC DEV
PROC DEV STOR SUBS OPER ENQ
Z1
Z2
Z3
Z4
Z5
Z6
Z7
Z8
Z9
Z10
Z11
Z12
Z13
Z14
Z15
USGNAMVC
USGTCBVC
USGAUSVC
USGADPVC
USGADUVC
===> ________
===> ________
===> ________
===>
===>
===>
===>
===>
===>
===>
===>
1
4
7
10
13
__
__
__
USGCUVVC
USGWFLVC
USGAUPVC
USGADDVC
USGADOVC
===> ________
===> ________
===> ________
===>
===>
===>
===>
===>
===>
===>
===>
2
5
8
11
14
__
__
__
USGSRBVC
USGTUSVC
USGAUDVC
USGADSVC
USGADEVC
===> ________
===> ________
===> ________
Command ===> ____________________________________________________
GG24-4131
===>
===>
===>
===>
===>
===>
===>
===>
3
6
9
12
15
__
__
__


Step 5: Graphic Report Parameters

Section 6: Command Line Layout

Section 7: Graphic Parameter Definition

GENERAL INFORMATION
The following variables are available for use in the command line:
&ZCMD
&AMT
NAME FOR HELP PANEL ===> ________
Name of HELP PANEL, if any
Enter or change the command line:

Command ===> &zcmd
TITLE FOR Y-AXIS
===> Average Number of Active Users
MINIMUM AXIS RANGE
===> 1
SELECTION RULE
===> 2
Scroll ===> &AMT

Axis will contain at least this
number of data points
Specify 0, 1, 2 or 3
Specify a variable name in each of the entry fields to replace each Z above.
Z1 ===>
Z2 ===>
Z3 ===>
Command ===> ____________________________________________________
GG24-4131

Step 5: Graphic Report Parameters ...
Command ===> ____________________________________________________
GG24-4131




LABEL INFORMATION FOR BAR TYPE I

PRIMARY LABEL
SECONDARY LABEL
PRIMARY COMPOSITE
SECONDARY COMPOSITE
===>
===>
===>
===>
USGNAMVC
________
_____
_____
Variable name containing label

Prefix of label
Prefix of label
LABEL INFORMATION FOR BAR TYPE II

PRIMARY LABEL
SECONDARY LABEL
PRIMARY COMPOSITE
SECONDARY COMPOSITE
===>
===>
===>
===>
________
________
_____
_____

Prefix of label
Prefix of label
Command ===> ____________________________________________________
GG24-4131

COLUMN SPECIFICATION FOR GRAPHIC BAR TYPES

NAME
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
USGADPVC
USGADDVC
USGADSVC
USGADUVC
USGADOVC
USGADEVC
USGAUPVC
USGAUDVC
________
________
LEGEND ID
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
14
08
15
28
29
09
19
18
__
__
TRANS ID
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
0
0
0
0
0
0
0
0
_
_
BAR TYPE ID
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
1
1
1
1
1
1
1
1
_
_
Command ===> ____________________________________________________
GG24-4131
79

Step 6: Verification Step
This is a confirmation/cancellation panel for report: USGTREND
The following actions are allowed:
Type
SAVE
Type
CANCEL command to cancel processing
command to save report
Press END
key to go one step backwards
Press ENTER
key to see the sample report
Command ===> ____________________________________________________
GG24-4131
80

This chapter details the IPS Parmlib member used in the demonstrations.
CPU=10.0,IOC=5.0,MSO=3.0,SRB=10.0 /* Service coefficients

/***
/*** Installation Performance Specifications
/***
APGRNG=(6-15)
/* IPS controls DPs 96 to 255
PVLDP=F91
/* Initiator priority
IOQ=PRTY
/* Priority I/O queuing
/***
/*** Domain Definitions
/***
DMN=1,CNSTR=(2,50),DSRV=(0,1000)
/* Btch-short
DMN=2,CNSTR=(1,50),DSRV=(0,5000)
/* Btch-long
DMN=3,CNSTR=(35,100),ASRV=(1000,1000) /* TSO-short
DMN=4,CNSTR=(10,100),ASRV=(1000,1000) /* TSO-medium
DMN=5,CNSTR=(10,100),ASRV=(1000,1000) /* TSO-long
DMN=6,CNSTR=(10,100),DSRV=(0,9999)
/* TSO-longer
DMN=7,CNSTR=(23,255),DSRV=(0,9999)
/* STCs
DMN=9,CNSTR=(1,2)
/* RMF
DMN=12,CNSTR=(2,20),ASRV=(1000,1000) /* TSO-extra
/***
/*** Performance Group definitions
/***
PGN=1,(DMN=1,DP=M4,DUR=40K)
/* Batch classes
(DMN=1,DP=M3,DUR=85K)
/* Batch
(DMN=2,DP=M2)
/*
PGN=2,(DMN=3,DP=F53,DUR=400)
/* TSO - short
(DMN=4,DP=F52,DUR=1400)
/*
- medium
(DMN=5,DP=F51,DUR=40K)
/*
- long
(DMN=6,DP=M4)
/*
- longer
PGN=3,(DMN=7,DP=M5)
/* STARTED TASKS
PGN=4,(DMN=7,DP=F93)
/* JES2
/* VTAM
/* RMF monitors
/* DB2
/* DFHSM
/* APPC ASCH
/* Netview
/* LLA VLF
PGN=22,(DMN=7,DP=M5)
/* GRS
*/
***/
***/
***/
*/
*/
*/
***/
***/
***/
*/
*/
*/
*/
*/
*/
*/
*/
*/
***/
***/
***/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
*/
81
82
Part 4. Foils
This part provides foils that can be used for presentations.
You can reproduce a page in this document as a transparency, if that page has
the copyright notice on it. The copyright notice must appear on each page being
reproduced.
83
Usage of RMF Monitor III

IBM Trademarks
The following items, used throughout the RMF presentation,

are trademarks of the IBM Corporation:
ESA/390
ISPF
MVS/ESA
TSO/E
OpenEdition Services MVS
GG24-4131
Copyright IBM Corp 1993

Agenda
Monitor III concepts
Monitor III Usage
Demonstration 1
Demonstration 2
Monitoring System Performances
Additional Features
User Written Reports
GG24-4131
Monitor III Usage
GG24-4131
Basics-1

Use Monitor III to:
Determine whether your MVS system runs without
performance problems
Identify jobs that are delayed and the reason for the delay
Identify the resources associated with delays
Identify resources with exceptional utilization
Monitor the availability of system resources
GG24-4131
Basics-2

Data Gatherer
One session per MVS system which:

Collects data for short-term problem solving
Collects data about system delays
Collects data about system resource utilization
Keeps data available in an in-storage buffer for immediate
problem determination
Records compressed data in VSAM data sets for later use
Creates SMF records
GG24-4131
Basics-3

Data Reporter
Multiple sessions per MVS system possible that:

Run under TSO/E and ISPF
Report on current data,
or report on earlier collected data,

or report on data from another system
Provide powerful features to monitor the system and
identify performance problems:
GG24-4131
Continous Monitoring (GO Mode)

Exception Processing
Cursor Sensitive Control
Basics-4

Cycle/Mintime/Range
Exact Count/Sampling Technique
States of a Job
Monitor III Metrics
GG24-4131
Basics-5

How does RMF Monitor III gather data?
Cycle = {50ms, ..., 1000ms, ..., 9999ms}

Mintime = {10s, ..., 100s, ..., 999s}
Formats the collected data into samples (at the end of
each cycle)
Stores the Set-of-Samples into the in-storage buffer and
VSAM data sets (at the end of each Mintime)

Data gathering techniques
Exact count technique
Sampling technique
GG24-4131
Basics-6

Job States
Using
Processor
Device
Delayed
Processor
Device
Storage
Subsystems
Operator
-TCB, SRB
-Connect
-IOS queue
-Disconnect
-Pending
-Common Storage Paging
-Local Storage Paging
-Virtual I/O
-Swap in
-Swapped out and ready
-HSM
-JES
-XCF
-Mount
-Message
Enqueue
Idle
Terminal Wait
Timer Wait
JES Queue
(Unknown)
GG24-4131
Basics-7

How does RMF Monitor III report data?

Common Metrics
Workflow (%) for jobs (address spaces) and resources

(device and processor)
Using (%) for jobs (address spaces)
Delay (%) for jobs (address spaces)
Average Active Users using or delayed for resources
Reporting Range
Multiple of Mintimes
Reports
GG24-4131
Job/Group oriented (DELAY, GROUP, SYSINFO, DEV, ENQ,

PROC, STOR, STORF, STORS, JOB)
Resource oriented (DEVR, ENQR, HSM, JES, XCF, STORR,
STORC, STORCR)
Workflow/Exception oriented (WFEX)
Availability of resources (WFEX)
Basics-8

GG24-4131
Basics-9

Monitor III Usage
Usage of Monitor III
Startup
Tabular/Graphic Reports
Report Options/Commands
Sequence to use RMF Monitor III Reports
Cursor Sensitivity
GG24-4131
Basics-10

ERB3PRM

Enter selection number or command on Selection line.

Basic Reports
1 WFEX
3 GROUP
Group => DMN001__
4 GROUPS Group reports selection
5 JOB
Jobs primary delay
Job
=> ROBERTV_
6 JOBS
Detail Reports
7 DELAY
Delays (DLY)
8 DEV
Device delays (DD)
9 DEVR
10 ENQ
ENQ delays (ED)
11 ENQR
ENQ resource (ER)
12 PROC
14 SUBS
Other Selections
DI DSINDEX Dataset index
OP OPTIONS Option selection
X EXIT
End display session
Licensed Materials - Property of IBM
Selection ===> _
GG24-4131
Basics-11

RMF Monitor III Tabular Reports

Samples:
100 Time: 03/31/93 at 15.15.00
Line 1 of 30
Range +
100 Sec

Processor(s) Online: 6
OPT = IEAOPT00
Average CPU Util:
74%
Average TCB+SRB: 66%
ICS = IEAICS00
Name
WFL --Users-- RESP TRANS VEC -AVG USG% TOT ACT Time /SEC UTL PROC DEV
*SYSTEM 50
*TSO
54
*BATCH 45
*STC
49
*ASCH
*OMVS
DMN000 70
DMN001 30
DMN002 59
DMN003 90
DMN004 31
DMN005 41
DMN006 60
DMN007 39
DMN009 67
DMN012
PG000
70
PG001
45
PG002
54
PG003
63
Command ===>
GG24-4131
576 14
11.76 0 4.7 3.1
1.6 3.0 0.0 1.3 0.7 1.0
478
8
11.76 0 2.8 1.7
0.6 2.0 0.0 1.2 0.0 0.0
7
4
0.00 0 1.4 0.7
0.9 0.7 0.0 0.0 0.0 1.0
91
2
0.00 0 0.5 0.7
0.0 0.3 0.0 0.1 0.7 0.0
0
0
0.00 0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
0
0
0.00 0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
39
1 .000 0.00 0 0.2 0.3
0.0 0.2 0.0 0.0 0.0 0.0
2
2 15.2 0.08 0 0.2 0.5
0.1 0.6 0.0 0.0 0.0 1.0
5
2 12.6 0.04 0 1.2 0.2
0.8 0.1 0.0 0.0 0.0 0.0
465
0 .033 11.75 0 0.2 0.1
0.0 0.0 0.0 0.0 0.0 0.0
1
0 .178 3.50 0 0.1 0.1
0.0 0.1 0.0 0.2 0.0 0.0
3
2 .493 1.91 0 0.6 0.6
0.1 0.5 0.0 1.0 0.0 0.0
5
4 6.43 0.64 0 2.0 0.9
0.5 1.4 0.0 0.0 0.0 0.0
53
1 .000 0.00 0 0.3 0.4
0.0 0.1 0.0 0.1 0.7 0.0
2
0 .000 0.00 0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
1
0 .000 0.00 0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0
39
1 .000 0.00 0 0.2 0.3
0.0 0.2 0.0 0.0 0.0 0.0
7
4 21.5 0.08 0 1.4 0.7
0.9 0.7 0.0 0.0 0.0 1.0
474
8 .516 11.75 0 2.8 1.7
0.6 2.0 0.0 1.2 0.0 0.0
39
0 .000 0.00 0 0.1 0.1
0.0 0.0 0.0 0.0 0.1 0.0
________________________________________________ Scroll ===> CSR
Basics-12

RMF Monitor III Graphic Reports

Samples:
100 Time: 03/31/93 at 15.15.00
Line 1 of 30
Range +
100 Sec
GG24-4131
Basics-13

RMF Monitor III Report Options
RMF SYSINFO Report Options
Line 1 of 5
Select (S), exclude (X) or fill-in groups for the SYSINFO report. Press END.
Selections made here also affect the Storage Summary (STORS) report.
Date ===> 03/31/93
Summary ===> NO
RPG data ===> NO
Sel
_
_
_
_
_
_
_
Group
______
______
*ALL
DMN003
DMN012
PG005
PG020
Sel
_
_
S
_
_
_
_
Group
______
______
*DMN
DMN004
PG000
PG008
PG021
Time ===> 15.15.00
Range ===> 100
Summary lines on SYSINFO report (YES NO)

RPG data lines on SYSINFO report (YES NO)
Sel
_
_
_
_
_
_
_
Group
______
______
*PG
DMN005
PG001
PG013
PG022
Sel Group
_ ______
_ ______
_ DMN000
_ DMN006
_ PG002
_ PG014
_
Sel Group
_ ______
_ ______
_ DMN001
_ DMN007
_ PG003
_ PG017
_
Sel
_
_
_
_
_
_
_
Group
______
______
DMN002
DMN009
PG004
PG019
GG24-4131
Basics-14

RMF Monitor III Report Options ...

Samples:
100 Time: 03/31/93 at 15.15.00
Line 1 of 30
Range +
100 Sec

OPT = IEAOPT00
Average CPU Util:
74%
ICS = IEAICS00
Name
DMN000
DMN001
DMN002
DMN003
DMN004
DMN005
DMN006
DMN007
DMN009
DMN012
70 39
30
2
59
5
90 465
31
1
41
3
60
5
39 53
67
2
1
1
2
2
0
0
2
4
1
0
0
.000
15.2
12.6
.033
.178
.493
6.43
.000
.000
.000
0.00
0.08
0.04
11.75
3.50
1.91
0.64
0.00
0.00
0.00
0
0
0
0
0
0
0
0
0
0
0.2
0.2
1.2
0.2
0.1
0.6
2.0
0.3
0.0
0.0
0.3
0.5
0.2
0.1
0.1
0.6
0.9
0.4
0.0
0.0
0.0
0.1
0.8
0.0
0.0
0.1
0.5
0.0
0.0
0.0
0.2
0.6
0.1
0.0
0.1
0.5
1.4
0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
1.0
0.0
0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.7
0.0
0.0
0.0
1.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Changed option(s) now in effect.

GG24-4131
Basics-15

Using RMF Monitor III Reports
GG24-4131
Basics-16

Cursor Sensitivity
GG24-4131
Basics-17
Demonstration-2
Monitoring System Performance
GG24-4131
Demonstration 2-1
Demonstration-1
Analyze System Workloads

Determine Components of Group Response Time
Determine Delay Reasons
Reports
SYSINFO
GROUP
PROC
JOB
GG24-4131
Demonstration 1-1

Demonstration 1
GG24-4131
Demonstration 1-2

Demonstration 1 ...

Samples:
100 Time: 03/31/93 at 15.10.00
Line 1 of 30
Range +
100 Sec
GG24-4131
Demonstration 1-3

Demonstration 1 ...

Line 1 of 30
Samples:
100 Time: 03/31/93 at 15.10.00 Range +
100 Sec
OPT = IEAOPT00
Average CPU Util:
94%
ICS = IEAICS00
Name
*SYSTEM 49
*TSO
66
*BATCH 30
*STC
70
*ASCH
*OMVS
DMN000 78
DMN001 23
DMN002 32
DMN003 73
DMN004 40
DMN005 68
DMN006 68
DMN007 67
DMN009 100
DMN012
PG000
78
PG001
30
PG002
66
PG003
49
Command ===>
GG24-4131
577
479
11
87
0
0
35
2
9
467
1
2
5
53
2
1
35
11
474
39
17
6
8
2
0
0
1
2
6
0
1
1
4
1
0
0
1
8
6
1
.000
27.8
37.7
.033
.121
1.45
11.1
.000
.000
.000
.000
58.7
1.06
.000
14.25
14.25
0.00
0.00
0.00
0.00
0.00
0.11
0.09
14.24
4.28
2.45
0.95
0.00
0.00
0.00
0.00
0.11
14.24
0.00
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.9
3.3
2.0
0.6
0.0
0.0
0.2
0.2
1.8
0.2
0.1
0.6
2.3
0.4
0.0
0.0
0.2
2.0
3.2
0.2
3.0
1.5
0.6
1.0
0.0
0.0
0.3
0.2
0.4
0.1
0.1
0.3
1.0
0.7
0.0
0.0
0.3
0.6
1.5
0.1
5.3
0.9
4.3
0.1
0.0
0.0
0.0
0.3
4.1
0.1
0.0
0.1
0.6
0.1
0.0
0.0
0.0
4.3
0.9
0.0
2.4
1.3
0.8
0.3
0.0
0.0
0.2
0.1
0.7
0.1
0.1
0.3
0.9
0.1
0.0
0.0
0.2
0.8
1.3
0.1
0.0 0.3
0.0 0.2
0.0 0.0
0.0 0.1
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.2
0.0 0.0
0.0 0.0
0.0 0.1
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.0
0.0 0.2
0.0 0.0
Scroll
0.2
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.2
===>
1.1
0.0
1.0
0.0
0.0
0.0
0.0
1.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1.0
0.0
0.0
CSR
Demonstration 1-4

Demonstration 1 ...
RMF 4.3.0 Group Response Time -- AQTS

Samples:
100 Time: 03/31/93 at 15.10.00 Range +
100 Sec
Group: PG001
Description: Batch classes
Primary Response Time Component: Processor delay
WFL
%
30
Users
Frames
TOT ACT %ACT
11
8
1
Total
Average Users
8.200
Response Time ACT 49.65
Average Users
Response Time ACT
Vector EXCP
UTIL
Rate
0 3,714
-AVG
PROC
2.04
12.3
USGDEV
0.58
3.50
PgIn
Rate
0.0
TRANS
Ended
Rate
0.1
--- Response Time ----- Ended TRANS-(Sec) Queued Active Total

9.000 49.65 58.65
-------------Average Delay-------------PROC DEV STOR SUBS OPER ENQ OTHER

4.33 0.78 0.00 0.00 0.00 1.04 0.00
26.1 4.71 0.00 0.00 0.00 6.27 0.00
---STOR Delay--- ---OUTR Swap Reason--- ---SUBS Delay--Page Swap OUTR

TI
TO
LW
XS JES HSM XCF
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Statistical error: 33%
Command ===>
GG24-4131
Demonstration 1-5

Demonstration 1 ...
Samples:
RMF 4.3.0 Processor Delays -- AQTS

100 Time: 03/31/93 at 15.10.00
Line 1 of 19
Range +
100 Sec
Jobname
DLY USG TCB+ VEC ----------- Holding Job(s) ----------C DMN PG % % SRB% Ratio % Name
% Name
% Name
IARHI
SCHIFFU
#SIMMONS
IARHD
IARHC
RBAKERAC
SCHUMAC1
IARHA
IARHT
TCHAND
UY81623
WASINATS
ALAN1@
##903010
##903007
KIRKIEC7
KIRKIEC6
##903011
WASINPM
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
B
2
2
2
2
2
***
***
2
***
***
***
***
***
***
1
2
2
***
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
63
58
53
51
41
35
29
25
23
12
11
8
7
4
4
3
3
2
1
23
30
47
16
17
14
12
12
5
3
5
5
1
4
3
1
1
5
0
25.2
30.6
52.1
19.0
15.3
11.2
11.6
9.9
9.0
3.3
6.5
4.7
1.9
2.5
1.1
1.0
1.0
2.9
0.0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
56
54
46
45
36
28
25
20
20
10
10
7
5
3
4
3
3
2
1
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
ROBK
IARHT
Report is for performance group 1 only.

Command ===>
GG24-4131
23
22
16
18
15
15
12
13
11
6
8
5
5
3
2
2
2
1
1
#SIMMONS
#SIMMONS
MINYANG
JEAHA
MINYANG
MINYANG
SCHIFFU
JEAHA
#SIMMONS
RHOWARD
MINYANG
ANN
ANN
CLIANG
KIM1
JEAHA
KIM1
RBAKERAC
SCHIFFU
22
18
16
17
14
15
12
9
9
5
7
3
3
3
2
2
1
1
1
MINYANG
MINYANG
SCHIFFU
MINYANG
#SIMMONS
SCHIFFU
MINYANG
CLIANG
ANN
WEISS
#SIMMONS
D10MRM1
JES2
H65JHF1
RHOWARD
D10MRM1
VTAM
H65JHF1
ANN
Scroll ===> CSR
Demonstration 1-6

Demonstration 1 ...
Samples:
Job: IARHI

100 Time: 03/31/93 at 15.10.00
Line 1 of 1
Range +
100 Sec
Primary delay: Job is waiting to use the processor.
Probable causes: 1) Job ROBK may be looping.

2) Higher priority work is using the system.
3) Improperly tuned dispatching priorities.
------------------------- Jobs Holding the Processor -------------------------Job:

ROBK
Job:
#SIMMONS
Job:
MINYANG
Holding:
56%
Holding:
23%
Holding:
22%
PROC Using:
88%
PROC Using:
47%
PROC Using:
34%
DEV Using:
0%
DEV Using:
0%
DEV Using:
1%
C ASID DMN PG % PROC DEV % % % PROC DEV STOR SUBS OPER ENQ Reason
B 0579 2 1 31 23 7 65 0 6 63
4
0
0
0
0 ROBK
Command ===>
GG24-4131
Scroll ===> CSR
Demonstration 1-7
Demonstration-2
Monitoring System Performance
Monitor System Performance

Customize Exceptions Criteria Sets
Identify Jobs and Resources with exceptional conditions
Reports
WFEX
STORC/STORCR
DELAY (Device/Enqueue)
GG24-4131
Demonstration 2-1

Demonstration 2

Line 1 of 23
Samples:
100 Time: 03/30/93 at 14.20.00 Range +
100 Sec
Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
10
51
STCs
53
2
42
ALL TSO
520
5
57
TSO short
510
0
81
ALL BATCH
3
3
42
TSO longer
3
2
63
ALL STC
97
3
49
TSO medium
0
0
67
ALL ASCH
Not avail
TSO long
3
2
44
ALL OMVS
Not avail
Btch short
2
2
37
*DMN000
45
1
85
Btch long
1
1
48
------------------------------ Exceptions ------------------------------------Name
Reason
D11DFF1T SUBS-HSM
JWHEEL
SUBS-HSM
CONSOLE
OPER-Message 43.0 % delay Awaiting reply to operator request 90.
CORVUS
DEV -HSML15
DRV021
Not avail
D54ECA
Not avail
D54ECB
Not avail
D54ECC
Not avail
D54ECD
Not avail
D54ECE
Not avail
D54ECF
Not avail
D54EC0
Not avail
D54EC1
Not avail
D54EC2
Not avail
D54EC3
Not avail
D54EC4
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
Demonstration 2-2

Demonstration 2 ...
RMF WFEX Report Options: Action Panel

Date ===> 03/30/93
Time ===> 14.20.00
Range ===> 100
Enter Action Code in the Action Column. To exit press END.

Action Codes: Select
(S)
Copy (C)
Move
(M)
Add
(AD)
Delete (D)
Move Block (MM)
Action
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
__
Command
GG24-4131
Line 1 of 37
Before (B)
After (A)
Class
Qualifier Indicator Label
Row Position
Only Add (AD) and After (A) are valid on this line.
SYSTEM
WF
*SYSTEM
1
1
TSO
WF
ALL TSO
1
2
BATCH
ALL
WF
ALL BATCH
1
3
STC
WF
ALL STC
1
4
ASCH
WF
ALL ASCH
1
5
OMVS
WF
ALL OMVS
1
6
DMN
0
WF
*DMN000
1
7
DMN
7
WF
STCs
2
1
DMN
3
WF
TSO short
2
2
DMN
6
WF
TSO longer
2
3
DMN
4
WF
TSO medium
2
4
DMN
5
WF
TSO long
2
5
DMN
1
WF
Btch short
2
6
DMN
2
WF
Btch long
2
7
PROC
EX-AVG
*PROC
STOR
EX-AVG
*STOR
STOR
EX-AVG
*STOR
STOR
EX-AVG
*STOR
STOR
EX-AVG
*STOR
STOR
EX-AVG
JOB
EX-UNAVAIL
===> ________________________________________________ Scroll ===> CSR
Demonstration 2-3

Demonstration 2 ...
Automatic Customization Defaults:
Class
Indicator
Setting for Yellow
Setting for Red
(1)
WF
WFL%<20 & ADU>2
WFL%<12 & ADU>2
(2)
WF
WFL%<30 & ADU>.2
WFL%<20 & ADU>.1
PROC
EX-AVG
STOR
EX-AVG
SQA%>5
STOR
EX-AVG
CSA%>2
STOR
EX-AVG
O N L F < (3)
STOR
EX-AVG
O N L X F < (4)
STOR
EX-AVG
STOR
EX-AVG
SCSA%>70
SCSA%>85
STOR
EX-AVG
SECS%>70
SECS%>85
JOB (5)
EX-UNAVAIL
Job not found
DEV (5)
EX-UNAVAIL
Device not found
DMN_w/_1
EX-ANY
WFL%<30 &DLY%>5
WFL%<20 & DLY%>5
DMN_w/_2
EX-ANY
WFL%<20 &DLY%>5
WFL%<10 & DLY%>5
DMN_1
EX-ANY
WFL%<30 &DLY%>5
DMN_2
EX-ANY
WFL%<10 &DLY%>5
AVAIL<#_of_cpus
TSQAO>0
WFL%<10 & DLY%>5
(1)
SYSTEM, BATCH, STC
(2)
TSO, ASCH, OMVS, DMNs
(3)
average number of frames online
(4)
average number of expanded frames online
(5)
line is displayed in turquoise
DMN_w/_1
Domain with DUR parameter in first half of list
DMN_w/_2
DMN_1
- - in second half of list

Domain without DUR parameter (1st period only)
DMN_2
- - (all other periods)
GG24-4131
Demonstration 2-4

Demonstration 2 ...
RMF WFEX Report Options: Definition and Criteria

Enter or edit information below. To view a list of criteria name values,
place the cursor in a blank Name field and press ENTER.
Exception will be displayed if all criteria in a set 1, set 2 or set 3 are met
Class
Qualifier
Indicator
Label
Alert
Text
===>
===>
===>
===>
===>
===>
Criteria set
Name <> Yel
AT___ >_ 25__
_____ __ ____
_____ __ ____
_____ __ ____
_____ __ ____
_____ __ ____
_____ __ ____
DMN___
For example: SYSTEM, BATCH, JOB, DEV, STC
1_______
For example: Jobname, volume serial, job class
EX-AVG____
WF, EX-ANY, EX-AVG, EX-GROUP or EX-UNAVAIL
__________
Label for workflow monitor or exception line
NONE_
Alerting signal: BLINK, BEEP, BOTH, NONE
______________________________________ Leave blank for defaul
1
Criteria set
Red
Name <> Yel
40__
_____ __ ____
____ or _____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
2
Criteria set
Red
Name <> Yel
____
_____ __ ____
____ or _____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
____
_____ __ ____
3
Red
____
____
____
____
____
____
____
Command ===> _________________________________________________________________
GG24-4131
Demonstration 2-5

Demonstration 2 ...
RMF WFEX Criteria Names for Class: DMN

Select (S) a maximum of 7 items to use in a criteria set. Press END.
_
_
_
_
S
_
_
_
_
_
_
_
_
_
_
_
_
_
AAU Avg active users

AAUS Total avg active
ADU Avg delayed users
ASTO% Active storage %
AT
ACT response time
AUU Avg using users
AVAIL No. jobs in group
COMM% Common stor dly %
CPU% TCB+SRB %
CPUS% CPU utilization %
CSA% CSA storage %
DEV% Device delay %
DLY% Overall delay %
ENQ% ENQ delay %
ESQO% ESQA overflow %
HSM% HSM delay %
ISTO% Idle storage %
JCSA% Job CSA use %
_ JECS% Job ECSA use %

_ JES% JES delay %
_ JESQ% Job ESQA use %
_ JSQA% Job SQA use %
_ LOCL% Local stor dly %
_ LPA% LPA storage %
_ MNT% MOUNT delay %
_ MSG% MSG delay %
_ ONLF Online RS frames
_ ONLXF Online EXP frames
_ OPER% Operator delay %
_ OUTR% Out/ready delay %
_ PROC% Processor delay %
_ QT
Queue time (ASCH)
_ RATE Transaction rate
_ RT
Response time tot
_ SCSA% System CSA use %
_ SECS% System ECSA use %
_ SESQ% System ESQA use

_ SQA% SQA storage %
_ SQAO% SQA overflow %
_ SSQA% System SQA use %
_ STOR% Storage delay %
_ SUBS% SUBS delay %
_ SWAP% Swap delay %
_ TET TRAN ELAP time
_ TSQAO Tot SQA overflow
_ USG% Overall using %
_ USGD% Device using %
_ USGP% Processor using
_ VIO% VIO storage dly
_ WFL% Workflow/Speed %
_ WFLG% Group workflow %
_ XCF% XCF delay %
Command ===> _________________________________________________________________
GG24-4131
Demonstration 2-6

Demonstration 2 ...

Line 1 of 25
Samples:
100 Time: 03/30/93 at 14.20.00 Range +
100 Sec
Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
10
51
STCs
53
2
42
ALL TSO
520
5
57
TSO short
510
0
81
ALL BATCH
3
3
42
TSO longer
3
2
63
ALL STC
97
3
49
TSO medium
0
0
67
Processors 139
4
78
TSO long
3
2
44
Devices
56
6
55
Btch short
2
2
37
*DMN000
45
1
85
Btch long
1
1
48
------------------------------ Exceptions ------------------------------------Name
Reason
D11DFF1T SUBS-HSM
JWHEEL
SUBS-HSM
*DMN001
AT > 25
25.52 sec
*SQA*
SSQA% > 80
88.2 %
CONSOLE
CORVUS
DEV -HSML15
DRV021
Not avail
D54ECA
Not avail
D54ECB
Not avail
D54ECC
Not avail
D54ECD
Not avail
D54ECE
Not avail
D54ECF
Not avail
D54EC5
Not avail
D54EC6
Not avail
D54EC7
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
Demonstration 2-7

Demonstration 2 ...
Demonstration 2 Analysis
Common Storage Usage
Enqueue Delays
Device Delays
GG24-4131
Demonstration 2-8

Demonstration 2 ...
Samples:
RMF 4.3.0 Common Storage -- AQTS

100 Time: 03/30/93 at 14.21.40
System Information
IPL Definitions
Peak Allocation Values
Average CSA to SQA Conversion
Average Use Summary
Available at End of Range
Job Information
Name
Act C DMN
%MVS
%REMAIN
JES2
S 7
*MASTER*
S 0
CATALOG
S 0
VTAM
S 7
TSO
S 0
DFHSM
S 7
DB2LMSTR
S 7
DB2LDIST
S 7
OMIIETE
S 7
RMF43
S 9
CRYPTO
S 7
DB2LIRLM
S 7
NETVIEW
S 7
NPM
S 0
AUTONET
S 7
FATH
H T ***
Command ===>
GG24-4131
ELAP
PG ASID Time
4 29
0
1
0 35
5 54
0 72
17 57
13 66
13 81
3 69
8 61
3 55
13 83
20 58
0 68
3 17
2 582
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.5D
6.7H
---- Percent ---CSA ECSA SQA ESQA

25
0
24
76
61 88
0
61 88
39 12
63
63
37
-- Percent Used CSA ECSA SQA ESQA

5
2 53 55
2
1 0
0
1 29 0
0
3
7 12
5
2
0 9
0
1
9 0
0
3
0 0
0
0
0 3
0
1
2 0
0
0
2 0
0
0
0 1
0
0
0 1
1
1
0 0
0
1
0 0
0
0
0 1
0
0
1 1
0
0
1 1
0
0
0 0
0
Line 1 of 654
Range +
100 Sec
------- Amount -------CSA ECSA SQA ESQA

3832K 28M 1364K 49M
940K 17M 1204K 31M
0
0
936K 17M 1203K 31M
2898K 11M 162K 18M
----CSA
197K
80928
20592
119K
65152
30652
130K
8427
30152
128
120
0
44424
44320
4864
16840
3464
106
Amount Used ----ECSA SQA ESQA

544K 722K 27M
371K 1168 67416
8101K
48 464
2131K 158K 2299K
67208 125K 198K
2500K
0 352
2120
0 224
704 38616 503
663K 528 4528
502K
48 2744
96048 18480 14496
2952 8216 635K
424 392 224
7992 128 288
75405 11240 480
200K 7968 480
198K 9560 368
1760 440 206
Scroll ===> CSR
Demonstration 2-9

Demonstration 2 ...
RMF 4.3.0 Common Storage Remaining -- AQTS

Line 1 of 109
Samples:
100 Time: 03/30/93 at 14.21.40 Range +
100 Sec
Jobname
ID
FATH
TSU15833
TNWILLI TSU09636
LALITHA TSU26463
LALITHA TSU26356
LALITHA TSU20770
RONDA
TSU17398
BEVK
TSU15161
D24PAE1 TSU11662
BAISE
TSU12386
ALTENBE TSU06082
ALTENBE TSU05989
D81HMY1 TSU03619
FARRELL TSU03860
D10LLC1 TSU10469
WALDMARI JOB26251
WNC
TSU10641
LALITHA TSU23462
TCP
STC20125
##CSFPK1 JOB17489
##DB2LD0 JOB17466
##DB2LS1 JOB17470
##DB2MD1 JOB17478
RACFCMQT JOB20194
NETSADMX JOB20266
Command ===>
GG24-4131
Job Ended
Date
Time
03/30/93
03/29/93
03/29/93
03/29/93
03/29/93
03/29/93
03/28/93
03/28/93
03/28/93
03/27/93
03/27/93
03/27/93
03/26/93
03/26/93
03/25/93
03/25/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
03/24/93
14.23.05
20.02.09
11.58.32
11.54.38
11.47.47
08.25.49
21.06.03
15.26.12
14.53.01
16.00.32
15.56.16
11.39.24
14.09.28
10.46.52
17.43.32
08.26.51
08.58.43
02.48.17
03.48.07
03.44.50
03.43.19
03.41.35
03.03.32
03.03.31
Amount of Common Storage

Not Released at End of Job
CSA ECSA SQA ESQA
128
288
288
288
288
288
288
288
288
288
288
288
288
288
288
288
288
0
0
0
0
0
0
0
2288
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1808
1424
1424
1424
1424
1424
1424
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Scroll ===> CSR
Demonstration 2-10

Demonstration 2 ...

Line 1 of 32
Samples:
100 Time: 03/30/93 at 14.23.20 Range +
100 Sec
Name
Users Active
Speed
Name
Users Active
Speed
*SYSTEM
620
16
35
STCs
53
2
47
ALL TSO
520
5
51
TSO short
509
0
73
ALL BATCH
10
9
18
TSO longer
2
2
61
ALL STC
90
3
52
TSO medium
1
0
39
Processors 138
3
74
TSO long
3
2
41
Devices
59
8
49
Btch short
6
6
9
*DMN000
39
1
78
Btch long
4
3
33
------------------------------ Exceptions ------------------------------------Name
Reason
*DMN001
AT > 40
47.38 sec
Btch short ENQ -SYSDSN
4.0 users SLM.ZRAALLEN.LOADLIB
CORVUS
DEV -HSM013
CORVUS
DEV -HSM013
D1CI2PVX ENQ -SYSDSN
D1CL2FDF ENQ -SYSDSN 100.0 % delay SLM.ZRAALLEN.LOADLIB
D1LI1LPA ENQ -SYSDSN 100.0 % delay SLM.ZRAALLEN.LOADLIB
D1LI1LPX ENQ -SYSDSN 100.0 % delay SLM.ZRAALLEN.LOADLIB
D11DFF1T SUBS-HSM
GREAGOR
DEV -HSM013
*SQA*
SSQA% > 80
88.0 %
ALL BATCH ENQ -SYSDSN
4.0 users SLM.ZRAALLEN.LOADLIB
CONSOLE
DRV021
Not avail
D54ECA
Not avail
D54ECB
Not avail
Command ===>
Scroll ===> CSR
GG24-4131
Demonstration 2-11

Demonstration 2 ...
Sequences to Analyze Enqueue Delays:
WFEX Field
Invoked Report
DMN001
GROUP (analyze components of active response time)

Continue with:
ENQ
Btch short
ALL BATCH
ENQ
-SYSDSN
DELAY (overview of all delays related to this groups)

Continue with
JOB
through Jobname, % Delayed for
ENQR
through Primary Reason
ENQ (Relation: Job delayed - Resource - Job(s) holding)

Continue with:
JOB
through Jobname, Holding
ENQR
through Resource Waiting
ENQR (Relation: Resource - Job(s) delayed - Job(s) holding)

Continue with:
JOB
D1CI2PVX
D1CL2FDF
...
GG24-4131
ENQ column
through Delayed, Holding
JOB (up to 3 jobs found holding the resource)

Continue with:
.JOB
via Job
ENQR
through Primary Reason
Demonstration 2-12

Demonstration 2 ...
Samples:
Job: CORVUS

100 Time: 03/30/93 at 14.23.20
Line 1 of 4
Range +
100 Sec
Primary delay: Excessive disconnect time on volume HSM013.
Probable causes: 1) Sequential access of data with short blocksize.

2) Active datasets spaced widely across volume.
3) Overloaded channel paths causing reconnect delays.
-------------------------- Volume HSM013 Device Data -------------------------Number:

06C
Active:
68%
Pending:
2%
Average Users
Device:
33903
Connect:
4%
Delay DB: 1%
Delayed
Shared:
Yes
Disconnect: 62%
Delay CU: 0%
0.8
Delay DP: 0%
C ASID DMN PG % PROC DEV % % % PROC DEV STOR SUBS OPER ENQ Reason
T 0137 *** 2 7
1 4 70 21 5
0 70
0
0
0
0 HSM013
4 2 15
1 1 11 0 0
0 11
0
0
0
0 HSM013
3 2
0 0 0 21 0
0
0
0
0
0
0
5 2 5
0 3 59 0 5
0 59
0
0
0
0 HSM013
Command ===>
GG24-4131
Scroll ===> CSR
Demonstration 2-13

Demonstration 2 ...
RMF 4.3.0 Device Resource Delays -- AQTS

Line 1 of 5
Samples:
100 Time: 03/30/93 at 14.23.20 Range +
100 Sec
Volume
Act Resp ACT CON DSC PND %, DEV/CU
/Num S Rate Time % % % Reasons Type
USG DLY
Jobname C DMN PG % %
HSM013 S
06C
CORVUS
GREAGOR
DURAND
MJORDAN
FIGURA
9.1 .092
68
4 62 PND
DB
2 33903
1 3990-3
Command ===>
GG24-4131
T
T
T
T
T
***
***
***
***
***
2
2
2
2
2
3 61
0 9
0 4
0 3
1 0
Scroll ===> CSR
Demonstration 2-14
Additional Features
Tutorial/Help
Print Reports
Modify Reports
GG24-4131
Additionnal Features-1

Additional Features
Options
Session Initialization Options (including: output data set for

Hardcopy and input data set containing the IPS member)
Color Options (Graphic colors and patterns)
Language Options (Date/Time format)
Report Options
Option Sets (customization results, color, and language

options)
Tutorial and Help

Hardcopy and ICU
RMF Utility
SMF Records
GG24-4131

Each report can be changed
Content is determined by options and format tables
Layout can be modified via change/replacement of panels
User exits available:
Reporter data exists in ISPF tables
Modification using ISPF services
Creation of own reports
Report format definition utility ERBRMFU for modification of

existing and creation of new reports
GG24-4131

Phase 1
ERB3RHP1 Input
Proc.
updates phase driver table and

creates ISPF table for report
Output
name of table in var.

PHDRTAB1
Input
variable PHDRTAB1
Proc.
creates new ISPF table
Output
name of new table in

PHDRTAB2
Build ISPF table
Phase 2
N/A
Modify tables
Phase 3
ERB3RDSP Input
Display tables
Phase 4
N/A
PHDRTAB2, ERBFMTS3,
ERBPTGS3
Proc.
creates of new ISPF table
Output
report defined by var. ERBREPC
Input
user defined variables (phase 2)
Proc.
cleans up resources allocated

by phase 2
Output
none
Clean up
GG24-4131
Address of Set-of-Samples

Reporter Control Tables
Phase driver table ERBPHDS3
defines function (process) to be performed in each phase
describes names of data tables created in phase 1 and 2
contains an entry for each report type
Tabular report format table ERBFMTS3
defines layout of each RMF report (display and hardcopy)
contains related panel and help panel name
describes in detail header,sub-header lines and scrollable data
Parameter table for graphic reports ERBPTGS3
defines layout of each graphic RMF report
contains associated help panel name
Example: ERBPHDS3 entry for SYSINFO report

Variable
Content
PHDREPNA
SYSINFO
PHDREPSE
11
PHDRPH1
PGM(ERB3RHP1) PARM(SYSINFO)
PHDRPH2
null
PHDRPH3
PGM(ERB3RDRS)
PHDRPH4
null
PHDRTAB1
ERBSYST3
PHDRTAB2
ERBSYST3
GG24-4131

RMF Utility
RMF Utility Dialog
Dialog Step
Requested Information
RMF Utility Primary Panel

Action
Action: Create, Modify, or Delete

Report Name
Prototype Name (for a new report)
Control Table Modifications

Phase Driver Definition
Select String (Program or Command File) for each phase

Table Names for phase 1 and 2.
Report Format Definition
Tabular and/or Graphic Report

Name of Report Panel
Name of Help Panel
Variable names of line and sequence number to access the
data
Tabular Parameter Definition

Report
Report
Report
Report
Header layout
Subheader layout
Column layout
Cmmndline layout
Definition of text and variable positions on screen

Assignment of ISPF variable substitutions
Graphic Parameter Definition

Layout
Title of Y-Axis, Minimum Axis Range, Selection Rule
Label Defintions
Bar Labels for up to 2 bar types
Column Specification
Assignement of ISPF variable substitution
Verification Step
Confirmation
GG24-4131
SAVE or CANCEL
Display modified report

Example of User Report: System Trend

-- AQTS
Line 1 of 20
Samples(AVG):
100 (Tot): 2000 Range: 00.33.20 /Line:
100 Sec
Startime: 03/30/93 at 14.30.00
Endtime: 03/30/93 at 13.58.20
Time
T143000
T142820
T142640
T142500
T142320
T142140
T142000
T141820
T141640
T141500
T141320
T141140
T141000
T140820
T140640
T140500
T140320
T140140
T140000
CPU SRB
%
%
39
53
50
35
35
39
40
41
49
54
61
69
59
64
52
48
58
54
63
2.5
2.5
2.5
2.2
2.2
2.2
2.2
1.9
2.2
2.2
2.0
2.7
2.2
2.5
1.9
2.0
1.9
2.1
2.6
Range/Line:
Total Range:
100 Sec
2000 Sec
TCB
%
WFL -Users-- -AVG USG- -Avg. Act. Users Delayed For% TOT ACT PROC DEV PROC DEV STOR SUBS OPER ENQ
31.0
41.9
39.8
26.5
26.3
30.7
32.1
33.8
39.9
45.4
52.2
57.2
50.7
54.1
44.2
39.7
50.6
44.4
50.8
45
52
52
38
35
39
51
52
53
54
48
59
57
62
62
67
67
57
50
623
623
623
621
620
620
620
620
620
623
623
623
623
623
622
619
618
620
619
11
13
14
15
16
16
10
9
11
12
12
12
11
10
8
9
9
12
14
2.5
3.6
3.8
2.0
2.2
2.6
2.9
2.5
3.2
3.4
3.8
4.3
3.8
4.0
3.4
3.2
3.6
3.5
4.0
2.8
4.4
4.6
4.2
4.2
4.5
3.6
3.0
3.4
3.6
2.7
3.9
3.4
3.4
2.5
3.6
3.3
4.4
4.5
1.5
1.0
1.8
1.1
0.8
0.8
0.8
0.6
1.0
0.8
1.0
1.8
1.1
1.1
1.0
0.8
0.7
0.7
2.3
Command ===>
GG24-4131
00.33.20
2.4
4.2
4.8
4.5
4.4
3.3
3.0
1.7
2.9
2.5
2.1
2.6
2.7
1.7
1.2
1.1
1.5
2.0
2.9
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1.4
0.9
0.1
1.1
1.5
1.6
0.9
1.7
0.8
1.2
1.6
0.7
0.7
0.4
0.1
0.2
0.7
2.1
2.0
1.3
1.2
0.7
0.6
1.4
0.2
1.5
1.1
0.7
0.4
1.5
0.3
0.9
1.1
1.3
1.2
0.5
1.2
1.2
0.0
0.0
0.3
3.0
4.0
5.0
0.0
0.0
0.5
0.9
1.0
0.2
0.0
0.2
0.0
0.0
0.0
0.0
0.0
Scroll ===> CSR

Example of User Report: System Trend ...

-- AQTS
Line 1 of 20
Samples(AVG):
100 (Tot): 2000 Range: 00.33.20 /Line:
100 Sec
GG24-4131

Summary
RMF Monitor III to analyze performance degradations

RMF Monitor III to monitor the system
Concepts (Delay - Reason - Cause)
Delayed Job(s) - for Resource - Holding Job(s)
Resource - Competing Jobs
Features:
Go Mode, Cursor Sensitivity, Automatic Customization, Helps,

Tutorials, ...
Enhancements:
GG24-4131
Use the RMF Utility to define own reports
Specifications
Customized CLIST
Panels Samples
GG24-4131
Specifications-1

Using the RMF Utility
Invocation
ERBRMFU (from TSO READY line or ISPF

command line)
Dialog
Provides 14 panels to define the new report

to RMF
Steps
The panels are grouped into 6 logical steps

1. Startup/Action panel
2. Phase Driver table information (saved in
ERBPHDS3)
3. Specify the desired report types and the
variable names for the line and sequence
number (necessary to access the table data)
4. Tabular report layout (saved in ERBFMTS3)
5. Graphic report layout (saved in ERBPTGS3)
6. Verification (Save to confirm changes/Cancel
to discard changes)
Navigation
ENTER to continue with the next panel

PF3 to go back to the last panel
GG24-4131
Specifications-2

Step 1: RMF Utility Startup Panel

To exit enter CANCEL or press the END key.
ACTION
===> MODIFY
MODIFY, CREATE or DELETE
or CREATE, see WARNING
REPORT NAME
===> USGTREND
PROTOTYPE NAME ===> SYSINFO
Name of report
Name of existing report to be

used as a prototype (optional)
Command ===> ____________________________________________________
GG24-4131
Specifications-3

Step 2: Phase Driver Definition

Section 1: Phase Driver Information

Select Strings format is:PGM(nnnnnnnn) PARM(mmm) or CMD(nnnnnnnn mmm)
SELECTION CHARACTERS ===> USGTREND
Selection on Primary Option Panel
PHASE 1 SELECT STRING ===> CMD(USGTREND)

TABLE NAME ===> USGTREND Name of reporter phase 1 result table
TABLE NAME ===> USGTREND
Optional name of phase 2 result table
PHASE 3 SELECT STRING ===> PGM(ERB3RDSP)

Command ===> ____________________________________________________
GG24-4131
Specifications-4

Step 3: Report Format Definition

Section 2: Report Format Information

REPORT MODE
===> BOTH
TABULAR, GRAPHIC or BOTH
PANEL NAME
===> ERB3SYS
Name of tabular report panel
HELP PANEL NAME
===> ERB3SYS0
Name of HELP panel
LOGICAL LINE NUMBER ===> USGLOGLN

SEQUENCE NUMBER
===> USGSEQUN

Command ===> ____________________________________________________
GG24-4131
Specifications-5

Step 4: Tabular Report Parameters

Enter or change the report header lines. To continue press ENTER.

The following
&ERBSID
&ERBSAMPL
&ERBTIME
variables are available for use in the header:

&ERBHCTXT
&ERBRMFVD
&ERBDATE
&ERBRANGE
Enter or change up to two report heading lines:

-- &Z_
&ERBHCTXT Samples(AVG): &Z_
(Tot): &Z_
Range: &USGRGTIM /Line:
Sec
Command ===> ____________________________________________________
GG24-4131
Specifications-6


The following report header lines have been specified:

-- Z1
&ERBHCTXT Samples(AVG): Z2
(Tot): Z3
Range: &USGRGTIM /Line: Z4
Sec
above.To continue press ENTER.To go backwards press END.To quit enter CANCEL.
&ERBSID
&ERBDATE
&ERBRMFVD
===> _________
===>
===>
===>
===>
1
__
__
__
&ERBHCTXT ===> __
&ERBTIME ===> __
&USGSAMPL ===> 2
===> _________ ===> __
&ERBSAMPL
&ERBRANGE
&USGSMPLT
===> _________
===>
===>
===>
===>
__
4
3
__
Command ===> ____________________________________________________
GG24-4131
Specifications-7


Enter or change the report subheader lines. To continue press ENTER.

The following variables are available for use in the subheader:
Enter or change up to five report subheading lines:

Startime: &USGDATE at &USGTIME Range/Line: &Z_
Sec
Endtime: &ERBDATE at &ERBTIME Total Range: &Z_
Sec &USGRGTIM
_______________________________________________________________________________
_______________________________________________________________________________
_______________________________________________________________________________
Command ===> ____________________________________________________
GG24-4131
Specifications-8


The following report subheader lines have been specified:

Startime: &USGDATE at &USGTIME Range/Line: Z1
Sec
Endtime: &ERBDATE at &ERBTIME Total Range: Z2
Sec
&USGRGTIM
===>
===>
===>
===>
===>
===>
===>
&ERBRANGE
_________
_________
_________
_________
_________
_________
_________
===>
===>
===>
===>
===>
===>
===>
===>
1
__
__
__
__
__
__
__
===>
===>
===>
===>
===>
===>
===>
&USGRANGE ===> 2
_________ ===> __
_________ ===> __
_________ ===> __
_________ ===> __
_________ ===> __
_________ ===> __
_________ ===> __
===>
===>
===>
===>
===>
===>
===>
===>
_________
_________
_________
_________
_________
_________
_________
_________
===>
===>
===>
===>
===>
===>
===>
===>
__
__
__
__
__
__
__
__
Command ===> ____________________________________________________
GG24-4131
Specifications-9


Line 1 of 1

DATA ATTRIBUTE CHARACTERS ===> _?|
Define meaning in attribute section

of associated table display (ERB3SYS).
Enter or change up to three column header lines:

CPU SRB TCB WFL -Users-- -AVG USG- -Avg. Act. Users Delayed ForTime
%
%
%
% TOT ACT PROC DEV PROC DEV STOR SUBS OPER ENQ
_______________________________________________________________________________
Enter or change up to three model lines:
?Z
?Z ?Z
?Z
?Z ?Z ?Z ?Z ?Z
?Z ?Z ?Z ?Z ?Z ?Z
_______________________________________________________________________________
_______________________________________________________________________________
The following variables are available for use in the model lines:
Command ===> ____________________________________________________
GG24-4131
Specifications-10


Line 1 of 8
The following report column header and model lines have been specified:
CPU SRB TCB WFL -Users-- -AVG USG- -Avg. Act. Users Delayed ForTime
%
%
%
% TOT ACT PROC DEV PROC DEV STOR SUBS OPER ENQ
Z1
Z2 Z3
Z4
Z5 Z6
Z7
Z8
Z9
Z10 Z11 Z12 Z13 Z14 Z15
USGNAMVC
USGTCBVC
USGAUSVC
USGADPVC
USGADUVC
===> ________
===> ________
===> ________
===>
===>
===>
===>
===>
===>
===>
===>
1
4
7
10
13
__
__
__
USGCUVVC
USGWFLVC
USGAUPVC
USGADDVC
USGADOVC
===> ________
===> ________
===> ________
===>
===>
===>
===>
===>
===>
===>
===>
2
5
8
11
14
__
__
__
USGSRBVC
USGTUSVC
USGAUDVC
USGADSVC
USGADEVC
===> ________
===> ________
===> ________
===>
===>
===>
===>
===>
===>
===>
===>
3
6
9
12
15
__
__
__
Command ===> ____________________________________________________
GG24-4131
Specifications-11


Section 6: Command Line Layout

The following variables are available for use in the command line:
&ZCMD
&AMT
Enter or change the command line:
Command ===> &zcmd
Scroll ===> &AMT
Specify a variable name in each of the entry fields to replace each Z above.
Z1 ===>
Z2 ===>
Z3 ===>
Command ===> ____________________________________________________
GG24-4131
Specifications-12

Step 5: Graphic Report Parameters


GENERAL INFORMATION
NAME FOR HELP PANEL ===> ________

TITLE FOR Y-AXIS
===> Average Number of Active Users
MINIMUM AXIS RANGE ===> 1

SELECTION RULE
Name of HELP PANEL, if any
===> 2
Axis will contain at least this

number of data points
Specify 0, 1, 2 or 3
Command ===> ____________________________________________________
GG24-4131
Specifications-13



LABEL INFORMATION FOR BAR TYPE I

PRIMARY LABEL
SECONDARY LABEL
PRIMARY COMPOSITE
SECONDARY COMPOSITE
===>
===>
===>
===>
USGNAMVC
________
_____
_____

Prefix of label
Prefix of label
LABEL INFORMATION FOR BAR TYPE II

PRIMARY LABEL
SECONDARY LABEL
PRIMARY COMPOSITE
SECONDARY COMPOSITE
===>
===>
===>
===>
________
________
_____
_____

Prefix of label
Prefix of label
Command ===> ____________________________________________________
GG24-4131
Specifications-14



COLUMN SPECIFICATION FOR GRAPHIC BAR TYPES

NAME
1. ===> USGADPVC
2. ===> USGADDVC
3. ===> USGADSVC
4. ===> USGADUVC
5. ===> USGADOVC
6. ===> USGADEVC
7. ===> USGAUPVC
8. ===> USGAUDVC
9. ===> ________
10. ===> ________
LEGEND ID
===> 14
===> 08
===> 15
===> 28
===> 29
===> 09
===> 19
===> 18
===> __
===> __
TRANS ID
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
0
0
0
0
0
0
0
0
_
_
BAR TYPE ID
===>
===>
===>
===>
===>
===>
===>
===>
===>
===>
1
1
1
1
1
1
1
1
_
_
Command ===> ____________________________________________________
GG24-4131
Specifications-15

Step 6: Verification Step
This is a confirmation/cancellation panel for report: USGTREND
The following actions are allowed:
Type SAVE
command to save report
Type CANCEL command to cancel processing

Press END
key to go one step backwards
Press ENTER key to see the sample report
Command ===> ____________________________________________________
GG24-4131
Specifications-16
150
Index
A
A A U 19
allocate 2, 4
analyze reasons for delays 9
APAR OY57005 46
ASCH 51, 52
automatic customization 45, 47
average active users 19, 37
ECSA 50, 55
ENQ 26, 56, 57
ENQR 57
Enqueue Delay 56
ERB3PRM 64
ERBFMST3 63
ERBFMTS3 63
ERBPHDS3 63
ERBPLIB 4
ERBPTGS3 63
ERBRMF3X clist 4
ERBRMFU 63
ESQA 55
exact count technique 15
exception criteria 48, 50
exceptions 49
B
basic concepts 9, 21
BREF 27, 34, 45
C
CNSTR 47
command 23, 25, 26
commands 13, 27
Common Metrics 19
Common Storage 55
Common Storage Remaining report
Common Storage report 43
Criteria 50
Criteria panel 49
criteria specification 56
CSA 50, 55
Cursor Sensitivity 29
Customization 5, 6, 52, 61
cycle 17
cycle time 15, 21
F
FREF
D
Data Gatherer 13
Data Gathering 21
Data Reporter 13
Data Set Index 23
delay 17, 24, 25, 26, 28, 29, 35, 38, 39, 40, 57, 65
delay percentage 22
delay samples 19
delays 28, 31, 32
determine actions 11
DEV 26
Device and Enqueue Delay report 43
Device Resource report 58
DEVR 59, 64
DI option 4
DLY%) and 21
domain 24, 36
domains 23, 34, 47
DSINDEX 25, 28
DUR 47, 48
27, 52, 56, 66
55
G
gatherer 63
GO Mode 52
graphic 24, 25, 32, 34, 35, 44, 45, 61
group 50
group of jobs 23
Group report 35
group response time 62
Group Response Time report 31, 33, 35, 36, 50, 57
group-oriented 20
GROUPS 23, 32, 35
groups of jobs 11
H
HARDCOPY 61
HELP 41, 59, 61
HSM 26
I
ICS 34
ICU 61
identify 11
idle 17, 21
IEAIPS00 5, 6, 7
Import 3
in-storage buffer 28
individual jobs 11
Input Data Set 44
151
IOS 64
IPS 1, 5, 6, 34, 41, 47
ISPEXEC 66
ISPF 23, 61, 63
ISPPLIB 66
J
JES 26
job groups 45
Job report 31, 40
Job reports 33
Job States 17
JOBS 23
M
mintime 15, 20, 21, 22, 52
Mintimes 66
Mode 32, 44
monitor system activity 9
MVS JCL 1
N
not-delayed 17
not-using 17
O
OMVS 6, 45, 47, 51, 52
OpenEdition 62
OPT 34
option 43
option sets 61
Options 26, 51
OPTSET 7
o v e r v i e w 9, 24
P
panel-driven dialog 63
performance group 24, 36, 39
performance groups 23, 26, 34
Phase Concept 63
Postprocessor 62
pre-allocated data sets 28, 52
Primary 5, 66
primary cause 11
Primary Menu 1, 23, 29, 32, 44, 64
Primary panel 6
problem determination 9
PROC 26, 38
Processor Delay 33
Processor Delay report 31, 37, 38
R
Range 32, 34, 37, 44, 56, 66
real-time monitoring 28
Report Options 26
reporter 9, 23
Reporting 21
response time 9, 31, 33, 34, 35, 36, 37, 57
REXX CLIST 1
RMF Utility 63, 64, 67
RMF Utility Dialog 64
RMFDS00 4
RMFWDM 23
RO 26
ROPTIONS 26
S
sampling 15
sanity check 37
SECS% 50
selector 23
sequential data sets 1
SESSION 5, 6, 32, 44
SESSION option 5, 32, 44
Session Options 47
set-of-samples 21
SLIP 48
SMF 61
speed at 19
speedometers 52
SQA 50, 52, 55
SRM 36
SRM response time 36
S S Q A % 50, 55
Start RMF 6
STOR 26, 50
STORAGE 23
STORC 25, 26, 55
STORCR 25, 55
STORF 26
STORS 26
SUBS 23
SY DMN 27
SYSEXEC 3
SYSINFO 23, 24, 25, 26, 28, 34, 64, 65
Sysinfo report 41
SYSPROC 3, 66
system counters 15
system events 15
System Information 24
System Information report 31, 32
System Trend 65
System Trend User report 1
T
tabular
152
24, 29, 34, 35, 45
time range 21
TOGGLE command
Toggle key 25
TSO 23, 63
25
U
unknown 17
User Report 65
Using 13, 15, 16, 17, 19, 21, 22, 25, 28, 35, 38, 40, 65
using samples 19
V
VSAM data sets 4, 13
VSAM Monitor III data sets
W
WF 23
WFEX 7, 23, 28, 45, 47, 50, 51, 52, 56, 58
workflow 19, 21, 22, 24, 28, 45, 47, 49, 51, 53, 56
Workflow/Exception 28
Workflow/Exception report 20, 23
Workflow/Exceptions report 45
X
XCF
26, 62
Index
153
ITSO Technical Bulletin Evaluation
RED000
Using Resource Measurement Facility Monitor III Efficiently

Publication No. GG24-4131-00
Your feedback is very important to help us maintain the quality of ITSO Bulletins. Please fill out this
questionnaire and return it using one of the following methods:
Mail it to the address on the back (postage paid in U.S. only)

Give it to an IBM marketing representative for mailing
Fax it to: Your International Access Code + 1 914 432 8246
Send a note to REDBOOK@VNET.IBM.COM
Please rate on a scale of 1 to 5 the subjects below.

(1 = very good, 2 = good, 3 = average, 4 = poor, 5 = very poor)
Overall Satisfaction
____
Organization of the book

Accuracy of the information
Relevance of the information
Completeness of the information
Value of illustrations
____
____
____
____
____
Grammar/punctuation/spelling
Ease of reading and understanding
Ease of finding information
Level of technical detail
Print quality
____
____
____
____
____
Please answer the following questions:

a)
If you are an employee of IBM or its subsidiaries:

Do you provide billable services for 20% or more of your time?
Yes ____ No ____
Are you in a Services Organization?
Yes ____ No ____
b)
Are you working in the USA?
Yes ____ No ____
c)
Was the Bulletin published in time for your needs?
Yes ____ No ____
d)
Did this Bulletin meet your needs?
Yes ____ No ____
If no, please explain:
What other topics would you like to see in this Bulletin?
What other Technical Bulletins would you like to see published?
Comments/Suggestions:
Name
Company or Organization
Phone No.
( THANK YOU FOR YOUR FEEDBACK! )
Address
ITSO Technical Bulletin Evaluation

GG24-4131-00
Fold and Tape
RED000
Please do not staple
IBM
Fold and Tape
NO POSTAGE
NECESSARY
IF MAILED IN THE
UNITED STATES
BUSINESS REPLY MAIL

FIRST CLASS MAIL
PERMIT NO. 40
ARMONK, NEW YORK
POSTAGE WILL BE PAID BY ADDRESSEE
IBM International Technical Support Organization

Mail Station P402
522 SOUTH ROAD
POUGHKEEPSIE NY
USA 12601-5400
Fold and Tape
GG24-4131-00
Please do not staple
Fold and Tape
IBM
Printed in U.S.A.
GG24-4131-00

GG 244131

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

GG 244131

Загружено:

Авторское право:

Доступные форматы

Using Resource Measurement Facility Monitor III Efficiently

Document Number GG24-4131-00

International Technical Support Organization

First Edition (November 1993)

Copyright IBM Corp. 1993

RMF MonIII Demonstration

Part 1. How to Use the Diskette

Chapter 1. RMF Monitor III Basics

Chapter 2. Demonstration 1 - Analyzing System Workloads

Chapter 3. Demonstration 2 - Monitoring System Performance

Copyright IBM Corp. 1993

Chapter 6. Installation Performance Specification

Chapter 5. Specifications for System Trend Report

RMF MonIII Demonstration

Copyright IBM Corp. 1993

RMF MonIII Demonstration

How This Document is Organized

Part 2, Text for Demonstration provides an overview of RMF Monitor III

Part 3, Specifications provides additionnal and more technical

Resource Measurement Facility Version 4 Users Guide, GC28-1058

Resource Measurement Facility Version 4 Release 3.0 Technical Overview,

Resource Measurement Facility Getting Started on Performance

Copyright IBM Corp. 1993

RMF MonIII Demonstration

Part 1. How to Use the Diskette

About four available cylinders on a diskpack.

A PS/2, a PC, or the equivalent with DOS or OS/2 to transfer the

Two files ERB3PRM.PAN, and ERB3USR.PAN, which are, respectively, a

File IEAIPS00.PLB which is an IEAIPS parmlib member that should be used

Unloading the Files

SEND a:\RMFCC98.SEQ x:RMFCC98.SEQ LRECL (32752)

SEND a:\RMFCC99.SEQ x:RMFCC99.SEQ LRECL (32752)

(262016 bytes to transfer)

SEND a:\SQTOVS.JCL x:dsname(SQTOVS) ASCII CRLF

SEND a:\VSDEF.JCL x:dsname(VSDEF) ASCII CRLF

SEND a:\IEAIPS00.PLB x:parname(IEAIPS00) ASCII CRLF

SEND a:\ERB3PRM.PAN x:panname(ERB3PRM) ASCII CRLF

SEND a:\ERB3USR.PAN x:panname(ERB3USR) ASCII CRLF

SEND a:\USGTREND.CLI x:cdsname(USGTREND) ASCII CRLF

a: is the diskette reader of your PC or PS.

x: is the window of your MVS system on your PC or PS.

dsname is the name of an existing partitionned data set to receive the

parname is the name of an existing partionned data set to receive the

panname is the name of an existing partitionned data set to receive

cdsname is the name of an existing partitionned data set to receive a

RMFCC98.SEQ (idem for RMFCC99.SEQ) is the name of a sequential

RMF MonIII Demonstration

The cdsname is put into your SYSEXEC or your SYSPROC concatenation.

Part 1. How to Use the Diskette

ALLOC FI(RMFDS00) DA(userx.RMFCC99) SHR

On your MVS system, log on to a Monitor III session reporting; in TSO:

You must have the menu as displayed in Figure 1.

RMF Monitor III Primary Menu -- Demo

Figure 1. Demonstration Primary Menu

RMF MonIII Demonstration

Choose the DI option, typing DI on the command line. An equivalent

RMF 4.3.0 Data Set Index -- AQTS

---------------Data Set Name--------

Graphic version of this report is not available.

Figure 2. Monitor III Used Data Set

7. Set up the options.

Part 1. How to Use the Diskette

RMF Session Options