Академический Документы
Профессиональный Документы
Культура Документы
Lawrence To
Roderick Manalac
Center of Expertise
Worldwide Customer Support
Oracle Corporation
One of the major architectural changes introduced in Oracle 7 is the shared pool component of the SGA. It came along with
little documentation about taking proper care of it and only had one tunable parameter directly associated with it. As a result,
most people at best only have a murky view of what is really going on beneath the surface. Every so often they will see
creatures lurking around down there, and get bitten with an ORA-4031 or memory leak error. In the interim, pieces of
advice on how to avoid drowning have been offered. This paper will attempt to centralize those pieces of information. In
addition, we wish to shed some light on the inner workings of this structure and show how it relates to the rest of the
mechanisms involved in processing user requests.
For purposes of simplification we have deferred detailed discussion of how the shared pool relates to Multithreaded Servers
(MTS) and the Parallel Query Option (PQO) to a future date. We also do not offer any radical new advice on how to perform
capacity planning of the shared pool in this edition of the paper.
This paper will start with a high level view of memory management and work its way down to the actual data structures that
reside within the Oracle shared pool. It is sort of like a National Geographic special with Jacques Cousteau taking his ship
Calypso from the surface of the ocean down to the coral reefs below and discovering an whole new ecosystem. We hope you
enjoy the adventure with us.
This paper has been created as an instructional outline for members of support. It comprises of a collection of work from
different developers and can be used as a resource in understanding and debugging heap managment and library cache
problems.
History
Before Oracle7, every Oracle session had a unique process or server associated with it. This server was responsible for
parsing and optimizing all SQL and PL/SQL submitted by the client. Typically in OLTP environments, these statements
written by the end user in an ad-hoc manner; rather they were hard-coded or generated by an application. As the number of
users of a particular application grew, Oracle became aware that the servers were duplicating much of their efforts. Oracle
believed that some time and resources could be saved if output of these efforts could be shared. Initially, Oracle examined
how PL/SQL (especially stored PL/SQL - a new feature in Oracle7) could be better shared but later widened the scope to
include other objects such as cursors. The desired result was that space would be saved by having one copy of a stored
PL/SQL object available for all to share. Also, time would be saved if an execution plan and other pieces of information
were made available to all servers executing the same SQL.
Note: the concept was expanded further to allow a large number of sessions to be serviced by a small number of servers.
This requires having an initial area to store objects that can be shared, devising a way of managing those objects, and having
a mechanism to allocate and deallocate memory for those objects. Oracle accomplished these objectives by revising three
things:
1. The variable portion of the SGA was expanded
2. A method to manipulate and access shared objects in the variable portion of the SGA
3. Generic Oracle memory management was enhanced
Before we get into the details of how those three items were done, we will provide some background into how memory
management works in general. We then explain how Oracle attempts to genericize and optimize usage of the memory that it
uses.
Memory Management
Memory management will vary widely from operating system to operating system. Basically, most have a concept of virtual
memory addressing. There usually is a memory manager that determines how executable code is laid out across an address
space and where memory is allocated from when requested. OS memory managers also provide to varying degrees some
level of protection to prevent threads of execution (e.g. Unix processes) from reading or writing memory allocated to another
thread. It may also delineate user space from system space. But it also allows applications to share memory if it determines
there is memory that can be safely shared by multiple threads.
There is a set of operating system dependent code routines (OSD’s) and low level services that Oracle provides so that the
majority of Oracle server code can remain generic across all platfoms. One of these low level services is the generic memory
services. These services work with the OSD’s to allow for creation, destruction, and management of memory that is private
to a thread and memory that will be public to all Oracle threads.
Figures in Chapter 1 of the Oracle7 Server Concepts Manual presents a fairly nice view of the Oracle system architecture. It
relates the main memory structure of Oracle (the SGA) with the various processes that attach to it. The PGA is not displayed
until Chapter 9. Meanwhile, slightly different representations of this view appear in Chapter 1 of the Oracle7 Server for
Unix Administrator's Reference Guide (R7.1). It extends our picture by displaying the concept of single task versus two task
architectures and including a picture that associates the PGA with a specific dedicated server process. Chapter 2 of the
Understanding SQL*Net Guide (V2.1) presents us with a more detailed view of how the two task architecture is laid out.
Finally, Chapter 2 of the Programmer's Guide to the Oracle Call Interfaces (R7.1) shows a view of the client cursor area. It
also discusses the dynamics of communications that transpire between client and server when processing a SQL statement.
Figure 1 is an ambitious attempt to combine these concepts into a memory representation of the typical Oracle session in a
Unix two-task environment.
In this example, at the lowest address is the actual compiled code of the program.
Notes: the compiled code can be broken up into several sections including a text section that is shared by all processes executing the same
program. This code is made up of several modules maintained by different development groups within Oracle. For example the SQL*Net
module is only responsible for transporting data back and forth between client and server. The data itself is composed of user requests and
data replies that are created by the other modules. Another aside: the client picture actually shows how three separate applications would
integrate with the Oracle user program interface (UPI) via different layers of code.
At the high addresses, the operating system keeps tracks which routine the process is currently in and which calling routine
to return to. Just above the code section is the operating system managed heap space or the memory that is allocated for use
by the program. For example, a UNIX procss can allocate more space for itself with a malloc system call. Oracle calls
malloc/free to resize the PGA when necessary.
The remaining memory section to be discussed is the SGA or shared global area. It is really a piece of memory that is shared
by multiple threads of execution. All servers attach the same piece of memory to the same virtual address space. It is
within the SGA that the shared pool resides.
A D D R E S S S P A C E
SGA
Shared
(Shared Pool,
Memory
Buffer Cache, etc.)
O.S.
Heap
PGA
LDA & CDA
(UGA, CGA, etc.)
Pro*C
SQL*Plus
OCI
Oracle RDBMS
OCILIB SQLLIB Code
and
UPI OPI Data
TTC TTC
SQL*Net SQL*Net
User/Client Server
Figure 1.
Note: On UNIX, client and server are two separate processes; each with its own address space. Only the server will attach to the
SGA. Other operating systems such as VMS allows you to link the applications into a single task executable safely because the OS
provides feature to protect portion of memory from being modified by application specific code. If that protection was not present, then
Oracle client code that is linked single task can directly address and change contents of Oracle shared memory segments leading to
corruption of internal Oracle data structures.
The RDBMS memory is built on top of two memory structures: the PGA and the SGA. In other words, Oracle abstracts the
memory it uses into the PGA or SGA.
PGA
PGA, process global area or private global area, is memory needed for the operation of one process. There is one (and only
one) independent PGA for each process in an instance. An initial portion, fixed PGA, is allocated at process startup and it
will expand and contract dynamically. Once the process terminates, PGA memory is returned to the operating system.
For security reasons, information need only be seen or modified by a particular process is kept private within that thread of
execution. Oracle assumes serveral things about private memory: 1) no other process can access it , and the 2) client side
routines can not access it directly.
SGA
Consequently, for performance reasons there may be data that is better to be shared than to have all threads maintain a
private copy. In other words, accessing data that is already cached in a public memory area is faster than performing disk IO
or executing a similar set of instructions to create a private version of the data. The other advantage is that it decreases
unnecessary memory usage by moving immutable memory structures that are common to several processes into shared
memory which is found the shared pool.. It attempts to share as much as one can and to make memory allocation more
flexible and easy to use for the server processes. For this, Oracle requires a piece of memory that it assumes to be sharable
by all processes within that instance. For example, several processes executing the same sql statement can share the same
execution plan and avoid unnecessary reoptimization. It also assumes that this memory will be available for use as long as
the instance is running. The SGA, shared global area, is created during instance startup and is deallocated at shutdown.
Thereby, it does not dynamically change in size during its lifetime.
For each instance, there is one SGA which each process attaches to a fix address. [note: this is due to Oracle referencing
objects by their virtual address rather than by an offset into SGA] The SGA is composed of four regions (fixed data structure
size, shared pool, database block buffers, redo buffer- see figure 3).
The size of the fixed portion is not controlled by the user and consists of fixed Oracle data structures. On the other hand, the
variable “SGA” size is determined by shared pool size and memory needed to store structures such as handles, enqueues,
state objects, etc. Finally db block buffers and redo buffer constitute the last two major structures in the SGA and do not fall
within the scope of this paper.
SGA
Log Buffer
figure 3
CLASSES
Oracle operations allocate space for the data it uses in either the PGA or SGA. The memory location is determined by a
couple of factors: 1) whether the data is private or public and 2) the period of time the data is needed (a.k.a. its expected
lifetime or persistence). There are 4 different lifetimes: call, session, process, and instance.
A call’s lifetime is basically the time needed to execute an Oracle Programmatic Interface (OPI) call. Memory that only
needs to exist for the lifetime of a call and is private can be placed into PGA. In particular into an area called the CGA, call
global area. Examples include memory used for DDL operations, DML operations, local PL/SQL variables, and sorting data.
It just happens that these are not calls which data needs to be public.
A session’s lifetime lasts from the time a client connects to the database until it disconnects. Data used by a session persists
for the duration of a session or when the user logs out. Examples are NLS parameters, optimizer goals, sql_trace, alter
session information, PLS 2.0 package global states, cursor states, and other "login" data structures (used for security). We
call the area that where the data is stored the User Global Area (UGA). For MTS and XA, it is necessary for user data to be
public in the SGA because different servers may need to access such data during the lifetime of the session. Otherwise, it is
sufficient for the UGA to be private and part of the PGA.
A process’ lifetime lasts from the time a process is started until it dies. Memory used by a process is considered private and
therefore part of the PGA. Examples are the fixed PGA, CGA, and sometimes the UGA (as mentioned above).
Finally the lifetime of an instance is from the time a startup nomount is issued until shutdown or crash. Memory for data that
exists for the lifetime of an instance is generally public. The database buffer cache, log buffer, and shared pool are examples
of SGA memory areas.
Based on knowledge of how long a data structure is needed and whether it needs to be public or private, one can guess at
where memory for that stucture will be allocated. An exercise for the reader is to figure out why session information is
found in the shared pool or SGA when MTS is enabled.
PGA SGA
Persists until
process dies
Persistence Classes
This picture gives examples of what can be found in the PGA and SGA and the life span of data.
Note: The UGA is contained in the SGA (shared pool) when MTS is enabled.
The shared pool is a component of the SGA. This implies it is public and accessible to all server processes and the structures
can have a life span of the instance. The contents have a potential life span greater than the process that created it. Among
other things, shared cursors, PL/SQL objects can be found in the shared pool. There are primary two components of the
shared pool not including the overhead: the row cache (also called the dictionary cache), and the library cache.
The row cache contains a in memory version of portions of the data dictionary. For performance reasons, it may contain
other derived columns.
The library cache is in memory collection of shared objects that can be referenced by any server process. This cache
contains the majority of shared objects such as packages, procedures, shared cursors, tables, views, and other dependencies.
Server processes can reference any accessible material (objects).
Note: the overhead contains buffer cache descriptors, handles, enqueues, state objects, etc.
How is it managed?
The shared pool and the PGA are managed by a generic Oracle memory manager, often referred to the KGH heap manager.
Heap manager is not a process but a piece a code that is executed by all threads of execution when needed.
The heap manager’s main purpose is to accomodate server process memory requests and to “free” memory when required.
It’s generic across all requests and it hides operating system dependencies. For the PGA, it may interact with the operating
system to allocate (e.g., malloc/sbrk system call) or free memory (e.g., free system call). However in the shared pool,
memory is allocated upfront and the heap manager manages all free space, gives space to accomodate requests, and works
with other Oracle software modules to get back space for reuse.
Oracle considers the initial memory allocation for the shared pool to be the SGA Heap. This memory architecture is
comprised of heaps and subheaps which are the building blocks of Oracle’s dynamic memory structure.
Since the shared pool is accessible to all server processes, Oracle has mechanisms to synchronize operations and recover
from aborted processes. Synchronization and recovery are implemented using latches, state objects, and enqueues.
Synchronization is necessary to allow only one user to modify any particular location of memory or read it while another
process is writing. Recovery is necessary when an user process terminates while making a change to an SGA structure.
Shared memory recovery is initiated and the memory is freed to be accessible to the shared pool.
The library cache manager is designed and implemented to manage a shared library cache but in the single-user user-side
environment (e.g. DOS), the same library cache manager can manage a single-user library cache. It also provides recovery
and clean-up of library cache data structures that are abandoned by dead processes by using the client environment’s cleanup
process to perform the recovery. In a multi-instance environment (i.e. Oracle Parallel Server), the manager will provide
concurrency and consistency control among the multiple SGA’s.
Concepts
The main purpose of the library cache is to provide a mechanism to locate and store any library cache object quickly. A
hashing mechanism is used to locate a handle which contains the identity (name) of the object. The library cache handle then
points us to one or more the library cache objects and their contents.
The library cache caches different types of library objects (e.g. packages, procedures, functions, shared cursors, anonymous
PL/SQL blocks, table definitions, view definitions, form definitions).
Library cache memory is allocated out of the top most heap or the generic SGA heap. When the library cache, KGL, needs
more memory, it will call the heap manager (KGH) to allocate it. The library cache consists of a hash table which consists of
an array of hash buckets. Each hash bucket is a doubly linked list of library cache object handles. Each library cache object
handle points to a library cache object and has a reference list. The library cache object is further broken down into other
components such as a dependency table, a child table, and an authorization table (to name a few).
The library cache manager will apply a modulo hash function to a given object’s namespace, object name, owner, and
database link to determine the hash bucket where the object should be found. It then walks down the corresponding linked
list to see if the object is there. If the object does not exist, the library cache manager will create an empty object with the
given name, insert it in the hash table, and request the client load it by calling the client's environment-dependent load
function. Basically, the client would read from disk, call the heap manager to allocate memory, and load the object.
Note: Resizing a hash table is done automatically and again causes a small hiccup (estimated 3 to 5 seconds) in the system. Expanding the
hash table is prompted whenever there are, on average, multiple handles off each bucket. For applications that will reference a large
number of library cache objects, one can use an undocumented parameter, _kgl_bucket_count, to set the initial number of hash buckets to
a higher value to reduce the likelihood of a hash table resize operation. Since the hiccup is small and infrequent and the current value
should be sufficient, we suggest that most applications should not modify this parameter.
LIBRARY CACHE
Hash Bucket
array of hash buckets lock handles
HASH BUCKET
HASH BUCKET
The handle uses namespaces to partition library cache objects by types. These are examples of different types of
namespaces: one namespace holds all namespaces depended on PL/SQL objects, one for package bodies and table bodies;
one for shared cursors; one for triggers; one for indexes; and one for clusters. The namespace describes the type of an item
kept in the library cache. The name consists of the object owner name, the object name, the database link name, and the
database link owner name. A comprehensive list can be viewed from v$librarycache.
A handle can be freed if there are no current references to it and it has not been expressly marked to be kept. We use this to
determine when a handle should be unpinned in memory.
LIBRARY CACHE
OBJECT
authorization table
privilege list
type
object type
status flags
status flags
dependency table
The dependency table contains references to other library cache objects. For example, a procedure may be dependent on
several tables. The dependency table will contain pointers to these dependent handles (tables). The relative position of each
reference in the table is important and does not change because it is addressed in the dependent object by its relative position
in the table and not by its name. This dependency information is also stored in a data dictionary table (DEPENDENCY$) or
simply can be viewed in V$OBJECT_DEPENDENCY.
When the library cache manager discovers that the timestamp of an object has changed (during the release of the exclusive
pin on the object at commit time or during the re-initialization of the object after it is being purged), it invalidates all
dependency references to the object and breaks the locks on it. When the library cache manager discovers that an object has
changed (regardless of whether its timestamp has changed) and that its changes are very likely to be committed successfully,
it invalidates all read-only dependents of this object.
child table
The child table consists of references to child library cache objects (e.g., different versions of a cursor). Each child reference
in the hash table is again addressed within the parent object by its relative position in the table since all the children are
textually identical. Child objects or anonymous objects do not have a name in its handle but are referenced via the parent
object which does have a name. In other words, child objects can only be addressed via the parent object. For example, the
hash value of a SQL statement is based on the text of the SQL statement itself. If two users issue ‘SELECT * FROM EMP’
where the text may match but the EMP table is private to each user, then what one sees is a parent object whose name is the
SQL. It will have two children corresponding to the two versions of the SQL that was issued.
Note: A hashing optimization was attempted in 7.1.6 where we only inspected the first and last 64 bytes. However, it was undone when
some applications had many similar SQL statements that hashed to the same value.
4-14 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
Authorization Table
This table contains privileges that have been granted on the library object. Each item in the list consists of an identification
number of a user or role, privileges that each user or role has on the object, and optionally an identification number of a part
of the object that the privilege is on (e.g. column number of a table).
Type
Here are examples of different library cache object types: shared cursor (SQL cursor or PL/SQL anonymous block), index,
table, cluster, view, synonym, sequence, procedure, function, package, table body, package body, and trigger. More will be
added in later versions of Oracle.
Status Flags
There are a number of status flags combination of which indicate the state of the library cache object. The status flags
indicate the following:
1. the object is existent.
2. the object is non-existent.
3. the object is locally represented.
4. the object is being created
5. the object is being altered
6. the object is being dropped.
7. the object is being updated.
This will be discussed in more detail in version 2 of the paper.
2
2
P-Code (Debugging)
3
3
M-Code
4
PL/SQL shared cursor 4
library cache entry errors library cache entry
5
5
SQL Plan
6
6
7
7
From the previous diagram, one can see that a library cache entry can be composed of up to eight heaps or subheaps but it is
not necessary for all eight heaps to be allocated. PL/SQL objects typically use 6 heaps: the common object information heap,
source heap, diana heap, p-code heap, m-code heap and the errors heap. The common object information heap contains the
global name, dependency list, security list and variety of other bookkeeping information about the object. The source heap
contains the PL/SQL source for the compilation unit. The p-code heap was originally intended to store portable executable
code. However, it was never implemented (all code is considered machine dependent), so the p-code heap is primarily used
to store debugging information instead. The m-code heap was intended to contain only machine dependent code, but it
actually contains all executable code. The diana contains the parse and syntax trees, description of the meta-data and all its
references. Any errors posted during compilation is found in the errors heap.
The Oracle kernel’s SQL processor uses 2 heaps to represent a shared SQL statement: the object information heap and
query/execution plan heap. The name of object also serves as the source for the shared SQL statement. The plan heap
contains the query/execution plan for the SQL statement.
Both pins and locks will wait until granted; that is, one will wait for a lock or pin to be acquired. If the object is locked or
pinned in a conflicting mode, the library cache manager will make the client wait until that the desired lock or pin can be
granted..
Locks
Locks are only taken on handles.
There are 3 modes for locks: null, share, and exclusive. A null or shared mode is placed on the object when it is only
intended to be read (e.g. executing an object, referencing the object during compilation). An exclusive lock is needed to
modify the object. There is only a slight difference between a null lock and a shared lock. A null lock on an object can be
broken and usually occurs when another client attempts to pin the same object in exclusive mode. For example, a null lock
on a read-only object is broken when there is an exclusive pin on any of the parent objects it depends on. All locks on read-
only objects have to be in null mode. On the other hand, an exclusive lock is placed on a library cache object if it is intended
to be written or modified (e.g. compiling or creating a new object or recompiling an existing object). Null locks express
interest on the handle. They are usually used on cursors when doing a read-only operation. The null lock is invalidated
when the cursor is invalidated. This invalidated null lock notifies any process attempting to use this cursor that this object
is invalid.
The persistence of the lock depends on whether it is associated to a session, a transaction, or a call. If it is associated with a
session, then it is released when the session closes. If it is associated with a transaction, then the lock will not be released
until the transaction commits or rollback while the lock will released at the completion of the call if it is associated with a
call.
Pins
After locking a library cache object, a process must then pin the object before accessing it. A process pinning a library cache
object provides a mask indicating which data blocks (heaps) are to be pinned and loaded. There are two pin modes: shared
and exclusive. A library cache object is pinned in share mode when it is to be read only and in exclusive mode when it is to
be modified.
Pinning/unpinning also helps determine if a heap can be freed. A recreateable chunk (or heap) can be pinned or unpinned.
At the time a chunk is allocated, it is pinned by the heap manager. Whenever the client is not actively using the chunk of
space, he should unpin it. If an allocation request is made and there is not enough space available to satisfy it, the heap
manager can, by invoking a client supplied callback routine, request a client to free an unpinned chunk of space.
Example
Here is an example of using locks and pins when altering (recompiling) a procedure. First, the library cache object for the
procedure is locked in exclusive mode. This prevents others from performing the same operation or any operation which
would drop or replace the interested object. Locking also prevents others from creating new procedures, packages, or
functions that will reference the object. Next, it is pinned in share mode to get to its definition and authorization information
so necessary security and error checking can be performed. Then, the share pin is released, the object is repinned in
exclusive mode, and the procedure is recompiled. Finally, all of its dependents have to be invalidated.
In particular, the library cache latches are required to prevent multiple access to a share library cache entry. Library cache
latches come in three general flavors: the library cache latch, the pin latch and the load-lock latch. The library cache latch is
the latch at the highest level. The library cache latch is needed prior to getting a lock on a handle (e.g., to drop or free the
handle, to invalidate the handle, etc.). A process attempting to pin a heap needs to acquire the library cache pin latch. To
load a library cache entry, the library cache load lock latch is required. This prevents the possibility of multiple processes
loading the same object simultaneously. Since there is only one library cache latch (e.g., to walk the linked list of LCOs),
one library cache pin latch (e.g., for all library cache objects), and one library cache load latch (e.g., to load new library
cache entries), latch contention must be monitored. Of the three, the library cache latch is used and contended for most
often. The library cache latch is the most versatile latch since it is acquired prior to hashing to a handle, dropping a handle,
and creating a new handle.
Oracle soon realised that these latches could be enormous bottlenecks as the level of concurrent activity in the system
increases. In Oracle 7.2, we have multiple library cache latches to help alleviate this problem. Each latch will be designated
to several specific hash values. In this model, the library cache hash table will be broken into ranges. A handle which hashes
into bucket M will be protected by latch id# (M mod N), where the latch ids are 0,1, ..., N-1. This means that there will be a
list of free lock descriptors, pin descriptors, handles, heap descriptors, etc. for each latch. This will allow users to perform
most operations through KGL (like locking, pinning, invalidation, etc.) using the one latch that is required for the handle of
the object being operated on. (More information can be found in Amit Jasuja’s Multiple Latches in KGL paper).
INTRODUCTION
As described in the previous sections, every object and shareable segment is found in the shared pool. The shared pool is
limited by the size of the available heap memory and efficient use of this memory will be dependent on application design
and the activity occurring in the database. This section focuses on application design and the effects of this on the shared
pool. Its main emphasis is on describing ways to tune and monitor the shared pool.
Shared pool tuning is much different from other areas of database tuning where there are many init.ora parameters that one
can tweak and adjust. Other than increasing and decreasing the shared pool size, most tuning concepts will center around
understanding the application and then using that knowledge to keep important packages in the shared pool, set or unset
cursor_space_for_time, or set session_cache_cursors to a relevant value. One or a combination of these operations hopefully
will result in reduced misses in the library cache and in reduced fragmentation. For the most part, application coding
standards and techniques are required to optimally tune the shared pool and efficiently use the sharing mechanisms of the
library cache. A bad application will potentially render other tuning options futile.
One can quickly discover if there is a performance problem in the library cache by monitoring the cache hit ratios. A cache
miss in the library cache or dictionary cache is potentially more expensive than a miss in the buffer cache. For this reason,
one must allocate sufficient memory for the shared pool first. Furthermore, the algorithm that Oracle uses to manage data in
the shared pool tends to hold dictionary data in memory longer than library cache data. Thus, tuning the library cache to an
acceptable cache hit ratio often ensures that the data dictionary cache hit ratio is also acceptable.
Description
A library cache miss can occur during the parse or execute phase.
Parse: If an application makes a parse call for a SQL statement and the parsed representation of the statement does not exist
in a shared SQL area in the library cache, Oracle parses and allocates space in the shared SQL area. You can reduce library
cache misses by ensuring that SQL statements can share a shared SQL area whenever possible.
Execute: If an application makes an execute call for a SQL statement and the shared SQL area containing the parsed
representation of the statement has been deallocated from the library cache to make room for another statement, Oracle
implicitly reparses the statement, allocates a new shared SQL area for it and executes it. You may be able to reduce the
library cache misses on execution calls by allocating more memory to the library cache.
Diagnosis
V$LIBRARYCACHE view will be used to determined if there is a performance problem within the library cache. The
namespace describes the type of an item kept in the library cache.
The following query outlines the ratio of cache handles and cache objects to be pinned for each namespace. It also gives an
idea of what type of activity is occurring in the library cache.
select namespace, gets, gethitratio, pins, pinhitratio, reloads, invalidations from v$librarycache;
When there is large numbers of gets and pins (over 1000) and the gethitratio and pinhitratio is low (less than 85%), the
shared pool size needs to be increased or other tuning techniques discussed in this paper needs to be investigated. Reloads
indicate that library objects have to be reinitialized and reloaded with data because they have been aged out or invalidated.
Total reloads should be near 0. If the ratio of RELOADS to PINS is more than 1%, then you should reduce the library cache
misses. High invalidations indicate that non-persistent library objects (like shared SQL areas) have been invalidated
probably due to role changes or altering tables. Development may be occurring which changes or recreate tables and
privileges which may cause unnecessary invalidations. A closer look of the application may be required.
However, it is very difficult to tune one particular namespace. For that reason, this query is normally used to give an idea of
total misses and access attempts in the library cache. The sum(pins) indicates the number of times that SQL statements,
PL/SQL blocks and object definitions were accessed for execution. The sum(reloads) indicates the number of times those
executions resulted in library cache misses causing Oracle to implicitly reparse a statement or block or reload an object
definition because it has been aged out.
V$ROWCACHE view gives statistics for data dictionary activity. Each row contain statistics for one data dictionary cache.
In version 6, Oracle gives the capability of tuning each parameter. In Oracle7 and higher, the dictionary cache which is part
of the row cache grabs memory from the Generic Heap. When required, the row cache will grow to accomodate more data
dictionary entries if space is available. The following queries will indicate if there are data dictionary cache misses.
Action
One can reduce library cache misses by (1) allocating additional memory for the library cache or (2) writing identical SQL
statements or shareable code. Increasing shareable memory is as simple as increasing the size of the shared_pool_size. One
4-20 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
must be careful not to induce paging or swapping due to insufficient memory. In order for different occurrences of a SQL
statement or PL/SQL block to share a shared SQL area, they must have identical text including space and case, reference the
same schema objects, bind variables must match in name and data type and use the same optimizer approach. This will be
discussed further later in this section.
If the total hit ratio is high in the row cache, then one should monitor the amount of memory allocated in the ‘dictionary
cache’ and ‘free memory’ via V$SGASTAT. If the ‘dictionary cache’ is relatively unchanging (not growing throughout the
day) and ‘free memory’ is large and also showing little fluctuations (greater than dictionary cache), then the bad hit ratio may
be caused by users dropping and recreating objects or modifying grants. This should be reflected by hit ratios on subordinate
row cache types found in V$ROWCACHE. If there is very low free memory and the overall hit ratio is below 90% , then
increasing the shared pool should help tune the row cache.
MEMORY FRAGMENTATION
The primary problem that occurs is that free memory in the shared pool becomes fragmented into small pieces over time.
Any attempt to allocate a large piece of memory in the shared pool will cause large amount of objects in the library cache to
be flushed out and may result in an ORA-4031 out of shared memory error. (information from Juan Luanza’s shared pool
tuning paper)
DIAGNOSIS
i) ORA-4031 ERROR
One way to diagnose that this is happening is to look for ORA-4031 errors being returned from applications. When an
attempt is made to allocate a large contiguous piece of shared memory, and not enough contiguous memory can be created in
the shared pool, the database will signal this error.
Before this error is signaled, all objects in the shared pool that are not currently pinned or in use will be flushed from the
shared pool, and their memory will be freed and merged. This error only occurs when there is still not a large enough
contiguous piece of free memory after this happens. There may be very large amounts of total free memory in the shared
pool, but just not enough contiguous memory.
Note: From 7.0.15, the compiled code for a package was split into more than one piece, each piece being only about 12K in
size. So, the 64K restriction was lifted; however, packages larger 100K still may have problems compiling. Furthermore,
with releases 7.2/2.3 of Oracle7/PLSQL, loading a library unit (package, function, procedure) into the shared pool does NOT
require one contiguous piece of memory in the shared pool. This means that chances of getting ORA-4031 is dramatically
reduced.
An init.ora parameter can be set so that whenever an ORA-4031 error is signaled a dump will occur into a trace file. By
looking for these trace files, the DBA can determine that these errors are occurring. This is useful when applications do not
always report errors signaled by oracle, or if users do not report the errors to the DBAs. The parameter is the following:
If you are using 7.0.16 or higher you can use the following:
This will cause a dump of the oracle state objects to occur when this error is signaled. By looking in the dump for
'load=X' and then looking up a few lines for 'name=' you can often tell whether an object was being loaded into the shared
pool when this error occurred. If an object was being loaded then it is likely that this load is the cause of the problem and the
object should be 'kept' in the shared pool. The object being loaded is the object printed after the 'name='. Do not use the
'level 4' option in versions before 7.0.16 because a bug existed that often caused the system to crash with this option enabled
due to a latch level violation.
iii) X$KSMLRU
There is a fixed table called X$KSMLRU that tracks allocations in the shared pool that cause other objects in the shared
pool to be aged out. This fixed table can be used to identify what is causing the large allocation. The columns of this
fixed table are the following:
If this comment is something like 'MPCODE' or 'PLSQL%' then there is a large PL/SQL object being loaded into the
shared pool. This PL/SQL object will need to be 'kept' in the shared pool.
If this comment is 'kgltbtab' then the allocation is for a dependency table in the library cache. This is only a problem when
several hundred users are logged on using distinct user ids. The solution in this case is to use fully qualified names for all
table references. This problem will not occur in 7.1.3 or later.
If you are running MTS and the comment is something like 'Fixed UGA' then the problem is that the init.ora parameter
'open_cursors' is set too high.
KSMLRSIZ - amount of contiguous memory being allocated. Values over around 5K start to be a problem, values over 10K
are a serious problem, and values over 20K are very serious problems. Anything less then 5K should not be a problem.
KSMLRNUM - number of objects that were flushed from the shared pool in order allocate the memory.
KSMLRHON - the name of the object being loaded into the shared pool if the object is a PL/SQL object or a cursor.
The advantage of X$KSMLRU is that it allows you to identify problems with fragmentation that are effecting performance,
but that are not bad enough to be causing ORA-4031 errors to be signaled. If a lot of objects are being periodically flushed
from the shared pool then this will cause response time problems and will likely cause library cache latch contention
problems when the objects are reloaded into the shared pool.
One unusual thing about the X$KSMLRU fixed table is that the contents of the fixed table are erased whenever someone
selects from the fixed table. This is done since the fixed table stores only the largest allocations that have occurred. The
values are reset after being selected so that subsequent large allocations can be noted even if they were not quite as large as
others that occurred previously. Because of this resetting, the output of selecting from this table should be carefully noted
since it cannot be reselected if it is forgotten. Also you should take care that there are not multiple people on one database
that select from this table because only one of them will select the real data.
ACTION
i) KEEPING OBJECTS
The primary source of problems is large PL/SQL objects. The means of correcting these errors is to 'keep” large PL/SQL
objects in the shared pool at startup time. This will load the objects into the shared pool and will make sure that the objects
are never aged out of the shared pool. If the objects are never aged out then there will not be a problem with trying to load
them and not having enough memory.
Objects are 'kept' in the shared pool using the dbms_shared_pool package that is defined in the dbmspool.sql file. For
example:
execute dbms_shared_pool.keep('SYS.STANDARD');
All large packages that are shipped should be 'kept' if the customer uses PL/SQL. This includes 'STANDARD',
'DBMS_STANDARD', and 'DIUTIL'.
One restriction on the 'keep' procedure is that it only works on packages. If the customer has large procedures or large
anonymous blocks, then these will need to be put into packages and marked kept.
You can determine what large stored objects are in the shared pool by selecting from the V$DB_OBJECT_CACHE fixed
view. This will also tell you which objects have been marked kept. This can be done with the following query:
Note that this query will not catch PL/SQL objects that are only rarely used and therefore the PL/SQL object is not currently
loaded in the shared pool.
To determine what large PL/SQL objects are currently loaded in the shared pool and are not marked 'kept' and therefore may
cause a problem, execute the following:
Note: Oracle 7.1.6 and on, bugs 192829 and 205976 have been fixed. Now you can pin procedures and triggers with the
dbms_shared_pool procedure. Either procedures or packages can be pinned with the ‘P’ flag, which is the default value.
Triggers are pinned with ‘R’ and anonymous PL/SQL blocks need any letter other than [p,P,r,R] as a flag.
Another thing that can be done to reduce the amount of fragmentation is to reduce or eliminate the number of SQL
statements in the shared pool that are duplicates of each other except for a constant that is embedded in the statement. The
statements should be replaced with one statement that uses a bind variable instead of a constant.
For example:
select * from emp where empno=1;
select * from emp where empno=2;
select * from emp where empno=3;
Should all be replaced with:
select * from emp where empno=:1;
You can identify statements that potentially fall into this class with a query like the following:
It is possible for a sql statement to not be shared because the max bind variable lengths of the bind variables in the statement
do not match. This is automatically taken care of for precompiler programs and forms programs, but could be a problem for
programs that directly use OCI. The bind call in OCI takes two arguments, one is the max length of the value, and the other
is a pointer to the actual length. If the current length is always passed in as the max length instead of the max possible
length for the variable, then this could cause the sql statement not to be shared.
To identify statements that might potentially have this problem execute the following statement:
Large anonymous PL/SQL blocks should be turned into small anonymous PL/SQL blocks that call packaged functions. The
packages should be 'kept' in memory. This includes anonymous PL/SQL blocks that are used for trigger definitions. Large
anonymous blocks can be identified with the following query:
Note that this query will not catch PL/SQL blocks that are only rarely used and therefore the PL/SQL block is not currently
loaded in the shared pool.
Another option that can be used when an anonymous block cannot be turned into a package is to mark the anonymous block
with some string so that it can be identified in v$sqlarea and marked 'kept'.
You can then use the following procedure to select these statements out of the shared pool and mark them 'kept' using the
dbms_shared_pool.keep package.
declare
/* DONT_KEEP_ME */
addr varchar2(10);
hash number;
cursor anon is
select address, hash_value
from v$sqlarea
where command_type = 47 -- command type for anonymous block
and sql_text like '% KEEP_ME %'
and sql_text not like '%DONT_KEEP_ME%';
begin
open anon;
loop
fetch anon into addr, hash;
exit when anon%notfound;
dbms_shared_pool.keep(addr || ',' || to_char(hash), 'C');
end loop;
end;
Another big problem that can occur in Oracle7 on multiprocessors that have a large number of CPUs is contention for the
library cache latch.
DIAGNOSIS
i) V$LATCH
Selecting from V$LATCH will show you which latches have the worst hit rates and more importantly which latches are
causing alot of sleeps. If the library cache latch is causing the most number of sleeps then you may have a problem. One
thing to watch out for here is that this information is accumulated since the database starts, and so it may not show problems
that are intermittent in nature.
ii) V$SESSION_WAIT
By selecting from V$SESSION_WAIT during a slowdown period you can usually determine very accurately whether you
have a problem with latching and which latch is causing the problem. If you see a large number (more then 3 or 4) of
processes waiting for the library cache or library cache pin latch, then
there may be a problem. Run the following query to determine this:
It is also very useful to just select from v$session_wait to determine what else is causing a slowdown:
CORRECTION
i) FRAGMENTATION
The primary cause of library cache latch contention is fragmentation of the shared pool. This can be diagnosed and
addressed as described in the fragmentation section of this document. If you are running on a system with just one or a very
small number of CPUs and you have a problem with library cache latch contention, then fragmentation is almost certainly
the source of the problem.
By increasing the amount of sharing that occurs on the system you can decrease the amount of missing and loading that
occurs in the library cache and therefore the load on the library cache latch. This is done by identifying statements that are
not being shared as described in the fragmentation section above.
To determine the percentage of sql statement parse calls that find a cursor to share you can execute the following:
Another way to decrease the load on the library cache latch is to reduce the number of parse calls that are coming into the
system. Even if the statement being parsed is found in the shared pool and shared, the load of a parse call is high because
the user must be authenticated to run the sql statement, and all name translations must be done for the SQL statement.
Reducing the amount of parsing is often as simple as setting 'HOLD_CURSOR=TRUE' for the precompilers. To identify the
sql statements that are receiving a lot of parse calls execute the following:
To identify the total amount of parsing going on in the system execute the following:
If this value increases at a rate greater than about 10 per second then this may be a problem.
iv) CURSOR_SPACE_FOR_TIME
Setting the init.ora parameter CURSOR_SPACE_FOR_TIME to TRUE can reduce the load on the library cache latch
somewhat. However, setting this parameter may add a lot of memory utilization, so before setting it to true make sure that
there is a lot of free memory on the system and that the number of hard page faults per minute is very low or zero.
v) SESSION_CACHED_CURSORS
In version 7.1 there is an init.ora parameter called SESSION_CACHED_CURSORS that can be set that will help in
situations where a user repeatedly parses the same statements. This can occur in many applications including FORMS based
application if users often switch between forms. Every time a user switches to a new form all the sql statements opened for
the old form will be closed. The SESSION_CACHED_CURSORS parameter will cause closed cursors to be cached within
the session so that a subsequent call to parse the statement will bypass the parse phase. This is similar to HOLD_CURSORS
in the precompilers. One thing to be careful about is that if this parameter is set to a high value, the amount of fragmentation
in the shared pool may be increased.
It can help to reduce the load on the library cache latch somewhat to use fully qualified names for tables in sql statements.
That is, instead of saying 'select * from emp', say 'select * from scott.emp'. This is especially helpful for sql statements that
are parsed very frequently. If all users log into the database using the same userid then this may be of little or no use.
vii) FORMS 4
SQL*forms version 4 generates less dynamic sql by making better use of bind variables. Therefore, less loading occurs in
the shared pool. One might consider switching to this new version sooner than you otherwise would have because of
this.
One very difficult judgment that needs to be make in Oracle7 is to determine the proper size of the shared pool. The
following provides some guidelines for this. It should be emphasized that these are just guidelines, there are no hard and fast
rules here and experimentation will be needed to determine a good value.
The shared pool size is highly application dependent. To determine the shared pool size that will be needed for a
production system it is generally necessary to first develop the application and run it on a test system and take some
measurements. The test system should be run with a very large value for the shared pool size to make the measurements
meaningful.
The amount of shared pool that needs to be allocated for objects that are stored in the database like packages and views is
easy to measure. You can just measure their size directly with the following statement:
This is especially effective because all large pl/sql object should be 'kept' in the shared pool at all times.
SQL
The amount of memory needed to store sql statements in the shared pool is more difficult to measure because of the needs of
dynamic sql. If an application has no dynamic sql then the amount of memory can simply be measured after the application
has run for a while by just selecting it out of the shared pool as follows:
If the application has a moderate or large amount of dynamic sql like most applications do, then a certain amount of
memory will be needed for the shared sql plus more for the dynamic sql. Sufficient memory should be allocated so that the
dynamic sql does not age the shared sql out of the shared pool.
The amount of memory for the shared sql can be approximated by the following:
The remaining memory in v$sqlarea is for dynamic sql. Some shared pool will need to be budgeted for this also, but there
are few rules here.
You will need to allow around 250 bytes of memory in the shared pool per concurrent user for each open cursor that the user
has whether the cursor is shared or not. During the peak usage time of the production system, you can measure this as
follows:
In a test system you can measure it by selecting the number of open cursors for a test user and multiplying by the total
number of users:
The per-user per-cursor memory is one of the classes of memory that shows up as 'library cache' in v$sgastat.
MTS
If you are using multi-threaded server, then you will need to allow enough memory for all the shared server users to put their
session memory in the shared pool. This can be measured for one user with the following query:
A more conservative value to use is the maximum session memory that was ever allocated by the user:
To select this value for all the currently logged on users the following query can be used:
OVERHEAD
You will need to add a minimum of 30% overhead to the values calculated above to allow for unexpected and unmeasured
usage of the shared pool.
Estimating Procedure
This will help estimate how big the shared pool should be at this moment
declare
object_mem number;
shared_sql number;
cursor_mem number;
mts_mem number;
used_pool_size number;
free_mem number;
pool_size varchar2(100);
begin
-- For a test system -- get usage for one user, multiply by # users
-- select (250 * value) bytes_per_user
-- from v$sesstat s, v$statname n
-- where s.statistic# = n.statistic#
-- and n.name = 'opened cursors current'
-- and s.sid = 25; -- where 25 is the sid of the process
-- MTS memory needed to hold session information for shared server users
-- This query computes a total for all currently logged on users (run
-- during peak period). Alternatively calculate for a single user and
-- multiply by # users.
select sum(value) into mts_mem from v$sesstat s, v$statname n
where s.statistic#=n.statistic#
and n.name='session uga memory max';
-- Free (unused) memory in the SGA: gives an indication of how much memory
-- is being wasted out of the total allocated.
select bytes into free_mem from v$sgastat
where name = 'free memory';
-- For non-MTS add up object, shared sql, cursors and 30% overhead.
used_pool_size := round(1.3*(object_mem+shared_sql+cursor_mem));
-- Display results
dbms_output.put_line ('Obj mem: '||to_char (object_mem) || ' bytes');
dbms_output.put_line ('Shared sql: '||to_char (shared_sql) || ' bytes');
dbms_output.put_line ('Cursors: '||to_char (cursor_mem) || ' bytes');
dbms_output.put_line ('MTS session: '||to_char (mts_mem) || ' bytes');
dbms_output.put_line ('Free memory: '||to_char (free_mem) || ' bytes ' || '(' ||
to_char(round(free_mem/1024/1024,2)) || 'M)');
dbms_output.put_line ('Shared pool utilization (total): '|| to_char(used_pool_size) || ' bytes ' || '(' ||
to_char(round(used_pool_size/1024/1024,2)) || 'M)');
4-30 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
dbms_output.put_line ('Shared pool allocation (actual): '|| pool_size ||' bytes ' || '(' ||
to_char(round(pool_size/1024/1024,2)) || 'M)');
dbms_output.put_line ('Percentage Utilized: '||to_char (round(used_pool_size/pool_size*100)) || '%');
end;
/
COMMON FALLACIES
There are a number of common fallacies about the shared pool that are often stated as fact.
FREE MEMORY
One fallacy is that the amount of 'free memory' reported in v$sgastat needs to be kept high. This is incorrect. The free
memory reported in this table is not like the free memory reported by operating system statistics. Since the shared pool acts
as a cache, nothing will ever be aged out of the shared pool until all the free memory has been used up. This is entirely
normal.
Free memory is more properly thought of as 'wasted memory'. You would rather see this value be low than very high. In
fact, a high value of free memory is sometimes a symptom that a lot of objects have been aged out of the shared pool and
therefore the system is experiencing fragmentation problems.
Some people think that frequently executing 'alter system flush shared_pool' improves the performance of the system and
decreases the amount of fragmentation. This is incorrect. Executing this statement causes a big spike in performance and
does nothing to improve fragmentation. Flushing the shared pool does not flush “pinned” or “kept” objects.
The only time when it might be useful to run this statement is between shifts of users so that the objects that are relevant to
the last shift of users can be flushed out before the next shift of users starts to use the system. This is almost never needed
though. However, I have seen bugs where flushing the shared pool did fix some locking contentions.
SUMMARY
The most important point that needs to be understood by everyone using Oracle7 and PL/SQL is that all large PL/SQL
objects must be made into packages and those packages must be kept in the shared pool. This point cannot be over
emphasized. Many customers, especially those running a lot of users, have had terrible performance problems that were
completely cleared up by doing this.
This sections outlines some of the key views utilized to monitor the shared pool and the library cache. Oracle 7 Reference
Guide and Oracle 7 DBA guide has complete descriptions of all the columns.
V$DB_OBJECT_CACHE
Shows database objects that are cached in the library cache. Objects include tables, indexes, clusters, synonyms definitions,
PL/SQL procedures and packages, and triggers. Objects with large sharable memory (over 5K) should be kept if possible.
Large loads values (over 5) may be reduced if the object (table, package, or sequence) can be cached or kept in the shared
pool.
V$LIBRARYCACHE
Statistics on the library cache management.
The get hit ratio, pin hit ratio, reloads and invalidations should be monitored throughout time. If the hit ratios drop, one can
first determine if the application or load on the database has changed recently. In particular, SQL AREA namespace should
have a high hit ratio. Low hit ratio may be mitigated by writing sharable code.
V$OBJECT_DEPENDENCY
Dynamic performance table that can be used to determine what objects are depended on by a package, procedure, or cursor
that is currently loaded in the shared pool.
This query describes the owner and name of objects that the application depends on when given the cursor text.
This query describes the owner and name of objects that the application depends on when given the sid. This only evaluates
the current statement.
V$SGASTAT
It has detailed information on the System Global Area and its fixed and varied allocations of memory. One should monitor
the difference (deltas) between the value of bytes per component of the SGA. This should be used for capacity planning and
4-32 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
describes what activity is eating up most of the space in the SGA. In particular, pay close attention to the amount of ‘free
memory’, ‘library cache’, ‘sql area’, and ‘dictionary cache’. Ideally, ‘free memory’ should be low (< 1 MB) while ‘library
cache’, ‘sql area’, and ‘dictionary cache’ should be stable with high hit ratios from v$librarycache and v$rowcache.
V$SQLAREA
It contains statistics on the shared cursor cache. Each row has statistics on one shared cursor.
It has the most versatile purposes. V$SQLAREA can be used to find the number of open cursors (counting
USERS_OPENING), locate common potentially sharable code (using SQL_TEXT), locate cursors with many versions
(using VERSION_COUNT), cursors or anonymous blocks that should be called via packages (looking for large
SHARABLE_MEM) and the number reads and writes completed by this cursor. Here are a list of some of the queries that
one may want to run. Some are repeats from discussions above.
Locates almost identical written code that are not shared usually because bind variables are not used.
Identifies cursors that have identical sql text but are not shared because of differences in definition or declaration. With
OCI, max bind size may vary which will cause the cursor not to be shared.
Large anonymous PL/SQL blocks should be turned into small anonymous PL/SQL blocks that call packaged functions. The
packages should be 'kept' in memory. This includes anonymous PL/SQL blocks that are used for trigger definitions. Large
anonymous blocks can be identified with the following query:
These queries track and measure the amount sharable memory used for sql and estimates for dynamic sql.
Note: To only measure open cursors then add this to the where clause, ‘and users_opening>0’.
V$SQL
This table is identical to V$SQLAREA except that it does not join all SQL statements with the same text together. There
may be multiple rows in this table with the same SQL statement if there are multiple versions of the cursor in the cursor
cache. Version count is not found in V$SQL because each SQL will have its own row.
V$OPEN_CURSOR
Cursors that each user session currently has opened and parsed.
V$ROWCACHE
Statistics for data dictionary activity. Each row contain statistics for one data dictionary cache.
V$SESSION_CURSOR_CACHE
Displays information on cursor usage for the current session. This is very helpful in determining the maximum number of
cursors allowed opened at one time, current number cursors in use, cursor open information, and the hit ratio. If memory
permits and hit ratio is low, then one should consider increasing session_cached_cursors.
select maximum, count, opened_once, open, opens, hits, hit_ratio from v$session_cursor_cache;
V$SYSTEM_CURSOR_CACHE
Displays similar information to the V$SESSION_CURSOR_CACHE view except that this information is system wide.
V$SYSSTAT
The current system wide value for each statistic. The following parameters should be monitor and measured throughout
time. The following is a list of important parameters.
1. open cursors current == how many cursors open currently. Should be ran during busy peaks of the application
cycle.
2. open cursors cumulative == total number of cursors opened since database open. Used with bstat/estat to
measure number of cursors between time interval.
3. cursor authentications == ## times that cursor privileges have been verified , either for select or because
privileges were revoked from an object , causing all users of the cursors to be reauthenticated.
4. session uga memory == total number of session uga memory (good estimate if thinking about implementing
MTS)
5. session uga memory max == maximum size of the uga
6. session pga memory == total pga memory allocated
7. session pga memory max == maximum pga memory allocated
8. parse count should not increase more than 10 per second.
9. session cursor cache count == count of session cursors cached.
10. session cursor cache hits == used with session cursor cache count to find the total hit ratio.
Shared pool tuning usually involves monitoring throughout a complete application cycle instead of a subset of the cycle.
drop.sql
This scripts allows the user to quickly drop all objects involved in the shared pool scripts.
drop sequence stat$interval;
drop table stats$begin_stats ;
drop table stats$end_stats ;
drop table stats$begin_latch ;
drop table stats$end_latch ;
pool_obj.sql
This script creates objects for shared pool bstat/estat. The sequence number is the unique key that will be joined to get all
information for specific run. It is therefore the unique identifier for a particular run of BSTAT/ESTAT.
/* summary stats */
create table hist$dates (test number,
type varchar2(10), stats_gather_times varchar2(100));
/* comparison of ratios */
create table hist$ratios
(test number, Name varchar2(40), Value number, Ratio varchar2(30));
/* create end statistics procedure which inserts into end tables and into
history and summary tables */
e.time_waited time_waited,
e.event event
from stats$end_event e
where e.event not in ( select b.event from stats$begin_event b
where b.test=counter)
and e.test=counter;
insert into hist$dc
select counter, b.parameter name,
e.gets-b.gets get_reqs,
e.getmisses-b.getmisses get_miss,
e.scans-b.scans scan_reqs,
e.scanmisses-b.scanmisses scan_miss,
e.modifications-b.modifications mod_reqs,
e."COUNT" ,
e.usage cur_usage,
e.fixed fixed
from stats$begin_dc b, stats$end_dc e
where b.cache#=e.cache#
and nvl(b.subordinate#,-1) = nvl(e.subordinate#,-1)
and b.test=counter and e.test=counter;
insert into hist$lib select counter, e.namespace, e.gets-b.gets gets,
e.gethits-b.gethits gethits,
e.pins-b.pins pins,
e.pinhits-b.pinhits pinhits,
e.reloads - b.reloads reloads,
e.invalidations - b.invalidations invals
from stats$begin_lib b, stats$end_lib e
where b.namespace = e.namespace
and b.test=counter and e.test=counter;
commit;
end;
/
p_bstat.sql
Rem ********************************************************************
Rem Gather start statistics
Rem ********************************************************************
from v$latch;
commit;
p_estat.sql
This script calls the estat procedure and gathers statistis and summary data.
execute estat;
p_query.sql
This script creates a report reflecting the statistics and hit ratios of the most recent run.
spool results
set echo on
set pagesize 60
set linesize 80
set numwidth 10
spool report.txt
Rem Select Library cache statistics. The pin hit rate shoule be high.
column "Gethit%" format 90.999
column "Pinhit%" like gethitratio
column reloads format 9,999,990
column invals like reloads
select namespace library,
gets,
round(decode(gethits,0,1,gethits)/decode(gets,0,1,gets),3)
"Gethit%",
pins,
round(decode(pinhits,0,1,pinhits)/decode(pins,0,1,pins),3)
"Pinhit%",
reloads, invals
from hist$lib
where test = (select max(a.test) from hist$lib a)
;
set numwidth 8;
column name format a25;
column "Gethit%' format 90.999
Rem Select Dictionary Cache (rowcache) Statistics
Rem get_miss and scan_miss should be very low compared to the requests.
Rem cur_usage is the number of entries in the cache that are being used.
select name, get_reqs, get_miss, scan_reqs, scan_miss, mod_reqs, count,
cur_usage, round((get_reqs-get_miss)/decode(get_reqs,0,1,get_reqs),3) "Gethit%"
from hist$dc
where get_reqs != 0 or scan_reqs != 0 or mod_reqs != 0
and test= (select max(test) from hist$lib);
set numwidth 27
Rem Average length of the dirty buffer write queue. If this is larger than
Rem the value of the db_block_write_batch init.ora parameter, then consider
Rem increasing the value of db_block_write_batch and check for disks that
Rem are doing many more IOs than other disks.
select queue.change/writes.change "Average Write Queue Length"
from hist$stats queue, hist$stats writes
where queue.name = 'summed dirty queue length'
and writes.name = 'write requests'
and queue.test=writes.test and
queue.test = (select max(test) from hist$lib);
set numwidth 12
column latch_name format a30
column hit_ratio format 90.999
Rem Statistics on no_wait gets of latches. A no_wait get does not wait for the
Rem latch to become free, it immediately times out.
select name latch_name,
immed_gets nowait_gets,
immed_miss nowait_miss,
round(decode(immed_gets-immed_miss,0,1,immed_gets-immed_miss)/
decode(immed_gets,0,1,immed_gets),
3)
hit_ratio
from hist$latches
where immed_gets != 0 and
test = (select max(test) from hist$lib)
order by name;
spool off;
comp_ratios.sql
This script generates some key ratios for BSTAT/ESTAT analysis. This script will populate the comp_ratio table.
declare
cons_gets number;
dblk_gets number;
total_gets number;
phy_reads number;
phy_writes number;
lcache number;
4-42 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
rcache number;
rdo_sreq number;
rdo_retry number;
rdo_ent number;
rdo_size number;
rbs_wait number;
bbwaits number;
fbwaits number;
lch_miss number;
lch_imiss number;
tfcont_row number;
tfby_rid number;
tsrow_got number;
fbuff_ins number;
fbuff_req number;
cum_cur number;
sort_dsk number;
sort_mem number;
sort_row number;
usr_roll number;
usr_com number;
rec_calls number;
usr_calls number;
parse_cnt number;
dbwr_cross number;
dbfree number;
dbmkfree number;
dbscan number;
dblru number;
bg_stats date;
en_stats date;
sum_req number;
wr_req number;
stat_time number;
counter number;
begin
select max(test) into counter from hist$stats;
begin
select change into cons_gets from hist$stats
where name = 'consistent gets' and test=counter;
exception when no_data_found then
cons_gets:=1;
end;
begin
select change into dblk_gets from hist$stats
where name = 'db block gets' and test=counter;
exception when no_data_found then
dblk_gets:=1;
end;
total_gets:= cons_gets + dblk_gets;
begin
select change into phy_reads from hist$stats
where name = 'physical reads' and test=counter;
exception when no_data_found then
ORACLE CONFIDENTIAL, Release 7.3 4-43
Shared Pool Internals Technical Reports Compendium, Volume I, 1995
phy_reads:=1;
end;
begin
select change into phy_writes from hist$stats
where name = 'physical writes' and test=counter;
exception when no_data_found then
phy_writes:=1;
end;
begin
select change into rdo_sreq from hist$stats
where name = 'redo log space requests' and test=counter;
exception when no_data_found then
rdo_sreq:=1;
end;
begin
select change into rdo_ent from hist$stats
where name = 'redo entries' and test=counter;
exception when no_data_found then
rdo_ent:=1;
end;
begin
select change into rdo_retry from hist$stats
where name = 'redo buffer allocation retries' and test=counter;
exception when no_data_found then
rdo_retry:=1;
end;
begin
select change into rdo_size from hist$stats
where name = 'redo size' and test=counter;
exception when no_data_found then
rdo_size:=1;
end;
begin
select event_count into bbwaits from hist$event
where event = 'buffer busy waits' and test=counter;
exception when no_data_found then
bbwaits:=1;
end;
begin
select event_count into fbwaits from hist$event
where event = 'free buffer waits' and test=counter;
exception when no_data_found then
fbwaits:=1;
end;
begin
select change into tfcont_row from hist$stats
where name = 'table fetch continued row' and test=counter;
exception when no_data_found then
tfcont_row:=1;•
end;
begin
select change into tfby_rid from hist$stats
where name = 'table fetch by rowid' and test=counter;
exception when no_data_found then
tfby_rid:=1;
4-44 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
end;
begin
select change into tsrow_got from hist$stats
where name = 'table scan rows gotten' and test=counter;
exception when no_data_found then
tsrow_got:=1;
end;
begin
select change into fbuff_ins from hist$stats
where name = 'free buffer inspected' and test=counter;
exception when no_data_found then
fbuff_ins:=1;
end;
begin
select change into fbuff_req from hist$stats
where name = 'free buffer requested' and test=counter;
exception when no_data_found then
fbuff_req:=1;
end;
begin
select change into cum_cur from hist$stats
where name in ('cumulative opened cursors',
'opened cursors cumulative') and test=counter;
exception when no_data_found then
cum_cur:=1;
end;
begin
select change into sort_dsk from hist$stats
where name = 'sorts (disk)' and test=counter;
exception when no_data_found then
sort_dsk:=1;
end;
begin
select change into sort_mem from hist$stats
where name = 'sorts (memory)' and test=counter;
exception when no_data_found then
sort_mem:=1;
end;
begin
select change into sort_row from hist$stats
where name = 'sorts (rows)' and test=counter;
exception when no_data_found then
sort_row:=1;
end;
begin
select change into usr_roll from hist$stats
where name = 'user rollbacks' and test=counter;
exception when no_data_found then
usr_roll:=1;
end;
begin
select change into usr_com from hist$stats
where name = 'user commits' and test=counter;
exception when no_data_found then
usr_com:=1;
ORACLE CONFIDENTIAL, Release 7.3 4-45
Shared Pool Internals Technical Reports Compendium, Volume I, 1995
end;
begin
select change into rec_calls from hist$stats
where name = 'recursive calls' and test=counter;
exception when no_data_found then
rec_calls:=1;
end;
begin
select change into usr_calls from hist$stats
where name = 'user calls' and test=counter;
exception when no_data_found then
usr_calls:=1;
end;
begin
select change into parse_cnt from hist$stats
where name = 'parse count' and test=counter;
exception when no_data_found then
parse_cnt:=1;
end;
begin
select (sum(pins) - sum(reloads))/sum(pins) into lcache from hist$lib
where test=counter;
exception when no_data_found then
lcache:=1;
end;
begin
select (sum(get_reqs) - sum(get_miss))/sum(get_reqs) into rcache
from hist$dc where test=counter;
exception when no_data_found then
rcache:=1;
end;
begin
select sum(misses)/sum(gets), sum(immed_miss)/sum(immed_gets)
into lch_miss, lch_imiss from hist$latches where test=counter;
exception when no_data_found then
lch_miss:=1;
lch_imiss:=1;
end;
begin
select change into dbwr_cross from hist$stats
where name = 'DBWR cross instance writes' and test=counter;
exception when no_data_found then
dbwr_cross:=1;
end;
begin
select change into sum_req from hist$stats
where name = 'summed dirty queue length' and test=counter;
exception when no_data_found then
sum_req:=1;
end;
begin
select change into dbfree from hist$stats
where name = 'DBWR free buffers found' and test=counter;
exception when no_data_found then
dbfree:=1;
4-46 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
end;
begin
select change into dbmkfree from hist$stats
where name = 'DBWR make free requests' and test=counter;
exception when no_data_found then
dbmkfree:=1;
end;
begin
select change into dbscan from hist$stats
where name = 'DBWR buffers scanned' and test=counter;
exception when no_data_found then
dbscan:=1;
end;
begin
select change into dblru from hist$stats
where name = 'DBWR lru scans' and test=counter;
exception when no_data_found then
dblru:=1;
end;
begin
select change into wr_req from hist$stats
where name = 'write requests' and test=counter;
exception when no_data_found then
wr_req:=1;
end;
select to_date(stats_gather_times, 'dd-MON-yy hh24:mi:ss') into bg_stats
from hist$dates where type='Start' and test=counter;
select to_date(stats_gather_times, 'dd-MON-yy hh24:mi:ss') into en_stats
from hist$dates where type='End' and test=counter;
select 60*24*(en_stats-bg_stats) into stat_time from dual;
Note: to gather information about a single cursor, replace above where clause with “ sql_text like ‘%cursor text%’ ”.
select s.sql_text, o.owner, o.name, o.namespace, o.sharable_mem, o.loads, locks, pins, kept
from v$db_object_cache o,v$object_dependency d, v$sql s
where o.owner =d.to_owner and o.name=d.to_name and
d.from_address= s.address and d.from_hash= s.hash_value;
Note: to gather information for a single cursor, add this line to the where clause: “and s.sql_text like ‘%cursor text%’ “.
INIT.ORA parameters
Here is a list of several init.ora parameters that will affect apace allocation and performance of the shared pool. Please refer
to Oracle 7 Server Reference or Oracle 7 DBA guide for more detailed explanation.
1. closed_cache_open_cursors
2. cursor_space_for_time
3. open_cursors
4. row_cache_cursors
5. session_cache_cursors
6. sequence_cache_entries
7. sequence _cache_hash_buckets
8. shared_pool_size
9. shared_pool_reserved_size
10. _kgl_latch_count
11. _kgl_bucket_count
On busy systems, the RDBMS may have difficulty finding a contiguous piece of memory to satisfy a large request for
memory. Because the RDBMS will search for and free currently unused memory, the search for this large piece of memory
may disrupt the behavior of the share pool, leading to more fragmentation and poor performance.
RDBMS 7.1.5 allows DBAs to reserve memory within the shared pool to satisfy these large allocations during RDBMS
operations such as PL/SQL compilation and trigger compilation. Smaller objects will not fragment the reserved list, helping
to ensure the reserved list will have large contiguous chunks of memory. Once the memory allocated from the reserved list is
freed, it returns to the reserved list.
The size of the reserved list, as well as the minimum size of the objects that can be allocated from the reserved list are
controlled via init.ora parameters: shared_pool_reserved_size and shared_pool_reserved_min_alloc.
shared_pool_reserved_size
The init.ora parameter shared_pool_reserved_size controls the amount of shared_pool_size reserved for large allocations. In
order to create a reserved list, shared_pool_reserved_size must be greater than shared_pool_reserved_min_alloc.
units : bytes
default: 0 (no reserved list)
minimum: > shared_pool_reserved_min_alloc
maximum: 1/2 shared_pool_size
shared_pool_reserved_min_alloc
The init.ora parameter shared_pool_reserved_min_alloc controls allocation for the reserved memory. Only allocations larger
than shared_pool_reserved_min_alloc are allowed to allocate space from the reserved list if a chunk of memory of sufficient
size is not found on the shared pool's free lists.
units : bytes
default: 5000
minimum: 5000
maximum: < shared_pool_reserved_size
The default value for shared_pool_reserved_min_alloc should be adequate for almost all systems.
Before the RDBMS signals the ORA-4031 error, it incrementally flushes unused objects from the shared pool until there is
sufficient memory to satisfy the allocation request. In most cases, incrementally flushing objects from the shared pool frees
enough memory for the allocation to complete succesfully. If the RDBMS signals an ORA-4031 error, it has flushed all
objects currently not in use on the system without finding a large enough piece of contiguous memory.
On a busy system, the larger the space allocation, the more likely the RDBMS will signal the ORA-4031 error. Flushing all
objects, however, will impact other users on the system, possibly causing a degradation in performance.
The aborted_request_threshold procedure allows the DBA to localize the impact the ORA-4031 error to the process that
couldn't allocate memory. The procedure takes a numeric value between 5000 and 2147483647, representing the size, in
bytes, of the threshold.
These columns of V$SHARED_POOL_RESERVED are only valid if the parameter shared_pool_reserved_size is set to a
valid value.
The next set of columns contain values which are valid even if shared_pool_reserved_size is not set.
REQUEST_FAILURES is the number of times that no memory was found to satisfy a request (e.g., number of times ORA-
4031 occurred)
4-52 ORACLE CONFIDENTIAL , Release 7.3
Technical Reports Compendium, Volume I, 1996 Shared Pool Internals
LAST_FAILURE_SIZE is the request size of the last failed request (e.g., the request size of last ORA-4031).
ABORTED_REQUEST_THRESHOLD is the minimum size of a request which will signal an ORA-4031 error without
flushing objects. See the procedure aborted_request_threshold described above.
LAST_ABORTED_SIZE is the last size of the request which returned an ORA-4031 error without flushing objects from the
LRU list.
Tuning shared_pool_reserved_size
Ideally, shared_pool_reserved_size should be made large enough to satisfy any request scanning for memory on the reserved
list without flushing objects from the shared pool. The amount of operating system memory, however, may constrain the
size of the SGA, and therefore the size of the shared pool such that this is not a feasible goal.
If the DBA has a system with ample free memory to increase his SGA, the goal is to have:
REQUEST_MISS =0
If neither of these goals are met, increase shared_pool_reserved_size; the DBA also needs to increase shared_pool_size by
the same amount, since the reserved list is taken from the shared pool.
The DBA has two options, depending on his SGA size constraints:
The first option will increase the amount of memory available on the reserved list without impacting users not allocating
memory from the reserved list. The second options reduces the number of allocations allowed to use memory from the
reserved list; doing so, however, will increase normal shared pool perhaps impacting other users on the system.
o Decrease shared_pool_reserved_size
o Decrease shared_pool_reserved_min_alloc (if not the default
value)
Then the DBA has two options if he has enabled the reserved list:
o Decrease shared_pool_reserved_size
o Decrease shared_pool_reserved_min_alloc (if set larger than the default)
o Increase shared_pool_size
Procedure free_unused_memory
This text is also in the specification for this procedure in dbmsutil.sql. It is part of package dbms_session.
Procedure free_unused_memory --
Procedure for users to reclaim unused memory after performing operations requiring large amounts of memory (where large
is >100K). Note that this procedure should only be used in cases where memory is at a premium.
One can monitor user memory by tracking the statistics "session uga memory" and "session pga memory" in the
v$sesstat/v$statname fixed views. Monitoring these statistics will also show how much memory this procedure has freed.
The behavior of this procedure depends upon the configuration of the server operating on behalf of the client:
In order to free memory using this procedure, the memory must not be in use.
Once an operation allocates memory, only the same type of operation can reuse the allocated memory. For example, once
memory is allocated for sort, even if the sort is complete and the memory is no longer in use, only another sort can reuse the
sort-allocated memory. For both sort and compilation, after the operation is complete, the memory is no longer in use and
the user can invoke this procedure to free the unused memory.
An indexed table implicitly allocates memory to store values assigned to the indexed table's elements. Thus, the more
elements in an indexed table, the more memory the RDBMS allocates to the indexed table. As long as there are elements
within the indexed table, the memory associated with an indexed table is in use.
The scope of indexed tables determines how long their memory is in use. Indexed tables declared globally are indexed tables
declared in packages or package bodies. They allocate memory from session memory. For an indexed table declared
globally, the memory will remain in use for the lifetime of a user's login (lifetime of a user's session), and is freed after the
user disconnects from ORACLE.
Indexed tables declared locally are indexed tables declared within functions, procedures, or anonymous blocks. These
indexed tables allocate memory from PGA memory. For an indexed table declared locally, the memory will remain in use
for as long as the user is still executing the procedure, function, or anonymous block in which the indexed table is declared.
After the procedure, function, or anonymous block is finished executing, the memory is then available for other locally
declared indexed tables to use (i.e., the memory is no longer in use).
Assigning an uninitialized, "empty," indexed table to an existing index table is a method to explicitly re-initialize the indexed
table and the memory associated with the indexed table. After this operation, the memory associated with the indexed table
will no longer be in use, making it available to be freed by calling this procedure. This method is particularly useful on
indexed tables declared globally which can grow during the lifetime of a user's session, as long as the user no longer needs
the contents of the indexed table.
The memory rules associated with an indexed table's scope still apply; this method and this procedure, however, allow users
to intervene and to explictly free the memory associated with an indexed table.
The PL/SQL fragment below illustrates the method and the use of procedure free_unused_user_memory.
Performance Implication:
introduced in 7.2
On a site with heavy memory usage (things are getting kicked out of the hared pool consistently) the processes requesting
memory can get into a state where they keep interupting each other's searches through the LRU list looking for chunks to
free. This happens because the process currently holding the shared pool latch modifies the list. The culprit of this problem
are KGL handles of objects that have dependencies on ther objects protected under different KGL latches. As these types of
handles build up on the LRU list, the problem progressively gets worse until the system hangs. The system may be freed up
every so often when someone gets a parent latch and frees up some these problem handles. The real fix is going into 7.3.
Any backport of this fix to 7.2 (note the problem is introduced in 7.2) should also include the fix to bug #318582.
fixed in 7.3
fix checked into 7.3.2 kgl2.c rev. 1.102
bug 318582 Handles and memory (shared pool) can not be freed resulting ora 4031
Introduced in 7.2.2
Problem:
Sites may run into this if they are not sharing cursors. Here is an explanation of the problem: The library cache now has two
types of latches, the parent latch and one or more child latches. When the parent latch is held, all child latches are effectively
held. Each library cache object is made up of a KGL handle and number of "heaps", the first being heap 0. Heap 0 is special
because it contains control information about the other heaps making up the library cache object. When the heap manager
requests that some space be freed, it asks that the handle, heap 0 , and other heaps be freed individually. Before calling back
to the library cache, the heap manager has to grab the appropriate child latch, or in some cases the parent latch,
corresponding the memory to be freed. Here is the new behavior in KGL starting in 7.2...
1. When a handle is requested to be freed, it will not be freed if the parent latch is not held and heap 0 still is loaded.
2. heap 0 can only be freed if the parent latch is obtained.
3. If a child latch is held when the memory request that ultimately causes some memory to be freed is made, the
parent latch cannot be obtained.
.
In most cases, heap 0 and the handle make up a small percentage of the memory in the library cache. But when cursors are
not shared, as in the testcase, these objects can use up significant amounts of memory. Now here is the situation where the
error can occur. If there are enough handles and heap 0's in memory such that a memory request can not be satisified without
kicking some these objects out AND the memory request is made with a child latch held, the following things happen.
1. The Heap 0's cannot be freed because the child latch prevents the heap manager from getting the parent latch.
2. The handles can't be freed because the heap 0's are still around.
3. No memory is freed and the ORa-4031 error occurs.
.
Heap 0 needs the parent latch because it might have references to other objects in the library cache protected by different
child latches. However, most objects do not have these references. The fix is to keep track of the objects that have foreign
latch references and only require the parent latch to free heap 0 for those objects.
Fixed in 7.2.3
kgl.c rev. 1.180.720.13 kgl2.c rev. 1.75.720.16
Problem.
Hashing on the first and last 256 bytes improved performance but did not catch all occurences of sql hashing to the same
bucket. A new patch which uses the old hashing algorithm has been given to osd-esc. This new hashing function is filling
up several hash links to quickly even though the objects are different. In some cases the sql they are running is hashing to
the same value. This is where the first 64 bytes and last 64 bytes of each statement are identical.
debugging techniques:
alter session set events 'immediate trace name library_cache level 2'; will show hundreds of '*' in some buckets. sults in
some buckets filling with thousands of statements.
Fix:
@ The 7.2.2 fix for this bug is bundled with fixes for 318582 and 322904
@ Please refer to 322904 for 7.2.2 tag
bug 257514 triggers referencing rowids may reference wrong rowid if done recursively
Introduced: 7.1
Problem:
The bug occurs because the same trigger is being executed at different pga depths. We make the decision not to rebind the
pgaoer->riddef in kxtini() when the cursor holding the trigger text has already been parsed once. This is fine if the pgaoer
does not change. But each level of pga depth has its own pgaoer structure. The problem is that we are not rebinding to the
proper memory location on the stack of pgaoer structures. A higher level example is that if a trigger referencing rowid (or
setting it) is fired by directly issuing a DML statement and then is later fired in the same session by the same SQL
statmentment, but recursively, it may see the wrong rowid.
Fix:
Regression testing is occurring.
Does not reproduce in 7.3
bugs 192829 and 205976 Pinning Procedures and Triggers with dbms_shared_pool procedure
fixed in 7.1.6
Fix:
From 7.0.15, the compiled code for a package was split into more than one piece, each piece being only about 12K in size.
So, the 64K restriction introduced in 7.0.13 was lifted. Large packages (>100K) may still have problems compiling.