Wasting expensive RAM resources is a bad thing and the goal of the Oracle DBA is to maximize the SGA

region with intelligent SGA sizing techniques in relation tosga_max_size. Beware of AMM:

There are reports where automatic memory management (implemented by setting the sga_target and sga_max_size parameters) can cause performance problem, especially for databases with undersized RAM resources.

Oracle (AMM) resize operations can hurt performance.

In 11g and beyond, Oracle automatic memory management is configured using the memory_target and memory_max_target initialization parameters. The memory_target parameter specifies the amount of shared memory available for Oracle to use when dynamically controlling the SGA and PGA. Thememory_max_target AMM parameter specifies the max size thatmemory_target may take.

Note that larger shops many forgo AMM and manually size their shared_pool_size anddb_cache_size. A human DBA can frequently do a better job in sizing the RAM pools, especially when using predictive tuning techniques, which anticipate changes in workload and adjust the pools before a shortage occurs. If you only have Oracle on the server, start by reserving 10% of RAM for Linux or 20% or RAM for Windows. With whatever RAM is left-over you allocate to SGA and PGA:

sga_max_size -- This 10g parameter sets the hard limit up to which sga_targetcan dynamically adjust sizes. At database start time, Oracle will allocatesga_max_size in RAM (or set sga_max_size to the sum of the existing pool sizes), so in order not to waste RAM it may be a good idea to havesga_max_size and sga_target at the same value, but there may be times when you want to have the capability to adjust for peak loads. By setting this parameter higher than sga_target, you allow dynamic adjustment of the sga_targetparameter. For related notes, see MOSC Notes 295626.1, 396940.1, 270065.1 and 256913.1. sga_target -- This parameter is for Oracle Database 10g ONLY and reflects the total size of memory footprint a SGA can consume. It includes in its boundaries the fixed SGA and other internal allocations, the (redo) log buffers, the shared pool, Java pool, streams pool, buffer cache, keep/recycle caches, and if they are specified, the nonstandard block size caches.

There are reported cases where sga_target cannot exceed two gigabytes. This is because of the need to edit edit the /etc/security/limits file and look for Soft DATA segment, it should have the value of -1 (Unlimited).
AIX Note for sga_target:

What should be my sga_max_size? The default that oracle suggests at the DBCA is 40%, however, the appropriate size for this parameter depends on the space requirements of your actual SGA. Your application requirements will determine this, things like the amount of concurrent users and the activities they are really performing. These types of things can only be monitored when running processes. There is no formula for an exact value of these parameters. The best thing you can do is to tune based the advisors suggestions. Memory consumption should be checked when your real workload is in progress, andsga_max_size tuning should be performed using Enterprise Manager to tune based on realistic metrics. You can see the Oracle memory segments with the ipcs -pmb command.

Question: Can you please explain the Oracle sga_max_size parameter? Answer: Wasting expensive RAM resources is a bad thing and the goal of the Oracle DBA is to maximize the SGA region with intelligent SGA sizing techniques in relation tosga_max_size. The sga_max_size was introduced for databases where the DBA did not know how to optimize the SGA pools manually. Today, a competent DBA can do a better job that automatic shared memory management, and large Oracle database still use manual SGA pool sizing. The Oracle sga_max_size parameter sets the hard limit up to which sga_target can dynamically adjust sizes. Usually, sga_max_size and sga_target will be the same value, but there may be times when you want to have the capability to adjust for peak loads. By setting the Oracle sga_max_size parameter higher than sga_target, you allow dynamic adjustment of the sga_target parameter.
There are reported cases where sga_target cannot exceed two gigabytes. This is because of the need to edit the /etc/security/limitsfile and look for Soft DATA segment, it should have the value of -1 (Unlimited).
AIX Note for sga_target: Beware OF AMM:

There are reports where automatic memory management (implemented by setting the sga_targetand sga_max_size parameters) can cause performance problem, especially for databases with undersized RAM resources. See Oracle (AMM) resize operations can hurt performance. In sum, all large mission-critical databases should have an expert DBA manually configure and optimize their SGA pool sizes. Also, larger shops many forgo AMM and manually size their shared_pool_size anddb_cache_size. A human DBA can frequently do a better job in sizing the RAM pools, especially when using predictive tuning techniques, which anticipate changes in workload and adjust the pools before a shortage occurs.

Note that sga_max_size is for Oracle 10g only, and in 11g and beyond, Oracle automatic memory management is configured using the memory_target and memory_max_target initialization parameters. The memory_target parameter specifies the amount of shared memory available for Oracle to use when dynamically controlling the SGA and PGA. The memory_max_target AMM parameter specifies the max size that memory_target may take. See my additional notes on Oracle sga_max_size: I'm consolidating 15 Oracle instances onto a single server and I need advice on the proper way to size my SGA for each instance.

Answer:
In a shared environment, it's important to observe the optimal sizes for eachshared_pool_size (v$shared_pool_advice) and the optimal data buffer size (usingv$db_cache_advice). I have complete Oracle SGA sizing details in my book "Oracle Tuning: The Definitive Reference".
Make sure to tread these detailed notes on SGA sizing.

2007 PGA Update: Oracle technology is constantly changing, so don't miss my new notes on updates to Oracle PGA behavior. Also see these important notes on over-riding the Oracle PGA defaults. In 10g, you may consider using Automatic Memory Management (AMM) and set thesga_max_size parameter to give the total SGA allocation. Wasting expensive RAM resources is a bad thing and the goal of the Oracle DBA is to maximize the SGA region with intelligent SGA sizing techniques. If you only have Oracle on the server, start by reserving 10% of RAM for Linux or 20% or RAM for Windows. With whatever RAM is left-over:

SGA Sizing - Optimize the instance by determining the optimal size fordb_cache_size, shared_pool_size, etc. PGA Sizing - Determine the optimal total RAM for PGA regions (pga_aggregate_target) to minimize disk sorts and maximize hash joins

Sizing your SGA

The SGA sizing tasks include optimizing settings for sga_max_size (not recommended for large databases where manual allocation is more effective) and the various configuration parameters for db_cache_size, db_xk_cache_size, shared_pool_size, large_pool_size, and other memory objects give the dba better control of system global area (SGA) component sizing. Some parameters to consider in SGA sizing include:

sga_max_size -- This parameter sets the hard limit up to which sga_target can dynamically adjust sizes. Usually, sga_max_size and sga_target will be the same value, but there may be times when you want to have the capability to adjust for peak loads. By setting this parameter higher than sga_target, you allow dynamic adjustment of the sga_target parameter. sga_target -- This parameter is new in Oracle Database 10g and reflects the total size of memory footprint a SGA can consume. It includes in its boundaries the fixed SGA and other internal allocations, the (redo) log buffers, the shared pool, Java pool, streams pool, buffer cache, keep/recycle caches, and if they are specified, the nonstandard block size caches.

SGA Sizing on a dedicated server

For dedicated Oracle servers, the maximum total RAM SGA size can be computed as follows: OS Reserved RAM This is RAM required to run the OS kernel and system functions, 20% of total RAM for MS-Windows, and 10% of total RAM for UNIX/Linux Oracle Database Connections RAM Each Oracle connection requires OS RAM regions for sorting and hash joins. (This does not apply when using the Oracle multithreaded server or pga_aggregate_target .) The maximum amount of RAM required for a session is as follows: 2 megabytes RAM session overhead

+ sort_area_size + hash_area_size Oracle SGA Sizing for RAM This is determined by the Oracle parameter settings. The total is easily found by either the show sga command or the value of thesga_max_size parameter.
We should subtract 20 percent from the total available RAM to allow for MS-Windows overhead. Windows uses RAM resources even when idle, and the 20 percent deduction is necessary to get the real free RAM on an idle server. Once the amount of RAM on the server is known, we will be in a position to size the Oracle SGA for optimal RAM usage. Oracle RAM on MS-Windows To see how much RAM you have on your MS-Windows server, you can go to start --> settings > control panel --> system, and click on the "general" tab (refer to Figure 1). Here we see that this server has 1,250 megabytes of RAM.

The MS-windows system display screen. Now that we know how to tell the size of our MS-Windows RAM and the size of the SGA, we have to consider the RAM usage for Oracle connections. Monitoring Server Resources in MS-Windows In MS-Windows we can use the performance manager screen to observe the resource consumption of the Oracle Windows server (refer to Figure 2). The performance manager is hidden deep inside the Windows menus, but can be found by following start > settings > control panel > administrative tools > performance.

The MS-Windows server performance monitor. The MS-Windows performance monitor plots three metrics:

Green (CPU) - This is the percentage of CPU resources consumed Yellow (RAM) - This is the number of RAM pages per seconds used Blue (DISK) - This is the disk I/O queue length percentage

Let's take a closer look at the MS-Windows performance monitor. Figure 2 is a time-based snapshot of an Oracle databases resource consumption at startup time. These lines form signatures (known usage patterns) that reveals some interesting patterns inside Oracle: 1. RAM Usage The yellow line is RAM usage, and we see the first spike in the RAM when the SGA is allocated and a short spike in RAM as the database is mounted. 2. DISK Usage The blue line is the disk I/O, and we see the disk I/O activity peg at the point where we mount the database. This is because Oracle must touch every data file header to read the system change number (SCN). 3. CPU Usage The green line is CPU and it is interesting to note that the CPU never goes above 50% during Oracle database startup.

Conclusion For PGA sizing, we need to know the high water mark (HWM) of Oracle connections and the average RAM needs to minimize disk sorts and maximize hash joins. The high water mark of connected Oracle sessions can be determined in several ways. One popular method uses Oracle login and logoff system-level triggers to record sessions in a statistics table. Another method uses Oracle STATSPACK to display the values from the stats$sysstat table, or the v$resource_limit view (only after release 8.1.7, because of a bug). In sum, SGA sizing is relatively straightforward push-pull of the competing RAM needs of the internal SGA pools (buffer, library cache), the external RAM for PGA, and the OS overhead. If you like Oracle tuning, you may enjoy my new book "Oracle Tuning: The Definitive Reference", over 900 pages of BC's favorite tuning tips & scripts. You can buy it direct from the publisher for 30%-off and get instant access to the code depot of Oracle tuning scripts.
n this post, I am going to be demonstrating the basics of working of the sga_max_size and sga_target and also to cover some myths of playing around with these parameters. To begin with, sga_max_size is set in the spfile to a value of 200M and then I try to set the sga_target and the story unfolds as below.

SQL> sho parameter spfile NAME TYPE VALUE ------------------------------------ ----------- ----------------------------spfile string /opt/app/oracle/product/10.2.0 /dbs/spfileprod_db.ora SQL> sho parameter sga_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 0

SQL> sho sga

Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers

209715200 1977560 155194152 50331648 2211840

bytes bytes bytes bytes bytes

SQL> sho parameter shared_pool_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------shared_pool_size big integer 84M SQL> sho parameter db_cache_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------db_cache_size big integer 48M SQL> sho parameter large NAME TYPE VALUE ------------------------------------ ----------- ----------------------------large_pool_size big integer 0 SQL> sho parameter java_pool_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------java_pool_size big integer 24M SQL> sho parameter streams_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------streams_pool_size big integer 0

Now, trying to set the sga_target to a value of 210M which is greater than that of sga_max_size gives out an error of ORA-02097 and ORA-00823, this shows that the sga_target can never be greater than the sga_max_size.

SQL> alter system set sga_target=210M scope=both; alter system set sga_target=210M scope=both *

ERROR at line 1: ORA-02097: parameter cannot be modified because specified value is invalid ORA-00823: Specified value of sga_target greater than sga_max_size

However, sga_target can be set less than or equal to sga_max_size.

SQL> alter system set sga_target=200M scope=both; System altered. SQL> sho parameter sga_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 200M

But an interesting point to be noted in here is that as soon as the sga_target is set to a non-zero value, it starts affecting the memory areas of the SGA which is evident from the change in the value of Database Buffers (db_cache_size) from 50331648 (48M) from the previous execution of SHO SGA to 92274688 (88M), while every other memory area remains unchanged.

SQL> sho sga Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers 209715200 1977560 113251112 92274688 2211840 bytes bytes bytes bytes bytes

SQL>

sho parameter db_Cache_size

NAME TYPE VALUE ------------------------------------ ----------- ----------------------------db_cache_size big integer 88M SQL> sho parameter shared_pool_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------shared_pool_size big integer 84M

SQL> sho parameter java_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------java_pool_size big integer 24M SQL> sho parameter streams_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------streams_pool_size big integer 0 SQL> sho parameter large_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------large_pool_size big integer 0

Let's try to unset the sga_target and see if Database Buffers (db_cache_size) reverts back to what it was, but it does not.

SQL> alter system set sga_target=0 scope=both; System altered. SQL> sho sga Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers SQL> sho parameter sga_ 209715200 1977560 113251112 92274688 2211840 bytes bytes bytes bytes bytes

NAME --------------------------------------sga_max_size sga_target

TYPE VALUE ----------- -------------------------big integer 200M big integer 0

SQL> NAME

sho parameter db_Cache_size TYPE VALUE

------------------------------------ ----------- ----------------------------db_cache_size big integer 88M

It was hoped that bouncing the database would make a difference, on the contrary it did not. It is another discussion as to why the Database Buffers (db_Cache_size) does not change back to what it was even after unsetting the sga_target and bouncing the database, let's not get into the nitty-gritty of that.

SQL> sho sga Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers 209715200 1977560 113251112 92274688 2211840 bytes bytes bytes bytes bytes

SQL> sho parameter db_Cache_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------db_cache_size big integer 88M SQL> shut immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers Database mounted. Database opened. 209715200 1977560 113251112 92274688 2211840 bytes bytes bytes bytes bytes

SQL> sho parameter sga_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 0

SQL> sho parameter db_Cache_size

NAME TYPE VALUE ------------------------------------ ----------- ----------------------------db_cache_size big integer 88M

The Oracle server does not let you to resize the sga_max_size on the fly since it is not a dynamically changeable parameter. Let's now change sga_target to the value of sga_max_size. After this I will comment the sga_max_size and add sga_target=200M in the pfile, convert it into spfile and bounce the database.

SQL> sho parameter sga_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 0

SQL> alter system set sga_max_size=0 scope=both; alter system set sga_max_size=0 scope=both * ERROR at line 1: ORA-02095: specified initialization parameter cannot be modified

SQL> alter system set sga_target=200M scope=both; System altered. SQL> sho parameter sgA_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 200M

### MAIN SGA PARAMS ### ###sga_max_size=200M sga_target=200M SQL> create spfile from pfile; File created. SQL> startup ORACLE instance started.

Total System Global Area Fixed Size Variable Size Database Buffers Redo Buffers Database mounted. Database opened. SQL> sho parameter sga_

209715200 1977528 75502408 130023424 2211840

bytes bytes bytes bytes bytes

NAME --------------------------------------sga_max_size sga_target SQL> sho parameter db_cache_size

TYPE VALUE ----------- -------------------------big integer 200M big integer 200M

NAME TYPE VALUE ------------------------------------ ----------- ----------------------------db_cache_size big integer 0 SQL> sho parameter shared_pool_size NAME TYPE VALUE ------------------------------------ ----------- ----------------------------shared_pool_size big integer 0 SQL> sho parameter large_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------large_pool_size big integer 0 SQL> sho parameter java_pool_s NAME TYPE VALUE ------------------------------------ ----------- ----------------------------java_pool_size big integer 0 SQL> sho parameter streams_pool NAME TYPE VALUE ------------------------------------ ----------- ----------------------------streams_pool_size big integer 0 SQL>

It now shows that as soon as the sga_target is set and set_max_size is unset, all the other memory area related parameters get reset to a value of zero(0) - making it clear that the Automatic Shared Memory Management is fully functional and that Memory Manager (MMAN) background process is actively doing its job. Also has the Database Buffers (db_cache_size) taken a higher value of 130023424 (124M) against the previous 88M.

There is yet another point to be noted here, besides sga_max_size being commented in spfile - it takes a value equal to the sga_target.

SQL> sho parameter sga_ NAME --------------------------------------sga_max_size sga_target TYPE VALUE ----------- -------------------------big integer 200M big integer 200M

UNIX$> grep -i "sga" initprod_db.ora ###sga_max_size=200M sga_target=200M

UNIX$> grep -i "sga" $ORACLE_HOME/dbs/spfileprod_db.ora *.sga_target=200M

The conclusions that we can draw from these little experiments are listed below. 1) sga_max_size cannot be changed without bouncing the database. 2) sga_target can be changed on the fly without bouncing the database but it can never be set to a value greater than the sga_max_size. 3) If the sga_max_size is not set in either of the pfile or spfile and the sga_target is set then the sga_max_size takes the value of sga_target when you do a sho parameter sga_

How to increase SGA_MAX_SIZE in Oracle

Phillip Bracken, Database Architect, Wachovia Corporate & Investment Banking

How can I increase SGA_MAX_SIZE in Oracle ? I got the error "ORA-03113: end-of-file on communication channel" during startup. Please help me with the steps to increase the SGA. I'm connected to Oracle9i Enterprise Edition Release 9.0.1.1.1 (Production) with the partitioning option.
SQL> connect sys as sysdba Enter password: ****** Connected. SQL> show sga; Total

System Global Area 147615836 bytes Fixed Size 282716 bytes Variable Size 113246208 bytes Database Buffers 33554432 bytes Redo Buffers 532480 bytes SQL> show parameter spfile NAME TYPE VALUE ------------------------------------ -----------

------------------------------ spfile string %ORACLE_HOME%DATABASESPFILE% ORACLE_SID%.ORA SQL> create pfile from spfile; set sga_max_size=20m scope=spfile; File created. SQL> alter system

System altered.

SQL> shutdown immediate;

Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORA03113: end-of-file on communication channel

Oracle 11g: What Happens When Setting SGA_MAX_SIZE less than SGA_TARGET?
April 14, 2011No comments

Have you ever hosed up your memory parameters such that you cannot get Oracle to even enter the startup nomount state? Here Ill do just that by setting sga_max_size to a value less than sga_target, and then trying to start the database. First a sanity check so you can see the memory parameters. In this case I am using ASMM (automatic shared memory management), and not the new 11g AMM (automatic memory management) features.

sqlplus / as sysdba SQL> show parameter memory_max_target

NAME ----memory_max_target

TYPE

VALUE

------------------------------------ ----------- ------------------------big integer 0

SQL> show parameter sga_max_size

NAME ----sga_max_size

TYPE

VALUE

------------------------------------ ----------- ------------------------big integer 1056M

SQL> show parameter sga_target

NAME ----sga_target

TYPE

VALUE

------------------------------------ ----------- ------------------------big integer 1G

SQL> show parameter pga_aggregate_target;

NAME ----pga_aggregate_target

TYPE

VALUE

------------------------------------ ----------- ------------------------big integer 350M

OK, lets have some fun. Well set our parameters to something that doesnt make sense by setting sga_max_size to 0. Then well shutdown immediate.

SQL> alter system set sga_max_size=0 scope=spfile; System altered.

SQL> shutdown immediate; Database closed. Database dismounted. ORACLE instance shut down.
Now well start our DB back upand notice that we cant! Panic! We get the ORA-00823 error.

SQL> startup; ORA-01078: failure in processing system parameters ORA-00823: Specified value of sga_target greater than sga_max_size

Uh ohnot even a nomount works. Makes sense though. Nomount reads the pfile and allocates the memory for the instance, so no wonder it is failing there if we cant properly allocate memory. I guess we cant just fire this thing into nomount state and run our alter system commands.

SQL> startup nomount ORA-01078: failure in processing system parameters ORA-00823: Specified value of sga_target greater than sga_max_size
OK, no sweat. Ill create my text based pfile from the spfile, edit this and be on my way.

SQL> create pfile from spfile; create pfile from spfile * ERROR at line 1: ORA-01565: error in identifying file '?/dbs/spfile@.ora' ORA-27037: unable to obtain file status Linux-x86_64 Error: 2: No such file or directory Additional information: 3
Huh? Wheres my spfile? Can you feel the hairs stand on the back of you neck as the panic sets in that your DB is hosed? No worried, the spfile has to be there somewhere. After all, this was a previously running system. Lets looks in the alert.log for how my instance booted previously.

cd /u01/app/oracle/diag/rdbms/orcl/orcl/trace vi alert_orcl.log
A quick scan through the alert log reveals that there is indeed an spfile somewhere out in the universe:

spfile

= "+DATA/orcl/spfileorcl.ora"

Ahh, thats right. This instance is using ASM (automatic storage management). To create that pfile I need to tell the command exactly where that spfile is:

SQL> create pfile from spfile='+DATA/orcl/spfileorcl.ora'; File created.

Now were cooking! Theres the init.ora file we need:

[oracle@localhost dbs]$ cd $ORACLE_HOME/dbs [oracle@localhost dbs]$ ls -lart initorcl.ora -rw-r----- 1 oracle oinstall 1061 Apr 14 09:20 initorcl.ora

And we happily edit it in vi and change sga_max_size=0 to sga_max_size=1056M Now our instance starts!

SQL> startup nomount pfile='$ORACLE_HOME/dbs/initorcl.ora'; ORACLE instance started. Total System Global Area 1102344192 bytes Fixed Size Variable Size Database Buffers Redo Buffers 2212576 bytes 335547680 bytes 754974720 bytes 9609216 bytes

Overwrite our spfile with the new values from our pfile:

SQL> create spfile='+DATA/orcl/spfileorcl.ora' from pfile; File created.

And then shutdown the database and restart. Voila, everything is back to life:

SQL> shutdown; ORA-01507: database not mounted ORACLE instance shut down.

SQL> startup; ORACLE instance started.

Total System Global Area 1102344192 bytes Fixed Size Variable Size Database Buffers Redo Buffers Database mounted. Database opened. 2212576 bytes 335547680 bytes 754974720 bytes 9609216 bytes

ga_target vs sga_max_size
SGA_TARGET specifies the total size of all SGA components. If the SGA_TARGET is set, then the following memory pools are automatically sized : Buffer cache (DB_CACHE_SIZE) Shared pool (SHARED_POOL_SIZE) Large pool (LARGE_POOL_SIZE) Java pool ( JAVA_POOL_SIZE) SGA_MAX_SIZE specifies the hard limit upto which the SGA_TARGET can dynamically grow. While executing DBCA, Oracle suggests that the estimated SGA_MAX_SIZE is to set aside 40% of memory. However, it should be set according to your requirement that

depends on multiple factors such as no of concurrent users, volume of transactions and growth rate of database. Under normal operation, you can set the SGA_MAX_SIZE equals to the SGA_TARGET. Sometimes, we need to perform some extra-heavy batch processing jobs that leads to more SGA size. At this circumstance, you must have capability to adjust for peak loads. That is why, you set hard limit for your SGA_MAX_SIZE. SGA_MAX_SIZE cannot be changed dynamically without bouncing the database whereas SGA_TARGET can be changed dynamically without bouncing the database. If you try to modify SGA_MAX_SIZE dynamically, you will get an error of ORA-02095: specified initialization parameter cannot be modified. SGA_TARGET can never be greater than SGA_MAX_SIZE. If you try to set the SGA_TARGET to a value which is greater than that of SGA_MAX_SIZE, then Oracle will throw an error of ORA-02097: parameter cannot be modified because specified value is invalid ORA-00823: specified value of SGA_TARGET greater than SGA_MAX_SIZE. If the SGA_MAX_SIZE is not set and the SGA_TARGET is set, then the SGA_MAX_SIZE takes the value of SGA_TARGET. If you set the SGA_MAX_SIZE greater than your server memory capacity and bounce the database, you will get an error of ORA-27102 : out of memory SVR4 Error : 12 : not enough space

Memory Tuning - How OS Page Sizes Can Impact Oracle Performance

Theory Modern operating systems provide each process it's own virtual address space that is potentially larger than physical memory. It does this by mapping the virtual addresses onto actual physical memory pages. The following diagram shows how large virtual address spaces are mapped onto small physical memory.

The virtual to physical relationship is maintained in mapping tables. Linux maintains a separate map for each process. The map contains one entry for each page where the size of the page is set by the operating system. The CPU attempts to speed up the mapping of virtual memory to physical addresses by caching a subset of the mapping entries in its internal cache. Modern large memory systems and the resulting large processes have increased the size of memory mapping tables faster than the cpu cache sizes have increased. The processor cache is usually very small with only maybe 100 entries in may processors. This mismatch causes faults that can have significant performance implications. System Analysis Linux uses physical memory to maintain the process memory mapping tables and swaps those mapping tables in and out of the processor cache on context switches and on memory page table faults. This physical memory is unavailable for other uses essentially reducing the amount of memory available to the rest of the system. The amount of memory used for page tables is the sum of all the virtual process sizes / the page table size multiplied by the size of a page table entry. So a system with a 20GB Oracle SGA uses 50MB of physical memory for page tables just for the SGA: 20GB Process Size / 4K page size * 10 = 50Megabytes. (5 million entries) Every connection to an Oracle database runs in its own process that has direct access to the Oracle DB through shared memory. This doesn't seem too bad until you consider that page tables for the entire shared memory space are duplicated across each process. So a 20GB Oracle process with 200 database connections would 5GB for the page tables just for the SGA portion of each process: 20GB Process Size/ 4K page size * 10 entry size * 200 connections = 10GB. (1 Billion page table entries) This means a 32GB Oracle server with a 20GB SGA and 200 connections uses 30% of the physical memory just for the page tables. It also means that the oracle processes, the OS and other processes no longer fit into memory resulting in possibly significant paging. You cannot rely on ps commands or other process monitorswhen investigating this because Linux always uses all available memory allocating any "extra" to file system buffers and other needs. You can get your actual page table usage in your system by dumping the contents of /etc/procinfo with cat /etc/procinfo.

Linux provides the ability to change the page size from 4KB to 2MB potentially returning large amounts of memory to the swappable pool. Calculations show that converting just the Oracle SGA in the previous example would save almost 10GB. 20GB Process size / 2MB page size * 10 entry size * 200 connections = 20MB (2 million entries) Why Aren't Huge Pages the Default? Huge (2MB) pages are different from the normal 4K pages in that they are locked into memory and do not swap. This means that extreme care must be taken when selecting the number and size of the huge page space. The Huge Page space should be large enough to hold the Oracle SGA while leaving enough other memory to run the non-SGA client code, the operating system and other processes. Teams must understand their memory needs so that htey can set the correct number of huge pages. Production Example I was talking with a colleague about their system and we did some back of the napkin calculations. They had an Oracle RAC with two 72GB servers each with 500 database connections. This ran a little slower than expected when both machines were up but their real problem was that they were unable to fail over to a single node even though they had plenty of CPU capacity. The following calculations show why. Before Page Table Entries (SGA) = 35GB / 4KB = 8.75MB Page Table Entries (1000 connections) = 1000 * 8.75MB = 8.75GB Memory (1000 connections) = 10B * 8.75G = 87.5GB

After Page Table Entries (SGA) = 35GB / 2MB = 17.5KB Page Table Entries (1000 connections) = 1000 * 17.5KB = 17.5MB Memory (1000 connections) = 10B * 17.5M = 175MB

The team converted 35GB of memory to Huge Pages so that the entire SGA fit into the huge pages. This returned 40GB of free memory to each machine and made it possible fall back to a single machine when they had planned or unplanned outages on a single server. A smaller system at another site recovered 7GB of physical memory on their 32GB servers by converting their 20GB oracle SGA to Huge Pages. This ended all their system paging. Recap Intelligent conversion from small memory pages to large memory pages modern modern large RAM systems can have significant performance implications: Larger page sizes result in fewer page table entries in the CPU cache. This results in fewer cache misses and less CPU stalling.

 The memory savings resulting from the 1/500 reduction in page table entries is multiplied by the number of processes sharing the same shared memory. Every team with large Oracle Databases should investigate the use of Huge Pages no matter what their operating system.

Configuring HugePages for Oracle on Linux (x86-64)

For large SGA sizes, HugePages can give substantial benefits in virtual memory management. Without HugePages, the memory of the SGA is divided into 4K pages, which have to be managed by the Linux kernel. Using HugePages, the page size can be increased to anything between 2MB and 256MB, thereby reducing the total number of pages to be managed by the kernel and therefore reducing the amount of memory required to hold the page table in memory. In addition to these changes, the memory associated with HugePages can not be swapped out, which forces the SGA to stay memory resident. The savings in memory and the effort of page management make HugePages pretty much mandatory for Oracle 11g systems running on x86-64 architectures. Note. Automatic Memory Management (AMM) is not compatible with HugePages, so apart from ASM instances and small unimportant databases, you will probably have no need for AMM on a real system. Instead, Automatic Shared Memory Management and Automatic PGA Management should be used as they are compatible with HugePages. Run the following command to determine the current HugePage usage. The default HugePage size is 2MB on Oracle Linux 5.x and as you can see from the output below, by default no HugePages are defined. $ grep Huge /proc/meminfo HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB $

Depending on the size of your SGA, you may wish to increase the value of Hugepagesize. Create a file called "hugepages_setting.sh" with the following contents. #!/bin/bash # # hugepages_settings.sh # # Linux bash script to compute values for the # recommended HugePages/HugeTLB configuration # # Note: This script does calculation for all shared memory # segments available when the script is run, no matter it # is an Oracle RDBMS shared memory segment or not. # Check for the kernel version KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'` # Find out the HugePage size HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`

# Start from 1 pages to be on the safe side and guarantee 1 free HugePage NUM_PG=1 # Cumulative number of pages required to handle the running shared memory segments for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"` do MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q` if [ $MIN_PG -gt 0 ]; then NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q` fi done # Finish with results case $KERN in '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`; echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;; '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;; *) echo "Unrecognized kernel version $KERN. Exiting." ;; esac # End

Make the file executable. $ chmod u+x hugepages_setting.sh

Make sure all the Oracle services are running as normal on the server, then run the script and make a note of the recommended "vm.nr_hugepages" value. $ ./hugepages_settings.sh Recommended setting: vm.nr_hugepages = 191 $

Edit the "/etc/sysctl.conf" file as the "root" user, adding the following entry, adjusted based on your output from the script. You should set the value greater than or equal to the value displayed by the script. You only need 1 or 2 spare pages. vm.nr_hugepages=192

Run the following command as the "root" user. # sysctl -p

You can now see the HugePages have been created, but are currently not being used. $ grep Huge /proc/meminfo HugePages_Total: 192 HugePages_Free: 192 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB

Add the following entries into the "/etc/security/limits.conf" script, where the setting is the size of the HugePages allocation in KB (HugePages * Hugepagesize * 1024). * soft memlock 393216 * hard memlock 393216

Check the MEMORY_TARGET parameters are not set for the database and SGA_TARGET and PGA_AGGREGATE_TARGET parameters are being used instead. SQL> show parameter target NAME --------------------------------------archive_lag_target db_flashback_retention_target fast_start_io_target fast_start_mttr_target memory_max_target memory_target parallel_servers_target pga_aggregate_target sga_target SQL> TYPE VALUE ----------- -------------------------integer integer integer integer big integer big integer integer big integer big integer 0 1440 0 0 0 0 16 125M 376M

Restart the server and restart the database services as required. Check the HugePages information again. $ grep Huge /proc/meminfo HugePages_Total: 192 HugePages_Free: 52 HugePages_Rsvd: 49 HugePages_Surp: 0 Hugepagesize: 2048 kB $