How to Verify Memory Leaks on Unix

Rui Wang

May 11, 2010

Recently, we were experiencing memory usage problem while we created a domain index (Oracle

Text). The circumstance we are facing is that the memory was eaten up shortly after we started the

simple create domain index statement.

As we known, creating that domain index probably would takes over 2 days. It’s also what we’ve

been told by application vendor. After we started the process to create that domain index, we

monitored it by using solaris command “prstat” and noticed that the value of column ‘SIZE’ was

increasing continuously. The investigation we made in Toad also presented that the

“PGA_ALLOC_MEM” kept increasing crazily. And shortly, only this single process took over 8G on

“PGA_ALLOC_MEM”. Upon this, we doubt that that’s a memory leaks.

Before we start to overcome it, we need to verify if that’s memory leaks first. Thanks for oracle

metalink article Note: 477522.1 How To Troubleshoot Memory Leaks on UNIX , we are able to easily

verify it with given script.

The first step is to create a solaris shell script as following, named mem.shl.

while true

do

for pid in `cat pids`

do var=`date|cut -d' ' -f4`

echo "$var :\c" >> ps.log.$pid

ps -eo pid,vsz,rss,pcpu,args | grep $pid |grep -v grep >> ps.log.$pid

done
 sleep 30

done

And then, establish sqlplus connection to database and use SPID of this oracle session as input

content of file pids.

29195

Okay, we are now ready to create domain index within above oracle session. Shortly after the

issuing of create index command, please start mem.shl script to monitor that process.

sh mem.shl

While the create index command is running, you’ll be noticed that there is output file created,

called ps.log.19704. The shell script mem.shl will produce memory calculation every 30 seconds

and write it to file ps.log.19704. The typical output of file ps.log.29195looks like below.

7 :29195 1400848 1384184 4.2 oracleWEBCT (LOCAL=NO)

7 :29195 1581008 1564296 10.5 oracleWEBCT (LOCAL=NO)

7 :29195 1758928 1742264 11.6 oracleWEBCT (LOCAL=NO)

7 :29195 1933328 1916648 12.1 oracleWEBCT (LOCAL=NO)

7 :29195 2100432 2083728 11.9 oracleWEBCT (LOCAL=NO)

7 :29195 2281232 2264552 12.0 oracleWEBCT (LOCAL=NO)

7 :29195 2459920 2443248 12.2 oracleWEBCT (LOCAL=NO)

7 :29195 2641616 2624904 12.3 oracleWEBCT (LOCAL=NO)

7 :29195 2824144 2807448 12.2 oracleWEBCT (LOCAL=NO)

7 :29195 3004496 2987824 12.1 oracleWEBCT (LOCAL=NO)

From output above (the column in color red), it’s very clear that the memory consumption is

increasing linearly. That’s for sure is memory leaks.

For intuitive perspective, it’s better to create PivotTable in Microsoft Office Excel. Basically, the

figure 1 below shows typical memory leaks and figure 2 presents normal memory usage. Usually, the

memory leak process starts up and grows to specific amount of memory usage for a period of time
and, at one point of time, memory growth rises sharply to quite larger size of memory, like plotted

in figure 1.

Figure 1. Process with Memory Leak

Figure 2. Process without memory Leak

For similar solution for other version of Unix, please refer to oracle metalink.
About the Author

Rui Wang currently works as Oracle DBA in Canada. He is responsible for

database performance tuning and high availability. With over 10 years
experience in architecting and building oracle systems, Rui is an evangelist
for oracle technology and products. Rui is OCP and possess master degree in
computer science from Simon Fraser University in Canada. Rui is also an expert of
integration of EDI and database/software system with proven successful projects.

Visit Rui’s blog at http://www.oraclepoint.com/oralife

or join forum http://www.oraclepoint.com for much more resource on oracle.