Answer/Solution

We finally discovered the cause of the problem. What was happening is that in pe
riods of inactivity the database connections from the web servers had no activit
y and were being severed after 1hr by the Cisco Firewall and/or Cisco ACE 4710 l
oad balancer (both have 1hr inactivity timeout thresholds by default).
To get around this problem, we added SQLNET.EXPIRE_TIME= 10? to the sqlnet.ora an
d reloaded the listeners. This is a dead connection detection parameter that wil
l tell the database server to check that the connections are still alive. How th
is helps is that it checks the database connections with the web servers every 1
0 mins and thus the firewall and/or load balancer will thus see activity and thu
s NOT severe the connections due to inactivity.
This was actually implemented on our old Oracle database Server (Solaris 8/Oracl
e 9i) but NOT implemented on the new Linux/Oracle 11G Server. And unfortunately
was overlooked until now.