04/12/2014

FrontingTomcatwithApacheorIISBestpractices

Summary
RunningclusterofTomcatserversbehindtheWebservercanbedemandingtaskifyouwishtoarchivemaximum
performanceandstability.Thisarticledescribesbestpracticeshowtoaccomplishthat.
ByMladenTurk

FrontingTomcat
OnemightaskaquestionWhytoputtheWebserverinfrontofTomcatatall?ThankstothelatestadvancesinJavaVirtual
Machines(JVM)technologyandtheTomcatcoreitself,theTomcatstandaloneisquitecomparablewithperformanceto
thenativewebservers.Evenwhendeliveringstaticcontentitisonly10%slowerthanrecentApache2webservers.
Theansweris:scalability.
Tomcatcanservemanyconcurrentusersbyassigningaseparatethreadofexecutiontoeachconcurrentclientconnection.
Itcandothatnicelybutthereisaproblemwhenthenumberofthoseconcurrentconnectionsrise.ThetimetheOperating
Systemwillspendonmanagingthosethreadswilldegradetheoverallperformance.JVMwillspendmoretimemanaging
andswitchingthosethreadsthendoingarealjob,servingtherequests.
Besidestheconnectivitythereisonemoresignificantproblem,anditcausedbytheapplicationsrunningontheTomcat.A
typicalapplicationwillprocessclientdata,accessthedatabase,dosomecalculationsandpresentthedatabacktothe
client.Allthatcanbeatimeconsumingjobthatinmostcasesmustbefinishedinsidehalfasecond,toachieveuser
perceptionofaworkingapplication.Simplemathwillshowthatfora10msapplicationresponsetimeyouwillbeableto
serveatmost50concurrentusers,beforeyourusersstartcomplaining.Sowhattodoifyouneedtosupportmoreusers?
Thesimplestthingistobuyafasterhardware,addmoreCPUoraddmoreboxes.Atwo2wayboxesareusuallycheaper
thena4wayone,soaddingmoreboxesisgenerallyacheapersolutionthenbuyingamainframe.
FirstthingtoeasetheloadfromtheTomcatistousetheWebserverforservingstaticcontentlikeimages,etc..

Figure1.Genericconfiguration
Figure1.showsthesimplestpossibleconfigurationscenario.HeretheWebserverisusedtodeliverstaticcontextwhile
Tomcatonlydoestherealjobservingapplication.Inmostcasesthisisallthatyouwillneed.With4wayboxand10ms
applicationtimeyou'llbecapableofserving200concurrentusers,thusgiving3.5millionhitsperday,thatisbyallmeans
arespectablenumber.
ForthatkindofloadyougenerallydonotneedtheWebserverinfrontofTomcat.Butherecomesthesecondreasonwhy
toputtheWebserverinfront,andthatiscreatinganDMZ(demilitarizedzone).PuttingWebserveronacomputerhost
insertedasa"neutralzone"betweenacompany'sprivatenetworkandtheinternetorsomeotheroutsidepublicnetwork
givestheapplicationshostedonTomcatcapabilitytoaccesscompanyprivatedata,whilesecuringtheaccesstoother
privateresources.
https://people.apache.org/~mturk/docs/article/ftwai.html

1/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

Figure2.Securegenericconfiguration
BesidehavingDMZandsecureaccesstoaprivatenetworktherecanbemanyotherfactorsliketheneedforthecustom
authenticationforexample.
IfyouneedtohandlemoreloadyouwilleventuallyhavetoaddmoreTomcatapplicationservers.Thereasonforthatcan
beeithercausedbythefactthatyourclientloadjustcannotbehandledbyasingleboxorthatyouneedsomesortof
failoverincaseoneofthenodesbreaks.

Figure3.Loadbalancingconfiguration

https://people.apache.org/~mturk/docs/article/ftwai.html

2/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

ConfigurationcontainingmultipleTomcatapplicationserversneedsaloadbalancerbetweenwebserverandTomcat.For
Apache1.3,Apache2.0andIISWebserversyoucanuseJakartaTomcatConnector(alsoknownasJK),becauseitoffers
bothsoftwareloadbalancingandstickysessions.FortheupcomingApache2.1/2.2usetheadvancedmod_proxy_balancer
thatisanewmoduledesignedandintegratedwithintheApachehttpdcore.

CalculatingLoad
WhendeterminingthenumberofTomcatserversthatyouwillneedtosatisfytheclientload,thefirstandmajortaskis
determiningtheAverageApplicationResponseTime(hereafterAART).Assaidbefore,tosatisfytheuserexperiencethe
applicationhastorespondwithinhalfofsecond.Thecontentreceivedbytheclientbrowserusuallytriggerscoupleof
physicalrequeststotheWebserver(e.g.images).Thewebpageusuallyconsistsofhtmlandimagedata,soclientissuesa
seriesofrequests,andthetimethatallthisgetsprocessedanddeliverediscalledAART.TogetmostoutofTomcatyou
shouldlimitthenumberofconcurrentrequeststo200perCPU.
Sowecancomewiththesimpleformulatocalculatethemaximumnumberofconcurrentconnectionsaphysicalboxcan
handle:
500
Concurrentrequests=(max200)*NumberofCPU's
AART(ms)

TheotherthingthatyoumustcareistheNetworkthroughputbetweentheWebserverandTomcatinstances.This
introducesanewvariablecalledAverageApplicationResponseSize(hereafterAARS),thatisthenumberofbytesofall
contextonawebpagepresentedtotheuser.Onastandard100Mbpsnetworkcardwith8BitsperByte,themaximum
theoreticalthroughputis12.5MBytes.
12500
Concurrentrequests=
AARS(KBytes)

Fora20KBAARSthiswillgiveatheoreticalmaximumof625concurrentrequests.Youcanaddmorecardsorusefaster
1Gbpshardwareifneedtohandlemoreload.

TheformulasabovewillgiveyourudimentaryestimationofthenumberofTomcatboxesandCPU'sthatyouwillneedto
handlethedesirednumberofconcurrentclientrequests.Ifyouhavetodeploytheconfigurationwithouthavingactual
hardware,theclosestyoucangetistomeasuretheAARTonatestplatformandthencomparethehardwarevendor
Specmarks.

FrontingTomcatwithApache
IfyouneedtoputtheApacheinfrontofTomcatusetheApache2withworkerMPM.YoucanuseApache1.3orApache2
withpreforkMPMforhandlingsimpleconfigurationslikeshownontheFigure1.IfyouneedtofrontseveralTomcat
boxesandimplementloadbalancinguseApache2andworkerMPMcompiledin.
MPMorMultiProcessingModuleisApache2corefeatureanditisresponsibleforbindingtonetworkportsonthe
machine,acceptingrequests,anddispatchingchildrentohandletherequests.MPMsmustbechosenduringconfiguration,
andcompiledintotheserver.Compilersarecapableofoptimizingalotoffunctionsifthreadsareused,butonlyifthey
knowthatthreadsarebeingused.BecausesomeMPMsusethreadsonUnixandothersdon't,Apachewillalwaysperform
betteriftheMPMischosenatconfigurationtimeandbuiltintoApache.
WorkerMPMoffersahigherscalabilitycomparedtoastandardpreforkmechanismwhereeachclientconnectioncreatesa
separateApacheprocess.Itcombinesthebestfromtwoworlds,havingasetofchildprocesseseachhavingasetof
separatethreads.Therearesitesthatarerunning10K+concurrentconnectionsusingthistechnology.

https://people.apache.org/~mturk/docs/article/ftwai.html

3/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

ConnectingtoTomcat
InasimplestscenariowhenyouneedtoconnecttosingleTomcatinstanceyoucanusemod_proxythatcomesasapartof
everyApachedistribution.However,usingthemod_jkconnectorwillprovideapproximatelydoubletheperformance.
Thereareseveralreasonsforthatandthemajoristhatmod_jkmanagesapersistentconnectionpooltotheTomcat,thus
avoidingopeningandclosingconnectionstoTomcatforeachrequest.Theotherreasonisthatmod_jkusesacustom
protocolnamedAJPanbythatavoidsassemblinganddisassemblingheaderparametersforeachrequestthatarealready
processedontheWebserver.YoucanfindmoredetailsaboutAJPprotocolontheJakartaTomcatconnectorssite.
Forthosereasonsyoucanusemod_proxyonlyforthelowloadsitesorforthetestingpurposes.FromnowonI'llfocuson
mod_jkforfrontingTomcatwithApache,becauseitoffersbetterperformanceandscalability.
OneofthemajordesignparameterswhenfrontingTomcatwithApacheoranyotherWebserveristosynchronizethe
maximumnumberofconcurrentconnections.DevelopersoftenleavedefaultconfigurationvaluesfrombothApacheand
Tomcat,andarefacedwithspuriouserrormessagesintheirlogfiles.Thereasonforthatisverysimple.Tomcatand
Apachecaneachacceptonlyapredefinednumberofconnections.Ifthosetwoconfigurationparametersdiffers,usually
withTomcathavinglowerconfigurednumberofconnections,youwillbefacedwiththesporadicconnectionerrors.Ifthe
loadgetsevenhigher,youruserswillstartreceivingHTTP500servererrorsevenifyourhardwareiscapableofdealing
withtheload.
DeterminingthenumberofmaximumofconnectionstotheTomcatincaseofApachewebserverdependsontheMPM
used.
MPM
Prefork
Worker
WinNT
Netware

configurationparameter
MaxClients
MaxClients
ThreadsPerChild
MaxThreads

OntheTomcatsidetheconfigurationparameterthatlimitsthenumberofallowedconcurrentrequestsismaxProcessors
withdefaultvalueof20.ThisnumberneedstobeequaltotheMPMconfigurationparameter.

Loadbalancing
Loadbalancingisoneofthewaystoincreasethenumberofconcurrentclientconnectionstotheapplicationserver.There
aretwotypesofloadbalancersthatyoucanuse.Thefirstoneishardwareloadbalancerandthesecondoneissoftware
loadbalancer.Ifyouareusingloadbalancinghardware,insteadofamod_jkorproxy,itmustsupportacompatiblepassive
oractivecookiepersistencemechanism,andSSLpersistence.
Mod_jkhasanintegratedvirtualloadbalancerworkerthatcancontainanynumberofphysicalworkersorparticular
physicalnodes.Eachofthenodescanhaveitsownbalancefactorortheworker'squotaorlbfactor.Lbfactorishowmuch
weexpectthisworkertowork,ortheworkers'sworkquota.Thisparameterisusuallydependentonthehardwaretopology
itself,anditofferstocreateaclusterwithdifferenthardwarenodeconfigurations.Eachlbfactoriscomparedtoallother
lbfactorsintheclusteranditsrelationshipgivestheactualload.Ifthelbfactorsareequaltheworkersloadwillbeequalas
well(e.g.11,22,5050,etc...).Iffirstnodehaslbfactor2whilesecondhaslbfactor1,thanthefirstnodewillreceivetwo
timesmorerequeststhansecondone.Thisasymmetricloadconfigurationenablestohavenodeswithdifferenthardware
architecture.
Inthesimplestloadbalancertopologywithonlytwonodesinthecluster,thenumberofconcurrentconnectionsonaweb
serversidecanbeastwiceashighthenonaparticularnode.But...
1+1!=2

Theupperstatementmeansthatthesumofallowedconnectionsonaparticularnodesdoesnotgivethetotalnumberof
connectionsallowed.Thismeansthateachnodehastoallowaslightlyhighernumberofconnectionsthanthedesiredtotal
sum.Thisnumberisusuallya20%higheranditmeansthat
https://people.apache.org/~mturk/docs/article/ftwai.html

4/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

1*1.2+1*1.2==2

Soifyouwishtohavea100concurrentconnectionswithtwonodes,eachofthenodewillhavetohandlethemaximumof
60connections.The20%marginfactorisexperimental,anddependsontheApacheserverused.ForpreforkMPMsitcan
riseupto50%,whilefortheNTorNetwareitsvalueis0%.Thereasonforthatisthateachparticularchildprocess
menagesitsownbalancestatisticsthusgivingthis20%errorformultiplechildprocesswebservers.
worker.node1.type=ajp13
worker.node1.host=10.0.0.10
worker.node1.lbfactor=1

worker.node2.type=ajp13
worker.node2.host=10.0.0.11
worker.node2.lbfactor=2
worker.node3.type=ajp13
worker.node3.host=10.0.0.12
worker.node3.lbfactor=1

worker.list=lbworker
worker.lbworker.type=lb
worker.lbworker.balance_workers=node1,node2,node3

Theminimumconfigurationforathreenodeclustershownintheupperexamplewillgivethe25%50%25%distribution
oftheload,meaningthatthenode2willgetasmuchloadastherestofthetwomembers.Itwillalsoimposethefollowing
numberofmaxProcessorsforeachparticularnodeincaseoftheMaxClients=200.
node1:
<Connector...maxProcessors="60".../>
node2:
<Connector...maxProcessors="120".../>
node3:
<Connector...maxProcessors="60".../>

Usingsimplemaththeloadshouldbe5010050butweneededtoaddthe20%loaddistributionerror.Incasethis20%
additionalloadisnotsufficient,youwillneedtosetthehighervalueuptothe50%.Ofcoursetheaveragenumberof
connectionsforeachparticularnodewillstillfollowtheloadbalancerdistributionquota.

Stickysessionsandfailower
Oneofthemajorproblemswithhavingmultiplebackendapplicationserversisdeterminingtheclientserverrelationship.
Oncetheclientmakesarequesttoaserverapplicationthatneedstotrackuseractionsoveradesignatedtimeperiod,some
sortofstatehastobeenforcedinsideastatelesshttpprotocol.Tomcatissuesasessionidentifierthatuniquelydistinguishes
eachuser.TheproblemwiththatsessionidentifieristhathedoesnotcarryanyinformationabouttheparticularTomcat
instancethatissuedthatidentifier.
TomcatinthatcaseaddsanextrajvmRouteconfigurablemarktothatsession.ThejvmRoutecanbeanynamethatwill
uniquelyidentifytheparticularTomcatinstanceinthecluster.Ontheothersideofthewirethemod_jkwillusethat
jvmRouteasthenameoftheworkerinit'sloadbalancerlist.ThismeansthatthenameoftheworkerandthejvmRoute
mustbeequal.
jvmRouteisappendedtothesessionidentifier:
http://host/app;jsessionid=0123456789ABCDEF0123456789ABCDEF.jvmRouteName

Whenhavingmultiplenodesinaclusteryoucanimproveyourapplicationavailabilitybyimplementingfailover.The
failovermeansthatiftheparticularelectednodecannotfulfilltherequesttheanothernodewillbeselectedautomatically.
Incaseofthreenodesyouareactuallydoublingyourapplicationavailability.Theapplicationresponsetimewillbeslower
duringfailover,butnoneofyouruserswillberejected.Insidethemod_jkconfigurationthereisaspecialconfiguration
parametercalledworker.retriesthathasdefaultvalueof3,butthatneedstobeadjustedtotheactualnumberofnodesin
thecluster.
...
https://people.apache.org/~mturk/docs/article/ftwai.html

5/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

worker.list=lbworker
worker.lbworker.type=lb
#Adjusttothenumberofworkers
worker.retries=4
worker.lbworker.balance_workers=node1,node2,node3,node4

Ifyouaddmorethenthreeworkerstotheloadbalanceradjusttheretriesparametertoreflectthatnumber.Itwillensure
thatevenintheworsecasescenariotherequestgetsservedifthereisasingleoperablenode.Ofcourse,therequestwillbe
rejectediftherearenofreeconnectionsavailableontheTomcatside,soyoushouldincreasetheallowednumberof
connectionsoneachTomcatinstance.Inthethreenodescenario(121)ifoneofthenodesgoesdown,theothertwowill
havetotakeitsload.SoiftheloadisdividedequallyyouwillneedtosetthefollowingTomcatconfiguration:
node1:
<Connector...maxProcessors="120".../>
node2:
<Connector...maxProcessors="160".../>
node3:
<Connector...maxProcessors="120".../>

Thisconfigurationwillensurethat200concurrentconnectionswillalwaysbeallowablenomatterwhichofthenodesgoes
down.Thereasonfordoublingthenumberofprocessorsonnode1andnode3isbecausetheyneedtohandletheadditional
loadincasenode2goesdown(load11).Node2alsoneedstheadjustmentbecauseifoneoftheothertwonodesgoes
down,theloadwillbe12.Asyoucanseethe20%loaderrorisalwayscalculatedin.

Figure4.Threenodeexampleloadbalancer
https://people.apache.org/~mturk/docs/article/ftwai.html

6/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

Figure5.Failoverfornode2
AsshowninthetwofiguresabovesettingmaxProcessorsdependsbothon20%loadbalancererrorandexpectedsingle
nodefailure.Thecalculationmustincludethenodewiththehighestlbfactorastheworstcasescenario.

DomainClusteringmodel
SinceJKversion1.2.8thereisanewdomainclusteringmodelanditoffershorizontalscalabilityandperformanceof
tomcatcluster.
Tomcatclusterdoesonlyallowsessionreplicationtoallnodesinthecluster.Onceyouworkwithmorethan34nodes
thereistoomuchoverheadandriskinreplicatingsessionstoallnodes.Wesplitallnodesintoclusteredgroups.Thenewly
introducedworkerattributedomainletmod_jkknow,towhichothernodesasessiongetsreplicated(allworkerswiththe
samevalueinthedomainattribute).Soaloadbalancingworkerknows,onwhichnodesthesessionisalive.Ifanodefails
orisbeingtakendownadministratively,mod_jkchoosesanothernodethathasareplicaofthesession.
Forexampleifyouhaveaclusterwithfournodesyoucanmaketwovirtualdomainsandreplicatethesessionsonlyinside
thedomains.Thiswilllowerthereplicationnetworktrafficbyhalf

https://people.apache.org/~mturk/docs/article/ftwai.html

7/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

Figure6.Domainmodelclustering
Fortheaboveexampletheconfigurationwouldlooklike:
worker.node1.type=ajp13
worker.node1.host=10.0.0.10
worker.node1.lbfactor=1
worker.node1.domain=A

worker.node2.type=ajp13
worker.node2.host=10.0.0.11
worker.node2.lbfactor=1
worker.node2.domain=A
worker.node3.type=ajp13
worker.node3.host=10.0.0.12
worker.node3.lbfactor=1
worker.node3.domain=B
worker.node4.type=ajp13
worker.node4.host=10.0.0.13
worker.node4.lbfactor=1
worker.node4.domain=B

worker.list=lbworker
worker.lbworker.type=lb
worker.lbworker.balance_workers=node1,node2,node3,node4

NowassumeyouhavemultipleApachesandTomcats.TheTomcatsareclusteredandmod_jkusesstickysessions.Now
youaregoingtoshutdown(maintenance)onetomcat.AllApachewillstartconnectionstoalltomcats.Youendupwith
alltomcatsgettingconnectionsfromallapacheprocesses,sothenumberofthreadsneededinsidethetomcatswillexplode.
Ifyougroupthetomcatstodomainasexplainedabove,theconnectionsnormallywillstayinsidethedomainandyouwill
needmuchlessthreads.

FrontingTomcatwithIIS
https://people.apache.org/~mturk/docs/article/ftwai.html

8/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

JustlikeApacheWebserverforWindows,MicrosoftIISmaintainsaseparatechildprocessandthreadpoolforserving
concurrentclientconnections.FornonserverproductslikeWindows2000ProfessionalorWindowsXPthenumberof
concurrentconnectionsislimitedto10.Thismeanthatyoucannotuseworkstationproductsforproductionserversunless
the10connectionslimitwillfulfilyourneeds.Theserverrangeofproductsdoesnotimposethat10connectionlimit,but
justlikeApache,the2000connectionsisalimitwhenthethreadcontextswitchingwilltakeitsshareandslowdownthe
effectivenumberofconcurrentconnections.Ifyouneedhigherloadyouwillneedtodeployadditionalwebserversand
useWindowsNetworkLoadBalancer(WNLB)infrontofTomcatservers.

Figure7.WNLBHighloadconfiguration
FortopologiesusingWindowsNetworkLoadBalancerthesamerulesareinplaceasfortheApachewithworkerMPM.
ThismeansthateachTomcatinstancewillhavetohandle20%higherconnectionloadpernodethanitsreallbfactor.The
workers.propertiesconfigurationmustbeidenticaloneachnodethatconstitutesWNLB,meaningthatyouwillhaveto
https://people.apache.org/~mturk/docs/article/ftwai.html

9/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

configureallfourTomcatnodes.

Apache2.2andnewmod_proxy
ForthenewApache2.1/2.2mod_proxyhasbeenrewritenandhasanewAJPcapableprotocolmodule(mod_proxy_ajp)
andintegratedsoftwareloadbalancer(mod_proxy_balancer).
Becauseitcanmaintainaconstantconnectionpooltobackedserversitcanreplacethemod_jkfunctionality.
LoadModuleproxy_modulemodules/mod_proxy.so
LoadModuleproxy_ajp_modulemodules/mod_proxy_ajp.so
LoadModuleproxy_balancer_modulemodules/mod_proxy_balancer.so
...
<Proxybalancer://mycluster>
BalancerMemberajp://10.0.0.10:8009min=10max=100route=node1loadfactor=1
BalancerMemberajp://10.0.0.11:8009min=20max=200route=node2loadfactor=2
</Proxy>
ProxyPass/servletsexamplesbalancer://mycluster/servletsexamples

TheaboveexampleshowshoweasyistoconfigureaTomcatclusterwithproxyloadbalancer.Oneofthemajor
advantagesofusingproxyistheintegratedcaching,andnoneedtocompileexternalmodule.
Mod_proxy_balancerhasintegratedmanagerfordynamicparameterchanges.Itofferschangingsessionroutesordisabling
anodeformaintenance.
<Location/balancermanager>
SetHandlerbalancermanager
Orderdeny,allow
Allowfromlocalhost
</Location>

https://people.apache.org/~mturk/docs/article/ftwai.html

10/11

04/12/2014

FrontingTomcatwithApacheorIISBestpractices

Figure8.ChangingBalancerMemberparameters
Thefuturedevelopmentofmod_proxywillincludetheoptiontodynamicallydiscovertheparticularnodetopology.Itwill
alsoallowtodynamicallyupdateloadfactorsandsessionroutes.

AbouttheAuthor
MladenTurkisaDeveloperandConsultantforJBossIncinEurope,whereheisresponsiblefornativeintegration.Heisa
longtimecommiterforJakartaTomcatConnectors,ApacheHttpdandApachePortableRuntimeprojects.

LinksandResources
JakartaTomcatconnectorsdocumentation
Apache2.0documentation
Apache2.1documentation
https://people.apache.org/~mturk/docs/article/ftwai.html

11/11