Java Virtual Machine Neutral
Java Virtual Machine Neutral
08
JavaPerformanceTuning MichaelFinocchiaro
ThiswhitepaperpresentsthebasicsofJavaPerformanceTuningforlargeApplicationServers.
http://mfinocchiaro.wordpress.com
TableofContents
JavaPerformanceTuning.................................................................................................................. 3 GarbageCollection.......................................................................................................................................... 3 MemoryManagement,JavaandImpactonApplicationServers .................................................... 3 JavaVirtualMachines .................................................................................................................................... 3 TheJavaHeap................................................................................................................................................... 3 HotSpotVM........................................................................................................................................................ 3 JavaHeapDescription ........................................................................................................................ 3 YoungGeneration ........................................................................................................................................... 3 OldGeneration ................................................................................................................................................. 3 PermanentGeneration.................................................................................................................................. 4 DefaultGarbageCollectionAlgorithms ........................................................................................ 4 ScavengeGarbageCollection....................................................................................................................... 4 FullGarbageCollection ................................................................................................................................. 4 AlternativeGarbageCollectors ....................................................................................................... 4 ParallelCopyGC............................................................................................................................................... 4 ConcurrentMarkandSweep...................................................................................................................... 4 SummaryofJava1.4andJava5CommandLineOptions ......................................................... 5 SizingtheJVMs ...................................................................................................................................... 5 MonitoringGarbageCollectionDuration&Frequency........................................................... 5 JavaCommandLine ............................................................................................................................. 6 ManagingSystemMemory ................................................................................................................ 6 BalancingSystemMemory........................................................................................................................... 6
JavaPerformanceTuning
GarbageCollection
TheJavaprogramminglanguageisobjectorientedandincludesautomaticgarbagecollection.Garbage collectionistheprocessofreclaimingmemorytakenupbyunreferencedobjects. AcomprehensivediscussiononGarbageCollectioncanbefoundat: http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html Thefollowingsectionswilltrytoresumesomeoftheconceptsfromthisdocumentandtheirimpacton ApplicationServerperformance.
MemoryManagement,JavaandImpactonApplicationServers
Thetaskofmemorymanagementthatwasalwayschallengingwithcompiledobjectorientedlanguages suchasC++.Ontheotherhand,Javaisaninterpretivelanguagethattakesthismemorymanagement outofthehandsofdevelopersandgivesitdirectlytothevirtualmachinewherethecodewillberun. Thismeansthatforbestperformanceandstability,itiscriticalthattheJavaparametersforthevirtual machinebeunderstoodandmanagedbytheApplicationServerdeploymentteam.Thissectionwill describethevariouspartsoftheJavaheapandthenlistsomeusefulparametersandtuningtipsfor ensuringcorrectruntimestabilityandperformanceofApplicationServers.
JavaVirtualMachines
TheJavaspecificationastowhatisstandardforagivenreleaseiswrittenandmaintainedbythe JavaSoftdivisionofSunMicrosystems.ThisspecificationisthendeliveredtootherJVMproviders(IBM, HP,etc).JavaSoftprovidesthestandardimplementationonWindows,Solaris,andLINUX.Other platformsarerequiredtodeliverthecodefunctionalitybuttheirJVMoptionscanbedifferent.Java optionsthatareprecededwithXorXXaretypicallyplatformspecific.Thatbeingsaid,many optionsareusedonallplatforms.OnemustreadindetailtheREADMEnotesfromthevariousreleases onvariousplatformstobekeptuptodatewiththevariations.Thisguidewillmentionthemostcritical onesanddistinguishbetweenthosewhichareimplementedonmostplatformsandthosewhichare platformspecific.
TheJavaHeap
TheJavaheapisdividedintothreemainsections:YoungGeneration,OldGenerationandthePermanent Generationasshowninthefigurebelow.
Also,notshownhere,isanotherareacalledtheCodeCachewhichistypicallyabout50MbinSize.
HotSpotVM
TheHotSpotVMisthegarbagecollectorthatcomeswiththeJavaSoftvirtualmachine.Itsspecification isdeliveredtoallJVMproviders(HP,IBM,BEA,etc)andthestandardimplementationisdeployedby JavaSoftonSolaris,WindowsandLINUX.
JavaHeapDescription
YoungGeneration
TheEdenSpaceoftheYoungGenerationholdsallthenewlycreatedobjects.Whenthisgenerationfills, theScavengeGarbageCollectorclearsoutofmemoryallobjectsthatareunreferenced.Objectsthat survivethisscavengemovedtotheFromSurvivorSpace.TheSurvivorSpaceisasectionoftheYoung Generationfortheseintermediatelifeobjects.IthastwoequallysizedsubspacesToandFrom whichareusedbyitsalgorithmforfastswitchingandcleanup.OncetheScavangeGCiscomplete,the pointersonthetwospacesarereversed:TobecomesFromandFrombecomesTo. TheYoungGenerationissizedwiththeXmnoptionontheJavacommandline.Itshouldneverexceed halfoftheentireheapandistypicallysetto1/2oftheheapforJVMslessthan1.5Gbandto1/3ofthe heapforJVMslargerthan1.5Gb.NotethattheargumenttotheXm{n,s,x}optionscanbesuffixedwith morMformegabytesandgorGforgigabytes.Forexample,a256MbYoungGenerationis specifiedbyXmn256m.Asafurtherexample,a1GbYoungGenerationisspecifiedwithXmn1gor Xmn1000m. TheSurvivorSpaceintheYoungGenerationissizedasaratioofoneofthesubspacestotheEden SpacethisiscalledtheSurvivorRatio.Forexample,ifXmnissetto400mandXX:SurvivorRatioisset
to4,thenthetotalSurvivorSpacewillbe133.2MbwithToandFromeachbeing66.6Mbandthe EdenSpacebeing266.8Mb.Thesurvivorratio=266.8(Eden)/66.6(To)=4.
OldGeneration
OnceanobjectsurvivesagivennumberofScavengeGCs,itispromoted(ortenured)fromtheTo SpacetotheOldGeneration.Objectsinthisspacearenevergarbagecollectedexceptinthetwocases: FullGarbageCollectionorConcurrentMarkandSweepGarbageCollection.IftheOldGenerationisfull andthereisnowayfortheheaptoexpand,anOutofMemoryerror(OOME)isthrownandtheJVMwill crash. TheOldGenerationissizedwiththeXmsandXmxparameters,whereXmsistheinitialheapsize allocatedatstartupandXmxisthemaximumheapsizereservedbytheJVMatstartup.Iftheheapsize exceedsfreememoryonthesystem,swappingwilloccurandperformancewillbeseriouslydegraded. ThemaximumvalueforXmxisarchitecturedependent. Architecture Xmxlimit(fortotalJVMsizeadd~200Mb) ~1300m(no/3Gbflag)1 32bitWindows(XP,Server2003) AIX5.3LonPOWER HPUX11iV1onPARISC HPUX11iV2onIA64 SolarisonUltraSPARCorT1orT2 Notes:
1.The/3Gbflagcanbeaddedtotheboot.iniflagonaWindowsXPSP2orWindows2003Servermachineinordertoexpand thelowermemoryrangetoallow32bitexecutablestoaddressslightlymorememorythanwithoutthisflag.Notethatthetwo valuesgivenareapproximateanddependonthedevicedriversloadedbytheoperatingsystem.Formoreinformationfrom Microsoftonthe/3Gbswitch,pleaseseeMicrosoftKnowledgeBasearticles291988and833721.Formoreinformationon memoryaddressabilityontheWindowsplatform,pleaseseethewhitepaperonthissite: http://download.boulder.ibm.com/ibmdl/pub/software/dw/jdk/diagnosis/dw3gbswitch3.pdf NotealsothatwithJava6,the/3GbflagonWindowswillNOTbesupportedbytheJVM. 2.ThePARISCarchitectureisbeingphasedoutandisnotcurrentlyrecommendedfornewdeployments.OnPARISC,one must,asroot,executethefollowingcommandonthejavaexecuteableinthe/opt/java1.5/bin/PARISC2.0/directorywithboth ApplicationServerandTomcatshutdown: chatr+q3penablejava Thiswillenableapproximately2.0to2.2GbofspacefortheXms/Xmxparametersdependingonloadeddevicedrivers,etc.
PermanentGeneration
ThePermanentGenerationiswhereclassfilesarekept.Thesearetheresultofcompiledclassesandjsp pages.Ifthisspaceisfull,ittriggersaFullGarbageCollection.IftheFullGarbageCollectioncannotclean outoldunreferencedclassesandthereisnoroomlefttoexpandthePermanentSpace,anOutof Memoryerror(OOME)isthrownandtheJVMwillcrash. ThePermanentGenerationissizedwiththeXX:PermSizeandXX:MaxPermSizeparameters.For example,tospecifyastartupPermanentGenerationof48MbandamaximumPermanentGenerationof 128Mb,usetheparameters:XX:PermSize=48mXX:MaxPermSize=128.Itisexceedinglyrarethatmore than128MbofmemoryisrequiredforthePermanentGeneration. NotealsothatthePermanentGenerationistackedontotheendoftheOldGeneration.Thereisalsoa smallcodecacheof50MbforinternalJVMmemorymanagement.Thismeansthatthetotalinitialheap size=Xms+XX:PermSize+~50Mbandthatthemaximumtotalheapsize=Xmx+XX:+MaxPermSize +~50Mb.Forexample,ifXms/Xmxaresetto512mandXX:PermSize/MaxPermSizearesetto128m, thetotalVMwillactuallyabout700Mbinsize.
DefaultGarbageCollectionAlgorithms
ScavengeGarbageCollection
ScavengeGarbageCollection(alsoknownasaMinorCollection)occurswhentheEdenSpaceisfull.By default,itissinglethreadedbutdoesnotinterrupttheotherthreadsworkingonobjects.Itcanbe parallelizedstartingatJava1.4butiftoomoreParallelGCThreadsarespecifiedthanCPUcoreswhere theJVMisrunning,thiscancausebottlenecks.Forthisreason,itissuggestedtobecarefulinwhenand wheretousetheparalleloptions.Thesewillbediscussedfurtheron.
FullGarbageCollection
AFullGarbageCollection(FullGC)occursundertheseconditions: TheJavaapplicationexplicitlycallsSystem.gc().Thiscanbeavoidedbyimplementingthe XX:+DisableExplicitGCparameterinthestartupcommandforallApplicationServerJVMs(Tomcat, ApplicationServerJVM,etc.)
TheRMIprotocolexplicitlycallsSystem.gc()onaregularbasisundernormaloperation.Thiscanbe avoidedbyimplementingtheXX:+DisableExplicitGCparameterinthestartupcommandforall ApplicationServerJVMs. Amemoryspace,eitherOldorPermanent,isfullandtoaccommodatenewobjectsorclasses,itneeds tobeexpandedtowardsitsmaxsize,iftherelevantparametershavedifferentvalues.Inotherwords,if XmsandXmxhavedifferentvaluesandifthesizeofOldneedsbeincreasedfromXmstowardsXmx toaccommodatemoreobjects,aFullGCiscalled.Similarly,ifXX:PermSizeandXX:MaxPermSizehave differentvaluesandthePermanentSpaceneedstobeincreasedtowardsXX:MaxPermSizeto accommodatenewjavaclasses,aFullGCiscalled.ThiscanbeavoidedbyalwayssettingXmsandXmx aswellasXX:PermSizeandXX:MaxPermSizetothesamevalue. TheTenuredSpaceisfullandtheOldGenerationisalreadyatthecapacitydefinedbyXmx.Thiscanbe avoidedbytuningtheYoungGenerationsothatmoreobjectsarefilteredoutbeforebeingpromotedto theOldGeneration,byincreasingtheXmxvalueand/orbyimplementingtheConcurrentMarkand Sweep(CMS)collector.CMSwillbediscussedbelow. ThePermanentSpaceisfullandthePermanentGenerationisalreadyatthecapacitydefinedby XX:MaxPermSize.ThiscanbeavoidedensuringthatthePermanentSpaceislargeenoughandthatthe Xnoclassgcoptionisenabled. TheHotSpotVMthinksthereisnotfreespaceintheheaptocollectalltheobjectsinEdenincaseofan emergencycollection.ThiscanbeavoidedbyusingtheXX:MaxLiveObjectEvacuationRatio=30optionon JVMspriorto1.4.2_13,andbytheXX:+HandlePromotionFailureoptiononJVMsfrom1.4.2_14and higheraswellasJava5JVMs. AConcurrentMarkandSweep(CMS)GCoperationdoesnothaveenoughspaceinOldtocomplete. CMSwillbediscussedbelow. AFullGarbagecollectionisdisruptiveinthesensethatallworkingthreadsarestoppedandoneJVM threadthenwillscantheentireheaptwicetryingtocleanoutunreferencedobjects.Atthesametime, objectswithfinalizerclausesareprocessed.Oncethesecondscaniscomplete,ifsomeobjectsonthe finalizerstackhavenotyetbeenprocessed,theyareleftonthequeueforthenextFullGC.Thisisan expensiveprocesswhichcausesdelaysinresponsetime.ThegoaloftuningtheJVMistominimizethe FullGCswhileensuringthatanOOMEdoesnotoccur.
AlternativeGarbageCollectors
ParallelCopyGC
TheXX:+UseParNewGCturnsonamultithreadedcollectorfortheEdenspace.Idlethreadsareusedfor routinegarbagecollectiontasks.Thenumberofthreadsavailableisdefinedby XX:ParallelGCThreads=#.ThiscandrasticallyreducetheamountoftimespentinScavengeGCbutneeds tobeusedwithprecaution.ThesumofParallelGCThreadsintheJVMsbeingrunonthemachineshould
notexceedtheavailablenumberofCPUcoresonthemachine.Thiscanresultisthreadcontention insidetheJVMandreducedperformance.
ConcurrentMarkandSweep
TheConcurrentMarkandSweepCollectorisaparallelcollectorfortheOldGeneration.Itbreaksthe garbagecollectionintoaninitialmarkingpause,aconcurrentmarkingphase,asecondremarkingclause, andaconcurrentsweepingphase.ThishasthepositiveimpactofavoidingFullGCandOOME.Ithasthe negativesideeffectofbeingveryexpensiveasoneormoreCPUcoreswillbededicatedtoCMS throughouttheentireconcurrentmarkingphaseandoneCPUwillbededicatedtoCMSthroughoutthe entireconcurrentsweepingphase.
Thefigureabovedemonstratesthedifferencebetweenthesinglethreaded(herecalledSerial)Mark SweepCompactCollectorandtheConcurrentMarkSweepcollectorwherethepausesshorterand manyofthephasesaredonewithmultiplethreads. TheCMScollectorisenabledusingtheXX:+UseConcMarkSweepGCflag.Itisalsosuggestedtousethe XX:+CMSParallelRemarkEnabledflagsothattheRemarkingphasehappenswithmultiplethreadsas illustratedabove. CMSshouldbeusedonlyonrelativelylargesystems(morethan4CPUcores)whereOOMEis experiencedregularlyduetotheCPUcostdiscussedearlier.Heapsneedtoattheirmaximumsizewith theXmnat1/4ofthetotalheapforthealgorithmtofunctionefficiently. NotethatturningonCMSalsoimplicitlyturnsonXX:+UseParNewGC.Forthisreason,wheneverCMSis activated,theParallelGCThreadsoptionmustalwaysbeexplicitlysettolimitthenumberofthreadsas discussedaboveinthesectiononParallelCopyGC.
SummaryofJava1.4andJava5CommandLineOptions
Option Xmn<x><m|g> Description YoungGenerationSize,suffixed withm(Mb)org(Gb).Itis recommendedthat Recommended <1.5Gbheap:1/2ofXmx
Xmx<x><m|g>
XX:SurvivorRatio=<x>
RatioofoneSurvivorSpaceto theEdenspace. InitialPermanentGeneration size MaximumPermanent Generationsize IgnoreallcallstoSystem.gc() Loggcactivityto<filename> TurnonGClogging TurnondetailedGClogging TurnontimestampsinGC logging Monitordetailswhenastack traceiscalledforthreadand monitorlockanalysis Useathreadlocaladdress blocktoreducecontentionin edenfornewallocations InitialTLABSize AllowdynamicresizingofTLAB UseParallelCopyCollector
XX:PermSize=<x><morg>
128m
XX:MaxPermSize=<x><morg>
128m
XX:+JavaMonitorsInStackTrace
DeployedinallJVMs
XX:+UseTLAB
DeployedinallJVMs
XX:ParallelGCThreads=<x>
Numberofthreadsdedicatedto Startat2butensurethatsum
ParallelCopyGC
ofParallelGCThreadsacrossall JVMsislessthannumberof availableCPUcores.Always explicitlyspecifywhenusing CMS. Useonsystemswith>4CPU coresONLYifothermethodsof eliminatingOOMEhavebeen exhausted AlwaysspecifywhenusingCMS
XX:+UseConcMarkSweepGC
EnableCMSalgorithm
XX:+CMSParallelRemarkEnabled
EnableParallelRemarkingin CMS
XX:MaxLiveObjectEvacuationRatio PercentageofOldusedbefore OnlyusewithJVMsupto theHotSpotcollectorthrowsan 1.4.2_12. OldFullexceptionandcausesa FullGC XX:+HandlePromotionFailure Fixforissuementionedabove. UsewithallJVMsover1.4.2_13 andwithJava5. 1UseifOOMEisexperienced.
SizingtheJVMs
ThedefaultvaluesofthebasicJavaparameters: Xms=3670k Xmx=64m XX:SurvivorRatio=32(onSolaris,8onHPUX,32onWindows) XX:NewSize=2228k XX:MaxNewSize=unlimited XX:NewRatio=2(onSolaris,defaultis8onWindows) XX:PermSize=16m XX:MaxPermSize=64m TheApplicationServerandservletengineexperiencetwoproblemswiththesedefaultVMdefaultheap parameters:
MonitoringGarbageCollectionDuration&Frequency
Eachvendorprovidesitsowntoolsformonitoringgarbagecollection.Thissectiondescribessometools availableforWindowsandSolarisplatforms.Foradditionalinformationongarbagecollection,seethe followingsites: Sun: http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html HPUX: http://www.hp.com/products1/unix/java/infolibrary/prog_guide/hotspot.html AIX http://www128.ibm.com/developerworks/java/jdk/diagnosis/
HPjtune
HPUXprovidesagarbagecollectionloganalysistoolcalledHPjtunethatplotsgarbagecollectionactivity graphically.ItisprimarilyusedfortheHPUXJVMbut,whenonlyverbose:gcorXloggcarepresentin theSolarisorWindowsJVMs,itcanbeusedtoreadthoseaswell.
jps
FromtheJava5commandline,runthejpscommandtoobtainalistofrunningJVMprocessesthatmay bemonitored.Theprocessidentifier(pid)andmainclassaredisplayedforeachsuchprocess.
jstat
JstatispartofthestandardJava5distributionandiscapableorprintingoutstatisticsinrealtimeona runningJVM.Forexample,specifyingthegcoption: S0CS1CS0US1UECEUOCOUPCPUYGCYGCTFGCFGCTGCT 136512.0136512.00.028820.6546176.0508081.81277952.0121102.898304.045056.9235 60.18100.00060.181
visualgc
Thevisualgctoolisavailableatjava.sun.com(http://java.sun.com/performance/jvmstat/) Thesameinformationthatyoucangetfromjstatisavailablegraphicallyusingthevisualgctool.For example,enterthefollowingforavisualdisplay: D:\jvmstat\bat>visualgc9040 Thefollowingpictureshowsasampleapplicationdisplaytakenatadifferentpointintimefromthe previousexample:TheleftwindowshowsthecurrentsizesofthePerm,Old,Edenandsurvivorspaces (S0andS1)intheheap.TherightwindowalsoshowstimingsforsignificantJVMeventssuchasgarbage collections.Therightwindowshowssizesofthesesamespacesintheheapasstripcharts.Throughthe stripcharts,somehistoryisavailableforspottingtrends.Unfortunately,thevisualgctooldoesnot supportloggingofgarbagecollectionactivitytofile.
JavaCommandLine
ThejavacommandacceptsthedirectiveXloggcthatspecifiesafilenamewherethegarbagecollection diagnosticdataiswritten.Detailsaboutthegarbagecollectionactivity,suchassizeoftheyoungandold generationbeforeandaftergarbagecollection,sizeoftotalheap,elapsedtimeittakesforagarbage collectiontohappeninyoungandoldgeneration,andsizeofobjectspromotedateverygarbage collection,andotherdata,canberecordedbyusingtheboththeXX:+PrintGCDetailsand XX:+PrintGCTimeStampsargument.Evenmoredetail(butnotrecommendedforproduction environments)canbeobtainedwiththeXX:+PrintTenuringDistribution.
ManagingSystemMemory
AnintegralpartofoptimizingApplicationServerperformanceispropermemorymanagement.The frequencyanddurationofJavagarbagecollection(heapmanagement)isgreatlyinfluencedby transactionloadandavailablememory. BoththeApplicationServerandthedatabaseservercanefficientlycachepersistentdatainmemoryto minimizediskreadsandthusincreasetransactionthroughput.Inaddition,theWebserver,servlet engineandoperatingsystemrequiresignificantamountsofmemory.Thegoalistobalanceallmemory requirementswithphysicalmemoryandavoidexcessivevirtualmemorypaging.
BalancingSystemMemory
Usethissectionasastartingpointwhendeterminingmemoryallocationonyourinitialinstallationof theApplicationServer.Theinformationprovidedassumesthattheserversinvolvedarededicatedto
runningyourApplicationServersolution.Theinformationshouldnotbetakenasfixedrulesthatmust befollowed;therecanbemanysitespecificreasonstouseadifferentallocationscheme. Note:Afteryouhaveyourproductionsystemrunning,usetheperformanceanalysistechniques describedinthisguidetoarriveatthememoryallocationschemethatbestmeetsyourproduction system.UsetheinformationinthefollowingMemoryAllocationGuidelinestableasasetofguidelines formemoryallocation.ThefirstcolumnliststhetypesofprocessesneededinaApplicationServer installationandcolumnstwothroughfourindicatethepercentageoftotalmemorytobeallocatedfor thetypeofprocess. Thesecondcolumnofthetablegivespercentagesforasingleserversystem(ApplicationServer,Oracle, Webserver,andservletengineallononesystem).Thethirdandfourthcolumnsgivepercentagesfora splitApplicationServerserver/Oracleserverconfiguration. Application/Database ProcessType ApplicationServer Oracle ServletEngine WebServer SearchEngine TotalUsedMemory SingleServer 21.00% 21.00% 21.00% 5.00% 5.00% 73.0% SplitServers/ Application Server 30.00% 30.00% 5.00% 5.00% 70.0% SplitServers/ OracleServer 70.00% 70.0%