 |
Tuning Garbage Collection
with the 1.3.1 Java Virtual
Machine |
See also Performance
Docs |
Introduction
The Java 2 Platform is increasingly used for large server
applications such as web services. These applications demand scalability,
and directly benefit from large numbers of threads, processors, sockets and
memory. Yet 'big iron' performance has a reputation as an art form,
requiring special expertise beyond what is needed for performance on smaller
systems. Fortunately, the Java Virtual Machine (JVM)* and Solaris
operating environment provide effective implementations of threads, I/O and
memory management. This document addresses a common speed bump on the road
to scalable high performance: poorly tuned garbage collection
(GC).
Amdahl observed that most workloads cannot be perfectly
parallelized; some portion is always sequential and does not benefit from
parallelism. This is also true for the Java 2 Platform. In
particular, JVMs up to and including version 1.3.1 do not have parallel garbage
collection, so the impact of GC on a multiprocessor system grows relative
to an otherwise parallel application.
The graph below models an ideal system that is perfectly scalable with the
exception of GC. The top line (red) is an application spending only 1% of
the time in GC on a uniprocessor; this translates into more than 20% loss in
throughput at 32 processors. At 10%, not considered an outrageous amount
of time in GC in uniprocessor applications, more than 75% of throughput is lost
when scaling up.
This demonstrates that issues that appear lost in the noise when developing
on small systems may become principal bottlenecks when scaling up. The
silver lining is that small improvements in such a bottleneck can produce large
gains in performance. For a sufficiently large system it becomes well
worthwhile to tune garbage collection.
This document is written from the
perspective of 1.3.1 JVM on the Solaris (SPARC Platform Edition) operating
environment, because that platform provides the most scalable hardware/software
Java 2 platform today. However, the descriptive text applies to other
supported platforms, including Linux, Microsoft Windows, and the Solaris (Intel
Architecture) operating environment, to the extent that scalable hardware is
available. Although command line options are consistent across platforms,
some platforms may have different defaults than described here.
Generations
One of Java 2 Platform's great strengths is that it shields
the substantial complexity of memory allocation and garbage collection from the
developer. However, once GC has become the principal bottleneck, it
becomes worth understanding aspects of this hidden implementation. Garbage
collectors make assumptions about the way applications use objects, and these
are reflected in tunable parameters that can be adjusted for improved
performance without sacrificing the power of the abstraction.
An object
is garbage when it can no longer be reached from any pointer in the running
program. The most straightforward garbage collection algorithms simply
iterate over every reachable object; any objects left over are then known to be
garbage. This approach takes time proportional to the number of living
objects, which is prohibitive for large applications maintaining lots of living
data.
The 1.3 JVM incorporates a number of different garbage collection
algorithms that are combined using generational collection. While
naive garbage collection examines every living object in the heap, generational
collection exploits several empirically observed properties of most applications
to avoid extra work.
The most important of these properties is infant
mortality. The blue area in the diagram below is a typical
distribution for the lifetimes of objects. The sharp peak at the left
represents objects that can be reclaimed shortly after being allocated.
Iterator objects, for example, are often alive for the duration of a single
loop.
Some objects do live longer, and so the distribution stretches out to the the
right. For instance, there are typically some objects allocated at
initialization that live until the process exits. Between these two
extremes are objects that live for the duration of some intermediate
computation, seen here as the lump to the right of the infant mortality
peak. Some applications have very different looking distributions, but a
surprisingly large number possess this general shape. Efficient collection
is made possible by focusing on the fact that a majority of objects die
young.
To do this, memory is managed in generations: memory pools
holding objects of different ages. Garbage collection occurs in each
generation when it fills up; these collections are represented on the diagram
above with vertical bars. Objects are allocated in eden, and
because of infant mortality most objects die there. When Eden fills up it
causes a minor collection, in which some surviving objects are moved to
an older generation. When older generations need to be collected there is
a major collection that is often much slower because it involves all
living objects.
The diagram shows a well-tuned system in which
most objects die before they survive to the first garbage collection. The
longer an object survives, the more collections it will endure and the slower GC
becomes. By arranging for most objects to survive less than one
collection, garbage collection can be very efficient. This happy situation
can be upset by applications with unusual lifetime distributions, or by poorly
sized generations that cause collections to be too frequent.
The
default garbage collection parameters were designed to be effective for most
small applications. They aren't optimal for many server
applications. This leads to the central tenet of this document:
|
If GC has become a bottleneck, you may wish to customize the generation
sizes. Check the verbose GC output, and then explore the sensitivity
of your individual performance metric to the GC
parameters. |
Types of collection
Each generation has an associated type of
garbage collection that can be configured to make different algorithmic
time, space and pause tradeoffs. In 1.3, the JVM implements three very
different garbage collectors:
- Copying (sometimes called scavenge): this collector very
efficiently moves objects between two or more generations. The source
generations are left empty, allowing remaining dead objects to be reclaimed
quickly. However, since it requires empty space to operate, copying
requires more footprint. In 1.3.1 copying collection is used for all
minor collections.
- Mark-compact: this collector allows generations to be collected in
place without reserving extra memory; however, compaction is significantly
slower than copying. In 1.3.1 mark-compact is used for major
collections.
- Incremental (sometimes called train): this collector is used
only if -Xincgc is passed on
the command line. By careful bookkeeping, incremental GC collects just a
portion of the old generation at a time, trying to spread the large pause of a
major collection over many minor collections. However, it is even slower
than mark-compact when considering overall throughput.
Since copying
is very fast, a tuning goal is to collect as many objects as possible by copying
rather than by compaction or incremental collection.
The default
arrangement of generations looks something like this.
At initialization, a maximum address space is virtually reserved but not
allocated physical memory unless it is needed. The complete address space
reserved for object memory can be divided into the young and old
generations.
The young generation consists of eden plus two
survivor spaces . Objects are initially allocated in eden.
One survivor space is empty at any time, and serves as the destination of the
next copying collection of any living objects in eden and the other survivor
space. Objects are copied between survivor spaces in this way until they
age enough to be tenured (copied to the old generation.)
(Other
virtual machines, including the production JVM version 1.2 for the Solaris
operating environment, used two equally sized spaces for copying rather than one
large eden plus two small spaces. This means the options for sizing the
young generation are not directly comparable; see the Performance FAQ
for an example.)
The old generation is collected in place by
mark-compact. One portion called the permanent generation is
special because it holds all the reflective data of the JVM itself, such as
class and method objects.
Performance considerations
There are two primary measures of garbage
collection performance. Throughput is the percentage of total time
not spent in garbage collection, considered over long periods of time.
Throughput includes time spent in allocation (but tuning for speed of allocation
is generally not needed.) Pauses are the times when an application
appears unresponsive because garbage collection is going on.
Users have
different requirements of garbage collection. For example, some consider
the right metric for a web server to be throughput, since pauses during garbage
collection may be tolerable, or simply obscured by network latencies. But
for an interactive graphical program, even short pauses may upset the user
experience.
Some users are sensitive to other considerations.
Footprint is the working set of a process, measured in pages and cache
lines. On systems with limited physical memory or many processes,
footprint may dictate scalability. Promptness is the time between
when an object becomes dead and when the memory becomes available, an important
consideration for distributed systems including RMI.
In general, a
particular generation sizing chooses a trade-off between these
considerations. For example, a very large young generation may maximize
throughput, but does so at the expense of footprint and promptness. Pauses
can be minimized by using a small young generation and incremental collection,
at the expense of throughput.
There is no one right way to size
generations; the best choice is determined by the way the application uses
memory as well as user requirements. For this reason the JVM's default GC
choices may not be optimal, and may be overridden by the user in the form of
command line options below.
Measurement
Throughput and footprint are best measured using metrics particular to
the application. For example, throughput of a web server may be
tested using a client load generator, while footprint of the server might be
measured on the Solaris operating environment using the pmap command. On the other hand,
pauses due to GC are easily estimated by inspecting the diagnostic output
of the JVM itself.
The command line argument -verbose:gc prints information at every
collection. For example, here is output from a large server
application:
[GC
325407K->83000K(776768K), 0.2300771 secs]
[GC
325816K->83372K(776768K), 0.2454258 secs]
[Full GC 267628K->83769K(776768K),
1.8479984 secs]
Here we see two minor collections and one major
one. The numbers before and after the arrow indicate the combined size of
live objects before and after the GC. (After minor collections the count
includes objects that aren't necessarily alive but can't be reclaimed, either
because they are directly alive, or because they are within or referenced from
the old generation.) The number in parenthesis is the total available
space, which is the total heap minus one of the survivor spaces.
Sizing the generations
A number of parameters affect generation size. This
diagram illustrates the ones most important to tuning the 1.3.1 JVM.
Many parameters are actually ratios x:y, and these are
depicted with black (representing x) and grey (representing
y) size bars: 
Total heap
Since collections occur when generations fill up, throughput
is inversely proprotional to the amount of memory available. Total
available memory is the most important knob affecting GC performance.
By default, the JVM grows or shrinks the heap at each collection to try
to keep the proportion of free space to living objects at each collection within
a specific range. This target range is set as a percentage by the
parameters -XX:MinHeapFreeRatio=<minimum>
and -XX:MaxHeapFreeRatio=<maximum>,
and the total size is bounded below by -Xms and above by -Xmx . The default parameters for
the Solaris (SPARC Platform Edition) operating environment are shown in this
table:
-XX:MinFreeHeapRatio=
|
40
|
-XX:MaxHeapFreeRatio=
|
70
|
-Xms
|
3584k
|
-Xmx
|
64m
|
Large server
apps often experience two problems with these defaults. One is slow
startup, because the initial heap is small and must be resized over many major
collections. A more pressing problem is that the default maximum heap size
is unreasonably small for most server applications. The rules of thumb for
server applications are:
Unless you have
problems with pauses, try granting as much memory as possible to the
JVM. The default size (64MB) is often too small.
Setting
-Xms and -Xmx to the same value increases
predictability by removing the most important sizing decision from the
JVM. On the other hand, the JVM can't compensate if you make a poor
choice.
Be sure to increase the memory as you increase the number
of processors, since allocation can be parallelized, but GC is not
parallel.
|
The young generation
The second most influential knob is the proportion
of the heap dedicated to the young generation. The bigger the young
generation, the less often minor collections occur. However, for a bounded
heap size a larger young generation implies a smaller old generation, which will
increase the frequency of major collections. The optimal choice depends on
the lifetime distribution of the application.
By default, the young
generation size is controlled by NewRatio. For example, setting -XX:NewRatio=3 means that the ratio
between the young and old generation is 1:3; in other words, the combined size
of eden and the survivor spaces will be one fourth of the heap.
The
parameters NewSize and MaxNewSize bound the young generation
size below and above. Setting these equal to one another fixes the young
generation, just as setting -Xms
and -Xmx equal fixes the total
heap size. This is useful for tuning the young generation at a finer
granularity than the integral multiples allowed by NewRatio.
Because the
young generation uses copying collection, enough free memory must be reserved in
the old generation to ensure that a minor collection can complete. In the
worst case, this reserved memory is equal to the size of eden plus the objects
in non-empty survivor space. When there isn't enough memory available in
the old generation for this worst case, a major collection will occur
instead. This policy is fine for small applications, because the memory
reserved in the old generation is typically only virtually committed but not
actually used. But for applications needing the largest possible heap, an
eden bigger than half the virtually committed size of the heap is useless: only
major collections would occur.
If desired, the parameter SurvivorRatio can be used to tune the
size of the survivor spaces, but this is often not as important to
performance. For example, -XX:SurvivorRatio=6 sets the ratio
between each survivor space and eden to be 1:6; in other words, each survivor
space will be one eighth of the young generation (not one seventh,
because there are two survivor spaces).
If survivor spaces are too
small, copying collection overflows directly into the old generation. If
survivor spaces are too large, they will be uselessly empty. At each
garbage collection the JVM chooses a threshold number of times an object can be
copied before it is tenured. This threshold is chosen to keep the
survivors half full. (For the intrepid, a 1.3.1 option -XX:+PrintTenuringDistribution can be
used to show this threshold and the ages of objects in the new generation.
It is also useful for observing the lifetime distribution of an
application.)
Here are the default values for the Solaris (SPARC Platform
Edition) operating environment:
NewRatio
|
2 (client JVM:
8)
|
NewSize
|
2172k
|
MaxNewSize
|
32m
|
SurvivorRatio
|
25
|
The rules of
thumb for server applications are:
First decide the
total amount of memory you can afford to give the JVM. Then graph
your own performance metric against young generation sizes to find the
best setting.
Unless you find problems with excessive major
collection or pause times, grant plenty of memory to the young
generation. The default MaxNewSize (32MB) is generally
too small.
Increasing the young generation becomes
counterproductive at half the total heap or less.
Be sure to
increase the young generation as you increase the number of processors,
since allocation can be parallelized, but GC is not
parallel.
|
Other considerations
For most applications the permanent generation is
not relevant to GC performance. However, some applications dynamically
generate and load many classes. For instance, some implementations of JSPs
do this. If necessary, the maximum permanent generation size can be
increased with MaxPermSize.
Some applications
interact with garbage collection by using finalization and weak/soft/phantom
references. These features can create performance artifacts at a
Java-programming-language level; an example is relying on finalization to close
file descriptors, which makes an external resource (descriptors) dependent on GC
promptness. Relying on GC to manage resources other than memory is almost
always a bad idea.
Another way apps can interact with garbage
collection is by invoking GCs explicitly, such as through the System.gc() call. These calls force
major collection, and inhibit scalability on large systems. The
performance impact of explicit GCs can be measured using the unsupported flag
-XX:+DisableExplicitGC.
One of
the most commonly encountered uses of explicit GC occurs with RMI's distributed
garbage collection (DGC). Applications using RMI refer to objects in other
JVMs. Garbage can't be collected in these distributed applications without
occasional local collection, so RMI forces periodic full collection. The
frequency of these collections can be controlled with properties. For
example,
java
-Dsun.rmi.dgc.client.gcInterval=3600000
-Dsun.rmi.dgc.server.gcInterval=3600000 ...
specifies explicit collection once per hour instead of the default rate of
once per minute. However, this may also cause some objects to take much longer
to be reclaimed. These properties can be set as high as Long.MAX_VALUE to
make the time between explicit collections effectively infinite, if there is no
desire for an upper bound on the timeliness of DGC activity.
The Solaris 8
operating environment supports an alternate version of libthread that binds
threads to LWPs directly; this may help avoid starvation of the finalization
thread. To try this, set the environment variable LD_LIBRARY_PATH to include /usr/lib/lwp before launching the
JVM.
Soft references are cleared less aggressively in the server JVM than
the client. The rate of clearing can be slowed by increasing a parameter
in this way: -XX:SoftRefLRUPolicyMSPerMB=10000. The
default is value 1000, or one second per megabyte.
For large dedicated
systems, there are other special options available
to boost performance.
Conclusion
Garbage collection can become a bottleneck in highly parallel
systems. By understanding how GC works, it is possible to use a variety of
command line options to minimize that impact.
The demands of large
servers are being met with larger hardware configurations that ever
before. For these systems, the 1.4 line of JVMs will provide additional
solutions, including a 64 bit address space for even larger generations and
concurrent collection to hide the pauses associated with major collection.
As used on the web site, the terms "Java Virtual Machine and
"JVM" mean a virtual machine for the Java platform.