JEP draft: Generational Shenandoah

AuthorsBernd Mathiske, Kelvin Nilsen, William Kemper
OwnerBernd Mathiske
TypeFeature
ScopeImplementation
StatusSubmitted
Componenthotspot / gc
Discussionhotspot dash gc dash dev at openjdk dot java dot net
EffortL
DurationL
Reviewed byRoman Kennke
Created2021/02/01 22:49
Updated2021/03/03 18:49
Issue8260865

Summary

Enhance the Shenandoah garbage collector with generational collection capabilities to improve sustainable throughput, load spike resilience, and memory utilization.

Goals

No trade-off between the memory frugality and elasticity of G1 vs. 
the short pause times of single-generation Shenandoah.

Non-Goals

Success Metrics

Motivation

Garbage collectors with concurrent compaction are capable of completely blending GC pause times into the single-digit millisecond range of other common JVM pauses, while also leaving mutator execution speed nearly unfettered. The Shenandoah garbage collector in OpenJDK provides this ideal GC behavior for latency-sensitive Java applications. However, it can only achieve this within a limited operational envelope (combinations of heap occupancy and allocation rate). In other words, Shenandoah isrelatively CPU-hungry and memory-hungry, mostly the latter. Compared to the generational collectors G1, CMS, and Parallel, it tends to require more heap headroom andwork harder to recover space occupied by unreachable objects.

A generational collector is not necessarily noticeably disadvantaged if the generational hypothesis, “most objects die young” [Ungar ‘84], does not hold. Auto-tuning in region-based collectors can adjust generation sizes and copying (“promotion”) policies dynamically, based on observed object demographics. Worst case, surviving objects get copied a few times “too often”, and this becomes the norm, not the exception. This cost is, however, dwarfed by repeated concurrent marking of long-lived objects in the old generation. In that case, there is not much difference left compared to a single-generation collector.

A concurrent collector that is also generational and can dynamically adjust its young generation’s size and related operational parameters can both achieve low pause times and stay competitive on all other performance aspects.

Description

This enhancement of the Shenandoah garbage collector separates the Java heap into two generations. Like in other generational collectors, GC efforts then focus on the "young" generation, the one in which allocations by the mutator occur and where ephemeral objects can be reclaimed with reduced effort. We propose the following approaches for an initial implementation.

The collection algorithms operating on each generation are closely based on traditional Shenandoah. Within the young generation, Generational Shenandoah replicates the same heuristics as traditional Shenandoah to distinguish areas of memory that hold newly allocated objects from areas of memory holding objects that survived one or more recent young-generation GC passes.

Each generation is formed by a subset of the Shenandoah heap’s regions. At any given time, a region is considered either free or dedicated to either the young or the old generation. The size of each generation is given by its occupied regions plus a quota of free regions. Overreach into the free quota of the respective other generation is tolerated, but it accelerates collection triggering and can lead to degenerate and full collections. The algorithms to control collection phase scheduling will be refined based on tests once a prototype is available. This also applies to dynamic young generation sizing and other auto-tuning mechanisms such as tenuring age.

Shenandoah has a unique Load Reference Barrier (LRB) that supports 32-bit builds and compressedOops in 64-bit builds. To not increase its mutator impact, this same LRB is used for both generations, without any changes. This leads to a singleton evacuator, which is shared between old and young collection efforts. Typical evacuation phases collect garbage either exclusively from young memory regions or from a combination of young and old memory regions. This behavior mimics G1’s young and mixed collections. The principal improvement over G1 is that generational Shenandoah’s “mixed” collections are concurrent to the mutator.

The generation-specific marking phases, however, are largely decoupled from each other. Concurrent old generation marking proceeds in the background while young generation marking occurs multiple times. It can also be suspended as needed to execute other collection phases. Once old generation marking has completed, subsequent evacuations and reference updates will include old generation regions until the entire old-generation collection set has been processed.

As remembered set implementation, we use existing card marking code and supplement code for remembered scanning that is concurrent with mutator execution.

Shenandoah’s existing SATB barriers are generalized to serve the combined needs of young-generation and old-generation concurrent marking. The post-processing of SATB buffers treats references to old-generation memory differently than references to young-generation memory. But the fast path through these barriers remains unchanged.

Building and Invoking

The new generational feature forms part of the Shenandoah code base in OpenJDK. But it has no runtime effect unless it is activated by this JVM command line option:

-XX:ShenandoahGCMode=generational

These existing options for generational garbage collectors (such as Parallel, G1, CMS) go into effect:

-XX:NewRatio=<n>
-XX:NewSize=<size>
-XX:MaxNewSize=<size>

Please see the project wiki (once available) for more information on how to setup and tune Generational Shenandoah.

Alternatives

Azul Systems’ C4, is already generational, but not available in open source. ZGC would be another excellent code base to further develop into a generational concurrent collector. However, neither of these options supports “compressedOops” (32-bit object references in 64-bit JVMs) and the vast majority of Java heaps we see (e.g., in cloud services) are well below 32 GB in size, which is the upper bound for using this space-saving and performance-improving feature.

Testing

Most of our existing functional and stress tests are collector-agnostic and can be reused as-is. Additional test run configurations for the new generational mode will be integrated. Furthermore, functional, performance, and stress tests specific to Generational Shenandoah will be added.

Risks and Assumptions

Dependencies