JEP 238: Multi-Release JAR Files
|Component||tools / jar|
|Discussion||core dash libs dash dev at openjdk dot java dot net|
|Reviewed by||Alan Bateman, Brian Goetz, Paul Sandoz, Steve Drach|
|Endorsed by||Brian Goetz|
Extend the JAR file format to allow multiple, Java-release-specific versions of class files to coexist in a single archive.
Enhance the Java Archive Tool (
jar) so that it can create multi-release JAR files.
Implement multi-release JAR files in the JRE, including support in the standard class loaders and
Enhance other critical tools (e.g.,
jdeps, etc.) to interpret multi-release JAR files.
Support multi-release modular JAR files for goals 1 to 3.
Preserve performance: The performance of tools and components that use multi-release JAR files must not be significantly impacted. In particular, performance when accessing ordinary (i.e., not multi-release) JAR files must not be degraded.
Third party libraries and frameworks typically support a range of Java platform versions, generally going several versions back. As a consequence they often do not take advantage of language or API features available in newer releases since it is difficult to express conditional platform dependencies, which generally involves reflection, or to distribute different library artifacts for different platform versions.
This creates a disincentive for libraries and frameworks to use new features, that in turn creates a disincentive for users to upgrade to new JDK versions---a vicious circle that impedes adoption, to everyone's detriment.
Some libraries and frameworks, furthermore, use internal APIs of the JDK that will be made inaccessible in Java 9 when module boundaries are strictly enforced. This also creates a disincentive to support new platform versions when there are public, supported API replacements for such internal APIs.
A JAR file has a content root, which contains classes and resources, as
well as a
META-INF directory which contains metadata about the JAR.
By adding some versioning metadata to specific groups of files the JAR
format can encode, in a compatible way, multiple versions of a library
for different target Java
A multi-release JAR ("MRJAR") will contain the main attribute:
declared in the main section of the JAR
MANIFEST.MF. The attribute
name is also declared as a constant
Like other main attributes the name declared in the
case insensitive. The value is also case-insensitive, but there must be
no preceding or trailing white space (such a restriction helps ensure
the performance goal is met).
A multi-release JAR ("MRJAR") will contain additional directories for classes and resources specific to particular Java platform releases. A JAR for a typical library might look like this:
jar root - A.class - B.class - C.class - D.class
Suppose there are alternate versions of A and B that can take advantage of Java 8 features, and yet another version of A that can take advantage of Java 9 features. We can bundle them into a single JAR as follows:
jar root - A.class - B.class - C.class - D.class - META-INF - versions - 8 - A.class - B.class - 9 - A.class
In a JDK that does not support MRJARs, only the classes and resources in the root directory will be visible, and the two packagings will be indistinguishable. In a JDK that does support MRJARs, the directories corresponding to any later Java platform release would be ignored; it would search for classes and resources first in the Java platform-specific directory corresponding to the currently-running major Java platform release version, then search those for lower versions, and finally the JAR root. On a Java 9 JDK, it would be as if there were a JAR-specific class path containing first the version 9 files, then the version 8 files, and then the JAR root; on a Java 8 JDK, this class path would contain only the version 8 files and the JAR root.
By this scheme, it is possible for versions of a class designed for a later Java platform release to override the version of that same class designed for an earlier Java platform release. In the example above, when running on an MRJAR-aware Java 8 JDK, it would see the 8-specific versions of A and B and the general versions of C and D; on an MRJAR-aware Java 9 JDK, it would see the 9-specific version of A and the 8-specific version of B; on older or non-MRJAR-aware JDKs it would only see the base versions of all.
JAR metadata, such as that found in the
MANIFEST.MF file and the
META-INF/services directory, need not be versioned. An MRJAR is
essentially one unit of release, so it has just one release version
(which is no different from a normal JAR, distributed say via Maven
Central), even though internally it contains multiple versions of a
library implementation for use on different Java platform releases.
Every version of the library should offer the same API; investigation is
required to determine whether this should be strict backwards
compatibility where the API is exactly the same (byte code signature
equality), or whether this can be relaxed to some degree without
necessarily enabling the introduction of new enhancements that would
blur the notion of one unit of release. This may imply, at a minimum,
that a public class present in a release-specific directory should also
be present in the root, though it need not be present in an earlier
release directory. The run-time system will not verify this property,
but tooling can and should detect such API compatibility issues, and a
library method may also be provided to perform such varification (for
Ultimately, this mechanism enables library and framework developers to decouple the use of APIs in a specific Java platform release version from the requirement that all their users migrate to that version. Library and framework maintainers can gradually migrate to and support new features while still carrying around support for the old features, breaking the chicken-and-egg cycle so that a library can be "Java 9-ready" without actually requiring Java 9.
The following components of the JDK will be modified in order to support multi-release JAR files.
URLClassLoadermust read selected versions of class files as indicated by the running Java platform version. The module-based class loader introduced with Project Jigsaw will require similar modifications.
The protocol handler for the
jarURL scheme and
java.util.jar.JarFileclass must select the appropriate version of a class from a multi-release JAR.
The Java compiler (
javac), via the underlying
ZipFileSystemAPIs, must read selected versions of class files as specified by the
-releasecommand-line options. The tools
wsgenwill leverage the underlying changes to
The Java Archive tool (
jar) will be enhanced so that it can create multi-release JAR files.
The JAR packing tool (
unpack200) must be updated (see JDK-8066272).
javaptool must be updated to enable selection of versioned class files.
jdepstool will require modifications to display version information and follow version specific class file dependencies.
The JAR specification must be revised to describe the multi-release JAR file format and any related changes (e.g., possible additions to the manifest).
By default the behaviour of
java.util.jar.JarFile and the
protocol handlers will remain the same. It is necessary to opt-in to
JarFile pointing to a MRJAR for version selection of
entries. Likewise it is necessary to opt-in for
jar URLs (see next
section for details).
JarFile instances created by the runtime for class loading will opt-in
and create instances that are configured to select entries according to
the version of the running Java platform. Such as
is referred to as being runtime versioned.
Class loader resources
A resource URL, produced by a class loader, identifying a resource in a
MRJAR will contain additional information signalling to the URL protocol
handler that the associated entry is to be processed as runtime
versioned. The URL fragment
#runtime will be present on such
resources. The URL protocol handler will detect that fragment value and
Changing the structure of resources URLs is not without risk. It is not unprecedented for code to directly process the characters of a URL without decomposing that URL into it's relevant components. Such code, for example,
is obviously incorrect but currently works and will break for versioned resource URLs.
Introducing a new URL scheme
mrjar may have less impact in certain
respects. Note that Project Jigsaw bringing Java modularity to the Java
platform will introduce a new URL scheme for resources contained in the
modular runtime image.
An alternative is to reify the resource URL to point to the versioned
entry. For a URL
r to a versioned resource in a MRJAR:
URL r = loader.getResource("foo/baz/resource.txt");
it's value may be:
A common approach is to use a static reflective check to determine if an API feature is present or not and accordingly select an appropriate class that respectively depends on that feature or not. The reflective cost is incurred at class initialization and not every time the dependent feature is used. A Java platform release is selected for compilation with the source and target flags set to a lower release to generate class files compatible with that lower release. This approach is often augmented with tools such as Animal Sniffer to check for API incompatibilities, where in addition to enforcing API compatibility code can be annotated to state whether it depends on a later Java platform release. There are a number of limitations with this approach:
The reflective checks need to be carefully maintained.
It is not possible to utilize newer language features.
If a platform release API feature is removed (perhaps an internal API) then dependent code will fail to compile.
"Fat" class files were considered, where a class may have one or more methods targeted to different Java platform versions. This was deemed too complicated in terms of the language and runtime features required to support such method declarations and dynamic selection.
Method handles (
invokedynamic) cannot be used because of the need to
maintain binary compatibility.
Risks and Assumptions
It is anticipated that the production of MRJARs is primarily compatible with existing popular build tools and therefore IDEs that support such tools, but the developer experience could be improved with enhancements.
The source layout and building of an MRJAR file can be supported by
Maven using a multi-module project. For example, see this
example Maven project that can produce a, currently rudimentary, MRJAR
file. There would be a sub-project for the root and specific Java
platform releases, and a sub-project to assemble the aforementioned
sub-projects into an MVJAR. The assembly process could be enhanced,
perhaps using a specific Maven plugin, leveraging the same features as
jar tool to enforce backwards compatibility.
The design and implementation of the runtime processing of MRJARs
currently assumes that a runtime uses the URL class loader or a custom
class loader leverages
JarFile to obtain platform-specific class
files. Runtimes whose class loaders use
ZipFile to load classes will
not be MRJAR aware. Popular application frameworks and tools, such as
Jetty, Tomcat, and Maven, etc., need to be checked for compatibility.
JEP 247 (Compile for Older Platform Versions), which supports compiling against older versions of the platform libraries, may aid build tools in the production of multi-release JAR files.