JEP 178: Statically-Linked JNI Libraries

OwnerBob Vandette
Created2013/02/18 20:00
Updated2014/08/04 19:58
TypeFeature
StatusClosed / Delivered
Componentcore-libs
ScopeSE
Discussionjdk8 dash dev at openjdk dot java dot net
EffortS
DurationS
Priority4
Reviewed byAlan Bateman, Alex Buckley
Endorsed byMark Reinhold
Release8
Issue8046168
Relates to8005716: Enhance JNI specification to allow support of static JNI libraries in Embedded JREs

Summary

Enhance the JNI specification to support statically linked native libraries.

Goals

  1. Modify the Java SE specification, and the JDK, to enable developers to package a Java runtime, native application code, and Java application code together into a single binary executable that does not require the use of shared native libraries.

  2. Require no changes to existing Java code in order to use a static native library as opposed to a dynamic native library. A method invocation of the form System.loadLibrary("foo"), in particular, should be able to load the "foo" library regardless of whether that library is provided in static or dynamic form.

  3. Allow a Java application to use a combination of static and dynamic native libraries, although static libraries must be in memory prior to any attempt to use them.

  4. Allow JVMTI Java Agents to be optionally statically linked with Java runtimes.

Non-Goals

It is not a goal to preserve complete native C/C++ source compatibility for existing dynamic native libraries that are converted to static form. Existing uses of the JNI_OnLoad and JNI_OnUnLoad functions will need to be modified in order to allow multiple static libraries to co-exist.

Motivation

There are two major scenarios in which static JNI libraries can be useful:

  1. Native applications that embed the JRE may wish to use statically linked JNI code rather than dynamically-linked libraries.

  2. Java applications running in environments that limit or do not support shared libraries require a JRE and all of its native API-library code to be linked into a single executable.

As an additional benefit, with statically linked JNI libraries an object-file linker can optimize the entire executable, potentially reducing its size.

Description

Two main problems need to be addressed to add support for static JNI libraries:

  1. The current Java API that initiates the dynamic-library loading process needs to be enhanced to support built-in static libraries. A Java application that uses a static JNI library needs a way to notify the VM that the library code is already included in the application image. In this situation, a System.loadLibrary request for a static library should skip the usual platform-specific dynamic loading process.

    The current JNI specification alludes to this type of support, though the Hotspot VM does not implement that behavior.

  2. The JNI_OnLoad and JNI_OnUnload function interface need to be enhanced to support library specific names since only a single function name can exist within an application. This could be implemented by appending the library name to these well-known-names. For example libnet.so could use JNI_OnLoad_net, JNI_OnUnload_net.

This feature requires changes to both the Java SE library-loading APIs and the JNI specification. What follows is an initial draft of the specification updates in both areas.

Java API Changes

The specifications of the java.lang.System.load and java.lang.Runtime.load methods will be revised to read:

Loads the native library specified by the filename argument. The filename argument must be an absolute path name.

If the filename argument, when stripped of any platform-specific library prefix, path, and file extension, indicates a library whose name is L, and a native library called L is statically linked with the VM, then the JNI_OnLoad_L function exported by the library is invoked rather than attempting to load a dynamic library. A filename matching the argument does not have to exist in the file system. See the JNI Specification for more details.

Otherwise, the filename argument is mapped to a native library image in an implementation-dependent manner.

The specifications of when these methods throw an UnsatisfiedLinkError will be revised to read:

UnsatisfiedLinkError - if either the filename is not an absolute path name, the native library is not statically linked with the VM, or the library cannot be mapped to a native library image by the host system.

The specifications of the java.lang.System.loadLibrary and java.lang.Runtime.loadLibrary methods will be revised to read:

Loads the native library specified by the libname argument. The libname must not contain any platform-specific prefix, file extension, or path.

If a native library called libname is statically linked with the VM, then the JNI_OnLoad_libname function exported by the library is invoked. See the JNI Specification for more details.

Otherwise, the libname is loaded from a system library location and mapped to a native-library image in an implementation-dependent manner.

The specifications of when these methods throw an UnsatisfiedLinkError will be revised to read:

UnsatisfiedLinkError - if either the libname argument contains a file path, the native library is not statically linked with the VM, or the library cannot be mapped to a native-library image by the host system.

JNI Specification Changes

The JNI version specification will be incremented to JNI_VERSION_1_8. Statically-linked libraries will only be supported this version or greater.

JVMTI -agentlib Command Line Option Specification Changes

The -agentlib command line specification descriptions will be revised in JDK 8 to read:

If the library argument, indicates a library whose name is L, and a native library called L is statically linked with the VM, then an Agent_OnLoad_L function must be exported by the agent. A library matching the argument does not have to exist in the file system. This Agent_OnLoad_L function will be invoked by the VM as described in the JVMTI specification. The options will be passed to the Agent_OnLoad_L function when called.

Otherwise, The name following -agentlib: is the name of the library to load. Lookup of the library, both its full name and location, proceeds in a platform-specific manner. Typically, the agent-lib-name is expanded to an operating system specific file name. The options will be passed to the agent on start-up. For example, if the option -agentlib:foo=opt1,opt2 is specified, the VM will attempt to load the shared library foo.dll from the system PATH under WindowsTM or libfoo.so from the LD_LIBRARY_PATH under the SolarisTM operating environment.

JVMTI -agentpath Command Line Option Specification Changes

The -agentpath command line specification descriptions will be revised in JDK 8 to read:

If the filename argument, when stripped of any platform-specific library prefix, path, and file extension, indicates a library whose name is L, and a native library called L is statically linked with the VM, then the Agent_OnLoad_L function must be exported by the agent. A filename matching the argument does not have to exist in the file system. This Agent_OnLoad_L function is invoked by the VM, as described in the JVMTI specification. The options will be passed to the Agent_OnLoad_L function when called.

Otherwise, The path following -agentpath: is the absolute path from which to load the library. No library name expansion will occur. The options will be passed to the agent on start-up. For example, if the option -agentpath:/myLibs/foo.so=opt1,opt2 is specified, the VM will attempt to load the shared library /myLibs/foo.so.

JVMTI Native Interface Specification Changes

com.sun.tools.attach.VirtualMachine.loadAgentLibrary

This language will be added to the javadocs for this method:

If the agent is statically linked with the VM that would otherwise load it, the specific Agent_OnAttach function name that is called will be library specific as defined in the -agentlib JVMTI specification section.

com.sun.tools.attach.VirtualMachine.loadAgentPath

This language will be added to the javadocs for this method:

If the agent is statically linked with the VM that would otherwise load it, the specific Agent_OnAttach function name that is called will be library specific as defined in the -agentpath JVMTI specification.

The JVMTI version specification will be incremented to JDK18_JVMTI_VERSION.
JDK18_JVMTI_VERSION will be set to 0x30010203, which translates to 1.2.3.

This new functionality will be supported in Virtual Machines that support JDK18_JVMTI_VERSION or greater.

Impact