Java Platform Module System: Issue Summary

Mark Reinhold

2016/12/9 13:53 -0800 [e9a93d2eb8a4]

This is the issue list for JSR 376, the Java Platform Module System. The issues documented here have been collected from the JSR 376 Expert Group mailing list, the JSR 376 comments list, the JSR 376 observers’ list, and the OpenJDK jigsaw-dev list. References to pertinent messages, though not all related messages, are included.

This is a living document, which will be updated regularly as work on the issues progresses. Each issue is assigned a unique #HashTag for use in the e-mail messages and other textual media in which these issues are discussed so that such discussions can be discovered, tracked, and summarized.

Issues by category

#ModuleNameSyntax · #ModuleNameCharacters ? · #CompileTimeDependences  · #ModuleAnnotations  · #ModuleDeprecation  · #ExportAnnotation

#MultiModuleExecutableJARs · #MultiModuleJARs · #ReifiedModuleGraphs · #ModuleNameInManifest · #AddExportsInManifest 

#ClassFileModuleName ? · #ClassFileAccPublic  · #ClassFileAccModule ? · #StandardModuleAttributes

#CyclicDependences · #MutableConfigurations · #LazyConfigurationAndInstantiation · #CustomizableAutomaticModuleNameMapping 

#ServiceLoaderEnhancements 

#ClassFilesAsResources  · #ResourceEncapsulation  · #ResourceExistenceAndSize · #ReflectiveAccessToNonExportedTypes ? · #AwkwardStrongEncapsulation ? · #ReflectionWithoutReadability  · #ReadabilityAddedByLayerCreator ? · #IndirectQualifiedReflectiveAccess ?

#AvoidConcealedPackageConflicts · #PlatformClassLoader  · #ClassLoaderNames 

#StaticLayerConfiguration · #MultipleModuleVersions · #VersionsInModuleNames  · #VersionedDependences ? · #VersionSyntax

#NonHierarchicalLayers ? · #DiscardableModules

#BootstrapClassLoaderSearchInJVMTI  · #ReflectiveAccessByInstrumentationAgents 

An issue marked with ‘’ has been resolved; an issue marked with ‘?’ has an open proposal or discussion thread.

Issues by status

Resolved

#AddExportsInManifest · #BootstrapClassLoaderSearchInJVMTI · #ClassFileAccPublic · #ClassFilesAsResources · #ClassLoaderNames · #CompileTimeDependences · #CustomizableAutomaticModuleNameMapping · #ModuleAnnotations · #ModuleDeprecation · #PlatformClassLoader · #ReflectionWithoutReadability · #ReflectiveAccessByInstrumentationAgents · #ResourceEncapsulation · #ServiceLoaderEnhancements · #VersionsInModuleNames

Proposal posted or discussion active

#AwkwardStrongEncapsulation · #ClassFileAccModule · #ClassFileModuleName · #CyclicDependences · #DiscardableModules · #IndirectQualifiedReflectiveAccess · #LazyConfigurationAndInstantiation · #ModuleNameCharacters · #MutableConfigurations · #NonHierarchicalLayers · #ReadabilityAddedByLayerCreator · #ReflectiveAccessToNonExportedTypes · #VersionedDependences

Open

#AvoidConcealedPackageConflicts · #ExportAnnotation · #ModuleNameInManifest · #ModuleNameSyntax · #MultiModuleExecutableJARs · #MultiModuleJARs · #MultipleModuleVersions · #ReifiedModuleGraphs · #ResourceExistenceAndSize · #StandardModuleAttributes · #StaticLayerConfiguration · #VersionSyntax

Module declarations

#ModuleNameSyntax — Some people find the use of the same syntax for module names and package names to be confusing. Should the recommended syntax for module names be changed to something else, e.g., lower-case words separated by underscores? Or should the syntax of module declarations be revised to expand requires into requires module, and exports into exports package, in order to disambiguate these two kinds of names? Or is there some other solution? [Rémi Forax, Stephen Colebourne, Paul Benedict]

#ModuleNameCharacters — Module names are presently constrained to be Java identifiers. Some existing module systems allow additional characters in module names, such as hyphens and slashes. Should this restriction be lifted or, perhaps, should it somehow be made layer-specific? [David Lloyd]

Proposal (revised) Do not change the treatment of module names in source code; they will remain qualified names. Revise the encoding of module names in compiled module-declaration class files to lift the current constraints but adopt new, less onerous constraints that still provide for the future evolution of the platform. Revise the format of class files to structure module and package names in a manner consistent with that already used for other kinds of constrained names. [Mark Reinhold; first proposal]

#CompileTimeDependences — Provide a means to specify a module dependence that is mandatory at compile time but optional at run time, for use with libraries that are not strictly necessary but can be leveraged if present at run time. (Detail: If a dependence is “optional at run time” then does the module system try to satisfy it during resolution but fail silently if it cannot, or does it ignore it during resolution but add the corresponding read edge to that module if it was resolved for some other reason?) [Philippe Marschall, Stephen Colebourne, Juergen Hoeller, Stephane Epardaud, Paul Benedict]

Resolution Extend the language of module declarations to allow the static modifier to be used on a requires directive, to indicate that a dependence is mandatory at compile time but optional in later phases. [Mark Reinhold]

#ModuleAnnotations — Should it be possible to annotate module declarations? As previously discussed this is technically feasible but it would add significant complexity. [Peter Kriens, Stephane Epardaud, Paul Benedict]

Resolution Allow arbitrary annotations to be placed on module declarations. [Mark Reinhold]

#ModuleDeprecation — If module declarations cannot be annotated, should there be some other way to deprecate modules? [Rémi Forax]

Resolution Allow arbitrary annotations to be placed on module declarations, and revise the java.lang.Deprecated annotation so that it can be applied to modules. [Mark Reinhold]

#ExportAnnotation — Define a standard, source-retention Export annotation which can be used in a package-info.java file to indicate that the package is to be exported. Tools that create module-info.java files could then interpret this annotation in order to generate corresponding exports clauses. [Paul Benedict]

Module artifacts

#MultiModuleExecutableJARs — Provide a means to create an executable modular “uber-JAR” that contains more than one module, preserving module identities and boundaries, so that an entire application can be shipped as a single artifact. [This issue has not come up in e-mail but was raised multiple times in person at JavaOne 2015, Devoxx BE 2015, and FOSDEM 2016.]

#MultiModuleJARs — Provide a means to package more than one module in a single artifact, while preserving module identities and boundaries. (This is a generalization of #MultiModuleExecutableJARs.) [Stefan Fuchs]

#ReifiedModuleGraphs — Define a means to record, in some type of artifact, the module graph that results from resolving a given main module against a given application module path. The artifact could contain full copies of the corresponding module artifacts, or references to such artifacts, or some combination of the two. At run time such an artifact could be used to create a new layer. [Peter Kriens]

#ModuleNameInManifest — For ease of migration, consider allowing the name of a module to be specified in the MANIFEST.MF file of a JAR file. If a JAR file with such a manifest entry is not a modular JAR (i.e., it has no module-info.class file), but is used as an automatic module, then the name of the automatic module is that specified in the manifest rather than that derived from the name of the JAR file itself. This would allow the maintainer of an existing library to specify a preferred module name simply by updating the library’s manifest. If a JAR file with such a manifest entry is a modular JAR then the module name in the module-info.class file determines the module name, and the name in the manifest is expected to match. In both cases, the presence of the module name in the manifest is useful to tools that must run on older versions of the platform, since they can use the existing java.util.jar API to read the module name. [Paul Benedict]

#AddExportsInManifest — Using a command-line option such as --add-exports to make JDK-internal APIs available to existing code is difficult, if not impossible, for applications that are delivered as executable JAR files. To ease migration, consider allowing such a JAR file to include the equivalent of such options in its MANIFEST.MF file. (Cf. #ReflectiveAccessToNonExportedTypes.) [Simon Nash]

Resolution Define two new JAR-file manifest attributes to allow a package in an explicit module to be exported, or exported privately, at run time. [Mark Reinhold]

Module descriptors

#ClassFileModuleName — The name of a module is not a simple UTF-8 string but is, rather, derived from the value of the this_class field of the ClassFile structure, which is awkward. [Rémi Forax]

Proposal Revise the binary form of module declarations to add an explicit module-name field and to zero out the this_class field. [Mark Reinhold]

#ClassFileAccPublic — The ACC_PUBLIC constant in a requires_flags should be encoded as 0x0001, as it is elsewhere in the JVMS, rather than as 0x0020, which has different meanings in other contexts. [Rémi Forax]

Resolution Rename this constant to ACC_TRANSITIVE. [Mark Reinhold]

#ClassFileAccModule — The ACC_MODULE constant is currently specified to have the value 0x8000. This is the last available bit remaining across all of the various access_flags fields of a class file, and thus should be reserved for some unspecified future purpose where it may be useful to use the same value in all such fields. Alternative candidates for ACC_MODULE include 0x0040 (overlaps with ACC_VOLATILE and ACC_BRIDGE) and 0x0080 (ACC_TRANSIENT and ACC_VARARGS). [Rémi Forax]

Proposal Do not change the value of this constant; there are other, better ways to cope with the future. [Mark Reinhold]

#StandardModuleAttributes — The current draft defines a set of optional module-info class-file attributes for extended information not required by the module system but useful to tools: A version string, the name of the module’s main class, the names of the module’s non-exported packages, and, for modules that contain native code, the operating-system name and version and the processor-architecture name. Is this the right set of extended attributes? Should others be considered? Candidates include, but are not limited to: Maintainer contact information, project URL, originating source-code repository URL and changeset identification, and copyright and license notices (or references to same). (Cf. the OS-specific module packaging requirement.) [Peter Kriens]

Module graphs

#CyclicDependences — The current draft disallows cycles when the module graph is initially resolved at compile time, link time, and run time. Cycles can arise later on at run time if readability edges are added for automatic modules, or via reflection. The rationale for disallowing cycles during resolution is that it makes the module graph easier to reason about, it simplifies the module system itself, and that, philosophically, any modules involved in a cycle are logically one module anyway, so they should be defined as such in the first place. This constraint is not, however, a documented requirement, and some have argued that cycles should be permitted. [Jochen Theodorou, Stephane Epardaud]

Discussion [Mark Reinhold]

#MutableConfigurations — Configurations are immutable: Once created, they cannot be modified. Some have argued that this will make it impossible for container applications to add and remove modules at run time. Should configurations, and hence the aspects of layers that depend upon configurations, be made mutable instead? [David Lloyd]

Discussion [Mark Reinhold]

#LazyConfigurationAndInstantiation — The construction of a configuration loads all of the relevant module descriptors, and the instantiation of a configuration into a layer defines all of its modules to the virtual machine. There is some concern that this could result in significant startup overhead for layers with thousands or tens of thousands of modules, which some consider to be realistic scenarios. If the startup overhead is actually that bad in such settings, should configuration and layer creation be revised to use incremental algorithms? [David Lloyd]

Discussion [Mark Reinhold]

#CustomizableAutomaticModuleNameMapping — Provide a means to customize the algorithm that computes the names and version strings of automatic modules. [Paul Benedict]

Resolution Extend the java.lang.module.ModuleDescriptor.Builder API so that it can be used to create automatic as well as explicit modules. [Mark Reinhold]

Services

#ServiceLoaderEnhancements — The module system encourages the use of services for loose coupling, but the ServiceLoader class is not very flexible. Consider enhancing it so that (1) neither a provider class nor its no-args constructor need be declared public, (2) a provider can be a singleton, or perhaps a collection of singletons, and (3) the classes of the available providers can be inspected and selected prior to instantiation. [Stephen Colebourne]

Resolution Enhance the ServiceLoader API to implement suggestions (2) and (3), but not (1). [Mark Reinhold]

Reflection

#ClassFilesAsResources — If a type is visible and was loaded from a class file then it should be possible to read that file by invoking the getResourceAsStream method of the type’s class loader, as it is in earlier releases. [Uwe Schindler, Rafael Winterhalter]

Resolution Special-case resource names ending in ".class" so that they can be located by code in any module. [Mark Reinhold; original proposal]

#ResourceEncapsulation — The Module::getResourceAsStream method can be used to read the resources of any named module, without restriction, which violates the resource encapsulation requirement. This method should be restricted somehow so that only “suitably-privileged” code (for some definition of that term) can access resources in a named module other than its own. An alternative is to drop this requirement (cf. #ClassFilesAsResources). [Mark Reinhold]

Resolution Extend the notion of open modules and open packages, introduced for #ReflectiveAccessToNonExportedTypes, to govern whether a resource defined in a named module can be located by code in some other module. [Mark Reinhold; original proposal]

#ResourceExistenceAndSize — Given a Module object and a resource name, should there be a way to determine whether that resource exists in that module and, if it does exist, what its size is, without actually opening the resource as a stream? One way to do this would be to introduce a new abstraction to represent resources. [David Lloyd]

#ReflectiveAccessToNonExportedTypes — Some kinds of framework libraries require reflective access to members of the non-exported types of other modules; examples include dependency injection (Guice), persistence (JPA), debugging tools, code-automation tools, and serialization (XStream). In some cases the particular library to be used is not known until run time (e.g., Hibernate and EclipseLink both implement JPA). This capability is also sometimes used to work around bugs in unchangeable code. Reflective access to non-exported packages can, at present, only be enabled via command-line flags, which is extremely awkward (cf. #AddExportsInManifest). Provide an easier way for reflective code to access such non-exported types. [Reinier Zwitserloot, Paul Benedict, Alan Snyder, Rémi Forax, Simon Nash]

Proposal (revised²) Extend the language of module declarations with the concept of open modules and introduce the new per-package directive open to allow all of the non-public elements of a package, or else such elements of specific packages, to be accessed via the AccessibleObject::setAccessible method of the core reflection API. Rename requires public to requires transitive to avoid confusion. [Mark Reinhold; first proposal; second proposal]

#AwkwardStrongEncapsulation — A non-public element of an exported package can still be accessed via the AccessibleObject::setAccessible method of the core reflection API. The only way to strongly encapsulate such an element is to move it to a non-exported package. This makes it awkward, at best, to encapsulate the internals of a package that defines a public API. [Martin Buchholz & Aleksey Shipilev, private communication, August 2016]

Proposal (see that for #ReflectiveAccessToNonExportedTypes)

#ReflectionWithoutReadability — Having to add read edges dynamically just to enable reflection is painful, and could slow migration and adoption. Consider relaxing the access model so that reflection does not require, or perhaps simply assumes, readability. [Rémi Forax, Peter Levart]

Resolution Adopt the second suggested alternative. Revise the core reflection APIs in the java.lang.reflect package to assume that any module that contains code that invokes a reflective operation can read the module that defines the types that are the subject of that operation. [Mark Reinhold]

#ReadabilityAddedByLayerCreator — Provide a means by which the code that creates a layer can add readability edges from the modules in that layer to other modules, whether those modules are in that layer or in other layers. [Thomas Watson]

Proposal Revise java.lang.reflect.Layer so that the multi-parent layer-creation methods added for #NonHierarchicalLayers return a capability object that defines an addReads method. [Mark Reinhold]

#IndirectQualifiedReflectiveAccess — Provide a means by which a client module can grant qualified reflective access to a framework module that is not known at compile time, assuming that the name of some other module that represents the framework module is known. A canonical example of this case is a client POJO module compiled against a module that defines the JPA API. At run time the client module, or a container or some other code acting on its behalf, must grant qualified reflective access to its POJO packages to the JPA implementation in actual use. [David Lloyd, Mark Reinhold]

Proposal Revise java.lang.reflect.Module::addOpens so that a module can delegate access to packages that are open to it; extend the layer capability proposed for #ReadabilityAddedByLayerCreator with an addOpens method; and extend the java.lang.invoke API so that framework authors can adopt method handles in preference to the core reflection API. [Mark Reinhold]

Class loaders

#AvoidConcealedPackageConflicts — Provide a simple means to load two modules, without using reflection, when they contain non-exported packages of the same name. This could be done by loading them via different class loaders or, alternatively, via #StaticLayerConfiguration. [Karl Sanders]

#PlatformClassLoader — Add a static method to java.lang.ClassLoader to return a class loader via which all of the built-in Java SE and JDK types are visible. (This is, at present, the loader formerly known as the “extension” class loader.) [Alan Snyder]

Resolution Add the suggested method.

#ClassLoaderNames — Enhance class loaders to have optional names, so that external module systems can provide better diagnostics. When the run-time system generates a stack trace or an exception message that mentions a module name and version, if present, then it should also insert the name of that module’s class loader, if present. [David Lloyd, Mark Reinhold].

Resolution Add a string-returning getName() method to java.lang.ClassLoader and corresponding constructors to that class, java.net.URLClassLoader, and java.security.SecureClassLoader. Arrange for stack traces to convey class-loader names when present. [Mark Reinhold]

Versioning

#StaticLayerConfiguration — Layers can, at present, only be created dynamically, via reflection. If there were a way to specify them statically, i.e., at startup time, then they could be used to solve the kinds of version conflicts that are presently addressed via shading or shadowing. [Ron Pressler, Nicolai Parlog]

#MultipleModuleVersions — Allow multiple distinct modules of a given name to be loaded in a convenient fashion, without using reflection. This could be done by creating new layers automatically, or by relaxing the constraints on multiple versions within a layer, or by some other means (cf. #StaticLayerConfiguration, #AvoidConcealedPackageConflicts). Addressing this issue may entail reconsidering the multiple versions non-requirement. [Mike Hearn]

#VersionsInModuleNames — Some have argued that library maintainers will be tempted to encode major version numbers, or even full version numbers, in module names. Is there some way we can guide people away from doing that? [Mike Hearn, Paul Benedict]

Resolution Revise the automatic-module naming algorithm not to create module names that end in digit characters, and revise javac to emit a lint warning when such module names are encountered in source code. [Mark Reinhold]

#VersionedDependences — Consider allowing specific version strings, or perhaps version constraints, as an optional element of requires clauses in module declarations. Failing that, consider allowing specific version strings, or perhaps version constraints, to be added to the dependences recorded in a compiled module descriptor; this would, e.g., allow a compiler or build system to record the versions of the modules against which a particular module was compiled, for use by other tools. In either case, if such version information is merely informative then it will still honor the version selection non-requirement; if such version information is interpreted by the module system then that requirement may come into question. [Cristiano Mariano, Stephane Epardaud]

Proposal When compiling a module that depends on some other modules, record the version strings of those modules, if available, in the resulting module descriptor. [Mark Reinhold]

#VersionSyntax — Revise the syntax of module version strings to accommodate a wider variety of existing version-string schemes [David Lloyd]. Alternatively, abandon the requirement to impose a total order on version strings so that we don’t even have to bother to parse them [Rémi Forax].

Layers

#NonHierarchicalLayers — Layers are presently constrained to be hierarchical, i.e., each layer has at most one parent. Should this restriction be relaxed so that a layer can have more than one parent? Some have argued that this will be essential to the adoption of the module system by a future version of the Java EE Platform. It would also enable bidirectional interoperation with existing module systems such as OSGi. [David Lloyd, Mark Reinhold, Thomas Watson]

Proposal Add static methods that take a list of parents, rather than a single parent, to java.lang.module.Configuration and java.lang.reflect.Layer. [Mark Reinhold]

#DiscardableModules — Provide a means by which a sub-graph of modules within a layer can be discarded, so that both those modules and their class loaders can be reclaimed. [Thomas Watson]

Discussion [Mark Reinhold]

Tooling

#BootstrapClassLoaderSearchInJVMTI — The JVM TI API defines a function, AddToBootstrapClassLoaderSearch, by which an instrumentation agent can append a file-system path element to the bootstrap class loader’s class path. The bootstrap class path, as such, no longer exists, but this function remains supported for now. Should this function be deprecated, with the intent to remove it in a future release, or should it remain supported? It appears to be critical to the operation of at least one popular instrumentation agent. [Andrew Dinn]

Resolution Retain this method. Do not deprecate it. [Mark Reinhold]

#ReflectiveAccessByInstrumentationAgents — As noted in the related issue #ReflectiveAccessToNonExportedTypes, the core reflection API cannot be used to gain access to members of packages that are not exported by their defining modules. Even if that issue is addressed for the use cases cited, a distinct use case is that of instrumentation agents that need to enable injected code to invoke non-public methods and access non-public fields. A possible solution for this use case is specifically to provide instrumentation agents with the ability to use core reflection, or some equivalent, even on elements of non-exported packages. [Andrew Dinn]

Resolution Extend the java.lang.instrument.Instrumentation interface, and the corresponding JVM TI native API, to support a redefineModule method, roughly analogous to the existing redefineClasses method. [Mark Reinhold; evaluation by Andrew Dinn]