Supporting OSGi Bundles in the Java Module System

Mandy Chung and Stanley Ho
Version 0.1
April 28, 2008
Copyright 2008 Sun Microsystems, Inc.

JSR 277 Early Draft defines the framework for the Java Module System and provides the API to access Java modules such as searching for modules, loading and initializing a module, and also managing its life time. JSR 277 enables one or more module systems run on a single Java virtual machine. This document1 defines how JSR 277 supports OSGi bundles as defined in JSR 291. This document is a work in progress and is expected to evolve based on more input from the Expert Group.

Section 1 describes various definitions in the Java Module System. Section 2 describes the requirements for supporting OSGi bundles in the Java Module System and section 3 describes the proposed API changes. Section 4 specifies how to map an OSGi bundle to a ModuleDefinition so they can be consumed by other modules in the Java Module System. Section 5 describes the repository mechanism that enables the OSGi module system to be plugged into the Java Module System framework to make OSGi bundles accessible by other module systems at runtime. Appendix A shows the code example illustrating how a JAM module system implementation uses the JSR 277 API to search and import OSGi bundles.

1This draft is built upon from the basis of the past EG discussions, proposals and suggestions for the interoperability support especially the ideas and invaluable inputs from Glyn Normington, Richard Hall and Bryan Atsatt.

0. Current Status

The JSR 277 EDR2 is being updated to make the distinction between the framework/abstractions for the Java Module System and the JAM module system (See Section 1) clearer.

This section highlights the main items that remain to be sorted out.

  1. Versioning scheme (see Section 4.2)
  2. Security
  3. Java Module Events

1. Definitions

Java module

A development abstraction that exists at compile-time in the Java programming language and is reified in the Java Virtual Machine.

Module Definition

A deployment abstraction that contains metadata, classes and resources, and is reified as ModuleDefinition as part of a Java Module System implementation.

JAM module

A concrete, portable deployment format which implements the Module Definition. Amongst other details, it has metadata, classes, resources, and can contain embedded JAR files and native libraries. The distribution format is called JAM (JAva Module) that is based on the JAR file format.

Java Module System

A set of runtime abstractions that includes ModuleSystem, Repository, ModuleDefinition (see "Module Definition" above), Module, ImportDependency and others.

JAM Module System

A concrete implementation of the Java Module System that supports JAM modules. It is the default implementation of the Java Module System for the Java SE platform.

OSGi Module System

A concrete implementation of the Java Module System by an OSGi container.

2. Requirements

  1. It shall be possible for an OSGi container to implement the Java Module System.
  2. It shall be possible for a JAM module to express an import dependency on any Module Definition in any Java Module System.
Below provides an example to illustrate how a JAM module imports OSGi bundles.

A Wombat application is a JAM module that processes shopping orders for a website. It depends on the Apache Xerces XML parser and the Apache Derby and both of them are available as OSGi bundles. The module definition of the Wombat application looks like this:

  //
  // com/wombat/app/module-info.java
  //
  @Version("1.0") 
  @ImportModules({
      @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+")
      @ImportModule(name="org.apache.derby", version="10.0+")
  })
  module com.wombat.app;
  
This Wombat application is packaged as a JAM file named "com.wombat.app-1.0.jam" and the OSGi bundles it depends on are packaged as JAR files containing the following manifests,2 per the OSGi specifications:
  org.apache.xerces.parsers:
     Bundle-SymbolicName: org.apache.xerces.parsers
     Bundle-Version: 2.9.10

     Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging",
                     org.apache.xerces.jaxp; version=2.6.6,
                     org.apache.xerces.framework; version=2.6.6,
                     org.apache.xerces.readers; version=2.6.6,
                     org.apache.xerces.utils; version=2.6.6,
                     org.apache.commons.logging; version=1.0.4
     Import-Package: javax.xml.parsers; version=1.2.0,
                     org.w3c.dom; version=1.0.0,
                     org.xml.sax; version=2.0.1
     Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional
                     
  org.apache.derby:
     Bundle-SymbolicName: org.apache.derby
     Bundle-Version: 10.0.2
     Export-Package: org.apache.derby.authentication,
                     org.apache.derby.database,
                     org.apache.derby.io,
                     org.apache.derby.jdbc
     Import-Package: java.sql
  
2These example manifests are for illustration purpose and they do not represent the manifest in the actual Apache Xerces XML Parser and Apache Derby bundles.

3. Proposed API Changes

This section describes the proposed API changes for supporting OSGi bundles in the Java Module System.

3.1 ModuleSystem class

The ModuleSystem specification is updated to support multiple implementations running in a single Java virtual machine. This update is necessary to meet the requirement (1). See JSR 277 EDR2 for the full specification.

A ModuleSystem implementation is responsible for creation, management, and release of Java Modules owned by this module system. The ModuleSystem specification does not define the resolution algorithm and the distribution format that a module system supports. Instead, the resolution algorithm and the distribution format are specific to a ModuleSystem implementation.

3.1.1 Module Initialization

This section is a clarification to the JSR 277 EDR about the initialization of a Module instance.

The ModuleSystem.getModule(ModuleDefinition) method returns a fully initialized Module instance3 for a given ModuleDefinition. A Module instance is fully initialized if the following conditions are all met:

  1. Its import dependencies have been resolved. ModuleDefinitions for the imported Java modules satisfying the import dependencies and all its constraints are found using the algorithm defined by this ModuleSystem.
  2. Module instances for its imports are successfully instantiated and initialized by its owning ModuleSystem.
  3. This ModuleSystem has performed its own type consistency checking successfully in the Module instance.
  4. If this ModuleSystem supports an initializer4 to be invoked before a Module instance is fully initialized, the initializer for this Module instance is invoked.
In addition, a ModuleSystem can support import constraints specific to its algorithm to tailor the resolution of the imports for a ModuleDefinition. The constraints specified in a ModuleDefinition is only known to its owning ModuleSystem and other ModuleSystem implementations are not required to understand them.

3Initializing an OSGi bundle is equivalent to resolving, wiring and starting an OSGi bundle.

4The ModuleSystem specification does not define a generic initializer mechanism for ModuleDefinitions. The JAM module system supports the module initializer through the ModuleInitializer API and the ModuleInitializerClass annotation (see JSR 277 EDR 2). The OSGi module system supports the module initializer through the bundle activator.

3.1.2 Constraint Checking

A new method ModuleSystem.getModules is added to allow a ModuleSystem implementation to instantiate Module instances for multiple ModuleDefinitions in the same resolution and also to enforce constraints specified in these Modules.
  ModuleSystem class:
      /**
       * Returns the list of Module instances for the imports 
       * in the same resolution for the specified importer.
       * The returned Module instances are instantiated and initialized
       * using the algorithm specific to this ModuleSystem.
       *
       * This method is called by a ModuleSystem when initializing
       * a Module instance (importer) that imports Modules (imports) 
       * from this ModuleSystem.
       */
      public List<Module> getModules(ModuleDefinition importer,
                                             List<ModuleDefinition> imports)
          throws ModuleInitializationException;
  
OSGi allows to put constraints on the importer through the metadata for an exported package in another bundle. The OSGi module system can enforce the constraints on the importer in the implementation of this getModules method. The importer can be a ModuleDefinition from other ModuleSystem.

For example, the Apache Xerces XML parser bundle in Example 1 of Section 1 exports the org.apache.xerces.parsers package whose export definition has a "use" constraint. This export definition puts a constraint on the importer of the org.apache.xerces.parsers package to use the org.apache.commons.logging package wired to the org.apache.xerces.parsers package in the same resolution. The 1.0 version of the Wombat application imports the org.apache.xerces.parsers bundle and the org.apache.derby bundle. The org.apache.xerces.parsers package will get wired to the version 1.0.4 logging package and the constraint is satisfied and thus it can be wired successfully.

Let's say a new version of the Wombat application (say version 1.2) is updated and it depends on an additional Apache Commons Logging utility which is also an OSGi bundle.

  //
  // com/wombat/app/module-info.java
  //
  @Version("1.2") 
  @ImportModules({
      @ImportModule(name="org.apache.xerces.parsers", version="2.6.6+")
      @ImportModule(name="org.apache.derby", version="10.0+")
      @ImportModule(name="org.apache.commons.logging", version="2.0+")
  })
  module com.wombat.app;
  
The bundle manifest header for the Apache Commons Logging utility is:
  org.apache.commons.logging:
     Bundle-SymbolicName: org.apache.commons.logging
     Bundle-Version: 2.0
     Export-Package: org.apache.commons.logging; version=2.0
  
The 1.2 version of the Wombat application imports the org.apache.commons.logging bundle that violates the constraint if the org.apache.xerces.parsers package is wired to the version 1.0.4 logging package but the Wombat application requires the version 2.0 logging package. The version 1.2 of the Wombat application should fail to initialize due to this constraint. In other words, the getModules() method of the OSGi module system should throw a ModuleInitializationException when the OSGi module system determines that the constraint is violated.

3.2 ModuleDefinition class

Two new methods are added in the ModuleDefinition class to return the exported packages and member packages respectively. The export and member definitions contained in the OSGi metadata are in package granularity. In addition, a new PackageDefinition class is added to allow an OSGi bundle to expose the metadata for an exported package. This is required to meet the requirements (1) and (2).
  ModuleDefinition class:
      public abstract Set<PackageDefinition> getExportedPackageDefinitions();
      public abstract Set<PackageDefinition> getMemberPackageDefinitions();
      
  PackageDefinition class:
      public abstract class PackageDefinition {
          public String getName();
          public Version getVersion();
          public Set<String> getAttributeNames();
          public String getAttribute(String name); 
      }
  
The version and attributes in the PackageDefinition are optional and for information only and to aid diagnosis. A ModuleSystem importing a ModuleDefinition from other ModuleSystem is not required to understand the version and attributes of its exported PackageDefinitions.

3.3 ImportDependency class

The ImportDependency class is updated as follows.
  1. An import type is added to indicate if the ImportDependency is for module-level or package-level. "module" and "package" are two defined import types.
  2. The ImportDependency class can have attributes for a module system to include additional information about an import dependency.
This is required to meet the requirement (1).
  ImportDependency class:
         // static factory methods
       public static ImportDependency 
           newImportModuleDependency(String moduleName,
                                     VersionConstraint constraint,
                                     boolean reexport,
                                     boolean optional,
                                     Map<String, String> attributes);
       public static ImportDependency
           newImportPackageDependency(String packageName,
                                      VersionConstraint constraint,
                                      boolean optional,
                                      Map<String, String> attributes);
       public String getType();
       public String getAttribute(String name);
       public Set<String> getAttributeNames();
  

4. Mapping OSGi Bundles to ModuleDefinitions

This section specifies how an OSGi bundle maps to a ModuleDefinition to expose in the Java Module System so that other ModuleDefinitions can import them.

4.1 Bundle-SymbolicName

The bundle symbolic name maps to a module name (i.e. ModuleDefinition.getName()). The directives for the Bundle-SymbolicName header maps to the module attributes.

For example:

      Bundle-SymbolicName: com.acme.foo
  
The Java Module System and OSGi do not enforce any naming convention. It is encouraged to use the reverse domain name convention to name OSGi bundles and Java modules to avoid the name space conflict.

4.2 Bundle-Version

A bundle version maps to a module version5 (i.e. ModuleDefinition.getVersion()). Bundle-Version is an optional header and the default value is 0.0.0. The bundle version format is:
      major[.minor[.micro]][.qualifier]
  
The module version format is:
      major[.minor[.micro[.update]]][-qualifier]
  
If the bundle version contains a qualifier, the delimiter prior to the qualifier will need to be changed from a period ('.') to a dash ('-'). For example, the bundle version 3.1.4.pi maps to the module version 3.1.4-pi.

5Difference in OSGi and JSR 277 versioning scheme
The versioning scheme defined in the JSR 277 Early Draft is loosely based on the existing versioning schemes that are widely used in the Java platform today and for backward compatibility reason (see JSR 277 EDR chapter 5). Many existing products including the JDK use the version format with the micro, minor, micro, and update numbers. A version with no qualifier is higher than the version with the same version number but with a qualifier. This is more intuitive and has been the convention the JDK has been using. The Expert Group has discussed the difference with the OSGi versioning scheme and agreed with the JSR 277 versioning scheme defined.

Open Issue:
Need to investigate the version mapping due to the difference in the comparison of two versions - one with a qualifier and the other without a qualifier.

When two bundle versions have the same major, minor, and micro numbers, the bundle version that has a qualifier is lower than the bundle version that has no qualifier. e.g. 7.8.9.pi < 7.8.9

When two module versions have the same major, minor, micro, and update numbers, the module version that has a qualifier is higher than the module version that has no qualifier. e.g. 7.8.9 < 7.8.9-b04-alpha

One possible solution would be:

  OSGi version                      JSR 277 Version
  
  major.minor.micro            ->   major.minor.micro-0
  major.minor.micro.qualifier  ->   major.minor.micro-1-qualifier  
  

4.3 Import-Package

The Import-Package header maps to the import dependencies for a Java module (i.e. ModuleDefinition.getImportDependencies()).

The Import-Package header contains one or more import definitions, each of which describes a single package import for a bundle. Each import definition maps to an ImportDependency instance with the "package" type as follows:

Example,
      Import-Package: p;
          version="[1.23, 1.24]";
          resolution:=optional
  
maps to the import dependencies equivalent to:
      ImportDependency importP = 
          ImportDependency.newImportPackageDependency("p", 
                                                      VersionConstraint.valueOf("[1.23,1.24]",
                                                      true /* optional */);
  

The version-range mapping:

The OSGi version-range maps to a VersionConstraint as follows:

4.4 Export-Package

The Export-Package header maps to the exported package definitions for a Java module (i.e. ModuleDefinition.getExportedPackageDefinitions()).

The Export-Package header contains one or more export definitions, each of which describes a single package export for a bundle. Each export definition maps to a PackageDefinition instance as follows:

4.5 Require-Bundle

The required bundles maps to the import dependencies for a Java module (i.e. ModuleDefinition.getImportDependencies()). Each required bundle maps to an ImportDependency instance with the "module" type: Example,
      Require-Bundle: com.acme.facade;visibility:=reexport,
          com.acme.bar;visibility:=reexport;resolution:=optional
  
maps to the import dependencies equivalent to:
      ImportDependency facade = 
          ImportDependency.newImportModuleDependency("com.acme.facade", 
                                                     VersionConstraint.DEFAULT,
                                                     true /* reexport */,
                                                     false /* optional */);
                                                    
      ImportDependency bar = 
          ImportDependency.newImportModuleDependency("com.acme.bar", 
                                                     VersionConstraint.DEFAULT,
                                                     true /* reexport */,
                                                     true /* optional */);
  

4.6 Other Manifest Headers

The above sections cover the manifest headers that provide the metadata for the OSGi resolver (see Section 3.5 of the OSGi Service Platform Core Specification Release 4, Version 4.1). The other bundle manifest headers, including Bundle-Vendor, Bundle-Description and DynamicImport-Package, do not affect the module resolution. This specification does not need to define how to map them to ModuleDefinition. However, implementations are encouraged to include them as the module attributes (i.e. ModuleDefinition.getAttribute()) as additional information to aid diagnosis.

4.7 Fragment Bundles

Fragment bundles are not exposed as ModuleDefinitions in the Java Module System. Instead, they are exposed as part of the ModuleDefinition of its host bundle to which they are attached to.

4.8 Example

The manifest in the org.apache.xerces.parsers bundle shown in Example 1 of Section 2 is:
  org.apache.xerces.parsers:
     Bundle-SymbolicName: org.apache.xerces.parsers
     Bundle-Version: 2.9.10

     Export-Package: org.apache.xerces.parsers; version=2.6.6; uses="org.apache.commons.logging",
                     org.apache.xerces.jaxp; version=2.6.6,
                     org.apache.xerces.framework; version=2.6.6,
                     org.apache.xerces.readers; version=2.6.6,
                     org.apache.xerces.utils; version=2.6.6,
                     org.apache.commons.logging; version=1.0.4
     Import-Package: javax.xml.parsers; version=1.2.0,
                     org.w3c.dom; version=1.0.0,
                     org.xml.sax; version=2.0.1
     Require-Bundle: org.osgi.util.xml; version=1.0.0; visibility:=reexport; resolution:=optional
  

Below shows the ModuleDefinition for this OSGi bundle when exposed in the Java Module System. For clarity, we only show one Export-Package entry and one Import-Package entry.

Method of ModuleDefinition Returned Value Bundle Manifest Header
getName() "org.apache.xerces.parsers" Bundle-SymbolicName
getVersion() Version.valueOf("2.9.10") Bundle-Version
getImportDependencies()
ImportDependency.newImportModuleDependency("org.osgi.util.xml", 
                                           VersionConstraint.valueOf("1.0.0+"),
                                           true /* reexport */,
                                           true /* optional */);
Require-Bundle
ImportDependency.newImportPackageDependency("javax.xml.parsers", 
                                            VersionConstraint.valueOf("1.2.0+"),
                                            false /* optional */);          
Import-Package
getExportedPackageDefinition() PackageDefinition with:
 name="org.apache.xerces.parsers",
 version="2.6.6"
 attributes=(("uses", "org.apache.commons.logging")}
Export-Package

5. Enabling the OSGi Module System in the Framework

To enable the OSGi module system in the framework, an OSGi Repository implementation should be plugged into the runtime. The OSGi repository is responsible for discovering OSGi bundles and exposing them as ModuleDefinitions so that OSGi bundles are available for other module systems to use. A ModuleSystem implementation finds OSGi bundles via the repository delegation model and therefore the OSGi repository has to be configured as an ancestor of the repository where the Java module depending on OSGi bundles resides. Otherwise, it will fail to find the importing OSGi bundles.

One or more repositories can be created for the OSGi module system and interoperate with other module systems via the repository delegation model.

The following picture depicts the repository tree set up to run the Wombat application described in Example 1 of Section 2. The OSGi repository is configured as the parent of the application repository. This particular OSGi repository implementation includes the Apache Felix OSGi runtime for loading and resolving OSGi bundles.

 
               ------------------------
               | Bootstrap Repository |
               ------------------------
                          |
                          |
                 --------------------
                 | Global Repository |
                 --------------------
                          |   
                          |                       xxxxxxxxxxxxxxxx
                 -------------------              x Apache Felix x
                 | OSGi Repository |   <====>     x     OSGi     x ---> org.apache.xerces.parsers version 2.10
                 -------------------              x   Runtime    x      org.apache.derby version 10.0.2
                          |                       xxxxxxxxxxxxxxxx      org.apache.derby version 9.1
                          |
                  ----------------- 
                  |  Application  | 
                  |  Repository   |
                  -----------------
                  com.wombat.app-1.0.jam
  
For example, the following command will launch the Wombat application in the "/wombat-application" directory.
                
  > java -repository /wombat-application -module com.wombat.app         
  
The com.wombat.app module is located in the application repository which is a repository for the JAM module system. The JAM module system first loads the com.wombat.app module. To initialize this JAM module, the JAM module system looks at its import dependencies and performs a search of two imported OSGi bundles through the repository delegation model from the OSGi repository. The JAM module system then requests the OSGi module system associated with the OSGi repository to get the Module instances for the imports (see Section 3.1). Once the imported OSGi bundles are loaded and started, the initialization process for the com.wombat.app module continues.

6. Delegation of Class and Resource Loading

The class loader of a Module instance (i.e. returned by the Module.getClassLoader() method) must be capable to load all classes and resources in the module.

As described in Section 5 above, when the com.wombat.app JAM module is initialized, two other Module instances for its imports representing the resolved OSGi bundles are created in the system. The class loader for the Module instance for the org.apache.xerces.parsers bundle must be capable to load all classes and resources in it. Similarly for the org.apache.derby bundle.

7. Security

TBD.

8. Implementation Notes

The following are the notes for the implementation of the OSGi repository and the implementation of the Java Module System.
  1. The repository delegation hierarchy is a tree and thus cycles involving multiple module systems are inherently unsupported.
  2. The repository delegation model is designed to offer isolation between ModuleDefinitions in different repositories. Although a module system could have access to multiple repositories, the module system should adhere to the repository delegation module. Otherwise, it would break the isolation model the repository provides.
  3. Java SE 7 is expected to have parallel class loading support. All module systems plugged in the framework are required have parallel class loading enabled in order to avoid potential deadlocks.
  4. Split packages without shadowing are explicitly permitted in OSGi whereas the JAM module system does not allow split packages. So importing OSGi bundles with split packages in a JAM module will result in module initialization failure.
  5. When a resolution involves multiple module systems, a module system implementation should take the possible potential issues (such as hanging) into account in their design to prevent a foreign module system from bringing down the module system or the entire JVM. A module system could implement time out policy to prevent from hanging the module system.

8. References

A. Appendix

The following illustrates how the JAM module system implementation uses the JSR 277 API to search and import OSGi bundles This example does not cover the exact resolution algorithm and the implementation of the module system runtime.
Example 1 of Section 1 has a com.wombat.app JAM module importing two OSGi bundles.
com.wombat.app ----> org.apache.xerces.parsers
               |
               |---> org.apache.derby
// the Java runtime will first find the module com.wombat.app
ModuleDefinition wombatModDef = repository.find("com.wombat.app", "1.0+");
// This call blocks until com.wombat.app is fully initialized
Module wombat = wombatModDef.getModuleInstance();
JAM Module System Runtime
// find imports for wombat
List<ModuleDefinition> imports = ...;
Map<ModuleSystem, List<ModuleDefinition>> foreignImportMap = new HashMap....;
for (ModuleDefinition md : imports) {
    if (md is from a foreign module system (ms)) {
        // the case to add a new entry in foreignImportMap 
        // is not shown in this pseudo-code.
        // 
        // org.apache.xerces.parsers and org.apache.derby will be added in this map       
        foreignImportMap.get(ms).add(md); 
    } else {
        // JAM resolution algorithm
        ...
    }
}

// Gets imports from foreign module systems
// Should do this in a separate thread since this is a synchronous call
for (ModuleSystem ms : foreignImportMap.keySet()) {
   // The getModules() method will allow a module system to know if 
   // a set of ModuleDefinition are resolved in the same resolution. 
   List<Module> imports = ms.getModules(wombatModDef, foreignImportMap.get(ms));
   // interconnect the imports with wombat
   ...
}

// continue the module initialization process such as shadow validatation
// and execute initializers
...

// Module initialization completed
if (succeeded) {
   com.wombat.app is now fully initialized
} else {
   // signal the getModule method to throw ModuleInitializationException
   ...
}