Procedure for Java SE 6 Technology Certification

This chapter describes the Java SE 6 technology compatibility testing procedure and compatibility requirements for JCK 6b.

This chapter contains the following sections:

• 2.1 Certification Overview

• 2.2 Compatibility Requirements

• 2.3 JCK Test Appeals Process

• 2.4 Reference Compiler for Java SE 6 Technology

• 2.5 Reference Runtime for Java SE 6 Technology

• 2.6 Specifications for Java SE 6 Technology

• 2.7 Libraries for Java SE 6 Technology

2.1 Certification Overview

The certification process for the Java SE 6 technology consists of the following activities:

• Install JCK 6b and execute it in accordance with the instructions in this User's Guide.

• Ensure that you meet the requirements outlined in 2.2 Compatibility Requirements.

• Certify to Java Licensee Engineering that you have finished testing and that you meet all the compatibility requirements.

2.2 Compatibility Requirements

The compatibility requirements for the Java SE 6 technology consist of meeting the requirements set forth by the rules and associated definitions contained in this section.

2.2.1 Definitions

These definitions are for use only with these compatibility requirements and are not intended for any other purpose.

Table 2-1 Definitions

2.2.2 Rules for Java SE 6 Technology Products

The following rules apply for each version of an operating system, software component, and hardware platform Documented as supporting the Product:

1. The Product must be able to satisfy all applicable compatibility requirements, including passing all Conformance Tests, in every Product Configuration and in every combination of Product Configurations, except only as specifically exempted by these Rules.

   For example, if a Product provides distinct Operating Modes to optimize performance, then that Product must satisfy all applicable compatibility requirements for a Product in each Product Configuration, and combination of Product Configurations, of those Operating Modes.

   1. If an Operating Mode controls a Resource necessary for the basic execution of the Test Suite, testing may always use a Product Configuration of that Operating Mode providing that Resource, even if other Product Configurations do not provide that Resource. Notwithstanding such exceptions, each Product must have at least one set of Product Configurations of such Operating Modes that is able to pass all the Conformance Tests.

      For example, a Product with an Operating Mode that controls a security policy (i.e., Security Resource) which has one or more Product Configurations that cause Conformance Tests to fail may be tested using a Product Configuration that allows all Conformance Tests to pass.

   2. A Product Configuration of an Operating Mode that causes the Product to report only version, usage, or diagnostic information is exempted from these compatibility rules.

   3. A Product may contain an Operating Mode that selects the Edition with which it is compatible. The Product must meet the compatibility requirements for the corresponding Edition for all Product Configurations of this Operating Mode. This Operating Mode must affect no smaller unit of execution than an entire application.

   4. A Runtime Product may have Operating Modes that enable the optional Java Virtual Machine Tool Interface (JVM TI) functionality of the Java Platform Debugger Architecture (JPDA) technology. If JVM TI functionality is provided, the Product must pass the conformance tests that test JVM TI. Testing of JVM TI functionality may always use a Product Configuration that enables the functionality.

   5. A Runtime Product may have Operating Modes that enable the optional Java Debug Wire Protocol (JDWP) functionality of the JPDA technology. If JDWP functionality is provided, the Product must pass the conformance tests that test JDWP. Testing of JDWP functionality may always use a Product Configuration that enables the functionality.

   6. A Runtime Product may have Operating Modes that enable the required java.lang.instrument functionality. Testing of such functionality may always use a Product Configuration that enables the functionality.

2. Some Conformance Tests may have properties that may be changed. Properties that can be changed are identified in the configuration interview. Apart from changing such properties and other allowed modifications described in this User's Guide (if any), no source or binary code for a Conformance Test may be altered in any way without prior written permission. Any such allowed alterations to the Conformance Tests would be posted to the Java Licensee Engineering web site and apply to all licensees.

3. The testing tools supplied as part of the Test Suite or as updated by the Maintenance Lead must be used to certify compliance.

4. The Exclude List associated with the Test Suite cannot be modified.

5. The Maintenance Lead can define exceptions to these Rules. Such exceptions would be made available to and apply to all licensees.

6. All hardware and software component additions, deletions, and modifications to a Documented supporting hardware/software platform, that are not part of the Product but required for the Product to satisfy the compatibility requirements, must be Documented and available to users of the Product.

   For example, if a patch to a particular version of a supporting operating system is required for the Product to pass the Conformance Tests, that patch must be Documented and available to users of the Product.

7. A Runtime Product must contain the full set of public and protected classes and interfaces for all the Libraries. Those classes and interfaces must contain exactly the set of public and protected methods, constructors, and fields defined by the Specifications for those Libraries. No subsetting, supersetting, or modifications of the public and protected API of the Libraries are allowed except only as specifically exempted by these Rules.

   1. If a Product includes Technologies in addition to the Technology Under Test, then it must contain the full set of combined public and protected classes and interfaces. The API of the Product must contain the union of the included Technologies. No further modifications to the APIs of the included Technologies are allowed.

   2. A Product may provide a newer version of an Endorsed Standard. Upon request, the Maintenance Lead will make available alternate Conformance Tests as necessary to conform with such newer version of an Endorsed Standard. Such alternate tests is to be made available to and apply to all licensees. If a Product provides a newer version of an Endorsed Standard, the version of the Endorsed Standard supported by the Product must be Documented.

   3. The Maintenance Lead may authorize the use of newer Versions of a Technology included in the Technology Under Test. A Product that provides a newer Version of a Technology must meet the Compatibility Requirements for that newer Version, and must Document that it supports the newer Version.

      For example, the Java SE 6 technology Maintenance Lead authorized use of a newer version of a Java technology, Java API for XML Processing (JAXP) that is also available independently of Java SE 6 technology and is evolving quickly to meet market demands such as supporting the latest XML language features.

8. Except for tests specifically required by this TCK to be rebuilt (if any), the binary and sources of Conformance Tests supplied as part of the Test Suite or as updated by the Maintenance Lead must be used to certify compliance.

9. The functional programmatic behavior of any binary class or interface must be that defined by the Specifications.

10. If a Runtime Product supplies neither 1) a Documented interface that allows a program running in the Runtime to directly interact with Native Code nor 2) a Documented interface that allows Native Code to directly interact with a program running in the Runtime, then the Product is not required to pass the Conformance Tests that test the JNI implementation. If, however, the Product supplies either (1) or (2) above, the Product must pass the Conformance Tests that test the JNI implementation.

2.2.3 Rules for Compiler Products

These Rules, in addition to the Rules for 2.2.2 Rules for Java SE 6 Technology Products Products, apply to all products that include a Compiler.

1. For each Product Configuration that causes the Compiler to generate Executable Output for some Version of the Technology, the Compiler, when run in that Product Configuration, must be able to satisfy all applicable compatibility requirements, including passing all Conformance Tests for a Compiler associated with that Version.

   For example, if a Compiler has a Product Configuration that causes it to generate Executable Output for Version 1.2 of the Technology, then that Compiler, when run in that Product Configuration, must satisfy all applicable compatibility requirements and pass the Conformance Tests for a Version 1.2 Compiler. If that Compiler also has a Product Configuration that causes it to generate Executable Output for Version 1.1 of the Technology, then it must also, when run in this second Product Configuration, satisfy all applicable compatibility requirements and pass the Conformance Tests for a Version 1.1 Compiler.

2. For each Product Configuration that causes the Compiler to generate Executable Output for some Version of the Technology, the Executable Output of the Compiler, when the Compiler is run in that Product Configuration, must execute properly on a Reference Runtime associated with that Version.

   For example, if a Compiler has a Product Configuration that causes it to generate Executable Output for Version 1.2 of the Technology, then that Executable Output must execute properly on the Reference Runtime associated with Version 1.2 of the Technology. If that Compiler also has a Product Configuration that causes it to generate Executable Output for Version 1.1 of the Technology, then that Executable Output must execute properly on the Reference Runtime associated with Version 1.1 of the Technology.

3. The Compiler must not produce Executable Output from source code that does not conform to the Specifications.

   For example, a Compiler must not produce Executable Output from source code that contains any language feature, such as a keyword, published by any party, that is not defined or enabled by the Specifications.

4. The presence of a comment or directive in source code, when compiled into Executable Output by the Compiler, must not cause the functional programmatic behavior of a class or interface or application to vary, when executed by the Runtime, from the functional programmatic behavior of the class or interface or application in the absence of that comment or directive.

5. The Executable Output of the Compiler must be in class file format defined by the Specifications.

6. Annotation processors activated in each Product Configuration must be documented.

7. For each Technology that is included in the Product, there must be a Product Configuration that causes the Compiler to process annotations as required by that Technology.

8. For each Product Configuration that causes the Compiler to create Executable Output as well as to process annotations, and for each class or interface for which both the Compiler and the Reference Compiler produce Executable Output, the Executable Output generated by the Compiler must be functionally equivalent to the Executable Output generated by the Reference Compiler.

2.2.4 Rules for Products That Include a Development Kit

The following Rules, in addition to the 2.2.2 Rules for Java SE 6 Technology Products and 2.2.3 Rules for Compiler Products, apply for each version of an operating system, software component, and hardware platform Documented as supporting software development.

2.2.4.1 Rules for Products That Include a Schema Compiler and Generator

These Rules, in addition to the 2.2.2 Rules for Java SE 6 Technology Products and 2.2.3 Rules for Compiler Products, apply to all products that include a Development Kit.

1. In cases where Schema Compiler Output contains Executable Output, that Schema Compiler Output must execute properly on a Reference Runtime.

2. The Schema Compiler must not produce Schema Compiler Output from schema that do not conform to the W3C XML Schema recommendation [XSD Part 1][XSD Part 2].

   1. The Schema Compiler in non-default Product Configurations may produce Schema Compiler Output when an XML Schema contains constructs for which a binding has not been defined by the Specification.

   2. The Schema Compiler in non-default Product Configurations may produce Schema Compiler Output when a custom binding declaration is encountered.

3. The Schema Compiler Output of the Schema Compiler must be either in class format defined by the Java Virtual Machine Specifications or in source file format defined by Java Language Specification (JLS).

4. For each Product Configuration that causes the Schema Generator to produce Schema Generator Output, the Output of Schema Generator must fully meet W3C requirements for the XML Schema language.

5. The Schema Generator must not produce Schema Generator Output when a program element violates mapping constraints defined by the Specifications.

6. The Schema Generator may produce Schema Generator Output when a program element contains constructs for which a mapping has not been defined by the Specification.

7. A schema produced by the Schema Generator from source code containing a comment or a directive, or from a class file containing non-standard attributes, must be equivalent to a schema produced by the Schema Generator from the same source code with the comment or directive removed, or from the class file with the attributes removed. Two schemas are equivalent if and only if they validate the same set of XML documents (any XML document is either valid against both schemas or invalid against both schemas).

2.2.4.2 Rules for Products That Include Java-to-WSDL and WSDL-to-Java Tools

These Rules, in addition to the 2.2.2 Rules for Java SE 6 Technology Products and 2.2.3 Rules for Compiler Products, apply to all products that include a Development Kit.

1. Source Code in WSDL-to-Java Output when compiled by a Reference Compiler must execute properly when run on a Reference Runtime.

2. Source Code in WSDL-to-Java Output must be in source file format defined by the Java Language specification (JLS).

3. Java-to-WSDL Output must fully meet W3C requirements for the Web Services Description language (WSDL) 1.1.

4. A Java-to-WSDL Tool must not produce Java-to-WSDL Output from source code that does not conform to the Java Language Specification (JLS).

2.3 JCK Test Appeals Process

Sun has a well-established process for managing challenges to its Java SE 6 technology Test Suite and plans to continue using a similar process in the future. Sun, as the Java SE 6 technology Maintenance Lead, will authorize representatives from the Java Licensee Engineering group to be the point of contact for all test challenges. Typically this is to be the engineer assigned to a company as part of its Java SE 6 technology JCK support.

If a test is determined to be invalid in function or if its basis in the specification is suspect, the test may be challenged by any licensee of the Java SE 6 technology JCK. Each test validity issue must be covered by a separate test challenge. Test validity or invalidity is to be determined based on its technical correctness such as:

• Test has bugs (i.e., program logic errors)

• Specification item covered by the test is ambiguous

• Test does not match the specification

• Test assumes unreasonable hardware and/or software requirements

• Test is biased to a particular implementation

Challenges based upon issues unrelated to technical correctness as defined by the specification will normally be rejected.

Test challenges must be made in writing to Java Licensee Engineering and include all relevant information as described in the 2.3.1 Test Challenge Form. The process used to determine the validity or invalidity of a test (or related group of tests) is described in JCK Test Appeals Steps.

All tests found to be invalid will either be placed on the Exclude List for that version of the Java SE 6 technology JCK or have an alternate test made available.

• Tests that are placed on the Exclude List is to be placed on the Exclude List within one business day after the determination of test validity. The new Exclude List is to be made available to all Java SE 6 technology JCK licensees on the Java SE 6 technology JCK web site.

• Sun, as Maintenance Lead has the option of creating alternative tests to address any challenge. Alternative tests (and criteria for their use) is to be made available on the Java SE 6 technology JCK web site.

Note - Passing an alternative test is deemed equivalent to passing the original test.

JCK Test Appeals Steps

1. Java SE 6 technology JCK licensee writes a test challenge to Java Licensee Engineering contesting the validity of one or a related set of Java SE 6 technology tests.

   A detailed justification for why each test should be invalidated must be included with the challenge as described by the 2.3.1 Test Challenge Form.

2. Java Licensee Engineering evaluates the challenge.

   If the appeal is incomplete or unclear, it is returned to the submitting licensee for correction. If all is in order, Java Licensee Engineering will check with the responsible test developers to review the purpose and validity of the test before writing a response. Java Licensee Engineering will attempt to complete the response within 5 business days. If the challenge is similar to a previously rejected test challenge (i.e., same test and justification), Java Licensee Engineering will send the previous response to the licensee.

3. The challenge and any supporting materials from test developers is sent to the specification engineers for evaluation.

   A decision of test validity or invalidity is normally made within 15 working days of receipt of the challenge. All decisions is to be documented with an explanation of why test validity was maintained or rejected.

4. The licensee is informed of the decision and proceeds accordingly.

   If the test challenge is approved and one or more tests are invalidated, Sun places the tests on the Exclude List for that version of the Java SE 6 technology JCK (effectively removing the test(s) from the Test Suite). All tests placed on the Exclude List will have a bug report written to document the decision and made available to all licensees through the bug reporting database. If the test is valid but difficult to pass due to hardware or operating system limitations, Sun may choose to provide an alternate test to use in place of the original test (all alternate tests are made available to the licensee community).

5. If the test challenge is rejected, the licensee may choose to escalate the decision to the Executive Committee (EC), however, it is expected that the licensee would continue to work with Sun to resolve the issue and only involve the EC as a last resort.

2.3.1 Test Challenge Form

Provide the following information to Java Licensee Engineering.

Test Challenger Name and Company:
Specification Name(s) and Version(s):
Test Suite Name and Version:
Exclude List Version:
Test Name:
Complaint (argument for why test is invalid):

2.3.2 Test Challenge Response Form

The following information is provided in response to a test challenge.

Test Defender Name and Company:
Test Defender Role in Defense (e.g., test developer, Maintenance Lead, etc.):
Specification Name(s) and Version(s):
Test Suite Name and Version:
Test Name:
Defense (argument for why test is valid):
[Multiple challenges and corresponding responses may be listed here.]
Implications of test invalidity (e.g., other affected tests and test
framework code, creation or exposure of ambiguities in spec (due
to unspecified requirements), invalidation of the reference
implementation, creation of serious holes in test suite):
Alternatives (e.g., are alternate test(s) appropriate?):

2.4 Reference Compiler for Java SE 6 Technology

The designated Reference Compiler for compatibility testing of Java SE 6 technology implementations is the Sun javac compiler in JDK release 6 executed using a Reference Runtime.

2.5 Reference Runtime for Java SE 6 Technology

Designated Reference Runtimes for compatibility testing of Java SE 6 technology implementations are the Java SE 6 technology Java Runtime Environment (JRE^TM) for Solaris Operating System on SPARC^® technology, and for Win32 on Windows 2000 and Windows XP.

2.6 Specifications for Java SE 6 Technology

The Specifications for Java SE 6 technology are found on the Java Licensee Engineering web site.

2.7 Libraries for Java SE 6 Technology

Table 2-2 provides a list of packages that constitute the required class libraries for the Java SE 6 technology.

Table 2-2 Required Class Libraries for Java SE 6 Technology

¹Applicable for JAX-WS 2.1.

2.7.1 Endorsed Standards for Java SE 6 Technology

The endorsed standards for the Java SE 6 technology constitute all classes and interfaces that are defined in the packages listed in the following location:

http://java.sun.com/javase/6/docs/technotes/guides/standards/index.html

Classes and interfaces defined in the subpackages of listed packages are not endorsed standards unless those subpackages are themselves listed.

Sun, as specification maintenance lead, may add additional endorsed standards to this list, but will not remove any items. Modified versions of the Endorsed Standards list will be posted to http://java.sun.com/javase/6/docs/technotes/guides/standards/index.html and apply to all licensees.