JEP 300: Augment Use-Site Variance with Declaration-Site Defaults

OwnerDan Smith
Created2014/05/19 23:16
Updated2016/12/06 23:43
TypeFeature
StatusCandidate
Componenttools / javac
ScopeSE
Discussionplatform dash jep dash discuss at openjdk dot java dot net
EffortM
DurationL
Priority3
Reviewed byBrian Goetz
Endorsed byBrian Goetz
Issue8043488
Relates toJEP 218: Generics over Primitive Types

Summary

Enhance the Java Programming Language so that a generic class or interface declaration can indicate whether each type parameter is, by default, invariant, covariant, or contravariant, thus allowing more intuitive subtyping relationships between parameterizations of the class or interface. This supplements the existing mechanism for variance, wildcards, which are written at type use sites.

Non-Goals

Any changes to the standard libraries in order to take advantage of the language enhancement would be performed under a separate JEP.

A language might be designed to infer the variance of type parameters, rather than requiring explicit syntax [1]. Such analysis is outside the scope of this JEP.

The feature is not a replacement for wildcards. It provides a less-verbose way to get the behavior of wildcards in certain cases, but is not intended to replace all common use cases.

Changes to the behavior of existing code—such as improvements to or simplifications of the behavior of existing wildcards—are outside the scope of this JEP.

Motivation

Certain class type parameters are used by the class declaration in such a way that they are inherently variant: for example, the type Iterator<Number> has no more functionality than the type Iterator<? extends Number>, both of which can be used to read Numbers; similarly, Predicate<String> has no more functionality than Predicate<? super String>, both of which can be used to test a String. Since invariant uses of these type arguments are less flexible than their wildcard equivalents, while providing no extra power, a reasonable practice is to always use a wildcard when mentioning the type. This strategy maximizes flexibility without compromising expressiveness.

But there are problems:

The solution is to shift this burden from programmers to compilers, allowing a class or interface type parameter, when appropriate, to be declared covariant or contravariant.

interface Predicate<contravariant T> {
  boolean test(T arg);
}

Then, the compiler can automatically treat every use of the type—e.g., Predicate<String>—as if it had wildcards—Predicate<? super String>. Clients get the benefit of more flexible types without having to do anything.

A 2011 PLDI paper [1] performed an analysis of some open-source Java libraries that make use of generics, including the Java core libraries, Guava, and Apache Commons. Findings:

Presumably a large number of clients of these libraries would benefit from both a reduced need for wildcards and increased flexibility in their types (the paper does not examine library clients).

Description

The new language feature encompasses the following:

(A note on terminology: for simplicity, throughout this JEP, the term class also refers to interfaces.)

Details of these four components:

Type parameter syntax. The strawman syntax is to use covariant T in the declaration of T in order to indicate that it is covariant; contravariant T indicates that T is contravariant. Thus, the Function interface might be declared as:

interface Function<contravariant T, covariant R> {
    R apply(T arg);
}

We may also consider some other syntax. Other languages use symbols (interface Function<-T,+R> { ... }) or keywords (interface Function<in T, out R> { ... }).

Well-formedness checking. A variant type parameter should only be used in certain contexts that support its variance. For example, if the type of a method's parameter is a type variable T, then T should be invariant or contravariant—not covariant. For more complex types, the analysis recurs: if the type of a method's parameter is Predicate<T>, then T should be invariant or covariant. The rules governing appropriate usage of type variables must be developed and implemented for every context in which a type variable can appear.

At the use site, a mismatch between the variance of a type parameter and the variance of a wildcard might also be detected—Iterator<? super String>, for example, is nonsensical. (We must take some care here, however, because such types—though useless—may exist in the wild, raising migration compatibility concerns.)

It is possible to define type-checking behavior without strictly enforcing these well-formedness rules, and there is some flexibility in their design. But being more permissive about well-formedness generally leads to unintuitive behavior at the use site.

Type checking. Two strategies may be used to specify and implement widening of reference types, consistent with the declared variance of type parameters.

There are advantages and disadvantages to each approach. Enhanced subtyping is more direct and more closely aligned with users' intuition, while implicit wildcards is more flexible, placing fewer constraints on the well-formedness rules. In principle, they are likely equivalent, and if desired it might be possible for the specification and implementation to use different approaches.

Note that many other aspects of type checking are orthogonal to this feature. For example: raw types continue to behave as currently specified; "diamond" class instance creation expressions use the same inference algorithm as before; generic array creation is still prohibited.

Class file changes. To allow separate compilation, variance of type parameters must be encoded in the class file. There are a variety of possible approaches, including modifying the Signature attribute, introducing a new attribute, or piggybacking on an existing mechanism (like RuntimeVisibleTypeAnnotations).

Reflection APIs should probably understand this encoding and directly expose it to clients (e.g., a TypeVariable.getVariance method).

Development plan

In a first phase, the following artifacts will be produced:

After the first phase, we will reevaluate whether the proposed feature is in accordance with initial expectations, and if so, proceed to a second phase, formally changing the language via the Java Community Process. This will involve i) making adjustments to the behavior of the first milestone as proposed by the consensus of an Expert Group, and ii) producing compliance tests (for the JCK-compiler suite).

Alternatives

The existing alternative in the Java language is use-site variance (wildcards). This is a useful feature, but has its limitations, as noted in the "Motivation" section.

There are good reasons for both use-site and declaration-site variance in a language; they are complementary [1] [2]. Use-site gives users flexibility to tailor a type (like List) to their particular needs, while declaration-site relieves users of a clerical burden when there is only one reasonable variant usage of a type (like Iterator).

There are various ways in which the scope of the feature could be limited, although the problem is so general that these limited approaches seem inadequate:

Testing

As a proposed language change, the enhancement should be accompanied by new JCK-compiler tests.

Behavior of existing compiler tests will be unchanged.

Risks and Assumptions

Dependencies

JDK-8154901 identifies various problems with the design of generics in the Java Language Specification and javac. These problems should be addressed, or at least carefully considered, before making further language changes in this area.

JEP 218 proposes other enhancements to generics, both in the language and the JVM. Design decisions should be made with both JEPs in mind.

It would be disappointing for users if this feature were delivered without accompanying changes to existing classes in the Java Class Library. These library changes will need to be handled in a subsequent JEP.

Minor changes may be necessary for java.lang.reflect, javax.lang.model, javadoc, javap, pack200, and ASM. This is mostly to correctly process/expose the variance of type parameters, as encoded in class files.

Language documentation and tutorials should explain and encourage use of the new feature.

References

[[1]]: #1 [1] John Altidor, Shan Shan Huang, & Yannis Smaragdakis. Taming the Wildcards: Combining Definition- and Use-Site Variance. http://jgaltidor.github.io/variance_pldi11.pdf.

[[2]]: #2 [2] Ross Tate. Mixed-Site Variance. http://www.cs.cornell.edu/~ross/publications/mixedsite/mixedsite-tate-fool13.pdf.