JEP 126: Lambda Expressions & Virtual Extension Methods

AuthorJoseph D. Darcy
OrganizationOracle
OwnerBrian Goetz
Created2011/11/1
Updated2013/2/21
TypeFeature
StateFunded
Component--/--
ScopeSE
JSR269 MR, 335
Discussionlambda dash dev at openjdk dot java dot net
Start2011/Q4
Blocks107, 109
EffortXL
DurationXL
Reviewed-byBrian Goetz
Endorsed-byBrian Goetz
Funded-byOracle
Release8
TargetM7

Summary

Add lambda expressions (closures) and supporting features, including method references, enhanced type inference, and virtual extension methods, to the Java programming language and platform.

Goals

The primary features of lambda expressions and virtual extension methods, along with their set of secondary supporting features, further several platform goals:

Besides adding a now-common feature to the Java programming language, lambda expressions open up possibilities for improved multicore support by enabling internal iteration idioms.

The supporting language features around lambda include virtual extension methods, which will allow interfaces to be evolved in a source and binary compatible fashion.

In addition to language changes, coordinated libraries and JVM changes will occur as well.

Note that the active and ongoing Project Lambda OpenJDK project pre-dates the JEP process, as does the corresponding JSR, JSR 335, which is targeted for Java SE 8 (JSR 336).

Non-Goals

The language features of function types and general control abstraction are not goals of adding lambda expressions to Java. However, the intention is to not preclude the addition of such features in the future.

Motivation

Many other commonly used object-oriented programming languages, including those hosted on the JVM (for example Groovy, Ruby, and Scala) and hosted on other virtual machines (C# on the CLR), include support for closures. Therefore, Java programmers are increasingly familiar with the language feature and the programming models it enables.

Of particular interest is enabling the idiom of internal iteration. Arrays and collections currently support external iteration, where the control logic of an iteration lives outside of the data structure being traversed. For example, a for-each loop over an array or a collection is an example of external iteration. The semantics of the for loop in Java mandates strict serial iteration, which means that the only means the programmer has of iterating over a standard collection will not allow the use of all the available cores.

With internal iteration, the data structure is passed a piece of code to execute, as a lambda expression, and the data structure is responsible for partitioning the computation and reporting the results. Since the data structure is familiar with its own internal details it can potentially choose a better scheduling of the computation by tuning options such as

A prototypical example of the internal iteration style is a sequence of filter-map-reduce operations such as:

int maxFooWeight =
    collection.filter( /* isFoo Predicate as a lambda */)
              .map(    /* Map a Foo to its weight with a lambda */)
              .max();  /* Reduction step */

The lambda expressions are expressions with concise syntax to represent the desired operation. This style is in lieu of one or more explicit for loops which would unnecessary constrain the iteration order over the collection. In addition, a well-designed algorithm can not only execute these sets of operations in parallel, but can also aggregate the three operations into a single parallel pass.

Project Lambda also includes virtual extension methods, which will address the long-standing limitation of not being able to add methods to widely-used interfaces because of source compatibility concerns.

By adding extension methods to the existing collection interfaces, such as java.util.Collection and java.util.List, existing implementations of those types can participate in the new programming idiom. Implementations of those types in the JDK (and elsewhere) can override the default implementations of the extension methods from the superinterfaces to provide higher-performance or otherwise specialized implementations.

Description

The in-progress OpenJDK project page for Project Lambda and corresponding JSR have the most up-to-date information about the details of the effort.

The platform components impacted by the overall Project Lambda effort include:

Alternatives

Project Lambda started with a straw-man proposal and has gone through several “State of the Lambda” iterations which have evolved both the syntax and feature set of the project.

The work on lambda expression portion of Project Lambda is informed by numerous previous efforts, including but not limited to:

The feature set of Project Lambda is judged to better address the needs of evolving the Java platform than the earlier proposals.

The design of virtual extension methods is likewise informed by a large body of prior work in the programming language community including:

Compared to other language environments, the Java language has always had much more predictable semantics, built-in primitive types are of a known size, exceptions are thrown at precise locations, the order of execution of expressions is precisely defined, and so on. Loosening the semantics of the built-in for and while loops to allow auto-parallelization attempts was judged to be neither desirable nor sufficient to meet the goals of supporting multi-core.

Testing

In addition to unit/regression tests, as a new language feature JCK tests will also be developed.

The utility of the feature will also be validated through usage in the JDK platform libraries.

Risks and Assumptions

As a large effort touching many aspects of the platform, there is the possibility for unforeseen complications and interactions to arise.

By starting early and planning for several iterations of, say, the supporting libraries work, the impact of those complications should be mitigated.

Dependences

The currently preferred implementation approach for lambda expressions relies on invokedynamic and method handles introduced by JSR 292. Therefore, acceptable performance of lambda depends on acceptable performance of method handles, amongst other factors.

There is expected to be feedback between the libraries work (JEPs 107 and 109) and the language features.

Impact