JEP 126: Lambda Expressions & Virtual Extension Methods
|Author||Joseph D. Darcy|
|Status||Closed / Delivered|
|Component||tools / javac|
|JSRs||269 MR, 335|
|Discussion||lambda dash dev at openjdk dot java dot net|
|Reviewed by||Brian Goetz|
|Endorsed by||Brian Goetz|
|Blocks||JEP 101: Generalized Target-Type Inference|
|JEP 107: Bulk Data Operations for Collections|
|JEP 109: Enhance Core Libraries with Lambda|
|JEP 155: Concurrency Updates|
Add lambda expressions (closures) and supporting features, including method references, enhanced type inference, and virtual extension methods, to the Java programming language and platform.
The primary features of lambda expressions and virtual extension methods, along with their set of secondary supporting features, further several platform goals:
- Simplifying the creation and consumption of more abstract, higher-performance libraries
- Supporting smoother library evolution with migration compatibility
Besides adding a now-common feature to the Java programming language, lambda expressions open up possibilities for improved multicore support by enabling internal iteration idioms.
The supporting language features around lambda include virtual extension methods, which will allow interfaces to be evolved in a source and binary compatible fashion.
In addition to language changes, coordinated libraries and JVM changes will occur as well.
The language features of function types and general control abstraction are not goals of adding lambda expressions to Java. However, the intention is to not preclude the addition of such features in the future.
Many other commonly used object-oriented programming languages, including those hosted on the JVM (for example Groovy, Ruby, and Scala) and hosted on other virtual machines (C# on the CLR), include support for closures. Therefore, Java programmers are increasingly familiar with the language feature and the programming models it enables.
Of particular interest is enabling the idiom of internal iteration.
Arrays and collections currently support external iteration, where
the control logic of an iteration lives outside of the data structure
being traversed. For example, a
for-each loop over an array or a
collection is an example of external iteration. The semantics of the
for loop in Java mandates strict serial iteration, which means that
the only means the programmer has of iterating over a standard
collection will not allow the use of all the available cores.
With internal iteration, the data structure is passed a piece of code to execute, as a lambda expression, and the data structure is responsible for partitioning the computation and reporting the results. Since the data structure is familiar with its own internal details it can potentially choose a better scheduling of the computation by tuning options such as
- Alternate execution order
- Concurrent execution using thread pools
- Parallel execution using partitioning and work stealing. The fork/join framework, added in Java SE 7, is one such candidate parallel execution framework and offers performance-robust partitioning of work over a wide range of core counts.
A prototypical example of the internal iteration style is a sequence of filter-map-reduce operations such as:
int maxFooWeight = collection.filter( /* isFoo Predicate as a lambda */) .map( /* Map a Foo to its weight with a lambda */) .max(); /* Reduction step */
The lambda expressions are expressions with concise syntax to
represent the desired operation. This style is in lieu of one or more
for loops which would unnecessary constrain the iteration
order over the collection. In addition, a well-designed algorithm can
not only execute these sets of operations in parallel, but can also
aggregate the three operations into a single parallel pass.
Project Lambda also includes virtual extension methods, which will address the long-standing limitation of not being able to add methods to widely-used interfaces because of source compatibility concerns.
By adding extension methods to the existing collection interfaces,
implementations of those types can participate in the new programming
idiom. Implementations of those types in the JDK (and elsewhere) can
override the default implementations of the extension methods from the
superinterfaces to provide higher-performance or otherwise specialized
The platform components impacted by the overall Project Lambda effort include:
- The specification of the Java programming language
- The specification of the Java virtual machine
- The reference implementation of the language changes in
- Class file changes to support compilation of lambda expressions and virtual extension methods
- (Possible) JVM enhancements to improve execution of lambdas
- JVM changes to support virtual extension methods
- Core API changes to add virtual extension methods
- Core API changes to support use of lambda expressions
- Updates to the JDK libraries to use the new extension methods
- Updates to the reflective APIs, such as core reflection and
javax.lang.model, to expose lambda and extension method related information
- Updates to class file tools like
unpack200to understand new JVM attributes
- (Possible) serialization updates, including IIOP serialization
- Enhancements to javadoc to indicate which interfaces can be used for lambdas
- Language runtime in
java.lang.*to support the translation of lambda expressions and virtual extension methods
The work on lambda expression portion of Project Lambda is informed by numerous previous efforts, including but not limited to:
The feature set of Project Lambda is judged to better address the needs of evolving the Java platform than the earlier proposals.
The design of virtual extension methods is likewise informed by a large body of prior work in the programming language community including:
- multiple inheritance in C++
- static extension methods in C#
- traits in Fortress
- traits in Scala
- mixins in Strongtalk
Compared to other language environments, the Java language has always
had much more predictable semantics, built-in primitive types are of
a known size, exceptions are thrown at precise locations, the order of
execution of expressions is precisely defined, and so on. Loosening
the semantics of the built-in
while loops to allow
auto-parallelization attempts was judged to be neither desirable nor
sufficient to meet the goals of supporting multi-core.
In addition to unit/regression tests, as a new language feature JCK tests will also be developed.
The utility of the feature will also be validated through usage in the JDK platform libraries.
Risks and Assumptions
As a large effort touching many aspects of the platform, there is the possibility for unforeseen complications and interactions to arise.
By starting early and planning for several iterations of, say, the supporting libraries work, the impact of those complications should be mitigated.
The currently preferred implementation approach for lambda expressions relies
invokedynamic and method handles introduced by
JSR 292. Therefore,
acceptable performance of lambda depends on acceptable performance
of method handles, amongst other factors.
There is expected to be feedback between the libraries work (JEPs 107 and 109) and the language features.
- Other JDK components: The impact of other JDK components is outlined in the description section.
- Compatibility: Virtual extension methods are intended to allow source compatible evolution of interfaces.add virtual extension methods
- Security: Security review will be needed on aspect of the runtime implementation of lambdas.
- Performance/scalability: Performance of lambda-friendly idioms versus traditional idioms should be tracked.
- Documentation: User guides and supporting documentation needs to be written.
- TCK: As a large platform feature, language and library TCKs need to be developed.