JEP draft: A Java 2D Metal API based rendering pipeline for macOS

Authorprr
OwnerPhilip Race
TypeFeature
ScopeJDK
StatusDraft
Releasetbd
Componentclient-libs / 2d
Reviewed byJayathirth D V, Kevin Rushforth, Sergey Bylokhov
Created2020/01/31 21:40
Updated2020/04/04 01:21
Issue8238361

Summary

Implement a Java 2D internal rendering pipeline for macOS using the Apple Metal APIs as an alternative to the deprecated Apple OpenGL APIs.

Goals

Non-Goals

Success Metrics

The success criteria for the new rendering pipeline is based on:

  1. Functional completeness - no functional regressions, pass TCK.
  2. Rendering Performance - measured and perceived performance should match or exceed OpenGL.

Refer to the "Testing" section for more details.

Motivation

There are two major factors motivating this introduction of a new Java 2D Metal based rendering pipeline on macOS.

  1. Deprecation of the OpenGL framework on macOS. Java 2D on macOS is completely reliant on OpenGL for its internal rendering pipeline for UI applications. Apple deprecated the OpenGL rendering library in macOS 10.14. See macOS 10.14 Release Notes Consequently Java 2D needs a replacement rendering pipeline implementation.
  2. The Metal framework is Apple's replacement for OpenGL and independently of the deprecation Apple claim benefits in application performance over OpenGL.

Description

Most Java SE UI applications are written using the Swing toolkit. This is a lightweight toolkit that renders using the Java 2D API. Internally Java 2D can use software rendering plus a blit to the screen or it can use a platform-specific API, such as X11/Xrender or Direct3D, or on MacOS, OpenGL. These platform APIs often offer much better performance of the application than software rendering and generally off-load the CPU. Metal is the new Apple platform API for such rendering replacing the deprecated OpenGL API. Note: the name has nothing to do with the Swing Metal Look And Feel. That is just a coincidence.

Using the Apple Metal API framework we will update internal JDK code, both Java and native, and create new Java and native code to utilise the Metal framework in a similar way as we already do for OpenGL on Mac, and for Direct3D on Microsoft Windows. Consequently this means there is an existing design/framework for this to fit in with. Opportunities to leverage new capabilities of the Metal framework, such as more concurrent rendering will be investigated for potential.

Refactoring outside of the OpenGL/Metal code is expected to be minimal.

This implementation of a new rendering pipeline does not introduce any new Java API.

However if necessary we will update the Java AWT Native Interface to provide access to a Metal native drawing surface. That is a native code interface, and does not affect Java language applications.

The Metal pipeline will co-exist with the OpenGL rendering pipeline. On startup of a UI application one or the other will be chosen. For the initial implementation OpenGL will remain the default, with Metal used only if it is specified on startup, or initialisation of OpenGL fails - as might happen in a future version of macOS with no OpenGL support.

Assuming Apple provide a sufficient period of time before removing OpenGL support, the initial integration of this JEP will require that applications opt-in to metal by specifying
-Dsun.java2d.metal=true on the command line used to start the JDK. We therefore expect both OpenGL and Metal rendering pipelines to co-exist on macOS platform for some time. The metal rendering pipeline will be made the default in a later release. Prior to integration of the JEP, all work described here will be conducted under Project Lanai. See Project Lanai

Alternatives

There are no other standard macOS alternatives to Metal for an accelerated graphics pipeline.

Testing

The testing is categorized in two parts - functionality and performance.

Testing the functionality of the new pipeline does not require new test development since no Java 2D API is changed. Consequently, for the most part, existing tests will be used for testing of the new rendering pipeline.

We also have some pre-existing performance tests (see J2DBench) below. However this does not preclude other real world performance testing being developed and one focus here will be on the performance of the IntelliJ IDEA IDE as a representative stress test of performance.

Tests for functional completeness :

  1. JDK jtreg regression tests.
  2. JCK Tests
  3. JDK's Java 2D and Swing Demos.
  4. IDEs such as Intellij IDE and Netbeans as examples of large scale real world applications.

Tests for Performance Measurements :

  1. J2DBench Bench-marking Tests. J2DBench is a Java 2D test application included in JDK.
  2. RenderPerfTest - A custom stress test that renders 'N' objects of the same primitive type and measures Frames per second (FPS). This test is in the Lanai Project repo.
  3. IntelliJ IDEA IDE performance

The above lists are subject to some change as the work progresses.

Risks and Assumptions

Although unlikely, until we are nearly complete, we cannot be sure that Metal will perform as well as supposed in all cases. However prototype work to date under Project Lanai suggests we are well on the way to exceeding our goals/metrics. Metal may not provide equivalent functionality in all cases. For example, so far no way has been discovered to efficiently implement the XOR operation, which although niche, is required.

Dependencies

There are no dependencies.