JEP 349: JFR Event Streaming

AuthorsErik Gahlin, Markus Grönlund
OwnerErik Gahlin
TypeFeature
ScopeJDK
StatusTargeted
Release14
Componenthotspot / jfr
Discussionhotspot dash jfr dash dev at openjdk dot java dot net
EffortM
DurationS
Reviewed byKaren Kinnear, Mikael Vidstedt
Endorsed byMikael Vidstedt
Created2017/07/11 19:20
Updated2019/10/18 17:54
Issue8184193

Summary

Expose JDK Flight Recorder data for continuous monitoring.

Goals

Non-Goals

Motivation

The HotSpot VM emits more than 500 data points using JFR, most of them not available through other means besides parsing log files.

To consume the data today, a user must start a recording, stop it, dump the contents to disk and then parse the recording file. This works well for application profiling, where typically at least a minute of data is being recorded at a time, but not for monitoring purposes. An example of monitoring usage is a dashboard which displays dynamic updates to the data.

There is overhead associated with creating a recording, such as:

If there were a way to read data being recorded from the disk repository without creating a new recording file, much of this overhead could be avoided.

Description

The package jdk.jfr.consumer, in module jdk.jfr, is extended with functionality to subscribe to events asynchronously. Users can read recording data directly, or stream, from the disk repository without dumping a recording file. The way to interact with a stream is to register a handler, for example a lambda function, to be invoked in response to the arrival of an event.

The following example prints the overall CPU usage and locks contended for more than 10 ms.

try (var rs = new RecordingStream()) {
  rs.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1));
  rs.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMillis(10));
  rs.onEvent("jdk.CPULoad", event -> {
    System.out.println(event.getFloat("machineTotal"));
  });
  rs.onEvent("jdk.JavaMonitorEnter", event -> {
    System.out.println(event.getClass("monitorClass"));
  });
  rs.start();
}

The RecordingStream class implements the interface jdk.jfr.consumer.EventStream that provides a uniform way to filter and consume events regardless if the source is a live stream or a file on disk.

public interface EventStream extends AutoCloseable {
  public static EventStream openRepository();
  public static EventStream openRepository(Path directory);
  public static EventStream openFile(Path file);

  void setStartTime(Instant startTime);
  void setEndTime(Instant endTime);
  void setOrdered(boolean ordered);
  void setReuse(boolean reuse);

  void onEvent(Consumer<RecordedEvent> handler);
  void onEvent(String eventName, Consumer<RecordedEvent handler);
  void onFlush(Runnable handler);
  void onClose(Runnable handler);
  void onError(Runnable handler);
  void remove(Object handler);

  void start();
  void startAsync();

  void awaitTermination();
  void awaitTermination(Duration duration);
  void close();
}

There are three factory methods to create a stream. EventStream::openRepository(Path) constructs a stream from a disk repository. This is a way to monitor other processes by working directly against the file system. The location of the disk repository is stored in the system property "jdk.jfr.repository" that can be read using the attach API. It is also possible to perform in-process monitoring using the EventStream::openRepository() method. Unlike RecordingStream, it does not start a recording. Instead, the stream receives events only when recordings are started by external means, for example using JCMD or JMX. The method EventStream::openFile(Path) creates a stream from a recording file. It complements the RecordingFile class that already exists today.

The interface can also be used to set the amount of data to buffer and if events should be ordered chronologically. To minimize allocation pressure, there is also an option to control if a new event object should be allocated for each event, or if a previous object can be reused. A stream can be started in the current thread or asynchronously.

Events stored in thread-local buffers are flushed periodically to the disk repository by the Java Virtual Machine (JVM) once every second. A separate thread parses the most recent file, up to the point in which data has been written, and pushes the events to subscribers. To keep overhead low, only actively subscribed events are read from the file. To receive a notification when a flush is complete, a handler can be registered using the EventStream::onFlush(Runnable) method. This is an opportunity to aggregate or push data to external systems while the JVM is preparing the next set of events.

Alternatives

JMX notifications provide a means for the JDK and third-party applications to expose information for continuous monitoring. There are, however, drawbacks that make JMX unsuited for the purpose of this JEP.

Testing

Risks and Assumptions