Code Tools: jmh

JMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targetting the JVM.

Basic Considerations

The recommended way to run a JMH benchmark is to use Maven to setup a standalone project that depends on the jar files of your application. This approach is preferred to ensure that the benchmarks are correctly initialized and produce reliable results. It is possible to run benchmarks from within an existing project, and even from within an IDE, however setup is more complex and the results are less reliable.

In all cases, the key to using JMH is enabling the annotation- or bytecode-processors to generate the synthetic benchmark code. Maven archetypes are the primary mechanism used to enable this. We strongly recommend new users make use of the archetype to setup the correct environment.

Preferred Usage: Command Line

When dealing with large projects, it is customary to keep the benchmarks in a separate subproject, which then depends on the tested modules via the usual build dependencies.

IDE support

While the command line approach is the suggested one, some people prefer to use IDEs. The user experience varies between different IDEs, but we will outline the general things here. Running benchmarks from the IDE is generally not recommended due to generally uncontrolled environment in which the benchmarks run.

Other build systems

We do not ship the build scripts for other build systems, but there are community-supported bindings to Gradle, sbt, and probably other builders, see Links section below. If you want to build with an alternative build system, you may reference the Ant sample which describes the steps to build JMH benchmark projects with Ant.

Building the "Bleeding Edge" JMH

In some cases, you want bleeding edge code which contains fixes, features, and APIs not available yet in released versions. This section explains how to get the "bleeding edge" JMH in your project.

  1. Check out JMH source with Mercurial:
    $ hg clone jmh
  2. Build JMH. You can optionally skip the tests:
    $ cd jmh/
    $ mvn clean install -DskipTests

    You only need to do this step once, and Maven will deploy JMH into Maven repo on your local machine.

  3. If you already have the benchmark project, then it is enough to change JMH dependencies versions to 1.0-SNAPSHOT. If not, create the JMH benchmark project and change the version there.

  4. Done! Build the benchmark project, and run it:

    $ mvn clean install
    $ java -jar target/benchmarks.jar

Getting Support

Make sure you did this before publishing the benchmark, and/or requesting the JMH feature: Make sure you tried these things before getting support:

If all these did not help, you are welcome at the JMH mailing list.


Related community-supported projects: