java - 如何从 JUnit 测试中运行 JMH？

Question

如何使用 JUnit 测试在现有项目中运行 JMH 基准测试？官方文档建议制作一个单独的项目，使用 Maven shade 插件，并在 main 方法中启动 JMH。这是必要的吗？为什么推荐它？

Answer 1

我一直在使用 JUnit 在我现有的 Maven 项目中运行 JMH，没有明显的不良影响。我无法回答为什么作者建议以不同的方式做事。我没有观察到结果的差异。 JMH 启动一个单独的 JVM 来运行基准测试以隔离它们。这是我的工作:

• 将 JMH 依赖项添加到您的 POM:

  <dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.21</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.21</version>
    <scope>test</scope>
  </dependency>
  

  请注意，我已将它们放在 test 范围内。

  在 Eclipse 中，您可能需要手动配置注释处理器。 NetBeans 会自动处理这个问题。

• 创建您的 JUnit 和 JMH 类。我选择将两者合并到一个类中，但这取决于你。请注意，OptionsBuilder.include 是真正确定将从您的 JUnit 测试中运行哪些基准的东西!

  import java.util.ArrayList;
  import java.util.List;
  import java.util.Random;
  import java.util.concurrent.TimeUnit;
  import org.junit.Test;
  import org.openjdk.jmh.annotations.*;
  import org.openjdk.jmh.infra.Blackhole;
  import org.openjdk.jmh.runner.Runner;
  import org.openjdk.jmh.runner.options.*;
  
  
  public class TestBenchmark 
  {
  
        @Test public void 
      launchBenchmark() throws Exception {
  
              Options opt = new OptionsBuilder()
                  // Specify which benchmarks to run. 
                  // You can be more specific if you'd like to run only one benchmark per test.
                  .include(this.getClass().getName() + ".*")
                  // Set the following options as needed
                  .mode (Mode.AverageTime)
                  .timeUnit(TimeUnit.MICROSECONDS)
                  .warmupTime(TimeValue.seconds(1))
                  .warmupIterations(2)
                  .measurementTime(TimeValue.seconds(1))
                  .measurementIterations(2)
                  .threads(2)
                  .forks(1)
                  .shouldFailOnError(true)
                  .shouldDoGC(true)
                  //.jvmArgs("-XX:+UnlockDiagnosticVMOptions", "-XX:+PrintInlining")
                  //.addProfiler(WinPerfAsmProfiler.class)
                  .build();
  
              new Runner(opt).run();
          }
  
      // The JMH samples are the best documentation for how to use it
      // http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/
      @State (Scope.Thread)
      public static class BenchmarkState
      {
          List<Integer> list;
  
            @Setup (Level.Trial) public void
          initialize() {
  
                  Random rand = new Random();
  
                  list = new ArrayList<>();
                  for (int i = 0; i < 1000; i++)
                      list.add (rand.nextInt());
              }
      }
  
        @Benchmark public void 
      benchmark1 (BenchmarkState state, Blackhole bh) {
  
              List<Integer> list = state.list;
  
              for (int i = 0; i < 1000; i++)
                  bh.consume (list.get (i));
          }
  }
  

• JMH 的注释处理器似乎不适用于 NetBeans 中的保存时编译。每当您修改基准时，您可能需要执行完整的 Clean and Build。 (任何建议表示赞赏!)

• 运行您的 launchBenchmark 测试并观察结果!

  -------------------------------------------------------
   T E S T S
  -------------------------------------------------------
  Running com.Foo
  # JMH version: 1.21
  # VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
  # VM invoker: /usr/lib/jvm/java-8-jdk/jre/bin/java
  # VM options: <none>
  # Warmup: 2 iterations, 1 s each
  # Measurement: 2 iterations, 1 s each
  # Timeout: 10 min per iteration
  # Threads: 2 threads, will synchronize iterations
  # Benchmark mode: Average time, time/op
  # Benchmark: com.Foo.benchmark1
  
  # Run progress: 0.00% complete, ETA 00:00:04
  # Fork: 1 of 1
  # Warmup Iteration   1: 4.258 us/op
  # Warmup Iteration   2: 4.359 us/op
  Iteration   1: 4.121 us/op
  Iteration   2: 4.029 us/op
  
  
  Result "benchmark1":
    4.075 us/op
  
  
  # Run complete. Total time: 00:00:06
  
  REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
  why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
  experiments, perform baseline and negative tests that provide experimental control, make sure
  the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
  Do not assume the numbers tell you what you want them to tell.
  
  Benchmark                                Mode  Cnt  Score   Error  Units
  Foo.benchmark1                           avgt    2  4.075          us/op
  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.013 sec
  

• Runner.run 甚至返回 RunResult 对象，您可以在这些对象上进行断言等。