benchmarking - 如何在 jmh 基准测试中生成方法？

Question

我使用 jmh( http://openjdk.java.net/projects/code-tools/jmh/ ) 来对某些方法进行基准测试。此外，我还有一组参数，我想将其用作运行此方法的参数。
是否可以为每个特定参数值生成一个方法(使用@GenerateMicroBenchmark 注释)？

现在我使用类似的实现，但不是那么方便，因为我要手工编写很多统一的代码:

interface State {
    int action();
    void prepare();
}

class Method {
    ...;
    toString() { return "State1_" + param; }
}

{
    Method[] states;
    curState = -1;
    count = 0;
    int[] params = {1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 11000, 12000};
    for (int param: params) {
      states[count++] = new Method(param);
    }
}

@Setup(Level.Invocation)
public void prepareState() {
  if (curState != -1) {
    states[curState].prepare();
  }
}

@GenerateMicroBenchmark
public int param_1000() {
    curState = 0;
    return states[curState].action();
}

@GenerateMicroBenchmark
public int param_2000() {
    curState = 1;
    return states[curState].action();
}

@GenerateMicroBenchmark
public int param_3000() {
    curState = 2;
    return states[curState].action();
}
...
@GenerateMicroBenchmark
public int param_12000() {
    curState = 11;
    return states[curState].action();
}

Answer 1

基准通常不会在没有严重损坏的情况下重用。我见过人们试图简化他们的基准的大多数尝试都让他们难以置信。例如，在此处使用数组可能会抵消纳米基准测试的结果(例如，如果 action() 很小)。 Level.Invocation通常也是一个坏主意，如 its Javadoc 中所述.

底线是，仅仅因为 API 允许某些使用，并不一定意味着您应该使用它。这是你应该做的:

@State(Scope.Thread)
class MyBenchmark {
    Method[] states;

    @Setup
    public void setup() {
        int[] params = {1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000, 11000, 12000};
        int count = 0;
        for (int param: params) {
            states[count++] = new Method(param);
        }
    }

    int doWork(int idx) {
        states[idx].prepare();
        return states[idx].action();
    }

    @GenerateMicroBenchmark
    public int param_1000() {
       doWork(0);
    }

    ...

    @GenerateMicroBenchmark
    public int param_12000() {
       doWork(11);
    }
}

...甚至:

@State(Scope.Thread)
class MyBenchmark {
    Method p1000, p2000, ..., p12000;

    @Setup
    public void setup() {
        p1000 = new Method(p1000);
        p2000 = new Method(p2000);
        ...
        p12000 = new Method(p12000);
    }

    @GenerateMicroBenchmark
    public int param_1000() {
       p1000.prepare();
       return p1000.action();
    }

    ...

    @GenerateMicroBenchmark
    public int param_12000() {
       p12000.prepare();
       return p12000.action();
    }
}

另一种方法是从外部接受参数，并使用 Java API 来处理参数。举个例子:

@State(Scope.Thread)
class MyBenchmark {
    final int param = Integer.getInteger("param", 1000); 

    Method m;

    @Setup
    public void setup() {
        m = new Method(param);
    }

    @GenerateMicroBenchmark
    public int work() {
       m.prepare();
       return m.action();
    }

     public static void main(String[] args) throws RunnerException {
         Options opts = new OptionsBuilder()
                 .include(".*")
                 .jvmArgs("-Dparam=2000")
                 .build();

         RunResult runResult = new Runner(opts).runSingle();
         Result result = runResult.getPrimaryResult();

         System.out.println();
         System.out.println("API replied benchmark score: " + result.getScore() + " " + result.getScoreUnit() + " over " + result.getStatistics().getN() + " iterations");
     }
}