java - 为什么 JUnit 只在第一次失败之前运行 Theory 的测试用例？

Question

最近一个新概念Theories已添加到 JUnit(自 v4.4 起)。

简而言之，你可以用@Theory注解(而不是@Test)标记你的测试方法，使你的测试方法参数化并声明一个参数数组，在同一类的某处用 @DataPoints 注释标记。

JUnit 将依次运行您的参数化测试方法，依次传递从 @DataPoints 检索到的参数。但直到第一次这样的调用失败(由于任何原因)。

这个概念似乎与TestNG的@DataProviders非常相似，但是当我们使用数据提供者时，所有的场景都在运行，不管它们的执行结果如何。它很有用，因为您可以看到有多少方案有效/无效，并且您可以更有效地修复您的程序。

那么，我想知道为什么不对每个@DataPoint 执行@Theory 标记的方法？ (从 Theories runner 继承并制作一个忽略失败的自定义 runner 似乎并不难，但为什么我们没有开箱即用的这种行为？)

UPD:我已经创建了 Theories runner 的容错版本并使其可供公众访问:https://github.com/rgorodischer/fault-tolerant-theories

为了将其与标准 Theories runner 进行比较，运行 StandardTheoriesBehaviorDemo，然后运行 ​​FaultTolerantTheoriesBehaviorDemo，它们位于 src/test/... 文件夹下。

Answer 1

Reporting multiple failures in a single test is generally a sign that the test does too much, compared to what a unit test ought to do. Usually this means either that the test is really a functional/acceptance/customer test or, if it is a unit test, then it is too big a unit test.

JUnit is designed to work best with a number of small tests. It executes each test within a separate instance of the test class. It reports failure on each test. Shared setup code is most natural when sharing between tests. This is a design decision that permeates JUnit, and when you decide to report multiple failures per test, you begin to fight against JUnit. This is not recommended.

Long tests are a design smell and indicate the likelihood of a design problem. Kent Beck is fond of saying in this case that "there is an opportunity to learn something about your design." We would like to see a pattern language develop around these problems, but it has not yet been written down. Source: http://junit.sourceforge.net/doc/faq/faq.htm#tests_12

要忽略断言失败，您还可以使用 JUnit 错误收集器规则:

The ErrorCollector rule allows execution of a test to continue after the first problem is found (for example, to collect all the incorrect rows in a table, and report them all at once)

例如，您可以编写这样的测试。

public static class UsesErrorCollectorTwice {
  @Rule
  public ErrorCollector collector= new ErrorCollector();

  @Test
  public void example() {
    String x = [..]
    collector.checkThat(x, not(containsString("a")));
    collector.checkThat(y, containsString("b"));             
  }
}

错误收集器使用 hamcrest Matchers。这是积极的还是消极的，取决于您的偏好。