javascript - 为什么 SauceLabs 在我的 QUnit 测试明显通过时却说它失败了？

Question

我们正在尝试使用 SauceLabs 来验证我们基于浏览器的 QUnit 测试是否通过了流行的设备/浏览器组合......

当我们在浏览器中查看时，测试通过:https://ordenado.herokuapp.com/

但出于某种原因，SauceLabs 告诉我们他们“失败了”......

参见:https://saucelabs.com/tests/5b0f07813a7f4934bb44b07606ea2fd5

使用的 cURL 命令

作为引用，我们使用了以下 curl 命令:

curl https://saucelabs.com/rest/v1/ordem/js-tests \
-X POST \
-u ordem:SECRET_KEY \
-H 'Content-Type: application/json' \
--data '{
    "platforms": [
      ["Windows 8.1", "internet explorer", "11"],
      ["Windows 8", "internet explorer", "10"],
      ["Windows 8.1", "firefox", "beta"],
      ["Windows 8", "firefox", "37"],
      ["Windows 7", "firefox", "32"],
      ["OS X 10.8", "safari", "6"],
      ["OS X 10.8", "chrome", "37"],
      ["Linux", "chrome", "30"],
      ["Linux", "firefox", "dev"],
      ["OS X 10.10","iphone", "7.0"],
      ["OS X 10.10","iphone", "8.2"],
      ["OS X 10.10","ipad", "7.0"],
      ["OS X 10.10","ipad", "8.2"]
    ],
    "url": "https://qunit.herokuapp.com/test/test.html?coverage=true",
    "framework": "qunit",
    "name":"ordem",
    "public": "public",
    "build": "build-007"
}'

另外，有没有其他人遇到过以下错误:

"The Sauce VMs failed to start the browser or device"

访问:https://saucelabs.com/u/ordem完整的测试列表。单击 Sauce 声称“失败”的任何一个并观看视频以查看测试通过!!

非常感谢任何见解!

Answer 1

您应该添加几个 Hook 来报告来自 SauceLabs 的 qunit 测试结果。

有an article如何开始使用 qunit。那里提到了repository with example以下代码片段在哪里:

var log = [];
QUnit.done = function (test_results) {
  var tests = log.map(function(details){
    return {
      name: details.name,
      result: details.result,
      expected: details.expected,
      actual: details.actual,
      source: details.source
    }
  });
  test_results.tests = tests;

  // delaying results a bit cause in real-world
  // scenario you won't get them immediately
  setTimeout(function () { window.global_test_results = test_results; }, 2000);
};
QUnit.testStart(function(testDetails){
  QUnit.log = function(details){
    if (!details.result) {
     details.name = testDetails.name;
     log.push(details);
    }
 }
});

在您的测试和结果正确报告之前添加此代码