karma-runner - karma-sauce-launcher 断开每次测试运行，导致 IE 和 Safari 运行失败

Question

通过所涉及的每个组件的多个版本(karma、socket.io、sauce-connect)，我已经有这个确切的问题几个月了。目前我在 Karma 12.16 上运行。

基本上，当使用 https://github.com/karma-runner/karma-sauce-launcher 、Safari 和 IE 浏览器启动时会断开每次测试运行，导致仅部分测试运行。相关的 karma 日志是

Driving the web on session: abf7c976abf1420a9c72b3e06c655040    
DEBUG [wd]:  > RESPONSE init({"base":"SauceLabs","browserName":"internet explorer","platform":"Windows 8.1","version":"11","tags":[],"name":"Karma and Sauce Labs demo","record-video":false,"record-screenshots":true,"build":null,"device-orientation":null,"disable-popup-handler":true}) "abf7c976abf1420a9c72b3e06c655040",null
INFO [launcher.sauce]: internet explorer 11 (Windows 8.1) session at https://saucelabs.com/tests/abf7c976abf1420a9c72b3e06c655040
DEBUG [launcher.sauce]: WebDriver channel for internet explorer 11 (Windows 8.1) instantiated, opening http://localhost:9876/?id=15483642
DEBUG [wd]:  > CALL get("http://localhost:9876/?id=15483642") 
DEBUG [wd]:  > POST /session/:sessionID/url {"url":"http://localhost:9876/?id=15483642"}
DEBUG [web-server]: serving: /Volumes/OSXSTORAGE/Users/a/Dropbox/a/WebstormProjects/a-ui/node_modules/karma/static/client.html
DEBUG [web-server]: serving: /Volumes/OSXSTORAGE/Users/a/Dropbox/a/WebstormProjects/a-ui/node_modules/karma/static/karma.js
DEBUG [wd]:  > RESPONSE get("http://localhost:9876/?id=15483642") 
DEBUG [karma]: A browser has connected on socket xXySXoeppVtPhAr8FVR4
INFO [IE 11.0.0 (Windows)]: Connected on socket xXySXoeppVtPhAr8FVR4 with id 15483642
DEBUG [launcher]: internet explorer 11 (Windows 8.1) on SauceLabs (id 15483642) captured in 17.693 secs
DEBUG [web-server]: serving: /Volumes/OSXSTORAGE/Users/a/Dropbox/a/WebstormProjects/a-ui/node_modules/karma/static/context.html
DEBUG [web-server]: serving (cached): /Volumes/OSXSTORAGE/Users/a/Dropbox/a/WebstormProjects/a-ui/src/components/angular/angular.js
................................................................................
..............DEBUG [karma]: A browser has connected on socket mGf0VzQ8DKdfW_NSFVR5
DEBUG [IE 11.0.0 (Windows)]: New connection mGf0VzQ8DKdfW_NSFVR5 (already have xXySXoeppVtPhAr8FVR4)
WARN [IE 11.0.0 (Windows)]: Disconnected (1 times), because no message in 10000 ms.

IE 11.0.0 (Windows): Executed 94 of 94 DISCONNECTED (11.481 secs / 0.757 secs)
DEBUG [launcher.sauce]: Shutting down the internet explorer 11 (Windows 8.1) driver
DEBUG [wd]:  > CALL get("about:blank") 
DEBUG [wd]:  > POST /session/:sessionID/url {"url":"about:blank"}
DEBUG [karma]: Run complete, exitting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [wd]:  > RESPONSE get("about:blank") 
DEBUG [wd]:  > CALL quit() 
DEBUG [wd]:  > DELETE /session/:sessionID 
DEBUG [wd]: 
Ending your web drivage..

DEBUG [wd]:  > RESPONSE quit() 

我曾尝试将 browserNoActivityTimeout 增加到 60000，但这只会导致测试在 60 秒而不是 10 秒后失败。这些失败中的每一个都有以下内容:

DEBUG [karma]: A browser has connected on socket mGf0VzQ8DKdfW_NSFVR5
DEBUG [IE 11.0.0 (Windows)]: New connection mGf0VzQ8DKdfW_NSFVR5 (already have xXySXoeppVtPhAr8FVR4)
WARN [IE 11.0.0 (Windows)]: Disconnected (1 times), because no message in 10000 ms.   
IE 11.0.0 (Windows): Executed 94 of 94 DISCONNECTED (11.481 secs / 0.757 secs)

根据浏览器的不同，有时它只会执行 50 个测试(Executed 50 of 94 DISCONNECTED)，而在其他情况下它会执行所有测试，但仍被视为测试运行失败。我从未在 Chrome 或 Firefox 中看到过这种情况。

每次都会在本地运行完全相同的 karma.conf 文件，不会出现任何问题。有没有人遇到过这个？我已经联系了 Saucelabs 支持，但他们开始坚持认为这是一个 karma-sauce-launcher 问题。

Karma.conf:

var fs = require('fs');

module.exports = function(config) {

  // Use ENV vars on Travis and sauce.json locally to get credentials
  if (!process.env.SAUCE_USERNAME) {
    if (!fs.existsSync('sauce.json')) {
      console.log('Create a sauce.json with your credentials based on the sauce-sample.json file.');
      process.exit(1);
    } else {
      process.env.SAUCE_USERNAME = require('./../sauce').username;
      process.env.SAUCE_ACCESS_KEY = require('./../sauce').accessKey;
    }
  }

  // Browsers to run on Sauce Labs
  var customLaunchers = {

    sl_safari_ml: {
      base: 'SauceLabs',
      browserName: 'safari',
      platform: 'OS X 10.9'

    }
  };

  config.set({

    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: '../',


    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['jasmine'],


    // list of files / patterns to load in the browser
    files: [
      'src/components/jquery/dist/jquery.js',
      'src/components/angular/angular.js',
      'src/components/angular-mocks/angular-mocks.js',
      'src/components/angular-cookies/angular-cookies.js',
      'src/components/angular-sanitize/angular-sanitize.js',
      'src/components/angular-ui-router/release/angular-ui-router.js',
      'src/components/angular-animate/angular-animate.js',
      'src/components/angular-strap/dist/angular-strap.js',
      'src/components/angular-strap/dist/angular-strap.tpl.js',
      'src/components/speakingurl/lib/index.js',
      'src/app/**/*.js',
      'src/app/common/templates/*.html'


    ],

    exclude: [
      'src/app/**/*.e2e.spec.js'

    ],
    ngHtml2JsPreprocessor: {
      stripPrefix: 'src/app/common/',
      moduleName: 'mywire.templates'
    },
    preprocessors: {
      // load templates
      'src/app/common/templates/*.html': ['ng-html2js']
    },
    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['dots'],


    // web server port
    port: 9876,

    colors: true,

    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_INFO,

    sauceLabs: {
      testName: 'Karma and Sauce Labs demo',
      startConnect: false
    },
    browserNoActivityTimeout: 60000,
    captureTimeout: 120000,
    customLaunchers: customLaunchers,

    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: Object.keys(customLaunchers),
    singleRun: true
  });
};

Answer 1

从 Karma 0.12 版开始，添加了更多超时设置。

将它们添加到我的 karma 配置中为我解决了这个问题。

browserDisconnectTimeout

类型:号码

默认值:2000

说明:Karma 等待多长时间
要重新连接的浏览器(以毫秒为单位)。带有片状连接，它很漂亮
常见的情况是浏览器断开连接但实际执行正在运行
没有任何问题。 Karma 不会将断开连接视为
立即失败，而是等待 browserDisconnectTimeout 毫秒。如果
在此期间浏览器重新连接，一切正常。

——

browserDisconnectTolerance

类型:号码

默认值:0

描述:允许的断开连接数。
disconnectTolerance 值表示在断开连接的情况下浏览器将尝试的最大尝试次数。通常，任何断开连接都被视为失败，但是当 karma 服务器和浏览器之间存在不稳定的网络链接时，此选项允许定义容错级别。

——

browserNoActivityTimeout

类型:号码

默认值:10000

描述:Karma 在断开浏览器之前等待来自浏览器的消息的时间(以毫秒为单位)。
如果在执行过程中 Karma 在 browserNoActivityTimeout ms 内没有收到来自浏览器的任何消息，它将断开浏览器的连接。

来源: http://karma-runner.github.io/0.12/config/configuration-file.html

更多提示: http://oligofren.wordpress.com/2014/05/27/running-karma-tests-on-browserstack/