Android Firebase 测试实验室 - 使用 Orchestrator 进行仪器测试，其目的是什么？

Question

来自 the docs

New: instrumentation tests with Orchestrator Android Test Orchestrator runs each of your app's instrumentation tests independently, which has several benefits, but also one drawback:

Benefits

No shared state: Each test runs in its own Instrumentation instance, so the shared state doesn't accumulate across tests.

Isolated crashes: If test crashes, it takes down only its own instance of Instrumentation, so the other tests in your suite still run.

Drawback

Longer runtime: Each test running its own Instrumentation instance means that the testing process takes slightly longer overall. The increased runtime could impact your quota usage or billed time and might cause you to hit devices' time-out limits.

现在它建议在本地运行新的编排器以确保其正常工作。所以我在将 testOptions 添加到模块的 build.gradle 文件后运行了它:

testOptions {
        execution 'ANDROID_TEST_ORCHESTRATOR'
    }

具有 gradle 依赖项:

 androidTestImplementation 'com.android.support.test:runner:1.0.1'
        androidTestUtil 'com.android.support.test:orchestrator:1.0.1'

但我在执行 ./gradlew connectedCheck 时没有看到任何区别

我尝试插入物理设备并同时运行模拟器 api 26。所以我现在有两个设备连接到 adb。然后执行./gradlew connectedCheck。我在控制台终端中看到以下输出:

#./gradlew connectedCheck                  
Parallel execution with configuration on demand is an incubating feature.
> Configure project :app 
Configuration 'compile' in project ':app' is deprecated. Use 'implementation' instead.
The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead.
        at build_495ocpxnm4a4fvowoj4qizq7q.run(/Users/me/Development/QA/android/MyApp/app/build.gradle:303)
        (Run with --stacktrace to get the full stack trace of this deprecation warning.)
The CompileOptions.bootClasspath property has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the CompileOptions.bootstrapClasspath property instead.
[newrelic.info] Android Gradle plugin version 3.+ detected. Using transform API
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedFolders(FileCollection)
> Task :app:processLabsDebugGoogleServices 
Parsing json file: /Users/me/Development/QA/android/Myapp/app/google-services.json
Starting 39 tests on Nexus_6P_API_26_oreo_xxhdpi(AVD) - 8.0.0
Starting 39 tests on SM-G610F - 7.0
<====---------> 33% EXECUTING [4m 40s]

它正在两台设备上运行测试。这是编排器所做的吗？

编排器的目的是什么？

Answer 1

Android Test Orchestrator确保您的每个测试都在隔离的环境中运行。这意味着先前测试的结果不太可能影响 future 测试的结果(由于共享状态)。

Read more在 Android 开发者博客和 this question 上.