ruby-on-rails - Rails Rspec 套件有故障，但在单独运行时通过

Question

事实:

运行我的整个规范套件将导致 610 个规范中的 21 个一致错误。

如果我运行任何单独的规范文件(例如:messages_controller_spec.rb)，它们都会通过。

如果我单独运行任何失败的规范，它们都会通过。

这些错误主要是 ActionMailer 失败，但也有一些是其他错误。
一个令人困惑的方面是，一些规范失败是因为数据库中的行比预期多，而另一些规范失败是因为比预期少了一行。也就是说，如果它是一个清理或缓存问题，它似乎应该始终是多一个或少一个。

我目前正在通过内联运行 Rails 4.1.1、Ruby 2.0.0p451、Rspec 2.14.8、Sidekiq!

Gemfile(用于测试)

group :development, :test do
  gem 'better_errors'
  gem 'binding_of_caller'
  gem 'faker'
  gem 'guard-rspec'
  gem 'pry'
  gem 'rspec-rails'
  gem 'spork-rails'
  gem 'sqlite3'
  gem 'thin'
end

group :test do
  gem 'capybara'
  gem 'capybara-email'
  gem 'capybara-webkit'
  gem 'database_cleaner'
  gem 'fabrication'
  gem 'launchy'
  gem 'selenium-webdriver'
  gem 'shoulda-matchers'
  gem 'webmock'
  gem 'vcr'
end

请注意，我已经检查了十几个类似的问题，但都无法解决这个问题。所以，澄清一下:

我没有使用 ARGV

我没有使用 before(:all) - 我使用 before(:each)。

在 :suite 和 :each 规范之前，我都尝试过 Rails.cache.clear。

我将数据库清理器设置为 :truncation 以进行所有清理(比 :transaction 慢，但结果更好 - 使用 :transaction 重新加载更新值时遇到问题)

为了方便起见，也许一些例子会有所帮助:

scheduler_spec.rb(显示运行整个套件时失败的测试)

require 'spec_helper'
require 'rake'
require 'sidekiq/testing'
Sidekiq::Testing.inline!

describe "scheduler", :vcr do
  describe ":wipe_abandoned_images" do
    let!(:abandoned_old_image)   { Fabricate(:image) }
    let!(:abandoned_young_image) { Fabricate(:image) }
    let!(:adopted_image)         { Fabricate(:image) }
    let(:run_cleaner) do
      Rake::Task[:wipe_abandoned_images].reenable
      Rake.application.invoke_task :wipe_abandoned_images
    end

  before do
      abandoned_young_image.update_columns(listing_id: nil, updated_at: 6.days.ago)
      abandoned_old_image.update_columns(  listing_id: nil, updated_at: 9.days.ago)
      Rake.application.rake_require 'tasks/scheduler'
      Rake::Task.define_task(:environment) #Stub env. Rspec runs the App, so dont want Rake to run it again.
  end

  context "for claimed images" do
      it "leaves the image" do
        adopted_image_id = adopted_image.id
        run_cleaner
          expect(Image.all.count                ).to eq(2)
          expect(Image.find(adopted_image_id) ).to be_present
      end
    end
  end
end

请注意，我使用的是 Sidekiq 的内联!在之前的应用程序上测试配置成功且没有这个问题。

forgot_passwords_controller_spec.rb(显示运行整个套件时失败的测试)

require 'spec_helper'
require 'sidekiq/testing'
Sidekiq::Testing.inline!

describe ForgotPasswordsController do
  let!(:jen) { Fabricate(:user, email: 'jen@example.com') }

  describe "POST create" do
    context "with valid email provided" do
      before { post :create, email: 'jen@example.com' }
      after do
        ActionMailer::Base.deliveries.clear
        Sidekiq::Worker.clear_all
      end

      it 'sends the reset email to the users provided email' do
        expect(ActionMailer::Base.deliveries.count).to eq(1)
      end
    end
  end
end

以下是我以各种方式运行规范时发生的情况:

ForgotPasswordsController 规范通过 SublimeText2 中的 RubyTest 插件传递

2014-08-28T03:42:43Z 32968 TID-ov5g65p44 INFO: Sidekiq client with redis options {} ........... Finished in 0.95479 seconds 11 examples, 0 failures Randomized with seed 40226 [Finished in 5.8s]

调度程序测试通过 SublimeText2 中的 RubyTest

.Rationing out invitations to users... done. .Rationing out invitations to users... done. ..Sweeping the server for abandoned images... 2014-08-28T01:49:02Z 32426 TID-owjt9ggh8 INFO: Sidekiq client with redis options {} done. .Sweeping the server for abandoned images... done. .Sweeping the server for abandoned images... done. .

Finished in 1.52 seconds 7 examples, 0 failures Randomized with seed 37996 [Finished in 8.6s]

测试通过控制台中的 rspec
$ rspec ./spec/lib/tasks/scheduler_spec.rb

.Rationing out invitations to users... done. .Rationing out invitations to users... done. ..Sweeping the server for abandoned images... 2014-08-28T02:14:43Z 32456 TID-ouiui9g8c INFO: Sidekiq client with redis options {} done. .Sweeping the server for abandoned images... done. .Sweeping the server for abandoned images... done. .

Finished in 1.32 seconds 7 examples, 0 failures Randomized with seed 19172

运行整个 Rspec 套件时测试失败
如果单独运行或作为它们来自的规范文件运行，这些失败将通过。
$ rspec

Finished in 49.71 seconds 610 examples, 21 failures, 10 pending

失败的例子:

有时它有一个额外的值

13) scheduler :wipe_abandoned_images for abandoned images under 1 week old leaves the image Failure/Error: expect(Image.all.count ).to eq(2)

   expected: 2
        got: 3

   (compared using ==)
 # ./spec/lib/tasks/scheduler_spec.rb:78:in `block (4 levels) in <top (required)>'

有时会丢失或不加载值

18) ForgotPasswordsController POST create with valid email provided sends the reset email to the users provided email Failure/Error: expect(ActionMailer::Base.deliveries.count).to eq(1)

   expected: 1
        got: 0

   (compared using ==)
 # ./spec/controllers/forgot_passwords_controller_spec.rb:22:in `block (4 levels) in <top (required)>'

这是失败的列表

rspec ./spec/controllers/messages_controller_spec.rb:161 # MessagesController POST create message about listing to user from guest with valid information with EXISTING, UN-confirmed guest with EXPIRED token sends another confirmation email with link to the guest
rspec ./spec/controllers/messages_controller_spec.rb:114 # MessagesController POST create message about listing to user from guest with valid information with NEW, UN-confirmed, and valid guest email sends an invitation for the guest to be put on safe-email list
rspec ./spec/controllers/invitations_controller_spec.rb:30 # InvitationsController POST create with valid email & available invitations sends an email
rspec ./spec/controllers/invitations_controller_spec.rb:33 # InvitationsController POST create with valid email & available invitations sends an email to the recipient_email address
rspec ./spec/controllers/users_controller_spec.rb:161 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends the email to the registering user
rspec ./spec/controllers/users_controller_spec.rb:158 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends the email
rspec ./spec/controllers/users_controller_spec.rb:164 # UsersController POST create with invitation token in params with valid token & input confirmation email sending sends an email with a confirmation link in the body
rspec ./spec/controllers/users_controller_spec.rb:354 # UsersController GET confirm_with_token with valid token has a welcome message in the email
rspec ./spec/controllers/users_controller_spec.rb:348 # UsersController GET confirm_with_token with valid token sends a welcome email
rspec ./spec/controllers/users_controller_spec.rb:351 # UsersController GET confirm_with_token with valid token sends the welcome email to the user
rspec ./spec/controllers/searches_controller_spec.rb:19 # SearchesController GET search GET search with specific category selected returns the matching OR partial-matching table row objects
rspec ./spec/controllers/searches_controller_spec.rb:22 # SearchesController GET search GET search with specific category selected only returns values from the selected category
rspec ./spec/lib/tasks/scheduler_spec.rb:75 # scheduler :wipe_abandoned_images for abandoned images under 1 week old leaves the image
rspec ./spec/lib/tasks/scheduler_spec.rb:68 # scheduler :wipe_abandoned_images for abandoned images over 1 week old deletes the images
rspec ./spec/lib/tasks/scheduler_spec.rb:84 # scheduler :wipe_abandoned_images for claimed images leaves the image
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:24 # ForgotPasswordsController POST create with valid email provided sets the email subject to notify the user of the reset link
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:27 # ForgotPasswordsController POST create with valid email provided sends the link with token in the body of the email
rspec ./spec/controllers/forgot_passwords_controller_spec.rb:21 # ForgotPasswordsController POST create with valid email provided sends the reset email to the users provided email
rspec ./spec/controllers/reset_passwords_controller_spec.rb:70 # ResetPasswordsController POST create with a valid token sets the email subject to notify the user of the reset password
rspec ./spec/controllers/reset_passwords_controller_spec.rb:67 # ResetPasswordsController POST create with a valid token sends a confirmation email to the user that their password has been changed
rspec ./spec/controllers/reset_passwords_controller_spec.rb:73 # ResetPasswordsController POST create with a valid token sends the link with token in the body of the email

Answer 1

我无法解释真正导致错误的原因。但这一定是由于全局设置了 Sidekiq 测试模式。从规范的头部部分删除 Sidekiq 设置并尝试以下操作:

    before do 
    Sidekiq::Testing.inline! do
      post :create, email: 'jen@example.com' 
     end  
    end
    after do
      ActionMailer::Base.deliveries.clear
      Sidekiq::Worker.clear_all
    end

    it 'sends the reset email to the users provided email' do
      expect(ActionMailer::Base.deliveries.count).to eq(1)
    end