ruby-on-rails - 仅当没有 delete_all 的多线程时，rake 任务中的 Rails 循环依赖错误

Question

我正在尝试创建一个 rake 任务来更新一些数据。问题是它有点慢所以我想我会让它多线程来加速它。问题是当我在 1 个线程中运行它时它很好但是当我在 5 个线程中运行它时我有时只得到

rake aborted!
Circular dependency detected while autoloading constant ClassInfo
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:492:in `load_missing_constant'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:184:in `const_missing'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:526:in `load_missing_constant'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:184:in `const_missing'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:66:in `block in load_class_from_link'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:187:in `block in each'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `upto'
/home/Tomer/.rvm/gems/ruby-2.2.2/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/node_set.rb:186:in `each'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:59:in `load_class_from_link'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:43:in `block (3 levels) in load_classes_from_catalog'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:42:in `each'
/home/Tomer/workspace/DrexelRegistrationBackend/app/helpers/html_parse_helper.rb:42:in `block (2 levels) in load_classes_from_catalog'

我正在使用 nokogiri 并打开一个 url 和 html 解析它，然后将它插入到我的数据库中。奇怪的是，如果我在开头插入 ClassInfo.delete_all，错误就会消失，但我不想那样做。我需要检查它是否在我的数据库中，所以我检查它是否存在于我崩溃的这一行中

myclass = ClassInfo.find_by(class_id: class_id)

它没有连接或依赖于它的其他类这是我的迁移

create_table :class_infos do |t|
      t.text :class_id, index: true
      t.text :subject_code
      t.text :subject
      t.text :course_number
      t.text :title
      t.float :credits_lower_range
      t.float :credits_upper_range
      t.text :description
      t.boolean :writing_intensive
      t.text :college
      t.text :repeat_status
      t.text :prereqs
      t.text :restrictions
      t.text :coreqs
      t.text :term_type
      t.text :student_type
      t.timestamps null: false
    end

这是模型

class ClassInfo < ActiveRecord::Base
end

非常奇怪的是，在它崩溃后，我通常可以立即再次运行 rake 任务，并且第二次运行正常。很奇怪

这里主要是我的 rake 调用的整个模块

require 'open-uri'

module HtmlParseHelper

def self.parse_catalog
    Thread.abort_on_exception=true
    @@threads = Array.new
       load_classes_from_catalog(url_sections)  #an array of 5 arrays contaning urls objects
    @@threads.each { |thr| thr.join }
end


def self.load_classes_from_catalog(urlArr)
    arrs.each{|urlSection|
      @@threads.push(
           Thread.new do
             urlSection.each{|url|                     
load_class_from_link(url[0],url[1],url[2],url[3])
                 }
                 ActiveRecord::Base.connection.close
               end
          )
        }
  end


def self.load_class_from_link(link, term_type, student_type, subject)

    begin
    doc = Nokogiri::HTML(open(link))do |config|
      config.strict.noblanks
    end
    class_id = doc.css('div.courseblock').children[0].text
    drexel_class = ClassInfo.find_by(class_id: class_id)   #where it crashes
    #.....
    drexel_class.save
end
end

Answer 1

https://github.com/rails/rails/commit/112077c255879351edf4530791cc4bcc7bd4005b

最后是急切加载把我搞砸了。

我刚刚添加了

Rails.application.eager_load!

从一开始就成功了