ruby-on-rails - Ruby 线程的重复结果

Question

我需要通过从外部服务器获取图像来改进构建布料外观的 rake 任务。

当我尝试创建多个线程时，结果重复。

但是如果我在每个 Thread.new 之前放置 sleep 0.1，代码就可以工作了!为什么？

new_looks = []
threads = []

for look in looks
  # sleep 0.1 - when I put it, works!
  threads << Thread.new do
    # a external http request is being done here
    new_looks << Look.new(ref: look["look_ref"])
  end
end

puts  'waiting threads to finish...'
threads.each(&:join)

puts  'saving...'
new_looks.sort_by(&:ref).each(&:save)

Answer 1

数组通常不是线程安全的。切换到线程安全的数据结构，例如 Queue:

new_look_queue = Queue.new
threads = looks.map do |look|
  Thread.new do
    new_look_queue.enq Look.new(ref: look["look_ref"])
  end
end

puts  'waiting threads to finish...'
threads.each(&:join)

puts  'saving...'
new_looks = []
while !new_look_queue.empty?
  new_look_queue << queue.deq
end
new_looks.sort_by(&:ref).each(&:save)

Queue#enq 将新条目放入队列； Queue#deq 取出一个，如果没有则阻塞。

如果不需要按顺序保存new_looks，代码会变得更简单:

puts 'saving...'
while !new_look_queue.empty?
  new_look_queue.deq.save
end

或者，甚至更简单，只需在线程内进行保存。

---

如果你有很多外观，上面的代码会创建更多的线程。过多的线程会导致处理请求的时间过长，并消耗过多的内存。在这种情况下，考虑创建一些生产者线程:

NUM_THREADS = 8

和以前一样，有一个已完成工作的队列:

new_look_queue = Queue.new

但现在还有一系列工作要做:

look_queue = Queue.new
looks.each do |look|
  look_queue.enq look
end

每个线程都会一直存在，直到它停止工作，所以让我们向队列中添加一些“停止工作”符号，每个线程一个:

NUM_THREADS.times do {look_queue.enq :done}

现在线程:

threads = NUM_THREADS.times.map do
  Thread.new do
    while (look = look_queue.deq) != :done
      new_look_queue.enq Look.new(ref: look["look_ref"])
    end
  end
end

处理new_look_queue同上。