concurrency - Clojure的pmap函数为URL提取操作生成了多少个线程？

Question

pmap函数的文档让我想知道对于通过Web提取XML提要的集合将有多高效。我不知道pmap会产生多少个并发的提取操作，最大值是多少。

Answer 1

如果查看源代码，则会看到:

> (use 'clojure.repl)
> (source pmap)
(defn pmap
  "Like map, except f is applied in parallel. Semi-lazy in that the
  parallel computation stays ahead of the consumption, but doesn't
  realize the entire result unless required. Only useful for
  computationally intensive functions where the time of f dominates
  the coordination overhead."
  {:added "1.0"}
  ([f coll]
   (let [n (+ 2 (.. Runtime getRuntime availableProcessors))
         rets (map #(future (f %)) coll)
         step (fn step [[x & xs :as vs] fs]
                (lazy-seq
                 (if-let [s (seq fs)]
                   (cons (deref x) (step xs (rest s)))
                   (map deref vs))))]
     (step rets (drop n rets))))
  ([f coll & colls]
   (let [step (fn step [cs]
                (lazy-seq
                 (let [ss (map seq cs)]
                   (when (every? identity ss)
                     (cons (map first ss) (step (map rest ss)))))))]
     (pmap #(apply f %) (step (cons coll colls))))))

(+ 2 (.. Runtime getRuntime availableProcessors))是那里的重要线索。 pmap将获取第一个 (+ 2 processors)工作，并通过 future异步运行它们。因此，如果您有2个核心，那么它将一次启动4个工作，试图保持领先地位，但最大值应为2 + n。
future最终使用代理I / O线程池，该池支持无限数量的线程。它会随着工作量的增长而增长，如果未使用线程，则会收缩。