高请求场景下的 Java 线程池与新线程

Question

我有一些用于 REST 服务的旧 Java 代码，它为每个传入请求使用单独的线程。 IE。主循环将在 socket.accept() 上循环并将套接字移交给 Runnable，然后 Runnable 将启动自己的后台线程并调用自身运行。这在一段时间内非常有效，直到最近我注意到在高负载下接受处理请求的延迟将变得 Not Acceptable 。当我说非常好时，我的意思是它每秒处理 100-200 个请求而没有显着的 CPU 使用率。只有当其他守护进程也增加负载时性能才会下降，然后只有一次负载超过 5。当机器处于其他进程组合的高负载(5-8)时，从接受到处理的时间会变得非常长( 500 毫秒到 3000 毫秒)，而实际处理时间保持在 10 毫秒以下。这都是在双核 centos 5 系统上。

我已经习惯了 .NET 上的线程池，我认为线程创建是罪魁祸首，我认为我会在 Java 中应用相同的模式。现在我的 Runnable 是用 ThreadPool.Executor 执行的(并且池使用 ArrayBlockingQueue)。同样，它在大多数情况下都能很好地工作，除非机器负载变高，然后从创建可运行对象到调用 run() 的时间显示出大致相同的荒谬时间。但更糟糕的是，在线程池逻辑就位的情况下，系统负载几乎翻了一番 (10-16)。所以现在我遇到了双倍负载的相同延迟问题。

我的怀疑是队列的锁争用比之前没有锁的新线程启动成本要差。任何人都可以分享他们对新线程与线程池的经验。如果我的怀疑是正确的，那么有人有另一种方法来处理没有锁争用的线程池吗？

我很想让整个系统成为单线程，因为我不知道我的线程有多大帮助，而且 IO 似乎不是问题，但我确实收到了一些长期存在的请求那会阻止一切。

谢谢，阿恩

更新:我切换到 Executors.newFixedThreadPool(100);，虽然它保持了相同的处理能力，但负载几乎立即翻了一番，并且运行 12 小时显示负载始终保持在 2 倍。我想在我的情况下，每个请求一个新线程更便宜。

Answer 1

配置为:

new ThreadPoolExecutor(10, 100, 30, TimeUnit.SECONDS, 
        new ArrayBlockingQueue<Runnable>(100))

然后一旦有10个线程在并发处理请求，就会有更多的请求加入到队列中，除非队列中达到100个请求，此时会开始创建新的线程，除非已经有100个线程，当处理该命令将被拒绝。

javadocs of ThreadPoolExecutor的部分(复制在下面)可能值得再读一读。

基于它们，以及您明显愿意运行 100 个线程的意愿，以及您接受所有请求并最终处理它们的愿望。我建议尝试以下变体:

new ThreadPoolExecutor(100, 100, 0, TimeUnit.SECONDS, 
        new LinkedBlockingQueue<Runnable>())

顺便说一句，这就是您从 Executors.newFixedThreadPool(100);

获得的内容

---

Queuing

Any BlockingQueue may be used to transfer and hold submitted tasks. The use of this queue interacts with pool sizing:

• If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
• If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
• If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.

There are three general strategies for queuing:

1. Direct handoffs. A good default choice for a work queue is a SynchronousQueue that hands off tasks to threads without otherwise holding them. Here, an attempt to queue a task will fail if no threads are immediately available to run it, so a new thread will be constructed. This policy avoids lockups when handling sets of requests that might have internal dependencies. Direct handoffs generally require unbounded maximumPoolSizes to avoid rejection of new submitted tasks. This in turn admits the possibility of unbounded thread growth when commands continue to arrive on average faster than they can be processed.
2. Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each task is completely independent of others, so tasks cannot affect each others execution; for example, in a web page server. While this style of queuing can be useful in smoothing out transient bursts of requests, it admits the possibility of unbounded work queue growth when commands continue to arrive on average faster than they can be processed.
3. Bounded queues. A bounded queue (for example, an ArrayBlockingQueue) helps prevent resource exhaustion when used with finite maximumPoolSizes, but can be more difficult to tune and control. Queue sizes and maximum pool sizes may be traded off for each other: Using large queues and small pools minimizes CPU usage, OS resources, and context-switching overhead, but can lead to artificially low throughput. If tasks frequently block (for example if they are I/O bound), a system may be able to schedule time for more threads than you otherwise allow. Use of small queues generally requires larger pool sizes, which keeps CPUs busier but may encounter unacceptable scheduling overhead, which also decreases throughput.