c# - 网络请求数量有限

Question

使用以下代码:

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var client = new HttpClient();
    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    foreach(var uri in uris)
    {
        var url = uri.ToString();

        var task = client.GetStringAsync(uri);
        task.ContinueWith(t => Console.WriteLine("Done {0}", url), TaskContinuationOptions.OnlyOnRanToCompletion);
        task.ContinueWith(t => Console.WriteLine("Failed {0}", url), TaskContinuationOptions.OnlyOnFaulted);
        task.ContinueWith(t => Console.WriteLine("Cancelled {0}", url), TaskContinuationOptions.OnlyOnCanceled);
    }

    Console.ReadKey();
}

根据 fiddler 的说法，我最多可以同时请求 15-20 个 url。所有这些 url 都是唯一的，并且不指向同一个主机。

这是怎么回事？

Answer 1

运行此程序的机器上的 CPU 有多少个内核？您的机器可以处理的并发操作数是有限制的。此外，TPL 会根据手头的任务自动决定要调用的并行度的正确数量。启动 1000 个线程来完成一项任务并不总是更有效。有大量无意中听到管理线程之间传递的消息。

这可能没有任何性能改进，但这应该更符合并行性:

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    foreach(var uri in uris)
    {
        var client = new HttpClient();
        var url = uri.ToString();

        var task = client.GetStringAsync(uri);
        task.ContinueWith(t => Console.WriteLine("Done {0}", url), TaskContinuationOptions.OnlyOnRanToCompletion);
        task.ContinueWith(t => Console.WriteLine("Failed {0}", url), TaskContinuationOptions.OnlyOnFaulted);
        task.ContinueWith(t => Console.WriteLine("Cancelled {0}", url), TaskContinuationOptions.OnlyOnCanceled);
    }

    Console.ReadKey();
}

甚至:

static void Main(string[] args)
{
    ServicePointManager.DefaultConnectionLimit = 1000;

    var uris = File.ReadAllLines(@"C:\urls.txt").Select(x => new Uri(x));

    Parallel.ForEach(uris, uri => {
        WebRequest myRequest = WebRequest.Create(uri.toString());
        // handle response synchronously
    });

}