gpt4 book ai didi

C# Parallel WebClient - 操作超时

转载 作者:太空宇宙 更新时间:2023-11-03 13:34:21 25 4
gpt4 key购买 nike

我是新手,想知道为什么我在 parallel 中的 webclient downloadstring() 遇到错误。我不知道是不是因为我的连接速度慢。这是我的代码:

for (int i = 2; i <= 5; i++)
{
string ebayLink = "http://www.ebay.de/sch/Studium-Wissen-/1105/i.html?LH_Auction=1&_sop=1&_nkw=&_pgn=" + i;
//string ebayLink = "http://www.ebay.de/sch/Schule-Ausbildung-/40434/i.html?LH_Auction=1&_sop=1&_nkw=&_pgn=" + i;
ebayLink = "http://www.ebay.de/sch/i.html?LH_Auction=1&_sacat=0&_from=R40&_nkw=B%C3%BCcher&_sop=1&_pgn=" + i;

HtmlWeb hw = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = hw.Load(ebayLink);


List<string> eanList = new List<string>();

List<string> links = new List<string>();

foreach (var link in doc.DocumentNode.SelectNodes("//a[@href]"))
{
string url = link.GetAttributeValue("href", "");
if (url.Contains(".de/itm") && !links.Contains(url) && !url.Contains("pt=Zeitschriften") && !url.Contains("pt=Belletristik"))
{
links.Add(url);
}
}

Parallel.ForEach(links, link =>
{
WebClient wc = new WebClient();
string html = wc.DownloadString(link);

EbayItem ebayItem = new EbayItem(html);

string ean = ebayItem.ean;


string amazonUsedPrice = string.Empty;

amazonUsedPrice = getAmazonUsedPrice(ean);

Product product = new Product();
product.EbayUrl = link;
product.Ean = ean;
product.AmazonPriceString = amazonUsedPrice;
product.ebayItem = ebayItem;
productList.Add(product);


}
);}

错误发生在string html = wc.DownloadString(link);。我在输出中看到它在达到至少 20 个链接时停止。

最佳答案

您的连接正在等待先前的连接关闭,因此超时。到同一主机的并发连接的默认限制是 2。在进入 Parallel 调用之前尝试增加该限制:

System.Net.ServicePointManager.DefaultConnectionLimit = int.MaxValue;

阅读有关 DefaultConnectionLimit 的更多信息 here .

Property Value

Type: System.Int32

The maximum number of concurrent connections allowed by a ServicePoint object. The default value is 2.

关于C# Parallel WebClient - 操作超时,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/19166863/

25 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com