vb.net - 非阻塞网络请求 vb.net

Question

我正在编写一个程序，必须尽快处理大约 5000 个字符串。其中大约 2000 个字符串必须通过 web 请求翻译到 mymemory.translated.net。 (参见下面的代码，JSON 部分已删除，因为此处不需要)

Try

          url = "http://api.mymemory.translated.net/get?q=" & Firstpart & "!&langpair=de|it&<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="a5c1c098d6cac8c0c8c4ccc9e5c6cdd7ccd6d1c8c4d68bc6cac8" rel="noreferrer noopener nofollow">[email protected]</a>"

          request = DirectCast(WebRequest.Create(url), HttpWebRequest)
          response = DirectCast(request.GetResponse(), HttpWebResponse)
          myreader = New StreamReader(response.GetResponseStream())

          Dim rawresp As String
          rawresp = myreader.ReadToEnd()
          Debug.WriteLine("Raw:" & rawresp)


          Catch ex As Exception
              MessageBox.Show(ex.ToString)

          End Try

代码本身工作正常，问题是它是一个阻塞代码，每个字符串需要大约 1 秒。我所有的琴弦都花了半个多小时。我需要将此代码转换为非阻塞代码并同时进行多个调用。有人可以告诉我该怎么做吗？我正在考虑一个后台 worker ，但这不会加快速度..它只会在不同的线程上执行代码...

谢谢!

Answer 1

问题在于您不仅仅受到最大并发操作数的阻碍。 HttpWebRequests 本质上是受到限制的(我相信默认策略在任何给定时间只允许 2 个请求)，因此您也必须覆盖该行为。请引用下面的代码。

Imports System.Diagnostics
Imports System.IO
Imports System.Net
Imports System.Threading
Imports System.Threading.Tasks

Public Class Form1

  ''' <summary>
  ''' Test entry point.
  ''' </summary>
  Private Sub Form1_Load() Handles MyBase.Load
    ' Generate enough words for us to test thoroughput.
    Dim words = Enumerable.Range(1, 100) _
      .Select(Function(i) "Word" + i.ToString()) _
      .ToArray()

    ' Maximum theoretical number of concurrent requests.
    Dim maxDegreeOfParallelism = 24
    Dim sw = Stopwatch.StartNew()

    ' Capture information regarding current SynchronizationContext
    ' so that we can perform thread marshalling later on.
    Dim uiScheduler = TaskScheduler.FromCurrentSynchronizationContext()
    Dim uiFactory = New TaskFactory(uiScheduler)

    Dim transformTask = Task.Factory.StartNew(
      Sub()
        ' Apply the transformation in parallel.
        ' Parallel.ForEach implements clever load
        ' balancing, so, since each request won't
        ' be doing much CPU work, it will spawn
        ' many parallel streams - likely more than
        ' the number of CPUs available.
        Parallel.ForEach(words, New ParallelOptions With {.MaxDegreeOfParallelism = maxDegreeOfParallelism},
          Sub(word)
            ' We are running on a thread pool thread now.
            ' Be careful not to access any UI until we hit
            ' uiFactory.StartNew(...)

            ' Perform transformation.
            Dim url = "http://api.mymemory.translated.net/get?q=" & word & "!&langpair=de|it&<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="ed8988d09e828088808c8481ad8e859f849e99808c9ec38e8280" rel="noreferrer noopener nofollow">[email protected]</a>"
            Dim request = DirectCast(WebRequest.Create(url), HttpWebRequest)

            ' Note that unless you specify this explicitly,
            ' the framework will use the default and you
            ' will be limited to 2 parallel requests
            ' regardless of how many threads you spawn.
            request.ServicePoint.ConnectionLimit = maxDegreeOfParallelism

            Using response = DirectCast(request.GetResponse(), HttpWebResponse)
              Using myreader As New StreamReader(response.GetResponseStream())
                Dim rawresp = myreader.ReadToEnd()

                Debug.WriteLine("Raw:" & rawresp)

                ' Transform the raw response here.
                Dim processed = rawresp

                uiFactory.StartNew(
                  Sub()
                    ' This is running on the UI thread,
                    ' so we can access the controls,
                    ' i.e. add the processed result
                    ' to the data grid.
                    Me.Text = processed
                  End Sub, TaskCreationOptions.PreferFairness)
              End Using
            End Using
          End Sub)
      End Sub)

    transformTask.ContinueWith(
      Sub(t As Task)
        ' Always stop the stopwatch.
        sw.Stop()

        ' Again, we are back on the UI thread, so we
        ' could access UI controls if we needed to.
        If t.Status = TaskStatus.Faulted Then
          Debug.Print("The transformation errored: {0}", t.Exception)
        Else
          Debug.Print("Operation completed in {0} s.", sw.ElapsedMilliseconds / 1000)
        End If
      End Sub,
      uiScheduler)
  End Sub

End Class