ruby-on-rails - 使用 ruby​​ 2.5.3 在特定站点上发出获取请求时出现超时错误

Question

任何人都可以从网站 www.coupang.com 得到正确的回应吗？我不断向“https://www.coupang.com/”发出请求，但 10 次中有 9 次出现错误。(有时它有效!令人惊讶。)

Traceback (most recent call last):
        14: from lib/add_sup/test.rb:7:in `<main>'
        13: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:485:in `get_response'
        12: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:609:in `start'
        11: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:910:in `start'
        10: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:487:in `block in get_response'
         9: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:1365:in `request_get'
         8: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:1464:in `request'
         7: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:1491:in `transport_request'
         6: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:1491:in `catch'
         5: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http.rb:1494:in `block in transport_request'
         4: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http/response.rb:29:in `read_new'
         3: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/http/response.rb:40:in `read_status_line'
         2: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/protocol.rb:167:in `readline'
         1: from /Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/protocol.rb:157:in `readuntil'
/Users/j/.rbenv/versions/2.5.3/lib/ruby/2.5.0/net/protocol.rb:181:in `rbuf_fill': Net::ReadTimeout (Net::ReadTimeout)

我也尝试过使用 python3 发出请求并且它工作正常。我认为我的 ruby​​ 配置或 ruby​​ 本身有问题。

require 'nokogiri'
require 'open-uri'
require 'net/http'


uri = URI("https://www.coupang.com/")
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)

我将感谢您对此事的友好想法。谢谢!

Answer 1

他们使用的是 akamai，所以首先 - 他们期望 HTTP/2(你需要一个 http2 gem)并且他们有一些相当严格的用户代理嗅探。

这是一个使用 net-http2 的例子

client = NetHttp2::Client.new "https://www.coupang.com/"
res = client.call :get, '/', headers: { "User-Agent" => "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6)" }
puts res.body if res.ok?