ruby - 100% 无休止的 CPU 使用率在没有证书的情况下获取 SSL 页面

Question

在 Ruby 1.9.3 上使用 Mechanize 2.6.0 我正在尝试通过 HTTPS 从 Windows 7x64 获取网页。当我尝试 get() URL 时，CPU 使用率达到 100%，并且该方法永远不会返回:

require 'mechanize'
uri = "https://my.com/wiki/api.php?action=query&titles=US4&prop=info&format=xml"
agent = Mechanize.new
u,p   = %w[myusername mypassword]
agent.add_auth( uri, u, p )
agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE 
info = agent.get( uri )

当我中断它时，我得到这些堆栈跟踪(三个不同的运行):

>> info = agent.get( page_api )
IRB::Abort: abort then interrupt!
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:27:in `call'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:27:in `parse'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:716:in `response_authenticate'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:306:in `fetch'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize.rb:431:in `get'
        from (irb):10
        from C:/Ruby193/bin/irb:12:in `<main>'
>> info = agent.get( page_api )
IRB::Abort: abort then interrupt!
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:29:in `call'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:29:in `new'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:29:in `parse'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:716:in `response_authenticate'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:306:in `fetch'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize.rb:431:in `get'
        from (irb):11
        from C:/Ruby193/bin/irb:12:in `<main>'
>> info = agent.get( page_api )
IRB::Abort: abort then interrupt!
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:114:in `call'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:114:in `token'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/www_authenticate_parser.rb:31:in `parse'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:716:in `response_authenticate'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:306:in `fetch'
        from C:/Ruby193/lib/ruby/gems/1.9.1/gems/mechanize-2.6.0/lib/mechanize.rb:431:in `get'
        from (irb):12
        from C:/Ruby193/bin/irb:12:in `<main>'

如何解决此问题并通过 Windows 上的 Ruby 正确获取 HTTPS URL？ (如果有比 Mechanize 更好的解决方案——因为无论如何我只需要将页面源提供给 Nokogiri——我愿意完全不使用 Mechanize。)

另一个数据点:在 OS X 上尝试相同的代码会产生相同的结果。

---

这是页面的实际内容，使用下面我的解决方法答案中描述的替代获取方法:

p fetch_https_without_ssl_verification(uri, u, p)
#=> "\t\t       <?xml version=\"1.0\"?><api><query><normalized><n from=\"Devtools/UI_Composer/DesignSpec/US7294\" to=\"Devtools/UI Composer/DesignSpec/US7294\" /></normalized><pages><page ns=\"0\" title=\"Devtools/UI Composer/DesignSpec/US7294\" missing=\"\" /></pages></query></api>"

Answer 1

如果您只需要 URL 的内容(就像我一样)，那么使用 curl而不是 Mechanize 更容易，而且它有效:

def fetch_https_without_ssl_verification( uri, user=nil, pass=nil )
  `curl -s -k #{%Q{-u "#{user}#{":"<<pass if pass}"} if user} "#{uri}"`
end