发送 POST 请求时 Ruby 的 Mechanize 错误 401(Steam 交易报价发送)

Question

我正在尝试使用 mechanize 发送 Steam 交易报价，我进行了登录以获取所需的 cookie，但是当我尝试发送 Steam 交易报价时，我收到错误 401 Unauthorized。

我从 python 移植了 this code 唯一的区别是，据我所知，与 ruby​​ 的 Mechanize 相比，python 的请求库如何处理 POST 请求中的 cookie，您可以验证我在登录请求中获取了所有 cookie通过输出 Mechanize cookie 并根据 this 我拥有所有必要的 cookie

这是我的代码，您可以复制粘贴它并执行它，唯一的问题是最后几行。

require 'mechanize'
require 'json'
require 'open-uri'
require 'openssl'
require 'base64'
require 'time'

def fa(shared_secret)
      timestamp = Time.new.to_i
      math = timestamp / 30
      math = math.to_i
      time_buffer =[math].pack('Q>')

      hmac = OpenSSL::HMAC.digest('sha1', Base64.decode64(shared_secret), time_buffer)

      start = hmac[19].ord & 0xf
      last = start + 4
      pre = hmac[start..last]
      fullcode = pre.unpack('I>')[0] & 0x7fffffff

      chars = '23456789BCDFGHJKMNPQRTVWXY'
      code= ''
      for looper in 0..4 do
        copy = fullcode #divmod
        i = copy % chars.length #divmod
        fullcode = copy / chars.length #divmod
        code = code + chars[i]
      end
      puts code
      return code

end

def pass_stamp(username,password,mech)
      response = mech.post('https://store.steampowered.com/login/getrsakey/', {'username' => username})

      data = JSON::parse(response.body)
      mod = data["publickey_mod"].hex
      exp = data["publickey_exp"].hex
      timestamp = data["timestamp"]

      key   = OpenSSL::PKey::RSA.new
      key.e = OpenSSL::BN.new(exp)
      key.n = OpenSSL::BN.new(mod)
      ep = Base64.encode64(key.public_encrypt(password.force_encoding("utf-8"))).gsub("\n", '')
      return {'password' => ep, 'timestamp' => timestamp }
end

user = 'user'
password = 'password'

session = Mechanize.new { |agent|
  agent.user_agent_alias = 'Windows Mozilla'
  agent.follow_meta_refresh = true
  agent.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, password)
  agent.log = Logger.new("mech.log")
}

data = pass_stamp(user,password, session)
ep = data["password"]
timestamp = data["timestamp"]
session.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user,  ep)

send = {
      'password' => ep,
      'username' => user,
      'twofactorcode' =>fa('twofactorcode'), #update
      'emailauth' => '',
      'loginfriendlyname' => '',
      'captchagid' => '-1',
      'captcha_text' => '',
      'emailsteamid' => '',
      'rsatimestamp' => timestamp,
      'remember_login' => 'false'
}

login = session.post('https://store.steampowered.com/login/dologin', send )
responsejson = JSON::parse(login.body)
if responsejson["success"] != true
      puts "didn't sucded"
      puts "probably 2fa code time diffrence,  retry "
      exit
end

responsejson["transfer_urls"].each { |url|
      getcookies = session.post(url, responsejson["transfer_parameters"])
}

session.get("https://steamcommunity.com/") do |page| ## to verify that you are logged in check this HTML
     File.open('./body.html', 'w') {|f| f.puts page.content}
end

sessionid = ''
session.cookies.each { |c|
      string = c.dup.to_s
      if string.include?('sessionid')
            sessionid = string.gsub('sessionid=', '')
      end
}

offer_link = 'https://steamcommunity.com/tradeoffer/new/?partner=410155236&token=H-yK-GFt'
token = offer_link.split('token=', 2)[1]
theirs = [{"appid" => 753,"contextid"=> "6","assetid" => "6705710171","amount" => 1 }]
mine =  []
params = {
      'sessionid' => sessionid,
      'serverid' => 1,
      'partner' => '76561198370420964',
      'tradeoffermessage' => '',
      'json_tradeoffer' => {
            "new_version" => true,
           "version" => 4,
           "me" => {
                "assets" => mine, #create this array
                "currency" => [],
                "ready" => false
           },
           "them" => {
           "assets" => theirs, #create this array
           "currency" => [],
           "ready" => false
            }
      },
      'captcha' => '',
      'trade_offer_create_params' => {'trade_offer_access_token' => token}
}
#the issue begins from here
begin
      send_offer = session.post(
        'http://steamcommunity.com/tradeoffer/new/send/',
        params,
        {'Referer' =>  "#{offer_link}", 'Origin' => 'https://steamcommunity.com/tradeoffer/new/send' }
      )
      puts send_offer.body
rescue Mechanize::UnauthorizedError => e
      puts e
      puts e.page.content
end

Answer 1

我通过调试 python POST 请求发现了这个问题。
发生了什么:当我登录时，我确实得到了 sessionid，但是 sessionid 对“store.steampowered.com”和“help.steampowered.com”有效，恰好是“.storesteapowered.com”。
在我的代码中，我盲目地识别了我的 session cookie(没有注意它属于哪个网站)，因此在 POST 请求参数中发送的 sessionid 变量不等于 POST 请求正在发送的 cookie标题所以我得到了 401 Unauthorized。
所以我们需要为 steamcommunity.com 设置/获取 session ID。
修复:
1)为 steamcommunity.com 设置一个随机的 CSRF sessionid cookie，或者像我一样，将 steampowered.com 的 session id cookie 设置为 steamcommunity.com(在代码中标记)
2) 在 params => 'json_tradeoffer' => "new_version" 中应该是 "newversion" 以避免错误 400 BAD REQUEST
3)post请求的头部应该是:

{'Referer' =>'https://steamcommunity.com/tradeoffer/new', 'Origin' =>'https://steamcommunity.com' }

4) 使用 to_json 将 params => json_tradeoffer 和 params => 'trade_offer_create_params' 值转换为字符串
重要 :此代码用于 1 个报价发送，如果您要发送 1 个以上的报价，您 必须 始终更新您的 sessionid 变量，因为每次与 steamcommunity.com 通信时，cookie 值都会改变
这是修复的代码:

require 'mechanize'
require 'json'
require 'open-uri'
require 'openssl'
require 'base64'
require 'time'

def fa(shared_secret)
      timestamp = Time.new.to_i
      math = timestamp / 30
      math = math.to_i
      time_buffer =[math].pack('Q>')

      hmac = OpenSSL::HMAC.digest('sha1', Base64.decode64(shared_secret), time_buffer)

      start = hmac[19].ord & 0xf
      last = start + 4
      pre = hmac[start..last]
      fullcode = pre.unpack('I>')[0] & 0x7fffffff

      chars = '23456789BCDFGHJKMNPQRTVWXY'
      code= ''
      for looper in 0..4 do
        copy = fullcode #divmod
        i = copy % chars.length #divmod
        fullcode = copy / chars.length #divmod
        code = code + chars[i]
      end
      puts code
      return code

end

def pass_stamp(username,password,mech)
      response = mech.post('https://store.steampowered.com/login/getrsakey/', {'username' => username})

      data = JSON::parse(response.body)
      mod = data["publickey_mod"].hex
      exp = data["publickey_exp"].hex
      timestamp = data["timestamp"]

      key   = OpenSSL::PKey::RSA.new
      key.e = OpenSSL::BN.new(exp)
      key.n = OpenSSL::BN.new(mod)
      ep = Base64.encode64(key.public_encrypt(password.force_encoding("utf-8"))).gsub("\n", '')
      return {'password' => ep, 'timestamp' => timestamp }
end

user = 'user'
password = 'password'

session = Mechanize.new { |agent|
  agent.user_agent_alias = 'Windows Mozilla'
  agent.follow_meta_refresh = true
  agent.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, password)
  agent.log = Logger.new("mech.log")
}

data = pass_stamp(user,password, session)
ep = data["password"]
timestamp = data["timestamp"]
session.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user,  ep)

send = {
      'password' => ep,
      'username' => user,
      'twofactorcode' =>fa('twofactorcode'), #update
      'emailauth' => '',
      'loginfriendlyname' => '',
      'captchagid' => '-1',
      'captcha_text' => '',
      'emailsteamid' => '',
      'rsatimestamp' => timestamp,
      'remember_login' => 'false'
}

login = session.post('https://store.steampowered.com/login/dologin', send )
responsejson = JSON::parse(login.body)
if responsejson["success"] != true
      puts "didn't sucded"
      puts "probably 2fa code time diffrence,  retry "
      exit
end

responsejson["transfer_urls"].each { |url|
      getcookies = session.post(url, responsejson["transfer_parameters"])
}


## SET COOKIE FOR STEAM COMMUNITY.COM
steampowered_sessionid = ''
session.cookies.each { |c|
      if c.name == "sessionid"
            steampowered_sessionid = c.value
            puts c.domain
      end
}
cookie = Mechanize::Cookie.new :domain => 'steamcommunity.com', :name =>'sessionid', :value =>steampowered_sessionid, :path => '/'
session.cookie_jar << cookie
sessionid = steampowered_sessionid
### END SET COOKIE
offer_link = 'https://steamcommunity.com/tradeoffer/new/?partner=410155236&token=H-yK-GFt'
token = offer_link.split('token=', 2)[1]
theirs = [{"appid" => 753,"contextid"=> "6","assetid" => "6705710171","amount" => 1 }]
mine =  []
params = {
      'sessionid' => sessionid,
      'serverid' => 1,
      'partner' => '76561198370420964',
      'tradeoffermessage' => '',
      'json_tradeoffer' => {
            "newversion" => true, ## FIXED newversion to avoid 400 BAD REQUEST
           "version" => 4,
           "me" => {
                "assets" => mine, #create this array
                "currency" => [],
                "ready" => false
           },
           "them" => {
                 "assets" => theirs, #create this array
                 "currency" => [],
                 "ready" => false
            }
      }.to_json, # ADDED TO JSON TO AVOID 400 BAD REQUEST
      'captcha' => '',
      'trade_offer_create_params' => {'trade_offer_access_token' => token}.to_json ## ADDED TO JSON FIX TO AVOID ERROR 400 BAD REQUEST
}

begin
      send_offer = session.post(
       'https://steamcommunity.com/tradeoffer/new/send',
        params,
        {'Referer' =>  'https://steamcommunity.com/tradeoffer/new', 'Origin' => 'https://steamcommunity.com' } ##FIXED THIS
      )
      puts send_offer.body
rescue Mechanize::UnauthorizedError => e
      puts e
      puts e.page.content
end