python - CloudFlare 抓取 403 禁止

Question

我在尝试向 https://www.off---white.com/en/IT/orders/populate.json 发送发布请求时收到“403 禁止”错误为了将以下页面上的商品添加到购物车:https://www.off---white.com/en/IT/women/products/owea143e18a640731001 。

这是我的代码:

import cfscrape
scrape = cfscrape.create_scraper()
payload = {"variant_id": "107917", "quantity": "1"}
headers={
    "accept":"application/json, text/javascript, */*; q=0.01",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9",
    "content-length": "35",
    "content-type": "application/json; charset=UTF-8",
    "refer": "https://www.off---white.com/en/IT/women/products/owea143e18a640731001",
    "origin":"https://www.off---white.com",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36",
    "x-requested-with": "XMLHttpRequest"
    }

res = scrape.post("https://www.off---white.com/en/IT/orders/populate.json",headers=headers,data=payload)
print(res.status_code)  #=403 :(

谢谢。

编辑:显然我需要cookie。我按如下方式更改了代码，但仍然收到 403 禁止。

cookie, user_agent=scrape.get_cookie_string("https://www.off---white.com/en/IT/women/products/owea143e18a640731001")
headers={
    "authority":"www.off---white.com",
    "accept":"application/json, text/javascript, */*; q=0.01",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9",
    "content-length": "35",
    "content-type": "application/json; charset=UTF-8",
    "refer": "https://www.off---white.com/en/IT/women/products/owea143e18a640731001",
    "origin":"https://www.off---white.com",
    "cookie": cookie,
    "user-agent": user_agent,
    "x-requested-with": "XMLHttpRequest"
    }
res = scrape.post("https://www.off---white.com/en/IT/orders/populate.json",headers=headers,data=payload)

关于还缺少什么的一些提示？

EDIT2:我也尝试过以下操作:

tokens, user_agent=scrape.get_tokens("https://www.off---white.com/en/IT/women/products/owea143e18a640731001")
headers={
    "authority":"www.off---white.com",
    "accept":"application/json, text/javascript, */*; q=0.01",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9",
    "content-length": "35",
    "content-type": "application/json; charset=UTF-8",
    "refer": "https://www.off---white.com/en/IT/women/products/owea143e18a640731001",
    "origin":"https://www.off---white.com",
    "user-agent": user_agent,
    "x-requested-with": "XMLHttpRequest"
    }
res = scrape.post("https://www.off---white.com/en/IT/orders/populate.json",headers=headers,data=payload,cookies=tokens)

再次，没有运气。另外，无论我如何提交 cookie，我注意到每当我拥有它们时，请求都会立即被拒绝，cfscrape 的 node.js 窗口甚至不会打开。

Answer 1

您确定内容长度正确吗？这可能是一个问题。通常您不需要包含此内容，它会在稍后自行添加。另一个问题很可能是cookie。我没有看到您通过 POST 请求发回 cookie。有 2 个必需的 cookie，其中一个是您未发送的 _hs_session_v3。

编辑:在浏览器中执行此操作: 1. 向 off---white.com 发出 GET 请求。保存所有这些cookie。 2.来自off---white.com/en/IT/women/products/…的GET请求。保存这些cookie。 3. POST 请求将起作用。要对此进行测试，请执行以下操作: 清除所有浏览器缓存/cookie。转到off---white.com/en/IT/women/products/…，它甚至没有加载页面..为什么不呢？您的 cookie 尚未设置。该网站正在使用您的 cookie 来确定您的请求是否有效。您正在请求一个依赖 cookie 来获得有效响应的页面。

当我点击添加到包时，这是curl:curl ' https://www.off---white.com/en/IT/orders/populate.json ' -H 'cookie: __cfduid=db351a20a2752d5459220f241c66440371539900632; miss_cookie_law=true； guest_token=Im41TjNZWFBFRFFwaUdvNkpTeVJDZnci--1fca5ea27af50cba12cd96f088975f5f5e0647ac； __riskifiedBeaconSessionId=c0f01056-f52a4b5a-76f25d73-cdb11d12-c2f20dbc-8173a1e0; cf_间隙=eb87f25251aeab5062f65bc7a5261b9cbef35809-1539947407-1800-150; _hs_session_v3=UHRUWUtUVDJKTTExeEkxTVNQVlFMSVQySUR1VXlZZFJGaGZJck0zOTBPbVZSWjUvL01pR2J1UktjOXFPMTZiK2pVbm5jVGtkUU9GRVpJVGR0VWkxZGUyWGNvak5kZXllR2tEMG5y OERHMmVMaHJxaUxYRHRGVUp3NXAvZERGSm9nREZBc0Z0bDJsM241VHhzdU16S05WT3BUaFlNbVhwYkd3cGNjdHAzSHpRPS0tZ2pOR25tNzJMTWlTY3U1MDJTaW9iQT09--44bbe09a45974b9e 7584588b186650946c1b6f73'-H'来源:https://www.off---white.com ' -H '接受编码:gzip、deflate、br' -H '接受语言:en-US,en;q=0.9' -H 'x-requested-with: XMLHttpRequest' -H 'pragma: no-cache ' -H '用户代理:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML，如 Gecko) Chrome/69.0.3497.100 Safari/537.36' -H '内容类型:application/json; charset=UTF-8' -H '接受:application/json、text/javascript、/； q=0.01' -H '缓存控制:无缓存' -H '权限:www.off---white.com' -H '引用者:https://www.off---white.com/en/IT/women/products/owea143e18a640731001 ' -H 'dnt: 1' --data-binary '{"variant_id":null, "quantity":1}' --compressed