python - 为什么python脚本无法通过代理下载网页

Question

我是 python 的新手，正在尝试使用套接字。所以我写了一个简单的 http 客户端，但令我惊讶的是它无法访问 firefox 可以访问的网页，但它们使用相同的 header

import socket
clientsocket= socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect(("213.229.83.205",80))#connect to proxy at given address
print "connected to 213.229.83.205"
sdata= """GET http://google.co.ug/ HTTP/1.1
Host: google.co.ug
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Proxy-Connection: keep-alive
Cookie: cookie <-- Real cookie deleted

"""
print "sending request"
clientsocket.send(sdata);
rdata=clientsocket.recv(10240)
if not rdata: print "no data found"
else:
    print "receiving data !"
    myfile=open("c:/users/markdenis/desktop/google.html","w")
    myfile.write(str(rdata))
    myfile.close()
    print "data written to file on desktop"
clientsocket.close()
raw_input()#system(pause)

当我运行它时，它显示:

connected to 213.229.83.205
sending request
no data found

Answer 1

HTTP 协议(protocol)要求在每个 header 的末尾使用 \r\n 并在 HTTP header 末尾的空行中添加额外的内容。您没有明确说明 sdata 缓冲区中的行结尾，因此您的缓冲区仅以 \n 行结尾结尾。

在 Windows、Linux 和 OS X 上测试，可以肯定:

>>> x = """a
b
c"""
>>> x
'a\\nb\\nc\\n'

你需要的地方:

>>> x = "a\r\nb\r\nc\r\n"
>>> x
'a\\r\\nb\\r\\nc\\r\\n'

添加 \r\n 并试一试。直接在缓冲区中执行它会得到一组额外的 \n，所以将其拆分:

sdata = "GET http://google.co.ug/ HTTP/1.1\r\n"
sdata += "Host: google.co.ug\r\n"
sdata += "User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0\r\n"
sdata += "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"
sdata += "Accept-Language: en-us,en;q=0.5\r\n"
sdata += "Accept-Encoding: gzip, deflate\r\n"
sdata += "Proxy-Connection: keep-alive\r\n"
sdata += "\r\n"