python 请求 - 使用 'idna' 编解码器编码失败(UnicodeError : label empty or too long) error

Question

我一直在使用 requests 包的 api 调用突然返回以下错误:“UnicodeError:使用‘idna’编解码器编码失败(UnicodeError:标签为空或太长)”

我不知道如何解决这个问题。我的代码如下所示，为此示例伪造了某些凭据:

api_key= '123abc'
password = '12345' #password that only idiots use on their luggage
shop_name = 'myshopname'
shop_url = 'https://%s:%s@%s.myecommercesite.com/admin/customers/1234567.json' %(api_key, password, shop_name)

a = requests.get(shop_url)

当我打印 shop_url 并将其粘贴到我的浏览器中时，我得到了返回的数据，这是我在 json 中期望的。但是当我运行这个请求时，我得到了 idna 编解码器错误。

这曾经没有问题，但显然某处发生了变化，我不确定它是与电子商务网站有关还是与请求有关，或者是什么导致了这种情况。

有没有人遇到过这种类型的错误或知道如何修复它？

如果我打印 url，它看起来像:https://123abc:12345@myshopname.myecommercesite.com/admin/customers/1234567.json

编辑2:忘记在我的代码示例中包含 %(api_key, password, shop_name)编辑:以下是完整的错误消息:

UnicodeError                              Traceback (most recent call last)
~/anaconda3/lib/python3.6/encodings/idna.py in encode(self, input, errors)
    164                 if not (0 < len(label) < 64):
--> 165                     raise UnicodeError("label empty or too long")
    166             if len(labels[-1]) >= 64:

UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

UnicodeError                              Traceback (most recent call last)
<ipython-input-15-f834b116b751> in <module>()
----> 1 a = requests.get(shop_url)

~/anaconda3/lib/python3.6/site-packages/requests/api.py in get(url, params, **kwargs)
     70 
     71     kwargs.setdefault('allow_redirects', True)
---> 72     return request('get', url, params=params, **kwargs)
     73 
     74 

~/anaconda3/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     56     # cases, and look like a memory leak in others.
     57     with sessions.Session() as session:
---> 58         return session.request(method=method, url=url, **kwargs)
     59 
     60 

~/anaconda3/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    497 
    498         settings = self.merge_environment_settings(
--> 499             prep.url, proxies, stream, verify, cert
    500         )
    501 

~/anaconda3/lib/python3.6/site-packages/requests/sessions.py in merge_environment_settings(self, url, proxies, stream, verify, cert)
    670             # Set environment's proxies.
    671             no_proxy = proxies.get('no_proxy') if proxies is not None else None
--> 672             env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
    673             for (k, v) in env_proxies.items():
    674                 proxies.setdefault(k, v)

~/anaconda3/lib/python3.6/site-packages/requests/utils.py in get_environ_proxies(url, no_proxy)
    690     :rtype: dict
    691     """
--> 692     if should_bypass_proxies(url, no_proxy=no_proxy):
    693         return {}
    694     else:

~/anaconda3/lib/python3.6/site-packages/requests/utils.py in should_bypass_proxies(url, no_proxy)
    674     with set_environ('no_proxy', no_proxy_arg):
    675         try:
--> 676             bypass = proxy_bypass(netloc)
    677         except (TypeError, socket.gaierror):
    678             bypass = False

~/anaconda3/lib/python3.6/urllib/request.py in proxy_bypass(host)
   2610             return proxy_bypass_environment(host, proxies)
   2611         else:
-> 2612             return proxy_bypass_macosx_sysconf(host)
   2613 
   2614     def getproxies():

~/anaconda3/lib/python3.6/urllib/request.py in proxy_bypass_macosx_sysconf(host)
   2587     def proxy_bypass_macosx_sysconf(host):
   2588         proxy_settings = _get_proxy_settings()
-> 2589         return _proxy_bypass_macosx_sysconf(host, proxy_settings)
   2590 
   2591     def getproxies_macosx_sysconf():

~/anaconda3/lib/python3.6/urllib/request.py in _proxy_bypass_macosx_sysconf(host, proxy_settings)
   2560             if hostIP is None:
   2561                 try:
-> 2562                     hostIP = socket.gethostbyname(hostonly)
   2563                     hostIP = ip2num(hostIP)
   2564                 except OSError:

UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)

Answer 1

这似乎是 socket 模块的问题。当 URL 超过 64 个字符时，它将失败。这仍然是一个悬而未决的问题 https://bugs.python.org/issue32958

The error can be consistently reproduced when the first substring of the url hostname is greater than 64 characters long, as in "0123456789012345678901234567890123456789012345678901234567890123.example.com". This wouldn't be a problem, except that it doesn't seem to separate out credentials from the first substring of the hostname so the entire "[user]:[secret]@XXX" section must be less than 65 characters long. This is problematic for services that use longer API keys and expect their submission over basic auth.

还有一个替代方案:

您似乎正在尝试使用 Shopify API，所以我将以它为例。

在 base64 中对 {api_key}:{password} 进行编码，并将此值发送到请求的 header 中，例如。 {'Authorization': 'Basic {token_base_64}'}

请看下面的例子:

import base64
import requests

auth = "[API KEY]:[PASSWORD]"
b64_auth = base64.b64encode(auth.encode()).decode("utf-8")

headers = {
    "Authorization": f"Basic {b64_auth}"
}

response = requests.get(
    url="https://[YOUR-SHOP].myshopify.com/admin/[ENDPOINT]",
    headers=headers
)