django - Django Elasticsearch AWS httplib UnicodeDecodeError

Question

我正在尝试使用Django(没有Haystack)设置Elasticsearch。
一切都在本地完美运行。
但是，当我尝试在基于AWS的IAM身份验证中使用elasticsearch-py 客户端时，出现此错误:

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 827, in _send_output
msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position132: ordinal not in range(128)

我尝试使用 aws-es-connection和 requests-aws4auth，但收到相同的错误。

仅当我允许在AWS上进行开放访问并且仅使用elasticsearch-py时，它才起作用

from elasticsearch import Elasticsearch, RequestsHttpConnection
ES_CLIENT = Elasticsearch(
    ['search-domain-xxx.us-east-1.es.amazonaws.com'],
connection_class=RequestsHttpConnection
)

但是我想要更安全的东西...
我认为这是utf-8 / unicode / str问题，但我无法解决它:(

Answer 1

我终于设法通过使用自定义序列化程序来修复该错误:

from elasticsearch import Elasticsearch, RequestsHttpConnection, serializer, compat, exceptions

class JSONSerializerPython2(serializer.JSONSerializer):
"""Override elasticsearch library serializer to ensure it encodes utf characters during json dump.
See original at: https://github.com/elastic/elasticsearch-py/blob/master/elasticsearch/serializer.py#L42
A description of how ensure_ascii encodes unicode characters to ensure they can be sent across the wire
as ascii can be found here: https://docs.python.org/2/library/json.html#basic-usage
"""
def dumps(self, data):
    # don't serialize strings
    if isinstance(data, compat.string_types):
        return data
    try:
        return json.dumps(data, default=self.default, ensure_ascii=True)
    except (ValueError, TypeError) as e:
        raise exceptions.SerializationError(data, e)

然后将序列化器传递给elasticsearch

from elasticsearch import Elasticsearch
es = Elasticsearch(..., serializer=JSONSerializerPython2())

我找到了解决方案 here