python - 在 Ubuntu Server 16.04LTS 上将 JSON 写入文件导致 Unicode 错误

Question

我确信我遇到的问题与操作系统或版本没有直接关系，而是与某种设置有关。在 python django 应用程序中，我将 JSON 写入一个文件，该文件可以包含其他语言的字符，例如 我、ל 和 は。据我所知，在数据流的任何时间点我都没有更改编码。

在本地开发期间，这不是问题。

    with open(self._json_path, 'w') as f:
        json.dump(test_dict, f, indent=2, ensure_ascii=False)
    answer, wordid, question = self._unpack_dict(test_dict)

一旦我部署到实时网络服务器，我就开始得到:

'ascii' codec can't encode character '\u90fd' in position 1: ordinal not in range(128)

我知道 test_dict 中的数据已正确编码。一旦 json.dump 发生，它就会出错。如果我打开创建的文件，它会在我放入其中的第一个非拉丁字符处失败。

我已经经历过this post ，但无法解决问题。添加 ,encoding='utf8' 会导致上述代码的输出创建一个文件，但在其中不放置任何内容。再次，我知道 test_dict 有数据，因为数据正确显示在网页上。**空白文件的答案:**在故障排除中，我从 dump 切换到 dumps，这导致生成文件但未填充文件。**主要问题的答案:**encoding='utf8' 不正确，是 encoding='utf-8'

我也尝试过重建虚拟环境。

在服务器上，这是一些结果:

echo $LANG
en_US.UTF-8
python -c "import sys; print(sys.stdout.encoding)"
UTF-8
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

本地环境:

echo $LANG
en_US.UTF-8
python -c "import sys; print(sys.stdout.encoding)"
UTF-8
LANG=en_US.UTF-8
LANGUAGE=en_US
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

两个最显着的区别是:

服务器:Python 3.5

本地环境:Python 3.7.2

apache2/error.log

[Fri Feb 15 09:49:15.899753 2019] [wsgi:error] [pid 2489] GETTING NEW TEST
[Fri Feb 15 09:49:15.957897 2019] [wsgi:error] [pid 2489] Saving the test dictionary
[Fri Feb 15 09:49:16.006470 2019] [wsgi:error] [pid 2489] Internal Server Error: /test/addohm/
[Fri Feb 15 09:49:16.006506 2019] [wsgi:error] [pid 2489] Traceback (most recent call last):
[Fri Feb 15 09:49:16.006509 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in in$
[Fri Feb 15 09:49:16.006511 2019] [wsgi:error] [pid 2489]     response = get_response(request)
[Fri Feb 15 09:49:16.006514 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_r$
[Fri Feb 15 09:49:16.006517 2019] [wsgi:error] [pid 2489]     response = self.process_exception_by_middleware(e, request)
[Fri Feb 15 09:49:16.006520 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_r$
[Fri Feb 15 09:49:16.006522 2019] [wsgi:error] [pid 2489]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Feb 15 09:49:16.006525 2019] [wsgi:error] [pid 2489]   File "/path/to/website/duotool/main/views.py", line 125, in test
[Fri Feb 15 09:49:16.006531 2019] [wsgi:error] [pid 2489]     form = TestForm(wordsdict)
[Fri Feb 15 09:49:16.006533 2019] [wsgi:error] [pid 2489]   File "/path/to/website/duotool/main/forms.py", line 21, in __init__
[Fri Feb 15 09:49:16.006536 2019] [wsgi:error] [pid 2489]     json.dumps(test_dict, f, indent=2, ensure_ascii=False)
[Fri Feb 15 09:49:16.006538 2019] [wsgi:error] [pid 2489]   File "/usr/lib/python3.5/json/__init__.py", line 179, in dump
[Fri Feb 15 09:49:16.006541 2019] [wsgi:error] [pid 2489]     fp.write(chunk)
[Fri Feb 15 09:49:16.006545 2019] [wsgi:error] [pid 2489] UnicodeEncodeError: 'ascii' codec can't encode character '\\u7ea6' in position 1: ordinal not in range(128)
[Fri Feb 15 09:49:16.006550 2019] [wsgi:error] [pid 2489]
[Fri Feb 15 09:51:36.924025 2019] [wsgi:error] [pid 2627] [client 124.9.54.252:55825] Timeout when reading response headers from daemon process 'duotool.addohm.net': /var/www/django/du$
[Fri Feb 15 09:51:43.283083 2019] [wsgi:error] [pid 2631] [client 124.9.54.252:55971] Truncated or oversized response headers received from daemon process 'duotool.addohm.net': /var/ww$
[Fri Feb 15 09:51:43.283324 2019] [wsgi:error] [pid 2630] [client 124.9.54.252:55888] Truncated or oversized response headers received from daemon process 'duotool.addohm.net': /var/ww$
[Fri Feb 15 09:53:18.572055 2019] [wsgi:error] [pid 2489] GETTING NEW TEST
[Fri Feb 15 09:53:18.631474 2019] [wsgi:error] [pid 2489] Saving the test dictionary
[Fri Feb 15 09:53:18.675315 2019] [wsgi:error] [pid 2489] Internal Server Error: /test/addohm/
[Fri Feb 15 09:53:18.675335 2019] [wsgi:error] [pid 2489] Traceback (most recent call last):
[Fri Feb 15 09:53:18.675338 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in in$
[Fri Feb 15 09:53:18.675341 2019] [wsgi:error] [pid 2489]     response = get_response(request)
[Fri Feb 15 09:53:18.675344 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_r$
[Fri Feb 15 09:53:18.675347 2019] [wsgi:error] [pid 2489]     response = self.process_exception_by_middleware(e, request)
[Fri Feb 15 09:53:18.675349 2019] [wsgi:error] [pid 2489]   File "/path/to/website/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_r$
[Fri Feb 15 09:53:18.675352 2019] [wsgi:error] [pid 2489]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Fri Feb 15 09:53:18.675354 2019] [wsgi:error] [pid 2489]   File "/path/to/website/duotool/main/views.py", line 125, in test
[Fri Feb 15 09:53:18.675357 2019] [wsgi:error] [pid 2489]     form = TestForm(wordsdict)
[Fri Feb 15 09:53:18.675359 2019] [wsgi:error] [pid 2489]   File "/path/to/website/duotool/main/forms.py", line 21, in __init__
[Fri Feb 15 09:53:18.675362 2019] [wsgi:error] [pid 2489]     json.dumps(test_dict, f, indent=2, ensure_ascii=False)
[Fri Feb 15 09:53:18.675365 2019] [wsgi:error] [pid 2489]   File "/usr/lib/python3.5/json/__init__.py", line 179, in dump
[Fri Feb 15 09:53:18.675367 2019] [wsgi:error] [pid 2489]     fp.write(chunk)
[Fri Feb 15 09:53:18.675371 2019] [wsgi:error] [pid 2489] UnicodeEncodeError: 'ascii' codec can't encode character '\\u90fd' in position 1: ordinal not in range(128)
[Fri Feb 15 09:53:18.675376 2019] [wsgi:error] [pid 2489]
[Fri Feb 15 09:53:19.498167 2019] [wsgi:error] [pid 2652] /path/to/website/duotool

我看不出两个版本之间有任何差异会导致这种情况。我该去哪里？

Answer 1

我认为您应该在打开文件时指定要写入的文件的编码。这应该有效:

   file = open(self._json_path, 'w',encoding='utf-8')
   file.write(json.dumps(your_json))
   file.close()