python - 谷歌应用程序引擎批量加载器 "Unexpected Thread Death"

Question

我正在尝试使用批量加载器功能将大小适中的 csv 文件上传到 Google 应用引擎，但它似乎在中途死亡，结果如下:

[INFO    ] Logging to bulkloader-log-20110328.181531
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10
[INFO    ] Opening database: bulkloader-progress-20110328.181531.sql3
[INFO    ] Connecting to notmyrealappname.appspot.com/_ah/remote_api
[INFO    ] Starting import; maximum 10 entities per post
...............................................................[INFO    ] Unexpected thread death: WorkerThread-7
[INFO    ] An error occurred. Shutting down...
.........[ERROR   ] Error in WorkerThread-7: <urlopen error [Errno -2] Name or service not known>

[INFO    ] 1740 entites total, 0 previously transferred
[INFO    ] 720 entities (472133 bytes) transferred in 32.3 seconds
[INFO    ] Some entities not successfully transferred

它上传了我尝试上传的 19k 条目中的大约 700 个，我想知道为什么它失败了。我检查了 csv 文件是否有错误，例如可能会导致 python csv 阅读器无法使用的额外逗号，并且非 ascii 字符已被删除。

Answer 1

提升批量限制 (batch_size) 和 rps 限制 (rps_limit) 是有效的，我使用 1000 作为批量大小，将 rps 限制设置为 500:

appcfg.py upload_data --url= --application= --filename=  --email= --batch_size=1000 --rps_limit=500