python - Pip install killed - 内存不足 - 如何绕过它？

Question

我正在尝试安装 spaCy，一个用于自然语言处理的库。

root@vps:~# python3 -m pip install spacy
Collecting spacy
  Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: spacy
Killed

过了一会儿，它被杀死了。我已经在 Stack Overflow 上寻找答案并尝试过，这是由于内存不足(我的 VPS 上有 1.2 Gb 可用，无法扩展)。快速 tail -f/var/log/kern.log 给我 Out of memory: Kill process 31729 (python3) score 558 or sacrific child.

我已经尝试添加 --no-cache-dir，但没有帮助。手动复制文件时似乎缺少某些内容。有什么方法可以在低 RAM 服务器上安装此软件包？

编辑:python3 -m pip install spacy -vvv

的完整日志

Converted retries value: Retry(total=5, connect=None, read=None, redirect=None)                                                                                                              -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N                                                                                                             one, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None)                                                                                                              -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N                                                                                                             one, read=None, redirect=None)
Collecting spacy
  1 location(s) to search for versions of spacy:
  * https://pypi.python.org/simple/spacy/
  Getting page https://pypi.python.org/simple/spacy/
  Looking up "https://pypi.python.org/simple/spacy/" in the cache
  Returning cached "301 Moved Permanently" response (ignoring date and etag info                                                                                                             rmation)
  Looking up "https://pypi.org/simple/spacy/" in the cache
  Current age based on date: 29458
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  Starting new HTTPS connection (1): pypi.org
  https://pypi.org:443 "GET /simple/spacy/ HTTP/1.1" 304 0
  Analyzing links from page https://pypi.org/simple/spacy/
    <!-- Found link, skipped linked (hundreds of lines) -->
    Found link https://files.pythonhosted.org/packages/a7/84/f35f44560f0ab9ef23c                                                                                                             4a334a513f2b6c8a95d7b3347ef68a0247da63c08/spacy-2.1.6.tar.gz#sha256=b708bb582597                                                                                                             6220de78ad7e20e827912ca95653738a812cc2e5d194e3060ae8 (from https://pypi.org/simp                                                                                                             le/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*), version: 2.1                                                                                                             .6
  Using version 2.1.6 (newest of versions: 0.31, 0.32, 0.33, 0.40, 0.51, 0.52, 0                                                                                                             .60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.67, 0.68, 0.70, 0.80, 0.81, 0.82, 0.83, 0.8                                                                                                             4, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.97, 0.98,                                                                                                              0.99, 0.100.0, 0.100.1, 0.100.2, 0.100.3, 0.100.4, 0.100.5, 0.100.6, 0.100.7, 0                                                                                                             .101.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.2.0, 1.3.0, 1.                                                                                                             4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.5, 1.8.0, 1.8.1, 1.8.2                                                                                                             , 1.9.0, 1.10.0, 1.10.1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7,                                                                                                              2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.0.12, 2.0.13, 2.0.15, 2.0.16, 2.0.17, 2.0.18, 2                                                                                                             .1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6)
  Looking up "https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247                                                                                                             babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_6                                                                                                             4.whl" in the cache
  Current age based on date: 98439
  Freshness lifetime from max-age: 365000000
  The response is "fresh", returning cached response
  365000000 > 98439
  Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd2938424                                                                                                             7babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_                                                                                                             64.whl
  Downloading from URL https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3b                                                                                                             d29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylin                                                                                                             ux1_x86_64.whl#sha256=2441ad9e6195669c13f4035dd77c4206414a40ebb8543c69f3386d984a                                                                                                             bfb942 (from https://pypi.org/simple/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.                                                                                                             1.*,!=3.2.*,!=3.3.*)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from spacy)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3                                                                                                             .5/dist-packages (from spacy)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/pytho                                                                                                             n3.5/dist-packages (from spacy)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /usr/local/lib/python3.5/                                                                                                             dist-packages (from spacy)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /usr/local/lib/python3.5                                                                                                             /dist-packages (from spacy)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.5/dist-pa                                                                                                             ckages (from spacy)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/l                                                                                                             ocal/lib/python3.5/dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-p                                                                                                             ackages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5                                                                                                             /dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from thinc<7.1.0,>=7.0.8->spacy)
Installing collected packages: spacy
Killed

Answer 1

这已经在github上报道过了

https://github.com/explosion/spaCy/issues/3396

下面是一些关于如何解决问题的好帖子

fork() failing with Out of memory error

Memory error while using pip install Matplotlib

out of memory issue in installing packages on Ubuntu server

https://chirale.org/2017/01/15/memory-error-on-pip-install-solved/

在摘要中，以下是两个选项:

1. 尝试使用 --no-cache-dir，不幸的是，它对您没有帮助

2. 增加交换空间

$ create swap file of 512 MB
dd if=/dev/zero of=/swapfile bs=1024 count=524288

$ modify permissions
chown root:root /swapfile
chmod 0600 /swapfile

$ setup swap area
mkswap /swapfile

$ turn swap on
swapon /swapfile