proxy - 如何运行通过代理使用需求文件的 Ansible pip 任务？

Question

我们最近设置了一个开发箱，我们希望能够在上面安装我们的软件。我们正在使用 Ansible，它在我们的本地环境中一切正常，但这个新盒子被设置为通过代理访问外部世界。这意味着要让所有 pip 任务在 ansible 中工作，我需要在每个任务上设置环境，详细说明代理详细信息(根据 http://docs.ansible.com/ansible/latest/user_guide/playbooks_environment.html )。

- name: upgrade pip
  pip:
    virtualenv: "{{ wcsftp_sources_dir }}/{{ wcsftp_rev }}"
    virtualenv_command: virtualenv-2.7
    name: pip
    version: "{{ pip_version }}"
    umask: '0022'
  environment:
    http_proxy: http://proxy.example.com:8080
    https_proxy: http://proxy.example.com:8080
  tags:
    - dependencies
    - virtualenv

这工作正常，但是当我运行一个使用需求文件的 pip 任务时，我收到了错误，我假设这些错误是由于代理详细信息没有传递到安装每个必需软件的调用中。

- name: create virtualenv for wcsftp
  pip:
    virtualenv: "{{ wcsftp_sources_dir }}/{{ wcsftp_rev }}"
    virtualenv_command: virtualenv-2.7
    requirements: "{{ wcsftp_sources_dir }}/{{ wcsftp_rev }}/wcsftp/requirements.txt"
    umask: '0022'
  environment:
    GIT_AUTH: "{{ gituser }}:{{ gitpass }}"
    GIT_ASKPASS: /usr/local/bin/git_env_askpass.py
    http_proxy: http://proxy.example.com:8080
    https_proxy: http://proxy.example.com:8080
    HTTP_PROXY: http://proxy.example.com:8080
    HTTPS_PROXY: http://proxy.example.com:8080
  tags:
    - dependencies
    - virtualenv

导致以下错误:

build   10-Apr-2018 10:14:56    TASK [create virtualenv for wcsftp] ********************************************
build   10-Apr-2018 10:15:00    fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/srv/wcsftp/master/bin/pip2 install -r /srv/wcsftp/master/wcsftp/requirements.txt", "failed": true, "msg": "stdout: Collecting git+https://github.com/prometheus/client_python.git (from -r /srv/wcsftp/master/wcsftp/requirements.txt (line 15))
Cloning https://github.com/prometheus/client_python.git to ./pip-VBNsmQ-build\nCollecting Twisted==17.5.0 (from -r /srv/wcsftp/master/wcsftp/requirements.txt (line 1))
Downloading Twisted-17.5.0.tar.bz2 (3.0MB)
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/incremental/: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:579) -- Some packages may not be found!
Couldn't find index page for 'incremental' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:579) -- Some packages may not be found!
No local packages or download links found for incremental>=16.10.1
Traceback (most recent call last):
File \"<string>\", line 1, in <module>
File \"/tmp/pip-build-HAEyHG/Twisted/setup.py\", line 21, in <module>
setuptools.setup(**_setup[\"getSetupArgs\"]())
File \"/usr/lib64/python2.7/distutils/core.py\", line 112, in setup
_setup_distribution = dist = klass(attrs)
File \"/srv/wcsftp/master/lib/python2.7/site-packages/setuptools/dist.py\", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File \"/srv/wcsftp/master/lib/python2.7/site-packages/setuptools/dist.py\", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File \"/srv/wcsftp/master/lib/python2.7/site-packages/pkg_resources.py\", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File \"/srv/wcsftp/master/lib/python2.7/site-packages/pkg_resources.py\", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File \"/srv/wcsftp/master/lib/python2.7/site-packages/pkg_resources.py\", line 874, in obtain
return installer(requirement)
File \"/srv/wcsftp/master/lib/python2.7/site-packages/setuptools/dist.py\", line 339, in fetch_build_egg
return cmd.easy_install(req)
File \"/srv/wcsftp/master/lib/python2.7/site-packages/setuptools/command/easy_install.py\", line 617, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('incremental>=16.10.1')
----------------------------------------
:stderr: Command \"python setup.py egg_info\" failed with error code 1 in /tmp/pip-build-HAEyHG/Twisted/

如果我在盒子上手动运行以下命令，那么一切都安装正常，包括增量:

pip install twisted==17.5.0 --proxy=http://proxy.example.com:8080

如果我在手动安装 twisted 后再次运行 ansible，我会得到完全相同的 SSL: UNKNOWN_PROTOCOL 错误，但这次它使用了 Twisted-17.5.0.tar.bz2 的缓存版本。

我尝试手动运行命令并使用 pip 和 pip2 指定代理信息:

/srv/wcsftp/master/bin/pip2 install -r /srv/wcsftp/master/wcsftp/requirements.txt --proxy=http://webproxy.e.corp.services:80

又报了不同的错误，但还是和无法访问外部站 pip 下载增量包有关:

Collecting Twisted==17.5.0 (from -r /srv/wcsftp/master/wcsftp/requirements.txt (line 1))
  Using cached Twisted-17.5.0.tar.bz2
    Complete output from command python setup.py egg_info:
    Download error on https://pypi.python.org/simple/incremental/: [Errno 101] Network is unreachable -- Some packages may not be found!
    Couldn't find index page for 'incremental' (maybe misspelled?)
    Download error on https://pypi.python.org/simple/: [Errno 101] Network is unreachable -- Some packages may not be found!
    No local packages or download links found for incremental>=16.10.1

Answer 1

查看 pip 关于环境变量的文档 https://pip.pypa.io/en/stable/user_guide/#environment-variables

你会看到

pip's command line options can be set with environment variables using the format PIP_<UPPER_LONG_NAME> . Dashes (-) have to be replaced with underscores (_).

使用您的示例代码，试试这个:

- name: create virtualenv for wcsftp
  pip:
    virtualenv: "{{ wcsftp_sources_dir }}/{{ wcsftp_rev }}"
    virtualenv_command: virtualenv-2.7
    requirements: "{{ wcsftp_sources_dir }}/{{ wcsftp_rev }}/wcsftp/requirements.txt"
    umask: '0022'
  environment:
    GIT_AUTH: "{{ gituser }}:{{ gitpass }}"
    GIT_ASKPASS: /usr/local/bin/git_env_askpass.py
    PIP_PROXY: http://proxy.example.com:8080
  tags:
    - dependencies
    - virtualenv

希望这能解决您的问题!快乐的自动化!