docker - sam build --use-container失败，但sam build成功

Question

该问题发生在以下设置中:

在OpenStack中启动的虚拟机

OS是Ubuntu 16.04 LTS

安装了virtualenv的Python版本3.7.6

SAM CLI版本0.39.0

要复制问题，您可以使用上面的设置并执行以下步骤:

$ sam init

$ Choice: 1

$ Runtime 9选择python3.7

$ Project name[sam-app]:sam-app

$ Template selection: 1选择Hello World示例

等待生成应用程序。

$ cd sam-app

$ python3 -m virtualenv venv

$ source venv/bin/activate

$(venv) source venv/bin/activate

$(venv) sam build

将显示以下输出:

Building resource 'HelloWorldFunction'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

但是，如果使用 --use-container标志，将出现以下错误

Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
Starting Build inside a container
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
Building resource 'HelloWorldFunction'

Fetching lambci/lambda:build-python3.7 Docker container image......
Mounting /home/ubuntu/test/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
    requirements_path=self.manifest_path,
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 198, in build_site_packages
    wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 222, in _download_dependencies
    deps = self._download_all_dependencies(requirements_filename, directory)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 305, in _download_all_dependencies
    self._pip.download_all_dependencies(requirements_filename, directory)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 594, in download_all_dependencies
    raise NoSuchPackageError(str(package_name))
aws_lambda_builders.workflows.python_pip.packager.NoSuchPackageError: Could not satisfy the requirement: requests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: Could not satisfy the requirement: requests
Builder workflow failed
Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
    requirements_path=self.manifest_path,
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 198, in build_site_packages
    wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 222, in _download_dependencies
    deps = self._download_all_dependencies(requirements_filename, directory)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 305, in _download_all_dependencies
    self._pip.download_all_dependencies(requirements_filename, directory)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 594, in download_all_dependencies
    raise NoSuchPackageError(str(package_name))
aws_lambda_builders.workflows.python_pip.packager.NoSuchPackageError: Could not satisfy the requirement: requests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: Could not satisfy the requirement: requests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/__main__.py", line 126, in main
    mode=params.get("mode", None),
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/builder.py", line 125, in build
    return workflow.run()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 76, in wrapper
    func(self, *args, **kwargs)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 276, in run
    raise WorkflowFailedError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - Could not satisfy the requirement: requests
Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - Could not satisfy the requirement: requests"}}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 107282, 'exitReason': 'BuildError', 'exitCode': 1, 'requestId': '62d1fc73-70e5-4592-8c78-8fa273684592', 'installationId': 'ce8ffa14-684f-4628-97fe-288848fcf73d', 'sessionId': '4ca5bb42-5cf1-4ae5-89ad-9ed00366fefb', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.39.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - Could not satisfy the requirement: requests

Answer 1

出现上述问题是由于python PythonPipBuilder无法解决依赖关系。从那以后，我登录到docker并手动安装依赖项，随后我怀疑这是由于docker网络问题所致。

随后，通过添加其他标志来构建sam应用程序，该标志使docker能够使用主机网络。
$(venv) sam build --use-container --docker-network host --debug
命令的输出:

Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
Starting Build inside a container
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
Building resource 'HelloWorldFunction'

Fetching lambci/lambda:build-python3.7 Docker container image......
Mounting /home/ubuntu/test/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
Full dependency closure: {requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
initial compatible: {requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
initial incompatible: set()
Downloading missing wheels: set()
compatible wheels after second download pass: {requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Build missing wheels from sdists (C compiling True): set()
compatible after building wheels (no C compiling): {requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Build missing wheels from sdists (C compiling False): set()
compatible after building wheels (C compiling): {requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Final compatible: {chardet==3.0.4(wheel), requests==2.23.0(wheel), certifi==2019.11.28(wheel), urllib3==1.25.8(wheel), idna==2.9(wheel)}
Final incompatible: set()
Final missing wheels: set()
PythonPipBuilder:ResolveDependencies succeeded
Running PythonPipBuilder:CopySource
PythonPipBuilder:CopySource succeeded
Build inside container returned response {"jsonrpc": "2.0", "id": 1, "result": {"artifacts_dir": "/tmp/samcli/artifacts"}}
Build inside container was successful. Copying artifacts from container to host
Copying from container: /tmp/samcli/artifacts/. -> /home/ubuntu/test/sam-app/.aws-sam/build/HelloWorldFunction
Build inside container succeeded

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 5544, 'exitReason': 'success', 'exitCode': 0, 'requestId': 'bd9e7b0a-82ac-40a8-a172-a2d49f0633ff', 'installationId': 'ce8ffa14-684f-4628-97fe-288848fcf73d', 'sessionId': '48762175-e559-4759-b2fa-6a66822e381e', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.39.0'}}]}

我的同事Phetsouvanh Silivanxay博士也一起对这个问题进行了故障排除。