python - 使用分页测试 boto3/botocore

Question

我一直在尝试将单元测试添加到我的 AWS 脚本中。我一直在使用 botocore.stub 来 stub API 调用。

我需要为各种调用添加分页，但我似乎找不到一种方法来编写包含分页的测试。

这是一个非分页测试的例子，我想知道如何重构这个测试和函数来使用分页:

 # -*- coding: utf-8 -*-
 import unittest
 import boto3
 from botocore.stub import Stubber
 from datetime import datetime


 def describe_images(client, repository):
     return client.describe_images(repositoryName=repository)


 class TestCase(unittest.TestCase):
     def setUp(self):
         self.client = boto3.client('ecr')

     def test_describe_images(self):
         describe_images_response = {
             'imageDetails': [
                 {
                     'registryId': 'string',
                     'repositoryName': 'string',
                     'imageDigest': 'string',
                     'imageTags': [
                         'string',
                     ],
                     'imageSizeInBytes': 123,
                     'imagePushedAt': datetime(2015, 1, 1)
                 },
             ],
             'nextToken': 'string'
         }
         stubber = Stubber(self.client)
         expected_params = {'repositoryName': 'repo_name'}
         stubber.add_response(
             'describe_images',
             describe_images_response,
             expected_params
         )
         with stubber:
             response = describe_images(self.client, 'repo_name')

         self.assertEqual(describe_images_response, response)


 if __name__ == '__main__':
     unittest.main()

如果我更新函数以包含这样的分页:

 def describe_images(client, repository):
     paginator = client.get_paginator('describe_images')

     response_iterator = paginator.paginate(
         repositoryName=repository
     )
     return response_iterator

我们似乎有所进展。由于平等性发生了变化，测试失败了:

F
======================================================================
FAIL: test_describe_images (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "desc_imgs_paginated.py", line 47, in test_describe_images
    self.assertEqual(describe_images_response, response)
AssertionError: {'imageDetails': [{'registryId': 'string'[178 chars]ing'} != <botocore.paginate.PageIterator object at 0x1058649b0>

----------------------------------------------------------------------
Ran 1 test in 0.075s

FAILED (failures=1)

当我尝试遍历生成器时::

def describe_images(client, repository):
     paginator = client.get_paginator('describe_images')

     response_iterator = paginator.paginate(
         repositoryName=repository
     )
     return [r for r in response_iterator]

我收到以下错误:

 E
 ======================================================================
 ERROR: test_describe_images (__main__.TestCase)
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "desc_imgs_paginated.py", line 45, in test_describe_images
     response = describe_images(self.client, repo_name)
   File "desc_imgs_paginated.py", line 14, in describe_images
     return '.join([r for r in response_iterator])
   File "desc_imgs_paginated.py", line 14, in <listcomp>
     return '.join([r for r in response_iterator])
   File "lib/python3.6/site-packages/botocore/paginate.py", line 255, in __iter__
     response = self._make_request(current_kwargs)
   File "lib/python3.6/site-packages/botocore/paginate.py", line 332, in _make_request
     return self._method(**current_kwargs)
   File "lib/python3.6/site-packages/botocore/client.py", line 312, in _api_call
     return self._make_api_call(operation_name, kwargs)
   File "lib/python3.6/site-packages/botocore/client.py", line 579, in _make_api_call
     api_params, operation_model, context=request_context)
   File "lib/python3.6/site-packages/botocore/client.py", line 631, in _convert_to_request_dict
     params=api_params, model=operation_model, context=context)
   File "lib/python3.6/site-packages/botocore/hooks.py", line 227, in emit
     return self._emit(event_name, kwargs)
   File "lib/python3.6/site-packages/botocore/hooks.py", line 210, in _emit
     response = handler(**kwargs)
   File "lib/python3.6/site-packages/botocore/stub.py", line 337, in _assert_expected_params
     self._assert_expected_call_order(model, params)
   File "lib/python3.6/site-packages/botocore/stub.py", line 323, in _assert_expected_call_order
     pformat(params)))
 botocore.exceptions.StubResponseError: Error getting response stub for operation DescribeImages: Unexpected API Call: called with parameters:
 {nextToken: string, repositoryName: repo_name}

 ----------------------------------------------------------------------
 Ran 1 test in 0.051s

 FAILED (errors=1)

我是否缺少正确的测试方法？或者这是 boto3/botocore 中的错误？

Answer 1

问这个问题已经有一段时间了，但是因为没有答案..

在您的设置中，您提供了一个响应字典，如下所示

describe_images_response = {
    'imageDetails': [
        {
            'registryId': 'string',
            'repositoryName': 'string',
            'imageDigest': 'string',
            'imageTags': [
                'string',
            ],
            'imageSizeInBytes': 123,
            'imagePushedAt': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

这里的关键是第一个响应将包含一个 nextToken 值。这将导致来自分页器的第二个请求。因此，您必须为 stub 提供额外的响应，最终您需要以不包含nextToken

的响应结束

现在回顾一下您的设置，只有一个 add_response 调用 stub

stubber.add_response(
    'describe_images',
    describe_images_response,
    expected_params
)

最终结果是，当分页器发出第二个请求时，设置中没有指定响应。

这会导致异常，希望现在的消息更有意义

 botocore.exceptions.StubResponseError: Error getting response stub for operation DescribeImages: Unexpected API Call: called with parameters:
 {nextToken: string, repositoryName: repo_name}

由于没有设置第二个响应，你得到了一个意外的请求异常，在这个请求中你可以看到nextToken参数的规范。