python - 模拟 map_async 函数参数产生 PicklingError

Question

尝试围绕执行 map_async() 的函数编写一些单元测试手术。更具体地说，我想确认在某个进程中发生异常时某些文件会被清理。下面提供了具有意图的示例伪代码。

foo.py

def write_chunk(chunk):
    ... create file from chunk
    return created_filename

class Foo:
    def write_parallel(chunks):
        filenames = set()
        try:
            pool = Pool(processes=2)
            pool.map_async(write_chunk, chunks, callback=filenames.add)
        except Exception:
            //handle exception
        finally:
            cleanup_files(filenames)

test_foo.py

@patch("foo.write_chunk")
def test_write_parallel_exception_cleanup(self, mock_write_chunk):
    def mock_side_effect(chunk):
        if "chunk_1" == chunk:
            raise Exception
        else:
            return chunk
    mock_write_chunk.side_effect = mock_side_effect

    foo = Foo()
    foo.write_parallel({"chunk_1", "chunk_2"})
    //assert "chunk_2" cleaned up and exception is thrown.

但是，当我去执行测试时，我收到以下 PicklingError: PicklingError: Can't pickle <class 'mock.MagicMock'>: it's not the same object as mock.MagicMock .

有什么想法如何执行用我自己的模拟函数替换映射函数的所需结果吗？

Answer 1

因此，由于问题源于尝试 Mock 和 Pickle 该函数，因此我决定将该功能提取到一个单独的函数中，模拟该函数，同时允许对原始函数进行 pickle。见下文:

foo.py

def write_chunk(chunk):
    return write_chunk_wrapped(chunk)

def write_chunk_wrapped(chunk)
    ... create file from chunk
    return created_filename

class Foo:
    def write_parallel(chunks):
        filenames = set()
        try:
            pool = Pool(processes=2)
            pool.map_async(write_chunk, chunks, callback=filenames.add)
        except Exception:
            //handle exception
        finally:
            cleanup_files(filenames)

test_foo.py

@patch("foo.write_chunk_wrapped")
def test_write_parallel_exception_cleanup(self, mock_write_chunk_wrapped):
    def mock_side_effect(chunk):
        if "chunk_1" == chunk:
            raise Exception
        else:
            return chunk
    mock_write_chunk_wrapped.side_effect = mock_side_effect

    foo = Foo()
    foo.write_parallel({"chunk_1", "chunk_2"})
    //assert "chunk_2" cleaned up and exception is thrown.