python - 使用 pytest，为什么单个测试结果与运行所有测试不同？

Question

将 pytest 与 Python 3.6 结合使用，我正在为一个项目进行管道模拟。我进行了针对不同阀门排列等的测试。

奇怪的是，如果我在一个目录中运行所有测试，所有测试都会通过。如果我尝试运行单个测试类，它会失败。我已经尝试添加设置/拆卸方法以将其与其余代码分开，以防它以某种方式受到其他类的影响。

下面是测试代码(ffc是组件库，fff是方法库):

class TestGate3TankLevels:
    @classmethod
    def setup_class(cls):
        ffc.tank2.level = 18.0
        fff.change_tank_level(ffc.tank2, ffc.tank2.level)
        fff.gate1_open()
        fff.gate2_open()
        fff.gate3_open()
        fff.gate4_open()

    def test_gate3_tank_levels(self):
        assert ffc.gate1.position == 100
        assert ffc.gate1.flow_out == 19542.86939891452
        assert ffc.gate1.press_out == 13.109851301499999

        assert ffc.gate2.position == 100
        assert ffc.gate2.flow_out == 19542.86939891452
        assert ffc.gate2.press_out == 6.5549256507499996

        assert ffc.gate3.position == 100
        assert ffc.gate3.flow_in == 19542.86939891452
        assert ffc.gate3.press_in == 13.109851301499999
        assert ffc.gate3.flow_out == 19542.86939891452
        assert ffc.gate3.press_out == 13.109851301499999
        assert ffc.gate6.flow_in == 19542.86939891452
        assert ffc.gate6.press_in == 13.109851301499999

        assert ffc.gate4.position == 100
        assert ffc.gate4.flow_in == 0.0
        assert ffc.gate4.press_in == 0.0
        assert ffc.gate4.flow_out == 0.0
        assert ffc.gate4.press_out == 0.0

    @classmethod
    def teardown_class(cls):
        ffc.tank2.level = 32.0
        fff.change_tank_level(ffc.tank2, ffc.tank2.level)

单独运行时，我得到以下结果:

F
test_fuel_components.py:408 (TestGate3TankLevels.test_gate3_tank_levels)
19542.86939891452 != 39085.73879782904

Expected :39085.73879782904
Actual   :19542.86939891452
 <Click to see difference>

self = <VirtualPLC.tests.models.fuel_farm.test_fuel_components.TestGate3TankLevels object at 0x7f9b35289a20>

    def test_gate3_tank_levels(self):
        assert ffc.gate1.position == 100
        assert ffc.gate1.flow_out == 19542.86939891452
        assert ffc.gate1.press_out == 13.109851301499999

        assert ffc.gate2.position == 100
        assert ffc.gate2.flow_out == 19542.86939891452
        assert ffc.gate2.press_out == 6.5549256507499996

        assert ffc.gate3.position == 100
        assert ffc.gate3.flow_in == 19542.86939891452
        assert ffc.gate3.press_in == 13.109851301499999
        assert ffc.gate3.flow_out == 19542.86939891452
        assert ffc.gate3.press_out == 13.109851301499999
>       assert ffc.gate6.flow_in == 19542.86939891452
E       assert 39085.73879782904 == 19542.86939891452
E        +  where 39085.73879782904 = <PipingSystems.valve.valve.Gate object at 0x7f9b3515ce80>.flow_in
E        +    where <PipingSystems.valve.valve.Gate object at 0x7f9b3515ce80> = ffc.gate6

test_fuel_components.py:423: AssertionError

但是，如果我在 test_fuel_components.py 文件中运行所有测试，那么一切都会通过:

Testing started at 2:16 PM ...
/home/cody/PycharmProjects/VirtualPLC/venv/bin/python /home/cody/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/182.3341.8/helpers/pycharm/_jb_pytest_runner.py --path /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm/test_fuel_components.py
Launching py.test with arguments /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm/test_fuel_components.py in /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm

============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.6.2, py-1.5.3, pluggy-0.6.0
rootdir: /home/cody/PycharmProjects/VirtualPLC/tests/models/fuel_farm, inifile:collected 20 items

test_fuel_components.py ....................                             [100%]

========================== 20 passed in 0.16 seconds ===========================
Process finished with exit code 0

如果我将这个测试类移动到文件中的另一个位置(现在它在所有其他测试的末尾)，它会导致任何后续测试失败，以及失败本身。

我不明白一个测试类为什么会自己失败，但在与其他测试一起运行时却没问题。

编辑 Link to other tests (这里太长了)

Answer 1

所以这里的问题是，当 jar 体型号为imported时，每个 jar 体和阀门的状态都被初始化一次。 ，就像这里的 tank1:

# Storage tanks
# Assumes 36 ft tall tank w/ 1 million gallon capacity = 27778 gallons per foot
# Assumes 16 inch diam transfer piping
tank1 = tank.Tank("Tank 1", level=36.0, fluid_density=DENSITY, spec_gravity=SPEC_GRAVITY, outlet_diam=16,
                  outlet_slope=0.25)
tank1.static_tank_press = tank1.level
tank1.gravity_flow(tank1.pipe_diam, tank1.pipe_slope, tank1.pipe_coeff)

如果不了解这个项目应该做什么，很难对这个设计提出具体的批评，但这个设计有大量的全局状态，这使得测试非常困难。

无论现有设计的优缺点如何，如果您希望测试可靠且独立，您需要做的是在每次测试开始时将每个储 jar 的状态重置为已知状态。考虑到这里涉及的全局状态有多少，您的选择非常有限。

当您全局存储这么多状态时，我的建议是在每次测试后使用 setUp 和 tearDown 方法重置回已知状态。

这是未经测试的，但您正在寻找这样的东西:

original_states = {}
tanks = [tank1, tank2, tank3, tank4, tank5, tank6]
tank_attrs = [ "name", "level", "fluid_density", "spec_grav",
               "tank_press", "flow_out", "pipe_diam", "pipe_slope" ]
def setUp(self):
    for tank in self.tanks:
        self.original_states[id(tank)] = {}
        for attr in self.tank_attrs:
            self.original_states[id(tank)] = getattr(tank, attr)

def tearDown(self):
    try:
        for tank in self.tanks:
            for attr, value in self.original_states[id(tank)].items():
                setattr(tank, attr, value)
    finally:
        self.original_states = {}

它基本上存储每次测试前的原始状态，然后在每次测试后恢复它。其中一些属性具有一些魔力，因此可能需要反复试验才能使其 100% 发挥作用。