python - 将迭代器返回函数转换为 "proper iterable"返回函数的标准方法？

Question

为了避免混淆，让我定义:

proper iterable: an iterable object that is not an iterator.

---

问:Python 的标准库是否已经提供了一种将“迭代器返回函数”转换为“适当的可迭代返回函数”的方法？

---

我以为我在哪里见过这个，但现在我找不到了。特别是，我浏览了 itertools 的文档，但没有发现它。

---

FWIW，这个本土实现似乎有效:

def to_iterable_maker(iterator_maker):

    def iterable_maker(*args, **kwargs):
        class nonce_iterable(object):
            def __iter__(self):
                return iterator_maker(*args, **kwargs)
        return nonce_iterable()

    return iterable_maker

...但是其中的一次性 nonce_iterable 类在我看来很笨拙。我确信从标准库中实现这样的东西会好得多。

---

@尼基塔

试试这个:

import itertools

base = range(3)
poops_out = itertools.permutations(base)

print list(poops_out)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(poops_out)
# []

myperms = to_iterable_maker(itertools.permutations)
keeps_going = myperms(base)

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

itertools.permutations 和 to_iterable_maker(itertools.permutations) 返回的值之间存在差异。我的问题是:标准库是否已经提供了类似于 to_iterable_maker 的东西？

Answer 1

这没有意义 - “迭代器返回函数”变成“可迭代返回函数”。如果一个函数返回一个迭代器，那么它已经返回一个可迭代对象，因为迭代器是可迭代对象，因为它们需要具有 __iter__ 方法。

来自 the docs :

iterable

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() or __getitem__() method. Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), ...). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also iterator, sequence, and generator.

iterator

An object representing a stream of data. Repeated calls to the iterator’s __next__() method (or passing it to the built-in function next()) return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its __next__() method just raise StopIteration again.

Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable

and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a list) produces a fresh new iterator each time you pass it to the iter() function or use it in a for loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

更新:

我的意思是...

(按步骤显示比较)

案例 1:

1. f = to_iterable_maker(iterator_maker);
2. i = f(some_var), i 是 nonce_iterable 和 __iter__;
3. j = iter(i)，j是iterator_maker(some_var)返回的迭代器；
4. next(j)，返回一些依赖于 some_var 的值。

案例 2:

1. f = iterator_maker;
2. i = f(some_var), i 是等于iterator_maker(some_var) 的迭代器，它有__iter__(根据迭代器协议(protocol))；
3. j = iter(i)，j是iterator_maker(some_var)返回的迭代器，因为调用了__iter__ 在迭代器上返回自身，所以 j is i 返回 true;
4. next(j)，返回一些依赖于 some_var 的值。

如您所见，除了准备步骤的额外复杂性外，没有什么真正的改变。

也许您可以提供有关您试图通过这种“包装”实现的目标的更多信息，以了解真正的问题。

根据你的问题，我想不出有任何库函数可以将迭代器变成可迭代的，因为它已经是了。如果你想复制迭代器，可以看看 itertools.tee() .

UPD2:

所以，现在我明白了，目标是将单遍迭代器转换为多遍迭代器...

我的回答:

"does the Standard Library already provide something analogous to to_iterable_maker?"

是“否”。但最接近的是itertools.tee() ，它可以帮助您将单个迭代器克隆为多个，您可以在之后使用。关于你的例子:

import itertools

base = range(3)
poops_out = itertools.permutations(base)

iterators = itertools.tee(poops_out, 4)

#You shouldn't use original iterator after clonning, so make it refer to a clone
#to be used again, otherwise ignore the following line
poops_out, iterators = iterators[0], iterators[1:]

for it in iterators:
    print list(it)

#Prints:
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

从迭代器获取可迭代对象的另一种常见方法是使用 list() 或 tuple() 对其进行转换，这将允许多次通过:

import itertools

base = range(3)

poops_out = itertools.permutations(base)

#Obviously poops_out gets consumed at the next line, so it won't iterate anymore
keeps_going = tuple(poops_out)

print list(poops_out)
# []

print list(poops_out)
# []

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

上述两种方法都可能会占用大量内存，因此有时不可行。在这种情况下，您找到的解决方案会很有效。另一个我能想到的实现，它有点面向对象，但在其他方面与你的没有太大区别:

class IterableMaker(object):
    '''Wraps function returning iterator into "proper" iterable'''

    def __init__(self, iterator_maker):
        self.f = iterator_maker

    def __call__(self, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs
        return self

    def __iter__(self):
        return self.f(*self.args, **self.kwargs)

用法是一样的:

import itertools

class IterableMaker(object):

    def __init__(self, iterator_maker):
        self.f = iterator_maker

    def __call__(self, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs
        return self

    def __iter__(self):
        return self.f(*self.args, **self.kwargs)

base = range(3)
poops_out = itertools.permutations(base)

print list(poops_out)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(poops_out)
# []

my_perm = IterableMaker(itertools.permutations)

keeps_going = my_perm(base)

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]