python - 字典大小在哪里变化？

Question

在我的Python Utilities Github repo我有一个函数可以从字符串、映射和序列中删除非打印字符和无效的 Unicode 字节:

def filterCharacters(s):
    """
    Strip non printable characters

    @type s dict|list|tuple|bytes|string
    @param s Object to remove non-printable characters from

    @rtype dict|list|tuple|bytes|string
    @return An object that corresponds with the original object, nonprintable characters removed.
    """

    validCategories = (
        'Lu', 'Ll', 'Lt', 'LC', 'Lm', 'Lo', 'L', 'Mn', 'Mc', 'Me', 'M', 'Nd', 'Nl', 'No', 'N', 'Pc',
        'Pd', 'Ps', 'Pe', 'Pi', 'Pf', 'Po', 'P', 'Sm', 'Sc', 'Sk', 'So', 'S', 'Zs', 'Zl', 'Zp', 'Z'
    )
    convertToBytes = False

    if isinstance(s, dict):
        new = {}
        for k,v in s.items(): # This is the offending line
            new[k] = filterCharacters(v)
        return new

    if isinstance(s, list):
        new = []
        for item in s:
            new.append(filterCharacters(item))
        return new

    if isinstance(s, tuple):
        new = []
        for item in s:
            new.append(filterCharacters(item))
        return tuple(new)

    if isinstance(s, bytes):
        s = s.decode('utf-8')
        convertToBytes = True

    if isinstance(s, str):
        s = ''.join(c for c in s if unicodedata.category(c) in validCategories)
        if convertToBytes:
            s = s.encode('utf-8')
        return s

    else:
        return None

有时这个函数会抛出异常:

Traceback (most recent call last):
  File "./util.py", line 56, in filterCharacters
    for k,v in s.items():
RuntimeError: dictionary changed size during iteration

我看不到我在哪里更改作为参数发送的字典。那么为什么会抛出这个异常呢？

谢谢!

Answer 1

在 python 3 中，dict.items() 返回 dict_view 对象(不是 python 2 中的 list)。查看 CPython 代码，我注意到像

这样的评论

Objects/dictobject.c

dict_items(register PyDictObject *mp) 
{
    ...
    /* Preallocate the list of tuples, to avoid allocations during
     * the loop over the items, which could trigger GC, which
     * could resize the dict. :-(
     */
    ...

    if (n != mp->ma_used) {
        /* Durnit.  The allocations caused the dict to resize.
         * Just start over, this shouldn't normally happen.
         */
        Py_DECREF(v);
        goto again;
    }
    ...
}

因此，不仅字典删除和插入可能导致显示此错误，任何分配也可能导致显示!哦!

调整大小的过程也很有趣。看看

static int
dictresize(PyDictObject *mp, Py_ssize_t minused)
{
    ...
}

但这都是内部结构。

解决方案

尝试将 dict_view 转换为 list

if isinstance(s, dict):
    new = {}
    items = [i for i in s.items()]
    for k,v in items:
        new[k] = filterCharacters(v)
    return new