python itertools groupby 返回元组

Question

我需要解析展平结构并使用提供的键列表创建嵌套结构。我已经解决了这个问题，但我正在寻求改进，我想了解我可以在我的代码中更改什么。有人可以审查它并使用更好的知识进行重构吗？

src_data = [
  {
    "key1": "XX",
    "key2": "X111",
    "key3": "1aa",
    "key4": 1
  },
  {
    "key1": "YY",
    "key2": "Y111",
    "key3": "1bb",
    "key4": 11
  },
  {
    "key1": "ZZ",
    "key2": "Z111",
    "key3": "1cc",
    "key4": 2.4
  },
  {
    "key1": "AA",
    "key2": "A111",
    "key3": "1cc",
    "key4": 33333.2122
  },
  {
    "key1": "BB",
    "key2": "B111",
    "key3": "1bb",
    "key4": 2
  },
]

这是我迄今为止开发的代码，用于创建最终结果。

def plant_tree(ll):
    master_tree = {}

    for i in ll:
        tree = master_tree
        for n in i:
            if n not in tree:
                tree[n] = {}
            tree = tree[n]
    return master_tree



def make_nested_object(tt, var):
    elo = lambda l: reduce(lambda x, y: {y: x}, l[::-1], var)
    return {'n_path': tt, 'n_structure': elo(tt)}



def getFromDict(dataDict, mapList):
    return reduce(operator.getitem, mapList, dataDict)


def set_nested_item(dataDict, mapList, val):
    """Set item in nested dictionary"""
    reduce(getitem, mapList[:-1], dataDict)[mapList[-1]] = val
    return dataDict



def update_tree(data_tree):
    # MAKE NESTED OBJECT
    out = (make_nested_object(k, v) for k,v, in res_out.items())


    for dd in out:
        leaf_data = dd['n_structure']
        leaf_path = dd['n_path']
        data_tree = set_nested_item(data_tree, leaf_path, getFromDict(leaf_data, leaf_path))
    return data_tree

这是这个问题的自定义 itemgeter 函数

def customed_itemgetter(*args):
    # this handles the case when one key is provided
    f = itemgetter(*args)
    if len(args) > 2:
        return f
    return lambda obj: (f(obj),)

定义嵌套层次

nesting_keys = ['key1', 'key3', 'key2']

grouper = customed_itemgetter(*nesting_keys)
ii = groupby(sorted(src_data, key=grouper), grouper)

res_out = {key: [{k:v for k,v in i.items() if k not in nesting_keys} for i in group] for key,group in ii}
#
ll = ([dd[x] for x in nesting_keys] for dd in src_data)
data_tree = plant_tree(ll)

获取结果

result = update_tree(data_tree)

如何改进我的代码？

Answer 1

如果itemgetter [Python-doc]给定一个元素，它返回该单个元素，并且不将它包装在一个单例元组中。

但是我们可以为此构造一个函数，例如:

from operator import itemgetter

def itemgetter2(*args):
    f = itemgetter(*args)
    if len(args) > 2:
        return f
    return lambda obj: (f(obj),)

然后我们可以使用新的 itemgetter2，例如:

grouper = <b>itemgetter2(</b>*ll<b>)</b>
ii = groupby(sorted(src_data, key=grouper), grouper)

编辑:根据您的问题，您想要执行多级分组，我们可以为此创建一个函数，例如:

def multigroup(groups, iterable, index=0):
    if len(groups) <= index:
        return list(iterable)
    else:
        f = itemgetter(groups[index])
        i1 = index + 1
        return {
            k: multigroup(groups, vs, index=i1)
            for k, vs in groupby(sorted(iterable, key=f), f)
        }

对于问题中的 data_src，这会生成:

>>> multigroup(['a', 'b'], src_data)
{1: {2: [{'a': 1, 'b': 2, 'z': 3}]}, 2: {3: [{'a': 2, 'b': 3, 'e': 2}]}, 4: {3: [{'a': 4, 'x': 3, 'b': 3}]}}

不过，您可以在 list(..) 调用中对值进行后处理。例如，我们可以生成没有分组列中元素的字典:

def multigroup(groups, iterable):
    group_set = set(groups)
    fs = [itemgetter(group) for group in groups]
    def mg(iterable, index=0):
        if len(groups) <= index:
            return [
                {k: v for k, v in item.items() if k not in group_set}
                for item in iterable
            ]
        else:
            i1 = index + 1
            return {
                k: mg(vs, index=i1)
                for k, vs in groupby(sorted(iterable, key=fs[index]), fs[index])
            }
    return mg(iterable)

对于给定的样本输入，我们得到:

>>> multigroup(['a', 'b'], src_data)
{1: {2: [{'z': 3}]}, 2: {3: [{'e': 2}]}, 4: {3: [{'x': 3}]}}

或者对于新的示例数据:

>>> pprint(multigroup(['key1', 'key3', 'key2'], src_data))
{'AA': {'1cc': {'A111': [{'key4': 33333.2122}]}},
 'BB': {'1bb': {'B111': [{'key4': 2}]}},
 'XX': {'1aa': {'X111': [{'key4': 1}]}},
 'YY': {'1bb': {'Y111': [{'key4': 11}]}},
 'ZZ': {'1cc': {'Z111': [{'key4': 2.4}]}}}