Python csv 转 json (D3)

Question

我试图从我生成的 .csv 创建一个 JSON 文件(用于 D3)，如下所示:

uat,soe1.1,deploy-mash-app40-uat,3.8.2.8,org.cgl.kfs.mas.mashonline,mashonline-ui-static
uat,soe1.1,deploy-mash-app22-uat-has,1.0.1.RC1,org.cgl.kfs.mas.mashonline,realtime_balances_mims_feeder
stg,soe1.1,deploy-coin-app2-stg,1.1.2,org.mbl.coin.ui.visormobile,vm-web-ui
stg,soe1.1,deploy-coin-app2-stg,1.2.14,org.mbl.coin.ui.factfind,factfind-web-ui

尝试了多种方法，包括 StackOverflow 中的几乎所有帖子。我想要的 D3 JSON 是这样的:

{
    "name": "flare",
    "children": [
        {
            "name": "uat",
            "children": [
                {
                    "name": "soe1.1",
                    "children": [
                        {
                            "name": "deploy-mash-app40-uat",
                            "children": [
                                {
                                    "name": "mashonline-ui-static",
                                    "children": [
                                        {
                                            "name": "com.cgl.bfs.mas.mashonline",
                                            "size": 3938
                                        },
                                        {
                                            "name": "3.8.2.8",
                                            "size": 3812
                                        }
                                    ]
                                }
                            ]
                        },
                        {
                            "name": "deploy-mash-app22-uat-has",
                            "children": [
                                {
                                    "name": "realtime_balances_mims_feeder",
                                    "children": [
                                        {
                                            "name": "1.0.1.RC1",
                                            "size": 3534
                                        },
                                        {
                                            "name": "com.cgl.bfs.mas.mashonline",
                                            "size": 5731
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ]
        },
        {
            "name": "stg",
            "children": [
                {
                    "name": "soe1.1",
                    "children": [
                        {
                            "name": "deploy-coin-app2-stg",
                            "children": [
                                {
                                    "name": "vm-web-ui",
                                    "children": [
                                        {
                                            "name": "1.1.2",
                                            "size": 3812
                                        },
                                        {
                                            "name": "com.mbl.coin.ui.visormobile",
                                            "size": 6714
                                        }
                                    ]
                                },
                                {
                                    "name": "factfind-web-ui",
                                    "children": [
                                        {
                                            "name": "1.2.14",
                                            "size": 5731
                                        },
                                        {
                                            "name": "com.mbl.coin.ui.factfind",
                                            "size": 7840
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

基本上，将最后两列值作为第 4 列的兄弟列。提前致谢(我也是 python 新手)。

尝试过 Link1 Link2和很多其他链接，但我无法使其工作

我运行的代码如下(感谢上述链接之一)，但我发现在到达单元格时很难添加“name”、“children”节点。

import json
import csv

tree = {}
name = "name"
children = "children"
reader = csv.reader(open("cleaned_new_test.txt", 'rb'))
reader.next()
for row in reader:
    print tree
    subtree = tree
    for i, cell in enumerate(row):
        if cell:
            if cell not in subtree:
                subtree[cell] = {} if i<len(row)-1 else 1
                print subtree
            subtree = subtree[cell]

print json.dumps(tree, indent=4)

Answer 1

以下是从 csv 文件获取 json 的一种方法:

import csv
from collections import OrderedDict
import json

def fmt(d):
    l = []
    for (k,v) in d.items():
        j = OrderedDict()
        j['name'] = k
        if isinstance(v, dict):
            j['children'] = fmt(v)
        elif isinstance(v, list):
            for (k,v) in v:
                j[k] = v
        l.append(j)
    return l

# Build OrderedDict
d1 = OrderedDict()
with open('input.txt') as f:
    reader = csv.reader(f, )
    for row in reader:
        print(row)
        # Extract the columns you want to use as "leaves"
        leaves = [row[-2], row[-3]]
        for l in leaves: row.remove(l)
        # Build a dictionary based on remaining row elements
        ctx = d1
        for e in row:
            if e not in ctx: ctx[e] = OrderedDict()
            ctx = ctx[e]
        # Re-insert leaves
        for l in leaves:
            ctx[l] = None

print(json.dumps(d1, indent=4))
print('---')


# Insert missing items (ctx = context)
ctx = d1['uat']['soe1.1']['deploy-mash-app40-uat']['mashonline-ui-static']
ctx['org.cgl.kfs.mas.mashonline']   = [('size', 3938)]
ctx['3.8.2.8']                      = [('size', 3812)]

ctx = d1['uat']['soe1.1']['deploy-mash-app22-uat-has']['realtime_balances_mims_feeder']
ctx['1.0.1.RC1']                    = [('size', 3534)]
ctx['org.cgl.kfs.mas.mashonline']   = [('size', 5731)]

ctx = d1['stg']['soe1.1']['deploy-coin-app2-stg']['vm-web-ui']
ctx['1.1.2']                        = [('size', 3812)]
ctx['org.mbl.coin.ui.visormobile']  = [('size', 6714)]

ctx = d1['stg']['soe1.1']['deploy-coin-app2-stg']['factfind-web-ui']
ctx['1.2.14']                       = [('size', 5731)]
ctx['org.mbl.coin.ui.factfind']     = [('size', 7840)]

# Wrap "formatted" in another dictionary
d2 = {"name": "flare", "children": fmt(d1)}

j = json.dumps(d2, indent=4)
print(j)

输出:

{    "name": "flare",     "children": [        {            "name": "uat",             "children": [                {                    "name": "soe1.1",                     "children": [                        {                            "name": "deploy-mash-app40-uat",                             "children": [                                {                                    "name": "mashonline-ui-static",                                     "children": [                                        {                                            "name": "org.cgl.kfs.mas.mashonline",                                             "size": 3938                                        },                                         {                                            "name": "3.8.2.8",                                             "size": 3812                                        }                                    ]                                }                            ]                        },                         {                            "name": "deploy-mash-app22-uat-has",                             "children": [                                {                                    "name": "realtime_balances_mims_feeder",                                     "children": [                                        {                                            "name": "org.cgl.kfs.mas.mashonline",                                             "size": 5731                                        },                                         {                                            "name": "1.0.1.RC1",                                             "size": 3534                                        }                                    ]                                }                            ]                        }                    ]                }            ]        },         {            "name": "stg",             "children": [                {                    "name": "soe1.1",                     "children": [                        {                            "name": "deploy-coin-app2-stg",                             "children": [                                {                                    "name": "vm-web-ui",                                     "children": [                                        {                                            "name": "org.mbl.coin.ui.visormobile",                                             "size": 6714                                        },                                         {                                            "name": "1.1.2",                                             "size": 3812                                        }                                    ]                                },                                 {                                    "name": "factfind-web-ui",                                     "children": [                                        {                                            "name": "org.mbl.coin.ui.factfind",                                             "size": 7840                                        },                                         {                                            "name": "1.2.14",                                             "size": 5731                                        }                                    ]                                }                            ]                        }                    ]                }            ]        }    ]}

It's not the prettiest, but it gets the job done.

Some notes:

• Adding the size elements after the fact is ugly, there might be a better way to do this. (I'm referring to the code that starts with the comment "Insert missing items"). In this section, you can specify additional key:value pairs to add as a list (key,value) 2-tuples.

• This section could have been written as:

  # Insert missing items (ctx = context)
  d1['uat']['soe1.1']['deploy-mash-app40-uat']['mashonline-ui-static']['org.cgl.kfs.mas.mashonline']              = [('size', 3938)]
  d1['uat']['soe1.1']['deploy-mash-app40-uat']['mashonline-ui-static']['3.8.2.8']                                 = [('size', 3812)]
  d1['uat']['soe1.1']['deploy-mash-app22-uat-has']['realtime_balances_mims_feeder']['1.0.1.RC1']                  = [('size', 3534)]
  d1['uat']['soe1.1']['deploy-mash-app22-uat-has']['realtime_balances_mims_feeder']['org.cgl.kfs.mas.mashonline'] = [('size', 5731)]
  d1['stg']['soe1.1']['deploy-coin-app2-stg']['vm-web-ui']['1.1.2']                                               = [('size', 3812)]
  d1['stg']['soe1.1']['deploy-coin-app2-stg']['vm-web-ui']['org.mbl.coin.ui.visormobile']                         = [('size', 6714)]
  d1['stg']['soe1.1']['deploy-coin-app2-stg']['factfind-web-ui']['1.2.14']                                        = [('size', 5731)]
  d1['stg']['soe1.1']['deploy-coin-app2-stg']['factfind-web-ui']['org.mbl.coin.ui.factfind']                      = [('size', 7840)]
  

  (没有 ctx 引用内容)。我只是使用定义的 ctx 作为字典结构中的位置，然后使用它来设置更深的字典值，以使行更短且更易于管理。

• 预期的 json 好多了，但仍然有点偏差。也就是说，您指定像 com.cgl.bfs.mas.mashonline 这样的标识符，但您的 csv 具有 org.cgl.bfs.mas.mashonline (“com”与“org” ”)。此外，json 中“leaf”元素的顺序不一致。在我的脚本输出的 json 中，第 5 列出现在第 4 列之前。在您的 json 中，第一个元素以这种方式出现，但最后三个元素出现的顺序交换了(4 在 5 之前)。如果您想要此交换顺序，请更改:

  leaves = [row[-2], row[-3]]
  

  至

  leaves = [row[-3], row[-2]]