python - 在 csv 文件中创建重复行以分隔列中的多个值 (python)

Question

我正在尝试在 Python 中构建一些代码，以将列中的多个值分隔成单独的行，并根据时间戳的同一天聚合 Active-Ticket 的列，是否有任何内部可以使用库还是需要安装外部库？

我的示例文件是(目前，Active-Tickets 列为空):

输入.csv

Timestamp,CaseID,Active-Tickets   
14FEB2017:10:55:23,K456 G578 T213,        
13FEB2017:10:56:12,F891 A63,
14FEB2017:11:59:14,T427 T31212 F900000,
15FEB2017:03:55:23,K456 G578 T213,        
14FEB2017:05:56:12,F891 A63,

我想要实现的目标:

输出.csv

Timestamp,CaseID,Active-Tickets
14FEB2017:10:55:23,K456,8 (because there are 8 cases happened on the same day)
14FEB2017:10:55:23,G578,8
14FEB2017:10:55:23,T213,8        
13FEB2017:10:56:12,F891,2 (because there are 2 cases happened on the same day)
13FEB2017:10:56:12,A63,2
14FEB2017:11:59:14,T427,8
14FEB2017:11:59:14,T31212,8
14FEB2017:11:59:14,F900000,8
15FEB2017:03:55:23,K456,3 (because there are 3 cases happened on the same day)
15FEB2017:03:55:23,G578,3
15FEB2017:03:55:23,T213,3        
14FEB2017:05:56:12,F891,8
14FEB2017:05:56:12,A63,8

我的想法是:

1. Take the values for the column Timestamp

2. Check if the date is the same,

3. Store all of the CaseID separated by space into a list based on the date,

4. Count the number of element in the list for each date then

5. Return the values for the counted elements into Active-Tickets.

但是这里的问题是，数据量不小，假设一天最少有50个案例，那么我认为我的方式是不可能的。

Answer 1

这是使用 itertools.chain.from_iterable() 实现此目的的一种方法。它只将计数保留在内存中，因此可能适用于您的情况。它分两次读取 csv 文件。一次获取计数，一次写入输出，但仅使用迭代器进行读取，因此应降低内存需求。

代码:

import csv
import itertools as it
from collections import Counter

# read through file and get counts per date
with open('test.csv', 'rU') as f:
    reader = csv.reader(f)
    header = next(reader)
    dates = it.chain.from_iterable(
        [date for _ in ids.split()]
        for date, ids in ((x[0].split(':')[0], x[1]) for x in reader))
    counts = Counter(dates)

# read through file again, and output as individual records with counts
with open('test.csv', 'rU') as f:
    reader = csv.reader(f)
    header = next(reader)
    records = it.chain.from_iterable(
        [(l[0], d) for d in l[1].split()] for l in reader)
    new_lines = (l + (str(counts[l[0].split(':')[0]]), ) for l in records)

    with open('test2.csv', 'wb') as f_out:
        writer = csv.writer(f_out)
        writer.writerow(header)
        writer.writerows(new_lines)

结果:

Timestamp,CaseID,Active-Tickets
14FEB2017:10:55:23,K456,8
14FEB2017:10:55:23,G578,8
14FEB2017:10:55:23,T213,8
13FEB2017:10:56:12,F891,2
13FEB2017:10:56:12,A63,2
14FEB2017:11:59:14,T427,8
14FEB2017:11:59:14,T31212,8
14FEB2017:11:59:14,F900000,8
15FEB2017:03:55:23,K456,3
15FEB2017:03:55:23,G578,3
15FEB2017:03:55:23,T213,3
14FEB2017:05:56:12,F891,8
14FEB2017:05:56:12,A63,8

2.6 中的计数器

collections.Counter 已向后移植到 python 2.5+ ( Here )