python - 如何通过检查从 CSV 文件中删除重复项？

Question

我有一个像这样的 CSV:

col-1(ID)       col-2(val-List)

1               [1]
1               [1,2,3]
2               [1,2]
2               [1]
3               [10]
3               [10]

我想从此文件中删除重复项，最后，我只需要一个具有更大列表长度的行，例如:

编辑:

如果内部列表的 ID 和长度相同，我想保留一行。

col-1(ID)       col-2(Val-List)

1               [1,2,3]
2               [1,2]
3               [10]

我尝试了很多但没有成功:我正在尝试使用 CSV 模块，但不知道我应该如何保持先前 Val-List 的长度并与下一个匹配 ID 进行比较。

import csv 
list_1 = []
with open('test123.csv', 'r', encoding='latin-1') as file:
    csvReader = csv.reader(file, delimiter=',')

    for row in csvReader:
        key = (row[0])
        # but how should I use this id to get my desired results?

Answer 1

为什么不让 pandas 来做这些工作？

import pandas

# Read in the CSV
df = pandas.read_csv('test123.csv', encoding='latin-1')

# Compute the list lengths
df['lst_len'] = df['col-2(val-List)'].map(lambda x: len(list(x)))

# Sort in reverse order by list lengths
df = df.sort_values('lst_len', ascending=False)

# Drop duplicates, preserving first (longest) list by ID
df = df.drop_duplicates(subset='col-1(ID)')

# Remove extra column that we introduced, write to file
df = df.drop('lst_len', axis=1)
df.to_csv('clean_test123.csv', index=False)