python - 合并多个 CSV 文件并按字段删除重复项

Question

我需要匹配多个 CSV 文件中的数据。例如，如果我有三个 CSV 文件。

输入1个csv

PANYNJ LGA WEST 1,available, LGA West GarageFlushing
PANYNJ LGA WEST 4,unavailable,LGA West Garage
iPark - Tesla,unavailable,530 E 80th St

输入2个csv

PANYNJ LGA WEST 4,unavailable,LGA West Garage
PANYNJ LGA WEST 5,available,LGA West Garage

输入3个csv

PANYNJ LGA WEST 5,available,LGA West Garage
imPark - Tesla,unavailable,611 E 83rd St

第一列是名称，第二列是状态，最后一列是地址。如果这三个文档具有相同的名称，我想将它们合并到一个 csv 文件中。我想要的输出文件是这样的

输出csv

PANYNJ LGA WEST 1,available, LGA West GarageFlushing
PANYNJ LGA WEST 4,unavailable,LGA West Garage
iPark - Tesla,unavailable,530 E 80th St
PANYNJ LGA WEST 5,available,LGA West Garage
imPark - Tesla,unavailable,611 E 83rd St

我正在尝试使用 pandas 或 CSV 修复此问题，但我不确定如何解决此问题。

非常感谢任何帮助!

Answer 1

使用pandas，您可以使用pd.concat，然后使用pd.drop_duplicates:

import pandas as pd
from io import StringIO

str1 = StringIO("""PANYNJ LGA WEST 1,available, LGA West GarageFlushing
PANYNJ LGA WEST 4,unavailable,LGA West Garage
iPark - Tesla,unavailable,530 E 80th St""")

str2 = StringIO("""PANYNJ LGA WEST 4,unavailable,LGA West Garage
PANYNJ LGA WEST 5,available,LGA West Garage""")

str3 = StringIO("""PANYNJ LGA WEST 5,available,LGA West Garage
imPark - Tesla,unavailable,611 E 83rd St""")

# replace str1, str2, str3 with 'file1.csv', 'file2.csv', 'file3.csv'
df1 = pd.read_csv(str1, header=None)
df2 = pd.read_csv(str2, header=None)
df3 = pd.read_csv(str3, header=None)

res = pd.concat([df1, df2, df3], ignore_index=True)\
        .drop_duplicates(0)

print(res)

                   0            1                         2
0  PANYNJ LGA WEST 1    available   LGA West GarageFlushing
1  PANYNJ LGA WEST 4  unavailable           LGA West Garage
2      iPark - Tesla  unavailable             530 E 80th St
4  PANYNJ LGA WEST 5    available           LGA West Garage
6     imPark - Tesla  unavailable             611 E 83rd St