python : Find Duplicate Items

Question

我在 csv 的列中有数据。我有一个来自它的两列的数组。我正在使用 list 的列表。我有这样的字符串列表 [[A,Bcdef],[Z,Wexy]我想识别重复条目，即 [A,Bcdef] 和 [A,Bcdef]

import csv
import StringIO
import os, sys
import hashlib
from collections import Counter
from collections import defaultdict
from itertools import takewhile, count
columns = defaultdict(list)
with open('person.csv','rU') as f:
    reader = csv.DictReader(f) # read rows into a dictionary format
listoflists = [];
    for row in reader: # read a row as {column1: value1, column2: value2,...}

        a_list = [];
        for (c,n) in row.items():
            if c =="firstName":
                try:
                    a_list.append(n[0])
                except IndexError:
                    pass    
        for (c,n) in row.items():
            if c=="lastName":
                try:
                    a_list.append(n);
                except IndexError:
                    pass
        #print list(a_list); 
        listoflists.append(a_list);
        #i += 1      
    print len(listoflists);

我已经尝试了这里提出的几个解决方案

• 使用set (listoflist) 总是返回:unhashable type: 'list'
• 函数:返回:“列表”对象没有属性“值”

例如:

results = list(filter(lambda x: len(x) > 1, dict1.values()))
if len(results) > 0:
        print('Duplicates Found:')
        print('The following files are identical.  the content is   identical')
        print('___________________')
        for result in results:
            for subresult in result:
                print('\t\t%s' % subresult)
            print('___________________')
    else:
        print('No duplicate files found.')

欢迎提出任何建议。

Answer 1

除了列表，您还可以使用可散列的元组。