Python 文件预处理(将列从离散值范围转换为连续值范围。)

Question

我有以下形式的数据集:

user_id::item_id1::rating::timestamp
user_id::item_id2::rating::timestamp
user_id::item_id3::rating::timestamp
user_id::item_id4::rating::timestamp

我要求 item_ids(按排序顺序有 n 个不同的项目 ID。后续行可以具有相同的项目 ID 或不同但保证排序)从 1 到 n 是连续的，它们当前的范围是从 1 到k

对于 k >> n

我有以下代码，但它不是很正确，我已经用了几个小时了，所以非常感谢对此的任何帮助，或者如果有更简单的方法可以在 python 中执行此操作，我将非常感谢指导对此也是如此。

我目前有以下代码:

def reOrderItemIds(inputFile,outputFile):
        #This is a list in the range of 1 to 10681.
        itemIdsRange = set(range(1,10682))
        #currKey = 1
        currKey = itemIdsRange.pop()
        lastContiguousKey=1
        #currKey+1
        contiguousKey=itemIdsRange.pop()
        f = open(inputFile)
        g = open(outputFile,"w")
        oldKeyToNewKeyMap = dict()
        for line in f:
                if int(line.split(":")[1]) == currKey and int(line.split(":")[1])==lastContiguousKey:
                        g.write(line)
                elif int(line.split(":")[1])!=currKey and int(line.split(":")[1])!=contiguousKey:
                        oldKeyToNewKeyMap[line.split(":")[1]]=contiguousKey
                        lastContiguousKey=contiguousKey
                        #update current key to the value of the current key.
                        currKey=int(line.split(":")[1])
                        contiguousKey=itemIdsRange.pop()
                        g.write(line.split(":")[0]+":"+str(lastContiguousKey)+":"+line.split(":")[2]+":"+line.split(":")[3])
                elif int(line.split(":")[1])==currKey and int(line.split(":")[1])!=contiguousKey:
                        g.write(line.split(":")[0]+":"+str(lastContiguousKey)+":"+line.split(":")[2]+":"+line.split(":")[3])

                elif int(line.split(":")[1])!=currKey and int(line.split(":")[1])==contiguousKey:
                        currKey = int(line.split(":")[1])
                        lastContiguousKey=contiguousKey
                        oldKeyToNewKeyMap[line.split(":")[1]] = lastContiguousKey
                        contiguousKey=itemIdsRange.pop()
                        g.write(line.split(":")[0]+":"+str(lastContiguousKey)+":"+line.split(":")[2]+":"+line.split(":")[3])
        f.close()
        g.close()

例子:

1::1::3::100
10::1::5::104
20::2::3::110
1::5::2::104

我要求输出的形式是:

1::1::3::100
10::1::5::104
20::2::3::110
1::3::2::104

因此只有 item_ids 列发生变化，其他所有内容都保持不变。

如有任何帮助，我们将不胜感激!

Answer 1

因为您的数据已经按 item_id 排序 - 您可以使用 itertools.groupby() 来简化解决方案。

from operator import itemgetter
from itertools import groupby

item_id = itemgetter(1)
def reOrderItemIds(inputFile,outputFile):
    n = 1
    with open(inputFile)as infile, open(outputFile,"w") as outfile:
        dataset = (line.split('::') for line in infile)
        for key, group in groupby(dataset, item_id):
            for line in group:
                line[1] = str(n)
                outfile.write('::'.join(line))
            n += 1