python - 在大数据集上使用 csv 来分隔参数时遇到问题

Question

我正在处理一个大型数据集 (OMNI)，我正在寻找解析数据并将每一行数据放入 数组 列表中的方法。我是 Python 的新手，所以我边学边学。

这是我所拥有的:

import Tkinter, tkFileDialog
import csv 

#Choose the file that you want to read from
root = Tkinter.Tk()
root.withdraw()


file_path = tkFileDialog.askopenfilename()
current_file = open(file_path , "r")

#OMNI_2001 = {}

reader = csv.reader(current_file, delimiter= ' ')

output_file = open('newdata.txt','w')
out = csv.writer(output_file)

for row in reader:
    out.writerow(row)
    print row
#print row[0::1]

我读入的一行数据是这样的:

2001 182  0  0 60 60   7   2  71   -695    320  0.22   -173    6.07    5.23    0.46   -2.00    0.69   -1.93    0.38    2.09   331.0  -329.5    24.5    19.8   8.66  101479.  1.90   0.64   2.25   8.0    6.67   29.65    3.55   12.73   -1.78   -0.70   288  -142   146    -3   -22    20    19   0.99

但是在我输出新数据之后是这样的:

2001,182,,0,,0,60,60,,,7,,,2,,71,,,-695,,,,320,,0.22,,,-173,,,,6.07,,,,5.23,,,,0.46,,,-2.00,,,,0.69,,,-1.93,,,,0.38,,,,2.09,,,331.0,,-329.5,,,,24.5,,,,19.8,,,8.66,,101479.,,1.90,,,0.64,,,2.25,,,8.0,,,,6.67,,,29.65,,,,3.55,,,12.73,,,-1.78,,,-0.70,,,288,,-142,,,146,,,,-3,,,-22,,,,20,,,,19,,,0.99

我在做什么导致这么多额外的逗号？另外，我将如何删除不需要的条目？

Answer 1

您的 csv 文件在项目之间有多个空格。 delimiter=' ' 使读者将每个 空格视为新列的分隔符。这就是为什么这些行有这么多“额外”列的原因。

使用skipinitialspace=True导致紧跟在定界符后面的空格被忽略。这将消除虚假的额外列。

import Tkinter, tkFileDialog
import csv 

#Choose the file that you want to read from
root = Tkinter.Tk()
root.withdraw()

file_path = tkFileDialog.askopenfilename()
with open(file_path , 'rb') as current_file:
    reader = csv.reader(current_file, delimiter= ' ', 
                        skipinitialspace=True)
    with open('newdata.txt','wb') as output_file:
        out = csv.writer(output_file)
        for row in reader:
            out.writerow(row)
            print row
            #print row[0::1]