python - 如何让这个过程更快？

Question

我正在尝试从大型 csv 文件(大约 250 000 行)填充数组列表，但这需要很长时间。我确信有一种方法可以使该过程更快，但我不知道如何!

这是代码:

import csv
import numpy as np

energy = []
ondeIG =[]
time =[]
envelope = []

with open('file.csv') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:        
        time = np.hstack([time, row['Time']])
        energy = np.hstack([energy, row['Energy']])
        ondeIG = np.hstack([ondeIG, row['OndeIG']])
        envelope = np.hstack([envelope, row['envelope']])

谢谢!

Answer 1

np.hstack() 每次都会构造一个新的 ndarray，这是昂贵的。您可以通过附加就地更新列表:

with open('file.csv') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:        
        time.append(row['Time'])
        energy.append(row['Energy'])
        ondeIG.append(row['OndeIG'])
        envelope.append(row['envelope'])