python - 解析csv文件并根据相对大小将行写入文件

Question

我有一长串不同时间段的天气变量列表，我已经过滤掉了那些不符合特定标准的变量。例如，所有数据点仅位于上午 11 点(11 点)和下午 5 点(17 点)之间。 11 点到 17 点之间的数据代表单个事件，并不是每天都包含一个事件。我正在尝试确定哪些日子举办了事件。我知道，每当时间 (HH24) 列中的值低于之前的值时，就会发生一个新事件。例如。如果值 16(下午 5 点)后面跟着 11、12、13、14 或 15，那么我就知道数据已进展到新的一天和事件。

我尝试编写的代码将获取包含最后一个值的行(例如 17)并将其写入文件，然后写入下一行。这样，新的 csv 文件将包含每个事件的开始时间(和其他信息)和结束时间。我假设我需要使用 for 命令，但我不确定如何使用 csv writer 来应对这个特定的挑战。我的代码大纲如下所示，其中包含我需要帮助的部分#

import csv

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input, open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted:
    reader = csv.DictReader(input, delimiter=",", skipinitialspace=True)
    fieldnames = reader.fieldnames
    writer_wanted = csv.DictWriter(wanted, fieldnames, delimiter=",")
    writer_wanted.writeheader()

    for line_number, row in enumerate(reader):
        try:
            if float(row["HH24"]) < #the value in the subsequent row:
                writer_wanted.writerow(row) # and also write subsequent row
        except:
            print "Failed to parse line", line_number
            print row

我的数据文件如下所示。我已经展示了 HH24 从高值到低值的转变，因此您可以明白我的意思。

hd,Station Number,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local standard time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Universal coordinated time,Precipitation since last (AWS) observation in mm,Quality of precipitation since last (AWS) observation value,Air Temperature in degrees Celsius,Quality of air temperature,Air temperature (1-minute maximum) in degrees Celsius,Quality of air temperature (1-minute maximum),Air temperature (1-minute minimum) in degrees Celsius,Quality of air temperature (1-minute minimum),Wet bulb temperature in degrees Celsius,Quality of Wet bulb temperature,Wet bulb temperature (1 minute maximum) in degrees Celsius,Quality of wet bulb temperature (1 minute maximum),Wet bulb temperature (1 minute minimum) in degrees Celsius,Quality of wet bulb temperature (1 minute minimum),Dew point temperature in degrees Celsius,Quality of dew point temperature,Dew point temperature (1-minute maximum) in degrees Celsius,Quality of Dew point Temperature (1-minute maximum),Dew point temperature (1 minute minimum) in degrees Celsius,Quality of Dew point Temperature (1 minute minimum),Relative humidity in percentage %,Quality of relative humidity,Relative humidity (1 minute maximum) in percentage %,Quality of relative humidity (1 minute maximum),Relative humidity (1 minute minimum) in percentage %,Quality of Relative humidity (1 minute minimum),Wind (1 minute) speed in km/h,Wind (1 minute) speed quality,Minimum wind speed (over 1 minute) in km/h,Minimum wind speed (over 1 minute) quality,Wind (1 minute) direction in degrees true,Wind (1 minute) direction quality,Standard deviation of wind (1 minute),Standard deviation of wind (1 minute) direction quality,Maximum wind gust (over 1 minute) in km/h,Maximum wind gust (over 1 minute) quality,Visibility (automatic - one minute data) in km,Quality of visibility (automatic - one minute data),Mean sea level pressure in hPa,Quality of mean sea level pressure,Station level pressure in hPa,Quality of station level pressure,QNH pressure in hPa,Quality of QNH pressure,#
    hd,40842,2000,3,22,13,40,2000,3,22,13,40,2000,3,22,13,40,0,N,20.4,N,20.5,N,20.4,N,20.2,N,20.2,N,20.1,N,20.1,N,20.1,N,20,N,98,N,,N,,N,9,N,8,N,18,N,7,N,11,N,,N,1013.3,N,1012.2,N,1013.3,N,#
    hd,40842,2000,3,22,13,47,2000,3,22,13,47,2000,3,22,13,47,0,N,20.5,N,20.5,N,20.5,N,20.2,N,20.2,N,20.2,N,20.1,N,20.1,N,20,N,97,N,,N,,N,4,N,0,N,56,N,75,N,5,N,,N,1013.2,N,1012.1,N,1013.2,N,#
    hd,40842,2000,3,23,11,0,2000,3,23,11,0,2000,3,23,11,0,0,N,23.4,N,23.4,N,23.3,N,21.3,N,21.4,N,21.3,N,20.2,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,66,N,2,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#
    hd,40842,2000,3,23,11,1,2000,3,23,11,1,2000,3,23,11,1,0,N,23.4,N,23.4,N,23.4,N,21.4,N,21.4,N,21.3,N,20.3,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,68,N,3,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#

Answer 1

您希望在日期更改时写入一行，因此我认为最好创建一个日期变量进行比较。(我在评论中提到为什么仅比较“HH24”值无法确定何时到达新日期)

跟踪和写出前行(因为您已经处理了它)比后续行要容易得多，所以您应该这样想关于进行。类似以下内容应该有所帮助(未经测试):

...
import datetime
prev_row = None
for line_number, row in enumerate(reader):
    try:
        dt = datetime.date(year=row["Year"], month=row["Month"], day=row["day"])
        if prev_row is not None and dt > prev_row['dt']:
            writer_wanted.writerow(prev_row['row'])
            writer_wanted.writerow(row)
        prev_row = {'row':row, 'dt':dt}
    except:
        print "Failed to parse line", line_number
        print row

编辑:

您的程序中此行的第一部分:

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input

打开名为 .csv输入文件(因为 mode 是 'rb' - docs here )。

同一行的下一部分:

open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted

打开指定文件进行输出(“wb”模式) - 请参阅与上面相同的引用。

此时，变量名称input ，和wanted现在，两者都引用 file type 的对象.

您的程序接下来使用 csv以特定方式读取文件的模块，有助于解析逗号分隔的文本文件；并将该引用分配给 reader多变的。同样，它分配变量 writer_wanted到 csv.DictWriter ，这将有助于在将行写入输出文件时进行格式化，由wanted引用。 .

之后，一次读入一行:

for line_number, row in enumerate(reader):

一次一行将被写出:

writer_wanted.writerow(row)

如果您想要更多详细信息，最好的选择是阅读一些 Python 教程(Google 是您的 friend )。