python - 根据日期和时间分割数据

Question

当前正在编写带有示例数据的代码。我需要根据时间分离数据。

当前的标题和行如下所示

 time,pressure,temperature,salinity,density
 9/12/2014 0:00,176.31,4.5914,34.90789056,1028.46834
 .
 .

时间采用“ mm / dd / yyy HH：mm”格式，如2014年9月12日0:00


我想用日期列mm / dd / yyy和HH：mm分割时间

我想编写一个例程，其中：对于日期mm / dd / yyy，如果行之间的时间变化<30分钟，则当行之间的时间> 30分钟将后续数据输出到outfile2时，将此数据输出到outfile1


我想为每个日期循环一次，每次向文本文件添加行时都不要覆盖它们，因此：

outfile1

 time,pressure,temperature,salinity,density
 9/12/2014 0:00,176.31,4.5914,34.90789056,1028.46834
 .
 .
 9/13/2014 0:00,176.31,4.5914,34.90789056,1028.46834
 .
 .

outfile2

 time,pressure,temperature,salinity,density
 9/12/2014 2:00,176.31,4.5914,34.90789056,1028.46834
 .
 .
 9/13/2014 2:00,176.31,4.5914,34.90789056,1028.46834

这是尝试拆分在不同时间池化的数据。有问题的设备每天两次，每秒两次池数据。代码样本将其平均为分钟，然后更改定界符。

这里的问题是找到一个有效的代码，如果第（i）行和第（j）行之间的HH：mm之差> 30分钟，则该代码将首先分割日期和时间，然后分割mm / dd / yyy日期
将这些值添加为outfile2中的条目

低于已写的内容。这只是目前平均输入条目到秒到分钟数据的代码。我想将下面提到的步骤（1）合并到下面的代码中。然后，一旦处理完成，写步骤（2）

    import pandas as pd

    df = pd.read_csv('C:\\Users\\ctaghili\\Desktop\\2014_15s.txt', index_col=1, 
    sep=",")
    # Read in file with time being the index

    name_change = {'time': 'Time', 'ctdpf_ckl_seawater_pressure': 'Pressure',
                 'ctdpf_ckl_seawater_temperature': 'Temperature',
                 'practical_salinity': 'Salinity'}
    df = df.rename(columns=name_change)
    # change the headers of each column


    df = df.rolling(60).mean()
    df = df.iloc[::60, :]
    # take the mean of every 60 entries

    select_cols = ['Pressure', 'Temperature', 'Salinity', 'density']
    df = df[select_cols]
    # data frame which contains the pressure, temp and salinity columns with 
    # the time column as the index

    data = df.iloc[1:630]
    # with the way I averaged data the first line is useless
    data.to_csv('C:\\Users\\ctaghili\\Desktop\\Output1.txt', sep='\t')

Answer 1

您不必一定要使用熊猫来完成此任务。在这里，我将指导您使用Python编写csv库。在下面的说明中，请转到底部以获取完整的代码块。

假设您有一个名为file1.csv的.csv文件

time,pressure,temperature,salinity,density
9/12/2014 0:00,176.31,4.5914,34.90789056,1028.46834
9/13/2014 0:20,176.31,4.5914,34.90789056,1028.46834
9/14/2014 0:21,176.31,4.5914,34.90789056,1028.46834
9/15/2014 2:00,176.31,4.5914,34.90789056,1028.46834
9/16/2014 1:20,176.31,4.5914,34.90789056,1028.46834
9/17/2014 0:29,176.31,4.5914,34.90789056,1028.46834
9/18/2014 1:00,176.31,4.5914,34.90789056,1028.46834
9/19/2014 3:20,176.31,4.5914,34.90789056,1028.46834

它有不同的时间。有些时间少于30分钟，有些则更多。我们如何将其分成两个单独的输出文件？

import csv

接下来，我们打开file1.csv文件并使用

with open('file1.csv') as csv_file:
    read = csv.reader(csv_file,delimiter=',')

由于存在标题，我使用了一个变量num_lines来记录我们所在的行。

with open('file1.csv') as csv_file:
    read = csv.reader(csv_file,delimiter=',')
    num_lines = 0

现在我们要创建一个writer变量来写入两个新的csv文件，我将这些文件称为out1.csv和out2.csv。 “ w”表示写。

with open('out1.csv',mode='w') as out_1,open('out2.csv',mode='w') as out_2:
            out2_w = csv.writer(out_2,delimiter=',') #make writer,with delimiter ','
            out1_w = csv.writer(out_1,delimiter=',')

最后，我们遍历每一行并检查数据和时间，以查看分钟数是否低于30。

for row in read:
     if num_lines == 0:
          out1_w.writerow(row)
          out2_w.writerow(row)
          num_lines += 1
     else:
          time = row[0].split(':')
          # time[1] is two digits,time[0][-1] is 0-9,time[0][-2] is ' ' or 1-9
          if int(time[1]) < 30 and int(time[0][-2] + time[0][-1]) == 0:
               out1_w.writerow(row)
          else:
               out2_w.writerow(row)

现在在一起：

import csv

with open('file1.csv') as csv_file:
    read = csv.reader(csv_file,delimiter=',')
    num_lines = 0
    with open('out1.csv',mode='w') as out_1,open('out2.csv',mode='w') as out_2:
         out2_w = csv.writer(out_2,delimiter=',')
         out1_w = csv.writer(out_1,delimiter=',')
         for row in read:
              if num_lines == 0:
                   out1_w.writerow(row)
                   out2_w.writerow(row)
                   num_lines += 1
              else:
                  # example: row[0] = '9/12/2014 0:00'
                  time = row[0].split(':')
                  # example: time = ['9/12/2014 0','00']
                  if int(time[1]) < 30 and int(time[0][-2] + time[0][-1]) == 0:
                       out1_w.writerow(row)
                  else:
                       out2_w.writerow(row)

此产品

cat out1.csv
time,pressure,temperature,salinity,density
9/12/2014 0:00,176.31,4.5914,34.90789056,1028.46834
9/13/2014 0:20,176.31,4.5914,34.90789056,1028.46834
9/14/2014 0:21,176.31,4.5914,34.90789056,1028.46834
9/17/2014 0:29,176.31,4.5914,34.90789056,1028.46834
cat out2.csv
time,pressure,temperature,salinity,density
9/15/2014 2:00,176.31,4.5914,34.90789056,1028.46834
9/16/2014 1:20,176.31,4.5914,34.90789056,1028.46834
9/18/2014 1:00,176.31,4.5914,34.90789056,1028.46834
9/19/2014 3:20,176.31,4.5914,34.90789056,1028.46834

现在，您可以将pandas用作pandas.read_csv并将其转换为数据帧。但是从这里开始你应该很好。