python - 获取时间差不是给定单位的索引

Question

我必须在计算机上手动插入一些信息，因此我必须检查所有数据是否输入正确。

我的数据如下所示:

timestamp,value
2015-03-02 15:00:45,1
2015-03-02 15:01:00,2
2015-03-02 15:01:15,3
2015-03-02 15:01:30,4
2015-03-02 15:01:45,5
2015-03-02 15:02:00,6
2015-03-02 15:02:15,7
2015-03-02 15:02:30,8
2015-03-02 15:02:45,9
2015-03-02 15:03:00,10
2015-03-02 15:03:15,11
2015-03-02 15:03:30,12
2015-03-02 15:03:45,13
2015-03-02 15:04:00,14
2015-03-02 15:04:15,15
2015-03-02 15:04:30,16
2015-03-02 15:04:45,17
2015-03-02 15:05:00,18
2015-03-02 17:00:45,19
2015-03-02 17:01:00,20
2015-03-02 17:01:15,21
2015-03-02 17:01:30,22
2015-03-02 17:01:45,23
2015-03-02 17:02:00,24
2015-03-02 17:02:15,25
2015-03-02 17:02:30,26
2015-03-02 17:02:45,27
2015-03-02 17:03:00,28
2015-03-02 17:03:15,29
2015-03-02 17:03:30,30
2015-03-02 17:03:45,31
2015-03-02 17:04:00,32
2015-03-02 17:04:15,33
2015-03-02 17:04:30,34
2015-03-02 17:44:15,33
2015-03-02 17:44:30,34
2015-03-02 17:44:45,35
2015-03-02 17:45:00,36
2015-03-02 17:45:15,37
2015-03-02 17:45:30,38
2015-03-02 17:45:45,39
2015-03-02 17:46:00,40
2015-03-02 17:46:15,41
2015-03-17 15:00:45,1
2015-03-17 15:01:00,2
2015-03-17 15:01:15,3
2015-03-17 15:01:30,4
2015-03-17 15:01:45,5
2015-03-17 15:02:00,6
2015-03-17 15:02:15,7
2015-03-17 15:02:30,8
2015-03-17 15:02:45,9
2015-03-17 15:03:00,10
2015-03-17 15:03:15,11
2015-03-17 15:03:30,12
2015-03-17 15:03:45,13
2015-03-17 15:04:00,14
2015-03-17 15:04:15,15
2015-03-17 15:04:30,16
2015-03-17 15:04:45,17
2015-03-17 15:05:00,18
2015-03-17 17:00:45,19
2015-03-17 17:01:00,20
2015-03-17 17:01:15,21
2015-03-17 17:01:30,22
2015-03-17 17:01:45,23
2015-03-17 17:02:00,24
2015-03-17 17:02:15,25
2015-03-17 17:02:30,26
2015-03-17 17:02:45,27
2015-03-17 17:03:00,28
2015-03-17 17:03:15,29
2015-03-17 17:03:30,30
2015-03-17 17:03:45,31
2015-03-17 17:04:00,32
2015-03-17 17:04:15,33
2015-03-17 17:04:30,34
2015-03-17 17:44:15,33
2015-03-17 17:44:30,34
2015-03-17 17:44:45,35
2015-03-17 17:45:00,36
2015-03-17 17:45:15,37
2015-03-17 17:45:30,38
2015-03-17 17:45:45,39
2015-03-17 17:46:00,40
2015-03-17 17:46:15,41

我想要的输出应该是这样的:奇数表示一个区间的开始，甚至最后一个(仍然包括在内)。

2015-03-02 15:00:45,1
2015-03-02 15:05:00,18
2015-03-02 17:00:45,19
2015-03-02 17:04:30,34
2015-03-02 17:44:15,33
2015-03-02 17:46:15,41
2015-03-17 15:00:45,1
2015-03-17 15:05:00,18
2015-03-17 17:00:45,19
2015-03-17 17:04:30,34
2015-03-17 17:44:15,33
2015-03-17 17:46:15,41

通过此方法，我们可以查看数据事务和重新输入是否有效。

我到目前为止的尝试都不起作用，因为它们没有正确地设置所有断点。

mintime = pd.to_datetime(tiere.loc[(tiere.timestamp.shift(-1)-tiere.timestamp)>"00:01:00","timestamp"].values[0:],format="%Y-%m-%d %H:%M:%S").sort_values()
#add to time max and get unique timestamps and sort them works only if tiere resample is NOT ON!!!
maxtime = pd.to_datetime(tiere.loc[(tiere.timestamp-tiere.timestamp.shift(1))>"00:01:00","timestamp"].values[0:],format="%Y-%m-%d %H:%M:%S").sort_values()
#add to time min and get unique timestamps and sort them. works only if tiere resample is NOT ON!!!
min2 = (pd.to_datetime(tiere.loc[(tiere.timestamp.shift(1)-tiere.timestamp)>"00:01:00","timestamp"].values[0:],format="%Y-%m-%d %H:%M:%S").sort_values())
#add to time max and get unique timestamps and sort them works only if tiere resample is NOT ON!!!
max2 = (pd.to_datetime(tiere.loc[(tiere.timestamp-tiere.timestamp.shift(-1))>"00:01:00","timestamp"].values[0:],format="%Y-%m-%d %H:%M:%S").sort_values())
breakpoints = mintime.union(mintimestamp_tiere).union(min2).union(maxtime).union(maxtimestamp_tiere).union(forgottentimedates).union(max2).delete(7)

Answer 1

您可以使用diff ，而不是用减法来代替 shift，并使用 Timedelta 创建一个 mask将差异与一分钟(或任何时间差异)进行比较。为了确保获得数据帧的第一行和最后一行，符号 ~ 用于获取差异小于一分钟的反向选择，例如:

tiere.timestamp = pd.to_datetime(tiere.timestamp) #convert the data to datetime first

mask = (~(tiere.timestamp.diff() < pd.Timedelta(minutes=1))|
        ~(tiere.timestamp.diff(-1).abs() < pd.Timedelta(minutes=1)))
breakpoints = tiere[mask]

print (breakpoints )
             timestamp  value
0  2015-03-02 15:00:45      1
17 2015-03-02 15:05:00     18
18 2015-03-02 17:00:45     19
33 2015-03-02 17:04:30     34
34 2015-03-02 17:44:15     33
42 2015-03-02 17:46:15     41
43 2015-03-17 15:00:45      1
60 2015-03-17 15:05:00     18
61 2015-03-17 17:00:45     19
76 2015-03-17 17:04:30     34
77 2015-03-17 17:44:15     33
85 2015-03-17 17:46:15     41