Python:滑动窗口均值，忽略缺失数据

Question

我目前正在尝试处理一个包含缺失值的实验性时间序列数据集。我想计算这个数据集随时间的滑动窗口平均值，同时处理 nan 值。我这样做的正确方法是在每个窗口内计算有限元素的总和并将其除以它们的数量。这种非线性迫使我使用非卷积方法来面对这个问题，因此我在这部分过程中遇到了严重的时间瓶颈。作为我试图完成的代码示例，我提供以下内容:

import numpy as np
#Construct sample data
n = 50
n_miss = 20
win_size = 3
data= np.random.random(50)
data[np.random.randint(0,n-1, n_miss)] = None

#Compute mean
result = np.zeros(data.size)
for count in range(data.size):
    part_data = data[max(count - (win_size - 1) / 2, 0): min(count + (win_size + 1) / 2, data.size)]
    mask = np.isfinite(part_data)
    if np.sum(mask) != 0:
        result[count] = np.sum(part_data[mask]) / np.sum(mask)
    else:
        result[count] = None
print 'Input:\t',data
print 'Output:\t',result

输出:

Input:  [ 0.47431791  0.17620835  0.78495647  0.79894688  0.58334064  0.38068788
  0.87829696         nan  0.71589171         nan  0.70359557  0.76113969
  0.13694387  0.32126573  0.22730891         nan  0.35057169         nan
  0.89251851  0.56226354  0.040117           nan  0.37249799  0.77625334
         nan         nan         nan         nan  0.63227417  0.92781944
  0.99416471  0.81850753  0.35004997         nan  0.80743783  0.60828597
         nan  0.01410721         nan         nan  0.6976317          nan
  0.03875394  0.60924066  0.22998065         nan  0.34476729  0.38090961
         nan  0.2021964 ]
Output: [ 0.32526313  0.47849424  0.5867039   0.72241466  0.58765847  0.61410849
  0.62949242  0.79709433  0.71589171  0.70974364  0.73236763  0.53389305
  0.40644977  0.22850617  0.27428732  0.2889403   0.35057169  0.6215451
  0.72739103  0.49829968  0.30119027  0.20630749  0.57437567  0.57437567
  0.77625334         nan         nan  0.63227417  0.7800468   0.85141944
  0.91349722  0.7209074   0.58427875  0.5787439   0.7078619   0.7078619
  0.31119659  0.01410721  0.01410721  0.6976317   0.6976317   0.36819282
  0.3239973   0.29265842  0.41961066  0.28737397  0.36283845  0.36283845
  0.29155301  0.2021964 ]

是否可以在不使用 for 循环的情况下通过 numpy 操作生成此结果？

Answer 1

您可以使用 rolling 来做到这一点 Pandas 的功能:

import numpy as np
import pandas as pd

#Construct sample data
n = 50
n_miss = 20
win_size = 3
data = np.random.random(n)
data[np.random.randint(0, n-1, n_miss)] = None

windowed_mean = pd.Series(data).rolling(window=win_size, min_periods=1).mean()

print(pd.DataFrame({'Data': data, 'Windowed mean': windowed_mean}) )

输出:

        Data  Windowed mean
0   0.589376       0.589376
1   0.639173       0.614274
2   0.343534       0.524027
3   0.250329       0.411012
4   0.911952       0.501938
5        NaN       0.581141
6   0.224964       0.568458
7        NaN       0.224964
8   0.508419       0.366692
9   0.215418       0.361918
10       NaN       0.361918
11  0.638118       0.426768
12  0.587478       0.612798
13  0.097037       0.440878
14  0.688689       0.457735
15  0.858593       0.548107
16  0.408903       0.652062
17  0.448993       0.572163
18       NaN       0.428948
19  0.877453       0.663223
20       NaN       0.877453
21       NaN       0.877453
22  0.021798       0.021798
23  0.482054       0.251926
24  0.092387       0.198746
25  0.251766       0.275402
26  0.093854       0.146002
27       NaN       0.172810
28       NaN       0.093854
29       NaN            NaN
30  0.965669       0.965669
31  0.695999       0.830834
32       NaN       0.830834
33       NaN       0.695999
34       NaN            NaN
35  0.613727       0.613727
36  0.837533       0.725630
37       NaN       0.725630
38  0.782295       0.809914
39       NaN       0.782295
40  0.777429       0.779862
41  0.401355       0.589392
42  0.491709       0.556831
43  0.127813       0.340292
44  0.781625       0.467049
45  0.960466       0.623301
46  0.637618       0.793236
47  0.651264       0.749782
48  0.154911       0.481264
49  0.159145       0.321773