python - 使用 Numpy 进行最大池化反向传播

Question

我正在使用 Numpy 实现一个 CNN，但我找不到一种方法来像前向传播那样有效地实现最大池化的反向传播。
这就是我在前向传播中所做的:

def run(self, x, is_training=True):
    """
    Applying MaxPooling on `x`
    :param x: input - [n_batches, channels, height, width]
    :param is_training: a boolean indicating whether training or not
    :return: output of the MaxPooling on `x`
    """
    n_batch, ch_x, h_x, w_x = x.shape
    h_poolwindow, w_poolwindow = self.pool_size

    out_h = int((h_x - h_poolwindow) / self.stride) + 1
    out_w = int((w_x - w_poolwindow) / self.stride) + 1

    windows = as_strided(x,
                         shape=(n_batch, ch_x, out_h, out_w, *self.pool_size),
                         strides=(x.strides[0], x.strides[1],
                                  self.stride * x.strides[2],
                                  self.stride * x.strides[3],
                                  x.strides[2], x.strides[3])
                         )
    out = np.max(windows, axis=(4, 5))

    if is_training:
        self.cache['X'] = x
    return out

我当前的反向传播实现:

def backprop(self, dA_prev):
    """
    Backpropagation in a max-pooling layer
    :return: the derivative of the cost layer with respect to the current layer
    """
    x = self.cache['X']
    n_batch, ch_x, h_x, w_x = x.shape
    h_poolwindow, w_poolwindow = self.pool_size

    dA = np.zeros(shape=x.shape)  # dC/dA --> gradient of the input
    for n in range(n_batch):
        for ch in range(ch_x):
            curr_y = out_y = 0
            while curr_y + h_poolwindow <= h_x:
                curr_x = out_x = 0
                while curr_x + w_poolwindow <= w_x:
                    window_slice = x[n, ch, curr_y:curr_y + h_poolwindow, curr_x:curr_x + w_poolwindow]
                    i, j = np.unravel_index(np.argmax(window_slice), window_slice.shape)
                    dA[n, ch, curr_y + i, curr_x + j] = dA_prev[n, ch, out_y, out_x]

                    curr_x += self.stride
                    out_x += 1

                curr_y += self.stride
                out_y += 1
    return dA

我可以矢量化吗？

Answer 1

我设法通过将前向传播更改为:

windows = as_strided(x,
                     shape=(n_batch, ch_x, out_h, out_w, *self.pool_size),
                     strides=(x.strides[0], x.strides[1],
                              self.stride * x.strides[2],
                              self.stride * x.strides[3],
                              x.strides[2], x.strides[3])
                     )
out = np.max(windows, axis=(4, 5))

maxs = out.repeat(2, axis=2).repeat(2, axis=3)
x_window = x[:, :, :out_h * self.stride, :out_w * self.stride]
mask = np.equal(x_window, maxs).astype(int)

if is_training:
    self.cache['X'] = x
    self.cache['mask'] = mask
return out

并将反向传播更改为:

mask = self.cache['mask']
dA = dA_prev.repeat(h_poolwindow, axis=2).repeat(w_poolwindow, axis=3)
dA = np.multiply(dA, mask)
pad = np.zeros(x.shape)
pad[:, :, :dA.shape[2], :dA.shape[3]] = dA
return pad