python - 值错误 : Num gradients 1 generated for op name: "mask/Mask"

Question

我关注了这个link创建一个名为 mask 的自定义操作。tensorflow op的主体是

def tf_mask(x, labels, epoch_, name=None):  # add "labels" to the input
    with ops.name_scope(name, "Mask", [x, labels, epoch_]) as name:
        z = py_func(np_mask,
                [x, labels, epoch_],   # add "labels, epoch_" to the input list
                [tf.float32],
                name=name,
                grad=our_grad)
        z = z[0]
        z.set_shape(x.get_shape())
        return z

实际上几乎遵循引用的链接。但是，我遇到了这个错误:

ValueError: Num gradients 1 generated for op name: "mask/Mask"
op: "PyFunc"
input: "conv2/Relu"
input: "Placeholder_2"
input: "Placeholder_3"
attr {
  key: "Tin"
  value {
    list {
      type: DT_FLOAT
      type: DT_FLOAT
      type: DT_FLOAT
    }
  }
}
attr {
  key: "Tout"
  value {
    list {
      type: DT_FLOAT
    }
  }
}
attr {
  key: "_gradient_op_type"
  value {
    s: "PyFuncGrad302636"
  }
}
attr {
  key: "token"
  value {
    s: "pyfunc_0"
  }
}
 do not match num inputs 3

如果需要，这就是我定义 our_grad 函数来计算梯度的方式。

def our_grad(cus_op, grad):
  """Compute gradients of our custom operation.
   Args:
    param cus_op: our custom op tf_mask
    param grad: the previous gradients before the operation
   Returns:
    gradient that can be sent down to next layer in back propagation
    it's an n-tuple, where n is the number of arguments of the operation   
  """
    x = cus_op.inputs[0]
    labels = cus_op.inputs[1]
    epoch_ = cus_op.inputs[2]   
    n_gr1 = tf_d_mask(x)
    n_gr2 = tf_gradient2(x, labels, epoch_)

    return tf.multiply(grad, n_gr1) + n_gr2

和 py_func 函数(与引用的链接相同)

def py_func(func, inp, tout, stateful=True, name=None, grad=None):
    """
    I omitted the introduction to parameters that are not of interest
    :param func: a numpy function
    :param inp: input tensors
    :param grad: a tensorflow function to get the gradients (used in bprop, should be able to receive previous 
                gradients and send gradients down.)

    :return: a tensorflow op with a registered bprop method
    """
    # Need to generate a unique name to avoid duplicates:
    rnd_name = 'PyFuncGrad' + str(np.random.randint(0, 1000000))
    tf.RegisterGradient(rnd_name)(grad)
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": rnd_name}):
        return tf.py_func(func, inp, tout, stateful=stateful, name=name)

真的需要社区的帮助!

谢谢!

Answer 1

好吧，经过一番努力，我自己解决了这个问题。

错误消息是在 tensorflow 函数 gradients_impl.py 中生成的。

def _VerifyGeneratedGradients(grads, op):
  """Verify that gradients are valid in number and type.

  Args:
    grads: List of generated gradients.
    op: Operation for which the gradients where generated.

  Raises:
    ValueError: if sizes of gradients and inputs don't match.
    TypeError: if type of any gradient is not valid for its input.
  """
  if len(grads) != len(op.inputs):
    raise ValueError("Num gradients %d generated for op %s do not match num "
                     "inputs %d" % (len(grads), op.node_def, len(op.inputs)))

现在我们可以看到这个错误意味着我们的操作 Mask，我们给了它三个输入。 TensorFlow 希望我们为生成梯度 our_grad 的函数提供三个输出，每个输出 w.r.t 前向传播函数的每个输入。

请注意，实际上我们只需要第一个输入的梯度，在反向传播中将不再使用其他两个梯度，我们可以再返回两个具有正确形状的 fake_gradients。