tensorflow - 将 tensorflow 数据格式从 nhwc( channel 在后)切换到 nchw( channel 在前)时出现问题

Question

我目前正在尝试为 OCR 任务训练 CRNN(卷积循环神经网络)。我通常保持默认的数据格式，即 NHWC( channel 最后)。但是由于一些部署限制，我想使用 NCHW( channel 优先)数据格式。

为此，我使用了 tensorflow.keras.backend.set_image_data_format('channels_first')。

我正在通过使用相应地更改我的数据管道:

if my_configs.data_format == 'channels_first':
    img = tf.transpose(img, [2, 0, 1])

我的模型有卷积层，后面跟着双向 lstm 层。这是我的模型的一部分:

    conv_7 = Conv2D(64, (2,2), activation = "relu")(pool_6)
    
    if data_format == 'channels_first':
        squeezed = Lambda(lambda x: tf.squeeze(x, 2))(conv_7)
        # squeezed = Reshape((1,64,31))(squeezed)
    else:
        squeezed = Lambda(lambda x: tf.squeeze(x, 1))(conv_7)

    if data_format == "channels_first":
        use_bias = False
    else:
        use_bias = True
    # bidirectional LSTM layers with units=128
    blstm_1 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(squeezed)
    blstm_2 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(blstm_1)

现在，这是我的问题。如果我在我的 LSTM 层中保持偏差，所以 use_bias=True，那么我会得到一个关于尺寸的错误。这是错误:

Traceback (most recent call last):
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1812, in _create_c_op
    c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be at least rank 3 but is rank 2 for '{{node BiasAdd}} = BiasAdd[T=DT_FLOAT, data_format="NCHW"](add, bias)' with input shapes: [?,512], [512].      

During handling of the above exception, another exception occurred:       

Traceback (most recent call last):   
  File "run_training.py", line 292, in <module>
    model = model_architecture_1_bis(len(char_list), data_format)
  File "D:\GdPicture\dev\MachineLearning\OCR\ModelsArchitectures.py", line 178, in model_architecture_1_bis    
    blstm_1 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(squeezed)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 530, in __call__
    return super(Bidirectional, self).__call__(inputs, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 925, in __call__       
    return self._functional_construction_call(inputs, args, kwargs,       
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1117, in _functional_construction_call
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 643, in call
    y = self.forward_layer(forward_inputs,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 663, in __call__        
    return super(RNN, self).__call__(inputs, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 925, in __call__       
    return self._functional_construction_call(inputs, args, kwargs,       
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1117, in _functional_construction_call
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1183, in call        
    runtime) = lstm_with_backend_selection(**normal_lstm_kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1557, in lstm_with_backend_selection
    last_output, outputs, new_h, new_c, runtime = defun_standard_lstm(    
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 2828, in __call__
    graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 3213, in _maybe_define_function 
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 3065, in _create_graph_function 
    func_graph_module.func_graph_from_py_func(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1305, in standard_lstm
    last_output, outputs, new_states 
= K.rnn(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\backend.py", line 4213, in rnn
    output_time_zero, _ = step_function(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1293, in step        
    z = K.bias_add(z, bias)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\backend.py", line 5772, in bias_add
    return nn.bias_add(x, bias, data_format='NCHW')
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 3366, in bias_add
    return gen_nn_ops.bias_add(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", 
line 693, in bias_add
    _, _, _op, _outputs = _op_def_library._apply_op_helper(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 742, in _apply_op_helper
    op = g._create_op_internal(op_type_name, inputs, dtypes=None,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\func_graph.py", line 591, in _create_op_internal
    return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 3477, in _create_op_internal     
    ret = Operation(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1974, in __init__
    self._c_op = _create_c_op(self._graph, node_def, inputs,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1815, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be at least rank 3 but is rank 2 for '{{node BiasAdd}} = BiasAdd[T=DT_FLOAT, data_format="NCHW"](add, bias)' with input shapes: [?,512], [512].

另一方面，如果我消除了偏差，所以 use_bias=False，那么我没有得到关于维度的错误，但我的模型很糟糕。我得到了大约 50% 的准确率(在这种情况下，准确率意味着我正确地识别了一个词)。我认为这些糟糕的结果是由于没有在我的 LSTM 层中使用偏差，但我可能是错的。

所以我的问题是，消除偏差对结果的影响有那么大吗？另外，当我使用偏差时，您认为尺寸误差的来源是什么？

Answer 1

So my question is, can removing the bias affect the results that much?

我怀疑偏见是造成如此低绩效的原因。如果我们退后一步看看网络中发生了什么，每一层都在应用一系列 W * X + B 形式的线性变换，后跟或不跟非线性变换 tanh, sigmoid....

线性部分 W * X 是应用多种变换的数学等价物，例如:拉伸(stretch)空间、收缩空间、旋转....

这张图片来自优秀的3 Blue 1 Brown channel

X + B 等同于在空间中移动原点，其他条件相同。

从深度学习的角度来看，这意味着梯度中的很大一部分方差位于空间中的一个方向，并且通过简单地沿该方向平移输入，您可以显着降低成本函数(即:与所有可能的线性和非线性组合)。

这并非不可能，但似乎不太可能发生。

And also, what do you think is the source of that dimensions error when I use the bias?

查看评论以获得更精确的信息。