python - 具有 mse 损失的 TensorFlow 未初始化值错误

Question

我正在尝试使用 TensorFlow r1.2 训练具有 mse 损失函数的自动编码器，但我不断收到 FailedPreconditionError，它指出与计算 mse 相关的变量之一未初始化(请参阅下面的完整堆栈跟踪打印输出)。我在 Jupyter notebook 中运行它，我使用的是 Python 3。

我将我的代码缩减为一个最小的示例，如下所示

import tensorflow as tf
import numpy as np
from functools import partial


# specify network

def reset_graph(seed=0):
    tf.reset_default_graph()
    tf.set_random_seed(seed)
    np.random.seed(seed)
reset_graph()

n_inputs = 100
n_hidden = 6
n_outputs = n_inputs

learning_rate = 0.001
l2_reg = 0.001

X = tf.placeholder(tf.float32, shape=[None, n_inputs])

he_init = tf.contrib.layers.variance_scaling_initializer()
l2_regularizer = tf.contrib.layers.l2_regularizer(l2_reg)
my_dense_layer = partial(tf.layers.dense,
                         activation=tf.nn.elu,
                         kernel_initializer=he_init,
                         kernel_regularizer=l2_regularizer)

hidden1 = my_dense_layer(X, n_hidden1)
outputs = my_dense_layer(hidden1, n_outputs, activation=None)

reconstruction_loss = tf.reduce_mean(tf.metrics.mean_squared_error(X, outputs))

reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
loss = tf.add_n([reconstruction_loss] + reg_losses)

optimizer = tf.train.AdamOptimizer(learning_rate)
training_op = optimizer.minimize(loss)

init = tf.global_variables_initializer()


# generate 1000 random examples 

sample_X = np.random.rand(1000, 100)


# train network

n_epochs = 10
batch_size = 50
with tf.Session() as sess:
    sess.run(init) # init.run()
    for epoch in range(n_epochs):
        n_batches = sample_X.shape[0] // batch_size
        for iteration in range(n_batches):
            start_idx = iteration*batch_size
            if iteration == n_batches-1:
                end_idx = sample_X.shape[0]
            else:
                end_idx = start_idx + batch_size
            sys.stdout.flush()   

            X_batch = sample_X[start_idx:end_idx]
            sess.run(training_op, feed_dict={X: X_batch})

            loss_train = reconstruction_loss.eval(feed_dict={X: X_batch})
            print(round(loss_train, 5))

当我将定义 reconstruction_loss 的行替换为不使用 tf.metrics 时，如下所示

reconstruction_loss = tf.reduce_mean(tf.square(tf.norm(outputs - X)))

我没有得到异常。

我检查了几个类似的 SO 问题，但没有一个能解决我的问题。例如，一个可能的原因，建议在 FailedPreconditionError: Attempting to use uninitialized in Tensorflow 的答案中, 未能初始化 TF 图中的所有变量，但我的脚本使用 init = tf.global_variables_initializer() 初始化所有 TF 变量，然后使用 sess.run(init) .另一个可能的原因是 Adam 优化器创建了自己的变量，需要在指定优化器后对其进行初始化(参见 Tensorflow: Using Adam optimizer )。但是，我的脚本在优化器之后定义了变量初始值设定项，正如该问题的已接受答案中所建议的那样，所以这也不是我的问题。

任何人都可以发现我的脚本有什么问题或提出一些建议来尝试找出此错误的原因吗？

下面是错误的堆栈跟踪。

---------------------------------------------------------------------------
FailedPreconditionError                   Traceback (most recent call last)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
   1138     try:
-> 1139       return fn(*args)
   1140     except errors.OpError as e:

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
   1120                                  feed_dict, fetch_list, target_list,
-> 1121                                  status, run_metadata)
   1122 

~\AppData\Local\Continuum\Anaconda3\lib\contextlib.py in __exit__(self, type, value, traceback)
     88             try:
---> 89                 next(self.gen)
     90             except StopIteration:

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py in raise_exception_on_not_ok_status()
    465           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 466           pywrap_tensorflow.TF_GetCode(status))
    467   finally:

FailedPreconditionError: Attempting to use uninitialized value mean_squared_error/total
     [[Node: mean_squared_error/total/read = Identity[T=DT_FLOAT, _class=["loc:@mean_squared_error/total"], _device="/job:localhost/replica:0/task:0/cpu:0"](mean_squared_error/total)]]

During handling of the above exception, another exception occurred:

FailedPreconditionError                   Traceback (most recent call last)
<ipython-input-55-aac61c488ed8> in <module>()
     64             sess.run(training_op, feed_dict={X: X_batch})
     65 
---> 66             loss_train = reconstruction_loss.eval(feed_dict={X: X_batch})
     67             print(round(loss_train, 5))

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in eval(self, feed_dict, session)
    604 
    605     """
--> 606     return _eval_using_default_session(self, feed_dict, self.graph, session)
    607 
    608 

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in _eval_using_default_session(tensors, feed_dict, graph, session)
   3926                        "the tensor's graph is different from the session's "
   3927                        "graph.")
-> 3928   return session.run(tensors, feed_dict)
   3929 
   3930 

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata)
    787     try:
    788       result = self._run(None, fetches, feed_dict, options_ptr,
--> 789                          run_metadata_ptr)
    790       if run_metadata:
    791         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    995     if final_fetches or final_targets:
    996       results = self._do_run(handle, final_targets, final_fetches,
--> 997                              feed_dict_string, options, run_metadata)
    998     else:
    999       results = []

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1130     if handle is None:
   1131       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1132                            target_list, options, run_metadata)
   1133     else:
   1134       return self._do_call(_prun_fn, self._session, handle, feed_dict,

~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
   1150         except KeyError:
   1151           pass
-> 1152       raise type(e)(node_def, op, message)
   1153 
   1154   def _extend_graph(self):

FailedPreconditionError: Attempting to use uninitialized value mean_squared_error/total
     [[Node: mean_squared_error/total/read = Identity[T=DT_FLOAT, _class=["loc:@mean_squared_error/total"], _device="/job:localhost/replica:0/task:0/cpu:0"](mean_squared_error/total)]]

Caused by op 'mean_squared_error/total/read', defined at:
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\__main__.py", line 3, in <module>
    app.launch_new_instance()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
    app.start()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 474, in start
    ioloop.IOLoop.instance().start()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\zmq\eventloop\ioloop.py", line 177, in start
    super(ZMQIOLoop, self).start()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\ioloop.py", line 888, in start
    handler_func(fd_obj, events)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 276, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 228, in dispatch_shell
    handler(stream, idents, msg)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 390, in execute_request
    user_expressions, allow_stdin)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 501, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2698, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2802, in run_ast_nodes
    if self.run_code(code, result):
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2862, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-55-aac61c488ed8>", line 32, in <module>
    reconstruction_loss = tf.reduce_mean(tf.metrics.mean_squared_error(X, outputs))
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\metrics_impl.py", line 1054, in mean_squared_error
    updates_collections, name or 'mean_squared_error')
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\metrics_impl.py", line 331, in mean
    total = _create_local('total', shape=[])
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\metrics_impl.py", line 196, in _create_local
    validate_shape=validate_shape)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1679, in variable
    caching_device=caching_device, name=name, dtype=dtype)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\variables.py", line 200, in __init__
    expected_shape=expected_shape)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\variables.py", line 319, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 1303, in identity
    result = _op_def_lib.apply_op("Identity", input=input, name=name)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value mean_squared_error/total
     [[Node: mean_squared_error/total/read = Identity[T=DT_FLOAT, _class=["loc:@mean_squared_error/total"], _device="/job:localhost/replica:0/task:0/cpu:0"](mean_squared_error/total)]]

Answer 1

看起来你在初始化时所做的一切都是正确的，所以我怀疑你的错误是你错误地使用了 tf.metrics.mean_squared_error。

类的指标包允许您计算一个值，而且还可以通过多次调用 sess.run 来累积该值。注意文档中 tf.metrics.mean_square_error 的返回值:

https://www.tensorflow.org/api_docs/python/tf/metrics/mean_squared_error

您会返回两个mean_square_error，正如您预期的那样，以及一个update_op。 update_op 的目的是让 tensorflow 计算 update_op 并累积均方误差。每次调用 mean_square_error 时，您都会获得累加值。当您想要重置该值时，您可以运行 sess.run(tf.local_variables_initializer())(注意是局部的而不是全局的，以清除度量包定义的“局部”变量)。

我认为 metrics 包不打算按照您使用它的方式使用。我认为您的意图是仅根据当前批处理计算 mse 作为您的损失，而不是通过多次调用累积值(value)。我什至不确定对于这样的累积值，差异化会如何发挥作用。

所以我认为您问题的答案是:不要以这种方式使用指标包。使用指标进行报告，并在测试数据集的多次迭代中累积结果，例如，而不是用于生成损失函数。

我想你的意思是使用tf.losses.mean_squared_error