tensorflow - 使用 Tensorflow 后端运行 Keras 时如何获得可重现的结果

Question

每次我在jupyter笔记本中使用Keras运行LSTM网络时，我都会得到不同的结果，我在Google上搜索了很多，并且尝试了一些不同的解决方案，但它们都不起作用，以下是我尝试过的一些解决方案:

1. 设置 numpy 随机种子

   random_seed=2017
从 numpy.random 导入种子
种子(random_seed)

2. 设置 tensorflow 随机种子

   来自tensorflow导入set_random_seed
set_random_seed(random_seed)

3. 设置内置随机种子

   随机导入
random.seed(random_seed)

4. 设置 PYTHONHASHSEED

   导入操作系统
os.environ['PYTHONHASHSEED'] = '0'

5. 在 jupyter Notebook kernel.json 中添加 PYTHONHASHSEED

   {
“语言”:“ python ”，
"display_name": "Python 3",
“env”:{“PYTHONHASHSEED”:“0”}，
“argv”:[
“Python”，
“-m”，
“ipykernel_launcher”，
“-F”，
“{连接文件}”
]
}

我的环境版本是:

Keras: 2.0.6
Tensorflow: 1.2.1
CPU or GPU: CPU

这是我的代码:

model = Sequential()
model.add(LSTM(16, input_shape=(time_steps,nb_features), return_sequences=True))
model.add(LSTM(16, input_shape=(time_steps,nb_features), return_sequences=False))
model.add(Dense(8,activation='relu'))        
model.add(Dense(1,activation='linear'))
model.compile(loss='mse',optimizer='adam')

Answer 1

您的模型定义中肯定缺少种子。详细文档可以在这里找到:https://keras.io/initializers/ 。

本质上，您的层使用随机变量作为其参数的基础。因此每次都会得到不同的输出。

一个例子:

model.add(Dense(1, activation='linear', 
               kernel_initializer=keras.initializers.RandomNormal(seed=1337),
               bias_initializer=keras.initializers.Constant(value=0.1))

Keras 本身在其常见问题解答部分中有一个关于获得可重现结果的部分:( https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development )。他们有以下代码片段来产生可重现的结果:

import numpy as np
import tensorflow as tf
import random as rn

# The below is necessary in Python 3.2.3 onwards to
# have reproducible behavior for certain hash-based operations.
# See these references for further details:
# https://docs.python.org/3.4/using/cmdline.html#envvar-PYTHONHASHSEED
# https://github.com/fchollet/keras/issues/2280#issuecomment-306959926

import os
os.environ['PYTHONHASHSEED'] = '0'

# The below is necessary for starting Numpy generated random numbers
# in a well-defined initial state.

np.random.seed(42)

# The below is necessary for starting core Python generated random numbers
# in a well-defined state.

rn.seed(12345)

# Force TensorFlow to use single thread.
# Multiple threads are a potential source of
# non-reproducible results.
# For further details, see: https://stackoverflow.com/questions/42022950/which-seeds-have-to-be-set-where-to-realize-100-reproducibility-of-training-res

session_conf = tf.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)

from keras import backend as K

# The below tf.set_random_seed() will make random number generation
# in the TensorFlow backend have a well-defined initial state.
# For further details, see: https://www.tensorflow.org/api_docs/python/tf/set_random_seed

tf.set_random_seed(1234)

sess = tf.Session(graph=tf.get_default_graph(), config=session_conf)
K.set_session(sess)