java - 有没有办法为 TensorFlow 进程设置 JAVA_OPTS

Question

现在我们使用 TensorFlow 将模型加载到 HDFS 中。该模型大于 1G，并且在达到堆大小时抛出 OOM。

2018-05-21 13:26:34.339483: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
18/05/21 13:26:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
INFO:tensorflow:Restoring parameters from hdfs://namenode:port/saved_model/0/variables/variables
INFO:tensorflow:Restoring parameters from hdfs://namenode:port/saved_model/0/variables/variables
2018-05-21 13:26:40.479367: W tensorflow/core/framework/allocator.cc:101] Allocation of 411041792 exceeds 10% of system memory.
hdfsPread: NewByteArray error:
java.lang.OutOfMemoryError: Java heap space

TensorFlow 现在正在使用 libhdfs，它可能会公开一些使用参数或环境变量设置 JAVA_OPTS 的方法。

Answer 1

最后，这将适用于 libhdfs:export LIBHDFS_OPTS="-Xms2048m -Xmx2048m"。