tensorflow - 如何分析在 tf-serving 上运行的 tensorflow 模型？

Question

据我所知，当我在 python 脚本上运行 tensorflow 模型时，我可以使用以下代码片段来分析模型中每个 block 的时间线。

from tensorflow.python.client import timeline

options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
batch_positive_score = sess.run([positive_score], feed_dict, options=options, run_metadata=run_metadata)
fetched_timeline = timeline.Timeline(run_metadata.step_stats)
chrome_trace = fetched_timeline.generate_chrome_trace_format()
with open('./result/timeline.json', 'w') as f:
    f.write(chrome_trace)

但是如何分析加载在 tensorflow-serving 上的模型？

Answer 1

我认为您可以使用 tf.profiler，即使在服务期间也是如此，因为它最终是一个 Tensorflow 图，并且在训练期间所做的更改(根据我的理解，包括分析)也将反射(reflect)在服务中。

请找到以下 Tensorflow 代码:

# User can control the tracing steps and
# dumping steps. User can also run online profiling during training.
#
# Create options to profile time/memory as well as parameters.
builder = tf.profiler.ProfileOptionBuilder
opts = builder(builder.time_and_memory()).order_by('micros').build()
opts2 = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()

# Collect traces of steps 10~20, dump the whole profile (with traces of
# step 10~20) at step 20. The dumped profile can be used for further profiling
# with command line interface or Web UI.
with tf.contrib.tfprof.ProfileContext('/tmp/train_dir',
                                      trace_steps=range(10, 20),
                                      dump_steps=[20]) as pctx:
  # Run online profiling with 'op' view and 'opts' options at step 15, 18, 20.
  pctx.add_auto_profiling('op', opts, [15, 18, 20])
  # Run online profiling with 'scope' view and 'opts2' options at step 20.
  pctx.add_auto_profiling('scope', opts2, [20])
  # High level API, such as slim, Estimator, etc.
  train_loop()

之后，我们可以在命令提示符下运行下面提到的命令:

bazel-bin/tensorflow/core/profiler/profiler \
    --profile_path=/tmp/train_dir/profile_xx
tfprof> op -select micros,bytes,occurrence -order_by micros

# Profiler ui available at: https://github.com/tensorflow/profiler-ui
python ui.py --profile_context_path=/tmp/train_dir/profile_xx

可视化时间和内存的代码:

# The following example generates a timeline.
tfprof> graph -step -1 -max_depth 100000 -output timeline:outfile=<filename>

generating trace file.

******************************************************
Timeline file is written to <filename>.
Open a Chrome browser, enter URL chrome://tracing and load the timeline file.
******************************************************

将 TensorFlow 图运行时间归因于您的 Python 代码:

tfprof> code -max_depth 1000 -show_name_regexes .*model_analyzer.*py.* -select micros -account_type_regexes .* -order_by micros

显示您的模型变量和参数数量:

tfprof> scope -account_type_regexes VariableV2 -max_depth 4 -select params

显示最昂贵的操作类型:

tfprof> op -select micros,bytes,occurrence -order_by micros

自动配置文件:

tfprof> advise

有关这方面的更多详细信息，您可以引用以下链接:

理解本页提到的所有类=>

https://www.tensorflow.org/api_docs/python/tf/profiler

代码在下面的链接中给出了详细信息:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/profiler/README.md