python - Apache MXNet - 胶子和模块之间的转换(反之亦然)？

Question

我想知道如何在两个版本之间进行转换，因为似乎量化功能主要用于 syms, arg_params, aux_params 元组样式传递，可以很好地包裹模块，但是不是胶子模型(如果我错了请纠正我)。

这是一个训练 cnn 模型的小代码片段:

batch_size = 64
num_inputs = 784
num_outputs = 10
data_iter = mx.io.NDArrayIter(x, y, batch_size=batch_size)

num_fc = 512
net = gluon.nn.HybridSequential()
with net.name_scope():
    net.add(gluon.nn.Conv2D(channels=20, kernel_size=5, activation='relu'))
    net.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))
    net.add(gluon.nn.Conv2D(channels=50, kernel_size=5, activation='relu'))
    net.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))
    net.add(gluon.nn.Flatten())
    net.add(gluon.nn.Dense(num_fc, activation="relu"))
    net.add(gluon.nn.Dense(num_outputs))

net.hybridize()
# Parameter initialization
net.collect_params().initialize(mx.init.Xavier(magnitude=2.24), ctx=ctx)
trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': .1})
softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()
for i, batch in enumerate(data_iter):
    data = batch.data[0].as_in_context(ctx)
    label = batch.label[0].as_in_context(ctx)
    with autograd.record():
        output = net(data)
        loss = softmax_cross_entropy(output, label)
    loss.backward()
    trainer.step(data.shape[0])

如果我想量化胶子模型，我会尝试将胶子序列化到磁盘中，然后将其作为模块带回。这可能会引起麻烦:

import os
net.export('mxnet')
mod = mx.module.Module.load('mxnet', 0) # 0 epoch

根据模块 API:

mod.bind( data_shapes = data_iter.provide_data, 
          label_shapes = data_iter.provide_label)
mod.predict(x)

但它在预测时不起作用，具有以下堆栈跟踪:

----------------------------------------------
KeyError     Traceback (most recent call last)
<ipython-input-10-f53137bb5e95> in <module>()
      1 mod.bind( data_shapes = data_iter.provide_data, 
----> 2           label_shapes = data_iter.provide_label)
      3 mod.predict(x)

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/module.py in bind(self, data_shapes, label_shapes, for_training, inputs_need_grad, force_rebind, shared_module, grad_req)
    434                                                      fixed_param_names=self._fixed_param_names,
    435                                                      grad_req=grad_req, group2ctxs=self._group2ctxs,
--> 436                                                      state_names=self._state_names)
    437         self._total_exec_bytes = self._exec_group._total_exec_bytes
    438         if shared_module is not None:

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/executor_group.py in __init__(self, symbol, contexts, workload, data_shapes, label_shapes, param_names, for_training, inputs_need_grad, shared_group, logger, fixed_param_names, grad_req, state_names, group2ctxs)
    281 
    282         eprint(sys._getframe().f_lineno, data_shapes, label_shapes)
--> 283         self.bind_exec(data_shapes, label_shapes, shared_group)
    284 
    285     def decide_slices(self, data_shapes):

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/executor_group.py in bind_exec(self, data_shapes, label_shapes, shared_group, reshape)
    388         if label_shapes is not None:
    389             self.label_names = [i.name for i in self.label_shapes]
--> 390         self._collect_arrays()
    391 
    392     def reshape(self, data_shapes, label_shapes):

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/executor_group.py in _collect_arrays(self)
    324             self.label_arrays = [[(self.slices[i], e.arg_dict[name])
    325                                   for i, e in enumerate(self.execs)]
--> 326                                  for name, _ in self.label_shapes]
    327         else:
    328             self.label_arrays = None

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/executor_group.py in <listcomp>(.0)
    324             self.label_arrays = [[(self.slices[i], e.arg_dict[name])
    325                                   for i, e in enumerate(self.execs)]
--> 326                                  for name, _ in self.label_shapes]
    327         else:
    328             self.label_arrays = None

~/anaconda3/envs/idp3/lib/python3.6/site-packages/mxnet/module/executor_group.py in <listcomp>(.0)
    323                 eprint(323, e.arg_dict.keys())
    324             self.label_arrays = [[(self.slices[i], e.arg_dict[name])
--> 325                                   for i, e in enumerate(self.execs)]
    326                                  for name, _ in self.label_shapes]
    327         else:

KeyError: 'softmax_label'

这是提示我在 e.arg_dict 中缺少该标签。

我打印出了e.arg_dict:

(['数据'、'hybridsequential1_conv0_weight'、'hybridsequential1_conv0_bias'、'hybridsequential1_conv1_weight'、'hybridsequential1_conv1_bias'、'hybridsequential1_dense0_weight'、'hybridsequential1_dense0_bias'、'hybridsequential1_dense1_weight' , 'hybridsequential1_dense1_bias'])

事实上，softmax_label 并不在那里。这个标签来自哪里以及如何正确地将模块转换为胶子？

Answer 1

对于您问题的第一部分(标签来自哪里？):

当您将 label_shapes = data_iter.provide_label 参数添加到 mod.bind 调用时，默认情况下会添加 softmax 标签。您可以通过显式设置 label_shapes = None 来删除它。查看https://discuss.mxnet.io/t/gluon-module-what-is-label-name-and-why-do-i-need-labels-for-modules-to-run-bind/1433的答案了解详情。

对于问题的第二部分(如何正确地将模块转换为胶子模型？):

要将符号模型转换为胶子模型，您可以

• 使用 mod.save_checkpoint 将符号模型保存到磁盘或mod.save_params
• 使用 gluon 重新创建网络架构或重用net
• 使用 net.load_params(filename, ctx=ctx) 加载参数

例如:

mod.save_params('mxnet.params')
net2 = gluon.nn.HybridSequential()
with net2.name_scope():
    net2.add(gluon.nn.Conv2D(channels=20, kernel_size=5, activation='relu'))
    net2.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))
    net2.add(gluon.nn.Conv2D(channels=50, kernel_size=5, activation='relu'))
    net2.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))
    net2.add(gluon.nn.Flatten())
    net2.add(gluon.nn.Dense(num_fc, activation="relu"))
    net2.add(gluon.nn.Dense(num_outputs))

net2.hybridize()
net2.load_params('mxnet.params', ctx=ctx)