python - 与 python 模型相比，tensorflow lite 模型给出了非常不同的精度值

Question

我正在使用 tensorflow 1.10 Python 3.6

我的代码基于预制的 iris classification model由 TensorFlow 提供。这意味着，我使用的是 Tensorflow DNN 预制分类器，具有以下区别:

• 10 个特征而不是 4 个。
• 5 节课而不是 3 节课。

测试和训练文件可以从以下链接下载: https://www.dropbox.com/sh/nmu8i2i8xe6hvfq/AADQEOIHH8e-kUHQf8zmmDMDa?dl=0

我编写了一个代码将这个分类器导出为 tflite 格式，但是 python 模型中的准确度高于 75%，但是当导出时准确度大约下降到 45%，这意味着大约 30% 的准确度丢失(这是太多了)。我已经尝试过使用不同数据集的代码，并且在所有这些代码中，导出后的准确性都下降了很多!这让我觉得 TocoConverter 函数出了点问题，或者我可能错误地导出到 tflite，缺少参数或类似的东西。

这是我生成模型的方式:

classifier = tf.estimator.DNNClassifier(
        feature_columns=my_feature_columns,
        hidden_units=[100, 500],
        optimizer=tf.train.AdagradOptimizer(learning_rate=0.003),
        n_classes=num_labels,
        model_dir="myModel")

这是我用来转换为 tflite 的函数:

converter = tf.contrib.lite.TocoConverter.from_frozen_graph(final_model_path, input_arrays, output_arrays, input_shapes={"dnn/input_from_feature_columns/input_layer/concat": [1, 10]})
        tflite_model = converter.convert()

我分享了完整的代码，其中我还计算了生成的 .tflite 文件的准确性。

import argparse
import tensorflow as tf

import pandas as pd
import csv

from tensorflow.python.tools import freeze_graph
from tensorflow.python.tools import optimize_for_inference_lib
import numpy as np


parser = argparse.ArgumentParser()
parser.add_argument('--batch_size', default=100, type=int, help='batch size')
parser.add_argument('--train_steps', default=1000, type=int,
                    help='number of training steps')

features_global = None
feature_spec = None

MODEL_NAME = 'myModel'

def load_data(train_path, test_path):
    """Returns the iris dataset as (train_x, train_y), (test_x, test_y)."""

    with open(train_path, newline='') as f:
        reader = csv.reader(f)
        column_names = next(reader)

    y_name = column_names[-1]

    train = pd.read_csv(train_path, names=column_names, header=0)
    train_x, train_y = train, train.pop(y_name)

    test = pd.read_csv(test_path, names=column_names, header=0)
    test_x, test_y = test, test.pop(y_name)

    return (train_x, train_y), (test_x, test_y)


def train_input_fn(features, labels, batch_size):
    """An input function for training"""
    # Convert the inputs to a Dataset.
    dataset = tf.data.Dataset.from_tensor_slices((dict(features), labels))

    # Shuffle, repeat, and batch the examples.
    dataset = dataset.shuffle(1000).repeat().batch(batch_size)

    # Return the dataset.
    return dataset


def eval_input_fn(features, labels, batch_size):
    """An input function for evaluation or prediction"""
    features=dict(features)
    if labels is None:
        # No labels, use only features.
        inputs = features
    else:
        inputs = (features, labels)

    # Convert the inputs to a Dataset.
    dataset = tf.data.Dataset.from_tensor_slices(inputs)

    # Batch the examples
    assert batch_size is not None, "batch_size must not be None"
    dataset = dataset.batch(batch_size)

    # Return the dataset.
    return dataset


def main(argv):
    args = parser.parse_args(argv[1:])

    train_path = "trainData.csv"
    test_path = "testData.csv"

    # Fetch the data
    (train_x, train_y), (test_x, test_y) = load_data(train_path, test_path)

    # Load labels
    num_labels = 5

    # Feature columns describe how to use the input.
    my_feature_columns = []
    for key in train_x.keys():
        my_feature_columns.append(tf.feature_column.numeric_column(key=key))

    # Build 2 hidden layer DNN
    classifier = tf.estimator.DNNClassifier(
        feature_columns=my_feature_columns,
        hidden_units=[100, 500],
        optimizer=tf.train.AdagradOptimizer(learning_rate=0.003),
        # The model must choose between 'num_labels' classes.
        n_classes=num_labels,
        model_dir="myModel")

    # Train the Model
    classifier.train(
        input_fn=lambda:train_input_fn(train_x, train_y,
                                                args.batch_size),
        steps=args.train_steps)

    # Evaluate the model.
    eval_result = classifier.evaluate(
        input_fn=lambda:eval_input_fn(test_x, test_y,
                                                args.batch_size))

    print('\nTest set accuracy: {accuracy:0.3f}\n'.format(**eval_result))

    # Export model
    feature_spec = tf.feature_column.make_parse_example_spec(my_feature_columns)
    serve_input_fun = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
    saved_model_path = classifier.export_savedmodel(
            export_dir_base="out",
            serving_input_receiver_fn=serve_input_fun,
            as_text=True,
            checkpoint_path=classifier.latest_checkpoint(),
        )
    tf.reset_default_graph()
    var = tf.Variable(0)
    with tf.Session() as sess:
        # First let's load meta graph and restore weights
        sess.run(tf.global_variables_initializer())
        latest_checkpoint_path = classifier.latest_checkpoint()
        saver = tf.train.import_meta_graph(latest_checkpoint_path + '.meta')
        saver.restore(sess, latest_checkpoint_path)

        input_arrays = ["dnn/input_from_feature_columns/input_layer/concat"]
        output_arrays = ["dnn/logits/BiasAdd"]

        frozen_graph_def = tf.graph_util.convert_variables_to_constants(
            sess, sess.graph_def,
            output_node_names=["dnn/logits/BiasAdd"])

        frozen_graph = "out/frozen_graph.pb"

        with tf.gfile.FastGFile(frozen_graph, "wb") as f:
                f.write(frozen_graph_def.SerializeToString())

        # save original graphdef to text file
        with open("estimator_graph.pbtxt", "w") as fp:
            fp.write(str(sess.graph_def))
        # save frozen graph def to text file
        with open("estimator_frozen_graph.pbtxt", "w") as fp:
            fp.write(str(frozen_graph_def))

        input_node_names = input_arrays
        output_node_name = output_arrays
        output_graph_def = optimize_for_inference_lib.optimize_for_inference(
                frozen_graph_def, input_node_names, output_node_name,
                tf.float32.as_datatype_enum)

        final_model_path = 'out/opt_' + MODEL_NAME + '.pb'
        with tf.gfile.FastGFile(final_model_path, "wb") as f:
            f.write(output_graph_def.SerializeToString())

        tflite_file = "out/iris.tflite"

        converter = tf.contrib.lite.TocoConverter.from_frozen_graph(final_model_path, input_arrays, output_arrays, input_shapes={"dnn/input_from_feature_columns/input_layer/concat": [1, 10]})
        tflite_model = converter.convert()
        open(tflite_file, "wb").write(tflite_model)

        interpreter = tf.contrib.lite.Interpreter(model_path=tflite_file)
        interpreter.allocate_tensors()

        # Get input and output tensors.
        input_details = interpreter.get_input_details()
        output_details = interpreter.get_output_details()

        # Test model on random input data.
        input_shape = input_details[0]['shape']
        # change the following line to feed into your own data.
        input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
        resultlist = list()
        df = pd.read_csv(test_path)
        expected = df.iloc[:, -1].values.tolist()
        with open(test_path, newline='') as f:
            reader = csv.reader(f)
            column_names = next(reader)
            for x in range(0, len(expected)):
                linea = next(reader)
                linea = linea[:len(linea) - 1]
                input_data2 = np.array(linea, dtype=np.float32)
                interpreter.set_tensor(input_details[0]['index'], [input_data2])
                interpreter.invoke()
                output_data = interpreter.get_tensor(output_details[0]['index'])
                #print(output_data)
                max = 0;
                longitud = len(output_data[0])

                for k in range(0, longitud):
                    if (output_data[0][k] > output_data[0][max]):
                        max = k
                resultlist.append(max)
            print(resultlist)

        coincidences = 0
        for pred_dict, expec in zip(resultlist, expected):
            if pred_dict == expec:
                coincidences = coincidences + 1

        print("tflite Accuracy: " + str(coincidences / len(expected)))


if __name__ == '__main__':
    tf.logging.set_verbosity(tf.logging.INFO)
    tf.app.run(main)

我希望你们中的一些人能够找出错误，或者给出一个可能的解决方案

Answer 1

这个问题得到了回答here可能有帮助。

在回答分享中提到，做一些

pre-processing

在将图像送入“interpreter.invoke()”之前在图像上解决问题(如果这是首先出现的问题)。

为了详细说明这里是来自共享链接的 block 引用:

The below code you see is what I meant by pre-processing:

test_image = cv2.imread(file_name)

test_image = cv2.resize(test_image,(299,299),cv2.INTER_AREA)

test_image = np.expand_dims((test_image)/255,axis=0).astype(np.float32)

interpreter.set_tensor(input_tensor_index, test_image)

interpreter.invoke()

digit = np.argmax(output()[0])

#print(digit)

prediction = result[digit]

As you can see there are two crucial commands/pre-processing done on the image once it is read using "imread()":

i) The image should be resized to the size that is the "input_height" and "input_width" values of the input image/tensor that was used during the training. In my case (inception-v3) this was 299 for both "input_height" and "input_width". (Read the documentation of the model for this value or look for this variable in the file that you used to train or retrain the model)

ii) The next command in the above code is:

test_image = np.expand_dims((test_image)/255,axis=0).astype(np.float32)

I got this from the "formulae"/model code:

test_image = np.expand_dims((test_image - input_mean)/input_std, axis=0).astype(np.float32)

Reading the documentation revealed that for my architecture input_mean = 0 and input_std = 255.

希望这对您有所帮助。