python-3.x - 如何使用tensorflow的one-hot编码正确编码标签？

Question

我一直在尝试使用 python 3.6 学习 Tensorflow，并决定使用埃塞克斯大学面部数据库 ( http://cswww.essex.ac.uk/mv/allfaces/index.html ) 的数据构建面部识别程序。到目前为止，我一直在遵循 Tensorflow 的 MNIST Expert 指南，但是当我开始测试时，每个时期的准确度都是 0，所以我知道出了问题。我对自己处理标签的方式感到最不安，所以我认为这就是问题所在。

数据集中的标签可以是数字 ID(例如 987323)，也可以是某人的姓名(例如“fordj”)。我处理这个问题的想法是创建一个“预编码”encode_labels函数，它为测试和训练集中的每个唯一标签提供了自己唯一的整数值。我检查以确保测试和训练集中的每个唯一标签都具有相同的唯一值。它还返回一个字典，以便我可以轻松地从编码版本映射回原始标签。如果我不执行此步骤并在检索标签时传递标签(即“fordj”)，则会收到一条错误消息

UnimplementedError (see above for traceback): Cast string to int32 is not supported [[Node: Cast = CastDstT=DT_INT32, SrcT=DT_STRING, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

我对此的解释是，由于许多标签是人名， tensorflow 无法将“fordj”这样的标签转换为 tf.int32。获取标签和路径的代码在这里:

def get_paths_and_labels(path):
    """ image_paths  :  list of relative image paths
        labels       :  mix of alphanumeric characters """
    image_paths = [path + image for image in os.listdir(path)]
    labels = [i.split(".")[-3] for i in image_paths]
    labels = [i.split("/")[-1] for i in labels]
    return image_paths, labels

def encode_labels(train_labels, test_labels):
    """ Assigns a numeric value to each label since some are subject's names """
    found_labels = []
    index = 0
    mapping = {}
    for i in train_labels:
        if i in found_labels:
            continue
        mapping[i] = index
        index += 1
        found_labels.append(i)
    return [mapping[i] for i in train_labels], [mapping[i] for i in test_labels], mapping

以下是我分配训练和测试标签的方法。然后我想使用tensorflow的one-hot编码器为我再次编码它们。

def main():
    # Grabs the labels and each image's relative path 
    train_image_paths, train_labels = get_paths_and_labels(TRAIN_PATH)
    # Smallish dataset so I can read it all into memory
    train_images = [cv2.imread(image) for image in train_image_paths]

    test_image_paths, test_labels = get_paths_and_labels(TEST_PATH)
    test_images = [cv2.imread(image) for image in test_image_paths]

    num_classes = len(set(train_labels))

    # Placeholders
    x = tf.placeholder(tf.float32, shape=[None, IMAGE_SIZE[0] * IMAGE_SIZE[1]])
    y_ = tf.placeholder(tf.float32, shape=[None, num_classes])
    x_image = tf.reshape(x, [-1, IMAGE_SIZE[0], IMAGE_SIZE[1], 1])

    # One-hot labels
    train_labels, test_labels, mapping = encode_labels(train_labels, test_labels)

    train_labels = tf.one_hot(indices=tf.cast(train_labels, tf.int32), depth=num_classes)
    test_labels = tf.one_hot(indices=tf.cast(test_labels, tf.int32), depth=num_classes)

我确信我做错了什么。我知道 sklearn 有一个 LabelEncoder，尽管我还没有尝试过。感谢您对此的任何建议，感谢所有帮助!

Answer 1

The way I'm interpreting this is that since many of the labels are people's names, tensorflow can't convert a label like "fordj" to a tf.int32.

你是对的。 tensorflow 无法做到这一点。相反，您可以创建从 nome 到唯一(且渐进)ID 的映射函数。完成此操作后，您就可以使用其 one-hot 表示形式对每个数字 ID 进行正确的一编码。您已经有了数字 ID 和字符串标签之间的关系，因此您可以执行以下操作:

train_labels, test_labels, mapping = encode_labels(train_labels, test_labels)
numeric_train_ids = [labels[idx] for idx in train_labels]
numeric_test_ids = [labels[idx] for idx in test_labels]     

one_hot_train_labels = tf.one_hot(indices=numeric_train_ids, depth=num_classes)
one_hot_test_labels = tf.one_hot(indices=numeric_test_ids, depth=num_classes)