tensorflow - 如何通过 tensorflow 实现成对损失函数？

Question

我正在通过 tensorflow 实现定制的成对损失函数。举个简单的例子，训练数据有5个实例，其标签为

y=[0,1,0,0,0]

假设预测是

y'=[y0',y1',y2',y3',y4']

在这种情况下，一个简单的损失函数可能是

min f=(y0'-y1')+(y2'-y1')+(y3'-y1')+(y4'-y1')

自y[1]=1开始。我只是想确保预测 y0',y2',y3',y4' 与 y1' 一样“远”。

但是，我不知道如何在tensorflow中实现它。在我当前的实现中，我使用小批量并将训练标签设置为占位符，例如:y = tf.placeholder("float", [无, 1])。在这种情况下，我无法构造损失函数，因为我不知道训练数据的大小，也不知道哪个实例因“无”而具有标签“1”或“0”。

谁能建议如何在 tensorflow 中做到这一点？谢谢!

Answer 1

您可以在模型外部预处理数据。

例如:

首先将正例和负例分为 2 组输入:

# data.py

import random

def load_data(data_x, data_y):
    """
    data_x: list of all instances
    data_y: list of their labels
    """
    pos_x = []
    neg_x = []
    for x, y in zip(data_x, data_y):
        if y == 1:
            pos_x.append(x)
        else:
            neg_x.append(x)

    ret_pos_x = []
    ret_neg_x = []

    # randomly sample k negative instances for each positive one
    for x0 in pos_x:
        for x1 in random.sample(neg_x, k):
            ret_pos_x.append(x0)
            ret_neg_x.append(x1)

    return ret_pos_x, ret_neg_x

接下来，在您的模型中定义 2 个占位符，而不是 1 个:

# model.py

import tensorflow as tf

class Model:
    def __init__(self):
        # shape: [batch_size, dim_x] (assume x are vectors of dim_x)
        self.pos_x = tf.placeholder(tf.float32, [None, dim_x])  
        self.neg_x = tf.placeholder(tf.float32, [None, dim_x])

        # shape: [batch_size]
        # NOTE: variables in some_func should be reused
        self.pos_y = some_func(self.pos_x)
        self.neg_y = some_func(self.neg_x)

        # A more generalized form: loss = max(0, margin - y+ + y-)
        self.loss = tf.reduce_mean(tf.maximum(0.0, 1.0 - self.pos_y + self.neg_y))
        self.train_op = tf.train.AdamOptimizer(learning_rate).minimize(self.loss)

最后迭代您的数据以提供模型:

# main.py

import tensorflow as tf 

from model import Model
from data import load_data

data_x, data_y = ...  # read from your file
pos_x, neg_x = load_data(data_x, data_y)

model = Model()
with tf.Session() as sess:
    # TODO: randomize the order
    for beg in range(0, len(pos_x), batch_size):
        end = min(beg + batch_size, len(pos_x))

        feed_dict = {
            model.pos_x: pos_x[beg:end],
            model.neg_x: neg_x[beg:end]
        }
        _, loss = sess.run([model.train_op, model.loss], feed_dict)
        print "%s/%s, loss = %s" % (beg, len(pos_x), loss)