tensorflow - tf.dataset、多个路径输入以及每批映射以加载图像

Question

我正在加载包含多个输入图像的数据集。输入图像路径只能在批处理时解码，以便处理大型数据集。

数据集是N个图像路径输入和M个浮点输出。每个输入的图像具有不同的分辨率。

Data is ([img_input_1.png, img_input_2.png, ...], [0.65, 0.7, 0.8])

该模型在符号模式下使用 Keras 功能 api。

这是最近编辑的代码

from itertools import zip_longest

def read_image(path, shape):
    try:
        image = tf.io.read_file(path)
        image = tf.image.decode_png(image)
        image = tf.image.resize(image, [shape[1],shape[2]])
        image /= 255.0
        return image
    except:
        print('ERROR: preprocess_image: bad path', path)    

def load_image(x, y, shp):
    pout = [(k, x[k]) for k in x.keys()]
    l1   = tf.convert_to_tensor(list(x))
    l2   = tf.convert_to_tensor(list(x.values()))

    pl = tf.map_fn(
        lambda args: (read_image(args[0], shp), args[1]), [l1, l2], dtype=(tf.float32, tf.float32)
    )
    pl = {path: (pl[0][i], pl[1][i]) for i, path in enumerate(x)}
    return (pl,y)

def dataset_prep(json_data, seq, batch_size):
    # LOAD DATA FROM JSON
    x,y = json_parse_x_y(json_data[seq])
    xx  = [*zip_longest(*x)] # NOTE: goes from variable sized input to {'input_N':...}
    yy  = [*zip_longest(*y)]

    # GET SHAPES (hard coded atm)
    lns = [[len(xxx)] for xxx in xx]
    rzs = [[24,512,1],[96,512,1]] # TEMP TODO! grab grom [(v['h'],v['w'],v['c']) for v in xx]
    shp = [*zip_longest(*[lns,rzs])]
    shp = [list(s) for s in shp]
    shp = [[*itertools.chain.from_iterable(s)] for s in shp]

    xd  = dict([[ "input_{}".format(i+1),np.array(y)] for i,y in [*enumerate(xx)]])
    yd  = dict([["output_{}".format(i+1),np.array(y)] for i,y in [*enumerate(yy)]])

    ds  = tf.data.Dataset.from_tensor_slices((xd, yd))

    ds  = ds.shuffle(10000)
    ds  = ds.repeat()
    ds  = ds.map(map_func=lambda x,y: load_image(x, y, shp), num_parallel_calls=AUTOTUNE)
    ds  = ds.batch(batch_size) if batch_size else ds 
    ds  = ds.prefetch(AUTOTUNE) 
    return ds

这是我收到的错误:

Traceback (most recent call last):
    File "/home/me/.local/bin/wavfeat", line 11, in <module>
        load_entry_point('wavfeat==0.1.0', 'console_scripts', 'wavfeat')()
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/__main__.py", line 91, in main
        analysis_batch_sql(obj)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 50, in analysis_batch_sql
        qy = [*map(lambda c: run_elm(c[0], c[1]), ch)]
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 50, in <lambda>
        qy = [*map(lambda c: run_elm(c[0], c[1]), ch)]
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 23, in run_elm
        out = fn(input, elm)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 196, in one_sec_onset_train
        return train(input, elm)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 182, in train
        ts = dataset_prep(jd, 'train', bc)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 123, in dataset_prep
        ds  = ds.map(map_func=lambda x,y: load_image(x, y, shp), num_parallel_calls=AUTOTUNE)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1146, in map
        self, map_func, num_parallel_calls, preserve_cardinality=True)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3264, in __init__
        use_legacy_function=use_legacy_function)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2591, in __init__
        self._function = wrapper_fn._get_concrete_function_internal()
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1366, in _get_concrete_function_internal
        *args, **kwargs)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1360, in _get_concrete_function_internal_garbage_collected
        graph_function, _, _ = self._maybe_define_function(args, kwargs)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1648, in _maybe_define_function
        graph_function = self._create_graph_function(args, kwargs)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1541, in _create_graph_function
        capture_by_value=self._capture_by_value),
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 716, in func_graph_from_py_func
        func_outputs = python_func(*func_args, **func_kwargs)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2585, in wrapper_fn
        ret = _wrapper_helper(*args)
    File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2530, in _wrapper_helper
        ret = func(*nested_args)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 123, in <lambda>
        ds  = ds.map(map_func=lambda x,y: load_image(x, y, shp), num_parallel_calls=AUTOTUNE)
    File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_data_loader.py", line 91, in load_image
        print("x['input_1'].values(): ", x['input_1'].values())
AttributeError: 'Tensor' object has no attribute 'values'

我在做什么导致路径无法加载？

编辑:

尝试潘德雷的修复时，我遇到输入错误。这是 from_tensor_slices 和 ds.map 之前以及之后的数据:

pre_from_tensor_slices x:  {'input_1': array(['/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/7388_39216_30--id=7388__sql_table=oac_1__sql_idx=405167__pitch=30__onset=39216.png',
       '/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/2447_864_27--id=2447__sql_table=oac_1__sql_idx=415458__pitch=27__onset=864.png',
       '/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/2386_20208_38--id=2386__sql_table=oac_1__sql_idx=433248__pitch=38__onset=20208.png',
       ...,
       '/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/6261_24528_57--id=6261__sql_table=oac_1__sql_idx=449753__pitch=57__onset=24528.png',
       '/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/3727_22944_31--id=3727__sql_table=oac_1__sql_idx=407620__pitch=31__onset=22944.png',
       '/media/me/sp_data/sp_data/datasets/chr_01/one_sec_onset_11_oac-leg/1668_7056_60--id=1668__sql_table=oac_1__sql_idx=381152__pitch=60__onset=7056.png'],
      dtype='<U162'), 'input_2': array(['/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/7388_39216_30--id=7388__sql_table=oac_1__sql_idx=405167__pitch=30__onset=39216.png',
       '/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/2447_864_27--id=2447__sql_table=oac_1__sql_idx=415458__pitch=27__onset=864.png',
       '/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/2386_20208_38--id=2386__sql_table=oac_1__sql_idx=433248__pitch=38__onset=20208.png',
       ...,
       '/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/6261_24528_57--id=6261__sql_table=oac_1__sql_idx=449753__pitch=57__onset=24528.png',
       '/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/3727_22944_31--id=3727__sql_table=oac_1__sql_idx=407620__pitch=31__onset=22944.png',
       '/media/me/sp_data/sp_data/datasets/mel_01/one_sec_onset_11_oac-leg/1668_7056_60--id=1668__sql_table=oac_1__sql_idx=381152__pitch=60__onset=7056.png'],
      dtype='<U162')}
pre_from_tensor_slices y:  {'output_1': array([0.817, 0.018, 0.421, ..., 0.511, 0.478, 0.147])}
_________________________
y:  {'output_1': <tf.Tensor 'args_2:0' shape=() dtype=float64>}
x:  {'input_1': <tf.Tensor 'args_0:0' shape=() dtype=string>, 'input_2': <tf.Tensor 'args_1:0' shape=() dtype=string>}
x.values():  dict_values([<tf.Tensor 'args_0:0' shape=() dtype=string>, <tf.Tensor 'args_1:0' shape=() dtype=string>])
x['input_1']:  Tensor("args_0:0", shape=(), dtype=string)

运行 x['input_1'].values() 会抛出错误:“Tensor”对象没有属性“values”

我在map_fn 周围遇到错误

  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 284, in _constant_impl
    allow_broadcast=allow_broadcast))
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 455, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

编辑2

尝试最新的我收到以下错误

Traceback (most recent call last):
  File "/home/me/.local/bin/wavfeat", line 11, in <module>
    load_entry_point('wavfeat==0.1.0', 'console_scripts', 'wavfeat')()
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/__main__.py", line 91, in main
    analysis_batch_sql(obj)
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 50, in analysis_batch_sql
    qy = [*map(lambda c: run_elm(c[0], c[1]), ch)]
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 50, in <lambda>
    qy = [*map(lambda c: run_elm(c[0], c[1]), ch)]
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/analysis_run_csv.py", line 23, in run_elm
    out = fn(input, elm)
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 216, in one_sec_onset_train
    return train(input, elm)
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 203, in train
    vs = validation_prep(jd, 'validation', bc)
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_onset.py", line 176, in validation_prep
    ds  = ds.map(map_func=load_and_preprocess_from_path_label, num_parallel_calls=AUTOTUNE)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1146, in map
    self, map_func, num_parallel_calls, preserve_cardinality=True)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3264, in __init__
    use_legacy_function=use_legacy_function)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2591, in __init__
    self._function = wrapper_fn._get_concrete_function_internal()
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1366, in _get_concrete_function_internal
    *args, **kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1360, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1648, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1541, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 716, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2585, in wrapper_fn
    ret = _wrapper_helper(*args)
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2530, in _wrapper_helper
    ret = func(*nested_args)
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_data_loader.py", line 47, in load_and_preprocess_from_path_label
    pl   = dict([(pk, tf.map_fn(load_and_preprocess_image, po, dtype=tf.float32)) for pk,po in pout])
  File "/home/me/.local/lib/python3.6/site-packages/wavfeat/one_sec_data_loader.py", line 47, in <listcomp>
    pl   = dict([(pk, tf.map_fn(load_and_preprocess_image, po, dtype=tf.float32)) for pk,po in pout])
  File "/home/me/.local/lib/python3.6/site-packages/tensorflow/python/ops/map_fn.py", line 214, in map_fn
    raise ValueError("elems must be a 1+ dimensional Tensor, not a scalar")
ValueError: elems must be a 1+ dimensional Tensor, not a scalar

Answer 1

附加组件:不使用字典结构

这是一个完整的代码(除了定义 json_parse_x_y 和声明 AUTOTUNE)，可以在不使用 dict 结构的情况下实现您尝试的目标。

我测试了 make_dataset 是否有效(请参阅下面的示例)，因此，如果您遇到问题，应该是由于有关 load_tensors 的规范错误造成的。

from itertools import zip_longest

import tensorflow as tf

# additionnally, `json_parse_x_y` must be defined
# and `AUTOTUNE` must be declared (in my example, I set it to 2)


def read_image(path, shape):
    """Read an image of givent filepath and tensor shape.

    Return a float tensor of given shape.
    """
    try:
        image = tf.io.read_file(path)
        image = tf.image.decode_png(image)
        image = tf.image.resize(image, [shape[1], shape[2]])
        image /= 255.0
        return image
    except:
        raise FileNotFoundError("preprocess_image: bad path '%s'" % path)


def load_images(paths, shapes):
    """Load an ensemble of images (associated with a single sample).

    paths  : rank-1 string Tensor
    shapes : list of images' shapes (same length as `paths`)

    Return a tuple of float tensors containing the loaded images.
    """
    return tuple((
        read_image(paths[i], shapes[i])
        for i in range(len(shapes))
    ))


def load_tensors(json_data, seq):
    """Load images descriptors from a json dump.

    Return a tuple containing:
        * a rank-2 tensor containing lists of image paths (str)
        * a rank-2 tensor containing resolution values (float)
        * a list of image shapes, of same length as the rank-2
          tensor's second axis
    """
    x,y = json_parse_x_y(json_data[seq])
    xx  = [*zip_longest(*x)] # NOTE: goes from variable sized input to {'input_N':...}
    yy  = [*zip_longest(*y)]

    # GET SHAPES (hard coded atm)
    lns = [[len(xxx)] for xxx in xx]
    rzs = [[24,512,1],[96,512,1]] # TEMP TODO! grab grom [(v['h'],v['w'],v['c']) for v in xx]
    shp = [*zip_longest(*[lns,rzs])]
    shp = [list(s) for s in shp]
    shp = [[*itertools.chain.from_iterable(s)] for s in shp]
    return (xx, yy, shp)


def make_dataset(xx, yy, shp, batch_size):
    """Build a Dataset instance containing loaded images.

    xx, yy, shp : see the specification of `load_tensors`'s outputs
    batch_size  : batch size to set on the Dataset

    Return a Dataset instance where each batched sample is a tuple
    containing two elements: first, a tuple containing N loaded images'
    rank-3 tensors; second, a rank-1 tensor containing M float values.
    (to be clear: batching adds a dimension to all those tensors)
    """
    data = tf.data.Dataset.from_tensor_slices((xx, yy))
    data = data.shuffle(10000)
    data = data.map(lambda x, y: (load_images(x, shapes), y))
    data = data.repeat()
    data = data.batch(batch_size) if batch_size else data
    data = data.prefetch(AUTOTUNE) 
    return data


def dataset_prep(json_data, seq, batch_size):
    """Full pipeline to making a Dataset from json."""
    xx, yy, shapes = load_tensors(json_data, seq)
    return make_dataset(xx, yy, shapes)

例如，使用“手工制作”值；所有图像实际上都是 this classic image ，形状为 [512, 512, 3]。

import numpy as np
import tensorflow as tf

# import previous code

# Here, N = 2, and I make 2 samples.
x = tf.convert_to_tensor(np.array([
    ['image_1a.png', 'image_1b.png'],
    ['image_2a.png', 'image_2b.png']
]))
shapes = [[1, 512, 512], [1, 512, 512]]  # images are initially [512, 512, 3]
# Here, M = 3, and I make 2 samples. Values are purely random.
y = tf.convert_to_tensor(np.array([
    [.087, .92, .276],
    [.242, .37, .205]
]))

# This should work.
data = make_dataset(x, y, shapes, batch_size=1)
# Output signature is <PrefetchDataset shapes:
#     (((None, 512, 512, None), (None, 512, 512, None)), (None, 3)),
#     types: ((tf.float32, tf.float32), tf.float64)
# >
# Where the first None is actually `batch_size`
# and the second is, in this case, 3.

<小时/>

回答当前问题:

好吧，您现在遇到的问题是修改后的 load_image 函数不符合数据集的规范，因此引发异常。请在下面找到似乎有效的完整编辑代码(我在计算机上使用自定义图像运行了测试，其中 xd/yd 字典初始化为看起来像您报告的 x 和y 数据集中张量)。它不太漂亮，我个人建议放弃 dict 结构，但它有效:

from itertools import zip_longest

def read_image(path, shape):
    try:
        image = tf.io.read_file(path)
        image = tf.image.decode_png(image)
        image = tf.image.resize(image, [shape[1],shape[2]])
        image /= 255.0
        return image
    except:
        raise FileNotFoundError("preprocess_image: bad path '%s'" % path)

# CHANGED: load_image is actually useless

def dataset_prep(json_data, seq, batch_size):
    # LOAD DATA FROM JSON
    x,y = json_parse_x_y(json_data[seq])
    xx  = [*zip_longest(*x)] # NOTE: goes from variable sized input to {'input_N':...}
    yy  = [*zip_longest(*y)]

    # GET SHAPES (hard coded atm)
    lns = [[len(xxx)] for xxx in xx]
    rzs = [[24,512,1],[96,512,1]] # TEMP TODO! grab grom [(v['h'],v['w'],v['c']) for v in xx]
    shp = [*zip_longest(*[lns,rzs])]
    shp = [list(s) for s in shp]
    shp = [[*itertools.chain.from_iterable(s)] for s in shp]

    xd  = dict([[ "input_{}".format(i+1),np.array(y)] for i,y in [*enumerate(xx)]])
    yd  = dict([["output_{}".format(i+1),np.array(y)] for i,y in [*enumerate(yy)]])

    ds  = tf.data.Dataset.from_tensor_slices((xd, yd))
    ds  = ds.shuffle(10000)

    # CHANGED: the following line, to run images import (also moved epeat instruction later)
    ds  = ds.map(
        lambda x, y: (
            {key: read_image(path, shp[i]) for i, (key, path) in enumerate(x.items())},
            y
        ),
        num_parallel_calls=AUTOTUNE
    )
    ds  = ds.repeat()
    ds  = ds.batch(batch_size) if batch_size else ds 
    ds  = ds.prefetch(AUTOTUNE) 
    return ds

<小时/>

初始答案(问题编辑之前):

我只会处理这个答案中由 load_image 引发的异常，但可能还需要对其余部分执行额外的工作 - 我没有对此进行测试，因为手头没有方便的数据集.

异常消息实际上非常明确:您正在传递一个标量元素(例如 n in [(k, tf.map_fn(lambda x: read_image (x, shp), n, dtype=tf.float32)) for k,n in pout]) 作为 tf.map_fn 的 elems 参数，当它需要一个张量(或(可能是嵌套的)张量列表或元组)，如其文档中明确指定的。

您还在引用的代码行中以错误的方式使用了 tf.map_fn ，因为基本上您将其与 python 意图列表混合在一起，而您应该使用其中之一。

使用意图列表(同时替换 load_image 函数中无用的前几行):pl = {path: (load_image(path, shp), res) for path, res in x.items()}

使用tf.map_fn:

# Read all images, return two tensors, one with images, the other with resolutions.
# (so, resolutions inclusion in this is actually useless and should be redesigned)
pl = tf.map_fn(
    lambda args: (read_image(args[0], shp), args[1]),
    [tf.convert_to_tensor(list(x)), tf.convert_to_tensor(list(x.values()))],
    dtype=(tf.float32, tf.float32)
)
# If you really, really want to return a dict, but is it an optimal design?
pl = {path: (pl[0][i], pl[1][i]) for i, path in enumerate(x)}

我不知道返回以这种方式指定的字典是否与数据集实例化是最佳的(甚至兼容)，但是如果代码的其余部分正常工作，这应该可以解决问题。

无论如何，如果您想迭代字典，请继续使用第一个版本或第二个版本的修改版本(这可能具有并行化图像读取的优势)。

我希望这有帮助:-)