python - Keras 关于数组形状的错误，但形状似乎是正确的

Question

我正在尝试使用 Keras 和 python 训练一个简单的模型。文本经过完美的预处理。但是当我尝试安装它时，出现以下错误:

File "main.py", line 47, in <module>
    model.fit(x_train, y_train, batch_size=32, epochs=3)
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
    batch_size=batch_size)
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training.py", line 789, in _standardize_user_data
    exception_prefix='target')
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training_utils.py", line 138, in standardize_input_data
    str(data_shape))
ValueError: Error when checking target: expected dense_2 to have shape (121885,) but got array with shape (1000,)

但是 print(x_train.shape) 和 print(y_train.shape) 都返回 (121885, 1000)。我看不出有什么理由。

代码:https://pastebin.com/afnzBf6B

from keras.preprocessing.text import Tokenizer
from keras.layers import Dense
from keras.models import Sequential

data = open('movie_lines.txt', encoding='ISO-8859-1')
lines = [line for line in data]
filtered_lines = []

for line in lines:
    sentence = line.split('+++$+++')[4].strip(' ')
    filtered_lines.append(sentence)

train_size = int(len(filtered_lines) * .8)
train_portion = filtered_lines[:train_size]
test_portion = filtered_lines[train_size:]

x_lines = train_portion[::2]
y_lines = train_portion[1::2]
x_test = test_portion[::2]
y_test = test_portion[1::2]
vocab_size = 1000
print('Prepared data')

def prepare_text(text):
    tokenizer = Tokenizer(num_words=vocab_size)
    tokenizer.fit_on_texts(text)
    matrix = tokenizer.texts_to_matrix(text)
    return matrix


x_train = prepare_text(x_lines)
print('matrixed x')
y_train = prepare_text(y_lines)
print('matrixed y')
print(f'X shape: {x_train.shape}')
print(f'Y shape: {y_train.shape}')

model = Sequential()
model.add(Dense(512, input_shape=(vocab_size,), activation='relu'))
model.add(Dense(len(y_lines), activation='softmax'))

model.compile(
        loss='categorical_crossentropy',
        optimizer='adam',
        metrics=['accuracy',]
        )
print('Created and compiled model')

model.fit(x_train, y_train, epochs=3)

score = model.evaluate(x_test, y_test, batch_size=32, epochs=3)
print('Test Score:'+score[0])
print('Test Accuracy:'+score[1])

Answer 1

在预处理阶段您使用 texts_to_matrix()方法(使用默认参数)以 one-hot 编码格式将给定序列返回为矩阵的行。现在，如果您想仅使用密集层从一个单热编码序列转到另一个单热编码序列，则需要将最后一层中的单元数设置为词汇表大小(即，矩阵)并使用sigmoid作为最后一层的激活函数:

model = Sequential()
model.add(Dense(512, input_shape=(vocab_size,), activation='relu'))
model.add(Dense(vocab_size, activation='sigmoid'))