python - 如何使用keras实现一个简单的神经网络

Question

使用以下库:

import keras
import tensorflow as tf
from keras.datasets import mnist
from keras.utils.np_utils import to_categorical
from keras import optimizers
from keras.models import Sequential
from keras.layers import Dense, Activation, Flatten

我需要创建一个简单的网络，如图所示:

NN 应该采用与我们输入的数字形状相同的张量。第一个隐藏层应该输出 300 维向量并使用 sigmoid 作为激活函数。第二个隐藏层也要输出300维的向量，使用relu作为激活函数。第三个隐藏层(即输出层)应该输出预测的大小，并在最后使用 softmax 激活进行多类分类

目前，我有:

model = None

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(5, input_shape=(300,)))
model.add(Activation(___))
model.add(tf.keras.layers.Dense(6))
model.add(Activation(___))
model.add(tf.keras.layers.Dense(7))
model.add(Activation(___))

我不确定 Dense 中包含什么。我只是从这里引用它 https://keras.io/api/models/sequential/

我理解第一个激活函数应该是sigmoid，这里sigmoid是1/(1 + np.exp(-x))

我也知道relu是max(0.0, x) softmax是

def softmax(vector):
    e = exp(vector)
    return e / e.sum()

但是我不确定如何将所有这些放在一起来创建神经网络并获得输出:

如果有人愿意提供帮助，我将不胜感激，因为这是我第一次尝试初始化神经网络。谢谢!

Answer 1

在 keras.layers.activations 的文档中你可以看到

tf.keras.activations.sigmoid(x)
tf.keras.activations.softmax(x, axis=-1) 
tf.keras.activations.relu(x, alpha=0.0, max_value=None, threshold=0.0)

所以代码可以是:

import tensorflow as tf

model = tf.keras.Sequential(name='sequential_2')

model.add(tf.keras.layers.Dense(300, name='dense_5', input_shape=(784,)))
model.add(tf.keras.layers.Activation(tf.keras.activations.sigmoid, name='activation_5'))

model.add(tf.keras.layers.Dense(300, name='dense_6'))
model.add(tf.keras.layers.Activation(tf.keras.activations.relu, name='activation_6'))

model.add(tf.keras.layers.Dense(10, name='dense_7'))
model.add(tf.keras.layers.Activation(tf.keras.activations.softmax, name='activation_7'))

model.summary()

结果:

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_5 (Dense)             (None, 300)               235500    
                                                                 
 activation_5 (Activation)   (None, 300)               0         
                                                                 
 dense_6 (Dense)             (None, 300)               90300     
                                                                 
 activation_6 (Activation)   (None, 300)               0         
                                                                 
 dense_7 (Dense)             (None, 10)                3010      
                                                                 
 activation_7 (Activation)   (None, 10)                0         
                                                                 
=================================================================
Total params: 328,810
Trainable params: 328,810
Non-trainable params: 0
_________________________________________________________________

---

使用不同的imports你可以减少代码并且仍然得到相同的summary:

#import tensorflow as tf  # no need it at this moment
from keras import Sequential
from keras.layers import Dense, Activation
from keras.activations import sigmoid, relu, softmax

model = Sequential(name='sequential_2')

model.add(Dense(300, name='dense_5', input_shape=(784,)))
model.add(Activation(sigmoid, name='activation_5'))

model.add(Dense(300, name='dense_6'))
model.add(Activation(relu, name='activation_6'))

model.add(Dense(10, name='dense_7'))
model.add(Activation(softmax, name='activation_7'))

model.summary()

但您也可以减少代码以获得相同的模型:

#import tensorflow as tf  # no need it at this moment
from keras import Sequential
from keras.layers import Dense

model = Sequential(name='sequential_2')

model.add(Dense(300, name='dense_5', activation='sigmoid', input_shape=(784,)))
model.add(Dense(300, name='dense_6', activation='relu'))
model.add(Dense(10,  name='dense_7', activation='softmax'))

model.summary()

但是summary也会减少:

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_5 (Dense)             (None, 300)               235500    
                                                                 
 dense_6 (Dense)             (None, 300)               90300     
                                                                 
 dense_7 (Dense)             (None, 10)                3010      
                                                                 
=================================================================
Total params: 328,810
Trainable params: 328,810
Non-trainable params: 0
_________________________________________________________________

您也可以跳过 name= 以减少代码。

#import tensorflow as tf  # no need it at this moment
from keras import Sequential
from keras.layers import Dense

model = Sequential()

model.add(Dense(300, activation='sigmoid', input_shape=(784,)))
model.add(Dense(300, activation='relu'))
model.add(Dense(10,  activation='softmax'))

model.summary()

---

顺便说一句:

如果你想使用 MNIST 的图像形状为 (28,28) 那么你可以添加层 Flatten 来自动转换形状 (28,28) 到形状 (784)

#import tensorflow as tf  # no need it at this moment
from keras import Sequential
from keras.layers import Dense, Flatten

model = Sequential()

#model.add(Flatten(input_shape=(1,28,28)))
model.add(Flatten(input_shape=(28,28)))
model.add(Dense(300, activation='sigmoid'))
model.add(Dense(300, activation='relu'))
model.add(Dense(10,  activation='softmax'))

model.summary()

结果:

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 784)               0         
                                                                 
 dense (Dense)               (None, 300)               235500    
                                                                 
 dense_1 (Dense)             (None, 300)               90300     
                                                                 
 dense_2 (Dense)             (None, 10)                3010      
                                                                 
=================================================================
Total params: 328,810
Trainable params: 328,810
Non-trainable params: 0
_________________________________________________________________

如果你在 fit() 之后使用 summary() 那么你甚至可以在 中跳过 input_shape=(28,28) >Flatten() 因为 fit() 会设置它。

---

编辑:

完整的工作代码。

在 tensorflow 2.8.0、Python 3.10、Linux Mint 21.0 上测试(基于 Ubuntu 22.04)

import tensorflow as tf
from keras import Sequential
from keras.layers import Dense, Flatten
from keras.utils.np_utils import to_categorical
#from keras.losses import categorical_crossentropy
from keras.datasets import mnist
import numpy as np

print('\n--- version ---\n')

print('tensorflow:', tf.__version__)
#print('tensorflow:', tf.version.VERSION)
#print('tensorflow:', keras.__version__)

print('\n--- data ---\n')

(x_train, y_train), (x_test, y_test) = mnist.load_data()
print('train.shape  :', x_train.shape)
print('test.shape   :', x_test.shape)

print('image.shape  :', x_train[0].shape)
print('image.flatten:', x_train[0].flatten().shape)

y_train = to_categorical(y_train)  # `Y` will have shape `(10)` and last layer will also need `Dense(10)`
y_test  = to_categorical(y_test)   # `Y` will have shape `(10)` and last layer will also need `Dense(10)`

print('\n--- model ---\n')

model = Sequential()

#model.add(Flatten(input_shape=(1,28,28)))  # if `summary()` used before `fit()` then you have to set `input_shape`
model.add(Flatten())                        # if `summary()` used after  `fit()` then you can skip `input_shape` (because `fit` will set it)
model.add(Dense(300, activation='sigmoid'))
model.add(Dense(300, activation='relu'))
model.add(Dense(10,  activation='softmax'))

#model.compile(loss=categorical_crossentropy, optimizer='adam', metrics=['accuracy'])  # function `categorical_crossentropy`
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])  # string `'categorical_crossentropy'`
print('\n--- fit/train ---\n')

batch_size = 20
epochs = 5

model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=True, validation_data=(x_test, y_test))

print('\n--- evaluate/test ---\n')

score = model.evaluate(x_test, y_test, verbose=True)
print(score)

print('\n--- predict ---\n')

# for N-elements it needs shape `(N, 28, 28)`
# for single element it needs shape `(1, 28, 28)`, not `(28, 28)`
#items = x_test[0]                          # WRONG
#items = x_test[0][np.newaxis, ...]         # OK
#items = x_test[0][None, ...]               # OK
#items = x_test[0:1]                        # OK
#items = np.array( [x_test[0]] )            # OK
#items = np.expand_dims(x_test[0], axis=0)  # OK
#items = x_test[0].reshape(-1,28,28)        # OK
items = x_test[0].reshape(1,28,28)          # OK
print('items.shape:', items.shape)

results = model.predict(items)   # it needs `numpy.array` even for single element (shape: `(1, 28, 28)`)

for item in results:
    print('predict:', item)
    print('max    :', np.max(item))
    print('argmax :', np.argmax(item))    
    
#print('result:', results[0])
#print('max:', np.max(results[0]), np.argmax(results[0]))

print('\n--- summary ---\n')

model.summary() 

#print('\n--- show image ---\n')

#import matplotlib.pyplot as plt
#plt.imshow(x_test[0])
#plt.show()

结果:

--- version ---

tensorflow: 2.8.0

--- data ---

train.shape  : (60000, 28, 28)
test.shape   : (10000, 28, 28)
image.shape  : (28, 28)
image.flatten: (784,)

--- model ---

--- fit/train ---

Epoch 1/5
3000/3000 [==============================] - 26s 8ms/step - loss: 0.4459 - accuracy: 0.8600 - val_loss: 0.3425 - val_accuracy: 0.8907
Epoch 2/5
3000/3000 [==============================] - 31s 10ms/step - loss: 0.3133 - accuracy: 0.9011 - val_loss: 0.2753 - val_accuracy: 0.9128
Epoch 3/5
3000/3000 [==============================] - 22s 7ms/step - loss: 0.2702 - accuracy: 0.9153 - val_loss: 0.2291 - val_accuracy: 0.9271
Epoch 4/5
3000/3000 [==============================] - 22s 7ms/step - loss: 0.2482 - accuracy: 0.9218 - val_loss: 0.2142 - val_accuracy: 0.9346
Epoch 5/5
3000/3000 [==============================] - 21s 7ms/step - loss: 0.2246 - accuracy: 0.9292 - val_loss: 0.2061 - val_accuracy: 0.9362

--- evaluate/test ---

313/313 [==============================] - 2s 6ms/step - loss: 0.2061 - accuracy: 0.9362
[0.20612983405590057, 0.9362000226974487]

--- predict ---

items.shape: (1, 28, 28)
predict: [2.2065036e-08 3.6986236e-09 2.2580415e-04 2.4656517e-06 6.2174437e-12
 6.3476205e-07 1.2726037e-15 9.9974352e-01 1.7498225e-07 2.7370381e-05]
max    : 0.9997435
maxarg : 7

--- summary ---

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 784)               0         
                                                                 
 dense (Dense)               (None, 300)               235500    
                                                                 
 dense_1 (Dense)             (None, 300)               90300     
                                                                 
 dense_2 (Dense)             (None, 10)                3010      
                                                                 
=================================================================
Total params: 328,810
Trainable params: 328,810
Non-trainable params: 0
_________________________________________________________________