python - 更改 NN 中的单个权重

Question

import tensorflow as tf
from tensorflow import keras
import numpy as np

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(2,)),
    keras.layers.Dense(20, activation=tf.nn.relu),
    keras.layers.Dense(20, activation=tf.nn.relu),
    keras.layers.Dense(1)
])


# an individual weight is like:
print(model.weights[4][0])
# returns tf.Tensor([0.3985532], shape=(1,), dtype=float32)

我正在做一个(有些愚蠢的)实验，想手动停用网络中的某些神经元。根据我的阅读，最好的方法是使用 mask 或调整重量。对于后者，我可以打印单个神经元的值，但现在我想“设置”它。问题是我不能说 tensor = 0.0 因为它也包含形状和类型。有什么想法吗？

Answer 1

您可以使用“assign”来更改值，如下所示:

import tensorflow as tf
from tensorflow import keras
import numpy as np

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(2,)),
    keras.layers.Dense(20, activation=tf.nn.relu),
    keras.layers.Dense(20, activation=tf.nn.relu),
    keras.layers.Dense(1)
])


# an individual weight is like:
print(model.weights[4][0])

weights=model.weights[4].numpy() # get the kernel weights of the layer as numpy array
weights[0]=0 #set tensor model.weights[4][0] to zero
model.weights[4].assign(weights)
print(model.weights[4][0]) # displays tf.Tensor([0.], shape=(1,), dtype=float32)

您还可以使用“assign”和“tensor_scatter_nd_update”，如下所示:

# an individual weight is like:
print(model.weights[4][0])

indices = [[0,0]] #indices to modify 
model.weights[4].assign(tf.tensor_scatter_nd_update(model.weights[4], indices, [0]))

print(model.weights[4][0]) # displays tf.Tensor([0.], shape=(1,), dtype=float32)