machine-learning - pytorch问题: how to add bias term and extract its value? 类与顺序模型？和softmax

Question

我在 pytorch 中有一个基本的神经网络模型，如下所示:

import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(input_dim, hidden_dim)
        self.sigmoid = nn.Sigmoid()
        self.fc2 = nn.Linear(hidden_dim, output_dim)

    def forward(self, x):
        out = self.fc1(x)
        out = self.sigmoid(out)
        out = self.fc2(out)
        return out
net = Net(400, 512,10)

如何从 net.parameters() 中提取偏差/截距项？这个模型是否等同于使用equential()？

net = nn.Sequential(nn.Linear(input_dim, hidden_dim[0]),
                      nn.Sigmoid(),
                      nn.Linear(hidden_dim[0], hidden_dim[1]),
                      nn.Sigmoid(),
                      nn.Linear(hidden_dim[1], output_dim))

对于多类分类，nn.Softmax() 在任一模型的末尾是可选的吗？如果我理解正确的话，使用软件它会输出某个类别的概率，但如果不使用它，它会返回预测输出？

预先感谢您回答我的新手问题。

Answer 1

我们来一一解答问题。 这个模型等价于使用equential()简短回答:不。您可以看到添加了两个 Sigmoid 层和两个线性层。您可以打印网络并查看结果:

net = Net(400, 512,10)

print(net.parameters())
print(net)
input_dim = 400
hidden_dim = 512
output_dim = 10

model = Net(400, 512,10)

net = nn.Sequential(nn.Linear(input_dim, hidden_dim),
                      nn.Sigmoid(),
                      nn.Linear(hidden_dim, hidden_dim),
                      nn.Sigmoid(),
                      nn.Linear(hidden_dim, output_dim))

print(net)

输出为:

Net(
  (fc1): Linear(in_features=400, out_features=512, bias=True)
  (sigmoid): Sigmoid()
  (fc2): Linear(in_features=512, out_features=10, bias=True)
)

Sequential(
  (0): Linear(in_features=400, out_features=512, bias=True)
  (1): Sigmoid()
  (2): Linear(in_features=512, out_features=512, bias=True)
  (3): Sigmoid()
  (4): Linear(in_features=512, out_features=10, bias=True)
)

我希望您能看到它们的不同之处。

您的第一个问题:如何从 net.parameters() 中提取偏差/截距项

答案:

model = Net(400, 512,10)

bias = model.fc1.bias

print(bias)

输出是:

tensor([ 3.4078e-02,  3.1537e-02,  3.0819e-02,  2.6163e-03,  2.1002e-03,
         4.6842e-05, -1.6454e-02, -2.9456e-02,  2.0646e-02, -3.7626e-02,
         3.5531e-02,  4.7748e-02, -4.6566e-02, -1.3317e-02, -4.6593e-02,
        -8.9996e-03, -2.6568e-02, -2.8191e-02, -1.9806e-02,  4.9720e-02,
        ---------------------------------------------------------------
        -4.6214e-02, -3.2799e-02, -3.3605e-02, -4.9720e-02, -1.0293e-02,
         3.2559e-03, -6.6590e-03, -1.2456e-02, -4.4547e-02,  4.2101e-02,
        -2.4981e-02, -3.6840e-03], requires_grad=True)