matlab - 反向传播算法(Matlab): output values are saturating to 1

Question

我根据这些注释在 Matlab 中编写了反向传播算法:http://dl.dropbox.com/u/7412214/BackPropagation.pdf

我的网络采用长度为 43 的输入/特征向量，隐藏层中有 20 个节点(我可以更改任意参数选择)，并且有一个输出节点。我想训练我的网络以获取 43 个特征并输出 0 到 100 之间的单个值。输入数据被归一化为零均值和单位标准差(通过 z = x - 均值/std)，然后我附加了“1 ” 术语输入向量来表示偏差。我的 targetValues 只是 0 到 100 之间的单个数字。

这是我的代码的相关部分:

(按照我的约定，层 I (i) 指输入层，J (j) 指隐藏层，K (k) 指输出层，在本例中为单个节点。)

for train=1:numItrs
        for iterator=1:numTrainingSets

            %%%%%%%% FORWARD PROPAGATION %%%%%%%%

            % Grab the inputs, which are rows of the inputFeatures matrix
            InputLayer = inputFeatures(iterator, :)'; %don't forget to turn into column 
            % Calculate the hidden layer outputs: 
            HiddenLayer = sigmoidVector(WeightMatrixIJ' * InputLayer); 
            % Now the output layer outputs:
            OutputLayer = sigmoidVector(WeightMatrixJK' * HiddenLayer);

            %%%%%%% Debug stuff %%%%%%%% (for single valued output)
            if (mod(train+iterator, 100) == 0)
               str = strcat('Output value: ', num2str(OutputLayer), ' | Test value: ', num2str(targetValues(iterator, :)')); 
               disp(str);
            end 




            %%%%%%%% BACKWARDS PROPAGATION %%%%%%%%

            % Propagate backwards for the hidden-output weights
            currentTargets = targetValues(iterator, :)'; %strip off the row, make it a column for easy subtraction
            OutputDelta = (OutputLayer - currentTargets) .* OutputLayer .* (1 - OutputLayer); 
            EnergyWeightDwJK = HiddenLayer * OutputDelta'; %outer product
            % Update this layer's weight matrix:
            WeightMatrixJK = WeightMatrixJK - epsilon*EnergyWeightDwJK; %does it element by element

            % Propagate backwards for the input-hidden weights
            HiddenDelta = HiddenLayer .* (1 - HiddenLayer) .* WeightMatrixJK*OutputDelta; 
            EnergyWeightDwIJ = InputLayer * HiddenDelta'; 
            WeightMatrixIJ = WeightMatrixIJ - epsilon*EnergyWeightDwIJ; 

        end

    end

权重矩阵初始化如下:

WeightMatrixIJ = rand(numInputNeurons, numHiddenNeurons) - 0.5; 
WeightMatrixJK = rand(numHiddenNeurons, numOutputNeurons) - 0.5; 
%randoms b/w (-0.5, 0.5)

“sigmoidVector”函数获取向量中的每个元素并应用y = 1/(1 + exp(-x))。

从代码开始，调试消息如下所示:

Output value:0.99939 | Test value:20
Output value:0.99976 | Test value:20
Output value:0.99985 | Test value:20
Output value:0.99989 | Test value:55
Output value:0.99991 | Test value:65
Output value:0.99993 | Test value:62
Output value:0.99994 | Test value:20
Output value:0.99995 | Test value:20
Output value:0.99995 | Test value:20
Output value:0.99996 | Test value:20
Output value:0.99996 | Test value:20
Output value:0.99997 | Test value:92
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99997 | Test value:20
Output value:0.99998 | Test value:20
Output value:0.99998 | Test value:20
Output value:0.99999 | Test value:20
Output value:0.99999 | Test value:20
Output value:1 | Test value:20
Output value:1 | Test value:62
Output value:1 | Test value:70
Output value:1 | Test value:77
Output value:1 | Test value:20
** stays saturated at 1 **

显然，我希望网络将输出值训练在 0 到 100 之间，以尝试匹配这些目标值!

感谢您的帮助，如果您需要更多信息，我会尽力提供。

Answer 1

sigmoid 函数限制在 (0,1) 范围内，因此它永远不会达到您的目标值(因为它们都大于 1)。您应该调整目标值，使它们也在 sigmoid 的范围内。由于您知道目标值限制在范围 (0,100) 内，因此只需将它们全部除以 100 即可。