machine-learning - 处理 Logistic 回归的 NaN(缺失)值 - 最佳实践？

Question

我正在处理患者信息数据集，并尝试使用 MATLAB 根据该数据计算倾向得分。删除具有许多缺失值的特征后，我仍然留下几个缺失(NaN)值。

当我尝试使用以下 Matlab 代码(来自 Andrew Ng 的 Coursera 机器学习类(class))执行逻辑回归时，由于这些缺失值，我收到了错误，因为我的成本函数和梯度向量的值变为 NaN:

[m, n] = size(X);
X = [ones(m, 1) X];    
initial_theta = ones(n+1, 1);
[cost, grad] = costFunction(initial_theta, X, y);
options = optimset('GradObj', 'on', 'MaxIter', 400);

[theta, cost] = ...
    fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

注意:sigmoid 和 costfunction 是我为整体易用性而创建的工作函数。

如果我将所有 NaN 值替换为 1 或 0，则计算可以顺利执行。但是我不确定这是否是处理此问题的最佳方法，而且我也想知道我应该选择什么替换值(在一般)以获得使用缺失数据执行逻辑回归的最佳结果。 使用特定数字(0 或 1 或其他数字)替换数据中所述缺失值有什么好处/缺点吗？

注意:我还将所有特征值标准化为 0-1 范围内。

对此问题的任何见解都将受到高度赞赏。谢谢您

Answer 1

正如前面所指出的，无论编程平台如何，这是人们处理的普遍问题。这称为“缺失数据插补”。

将所有缺失值强制为特定数字肯定有缺点。根据数据的分布，它可能会很剧烈，例如，在零多于一的二进制稀疏数据中将所有缺失值设置为 1。

幸运的是，MATLAB 有一个名为 knnimpute 的函数，可以通过其最近邻估计缺失的数据点。

根据我的经验，我经常发现 knnimpute 很有用。然而，当您的数据中缺少太多站点时，它可能会出现不足；缺失站点的邻居也可能不完整，从而导致估计不准确。下面，我想出了一个解决方案；它首先估算最少不完整的列，(可选)为邻居施加安全的预定义距离。我希望这会有所帮助。

function data = dnnimpute(data,distCutoff,option,distMetric)
% data = dnnimpute(data,distCutoff,option,distMetric)
%
%   Distance-based nearest neighbor imputation that impose a distance
%     cutoff to determine nearest neighbors, i.e., avoids those samples 
%     that are more distant than the distCutoff argument.
%
%   Imputes missing data coded by "NaN" starting from the covarites 
%     (columns) with the least number of missing data. Then it continues by 
%     including more (complete) covariates in the calculation of pair-wise 
%     distances.
%
%   option, 
%       'median'      - Median of the nearest neighboring values
%       'weighted'    - Weighted average of the nearest neighboring values
%       'default'     - Unweighted average of the nearest neighboring values
%
%   distMetric,
%       'euclidean'   - Euclidean distance (default)
%       'seuclidean'  - Standardized Euclidean distance. Each coordinate
%                       difference between rows in X is scaled by dividing
%                       by the corresponding element of the standard
%                       deviation S=NANSTD(X). To specify another value for
%                       S, use D=pdist(X,'seuclidean',S).
%       'cityblock'   - City Block distance
%       'minkowski'   - Minkowski distance. The default exponent is 2. To
%                       specify a different exponent, use
%                       D = pdist(X,'minkowski',P), where the exponent P is
%                       a scalar positive value.
%       'chebychev'   - Chebychev distance (maximum coordinate difference)
%       'mahalanobis' - Mahalanobis distance, using the sample covariance
%                       of X as computed by NANCOV. To compute the distance
%                       with a different covariance, use
%                       D =  pdist(X,'mahalanobis',C), where the matrix C
%                       is symmetric and positive definite.
%       'cosine'      - One minus the cosine of the included angle
%                       between observations (treated as vectors)
%       'correlation' - One minus the sample linear correlation between
%                       observations (treated as sequences of values).
%       'spearman'    - One minus the sample Spearman's rank correlation
%                       between observations (treated as sequences of values).
%       'hamming'     - Hamming distance, percentage of coordinates
%                       that differ
%       'jaccard'     - One minus the Jaccard coefficient, the
%                       percentage of nonzero coordinates that differ
%       function      - A distance function specified using @, for
%                       example @DISTFUN.
%  
if nargin < 3
    option = 'mean';
end
if nargin < 4
    distMetric = 'euclidean';
end

nanVals = isnan(data);
nanValsPerCov = sum(nanVals,1);
noNansCov = nanValsPerCov == 0;
if isempty(find(noNansCov, 1))
    [~,leastNans] = min(nanValsPerCov);
    noNansCov(leastNans) = true;
    first = data(nanVals(:,noNansCov),:);
    nanRows = find(nanVals(:,noNansCov)==true); i = 1;
    for row = first'
        data(nanRows(i),noNansCov) = mean(row(~isnan(row)));
        i = i+1;
    end
end

nSamples = size(data,1);
if nargin < 2
    dataNoNans = data(:,noNansCov);
    distances = pdist(dataNoNans);
    distCutoff = min(distances);
end
[stdCovMissDat,idxCovMissDat] = sort(nanValsPerCov,'ascend');
imputeCols = idxCovMissDat(stdCovMissDat>0);
% Impute starting from the cols (covariates) with the least number of 
% missing data. 
for c = reshape(imputeCols,1,length(imputeCols))    
    imputeRows = 1:nSamples;
    imputeRows = imputeRows(nanVals(:,c));   
    for r = reshape(imputeRows,1,length(imputeRows))
        % Calculate distances
        distR = inf(nSamples,1);
        %
        noNansCov_r = find(isnan(data(r,:))==0);
        noNansCov_r = noNansCov_r(sum(isnan(data(nanVals(:,c)'==false,~isnan(data(r,:)))),1)==0);
        %
        for i = find(nanVals(:,c)'==false)
            distR(i) = pdist([data(r,noNansCov_r); data(i,noNansCov_r)],distMetric);            
        end
        tmp = min(distR(distR>0));        
        % Impute the missing data at sample r of covariate c
        switch option
            case 'weighted'
                data(r,c) = (1./distR(distR<=max(distCutoff,tmp)))' * data(distR<=max(distCutoff,tmp),c) / sum(1./distR(distR<=max(distCutoff,tmp)));
            case 'median'
                data(r,c) = median(data(distR<=max(distCutoff,tmp),c),1);
            case 'mean'
                data(r,c) = mean(data(distR<=max(distCutoff,tmp),c),1);
        end
        % The missing data in sample r is imputed. Update the sample 
        % indices of c which are imputed. 
        nanVals(r,c) = false;  
    end    
    fprintf('%u/%u of the covariates are imputed.\n',find(c==imputeCols),length(imputeCols));
end