Python Scipy : scipy. stats.spearmanr 返回 nans

Question

编辑:我认为基本解决了。

我正在使用 scipy.stats 中的 spearmanr 来查找多个不同样本中变量之间的相关性。我有大约 2500 个变量和 36 个样本(或“观察”)

如果我使用所有 36 个样本计算相关性，spearmanr 工作正常。如果我只使用前 18 个样本，它也能正常工作。但是，如果我使用后 18 个样本，则会出现错误并返回 nans。

这是错误:

/Home/s1215235/.local/lib/python2.7/site-packages/numpy/lib/function_base.py:1945: RuntimeWarning: invalid value encountered in true_divide
return c / sqrt(multiply.outer(d, d))
/Home/s1215235/.local/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1718: RuntimeWarning: invalid value encountered in greater
cond1 = (scale > 0) & (x > self.a) & (x < self.b)
/Home/s1215235/.local/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1718: RuntimeWarning: invalid value encountered in less
cond1 = (scale > 0) & (x > self.a) & (x < self.b)
/Home/s1215235/.local/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1719: RuntimeWarning: invalid value encountered in less_equal
cond2 = cond0 & (x <= self.a)

这是代码:

populationdata = np.vstack(thing).astype(np.float)
rho, pval = stats.spearmanr(populationdata[:,sampleindexes], axis = 1)

(populationdata 是一个充满 float 的 numpy 数组；[:,sampleindexes] 只允许使用少数列。

这就是 rho 返回的内容:

[[ 1.                 nan         nan ...,  1.         -0.05882353
  -0.08574929]
 [        nan         nan         nan ...,         nan         nan
          nan]
 [        nan         nan         nan ...,         nan         nan
          nan]
 ..., 
 [ 1.                 nan         nan ...,  1.         -0.05882353
  -0.08574929]
 [-0.05882353         nan         nan ..., -0.05882353  1.          0.68599434]
 [-0.08574929         nan         nan ..., -0.08574929  0.68599434  1.        ]]

Answer 1

在评论中指出“虽然有很多 0。” 所以 populationdata[:,sampleindexes] 可能有全为 0 的行。那将导致 spearmanr 生成 nan。例如，

In [3]: spearmanr([[0, 0, 0], [1, 2, 3]], axis=1)
/Users/warren/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.py:1957: RuntimeWarning: invalid value encountered in true_divide
  return c / sqrt(multiply.outer(d, d))
/Users/warren/anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1728: RuntimeWarning: invalid value encountered in greater
  cond1 = (scale > 0) & (x > self.a) & (x < self.b)
/Users/warren/anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1728: RuntimeWarning: invalid value encountered in less
  cond1 = (scale > 0) & (x > self.a) & (x < self.b)
/Users/warren/anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1729: RuntimeWarning: invalid value encountered in less_equal
  cond2 = cond0 & (x <= self.a)
Out[3]: (nan, nan)