python - 非常大的矩阵(包括 NaN)的热图

Question

我想看看 NaN 是否集中在某个地方，或者它们的分布是否有任何模式。

这个想法是使用 python 绘制矩阵的热图(即 200K 行和 1k 列)，并为 NaN 值设置特殊颜色(其余值可以用相同的颜色表示，这并不'没关系)

可能的显示示例:

先谢谢大家了

Answer 1

1:200 的纵横比非常糟糕，而且由于可能会遇到内存问题，因此您可能应该将其分成几个 Nx1k block 。

话虽这么说，这是我的解决方案(受到您的示例图像的启发):

from mpl_toolkits.axes_grid1 import AxesGrid

# generate random matrix
xDim = 2000
yDim = 4000
# number of nans
nNans = xDim*yDim*.1
rands = np.random.rand(yDim, xDim)

# create a skewed distribution for the nans
x = np.clip(np.random.gamma(2, yDim*.125, size=nNans).astype(np.int),0 ,yDim-1)
y = np.random.randint(0,xDim,size=nNans)
rands[x,y] = np.nan

# find the nans:
isNan = np.isnan(rands)

fig = plt.figure()

# make axesgrid so we can put a histogram-like plot next to the data
grid = AxesGrid(fig, 111, nrows_ncols=(1, 2), axes_pad=0.05)

# plot the data using binary colormap
grid[0].imshow(isNan, cmap=cm.binary)

# plot the histogram
grid[1].plot(np.sum(isNan,axis=1), range(isNan.shape[0]))

# set ticks and limits, so the figure looks nice
grid[0].set_xticks([0,250,500,750,1000,1250,1500,1750])
grid[1].set_xticks([0,250,500,750])
grid[1].set_xlim([0,750])
grid.axes_llc.set_ylim([0, yDim])
plt.show()

它是这样的: