python - pandas - 访问 SparseDataFrame 的行

Question

我有一个密度非常低(设置了 0.1% 的条目)的大型 SparseDataFrame(例如，20k 索引 x 10k 列)。我正在尝试访问特定的行数据框，但我似乎做不到。虽然访问列很好。这是一个说明问题的小例子:

import numpy as np
import pandas as pd

df = pd.DataFrame(np.arange(15).reshape(5,3), index=list('abcde'))
df.loc['b',1] = np.nan  # for good measure...
sparse = df.to_sparse()

sparse[1]  # This is OK.
df.loc['b']  # This is also OK.
sparse.loc['b'] # This blows up.

这是回溯:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/indexing.py", line 1020, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/indexing.py", line 1145, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/indexing.py", line 68, in _get_label
    return self.obj._xs(label, axis=axis, copy=True)
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/frame.py", line 2149, in xs
    new_values, copy = self._data.fast_2d_xs(loc, copy=copy)
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/internals.py", line 2714, in fast_2d_xs
    result[i] = blk._try_coerce_result(blk.iget((j, loc)))
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/core/internals.py", line 275, in iget
    return self.values[i]
  File "/Users/.../.virtualenvs/exp/lib/python2.7/site-packages/pandas/sparse/array.py", line 286, in __getitem__
    data_slice = self.values[key]
IndexError: too many indices

请注意，在“正常”、密集的 DataFrame 对象上它运行良好。然而，由于我的尺寸很大，这对我来说是一个很大的不便:

1. 转置数据框(需要很长时间)
2. 转换为密集数据帧(占用太多内存)

我对 Pandas 比较陌生，所以我可能遗漏了一些东西。无论如何，我们将不胜感激!

Answer 1

根据@Jeff 对我的问题的评论，到目前为止，Pandas 的稀疏数据结构似乎还没有完全支持索引。希望这会很快改变!

GitHub 上的相关问题:

• > https://github.com/pydata/pandas/issues/4400
• > https://github.com/pydata/pandas/issues/6076