python - Pandas SparseDtype 不适用于 GroupBy

Question

data.groupby(by="DAY").agg({"CLOSING_DATE": min})

当我尝试对数据框进行分组以获取稀疏列的最旧日期(CLOSING_DATE 大部分为空)时，为什么会出现以下错误？

Traceback (most recent call last):
File "<ipython-input-23-37f9fe161304>", line 1, in <module>
data[:10000].groupby(by="DAY").agg({"CLOSING_DATE": min})
File "/home/user/miniconda3/envs/churn/lib/python3.8/site-packages/pandas/core/groupby/generic.py", line 951, in aggregate
result, how = self._aggregate(func, *args, **kwargs)
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/base.py", line 416, in _aggregate
result = _agg(arg, _agg_1dim)
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/base.py", line 383, in _agg
result[fname] = func(fname, agg_how)
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/base.py", line 367, in _agg_1dim
return colg.aggregate(how)
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/generic.py", line 252, in aggregate
return getattr(self, cyfunc)()
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1553, in min
return self._agg_general(
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1000, in _agg_general
result = self._cython_agg_general(
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1035, in _cython_agg_general
result, agg_names = self.grouper.aggregate(
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/ops.py", line 591, in aggregate
return self._cython_operation(
File "/home/user/miniconda3/envs/py_env/lib/python3.8/site-packages/pandas/core/groupby/ops.py", line 471, in _cython_operation
raise NotImplementedError(f"{values.dtype} dtype not supported")
NotImplementedError: Sparse[float64, nan] dtype not supported

Answer 1

这是 Pandas 中的一个错误，与最近对 cython 优化 groupbys 的重构有关:
https://github.com/pandas-dev/pandas/issues/38980
你有两个选择:

将您正在使用的 pandas 版本降级到 1.1.4 并等待错误修复(可能大约 4-6 周)

使用 to_dense() 在 groupby 之前将稀疏矩阵转换为稠密矩阵