python - pandas:当数据为NaN时，无法进行逻辑运算

Question

我在 Pandas 中有一个很大的 DataFrame，并且 2 列可以有值，或者在未分配给任何值时为 NaN(空)。

我想根据这 2 列填充第三列。当不是 NaN 时，它需要一些值。其工作原理如下:

In [16]: import pandas as pd

In [17]: import numpy as np

In [18]: df = pd.DataFrame([[np.NaN, np.NaN],['John', 'Malone'],[np.NaN, np.NaN]], columns = ['col1', 'col2'])

In [19]: df
Out[19]:
   col1    col2
0   NaN     NaN
1  John  Malone
2   NaN     NaN

In [20]: df['col3'] = np.NaN

In [21]: df.loc[df['col1'].notnull(),'col3'] = 'I am ' + df['col1']

In [22]: df
Out[22]:
   col1    col2       col3
0   NaN     NaN        NaN
1  John  Malone  I am John
2   NaN     NaN        NaN

这也有效:

In [29]: df.loc[df['col1']== 'John','col3'] = 'I am ' + df['col2']

In [30]: df
Out[30]:
   col1    col2         col3
0   NaN     NaN          NaN
1  John  Malone  I am Malone
2   NaN     NaN          NaN

但是，如果我没有将所有值设置为 NaN，然后​​尝试最后一个 loc，则会出现错误!

In [31]: df = pd.DataFrame([[np.NaN, np.NaN],[np.NaN, np.NaN],[np.NaN, np.NaN]], columns = ['col1', 'col2'])

In [32]: df
Out[32]:
   col1  col2
0   NaN   NaN
1   NaN   NaN
2   NaN   NaN

In [33]: df['col3'] = np.NaN

In [34]: df.loc[df['col1']== 'John','col3'] = 'I am ' + df['col2']
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
c:\python33\lib\site-packages\pandas\core\ops.py in na_op(x, y)
    552             result = expressions.evaluate(op, str_rep, x, y,
--> 553                                           raise_on_error=True, **eval_kwargs)
    554         except TypeError:

c:\python33\lib\site-packages\pandas\computation\expressions.py in evaluate(op, op_str, a, b, raise_on_error, use_numexpr, **eval_kwargs)
    217         return _evaluate(op, op_str, a, b, raise_on_error=raise_on_error,
--> 218                          **eval_kwargs)
    219     return _evaluate_standard(op, op_str, a, b, raise_on_error=raise_on_error)

c:\python33\lib\site-packages\pandas\computation\expressions.py in _evaluate_standard(op, op_str, a, b, raise_on_error, **eval_kwargs)
     70         _store_test_result(False)
---> 71     return op(a, b)
     72

c:\python33\lib\site-packages\pandas\core\ops.py in _radd_compat(left, right)
    805     try:
--> 806         output = radd(left, right)
    807     except TypeError:

c:\python33\lib\site-packages\pandas\core\ops.py in <lambda>(x, y)
    802 def _radd_compat(left, right):
--> 803     radd = lambda x, y: y + x
    804     # GH #353, NumPy 1.5.1 workaround

TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('<U32') dtype('<U32') dtype('<U32')

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-34-3b2873f8749b> in <module>()
----> 1 df.loc[df['col1']== 'John','col3'] = 'I am ' + df['col2']

c:\python33\lib\site-packages\pandas\core\ops.py in wrapper(left, right, name, na_op)
    616                 lvalues = lvalues.values
    617
--> 618             return left._constructor(wrap_results(na_op(lvalues, rvalues)),
    619                                      index=left.index, name=left.name,
    620                                      dtype=dtype)

c:\python33\lib\site-packages\pandas\core\ops.py in na_op(x, y)
    561                 result = np.empty(len(x), dtype=x.dtype)
    562                 mask = notnull(x)
--> 563                 result[mask] = op(x[mask], y)
    564             else:
    565                 raise TypeError("{typ} cannot perform the operation {op}".format(typ=type(x).__name__,op=str_rep))

c:\python33\lib\site-packages\pandas\core\ops.py in _radd_compat(left, right)
    804     # GH #353, NumPy 1.5.1 workaround
    805     try:
--> 806         output = radd(left, right)
    807     except TypeError:
    808         raise

c:\python33\lib\site-packages\pandas\core\ops.py in <lambda>(x, y)
    801
    802 def _radd_compat(left, right):
--> 803     radd = lambda x, y: y + x
    804     # GH #353, NumPy 1.5.1 workaround
    805     try:

TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('<U32') dtype('<U32') dtype('<U32')

这就好像 Pandas 不喜欢 Column value == some text 如果所有值都是 NaN？？？

救命!

Answer 1

我认为，实际上这行代码所做的就是将一个字符串添加到第 1 列值(如果有任何值不为空)。

df.loc[df['col1'].notnull(),'col3'] = 'I am ' + df['col1']

所以你可以只检查是否有任何不为空的值，然后仅在有时执行操作:

if df['col1'].notnull().any():
    df['col3'] = 'I am ' + df['col1']

在以这种方式运行之前，您也不需要创建 col3 列。