python - Pandas 中的多处理错误

Question

我尝试在 pandas 中使用多重处理:

from multiprocessing import Pool, cpu_count
def applyParallel(DF, func, data):
    with Pool(cpu_count()) as p:
        ret_list = p.map(func, [(DF, group) for name, group in data])
    return pd.concat(ret_list, axis  = 1)

def apply_row_foo(input_df, data_group):
    dfn = pd.DataFrame()
    for i in data_group['name']:
        dfn['result_' + str(i)] = input_df['student'].apply(lambda x : myfunction(x, i))
    return dfn

if __name__ == '__main__':
    z = applyParallel(df.groupby(df.index % 4), apply_row_foo, DATA)

TypeError: apply_row_foo() missing 1 required positional argument: data_group

如何解决这个问题？谢谢。

Answer 1

您已经定义了 apply_row_foo 来接受两个参数，但实际上您是在单个元组中使用这两个参数来调用它。解决此问题的一种方法是让 apply_row_foo 接受元组参数:

def apply_row_foo(params):
    input_df, data_group = params