python - Pandas 在 2 列中应用并在一行中替换它们

Question

我正在尝试提高我的代码的性能，我想标记数据框的 2 列，我是这样的

submission_df['question1'] = submission_df.apply(lambda row: nltk.word_tokenize(row['question1']), axis=1)
submission_df['question2'] = submission_df.apply(lambda row: nltk.word_tokenize(row['question2']), axis=1)

我想也许我可以将它们合并在一行中，这样我只会在所有行(200 万行)上迭代一次，所以我想到了这样的事情

submission_df['question1'],submission_df['question2'] = submission_df.apply
    (lambda row: 
     (nltk.word_tokenize(row['question1']),
      nltk.word_tokenize(row['question2'])), axis=1)   

但没有用，也许还有其他方法可以改进它，而不是使用 apply 方法。

Answer 1

您可以简单地使用 apply 为带有 astype(str) 的选定列，即

submission_df[['question1','question2']]=submission_df[['question1','question2']].astype(str).apply(lambda row: [nltk.word_tokenize(row['question1']),nltk.word_tokenize(row['question2'])], axis=1)

示例:

import nltk
df = pd.DataFrame({"A":["Nice to meet you ","Nice to meet you ","Nice to meet you ",8,9,10],"B":[7,6,7,"Nice to meet you ","Nice to meet you ","Nice to meet you "]})
df[['A','B']] = df[['A','B']].astype(str).apply(lambda row: [nltk.word_tokenize(row['A']),nltk.word_tokenize(row['B'])], axis=1)

输出:

                          A                      B0  [Nice, to, meet, you]                    [7]1  [Nice, to, meet, you]                    [6]2  [Nice, to, meet, you]                    [7]3                    [8]  [Nice, to, meet, you]4                    [9]  [Nice, to, meet, you]5                   [10]  [Nice, to, meet, you]