python - 将 DataFrame 中的值替换为 pos_tags 字典

Question

这里我有一个 Pandas Dataframe，其中包含包含文本的“body”列。

         body
0   David Beckham's dreams of kick starting his ow...
1   Ascension Island. Picture: NASA, via Wikicommo...
2   So far this downturn, almost 10,000 direct min...
3   \nHOUSTON - Wendy Davis continued to capitaliz...
4   If something can't go on for ever, it won't. -...
5   \nPublished 04/10/2014 | 02:30\nTaoiseach Enda...
6   Ebola is having catastrophic economic conseque...
7   A British man has been raped at the Oktoberfes...
8   \nA top fashion journalist has sharply critiqu...
9   All over Ontario, giant wind turbines are spro...
10  Geneva - The Red Cross said on Monday that Sud...
11  \nPop quiz: What do pickles, vinegar, tempeh, ...

... ...
2284 rows × 1 columns

我想获取一个 DataFrame，将“body”下的文本转换为标签形式。我将其作为一个基本案例:

from nltk import pos_tag
pog = dict()
for txt in df['body'][0:3].str.split():
    text = nltk.pos_tag(txt)
    for postag in text:
        pog[postag[0]] = postag[1]
print(pog)

输出是:

{'David': 'NNP', "Beckham's": 'NNP', 'dreams': 'NNS', 'of': 'IN','kick': 'NN', 'starting': 'VBG', 'his': 'PRP$', 'own': 'JJ', 'American': 'JJ', 'soccer': 'NN', ...}

然后我写道:

df['body'] = df['body'].replace(pog)
print(df)

输出与上面的 DataFrame 完全相同，没有任何变化。我的想法是使用字典将原始 DataFrame 中的单词替换为标签。

我只是想知道为什么，如果有人有更好的主意用标签替换单词，请展示，谢谢。

Answer 1

在 pandas 中，您可以链接 apply 函数来获取输出。

## sample data frame
df = pd.DataFrame({'senten': ['I am not dancing','You are playing']})

df['new_sent'] = (df['senten']
                  .apply(word_tokenize)
                  .apply(pos_tag)
                  .apply(lambda x: ' '.join([y[1] for y in x])))

print(df)

             senten        new_sent
0  I am not dancing  PRP VBP RB VBG
1   You are playing     PRP VBP VBG