python - 如何计算tokoneized数据框中的匹配项

Question

我有一个数据框，其中包含 599 个标记化文本，每行一个。我还有这些列表:

grwoth = ['growth', 'grow', 'growing', 'grows']
syergies = ['synergies', 'synergy' ,'accretive', 'accretion','efficiencies' ,'efficient', 'efficiently' ]
intangibles = ['brand','branded','branding','brands','goodwill','patent','patents','goodwil']
customers = ['customer', 'customers' ,'consumer' ,'consumers' ]
technology = ['technological', 'technologically', 'technologies', 'technology', 'innovate', 'innovation', 'innovations', 'innovative', 'innovator', 'innovators']
human = ['employee', 'employees', 'employees', 'team', 'teamed', 'teaming', 'teams', 'Expertise' ]

我想在数据框中为每个列表创建一个新列，并计算每个文本中列表中的单词被计数的频率。

我尝试将它们输入到我的原始数据框中(没有标记化)，但这也不起作用。我有以下代码:

%%time

    growth = ['growth', 'grow', 'growing', 'grows']
    synergies = ['synergies', 'synergy' ,'accretive', 'accretion','efficiencies' ,'efficient', 'efficiently' ]
    intangibles = ['brand','branded','branding','brands','goodwill','patent','patents','goodwil']
    customers = ['customer', 'customers' ,'consumer' ,'consumers' ]
    technology = ['technological', 'technologically', 'technologies', 'technology', 'innovate', 'innovation', 'innovations', 'innovative', 'innovator', 'innovators']
    human = ['employee', 'employees', 'employees', 'team', 'teamed', 'teaming', 'teams', 'expertise' ]
    the = 'Wire'

    result_list=[]
    count_growth = 0
    count_human = 0
    count_technology= 0
    count_customers = 0
count_intagibles = 0
count_synergies = 0
count_the = 0
for file in file_list:
    name = file[len(input_path):]
    date = name[11:17]
    type_1 = name[17:20]
    with open(file, "r", encoding="utf-8", errors="surrogateescape") as rfile:
            # We need to encode/decode as some text files are not in utf-8 format
            text = rfile.read()
            text = text.encode('utf-8', 'ignore')
            text = text.decode('utf-8', 'ignore')

    for word in text.split():
        if word in growth:
            count_growth = count_growth +1
        if word in synergies:
            count_synergies = count_synergies +1 
        if word in intagibles:
            count_intagibles = count_intagibles+1
        if word in customers:
            count_customers = count_customers +1
        if word in technology:
            count_technology = count_technology +1
        if word in human:
            count_human = count_human +1
        if word == 'The':
            count_the = count_the +1
    length = len(text.split())




    a={"File": name, "Text": text,'the':count_the, 'Datum': date, 'File_type': type_1, 'length':length, 'grwoth':count_growth, 'synergies': count_synergies,'intagibles':count_intagibles,'customers':count_customers, 'technology':count_technology,'human':count_human,}
    result_list.append(a)

这里的问题是，它创建了总和，但不是像长度那样为每行创建总和。

提前感谢您提供任何解决方案!

Answer 1

您只需清除 for 循环内的变量即可。这样它就可以输出各个文件的计数，就像输出长度一样。

我希望我正确理解了你想要做什么。

代码如下:

for file in file_list:
    count_growth = 0
    count_human = 0
    count_technology= 0
    count_customers = 0
    count_intagibles = 0
    count_synergies = 0
    count_the = 0
    name = file[len(input_path):]
    date = name[11:17]
    type_1 = name[17:20]
    with open(file, "r", encoding="utf-8", errors="surrogateescape") as rfile:
            # We need to encode/decode as some text files are not in utf-8 format
            text = rfile.read()
            text = text.encode('utf-8', 'ignore')
            text = text.decode('utf-8', 'ignore')

    for word in text.split():
        if word in growth:
            count_growth = count_growth +1
        if word in synergies:
            count_synergies = count_synergies +1 
        if word in intagibles:
            count_intagibles = count_intagibles+1
        if word in customers:
            count_customers = count_customers +1
        if word in technology:
            count_technology = count_technology +1
        if word in human:
            count_human = count_human +1
        if word == 'The':
            count_the = count_the +1
    length = len(text.split())    
    a={"File": name, "Text": text,'the':count_the, 'Datum': date, 'File_type': type_1, 'length':length, 'grwoth':count_growth, 'synergies': count_synergies,'intagibles':count_intagibles,'customers':count_customers, 'technology':count_technology,'human':count_human,}
    result_list.append(a)