python - 频率矩阵 - 如何用 Python 构建

Question

我对 Python 的经验很少，并试图构建一个频率矩阵，该矩阵将总结有多少出版物标题包含特定关键字，并按学院和关键字组 (SDG) 进行分割。我当前的脚本使用测试数据，并显示当教师的标题具有关键字组之一中的关键字时出现的情况。我不知道如何将其组织成矩阵(按教师和 SDG 计算出现次数)并将其输出到 csv。

我的预期输出将具有以下 3 个列标题:

师资队伍、包含关键字的标题数量、SDG(关键字组)

除了获取频率之外，我还想找到一种能够使用不同长度的关键字列(data1)的方法。我怎样才能做到这一点？

任何帮助表示赞赏。谢谢

import pandas
import re   

data1 = {'SDG1':['Africa', 'Basic services', 'Class', 'Vulnerable','Wealth distribution'], 'SDG2':['Agricultural Orientation index',  'Agriculture', 'Consume', 'abcd', 'defg']} 
keywords = pandas.DataFrame(data1) 
keywords = keywords.applymap(lambda s:s.lower() if type(s) == str else s)

data2 = {'Title':['Africa blabla Class', 'Basic services blabla Class', 'Wealth abcdabcd', 'Wealth distribution 1', 'Wealth distribution 2','Consume blabla'], 'Faculty':['Eng', 'FHM', 'Eng', 'Architecture', 'Eng', 'Architecture']} 
titles =  pandas.DataFrame(data2) 
titles = titles.applymap(lambda s:s.lower() if type(s) == str else s)
dict_Fac = pandas.Series(titles.Faculty.values,index=titles.Title).to_dict()

res = False
NumberofTitleswithKeywords=0
for key,value in dict_Fac.items():
    title =  key
    for column in keywords:
        patterns = keywords[column]
        res = False
        for pattern in patterns:                          
            #print('Looking for "%s" in "%s" ->' % (pattern, title), end=' ')

            if re.search(pattern, title):
                res = True                   
        if res:
            print (value)
            print("FoundKeywordfrom"+str(column))

Answer 1

我假设您只是想检查标题是否包含 SDG1 或 SDG2 中的关键字。这可以通过使用Series.str.findall来完成:

import pandas
import re

data1 = {'SDG1':['Africa', 'Basic services', 'Class', 'Vulnerable','Wealth distribution'], 'SDG2':['Agricultural Orientation index',  'Agriculture', 'Consume', 'abcd', 'defg']}

data2 = {'Title':['Africa blabla Class', 'Basic services blabla Class', 'Wealth abcdabcd', 'Wealth distribution 1', 'Wealth distribution 2','Consume blabla'], 'Faculty':['Eng', 'FHM', 'Eng', 'Architecture', 'Eng', 'Architecture']}
titles =  pandas.DataFrame(data2)
titles = titles.applymap(lambda s:s.lower() if type(s) == str else s)

sdg1 = r"|\b".join(i+r"\b" for i in data1["SDG1"])
sdg2 = r"|\b".join(i+r"\b" for i in data1["SDG2"])

titles["SDG1"] = titles["Title"].str.findall(sdg1,flags=re.IGNORECASE)
titles["SDG2"] = titles["Title"].str.findall(sdg2,flags=re.IGNORECASE)

print (titles)

#
                         Title       Faculty                     SDG1       SDG2
0          africa blabla class           eng          [africa, class]         []
1  basic services blabla class           fhm  [basic services, class]         []
2              wealth abcdabcd           eng                       []         []
3        wealth distribution 1  architecture    [wealth distribution]         []
4        wealth distribution 2           eng    [wealth distribution]         []
5               consume blabla  architecture                       []  [consume]