python - 如何高效地查找文档中的短语

Question

我有一个很大的短语列表(单个和多个单词；一些重叠)并且我有很多文档。最后，我只想存储每个文档的短语列表(来自大型短语列表)，而不是整个文档。实现这一目标的有效方法是什么？ (最好是Python)

示例:

phrase_list = ['cat', 'dog', 'tree', 'tree house'] // actually a few thousend if not million

// a list of a few thousend documents with longer text
doc_dictionary = {'doc1':"""the cat sat under the tree""",
                  'doc2':"""the dog chased the cat""",
                  'doc3':"""the boy loves his tree house"",}

result_dict = {'doc1': ['cat','tree'], 'doc2': ['dog', 'cat'], 'doc3': ['tree house']}

Answer 1

听起来您需要一个索引器和搜索引擎，例如 Lucene for Java。也许PyLucene端口会有帮助。