python - 如何获取 CFG 语法词典中没有的单词？

Question

如何让程序返回语法未涵盖的单词列表？例如，考虑下面的代码:

    import nltk
    # Define the cfg grammar.
    grammar = nltk.CFG.fromstring("""
    S -> NP VP
    VP -> V NP
    NP -> det N | N
    V -> "eats" | "drinks"
    N -> "President" | "apple"
    det -> "The" | "a" | "an"
    """)
    sentence = "The President Michel eats banana"

    # Load the grammar into the ChartParser.
    cp = nltk.ChartParser(grammar)

    # Generate and print the parse from the grammar given the sentence tokens.
    for tree in cp.parse(sentence.split()):
        print(tree)

它只显示错误消息:ValueError:语法未涵盖某些输入单词:“'Michel'，'banana'”。

但是，我想让这些语法未涵盖的单词在程序的其他地方使用它们。

Answer 1

您可以使用grammar.check_coverage(sentence.split())，但它会引发相同的异常并显示缺失单词的列表。但是，查看 check_coverage 方法的源代码:

def check_coverage(self, tokens):
    """
    Check whether the grammar rules cover the given list of tokens.
    If not, then raise an exception.

    :type tokens: list(str)
    """
    missing = [tok for tok in tokens
               if not self._lexical_index.get(tok)]
    if missing:
        missing = ', '.join('%r' % (w,) for w in missing)
        raise ValueError("Grammar does not cover some of the "
                         "input words: %r." % missing)

您可以根据他们的示例编写一个新函数，例如:

def get_missing_words(grammar, tokens):
    """
    Find list of missing tokens not covered by grammar
    """
    missing = [tok for tok in tokens
               if not grammar._lexical_index.get(tok)]
    return missing

并在示例中使用 get_missing_words(grammar, Sentence.split()) 来获取 ['Michel', 'banana']。