python - 在 Python 中将单词解析为(前缀、词根、后缀)

Question

我正在尝试为一些文本数据创建一个简单的解析器。 (文本使用 NLTK 没有任何解析器的语言。)

基本上，我的前缀数量有限，可以是一个或两个字母；一个词可以有多个前缀。我也有一个或两个字母的后缀数量有限。它们之间的任何东西都应该是这个词的“词根”。许多单词会有更多的可能解析，所以我想输入一个单词并以元组(前缀、根、后缀)的形式返回可能的解析列表。

虽然我不知道如何构建代码。我粘贴了我尝试过的一种方法的示例(使用一些虚拟的英语数据使其更易于理解)，但这显然是不正确的。一方面，它真的很丑陋和多余，所以我相信有更好的方法来做到这一点。另一方面，它不适用于具有多个前缀或后缀或同时具有前缀和后缀的单词。

有什么想法吗？

prefixes = ['de','con']
suffixes = ['er','s']

def parser(word):
    poss_parses = []
    if word[0:2] in prefixes:
        poss_parses.append((word[0:2],word[2:],''))
    if word[0:3] in prefixes:
        poss_parses.append((word[0:3],word[3:],''))
    if word[-2:-1] in prefixes:
        poss_parses.append(('',word[:-2],word[-2:-1]))
    if word[-3:-1] in prefixes:
        poss_parses.append(('',word[:-3],word[-3:-1]))
    if word[0:2] in prefixes and word[-2:-1] in suffixes and len(word[2:-2])>2:
        poss_parses.append((word[0:2],word[2:-2],word[-2:-1]))
    if word[0:2] in prefixes and word[-3:-1] in suffixes and len(word[2:-3])>2:
        poss_parses.append((word[0:2],word[2:-2],word[-3:-1]))
    if word[0:3] in prefixes and word[-2:-1] in suffixes and len(word[3:-2])>2:
        poss_parses.append((word[0:2],word[2:-2],word[-2:-1]))
    if word[0:3] in prefixes and word[-3:-1] in suffixes and len(word[3:-3])>2:
        poss_parses.append((word[0:3],word[3:-2],word[-3:-1]))
    return poss_parses



>>> wordlist = ['construct','destructer','constructs','deconstructs']
>>> for w in wordlist:
...   parses = parser(w)
...   print w
...   for p in parses:
...     print p
... 
construct
('con', 'struct', '')
destructer
('de', 'structer', '')
constructs
('con', 'structs', '')
deconstructs
('de', 'constructs', '')

Answer 1

Pyparsing 将字符串索引和标记提取包装到它自己的解析框架中，并允许您使用简单的算术语法来构建您的解析定义:

wordlist = ['construct','destructer','constructs','deconstructs']

from pyparsing import StringEnd, oneOf, FollowedBy, Optional, ZeroOrMore, SkipTo

endOfString = StringEnd()
prefix = oneOf("de con")
suffix = oneOf("er s") + FollowedBy(endOfString)

word = (ZeroOrMore(prefix)("prefixes") + 
        SkipTo(suffix | endOfString)("root") + 
        Optional(suffix)("suffix"))

for wd in wordlist:
    print wd
    res = word.parseString(wd)
    print res.dump()
    print res.prefixes
    print res.root
    print res.suffix
    print

结果在一个名为 ParseResults 的丰富对象中返回，该对象可以作为简单列表、具有命名属性的对象或字典来访问。这个程序的输出是:

construct
['con', 'struct']
- prefixes: ['con']
- root: struct
['con']
struct


destructer
['de', 'struct', 'er']
- prefixes: ['de']
- root: struct
- suffix: ['er']
['de']
struct
['er']

constructs
['con', 'struct', 's']
- prefixes: ['con']
- root: struct
- suffix: ['s']
['con']
struct
['s']

deconstructs
['de', 'con', 'struct', 's']
- prefixes: ['de', 'con']
- root: struct
- suffix: ['s']
['de', 'con']
struct
['s']