gpt4 book ai didi

python - 元组没有属性 'isdigit'

转载 作者:太空宇宙 更新时间:2023-11-04 06:54:55 26 4
gpt4 key购买 nike

我需要使用 NLTK 模块进行一些文字处理,但出现此错误:AttributeError: 'tuple' 对象没有属性 'isdigit'

有人知道如何处理这个错误吗?

Traceback (most recent call last):
File "preprocessing-edit.py", line 36, in <module>
postoks = nltk.tag.pos_tag(tok)
NameError: name 'tok' is not defined

PS C:\Users\moham\Desktop\Presentation> python preprocessing-edit.py
Traceback (most recent call last):
File "preprocessing-edit.py", line 37, in <module>
postoks = nltk.tag.pos_tag(tok)
File "c:\python34\lib\site-packages\nltk-3.1-py3.4.egg\nltk\tag\__init__.py", line 111, in pos_tag
return _pos_tag(tokens, tagset, tagger)
File "c:\python34\lib\site-packages\nltk-3.1-py3.4.egg\nltk\tag\__init__.py", line 82, in _pos_tag
tagged_tokens = tagger.tag(tokens)
File "c:\python34\lib\site-packages\nltk-3.1-py3.4.egg\nltk\tag\perceptron.py", line 153, in tag
context = self.START + [self.normalize(w) for w in tokens] + self.END
File "c:\python34\lib\site-packages\nltk-3.1-py3.4.egg\nltk\tag\perceptron.py", line 153, in <listcomp>
context = self.START + [self.normalize(w) for w in tokens] + self.END
File "c:\python34\lib\site-packages\nltk-3.1-py3.4.egg\nltk\tag\perceptron.py", line 228, in normalize
elif word.isdigit() and len(word) == 4:
AttributeError: 'tuple' object has no attribute 'isdigit'

import nltk

with open ("SHORT-LIST.txt", "r",encoding='utf8') as myfile:
text = (myfile.read().replace('\n', ''))

#text = "program managment is complicated issue for human workers"

# Used when tokenizing words
sentence_re = r'''(?x) # set flag to allow verbose regexps
([A-Z])(\.[A-Z])+\.? # abbreviations, e.g. U.S.A.
| \w+(-\w+)* # words with optional internal hyphens
| \$?\d+(\.\d+)?%? # currency and percentages, e.g. $12.40, 82%
| \.\.\. # ellipsis
| [][.,;"'?():-_`] # these are separate tokens
'''

lemmatizer = nltk.WordNetLemmatizer()
stemmer = nltk.stem.porter.PorterStemmer()


grammar = r"""
NBAR:
{<NN.*|JJ>*<NN.*>} # Nouns and Adjectives, terminated with Nouns

NP:
{<NBAR>}
{<NBAR><IN><NBAR>} # Above, connected with in/of/etc...
"""
chunker = nltk.RegexpParser(grammar)

tok = nltk.regexp_tokenize(text, sentence_re)

postoks = nltk.tag.pos_tag(tok)

#print (postoks)

tree = chunker.parse(postoks)

from nltk.corpus import stopwords
stopwords = stopwords.words('english')


def leaves(tree):
"""Finds NP (nounphrase) leaf nodes of a chunk tree."""
for subtree in tree.subtrees(filter = lambda t: t.label()=='NP'):
yield subtree.leaves()

def normalise(word):
"""Normalises words to lowercase and stems and lemmatizes it."""
word = word.lower()
word = stemmer.stem_word(word)
word = lemmatizer.lemmatize(word)
return word

def acceptable_word(word):
"""Checks conditions for acceptable word: length, stopword."""
accepted = bool(2 <= len(word) <= 40
and word.lower() not in stopwords)
return accepted


def get_terms(tree):
for leaf in leaves(tree):
term = [ normalise(w) for w,t in leaf if acceptable_word(w) ]
yield term

terms = get_terms(tree)


with open("results.txt", "w+") as logfile:
for term in terms:
for word in term:
result = word
logfile.write("%s\n" % str(word))
# print (word),
# (print)

logfile.close()

最佳答案

另一种简单的方法是更改​​这部分:

tok = nltk.regexp_tokenize(text, sentence_re)
postoks = nltk.tag.pos_tag(tok)

并将其替换为 nltk 标准单词分词器:

toks = nltk.word_tokenize(text)
postoks = nltk.tag.pos_tag(toks)

从理论上讲,性能和结果应该没有太大差异。

关于python - 元组没有属性 'isdigit',我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34097264/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com