pandas - 值太多，无法在 Python 中使用 NLTK 和 Pandas 解压

Question

我正在尝试使用 NLTK 和 Pandas 模块来尝试不同的方法来使 NLTK 的朴素贝叶斯正常工作，但我收到了“太多值无法解压”错误。

import pandas as pd
from pandas import DataFrame, Series
import numpy as np
import re
import nltk

### Remove cases with missing name or missing ethnicity information
def read_file():
    data = pd.read_csv("C:\sample.csv", encoding="utf-8")
    frame = DataFrame(data)
    frame.columns = ["Name", "Gender"]

    return frame
#read_file()

def gender_features(word):
    return {'last_letter': word[-1]}
#gender_features()

frame = read_file()
featuresets = [(gender_features(n), gender) for (n, gender) in frame]
train_set, test_set = features[500:], featuresets[:500]
classifier = nltkNaiveBayesClassifier.train(train_set)

Answer 1

我怀疑您在使用 panadas.DataFrame 时试图做比名称分类更大的事情，因为 DataFrame 对象通常在您的 RAM 有限并且想要使迭代数据以提取特征时使用磁盘空间:

a 2-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dict of Series objects. It is generally the most commonly used pandas object. Like Series, DataFrame accepts many different kinds of input:

• Dict of 1D ndarrays, lists, dicts, or Series
• 2-D numpy.ndarray
• Structured or record ndarray
• A Series
• Another DataFrame

我建议您先阅读 pandas 教程来了解该库:http://pandas.pydata.org/pandas-docs/dev/tutorials.html

然后从http://www.nltk.org/book/ch06.html了解NLTK分类

<小时/>

首先，访问 pandas.DataFrame 对象的方式存在一些问题。

要迭代数据帧的行，您应该这样做:

# Read file into pandas dataframe
df = DataFrame(pd.read_csv('sample.csv'))
df.columns = ['name', 'gender']

for index, row in df.iterrows():
    print row['name'], row['gender']

接下来要训练分类器，您应该执行以下操作:

import numpy as np
import pandas as pd
from pandas import DataFrame, Series

from nltk.corpus import names
from nltk.classify import NaiveBayesClassifier as nbc

# Create a sample.csv file
male_names = [','.join([i,'m']) for i in names.words('male.txt')]
female_names = [','.join([i,'m']) for i in names.words('female.txt')]
with open('sample.csv', 'w') as fout:
    fout.write('\n'.join(male_names+female_names))

# Feature extractor function.
def gender_features(word):
    return {'last_letter': word[-1]}

# Read file into pandas dataframe
df = DataFrame(pd.read_csv('sample.csv'))
df.columns = ['name', 'gender']

# Extract features. 
featuresets = [(gender_features(name), gender) for index, (name, gender) in df.iterrows()]
# Split train and test set
train_set, test_set = featuresets[500:], featuresets[:500]
# Train a classifier
classifier = nbc.train(train_set)
# Test classifier on "Neo"
print classifier.classify(gender_features('Neo'))

[输出]:

m