gpt4 book ai didi

python - Scikit Learn RandomForest 内存错误

转载 作者:太空狗 更新时间:2023-10-30 01:01:42 26 4
gpt4 key购买 nike

我正在尝试在 mnist 手写数字数据集上运行 scikit 学习随机森林算法。在算法训练期间,系统会出现内存错误。请告诉我应该如何解决这个问题。

CPU 统计数据: Intel Core 2 Duo,4GB 内存

数据集的形状是60000, 784。 linux终端的完整错误如下:

> File "./reducer.py", line 53, in <module>
> main() File "./reducer.py", line 38, in main
> clf = clf.fit(data,labels) #training the algorithm File "/usr/lib/pymodules/python2.7/sklearn/ensemble/forest.py", line 202,
> in fit
> for i in xrange(n_jobs)) File "/usr/lib/pymodules/python2.7/joblib/parallel.py", line 409, in
> __call__
> self.dispatch(function, args, kwargs) File "/usr/lib/pymodules/python2.7/joblib/parallel.py", line 295, in
> dispatch
> job = ImmediateApply(func, args, kwargs) File "/usr/lib/pymodules/python2.7/joblib/parallel.py", line 101, in
> __init__
> self.results = func(*args, **kwargs) File "/usr/lib/pymodules/python2.7/sklearn/ensemble/forest.py", line 73, in
> _parallel_build_trees
> sample_mask=sample_mask, X_argsorted=X_argsorted) File "/usr/lib/pymodules/python2.7/sklearn/tree/tree.py", line 476, in fit
> X_argsorted=X_argsorted) File "/usr/lib/pymodules/python2.7/sklearn/tree/tree.py", line 357, in
> _build_tree
> np.argsort(X.T, axis=1).astype(np.int32).T) File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line
> 680, in argsort
> return argsort(axis, kind, order) MemoryError

最佳答案

设置 n_jobs=1 或升级到 scikit-learn 的最新版本。问题是当前发布的版本使用多个进程来并行拟合树,这意味着数据(Xy)需要被复制到这些进程。下一个版本将使用线程而不是进程,因此树学习者共享内存。

关于python - Scikit Learn RandomForest 内存错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/23118309/

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com