python - scikit学习凝聚聚类错误

Question

我正在尝试使用 sklearn 进行凝聚聚类。在拟合步骤中，我收到此错误。该错误不会一直出现，如果我更改数据点的数量，那么我可能不会得到错误和凝聚聚类。我不太确定如何调试这个。我已经使用 fillnan 确保我的数据数组中没有 NaN 值。任何关于为什么会发生这种情况的想法都会有所帮助。

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-38-8acbe956f76e> in <module>()
     13     agg = AgglomerativeClustering(n_clusters=k,affinity="euclidean",linkage="ward")
     14     init = time.time()
---> 15     agg.fit(data)
     16     atime = time.time()
     17     labels = agg.labels_

C:\Python27\lib\site-packages\sklearn\cluster\hierarchical.pyc in fit(self, X, y)
    754                                        n_components=self.n_components,
    755                                        n_clusters=n_clusters,
--> 756                                        **kwargs)
    757         # Cut the tree
    758         if compute_full_tree:

C:\Python27\lib\site-packages\sklearn\externals\joblib\memory.pyc in __call__(self, *args, **kwargs)
    279 
    280     def __call__(self, *args, **kwargs):
--> 281         return self.func(*args, **kwargs)
    282 
    283     def call_and_shelve(self, *args, **kwargs):

C:\Python27\lib\site-packages\sklearn\cluster\hierarchical.pyc in ward_tree(X, connectivity, n_components, n_clusters, return_distance)
    189                           'for the specified number of clusters',
    190                           stacklevel=2)
--> 191         out = hierarchy.ward(X)
    192         children_ = out[:, :2].astype(np.intp)
    193 

C:\Python27\lib\site-packages\scipy\cluster\hierarchy.pyc in ward(y)
    463 
    464     """
--> 465     return linkage(y, method='ward', metric='euclidean')
    466 
    467 

C:\Python27\lib\site-packages\scipy\cluster\hierarchy.pyc in linkage(y, method, metric)
    662             Z = np.zeros((n - 1, 4))
    663             _hierarchy.linkage(dm, Z, n,
--> 664                                int(_cpy_euclid_methods[method]))
    665     return Z
    666 

scipy\cluster\_hierarchy.pyx in scipy.cluster._hierarchy.linkage (scipy\cluster\_hierarchy.c:8759)()

C:\Python27\lib\site-packages\scipy\cluster\_hierarchy.pyd in View.MemoryView.memoryview_copy_contents (scipy\cluster\_hierarchy.c:22026)()

C:\Python27\lib\site-packages\scipy\cluster\_hierarchy.pyd in View.MemoryView._err_extents (scipy\cluster\_hierarchy.c:21598)()

ValueError: got differing extents in dimension 0 (got 704882705 and 4999850001)

Answer 1

这是一个溢出问题，请注意 4999850001 - 2**32 = 704882705(输出的最后一行)。有些东西太大了，无法容纳 32 位整数。您应该尝试使用更少的数据点。