python - 如何使用 PyTorch 对数组中的数字执行无监督聚类

Question

我得到了这个数组，我想将这些数字聚类/分组为相似的值。
输入数组的一个例子:

array([ 57, 58, 59, 60, 61, 78, 79, 80, 81, 82, 83, 101, 102, 103, 104, 105, 106] 

预期结果 :

array([57,58,59,60,61]), ([78,79,80,81,82,83]), ([101,102,103,104,105,106]) 

我尝试使用聚类，但如果我不知道要拆分多少个，我认为它不会起作用。

true = np.where(array>=1)
-> (array([ 57,  58,  59,  60,  61,  78,  79,  80,  81,  82,  83, 101, 102,
    103, 104, 105, 106], dtype=int64),)

Answer 1

动态分箱需要明确的标准并且不是一个容易自动化的问题，因为每个阵列可能需要一组不同的阈值来有效地分箱。
我想 Gaussian mixtures与 silhouette score criteria是你最好的选择。这是您要实现的目标的代码。轮廓分数可帮助您确定应使用的聚类/高斯分布的数量，并且对于 1D 数据非常准确且可解释。

import numpy as np
from sklearn.mixture import GaussianMixture
from sklearn.metrics import silhouette_score
import scipy
import matplotlib.pyplot as plt
%matplotlib inline

#Sample data
x = [57, 58, 59, 60, 61, 78, 79, 80, 81, 82, 83, 101, 102, 103, 104, 105, 106]

#Fit a model onto the data
data = np.array(x).reshape(-1,1)

#change value of clusters to check best silhoutte score
print('Silhoutte scores')
scores = []
for n in range(2,11):
    model = GaussianMixture(n).fit(data)
    preds = model.predict(data)
    score = silhouette_score(data, preds)
    scores.append(score)
    print(n,'->',score)

n_best = np.argmax(scores)+2 #because clusters start from 2

model = GaussianMixture(n_best).fit(data) #best model fit
    
#Get list of means and variances
mu = np.abs(model.means_.flatten())
sd = np.sqrt(np.abs(model.covariances_.flatten()))

#Plotting
extend_window = 50  #this is for zooming into or out of the graph, higher it is , more zoom out
x_values = np.arange(data.min()-extend_window, data.max()+extend_window, 0.1) #For plotting smooth graphs
plt.plot(data, np.zeros(data.shape), linestyle='None', markersize = 10.0, marker='o') #plot the data on x axis

#plot the different distributions (in this case 2 of them)
for i in range(num_components):
    y_values = scipy.stats.norm(mu[i], sd[i])
    plt.plot(x_values, y_values.pdf(x_values))

#split data by clusters
pred = model.predict(data)
output = np.split(x, np.sort(np.unique(pred, return_index=True)[1])[1:])
print(output)

Silhoutte scores
2 -> 0.699444729378163
3 -> 0.8962176943475543  #<--- selected as nbest
4 -> 0.7602523591781903
5 -> 0.5835620702692205
6 -> 0.5313888070615105
7 -> 0.4457049486461251
8 -> 0.4355742296918767
9 -> 0.13725490196078433
10 -> 0.2159663865546218

这将创建 3 个具有以下分布的高斯分布以将数据拆分为集群。

数组输出最终按相似值拆分

#output - 
[array([57, 58, 59, 60, 61]),
 array([78, 79, 80, 81, 82, 83]),
 array([101, 102, 103, 104, 105, 106])]