python - 螃蟹 : why is crab's UserBasedRecommender so slow?

Question

螃蟹的基准是(http://www.slideshare.net/marcelcaraciolo/crab-a-python-framework-for-building-recommender-systems page-37)

Benchmarks Pure Python w/ Python w/ Scipy Dataset dicts and NumpyMovieLens 100k 15.32 s 9.56 s http://www.grouplens.org/node/73 Old Crab New Crab

however, in my case, I need to take more than 30 minutes to do it, I don't know the reason

my code is

model = MatrixPreferenceDataModel(recommend_data.data)
similarity = UserSimilarity(model, pearson_correlation)
recommender = UserBasedRecommender(model, similarity, with_preference=True)
recommender.recommend("6")

我的数据是 NumpyMovieLens 100k，其中包含 1000 个用户对 1700 部电影的 100,000 个评分。

Answer 1

当我一年多前最后一次查看 Crab 时，它使用的是密集数组，它应该使用 sparse matrices。 (例如 scipy.sparse )。这使得它变得非常慢以及内存消耗。随便扫了一眼，Crab好像快一年没更新了，估计还是老样子。