java - 无论如何加速mapdb？

Question

我用整数键和字符串值测试了 mapdb，以在其中插入 10,000,000 个元素。这是我看到的:

Processed 1.0E-5  percent of the data  / time so far = 0  seconds 
Processed 1.00001  percent of the data  / time so far = 7  seconds 
Processed 2.00001  percent of the data  / time so far = 14  seconds 
Processed 3.00001  percent of the data  / time so far = 20  seconds 
Processed 4.00001  percent of the data  / time so far = 26  seconds 
Processed 5.00001  percent of the data  / time so far = 33  seconds 
Processed 6.00001  percent of the data  / time so far = 39  seconds 
Processed 7.00001  percent of the data  / time so far = 45  seconds 
Processed 8.00001  percent of the data  / time so far = 53  seconds 
Processed 9.00001  percent of the data  / time so far = 60  seconds 
Processed 10.00001  percent of the data  / time so far = 66  seconds 
Processed 11.00001  percent of the data  / time so far = 73  seconds 
Processed 12.00001  percent of the data  / time so far = 80  seconds 
Processed 13.00001  percent of the data  / time so far = 88  seconds 
Processed 14.00001  percent of the data  / time so far = 96  seconds 
Processed 15.00001  percent of the data  / time so far = 102  seconds 
Processed 16.00001  percent of the data  / time so far = 110  seconds 
Processed 17.00001  percent of the data  / time so far = 119  seconds 
Processed 18.00001  percent of the data  / time so far = 127  seconds 
Processed 19.00001  percent of the data  / time so far = 134  seconds 
Processed 20.00001  percent of the data  / time so far = 141  seconds 
Processed 21.00001  percent of the data  / time so far = 149  seconds 
Processed 22.00001  percent of the data  / time so far = 157  seconds 
Processed 23.00001  percent of the data  / time so far = 164  seconds 
Processed 24.00001  percent of the data  / time so far = 171  seconds 
Processed 25.00001  percent of the data  / time so far = 178  seconds 
.... 

在 178 秒内将大约 250 万个实例放入 map 中。对于 1000 万，大约需要 12 分钟。

然后我切换到更复杂的值并且速度大幅下降(将整个 10,000,000 个实例添加到 map 中需要 3-4 天)。有人对加快 mapdb 插入有任何建议吗？之前有任何与 MabDB 速度相关的经验/问题吗？

这里还有评价:http://kotek.net/blog/3G_map

更新:我使用了创建 map 的通用程序。这是一个伪代码:

DB db = DBMaker.newFileDB()....; 
... map = db.getHashMap(...);
loop (...) {   
map.put(...);
} 
db.commit();

Answer 1

此处为 MapDB 作者。

对于开始使用专门的序列化器，它们会更快一些:

Map m = dbmaker.createHashMap("a").keySerializer(Serializer.LONG).valueSerializer(Serializer.LONG).makeOrGet()

接下来，对于导入，我建议将 Data Pump 与 TreeMap 结合使用。一个例子在这里: https://github.com/jankotek/MapDB/blob/master/src/test/java/examples/Huge_Insert.java