java - mapdb 内存不足

Question

我在测试 mapdb 时遇到内存不足错误。鉴于该项目的整个想法是将数据结构序列化到磁盘并避免内存问题，我认为我做错了什么。有什么想法我做错了吗？还是有bug？

    @Test
    public void testLarge() throws Exception {
        final HTreeMap<UUID, String> storage = DBMaker.newTempHashMap();

        String string = createDataSize(250);

        ArrayList<UUID> keys = new ArrayList<>();
        for (int i = 0; i < 320000; i++) {
            final UUID key = UUID.randomUUID();
            storage.put(key, string);
            keys.add(key);
        }

        for (UUID key : keys) {
            assertNotNull(storage.get(key));
        }

        for (UUID key : keys) {
            storage.remove(key);
        }

        assertEquals("nothing left", 0, storage.size());
    }

    /**
     * Creates a message of size @msgSize in KB.
     */
    private static String createDataSize(int msgSize) {
        // Java chars are 2 bytes
        msgSize = msgSize / 2;
        msgSize = msgSize * 1024;
        StringBuilder sb = new StringBuilder(msgSize);
        for (int i = 0; i < msgSize; i++) {
            sb.append('a');
        }
        return sb.toString();
    }
}

堆栈跟踪。我的函数中的第 29 行对应于“assertNotNull(storage.get(key));”线。

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2367)
    at java.lang.String.<init>(String.java:168)
    at org.mapdb.SerializerBase.deserializeString(SerializerBase.java:724)
    at org.mapdb.SerializerBase.deserialize(SerializerBase.java:932)
    at org.mapdb.SerializerBase.deserialize(SerializerBase.java:731)
    at org.mapdb.HTreeMap$1.deserialize(HTreeMap.java:134)
    at org.mapdb.HTreeMap$1.deserialize(HTreeMap.java:123)
    at org.mapdb.StorageDirect.recordGet2(StorageDirect.java:536)
    at org.mapdb.StorageDirect.get(StorageDirect.java:201)
    at org.mapdb.EngineWrapper.get(EngineWrapper.java:50)
    at org.mapdb.AsyncWriteEngine.get(AsyncWriteEngine.java:163)
    at org.mapdb.EngineWrapper.get(EngineWrapper.java:50)
    at org.mapdb.CacheHashTable.get(CacheHashTable.java:85)
    at org.mapdb.HTreeMap.get(HTreeMap.java:387)
    at com.sample.StorageTest.testLarge(StorageTest.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:292)

根据 unholysampler 的建议，我进行了以下更改，解决了问题。

diff -r 35918e46551a src/test/java/com/sample/StorageTest.java
--- a/src/test/java/com/sample/StorageTest.java Thu Mar 21 13:40:16 2013 -0600
+++ b/src/test/java/com/sample/StorageTest.java Thu Mar 21 13:42:24 2013 -0600
@@ -16,7 +16,9 @@

     @Test
     public void testLarge() throws Exception {
-        final HTreeMap<UUID, String> storage = DBMaker.newTempHashMap();
+        File tmpFile = File.createTempFile("largeTest", null);
+        DB db = DBMaker.newFileDB(tmpFile).deleteFilesAfterClose().journalDisable().make();
+        final HTreeMap<UUID, String> storage = db.getHashMap("name");

         String string = createDataSize(250);

@@ -25,6 +27,9 @@
             final UUID key = UUID.randomUUID();
             storage.put(key, string);
             keys.add(key);
+            if (i%100==0) {
+                db.commit();
+            }
         }

         for (UUID key : keys) {

Answer 1

文档显示有一个 commit() 方法。

db.commit();  //persist changes into disk

testLarge() 从未调用 commit()，因此尚未保留任何内容。

您也没有调用 close()，因此当您移动到另一个测试时， map 可能仍在内存中。

注意:我从未与该库合作过，但这是我最好的猜测。