java - 内存映射文件的性能特点

Question

背景:

I have a Java application which does intensive IO on quite large memory mapped files (> 500 MB). The program reads data, writes data, and sometimes does both.

All read/write functions have similar computation complexity.

I benchmarked the IO layer of the program and noticed strange performance characteristics of memory mapped files:

• It performs 90k reads per second (read 1KB every iteration at random position)
• It performs 38k writes per second (write 1KB every iteration sequentially)
• It performs 43k writes per second (write 4 bytes every iteration at random position)
• It performs only 9k read/write combined operation per second (read 12 bytes then write 1KB every iteration, at random position)

The programs on 64-bit JDK 1.7, Linux 3.4.

The machine is an ordinary Intel PC with 8 threads CPU and 4GB physical memory. Only 1 GB was assigned to JVM heap when conducting the benchmark.

If more details are needed, here is the benchmark code: https://github.com/HouzuoGuo/Aurinko2/blob/master/src/test/scala/storage/Benchmark.scala

And here is the implementation of the above read, write, read/write functions: https://github.com/HouzuoGuo/Aurinko2/blob/master/src/main/scala/aurinko2/storage/Collection.scala

所以我的问题是:

• 给定固定的文件大小和内存大小，哪些因素会影响内存映射文件的随机读取性能？
• 给定固定的文件大小和内存大小，哪些因素会影响内存映射文件的随机写入性能？
• 如何解释读/写组合操作的基准测试结果？ (我期望它每秒执行超过 20K 次迭代)。

谢谢。

Answer 1

内存映射文件性能取决于磁盘性能、文件系统类型、可用于文件系统缓存的可用内存和读/写 block 大小。 linux 上的页面大小是 4K。因此，您应该期待 4k 读/写的最佳性能。如果页面未映射，则随机位置的访问会导致页面错误，并将读取新的页面。通常，如果您希望将文件视为一个内存数组(或 Java 中的 ByteBuffer )，则需要内存映射文件。