c - 重新编码 malloc 和页面大小

Question

我正在用 mmap 函数重新编码 malloc。我正在使用最适合的算法，并且能够在一页中分配和取消分配。当我想分配小于页面大小时，我的 malloc 函数运行良好。

但我不明白我应该如何处理大于页面大小的东西的分配？

Answer 1

默认的 libc malloc 实现已经使用mmap 和MAP_ANONYMOUS 分配大的单一内存分配。

为了演示，编译这个:

#include <stdio.h>
#include <stdlib.h>

int
main (int argc, char **argv)
{
  void *i = malloc (100 * 1024 * 1024);
  exit (0);
}

然后在strace下运行:

$ strace ./x 2>&1

execve("./x", ["./x"], [/* 21 vars */]) = 0
brk(0)                                  = 0x135d000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0fbb000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=105169, ...}) = 0
mmap(NULL, 105169, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f7ad0fa1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\30\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1811128, ...}) = 0
mmap(NULL, 3925176, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f7ad09dc000
mprotect(0x7f7ad0b91000, 2093056, PROT_NONE) = 0
mmap(0x7f7ad0d90000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b4000) = 0x7f7ad0d90000
mmap(0x7f7ad0d96000, 17592, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0d96000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0fa0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0f9f000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0f9e000
arch_prctl(ARCH_SET_FS, 0x7f7ad0f9f700) = 0
mprotect(0x7f7ad0d90000, 16384, PROT_READ) = 0
mprotect(0x600000, 4096, PROT_READ)     = 0
mprotect(0x7f7ad0fbd000, 4096, PROT_READ) = 0
munmap(0x7f7ad0fa1000, 105169)          = 0
mmap(NULL, 104861696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7aca5db000
exit_group(0)                           = ?

这一行...

mmap(NULL, 104861696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7aca5db000

...是大块的分配。

这就是您问题的答案。你分配一个大尺寸和 MAP_PRIVATE|MAP_ANONYMOUS。

来自 malloc 的手册页

Normally, malloc() allocates memory from the heap, and adjusts the size of the heap as required, using sbrk(2). When allocating blocks of memory larger than MMAP_THRESHOLD bytes, the glibc malloc() implementation allocates the memory as a private anonymous mapping using mmap(2). MMAP_THRESHOLD is 128 kB by default, but is adjustable using mallopt(3). Allocations performed using mmap(2) are unaffected by the RLIMIT_DATA resource limit (see getrlimit(2)).

来自 mallopt 的手册页

M_MMAP_THRESHOLD

When an allocation request larger than the given value cannot be satisfied by an existing free chunk, the memory is guaranteed to be obtained with mmap(). Smaller requests might be allocated with either of mmap() or sbrk(). mmap()-allocated memory can be immediately returned to the OS when it is freed, but this is not true for all memory allocated with sbrk(); however, memory allocated by mmap() and later freed is neither joined nor reused, so the overhead is greater. Default: 128*1024.

M_MMAP_MAX

The given value sets the maximum number of mmap()-allocated chunks allowed to be in use at a given time (even if the size of the allocation request exceeds the value of the M_MMAP_THRESHOLD parameter). This is useful on systems where the mmap() implementation scales poorly. A value of 0 disables the use of mmap(). Default: 65536.

如果您将 M_MMAP_TRESHOLD 调低至零(我不知道它是否会那么有效)，我猜每个分配都将通过 mmap 完成，无需更改一行以上的代码。但无论如何，您的问题的答案是 mmap，其参数与上述类似。

请注意，有太多单独的 mmap 区域可能是不可取的，在这种情况下，您需要在单个区域或多个区域内进行分配。在不更改地址的情况下增加 mmap 区域可能会很困难，这可能会在分配碎片化时使将内存返回给操作系统变得困难。