c - 如果请求超出可用物理内存，如何使malloc/calloc失败(即，不要使用swap)

Question

malloc/calloc显然使用交换空间来满足超出可用空闲内存的请求。随着磁盘使用指示灯持续亮起，这几乎使系统挂起。在我遇到这种情况之后，我还不确定为什么，我编写了以下5行测试程序来检查这是否确实是系统挂起的原因，

/* --- test how many bytes can be malloc'ed successfully --- */
#include <stdio.h>
#include <stdlib.h>
int main ( int argc, char *argv[] ) {
  unsigned int nmalloc = (argc>1? atoi(argv[1]) : 10000000 ),
               size    = (argc>2? atoi(argv[2]) : (0) );
  unsigned char *pmalloc = (size>0? calloc(nmalloc,size):malloc(nmalloc));
  fprintf( stdout," %s malloc'ed %d elements of %d bytes each.\n",
    (pmalloc==NULL? "UNsuccessfully" : "Successfully"),
    nmalloc, (size>0?size:1) );
  if ( pmalloc != NULL ) free(pmalloc);
  } /* --- end-of-function main() --- */

如果您的两个命令行参数的乘积超过物理内存，则确实会挂断系统。最简单的解决方案是通过某种方式自动使malloc/calloc失败。更加困难且不可移植的是编写一个小的包装器，该包装器使用popen()的一个免费命令，解析输出，并且仅在可用的“空闲”内存可以满足该请求的情况下才调用malloc/calloc，也许会有一点安全性内置的。

有没有更简单，更便携的方法来实现这一目标？ (显然类似于 can calloc or malloc be used to allocate ONLY physical memory in OSX?这个问题，但我希望得到某种"is"的答案。)

埃迪特--------------
决定遵循汤姆的/proc/meminfo建议。也就是说，不是popen()“免费”，而是直接解析现有且易于解析的/proc/meminfo文件。然后是表格的单行宏

#define noswapmalloc(n) ( (n) < 1000l*memfree(NULL)/2? malloc(n) : NULL )

完成工作。如下所示，memfree()并不像我想要的那样可移植，但是如果/当有需要时，可以轻松，透明地将其替换为更好的解决方案，而现在并非如此。

#include <stdio.h>
#include <stdlib.h>
#define _GNU_SOURCE                     /* for strcasestr() in string.h */
#include <string.h>
char    *strcasestr();                  /* non-standard extension */

/* ==========================================================================
 * Function:    memfree ( memtype )
 * Purpose:     return number of Kbytes of available memory
 *              (as reported in /proc/meminfo)
 * --------------------------------------------------------------------------
 * Arguments:   memtype (I)     (char *) to null-terminated, case-insensitive
 *                              (sub)string matching first field in
 *                              /proc/meminfo (NULL uses MemFree)
 * --------------------------------------------------------------------------
 * Returns:     ( int )         #Kbytes of memory, or -1 for any error
 * --------------------------------------------------------------------------
 * Notes:       o
 * ======================================================================= */
/* --- entry point --- */
int     memfree ( char *memtype ) {
  /* ---
   * allocations and declarations
   * ------------------------------- */
  static char memfile[99] = "/proc/meminfo"; /* linux standard */
  static char deftype[99] = "MemFree";  /* default if caller passes null */
  FILE  *fp = fopen(memfile,"r");       /* open memfile for read */
  char  memline[999];                   /* read memfile line-by-line */
  int   nkbytes = (-1);                 /* #Kbytes, init for error */
  /* ---
   * read memfile until line with desired memtype found
   * ----------------------------------------------------- */
  if ( memtype == NULL ) memtype = deftype; /* caller wants default */
  if ( fp == NULL ) goto end_of_job;    /* but we can't get it */
  while ( fgets(memline,512,fp)         /* read next line */
  !=      NULL ) {                      /* quit at eof (or error) */
    if ( strcasestr(memline,memtype)    /* look for memtype in line */
    !=   NULL ) {                       /* found line with memtype */
      char *delim = strchr(memline,':'); /* colon following MemType */
      if ( delim != NULL )              /* NULL if file format error? */
        nkbytes = atoi(delim+1);        /* num after colon is #Kbytes */
      break; }                          /* no need to read further */
    } /* --- end-of-while(fgets()!=NULL) --- */
  end_of_job:                           /* back to caller with nkbytes */
    if ( fp != NULL ) fclose(fp);       /* close /proc/meminfo file */
    return ( nkbytes );                 /* and return nkbytes to caller */
  } /* --- end-of-function memfree() --- */

#if defined(MEMFREETEST)
int     main ( int argc, char *argv[] ) {
  char  *memtype = ( argc>1? argv[1] : NULL );
  int   memfree();
  printf ( " memfree(\"%s\") = %d Kbytes\n Have a nice day.\n",
        (memtype==NULL?" ":memtype), memfree(memtype) );
  } /* --- end-of-function main() --- */
#endif

Answer 1

扩展我对原始问题的评论:

如果要禁用交换，请使用swapoff命令(sudo swapoff -a)。我通常以这种方式运行我的机器，以避免在Firefox执行不应执行的操作时冻结。您可以使用setrlimit()(或ulimit命令)来设置最大VM大小，但这不能适当补偿突然决定是内存消耗的其他某些过程(请参见上文)。

即使您选择上述选项之一，也应阅读本答案的其余部分，以了解如何避免在第一次调用calloc()时进行不必要的初始化。

至于精确的测试工具，事实证明您正在触发GNU calloc()的优化的不幸异常。

这是我对另一个答案所作的评论(现已删除)，事实证明这并不是严格意义上的准确性:

I checked the glibc source for the default gnu/linux malloc library, and verified that calloc() does not normally manually clear memory which has just been mmap'd. And malloc() doesn't touch the memory at all.

事实证明，我错过了 calloc优化的一个异常(exception)。由于GNU malloc实现初始化malloc系统的方式，对 calloc的首次调用始终使用 memset()将新分配的存储设置为0。对 calloc()的所有其他调用都将通过整个 calloc逻辑，从而避免了在存储位置上调用 memset()的情况。已经新鲜出炉了。

因此，对测试程序的以下修改显示了截然不同的行为:

#include <stdio.h>
#include <stdlib.h>
int main ( int argc, char *argv[] ) {
  /* These three lines were added */
  void* tmp = calloc(1000, 1); /* force initialization */
  printf("Allocated 1000 bytes at %p\n", tmp);
  free(tmp);
  /* The rest is unchanged */
  unsigned int nmalloc = (argc>1? atoi(argv[1]) : 10000000 ),
               size    = (argc>2? atoi(argv[2]) : (0) );
  unsigned char *pmalloc = (size>0? calloc(nmalloc,size):malloc(nmalloc));
  fprintf( stdout," %s malloc'ed %d elements of %d bytes each.\n",
    (pmalloc==NULL? "UNsuccessfully" : "Successfully"),
    nmalloc, (size>0?size:1) );
  if ( pmalloc != NULL ) free(pmalloc);
}

请注意，如果将 MALLOC_PERTURB_设置为非零值，那么它将用于初始化malloc()的块，并强制将calloc()的块初始化为0。在下面的测试中将使用它。

下面，我使用 /usr/bin/time来显示执行期间的页面错误数。请注意次要错误的数量，这是操作系统在匿名mmap'd区域中零初始化先前未引用的页面的结果(以及其他一些情况，例如映射Linux页面高速缓存中已经存在的页面)。另外还要查看常驻集大小，以及执行时间。

$ gcc -Og -ggdb -Wall -o mall mall.c

$ # A simple malloc completes instantly without page faults
$ /usr/bin/time ./mall 4000000000
Allocated 1000 bytes at 0x55b94ff56260
 Successfully malloc'ed -294967296 elements of 1 bytes each.
0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 1600maxresident)k
0inputs+0outputs (0major+61minor)pagefaults 0swaps

$ # Unless we tell malloc to initialise memory
$ MALLOC_PERTURB_=35 /usr/bin/time ./mall 4000000000
Allocated 1000 bytes at 0x5648c2436260
 Successfully malloc'ed -294967296 elements of 1 bytes each.
0.19user 1.23system 0:01.43elapsed 99%CPU (0avgtext+0avgdata 3907584maxresident)k
0inputs+0outputs (0major+976623minor)pagefaults 0swaps

# Same, with calloc. No page faults, instant completion.
$ /usr/bin/time ./mall 1000000000 4
Allocated 1000 bytes at 0x55e8257bb260
 Successfully malloc'ed 1000000000 elements of 4 bytes each.
0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 1656maxresident)k
0inputs+0outputs (0major+62minor)pagefaults 0swaps

$ # Again, setting the magic malloc config variable changes everything
$ MALLOC_PERMUTE_=35 /usr/bin/time ./mall 1000000000 4
Allocated 1000 bytes at 0x5646f391e260
 Successfully malloc'ed 1000000000 elements of 4 bytes each.
0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 1656maxresident)k
0inputs+0outputs (0major+62minor)pagefaults 0swaps