c++ - 在 Visual Studio 中 fwrite 超过 4GB

Question

我正在尝试设置我的 Visual Studio ，以便我可以一次fwrite 8GB。

通过使用监视器跟踪内存，我可以看到 malloc 分配了 8GB。

然而，fwrite 的返回值为零，输出文件大小仅为 4GB。

size_t s = fwrite(result, sizeof(unsigned int), 0x80000000, fout);

我正在使用 x64， Release模式。

我应该使用任何其他设置吗？

Answer 1

使用最古老的 C 运行时函数来处理大数据没有太多意义，因为 Windows API 和 C++ 都有更好的方法来处理更大的数据，例如内存映射文件。对于 C++ 内存映射文件，boost 有几个实现。

如果你真的想使用 fwrite，那么尝试将它分开，因为 Visual C++ 的 fwrite 实现不会一次性写入大数组:

fwrite 是在 Windows WriteFile 函数之上实现的，它只能写入一个 DWORD 字节，此外还可以从它的实现中看到这个评论，在 C:\Program Files (x86)\微软 Visual Studio 12.0\VC\crt\src\fwrite.c 。

/***
*size_t fwrite(void *buffer, size_t size, size_t count, FILE *stream) -
*       write to the specified stream from the specified buffer.
*
*Purpose:
*       Write 'count' items of size 'size' to the specified stream from
*       the specified buffer. Return when 'count' items have been written
*       or no more items can be written to the stream.
*
*Entry:
*       buffer  - pointer to user's buffer
*       size    - size of the item to write
*       count   - number of items to write
*       stream  - stream to write to
*
*Exit:
*       Returns the number of (whole) items that were written to the stream.
*       This may be less than 'count' if an error or eof occurred. In this
*       case, ferror() or feof() should be used to distinguish between the
*       two conditions.
*
*Notes:
*       fwrite will attempt to buffer the stream (side effect of the _flsbuf
*       call) if necessary.
*
*       No more than 0xFFFE bytes may be written out at a time by a call to
*       write(). Further, write() does not handle huge buffers. Therefore,
*       in large data models, the write request is broken down into chunks
*       that do not violate these considerations. Each of these chunks is
*       processed much like an fwrite() call in a small data model (by a
*       call to _nfwrite()).
*...

请注意开始的部分 通过调用 write() 一次可以写出不超过 0xFFFE 字节 - 除了简单之外，用大块调用它没有任何优势内存 vs 使用较小的 block 进行多次调用并检查返回值。