c++ - 使用 std::ifstream::read 读取大二进制 block > INT32_MAX

Question

我有一个大文件，大约 4GB。我需要读取该文件中大于 INT32_MAX 的部分。 std::ifstream::read() 接受一个 std::streamsize 作为第二个输入(要读取的字节数)。因为我使用的是 64 位，所以它是一个 typedef 到 ptrdiff_t，它应该是一个 int64_t。所以我希望能够立即读取 9223372036854775807。我下面的例子证明我错了。当我读取超过 INT32_MAX 时设置故障位。

我错过了什么？

#include <fstream>
#include <iostream>
#include <limits>

int main() {
  std::cout << "Maximum of std::streamsize: "
            << std::numeric_limits<std::streamsize>::max() << std::endl;
  std::cout << "INT32_MAX: " << std::numeric_limits<int32_t>::max()
            << std::endl;

  auto const filename = R"(C:\TEMP\test.dat)";  // a large file > INT32_MAX

  auto dataStream = std::ifstream();
  dataStream.open(filename, std::ios_base::binary);
  dataStream.seekg(0, dataStream.end);
  size_t filesize = dataStream.tellg();
  std::cout << "Size of file: " << filesize << std::endl;

  // buffer for the whole file
  auto buffer = new uint8_t[filesize];

  dataStream.seekg(0, dataStream.beg);
  std::cout << "Reading INT32_MAX bytes..." << std::endl;
  dataStream.read(reinterpret_cast<char*>(buffer),
                  std::numeric_limits<int32_t>::max());
  std::cout << "Read failed: " << dataStream.fail() << std::endl;

  dataStream.seekg(0, dataStream.beg);
  std::cout << "Reading INT32_MAX + 1 bytes..." << std::endl;
  dataStream.read(
      reinterpret_cast<char*>(buffer),
      static_cast<int64_t>(std::numeric_limits<int32_t>::max()) + 1);
  std::cout << "Read failed: " << dataStream.fail() << std::endl;

  delete[] buffer;
}

我编译了:

$ g++ --version
g++.exe (Rev2, Built by MSYS2 project) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

在 Windows 7 笔记本电脑上提供:

Maximum of std::streamsize: 9223372036854775807
INT32_MAX: 2147483647
Size of file: 4001202702
Reading INT32_MAX bytes...
Read failed: 0
Reading INT32_MAX + 1 bytes...
Read failed: 1

我通过读取多个 INT32_MAX 大小的 block 解决了这个问题。我很想知道为什么失败了。

编辑:我做了更多测试。使用 GCC 8.3 在 Linux 上编译:有效，使用 MSVC15 编译:有效。

所以我猜 MinGW-W64 自带的 libstdc++ 有问题。

EDIT2:问题仍然存在

$ gcc --version
gcc.exe (Rev8, Built by MSYS2 project) 10.3.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Answer 1

你能试试这个吗

auto buffer = new int32_t[filesize];