c++ - 大约 2Gb 后 C++ ofstream 写入性能下降

Question

我目前正在编写一个 C++ 程序，该程序需要通过按顺序写入缓冲区来在文件中写入大量数据(通常约为 5Gb)。
在这个例子中，我将 400Mb 的缓冲区写入 std::ofstream。我的磁盘是 SSD，几乎是空的。 4 个缓冲区后，写入性能下降。有人知道为什么以及是否可以避免这种情况吗？
这是我的代码:

#include <iostream>
#include <iomanip>
#include <fstream>
#include <chrono>

int main()
{
  unsigned int bufferSize = 4e8; //400Mb
  unsigned char* buffer = new unsigned char[bufferSize];
  for (unsigned int i = 0; i < bufferSize; i++)
    buffer[i] = (unsigned char) i % 256; // just "randomly" writing the buffer
  std::ofstream ofs("test.bin");
  std::chrono::steady_clock::time_point start_time;
  std::chrono::steady_clock::time_point stop_time;
  std::chrono::duration<double> duration;

  for (int i = 1; i <= 10; i++)
  {
    start_time = std::chrono::steady_clock::now();
    ofs.write((char*) buffer, bufferSize);
    stop_time = std::chrono::steady_clock::now();
    duration = stop_time - start_time;
    std::cout << "i = " << i << ", time spent copying the buffer = " << std::chrono::duration_cast<std::chrono::nanoseconds>(duration).count() / 1e6  << "ms" << std::endl;
  }

  ofs.close();
  delete[] buffer;
  return 0;
}

这是我运行它时得到的:

nirupamix@machine:~/$ ./main.out
i = 1, time spent copying the buffer = 166.267ms
i = 2, time spent copying the buffer = 170.698ms
i = 3, time spent copying the buffer = 177.484ms
i = 4, time spent copying the buffer = 210.693ms
i = 5, time spent copying the buffer = 475.933ms
i = 6, time spent copying the buffer = 793.295ms
i = 7, time spent copying the buffer = 822.195ms
i = 8, time spent copying the buffer = 828.539ms
i = 9, time spent copying the buffer = 850.651ms
i = 10, time spent copying the buffer = 794.542ms

谢谢你的时间!

Answer 1

向SSD写入大量数据时可能会出现几个问题

终端节流 - 当数据写入时间足够长时，SDD 会升温并变慢

您的数据不应该足够大。

SSD 缓存已满

根据您的 SSD 的年龄和技术，您的 SSD 可能有一些 DRAM 来缓存写入，然后再进入较慢的 SLC 缓存，最后是否已满实际 QLC 存储。 (更多见 anandtech。