gpt4 book ai didi

c++ - 尽可能快地访问数百个文件

转载 作者:行者123 更新时间:2023-11-30 04:15:03 24 4
gpt4 key购买 nike

所以我有一个程序最多可以接收 1000 个文件路径,我的想法是读取 3 个特定字节以返回日期,这一切都很好。

当它启动时问题就开始了,内存使用率飞到最大值,几秒钟内我的电脑就死机了。

我猜打开过程会占用几兆内存或其他东西......关于如何在不使用大量内存的情况下实现我需要的东西有什么想法吗?

注意:我打开的文件大小约为 15 GB

int main(int argc, char *argv[])
{
string paths[1000] = {};
int date[3] = {0};

cout << "Arg count: " << argc << endl;
if (argc <= 1)
paths[0] = "PRIV.EDB";
else
for(int i = 1;i<argc;i++){
paths[i-1] = argv[i];
}

cout << "Start\n\n";
for (int i=0;i<1000;i++)
{
if (paths[i].empty())
break;
cout << paths[i] << endl;
ifstream pFile;
pFile.open(paths[i], ios::binary);
pFile.seekg(195);
date[0] = pFile.get();
date[1] = pFile.get();
date[2] = pFile.get();
cout << date[0] << " : " << date[1] << " : " << date[2] << " \n";
cout << endl;
pFile.clear();
pFile.close();
}
cout << "Fin\n";

if (argc <= 1)
getchar();

return date[0];
}

最佳答案

使用内存映射文件以获得更好的性能。

http://msdn.microsoft.com/en-us/library/dd997372.aspx

MSDN:

These memory-mapped files are suitable for working with extremely large source files

维基百科:

The primary benefit of memory mapping a file is increasing I/O performance, especially when used on large files... Accessing memory mapped files is faster than using direct read and write operations for two reasons. Firstly, a system call is orders of magnitude slower than a simple change to a program's local memory. Secondly, in most operating systems the memory region mapped actually is the kernel's page cache (file cache), meaning that no copies need to be created in user space.

为了便于实现,请参阅: http://www.boost.org/doc/libs/1_54_0/doc/html/interprocess.html

使用:

提升/进程间/文件映射

提升/进程间/mapped_region

关于c++ - 尽可能快地访问数百个文件,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/18572695/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com