c - 文件系统性能

Question

我正在尝试探索应用程序的访问特性如何显着影响文件系统性能。我想将请求大小从 1KB (1024) 更改为 64KB (65536)。

当我达到 8KB 时，即出现写入文件错误。我没有分配足够的空间吗？

./test
error writing file (rc = 200704, errno 0)

我在这里调整 1KB - 64KB 的大小:

request_size = 32*(1<<13); /* 1KB is 1^10 (or 1<<10) */

所以，这应该是 8kb = 8192

我现在应该请求 4mb 文件吗？我不确定那会是什么转变？

int main(int c, char *argv[]) {

    int fd;
    void *buffer, *buffer_aligned;
    int i, rc;
    unsigned long file_size, request_size;
    struct timeval start, stop;
    double seconds;

    /* allocate a 64KB page-aligned buffer */
    buffer = malloc(65536+4096);
    buffer_aligned = buffer+(4096-(((unsigned long)buffer)%4096));

    /* write a 1MB file with 32KB requests  */
    fd = open("foo", O_CREAT|O_RDWR, 0644);
    file_size = 1*(1<<20);  /* 1MB is 2^20 (or 1<<20). 1GB is 2^30 (or 1<<30) */
    request_size = 32*(1<<13); /* 1KB is 1^10 (or 1<<10) */
    gettimeofday(&start, NULL);
    for (i=0; i<file_size/request_size; i++) {
            if ((rc=write(fd, buffer_aligned, request_size))!=request_size) {
                    fprintf(stderr, "error writing file (rc = %i, errno %i)\n", rc, errno);
                    return 1;
            }
    }       
    gettimeofday(&stop, NULL);
    seconds = ((stop.tv_sec*1e6+stop.tv_usec)-(start.tv_sec*1e6+start.tv_usec))/1e6;
    printf("file size is %uMB\n", file_size>>20);
    printf("request size is %uKB\n", request_size>>13);
    printf("elapsed time is %.2lf seconds\n", seconds);
    printf("bandwidth is %.2lf MB/sec\n", file_size/seconds/(1<<20));
    close(fd);

    /* free buffer */
    free(buffer);

    return 0;
}

这不是 8KB 吗？

request_size = 32*(1<<13); /* 1KB is 1^10 (or 1<<10) */

Answer 1

可能发生的情况是，您在一次调用 write 中写入的数据量太大，以至于无法始终在第一次调用时完全写入。来自 write 的 GNU libc 文档:

The return value is the number of bytes actually written. This may be size, but can always be smaller. Your program should always call write in a loop, iterating until all the data is written.

• http://www.gnu.org/software/libc/manual/html_node/I_002fO-Primitives.html

(我曾尝试查看其他 libc 实现的文档，但它们都没有像 GNU 一样详细地描述此行为。他们只是说该函数“尝试写入 n 字节的数据，并返回字节数written”。有人提到这取决于文件描述符是否处于非阻塞模式。在任何情况下，write 通常直接映射到操作系统调用，因此行为也取决于操作系统。 )

所以基本上，您需要在 write 周围放置一个循环，检查直到它已写入 request_size 字节。像这样的东西:

unsigned long data_remaining = request_size;
void *data_offset = buffer_aligned;
while (data_remaining > 0)
{
    rc = write(fd, data_offset, data_remaining);
    if (rc == -1) {
        fprintf(stderr, "error writing file (rc = %i, errno %i)\n", rc, errno);
        return 1;
    }
    data_remaining -= rc;
    data_offset += rc;  /* you don't care about the data being written, so this is kind of unnecessary in your case */
}