language-agnostic - Google 文件系统中的 block 大小问题

Question

Google File System Paper ——

Chunk size is one of the key design parameters. We have chosen 64 MB, which is much larger than typical file sys- tem block sizes. Each chunk replica is stored as a plain Linux file on a chunkserver and is extended only as needed. Lazy space allocation avoids wasting space due to internal fragmentation, perhaps the greatest objection against such a large chunk size.

什么是惰性空间分配，它将如何解决内部碎片问题？

A small file consists of a small number of chunks, perhaps just one. The chunkservers storing those chunks may become hot spots if many clients are accessing the same file ... We fixed this problem by storing such executables with a higher replication factor and by making the batch- queue system stagger application start times.

什么是惊人的应用程序启动时间，它如何避免块服务器成为热点？

Answer 1

延迟空间分配意味着文件系统在写入文件之前实际上并不提供文件空间。它们通常被称为 sparse files .例如，如果只使用了 64MB 块文件的前 2MB，那么实际在磁盘上将只使用 2MB。

交错的应用程序启动时间仅意味着它们不会立即启动所有内容。如果每个应用程序在启动时都需要读取存储在 GFS 中的几个配置文件，如果它们都同时启动，就会出现负载问题。分散启动时间可以缓解这种情况。