hadoop - VIntWritable 与 IntWritable

Question

我明白 VIntWritable与 IntWritable 相比，可以显着减少存储整数所需的大小.

我的问题是:使用 VIntWritable 而不是 IntWritable 的成本是多少？是(仅)压缩所需的时间吗？换句话说，什么时候应该使用 IntWritable 而不是 VIntWritable？

Answer 1

How do you choose between a fixed-length and a variable-length encoding?

Fixedlength encodings are good when the distribution of values is fairly uniform across the whole value space, such as a (well-designed) hash function. Most numeric variables tend to have nonuniform distributions, and on average the variable-length encoding will save space. Another advantage of variable-length encodings is that you can switch from VIntWritable to VLongWritable, because their encodings are actually the same. So by choosing a variable-length representation, you have room to grow without committing to an 8-byte long representation from the beginning.

我刚从 definitive guide book 上捡到这个第98页