Hadoop LZO native 库和 JAR 不可用

Question

我从 http://apache.techartifact.com/mirror/hadoop/common/hadoop-1.1.2/hadoop-1.1.2-bin.tar.gz 下载了 Hadoop 1.1.2 .它的 native 目录没有 LZO native 库。

$ 找到 ~/softwares/hadoop-1.1.2/lib/native/

/home/deepakkv/softwares/hadoop-1.1.2/lib/native/
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32/libhadoop.so
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32/libhadoop.la
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32/libhadoop.a
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32/libhadoop.so.1
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-i386-32/libhadoop.so.1.0.0

/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64/libhadoop.so
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64/libhadoop.la
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64/libhadoop.a
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64/libhadoop.so.1
/home/deepakkv/softwares/hadoop-1.1.2/lib/native/Linux-amd64-64/libhadoop.so.1.0.0

$

当我在本地模式下使用 PigServer 运行 PIG 脚本时，我看到以下错误

parquet.hadoop.BadConfigurationException:找不到类 com.hadoop.compression.lzo.LzoCodec

如何获取 jar 文件和 natives？

Answer 1

LZO 是 GPL 许可的，因此它不能与 hadoop 一起发布(不像基于 BSD 的 snappy)你可以从 https://github.com/toddlipcon/hadoop-lzo 构建 LZO或从https://code.google.com/p/hadoop-gpl-packing/下载