hadoop - 构建 Impala 是否依赖于 Hive、HBase 和 Sentry？

Question

我有一个 Hadoop 集群，有一个主节点和 3 个从节点。现在，我想在此集群上添加 Apache Impala 功能。我已经从 here 下载了 tarball .我想构建 Impala，但不确定先决条件是什么。有两个不同的来源:

1. This ，来自文档，其中说要求是:MySQL(或 PostgreSQL)、Hive metastore 和 Java 依赖项(很明显)。

2. apache-impala 目录中的 README.md 文件在解压 tar 包后创建。引用它:

   Impala can be built with pre-built components, downloaded from S3, or can be built with an in-place toolchain located in the thirdparty directory (not recommended). The components needed to build Impala are Apache Hadoop, Hive, HBase, and Sentry.

我对这两个来源感到困惑。我应该怎么办？一组明确的 Apache Impala 依赖项会很棒!

Answer 1

如果您仔细阅读Impala Requirements您会看到隐含了 Hadoop 支持，而哨兵需求隐藏在 Impala Security 中。页面底部附近的链接。

在 Java Dependencies 部分下，它说:

All Java dependencies are packaged in the impala-dependencies.jar file, which is located at /usr/lib/impala/lib/. These map to everything that is built under fe/target/dependency.

查看对应的pom.xml你会看到所有的依赖关系。 Grepping artifactId 显示以下内容:

$ grep artifactId fe/pom.xml 
    <artifactId>impala-parent</artifactId>
  <artifactId>impala-frontend</artifactId>
      <artifactId>json-smart</artifactId>
      <artifactId>impala-data-source-api</artifactId>
      <artifactId>hadoop-hdfs</artifactId>
      <artifactId>hadoop-common</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>hadoop-auth</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>hadoop-aws</artifactId>
      <artifactId>hadoop-azure-datalake</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>sentry-core-common</artifactId>
      <artifactId>yarn-extras</artifactId>
      <artifactId>sentry-core-model-db</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>sentry-provider-common</artifactId>
      <artifactId>sentry-provider-db</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>sentry-provider-file</artifactId>
      <artifactId>sentry-provider-cache</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>sentry-policy-common</artifactId>
      <artifactId>sentry-binding-hive</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>sentry-policy-engine</artifactId>
      <artifactId>sentry-service-api</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>parquet-hadoop-bundle</artifactId>
      <artifactId>hbase-client</artifactId>
           <artifactId>json-smart</artifactId>
      <artifactId>hbase-common</artifactId>
           <artifactId>json-smart</artifactId>
      <artifactId>hbase-protocol</artifactId>
      <artifactId>commons-lang</artifactId>
      <artifactId>java-cup</artifactId>
      <artifactId>libthrift</artifactId>
      <artifactId>hive-service</artifactId>
          <artifactId>hive-llap-server</artifactId>
          <artifactId>json-smart</artifactId>
      <artifactId>hive-serde</artifactId>

因此 README.md 中声明您需要 Hadoop、Hive、HBase 和 Sentry 来构建 Impala 是正确的。