hadoop - 尝试使用Jdbc访问Hive表时出错

Question

我需要编写一个API来从Hive表中获取数据，以便对其进行处理，因此，我尝试使用Jdbc访问Hive表。
一个简单的“从表中选择*”可以正常工作，但是“从表中选择count(*)会引发错误:

2017-04-20 18:05:57,709 INFO  [main] jdbc.Utils (Utils.java:parseURL(310)) - Supplied authorities: 10.94.154.125:10000
2017-04-20 18:05:57,712 INFO  [main] jdbc.Utils (Utils.java:parseURL(397)) - Resolved authority: 10.94.154.125:10000
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2017-04-20 18:05:57,790 INFO  [main] jdbc.HiveConnection (HiveConnection.java:openTransport(203)) - Will try to open client transport with JDBC Uri: jdbc:hive2://10.94.154.125:10000/CloudData
Running: select count(*) from  ipticket
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:296)
    at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:392)
    at HiveJdbcClient.main(HiveJdbcClient.java:35)

代码:

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;

public class HiveJdbcClient {
  private static String driverName = "org.apache.hive.jdbc.HiveDriver";

  public static void main(String[] args) throws SQLException {
    try {
      Class.forName(driverName);
    } catch (ClassNotFoundException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
      System.exit(1);
    }
    Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/CloudData", "", "");
    Statement stmt = con.createStatement();
    String tableName = "table1";
  String  sql = "select count(*) from  " + tableName;
    System.out.println("Running: " + sql);
    try{
         ResultSet res = stmt.executeQuery(sql);
         while (res.next()) {
              System.out.println(res.getString(1) + "\t" + res.getString(2)+"\t"+res.getString(3) + "\t" + res.getString(4));
            }

    }catch (SQLException f){
        f.printStackTrace();
    }

  }
}

Answer 1

因为Select count()启动了MapReduce作业，所以可能您的配置文件有问题
尝试在您的集群上测试 map 减少作业
执行这个
并使用mapreduce示例的jar版本更改hadoop-mapreduce-examples -... jar(如果您拥有hadoop 2.6，则其外观类似于/hadoop-mapreduce-examples-2.6.0.jar)

cd $HADOOP_HOME then hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-..*.jar pi 2 5

如果您有错误，则MR未配置为正常工作