java - 在我的本地计算机上创建 hadoop 客户端时出现问题

Question

我有一个在 aws 上运行的名称节点和数据节点。我配置了 foxyproxy 并检查了以下哪些是有效的:

 Ganglia Metrics Reports    master-public-dns/ganglia/
 Hadoop ResourceManager master-public-dns-name:9026
 Hadoop NameNode    master-public-dns-name:9101

我的 Java 代码是:

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import com.google.protobuf.ServiceException;

public class HBaseConnection {

public static void main(String [] args){

    System.out.println("HBaseConnection");

}

public void getConnection() throws IOException, ServiceException{

    Configuration config = HBaseConfiguration.create();
    config.clear();
    config.set("hbase.zookeeper.quorum", "MASTER_NODE_DNS");
    config.set("hbase.zookeeper.property.clientPort","2181");
    System.out.println("Getting HBaseConnection..");

    try {
        HBaseAdmin.checkHBaseAvailable(config);
    } catch (MasterNotRunningException e) {
        p("HBase is not running.");
        p(e.toString());
        System.exit(1);
    }

    QueryHBase query = new QueryHBase();
    query.run(config);



}

private static void p(String msg) {
    System.out.println(msg);
}
}

我无法连接到主节点并收到以下错误:

14/10/23 07:17:35 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error,         closing socket connection and attempting reconnect
java.net.ConnectException: Operation timed out
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

我已启用主安全组配置中的所有端口。

这个问题一直困扰着我。

Answer 1

确保连接到您的 Zookeeper 整体，它可能不与 NameNode 或 ResourceManager 位于同一位置。您应该确保可以访问 zookeeper-host:2181。