hadoop - Zookeeper 连接到本地主机

Question

在运行将数据插入 HBase 的 Pig 脚本时，出现以下错误。

2013-10-25 14:57:03,147 [main-SendThread(localhost:2181)] INFO 
org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2013-10-25 14:57:03,147 [main-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session
2013-10-25 14:57:03,169 [main-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x41ead7bb150092, negotiated timeout = 180000

另外，当我看到 slave 日志时，它说无法与 master 建立连接，但是 master 说连接已成功建立。以下是两者的日志:

主日志:

zookeeper.ZooKeeper: Initiating client connection, connectString=slave:2181,hadoop-master:2181,ubuntu:2181 sessionTimeout=180000 watcher=hconnection
13/10/24 19:51:56 INFO zookeeper.ClientCnxn: Opening socket connection to server slave:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/24 19:51:56 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 104744@hadoop-master
13/10/24 19:51:56 INFO zookeeper.ClientCnxn: Socket connection established to slave:2181, initiating session
13/10/24 19:51:56 INFO zookeeper.ClientCnxn: Session establishment complete on server slave:2181, sessionid = 0x141ead77c250002, negotiated timeout = 180000

从属日志

2013-10-24 19:51:32,174 INFO org.apache.zookeeper.server.quorum.FastLeaderElection: New election. My id =  1, proposed zxid=0x80000002a
2013-10-24 19:51:32,180 WARN org.apache.zookeeper.server.quorum.QuorumCnxManager: Cannot open channel to 0 at election address hadoop-master:3888
java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

以下是我的配置:

hbase-env.sh

export HBASE_MANAGES_ZK=true

hbase-site.xml

<configuration>
    <property>
        <name>hbase.master</name> 
        <value>hadoop-master:60000</value>
        <description>
            The host and port that the HBase master runs at.
            A value of 'local' runs the master and a regionserver in a single process.
        </description>
    </property>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://hadoop-master:54310/hbase</value>
        <description>
            The directory shared by RegionServers.
        </description>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
        <description>
            The mode the cluster will be in. Possible values are
            false: standalone and pseudo-distributed setups with managed Zookeeper
            true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)
        </description>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
        <description>
            Property from ZooKeeper's config zoo.cfg.
            The port at which the clients will connect.
        </description>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>hadoop-master,slave,ubuntu</value>
        <description>
            Comma separated list of servers in the ZooKeeper Quorum.
            For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
            By default this is set to localhost for local and pseudo-distributed modes
            of operation. For a fully-distributed setup, this should be set to a full
            list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
            this is the list of servers which we will start/stop ZooKeeper on.
        </description>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
        <description>
            Default block replication.
            The actual number of replications can be specified when the file is created.
            The default is used if replication is not specified in create time.
        </description>
    </property>
</configuration>

通过输入命令 jps 我得到以下输出:

硕士:

104744 HMaster
91841 JobTracker
80184 TaskTracker
91475 DataNode
91222 NameNode
105062 HRegionServer
91747 SecondaryNameNode
104666 HQuorumPeer

在从站:

11533 HQuorumPeer
2444 SecondaryNameNode
5970 TaskTracker
11756 HRegionServer

Answer 1

这可能是运行 Pig 脚本时您的 HBase 配置不在类路径中的问题。您可以尝试的一件事是在 Pig 脚本中设置 zookeeper 连接设置，以查看它是否连接到正确的实例。如果是这样，您可以在启动脚本之前将 HBase 配置添加到类路径。