hadoop - 运行 mapreduce 作业时无法连接到 Zookeeper

Question

我正在使用 Accumulo 表作为输入运行 map reduce 作业，并将数据存储在 Accumulo 中的另一个表中。这是运行方法

public int run(String[] args) throws Exception {

        Opts opts = new Opts();
        opts.parseArgs(PivotTable.class.getName(), args);

        Configuration conf = getConf();

        conf.set("formula", opts.formula);

        Job job = Job.getInstance(conf);

        job.setJobName("Pivot Table Generation");
        job.setJarByClass(PivotTable.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);

        job.setMapperClass(PivotTableMapper.class);
        job.setCombinerClass(PivotTableCombiber.class);
        job.setReducerClass(PivotTableReducer.class);

        AccumuloInputFormat.setInputTableName(job, opts.dataTable);

        BatchWriterConfig bwConfig = new BatchWriterConfig();

        AccumuloOutputFormat.setBatchWriterOptions(job, bwConfig);
        AccumuloOutputFormat.setDefaultTableName(job, opts.pivotTable);
        AccumuloOutputFormat.setCreateTables(job, true);

        job.setInputFormatClass(AccumuloInputFormat.class);
        job.setOutputFormatClass(AccumuloOutputFormat.class);

        opts.setAccumuloConfigs(job);

        return job.waitForCompletion(true) ? 0 : 1;
    }

但问题是，当我运行该作业时，我收到一个异常，提示它无法连接到 zookeeper。

Error: java.lang.RuntimeException: Failed to connect to zookeeper (zookeeper.1:22181) within 2x zookeeper timeout period 30000
        at org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:124)
        at org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:164)
        at org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:43)
        at org.apache.accumulo.fate.zookeeper.ZooReader.getZooKeeper(ZooReader.java:47)
        at org.apache.accumulo.fate.zookeeper.ZooCache.getZooKeeper(ZooCache.java:59)
        at org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:159)
        at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:289)
        at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:238)
        at org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceID(ZooKeeperInstance.java:169)
        at org.apache.accumulo.core.client.ZooKeeperInstance.<init>(ZooKeeperInstance.java:159)
        at org.apache.accumulo.core.client.ZooKeeperInstance.<init>(ZooKeeperInstance.java:140)
        at org.apache.accumulo.core.client.mapreduce.RangeInputSplit.getInstance(RangeInputSplit.java:364)
        at org.apache.accumulo.core.client.mapreduce.AbstractInputFormat$AbstractRecordReader.initialize(AbstractInputFormat.java:495)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

我检查了 zookeeper 是否已启动并且正在运行。我运行 telnet 以查看端口是否已启动并且已启动。

我正在使用 $ACCUMULO_HOME/bin/tool.sh 来运行作业。任何帮助将不胜感激。

Answer 1

这是我的 hadoop slaves 中的主机文件的问题。主机名映射不正确。