apache - reducer 卡在16％-6ren

apache - reducer 卡在16％

转载作者：行者123 更新时间：2023-12-02 21:43:47

我的问题是映射器已完成，并且显示为100％，但另一方面，我的 reducer 却卡在16％。我的hadoop集群由2个数据节点和一个名称节点组成，在其上配置了辅助名称节点和作业跟踪器。 Tasktracker GUI显示 reducer 卡在15％的位置，尽管在15分钟内缓慢增加到16％。我使用的是hadoop版本0.20.2

我确实尝试了mapred-site.xml中的一些调整，但没有任何效果。

还将jobtracker和tasktracker日志附加到消息。

我不知道出了什么问题。

    <configuration>
<property>
  <name>mapred.job.tracker</name>
 <value>10.0.0.93:54311</value>
  <description>The host and port that the MapReduce job tracker runs at.</description>
</property>

<property>
    <name>mapred.child.java.opts</name>
        <value>-Xmx2024m</value>
          </property>

<property>
    <name>mapred.reduce.slowstart.completed.maps</name>
        <value>0.80</value>
          </property>
<property>
    <name>tasktracker.http.threads</name>
        <value>80</value>
          </property>
<property>
    <name>mapred.reduce.parallel.copies</name>
        <value>.4</value>
          </property>
<property>
    <name>mapred.tasktracker.shuffle.fadvise</name>
        <value>false</value>
          </property>

</configuration>

Jobtracker日志

2014-11-10 08:56:55,852 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:69  completedMapsInputSize:29478557296  completedMapsOutputSize:1518
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000000_0
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000068_0' has completed task_201411100854_0001_m_000068 successfully.
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:70  completedMapsInputSize:29746992753  completedMapsOutputSize:1540
2014-11-10 08:56:58,857 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000069_0' has completed task_201411100854_0001_m_000069 successfully.
2014-11-10 08:56:58,857 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:71  completedMapsInputSize:29831867549  completedMapsOutputSize:1562
2014-11-10 08:56:58,861 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201411100854_0001_m_000067_1' to tip task_201411100854_0001_m_000067, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 08:57:01,866 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000067_1' from 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 08:59:31,995 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000000_0
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000000_0
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000000_0 ... killing it
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000000_0: Too many fetch-failures
2014-11-10 09:04:32,242 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000000_1' to tip task_201411100854_0001_m_000000, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:04:32,242 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000000
2014-11-10 09:04:35,244 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000000_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:04:44,255 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000000_1' has completed task_201411100854_0001_m_000000 successfully.
2014-11-10 09:04:44,255 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:72  completedMapsInputSize:30905609374  completedMapsOutputSize:1584
2014-11-10 09:07:11,362 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:09:47,481 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000001_0 ... killing it
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000001_0: Too many fetch-failures
2014-11-10 09:14:47,698 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000001_1' to tip task_201411100854_0001_m_000001, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:14:47,698 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000001
2014-11-10 09:14:50,693 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000001_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:14:59,708 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000001_1' has completed task_201411100854_0001_m_000001 successfully.
2014-11-10 09:14:59,708 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:73  completedMapsInputSize:31979351199  completedMapsOutputSize:1606
2014-11-10 09:17:26,812 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:20:02,916 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000002_0 ... killing it
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000002_0: Too many fetch-failures
2014-11-10 09:25:03,128 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000002_1' to tip task_201411100854_0001_m_000002, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:25:03,128 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000002
2014-11-10 09:25:06,146 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000002_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:25:15,138 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000002_1' has completed task_201411100854_0001_m_000002 successfully.
2014-11-10 09:25:15,138 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:74  completedMapsInputSize:33053093024  completedMapsOutputSize:1628

任务跟踪器日志

2014-11-10 09:06:12,777 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:43401, bytes: 22, op: MAPRED_SHUFFLE, cliID: attempt_201411100854_0001_m_000000_1
2014-11-10 09:06:17,272 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:23,275 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:26,276 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:32,279 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:38,281 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:41,283 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:47,286 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:53,288 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:06:56,290 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:02,292 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:08,295 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:11,297 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:17,299 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:23,302 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:26,304 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:32,306 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:38,309 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:41,310 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:47,313 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:53,315 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:07:56,317 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:02,320 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:08,322 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:11,324 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:17,327 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:23,330 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:26,331 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:32,334 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:38,337 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:41,339 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:47,341 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:53,344 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:08:56,346 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:02,348 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:08,351 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:11,353 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:17,355 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:23,358 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:26,360 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:32,363 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:38,366 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:41,367 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:47,370 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:53,373 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:09:56,375 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:02,377 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:08,380 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:11,382 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:17,385 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:23,387 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:26,389 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:32,392 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:38,394 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:41,396 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:47,399 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:53,401 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:10:56,403 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:02,405 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:08,408 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:11,410 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:17,412 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:23,415 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:26,416 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:32,419 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:38,422 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:41,423 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:47,426 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:53,429 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:11:56,430 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:02,433 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:08,435 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:11,437 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:17,440 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:23,442 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:26,444 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:32,446 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:38,449 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:41,451 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:47,454 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:53,456 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:12:56,458 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:02,460 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:08,463 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:11,465 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:17,467 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:23,470 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:26,471 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:32,474 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:38,477 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:41,478 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:47,481 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:53,483 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:13:56,485 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:02,487 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:08,490 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:11,491 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:17,494 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:23,496 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:26,498 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:32,501 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:38,503 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:41,505 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:47,507 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:53,510 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:14:56,511 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:02,514 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:08,516 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:11,518 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:26,525 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:32,527 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:38,530 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:41,531 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:47,534 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:53,536 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:15:56,538 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:16:02,540 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:16:08,543 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201411100854_0001_m_000001_1 task's state:UNASSIGNED
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:10,983 WARN org.apache.hadoop.fs.FileSystem: "server093:54310" is a deprecated filesystem name. Use "hdfs://server093:54310/" instead.
2014-11-10 09:16:10,985 WARN org.apache.hadoop.fs.FileSystem: "server093:54310" is a deprecated filesystem name. Use "hdfs://server093:54310/" instead.
2014-11-10 09:16:10,992 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201411100854_0001_m_-136852211
2014-11-10 09:16:10,993 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201411100854_0001_m_-136852211 spawned.
2014-11-10 09:16:11,489 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201411100854_0001_m_-136852211 given task: attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:11,544 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:16:17,547 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) > 
2014-11-10 09:16:17,903 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:20,907 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:22,439 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:22,440 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201411100854_0001_m_000001_1 is done.
2014-11-10 09:16:22,440 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201411100854_0001_m_000001_1  was 0
2014-11-10 09:16:22,441 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2014-11-10 09:16:22,550 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201411100854_0001_m_-136852211 exited. Number of tasks it ran: 1
2014-11-10 09:16:23,549 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >

同一配置单元查询在单节点hadoop群集上运行良好。

最佳答案

我已经解决了我将主机名放在跨数据节点的所有配置文件中的所有位置，而不是使用IP地址。终于我的集群开始工作了。

关于apache - reducer 卡在16％，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/26836659/

文章推荐： hadoop - Hadoop的Pig脚本错误

文章推荐： javascript - ReactJS - 使用 iframe 静默更新 token

文章推荐： python - wordcount:reduce python程序抛出ValueError

android - 如何在 KitKat 中将文件写入可移动 SD 卡(不是外部 SD 卡)？
在 Android 的 API > 19 中是否有任何方法可以获取可移动 SD 卡的路径？与外部 SD 卡一样，我们有 Environment.getExternalStorageDirectory
Android API 19 及更高版本。写入 microSD 卡(外置 SD 卡)
一些 Android 设备有 microSD(或其他存储卡)插槽，通常安装为 /storage/sdcard1 据我所知，自 Android 4.4 起 Google 限制了对此内存的访问，并在 An
Java 卡检测到与智能卡的通信错误
我使用 Java Card 2.1.2 SDK 和 GPShell 作为与设备通信的方式在 Java Card 上构建一个项目。我从 GpShell 测试了 helloworld 示例，并成功发送了
安卓广播接收器和双 SIM 卡
我开发了一个应用程序，它有一个来电接收器，它适用于所有手机。一位用户有一部双 SIM 卡安卓手机。该应用程序适用于第一张 SIM 卡。但是当有人调用他的第二张 SIM 卡时，我们的应用程序不会被调用。
具有固定高度和响应图像的 CSS 卡
我有一个带预览的文件输入。这是笔 Codepen 我想强制高度，我无法理解我该怎么做。我想将此组件的高度固定为 300px(示例)，我还需要保持加载图像的正确纵横比，用灰色背景填充空白。现在我保持宽
gsm - 提供商如何检测到重复的 SIM 卡？
关闭。这个问题不符合Stack Overflow guidelines .它目前不接受答案。想改进这个问题？将问题更新为 on-topic对于堆栈溢出。 6年前关闭。 Improve this qu
kivy - 访问可移动 SD 卡
我正在使用此代码访问 SD card : import os from os.path import join from jnius import autoclass #from android.pe
io - 通过无竞争条件的缓冲区将数据写入 SD 卡
我正在为数据记录设备编写固件。它以 20 Hz 的频率从传感器读取数据并将数据写入 SD 卡。但是，向SD卡写入数据的时间并不一致(大约200-300 ms)。因此，一种解决方案是以一致的速率将数据写
html - Bootstrap ，卡+视频不适合手机屏幕
我正在使用以下代码将视频放到网站上，但是在垂直方向上，手机屏幕上只能看到视频的左半部分我不是网络开发人员。有人可以告诉我确切的内容吗，如何使其正确放置在手机屏幕上？是在youtube iframe
javascript - 在单行而不是列中显示 Vuetify 卡？
我正在使用 Vuetify 1.5 和 Vuetify 网格系统来设置我的布局。现在我有一个组件 HelloWorld我将其导入到我的 Parent 中成分。我已经在我的 HelloWorld 中设置
python - 如何并排打印我的 ASCII 卡？
我使用 python 制作了一个简单的二十一点游戏。我制作了游戏的其余部分，但我正在努力放入 ASCII 卡，所以这只是代码的一小部分。我尝试将 * len(phand) 放在附加行的末尾。虽然这确实
twitter - 如何测试 Twitter 卡？
我正在使用玩家卡设置 Twitter 卡。它可以在预览工具中运行，但文档说它需要在“twitter.com 现代桌面浏览器？ native iOs 和 Android Twitter 应用程序？mob
gsm - 如何以编程方式访问 SIM 卡？
任何旧的 GSM 兼容 SIM 卡(3G USIM 的奖励)。我想我需要一些硬件？谁能为业余爱好者推荐一些便宜的东西，以及一些更专业的东西？我认为会有一个带有硬件的 API 的完整文档，所以也许这
python - 如何并排打印我的 ASCII 卡？
我使用 python 制作了一个简单的二十一点游戏。我制作了游戏的其余部分，但我正在努力放入 ASCII 卡，所以这只是代码的一小部分。我尝试将 * len(phand) 放在附加行的末尾。虽然这确实
process - 用于并行计算的 CPU 卡？
我记得前一段时间读到有 cpu 卡供系统添加额外的处理能力来进行大规模并行化。任何人都有这方面的经验和任何资源来研究项目的硬件和软件方面吗？这项技术是否不如传统集群？它更注重功率吗？最佳答案有两个
java - 外部存储并不指大容量内部 SD 卡
我检查外部存储是否已安装并且可用于读/写，然后从中读取。我使用的是确切的官方 Android 示例代码 ( from here )。它说外部存储未安装。 getExternalFilesDir(nu
Android 应用程序下载到 SD 卡
在 Android 2.1 及更低版本中，Android 应用程序可以请求下载到 SD 卡上吗？另外我想知道应用程序是否可以请求一些包含视频的文件夹下载到 SD 卡上？以及如何做到这一点？提前致谢。
c++ - 驱动程序拒绝访问 PCI 卡
我们编写了一个 Windows 设备驱动程序来访问我们的自定义 PCI 卡。驱动程序使用 CreateFile 获取卡的句柄。我们最近在一次安装中遇到了问题，卡似乎停止工作了。我们尝试更换卡(更换似
Android:查找设备是否具有双 SD 卡
有些新设备(例如 Samsung Galaxy)带有两个 SD 卡。我想知道是否有任何方法可以确定设备是否有两张 SD 卡或一张 SD 卡。谢谢最佳答案我认为唯一的方法是使用检查可用根的列表 F
android - 无法访问 SD 卡
我正在尝试将文件读/写到 SD 卡。我已经尝试在我的真实手机和 Eclipse 中的模拟器上执行此操作。在这两种设备上，对/mnt/sdcard/或/sdcard 的权限仅为“d--------”，我

行者123

个人简介

我是一名优秀的程序员,十分优秀！

作者热门文章

滴滴打车优惠券免费领取

全站热门文章

首页

博学

6Ren·AI

商城

apache - reducer 卡在16％