hadoop - Hadoop尝试将JVM挂断

Question

Hadoop 0.20.2

有几个作业需要一个一个地执行，并且某些尝试作业的JVM无法杀死。日志如下。如果您看到“ JVM Not Killed jvm_201208192339_6873_m_1286217329但只是删除了”，则任务跟踪器似乎找不到JVMId。我看过源代码。但是我找不到任务跟踪程序找不到JVMId的原因。顺便说一句，有13个任务跟踪程序，但是只有3个任务跟踪程序有此问题，我是否忘记配置某些内容？
有人帮我找到原因吗？谢谢。 ^ O ^

2012-09-20 13:52:56,655 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201208192339_6873_m_004334_0

2012-09-20 13:52:56,655 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201208192339_6873_m_004334_0

2012-09-20 13:52:56,655 INFO org.apache.hadoop.mapred.JvmManager: JVM Not killed jvm_201208192339_6873_m_1286217329 but just removed

2012-09-20 13:52:56,655 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 8

2012-09-20 13:52:56,655 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201208192339_6873_m_004334_0 not found in cache

2012-09-20 13:52:56,962 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201208192339_6873_m_004334_0 task's state:KILLED_UNCLEAN

2012-09-20 13:52:56,962 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201208192339_6873_m_004334_0 which needs 1 slots

2012-09-20 13:52:56,962 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 8 and trying to launch attempt_201208192339_6873_m_004334_0 which needs 1 slots

2012-09-20 13:52:56,968 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201208192339_6873_m_677724590

2012-09-20 13:52:56,968 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201208192339_6873_m_677724590 spawned.

2012-09-20 13:52:56,974 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /disk10/hdfs/mapred/local/ttprivate/taskTracker/root/jobcache/job_201208192339_6873/attempt_201208192339_6873_m_004334_0.cleanup/taskjvm.sh

2012-09-20 13:52:58,017 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201208192339_6873_m_677724590 given task: attempt_201208192339_6873_m_004334_0

2012-09-20 13:52:58,557 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201208192339_6873_m_004334_0 0.0%

2012-09-20 13:52:58,564 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201208192339_6873_m_004334_0 0.0% cleanup

2012-09-20 13:52:58,566 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201208192339_6873_m_004334_0 is done.

2012-09-20 13:52:58,566 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201208192339_6873_m_004334_0 was -1

2012-09-20 13:52:58,566 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 8

Answer 1

最终，出现此问题的节点还有另一个问题是其操作系统与硬件不匹配。在新操作系统中运行作业一段时间后，该问题不再出现。原来的旧操作系统在网络中表现不佳。它将降低网络带宽。