java - Hive 查询失败(AMI - 3.11.0、Hive- 0.13.1)

Question

此任务的诊断消息:容器 [pid=3347,containerID=container_1490354262227_0013_01_000104] 正在超出物理内存限制运行。当前使用情况:已使用 1 GB 物理内存中的 1.0 GB；使用了 1.5 GB 的 5 GB 虚拟内存。杀死容器。转储container_1490354262227_0013_01_000104的进程树: |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 3360 3347 334 7 3347 (java) 7596 396 1537003520 262629/usr/java/latest/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx864m -Djava.io.tmpdir=/mnt3/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1490354262227_0013/container_1490354262227_0013_01_000104/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1490 354262227_0013/container_1490354262227_0013_01_000104-Dyarn.app .container.log.filesize=0 -Dhadoop.root.logger=INFO，CLA org.apache.hadoop.mapred.YarnChild 10.35.178.86 49938 attempts_1490354262227_0013_m_000004_3 104 |- 3347 2563 3347 3347 (bash) 0 1 115806208 698/bin/bash -c/usr/java/latest/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=警告 -Xmx864m -Djava.io.tmpdir=/mnt3/var/lib/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application_1490354262227_0013/container_1490354262227_0013_01_000104/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/mnt/var/log/hadoop/userlogs/application_1490354262227_0013/container_1490354262227_0013_01_000104 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 10.35.178.86 49938 attempts_1490354262227_0013_m _000004_3 104 1>/mnt/var/log/hadoop/用户日志/application_1490354262227_0013/container_1490354262227_0013_01_000104/stdout 2>/mnt/var/log/hadoop/userlogs/application_1490354262227_0013/container_1490354262227_0013 _01_000104/stderr

Answer 1

Container [pid=3347,containerID=container_1490354262227_0013_01_000104] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 1.5 GB of 5 GB virtual memory used.

看起来您的进程需要更多内存，并且超出了定义的限制。

您需要增加容器大小<​​/p>

SET hive.tez.container.size=4096MB
SET hive.auto.convert.join.noconditionaltask.size=1370MB

了解更多相关信息here .