windows - 如何以 Windows 特权帐户运行 Hadoop NodeManager

Question

我正在开发以下版本:

操作系统- Windows 8

Hadoop- 2.3(单节点集群)，Pig- 0.12.1，Hive- 0.13

我可以使用 Windows 命令提示符(没有管理员权限)启动 Namenode、Datanode、ResourceManager 和 NodeManager。当我尝试运行 mapreduce 程序时，映射任务没有启动，它始终显示为“Mapping-0%”。

注意:如果我使用管理员权限从 Windows 命令提示符启动“NodeManager”，则 MapReduce 程序可以正常工作。

节点管理器日志:

14/10/07 14:44:18 INFO http.HttpServer2: adding path spec: /node/*
14/10/07 14:44:18 INFO http.HttpServer2: adding path spec: /ws/*
14/10/07 14:44:18 INFO http.HttpServer2: Jetty bound to port 8042
14/10/07 14:44:18 INFO mortbay.log: jetty-6.1.26
14/10/07 14:44:18 INFO mortbay.log: Extract jar:file:/C:/kaveen/BDSDK/1
.1.0.8/SDK/Hadoop/share/hadoop/yarn/hadoop-yarn-common-2.3.0.jar!/webapps/node t
o C:\Users\kaveen\AppData\Local\Temp\Jetty_localhost_8042_node____j463i9\webapp
14/10/07 14:44:19 INFO mortbay.log: Started SelectChannelConnector@localhost:804
2
14/10/07 14:44:19 INFO webapp.WebApps: Web app /node started at 8042
14/10/07 14:44:19 INFO webapp.WebApps: Registered webapp guice modules
14/10/07 14:44:19 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0
:8031
14/10/07 14:44:19 INFO nodemanager.NodeStatusUpdaterImpl: Registering with RM us
ing finished containers :[]
14/10/07 14:44:19 INFO security.NMContainerTokenSecretManager: Rolling master-ke
y for container-tokens, got key with id 1336526610
14/10/07 14:44:19 INFO security.NMTokenSecretManagerInNM: Rolling master-key for
 nm-tokens, got key with id :-93940432
14/10/07 14:44:19 INFO nodemanager.NodeStatusUpdaterImpl: Registered with Resour
ceManager as 127.0.0.1:49344 with total resource of <memory:8192, vCores:8>
14/10/07 14:44:19 INFO nodemanager.NodeStatusUpdaterImpl: Notifying ContainerMan
ager to unblock new container-requests

ResourceManager 日志:

14/10/07 14:44:19 INFO util.RackResolver: Resolved 127.0.0.1 to /default-rack
14/10/07 14:44:19 INFO resourcemanager.ResourceTrackerService: NodeManager from
node 127.0.0.1(cmPort: 49344 httpPort: 8042) registered with capability: <memory
:8192, vCores:8>, assigned nodeId 127.0.0.1:49344
14/10/07 14:44:19 INFO rmnode.RMNodeImpl: 127.0.0.1:49344 Node Transitioned from
 NEW to RUNNING
14/10/07 14:44:19 INFO capacity.CapacityScheduler: Added node 127.0.0.1:49344 cl
usterResource: <memory:16384, vCores:16>
14/10/07 14:44:19 INFO rmnode.RMNodeImpl: Node 127.0.0.1:49344 reported UNHEALTH
Y with details: 1/1 local-dirs turned bad: /tmp/hadoop-kaveen/nm-local-dir;1/1 l
og-dirs turned bad: C:/kaveen/BDSDK/1.1.0.8/SDK/Hadoop/logs/userlogs
14/10/07 14:44:19 INFO rmnode.RMNodeImpl: 127.0.0.1:49344 Node Transitioned from
 RUNNING to UNHEALTHY
14/10/07 14:44:19 INFO capacity.CapacityScheduler: Removed node 127.0.0.1:49344
clusterResource: <memory:8192, vCores:8>

在集群节点 "http://localhost:8088/cluster/nodes" 看起来像下面的截图。

没有创建节点。

有谁知道如何在没有管理员权限的情况下在 Windows 上运行“NodeManager”？

提前致谢...

Answer 1

快速修复是在管理员权限下运行 C:hadoop\sbin\start-dfs 和 C:hadoop\sbin\start-yarn。