hadoop - 在本地运行hadoop作业的问题

Question

我已经运行了一个Hadoop工作，如下所示:

hadoop jar VoteCount.jar / user / joy / District1 / user / joy / Result_District1

16/03/26 16:18:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/03/26 16:18:30 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
16/03/26 16:18:30 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
16/03/26 16:18:30 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/03/26 16:18:30 INFO input.FileInputFormat: Total input paths to process : 10
16/03/26 16:18:30 INFO mapreduce.JobSubmitter: number of splits:10
16/03/26 16:18:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local911715097_0001
16/03/26 16:18:30 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
16/03/26 16:18:30 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/03/26 16:18:30 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
16/03/26 16:18:30 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
16/03/26 16:18:30 INFO mapred.LocalJobRunner: Waiting for map tasks
16/03/26 16:18:30 INFO mapred.LocalJobRunner: Starting task: attempt_local911715097_0001_m_000000_0
16/03/26 16:18:30 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
16/03/26 16:18:30 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
16/03/26 16:18:30 INFO mapred.Task:  Using ResourceCalculatorProcessTree : null
16/03/26 16:18:30 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/joy/District1/booth-1.txt:0+70
16/03/26 16:18:30 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
16/03/26 16:18:30 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
16/03/26 16:18:30 INFO mapred.MapTask: soft limit at 83886080
16/03/26 16:18:30 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
16/03/26 16:18:30 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
16/03/26 16:18:30 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
16/03/26 16:18:30 INFO mapred.LocalJobRunner: 
16/03/26 16:18:30 INFO mapred.MapTask: Starting flush of map output
16/03/26 16:18:30 INFO mapred.MapTask: Spilling map output
16/03/26 16:18:30 INFO mapred.MapTask: bufstart = 0; bufend = 110; bufvoid = 104857600
16/03/26 16:18:30 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600

但是我无法浏览该作业，在浏览器中键入 http://localhost:8080/时，它将继续拒绝连接，并通过以下命令检查是否已创建输出文件夹:

    hadoop fs -ls /user/joy/Result_District1

显示正在生成一个临时文件夹，未生成part-XXXXX文件。

我的hadoop配置如下:

等/hadoop/hdfs-site.xml

<configuration>
<property>
    <name>dfs.replication</name>
    <value>1</value>
</property>

等/hadoop/core-site.xml

 configuration>
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
</property>
</configuration>

Answer 1

尝试http://localhost:50070/访问HDFS。

您的工作似乎无法正常运行。您可以粘贴整个日志吗？