hadoop - KiteSdk 1.1.0 csv导入IOError

Question

在上使用 HDP-2.5 并在Ubuntu-14.04 上运行此命令并

$ ./kite-dataset csv-import ./test.csv  test_schema

尝试使用KiteSdk import raw csv将数据 ver.1-1-0放入Hive
并具有以下 IOError :

1 job failure(s) occurred: org.kitesdk.tools.CopyTask: Kite(dataset:file:/tmp/444e6fc4-10e2-407d-afaf-723c408a6d... ID=1 (1/1)(1): java.io.FileNotFoundException: File file:/hdp/apps/2.5.0.0-1245/mapreduce/mapreduce.tar.gz does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:624) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:850) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:614) at org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:125) at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:468) at org.apache.hadoop.fs.FilterFs.resolvePath(FilterFs.java:158) at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2195) at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2191) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2191) at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:603) at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:457) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:142) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.submit(CrunchControlledJob.java:329) at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.startReadyJobs(CrunchJobControl.java:204) at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.pollJobStatusAndStartNewOnes(CrunchJobControl.java:238) at org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:112) at org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:55) at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:83) at java.lang.Thread.run(Thread.java:745)

我检查了文件 "hdfs:/hdp/apps/2.5.0.0-1245/mapreduce/mapreduce.tar.gz"存在，并且在很长一段时间内都找不到解决该错误的方法。

任何帮助是极大的赞赏。

Answer 1

我遇到了相同的错误，我通过创建/hdp/apps/2.5.0.0-1245/mapreduce然后解决了该问题:
cp /usr/hdp/current/hadoop-client/mapreduce.tar.gz /hdp/apps/2.5.0.0-1245/mapreduce

然后这创建了一个新的错误:org.kitesdk.tools.CopyTask:Kite(dataset:file:/ tmp / 413a41a2-8813-4056-9433-3c5e073d80 ... ID = 1(1/1)(1):java。 io.FileNotFoundException:文件不存在:hdfs://sandbox.hortonworks.com:8020 / tmp / crunch-283520469 / p1 / REDUCE

我仍在尝试进行故障排除。