gpt4 book ai didi

java.lang.UnsupportedOperationException : Not implemented by the DistributedFileSystem FileSystem implementation during FileSystem. 获取()

转载 作者:可可西里 更新时间:2023-11-01 15:00:19 30 4
gpt4 key购买 nike

请查找随附的代码片段。我正在使用此代码将文件从 hdfs 下载到我的本地文件系统 -

    Configuration conf = new Configuration();

FileSystem hdfsFileSystem = FileSystem.get(conf);

Path local = new Path(destinationPath);
Path hdfs = new Path(sourcePath);

String fileName = hdfs.getName();

if (hdfsFileSystem.exists(hdfs))
{
hdfsFileSystem.copyToLocalFile(false, hdfs, local, true);
logger.info("File " + fileName + " copied to local machine on location: " + destinationPath);
}
else
{
logger.error("File " + fileName + " does not exist on HDFS on location: " + sourcePath);
}

运行它会得到以下输出 -

Exception in thread "main" java.lang.UnsupportedOperationException: Not implemented by the DistributedFileSystem FileSystem implementation
at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:217)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2624)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2634)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
at com.flipkart.ads.importer.HdfsToLocalImporter.importFile(HdfsToLocalImporter.java:35)
at com.flipkart.ads.importer.HdfsToLocalImporter.main(HdfsToLocalImporter.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Process finished with exit code 1

我错过了什么?

最佳答案

已解决

没想到。这是一个 jar 不匹配问题。 Hadoop-code 和 Hadoop-commons 提供相同的 jar,我已经包含了这两个依赖项。

关于java.lang.UnsupportedOperationException : Not implemented by the DistributedFileSystem FileSystem implementation during FileSystem. 获取(),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/33015753/

30 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com