java - Hadoop Library导入但无法在File System中设置 "get"方法

Question

我正在尝试设置对 HDFS 的调用以使用文件系统从中获取文件:
FileSystem fs = new FileSystem.get(new URI.create(uri), conf);

但我还是遇到了这两个错误，所以我调用了所有的库

    Error:(46, 39) java: cannot find symbol
    symbol:   class get
    location: class org.apache.hadoop.fs.FileSystem

    Error:(46, 47) java: cannot find symbol
    symbol:   method create(java.lang.String)
    location: class org.apache.commons.httpclient.URI

这是针对 hadoop 2.7.7 版本的。我已经尝试导入孔库org.apache.hadoop.fs.*;但是当我尝试在行中使用它们时，方法 get 和 create 仍然不存在，如下所示:FileSystem fs = new FileSystem.get(URI.create(usi), conf)

import org.apache.commons.httpclient.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;   

public class Test { 
  public static void main(String []args) {

    String uri = "hdfs://localhost:9000/data.json";
    Configuration conf = new Configuration();

    //cannot find symbol get
    //Cannot find symbol create
    FileSystem fs = new FileSystem.get(new URI.create(uri), conf);

    InputStream in = null;

    try {
        in = fs.open(new Path(uri));
        IOUtils.copyBytes(in, System.out, 4096, false);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        IOUtils.closeStream(in);
    }
  }
}

这是我的 POM.xml 文件:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.boeing.bedl</groupId>
<artifactId>IngestionData</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>2.7.7</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-yarn-common</artifactId>
        <version>2.7.7</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-mapreduce-client-core</artifactId>
        <version>2.7.7</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-mapreduce-client-common</artifactId>
        <version>2.7.7</version>
    </dependency>

   </dependencies>
</project>

Answer 1

我会说问题是您导入了 import org.apache.commons.httpclient.URI; 而不是 java.net.URI

这就是为什么你会因为缺少 .create 方法而出错，它是 java.net.URI 的一部分，而你会在 .get 方法上出错，因为你传递了URI 类型错误。

(我猜是 IDE 的一些自动导入？:))