作者热门文章
- android - RelativeLayout 背景可绘制重叠内容
- android - 如何链接 cpufeatures lib 以获取 native android 库?
- java - OnItemClickListener 不起作用,但 OnLongItemClickListener 在自定义 ListView 中起作用
- java - Android 文件转字符串
我打算通过 Java 从 HDFS 获取文件
public static void main(String[] args) throws IOException, URISyntaxException
{ //1. Get the instance of Configuration
Configuration configuration = new Configuration();
File workaround = new File(".");
System.getProperties().put("hadoop.home.dir", workaround.getAbsolutePath());
new File("./bin").mkdirs();
new File("./bin/winutils.exe").createNewFile();
configuration.set("fs.hdfs.impl",
org.apache.hadoop.hdfs.DistributedFileSystem.class.getName()
);
configuration.set("fs.file.impl",
org.apache.hadoop.fs.LocalFileSystem.class.getName()
);
//2. URI of the file to be read
URI uri = new URI("hdfs://10.1.7.24:8020/user/hue/sparkinput/tripdata.csv");
//3. Get the instance of the HDFS
FileSystem hdfs = FileSystem.get(uri, configuration);
//4. A reference to hold the InputStream
InputStream inputStream = null;
String pathOfFile=uri.toString();
try{
//5. Prepare the Path, i.e similar to File class in Java, Path represents file in HDFS
Path path = new Path(pathOfFile);
//6. Open a Input Stream to read the data from HDFS
inputStream = hdfs.open(path);
//7. Use the IOUtils to flush the data from the file to console
IOUtils.copyBytes(inputStream, System.out, 4096, false);
}finally{
//8. Close the InputStream once the data is read
IOUtils.closeStream(inputStream);
}
/* Testing testing = new Testing();
testing.getMongoTables("sales");*/
这是异常(exception)
log4j:ERROR Could not find value for key log4j.appender.FILE
log4j:ERROR Could not instantiate appender named "FILE".
2018-01-11 16:35:11 WARN NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/crypto/key/KeyProviderenter code here
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2397)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
at Testing.main(Testing.java:172)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.crypto.key.KeyProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
最佳答案
您的 java 程序无法找到 hadoop 库,更正 hadoop jar 的路径将解决您的问题。
这个警告显示了你的问题
NativeCodeLoader - Unable to load native-hadoop library for your platform
关于java - Hadoop NoClassDef hadoop/crypto/key/KeyProvider,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/48206228/
我在 HDFS 的加密区域中有数据。我可以使用 hive 用户读取数据,但是当我创建一个 hive 表并尝试通过 beeline 查询它时,我得到了这个异常: Error: java.io.IOExc
我打算通过 Java 从 HDFS 获取文件 public static void main(String[] args) throws IOException, URISyntaxException
我是一名优秀的程序员,十分优秀!