java - 如何在Java中从S3读取Snappy压缩文件

Question

当前，我们正在Hadoop中运行MapReduce作业，其中输出被压缩到SnappyCompression中。然后，我们将输出文件移至S3。现在，我想通过Java从S3读取压缩文件。

Answer 1

我找到了从S3读取快速压缩文件的答案。首先，您应该从S3获取对象内容。然后解压缩文件。

    S3Object s3object = s3Client.getObject(new GetObjectRequest(bucketName,Path));
    InputStream inContent = s3object.getObjectContent();
    CompressionCodec codec = (CompressionCodec) ReflectionUtils.newInstance(SnappyCodec.class, new Configuration());
    InputStream inStream = codec.createInputStream(new BufferedInputStream(inContent));
    InputStreamReader  inRead = new InputStreamReader(inStream);
    BufferedReader br = new BufferedReader(inRead);
    String line=null;
    while ((line = br.readLine()) != null){
        system.out.println(line);
    }