java - Google App Engine - 使用 BlobstoreInputStream 从 Blobstore 逐行读取

Question

我真的希望你能帮助我!

目标是能够通过我的(java - google app engine)应用程序上传数据文件(.tsv)并获取其内容并将其写入 mysql 数据库。

经过一番研究，blobstore 似乎是最佳选择，因为文件大小约为 70-100MB。

所以几天前我开始使用这个示例项目来帮助我开始 https://github.com/crhym3/java-blobstore-gcs-sample .

我的项目启动并运行没有问题。由于我想访问内容而不是下载文件(这就是示例项目正在执行的操作)，因此我尝试按照多个来源中的建议使用 BlobstoreInputStream 来执行此操作。因此，我从示例项目中更改的唯一文件是 ServeBlobServlet。我没有提供整个 blob，而是尝试通过 BlobstoreInputStream 获取第一行。

public class ServeBlobServlet extends HttpServlet {

BlobstoreService blobstore = BlobstoreServiceFactory.getBlobstoreService();

private static final Logger log = 
        Logger.getLogger(ServeBlobServlet.class.getName());

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
        throws ServletException, IOException {

    String[] parts = req.getRequestURI().split("/");
    String gsObjectName = URLDecoder.decode(parts[parts.length - 1], "UTF-8");

    log.info("Serving GCS object: " + gsObjectName);
    BlobKey blobKey = blobstore.createGsBlobKey(gsObjectName);

    //blobstore.serve(blobKey, resp);
    System.out.println(getFirstLine(blobKey));
}

private String getFirstLine(BlobKey blobkey) throws IOException {
    BufferedReader reader = new BufferedReader(new InputStreamReader(new BlobstoreInputStream(blobkey)));
    String line = reader.readLine();
    reader.close();
    return line;
}

正如你所看到的。实际上，我只是取消注释了 blobstore.serve 行并添加了标准输出行(并编写了 getFirstLine 方法)。

尽管我使用与 blobstore.serve 相同的 BlobKey(工作正常)，但我收到此错误。

Uncaught exception from servlet
com.google.appengine.api.blobstore.BlobstoreInputStream$BlobstoreIOException: BlobstoreInputStream received an invalid blob key: AMIfv97t_VDPdp2Qg7UXkE3oc6nRe7yrr86jo6th4-cCU2w8HnjOF5iSOYyAlsrQDkWVLsOMYMdHlbl-AE14swvD5n03N4XStmumVlNeNtNANXI8OWNpodBmJrBB4QS7Ru_AXK5rYvV4-KizTi6kPznYvGQ6gCd1oizh0xtRwO0jkuC-K4l648XpGhZF65H-eb1JJtxWDEsMlQ9LbmRzEPJZHV3tfZ5YRLFnlzOEGnWZxj92BpORnBq8Ly_PMTA-xSNDuFsMBAJwYzdDwG3WVwDpnkxxkqvTLfTRxXKm1WgMZEtlP-0w4NbVwuJ8QNaTAzsDLCs14PFFvWV4pVMd9coXvF3KmC-Z6T4S8yC0nGK9ST0x8o0-CNI
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:121)
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:92)
    at com.google.appengine.api.blobstore.BlobstoreInputStream.<init>(BlobstoreInputStream.java:104)
    at com.moviemap.server.ServeBlobServlet.getFirstLine(ServeBlobServlet.java:43)
    at com.moviemap.server.ServeBlobServlet.doGet(ServeBlobServlet.java:39)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:37)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:260)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:78)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:148)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:469)
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
    at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:234)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:235)
    at java.lang.Thread.run(Thread.java:745)

我尝试了几种不同的方法和 BlobKey 构造函数，但没有任何效果。同样的错误不断出现。

提前感谢任何花时间提供帮助的人!欣赏它!

Answer 1

答案在这里

konqi 对上述问题的评论绝对正确。我开始的项目(问题中的链接)不适合直接上传和读取 blob。正如 konqi 所说，我所做的(项目所做的)是将 blob 上传到谷歌云存储。

所以答案是:如果您想将 blob 上传到 google blobstore 并直接读取它，您应该按照此处的教程 https://cloud.google.com/appengine/docs/java/blobstore/#Java_Using_the_Blobstore 进行操作但您可以这样阅读它，而不是提供 blob(请参阅教程中的段落):

    String keyString = req.getParameter("blob-key");
    BlobKey blobkey = new BlobKey(keyString);
    BufferedReader reader = new BufferedReader(new InputStreamReader(new BlobstoreInputStream(blobkey)));
    String line = null;
    while((line = reader.readLine()) != null) {
        //Do whatever you like with the line
    }
    reader.close();

希望这对某人有帮助