java.lang.OutOfMemory错误: Java heap space - blobstoreService

Question

我正在尝试从存储在 Google Cloud Storage 上的文件中读取字节并将其通过 HTTP POST 请求发送，但我在使用较大的文件时出现此异常，代码正在运行小文件:

此行抛出异常:

writer.write(blobstoreService.fetchData(new BlobKey(video.getBlobkey()), start, end));

这是我的代码:

    URLConnection connection = new URL("http://myurl.com/").openConnection();
    //set time out to infinite
    connection.setConnectTimeout(0);
    connection.setDoOutput(true);
    connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
    BufferedOutputStream writer = null;
    OutputStream output = connection.getOutputStream();
    writer = new BufferedOutputStream(output); // true = autoFlush, important!

    //........

    BlobInfoFactory blobInfoFactory = new BlobInfoFactory();
    BlobInfo blobInfo = blobInfoFactory.loadBlobInfo(new BlobKey(video.getBlobkey()));
    Long blobSize = blobInfo.getSize();
    //max read on fetch
    long maxReadFetch = 1015807;
    //read the file in one time temporary
    long i = 0;
    long start = 0;
    long end = 0;
    while(i < blobSize) {
        start = i;
        end = i + maxReadFetch;
        //determine end
        if(end > blobSize) {
            end = blobSize;
        } else {
            end--;
        }
        writer.write(blobstoreService.fetchData(new BlobKey(video.getBlobkey()), start, end));

        i += maxReadFetch;
    }
    writer.flush(); // Important! Output cannot be closed. Close of writer will close output as well.
} finally {
    if (input != null) try { input.close(); } catch (IOException logOrIgnore) {}
}

堆栈跟踪:

Uncaught exception from servlet java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2961) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:111) at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection$BufferingOutputStream.write(URLFetchServiceStreamHandler.java:460) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at java.io.FilterOutputStream.write(FilterOutputStream.java:97) at controller.CtrlWistia.upload(CtrlWistia.java:200) at controller.CtrlWistia.add(CtrlWistia.java:126) at controller.CtrlWistia.ajax(CtrlWistia.java:79) at controller.CtrlAjax.main(CtrlAjax.java:66) at vidaao.AjaxServlet.doPost(AjaxServlet.java:37) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176) at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145) at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92) at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)

Answer 1

请注意，即使您一次读取的数据少于 1MB，您也会将该数据写入 OutputStream，该数据将保留在内存中，直到您提交请求。

根据docs :

App Engine's implementation of URLConnection does not maintain a persistent connection with the remote host. When the app sets request data or writes to the output stream, the request data is kept in memory. When the app accesses any data about the response, such as getting the input stream (or calling the connect() method), App Engine calls the URL Fetch service with the request data, gets the response, closes the connection and returns the response data.