java - Spring ，Java : Streaming file download for avoiding out of memory errors

Question

我正在开发一个 Spring-MVC 应用程序，用户可以在其中下载文件。用户可以点击触发下载机制的附件。

昨天，当多次下载并且其中两次有大约 2 GB 的文件时，它导致内存不足错误(日志如下)。

为了避免这个问题，解决这个问题的一种方法似乎是将下载数据分块流式传输，并且只在服务层处理这些 block ，而不是整个文件。

不幸的是，我不知道如何推进这个，任何帮助都会很好。如果此选项无法运行，请提供有关如何解决此问题的任何建议。

错误日志:

HTTP Status 500 - Handler processing failed; nested exception is java.lang.OutOfMemoryError: Direct buffer memory

type Exception report

message Handler processing failed; nested exception is java.lang.OutOfMemoryError: Direct buffer memory

description The server encountered an internal error that prevented it from fulfilling this request.

exception

org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Direct buffer memory
    org.springframework.web.servlet.DispatcherServlet.triggerAfterCompletionWithError(DispatcherServlet.java:1303)
    org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:977)
    org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
    org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967)
    org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
    org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843)

Controller 代码:

    @RequestMapping(value = "/download/attachment/{attachid}", method = RequestMethod.GET)
        public void getAttachmentFromDatabase(@PathVariable("attachid") int attachid,
    , HttpServletResponse response,) {

response.setContentType("application/octet-stream");
GroupAttachments groupAttachments = this.groupAttachmentsService.getAttachmenById(attachid);
response.setHeader("Content-Disposition", "attachment; filename=\"" + groupAttachments.getFileName() + "\"");
                            response.setContentLength(groupAttachments.getSendAttachment().length);
                            FileCopyUtils.copy(groupAttachments.getSendAttachment(), response.getOutputStream());
    response.flushBuffer();

    }

服务层:

@Override
    public GroupAttachments getAttachmenById(int attachId) {
        Person person = this.personService.getCurrentlyAuthenticatedUser();
        GroupAttachments groupAttachments = this.groupAttachmentsDAO.getAttachmenById(attachId);

        GroupMembers groupMembers = this.groupMembersService.returnMembersMatchingUsernameAccountId(person.getUsername(),
                groupAttachments.getGroupId());
        if (!(groupMembers == null)) {
            if (person.getUsername().equals(groupMembers.getMemberUsername())) {
                try {
                    Path path = Paths.get(msg + groupAttachments.getGroupId() + "/" +
                            groupAttachments.getFileIdentifier());
                    groupAttachments.setSendAttachment(Files.readAllBytes(path));
                    return groupAttachments;
                } catch (IOException ignored) {
                    this.groupAttachmentsDAO.removeAttachment(attachId);
                    return null;
                }
            }
            return null;
        } else {
            return null;
        }
    }

谢谢。 :-)

更新

新的下载机制:

Controller :

 public ResponseEntity<byte[]> getAttachmentFromDatabase(@PathVariable("attachid") int attachid,
                                                    @PathVariable("groupaccountid") Long groupAccountId, @PathVariable("api") String api,
                                                    HttpServletResponse response,
                                                    @PathVariable("type") boolean type) {

 Path path = this.groupAttachmentsService.getAttachmentPathById(attachid);

        GroupAttachments groupAttachments = this.groupAttachmentsService.getAttachmentObjectOnlyById(attachid);
                        response.setContentType("application/octet-stream");
                        response.setHeader("Content-Disposition", "attachment; filename=\""+groupAttachments.getFileName()+"\"");
  try {
OutputStream outputStream = response.getOutputStream();

Files.copy(path,outputStream);
outputStream.flush();
outputStream.close();
response.flushBuffer();
}

服务层:

@Override
    public Path getAttachmentPathById(int attachId){
        Person person = this.personService.getCurrentlyAuthenticatedUser();
        GroupAttachments groupAttachments = this.groupAttachmentsDAO.getAttachmenById(attachId);

        GroupMembers groupMembers = this.groupMembersService.returnMembersMatchingUsernameAccountId(person.getUsername(),
                groupAttachments.getGroupId());
        if (!(groupMembers == null)) {
            if (person.getUsername().equals(groupMembers.getMemberUsername())) {
                try {
                    return Paths.get(msg + groupAttachments.getGroupId() + "/" +
                            groupAttachments.getFileIdentifier());
                } catch (Exception ignored) {
                    return null;
                }
            }
            return null;
        } else {
            return null;
        }
    }

Answer 1

首先停止在您的服务中加载全部内容，因为您正在将所有文件内容加载到内存中。

创建一个为 GroupAttachments 构造 Path 的方法，我会在 GroupAttachments 上自行创建它。

public class GroupAttachments {

    public Path getPath() {
        return Paths.get(msg + getGroupId() + "/" + getFileIdentifier());
    }
}

然后在你的 Controller 中简单地做

@RequestMapping(value = "/download/attachment/{attachid}", method = RequestMethod.GET)
public void getAttachmentFromDatabase(@PathVariable("attachid") int attachid, HttpServletResponse response) {

  response.setContentType("application/octet-stream");
  GroupAttachments groupAttachments = this.groupAttachmentsService.getAttachmenById(attachid);
  Path path = groupAttachmetns.getPath(); // calculates the java.nio.file.Path  
  response.setHeader("Content-Disposition", "attachment; filename=\"" + path.getFileName() + "\"");
  response.setContentLength(Files.size(path);
  Files.copy(path, response.getOutputStream());
  response.flushBuffer();

}

恕我直言，没必要让它变得更复杂。