java - 使用 cxf web 服务上传大文件时出现内存不足错误

Question

我使用 cxf 和 JAX-WS 编写了一个在 Jetty 服务器上运行的 Web 服务。在客户端和服务器中启用 MTOM。我正在尝试上传一个大文件(>2GB)。我正在尝试上传一个大小为 2572848KB 的文件，并且从客户端应用程序我正在尝试将该文件上传到服务器。最多 2572847KB 的文件已成功上传。在客户端出现 java.lang.outofmemoryerror ，在服务器端也出现一些异常。我尝试重复这些步骤，但在上传的最后阶段遇到了同样的异常。

客户端异常:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:350)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:332)
    at org.apache.cxf.io.CachedOutputStream.writeCacheTo(CachedOutputStream.java:297)
    at org.apache.cxf.interceptor.AbstractLoggingInterceptor.writePayload(AbstractLoggingInterceptor.java:125)
    at org.apache.cxf.interceptor.LoggingOutInterceptor$LoggingCallback.onClose(LoggingOutInterceptor.java:142)
    at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:182)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:640)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:519)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:449)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:352)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:304)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:88)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:134)
    at com.sun.proxy.$Proxy34.uploadFile(Unknown Source)

服务器端异常:

Apr 26, 2014 5:08:50 PM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
WARNING: Interceptor for {http://testupload.java.com/}TestUploadServerImplService#{http://testupload.java.com/}uploadFile has thrown exception, unwinding now
org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:288)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:270)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:105)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:323)
    at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:319)
    at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:287)
    at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:939)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
    at org.eclipse.jetty.server.Server.handle(Server.java:346)
    at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:589)
    at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1065)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:915)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
    at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:411)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
    at java.lang.Thread.run(Unknown Source)
Caused by: com.ctc.wstx.exc.WstxIOException: null
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1402)
    at com.ctc.wstx.sw.BaseStreamWriter.writeEndDocument(BaseStreamWriter.java:532)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:282)
    ... 25 more
Caused by: org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:149)
    at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:96)
    at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:100)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:225)
    at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:198)
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1400)
    ... 27 more

在一些帖子中，他们提到服务器响应的大小很大。如有任何帮助，我们将不胜感激。

客户端代码:

JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
Map<String, Object> props = new HashMap<String, Object>();
props.put("mtom-enabled", Boolean.TRUE);
factory.setProperties(props);
long timeout = 10000L;

factory.setServiceClass(TestUploadServer.class);
factory.setAddress("http://localhost:8080/TestUploadServer/services/TestUploadServerImplPort");
// logging feature with size limit and output destination.
factory.getFeatures().add(new LoggingFeature(1024));
// factory.getFeatures().add(new LoggingFeature("<stdout>",
// "location"));
// factory.getInInterceptors().add(new LoggingInInterceptor());
// factory.getOutInterceptors().add(new LoggingOutInterceptor());
TestUploadServer uploadWebservice = (TestUploadServer) factory.create();

Client client = ClientProxy.getClient(uploadWebservice);
if (client != null) {
    HTTPConduit conduit = (HTTPConduit) client.getConduit();
    HTTPClientPolicy policy = new HTTPClientPolicy();
    policy.setConnectionTimeout(0);
    policy.setReceiveTimeout(0);
    conduit.setClient(policy);
}
FileUploader file = new FileUploader();

file.setName(FILENAME);
file.setFileType(FILETYPE);

DataSource source = new FileDataSource(new File(FILEPATH));
file.setDfile(new DataHandler(source));
uploadWebservice.uploadFile(file);
System.out.println("Finished Copying......");

Answer 1

Logginginterceptor 导致我的客户代码出现问题。一旦服务器回复附件，附件就会被放置在我的内存中，这是巨大的。因此，我遇到了内存不足错误。只需删除日志拦截器客户端的代码即可工作。尝试使用 12 GB 并工作正常...