java - 发生 org.hibernate.exception.GenericJDBCException : Could not open connection for long running process

Question

在我的应用程序中，我为一项工作使用了 20 个线程。每个线程调用远程 Web 服务并更新 Oracle 数据库。 Jboss 重启后，作业无法更新数据库，除了我在日志中发现的异常:

2017-11-08 23:36:20,706 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (EJB default - 21) javax.resource.ResourceException: IJ000460: Error checking for a transaction
2017-11-08 23:36:20,706 ERROR [org.jboss.as.ejb3] (EJB default - 16) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,707 ERROR [org.jboss.as.ejb3] (EJB default - 20) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,707 ERROR [org.jboss.as.ejb3] (EJB default - 19) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,710 ERROR [org.jboss.as.ejb3] (EJB default - 21) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,711 ERROR [org.jboss.as.ejb3.invocation] (EJB default - 16) JBAS014134: EJB Invocation failed on component PersonServiceDaoImpl for method public abstract long com.kaviletta.re.caltek.db.dao.PersonServiceDao.getNotUpdatedAddressCountBySubmissionId(int): javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:162) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:341) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:238) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at com.kaviletta.re.caltek.db.dao.PersonServiceDao$$$view6.getNotUpdatedAddressCountBySubmissionId(Unknown Source) [classes:]
    at com.kaviletta.re.caltek.rest.service.impl.AsynchronousServiceImpl.asyncAddressDetails(AsynchronousServiceImpl.java:61) [classes:]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55]
    at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55]
    at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:272) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:339) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:238) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AsyncFutureInterceptorFactory$1$1.runInvocation(AsyncFutureInterceptorFactory.java:89) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AsyncInvocationTask.run(AsyncInvocationTask.java:73) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_55]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:316) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.jboss.as.jpa.container.QueryNonTxInvocationDetacher.getSingleResult(QueryNonTxInvocationDetacher.java:69) [jboss-as-jpa-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]

线程用于数据库操作的代码:

    @PersistenceContext
EntityManager entityManager;
protected EntityManager getEntityManager() {
    return entityManager;
}

protected void setEntityManager(EntityManager manager) {
    this.entityManager = manager;
}
 public long getNotUpdatedAddressCountBySubmissionId(int submissionId) {
            Query q = getEntityManager().createQuery(
                    "select count(address.addressId.submissionId) from AddressStruc address where processState!=1  and address.addressId.submissionId = :submissionId");
            q.setParameter("submissionId", submissionId);
            return (long) q.getSingleResult();

        }

        public void updateStudentdAddress(List<AddressStruc> studnetsAddressDetails) {
            try {
                for (AddressStruc studentdAddress : studnetsAddressDetails) {
                    AddressStruc addressDetails = entityManager.find(AddressStruc.class, new AddressId(
                            studentAddress.getAddressId().getSubmissionId(), studentAddress.getAddressId().getRowId()));
                    if (addressDetails != null) {
                        entityManager.merge(updateStudentDetails(addressDetails, studentAddress));
                        entityManager.flush();
                    }

                }
            } catch (Exception e) {
                log.error("Exception occured while updating Address(method-->updateStudentAddress))-->" + e);
            }
        }

异步方法调用 updateStudentAddress 方法

public AsyncResult<StudentStatusModel> asyncAddressDetails(int startRow, int noOfRecord, int submissionId) {
    // method for fetching record based on the start row provided
    Collection<AddressStruc> addressList = studentEjb.getAddressBySubmissionId(startRow, noOfRecord, submissionId);
    StudentStatusModel status = new StudentStatusModel();
    status.setStartRow(startRow);
    status.setRetry(false);
    status.setMessage(null);
    // invoke the webservice and generate student for the given address
    if (StudentUtil.webServiceFailed <= 5) {
        OutputModel outputModel = invokeWebservice(addressList, status);

        if ((outputModel != null) && (outputModel.getStudentModels().size() > 0)) {
        // update the address details based on the submissionId+rowId
        studentEjb.updateStudentAddress(new ArrayList<>(convertModelToEntity(outputModel).values()));
    }
}
return new AsyncResult<StudentStatusModel>(status);
}

我正在使用容器管理的实体管理器进行数据库转换。

每个线程调用 updateStudentAddress 方法并更新 100 条记录。

由于这个作业是一个长时间运行的作业，一段时间后 jboss 无法与 oracle 建立进一步的连接。

有什么办法可以解决这个问题吗？

Answer 1

我觉得问题可能是里面的异常处理:asyncAddressDetails

在 RuntimeException 或 ApplicationException 导致事务回滚后，我们在 jboss7 中发生了此错误，当捕获到此异常时，应用程序代码随后在当前事务上下文完成之前对 EntityManager 进行了进一步操作。

例子:

void methodInNewTransaction() {
  try {
    dao.inRequiredTransaction();
  } catch (RuntimeException ex) {
    ; // ignore it
  }
  dao.secondCallInRequiredTransaction();
}

当忽略 RuntimeException 时，secondCallInRequiredTransaction 可能会导致错误。

如何解决这个问题，我看到了以下可能性:

• 防止异常发生
• 使用@ApplicationException-Annotation 来控制发生的异常的事务影响
• 重新抛出异常，以便不再使用外部事务上下文