java - 如何跟踪和防止分进程运行的c3po出现死锁？

Question

我有一个非常简单的计算，它生成字母矩阵，可能找到矩阵中的所有单词。单词中的字母是相邻的单元格。

   for (int i = 0; i < 500; i++) {
        System.out.println(i);
        Matrix matrix = new Matrix(4);
        matrix.scanWordsRandomly(9);
        matrix.printMatrix();
        System.out.println(matrix.getSollSize());
        matrix.write_to_db();
    }

这是持久化代码。

public void write_to_db() {
    Session session = null;
    try {
        session = HibernateUtil.getSessionFactory().openSession();
        session.beginTransaction();
        Matrixtr onematrixtr = new Matrixtr();
        onematrixtr.setDimension(dimension);
        onematrixtr.setMatrixstr(this.toString());
        onematrixtr.setSolsize(getSollSize());
        session.save(onematrixtr);
        for (Map.Entry<Kelimetr, List<Cell>> sollution : sollutions.entrySet()) {
            Kelimetr kelimetr = sollution.getKey();
            List<Cell> solpath = sollution.getValue();
            Solstr onesol = new Solstr();
            onesol.setKelimetr(kelimetr);
            onesol.setMatrixtr(onematrixtr);
            onesol.setSoltext(solpath.toString().replace("[", "").replace("]", "").replace("true", "").replace("false", ""));
            session.save(onesol);
        }

        session.getTransaction().commit();
        session.close();

    }
    catch (HibernateException he) {
        System.out.println("DB Error : " + he.getMessage());
        session.close();
    } 
    catch (Exception ex) {
        System.out.println("General Error : " + ex.getMessage());
    }
}

这是 hibernate 配置文件。

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
  <session-factory>
    <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
    <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
    <property name="hibernate.connection.url">jdbc:mysql://localhost:3306/kelimegame_db_dev?autoReconnect=true&amp;useUnicode=true&amp;characterEncoding=UTF-8</property>
    <property name="hibernate.connection.username">root</property>
    <property name="hibernate.connection.password">!.Wlu9RrCA</property>
    <property name="hibernate.show_sql">false</property>
    <property name="hibernate.query.factory_class">org.hibernate.hql.classic.ClassicQueryTranslatorFactory</property>
    <property name="hibernate.format_sql">false</property>
    <!-- Use the C3P0 connection pool provider -->
    <property name="hibernate.c3p0.acquire_increment">50</property>
    <property name="hibernate.c3p0.min_size">10</property>
    <property name="hibernate.c3p0.max_size">100</property>
    <property name="hibernate.c3p0.timeout">300</property>
    <property name="hibernate.c3p0.max_statements">5</property>
    <property name="hibernate.c3p0.idle_test_period">3000</property>
    <mapping resource="kelimegame/entity/Progress.hbm.xml"/>
    <mapping resource="kelimegame/entity/Solstr.hbm.xml"/>
    <mapping resource="kelimegame/entity/Kelimetr.hbm.xml"/>
    <mapping resource="kelimegame/entity/User.hbm.xml"/>
    <mapping resource="kelimegame/entity/Achievement.hbm.xml"/>
    <mapping resource="kelimegame/entity/Matrixtr.hbm.xml"/>
  </session-factory>
</hibernate-configuration>

在找到所有可能的解决方案后，我使用 hibernate 保存矩阵和解决方案。我也在使用 c3pO 库。我没有生成任何线程。所有工作都以非常简单的迭代方式完成。但是我在不同的进程中运行 jar。我正在从不同的终端执行此操作:

java -jar NewDB.jar

我遇到了如下死锁:

Apr 25, 2013 8:38:05 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
WARNING: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7f0c09f9 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Apr 25, 2013 9:08:23 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
WARNING: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7f0c09f9 -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@2933f261
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#1
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@116dd369
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#0
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@41529b6f
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#2
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@165ab5ea
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@1d5d211d
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@4d2905fa
Pool thread stack traces:
    Thread[C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#1,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:662)
    Thread[C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#0,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:662)
    Thread[C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#2,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:662)


Apr 25, 2013 9:41:29 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
WARNING: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7f0c09f9 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
Apr 25, 2013 9:55:18 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
WARNING: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7f0c09f9 -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5a337b7d
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#0
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@69f079ce
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#1
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@2accf9b8
            on thread: C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#2
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@771eb4fb
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@fc07d6
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@2266731b
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@740f0341
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@59edbee
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@78e924
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@2123aba
        com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7acd8a65
Pool thread stack traces:
    Thread[C3P0PooledConnectionPoolManager[identityToken->z8kfsx8uibeyqevbbapc|4045cf35]-HelperThread-#0,5,main]
        java.text.NumberFormat.getInstance(NumberFormat.java:769)
        java.text.NumberFormat.getInstance(NumberFormat.java:393)
        java.text.MessageFormat.subformat(MessageFormat.java:1262)
        java.text.MessageFormat.format(MessageFormat.java:860)
        java.text.Format.format(Format.java:157)
        java.text.MessageFormat.format(MessageFormat.java:836)
        com.mysql.jdbc.Messages.getString(Messages.java:106)
        com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2552)
        com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
        com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
        com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
        com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:943)
        com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4113)
        com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1308)
        com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2336)
        com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2176)
        com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2158)
        com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
        com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
        sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
        com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
        com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:183)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:172)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:188)Killed
caglar@ubuntu:~/NetBeansProjects/NewDB/dist$ 

我的问题如下:

1. c3po 中的这个死锁会发生吗，因为我在单独的过程？
2. 我应该在其中使用一个进程和多个线程吗？过程？
3. 我如何追踪这个死锁并了解它的原因？有没有办法跟踪导致死锁的多个 JVM？

Answer 1

这很有趣。

您已经发布了两个明显的死锁。第一个是由 c3p0 尝试关闭()连接引起的，并且这些关闭()操作既没有成功也没有及时失败并出现异常。第二个 APPARENT DEADLOCK 显示了连接获取的问题:c3p0 正在尝试获取新的连接，并且这些尝试既没有成功也没有及时失败并出现异常。非常不同的操作被卡住的事实表明，您的 dbms 在您正在做的事情或类似事情的压力下锁定可能是一个更普遍的问题。对您的数据库运行多个进程应该没有问题，但您需要了解限制。

关于您的配置有一些有趣的事情:

1) hibernate.c3p0.max_statements=5 是一个非常糟糕的主意，几乎适用于任何池，尤其是这么大的池。您最多有 100 个连接，并且您只允许在所有连接之间缓存总共 5 个语句。这可能会给池和 DBMS 带来压力，因为您将不断地翻阅 PreparedStatements 并且语句缓存会为此做很多簿记工作。您可能意味着 每个连接 有 5 个缓存语句，但这不是您配置的内容。您已经为您的池设置了全局最大值。也许试试 hibernate.c3p0.maxStatementsPerConnection=5 代替？或者将 max_statements 设置为零以关闭语句缓存，至少在解决死锁之前。见http://www.mchange.com/projects/c3p0/#configuring_statement_pooling

2) 如果您在多个进程而不是多个线程中运行计算，您真的需要每个进程保持 50 - 100 个连接吗？事情很可能会卡住，因为您正在给 dbms 施加太多未完成的连接，因为您的多个进程中的每一个都获得了大量资源密集型连接。在任何进程中，您不需要比在该进程中同时运行的客户端线程更多的连接。我会将 hibernate.c3p0.acquire_increment 和可能的 hibernate.c3p0.max_size 设置为更小的值。

3) 如果您确实需要所有这些连接同时运行，您可以通过将配置参数 numHelperThreads 增加到大于其默认值 3 的某个值来减少池死锁的漏洞。您可能希望 numHelperThreads 类似于机器上可用内核数的两倍。考虑到你正在运行多个进程，你可能会发现你的 CPU 已经饱和，这会导致卡住。所以要注意这一点。

基本上，尝试更新您的配置，以便尽可能高效地使用资源——文件句柄、网络连接、CPU，这样您就不会不必要地对池/语句缓存/dbms 施加超出您需要的压力是。

如果这些建议不能解决问题，请发布您的池的填充配置。 c3p0 在池初始化时将其配置转储到 INFO 级别。

祝你好运!