python - "*** Error in ` python': free(): corrupted unsorted chunks: 0x0000000000ff2460 ***"in Python 的原因及解决方法

Question

我使用 Python 创建了一项服务，该服务使用 pymssql 连接到 SQL Azure，并且仅进行 SELECT 查询。 1 天(或更长时间)后，连接在查询中开始失败，最后服务以错误结束:

Error in `python': free(): corrupted unsorted chunks: 0x0000000000ff2460

我不确定是否只有一个错误或多个错误(也许第一个错误会引发更多错误)。

连接代码在这里:

  connectionDb = pymssql.connect(host=self.HOST_DATA_BASE, user=self.USER_DATA_BASE, password=self.PASSWORD_DATA_BASE, database=self.DATA_BASE_NAME)    

我以下面的形式执行查询:

  cursor=connectionDb.cursor()
  cursor.execute("select * from vehicles") 
  rows = cursor.fetchall()

  if (rows!=None):
     return rows

最初，连接工作正常。一段时间不活动后会出现此问题。

我尝试简化查询，但我不认为这是错误的原因。

*也许pymssql中可能存在错误？

Answer 1

好的，我能够验证这是 p​​ymssql 中的内存问题。为此有一张公开的票。

我使用Python垃圾收集库(gc)来打印Python尚未释放多少内存。

下面是垃圾收集输出。正如您所看到的，“集合”继续增加，这意味着 pymssql 没有释放内存，即使它不再使用。

garbage collector output is [{'collections': 1822693, 'collected': 54744, 'uncollectable': 0}, {'collections': 165699, 'collected': 863905, 'uncollectable': 0}, {'collections': 4173, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1823867, 'collected': 54744, 'uncollectable': 0}, {'collections': 165806, 'collected': 863905, 'uncollectable': 0}, {'collections': 4176, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1825043, 'collected': 54744, 'uncollectable': 0}, {'collections': 165912, 'collected': 863905, 'uncollectable': 0}, {'collections': 4178, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1826218, 'collected': 54744, 'uncollectable': 0}, {'collections': 166019, 'collected': 863905, 'uncollectable': 0}, {'collections': 4180, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1827393, 'collected': 54744, 'uncollectable': 0}, {'collections': 166126, 'collected': 863905, 'uncollectable': 0}, {'collections': 4182, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1828567, 'collected': 54744, 'uncollectable': 0}, {'collections': 166233, 'collected': 863905, 'uncollectable': 0}, {'collections': 4185, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1829741, 'collected': 54744, 'uncollectable': 0}, {'collections': 166340, 'collected': 863905, 'uncollectable': 0}, {'collections': 4188, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1830917, 'collected': 54744, 'uncollectable': 0}, {'collections': 166446, 'collected': 863905, 'uncollectable': 0}, {'collections': 4190, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1832092, 'collected': 54744, 'uncollectable': 0}, {'collections': 166553, 'collected': 863905, 'uncollectable': 0}, {'collections': 4192, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1833267, 'collected': 54744, 'uncollectable': 0}, {'collections': 166660, 'collected': 863905, 'uncollectable': 0}, {'collections': 4194, 'collected': 359393, 'uncollectable': 0}]
garbage collector output is [{'collections': 1834441, 'collected': 54744, 'uncollectable': 0}, {'collections': 166767, 'collected': 863905, 'uncollectable': 0}, {'collections': 4197, 'collected': 359393, 'uncollectable': 0}]


free(): corrupted unsorted chunks

当我切换到 pyodbc 时，我运行了相同的代码并得到了以下结果:

garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]
garbage collector output is [{'collections': 218, 'collected': 806, 'uncollectable': 0}, {'collections': 19, 'collected': 115, 'uncollectable': 0}, {'collections': 1, 'collected': 5, 'uncollectable': 0}]

正如您所看到的，“集合”元素保持不变。这证明pymssql存在内存泄漏问题。我的建议是使用 pyodbc、pytds 或 ctds。我在github ticket上对此发表了观点。