postgresql - 使用 python psycopg2 插入行时，docker postgres 进程终止

Question

数据库是docker中的postgresql-9.5.1。我的主机有 3.75 GB 内存，linux。在某些方法中，我使用带有以下代码的 psycopg2 一个接一个地插入 490000 行。

student_list = [(name, surname, explanation)]
args_str = ','.join(cur.mogrify("(%s,%s,%s)", x) for x in student_list)
cur.execute('INSERT INTO students (name, surname, explanation) VALUES ' + args_str)

这使我的数据库 docker 内存似乎已满并出现以下错误:

LOG: server process (PID 11219) was terminated by signal 9: Killed
DETAIL: Failed process was running
LOG: terminating any other active server processes
docker@test_db WARNING: terminating connection because of crash of another server process
docker@test_db DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
docker@test_db HINT: In a moment you should be able to reconnect to the database and repeat your command.
docker@test_db WARNING: terminating connection because of crash of another server process
docker@test_db DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
... docker@test_db FATAL: the database system is in recovery mode
LOG: all server processes terminated; reinitializing
LOG: database system was interrupted; last known up at 2017-06-06 09:39:40 UTC
LOG: database system was not properly shut down; automatic recovery in progress
docker@test_db FATAL: the database system is in recovery mode
docker@test_db FATAL: the database system is in recovery mode
docker@test_db FATAL: the database system is in recovery mode
LOG: autovacuum launcher started

脚本给出该日志:

Inner exception
SSL SYSCALL error: EOF detected

我尝试在连续查询之间设置一些休眠时间，但得到了相同的结果。有什么限制吗？我也尝试为每个查询连接和断开连接，但得到了相同的结果。这些是我的连接和断开连接方法。

def connect():
    conn = psycopg2.connect(database=database_name,
                            user=database_user,
                            host=database_host,
                            port=database_port)
    conn
  .set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
    cur = conn.cursor()
    return conn, cur

def disconnect(conn, cur):
    cur.close()
    conn.close()

Answer 1

这是我做的。其实我的内存已经够满了。这就是为什么 linux 操作系统过去常常终止 Postgresql 中的进程。每个插入过程中都有 1M 个值。诀窍是我将数据列表分成 block 并尝试 100k x 100k。这很好用。感谢您的帮助。