postgresql - 在 Docker Compose 多容器应用程序中，如何防止 Postgres 运行先前发出的语句

Question

我正在运行 Scrapy带有将数据写入 PostgreSQL 数据库的项目管道的项目。这是一个Docker Compose具有以下 docker-compose.yml 的多容器应用程序:

version: '3'

services:
  privoxy:
    build: ./privoxy
    links:
      - tor

  tor:
    build:
      context: ./tor
      args:
        password: "foo"

  scraper:
    build: ./scraper
    environment:
      - http_proxy=http://privoxy:8118
      - PGPASSWORD=iperpassword
    links:
      - tor
      - privoxy
      - db
    volumes:
      - scraper-data:/scraper/apkmirror_scraper/data
      - scraper-jobdir:/scraper/apkmirror_scraper/crawls

  db:
    image: postgres
    environment:
      - POSTGRES_PASSWORD=iperpassword
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  scraper-data:
  scraper-jobdir:
  db-data:

我正在使用来自官方存储库 ( https://hub.docker.com/_/postgres/ ) 的 postgres 图像，并将 /var/lib/postgresql/data 目录制作成一个卷以下 https://docs.docker.com/compose/compose-file/#volume-configuration-reference .

问题是，当我启动应用程序时，db 服务输出的日志和错误与我之前尝试过的有关，而不是我现在尝试做的。这是我看到的:

apkmirrorscrapercompose_tor_1 is up-to-date
apkmirrorscrapercompose_db_1 is up-to-date
apkmirrorscrapercompose_privoxy_1 is up-to-date
Recreating apkmirrorscrapercompose_scraper_1
Attaching to apkmirrorscrapercompose_tor_1, apkmirrorscrapercompose_db_1, apkmirrorscrapercompose_privoxy_1, apkmirrorscrapercompose_scraper_1
scraper_1  |                                  List of databases
scraper_1  |    Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
scraper_1  | -----------+----------+----------+------------+------------+-----------------------
scraper_1  |  postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
scraper_1  |  template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
scraper_1  |            |          |          |            |            | postgres=CTc/postgres
scraper_1  |  template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
scraper_1  |            |          |          |            |            | postgres=CTc/postgres
scraper_1  | (3 rows)
scraper_1  | 
scraper_1  | Postgres is up - executing command
scraper_1  | Tor appears to be working. Proceeding with command...
scraper_1  | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: apkmirror_scraper)
scraper_1  | 2017-06-30 12:32:59 [scrapy.utils.log] INFO: Overridden settings: {'BOT_NAME': 'apkmirror_scraper', 'NEWSPIDER_MODULE': 'apkmirror_scraper.spiders', 'SPIDER_MODULES': ['apkmirror_scraper.spiders']}
scraper_1  | 2017-06-30 12:32:59 [scrapy.middleware] INFO: Enabled extensions:
scraper_1  | ['scrapy.extensions.corestats.CoreStats',
scraper_1  |  'scrapy.extensions.telnet.TelnetConsole',
scraper_1  |  'scrapy.extensions.memusage.MemoryUsage',
scraper_1  |  'scrapy.extensions.closespider.CloseSpider',
scraper_1  |  'scrapy.extensions.logstats.LogStats']
db_1       | The files belonging to this database system will be owned by user "postgres".
db_1       | This user must also own the server process.
db_1       | 
db_1       | The database cluster will be initialized with locale "en_US.utf8".
db_1       | The default database encoding has accordingly been set to "UTF8".
db_1       | The default text search configuration will be set to "english".
db_1       | 
db_1       | Data page checksums are disabled.
db_1       | 
db_1       | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1       | creating subdirectories ... ok
db_1       | selecting default max_connections ... 100
db_1       | selecting default shared_buffers ... 128MB
db_1       | selecting dynamic shared memory implementation ... posix
db_1       | creating configuration files ... ok
db_1       | running bootstrap script ... ok
db_1       | performing post-bootstrap initialization ... ok
db_1       | 
db_1       | WARNING: enabling "trust" authentication for local connections
db_1       | You can change this by editing pg_hba.conf or using the option -A, or
db_1       | --auth-local and --auth-host, the next time you run initdb.
db_1       | syncing data to disk ... ok
db_1       | 
db_1       | Success. You can now start the database server using:
db_1       | 
db_1       |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1       | 
db_1       | waiting for server to start....LOG:  database system was shut down at 2017-06-29 13:40:40 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       |  done
db_1       | server started
db_1       | ALTER ROLE
db_1       | 
db_1       | 
db_1       | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1       | 
db_1       | LOG:  received fast shutdown request
db_1       | waiting for server to shut down....LOG:  aborting any active transactions
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       |  done
db_1       | server stopped
db_1       | 
db_1       | PostgreSQL init process complete; ready for start up.
db_1       | 
db_1       | LOG:  database system was shut down at 2017-06-29 13:40:41 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:41:53 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:43:22 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 13:52:02 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled
db_1       | LOG:  database system is ready to accept connections
db_1       | LOG:  autovacuum launcher started
db_1       | ERROR:  column "developer" of relation "apks" does not exist at character 31
db_1       | STATEMENT:  INSERT INTO apks (url, title, developer, app, version_name, version_code, architectures, package, apk_file_size, android_min_version, android_target_version, supported_dpis, md5_signature, time_uploaded, time_scraped, image_urls, images, file_urls, files, created_on, updated_on) VALUES ('http://www.apkmirror.com/apk/sony-mobile-communications/backup-restore-unknown-developer/backup-restore-unknown-developer-1-2-a-1-23-release/backup-restore-1-2-a-1-23-android-apk-download/', 'Backup & restore 1.2.A.1.23', 'Sony Mobile Communications', 'Backup & restore', '1.2.A.1.23', '2360343', ARRAY['arm'], 'com.sonymobile.synchub', 9541452, '6.0', '7.1', ARRAY['nodpi'], 'eda39fedfc33ea939a5bb4926b3980c1', '2017-04-25T16:30:00+00:00'::timestamptz, '2017-06-29T14:10:39+00:00'::timestamptz, ARRAY['http://www.apkmirror.com/wp-content/themes/APKMirror/ap_resize/ap_resize.php?src=http%3A%2F%2Fwww.apkmirror.com%2Fwp-content%2Fuploads%2F2017%2F04%2F58ff36aea8e0c.png&w=96&h=96&q=100'], '{}', ARRAY['http://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=203068'], '{}', '2017-06-29T14:19:21.661193'::timestamp, '2017-06-29T14:19:21.661207'::timestamp) RETURNING apks.id
db_1       | LOG:  received smart shutdown request
db_1       | LOG:  autovacuum launcher shutting down
db_1       | LOG:  shutting down
db_1       | LOG:  database system is shut down
db_1       | LOG:  database system was shut down at 2017-06-29 16:28:54 UTC
db_1       | LOG:  MultiXact member wraparound protections are now enabled

虽然我没有显示应用程序的详细信息，但我的主要观点是我肯定错误消息中显示的数据来自之前的运行，而不是来 self 当前尝试的 APK 下载页面刮。

简而言之，似乎 PostgreSQL 正在“缓存”以前的语句并尝试重新运行它们，即使在我停止应用程序后，执行 docker-compose build 和 docker-compose up 。在我看来，这种行为必须以某种方式由 /var/lib/postgresql/data 中的数据驱动，因为这个卷是唯一在重新启动容器时持久存在的东西。

真的是这样吗？如果是这样，我该如何禁用这种“语句缓存”行为？

Answer 1

这只是意味着 docker-compose 正在重新启动您之前停止的容器(或者重新附加到它，如果它已经在运行)，并且因为不需要重新构建图像:

apkmirrorscrapercompose_db_1 is up-to-date

因此，您在日志中“再次”看到的是该容器的 docker 日志，其中包括 postgres 的所有生命生成日志。

只需删除容器并重新启动(db 数据保存在卷中，因此它会存活):

docker-compose rm db