postgresql - 将数据从 AWS S3 复制到 Aurora Postgres

Question

我正在尝试将数据从 AWS S3 复制到 Aurora Postgres。目前我的流程如下:

1. 从 S3 本地下载文件(在 EC2 实例上)
2. 运行“COPY FROM STDIN ...”命令将数据从输入流加载到 Aurora postgres。

我想知道是否有直接从 S3 复制到 Aurora postgres 的命令。

Answer 1

这篇来自 AWS 的博客文章提到能够使用 AWS Database Migration service 来完成它

https://aws.amazon.com/blogs/database/stream-data-into-an-aurora-postgresql-database-using-aws-dms-and-amazon-kinesis-data-firehose/

AWS DMS can read data from source S3 buckets and load them into a target database. To do this, provide access to an S3 bucket containing one or more data files. In that S3 bucket, include a JSON file that describes the mapping between the data and the database tables of the data in those files.

The source data files must be in comma-separated value (CSV) format. Name the files using the naming convention shown following. In this convention, schemaName is the source schema and tableName is the name of a table within that schema.