sql - Redshift 光谱 : how to import only certain files

Question

使用 Redshift 光谱时，您似乎只能导入提供位置直到文件夹的数据，并且导入文件夹内的所有文件。

有没有办法从包含多个文件的文件夹中导入仅导入一个文件。当提供带有 filename 的完整路径时，我认为它将文件视为 list 文件并给出错误: list 太大或不支持 JSON。

有没有其他办法？

Answer 1

您无意中回答了自己的问题:使用 list 文件

来自 CREATE EXTERNAL TABLE - Amazon Redshift :

LOCATION { 's3://bucket/folder/' | 's3://bucket/manifest_file' }

The path to the Amazon S3 bucket or folder that contains the data files or a manifest file that contains a list of Amazon S3 object paths. The buckets must be in the same AWS Region as the Amazon Redshift cluster.

If the path specifies a manifest file, the s3://bucket/manifest_file argument must explicitly reference a single file—for example,'s3://mybucket/manifest.txt'. It can't reference a key prefix.

The manifest is a text file in JSON format that lists the URL of each file that is to be loaded from Amazon S3 and the size of the file, in bytes. The URL includes the bucket name and full object path for the file. The files that are specified in the manifest can be in different buckets, but all the buckets must be in the same AWS Region as the Amazon Redshift cluster.

我不确定为什么它需要每个文件的长度。它可用于在多个节点之间分配工作负载。