amazon-web-services - 资源更新/创建后如何运行 AWS Glue Crawler？

Question

我在我的 Terraform 中定义了一个资源来创建一个我不想安排的 Glue Crawler。但我希望它在创建和更新后运行。我在文档中找不到任何关于如何触发它的内容。

resource "aws_glue_crawler" "my_crawler" {
  database_name = "my_db"
  name          = "my_crawler"
  role          = "arn:aws:iam::111111111111:role/service-role/someRole"

  s3_target {
    path = "s3://my_bucket/key/prefix"
  }

}

Answer 1

你可以使用 local-exec provisioner在 Glue 爬虫创建后使用 AWS CLI 触发它:

resource "aws_glue_crawler" "my_crawler" {
  database_name = "my_db"
  name          = "my_crawler"
  role          = "arn:aws:iam::111111111111:role/service-role/someRole"

  s3_target {
    path = "s3://my_bucket/key/prefix"
  }

  provisioner "local-exec" {
    command = "aws glue start-crawler --name ${self.name}"
  }
}

这只会在创建爬虫时触发，而不会在任何其他点触发，例如如果您更改了 s3_target.path 或其他任何内容。

如果您希望能够在更改 s3_target.path 时触发它，您需要使用 null_resource with a trigger :

resource "aws_glue_crawler" "my_crawler" {
  database_name = "my_db"
  name          = "my_crawler"
  role          = "arn:aws:iam::111111111111:role/service-role/someRole"

  s3_target {
    path = "s3://my_bucket/key/prefix"
  }
}

resource "null_resource" "run_crawler" {
  # Changes to the crawler's S3 path requires re-running
  triggers = {
    s3_path = "${aws_glue_crawler.my_crawler.s3_target.0.path}"
  }

  provisioner "local-exec" {
    command = "aws glue start-crawler --name ${aws_glue_crawler.my_crawler.name}"
  }
}