aws-cloudformation - Cloudformation自定义模块

Question

Cloudformation 中是否有类似于 terraform 模块 的内容，您可以在其中创建参数化模板(资源组，而不是 CF 模板)，然后在您的 < em>CF 模板多次使用不同的参数？

我在我的 CF 模板中看到很多 bolierplate YAML，我正在寻找一种重构它的方法。我以前使用过 terraform，它提供了该功能。

示例:

我正在创建许多 AWS Glue 作业，其中许多作业只有 2 个参数不同，但每个定义都是 25 行代码。

Resources:
  myGlueJob1:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob1
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": PARAM1-THAT-DIFFERS
        "--ga_view_id": PARAM2-THAT-DIFFERS
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
  myGlueJob2:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob2
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": PARAM1-THAT-DIFFERS
        "--ga_view_id": PARAM2-THAT-DIFFERS
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"

我可以想象一个看起来像这样的解决方案:

Module:
    Type: Me::MyGlueJob
    Resouces:
        Type: AWS::Glue::Job
        Properties:
          ExecutionProperty:
            MaxConcurrentRuns: 1
          MaxRetries: 3
          Name: myGlueJob2
          Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
          Command:
            Name: glueetl
            ScriptLocation: XXXXXX
          DefaultArguments:
            "--ga_project_id": {{ MY_PARAM1 }}
            "--ga_view_id": {{ MY_PARAM2 }}
            "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
            "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
            "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
            "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
            "--job-language": scala
            "--class": GlueApp
            "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
            "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
            "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
Resources:
  myGlueJob1:
    Type: Me::MyGlueJob
    Properties:
        MY_PARAM1: value-for-job1
        MY_PARAM2: value-for-job1
  myGlueJob2:
    Type: Me::MyGlueJob
    Properties:
        MY_PARAM1: value-for-job2
        MY_PARAM2: value-for-job2

任何有关最佳实践的提示将不胜感激。

Answer 1

您应该能够使用jinja2 templates to generate actual CloudFormation模板。

在您的情况下，它应该如下所示:

{% set job_params = [
  ["value-for-job1", "value-for-job1"],
  ["value-for-job2", "value-for-job2"]
] %}

Resources:
{% for params in job_params %}
  myGlueJob{{loop.index}}:
    Type: AWS::Glue::Job
    Properties:
      ExecutionProperty:
        MaxConcurrentRuns: 1
      MaxRetries: 3
      Name: myGlueJob1
      Role: arn:aws:iam::xxxxxxx:role/XXXXXXXXXXXXXX
      Command:
        Name: glueetl
        ScriptLocation: XXXXXX
      DefaultArguments:
        "--ga_project_id": "{{params[0]}}"
        "--ga_view_id": "{{params[1]}}"
        "--ga_service_account_keyfile": gc.key-SAME_FOR_ALL_RESOURCES
        "--date": YESTERDAY-SAME_FOR_ALL_RESOURCES
        "--temp_gcs_bucket": "foobar-SAME_FOR_ALL_RESOURCES"
        "--output_path": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--job-language": scala
        "--class": GlueApp
        "--TempDir": "s3://some-other-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-files": "s3://some-s3-path-SAME_FOR_ALL_RESOURCES"
        "--extra-jars": "s3://looooooooooooooooooooooooots-of-s3-paths-SAME_FOR_ALL_RESOURCES"
{% endfor %}