python - AWS Glue Job Cloudformation - Cloudformation 中设置的值不固定

Question

下面的 Cloudformation 设置未按我的预期运行。以下变量未使用下面的模板进行设置。手动设置变量后，作业将成功运行。

• IAM 角色
• 类型
• 语言

---

Description: "AWS Glue Job Test"
Resources:
  MyJobRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Effect: "Allow"
            Principal:
              Service:
                - "glue.amazonaws.com"
            Action:
              - "sts:AssumeRole"
      Path: "/"
      Policies:
        -
          PolicyName: "root"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: "*"
                Resource: "*"
  MyJob:
    Type: AWS::Glue::Job
    Properties:
      Name: cf-job1
      Command:
        Name: test-etl1
        ScriptLocation: "s3://project_bucket/releases/latest/mixpanel_job.py"
        PythonVersion: "3"
      Description: "Testing setup config"
      ExecutionProperty:
        MaxConcurrentRuns: 2
      MaxRetries: 2
      GlueVersion: "3.0"
      WorkerType: "G.1X"
      NumberOfWorkers: 2
      Timeout: 2880
      DefaultArguments:
        "--class": "GlueApp"
        "--enable-continuous-cloudwatch-log": "true"
        "--enable-job-insights": "true"
        "--enable-metrics": "true"
        "--enable-spark-ui": "true"
        "--extra-jars": "s3://project_bucket/releases/latest/jars/delta-core_2.12-1.0.1.jar"
        "--extra-py-files": "s3://project_bucket/releases/latest/lib.zip"
        "--job-bookmark-option": "job-bookmark-disable"
        "--job-language": "python"
        "--spark-event-logs-path": "s3://project_bucket/logs/"
      Role: !Ref MyJobRole

Result of Stack

Answer 1

这是 Glue 团队中 AWS Cloudformation 的一个已知问题。

只能使用指向另一个 ROLE 的物理 ARN 字符串值来设置运行作业的默认 IAM 角色

--作业语言参数设置不正确，这是一个已知问题。这必须通过 Glue 控制台手动完成

控制台中的Type参数仍然没有cloudformation参数，必须手动设置。

目前还无法实现真正​​的自动化。