amazon-web-services - 如何在AWS上创建集群期间动态更新kubeconfig？

Question

我使用模板云形式在aws上创建集群:

AWSTemplateFormatVersion: "2010-09-09"
Description: Deploys an EKS cluster in a new VPC (qs-1p7nknoht)
Metadata:
  LintSpellExclude:
    - Kubernetes
    - ARNs
    - Resource Names
    - autoscaler
    - IOPS
    - EfsStorageClass
    - Lambda
    - maxIO
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: VPC network configuration
        Parameters:
          - NumberOfAZs
          - AvailabilityZones
          - VPCCIDR
          - PrivateSubnet1CIDR
          - PrivateSubnet2CIDR
          - PrivateSubnet3CIDR
          - PublicSubnet1CIDR
          - PublicSubnet2CIDR
          - PublicSubnet3CIDR
          - RemoteAccessCIDR
          - ProvisionBastionHost
      - Label:
          default: Amazon EC2 configuration
        Parameters:
          - KeyPairName
      - Label:
          default: Amazon EKS configuration
        Parameters:
          - NodeInstanceType
          - NumberOfNodes
          - MaxNumberOfNodes
          - NodeGroupName
          - NodeVolumeSize
          - ManagedNodeGroup
          - ManagedNodeGroupAMIType
          - AdditionalEKSAdminArns
          - KubernetesVersion
      - Label:
          default: Optional Kubernetes add-ins
        Parameters:
          - ClusterAutoScaler
          - EfsStorageClass
          - EfsPerformanceMode
          - EfsThroughputMode
          - EfsProvisionedThroughputInMibps
          - MonitoringStack
      - Label:
          default: AWS Quick Start configuration
        Parameters:
          - QSS3BucketName
          - QSS3KeyPrefix
          - QSS3BucketRegion
          - LambdaZipsBucketName
    ParameterLabels:
      AvailabilityZones:
        default: Availability Zones
      KeyPairName:
        default: SSH key name
      PrivateSubnet1CIDR:
        default: Private subnet 1 CIDR
      PrivateSubnet2CIDR:
        default: Private subnet 2 CIDR
      PrivateSubnet3CIDR:
        default: Private subnet 3 CIDR
      PublicSubnet1CIDR:
        default: Public subnet 1 CIDR
      PublicSubnet2CIDR:
        default: Public subnet 2 CIDR
      PublicSubnet3CIDR:
        default: Public subnet 3 CIDR
      QSS3BucketName:
        default: Quick Start S3 bucket name
      QSS3KeyPrefix:
        default: Quick Start S3 key prefix
      QSS3BucketRegion:
        default: Quick Start S3 bucket region
      RemoteAccessCIDR:
        default: Allowed external access CIDR
      VPCCIDR:
        default: VPC CIDR
      NodeInstanceType:
        default: Nodes instance type
      NumberOfNodes:
        default: Number of nodes
      MaxNumberOfNodes:
        default: Maximum number of nodes  
      NodeGroupName:
        default: Node group name
      NodeVolumeSize:
        default: Node volume size
      ManagedNodeGroup:
        default: Managed node group
      ManagedNodeGroupAMIType:
        default: Managed node group AMI type
      AdditionalEKSAdminArns:
        default: Additional EKS admin ARNs
      KubernetesVersion:
        default: Kubernetes version
      LambdaZipsBucketName:
        default: Lambda zips bucket name
      ClusterAutoScaler:
        default: Cluster autoscaler
      EfsStorageClass:
        default: EFS storage class
      EfsPerformanceMode:
        default: EFS performance mode
      EfsThroughputMode:
        default: EFS throughput mode
      EfsProvisionedThroughputInMibps:
        default: EFS provisioned throughput in Mibps
      MonitoringStack:
        default: Monitoring Stack
      NumberOfAZs:
        default: Number of Availability Zones
      ProvisionBastionHost:
        default: Provision Bastion Host
Parameters:
  AvailabilityZones:
    Description: The list of Availability Zones to use for the subnets in the VPC. Three
      Availability Zones are used for this deployment, and the logical order of your
      selections is preserved.
    Type: List<AWS::EC2::AvailabilityZone::Name>
  KeyPairName:
    Description: The name of an existing public/private key pair, which allows you
      to securely connect to your instance after it launches
    Type: AWS::EC2::KeyPair::KeyName
  PrivateSubnet1CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.0.0/19
    Description: The CIDR block for private subnet 1 located in Availability Zone 1
    Type: String
  PrivateSubnet2CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.32.0/19
    Description: The CIDR block for private subnet 2 located in Availability Zone 2
    Type: String
  PrivateSubnet3CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.64.0/19
    Description: The CIDR block for private subnet 3 located in Availability Zone 3
    Type: String
  PublicSubnet1CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.128.0/20
    Description: CIDR block for the public (DMZ) subnet 1 located in Availability
      Zone 1
    Type: String
  PublicSubnet2CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.144.0/20
    Description: The CIDR block for the public (DMZ) subnet 2 located in Availability
      Zone 2
    Type: String
  PublicSubnet3CIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.160.0/20
    Description: The CIDR block for the public (DMZ) subnet 3 located in Availability
      Zone 3
    Type: String
  QSS3BucketName:
    AllowedPattern: ^[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$
    ConstraintDescription: Quick Start bucket name can include numbers, lowercase
      letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen
      (-).
    Default: aws-quickstart
    Description: S3 bucket name for the Quick Start assets. This string can include
      numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start
      or end with a hyphen (-).
    Type: String
  QSS3KeyPrefix:
    AllowedPattern: ^[0-9a-zA-Z-/.]*$
    ConstraintDescription: Quick Start key prefix can include numbers, lowercase letters,
      uppercase letters, hyphens (-), dots(.) and forward slash (/).
    Default: quickstart-amazon-eks/
    Description: S3 key prefix for the Quick Start assets. Quick Start key prefix
      can include numbers, lowercase letters, uppercase letters, hyphens (-), dots(.) and
      forward slash (/).
    Type: String
  QSS3BucketRegion:
    Default: 'us-east-1'
    Description: The AWS Region where the Quick Start S3 bucket (QSS3BucketName) is
      hosted. When using your own bucket, you must specify this value.
    Type: String
  RemoteAccessCIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/x
    Description: The CIDR IP range that is permitted to access the instances. We recommend
      that you set this value to a trusted IP range.
    Type: String
  VPCCIDR:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28
    Default: 10.0.0.0/16
    Description: The CIDR block for the VPC
    Type: String
  AdditionalEKSAdminArns:
    Default: ""
    Description: "[OPTIONAL] Comma separated list of IAM user/role Amazon Resource Names (ARNs) to be granted admin access to the EKS cluster"
    Type: CommaDelimitedList
  NodeInstanceType:
    Default: t3.medium
    AllowedValues:
      - t2.small
      - t2.medium
      - t2.large
      - t2.xlarge
      - t2.2xlarge
      - t3.nano
      - t3.micro
      - t3.small
      - t3.medium
      - t3.large
      - t3.xlarge
      - t3.2xlarge
      - m3.medium
      - m3.large
      - m3.xlarge
      - m3.2xlarge
      - m4.large
      - m4.xlarge
      - m4.2xlarge
      - m4.4xlarge
      - m4.10xlarge
      - m5.large
      - m5.xlarge
      - m5.2xlarge
      - m5.4xlarge
      - m5.12xlarge
      - m5.24xlarge
      - c4.large
      - c4.xlarge
      - c4.2xlarge
      - c4.4xlarge
      - c4.8xlarge
      - c5.large
      - c5.xlarge
      - c5.2xlarge
      - c5.4xlarge
      - c5.9xlarge
      - c5.18xlarge
      - i3.large
      - i3.xlarge
      - i3.2xlarge
      - i3.4xlarge
      - i3.8xlarge
      - i3.16xlarge
      - r3.xlarge
      - r3.2xlarge
      - r3.4xlarge
      - r3.8xlarge
      - r4.large
      - r4.xlarge
      - r4.2xlarge
      - r4.4xlarge
      - r4.8xlarge
      - r4.16xlarge
      - x1.16xlarge
      - x1.32xlarge
      - p2.xlarge
      - p2.8xlarge
      - p2.16xlarge
      - p3.2xlarge
      - p3.8xlarge
      - p3.16xlarge
      - r5.large
      - r5.xlarge
      - r5.2xlarge
      - r5.4xlarge
      - r5.12xlarge
      - r5.24xlarge
      - r5d.large
      - r5d.xlarge
      - r5d.2xlarge
      - r5d.4xlarge
      - r5d.12xlarge
      - r5d.24xlarge
      - z1d.large
      - z1d.xlarge
      - z1d.2xlarge
      - z1d.3xlarge
      - z1d.6xlarge
      - z1d.12xlarge
    ConstraintDescription: Must be a valid EC2 instance type
    Description: The type of EC2 instance for the node instances.
    Type: String
  NumberOfNodes:
    Default: 3
    Description: The number of Amazon EKS node instances. The default is one for each of the three Availability Zones.
    Type: Number
  MaxNumberOfNodes:
    Default: 3
    Description: The maximum number of Amazon EKS node instances. The default is three node.
    Type: Number  
  NodeGroupName:
    Default: Default
    Description: The name for EKS node group.
    Type: String
  NodeVolumeSize:
    Default: 20
    Description: "The size for the node's root EBS volumes."
    Type: String
  ManagedNodeGroup:
    AllowedValues: [ "yes", "no" ]
    Default: "no"
    Description: Choose if you want to use a managed node group. If you select "yes", you must select Kubernetes Version 1.14 or higher.
    Type: String
  ManagedNodeGroupAMIType:
    Description: Select one of the two AMI types for your managed node group (only applies if you chose "yes" for ManagedNodeGroup). GPU instance types should use the AL2_x86_64_GPU AMI type, which uses the Amazon EKS-optimized Linux AMI with GPU support. Non-GPU instances should use the AL2_x86_64 AMI type, which uses the Amazon EKS-optimized Linux AMI.
    AllowedValues: [ "AL2_x86_64", "AL2_x86_64_GPU", ""]
    Default: "AL2_x86_64"
    Type: String
  KubernetesVersion:
    Type: String
    AllowedValues: [ "1.13", "1.14", "1.15" ]
    Default: "1.15"
    Description: The Kubernetes control plane version.
  LambdaZipsBucketName:
    Description: '[OPTIONAL] The name of the S3 bucket where the Lambda zip files should be placed. If you leave this parameter blank, an S3 bucket will be created.'
    Type: String
    Default: ''
  ClusterAutoScaler:
    Type: String
    AllowedValues: [ Enabled, Disabled ]
    Default: Disabled
    Description: Choose Enabled to enable Kubernetes cluster autoscaler.
  EfsStorageClass:
    Type: String
    AllowedValues: [ Enabled, Disabled ]
    Default: Disabled
    Description: Choose Enabled to enable EFS storage class, which will create the required EFS volume.
  EfsPerformanceMode:
    Type: String
    AllowedValues: [ generalPurpose, maxIO ]
    Default: generalPurpose
    Description: Choose maxIO mode to provide greater IOPS with an increased latency. Only has an effect when EfsStorageClass is enabled.
  EfsThroughputMode:
    Type: String
    AllowedValues: [ bursting, provisioned ]
    Default: bursting
    Description: Choose provisioned for throughput that is not dependent on the amount of data stored in the file system. Only has an effect when EfsStorageClass is enabled.
  EfsProvisionedThroughputInMibps:
    Type: Number
    MinValue: 0
    Default: 0
    Description: Set to 0 if EfsThroughputMode is set to bursting. Only has an effect when EfsStorageClass is enabled.
  MonitoringStack:
    Type: String
    AllowedValues: [ "Prometheus + Grafana", "None" ]
    Default: "None"
    Description: Enable Monitoring stack with "Prometheus+Grafana"
  NumberOfAZs:
    Type: String
    AllowedValues: ["2", "3"]
    Default: "3"
    Description: Number of Availability Zones to use in the VPC. This must match your selections in the list of Availability Zones parameter.
  ProvisionBastionHost:
    Type: String
    AllowedValues: [ "Enabled", "Disabled" ]
    Default: "Enabled"
    Description: "Skip creating a bastion host by setting this is set to Disabled."
Rules:
  EKSSupport:
    Assertions:
      - AssertDescription: Your AWS Region does *NOT* yet support Amazon EKS
        Assert: !Contains
          -  - us-west-2
             - us-east-1
             - us-east-2
             - sa-east-1
             - eu-west-1
             - eu-west-2
             - eu-west-3
             - eu-north-1
             - eu-central-1
             - ap-southeast-1
             - ap-southeast-2
             - ap-northeast-1
             - ap-northeast-2
             - ap-south-1
             - ca-central-1
          - !Ref 'AWS::Region'
  EKSVersion_ManagedNodeGroup:
    RuleCondition: !Equals [ !Ref 'ManagedNodeGroup', 'yes' ]
    Assertions:
      - AssertDescription: To use Managed Node Groups you must use EKS version 1.14 or higher
        Assert: !Contains
          - [ '1.15', '1.14' ]
          - !Ref 'KubernetesVersion'
  ClusterAutoScalerVerification:
    RuleCondition: !Equals [ !Ref 'ManagedNodeGroup', 'yes' ]
    Assertions:
      - AssertDescription: To use Cluster AutoScaler you should not use Managed Node Groups
        Assert: !Contains
          - - 'Disabled'
          - !Ref 'ClusterAutoScaler'
Conditions:
  3AZDeployment: !Equals [!Ref NumberOfAZs, "3"]
  2AZDeployment: !Or
    - !Equals [!Ref NumberOfAZs, "2"]
    - !Equals [!Ref NumberOfAZs, "3"]
  UsingDefaultBucket: !Equals [!Ref QSS3BucketName, 'aws-quickstart']
Resources:
  VPCStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub
        - 'https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}submodules/quickstart-aws-vpc/templates/aws-vpc.template'
        - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion]
          S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
      Parameters:
        AvailabilityZones: !Join [ ',', !Ref 'AvailabilityZones' ]
        KeyPairName: !Ref 'KeyPairName'
        NumberOfAZs: !Ref 'NumberOfAZs'
        PrivateSubnet1ACIDR: !Ref 'PrivateSubnet1CIDR'
        PrivateSubnet2ACIDR: !Ref 'PrivateSubnet2CIDR'
        PrivateSubnet3ACIDR: !Ref 'PrivateSubnet3CIDR'
        PrivateSubnetATag2: "kubernetes.io/role/internal-elb="
        PublicSubnet1CIDR: !Ref 'PublicSubnet1CIDR'
        PublicSubnet2CIDR: !Ref 'PublicSubnet2CIDR'
        PublicSubnet3CIDR: !Ref 'PublicSubnet3CIDR'
        PublicSubnetTag2: "kubernetes.io/role/elb="
        VPCCIDR: !Ref 'VPCCIDR'
  EKSStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub
        - 'https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/amazon-eks.template.yaml'
        - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion]
          S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
      Parameters:
        PublicSubnet1ID: !GetAtt VPCStack.Outputs.PublicSubnet1ID
        PublicSubnet2ID: !If
          - 2AZDeployment
          - !GetAtt VPCStack.Outputs.PublicSubnet2ID
          - !Ref AWS::NoValue
        PublicSubnet3ID: !If
          - 3AZDeployment
          - !GetAtt VPCStack.Outputs.PublicSubnet3ID
          - !Ref AWS::NoValue
        KeyPairName: !Ref KeyPairName
        QSS3BucketName: !Ref QSS3BucketName
        QSS3KeyPrefix: !Ref QSS3KeyPrefix
        QSS3BucketRegion: !Ref QSS3BucketRegion
        PrivateSubnet1ID: !GetAtt VPCStack.Outputs.PrivateSubnet1AID
        PrivateSubnet2ID: !If
          - 2AZDeployment
          - !GetAtt VPCStack.Outputs.PrivateSubnet2AID
          - !Ref AWS::NoValue
        PrivateSubnet3ID: !If
          - 3AZDeployment
          - !GetAtt VPCStack.Outputs.PrivateSubnet3AID
          - !Ref AWS::NoValue
        NumberOfNodes: !Ref NumberOfNodes
        MaxNumberOfNodes: !Ref MaxNumberOfNodes
        NodeGroupName: !Ref NodeGroupName
        NodeVolumeSize: !Ref NodeVolumeSize
        ManagedNodeGroup: !Ref ManagedNodeGroup
        ManagedNodeGroupAMIType: !Ref ManagedNodeGroupAMIType
        LambdaZipsBucketName: !Ref LambdaZipsBucketName
        NodeInstanceType: !Ref NodeInstanceType
        RemoteAccessCIDR: !Ref RemoteAccessCIDR
        AdditionalEKSAdminArns: !Join [ ",", !Ref AdditionalEKSAdminArns ]
        VPCID: !GetAtt VPCStack.Outputs.VPCID
        KubernetesVersion: !Ref KubernetesVersion
        ProvisionClusterAutoScaler: !Ref ClusterAutoScaler
        EfsStorageClass: !Ref EfsStorageClass
        EfsPerformanceMode: !Ref EfsPerformanceMode
        EfsThroughputMode: !Ref EfsThroughputMode
        EfsProvisionedThroughputInMibps: !Ref EfsProvisionedThroughputInMibps
        ProvisionMonitoringStack: !Ref MonitoringStack
        ProvisionBastionHost: !Ref ProvisionBastionHost
Outputs:
  KubeConfigPath:
    Value: !GetAtt EKSStack.Outputs.KubeConfigPath
  HelmLambdaArn:
    Value: !GetAtt EKSStack.Outputs.HelmLambdaArn
  KubeManifestLambdaArn:
    Value: !GetAtt EKSStack.Outputs.KubeManifestLambdaArn
  KubeGetLambdaArn:
    Value: !GetAtt EKSStack.Outputs.KubeGetLambdaArn
  EKSClusterName:
    Value: !GetAtt EKSStack.Outputs.EKSClusterName
  BastionIP:
    Value: !GetAtt EKSStack.Outputs.BastionIP
  BastionSecurityGroup:
    Value: !GetAtt EKSStack.Outputs.BastionSecurityGroup
  NodeGroupSecurityGroup:
    Value: !GetAtt EKSStack.Outputs.NodeGroupSecurityGroup

我还通过以下ARN创建了一个IAM用户来访问我的群集:arn:aws:iam::XXXXXXXXXXXX:role / testrole

为了使用此角色访问我的集群，aws建议更新配置映射aws-auth并添加类似以下内容: $ kubectl edit configmap aws-auth -n kube-syste m

mapRoles: |
  - rolearn: arn:aws:iam::XXXXXXXXXXXX:role/testrole
    username: testrole
    groups:
      - system:masters

在此处链接od recommandation: https://aws.amazon.com/fr/premiumsupport/knowledge-center/eks-api-server-unauthorized-error/

每天，我下类时必须删除集群，并在星期一重新创建集群(由于控制平面成本)。但是，我不想每天都在使用云休眠模板创建集群时手动更新配置映射aws-auth以添加之前的代码。

我如何在我的cloudfourmation模板中添加此角色，以便使用自动添加的默认角色创建集群？

提前致谢

Answer 1

在使用模板创建集群的过程中，有一个步骤要求您添加其他信息。您可以在此处指定您的arn，然后默认情况下它将自动添加到您的集群中。您也可以在云形成文件中的以下位置进行设置:

  AdditionalEKSAdminArns:
    default: Additional EKS admin ARNs