gpt4 book ai didi

aws-glue - 来自 AWS secret 管理器的 AWS Glue 连接

转载 作者:行者123 更新时间:2023-12-05 06:25:33 28 4
gpt4 key购买 nike

在 AWS Glue 中创建 JDBC 连接时,有什么方法可以从 AWS secret manager 获取密码而不是手动硬编码吗?

最佳答案

我必须在我当前的项目中这样做才能连接到 Cassandra 数据库,这就是我的做法。我将实际的 secrets_key 作为作业参数传递 --SECRETS_KEY my/secrets/key

// here's method to pull from secrets manager
def retrieveSecrets(secrets_key: String) :Map[String,String] = {
val awsSecretsClient = AWSSecretsManagerClientBuilder.defaultClient()
val getSecretValueRequest = new GetSecretValueRequest().withSecretId(secrets_key)
val secretValue = awsSecretsClient.getSecretValue(getSecretValueRequest)
val secretJson = secretValue.getSecretString()
val result = JSON.parseFull(secretJson)
val jsonMap:Map[String,String] = result.get.asInstanceOf[Map[String, String]]
return jsonMap
}

// main script where I invoke this call BEFORE the glue stuff is setup..
def main(sysArgs: Array[String]) {
println("***** RETRIEVING Secrets from SecretsManager ****** ")
// pull secrets from map
val secretsMap = retrieveSecrets(secrets_key)
val host_names = secretsMap.get("HOST_NAMES").get
val user_id = secretsMap.get("USER_ID").get
val password = secretsMap.get("PASSWORD").get

// I needed to configure this BEFORE we create the spark Context
val conf = setupCassandraConfiguration(host_names, user_id, password)
val sparkContext: SparkContext = new SparkContext(conf)
// now create the glue context using the spark context created with the cassandra options attached
val glueContext: GlueContext = new GlueContext(sparkContext)
Job.init(args("JOB_NAME"), glueContext, args.asJava)
val spark = glueContext.getSparkSession

println("***** ALL contexts created - starting loading tables ****** ")
import spark.implicits._
val myDF = spark.read.format("org.apache.spark.sql.cassandra").options(Map( "table" -> "my_table", "keyspace" -> "my_keyspace" )).load()
}

关于aws-glue - 来自 AWS secret 管理器的 AWS Glue 连接,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56859250/

28 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com