gpt4 book ai didi

amazon-web-services - AWS Lambda 函数在查询 Athena 时失败

转载 作者:行者123 更新时间:2023-12-05 06:12:13 27 4
gpt4 key购买 nike

我正在尝试编写一个简单的 Lambda 函数来查询 Athena 中的表。但几秒钟后,我在 Cloudwatch 日志中看到“状态:失败”。没有关于失败原因的描述性错误消息。

我的测试代码如下:

import json
import time
import boto3

# athena constant
DATABASE = 'default'
TABLE = 'test'

# S3 constant
S3_OUTPUT = 's3://test-output/'

# number of retries
RETRY_COUNT = 1000

def lambda_handler(event, context):
# created query
query = "SELECT * FROM default.test limit 2"
# % (DATABASE, TABLE)
# athena client
client = boto3.client('athena')

# Execution
response = client.start_query_execution(
QueryString=query,
QueryExecutionContext={
'Database': DATABASE
},
ResultConfiguration={
'OutputLocation': S3_OUTPUT,
}
)

# get query execution id
query_execution_id = response['QueryExecutionId']
print(query_execution_id)

# get execution status
for i in range(1, 1 + RETRY_COUNT):

# get query execution
query_status = client.get_query_execution(QueryExecutionId=query_execution_id)
query_execution_status = query_status['QueryExecution']['Status']['State']

if query_execution_status == 'SUCCEEDED':
print("STATUS:" + query_execution_status)
break

if query_execution_status == 'FAILED':
#raise Exception("STATUS:" + query_execution_status)
print("STATUS:" + query_execution_status)

else:
print("STATUS:" + query_execution_status)
time.sleep(i)
else:
# Did not encounter a break event. Need to kill the query
client.stop_query_execution(QueryExecutionId=query_execution_id)
raise Exception('TIME OVER')

# get query results
result = client.get_query_results(QueryExecutionId=query_execution_id)
print(result)
return

日志显示如下:

2020-08-31T10:52:12.443-04:00

START RequestId: e5434651-d36e-48f0-8f27-0290 Version: $LATEST

2020-08-31T10:52:13.481-04:00

88162f38-bfcb-40ae-b4a3-0b5a21846e28

2020-08-31T10:52:13.500-04:00

STATUS:QUEUED

2020-08-31T10:52:14.519-04:00

STATUS:RUNNING

2020-08-31T10:52:16.540-04:00

STATUS:RUNNING

2020-08-31T10:52:19.556-04:00

STATUS:RUNNING

2020-08-31T10:52:23.574-04:00

STATUS:RUNNING

2020-08-31T10:52:28.594-04:00

STATUS:FAILED

2020-08-31T10:52:28.640-04:00

....more status: FAILED
....

END RequestId: e5434651-d36e-48f0-8f27-0290

REPORT RequestId: e5434651-d36e-48f0-8f27-0290 Duration: 30030.22 ms Billed Duration: 30000 ms Memory Size: 128 MB Max Memory Used: 72 MB Init Duration: 307.49 ms

2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds

我认为我拥有授予该角色的 S3 存储桶访问权限(如果没有,我会看到错误消息)。存储桶中也没有创建文件。我不确定这里出了什么问题。我错过了什么?谢谢

最佳答案

日志中的最后一行显示

2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds 

在我看来,Lambda 函数的超时设置为 30 秒。尝试将其增加到超过 Athena 查询所需的时间(最长为 15 分钟)。

关于amazon-web-services - AWS Lambda 函数在查询 Athena 时失败,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63673741/

27 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com