amazon-web-services - 递归 AWS Lambda 函数调用 - 最佳实践

Question

我的任务是查看基于 AWS Lambda 构建的服务，该服务执行打开和关闭 VM 的长期运行任务。请注意，我来自 Azure 团队，所以我不熟悉 AWS 服务的样式或最佳实践。

原始开发人员采用的方法是将整个工作负载发送到一个 Lambda 函数，然后让该函数获取一部分工作负载，然后使用剩余的工作负载递归调用自身，直到所有项目都消失(工作负载 = 0) .

伪代码:

// Assume this gets sent to a HTTP Lambda endpoint as a whole
let workload = [1, 2, 3, 4, 5, 6, 7, 8]

// The Lambda HTTP endpoint
function Lambda(workload) {
    if (!workload.length) {
        return "No more work!"
    }
    const toDo = workload.splice(0, 2) // get first two items
    doWork(toDo)

    // Then... except it builds a new HTTP request with aws sdk
    Lambda(workload) // 3, 4, 5, 6, 7, 8, etc.
}

这似乎非常低效且不可靠(如果我错了请纠正我)。在这个过程中存储了很多状态，在我看来这会产生很多故障点。

我的计划是建议我们重新设计整个服务，改为使用 Queue/Worker 类型的框架，理想情况下端点一次处理一个工作负载，并且是无状态的。

队列将由一项服务(Jenkins？Lambda？手动？)填充，然后第二项服务将从队列中读取(理想情况下还可以根据需要横向扩展)。

Answer 1

更新:AWS EventBridge 现在看起来像是首选解决方案。

---

我想到的是“耦合”，看这里:https://www.jeffersonfrank.com/insights/aws-lambda-design-considerations

Coupling
Coupling goes beyond Lambda design considerations—it’s more about the system as a whole. Lambdas within a microservice are sometimes tightly coupled, but this is nothing to worry about as long as the data passed between Lambdas within their little black box of a microservice is not over-pure HTTP and isn’t synchronous.

Lambdas shouldn’t be directly coupled to one another in a Request Response fashion, but asynchronously. Consider the scenario when an S3 Event invokes a Lambda function, then that Lambda also needs to call another Lambda within that same microservice and so on.

aws lambda 耦合

You might be tempted to implement direct coupling, like allowing Lambda 1 to use the AWS SDK to call Lambda 2 and so on. This introduces some of the following problems:

1. If Lambda 1 is invoking Lambda 2 synchronously, it needs to wait for the latter to be done first. Lambda 1 might not know that Lambda 2 also called Lambda 3 synchronously, and Lambda 1 may now need to wait for both Lambda 2 and 3 to finish successfully. Lambda 1 might timeout as it needs to wait for all the Lambdas to complete first, and you’re also paying for each Lambda while they wait.

2. What if Lambda 3 has a concurrency limit set and is also called by another service? The call between Lambda 2 and 3 will fail until it has concurrency again. The error can be returned to all the way back to Lambda 1 but what does Lambda 1 then do with the error? It has to store that the S3 event was unsuccessful and that it needs to replay it.

这个过程可以重新设计为事件驱动:lambda 耦合

这不仅是直接耦合方法引入的所有问题的解决方案，而且还提供了一种在每个 Lambda 发生错误时重放 DLQ 的方法。消息不会丢失，也无需外存，需求与处理解耦。