python - 应用程序引擎(gae)、python、ndb、延迟库或任务链、cron 作业还是全部？

Question

我不太熟悉任务链、拉队列或推送队列、cron 作业、延迟库等。但我知道我需要使用其中之一来完成手头的任务。我不确定最好的方法是什么。

我有一个 ndb.Model ，其某个属性需要更新。

我将有成千上万个需要更新的 ndb.Model 实例/实体，实际上每个实例上都有相同的属性。

此属性不需要经常更新，因为最终用户不会访问它。

我研究过的解决方案:

• Map reduce 似乎有点矫枉过正，我想我读过一些关于它需要 cvs 文件的内容，这让我不敢相信。

• 似乎任务链与 cron 作业相结合可以解决这个问题，但是..我不知道这是否可能。我对这两者都是新手，需要一些确认。

我已经了解了延迟库，这是最好的选择吗？

Answer 1

cron 作业本身实际上是一个计划任务，所以是的，cron + 任务队列是可能的:)

嗯，浏览文档以获得更好的想法总是好的:

• Scheduled Tasks
• Task Queues
• deferred library

在 this article有一个指导说明也可能有帮助:

When to use ext.deferred

You may be wondering when to use ext.deferred, and when to stick with the built-in task queue API. Here are our suggestions.

You may want to use the deferred library if:

• You only use the task queue lightly.
• You want to refactor existing code to run on the Task Queue with a minimum of changes.
• You're writing a one off maintenance task, such as schema migration.
• Your app has many different types of background tasks, and writing a separate handler for each would be burdensome.
• Your task requires complex arguments that aren't easily serialized without using Pickle.
• You are writing a library for other apps that needs to do background work.

You may want to use the Task Queue API if:

• You need complete control over how tasks are queued and executed.
• You need better queue management or monitoring than deferred provides.
• You have high throughput, and overhead is important.
• You are building larger abstractions and need direct control over tasks.
• You like the webhook model better than the RPC model.

Naturally, you can use both the Task Queue API and the deferred library side-by-side, if your app has requirements that fit into both groups.

Google App Engine Pipeline API (实际上是mapreduce + 任务队列)在this article中有很好的描述。 :

The Google App Engine Pipeline API connects together complex, workflows (including human tasks). The goals are flexibility, workflow reuse, and testability.