gpt4 book ai didi

Using Celery tasks in Django to update aggregated data(使用Django中的芹菜任务更新聚合数据)

转载 作者:bug小助手 更新时间:2023-10-25 09:15:47 26 4
gpt4 key购买 nike



In my Django project, I have a use case where I have some complex Python functions which are computationally expensive: they involve numerous calculations based on a recurrence relation. They are already well optimized but still take a few seconds to run in some cases.

在我的Django项目中,我有一个用例,其中我有一些复杂的Python函数,它们的计算代价很高:它们涉及基于递归关系的大量计算。它们已经得到了很好的优化,但在某些情况下仍需要几秒钟才能运行。


In this context, I'm thinking about using Celery tasks to carry out the calculations and to update aggregated data in dedicated tables of my PostgreSQL database. However, I have a feeling it's not going to be easy to ensure the integrity of the data and to avoid unnecessary calculations. Race conditions and unreliable error handling / retry mechanisms are some of the pitfalls I have in mind.

在这种情况下,我正在考虑使用芹菜任务来执行计算并更新我的PostgreSQL数据库的专用表中的聚合数据。然而,我有一种感觉,要确保数据的完整性并避免不必要的计算并不容易。竞争条件和不可靠的错误处理/重试机制是我考虑的一些陷阱。


What are the best practices in this case? What do I need to pay particular attention to?

这种情况下的最佳实践是什么?我需要特别注意什么?


Are you aware of open source Django projects doing something similar?

您是否知道开源Django项目也在做类似的事情?


I'm using Django 4.2, PostgreSQL 15, Redis 7.2 (as a cache), Celery 5.3, RabbitMQ 3.12 (as a broker).

我使用Django 4.2、PostgreSQL 15、Redis 7.2(作为缓存)、Celery 5.3、RabbitMQ 3.12(作为代理)。


更多回答

I can't really understand what your problem is. Maybe a simple transaction can solve your case? docs.djangoproject.com/en/4.2/topics/db/transactions

我真的不明白你的问题是什么。也许一笔简单的交易就能解决你的案子?Docs.djangoproject.com/en/4.2/topics/db/transactions

Indeed, transaction.atomic is a good example of the kind of good habits I'm looking for. I'm simply looking for best practices like that one. I have to admit my question is more theoretical than what's used to be found on SO, but I would like to see some elegant pieces of code & concrete examples tackling that kind of issue, to be sure not to forget anything.

事实上,Transaction是我正在寻找的那种好习惯的一个很好的例子。我只是在寻找这样的最佳实践。我必须承认,我的问题比过去在上面找到的问题更具理论性,但我希望看到一些优雅的代码片段和解决这类问题的具体示例,以确保不会忘记任何东西。

优秀答案推荐
更多回答

26 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com