In my Django project, I have a use case where I have some complex Python functions which are computationally expensive: they involve numerous calculations based on a recurrence relation. They are already well optimized but still take a few seconds to run in some cases.
在我的Django项目中,我有一个用例,其中我有一些复杂的Python函数,它们的计算代价很高:它们涉及基于递归关系的大量计算。它们已经得到了很好的优化,但在某些情况下仍需要几秒钟才能运行。
In this context, I'm thinking about using Celery tasks to carry out the calculations and to update aggregated data in dedicated tables of my PostgreSQL database. However, I have a feeling it's not going to be easy to ensure the integrity of the data and to avoid unnecessary calculations. Race conditions and unreliable error handling / retry mechanisms are some of the pitfalls I have in mind.
在这种情况下,我正在考虑使用芹菜任务来执行计算并更新我的PostgreSQL数据库的专用表中的聚合数据。然而,我有一种感觉,要确保数据的完整性并避免不必要的计算并不容易。竞争条件和不可靠的错误处理/重试机制是我考虑的一些陷阱。
What are the best practices in this case? What do I need to pay particular attention to?
这种情况下的最佳实践是什么?我需要特别注意什么?
Are you aware of open source Django projects doing something similar?
您是否知道开源Django项目也在做类似的事情?
I'm using Django 4.2, PostgreSQL 15, Redis 7.2 (as a cache), Celery 5.3, RabbitMQ 3.12 (as a broker).
我使用Django 4.2、PostgreSQL 15、Redis 7.2(作为缓存)、Celery 5.3、RabbitMQ 3.12(作为代理)。
更多回答
Indeed, transaction.atomic
is a good example of the kind of good habits I'm looking for. I'm simply looking for best practices like that one. I have to admit my question is more theoretical than what's used to be found on SO, but I would like to see some elegant pieces of code & concrete examples tackling that kind of issue, to be sure not to forget anything.
事实上,Transaction是我正在寻找的那种好习惯的一个很好的例子。我只是在寻找这样的最佳实践。我必须承认,我的问题比过去在上面找到的问题更具理论性,但我希望看到一些优雅的代码片段和解决这类问题的具体示例,以确保不会忘记任何东西。
我是一名优秀的程序员,十分优秀!