python - 如何在硬超时时重试 celery 任务？

Question

我有一个 celery 任务如下:

@celery.task(name='tasks.ht_run', default_retry_delay=30, max_retries=15, time_limit=1)  
def ht_run(str_command):
    try:
        f = os.popen(str_command)
        output = f.read()
        if output == '':
            raise Exception
    except Exception as exc:
        raise ht_run.retry(exc=exc)

    return output.split('\n')

它的名字是这样的: appserver.ht_run.delay(字符串)

虽然我希望它在超时时重试，但如果失败了。在 Celery 窗口中，我收到以下错误:

[2014-12-10 11:50:22,128: ERROR/MainProcess] Task tasks.ht_run[6a83793a-8bd6-47fc-bf74-0b673bf961f2] raised unexpected: TimeLimitExceeded(1,)
Traceback (most recent call last):
  File "/Users/andy.terhune/cgenv/lib/python2.6/site-packages/billiard/pool.py", line 639, in on_hard_timeout
    raise TimeLimitExceeded(job._timeout)
TimeLimitExceeded: TimeLimitExceeded(1,)
[2014-12-10 11:50:22,128: ERROR/MainProcess] Hard time limit (1s) exceeded for tasks.ht_run[6a83793a-8bd6-47fc-bf74-0b673bf961f2]
[2014-12-10 11:50:23,644: ERROR/MainProcess] Process 'Worker-28' pid:2081 exited with 'signal 9 (SIGKILL)'

我怎样才能让它超时并在超时时重试？

Answer 1

感谢 user2097159，

尝试了一个软超时并且工作得很好，

@celery.task(
    name='tasks.ht_run', 
    default_retry_delay=30,
    max_retries=15,
    soft_time_limit=1)
def ht_run(str_command):
    try:
        f = os.popen(str_command)
        output = f.read()
        if output == '':
            raise Exception
    except Exception as exc:
        raise ht_run.retry(exc=exc)

    return output.split('\n')

...

[2014-12-10 12:16:55,580: WARNING/MainProcess] Soft time limit (1s) exceeded for tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9]
[2014-12-10 12:16:55,603: INFO/MainProcess] Received task: tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9] eta:[2014-12-10 18:17:25.593613+00:00]
[2014-12-10 12:16:55,603: DEBUG/MainProcess] basic.qos: prefetch_count->121
[2014-12-10 12:16:56,897: INFO/MainProcess] Task tasks.ht_run[f31bbd15-e755-440c-9261-cd0864ceb3a9] retry: Retry in 30s: SoftTimeLimitExceeded()