python - RQ : redis. 异常。ResponseError : Command # 3 . .. 管道导致错误:使用内存时不允许 OOM 命令 > 'maxmemory'

Question

我正在使用 RQ 在我的 django 应用程序中运行后台任务，因为它应该是完成工作的最简单方法之一。该任务包括检查某些 API(如果有任何信息已更新)并将任何新信息插入我自己的数据库。

直到几天前，它工作正常，但我现在遇到无法解决的错误。

来自错误消息的最后几行:

使用内存时不允许 OOM 命令 > 'maxmemory'

我一开始以为我传递了太多数据给工作人员。但是，我最终减少了传递给具有 5 个键值对的单个字典的数据，但我仍然收到错误消息(请参阅底部的完整消息)。然而，直到上周，我才通​​过了 20 多部词典，每部词典都有更多的元素，而且工作正常。

我检查了here和 here , 但它似乎与我的问题不同。

知道我为什么会收到此错误以及如何解决它吗？

---

错误信息:

Internal Server Error: /en/results/
Traceback (most recent call last):
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2518, in _execute_transaction
    response = self.parse_response(connection, '_')
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2584, in parse_response
    self, connection, command_name, **options)
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/my/path/lib/python3.5/site-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.ExecAbortError: Transaction discarded because of previous errors.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/my/path/lib/python3.5/site-packages/django/core/handlers/base.py", line 149, in get_response
    response = self.process_exception_by_middleware(e, request)
  File "/my/path/lib/python3.5/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/my/path/app/results/views.py", line 259, in results_list
    price_update_to_queue(outbound_trips)
  File "/my/path/app/results/views.py", line 331, in price_update_to_queue
    queue.enqueue(update_prices, queries)
  File "/my/path/lib/python3.5/site-packages/rq/queue.py", line 274, in enqueue
    job_id=job_id, at_front=at_front, meta=meta)
  File "/my/path/lib/python3.5/site-packages/django_rq/queues.py", line 60, in enqueue_call
    return self.original_enqueue_call(*args, **kwargs)
  File "/my/path/lib/python3.5/site-packages/django_rq/queues.py", line 56, in original_enqueue_call
    return super(DjangoRQ, self).enqueue_call(*args, **kwargs)
  File "/my/path/lib/python3.5/site-packages/rq/queue.py", line 227, in enqueue_call
    job = self.enqueue_job(job, at_front=at_front)
  File "/my/path/lib/python3.5/site-packages/rq/queue.py", line 298, in enqueue_job
    pipe.execute()
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2626, in execute
    return execute(conn, stack, raise_on_error)
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2523, in _execute_transaction
    raise errors[0][1]
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2510, in _execute_transaction
    self.parse_response(connection, '_')
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 2584, in parse_response
    self, connection, command_name, **options)
  File "/my/path/lib/python3.5/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/my/path/lib/python3.5/site-packages/redis/connection.py", line 582, in read_response
    raise response
redis.exceptions.ResponseError: Command # 3 (HMSET b'rq:job:8df23896-d52d-4585-aa60-9f5f9a39292a' created_at 2017-06-26T13:13:33Z timeout 500 origin default data b'\x80\x04\x95\xff\x00\x00\x00\x00\x00\x00\x00(\x8c\x1bresults.tasks.update_prices\x94N]\x94}\x94(\x8c\x03arr\x94\x8c\x0bDE-CGN-00-0\x94\x8c\x03dep\x94\x8c\x0bDE-BER-02-0\x94\x8c\x03rtn\x94K\x00\x8c\x06seller\x94\x8c\x04KIWI\x94\x8c\x06dep_dt\x94\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xe1\x07\x18\x00\x00\x00\x00\x00\x00\x94\x8c\x0bpsycopg2.tz\x94\x8c\x13FixedOffsetTimezone\x94\x93\x94KxN\x86\x94R\x94}\x94\x8c\x07_offset\x94\x8c\x08datetime\x94\x8c\ttimedelta\x94\x93\x94K\x00M \x1cK\x00\x87\x94R\x94sb\x86\x94R\x94ua\x85\x94}\x94t\x94.' description results.tasks.update_prices([{'arr': 'DE-CGN-00-0', 'dep': 'DE-BER-02-0', 'rtn': 0, 'seller': 'KIWI', 'dep_dt': datetime.datetime(2017, 7, 24, 0, 0, tzinfo=psycopg2.tz.FixedOffsetTimezone(offset=120, name=None))}]) enqueued_at 2017-06-26T13:13:33Z status queued) of pipeline caused error: OOM command not allowed when used memory > 'maxmemory'.
[26/Jun/2017 13:13:33] "GET /en/results/?From=Berlin&To=Cologne&OutboundDate=24-07-2017&ReturnDate= HTTP/1.1" 500 174583

这就是我传递任务的方式

#views.py
import django_rq
from .tasks import update_prices

...
queue = django_rq.get_queue('default')
queue.enqueue(update_prices, queries)

和

#tasks.py
from django_rq import job

@job
def update_prices(queries):

    ...
    return queries_to_background(queries)

Answer 1

之前遇到过类似的问题。我的 Flask 应用程序部署到 Heroku 上，我在 Heroku 上使用 redistogo，免费计划的内存非常少，这导致我出现超出最大内存的错误。然后切换到heroku-redis，问题解决。