From b88c3f87c034a67ffdf997d58ded7afe7079a833 Mon Sep 17 00:00:00 2001 From: Michael Bunsen Date: Fri, 17 Apr 2026 00:04:19 -0700 Subject: [PATCH] fix(deps): pin redis-py 5.3.1 to avoid ConnectionPool deadlock MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit redis-py 5.2.1 has a self-deadlock in `ConnectionPool.release()` where `PubSub.__del__` triggered by GC while the pool holds its `_lock` blocks forever trying to re-acquire the same non-reentrant lock. Fixed upstream in redis-py 5.3.0 by switching to an `RLock` (redis-py#3677), regressed in 6.x (celery/celery#9622). Observed symptom: celery beat stops scheduling all tasks with no error, no crash, no log output. Main thread parked on `futex_wait_queue` inside `ConnectionPool.release()`. Captured via py-spy on a hung beat container. Stack at hang: tick → apply_async → send_task → on_task_call (celery/backends/redis.py: Redis result backend subscribes to the result key via PubSub for every apply_async) ResultConsumer.consume_from → hits ConnectionError on a stale pubsub reconnect_on_error → retry_over_time → _reconnect_pubsub mget → get_connection → make_connection → Connection.__init__ ← GC fires PubSub.__del__ mid-__init__ PubSub.reset → ConnectionPool.release → stuck on self._lock Pinning 5.3.1 (not latest 6.x/7.x) is deliberate: per the thread on celery/celery#9622, redis-py 6+ ships a newer ConnectionPool implementation that reintroduces the same class of issue. Co-Authored-By: Claude Opus 4.7 (1M context) --- requirements/base.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements/base.txt b/requirements/base.txt index 037eeea17..089a3d80a 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -2,7 +2,7 @@ python-slugify==8.0.1 # https://github.com/un33k/python-slugify Pillow==10.2.0 # https://github.com/python-pillow/Pillow argon2-cffi==21.3.0 # https://github.com/hynek/argon2_cffi whitenoise==6.5.0 # https://github.com/evansd/whitenoise -redis==5.2.1 # https://github.com/redis/redis-py +redis==5.3.1 # https://github.com/redis/redis-py — pin 5.3.x; 5.2.x deadlocks in ConnectionPool.release (redis-py#3677), 6.x reintroduces it (celery#9622) hiredis==2.2.3 # https://github.com/redis/hiredis-py celery==5.4.0 # pyup: < 6.0 # https://github.com/celery/celery django-celery-beat==2.5.0 # https://github.com/celery/django-celery-beat