New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PubSub error when running in threads with redis-py 5.0.3 #3184
Comments
Running into a similar issue with Celery/Redis. Downgrading to 4.6.0 didn't help. For me, the issue triggers when I try to get the results of celery tasks Stack trace when gathering celery task results. results = await asyncio.gather(*[asyncio.to_thread(task.get) for task in tasks])
File "/usr/local/lib/python3.9/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/result.py", line 251, in get
return self.backend.wait_for_pending(
File "/usr/local/lib/python3.9/site-packages/celery/backends/asynchronous.py", line 221, in wait_for_pending
for _ in self._wait_for_pending(result, **kwargs):
File "/usr/local/lib/python3.9/site-packages/celery/backends/asynchronous.py", line 287, in _wait_for_pending
for _ in self.drain_events_until(
File "/usr/local/lib/python3.9/site-packages/celery/backends/asynchronous.py", line 54, in drain_events_until
yield self.wait_for(p, wait, timeout=interval)
File "/usr/local/lib/python3.9/site-packages/celery/backends/asynchronous.py", line 63, in wait_for
wait(timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/celery/backends/redis.py", line 161, in drain_events
message = self._pubsub.get_message(timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1690, in get_message
response = self.parse_response(block=(timeout is None), timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1542, in parse_response
response = self._execute(conn, try_read)
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1518, in _execute
return conn.retry.call_with_retry(
File "/usr/local/lib/python3.9/site-packages/redis/retry.py", line 46, in call_with_retry
return do()
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1519, in <lambda>
lambda: command(*args, **kwargs),
File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1536, in try_read
if not conn.can_read(timeout=timeout):
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 869, in can_read
return self._parser.can_read(timeout)
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 344, in can_read
return self._buffer and self._buffer.can_read(timeout)
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 242, in can_read
return bool(self.unread_bytes()) or self._read_from_socket(
File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 237, in _read_from_socket
buf.seek(current_pos)
ValueError: I/O operation on closed file. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
pubsub.py:
In redis-py 4.6.0 it works as expected:
With redis-py 5.0.3 it constantly fails with:
Tested in Python3.8 and Python3.11, same result.
The text was updated successfully, but these errors were encountered: