处理redis连接重置

我们在我们的应用程序中非常随机地面对 [Errno 54] Connection reset by peer 并且看起来它是由 redis 服务器而不是客户端触发的。 Python 的 redis 客户端有 backoff 策略实现,但它无法处理这种情况。

官方 repo 上也有 github 问题,最近很多人评论证实了这个问题。

步骤重现

$ ipython
# in python client
import redis
from redis.retry import Retry
from redis.exceptions import (TimeoutError, ConnectionError)
from redis.backoff import ExponentialBackoff

# connect client with exponential backoff retry
client = redis.StrictRedis(retry=Retry(ExponentialBackoff(cap=10, base=1), 25), retry_on_error=[ConnectionError, TimeoutError, ConnectionResetError], health_check_interval=1)

client.keys()
# print all keys

现在直接从redis服务器重置连接

$ redis-cli
RESET

等待 120 秒或更长时间,然后再次运行客户端

# in client
client.keys()
---------------------------------------------------------------------------
ConnectionResetError                      Traceback (most recent call last)
<ipython-input-91-011ce9f936fc> in <module>
----> 1 client.keys("rq*")

~/path-to-python-env/env/lib/python3.8/site-packages/redis/commands/core.py in keys(self, pattern, **kwargs)
   1386         For more information check https://redis.io/commands/keys
   1387         """
-> 1388         return self.execute_command("KEYS", pattern, **kwargs)
   1389 
   1390     def lmove(self, first_list, second_list, src="LEFT", dest="RIGHT"):

~/path-to-python-env/env/lib/python3.8/site-packages/redis/client.py in execute_command(self, *args, **options)
   1168         pool = self.connection_pool
   1169         command_name = args[0]
-> 1170         conn = self.connection or pool.get_connection(command_name, **options)
   1171 
   1172         try:

~/path-to-python-env/env/lib/python3.8/site-packages/redis/connection.py in get_connection(self, command_name, *keys, **options)
   1315             # closed. either way, reconnect and verify everything is good.
   1316             try:
-> 1317                 if connection.can_read():
   1318                     raise ConnectionError("Connection has data")
   1319             except ConnectionError:

~/path-to-python-env/env/lib/python3.8/site-packages/redis/connection.py in can_read(self, timeout)
    793         if not sock:
    794             self.connect()
--> 795         return self._parser.can_read(timeout)
    796 
    797     def read_response(self, disable_decoding=False):

~/path-to-python-env/env/lib/python3.8/site-packages/redis/connection.py in can_read(self, timeout)
    315 
    316     def can_read(self, timeout):
--> 317         return self._buffer and self._buffer.can_read(timeout)
    318 
    319     def read_response(self, disable_decoding=False):

~/path-to-python-env/env/lib/python3.8/site-packages/redis/connection.py in can_read(self, timeout)
    222 
    223     def can_read(self, timeout):
--> 224         return bool(self.length) or self._read_from_socket(
    225             timeout=timeout, raise_on_timeout=False
    226         )

~/path-to-python-env/env/lib/python3.8/site-packages/redis/connection.py in _read_from_socket(self, length, timeout, raise_on_timeout)
    192                 sock.settimeout(timeout)
    193             while True:
--> 194                 data = self._sock.recv(socket_read_size)
    195                 # an empty string indicates the server shutdown the socket
    196                 if isinstance(data, bytes) and len(data) == 0:

ConnectionResetError: [Errno 54] Connection reset by peer

第二次运行它可以完美运行,尽管它应该通过重试策略来处理。

client.keys()
# prints all keys

配置

redis server - 6.2.6
python redis - 4.1.0

我们可以围绕 redis 客户端编写自己的 try/catch,但我们正在使用一些库,例如 rqflask-cache ,它们在内部使用 redis,并且没有修改其流程的接口。

任何帮助深表感谢。

stack overflow Handle redis connection reset
原文答案

答案:

作者头像

尝试使用连接池,因为它将保持活动状态并自动尝试在后台恢复。在 REPL

import redis
pool = redis.ConnectionPool(host="localhost", port=6379)
r = redis.Redis(connection_pool=pool)
r.ping()

在另一个 shell 中,您可以通过运行 redis-cli 进行测试

client LIST
client kill IP:PORT

然后运行

r.ping

你应该会在 redis-cli 中看到一个新的连接

client LIST
作者头像

您还可以设置健康检查值,
client = redis.Redis(..., health_check_interval=30)
它将重新建立连接。 https://github.com/redis/redis-py/issues/1186 中的更多详细信息