ssl_cert_reqs seems to not work

[knyghty]

I have:

RQ_QUEUES = {
    "default": {
        "DEFAULT_TIMEOUT": datetime.timedelta(hours=1).seconds,
        "URL": REDIS_URL,
    }
}

if REDIS_URL.startswith("rediss://"):  # pragma: no cover
    RQ_QUEUES["default"]["REDIS_CLIENT_KWARGS"] = {"ssl_cert_reqs": None}

I have checked:

>>> settings.RQ_QUEUES
{'default': {'DEFAULT_TIMEOUT': 3600, 'URL': 'rediss://:secret@esecret:11640', 'REDIS_CLIENT_KWARGS': {'ssl_cert_reqs': None}}}

But when I try to queue an email I get a traceback:

Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/connection.py", line 277, in connect
    sock = self.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/retry.py", line 62, in call_with_retry
    return do()
           ^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/connection.py", line 278, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/connection.py", line 748, in _connect
    sslsock = context.wrap_socket(sock, server_hostname=self.host)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/ssl.py", line 455, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/ssl.py", line 1042, in _create
    self.do_handshake()
  File "/app/.heroku/python/lib/python3.12/ssl.py", line 1320, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/.heroku/python/lib/python3.12/site-packages/django/core/mail/__init__.py", line 88, in send_mail
    return mail.send()
           ^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/django/core/mail/message.py", line 301, in send
    return self.get_connection(fail_silently).send_messages([self])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/django_rq_email_backend/backends.py", line 13, in send_messages
    return [send_email.delay(message, **kwargs) for message in email_messages]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/decorators.py", line 104, in delay
    return queue.enqueue_call(
           ^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/django_rq/queues.py", line 68, in enqueue_call
    return self.original_enqueue_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/django_rq/queues.py", line 64, in original_enqueue_call
    return super(DjangoRQ, self).enqueue_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/queue.py", line 719, in enqueue_call
    return self.enqueue_job(job, pipeline=pipeline, at_front=at_front)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/queue.py", line 1094, in enqueue_job
    return self._enqueue_job(job, pipeline=pipeline, at_front=at_front)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/queue.py", line 1114, in _enqueue_job
    job.redis_server_version = self.get_redis_server_version()
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/queue.py", line 208, in get_redis_server_version
    self.redis_server_version = get_version(self.connection)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/rq/utils.py", line 293, in get_version
    tuple(int(i) for i in str(connection.info("server")["redis_version"]).split('.')[:3]),
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/commands/core.py", line 996, in info
    return self.execute_command("INFO", section, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/client.py", line 545, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/connection.py", line 1074, in get_connection
    connection.connect()
  File "/app/.heroku/python/lib/python3.12/site-packages/redis/connection.py", line 283, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 1 connecting to secret:11640. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000).

I have my Django cache set up in the same way:

CACHES: dict[str, typing.Any] = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": REDIS_URL,
    }
}

if REDIS_URL.startswith("rediss://"):  # pragma: no cover
    CACHES["default"]["OPTIONS"] = {"ssl_cert_reqs": None}

And I confirmed this is working properly:

>>> cache.set("a", True)
>>> cache.get("a")
True

[EnochLucas]

This is a pretty urgent issue for heroku deployments. I was able to get it working by throwing a whole lot of guesses together, not sure which of these things in particular fixed the issue, but it was urgent, so I tried a whole lot at once.

1. I switched to using the REDIS_TLS_URL instead of the REDIS_URL (I think this is what fixed it for me, not 100% sure though)
2. I added this configuration to the queue:
   'SSL_CERT_REQS': 'none',
3. I added this configuration to the queue:

'CONNECTION_KWARGS': {  # Eventual additional Redis connection arguments
    'ssl': False
},

Altogether it looks something like this:

'default': {
    'URL': os.getenv('REDIS_TLS_URL'),
    'DEFAULT_TIMEOUT': 360,
    'SSL_CERT_REQS': 'none',
    'REDIS_CLIENT_KWARGS': {    # Eventual additional Redis connection arguments
        'ssl_cert_reqs': None,
    },
    'CONNECTION_KWARGS': {  # Eventual additional Redis connection arguments
        'ssl': False
    },
},

I'm not sure the last 2 are required you probably just need to use the REDIS_TLS_URL, but I thought I'd throw them in just to see if they would stick as this was urgent for us. I also wanted to make sure that I posted this in case you or anyone else is still having this issue.

[onAutopylot]

The documentation likely needs a small revision.

We need to update the RQ_QUEUES setting in settings.py. The documentation suggests the addition of 'REDIS_CLIENT_KWARGS':

{'default': {
        'HOST': 'localhost',
        'PORT': 6379,
        'DB': 0,
        'USERNAME': 'some-user',
        'PASSWORD': 'some-password',
        'DEFAULT_TIMEOUT': 360,
        'REDIS_CLIENT_KWARGS': {    # Eventual additional Redis connection arguments
            'ssl_cert_reqs': None,
        }
        },}

But actually, in version 2.10.2 the config key needed is "SSL_CERT_REQS":

def get_redis_connection(config, use_strict_redis=False):
    """
    Returns a redis connection from a connection config
    """
    redis_cls = redis.StrictRedis if use_strict_redis else redis.Redis

    if 'URL' in config:
        if config.get('SSL') or config.get('URL').startswith('rediss://'): # rediss:// for both REDIS_TLS_URL & REDIS_URL
            return redis_cls.from_url(
                config['URL'],
                db=config.get('DB'),
                ssl_cert_reqs=config.get('SSL_CERT_REQS', 'required'),

resulting in:

{'default': {
        'HOST': 'localhost',
        'PORT': 6379,
        'DB': 0,
        'USERNAME': 'some-user',
        'PASSWORD': 'some-password',
        'DEFAULT_TIMEOUT': 360,
        'REDIS_CLIENT_KWARGS': {    # Eventual additional Redis connection arguments
            'ssl_cert_reqs': None,
        },
        'SSL_CERT_REQS':'none'
        }
}

On the published app, Heroku sets both the environment variables REDIS_TLS_URL and REDIS_URL to the same value, so no update to settings is immediately required there, though going forward it's likely best to ensure REDIS_TLS_URL points to the REDIS_URL environment variable since they will likely depreciate the REDIS_TLS_URL environment variable :
REDIS_TLS_URL = os.environ["REDIS_URL"] # os.environ["REDIS_TLS_URL"]

Heroku users see also:

• https://help.heroku.com/HC0F8CUS/heroku-key-value-store-connection-issues
• https://help.heroku.com/45F0AY28/what-are-the-upcoming-changes-to-the-redis_url-config-var-for-heroku-key-value-store-mini-add-ons

[ildarlomov]

@onAutopylot thank you!