Pool timeout error

[neriumrevolta]

There appears to be an issue with the DB that happens roughly 20-30 minutes after the Radio is started/restarted. So far we've only noticed this happening in the Radio running in our kubernetes cluster, when the Radio is ran locally it works fine. Examples of the error:

1. In the GraphQL playground:

{
  "data": null,
  "errors": [
    {
      "message": "pool timed out while waiting for an open connection",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "queryActiveIndexers"
      ]
    }
  ]
}

2. In the logs:

  2024-03-12T09:51:33.917273Z  WARN listener_radio::operator: Error during pruning by retention, err: pool timed out while waiting for an open connection
    at src/operator/mod.rs:198

  2024-03-12T09:51:36.918274Z  INFO listener_radio::operator: Monitoring summary, total_messages: 0, total_num_pruned: 0

This seems to be related to the message pruning logic.