Installation fails on DO managed k8s

[r0fls]

Hi, I followed the setup guide here and I'm getting these errors https://us5.datadoghq.com/signup/agent#kubernetes:

2024-10-13 15:55:55 UTC | CORE | ERROR | (pkg/collector/worker/check_logger.go:71 in Error) | check:cilium | Error running check: [{"message":"HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdeea14d0>: Failed to establish a new connection: [Errno 111] Connection refused'))","traceback":"Traceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 196, in _new_conn\n    sock = connection.create_connection(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n    raise err\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n    sock.connect(sa)\nConnectionRefusedError: [Errno 111] Connection refused\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 789, in urlopen\n    response = self._make_request(\n               ^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 495, in _make_request\n    conn.request(\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 398, in request\n    self.endheaders()\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 1293, in endheaders\n    self._send_output(message_body, encode_chunked=encode_chunked)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 1052, in _send_output\n    self.send(msg)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 990, in send\n    self.connect()\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 236, in connect\n    self.sock = self._new_conn()\n                ^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 211, in _new_conn\n    raise NewConnectionError(\nurllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1cdeea14d0>: Failed to establish a new connection: [Errno 111] Connection refused\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/adapters.py\", line 667, in send\n    resp = conn.urlopen(\n           ^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 843, in urlopen\n    retries = retries.increment(\n              ^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/retry.py\", line 519, in increment\n    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]\n    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nurllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdeea14d0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/base.py\", line 1313, in run\n    self.check(instance)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/base_check.py\", line 142, in check\n    self.process(scraper_config)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 578, in process\n    for metric in self.scrape_metrics(scraper_config):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 505, in scrape_metrics\n    response = self.poll(scraper_config)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 842, in poll\n    response = self.send_request(endpoint, scraper_config, headers)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 868, in send_request\n    return http_handler.get(endpoint, stream=True, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 352, in get\n    return self._request('get', url, options)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 416, in _request\n    response = self.make_request_aia_chasing(request_method, method, url, new_options, persist)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 422, in make_request_aia_chasing\n    response = request_method(url, **new_options)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/api.py\", line 73, in get\n    return request(\"get\", url, params=params, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/api.py\", line 59, in request\n    return session.request(method=method, url=url, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/sessions.py\", line 589, in request\n    resp = self.send(prep, **send_kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/sessions.py\", line 703, in send\n    r = adapter.send(request, **kwargs)\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/adapters.py\", line 700, in send\n    raise ConnectionError(e, request=request)\nrequests.exceptions.ConnectionError: HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdeea14d0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"}]                                                                                                                    
2024-10-13 15:56:05 UTC | CORE | WARN | (pkg/logs/client/http/destination.go:336 in unconditionalSend) | failed to post http payload. code=403 host=contlcycle-intake.datadoghq.com response={"errors":[{"status":"403","title":"Forbidden","detail":"Api key is either forbidden or blacklisted"}]}                                                                      
2024-10-13 15:56:05 UTC | CORE | WARN | (pkg/logs/client/http/destination.go:245 in sendAndRetry) | Could not send payload: client error                                             
2024-10-13 15:56:10 UTC | CORE | ERROR | (pkg/collector/worker/check_logger.go:71 in Error) | check:cilium | Error running check: [{"message":"HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdee8e810>: Failed to establish a new connection: [Errno 111] Connection refused'))","traceback":"Traceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 196, in _new_conn\n    sock = connection.create_connection(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n    raise err\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n    sock.connect(sa)\nConnectionRefusedError: [Errno 111] Connection refused\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 789, in urlopen\n    response = self._make_request(\n               ^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 495, in _make_request\n    conn.request(\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 398, in request\n    self.endheaders()\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 1293, in endheaders\n    self._send_output(message_body, encode_chunked=encode_chunked)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 1052, in _send_output\n    self.send(msg)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/http/client.py\", line 990, in send\n    self.connect()\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 236, in connect\n    self.sock = self._new_conn()\n                ^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connection.py\", line 211, in _new_conn\n    raise NewConnectionError(\nurllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1cdee8e810>: Failed to establish a new connection: [Errno 111] Connection refused\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/adapters.py\", line 667, in send\n    resp = conn.urlopen(\n           ^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/connectionpool.py\", line 843, in urlopen\n    retries = retries.increment(\n              ^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/urllib3/util/retry.py\", line 519, in increment\n    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]\n    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nurllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdee8e810>: Failed to establish a new connection: [Errno 111] Connection refused'))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/base.py\", line 1313, in run\n    self.check(instance)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/base_check.py\", line 142, in check\n    self.process(scraper_config)\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 578, in process\n    for metric in self.scrape_metrics(scraper_config):\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 505, in scrape_metrics\n    response = self.poll(scraper_config)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 842, in poll\n    response = self.send_request(endpoint, scraper_config, headers)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/checks/openmetrics/mixins.py\", line 868, in send_request\n    return http_handler.get(endpoint, stream=True, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 352, in get\n    return self._request('get', url, options)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 416, in _request\n    response = self.make_request_aia_chasing(request_method, method, url, new_options, persist)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/datadog_checks/base/utils/http.py\", line 422, in make_request_aia_chasing\n    response = request_method(url, **new_options)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/api.py\", line 73, in get\n    return request(\"get\", url, params=params, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/api.py\", line 59, in request\n    return session.request(method=method, url=url, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/sessions.py\", line 589, in request\n    resp = self.send(prep, **send_kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/sessions.py\", line 703, in send\n    r = adapter.send(request, **kwargs)\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/datadog-agent/embedded/lib/python3.11/site-packages/requests/adapters.py\", line 700, in send\n    raise ConnectionError(e, request=request)\nrequests.exceptions.ConnectionError: HTTPConnectionPool(host='10.138.32.241', port=9962): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1cdee8e810>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"}]    

I use DD for work profesionally and was going to use it for a hobby project on Digital Ocean, but I'm not sure if it's unsupported? Anyway, I will probably be uninstalling it since I was hoping it would be easier than using Grafana's OS solutions (since those are entirely free), but it doesn't look like it.