Failed to retrieve item: status 408: sync did not complete within timeout, please retry the request

[ankycooper]

Your environment

Operator Version: 1.6.0

Connect Server Version: 1.5.7

Kubernetes Version: v1.27.11+k3s1

What happened?

Tried to get test creds via following yaml
apiVersion: onepassword.com/v1 kind: OnePasswordItem metadata: name: test spec: itemPath: "vaults/Systems/items/bi-ha"

What did you expect to happen?

Get secrets as they are present in 1passord on the above path

Steps to reproduce

1. install via helm and apply the yaml

Notes & Logs

1.7107159215368934e+09 INFO controller_onepassworditem Reconciling OnePasswordItem {"Request.Namespace": "default", "Request.Name": "test"}
1.7107159315644588e+09 ERROR Reconciler error {"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "OnePasswordItem": {"name":"test","namespace":"default"}, "namespace": "default", "name": "test", "reconcileID": "e13a0d04-cff1-4b60-a00a-641f53ac5ea5", "error": "Failed to retrieve item: status 408: sync did not complete within timeout, please retry the request"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:326
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234

[ankycooper]

Tried lot of different combinations

Diff versions of K3s -- same issue
Diff OS (ubuntu-22.04/debian-12) -- same issue
Disabled app armour -- same issue
single node/multi node k3s -- same issue

Tried rancher desktop -- works

Install k3s on any os using curl -sfL https://get.k3s.io | K3S_TOKEN={{ k3s_token }} sh -

install the operator helm with OP creds and try and retrieve any secrets.

[ankycooper]

1Password/connect-helm-charts#124

so far it does not work on "debian/ubuntu" but works on "amazon/fedora"