Cannot backup local PV using Opt-In

[Sapp00]

What steps did you take and what happened:
I have a local persistent volume that I want to backup using the opt-in approach. When I run velero backup create, it says "no applicable volumesnapshotter found".
Velero is installed using helm and the backup is using a schedule.

Helm values:

---
resources:
  requests:
    cpu: 500m
    memory: 128Mi
  limits:
    cpu: 1000m
    memory: 512Mi
dnsPolicy: Default
initContainers:
  - name: velero-plugin-for-aws
    image: velero/velero-plugin-for-aws:v1.5.2
    imagePullPolicy: IfNotPresent
    volumeMounts:
      - mountPath: /target
        name: plugins

podSecurityContext: {}
containerSecurityContext: {}
dnsConfig:
  nameservers:
    - 10.10.10.1

metrics:
  enabled: true
  scrapeInterval: 30s
  scrapeTimeout: 10s

  podAnnotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "8085"
    prometheus.io/path: "/metrics"

upgradeCRDs: true
cleanUpCRDs: false

configuration:
  backupStorageLocation:
    - name: default
      provider: aws
      bucket: velero-backup
      accessMode: ReadWrite
      config:
        region: minio
        s3ForcePathStyle: true
        s3Url: http://minio.local

  volumeSnapshotLocation:
    - name: default
      provider: aws
      config:
        region: minio

  uploaderType: restic
  backupSyncPeriod: 1h
  fsBackupTimeout: 4h
  defaultBackupStorageLocation: default
  defaultBackupTTL: 72h
  logLevel: info
  metricsAddress: :8085
  namespace: velero
  defaultVolumesToFsBackup: false

rbac:
  create: true

serviceAccount:
  server:
    create: true
    name:
    annotations:
    labels:

credentials:
  useSecret: true
  existingSecret: minio-creds

backupsEnabled: true
snapshotsEnabled: true

deployNodeAgent: true

nodeAgent:
  podVolumePath: /var/lib/kubelet/pods
  privileged: false
  resources:
    requests:
      cpu: 500m
      memory: 512Mi
    limits:
      cpu: 1000m
      memory: 1024Mi

Schedule:

---
apiVersion: velero.io/v1
kind: Schedule
metadata:
  creationTimestamp: null
  name: hassio-schedule
  namespace: velero
spec:
  schedule: '@every 48h'
  template:
    csiSnapshotTimeout: 0s
    hooks: {}
    includedNamespaces:
      - homeassistant
    includedResources:
      - pvc
      - pv
    excludedNamespaces:
    excludedResources:
    metadata: {}
    ttl: 2160h0m0s
    storageLocation: default
  useOwnerReferencesInBackup: false

I then run it using velero backup create --from-schedule hassio-schedule and it completes "successfully".

The respective PV is used by a deployment:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: homeassist-deployment
  labels:
    app.kubernetes.io/name: homeassistant
    app.kubernetes.io/version: "2024.9"
    app.kubernetes.io/component: server
    app.kubernetes.io/part-of: homeassistant
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: homeassistant
  template:
    metadata:
      labels:
        app.kubernetes.io/name: homeassistant
        app.kubernetes.io/version: "2024.9"
        app.kubernetes.io/component: server
        app.kubernetes.io/part-of: homeassistant
      annotations:
        backup.velero.io/backup-volumes: homeassist-data
    spec:
[...]

and the PVC is also annotated:

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: null
  labels:
    app: homeassist
  name: homeassist-pvc
  annotations:
    backup.velero.io/backup-volumes: homeassist-data
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

PV:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  creationTimestamp: null
  labels:
    app: homeassist
    type: local
  name: homeassist-data
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  local:
    path: "/var/local/hassio"

(other question: do I need to annotate PVC and pod or is one sufficient?)

What did you expect to happen:

The full backup should be performed. When I restore, I only see the PVC being created without any content. In the S3 bucket, there is also nothing in the restic directory.

The following information will help us better understand what's going on:

Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

• kubectl logs deployment/velero -n velero https://pastebin.com/r8c0XC0d
• velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml: https://pastebin.com/qzv5e3Ad
• velero backup logs <backupname>: https://pastebin.com/keGPj6GK

Environment:

• Velero version (use velero version): v1.10.1 (client) / v1.14.1 (server)
• Velero features (use velero client config get features):
• Kubernetes version (use kubectl version): v1.30.3
• Kubernetes installer & version: using Talos
• Cloud provider or hardware configuration: Virtualized Talos Linux
• OS (e.g. from /etc/os-release): Talos v1.7.6

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"

[ywk253100]

You need to annotate the Pods rather than the PVCs, refer to https://velero.io/docs/v1.14/file-system-backup/#using-opt-in-pod-volume-backup