Galaxy with CVMFS seems to work on newer kubernetes

[cyrilcros]

**** This isn't a bug, I just want to point out that CVMFS seems to work better now.... ***

Hi, I saw this pull request cvmfs-contrib/cvmfs-csi#35 on the cernops/cvmfs-csi and decided to try it. I added a few edits in a new_provisioner branch https://github.com/cyrilcros/cvmfs-csi/blob/new_provisioner/deployments/helm/cvmfs-csi/values.yaml

That's been working well for me, notably because it has a different csi-cvmfsplugin image that solves some issues (cvmfs-contrib/cvmfs-csi#26). I just added some quick other changes like in this pull request https://github.com/cernops/cvmfs-csi/pull/22/files , or removing the attach component as in cvmfs-contrib/cvmfs-csi#29 .

I am getting 504 timeout errors from nginx I am now trying to fix, but now the PVCs are created as they should on Kubernetes 1.21...

[almahmoud]

Thank you for pointing out that PR!!
I'll work on porting the new version on our end to have Galaxy's CVMFS work post-1.19.

[nuwang]

This is super helpful, thanks @cyrilcros. It still looks like the repo is generally behind on maintenance, and the last merged PR was in May, 2020. Perhaps we need to maintain a fork ourselves at https://github.com/CloudVE/cvmfs-csi that will generally work for this helm chart. Another, longer term alternative that was contemplated was to sync all the CVMFS data over to S3 and to switch to S3FS as an alternative using something like: https://github.com/ctrox/csi-s3.

[cyrilcros]

Yes, from what I have seen the Helm chart needed just minor work, but the CSI plugin itself needs an update. The fbarreir/csi-cvmfsplugin:9 Docker image (from this guy I assume: Fernando Barreiro ) seems to use a modernized version, ideally you may get the source code from him. I don't see it on his public Github....

[ksuderman]

@cyrilcros I am trying to get all of these changes merged into our helm charts, but the fbarreir/csi-cvmfsplugin:9 Docker image does not seem to exist on Docker Hub or quay.io and I can not find any contact information for Fernando Barreiro. I was hoping the brinkmanlab/csi-cvmfsplugin:1.2.0 Docker image would be a drop in replacement. The PVCa and PVa are created, but I get the following error when launching Galaxy:

MountVolume.MountDevice failed for volume "pvc-2f420977-f878-4e0b-ae6a-95c387b4c426" : rpc error: code = Internal desc = mount failed: exit status 32 Mounting command: mount Mounting arguments: -t cvmfs -o config=/etc/cvmfs/config-csi-cvmfs-pvc-2f420977-f878-4e0b-ae6a-95c387b4c426-a48189e8-d068-11ec-a3a6-fa163e3c7d6b data.galaxyproject.org /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2f420977-f878-4e0b-ae6a-95c387b4c426/globalmount Output: CernVM-FS: running with credentials 999:997 CernVM-FS: loading Fuse module... Failed to fetch manifest (22 - history init failure)

Is this still working for you?