Issues with bput

[jnimoth]

A user of ours experiences issues when trying to upload with gocmd bput. I tried myself today and also got a lot of errors when trying bput command.

He a simple example:

$ ll bput_test
total 3.4G
-rw-rw-r-- 1 jelte jelte 954M Dec 16 14:33 1GB_Testfile
-rw-rw-r-- 1 jelte jelte 1.9G Dec 16 14:33 2GB_Testfile
-rw-rw-r-- 1 jelte jelte 477M Dec 16 14:33 500MB_Testfile
-rw-rw-r-- 1 jelte jelte  48M Dec 16 14:33 50MB_Testfile

Trying to upload via gocmd bput -d --min_file_num 1 bput_test/ gives an error (see attached log bput.log).

I also end up with a non-good replica in .gocmd_staging as well as parts of the files being transferred:

$ ils -L bput_test
/testrug/home/[email protected]/bput_test:
  rdms-testers      0 rootResc;mnt_nfsirods    500000000 2024-12-16.15:02 & 500MB_Testfile
        generic    /mnt/nfsirods/home/[email protected]/bput_test/500MB_Testfile
  rdms-testers      0 rootResc;mnt_nfsirods     50000000 2024-12-16.15:02 & 50MB_Testfile
        generic    /mnt/nfsirods/home/[email protected]/bput_test/50MB_Testfile
$ ils -L .gocmd_staging
/testrug/home/[email protected]/.gocmd_staging:
  rdms-testers      0 rootResc;mnt_nfsirods            0 2024-12-16.15:02 ? bundle_e7752d5110e9db4b6ce5ade52e2c6da4.tar
        generic    /mnt/nfsirods/home/[email protected]/.gocmd_staging/bundle_e7752d5110e9db4b6ce5ade52e2c6da4.tar

If I remove the 1GB and 2GB files from the folder, the bput transfer succeeds with the same parameters.

I also tried the original folder once more, but specified a bigger --max_file_size than the default (precisely: gocmd bput -d --min_file_num 1 --max_file_size "21474836480" bput_test/. This also resulted in an error, but a different one (see attached log bput_again.log). In this case, the upload location is completely empty and I also end up with a non-good replica in .gocmd_staging.

$ ils -L .gocmd_staging
/testrug/home/[email protected]/.gocmd_staging:
  rdms-testers      0 rootResc;mnt_nfsirods            0 2024-12-16.15:15 ? bundle_b5a4a757b693c4bfb0c96b46fa6b0ad7.tar
        generic    /mnt/nfsirods/home/[email protected]/.gocmd_staging/bundle_b5a4a757b693c4bfb0c96b46fa6b0ad7.tar

I also saw that when I use the --icat flag, the same upload seems to succeed, but I do not understand why so far.

So

$ gocmd bput --icat -d --min_file_num 1 bput_test/

results in all files being upload. No errors or non-good replicas in this case (see attached log bput_icat_default_size.log).

We are using iRODS 4.3.2 + PAM in the background. It is for me currently not clear if this is an issue with bput, our system, or wrong use of the command.

bput.log
bput_again.log
bput_icat_default_size.log