Bug #23229
open
Permission denied exception trying to create cache when uploading via arv-put
Description
This issue has been reported to us. It looks like the cache creation is inconsistent. It creates a arv-put cache in the home of the user which seems to be fine. But then it tries to create a discovery cache in the location where the upload is started. We assume the user is lacking some permissions there, so this breaks.
[marbacd2@ec-hub2-kau rnaseq_PS-12053]$ arv keep put --update-collection arkau-4zz18-ss13ebjw5d0lbrv bksnake
2025-10-22 17:21:25 arvados.arv_put[3268474] INFO: Calculating upload size, this could take some time...
2025-10-22 17:21:25 arvados.arv_put[3268474] INFO: Creating new cache file at /home/marbacd2/.cache/arvados/arv-put/16438654ef2e59b2fb2cea199f6c95b3
0M / 2663M 0.0% 2025-10-22 17:21:26 arvados.arv_put[3268474] WARNING: Abnormal termination:
Traceback (most recent call last):
File "/apps/rocs/2024.04/common/x86-64-v4/software/arvados/3.0.0-GCCcore-13.3.0-Python-3.12.3/lib/python3.12/site-packages/arvados/commands/put.py", line 622, in start
self._upload_files()
File "/apps/rocs/2024.04/common/x86-64-v4/software/arvados/3.0.0-GCCcore-13.3.0-Python-3.12.3/lib/python3.12/site-packages/arvados/commands/put.py", line 838, in _upload_files
with open(source, 'rb') as source_fd:
^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache/arvados/discovery/e9d8087a1b07d268d229eb49687f9d7b.tmp'
We instructed the user to launch this from his home or set XDG_CACHE_HOME. However, it seems like a bug. It should not try to write anything into this location.
Updated by Brett Smith 5 months ago
- Release set to 83
- Category set to SDKs
Hi Moritz, is it possible to get at least a partial copy of the user's environment when they ran into this issue? At least their HOME and XDG_* variables? I agree that if we can't write a cache we should just continue on without it. However, I'm wondering if we have a second bug where we're trying to write the cache to the wrong place in some situations. Knowing the user's setup at the time would help figure that out. Thanks.
Updated by Moritz Gilsdorf 5 months ago
Hi Brett,
I got this information from the User. Interestingly it does not seem like he is lacking write permissions.
[marbacd2@ec-hub1-kau ~]$ echo "$HOME" /home/marbacd2 Had no XDG_* Variables set previously Permissions: [marbacd2@ec-hub1-kau ~]$ ll /projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/ total 247 -rw-rw---- 1 marbacd2 EMEA_SMR_CORE_clincomp 52871 May 21 09:05 README.md -rw-rw---- 1 marbacd2 EMEA_SMR_CORE_clincomp 6 May 21 09:05 VERSION drwxrws--- 5 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:07 bedapub_bksnake -rw-rw-r-- 1 schmucr1 EMEA_SMR_CORE_clincomp 23830 May 22 11:03 bk_rulegraph_2025-05-22_110233.pdf drwxrws--- 2 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:05 config drwxrws--- 3 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:08 logs -rw-rw---- 1 marbacd2 EMEA_SMR_CORE_clincomp 5062 May 21 09:05 lsf.yaml drwxrws--- 2 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:05 profile_slurm drwxrws--- 4 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:05 resources drwxrws--- 15 marbacd2 EMEA_SMR_CORE_clincomp 8192 May 22 12:07 results -rw-rw---- 1 marbacd2 EMEA_SMR_CORE_clincomp 20981 May 21 09:05 run.py -rwxrwx--- 1 marbacd2 EMEA_SMR_CORE_clincomp 7732 May 21 09:05 run.sh -rw-rw---- 1 marbacd2 EMEA_SMR_CORE_clincomp 7494 May 21 09:05 vc_run.sh drwxrws--- 7 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:05 workflow
Updated by Brett Smith 5 months ago
Moritz Gilsdorf wrote in #note-2:
I got this information from the User. Interestingly it does not seem like he is lacking write permissions.
The output of ls -AlR /projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache would be helpful. .cache wasn't included in the last output, and I'm wondering if that directory exists owned by schmucr1 without group/other write permissions.
Updated by Moritz Gilsdorf 5 months ago
Hi @Brett Smith,
Here is the output.
[marbacd2@ec-hub2-kau biokitr]$ ls -AlR /projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache
/projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache:
total 2
drwxrws--- 3 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:08 arvados
drwxr-xr-x 2 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 21 09:15 fontconfig
/projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache/arvados:
total 1
drwxrws--- 2 marbacd2 EMEA_SMR_CORE_clincomp 4096 May 22 12:06 discovery
/projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache/arvados/discovery:
total 160
-rw------- 1 schmucr1 EMEA_SMR_CORE_clincomp 148316 May 22 12:06 e9d8087a1b07d268d229eb49687f9d7b.tmp
/projects/site/pred/pmda_onc_gg_clin/2024/morpheus_melanoma/rnaseq_PS-12053/bksnake/bksnake/.cache/fontconfig:
total 65
-rw-rw---- 1 marbacd2 games 57000 May 21 09:15 5f28344a038062402944c3da52536cd5-le64.cache-8
-rw-r----- 1 marbacd2 games 200 May 21 09:15 CACHEDIR.TAG