Bug #16367: forced download of arvados/jobs to client - Arvados

Actions

Copy link

Bug #16367

open

forced download of arvados/jobs to client

Added by Michael Crusoe over 4 years ago. Updated almost 2 years ago.

Status:

New

Priority:

Normal

Assigned To:

Category:

Target version:

Start date:

Due date:

% Done:

Estimated time:

Story points:

Release:

Postponed

Release relationship:

Auto

Description

Even when not part of the workflow :-(

$ arvados-cwl-runner msa.cwl msa_arv.yaml
INFO /home/michael/src/2020-covid-19-bh/env3.7/bin/arvados-cwl-runner 2.1.0.dev20200407192516, arvados-python-client 2.0.1, cwltool 3.0.20200324120055
INFO Resolved 'msa.cwl' to 'file:///home/michael/src/2020-covid-19-bh/msa/msa.cwl'
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using empty collection d41d8cd98f00b204e9800998ecf8427e+0
INFO Using collection cache size 256 MiB
INFO ['docker', 'pull', 'arvados/jobs:2.1.0.dev20200407192516']
2.1.0.dev20200407192516: Pulling from arvados/jobs
fc7181108d40: Pull complete

Files

Download all files

arvados.log (10.2 KB) arvados.log		Michael Crusoe, 04/24/2020 04:47 PM
msa_arv.yaml (139 Bytes) msa_arv.yaml		Michael Crusoe, 04/24/2020 04:57 PM
msa.cwl (734 Bytes) msa.cwl		Michael Crusoe, 04/24/2020 04:57 PM
iqtree.cwl (5.31 KB) iqtree.cwl		Michael Crusoe, 04/24/2020 04:57 PM
mafft.cwl (1.06 KB) mafft.cwl		Michael Crusoe, 04/24/2020 04:57 PM
arvados-debug.log (15.1 KB) arvados-debug.log		Michael Crusoe, 04/24/2020 05:05 PM
working-venv.tar.xz (12.5 MB) working-venv.tar.xz		Michael Crusoe, 04/24/2020 05:52 PM
not-working-venv.tar.xz (11.8 MB) not-working-venv.tar.xz		Michael Crusoe, 04/24/2020 05:52 PM
log (15.1 KB) log		Michael Crusoe, 05/01/2020 04:19 PM

Actions

Copy link Download all files

Updated by Michael Crusoe over 4 years ago

File msa_arv.yaml msa_arv.yaml added
File iqtree.cwl iqtree.cwl added
File msa.cwl msa.cwl added
File mafft.cwl mafft.cwl added

Adding the workflow and input job object

Actions

Copy link

Updated by Michael Crusoe over 4 years ago

The upload fails, I've retried multiple times:

2020-04-24 18:55:23 arvados.arv_put[153525] INFO: Resuming upload from cache file /home/michael/.cache/arvados/arv-put/bd0e10119db97007dbc105c8dcc5aeea
0M / 261M 0.0% 2020-04-24 18:56:10 arvados.arv_put[153525] WARNING: Abnormal termination:
Traceback (most recent call last):
  File "/home/michael/src/2020-covid-19-bh/env3.7/lib/python3.7/site-packages/arvados/arvfile.py", line 712, in commit_bufferblock
    block.set_state(_BufferBlock.PENDING)
  File "/home/michael/src/2020-covid-19-bh/env3.7/lib/python3.7/site-packages/arvados/arvfile.py", line 270, in synchronized_wrapper
    return orig_func(self, *args, **kwargs)
  File "/home/michael/src/2020-covid-19-bh/env3.7/lib/python3.7/site-packages/arvados/arvfile.py", line 360, in set_state
    raise StateChangeError("Invalid state change from %s to %s" % (self._state, nextstate), self._state, nextstate)
arvados.arvfile.StateChangeError: Invalid state change from 1 to 1

Actions

Copy link

Updated by Michael Crusoe over 4 years ago

File arvados-debug.log arvados-debug.log added

I did delete ~/.cache/arvados but the error persists

Here's the --debug log

Actions

Copy link

Updated by Lucas Di Pentima over 4 years ago

Project changed from Arvados to 35

Actions

Copy link

Updated by Lucas Di Pentima over 4 years ago

Project changed from 35 to Arvados

Actions

Copy link

Updated by Lucas Di Pentima over 4 years ago

Michael Crusoe wrote:

The upload fails, I've retried multiple times:

[...]

arv-put's error indicates that it isn't handling (or even noticing?) the 413 error when trying to PUT a block, that may need a separate bugfix ticket.

Actions

Copy link Download all files