Arvados

I reran the job with a sdk version: 722e147756526579ba32a31f967e9e00d47fd3ed

(before I used 92768ce858673678aa7924f83ad41e2a9f8dd678)

https://workbench.wx7k5.arvadosapi.com/pipeline_instances/wx7k5-d1hrv-ejxs4nxc0dy21xr

and it worked, no longer blocked.

This job used 722e147756526579ba32a31f967e9e00d47fd3ed

https://workbench.wx7k5.arvadosapi.com/collections/e6ee0c7858d44fec27cd245ed98b0117+91/wx7k5-8i9sb-o0dzi5z6vlq204g.log.txt

and failed with the same error.

Also I think its currently blocking #7933, its a little hard to tell, but I think this is making the job end prematurely.

• It takes >60 seconds to process the buffered stderr after the slurm jobstep exits. The process already exited 1 for some unrelated reason, but we don't see the relevant part of stderr because slurm cut us off before we got to it.
• The task process forks/detaches a child process that is still running after the task process exits, and stderr is still coming from that daemon process for >= 60s when slurm decides something is wrong. (But: wouldn't "docker run" shut down the container and kill off any such daemon processs before exiting?)

https://workbench.wx7k5.arvadosapi.com/jobs/wx7k5-8i9sb-6k7kp8hgr81hicn#Status

This job has the stderr print to a file, and the error does not show up.