Bug #13050
closedTutorial arvados-cwl-runner job failing on qr1hi
Description
I'm just running through some arvados tutorial material on qr1hi but my workflow keeps failing. Am I missing something basic?
Command: arvados-cwl-runner bwa-mem.cwl bwa-mem-input.yml
error log: https://workbench.qr1hi.arvadosapi.com/jobs/qr1hi-8i9sb-www5rwum7f2atdb#Log
text in error log:
Timestamp Node Slot Log type Task Message
2/7/2018 1:59:43 PM crunch running from /usr/local/arvados/src/sdk/cli/bin/crunch-job with arvados-cli Gem version(s) 0.1.20170817171636, 0.1.20170322173355, 0.1.20161017193526
2/7/2018 1:59:43 PM crunch check slurm allocation
2/7/2018 1:59:43 PM crunch node compute2 - 1 slots
2/7/2018 1:59:43 PM crunch start
2/7/2018 1:59:43 PM crunch clean work dirs: start
2/7/2018 1:59:43 PM crunch stderr starting: ['srun','--nodelist=compute2','-D','/tmp','bash','-ec',' arv-mount --unmount-timeout 10 --unmount-all ${CRUNCH_TMP} rm -rf ${JOB_WORK} ${CRUNCH_INSTALL} ${CRUNCH_TMP}/task ${CRUNCH_TMP}/src* ${CRUNCH_TMP}/*.cid ']
2/7/2018 1:59:47 PM crunch clean work dirs: exit 0
2/7/2018 1:59:47 PM crunch Install docker image fed3a0c5b7af2f800207abd3035ede74+129
2/7/2018 1:59:47 PM crunch docker image hash is sha256:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a
2/7/2018 1:59:47 PM crunch load docker image: start
2/7/2018 1:59:48 PM crunch stderr starting: ['srun','--nodelist=compute2','/bin/bash','-o','pipefail','-ec',' loaded() { id=$(/usr/bin/docker inspect --format="{{.ID}}" sha256\\:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a) || return 1 echo "image ID is $id" [[ ${id} = sha256\\:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a ]] } if loaded >&2 2>/dev/null; then echo >&2 "image is already present" exit 0 fi echo >&2 "docker image is not present; loading" arv-get fed3a0c5b7af2f800207abd3035ede74\\+129\\/sha256\\:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a\\.tar | /usr/bin/docker load if ! loaded >&2; then echo >&2 "`docker load` exited 0, but image is not found (!)" exit 1 fi echo >&2 "image loaded successfully" ']
2/7/2018 1:59:48 PM crunch stderr docker image is not present; loading
2/7/2018 1:59:48 PM crunch stderr image ID is sha256:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a
2/7/2018 1:59:48 PM crunch stderr image loaded successfully
2/7/2018 1:59:48 PM crunch load docker image: exit 0
2/7/2018 1:59:48 PM crunch check --memory-swap feature: start
2/7/2018 1:59:48 PM crunch stderr starting: ['srun','--nodes=1','/usr/bin/docker','run','--help']
2/7/2018 1:59:49 PM crunch check --memory-swap feature: exit 0
2/7/2018 1:59:49 PM crunch check whether default user is UID 0: start
2/7/2018 1:59:49 PM crunch stderr starting: ['srun','--nodes=1','/bin/sh','-ec','/usr/bin/docker run sha256:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a id --user']
2/7/2018 1:59:49 PM crunch stderr [main] unrecognized command 'id'
2/7/2018 1:59:49 PM crunch stderr srun: error: compute2: task 0: Exited with exit code 1
2/7/2018 1:59:49 PM crunch check whether default user is UID 0: exit 1
2/7/2018 1:59:49 PM crunch check whether user 'crunch' is UID 0: start
2/7/2018 1:59:49 PM crunch stderr starting: ['srun','--nodes=1','/bin/sh','-ec','/usr/bin/docker run --user=crunch sha256:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a id --user']
2/7/2018 1:59:49 PM crunch stderr /usr/bin/docker: Error response from daemon: linux spec user: unable to find user crunch: no matching entries in passwd file.
2/7/2018 1:59:49 PM crunch stderr srun: error: compute2: task 0: Exited with exit code 125
2/7/2018 1:59:49 PM crunch check whether user 'crunch' is UID 0: exit 125
2/7/2018 1:59:49 PM crunch check whether user 'nobody' is UID 0: start
2/7/2018 1:59:49 PM crunch stderr starting: ['srun','--nodes=1','/bin/sh','-ec','/usr/bin/docker run --user=nobody sha256:c66bf27205ddbb602243fa047113e9ae0f585d03cf16993d778e45fcb3c9950a id --user']
2/7/2018 1:59:49 PM crunch stderr /usr/bin/docker: Error response from daemon: linux spec user: unable to find user nobody: no matching entries in passwd file.
2/7/2018 1:59:49 PM crunch stderr srun: error: compute2: task 0: Exited with exit code 125
2/7/2018 1:59:49 PM crunch check whether user 'nobody' is UID 0: exit 125
2/7/2018 1:59:49 PM crunch Could not find a user in container that is not UID 0 (tried default user, crunch nobody) or there was a problem running 'id' in the container. at /usr/local/arvados/src/sdk/cli/bin/crunch-job line 496
2/7/2018 1:59:49 PM crunch release job allocation
2/7/2018 1:59:50 PM crunch Freeze not implemented
2/7/2018 1:59:50 PM crunch collate
2/7/2018 1:59:50 PM crunch collated output manifest text to send to API server is 0 bytes with access tokens