Arvados

9442-cwl-runtime-outdir has fix.

When running a workflow, the outdir= option sets output directory for the workflow as a whole, not the output directory for each individual tool. This fix sets the outdir= option for individual tool jobs.

Reviewing 966acb6

Will this code behave well when self.work_api == 'containers'? Since $(task.outdir) is specific to run-command, which I don't think we're using for container requests, I'm worried setting this string unconditionally will cause trouble for container requests.

Does it make sense to set this as a default value, rather than always overwriting it? (kwargs.setdefault('outdir', '$(task.outdir)')) That would make this class more extensible in the future, although I admit it's hard to say how important that is right now.

It would be great to see tests for this, for both jobs and containers work APIs.

Thanks.

Trying to run arvados-cwl-runner without specifying the api flag (since I was unsure what it was for) caused this error:

Traceback (most recent call last):
6/28/2016 10:41:31 AM     compute0     1     task-print     0     File "/tmp/crunch-job/src/crunch_scripts/cwl-runner", line 50, in <module>
6/28/2016 10:41:31 AM     compute0     1     task-print     0     runner = arvados_cwl.ArvCwlRunner(api_client=arvados.api('v1', model=OrderedJsonModel()), work_api="jobs")
6/28/2016 10:41:31 AM     compute0     1     task-print     0     TypeError: __init__() got an unexpected keyword argument 'work_api'

9442-cwl-runtime-outdir updated with containers fixed, testing, and a fix for the cwl-runner crunch script.

The branch at 8ffcc8c looks good to me. A couple of parting nits:

In __init__.py, "# todo: autodetect API to use." - It could be nice to extend this comment to suggest that we should check the local environment to try to see whether we're currently running as an Arvados job or container, and set work_api to match, and then detect the API server's preference if that turns up nothing. A strategy like that would keep the Crunch script working as-is, whereas a strategy that only asks the API server would break it.

In "%s/%s" % (self.outdir, self.stdout), using os.path.join would be a little cleaner and safer.

Thanks.