Arvados

First pass on review:

The rabix/experiments repo seems to duplicate a lot of the work in the common-workflow-language repository, but less thoroughly and hasn't been updated since August. Does common-workflow-language supersede rabix/experiments?

In the common-workflow-language repo:

I could not get the reference implementation to pass its unit tests:

hitchcock:/home/twp/common-workflow-language/reference% python setup.py test
running test
Searching for jsonschema>=2.4.0
Reading https://pypi.python.org/simple/jsonschema/
Best match: jsonschema 2.4.0
Downloading https://pypi.python.org/packages/source/j/jsonschema/jsonschema-2.4.0.zip#md5=f645c88123189976058fcf550c02e50f
Processing jsonschema-2.4.0.zip
Writing /tmp/easy_install-xZWzjY/jsonschema-2.4.0/setup.cfg
Running jsonschema-2.4.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xZWzjY/jsonschema-2.4.0/egg-dist-tmp-DwEYCO
zip_safe flag not set; analyzing archive contents...
jsonschema.tests.test_jsonschema_test_suite: module references __file__

Installed /home/twp/common-workflow-language/reference/jsonschema-2.4.0-py2.7.egg
running egg_info
writing requirements to cwltool.egg-info/requires.txt
writing cwltool.egg-info/PKG-INFO
writing top-level names to cwltool.egg-info/top_level.txt
writing dependency_links to cwltool.egg-info/dependency_links.txt
writing entry points to cwltool.egg-info/entry_points.txt
reading manifest file 'cwltool.egg-info/SOURCES.txt'
writing manifest file 'cwltool.egg-info/SOURCES.txt'
running build_ext
test_job_order (tests.test_examples.TestExamples) ... ERROR

======================================================================
ERROR: test_job_order (tests.test_examples.TestExamples)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/twp/common-workflow-language/reference/tests/test_examples.py", line 8, in test_job_order
    job = t.job(from_url("../examples/bwa-mem-job.json"))
TypeError: job() takes at least 3 arguments (2 given)

----------------------------------------------------------------------
Ran 1 test in 0.129s

FAILED (errors=1)

The conformance suite also fails: each test fails with "No module named jsonschema.exceptions". I tried installing the reference implementation into a virtualenv and activating that, but then each test failed with "No module named requests".

I realize that the schema and documentation are in flux, but these things that jumped out at me on a first reading:

• The tool schema #requirements/resources doesn't specify units for diskspace or mem. This probably can't be enforced at the schema level but needs to be documented formally.
• examples/cat2-tool.json refers to a #inputs/file1/path parameter that isn't present.