Project

General

Profile

Actions

Bug #12857

closed

arvados-cwl-runner does not work with mixed workflow/tool cwlVersion

Added by Joshua Randall over 8 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-

Description

When I attempt to run a workflow using cwlVersion: v1.0 which then references a CommandLineTool with cwlVersion: draft-3, arvados-cwl-runner gives a validation error: "Version 'v1.1.0-dev1' is a development or deprecated version." The same workflow runs with cwlref-runner without issue.

I have created a minimal example that demonstrates this.

Run with arvados-cwl-runner:

$ arvados-cwl-runner wf-v1.0.cwl --msg "hello world" 
2017-12-20 16:33:32 cwltool INFO: /usr/bin/arvados-cwl-runner dc78526ba494973df7d298825e20503353e92adf 1.0.20171116210428, arvados-python-client 0.1.20171109204045, cwltool 1.0.20170928192020
2017-12-20 16:33:32 cwltool INFO: Resolved 'wf-v1.0.cwl' to 'file:///home/mercury/checkouts/arvados-pipelines/cwl/wf-v1.0.cwl'
2017-12-20 16:33:34 cwltool ERROR: Workflow error, try again with --debug for more information:
Tool definition file:///home/mercury/checkouts/arvados-pipelines/cwl/tool-draft-3.cwl failed validation:
  Version 'v1.1.0-dev1' is a development or deprecated version.
   Update your document to a stable version (v1.0, draft-3, draft-2) or use --enable-dev to enable support for development and deprecated versions.

Run with cwlref-runner:

$ cwl-runner wf-v1.0.cwl --msg "hello world" 
/home/mercury/checkouts/arvados-pipelines/cwl/venv/bin/cwl-runner 1.0.20171107133715
Resolved 'wf-v1.0.cwl' to 'file:///home/mercury/checkouts/arvados-pipelines/cwl/wf-v1.0.cwl'
[workflow wf-v1.0.cwl] start
[step tool] start
[job tool] /tmp/tmpoLM698$ echo \
    'hello world' > /tmp/tmpoLM698/stdout
[job tool] completed success
[step tool] completed success
[workflow wf-v1.0.cwl] completed success
{
    "out": {
        "checksum": "sha1$22596363b3de40b06f981fb85d82312e8c0ed511",
        "basename": "stdout",
        "location": "file:///home/mercury/checkouts/arvados-pipelines/cwl/stdout",
        "path": "/home/mercury/checkouts/arvados-pipelines/cwl/stdout",
        "class": "File",
        "size": 12
    }
}
Final process status is success

This is on Ubuntu 16.04.1, with Arvados/CWL versions:

$ dpkg -l |grep cwl
ii  python-arvados-cwl-runner           1.0.20171116210428-3              all          The Arvados CWL runner
ii  python-cwltool                      1.0.20170928192020-2              all          Common workflow language reference implementation

I note also that passing `--enable-dev` to arvados-cwl-runner has no effect, so if I actually had a v1.1.0-dev1 workflow, I expect that wouldn't work either.

If I modify the workflow and tool so that they both declare themselves as cwlVersion: v1.0 this error does not occur.

Actions #1

Updated by Joshua Randall over 8 years ago

Gist of the minimal example demonstrating this issue: https://gist.github.com/jrandall/0fbc17592ceccc788c98000f7e2f941c

Actions #2

Updated by Peter Amstutz about 6 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF