Arvados » Arvados Epics

Support versioning for registered workflows in Workbench.

Possible features:

• When launching a workflow, user can choose among multiple versions
• Can easily determine which version of the workflow was used to run a particular pipeline.
• Text of workflow that is used for execution is more like the text that the user started from (instead of workflow packing).
• Text of workflow as submitted (i.e. original) is also saved so that users can compare the "original" they submitted against their VCS later (good for auditability).
• Requirement from customer: when making a copy, record the upstream collection it was copied from, indicate in the user interface that the upstream version has changed, offer to bump up to current version.

Possible design:

• Store CWL in a collection with "type: workflow" property
• Use collection versioning
• Store workflow metadata in properties
  • entry point
  • input spec
  • git commit/branch/repo? if uploaded a git directory
  • docker jobs image to use
• Deprecate "workflow" records
• Identify the workflow version used for a given pipeline by matching the PDH

Implementation:

• arvados-cwl-runner support for creating/updating new-style registered workflows
• Workbench 2 support (this is probably the majority of the work)
  • picking
  • viewing
  • launching

Proposed new behavior for a-c-r: always create a workflow collection when running a workflow, so we have accessible record of what was run.