Arvados

Sorry the error message got formatted all on one line for some reason. Here it is in a more readable format:

2017-11-17T11:31:37.852090800Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-ploidy.vcf.gz (403 bytes)
2017-11-17T11:31:37.854757100Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-ploidy.vcf.gz.tbi (385 bytes)
2017-11-17T11:31:37.854791000Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-regions-merged.bed (285572 bytes)
2017-11-17T11:31:37.854875100Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-regions-merged.bed.gz (94913 bytes)
2017-11-17T11:31:37.854917900Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-regions-merged.bed.gz.tbi (13964 bytes)
2017-11-17T11:31:37.854941700Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-regions.bed (285572 bytes)
2017-11-17T11:31:37.855083700Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/stats/runStats.tsv (170 bytes)
2017-11-17T11:31:37.856529200Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/stats/runStats.xml (451 bytes)
2017-11-17T11:31:37.856914000Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/genome.S1.vcf.gz (24603795 bytes)
2017-11-17T11:31:37.868911500Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/genome.S1.vcf.gz.tbi (34744 bytes)
2017-11-17T11:31:37.868975600Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/genome.vcf.gz (24603795 bytes)
2017-11-17T11:31:37.878724000Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/genome.vcf.gz.tbi (34744 bytes)
2017-11-17T11:31:37.878796700Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/variants.vcf.gz (4136633 bytes)
2017-11-17T11:31:37.880521500Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/results/variants/variants.vcf.gz.tbi (59036 bytes)
2017-11-17T11:31:37.880587900Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/runWorkflow.py (7795 bytes)
2017-11-17T11:31:37.882003000Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/runWorkflow.py.config.pickle (4785 bytes)
2017-11-17T11:31:37.882031100Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/workflow.error.log.txt (0 bytes)
2017-11-17T11:31:37.882054800Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/workflow.exitcode.txt (2 bytes)
2017-11-17T11:31:37.882076800Z Uploading strelka2/chr19/NA24385-chr19_0_16089283-block-work/workflow.warning.log.txt (0 bytes)
2017-11-17T11:31:37.882133000Z While uploading output files: Symlink in output "/strelka2/chr19/NA24385-chr19_0_16089283-block.vcf.gz" points to invalid location "genome.S1.vcf.gz": lstat /tmp/603101599/strelka2/chr19/genome.S1.vcf.gz: no such file or directory
2017-11-17T11:31:37.882144900Z Cancelled

I think what it is trying to tell you that the output file strelka2/chr19/NA24385-chr19_0_16089283-block.vcf.gz is a symlink to genome.S1.vcf.gz, but genome.S1.vcf.gz doesn't exist.

It looks like it might be resolving the relative symlink incorrectly, which would be a bug. I'll have to investigate.

Thanks Peter. I'll work around this by avoiding the symlinks and copying the files out of the strelka2 specific directory into the out directory. If you end up digging into it, I'd be interested to know if it's the relative symlinks or the nested double symlink that causes issues, in case I do any symlinking elsewhere. I'll report back after testing with the copies output.

Here's a minimal CWL test case:

class: CommandLineTool
cwlVersion: v1.0
inputs: []
outputs:
  out:
    type: File
    outputBinding:
      glob: Test1-chrM_0_1000-block.vcf.gz
requirements:
  ShellCommandRequirement: {}
arguments:
  - shellQuote: false
    valueFrom: |
      mkdir -p Test1-chrM_0_1000-block-work/results/variants
      echo genome | gzip -c > Test1-chrM_0_1000-block-work/results/variants/genome.S1.vcf.gz
      cd Test1-chrM_0_1000-block-work/results/variants
      ln -s genome.S1.vcf.gz genome.vcf.gz
      cd ../../..
      ln -s Test1-chrM_0_1000-block-work/results/variants/genome.vcf.gz Test1-chrM_0_1000-block.vcf.gz

This fails on current master but passes on #13100 branch.