Bug #11693
closed
[Crunch2] Correctly handle symlinks to input files in output dir
Added by Peter Amstutz over 7 years ago.
Updated over 7 years ago.
Estimated time:
(Total: 0.00 h)
Description
Crunch v1 and cwltool allow the output directory to have symlinks to input files, which are then efficiently propagated to output. Unfortunately, crunch-run fails in this case. Some scripts rely on this behavior. It needs to support dereferencing symlinks and adding them to the output collection.
- Description updated (diff)
- Assigned To set to Peter Amstutz
Reviewed a340487a7
Did a jenkins test run because my local tests kept failing for some reason, all ok: https://ci.curoverse.com/job/developer-run-tests/286/
Just a minor comment and a couple of questions:
- File
services/crunch-run/crunchrun.go:962: outputSuffix already assigned on L952
- What should happen when the symlink is pointing to a mount, but that mount is not a collection? (asking because of the
...&& mnt.Kind == “collection” bit on L958). Can a mount be other thing than a collection? Shouldn't the collection mounts be filtered when building the binds list on L924?
Lucas Di Pentima wrote:
Reviewed a340487a7
Did a jenkins test run because my local tests kept failing for some reason, all ok: https://ci.curoverse.com/job/developer-run-tests/286/
Just a minor comment and a couple of questions:
- File
services/crunch-run/crunchrun.go:962: outputSuffix already assigned on L952
Copy and paste error. Fixed.
- What should happen when the symlink is pointing to a mount, but that mount is not a collection? (asking because of the
...&& mnt.Kind == “collection” bit on L958). Can a mount be other thing than a collection? Shouldn't the collection mounts be filtered when building the binds list on L924?
Inputs are always collections. Symlinks to files in temporary directories isn't supported.
Filtering the binds array to just collections is a good idea. Fixed.
Now at aceb2d1ed239fa82fcb8bb352b632a8d92251dac
- Status changed from In Progress to Resolved
Applied in changeset arvados|commit:f26f70d0a60798065c5f7a5cb91b95587cc9e9ef.
Also available in: Atom
PDF
Updated by 
Updated by