Bug #4957
open
[SDKs] arv keep put blows up when trying to put a danging symlink
Added by Ward Vandewege almost 10 years ago.
Updated over 3 years ago.
Description
For example:
$ arv keep put mirobot_updater
Traceback (most recent call last):
File "/usr/local/bin/arv-put", line 4, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 411, in main
bytes_expected = expected_bytes_for(args.paths)
File "/usr/local/lib/python2.7/dist-packages/arvados/commands/put.py", line 333, in expected_bytes_for
bytesum += os.path.getsize(os.path.join(path, filename))
File "/usr/lib/python2.7/genericpath.py", line 49, in getsize
return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: 'mirobot_updater/resources/app/node_modules/.bin/serialportlist'
and
$ ls mirobot_updater/resources/app/node_modules/.bin/serialportlist -laF
lrwxrwxrwx 1 wardtest wardtest 35 Jan 10 19:41 mirobot_updater/resources/app/node_modules/.bin/serialportlist -> ../serialport/bin/serialportList.js
arv keep put should just skip things it can not save, and ideally list at the end of the run which file(s) it skipped.
- Description updated (diff)
- Subject changed from [SDK] arv keep put blows up when trying to put a danging symlink to [SDKs] arv keep put blows up when trying to put a danging symlink
- Category set to SDKs
- Target version changed from Bug Triage to Arvados Future Sprints
It does this if you specify any kind of nonexistent file, not just a dangling symlink. I assume you want nicer error reporting? There's a lot of room for improvement in all the CLI tools there—exception tracebacks are still the way most problems are presented to users.
Brett Smith wrote:
It does this if you specify any kind of nonexistent file, not just a dangling symlink. I assume you want nicer error reporting? There's a lot of room for improvement in all the CLI tools there—exception tracebacks are still the way most problems are presented to users.
Yeah, I think the desired behavior would be that arv keep put skips any files it can not upload, and prints out the list of errors at the end. It shouldn't abort the upload because there's one dangling symlink in a directory tree.
- Target version changed from Arvados Future Sprints to 2015-03-11 sprint
Ward Vandewege wrote:
Yeah, I think the desired behavior would be that arv keep put skips any files it can not upload, and prints out the list of errors at the end. It shouldn't abort the upload because there's one dangling symlink in a directory tree.
- Do you want it to skip on any OS error when it tries to open the file?
- Do you want to adjust the behavior if there's an error while reading the file after opening?
- What should the exit code be in this case?
- Should there be different exit codes for the case of "couldn't upload some files" vs. "couldn't upload any files?"
- Assigned To set to Peter Amstutz
- Target version changed from 2015-03-11 sprint to 2015-04-01 sprint
- Assigned To deleted (
Peter Amstutz)
Perhaps rsync's behavior in the face of IO errors is a good model to follow. "Some stuff failed along the way, so exiting non-zero. But I did everything I could."
- Description updated (diff)
- Assigned To set to Peter Amstutz
Tom Clegg wrote:
Perhaps rsync's behavior in the face of IO errors is a good model to follow. "Some stuff failed along the way, so exiting non-zero. But I did everything I could."
Yeah. That's right.
- Assigned To deleted (
Peter Amstutz)
- Target version changed from 2015-04-01 sprint to Arvados Future Sprints
- Target version deleted (
Arvados Future Sprints)
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by 
Updated by