Actions
Story #11419
closed
[SDKs] support text-mode open() in Python 3
Added by Tom Clegg over 7 years ago. Updated almost 6 years ago.
Start date:
03/21/2017
Due date:
% Done:
100%
Estimated time:
(Total: 0.00 h)
Story points:
-
Release:
Release relationship:
Auto
Updated by Tom Clegg over 7 years ago
- Subject changed from open files in text or binary mode to support text-mode open() in Python 3
Updated by Tom Clegg over 7 years ago
- Subject changed from support text-mode open() in Python 3 to [SDKs] support text-mode open() in Python 3
- Assigned To deleted (
Tom Clegg) - Parent task deleted (
#11308)
Updated by Tom Clegg over 7 years ago
- Tracker changed from Task to Feature
- Target version deleted (
2017-05-10 sprint)
Updated by Tom Morris over 7 years ago
- Target version set to Arvados Future Sprints
Updated by Tom Morris over 6 years ago
- Related to Story #11308: Support Python3 for arvados-python-client & command line utilities added
Updated by Tom Morris over 6 years ago
- Target version changed from Arvados Future Sprints to To Be Groomed
Updated by Tom Morris about 6 years ago
- Blocks Story #14532: [Epic] Port to Python 3 to for Python 2 sunset in December 2019 added
Updated by Tom Morris about 6 years ago
- Target version changed from To Be Groomed to 2018-12-12 Sprint
Updated by Tom Clegg about 6 years ago
11419-text-mode @ 0565661b2c4fb220428ea37c2e0728e4b07d2465 https://ci.curoverse.com/view/Developer/job/developer-run-tests/989/
This change mostly comes down to wrapping the ArvadosFileReader/Writer in an io.BufferedReader/Random and then in an io.TextIOWrapper, if text mode is requested.
With the previous code, the "block boundary isn't on a character boundary" test crashed in Python 2 when using text mode, and the same wrapper fixes it, so I figured it would make sense to fix both at once. However, this does cause a slight API change that could conceivably break existing code. In text mode, the file-like object returned by Collection.open()...- ...is not an ArvadosFileReader/Writer. Code that tests
isinstance(f, ...)will behave differently. I found no examples of this in our own repo. - ...cannot seek to arbitrary positions (see TextIOBase seek restrictions). The only occurrence of this I found in our repo was in the "open in text mode in Python 2" test.
- ...does not accept close(flush=False). This revealed (via sdk/cwl test failures) that source:sdk/python/arvados/commands/run.py was accidentally using text mode to copy arbitrary files.
Updated by Tom Clegg about 6 years ago
This broke some things: https://ci.curoverse.com/job/developer-run-tests-sdk-python-ruby/275/console
Updated by Tom Clegg about 6 years ago
Updated by
Updated by Tom Clegg about 6 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|7317196f547cf8c5e0ec87d1526322c305faba30.
Updated by Peter Amstutz about 6 years ago
This has a regression:
https://ci.curoverse.com/view/CWL/job/run-cwl-test-4xphq/799/console
2018-12-14 20:28:39 arvados.cwl-runner ERROR: [step-valuefrom5-wf.cwl] While getting final output object: 'TextIOWrapper' object has no attribute 'size'
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/arvados_cwl/runner.py", line 465, in done
if f.size() > 0:
AttributeError: 'TextIOWrapper' object has no attribute 'size'
Updated by Peter Amstutz about 6 years ago
- Status changed from Resolved to Feedback
- Target version changed from 2018-12-12 Sprint to 2018-12-21 Sprint
Updated by Tom Clegg about 6 years ago
11419-cwl-read-json-binary-mode @ fef817dc5d6d1e0160c6c14ba4f41581e241f33a
Updated by Tom Clegg about 6 years ago
- Status changed from Feedback to Resolved
Actions