Bug #3825
closed
[Crunch] crunch-job should save (very) large log files without filling up tmp space
Added by Ward Vandewege over 10 years ago.
Updated over 10 years ago.
Estimated time:
(Total: 0.00 h)
Description
Instead of writing to a temporary file and then calling arv-put, pipe the log directly to an arv-put process.
- Description updated (diff)
- Subject changed from [Crunch] fix crunch-dispatch so it does not fill up tmp space when there a job has a (very) large log file to [Crunch] crunch-job should save (very) large log files without filling up tmp space
- Description updated (diff)
- Category set to Crunch
- Target version changed from Arvados Future Sprints to 2014-10-08 sprint
- Assigned To set to Tim Pierce
- Status changed from New to In Progress
Reviewing 3825-crunch-pipe-to-arv-put at c6a6231
- I think it would read better if
output_log_*
were renamed to log_writer_*
- The word "output" usually means "output of a task or job" around here, and that's already a bit overloaded and confusing.
- Perhaps
write_output_log
should rename to something slightly different than write_log_writer
, though. Maybe write_to_log_writer
or log_writer_send
?
- I do like the idea of using the whole substring
log_writer
(née output_log
) in all of the related functions. It makes a really clear distinction between "this log pipe" and "some mysterious perl feature" and "some other logging-related thing we do here" etc.
I think a semicolon, although not strictly necessary, would be better form here:
Other than that, LGTM!
- Status changed from In Progress to Resolved
- % Done changed from 67 to 100
Applied in changeset arvados|commit:d65963976174bd9c94fdb0b91eeb8a281f01e7b3.
- Target version changed from 2014-10-08 sprint to 2014-10-29 sprint
- Target version changed from 2014-10-29 sprint to 2014-10-08 sprint
Also available in: Atom
PDF