Story #14996
open
Log crunch-run resource usage statistics
0%
Description
Currently we have crunchstat.txt for container stats and hoststat.txt for node level stats, but we have no way of recording the time and resources for file upload by crunch-run after the container exits. This would be very useful since, for I/O intensive jobs like bcl2fastq, this can represent as much time as the compute phase of the job.
Updated by Peter Amstutz almost 6 years ago
In order to collect fine-grained metrics, one option is for crunch-run to have a bootstrap mode that creates a cgroup and runs crunch-run and arv-mount inside it.
Updated by Eric Biagiotti almost 6 years ago
- Story points set to 2.0
Go cgroup module should make this easier https://godoc.org/github.com/containerd/cgroups. Might be worth trying to have crunch-run move itself to a cgroup in bootstrap mode. If that doesn't work, we'd have to start another process.
Once crunch-run and arv-mount are in their own cgroup, the arvados crunchstat go module can automatically interpret stats.
Updated by Tom Morris over 5 years ago
- Target version changed from To Be Groomed to Arvados Future Sprints
Updated by Peter Amstutz over 3 years ago
- Target version deleted (
Arvados Future Sprints)
Updated by