Feature #8017
closed
[Crunch2] support runtime constraints
Added by Peter Amstutz about 9 years ago.
Updated over 8 years ago.
Assigned To:
Radhika Chippada
Estimated time:
(Total: 0.00 h)
Release relationship:
Auto
Description
Propagate runtime constraints from Container to slurm in crunch-dispatch-slurm.
See runtime_constraints at Containers API
We should support (at least) the following cases:
- request 12 GB RAM, do not specify number of CPU cores: {"ram":12000000000}
- request 12 GB RAM and 4 CPU cores: {"ram":12000000000, "vcpus":4}
- do not propagate any constraints: {}
The Container record's runtime_constraints hash is expected to have a numeric value (or null/missing) for each element -- never a range of acceptable values like the ContainerRequest examples.
Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
- Target version set to 2016-05-11 sprint
- Assigned To set to Peter Amstutz
- Story points set to 1.0
- Assigned To changed from Peter Amstutz to Radhika Chippada
- Description updated (diff)
- Description updated (diff)
- Status changed from New to In Progress
The story description said: (1) The Container record's runtime_constraints hash is expected to have a numeric value for each element, (2) Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
- Based on this, I updated the slurm dispatch code to expect these numeric runtime_constraints in the Container record and pass them to sbatch command.
- I used strconv.Itoa to pass the numeric values as strings to sbatch. Please let me know if this needs any corrections.
Commit f1adedeb
A careful reading of the SLURM page suggests that we should be using --mem-per-cpu instead of --mem. So, you need to divide ram by the number of cpus and round up. Also, the slurm --mem-per-cpu option takes MegaBytes but the crunch v2 API gives RAM in bytes. You need to divide by 2^20 and round up.
Thanks Peter. Updated per note 9 at b5e2bbb9
One more tweak, this should be int64 instead of int because int in Go can be either 32 or 64 bits.
RuntimeConstraints map[string]int `json:"runtime_constraints"`
Updated to RuntimeConstraints to use int64. Thanks.
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:a32c69b81296860a30cc33909226d9294f411adf.
Also available in: Atom
PDF
Updated by 
Updated by Radhika Chippada 
Updated by