Feature #8017
closed
[Crunch2] support runtime constraints
Description
Propagate runtime constraints from Container to slurm in crunch-dispatch-slurm.
See runtime_constraints at Containers API
We should support (at least) the following cases:- request 12 GB RAM, do not specify number of CPU cores: {"ram":12000000000}
- request 12 GB RAM and 4 CPU cores: {"ram":12000000000, "vcpus":4}
- do not propagate any constraints: {}
The Container record's runtime_constraints hash is expected to have a numeric value (or null/missing) for each element -- never a range of acceptable values like the ContainerRequest examples.
Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
Updated by Tom Clegg almost 10 years ago
- Target version set to 2016-05-11 sprint
Updated by Tom Clegg almost 10 years ago
- Assigned To set to Peter Amstutz
- Story points set to 1.0
Updated by Radhika Chippada almost 10 years ago
- Assigned To changed from Peter Amstutz to Radhika Chippada
Updated by Peter Amstutz almost 10 years ago
Updated by Radhika Chippada almost 10 years ago
- Status changed from New to In Progress
Updated by Radhika Chippada almost 10 years ago
The story description said: (1) The Container record's runtime_constraints hash is expected to have a numeric value for each element, (2) Every Container record is guaranteed to have a value for "ram" and "vcpus" by the time it's eligible for dispatch.
- Based on this, I updated the slurm dispatch code to expect these numeric runtime_constraints in the Container record and pass them to sbatch command.
- I used strconv.Itoa to pass the numeric values as strings to sbatch. Please let me know if this needs any corrections.
Commit f1adedeb
Updated by Peter Amstutz almost 10 years ago
A careful reading of the SLURM page suggests that we should be using --mem-per-cpu instead of --mem. So, you need to divide ram by the number of cpus and round up. Also, the slurm --mem-per-cpu option takes MegaBytes but the crunch v2 API gives RAM in bytes. You need to divide by 2^20 and round up.
Updated by Radhika Chippada almost 10 years ago
Thanks Peter. Updated per note 9 at b5e2bbb9
Updated by Peter Amstutz almost 10 years ago
One more tweak, this should be int64 instead of int because int in Go can be either 32 or 64 bits.
RuntimeConstraints map[string]int `json:"runtime_constraints"`
Updated by Radhika Chippada almost 10 years ago
Updated to RuntimeConstraints to use int64. Thanks.
Updated by Radhika Chippada almost 10 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:a32c69b81296860a30cc33909226d9294f411adf.