Arvados

Story #9581

Updated by Brett Smith over 8 years ago

h3. Use case 

 Administrators want additional control over how Arvados containers are dispatched to SLURM.    Some options they might commonly want to specify are @--account@, @--clusters@, or @--partition@.    crunch-dispatch-slurm should provide a simple mechanism for administrators to set arbitrary additional arguments to sbatch. 

 h3. Implementation 

 Add a command line flag, "-config path/to/config.json" (see source:services/keep-balance/main.go for an example).    crunch-dispatch-slurm reads configuration entries from this JSON file.    If no @-config@ switch is given, it tries to read configuration from @/etc/arvados/crunch-dispatch-slurm/config.json@.    If that file does not exist, crunch-dispatch-slurm continues running with default configuration.    If there's any other error reading the file (permissions problem, invalid JSON), or if the user specified a config file that doesn't exist, that's a fatal error, and the program should abort with a useful error message. 

 Convert existing command line flags to config file entries. 

 <pre> 
 type Config struct { 
         SbatchArguments    []string 
         PollPeriod         time.Duration 
         CrunchRunCommand string 
 } 
 </pre> 

 The Append the configured SbatchArguments should be inserted into to the command existing arguments passed to @exec.Command()@, *after* the generic options @sbatch --share@ and *before* the job-specific options @--job-name@, @--mem-per-cpu@, and @--cpus-per-task@.    This is because the administrator's configuration should take precedence over default behavior switches, but not over job-specific switches which have more details about how the container should be run. exec.Command(). 

 h3. Tests 

 Just unit tests for sbatchFunc. 
 * Handles nil SbatchArguments correctly 
 * Handles non-nil SbatchArguments correctly 

Back