Story #13731
closed
Design new nodemanager config data organization for cluster-wide config
Added by Tom Morris over 6 years ago.
Updated over 4 years ago.
Estimated time:
(Total: 0.00 h)
Description
Add to Cluster configuration
Here's an example of the Puppet Hiera config, this is a yaml file used to populate the current .ini file, so one obvious approach would be to use the heira configuration format in "arvados.yml" instead:
arvados-node-manager:
Logging:
level: DEBUG
apiclient: WARNING
googleapiclient: WARNING
Daemon:
dispatcher: slurm
max_nodes: 8
poll_time: 10
boot_fail_after: 1200
Arvados:
token: (redacted)
timeout: 20
Cloud:
provider: ec2
shutdown_windows: 21, 999999
Cloud Credentials:
key: (redacted)
secret: "(redacted)"
region: us-east-1
timeout: 20
Cloud Create:
ping_host: 4xphq.arvadosapi.com
ex_keyname: ward
image_id: ami-0a01b48b88d14541e
subnet_id: subnet-24f5ad62
security_groups: sg-4fc53e2a
Cloud List:
instance-state-name: running
tag_arvados-class: dynamic-compute
tag_cluster: 4xphq
Size m4.medium:
cores: 1
scratch: 4000
Size m4.large:
cores: 2
scratch: 32000
Size m4.xlarge:
cores: 4
scratch: 80000
- Status changed from New to In Progress
- Status changed from In Progress to New
- Story points set to 0.5
- Description updated (diff)
- Assigned To set to Tom Clegg
- Blocks Feature #13611: Nodemanager reads instance sizes from the system config file to avoid duplication added
- Description updated (diff)
- Target version changed from 2018-07-18 Sprint to 2018-08-01 Sprint
- Assigned To changed from Tom Clegg to Lucas Di Pentima
- Status changed from New to In Progress
- Target version changed from 2018-08-01 Sprint to 2018-08-15 Sprint
Added nodemanager's config items to the arvados-config.yml example.
You can check the differences here: https://dev.arvados.org/projects/arvados/wiki/Cluster_configuration/diff?utf8=✓&version=13&version_from=12&commit=View+differences (although it doesn't show indentation correctly)
As a summary:
- Added
[Logging] section as a top-level LoggingLevel key so that we can use default logging levels, maybe with the possibility of adding per-service values too
[Cloud Credentials] is replaced by the Providers key, I expanded it a little so that it's more clean how we can manage regions/services/credentials- Node sizes were already existing as top-level items
- Moved all cloud node handling specifics inside a
Cloud key on NodeManager's own key
- Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint
- Status changed from In Progress to New
- Assigned To deleted (
Lucas Di Pentima)
- Target version changed from 2018-09-05 Sprint to Arvados Future Sprints
Moving this to the backlog since we're redesigning this corner of the world.
- Target version changed from Arvados Future Sprints to 2019-02-13 Sprint
- Target version changed from 2019-02-13 Sprint to Arvados Future Sprints
- Status changed from New to Closed
- Target version deleted (
Arvados Future Sprints)
Also available in: Atom
PDF
Updated by Tom Morris 
Updated by 
Updated by 
Updated by