Project

General

Profile

Actions
Copy link

Idea #13731

closed

Design new nodemanager config data organization for cluster-wide config

Added by Tom Morris over 7 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Start date:
08/13/2018
Due date:
Story points:
0.5

Description

Add to Cluster configuration

Here's an example of the Puppet Hiera config, this is a yaml file used to populate the current .ini file, so one obvious approach would be to use the heira configuration format in "arvados.yml" instead:

arvados-node-manager:
  Logging:
    level: DEBUG
    apiclient: WARNING
    googleapiclient: WARNING
  Daemon:
    dispatcher: slurm
    max_nodes: 8
    poll_time: 10
    boot_fail_after: 1200
  Arvados:
    token: (redacted)
    timeout: 20
  Cloud:
    provider: ec2
    shutdown_windows: 21, 999999
  Cloud Credentials:
    key: (redacted)
    secret: "(redacted)" 
    region: us-east-1
    timeout: 20
  Cloud Create:
    ping_host: 4xphq.arvadosapi.com
    ex_keyname: ward
    image_id: ami-0a01b48b88d14541e
    subnet_id: subnet-24f5ad62
    security_groups: sg-4fc53e2a
  Cloud List:
    instance-state-name: running
    tag_arvados-class: dynamic-compute
    tag_cluster: 4xphq
  Size m4.medium:
    cores: 1
    scratch: 4000
  Size m4.large:
    cores: 2
    scratch: 32000
  Size m4.xlarge:
    cores: 4
    scratch: 80000

Subtasks 1 (0 open — 1 closed)

Task #13732: ReviewClosedNico César08/13/2018Actions

Related issues 1 (0 open — 1 closed)

Blocks Arvados - Feature #13611: Nodemanager reads instance sizes from the system config file to avoid duplicationClosedActions
Actions
Copy link
 #1

Updated by Tom Morris over 7 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #2

Updated by Tom Morris over 7 years ago

  • Status changed from In Progress to New
  • Story points set to 0.5
Actions
Copy link
 #3

Updated by Tom Clegg over 7 years ago

  • Description updated (diff)
Actions
Copy link
 #4

Updated by Tom Clegg over 7 years ago

  • Assigned To set to Tom Clegg
Actions
Copy link
 #5

Updated by Tom Clegg over 7 years ago

  • Blocks Feature #13611: Nodemanager reads instance sizes from the system config file to avoid duplication added
Actions
Copy link
 #6

Updated by Peter Amstutz over 7 years ago

  • Description updated (diff)
Actions
Copy link
 #7

Updated by Tom Clegg over 7 years ago

  • Target version changed from 2018-07-18 Sprint to 2018-08-01 Sprint
Actions
Copy link
 #8

Updated by Lucas Di Pentima over 7 years ago

  • Assigned To changed from Tom Clegg to Lucas Di Pentima
Actions
Copy link
 #9

Updated by Lucas Di Pentima over 7 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #10

Updated by Lucas Di Pentima over 7 years ago

  • Target version changed from 2018-08-01 Sprint to 2018-08-15 Sprint
Actions
Copy link
 #11

Updated by Lucas Di Pentima over 7 years ago

Added nodemanager's config items to the arvados-config.yml example.
You can check the differences here: https://dev.arvados.org/projects/arvados/wiki/Cluster_configuration/diff?utf8=✓&version=13&version_from=12&commit=View+differences (although it doesn't show indentation correctly)

As a summary:

  • Added [Logging] section as a top-level LoggingLevel key so that we can use default logging levels, maybe with the possibility of adding per-service values too
  • [Cloud Credentials] is replaced by the Providers key, I expanded it a little so that it's more clean how we can manage regions/services/credentials
  • Node sizes were already existing as top-level items
  • Moved all cloud node handling specifics inside a Cloud key on NodeManager's own key
Actions
Copy link
 #12

Updated by Lucas Di Pentima over 7 years ago

  • Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint
Actions
Copy link
 #13

Updated by Tom Morris over 7 years ago

  • Status changed from In Progress to New
  • Assigned To deleted (Lucas Di Pentima)
  • Target version changed from 2018-09-05 Sprint to Arvados Future Sprints

Moving this to the backlog since we're redesigning this corner of the world.

Actions
Copy link
 #14

Updated by Tom Morris about 7 years ago

  • Target version changed from Arvados Future Sprints to 2019-02-13 Sprint
Actions
Copy link
 #15

Updated by Tom Morris about 7 years ago

  • Target version changed from 2019-02-13 Sprint to Arvados Future Sprints
Actions
Copy link
 #16

Updated by Peter Amstutz over 6 years ago

  • Status changed from New to Closed
Actions
Copy link
 #17

Updated by Peter Amstutz over 5 years ago

  • Target version deleted (Arvados Future Sprints)
Actions
Copy link

Also available in: Atom PDF