Cluster configuration¶

We are (2018) consolidating configuration from per-microservice yaml/json/ini files into a single cluster configuration document that is used by all components.

• Long term: system nodes automatically keep their configs synchronized (using something like consul).
• Short term: sysadmin uses tools like puppet and terraform to ensure /etc/arvados/config.yml is identical on all system nodes.
• Hosts without config files (e.g., hosts outside the cluster) can retrieve the config document from the API server.

Discovery document¶

Previously, we copied selected config values from the API server config into the API discovery document so clients could see them. When clients can get the configuration document itself, this won't be needed. The discovery document should advertise APIs provided by the server, not cluster configuration.

Example config file¶

(Format not yet frozen!)

Clusters:
  xyzzy:
    BlobSigningKey: ungu355able
    BlobSignatureTTL: 172800
    Volumes:
      xyzzy-keep-0:
        Type: s3
        Region: us-east
        Bucket: xyzzy-keep-0
        # [rest of keepstore volume config goes here]
    Providers:
      AWS:
        # [credentials and stuff go here]
    InstanceTypes:
    - Name: m4.large
      VCPUs: 2
      RAM: 8000000000
      Scratch: 31000000000
      Price: 0.1
    - Name: m4.large-1t
      # same instance type as m4.large but our scripts attach more scratch
      ProviderType: m4.large
      VCPUs: 2
      RAM: 8000000000
      Scratch: 999000000000
      Price: 0.12
    - Name: m4.xlarge
      VCPUs: 4
      RAM: 16000000000
      Scratch: 78000000000
      Price: 0.2
    - Name: m4.8xlarge
      VCPUs: 40
      RAM: 160000000000
      Scratch: 156000000000
      Price: 2
    - Name: m4.16xlarge
      VCPUs: 64
      RAM: 256000000000
      Scratch: 310000000000
      Price: 3.2
    - Name: c4.large
      VCPUs: 2
      RAM: 3750000000
      Price: 0.1
    - Name: c4.8xlarge
      VCPUs: 36
      RAM: 60000000000
      Price: 1.591
    NodeProfiles:
      # Key is a profile name; can be specified on service prog command line, defaults to $(hostname)
      keep:
        # Don’t run other services automatically -- only specified ones
        Default: {Disable: true}
        Keepstore: {Listen: ":25107"}
      apiserver:
        Default: {Disable: true}
        RailsAPI: {Listen: ":9000", TLS: true}
        Controller: {Listen: ":9100"}
        Websocket: {Listen: ":9101"}
        Health: {Listen: ":9199"}
      keep:
        Default: {Disable: true}
        KeepProxy: {Listen: ":9102"}
        KeepWeb: {Listen: ":9103"}
      *:
        # This section used for a node whose profile name is not listed above
        Default: {Disable: false} # (this is the default behavior)
    PostgreSQL:
      Host: localhost
      Port: 5432
      Username: arvados
      Password: s3cr3t
      Database: arvados_production
      Encoding: utf8
    WebRoutes:
      # “default” means route according to method/path
      xyzzy.arvadosapi.com: default
      # “collections” means always route to keep-web
      collections.xyzzy.arvadosapi.com: collections
      # leading * is a wildcard (longest match wins)
      "*--collections.xyzzy.arvadosapi.com": collections
      cloud.curoverse.com: workbench
      workbench.xyzzy.arvadosapi.com: workbench
      "*.xyzzy.arvadosapi.com": default
    HTTPRequestTimeout: 5m