Arvados

h1. Cluster configuration

We are (2018) consolidating configuration from per-microservice yaml/json/ini files into a single cluster configuration document that is used by all components.

* Long term: system nodes automatically keep their configs synchronized (using something like consul).

* Short term: sysadmin uses tools like puppet and terraform to ensure /etc/arvados/config.yml is identical on all system nodes.

* Hosts without config files (e.g., hosts outside the cluster) can retrieve the config document from the API server.

h2. Discovery document

Previously, we copied selected config values from the API server config into the API discovery document so clients could see them. When clients can get the configuration document itself, this won't be needed. The discovery document should advertise APIs provided by the server, not cluster configuration.

h2. Example config file

(Format not yet frozen!)

<pre><code class="yaml">

Clusters:

  xyzzy:

    BlobSigningKey: ungu355able

    BlobSignatureTTL: 172800

    Volumes:

      xyzzy-keep-0:

        Type: s3

        Region: us-east

        Bucket: xyzzy-keep-0

        # [rest of keepstore volume config goes here]

    Providers:

      AWS:

        # [credentials and stuff go here]

    InstanceTypes:

    - Name: m4.large

      VCPUs: 2

      RAM: 8000000000

      Scratch: 31000000000

      Price: 0.1

    - Name: m4.large-1t

      # same instance type as m4.large but our scripts attach more scratch

      ProviderType: m4.large

      VCPUs: 2

      RAM: 8000000000

      Scratch: 999000000000

      Price: 0.12

    - Name: m4.xlarge

      VCPUs: 4

      RAM: 16000000000

      Scratch: 78000000000

      Price: 0.2

    - Name: m4.8xlarge

      VCPUs: 40

      RAM: 160000000000

      Scratch: 156000000000

      Price: 2

    - Name: m4.16xlarge

      VCPUs: 64

      RAM: 256000000000

      Scratch: 310000000000

      Price: 3.2

    - Name: c4.large

      VCPUs: 2

      RAM: 3750000000

      Price: 0.1

    - Name: c4.8xlarge

      VCPUs: 36

      RAM: 60000000000

      Price: 1.591

    SystemNodes:

      # Key is a class/type/profile, specified on command line, defaulting to hostname

      keep:

        # Don’t run other services automatically -- only specified ones

        Default: {Disable: true}

        Keepstore: {Listen: ":25107"}

      apiserver:

        Default: {Disable: true}

        RailsAPI: {Listen: ":9000", TLS: true}

        Controller: {Listen: ":9100"}

        Websocket: {Listen: ":9101"}

        Health: {Listen: ":9199"}

      keep:

        Default: {Disable: true}

        KeepProxy: {Listen: ":9102"}

        KeepWeb: {Listen: ":9103"}

      *:

        # This section used for a system node not listed above

        Default: {Disable: false} # (this is the default behavior)

    PostgreSQL:

      Host: localhost

      Port: 5432

      Username: arvados

      Password: s3cr3t

      Database: arvados_production

      Encoding: utf8

    DNS:

      # “default” means route according to method/path

      xyzzy.arvadosapi.com: default

      # “collections” means always route to keep-web

      collections.xyzzy.arvadosapi.com: collections

      # leading * is a wildcard (longest match wins)

      "*--collections.xyzzy.arvadosapi.com": collections

      cloud.curoverse.com: workbench

      workbench.xyzzy.arvadosapi.com: workbench

      "*.xyzzy.arvadosapi.com": default

    HTTPRequestTimeout: 5m

</code></pre>