Arvados

Rewrote the code as discussed with @cure. Branched from #18791 and modified to manage different certs depending on single/multiple hostnames.

Tested locally on vagrant, with snakeoil certs, and it passes.
Tested on AWS, with a single instance VM and LE certs and it passes.

CAVEAT: LE, by policy, does not issue certificates to AWS domain names, so a valid, non-aws domain name needs to be assigned to the node.

Attached are the config files used for testing and the output file of the run, following instructions described in https://doc.arvados.org/install/salt-single-host.html

Javier Bértoli wrote:

Rewrote the code as discussed with @cure. Branched from #18791 and modified to manage different certs depending on single/multiple hostnames.

Tested locally on vagrant, with snakeoil certs, and it passes.
Tested on AWS, with a single instance VM and LE certs and it passes.

CAVEAT: LE, by policy, does not issue certificates to AWS domain names, so a valid, non-aws domain name needs to be assigned to the node.

• Well, that seems like a worthwhile thing to add to our documentation. Maybe something like this:

Please note: When using AWS, EC2 instances can have a default hostname that ends with `amazonaws.com`. Let's Encrypt has a blacklist of domain names for which it will not issue certificates, and that blacklist includes the `amazonaws.com` domain. In order to use Let's Encrypt certificates on AWS EC2, you will need to bring your own domain name and point a hostname in that domain at your EC2 instance.

Attached are the config files used for testing and the output file of the run, following instructions described in https://doc.arvados.org/install/salt-single-host.html

• in `tools/salt-install/config_examples/single_host/single_hostname/states/snakeoil_certs.sls` line 5-6

+# WARNING: This file is only used for testing purposes, and should not be used
+# in a production environment

Just remove that please. It's not true (who are we to assume the user's purpose?) and it doesn't convey any useful information here. The issues with self-signed certificates are pretty obvious/visible and any caveats belong in the documentation.

• in `tools/salt-install/local.params.example.single_host_single_hostname`

+# If you going to provide your own certificates for Arvados, the provision script can
+# help you deploy them. In order to do that, you need to set `SSL_MODE=bring-your-own` above,
+# and copy the required certificates under the directory specified in the next line.
+# The certs will be copied from this directory by the provision script.
 # The directory to check for the config files (pillars, states) you want to use.
 # There are a few examples under 'config_examples'.
 # CONFIG_DIR="local_config_dir" 

All these comments are confusing. The "next line" is another comment, so... From all this I can't actually say where the certs need to go. Is the definition of CUSTOM_CERTS_DIR simply missing? The example in
tools/salt-install/local.params.example.single_host_multiple_hostnames has even more comments but also a line defining CUSTOM_CERTS_DIR.

• For the single host `bring-your-own` certificates case, please add to the docs where the cert file needs to go and what it should be called.

I'm still doing testing but overall it looks good with the above changes. I'll update this when I am done testing.

Addressed comments on commits 3d3790ee6..3a26616ce, branch 18785-single-host-single-name-with-letsencrypt

I pushed a number of followon changes, ready for another look in fc1bcfc9058f6bcdcbfe22fef7c85ef2b683af76 on branch 18785-single-host-single-name-with-letsencrypt.

Added a test for the presence of /etc/cloud/cloud.cfg.d in 0eb3e44248cce822c97aff3882be37830ca9502b on branch 18785-single-host-single-name-with-letsencrypt

@cure, it lgtm.