Arvados

On the topic of metrics and health checks, we should add a page to the "Admin" section of documentation that describes which components have endpoints and how to use the health check aggregator. That would address the situation where no one remembers where we are with the project of implementing health checks / metrics, at least it would be written down.

Documenting health checks / metrics story: #13791

13198-keep-web-metrics @ 0011b5236fc9a562bc13f943f9a431c496b2b7cd https://ci.curoverse.com/job/developer-run-tests/815/

Tried manually and all seems to work great.
One question though: Are the "time to fetch block" and "cache hits/misses" going to be implemented later / discarded? If yes, then it LGTM.

Right, this branch (just merged) only offers request timing.

Keeping issue open for the keep-web-specific metrics.

13198-keep-web-metrics @ 413db07b4c81ea08663f90f31ee03227349d2be4 https://ci.curoverse.com/view/All/job/developer-run-tests/832/

This doesn't have "time to fetch block". It just exports the cache metrics we were already collecting (and exporting in status.json) and exports them as prometheus counters/gauges.

This LGTM, but it I think it would be nice to avoid computing increments twice for every counter, couldn't /status.json be some sort of prometheus client?

Updates at 6db4c94527903f403e77ba4fce2d1d5fe4e29b03 LGTM.

• adds keep-web section to https://doc.arvados.org/admin/metrics.html

Just one comment:

• I noticed that the documentation style changed with these new additions. All other items are listed in a tabular way instead of bullet point lists, I think the tabular way is clearer to read.

Other than that, lgtm.