Arvados

h1. Crunch2 installation

(DRAFT -- when ready, this will move to doc.arvados.org→install)

{{toc}}

h2. Set up a crunch-dispatch service

Currently, dispatching containers via SLURM is supported.

Install crunch-dispatch-slurm on a node that can submit SLURM jobs. This can be the slurm controller node, a worker node, or any other node that has the appropriate SLURM/munge configuration.

<pre><code class="shell">

sudo apt-get install crunch-dispatch-slurm

</code></pre>

Create a privileged token for use by the dispatcher. If you have multiple dispatch processes, you should give each one a different token.

<pre><code class="shell">

apiserver:~$ cd /var/www/arvados-api/current

apiserver:/var/www/arvados-api/current$ sudo -u webserver-user RAILS_ENV=production bundle exec script/create_superuser_token.rb

zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

</code></pre>

Save the token on the dispatch node, in <code>/etc/sv/env/ARVADOS_API_TOKEN</code>

Example runit script:

<pre><code class="shell">

#!/bin/sh

export ARVADOS_API_HOST=uuid_prefix.your.domain

exec chpst -e ./env -u crunch crunch-dispatch-slurm

</code></pre>

Ensure the @crunch@ user exists -- and has the same UID, GID, and home directory -- on the dispatch node and all SLURM compute nodes. Ensure the @crunch@ user can run docker containers on SLURM compute nodes.

h2. Install crunch-run on all compute nodes

<pre><code class="shell">

sudo apt-get install crunch-run

</code></pre>

h2. Enable cgroup accounting on all compute nodes

Check https://docs.docker.com/engine/installation/linux/ for instructions specific to your distribution.

For example, on Ubuntu:

# Update @/etc/default/grub@ to include: <pre>

GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"

</pre>

# @sudo update-grub@

# Reboot

h2. Configure docker

h2. Test the dispatcher