Arvados

h1. Development and testing with Docker

Arvados is primarily intended as a system to be run on large server clusters.  For development and testing purposes, we have provided a process to build and deploy the Arvados system in "Docker":https://www.docker.io/ containers on a single machine.

http://doc.arvados.org/install/install-docker.html shows how to install and update a development system.

h2. Additional sources of information

The "docker/README.md":https://github.com/curoverse/arvados/blob/master/docker/README.md file in the source distribution has more notes.

h2. Installing the client libraries

@./install_sdk.sh@ (in the docker directory) will install the CLI tools like arv and arv-put. If that does not work for you, try installing these by hand:

* *RVM* -- if you do not already have RVM installed on your workstation, install it now.

<pre>

sudo apt-get install curl libcurl3 libcurl3-gnutls libcurl4-openssl-dev python-pip

curl -sSL https://get.rvm.io | bash -s stable

source ~/.rvm/scripts/rvm

rvm install 2.1.0

</pre>

* The arvados-cli gem:

<pre>

gem install arvados-cli

sudo pip install --upgrade httplib2

</pre>

h2. Debugging containers

Once it's running, you can get run a shell inside a container using nsenter.

Install nsenter:

<pre>

docker run --rm jpetazzo/nsenter cat /nsenter > /tmp/nsenter

sudo install -m 0755 /tmp/nsenter /usr/local/bin/

</pre>

Install this shortcut:

<pre>

sudo tee /usr/local/bin/nse <<'EOF'

#!/bin/sh

PID=$(docker inspect --format {{.State.Pid}} "$*")

exec nsenter --target $PID --mount --uts --ipc --net --pid

EOF

chmod +x /usr/local/bin/nse

</pre>

Then you can run a shell in a container like this:

<pre>

nse keep_server_0

</pre>

h3. Console

@./arvdock@ starts containers in detached mode. You can attach your terminal to them in order to see stdout/stderr:

<pre>

docker attach --sig-proxy=false keep_server_0

</pre>

Press control-C to detach. (If you didn't use --sig-proxy=false, control-C will send SIGINT into the container.)

h2. Tutorials

Once you have installed the servers and started them with @arvdock start@, and installed the client libraries, you should be able to work through the "user tutorials":http://doc.arvados.org/user/ (using your host instead of a shell VM) to upload data into Arvados and run Crunch jobs in local development mode.