X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/e513251f7b0f6acdc0c0d6df5792c18358030221..c5cb2cebf5912642dc576760eaa687fa1d9c9cce:/doc/install/install-compute-node.html.textile.liquid diff --git a/doc/install/install-compute-node.html.textile.liquid b/doc/install/install-compute-node.html.textile.liquid index 7eff16ba56..7d13f773a6 100644 --- a/doc/install/install-compute-node.html.textile.liquid +++ b/doc/install/install-compute-node.html.textile.liquid @@ -8,22 +8,22 @@ h2. Install dependencies First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/install-manual-prerequisites.html#repos. +{% include 'note_python27_sc' %} + On Debian-based systems: -
~$ sudo apt-get install perl python-virtualenv fuse python-arvados-python-client python-arvados-fuse crunchstat iptables ca-certificates
+
~$ sudo apt-get install perl python-virtualenv fuse python-arvados-python-client python-arvados-fuse crunchstat arvados-docker-cleaner iptables ca-certificates
 
On Red Hat-based systems: -
~$ sudo yum install perl python27-python-virtualenv fuse python27-python-arvados-python-client python27-python-arvados-fuse crunchstat iptables ca-certificates
+
~$ sudo yum install perl python27-python-virtualenv fuse python27-python-arvados-python-client python27-python-arvados-fuse crunchstat arvados-docker-cleaner iptables ca-certificates
 
-{% include 'note_python27_sc' %} - h2. Install Docker Compute nodes must have Docker installed to run jobs inside containers. This requires a relatively recent version of Linux (at least upstream version 3.10, or a distribution version with the appropriate patches backported). Follow the "Docker Engine installation documentation":https://docs.docker.com/ for your distribution. @@ -74,7 +74,49 @@ user_allow_other
-h2. Crunch user account +h2. Configure the Docker cleaner + +The arvados-docker-cleaner program removes least recently used docker images as needed to keep disk usage below a configured limit. + +{% include 'notebox_begin' %} +This also removes all containers as soon as they exit, as if they were run with `docker run --rm`. If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with `--remove-stopped-containers never`. +{% include 'notebox_end' %} + +On Debian-based systems, install runit: + + +
~$ sudo apt-get install runit
+
+
+ +On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor. + +Configure runit to run the image cleaner using a suitable quota for your compute nodes and workload: + + +
~$ cd /etc/sv
+/etc/sv$ sudo mkdir arvados-docker-cleaner; cd arvados-docker-cleaner
+/etc/sv/arvados-docker-cleaner$ sudo mkdir log log/main
+/etc/sv/arvados-docker-cleaner$ sudo sh -c 'cat >log/run' <<'EOF'
+#!/bin/sh
+exec svlogd -tt main
+EOF
+/etc/sv/arvados-docker-cleaner$ sudo sh -c 'cat >run' <<'EOF'
+#!/bin/sh
+exec python3 -m arvados_docker.cleaner --quota 50G
+EOF
+/etc/sv/arvados-docker-cleaner$ sudo chmod +x run log/run
+
+
+ +If you are using a different daemon supervisor, or if you want to test the daemon in a terminal window, an equivalent shell command to run arvados-docker-cleaner is: + + +
python3 -m arvados_docker.cleaner --quota 50G
+
+
+ +h2. Add a Crunch user account Create a Crunch user account, and add it to the @fuse@ and @docker@ groups so it can use those tools: