8831: Add crunchrunner to compute node dependencies.

[arvados.git] / doc / install / install-compute-node.html.textile.liquid

diff --git a/doc/install/install-compute-node.html.textile.liquid b/doc/install/install-compute-node.html.textile.liquid
index aa4f37d639704f33dc10b2f9e71db1c36a6c129a..a2e3e01ecb117aa0170e1349914dfe9b3a07d7ca 100644 (file)
--- a/doc/install/install-compute-node.html.textile.liquid
+++ b/doc/install/install-compute-node.html.textile.liquid
@@ -8,22 +8,22 @@ h2. Install dependencies
 
 First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/install-manual-prerequisites.html#repos.
 
 
 First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/install-manual-prerequisites.html#repos.
 

+{% include 'note_python27_sc' %}
+

 On Debian-based systems:
 
 <notextile>
 On Debian-based systems:
 
 <notextile>

-<pre><code>~$ <span class="userinput">sudo apt-get install perl python-virtualenv fuse python-arvados-python-client python-arvados-fuse crunchstat arvados-docker-cleaner iptables ca-certificates</span>
+<pre><code>~$ <span class="userinput">sudo apt-get install perl python-virtualenv fuse python-arvados-python-client python-arvados-fuse crunchrunner crunchstat arvados-docker-cleaner iptables ca-certificates</span>

 </code></pre>
 </notextile>
 
 On Red Hat-based systems:
 
 <notextile>
 </code></pre>
 </notextile>
 
 On Red Hat-based systems:
 
 <notextile>

-<pre><code>~$ <span class="userinput">sudo yum install perl python27-python-virtualenv fuse python27-python-arvados-python-client python27-python-arvados-fuse crunchstat arvados-docker-cleaner iptables ca-certificates</span>
+<pre><code>~$ <span class="userinput">sudo yum install perl python27-python-virtualenv fuse python27-python-arvados-python-client python27-python-arvados-fuse crunchrunner crunchstat arvados-docker-cleaner iptables ca-certificates</span>

 </code></pre>
 </notextile>
 
 </code></pre>
 </notextile>
 

-{% include 'note_python27_sc' %}
-

 h2. Install Docker
 
 Compute nodes must have Docker installed to run jobs inside containers.  This requires a relatively recent version of Linux (at least upstream version 3.10, or a distribution version with the appropriate patches backported).  Follow the "Docker Engine installation documentation":https://docs.docker.com/ for your distribution.
 h2. Install Docker
 
 Compute nodes must have Docker installed to run jobs inside containers.  This requires a relatively recent version of Linux (at least upstream version 3.10, or a distribution version with the appropriate patches backported).  Follow the "Docker Engine installation documentation":https://docs.docker.com/ for your distribution.


@@ -54,7 +54,7 @@ Install SLURM following "the same process you used to install the Crunch dispatc
 
 h2. Copy configuration files from the dispatcher (API server)
 
 
 h2. Copy configuration files from the dispatcher (API server)
 

-The @/etc/slurm-llnl/slurm.conf@ and @/etc/munge/munge.key@ files need to be identicaly across the dispatcher and all compute nodes. Copy the files you created in the "Install the Crunch dispatcher":install-crunch-dispatch.html step to this compute node.
+The @slurm.conf@ and @/etc/munge/munge.key@ files need to be identical across the dispatcher and all compute nodes. Copy the files you created in the "Install the Crunch dispatcher":install-crunch-dispatch.html step to this compute node.

 
 h2. Configure FUSE
 
 
 h2. Configure FUSE
 


@@ -79,22 +79,16 @@ h2. Configure the Docker cleaner
 The arvados-docker-cleaner program removes least recently used docker images as needed to keep disk usage below a configured limit.
 
 {% include 'notebox_begin' %}
 The arvados-docker-cleaner program removes least recently used docker images as needed to keep disk usage below a configured limit.
 
 {% include 'notebox_begin' %}

-This also removes all containers as soon as they exit, as if they were run with `docker run --rm`. If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with `--remove-stopped-containers never`.
+This also removes all containers as soon as they exit, as if they were run with @docker run --rm@. If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with @--remove-stopped-containers never@.

 {% include 'notebox_end' %}
 
 {% include 'notebox_end' %}
 

-On Debian-based systems, install runit:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install runit</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor.
+Install runit to supervise the Docker cleaner daemon.  {% include 'install_runit' %}

 
 Configure runit to run the image cleaner using a suitable quota for your compute nodes and workload:
 
 <notextile>
 
 Configure runit to run the image cleaner using a suitable quota for your compute nodes and workload:
 
 <notextile>

-<pre><code>~$ <span class="userinput">cd /etc/sv</span>
+<pre><code>~$ <span class="userinput">sudo mkdir -p /etc/sv</span>
+~$ <span class="userinput">cd /etc/sv</span>

 /etc/sv$ <span class="userinput">sudo mkdir arvados-docker-cleaner; cd arvados-docker-cleaner</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo mkdir log log/main</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo sh -c 'cat &gt;log/run' &lt;&lt;'EOF'
 /etc/sv$ <span class="userinput">sudo mkdir arvados-docker-cleaner; cd arvados-docker-cleaner</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo mkdir log log/main</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo sh -c 'cat &gt;log/run' &lt;&lt;'EOF'


@@ -106,6 +100,7 @@ EOF</span>
 exec python3 -m arvados_docker.cleaner --quota <b>50G</b>
 EOF</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo chmod +x run log/run</span>
 exec python3 -m arvados_docker.cleaner --quota <b>50G</b>
 EOF</span>
 /etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo chmod +x run log/run</span>

+/etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo ln -s "$(pwd)" /etc/service/</span>

 </code></pre>
 </notextile>
 
 </code></pre>
 </notextile>
 


@@ -152,8 +147,7 @@ if ! test -f /root/node.json ; then
 import arvados, json, socket
 fqdn = socket.getfqdn()
 hostname, _, domain = fqdn.partition('.')
 import arvados, json, socket
 fqdn = socket.getfqdn()
 hostname, _, domain = fqdn.partition('.')

-ip_address = socket.gethostbyname(fqdn)
-node = arvados.api('v1').nodes().create(body={'hostname': hostname, 'domain': domain, 'ip_address': ip_address}).execute()
+node = arvados.api('v1').nodes().create(body={'hostname': hostname, 'domain': domain}).execute()

 with open('/root/node.json', 'w') as node_file:
     json.dump(node, node_file, indent=2)
 EOF
 with open('/root/node.json', 'w') as node_file:
     json.dump(node, node_file, indent=2)
 EOF


@@ -183,4 +177,3 @@ And remove your token from the environment:
 </code>
 </pre>
 </notextile>
 </code>
 </pre>
 </notextile>

-