X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/8884a9fff1ee4d5f6df3fc3ef43aed3a9eec9ea2..e39e11d5850c1a2509f377e2f2551105507f59e3:/doc/install/install-compute-node.html.textile.liquid diff --git a/doc/install/install-compute-node.html.textile.liquid b/doc/install/install-compute-node.html.textile.liquid index dd64cd03a0..767b8e3b81 100644 --- a/doc/install/install-compute-node.html.textile.liquid +++ b/doc/install/install-compute-node.html.textile.liquid @@ -4,38 +4,41 @@ navsection: installguide title: Install a compute node ... -This installation guide assumes you are on a 64 bit Debian or Ubuntu system. - h2. Install dependencies -First add the Arvados apt repository, and then install a number of packages. +First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/install-manual-prerequisites.html#repos. + +On Debian-based systems: -
~$ echo "deb http://apt.arvados.org/ wheezy main" | sudo tee /etc/apt/sources.list.d/apt.arvados.org.list
-~$ sudo /usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7
-~$ sudo /usr/bin/apt-get update
-~$ sudo /usr/bin/apt-get install python-pip python-pyvcf python-gflags python-google-api-python-client python-virtualenv libattr1-dev libfuse-dev python-dev python-llfuse fuse crunchstat python-arvados-fuse iptables ca-certificates lxc apt-transport-https docker.io
+
~$ sudo apt-get install perl python-virtualenv fuse python-arvados-python-client python-arvados-fuse crunchstat iptables ca-certificates
 
-h2. Install slurm and munge +On Red Hat-based systems: -
~$ sudo /usr/bin/apt-get install slurm-llnl munge
+
~$ sudo yum install perl python27-python-virtualenv fuse python27-python-arvados-python-client python27-python-arvados-fuse crunchstat iptables ca-certificates
 
-h2. Copy configuration files from the dispatcher (api) +{% include 'note_python27_sc' %} -The @/etc/slurm-llnl/slurm.conf@ and @/etc/munge/munge.key@ files need to be identicaly across the dispatcher and all compute nodes. Copy the files you created in the "Install the Crunch dispatcher":{{site.baseurl}} step to this compute node. +h2. Set up Docker -h2. Crunch user account +Compute nodes must have Docker installed to run jobs inside containers. This requires a relatively recent version of Linux (at least upstream version 3.10, or a distribution version with the appropriate patches backported). Follow the "Docker Engine installation documentation":https://docs.docker.com/ for your distribution. -* @adduser crunch@ +For Debian-based systems, the Arvados package repository includes a backported @docker.io@ package with a known-good version you can install. -The crunch user should have the same UID, GID, and home directory on all compute nodes and on the dispatcher (api server). +h2. Set up SLURM + +Install SLURM following "the same process you used to install the Crunch dispatcher":{{ site.baseurl }}/install/install-crunch-dispatch.html#slurm. + +h2. Copy configuration files from the dispatcher (API server) + +The @/etc/slurm-llnl/slurm.conf@ and @/etc/munge/munge.key@ files need to be identicaly across the dispatcher and all compute nodes. Copy the files you created in the "Install the Crunch dispatcher":{{site.baseurl}} step to this compute node. -h2. Configure fuse +h2. Configure FUSE Install this file as @/etc/fuse.conf@: @@ -53,6 +56,17 @@ user_allow_other
+h2. Crunch user account + +Create a Crunch user account, and add it to the @fuse@ and @docker@ groups so it can use those tools: + + +
~$ sudo useradd --groups fuse,docker crunch
+
+
+ +The crunch user should have the same UID, GID, and home directory across all compute nodes and the dispatcher (API server). + h2. Tell the API server about this compute node Load your API superuser token on the compute node: @@ -72,8 +86,14 @@ Then execute this script to create a compute node object, and set up a cron job

 #!/bin/bash
+set -e
 if ! test -f /root/node.json ; then
-    arv node create --node "{\"hostname\": \"$(hostname)\"}" > /root/node.json
+    python - <<EOF
+import arvados, json, socket
+node = arvados.api('v1').nodes().create(body={'hostname': socket.gethostname()}).execute()
+with open('/root/node.json', 'w') as node_file:
+    json.dump(node, node_file, indent=2)
+EOF
 
     # Make sure /dev/fuse permissions are correct (the device appears after fuse is loaded)
     chmod 1660 /dev/fuse && chgrp fuse /dev/fuse