X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/b7ec383316ca68220f9f54fe1602766284854173..9fcd7e96ad9ed8c7f3a343663a06ee9cbaa15aa9:/doc/install/install-crunch-dispatch.html.textile.liquid

diff --git a/doc/install/install-crunch-dispatch.html.textile.liquid b/doc/install/install-crunch-dispatch.html.textile.liquid
index 231d1f45e8..46ce04289e 100644
--- a/doc/install/install-crunch-dispatch.html.textile.liquid
+++ b/doc/install/install-crunch-dispatch.html.textile.liquid
@@ -5,37 +5,99 @@ title: Install the Crunch dispatcher
 
 ...
 
-
-
 The dispatcher normally runs on the same host/VM as the API server.
 
-h4. Perl SDK dependencies
+h2. Perl SDK dependencies
 
 Install the Perl SDK on the controller.
 
 * See "Perl SDK":{{site.baseurl}}/sdk/perl/index.html page for details.
 
-h4. Python SDK dependencies
+h2. Python SDK dependencies
 
 Install the Python SDK and CLI tools on controller and all compute nodes.
 
 * See "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html page for details.
 
-h4. Likely crunch job dependencies
+h2(#slurm). Set up SLURM
+
+On the API server, install SLURM and munge, and generate a munge key.
+
+On Debian-based systems:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo /usr/bin/apt-get install slurm-llnl munge</span>
+~$ <span class="userinput">sudo /usr/sbin/create-munge-key</span>
+</code></pre>
+</notextile>
 
-On compute nodes:
+On Red Hat-based systems, "install SLURM and munge from source following their installation guide":https://computing.llnl.gov/linux/slurm/quickstart_admin.html.
 
-* @pip install --upgrade pyvcf@
+Now we need to give SLURM a configuration file in @/etc/slurm-llnl/slurm.conf@. Here's an example:
 
-h4. Crunch user account
+<notextile>
+<pre>
+ControlMachine=uuid_prefix.your.domain
+SlurmctldPort=6817
+SlurmdPort=6818
+AuthType=auth/munge
+StateSaveLocation=/tmp
+SlurmdSpoolDir=/tmp/slurmd
+SwitchType=switch/none
+MpiDefault=none
+SlurmctldPidFile=/var/run/slurmctld.pid
+SlurmdPidFile=/var/run/slurmd.pid
+ProctrackType=proctrack/pgid
+CacheGroups=0
+ReturnToService=2
+TaskPlugin=task/affinity
+#
+# TIMERS
+SlurmctldTimeout=300
+SlurmdTimeout=300
+InactiveLimit=0
+MinJobAge=300
+KillWait=30
+Waittime=0
+#
+# SCHEDULING
+SchedulerType=sched/backfill
+SchedulerPort=7321
+SelectType=select/cons_res
+SelectTypeParameters=CR_CPU_Memory
+FastSchedule=1
+#
+# LOGGING
+SlurmctldDebug=3
+#SlurmctldLogFile=
+SlurmdDebug=3
+#SlurmdLogFile=
+JobCompType=jobcomp/none
+#JobCompLoc=
+JobAcctGatherType=jobacct_gather/none
+#
+# COMPUTE NODES
+NodeName=DEFAULT
+PartitionName=DEFAULT MaxTime=INFINITE State=UP
+PartitionName=compute Default=YES Shared=yes
+
+NodeName=compute[0-255]
+
+PartitionName=compute Nodes=compute[0-255]
+</pre>
+</notextile>
+
+Please make sure to update the value of the @ControlMachine@ parameter to the hostname of your dispatcher (API server).
 
-On compute nodes and controller:
+h2. Enable SLURM job dispatch
 
-* @adduser crunch@
+In your API server's @application.yml@ configuration file, add the line @crunch_job_wrapper: :slurm_immediate@ under the appropriate section.  (The second colon is not a typo.  It denotes a Ruby symbol.)
 
-The crunch user should have the same UID, GID, and home directory on all compute nodes and on the controller.
+h2. Crunch user account
 
-h4. Repositories
+Run @sudo adduser crunch@.  The crunch user should have the same UID, GID, and home directory on all compute nodes and on the dispatcher (API server).
+
+h2. Git Repositories
 
 Crunch scripts must be in Git repositories in the directory configured as @git_repositories_dir@/*.git (see the "API server installation":install-api-server.html#git_repositories_dir).
 
@@ -56,12 +118,12 @@ Without getting this error:
 ArgumentError: Specified script_version does not resolve to a commit
 </pre>
 
-h4. Running jobs
+h2. Running jobs
 
 * @services/api/script/crunch-dispatch.rb@ must be running.
 * @crunch-dispatch.rb@ needs @services/crunch/crunch-job@ in its @PATH@.
-* @crunch-job@ needs @sdk/perl/lib@ and @warehouse-apps/libwarehouse-perl/lib@ in its @PERLLIB@
-* @crunch-job@ needs @ARVADOS_API_HOST@ (and, if necessary in a development environment, @ARVADOS_API_HOST_INSECURE@)
+* @crunch-job@ needs the installation path of the Perl SDK in its @PERLLIB@.
+* @crunch-job@ needs the @ARVADOS_API_HOST@ (and, if necessary in a development environment, @ARVADOS_API_HOST_INSECURE@) environment variable set.
 
 Example @/var/service/arvados_crunch_dispatch/run@ script:
 
@@ -77,6 +139,10 @@ export PATH="$PATH":/path/to/arvados/services/crunch
 export ARVADOS_API_HOST={{ site.arvados_api_host }}
 export CRUNCH_DISPATCH_LOCKFILE=/var/lock/crunch-dispatch
 
+# This is the path to docker on your compute nodes. You might need to
+# change it to "docker", "/opt/bin/docker", etc.
+export CRUNCH_JOB_DOCKER_BIN=docker.io
+
 fuser -TERM -k $CRUNCH_DISPATCH_LOCKFILE || true
 
 ## Only if your SSL cert is unverifiable: