- Additional services:
- install/install-ws.html.textile.liquid
- install/install-arv-git-httpd.html.textile.liquid
- - install/install-shell-server.html.textile.liquid
+ - install/install-shell-server.html.textile.liquid
+ - Containers API support on cloud:
+ - install/install-dispatch-cloud.html.textile.liquid
- Containers API support on SLURM:
- install/crunch2-slurm/install-prerequisites.html.textile.liquid
- install/crunch2-slurm/install-slurm.html.textile.liquid
- install/crunch2-slurm/install-compute-node.html.textile.liquid
- install/crunch2-slurm/install-dispatch.html.textile.liquid
- install/crunch2-slurm/install-test.html.textile.liquid
- - install/install-nodemanager.html.textile.liquid
- - install/install-compute-ping.html.textile.liquid
- - Containers API support on cloud (beta):
- - install/install-dispatch-cloud.html.textile.liquid
- External dependencies:
- install/install-postgresql.html.textile.liquid
- install/ruby.html.textile.liquid
Compute nodes must have Docker installed to run containers. This requires a relatively recent version of Linux (at least upstream version 3.10, or a distribution version with the appropriate patches backported). Follow the "Docker Engine installation documentation":https://docs.docker.com/ for your distribution.
-For Debian-based systems, the Arvados package repository includes a backported @docker.io@ package with a known-good version you can install.
-
h2(#configure_docker_daemon). Configure the Docker daemon
Crunch runs Docker containers with relatively little configuration. You may need to start the Docker daemon with specific options to make sure these jobs run smoothly in your environment. This section highlights options that are useful to most installations. Refer to the "Docker daemon reference":https://docs.docker.com/reference/commandline/daemon/ for complete information about all available options.
The best way to configure these options varies by distribution.
-* If you're using our backported @docker.io@ package, you can list these options in the @DOCKER_OPTS@ setting in @/etc/default/docker.io@.
-* If you're using another Debian-based package, you can list these options in the @DOCKER_OPTS@ setting in @/etc/default/docker@.
+* If you're using the Debian package, you can list these options in the @DOCKER_OPTS@ setting in @/etc/default/docker@.
* On Red Hat-based distributions, you can list these options in the @other_args@ setting in @/etc/sysconfig/docker@.
h3. Default ulimits
layout: default
navsection: installguide
title: Install the cloud dispatcher
-
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-The cloud dispatch service is an *experimental* service for running containers on cloud VMs. It eliminates the need for SLURM, Node Manager, and SLURM dispatcher. It works with Microsoft Azure and Amazon EC2; future versions will also support Google Compute Engine.
+# "Update config.yml":#update-config
+# "Install arvados-dispatch-cloud":#install-packages
+# "Confirm working installation":#confirm-working
+
+The cloud dispatch service is for running containers on cloud VMs. It works with Microsoft Azure and Amazon EC2; future versions will also support Google Compute Engine.
The cloud dispatch service can run on any node that can connect to the Arvados API service, the cloud provider's API, and the SSH service on cloud VMs. It is not resource-intensive, so you can run it on the API server node.
</code></pre>
</notextile>
-h2. Create a dispatcher token
-
-If you haven't already done so, create an Arvados superuser token to use as SystemRootToken in your cluster config file.
-
-{% include 'create_superuser_token' %}
+h2(#update-config). Update config.yml
-h2. Create a private key
+h3. Create a private key
Generate an SSH private key with no passphrase. Save it in the cluster configuration file (see @PrivateKey@ in the example below).
</code></pre>
</notextile>
-h2. Configure the dispatcher
+h3. Configure CloudVMs
Add or update the following portions of your cluster configuration file, @/etc/arvados/config.yml@. Refer to "config.defaults.yml":{{site.baseurl}}/admin/config.html for information about additional configuration options.
<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- ManagementToken: xyzzy
- SystemRootToken: <span class="userinput">zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz</span>
- Services:
- Controller:
- ExternalURL: "https://<span class="userinput">uuid_prefix.arvadosapi.com</span>"
+<pre><code> Services:
DispatchCloud:
InternalURLs:
"http://localhost:9006": {}
Minimal configuration example for Amazon EC2:
<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- Containers:
+<pre><code> Containers:
CloudVMs:
ImageID: ami-01234567890abcdef
Driver: ec2
Minimal configuration example for Azure:
<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- Containers:
+<pre><code> Containers:
CloudVMs:
ImageID: "https://zzzzzzzz.blob.core.windows.net/system/Microsoft.Compute/Images/images/zzzzz-compute-osDisk.55555555-5555-5555-5555-555555555555.vhd"
Driver: azure
</code></pre>
</notextile>
-h2. Test your configuration
-
-First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/install-manual-prerequisites.html#repos.
-
-Next, install the arvados-server package.
-
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-server</span>
-</code></pre>
-</notextile>
-
-On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-server</span>
-</code></pre>
-</notextile>
+h3. Test your configuration
Run the @cloudtest@ tool to verify that your configuration works. This creates a new cloud VM, confirms that it boots correctly and accepts your configured SSH private key, and shuts it down.
Refer to the "cloudtest tool documentation":../admin/cloudtest.html for more information.
-h2. Install the dispatcher
+h2. Install arvados-dispatch-cloud
+
+{% include 'notebox_begin' %}
+
+The arvados-dispatch-cloud package includes configuration files for systemd. If you're using a different init system, configure a service to start and stop an @arvados-dispatch-cloud@ process as desired.
-On Red Hat-based systems:
+{% include 'notebox_end' %}
+
+h3. Centos 7
<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-dispatch-cloud</span>
-~$ <span class="userinput">sudo systemctl enable arvados-dispatch-cloud</span>
+<pre><code># <span class="userinput">yum install arvados-dispatch-cloud</span>
</code></pre>
</notextile>
-On Debian-based systems:
+h3. Debian and Ubuntu
<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-dispatch-cloud</span>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-dispatch-cloud</span>
</code></pre>
</notextile>
-{% include 'notebox_begin' %}
-
-The arvados-dispatch-cloud package includes configuration files for systemd. If you're using a different init system, configure a service to start and stop an @arvados-dispatch-cloud@ process as desired.
-
-{% include 'notebox_end' %}
-
-h2. Verify the dispatcher is running
+h2(#confirm-working). Confirm working installation
-Use your ManagementToken to test the dispatcher's metrics endpoint.
+Use your @ManagementToken@ to test the dispatcher's metrics endpoint.
<notextile>
<pre><code>~$ <span class="userinput">token="xyzzy"</span>
projects / arvados.git / commitdiff