--- /dev/null
+---
+layout: default
+navsection: installguide
+title: Configuration file
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+The configuration file is normally found at @/etc/arvados/config.yml@. This configuration file should be kept in sync across every node in the cluster, except compute nodes (which usually do not require config.yml). We recommend using a devops configuration management tool such as Puppet to synchronize the config file.
+
+h2. Syntax
+
+The configuration file is in "YAML":https://yaml.org/ format. This is a block syntax where indentation is significant (similar to Python). By convention we use two space indent. The first line of the file is always "Clusters:", underneath it at the first indent level is the cluster id. All the actual cluster configuration comes under the cluster id. This means all configuration parameters are indented by at least two levels (four spaces). Comments start with @#@ . Example
+
+<pre>
+Clusters: # Clusters block, everything else is listed under this
+ abcde: # Cluster ID
+ ExampleConfigKey: "fghijk" # An example configuration string for cluster 'abcde'
+ ExampleConfigGroup: # A group of keys
+ ExampleDurationConfig: 12s # Example duration
+ ExampleSizeConfig: 99KiB # Example with a size suffix
+</pre>
+
+Each configuration group may only appear once. When a configuration key is within a config group, it will be written with the group name leading, for example @ExampleConfigGroup.ExampleSizeConfig@.
+
+Duration suffixes are s=seconds, m=minutes or h=hours.
+
+Size suffixes are K=10 ^3^, Ki=2 ^10^ , M=10 ^6^, Mi=2 ^20^, G=10 ^9^, Gi=2 ^30^, T=10 ^12^, Ti=2 ^40^, P=10 ^15^, Pi=2 ^50^, E=10 ^18^, Ei=2 ^60^. You can optionally follow with a "B" (eg "MB" or "MiB") for readability (it does not affect the units.)
+
+h2. Essential Configuration
+
+@SystemRootToken@ is used by Arvados system services to authenticate as the system (root) user when communicating with the API server.
+
+@ManagementToken@ is used to authenticate access to system metrics.
+
+@API.RailsSessionSecretToken@ is required by the API server.
+
+@Collections.BlobSigningKey@ is used to construct blob signatures, as part of storage layer access control.
+
+You can generate a random token for each of these items at the command line like this:
+
+<notextile>
+<pre><code>~$ <span class="userinput">tr -dc 0-9a-zA-Z </dev/urandom | head -c40; echo</span>
+</code></pre>
+</notextile>
+
+Sample configuration file fragment. Fill in each configuration item with the tokens you generate.
+
+<pre>
+Clusters:
+ abcde:
+ SystemRootToken: "$system_root_token"
+ ManagementToken: "$management_token"
+ API:
+ RailsSessionSecretToken: "$secret_token"
+ Collections:
+ BlobSigningKey: "$blob_signing_key"
+</pre>
+
+h2. Services
+
+The @Services@ section of the configuration helps Arvados components contact one another.
+
+Each service has one or more @InternalURLs@ and an @ExternalURL@.
+
+The @InternalURLs@ describe where the service actually runs.
+
+The @ExternalURL@ is how clients contact the service.
+
+<pre>
+Clusters:
+ abcde:
+ Services:
+ Controller:
+ ExternalURL: "https://abcde.example.com"
+ InternalURLs:
+ "http://abcde.example.com:8000": {}
+ RailsAPI:
+ # Does not have an ExternalURL
+ InternalURLs:
+ "http://abcde.example.com:8001": {}
+
+</pre>
The Arvados storage subsystem is called "keep". The compute subsystem is called "crunch".
# "Supported GNU/Linux distributions":#supportedlinux
+# "Choosing which components to install":#components
# "Identity provider":#identity
# "Storage backend (Keep)":#storage
# "Container compute scheduler (Crunch)":#scheduler
|CentOS 7|Supported|Latest|
|Debian 10 ("buster")|Supported|Latest|
|Debian 9 ("stretch")|Supported|Latest|
-|Ubuntu 16.04 ("xenial")|Supported|Latest|
|Ubuntu 18.04 ("bionic")|Supported|Latest|
+|Ubuntu 16.04 ("xenial")|Supported|Latest|
|Ubuntu 14.04 ("trusty")|EOL|5f943cd451acfbdcddd84e791738c3aa5926bfed (2019-07-10)|
|Debian 8 ("jessie")|EOL|5f943cd451acfbdcddd84e791738c3aa5926bfed (2019-07-10)|
|Ubuntu 12.04 ("precise")|EOL|8ed7b6dd5d4df93a3f37096afe6d6f81c2a7ef6e (2017-05-03)|
Arvados packages are published for current Debian releases (until the EOL date), current Ubuntu LTS releases (until the end of standard support), and the latest version of CentOS.
+h2(#components). Choosing which components to install
+
+Arvados consists of many components, some of which may be omitted (at the cost of reduced functionality.) It may also be helpful to review the "Arvados Architecture":{{site.baseurl}}/architecture to understand how these components interact.
+
+table(table table-bordered table-condensed).
+|\3=. *Core*|
+|"Postgres database":install-postgresql.html |Stores data for the API server.|Required.|
+|"API server":install-api-server.html |Core Arvados logic for managing users, groups, collections, containers, and enforcing permissions.|Required.|
+|\3=. *Keep (storage)*|
+|"Keepstore":install-keepstore.html |Stores content-addressed blocks in a variety of backends (local filesystem, cloud object storage).|Required.|
+|"Keepproxy":install-keepproxy.html |Gateway service to access keep servers from external networks.|Required to be able to use arv-put, arv-get, or arv-mount outside the private Arvados network.|
+|"Keep-web":install-keep-web.html |Gateway service providing read/write HTTP and WebDAV support on top of Keep.|Required to be able to download files from Keep over plain HTTP in Workbench.|
+|"Keep-balance":install-keep-balance.html |Storage cluster maintenance daemon responsible for moving blocks to their optimal server location, adjusting block replication levels, and trashing unreferenced blocks.|Required to free deleted data from underlying storage, and to ensure proper replication and block distribution (including support for storage classes).|
+|\3=. *User interface*|
+|"Single Sign On server":install-sso.html |Web based login to Workbench.|Depends on identity provider. Not required for Google. Required for LDAP or standalone database.|
+|"Workbench":install-workbench-app.html, "Workbench2":install-workbench2-app.html |Primary graphical user interface for working with file collections and running containers.|Optional. Depends on API server, SSO server, keep-web, websockets server.|
+|"Workflow Composer":install-composer.html |Graphical user interface for editing Common Workflow Language workflows.|Optional. Depends on git server (arv-git-httpd).|
+|\3=. *Additional services*|
+|"Websockets server":install-ws.html |Event distribution server.|Required to view streaming container logs in Workbench.|
+|"Shell server":install-shell-server.html |Synchronize (create/delete/configure) Unix shell accounts with Arvados users.|Optional.|
+|"Git server":install-arv-git-httpd.html |Arvados-hosted git repositories, with Arvados-token based authentication.|Optional, but required by Workflow Composer.|
+|\3=. *Crunch (running containers)*|
+|"crunch-dispatch-slurm":crunch2-slurm/install-prerequisites.html |Run analysis workflows using Docker containers distributed across a SLURM cluster.|Optional if you wish to use Arvados for data management only.|
+|"Node Manager":install-nodemanager.html |Allocate and free cloud VM instances on demand based on workload.|Optional, not needed for a static SLURM cluster (such as on-premise HPC).|
+
h2(#identity). Identity provider
Choose which backend you will use to authenticate users.
h2(#machines). Hardware (or virtual machines)
-For a production installation, here is a suggested configuration:
+Choose how to allocate Arvados services to machines. We recommend that each machine start with a clean installation of your preferred Linux distribution.
+
+For a production installation, this is a reasonable configuration:
<div class="offset1">
table(table table-bordered table-condensed).
h2(#dnstls). DNS entries and TLS certificates
-The following services are normally public-facing and require DNS entries and corresponding TLS certificates. Get certificates from your preferred TLS certificate provider. We recommend using "Let's Encrypt":https://letsencrypt.org/ . You can run several services on same node, but each distinct hostname requires its own TLS certificate. A later part of this guide will describe how to set up Nginx virtual hosts.
+The following services are normally public-facing and require DNS entries and corresponding TLS certificates. Get certificates from your preferred TLS certificate provider. We recommend using "Let's Encrypt":https://letsencrypt.org/ . You can run several services on same node, but each distinct hostname requires its own TLS certificate.
+
+This guide uses the following hostname conventions. A later part of this guide will describe how to set up Nginx virtual hosts.
<div class="offset1">
table(table table-bordered table-condensed).
|_. Function|_. Hostname|
-|Arvados API|@ClusterID@.your.domain|
-|Arvados Git server|git.@ClusterID@.your.domain|
-|Arvados Websockets endpoint|ws.@ClusterID@.your.domain|
-|Arvados SSO Server|auth.your.domain|
-|Arvados Workbench|workbench.@ClusterID@.your.domain|
-|Arvados Workbench 2|workbench2.@ClusterID@.your.domain|
-|Arvados Keepproxy server|keep.@ClusterID@.your.domain|
-|Arvados Keep-web server|download.@ClusterID@.your.domain
+|Arvados API|@ClusterID@.example.com|
+|Arvados Git server|git.@ClusterID@.example.com|
+|Arvados Websockets endpoint|ws.@ClusterID@.example.com|
+|Arvados SSO Server|auth.example.com|
+|Arvados Workbench|workbench.@ClusterID@.example.com|
+|Arvados Workbench 2|workbench2.@ClusterID@.example.com|
+|Arvados Keepproxy server|keep.@ClusterID@.example.com|
+|Arvados Keep-web server|download.@ClusterID@.example.com
_and_
-*.collections.@ClusterID@.your.domain or
-*<notextile>--</notextile>collections.@ClusterID@.your.domain or
-collections.@ClusterID@.your.domain (see the "keep-web install docs":install-keep-web.html)|
+*.collections.@ClusterID@.example.com or
+*<notextile>--</notextile>collections.@ClusterID@.example.com or
+collections.@ClusterID@.example.com (see the "keep-web install docs":install-keep-web.html)|
</div>
{% include 'notebox_begin' %}
projects / arvados.git / commitdiff