SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-If you plan to use custom certificates, please set the variable <i>USE_LETSENCRYPT=no</i> and copy your certificates to the directory specified with the variable @CUSTOM_CERTS_DIR@ (usually "./certs") in the remote directory where you copied the @provision.sh@ script. From this dir, the provision script will install the certificates required for the role you're installing.
+Copy your certificates to the directory specified with the variable @CUSTOM_CERTS_DIR@ in the remote directory where you copied the @provision.sh@ script. The provision script will find the certificates there.
The script expects cert/key files with these basenames (matching the role except for <i>keepweb</i>, which is split in both <i>download / collections</i>):
* "collections" # Part of keepweb
* "keepproxy"
-Ie., for 'keepproxy', the script will look for
+E.g. for 'keepproxy', the script will look for
<notextile>
<pre><code>${CUSTOM_CERTS_DIR}/keepproxy.crt
${CUSTOM_CERTS_DIR}/keepproxy.key
</code></pre>
</notextile>
+
+Make sure that all the FQDNs that you will use for the public-facing applications (API/controller, Workbench, Keepproxy/Keepweb) are reachable.
The <i>multi_host</i> example includes Let's Encrypt salt code to automatically request and install the certificates for the public-facing hosts (API/controller, Workbench, Keepproxy/Keepweb) using AWS' Route53.
-{% include 'install_custom_certificates' %}
+{% include 'multi_host_install_custom_certificates' %}
+
+If you want to use valid certificates provided by Let's Encrypt, set the variable <i>SSL_MODE=lets-encrypt</i> and make sure that all the FQDNs that you will use for the public-facing applications (API/controller, Workbench, Keepproxy/Keepweb) are reachable.
h3(#further_customization). Further customization of the installation (modifying the salt pillars and states)
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-# "Single host install using the provision.sh script":#single_host
-# "Choose the desired configuration":#choose_configuration
-## "Single host / single hostname":#single_host_single_hostnames
-## "Single host / multiple hostnames (Alternative configuration)":#single_host_multiple_hostnames
-## "Further customization of the installation (modifying the salt pillars and states)":#further_customization
+# "Limitations of the single host install":#limitations
+# "Prerequisites":#prerequisites
+# "Download the installer":#single_host
+# "Choose the SSL configuration":#certificates
+## "Using a self-signed certificate":#self-signed
+## "Using a Let's Encrypt certificate":#lets-encrypt
+## "Bring your own certificate":#bring-your-own
+# "Further customization of the installation (modifying the salt pillars and states)":#further_customization
# "Run the provision.sh script":#run_provision_script
-# "Final configuration steps":#final_steps
-## "Install the CA root certificate (required in both alternatives)":#ca_root_certificate
-## "DNS configuration (single host / multiple hostnames)":#single_host_multiple_hostnames_dns_configuration
+# "Install the CA root certificate":#ca_root_certificate
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
# "After the installation":#post_install
-h2(#single_host). Single host install using the provision.sh script
+h2(#limitations). Limitations of the single host install
-<b>NOTE: The single host installation is not recommended for production use.</b>
+<b>NOTE: The single host installation is a good choice for evaluating Arvados, but it is not recommended for production use.</b>
-{% include 'branchname' %}
+Using the default configuration, this installation method has a number of limitations:
-This is a package-based installation method. Start with the @provision.sh@ script which is available by cloning the @{{ branchname }}@ branch from "https://git.arvados.org/arvados.git":https://git.arvados.org/arvados.git . The @provision.sh@ script and its supporting files can be found in the "arvados/tools/salt-install":https://git.arvados.org/arvados.git/tree/refs/heads/{{ branchname }}:/tools/salt-install directory in the Arvados git repository.
+* all services run on the same machine, and they will compete for resources. This includes any compute jobs.
+* it uses the local machine disk for Keep storage (under the @/tmp@ directory). There may not be a lot of space available.
+* it installs the @crunch-dispatch-local@ dispatcher, which can run just eight concurrent CWL jobs. These jobs will be executed on the same machine that runs all the Arvados services and may well starve them of resources.
-This procedure will install all the main Arvados components to get you up and running in a single host. The whole installation procedure takes somewhere between 15 to 60 minutes, depending on the host resources and its network bandwidth. As a reference, on a virtual machine with 1 core and 1 GB RAM, it takes ~25 minutes to do the initial install.
+It is possible to start with the single host installation method and modify the Arvados configuration file later to address these limitations. E.g. switch to a "different storage volume setup":{{site.baseurl}}/install/configure-s3-object-storage.html for Keep, and switch to "the cloud dispatcher":{{site.baseurl}}/install/crunch2-cloud/install-dispatch-cloud.html to provision compute resources dynamically.
-The @provision.sh@ script will help you deploy Arvados by preparing your environment to be able to run the installer, then running it. The actual installer is located at "arvados-formula":https://git.arvados.org/arvados-formula.git/tree/refs/heads/{{ branchname }} and will be cloned during the running of the @provision.sh@ script. The installer is built using "Saltstack":https://saltproject.io/ and @provision.sh@ performs the install using master-less mode.
+h2(#prerequisites). Prerequisites and planning
-After setting up a few variables in a config file (next step), you'll be ready to run it and get Arvados deployed.
+Prerequisites:
-h2(#choose_configuration). Choose the desired configuration
+* git
+* a dedicated (virtual) machine for your Arvados server with at least 2 cores and 8 GiB of RAM, running a "supported Arvados distribution":{{site.baseurl}}/install/install-manual-prerequisites.html#supportedlinux
+* a DNS hostname that resolves to the IP address of your Arvados server
+* ports 443, 8800-8805 need to be reachable from your client (configurable in @local.params@, see below)
+* port 80 needs to be reachable from everywhere on the internet (only when using "Let's Encrypt":#lets-encrypt)
+* an SSL certificate matching the hostname in use (only when using "bring your own certificate":#bring-your-own)
-For documentation's sake, we will use the cluster name <i>arva2</i> and the domain <i>arv.local</i>. If you don't change them as required in the next steps, installation won't proceed.
+h2(#single_host). Download the installer
-Arvados' single host installation can be done in two fashions:
+{% include 'branchname' %}
-* Using a single hostname, assigning <i>a different port (other than 443) for each user-facing service</i>: This choice is easier to setup, but the user will need to know the port/s for the different services she wants to connect to.
-* Using multiple hostnames on the same IP: this setup involves a few extra steps but each service will have a meaningful hostname so it will make easier to access them later.
+This procedure will install all the main Arvados components to get you up and running in a single host.
-Once you decide which of these choices you prefer, copy one the two example configuration files and directory, and edit them to suit your needs.
+This is a package-based installation method, however the installation script is currently distributed in source form via @git@:
+
+<notextile>
+<pre><code>git clone https://git.arvados.org/arvados.git
+git checkout {{ branchname }}
+cd arvados/tools/salt-install
+</code></pre>
+</notextile>
+
+The @provision.sh@ script will help you deploy Arvados by preparing your environment to be able to run the installer, then running it. The actual installer is located in the "arvados-formula git repository":https://git.arvados.org/arvados-formula.git/tree/refs/heads/{{ branchname }} and will be cloned during the running of the @provision.sh@ script. The installer is built using "Saltstack":https://saltproject.io/ and @provision.sh@ performs the install using master-less mode.
+
+First, copy the configuration files:
-h3(#single_host_single_hostnames). Single host / single hostname
<notextile>
<pre><code>cp local.params.example.single_host_single_hostname local.params
cp -r config_examples/single_host/single_hostname local_config_dir
</code></pre>
</notextile>
-Edit the variables in the <i>local.params</i> file. Pay attention to the <b>*_PORT, *_TOKEN</b> and <b>*KEY</b> variables.
+Edit the variables in the <i>local.params</i> file. Pay attention to the <b>*_PORT, *_TOKEN</b> and <b>*KEY</b> variables. The *SSL_MODE* variable is discussed in the next section.
+
+h2(#certificates). Choose the SSL configuration (SSL_MODE)
+
+Arvados requires an SSL certificate to work correctly. This installer supports these options:
-The <i>single_host</i> examples use self-signed SSL certificates, which are deployed using the same mechanism used to deploy custom certificates.
+* @self-signed@: let the installer create a self-signed certificate
+* @lets-encrypt@: automatically obtain and install an SSL certificate for your hostname
+* @bring-your-own@: supply your own certificate in the `certs` directory
-{% include 'install_custom_certificates' %}
+h3(#self-signed). Using a self-signed certificate
-If you want to use valid certificates provided by Let's Encrypt, please set the variable <i>USE_LETSENCRYPT=yes</i> and make sure that all the FQDNs that you will use for the public-facing applications (API/controller, Workbench, Keepproxy/Keepweb) are reachable.
+In the default configuration, this installer uses self-signed certificate(s):
-h3(#single_host_multiple_hostnames). Single host / multiple hostnames (Alternative configuration)
<notextile>
-<pre><code>cp local.params.example.single_host_multiple_hostnames local.params
-cp -r config_examples/single_host/multiple_hostnames local_config_dir
+<pre><code>SSL_MODE="self-signed"
</code></pre>
</notextile>
-Edit the variables in the <i>local.params</i> file.
+When connecting to the Arvados web interface for the first time, you will need to accept the self-signed certificate as trusted to bypass the browser warnings.
-h3(#further_customization). Further customization of the installation (modifying the salt pillars and states)
+h3(#lets-encrypt). Using a Let's Encrypt certificate
-If you want or need further customization, you can edit the Saltstack pillars and states files. Pay particular attention to the <i>pillars/arvados.sls</i> one. Any extra <i>state</i> file you add under <i>local_config_dir/states</i> will be added to the salt run and applied to the host.
+To automatically get a valid certificate via Let's Encrypt, change the configuration like this:
-h2(#run_provision_script). Run the provision.sh script
+<notextile>
+<pre><code>SSL_MODE="lets-encrypt"
+</code></pre>
+</notextile>
-When you finished customizing the configuration, you are ready to copy the files to the host (if needed) and run the @provision.sh@ script:
+The hostname for your Arvados cluster must be defined in @HOSTNAME_EXT@ and resolve to the public IP address of your Arvados instance, so that Let's Encrypt can validate the domainname ownership and issue the certificate.
+
+When using AWS, EC2 instances can have a default hostname that ends with <i>amazonaws.com</i>. Let's Encrypt has a blacklist of domain names for which it will not issue certificates, and that blacklist includes the <i>amazonaws.com</i> domain, which means the default hostname can not be used to get a certificate from Let's Encrypt.
+
+h3(#bring-your-own). Bring your own certificate
+
+To supply your own certificate, change the configuration like this:
<notextile>
-<pre><code>scp -r provision.sh local* tests user@host:
-# if you use custom certificates (not Let's Encrypt), make sure to copy those too:
-# scp -r certs user@host:
-ssh user@host sudo ./provision.sh
+<pre><code>SSL_MODE="bring-your-own"
+CUSTOM_CERTS_DIR="${SCRIPT_DIR}/certs"
</code></pre>
</notextile>
-or, if you saved the @local.params@ in another directory or with some other name
+Copy your certificate files to the directory specified with the variable @CUSTOM_CERTS_DIR@. The provision script will find it there. The certificate and its key need to be copied to a file named after @HOSTNAME_EXT@. For example, if @HOSTNAME_EXT@ is defined as @my-arvados.example.net@, the script will look for
<notextile>
-<pre><code>scp -r provision.sh local* tests user@host:
-ssh user@host sudo ./provision.sh -c /path/to/your/local.params.file
+<pre><code>${CUSTOM_CERTS_DIR}/my-arvados.example.net.crt
+${CUSTOM_CERTS_DIR}/my-arvados.example.net.key
</code></pre>
</notextile>
-and wait for it to finish.
+All certificate files will be used by nginx. You may need to include intermediate certificates in your certificate file. See "the nginx documentation":http://nginx.org/en/docs/http/configuring_https_servers.html#chains for more details.
+
+h2(#further_customization). Further customization of the installation (modifying the salt pillars and states)
+
+If you want or need further customization, you can edit the Saltstack pillars and states files. Pay particular attention to the <i>pillars/arvados.sls</i> one. Any extra <i>state</i> file you add under <i>local_config_dir/states</i> will be added to the salt run and applied to the host.
+
+h2(#run_provision_script). Run the provision.sh script
-If everything goes OK, you'll get some final lines stating something like:
+When you finished customizing the configuration, you are ready to copy the files to the host (if needed) and run the @provision.sh@ script:
<notextile>
-<pre><code>arvados: Succeeded: 109 (changed=9)
-arvados: Failed: 0
+<pre><code>scp -r provision.sh local* tests user@host:
+# if you have set SSL_MODE to "bring-your-own", make sure to also copy the certificate files:
+# scp -r certs user@host:
+ssh user@host sudo ./provision.sh
</code></pre>
</notextile>
-h2(#final_steps). Final configuration steps
+and wait for it to finish. The script will need 5 to 10 minutes to install and configure everything.
+
+If everything goes OK, you'll get final output that looks similar to this:
-Once the deployment went OK, you'll need to perform a few extra steps in your local browser/host to access the cluster.
+<notextile>
+<pre><code>arvados: Succeeded: 151 (changed=36)
+arvados: Failed: 0
+</code></pre>
+</notextile>
-h3(#ca_root_certificate). Install the CA root certificate (required in both alternatives)
+h2(#ca_root_certificate). Install the CA root certificate (SSL_MODE=self-signed only)
-Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
+Arvados uses SSL to encrypt communications. The web interface uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so you can add it to your workstation.
</code></pre>
</notextile>
-h3(#single_host_multiple_hostnames_dns_configuration). DNS configuration (single host / multiple hostnames)
-
-When using multiple hostnames, after the setup is done, you need to set up your DNS to be able to access the cluster.
-
-If you don't have access to the domain's DNS to add the required entries, the simplest way to do it is to edit your @/etc/hosts@ file (as root):
-
-<notextile>
-<pre><code>export CLUSTER="arva2"
-export DOMAIN="arv.local"
-export HOST_IP="127.0.0.2" # This is valid either if installing in your computer directly
- # or in a Vagrant VM. If you're installing it on a remote host
- # just change the IP to match that of the host.
-echo "${HOST_IP} api keep keep0 collections download ws workbench workbench2 ${CLUSTER}.${DOMAIN} api.${CLUSTER}.${DOMAIN} keep.${CLUSTER}.${DOMAIN} keep0.${CLUSTER}.${DOMAIN} collections.${CLUSTER}.${DOMAIN} download.${CLUSTER}.${DOMAIN} ws.${CLUSTER}.${DOMAIN} workbench.${CLUSTER}.${DOMAIN} workbench2.${CLUSTER}.${DOMAIN}" >> /etc/hosts
-</code></pre>
-</notextile>
-
h2(#initial_user). Initial user and login
-At this point you should be able to log into the Arvados cluster. The initial URL will be:
-
-* https://workbench.arva2.arv.local
-
-or, in general, the url format will be:
+At this point you should be able to log on to your new Arvados cluster. The workbench URL will be
-* https://workbench.@<cluster>.<domain>@
+* https://@HOSTNAME_EXT@
-By default, the provision script creates an initial user for testing purposes. This user is configured as administrator of the newly created cluster.
-
-Assuming you didn't change these values in the @local.params@ file, the initial credentials are:
-
-* User: 'admin'
-* Password: 'password'
-* Email: 'admin@arva2.arv.local'
+By default, the provision script creates an initial user for testing purposes. This user is configured as administrator of the newly created cluster. The username, password and e-mail address for the initial user are configured in the @local.params@ file. Log in with the e-mail address and password.
h2(#test_install). Test the installed cluster running a simple workflow
"owner_uuid":"arva2-tpzed-000000000000000",
...
}
-Uploading arvados/jobs' docker image to the project
-2.1.1: Pulling from arvados/jobs
-8559a31e96f4: Pulling fs layer
-...
-Status: Downloaded newer image for arvados/jobs:2.1.1
-docker.io/arvados/jobs:2.1.1
-2020-11-23 21:43:39 arvados.arv_put[32678] INFO: Creating new cache file at /home/vagrant/.cache/arvados/arv-put/c59256eda1829281424c80f588c7cc4d
-2020-11-23 21:43:46 arvados.arv_put[32678] INFO: Collection saved as 'Docker image arvados jobs:2.1.1 sha256:0dd50'
-arva2-4zz18-1u5pvbld7cvxuy2
Creating initial user ('admin')
Setting up user ('admin')
{
This is a package-based installation method.
-
-
h2(#provisioning_arvados). Provisioning Arvados with Saltstack
The "tools/salt-install":https://git.arvados.org/arvados.git/tree/{{ branchname }}:/tools/salt-install directory in the Arvados git repository contains a script that you can run in the node/s where you want to install Arvados' components (the @provision.sh@ script) and a few configuration examples for different setups, that you can use to customize your installation.
s#domain_fixme_or_this_wont_work#local#g;
s#CONTROLLER_EXT_SSL_PORT=443#CONTROLLER_EXT_SSL_PORT=8443#g;
s#RELEASE=\"production\"#RELEASE=\"development\"#g;
- s/# VERSION=.*$/VERSION=\"latest\"/g;
- s/#\ BRANCH=\"main\"/\ BRANCH=\"main\"/g' \
+ s/# VERSION=.*$/VERSION=\"latest\"/g;' \
/vagrant/local.params.example.single_host_multiple_hostnames > /tmp/local.params.single_host_multiple_hostnames"
+ # s/#\ BRANCH=\"main\"/\ BRANCH=\"main\"/g;' \
arv.vm.provision "shell",
path: "provision.sh",
# A single_host single_hostname example
config.vm.define "arvados-sh-sn" do |arv|
- arv.vm.box = "bento/debian-10"
+ arv.vm.box = "bento/debian-11"
arv.vm.hostname = "zeppo"
# CPU/RAM
config.vm.provider :virtualbox do |v|
arv.vm.provision "shell",
inline: "cp -vr /vagrant/config_examples/single_host/single_hostname /home/vagrant/local_config_dir;
cp -vr /vagrant/tests /home/vagrant/tests;
- sed 's#HOSTNAME_EXT=\"\"#HOSTNAME_EXT=\"zeppo.local\"#g;
+ sed 's#HOSTNAME_EXT=\"hostname_ext_fixme_or_this_wont_work\"#HOSTNAME_EXT=\"zeppo.local\"#g;
s#cluster_fixme_or_this_wont_work#zeppo#g;
- s/#\ BRANCH=\"main\"/\ BRANCH=\"main\"/g;
s#domain_fixme_or_this_wont_work#local#g;' \
/vagrant/local.params.example.single_host_single_hostname > /tmp/local.params.single_host_single_hostname"
+ # s/#\ BRANCH=\"main\"/\ BRANCH=\"main\"/g;
arv.vm.provision "shell",
path: "provision.sh",
args: [
- # "--debug",
+ "--debug",
"--config /tmp/local.params.single_host_single_hostname",
"--test",
"--vagrant"
--- /dev/null
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: AGPL-3.0
+
+dns:
+ pkg.installed:
+ - pkgs:
+ - dnsmasq
password: "__DATABASE_PASSWORD__"
user: __CLUSTER___arvados
encoding: en_US.utf8
+ client_encoding: UTF8
tls:
# certificate: ''
system_root: __SYSTEM_ROOT_TOKEN__
management: __MANAGEMENT_TOKEN__
anonymous_user: __ANONYMOUS_USER_TOKEN__
- rails_secret: YDLxHf4GqqmLXYAMgndrAmFEdqgC0sBqX7TEjMN2rw9D6EVwgx
### KEYS
secrets:
# <cluster>-nyw5e-<volume>
__CLUSTER__-nyw5e-000000000000000:
AccessViaHosts:
- 'http://__HOSTNAME_INT__:25107':
+ 'http://__IP_INT__:25107':
ReadOnly: false
Replication: 2
Driver: Directory
Controller:
ExternalURL: 'https://__HOSTNAME_EXT__:__CONTROLLER_EXT_SSL_PORT__'
InternalURLs:
- 'http://__HOSTNAME_INT__:8003': {}
+ 'http://__IP_INT__:8003': {}
Keepproxy:
ExternalURL: 'https://__HOSTNAME_EXT__:__KEEP_EXT_SSL_PORT__'
InternalURLs:
- 'http://__HOSTNAME_INT__:25100': {}
+ 'http://__IP_INT__:25100': {}
Keepstore:
InternalURLs:
- 'http://__HOSTNAME_INT__:25107': {}
+ 'http://__IP_INT__:25107': {}
RailsAPI:
InternalURLs:
- 'http://__HOSTNAME_INT__:8004': {}
+ 'http://__IP_INT__:8004': {}
WebDAV:
ExternalURL: 'https://__HOSTNAME_EXT__:__KEEPWEB_EXT_SSL_PORT__'
InternalURLs:
- 'http://__HOSTNAME_INT__:9003': {}
+ 'http://__IP_INT__:9003': {}
WebDAVDownload:
ExternalURL: 'https://__HOSTNAME_EXT__:__KEEPWEB_EXT_SSL_PORT__'
WebShell:
Websocket:
ExternalURL: 'wss://__HOSTNAME_EXT__:__WEBSOCKET_EXT_SSL_PORT__/websocket'
InternalURLs:
- 'http://__HOSTNAME_INT__:8005': {}
+ 'http://__IP_INT__:8005': {}
Workbench1:
ExternalURL: 'https://__HOSTNAME_EXT__:__WORKBENCH1_EXT_SSL_PORT__'
Workbench2:
--- /dev/null
+---
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: AGPL-3.0
+
+aws_credentials:
+ region: __LE_AWS_REGION__
+ access_key_id: __LE_AWS_ACCESS_KEY_ID__
+ secret_access_key: __LE_AWS_SECRET_ACCESS_KEY__
pkg:
docker:
use_upstream: package
+ daemon_config: {"dns": ["__IP_INT__"]}
--- /dev/null
+---
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: AGPL-3.0
+
+### LETSENCRYPT
+letsencrypt:
+ use_package: true
+ pkgs:
+ - certbot: latest
+ - python3-certbot-nginx
+ config:
+ server: https://acme-v02.api.letsencrypt.org/directory
+ email: __INITIAL_USER_EMAIL__
+ authenticator: nginx
+ agree-tos: true
+ keep-until-expiring: true
+ expand: true
+ max-log-backups: 0
+ deploy-hook: systemctl reload nginx
+
+ domainsets:
+ __HOSTNAME_EXT__:
+ - __HOSTNAME_EXT__
### SITES
servers:
managed:
- arvados_api:
+ arvados_api.conf:
enabled: true
overwrite: true
config:
- server:
- - listen: '__HOSTNAME_INT__:8004'
+ - listen: '__IP_INT__:8004'
- server_name: api
- root: /var/www/arvados-api/current/public
- index: index.html index.htm
default: 1
'127.0.0.0/8': 0
upstream controller_upstream:
- - server: '__HOSTNAME_INT__:8003 fail_timeout=10s'
+ - server: '__IP_INT__:8003 fail_timeout=10s'
### SITES
servers:
managed:
### DEFAULT
- arvados_controller_default:
+ arvados_controller_default.conf:
enabled: true
overwrite: true
config:
- server:
- server_name: _
- listen:
- - 80 default_server
+ - 80
- location /.well-known:
- root: /var/www
- location /:
- return: '301 https://$host$request_uri'
- arvados_controller_ssl:
+ arvados_controller_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- proxy_set_header: 'X-Real-IP $remote_addr'
- proxy_set_header: 'X-Forwarded-For $proxy_add_x_forwarded_for'
- proxy_set_header: 'X-External-Client $external_client'
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/__CLUSTER__.__DOMAIN__.error.log
- client_max_body_size: 128m
### STREAMS
http:
upstream keepproxy_upstream:
- - server: '__HOSTNAME_INT__:25100 fail_timeout=10s'
+ - server: '__IP_INT__:25100 fail_timeout=10s'
servers:
managed:
- arvados_keepproxy_ssl:
+ ### DEFAULT
+ arvados_keepproxy_default.conf:
enabled: true
overwrite: true
+ config:
+ - server:
+ - server_name: keep.__CLUSTER__.__DOMAIN__
+ - listen:
+ - 80
+ - location /:
+ - return: '301 https://$host$request_uri'
+
+ arvados_keepproxy_ssl.conf:
+ enabled: true
+ overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- client_max_body_size: 64M
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/keepproxy.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/keepproxy.__CLUSTER__.__DOMAIN__.error.log
### STREAMS
http:
upstream collections_downloads_upstream:
- - server: '__HOSTNAME_INT__:9003 fail_timeout=10s'
+ - server: '__IP_INT__:9003 fail_timeout=10s'
servers:
managed:
### COLLECTIONS / DOWNLOAD
- arvados_collections_download_ssl:
+ arvados_collections_download_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- client_max_body_size: 0
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/keepweb.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/keepweb.__CLUSTER__.__DOMAIN__.error.log
### SITES
servers:
managed:
- # Remove default webserver
+ # Update default config to redirect to https
default:
- enabled: false
+ enabled: true
+ overwrite: true
+ config:
+ - server:
+ - server_name: _
+ - listen:
+ - 80 default_server
+ - location /.well-known:
+ - root: /var/www
+ - location /:
+ - return: '301 https://$host$request_uri'
### STREAMS
http:
upstream webshell_upstream:
- - server: '__HOSTNAME_INT__:4200 fail_timeout=10s'
+ - server: '__IP_INT__:4200 fail_timeout=10s'
### SITES
servers:
managed:
- arvados_webshell_ssl:
+ arvados_webshell_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- add_header: "'Access-Control-Allow-Methods' 'GET, POST, OPTIONS'"
- add_header: "'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'"
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/webshell.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/webshell.__CLUSTER__.__DOMAIN__.error.log
### STREAMS
http:
upstream websocket_upstream:
- - server: '__HOSTNAME_INT__:8005 fail_timeout=10s'
+ - server: '__IP_INT__:8005 fail_timeout=10s'
servers:
managed:
- arvados_websocket_ssl:
+ arvados_websocket_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- client_max_body_size: 64M
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/ws.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/ws.__CLUSTER__.__DOMAIN__.error.log
### SITES
servers:
managed:
- arvados_workbench2_ssl:
+ arvados_workbench2_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- return: 503
- location /config.json:
- return: {{ "200 '" ~ '{"API_HOST":"__HOSTNAME_EXT__:__CONTROLLER_EXT_SSL_PORT__"}' ~ "'" }}
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/workbench2.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/workbench2.__CLUSTER__.__DOMAIN__.error.log
### STREAMS
http:
upstream workbench_upstream:
- - server: '__HOSTNAME_INT__:9000 fail_timeout=10s'
+ - server: '__IP_INT__:9000 fail_timeout=10s'
### SITES
servers:
managed:
- arvados_workbench_ssl:
+ arvados_workbench_ssl.conf:
enabled: true
overwrite: true
+ requires:
+ __CERT_REQUIRES__
config:
- server:
- server_name: __HOSTNAME_EXT__
- proxy_set_header: 'Host $http_host'
- proxy_set_header: 'X-Real-IP $remote_addr'
- proxy_set_header: 'X-Forwarded-For $proxy_add_x_forwarded_for'
- - include: 'snippets/arvados-snakeoil.conf'
+ - include: snippets/ssl_hardening_default.conf
+ - ssl_certificate: __CERT_PEM__
+ - ssl_certificate_key: __CERT_KEY__
- access_log: /var/log/nginx/workbench.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/workbench.__CLUSTER__.__DOMAIN__.error.log
overwrite: true
config:
- server:
- - listen: '__HOSTNAME_INT__:9000'
+ - listen: '__IP_INT__:9000'
- server_name: workbench
- root: /var/www/arvados-workbench/current/public
- index: index.html index.htm
--- /dev/null
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+{%- set orig_cert_dir = salt['pillar.get']('extra_custom_certs_dir', '/srv/salt/certs') %}
+{%- set dest_cert_dir = '/etc/nginx/ssl' %}
+{%- set certs = salt['pillar.get']('extra_custom_certs', []) %}
+
+{% if certs %}
+extra_custom_certs_file_directory_certs_dir:
+ file.directory:
+ - name: /etc/nginx/ssl
+ - require:
+ - pkg: nginx_install
+
+ {%- for cert in certs %}
+ {%- set cert_file = 'arvados-' ~ cert ~ '.pem' %}
+ {#- set csr_file = 'arvados-' ~ cert ~ '.csr' #}
+ {%- set key_file = 'arvados-' ~ cert ~ '.key' %}
+ {% for c in [cert_file, key_file] %}
+extra_custom_certs_file_copy_{{ c }}:
+ file.copy:
+ - name: {{ dest_cert_dir }}/{{ c }}
+ - source: {{ orig_cert_dir }}/{{ c }}
+ - force: true
+ - user: root
+ - group: root
+ - unless: cmp {{ dest_cert_dir }}/{{ c }} {{ orig_cert_dir }}/{{ c }}
+ - require:
+ - file: extra_custom_certs_file_directory_certs_dir
+ {%- endfor %}
+ {%- endfor %}
+{%- endif %}
--- /dev/null
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: AGPL-3.0
+
+dns:
+ pkg.installed:
+ - pkgs:
+ - dnsmasq
{%- from "arvados/map.jinja" import arvados with context %}
{%- set tpldir = curr_tpldir %}
+# We need the external hostname to resolve to the internal IP for docker. We
+# tell docker to resolve via the local dnsmasq, which reads from /etc/hosts by
+# default.
+arvados_local_access_to_hostname_ext:
+ host.present:
+ - ip: __IP_INT__
+ - names:
+ - __HOSTNAME_EXT__
+
arvados_test_salt_states_examples_single_host_etc_hosts_host_present:
host.present:
- ip: 127.0.1.1
- names:
- {{ arvados.cluster.name }}.{{ arvados.cluster.domain }}
- # FIXME! This just works for our testings.
+ # FIXME! This just works for our testing.
# Won't work if the cluster name != host name
{%- for entry in [
'api',
{%- from "arvados/map.jinja" import arvados with context %}
{%- set tpldir = curr_tpldir %}
+{%- set orig_cert_dir = salt['pillar.get']('extra_custom_certs_dir', '/srv/salt/certs') %}
+
include:
- nginx.passenger
- nginx.config
# we'll keep it simple here.
{%- set arvados_ca_cert_file = '/etc/ssl/private/arvados-snakeoil-ca.pem' %}
{%- set arvados_ca_key_file = '/etc/ssl/private/arvados-snakeoil-ca.key' %}
-{%- set arvados_cert_file = '/etc/ssl/private/arvados-snakeoil-cert.pem' %}
-{%- set arvados_csr_file = '/etc/ssl/private/arvados-snakeoil-cert.csr' %}
-{%- set arvados_key_file = '/etc/ssl/private/arvados-snakeoil-cert.key' %}
{%- if grains.get('os_family') == 'Debian' %}
{%- set arvados_ca_cert_dest = '/usr/local/share/ca-certificates/arvados-snakeoil-ca.crt' %}
{%- set update_ca_cert = '/usr/sbin/update-ca-certificates' %}
{%- set openssl_conf = '/etc/ssl/openssl.cnf' %}
+
+extra_snakeoil_certs_ssl_cert_pkg_installed:
+ pkg.installed:
+ - name: ssl-cert
+ - require_in:
+ - sls: postgres
+
{%- else %}
{%- set arvados_ca_cert_dest = '/etc/pki/ca-trust/source/anchors/arvados-snakeoil-ca.pem' %}
{%- set update_ca_cert = '/usr/bin/update-ca-trust' %}
{%- set openssl_conf = '/etc/pki/tls/openssl.cnf' %}
+
{%- endif %}
-arvados_test_salt_states_examples_single_host_snakeoil_certs_dependencies_pkg_installed:
+extra_snakeoil_certs_dependencies_pkg_installed:
pkg.installed:
- pkgs:
- openssl
- ca-certificates
-arvados_test_salt_states_examples_single_host_snakeoil_certs_arvados_snake_oil_ca_cmd_run:
+# Remove the RANDFILE parameter in openssl.cnf as it makes openssl fail in Ubuntu 18.04
+# Saving and restoring the rng state is not necessary anymore in the openssl 1.1.1
+# random generator, cf
+# https://github.com/openssl/openssl/issues/7754
+#
+extra_snakeoil_certs_file_comment_etc_openssl_conf:
+ file.comment:
+ - name: /etc/ssl/openssl.cnf
+ - regex: ^RANDFILE.*
+ - onlyif: grep -q ^RANDFILE /etc/ssl/openssl.cnf
+ - require_in:
+ - cmd: extra_snakeoil_certs_arvados_snakeoil_ca_cmd_run
+
+extra_snakeoil_certs_arvados_snakeoil_ca_cmd_run:
# Taken from https://github.com/arvados/arvados/blob/master/tools/arvbox/lib/arvbox/docker/service/certificate/run
cmd.run:
- name: |
- # These dirs are not to CentOS-ish, but this is a helper script
+ # These dirs are not too CentOS-ish, but this is a helper script
# and they should be enough
mkdir -p /etc/ssl/certs/ /etc/ssl/private/ && \
openssl req \
- test -f {{ arvados_ca_cert_file }}
- openssl verify -CAfile {{ arvados_ca_cert_file }} {{ arvados_ca_cert_file }}
- require:
- - pkg: arvados_test_salt_states_examples_single_host_snakeoil_certs_dependencies_pkg_installed
+ - pkg: extra_snakeoil_certs_dependencies_pkg_installed
-arvados_test_salt_states_examples_single_host_snakeoil_certs_arvados_snake_oil_cert_cmd_run:
+{%- set arvados_cert_file = orig_cert_dir ~ '/arvados-__HOSTNAME_EXT__.pem' %}
+{%- set arvados_csr_file = orig_cert_dir ~ '/arvadoos-__HOSTNAME_EXT__.csr' %}
+{%- set arvados_key_file = orig_cert_dir ~ '/arvados-__HOSTNAME_EXT__.key' %}
+
+extra_snakeoil_certs_arvados_snakeoil_cert___HOSTNAME_EXT___cmd_run:
cmd.run:
- name: |
- cat > /tmp/openssl.cnf <<-CNF
+ cat > /tmp/__HOSTNAME_EXT__.openssl.cnf <<-CNF
[req]
default_bits = 2048
prompt = no
default_md = sha256
- req_extensions = rext
distinguished_name = dn
+ req_extensions = rext
+ [rext]
+ subjectAltName = @alt_names
[dn]
C = CC
ST = Some State
L = Some Location
- O = Arvados Formula
- OU = arvados-formula
+ O = Arvados Provision Example Single Host / Single Hostname
+ OU = arvados-provision-example-single_host_single_hostname
CN = {{ arvados.cluster.name }}.{{ arvados.cluster.domain }}
emailAddress = admin@{{ arvados.cluster.name }}.{{ arvados.cluster.domain }}
- [rext]
- subjectAltName = @alt_names
[alt_names]
{%- for entry in grains.get('ipv4') %}
IP.{{ loop.index }} = {{ entry }}
{%- endfor %}
- {%- for entry in [
- 'keep',
- 'collections',
- 'download',
- 'keepweb',
- 'ws',
- 'workbench',
- 'workbench2',
- ]
- %}
- DNS.{{ loop.index }} = {{ entry }}
- {%- endfor %}
- DNS.8 = {{ arvados.cluster.name }}.{{ arvados.cluster.domain }}
- DNS.9 = '__HOSTNAME_EXT__'
- DNS.10 = '__HOSTNAME_INT__'
+ DNS.1 = {{ arvados.cluster.name }}.{{ arvados.cluster.domain }}
+ DNS.2 = '__HOSTNAME_EXT__'
CNF
# The req
openssl req \
- -config /tmp/openssl.cnf \
+ -config /tmp/__HOSTNAME_EXT__.openssl.cnf \
-new \
-nodes \
-sha256 \
-out {{ arvados_csr_file }} \
- -keyout {{ arvados_key_file }} > /tmp/snake_oil_certs.output 2>&1 && \
+ -keyout {{ arvados_key_file }} > /tmp/snake_oil_certs.__HOSTNAME_EXT__.output 2>&1 && \
# The cert
openssl x509 \
-req \
-days 365 \
-in {{ arvados_csr_file }} \
-out {{ arvados_cert_file }} \
- -extfile /tmp/openssl.cnf \
+ -extfile /tmp/__HOSTNAME_EXT__.openssl.cnf \
-extensions rext \
-CA {{ arvados_ca_cert_file }} \
-CAkey {{ arvados_ca_key_file }} \
- test -f {{ arvados_key_file }}
- openssl verify -CAfile {{ arvados_ca_cert_file }} {{ arvados_cert_file }}
- require:
- - pkg: arvados_test_salt_states_examples_single_host_snakeoil_certs_dependencies_pkg_installed
- - cmd: arvados_test_salt_states_examples_single_host_snakeoil_certs_arvados_snake_oil_ca_cmd_run
- # We need this before we can add the nginx's snippet
+ - pkg: extra_snakeoil_certs_dependencies_pkg_installed
+ - cmd: extra_snakeoil_certs_arvados_snakeoil_ca_cmd_run
- require_in:
- - file: nginx_snippet_arvados-snakeoil.conf
+ - file: extra_custom_certs_file_copy_arvados-__HOSTNAME_EXT__.pem
+ - file: extra_custom_certs_file_copy_arvados-__HOSTNAME_EXT__.key
-{%- if grains.get('os_family') == 'Debian' %}
-arvados_test_salt_states_examples_single_host_snakeoil_certs_ssl_cert_pkg_installed:
- pkg.installed:
- - name: ssl-cert
- - require_in:
- - sls: postgres
-
-arvados_test_salt_states_examples_single_host_snakeoil_certs_certs_permissions_cmd_run:
+ {%- if grains.get('os_family') == 'Debian' %}
+extra_snakeoil_certs_certs_permissions___HOSTNAME_EXT___cmd_run:
file.managed:
- name: {{ arvados_key_file }}
- owner: root
- group: ssl-cert
- require:
- - cmd: arvados_test_salt_states_examples_single_host_snakeoil_certs_arvados_snake_oil_cert_cmd_run
- - pkg: arvados_test_salt_states_examples_single_host_snakeoil_certs_ssl_cert_pkg_installed
- - require_in:
- - file: nginx_snippet_arvados-snakeoil.conf
-{%- endif %}
+ - cmd: extra_snakeoil_certs_arvados_snakeoil_cert___HOSTNAME_EXT___cmd_run
+ - pkg: extra_snakeoil_certs_ssl_cert_pkg_installed
+ {%- endif %}
DATABASE_PASSWORD=please_set_this_to_some_secure_value
# SSL CERTIFICATES
-# Arvados REQUIRES valid SSL to work correctly. Otherwise, some components will fail
-# to communicate and can silently drop traffic. You can try to use the Letsencrypt
-# salt formula (https://github.com/saltstack-formulas/letsencrypt-formula) to try to
-# automatically obtain and install SSL certificates for your instances or set this
-# variable to "no", provide and upload your own certificates to the instances and
-# modify the 'nginx_*' salt pillars accordingly (see CUSTOM_CERTS_DIR below)
-USE_LETSENCRYPT="yes"
-USE_LETSENCRYPT_IAM_USER="yes"
+# Arvados requires SSL certificates to work correctly. This installer supports these options:
+# * self-signed: let the installer create self-signed certificate(s)
+# * bring-your-own: supply your own certificate(s) in the `certs` directory
+# * lets-encrypt: automatically obtain and install SSL certificates for your hostname(s)
+#
+# See https://doc.arvados.org/intall/salt-multi-host.html for more information.
+SSL_MODE="lets-encrypt"
+USE_LETSENCRYPT_ROUTE53="yes"
# For collections, we need to obtain a wildcard certificate for
# '*.collections.<cluster>.<domain>'. This is only possible through a DNS-01 challenge.
# For that reason, you'll need to provide AWS credentials with permissions to manage
LE_AWS_SECRET_ACCESS_KEY="thisistherandomstringthatisyoursecretkey"
# If you going to provide your own certificates for Arvados, the provision script can
-# help you deploy them. In order to do that, you need to set `USE_LETSENCRYPT=no` above,
+# help you deploy them. In order to do that, you need to set `SSL_MODE=bring-your-own` above,
# and copy the required certificates under the directory specified in the next line.
# The certs will be copied from this directory by the provision script.
# Please set it to the FULL PATH to the certs dir if you're going to use a different dir
# Formulas versions
# ARVADOS_TAG="2.2.0"
# POSTGRES_TAG="v0.43.0"
-# NGINX_TAG="temp-fix-missing-statements-in-pillar"
+# NGINX_TAG="v2.8.0"
# DOCKER_TAG="v2.0.7"
# LOCALE_TAG="v0.3.4"
# LETSENCRYPT_TAG="v2.1.0"
# The domainname you want tou give to your cluster's hosts
DOMAIN="domain_fixme_or_this_wont_work"
-# Host SSL port where you want to point your browser to access Arvados
-# Defaults to 443 for regular runs, and to 8443 when called in Vagrant.
-# You can point it to another port if desired
-# In Vagrant, make sure it matches what you set in the Vagrantfile (8443)
+# External ports used by the Arvados services
CONTROLLER_EXT_SSL_PORT=443
KEEP_EXT_SSL_PORT=25101
-# Both for collections and downloads
KEEPWEB_EXT_SSL_PORT=9002
WEBSHELL_EXT_SSL_PORT=4202
WEBSOCKET_EXT_SSL_PORT=8002
WORKBENCH2_EXT_SSL_PORT=3001
INITIAL_USER="admin"
-
# If not specified, the initial user email will be composed as
# INITIAL_USER@CLUSTER.DOMAIN
INITIAL_USER_EMAIL="admin@cluster_fixme_or_this_wont_work.domain_fixme_or_this_wont_work"
DATABASE_PASSWORD=please_set_this_to_some_secure_value
# SSL CERTIFICATES
-# Arvados REQUIRES valid SSL to work correctly. Otherwise, some components will
-# fail to communicate and can silently drop traffic. Set USE_LETSENCRYPT="yes"
-# to use the Let's Encrypt salt formula
-# (https://github.com/saltstack-formulas/letsencrypt-formula) to automatically
-# obtain and install SSL certificates for your hostname(s).
+# Arvados requires SSL certificates to work correctly. This installer supports these options:
+# * self-signed: let the installer create self-signed certificate(s)
+# * bring-your-own: supply your own certificate(s) in the `certs` directory
+# * lets-encrypt: automatically obtain and install SSL certificates for your hostname(s)
#
-# Alternatively, set this variable to "no" and provide and upload your own
-# certificates to the instances and modify the 'nginx_*' salt pillars
-# accordingly
-USE_LETSENCRYPT="no"
+# See https://doc.arvados.org/intall/salt-single-host.html#certificates for more information.
+SSL_MODE="self-signed"
-# If you going to provide your own certificates for Arvados, the provision script can
-# help you deploy them. In order to do that, you need to set `USE_LETSENCRYPT=no` above,
-# and copy the required certificates under the directory specified in the next line.
-# The certs will be copied from this directory by the provision script.
-# Please set it to the FULL PATH to the certs dir if you're going to use a different dir
-# Default is "${SCRIPT_DIR}/certs", where the variable "SCRIPT_DIR" has the path to the
-# directory where the "provision.sh" script was copied in the destination host.
+# CUSTOM_CERTS_DIR is only used when SSL_MODE is set to "bring-your-own".
+# See https://doc.arvados.org/intall/salt-single-host.html#bring-your-own for more information.
# CUSTOM_CERTS_DIR="${SCRIPT_DIR}/certs"
-# The script expects cert/key files with these basenames (matching the role except for
-# keepweb, which is split in both download/collections):
-# "controller"
-# "websocket"
-# "workbench"
-# "workbench2"
-# "webshell"
-# "download" # Part of keepweb
-# "collections" # Part of keepweb
-# "keepproxy"
-# Ie., 'keepproxy', the script will lookup for
-# ${CUSTOM_CERTS_DIR}/keepproxy.crt
-# ${CUSTOM_CERTS_DIR}/keepproxy.key
# The directory to check for the config files (pillars, states) you want to use.
# There are a few examples under 'config_examples'.
# CONFIG_DIR="local_config_dir"
+
# Extra states to apply. If you use your own subdir, change this value accordingly
# EXTRA_STATES_DIR="${CONFIG_DIR}/states"
# Formulas versions
# ARVADOS_TAG="2.2.0"
# POSTGRES_TAG="v0.43.0"
-# NGINX_TAG="temp-fix-missing-statements-in-pillar"
+# NGINX_TAG="v2.8.0"
# DOCKER_TAG="v2.0.7"
# LOCALE_TAG="v0.3.4"
# LETSENCRYPT_TAG="v2.1.0"
# These are the basic parameters to configure the installation
-# The FIVE ALPHANUMERIC CHARACTERS name you want to give your cluster
+# The Arvados cluster ID, needs to be five alphanumeric characters.
CLUSTER="cluster_fixme_or_this_wont_work"
-# The domainname you want tou give to your cluster's hosts
+# The domainname for your cluster's hosts
DOMAIN="domain_fixme_or_this_wont_work"
-# Set this value when installing a cluster in a single host with a single hostname
-# to access all the instances. Not used in the other examples.
-# When using virtualization (ie AWS), this should be
-# the EXTERNAL/PUBLIC hostname for the instance.
-# If empty, ${CLUSTER}.${DOMAIN} will be used
-HOSTNAME_EXT=""
-# The internal hostname for the host. In the example files, only used in the
-# single_host/single_hostname example
-HOSTNAME_INT="127.0.1.1"
-# Host SSL port where you want to point your browser to access Arvados
-# Defaults to 443 for regular runs, and to 8443 when called in Vagrant.
-# You can point it to another port if desired
-# In Vagrant, make sure it matches what you set in the Vagrantfile (8443)
-CONTROLLER_EXT_SSL_PORT=9443
-KEEP_EXT_SSL_PORT=35101
-# Both for collections and downloads
-KEEPWEB_EXT_SSL_PORT=11002
-WEBSHELL_EXT_SSL_PORT=14202
-WEBSOCKET_EXT_SSL_PORT=18002
-WORKBENCH1_EXT_SSL_PORT=9444
-WORKBENCH2_EXT_SSL_PORT=9445
+# Set this value when installing a cluster in a single host with a single
+# hostname to access all the instances. HOSTNAME_EXT should be set to the
+# external hostname for the instance.
+HOSTNAME_EXT="hostname_ext_fixme_or_this_wont_work"
-INITIAL_USER="admin"
+# The internal IP address for the host.
+IP_INT="ip_int_fixme_or_this_wont_work"
+
+# External ports used by the Arvados services
+CONTROLLER_EXT_SSL_PORT=8800
+KEEP_EXT_SSL_PORT=8801
+KEEPWEB_EXT_SSL_PORT=8802
+WEBSHELL_EXT_SSL_PORT=8803
+WEBSOCKET_EXT_SSL_PORT=8804
+WORKBENCH1_EXT_SSL_PORT=8805
+WORKBENCH2_EXT_SSL_PORT=443
+INITIAL_USER="admin"
# If not specified, the initial user email will be composed as
# INITIAL_USER@CLUSTER.DOMAIN
INITIAL_USER_EMAIL="admin@cluster_fixme_or_this_wont_work.domain_fixme_or_this_wont_work"
INITIAL_USER_PASSWORD="password"
-# YOU SHOULD CHANGE THESE TO SOME RANDOM STRINGS
+# Populate these values with random strings
BLOB_SIGNING_KEY=blobsigningkeymushaveatleast32characters
MANAGEMENT_TOKEN=managementtokenmushaveatleast32characters
SYSTEM_ROOT_TOKEN=systemroottokenmushaveatleast32characters
DATABASE_PASSWORD=please_set_this_to_some_secure_value
# SSL CERTIFICATES
-# Arvados REQUIRES valid SSL to work correctly. Otherwise, some components will
-# fail to communicate and can silently drop traffic. Set USE_LETSENCRYPT="yes"
-# to use the Let's Encrypt salt formula
-# (https://github.com/saltstack-formulas/letsencrypt-formula) to automatically
-# obtain and install SSL certificates for your hostname(s).
+# Arvados requires SSL certificates to work correctly. This installer supports these options:
+# * self-signed: let the installer create self-signed certificate(s)
+# * bring-your-own: supply your own certificate(s) in the `certs` directory
+# * lets-encrypt: automatically obtain and install SSL certificates for your hostname(s)
#
-# Alternatively, set this variable to "no" and provide and upload your own
-# certificates to the instances and modify the 'nginx_*' salt pillars
-# accordingly
-USE_LETSENCRYPT="no"
+# See https://doc.arvados.org/intall/salt-single-host.html#certificates for more information.
+SSL_MODE="self-signed"
+
+# CUSTOM_CERTS_DIR is only used when SSL_MODE is set to "bring-your-own".
+# See https://doc.arvados.org/intall/salt-single-host.html#bring-your-own for more information.
+# CUSTOM_CERTS_DIR="${SCRIPT_DIR}/certs"
# The directory to check for the config files (pillars, states) you want to use.
# There are a few examples under 'config_examples'.
# CONFIG_DIR="local_config_dir"
+
# Extra states to apply. If you use your own subdir, change this value accordingly
# EXTRA_STATES_DIR="${CONFIG_DIR}/states"
# Formulas versions
# ARVADOS_TAG="2.2.0"
# POSTGRES_TAG="v0.43.0"
-# NGINX_TAG="temp-fix-missing-statements-in-pillar"
+# NGINX_TAG="v2.8.0"
# DOCKER_TAG="v2.0.7"
# LOCALE_TAG="v0.3.4"
# LETSENCRYPT_TAG="v2.1.0"
DOMAIN=""
# Hostnames/IPs used for single-host deploys
-HOSTNAME_EXT=""
-HOSTNAME_INT="127.0.1.1"
+IP_INT="127.0.1.1"
# Initial user setup
INITIAL_USER=""
WORKBENCH1_EXT_SSL_PORT=443
WORKBENCH2_EXT_SSL_PORT=3001
-USE_LETSENCRYPT="no"
+SSL_MODE="self-signed"
+USE_LETSENCRYPT_ROUTE53="no"
CUSTOM_CERTS_DIR="${SCRIPT_DIR}/certs"
## These are ARVADOS-related parameters
# Other formula versions we depend on
POSTGRES_TAG="v0.43.0"
-NGINX_TAG="temp-fix-missing-statements-in-pillar"
+NGINX_TAG="v2.8.0"
DOCKER_TAG="v2.0.7"
LOCALE_TAG="v0.3.4"
LETSENCRYPT_TAG="v2.1.0"
fi
# Only used in single_host/single_name deploys
-if [ "x${HOSTNAME_EXT}" = "x" ] ; then
+if [ ! -z "${HOSTNAME_EXT}" ] ; then
+ # We need to add some extra control vars to manage a single certificate vs. multiple
+ USE_SINGLE_HOSTNAME="yes"
+ # Make sure that the value configured as IP_INT is a real IP on the system.
+ # If we don't error out early here when there is a mismatch, the formula will
+ # fail with hard to interpret nginx errors later on.
+ ip addr list |grep -q " ${IP_INT}/"
+ if [[ $? -ne 0 ]]; then
+ echo "Unable to find the IP_INT address '${IP_INT}' on the system, please correct the value in local.params. Exiting..."
+ exit 1
+ fi
+else
+ USE_SINGLE_HOSTNAME="no"
+ # We set this variable, anyway, so sed lines do not fail and we don't need to add more
+ # conditionals
HOSTNAME_EXT="${CLUSTER}.${DOMAIN}"
fi
git clone --quiet https://github.com/saltstack-formulas/docker-formula.git ${F_DIR}/docker
( cd docker && git checkout --quiet tags/"${DOCKER_TAG}" -b "${DOCKER_TAG}" )
+echo "...locale"
git clone --quiet https://github.com/saltstack-formulas/locale-formula.git ${F_DIR}/locale
( cd locale && git checkout --quiet tags/"${LOCALE_TAG}" -b "${LOCALE_TAG}" )
-git clone --quiet https://github.com/netmanagers/nginx-formula.git ${F_DIR}/nginx
+echo "...nginx"
+git clone --quiet https://github.com/saltstack-formulas/nginx-formula.git ${F_DIR}/nginx
( cd nginx && git checkout --quiet tags/"${NGINX_TAG}" -b "${NGINX_TAG}" )
+echo "...postgres"
git clone --quiet https://github.com/saltstack-formulas/postgres-formula.git ${F_DIR}/postgres
( cd postgres && git checkout --quiet tags/"${POSTGRES_TAG}" -b "${POSTGRES_TAG}" )
+echo "...letsencrypt"
git clone --quiet https://github.com/saltstack-formulas/letsencrypt-formula.git ${F_DIR}/letsencrypt
( cd letsencrypt && git checkout --quiet tags/"${LETSENCRYPT_TAG}" -b "${LETSENCRYPT_TAG}" )
+echo "...arvados"
git clone --quiet https://git.arvados.org/arvados-formula.git ${F_DIR}/arvados
# If we want to try a specific branch of the formula
s#__CLUSTER__#${CLUSTER}#g;
s#__DOMAIN__#${DOMAIN}#g;
s#__HOSTNAME_EXT__#${HOSTNAME_EXT}#g;
- s#__HOSTNAME_INT__#${HOSTNAME_INT}#g;
+ s#__IP_INT__#${IP_INT}#g;
s#__INITIAL_USER_EMAIL__#${INITIAL_USER_EMAIL}#g;
s#__INITIAL_USER_PASSWORD__#${INITIAL_USER_PASSWORD}#g;
s#__INITIAL_USER__#${INITIAL_USER}#g;
mkdir -p ${T_DIR}
# Replace cluster and domain name in the test files
for f in $(ls "${SOURCE_TESTS_DIR}"/*); do
- sed "s#__CLUSTER__#${CLUSTER}#g;
+ FILTERS="s#__CLUSTER__#${CLUSTER}#g;
s#__CONTROLLER_EXT_SSL_PORT__#${CONTROLLER_EXT_SSL_PORT}#g;
s#__DOMAIN__#${DOMAIN}#g;
- s#__HOSTNAME_INT__#${HOSTNAME_INT}#g;
+ s#__IP_INT__#${IP_INT}#g;
s#__INITIAL_USER_EMAIL__#${INITIAL_USER_EMAIL}#g;
s#__INITIAL_USER_PASSWORD__#${INITIAL_USER_PASSWORD}#g
s#__INITIAL_USER__#${INITIAL_USER}#g;
s#__DATABASE_PASSWORD__#${DATABASE_PASSWORD}#g;
- s#__SYSTEM_ROOT_TOKEN__#${SYSTEM_ROOT_TOKEN}#g" \
- "${f}" > ${T_DIR}/$(basename "${f}")
+ s#__SYSTEM_ROOT_TOKEN__#${SYSTEM_ROOT_TOKEN}#g"
+ if [ "$USE_SINGLE_HOSTNAME" = "yes" ]; then
+ FILTERS="s#__CLUSTER__.__DOMAIN__#${HOSTNAME_EXT}#g;
+ $FILTERS"
+ fi
+ sed "$FILTERS" \
+ "${f}" > ${T_DIR}/$(basename "${f}")
done
chmod 755 ${T_DIR}/run-test.sh
s#__CONTROLLER_EXT_SSL_PORT__#${CONTROLLER_EXT_SSL_PORT}#g;
s#__DOMAIN__#${DOMAIN}#g;
s#__HOSTNAME_EXT__#${HOSTNAME_EXT}#g;
- s#__HOSTNAME_INT__#${HOSTNAME_INT}#g;
+ s#__IP_INT__#${IP_INT}#g;
s#__INITIAL_USER_EMAIL__#${INITIAL_USER_EMAIL}#g;
s#__INITIAL_USER_PASSWORD__#${INITIAL_USER_PASSWORD}#g;
s#__INITIAL_USER__#${INITIAL_USER}#g;
# States, extra states
if [ -d "${F_DIR}"/extra/extra ]; then
- if [ "$DEV_MODE" = "yes" ]; then
+ SKIP_SNAKE_OIL="snakeoil_certs"
+
+ if [[ "$DEV_MODE" = "yes" || "${SSL_MODE}" == "self-signed" ]] ; then
# In dev mode, we create some snake oil certs that we'll
- # use as CUSTOM_CERTS, so we don't skip the states file
- SKIP_SNAKE_OIL="dont_snakeoil_certs"
- else
- SKIP_SNAKE_OIL="snakeoil_certs"
+ # use as CUSTOM_CERTS, so we don't skip the states file.
+ # Same when using self-signed certificates.
+ SKIP_SNAKE_OIL="dont_add_snakeoil_certs"
fi
for f in $(ls "${F_DIR}"/extra/extra/*.sls | grep -v ${SKIP_SNAKE_OIL}); do
echo " - extra.$(basename ${f} | sed 's/.sls$//g')" >> ${S_DIR}/top.sls
done
- # Use custom certs
- if [ "x${USE_LETSENCRYPT}" != "xyes" ]; then
+ # Use byo or self-signed certificates
+ if [ "${SSL_MODE}" != "lets-encrypt" ]; then
mkdir -p "${F_DIR}"/extra/extra/files
fi
fi
if [ -z "${ROLES}" ]; then
# States
echo " - nginx.passenger" >> ${S_DIR}/top.sls
- # Currently, only available on config_examples/multi_host/aws
- if [ "x${USE_LETSENCRYPT}" = "xyes" ]; then
- if [ "x${USE_LETSENCRYPT_IAM_USER}" != "xyes" ]; then
+ if [ "${SSL_MODE}" = "lets-encrypt" ]; then
+ if [ "${USE_LETSENCRYPT_ROUTE53}" = "yes" ]; then
grep -q "aws_credentials" ${S_DIR}/top.sls || echo " - extra.aws_credentials" >> ${S_DIR}/top.sls
fi
grep -q "letsencrypt" ${S_DIR}/top.sls || echo " - letsencrypt" >> ${S_DIR}/top.sls
else
- # Use custom certs
+ # Use custom certs, as both bring-your-own and self-signed are copied using this state
# Copy certs to formula extra/files
# In dev mode, the files will be created and put in the destination directory by the
# snakeoil_certs.sls state file
echo " - nginx_workbench_configuration" >> ${P_DIR}/top.sls
echo " - postgresql" >> ${P_DIR}/top.sls
- # Currently, only available on config_examples/multi_host/aws
- if [ "x${USE_LETSENCRYPT}" = "xyes" ]; then
- if [ "x${USE_LETSENCRYPT_IAM_USER}" != "xyes" ]; then
+ if [ "${SSL_MODE}" = "lets-encrypt" ]; then
+ if [ "${USE_LETSENCRYPT_ROUTE53}" = "yes" ]; then
grep -q "aws_credentials" ${P_DIR}/top.sls || echo " - aws_credentials" >> ${P_DIR}/top.sls
fi
- grep -q "letsencrypt" ${P_DIR}/top.sls || echo " - letsencrypt" >> ${P_DIR}/top.sls
+ grep -q "letsencrypt" ${P_DIR}/top.sls || echo " - letsencrypt" >> ${P_DIR}/top.sls
# As the pillar differ whether we use LE or custom certs, we need to do a final edition on them
- for c in controller websocket workbench workbench2 webshell download collections keepproxy; do
- sed -i "s/__CERT_REQUIRES__/cmd: create-initial-cert-${c}.${CLUSTER}.${DOMAIN}*/g;
- s#__CERT_PEM__#/etc/letsencrypt/live/${c}.${CLUSTER}.${DOMAIN}/fullchain.pem#g;
- s#__CERT_KEY__#/etc/letsencrypt/live/${c}.${CLUSTER}.${DOMAIN}/privkey.pem#g" \
+ for c in controller websocket workbench workbench2 webshell keepweb keepproxy; do
+ if [ "${USE_SINGLE_HOSTNAME}" = "yes" ]; then
+ # Are we in a single-host-single-hostname env?
+ CERT_NAME=${HOSTNAME_EXT}
+ else
+ # We are in a single-host-multiple-hostnames env
+ CERT_NAME=${c}.${CLUSTER}.${DOMAIN}
+ fi
+
+ sed -i "s/__CERT_REQUIRES__/cmd: create-initial-cert-${CERT_NAME}*/g;
+ s#__CERT_PEM__#/etc/letsencrypt/live/${CERT_NAME}/fullchain.pem#g;
+ s#__CERT_KEY__#/etc/letsencrypt/live/${CERT_NAME}/privkey.pem#g" \
${P_DIR}/nginx_${c}_configuration.sls
done
else
echo "extra_custom_certs_dir: /srv/salt/certs" > ${P_DIR}/extra_custom_certs.sls
echo "extra_custom_certs:" >> ${P_DIR}/extra_custom_certs.sls
- for c in controller websocket workbench workbench2 webshell download collections keepproxy; do
- grep -q ${c} ${P_DIR}/extra_custom_certs.sls || echo " - ${c}" >> ${P_DIR}/extra_custom_certs.sls
-
- # As the pillar differ whether we use LE or custom certs, we need to do a final edition on them
- sed -i "s/__CERT_REQUIRES__/file: extra_custom_certs_file_copy_arvados-${c}.pem/g;
- s#__CERT_PEM__#/etc/nginx/ssl/arvados-${c}.pem#g;
- s#__CERT_KEY__#/etc/nginx/ssl/arvados-${c}.key#g" \
+ for c in controller websocket workbench workbench2 webshell keepweb keepproxy; do
+ # Are we in a single-host-single-hostname env?
+ if [ "${USE_SINGLE_HOSTNAME}" = "yes" ]; then
+ # Are we in a single-host-single-hostname env?
+ CERT_NAME=${HOSTNAME_EXT}
+ else
+ # We are in a multiple-hostnames env
+ CERT_NAME=${c}
+ fi
+
+ if [[ "$SSL_MODE" == "bring-your-own" ]]; then
+ copy_custom_cert ${CUSTOM_CERTS_DIR} ${CERT_NAME}
+ fi
+
+ grep -q ${CERT_NAME} ${P_DIR}/extra_custom_certs.sls || echo " - ${CERT_NAME}" >> ${P_DIR}/extra_custom_certs.sls
+
+ # As the pillar differs whether we use LE or custom certs, we need to do a final edition on them
+ sed -i "s/__CERT_REQUIRES__/file: extra_custom_certs_file_copy_arvados-${CERT_NAME}.pem/g;
+ s#__CERT_PEM__#/etc/nginx/ssl/arvados-${CERT_NAME}.pem#g;
+ s#__CERT_KEY__#/etc/nginx/ssl/arvados-${CERT_NAME}.key#g" \
${P_DIR}/nginx_${c}_configuration.sls
done
fi
else
# If we add individual roles, make sure we add the repo first
echo " - arvados.repo" >> ${S_DIR}/top.sls
- # We add the custom_certs state
- grep -q "custom_certs" ${S_DIR}/top.sls || echo " - extra.custom_certs" >> ${S_DIR}/top.sls
+ # We add the extra_custom_certs state
+ grep -q "extra.custom_certs" ${S_DIR}/top.sls || echo " - extra.custom_certs" >> ${S_DIR}/top.sls
# And we add the basic part for the certs pillar
- if [ "x${USE_LETSENCRYPT}" != "xyes" ]; then
+ if [ "${SSL_MODE}" != "lets-encrypt" ]; then
# And add the certs in the custom_certs pillar
echo "extra_custom_certs_dir: /srv/salt/certs" > ${P_DIR}/extra_custom_certs.sls
echo "extra_custom_certs:" >> ${P_DIR}/extra_custom_certs.sls
### If we don't install and run LE before arvados-api-server, it fails and breaks everything
### after it. So we add this here as we are, after all, sharing the host for api and controller
# Currently, only available on config_examples/multi_host/aws
- if [ "x${USE_LETSENCRYPT}" = "xyes" ]; then
- if [ "x${USE_LETSENCRYPT_IAM_USER}" != "xyes" ]; then
+ if [ "${SSL_MODE}" = "lets-encrypt" ]; then
+ if [ "${USE_LETSENCRYPT_ROUTE53}" = "yes" ]; then
grep -q "aws_credentials" ${S_DIR}/top.sls || echo " - aws_credentials" >> ${S_DIR}/top.sls
fi
grep -q "letsencrypt" ${S_DIR}/top.sls || echo " - letsencrypt" >> ${S_DIR}/top.sls
else
# Use custom certs
- copy_custom_cert ${CUSTOM_CERTS_DIR} controller
+ if [ "${SSL_MODE}" = "bring-your-own" ]; then
+ copy_custom_cert ${CUSTOM_CERTS_DIR} controller
+ fi
grep -q controller ${P_DIR}/extra_custom_certs.sls || echo " - controller" >> ${P_DIR}/extra_custom_certs.sls
fi
grep -q "arvados.${R}" ${S_DIR}/top.sls || echo " - arvados.${R}" >> ${S_DIR}/top.sls
# States
grep -q "nginx.passenger" ${S_DIR}/top.sls || echo " - nginx.passenger" >> ${S_DIR}/top.sls
# Currently, only available on config_examples/multi_host/aws
- if [ "x${USE_LETSENCRYPT}" = "xyes" ]; then
- if [ "x${USE_LETSENCRYPT_IAM_USER}" != "xyes" ]; then
+ if [ "${SSL_MODE}" = "lets-encrypt" ]; then
+ if [ "x${USE_LETSENCRYPT_ROUTE53}" = "xyes" ]; then
grep -q "aws_credentials" ${S_DIR}/top.sls || echo " - aws_credentials" >> ${S_DIR}/top.sls
fi
grep -q "letsencrypt" ${S_DIR}/top.sls || echo " - letsencrypt" >> ${S_DIR}/top.sls
else
# Use custom certs, special case for keepweb
if [ ${R} = "keepweb" ]; then
- copy_custom_cert ${CUSTOM_CERTS_DIR} download
- copy_custom_cert ${CUSTOM_CERTS_DIR} collections
+ if [ "${SSL_MODE}" = "bring-your-own" ]; then
+ copy_custom_cert ${CUSTOM_CERTS_DIR} download
+ copy_custom_cert ${CUSTOM_CERTS_DIR} collections
+ fi
else
- copy_custom_cert ${CUSTOM_CERTS_DIR} ${R}
+ if [ "${SSL_MODE}" = "bring-your-own" ]; then
+ copy_custom_cert ${CUSTOM_CERTS_DIR} ${R}
+ fi
fi
fi
# webshell role is just a nginx vhost, so it has no state
fi
# Currently, only available on config_examples/multi_host/aws
- if [ "x${USE_LETSENCRYPT}" = "xyes" ]; then
- if [ "x${USE_LETSENCRYPT_IAM_USER}" != "xyes" ]; then
+ if [ "${SSL_MODE}" = "lets-encrypt" ]; then
+ if [ "${USE_LETSENCRYPT_ROUTE53}" = "yes" ]; then
grep -q "aws_credentials" ${P_DIR}/top.sls || echo " - aws_credentials" >> ${P_DIR}/top.sls
fi
grep -q "letsencrypt" ${P_DIR}/top.sls || echo " - letsencrypt" >> ${P_DIR}/top.sls
# Now run the install
salt-call --local state.apply -l ${LOG_LEVEL}
+# Finally, make sure that /etc/hosts is not overwritten on reboot
+if [ -d /etc/cloud/cloud.cfg.d ]; then
+ # TODO: will this work on CentOS?
+ sed -i 's/^manage_etc_hosts: true/#manage_etc_hosts: true/g' /etc/cloud/cloud.cfg.d/*
+fi
+
# FIXME! #16992 Temporary fix for psql call in arvados-api-server
if [ "x${DELETE_PSQL}" = "xyes" ]; then
echo "Removing .psql file"
echo "Arvados project uuid is '${project_uuid}'"
-echo "Uploading arvados/jobs' docker image to the project"
-VERSION="2.1.1"
-arv-keepdocker --pull arvados/jobs "${VERSION}" --project-uuid "${project_uuid}"
-
# Create the initial user
echo "Creating initial user '__INITIAL_USER__'"
user_uuid=$(arv --format=uuid user list --filters '[["email", "=", "__INITIAL_USER_EMAIL__"], ["username", "=", "__INITIAL_USER__"]]')
projects / arvados.git / commitdiff