SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
+# "Introduction":#introduction
# "Hosts preparation":#hosts_preparation
-## "Hosts setup using terraform (experimental)":#hosts_setup_using_terraform
## "Create a compute image":#create_a_compute_image
# "Multi host install using the provision.sh script":#multi_host
# "Choose the desired configuration":#choose_configuration
# "Run the provision.sh script":#run_provision_script
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
+# "After the installation":#post_install
+
+h2(#introduction). Introduction
+
+Arvados components can be installed in a distributed infrastructure, whether it is an "on-prem" with physical or virtual hosts, or a cloud environment.
+
+As infrastructures vary a great deal from site to site, these instructions should be considered more as 'guidelines' than fixed steps to follow.
+
+We provide an "installer script":salt.html that can help you deploy the different Arvados components. At the time of writing, the provided examples are suitable to install Arvados on AWS.
+
+
h2(#hosts_preparation). Hosts preparation
## arvados controller
## arvados websocket
## arvados cloud dispatcher
+## arvados keepbalance
# WORKBENCH node:
## arvados workbench
## arvados workbench2
Note that these hosts can be virtual machines in your infrastructure and they don't need to be physical machines.
-h3(#hosts_setup_using_terraform). Hosts setup using terraform (experimental)
-
-We added a few "terraform":https://terraform.io/ scripts (https://github.com/arvados/arvados/tree/master/tools/terraform) to let you create these instances easier.
-Check "the Arvados terraform documentation":/doc/install/terraform.html for more details.
+Again, if your infrastructure differs from the setup proposed above (ie, using RDS or an existing DB server), remember that you will need to edit the configuration files for the scripts so they work with your infrastructure.
h2(#multi_host). Multi host install using the provision.sh script
-{% if site.current_version %}
-{% assign branchname = site.current_version | slice: 1, 5 | append: '-dev' %}
-{% else %}
-{% assign branchname = 'master' %}
-{% endif %}
+{% include 'branchname' %}
-This is a package-based installation method. Start with the @provision.sh@ script which is available by cloning the @{{ branchname }}@ branch from "https://git.arvados.org/arvados.git":https://git.arvados.org/arvados.git . The @provision.sh@ script and its supporting files can be found in the "arvados/tools/salt-install":https://git.arvados.org/arvados.git/tree/refs/heads/{{ branchname }}:/tools/salt-install directory in the Arvados git repository.
+This is a package-based installation method. Start with the @provision.sh@ script which is available by cloning the @{{ branchname }}@ branch from "https://git.arvados.org/arvados.git":https://git.arvados.org/arvados.git . The @provision.sh@ script and its supporting files can be found in the "arvados/tools/salt-install":https://git.arvados.org/arvados.git/tree/refs/heads/{{ branchname }}:/tools/salt-install directory in the Arvados git repository.
This procedure will install all the main Arvados components to get you up and running in a multi-host environment.
h3(#create_a_compute_image). Create a compute image
-In a multi-host installation, containers are dispatched in docker daemons running in the <i>compute instances</i>, which need some special setup. We provide a "compute image builder script":https://github.com/arvados/arvados/tree/master/tools/compute-images that you can use to build a template image following "these instructions":https://doc.arvados.org/main/install/crunch2-cloud/install-compute-node.html . Once you have that image created, you can use the image reference in the Arvados configuration in the next steps.
+In a multi-host installation, containers are dispatched in docker daemons running in the <i>compute instances</i>, which need some special setup. We provide a "compute image builder script":https://github.com/arvados/arvados/tree/main/tools/compute-images that you can use to build a template image following "these instructions":https://doc.arvados.org/main/install/crunch2-cloud/install-compute-node.html . Once you have that image created, you can use the image ID in the Arvados configuration in the next steps.
h2(#choose_configuration). Choose the desired configuration
Edit the variables in the <i>local.params</i> file. Pay attention to the <b>*_INT_IP, *_TOKEN</b> and <b>*KEY</b> variables. Those variables will be used to do a search and replace on the <i>pillars/*</i> in place of any matching __VARIABLE__.
-The <i>multi_host</i> include LetsEncrypt salt code to automatically request and install the certificates for the public-facing hosts (API, Workbench) so it will need the hostnames to be reachable from the Internet. If this cluster will not be the case, please set the variable <i>USE_LETSENCRYPT=no</i>.
+The <i>multi_host</i> example includes Let's Encrypt salt code to automatically request and install the certificates for the public-facing hosts (API/controller, Workbench, Keepproxy/Keepweb) using AWS' Route53.
+
+{% include 'multi_host_install_custom_certificates' %}
+
+If you want to use valid certificates provided by Let's Encrypt, set the variable <i>SSL_MODE=lets-encrypt</i> and make sure that all the FQDNs that you will use for the public-facing applications (API/controller, Workbench, Keepproxy/Keepweb) are reachable.
h3(#further_customization). Further customization of the installation (modifying the salt pillars and states)
-You will need further customization to suit your environment, which can be done editing the Saltstack pillars and states files. Pay particular attention to the <i>pillars/arvados.sls</i> file, where you will need to provide some information that can be retrieved as output of the terraform run.
+You will need further customization to suit your environment, which can be done editing the Saltstack pillars and states files. Pay particular attention to the <i>pillars/arvados.sls</i> file, where you will need to provide some information that describes your environment.
Any extra <i>state</i> file you add under <i>local_config_dir/states</i> will be added to the salt run and applied to the hosts.
A few Arvados nodes need to be installed in certain order. The required order is
-#. Database
-#. API server
-#. The other nodes can be installed in any order after the two above
+* Database
+* API server
+* The other nodes can be installed in any order after the two above
h2(#run_provision_script). Run the provision.sh script
The distribution of role as described above can be applied running these commands:
-#. Database
+h4. Database
<notextile>
<pre><code>scp -r provision.sh local* user@host:
ssh user@host sudo ./provision.sh --config local.params --roles database
</code></pre>
</notextile>
-#. API
+h4. API
<notextile>
<pre><code>scp -r provision.sh local* user@host:
-ssh user@host sudo ./provision.sh --config local.params --roles api,controller,websocket,dispatcher
+ssh user@host sudo ./provision.sh --config local.params --roles api,controller,websocket,dispatcher,keepbalance
</code></pre>
</notextile>
-#. Keepstore/s
+h4. Keepstore(s)
<notextile>
<pre><code>scp -r provision.sh local* user@host:
ssh user@host sudo ./provision.sh --config local.params --roles keepstore
</code></pre>
</notextile>
-#. Workbench
+h4. Workbench
<notextile>
<pre><code>scp -r provision.sh local* user@host:
ssh user@host sudo ./provision.sh --config local.params --roles workbench,workbench2,webshell
</code></pre>
</notextile>
-#. Keepproxy / Keepweb
+h4. Keepproxy / Keepweb
<notextile>
<pre><code>scp -r provision.sh local* user@host:
ssh user@host sudo ./provision.sh --config local.params --roles keepproxy,keepweb
</code></pre>
</notextile>
-#. Shell (here we copy the CLI test workflow too)
+h4. Shell (here we copy the CLI test workflow too)
<notextile>
<pre><code>scp -r provision.sh local* tests user@host:
ssh user@host sudo ./provision.sh --config local.params --roles shell
INFO Final process status is success
</code></pre>
</notextile>
+
+h2(#post_install). After the installation
+
+Once the installation is complete, it is recommended to keep a copy of your local configuration files. Committing them to version control is a good idea.
+
+Re-running the Salt-based installer is not recommended for maintaining and upgrading Arvados, please see "Maintenance and upgrading":{{site.baseurl}}/admin/maintenance-and-upgrading.html for more information.
projects / arvados.git / blobdiff