X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/415ecc439212c2a670b1df05c3e8b1a90245243e..8626abb0a44cfc303bef3552a7bc57163c79231a:/doc/install/install-api-server.html.textile.liquid
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index 7bfad83692..fb43f783f0 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -16,24 +16,24 @@ h3(#install_postgres). Install PostgreSQL
{% include 'install_postgres' %}
-h3(#build_tools_apiserver). Build tools
-
-On older distributions, you may need to use a backports repository to satisfy these requirements. For example, on older Red Hat-based systems, consider using the "postgresql92":https://www.softwarecollections.org/en/scls/rhscl/postgresql92/ and "nginx16":https://www.softwarecollections.org/en/scls/rhscl/nginx16/ Software Collections.
+h2(#install_apiserver). Install API server and dependencies
On a Debian-based system, install the following packages:
-~$ sudo apt-get install bison build-essential libcurl4-openssl-dev git nginx arvados-api-server
+~$ sudo apt-get install bison build-essential libcurl4-openssl-dev git arvados-api-server
On a Red Hat-based system, install the following packages:
-~$ sudo yum install bison make automake gcc gcc-c++ libcurl-devel nginx git arvados-api-server
+~$ sudo yum install bison make automake gcc gcc-c++ libcurl-devel git arvados-api-server
+{% include 'install_git' %}
+
h2. Set up the database
Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
@@ -84,18 +84,15 @@ Edit @/etc/arvados/api/database.yml@ and replace the @xxxxxxxx@ database passwor
h2(#configure_application). Configure the API server
-Edit @/etc/arvados/api/application.yml@ and to configure the settings described in the following sections. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. The settings in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided as a starting template only.
+Edit @/etc/arvados/api/application.yml@ to configure the settings described in the following sections. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. The settings in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided as a starting template only.
-Only put local configuration in @application.yml@, do not edit @application.default.yml@.
+@config/application.default.yml@ documents additional configuration settings not listed here. You can "view the current source version":https://dev.arvados.org/projects/arvados/repository/revisions/master/entry/services/api/config/application.default.yml for reference.
-h3(#uuid_prefix). uuid_prefix
+Only put local configuration in @application.yml@. Do not edit @application.default.yml@.
-The @uuid_prefix@ is used for all database identifiers to identify the record as originating from this site. It is a string consisting of exactly 5 lowercase ASCII letters and digits. Generate a random value and set it in @application.yml@:
+h3(#uuid_prefix). uuid_prefix
-
-~$ ruby -e 'puts rand(2**40).to_s(36)[0,5]'
-zzzzz
-
+Define your @uuid_prefix@ in @application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 lowercase ASCII letters and digits.
Example @application.yml@:
@@ -105,7 +102,7 @@ Example @application.yml@:
h3. secret_token
-The @secret_token@ is used for for signing cookies. IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it to @application.yml@:
+The @secret_token@ is used for for signing cookies. IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it in @application.yml@:
~$ ruby -e 'puts rand(2**400).to_s(36)'
@@ -120,7 +117,7 @@ Example @application.yml@:
h3(#blob_signing_key). blob_signing_key
-The @blob_signing_key@ is used to enforce access control to Keep blocks. This same key must be provided to the Keepstore daemons when "installing Keepstore servers.":install-keepstore.html IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it to @application.yml@:
+The @blob_signing_key@ is used to enforce access control to Keep blocks. This same key must be provided to the Keepstore daemons when "installing Keepstore servers.":install-keepstore.html IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it in @application.yml@:
~$ ruby -e 'puts rand(2**400).to_s(36)'
@@ -160,45 +157,45 @@ Example @application.yml@:
workbench_address: https://workbench.zzzzz.example.com
-h3. websockets_address
+h3. websocket_address
-Set @websockets_address@ to the @@wss://@ URL of the API server websocket endpoint after following "Set up Web servers.":#set_up
+Set @websocket_address@ to the @wss://@ URL of the API server websocket endpoint after following "Set up Web servers":#set_up. The path of the default endpoint is @/websocket@.
Example @application.yml@:
- websockets_address: https://ws.zzzzz.example.com
+ websocket_address: wss://ws.zzzzz.example.com/websocket
-h3(#git_repositories_dir). git_repositories_dir, git_internal_dir
+h3(#git_repositories_dir). git_repositories_dir
-The @git_repositories_dir@ setting specifies the directory is where user git repositories will be stored. By default this is @/var/lib/arvados/git@. You can change the location by defining it in @application.yml@.
+The @git_repositories_dir@ setting specifies the directory where user git repositories will be stored.
-The @git_internal_dir@ setting specifies the directory is where the system internal git repository will be stored. By default this is @/var/lib/arvados/internal.git@. This repository stores git branches that have been used to run crunch jobs.
+The git server setup process is covered on "its own page":install-arv-git-httpd.html. For now, create an empty directory in the default location:
-Example @application.yml@:
+
+~$ sudo mkdir -p /var/lib/arvados/git/repositories
+
+
+If you intend to store your git repositories in a different location, specify that location in @application.yml@.
+
+Default setting in @application.default.yml@:
- git_repositories_dir: /var/lib/arvados/git
- git_internal_dir: /var/lib/arvados/internal.git
+ git_repositories_dir: /var/lib/arvados/git/repositories
-h3. Additional settings
-
-The file @config/application.default.yml@ (online at "https://arvados.org/projects/arvados/repository/revisions/master/entry/services/api/config/application.default.yml":https://arvados.org/projects/arvados/repository/revisions/master/entry/services/api/config/application.default.yml documents a number of additional configuration settings. Only put local configuration in @application.yml@, do not edit @application.default.yml@.
+h3(#git_internal_dir). git_internal_dir
-h2. Prepare the API server deployment
+The @git_internal_dir@ setting specifies the location of Arvados' internal git repository. By default this is @/var/lib/arvados/internal.git@. This repository stores git commits that have been used to run Crunch jobs. It should _not_ be a subdirectory of @git_repositories_dir@.
-Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
+Example @application.yml@:
-{% include 'notebox_begin' %}
-You can safely ignore the following error message you may see when loading the database structure:
-ERROR: must be owner of extension plpgsql
-{% include 'notebox_end' %}
-
-This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
+ git_internal_dir: /var/lib/arvados/internal.git
+
+
h2(#set_up). Set up Web servers
@@ -206,11 +203,9 @@ For best performance, we recommend you use Nginx as your Web server front-end, w
-- Install Nginx via your distribution or a backports repository.
-
-- Install Phusion Passenger for Nginx.
+- Install Nginx and Phusion Passenger.
-Puma is already included with the API server's gems. We recommend you use a tool like runit or something similar. Here's a sample run script for that:
+Puma is already included with the API server's gems. We recommend you run it as a service under runit or a similar tool. Here's a sample runit script for that:
#!/bin/bash
@@ -227,8 +222,12 @@ echo ws-only > "$envdir/ARVADOS_WEBSOCKETS"
cd /var/www/arvados-api/current
echo "Starting puma in `pwd`"
-# You may need to change arguments below to match your deployment, especially -u.
-exec chpst -m 1073741824 -u www-data:www-data -e "$envdir" \
+# Change arguments below to match your deployment, "webserver-user" and
+# "webserver-group" should be changed to the user and group of the web server
+# process. This is typically "www-data:www-data" on Debian systems by default,
+# other systems may use different defaults such the name of the web server
+# software (for example, "nginx:nginx").
+exec chpst -m 1073741824 -u webserver-user:webserver-group -e "$envdir" \
bundle exec puma -t 0:512 -e production -b tcp://127.0.0.1:8100
@@ -258,17 +257,37 @@ upstream websockets {
proxy_http_version 1.1;
+# When Keep clients request a list of Keep services from the API server, the
+# server will automatically return the list of available proxies if
+# the request headers include X-External-Client: 1. Following the example
+# here, at the end of this section, add a line for each netmask that has
+# direct access to Keep storage daemons to set this header value to 0.
+geo $external_client {
+ default 1;
+ 10.20.30.0/24 0;
+}
+
server {
listen [your public IP address]:443 ssl;
server_name uuid_prefix.your.domain;
ssl on;
+ ssl_certificate /YOUR/PATH/TO/cert.pem;
+ ssl_certificate_key /YOUR/PATH/TO/cert.key;
index index.html index.htm index.php;
+ # This value effectively limits the size of API objects users can create,
+ # especially collections. If you change this, you should also set
+ # `max_request_size` in the API server's application.yml file to the same
+ # value.
+ client_max_body_size 128m;
+
location / {
proxy_pass http://api;
proxy_redirect off;
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
@@ -283,12 +302,16 @@ server {
server_name ws.uuid_prefix.your.domain;
ssl on;
+ ssl_certificate /YOUR/PATH/TO/cert.pem;
+ ssl_certificate_key /YOUR/PATH/TO/cert.key;
index index.html index.htm index.php;
location / {
proxy_pass http://websockets;
proxy_redirect off;
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
@@ -300,7 +323,26 @@ server {
-- Restart Nginx.
+Restart Nginx:
+
+~$ sudo nginx -s reload
+
+
+
+
+h2. Prepare the API server deployment
+
+Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
+
+{% include 'notebox_begin' %}
+You can safely ignore the following messages if they appear while this script runs:
+
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will
+break this application for all non-root users on this machine.
+fatal: Not a git repository (or any of the parent directories): .git
+{% include 'notebox_end' %}
+
+This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
+