---
layout: default
navsection: installguide
title: Install the API server
...
h2. Install prerequisites
The Arvados package repository includes an API server package that can help automate much of the deployment.
h3(#install_ruby_and_bundler). Install Ruby and Bundler
{% include 'install_ruby_and_bundler' %}
h3(#install_postgres). Install PostgreSQL
{% include 'install_postgres' %}
h2(#install_apiserver). Install API server and dependencies
On a Debian-based system, install the following packages:
~$ sudo apt-get install bison build-essential libcurl4-openssl-dev git arvados-api-server
On a Red Hat-based system, install the following packages:
~$ sudo yum install bison make automake gcc gcc-c++ libcurl-devel git arvados-api-server
h2. Set up the database
Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
~$ ruby -e 'puts rand(2**128).to_s(36)'
6gqa1vu492idd7yca9tfandj3
Create a new database user.
~$ sudo -u postgres createuser --encrypted -R -S --pwprompt arvados
[sudo] password for you: yourpassword
Enter password for new role: paste-password-you-generated
Enter it again: paste-password-again
{% include 'notebox_begin' %}
This user setup assumes that your PostgreSQL is configured to accept password authentication. Red Hat systems use ident-based authentication by default. You may need to either adapt the user creation, or reconfigure PostgreSQL (in @pg_hba.conf@) to accept password authentication.
{% include 'notebox_end' %}
Create the database:
~$ sudo -u postgres createdb arvados_production -T template0 -E UTF8 -O arvados
h2. Set up configuration files
The API server package uses configuration files that you write to @/etc/arvados/api@ and ensures they're consistently deployed. Create this directory and copy the example configuration files to it:
~$ sudo mkdir -p /etc/arvados/api
~$ sudo chmod 700 /etc/arvados/api
~$ cd /var/www/arvados-api/current
/var/www/arvados-api/current$ sudo cp config/database.yml.example /etc/arvados/api/database.yml
/var/www/arvados-api/current$ sudo cp config/application.yml.example /etc/arvados/api/application.yml
h2. Configure the database connection
Edit @/etc/arvados/api/database.yml@ and replace the @xxxxxxxx@ database password placeholders with the PostgreSQL password you generated above.
h2(#configure_application). Configure the API server
Edit @/etc/arvados/api/application.yml@ to configure the settings described in the following sections. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. The settings in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided as a starting template only.
@config/application.default.yml@ documents additional configuration settings not listed here. You can "view the current source version":https://arvados.org/projects/arvados/repository/revisions/master/entry/services/api/config/application.default.yml for reference.
Only put local configuration in @application.yml@. Do not edit @application.default.yml@.
h3(#uuid_prefix). uuid_prefix
Define your @uuid_prefix@ in @application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 lowercase ASCII letters and digits.
Example @application.yml@:
uuid_prefix: zzzzz
h3. secret_token
The @secret_token@ is used for for signing cookies. IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it in @application.yml@:
~$ ruby -e 'puts rand(2**400).to_s(36)'
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
Example @application.yml@:
secret_token: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
h3(#blob_signing_key). blob_signing_key
The @blob_signing_key@ is used to enforce access control to Keep blocks. This same key must be provided to the Keepstore daemons when "installing Keepstore servers.":install-keepstore.html IMPORTANT: This is a site secret. It should be at least 50 characters. Generate a random value and set it in @application.yml@:
~$ ruby -e 'puts rand(2**400).to_s(36)'
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Example @application.yml@:
blob_signing_key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
h3(#omniauth). sso_app_secret, sso_app_id, sso_provider_url
The following settings enable the API server to communicate with the "Single Sign On (SSO) server":install-sso.html to authenticate user log in.
Set @sso_provider_url@ to the base URL where your SSO server is installed. This should be a URL consisting of the scheme and host (and optionally, port), without a trailing slash.
Set @sso_app_secret@ and @sso_app_id@ to the corresponding values for @app_secret@ and @app_id@ used in the "Create arvados-server client for Single Sign On (SSO)":install-sso.html#client step.
Example @application.yml@:
sso_app_id: arvados-server
sso_app_secret: wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
sso_provider_url: https://sso.example.com
h3. workbench_address
Set @workbench_address@ to the URL of your workbench application after following "Install Workbench.":install-workbench-app.html
Example @application.yml@:
workbench_address: https://workbench.zzzzz.example.com
h3. websockets_address
Set @websockets_address@ to the @wss://@ URL of the API server websocket endpoint after following "Set up Web servers.":#set_up
Example @application.yml@:
websockets_address: wss://ws.zzzzz.example.com
h3(#git_repositories_dir). git_repositories_dir
The @git_repositories_dir@ setting specifies the directory where user git repositories will be stored. By default this is @/var/lib/arvados/git@.
Example @application.yml@:
git_repositories_dir: /var/lib/arvados/git
Make sure a clone of the arvados repository exists in @git_repositories_dir@.
~$ sudo mkdir -p /var/lib/arvados/git
~$ sudo git clone --bare git://git.curoverse.com/arvados.git /var/lib/arvados/git/arvados.git
h3(#git_internal_dir). git_internal_dir
The @git_internal_dir@ setting specifies the location of Arvados' internal git repository. By default this is @/var/lib/arvados/internal.git@. This repository stores git commits that have been used to run Crunch jobs. It should _not_ be a subdirectory of @git_repositories_dir@.
Example @application.yml@:
git_internal_dir: /var/lib/arvados/internal.git
h2. Prepare the API server deployment
Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
{% include 'notebox_begin' %}
You can safely ignore the following error message you may see when loading the database structure:
ERROR: must be owner of extension plpgsql
{% include 'notebox_end' %}
This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
h2(#set_up). Set up Web servers
For best performance, we recommend you use Nginx as your Web server front-end, with a Passenger backend for the main API server and a Puma backend for API server Websockets. To do that:
- Install Nginx and Phusion Passenger.
Puma is already included with the API server's gems. We recommend you run it as a service under runit or a similar tool. Here's a sample runit script for that:
#!/bin/bash
set -e
exec 2>&1
# Uncomment the line below if you're using RVM.
#source /etc/profile.d/rvm.sh
envdir="`pwd`/env"
mkdir -p "$envdir"
echo ws-only > "$envdir/ARVADOS_WEBSOCKETS"
cd /var/www/arvados-api/current
echo "Starting puma in `pwd`"
# You may need to change arguments below to match your deployment, especially -u.
exec chpst -m 1073741824 -u www-data:www-data -e "$envdir" \
bundle exec puma -t 0:512 -e production -b tcp://127.0.0.1:8100
Edit the http section of your Nginx configuration to run the Passenger server, and act as a front-end for both it and Puma. You might add a block like the following, adding SSL and logging parameters to taste:
server {
listen 127.0.0.1:8000;
server_name localhost-api;
root /var/www/arvados-api/current/public;
index index.html index.htm index.php;
passenger_enabled on;
# If you're using RVM, uncomment the line below.
#passenger_ruby /usr/local/rvm/wrappers/default/ruby;
}
upstream api {
server 127.0.0.1:8000 fail_timeout=10s;
}
upstream websockets {
# The address below must match the one specified in puma's -b option.
server 127.0.0.1:8100 fail_timeout=10s;
}
proxy_http_version 1.1;
# When Keep clients request a list of Keep services from the API server, the
# server will automatically return the list of available proxies if
# the request headers include X-External-Client: 1. Following the example
# here, at the end of this section, add a line for each netmask that has
# direct access to Keep storage daemons to set this header value to 0.
geo $external_client {
default 1;
10.20.30.0/24 0;
}
server {
listen [your public IP address]:443 ssl;
server_name uuid_prefix.your.domain;
ssl on;
ssl_certificate /YOUR/PATH/TO/cert.pem;
ssl_certificate_key /YOUR/PATH/TO/cert.key;
index index.html index.htm index.php;
location / {
proxy_pass http://api;
proxy_redirect off;
proxy_connect_timeout 90s;
proxy_read_timeout 300s;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
proxy_set_header X-External-Client $external_client;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen [your public IP address]:443 ssl;
server_name ws.uuid_prefix.your.domain;
ssl on;
ssl_certificate /YOUR/PATH/TO/cert.pem;
ssl_certificate_key /YOUR/PATH/TO/cert.key;
index index.html index.htm index.php;
location / {
proxy_pass http://websockets;
proxy_redirect off;
proxy_connect_timeout 90s;
proxy_read_timeout 300s;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
- Restart Nginx.