3 navsection: installguide
4 title: Install the API server
7 h2. Install prerequisites
9 The Arvados package repository includes an API server package that can help automate much of the deployment. It requires:
12 * "Ruby 2.1 and bundler":install-manual-prerequisites-ruby.html
13 * Build tools and the curl and PostgreSQL development libraries, to build gem dependencies
16 On older distributions, you may need to use a backports repository to satisfy these requirements. For example, on older Red Hat-based systems, consider using the "postgresql92":https://www.softwarecollections.org/en/scls/rhscl/postgresql92/ and "nginx16":https://www.softwarecollections.org/en/scls/rhscl/nginx16/ Software Collections.
18 On a Debian-based system, install the following packages:
21 <pre><code>~$ <span class="userinput">sudo apt-get install bison build-essential libpq-dev libcurl4-openssl-dev postgresql git nginx arvados-api-server</span>
25 On a Red Hat-based system, install the following packages:
28 <pre><code>~$ <span class="userinput">sudo yum install bison make automake gcc gcc-c++ libcurl-devel postgresql-server postgresql-devel nginx git arvados-api-server</span>
32 {% include 'notebox_begin' %}
34 If you intend to use specific versions of these packages from Software Collections, you may have to adapt some of the package names to match; e.g., @postgresql92-postgresql-server postgresql92-postgresql-devel nginx16@.
36 {% include 'notebox_end' %}
38 h2. Set up the database
40 Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
43 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**128).to_s(36)'</span>
44 6gqa1vu492idd7yca9tfandj3
45 </code></pre></notextile>
47 Create a new database user.
50 <pre><code>~$ <span class="userinput">sudo -u postgres createuser --encrypted -R -S --pwprompt arvados</span>
51 [sudo] password for <b>you</b>: <span class="userinput">yourpassword</span>
52 Enter password for new role: <span class="userinput">paste-password-you-generated</span>
53 Enter it again: <span class="userinput">paste-password-again</span>
54 </code></pre></notextile>
56 {% include 'notebox_begin' %}
58 This user setup assumes that your PostgreSQL is configured to accept password authentication. Red Hat systems use ident-based authentication by default. You may need to either adapt the user creation, or reconfigure PostgreSQL (in @pg_hba.conf@) to accept password authentication.
60 {% include 'notebox_end' %}
65 <pre><code>~$ <span class="userinput">sudo -u postgres createdb arvados_production -T template0 -E UTF8 -O arvados</span>
69 h2. Set up configuration files
71 The API server package uses configuration files that you write to @/etc/arvados/api@ and ensures they're consistently deployed. Create this directory and copy the example configuration files to it:
74 <pre><code>~$ <span class="userinput">sudo mkdir -p /etc/arvados/api</span>
75 ~$ <span class="userinput">sudo chmod 700 /etc/arvados/api</span>
76 ~$ <span class="userinput">cd /var/www/arvados-api/current</span>
77 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/initializers/omniauth.rb.example /etc/arvados/api/omniauth.rb</span>
78 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/database.yml.sample /etc/arvados/api/database.yml</span>
79 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/application.yml.example /etc/arvados/api/application.yml</span>
83 h2. Configure the database connection
85 Edit @/etc/arvados/api/database.yml@ and replace the @xxxxxxxx@ database password placeholders with the PostgreSQL password you generated above.
87 h2. Configure the API server
89 Edit @/etc/arvados/api/application.yml@ following the instructions below. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. Values in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided for installation convenience only.
91 Always put your local configuration in @application.yml@ instead of editing @application.default.yml@.
93 h3(#uuid_prefix). uuid_prefix
95 Define your @uuid_prefix@ in @application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 alphanumeric characters (lowercase ASCII letters and digits).
97 h3(#git_repositories_dir). git_repositories_dir
99 This field defaults to @/var/lib/arvados/git@. You can override the value by defining it in @application.yml@.
101 Make sure a clone of the arvados repository exists in @git_repositories_dir@.
104 <pre><code>~$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git</span>
105 ~$ <span class="userinput">sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git</span>
106 </code></pre></notextile>
110 Generate a new secret token for signing cookies:
113 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**400).to_s(36)'</span>
114 zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
115 </code></pre></notextile>
117 Then put that value in the @secret_token@ field.
121 If you want access control on your "Keepstore":install-keepstore.html server(s), you should set @blob_signing_key@ to the same value as the permission key you provide to your Keepstore daemon(s).
123 h3. workbench_address
125 Fill in the url of your workbench application in @workbench_address@, for example
127 https://workbench.@prefix_uuid@.your.domain
129 h3(#omniauth). sso_app_id, sso_app_secret, sso_provider_url
131 For @sso_app_id@ and @sso_app_secret@, provide the same @app_id@ and @app_secret@ used in the "Create arvados-server client for Single Sign On (SSO)":install-sso.html#client step.
133 For @sso_provider_url@, provide the base URL where your SSO server is installed: just the scheme and host, with no trailing slash.
136 <pre><code> sso_app_id: arvados-server
137 sso_app_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
138 sso_provider_url: https://sso.example.com
144 Consult @/var/www/arvados-api/current/config/application.default.yml@ for a full list of configuration options. (But don't edit it. Edit @application.yml@ instead.)
146 h2. Prepare the API server deployment
148 Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
150 {% include 'notebox_begin' %}
151 You can safely ignore the following error message you may see when loading the database structure:
153 <pre><code>ERROR: must be owner of extension plpgsql</code></pre></notextile>
154 {% include 'notebox_end' %}
156 This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
158 h2. Set up Web servers
160 For best performance, we recommend you use Nginx as your Web server front-end, with a Passenger backend for the main API server and a Puma backend for API server Websockets. To do that:
164 <li>Install Nginx via your distribution or a backports repository.</li>
166 <li><a href="https://www.phusionpassenger.com/documentation/Users%20guide%20Nginx.html">Install Phusion Passenger for Nginx</a>.</li>
168 <li><p>Puma is already included with the API server's gems. We recommend you use a tool like <a href="http://smarden.org/runit/">runit</a> or something similar. Here's a sample run script for that:</p>
170 <pre><code>#!/bin/bash
173 # Uncomment the line below if you're using RVM.
174 #source /etc/profile.d/rvm.sh
176 envdir="/etc/sv/puma/env"
178 echo "Starting puma from ${root}"
182 cd /var/www/arvados-api/current
183 # You may need to change arguments below to match your deployment, especially -u.
184 exec chpst -e "${envdir}" -m 1073741824 -u www-data:www-data bundle exec puma -t 0:512 -e production -b tcp://127.0.0.1:8100
188 <li><p>Edit the http section of your Nginx configuration to run the Passenger server, and act as a front-end for both it and Puma. You might add a block like the following, adding SSL and logging parameters to taste:</p>
191 listen 127.0.0.1:8000;
192 server_name localhost-api;
194 root /var/www/arvados-api/current/public;
195 index index.html index.htm index.php;
197 passenger_enabled on;
198 # If you're using RVM, uncomment the line below.
199 #passenger_ruby /usr/local/rvm/wrappers/default/ruby;
203 server 127.0.0.1:8000 fail_timeout=10s;
206 upstream websockets {
207 # The address below must match the one specified in puma's -b option.
208 server 127.0.0.1:8100 fail_timeout=10s;
211 proxy_http_version 1.1;
214 listen <span class="userinput">[your public IP address]</span>:443 ssl;
215 server_name <span class="userinput">uuid-prefix.your.domain</span>;
219 index index.html index.htm index.php;
222 proxy_pass http://api;
225 proxy_set_header X-Forwarded-Proto https;
226 proxy_set_header Host $http_host;
227 proxy_set_header X-External-Client $external_client;
228 proxy_set_header X-Real-IP $remote_addr;
229 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
234 listen <span class="userinput">[your public IP address]</span>:443 ssl;
235 server_name ws.<span class="userinput">uuid-prefix.your.domain</span>;
239 index index.html index.htm index.php;
242 proxy_pass http://websockets;
245 proxy_set_header Upgrade $http_upgrade;
246 proxy_set_header Connection "upgrade";
247 proxy_set_header Host $host;
248 proxy_set_header X-Real-IP $remote_addr;
249 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
255 <li>Restart Nginx.</li>