3 navsection: installguide
4 title: Install the API server
7 h2. Install prerequisites
9 The Arvados package repository includes an API server package that can help automate much of the deployment.
11 h3(#install_ruby_and_bundler). Install Ruby and Bundler
13 {% include 'install_ruby_and_bundler' %}
15 h3(#install_postgres). Install Postgres
17 {% include 'install_postgres' %}
19 h3(#build_tools_apiserver). Build tools
21 * Build tools and the curl and PostgreSQL development libraries, to build gem dependencies
24 On older distributions, you may need to use a backports repository to satisfy these requirements. For example, on older Red Hat-based systems, consider using the "postgresql92":https://www.softwarecollections.org/en/scls/rhscl/postgresql92/ and "nginx16":https://www.softwarecollections.org/en/scls/rhscl/nginx16/ Software Collections.
26 On a Debian-based system, install the following packages:
29 <pre><code>~$ <span class="userinput">sudo apt-get install bison build-essential libpq-dev libcurl4-openssl-dev postgresql git nginx arvados-api-server</span>
33 On a Red Hat-based system, install the following packages:
36 <pre><code>~$ <span class="userinput">sudo yum install bison make automake gcc gcc-c++ libcurl-devel postgresql-server postgresql-devel nginx git arvados-api-server</span>
40 h2. Set up the database
42 Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
45 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**128).to_s(36)'</span>
46 6gqa1vu492idd7yca9tfandj3
47 </code></pre></notextile>
49 Create a new database user.
52 <pre><code>~$ <span class="userinput">sudo -u postgres createuser --encrypted -R -S --pwprompt arvados</span>
53 [sudo] password for <b>you</b>: <span class="userinput">yourpassword</span>
54 Enter password for new role: <span class="userinput">paste-password-you-generated</span>
55 Enter it again: <span class="userinput">paste-password-again</span>
56 </code></pre></notextile>
58 {% include 'notebox_begin' %}
60 This user setup assumes that your PostgreSQL is configured to accept password authentication. Red Hat systems use ident-based authentication by default. You may need to either adapt the user creation, or reconfigure PostgreSQL (in @pg_hba.conf@) to accept password authentication.
62 {% include 'notebox_end' %}
67 <pre><code>~$ <span class="userinput">sudo -u postgres createdb arvados_production -T template0 -E UTF8 -O arvados</span>
71 h2. Set up configuration files
73 The API server package uses configuration files that you write to @/etc/arvados/api@ and ensures they're consistently deployed. Create this directory and copy the example configuration files to it:
76 <pre><code>~$ <span class="userinput">sudo mkdir -p /etc/arvados/api</span>
77 ~$ <span class="userinput">sudo chmod 700 /etc/arvados/api</span>
78 ~$ <span class="userinput">cd /var/www/arvados-api/current</span>
79 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/database.yml.sample /etc/arvados/api/database.yml</span>
80 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/application.yml.example /etc/arvados/api/application.yml</span>
84 h2. Configure the database connection
86 Edit @/etc/arvados/api/database.yml@ and replace the @xxxxxxxx@ database password placeholders with the PostgreSQL password you generated above.
88 h2. Configure the API server
90 Edit @/etc/arvados/api/application.yml@ following the instructions below. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. Values in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided for installation convenience only.
92 Always put your local configuration in @application.yml@ instead of editing @application.default.yml@.
94 h3(#uuid_prefix). uuid_prefix
96 Define your @uuid_prefix@ in @application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 alphanumeric characters (lowercase ASCII letters and digits).
98 h3(#git_repositories_dir). git_repositories_dir
100 This field defaults to @/var/lib/arvados/git@. You can override the value by defining it in @application.yml@.
102 Make sure a clone of the arvados repository exists in @git_repositories_dir@.
105 <pre><code>~$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git</span>
106 ~$ <span class="userinput">sudo git clone --bare git://git.curoverse.com/arvados.git /var/lib/arvados/git/arvados.git</span>
107 </code></pre></notextile>
111 Generate a new secret token for signing cookies:
114 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**400).to_s(36)'</span>
115 zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
116 </code></pre></notextile>
118 Then put that value in the @secret_token@ field.
122 If you want access control on your "Keepstore":install-keepstore.html server(s), you should set @blob_signing_key@ to the same value as the permission key you provide to your Keepstore daemon(s).
124 h3. workbench_address
126 Fill in the url of your workbench application in @workbench_address@, for example
128 https://workbench.@uuid_prefix@.your.domain
130 h3(#omniauth). sso_app_id, sso_app_secret, sso_provider_url
132 For @sso_app_id@ and @sso_app_secret@, provide the same @app_id@ and @app_secret@ used in the "Create arvados-server client for Single Sign On (SSO)":install-sso.html#client step.
134 For @sso_provider_url@, provide the base URL where your SSO server is installed: just the scheme and host, with no trailing slash.
137 <pre><code> sso_app_id: arvados-server
138 sso_app_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
139 sso_provider_url: https://sso.example.com
145 Consult @/var/www/arvados-api/current/config/application.default.yml@ for a full list of configuration options. (But don't edit it. Edit @application.yml@ instead.)
147 h2. Prepare the API server deployment
149 Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
151 {% include 'notebox_begin' %}
152 You can safely ignore the following error message you may see when loading the database structure:
154 <pre><code>ERROR: must be owner of extension plpgsql</code></pre></notextile>
155 {% include 'notebox_end' %}
157 This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
159 h2. Set up Web servers
161 For best performance, we recommend you use Nginx as your Web server front-end, with a Passenger backend for the main API server and a Puma backend for API server Websockets. To do that:
165 <li>Install Nginx via your distribution or a backports repository.</li>
167 <li><a href="https://www.phusionpassenger.com/documentation/Users%20guide%20Nginx.html">Install Phusion Passenger for Nginx</a>.</li>
169 <li><p>Puma is already included with the API server's gems. We recommend you use a tool like <a href="http://smarden.org/runit/">runit</a> or something similar. Here's a sample run script for that:</p>
171 <pre><code>#!/bin/bash
176 # Uncomment the line below if you're using RVM.
177 #source /etc/profile.d/rvm.sh
181 echo ws-only > "$envdir/ARVADOS_WEBSOCKETS"
183 cd /var/www/arvados-api/current
184 echo "Starting puma in `pwd`"
186 # You may need to change arguments below to match your deployment, especially -u.
187 exec chpst -m 1073741824 -u www-data:www-data -e "$envdir" \
188 bundle exec puma -t 0:512 -e production -b tcp://127.0.0.1:8100
192 <li><p>Edit the http section of your Nginx configuration to run the Passenger server, and act as a front-end for both it and Puma. You might add a block like the following, adding SSL and logging parameters to taste:</p>
195 listen 127.0.0.1:8000;
196 server_name localhost-api;
198 root /var/www/arvados-api/current/public;
199 index index.html index.htm index.php;
201 passenger_enabled on;
202 # If you're using RVM, uncomment the line below.
203 #passenger_ruby /usr/local/rvm/wrappers/default/ruby;
207 server 127.0.0.1:8000 fail_timeout=10s;
210 upstream websockets {
211 # The address below must match the one specified in puma's -b option.
212 server 127.0.0.1:8100 fail_timeout=10s;
215 proxy_http_version 1.1;
218 listen <span class="userinput">[your public IP address]</span>:443 ssl;
219 server_name <span class="userinput">uuid_prefix.your.domain</span>;
223 index index.html index.htm index.php;
226 proxy_pass http://api;
229 proxy_set_header X-Forwarded-Proto https;
230 proxy_set_header Host $http_host;
231 proxy_set_header X-External-Client $external_client;
232 proxy_set_header X-Real-IP $remote_addr;
233 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
238 listen <span class="userinput">[your public IP address]</span>:443 ssl;
239 server_name ws.<span class="userinput">uuid_prefix.your.domain</span>;
243 index index.html index.htm index.php;
246 proxy_pass http://websockets;
249 proxy_set_header Upgrade $http_upgrade;
250 proxy_set_header Connection "upgrade";
251 proxy_set_header Host $host;
252 proxy_set_header X-Real-IP $remote_addr;
253 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
259 <li>Restart Nginx.</li>