3 navsection: installguide
4 title: Install the API server
7 h2. Install prerequisites
9 The Arvados package repository includes an API server package that can help automate much of the deployment.
11 h3(#install_ruby_and_bundler). Install Ruby and Bundler
13 {% include 'install_ruby_and_bundler' %}
15 h3(#install_postgres). Install PostgreSQL
17 {% include 'install_postgres' %}
19 h3(#build_tools_apiserver). Build tools
21 On older distributions, you may need to use a backports repository to satisfy these requirements. For example, on older Red Hat-based systems, consider using the "postgresql92":https://www.softwarecollections.org/en/scls/rhscl/postgresql92/ and "nginx16":https://www.softwarecollections.org/en/scls/rhscl/nginx16/ Software Collections.
23 On a Debian-based system, install the following packages:
26 <pre><code>~$ <span class="userinput">sudo apt-get install bison build-essential libcurl4-openssl-dev git nginx arvados-api-server</span>
30 On a Red Hat-based system, install the following packages:
33 <pre><code>~$ <span class="userinput">sudo yum install bison make automake gcc gcc-c++ libcurl-devel nginx git arvados-api-server</span>
37 h2. Set up the database
39 Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
42 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**128).to_s(36)'</span>
43 6gqa1vu492idd7yca9tfandj3
44 </code></pre></notextile>
46 Create a new database user.
49 <pre><code>~$ <span class="userinput">sudo -u postgres createuser --encrypted -R -S --pwprompt arvados</span>
50 [sudo] password for <b>you</b>: <span class="userinput">yourpassword</span>
51 Enter password for new role: <span class="userinput">paste-password-you-generated</span>
52 Enter it again: <span class="userinput">paste-password-again</span>
53 </code></pre></notextile>
55 {% include 'notebox_begin' %}
57 This user setup assumes that your PostgreSQL is configured to accept password authentication. Red Hat systems use ident-based authentication by default. You may need to either adapt the user creation, or reconfigure PostgreSQL (in @pg_hba.conf@) to accept password authentication.
59 {% include 'notebox_end' %}
64 <pre><code>~$ <span class="userinput">sudo -u postgres createdb arvados_production -T template0 -E UTF8 -O arvados</span>
68 h2. Set up configuration files
70 The API server package uses configuration files that you write to @/etc/arvados/api@ and ensures they're consistently deployed. Create this directory and copy the example configuration files to it:
73 <pre><code>~$ <span class="userinput">sudo mkdir -p /etc/arvados/api</span>
74 ~$ <span class="userinput">sudo chmod 700 /etc/arvados/api</span>
75 ~$ <span class="userinput">cd /var/www/arvados-api/current</span>
76 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/database.yml.sample /etc/arvados/api/database.yml</span>
77 /var/www/arvados-api/current$ <span class="userinput">sudo cp config/application.yml.example /etc/arvados/api/application.yml</span>
81 h2. Configure the database connection
83 Edit @/etc/arvados/api/database.yml@ and replace the @xxxxxxxx@ database password placeholders with the PostgreSQL password you generated above.
85 h2. Configure the API server
87 Edit @/etc/arvados/api/application.yml@ following the instructions below. The deployment script will consistently deploy this to the API server's configuration directory. The API server reads both @application.yml@ and its own @config/application.default.yml@ file. Values in @application.yml@ take precedence over the defaults that are defined in @config/application.default.yml@. The @config/application.yml.example@ file is not read by the API server and is provided for installation convenience only.
89 Always put your local configuration in @application.yml@ instead of editing @application.default.yml@.
91 h3(#uuid_prefix). uuid_prefix
93 Define your @uuid_prefix@ in @application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 alphanumeric characters (lowercase ASCII letters and digits).
95 h3(#git_repositories_dir). git_repositories_dir
97 This field defaults to @/var/lib/arvados/git@. You can override the value by defining it in @application.yml@.
99 Make sure a clone of the arvados repository exists in @git_repositories_dir@.
102 <pre><code>~$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git</span>
103 ~$ <span class="userinput">sudo git clone --bare git://git.curoverse.com/arvados.git /var/lib/arvados/git/arvados.git</span>
104 </code></pre></notextile>
108 Generate a new secret token for signing cookies:
111 <pre><code>~$ <span class="userinput">ruby -e 'puts rand(2**400).to_s(36)'</span>
112 zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
113 </code></pre></notextile>
115 Then put that value in the @secret_token@ field.
119 If you want access control on your "Keepstore":install-keepstore.html server(s), you should set @blob_signing_key@ to the same value as the permission key you provide to your Keepstore daemon(s).
121 h3. workbench_address
123 Fill in the url of your workbench application in @workbench_address@, for example
125 https://workbench.@uuid_prefix@.your.domain
127 h3(#omniauth). sso_app_id, sso_app_secret, sso_provider_url
129 For @sso_app_id@ and @sso_app_secret@, provide the same @app_id@ and @app_secret@ used in the "Create arvados-server client for Single Sign On (SSO)":install-sso.html#client step.
131 For @sso_provider_url@, provide the base URL where your SSO server is installed: just the scheme and host, with no trailing slash.
134 <pre><code> sso_app_id: arvados-server
135 sso_app_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
136 sso_provider_url: https://sso.example.com
142 Consult @/var/www/arvados-api/current/config/application.default.yml@ for a full list of configuration options. (But don't edit it. Edit @application.yml@ instead.)
144 h2. Prepare the API server deployment
146 Now that all your configuration is in place, run @/usr/local/bin/arvados-api-server-upgrade.sh@. This will install and check your configuration, install necessary gems, and run any necessary database setup.
148 {% include 'notebox_begin' %}
149 You can safely ignore the following error message you may see when loading the database structure:
151 <pre><code>ERROR: must be owner of extension plpgsql</code></pre></notextile>
152 {% include 'notebox_end' %}
154 This command aborts when it encounters an error. It's safe to rerun multiple times, so if there's a problem with your configuration, you can fix that and try again.
156 h2. Set up Web servers
158 For best performance, we recommend you use Nginx as your Web server front-end, with a Passenger backend for the main API server and a Puma backend for API server Websockets. To do that:
162 <li>Install Nginx via your distribution or a backports repository.</li>
164 <li><a href="https://www.phusionpassenger.com/documentation/Users%20guide%20Nginx.html">Install Phusion Passenger for Nginx</a>.</li>
166 <li><p>Puma is already included with the API server's gems. We recommend you use a tool like <a href="http://smarden.org/runit/">runit</a> or something similar. Here's a sample run script for that:</p>
168 <pre><code>#!/bin/bash
173 # Uncomment the line below if you're using RVM.
174 #source /etc/profile.d/rvm.sh
178 echo ws-only > "$envdir/ARVADOS_WEBSOCKETS"
180 cd /var/www/arvados-api/current
181 echo "Starting puma in `pwd`"
183 # You may need to change arguments below to match your deployment, especially -u.
184 exec chpst -m 1073741824 -u www-data:www-data -e "$envdir" \
185 bundle exec puma -t 0:512 -e production -b tcp://127.0.0.1:8100
189 <li><p>Edit the http section of your Nginx configuration to run the Passenger server, and act as a front-end for both it and Puma. You might add a block like the following, adding SSL and logging parameters to taste:</p>
192 listen 127.0.0.1:8000;
193 server_name localhost-api;
195 root /var/www/arvados-api/current/public;
196 index index.html index.htm index.php;
198 passenger_enabled on;
199 # If you're using RVM, uncomment the line below.
200 #passenger_ruby /usr/local/rvm/wrappers/default/ruby;
204 server 127.0.0.1:8000 fail_timeout=10s;
207 upstream websockets {
208 # The address below must match the one specified in puma's -b option.
209 server 127.0.0.1:8100 fail_timeout=10s;
212 proxy_http_version 1.1;
215 listen <span class="userinput">[your public IP address]</span>:443 ssl;
216 server_name <span class="userinput">uuid_prefix.your.domain</span>;
220 index index.html index.htm index.php;
223 proxy_pass http://api;
226 proxy_set_header X-Forwarded-Proto https;
227 proxy_set_header Host $http_host;
228 proxy_set_header X-External-Client $external_client;
229 proxy_set_header X-Real-IP $remote_addr;
230 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
235 listen <span class="userinput">[your public IP address]</span>:443 ssl;
236 server_name ws.<span class="userinput">uuid_prefix.your.domain</span>;
240 index index.html index.htm index.php;
243 proxy_pass http://websockets;
246 proxy_set_header Upgrade $http_upgrade;
247 proxy_set_header Connection "upgrade";
248 proxy_set_header Host $host;
249 proxy_set_header X-Real-IP $remote_addr;
250 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
256 <li>Restart Nginx.</li>