X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/2054cdb05d79a3c45e8346661adc81062c383b16..fe3a451ad1d3613fd0953919fc44e5df18219fc4:/doc/install/install-api-server.html.textile.liquid
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index b65fe6975d..b8442eb060 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -1,183 +1,227 @@
---
layout: default
navsection: installguide
-title: Install the API server
+title: Install API server and Controller
...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
-This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
-h2. Install prerequisites
+# "Introduction":#introduction
+# "Install dependencies":#dependencies
+# "Set up database":#database-setup
+# "Update config.yml":#update-config
+# "Update nginx configuration":#update-nginx
+# "Install arvados-api-server and arvados-controller":#install-packages
+# "Confirm working installation":#confirm-working
-~$ sudo apt-get install \
- bison build-essential gettext libcurl3 libcurl3-gnutls \
- libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
- libssl-dev libxslt1.1 postgresql git wget zlib1g-dev
-
~$ cd $HOME # (or wherever you want to install)
-~$ git clone https://github.com/curoverse/arvados.git
-
~$ cd arvados/services/api
-~/arvados/services/api$ bundle install
-
~/arvados/services/api$ cp -i config/environments/production.rb.example config/environments/production.rb
-
~/arvados/services/api$ cp -i config/application.yml.example config/application.yml
-
SystemRootToken: "$system_root_token"
+ ManagementToken: "$management_token"
+ API:
+ RailsSessionSecretToken: "$rails_secret_token"
+ Collections:
+ BlobSigningKey: "blob_signing_key"
+
+
-h3(#uuid_prefix). uuid_prefix
+@SystemRootToken@ is used by Arvados system services to authenticate as the system (root) user when communicating with the API server.
-Define your @uuid_prefix@ in @config/application.yml@ by setting the @uuid_prefix@ field in the section for your environment. This prefix is used for all database identifiers to identify the record as originating from this site. It must be exactly 5 alphanumeric characters (lowercase ASCII letters and digits).
+@ManagementToken@ is used to authenticate access to system metrics.
-h3(#git_repositories_dir). git_repositories_dir
+@API.RailsSessionSecretToken@ is required by the API server.
-This field defaults to @/var/lib/arvados/git@. You can override the value by defining it in @config/application.yml@.
+@Collections.BlobSigningKey@ is used to control access to Keep blocks.
-Make sure a clone of the arvados repository exists in @git_repositories_dir@.
+You can generate a random token for each of these items at the command line like this:
~/arvados/services/api$ sudo mkdir -p /var/lib/arvados/git
-~/arvados/services/api$ sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git
-
~$ tr -dc 0-9a-zA-Z </dev/urandom | head -c50; echo
+
+
-Generate a new secret token for signing cookies:
+h3. PostgreSQL.Connection
~/arvados/services/api$ ruby -e 'puts rand(2**400).to_s(36)'
-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-
PostgreSQL:
+ Connection:
+ host: localhost
+ user: arvados
+ password: $postgres_password
+ dbname: arvados_production
+
+
-h2. Set up the database
+Replace the @$postgres_password@ placeholder with the password you generated during "database setup":#database-setup .
-Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
+h3. Services
~/arvados/services/api$ ruby -e 'puts rand(2**128).to_s(36)'
-6gqa1vu492idd7yca9tfandj3
-
Services:
+ Controller:
+ ExternalURL: "https://ClusterID.example.com"
+ InternalURLs:
+ "http://localhost:8003": {}
+ RailsAPI:
+ # Does not have an ExternalURL
+ InternalURLs:
+ "http://localhost:8004": {}
+
+
-Create a new database user with permission to create its own databases.
+Replace @ClusterID.example.com@ with the hostname that you previously selected for the API server.
+
+The @Services@ section of the configuration helps Arvados components contact one another (service discovery). Each service has one or more @InternalURLs@ and an @ExternalURL@. The @InternalURLs@ describe where the service runs, and how the Nginx reverse proxy will connect to it. The @ExternalURL@ is how external clients contact the service.
+
+h2(#update-nginx). Update nginx configuration
+
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-api-and-controller.conf@ with the following configuration. Options that need attention are marked in red.
+
+proxy_http_version 1.1;
+
+# When Keep clients request a list of Keep services from the API
+# server, use the origin IP address to determine if the request came
+# from the internal subnet or it is an external client. This sets the
+# $external_client variable which in turn is used to set the
+# X-External-Client header.
+#
+# The API server uses this header to choose whether to respond to a
+# "available keep services" request with either a list of internal keep
+# servers (0) or with the keepproxy (1).
+#
+# Following the example here, update the 10.20.30.0/24 netmask
+# to match your private subnet.
+# Update 1.2.3.4 and add lines as necessary with the public IP
+# address of all servers that can also access the private network to
+# ensure they are not considered 'external'.
+
+geo $external_client {
+ default 1;
+ 127.0.0.0/24 0;
+ 10.20.30.0/24 0;
+ 1.2.3.4/32 0;
+}
+
+# This is the port where nginx expects to contact arvados-controller.
+upstream controller {
+ server localhost:8003 fail_timeout=10s;
+}
+
+server {
+ # This configures the public https port that clients will actually connect to,
+ # the request is reverse proxied to the upstream 'controller'
+
+ listen 443 ssl;
+ server_name ClusterID.example.com;
+
+ ssl_certificate /YOUR/PATH/TO/cert.pem;
+ ssl_certificate_key /YOUR/PATH/TO/cert.key;
+
+ # Refer to the comment about this setting in the passenger (arvados
+ # api server) section of your Nginx configuration.
+ client_max_body_size 128m;
+
+ location / {
+ proxy_pass http://controller;
+ proxy_redirect off;
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
+
+ proxy_set_header X-Forwarded-Proto https;
+ proxy_set_header Host $http_host;
+ proxy_set_header X-External-Client $external_client;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ }
+}
+
+server {
+ # This configures the Arvados API server. It is written using Ruby
+ # on Rails and uses the Passenger application server.
+
+ listen localhost:8004;
+ server_name localhost-api;
+
+ root /var/www/arvados-api/current/public;
+ index index.html index.htm index.php;
+
+ passenger_enabled on;
+
+ # If you are using RVM, uncomment the line below.
+ # If you're using system ruby, leave it commented out.
+ #passenger_ruby /usr/local/rvm/wrappers/default/ruby;
+
+ # This value effectively limits the size of API objects users can
+ # create, especially collections. If you change this, you should
+ # also ensure the following settings match it:
+ # * `client_max_body_size` in the previous server section
+ # * `API.MaxRequestSize` in config.yml
+ client_max_body_size 128m;
+}
+
+~/arvados/services/api$ sudo -u postgres createuser --createdb --encrypted -R -S --pwprompt arvados
-[sudo] password for you: yourpassword
-Enter password for new role: paste-password-you-generated
-Enter it again: paste-password-again
-
~/arvados/services/api$ cp -i config/database.yml.sample config/database.yml
-~/arvados/services/api$ edit config/database.yml
-
~/arvados/services/api$ RAILS_ENV=production bundle exec rake db:setup
-
~/arvados/services/api$ su postgres createdb arvados_production -E UTF8 -O arvados
-~/arvados/services/api$ RAILS_ENV=production bundle exec rake db:structure:load
-~/arvados/services/api$ RAILS_ENV=production bundle exec rake db:seed
+$ curl https://ClusterID.example.com/arvados/v1/config
-{% include 'notebox_begin' %}
-You can safely ignore the following error message you may see when loading the database structure:
-
-ERROR: must be owner of extension plpgsql
-{% include 'notebox_end' %}
-
-h2(#omniauth). Set up omniauth
+Confirm working Rails API server:
-First copy the omniauth configuration file:
-
-
-~/arvados/services/api$ cp -i config/initializers/omniauth.rb.example config/initializers/omniauth.rb
+$ curl https://ClusterID.example.com/discovery/v1/apis/arvados/v1/rest
-Edit @config/initializers/omniauth.rb@ to configure the SSO server for authentication. @APP_ID@ and @APP_SECRET@ correspond to the @app_id@ and @app_secret@ set in "Create arvados-server client for Single Sign On (SSO)":install-sso.html#client and @CUSTOM_PROVIDER_URL@ is the address of your SSO server.
-
-
-APP_ID = 'arvados-server'
-APP_SECRET = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
-CUSTOM_PROVIDER_URL = 'https://sso.example.com/'
-
-
-
-h2. Start the API server
+Confirm that you can use the system root token to act as the system root user:
-h3. Development environment
-
-If you plan to run in development mode, you can now run the development server this way:
-
-
-~/arvados/services/api$ bundle exec rails server --port=3030
+
+$ curl -H "Authorization: Bearer $system_root_token" https://ClusterID.example.com/arvados/v1/users/current
-h3. Production environment
-
-We recommend "Passenger":https://www.phusionpassenger.com/ to run the API server in production.
+h3. Troubleshooting
-Point it to the services/api directory in the source tree.
+If you are getting TLS errors, make sure the @ssl_certificate@ directive in your nginx configuration has the "full certificate chain":http://nginx.org/en/docs/http/configuring_https_servers.html#chains
-To enable streaming so users can monitor crunch jobs in real time, make sure to add the following to your Passenger configuration:
+Logs can be found in @/var/www/arvados-api/current/log/production.log@ and using @journalctl -u arvados-controller@.
-
-PassengerBufferResponse off
-
-
+See also the admin page on "Logging":{{site.baseurl}}/admin/logging.html .