X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/443f3228eb4c56849f77ae9c421dd1cc6fdbc5f1..5fcca42249b8b35f50beb9ed4c51d090d76c1767:/doc/install/install-keepproxy.html.textile.liquid?ds=sidebyside diff --git a/doc/install/install-keepproxy.html.textile.liquid b/doc/install/install-keepproxy.html.textile.liquid index a6bb5d4bd9..0839c0e521 100644 --- a/doc/install/install-keepproxy.html.textile.liquid +++ b/doc/install/install-keepproxy.html.textile.liquid @@ -3,9 +3,24 @@ layout: default navsection: installguide title: Install Keepproxy server ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. + +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} + +# "Introduction":#introduction +# "Update config.yml":#update-config +# "Update nginx configuration":#update-nginx +# "Install keepproxy package":#install-packages +# "Start the service":#start-service +# "Restart the API server and controller":#restart-api +# "Confirm working installation":#confirm-working + +h2(#introduction). Introduction The Keepproxy server is a gateway into your Keep storage. Unlike the Keepstore servers, which are only accessible on the local LAN, Keepproxy is suitable for clients located elsewhere on the internet. Specifically, in contrast to Keepstore: -* A client writing through Keepproxy generates less network traffic: the client sends a single copy of a data block, and Keepproxy sends copies to the appropriate Keepstore servers. +* A client writing through Keepproxy sends a single copy of a data block, and Keepproxy distributes copies to the appropriate Keepstore servers. * A client can write through Keepproxy without precomputing content hashes. Notably, the browser-based upload feature in Workbench requires Keepproxy. * Keepproxy checks API token validity before processing requests. (Clients that can connect directly to Keepstore can use it as scratch space even without a valid API token.) @@ -13,106 +28,93 @@ By convention, we use the following hostname for the Keepproxy server:
~$ sudo apt-get install keepproxy
-
+ Services:
+ Keepproxy:
+ ExternalURL: https://keep.ClusterID.example.com
+ InternalURLs:
+ "http://localhost:25107": {}
+
~$ sudo yum install keepproxy
-
-~$ keepproxy -h
-Usage of keepproxy:
- -default-replicas=2: Default number of replicas to write if not specified by the client.
- -listen=":25107": Interface on which to listen for requests, in the format ipaddr:port. e.g. -listen=10.0.1.24:8000. Use -listen=:port to listen on all network interfaces.
- -no-get=false: If set, disable GET operations
- -no-put=false: If set, disable PUT operations
- -pid="": Path to write pid file
- -timeout=15: Timeout on requests to internal Keep services (default 15 seconds)
-
-upstream keepproxy {
+ server 127.0.0.1:25107;
+}
-{% assign railscmd = "bundle exec ./script/get_anonymous_user_token.rb" %}
-{% assign railsout = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" %}
-The Keepproxy server needs a token to talk to the API server. On the API server, use the following command to create the token. {% include 'install_rails_command' %}
+server {
+ listen *:443 ssl;
+ server_name keep.ClusterID.example.com;
-h3. Set up the Keepproxy service
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_http_version 1.1;
+ proxy_request_buffering off;
-Install runit to supervise the keepproxy daemon. {% include 'install_runit' %}
+ ssl on;
+ ssl_certificate /YOUR/PATH/TO/cert.pem;
+ ssl_certificate_key /YOUR/PATH/TO/cert.key;
-The run script for the keepproxy service should set the environment variables @ARVADOS_API_TOKEN@ (with the token you just generated), @ARVADOS_API_HOST@, and, if needed, @ARVADOS_API_HOST_INSECURE@. The core keepproxy command to run is:
+ # Clients need to be able to upload blocks of data up to 64MiB in size.
+ client_max_body_size 64m;
-
-ARVADOS_API_TOKEN={{railsout}} ARVADOS_API_HOST=uuid_prefix.your.domain exec keepproxy
-
-
+ location / {
+ proxy_pass http://keepproxy;
+ }
+}
+
-upstream keepproxy { - server 127.0.0.1:25107; -} +{% include 'start_service' %} -server { - listen [your public IP address]:443 ssl; - server_name keep.uuid_prefix.your.domain; +{% include 'restart_api' %} - proxy_connect_timeout 90s; - proxy_read_timeout 300s; - proxy_set_header X-Real-IP $remote_addr; +h2(#confirm-working). Confirm working installation - ssl on; - ssl_certificate /etc/nginx/keep.uuid_prefix.your.domain-ssl.crt; - ssl_certificate_key /etc/nginx/keep.uuid_prefix.your.domain-ssl.key; +Log into a host that is on a network external to your private Arvados network. The host should be able to contact your keepproxy server (eg @keep.ClusterID.example.com@), but not your keepstore servers (eg keep[0-9].ClusterID.example.com). - # Clients need to be able to upload blocks of data up to 64MiB in size. - client_max_body_size 64m; +@ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ must be set in the environment. - location / { - proxy_pass http://keepproxy; - } -} -
~$ uuid_prefix=`arv --format=uuid user current | cut -d- -f1`
-~$ echo "Site prefix is '$uuid_prefix'"
-~$ read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"
-{
- "service_host":"keep.$uuid_prefix.your.domain",
- "service_port":443,
- "service_ssl_flag":true,
- "service_type":"proxy"
-}
-EOF
-
+$ arv keep_service accessible
+[...]
+
+
+
+If keepstore does not show up in the "accessible" list, and you are accessing it from within the private network, check that you have "properly configured the @geo@ block for the API server":install-api-server.html#update-nginx .
+
+Install the "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html
+
+You should now be able to use @arv-put@ to upload collections and @arv-get@ to fetch collections. Be sure to execute this from _outside_ the cluster's private network.
+
+{% include 'arv_put_example' %}