title: Install Keepproxy server
...
-The Keepproxy server is a gateway into your Keep storage. Unlike the Keepstore servers, which are only accessible on the local LAN, Keepproxy is designed to provide secure access into Keep from anywhere on the internet.
+The Keepproxy server is a gateway into your Keep storage. Unlike the Keepstore servers, which are only accessible on the local LAN, Keepproxy is suitable for clients located elsewhere on the internet. Specifically, in contrast to Keepstore:
+* A client writing through Keepproxy generates less network traffic: the client sends a single copy of a data block, and Keepproxy sends copies to the appropriate Keepstore servers.
+* A client can write through Keepproxy without precomputing content hashes. Notably, the browser-based upload feature in Workbench requires Keepproxy.
+* Keepproxy checks API token validity before processing requests. (Clients that can connect directly to Keepstore can use it as scratch space even without a valid API token.)
-By convention, we use the following hostname for the Keepproxy:
+By convention, we use the following hostname for the Keepproxy server:
<div class="offset1">
table(table table-bordered table-condensed).
<notextile>
<pre><code>~$ <span class="userinput">keepproxy -h</span>
-Usage of default:
+Usage of keepproxy:
-default-replicas=2: Default number of replicas to write if not specified by the client.
-listen=":25107": Interface on which to listen for requests, in the format ipaddr:port. e.g. -listen=10.0.1.24:8000. Use -listen=:port to listen on all network interfaces.
-no-get=false: If set, disable GET operations
-no-put=false: If set, disable PUT operations
-pid="": Path to write pid file
+ -timeout=15: Timeout on requests to internal Keep services (default 15 seconds)
</code></pre>
</notextile>
-It's recommended to run Keepproxy under "runit":http://smarden.org/runit/ or something similar.
-
h3. Create an API token for the Keepproxy server
The Keepproxy server needs a token to talk to the API server.
On the <strong>API server</strong>, use the following command to create the token:
<notextile>
-<pre><code>~/arvados/services/api/script$ <span class="userinput">RAILS_ENV=production ./get_anonymous_user_token.rb</span>
+<pre><code>/var/www/arvados-api/current/script$ <span class="userinput">RAILS_ENV=production bundle exec ./get_anonymous_user_token.rb</span>
hoShoomoo2bai3Ju1xahg6aeng1siquuaZ1yae2gi2Uhaeng2r
</code></pre></notextile>
-The value for the @api_token@ field should be added to Keepproxy's environment as ARVADOS_API_TOKEN. Make sure to also set ARVADOS_API_HOST to @uuid_prefix@.your.domain.
+h3. Set up the Keepproxy service
+
+We recommend you run Keepproxy under "runit":http://smarden.org/runit/ or a similar supervisor. Make sure the launcher sets the envirnoment variables @ARVADOS_API_TOKEN@ (with the token you just generated), @ARVADOS_API_HOST@, and, if needed, @ARVADOS_API_HOST_INSECURE@. The core keepproxy command to run is:
+
+<notextile>
+<pre><code>ARVADOS_API_TOKEN=<span class="userinput">[generated token]</span> ARVADOS_API_HOST=<span class="userinput">uuid_prefix.your.domain</span> exec keepproxy
+</code></pre>
+</notextile>
h3. Set up a reverse proxy with SSL support
This is best achieved by putting a reverse proxy with SSL support in front of Keepproxy. Keepproxy itself runs on port 25107 by default; your reverse proxy can run on port 443 and pass requests to Keepproxy on port 25107.
-If possible, the proxy should be configured to add CORS headers to its own error responses -- otherwise in-browser applications can't report proxy errors. For example, in nginx >= 1.7.5:
-
<notextile><pre>
+upstream keepproxy {
+ server 127.0.0.1:<span class="userinput">25107</span>;
+}
+
server {
- server_name keep.example.com
- ...
- add_header 'Access-Control-Allow-Methods' 'GET, HEAD, POST, PUT, OPTIONS' always
- add_header 'Access-Control-Allow-Origin' '*' always
- add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Length, Content-Type, X-Keep-Desired-Replicas' always
- add_header 'Access-Control-Max-Age' '86486400' always
+ listen <span class="userinput">[your public IP address]</span>:443 ssl;
+ server_name keep.<span class="userinput">uuid_prefix</span>.your.domain
+
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
+ proxy_set_header X-Real-IP $remote_addr;
+
+ ssl on;
+ ssl_certificate /etc/nginx/keep.<span class="userinput">uuid_prefix</span>.your.domain-ssl.crt;
+ ssl_certificate_key /etc/nginx/keep.<span class="userinput">uuid_prefix</span>.your.domain-ssl.key;
+
+ # Clients need to be able to upload blocks of data up to 64MiB in size.
+ client_max_body_size 64m;
+
+ location / {
+ proxy_pass http://keepproxy;
+ }
}
</pre></notextile>
-*Warning:* Make sure you don't inadvertently add CORS headers for services _other than keepproxy_ while you're doing this.
+Note: if the Web uploader is failing to upload data and there are no logs from keepproxy, be sure to check the nginx proxy logs. In addition to "GET" and "PUT", The nginx proxy must pass "OPTIONS" requests to keepproxy, which should respond with appropriate Cross-origin resource sharing headers. If the CORS headers are not present, brower security policy will cause the upload request to silently fail. The CORS headers are generated by keepproxy and should not be set in nginx.
h3. Tell the API server about the Keepproxy server
The API server needs to be informed about the presence of your Keepproxy server. Please execute the following commands on your <strong>shell server</strong>.
<notextile>
-<pre><code>~$ <span class="userinput">prefix=`arv --format=uuid user current | cut -d- -f1`</span>
-~$ <span class="userinput">echo "Site prefix is '$prefix'"</span>
+<pre><code>~$ <span class="userinput">uuid_prefix=`arv --format=uuid user current | cut -d- -f1`</span>
+~$ <span class="userinput">echo "Site prefix is '$uuid_prefix'"</span>
~$ <span class="userinput">read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"</span>
<span class="userinput">{
- "service_host":"keep.$prefix.your.domain",
+ "service_host":"<strong>keep.$uuid_prefix.your.domain</strong>",
"service_port":443,
"service_ssl_flag":true,
"service_type":"proxy"
}
EOF</span>
</code></pre></notextile>
-
projects / arvados.git / blobdiff