1 // Keep-web provides read-only HTTP access to files stored in Keep. It
2 // serves public data to anonymous and unauthenticated clients, and
3 // serves private data to clients that supply Arvados API tokens. It
4 // can be installed anywhere with access to Keep services, typically
5 // behind a web proxy that supports TLS.
7 // See http://doc.arvados.org/install/install-keep-web.html.
11 // Serve HTTP requests at port 1234 on all interfaces:
13 // keep-web -address=:1234
15 // Serve HTTP requests at port 1234 on the interface with IP address 1.2.3.4:
17 // keep-web -address=1.2.3.4:1234
19 // Proxy configuration
21 // Keep-web does not support SSL natively. Typically, it is installed
22 // behind a proxy like nginx.
24 // Here is an example nginx configuration.
27 // upstream keep-web {
28 // server localhost:1234;
32 // server_name dl.example.com *.dl.example.com ~.*--dl.example.com;
33 // ssl_certificate /root/wildcard.example.com.crt;
34 // ssl_certificate_key /root/wildcard.example.com.key;
36 // proxy_pass http://keep-web;
37 // proxy_set_header Host $host;
38 // proxy_set_header X-Forwarded-For $remote_addr;
43 // It is not necessary to run keep-web on the same host as the nginx
44 // proxy. However, TLS is not used between nginx and keep-web, so
45 // intervening networks must be secured by other means.
49 // The following "same origin" URL patterns are supported for public
50 // collections (i.e., collections which can be served by keep-web
51 // without making use of any credentials supplied by the client). See
52 // "Same-origin mode" below.
54 // http://dl.example.com/c=uuid_or_pdh/path/file.txt
55 // http://dl.example.com/c=uuid_or_pdh/path/t=TOKEN/file.txt
57 // The following "multiple origin" URL patterns are supported for all
60 // http://uuid_or_pdh--dl.example.com/path/file.txt
61 // http://uuid_or_pdh--dl.example.com/t=/path/file.txt
62 // http://uuid_or_pdh--dl.example.com/t=TOKEN/path/file.txt
64 // In the "multiple origin" form, the string "--" can be replaced with
65 // "." with identical results (assuming the upstream proxy is
66 // configured accordingly). These two are equivalent:
68 // http://uuid_or_pdh--dl.example.com/path/file.txt
69 // http://uuid_or_pdh.dl.example.com/path/file.txt
71 // The first form minimizes the cost and effort of deploying a
72 // wildcard TLS certificate for *.dl.example.com. The second form is
73 // likely to be easier to configure, and more efficient to run, on an
76 // In all of the above forms, the "dl.example.com" part can be
79 // In all of the above forms, the "uuid_or_pdh" part can be either a
80 // collection UUID or a portable data hash with the "+" character
83 // Assuming there is a collection with UUID
84 // zzzzz-4zz18-znfnqtbbv4spc3w and portable data hash
85 // 1f4b0bc7583c2a7f9102c395f4ffc5e3+45, the following URLs are
88 // http://zzzzz-4zz18-znfnqtbbv4spc3w.dl.example.com/foo
89 // http://zzzzz-4zz18-znfnqtbbv4spc3w.dl.example.com/t=/foo
90 // http://zzzzz-4zz18-znfnqtbbv4spc3w--dl.example.com/t=/foo
91 // http://1f4b0bc7583c2a7f9102c395f4ffc5e3-45--foo.example.com/foo
92 // http://1f4b0bc7583c2a7f9102c395f4ffc5e3-45--.invalid/foo
94 // Authorization mechanisms
96 // A token can be provided in an Authorization header:
98 // Authorization: OAuth2 o07j4px7RlJK4CuMYp7C0LDT4CzR1J1qBE5Avo7eCcUjOTikxK
100 // A base64-encoded token can be provided in a cookie named "api_token":
102 // Cookie: api_token=bzA3ajRweDdSbEpLNEN1TVlwN0MwTERUNEN6UjFKMXFCRTVBdm83ZUNjVWpPVGlreEs=
104 // A token can be provided in an URL-encoded query string:
106 // GET /foo.txt?api_token=o07j4px7RlJK4CuMYp7C0LDT4CzR1J1qBE5Avo7eCcUjOTikxK
108 // A suitably encoded token can be provided in a POST body if the
109 // request has a content type of application/x-www-form-urlencoded or
110 // multipart/form-data:
113 // Content-Type: application/x-www-form-urlencoded
115 // api_token=o07j4px7RlJK4CuMYp7C0LDT4CzR1J1qBE5Avo7eCcUjOTikxK
117 // If a token is provided in a query string or in a POST request, the
118 // response is an HTTP 303 redirect to an equivalent GET request, with
119 // the token stripped from the query string and added to a cookie
124 // Client-provided authorization tokens are ignored if the client does
125 // not provide a Host header.
127 // In order to use the query string or a POST form authorization
128 // mechanisms, the client must follow 303 redirects; the client must
129 // accept cookies with a 303 response and send those cookies when
130 // performing the redirect; and either the client or an intervening
131 // proxy must resolve a relative URL ("//host/path") if given in a
132 // response Location header.
136 // Normally, Keep-web accepts requests for multiple collections using
137 // the same host name, provided the client's credentials are not being
138 // used. This provides insufficient XSS protection in an installation
139 // where the "anonymously accessible" data is not truly public, but
140 // merely protected by network topology.
142 // In such cases -- for example, a site which is not reachable from
143 // the internet, where some data is world-readable from Arvados's
144 // perspective but is intended to be available only to users within
145 // the local network -- the upstream proxy should configured to return
146 // 401 for all paths beginning with "/c=".
150 // Without the same-origin protection outlined above, a web page
151 // stored in collection X could execute JavaScript code that uses the
152 // current viewer's credentials to download additional data from
153 // collection Y -- data which is accessible to the current viewer, but
154 // not to the author of collection X -- from the same origin
155 // (``https://dl.example.com/'') and upload it to some other site
156 // chosen by the author of collection X.
160 // TODO(TC): Implement
164 // Normally, Keep-web is installed using a wildcard DNS entry and a
165 // wildcard HTTPS certificate, serving data from collection X at
166 // ``https://X--dl.example.com/path/file.ext''.
168 // It will also serve publicly accessible data at
169 // ``https://dl.example.com/collections/X/path/file.txt'', but it does not
170 // accept any kind of credentials at paths like these.
172 // In "trust all content" mode, Keep-web will accept credentials (API
173 // tokens) and serve any collection X at
174 // "https://dl.example.com/collections/X/path/file.ext". This is
175 // UNSAFE except in the special case where everyone who is able write
176 // ANY data to Keep, and every JavaScript and HTML file written to
177 // Keep, is also trusted to read ALL of the data in Keep.
179 // In such cases you can enable trust-all-content mode.
181 // keep-web -trust-all-content [...]
183 // In the general case, this should not be enabled: