X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/102df19458ef2c97d1ef4ba0e571e3204d7073e6..531fd553a1b83c546066c1d2a2619f86e17b6d20:/doc/install/install-keepstore.html.textile.liquid diff --git a/doc/install/install-keepstore.html.textile.liquid b/doc/install/install-keepstore.html.textile.liquid index d6a529d38b..3cb922642e 100644 --- a/doc/install/install-keepstore.html.textile.liquid +++ b/doc/install/install-keepstore.html.textile.liquid @@ -14,6 +14,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0 # "Install keepstore package":#install-packages # "Restart the API server and controller":#restart-api # "Confirm working installation":#confirm-working +# "Note on storage management":#note h2. Introduction @@ -35,8 +36,8 @@ By convention, we use the following hostname pattern:
table(table table-bordered table-condensed). |_. Hostname| -|keep0.@ClusterID@.example.com| -|keep1.@ClusterID@.example.com| +|@keep0.ClusterID.example.com@| +|@keep1.ClusterID.example.com@|
Keepstore servers should not be directly accessible from the Internet (they are accessed via "keepproxy":install-keepproxy.html), so the hostnames only need to resolve on the private network. @@ -51,6 +52,8 @@ Fill in the @Volumes@ section of @config.yml@ for each storage volume. Availabl * If you are using S3-compatible object storage (including Amazon S3, Google Cloud Storage, and Ceph RADOS), follow the setup instructions on "S3 Object Storage":configure-s3-object-storage.html * If you are using Azure Blob Storage, follow the setup instructions on "Azure Blob Storage":configure-azure-blob-storage.html +There are a number of general configuration parameters for Keepstore. They are described in the "configuration reference":{{site.baseurl}}/admin/config.html. In particular, you probably want to change @API/MaxKeepBlobBuffers@ to align Keepstore's memory usage with the available memory on the machine that hosts it. + h3. List services Add each keepstore server to the @Services.Keepstore@ section of @/etc/arvados/config.yml@ . @@ -60,9 +63,9 @@ Add each keepstore server to the @Services.Keepstore@ section of @/etc/arvados/c Keepstore: # No ExternalURL because they are only accessed by the internal subnet. InternalURLs: - "http://keep0.ClusterID.example.com:25107/": {} - "http://keep1.ClusterID.example.com:25107/": {} - # and so forth + "http://keep0.ClusterID.example.com:25107": {} + "http://keep1.ClusterID.example.com:25107": {} + # and so forth @@ -76,26 +79,33 @@ Add each keepstore server to the @Services.Keepstore@ section of @/etc/arvados/c h2(#confirm-working). Confirm working installation -Install the "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html +Log into a host that is on your private Arvados network. The host should be able to contact your your keepstore servers (eg keep[0-9].ClusterID.example.com). @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ must be set in the environment. -You should now be able to use @arv-put@ to upload collections and @arv-get@ to fetch collections: +@ARVADOS_API_HOST@ should be the hostname of the API server. + +@ARVADOS_API_TOKEN@ should be the system root token. + +Install the "Command line SDK":{{site.baseurl}}/sdk/cli/install.html + +Check that the keepstore server is in the @keep_service@ "accessible" list: + + +

+$ arv keep_service accessible
+[...]
+
+
-
-$ echo "hello world!" > hello.txt
+If keepstore does not show up in the "accessible" list, and you are accessing it from within the private network, check that you have "properly configured the @geo@ block for the API server":install-api-server.html#update-nginx .
 
-$ arv-put --portable-data-hash hello.txt
-2018-07-12 13:35:25 arvados.arv_put[28702] INFO: Creating new cache file at /home/example/.cache/arvados/arv-put/1571ec0adb397c6a18d5c74cc95b3a2a
-0M / 0M 100.0% 2018-07-12 13:35:27 arvados.arv_put[28702] INFO:
+Next, install the "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html
 
-2018-07-12 13:35:27 arvados.arv_put[28702] INFO: Collection saved as 'Saved at 2018-07-12 17:35:25 UTC by example@example'
-59389a8f9ee9d399be35462a0f92541c+53
+You should now be able to use @arv-put@ to upload collections and @arv-get@ to fetch collections.  Be sure to execute this from _inside_ the cluster's private network.  You will be able to access keep from _outside_ the private network after setting up "keepproxy":install-keepproxy.html .
 
-$ arv-get 59389a8f9ee9d399be35462a0f92541c+53/hello.txt
-hello world!
-
+{% include 'arv_put_example' %} -h3. Note on storage management +h2(#note). Note on storage management On its own, a keepstore server never deletes data. Instead, the keep-balance service determines which blocks are candidates for deletion and instructs the keepstore to move those blocks to the trash. Please see the "Balancing Keep servers":{{site.baseurl}}/admin/keep-balance.html for more details.