X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/ced8b355e88199516443aeb7504ebaf57f2df82c..608a1591840dfbeac3d5580c505b50bfface5929:/doc/install/install-keep-balance.html.textile.liquid diff --git a/doc/install/install-keep-balance.html.textile.liquid b/doc/install/install-keep-balance.html.textile.liquid index 19249b9ab4..a7f31dfe5f 100644 --- a/doc/install/install-keep-balance.html.textile.liquid +++ b/doc/install/install-keep-balance.html.textile.liquid @@ -3,12 +3,17 @@ layout: default navsection: installguide title: Install Keep-balance ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. + +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} Keep-balance deletes unreferenced and overreplicated blocks from Keep servers, makes additional copies of underreplicated blocks, and moves blocks into optimal locations as needed (e.g., after adding new servers). {% include 'notebox_begin' %} -If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, use the @keepstore -never-delete=true@ flag or remove the @-commit-trash@ flag from your keep-balance startup script. +If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, edit your keep-balance startup script to use the flags @-commit-pulls=false -commit-trash=false@. {% include 'notebox_end' %} @@ -16,6 +21,8 @@ h2. Install keep-balance Keep-balance can be installed anywhere with network access to Keep services. Typically it runs on the same host as keepproxy. +*A cluster should have only one keep-balance process running at a time.* + On Debian-based systems: @@ -46,30 +53,22 @@ Options: -h3. Create a keep-balance token - -Create an Arvados superuser token for use by keep-balance. *On the API server*, run: - - -
apiserver:~$ cd /var/www/arvados-api/current
-apiserver:/var/www/arvados-api/current$ sudo -u webserver-user RAILS_ENV=production bundle exec script/create_superuser_token.rb
-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-
-
- -h3. Create a configuration file +h3. Create a keep-balance configuration file -On the host running keep-balance, create @/etc/arvados/keep-balance/keep-balance.yml@ using the token you generated in the previous step. Follow this YAML format: +On the host running keep-balance, create @/etc/arvados/keep-balance/keep-balance.yml@ using the SystemRootToken from your cluster configuration file. Follow this YAML format: -
Client:
+
Listen: :9005
+Client:
   APIHost: uuid_prefix.your.domain:443
   AuthToken: zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
 KeepServiceTypes:
   - disk
+ManagementToken: xyzzy
 RunPeriod: 10m
 CollectionBatchSize: 100000
 CollectionBuffers: 1000
+LostBlocksFile: /tmp/keep-balance-lost-blocks.txt    # If given, this file will be updated atomically during each successful run.
 
@@ -137,28 +136,14 @@ run: /etc/service/keep-balance: (pid 12520) 2s; run: log: (pid 12519) 2s
-h2. Enable delete operations on keepstore volumes - -Ensure your keepstore services have the "delete" operation enabled. If it is disabled (which is the default), unneeded blocks will be identified by keep-balance, but will never be deleted from the underlying storage devices. +h2. Enable garbage collection -Add the @-never-delete=false@ command line flag to your keepstore run script: +Ensure your cluster configuration has @Collections.BlobTrash: true@ (this is the default). -
keepstore -never-delete=false -volume=...
+
~$ arvados-server config-dump | grep BlobTrash:
+      BlobTrash: true
 
-{% comment %} -// To replace the above section when the keepstore page recommends YAML... - -Use the @EnableDelete@ flag in your YAML configuration file @/etc/arvados/keepstore/keepstore.yml@: - - -
...
-BlobSigningKeyFile: /etc/keepstore/blob-signing.key
-EnableDelete: true
-Listen: :25107
-...
-
-
-{% endcomment %} +If BlobTrash is false, unneeded blocks will be counted and logged by keep-balance, but they will not be deleted.