navsection: installguide
title: Install Keep-balance
...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
-Keep-balance deletes unreferenced and overreplicated blocks from Keep servers, makes additional copies of underreplicated blocks, and moves blocks into optimal locations as needed (e.g., after adding new servers).
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
-{% include 'notebox_begin' %}
-
-If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, use the @keepstore -never-delete=true@ flag or remove the @-commit-trash@ flag from your keep-balance startup script.
-
-{% include 'notebox_end' %}
-
-h2. Install keep-balance
-
-Keep-balance can be installed anywhere with access to Keep services. Typically it runs on the same host as keepproxy.
-
-On Debian-based systems:
+# "Introduction":#introduction
+# "Update config.yml":#update-config
+# "Install keep-balance package":#install-packages
+# "Start the service":#start-service
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install keep-balance</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems:
+h2(#introduction). Introduction
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install keep-balance</span>
-</code></pre>
-</notextile>
+Keep-balance deletes unreferenced and overreplicated blocks from Keep servers, makes additional copies of underreplicated blocks, and moves blocks into optimal locations as needed (e.g., after adding new servers). See "Balancing Keep servers":{{site.baseurl}}/admin/keep-balance.html for usage details.
-Verify that @keep-balance@ is functional:
-
-<notextile>
-<pre><code>~$ <span class="userinput">keep-balance -h</span>
-...
-Usage: keep-balance [options]
-
-Options:
- -commit-pulls
- send pull requests (make more replicas of blocks that are underreplicated or are not in optimal rendezvous probe order)
- -commit-trash
- send trash requests (delete unreferenced old blocks, and excess replicas of overreplicated blocks)
-...
-</code></pre>
-</notextile>
-
-h3. Create a keep-balance token
-
-Create a privileged Arvados API token for use by keep-balance. *On the API server*, run:
-
-<notextile>
-<pre><code>apiserver:~$ <span class="userinput">cd /var/www/arvados-api/current</span>
-apiserver:/var/www/arvados-api/current$ <span class="userinput">sudo -u <b>webserver-user</b> RAILS_ENV=production bundle exec script/create_superuser_token.rb</span>
-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-</code></pre>
-</notextile>
+Keep-balance can be installed anywhere with network access to Keep services, arvados-controller, and PostgreSQL. Typically it runs on the same host as keepproxy.
-h3. Create a configuration file
+*A cluster should have only one instance of keep-balance running at a time.*
-Create @/etc/arvados/keep-balance/keep-balance.yml@ using the token you generated in the previous step. Follow this YAML format:
-
-<notextile>
-<pre><code>Client:
- APIHost: <span class="userinput">uuid_prefix.your.domain</span>:443
- AuthToken: zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-KeepServiceTypes:
- - disk
-RunPeriod: 10m
-CollectionBatchSize: 100000
-CollectionBuffers: 1000
-</code></pre>
-</notextile>
-
-If your API server's SSL certificate is not signed by a recognized CA, add the @Insecure@ option to the @Client@ section:
-
-<notextile>
-<pre><code>Client:
- <span class="userinput">Insecure: true</span>
- APIHost: ...
-</code></pre>
-</notextile>
-
-h3. Start the service (option 1: systemd)
-
-If your system does not use systemd, skip this section and follow the "runit instructions":#runit instead.
-
-If your system uses systemd, the keep-balance service should already be set up. Start it and check its status:
+{% include 'notebox_begin' %}
-<notextile>
-<pre><code>~$ <span class="userinput">sudo systemctl restart keep-balance</span>
-~$ <span class="userinput">sudo systemctl status keep-balance</span>
-● keep-balance.service - Arvados Keep Balance
- Loaded: loaded (/lib/systemd/system/keep-balance.service; enabled)
- Active: active (running) since Sat 2017-02-14 18:46:01 UTC; 3 days ago
- Docs: https://doc.arvados.org/
- Main PID: 541 (keep-balance)
- CGroup: /system.slice/keep-balance.service
- └─541 /usr/bin/keep-balance -commit-pulls -commit-trash
-
-Feb 14 18:46:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:46:01 starting up: will scan every 10m0s and on SIGUSR1
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 Run: start
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 skipping zzzzz-bi6l4-rbtrws2jxul6i4t with service type "proxy"
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 clearing existing trash lists, in case the new rendezvous order differs from previous run
-</code></pre>
-</notextile>
+If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, set the @Collections.BalancePullLimit@ and @Collections.BalanceTrashLimit@ configuration entries to zero.
-h3(#runit). Start the service (option 2: runit)
+{% include 'notebox_end' %}
-Install runit to supervise the keep-balance daemon. {% include 'install_runit' %}
+h2(#update-config). Update the cluster config
-Create a supervised service.
+Edit the cluster config at @config.yml@ and set @Services.Keepbalance.InternalURLs@. This port is only used to publish metrics.
<notextile>
-<pre><code>~$ <span class="userinput">sudo mkdir /etc/service/keep-balance</span>
-~$ <span class="userinput">cd /etc/service/keep-balance</span>
-~$ <span class="userinput">sudo mkdir log log/main</span>
-~$ <span class="userinput">printf '#!/bin/sh\nexec keep-balance -commit-pulls -commit-trash 2>&1\n' | sudo tee run</span>
-~$ <span class="userinput">printf '#!/bin/sh\nexec svlogd main\n' | sudo tee log/run</span>
-~$ <span class="userinput">sudo chmod +x run log/run</span>
-~$ <span class="userinput">sudo sv exit .</span>
-~$ <span class="userinput">cd -</span>
+<pre><code> Services:
+ Keepbalance:
+ InternalURLs:
+ "http://<span class="userinput">keep.ClusterID.example.com</span>:9005/": {}
</code></pre>
</notextile>
-Use @sv stat@ and check the log file to verify the service is running.
+Ensure your cluster configuration has @Collections.BlobTrash: true@ (this is the default).
<notextile>
-<pre><code>~$ <span class="userinput">sudo sv stat /etc/service/keep-balance</span>
-run: /etc/service/keep-balance: (pid 12520) 2s; run: log: (pid 12519) 2s
-~$ <span class="userinput">tail /etc/service/keep-balance/log/main/current</span>
-2017/02/14 18:46:01 starting up: will scan every 10m0s and on SIGUSR1
-2017/02/14 18:56:01 Run: start
-2017/02/14 18:56:01 skipping zzzzz-bi6l4-rbtrws2jxul6i4t with service type "proxy"
-2017/02/14 18:56:01 clearing existing trash lists, in case the new rendezvous order differs from previous run
+<pre><code># arvados-server config-dump | grep BlobTrash:
+ BlobTrash: true
</code></pre>
</notextile>
-h2. Enable delete operations on keepstore volumes
+If BlobTrash is false, unneeded blocks will be counted and logged by keep-balance, but they will not be deleted.
-Ensure your keepstore services have the "delete" operation enabled. Otherwise, even when unneeded data is identified by keep-balance, it will never be deleted from the underlying storage devices.
+{% assign arvados_component = 'keep-balance' %}
-Either use the @EnableDelete@ flag in your YAML configuration file @/etc/arvados/keepstore/keepstore.yml@:
+{% include 'install_packages' %}
-<notextile>
-<pre><code>...
-BlobSigningKeyFile: /etc/keepstore/blob-signing.key
-<span class="userinput">EnableDelete: true</span>
-Listen: :25107
-...
-</code></pre>
-</notextile>
-
-Or use the @-never-delete=false@ command line flag in your run script:
-
-<notextile>
-<pre><code>keepstore <span class="userinput">-never-delete=false</span> -volume=...
-</code></pre>
-</notextile>
+{% include 'start_service' %}
projects / arvados.git / blobdiff