X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/0bb53c1cbcbcb8b3be50e6ecf3fdf0bb7cbd96b5..9881b1dc0b51b0a5da6dfd92fbe3a903c7330e99:/doc/install/install-keep-balance.html.textile.liquid diff --git a/doc/install/install-keep-balance.html.textile.liquid b/doc/install/install-keep-balance.html.textile.liquid index a7f31dfe5f..bb4ae7b3d8 100644 --- a/doc/install/install-keep-balance.html.textile.liquid +++ b/doc/install/install-keep-balance.html.textile.liquid @@ -9,141 +9,49 @@ Copyright (C) The Arvados Authors. All rights reserved. SPDX-License-Identifier: CC-BY-SA-3.0 {% endcomment %} -Keep-balance deletes unreferenced and overreplicated blocks from Keep servers, makes additional copies of underreplicated blocks, and moves blocks into optimal locations as needed (e.g., after adding new servers). +# "Introduction":#introduction +# "Update config.yml":#update-config +# "Install keep-balance package":#install-packages +# "Start the service":#start-service -{% include 'notebox_begin' %} - -If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, edit your keep-balance startup script to use the flags @-commit-pulls=false -commit-trash=false@. - -{% include 'notebox_end' %} +h2(#introduction). Introduction -h2. Install keep-balance +Keep-balance deletes unreferenced and overreplicated blocks from Keep servers, makes additional copies of underreplicated blocks, and moves blocks into optimal locations as needed (e.g., after adding new servers). See "Balancing Keep servers":{{site.baseurl}}/admin/keep-balance.html for usage details. -Keep-balance can be installed anywhere with network access to Keep services. Typically it runs on the same host as keepproxy. +Keep-balance can be installed anywhere with network access to Keep services, arvados-controller, and PostgreSQL. Typically it runs on the same host as keepproxy. -*A cluster should have only one keep-balance process running at a time.* +*A cluster should have only one instance of keep-balance running at a time.* -On Debian-based systems: - - -
~$ sudo apt-get install keep-balance
-
-
- -On Red Hat-based systems: - - -
~$ sudo yum install keep-balance
-
-
- -Verify that @keep-balance@ is functional: - - -
~$ keep-balance -h
-...
-Usage: keep-balance [options]
-
-Options:
-  -commit-pulls
-        send pull requests (make more replicas of blocks that are underreplicated or are not in optimal rendezvous probe order)
-  -commit-trash
-        send trash requests (delete unreferenced old blocks, and excess replicas of overreplicated blocks)
-...
-
-
- -h3. Create a keep-balance configuration file - -On the host running keep-balance, create @/etc/arvados/keep-balance/keep-balance.yml@ using the SystemRootToken from your cluster configuration file. Follow this YAML format: - - -
Listen: :9005
-Client:
-  APIHost: uuid_prefix.your.domain:443
-  AuthToken: zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-KeepServiceTypes:
-  - disk
-ManagementToken: xyzzy
-RunPeriod: 10m
-CollectionBatchSize: 100000
-CollectionBuffers: 1000
-LostBlocksFile: /tmp/keep-balance-lost-blocks.txt    # If given, this file will be updated atomically during each successful run.
-
-
- -If your API server's SSL certificate is not signed by a recognized CA, add the @Insecure@ option to the @Client@ section: - - -
Client:
-  Insecure: true
-  APIHost: ...
-
-
- -h3. Start the service (option 1: systemd) - -If your system does not use systemd, skip this section and follow the "runit instructions":#runit instead. - -If your system uses systemd, the keep-balance service should already be set up. Start it and check its status: - - -
~$ sudo systemctl restart keep-balance
-~$ sudo systemctl status keep-balance
-● keep-balance.service - Arvados Keep Balance
-   Loaded: loaded (/lib/systemd/system/keep-balance.service; enabled)
-   Active: active (running) since Sat 2017-02-14 18:46:01 UTC; 3 days ago
-     Docs: https://doc.arvados.org/
- Main PID: 541 (keep-balance)
-   CGroup: /system.slice/keep-balance.service
-           └─541 /usr/bin/keep-balance -commit-pulls -commit-trash
-
-Feb 14 18:46:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:46:01 starting up: will scan every 10m0s and on SIGUSR1
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 Run: start
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 skipping zzzzz-bi6l4-rbtrws2jxul6i4t with service type "proxy"
-Feb 14 18:56:01 zzzzz.arvadosapi.com keep-balance[541]: 2017/02/14 18:56:01 clearing existing trash lists, in case the new rendezvous order differs from previous run
-
-
- -h3(#runit). Start the service (option 2: runit) +{% include 'notebox_begin' %} -Install runit to supervise the keep-balance daemon. {% include 'install_runit' %} +If you are installing keep-balance on an existing system with valuable data, you can run keep-balance in "dry run" mode first and review its logs as a precaution. To do this, edit your keep-balance startup script to use the flags @-commit-pulls=false -commit-trash=false -commit-confirmed-fields=false@. -Create a supervised service. +{% include 'notebox_end' %} - -
~$ sudo mkdir /etc/service/keep-balance
-~$ cd /etc/service/keep-balance
-~$ sudo mkdir log log/main
-~$ printf '#!/bin/sh\nexec keep-balance -commit-pulls -commit-trash 2>&1\n' | sudo tee run
-~$ printf '#!/bin/sh\nexec svlogd main\n' | sudo tee log/run
-~$ sudo chmod +x run log/run
-~$ sudo sv exit .
-~$ cd -
-
-
+h2(#update-config). Update the cluster config -Use @sv stat@ and check the log file to verify the service is running. +Edit the cluster config at @config.yml@ and set @Services.Keepbalance.InternalURLs@. This port is only used to publish metrics. -
~$ sudo sv stat /etc/service/keep-balance
-run: /etc/service/keep-balance: (pid 12520) 2s; run: log: (pid 12519) 2s
-~$ tail /etc/service/keep-balance/log/main/current
-2017/02/14 18:46:01 starting up: will scan every 10m0s and on SIGUSR1
-2017/02/14 18:56:01 Run: start
-2017/02/14 18:56:01 skipping zzzzz-bi6l4-rbtrws2jxul6i4t with service type "proxy"
-2017/02/14 18:56:01 clearing existing trash lists, in case the new rendezvous order differs from previous run
+
    Services:
+      Keepbalance:
+        InternalURLs:
+          "http://keep.ClusterID.example.com:9005/": {}
 
-h2. Enable garbage collection - Ensure your cluster configuration has @Collections.BlobTrash: true@ (this is the default). -
~$ arvados-server config-dump | grep BlobTrash:
+
# arvados-server config-dump | grep BlobTrash:
       BlobTrash: true
 
If BlobTrash is false, unneeded blocks will be counted and logged by keep-balance, but they will not be deleted. + +{% assign arvados_component = 'keep-balance' %} + +{% include 'install_packages' %} + +{% include 'start_service' %}