arvados.git
4 years agoMerge branch '15699-workbench-copy-fail'
Tom Clegg [Tue, 15 Oct 2019 19:21:55 +0000 (15:21 -0400)]
Merge branch '15699-workbench-copy-fail'

refs #15699

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15699: Remove unused variables.
Tom Clegg [Tue, 15 Oct 2019 19:20:23 +0000 (15:20 -0400)]
15699: Remove unused variables.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15717-svc-select-port' refs #15717
Peter Amstutz [Tue, 15 Oct 2019 18:53:37 +0000 (14:53 -0400)]
Merge branch '15717-svc-select-port' refs #15717

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15721-config-cors'
Tom Clegg [Tue, 15 Oct 2019 18:46:50 +0000 (14:46 -0400)]
Merge branch '15721-config-cors'

fixes #15721

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15717: If there are several InternalURLs for the same host, try them all
Peter Amstutz [Tue, 15 Oct 2019 15:37:24 +0000 (11:37 -0400)]
15717: If there are several InternalURLs for the same host, try them all

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15721: Fix 503 on API.MaxConcurrentRequests==0 and empty response.
Tom Clegg [Tue, 15 Oct 2019 14:24:02 +0000 (10:24 -0400)]
15721: Fix 503 on API.MaxConcurrentRequests==0 and empty response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15424-wb2-welcome-page' refs #15424
Peter Amstutz [Fri, 11 Oct 2019 18:57:31 +0000 (14:57 -0400)]
Merge branch '15424-wb2-welcome-page' refs #15424

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15694-cwl-hang' closes #15694
Peter Amstutz [Fri, 11 Oct 2019 18:54:44 +0000 (14:54 -0400)]
Merge branch '15694-cwl-hang' closes #15694

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15424: Tweak default WelcomePageHTML
Peter Amstutz [Fri, 11 Oct 2019 18:48:39 +0000 (14:48 -0400)]
15424: Tweak default WelcomePageHTML

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15694: Improve test comments/assertions
Peter Amstutz [Fri, 11 Oct 2019 17:16:23 +0000 (13:16 -0400)]
15694: Improve test comments/assertions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15531-logincluster-migrate' refs #15531
Peter Amstutz [Fri, 11 Oct 2019 15:04:42 +0000 (11:04 -0400)]
Merge branch '15531-logincluster-migrate' refs #15531

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15699: Fix handling of streams with multiple refs to a block ID.
Tom Clegg [Fri, 11 Oct 2019 05:11:45 +0000 (01:11 -0400)]
15699: Fix handling of streams with multiple refs to a block ID.

The manifest normalization code in the Ruby SDK (and therefore
Workbench) was based on an incorrect assumption that each block
locator could only appear once in a given stream.

If a manifest referenced the same block more than once, copying a file
from that manifest into a new one would produce a new file with the
correct size, but wrong data.

The new code uses a different strategy that deduplicates block
references in common cases, although not in all possible cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15531: Don't activate new user if old user is not active
Peter Amstutz [Thu, 10 Oct 2019 21:55:32 +0000 (17:55 -0400)]
15531: Don't activate new user if old user is not active

Fix Python 2.7

Fix check.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoAdd the GOGC=10 environment variable to the Keepstore service file, to make
Ward Vandewege [Thu, 10 Oct 2019 16:47:38 +0000 (12:47 -0400)]
Add the GOGC=10 environment variable to the Keepstore service file, to make
Keepstore memory use tightly coupled to the number of buffers it is
configured to use.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

4 years ago15424: Add WelcomePageHTML to config, with default text
Peter Amstutz [Thu, 10 Oct 2019 15:10:06 +0000 (11:10 -0400)]
15424: Add WelcomePageHTML to config, with default text

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoFix wrong subcommand name.
Tom Clegg [Thu, 10 Oct 2019 13:56:46 +0000 (09:56 -0400)]
Fix wrong subcommand name.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoFix typo.
Tom Clegg [Thu, 10 Oct 2019 13:51:38 +0000 (09:51 -0400)]
Fix typo.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15694: Add test
Peter Amstutz [Wed, 9 Oct 2019 18:56:45 +0000 (14:56 -0400)]
15694: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15694: Catch exception from get_current_container in runtime_status_update
Peter Amstutz [Wed, 9 Oct 2019 18:10:11 +0000 (14:10 -0400)]
15694: Catch exception from get_current_container in runtime_status_update

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15655-logtail-encoding'
Eric Biagiotti [Wed, 9 Oct 2019 13:31:06 +0000 (09:31 -0400)]
Merge branch '15655-logtail-encoding'

refs #15655

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago15655: Fixes logtail encoding
Eric Biagiotti [Wed, 9 Oct 2019 12:40:30 +0000 (08:40 -0400)]
15655: Fixes logtail encoding

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago15531: Fix file ownership after git pull as root
Peter Amstutz [Tue, 8 Oct 2019 20:41:22 +0000 (16:41 -0400)]
15531: Fix file ownership after git pull as root

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Moved federation section in admin guide
Peter Amstutz [Tue, 8 Oct 2019 19:22:04 +0000 (15:22 -0400)]
15531: Moved federation section in admin guide

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch 'master' into 15531-logincluster-migrate
Peter Amstutz [Tue, 8 Oct 2019 19:21:32 +0000 (15:21 -0400)]
Merge branch 'master' into 15531-logincluster-migrate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Fixing up test framework
Peter Amstutz [Tue, 8 Oct 2019 19:16:47 +0000 (15:16 -0400)]
15531: Fixing up test framework

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoarvbox update config.yml when config override is updated refs #15531
Peter Amstutz [Tue, 8 Oct 2019 19:05:54 +0000 (15:05 -0400)]
arvbox update config.yml when config override is updated refs #15531

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Documentation updates
Peter Amstutz [Fri, 4 Oct 2019 19:03:37 +0000 (15:03 -0400)]
15531: Documentation updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15640-cwl-min-ram' refs #15640
Peter Amstutz [Fri, 4 Oct 2019 15:37:03 +0000 (11:37 -0400)]
Merge branch '15640-cwl-min-ram' refs #15640

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15640: RunInSingleContainer needs minimum of 128 MiB RAM
Peter Amstutz [Fri, 4 Oct 2019 14:23:00 +0000 (10:23 -0400)]
15640: RunInSingleContainer needs minimum of 128 MiB RAM

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15684-package-deps' refs #15684
Peter Amstutz [Thu, 3 Oct 2019 22:01:46 +0000 (18:01 -0400)]
Merge branch '15684-package-deps' refs #15684

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15684: Fix arvados-api-server and arvados-workbench packaging
Peter Amstutz [Thu, 3 Oct 2019 21:57:25 +0000 (17:57 -0400)]
15684: Fix arvados-api-server and arvados-workbench packaging

- Fix test_package_presence to use correct URL to download prebuilt dev debs
- Dev package version of api is now latest(api, arvados-server)
- Dev package version of workbench is now latest(workbench, arvados-server)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '14714-keep-balance-config'
Eric Biagiotti [Thu, 3 Oct 2019 19:33:59 +0000 (15:33 -0400)]
Merge branch '14714-keep-balance-config'

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Corrects keepstore service distinction in the upgrade notes
Eric Biagiotti [Thu, 3 Oct 2019 19:33:14 +0000 (15:33 -0400)]
14714: Corrects keepstore service distinction in the upgrade notes

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Improves error message when using the legacy keep-balance config
Eric Biagiotti [Thu, 3 Oct 2019 18:45:47 +0000 (14:45 -0400)]
14714: Improves error message when using the legacy keep-balance config

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoMerge branch '15014-wrong-node-count'
Tom Clegg [Thu, 3 Oct 2019 15:44:22 +0000 (11:44 -0400)]
Merge branch '15014-wrong-node-count'

fixes #15014

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '11016-doc-signing-ttl'
Tom Clegg [Thu, 3 Oct 2019 15:39:18 +0000 (11:39 -0400)]
Merge branch '11016-doc-signing-ttl'

refs #11016

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '14714-keep-balance-config'
Eric Biagiotti [Thu, 3 Oct 2019 15:09:39 +0000 (11:09 -0400)]
Merge branch '14714-keep-balance-config'

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoMerge branch '15654-wb-gem-update' refs #15654
Peter Amstutz [Thu, 3 Oct 2019 14:48:19 +0000 (10:48 -0400)]
Merge branch '15654-wb-gem-update' refs #15654

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15654: Update rubyzip gem
Peter Amstutz [Wed, 2 Oct 2019 20:27:40 +0000 (16:27 -0400)]
15654: Update rubyzip gem

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15654: Update nokogiri
Peter Amstutz [Mon, 30 Sep 2019 19:46:22 +0000 (15:46 -0400)]
15654: Update nokogiri

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge branch '15678-pkg-version'
Tom Clegg [Wed, 2 Oct 2019 17:09:42 +0000 (13:09 -0400)]
Merge branch '15678-pkg-version'

closes #15678

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15599-keepstore-iam-role'
Tom Clegg [Wed, 2 Oct 2019 15:48:55 +0000 (11:48 -0400)]
Merge branch '15599-keepstore-iam-role'

closes #15599

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15640-cwl-test-mem' refs #15640
Peter Amstutz [Tue, 1 Oct 2019 20:30:52 +0000 (16:30 -0400)]
Merge branch '15640-cwl-test-mem' refs #15640

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago14714: Removes metrics serving
Eric Biagiotti [Tue, 1 Oct 2019 20:23:38 +0000 (16:23 -0400)]
14714: Removes metrics serving

- Updates test gathering of metrics
- Simplifies balancer keep service discovery
- Moves keep-balance info from keepstore install docs to the new admin page
- Keepbalance now exits correctly when using -once

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoarvbox pin yarn to 1.17.3 no issue #
Peter Amstutz [Tue, 1 Oct 2019 19:22:58 +0000 (15:22 -0400)]
arvbox pin yarn to 1.17.3 no issue #

Work around for "Incorrect integrity when fetching from the cache"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15599: Reject configuration with both IAMRole and Access/SecretKey.
Tom Clegg [Tue, 1 Oct 2019 18:52:03 +0000 (14:52 -0400)]
15599: Reject configuration with both IAMRole and Access/SecretKey.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Use default/automatic credentials for example config.
Tom Clegg [Tue, 1 Oct 2019 18:47:25 +0000 (14:47 -0400)]
15599: Use default/automatic credentials for example config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoUpdate workbench bundle.
Tom Clegg [Tue, 1 Oct 2019 18:38:21 +0000 (14:38 -0400)]
Update workbench bundle.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15678: Fix version reporting.
Tom Clegg [Tue, 1 Oct 2019 17:46:14 +0000 (13:46 -0400)]
15678: Fix version reporting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15656-user-delete-container'
Tom Clegg [Tue, 1 Oct 2019 14:29:07 +0000 (10:29 -0400)]
Merge branch '15656-user-delete-container'

refs #15656

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15656: Note "delete container" is admin-only and not normally used.
Tom Clegg [Tue, 1 Oct 2019 14:24:02 +0000 (10:24 -0400)]
15656: Note "delete container" is admin-only and not normally used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago11016: Update BlobSigningTTL config docs.
Tom Clegg [Mon, 30 Sep 2019 19:31:55 +0000 (15:31 -0400)]
11016: Update BlobSigningTTL config docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoUpdate login-sync Gemfile.lock.
Tom Clegg [Tue, 1 Oct 2019 03:34:23 +0000 (23:34 -0400)]
Update login-sync Gemfile.lock.

refs #15673

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Improve error message when no role is defined.
Tom Clegg [Mon, 30 Sep 2019 23:22:27 +0000 (19:22 -0400)]
15599: Improve error message when no role is defined.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Warn if multiple roles are assigned.
Tom Clegg [Mon, 30 Sep 2019 23:22:25 +0000 (19:22 -0400)]
15599: Warn if multiple roles are assigned.

This is currently impossible on AWS. If it becomes possible in the
future, this may help with troubleshooting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Look up IAM role name from metadata if not configured.
Tom Clegg [Fri, 27 Sep 2019 21:30:47 +0000 (17:30 -0400)]
15599: Look up IAM role name from metadata if not configured.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Fix misleading error message when address already in use.
Tom Clegg [Fri, 27 Sep 2019 20:40:19 +0000 (16:40 -0400)]
15599: Fix misleading error message when address already in use.

Also warn about unexpected errors encountered while looking for the
current host's entry in InternalURLs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15599: Support IAM role credentials in keepstore.
Tom Clegg [Fri, 27 Sep 2019 20:39:51 +0000 (16:39 -0400)]
15599: Support IAM role credentials in keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15656: Fix missing permission check.
Tom Clegg [Thu, 26 Sep 2019 14:47:33 +0000 (10:47 -0400)]
15656: Fix missing permission check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15014: Update tests.
Tom Clegg [Thu, 26 Sep 2019 14:07:11 +0000 (10:07 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15014: Update tests.
Tom Clegg [Fri, 13 Sep 2019 19:45:28 +0000 (15:45 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15014: Update tests.
Tom Clegg [Fri, 13 Sep 2019 18:48:37 +0000 (14:48 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15014: Hide busy/idle nodes panel when crunch1 is not active.
Tom Clegg [Fri, 13 Sep 2019 17:24:09 +0000 (13:24 -0400)]
15014: Hide busy/idle nodes panel when crunch1 is not active.

The node counts are incorrect when the crunch1 dispatch processes are
not running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15673-faraday-pin'
Tom Clegg [Mon, 30 Sep 2019 20:44:34 +0000 (16:44 -0400)]
Merge branch '15673-faraday-pin'

refs #15673

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15673: Pin faraday below 0.16.
Tom Clegg [Mon, 30 Sep 2019 19:33:53 +0000 (15:33 -0400)]
15673: Pin faraday below 0.16.

This allows arvados-cli and login-sync to install correctly even if an
older arvados gem (without the faraday pin) is already installed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15673: Pin faraday gem version.
Tom Clegg [Mon, 30 Sep 2019 19:03:06 +0000 (15:03 -0400)]
15673: Pin faraday gem version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15673: Fix ruby warnings.
Tom Clegg [Mon, 30 Sep 2019 14:00:28 +0000 (10:00 -0400)]
15673: Fix ruby warnings.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15673: Show stderr before giving up failed command.
Tom Clegg [Mon, 30 Sep 2019 13:39:25 +0000 (09:39 -0400)]
15673: Show stderr before giving up failed command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoUse default Azure storage endpoint if configured value is "".
Tom Clegg [Mon, 30 Sep 2019 18:53:57 +0000 (14:53 -0400)]
Use default Azure storage endpoint if configured value is "".

This is the documented behavior.

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15640: Adjust memory requirements
Peter Amstutz [Mon, 30 Sep 2019 18:51:00 +0000 (14:51 -0400)]
15640: Adjust memory requirements

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years agoMerge remote-tracking branch 'origin/master' into 14714-keep-balance-config
Eric Biagiotti [Mon, 30 Sep 2019 16:31:13 +0000 (12:31 -0400)]
Merge remote-tracking branch 'origin/master' into 14714-keep-balance-config

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Simplifies server startup and removes globals
Eric Biagiotti [Mon, 30 Sep 2019 16:26:20 +0000 (12:26 -0400)]
14714: Simplifies server startup and removes globals

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Updates KeepServiceTypes legacy mapping and error messages
Eric Biagiotti [Mon, 30 Sep 2019 16:08:15 +0000 (12:08 -0400)]
14714: Updates KeepServiceTypes legacy mapping and error messages

- Config loading will now fail if the KeepServiceList config option is provided with any value other than "disk", which is equal to the new behavior.
- Changes default BalanceCollectionBatch config to zero.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoMerge branch '13647-keepstore-config'
Tom Clegg [Mon, 30 Sep 2019 15:13:24 +0000 (11:13 -0400)]
Merge branch '13647-keepstore-config'

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago14714: Adds missing doc
Eric Biagiotti [Fri, 27 Sep 2019 20:53:11 +0000 (16:53 -0400)]
14714: Adds missing doc

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago15531: Update arv-federation-migrate documentation for LoginCluster
Peter Amstutz [Fri, 27 Sep 2019 20:20:27 +0000 (16:20 -0400)]
15531: Update arv-federation-migrate documentation for LoginCluster

Also take out some mentions of the obsolete application.yml and
reorganize a couple other pages that describe obsolete features.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago14714: Updates/adds keep-balance docs
Eric Biagiotti [Fri, 27 Sep 2019 20:00:00 +0000 (16:00 -0400)]
14714: Updates/adds keep-balance docs

Also, a small typo fix in the admin collection versioning page.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago13647: Check ARVADOS_* env vars when loading config.
Tom Clegg [Fri, 27 Sep 2019 18:12:22 +0000 (14:12 -0400)]
13647: Check ARVADOS_* env vars when loading config.

Previously they were only checked when starting up a service, so they
weren't noticed by config-dump etc., and couldn't be used by the
keepstore config migration when loading the keep_services endpoint.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15531: Refactor federation-migrate
Peter Amstutz [Fri, 27 Sep 2019 17:59:56 +0000 (13:59 -0400)]
15531: Refactor federation-migrate

Split into functions for readability.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago13647: Don't check pending keepstore migrations during API startup.
Tom Clegg [Fri, 27 Sep 2019 17:38:35 +0000 (13:38 -0400)]
13647: Don't check pending keepstore migrations during API startup.

Fixes deadlock: If controller is already up when RailsAPI boots,
RailsAPI runs "config-dump -skip-legacy", which (until this change)
requests .../keep_services; controller forwards the request to
RailsAPI; and passenger queues the request until RailsAPI finishes
booting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago15531: Bump API server revision in discovery doc for feature test
Peter Amstutz [Fri, 27 Sep 2019 17:08:46 +0000 (13:08 -0400)]
15531: Bump API server revision in discovery doc for feature test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago14714: Uses lib/cmd prometheus registry
Eric Biagiotti [Thu, 26 Sep 2019 19:44:01 +0000 (15:44 -0400)]
14714: Uses lib/cmd prometheus registry

Also switched test logging to use ctxlog TestLogger

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoAvoid nginx startup race in test suite.
Tom Clegg [Thu, 26 Sep 2019 19:27:17 +0000 (15:27 -0400)]
Avoid nginx startup race in test suite.

If run_test_server.py doesn't wait for nginx to start up before
returning, run-tests.sh might check the pidfile, conclude nginx has
died, and abort.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago14714: Sets the token on the arvados client
Eric Biagiotti [Thu, 26 Sep 2019 15:22:18 +0000 (11:22 -0400)]
14714: Sets the token on the arvados client

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Removes context from Server functions. Adds sync.Once to setup
Eric Biagiotti [Thu, 26 Sep 2019 14:22:12 +0000 (10:22 -0400)]
14714: Removes context from Server functions. Adds sync.Once to setup

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Tests use cluster config
Eric Biagiotti [Wed, 25 Sep 2019 17:51:07 +0000 (13:51 -0400)]
14714: Tests use cluster config

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: keep-balance uses cluster config
Eric Biagiotti [Tue, 24 Sep 2019 13:58:40 +0000 (09:58 -0400)]
14714: keep-balance uses cluster config

- Removes dumpconfig flag.
- Removes the options to specify a keep service list or type. Will now balance all keep services of type disk reported by the keep_services endpoint.
- Debug flag removed. Uses SystemLogs.LogLevel from cluster config instead.
- Reorganizes the Server struct to use lib/service to do generic service things.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years ago14714: Adds keep-balance to cluster config loading
Eric Biagiotti [Fri, 6 Sep 2019 18:39:39 +0000 (14:39 -0400)]
14714: Adds keep-balance to cluster config loading

Also adds a deprecated config loading test and fixes the service file

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

4 years agoMerge branch '13647-keepstore-config'
Tom Clegg [Thu, 26 Sep 2019 15:15:56 +0000 (11:15 -0400)]
Merge branch '13647-keepstore-config'

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago13647: Migrate arvbox volume config.
Tom Clegg [Wed, 25 Sep 2019 21:15:11 +0000 (17:15 -0400)]
13647: Migrate arvbox volume config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago13647: Set X-External-Client header in arvbox.
Tom Clegg [Wed, 25 Sep 2019 19:29:23 +0000 (15:29 -0400)]
13647: Set X-External-Client header in arvbox.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch 'master' into 15531-logincluster-migrate
Peter Amstutz [Mon, 23 Sep 2019 21:35:07 +0000 (17:35 -0400)]
Merge branch 'master' into 15531-logincluster-migrate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Adjust behavior redirecting remote user to a local one, add test
Peter Amstutz [Mon, 23 Sep 2019 20:52:02 +0000 (16:52 -0400)]
15531: Adjust behavior redirecting remote user to a local one, add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Add README
Peter Amstutz [Mon, 23 Sep 2019 19:24:57 +0000 (15:24 -0400)]
15531: Add README

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Adjust redirect_to_new_user to intended behavior
Peter Amstutz [Mon, 23 Sep 2019 18:39:51 +0000 (14:39 -0400)]
15531: Adjust redirect_to_new_user to intended behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago15531: Fed migrate script passes test cases
Peter Amstutz [Mon, 23 Sep 2019 15:32:03 +0000 (11:32 -0400)]
15531: Fed migrate script passes test cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4 years ago13647: Add keepstore service entries to arvbox config.
Tom Clegg [Sat, 21 Sep 2019 06:28:37 +0000 (02:28 -0400)]
13647: Add keepstore service entries to arvbox config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago13647: Remove duplicate Keepproxy config.
Tom Clegg [Sat, 21 Sep 2019 06:27:34 +0000 (02:27 -0400)]
13647: Remove duplicate Keepproxy config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years ago13647: Fix send-HUP-to-keepproxy command.
Tom Clegg [Sat, 21 Sep 2019 06:26:52 +0000 (02:26 -0400)]
13647: Fix send-HUP-to-keepproxy command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4 years agoMerge branch '15648-centos7-upgrade-to-rh-python36'
Ward Vandewege [Fri, 20 Sep 2019 20:23:41 +0000 (16:23 -0400)]
Merge branch '15648-centos7-upgrade-to-rh-python36'

closes #15648

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>