arvados.git
5 years ago15716: Don't include modules on .rake files
Lucas Di Pentima [Fri, 18 Oct 2019 21:44:49 +0000 (18:44 -0300)]
15716: Don't include modules on .rake files

The file symbols.rake used to include CurrentApiClient, making its methods
available to the entire app as Object class methods.
This made several tests to pass successfully while the code they exercised
really fail on production and development environments.

The solution is to make a helper class that works as a namespace of those
methods.

This commit exposes 41 failing tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '15699-workbench-copy-fail'
Tom Clegg [Tue, 15 Oct 2019 19:21:55 +0000 (15:21 -0400)]
Merge branch '15699-workbench-copy-fail'

refs #15699

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15699: Remove unused variables.
Tom Clegg [Tue, 15 Oct 2019 19:20:23 +0000 (15:20 -0400)]
15699: Remove unused variables.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15717-svc-select-port' refs #15717
Peter Amstutz [Tue, 15 Oct 2019 18:53:37 +0000 (14:53 -0400)]
Merge branch '15717-svc-select-port' refs #15717

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15721-config-cors'
Tom Clegg [Tue, 15 Oct 2019 18:46:50 +0000 (14:46 -0400)]
Merge branch '15721-config-cors'

fixes #15721

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15717: If there are several InternalURLs for the same host, try them all
Peter Amstutz [Tue, 15 Oct 2019 15:37:24 +0000 (11:37 -0400)]
15717: If there are several InternalURLs for the same host, try them all

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15721: Fix 503 on API.MaxConcurrentRequests==0 and empty response.
Tom Clegg [Tue, 15 Oct 2019 14:24:02 +0000 (10:24 -0400)]
15721: Fix 503 on API.MaxConcurrentRequests==0 and empty response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15424-wb2-welcome-page' refs #15424
Peter Amstutz [Fri, 11 Oct 2019 18:57:31 +0000 (14:57 -0400)]
Merge branch '15424-wb2-welcome-page' refs #15424

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15694-cwl-hang' closes #15694
Peter Amstutz [Fri, 11 Oct 2019 18:54:44 +0000 (14:54 -0400)]
Merge branch '15694-cwl-hang' closes #15694

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15424: Tweak default WelcomePageHTML
Peter Amstutz [Fri, 11 Oct 2019 18:48:39 +0000 (14:48 -0400)]
15424: Tweak default WelcomePageHTML

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15694: Improve test comments/assertions
Peter Amstutz [Fri, 11 Oct 2019 17:16:23 +0000 (13:16 -0400)]
15694: Improve test comments/assertions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15531-logincluster-migrate' refs #15531
Peter Amstutz [Fri, 11 Oct 2019 15:04:42 +0000 (11:04 -0400)]
Merge branch '15531-logincluster-migrate' refs #15531

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15699: Fix handling of streams with multiple refs to a block ID.
Tom Clegg [Fri, 11 Oct 2019 05:11:45 +0000 (01:11 -0400)]
15699: Fix handling of streams with multiple refs to a block ID.

The manifest normalization code in the Ruby SDK (and therefore
Workbench) was based on an incorrect assumption that each block
locator could only appear once in a given stream.

If a manifest referenced the same block more than once, copying a file
from that manifest into a new one would produce a new file with the
correct size, but wrong data.

The new code uses a different strategy that deduplicates block
references in common cases, although not in all possible cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15531: Don't activate new user if old user is not active
Peter Amstutz [Thu, 10 Oct 2019 21:55:32 +0000 (17:55 -0400)]
15531: Don't activate new user if old user is not active

Fix Python 2.7

Fix check.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoAdd the GOGC=10 environment variable to the Keepstore service file, to make
Ward Vandewege [Thu, 10 Oct 2019 16:47:38 +0000 (12:47 -0400)]
Add the GOGC=10 environment variable to the Keepstore service file, to make
Keepstore memory use tightly coupled to the number of buffers it is
configured to use.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago15424: Add WelcomePageHTML to config, with default text
Peter Amstutz [Thu, 10 Oct 2019 15:10:06 +0000 (11:10 -0400)]
15424: Add WelcomePageHTML to config, with default text

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFix wrong subcommand name.
Tom Clegg [Thu, 10 Oct 2019 13:56:46 +0000 (09:56 -0400)]
Fix wrong subcommand name.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoFix typo.
Tom Clegg [Thu, 10 Oct 2019 13:51:38 +0000 (09:51 -0400)]
Fix typo.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15694: Add test
Peter Amstutz [Wed, 9 Oct 2019 18:56:45 +0000 (14:56 -0400)]
15694: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15694: Catch exception from get_current_container in runtime_status_update
Peter Amstutz [Wed, 9 Oct 2019 18:10:11 +0000 (14:10 -0400)]
15694: Catch exception from get_current_container in runtime_status_update

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15655-logtail-encoding'
Eric Biagiotti [Wed, 9 Oct 2019 13:31:06 +0000 (09:31 -0400)]
Merge branch '15655-logtail-encoding'

refs #15655

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago15655: Fixes logtail encoding
Eric Biagiotti [Wed, 9 Oct 2019 12:40:30 +0000 (08:40 -0400)]
15655: Fixes logtail encoding

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago15531: Fix file ownership after git pull as root
Peter Amstutz [Tue, 8 Oct 2019 20:41:22 +0000 (16:41 -0400)]
15531: Fix file ownership after git pull as root

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Moved federation section in admin guide
Peter Amstutz [Tue, 8 Oct 2019 19:22:04 +0000 (15:22 -0400)]
15531: Moved federation section in admin guide

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch 'master' into 15531-logincluster-migrate
Peter Amstutz [Tue, 8 Oct 2019 19:21:32 +0000 (15:21 -0400)]
Merge branch 'master' into 15531-logincluster-migrate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Fixing up test framework
Peter Amstutz [Tue, 8 Oct 2019 19:16:47 +0000 (15:16 -0400)]
15531: Fixing up test framework

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoarvbox update config.yml when config override is updated refs #15531
Peter Amstutz [Tue, 8 Oct 2019 19:05:54 +0000 (15:05 -0400)]
arvbox update config.yml when config override is updated refs #15531

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Documentation updates
Peter Amstutz [Fri, 4 Oct 2019 19:03:37 +0000 (15:03 -0400)]
15531: Documentation updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15640-cwl-min-ram' refs #15640
Peter Amstutz [Fri, 4 Oct 2019 15:37:03 +0000 (11:37 -0400)]
Merge branch '15640-cwl-min-ram' refs #15640

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15640: RunInSingleContainer needs minimum of 128 MiB RAM
Peter Amstutz [Fri, 4 Oct 2019 14:23:00 +0000 (10:23 -0400)]
15640: RunInSingleContainer needs minimum of 128 MiB RAM

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15684-package-deps' refs #15684
Peter Amstutz [Thu, 3 Oct 2019 22:01:46 +0000 (18:01 -0400)]
Merge branch '15684-package-deps' refs #15684

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15684: Fix arvados-api-server and arvados-workbench packaging
Peter Amstutz [Thu, 3 Oct 2019 21:57:25 +0000 (17:57 -0400)]
15684: Fix arvados-api-server and arvados-workbench packaging

- Fix test_package_presence to use correct URL to download prebuilt dev debs
- Dev package version of api is now latest(api, arvados-server)
- Dev package version of workbench is now latest(workbench, arvados-server)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14714-keep-balance-config'
Eric Biagiotti [Thu, 3 Oct 2019 19:33:59 +0000 (15:33 -0400)]
Merge branch '14714-keep-balance-config'

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Corrects keepstore service distinction in the upgrade notes
Eric Biagiotti [Thu, 3 Oct 2019 19:33:14 +0000 (15:33 -0400)]
14714: Corrects keepstore service distinction in the upgrade notes

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Improves error message when using the legacy keep-balance config
Eric Biagiotti [Thu, 3 Oct 2019 18:45:47 +0000 (14:45 -0400)]
14714: Improves error message when using the legacy keep-balance config

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '15014-wrong-node-count'
Tom Clegg [Thu, 3 Oct 2019 15:44:22 +0000 (11:44 -0400)]
Merge branch '15014-wrong-node-count'

fixes #15014

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '11016-doc-signing-ttl'
Tom Clegg [Thu, 3 Oct 2019 15:39:18 +0000 (11:39 -0400)]
Merge branch '11016-doc-signing-ttl'

refs #11016

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14714-keep-balance-config'
Eric Biagiotti [Thu, 3 Oct 2019 15:09:39 +0000 (11:09 -0400)]
Merge branch '14714-keep-balance-config'

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '15654-wb-gem-update' refs #15654
Peter Amstutz [Thu, 3 Oct 2019 14:48:19 +0000 (10:48 -0400)]
Merge branch '15654-wb-gem-update' refs #15654

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15654: Update rubyzip gem
Peter Amstutz [Wed, 2 Oct 2019 20:27:40 +0000 (16:27 -0400)]
15654: Update rubyzip gem

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15654: Update nokogiri
Peter Amstutz [Mon, 30 Sep 2019 19:46:22 +0000 (15:46 -0400)]
15654: Update nokogiri

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '15678-pkg-version'
Tom Clegg [Wed, 2 Oct 2019 17:09:42 +0000 (13:09 -0400)]
Merge branch '15678-pkg-version'

closes #15678

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15599-keepstore-iam-role'
Tom Clegg [Wed, 2 Oct 2019 15:48:55 +0000 (11:48 -0400)]
Merge branch '15599-keepstore-iam-role'

closes #15599

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15640-cwl-test-mem' refs #15640
Peter Amstutz [Tue, 1 Oct 2019 20:30:52 +0000 (16:30 -0400)]
Merge branch '15640-cwl-test-mem' refs #15640

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14714: Removes metrics serving
Eric Biagiotti [Tue, 1 Oct 2019 20:23:38 +0000 (16:23 -0400)]
14714: Removes metrics serving

- Updates test gathering of metrics
- Simplifies balancer keep service discovery
- Moves keep-balance info from keepstore install docs to the new admin page
- Keepbalance now exits correctly when using -once

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoarvbox pin yarn to 1.17.3 no issue #
Peter Amstutz [Tue, 1 Oct 2019 19:22:58 +0000 (15:22 -0400)]
arvbox pin yarn to 1.17.3 no issue #

Work around for "Incorrect integrity when fetching from the cache"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15599: Reject configuration with both IAMRole and Access/SecretKey.
Tom Clegg [Tue, 1 Oct 2019 18:52:03 +0000 (14:52 -0400)]
15599: Reject configuration with both IAMRole and Access/SecretKey.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Use default/automatic credentials for example config.
Tom Clegg [Tue, 1 Oct 2019 18:47:25 +0000 (14:47 -0400)]
15599: Use default/automatic credentials for example config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoUpdate workbench bundle.
Tom Clegg [Tue, 1 Oct 2019 18:38:21 +0000 (14:38 -0400)]
Update workbench bundle.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15678: Fix version reporting.
Tom Clegg [Tue, 1 Oct 2019 17:46:14 +0000 (13:46 -0400)]
15678: Fix version reporting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15656-user-delete-container'
Tom Clegg [Tue, 1 Oct 2019 14:29:07 +0000 (10:29 -0400)]
Merge branch '15656-user-delete-container'

refs #15656

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15656: Note "delete container" is admin-only and not normally used.
Tom Clegg [Tue, 1 Oct 2019 14:24:02 +0000 (10:24 -0400)]
15656: Note "delete container" is admin-only and not normally used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago11016: Update BlobSigningTTL config docs.
Tom Clegg [Mon, 30 Sep 2019 19:31:55 +0000 (15:31 -0400)]
11016: Update BlobSigningTTL config docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoUpdate login-sync Gemfile.lock.
Tom Clegg [Tue, 1 Oct 2019 03:34:23 +0000 (23:34 -0400)]
Update login-sync Gemfile.lock.

refs #15673

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Improve error message when no role is defined.
Tom Clegg [Mon, 30 Sep 2019 23:22:27 +0000 (19:22 -0400)]
15599: Improve error message when no role is defined.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Warn if multiple roles are assigned.
Tom Clegg [Mon, 30 Sep 2019 23:22:25 +0000 (19:22 -0400)]
15599: Warn if multiple roles are assigned.

This is currently impossible on AWS. If it becomes possible in the
future, this may help with troubleshooting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Look up IAM role name from metadata if not configured.
Tom Clegg [Fri, 27 Sep 2019 21:30:47 +0000 (17:30 -0400)]
15599: Look up IAM role name from metadata if not configured.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Fix misleading error message when address already in use.
Tom Clegg [Fri, 27 Sep 2019 20:40:19 +0000 (16:40 -0400)]
15599: Fix misleading error message when address already in use.

Also warn about unexpected errors encountered while looking for the
current host's entry in InternalURLs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15599: Support IAM role credentials in keepstore.
Tom Clegg [Fri, 27 Sep 2019 20:39:51 +0000 (16:39 -0400)]
15599: Support IAM role credentials in keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15656: Fix missing permission check.
Tom Clegg [Thu, 26 Sep 2019 14:47:33 +0000 (10:47 -0400)]
15656: Fix missing permission check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15014: Update tests.
Tom Clegg [Thu, 26 Sep 2019 14:07:11 +0000 (10:07 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15014: Update tests.
Tom Clegg [Fri, 13 Sep 2019 19:45:28 +0000 (15:45 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15014: Update tests.
Tom Clegg [Fri, 13 Sep 2019 18:48:37 +0000 (14:48 -0400)]
15014: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15014: Hide busy/idle nodes panel when crunch1 is not active.
Tom Clegg [Fri, 13 Sep 2019 17:24:09 +0000 (13:24 -0400)]
15014: Hide busy/idle nodes panel when crunch1 is not active.

The node counts are incorrect when the crunch1 dispatch processes are
not running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '15673-faraday-pin'
Tom Clegg [Mon, 30 Sep 2019 20:44:34 +0000 (16:44 -0400)]
Merge branch '15673-faraday-pin'

refs #15673

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15673: Pin faraday below 0.16.
Tom Clegg [Mon, 30 Sep 2019 19:33:53 +0000 (15:33 -0400)]
15673: Pin faraday below 0.16.

This allows arvados-cli and login-sync to install correctly even if an
older arvados gem (without the faraday pin) is already installed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15673: Pin faraday gem version.
Tom Clegg [Mon, 30 Sep 2019 19:03:06 +0000 (15:03 -0400)]
15673: Pin faraday gem version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15673: Fix ruby warnings.
Tom Clegg [Mon, 30 Sep 2019 14:00:28 +0000 (10:00 -0400)]
15673: Fix ruby warnings.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15673: Show stderr before giving up failed command.
Tom Clegg [Mon, 30 Sep 2019 13:39:25 +0000 (09:39 -0400)]
15673: Show stderr before giving up failed command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoUse default Azure storage endpoint if configured value is "".
Tom Clegg [Mon, 30 Sep 2019 18:53:57 +0000 (14:53 -0400)]
Use default Azure storage endpoint if configured value is "".

This is the documented behavior.

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15640: Adjust memory requirements
Peter Amstutz [Mon, 30 Sep 2019 18:51:00 +0000 (14:51 -0400)]
15640: Adjust memory requirements

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge remote-tracking branch 'origin/master' into 14714-keep-balance-config
Eric Biagiotti [Mon, 30 Sep 2019 16:31:13 +0000 (12:31 -0400)]
Merge remote-tracking branch 'origin/master' into 14714-keep-balance-config

refs #14714

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Simplifies server startup and removes globals
Eric Biagiotti [Mon, 30 Sep 2019 16:26:20 +0000 (12:26 -0400)]
14714: Simplifies server startup and removes globals

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Updates KeepServiceTypes legacy mapping and error messages
Eric Biagiotti [Mon, 30 Sep 2019 16:08:15 +0000 (12:08 -0400)]
14714: Updates KeepServiceTypes legacy mapping and error messages

- Config loading will now fail if the KeepServiceList config option is provided with any value other than "disk", which is equal to the new behavior.
- Changes default BalanceCollectionBatch config to zero.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '13647-keepstore-config'
Tom Clegg [Mon, 30 Sep 2019 15:13:24 +0000 (11:13 -0400)]
Merge branch '13647-keepstore-config'

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14714: Adds missing doc
Eric Biagiotti [Fri, 27 Sep 2019 20:53:11 +0000 (16:53 -0400)]
14714: Adds missing doc

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago15531: Update arv-federation-migrate documentation for LoginCluster
Peter Amstutz [Fri, 27 Sep 2019 20:20:27 +0000 (16:20 -0400)]
15531: Update arv-federation-migrate documentation for LoginCluster

Also take out some mentions of the obsolete application.yml and
reorganize a couple other pages that describe obsolete features.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14714: Updates/adds keep-balance docs
Eric Biagiotti [Fri, 27 Sep 2019 20:00:00 +0000 (16:00 -0400)]
14714: Updates/adds keep-balance docs

Also, a small typo fix in the admin collection versioning page.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13647: Check ARVADOS_* env vars when loading config.
Tom Clegg [Fri, 27 Sep 2019 18:12:22 +0000 (14:12 -0400)]
13647: Check ARVADOS_* env vars when loading config.

Previously they were only checked when starting up a service, so they
weren't noticed by config-dump etc., and couldn't be used by the
keepstore config migration when loading the keep_services endpoint.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15531: Refactor federation-migrate
Peter Amstutz [Fri, 27 Sep 2019 17:59:56 +0000 (13:59 -0400)]
15531: Refactor federation-migrate

Split into functions for readability.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13647: Don't check pending keepstore migrations during API startup.
Tom Clegg [Fri, 27 Sep 2019 17:38:35 +0000 (13:38 -0400)]
13647: Don't check pending keepstore migrations during API startup.

Fixes deadlock: If controller is already up when RailsAPI boots,
RailsAPI runs "config-dump -skip-legacy", which (until this change)
requests .../keep_services; controller forwards the request to
RailsAPI; and passenger queues the request until RailsAPI finishes
booting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15531: Bump API server revision in discovery doc for feature test
Peter Amstutz [Fri, 27 Sep 2019 17:08:46 +0000 (13:08 -0400)]
15531: Bump API server revision in discovery doc for feature test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14714: Uses lib/cmd prometheus registry
Eric Biagiotti [Thu, 26 Sep 2019 19:44:01 +0000 (15:44 -0400)]
14714: Uses lib/cmd prometheus registry

Also switched test logging to use ctxlog TestLogger

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoAvoid nginx startup race in test suite.
Tom Clegg [Thu, 26 Sep 2019 19:27:17 +0000 (15:27 -0400)]
Avoid nginx startup race in test suite.

If run_test_server.py doesn't wait for nginx to start up before
returning, run-tests.sh might check the pidfile, conclude nginx has
died, and abort.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14714: Sets the token on the arvados client
Eric Biagiotti [Thu, 26 Sep 2019 15:22:18 +0000 (11:22 -0400)]
14714: Sets the token on the arvados client

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Removes context from Server functions. Adds sync.Once to setup
Eric Biagiotti [Thu, 26 Sep 2019 14:22:12 +0000 (10:22 -0400)]
14714: Removes context from Server functions. Adds sync.Once to setup

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Tests use cluster config
Eric Biagiotti [Wed, 25 Sep 2019 17:51:07 +0000 (13:51 -0400)]
14714: Tests use cluster config

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: keep-balance uses cluster config
Eric Biagiotti [Tue, 24 Sep 2019 13:58:40 +0000 (09:58 -0400)]
14714: keep-balance uses cluster config

- Removes dumpconfig flag.
- Removes the options to specify a keep service list or type. Will now balance all keep services of type disk reported by the keep_services endpoint.
- Debug flag removed. Uses SystemLogs.LogLevel from cluster config instead.
- Reorganizes the Server struct to use lib/service to do generic service things.

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14714: Adds keep-balance to cluster config loading
Eric Biagiotti [Fri, 6 Sep 2019 18:39:39 +0000 (14:39 -0400)]
14714: Adds keep-balance to cluster config loading

Also adds a deprecated config loading test and fixes the service file

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '13647-keepstore-config'
Tom Clegg [Thu, 26 Sep 2019 15:15:56 +0000 (11:15 -0400)]
Merge branch '13647-keepstore-config'

refs #13647

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13647: Migrate arvbox volume config.
Tom Clegg [Wed, 25 Sep 2019 21:15:11 +0000 (17:15 -0400)]
13647: Migrate arvbox volume config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13647: Set X-External-Client header in arvbox.
Tom Clegg [Wed, 25 Sep 2019 19:29:23 +0000 (15:29 -0400)]
13647: Set X-External-Client header in arvbox.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch 'master' into 15531-logincluster-migrate
Peter Amstutz [Mon, 23 Sep 2019 21:35:07 +0000 (17:35 -0400)]
Merge branch 'master' into 15531-logincluster-migrate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Adjust behavior redirecting remote user to a local one, add test
Peter Amstutz [Mon, 23 Sep 2019 20:52:02 +0000 (16:52 -0400)]
15531: Adjust behavior redirecting remote user to a local one, add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Add README
Peter Amstutz [Mon, 23 Sep 2019 19:24:57 +0000 (15:24 -0400)]
15531: Add README

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Adjust redirect_to_new_user to intended behavior
Peter Amstutz [Mon, 23 Sep 2019 18:39:51 +0000 (14:39 -0400)]
15531: Adjust redirect_to_new_user to intended behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago15531: Fed migrate script passes test cases
Peter Amstutz [Mon, 23 Sep 2019 15:32:03 +0000 (11:32 -0400)]
15531: Fed migrate script passes test cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13647: Add keepstore service entries to arvbox config.
Tom Clegg [Sat, 21 Sep 2019 06:28:37 +0000 (02:28 -0400)]
13647: Add keepstore service entries to arvbox config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13647: Remove duplicate Keepproxy config.
Tom Clegg [Sat, 21 Sep 2019 06:27:34 +0000 (02:27 -0400)]
13647: Remove duplicate Keepproxy config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13647: Fix send-HUP-to-keepproxy command.
Tom Clegg [Sat, 21 Sep 2019 06:26:52 +0000 (02:26 -0400)]
13647: Fix send-HUP-to-keepproxy command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>