arvados.git
3 years ago16267: Merge branch 'master' into 16267-change-arvbox-deps
Ward Vandewege [Mon, 28 Sep 2020 15:58:54 +0000 (11:58 -0400)]
16267: Merge branch 'master' into 16267-change-arvbox-deps

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix more golint warnings.
Ward Vandewege [Mon, 28 Sep 2020 02:16:48 +0000 (22:16 -0400)]
Fix more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix more golint warnings.
Ward Vandewege [Sun, 27 Sep 2020 02:09:06 +0000 (22:09 -0400)]
Fix more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16874-configure-welcome-mail' refs #16874
Peter Amstutz [Fri, 25 Sep 2020 22:17:13 +0000 (18:17 -0400)]
Merge branch '16874-configure-welcome-mail' refs #16874

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16874: Fix test.
Peter Amstutz [Fri, 25 Sep 2020 22:04:29 +0000 (18:04 -0400)]
16874: Fix test.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16267: fix logic bug, use local checked out arvados directory in dev
Ward Vandewege [Fri, 25 Sep 2020 21:31:43 +0000 (17:31 -0400)]
16267: fix logic bug, use local checked out arvados directory in dev
       mode, not the copy under ~/.arvbox

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16874: Content of account_is_setup.text.erb uses Users.UserSetupMailText
Peter Amstutz [Fri, 25 Sep 2020 21:17:06 +0000 (17:17 -0400)]
16874: Content of account_is_setup.text.erb uses Users.UserSetupMailText

The most straightforward solution seems to be for UserSetupMailText to
still be an erb template and is evaluated in the same environment as
as before, but it gets the code from the config file.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16825-delete-token' closes #16825
Peter Amstutz [Fri, 25 Sep 2020 18:55:31 +0000 (14:55 -0400)]
Merge branch '16825-delete-token' closes #16825

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16825: Document deleting token at the CLI
Peter Amstutz [Fri, 25 Sep 2020 15:46:36 +0000 (11:46 -0400)]
16825: Document deleting token at the CLI

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoRecommend installing R package from https refs #16885
Peter Amstutz [Fri, 25 Sep 2020 15:07:02 +0000 (11:07 -0400)]
Recommend installing R package from https refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16267: implement review feedback.
Ward Vandewege [Fri, 25 Sep 2020 00:49:57 +0000 (20:49 -0400)]
16267: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16264-add-r-dependencies' into master
Ward Vandewege [Thu, 24 Sep 2020 21:00:58 +0000 (17:00 -0400)]
Merge branch '16264-add-r-dependencies' into master

closes #16264

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16264: Merge branch 'master' into 16264-add-r-dependencies
Ward Vandewege [Thu, 24 Sep 2020 21:00:25 +0000 (17:00 -0400)]
16264: Merge branch 'master' into 16264-add-r-dependencies

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16827-fix-R-sdk' refs #16827
Peter Amstutz [Thu, 24 Sep 2020 20:58:40 +0000 (16:58 -0400)]
Merge branch '16827-fix-R-sdk' refs #16827

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16884: Add public_project permission link to test fixtures
Peter Amstutz [Thu, 24 Sep 2020 20:01:20 +0000 (16:01 -0400)]
16884: Add public_project permission link to test fixtures

This is the permission link that gives the all users and anonymous
users read access to public favorites.  It is added by database seeds,
but it also needs to be present in the fixtures.

refs #16884
refs #16811

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16827: Don't append '/' to requests with query params. Bump version
Peter Amstutz [Wed, 23 Sep 2020 21:50:14 +0000 (17:50 -0400)]
16827: Don't append '/' to requests with query params.  Bump version

Also remove package-cross loading with "source" that generated warnings.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16827: Update the generated API to Arvados 2.0
Peter Amstutz [Wed, 23 Sep 2020 20:40:55 +0000 (16:40 -0400)]
16827: Update the generated API to Arvados 2.0

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16827: Fix getFileNamesFromResponse
Peter Amstutz [Wed, 23 Sep 2020 20:17:09 +0000 (16:17 -0400)]
16827: Fix getFileNamesFromResponse

Update tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16267: switch to `arvados-server install -type test` for installing
Ward Vandewege [Wed, 23 Sep 2020 23:46:21 +0000 (19:46 -0400)]
16267: switch to `arvados-server install -type test` for installing
       dependencies.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16264: add R dependencies as Debian packages to lib/install/deps.go
Ward Vandewege [Thu, 24 Sep 2020 00:56:12 +0000 (20:56 -0400)]
16264: add R dependencies as Debian packages to lib/install/deps.go

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16850-s3-keycount'
Tom Clegg [Wed, 23 Sep 2020 14:33:52 +0000 (10:33 -0400)]
Merge branch '16850-s3-keycount'

fixes #16850

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agoTweak database seeds so the reset-to-fixture API works
Peter Amstutz [Tue, 22 Sep 2020 21:15:19 +0000 (17:15 -0400)]
Tweak database seeds so the reset-to-fixture API works

refs #16811

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16811-public-favs' refs #16811
Peter Amstutz [Tue, 22 Sep 2020 20:32:54 +0000 (16:32 -0400)]
Merge branch '16811-public-favs' refs #16811

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16811: Make note about public favorites migration
Peter Amstutz [Tue, 22 Sep 2020 20:31:42 +0000 (16:31 -0400)]
16811: Make note about public favorites migration

Also add recommendation to back up database before upgrading.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch 'master' into 16811-public-favs
Peter Amstutz [Tue, 22 Sep 2020 19:57:34 +0000 (15:57 -0400)]
Merge branch 'master' into 16811-public-favs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16859-pdh-version' closes #16859
Peter Amstutz [Tue, 22 Sep 2020 18:30:40 +0000 (14:30 -0400)]
Merge branch '16859-pdh-version' closes #16859

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoTweak text saying which version of Ruby to install & add upgrade note
Peter Amstutz [Tue, 22 Sep 2020 17:20:51 +0000 (13:20 -0400)]
Tweak text saying which version of Ruby to install & add upgrade note

refs #16861

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16856-acr-disable-color' closes #16856
Peter Amstutz [Tue, 22 Sep 2020 17:12:09 +0000 (13:12 -0400)]
Merge branch '16856-acr-disable-color' closes #16856

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoUpdate our docs to state a minimum requirement of Ruby 2.5, since we
Ward Vandewege [Tue, 22 Sep 2020 15:52:35 +0000 (11:52 -0400)]
Update our docs to state a minimum requirement of Ruby 2.5, since we
don't test on 2.3 anymore.

closes #16868
refs #16861

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16791-api-flaky-test-fix'
Lucas Di Pentima [Tue, 22 Sep 2020 12:49:13 +0000 (09:49 -0300)]
Merge branch '16791-api-flaky-test-fix'
Closes #16791

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16859: default include_old_versions: true for GET by PDH or UUID
Peter Amstutz [Mon, 21 Sep 2020 20:59:51 +0000 (16:59 -0400)]
16859: default include_old_versions: true for GET by PDH or UUID

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16831-fix-arvbox-bundler-errors' into master
Ward Vandewege [Mon, 21 Sep 2020 20:30:29 +0000 (16:30 -0400)]
Merge branch '16831-fix-arvbox-bundler-errors' into master

closes #16831

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16831: Merge branch 'master' into 16831-fix-arvbox-bundler-errors
Ward Vandewege [Mon, 21 Sep 2020 20:29:29 +0000 (16:29 -0400)]
16831: Merge branch 'master' into 16831-fix-arvbox-bundler-errors

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16856: Fix tests.
Peter Amstutz [Mon, 21 Sep 2020 20:13:13 +0000 (16:13 -0400)]
16856: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16856: Log the cluster being submitted to
Peter Amstutz [Mon, 21 Sep 2020 18:44:57 +0000 (14:44 -0400)]
16856: Log the cluster being submitted to

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16856: Make spurious msg "Using empty collection" debug log level
Peter Amstutz [Mon, 21 Sep 2020 18:33:17 +0000 (14:33 -0400)]
16856: Make spurious msg "Using empty collection" debug log level

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16856: Pass --disable-color, workbench log doesn't do color codes
Peter Amstutz [Mon, 21 Sep 2020 18:25:58 +0000 (14:25 -0400)]
16856: Pass --disable-color, workbench log doesn't do color codes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix more golint warnings.
Ward Vandewege [Mon, 21 Sep 2020 02:16:52 +0000 (22:16 -0400)]
Fix more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix more golint warnings.
Ward Vandewege [Sat, 19 Sep 2020 14:51:25 +0000 (10:51 -0400)]
Fix more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoRename the ssh_executor package to sshexecutor, fix Golint warning.
Ward Vandewege [Thu, 17 Sep 2020 21:03:01 +0000 (17:03 -0400)]
Rename the ssh_executor package to sshexecutor, fix Golint warning.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16831: arvbox: bugfix: fix bundler version issue in documentation.
Ward Vandewege [Fri, 18 Sep 2020 21:06:41 +0000 (17:06 -0400)]
16831: arvbox: bugfix: fix bundler version issue in documentation.
       Upgrade images to Debian Buster. Clean up whitespace in the
       arvbox command.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16811: Add a test that system users/groups can't be deleted.
Peter Amstutz [Fri, 18 Sep 2020 23:21:09 +0000 (19:21 -0400)]
16811: Add a test that system users/groups can't be deleted.

Also tweak PublicFavoritesProject migration to use up/down instead of
change.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16791: Restores original deadline of 10 seconds on test.
Lucas Di Pentima [Fri, 18 Sep 2020 22:17:04 +0000 (19:17 -0300)]
16791: Restores original deadline of 10 seconds on test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16838-probe-metrics' into master
Ward Vandewege [Fri, 18 Sep 2020 20:36:08 +0000 (16:36 -0400)]
Merge branch '16838-probe-metrics' into master

closes #16838

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16838: Merge branch 'master' into 16838-probe-metrics
Ward Vandewege [Fri, 18 Sep 2020 20:35:31 +0000 (16:35 -0400)]
16838: Merge branch 'master' into 16838-probe-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16791: Fixes audit log cleaning tests.
Lucas Di Pentima [Fri, 18 Sep 2020 20:18:57 +0000 (17:18 -0300)]
16791: Fixes audit log cleaning tests.

Log fixtures lacking the created_at field set in the past in combination
with incorrect testing made us believe the LogTest was working ok.
Specifically, LogTest#test_delete_old_audit_logs_in_thread was not doing
the waiting for the log entries to be eliminated, so there was a race
between the AuditLogs thread cleaning the records and the test teardown doing
the rollback producing the intermittent failure of
use_ownership_and_permission_links_to_determine_which_logs_a_user_can_see.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoFix wb1 bug preventing deletion of container requests & workflows
Peter Amstutz [Fri, 18 Sep 2020 19:09:23 +0000 (15:09 -0400)]
Fix wb1 bug preventing deletion of container requests & workflows

refs #16839

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16838: implement review feedback.
Ward Vandewege [Fri, 18 Sep 2020 18:40:11 +0000 (14:40 -0400)]
16838: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16850: Add KeyCount field to S3 ListObjects response.
Tom Clegg [Fri, 18 Sep 2020 17:45:09 +0000 (13:45 -0400)]
16850: Add KeyCount field to S3 ListObjects response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agoPlayground shell accounts enabled by default refs #16601
Peter Amstutz [Fri, 18 Sep 2020 16:06:49 +0000 (12:06 -0400)]
Playground shell accounts enabled by default refs #16601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix some more golint warnings.
Ward Vandewege [Fri, 18 Sep 2020 00:07:02 +0000 (20:07 -0400)]
Fix some more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16842-s3-timestamp-precision'
Tom Clegg [Thu, 17 Sep 2020 17:31:17 +0000 (13:31 -0400)]
Merge branch '16842-s3-timestamp-precision'

fixes #16842

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16791: Fixes log uuids on fixtures.
Lucas Di Pentima [Thu, 17 Sep 2020 14:46:44 +0000 (11:46 -0300)]
16791: Fixes log uuids on fixtures.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16601-new-tutorial' refs #16601
Peter Amstutz [Thu, 17 Sep 2020 13:57:15 +0000 (09:57 -0400)]
Merge branch '16601-new-tutorial' refs #16601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16842: Truncate timestamp precision in new S3 driver, too.
Tom Clegg [Thu, 17 Sep 2020 13:20:16 +0000 (09:20 -0400)]
16842: Truncate timestamp precision in new S3 driver, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16601: Formatting touch-ups and typo fixes.
Peter Amstutz [Wed, 16 Sep 2020 22:08:15 +0000 (18:08 -0400)]
16601: Formatting touch-ups and typo fixes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16838: a-d-c: metrics: add runProbe success/failure duration metrics.
Ward Vandewege [Wed, 16 Sep 2020 20:53:00 +0000 (16:53 -0400)]
16838: a-d-c: metrics: add runProbe success/failure duration metrics.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16842: Truncate nanoseconds from timestamps in index response.
Tom Clegg [Wed, 16 Sep 2020 18:02:33 +0000 (14:02 -0400)]
16842: Truncate nanoseconds from timestamps in index response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agoMerge branch '16833-replace-epydoc' into master
Ward Vandewege [Wed, 16 Sep 2020 20:06:41 +0000 (16:06 -0400)]
Merge branch '16833-replace-epydoc' into master

closes #16833

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16833: Merge branch 'master' into 16833-replace-epydoc
Ward Vandewege [Wed, 16 Sep 2020 20:00:07 +0000 (16:00 -0400)]
16833: Merge branch 'master' into 16833-replace-epydoc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16833: replace epydoc with pdoc3
Ward Vandewege [Tue, 15 Sep 2020 20:43:12 +0000 (16:43 -0400)]
16833: replace epydoc with pdoc3

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16636-more-metrics' into master
Ward Vandewege [Tue, 15 Sep 2020 19:32:47 +0000 (15:32 -0400)]
Merge branch '16636-more-metrics' into master

closes #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: Merge branch 'master' into 16636-more-metrics
Ward Vandewege [Tue, 15 Sep 2020 19:32:15 +0000 (15:32 -0400)]
16636: Merge branch 'master' into 16636-more-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: fix logic bug in (*Scheduler) updateMetrics()
Ward Vandewege [Tue, 15 Sep 2020 19:30:55 +0000 (15:30 -0400)]
16636: fix logic bug in (*Scheduler) updateMetrics()

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: implement review feedback.
Ward Vandewege [Tue, 15 Sep 2020 17:59:33 +0000 (13:59 -0400)]
16636: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16826-unlogged-attrs-fix'
Lucas Di Pentima [Tue, 15 Sep 2020 14:56:33 +0000 (11:56 -0300)]
Merge branch '16826-unlogged-attrs-fix'
Closes #16826

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16834-stale-run-lock'
Tom Clegg [Tue, 15 Sep 2020 14:49:44 +0000 (10:49 -0400)]
Merge branch '16834-stale-run-lock'

fixes #16834

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16834: Comments about parsing "crunch-run --list" output.
Tom Clegg [Tue, 15 Sep 2020 14:49:13 +0000 (10:49 -0400)]
16834: Comments about parsing "crunch-run --list" output.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16811: Add "Public project" for storing favorites.
Peter Amstutz [Mon, 14 Sep 2020 20:42:07 +0000 (16:42 -0400)]
16811: Add "Public project" for storing favorites.

Workbench and documentation updated for "public favorites" group

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16826: Moves configuration set-up to TestCase's setup() method.
Lucas Di Pentima [Mon, 14 Sep 2020 20:36:43 +0000 (17:36 -0300)]
16826: Moves configuration set-up to TestCase's setup() method.

Having a failing assertion on the teardown() method before the config
restoration code has the side effect of failing all the tests following
the one that made an illegal config change. This updates fixes that.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Fixes config maps assignments, mostly on tests.
Lucas Di Pentima [Mon, 14 Sep 2020 20:23:09 +0000 (17:23 -0300)]
16826: Fixes config maps assignments, mostly on tests.

The new config check brought to the spotlight new tests that are setting
configurations incorrectly.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Enhances teardown config check.
Lucas Di Pentima [Mon, 14 Sep 2020 19:19:59 +0000 (16:19 -0300)]
16826: Enhances teardown config check.

It's not enough to check that .keys() returns a list of Symbols, because
ActiveRecord::OrderedOptions behaves somewhat like HashWithIndifferentAccess
so values can need to be accessed by symbol or string. So, we now check that
every hash-like config section is an OrderedOptions object and also returns
its keys as symbols.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
Ward Vandewege [Mon, 14 Sep 2020 19:14:32 +0000 (15:14 -0400)]
16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
       indicates the current longest wait time between seeing a
       container in the queue or requeueing, and starting its crunch-run
       process on a worker.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16826: Adds catch-all check for any config potentially changed during tests.
Lucas Di Pentima [Mon, 14 Sep 2020 18:42:15 +0000 (15:42 -0300)]
16826: Adds catch-all check for any config potentially changed during tests.

On test suite teardown, check if all config keys are still Symbols, so that
no mistake is made by writing tests that rely on config hashes with string
keys.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16813-avoid-noop-user-updates'
Tom Clegg [Mon, 14 Sep 2020 18:39:48 +0000 (14:39 -0400)]
Merge branch '16813-avoid-noop-user-updates'

fixes #16813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agorun_test_server.py: work around Python bug, appending to a FIFO in
Ward Vandewege [Mon, 14 Sep 2020 17:17:18 +0000 (13:17 -0400)]
run_test_server.py: work around Python bug, appending to a FIFO in
python >=3 <3.8 is broken (https://bugs.python.org/issue27805), more
thorougly this time.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: Merge branch 'master' into 16636-more-metrics
Ward Vandewege [Mon, 14 Sep 2020 17:03:18 +0000 (13:03 -0400)]
16636: Merge branch 'master' into 16636-more-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWork around Python bug, appending to a FIFO in python >=3 <3.8 is broken
Ward Vandewege [Mon, 14 Sep 2020 17:00:45 +0000 (13:00 -0400)]
Work around Python bug, appending to a FIFO in python >=3 <3.8 is broken
(https://bugs.python.org/issue27805).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16578-copy-tutorial' refs #16578
Peter Amstutz [Mon, 14 Sep 2020 15:47:07 +0000 (11:47 -0400)]
Merge branch '16578-copy-tutorial' refs #16578

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
Ward Vandewege [Mon, 14 Sep 2020 13:53:38 +0000 (09:53 -0400)]
16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
       seeing a container in the queue or requeueing, and starting its
       crunch-run process on a worker) across previous starts.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16814-remove-python2-arvbox' into master
Nico Cesar [Mon, 14 Sep 2020 13:37:44 +0000 (09:37 -0400)]
Merge branch '16814-remove-python2-arvbox' into master

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoimplementing https://dev.arvados.org/issues/16814#note-7
Nico Cesar [Mon, 14 Sep 2020 13:35:50 +0000 (09:35 -0400)]
implementing https://dev.arvados.org/issues/16814#note-7

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago16834: Drain instance if crunch-run exits but arv-mount doesn't.
Tom Clegg [Sat, 12 Sep 2020 19:51:25 +0000 (15:51 -0400)]
16834: Drain instance if crunch-run exits but arv-mount doesn't.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16826: Fixes config setting on tests about config maps knobs.
Lucas Di Pentima [Fri, 11 Sep 2020 21:59:54 +0000 (18:59 -0300)]
16826: Fixes config setting on tests about config maps knobs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Fixes the bug.
Lucas Di Pentima [Fri, 11 Sep 2020 21:44:20 +0000 (18:44 -0300)]
16826: Fixes the bug.

The issue is that the config loading code on Rails uses
ActiveSupport::OrderedOptions on config hashes, and this class always
returns keys as symbols. We were incorrectly testing the feature by
assigning a normal Hash to the config.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Adds test exposing the bug.
Lucas Di Pentima [Fri, 11 Sep 2020 21:42:04 +0000 (18:42 -0300)]
16826: Adds test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16636: add instances_time_from_shutdown_request_to_disappearance_seconds
Ward Vandewege [Fri, 11 Sep 2020 16:30:07 +0000 (12:30 -0400)]
16636: add instances_time_from_shutdown_request_to_disappearance_seconds
       metric

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16811: Make public favorites work
Peter Amstutz [Thu, 10 Sep 2020 21:41:54 +0000 (17:41 -0400)]
16811: Make public favorites work

Update API docs.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoBump gem versions in doc/Gemfile.lock, cf dependabot.
Ward Vandewege [Thu, 10 Sep 2020 20:51:35 +0000 (16:51 -0400)]
Bump gem versions in doc/Gemfile.lock, cf dependabot.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16813: Avoid writing db when nothing is changing in a batch update.
Tom Clegg [Thu, 10 Sep 2020 20:33:31 +0000 (16:33 -0400)]
16813: Avoid writing db when nothing is changing in a batch update.

Avoids superfluous log entries and potentially costly permission graph
updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agoMerge branch '15888-remove-more-python2-remnants'
Ward Vandewege [Thu, 10 Sep 2020 20:26:53 +0000 (16:26 -0400)]
Merge branch '15888-remove-more-python2-remnants'

refs #15888

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago15888: Merge branch 'master' into 15888-remove-more-python2-remnants
Ward Vandewege [Thu, 10 Sep 2020 20:26:21 +0000 (16:26 -0400)]
15888: Merge branch 'master' into 15888-remove-more-python2-remnants

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfix: make sure that any fpm_args defined in fpm-info.sh get
Ward Vandewege [Thu, 10 Sep 2020 20:18:12 +0000 (16:18 -0400)]
Bugfix: make sure that any fpm_args defined in fpm-info.sh get
introduced at the correct position in the fpm argument list. This only
affects non-Python packages (we already did the right thing for Python
packages).

refs #15348

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoremove python2 packages from Dockerfile.base in arvbox
Nico Cesar [Thu, 10 Sep 2020 19:33:55 +0000 (15:33 -0400)]
remove python2 packages from Dockerfile.base in arvbox

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoMerge branch '16808-add-strict-mode'
Ward Vandewege [Thu, 10 Sep 2020 19:06:34 +0000 (15:06 -0400)]
Merge branch '16808-add-strict-mode'

closes #16808

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16808: Merge branch 'master' into 16808-add-strict-mode
Ward Vandewege [Thu, 10 Sep 2020 19:06:00 +0000 (15:06 -0400)]
16808: Merge branch 'master' into 16808-add-strict-mode

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago15888: update `arvados-server install -type test` for the new
Ward Vandewege [Thu, 10 Sep 2020 18:58:26 +0000 (14:58 -0400)]
15888: update `arvados-server install -type test` for the new
       python3-only world in run-tests.sh

We're still pulling in all of python2 because of the dependency on the
linkchecker and python-epydoc, though.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16636-container-allocation-metrics'
Ward Vandewege [Thu, 10 Sep 2020 18:52:49 +0000 (14:52 -0400)]
Merge branch '16636-container-allocation-metrics'

refs #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16808: move strict flag to the config-check command only
Ward Vandewege [Thu, 10 Sep 2020 18:08:48 +0000 (14:08 -0400)]
16808: move strict flag to the config-check command only

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16578: Add an arv-copy integration test
Peter Amstutz [Thu, 10 Sep 2020 18:00:59 +0000 (14:00 -0400)]
16578: Add an arv-copy integration test

To avoid complexity of starting two Arvados instances, copies a
subproject container a collection from one parent to another on the
same cluster.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>