arvados.git
3 years ago17464: Don't deny or log GET on directories.
Peter Amstutz [Thu, 17 Jun 2021 20:47:33 +0000 (16:47 -0400)]
17464: Don't deny or log GET on directories.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add tests for paths by /users/ and by PDH
Peter Amstutz [Thu, 17 Jun 2021 19:49:04 +0000 (15:49 -0400)]
17464: Add tests for paths by /users/ and by PDH

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Clean up tests
Peter Amstutz [Thu, 17 Jun 2021 18:48:33 +0000 (14:48 -0400)]
17464: Clean up tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Update config variables & docs from feedback
Peter Amstutz [Thu, 17 Jun 2021 16:27:19 +0000 (12:27 -0400)]
17464: Update config variables & docs from feedback

Change KeepWebPermission -> WebDAVPermission

Add WebDAVLogEvents

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Activity report lists upload/download events
Peter Amstutz [Wed, 16 Jun 2021 21:22:04 +0000 (17:22 -0400)]
17464: Activity report lists upload/download events

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix test.
Peter Amstutz [Wed, 16 Jun 2021 20:06:36 +0000 (16:06 -0400)]
17464: Fix test.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add upload/download logging and permissions to keepproxy
Peter Amstutz [Wed, 16 Jun 2021 20:03:48 +0000 (16:03 -0400)]
17464: Add upload/download logging and permissions to keepproxy

Add tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Refactor tests and check that log events are posted
Peter Amstutz [Tue, 15 Jun 2021 20:59:40 +0000 (16:59 -0400)]
17464: Refactor tests and check that log events are posted

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Reset database after upload tests
Peter Amstutz [Tue, 15 Jun 2021 18:54:16 +0000 (14:54 -0400)]
17464: Reset database after upload tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Permission/logging testing WIP
Peter Amstutz [Mon, 14 Jun 2021 21:30:59 +0000 (17:30 -0400)]
17464: Permission/logging testing WIP

The upload tests are messing up the other tests by changing the
contents of the collection, still need to fix it.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix shadowed "sess" variable
Peter Amstutz [Fri, 11 Jun 2021 21:19:36 +0000 (17:19 -0400)]
17464: Fix shadowed "sess" variable

Testing WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Log collection uuid for FileSystem requests
Peter Amstutz [Fri, 11 Jun 2021 19:26:05 +0000 (15:26 -0400)]
17464: Log collection uuid for FileSystem requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add upload/download permission checks and logging
Peter Amstutz [Thu, 10 Jun 2021 20:50:56 +0000 (16:50 -0400)]
17464: Add upload/download permission checks and logging

Adds extra log line to normal logging that includes user uuid and full
name.

Also posts an event to the logs table.

Adds permission checks to but these haven't been tested yet.

Still need to add testing.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix config entry
Peter Amstutz [Mon, 7 Jun 2021 19:42:32 +0000 (15:42 -0400)]
17464: Fix config entry

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Start by writing the documentation page
Peter Amstutz [Mon, 7 Jun 2021 19:23:40 +0000 (15:23 -0400)]
17464: Start by writing the documentation page

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix auto-enable DebugLocksPanicMode.
Tom Clegg [Fri, 18 Jun 2021 15:28:08 +0000 (11:28 -0400)]
17464: Fix auto-enable DebugLocksPanicMode.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17609-diagnostics-cmd'
Tom Clegg [Fri, 18 Jun 2021 14:58:36 +0000 (10:58 -0400)]
Merge branch '17609-diagnostics-cmd'

closes #17609

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17464: Add DebugLocksPanicMode and fix some missing locks.
Tom Clegg [Fri, 18 Jun 2021 14:11:27 +0000 (10:11 -0400)]
17464: Add DebugLocksPanicMode and fix some missing locks.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17800-arvput-no-follow-links'
Lucas Di Pentima [Fri, 18 Jun 2021 11:03:15 +0000 (08:03 -0300)]
Merge branch '17800-arvput-no-follow-links'
Closes #17800

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17810-s3-escape-non-unreserved-chars'
Tom Clegg [Fri, 18 Jun 2021 02:59:00 +0000 (22:59 -0400)]
Merge branch '17810-s3-escape-non-unreserved-chars'

fixes #17810

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoThe bleeding edge branch in the arvados-formula repository is now called
Ward Vandewege [Thu, 17 Jun 2021 18:42:12 +0000 (14:42 -0400)]
The bleeding edge branch in the arvados-formula repository is now called
'main'.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17803-config-key-case-warning'
Tom Clegg [Wed, 16 Jun 2021 13:45:00 +0000 (09:45 -0400)]
Merge branch '17803-config-key-case-warning'

fixes #17803

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17803: Note todo's.
Tom Clegg [Tue, 15 Jun 2021 22:31:08 +0000 (18:31 -0400)]
17803: Note todo's.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17803: Fix unnecessarily convoluted test case.
Tom Clegg [Tue, 15 Jun 2021 22:22:51 +0000 (18:22 -0400)]
17803: Fix unnecessarily convoluted test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17810: Fix S3 signature verification.
Tom Clegg [Tue, 15 Jun 2021 18:57:59 +0000 (14:57 -0400)]
17810: Fix S3 signature verification.

All chars (other than "/") that are not unreserved must be
percent-encoded.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17800: Avoids saving a new empty collection.
Lucas Di Pentima [Mon, 14 Jun 2021 19:49:42 +0000 (16:49 -0300)]
17800: Avoids saving a new empty collection.

Sometimes the specified paths won't be eligible for uploading (ie: all are
symlinks and --no-follow-symlinks is used), in those cases arv-put just
prints a warning message and doesn't save anything to Keep/API.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17803: Ensure keys with mismatched case don't get used.
Tom Clegg [Mon, 14 Jun 2021 18:28:00 +0000 (14:28 -0400)]
17803: Ensure keys with mismatched case don't get used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17800: Improves help message. Shows warnings when ignoring given paths.
Lucas Di Pentima [Mon, 14 Jun 2021 17:26:21 +0000 (14:26 -0300)]
17800: Improves help message. Shows warnings when ignoring given paths.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17800: Fixes bug.
Lucas Di Pentima [Mon, 14 Jun 2021 14:01:36 +0000 (11:01 -0300)]
17800: Fixes bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17800: Exposes bug on test.
Lucas Di Pentima [Mon, 14 Jun 2021 13:38:14 +0000 (10:38 -0300)]
17800: Exposes bug on test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17803: Warn when config keys are ignored due to case mismatch.
Tom Clegg [Sun, 13 Jun 2021 14:00:45 +0000 (10:00 -0400)]
17803: Warn when config keys are ignored due to case mismatch.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Make costanalyzer logging a bit less verbose.
Tom Clegg [Fri, 11 Jun 2021 14:18:19 +0000 (10:18 -0400)]
17609: Make costanalyzer logging a bit less verbose.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Merge branch 'master'
Tom Clegg [Fri, 11 Jun 2021 14:02:58 +0000 (10:02 -0400)]
17609: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17351-arvput-keepclient-storage-support'
Lucas Di Pentima [Thu, 10 Jun 2021 21:54:58 +0000 (18:54 -0300)]
Merge branch '17351-arvput-keepclient-storage-support'
Closes #17351

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17609: Remove extra newlines.
Tom Clegg [Thu, 10 Jun 2021 20:06:22 +0000 (16:06 -0400)]
17609: Remove extra newlines.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Remove duplicate NoPrefixFormatter.
Tom Clegg [Thu, 10 Jun 2021 19:57:44 +0000 (15:57 -0400)]
17609: Remove duplicate NoPrefixFormatter.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17774-fix-cancel-cr'
Tom Clegg [Thu, 10 Jun 2021 19:09:46 +0000 (15:09 -0400)]
Merge branch '17774-fix-cancel-cr'

fixes #17774

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17774: Add non-empty mounts to test fixtures.
Tom Clegg [Thu, 10 Jun 2021 18:46:38 +0000 (14:46 -0400)]
17774: Add non-empty mounts to test fixtures.

Confirmed wb1 integration tests fail if the recent fix (parent of this
commit) is backed out.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17799-rails-upgrade-5.2.6'
Lucas Di Pentima [Thu, 10 Jun 2021 15:06:53 +0000 (12:06 -0300)]
Merge branch '17799-rails-upgrade-5.2.6'
Closes #17799

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17774: Test runtime_constraints and scheduling_parameters, too.
Tom Clegg [Thu, 10 Jun 2021 15:05:03 +0000 (11:05 -0400)]
17774: Test runtime_constraints and scheduling_parameters, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17774: Ignore addition of zero/default values to mounts when frozen.
Tom Clegg [Thu, 10 Jun 2021 14:42:30 +0000 (10:42 -0400)]
17774: Ignore addition of zero/default values to mounts when frozen.

Avoids errors when a client reads the CR and includes the resulting
mounts field (to which controller will have added zero/default values)
in a subsequent update request.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17799: Upgrades API & Workbench rails to 5.2.6
Lucas Di Pentima [Wed, 9 Jun 2021 23:26:20 +0000 (20:26 -0300)]
17799: Upgrades API & Workbench rails to 5.2.6

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17609: Tweak messages.
Tom Clegg [Tue, 8 Jun 2021 06:08:43 +0000 (02:08 -0400)]
17609: Tweak messages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Behave better when project creation fails.
Tom Clegg [Tue, 8 Jun 2021 05:53:12 +0000 (01:53 -0400)]
17609: Behave better when project creation fails.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Improve output formatting.
Tom Clegg [Tue, 8 Jun 2021 05:45:55 +0000 (01:45 -0400)]
17609: Improve output formatting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Add "run container" test.
Tom Clegg [Tue, 8 Jun 2021 05:45:40 +0000 (01:45 -0400)]
17609: Add "run container" test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Improve log text alignment.
Tom Clegg [Wed, 2 Jun 2021 19:23:56 +0000 (15:23 -0400)]
17609: Improve log text alignment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Use timeout on all http requests.
Tom Clegg [Wed, 2 Jun 2021 19:17:13 +0000 (15:17 -0400)]
17609: Use timeout on all http requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Fix limit args to work with older API.
Tom Clegg [Wed, 2 Jun 2021 19:12:58 +0000 (15:12 -0400)]
17609: Fix limit args to work with older API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Refactor tests to add some structure.
Tom Clegg [Tue, 1 Jun 2021 20:23:08 +0000 (16:23 -0400)]
17609: Refactor tests to add some structure.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Add webshell test.
Tom Clegg [Tue, 1 Jun 2021 15:28:54 +0000 (11:28 -0400)]
17609: Add webshell test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17609: Add diagnostics command.
Tom Clegg [Mon, 31 May 2021 20:52:36 +0000 (16:52 -0400)]
17609: Add diagnostics command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17717-costanalyzer-date-mode'
Ward Vandewege [Tue, 8 Jun 2021 14:52:47 +0000 (10:52 -0400)]
Merge branch '17717-costanalyzer-date-mode'

closes #17717

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17717: Merge branch 'master' into 17717-costanalyzer-date-mode
Ward Vandewege [Tue, 8 Jun 2021 14:49:38 +0000 (10:49 -0400)]
17717: Merge branch 'master' into 17717-costanalyzer-date-mode

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17717: more changes post review: remove unneeded debug print, switch
Ward Vandewege [Tue, 8 Jun 2021 14:47:31 +0000 (10:47 -0400)]
17717: more changes post review: remove unneeded debug print, switch
       Command to value receiver.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoRemove leftover dead link from the ToC in the 'Set up a shell node'
Ward Vandewege [Tue, 8 Jun 2021 12:55:53 +0000 (08:55 -0400)]
Remove leftover dead link from the ToC in the 'Set up a shell node'
page.

refs #16803

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17706-costanalyzer-uncommitted-container-requests'
Ward Vandewege [Tue, 8 Jun 2021 12:27:54 +0000 (08:27 -0400)]
Merge branch '17706-costanalyzer-uncommitted-container-requests'

closes #17706

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17706: Merge branch 'master' into 17706-costanalyzer-uncommitted-container-requests
Ward Vandewege [Tue, 8 Jun 2021 12:27:25 +0000 (08:27 -0400)]
17706: Merge branch 'master' into 17706-costanalyzer-uncommitted-container-requests

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17717: address further review comments: remove unused code and refactor
Ward Vandewege [Mon, 7 Jun 2021 22:00:46 +0000 (18:00 -0400)]
17717: address further review comments: remove unused code and refactor
       a bit.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17717: address review comments.
Ward Vandewege [Mon, 7 Jun 2021 18:01:10 +0000 (14:01 -0400)]
17717: address review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17465-pysdk-storage-classes-support'
Lucas Di Pentima [Mon, 7 Jun 2021 17:17:37 +0000 (14:17 -0300)]
Merge branch '17465-pysdk-storage-classes-support'
Closes #17465

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17351: Adds support for storage classes while writing blocks to Keep.
Lucas Di Pentima [Thu, 3 Jun 2021 00:23:24 +0000 (21:23 -0300)]
17351: Adds support for storage classes while writing blocks to Keep.

Also, enabled support for multiple storage classes. For some reason it was
limited to just one.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17465: Fixed tests and made the suite less noisy.
Lucas Di Pentima [Mon, 7 Jun 2021 14:02:22 +0000 (11:02 -0300)]
17465: Fixed tests and made the suite less noisy.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoTweak upgrading doc page to reflect the release of 2.2.0.
Ward Vandewege [Sat, 5 Jun 2021 12:12:06 +0000 (08:12 -0400)]
Tweak upgrading doc page to reflect the release of 2.2.0.

refs #17512

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17757-bootprobe-race'
Ward Vandewege [Fri, 4 Jun 2021 21:31:01 +0000 (17:31 -0400)]
Merge branch '17757-bootprobe-race'

closes #17757

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17757: Merge branch 'master' into 17757-bootprobe-race
Ward Vandewege [Fri, 4 Jun 2021 21:30:32 +0000 (17:30 -0400)]
17757: Merge branch 'master' into 17757-bootprobe-race

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17762: Add uuid-dev package
Peter Amstutz [Fri, 4 Jun 2021 19:29:05 +0000 (15:29 -0400)]
17762: Add uuid-dev package

refs #17762

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17757: On our compute node images, Docker should not start by default;
Ward Vandewege [Fri, 4 Jun 2021 15:22:47 +0000 (11:22 -0400)]
17757: On our compute node images, Docker should not start by default;
       ensure-encrypted-partitions.sh takes care of starting it.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17465: Don't repeat warnings when storage classes aren't supported.
Lucas Di Pentima [Thu, 3 Jun 2021 22:34:26 +0000 (19:34 -0300)]
17465: Don't repeat warnings when storage classes aren't supported.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoFix building of metapackages for Ubuntu 18.04 and Ubuntu 20.04.
Ward Vandewege [Thu, 3 Jun 2021 20:18:36 +0000 (16:18 -0400)]
Fix building of metapackages for Ubuntu 18.04 and Ubuntu 20.04.

refs #17498

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoTweak version-at-commit.sh for interim releases (X.Y.Z.A)
Peter Amstutz [Thu, 3 Jun 2021 16:03:02 +0000 (12:03 -0400)]
Tweak version-at-commit.sh for interim releases (X.Y.Z.A)

Assign correct intended dev version when there is a X.Y.Z.A tag (not
just X.Y.Z) in the history.

As of this writing we don't have tags like that in the arvados repo,
but we do have tags like that in the workbench2 repo, so this commit
is just about synchronizing the script for consistency between the two
repos.

refs #17512

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoRemove duplicate rm.
Tom Clegg [Thu, 3 Jun 2021 13:41:50 +0000 (09:41 -0400)]
Remove duplicate rm.

Fixes #17752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMake our arvados-api and arvados-workbench Gemfiles use the arvados
Ward Vandewege [Wed, 2 Jun 2021 21:28:41 +0000 (17:28 -0400)]
Make our arvados-api and arvados-workbench Gemfiles use the arvados
2.1.5 gem.

refs #17738

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17465: Collection.save() adds storage_classes_desired when set on instance.
Lucas Di Pentima [Wed, 2 Jun 2021 20:24:29 +0000 (17:24 -0300)]
17465: Collection.save() adds storage_classes_desired when set on instance.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17465: Synchronizes storage classes data between Collection & BlockManager.
Lucas Di Pentima [Wed, 2 Jun 2021 19:54:39 +0000 (16:54 -0300)]
17465: Synchronizes storage classes data between Collection & BlockManager.

Storage classes can be set at Collection instantiation time, and BlockManager
used to get that data when being instantiated by Collection, but desired
storage classes can change in the middle of a Collection instance lifetime,
and new blocks should get written on the correct classes, so the solution is
to pass a function for BlockManager to be able to query its parent Collection
instance settings at any time.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoNeed to export ARVADOS_BUILDING_ so arvados_version.py sees them.
Peter Amstutz [Wed, 2 Jun 2021 18:50:16 +0000 (14:50 -0400)]
Need to export ARVADOS_BUILDING_ so arvados_version.py sees them.

refs #17512

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17296-singularity'
Tom Clegg [Wed, 2 Jun 2021 17:49:03 +0000 (13:49 -0400)]
Merge branch '17296-singularity'

closes #17296

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17296: Merge branch 'master'
Tom Clegg [Wed, 2 Jun 2021 15:21:18 +0000 (11:21 -0400)]
17296: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17738-pin-gemfiles-to-2.1.4'
Ward Vandewege [Tue, 1 Jun 2021 21:03:05 +0000 (17:03 -0400)]
Merge branch '17738-pin-gemfiles-to-2.1.4'

closes #17738

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17738: Make our arvados-api and arvados-workbench Gemfiles use the arvados
Ward Vandewege [Tue, 1 Jun 2021 19:00:15 +0000 (15:00 -0400)]
17738: Make our arvados-api and arvados-workbench Gemfiles use the arvados
2.1.4 gem.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17738-remove-constant-modification'
Ward Vandewege [Tue, 1 Jun 2021 15:26:57 +0000 (11:26 -0400)]
Merge branch '17738-remove-constant-modification'

refs #17738

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17738: remove the monkey patch of the OS_VERSION constant from the
Ward Vandewege [Tue, 1 Jun 2021 14:55:09 +0000 (10:55 -0400)]
17738: remove the monkey patch of the OS_VERSION constant from the
       google-api-client gem; we run from a forked version of the
       gem which already has the necessary fix.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17465: Adds tests confirming KeepClient storage support on Collection.
Lucas Di Pentima [Tue, 1 Jun 2021 01:02:11 +0000 (22:02 -0300)]
17465: Adds tests confirming KeepClient storage support on Collection.

Also, made storage_classes_desired field be loaded into a Collection from the
API server.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17465: Adds KeepClient storage classes support to Collection class.
Lucas Di Pentima [Mon, 31 May 2021 23:36:00 +0000 (20:36 -0300)]
17465: Adds KeepClient storage classes support to Collection class.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17465: Adds tests for class storage support.
Lucas Di Pentima [Mon, 31 May 2021 20:05:57 +0000 (17:05 -0300)]
17465: Adds tests for class storage support.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17743-use-git-arvados-org-for-provision'
Javier Bértoli [Fri, 28 May 2021 22:46:04 +0000 (19:46 -0300)]
Merge branch '17743-use-git-arvados-org-for-provision'

closes #17743
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofeat(provision): moved the arvados-formula to Arvados.org
Javier Bértoli [Thu, 20 May 2021 19:11:52 +0000 (16:11 -0300)]
feat(provision): moved the arvados-formula to Arvados.org

refs #17655
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '17603-nginx-webshell-vhost-fix'
Javier Bértoli [Fri, 28 May 2021 21:21:50 +0000 (18:21 -0300)]
Merge branch '17603-nginx-webshell-vhost-fix'

closes #17603
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '17722-adjust-token-expiry' refs #17722
Peter Amstutz [Fri, 28 May 2021 20:43:13 +0000 (16:43 -0400)]
Merge branch '17722-adjust-token-expiry' refs #17722

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17722: Tweak tests to be clearer about new behavior
Peter Amstutz [Fri, 28 May 2021 20:40:49 +0000 (16:40 -0400)]
17722: Tweak tests to be clearer about new behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17605-letsencrypt-credentials-for-route53'
Javier Bértoli [Fri, 28 May 2021 20:19:22 +0000 (17:19 -0300)]
Merge branch '17605-letsencrypt-credentials-for-route53'

closes #17605
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17722: Update tests for adjusted MaxTokenLifetime behavior
Peter Amstutz [Fri, 28 May 2021 19:26:03 +0000 (15:26 -0400)]
17722: Update tests for adjusted MaxTokenLifetime behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17722: Adjust token expiry behavior
Peter Amstutz [Tue, 25 May 2021 20:45:19 +0000 (16:45 -0400)]
17722: Adjust token expiry behavior

* Admins tokens have API.MaxTokenLifetime by default, but admins can create
  token which are not restricted by API.MaxTokenLifetime.

* Login.TokenLifetime does no longer implies untrusted tokens

* Added Login.TrustLoginToken to determine whether tokens issued by login are
  considered "trusted"

* Updated documentation page

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17703-provision-version' refs #17703
Peter Amstutz [Fri, 28 May 2021 18:21:05 +0000 (14:21 -0400)]
Merge branch '17703-provision-version' refs #17703

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17703: Be more explicit about package version.
Peter Amstutz [Fri, 28 May 2021 18:20:25 +0000 (14:20 -0400)]
17703: Be more explicit about package version.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoWhen building compute node images, print the version of packer used.
Ward Vandewege [Fri, 28 May 2021 18:11:22 +0000 (14:11 -0400)]
When building compute node images, print the version of packer used.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17465: Adds storage classes tracking and old cluster support.
Lucas Di Pentima [Thu, 27 May 2021 21:38:30 +0000 (18:38 -0300)]
17465: Adds storage classes tracking and old cluster support.

* Tracks which classes were confirmed and ask for the remaining on following
  retries.
* When receiving a keepstore response without the proper header, it fallbacks
  to only saving the requested number of copies.
* Updates tests to reflect code changes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoRemove old Go installation when upgrading.
Tom Clegg [Thu, 27 May 2021 05:23:18 +0000 (01:23 -0400)]
Remove old Go installation when upgrading.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17738-frozen-string-error'
Lucas Di Pentima [Thu, 27 May 2021 15:51:42 +0000 (12:51 -0300)]
Merge branch '17738-frozen-string-error'
Closes #17738

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17738: Fixes the issue by strip!()ing a mutable version of OS_VERSION.
Lucas Di Pentima [Thu, 27 May 2021 14:28:50 +0000 (11:28 -0300)]
17738: Fixes the issue by strip!()ing a mutable version of OS_VERSION.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>