arvados.git
4 years ago16564: Upgrades rack to address CVE-2020-8184.
Lucas Di Pentima [Thu, 25 Jun 2020 13:36:01 +0000 (10:36 -0300)]
16564: Upgrades rack to address CVE-2020-8184.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoRestore link to fix test refs #16007
Peter Amstutz [Mon, 22 Jun 2020 20:06:22 +0000 (16:06 -0400)]
Restore link to fix test refs #16007

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMake it clear that the API token is used by all Arvados command line
Ward Vandewege [Mon, 22 Jun 2020 16:08:24 +0000 (12:08 -0400)]
Make it clear that the API token is used by all Arvados command line
tools, not just the 'arv' tool.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16480-keep-balance-index-timeout'
Tom Clegg [Mon, 22 Jun 2020 15:36:11 +0000 (11:36 -0400)]
Merge branch '16480-keep-balance-index-timeout'

fixes #16480

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16480: Merge branch 'master'
Tom Clegg [Mon, 22 Jun 2020 15:35:35 +0000 (11:35 -0400)]
16480: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoAdd links to the Arvados forum refs #16521
Peter Amstutz [Fri, 19 Jun 2020 21:20:59 +0000 (17:20 -0400)]
Add links to the Arvados forum refs #16521

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16007-validate-group-class' refs #16007
Peter Amstutz [Fri, 19 Jun 2020 15:38:18 +0000 (11:38 -0400)]
Merge branch '16007-validate-group-class' refs #16007

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Fix typo
Peter Amstutz [Fri, 19 Jun 2020 15:37:57 +0000 (11:37 -0400)]
16007: Fix typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16513-keep-exercise-improvements'
Ward Vandewege [Fri, 19 Jun 2020 15:37:42 +0000 (11:37 -0400)]
Merge branch '16513-keep-exercise-improvements'

refs #16513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16513: Merge branch 'master' into 16513-keep-exercise-improvements
Ward Vandewege [Fri, 19 Jun 2020 15:36:55 +0000 (11:36 -0400)]
16513: Merge branch 'master' into 16513-keep-exercise-improvements

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16526-ruby-and-python-build-script-updates'
Ward Vandewege [Fri, 19 Jun 2020 15:33:20 +0000 (11:33 -0400)]
Merge branch '16526-ruby-and-python-build-script-updates'

closes #16526

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16526: Merge branch 'master' into 16526-ruby-and-python-build-script-updates
Ward Vandewege [Fri, 19 Jun 2020 15:31:54 +0000 (11:31 -0400)]
16526: Merge branch 'master' into 16526-ruby-and-python-build-script-updates

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16480: Configurable timeout for entire keep-balance iteration.
Tom Clegg [Fri, 19 Jun 2020 14:34:01 +0000 (10:34 -0400)]
16480: Configurable timeout for entire keep-balance iteration.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: note about sharing with anonymous users, and renaming roles
Peter Amstutz [Thu, 18 Jun 2020 16:15:57 +0000 (12:15 -0400)]
16007: note about sharing with anonymous users, and renaming roles

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Doc updates/clarifications from feedback
Peter Amstutz [Thu, 18 Jun 2020 16:04:46 +0000 (12:04 -0400)]
16007: Doc updates/clarifications from feedback

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16482-bump-cwltool-version'
Ward Vandewege [Thu, 18 Jun 2020 15:53:41 +0000 (11:53 -0400)]
Merge branch '16482-bump-cwltool-version'

closes #16482

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16482: Merge branch 'master' into 16482-bump-cwltool-version
Ward Vandewege [Thu, 18 Jun 2020 15:52:23 +0000 (11:52 -0400)]
16482: Merge branch 'master' into 16482-bump-cwltool-version

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16007: Update docs, restore empty_collection, fix tests
Peter Amstutz [Wed, 17 Jun 2020 21:34:34 +0000 (17:34 -0400)]
16007: Update docs, restore empty_collection, fix tests

Also include link uuid in migration log message

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Migration works with large database
Peter Amstutz [Mon, 8 Jun 2020 14:43:47 +0000 (10:43 -0400)]
16007: Migration works with large database

Changing individual permissions on groups that affect a lot of objects
is expensive.  The migration now suppresses permission updates and
does a batch update at the end.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Fix FUSE test
Peter Amstutz [Tue, 2 Jun 2020 20:49:38 +0000 (16:49 -0400)]
16007: Fix FUSE test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Add migration to fix invalid groups & permission links
Peter Amstutz [Tue, 2 Jun 2020 20:45:21 +0000 (16:45 -0400)]
16007: Add migration to fix invalid groups & permission links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Reenable updating the permission level of a link
Peter Amstutz [Tue, 2 Jun 2020 17:26:48 +0000 (13:26 -0400)]
16007: Reenable updating the permission level of a link

Add test to ensure permission table is synchronized when updating the
level of an existing permission link.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Update group-sync tool for new restrictions on roles
Peter Amstutz [Tue, 2 Jun 2020 14:09:29 +0000 (10:09 -0400)]
16007: Update group-sync tool for new restrictions on roles

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Roles are owned by system user
Peter Amstutz [Mon, 1 Jun 2020 22:03:56 +0000 (18:03 -0400)]
16007: Roles are owned by system user

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Only users and roles can be granted permission
Peter Amstutz [Mon, 1 Jun 2020 18:59:51 +0000 (14:59 -0400)]
16007: Only users and roles can be granted permission

Permission link tail_uuid must be a user or group_class role.

Also disallow modifying permission links.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Make it so that only projects can own things
Peter Amstutz [Fri, 29 May 2020 21:25:05 +0000 (17:25 -0400)]
16007: Make it so that only projects can own things

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Validate group_class is set to 'project' or 'role'
Peter Amstutz [Fri, 29 May 2020 04:04:12 +0000 (00:04 -0400)]
16007: Validate group_class is set to 'project' or 'role'

Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16513: switch to CSV output only, send CSV to stdout and logs to stderr,
Ward Vandewege [Wed, 17 Jun 2020 16:31:39 +0000 (12:31 -0400)]
16513: switch to CSV output only, send CSV to stdout and logs to stderr,
       abort on ctrl-c, print final CSV line when ending.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16171-oidc-config'
Tom Clegg [Wed, 17 Jun 2020 16:35:32 +0000 (12:35 -0400)]
Merge branch '16171-oidc-config'

closes #16171

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Link install docs to additional OpenIDConnect configs.
Tom Clegg [Wed, 17 Jun 2020 16:34:41 +0000 (12:34 -0400)]
16171: Link install docs to additional OpenIDConnect configs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Fix typo.
Tom Clegg [Wed, 17 Jun 2020 14:24:38 +0000 (10:24 -0400)]
16171: Fix typo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16007-permission-table-rb' refs #16007
Peter Amstutz [Tue, 16 Jun 2020 21:25:38 +0000 (17:25 -0400)]
Merge branch '16007-permission-table-rb' refs #16007

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Update comments to discuss edge_id
Peter Amstutz [Tue, 16 Jun 2020 19:52:43 +0000 (15:52 -0400)]
16007: Update comments to discuss edge_id

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16480: Use longer timeout for keepstore index requests.
Tom Clegg [Tue, 16 Jun 2020 14:10:53 +0000 (10:10 -0400)]
16480: Use longer timeout for keepstore index requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Handle overlapping permissions correctly
Peter Amstutz [Mon, 15 Jun 2020 22:17:09 +0000 (18:17 -0400)]
16007: Handle overlapping permissions correctly

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Add tests for overlapping permission links
Peter Amstutz [Mon, 15 Jun 2020 18:25:30 +0000 (14:25 -0400)]
16007: Add tests for overlapping permission links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16425-skip-unused-dirs'
Tom Clegg [Mon, 15 Jun 2020 15:17:48 +0000 (11:17 -0400)]
Merge branch '16425-skip-unused-dirs'

fixes #16425

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16427-undelete'
Tom Clegg [Fri, 12 Jun 2020 20:25:47 +0000 (16:25 -0400)]
Merge branch '16427-undelete'

closes #16427

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16425: Don't scan dirs that we don't write in.
Tom Clegg [Fri, 12 Jun 2020 20:14:20 +0000 (16:14 -0400)]
16425: Don't scan dirs that we don't write in.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16526: update the run-build-packages-python-and-ruby.sh script so that
Ward Vandewege [Fri, 12 Jun 2020 15:26:24 +0000 (11:26 -0400)]
16526: update the run-build-packages-python-and-ruby.sh script so that
it can be asked to do ruby gems or python packages alone. The default
behavior remains unchanged: both ruby gems and python packages will be
built.Some bash comparison cleanups.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16171: Merge branch 'master'
Tom Clegg [Fri, 12 Jun 2020 15:18:45 +0000 (11:18 -0400)]
16171: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Merge branch 'master'
Tom Clegg [Fri, 12 Jun 2020 15:17:41 +0000 (11:17 -0400)]
16427: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch 'master' into 16007-permission-table-rb
Peter Amstutz [Fri, 12 Jun 2020 14:43:53 +0000 (10:43 -0400)]
Merge branch 'master' into 16007-permission-table-rb

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16513: add an option to keep-exercise for a timed run, with a CSV output
Ward Vandewege [Fri, 12 Jun 2020 14:47:26 +0000 (10:47 -0400)]
16513: add an option to keep-exercise for a timed run, with a CSV output
       printed at the end.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16524: Pin rsa package to < 4.1
Peter Amstutz [Fri, 12 Jun 2020 14:13:24 +0000 (10:13 -0400)]
16524: Pin rsa package to < 4.1

Because >= 4.1 does not support python 2 any more.  refs #16524

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16427: Option to recover from given collection's last log entry.
Tom Clegg [Thu, 11 Jun 2020 20:29:33 +0000 (16:29 -0400)]
16427: Option to recover from given collection's last log entry.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoFix double slash in URL.
Tom Clegg [Thu, 11 Jun 2020 19:26:06 +0000 (15:26 -0400)]
Fix double slash in URL.

refs #16523

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Add REVOKE_PERM and CAN_MANAGE_PERM constants
Peter Amstutz [Thu, 11 Jun 2020 17:12:04 +0000 (13:12 -0400)]
16007: Add REVOKE_PERM and CAN_MANAGE_PERM constants

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16427: Rename undelete -> recover-collection.
Tom Clegg [Wed, 10 Jun 2020 14:26:18 +0000 (10:26 -0400)]
16427: Rename undelete -> recover-collection.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Add new config keys to export list.
Tom Clegg [Tue, 9 Jun 2020 17:47:45 +0000 (13:47 -0400)]
16171: Add new config keys to export list.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Configurable "username" OIDC claim key.
Tom Clegg [Tue, 9 Jun 2020 14:55:08 +0000 (10:55 -0400)]
16171: Configurable "username" OIDC claim key.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Configurable "email" and "email_verified" OIDC claim keys.
Tom Clegg [Tue, 9 Jun 2020 14:22:09 +0000 (10:22 -0400)]
16171: Configurable "email" and "email_verified" OIDC claim keys.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoadded type ContainerRequestList
Nico Cesar [Mon, 8 Jun 2020 20:35:22 +0000 (16:35 -0400)]
added type ContainerRequestList

refs #16462

Arvados-DCO-1.1-Signed-off-by: <nico@nicocesar.com>

4 years agoMerge branch '16171-oidc'
Tom Clegg [Mon, 8 Jun 2020 14:17:55 +0000 (10:17 -0400)]
Merge branch '16171-oidc'

refs #16171

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Fix typo & use query parameters
Peter Amstutz [Fri, 5 Jun 2020 21:02:28 +0000 (17:02 -0400)]
16007: Fix typo & use query parameters

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16427: Add "undeleting collections" doc page in admin section.
Tom Clegg [Fri, 5 Jun 2020 20:23:01 +0000 (16:23 -0400)]
16427: Add "undeleting collections" doc page in admin section.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Refactoring and update comments.
Peter Amstutz [Fri, 5 Jun 2020 20:20:24 +0000 (16:20 -0400)]
16007: Refactoring and update comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16492-websocket-extensions-gem-upgrade'
Lucas Di Pentima [Fri, 5 Jun 2020 19:37:22 +0000 (16:37 -0300)]
Merge branch '16492-websocket-extensions-gem-upgrade'
Closes #16492

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16492: Upgrades websocket-extensions gem to fix a security alert.
Lucas Di Pentima [Fri, 5 Jun 2020 18:13:44 +0000 (15:13 -0300)]
16492: Upgrades websocket-extensions gem to fix a security alert.

We really don't use this gem as it's requested by actioncable from Rails 5
that we don't even load at runtime. But, as we cannot remove actioncable
from our Gemfile.lock files because of some other gems depending on 'rails',
we upgrade it to make GitHub happy.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16427: Support looking up old manifest for given log entry UUID.
Tom Clegg [Fri, 5 Jun 2020 17:10:22 +0000 (13:10 -0400)]
16427: Support looking up old manifest for given log entry UUID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16427-undelete'
Tom Clegg [Fri, 5 Jun 2020 15:16:17 +0000 (11:16 -0400)]
Merge branch '16427-undelete'

refs #16427

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Update func comments.
Tom Clegg [Fri, 5 Jun 2020 15:15:44 +0000 (11:15 -0400)]
16427: Update func comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Special handing for users with permissions on other users
Peter Amstutz [Thu, 4 Jun 2020 20:58:18 +0000 (16:58 -0400)]
16007: Special handing for users with permissions on other users

Revise & simplify permission traversal.  Don't traverse users except
when starting from the user (origin_uuid = starting_uuid).

This avoids disasterous queries where we re-traverse other users "just
in case" and end up recomputing the whole database.  As a tradeoff,
our epic readable_by query gets a touch more epic, as it now has to go
to the permissions table to check if there are other user permissions
the current user also is allowed to use.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16427: Merge branch 'master'
Tom Clegg [Fri, 5 Jun 2020 14:51:17 +0000 (10:51 -0400)]
16427: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Don't print legacy config path flags if they won't be used.
Tom Clegg [Thu, 4 Jun 2020 20:35:05 +0000 (16:35 -0400)]
16427: Don't print legacy config path flags if they won't be used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Improve -help / usage message.
Tom Clegg [Thu, 4 Jun 2020 20:34:44 +0000 (16:34 -0400)]
16427: Improve -help / usage message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Make test logging more obvious.
Tom Clegg [Thu, 4 Jun 2020 19:45:44 +0000 (15:45 -0400)]
16427: Make test logging more obvious.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Explain workerThreads choice.
Tom Clegg [Thu, 4 Jun 2020 19:44:13 +0000 (15:44 -0400)]
16427: Explain workerThreads choice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Explain choice of blob ttl.
Tom Clegg [Thu, 4 Jun 2020 19:44:11 +0000 (15:44 -0400)]
16427: Explain choice of blob ttl.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Return error instead of ok bool from util funcs.
Tom Clegg [Thu, 4 Jun 2020 18:49:32 +0000 (14:49 -0400)]
16427: Return error instead of ok bool from util funcs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Change issuer config to string to avoid trailing-slash pain.
Tom Clegg [Thu, 4 Jun 2020 18:17:28 +0000 (14:17 -0400)]
16171: Change issuer config to string to avoid trailing-slash pain.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Warn about OIDC issuer URL spelling sensitivity.
Tom Clegg [Thu, 4 Jun 2020 15:44:29 +0000 (11:44 -0400)]
16171: Warn about OIDC issuer URL spelling sensitivity.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Test non-Google OIDC login with fake issuer.
Tom Clegg [Thu, 4 Jun 2020 15:21:43 +0000 (11:21 -0400)]
16171: Test non-Google OIDC login with fake issuer.

Ensures the proper credentials are used.

Exposes go-oidc's sensitivity to different spellings of equivalent
issuer URLs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Move more code to one-time setup func.
Tom Clegg [Thu, 4 Jun 2020 13:54:35 +0000 (09:54 -0400)]
16171: Move more code to one-time setup func.

Fix Google credentials used for non-Google OIDC provider.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16482: bump a-c-r's cwltool dependency to version 3.0.20200530110633 so
Ward Vandewege [Wed, 3 Jun 2020 19:42:03 +0000 (15:42 -0400)]
16482: bump a-c-r's cwltool dependency to version 3.0.20200530110633 so
that CWL v1.2.0-dev3 tests are supported. This also requires an update
to the schema-salad dependency, and some adjustment to account for code
that moved around in cwltool. Also, a fix to make sure the output from
resource_stream gets turned into a string, not a binary object.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoBugfix: test_with_arvbox.sh in non-build mode now works again and uses
Ward Vandewege [Tue, 2 Jun 2020 21:43:40 +0000 (17:43 -0400)]
Bugfix: test_with_arvbox.sh in non-build mode now works again and uses
the logic from run-library.sh to determine the version of arvados/jobs
to pull. Also includes some extra error checking and a small comparison
operator bugfix.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoFixes arvbox config, adding Login.SSO.Enable: true. No issue #
Lucas Di Pentima [Tue, 2 Jun 2020 18:00:40 +0000 (15:00 -0300)]
Fixes arvbox config, adding Login.SSO.Enable: true. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '16482-bump-cwltool-version'
Ward Vandewege [Wed, 3 Jun 2020 13:40:31 +0000 (09:40 -0400)]
Merge branch '16482-bump-cwltool-version'

closes #16482

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16482: Merge branch 'master' into 16482-bump-cwltool-version
Ward Vandewege [Wed, 3 Jun 2020 13:39:41 +0000 (09:39 -0400)]
16482: Merge branch 'master' into 16482-bump-cwltool-version

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoBugfix: test_with_arvbox.sh in non-build mode now works again and uses
Ward Vandewege [Tue, 2 Jun 2020 21:43:40 +0000 (17:43 -0400)]
Bugfix: test_with_arvbox.sh in non-build mode now works again and uses
the logic from run-library.sh to determine the version of arvados/jobs
to pull. Also includes some extra error checking and a small comparison
operator bugfix.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16482: bump a-c-r's cwltool dependency to version 3.0.20200324120055 so
Ward Vandewege [Tue, 2 Jun 2020 21:30:56 +0000 (17:30 -0400)]
16482: bump a-c-r's cwltool dependency to version 3.0.20200324120055 so
       that CWL v1.2.0-dev3 tests are supported.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoFixes arvbox config, adding Login.SSO.Enable: true. No issue #
Lucas Di Pentima [Tue, 2 Jun 2020 18:00:40 +0000 (15:00 -0300)]
Fixes arvbox config, adding Login.SSO.Enable: true. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoSome refactoring in the build scripts: move duplicated code to calculate
Ward Vandewege [Tue, 2 Jun 2020 14:05:44 +0000 (10:05 -0400)]
Some refactoring in the build scripts: move duplicated code to calculate
the cwl runner package version to a function. Remove unused argument to
the `version_from_git` function.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoFix whitespace in tools/arvbox/lib/arvbox/docker/common.sh
Ward Vandewege [Mon, 1 Jun 2020 19:56:29 +0000 (15:56 -0400)]
Fix whitespace in tools/arvbox/lib/arvbox/docker/common.sh

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16171: Add OIDC config keys to export whitelist.
Tom Clegg [Mon, 1 Jun 2020 15:08:06 +0000 (11:08 -0400)]
16171: Add OIDC config keys to export whitelist.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Tidy up config test.
Tom Clegg [Mon, 1 Jun 2020 14:28:15 +0000 (10:28 -0400)]
16171: Tidy up config test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Don't use Google as example of non-Google OIDC issuer.
Tom Clegg [Mon, 1 Jun 2020 14:17:21 +0000 (10:17 -0400)]
16171: Don't use Google as example of non-Google OIDC issuer.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Fix test order dependency.
Tom Clegg [Mon, 1 Jun 2020 13:42:22 +0000 (09:42 -0400)]
16427: Fix test order dependency.

Test was incorrectly assuming keep2, keep3 data dirs existed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16171: Support non-Google OpenID Connect auth provider.
Tom Clegg [Sun, 31 May 2020 23:57:11 +0000 (19:57 -0400)]
16171: Support non-Google OpenID Connect auth provider.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Move blob signing test cases to arvados pkg.
Tom Clegg [Fri, 29 May 2020 20:38:24 +0000 (16:38 -0400)]
16427: Move blob signing test cases to arvados pkg.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Add test cases.
Tom Clegg [Fri, 29 May 2020 14:41:37 +0000 (10:41 -0400)]
16427: Add test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16427: Touch block timestamps to avoid garbage collection race.
Tom Clegg [Fri, 29 May 2020 14:36:13 +0000 (10:36 -0400)]
16427: Touch block timestamps to avoid garbage collection race.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16007: Add comment about override_edge_* parameters
Peter Amstutz [Fri, 29 May 2020 02:22:57 +0000 (22:22 -0400)]
16007: Add comment about override_edge_* parameters

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Ensure that updated permission edge overrides edges view
Peter Amstutz [Fri, 29 May 2020 01:41:46 +0000 (21:41 -0400)]
16007: Ensure that updated permission edge overrides edges view

An edge originating from a user can be traversed more than once, if
that edge is the same as the one being updated, ensure that it uses
the updated permission level and not the permission from the edges
view.  Necessary when revoking permissions.

Also moved comments into the body of the postgres functions to bring
them closer to the code, this also has the convenient effect of having
the comments appear in structure.sql function definitions so as to be
easier for future developers to find.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Enable permission correctness checking (only for tests)
Peter Amstutz [Thu, 28 May 2020 19:35:29 +0000 (15:35 -0400)]
16007: Enable permission correctness checking (only for tests)

* Explicitly set up a transaction in update_permissions
* Rename refresh_permission_view.rb -> update_permissions.rb
* Add skip_check_permissions_against_full_refresh

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: refresh_trashed uses a transaction
Peter Amstutz [Tue, 26 May 2020 19:12:56 +0000 (15:12 -0400)]
16007: refresh_trashed uses a transaction

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: More code comment detail about compute_permission_subgraph query
Peter Amstutz [Tue, 26 May 2020 18:11:24 +0000 (14:11 -0400)]
16007: More code comment detail about compute_permission_subgraph query

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Lots and lots lots of method documentation via code comments.
Peter Amstutz [Fri, 22 May 2020 19:00:22 +0000 (15:00 -0400)]
16007: Lots and lots lots of method documentation via code comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16007: Use incremental updates instead of materialized view for permissions
Peter Amstutz [Fri, 1 May 2020 18:26:35 +0000 (14:26 -0400)]
16007: Use incremental updates instead of materialized view for permissions

Separate 'trashed' from 'permissions' and remove 'trashed' from
permission computation.  Add postgres functions for computing trash
and update trashed_groups incrementally.  Make sure trash table gets
refreshed on database reset.  readable_by() now checks trash_at timestamp.

Drop materialized view and replace with a table that is updated
incrementally.  Add postgres functions for computing permissions.
Initialize materialized_permissions from search_permission_graph.
Call refresh_permissions in database_seeds.  Add index on
materialized_permissions.target_uuid.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16427: Add "touch" API to keepstore.
Tom Clegg [Thu, 28 May 2020 13:40:44 +0000 (09:40 -0400)]
16427: Add "touch" API to keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>