arvados.git
3 years agoMerge branch '17913-addressable-upgrade' into main. Closes #17913
Lucas Di Pentima [Tue, 13 Jul 2021 19:55:30 +0000 (16:55 -0300)]
Merge branch '17913-addressable-upgrade' into main. Closes #17913

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17913: Upgrades 'addressable' gem to 2.8.0
Lucas Di Pentima [Tue, 13 Jul 2021 19:14:49 +0000 (16:14 -0300)]
17913: Upgrades 'addressable' gem to 2.8.0

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17782-nodejs-update' into main
Lucas Di Pentima [Mon, 12 Jul 2021 15:14:28 +0000 (12:14 -0300)]
Merge branch '17782-nodejs-update' into main
Refs #17782

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoPin networkx to avoid new version that pulls in scipy/numpy
Peter Amstutz [Fri, 9 Jul 2021 14:54:30 +0000 (10:54 -0400)]
Pin networkx to avoid new version that pulls in scipy/numpy

refs #17879

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd missing link to "restricting-upload-download" to doc refs #17464
Peter Amstutz [Thu, 8 Jul 2021 21:02:36 +0000 (17:02 -0400)]
Add missing link to "restricting-upload-download" to doc refs #17464

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17879-update-schema-salad' into main refs #17879
Peter Amstutz [Thu, 8 Jul 2021 19:24:55 +0000 (15:24 -0400)]
Merge branch '17879-update-schema-salad' into main refs #17879

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17879: Add test that acr doesn't fail on sbg fields
Peter Amstutz [Thu, 8 Jul 2021 19:20:51 +0000 (15:20 -0400)]
17879: Add test that acr doesn't fail on sbg fields

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17879: Default --thread-count=0, upgrade schema-salad
Peter Amstutz [Thu, 8 Jul 2021 18:36:00 +0000 (14:36 -0400)]
17879: Default --thread-count=0, upgrade schema-salad

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoPut the default arvados-cwl-runner --thread-count back down to 1.
Peter Amstutz [Wed, 7 Jul 2021 20:23:27 +0000 (16:23 -0400)]
Put the default arvados-cwl-runner --thread-count back down to 1.

On account of reports of workflows deadlocking.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17858-cwl-exprtool-defaults' into main refs #17858
Peter Amstutz [Wed, 7 Jul 2021 20:18:58 +0000 (16:18 -0400)]
Merge branch '17858-cwl-exprtool-defaults' into main refs #17858

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Tweak logic to avoid KeyError
Peter Amstutz [Wed, 7 Jul 2021 20:11:54 +0000 (16:11 -0400)]
17858: Tweak logic to avoid KeyError

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17705-adc-arch-doc'
Tom Clegg [Wed, 7 Jul 2021 19:10:13 +0000 (15:10 -0400)]
Merge branch '17705-adc-arch-doc'

closes #17705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17782: Updates nodejs to v12, as required by workbench2.
Lucas Di Pentima [Wed, 7 Jul 2021 18:50:38 +0000 (15:50 -0300)]
17782: Updates nodejs to v12, as required by workbench2.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17858: Add ExpressionTool to classes checked for ids when visiting
Peter Amstutz [Wed, 7 Jul 2021 14:02:05 +0000 (10:02 -0400)]
17858: Add ExpressionTool to classes checked for ids when visiting

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Integration test passes
Peter Amstutz [Wed, 7 Jul 2021 17:42:13 +0000 (13:42 -0400)]
17858: Integration test passes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Add test to reproduce bug
Peter Amstutz [Wed, 7 Jul 2021 13:51:09 +0000 (09:51 -0400)]
17858: Add test to reproduce bug

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17841-add-duration'
Ward Vandewege [Tue, 6 Jul 2021 21:37:16 +0000 (17:37 -0400)]
Merge branch '17841-add-duration'

closes #17841

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17841: Merge branch 'main' into 17841-add-duration
Ward Vandewege [Tue, 6 Jul 2021 21:36:43 +0000 (17:36 -0400)]
17841: Merge branch 'main' into 17841-add-duration

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17841: update documentation
Ward Vandewege [Tue, 6 Jul 2021 21:36:13 +0000 (17:36 -0400)]
17841: update documentation

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17853-fix-write-with-rlock' into main
Tom Clegg [Tue, 6 Jul 2021 13:11:01 +0000 (09:11 -0400)]
Merge branch '17853-fix-write-with-rlock' into main

fixes #17853

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17705: Improve InstanceSetID wording, update link to mgmt API docs.
Tom Clegg [Mon, 5 Jul 2021 18:46:03 +0000 (14:46 -0400)]
17705: Improve InstanceSetID wording, update link to mgmt API docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17853: Fix map write when only RLock held.
Tom Clegg [Mon, 5 Jul 2021 18:32:39 +0000 (14:32 -0400)]
17853: Fix map write when only RLock held.

Update DebugLocksPanicMode to check RLock/Lock as appropriate.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix api server tests.
Ward Vandewege [Fri, 2 Jul 2021 01:24:45 +0000 (21:24 -0400)]
Fix api server tests.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix keepproxy tests.
Ward Vandewege [Thu, 1 Jul 2021 21:36:09 +0000 (17:36 -0400)]
Fix keepproxy tests.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix arv-git-httpd tests.
Ward Vandewege [Thu, 1 Jul 2021 21:17:10 +0000 (17:17 -0400)]
Fix arv-git-httpd tests.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17395-container-output-storage-class' into main
Peter Amstutz [Thu, 1 Jul 2021 20:13:58 +0000 (16:13 -0400)]
Merge branch '17395-container-output-storage-class' into main

refs #17395

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17395: Add missing migration file
Peter Amstutz [Thu, 1 Jul 2021 19:37:20 +0000 (15:37 -0400)]
17395: Add missing migration file

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17395: Update docs for newer storage classes behavior.
Peter Amstutz [Thu, 1 Jul 2021 20:10:30 +0000 (16:10 -0400)]
17395: Update docs for newer storage classes behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoThe bleeding edge branch is now called 'main'.
Ward Vandewege [Thu, 1 Jul 2021 18:00:01 +0000 (14:00 -0400)]
The bleeding edge branch is now called 'main'.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoThe bleeding edge branch for arvados-workbench2 is now called 'main'.
Ward Vandewege [Thu, 1 Jul 2021 16:57:02 +0000 (12:57 -0400)]
The bleeding edge branch for arvados-workbench2 is now called 'main'.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17778-doc-update'
Ward Vandewege [Thu, 1 Jul 2021 13:24:17 +0000 (09:24 -0400)]
Merge branch '17778-doc-update'

closes #17778

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17778: Merge branch 'master' into 17778-doc-update
Ward Vandewege [Thu, 1 Jul 2021 13:23:45 +0000 (09:23 -0400)]
17778: Merge branch 'master' into 17778-doc-update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17464-download-activity' refs #17464
Peter Amstutz [Wed, 30 Jun 2021 19:49:01 +0000 (15:49 -0400)]
Merge branch '17464-download-activity' refs #17464

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Couple of doc fixes
Peter Amstutz [Wed, 30 Jun 2021 19:47:16 +0000 (15:47 -0400)]
17464: Couple of doc fixes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17821-provision-dump-config-parameter'
Javier Bértoli [Wed, 30 Jun 2021 18:03:01 +0000 (15:03 -0300)]
Merge branch '17821-provision-dump-config-parameter'

closes #17821
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17821: Add a parameter to provision.sh to dump the configuration
Javier Bértoli [Thu, 17 Jun 2021 19:50:16 +0000 (16:50 -0300)]
17821: Add a parameter to provision.sh to dump the configuration

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17395: Update documentation. Add an additional test.
Peter Amstutz [Wed, 30 Jun 2021 14:30:38 +0000 (10:30 -0400)]
17395: Update documentation.  Add an additional test.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17841: address review comments.
Ward Vandewege [Tue, 29 Jun 2021 21:36:37 +0000 (17:36 -0400)]
17841: address review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17395: Add tests. Don't include storage classes in full text search
Peter Amstutz [Tue, 29 Jun 2021 20:10:47 +0000 (16:10 -0400)]
17395: Add tests.  Don't include storage classes in full text search

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17395: Add OutputStorageClasses support to crunch-run
Peter Amstutz [Tue, 29 Jun 2021 19:24:06 +0000 (15:24 -0400)]
17395: Add OutputStorageClasses support to crunch-run

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17841: add container duration to the totals in the csv files. Reduce
Ward Vandewege [Tue, 29 Jun 2021 17:26:23 +0000 (13:26 -0400)]
17841: add container duration to the totals in the csv files. Reduce
       cost precision in totals to 1 cent.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17395: Add output_storage_classes to container and container_request
Peter Amstutz [Mon, 28 Jun 2021 19:58:57 +0000 (15:58 -0400)]
17395: Add output_storage_classes to container and container_request

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17829-remove-omniauth' refs #17829
Peter Amstutz [Mon, 28 Jun 2021 16:17:39 +0000 (12:17 -0400)]
Merge branch '17829-remove-omniauth' refs #17829

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Fix test
Peter Amstutz [Thu, 24 Jun 2021 17:17:06 +0000 (13:17 -0400)]
17829: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Fix test
Peter Amstutz [Thu, 24 Jun 2021 15:17:04 +0000 (11:17 -0400)]
17829: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17833-arvput-cleaner-errors'
Lucas Di Pentima [Thu, 24 Jun 2021 14:58:11 +0000 (11:58 -0300)]
Merge branch '17833-arvput-cleaner-errors'
Closes #17833

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17829: Remove additional references to SSO and omniauth
Peter Amstutz [Thu, 24 Jun 2021 14:46:44 +0000 (10:46 -0400)]
17829: Remove additional references to SSO and omniauth

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Remove more SSO-related configuration
Peter Amstutz [Wed, 23 Jun 2021 14:22:16 +0000 (10:22 -0400)]
17829: Remove more SSO-related configuration

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Remove SSO from config, controller, and tests
Peter Amstutz [Wed, 23 Jun 2021 14:07:19 +0000 (10:07 -0400)]
17829: Remove SSO from config, controller, and tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Fix test
Peter Amstutz [Tue, 22 Jun 2021 21:11:51 +0000 (17:11 -0400)]
17829: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Update tests to use 'controller' provider
Peter Amstutz [Tue, 22 Jun 2021 20:49:32 +0000 (16:49 -0400)]
17829: Update tests to use 'controller' provider

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17829: Remove omniauth
Peter Amstutz [Tue, 22 Jun 2021 02:19:17 +0000 (22:19 -0400)]
17829: Remove omniauth

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17828-remove-bootsnap' refs #17828
Peter Amstutz [Thu, 24 Jun 2021 14:07:23 +0000 (10:07 -0400)]
Merge branch '17828-remove-bootsnap' refs #17828

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17812-provisioning-documentation-improvement'
Javier Bértoli [Wed, 23 Jun 2021 21:51:49 +0000 (18:51 -0300)]
Merge branch '17812-provisioning-documentation-improvement'

closes #17812

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17833: Catches KeepWriteError exceptions to avoid displaying stacktraces.
Lucas Di Pentima [Wed, 23 Jun 2021 21:01:14 +0000 (18:01 -0300)]
17833: Catches KeepWriteError exceptions to avoid displaying stacktraces.

Replication and storage classes errors now won't show the entire stacktrace.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17812: Organize and extend installation documentation
Javier Bértoli [Mon, 21 Jun 2021 21:02:54 +0000 (18:02 -0300)]
17812: Organize and extend installation documentation

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '17572-arvmount-storage-classes'
Lucas Di Pentima [Wed, 23 Jun 2021 19:13:56 +0000 (16:13 -0300)]
Merge branch '17572-arvmount-storage-classes'
Closes #17572

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17389-keepproxy-storage-classes-confirmed'
Lucas Di Pentima [Wed, 23 Jun 2021 18:12:39 +0000 (15:12 -0300)]
Merge branch '17389-keepproxy-storage-classes-confirmed'
Closes #17389

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17389: Adds clarifying comment.
Lucas Di Pentima [Wed, 23 Jun 2021 18:11:23 +0000 (15:11 -0300)]
17389: Adds clarifying comment.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17801-runtime-outdir' refs #17801
Peter Amstutz [Wed, 23 Jun 2021 14:11:48 +0000 (10:11 -0400)]
Merge branch '17801-runtime-outdir' refs #17801

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17828: Remove bootsnap, which just seems to cause trouble
Peter Amstutz [Tue, 22 Jun 2021 19:01:18 +0000 (15:01 -0400)]
17828: Remove bootsnap, which just seems to cause trouble

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17333-cwl-check-project-uuid' refs #17333
Peter Amstutz [Tue, 22 Jun 2021 14:47:20 +0000 (10:47 -0400)]
Merge branch '17333-cwl-check-project-uuid' refs #17333

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17333: Add check for valid project uuid, with test
Peter Amstutz [Mon, 21 Jun 2021 20:12:12 +0000 (16:12 -0400)]
17333: Add check for valid project uuid, with test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17572: Merge branch 'master' into 17572-arvmount-storage-classes
Lucas Di Pentima [Mon, 21 Jun 2021 19:40:42 +0000 (16:40 -0300)]
17572: Merge branch 'master' into 17572-arvmount-storage-classes

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17572: Fixes failing test.
Lucas Di Pentima [Mon, 21 Jun 2021 19:39:40 +0000 (16:39 -0300)]
17572: Fixes failing test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17572: Adds tests.
Lucas Di Pentima [Mon, 21 Jun 2021 19:14:59 +0000 (16:14 -0300)]
17572: Adds tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17464: MakeRESTRouter returns erro instead of panicking
Peter Amstutz [Mon, 21 Jun 2021 15:25:29 +0000 (11:25 -0400)]
17464: MakeRESTRouter returns erro instead of panicking

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17572: Adds support for storage classes on new collections.
Lucas Di Pentima [Fri, 11 Jun 2021 20:05:48 +0000 (17:05 -0300)]
17572: Adds support for storage classes on new collections.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17572: Adds documentation related to storage classes.
Lucas Di Pentima [Wed, 9 Jun 2021 22:02:40 +0000 (19:02 -0300)]
17572: Adds documentation related to storage classes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17801: Handle glob capture of keep URI with a trailing slash
Peter Amstutz [Fri, 18 Jun 2021 21:29:05 +0000 (17:29 -0400)]
17801: Handle glob capture of keep URI with a trailing slash

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: keep-web handles cases where user is unknown
Peter Amstutz [Fri, 18 Jun 2021 19:58:03 +0000 (15:58 -0400)]
17464: keep-web handles cases where user is unknown

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Unknown user gets "User" permission check instead of failure
Peter Amstutz [Fri, 18 Jun 2021 19:43:26 +0000 (15:43 -0400)]
17464: Unknown user gets "User" permission check instead of failure

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17776-more-throttling'
Ward Vandewege [Fri, 18 Jun 2021 18:50:35 +0000 (14:50 -0400)]
Merge branch '17776-more-throttling'

closes #17776
closes #17777
closes #17783

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17776: Merge branch 'master' into 17776-more-throttling
Ward Vandewege [Fri, 18 Jun 2021 18:47:44 +0000 (14:47 -0400)]
17776: Merge branch 'master' into 17776-more-throttling

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17464: Fix typo in exported config
Peter Amstutz [Fri, 18 Jun 2021 17:07:39 +0000 (13:07 -0400)]
17464: Fix typo in exported config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add note about special case involving sharing links
Peter Amstutz [Fri, 18 Jun 2021 16:58:52 +0000 (12:58 -0400)]
17464: Add note about special case involving sharing links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Update exported config
Peter Amstutz [Fri, 18 Jun 2021 16:50:52 +0000 (12:50 -0400)]
17464: Update exported config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Replace cache with LRU cache
Peter Amstutz [Fri, 18 Jun 2021 16:35:38 +0000 (12:35 -0400)]
17464: Replace cache with LRU cache

Error out if API error from users.current

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17823-use-saltstack-formulas-nginx-formula'
Javier Bértoli [Fri, 18 Jun 2021 15:45:40 +0000 (12:45 -0300)]
Merge branch '17823-use-saltstack-formulas-nginx-formula'

closes #17823

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17823: change references back to saltstack-formulas' nginx formula
Javier Bértoli [Fri, 18 Jun 2021 15:34:14 +0000 (12:34 -0300)]
17823: change references back to saltstack-formulas' nginx formula

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17464: Don't deny or log GET on directories.
Peter Amstutz [Thu, 17 Jun 2021 20:47:33 +0000 (16:47 -0400)]
17464: Don't deny or log GET on directories.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add tests for paths by /users/ and by PDH
Peter Amstutz [Thu, 17 Jun 2021 19:49:04 +0000 (15:49 -0400)]
17464: Add tests for paths by /users/ and by PDH

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Clean up tests
Peter Amstutz [Thu, 17 Jun 2021 18:48:33 +0000 (14:48 -0400)]
17464: Clean up tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Update config variables & docs from feedback
Peter Amstutz [Thu, 17 Jun 2021 16:27:19 +0000 (12:27 -0400)]
17464: Update config variables & docs from feedback

Change KeepWebPermission -> WebDAVPermission

Add WebDAVLogEvents

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Activity report lists upload/download events
Peter Amstutz [Wed, 16 Jun 2021 21:22:04 +0000 (17:22 -0400)]
17464: Activity report lists upload/download events

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix test.
Peter Amstutz [Wed, 16 Jun 2021 20:06:36 +0000 (16:06 -0400)]
17464: Fix test.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add upload/download logging and permissions to keepproxy
Peter Amstutz [Wed, 16 Jun 2021 20:03:48 +0000 (16:03 -0400)]
17464: Add upload/download logging and permissions to keepproxy

Add tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Refactor tests and check that log events are posted
Peter Amstutz [Tue, 15 Jun 2021 20:59:40 +0000 (16:59 -0400)]
17464: Refactor tests and check that log events are posted

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Reset database after upload tests
Peter Amstutz [Tue, 15 Jun 2021 18:54:16 +0000 (14:54 -0400)]
17464: Reset database after upload tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Permission/logging testing WIP
Peter Amstutz [Mon, 14 Jun 2021 21:30:59 +0000 (17:30 -0400)]
17464: Permission/logging testing WIP

The upload tests are messing up the other tests by changing the
contents of the collection, still need to fix it.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix shadowed "sess" variable
Peter Amstutz [Fri, 11 Jun 2021 21:19:36 +0000 (17:19 -0400)]
17464: Fix shadowed "sess" variable

Testing WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Log collection uuid for FileSystem requests
Peter Amstutz [Fri, 11 Jun 2021 19:26:05 +0000 (15:26 -0400)]
17464: Log collection uuid for FileSystem requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Add upload/download permission checks and logging
Peter Amstutz [Thu, 10 Jun 2021 20:50:56 +0000 (16:50 -0400)]
17464: Add upload/download permission checks and logging

Adds extra log line to normal logging that includes user uuid and full
name.

Also posts an event to the logs table.

Adds permission checks to but these haven't been tested yet.

Still need to add testing.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix config entry
Peter Amstutz [Mon, 7 Jun 2021 19:42:32 +0000 (15:42 -0400)]
17464: Fix config entry

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Start by writing the documentation page
Peter Amstutz [Mon, 7 Jun 2021 19:23:40 +0000 (15:23 -0400)]
17464: Start by writing the documentation page

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17464: Fix auto-enable DebugLocksPanicMode.
Tom Clegg [Fri, 18 Jun 2021 15:28:08 +0000 (11:28 -0400)]
17464: Fix auto-enable DebugLocksPanicMode.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17609-diagnostics-cmd'
Tom Clegg [Fri, 18 Jun 2021 14:58:36 +0000 (10:58 -0400)]
Merge branch '17609-diagnostics-cmd'

closes #17609

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17464: Add DebugLocksPanicMode and fix some missing locks.
Tom Clegg [Fri, 18 Jun 2021 14:11:27 +0000 (10:11 -0400)]
17464: Add DebugLocksPanicMode and fix some missing locks.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17389: Adds X-Keep-Storage-Classes-Confirmed header to responses on success.
Lucas Di Pentima [Fri, 18 Jun 2021 11:50:07 +0000 (08:50 -0300)]
17389: Adds X-Keep-Storage-Classes-Confirmed header to responses on success.

Also, don't treat partial success (ie: replicas written > 0) as success to
let the client decide what to do, as this would require a GoSDK API change on
Put methods, or the use of a special new method just for keepproxy that
returns fulfilled storage classes information.
In the case of partial successes from the client point of view, the only
thing that a client can do is retry the request with the same keepproxy, and
that would render the same result.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17800-arvput-no-follow-links'
Lucas Di Pentima [Fri, 18 Jun 2021 11:03:15 +0000 (08:03 -0300)]
Merge branch '17800-arvput-no-follow-links'
Closes #17800

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>