arvados.git
2 years ago17948: First working implementation (WIP)
Lucas Di Pentima [Thu, 5 Aug 2021 14:42:09 +0000 (11:42 -0300)]
17948: First working implementation (WIP)

There're some issues: RailsAPI's full text search index on collections
returns an error when creating collection with >~36k files. Also, PySDK
is returning a socket read timeout error, seemilngly disregarding the
'timeout' setting on the arvados.api() object.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18005-collection-versioning-regression' into main. Closes #18004
Lucas Di Pentima [Wed, 11 Aug 2021 19:40:59 +0000 (16:40 -0300)]
Merge branch '18005-collection-versioning-regression' into main. Closes #18004

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18005: Fixes the bug.
Lucas Di Pentima [Wed, 11 Aug 2021 18:37:46 +0000 (15:37 -0300)]
18005: Fixes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18005: Exposes the bug by simulating a keep-balance run a making a new version.
Lucas Di Pentima [Wed, 11 Aug 2021 18:31:11 +0000 (15:31 -0300)]
18005: Exposes the bug by simulating a keep-balance run a making a new version.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '17985-cwltool-binstub'
Ward Vandewege [Wed, 11 Aug 2021 15:20:24 +0000 (11:20 -0400)]
Merge branch '17985-cwltool-binstub'

closes #17985

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17985: fix binstub for cwltool in our arvados-cwl-runner package.
Ward Vandewege [Wed, 11 Aug 2021 15:19:35 +0000 (11:19 -0400)]
17985: fix binstub for cwltool in our arvados-cwl-runner package.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17936-arvput-batch-mode' into main. Closes #17936.
Lucas Di Pentima [Tue, 10 Aug 2021 18:57:21 +0000 (15:57 -0300)]
Merge branch '17936-arvput-batch-mode' into main. Closes #17936.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '17967-storage-classes-config' into main
Tom Clegg [Tue, 10 Aug 2021 14:43:25 +0000 (10:43 -0400)]
Merge branch '17967-storage-classes-config' into main

closes #17967

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17936: Updates arv-keepdocker to call arv-put with --batch argument.
Lucas Di Pentima [Mon, 9 Aug 2021 18:12:48 +0000 (15:12 -0300)]
17936: Updates arv-keepdocker to call arv-put with --batch argument.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17936: Adds --batch implementation, making the new test pass.
Lucas Di Pentima [Mon, 9 Aug 2021 17:58:41 +0000 (14:58 -0300)]
17936: Adds --batch implementation, making the new test pass.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17967: Update generated config.
Tom Clegg [Mon, 9 Aug 2021 17:23:44 +0000 (13:23 -0400)]
17967: Update generated config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17936: Adds --batch argument to arv-put, updates test.
Lucas Di Pentima [Mon, 9 Aug 2021 16:56:44 +0000 (13:56 -0300)]
17936: Adds --batch argument to arv-put, updates test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17967: Test new behavior for default storage_classes_desired.
Tom Clegg [Mon, 9 Aug 2021 15:29:10 +0000 (11:29 -0400)]
17967: Test new behavior for default storage_classes_desired.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17967: Link to config doc page from conf file and upgrade notes.
Tom Clegg [Mon, 9 Aug 2021 15:11:32 +0000 (11:11 -0400)]
17967: Link to config doc page from conf file and upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17967: Update storage classes docs.
Tom Clegg [Fri, 6 Aug 2021 14:19:59 +0000 (10:19 -0400)]
17967: Update storage classes docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17967: Update tests.
Tom Clegg [Thu, 5 Aug 2021 21:57:50 +0000 (17:57 -0400)]
17967: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17952-no-nodes-link' into main closes #17952
Peter Amstutz [Thu, 5 Aug 2021 16:13:42 +0000 (12:13 -0400)]
Merge branch '17952-no-nodes-link' into main closes #17952

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17967: Use StorageClasses.*.Default instead of ["default"].
Tom Clegg [Thu, 5 Aug 2021 15:04:37 +0000 (11:04 -0400)]
17967: Use StorageClasses.*.Default instead of ["default"].

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17952: Remove tests for presence of nodes / disks menu item
Peter Amstutz [Thu, 5 Aug 2021 14:25:00 +0000 (10:25 -0400)]
17952: Remove tests for presence of nodes / disks menu item

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17967: Read from volumes with high-priority storage classes first.
Tom Clegg [Thu, 5 Aug 2021 03:21:45 +0000 (23:21 -0400)]
17967: Read from volumes with high-priority storage classes first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17952: Remove links to obsolete "nodes" and "keep disks"
Peter Amstutz [Wed, 4 Aug 2021 21:38:09 +0000 (17:38 -0400)]
17952: Remove links to obsolete "nodes" and "keep disks"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17967: Add StorageClasses config section.
Tom Clegg [Wed, 4 Aug 2021 20:57:35 +0000 (16:57 -0400)]
17967: Add StorageClasses config section.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17574-storage-classes-confirmed' into main
Tom Clegg [Wed, 4 Aug 2021 15:25:12 +0000 (11:25 -0400)]
Merge branch '17574-storage-classes-confirmed' into main

closes #17574

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Merge branch 'main'
Tom Clegg [Wed, 4 Aug 2021 14:02:15 +0000 (10:02 -0400)]
17574: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17591-doc-improvement'
Ward Vandewege [Tue, 3 Aug 2021 21:33:22 +0000 (17:33 -0400)]
Merge branch '17591-doc-improvement'

closes #17591

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch 'main' into 17591-doc-improvement
Ward Vandewege [Tue, 3 Aug 2021 21:32:59 +0000 (17:32 -0400)]
Merge branch 'main' into 17591-doc-improvement

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17591: address review feedback.
Ward Vandewege [Tue, 3 Aug 2021 21:26:30 +0000 (17:26 -0400)]
17591: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoUpdate the package repositories page, we now have Debian 11 (bullseye)
Ward Vandewege [Tue, 3 Aug 2021 21:02:37 +0000 (17:02 -0400)]
Update the package repositories page, we now have Debian 11 (bullseye)
pacakges.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17574: Confirm storage classes as [] when replication is 0.
Tom Clegg [Tue, 3 Aug 2021 20:42:04 +0000 (16:42 -0400)]
17574: Confirm storage classes as [] when replication is 0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix map write when only RLock held.
Tom Clegg [Tue, 3 Aug 2021 15:46:20 +0000 (11:46 -0400)]
Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17591: be more explicit about the required TLS certificates and DNS
Ward Vandewege [Tue, 3 Aug 2021 15:15:20 +0000 (11:15 -0400)]
17591: be more explicit about the required TLS certificates and DNS
       hostnames.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd extra validation of our package version format in the build scripts,
Ward Vandewege [Fri, 30 Jul 2021 17:27:35 +0000 (13:27 -0400)]
Add extra validation of our package version format in the build scripts,
it must match

  ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+|)(~rc[0-9]+|~dev[0-9]+|)-[0-9]+$

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMore cleanup, remove references to Ubuntu 16.04 (xenial) which is no
Ward Vandewege [Thu, 29 Jul 2021 20:13:54 +0000 (16:13 -0400)]
More cleanup, remove references to Ubuntu 16.04 (xenial) which is no
longer supported.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoDebian 11 (bullseye) will be a supported distribution in the next major
Ward Vandewege [Thu, 29 Jul 2021 20:12:44 +0000 (16:12 -0400)]
Debian 11 (bullseye) will be a supported distribution in the next major
Arvados release.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17949-add-bullseye'
Ward Vandewege [Thu, 29 Jul 2021 19:52:44 +0000 (15:52 -0400)]
Merge branch '17949-add-bullseye'

closes #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17574: Fix map write when only RLock held.
Tom Clegg [Thu, 29 Jul 2021 18:40:10 +0000 (14:40 -0400)]
17574: Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17949: add support for Debian Bullseye in our build scripts. Remove some
Ward Vandewege [Thu, 29 Jul 2021 18:01:35 +0000 (14:01 -0400)]
17949: add support for Debian Bullseye in our build scripts. Remove some
       configuration for older, no longer supported distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17756-dispatch-lsf' into main
Tom Clegg [Thu, 29 Jul 2021 13:57:48 +0000 (09:57 -0400)]
Merge branch '17756-dispatch-lsf' into main

closes #17756

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Fix misleading errors after reaching update limit.
Tom Clegg [Thu, 29 Jul 2021 03:45:23 +0000 (23:45 -0400)]
17574: Fix misleading errors after reaching update limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Update test.
Tom Clegg [Wed, 28 Jul 2021 22:23:05 +0000 (18:23 -0400)]
17574: Update test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Fix wait on empty channel.
Tom Clegg [Wed, 28 Jul 2021 22:22:08 +0000 (18:22 -0400)]
17574: Fix wait on empty channel.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Update install/upgrade docs wrt PostgreSQL access.
Tom Clegg [Wed, 28 Jul 2021 20:20:33 +0000 (16:20 -0400)]
17574: Update install/upgrade docs wrt PostgreSQL access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Batch updates into transactions, skip when unchanged.
Tom Clegg [Wed, 28 Jul 2021 20:11:52 +0000 (16:11 -0400)]
17574: Batch updates into transactions, skip when unchanged.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix the example AWS InstanceTypes for the installer, none of these node
Ward Vandewege [Wed, 28 Jul 2021 19:28:17 +0000 (15:28 -0400)]
Fix the example AWS InstanceTypes for the installer, none of these node
types come with IncludedScratch.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17903-login-sync-centos7'
Ward Vandewege [Wed, 28 Jul 2021 15:11:50 +0000 (11:11 -0400)]
Merge branch '17903-login-sync-centos7'

closes #17903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17903: Merge branch 'main' into 17903-login-sync-centos7
Ward Vandewege [Wed, 28 Jul 2021 15:11:11 +0000 (11:11 -0400)]
17903: Merge branch 'main' into 17903-login-sync-centos7

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17830-reqid-header-propagation-fix' into main. Closes #17830
Lucas Di Pentima [Wed, 28 Jul 2021 13:43:07 +0000 (10:43 -0300)]
Merge branch '17830-reqid-header-propagation-fix' into main. Closes #17830

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17574: Process collections in a worker pool.
Tom Clegg [Wed, 28 Jul 2021 04:19:03 +0000 (00:19 -0400)]
17574: Process collections in a worker pool.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Add BalanceUpdateLimit config, fix tests.
Tom Clegg [Wed, 28 Jul 2021 04:18:53 +0000 (00:18 -0400)]
17574: Add BalanceUpdateLimit config, fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: add -pprof flag to service commands.
Tom Clegg [Wed, 28 Jul 2021 04:13:50 +0000 (00:13 -0400)]
17574: add -pprof flag to service commands.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Speed up manifest parsing.
Tom Clegg [Wed, 28 Jul 2021 04:04:56 +0000 (00:04 -0400)]
17574: Speed up manifest parsing.

Previous code spent a lot of time allocating, copying, and GCing
strings. Working with byte slices is much faser.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17830: Moves header setting code to cover more use cases.
Lucas Di Pentima [Tue, 27 Jul 2021 21:19:54 +0000 (18:19 -0300)]
17830: Moves header setting code to cover more use cases.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17830: Adds test cases & fixes one more issue.
Lucas Di Pentima [Tue, 27 Jul 2021 19:22:26 +0000 (16:22 -0300)]
17830: Adds test cases & fixes one more issue.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17903: use more distribution-agnostic commands to add/remove a user
Ward Vandewege [Tue, 27 Jul 2021 18:40:19 +0000 (14:40 -0400)]
17903: use more distribution-agnostic commands to add/remove a user
       to/from a group. This makes arvados-login-sync work on CentOS as
       well as Debian-based distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17574: Add -update-confirmed-fields=false option.
Tom Clegg [Tue, 27 Jul 2021 15:23:48 +0000 (11:23 -0400)]
17574: Add -update-confirmed-fields=false option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Get collections directly from DB instead of controller.
Tom Clegg [Tue, 27 Jul 2021 15:16:49 +0000 (11:16 -0400)]
17574: Get collections directly from DB instead of controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665
Lucas Di Pentima [Tue, 27 Jul 2021 14:35:21 +0000 (11:35 -0300)]
Merge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '17813-docker-to-singularity' into main
Tom Clegg [Mon, 26 Jul 2021 20:02:31 +0000 (16:02 -0400)]
Merge branch '17813-docker-to-singularity' into main

refs #17813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17813: Error out if sif cache collection cannot be created.
Tom Clegg [Mon, 26 Jul 2021 20:01:54 +0000 (16:01 -0400)]
17813: Error out if sif cache collection cannot be created.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17574: Update replication_confirmed fields after keep-balance run.
Tom Clegg [Mon, 26 Jul 2021 19:28:01 +0000 (15:28 -0400)]
17574: Update replication_confirmed fields after keep-balance run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix typo in documentation.
Ward Vandewege [Mon, 26 Jul 2021 19:09:44 +0000 (15:09 -0400)]
Fix typo in documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17388-arvcopy-storage-classes' into main. Closes #17388
Lucas Di Pentima [Mon, 26 Jul 2021 14:32:05 +0000 (11:32 -0300)]
Merge branch '17388-arvcopy-storage-classes' into main. Closes #17388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17388: Adds storage classes support to arv-copy.
Lucas Di Pentima [Fri, 23 Jul 2021 22:52:00 +0000 (19:52 -0300)]
17388: Adds storage classes support to arv-copy.

Also, updates tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17813: Handle case where the cache collection update fails
Peter Amstutz [Fri, 23 Jul 2021 18:35:15 +0000 (14:35 -0400)]
17813: Handle case where the cache collection update fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17813: Fix tests
Peter Amstutz [Thu, 22 Jul 2021 21:32:44 +0000 (17:32 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago16665: Fixes tests.
Lucas Di Pentima [Thu, 22 Jul 2021 21:02:53 +0000 (18:02 -0300)]
16665: Fixes tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17813: continue refactor & fix tests
Peter Amstutz [Thu, 22 Jul 2021 20:44:19 +0000 (16:44 -0400)]
17813: continue refactor & fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago16665: Wraps error type to be able to differentiate it in a case switch.
Lucas Di Pentima [Thu, 22 Jul 2021 20:30:51 +0000 (17:30 -0300)]
16665: Wraps error type to be able to differentiate it in a case switch.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago16665: Exposes bug through updated test.
Lucas Di Pentima [Thu, 22 Jul 2021 20:24:26 +0000 (17:24 -0300)]
16665: Exposes bug through updated test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17813: Refactor singularity image loading / caching / conversion
Peter Amstutz [Thu, 22 Jul 2021 20:22:39 +0000 (16:22 -0400)]
17813: Refactor singularity image loading / caching / conversion

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17830: Copies request's X-Request-Id header to response. Moves tests.
Lucas Di Pentima [Thu, 22 Jul 2021 18:31:11 +0000 (15:31 -0300)]
17830: Copies request's X-Request-Id header to response. Moves tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17830: Sets up X-Request-Id if not provided by client. Adds it to the response.
Lucas Di Pentima [Wed, 21 Jul 2021 23:51:10 +0000 (20:51 -0300)]
17830: Sets up X-Request-Id if not provided by client. Adds it to the response.

Uses the IDGenerator to explicitly set one request id if needed, so that it
can return it to the client without the need to dig it out from the railsAPI
response.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17830: Adds controller tests exposing the X-Request-Id header propagation bug.
Lucas Di Pentima [Tue, 20 Jul 2021 20:59:58 +0000 (17:59 -0300)]
17830: Adds controller tests exposing the X-Request-Id header propagation bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17830: Customizes ActionDispatch::RequestId middleware to our needs.
Lucas Di Pentima [Tue, 20 Jul 2021 19:23:36 +0000 (16:23 -0300)]
17830: Customizes ActionDispatch::RequestId middleware to our needs.

Moves all the request id code to the proper middleware, and updates tests.
As functional tests are "unit tests for controllers", things that happen inside
middlewares need to be tested from integration tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17830: Adds test exposing incorrect X-Request-Id format on railsAPI.
Lucas Di Pentima [Tue, 20 Jul 2021 14:26:21 +0000 (11:26 -0300)]
17830: Adds test exposing incorrect X-Request-Id format on railsAPI.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago17756: Fix test.
Tom Clegg [Thu, 22 Jul 2021 15:10:25 +0000 (11:10 -0400)]
17756: Fix test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17756: Note that bsub sudo user must exist on worker nodes.
Tom Clegg [Thu, 22 Jul 2021 15:06:29 +0000 (11:06 -0400)]
17756: Note that bsub sudo user must exist on worker nodes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17756: Move "no suitable instance type" reporting to dispatch lib.
Tom Clegg [Thu, 22 Jul 2021 14:36:14 +0000 (10:36 -0400)]
17756: Move "no suitable instance type" reporting to dispatch lib.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17863-bundler-update'
Ward Vandewege [Thu, 22 Jul 2021 14:27:45 +0000 (10:27 -0400)]
Merge branch '17863-bundler-update'

closes #17863

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17863: Merge branch 'main' into 17863-bundler-update
Ward Vandewege [Thu, 22 Jul 2021 14:25:59 +0000 (10:25 -0400)]
17863: Merge branch 'main' into 17863-bundler-update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17863: further cleanup in our rails postinst scripts and documentation:
Ward Vandewege [Thu, 22 Jul 2021 13:02:27 +0000 (09:02 -0400)]
17863: further cleanup in our rails postinst scripts and documentation:

* replace `bundle exec` calls with the appropriate binstub
* update deprecated use of `bundle install --path` to avoid warning

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17962-bump-lib-pq'
Ward Vandewege [Wed, 21 Jul 2021 12:46:46 +0000 (08:46 -0400)]
Merge branch '17962-bump-lib-pq'

closes #17962

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17962: Merge branch 'main' into 17962-bump-lib-pq
Ward Vandewege [Wed, 21 Jul 2021 12:46:12 +0000 (08:46 -0400)]
17962: Merge branch 'main' into 17962-bump-lib-pq

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17813: Change default value of runtime_auth_scopes to []
Peter Amstutz [Tue, 20 Jul 2021 21:01:24 +0000 (17:01 -0400)]
17813: Change default value of runtime_auth_scopes to []

It is documented to be an array of strings, so having the default be
an object just seems to be a mistake.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17813: Fix tests
Peter Amstutz [Tue, 20 Jul 2021 21:01:08 +0000 (17:01 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMake singularity suid in arvbox, crunch-dispatch-local runs as user
Peter Amstutz [Tue, 20 Jul 2021 20:28:30 +0000 (16:28 -0400)]
Make singularity suid in arvbox, crunch-dispatch-local runs as user

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17962: Bump the version of lib/pq to 1.10.2
Ward Vandewege [Tue, 20 Jul 2021 19:25:05 +0000 (15:25 -0400)]
17962: Bump the version of lib/pq to 1.10.2

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAnother doc tweak.
Ward Vandewege [Tue, 20 Jul 2021 17:08:46 +0000 (13:08 -0400)]
Another doc tweak.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd some more clarifications to the doc page about restricting uploads
Ward Vandewege [Tue, 20 Jul 2021 15:53:27 +0000 (11:53 -0400)]
Add some more clarifications to the doc page about restricting uploads
and downloads.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17813: Handle setting $HOME as a special case
Peter Amstutz [Tue, 20 Jul 2021 15:15:44 +0000 (11:15 -0400)]
17813: Handle setting $HOME as a special case

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17748-extra-claims-take2'
Nico Cesar [Tue, 20 Jul 2021 14:07:09 +0000 (10:07 -0400)]
Merge branch '17748-extra-claims-take2'

closes #17748

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

2 years agoFix typos in the documentation.
Ward Vandewege [Tue, 20 Jul 2021 12:47:52 +0000 (08:47 -0400)]
Fix typos in the documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoOIDC given name / family name fields support
Nico Cesar [Mon, 19 Jul 2021 22:03:38 +0000 (18:03 -0400)]
OIDC given name / family name fields support

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@nicocesar.com>

2 years ago17813: Upload .sif file to cache project & read it from keep
Peter Amstutz [Mon, 19 Jul 2021 20:13:33 +0000 (16:13 -0400)]
17813: Upload .sif file to cache project & read it from keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17863: update to using bundler 2.2.19 everywhere.
Ward Vandewege [Mon, 19 Jul 2021 15:17:46 +0000 (11:17 -0400)]
17863: update to using bundler 2.2.19 everywhere.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoFix arvbox demo image build.
Ward Vandewege [Mon, 19 Jul 2021 14:25:56 +0000 (10:25 -0400)]
Fix arvbox demo image build.

refs #17816

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17813: Singularity image caching wip
Peter Amstutz [Fri, 16 Jul 2021 21:26:31 +0000 (17:26 -0400)]
17813: Singularity image caching wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17756: Review feedback.
Tom Clegg [Fri, 16 Jul 2021 20:42:46 +0000 (16:42 -0400)]
17756: Review feedback.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17573-export-storage-classes' into main. Refs #17573
Lucas Di Pentima [Fri, 16 Jul 2021 19:03:20 +0000 (16:03 -0300)]
Merge branch '17573-export-storage-classes' into main. Refs #17573

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '17816-singularity-cwd' into main refs #17816
Peter Amstutz [Fri, 16 Jul 2021 15:27:06 +0000 (11:27 -0400)]
Merge branch '17816-singularity-cwd' into main refs #17816

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>