arvados.git
3 years ago17698: Add comment to clarify locking.
Tom Clegg [Mon, 23 Aug 2021 14:01:40 +0000 (10:01 -0400)]
17698: Add comment to clarify locking.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17698: Update PutBlock comment.
Tom Clegg [Mon, 23 Aug 2021 14:01:32 +0000 (10:01 -0400)]
17698: Update PutBlock comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17698: Rename newPutResult to match new putProgress type name.
Tom Clegg [Mon, 23 Aug 2021 14:00:50 +0000 (10:00 -0400)]
17698: Rename newPutResult to match new putProgress type name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17698: Fix AllWritable panic when no volumes are writable.
Tom Clegg [Mon, 23 Aug 2021 14:00:01 +0000 (10:00 -0400)]
17698: Fix AllWritable panic when no volumes are writable.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17698: Do concurrent writes when multiple volumes are needed.
Tom Clegg [Fri, 13 Aug 2021 15:06:34 +0000 (11:06 -0400)]
17698: Do concurrent writes when multiple volumes are needed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17726-singularity-doc' into main
Tom Clegg [Thu, 12 Aug 2021 20:05:59 +0000 (16:05 -0400)]
Merge branch '17726-singularity-doc' into main

closes #17726

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17947-centos7-package-conflict'
Ward Vandewege [Thu, 12 Aug 2021 13:07:06 +0000 (09:07 -0400)]
Merge branch '17947-centos7-package-conflict'

closes #17947

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17947: Merge branch 'main' into 17947-centos7-package-conflict
Ward Vandewege [Thu, 12 Aug 2021 13:06:36 +0000 (09:06 -0400)]
17947: Merge branch 'main' into 17947-centos7-package-conflict

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18005-collection-versioning-regression' into main. Closes #18004
Lucas Di Pentima [Wed, 11 Aug 2021 19:40:59 +0000 (16:40 -0300)]
Merge branch '18005-collection-versioning-regression' into main. Closes #18004

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17947: the Arvados python3-* CentOS packages should conflict with the
Ward Vandewege [Wed, 11 Aug 2021 19:29:07 +0000 (15:29 -0400)]
17947: the Arvados python3-* CentOS packages should conflict with the
       corresponding older rh-python36-* version.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18005: Fixes the bug.
Lucas Di Pentima [Wed, 11 Aug 2021 18:37:46 +0000 (15:37 -0300)]
18005: Fixes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18005: Exposes the bug by simulating a keep-balance run a making a new version.
Lucas Di Pentima [Wed, 11 Aug 2021 18:31:11 +0000 (15:31 -0300)]
18005: Exposes the bug by simulating a keep-balance run a making a new version.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17985-cwltool-binstub'
Ward Vandewege [Wed, 11 Aug 2021 15:20:24 +0000 (11:20 -0400)]
Merge branch '17985-cwltool-binstub'

closes #17985

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17985: fix binstub for cwltool in our arvados-cwl-runner package.
Ward Vandewege [Wed, 11 Aug 2021 15:19:35 +0000 (11:19 -0400)]
17985: fix binstub for cwltool in our arvados-cwl-runner package.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17726: Add Singularity info to user docs.
Tom Clegg [Tue, 10 Aug 2021 21:00:45 +0000 (17:00 -0400)]
17726: Add Singularity info to user docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17936-arvput-batch-mode' into main. Closes #17936.
Lucas Di Pentima [Tue, 10 Aug 2021 18:57:21 +0000 (15:57 -0300)]
Merge branch '17936-arvput-batch-mode' into main. Closes #17936.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17967-storage-classes-config' into main
Tom Clegg [Tue, 10 Aug 2021 14:43:25 +0000 (10:43 -0400)]
Merge branch '17967-storage-classes-config' into main

closes #17967

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17936: Updates arv-keepdocker to call arv-put with --batch argument.
Lucas Di Pentima [Mon, 9 Aug 2021 18:12:48 +0000 (15:12 -0300)]
17936: Updates arv-keepdocker to call arv-put with --batch argument.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17936: Adds --batch implementation, making the new test pass.
Lucas Di Pentima [Mon, 9 Aug 2021 17:58:41 +0000 (14:58 -0300)]
17936: Adds --batch implementation, making the new test pass.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Update generated config.
Tom Clegg [Mon, 9 Aug 2021 17:23:44 +0000 (13:23 -0400)]
17967: Update generated config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17936: Adds --batch argument to arv-put, updates test.
Lucas Di Pentima [Mon, 9 Aug 2021 16:56:44 +0000 (13:56 -0300)]
17936: Adds --batch argument to arv-put, updates test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Test new behavior for default storage_classes_desired.
Tom Clegg [Mon, 9 Aug 2021 15:29:10 +0000 (11:29 -0400)]
17967: Test new behavior for default storage_classes_desired.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Link to config doc page from conf file and upgrade notes.
Tom Clegg [Mon, 9 Aug 2021 15:11:32 +0000 (11:11 -0400)]
17967: Link to config doc page from conf file and upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: More detail about singularity/docker differences.
Tom Clegg [Mon, 9 Aug 2021 13:49:03 +0000 (09:49 -0400)]
17726: More detail about singularity/docker differences.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: Link singularity install docs. Add dispatch-lsf mention.
Tom Clegg [Mon, 9 Aug 2021 13:38:13 +0000 (09:38 -0400)]
17726: Link singularity install docs. Add dispatch-lsf mention.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Update storage classes docs.
Tom Clegg [Fri, 6 Aug 2021 14:19:59 +0000 (10:19 -0400)]
17967: Update storage classes docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Update tests.
Tom Clegg [Thu, 5 Aug 2021 21:57:50 +0000 (17:57 -0400)]
17967: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: Add singularity doc page.
Tom Clegg [Thu, 5 Aug 2021 19:14:57 +0000 (15:14 -0400)]
17726: Add singularity doc page.

Move Docker image collection format note from 2.2 to main (this change
was not in 2.2).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17952-no-nodes-link' into main closes #17952
Peter Amstutz [Thu, 5 Aug 2021 16:13:42 +0000 (12:13 -0400)]
Merge branch '17952-no-nodes-link' into main closes #17952

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Use StorageClasses.*.Default instead of ["default"].
Tom Clegg [Thu, 5 Aug 2021 15:04:37 +0000 (11:04 -0400)]
17967: Use StorageClasses.*.Default instead of ["default"].

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove tests for presence of nodes / disks menu item
Peter Amstutz [Thu, 5 Aug 2021 14:25:00 +0000 (10:25 -0400)]
17952: Remove tests for presence of nodes / disks menu item

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Read from volumes with high-priority storage classes first.
Tom Clegg [Thu, 5 Aug 2021 03:21:45 +0000 (23:21 -0400)]
17967: Read from volumes with high-priority storage classes first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove links to obsolete "nodes" and "keep disks"
Peter Amstutz [Wed, 4 Aug 2021 21:38:09 +0000 (17:38 -0400)]
17952: Remove links to obsolete "nodes" and "keep disks"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Add StorageClasses config section.
Tom Clegg [Wed, 4 Aug 2021 20:57:35 +0000 (16:57 -0400)]
17967: Add StorageClasses config section.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17574-storage-classes-confirmed' into main
Tom Clegg [Wed, 4 Aug 2021 15:25:12 +0000 (11:25 -0400)]
Merge branch '17574-storage-classes-confirmed' into main

closes #17574

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Merge branch 'main'
Tom Clegg [Wed, 4 Aug 2021 14:02:15 +0000 (10:02 -0400)]
17574: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17591-doc-improvement'
Ward Vandewege [Tue, 3 Aug 2021 21:33:22 +0000 (17:33 -0400)]
Merge branch '17591-doc-improvement'

closes #17591

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch 'main' into 17591-doc-improvement
Ward Vandewege [Tue, 3 Aug 2021 21:32:59 +0000 (17:32 -0400)]
Merge branch 'main' into 17591-doc-improvement

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17591: address review feedback.
Ward Vandewege [Tue, 3 Aug 2021 21:26:30 +0000 (17:26 -0400)]
17591: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate the package repositories page, we now have Debian 11 (bullseye)
Ward Vandewege [Tue, 3 Aug 2021 21:02:37 +0000 (17:02 -0400)]
Update the package repositories page, we now have Debian 11 (bullseye)
pacakges.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Confirm storage classes as [] when replication is 0.
Tom Clegg [Tue, 3 Aug 2021 20:42:04 +0000 (16:42 -0400)]
17574: Confirm storage classes as [] when replication is 0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix map write when only RLock held.
Tom Clegg [Tue, 3 Aug 2021 15:46:20 +0000 (11:46 -0400)]
Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17591: be more explicit about the required TLS certificates and DNS
Ward Vandewege [Tue, 3 Aug 2021 15:15:20 +0000 (11:15 -0400)]
17591: be more explicit about the required TLS certificates and DNS
       hostnames.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd extra validation of our package version format in the build scripts,
Ward Vandewege [Fri, 30 Jul 2021 17:27:35 +0000 (13:27 -0400)]
Add extra validation of our package version format in the build scripts,
it must match

  ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+|)(~rc[0-9]+|~dev[0-9]+|)-[0-9]+$

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMore cleanup, remove references to Ubuntu 16.04 (xenial) which is no
Ward Vandewege [Thu, 29 Jul 2021 20:13:54 +0000 (16:13 -0400)]
More cleanup, remove references to Ubuntu 16.04 (xenial) which is no
longer supported.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDebian 11 (bullseye) will be a supported distribution in the next major
Ward Vandewege [Thu, 29 Jul 2021 20:12:44 +0000 (16:12 -0400)]
Debian 11 (bullseye) will be a supported distribution in the next major
Arvados release.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17949-add-bullseye'
Ward Vandewege [Thu, 29 Jul 2021 19:52:44 +0000 (15:52 -0400)]
Merge branch '17949-add-bullseye'

closes #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Fix map write when only RLock held.
Tom Clegg [Thu, 29 Jul 2021 18:40:10 +0000 (14:40 -0400)]
17574: Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17949: add support for Debian Bullseye in our build scripts. Remove some
Ward Vandewege [Thu, 29 Jul 2021 18:01:35 +0000 (14:01 -0400)]
17949: add support for Debian Bullseye in our build scripts. Remove some
       configuration for older, no longer supported distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17756-dispatch-lsf' into main
Tom Clegg [Thu, 29 Jul 2021 13:57:48 +0000 (09:57 -0400)]
Merge branch '17756-dispatch-lsf' into main

closes #17756

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix misleading errors after reaching update limit.
Tom Clegg [Thu, 29 Jul 2021 03:45:23 +0000 (23:45 -0400)]
17574: Fix misleading errors after reaching update limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update test.
Tom Clegg [Wed, 28 Jul 2021 22:23:05 +0000 (18:23 -0400)]
17574: Update test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix wait on empty channel.
Tom Clegg [Wed, 28 Jul 2021 22:22:08 +0000 (18:22 -0400)]
17574: Fix wait on empty channel.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update install/upgrade docs wrt PostgreSQL access.
Tom Clegg [Wed, 28 Jul 2021 20:20:33 +0000 (16:20 -0400)]
17574: Update install/upgrade docs wrt PostgreSQL access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Batch updates into transactions, skip when unchanged.
Tom Clegg [Wed, 28 Jul 2021 20:11:52 +0000 (16:11 -0400)]
17574: Batch updates into transactions, skip when unchanged.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix the example AWS InstanceTypes for the installer, none of these node
Ward Vandewege [Wed, 28 Jul 2021 19:28:17 +0000 (15:28 -0400)]
Fix the example AWS InstanceTypes for the installer, none of these node
types come with IncludedScratch.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17903-login-sync-centos7'
Ward Vandewege [Wed, 28 Jul 2021 15:11:50 +0000 (11:11 -0400)]
Merge branch '17903-login-sync-centos7'

closes #17903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17903: Merge branch 'main' into 17903-login-sync-centos7
Ward Vandewege [Wed, 28 Jul 2021 15:11:11 +0000 (11:11 -0400)]
17903: Merge branch 'main' into 17903-login-sync-centos7

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17830-reqid-header-propagation-fix' into main. Closes #17830
Lucas Di Pentima [Wed, 28 Jul 2021 13:43:07 +0000 (10:43 -0300)]
Merge branch '17830-reqid-header-propagation-fix' into main. Closes #17830

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17574: Process collections in a worker pool.
Tom Clegg [Wed, 28 Jul 2021 04:19:03 +0000 (00:19 -0400)]
17574: Process collections in a worker pool.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Add BalanceUpdateLimit config, fix tests.
Tom Clegg [Wed, 28 Jul 2021 04:18:53 +0000 (00:18 -0400)]
17574: Add BalanceUpdateLimit config, fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: add -pprof flag to service commands.
Tom Clegg [Wed, 28 Jul 2021 04:13:50 +0000 (00:13 -0400)]
17574: add -pprof flag to service commands.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Speed up manifest parsing.
Tom Clegg [Wed, 28 Jul 2021 04:04:56 +0000 (00:04 -0400)]
17574: Speed up manifest parsing.

Previous code spent a lot of time allocating, copying, and GCing
strings. Working with byte slices is much faser.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17830: Moves header setting code to cover more use cases.
Lucas Di Pentima [Tue, 27 Jul 2021 21:19:54 +0000 (18:19 -0300)]
17830: Moves header setting code to cover more use cases.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test cases & fixes one more issue.
Lucas Di Pentima [Tue, 27 Jul 2021 19:22:26 +0000 (16:22 -0300)]
17830: Adds test cases & fixes one more issue.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17903: use more distribution-agnostic commands to add/remove a user
Ward Vandewege [Tue, 27 Jul 2021 18:40:19 +0000 (14:40 -0400)]
17903: use more distribution-agnostic commands to add/remove a user
       to/from a group. This makes arvados-login-sync work on CentOS as
       well as Debian-based distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Add -update-confirmed-fields=false option.
Tom Clegg [Tue, 27 Jul 2021 15:23:48 +0000 (11:23 -0400)]
17574: Add -update-confirmed-fields=false option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Get collections directly from DB instead of controller.
Tom Clegg [Tue, 27 Jul 2021 15:16:49 +0000 (11:16 -0400)]
17574: Get collections directly from DB instead of controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665
Lucas Di Pentima [Tue, 27 Jul 2021 14:35:21 +0000 (11:35 -0300)]
Merge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17813-docker-to-singularity' into main
Tom Clegg [Mon, 26 Jul 2021 20:02:31 +0000 (16:02 -0400)]
Merge branch '17813-docker-to-singularity' into main

refs #17813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17813: Error out if sif cache collection cannot be created.
Tom Clegg [Mon, 26 Jul 2021 20:01:54 +0000 (16:01 -0400)]
17813: Error out if sif cache collection cannot be created.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update replication_confirmed fields after keep-balance run.
Tom Clegg [Mon, 26 Jul 2021 19:28:01 +0000 (15:28 -0400)]
17574: Update replication_confirmed fields after keep-balance run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix typo in documentation.
Ward Vandewege [Mon, 26 Jul 2021 19:09:44 +0000 (15:09 -0400)]
Fix typo in documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17388-arvcopy-storage-classes' into main. Closes #17388
Lucas Di Pentima [Mon, 26 Jul 2021 14:32:05 +0000 (11:32 -0300)]
Merge branch '17388-arvcopy-storage-classes' into main. Closes #17388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17388: Adds storage classes support to arv-copy.
Lucas Di Pentima [Fri, 23 Jul 2021 22:52:00 +0000 (19:52 -0300)]
17388: Adds storage classes support to arv-copy.

Also, updates tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Handle case where the cache collection update fails
Peter Amstutz [Fri, 23 Jul 2021 18:35:15 +0000 (14:35 -0400)]
17813: Handle case where the cache collection update fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Thu, 22 Jul 2021 21:32:44 +0000 (17:32 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Fixes tests.
Lucas Di Pentima [Thu, 22 Jul 2021 21:02:53 +0000 (18:02 -0300)]
16665: Fixes tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: continue refactor & fix tests
Peter Amstutz [Thu, 22 Jul 2021 20:44:19 +0000 (16:44 -0400)]
17813: continue refactor & fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Wraps error type to be able to differentiate it in a case switch.
Lucas Di Pentima [Thu, 22 Jul 2021 20:30:51 +0000 (17:30 -0300)]
16665: Wraps error type to be able to differentiate it in a case switch.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago16665: Exposes bug through updated test.
Lucas Di Pentima [Thu, 22 Jul 2021 20:24:26 +0000 (17:24 -0300)]
16665: Exposes bug through updated test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Refactor singularity image loading / caching / conversion
Peter Amstutz [Thu, 22 Jul 2021 20:22:39 +0000 (16:22 -0400)]
17813: Refactor singularity image loading / caching / conversion

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17830: Copies request's X-Request-Id header to response. Moves tests.
Lucas Di Pentima [Thu, 22 Jul 2021 18:31:11 +0000 (15:31 -0300)]
17830: Copies request's X-Request-Id header to response. Moves tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Sets up X-Request-Id if not provided by client. Adds it to the response.
Lucas Di Pentima [Wed, 21 Jul 2021 23:51:10 +0000 (20:51 -0300)]
17830: Sets up X-Request-Id if not provided by client. Adds it to the response.

Uses the IDGenerator to explicitly set one request id if needed, so that it
can return it to the client without the need to dig it out from the railsAPI
response.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds controller tests exposing the X-Request-Id header propagation bug.
Lucas Di Pentima [Tue, 20 Jul 2021 20:59:58 +0000 (17:59 -0300)]
17830: Adds controller tests exposing the X-Request-Id header propagation bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Customizes ActionDispatch::RequestId middleware to our needs.
Lucas Di Pentima [Tue, 20 Jul 2021 19:23:36 +0000 (16:23 -0300)]
17830: Customizes ActionDispatch::RequestId middleware to our needs.

Moves all the request id code to the proper middleware, and updates tests.
As functional tests are "unit tests for controllers", things that happen inside
middlewares need to be tested from integration tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test exposing incorrect X-Request-Id format on railsAPI.
Lucas Di Pentima [Tue, 20 Jul 2021 14:26:21 +0000 (11:26 -0300)]
17830: Adds test exposing incorrect X-Request-Id format on railsAPI.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17756: Fix test.
Tom Clegg [Thu, 22 Jul 2021 15:10:25 +0000 (11:10 -0400)]
17756: Fix test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Note that bsub sudo user must exist on worker nodes.
Tom Clegg [Thu, 22 Jul 2021 15:06:29 +0000 (11:06 -0400)]
17756: Note that bsub sudo user must exist on worker nodes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Move "no suitable instance type" reporting to dispatch lib.
Tom Clegg [Thu, 22 Jul 2021 14:36:14 +0000 (10:36 -0400)]
17756: Move "no suitable instance type" reporting to dispatch lib.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17863-bundler-update'
Ward Vandewege [Thu, 22 Jul 2021 14:27:45 +0000 (10:27 -0400)]
Merge branch '17863-bundler-update'

closes #17863

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17863: Merge branch 'main' into 17863-bundler-update
Ward Vandewege [Thu, 22 Jul 2021 14:25:59 +0000 (10:25 -0400)]
17863: Merge branch 'main' into 17863-bundler-update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17863: further cleanup in our rails postinst scripts and documentation:
Ward Vandewege [Thu, 22 Jul 2021 13:02:27 +0000 (09:02 -0400)]
17863: further cleanup in our rails postinst scripts and documentation:

* replace `bundle exec` calls with the appropriate binstub
* update deprecated use of `bundle install --path` to avoid warning

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17962-bump-lib-pq'
Ward Vandewege [Wed, 21 Jul 2021 12:46:46 +0000 (08:46 -0400)]
Merge branch '17962-bump-lib-pq'

closes #17962

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17962: Merge branch 'main' into 17962-bump-lib-pq
Ward Vandewege [Wed, 21 Jul 2021 12:46:12 +0000 (08:46 -0400)]
17962: Merge branch 'main' into 17962-bump-lib-pq

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17813: Change default value of runtime_auth_scopes to []
Peter Amstutz [Tue, 20 Jul 2021 21:01:24 +0000 (17:01 -0400)]
17813: Change default value of runtime_auth_scopes to []

It is documented to be an array of strings, so having the default be
an object just seems to be a mistake.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Tue, 20 Jul 2021 21:01:08 +0000 (17:01 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMake singularity suid in arvbox, crunch-dispatch-local runs as user
Peter Amstutz [Tue, 20 Jul 2021 20:28:30 +0000 (16:28 -0400)]
Make singularity suid in arvbox, crunch-dispatch-local runs as user

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17962: Bump the version of lib/pq to 1.10.2
Ward Vandewege [Tue, 20 Jul 2021 19:25:05 +0000 (15:25 -0400)]
17962: Bump the version of lib/pq to 1.10.2

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAnother doc tweak.
Ward Vandewege [Tue, 20 Jul 2021 17:08:46 +0000 (13:08 -0400)]
Another doc tweak.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>