arvados.git
3 years ago17390: Document arv:OutputStorageClass extension
Peter Amstutz [Fri, 13 Aug 2021 18:11:33 +0000 (14:11 -0400)]
17390: Document arv:OutputStorageClass extension

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17390: Update arvados-cwl-runner docs
Peter Amstutz [Fri, 13 Aug 2021 14:33:34 +0000 (10:33 -0400)]
17390: Update arvados-cwl-runner docs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17390: Add unit tests for storage classes
Peter Amstutz [Thu, 12 Aug 2021 19:24:14 +0000 (15:24 -0400)]
17390: Add unit tests for storage classes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17390: Use cluster-configured default
Peter Amstutz [Tue, 10 Aug 2021 20:59:21 +0000 (16:59 -0400)]
17390: Use cluster-configured default

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17390: setting intermediate/final storage classes WIP
Peter Amstutz [Tue, 10 Aug 2021 20:54:40 +0000 (16:54 -0400)]
17390: setting intermediate/final storage classes WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17984-cwl-project-uuid-users' into main
Peter Amstutz [Fri, 13 Aug 2021 14:13:54 +0000 (10:13 -0400)]
Merge branch '17984-cwl-project-uuid-users' into main

refs #17984

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18013-token-rake-task-fix'
Ward Vandewege [Thu, 12 Aug 2021 20:22:51 +0000 (16:22 -0400)]
Merge branch '18013-token-rake-task-fix'

closes #18013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18013: Merge branch 'main' into 18013-token-rake-task-fix
Ward Vandewege [Thu, 12 Aug 2021 20:22:20 +0000 (16:22 -0400)]
18013: Merge branch 'main' into 18013-token-rake-task-fix

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18013: address review feedback.
Ward Vandewege [Thu, 12 Aug 2021 20:21:40 +0000 (16:21 -0400)]
18013: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17726-singularity-doc' into main
Tom Clegg [Thu, 12 Aug 2021 20:05:59 +0000 (16:05 -0400)]
Merge branch '17726-singularity-doc' into main

closes #17726

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17984: Fix tests
Peter Amstutz [Thu, 12 Aug 2021 19:43:18 +0000 (15:43 -0400)]
17984: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18013: the db:check_long_lived_tokens and db:fix_long_lived_tokens rake
Ward Vandewege [Thu, 12 Aug 2021 15:05:52 +0000 (11:05 -0400)]
18013: the db:check_long_lived_tokens and db:fix_long_lived_tokens rake
       tasks now take API.MaxTokenLifetime into account.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17947-centos7-package-conflict'
Ward Vandewege [Thu, 12 Aug 2021 13:07:06 +0000 (09:07 -0400)]
Merge branch '17947-centos7-package-conflict'

closes #17947

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17947: Merge branch 'main' into 17947-centos7-package-conflict
Ward Vandewege [Thu, 12 Aug 2021 13:06:36 +0000 (09:06 -0400)]
17947: Merge branch 'main' into 17947-centos7-package-conflict

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18005-collection-versioning-regression' into main. Closes #18004
Lucas Di Pentima [Wed, 11 Aug 2021 19:40:59 +0000 (16:40 -0300)]
Merge branch '18005-collection-versioning-regression' into main. Closes #18004

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17947: the Arvados python3-* CentOS packages should conflict with the
Ward Vandewege [Wed, 11 Aug 2021 19:29:07 +0000 (15:29 -0400)]
17947: the Arvados python3-* CentOS packages should conflict with the
       corresponding older rh-python36-* version.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18005: Fixes the bug.
Lucas Di Pentima [Wed, 11 Aug 2021 18:37:46 +0000 (15:37 -0300)]
18005: Fixes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18005: Exposes the bug by simulating a keep-balance run a making a new version.
Lucas Di Pentima [Wed, 11 Aug 2021 18:31:11 +0000 (15:31 -0300)]
18005: Exposes the bug by simulating a keep-balance run a making a new version.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17985-cwltool-binstub'
Ward Vandewege [Wed, 11 Aug 2021 15:20:24 +0000 (11:20 -0400)]
Merge branch '17985-cwltool-binstub'

closes #17985

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17985: fix binstub for cwltool in our arvados-cwl-runner package.
Ward Vandewege [Wed, 11 Aug 2021 15:19:35 +0000 (11:19 -0400)]
17985: fix binstub for cwltool in our arvados-cwl-runner package.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17726: Add Singularity info to user docs.
Tom Clegg [Tue, 10 Aug 2021 21:00:45 +0000 (17:00 -0400)]
17726: Add Singularity info to user docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17984: Improve error reporting a bit
Peter Amstutz [Tue, 10 Aug 2021 19:44:27 +0000 (15:44 -0400)]
17984: Improve error reporting a bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17984: validation accepts user uuid in --project-uuid
Peter Amstutz [Tue, 10 Aug 2021 19:29:45 +0000 (15:29 -0400)]
17984: validation accepts user uuid in --project-uuid

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17936-arvput-batch-mode' into main. Closes #17936.
Lucas Di Pentima [Tue, 10 Aug 2021 18:57:21 +0000 (15:57 -0300)]
Merge branch '17936-arvput-batch-mode' into main. Closes #17936.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17967-storage-classes-config' into main
Tom Clegg [Tue, 10 Aug 2021 14:43:25 +0000 (10:43 -0400)]
Merge branch '17967-storage-classes-config' into main

closes #17967

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17936: Updates arv-keepdocker to call arv-put with --batch argument.
Lucas Di Pentima [Mon, 9 Aug 2021 18:12:48 +0000 (15:12 -0300)]
17936: Updates arv-keepdocker to call arv-put with --batch argument.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17936: Adds --batch implementation, making the new test pass.
Lucas Di Pentima [Mon, 9 Aug 2021 17:58:41 +0000 (14:58 -0300)]
17936: Adds --batch implementation, making the new test pass.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Update generated config.
Tom Clegg [Mon, 9 Aug 2021 17:23:44 +0000 (13:23 -0400)]
17967: Update generated config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17936: Adds --batch argument to arv-put, updates test.
Lucas Di Pentima [Mon, 9 Aug 2021 16:56:44 +0000 (13:56 -0300)]
17936: Adds --batch argument to arv-put, updates test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Test new behavior for default storage_classes_desired.
Tom Clegg [Mon, 9 Aug 2021 15:29:10 +0000 (11:29 -0400)]
17967: Test new behavior for default storage_classes_desired.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Link to config doc page from conf file and upgrade notes.
Tom Clegg [Mon, 9 Aug 2021 15:11:32 +0000 (11:11 -0400)]
17967: Link to config doc page from conf file and upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: More detail about singularity/docker differences.
Tom Clegg [Mon, 9 Aug 2021 13:49:03 +0000 (09:49 -0400)]
17726: More detail about singularity/docker differences.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: Link singularity install docs. Add dispatch-lsf mention.
Tom Clegg [Mon, 9 Aug 2021 13:38:13 +0000 (09:38 -0400)]
17726: Link singularity install docs. Add dispatch-lsf mention.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Update storage classes docs.
Tom Clegg [Fri, 6 Aug 2021 14:19:59 +0000 (10:19 -0400)]
17967: Update storage classes docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Update tests.
Tom Clegg [Thu, 5 Aug 2021 21:57:50 +0000 (17:57 -0400)]
17967: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: Add singularity doc page.
Tom Clegg [Thu, 5 Aug 2021 19:14:57 +0000 (15:14 -0400)]
17726: Add singularity doc page.

Move Docker image collection format note from 2.2 to main (this change
was not in 2.2).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17952-no-nodes-link' into main closes #17952
Peter Amstutz [Thu, 5 Aug 2021 16:13:42 +0000 (12:13 -0400)]
Merge branch '17952-no-nodes-link' into main closes #17952

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Use StorageClasses.*.Default instead of ["default"].
Tom Clegg [Thu, 5 Aug 2021 15:04:37 +0000 (11:04 -0400)]
17967: Use StorageClasses.*.Default instead of ["default"].

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove tests for presence of nodes / disks menu item
Peter Amstutz [Thu, 5 Aug 2021 14:25:00 +0000 (10:25 -0400)]
17952: Remove tests for presence of nodes / disks menu item

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Read from volumes with high-priority storage classes first.
Tom Clegg [Thu, 5 Aug 2021 03:21:45 +0000 (23:21 -0400)]
17967: Read from volumes with high-priority storage classes first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove links to obsolete "nodes" and "keep disks"
Peter Amstutz [Wed, 4 Aug 2021 21:38:09 +0000 (17:38 -0400)]
17952: Remove links to obsolete "nodes" and "keep disks"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Add StorageClasses config section.
Tom Clegg [Wed, 4 Aug 2021 20:57:35 +0000 (16:57 -0400)]
17967: Add StorageClasses config section.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17574-storage-classes-confirmed' into main
Tom Clegg [Wed, 4 Aug 2021 15:25:12 +0000 (11:25 -0400)]
Merge branch '17574-storage-classes-confirmed' into main

closes #17574

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Merge branch 'main'
Tom Clegg [Wed, 4 Aug 2021 14:02:15 +0000 (10:02 -0400)]
17574: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17591-doc-improvement'
Ward Vandewege [Tue, 3 Aug 2021 21:33:22 +0000 (17:33 -0400)]
Merge branch '17591-doc-improvement'

closes #17591

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch 'main' into 17591-doc-improvement
Ward Vandewege [Tue, 3 Aug 2021 21:32:59 +0000 (17:32 -0400)]
Merge branch 'main' into 17591-doc-improvement

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17591: address review feedback.
Ward Vandewege [Tue, 3 Aug 2021 21:26:30 +0000 (17:26 -0400)]
17591: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate the package repositories page, we now have Debian 11 (bullseye)
Ward Vandewege [Tue, 3 Aug 2021 21:02:37 +0000 (17:02 -0400)]
Update the package repositories page, we now have Debian 11 (bullseye)
pacakges.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Confirm storage classes as [] when replication is 0.
Tom Clegg [Tue, 3 Aug 2021 20:42:04 +0000 (16:42 -0400)]
17574: Confirm storage classes as [] when replication is 0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix map write when only RLock held.
Tom Clegg [Tue, 3 Aug 2021 15:46:20 +0000 (11:46 -0400)]
Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17591: be more explicit about the required TLS certificates and DNS
Ward Vandewege [Tue, 3 Aug 2021 15:15:20 +0000 (11:15 -0400)]
17591: be more explicit about the required TLS certificates and DNS
       hostnames.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd extra validation of our package version format in the build scripts,
Ward Vandewege [Fri, 30 Jul 2021 17:27:35 +0000 (13:27 -0400)]
Add extra validation of our package version format in the build scripts,
it must match

  ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+|)(~rc[0-9]+|~dev[0-9]+|)-[0-9]+$

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMore cleanup, remove references to Ubuntu 16.04 (xenial) which is no
Ward Vandewege [Thu, 29 Jul 2021 20:13:54 +0000 (16:13 -0400)]
More cleanup, remove references to Ubuntu 16.04 (xenial) which is no
longer supported.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDebian 11 (bullseye) will be a supported distribution in the next major
Ward Vandewege [Thu, 29 Jul 2021 20:12:44 +0000 (16:12 -0400)]
Debian 11 (bullseye) will be a supported distribution in the next major
Arvados release.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17949-add-bullseye'
Ward Vandewege [Thu, 29 Jul 2021 19:52:44 +0000 (15:52 -0400)]
Merge branch '17949-add-bullseye'

closes #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Fix map write when only RLock held.
Tom Clegg [Thu, 29 Jul 2021 18:40:10 +0000 (14:40 -0400)]
17574: Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17949: add support for Debian Bullseye in our build scripts. Remove some
Ward Vandewege [Thu, 29 Jul 2021 18:01:35 +0000 (14:01 -0400)]
17949: add support for Debian Bullseye in our build scripts. Remove some
       configuration for older, no longer supported distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17756-dispatch-lsf' into main
Tom Clegg [Thu, 29 Jul 2021 13:57:48 +0000 (09:57 -0400)]
Merge branch '17756-dispatch-lsf' into main

closes #17756

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix misleading errors after reaching update limit.
Tom Clegg [Thu, 29 Jul 2021 03:45:23 +0000 (23:45 -0400)]
17574: Fix misleading errors after reaching update limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update test.
Tom Clegg [Wed, 28 Jul 2021 22:23:05 +0000 (18:23 -0400)]
17574: Update test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix wait on empty channel.
Tom Clegg [Wed, 28 Jul 2021 22:22:08 +0000 (18:22 -0400)]
17574: Fix wait on empty channel.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update install/upgrade docs wrt PostgreSQL access.
Tom Clegg [Wed, 28 Jul 2021 20:20:33 +0000 (16:20 -0400)]
17574: Update install/upgrade docs wrt PostgreSQL access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Batch updates into transactions, skip when unchanged.
Tom Clegg [Wed, 28 Jul 2021 20:11:52 +0000 (16:11 -0400)]
17574: Batch updates into transactions, skip when unchanged.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix the example AWS InstanceTypes for the installer, none of these node
Ward Vandewege [Wed, 28 Jul 2021 19:28:17 +0000 (15:28 -0400)]
Fix the example AWS InstanceTypes for the installer, none of these node
types come with IncludedScratch.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17903-login-sync-centos7'
Ward Vandewege [Wed, 28 Jul 2021 15:11:50 +0000 (11:11 -0400)]
Merge branch '17903-login-sync-centos7'

closes #17903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17903: Merge branch 'main' into 17903-login-sync-centos7
Ward Vandewege [Wed, 28 Jul 2021 15:11:11 +0000 (11:11 -0400)]
17903: Merge branch 'main' into 17903-login-sync-centos7

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17830-reqid-header-propagation-fix' into main. Closes #17830
Lucas Di Pentima [Wed, 28 Jul 2021 13:43:07 +0000 (10:43 -0300)]
Merge branch '17830-reqid-header-propagation-fix' into main. Closes #17830

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17574: Process collections in a worker pool.
Tom Clegg [Wed, 28 Jul 2021 04:19:03 +0000 (00:19 -0400)]
17574: Process collections in a worker pool.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Add BalanceUpdateLimit config, fix tests.
Tom Clegg [Wed, 28 Jul 2021 04:18:53 +0000 (00:18 -0400)]
17574: Add BalanceUpdateLimit config, fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: add -pprof flag to service commands.
Tom Clegg [Wed, 28 Jul 2021 04:13:50 +0000 (00:13 -0400)]
17574: add -pprof flag to service commands.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Speed up manifest parsing.
Tom Clegg [Wed, 28 Jul 2021 04:04:56 +0000 (00:04 -0400)]
17574: Speed up manifest parsing.

Previous code spent a lot of time allocating, copying, and GCing
strings. Working with byte slices is much faser.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17830: Moves header setting code to cover more use cases.
Lucas Di Pentima [Tue, 27 Jul 2021 21:19:54 +0000 (18:19 -0300)]
17830: Moves header setting code to cover more use cases.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test cases & fixes one more issue.
Lucas Di Pentima [Tue, 27 Jul 2021 19:22:26 +0000 (16:22 -0300)]
17830: Adds test cases & fixes one more issue.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17903: use more distribution-agnostic commands to add/remove a user
Ward Vandewege [Tue, 27 Jul 2021 18:40:19 +0000 (14:40 -0400)]
17903: use more distribution-agnostic commands to add/remove a user
       to/from a group. This makes arvados-login-sync work on CentOS as
       well as Debian-based distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Add -update-confirmed-fields=false option.
Tom Clegg [Tue, 27 Jul 2021 15:23:48 +0000 (11:23 -0400)]
17574: Add -update-confirmed-fields=false option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Get collections directly from DB instead of controller.
Tom Clegg [Tue, 27 Jul 2021 15:16:49 +0000 (11:16 -0400)]
17574: Get collections directly from DB instead of controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665
Lucas Di Pentima [Tue, 27 Jul 2021 14:35:21 +0000 (11:35 -0300)]
Merge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17813-docker-to-singularity' into main
Tom Clegg [Mon, 26 Jul 2021 20:02:31 +0000 (16:02 -0400)]
Merge branch '17813-docker-to-singularity' into main

refs #17813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17813: Error out if sif cache collection cannot be created.
Tom Clegg [Mon, 26 Jul 2021 20:01:54 +0000 (16:01 -0400)]
17813: Error out if sif cache collection cannot be created.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update replication_confirmed fields after keep-balance run.
Tom Clegg [Mon, 26 Jul 2021 19:28:01 +0000 (15:28 -0400)]
17574: Update replication_confirmed fields after keep-balance run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix typo in documentation.
Ward Vandewege [Mon, 26 Jul 2021 19:09:44 +0000 (15:09 -0400)]
Fix typo in documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17388-arvcopy-storage-classes' into main. Closes #17388
Lucas Di Pentima [Mon, 26 Jul 2021 14:32:05 +0000 (11:32 -0300)]
Merge branch '17388-arvcopy-storage-classes' into main. Closes #17388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17388: Adds storage classes support to arv-copy.
Lucas Di Pentima [Fri, 23 Jul 2021 22:52:00 +0000 (19:52 -0300)]
17388: Adds storage classes support to arv-copy.

Also, updates tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Handle case where the cache collection update fails
Peter Amstutz [Fri, 23 Jul 2021 18:35:15 +0000 (14:35 -0400)]
17813: Handle case where the cache collection update fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Thu, 22 Jul 2021 21:32:44 +0000 (17:32 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Fixes tests.
Lucas Di Pentima [Thu, 22 Jul 2021 21:02:53 +0000 (18:02 -0300)]
16665: Fixes tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: continue refactor & fix tests
Peter Amstutz [Thu, 22 Jul 2021 20:44:19 +0000 (16:44 -0400)]
17813: continue refactor & fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Wraps error type to be able to differentiate it in a case switch.
Lucas Di Pentima [Thu, 22 Jul 2021 20:30:51 +0000 (17:30 -0300)]
16665: Wraps error type to be able to differentiate it in a case switch.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago16665: Exposes bug through updated test.
Lucas Di Pentima [Thu, 22 Jul 2021 20:24:26 +0000 (17:24 -0300)]
16665: Exposes bug through updated test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Refactor singularity image loading / caching / conversion
Peter Amstutz [Thu, 22 Jul 2021 20:22:39 +0000 (16:22 -0400)]
17813: Refactor singularity image loading / caching / conversion

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17830: Copies request's X-Request-Id header to response. Moves tests.
Lucas Di Pentima [Thu, 22 Jul 2021 18:31:11 +0000 (15:31 -0300)]
17830: Copies request's X-Request-Id header to response. Moves tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Sets up X-Request-Id if not provided by client. Adds it to the response.
Lucas Di Pentima [Wed, 21 Jul 2021 23:51:10 +0000 (20:51 -0300)]
17830: Sets up X-Request-Id if not provided by client. Adds it to the response.

Uses the IDGenerator to explicitly set one request id if needed, so that it
can return it to the client without the need to dig it out from the railsAPI
response.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds controller tests exposing the X-Request-Id header propagation bug.
Lucas Di Pentima [Tue, 20 Jul 2021 20:59:58 +0000 (17:59 -0300)]
17830: Adds controller tests exposing the X-Request-Id header propagation bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Customizes ActionDispatch::RequestId middleware to our needs.
Lucas Di Pentima [Tue, 20 Jul 2021 19:23:36 +0000 (16:23 -0300)]
17830: Customizes ActionDispatch::RequestId middleware to our needs.

Moves all the request id code to the proper middleware, and updates tests.
As functional tests are "unit tests for controllers", things that happen inside
middlewares need to be tested from integration tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test exposing incorrect X-Request-Id format on railsAPI.
Lucas Di Pentima [Tue, 20 Jul 2021 14:26:21 +0000 (11:26 -0300)]
17830: Adds test exposing incorrect X-Request-Id format on railsAPI.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17756: Fix test.
Tom Clegg [Thu, 22 Jul 2021 15:10:25 +0000 (11:10 -0400)]
17756: Fix test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Note that bsub sudo user must exist on worker nodes.
Tom Clegg [Thu, 22 Jul 2021 15:06:29 +0000 (11:06 -0400)]
17756: Note that bsub sudo user must exist on worker nodes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Move "no suitable instance type" reporting to dispatch lib.
Tom Clegg [Thu, 22 Jul 2021 14:36:14 +0000 (10:36 -0400)]
17756: Move "no suitable instance type" reporting to dispatch lib.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17863-bundler-update'
Ward Vandewege [Thu, 22 Jul 2021 14:27:45 +0000 (10:27 -0400)]
Merge branch '17863-bundler-update'

closes #17863

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17863: Merge branch 'main' into 17863-bundler-update
Ward Vandewege [Thu, 22 Jul 2021 14:25:59 +0000 (10:25 -0400)]
17863: Merge branch 'main' into 17863-bundler-update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>