arvados.git
3 years ago17390: Use cluster-configured default
Peter Amstutz [Tue, 10 Aug 2021 20:59:21 +0000 (16:59 -0400)]
17390: Use cluster-configured default

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17390: setting intermediate/final storage classes WIP
Peter Amstutz [Tue, 10 Aug 2021 20:54:40 +0000 (16:54 -0400)]
17390: setting intermediate/final storage classes WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17984-cwl-project-uuid-users' into main
Peter Amstutz [Fri, 13 Aug 2021 14:13:54 +0000 (10:13 -0400)]
Merge branch '17984-cwl-project-uuid-users' into main

refs #17984

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18013-token-rake-task-fix'
Ward Vandewege [Thu, 12 Aug 2021 20:22:51 +0000 (16:22 -0400)]
Merge branch '18013-token-rake-task-fix'

closes #18013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18013: Merge branch 'main' into 18013-token-rake-task-fix
Ward Vandewege [Thu, 12 Aug 2021 20:22:20 +0000 (16:22 -0400)]
18013: Merge branch 'main' into 18013-token-rake-task-fix

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18013: address review feedback.
Ward Vandewege [Thu, 12 Aug 2021 20:21:40 +0000 (16:21 -0400)]
18013: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17726-singularity-doc' into main
Tom Clegg [Thu, 12 Aug 2021 20:05:59 +0000 (16:05 -0400)]
Merge branch '17726-singularity-doc' into main

closes #17726

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17984: Fix tests
Peter Amstutz [Thu, 12 Aug 2021 19:43:18 +0000 (15:43 -0400)]
17984: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoDisable SELinux on Centos-7 platform
Javier Bértoli [Thu, 12 Aug 2021 18:47:55 +0000 (15:47 -0300)]
Disable SELinux on Centos-7 platform

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago18013: the db:check_long_lived_tokens and db:fix_long_lived_tokens rake
Ward Vandewege [Thu, 12 Aug 2021 15:05:52 +0000 (11:05 -0400)]
18013: the db:check_long_lived_tokens and db:fix_long_lived_tokens rake
       tasks now take API.MaxTokenLifetime into account.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17947-centos7-package-conflict'
Ward Vandewege [Thu, 12 Aug 2021 13:07:06 +0000 (09:07 -0400)]
Merge branch '17947-centos7-package-conflict'

closes #17947

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17947: Merge branch 'main' into 17947-centos7-package-conflict
Ward Vandewege [Thu, 12 Aug 2021 13:06:36 +0000 (09:06 -0400)]
17947: Merge branch 'main' into 17947-centos7-package-conflict

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18004: Adds test exposing the race condition.
Lucas Di Pentima [Tue, 10 Aug 2021 15:15:04 +0000 (12:15 -0300)]
18004: Adds test exposing the race condition.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '18005-collection-versioning-regression' into main. Closes #18004
Lucas Di Pentima [Wed, 11 Aug 2021 19:40:59 +0000 (16:40 -0300)]
Merge branch '18005-collection-versioning-regression' into main. Closes #18004

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17947: the Arvados python3-* CentOS packages should conflict with the
Ward Vandewege [Wed, 11 Aug 2021 19:29:07 +0000 (15:29 -0400)]
17947: the Arvados python3-* CentOS packages should conflict with the
       corresponding older rh-python36-* version.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18005: Fixes the bug.
Lucas Di Pentima [Wed, 11 Aug 2021 18:37:46 +0000 (15:37 -0300)]
18005: Fixes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18005: Exposes the bug by simulating a keep-balance run a making a new version.
Lucas Di Pentima [Wed, 11 Aug 2021 18:31:11 +0000 (15:31 -0300)]
18005: Exposes the bug by simulating a keep-balance run a making a new version.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17985-cwltool-binstub'
Ward Vandewege [Wed, 11 Aug 2021 15:20:24 +0000 (11:20 -0400)]
Merge branch '17985-cwltool-binstub'

closes #17985

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17985: fix binstub for cwltool in our arvados-cwl-runner package.
Ward Vandewege [Wed, 11 Aug 2021 15:19:35 +0000 (11:19 -0400)]
17985: fix binstub for cwltool in our arvados-cwl-runner package.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18001: add Users/UserNotifierEmailBcc configuration option, which is the
Ward Vandewege [Tue, 10 Aug 2021 15:00:02 +0000 (11:00 -0400)]
18001: add Users/UserNotifierEmailBcc configuration option, which is the
       e-mail address that will be bcc'd on the new user welcome e-mail.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17726: Add Singularity info to user docs.
Tom Clegg [Tue, 10 Aug 2021 21:00:45 +0000 (17:00 -0400)]
17726: Add Singularity info to user docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17984: Improve error reporting a bit
Peter Amstutz [Tue, 10 Aug 2021 19:44:27 +0000 (15:44 -0400)]
17984: Improve error reporting a bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17984: validation accepts user uuid in --project-uuid
Peter Amstutz [Tue, 10 Aug 2021 19:29:45 +0000 (15:29 -0400)]
17984: validation accepts user uuid in --project-uuid

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17936-arvput-batch-mode' into main. Closes #17936.
Lucas Di Pentima [Tue, 10 Aug 2021 18:57:21 +0000 (15:57 -0300)]
Merge branch '17936-arvput-batch-mode' into main. Closes #17936.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17967-storage-classes-config' into main
Tom Clegg [Tue, 10 Aug 2021 14:43:25 +0000 (10:43 -0400)]
Merge branch '17967-storage-classes-config' into main

closes #17967

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMake Saltstack fail on first error
Javier Bértoli [Tue, 10 Aug 2021 11:20:31 +0000 (08:20 -0300)]
Make Saltstack fail on first error

no issue #
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoFix nginx_passenger parameters for Centos-7 and Ubuntu-18.04
Javier Bértoli [Mon, 9 Aug 2021 22:19:45 +0000 (19:19 -0300)]
Fix nginx_passenger parameters for Centos-7 and Ubuntu-18.04

refs #17535
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17936: Updates arv-keepdocker to call arv-put with --batch argument.
Lucas Di Pentima [Mon, 9 Aug 2021 18:12:48 +0000 (15:12 -0300)]
17936: Updates arv-keepdocker to call arv-put with --batch argument.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17936: Adds --batch implementation, making the new test pass.
Lucas Di Pentima [Mon, 9 Aug 2021 17:58:41 +0000 (14:58 -0300)]
17936: Adds --batch implementation, making the new test pass.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Update generated config.
Tom Clegg [Mon, 9 Aug 2021 17:23:44 +0000 (13:23 -0400)]
17967: Update generated config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17936: Adds --batch argument to arv-put, updates test.
Lucas Di Pentima [Mon, 9 Aug 2021 16:56:44 +0000 (13:56 -0300)]
17936: Adds --batch argument to arv-put, updates test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17967: Test new behavior for default storage_classes_desired.
Tom Clegg [Mon, 9 Aug 2021 15:29:10 +0000 (11:29 -0400)]
17967: Test new behavior for default storage_classes_desired.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Link to config doc page from conf file and upgrade notes.
Tom Clegg [Mon, 9 Aug 2021 15:11:32 +0000 (11:11 -0400)]
17967: Link to config doc page from conf file and upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: More detail about singularity/docker differences.
Tom Clegg [Mon, 9 Aug 2021 13:49:03 +0000 (09:49 -0400)]
17726: More detail about singularity/docker differences.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17726: Link singularity install docs. Add dispatch-lsf mention.
Tom Clegg [Mon, 9 Aug 2021 13:38:13 +0000 (09:38 -0400)]
17726: Link singularity install docs. Add dispatch-lsf mention.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17967: Update storage classes docs.
Tom Clegg [Fri, 6 Aug 2021 14:19:59 +0000 (10:19 -0400)]
17967: Update storage classes docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoSingularity requires squashfs-tools to build images.
Ward Vandewege [Fri, 6 Aug 2021 13:29:30 +0000 (09:29 -0400)]
Singularity requires squashfs-tools to build images.

refs #17755

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17967: Update tests.
Tom Clegg [Thu, 5 Aug 2021 21:57:50 +0000 (17:57 -0400)]
17967: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoDon't force the ruby version in the examples
Javier Bértoli [Thu, 5 Aug 2021 21:39:20 +0000 (18:39 -0300)]
Don't force the ruby version in the examples

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17726: Add singularity doc page.
Tom Clegg [Thu, 5 Aug 2021 19:14:57 +0000 (15:14 -0400)]
17726: Add singularity doc page.

Move Docker image collection format note from 2.2 to main (this change
was not in 2.2).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17952-no-nodes-link' into main closes #17952
Peter Amstutz [Thu, 5 Aug 2021 16:13:42 +0000 (12:13 -0400)]
Merge branch '17952-no-nodes-link' into main closes #17952

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdapt the provision scripts to manage Centos 7 too
Javier Bértoli [Thu, 5 Aug 2021 15:37:34 +0000 (12:37 -0300)]
Adapt the provision scripts to manage Centos 7 too

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoThe user's `api_client_authorization` token can't be obtained using filters
Javier Bértoli [Thu, 5 Aug 2021 15:29:55 +0000 (12:29 -0300)]
The user's `api_client_authorization` token can't be obtained using filters

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoSome more Centos required tweaks to the provision script.
Javier Bértoli [Thu, 5 Aug 2021 15:16:07 +0000 (12:16 -0300)]
Some more Centos required tweaks to the provision script.

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17967: Use StorageClasses.*.Default instead of ["default"].
Tom Clegg [Thu, 5 Aug 2021 15:04:37 +0000 (11:04 -0400)]
17967: Use StorageClasses.*.Default instead of ["default"].

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove tests for presence of nodes / disks menu item
Peter Amstutz [Thu, 5 Aug 2021 14:25:00 +0000 (10:25 -0400)]
17952: Remove tests for presence of nodes / disks menu item

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Read from volumes with high-priority storage classes first.
Tom Clegg [Thu, 5 Aug 2021 03:21:45 +0000 (23:21 -0400)]
17967: Read from volumes with high-priority storage classes first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17952: Remove links to obsolete "nodes" and "keep disks"
Peter Amstutz [Wed, 4 Aug 2021 21:38:09 +0000 (17:38 -0400)]
17952: Remove links to obsolete "nodes" and "keep disks"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17967: Add StorageClasses config section.
Tom Clegg [Wed, 4 Aug 2021 20:57:35 +0000 (16:57 -0400)]
17967: Add StorageClasses config section.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17574-storage-classes-confirmed' into main
Tom Clegg [Wed, 4 Aug 2021 15:25:12 +0000 (11:25 -0400)]
Merge branch '17574-storage-classes-confirmed' into main

closes #17574

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Merge branch 'main'
Tom Clegg [Wed, 4 Aug 2021 14:02:15 +0000 (10:02 -0400)]
17574: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17591-doc-improvement'
Ward Vandewege [Tue, 3 Aug 2021 21:33:22 +0000 (17:33 -0400)]
Merge branch '17591-doc-improvement'

closes #17591

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch 'main' into 17591-doc-improvement
Ward Vandewege [Tue, 3 Aug 2021 21:32:59 +0000 (17:32 -0400)]
Merge branch 'main' into 17591-doc-improvement

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17591: address review feedback.
Ward Vandewege [Tue, 3 Aug 2021 21:26:30 +0000 (17:26 -0400)]
17591: address review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate the package repositories page, we now have Debian 11 (bullseye)
Ward Vandewege [Tue, 3 Aug 2021 21:02:37 +0000 (17:02 -0400)]
Update the package repositories page, we now have Debian 11 (bullseye)
pacakges.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Confirm storage classes as [] when replication is 0.
Tom Clegg [Tue, 3 Aug 2021 20:42:04 +0000 (16:42 -0400)]
17574: Confirm storage classes as [] when replication is 0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix map write when only RLock held.
Tom Clegg [Tue, 3 Aug 2021 15:46:20 +0000 (11:46 -0400)]
Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17591: be more explicit about the required TLS certificates and DNS
Ward Vandewege [Tue, 3 Aug 2021 15:15:20 +0000 (11:15 -0400)]
17591: be more explicit about the required TLS certificates and DNS
       hostnames.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd extra validation of our package version format in the build scripts,
Ward Vandewege [Fri, 30 Jul 2021 17:27:35 +0000 (13:27 -0400)]
Add extra validation of our package version format in the build scripts,
it must match

  ^[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+|)(~rc[0-9]+|~dev[0-9]+|)-[0-9]+$

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMore cleanup, remove references to Ubuntu 16.04 (xenial) which is no
Ward Vandewege [Thu, 29 Jul 2021 20:13:54 +0000 (16:13 -0400)]
More cleanup, remove references to Ubuntu 16.04 (xenial) which is no
longer supported.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDebian 11 (bullseye) will be a supported distribution in the next major
Ward Vandewege [Thu, 29 Jul 2021 20:12:44 +0000 (16:12 -0400)]
Debian 11 (bullseye) will be a supported distribution in the next major
Arvados release.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17949-add-bullseye'
Ward Vandewege [Thu, 29 Jul 2021 19:52:44 +0000 (15:52 -0400)]
Merge branch '17949-add-bullseye'

closes #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Fix map write when only RLock held.
Tom Clegg [Thu, 29 Jul 2021 18:40:10 +0000 (14:40 -0400)]
17574: Fix map write when only RLock held.

Similar to dd8ae798774d29b35532b99b78cae67a151654be.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17949: add support for Debian Bullseye in our build scripts. Remove some
Ward Vandewege [Thu, 29 Jul 2021 18:01:35 +0000 (14:01 -0400)]
17949: add support for Debian Bullseye in our build scripts. Remove some
       configuration for older, no longer supported distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17756-dispatch-lsf' into main
Tom Clegg [Thu, 29 Jul 2021 13:57:48 +0000 (09:57 -0400)]
Merge branch '17756-dispatch-lsf' into main

closes #17756

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix misleading errors after reaching update limit.
Tom Clegg [Thu, 29 Jul 2021 03:45:23 +0000 (23:45 -0400)]
17574: Fix misleading errors after reaching update limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update test.
Tom Clegg [Wed, 28 Jul 2021 22:23:05 +0000 (18:23 -0400)]
17574: Update test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Fix wait on empty channel.
Tom Clegg [Wed, 28 Jul 2021 22:22:08 +0000 (18:22 -0400)]
17574: Fix wait on empty channel.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update install/upgrade docs wrt PostgreSQL access.
Tom Clegg [Wed, 28 Jul 2021 20:20:33 +0000 (16:20 -0400)]
17574: Update install/upgrade docs wrt PostgreSQL access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Batch updates into transactions, skip when unchanged.
Tom Clegg [Wed, 28 Jul 2021 20:11:52 +0000 (16:11 -0400)]
17574: Batch updates into transactions, skip when unchanged.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix the example AWS InstanceTypes for the installer, none of these node
Ward Vandewege [Wed, 28 Jul 2021 19:28:17 +0000 (15:28 -0400)]
Fix the example AWS InstanceTypes for the installer, none of these node
types come with IncludedScratch.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17903-login-sync-centos7'
Ward Vandewege [Wed, 28 Jul 2021 15:11:50 +0000 (11:11 -0400)]
Merge branch '17903-login-sync-centos7'

closes #17903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17903: Merge branch 'main' into 17903-login-sync-centos7
Ward Vandewege [Wed, 28 Jul 2021 15:11:11 +0000 (11:11 -0400)]
17903: Merge branch 'main' into 17903-login-sync-centos7

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17830-reqid-header-propagation-fix' into main. Closes #17830
Lucas Di Pentima [Wed, 28 Jul 2021 13:43:07 +0000 (10:43 -0300)]
Merge branch '17830-reqid-header-propagation-fix' into main. Closes #17830

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17574: Process collections in a worker pool.
Tom Clegg [Wed, 28 Jul 2021 04:19:03 +0000 (00:19 -0400)]
17574: Process collections in a worker pool.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Add BalanceUpdateLimit config, fix tests.
Tom Clegg [Wed, 28 Jul 2021 04:18:53 +0000 (00:18 -0400)]
17574: Add BalanceUpdateLimit config, fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: add -pprof flag to service commands.
Tom Clegg [Wed, 28 Jul 2021 04:13:50 +0000 (00:13 -0400)]
17574: add -pprof flag to service commands.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Speed up manifest parsing.
Tom Clegg [Wed, 28 Jul 2021 04:04:56 +0000 (00:04 -0400)]
17574: Speed up manifest parsing.

Previous code spent a lot of time allocating, copying, and GCing
strings. Working with byte slices is much faser.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17830: Moves header setting code to cover more use cases.
Lucas Di Pentima [Tue, 27 Jul 2021 21:19:54 +0000 (18:19 -0300)]
17830: Moves header setting code to cover more use cases.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test cases & fixes one more issue.
Lucas Di Pentima [Tue, 27 Jul 2021 19:22:26 +0000 (16:22 -0300)]
17830: Adds test cases & fixes one more issue.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17903: use more distribution-agnostic commands to add/remove a user
Ward Vandewege [Tue, 27 Jul 2021 18:40:19 +0000 (14:40 -0400)]
17903: use more distribution-agnostic commands to add/remove a user
       to/from a group. This makes arvados-login-sync work on CentOS as
       well as Debian-based distributions.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17574: Add -update-confirmed-fields=false option.
Tom Clegg [Tue, 27 Jul 2021 15:23:48 +0000 (11:23 -0400)]
17574: Add -update-confirmed-fields=false option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Get collections directly from DB instead of controller.
Tom Clegg [Tue, 27 Jul 2021 15:16:49 +0000 (11:16 -0400)]
17574: Get collections directly from DB instead of controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665
Lucas Di Pentima [Tue, 27 Jul 2021 14:35:21 +0000 (11:35 -0300)]
Merge branch '16665-keepproxy-spurious-413-status' into main. Closes #16665

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17813-docker-to-singularity' into main
Tom Clegg [Mon, 26 Jul 2021 20:02:31 +0000 (16:02 -0400)]
Merge branch '17813-docker-to-singularity' into main

refs #17813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17813: Error out if sif cache collection cannot be created.
Tom Clegg [Mon, 26 Jul 2021 20:01:54 +0000 (16:01 -0400)]
17813: Error out if sif cache collection cannot be created.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17574: Update replication_confirmed fields after keep-balance run.
Tom Clegg [Mon, 26 Jul 2021 19:28:01 +0000 (15:28 -0400)]
17574: Update replication_confirmed fields after keep-balance run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix typo in documentation.
Ward Vandewege [Mon, 26 Jul 2021 19:09:44 +0000 (15:09 -0400)]
Fix typo in documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17388-arvcopy-storage-classes' into main. Closes #17388
Lucas Di Pentima [Mon, 26 Jul 2021 14:32:05 +0000 (11:32 -0300)]
Merge branch '17388-arvcopy-storage-classes' into main. Closes #17388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17388: Adds storage classes support to arv-copy.
Lucas Di Pentima [Fri, 23 Jul 2021 22:52:00 +0000 (19:52 -0300)]
17388: Adds storage classes support to arv-copy.

Also, updates tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Handle case where the cache collection update fails
Peter Amstutz [Fri, 23 Jul 2021 18:35:15 +0000 (14:35 -0400)]
17813: Handle case where the cache collection update fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17755: Merge branch 'main' into 17755-add-singularity-to-compute-image
Ward Vandewege [Fri, 23 Jul 2021 18:04:16 +0000 (14:04 -0400)]
17755: Merge branch 'main' into 17755-add-singularity-to-compute-image

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17535: Modify provision script to run on Centos
Javier Bértoli [Fri, 23 Jul 2021 18:02:45 +0000 (15:02 -0300)]
17535: Modify provision script to run on Centos

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Thu, 22 Jul 2021 21:32:44 +0000 (17:32 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Fixes tests.
Lucas Di Pentima [Thu, 22 Jul 2021 21:02:53 +0000 (18:02 -0300)]
16665: Fixes tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: continue refactor & fix tests
Peter Amstutz [Thu, 22 Jul 2021 20:44:19 +0000 (16:44 -0400)]
17813: continue refactor & fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Wraps error type to be able to differentiate it in a case switch.
Lucas Di Pentima [Thu, 22 Jul 2021 20:30:51 +0000 (17:30 -0300)]
16665: Wraps error type to be able to differentiate it in a case switch.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago16665: Exposes bug through updated test.
Lucas Di Pentima [Thu, 22 Jul 2021 20:24:26 +0000 (17:24 -0300)]
16665: Exposes bug through updated test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Refactor singularity image loading / caching / conversion
Peter Amstutz [Thu, 22 Jul 2021 20:22:39 +0000 (16:22 -0400)]
17813: Refactor singularity image loading / caching / conversion

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17830: Copies request's X-Request-Id header to response. Moves tests.
Lucas Di Pentima [Thu, 22 Jul 2021 18:31:11 +0000 (15:31 -0300)]
17830: Copies request's X-Request-Id header to response. Moves tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>