arvados.git
3 years ago17417: switch our Rails codebases from therubyracer to mini_racer.
Ward Vandewege [Wed, 22 Dec 2021 20:47:08 +0000 (15:47 -0500)]
17417: switch our Rails codebases from therubyracer to mini_racer.
       Remove old dependency on `less` and `less-rails` from WB1.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18323-cwl-gpu' refs #18323
Peter Amstutz [Tue, 21 Dec 2021 19:26:12 +0000 (14:26 -0500)]
Merge branch '18323-cwl-gpu' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Document CUDARequirement
Peter Amstutz [Tue, 21 Dec 2021 19:24:55 +0000 (14:24 -0500)]
18323: Document CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18321-gpu-instancetype' refs #18321
Peter Amstutz [Tue, 21 Dec 2021 18:03:49 +0000 (13:03 -0500)]
Merge branch '18321-gpu-instancetype' refs #18321

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Fix tests
Peter Amstutz [Tue, 21 Dec 2021 17:00:28 +0000 (12:00 -0500)]
18321: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-cuda-azure-image'
Ward Vandewege [Tue, 21 Dec 2021 15:59:36 +0000 (10:59 -0500)]
Merge branch '18325-cuda-azure-image'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Check runtime constraints md5 with/without empty cuda section
Peter Amstutz [Mon, 20 Dec 2021 22:15:46 +0000 (17:15 -0500)]
18321: Check runtime constraints md5 with/without empty cuda section

* config & API check format of DriverVersion and HardwareCapability
* crunch-run only pays attention to CUDA.DeviceCount
* update docs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: adapt for naming differences in kernel/headers packages between
Ward Vandewege [Mon, 20 Dec 2021 15:32:20 +0000 (10:32 -0500)]
18325: adapt for naming differences in kernel/headers packages between
       Debian and Ubuntu.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18323: Fix the rest of the tests
Peter Amstutz [Mon, 20 Dec 2021 17:20:26 +0000 (12:20 -0500)]
18323: Fix the rest of the tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Test refactor WIP
Peter Amstutz [Mon, 20 Dec 2021 16:42:27 +0000 (11:42 -0500)]
18323: Test refactor WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: The nvidia apt repo link generation information at
Ward Vandewege [Mon, 20 Dec 2021 15:22:54 +0000 (10:22 -0500)]
18325: The nvidia apt repo link generation information at
       https://nvidia.github.io/libnvidia-container/ is wrong for
       Ubuntu, the dot needs to be stripped from the version number.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: also support building images with CUDA for Azure.
Ward Vandewege [Mon, 20 Dec 2021 15:01:23 +0000 (10:01 -0500)]
18325: also support building images with CUDA for Azure.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Account for CUDA in container reuse
Peter Amstutz [Fri, 17 Dec 2021 20:57:16 +0000 (15:57 -0500)]
18321: Account for CUDA in container reuse

Also ensure reuse across versions when CUDA isn't being used.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Add CUDARequirement to extension schema
Peter Amstutz [Fri, 17 Dec 2021 18:18:03 +0000 (13:18 -0500)]
18323: Add CUDARequirement to extension schema

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-compute-image-cuda'
Ward Vandewege [Fri, 17 Dec 2021 16:27:13 +0000 (11:27 -0500)]
Merge branch '18325-compute-image-cuda'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Add CUDARuntimeConstraints matching CUDAFeatures
Peter Amstutz [Fri, 17 Dec 2021 15:55:00 +0000 (10:55 -0500)]
18321: Add CUDARuntimeConstraints matching CUDAFeatures

Adjust CUDA version comparison.  Refactor tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
Ward Vandewege [Fri, 17 Dec 2021 14:49:33 +0000 (09:49 -0500)]
18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
       fix use of the $SUDO variable in the base packer script and add a
       final call to `apt-get clean` to it.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: no debian11 packages yet for libnvidia-container-tools, use
Ward Vandewege [Thu, 16 Dec 2021 21:05:45 +0000 (16:05 -0500)]
18325: no debian11 packages yet for libnvidia-container-tools, use
       workaround to install experimental rc1 with debian11 support,
       when on debian11.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: use a bigger root volume while building the AMI on AWS.
Ward Vandewege [Thu, 16 Dec 2021 19:46:36 +0000 (14:46 -0500)]
18325: use a bigger root volume while building the AMI on AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: add support for Nvidia GPUs as an optional feature to our compute
Ward Vandewege [Thu, 16 Dec 2021 19:12:49 +0000 (14:12 -0500)]
18325: add support for Nvidia GPUs as an optional feature to our compute
       node image builder script, for AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18590-fix-discovery-doc'
Ward Vandewege [Thu, 16 Dec 2021 15:39:11 +0000 (10:39 -0500)]
Merge branch '18590-fix-discovery-doc'

closes #18590

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: Merge branch 'main' into 18590-fix-discovery-doc
Ward Vandewege [Thu, 16 Dec 2021 15:38:14 +0000 (10:38 -0500)]
18590: Merge branch 'main' into 18590-fix-discovery-doc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: the default value for uuid in the groups/contents api endpoint
Ward Vandewege [Thu, 16 Dec 2021 00:22:49 +0000 (19:22 -0500)]
18590: the default value for uuid in the groups/contents api endpoint
       should not be expressed as `null` in the discovery doc, but as a
       string.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18301-wb1-cancel-button-fix'. Closes #18301.
Lucas Di Pentima [Wed, 15 Dec 2021 21:24:52 +0000 (18:24 -0300)]
Merge branch '18301-wb1-cancel-button-fix'. Closes #18301.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '18585-remove-java-sdk-v1'
Ward Vandewege [Wed, 15 Dec 2021 19:28:08 +0000 (14:28 -0500)]
Merge branch '18585-remove-java-sdk-v1'

closes #18585

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: Merge branch 'main' into 18585-remove-java-sdk-v1
Ward Vandewege [Wed, 15 Dec 2021 19:27:44 +0000 (14:27 -0500)]
18585: Merge branch 'main' into 18585-remove-java-sdk-v1

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: actually remove the old Java SDK.
Ward Vandewege [Wed, 15 Dec 2021 19:26:40 +0000 (14:26 -0500)]
18585: actually remove the old Java SDK.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18349-fed-request-id'
Tom Clegg [Wed, 15 Dec 2021 14:26:45 +0000 (09:26 -0500)]
Merge branch '18349-fed-request-id'

closes #18349

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Tue, 14 Dec 2021 18:13:36 +0000 (13:13 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18566: Update note with fixed singularity version.
Tom Clegg [Tue, 14 Dec 2021 18:12:47 +0000 (13:12 -0500)]
18566: Update note with fixed singularity version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18585: remove the old Java SDK, it has been deprecated for years and the
Ward Vandewege [Tue, 14 Dec 2021 16:51:27 +0000 (11:51 -0500)]
18585: remove the old Java SDK, it has been deprecated for years and the
       Java SDK v2 works much better.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18277-groups-visible-to-all'
Tom Clegg [Tue, 14 Dec 2021 16:26:14 +0000 (11:26 -0500)]
Merge branch '18277-groups-visible-to-all'

closes #18277

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18321: Fix config export
Peter Amstutz [Mon, 13 Dec 2021 18:47:28 +0000 (13:47 -0500)]
18321: Fix config export

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Incorporate CUDA request into picking a node type
Peter Amstutz [Mon, 13 Dec 2021 16:55:45 +0000 (11:55 -0500)]
18321: Incorporate CUDA request into picking a node type

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '12630-crunch-gpu' refs #12630
Peter Amstutz [Mon, 13 Dec 2021 20:17:41 +0000 (15:17 -0500)]
Merge branch '12630-crunch-gpu' refs #12630

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18277: Note configurable role visibility in permission model doc.
Tom Clegg [Mon, 13 Dec 2021 20:01:18 +0000 (15:01 -0500)]
18277: Note configurable role visibility in permission model doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test docker container config setup.
Tom Clegg [Mon, 13 Dec 2021 19:23:53 +0000 (14:23 -0500)]
12630: Test docker container config setup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test containerSpec setup and singularity exec args.
Tom Clegg [Mon, 13 Dec 2021 18:58:13 +0000 (13:58 -0500)]
12630: Test containerSpec setup and singularity exec args.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Update to docker client 19.03.15 for GPU support
Peter Amstutz [Fri, 10 Dec 2021 16:34:44 +0000 (11:34 -0500)]
12630: Update to docker client 19.03.15 for GPU support

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Remove distinction between cubin/PTX hardware capabilities
Peter Amstutz [Wed, 24 Nov 2021 20:26:46 +0000 (15:26 -0500)]
12630: Remove distinction between cubin/PTX hardware capabilities

Fix misspelled "HardwareCapability"

Add cuda_ fields runtime_constraints documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Use CUDADeviceCount instead of EnableCUDA
Peter Amstutz [Mon, 22 Nov 2021 18:35:20 +0000 (13:35 -0500)]
12630: Use CUDADeviceCount instead of EnableCUDA

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: first try at adding GPU support, no tests yet
Peter Amstutz [Mon, 22 Nov 2021 17:47:03 +0000 (12:47 -0500)]
12630: first try at adding GPU support, no tests yet

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18348-log-cluster-id'
Tom Clegg [Fri, 10 Dec 2021 16:19:52 +0000 (11:19 -0500)]
Merge branch '18348-log-cluster-id'

closes #18348

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Fri, 10 Dec 2021 16:19:29 +0000 (11:19 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18277: Add RoleGroupsVisibleToAll config, default true.
Tom Clegg [Fri, 10 Dec 2021 14:46:00 +0000 (09:46 -0500)]
18277: Add RoleGroupsVisibleToAll config, default true.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18566: Add note to install doc re singularity loop device bug/fix.
Tom Clegg [Fri, 10 Dec 2021 14:33:24 +0000 (09:33 -0500)]
18566: Add note to install doc re singularity loop device bug/fix.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18349: Confirm X-Request-Id propagates through federation.
Tom Clegg [Thu, 9 Dec 2021 19:00:48 +0000 (14:00 -0500)]
18349: Confirm X-Request-Id propagates through federation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18348: Add ClusterID field to service log entries.
Tom Clegg [Wed, 8 Dec 2021 19:51:13 +0000 (14:51 -0500)]
18348: Add ClusterID field to service log entries.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17339-s3aws-driver-memory-footprint'
Ward Vandewege [Tue, 7 Dec 2021 22:20:56 +0000 (17:20 -0500)]
Merge branch '17339-s3aws-driver-memory-footprint'

closes #17339

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: Merge branch 'main' into 17339-s3aws-driver-memory-footprint
Ward Vandewege [Tue, 7 Dec 2021 22:20:27 +0000 (17:20 -0500)]
17339: Merge branch 'main' into 17339-s3aws-driver-memory-footprint

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17667-doc-improvements'
Ward Vandewege [Tue, 7 Dec 2021 21:46:05 +0000 (16:46 -0500)]
Merge branch '17667-doc-improvements'

closes #17667

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17667: Merge branch 'main' into 17667-doc-improvements
Ward Vandewege [Tue, 7 Dec 2021 21:45:25 +0000 (16:45 -0500)]
17667: Merge branch 'main' into 17667-doc-improvements

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18547-use-volume-uuid-not-device-id'
Tom Clegg [Tue, 7 Dec 2021 20:07:30 +0000 (15:07 -0500)]
Merge branch '18547-use-volume-uuid-not-device-id'

fixes #18547

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17339: implement review feedback.
Ward Vandewege [Mon, 6 Dec 2021 22:01:26 +0000 (17:01 -0500)]
17339: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18301: Fixes the issue by adding 0 as one of the default values to be ignored.
Lucas Di Pentima [Tue, 7 Dec 2021 18:12:54 +0000 (15:12 -0300)]
18301: Fixes the issue by adding 0 as one of the default values to be ignored.

Also, pre-sort mounts' key lists before comparison, just in case.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18301: Adds test exposing the bug.
Lucas Di Pentima [Tue, 7 Dec 2021 18:11:29 +0000 (15:11 -0300)]
18301: Adds test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17667: a round of edits after review feedback.
Ward Vandewege [Tue, 7 Dec 2021 00:46:01 +0000 (19:46 -0500)]
17667: a round of edits after review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18547: Error out if two volumes return the same non-empty DeviceID.
Tom Clegg [Tue, 7 Dec 2021 00:32:34 +0000 (19:32 -0500)]
18547: Error out if two volumes return the same non-empty DeviceID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17667: add some missing information on the "Install prerequisites" page.
Ward Vandewege [Mon, 6 Dec 2021 17:48:57 +0000 (12:48 -0500)]
17667: add some missing information on the "Install prerequisites" page.
       Update the "Install PostgreSQL" page with a reference to Aurora
       RDS. Add a new page that details InternalURLs/ExternalURL.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: fix tests.
Ward Vandewege [Fri, 3 Dec 2021 21:29:49 +0000 (16:29 -0500)]
17339: fix tests.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: remove unnecessary memory allocation when reading from Keep with
Ward Vandewege [Fri, 3 Dec 2021 20:12:38 +0000 (15:12 -0500)]
17339: remove unnecessary memory allocation when reading from Keep with
       the S3 v2 driver. Also fix a few incorrect calls to log.Error()

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18376-nfs-readdirent'
Tom Clegg [Fri, 3 Dec 2021 16:30:23 +0000 (11:30 -0500)]
Merge branch '18376-nfs-readdirent'

refs #18376

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18547: Use volume UUID instead of DeviceID to deduplicate mounts.
Tom Clegg [Fri, 3 Dec 2021 15:54:59 +0000 (10:54 -0500)]
18547: Use volume UUID instead of DeviceID to deduplicate mounts.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18488-nonblocking-advisory-lock'
Tom Clegg [Fri, 3 Dec 2021 15:44:04 +0000 (10:44 -0500)]
Merge branch '18488-nonblocking-advisory-lock'

fixes #18488

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17339: remove unnecessary memory allocation when writing to Keep with
Ward Vandewege [Thu, 2 Dec 2021 20:59:18 +0000 (15:59 -0500)]
17339: remove unnecessary memory allocation when writing to Keep with
       the S3 v2 driver.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18511-java-sdk-groups-links' refs #18511
Peter Amstutz [Thu, 2 Dec 2021 15:31:27 +0000 (10:31 -0500)]
Merge branch '18511-java-sdk-groups-links' refs #18511

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18490-redundant-updates' refs #18490
Peter Amstutz [Wed, 1 Dec 2021 22:27:15 +0000 (17:27 -0500)]
Merge branch '18490-redundant-updates' refs #18490

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18490: Add comments.
Peter Amstutz [Wed, 1 Dec 2021 22:26:47 +0000 (17:26 -0500)]
18490: Add comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoDocumentation: fix AWS region name in the configuration reference and on
Ward Vandewege [Wed, 1 Dec 2021 20:48:54 +0000 (15:48 -0500)]
Documentation: fix AWS region name in the configuration reference and on
the S3 configuration page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge remote-tracking branch 'MajewskiKrzysztof/links-tail-uuid' into 18511-java...
Peter Amstutz [Wed, 1 Dec 2021 19:34:11 +0000 (14:34 -0500)]
Merge remote-tracking branch 'MajewskiKrzysztof/links-tail-uuid' into 18511-java-sdk-groups-links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge remote-tracking branch 'MajewskiKrzysztof/group-update' into 18511-java-sdk...
Peter Amstutz [Wed, 1 Dec 2021 19:33:30 +0000 (14:33 -0500)]
Merge remote-tracking branch 'MajewskiKrzysztof/group-update' into 18511-java-sdk-groups-links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd tails_uuid and tail_kind to Links API
majewsk6 [Wed, 1 Dec 2021 07:19:12 +0000 (08:19 +0100)]
Add tails_uuid and tail_kind to Links API

Arvados-DCO-1.1-Signed-off-by: Krzysztof Majewski majewski.kielce@gmail.com

3 years agoGroup update caused exception
majewsk6 [Wed, 1 Dec 2021 07:11:46 +0000 (08:11 +0100)]
Group update caused exception

Arvados-DCO-1.1-Signed-off-by: Krzysztof Majewski majewski.kielce@gmail.com

3 years agoMerge branch '18491-jwt-go-upgrade' into main. Closes #18491.
Lucas Di Pentima [Tue, 30 Nov 2021 15:59:12 +0000 (12:59 -0300)]
Merge branch '18491-jwt-go-upgrade' into main. Closes #18491.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18488: Close pg connection when waiting for lock.
Tom Clegg [Tue, 30 Nov 2021 15:57:08 +0000 (10:57 -0500)]
18488: Close pg connection when waiting for lock.

In practice, pg_advisory_lock() calls pile up and consume database
connection slots, even when the corresponding processes have exited.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18491: Removes an unnecessary check that made the test fail on module upgrade.
Lucas Di Pentima [Tue, 30 Nov 2021 15:37:00 +0000 (12:37 -0300)]
18491: Removes an unnecessary check that made the test fail on module upgrade.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18490: Add a where clause to avoid updating identical rows.
Peter Amstutz [Mon, 29 Nov 2021 21:15:23 +0000 (16:15 -0500)]
18490: Add a where clause to avoid updating identical rows.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18491: Upgrades github.com/Azure/go-autorest/autorest/azure/auth and its deps.
Lucas Di Pentima [Mon, 29 Nov 2021 19:19:38 +0000 (16:19 -0300)]
18491: Upgrades github.com/Azure/go-autorest/autorest/azure/auth and its deps.

Also, ran "go mod tidy" to remove unused package references.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17785-federated-token-regression' into main. Closes #17785
Lucas Di Pentima [Mon, 29 Nov 2021 15:57:57 +0000 (12:57 -0300)]
Merge branch '17785-federated-token-regression' into main. Closes #17785

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '18480-arvput-special-files-handling' into main. Closes #18480
Lucas Di Pentima [Fri, 26 Nov 2021 16:21:34 +0000 (13:21 -0300)]
Merge branch '18480-arvput-special-files-handling' into main. Closes #18480

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18480: Improves test by confirming that symlinked dirs aren't affected.
Lucas Di Pentima [Fri, 26 Nov 2021 15:56:32 +0000 (12:56 -0300)]
18480: Improves test by confirming that symlinked dirs aren't affected.

Also, terminate the producer process before doing any assertion to avoid
test's lockups.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18480: Applies the suggested patch, adjusts the tests to kill the producer.
Lucas Di Pentima [Fri, 26 Nov 2021 15:33:19 +0000 (12:33 -0300)]
18480: Applies the suggested patch, adjusts the tests to kill the producer.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18480: Adds test exposing the bug.
Lucas Di Pentima [Fri, 26 Nov 2021 15:20:47 +0000 (12:20 -0300)]
18480: Adds test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17785: Fixes ApiClientAuthorization.ExpiresAt type.
Lucas Di Pentima [Wed, 24 Nov 2021 20:12:36 +0000 (17:12 -0300)]
17785: Fixes ApiClientAuthorization.ExpiresAt type.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17785: Fixes ACA type so that it doesn't have pointers to string fields.
Lucas Di Pentima [Wed, 24 Nov 2021 19:53:02 +0000 (16:53 -0300)]
17785: Fixes ACA type so that it doesn't have pointers to string fields.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17785: Enhances test.
Lucas Di Pentima [Tue, 23 Nov 2021 22:20:11 +0000 (19:20 -0300)]
17785: Enhances test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17785: Controller forwards apiClientAuthorization requests to remotes.
Lucas Di Pentima [Mon, 22 Nov 2021 20:53:29 +0000 (17:53 -0300)]
17785: Controller forwards apiClientAuthorization requests to remotes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17785: Adds test exposing a problem when trying to handle other users' tokens.
Lucas Di Pentima [Wed, 8 Sep 2021 22:50:17 +0000 (19:50 -0300)]
17785: Adds test exposing a problem when trying to handle other users' tokens.

Trying to reproduce the "Login as user" feature from wb1 on a satellite
cluster: the admin user creates an api_client_authorization object owned
by the target user.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17840: Fix -list mode.
Tom Clegg [Mon, 22 Nov 2021 21:10:39 +0000 (16:10 -0500)]
17840: Fix -list mode.

Broken in 40f551004ab4e5f1d8ab02ddb55dca225ee8f6ac.

refs #17840

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18298-lsf-no-suitable-hosts'
Tom Clegg [Mon, 22 Nov 2021 18:45:21 +0000 (13:45 -0500)]
Merge branch '18298-lsf-no-suitable-hosts'

refs #18298

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18298: Add upgrade note re updated default BsubArgumentsList.
Tom Clegg [Mon, 22 Nov 2021 18:44:41 +0000 (13:44 -0500)]
18298: Add upgrade note re updated default BsubArgumentsList.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18308-enable-collection-versioning'
Ward Vandewege [Mon, 22 Nov 2021 14:49:47 +0000 (09:49 -0500)]
Merge branch '18308-enable-collection-versioning'

closes #18308

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18308: Merge branch 'main' into 18308-enable-collection-versioning
Ward Vandewege [Mon, 22 Nov 2021 14:49:22 +0000 (09:49 -0500)]
18308: Merge branch 'main' into 18308-enable-collection-versioning

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18298-lsf-no-suitable-hosts'
Tom Clegg [Fri, 19 Nov 2021 22:29:50 +0000 (17:29 -0500)]
Merge branch '18298-lsf-no-suitable-hosts'

refs #18298

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUsing curl against https requires the ca-certificates package, update
Ward Vandewege [Fri, 19 Nov 2021 01:15:07 +0000 (20:15 -0500)]
Using curl against https requires the ca-certificates package, update
the docs accordingly. Also remove the 'silent' flag from curl so that
errors are shown.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18298: Use bjobs select[] args, cancel on "no suitable host".
Tom Clegg [Thu, 18 Nov 2021 21:43:33 +0000 (16:43 -0500)]
18298: Use bjobs select[] args, cancel on "no suitable host".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18308: reduce the default value for PreserveVersionIfIdle to 10s.
Ward Vandewege [Thu, 18 Nov 2021 21:20:22 +0000 (16:20 -0500)]
18308: reduce the default value for PreserveVersionIfIdle to 10s.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18339-sweep-trash-lock'
Tom Clegg [Thu, 18 Nov 2021 20:01:06 +0000 (15:01 -0500)]
Merge branch '18339-sweep-trash-lock'

fixes #18339

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18339: Tweak upgrade note.
Tom Clegg [Thu, 18 Nov 2021 20:00:19 +0000 (15:00 -0500)]
18339: Tweak upgrade note.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18376: Retry up to 4 times on EBADCOOKIE from readdir.
Tom Clegg [Thu, 18 Nov 2021 19:31:01 +0000 (14:31 -0500)]
18376: Retry up to 4 times on EBADCOOKIE from readdir.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>