arvados.git
2 years agoMerge branch '18323-cwl-gpu2' refs #18323
Peter Amstutz [Wed, 5 Jan 2022 21:50:23 +0000 (16:50 -0500)]
Merge branch '18323-cwl-gpu2' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoRemove one more use of `go get` from run-library.sh.
Ward Vandewege [Wed, 5 Jan 2022 19:56:03 +0000 (14:56 -0500)]
Remove one more use of `go get` from run-library.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 21:06:28 +0000 (16:06 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 20:05:25 +0000 (15:05 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix to use cwltool:CUDARequirement
Peter Amstutz [Wed, 5 Jan 2022 20:03:30 +0000 (15:03 -0500)]
18323: Fix to use cwltool:CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoRemove deprecated use of `go get` from run-tests.sh. This should solve
Ward Vandewege [Wed, 5 Jan 2022 16:56:03 +0000 (11:56 -0500)]
Remove deprecated use of `go get` from run-tests.sh. This should solve
the occasional surprise modifications of go.mod/go.sum.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18562-preemptible'
Tom Clegg [Thu, 30 Dec 2021 19:00:12 +0000 (14:00 -0500)]
Merge branch '18562-preemptible'

fixes #18562

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18604-embed-config-default'
Tom Clegg [Thu, 30 Dec 2021 15:23:29 +0000 (10:23 -0500)]
Merge branch '18604-embed-config-default'

closes #18604

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17417-fix-wb1'
Ward Vandewege [Wed, 29 Dec 2021 22:53:52 +0000 (17:53 -0500)]
Merge branch '17417-fix-wb1'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18604: Require go1.17, use go:embed feature for config.default.yml.
Tom Clegg [Wed, 29 Dec 2021 16:14:14 +0000 (11:14 -0500)]
18604: Require go1.17, use go:embed feature for config.default.yml.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: remove the coffee-rails gem dependency in wb1, and convert the
Ward Vandewege [Wed, 29 Dec 2021 02:41:54 +0000 (21:41 -0500)]
17417: remove the coffee-rails gem dependency in wb1, and convert the
       two files where we used it to straight up javascript.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18562: Add upgrade note about new config key.
Tom Clegg [Mon, 27 Dec 2021 15:51:52 +0000 (10:51 -0500)]
18562: Add upgrade note about new config key.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: make sure that there is enough of a config.yml file for the wb1
Ward Vandewege [Fri, 24 Dec 2021 20:56:26 +0000 (15:56 -0500)]
17417: make sure that there is enough of a config.yml file for the wb1
       package install test to find all potential issues.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18562: Rename config to AlwaysUsePreemptibleInstances.
Tom Clegg [Thu, 23 Dec 2021 21:59:12 +0000 (16:59 -0500)]
18562: Rename config to AlwaysUsePreemptibleInstances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18562: Update doc and config comments for UsePreemptibleInstances.
Tom Clegg [Thu, 23 Dec 2021 21:51:06 +0000 (16:51 -0500)]
18562: Update doc and config comments for UsePreemptibleInstances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17417-goodbye-to-mini_racer'
Ward Vandewege [Thu, 23 Dec 2021 21:21:08 +0000 (16:21 -0500)]
Merge branch '17417-goodbye-to-mini_racer'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: remove more unused gems from the API server gemfile.
Ward Vandewege [Thu, 23 Dec 2021 20:36:00 +0000 (15:36 -0500)]
17417: remove more unused gems from the API server gemfile.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18562: Auto-enable preemptible when changing state to Committed.
Tom Clegg [Thu, 23 Dec 2021 15:54:07 +0000 (10:54 -0500)]
18562: Auto-enable preemptible when changing state to Committed.

(As opposed to during any update.)

Avoids a situation where a client creates a child CR with
preemptible=false, preemptible is automatically changed to true, but
then preemptible instances are removed from config, and the client
cannot change the CR state to committed without also explicitly
clearing the preemptible flag, because the automatically-enabled
preemptible flag is not allowed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17417: Testing if execjs is really needed.
Lucas Di Pentima [Thu, 23 Dec 2021 14:29:30 +0000 (11:29 -0300)]
17417: Testing if execjs is really needed.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18562: Fix UsePreemptibleInstances behavior.
Tom Clegg [Wed, 22 Dec 2021 22:49:23 +0000 (17:49 -0500)]
18562: Fix UsePreemptibleInstances behavior.

* Do not automatically set preemptible=true if there are no
  preemptible instance types available.

* Do not automatically set preemptible=true on a container request
  that has already been committed with preemptible=false.

* Do not reject updates to existing container requests with
  preemptible=true just because config has since changed and no longer
  enables it automatically.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18323: Revise in line with the cwltool version of this feature
Peter Amstutz [Wed, 22 Dec 2021 21:27:06 +0000 (16:27 -0500)]
18323: Revise in line with the cwltool version of this feature

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17417: switch our Rails codebases from therubyracer to mini_racer.
Ward Vandewege [Wed, 22 Dec 2021 20:47:08 +0000 (15:47 -0500)]
17417: switch our Rails codebases from therubyracer to mini_racer.
       Remove old dependency on `less` and `less-rails` from WB1.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18323-cwl-gpu' refs #18323
Peter Amstutz [Tue, 21 Dec 2021 19:26:12 +0000 (14:26 -0500)]
Merge branch '18323-cwl-gpu' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Document CUDARequirement
Peter Amstutz [Tue, 21 Dec 2021 19:24:55 +0000 (14:24 -0500)]
18323: Document CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18321-gpu-instancetype' refs #18321
Peter Amstutz [Tue, 21 Dec 2021 18:03:49 +0000 (13:03 -0500)]
Merge branch '18321-gpu-instancetype' refs #18321

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Fix tests
Peter Amstutz [Tue, 21 Dec 2021 17:00:28 +0000 (12:00 -0500)]
18321: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-cuda-azure-image'
Ward Vandewege [Tue, 21 Dec 2021 15:59:36 +0000 (10:59 -0500)]
Merge branch '18325-cuda-azure-image'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Check runtime constraints md5 with/without empty cuda section
Peter Amstutz [Mon, 20 Dec 2021 22:15:46 +0000 (17:15 -0500)]
18321: Check runtime constraints md5 with/without empty cuda section

* config & API check format of DriverVersion and HardwareCapability
* crunch-run only pays attention to CUDA.DeviceCount
* update docs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: adapt for naming differences in kernel/headers packages between
Ward Vandewege [Mon, 20 Dec 2021 15:32:20 +0000 (10:32 -0500)]
18325: adapt for naming differences in kernel/headers packages between
       Debian and Ubuntu.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18323: Fix the rest of the tests
Peter Amstutz [Mon, 20 Dec 2021 17:20:26 +0000 (12:20 -0500)]
18323: Fix the rest of the tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Test refactor WIP
Peter Amstutz [Mon, 20 Dec 2021 16:42:27 +0000 (11:42 -0500)]
18323: Test refactor WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: The nvidia apt repo link generation information at
Ward Vandewege [Mon, 20 Dec 2021 15:22:54 +0000 (10:22 -0500)]
18325: The nvidia apt repo link generation information at
       https://nvidia.github.io/libnvidia-container/ is wrong for
       Ubuntu, the dot needs to be stripped from the version number.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: also support building images with CUDA for Azure.
Ward Vandewege [Mon, 20 Dec 2021 15:01:23 +0000 (10:01 -0500)]
18325: also support building images with CUDA for Azure.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Account for CUDA in container reuse
Peter Amstutz [Fri, 17 Dec 2021 20:57:16 +0000 (15:57 -0500)]
18321: Account for CUDA in container reuse

Also ensure reuse across versions when CUDA isn't being used.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Add CUDARequirement to extension schema
Peter Amstutz [Fri, 17 Dec 2021 18:18:03 +0000 (13:18 -0500)]
18323: Add CUDARequirement to extension schema

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-compute-image-cuda'
Ward Vandewege [Fri, 17 Dec 2021 16:27:13 +0000 (11:27 -0500)]
Merge branch '18325-compute-image-cuda'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Add CUDARuntimeConstraints matching CUDAFeatures
Peter Amstutz [Fri, 17 Dec 2021 15:55:00 +0000 (10:55 -0500)]
18321: Add CUDARuntimeConstraints matching CUDAFeatures

Adjust CUDA version comparison.  Refactor tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
Ward Vandewege [Fri, 17 Dec 2021 14:49:33 +0000 (09:49 -0500)]
18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
       fix use of the $SUDO variable in the base packer script and add a
       final call to `apt-get clean` to it.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: no debian11 packages yet for libnvidia-container-tools, use
Ward Vandewege [Thu, 16 Dec 2021 21:05:45 +0000 (16:05 -0500)]
18325: no debian11 packages yet for libnvidia-container-tools, use
       workaround to install experimental rc1 with debian11 support,
       when on debian11.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: use a bigger root volume while building the AMI on AWS.
Ward Vandewege [Thu, 16 Dec 2021 19:46:36 +0000 (14:46 -0500)]
18325: use a bigger root volume while building the AMI on AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: add support for Nvidia GPUs as an optional feature to our compute
Ward Vandewege [Thu, 16 Dec 2021 19:12:49 +0000 (14:12 -0500)]
18325: add support for Nvidia GPUs as an optional feature to our compute
       node image builder script, for AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18590-fix-discovery-doc'
Ward Vandewege [Thu, 16 Dec 2021 15:39:11 +0000 (10:39 -0500)]
Merge branch '18590-fix-discovery-doc'

closes #18590

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: Merge branch 'main' into 18590-fix-discovery-doc
Ward Vandewege [Thu, 16 Dec 2021 15:38:14 +0000 (10:38 -0500)]
18590: Merge branch 'main' into 18590-fix-discovery-doc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: the default value for uuid in the groups/contents api endpoint
Ward Vandewege [Thu, 16 Dec 2021 00:22:49 +0000 (19:22 -0500)]
18590: the default value for uuid in the groups/contents api endpoint
       should not be expressed as `null` in the discovery doc, but as a
       string.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18301-wb1-cancel-button-fix'. Closes #18301.
Lucas Di Pentima [Wed, 15 Dec 2021 21:24:52 +0000 (18:24 -0300)]
Merge branch '18301-wb1-cancel-button-fix'. Closes #18301.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '18585-remove-java-sdk-v1'
Ward Vandewege [Wed, 15 Dec 2021 19:28:08 +0000 (14:28 -0500)]
Merge branch '18585-remove-java-sdk-v1'

closes #18585

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: Merge branch 'main' into 18585-remove-java-sdk-v1
Ward Vandewege [Wed, 15 Dec 2021 19:27:44 +0000 (14:27 -0500)]
18585: Merge branch 'main' into 18585-remove-java-sdk-v1

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: actually remove the old Java SDK.
Ward Vandewege [Wed, 15 Dec 2021 19:26:40 +0000 (14:26 -0500)]
18585: actually remove the old Java SDK.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18349-fed-request-id'
Tom Clegg [Wed, 15 Dec 2021 14:26:45 +0000 (09:26 -0500)]
Merge branch '18349-fed-request-id'

closes #18349

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Tue, 14 Dec 2021 18:13:36 +0000 (13:13 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18566: Update note with fixed singularity version.
Tom Clegg [Tue, 14 Dec 2021 18:12:47 +0000 (13:12 -0500)]
18566: Update note with fixed singularity version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18585: remove the old Java SDK, it has been deprecated for years and the
Ward Vandewege [Tue, 14 Dec 2021 16:51:27 +0000 (11:51 -0500)]
18585: remove the old Java SDK, it has been deprecated for years and the
       Java SDK v2 works much better.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18277-groups-visible-to-all'
Tom Clegg [Tue, 14 Dec 2021 16:26:14 +0000 (11:26 -0500)]
Merge branch '18277-groups-visible-to-all'

closes #18277

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18321: Fix config export
Peter Amstutz [Mon, 13 Dec 2021 18:47:28 +0000 (13:47 -0500)]
18321: Fix config export

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Incorporate CUDA request into picking a node type
Peter Amstutz [Mon, 13 Dec 2021 16:55:45 +0000 (11:55 -0500)]
18321: Incorporate CUDA request into picking a node type

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '12630-crunch-gpu' refs #12630
Peter Amstutz [Mon, 13 Dec 2021 20:17:41 +0000 (15:17 -0500)]
Merge branch '12630-crunch-gpu' refs #12630

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18277: Note configurable role visibility in permission model doc.
Tom Clegg [Mon, 13 Dec 2021 20:01:18 +0000 (15:01 -0500)]
18277: Note configurable role visibility in permission model doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test docker container config setup.
Tom Clegg [Mon, 13 Dec 2021 19:23:53 +0000 (14:23 -0500)]
12630: Test docker container config setup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test containerSpec setup and singularity exec args.
Tom Clegg [Mon, 13 Dec 2021 18:58:13 +0000 (13:58 -0500)]
12630: Test containerSpec setup and singularity exec args.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Update to docker client 19.03.15 for GPU support
Peter Amstutz [Fri, 10 Dec 2021 16:34:44 +0000 (11:34 -0500)]
12630: Update to docker client 19.03.15 for GPU support

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Remove distinction between cubin/PTX hardware capabilities
Peter Amstutz [Wed, 24 Nov 2021 20:26:46 +0000 (15:26 -0500)]
12630: Remove distinction between cubin/PTX hardware capabilities

Fix misspelled "HardwareCapability"

Add cuda_ fields runtime_constraints documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Use CUDADeviceCount instead of EnableCUDA
Peter Amstutz [Mon, 22 Nov 2021 18:35:20 +0000 (13:35 -0500)]
12630: Use CUDADeviceCount instead of EnableCUDA

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: first try at adding GPU support, no tests yet
Peter Amstutz [Mon, 22 Nov 2021 17:47:03 +0000 (12:47 -0500)]
12630: first try at adding GPU support, no tests yet

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18348-log-cluster-id'
Tom Clegg [Fri, 10 Dec 2021 16:19:52 +0000 (11:19 -0500)]
Merge branch '18348-log-cluster-id'

closes #18348

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Fri, 10 Dec 2021 16:19:29 +0000 (11:19 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18277: Add RoleGroupsVisibleToAll config, default true.
Tom Clegg [Fri, 10 Dec 2021 14:46:00 +0000 (09:46 -0500)]
18277: Add RoleGroupsVisibleToAll config, default true.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18566: Add note to install doc re singularity loop device bug/fix.
Tom Clegg [Fri, 10 Dec 2021 14:33:24 +0000 (09:33 -0500)]
18566: Add note to install doc re singularity loop device bug/fix.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18349: Confirm X-Request-Id propagates through federation.
Tom Clegg [Thu, 9 Dec 2021 19:00:48 +0000 (14:00 -0500)]
18349: Confirm X-Request-Id propagates through federation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18348: Add ClusterID field to service log entries.
Tom Clegg [Wed, 8 Dec 2021 19:51:13 +0000 (14:51 -0500)]
18348: Add ClusterID field to service log entries.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17339-s3aws-driver-memory-footprint'
Ward Vandewege [Tue, 7 Dec 2021 22:20:56 +0000 (17:20 -0500)]
Merge branch '17339-s3aws-driver-memory-footprint'

closes #17339

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: Merge branch 'main' into 17339-s3aws-driver-memory-footprint
Ward Vandewege [Tue, 7 Dec 2021 22:20:27 +0000 (17:20 -0500)]
17339: Merge branch 'main' into 17339-s3aws-driver-memory-footprint

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17667-doc-improvements'
Ward Vandewege [Tue, 7 Dec 2021 21:46:05 +0000 (16:46 -0500)]
Merge branch '17667-doc-improvements'

closes #17667

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17667: Merge branch 'main' into 17667-doc-improvements
Ward Vandewege [Tue, 7 Dec 2021 21:45:25 +0000 (16:45 -0500)]
17667: Merge branch 'main' into 17667-doc-improvements

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18547-use-volume-uuid-not-device-id'
Tom Clegg [Tue, 7 Dec 2021 20:07:30 +0000 (15:07 -0500)]
Merge branch '18547-use-volume-uuid-not-device-id'

fixes #18547

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17339: implement review feedback.
Ward Vandewege [Mon, 6 Dec 2021 22:01:26 +0000 (17:01 -0500)]
17339: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18301: Fixes the issue by adding 0 as one of the default values to be ignored.
Lucas Di Pentima [Tue, 7 Dec 2021 18:12:54 +0000 (15:12 -0300)]
18301: Fixes the issue by adding 0 as one of the default values to be ignored.

Also, pre-sort mounts' key lists before comparison, just in case.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18301: Adds test exposing the bug.
Lucas Di Pentima [Tue, 7 Dec 2021 18:11:29 +0000 (15:11 -0300)]
18301: Adds test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17667: a round of edits after review feedback.
Ward Vandewege [Tue, 7 Dec 2021 00:46:01 +0000 (19:46 -0500)]
17667: a round of edits after review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18547: Error out if two volumes return the same non-empty DeviceID.
Tom Clegg [Tue, 7 Dec 2021 00:32:34 +0000 (19:32 -0500)]
18547: Error out if two volumes return the same non-empty DeviceID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17667: add some missing information on the "Install prerequisites" page.
Ward Vandewege [Mon, 6 Dec 2021 17:48:57 +0000 (12:48 -0500)]
17667: add some missing information on the "Install prerequisites" page.
       Update the "Install PostgreSQL" page with a reference to Aurora
       RDS. Add a new page that details InternalURLs/ExternalURL.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: fix tests.
Ward Vandewege [Fri, 3 Dec 2021 21:29:49 +0000 (16:29 -0500)]
17339: fix tests.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17339: remove unnecessary memory allocation when reading from Keep with
Ward Vandewege [Fri, 3 Dec 2021 20:12:38 +0000 (15:12 -0500)]
17339: remove unnecessary memory allocation when reading from Keep with
       the S3 v2 driver. Also fix a few incorrect calls to log.Error()

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18376-nfs-readdirent'
Tom Clegg [Fri, 3 Dec 2021 16:30:23 +0000 (11:30 -0500)]
Merge branch '18376-nfs-readdirent'

refs #18376

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18547: Use volume UUID instead of DeviceID to deduplicate mounts.
Tom Clegg [Fri, 3 Dec 2021 15:54:59 +0000 (10:54 -0500)]
18547: Use volume UUID instead of DeviceID to deduplicate mounts.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18488-nonblocking-advisory-lock'
Tom Clegg [Fri, 3 Dec 2021 15:44:04 +0000 (10:44 -0500)]
Merge branch '18488-nonblocking-advisory-lock'

fixes #18488

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17339: remove unnecessary memory allocation when writing to Keep with
Ward Vandewege [Thu, 2 Dec 2021 20:59:18 +0000 (15:59 -0500)]
17339: remove unnecessary memory allocation when writing to Keep with
       the S3 v2 driver.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18511-java-sdk-groups-links' refs #18511
Peter Amstutz [Thu, 2 Dec 2021 15:31:27 +0000 (10:31 -0500)]
Merge branch '18511-java-sdk-groups-links' refs #18511

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18490-redundant-updates' refs #18490
Peter Amstutz [Wed, 1 Dec 2021 22:27:15 +0000 (17:27 -0500)]
Merge branch '18490-redundant-updates' refs #18490

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18490: Add comments.
Peter Amstutz [Wed, 1 Dec 2021 22:26:47 +0000 (17:26 -0500)]
18490: Add comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoDocumentation: fix AWS region name in the configuration reference and on
Ward Vandewege [Wed, 1 Dec 2021 20:48:54 +0000 (15:48 -0500)]
Documentation: fix AWS region name in the configuration reference and on
the S3 configuration page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge remote-tracking branch 'MajewskiKrzysztof/links-tail-uuid' into 18511-java...
Peter Amstutz [Wed, 1 Dec 2021 19:34:11 +0000 (14:34 -0500)]
Merge remote-tracking branch 'MajewskiKrzysztof/links-tail-uuid' into 18511-java-sdk-groups-links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge remote-tracking branch 'MajewskiKrzysztof/group-update' into 18511-java-sdk...
Peter Amstutz [Wed, 1 Dec 2021 19:33:30 +0000 (14:33 -0500)]
Merge remote-tracking branch 'MajewskiKrzysztof/group-update' into 18511-java-sdk-groups-links

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd tails_uuid and tail_kind to Links API
majewsk6 [Wed, 1 Dec 2021 07:19:12 +0000 (08:19 +0100)]
Add tails_uuid and tail_kind to Links API

Arvados-DCO-1.1-Signed-off-by: Krzysztof Majewski majewski.kielce@gmail.com

3 years agoGroup update caused exception
majewsk6 [Wed, 1 Dec 2021 07:11:46 +0000 (08:11 +0100)]
Group update caused exception

Arvados-DCO-1.1-Signed-off-by: Krzysztof Majewski majewski.kielce@gmail.com

3 years agoMerge branch '18491-jwt-go-upgrade' into main. Closes #18491.
Lucas Di Pentima [Tue, 30 Nov 2021 15:59:12 +0000 (12:59 -0300)]
Merge branch '18491-jwt-go-upgrade' into main. Closes #18491.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18488: Close pg connection when waiting for lock.
Tom Clegg [Tue, 30 Nov 2021 15:57:08 +0000 (10:57 -0500)]
18488: Close pg connection when waiting for lock.

In practice, pg_advisory_lock() calls pile up and consume database
connection slots, even when the corresponding processes have exited.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18491: Removes an unnecessary check that made the test fail on module upgrade.
Lucas Di Pentima [Tue, 30 Nov 2021 15:37:00 +0000 (12:37 -0300)]
18491: Removes an unnecessary check that made the test fail on module upgrade.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18490: Add a where clause to avoid updating identical rows.
Peter Amstutz [Mon, 29 Nov 2021 21:15:23 +0000 (16:15 -0500)]
18490: Add a where clause to avoid updating identical rows.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18491: Upgrades github.com/Azure/go-autorest/autorest/azure/auth and its deps.
Lucas Di Pentima [Mon, 29 Nov 2021 19:19:38 +0000 (16:19 -0300)]
18491: Upgrades github.com/Azure/go-autorest/autorest/azure/auth and its deps.

Also, ran "go mod tidy" to remove unused package references.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17785-federated-token-regression' into main. Closes #17785
Lucas Di Pentima [Mon, 29 Nov 2021 15:57:57 +0000 (12:57 -0300)]
Merge branch '17785-federated-token-regression' into main. Closes #17785

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>