arvados.git
2 years ago12630: Report errors from nvidia-modprobe & use Getenv
Peter Amstutz [Mon, 10 Jan 2022 19:11:37 +0000 (14:11 -0500)]
12630: Report errors from nvidia-modprobe & use Getenv

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Fix test
Peter Amstutz [Sat, 8 Jan 2022 19:02:33 +0000 (14:02 -0500)]
12630: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Fix usage of string.Split
Peter Amstutz [Fri, 7 Jan 2022 23:10:29 +0000 (18:10 -0500)]
12630: Fix usage of string.Split

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: request both 'compute' and 'utility' nvidia capabilities
Peter Amstutz [Fri, 7 Jan 2022 22:56:08 +0000 (17:56 -0500)]
12630: request both 'compute' and 'utility' nvidia capabilities

Add note about capabilities.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Call nvidia-modprobe, support CUDA_VISIBLE_DEVICES
Peter Amstutz [Fri, 7 Jan 2022 22:13:09 +0000 (17:13 -0500)]
12630: Call nvidia-modprobe, support CUDA_VISIBLE_DEVICES

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17417-add-arm64'
Ward Vandewege [Fri, 7 Jan 2022 19:53:04 +0000 (14:53 -0500)]
Merge branch '17417-add-arm64'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Fri, 7 Jan 2022 19:52:31 +0000 (14:52 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18577-fix-back_url'
Ward Vandewege [Fri, 7 Jan 2022 18:56:25 +0000 (13:56 -0500)]
Merge branch '18577-fix-back_url'

closes #18577

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18577: Merge branch 'main' into 18577-fix-back_url
Ward Vandewege [Fri, 7 Jan 2022 18:55:50 +0000 (13:55 -0500)]
18577: Merge branch 'main' into 18577-fix-back_url

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: address review comments.
Ward Vandewege [Fri, 7 Jan 2022 18:51:39 +0000 (13:51 -0500)]
17417: address review comments.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRelease branches were renamed.
Ward Vandewege [Thu, 6 Jan 2022 18:00:18 +0000 (13:00 -0500)]
Release branches were renamed.

closes #18249

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoBugfixes: doc: show the correct branch names for all the Salt install
Ward Vandewege [Thu, 6 Jan 2022 17:02:37 +0000 (12:02 -0500)]
Bugfixes: doc: show the correct branch names for all the Salt install
methods.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18323-cwl-gpu2' refs #18323
Peter Amstutz [Wed, 5 Jan 2022 21:50:23 +0000 (16:50 -0500)]
Merge branch '18323-cwl-gpu2' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 5 Jan 2022 21:38:52 +0000 (16:38 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 5 Jan 2022 21:35:02 +0000 (16:35 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRemove one more use of `go get` from run-library.sh.
Ward Vandewege [Wed, 5 Jan 2022 19:56:03 +0000 (14:56 -0500)]
Remove one more use of `go get` from run-library.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 21:06:28 +0000 (16:06 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 20:05:25 +0000 (15:05 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix to use cwltool:CUDARequirement
Peter Amstutz [Wed, 5 Jan 2022 20:03:30 +0000 (15:03 -0500)]
18323: Fix to use cwltool:CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18577: workbench1: remove references to `redirect_to :back`, this
Ward Vandewege [Wed, 5 Jan 2022 19:34:45 +0000 (14:34 -0500)]
18577: workbench1: remove references to `redirect_to :back`, this
       stopped working in Rails 5.1.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: add --only-build flag to the
Ward Vandewege [Wed, 5 Jan 2022 18:54:16 +0000 (13:54 -0500)]
17417: add --only-build flag to the
       run-build-test-packages-one-target.sh script.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Because of debian bug #983477, libfuse-dev for arm64 does not
Ward Vandewege [Wed, 5 Jan 2022 16:20:23 +0000 (11:20 -0500)]
17417: Because of debian bug #983477, libfuse-dev for arm64 does not
       install properly beside the amd64 version. This was fixed in
       debian11. Rather than making run-library.sh even more complicated
       by switching out libfuse-dev before every arm64 cross compiled
       build or by building a custom version of libfuse-dev, I disabled
       cross compilation the older Debian-based distributions we support.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: another optimization: instead of building the
Ward Vandewege [Wed, 5 Jan 2022 15:27:02 +0000 (10:27 -0500)]
17417: another optimization: instead of building the
       arvados-python-client sdist regardless of the value of
       $ONLY_BUILD, only do it when it is actually needed.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: fix bug to enable arm64 cross-compilation on amd64 for our go
Ward Vandewege [Wed, 5 Jan 2022 01:11:43 +0000 (20:11 -0500)]
17417: fix bug to enable arm64 cross-compilation on amd64 for our go
       packages. Also make sure package testing works (at least for
       amd64).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRemove deprecated use of `go get` from run-tests.sh. This should solve
Ward Vandewege [Wed, 5 Jan 2022 16:56:03 +0000 (11:56 -0500)]
Remove deprecated use of `go get` from run-tests.sh. This should solve
the occasional surprise modifications of go.mod/go.sum.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: apparently the arm64 repos have a different url on Ubuntu.
Ward Vandewege [Tue, 4 Jan 2022 21:38:50 +0000 (16:38 -0500)]
17417: apparently the arm64 repos have a different url on Ubuntu.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: apply some shellcheck suggestions to build/run-build-packages.sh
Ward Vandewege [Tue, 4 Jan 2022 20:30:46 +0000 (15:30 -0500)]
17417: apply some shellcheck suggestions to build/run-build-packages.sh

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: remove libcloud pin, it is no longer used.
Ward Vandewege [Tue, 4 Jan 2022 19:58:29 +0000 (14:58 -0500)]
17417: remove libcloud pin, it is no longer used.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: preseed the go module cache in our package build images. This
Ward Vandewege [Tue, 4 Jan 2022 19:50:27 +0000 (14:50 -0500)]
17417: preseed the go module cache in our package build images. This
       makes them (much) bigger, but it really speeds up a quick one-off
       package build.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: more refactoring: reduce use of global variables, move more code
Ward Vandewege [Tue, 4 Jan 2022 18:55:18 +0000 (13:55 -0500)]
17417: more refactoring: reduce use of global variables, move more code
       into wrapper functions in run_library.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: refactor, reduce use of global variables.
Ward Vandewege [Mon, 3 Jan 2022 21:55:42 +0000 (16:55 -0500)]
17417: refactor, reduce use of global variables.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18562-preemptible'
Tom Clegg [Thu, 30 Dec 2021 19:00:12 +0000 (14:00 -0500)]
Merge branch '18562-preemptible'

fixes #18562

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: no cross compilation support on CentOS.
Ward Vandewege [Thu, 30 Dec 2021 16:59:57 +0000 (11:59 -0500)]
17417: no cross compilation support on CentOS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18604-embed-config-default'
Tom Clegg [Thu, 30 Dec 2021 15:23:29 +0000 (10:23 -0500)]
Merge branch '18604-embed-config-default'

closes #18604

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: add native arm64 build support to our package build images. Add
Ward Vandewege [Thu, 30 Dec 2021 14:17:35 +0000 (09:17 -0500)]
17417: add native arm64 build support to our package build images. Add
       cross-compile arm64 build support to all our Debian and Ubuntu
       package build images (golang packages only).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 29 Dec 2021 23:01:22 +0000 (18:01 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17417-fix-wb1'
Ward Vandewege [Wed, 29 Dec 2021 22:53:52 +0000 (17:53 -0500)]
Merge branch '17417-fix-wb1'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18604: Require go1.17, use go:embed feature for config.default.yml.
Tom Clegg [Wed, 29 Dec 2021 16:14:14 +0000 (11:14 -0500)]
18604: Require go1.17, use go:embed feature for config.default.yml.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17417: remove the coffee-rails gem dependency in wb1, and convert the
Ward Vandewege [Wed, 29 Dec 2021 02:41:54 +0000 (21:41 -0500)]
17417: remove the coffee-rails gem dependency in wb1, and convert the
       two files where we used it to straight up javascript.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18562: Add upgrade note about new config key.
Tom Clegg [Mon, 27 Dec 2021 15:51:52 +0000 (10:51 -0500)]
18562: Add upgrade note about new config key.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17417: make sure that there is enough of a config.yml file for the wb1
Ward Vandewege [Fri, 24 Dec 2021 20:56:26 +0000 (15:56 -0500)]
17417: make sure that there is enough of a config.yml file for the wb1
       package install test to find all potential issues.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Thu, 23 Dec 2021 22:02:32 +0000 (17:02 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18562: Rename config to AlwaysUsePreemptibleInstances.
Tom Clegg [Thu, 23 Dec 2021 21:59:12 +0000 (16:59 -0500)]
18562: Rename config to AlwaysUsePreemptibleInstances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18562: Update doc and config comments for UsePreemptibleInstances.
Tom Clegg [Thu, 23 Dec 2021 21:51:06 +0000 (16:51 -0500)]
18562: Update doc and config comments for UsePreemptibleInstances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17417-goodbye-to-mini_racer'
Ward Vandewege [Thu, 23 Dec 2021 21:21:08 +0000 (16:21 -0500)]
Merge branch '17417-goodbye-to-mini_racer'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: remove more unused gems from the API server gemfile.
Ward Vandewege [Thu, 23 Dec 2021 20:36:00 +0000 (15:36 -0500)]
17417: remove more unused gems from the API server gemfile.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: add native build support for aarch64 (arm64) for our Debian 11
Ward Vandewege [Thu, 23 Dec 2021 18:00:06 +0000 (13:00 -0500)]
17417: add native build support for aarch64 (arm64) for our Debian 11
       target. Also speed up the docker image build by telling RVM and
       bundler to use all available cores.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18562: Auto-enable preemptible when changing state to Committed.
Tom Clegg [Thu, 23 Dec 2021 15:54:07 +0000 (10:54 -0500)]
18562: Auto-enable preemptible when changing state to Committed.

(As opposed to during any update.)

Avoids a situation where a client creates a child CR with
preemptible=false, preemptible is automatically changed to true, but
then preemptible instances are removed from config, and the client
cannot change the CR state to committed without also explicitly
clearing the preemptible flag, because the automatically-enabled
preemptible flag is not allowed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17417: Testing if execjs is really needed.
Lucas Di Pentima [Thu, 23 Dec 2021 14:29:30 +0000 (11:29 -0300)]
17417: Testing if execjs is really needed.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18562: Fix UsePreemptibleInstances behavior.
Tom Clegg [Wed, 22 Dec 2021 22:49:23 +0000 (17:49 -0500)]
18562: Fix UsePreemptibleInstances behavior.

* Do not automatically set preemptible=true if there are no
  preemptible instance types available.

* Do not automatically set preemptible=true on a container request
  that has already been committed with preemptible=false.

* Do not reject updates to existing container requests with
  preemptible=true just because config has since changed and no longer
  enables it automatically.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18323: Revise in line with the cwltool version of this feature
Peter Amstutz [Wed, 22 Dec 2021 21:27:06 +0000 (16:27 -0500)]
18323: Revise in line with the cwltool version of this feature

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17417: switch our Rails codebases from therubyracer to mini_racer.
Ward Vandewege [Wed, 22 Dec 2021 20:47:08 +0000 (15:47 -0500)]
17417: switch our Rails codebases from therubyracer to mini_racer.
       Remove old dependency on `less` and `less-rails` from WB1.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: add the scaffolding for multi-arch support for our Python
Ward Vandewege [Wed, 22 Dec 2021 19:53:20 +0000 (14:53 -0500)]
17417: add the scaffolding for multi-arch support for our Python
       packages. Cross-compilation for Python does not work yet, though
       native compilation on an arm64 system works now.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: when building the arvados-src, cwltest and libpam-arvados-go
Ward Vandewege [Wed, 22 Dec 2021 18:34:50 +0000 (13:34 -0500)]
17417: when building the arvados-src, cwltest and libpam-arvados-go
       packages, respect the --only-build flag.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: add arm64 support for our golang packages, functional for Debian
Ward Vandewege [Wed, 22 Dec 2021 18:34:07 +0000 (13:34 -0500)]
17417: add arm64 support for our golang packages, functional for Debian
       11 as of this commit.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 22 Dec 2021 14:27:30 +0000 (09:27 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18323-cwl-gpu' refs #18323
Peter Amstutz [Tue, 21 Dec 2021 19:26:12 +0000 (14:26 -0500)]
Merge branch '18323-cwl-gpu' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Document CUDARequirement
Peter Amstutz [Tue, 21 Dec 2021 19:24:55 +0000 (14:24 -0500)]
18323: Document CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18321-gpu-instancetype' refs #18321
Peter Amstutz [Tue, 21 Dec 2021 18:03:49 +0000 (13:03 -0500)]
Merge branch '18321-gpu-instancetype' refs #18321

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Fix tests
Peter Amstutz [Tue, 21 Dec 2021 17:00:28 +0000 (12:00 -0500)]
18321: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-cuda-azure-image'
Ward Vandewege [Tue, 21 Dec 2021 15:59:36 +0000 (10:59 -0500)]
Merge branch '18325-cuda-azure-image'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Check runtime constraints md5 with/without empty cuda section
Peter Amstutz [Mon, 20 Dec 2021 22:15:46 +0000 (17:15 -0500)]
18321: Check runtime constraints md5 with/without empty cuda section

* config & API check format of DriverVersion and HardwareCapability
* crunch-run only pays attention to CUDA.DeviceCount
* update docs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: adapt for naming differences in kernel/headers packages between
Ward Vandewege [Mon, 20 Dec 2021 15:32:20 +0000 (10:32 -0500)]
18325: adapt for naming differences in kernel/headers packages between
       Debian and Ubuntu.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18323: Fix the rest of the tests
Peter Amstutz [Mon, 20 Dec 2021 17:20:26 +0000 (12:20 -0500)]
18323: Fix the rest of the tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Test refactor WIP
Peter Amstutz [Mon, 20 Dec 2021 16:42:27 +0000 (11:42 -0500)]
18323: Test refactor WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: The nvidia apt repo link generation information at
Ward Vandewege [Mon, 20 Dec 2021 15:22:54 +0000 (10:22 -0500)]
18325: The nvidia apt repo link generation information at
       https://nvidia.github.io/libnvidia-container/ is wrong for
       Ubuntu, the dot needs to be stripped from the version number.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: also support building images with CUDA for Azure.
Ward Vandewege [Mon, 20 Dec 2021 15:01:23 +0000 (10:01 -0500)]
18325: also support building images with CUDA for Azure.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Account for CUDA in container reuse
Peter Amstutz [Fri, 17 Dec 2021 20:57:16 +0000 (15:57 -0500)]
18321: Account for CUDA in container reuse

Also ensure reuse across versions when CUDA isn't being used.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18323: Add CUDARequirement to extension schema
Peter Amstutz [Fri, 17 Dec 2021 18:18:03 +0000 (13:18 -0500)]
18323: Add CUDARequirement to extension schema

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18325-compute-image-cuda'
Ward Vandewege [Fri, 17 Dec 2021 16:27:13 +0000 (11:27 -0500)]
Merge branch '18325-compute-image-cuda'

closes #18325

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18321: Add CUDARuntimeConstraints matching CUDAFeatures
Peter Amstutz [Fri, 17 Dec 2021 15:55:00 +0000 (10:55 -0500)]
18321: Add CUDARuntimeConstraints matching CUDAFeatures

Adjust CUDA version comparison.  Refactor tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
Ward Vandewege [Fri, 17 Dec 2021 14:49:33 +0000 (09:49 -0500)]
18325: when using Nvidia GPUs on Debian 10, we need a newer Docker. Also
       fix use of the $SUDO variable in the base packer script and add a
       final call to `apt-get clean` to it.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: no debian11 packages yet for libnvidia-container-tools, use
Ward Vandewege [Thu, 16 Dec 2021 21:05:45 +0000 (16:05 -0500)]
18325: no debian11 packages yet for libnvidia-container-tools, use
       workaround to install experimental rc1 with debian11 support,
       when on debian11.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: use a bigger root volume while building the AMI on AWS.
Ward Vandewege [Thu, 16 Dec 2021 19:46:36 +0000 (14:46 -0500)]
18325: use a bigger root volume while building the AMI on AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18325: add support for Nvidia GPUs as an optional feature to our compute
Ward Vandewege [Thu, 16 Dec 2021 19:12:49 +0000 (14:12 -0500)]
18325: add support for Nvidia GPUs as an optional feature to our compute
       node image builder script, for AWS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18590-fix-discovery-doc'
Ward Vandewege [Thu, 16 Dec 2021 15:39:11 +0000 (10:39 -0500)]
Merge branch '18590-fix-discovery-doc'

closes #18590

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: Merge branch 'main' into 18590-fix-discovery-doc
Ward Vandewege [Thu, 16 Dec 2021 15:38:14 +0000 (10:38 -0500)]
18590: Merge branch 'main' into 18590-fix-discovery-doc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18590: the default value for uuid in the groups/contents api endpoint
Ward Vandewege [Thu, 16 Dec 2021 00:22:49 +0000 (19:22 -0500)]
18590: the default value for uuid in the groups/contents api endpoint
       should not be expressed as `null` in the discovery doc, but as a
       string.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18301-wb1-cancel-button-fix'. Closes #18301.
Lucas Di Pentima [Wed, 15 Dec 2021 21:24:52 +0000 (18:24 -0300)]
Merge branch '18301-wb1-cancel-button-fix'. Closes #18301.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '18585-remove-java-sdk-v1'
Ward Vandewege [Wed, 15 Dec 2021 19:28:08 +0000 (14:28 -0500)]
Merge branch '18585-remove-java-sdk-v1'

closes #18585

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: Merge branch 'main' into 18585-remove-java-sdk-v1
Ward Vandewege [Wed, 15 Dec 2021 19:27:44 +0000 (14:27 -0500)]
18585: Merge branch 'main' into 18585-remove-java-sdk-v1

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18585: actually remove the old Java SDK.
Ward Vandewege [Wed, 15 Dec 2021 19:26:40 +0000 (14:26 -0500)]
18585: actually remove the old Java SDK.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18349-fed-request-id'
Tom Clegg [Wed, 15 Dec 2021 14:26:45 +0000 (09:26 -0500)]
Merge branch '18349-fed-request-id'

closes #18349

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Tue, 14 Dec 2021 18:13:36 +0000 (13:13 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18566: Update note with fixed singularity version.
Tom Clegg [Tue, 14 Dec 2021 18:12:47 +0000 (13:12 -0500)]
18566: Update note with fixed singularity version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18585: remove the old Java SDK, it has been deprecated for years and the
Ward Vandewege [Tue, 14 Dec 2021 16:51:27 +0000 (11:51 -0500)]
18585: remove the old Java SDK, it has been deprecated for years and the
       Java SDK v2 works much better.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18277-groups-visible-to-all'
Tom Clegg [Tue, 14 Dec 2021 16:26:14 +0000 (11:26 -0500)]
Merge branch '18277-groups-visible-to-all'

closes #18277

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18321: Fix config export
Peter Amstutz [Mon, 13 Dec 2021 18:47:28 +0000 (13:47 -0500)]
18321: Fix config export

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18321: Incorporate CUDA request into picking a node type
Peter Amstutz [Mon, 13 Dec 2021 16:55:45 +0000 (11:55 -0500)]
18321: Incorporate CUDA request into picking a node type

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '12630-crunch-gpu' refs #12630
Peter Amstutz [Mon, 13 Dec 2021 20:17:41 +0000 (15:17 -0500)]
Merge branch '12630-crunch-gpu' refs #12630

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18277: Note configurable role visibility in permission model doc.
Tom Clegg [Mon, 13 Dec 2021 20:01:18 +0000 (15:01 -0500)]
18277: Note configurable role visibility in permission model doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test docker container config setup.
Tom Clegg [Mon, 13 Dec 2021 19:23:53 +0000 (14:23 -0500)]
12630: Test docker container config setup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Test containerSpec setup and singularity exec args.
Tom Clegg [Mon, 13 Dec 2021 18:58:13 +0000 (13:58 -0500)]
12630: Test containerSpec setup and singularity exec args.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago12630: Update to docker client 19.03.15 for GPU support
Peter Amstutz [Fri, 10 Dec 2021 16:34:44 +0000 (11:34 -0500)]
12630: Update to docker client 19.03.15 for GPU support

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Remove distinction between cubin/PTX hardware capabilities
Peter Amstutz [Wed, 24 Nov 2021 20:26:46 +0000 (15:26 -0500)]
12630: Remove distinction between cubin/PTX hardware capabilities

Fix misspelled "HardwareCapability"

Add cuda_ fields runtime_constraints documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: Use CUDADeviceCount instead of EnableCUDA
Peter Amstutz [Mon, 22 Nov 2021 18:35:20 +0000 (13:35 -0500)]
12630: Use CUDADeviceCount instead of EnableCUDA

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago12630: first try at adding GPU support, no tests yet
Peter Amstutz [Mon, 22 Nov 2021 17:47:03 +0000 (12:47 -0500)]
12630: first try at adding GPU support, no tests yet

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18348-log-cluster-id'
Tom Clegg [Fri, 10 Dec 2021 16:19:52 +0000 (11:19 -0500)]
Merge branch '18348-log-cluster-id'

closes #18348

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18566-singularity-loop-doc'
Tom Clegg [Fri, 10 Dec 2021 16:19:29 +0000 (11:19 -0500)]
Merge branch '18566-singularity-loop-doc'

closes #18566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18277: Add RoleGroupsVisibleToAll config, default true.
Tom Clegg [Fri, 10 Dec 2021 14:46:00 +0000 (09:46 -0500)]
18277: Add RoleGroupsVisibleToAll config, default true.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>