arvados.git
2 years ago18719: Fix TmpCollectionDirectory
Peter Amstutz [Sun, 6 Feb 2022 21:12:45 +0000 (16:12 -0500)]
18719: Fix TmpCollectionDirectory

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18719: Put back some logic to invalidate collection_record_file
Peter Amstutz [Sun, 6 Feb 2022 21:11:02 +0000 (16:11 -0500)]
18719: Put back some logic to invalidate collection_record_file

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18719: handle updates on flush()
Peter Amstutz [Sun, 6 Feb 2022 21:00:33 +0000 (16:00 -0500)]
18719: handle updates on flush()

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18719: Don't try to apply incremental updates to parent inode
Peter Amstutz [Sun, 6 Feb 2022 19:58:13 +0000 (14:58 -0500)]
18719: Don't try to apply incremental updates to parent inode

Just invalidate it and update it.

Don't crash commit_all race.

Running "aws s3 sync" seems to be working now.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18719: Handle collection_record differently
Peter Amstutz [Sun, 6 Feb 2022 03:05:19 +0000 (22:05 -0500)]
18719: Handle collection_record differently

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18719: Make sure to ignore old versions of collections
Peter Amstutz [Fri, 4 Feb 2022 22:18:58 +0000 (17:18 -0500)]
18719: Make sure to ignore old versions of collections

Simplify websocket events to just call invalidate, don't try to do anything clever.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFix typo in documentation.
Ward Vandewege [Thu, 3 Feb 2022 22:05:42 +0000 (17:05 -0500)]
Fix typo in documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18703-installer-upgrade-postgres-formula'
Javier Bértoli [Thu, 3 Feb 2022 17:16:25 +0000 (14:16 -0300)]
Merge branch '18703-installer-upgrade-postgres-formula'

closes #18703
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '18679-handle-keys-with-null-values-in-config'
Ward Vandewege [Thu, 3 Feb 2022 16:07:14 +0000 (11:07 -0500)]
Merge branch '18679-handle-keys-with-null-values-in-config'

closes #18679

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18679: if a config file contains a key with a null value, do not
Ward Vandewege [Wed, 2 Feb 2022 22:02:43 +0000 (17:02 -0500)]
18679: if a config file contains a key with a null value, do not
       override the default value for that key.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18703: upgrade Vagrantfile to debian-11
Javier Bértoli [Tue, 1 Feb 2022 23:15:40 +0000 (20:15 -0300)]
18703: upgrade Vagrantfile to debian-11

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18703: upgrade postgres-formula to 0.43.0 for debian-11
Javier Bértoli [Tue, 1 Feb 2022 23:14:59 +0000 (20:14 -0300)]
18703: upgrade postgres-formula to 0.43.0 for debian-11

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '17583-federated-token-reqs'. Closes #17583
Lucas Di Pentima [Tue, 1 Feb 2022 21:51:52 +0000 (18:51 -0300)]
Merge branch '17583-federated-token-reqs'. Closes #17583

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18632-docker-installation-from-packages'
Javier Bértoli [Fri, 28 Jan 2022 20:13:07 +0000 (17:13 -0300)]
Merge branch '18632-docker-installation-from-packages'

closes #18632
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18632: address review comments.
Javier Bértoli [Fri, 28 Jan 2022 18:36:03 +0000 (15:36 -0300)]
18632: address review comments.

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '16727-refresh-signatures'
Tom Clegg [Fri, 28 Jan 2022 16:07:08 +0000 (11:07 -0500)]
Merge branch '16727-refresh-signatures'

closes #16727

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago16727: Explain guessSignatureTTL field.
Tom Clegg [Fri, 28 Jan 2022 14:59:06 +0000 (09:59 -0500)]
16727: Explain guessSignatureTTL field.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18632: fix missing docker's pillar in shell node
Javier Bértoli [Thu, 27 Jan 2022 22:06:33 +0000 (19:06 -0300)]
18632: fix missing docker's pillar in shell node

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18632: fix docker pillar in provision script
Javier Bértoli [Thu, 27 Jan 2022 18:31:04 +0000 (15:31 -0300)]
18632: fix docker pillar in provision script

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago16727: Refresh block permission signatures on Sync and Read.
Tom Clegg [Tue, 25 Jan 2022 16:15:51 +0000 (11:15 -0500)]
16727: Refresh block permission signatures on Sync and Read.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18670-flaky-lsf-test'
Tom Clegg [Thu, 27 Jan 2022 14:23:02 +0000 (09:23 -0500)]
Merge branch '18670-flaky-lsf-test'

fixes #18670

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18670: Fix abandoned job tracker during race.
Tom Clegg [Wed, 26 Jan 2022 18:32:13 +0000 (13:32 -0500)]
18670: Fix abandoned job tracker during race.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18678-custom-certs-keepstore-fix'
Ward Vandewege [Tue, 25 Jan 2022 20:11:53 +0000 (15:11 -0500)]
Merge branch '18678-custom-certs-keepstore-fix'

closes #18678

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18678: installer: when custom certs are used, do not abort when a role
Ward Vandewege [Tue, 25 Jan 2022 19:55:38 +0000 (14:55 -0500)]
18678: installer: when custom certs are used, do not abort when a role
       is used that does not require any certificates.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18326-cuda-docs' refs #18326
Peter Amstutz [Tue, 25 Jan 2022 16:26:33 +0000 (11:26 -0500)]
Merge branch '18326-cuda-docs' refs #18326

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18326: Whitespace fixes
Peter Amstutz [Tue, 25 Jan 2022 16:22:44 +0000 (11:22 -0500)]
18326: Whitespace fixes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18487-vocab-config-check'. Closes #18487
Lucas Di Pentima [Tue, 25 Jan 2022 11:37:19 +0000 (08:37 -0300)]
Merge branch '18487-vocab-config-check'. Closes #18487

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18670: Fix unreliable test.
Tom Clegg [Tue, 25 Jan 2022 04:48:17 +0000 (23:48 -0500)]
18670: Fix unreliable test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18487: Logs vocabulary validating error messages along with other checks.
Lucas Di Pentima [Mon, 24 Jan 2022 20:18:07 +0000 (17:18 -0300)]
18487: Logs vocabulary validating error messages along with other checks.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18665-vscode-setup' refs #18665
Peter Amstutz [Mon, 24 Jan 2022 19:00:27 +0000 (14:00 -0500)]
Merge branch '18665-vscode-setup' refs #18665

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18665: Separate Set Host and Set Token steps
Peter Amstutz [Mon, 24 Jan 2022 18:43:30 +0000 (13:43 -0500)]
18665: Separate Set Host and Set Token steps

Being separated because the previous "paste the whole
contents of the dialog box" was actually more confusing than helpful.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17583: Forwards ApiClientAuthorization list requests to LoginCluster.
Lucas Di Pentima [Fri, 21 Jan 2022 20:27:36 +0000 (17:27 -0300)]
17583: Forwards ApiClientAuthorization list requests to LoginCluster.

Also, honor the bypass_federation parameter.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoDocumentation tweaks for the Salt installer.
Ward Vandewege [Fri, 21 Jan 2022 20:16:32 +0000 (15:16 -0500)]
Documentation tweaks for the Salt installer.

refs #18658

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17583: Expands test to expose one pending bug.
Lucas Di Pentima [Fri, 21 Jan 2022 20:15:21 +0000 (17:15 -0300)]
17583: Expands test to expose one pending bug.

ApiClientAuthorization LIST requests are not being forwarded on LoginCluster
federations.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18658-fix-custom-certs-deployment-on-multi-host'
Javier Bértoli [Fri, 21 Jan 2022 19:56:04 +0000 (16:56 -0300)]
Merge branch '18658-fix-custom-certs-deployment-on-multi-host'

closes #18658
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18658: address review comments.
Javier Bértoli [Fri, 21 Jan 2022 19:07:00 +0000 (16:07 -0300)]
18658: address review comments.

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18658: ensure custom SSL certs are copied to the correct host dir
Javier Bértoli [Thu, 20 Jan 2022 23:17:26 +0000 (20:17 -0300)]
18658: ensure custom SSL certs are copied to the correct host dir

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18658: update multi-host's nginx pillar to use rvm ruby on Ubuntu-18.04
Javier Bértoli [Thu, 20 Jan 2022 23:16:12 +0000 (20:16 -0300)]
18658: update multi-host's nginx pillar to use rvm ruby on Ubuntu-18.04

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18658: add missing state to deploy custom certs in multi-host env
Javier Bértoli [Thu, 20 Jan 2022 15:22:16 +0000 (12:22 -0300)]
18658: add missing state to deploy custom certs in multi-host env

Also, set the full path to the certs dir if not configured

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoSalt installer fixes:
Ward Vandewege [Thu, 20 Jan 2022 21:34:08 +0000 (16:34 -0500)]
Salt installer fixes:
* documentation: fix formatting
* multi-host aws example: remove AccessViaHosts from the config file, it
  is not necessary. Fix volume identifiers. Fix link to documentation by
  dispatcher key.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18487: Improves vocabulary documentation.
Lucas Di Pentima [Thu, 20 Jan 2022 21:31:51 +0000 (18:31 -0300)]
18487: Improves vocabulary documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18487: Fixes error reporting to include both JSON & vocabulary errors at once.
Lucas Di Pentima [Thu, 20 Jan 2022 20:00:17 +0000 (17:00 -0300)]
18487: Fixes error reporting to include both JSON & vocabulary errors at once.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18659-cast-devnum'
Tom Clegg [Thu, 20 Jan 2022 18:49:52 +0000 (13:49 -0500)]
Merge branch '18659-cast-devnum'

fixes #18659

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18659: Cast OS-specific device number type to uint64.
Tom Clegg [Thu, 20 Jan 2022 17:04:51 +0000 (12:04 -0500)]
18659: Cast OS-specific device number type to uint64.

Fixes compile error on GOOS=darwin.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18487: Fixes error checking on nonexistant vocabulary file. Adds tests.
Lucas Di Pentima [Tue, 18 Jan 2022 15:54:59 +0000 (12:54 -0300)]
18487: Fixes error checking on nonexistant vocabulary file. Adds tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18487: Improves vocabulary error checking, with tests.
Lucas Di Pentima [Tue, 18 Jan 2022 14:29:49 +0000 (11:29 -0300)]
18487: Improves vocabulary error checking, with tests.

* Adds JSON duplicated keys checking (json.Unmarshall silently accepts them).
* Adds line & column numbers to JSON syntax errors.
* Reports all detected non-syntax errors at once instead of returning on the
first one. (so that the site admin doesn't have to do multiple edit+check
cycles)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago18487: Adds vocabulary check to 'arvados-server config-check'
Lucas Di Pentima [Mon, 17 Jan 2022 19:20:01 +0000 (16:20 -0300)]
18487: Adds vocabulary check to 'arvados-server config-check'

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18513-log-signal-exit'
Tom Clegg [Tue, 18 Jan 2022 18:51:16 +0000 (13:51 -0500)]
Merge branch '18513-log-signal-exit'

closes #18513

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18513: Decipher status code in crunch-run.txt log.
Tom Clegg [Tue, 18 Jan 2022 15:13:50 +0000 (10:13 -0500)]
18513: Decipher status code in crunch-run.txt log.

Container exited with status code 137 (signal 9, SIGKILL)

Previously, docker mode just said "exited with code: 137" and
singularity mode didn't mention it at all.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18624-document-compute-image-requirements'
Ward Vandewege [Fri, 14 Jan 2022 20:51:48 +0000 (15:51 -0500)]
Merge branch '18624-document-compute-image-requirements'

closes #18624

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18624: implement review feedback.
Ward Vandewege [Fri, 14 Jan 2022 18:51:44 +0000 (13:51 -0500)]
18624: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18326: Update documentation to cover configuring GPU support
Peter Amstutz [Thu, 13 Jan 2022 22:20:39 +0000 (17:20 -0500)]
18326: Update documentation to cover configuring GPU support

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18624: crunch-run no longer needs to be preinstalled on our cloud
Ward Vandewege [Thu, 13 Jan 2022 13:39:10 +0000 (08:39 -0500)]
18624: crunch-run no longer needs to be preinstalled on our cloud
       compute images, arvados-dispatch-cloud copies it over.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18624: documentation: describe the arvados-dispatch-cloud requirements
Ward Vandewege [Wed, 12 Jan 2022 21:33:47 +0000 (16:33 -0500)]
18624: documentation: describe the arvados-dispatch-cloud requirements
       for compute nodes.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '12630-nvidia-devices' refs #12630
Peter Amstutz [Mon, 10 Jan 2022 22:01:25 +0000 (17:01 -0500)]
Merge branch '12630-nvidia-devices' refs #12630

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Log command line
Peter Amstutz [Mon, 10 Jan 2022 21:59:51 +0000 (16:59 -0500)]
12630: Log command line

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Tweak nvidia warnings
Peter Amstutz [Mon, 10 Jan 2022 20:39:26 +0000 (15:39 -0500)]
12630: Tweak nvidia warnings

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoWhen installing the bundler gem in run-tests.sh, always use
Ward Vandewege [Mon, 10 Jan 2022 20:26:29 +0000 (15:26 -0500)]
When installing the bundler gem in run-tests.sh, always use
--no-document

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago12630: Report errors from nvidia-modprobe & use Getenv
Peter Amstutz [Mon, 10 Jan 2022 19:11:37 +0000 (14:11 -0500)]
12630: Report errors from nvidia-modprobe & use Getenv

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18324-lsf-gpu' refs #18324
Peter Amstutz [Mon, 10 Jan 2022 16:26:16 +0000 (11:26 -0500)]
Merge branch '18324-lsf-gpu' refs #18324

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Fix test
Peter Amstutz [Sat, 8 Jan 2022 19:02:33 +0000 (14:02 -0500)]
12630: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch 'main' into 18324-lsf-gpu
Peter Amstutz [Fri, 7 Jan 2022 22:19:54 +0000 (17:19 -0500)]
Merge branch 'main' into 18324-lsf-gpu

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Fix usage of string.Split
Peter Amstutz [Fri, 7 Jan 2022 23:10:29 +0000 (18:10 -0500)]
12630: Fix usage of string.Split

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: request both 'compute' and 'utility' nvidia capabilities
Peter Amstutz [Fri, 7 Jan 2022 22:56:08 +0000 (17:56 -0500)]
12630: request both 'compute' and 'utility' nvidia capabilities

Add note about capabilities.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago12630: Call nvidia-modprobe, support CUDA_VISIBLE_DEVICES
Peter Amstutz [Fri, 7 Jan 2022 22:13:09 +0000 (17:13 -0500)]
12630: Call nvidia-modprobe, support CUDA_VISIBLE_DEVICES

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17417-add-arm64'
Ward Vandewege [Fri, 7 Jan 2022 19:53:04 +0000 (14:53 -0500)]
Merge branch '17417-add-arm64'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Fri, 7 Jan 2022 19:52:31 +0000 (14:52 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18577-fix-back_url'
Ward Vandewege [Fri, 7 Jan 2022 18:56:25 +0000 (13:56 -0500)]
Merge branch '18577-fix-back_url'

closes #18577

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18577: Merge branch 'main' into 18577-fix-back_url
Ward Vandewege [Fri, 7 Jan 2022 18:55:50 +0000 (13:55 -0500)]
18577: Merge branch 'main' into 18577-fix-back_url

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: address review comments.
Ward Vandewege [Fri, 7 Jan 2022 18:51:39 +0000 (13:51 -0500)]
17417: address review comments.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRelease branches were renamed.
Ward Vandewege [Thu, 6 Jan 2022 18:00:18 +0000 (13:00 -0500)]
Release branches were renamed.

closes #18249

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoBugfixes: doc: show the correct branch names for all the Salt install
Ward Vandewege [Thu, 6 Jan 2022 17:02:37 +0000 (12:02 -0500)]
Bugfixes: doc: show the correct branch names for all the Salt install
methods.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18323-cwl-gpu2' refs #18323
Peter Amstutz [Wed, 5 Jan 2022 21:50:23 +0000 (16:50 -0500)]
Merge branch '18323-cwl-gpu2' refs #18323

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 5 Jan 2022 21:38:52 +0000 (16:38 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 5 Jan 2022 21:35:02 +0000 (16:35 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRemove one more use of `go get` from run-library.sh.
Ward Vandewege [Wed, 5 Jan 2022 19:56:03 +0000 (14:56 -0500)]
Remove one more use of `go get` from run-library.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 21:06:28 +0000 (16:06 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18324: Incorporate BsubCUDAArguments. update test
Peter Amstutz [Wed, 5 Jan 2022 20:53:17 +0000 (15:53 -0500)]
18324: Incorporate BsubCUDAArguments.  update test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix test
Peter Amstutz [Wed, 5 Jan 2022 20:05:25 +0000 (15:05 -0500)]
18323: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18323: Fix to use cwltool:CUDARequirement
Peter Amstutz [Wed, 5 Jan 2022 20:03:30 +0000 (15:03 -0500)]
18323: Fix to use cwltool:CUDARequirement

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18577: workbench1: remove references to `redirect_to :back`, this
Ward Vandewege [Wed, 5 Jan 2022 19:34:45 +0000 (14:34 -0500)]
18577: workbench1: remove references to `redirect_to :back`, this
       stopped working in Rails 5.1.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: add --only-build flag to the
Ward Vandewege [Wed, 5 Jan 2022 18:54:16 +0000 (13:54 -0500)]
17417: add --only-build flag to the
       run-build-test-packages-one-target.sh script.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Because of debian bug #983477, libfuse-dev for arm64 does not
Ward Vandewege [Wed, 5 Jan 2022 16:20:23 +0000 (11:20 -0500)]
17417: Because of debian bug #983477, libfuse-dev for arm64 does not
       install properly beside the amd64 version. This was fixed in
       debian11. Rather than making run-library.sh even more complicated
       by switching out libfuse-dev before every arm64 cross compiled
       build or by building a custom version of libfuse-dev, I disabled
       cross compilation the older Debian-based distributions we support.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: another optimization: instead of building the
Ward Vandewege [Wed, 5 Jan 2022 15:27:02 +0000 (10:27 -0500)]
17417: another optimization: instead of building the
       arvados-python-client sdist regardless of the value of
       $ONLY_BUILD, only do it when it is actually needed.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: fix bug to enable arm64 cross-compilation on amd64 for our go
Ward Vandewege [Wed, 5 Jan 2022 01:11:43 +0000 (20:11 -0500)]
17417: fix bug to enable arm64 cross-compilation on amd64 for our go
       packages. Also make sure package testing works (at least for
       amd64).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoRemove deprecated use of `go get` from run-tests.sh. This should solve
Ward Vandewege [Wed, 5 Jan 2022 16:56:03 +0000 (11:56 -0500)]
Remove deprecated use of `go get` from run-tests.sh. This should solve
the occasional surprise modifications of go.mod/go.sum.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: apparently the arm64 repos have a different url on Ubuntu.
Ward Vandewege [Tue, 4 Jan 2022 21:38:50 +0000 (16:38 -0500)]
17417: apparently the arm64 repos have a different url on Ubuntu.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: apply some shellcheck suggestions to build/run-build-packages.sh
Ward Vandewege [Tue, 4 Jan 2022 20:30:46 +0000 (15:30 -0500)]
17417: apply some shellcheck suggestions to build/run-build-packages.sh

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: remove libcloud pin, it is no longer used.
Ward Vandewege [Tue, 4 Jan 2022 19:58:29 +0000 (14:58 -0500)]
17417: remove libcloud pin, it is no longer used.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: preseed the go module cache in our package build images. This
Ward Vandewege [Tue, 4 Jan 2022 19:50:27 +0000 (14:50 -0500)]
17417: preseed the go module cache in our package build images. This
       makes them (much) bigger, but it really speeds up a quick one-off
       package build.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: more refactoring: reduce use of global variables, move more code
Ward Vandewege [Tue, 4 Jan 2022 18:55:18 +0000 (13:55 -0500)]
17417: more refactoring: reduce use of global variables, move more code
       into wrapper functions in run_library.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18324: Adding BsubCUDAArguments WIP
Peter Amstutz [Mon, 3 Jan 2022 22:05:10 +0000 (17:05 -0500)]
18324: Adding BsubCUDAArguments WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17417: refactor, reduce use of global variables.
Ward Vandewege [Mon, 3 Jan 2022 21:55:42 +0000 (16:55 -0500)]
17417: refactor, reduce use of global variables.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18562-preemptible'
Tom Clegg [Thu, 30 Dec 2021 19:00:12 +0000 (14:00 -0500)]
Merge branch '18562-preemptible'

fixes #18562

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: no cross compilation support on CentOS.
Ward Vandewege [Thu, 30 Dec 2021 16:59:57 +0000 (11:59 -0500)]
17417: no cross compilation support on CentOS.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18604-embed-config-default'
Tom Clegg [Thu, 30 Dec 2021 15:23:29 +0000 (10:23 -0500)]
Merge branch '18604-embed-config-default'

closes #18604

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17417: add native arm64 build support to our package build images. Add
Ward Vandewege [Thu, 30 Dec 2021 14:17:35 +0000 (09:17 -0500)]
17417: add native arm64 build support to our package build images. Add
       cross-compile arm64 build support to all our Debian and Ubuntu
       package build images (golang packages only).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17417: Merge branch 'main' into 17417-add-arm64
Ward Vandewege [Wed, 29 Dec 2021 23:01:22 +0000 (18:01 -0500)]
17417: Merge branch 'main' into 17417-add-arm64

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17417-fix-wb1'
Ward Vandewege [Wed, 29 Dec 2021 22:53:52 +0000 (17:53 -0500)]
Merge branch '17417-fix-wb1'

refs #17417

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18604: Require go1.17, use go:embed feature for config.default.yml.
Tom Clegg [Wed, 29 Dec 2021 16:14:14 +0000 (11:14 -0500)]
18604: Require go1.17, use go:embed feature for config.default.yml.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>