arvados.git
3 years agoRemove long dead `run` and `pipeline` subcommands from the `arv`
Ward Vandewege [Sun, 10 Oct 2021 00:29:24 +0000 (20:29 -0400)]
Remove long dead `run` and `pipeline` subcommands from the `arv`
executable.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18259-singularity-doc'
Ward Vandewege [Fri, 8 Oct 2021 21:36:09 +0000 (17:36 -0400)]
Merge branch '18259-singularity-doc'

refs #18259

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18259: mention `mksquashfs mem` config setting in singularity.conf
Ward Vandewege [Fri, 8 Oct 2021 16:49:20 +0000 (12:49 -0400)]
18259: mention `mksquashfs mem` config setting in singularity.conf

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoThe minimum supported version of Python is 3.6
Ward Vandewege [Fri, 8 Oct 2021 21:20:43 +0000 (17:20 -0400)]
The minimum supported version of Python is 3.6

refs #18261

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix CWL tests mixed_version_v10_wf and command_input_file_expression
Peter Amstutz [Fri, 8 Oct 2021 20:06:58 +0000 (16:06 -0400)]
Fix CWL tests mixed_version_v10_wf and command_input_file_expression

refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18258-set-singularity-cachedir'
Ward Vandewege [Fri, 8 Oct 2021 16:50:08 +0000 (12:50 -0400)]
Merge branch '18258-set-singularity-cachedir'

closes #18258

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18258: also handle SINGULARITY_TMPDIR
Ward Vandewege [Fri, 8 Oct 2021 14:43:52 +0000 (10:43 -0400)]
18258: also handle SINGULARITY_TMPDIR

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18258: add missing error check on os.Mkdir
Ward Vandewege [Fri, 8 Oct 2021 14:38:17 +0000 (10:38 -0400)]
18258: add missing error check on os.Mkdir

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18258: set the SINGULARITY_CACHEDIR to a path underneath our temporary
Ward Vandewege [Fri, 8 Oct 2021 14:24:28 +0000 (10:24 -0400)]
18258: set the SINGULARITY_CACHEDIR to a path underneath our temporary
       space, and clean up after we're done building.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoarvbox fix, keepstore is part of arvados-server now
Peter Amstutz [Thu, 7 Oct 2021 18:01:15 +0000 (14:01 -0400)]
arvbox fix, keepstore is part of arvados-server now

refs #18216

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18113-change-cloudops-defaults'
Ward Vandewege [Thu, 7 Oct 2021 17:26:29 +0000 (13:26 -0400)]
Merge branch '18113-change-cloudops-defaults'

closes #18113

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18113: Merge branch 'main' into 18113-change-cloudops-defaults
Ward Vandewege [Thu, 7 Oct 2021 17:26:12 +0000 (13:26 -0400)]
18113: Merge branch 'main' into 18113-change-cloudops-defaults

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18113: change MaxCloudOpsPerSecond and MaxConcurrentInstanceCreateOps
Ward Vandewege [Wed, 29 Sep 2021 16:08:17 +0000 (12:08 -0400)]
18113: change MaxCloudOpsPerSecond and MaxConcurrentInstanceCreateOps
       defaults.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoRemove the failing pre-build (--only-deps) step for crunch
Peter Amstutz [Thu, 7 Oct 2021 14:52:30 +0000 (10:52 -0400)]
Remove the failing pre-build (--only-deps) step for crunch

It will just do what it needs to when the container starts.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoRemove the 'experimental' label in our config reference from things that
Ward Vandewege [Thu, 7 Oct 2021 14:14:23 +0000 (10:14 -0400)]
Remove the 'experimental' label in our config reference from things that
are not (PAM login support, a-d-c, Singularity support). Update the docs
for our LSF and Singularity support along the same lines, and update
formatting a bit to make the current limitations of the implementation
clear.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe support Singularity 3.7.4.
Ward Vandewege [Thu, 7 Oct 2021 13:57:02 +0000 (09:57 -0400)]
We support Singularity 3.7.4.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18238-singularity-home' refs #18238
Peter Amstutz [Thu, 7 Oct 2021 14:08:43 +0000 (10:08 -0400)]
Merge branch '18238-singularity-home' refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Another try at Singularity $HOME fix
Peter Amstutz [Thu, 7 Oct 2021 14:04:48 +0000 (10:04 -0400)]
18238: Another try at Singularity $HOME fix

Use just the --home parameter, which appears to have option of
behaving as a special case of --bind, and apparently (according to
cwltool experience) works across more Singularity versions.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18216-arvados-server-keepstore'
Tom Clegg [Wed, 6 Oct 2021 20:41:17 +0000 (16:41 -0400)]
Merge branch '18216-arvados-server-keepstore'

closes #18216

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18248-fix-ubuntu-18.04-testing'
Javier Bértoli [Wed, 6 Oct 2021 20:08:01 +0000 (17:08 -0300)]
Merge branch '18248-fix-ubuntu-18.04-testing'

closes #18248
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago18248: update comment addressing review suggestions
Javier Bértoli [Wed, 6 Oct 2021 20:01:16 +0000 (17:01 -0300)]
18248: update comment addressing review suggestions

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '18238-containers-in-arvbox' refs #18238
Peter Amstutz [Wed, 6 Oct 2021 19:58:02 +0000 (15:58 -0400)]
Merge branch '18238-containers-in-arvbox' refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix setting HOME on singularity. Fix a-c-r secondaryFiles.
Peter Amstutz [Wed, 6 Oct 2021 19:48:19 +0000 (15:48 -0400)]
Fix setting HOME on singularity.  Fix a-c-r secondaryFiles.

To fix CWL conformance tests.

refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17779-request-id-in-exception'
Tom Clegg [Wed, 6 Oct 2021 18:51:41 +0000 (14:51 -0400)]
Merge branch '17779-request-id-in-exception'

closes #17779

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18247-faraday-version'
Ward Vandewege [Wed, 6 Oct 2021 13:49:50 +0000 (09:49 -0400)]
Merge branch '18247-faraday-version'

refs #18247

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18247: Merge branch 'main' into 18247-faraday-version
Ward Vandewege [Wed, 6 Oct 2021 13:49:04 +0000 (09:49 -0400)]
18247: Merge branch 'main' into 18247-faraday-version

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18097-sync-groups-doc' into main. Refs #18097
Lucas Di Pentima [Wed, 6 Oct 2021 13:14:29 +0000 (10:14 -0300)]
Merge branch '18097-sync-groups-doc' into main. Refs #18097

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago18247: Merge branch 'main' into 18247-faraday-version
Ward Vandewege [Wed, 6 Oct 2021 13:00:22 +0000 (09:00 -0400)]
18247: Merge branch 'main' into 18247-faraday-version

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate the godoc path for keep-web in our documentation.
Ward Vandewege [Tue, 5 Oct 2021 21:33:11 +0000 (17:33 -0400)]
Update the godoc path for keep-web in our documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18253: documentation updates around the Arvados GPG package signing key.
Ward Vandewege [Tue, 5 Oct 2021 21:17:48 +0000 (17:17 -0400)]
18253: documentation updates around the Arvados GPG package signing key.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix CWL tests refs #18238
Peter Amstutz [Tue, 5 Oct 2021 01:33:00 +0000 (21:33 -0400)]
Fix CWL tests refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Make sure to pull default image
Peter Amstutz [Mon, 4 Oct 2021 21:40:40 +0000 (17:40 -0400)]
18238: Make sure to pull default image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Update tests to use newer arvados/jobs image
Peter Amstutz [Mon, 4 Oct 2021 19:25:32 +0000 (15:25 -0400)]
18238: Update tests to use newer arvados/jobs image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Fix API revision check
Peter Amstutz [Fri, 1 Oct 2021 20:31:06 +0000 (16:31 -0400)]
18238: Fix API revision check

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Check API version before using output_storage_classes
Peter Amstutz [Fri, 1 Oct 2021 14:26:42 +0000 (10:26 -0400)]
18238: Check API version before using output_storage_classes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Make backwards compatible when StorageClasses is not in config
Peter Amstutz [Fri, 1 Oct 2021 14:02:40 +0000 (10:02 -0400)]
18238: Make backwards compatible when StorageClasses is not in config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Add required container_engine parameter to builder
Peter Amstutz [Thu, 30 Sep 2021 20:13:28 +0000 (16:13 -0400)]
18238: Add required container_engine parameter to builder

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Update cwltool and schema-salad deps
Peter Amstutz [Thu, 30 Sep 2021 14:18:12 +0000 (10:18 -0400)]
18238: Update cwltool and schema-salad deps

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18247: fix arvados gem dependency on the faraday gem
Ward Vandewege [Tue, 5 Oct 2021 01:28:53 +0000 (21:28 -0400)]
18247: fix arvados gem dependency on the faraday gem

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18245: Update the "Ruby and Bundler" installation page:
Ward Vandewege [Fri, 1 Oct 2021 20:25:39 +0000 (16:25 -0400)]
18245: Update the "Ruby and Bundler" installation page:
- installing the system bundler package is no longer needed
- the RVM installation method already installs bundler, no need to do
  that explicitly
- update the references to the latest Ruby 2.5.x version
- remove reference to Ubuntu 16.04 which is no longer supported
- update the "install from source" documentation to make it work with
  the Arvados rails packages

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd Workbench to arvbox status, no issue #
Peter Amstutz [Thu, 30 Sep 2021 13:50:08 +0000 (09:50 -0400)]
Add Workbench to arvbox status, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17830: Add test for logging request ID.
Tom Clegg [Wed, 29 Sep 2021 14:17:11 +0000 (10:17 -0400)]
17830: Add test for logging request ID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoThe request_id passing was moved to Rails middleware as part of #17830.
Ward Vandewege [Tue, 28 Sep 2021 18:25:42 +0000 (14:25 -0400)]
The request_id passing was moved to Rails middleware as part of #17830.
Unfortunately, in that process the logging of the request_id parameter
in the rails logs was broken. This commit fixes that.

refs #17830

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18253-doc-gpg-key-updates'
Ward Vandewege [Wed, 6 Oct 2021 12:46:44 +0000 (08:46 -0400)]
Merge branch '18253-doc-gpg-key-updates'

closes #18253

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18253: Merge branch 'main' into 18253-doc-gpg-key-updates
Ward Vandewege [Wed, 6 Oct 2021 12:45:53 +0000 (08:45 -0400)]
18253: Merge branch 'main' into 18253-doc-gpg-key-updates

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18097: Updates arvados-sync-groups doc page.
Lucas Di Pentima [Tue, 5 Oct 2021 22:06:47 +0000 (19:06 -0300)]
18097: Updates arvados-sync-groups doc page.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoUpdate the godoc path for keep-web in our documentation.
Ward Vandewege [Tue, 5 Oct 2021 21:33:11 +0000 (17:33 -0400)]
Update the godoc path for keep-web in our documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18253: documentation updates around the Arvados GPG package signing key.
Ward Vandewege [Tue, 5 Oct 2021 21:17:48 +0000 (17:17 -0400)]
18253: documentation updates around the Arvados GPG package signing key.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18248: Ensure the OS-provided postgres packages are used in Debian's family
Javier Bértoli [Tue, 5 Oct 2021 18:47:03 +0000 (15:47 -0300)]
18248: Ensure the OS-provided postgres packages are used in Debian's family

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoFix CWL tests refs #18238
Peter Amstutz [Tue, 5 Oct 2021 01:33:00 +0000 (21:33 -0400)]
Fix CWL tests refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '18238-update-cwltool' refs #18238
Peter Amstutz [Mon, 4 Oct 2021 21:41:18 +0000 (17:41 -0400)]
Merge branch '18238-update-cwltool' refs #18238

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Make sure to pull default image
Peter Amstutz [Mon, 4 Oct 2021 21:40:40 +0000 (17:40 -0400)]
18238: Make sure to pull default image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Update tests to use newer arvados/jobs image
Peter Amstutz [Mon, 4 Oct 2021 19:25:32 +0000 (15:25 -0400)]
18238: Update tests to use newer arvados/jobs image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Arvbox Singularity / Docker updates wip
Peter Amstutz [Mon, 4 Oct 2021 17:37:42 +0000 (13:37 -0400)]
18238: Arvbox Singularity / Docker updates wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18248: enable debug for cwl-runner in tests
Javier Bértoli [Mon, 4 Oct 2021 15:25:23 +0000 (12:25 -0300)]
18248: enable debug for cwl-runner in tests

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago18238: Fix API revision check
Peter Amstutz [Fri, 1 Oct 2021 20:31:06 +0000 (16:31 -0400)]
18238: Fix API revision check

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Check API version before using output_storage_classes
Peter Amstutz [Fri, 1 Oct 2021 14:26:42 +0000 (10:26 -0400)]
18238: Check API version before using output_storage_classes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Make backwards compatible when StorageClasses is not in config
Peter Amstutz [Fri, 1 Oct 2021 14:02:40 +0000 (10:02 -0400)]
18238: Make backwards compatible when StorageClasses is not in config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Add required container_engine parameter to builder
Peter Amstutz [Thu, 30 Sep 2021 20:13:28 +0000 (16:13 -0400)]
18238: Add required container_engine parameter to builder

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18238: Update cwltool and schema-salad deps
Peter Amstutz [Thu, 30 Sep 2021 14:18:12 +0000 (10:18 -0400)]
18238: Update cwltool and schema-salad deps

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18248: edit provision's snakeoil cert for ubuntu-18.04
Javier Bértoli [Sun, 3 Oct 2021 23:43:14 +0000 (20:43 -0300)]
18248: edit provision's snakeoil cert for ubuntu-18.04

Comment RANDFILE line in /etc/ssl/openssl.cnf to fix ubuntu 18.04 testing

See https://github.com/openssl/openssl/issues/7754#issuecomment-541310006

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '18245-bundler-doc-update'
Ward Vandewege [Fri, 1 Oct 2021 20:52:46 +0000 (16:52 -0400)]
Merge branch '18245-bundler-doc-update'

closes #18245

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18245: Update the "Ruby and Bundler" installation page:
Ward Vandewege [Fri, 1 Oct 2021 20:25:39 +0000 (16:25 -0400)]
18245: Update the "Ruby and Bundler" installation page:
- installing the system bundler package is no longer needed
- the RVM installation method already installs bundler, no need to do
  that explicitly
- update the references to the latest Ruby 2.5.x version
- remove reference to Ubuntu 16.04 which is no longer supported
- update the "install from source" documentation to make it work with
  the Arvados rails packages

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd Workbench to arvbox status, no issue #
Peter Amstutz [Thu, 30 Sep 2021 13:50:08 +0000 (09:50 -0400)]
Add Workbench to arvbox status, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago18216: Move keepstore into arvados-server binary.
Tom Clegg [Thu, 30 Sep 2021 12:54:31 +0000 (08:54 -0400)]
18216: Move keepstore into arvados-server binary.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18003-support-exists-contains-in-filter-groups'
Ward Vandewege [Wed, 29 Sep 2021 19:35:00 +0000 (15:35 -0400)]
Merge branch '18003-support-exists-contains-in-filter-groups'

closes #18003

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18003: Merge branch 'main' into 18003-support-exists-contains-in-filter-groups
Ward Vandewege [Wed, 29 Sep 2021 19:34:29 +0000 (15:34 -0400)]
18003: Merge branch 'main' into 18003-support-exists-contains-in-filter-groups

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17830-fix-rails-request_id-logging'
Ward Vandewege [Wed, 29 Sep 2021 15:03:18 +0000 (11:03 -0400)]
Merge branch '17830-fix-rails-request_id-logging'

closes #17830

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17830: Add test for logging request ID.
Tom Clegg [Wed, 29 Sep 2021 14:17:11 +0000 (10:17 -0400)]
17830: Add test for logging request ID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17779: Fix invalid manifest throwing IOError instead of SyntaxError.
Tom Clegg [Wed, 29 Sep 2021 02:05:36 +0000 (22:05 -0400)]
17779: Fix invalid manifest throwing IOError instead of SyntaxError.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18003: add 'contains' and 'exists' to the permitted operators for filter
Ward Vandewege [Tue, 28 Sep 2021 20:38:34 +0000 (16:38 -0400)]
18003: add 'contains' and 'exists' to the permitted operators for filter
       groups.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17779: Fix misleading "error processing manifest" after IO errors.
Tom Clegg [Tue, 28 Sep 2021 18:25:47 +0000 (14:25 -0400)]
17779: Fix misleading "error processing manifest" after IO errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoThe request_id passing was moved to Rails middleware as part of #17830.
Ward Vandewege [Tue, 28 Sep 2021 18:25:42 +0000 (14:25 -0400)]
The request_id passing was moved to Rails middleware as part of #17830.
Unfortunately, in that process the logging of the request_id parameter
in the rails logs was broken. This commit fixes that.

refs #17830

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17779: Include request ID in exception messages.
Tom Clegg [Tue, 28 Sep 2021 18:24:55 +0000 (14:24 -0400)]
17779: Include request ID in exception messages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoAdd an example to the Python cookbook, "Delete a file from an existing
Ward Vandewege [Mon, 27 Sep 2021 16:29:52 +0000 (12:29 -0400)]
Add an example to the Python cookbook, "Delete a file from an existing
collection".

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate installer and images to Go 1.17.1.
Tom Clegg [Fri, 24 Sep 2021 20:32:34 +0000 (16:32 -0400)]
Update installer and images to Go 1.17.1.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUpdate go build tags to go1.17.1 gofmt.
Tom Clegg [Fri, 24 Sep 2021 20:29:59 +0000 (16:29 -0400)]
Update go build tags to go1.17.1 gofmt.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUpdate (unused?) scopes in our discovery doc to have arvados.org URLs.
Ward Vandewege [Fri, 24 Sep 2021 15:22:36 +0000 (11:22 -0400)]
Update (unused?) scopes in our discovery doc to have arvados.org URLs.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate a few more old domain references.
Ward Vandewege [Fri, 24 Sep 2021 15:21:00 +0000 (11:21 -0400)]
Update a few more old domain references.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '18184-singularity-374'
Tom Clegg [Fri, 24 Sep 2021 15:03:52 +0000 (11:03 -0400)]
Merge branch '18184-singularity-374'

refs #18184

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUpdate a few remaining Jenkins references to ci.arvados.org.
Ward Vandewege [Fri, 24 Sep 2021 15:02:19 +0000 (11:02 -0400)]
Update a few remaining Jenkins references to ci.arvados.org.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe now have the auto-redirect behavior on https://arvadosapi.com/
Ward Vandewege [Fri, 24 Sep 2021 14:52:53 +0000 (10:52 -0400)]
We now have the auto-redirect behavior on https://arvadosapi.com/

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFixes the 'branchname' variable setting used on salt doc pages. Refs #18130
Lucas Di Pentima [Tue, 21 Sep 2021 18:50:42 +0000 (15:50 -0300)]
Fixes the 'branchname' variable setting used on salt doc pages. Refs #18130

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '13697-database-timeout'
Tom Clegg [Thu, 23 Sep 2021 21:01:53 +0000 (17:01 -0400)]
Merge branch '13697-database-timeout'

fixes #13697

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago13967: Add guard comment re timing out large uploads.
Tom Clegg [Thu, 23 Sep 2021 21:01:09 +0000 (17:01 -0400)]
13967: Add guard comment re timing out large uploads.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18184: Install singularity 3.7.4 for dev/test.
Tom Clegg [Thu, 23 Sep 2021 20:53:20 +0000 (16:53 -0400)]
18184: Install singularity 3.7.4 for dev/test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoRemove -x from the salt install provision script.
Ward Vandewege [Thu, 23 Sep 2021 17:21:43 +0000 (13:21 -0400)]
Remove -x from the salt install provision script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago13697: Do not apply RequestTimeout to hijacked connections.
Tom Clegg [Thu, 23 Sep 2021 14:46:42 +0000 (10:46 -0400)]
13697: Do not apply RequestTimeout to hijacked connections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago13697: Remove HandlerWithContext.
Tom Clegg [Thu, 23 Sep 2021 13:44:29 +0000 (09:44 -0400)]
13697: Remove HandlerWithContext.

Use http.Server's BaseContext feature instead.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago13697: Cancel request context after API.RequestTimeout.
Tom Clegg [Thu, 23 Sep 2021 13:41:20 +0000 (09:41 -0400)]
13697: Cancel request context after API.RequestTimeout.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17749-s3-prefixes'
Tom Clegg [Thu, 23 Sep 2021 13:54:26 +0000 (09:54 -0400)]
Merge branch '17749-s3-prefixes'

closes #17749

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '8363-fuse-warnings'
Tom Clegg [Thu, 23 Sep 2021 13:49:31 +0000 (09:49 -0400)]
Merge branch '8363-fuse-warnings'

closes #8363

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18098-log-runtime-engine'
Tom Clegg [Thu, 23 Sep 2021 13:48:56 +0000 (09:48 -0400)]
Merge branch '18098-log-runtime-engine'

closes #18098

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix typo.
Ward Vandewege [Thu, 23 Sep 2021 13:21:25 +0000 (09:21 -0400)]
Fix typo.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago13697: Set Rails database timeout to API.RequestTimeout config.
Tom Clegg [Wed, 22 Sep 2021 20:57:57 +0000 (16:57 -0400)]
13697: Set Rails database timeout to API.RequestTimeout config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago18098: Mention runtime engine (e.g., docker) in crunch-run log.
Tom Clegg [Wed, 22 Sep 2021 20:08:50 +0000 (16:08 -0400)]
18098: Mention runtime engine (e.g., docker) in crunch-run log.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago8363: Save matching arv-mount log line in warningDetail.
Tom Clegg [Wed, 22 Sep 2021 19:34:15 +0000 (15:34 -0400)]
8363: Save matching arv-mount log line in warningDetail.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago8363: Fix spelling of errorDetail and warningDetail in docs.
Tom Clegg [Wed, 22 Sep 2021 18:32:31 +0000 (14:32 -0400)]
8363: Fix spelling of errorDetail and warningDetail in docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago8363: Update tests.
Tom Clegg [Wed, 22 Sep 2021 18:06:06 +0000 (14:06 -0400)]
8363: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '18164-link-perm-read' refs #18164
Peter Amstutz [Wed, 22 Sep 2021 18:02:36 +0000 (14:02 -0400)]
Merge branch '18164-link-perm-read' refs #18164

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>