arvados.git
2 years ago19513: Note configurable permission on doc section about roles.
Tom Clegg [Thu, 24 Nov 2022 19:54:43 +0000 (14:54 -0500)]
19513: Note configurable permission on doc section about roles.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19513: Add Users.CreateRoleGroups config option.
Tom Clegg [Wed, 23 Nov 2022 19:42:23 +0000 (14:42 -0500)]
19513: Add Users.CreateRoleGroups config option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18797-retry-docstrings'.
Brett Smith [Mon, 21 Nov 2022 20:59:22 +0000 (15:59 -0500)]
Merge branch '18797-retry-docstrings'.

Refs #18797. Closes #19788.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years agoAdd 2.4.4 to upgrading notes.
Peter Amstutz [Fri, 18 Nov 2022 14:35:00 +0000 (09:35 -0500)]
Add 2.4.4 to upgrading notes.

refs #19717

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19364-diag-docs'
Tom Clegg [Mon, 21 Nov 2022 16:26:04 +0000 (11:26 -0500)]
Merge branch '19364-diag-docs'

closes #19364

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19698-masked-error'
Tom Clegg [Mon, 21 Nov 2022 15:51:58 +0000 (10:51 -0500)]
Merge branch '19698-masked-error'

fixes #19698

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18797: Update style of arvados.retry docstrings. 18797-retry-docstrings
Brett Smith [Thu, 10 Nov 2022 21:58:21 +0000 (16:58 -0500)]
18797: Update style of arvados.retry docstrings.

I wrote these docstrings as a demonstration of new style rules for our
Python SDK. It is real, useful documentation, as well as a good
illustration of how to follow the new style guide, so we might as well
merge it.

This is primarily a style update to the existing docstrings. In places
where I've actually changed the meaning, I believe the new meaning is
more correct. In particular, RetryLoop's "attempts" count the number of
saved results, which may or may not be the same as the number of times
the loop has run depending on what the body does.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19364: Cancel container request when timing out. 19364-diag-docs
Tom Clegg [Mon, 21 Nov 2022 15:26:34 +0000 (10:26 -0500)]
19364: Cancel container request when timing out.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Proofreading.
Tom Clegg [Mon, 21 Nov 2022 14:54:42 +0000 (09:54 -0500)]
19364: Proofreading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Merge branch 'main' into 19698-masked-error 19698-masked-error
Tom Clegg [Mon, 21 Nov 2022 14:41:39 +0000 (09:41 -0500)]
19698: Merge branch 'main' into 19698-masked-error

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Remove redundant savepoint.
Tom Clegg [Mon, 21 Nov 2022 14:20:31 +0000 (09:20 -0500)]
19698: Remove redundant savepoint.

The existing savepoint is still valid here.  Creating another with the
same name is wasteful.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Merge branch 'main' into 19364-diag-docs
Tom Clegg [Fri, 18 Nov 2022 15:21:19 +0000 (10:21 -0500)]
19364: Merge branch 'main' into 19364-diag-docs

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Add admin>diagnostics page.
Tom Clegg [Thu, 17 Nov 2022 21:33:56 +0000 (16:33 -0500)]
19364: Add admin>diagnostics page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix unused debug variable.
Tom Clegg [Thu, 17 Nov 2022 01:46:09 +0000 (20:46 -0500)]
Fix unused debug variable.

refs #19779

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Document arvados-server check command.
Tom Clegg [Wed, 16 Nov 2022 16:28:06 +0000 (11:28 -0500)]
19364: Document arvados-server check command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19745-update-deps'
Tom Clegg [Wed, 16 Nov 2022 14:15:33 +0000 (09:15 -0500)]
Merge branch '19745-update-deps'

closes #19745

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19364-diag-docs'
Tom Clegg [Tue, 15 Nov 2022 21:01:15 +0000 (16:01 -0500)]
Merge branch '19364-diag-docs'

refs #19364

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix python packaging conflicts with older pip refs #19330
Peter Amstutz [Mon, 8 Aug 2022 22:28:58 +0000 (18:28 -0400)]
Fix python packaging conflicts with older pip refs #19330

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19692-quieter-runtime-status' refs #19692
Peter Amstutz [Tue, 15 Nov 2022 14:31:15 +0000 (09:31 -0500)]
Merge branch '19692-quieter-runtime-status' refs #19692

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19699-cwl-http-dl' refs #19699
Peter Amstutz [Tue, 15 Nov 2022 14:30:41 +0000 (09:30 -0500)]
Merge branch '19699-cwl-http-dl' refs #19699

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19364: Recommend diagnostics during install and upgrade.
Tom Clegg [Mon, 14 Nov 2022 22:35:32 +0000 (17:35 -0500)]
19364: Recommend diagnostics during install and upgrade.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Fix savepoint usage.
Tom Clegg [Mon, 14 Nov 2022 20:23:20 +0000 (15:23 -0500)]
19698: Fix savepoint usage.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19702-memory-overhead'
Tom Clegg [Mon, 14 Nov 2022 19:29:52 +0000 (14:29 -0500)]
Merge branch '19702-memory-overhead'

fixes #19702

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19712-remove-perl'
Tom Clegg [Mon, 14 Nov 2022 19:20:58 +0000 (14:20 -0500)]
Merge branch '19712-remove-perl'

refs #19712

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19688-cwl-fast-path' refs #19688
Peter Amstutz [Mon, 14 Nov 2022 19:03:08 +0000 (14:03 -0500)]
Merge branch '19688-cwl-fast-path' refs #19688

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoDon't send validation warning to runtime status. 19692-quieter-runtime-status
Peter Amstutz [Mon, 14 Nov 2022 15:36:39 +0000 (10:36 -0500)]
Don't send validation warning to runtime status.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add --prefer-cached-downloads, add tests 19699-cwl-http-dl
Peter Amstutz [Wed, 9 Nov 2022 03:24:52 +0000 (22:24 -0500)]
19699: Add --prefer-cached-downloads, add tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Pass through --varying-url-params to runner container
Peter Amstutz [Tue, 8 Nov 2022 18:31:53 +0000 (13:31 -0500)]
19699: Pass through --varying-url-params to runner container

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add --varying-url-params
Peter Amstutz [Tue, 8 Nov 2022 15:08:27 +0000 (10:08 -0500)]
19699: Add --varying-url-params

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add a couple debug statements for etags
Peter Amstutz [Mon, 7 Nov 2022 22:00:58 +0000 (17:00 -0500)]
19699: Add a couple debug statements for etags

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Tighten up etag handling a bit
Peter Amstutz [Mon, 7 Nov 2022 21:59:05 +0000 (16:59 -0500)]
19699: Tighten up etag handling a bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Report download done, don't try to stage deferred downloads
Peter Amstutz [Mon, 7 Nov 2022 21:46:37 +0000 (16:46 -0500)]
19699: Report download done, don't try to stage deferred downloads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Accomodate failed HEAD requests, add If-None-Match
Peter Amstutz [Mon, 7 Nov 2022 21:24:25 +0000 (16:24 -0500)]
19699: Accomodate failed HEAD requests, add If-None-Match

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Ensure collection name doesn't go over the limit
Peter Amstutz [Fri, 4 Nov 2022 01:14:00 +0000 (21:14 -0400)]
19699: Ensure collection name doesn't go over the limit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add option to defer downloads
Peter Amstutz [Fri, 4 Nov 2022 01:01:03 +0000 (21:01 -0400)]
19699: Add option to defer downloads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Fix test 19688-cwl-fast-path
Peter Amstutz [Mon, 14 Nov 2022 16:24:44 +0000 (11:24 -0500)]
19688: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19745: Avoid incompatible google-api-core update. 19745-update-deps
Tom Clegg [Mon, 14 Nov 2022 15:59:28 +0000 (10:59 -0500)]
19745: Avoid incompatible google-api-core update.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19745: Update ruby, node, wb2.
Tom Clegg [Mon, 14 Nov 2022 07:43:38 +0000 (02:43 -0500)]
19745: Update ruby, node, wb2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19745: Update golang.
Tom Clegg [Mon, 14 Nov 2022 07:30:10 +0000 (02:30 -0500)]
19745: Update golang.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Make sure the wrapper has git info too
Peter Amstutz [Mon, 14 Nov 2022 00:52:37 +0000 (19:52 -0500)]
19688: Make sure the wrapper has git info too

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Fix tests
Peter Amstutz [Sun, 13 Nov 2022 23:19:53 +0000 (18:19 -0500)]
19688: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Set step id to match original workflow id
Peter Amstutz [Sat, 12 Nov 2022 01:26:37 +0000 (20:26 -0500)]
19688: Set step id to match original workflow id

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Set label so subwf isn't displayed as "step"
Peter Amstutz [Fri, 11 Nov 2022 22:55:55 +0000 (17:55 -0500)]
19688: Set label so subwf isn't displayed as "step"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Make sure CommandLineTool gets wrapped properly
Peter Amstutz [Fri, 11 Nov 2022 14:53:18 +0000 (09:53 -0500)]
19688: Make sure CommandLineTool gets wrapped properly

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19588-update-passenger'
Tom Clegg [Thu, 10 Nov 2022 19:32:47 +0000 (14:32 -0500)]
Merge branch '19588-update-passenger'

refs #19588

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19712: Remove link to perl sdk docs. 19712-remove-perl
Tom Clegg [Wed, 9 Nov 2022 14:28:21 +0000 (09:28 -0500)]
19712: Remove link to perl sdk docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19712: Remove perl sdk.
Tom Clegg [Tue, 8 Nov 2022 19:45:55 +0000 (14:45 -0500)]
19712: Remove perl sdk.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19588: Update passenger. 19588-update-passenger
Tom Clegg [Tue, 8 Nov 2022 19:40:56 +0000 (14:40 -0500)]
19588: Update passenger.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19702: Update tests. 19702-memory-overhead
Tom Clegg [Tue, 8 Nov 2022 18:19:16 +0000 (13:19 -0500)]
19702: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Update tests
Peter Amstutz [Tue, 8 Nov 2022 17:24:21 +0000 (12:24 -0500)]
19688: Update tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19702: Increase default ReserveExtraRAM, allow keepstore overhead.
Tom Clegg [Tue, 8 Nov 2022 16:56:36 +0000 (11:56 -0500)]
19702: Increase default ReserveExtraRAM, allow keepstore overhead.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19710-fix-redmine-links'.
Brett Smith [Tue, 8 Nov 2022 15:40:38 +0000 (10:40 -0500)]
Merge branch '19710-fix-redmine-links'.

Closes #19710.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19710: Fix broken Redmine links throughout. 19710-fix-redmine-links
Brett Smith [Tue, 8 Nov 2022 14:34:42 +0000 (09:34 -0500)]
19710: Fix broken Redmine links throughout.

There's a handful of Redmine links throughout the source code that still
use arvados.org as their host part. None of the links work after the site
reorganization. I found them all by running:

    git grep -F /arvados.org/projects/

This commit fixes all these links. Most were fixed by simply updating
the host part to dev.arvados.org. The "Introduction to Arvados" wiki
page now redirects you to the front page of arvados.org, so I updated
that link to match. I confirmed by hand they all work now.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years agoMerge branch '19240-check-redirect'
Tom Clegg [Tue, 8 Nov 2022 14:22:01 +0000 (09:22 -0500)]
Merge branch '19240-check-redirect'

refs #19240

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Treat localhost as a private-network client.
Tom Clegg [Tue, 8 Nov 2022 14:21:49 +0000 (09:21 -0500)]
19240: Treat localhost as a private-network client.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19240-check-redirect'
Tom Clegg [Mon, 7 Nov 2022 15:40:17 +0000 (10:40 -0500)]
Merge branch '19240-check-redirect'

fixes #19240

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Update federation setup docs to list wb+wb2.
Tom Clegg [Mon, 7 Nov 2022 15:27:30 +0000 (10:27 -0500)]
19240: Update federation setup docs to list wb+wb2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Mention federation case in upgrade note.
Tom Clegg [Mon, 7 Nov 2022 15:09:01 +0000 (10:09 -0500)]
19240: Mention federation case in upgrade note.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Add upgrade note.
Tom Clegg [Fri, 4 Nov 2022 19:06:20 +0000 (15:06 -0400)]
19240: Add upgrade note.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19234-default-s3-region'
Tom Clegg [Fri, 4 Nov 2022 16:44:36 +0000 (12:44 -0400)]
Merge branch '19234-default-s3-region'

fixes #19234

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Update gofmt.
Tom Clegg [Fri, 4 Nov 2022 16:43:52 +0000 (12:43 -0400)]
19240: Update gofmt.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19240: Restrict login/logout redirects to listed URLs.
Tom Clegg [Fri, 4 Nov 2022 16:43:50 +0000 (12:43 -0400)]
19240: Restrict login/logout redirects to listed URLs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Recognize plain workflow uuid without arvwf: prefix
Peter Amstutz [Fri, 4 Nov 2022 01:49:53 +0000 (21:49 -0400)]
19688: Recognize plain workflow uuid without arvwf: prefix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19240: Fix url comparison.
Tom Clegg [Fri, 4 Nov 2022 01:33:02 +0000 (21:33 -0400)]
19240: Fix url comparison.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Fix @stubs decorator that was accidentally skipping tests
Peter Amstutz [Thu, 3 Nov 2022 19:44:20 +0000 (15:44 -0400)]
19688: Fix @stubs decorator that was accidentally skipping tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19234: Ignore Region when using custom Endpoint with s3v1 driver. 19234-default-s3-region
Tom Clegg [Wed, 2 Nov 2022 19:41:40 +0000 (15:41 -0400)]
19234: Ignore Region when using custom Endpoint with s3v1 driver.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19234: s3v2 + non-aws: default to us-east-1 signing settings.
Tom Clegg [Wed, 2 Nov 2022 19:01:15 +0000 (15:01 -0400)]
19234: s3v2 + non-aws: default to us-east-1 signing settings.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19683-arvbox-rubysdk-fix'. Closes #19683
Lucas Di Pentima [Wed, 2 Nov 2022 18:20:46 +0000 (15:20 -0300)]
Merge branch '19683-arvbox-rubysdk-fix'. Closes #19683

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19688: Make registered workflows lightweight wrappers
Peter Amstutz [Wed, 2 Nov 2022 03:17:51 +0000 (23:17 -0400)]
19688: Make registered workflows lightweight wrappers

Puts the actual workflow in a collection

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Trying a fast path submit for registered workflows
Peter Amstutz [Wed, 2 Nov 2022 01:34:35 +0000 (21:34 -0400)]
19688: Trying a fast path submit for registered workflows

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19683: Improves pattern matching on the arvados gem. 19683-arvbox-rubysdk-fix
Lucas Di Pentima [Tue, 1 Nov 2022 17:59:05 +0000 (14:59 -0300)]
19683: Improves pattern matching on the arvados gem.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19683: Improves code style.
Lucas Di Pentima [Tue, 1 Nov 2022 17:30:14 +0000 (14:30 -0300)]
19683: Improves code style.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19683: Builds & installs the arvados gem dev version.
Lucas Di Pentima [Tue, 1 Nov 2022 16:35:42 +0000 (13:35 -0300)]
19683: Builds & installs the arvados gem dev version.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '19582-aws-s3v2-driver'. Closes #19582.
Lucas Di Pentima [Mon, 31 Oct 2022 20:22:40 +0000 (17:22 -0300)]
Merge branch '19582-aws-s3v2-driver'. Closes #19582.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '19678-job-loader' refs #19678
Peter Amstutz [Mon, 31 Oct 2022 20:12:09 +0000 (16:12 -0400)]
Merge branch '19678-job-loader' refs #19678

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch 'main' into 19582-aws-s3v2-driver 19582-aws-s3v2-driver
Lucas Di Pentima [Mon, 31 Oct 2022 19:22:07 +0000 (16:22 -0300)]
Merge branch 'main' into 19582-aws-s3v2-driver

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18071-dblock-keep-balance-and-dispatch'
Tom Clegg [Mon, 31 Oct 2022 17:12:31 +0000 (13:12 -0400)]
Merge branch '18071-dblock-keep-balance-and-dispatch'

closes #18071

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Remove obsolete comment.
Tom Clegg [Mon, 31 Oct 2022 17:11:57 +0000 (13:11 -0400)]
18071: Remove obsolete comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19678: Fix for parameters called 'name' 19678-job-loader
Peter Amstutz [Fri, 28 Oct 2022 21:30:39 +0000 (17:30 -0400)]
19678: Fix for parameters called 'name'

Also fix regression involving default file references appearing in
nested processes (inline declaration of a tool within a workflow).

Also fixed some dependency issues preventing arvados/jobs developer
image from building.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19678: Use minimal job loader context to expand job order, same as cwltool
Peter Amstutz [Fri, 28 Oct 2022 18:36:32 +0000 (14:36 -0400)]
19678: Use minimal job loader context to expand job order, same as cwltool

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18071: Fix tests. 18071-dblock-keep-balance-and-dispatch
Tom Clegg [Fri, 28 Oct 2022 18:33:45 +0000 (14:33 -0400)]
18071: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Relax test timeout.
Tom Clegg [Fri, 28 Oct 2022 18:18:59 +0000 (14:18 -0400)]
18071: Relax test timeout.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Improve test timeout.
Tom Clegg [Fri, 28 Oct 2022 15:08:39 +0000 (11:08 -0400)]
18071: Improve test timeout.

Test can take longer when output is going to a terminal. Keep trying
if some progress is made every 3s, rather than giving up after 10s
total.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Use dblock to prevent multiple dispatchers from competing.
Tom Clegg [Fri, 28 Oct 2022 14:35:27 +0000 (10:35 -0400)]
18071: Use dblock to prevent multiple dispatchers from competing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Log host:port of other process(es) holding lock.
Tom Clegg [Thu, 27 Oct 2022 15:05:42 +0000 (11:05 -0400)]
18071: Log host:port of other process(es) holding lock.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18071: Use dblock to avoid concurrent keep-balance ops.
Tom Clegg [Wed, 26 Oct 2022 21:00:42 +0000 (17:00 -0400)]
18071: Use dblock to avoid concurrent keep-balance ops.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19377-verbose'
Tom Clegg [Thu, 27 Oct 2022 15:01:54 +0000 (11:01 -0400)]
Merge branch '19377-verbose'

closes #19377

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19377: Usage error if config is readable but does not match env. 19377-verbose
Tom Clegg [Thu, 27 Oct 2022 14:07:43 +0000 (10:07 -0400)]
19377: Usage error if config is readable but does not match env.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19563-log-cr-mem'
Tom Clegg [Wed, 26 Oct 2022 21:19:42 +0000 (17:19 -0400)]
Merge branch '19563-log-cr-mem'

refs #19563

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19563: Report process size in bytes, not pages. 19563-log-cr-mem
Tom Clegg [Tue, 25 Oct 2022 18:00:38 +0000 (14:00 -0400)]
19563: Report process size in bytes, not pages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19377: Improve log message about skipping health check during diags.
Tom Clegg [Tue, 25 Oct 2022 14:45:41 +0000 (10:45 -0400)]
19377: Improve log message about skipping health check during diags.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19563: Clean up tests.
Tom Clegg [Tue, 25 Oct 2022 14:22:53 +0000 (10:22 -0400)]
19563: Clean up tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19466-cwl-io' refs #19466
Peter Amstutz [Mon, 24 Oct 2022 21:05:18 +0000 (17:05 -0400)]
Merge branch '19466-cwl-io' refs #19466

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19466: Add link to section later in the page
Peter Amstutz [Mon, 24 Oct 2022 21:03:52 +0000 (17:03 -0400)]
19466: Add link to section later in the page

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19563: Log memory usage of arv-mount, crunch-run, and keepstore.
Tom Clegg [Mon, 24 Oct 2022 20:45:45 +0000 (16:45 -0400)]
19563: Log memory usage of arv-mount, crunch-run, and keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19466: Fix typo and test with cwl_ properties 19466-cwl-io
Peter Amstutz [Fri, 21 Oct 2022 20:57:29 +0000 (16:57 -0400)]
19466: Fix typo and test with cwl_ properties

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19466: Fix properties page typos
Peter Amstutz [Fri, 21 Oct 2022 20:54:45 +0000 (16:54 -0400)]
19466: Fix properties page typos

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19466: Fix incorrect cwl_ properties in table
Peter Amstutz [Fri, 21 Oct 2022 20:31:11 +0000 (16:31 -0400)]
19466: Fix incorrect cwl_ properties in table

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19466: Add an API documentation page about properties
Peter Amstutz [Fri, 21 Oct 2022 20:28:56 +0000 (16:28 -0400)]
19466: Add an API documentation page about properties

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19629-nokogiri-upgrade'. Closes #19629
Lucas Di Pentima [Fri, 21 Oct 2022 13:40:07 +0000 (10:40 -0300)]
Merge branch '19629-nokogiri-upgrade'. Closes #19629

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>