arvados.git
2 years agoMerge branch '19686-threadsafe-api-default'
Brett Smith [Thu, 8 Dec 2022 16:05:57 +0000 (11:05 -0500)]
Merge branch '19686-threadsafe-api-default'

Closes #19686.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19686: Specify ThreadSafeApiCache API version throughout
Brett Smith [Thu, 8 Dec 2022 15:51:58 +0000 (10:51 -0500)]
19686: Specify ThreadSafeApiCache API version throughout

This avoids logging a warning now that ThreadSafeApiCache accepts a
version argument just like other API client constructors.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19686: Fix test scaffolding after API changes
Brett Smith [Wed, 7 Dec 2022 20:21:43 +0000 (15:21 -0500)]
19686: Fix test scaffolding after API changes

Without clearing arv-put's API client this way, multiple calls to main()
reuse the same, pre-patch Keep object from ThreadSafeApiCache, so calls
don't get recorded as intended.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19686: Add tests for new Arvados client API
Brett Smith [Wed, 7 Dec 2022 04:16:28 +0000 (23:16 -0500)]
19686: Add tests for new Arvados client API

This commit ensures all public-facing functions have basic tests for
good inputs, as well as known-bad argument combinations (if any). This
includes testing functions like api_from_config that were previously
tested implicitly by other tests, but did not have their own dedicated
tests.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19686: api constructor returns ThreadSafeApiCache
Brett Smith [Tue, 29 Nov 2022 20:01:43 +0000 (15:01 -0500)]
19686: api constructor returns ThreadSafeApiCache

This is an API-compatible wrapper object that provides thread
safety. Returning this from api() helps keep users out of
trouble.

The changes to ThreadSafeApiCache are required to keep it API-compatible
with the original and keep tests passing. Assignments to the request_id
attribute need to be used for all future requests. It's a little unclear
if this is an intended API or just test scaffolding, but it's not too
difficult to keep working so I just did that.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19686: Introduce low-level api_client constructor
Brett Smith [Mon, 28 Nov 2022 21:30:28 +0000 (16:30 -0500)]
19686: Introduce low-level api_client constructor

When api() returns a ThreadSafeApiCache, we still want to provide a
mechanism to get a plain Resource object. The api_client() function is
that mechanism. It *just* builds the Resource object as api() did
before.

normalize_api_kwargs() takes the arguments passed to api() and turns
them into keyword arguments for api_client(). api_config_to_kwargs()
takes a configuration mapping nand turns them into keyword arguments for
api_client(). Both of these are small APIs, just returning one
dictionary from another.

With this reorganization, api(), api_from_config(), and
ThreadSafeApiClient() can all have simpler implementations.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19791: Remove duplicate word
Brett Smith [Wed, 7 Dec 2022 15:57:41 +0000 (10:57 -0500)]
19791: Remove duplicate word

No issue #

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years agoMerge branch '19791-python-api-overview'
Brett Smith [Tue, 6 Dec 2022 20:18:11 +0000 (15:18 -0500)]
Merge branch '19791-python-api-overview'

Closes #19791.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19791: Clarify the background discussion for using keyset_list_all
Brett Smith [Tue, 6 Dec 2022 18:23:07 +0000 (13:23 -0500)]
19791: Clarify the background discussion for using keyset_list_all

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19791: Add a note explaining the page's intended scope
Brett Smith [Tue, 6 Dec 2022 18:16:39 +0000 (13:16 -0500)]
19791: Add a note explaining the page's intended scope

I wrote a note to this effect during the branch review, and I hope
putting it here can help guide future authors (possibly including future
me).

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19791: Expand Python SDK examples into API client overview
Brett Smith [Fri, 25 Nov 2022 16:56:44 +0000 (11:56 -0500)]
19791: Expand Python SDK examples into API client overview

This change takes the skeleton of the existing examples page and expands
it into a fuller overview of the API client provided by the Python
SDK. It provides more explanation of how the client works; how it
corresponds to the Arvados API; more concrete examples of real tasks you
might actually want to do with the client; and a few different examples
to demonstrate how you might use different client code patterns for
different situations.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years agoMerge branch '19513-create-role-admin-only'
Tom Clegg [Tue, 6 Dec 2022 16:22:46 +0000 (11:22 -0500)]
Merge branch '19513-create-role-admin-only'

closes #19513

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix broken documentation link refs #19215
Peter Amstutz [Mon, 5 Dec 2022 22:02:59 +0000 (17:02 -0500)]
Fix broken documentation link refs #19215

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19215-installer' refs #19215
Peter Amstutz [Mon, 5 Dec 2022 18:43:01 +0000 (13:43 -0500)]
Merge branch '19215-installer' refs #19215

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Adjusts the documentation to reflect recent changes. 19215-installer
Lucas Di Pentima [Mon, 5 Dec 2022 18:24:32 +0000 (19:24 +0100)]
19215: Adjusts the documentation to reflect recent changes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Properly outputs LE's secret access key.
Lucas Di Pentima [Mon, 5 Dec 2022 18:05:09 +0000 (19:05 +0100)]
19215: Properly outputs LE's secret access key.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Fixes one remaining IP related variable.
Lucas Di Pentima [Fri, 2 Dec 2022 22:58:07 +0000 (19:58 -0300)]
19215: Fixes one remaining IP related variable.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Adds private subnet & NAT gateway for compute nodes.
Lucas Di Pentima [Fri, 2 Dec 2022 22:54:22 +0000 (19:54 -0300)]
19215: Adds private subnet & NAT gateway for compute nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19513: Rename flag to CanCreateRoleGroups, add test case. 19513-create-role-admin-only
Tom Clegg [Fri, 2 Dec 2022 20:09:55 +0000 (15:09 -0500)]
19513: Rename flag to CanCreateRoleGroups, add test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18842-arv-mount-disk-config' refs #18842
Peter Amstutz [Fri, 2 Dec 2022 18:28:28 +0000 (13:28 -0500)]
Merge branch '18842-arv-mount-disk-config' refs #18842

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Add -o pipefail to installer.sh
Peter Amstutz [Fri, 2 Dec 2022 17:11:09 +0000 (12:11 -0500)]
19215: Add -o pipefail to installer.sh

This avoids having "| tee" suppress errors that should have caused the
script to stop.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: One more refactor 18842-arv-mount-disk-config
Peter Amstutz [Fri, 2 Dec 2022 16:30:43 +0000 (11:30 -0500)]
18842: One more refactor

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Need to call ready.set() on base cases
Peter Amstutz [Fri, 2 Dec 2022 16:24:46 +0000 (11:24 -0500)]
18842: Need to call ready.set() on base cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Clean up keep cache set logic a little more
Peter Amstutz [Fri, 2 Dec 2022 16:14:51 +0000 (11:14 -0500)]
18842: Clean up keep cache set logic a little more

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Clean up keep cache set() a little bit
Peter Amstutz [Fri, 2 Dec 2022 16:04:27 +0000 (11:04 -0500)]
18842: Clean up keep cache set() a little bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Fix *_INT_IP defaults and Terraform CLI install text.
Peter Amstutz [Thu, 1 Dec 2022 22:37:28 +0000 (17:37 -0500)]
19215: Fix *_INT_IP defaults and Terraform CLI install text.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoarvbox changes: yank Composer. Add link to documentation.
Peter Amstutz [Thu, 1 Dec 2022 18:41:38 +0000 (13:41 -0500)]
arvbox changes: yank Composer.  Add link to documentation.

Also set locale.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoSet up ruby sdk, no issue #
Peter Amstutz [Thu, 3 Nov 2022 14:48:29 +0000 (10:48 -0400)]
Set up ruby sdk, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Move the note about installing Terraform CLI
Peter Amstutz [Thu, 1 Dec 2022 16:40:30 +0000 (11:40 -0500)]
19215: Move the note about installing Terraform CLI

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: A few tweaks about setting up compute image.
Peter Amstutz [Thu, 1 Dec 2022 16:15:52 +0000 (11:15 -0500)]
19215: A few tweaks about setting up compute image.

Make packer script create a log file.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: More documentation details
Peter Amstutz [Wed, 30 Nov 2022 22:13:04 +0000 (17:13 -0500)]
19215: More documentation details

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Polishing install process
Peter Amstutz [Wed, 30 Nov 2022 21:25:00 +0000 (16:25 -0500)]
19215: Polishing install process

* Coordinate terraform using installer.sh
* Doc updates (WIP)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Adds region_name as output.
Lucas Di Pentima [Wed, 30 Nov 2022 15:00:18 +0000 (12:00 -0300)]
19215: Adds region_name as output.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoWhen comparing manifest_text, strip signatures
Peter Amstutz [Tue, 29 Nov 2022 15:47:57 +0000 (10:47 -0500)]
When comparing manifest_text, strip signatures

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19699-cwl-dl-docs' refs #19699
Peter Amstutz [Tue, 29 Nov 2022 15:26:36 +0000 (10:26 -0500)]
Merge branch '19699-cwl-dl-docs' refs #19699

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Update from review comments 19699-cwl-dl-docs
Peter Amstutz [Tue, 29 Nov 2022 14:58:50 +0000 (09:58 -0500)]
19699: Update from review comments

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Add a couple of error handling tests
Peter Amstutz [Mon, 28 Nov 2022 22:29:34 +0000 (17:29 -0500)]
18842: Add a couple of error handling tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Turn disk write errors into KeepCacheError
Peter Amstutz [Mon, 28 Nov 2022 21:39:03 +0000 (16:39 -0500)]
18842: Turn disk write errors into KeepCacheError

* DiskCacheSlot properly keeps track of the file handle now
* Failure to write disk block to cache will attempt to shrink cache
and try again, if it still fails, turns into KeepCacheError
* init_cache writes a test block to check for working cache

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Adds hardcoded deploy_user output set to "admin".
Lucas Di Pentima [Mon, 28 Nov 2022 21:33:21 +0000 (18:33 -0300)]
19215: Adds hardcoded deploy_user output set to "admin".

Debian AMIs have their default non-root user set to "admin", so admins will
need to set up the local.params DEPLOY_USER var accordingly.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoFix manifest comparisons in new collection tests
Brett Smith [Mon, 28 Nov 2022 20:25:03 +0000 (15:25 -0500)]
Fix manifest comparisons in new collection tests

The previous version of these tests was comparing bound method objects
directly, rather than comparing the manifest texts returned by calling
them. Apparently in some setups this comparison returns True, but on my
development box it returns False, causing test failures. We haven't
fully diagnosed what causes the difference, but it seems clear enough
that comparing the method objects isn't the intent of these tests, but
rather they should compare the returned manifests. This commit updates
them to do so. No issue #.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19215: Fixes IAM policy example for Keepstore S3 access.
Lucas Di Pentima [Mon, 28 Nov 2022 14:58:33 +0000 (11:58 -0300)]
19215: Fixes IAM policy example for Keepstore S3 access.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Sets up the provision.sh script to use our own postgres formula fork.
Lucas Di Pentima [Mon, 28 Nov 2022 14:53:07 +0000 (11:53 -0300)]
19215: Sets up the provision.sh script to use our own postgres formula fork.

This is a temporary measure until it gets properly fixed. See:
https://github.com/saltstack-formulas/postgres-formula/issues/327

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Adds documentation on Terraform code.
Lucas Di Pentima [Mon, 28 Nov 2022 14:39:49 +0000 (11:39 -0300)]
19215: Adds documentation on Terraform code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19215: Adds initial version of terraform code for multi-host deploy in AWS.
Lucas Di Pentima [Wed, 23 Nov 2022 01:29:27 +0000 (22:29 -0300)]
19215: Adds initial version of terraform code for multi-host deploy in AWS.

It's separated in 3 sections: vpc, data-storage & services. This is to limit
the 'blast radius' of a potential error when applying changes, as recommended
in many places.
Each state should be applied in the order described above, and their outputs
feed the following states with important data.
The shared 'terraform.tfvars' file allows the operator to customize their
deployment.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years ago19513: Note configurable permission on doc section about roles.
Tom Clegg [Thu, 24 Nov 2022 19:54:43 +0000 (14:54 -0500)]
19513: Note configurable permission on doc section about roles.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19699: Add section about data import
Peter Amstutz [Wed, 23 Nov 2022 20:08:37 +0000 (15:08 -0500)]
19699: Add section about data import

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Refresh arvados-cwl-runner table of options
Peter Amstutz [Wed, 23 Nov 2022 19:53:27 +0000 (14:53 -0500)]
19699: Refresh arvados-cwl-runner table of options

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19513: Add Users.CreateRoleGroups config option.
Tom Clegg [Wed, 23 Nov 2022 19:42:23 +0000 (14:42 -0500)]
19513: Add Users.CreateRoleGroups config option.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18842: Added tests for specific disk cache behavior
Peter Amstutz [Tue, 22 Nov 2022 22:02:01 +0000 (17:02 -0500)]
18842: Added tests for specific disk cache behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18842: Use tempdir for cache directory in tests
Peter Amstutz [Tue, 22 Nov 2022 21:12:20 +0000 (16:12 -0500)]
18842: Use tempdir for cache directory in tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch 'main' into 18842-arv-mount-disk-config
Peter Amstutz [Tue, 22 Nov 2022 18:49:01 +0000 (13:49 -0500)]
Merge branch 'main' into 18842-arv-mount-disk-config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18797-retry-docstrings'.
Brett Smith [Mon, 21 Nov 2022 20:59:22 +0000 (15:59 -0500)]
Merge branch '18797-retry-docstrings'.

Refs #18797. Closes #19788.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years agoAdd 2.4.4 to upgrading notes.
Peter Amstutz [Fri, 18 Nov 2022 14:35:00 +0000 (09:35 -0500)]
Add 2.4.4 to upgrading notes.

refs #19717

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19364-diag-docs'
Tom Clegg [Mon, 21 Nov 2022 16:26:04 +0000 (11:26 -0500)]
Merge branch '19364-diag-docs'

closes #19364

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19698-masked-error'
Tom Clegg [Mon, 21 Nov 2022 15:51:58 +0000 (10:51 -0500)]
Merge branch '19698-masked-error'

fixes #19698

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18797: Update style of arvados.retry docstrings. 18797-retry-docstrings
Brett Smith [Thu, 10 Nov 2022 21:58:21 +0000 (16:58 -0500)]
18797: Update style of arvados.retry docstrings.

I wrote these docstrings as a demonstration of new style rules for our
Python SDK. It is real, useful documentation, as well as a good
illustration of how to follow the new style guide, so we might as well
merge it.

This is primarily a style update to the existing docstrings. In places
where I've actually changed the meaning, I believe the new meaning is
more correct. In particular, RetryLoop's "attempts" count the number of
saved results, which may or may not be the same as the number of times
the loop has run depending on what the body does.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 years ago19364: Cancel container request when timing out. 19364-diag-docs
Tom Clegg [Mon, 21 Nov 2022 15:26:34 +0000 (10:26 -0500)]
19364: Cancel container request when timing out.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Proofreading.
Tom Clegg [Mon, 21 Nov 2022 14:54:42 +0000 (09:54 -0500)]
19364: Proofreading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Merge branch 'main' into 19698-masked-error 19698-masked-error
Tom Clegg [Mon, 21 Nov 2022 14:41:39 +0000 (09:41 -0500)]
19698: Merge branch 'main' into 19698-masked-error

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Remove redundant savepoint.
Tom Clegg [Mon, 21 Nov 2022 14:20:31 +0000 (09:20 -0500)]
19698: Remove redundant savepoint.

The existing savepoint is still valid here.  Creating another with the
same name is wasteful.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Merge branch 'main' into 19364-diag-docs
Tom Clegg [Fri, 18 Nov 2022 15:21:19 +0000 (10:21 -0500)]
19364: Merge branch 'main' into 19364-diag-docs

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Add admin>diagnostics page.
Tom Clegg [Thu, 17 Nov 2022 21:33:56 +0000 (16:33 -0500)]
19364: Add admin>diagnostics page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix unused debug variable.
Tom Clegg [Thu, 17 Nov 2022 01:46:09 +0000 (20:46 -0500)]
Fix unused debug variable.

refs #19779

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19364: Document arvados-server check command.
Tom Clegg [Wed, 16 Nov 2022 16:28:06 +0000 (11:28 -0500)]
19364: Document arvados-server check command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19745-update-deps'
Tom Clegg [Wed, 16 Nov 2022 14:15:33 +0000 (09:15 -0500)]
Merge branch '19745-update-deps'

closes #19745

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19364-diag-docs'
Tom Clegg [Tue, 15 Nov 2022 21:01:15 +0000 (16:01 -0500)]
Merge branch '19364-diag-docs'

refs #19364

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix python packaging conflicts with older pip refs #19330
Peter Amstutz [Mon, 8 Aug 2022 22:28:58 +0000 (18:28 -0400)]
Fix python packaging conflicts with older pip refs #19330

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19692-quieter-runtime-status' refs #19692
Peter Amstutz [Tue, 15 Nov 2022 14:31:15 +0000 (09:31 -0500)]
Merge branch '19692-quieter-runtime-status' refs #19692

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19699-cwl-http-dl' refs #19699
Peter Amstutz [Tue, 15 Nov 2022 14:30:41 +0000 (09:30 -0500)]
Merge branch '19699-cwl-http-dl' refs #19699

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19364: Recommend diagnostics during install and upgrade.
Tom Clegg [Mon, 14 Nov 2022 22:35:32 +0000 (17:35 -0500)]
19364: Recommend diagnostics during install and upgrade.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19698: Fix savepoint usage.
Tom Clegg [Mon, 14 Nov 2022 20:23:20 +0000 (15:23 -0500)]
19698: Fix savepoint usage.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19702-memory-overhead'
Tom Clegg [Mon, 14 Nov 2022 19:29:52 +0000 (14:29 -0500)]
Merge branch '19702-memory-overhead'

fixes #19702

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19712-remove-perl'
Tom Clegg [Mon, 14 Nov 2022 19:20:58 +0000 (14:20 -0500)]
Merge branch '19712-remove-perl'

refs #19712

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19688-cwl-fast-path' refs #19688
Peter Amstutz [Mon, 14 Nov 2022 19:03:08 +0000 (14:03 -0500)]
Merge branch '19688-cwl-fast-path' refs #19688

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoDon't send validation warning to runtime status. 19692-quieter-runtime-status
Peter Amstutz [Mon, 14 Nov 2022 15:36:39 +0000 (10:36 -0500)]
Don't send validation warning to runtime status.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add --prefer-cached-downloads, add tests 19699-cwl-http-dl
Peter Amstutz [Wed, 9 Nov 2022 03:24:52 +0000 (22:24 -0500)]
19699: Add --prefer-cached-downloads, add tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Pass through --varying-url-params to runner container
Peter Amstutz [Tue, 8 Nov 2022 18:31:53 +0000 (13:31 -0500)]
19699: Pass through --varying-url-params to runner container

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add --varying-url-params
Peter Amstutz [Tue, 8 Nov 2022 15:08:27 +0000 (10:08 -0500)]
19699: Add --varying-url-params

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add a couple debug statements for etags
Peter Amstutz [Mon, 7 Nov 2022 22:00:58 +0000 (17:00 -0500)]
19699: Add a couple debug statements for etags

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Tighten up etag handling a bit
Peter Amstutz [Mon, 7 Nov 2022 21:59:05 +0000 (16:59 -0500)]
19699: Tighten up etag handling a bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Report download done, don't try to stage deferred downloads
Peter Amstutz [Mon, 7 Nov 2022 21:46:37 +0000 (16:46 -0500)]
19699: Report download done, don't try to stage deferred downloads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Accomodate failed HEAD requests, add If-None-Match
Peter Amstutz [Mon, 7 Nov 2022 21:24:25 +0000 (16:24 -0500)]
19699: Accomodate failed HEAD requests, add If-None-Match

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Ensure collection name doesn't go over the limit
Peter Amstutz [Fri, 4 Nov 2022 01:14:00 +0000 (21:14 -0400)]
19699: Ensure collection name doesn't go over the limit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19699: Add option to defer downloads
Peter Amstutz [Fri, 4 Nov 2022 01:01:03 +0000 (21:01 -0400)]
19699: Add option to defer downloads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Fix test 19688-cwl-fast-path
Peter Amstutz [Mon, 14 Nov 2022 16:24:44 +0000 (11:24 -0500)]
19688: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19745: Avoid incompatible google-api-core update. 19745-update-deps
Tom Clegg [Mon, 14 Nov 2022 15:59:28 +0000 (10:59 -0500)]
19745: Avoid incompatible google-api-core update.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19745: Update ruby, node, wb2.
Tom Clegg [Mon, 14 Nov 2022 07:43:38 +0000 (02:43 -0500)]
19745: Update ruby, node, wb2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19745: Update golang.
Tom Clegg [Mon, 14 Nov 2022 07:30:10 +0000 (02:30 -0500)]
19745: Update golang.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Make sure the wrapper has git info too
Peter Amstutz [Mon, 14 Nov 2022 00:52:37 +0000 (19:52 -0500)]
19688: Make sure the wrapper has git info too

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Fix tests
Peter Amstutz [Sun, 13 Nov 2022 23:19:53 +0000 (18:19 -0500)]
19688: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Set step id to match original workflow id
Peter Amstutz [Sat, 12 Nov 2022 01:26:37 +0000 (20:26 -0500)]
19688: Set step id to match original workflow id

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Set label so subwf isn't displayed as "step"
Peter Amstutz [Fri, 11 Nov 2022 22:55:55 +0000 (17:55 -0500)]
19688: Set label so subwf isn't displayed as "step"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19688: Make sure CommandLineTool gets wrapped properly
Peter Amstutz [Fri, 11 Nov 2022 14:53:18 +0000 (09:53 -0500)]
19688: Make sure CommandLineTool gets wrapped properly

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago19215: Fixes IAM policy example.
Lucas Di Pentima [Thu, 10 Nov 2022 21:48:07 +0000 (18:48 -0300)]
19215: Fixes IAM policy example.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '19588-update-passenger'
Tom Clegg [Thu, 10 Nov 2022 19:32:47 +0000 (14:32 -0500)]
Merge branch '19588-update-passenger'

refs #19588

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19712: Remove link to perl sdk docs. 19712-remove-perl
Tom Clegg [Wed, 9 Nov 2022 14:28:21 +0000 (09:28 -0500)]
19712: Remove link to perl sdk docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19712: Remove perl sdk.
Tom Clegg [Tue, 8 Nov 2022 19:45:55 +0000 (14:45 -0500)]
19712: Remove perl sdk.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19588: Update passenger. 19588-update-passenger
Tom Clegg [Tue, 8 Nov 2022 19:40:56 +0000 (14:40 -0500)]
19588: Update passenger.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19702: Update tests. 19702-memory-overhead
Tom Clegg [Tue, 8 Nov 2022 18:19:16 +0000 (13:19 -0500)]
19702: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago19688: Update tests
Peter Amstutz [Tue, 8 Nov 2022 17:24:21 +0000 (12:24 -0500)]
19688: Update tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>