arvados.git
20 months ago20422: Do GET when cache slot is empty instead of raising an error 20422-cache-slot
Peter Amstutz [Mon, 24 Apr 2023 20:59:27 +0000 (16:59 -0400)]
20422: Do GET when cache slot is empty instead of raising an error

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoAdjust default value of BalanceCollectionBuffers down to 4
Peter Amstutz [Fri, 21 Apr 2023 19:19:29 +0000 (15:19 -0400)]
Adjust default value of BalanceCollectionBuffers down to 4

Buffering a large number of collections in RAM is much less helpful
since keep-balance started accessing the database directly instead of
using the API.

refs #20421

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoAdjust trash_at of log collection to now+2 weeks refs #20378
Peter Amstutz [Fri, 21 Apr 2023 19:06:24 +0000 (15:06 -0400)]
Adjust trash_at of log collection to now+2 weeks refs #20378

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoMerge branch '20388-installer-prom-targets'. Closes #20388
Lucas Di Pentima [Fri, 21 Apr 2023 18:08:36 +0000 (15:08 -0300)]
Merge branch '20388-installer-prom-targets'. Closes #20388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20257-http-import' refs #20257
Peter Amstutz [Fri, 21 Apr 2023 16:23:27 +0000 (12:23 -0400)]
Merge branch '20257-http-import' refs #20257

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Fix use of dataclass 20257-http-import
Peter Amstutz [Fri, 21 Apr 2023 16:02:04 +0000 (12:02 -0400)]
20257: Fix use of dataclass

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Mark functions and classes internal not intended for use
Peter Amstutz [Fri, 21 Apr 2023 14:57:30 +0000 (10:57 -0400)]
20257: Mark functions and classes internal not intended for use

Adjust return value of http_to_keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20388: Adds missing arvados services as prometheus targets. 20388-installer-prom-targets
Lucas Di Pentima [Fri, 21 Apr 2023 14:19:26 +0000 (11:19 -0300)]
20388: Adds missing arvados services as prometheus targets.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20241-validate-ssh-keys'
Tom Clegg [Thu, 20 Apr 2023 14:49:16 +0000 (10:49 -0400)]
Merge branch '20241-validate-ssh-keys'

fixes #20241

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20257: Fix test
Peter Amstutz [Wed, 19 Apr 2023 21:59:18 +0000 (17:59 -0400)]
20257: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Don't log arvados.keep at INFO
Peter Amstutz [Wed, 19 Apr 2023 20:59:16 +0000 (16:59 -0400)]
20257: Don't log arvados.keep at INFO

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Set INFO log level explicitly
Peter Amstutz [Wed, 19 Apr 2023 20:43:17 +0000 (16:43 -0400)]
20257: Set INFO log level explicitly

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Add message before doing any HTTP activity
Peter Amstutz [Wed, 19 Apr 2023 20:08:39 +0000 (16:08 -0400)]
20257: Add message before doing any HTTP activity

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Tweak runtime status for 'activity'
Peter Amstutz [Wed, 19 Apr 2023 19:44:11 +0000 (15:44 -0400)]
20257: Tweak runtime status for 'activity'

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20241: Add ed25519-sk test key 20241-validate-ssh-keys
Brett Smith [Wed, 19 Apr 2023 19:26:35 +0000 (15:26 -0400)]
20241: Add ed25519-sk test key

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago20257: Record basic workflow status in "activity"
Peter Amstutz [Wed, 19 Apr 2023 19:18:44 +0000 (15:18 -0400)]
20257: Record basic workflow status in "activity"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Need to parse status code out of the header, because pyCurl
Peter Amstutz [Wed, 19 Apr 2023 19:02:55 +0000 (15:02 -0400)]
20257: Need to parse status code out of the header, because pyCurl

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20257: Refactor http-to-keep downloader
Peter Amstutz [Wed, 19 Apr 2023 14:07:05 +0000 (10:07 -0400)]
20257: Refactor http-to-keep downloader

* Moved into the SDK, so that other programs besides cwl-runner can use
it
* Migrated to pyCurl
* Updated tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago18799: Force UTF-8 encoding for discovery2pydoc output
Brett Smith [Wed, 19 Apr 2023 13:58:49 +0000 (09:58 -0400)]
18799: Force UTF-8 encoding for discovery2pydoc output

Refs #18799.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months agoMerge branch '18799-api-pydoc'
Brett Smith [Wed, 19 Apr 2023 12:55:56 +0000 (08:55 -0400)]
Merge branch '18799-api-pydoc'

Closes #18799.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago20241: Move authorized_keys to new code path, validate public key.
Tom Clegg [Tue, 18 Apr 2023 20:34:47 +0000 (16:34 -0400)]
20241: Move authorized_keys to new code path, validate public key.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18799: Make schema docstring more specific about how to select object keys 18799-api-pydoc
Brett Smith [Tue, 18 Apr 2023 15:13:08 +0000 (11:13 -0400)]
18799: Make schema docstring more specific about how to select object keys

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Link back to resource class from schema docstring
Brett Smith [Tue, 18 Apr 2023 15:07:54 +0000 (11:07 -0400)]
18799: Link back to resource class from schema docstring

This makes the connection bidirectional, which helps both understanding
and navigation.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months agoMerge branch '19620-remove-old-s3-driver'
Tom Clegg [Tue, 18 Apr 2023 15:13:41 +0000 (11:13 -0400)]
Merge branch '19620-remove-old-s3-driver'

closes #19620

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoAdd 2.6.1 to upgrade notes. refs #20345
Peter Amstutz [Mon, 17 Apr 2023 19:30:54 +0000 (15:30 -0400)]
Add 2.6.1 to upgrade notes. refs #20345

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoMerge branch '20123-access-token-doc' refs #20123
Peter Amstutz [Mon, 17 Apr 2023 18:43:01 +0000 (14:43 -0400)]
Merge branch '20123-access-token-doc' refs #20123

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20123: Fix tabs vs spaces 20123-access-token-doc
Peter Amstutz [Mon, 17 Apr 2023 18:41:57 +0000 (14:41 -0400)]
20123: Fix tabs vs spaces

refs #20123

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20123: Add note about scopes
Peter Amstutz [Mon, 17 Apr 2023 18:37:10 +0000 (14:37 -0400)]
20123: Add note about scopes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago20123: Make OpenID connect token a proper subsection
Peter Amstutz [Mon, 17 Apr 2023 18:25:05 +0000 (14:25 -0400)]
20123: Make OpenID connect token a proper subsection

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoMerge branch '20203-rails-cache'
Tom Clegg [Mon, 17 Apr 2023 15:06:54 +0000 (11:06 -0400)]
Merge branch '20203-rails-cache'

fixes #20203

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago19620: Remove goamz dependency in non-testing code. 19620-remove-old-s3-driver
Tom Clegg [Mon, 17 Apr 2023 14:50:05 +0000 (10:50 -0400)]
19620: Remove goamz dependency in non-testing code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18799: Make the PySDK's build subcommand extend build_py
Brett Smith [Fri, 14 Apr 2023 18:03:45 +0000 (14:03 -0400)]
18799: Make the PySDK's build subcommand extend build_py

See the added comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months agoMerge branch '20325-go-docker-distribution-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 18:00:56 +0000 (15:00 -0300)]
Merge branch '20325-go-docker-distribution-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago18799: Add Python 3.7 compatibility for TypedDict
Brett Smith [Fri, 14 Apr 2023 17:55:54 +0000 (13:55 -0400)]
18799: Add Python 3.7 compatibility for TypedDict

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Clean up redundant setuptools requirement listing
Brett Smith [Fri, 14 Apr 2023 17:54:52 +0000 (13:54 -0400)]
18799: Clean up redundant setuptools requirement listing

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Add missing f-string prefix
Brett Smith [Fri, 14 Apr 2023 17:53:32 +0000 (13:53 -0400)]
18799: Add missing f-string prefix

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago20325: Upgrades github.com/docker/distribution module dependency. 20325-go-docker-distribution-upgrade
Lucas Di Pentima [Fri, 14 Apr 2023 17:25:07 +0000 (14:25 -0300)]
20325: Upgrades github.com/docker/distribution module dependency.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20325-globalid-gem-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 16:15:17 +0000 (13:15 -0300)]
Merge branch '20325-globalid-gem-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20334-slow-dedup-migration'
Tom Clegg [Fri, 14 Apr 2023 16:02:25 +0000 (12:02 -0400)]
Merge branch '20334-slow-dedup-migration'

fixes #20334

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '20336-filter-bigint'
Tom Clegg [Fri, 14 Apr 2023 16:00:58 +0000 (12:00 -0400)]
Merge branch '20336-filter-bigint'

fixes #20336

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20203: Fix global var reset on db reset; skip in non-test mode. 20203-rails-cache
Tom Clegg [Fri, 14 Apr 2023 15:59:13 +0000 (11:59 -0400)]
20203: Fix global var reset on db reset; skip in non-test mode.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20325: Upgrades the globalid gem on both RailsAPI & WB1. 20325-globalid-gem-upgrade
Lucas Di Pentima [Fri, 14 Apr 2023 15:46:45 +0000 (12:46 -0300)]
20325: Upgrades the globalid gem on both RailsAPI & WB1.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20325-go-containerd-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 15:18:07 +0000 (12:18 -0300)]
Merge branch '20325-go-containerd-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago20325: Upgrades github.com/containerd/containerd dependency. 20325-go-containerd-upgrade
Lucas Di Pentima [Fri, 14 Apr 2023 14:28:47 +0000 (11:28 -0300)]
20325: Upgrades github.com/containerd/containerd dependency.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago18799: Mark self parameters as POSITIONAL_OR_KEYWORD
Brett Smith [Fri, 14 Apr 2023 14:06:23 +0000 (10:06 -0400)]
18799: Mark self parameters as POSITIONAL_OR_KEYWORD

On Python 3.7, if you mark a parameter POSITIONAL_ONLY, it will generate
a signature that uses the positional-only `/` syntax, *even though*
Python 3.7 cannot parse that syntax (it was added in Python 3.8, PEP 570).

POSITIONAL_OR_KEYWORD avoids that problem, and is probably more accurate
anyway.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago19620: Remove old "v1" S3 keepstore driver.
Tom Clegg [Thu, 13 Apr 2023 21:23:33 +0000 (17:23 -0400)]
19620: Remove old "v1" S3 keepstore driver.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18799: Move canonical discovery document to sdk/python
Brett Smith [Thu, 13 Apr 2023 20:36:10 +0000 (16:36 -0400)]
18799: Move canonical discovery document to sdk/python

The Python SDK build is currently the only code that needs it, and
moving the file here simplifies the build implementation. We can
reevaluate where it lives if another need arises.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago20336: Allow filtering on set of integers. 20336-filter-bigint
Tom Clegg [Thu, 13 Apr 2023 19:42:22 +0000 (15:42 -0400)]
20336: Allow filtering on set of integers.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20336: Reject >64 bit operands when filtering on integer columns.
Tom Clegg [Thu, 13 Apr 2023 17:27:34 +0000 (13:27 -0400)]
20336: Reject >64 bit operands when filtering on integer columns.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '20325-nokogiri-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 17:54:26 +0000 (14:54 -0300)]
Merge branch '20325-nokogiri-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago23025: Upgrades nokogiri gem on RailsAPI & Workbench1. 20325-nokogiri-upgrade
Lucas Di Pentima [Thu, 13 Apr 2023 17:32:01 +0000 (14:32 -0300)]
23025: Upgrades nokogiri gem on RailsAPI & Workbench1.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20325-go-x-net-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 17:25:27 +0000 (14:25 -0300)]
Merge branch '20325-go-x-net-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago20325: Upgrades golang.org/x/net. 20325-go-x-net-upgrade
Lucas Di Pentima [Thu, 13 Apr 2023 16:54:56 +0000 (13:54 -0300)]
20325: Upgrades golang.org/x/net.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20325-rack-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 16:41:33 +0000 (13:41 -0300)]
Merge branch '20325-rack-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago20325: Upgrades rack gem on RailsAPI & Workbench1 20325-rack-upgrade
Lucas Di Pentima [Thu, 13 Apr 2023 16:16:22 +0000 (13:16 -0300)]
20325: Upgrades rack gem on RailsAPI & Workbench1

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months ago20334: Batch permission updates during link dedup migration. 20334-slow-dedup-migration
Tom Clegg [Thu, 13 Apr 2023 15:47:46 +0000 (11:47 -0400)]
20334: Batch permission updates during link dedup migration.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18799: Make discovery2pydoc status check Py3.7-compatible
Brett Smith [Thu, 13 Apr 2023 15:42:09 +0000 (11:42 -0400)]
18799: Make discovery2pydoc status check Py3.7-compatible

The `status` attribute was added in Python 3.9.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Add discovery2pydoc to Python SDK manifest
Brett Smith [Thu, 13 Apr 2023 15:24:45 +0000 (11:24 -0400)]
18799: Add discovery2pydoc to Python SDK manifest

Same rationale as arvados_version: setup.py pulls it in.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Add background to API docstrings
Brett Smith [Thu, 13 Apr 2023 14:51:46 +0000 (10:51 -0400)]
18799: Add background to API docstrings

These explanations are intended to help orient users reading this
documentation module (whether via Pydoc or the web) to understand how it
relates to the API client object and the dictionaries it returns.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago20123: Add detail about OIDC token validation/cache implementation.
Tom Clegg [Thu, 13 Apr 2023 14:50:13 +0000 (10:50 -0400)]
20123: Add detail about OIDC token validation/cache implementation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20123: Add hints about OpenID Connect configuration.
Tom Clegg [Wed, 5 Apr 2023 13:50:36 +0000 (09:50 -0400)]
20123: Add hints about OpenID Connect configuration.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18799: Build Python SDK before generating web documentation
Brett Smith [Thu, 13 Apr 2023 14:07:05 +0000 (10:07 -0400)]
18799: Build Python SDK before generating web documentation

This gives preprocessing steps like discovery2pydoc a chance to run.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Run discovery2pydoc as part of SDK build
Brett Smith [Thu, 13 Apr 2023 13:56:27 +0000 (09:56 -0400)]
18799: Run discovery2pydoc as part of SDK build

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Support relative path to discovery document
Brett Smith [Mon, 10 Apr 2023 15:54:46 +0000 (11:54 -0400)]
18799: Support relative path to discovery document

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Commit canonical discovery document with check
Brett Smith [Mon, 10 Apr 2023 15:51:06 +0000 (11:51 -0400)]
18799: Commit canonical discovery document with check

This copy of the discovery document can be used by our build process,
e.g., to generate documentation. The integration test helps ensure it
does not fall out of date.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Remove old API documentation scripts
Brett Smith [Mon, 10 Apr 2023 14:53:16 +0000 (10:53 -0400)]
18799: Remove old API documentation scripts

These are no longer used.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Add deprecation notices to API pydoc
Brett Smith [Fri, 31 Mar 2023 21:38:33 +0000 (17:38 -0400)]
18799: Add deprecation notices to API pydoc

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Include schemas in API pydoc
Brett Smith [Fri, 31 Mar 2023 20:36:47 +0000 (16:36 -0400)]
18799: Include schemas in API pydoc

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months ago18799: Initial prototype of API pydoc generator
Brett Smith [Fri, 31 Mar 2023 18:32:34 +0000 (14:32 -0400)]
18799: Initial prototype of API pydoc generator

See the script docstring for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months agoUpdate R package version to 2.6.0
Peter Amstutz [Thu, 6 Apr 2023 18:46:00 +0000 (14:46 -0400)]
Update R package version to 2.6.0

Fortunately the actual release build script replaces the version
number automatically so this is just bringing it in alignment with the
released version.

refs #20145

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months agoAdd 2.6.0 release to upgrade notes 2.6.0
Peter Amstutz [Thu, 6 Apr 2023 16:08:37 +0000 (12:08 -0400)]
Add 2.6.0 release to upgrade notes

refs #20145

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago19889: Add an id to the container log method documentation
Brett Smith [Wed, 5 Apr 2023 16:19:23 +0000 (12:19 -0400)]
19889: Add an id to the container log method documentation

To support the release notes.
Refs #19889

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

20 months agoMerge branch '19889-live-log-webdav'
Tom Clegg [Tue, 4 Apr 2023 18:29:13 +0000 (14:29 -0400)]
Merge branch '19889-live-log-webdav'

closes #19889

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '20240-container-update-deadlock'
Tom Clegg [Tue, 4 Apr 2023 18:27:22 +0000 (14:27 -0400)]
Merge branch '20240-container-update-deadlock'

fixes #20240

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Change locking mode to EXCLUSIVE. 20240-container-update-deadlock
Tom Clegg [Tue, 4 Apr 2023 15:18:08 +0000 (11:18 -0400)]
20240: Change locking mode to EXCLUSIVE.

...in order to conflict with SELECT FOR UPDATE.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Don't accept/retry after deadlock in test case.
Tom Clegg [Fri, 31 Mar 2023 15:16:10 +0000 (11:16 -0400)]
20240: Don't accept/retry after deadlock in test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Lock containers table before cascading priority updates.
Tom Clegg [Fri, 31 Mar 2023 15:14:19 +0000 (11:14 -0400)]
20240: Lock containers table before cascading priority updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Expose deadlock in multi-level workflow test case.
Tom Clegg [Thu, 30 Mar 2023 21:19:13 +0000 (17:19 -0400)]
20240: Expose deadlock in multi-level workflow test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Fix test.
Tom Clegg [Thu, 30 Mar 2023 18:11:32 +0000 (14:11 -0400)]
20240: Fix test.

Automatic cancellation of child containers doesn't work as expected if
the parent container is in Queued state, which never happens in real
life, but did happen in the previous version of this test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago20240: Update test to reveal bug.
Tom Clegg [Thu, 23 Mar 2023 21:40:30 +0000 (17:40 -0400)]
20240: Update test to reveal bug.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago19889: Fix crash on gateway connect error. 19889-live-log-webdav
Tom Clegg [Tue, 4 Apr 2023 15:00:22 +0000 (11:00 -0400)]
19889: Fix crash on gateway connect error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '16379-installer-prometheus-grafana'. Closes #16379
Lucas Di Pentima [Tue, 4 Apr 2023 13:38:50 +0000 (10:38 -0300)]
Merge branch '16379-installer-prometheus-grafana'. Closes #16379

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

20 months agoMerge branch '20295-fix-collection-tree-caching-bug' refs #20295
Peter Amstutz [Tue, 4 Apr 2023 13:26:09 +0000 (09:26 -0400)]
Merge branch '20295-fix-collection-tree-caching-bug' refs #20295

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago18790: Fix logging noise from ENOENT.
Tom Clegg [Mon, 3 Apr 2023 15:23:32 +0000 (11:23 -0400)]
18790: Fix logging noise from ENOENT.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months ago18790: Fix .../containers/.../log routing.
Tom Clegg [Mon, 3 Apr 2023 15:23:03 +0000 (11:23 -0400)]
18790: Fix .../containers/.../log routing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '20264-trusted-clients-portnumber'
Tom Clegg [Fri, 31 Mar 2023 14:12:02 +0000 (10:12 -0400)]
Merge branch '20264-trusted-clients-portnumber'

fixes #20264
fixes #20140

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoMerge branch '20235-probe-after-upgrade'
Tom Clegg [Fri, 31 Mar 2023 14:10:26 +0000 (10:10 -0400)]
Merge branch '20235-probe-after-upgrade'

fixes #20235

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

20 months agoAdd unit test 20295-fix-collection-tree-caching-bug
Konrad Rudolph [Thu, 30 Mar 2023 17:25:25 +0000 (19:25 +0200)]
Add unit test

Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>

20 months agoFix collection tree caching bug
Konrad Rudolph [Thu, 30 Mar 2023 16:33:15 +0000 (18:33 +0200)]
Fix collection tree caching bug

Before this fix, accessing two files inside the same collection in
succession via `collection$get` will fail on the second file because the
private tree inside the `collection` is only populated with the relative
path of the first file.

Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>

20 months ago16379: Tweak doc on starring the dashboards on grafana 16379-installer-prometheus-grafana
Peter Amstutz [Thu, 30 Mar 2023 17:18:38 +0000 (13:18 -0400)]
16379: Tweak doc on starring the dashboards on grafana

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago16379: Reorganize local.params examples
Peter Amstutz [Thu, 30 Mar 2023 14:56:44 +0000 (10:56 -0400)]
16379: Reorganize local.params examples

Make sure the things that always need to be changed are at the top.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago16379: Small doc updates about prometheus and grafana
Peter Amstutz [Thu, 30 Mar 2023 14:30:41 +0000 (10:30 -0400)]
16379: Small doc updates about prometheus and grafana

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

20 months ago16379: Updates for grafana install
Peter Amstutz [Thu, 30 Mar 2023 13:27:35 +0000 (09:27 -0400)]
16379: Updates for grafana install

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago16379: Add prometheus and grafana to workbench aliases
Peter Amstutz [Tue, 28 Mar 2023 22:07:43 +0000 (18:07 -0400)]
16379: Add prometheus and grafana to workbench aliases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20264: Reject redirect target with userinfo. 20264-trusted-clients-portnumber
Tom Clegg [Tue, 28 Mar 2023 21:05:26 +0000 (17:05 -0400)]
20264: Reject redirect target with userinfo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago16379: doc updates
Peter Amstutz [Tue, 28 Mar 2023 20:38:25 +0000 (16:38 -0400)]
16379: doc updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago19889: Preserve WebDAV path when proxying to keep-web.
Tom Clegg [Tue, 28 Mar 2023 19:51:59 +0000 (15:51 -0400)]
19889: Preserve WebDAV path when proxying to keep-web.

WebDAV clients expect the path in the server response to match the
request.

Previously when proxying to keep-web we were rewriting the request
from /arvados/v1/containers/{uuid}/log/stderr.txt to
/by_id/{pdh}/stderr.txt, so the response referred to
/by_id/{pdh}/stderr.txt.

With this change, we leave the request path alone and use a new
X-Webdav-Prefix request header (/arvados/v1/containers/{uuid}/log in
this case) to tell keep-web to strip that part when accessing the
virtual filesystem.

New test uses cadaver, which fails on the previous version with

Could not access /arvados/v1/containers/zzzzz-dz642-queuedcontainer/log/ (not WebDAV-enabled?):
Did not find a collection resource.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '20270-salt-installer-less-instances'. Closes #20270
Lucas Di Pentima [Tue, 28 Mar 2023 19:40:01 +0000 (16:40 -0300)]
Merge branch '20270-salt-installer-less-instances'. Closes #20270

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago16379: Fixes missing 'monitoring' role on the workbench node.
Lucas Di Pentima [Tue, 28 Mar 2023 18:22:16 +0000 (15:22 -0300)]
16379: Fixes missing 'monitoring' role on the workbench node.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago16379: Removes prometheus config for keep1 & keepproxy nodes.
Lucas Di Pentima [Tue, 28 Mar 2023 18:08:15 +0000 (15:08 -0300)]
16379: Removes prometheus config for keep1 & keepproxy nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>