arvados.git
7 months ago20840: Fix broken url 20840-api-deprecate-notice
Peter Amstutz [Tue, 5 Sep 2023 18:06:49 +0000 (14:06 -0400)]
20840: Fix broken url

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago20840: Add deprecations to upgrade notes
Peter Amstutz [Sun, 3 Sep 2023 17:36:49 +0000 (13:36 -0400)]
20840: Add deprecations to upgrade notes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago20840: Add warning boxes to legacy APIs and rearrange API listing
Peter Amstutz [Sun, 3 Sep 2023 17:27:03 +0000 (13:27 -0400)]
20840: Add warning boxes to legacy APIs and rearrange API listing

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20915: Add missing import to _pycurlhelper
Brett Smith [Thu, 31 Aug 2023 13:27:03 +0000 (09:27 -0400)]
20915: Add missing import to _pycurlhelper

Closes #20915.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months agoMerge branch '20877-trashed-priority' refs #20877
Peter Amstutz [Tue, 29 Aug 2023 18:15:08 +0000 (14:15 -0400)]
Merge branch '20877-trashed-priority' refs #20877

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20839-pysdk-deprecations'
Brett Smith [Tue, 29 Aug 2023 15:41:10 +0000 (11:41 -0400)]
Merge branch '20839-pysdk-deprecations'

Closes #20839.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20877: Do not create log or output collections if parent is frozen or trashed 20877-trashed-priority
Peter Amstutz [Tue, 29 Aug 2023 15:04:03 +0000 (11:04 -0400)]
20877: Do not create log or output collections if parent is frozen or trashed

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20757-ctr-shell-settings'
Tom Clegg [Tue, 29 Aug 2023 13:16:00 +0000 (09:16 -0400)]
Merge branch '20757-ctr-shell-settings'

fixes #20757

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20836: Update test_select for Py<3.8 compatibility 20839-pysdk-deprecations
Brett Smith [Mon, 28 Aug 2023 20:26:41 +0000 (16:26 -0400)]
20836: Update test_select for Py<3.8 compatibility

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Mark deprecated PySDK functions as such
Brett Smith [Sun, 27 Aug 2023 02:13:39 +0000 (22:13 -0400)]
20839: Mark deprecated PySDK functions as such

This is in preparation to remove these from the SDK in Arvados 3.0.

These changes exacerbated the circular import between the top-level
arvados and util, which had knock-off effects on other import
statements, causing them to see the world differently (like whether
`arvados.api` is a module or function). Most of the import-related
shuffling is meant to mitigate all that.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Add order_key to select in keyset_list_all
Brett Smith [Sun, 27 Aug 2023 03:59:21 +0000 (23:59 -0400)]
20839: Add order_key to select in keyset_list_all

Like uuid, the main loop requires this field.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Replace arvados.util.mkdir_dash_p calls throughout
Brett Smith [Sun, 27 Aug 2023 02:52:13 +0000 (22:52 -0400)]
20839: Replace arvados.util.mkdir_dash_p calls throughout

In preparation to deprecate the function.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Replace arvados.util.list_all calls throughout
Brett Smith [Sun, 27 Aug 2023 02:01:32 +0000 (22:01 -0400)]
20839: Replace arvados.util.list_all calls throughout

In preparation to deprecate this function. This does leave a couple
calls in arv-docker-migrate19, but that command will also be deprecated
and removed at least as soon as the function.

The changes to the the sdk/cwl/tests/test_submit.py mocks are necessary
because of mechanical differences between list_all and keyset_list_all.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

test fixup

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20757: Split tests into separate funcs. 20757-ctr-shell-settings
Tom Clegg [Mon, 28 Aug 2023 18:40:05 +0000 (14:40 -0400)]
20757: Split tests into separate funcs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20839: Optimize list sort
Brett Smith [Sun, 27 Aug 2023 01:55:39 +0000 (21:55 -0400)]
20839: Optimize list sort

Trivial optimization I noticed while working around here.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Replace arvados.util.run_command calls in arv-copy
Brett Smith [Sun, 27 Aug 2023 01:42:20 +0000 (21:42 -0400)]
20839: Replace arvados.util.run_command calls in arv-copy

In preparation for deprecating this function.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20839: Remove old Python 2 TextIOWrapper shim
Brett Smith [Sat, 26 Aug 2023 23:32:53 +0000 (19:32 -0400)]
20839: Remove old Python 2 TextIOWrapper shim

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20757: `arvados-client shell` uses settings.conf if present.
Tom Clegg [Fri, 25 Aug 2023 16:28:56 +0000 (12:28 -0400)]
20757: `arvados-client shell` uses settings.conf if present.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20877-trashed-priority' refs #20877
Peter Amstutz [Fri, 25 Aug 2023 14:49:33 +0000 (10:49 -0400)]
Merge branch '20877-trashed-priority' refs #20877

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20894-instances-default' refs #20894
Peter Amstutz [Fri, 25 Aug 2023 14:04:25 +0000 (10:04 -0400)]
Merge branch '20894-instances-default' refs #20894

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoRevert "20894: Change default retention on prometheus to 90 days" 20894-instances-default
Peter Amstutz [Thu, 24 Aug 2023 14:10:35 +0000 (10:10 -0400)]
Revert "20894: Change default retention on prometheus to 90 days"

This reverts commit 15e554b5da9d4da9447e547abd536c53be069a21.

According to Lucas, this doesn't actually work.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20894: Change default retention on prometheus to 90 days
Peter Amstutz [Thu, 24 Aug 2023 14:06:41 +0000 (10:06 -0400)]
20894: Change default retention on prometheus to 90 days

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20894: Set LimitLogBytesPerJob to 0
Peter Amstutz [Wed, 23 Aug 2023 18:58:43 +0000 (14:58 -0400)]
20894: Set LimitLogBytesPerJob to 0

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20894: Wordsmithing
Peter Amstutz [Wed, 23 Aug 2023 17:24:10 +0000 (13:24 -0400)]
20894: Wordsmithing

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20894: Change InitialQuotaEstimate and SupervisorFraction defaults.
Peter Amstutz [Wed, 23 Aug 2023 17:08:45 +0000 (13:08 -0400)]
20894: Change InitialQuotaEstimate and SupervisorFraction defaults.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20877: Fix missing 'and'
Peter Amstutz [Wed, 23 Aug 2023 13:31:47 +0000 (09:31 -0400)]
20877: Fix missing 'and'

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20877: Don't retry trashed container requests
Peter Amstutz [Mon, 21 Aug 2023 20:56:22 +0000 (16:56 -0400)]
20877: Don't retry trashed container requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20877: When trashing a project, update priorities on container requests
Peter Amstutz [Mon, 21 Aug 2023 20:42:39 +0000 (16:42 -0400)]
20877: When trashing a project, update priorities on container requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20877: Add test for trashing project
Peter Amstutz [Mon, 21 Aug 2023 18:02:23 +0000 (14:02 -0400)]
20877: Add test for trashing project

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20853-pdoc'
Brett Smith [Mon, 21 Aug 2023 16:15:50 +0000 (12:15 -0400)]
Merge branch '20853-pdoc'

Closes #20853.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months agoAdd pdoc to arvbox, refs #20853
Peter Amstutz [Fri, 18 Aug 2023 20:41:19 +0000 (16:41 -0400)]
Add pdoc to arvbox, refs #20853

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20878-rh-package-download'
Brett Smith [Fri, 18 Aug 2023 13:28:43 +0000 (09:28 -0400)]
Merge branch '20878-rh-package-download'

Closes #20878.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20878: Document package download change to add a target distribution 20878-rh-package-download
Brett Smith [Thu, 17 Aug 2023 23:53:33 +0000 (19:53 -0400)]
20878: Document package download change to add a target distribution

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20878: Improve processed RPM check style
Brett Smith [Thu, 17 Aug 2023 22:28:17 +0000 (18:28 -0400)]
20878: Improve processed RPM check style

This is nicer bash style that makes it more consistent with the
preceding download code.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20878: Improve RPM download logic
Brett Smith [Thu, 17 Aug 2023 22:27:12 +0000 (18:27 -0400)]
20878: Improve RPM download logic

Instead of downloading and parsing a package file listing, just try to
download the package directly, and check whether that succeeds.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20878: Parameterize URL for RPM package builds
Brett Smith [Thu, 17 Aug 2023 22:20:37 +0000 (18:20 -0400)]
20878: Parameterize URL for RPM package builds

* Add support for rocky8.
* Prevent future errors by refusing to attempt a download for unknown
  targets.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months agoMerge branch '20858-jsonb-exists' refs #20858
Peter Amstutz [Wed, 16 Aug 2023 18:54:58 +0000 (14:54 -0400)]
Merge branch '20858-jsonb-exists' refs #20858

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20735-install-test'
Tom Clegg [Wed, 16 Aug 2023 13:32:32 +0000 (09:32 -0400)]
Merge branch '20735-install-test'

refs #20735

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Workaround nginx/glibc/docker issue.
Tom Clegg [Wed, 16 Aug 2023 13:04:18 +0000 (09:04 -0400)]
20735: Workaround nginx/glibc/docker issue.

Docker writes both "::1 localhost" and "127.0.0.1 localhost" in
/etc/hosts even when not enabling IPv6 [1], causing glibc to resolve
"localhost" to two identical IP addresses 127.0.0.1 and 127.0.0.1 [2],
causing nginx to error out on "listen localhost:45123;" [3]:

[emerg] a duplicate listen 127.0.0.1:45123 in /tmp/nginx.conf:29

[1] https://github.com/moby/moby/issues/35954#issuecomment-498406618
("you should be testing with IPv6 enabled")

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=14969 (open since
2012-12-17)

[2] https://trac.nginx.org/nginx/ticket/2400 (workaround added
2022-11-23)

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20680-cores-concurrency' refs #20680
Peter Amstutz [Tue, 15 Aug 2023 20:52:41 +0000 (16:52 -0400)]
Merge branch '20680-cores-concurrency' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20858: Enable 'exists' filter to use index 20858-jsonb-exists
Peter Amstutz [Tue, 15 Aug 2023 20:41:01 +0000 (16:41 -0400)]
20858: Enable 'exists' filter to use index

See comment in services/api/db/migrate/20230815160000_jsonb_exists_functions.rb

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20735: Fix default workbench2-source path.
Tom Clegg [Fri, 11 Aug 2023 13:47:10 +0000 (09:47 -0400)]
20735: Fix default workbench2-source path.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Override git directory ownership check.
Tom Clegg [Thu, 10 Aug 2023 21:18:27 +0000 (17:18 -0400)]
20735: Override git directory ownership check.

Without this, `git diff --shortstat` returns an error "warning: Not a
git repository."

The real problem, as reported by `git status`, is "fatal: detected
dubious ownership in repository at '/path'".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Fix auto version generator.
Tom Clegg [Thu, 10 Aug 2023 20:41:49 +0000 (16:41 -0400)]
20735: Fix auto version generator.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Revive lib/install dependency test.
Tom Clegg [Thu, 10 Aug 2023 19:22:55 +0000 (15:22 -0400)]
20735: Revive lib/install dependency test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Fix firefox package name on debian 11.
Tom Clegg [Thu, 10 Aug 2023 19:21:52 +0000 (15:21 -0400)]
20735: Fix firefox package name on debian 11.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Fixup singularity dependencies (test/dev-only).
Tom Clegg [Thu, 10 Aug 2023 19:20:15 +0000 (15:20 -0400)]
20735: Fixup singularity dependencies (test/dev-only).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '17244-cgroup2-cpu-max'
Tom Clegg [Tue, 15 Aug 2023 17:41:58 +0000 (13:41 -0400)]
Merge branch '17244-cgroup2-cpu-max'

fixes #17244

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago17244: Add upgrade note re number formatting change.
Tom Clegg [Tue, 15 Aug 2023 17:40:18 +0000 (13:40 -0400)]
17244: Add upgrade note re number formatting change.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoFurther fixes on single host installer deployment. No issue #
Lucas Di Pentima [Tue, 15 Aug 2023 17:39:44 +0000 (14:39 -0300)]
Further fixes on single host installer deployment. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months agoFixes deployment when using envvar BRANCH=main. No issue #
Lucas Di Pentima [Tue, 15 Aug 2023 13:53:05 +0000 (10:53 -0300)]
Fixes deployment when using envvar BRANCH=main. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago20853: Modernize pdoc installation process 20853-pdoc
Brett Smith [Mon, 14 Aug 2023 21:04:10 +0000 (17:04 -0400)]
20853: Modernize pdoc installation process

Install to a virtualenv to accommodate PEP 668.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20853: Switch from pdoc3 to pdoc
Brett Smith [Mon, 14 Aug 2023 21:00:57 +0000 (17:00 -0400)]
20853: Switch from pdoc3 to pdoc

pdoc is better maintained, both in terms of release cadence and in terms
of community code of conduct.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago17244: New debian10 testdata, add cpu.max and cpuinfo for others. 17244-cgroup2-cpu-max
Tom Clegg [Mon, 14 Aug 2023 19:40:36 +0000 (15:40 -0400)]
17244: New debian10 testdata, add cpu.max and cpuinfo for others.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago17244: Use cpu.max to get available fraction of CPUs.
Tom Clegg [Mon, 14 Aug 2023 19:09:27 +0000 (15:09 -0400)]
17244: Use cpu.max to get available fraction of CPUs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20649-ssh-help'
Tom Clegg [Mon, 14 Aug 2023 20:29:48 +0000 (16:29 -0400)]
Merge branch '20649-ssh-help'

fixes #20649

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20457-careful-near-quota'
Tom Clegg [Mon, 14 Aug 2023 20:29:09 +0000 (16:29 -0400)]
Merge branch '20457-careful-near-quota'

fixes #20457

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20680: Set concurrency to num_cpus * 2 with a minimum of 8 20680-cores-concurrency
Peter Amstutz [Mon, 14 Aug 2023 18:41:39 +0000 (14:41 -0400)]
20680: Set concurrency to num_cpus * 2 with a minimum of 8

In practice it makes sense to have two workers per core because
workers usually spend at least half their time waiting on the
database.

Setting a minimum concurrency of 8 avoids a deadlock situation when
controller calls keep-web, which calls back to controller.

This doesn't completely eliminate the problem, but it at least avoids
having deadlocked requests with a single user in the default
configuration (!).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20649: Mention `arvados-server cloudtest` in boot timeout log msg. 20649-ssh-help
Tom Clegg [Mon, 14 Aug 2023 15:14:24 +0000 (11:14 -0400)]
20649: Mention `arvados-server cloudtest` in boot timeout log msg.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20649: Fix panic on race, worker shutdown vs. container startup.
Tom Clegg [Mon, 14 Aug 2023 15:11:51 +0000 (11:11 -0400)]
20649: Fix panic on race, worker shutdown vs. container startup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20649: Don't wait timeoutSignal before sending first TERM signal.
Tom Clegg [Mon, 14 Aug 2023 15:10:44 +0000 (11:10 -0400)]
20649: Don't wait timeoutSignal before sending first TERM signal.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoFixes comment formatting. No issue #
Lucas Di Pentima [Mon, 14 Aug 2023 14:07:19 +0000 (11:07 -0300)]
Fixes comment formatting. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago20457: Fix flaky test.
Tom Clegg [Wed, 9 Aug 2023 21:07:35 +0000 (17:07 -0400)]
20457: Fix flaky test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20649: Log actual error from last failed boot probe.
Tom Clegg [Fri, 11 Aug 2023 21:47:32 +0000 (17:47 -0400)]
20649: Log actual error from last failed boot probe.

Not just the stderr content, which is useless if the error was
something like "connection failed".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20649: Log boot probe errors other than usual wait-for-boot errors.
Tom Clegg [Fri, 11 Aug 2023 21:47:06 +0000 (17:47 -0400)]
20649: Log boot probe errors other than usual wait-for-boot errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: Update test to match new unlock-at-quota behavior. 20457-careful-near-quota
Tom Clegg [Fri, 11 Aug 2023 17:31:29 +0000 (13:31 -0400)]
20457: Update test to match new unlock-at-quota behavior.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: Don't lock-cycle next-in-line ctr while waiting for quota.
Tom Clegg [Fri, 11 Aug 2023 14:17:30 +0000 (10:17 -0400)]
20457: Don't lock-cycle next-in-line ctr while waiting for quota.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: Fix total instances count.
Tom Clegg [Fri, 11 Aug 2023 14:12:25 +0000 (10:12 -0400)]
20457: Fix total instances count.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoAdd packages required to build newer singularity, refs #20735
Peter Amstutz [Thu, 10 Aug 2023 18:24:53 +0000 (14:24 -0400)]
Add packages required to build newer singularity, refs #20735

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20688-wb1-to-wb2-redirects' refs #20688
Peter Amstutz [Thu, 10 Aug 2023 17:11:30 +0000 (13:11 -0400)]
Merge branch '20688-wb1-to-wb2-redirects' refs #20688

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20649: cloudtest obeys DeployPublicKey config, like a-d-c.
Tom Clegg [Thu, 10 Aug 2023 14:45:46 +0000 (10:45 -0400)]
20649: cloudtest obeys DeployPublicKey config, like a-d-c.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20755-ec2-multiple-subnets'
Tom Clegg [Thu, 10 Aug 2023 14:34:11 +0000 (10:34 -0400)]
Merge branch '20755-ec2-multiple-subnets'

closes #20755

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: Fix flaky test.
Tom Clegg [Wed, 9 Aug 2023 21:07:35 +0000 (17:07 -0400)]
20457: Fix flaky test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: Exercise quota handling in dispatcher chaos test.
Tom Clegg [Wed, 9 Aug 2023 21:06:40 +0000 (17:06 -0400)]
20457: Exercise quota handling in dispatcher chaos test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20457: When passing prior quota level, raise maxConcurrency slowly.
Tom Clegg [Mon, 7 Aug 2023 19:26:20 +0000 (15:26 -0400)]
20457: When passing prior quota level, raise maxConcurrency slowly.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20680-rolling-deploy' refs #20680
Peter Amstutz [Wed, 9 Aug 2023 13:53:48 +0000 (09:53 -0400)]
Merge branch '20680-rolling-deploy' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20680: Increase MaxConcurrentRequests to 8 20680-rolling-deploy
Peter Amstutz [Wed, 9 Aug 2023 13:48:51 +0000 (09:48 -0400)]
20680: Increase MaxConcurrentRequests to 8

Necessary to avoid a API -> keep-web -> API loop causing a deadlock.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20755: Merge branch 'main' into 20755-ec2-multiple-subnets 20755-ec2-multiple-subnets
Tom Clegg [Wed, 9 Aug 2023 13:33:34 +0000 (09:33 -0400)]
20755: Merge branch 'main' into 20755-ec2-multiple-subnets

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20755: Fix test case by accepting empty DriverParameters.
Tom Clegg [Wed, 9 Aug 2023 13:32:37 +0000 (09:32 -0400)]
20755: Fix test case by accepting empty DriverParameters.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20755: Run cloudtest suite once per configured subnet, not just 1st.
Tom Clegg [Tue, 8 Aug 2023 21:31:24 +0000 (17:31 -0400)]
20755: Run cloudtest suite once per configured subnet, not just 1st.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20735-update-deps'
Tom Clegg [Tue, 8 Aug 2023 18:52:42 +0000 (14:52 -0400)]
Merge branch '20735-update-deps'

refs #20735

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Clarify singularity version/compatibility expectations.
Tom Clegg [Tue, 8 Aug 2023 18:51:21 +0000 (14:51 -0400)]
20735: Clarify singularity version/compatibility expectations.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20680: Only do a rolling update when there is multiple controllers
Peter Amstutz [Tue, 8 Aug 2023 17:42:25 +0000 (13:42 -0400)]
20680: Only do a rolling update when there is multiple controllers

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoFix error check.
Tom Clegg [Tue, 8 Aug 2023 14:18:29 +0000 (10:18 -0400)]
Fix error check.

refs #17244

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20680: installer does rolling update of controller
Peter Amstutz [Tue, 8 Aug 2023 13:36:15 +0000 (09:36 -0400)]
20680: installer does rolling update of controller

When there is a load balancer, disable each controller node in turn
before updating it.

Also, don't reduce controller_nr when a controller is down, having
worker_connections go up and down during deploy seems like in could
cause problems.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20754-docker-py-upgrade'
Brett Smith [Mon, 7 Aug 2023 21:18:56 +0000 (17:18 -0400)]
Merge branch '20754-docker-py-upgrade'

Closes #20754.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20735: Merge branch 'main' into 20735-update-deps 20735-update-deps
Tom Clegg [Mon, 7 Aug 2023 20:58:45 +0000 (16:58 -0400)]
20735: Merge branch 'main' into 20735-update-deps

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Update singularity version in docs and build script.
Tom Clegg [Mon, 7 Aug 2023 20:57:23 +0000 (16:57 -0400)]
20735: Update singularity version in docs and build script.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20705-crunchstat-warn-missing-data'
Tom Clegg [Mon, 7 Aug 2023 20:49:17 +0000 (16:49 -0400)]
Merge branch '20705-crunchstat-warn-missing-data'

fixes #20705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '20680-default-config-updates' refs #20680
Peter Amstutz [Mon, 7 Aug 2023 20:29:15 +0000 (16:29 -0400)]
Merge branch '20680-default-config-updates' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20680: Fix nginx_balancer_configuration & add note about defaults 20680-default-config-updates
Peter Amstutz [Mon, 7 Aug 2023 20:28:13 +0000 (16:28 -0400)]
20680: Fix nginx_balancer_configuration & add note about defaults

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago20705: 10% failure on stub Create to ensure running error path. 20705-crunchstat-warn-missing-data
Tom Clegg [Mon, 7 Aug 2023 14:31:56 +0000 (10:31 -0400)]
20705: 10% failure on stub Create to ensure running error path.

Fixes occasional test failures due to absence of rate-limiting errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago20735: Sync bundler version.
Tom Clegg [Mon, 7 Aug 2023 14:04:38 +0000 (10:04 -0400)]
20735: Sync bundler version.

See https://dev.arvados.org/issues/20735#note-6

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoFixes instance profile assignment. Refs #20610
Lucas Di Pentima [Fri, 4 Aug 2023 20:55:13 +0000 (17:55 -0300)]
Fixes instance profile assignment. Refs #20610

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months agoMerge branch '20610-installer-load-balancer'. Refs #20610
Lucas Di Pentima [Fri, 4 Aug 2023 19:08:32 +0000 (16:08 -0300)]
Merge branch '20610-installer-load-balancer'. Refs #20610

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago20754: Upgrade docker module for docker-cleaner 20754-docker-py-upgrade
Brett Smith [Fri, 4 Aug 2023 18:28:18 +0000 (14:28 -0400)]
20754: Upgrade docker module for docker-cleaner

The immediate motivation is dealing with the bug discussed in the
comments. Upgrading generally seems like a good idea since we've been
pinned at a specific version for so long.

Changing the Docker API version is required by the library: 1.21 is the
oldest version currently supported. Our Go stack is standardized on
1.35, so use that here too.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20610: Documentation changes, including simplification & fixes. 20610-installer-load-balancer
Lucas Di Pentima [Fri, 4 Aug 2023 18:08:04 +0000 (15:08 -0300)]
20610: Documentation changes, including simplification & fixes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago20754: Remove test requirements from docker-cleaner
Brett Smith [Fri, 4 Aug 2023 18:05:26 +0000 (14:05 -0400)]
20754: Remove test requirements from docker-cleaner

Neither are required with our current Python stack.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago20610: Changes the default installer config in order to simplify documentation.
Lucas Di Pentima [Fri, 4 Aug 2023 17:48:29 +0000 (14:48 -0300)]
20610: Changes the default installer config in order to simplify documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago20688: Update single host/multiple hostname template 20688-wb1-to-wb2-redirects
Peter Amstutz [Thu, 3 Aug 2023 14:31:05 +0000 (10:31 -0400)]
20688: Update single host/multiple hostname template

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>