arvados.git
15 months ago20862: Add 'sdk/ruby-google-api-client/' from commit '2f4be67955e48bb65d008ecd9ff6da9...
Tom Clegg [Wed, 23 Aug 2023 13:31:48 +0000 (09:31 -0400)]
20862: Add 'sdk/ruby-google-api-client/' from commit '2f4be67955e48bb65d008ecd9ff6da9d8fe77028'

git-subtree-dir: sdk/ruby-google-api-client
git-subtree-mainline: 6e91ab41ffa1509dd1d4e2b8ecb559dd429c6ea4
git-subtree-split: 2f4be67955e48bb65d008ecd9ff6da9d8fe77028

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20862: Update apps to latest ruby SDK.
Tom Clegg [Fri, 18 Aug 2023 20:09:07 +0000 (16:09 -0400)]
20862: Update apps to latest ruby SDK.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20862: Update ruby and cli gem dependencies.
Tom Clegg [Fri, 18 Aug 2023 17:21:27 +0000 (13:21 -0400)]
20862: Update ruby and cli gem dependencies.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20862: Fix use of deprecated stdlib.
Tom Clegg [Fri, 18 Aug 2023 17:16:06 +0000 (13:16 -0400)]
20862: Fix use of deprecated stdlib.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20862: Update dependencies, disable some Faraday usage.
Tom Clegg [Fri, 18 Aug 2023 19:50:36 +0000 (15:50 -0400)]
20862: Update dependencies, disable some Faraday usage.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20862: Update gemspec to reflect forked gem name.
Tom Clegg [Fri, 18 Aug 2023 19:36:14 +0000 (15:36 -0400)]
20862: Update gemspec to reflect forked gem name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20735-install-test'
Tom Clegg [Wed, 16 Aug 2023 13:32:32 +0000 (09:32 -0400)]
Merge branch '20735-install-test'

refs #20735

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Workaround nginx/glibc/docker issue.
Tom Clegg [Wed, 16 Aug 2023 13:04:18 +0000 (09:04 -0400)]
20735: Workaround nginx/glibc/docker issue.

Docker writes both "::1 localhost" and "127.0.0.1 localhost" in
/etc/hosts even when not enabling IPv6 [1], causing glibc to resolve
"localhost" to two identical IP addresses 127.0.0.1 and 127.0.0.1 [2],
causing nginx to error out on "listen localhost:45123;" [3]:

[emerg] a duplicate listen 127.0.0.1:45123 in /tmp/nginx.conf:29

[1] https://github.com/moby/moby/issues/35954#issuecomment-498406618
("you should be testing with IPv6 enabled")

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=14969 (open since
2012-12-17)

[2] https://trac.nginx.org/nginx/ticket/2400 (workaround added
2022-11-23)

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20680-cores-concurrency' refs #20680
Peter Amstutz [Tue, 15 Aug 2023 20:52:41 +0000 (16:52 -0400)]
Merge branch '20680-cores-concurrency' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20735: Fix default workbench2-source path.
Tom Clegg [Fri, 11 Aug 2023 13:47:10 +0000 (09:47 -0400)]
20735: Fix default workbench2-source path.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Override git directory ownership check.
Tom Clegg [Thu, 10 Aug 2023 21:18:27 +0000 (17:18 -0400)]
20735: Override git directory ownership check.

Without this, `git diff --shortstat` returns an error "warning: Not a
git repository."

The real problem, as reported by `git status`, is "fatal: detected
dubious ownership in repository at '/path'".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Fix auto version generator.
Tom Clegg [Thu, 10 Aug 2023 20:41:49 +0000 (16:41 -0400)]
20735: Fix auto version generator.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Revive lib/install dependency test.
Tom Clegg [Thu, 10 Aug 2023 19:22:55 +0000 (15:22 -0400)]
20735: Revive lib/install dependency test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Fix firefox package name on debian 11.
Tom Clegg [Thu, 10 Aug 2023 19:21:52 +0000 (15:21 -0400)]
20735: Fix firefox package name on debian 11.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Fixup singularity dependencies (test/dev-only).
Tom Clegg [Thu, 10 Aug 2023 19:20:15 +0000 (15:20 -0400)]
20735: Fixup singularity dependencies (test/dev-only).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '17244-cgroup2-cpu-max'
Tom Clegg [Tue, 15 Aug 2023 17:41:58 +0000 (13:41 -0400)]
Merge branch '17244-cgroup2-cpu-max'

fixes #17244

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago17244: Add upgrade note re number formatting change.
Tom Clegg [Tue, 15 Aug 2023 17:40:18 +0000 (13:40 -0400)]
17244: Add upgrade note re number formatting change.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoFurther fixes on single host installer deployment. No issue #
Lucas Di Pentima [Tue, 15 Aug 2023 17:39:44 +0000 (14:39 -0300)]
Further fixes on single host installer deployment. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months agoFixes deployment when using envvar BRANCH=main. No issue #
Lucas Di Pentima [Tue, 15 Aug 2023 13:53:05 +0000 (10:53 -0300)]
Fixes deployment when using envvar BRANCH=main. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago17244: New debian10 testdata, add cpu.max and cpuinfo for others. 17244-cgroup2-cpu-max
Tom Clegg [Mon, 14 Aug 2023 19:40:36 +0000 (15:40 -0400)]
17244: New debian10 testdata, add cpu.max and cpuinfo for others.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago17244: Use cpu.max to get available fraction of CPUs.
Tom Clegg [Mon, 14 Aug 2023 19:09:27 +0000 (15:09 -0400)]
17244: Use cpu.max to get available fraction of CPUs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20649-ssh-help'
Tom Clegg [Mon, 14 Aug 2023 20:29:48 +0000 (16:29 -0400)]
Merge branch '20649-ssh-help'

fixes #20649

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20457-careful-near-quota'
Tom Clegg [Mon, 14 Aug 2023 20:29:09 +0000 (16:29 -0400)]
Merge branch '20457-careful-near-quota'

fixes #20457

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20680: Set concurrency to num_cpus * 2 with a minimum of 8 20680-cores-concurrency
Peter Amstutz [Mon, 14 Aug 2023 18:41:39 +0000 (14:41 -0400)]
20680: Set concurrency to num_cpus * 2 with a minimum of 8

In practice it makes sense to have two workers per core because
workers usually spend at least half their time waiting on the
database.

Setting a minimum concurrency of 8 avoids a deadlock situation when
controller calls keep-web, which calls back to controller.

This doesn't completely eliminate the problem, but it at least avoids
having deadlocked requests with a single user in the default
configuration (!).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20649: Mention `arvados-server cloudtest` in boot timeout log msg. 20649-ssh-help
Tom Clegg [Mon, 14 Aug 2023 15:14:24 +0000 (11:14 -0400)]
20649: Mention `arvados-server cloudtest` in boot timeout log msg.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20649: Fix panic on race, worker shutdown vs. container startup.
Tom Clegg [Mon, 14 Aug 2023 15:11:51 +0000 (11:11 -0400)]
20649: Fix panic on race, worker shutdown vs. container startup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20649: Don't wait timeoutSignal before sending first TERM signal.
Tom Clegg [Mon, 14 Aug 2023 15:10:44 +0000 (11:10 -0400)]
20649: Don't wait timeoutSignal before sending first TERM signal.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoFixes comment formatting. No issue #
Lucas Di Pentima [Mon, 14 Aug 2023 14:07:19 +0000 (11:07 -0300)]
Fixes comment formatting. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20457: Fix flaky test.
Tom Clegg [Wed, 9 Aug 2023 21:07:35 +0000 (17:07 -0400)]
20457: Fix flaky test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20649: Log actual error from last failed boot probe.
Tom Clegg [Fri, 11 Aug 2023 21:47:32 +0000 (17:47 -0400)]
20649: Log actual error from last failed boot probe.

Not just the stderr content, which is useless if the error was
something like "connection failed".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20649: Log boot probe errors other than usual wait-for-boot errors.
Tom Clegg [Fri, 11 Aug 2023 21:47:06 +0000 (17:47 -0400)]
20649: Log boot probe errors other than usual wait-for-boot errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: Update test to match new unlock-at-quota behavior. 20457-careful-near-quota
Tom Clegg [Fri, 11 Aug 2023 17:31:29 +0000 (13:31 -0400)]
20457: Update test to match new unlock-at-quota behavior.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: Don't lock-cycle next-in-line ctr while waiting for quota.
Tom Clegg [Fri, 11 Aug 2023 14:17:30 +0000 (10:17 -0400)]
20457: Don't lock-cycle next-in-line ctr while waiting for quota.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: Fix total instances count.
Tom Clegg [Fri, 11 Aug 2023 14:12:25 +0000 (10:12 -0400)]
20457: Fix total instances count.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoAdd packages required to build newer singularity, refs #20735
Peter Amstutz [Thu, 10 Aug 2023 18:24:53 +0000 (14:24 -0400)]
Add packages required to build newer singularity, refs #20735

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months agoMerge branch '20688-wb1-to-wb2-redirects' refs #20688
Peter Amstutz [Thu, 10 Aug 2023 17:11:30 +0000 (13:11 -0400)]
Merge branch '20688-wb1-to-wb2-redirects' refs #20688

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20649: cloudtest obeys DeployPublicKey config, like a-d-c.
Tom Clegg [Thu, 10 Aug 2023 14:45:46 +0000 (10:45 -0400)]
20649: cloudtest obeys DeployPublicKey config, like a-d-c.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20755-ec2-multiple-subnets'
Tom Clegg [Thu, 10 Aug 2023 14:34:11 +0000 (10:34 -0400)]
Merge branch '20755-ec2-multiple-subnets'

closes #20755

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: Fix flaky test.
Tom Clegg [Wed, 9 Aug 2023 21:07:35 +0000 (17:07 -0400)]
20457: Fix flaky test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: Exercise quota handling in dispatcher chaos test.
Tom Clegg [Wed, 9 Aug 2023 21:06:40 +0000 (17:06 -0400)]
20457: Exercise quota handling in dispatcher chaos test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20457: When passing prior quota level, raise maxConcurrency slowly.
Tom Clegg [Mon, 7 Aug 2023 19:26:20 +0000 (15:26 -0400)]
20457: When passing prior quota level, raise maxConcurrency slowly.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20680-rolling-deploy' refs #20680
Peter Amstutz [Wed, 9 Aug 2023 13:53:48 +0000 (09:53 -0400)]
Merge branch '20680-rolling-deploy' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20680: Increase MaxConcurrentRequests to 8 20680-rolling-deploy
Peter Amstutz [Wed, 9 Aug 2023 13:48:51 +0000 (09:48 -0400)]
20680: Increase MaxConcurrentRequests to 8

Necessary to avoid a API -> keep-web -> API loop causing a deadlock.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20755: Merge branch 'main' into 20755-ec2-multiple-subnets 20755-ec2-multiple-subnets
Tom Clegg [Wed, 9 Aug 2023 13:33:34 +0000 (09:33 -0400)]
20755: Merge branch 'main' into 20755-ec2-multiple-subnets

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20755: Fix test case by accepting empty DriverParameters.
Tom Clegg [Wed, 9 Aug 2023 13:32:37 +0000 (09:32 -0400)]
20755: Fix test case by accepting empty DriverParameters.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20755: Run cloudtest suite once per configured subnet, not just 1st.
Tom Clegg [Tue, 8 Aug 2023 21:31:24 +0000 (17:31 -0400)]
20755: Run cloudtest suite once per configured subnet, not just 1st.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20735-update-deps'
Tom Clegg [Tue, 8 Aug 2023 18:52:42 +0000 (14:52 -0400)]
Merge branch '20735-update-deps'

refs #20735

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Clarify singularity version/compatibility expectations.
Tom Clegg [Tue, 8 Aug 2023 18:51:21 +0000 (14:51 -0400)]
20735: Clarify singularity version/compatibility expectations.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20680: Only do a rolling update when there is multiple controllers
Peter Amstutz [Tue, 8 Aug 2023 17:42:25 +0000 (13:42 -0400)]
20680: Only do a rolling update when there is multiple controllers

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months agoFix error check.
Tom Clegg [Tue, 8 Aug 2023 14:18:29 +0000 (10:18 -0400)]
Fix error check.

refs #17244

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20680: installer does rolling update of controller
Peter Amstutz [Tue, 8 Aug 2023 13:36:15 +0000 (09:36 -0400)]
20680: installer does rolling update of controller

When there is a load balancer, disable each controller node in turn
before updating it.

Also, don't reduce controller_nr when a controller is down, having
worker_connections go up and down during deploy seems like in could
cause problems.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months agoMerge branch '20754-docker-py-upgrade'
Brett Smith [Mon, 7 Aug 2023 21:18:56 +0000 (17:18 -0400)]
Merge branch '20754-docker-py-upgrade'

Closes #20754.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20735: Merge branch 'main' into 20735-update-deps 20735-update-deps
Tom Clegg [Mon, 7 Aug 2023 20:58:45 +0000 (16:58 -0400)]
20735: Merge branch 'main' into 20735-update-deps

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Update singularity version in docs and build script.
Tom Clegg [Mon, 7 Aug 2023 20:57:23 +0000 (16:57 -0400)]
20735: Update singularity version in docs and build script.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20705-crunchstat-warn-missing-data'
Tom Clegg [Mon, 7 Aug 2023 20:49:17 +0000 (16:49 -0400)]
Merge branch '20705-crunchstat-warn-missing-data'

fixes #20705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20680-default-config-updates' refs #20680
Peter Amstutz [Mon, 7 Aug 2023 20:29:15 +0000 (16:29 -0400)]
Merge branch '20680-default-config-updates' refs #20680

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20680: Fix nginx_balancer_configuration & add note about defaults 20680-default-config-updates
Peter Amstutz [Mon, 7 Aug 2023 20:28:13 +0000 (16:28 -0400)]
20680: Fix nginx_balancer_configuration & add note about defaults

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20705: 10% failure on stub Create to ensure running error path. 20705-crunchstat-warn-missing-data
Tom Clegg [Mon, 7 Aug 2023 14:31:56 +0000 (10:31 -0400)]
20705: 10% failure on stub Create to ensure running error path.

Fixes occasional test failures due to absence of rate-limiting errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Sync bundler version.
Tom Clegg [Mon, 7 Aug 2023 14:04:38 +0000 (10:04 -0400)]
20735: Sync bundler version.

See https://dev.arvados.org/issues/20735#note-6

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoFixes instance profile assignment. Refs #20610
Lucas Di Pentima [Fri, 4 Aug 2023 20:55:13 +0000 (17:55 -0300)]
Fixes instance profile assignment. Refs #20610

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months agoMerge branch '20610-installer-load-balancer'. Refs #20610
Lucas Di Pentima [Fri, 4 Aug 2023 19:08:32 +0000 (16:08 -0300)]
Merge branch '20610-installer-load-balancer'. Refs #20610

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20754: Upgrade docker module for docker-cleaner 20754-docker-py-upgrade
Brett Smith [Fri, 4 Aug 2023 18:28:18 +0000 (14:28 -0400)]
20754: Upgrade docker module for docker-cleaner

The immediate motivation is dealing with the bug discussed in the
comments. Upgrading generally seems like a good idea since we've been
pinned at a specific version for so long.

Changing the Docker API version is required by the library: 1.21 is the
oldest version currently supported. Our Go stack is standardized on
1.35, so use that here too.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20610: Documentation changes, including simplification & fixes. 20610-installer-load-balancer
Lucas Di Pentima [Fri, 4 Aug 2023 18:08:04 +0000 (15:08 -0300)]
20610: Documentation changes, including simplification & fixes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20754: Remove test requirements from docker-cleaner
Brett Smith [Fri, 4 Aug 2023 18:05:26 +0000 (14:05 -0400)]
20754: Remove test requirements from docker-cleaner

Neither are required with our current Python stack.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20610: Changes the default installer config in order to simplify documentation.
Lucas Di Pentima [Fri, 4 Aug 2023 17:48:29 +0000 (14:48 -0300)]
20610: Changes the default installer config in order to simplify documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20688: Update single host/multiple hostname template 20688-wb1-to-wb2-redirects
Peter Amstutz [Thu, 3 Aug 2023 14:31:05 +0000 (10:31 -0400)]
20688: Update single host/multiple hostname template

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: "workbench" host is now workbench2
Peter Amstutz [Thu, 3 Aug 2023 13:44:41 +0000 (09:44 -0400)]
20688: "workbench" host is now workbench2

"workbench2" host redirects to workbench

Right now, both workbench and workbench2 roles still need to be
assigned, because the 'workbench' role is now configuring nginx but
the 'workbench2' state is the one that actually installs the package.

Will need to make some upstream changes to the formula.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: Reorder redirects by priority
Peter Amstutz [Wed, 2 Aug 2023 21:18:32 +0000 (17:18 -0400)]
20688: Reorder redirects by priority

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: check for ssh ahead of time, sync only before deploy
Peter Amstutz [Wed, 2 Aug 2023 18:39:20 +0000 (14:39 -0400)]
20688: check for ssh ahead of time, sync only before deploy

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: Add nginx redirects from wb1 paths to wb2 paths to salt
Peter Amstutz [Fri, 21 Jul 2023 17:44:36 +0000 (13:44 -0400)]
20688: Add nginx redirects from wb1 paths to wb2 paths to salt

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: Fix quoting issue
Peter Amstutz [Fri, 21 Jul 2023 17:15:11 +0000 (13:15 -0400)]
20688: Fix quoting issue

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20688: Add wb1 to wb2 redirects to arvbox
Peter Amstutz [Tue, 18 Jul 2023 18:57:39 +0000 (14:57 -0400)]
20688: Add wb1 to wb2 redirects to arvbox

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20680: Rework worker settings for new controller behavior
Peter Amstutz [Thu, 3 Aug 2023 18:26:51 +0000 (14:26 -0400)]
20680: Rework worker settings for new controller behavior

We now have separate values for concurrent requests and queued
requests.  Ensure the arvados, passenger and nginx configurations align
with the correct values.

Renamed CONTROLLER_NGINX_WORKERS to CONTROLLER_MAX_WORKERS

Renamed CONTROLLER_MAX_CONCURRENT_REQUESTS to
CONTROLLER_MAX_QUEUED_REQUESTS

Adjusted config.yml defaults to reflect that MaxConcurrentRequests
means something different.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

16 months ago20755: Report metrics for ec2 instances per subnet.
Tom Clegg [Fri, 4 Aug 2023 14:46:00 +0000 (10:46 -0400)]
20755: Report metrics for ec2 instances per subnet.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20610: Documentation fixes & additions.
Lucas Di Pentima [Thu, 3 Aug 2023 20:53:26 +0000 (17:53 -0300)]
20610: Documentation fixes & additions.

Also, removed the 'api' role from the multi host local.params example.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20610: Fixes balancer nginx's request queue config depending on # of backends
Lucas Di Pentima [Thu, 3 Aug 2023 19:58:18 +0000 (16:58 -0300)]
20610: Fixes balancer nginx's request queue config depending on # of backends

Also, fixes templating issue on arvados' config.yml file related to this
setting.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20755: Allow cloud drivers to register their own metrics.
Tom Clegg [Thu, 3 Aug 2023 19:50:50 +0000 (15:50 -0400)]
20755: Allow cloud drivers to register their own metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20755: Support multiple/alternate subnets on EC2.
Tom Clegg [Thu, 3 Aug 2023 19:03:51 +0000 (15:03 -0400)]
20755: Support multiple/alternate subnets on EC2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20610: Unifies the 'api' role with the 'controller' role.
Lucas Di Pentima [Thu, 3 Aug 2023 18:51:13 +0000 (15:51 -0300)]
20610: Unifies the 'api' role with the 'controller' role.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20610: Improves provision.sh code readability.
Lucas Di Pentima [Thu, 3 Aug 2023 15:06:51 +0000 (12:06 -0300)]
20610: Improves provision.sh code readability.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20610: Deduplicates variable substitution code.
Lucas Di Pentima [Thu, 3 Aug 2023 14:05:24 +0000 (11:05 -0300)]
20610: Deduplicates variable substitution code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20610: Renames the ROLES map variable because the name was being used already.
Lucas Di Pentima [Wed, 2 Aug 2023 21:21:22 +0000 (18:21 -0300)]
20610: Renames the ROLES map variable because the name was being used already.

Also, adds "set -eu" to provision.sh and fixes related unbound var issues.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months ago20735: Update Go to 1.20.6.
Tom Clegg [Wed, 2 Aug 2023 20:11:11 +0000 (16:11 -0400)]
20735: Update Go to 1.20.6.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Update ruby and singularity.
Tom Clegg [Wed, 12 Jul 2023 15:35:48 +0000 (11:35 -0400)]
20735: Update ruby and singularity.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20735: Update to go 1.20.
Tom Clegg [Wed, 12 Jul 2023 15:34:45 +0000 (11:34 -0400)]
20735: Update to go 1.20.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20610: Moves code from local.params to its own common.sh file.
Lucas Di Pentima [Wed, 2 Aug 2023 13:38:07 +0000 (10:38 -0300)]
20610: Moves code from local.params to its own common.sh file.

provision & installer scripts now loads this new file that in turn loads
the params files.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

16 months agoMerge branch '17244-cgroup2'
Tom Clegg [Wed, 2 Aug 2023 13:24:32 +0000 (09:24 -0400)]
Merge branch '17244-cgroup2'

refs #17244

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago17244: Fix doc link.
Tom Clegg [Wed, 2 Aug 2023 13:23:40 +0000 (09:23 -0400)]
17244: Fix doc link.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months agoMerge branch '20797-rhel8-packages'
Brett Smith [Wed, 2 Aug 2023 01:08:36 +0000 (21:08 -0400)]
Merge branch '20797-rhel8-packages'

Closes #20797.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Merge Red Hat package install instructions 20797-rhel8-packages
Brett Smith [Tue, 1 Aug 2023 18:07:32 +0000 (14:07 -0400)]
20797: Merge Red Hat package install instructions

For now, since the instructions are *exactly* the same across distros
and versions, this makes sense as a streamlining measure. If we add more
variation to the process in the future, it probably makes sense to split
these out again.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Run microdnf with --assumeyes
Brett Smith [Tue, 1 Aug 2023 16:06:59 +0000 (12:06 -0400)]
20797: Run microdnf with --assumeyes

`docker run --rm -ti rockylinux:8-minimal microdnf --help` says,

> The "--assumeyes" option is turned on by default.

However, that behavior apparently changes in a future version:
<https://github.com/rpm-software-management/microdnf/pull/97>,
microdnf commit ca778a0b34c5e15b5b0de1e0af0f8892280fc0c2.

Passing `--assumeyes` provides more consistency and will help keep the
code working in later releases.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Prefer dnf in PostgreSQL install instructions
Brett Smith [Tue, 1 Aug 2023 16:02:30 +0000 (12:02 -0400)]
20797: Prefer dnf in PostgreSQL install instructions

I believe this is the current preferred tool, and provides consistency
with other RH8 documentation.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago17244: Update docs for -cgroup-parent-subsystem flag.
Tom Clegg [Mon, 31 Jul 2023 19:52:50 +0000 (15:52 -0400)]
17244: Update docs for -cgroup-parent-subsystem flag.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago17244: Clarify log message.
Tom Clegg [Mon, 31 Jul 2023 19:33:18 +0000 (15:33 -0400)]
17244: Clarify log message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20705: Warn if some stats are missing.
Tom Clegg [Mon, 31 Jul 2023 17:54:24 +0000 (13:54 -0400)]
20705: Warn if some stats are missing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

16 months ago20797: Update cross compilation detection logic
Brett Smith [Mon, 31 Jul 2023 15:37:38 +0000 (11:37 -0400)]
20797: Update cross compilation detection logic

The immediate bug that needs to be fixed is that we currently don't
support cross compilation on rocky8.

DRY up the code to try to make it easier to follow what's going on
here.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Note Alma/CentOS/Red Hat/Rocky 8 support in install docs
Brett Smith [Fri, 28 Jul 2023 19:53:45 +0000 (15:53 -0400)]
20797: Note Alma/CentOS/Red Hat/Rocky 8 support in install docs

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Copyedit "CentOS" for consistent capitalization
Brett Smith [Fri, 28 Jul 2023 19:12:44 +0000 (15:12 -0400)]
20797: Copyedit "CentOS" for consistent capitalization

I left the upgrade notes alone since they're more anchored in time, but
this should get the rest of the documentation.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Add basic documentation for how to add a new target distro
Brett Smith [Fri, 28 Jul 2023 18:51:18 +0000 (14:51 -0400)]
20797: Add basic documentation for how to add a new target distro

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

16 months ago20797: Add basic intro documentation to build README
Brett Smith [Fri, 28 Jul 2023 18:41:15 +0000 (14:41 -0400)]
20797: Add basic intro documentation to build README

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>