Brett Smith [Fri, 4 Aug 2023 18:28:18 +0000 (14:28 -0400)]
20754: Upgrade docker module for docker-cleaner
The immediate motivation is dealing with the bug discussed in the
comments. Upgrading generally seems like a good idea since we've been
pinned at a specific version for so long.
Changing the Docker API version is required by the library: 1.21 is the
oldest version currently supported. Our Go stack is standardized on
1.35, so use that here too.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Thu, 3 Aug 2023 13:44:41 +0000 (09:44 -0400)]
20688: "workbench" host is now workbench2
"workbench2" host redirects to workbench
Right now, both workbench and workbench2 roles still need to be
assigned, because the 'workbench' role is now configuring nginx but
the 'workbench2' state is the one that actually installs the package.
Will need to make some upstream changes to the formula.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 3 Aug 2023 18:26:51 +0000 (14:26 -0400)]
20680: Rework worker settings for new controller behavior
We now have separate values for concurrent requests and queued
requests. Ensure the arvados, passenger and nginx configurations align
with the correct values.
Renamed CONTROLLER_NGINX_WORKERS to CONTROLLER_MAX_WORKERS
Renamed CONTROLLER_MAX_CONCURRENT_REQUESTS to
CONTROLLER_MAX_QUEUED_REQUESTS
Adjusted config.yml defaults to reflect that MaxConcurrentRequests
means something different.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Tue, 1 Aug 2023 18:07:32 +0000 (14:07 -0400)]
20797: Merge Red Hat package install instructions
For now, since the instructions are *exactly* the same across distros
and versions, this makes sense as a streamlining measure. If we add more
variation to the process in the future, it probably makes sense to split
these out again.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 1 Aug 2023 16:06:59 +0000 (12:06 -0400)]
20797: Run microdnf with --assumeyes
`docker run --rm -ti rockylinux:8-minimal microdnf --help` says,
> The "--assumeyes" option is turned on by default.
However, that behavior apparently changes in a future version:
<https://github.com/rpm-software-management/microdnf/pull/97>,
microdnf commit ca778a0b34c5e15b5b0de1e0af0f8892280fc0c2.
Passing `--assumeyes` provides more consistency and will help keep the
code working in later releases.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 28 Jul 2023 19:01:17 +0000 (15:01 -0400)]
20797: Update for new createrepo package+executable name
In Debian 12 the package is named createrepo-c and it installs
/usr/bin/createrepo_c. With this change, the build script prefers that
name, but still looks for and uses `createrepo`.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 28 Jul 2023 18:09:02 +0000 (14:09 -0400)]
20797: Streamline setup of Python build variables
Instead of setting Debian defaults and overriding them,
1. set global defaults
2. set target-specific variables
3. set deb/rpm-specific defaults as needed
I hope this makes clearer what's actually a target-specific variable
vs. a regular difference between Debian and Red Hat, and so makes it
easier to add targets in the future.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Thu, 27 Jul 2023 18:39:56 +0000 (15:39 -0300)]
20610: Restricts backends' HTTP access by nginx rules.
I think this is better than implementing those controls through security
groups via Terraform, because the node's role information is already available
on the salt code.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Wed, 26 Jul 2023 21:06:41 +0000 (18:06 -0300)]
20610: Removes duplication on balancer & backend declarations.
Instead of having to explicitly declare those, just use the ROLES map
to get the balancer and controller nodes.
Also, explicitly gives PG access to Keep-Balance instead of assuming it's
installed on the same node as the controller.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Fri, 7 Jul 2023 20:57:38 +0000 (16:57 -0400)]
20684: Expand "Getting started at the command line" documentation
* Add more structure to the page
* Be explicit that we usually recommend using a VM
* Highlight that using our package repository is the simplest way to
install tools
* List all the tools you can install, with more details
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 5 Jul 2023 20:58:07 +0000 (16:58 -0400)]
20343: Define Python regexps with raw strings
Prevent the DeprecationWarning added to Python 3.6 for unrecognized
string escape sequences. Found candidates by running:
git grep --line-number -E '\bre\.\w+\([^r]*\\'
Most of the changes just add the raw string prefix `r` to these regexps,
but I did make some other readability improvements while I was at it,
including switching from regexps to plain string test methods where
appropriate.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 5 Jul 2023 16:20:57 +0000 (12:20 -0400)]
20710: Switch from pipes.quote to shlex.quote
pipes is deprecated in Python 3.11+ and slated to be removed in Python
3.13. Fortunately we only use pipes.quote, which is literally the same
as shlex.quote:
>>> import pipes
<stdin>:1: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
>>> import shlex
>>> pipes.quote is shlex.quote
True
Switch to that instead for continued support.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Fri, 30 Jun 2023 14:28:38 +0000 (10:28 -0400)]
20559: Trim session count explicitly instead of using LRU cache.
The LRU cache automatically implemented the session count limit by
dropping the oldest session, which would break our "one session per
token" rule (by evicting a session and creating a new one while the
old session was still in use) when there were more active sessions
than the configured limit.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
projects / arvados.git / log