Brett Smith [Wed, 2 Jul 2025 02:06:13 +0000 (22:06 -0400)]
22830: Write default controller MaxRequestSize in Ansible
In general, we should do #22830. But right now this is the only thing
that Ansible reads from the API section, so we can get a big usability
win for single-node installs by just duplicating the default here for now.
Refs #22830.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 30 Jun 2025 21:25:14 +0000 (17:25 -0400)]
23007: Fix expected test output for ruamel.yaml 0.18.13+
This release changed the wrapping of its output, which goes into a Keep
block, which goes into a manifest, which the CWL tests expect to match
an exact PDH. We might want to do something more sophisticated here but
this should get the Jenkins tests passing again for now.
Refs #23007.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 30 Jun 2025 18:54:36 +0000 (14:54 -0400)]
22965: Update default CI_REF
After reconfiguring a bunch of Jenkins jobs, it became clear that this
is a better default. We can consistently use the `ci-build` ref name to
point to the commit we want to build from that remote, for all remotes.
Refs #22965.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 10 Jun 2025 20:45:53 +0000 (16:45 -0400)]
22958: Update Python packages to depend on libcurl4
This is sort of a side effect of the switch to building Docker images
with Ansible, but I wanted it to be a separate commit for
clarity. `install-dev-tools.yml` installs `libcurl4-openssl-dev` because
it's clearly the preferred version: it depends on libcurl4, which in
turn is the version used by curl. The package build Dockerfiles use
`libcurl4-gnutls-dev` which depends on `libcurl3-gnutls` which is
clearly second-rate.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 10 Jun 2025 20:43:33 +0000 (16:43 -0400)]
22958: Use Ansible to build package build Docker images
This extends the install-dev-tools.yml playbook to introduce new
`arvados_build_one_target` and `arvados_build_all_targets` groups, to
clarify what happens inside the Docker container vs. outside. Then
`arvados_build_one_target` installs additional build tools that are
required for packages like fpm.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 10 Jun 2025 20:21:51 +0000 (16:21 -0400)]
22958: Introduce distro_dnf role to Ansible
Akin to distro_packages, this role is a common place that knows what
module streams we enable on different versions of RHEL, and can enable
any that are requested.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 10 Jun 2025 20:15:30 +0000 (16:15 -0400)]
22958: Introduce distro_packages role
The point of this role is to abstract away differences between
distributions. This is the one place that knows about packaging
differences across distributions and takes care of it, so we don't have
to keep re-encoding the same logic across our playbooks.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
This commit completely reorganizes `install-test-env.yml` to make it
possible to install a subset of dependencies using inventory
groups. This is a first step towards reusing the playbook in other
contexts like package build Dockerfiles.
If you put a host in the `arvados_test_all` group, the playbook still
manages all the same stuff it did before, it's just completely
reorganized.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 2 Jun 2025 14:50:35 +0000 (10:50 -0400)]
22957: Build package test Docker images with Ansible
This is not a huge win by itself, but it means we can manage these
images with a single playbook rather than one Dockerfile per image. The
bigger win will come when we do the same for package build Dockerfiles,
which can reuse a lot of the code we've already written in
`install-test-env.yml`.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 30 May 2025 17:13:52 +0000 (13:13 -0400)]
22944: Bugfix and DRY Ansible bootstrap tasks
By default, Ansible tries to gather host facts at the start of each
play. In order for the bootstrap task to work as intended, it must run
in an early play with `gather_facts: no`.
Make this easier by adding a role to do it. This removes some
bootstrapping of the `gnupg` and `xz-utils` packages. Both of these were
carried over from previous automations, and they don't seem to really be
required by Ansible directly. I have been able to run all these
playbooks as-is. They might be getting pulled in as dependencies of
other packages, or might be core to more recent Debian, but whatever the
reason, we should be fine.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 30 May 2025 14:50:30 +0000 (10:50 -0400)]
22957: Add requirements files to Ansible
We are at a point where we want to use collections that aren't included
in Ansible 8. This commit adds the infrastructure for us to do that in
the standard way, while taking the opportunity to only declare what we
need (no more installing the 240MiB Fortinet collection everywhere).
The change to the PostgreSQL HBA sources is required by stricter
validation in the newer community.postgresql module. The change from
parameter names from `db` to `login_db` is because the former is
deprecated in this version.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Fri, 30 May 2025 18:25:33 +0000 (15:25 -0300)]
22613: Use distribution-provided PostgreSQL service instead of upstream
This is to workaround an issue with the missing pgdg-keyring deb package
that the postgresql-formula attempts to download when using PostgreSQL
from postgresql.org.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
projects / arvados.git / log