Brett Smith [Fri, 30 May 2025 17:13:52 +0000 (13:13 -0400)]
22944: Bugfix and DRY Ansible bootstrap tasks
By default, Ansible tries to gather host facts at the start of each
play. In order for the bootstrap task to work as intended, it must run
in an early play with `gather_facts: no`.
Make this easier by adding a role to do it. This removes some
bootstrapping of the `gnupg` and `xz-utils` packages. Both of these were
carried over from previous automations, and they don't seem to really be
required by Ansible directly. I have been able to run all these
playbooks as-is. They might be getting pulled in as dependencies of
other packages, or might be core to more recent Debian, but whatever the
reason, we should be fine.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 30 May 2025 14:50:30 +0000 (10:50 -0400)]
22957: Add requirements files to Ansible
We are at a point where we want to use collections that aren't included
in Ansible 8. This commit adds the infrastructure for us to do that in
the standard way, while taking the opportunity to only declare what we
need (no more installing the 240MiB Fortinet collection everywhere).
The change to the PostgreSQL HBA sources is required by stricter
validation in the newer community.postgresql module. The change from
parameter names from `db` to `login_db` is because the former is
deprecated in this version.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 27 May 2025 18:33:47 +0000 (14:33 -0400)]
22779: Skip reading ~/.aws from the a-d-c service
See comments for background. Setting these variables lets a-d-c act more
like it did in previous versions while leaving a way for administrators
to opt in to newer behavior. We may do something different for Arvados
3.2.0, but this is where we want to be today. This was discussed between
Peter, Tom, Lucas, and myself. Refs #22779.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 8 May 2025 13:10:53 +0000 (09:10 -0400)]
21362: Search for bundler in the arvados-api-server package test
We need to search for Bundler the same way the package postinst does for
all the same reasons. I know this is pushing the limits of what I should
push directly to main, but:
* This is code we already have, and kind of by definition it needs to
work the same way here, so it's been reviewed in some capacity.
* I don't see any good way to DRY this up and share this code between
the postinst and the package test script. I mean I could
rig *something* up, there are options, but it would all be ad hoc
shell includes, nothing especially reliable. If someone feels strongly
about it I'm willing to take on a follow-up ticket, but at least this
unbreaks the build for now.
Refs #21362.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 8 May 2025 10:54:12 +0000 (06:54 -0400)]
21362: Add tar to arvados-api-server dependencies
This is required for Passenger to install nginx, whether it downloads a
binary or compiles source. We missed this dependency before because tar
was preinstalled in our package test Docker images. Refs #21362.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Mon, 5 May 2025 19:44:27 +0000 (15:44 -0400)]
22680: Make 'any_searchable_columns' which excludes datetime columns
Can't use datetime in all-column search because Postgresql will try to
parse the string literal into a datetime during query compile time and
throw an error. We'd prefer the comparison just evaluated to 'false'
but that's not what it does. Stack Overflow suggests there a solution
involving trapping the error with a SQL stored procedure but the more
immediate fix is to just ignore the columns entirely.
The fact that we haven't seen this error before means no one is
relying on this behavior, while fixing it allows us to have passing
tests that check for a concrete security concern in the credentials
API.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Mon, 5 May 2025 19:51:26 +0000 (15:51 -0400)]
21362: Streamline package test Dockerfiles
Basically a lot of the Gem/Bundler setup being done here should be
handled by the RailsAPI package instead. The test is more meaningful if
we let it happen there rather than preinstalling prerequisites in the
image.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 5 May 2025 19:49:16 +0000 (15:49 -0400)]
21362: Note what it means if Bundler removes "stale" gems
This is behavior I observed while trying to build Rocky 9 packages with
Ruby 3.3. RailsAPI is pinned to the last version of Nokogiri that
supports Ruby 2.7, which says that it requires Ruby <3.3.dev.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 5 May 2025 19:14:41 +0000 (15:14 -0400)]
21362: Add nginx compilation dependencies for rocky9 package
For reasons I don't understand, Passenger can't use a prebuilt nginx
binary on Rocky 9, it has to build it from source. Add package
dependencies for that to succeed.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 30 Apr 2025 18:29:13 +0000 (14:29 -0400)]
22822: Fix VIRTUAL_ENV path in Python packages
The code we have to update the `activate` script is failing on at least
some distributions, so activating the virtualenv appears to work but is
ineffective.
Trying to modify shell source on the fly is inherently a dicey
proposition. Instead of doing that, let's rely on the fact that builds
happen in an ephemeral Docker container, and just build the "source"
virtualenv in the same path where the package will install it,
eliminating the need for any rewriting.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
projects / arvados.git / log