Brett Smith [Wed, 10 Jan 2024 18:43:20 +0000 (13:43 -0500)]
21367/21230: Set up dev-jobs with virtualenv
The functional goal here is to do all pip installs inside the Dockerfile
inside a virtualenv to avoid the global pip install issues.
Changes that fall out of that:
* Installing python3-venv is all we need to go back to the -slim Debian
image.
* Stop installing other Python packages we'll just install inside the
virtualenv anyway.
* Rather than having separate arguments and code blocks for every
package we might want to install, just set up a dedicated build
context for the Docker image with all the packages we want to install,
and install them all unconditionally during the image build. This is
much less code and hopefully easier to follow.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 10 Jan 2024 18:33:36 +0000 (13:33 -0500)]
21367: Remove environment check
You only need for this for `arv-keepdocker` at the end of the process,
which can read from settings.conf just fine too. Even if it fails, the
developer will have the command handy to fix their environment and try
that again easily.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 6 Jan 2024 21:00:25 +0000 (16:00 -0500)]
20846: Use Python build variables more consistently
Rather than have the virtualenv process use its own python command,
ensure PYTHON3_EXECUTABLE points to a specific minor version, and use it
throughout the entire build process.
The changes to the virtualenv manipulation with sed are meant to clean
things up with fewer commands and tighter regexps, but not make any
functional change beyond referring to a specific minor Python version.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 5 Jan 2024 21:53:57 +0000 (16:53 -0500)]
21273: Build rocky8 Python packages against python39
We can no longer support the Python 3.6 that ships with rocky8. Building
against the latest appstream release is the most straightforward way to
continue building packages for the distro.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Thu, 4 Jan 2024 22:51:33 +0000 (17:51 -0500)]
21258: Ensure at least one boot failure.
With the previous approach, it was possible for all containers needing
a type4 instance to finish, and a different instance type to report a
quota error and cause the scheduler to shut down the now-unneeded
instance, all before the "guaranteed broken" node reached
TimeoutBooting. In such a case it would not be counted as a boot
failure.
To avoid this, the new approach induces boot failures on *all* type4
instances until 2x TimeoutBooting intervals have passed.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Thu, 4 Jan 2024 19:15:43 +0000 (14:15 -0500)]
20846/21230: Remove Python transitional metapackages from the package build
The build process for this is relatively expensive (we set up a
dedicated virtualenv for each one) and not PEP 668-compliant (it tries
to upgrade packages globally). We don't need these packages anymore for
the Arvados 3.0 release. Stop building all of them.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 4 Jan 2024 19:00:46 +0000 (14:00 -0500)]
20846/21230: Modernize virtualenv setup in package building
This change sets up the virtualenv earlier, then does all work inside
it, starting with installing build dependencies, to comply with PEP 668.
A couple optimizations that were easy to do as part of this work:
* We install modules in as few pip commands as possible, to reduce
the overhead of determining what's currently installed, what's
available from PyPI, etc.
* Removed a special case for CentOS 7 that we no longer need.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Wed, 3 Jan 2024 19:40:28 +0000 (14:40 -0500)]
21278: Remove the '/search.*' to '/search-results' redirect
It produces an infinite loop and I don't think the /search endpoint is
really compatible with the /search-results endpoint anyway, and it's
less likely for people to have old hyperlinks to search results than
to projects or collections.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Fri, 29 Dec 2023 16:16:42 +0000 (11:16 -0500)]
20846/18874: Update NodeJS/Yarn in debian12/ubuntu2204
This applies the change from c4d4088d3e7f3e3f896ed0bb482560ab702d5465 to
the debian12 and ubuntu2204 package build Dockerfiles. It uses a
slightly different yarn install recipe to avoid the "double ln" and
associated layer, but the end result is the same.
Refs #18874, #20846. These branches were in development in parallel, so
the changes from the former weren't applied to the latter.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 28 Dec 2023 19:29:30 +0000 (14:29 -0500)]
Use full paths for API documentation links
Most of the cross-reference links in our documentation point to a
specific page. These links are unusual in that they point to a
directory. Lately the linkchecker has been intermittently complaining
about them:
URL `file:///tmp/workspace/developer-run-tests-doc-and-sdk-R/doc/.site/api'
Name `API documentation'
Parent URL file:///tmp/workspace/developer-run-tests-doc-and-sdk-R/doc/.site/admin/scoped-tokens.html, line 306, col 576
Real URL file:///tmp/workspace/developer-run-tests-doc-and-sdk-R/doc/.site/api/
Warning [file-missing-slash] Added trailing slash to
directory.
Result Valid: directory
This causes a test failure. Spell out the rest of the link to prevent
that. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Thu, 21 Dec 2023 14:20:09 +0000 (09:20 -0500)]
21305: Add piwik and google analytics tracking to layout
This used to be a hidden branch that was cherry-picked called
"public-site-piwik", so this just brings it into the main branch to be
able to maintain it.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
projects / arvados.git / log