Brett Smith [Tue, 12 Aug 2025 02:30:47 +0000 (22:30 -0400)]
20311/23090: Set Python package license based on filename
The License metadata we were using previously is deprecated in favor of
License-Expression. However, the latest version of setuptools that
supports Python 3.8 doesn't set that. Use the license filename to set a
human-friendly license name.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Stephen Smith [Tue, 12 Aug 2025 13:14:11 +0000 (09:14 -0400)]
23083: Exempt background refreshes from queueing data explorer refreshes
Background refresh is not needed if the DE is already loading because background
refresh does not change parameters. This also prevents background refreshes from
inheriting background=false, extending loading screens unnecessarily.
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Brett Smith [Mon, 11 Aug 2025 22:35:20 +0000 (18:35 -0400)]
20311/23090: Restore systemd service to docker-cleaner package
This was accidentally lost in the transition: note that c1d84bfed3be75d342d800011501fcfef7a1b756 removes it from `data_files` in
`setup.py` but does not add it to `data-files` in `pyproject.toml`.
Refs #20311, #23090.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Tue, 5 Aug 2025 21:50:45 +0000 (18:50 -0300)]
23010: Uses new pg-formula supporting Ubuntu 24.04
Also, avoid installing postgresql-contrib as it depends on the currently
supported PG version on each distro, eg: install postgresql-16 on Ubuntu
24.04 so both v15 and v16 get installed (because v15 is the one
requested by the installer) and that creates issues with service ports.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Tue, 5 Aug 2025 20:59:10 +0000 (16:59 -0400)]
22436: Remove arvados-package
Because Arvados is deployed with third-party software like nginx, a
configuration management tool like Salt or Ansible is better suited to
orchestrate the install compared to a distribution package. We are
focusing future development on the Ansible installer.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 4 Aug 2025 19:33:32 +0000 (15:33 -0400)]
Restart Docker after updating daemon.json
This was not in the original version of the code because it was written
to build cloud compute images where we just needed to write the right
configuration. Now that it supports single-node installs and maybe more
in the future, we need to make sure services get new state. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Mon, 4 Aug 2025 17:59:57 +0000 (13:59 -0400)]
23025: Fix double redirect.
Previous code did not use the query token to look up the container, so
the first request would fail the container lookup and send a 303
moving the token to a cookie, and the resulting second request would
succeed at the container lookup only to find the container UUID does
not match the client's cookie, and send another 303 to clear site
data.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Mon, 4 Aug 2025 01:47:10 +0000 (21:47 -0400)]
20311: Remove deprecated setup.py invocations
This is possible now that we define pyproject.toml for all our Python
modules. I'm not bothering with `arvados-server install` and `arvbox`
since they're both slated for removal in Arvados 3.2.0.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 4 Aug 2025 00:48:17 +0000 (20:48 -0400)]
20311: Remove unused get-package-version script
In addition to not having references in the repository, it was never
updated to remove the `development-` prefix following b81baa970ca14b1ace4b9f0dc2ea0eae8330e788.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 1 Aug 2025 21:24:14 +0000 (17:24 -0400)]
20311: Generate _version.py from a setuptools subcommand
This restructures arvados_version.py so that metadata generation happens
in a dedicated class (useful for other build tooling like
build_docker_image.py) and then it provides a setuptools subcommand to
write _version.py where needed. This provides cleaner separation of
concerns and gives us a better base to build more tooling from.
As part of this change, we take care to no longer install tests as part
of the wheel we build. We shouldn't have been doing this anyway, and now
it's required for the tests to run as expected.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 1 Aug 2025 21:21:32 +0000 (17:21 -0400)]
20311: Remove sys.path manipulation from run_test_server.py
This has been obsolete since run-tests.sh started installing the PySDK
to its virtualenv as part of `install env`, and that's a better approach
since it also covers dependencies, etc.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 31 Jul 2025 14:22:41 +0000 (10:22 -0400)]
20311: Make discovery pydoc build a subcommand
The removed comment is no longer a concern now that we consistently use
`pip install`, and this makes it easier to further extend the build
process for PEP 517.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Thu, 31 Jul 2025 14:09:48 +0000 (10:09 -0400)]
23044: Pass ARVADOS_TEST_PRIVESC through in test cluster.
My intent was to enable the use of sudo in integration tests with
{RuntimeEngine: singularity}. This still doesn't work with
non-passwordless sudo, but at least it tries to use sudo when so
configured, and logs something in stderr.txt if it fails.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 24 Jul 2025 17:51:51 +0000 (13:51 -0400)]
23044: Enable use of crunch-run gateway with loopback driver.
Normally a gateway address of 127.0.0.1:* means crunch-run couldn't
figure out its external IP address, but in the case of the loopback
driver, it's correct.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Thu, 24 Jul 2025 00:01:17 +0000 (20:01 -0400)]
Run `bundle install` for multi_json
Should've been included in the previous commit. I thought this would be
a noop because we already had an indirect dependency on
`multi_json (1.15.0)`, but I guess (to its credit) bundler does check
everything. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 23 Jul 2025 19:58:28 +0000 (15:58 -0400)]
Only set Docker data-root for cloud compute nodes
b6d8d5665243578986cefc5c3fec850d6cf25b05 could cause single-node
installs to configure Docker with a data root at /tmp/docker-data, which
is probably not what people want. Instead set this in
build-compute-image.yml, and introduce a system to extend the Docker
daemon.json selectively. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 23 Jul 2025 19:14:17 +0000 (15:14 -0400)]
Do not chown/chmod /etc/arvados
On dedicated compute nodes this is unlikely to exist and no big deal,
but on single-node installs the admin may choose to change the
ownership (especially group) of this directory. Remove those specifiers
to avoid interfering with that. Ansible's defaults should be fine for
a newly-created directory. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 23 Jul 2025 19:12:35 +0000 (15:12 -0400)]
Configure Docker when installing it with crunch-dispatch-local
In order for crunch-dispatch-local to work, the local node needs to be
configured like any other compute node. Most of this is already done via
the dependency on arvados_compute, but that does not configure Docker
services, that's the job of compute_docker. Pull that in when necessary.
No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 23 Jul 2025 13:32:13 +0000 (09:32 -0400)]
Merge Python .gitignore files into top level
At least in my development environment, the symlinks generally don't
work and confuse tools. Most of the temporary-type files are already
covered by patterns in .gitignore, so the easiest fix from here is just
to ignore the full build and dist paths. No issue #.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 23 Jul 2025 01:56:09 +0000 (21:56 -0400)]
23064: Start contrib directory
The goal of this directory is to help users better understand what
components are "core" Arvados vs. additional tools and to give us a
place to start building more of the latter.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 22 Jul 2025 15:16:41 +0000 (11:16 -0400)]
22817: Remove Java SDK Docker test scripts
These were used by previous Jenkins configuration. Our current nodes are
sure to have Gradle, so we have eliminated the need for this indirection.
Closes #22817.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sun, 20 Jul 2025 17:47:24 +0000 (13:47 -0400)]
23006: Hardcode runner image inside arvbox
It is already a known limitation that arvbox can't build its own
arvados/jobs image because it runs Singularity. This pins it to a
known-good version while we work on transitioning away from arvbox.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
projects / arvados.git / log