arvados-dev.git
10 days agoMerge branch '22607-buildx' main
Brett Smith [Mon, 24 Feb 2025 22:36:35 +0000 (17:36 -0500)]
Merge branch '22607-buildx'

Refs #22607.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 days ago22607: Clarify new task name
Brett Smith [Mon, 24 Feb 2025 22:36:26 +0000 (17:36 -0500)]
22607: Clarify new task name

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 days ago22607: Install docker-buildx-plugin 22607-buildx
Brett Smith [Mon, 24 Feb 2025 20:31:04 +0000 (15:31 -0500)]
22607: Install docker-buildx-plugin

Required by arvbox, whose Dockerfiles use `RUN --mount`.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks agoMerge branch '22594-unused-packer-images'
Brett Smith [Thu, 20 Feb 2025 17:26:29 +0000 (12:26 -0500)]
Merge branch '22594-unused-packer-images'

Closes #22594.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks agoInstall amazon Packer plugin
Brett Smith [Thu, 20 Feb 2025 17:24:58 +0000 (12:24 -0500)]
Install amazon Packer plugin

Needed to build Curii compute nodes.
Refs #22489.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks agoNot using obscured SSH ports any more (refs #22540)
Peter Amstutz [Thu, 20 Feb 2025 16:02:06 +0000 (11:02 -0500)]
Not using obscured SSH ports any more (refs #22540)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 weeks ago22594: Add ci_ansible role to test-provision playbook 22594-unused-packer-images
Brett Smith [Thu, 20 Feb 2025 15:20:16 +0000 (10:20 -0500)]
22594: Add ci_ansible role to test-provision playbook

This is anticipating a near future where we want the test-provision
nodes to be able to run Ansible playbooks. I would rather include this
now when I start building initial test images from the playbook.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22594: Add an Ansible playbook for jenkins-image-provision
Brett Smith [Thu, 20 Feb 2025 15:16:25 +0000 (10:16 -0500)]
22594: Add an Ansible playbook for jenkins-image-provision

This modernizes the setup for test-provision and lets us get rid of the
last image setup shell scripts.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22594: Remove jenkins-image-arvados-formula-tests
Brett Smith [Thu, 20 Feb 2025 14:59:53 +0000 (09:59 -0500)]
22594: Remove jenkins-image-arvados-formula-tests

This image has been used to test our Arvados Salt formula. This image
actually does use a tool for that that isn't included in other
images (kitchen). But we haven't actually run these formula tests in
three years despite making arvados-formula changes in the meantime—we
mostly lean on test-provision to be a big integration test for us.

We're not likely to revisit this since we plan to move to an
Ansible-based installer. Just remove this image.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22594: Remove jenkins-image-k8s-tests
Brett Smith [Thu, 20 Feb 2025 14:56:02 +0000 (09:56 -0500)]
22594: Remove jenkins-image-k8s-tests

We never set up a Jenkins job to actually use this image, as evidenced
by the fact it never got updated past Debian 10. So much has changed
about Kubernetes in the meantime that I expect if we revisit support or
testing, we'll want to look at it with fresh eyes.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22594: Remove jenkins-image-federation-tests
Brett Smith [Thu, 20 Feb 2025 14:50:42 +0000 (09:50 -0500)]
22594: Remove jenkins-image-federation-tests

This package list is stale (python2.7-dev?) and a subset of what gets
installed in jenkins-image-arvados-tests. Jenkins jobs that want this
software should just use that image instead.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks agoMerge branch '22579-jenkins-image-cleanups'
Brett Smith [Wed, 19 Feb 2025 13:57:36 +0000 (08:57 -0500)]
Merge branch '22579-jenkins-image-cleanups'

Refs #22579.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22579: Remove jenkins-image-with-docker 22579-jenkins-image-cleanups
Brett Smith [Tue, 18 Feb 2025 16:42:27 +0000 (11:42 -0500)]
22579: Remove jenkins-image-with-docker

jenkins-image-arvados-tests already included all this functionality, and
we would rather just use that than maintain a separate image.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22579: Remove jenkins-image-workbench2
Brett Smith [Tue, 18 Feb 2025 16:29:44 +0000 (11:29 -0500)]
22579: Remove jenkins-image-workbench2

This supported Workbench 2 tests when it lived in a separate Git
repository. These tests are now run under jenkins-image-arvados-tests.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks ago22579: Update README
Brett Smith [Tue, 18 Feb 2025 16:27:52 +0000 (11:27 -0500)]
22579: Update README

With a link to the wiki that already has a lot more information.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 weeks agoPublish arvados-user-activity and arvados-cluster-activity, refs #22549
Peter Amstutz [Mon, 17 Feb 2025 21:57:39 +0000 (16:57 -0500)]
Publish arvados-user-activity and arvados-cluster-activity, refs #22549

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 weeks agoMerge branch '22489-jenkins-tests-ansible'
Brett Smith [Wed, 12 Feb 2025 20:23:44 +0000 (15:23 -0500)]
Merge branch '22489-jenkins-tests-ansible'

Refs #22489.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

3 weeks ago22489: Install Packer googlecompute plugin 22489-jenkins-tests-ansible
Brett Smith [Sun, 9 Feb 2025 23:19:05 +0000 (18:19 -0500)]
22489: Install Packer googlecompute plugin

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

3 weeks ago22489: Fix JSON syntax (trailing comma)
Brett Smith [Sun, 9 Feb 2025 21:57:47 +0000 (16:57 -0500)]
22489: Fix JSON syntax (trailing comma)

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

3 weeks ago22489: Fix JSON syntax
Brett Smith [Sun, 9 Feb 2025 21:55:29 +0000 (16:55 -0500)]
22489: Fix JSON syntax

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

3 weeks ago22489: Add Ansible playbooks to build Docker and test nodes
Brett Smith [Sun, 9 Feb 2025 15:26:51 +0000 (10:26 -0500)]
22489: Add Ansible playbooks to build Docker and test nodes

This lets us move away from building and running `arvados-server
install` and instead use the Ansible playbook we've already written to
build a test node. This also makes it easier to start sharing code
across Packer image builds.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 weeks agoMerge branch '22406-rails-db-bugfix'
Brett Smith [Mon, 27 Jan 2025 16:34:17 +0000 (11:34 -0500)]
Merge branch '22406-rails-db-bugfix'

Closes #22474.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 weeks ago22406: Install arvados-server on test nodes during build 22406-rails-db-bugfix
Brett Smith [Tue, 21 Jan 2025 20:18:58 +0000 (15:18 -0500)]
22406: Install arvados-server on test nodes during build

See comments for rationale. `arvados-server install` runs database setup
Rake tasks, which require `arvados-server` to be on $PATH to load
configuration.

We should probably stop doing that and just let `run-tests.sh` do the
database setup since it needs to anyway. That can happen as part of
migrating this Jenkins job to use the test environment Ansible playbook.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months agoMerge branch '22383-jenkins-install-ansible'
Brett Smith [Tue, 10 Dec 2024 15:22:44 +0000 (10:22 -0500)]
Merge branch '22383-jenkins-install-ansible'

Refs #22383.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Remove scaffolding from node-ready.sh 22383-jenkins-install-ansible
Brett Smith [Sat, 7 Dec 2024 13:18:02 +0000 (08:18 -0500)]
22383: Remove scaffolding from node-ready.sh

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Bugfix systemctl list arguments
Brett Smith [Fri, 6 Dec 2024 22:58:35 +0000 (17:58 -0500)]
22383: Bugfix systemctl list arguments

`--quiet` does not disable the legend on older versions.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Bugfix ssh service start
Brett Smith [Fri, 6 Dec 2024 21:45:20 +0000 (16:45 -0500)]
22383: Bugfix ssh service start

`read` returns a nonzero exit status if it reaches EOF while run
non-interactively. The old code relied on a partial line read and
tripped this up.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383/22186: Wait for cloud-init only when in use
Brett Smith [Fri, 6 Dec 2024 20:53:04 +0000 (15:53 -0500)]
22383/22186: Wait for cloud-init only when in use

This wait was added to get the build working on some Ubuntu base images,
but was not tested on Debian base images after being added. Those images
don't have cloud-init installed, so don't wait for it.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Install Ansible in Docker image
Brett Smith [Fri, 6 Dec 2024 20:27:03 +0000 (15:27 -0500)]
22383: Install Ansible in Docker image

To support the new compute node build.

I'm making the judgment call that jq and Python 3+its venv module are
generally useful enough to install on all images.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Handle different SSH service name on RHEL
Brett Smith [Fri, 6 Dec 2024 20:02:45 +0000 (15:02 -0500)]
22383: Handle different SSH service name on RHEL

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

2 months ago22383: Prefer plain `sudo` over `sudo su -c`
Brett Smith [Fri, 6 Dec 2024 19:31:47 +0000 (14:31 -0500)]
22383: Prefer plain `sudo` over `sudo su -c`

Improve readability by reducing indirection.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

3 months agoScript to publish R package fails fast and accepts host key
Peter Amstutz [Wed, 13 Nov 2024 23:32:13 +0000 (18:32 -0500)]
Script to publish R package fails fast and accepts host key

refs #22084

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 months agoRun CWL tests against 1.2.1 test suite, no issue #
Peter Amstutz [Thu, 7 Nov 2024 14:27:12 +0000 (09:27 -0500)]
Run CWL tests against 1.2.1 test suite, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 months agoMerge branch '22186-ubuntu-fix'. Refs #22186
Lucas Di Pentima [Wed, 6 Nov 2024 22:01:11 +0000 (19:01 -0300)]
Merge branch '22186-ubuntu-fix'. Refs #22186

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 months ago22186: Moves cloud-init execution to the common script. 22186-ubuntu-fix
Lucas Di Pentima [Wed, 6 Nov 2024 21:53:14 +0000 (18:53 -0300)]
22186: Moves cloud-init execution to the common script.

Also removes unused SSH keys.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 months ago22186: Waits for cloud-init to finish before attempting package installations.
Lucas Di Pentima [Wed, 6 Nov 2024 21:37:26 +0000 (18:37 -0300)]
22186: Waits for cloud-init to finish before attempting package installations.

This is following packer's advice at:
https://developer.hashicorp.com/packer/docs/debugging#issues-installing-ubuntu-packages

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 months agoMerge branch '22186-jenkins-docker-mirror' refs #22186
Peter Amstutz [Wed, 6 Nov 2024 18:01:02 +0000 (13:01 -0500)]
Merge branch '22186-jenkins-docker-mirror' refs #22186

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 months ago22186: Set up registry-mirrors in jenkins-image-with-docker 22186-jenkins-docker-mirror
Peter Amstutz [Wed, 6 Nov 2024 17:49:51 +0000 (12:49 -0500)]
22186: Set up registry-mirrors in jenkins-image-with-docker

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 months ago22186: Update debian base image and consolidate scripts a bit
Peter Amstutz [Wed, 6 Nov 2024 17:45:44 +0000 (12:45 -0500)]
22186: Update debian base image and consolidate scripts a bit

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago22206: Add release task to update third-party package pins
Brett Smith [Mon, 4 Nov 2024 16:01:28 +0000 (11:01 -0500)]
22206: Add release task to update third-party package pins

See <https://dev.arvados.org/projects/arvados/wiki/Release_Checklist>
for more details. Refs #22206.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21909: Update build-package-r to build the package with make
Brett Smith [Thu, 19 Sep 2024 20:31:00 +0000 (16:31 -0400)]
21909: Update build-package-r to build the package with make

This is necessary to generate code and documentation before packaging
everything up. The rest of the changes are just path adjustments now
that the script works out of `sdk/R/` instead of just `sdk/`.
Refs #21909.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21931: Support uploading the arvados-google-api-client gem
Brett Smith [Wed, 3 Jul 2024 14:52:45 +0000 (10:52 -0400)]
21931: Support uploading the arvados-google-api-client gem

Because this is updated rarely and does not follow Arvados versioning,
it is uploaded by a dedicated Jenkins job, not the usual
build-packages-multijob. But that's fine, we can still use the existing
tooling. Refs #21931.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21855: Update RPM filesystem path
Brett Smith [Thu, 20 Jun 2024 20:49:44 +0000 (16:49 -0400)]
21855: Update RPM filesystem path

Since we no longer expect to publish packages for CentOS after it
changed its release model. Refs #21855

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21363: Update name of SSH service
Brett Smith [Fri, 7 Jun 2024 14:58:01 +0000 (10:58 -0400)]
21363: Update name of SSH service

The SSH server service has always been named ssh.service (at least since
Ubuntu 20.04). Some distros provide an sshd.service alias, but Ubuntu
24.04 does not, so don't use it.

Note this makes the command consistent with jenkins-image-common.sh.

Refs #21363.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21363: Add ubuntu2404 to run_upload_packages
Brett Smith [Fri, 7 Jun 2024 13:10:38 +0000 (09:10 -0400)]
21363: Add ubuntu2404 to run_upload_packages

Refs #21363.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

9 months agoMerge branch '21840-twine-upload'
Brett Smith [Tue, 28 May 2024 16:05:40 +0000 (12:05 -0400)]
Merge branch '21840-twine-upload'

Refs #21840.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

9 months ago21840: Use twine for Python uploads
Brett Smith [Tue, 28 May 2024 16:00:47 +0000 (12:00 -0400)]
21840: Use twine for Python uploads

In addition to getting us away from a deprecated setup.py invocation,
this also lets the Python upload work a lot more like the Ruby upload:
you just upload distributions that Jenkins built previously.

Note uploading wheels will require a mirror change in the Jenkins build
script.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

9 months ago21840: Migrate from pipes.quote to shlex.quote
Brett Smith [Tue, 28 May 2024 15:40:30 +0000 (11:40 -0400)]
21840: Migrate from pipes.quote to shlex.quote

To avoid the pipes deprecation in modern Python.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

9 months agoUpdate release process TASKS
Peter Amstutz [Mon, 20 May 2024 13:59:05 +0000 (09:59 -0400)]
Update release process TASKS

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

9 months agoart: print git hash and message when running find-and-associate
Peter Amstutz [Mon, 20 May 2024 13:55:01 +0000 (09:55 -0400)]
art: print git hash and message when running find-and-associate

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoMerge branch '21521-aptly-flock'
Brett Smith [Mon, 29 Apr 2024 17:35:40 +0000 (13:35 -0400)]
Merge branch '21521-aptly-flock'

Refs #21521.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months ago21521: Wrap aptly commands in flock 21521-aptly-flock
Brett Smith [Fri, 26 Apr 2024 20:47:50 +0000 (16:47 -0400)]
21521: Wrap aptly commands in flock

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months ago21521: Make apt upload script less verbose
Brett Smith [Fri, 26 Apr 2024 20:41:55 +0000 (16:41 -0400)]
21521: Make apt upload script less verbose

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months ago21662: Refine distro detection in jenkins-image-common
Brett Smith [Fri, 19 Apr 2024 20:06:05 +0000 (16:06 -0400)]
21662: Refine distro detection in jenkins-image-common

* Use ID_LIKE to identify distros rather than having whole lists.
* Only install buster-backports when we're actually on buster.

Refs #21662.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months agoAdd a banner to the package promotion logging.
Peter Amstutz [Thu, 11 Apr 2024 13:46:24 +0000 (09:46 -0400)]
Add a banner to the package promotion logging.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoFix package publishing script to handle both deb and rpm in one command
Peter Amstutz [Thu, 11 Apr 2024 13:35:11 +0000 (09:35 -0400)]
Fix package publishing script to handle both deb and rpm in one command

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoUse createrepo_c, no issue #
Peter Amstutz [Thu, 11 Apr 2024 13:16:35 +0000 (09:16 -0400)]
Use createrepo_c, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoAdd set-sprint subcommand to 'art', no issue #
Peter Amstutz [Fri, 5 Apr 2024 20:20:37 +0000 (16:20 -0400)]
Add set-sprint subcommand to 'art', no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoUses createrepo_c instead of createrepo. Refs #21616
Lucas Di Pentima [Tue, 2 Apr 2024 12:23:15 +0000 (09:23 -0300)]
Uses createrepo_c instead of createrepo. Refs #21616

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoFixes in git checkout call. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 22:53:17 +0000 (19:53 -0300)]
Fixes in git checkout call. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoFurther fixes on envvar passthrough on packer scripts. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 22:46:48 +0000 (19:46 -0300)]
Further fixes on envvar passthrough on packer scripts. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoPasses arvados git hash as a envvar to the packer script. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 22:13:09 +0000 (19:13 -0300)]
Passes arvados git hash as a envvar to the packer script. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoAllow requesting a specific arvados commit hash for image buildind. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 20:26:40 +0000 (17:26 -0300)]
Allow requesting a specific arvados commit hash for image buildind. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoFixes sysctl call on provisioning scripts. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 18:10:43 +0000 (15:10 -0300)]
Fixes sysctl call on provisioning scripts. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoApplies settings from sysctl conf file. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 17:53:15 +0000 (14:53 -0300)]
Applies settings from sysctl conf file. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoUses Debian Bullseye as a base for wb2 jenkins image. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 14:36:57 +0000 (11:36 -0300)]
Uses Debian Bullseye as a base for wb2 jenkins image. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoUpdates WB2 image building script for Jenkins. Refs #21461
Lucas Di Pentima [Wed, 21 Feb 2024 14:28:32 +0000 (11:28 -0300)]
Updates WB2 image building script for Jenkins. Refs #21461

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months agoincrease fs.inotify.max_user_watches to run wb2 tests
Peter Amstutz [Tue, 16 Jan 2024 20:33:17 +0000 (15:33 -0500)]
increase fs.inotify.max_user_watches to run wb2 tests

refs #21393

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months agoChanges the SSH key type that packer creates to communicate with nodes.
Lucas Di Pentima [Fri, 12 Jan 2024 18:17:58 +0000 (15:17 -0300)]
Changes the SSH key type that packer creates to communicate with nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

This is a test to see if the packer-build-provision-testing-image pipeline
works with Debian 12 & Ubuntu 22.04, that seem to have deprecated RSA SSH
key types for authentication.

No issue #

13 months agoReverts previous change as it wasn't needed. No issue #
Lucas Di Pentima [Fri, 12 Jan 2024 15:51:00 +0000 (12:51 -0300)]
Reverts previous change as it wasn't needed. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months agoMoves packer installation to the common script. No issue #
Lucas Di Pentima [Thu, 11 Jan 2024 21:50:18 +0000 (18:50 -0300)]
Moves packer installation to the common script. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

14 months agoSets Debian image to v11. Refs #21330
Lucas Di Pentima [Tue, 2 Jan 2024 22:16:19 +0000 (19:16 -0300)]
Sets Debian image to v11. Refs #21330

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

14 months ago20846: Add debian12 and ubuntu2204 upload targets
Brett Smith [Fri, 29 Dec 2023 14:59:38 +0000 (09:59 -0500)]
20846: Add debian12 and ubuntu2204 upload targets

I also checked testing_to_stable_publish_packages.sh and
upload_packages_testing_to_stable.sh but there doesn't seem to be
anything to update in those. They seem to default to working on
deb-based distributions just fine.

Refs #20846.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

14 months agoAdd task to update public-facing clusters to stable release
Peter Amstutz [Fri, 8 Dec 2023 15:21:14 +0000 (10:21 -0500)]
Add task to update public-facing clusters to stable release

refs #21089

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

15 months agoInstalls 'createrepo-c' package instead of the deprecated 'createrepo'.
Lucas Di Pentima [Tue, 7 Nov 2023 19:49:25 +0000 (16:49 -0300)]
Installs 'createrepo-c' package instead of the deprecated 'createrepo'.
No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

15 months agoSets Debian image to v11. No issue #
Lucas Di Pentima [Tue, 7 Nov 2023 19:38:05 +0000 (16:38 -0300)]
Sets Debian image to v11. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

17 months agoAdd rocky8 to publish package script
Peter Amstutz [Thu, 21 Sep 2023 16:57:02 +0000 (12:57 -0400)]
Add rocky8 to publish package script

refs #20769

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

19 months agoMerge branch '20797-rh8-packages'
Brett Smith [Tue, 1 Aug 2023 21:29:32 +0000 (17:29 -0400)]
Merge branch '20797-rh8-packages'

Refs #20797. Closes #20807.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

19 months ago20797: Add support for rocky8 upload target 20797-rh8-packages
Brett Smith [Tue, 1 Aug 2023 18:52:49 +0000 (14:52 -0400)]
20797: Add support for rocky8 upload target

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months agoAccept "resolves #" (equivalent to "closes #"), update help message.
Tom Clegg [Thu, 1 Jun 2023 13:31:53 +0000 (09:31 -0400)]
Accept "resolves #" (equivalent to "closes #"), update help message.

no issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19920-move-unassigned-open-tickets'. Closes #19920
Lucas Di Pentima [Wed, 15 Feb 2023 15:52:40 +0000 (12:52 -0300)]
Merge branch '19920-move-unassigned-open-tickets'. Closes #19920

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoAdds worker pool to speed up processing lots of issues. Refs #19920 19920-move-unassigned-open-tickets
Lucas Di Pentima [Wed, 25 Jan 2023 21:29:47 +0000 (18:29 -0300)]
Adds worker pool to speed up processing lots of issues. Refs #19920

The single-threaded version was processing issues at about 1 per second,
too slow to move lots of issues in one go.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoIgnore .gitignore files for licensing. No issue #
Lucas Di Pentima [Wed, 25 Jan 2023 18:55:08 +0000 (15:55 -0300)]
Ignore .gitignore files for licensing. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoAdds 'associate-orphans' subcommand. Refs #19920
Lucas Di Pentima [Wed, 25 Jan 2023 18:38:35 +0000 (15:38 -0300)]
Adds 'associate-orphans' subcommand. Refs #19920

This subcommand takes a project name, a release ID and searches for
every open issue that's not assigned to a version and release,
associating it with the given release ID.
There's a dry-run parameter to allow the user to double check before
committing.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoAdds release filtering on issues. Refs #19920
Lucas Di Pentima [Wed, 25 Jan 2023 14:44:03 +0000 (11:44 -0300)]
Adds release filtering on issues. Refs #19920

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoAdds gitignore. Refs #19920
Lucas Di Pentima [Thu, 19 Jan 2023 14:32:18 +0000 (11:32 -0300)]
Adds gitignore. Refs #19920

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoArvados release tool maintenance, refs #19647
Peter Amstutz [Tue, 13 Dec 2022 15:35:05 +0000 (10:35 -0500)]
Arvados release tool maintenance, refs #19647

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFixes installer var replacement. No issue #
Lucas Di Pentima [Tue, 6 Dec 2022 23:13:33 +0000 (00:13 +0100)]
Fixes installer var replacement. No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoFix go.mod confusion.
Tom Clegg [Thu, 10 Nov 2022 17:08:07 +0000 (12:08 -0500)]
Fix go.mod confusion.

refs #19572

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoUpdate release process a bit
Peter Amstutz [Fri, 5 Aug 2022 21:05:01 +0000 (17:05 -0400)]
Update release process a bit

refs #19330

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoTweak to the order of release tasks: the reference to the default
Ward Vandewege [Wed, 1 Jun 2022 17:34:51 +0000 (13:34 -0400)]
Tweak to the order of release tasks: the reference to the default
package that the installer uses should only be updated after that
package is published.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoSalt installer change: standardize on putting the certs directory under
Ward Vandewege [Wed, 1 Jun 2022 14:22:49 +0000 (10:22 -0400)]
Salt installer change: standardize on putting the certs directory under
the local_config_dir directory, this simplifies things.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19092-upload-crunchstat_summary-to-pypi'
Ward Vandewege [Wed, 25 May 2022 15:41:53 +0000 (11:41 -0400)]
Merge branch '19092-upload-crunchstat_summary-to-pypi'

closes #19092

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19092: upload crunchstat_summary to Pypi 19092-upload-crunchstat_summary-to-pypi
Ward Vandewege [Wed, 25 May 2022 14:33:15 +0000 (10:33 -0400)]
19092: upload crunchstat_summary to Pypi

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoTweak fuse.conf after the fuse package is installed, not before.
Ward Vandewege [Fri, 20 May 2022 19:55:54 +0000 (15:55 -0400)]
Tweak fuse.conf after the fuse package is installed, not before.

refs #15370

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoFor running our tests, FUSE must be configured with the
Ward Vandewege [Fri, 20 May 2022 19:40:43 +0000 (15:40 -0400)]
For running our tests, FUSE must be configured with the
'user_allow_other' option enabled for Crunch to set up Keep mounts that
are readable by containers.

refs #15370

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoBump the buster base image for our test image.
Ward Vandewege [Fri, 20 May 2022 19:32:49 +0000 (15:32 -0400)]
Bump the buster base image for our test image.

refs #15370

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd docker to our tests image.
Ward Vandewege [Fri, 20 May 2022 16:38:58 +0000 (12:38 -0400)]
Add docker to our tests image.

refs #15370

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd a convenient way to disable our git hooks via an environment
Ward Vandewege [Thu, 19 May 2022 17:41:29 +0000 (13:41 -0400)]
Add a convenient way to disable our git hooks via an environment
variable.

refs #18872

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoOur base test image installs the correct version of Go (detected in the
Ward Vandewege [Wed, 13 Apr 2022 16:53:25 +0000 (12:53 -0400)]
Our base test image installs the correct version of Go (detected in the
usual way from the Arvados source tree).

Check out the git tree using the git protocol, not http.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>