arvados-dev.git
3 years agoMerge branch '17535-test-provision-jenkins'
Javier Bértoli [Thu, 5 Aug 2021 16:08:36 +0000 (13:08 -0300)]
Merge branch '17535-test-provision-jenkins'

refs #17535
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoUpdate run_upload_packages.py: add debian 11 (bullseye), remove some no
Ward Vandewege [Thu, 29 Jul 2021 20:49:02 +0000 (16:49 -0400)]
Update run_upload_packages.py: add debian 11 (bullseye), remove some no
longer supported distributions.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd script to create new release repositories.
Ward Vandewege [Thu, 29 Jul 2021 19:13:38 +0000 (15:13 -0400)]
Add script to create new release repositories.

refs #17949

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17535: missing command
Javier Bértoli [Thu, 22 Jul 2021 20:06:01 +0000 (17:06 -0300)]
17535: missing command

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: typo
Javier Bértoli [Thu, 22 Jul 2021 19:56:13 +0000 (16:56 -0300)]
17535: typo

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Use the backport line only in the Debian family
Javier Bértoli [Thu, 22 Jul 2021 19:46:56 +0000 (16:46 -0300)]
17535: Use the backport line only in the Debian family

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: typo
Javier Bértoli [Thu, 22 Jul 2021 19:36:26 +0000 (16:36 -0300)]
17535: typo

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Allow to specify distro as part of the image name
Javier Bértoli [Thu, 22 Jul 2021 19:16:57 +0000 (16:16 -0300)]
17535: Allow to specify distro as part of the image name

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Add a basic instance template for provision testing
Javier Bértoli [Thu, 22 Jul 2021 16:52:41 +0000 (13:52 -0300)]
17535: Add a basic instance template for provision testing

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Re-add netcat to the images
Javier Bértoli [Thu, 22 Jul 2021 16:11:35 +0000 (13:11 -0300)]
17535: Re-add netcat to the images

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Add missing Debian's pre-install command
Javier Bértoli [Thu, 22 Jul 2021 15:43:31 +0000 (12:43 -0300)]
17535: Add missing Debian's pre-install command

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17535: Modify the image-common script so it can work both in centos/debian
Javier Bértoli [Thu, 22 Jul 2021 15:37:19 +0000 (12:37 -0300)]
17535: Modify the image-common script so it can work both in centos/debian

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoUpdate check-copyright-headers.rb so it can handle the first push to an
Ward Vandewege [Fri, 16 Jul 2021 18:05:22 +0000 (14:05 -0400)]
Update check-copyright-headers.rb so it can handle the first push to an
empty repository.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAllow building a wb2 jenkins image from a specific wb2 git commit.
Ward Vandewege [Tue, 6 Jul 2021 21:23:15 +0000 (17:23 -0400)]
Allow building a wb2 jenkins image from a specific wb2 git commit.

refs #17782

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17844-create-an-image-for-arvados-formula-testing'
Javier Bértoli [Thu, 1 Jul 2021 20:22:15 +0000 (17:22 -0300)]
Merge branch '17844-create-an-image-for-arvados-formula-testing'

closes #17844
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoThe bleeding edge branch on the arvados repo is now called 'main'.
Ward Vandewege [Thu, 1 Jul 2021 18:48:53 +0000 (14:48 -0400)]
The bleeding edge branch on the arvados repo is now called 'main'.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17844: use curl to get files
Javier Bértoli [Thu, 1 Jul 2021 17:53:09 +0000 (14:53 -0300)]
17844: use curl to get files

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17844: use curl to get files
Javier Bértoli [Thu, 1 Jul 2021 17:39:05 +0000 (14:39 -0300)]
17844: use curl to get files

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoBump packer version to 1.7.2. Checkout arvados-dev from the new URL on
Ward Vandewege [Thu, 1 Jul 2021 15:25:47 +0000 (11:25 -0400)]
Bump packer version to 1.7.2. Checkout arvados-dev from the new URL on
github.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17844: add a template to build an image to test the Saltstack's arvados-formula
Javier Bértoli [Wed, 30 Jun 2021 21:25:32 +0000 (18:25 -0300)]
17844: add a template to build an image to test the Saltstack's arvados-formula

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoUpdate our commit hooks to use 'main' as the bleeding edge branch.
Ward Vandewege [Thu, 17 Jun 2021 14:24:50 +0000 (10:24 -0400)]
Update our commit hooks to use 'main' as the bleeding edge branch.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoIn preparation of the migration to 'main', make a copy of the commit
Ward Vandewege [Thu, 17 Jun 2021 13:44:25 +0000 (09:44 -0400)]
In preparation of the migration to 'main', make a copy of the commit
hooks that reference the old 'master' branch, so that we can migrate
gradually.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPreseed the run-tests.sh cache to make test runs faster (bugfix).
Ward Vandewege [Fri, 7 May 2021 15:56:09 +0000 (11:56 -0400)]
Preseed the run-tests.sh cache to make test runs faster (bugfix).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPreseed the run-tests.sh cache to make test runs faster.
Ward Vandewege [Thu, 6 May 2021 16:01:47 +0000 (12:01 -0400)]
Preseed the run-tests.sh cache to make test runs faster.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe need build-essential because of gihub.com/msteinert/pam which
Ward Vandewege [Wed, 5 May 2021 15:31:45 +0000 (11:31 -0400)]
We need build-essential because of gihub.com/msteinert/pam which
apparently uses cgo...

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPermissions fix.
Ward Vandewege [Wed, 5 May 2021 15:26:22 +0000 (11:26 -0400)]
Permissions fix.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump the version of the debian buster image we base our images on.
Ward Vandewege [Wed, 5 May 2021 15:19:25 +0000 (11:19 -0400)]
Bump the version of the debian buster image we base our images on.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump go to 1.16.3.
Ward Vandewege [Wed, 5 May 2021 15:16:16 +0000 (11:16 -0400)]
Bump go to 1.16.3.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoCostanalyzer has been moved to the arvados codebase, accessible via
Ward Vandewege [Thu, 18 Mar 2021 01:23:32 +0000 (21:23 -0400)]
Costanalyzer has been moved to the arvados codebase, accessible via
arvados-client.

refs #16950

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWhen a package can't be published, print which distribution the error is
Ward Vandewege [Thu, 25 Feb 2021 18:22:42 +0000 (13:22 -0500)]
When a package can't be published, print which distribution the error is
with.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfix for the `testing_to_stable_publish_packages.sh` script to reflect
Ward Vandewege [Thu, 25 Feb 2021 18:13:00 +0000 (13:13 -0500)]
Bugfix for the `testing_to_stable_publish_packages.sh` script to reflect
our switch to aptly.

Some refactoring to remove exitcode nonsense.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py: ignore errors writing timestamp files, but print
Ward Vandewege [Fri, 29 Jan 2021 15:41:55 +0000 (10:41 -0500)]
run_upload_packages.py: ignore errors writing timestamp files, but print
out a warning. We run this script from emphemeral machines now, so the
timestamp mechanism is not used.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py: give createrepo a cache directory for its
Ward Vandewege [Fri, 29 Jan 2021 15:26:15 +0000 (10:26 -0500)]
run_upload_packages.py: give createrepo a cache directory for its
package checksums, and tell it to use a simple file size/mtime
comparison instead of recalculating the hashes, when adding a package to
the repo.

This should speed up rpm package publishing.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py should make sure the directory it tries to put a
Ward Vandewege [Thu, 28 Jan 2021 22:48:12 +0000 (17:48 -0500)]
run_upload_packages.py should make sure the directory it tries to put a
timestamp file in exists (bugfix).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py should make sure the directory it tries to put a
Ward Vandewege [Thu, 28 Jan 2021 22:30:55 +0000 (17:30 -0500)]
run_upload_packages.py should make sure the directory it tries to put a
timestamp file in exists.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix version number regex to allow 4 number positions & values over 9
Peter Amstutz [Thu, 28 Jan 2021 16:51:43 +0000 (11:51 -0500)]
Fix version number regex to allow 4 number positions & values over 9

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16916-move-wb2-tests-to-jenkins-satellites'
Ward Vandewege [Fri, 8 Jan 2021 20:06:49 +0000 (15:06 -0500)]
Merge branch '16916-move-wb2-tests-to-jenkins-satellites'

closes #16916

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16916: add packer files for WB2 test image 16916-move-wb2-tests-to-jenkins-satellites
Ward Vandewege [Thu, 7 Jan 2021 22:47:42 +0000 (17:47 -0500)]
16916: add packer files for WB2 test image

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfixes for packaging-move-dev-to-attic.py
Ward Vandewege [Mon, 4 Jan 2021 21:04:47 +0000 (16:04 -0500)]
Bugfixes for packaging-move-dev-to-attic.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17219-refactor-attic-scripts'
Ward Vandewege [Mon, 4 Jan 2021 18:55:52 +0000 (13:55 -0500)]
Merge branch '17219-refactor-attic-scripts'

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAddress review comments. 17219-refactor-attic-scripts
Ward Vandewege [Mon, 4 Jan 2021 18:55:21 +0000 (13:55 -0500)]
Address review comments.

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agopackaging-move-dev-to-attic.py: remove hardcoded list of supported
Ward Vandewege [Mon, 4 Jan 2021 18:27:21 +0000 (13:27 -0500)]
packaging-move-dev-to-attic.py: remove hardcoded list of supported
distributions, instead validate the distribution name by pattern.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd packaging-move-dev-to-attic.py, which now supports aptly. This is a
Ward Vandewege [Mon, 4 Jan 2021 18:11:33 +0000 (13:11 -0500)]
Add packaging-move-dev-to-attic.py, which now supports aptly. This is a
modified version of an old internal script that supported freight.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoIterate over packages one by one in run_upload_packages.py
Ward Vandewege [Wed, 23 Dec 2020 03:05:52 +0000 (22:05 -0500)]
Iterate over packages one by one in run_upload_packages.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake the run_upload_packages.py script more verbose.
Ward Vandewege [Wed, 23 Dec 2020 02:27:43 +0000 (21:27 -0500)]
Make the run_upload_packages.py script more verbose.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not try to add packages to the repository that are already in it.
Ward Vandewege [Wed, 23 Dec 2020 01:45:43 +0000 (20:45 -0500)]
Do not try to add packages to the repository that are already in it.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17218-repository-update'
Ward Vandewege [Tue, 22 Dec 2020 22:09:18 +0000 (17:09 -0500)]
Merge branch '17218-repository-update'

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe've made a change in the structure of our apt repositories. 17218-repository-update
Ward Vandewege [Tue, 22 Dec 2020 19:56:34 +0000 (14:56 -0500)]
We've made a change in the structure of our apt repositories.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake it possible to deploy a single node.
Ward Vandewege [Wed, 16 Dec 2020 19:19:50 +0000 (14:19 -0500)]
Make it possible to deploy a single node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not run arv-keepdocker on every shell node, just on the main shell
Ward Vandewege [Wed, 16 Dec 2020 19:04:40 +0000 (14:04 -0500)]
Do not run arv-keepdocker on every shell node, just on the main shell
node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
Ward Vandewege [Wed, 16 Dec 2020 18:50:09 +0000 (13:50 -0500)]
run-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
when running apt-get -y upgrade. This caused a hang on install/upgrade
of the iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoNo need to sudo to run salt.
Ward Vandewege [Wed, 16 Dec 2020 14:54:33 +0000 (09:54 -0500)]
No need to sudo to run salt.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake sure to have DEBIAN_FRONTEND=noninteractive set when running
Ward Vandewege [Tue, 15 Dec 2020 15:33:24 +0000 (10:33 -0500)]
Make sure to have DEBIAN_FRONTEND=noninteractive set when running
apt-get -y upgrade. This caused a hang on install/upgrade of the
iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17013-upload-wheels-to-pypi' into master
Ward Vandewege [Mon, 23 Nov 2020 16:20:52 +0000 (11:20 -0500)]
Merge branch '17013-upload-wheels-to-pypi' into master

closes #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17013: also upload wheels to PyPI. To make the wheels universal, we 17013-upload-wheels-to-pypi
Ward Vandewege [Mon, 23 Nov 2020 15:27:42 +0000 (10:27 -0500)]
17013: also upload wheels to PyPI. To make the wheels universal, we
       still need a change to setup.cfg in each of our Python
       component directories.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe use Python3 now, exclusively.
Ward Vandewege [Mon, 23 Nov 2020 14:27:46 +0000 (09:27 -0500)]
We use Python3 now, exclusively.

refs #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd extra validation to upload_packages_testing_to_stable.sh: only
Ward Vandewege [Wed, 18 Nov 2020 19:37:48 +0000 (14:37 -0500)]
Add extra validation to upload_packages_testing_to_stable.sh: only
numbered releases should be allowed.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '17086-add-focal-fossa' into master
Ward Vandewege [Sat, 7 Nov 2020 13:46:24 +0000 (08:46 -0500)]
Merge branch '17086-add-focal-fossa' into master

refs #17086

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago17086: add support for Ubuntu 20.04, Focal Fossa. 17086-add-focal-fossa
Ward Vandewege [Thu, 5 Nov 2020 19:29:53 +0000 (14:29 -0500)]
17086: add support for Ubuntu 20.04, Focal Fossa.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoFix link to the Coding Standards wiki page.
Ward Vandewege [Wed, 21 Oct 2020 18:15:47 +0000 (14:15 -0400)]
Fix link to the Coding Standards wiki page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing, taking release clients into account.
Ward Vandewege [Fri, 16 Oct 2020 15:14:19 +0000 (11:14 -0400)]
Tweak version number parsing, taking release clients into account.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing (again).
Ward Vandewege [Fri, 16 Oct 2020 14:57:49 +0000 (10:57 -0400)]
Tweak version number parsing (again).

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing.
Ward Vandewege [Fri, 16 Oct 2020 14:47:21 +0000 (10:47 -0400)]
Tweak version number parsing.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoHandle the fact that ~ is not a valid character in docker image tags,
Ward Vandewege [Tue, 13 Oct 2020 19:11:18 +0000 (15:11 -0400)]
Handle the fact that ~ is not a valid character in docker image tags,
apparently.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy-salt.sh: remove superfluous definitions.
Ward Vandewege [Wed, 30 Sep 2020 14:19:14 +0000 (10:19 -0400)]
run-deploy-salt.sh: remove superfluous definitions.

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16921-run-deploy-salt' into master
Ward Vandewege [Wed, 30 Sep 2020 14:17:28 +0000 (10:17 -0400)]
Merge branch '16921-run-deploy-salt' into master

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the 16921-run-deploy-salt
Ward Vandewege [Tue, 29 Sep 2020 19:50:47 +0000 (15:50 -0400)]
16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the
       list of machines to be deployed, and also to issue the commands
       to them.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: some tweaks to support tordo. A bit of cleanup too.
Ward Vandewege [Tue, 29 Sep 2020 13:47:26 +0000 (09:47 -0400)]
run-deploy.sh: some tweaks to support tordo. A bit of cleanup too.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDoesn't need to have -p2222 any more
Peter Amstutz [Fri, 25 Sep 2020 14:31:15 +0000 (10:31 -0400)]
Doesn't need to have -p2222 any more

refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate build-package-r.sh to use r.arvados.org refs #16885
Peter Amstutz [Fri, 25 Sep 2020 14:22:17 +0000 (10:22 -0400)]
Update build-package-r.sh to use r.arvados.org refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoTweak after enabling `set -e`: handle grep properly. Also be more
Ward Vandewege [Thu, 3 Sep 2020 21:12:06 +0000 (17:12 -0400)]
Tweak after enabling `set -e`: handle grep properly. Also be more
specific about the version number that is retrieved.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoNo more tagging as 'latest' for our Docker arvados/jobs image
Ward Vandewege [Sat, 22 Aug 2020 00:02:58 +0000 (20:02 -0400)]
No more tagging as 'latest' for our Docker arvados/jobs image

refs #16565

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWe need a `git pull` after switching to the master branch, to make sure we see
Ward Vandewege [Wed, 19 Aug 2020 23:43:27 +0000 (19:43 -0400)]
We need a `git pull` after switching to the master branch, to make sure we see
the complete lists of branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure we don't accidentally delete our release branches.
Ward Vandewege [Wed, 19 Aug 2020 23:30:28 +0000 (19:30 -0400)]
Make sure we don't accidentally delete our release branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake run-deploy.sh run with set -e, always. We need to know when it
Ward Vandewege [Wed, 19 Aug 2020 02:19:36 +0000 (22:19 -0400)]
Make run-deploy.sh run with set -e, always. We need to know when it
fails.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoThe python pam and node manager packages have been removed, do not try
Ward Vandewege [Tue, 18 Aug 2020 20:10:41 +0000 (16:10 -0400)]
The python pam and node manager packages have been removed, do not try
to upload them.

refs #16580

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoRemove unattended-upgrades from our Jenkins build images, they interfere
Ward Vandewege [Tue, 18 Aug 2020 19:04:57 +0000 (15:04 -0400)]
Remove unattended-upgrades from our Jenkins build images, they interfere
with job startup when we want to install something via apt.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: when there is a shell node, use that to figure out the
Ward Vandewege [Thu, 13 Aug 2020 23:28:12 +0000 (19:28 -0400)]
run-deploy.sh: when there is a shell node, use that to figure out the
version of the arvados/jobs image to install.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agorefactoring: bring the name for the generated federation and k8s tests
Ward Vandewege [Fri, 31 Jul 2020 01:00:36 +0000 (21:00 -0400)]
refactoring: bring the name for the generated federation and k8s tests
images in line with the other jenkins images.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBump version of the Debian Buster image we use.
Ward Vandewege [Fri, 31 Jul 2020 00:46:59 +0000 (20:46 -0400)]
Bump version of the Debian Buster image we use.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorefactoring: rename packer templates and the resulting images to make it
Ward Vandewege [Fri, 31 Jul 2020 00:30:20 +0000 (20:30 -0400)]
refactoring: rename packer templates and the resulting images to make it
more clear what they are for.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16616-move-jenkins-packer-code'
Ward Vandewege [Thu, 30 Jul 2020 17:19:36 +0000 (13:19 -0400)]
Merge branch '16616-move-jenkins-packer-code'

refs #16616

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16616: move jenkins packer templates here. 16616-move-jenkins-packer-code
Ward Vandewege [Tue, 28 Jul 2020 17:50:12 +0000 (13:50 -0400)]
16616: move jenkins packer templates here.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBugfix for the check-copyright-headers.rb script: dtrt with symlinks.
Ward Vandewege [Fri, 19 Jun 2020 21:35:40 +0000 (17:35 -0400)]
Bugfix for the check-copyright-headers.rb script: dtrt with symlinks.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAnother bugfix for the check-copyright-headers.rb script.
Ward Vandewege [Tue, 16 Jun 2020 21:25:31 +0000 (17:25 -0400)]
Another bugfix for the check-copyright-headers.rb script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDo not print a noisy warning when there is no .licenseignore file.
Ward Vandewege [Mon, 15 Jun 2020 19:04:32 +0000 (15:04 -0400)]
Do not print a noisy warning when there is no .licenseignore file.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure to convert the globs in .licenseignore into regular
Ward Vandewege [Mon, 15 Jun 2020 18:57:55 +0000 (14:57 -0400)]
Make sure to convert the globs in .licenseignore into regular
expressions.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoReplace check-copyright-headers.sh with check-copyright-headers.sh.
Ward Vandewege [Mon, 15 Jun 2020 01:55:50 +0000 (21:55 -0400)]
Replace check-copyright-headers.sh with check-copyright-headers.sh.
Rename coding-standards.sh to coding-standards.rb, it was already a Ruby
script. All git hooks are now Ruby.

Fix bugs in check-copyright-headers: no longer print a 'fatal' error
when a branch is first pushed. No longer skip checking of new files in
some circumstances (git `commit` object instead of `blob` object).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd a script to clean up old compute node images on Azure.
Ward Vandewege [Thu, 7 May 2020 01:30:04 +0000 (21:30 -0400)]
Add a script to clean up old compute node images on Azure.

closes #16418

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoLanguage tweaks to COPYING and README, and url updates in README.
Ward Vandewege [Thu, 7 May 2020 01:05:27 +0000 (21:05 -0400)]
Language tweaks to COPYING and README, and url updates in README.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoInitial commit of the costanalyzer code.
Ward Vandewege [Thu, 30 Apr 2020 14:07:48 +0000 (10:07 -0400)]
Initial commit of the costanalyzer code.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoImprove grepping for the docker image to avoid false positives.
Ward Vandewege [Fri, 24 Apr 2020 01:06:32 +0000 (21:06 -0400)]
Improve grepping for the docker image to avoid false positives.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago* run-deploy.sh: fix logic bug in arv-keepdocker call
Ward Vandewege [Sat, 21 Mar 2020 01:45:03 +0000 (21:45 -0400)]
* run-deploy.sh: fix logic bug in arv-keepdocker call
* get version from python3-arvados-cwl-runner package

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoA few improvements to run-deploy.sh:
Ward Vandewege [Fri, 20 Mar 2020 21:12:39 +0000 (17:12 -0400)]
A few improvements to run-deploy.sh:

* deal with those annoying `Ignoring curb-0.9.10 because its extensions
are not built.  Try: gem pristine curb --version 0.9.10` warnings so we
don't display them, for real this time.

* prefix output of remote commands with the name of the host. That makes
determining which host is having issues easier when run with
--concurrency > 1

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd make-package-list script
Peter Amstutz [Fri, 14 Feb 2020 16:16:00 +0000 (16:16 +0000)]
Add make-package-list script

4 years agoCan unpublish an individual package
Peter Amstutz [Fri, 14 Feb 2020 15:56:26 +0000 (15:56 +0000)]
Can unpublish an individual package

4 years agoFix find pattern for debs
Peter Amstutz [Mon, 10 Feb 2020 19:43:10 +0000 (14:43 -0500)]
Fix find pattern for debs

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAdd unpublish script for removing from testing repo
Peter Amstutz [Mon, 10 Feb 2020 19:35:56 +0000 (14:35 -0500)]
Add unpublish script for removing from testing repo

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSimplify the deploy script (and fix a bug around detection of installed
Ward Vandewege [Wed, 5 Feb 2020 21:37:06 +0000 (16:37 -0500)]
Simplify the deploy script (and fix a bug around detection of installed
docker images).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoRun docker tag latest on the node from which we will run arv-keepdocker
Ward Vandewege [Wed, 5 Feb 2020 21:24:02 +0000 (16:24 -0500)]
Run docker tag latest on the node from which we will run arv-keepdocker
tag latest.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>