arvados-dev.git
3 years agoIn preparation of the migration to 'main', make a copy of the commit
Ward Vandewege [Thu, 17 Jun 2021 13:44:25 +0000 (09:44 -0400)]
In preparation of the migration to 'main', make a copy of the commit
hooks that reference the old 'master' branch, so that we can migrate
gradually.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPreseed the run-tests.sh cache to make test runs faster (bugfix).
Ward Vandewege [Fri, 7 May 2021 15:56:09 +0000 (11:56 -0400)]
Preseed the run-tests.sh cache to make test runs faster (bugfix).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPreseed the run-tests.sh cache to make test runs faster.
Ward Vandewege [Thu, 6 May 2021 16:01:47 +0000 (12:01 -0400)]
Preseed the run-tests.sh cache to make test runs faster.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe need build-essential because of gihub.com/msteinert/pam which
Ward Vandewege [Wed, 5 May 2021 15:31:45 +0000 (11:31 -0400)]
We need build-essential because of gihub.com/msteinert/pam which
apparently uses cgo...

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPermissions fix.
Ward Vandewege [Wed, 5 May 2021 15:26:22 +0000 (11:26 -0400)]
Permissions fix.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump the version of the debian buster image we base our images on.
Ward Vandewege [Wed, 5 May 2021 15:19:25 +0000 (11:19 -0400)]
Bump the version of the debian buster image we base our images on.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump go to 1.16.3.
Ward Vandewege [Wed, 5 May 2021 15:16:16 +0000 (11:16 -0400)]
Bump go to 1.16.3.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoCostanalyzer has been moved to the arvados codebase, accessible via
Ward Vandewege [Thu, 18 Mar 2021 01:23:32 +0000 (21:23 -0400)]
Costanalyzer has been moved to the arvados codebase, accessible via
arvados-client.

refs #16950

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWhen a package can't be published, print which distribution the error is
Ward Vandewege [Thu, 25 Feb 2021 18:22:42 +0000 (13:22 -0500)]
When a package can't be published, print which distribution the error is
with.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfix for the `testing_to_stable_publish_packages.sh` script to reflect
Ward Vandewege [Thu, 25 Feb 2021 18:13:00 +0000 (13:13 -0500)]
Bugfix for the `testing_to_stable_publish_packages.sh` script to reflect
our switch to aptly.

Some refactoring to remove exitcode nonsense.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py: ignore errors writing timestamp files, but print
Ward Vandewege [Fri, 29 Jan 2021 15:41:55 +0000 (10:41 -0500)]
run_upload_packages.py: ignore errors writing timestamp files, but print
out a warning. We run this script from emphemeral machines now, so the
timestamp mechanism is not used.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py: give createrepo a cache directory for its
Ward Vandewege [Fri, 29 Jan 2021 15:26:15 +0000 (10:26 -0500)]
run_upload_packages.py: give createrepo a cache directory for its
package checksums, and tell it to use a simple file size/mtime
comparison instead of recalculating the hashes, when adding a package to
the repo.

This should speed up rpm package publishing.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py should make sure the directory it tries to put a
Ward Vandewege [Thu, 28 Jan 2021 22:48:12 +0000 (17:48 -0500)]
run_upload_packages.py should make sure the directory it tries to put a
timestamp file in exists (bugfix).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun_upload_packages.py should make sure the directory it tries to put a
Ward Vandewege [Thu, 28 Jan 2021 22:30:55 +0000 (17:30 -0500)]
run_upload_packages.py should make sure the directory it tries to put a
timestamp file in exists.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix version number regex to allow 4 number positions & values over 9
Peter Amstutz [Thu, 28 Jan 2021 16:51:43 +0000 (11:51 -0500)]
Fix version number regex to allow 4 number positions & values over 9

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16916-move-wb2-tests-to-jenkins-satellites'
Ward Vandewege [Fri, 8 Jan 2021 20:06:49 +0000 (15:06 -0500)]
Merge branch '16916-move-wb2-tests-to-jenkins-satellites'

closes #16916

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16916: add packer files for WB2 test image 16916-move-wb2-tests-to-jenkins-satellites
Ward Vandewege [Thu, 7 Jan 2021 22:47:42 +0000 (17:47 -0500)]
16916: add packer files for WB2 test image

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfixes for packaging-move-dev-to-attic.py
Ward Vandewege [Mon, 4 Jan 2021 21:04:47 +0000 (16:04 -0500)]
Bugfixes for packaging-move-dev-to-attic.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17219-refactor-attic-scripts'
Ward Vandewege [Mon, 4 Jan 2021 18:55:52 +0000 (13:55 -0500)]
Merge branch '17219-refactor-attic-scripts'

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAddress review comments. 17219-refactor-attic-scripts
Ward Vandewege [Mon, 4 Jan 2021 18:55:21 +0000 (13:55 -0500)]
Address review comments.

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agopackaging-move-dev-to-attic.py: remove hardcoded list of supported
Ward Vandewege [Mon, 4 Jan 2021 18:27:21 +0000 (13:27 -0500)]
packaging-move-dev-to-attic.py: remove hardcoded list of supported
distributions, instead validate the distribution name by pattern.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd packaging-move-dev-to-attic.py, which now supports aptly. This is a
Ward Vandewege [Mon, 4 Jan 2021 18:11:33 +0000 (13:11 -0500)]
Add packaging-move-dev-to-attic.py, which now supports aptly. This is a
modified version of an old internal script that supported freight.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoIterate over packages one by one in run_upload_packages.py
Ward Vandewege [Wed, 23 Dec 2020 03:05:52 +0000 (22:05 -0500)]
Iterate over packages one by one in run_upload_packages.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake the run_upload_packages.py script more verbose.
Ward Vandewege [Wed, 23 Dec 2020 02:27:43 +0000 (21:27 -0500)]
Make the run_upload_packages.py script more verbose.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not try to add packages to the repository that are already in it.
Ward Vandewege [Wed, 23 Dec 2020 01:45:43 +0000 (20:45 -0500)]
Do not try to add packages to the repository that are already in it.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17218-repository-update'
Ward Vandewege [Tue, 22 Dec 2020 22:09:18 +0000 (17:09 -0500)]
Merge branch '17218-repository-update'

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe've made a change in the structure of our apt repositories. 17218-repository-update
Ward Vandewege [Tue, 22 Dec 2020 19:56:34 +0000 (14:56 -0500)]
We've made a change in the structure of our apt repositories.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake it possible to deploy a single node.
Ward Vandewege [Wed, 16 Dec 2020 19:19:50 +0000 (14:19 -0500)]
Make it possible to deploy a single node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not run arv-keepdocker on every shell node, just on the main shell
Ward Vandewege [Wed, 16 Dec 2020 19:04:40 +0000 (14:04 -0500)]
Do not run arv-keepdocker on every shell node, just on the main shell
node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
Ward Vandewege [Wed, 16 Dec 2020 18:50:09 +0000 (13:50 -0500)]
run-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
when running apt-get -y upgrade. This caused a hang on install/upgrade
of the iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoNo need to sudo to run salt.
Ward Vandewege [Wed, 16 Dec 2020 14:54:33 +0000 (09:54 -0500)]
No need to sudo to run salt.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake sure to have DEBIAN_FRONTEND=noninteractive set when running
Ward Vandewege [Tue, 15 Dec 2020 15:33:24 +0000 (10:33 -0500)]
Make sure to have DEBIAN_FRONTEND=noninteractive set when running
apt-get -y upgrade. This caused a hang on install/upgrade of the
iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '17013-upload-wheels-to-pypi' into master
Ward Vandewege [Mon, 23 Nov 2020 16:20:52 +0000 (11:20 -0500)]
Merge branch '17013-upload-wheels-to-pypi' into master

closes #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago17013: also upload wheels to PyPI. To make the wheels universal, we 17013-upload-wheels-to-pypi
Ward Vandewege [Mon, 23 Nov 2020 15:27:42 +0000 (10:27 -0500)]
17013: also upload wheels to PyPI. To make the wheels universal, we
       still need a change to setup.cfg in each of our Python
       component directories.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWe use Python3 now, exclusively.
Ward Vandewege [Mon, 23 Nov 2020 14:27:46 +0000 (09:27 -0500)]
We use Python3 now, exclusively.

refs #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd extra validation to upload_packages_testing_to_stable.sh: only
Ward Vandewege [Wed, 18 Nov 2020 19:37:48 +0000 (14:37 -0500)]
Add extra validation to upload_packages_testing_to_stable.sh: only
numbered releases should be allowed.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '17086-add-focal-fossa' into master
Ward Vandewege [Sat, 7 Nov 2020 13:46:24 +0000 (08:46 -0500)]
Merge branch '17086-add-focal-fossa' into master

refs #17086

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago17086: add support for Ubuntu 20.04, Focal Fossa. 17086-add-focal-fossa
Ward Vandewege [Thu, 5 Nov 2020 19:29:53 +0000 (14:29 -0500)]
17086: add support for Ubuntu 20.04, Focal Fossa.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoFix link to the Coding Standards wiki page.
Ward Vandewege [Wed, 21 Oct 2020 18:15:47 +0000 (14:15 -0400)]
Fix link to the Coding Standards wiki page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing, taking release clients into account.
Ward Vandewege [Fri, 16 Oct 2020 15:14:19 +0000 (11:14 -0400)]
Tweak version number parsing, taking release clients into account.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing (again).
Ward Vandewege [Fri, 16 Oct 2020 14:57:49 +0000 (10:57 -0400)]
Tweak version number parsing (again).

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing.
Ward Vandewege [Fri, 16 Oct 2020 14:47:21 +0000 (10:47 -0400)]
Tweak version number parsing.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoHandle the fact that ~ is not a valid character in docker image tags,
Ward Vandewege [Tue, 13 Oct 2020 19:11:18 +0000 (15:11 -0400)]
Handle the fact that ~ is not a valid character in docker image tags,
apparently.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy-salt.sh: remove superfluous definitions.
Ward Vandewege [Wed, 30 Sep 2020 14:19:14 +0000 (10:19 -0400)]
run-deploy-salt.sh: remove superfluous definitions.

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16921-run-deploy-salt' into master
Ward Vandewege [Wed, 30 Sep 2020 14:17:28 +0000 (10:17 -0400)]
Merge branch '16921-run-deploy-salt' into master

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the 16921-run-deploy-salt
Ward Vandewege [Tue, 29 Sep 2020 19:50:47 +0000 (15:50 -0400)]
16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the
       list of machines to be deployed, and also to issue the commands
       to them.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: some tweaks to support tordo. A bit of cleanup too.
Ward Vandewege [Tue, 29 Sep 2020 13:47:26 +0000 (09:47 -0400)]
run-deploy.sh: some tweaks to support tordo. A bit of cleanup too.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDoesn't need to have -p2222 any more
Peter Amstutz [Fri, 25 Sep 2020 14:31:15 +0000 (10:31 -0400)]
Doesn't need to have -p2222 any more

refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate build-package-r.sh to use r.arvados.org refs #16885
Peter Amstutz [Fri, 25 Sep 2020 14:22:17 +0000 (10:22 -0400)]
Update build-package-r.sh to use r.arvados.org refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoTweak after enabling `set -e`: handle grep properly. Also be more
Ward Vandewege [Thu, 3 Sep 2020 21:12:06 +0000 (17:12 -0400)]
Tweak after enabling `set -e`: handle grep properly. Also be more
specific about the version number that is retrieved.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoNo more tagging as 'latest' for our Docker arvados/jobs image
Ward Vandewege [Sat, 22 Aug 2020 00:02:58 +0000 (20:02 -0400)]
No more tagging as 'latest' for our Docker arvados/jobs image

refs #16565

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWe need a `git pull` after switching to the master branch, to make sure we see
Ward Vandewege [Wed, 19 Aug 2020 23:43:27 +0000 (19:43 -0400)]
We need a `git pull` after switching to the master branch, to make sure we see
the complete lists of branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure we don't accidentally delete our release branches.
Ward Vandewege [Wed, 19 Aug 2020 23:30:28 +0000 (19:30 -0400)]
Make sure we don't accidentally delete our release branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake run-deploy.sh run with set -e, always. We need to know when it
Ward Vandewege [Wed, 19 Aug 2020 02:19:36 +0000 (22:19 -0400)]
Make run-deploy.sh run with set -e, always. We need to know when it
fails.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoThe python pam and node manager packages have been removed, do not try
Ward Vandewege [Tue, 18 Aug 2020 20:10:41 +0000 (16:10 -0400)]
The python pam and node manager packages have been removed, do not try
to upload them.

refs #16580

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoRemove unattended-upgrades from our Jenkins build images, they interfere
Ward Vandewege [Tue, 18 Aug 2020 19:04:57 +0000 (15:04 -0400)]
Remove unattended-upgrades from our Jenkins build images, they interfere
with job startup when we want to install something via apt.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: when there is a shell node, use that to figure out the
Ward Vandewege [Thu, 13 Aug 2020 23:28:12 +0000 (19:28 -0400)]
run-deploy.sh: when there is a shell node, use that to figure out the
version of the arvados/jobs image to install.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agorefactoring: bring the name for the generated federation and k8s tests
Ward Vandewege [Fri, 31 Jul 2020 01:00:36 +0000 (21:00 -0400)]
refactoring: bring the name for the generated federation and k8s tests
images in line with the other jenkins images.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBump version of the Debian Buster image we use.
Ward Vandewege [Fri, 31 Jul 2020 00:46:59 +0000 (20:46 -0400)]
Bump version of the Debian Buster image we use.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorefactoring: rename packer templates and the resulting images to make it
Ward Vandewege [Fri, 31 Jul 2020 00:30:20 +0000 (20:30 -0400)]
refactoring: rename packer templates and the resulting images to make it
more clear what they are for.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16616-move-jenkins-packer-code'
Ward Vandewege [Thu, 30 Jul 2020 17:19:36 +0000 (13:19 -0400)]
Merge branch '16616-move-jenkins-packer-code'

refs #16616

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16616: move jenkins packer templates here. 16616-move-jenkins-packer-code
Ward Vandewege [Tue, 28 Jul 2020 17:50:12 +0000 (13:50 -0400)]
16616: move jenkins packer templates here.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBugfix for the check-copyright-headers.rb script: dtrt with symlinks.
Ward Vandewege [Fri, 19 Jun 2020 21:35:40 +0000 (17:35 -0400)]
Bugfix for the check-copyright-headers.rb script: dtrt with symlinks.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAnother bugfix for the check-copyright-headers.rb script.
Ward Vandewege [Tue, 16 Jun 2020 21:25:31 +0000 (17:25 -0400)]
Another bugfix for the check-copyright-headers.rb script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDo not print a noisy warning when there is no .licenseignore file.
Ward Vandewege [Mon, 15 Jun 2020 19:04:32 +0000 (15:04 -0400)]
Do not print a noisy warning when there is no .licenseignore file.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure to convert the globs in .licenseignore into regular
Ward Vandewege [Mon, 15 Jun 2020 18:57:55 +0000 (14:57 -0400)]
Make sure to convert the globs in .licenseignore into regular
expressions.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoReplace check-copyright-headers.sh with check-copyright-headers.sh.
Ward Vandewege [Mon, 15 Jun 2020 01:55:50 +0000 (21:55 -0400)]
Replace check-copyright-headers.sh with check-copyright-headers.sh.
Rename coding-standards.sh to coding-standards.rb, it was already a Ruby
script. All git hooks are now Ruby.

Fix bugs in check-copyright-headers: no longer print a 'fatal' error
when a branch is first pushed. No longer skip checking of new files in
some circumstances (git `commit` object instead of `blob` object).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd a script to clean up old compute node images on Azure.
Ward Vandewege [Thu, 7 May 2020 01:30:04 +0000 (21:30 -0400)]
Add a script to clean up old compute node images on Azure.

closes #16418

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoLanguage tweaks to COPYING and README, and url updates in README.
Ward Vandewege [Thu, 7 May 2020 01:05:27 +0000 (21:05 -0400)]
Language tweaks to COPYING and README, and url updates in README.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoInitial commit of the costanalyzer code.
Ward Vandewege [Thu, 30 Apr 2020 14:07:48 +0000 (10:07 -0400)]
Initial commit of the costanalyzer code.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoImprove grepping for the docker image to avoid false positives.
Ward Vandewege [Fri, 24 Apr 2020 01:06:32 +0000 (21:06 -0400)]
Improve grepping for the docker image to avoid false positives.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago* run-deploy.sh: fix logic bug in arv-keepdocker call
Ward Vandewege [Sat, 21 Mar 2020 01:45:03 +0000 (21:45 -0400)]
* run-deploy.sh: fix logic bug in arv-keepdocker call
* get version from python3-arvados-cwl-runner package

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoA few improvements to run-deploy.sh:
Ward Vandewege [Fri, 20 Mar 2020 21:12:39 +0000 (17:12 -0400)]
A few improvements to run-deploy.sh:

* deal with those annoying `Ignoring curb-0.9.10 because its extensions
are not built.  Try: gem pristine curb --version 0.9.10` warnings so we
don't display them, for real this time.

* prefix output of remote commands with the name of the host. That makes
determining which host is having issues easier when run with
--concurrency > 1

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd make-package-list script
Peter Amstutz [Fri, 14 Feb 2020 16:16:00 +0000 (16:16 +0000)]
Add make-package-list script

4 years agoCan unpublish an individual package
Peter Amstutz [Fri, 14 Feb 2020 15:56:26 +0000 (15:56 +0000)]
Can unpublish an individual package

4 years agoFix find pattern for debs
Peter Amstutz [Mon, 10 Feb 2020 19:43:10 +0000 (14:43 -0500)]
Fix find pattern for debs

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAdd unpublish script for removing from testing repo
Peter Amstutz [Mon, 10 Feb 2020 19:35:56 +0000 (14:35 -0500)]
Add unpublish script for removing from testing repo

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSimplify the deploy script (and fix a bug around detection of installed
Ward Vandewege [Wed, 5 Feb 2020 21:37:06 +0000 (16:37 -0500)]
Simplify the deploy script (and fix a bug around detection of installed
docker images).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoRun docker tag latest on the node from which we will run arv-keepdocker
Ward Vandewege [Wed, 5 Feb 2020 21:24:02 +0000 (16:24 -0500)]
Run docker tag latest on the node from which we will run arv-keepdocker
tag latest.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoMake sure all the ssh calls in run-deploy.sh use the supplied connection
Ward Vandewege [Wed, 5 Feb 2020 16:56:21 +0000 (11:56 -0500)]
Make sure all the ssh calls in run-deploy.sh use the supplied connection
parameters

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoUpdate deploy script to include API server for ce8i5
Peter Amstutz [Thu, 30 Jan 2020 16:23:49 +0000 (11:23 -0500)]
Update deploy script to include API server for ce8i5

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agorun deploy workbench2 for ce8i5
Peter Amstutz [Wed, 29 Jan 2020 21:10:35 +0000 (21:10 +0000)]
run deploy workbench2 for ce8i5

4 years agoIncrease js eval timeout for run-cwl-tests another up to 60s
Peter Amstutz [Mon, 27 Jan 2020 22:55:08 +0000 (17:55 -0500)]
Increase js eval timeout for run-cwl-tests another up to 60s

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoIncrease js eval timeout for run-cwl-tests
Peter Amstutz [Mon, 27 Jan 2020 14:58:12 +0000 (09:58 -0500)]
Increase js eval timeout for run-cwl-tests

Remove unused legacy run-cwl-test script

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix target check
Peter Amstutz [Fri, 24 Jan 2020 16:29:04 +0000 (11:29 -0500)]
Fix target check

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMake dev/testing selectable instead of having a second copy-pasted script
Peter Amstutz [Fri, 24 Jan 2020 16:22:58 +0000 (11:22 -0500)]
Make dev/testing selectable instead of having a second copy-pasted script

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSimplify PACKAGE_LIST sed line
Peter Amstutz [Thu, 9 Jan 2020 20:52:10 +0000 (15:52 -0500)]
Simplify PACKAGE_LIST sed line

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoupload_packages_testing_to_stable.sh add trailing '*' to package patterns
Peter Amstutz [Thu, 9 Jan 2020 19:57:23 +0000 (19:57 +0000)]
upload_packages_testing_to_stable.sh add trailing '*' to package patterns

run-cwl-test.sh: log ssh command on DEBUG

no issue #

4 years agoBugfix in run-deploy.sh: the run_command needs to return its output.
Ward Vandewege [Tue, 7 Jan 2020 15:36:39 +0000 (10:36 -0500)]
Bugfix in run-deploy.sh: the run_command needs to return its output.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoce8i5 now has a shell node, too.
Ward Vandewege [Fri, 20 Dec 2019 00:39:37 +0000 (19:39 -0500)]
ce8i5 now has a shell node, too.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd support for unmanaged-by-puppet clusters to the deploy script, and
Ward Vandewege [Thu, 19 Dec 2019 19:53:48 +0000 (14:53 -0500)]
Add support for unmanaged-by-puppet clusters to the deploy script, and
also specifically support ce8i5.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoFix comment in run-deploy.sh
Ward Vandewege [Thu, 19 Dec 2019 15:24:24 +0000 (10:24 -0500)]
Fix comment in run-deploy.sh

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

5 years agoAdd support for Debian 10 (aka 'Buster') to run_upload_packages.py 15845-add-debian10-upload
Fernando Monserrat [Wed, 20 Nov 2019 18:37:34 +0000 (15:37 -0300)]
Add support for Debian 10 (aka 'Buster') to run_upload_packages.py
refs #15845
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agorun-deploy.sh improvements:
Ward Vandewege [Wed, 25 Sep 2019 17:44:51 +0000 (13:44 -0400)]
run-deploy.sh improvements:
* make sure all arv-keepdocker calls include the project id.
* add some extra verbosity before arv-keepdocker call to tag the arvados/jobs image that exists

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy.sh fix: be consistent in how we call arv-keepdocker.
Ward Vandewege [Wed, 25 Sep 2019 17:05:54 +0000 (13:05 -0400)]
run-deploy.sh fix: be consistent in how we call arv-keepdocker.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy fixes:
Ward Vandewege [Fri, 13 Sep 2019 13:55:12 +0000 (09:55 -0400)]
run-deploy fixes:

* when getting GIT_COMMIT, exclude stderr
* unload RVM before running puppet

refs #15630

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBe consistent about how we run the arv command.
Ward Vandewege [Thu, 2 May 2019 15:10:21 +0000 (11:10 -0400)]
Be consistent about how we run the arv command.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy.sh should also handle pre-1.4 installations.
Ward Vandewege [Thu, 2 May 2019 15:00:48 +0000 (11:00 -0400)]
run-deploy.sh should also handle pre-1.4 installations.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoFix ARVADOS_API_TOKEN munging in logs.
Ward Vandewege [Thu, 2 May 2019 14:17:01 +0000 (10:17 -0400)]
Fix ARVADOS_API_TOKEN munging in logs.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorefs #14891 .Merge branch '14891-upload-to-testing-repo'
Fernando Monserrat [Tue, 5 Mar 2019 21:12:32 +0000 (18:12 -0300)]
refs #14891 .Merge branch '14891-upload-to-testing-repo'
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>