arvados-dev.git
3 years ago16916: add packer files for WB2 test image 16916-move-wb2-tests-to-jenkins-satellites
Ward Vandewege [Thu, 7 Jan 2021 22:47:42 +0000 (17:47 -0500)]
16916: add packer files for WB2 test image

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfixes for packaging-move-dev-to-attic.py
Ward Vandewege [Mon, 4 Jan 2021 21:04:47 +0000 (16:04 -0500)]
Bugfixes for packaging-move-dev-to-attic.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17219-refactor-attic-scripts'
Ward Vandewege [Mon, 4 Jan 2021 18:55:52 +0000 (13:55 -0500)]
Merge branch '17219-refactor-attic-scripts'

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAddress review comments. 17219-refactor-attic-scripts
Ward Vandewege [Mon, 4 Jan 2021 18:55:21 +0000 (13:55 -0500)]
Address review comments.

closes #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agopackaging-move-dev-to-attic.py: remove hardcoded list of supported
Ward Vandewege [Mon, 4 Jan 2021 18:27:21 +0000 (13:27 -0500)]
packaging-move-dev-to-attic.py: remove hardcoded list of supported
distributions, instead validate the distribution name by pattern.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd packaging-move-dev-to-attic.py, which now supports aptly. This is a
Ward Vandewege [Mon, 4 Jan 2021 18:11:33 +0000 (13:11 -0500)]
Add packaging-move-dev-to-attic.py, which now supports aptly. This is a
modified version of an old internal script that supported freight.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoIterate over packages one by one in run_upload_packages.py
Ward Vandewege [Wed, 23 Dec 2020 03:05:52 +0000 (22:05 -0500)]
Iterate over packages one by one in run_upload_packages.py

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake the run_upload_packages.py script more verbose.
Ward Vandewege [Wed, 23 Dec 2020 02:27:43 +0000 (21:27 -0500)]
Make the run_upload_packages.py script more verbose.

refs #17219

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not try to add packages to the repository that are already in it.
Ward Vandewege [Wed, 23 Dec 2020 01:45:43 +0000 (20:45 -0500)]
Do not try to add packages to the repository that are already in it.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17218-repository-update'
Ward Vandewege [Tue, 22 Dec 2020 22:09:18 +0000 (17:09 -0500)]
Merge branch '17218-repository-update'

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWe've made a change in the structure of our apt repositories. 17218-repository-update
Ward Vandewege [Tue, 22 Dec 2020 19:56:34 +0000 (14:56 -0500)]
We've made a change in the structure of our apt repositories.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake it possible to deploy a single node.
Ward Vandewege [Wed, 16 Dec 2020 19:19:50 +0000 (14:19 -0500)]
Make it possible to deploy a single node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoDo not run arv-keepdocker on every shell node, just on the main shell
Ward Vandewege [Wed, 16 Dec 2020 19:04:40 +0000 (14:04 -0500)]
Do not run arv-keepdocker on every shell node, just on the main shell
node.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agorun-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
Ward Vandewege [Wed, 16 Dec 2020 18:50:09 +0000 (13:50 -0500)]
run-deploy-salt.sh: make sure to have DEBIAN_FRONTEND=noninteractive set
when running apt-get -y upgrade. This caused a hang on install/upgrade
of the iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoNo need to sudo to run salt.
Ward Vandewege [Wed, 16 Dec 2020 14:54:33 +0000 (09:54 -0500)]
No need to sudo to run salt.

refs #16925

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMake sure to have DEBIAN_FRONTEND=noninteractive set when running
Ward Vandewege [Tue, 15 Dec 2020 15:33:24 +0000 (10:33 -0500)]
Make sure to have DEBIAN_FRONTEND=noninteractive set when running
apt-get -y upgrade. This caused a hang on install/upgrade of the
iptables-persistent package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '17013-upload-wheels-to-pypi' into master
Ward Vandewege [Mon, 23 Nov 2020 16:20:52 +0000 (11:20 -0500)]
Merge branch '17013-upload-wheels-to-pypi' into master

closes #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago17013: also upload wheels to PyPI. To make the wheels universal, we 17013-upload-wheels-to-pypi
Ward Vandewege [Mon, 23 Nov 2020 15:27:42 +0000 (10:27 -0500)]
17013: also upload wheels to PyPI. To make the wheels universal, we
       still need a change to setup.cfg in each of our Python
       component directories.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWe use Python3 now, exclusively.
Ward Vandewege [Mon, 23 Nov 2020 14:27:46 +0000 (09:27 -0500)]
We use Python3 now, exclusively.

refs #17013

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd extra validation to upload_packages_testing_to_stable.sh: only
Ward Vandewege [Wed, 18 Nov 2020 19:37:48 +0000 (14:37 -0500)]
Add extra validation to upload_packages_testing_to_stable.sh: only
numbered releases should be allowed.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '17086-add-focal-fossa' into master
Ward Vandewege [Sat, 7 Nov 2020 13:46:24 +0000 (08:46 -0500)]
Merge branch '17086-add-focal-fossa' into master

refs #17086

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago17086: add support for Ubuntu 20.04, Focal Fossa. 17086-add-focal-fossa
Ward Vandewege [Thu, 5 Nov 2020 19:29:53 +0000 (14:29 -0500)]
17086: add support for Ubuntu 20.04, Focal Fossa.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoFix link to the Coding Standards wiki page.
Ward Vandewege [Wed, 21 Oct 2020 18:15:47 +0000 (14:15 -0400)]
Fix link to the Coding Standards wiki page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing, taking release clients into account.
Ward Vandewege [Fri, 16 Oct 2020 15:14:19 +0000 (11:14 -0400)]
Tweak version number parsing, taking release clients into account.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing (again).
Ward Vandewege [Fri, 16 Oct 2020 14:57:49 +0000 (10:57 -0400)]
Tweak version number parsing (again).

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoTweak version number parsing.
Ward Vandewege [Fri, 16 Oct 2020 14:47:21 +0000 (10:47 -0400)]
Tweak version number parsing.

refs #16982

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoHandle the fact that ~ is not a valid character in docker image tags,
Ward Vandewege [Tue, 13 Oct 2020 19:11:18 +0000 (15:11 -0400)]
Handle the fact that ~ is not a valid character in docker image tags,
apparently.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy-salt.sh: remove superfluous definitions.
Ward Vandewege [Wed, 30 Sep 2020 14:19:14 +0000 (10:19 -0400)]
run-deploy-salt.sh: remove superfluous definitions.

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16921-run-deploy-salt' into master
Ward Vandewege [Wed, 30 Sep 2020 14:17:28 +0000 (10:17 -0400)]
Merge branch '16921-run-deploy-salt' into master

refs #16921

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the 16921-run-deploy-salt
Ward Vandewege [Tue, 29 Sep 2020 19:50:47 +0000 (15:50 -0400)]
16921: add run-deploy-salt.sh, which is simpler. It uses salt to get the
       list of machines to be deployed, and also to issue the commands
       to them.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: some tweaks to support tordo. A bit of cleanup too.
Ward Vandewege [Tue, 29 Sep 2020 13:47:26 +0000 (09:47 -0400)]
run-deploy.sh: some tweaks to support tordo. A bit of cleanup too.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDoesn't need to have -p2222 any more
Peter Amstutz [Fri, 25 Sep 2020 14:31:15 +0000 (10:31 -0400)]
Doesn't need to have -p2222 any more

refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate build-package-r.sh to use r.arvados.org refs #16885
Peter Amstutz [Fri, 25 Sep 2020 14:22:17 +0000 (10:22 -0400)]
Update build-package-r.sh to use r.arvados.org refs #16885

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoTweak after enabling `set -e`: handle grep properly. Also be more
Ward Vandewege [Thu, 3 Sep 2020 21:12:06 +0000 (17:12 -0400)]
Tweak after enabling `set -e`: handle grep properly. Also be more
specific about the version number that is retrieved.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoNo more tagging as 'latest' for our Docker arvados/jobs image
Ward Vandewege [Sat, 22 Aug 2020 00:02:58 +0000 (20:02 -0400)]
No more tagging as 'latest' for our Docker arvados/jobs image

refs #16565

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWe need a `git pull` after switching to the master branch, to make sure we see
Ward Vandewege [Wed, 19 Aug 2020 23:43:27 +0000 (19:43 -0400)]
We need a `git pull` after switching to the master branch, to make sure we see
the complete lists of branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure we don't accidentally delete our release branches.
Ward Vandewege [Wed, 19 Aug 2020 23:30:28 +0000 (19:30 -0400)]
Make sure we don't accidentally delete our release branches.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake run-deploy.sh run with set -e, always. We need to know when it
Ward Vandewege [Wed, 19 Aug 2020 02:19:36 +0000 (22:19 -0400)]
Make run-deploy.sh run with set -e, always. We need to know when it
fails.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoThe python pam and node manager packages have been removed, do not try
Ward Vandewege [Tue, 18 Aug 2020 20:10:41 +0000 (16:10 -0400)]
The python pam and node manager packages have been removed, do not try
to upload them.

refs #16580

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoRemove unattended-upgrades from our Jenkins build images, they interfere
Ward Vandewege [Tue, 18 Aug 2020 19:04:57 +0000 (15:04 -0400)]
Remove unattended-upgrades from our Jenkins build images, they interfere
with job startup when we want to install something via apt.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorun-deploy.sh: when there is a shell node, use that to figure out the
Ward Vandewege [Thu, 13 Aug 2020 23:28:12 +0000 (19:28 -0400)]
run-deploy.sh: when there is a shell node, use that to figure out the
version of the arvados/jobs image to install.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agorefactoring: bring the name for the generated federation and k8s tests
Ward Vandewege [Fri, 31 Jul 2020 01:00:36 +0000 (21:00 -0400)]
refactoring: bring the name for the generated federation and k8s tests
images in line with the other jenkins images.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBump version of the Debian Buster image we use.
Ward Vandewege [Fri, 31 Jul 2020 00:46:59 +0000 (20:46 -0400)]
Bump version of the Debian Buster image we use.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agorefactoring: rename packer templates and the resulting images to make it
Ward Vandewege [Fri, 31 Jul 2020 00:30:20 +0000 (20:30 -0400)]
refactoring: rename packer templates and the resulting images to make it
more clear what they are for.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16616-move-jenkins-packer-code'
Ward Vandewege [Thu, 30 Jul 2020 17:19:36 +0000 (13:19 -0400)]
Merge branch '16616-move-jenkins-packer-code'

refs #16616

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16616: move jenkins packer templates here. 16616-move-jenkins-packer-code
Ward Vandewege [Tue, 28 Jul 2020 17:50:12 +0000 (13:50 -0400)]
16616: move jenkins packer templates here.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBugfix for the check-copyright-headers.rb script: dtrt with symlinks.
Ward Vandewege [Fri, 19 Jun 2020 21:35:40 +0000 (17:35 -0400)]
Bugfix for the check-copyright-headers.rb script: dtrt with symlinks.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAnother bugfix for the check-copyright-headers.rb script.
Ward Vandewege [Tue, 16 Jun 2020 21:25:31 +0000 (17:25 -0400)]
Another bugfix for the check-copyright-headers.rb script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoDo not print a noisy warning when there is no .licenseignore file.
Ward Vandewege [Mon, 15 Jun 2020 19:04:32 +0000 (15:04 -0400)]
Do not print a noisy warning when there is no .licenseignore file.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMake sure to convert the globs in .licenseignore into regular
Ward Vandewege [Mon, 15 Jun 2020 18:57:55 +0000 (14:57 -0400)]
Make sure to convert the globs in .licenseignore into regular
expressions.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoReplace check-copyright-headers.sh with check-copyright-headers.sh.
Ward Vandewege [Mon, 15 Jun 2020 01:55:50 +0000 (21:55 -0400)]
Replace check-copyright-headers.sh with check-copyright-headers.sh.
Rename coding-standards.sh to coding-standards.rb, it was already a Ruby
script. All git hooks are now Ruby.

Fix bugs in check-copyright-headers: no longer print a 'fatal' error
when a branch is first pushed. No longer skip checking of new files in
some circumstances (git `commit` object instead of `blob` object).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoAdd a script to clean up old compute node images on Azure.
Ward Vandewege [Thu, 7 May 2020 01:30:04 +0000 (21:30 -0400)]
Add a script to clean up old compute node images on Azure.

closes #16418

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoLanguage tweaks to COPYING and README, and url updates in README.
Ward Vandewege [Thu, 7 May 2020 01:05:27 +0000 (21:05 -0400)]
Language tweaks to COPYING and README, and url updates in README.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoInitial commit of the costanalyzer code.
Ward Vandewege [Thu, 30 Apr 2020 14:07:48 +0000 (10:07 -0400)]
Initial commit of the costanalyzer code.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoImprove grepping for the docker image to avoid false positives.
Ward Vandewege [Fri, 24 Apr 2020 01:06:32 +0000 (21:06 -0400)]
Improve grepping for the docker image to avoid false positives.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago* run-deploy.sh: fix logic bug in arv-keepdocker call
Ward Vandewege [Sat, 21 Mar 2020 01:45:03 +0000 (21:45 -0400)]
* run-deploy.sh: fix logic bug in arv-keepdocker call
* get version from python3-arvados-cwl-runner package

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoA few improvements to run-deploy.sh:
Ward Vandewege [Fri, 20 Mar 2020 21:12:39 +0000 (17:12 -0400)]
A few improvements to run-deploy.sh:

* deal with those annoying `Ignoring curb-0.9.10 because its extensions
are not built.  Try: gem pristine curb --version 0.9.10` warnings so we
don't display them, for real this time.

* prefix output of remote commands with the name of the host. That makes
determining which host is having issues easier when run with
--concurrency > 1

refs #16220

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd make-package-list script
Peter Amstutz [Fri, 14 Feb 2020 16:16:00 +0000 (16:16 +0000)]
Add make-package-list script

4 years agoCan unpublish an individual package
Peter Amstutz [Fri, 14 Feb 2020 15:56:26 +0000 (15:56 +0000)]
Can unpublish an individual package

4 years agoFix find pattern for debs
Peter Amstutz [Mon, 10 Feb 2020 19:43:10 +0000 (14:43 -0500)]
Fix find pattern for debs

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAdd unpublish script for removing from testing repo
Peter Amstutz [Mon, 10 Feb 2020 19:35:56 +0000 (14:35 -0500)]
Add unpublish script for removing from testing repo

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSimplify the deploy script (and fix a bug around detection of installed
Ward Vandewege [Wed, 5 Feb 2020 21:37:06 +0000 (16:37 -0500)]
Simplify the deploy script (and fix a bug around detection of installed
docker images).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoRun docker tag latest on the node from which we will run arv-keepdocker
Ward Vandewege [Wed, 5 Feb 2020 21:24:02 +0000 (16:24 -0500)]
Run docker tag latest on the node from which we will run arv-keepdocker
tag latest.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoMake sure all the ssh calls in run-deploy.sh use the supplied connection
Ward Vandewege [Wed, 5 Feb 2020 16:56:21 +0000 (11:56 -0500)]
Make sure all the ssh calls in run-deploy.sh use the supplied connection
parameters

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoUpdate deploy script to include API server for ce8i5
Peter Amstutz [Thu, 30 Jan 2020 16:23:49 +0000 (11:23 -0500)]
Update deploy script to include API server for ce8i5

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agorun deploy workbench2 for ce8i5
Peter Amstutz [Wed, 29 Jan 2020 21:10:35 +0000 (21:10 +0000)]
run deploy workbench2 for ce8i5

4 years agoIncrease js eval timeout for run-cwl-tests another up to 60s
Peter Amstutz [Mon, 27 Jan 2020 22:55:08 +0000 (17:55 -0500)]
Increase js eval timeout for run-cwl-tests another up to 60s

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoIncrease js eval timeout for run-cwl-tests
Peter Amstutz [Mon, 27 Jan 2020 14:58:12 +0000 (09:58 -0500)]
Increase js eval timeout for run-cwl-tests

Remove unused legacy run-cwl-test script

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix target check
Peter Amstutz [Fri, 24 Jan 2020 16:29:04 +0000 (11:29 -0500)]
Fix target check

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMake dev/testing selectable instead of having a second copy-pasted script
Peter Amstutz [Fri, 24 Jan 2020 16:22:58 +0000 (11:22 -0500)]
Make dev/testing selectable instead of having a second copy-pasted script

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSimplify PACKAGE_LIST sed line
Peter Amstutz [Thu, 9 Jan 2020 20:52:10 +0000 (15:52 -0500)]
Simplify PACKAGE_LIST sed line

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoupload_packages_testing_to_stable.sh add trailing '*' to package patterns
Peter Amstutz [Thu, 9 Jan 2020 19:57:23 +0000 (19:57 +0000)]
upload_packages_testing_to_stable.sh add trailing '*' to package patterns

run-cwl-test.sh: log ssh command on DEBUG

no issue #

4 years agoBugfix in run-deploy.sh: the run_command needs to return its output.
Ward Vandewege [Tue, 7 Jan 2020 15:36:39 +0000 (10:36 -0500)]
Bugfix in run-deploy.sh: the run_command needs to return its output.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoce8i5 now has a shell node, too.
Ward Vandewege [Fri, 20 Dec 2019 00:39:37 +0000 (19:39 -0500)]
ce8i5 now has a shell node, too.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd support for unmanaged-by-puppet clusters to the deploy script, and
Ward Vandewege [Thu, 19 Dec 2019 19:53:48 +0000 (14:53 -0500)]
Add support for unmanaged-by-puppet clusters to the deploy script, and
also specifically support ce8i5.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoFix comment in run-deploy.sh
Ward Vandewege [Thu, 19 Dec 2019 15:24:24 +0000 (10:24 -0500)]
Fix comment in run-deploy.sh

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

5 years agoAdd support for Debian 10 (aka 'Buster') to run_upload_packages.py 15845-add-debian10-upload
Fernando Monserrat [Wed, 20 Nov 2019 18:37:34 +0000 (15:37 -0300)]
Add support for Debian 10 (aka 'Buster') to run_upload_packages.py
refs #15845
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agorun-deploy.sh improvements:
Ward Vandewege [Wed, 25 Sep 2019 17:44:51 +0000 (13:44 -0400)]
run-deploy.sh improvements:
* make sure all arv-keepdocker calls include the project id.
* add some extra verbosity before arv-keepdocker call to tag the arvados/jobs image that exists

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy.sh fix: be consistent in how we call arv-keepdocker.
Ward Vandewege [Wed, 25 Sep 2019 17:05:54 +0000 (13:05 -0400)]
run-deploy.sh fix: be consistent in how we call arv-keepdocker.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy fixes:
Ward Vandewege [Fri, 13 Sep 2019 13:55:12 +0000 (09:55 -0400)]
run-deploy fixes:

* when getting GIT_COMMIT, exclude stderr
* unload RVM before running puppet

refs #15630

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBe consistent about how we run the arv command.
Ward Vandewege [Thu, 2 May 2019 15:10:21 +0000 (11:10 -0400)]
Be consistent about how we run the arv command.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorun-deploy.sh should also handle pre-1.4 installations.
Ward Vandewege [Thu, 2 May 2019 15:00:48 +0000 (11:00 -0400)]
run-deploy.sh should also handle pre-1.4 installations.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoFix ARVADOS_API_TOKEN munging in logs.
Ward Vandewege [Thu, 2 May 2019 14:17:01 +0000 (10:17 -0400)]
Fix ARVADOS_API_TOKEN munging in logs.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agorefs #14891 .Merge branch '14891-upload-to-testing-repo'
Fernando Monserrat [Tue, 5 Mar 2019 21:12:32 +0000 (18:12 -0300)]
refs #14891 .Merge branch '14891-upload-to-testing-repo'
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agoissue #14891 add scripts to publish packages from testing to stable 14891-upload-to-testing-repo
Fernando Monserrat [Tue, 5 Mar 2019 21:11:06 +0000 (18:11 -0300)]
issue #14891 add scripts to publish packages from testing to stable
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agoissue #14891 adding python script to upload release candidates to testing repo
Fernando Monserrat [Tue, 5 Mar 2019 19:16:10 +0000 (16:16 -0300)]
issue #14891 adding python script to upload release candidates to testing repo
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agoNow python-arvados-cwl-runner includes a virtualenv, make sure to use its
Ward Vandewege [Thu, 7 Feb 2019 17:41:23 +0000 (12:41 -0500)]
Now python-arvados-cwl-runner includes a virtualenv, make sure to use its
python executable when importing the arvados_cwl python module.

refs #9945

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoRun CWL tests on the containers API.
Ward Vandewege [Thu, 11 Oct 2018 16:26:39 +0000 (12:26 -0400)]
Run CWL tests on the containers API.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd support for ubuntu 18.04 (aka 'bionic') to run_upload_packages.py
Ward Vandewege [Thu, 23 Aug 2018 13:26:48 +0000 (09:26 -0400)]
Add support for ubuntu 18.04 (aka 'bionic') to run_upload_packages.py

refs #13823

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoIncrease the timeout on a single CWL test to 900 seconds (15 minutes)
Peter Amstutz [Mon, 13 Aug 2018 19:59:45 +0000 (15:59 -0400)]
Increase the timeout on a single CWL test to 900 seconds (15 minutes)

This is to accomodate the observation that occasionally waiting for a
compute node to become available exceeds a 10 minute test timeout, but
had it had a little more time the test would have succeeded.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoAdd option to deploy just one node to the run_deploy.sh script.
Ward Vandewege [Tue, 19 Jun 2018 17:44:21 +0000 (13:44 -0400)]
Add option to deploy just one node to the run_deploy.sh script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago* Add a 'local' mode to jenkins/run-cwl-test.sh, where arvados-cwl-runner runs
Ward Vandewege [Fri, 15 Jun 2018 19:22:11 +0000 (15:22 -0400)]
* Add a 'local' mode to jenkins/run-cwl-test.sh, where arvados-cwl-runner runs
locally
* Make fewer assumptions about the remote Arvados cluster.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoWhen enforcing copyright headers, do not consider commits that are already part
Ward Vandewege [Fri, 25 May 2018 20:48:14 +0000 (16:48 -0400)]
When enforcing copyright headers, do not consider commits that are already part
of the git history of the repository.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoChange the pattern matching for our copyright header so that it doesn't croak
Ward Vandewege [Wed, 23 May 2018 21:47:20 +0000 (17:47 -0400)]
Change the pattern matching for our copyright header so that it doesn't croak
on files that are copyright someone else.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoBe a little more verbose about the presence of the copyright header check.
Ward Vandewege [Wed, 23 May 2018 21:21:16 +0000 (17:21 -0400)]
Be a little more verbose about the presence of the copyright header check.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd a git hook to ensure that all commits adhere to the copyright header
Ward Vandewege [Wed, 23 May 2018 21:17:15 +0000 (17:17 -0400)]
Add a git hook to ensure that all commits adhere to the copyright header
convention for the Arvados project.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoSupport -j argument on run-cwl-test.sh script, to parallelize the CWL
Ward Vandewege [Thu, 26 Apr 2018 02:22:11 +0000 (22:22 -0400)]
Support -j argument on run-cwl-test.sh script, to parallelize the CWL
tests.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agorefs #13038 fix script permission
Fernando Monserrat [Fri, 9 Feb 2018 20:30:47 +0000 (17:30 -0300)]
refs #13038 fix script permission
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago"Merge branch '13038-build-package-r'".
Fernando Monserrat [Fri, 9 Feb 2018 20:25:41 +0000 (17:25 -0300)]
"Merge branch '13038-build-package-r'".
refs #13038
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago#13056 fix missing quote 13038-build-package-r
Fernando Monserrat [Fri, 9 Feb 2018 20:21:05 +0000 (17:21 -0300)]
#13056 fix missing quote
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>