Brett Smith [Wed, 14 Oct 2015 22:18:58 +0000 (18:18 -0400)]
7341: run-deploy.sh retries failed Puppet agent runs.
It simply keeps retrying any failure until ten minutes have passed,
per Nico's suggestion in the comments.
Bonus bugfix: make sure run_puppet's ECODE variable gets set to the
exit status of Puppet (or the wrapper script), and not tee.
Ward Vandewege [Tue, 13 Oct 2015 14:18:55 +0000 (10:18 -0400)]
Make sure to run the arvados-api-server-upgrade.sh and
arvados-workbench-upgrade.sh scripts in the RVM environment in
run-deploy.sh.
no issue #
Ward Vandewege [Fri, 9 Oct 2015 17:09:20 +0000 (13:09 -0400)]
Make it possible to pass --debug through to the script run inside the Docker
build containers.
No issue #
Ward Vandewege [Fri, 9 Oct 2015 17:02:39 +0000 (13:02 -0400)]
Ensure proper permissions on /usr/local/arvados/src in the arvados-src package.
refs #7397
Nico Cesar [Wed, 7 Oct 2015 14:09:16 +0000 (10:09 -0400)]
7460: c97qk doesnt need mailchimp deployed. this caused deploy to fail in Ubuntu
refs #7460
Peter Amstutz [Wed, 7 Oct 2015 14:07:25 +0000 (10:07 -0400)]
Run CWL tests for both draft-2 and draft-3. No issue #
Peter Amstutz [Wed, 7 Oct 2015 14:02:43 +0000 (10:02 -0400)]
Update cwl builder process for repository reorganization. No issue #
Peter Amstutz [Mon, 5 Oct 2015 18:21:04 +0000 (14:21 -0400)]
Update pin of libcloud fork to dev4 refs #7286
Brett Smith [Mon, 5 Oct 2015 17:14:00 +0000 (13:14 -0400)]
Merge branch '7451-dont-recreate-venv3dir-wip'
Refs #7451. Closes #7452.
Brett Smith [Mon, 5 Oct 2015 16:53:42 +0000 (12:53 -0400)]
7451: Improve virtualenv setup in run-tests.sh.
The code to build the Python 3 virtualenv didn't check to see whether
the virtualenv already existed first. When reusing a Python 3
virtualenv, pip would get downgraded and start failing.
Refactor the virtualenv setup code to avoid this duplication and these
sorts of bugs.
While I was at it: rather than trying to parse package versions
ourselves to figure out when to upgrade setuptools and pip, just tell
pip the versions we want. 'pip>=7' is an approximation of what we're
currently doing. The main thing we care about is that we get a
version new enough to work with setuptools 18.
Tom Clegg [Mon, 5 Oct 2015 16:48:56 +0000 (12:48 -0400)]
Fix incorrect line numbers on compile errors reported by "go test" in coverage mode. No issue #
Brett Smith [Wed, 30 Sep 2015 15:11:59 +0000 (11:11 -0400)]
7356: CentOS 6 needs a Python 3 backport for websocket-client.
Refs #7356.
Ward Vandewege [Tue, 29 Sep 2015 17:49:35 +0000 (13:49 -0400)]
Merge branch '7330-postinst-for-sso-server'
refs #7330
Ward Vandewege [Tue, 29 Sep 2015 01:00:32 +0000 (21:00 -0400)]
More fixes for the SSO server packages.
refs #7330
Ward Vandewege [Mon, 28 Sep 2015 21:42:58 +0000 (17:42 -0400)]
A number of additional tweaks.
refs #7330
Ward Vandewege [Mon, 28 Sep 2015 04:23:16 +0000 (00:23 -0400)]
Add support for CentOS6 to the improved SSO packages.
refs #7330
Ward Vandewege [Sun, 27 Sep 2015 19:59:56 +0000 (15:59 -0400)]
Initial version of run-test-packages-sso.sh
refs #7330
Ward Vandewege [Sun, 27 Sep 2015 19:29:15 +0000 (15:29 -0400)]
Remove bash-ism from arvados-sso-server.postinst.
refs #7330
Ward Vandewege [Sun, 27 Sep 2015 15:22:32 +0000 (11:22 -0400)]
Add prerm and postrm scripts, various cleanups.
refs #7330
Ward Vandewege [Sat, 26 Sep 2015 01:42:29 +0000 (21:42 -0400)]
First commit.
refs #7330
Ward Vandewege [Fri, 25 Sep 2015 16:05:33 +0000 (12:05 -0400)]
Fix the test that avoids running puppet multiple times on installations
where workbench runs on the API server.
No issue #
Brett Smith [Thu, 24 Sep 2015 16:01:20 +0000 (12:01 -0400)]
7322: Don't backport virtualenv for Arvados.
The Arvados code only needs virtualenv 1.7, which is provided with all
of our supported distros.
run-tests.sh requires 1.8.3 to use the --setuptools switch, but that
can be handled without publishing a backported package.
Refs #7322.
Peter Amstutz [Wed, 23 Sep 2015 17:02:43 +0000 (13:02 -0400)]
Merge branch '7228-group-writable-tmp-cache' refs #7228
Peter Amstutz [Wed, 23 Sep 2015 13:09:25 +0000 (09:09 -0400)]
7228: Add chmod -R 2775 $RELEASE_PATH/tmp/cache/ so that cache is group writable.
Ward Vandewege [Mon, 21 Sep 2015 20:56:57 +0000 (16:56 -0400)]
Make the build step for our libcloud package a lot less noisy by default.
No issue #
Brett Smith [Wed, 16 Sep 2015 20:33:28 +0000 (16:33 -0400)]
Close unclosed quotes in arvados-api-server-upgrade.sh.
No issue #.
Brett Smith [Wed, 16 Sep 2015 19:25:40 +0000 (15:25 -0400)]
Merge branch 'pr/1'
Closes #7261.
radhika [Mon, 14 Sep 2015 22:56:43 +0000 (18:56 -0400)]
refs #6260
Merge branch '6260-test-datamanager'
radhika [Mon, 14 Sep 2015 22:53:16 +0000 (18:53 -0400)]
6260: since datamanager no longer needs the data-manager-token-file (only keep server does), remove the run-tests.sh update for this.
Tom Clegg [Sat, 12 Sep 2015 02:25:30 +0000 (22:25 -0400)]
Merge branch '7311-temp-base' closes #7311
Tom Clegg [Fri, 11 Sep 2015 21:04:53 +0000 (17:04 -0400)]
7311: Do not forget to make temp dirs.
Tom Clegg [Fri, 11 Sep 2015 14:44:16 +0000 (10:44 -0400)]
7311: Make recognizable temp dirs under a single base; allow override with --temp DIR
Joshua C. Randall [Thu, 10 Sep 2015 20:20:04 +0000 (20:20 +0000)]
Adds setuptools/pip upgrade in python3 virtualenv
Moves ruby environment setup to after python3 virtualenv
setup, because otherwise activating the python3 virtualenv
breaks ruby.
Peter Amstutz [Wed, 9 Sep 2015 15:42:39 +0000 (11:42 -0400)]
Add libcloud-pin and update to dev3 refs #7031
radhika [Tue, 8 Sep 2015 21:26:12 +0000 (17:26 -0400)]
6260: append to testargs, instead of replacing.
radhika [Tue, 8 Sep 2015 16:34:54 +0000 (12:34 -0400)]
Merge branch 'master' into 6260-test-datamanager
Peter Amstutz [Tue, 8 Sep 2015 13:44:21 +0000 (09:44 -0400)]
Merge branch '7031-build-forked-libcloud' refs #7031
Peter Amstutz [Tue, 8 Sep 2015 13:43:20 +0000 (09:43 -0400)]
Merge branch 'master' into 7031-build-forked-libcloud
Tom Clegg [Mon, 7 Sep 2015 18:13:18 +0000 (14:13 -0400)]
Merge branch '6263-go-coverage' refs #6263
Tom Clegg [Mon, 7 Sep 2015 18:10:53 +0000 (14:10 -0400)]
Merge branch '6263-gitolite-test' refs #6263
Ward Vandewege [Fri, 4 Sep 2015 15:04:17 +0000 (11:04 -0400)]
Make sure the library can always be found.
No issue #
Peter Amstutz [Fri, 4 Sep 2015 14:00:23 +0000 (10:00 -0400)]
Bump version to dev2
Peter Amstutz [Mon, 31 Aug 2015 18:10:53 +0000 (14:10 -0400)]
7031: Fix python prefix
Peter Amstutz [Mon, 31 Aug 2015 17:38:37 +0000 (13:38 -0400)]
7031: Check out our forked libcloud from git and make packages from that.
radhika [Thu, 3 Sep 2015 19:55:31 +0000 (15:55 -0400)]
Merge branch 'master' into 6260-test-datamanager
Ward Vandewege [Thu, 3 Sep 2015 14:06:16 +0000 (10:06 -0400)]
Another tweak to the scopes required for run-deploy.sh to do its thing.
No issue #
Ward Vandewege [Wed, 2 Sep 2015 21:05:35 +0000 (17:05 -0400)]
Fix documented scope needed for run-deploy.sh
No issue #
Tom Clegg [Wed, 2 Sep 2015 06:48:18 +0000 (02:48 -0400)]
6263: Skip re-installing virtualenv if it is already set up. Upgrade pip along with setuptools.
Tom Clegg [Wed, 2 Sep 2015 04:49:33 +0000 (00:49 -0400)]
6263: Check for gitolite in sanity checks.
Ward Vandewege [Tue, 1 Sep 2015 20:57:06 +0000 (16:57 -0400)]
Merge branch '6939-package-sso-server'
closes #6939
Ward Vandewege [Tue, 1 Sep 2015 20:56:17 +0000 (16:56 -0400)]
Merge branch 'master' into 6939-package-sso-server
Conflicts:
jenkins/run-build-packages-all-targets.sh
refs #6939
Ward Vandewege [Tue, 1 Sep 2015 18:52:01 +0000 (14:52 -0400)]
Split out shared functions into a new run-library.sh.
refs #6939
Ward Vandewege [Tue, 1 Sep 2015 18:33:18 +0000 (14:33 -0400)]
First batch of improvements based on review feedback.
refs #6939
Ward Vandewege [Mon, 31 Aug 2015 21:04:23 +0000 (17:04 -0400)]
Add build script for the new Arvados SSO server package.
Also make the run-build-packages-all-targets.sh script a bit smarter so
that we can use it to build the SSO server package.
Add a run-build-packages-one-target.sh script to build any of our
Arvados packages for just one target.
refs #6939
Ward Vandewege [Fri, 28 Aug 2015 15:37:35 +0000 (11:37 -0400)]
Optimization: do not run bundle install for the API server package when
we aren't including the vendor directory in the package we're building.
No issue #
Ward Vandewege [Fri, 28 Aug 2015 15:37:35 +0000 (11:37 -0400)]
Optimization: do not run bundle install for the API server package when
we aren't including the vendor directory in the package we're building.
No issue #
Peter Amstutz [Fri, 28 Aug 2015 14:40:06 +0000 (10:40 -0400)]
Preinstall forked version of libcloud, because nodemanager "pip install" won't
pick it up by default. refs #6507
Ward Vandewege [Fri, 28 Aug 2015 13:51:43 +0000 (09:51 -0400)]
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
No issue #
Ward Vandewege [Fri, 28 Aug 2015 13:51:43 +0000 (09:51 -0400)]
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
No issue #
Tom Clegg [Thu, 27 Aug 2015 19:47:45 +0000 (15:47 -0400)]
6263: Generate Go coverage reports in $WORKSPACE/tmp/coverage-*
Ward Vandewege [Thu, 27 Aug 2015 18:51:29 +0000 (14:51 -0400)]
Update/clarify package assumptions for api server and workbench.
No issue #
Tom Clegg [Wed, 26 Aug 2015 15:29:32 +0000 (11:29 -0400)]
Merge branch '6781-arvados-login-sync' refs #6781
Ward Vandewege [Wed, 19 Aug 2015 18:49:00 +0000 (14:49 -0400)]
Stop building broken debs for our gems. We don't use them and they have
never worked. We can reintroduce a working version later.
refs #6781
Tom Clegg [Thu, 13 Aug 2015 15:02:51 +0000 (11:02 -0400)]
6781: Add arvados-login-sync.
radhika [Tue, 25 Aug 2015 16:38:35 +0000 (12:38 -0400)]
6260: add datamanager to run-tests.sh
Nico Cesar [Mon, 24 Aug 2015 21:35:20 +0000 (17:35 -0400)]
7041: pyvcf was missing from backports in trusty
refs #7041
Brett Smith [Thu, 20 Aug 2015 15:20:54 +0000 (11:20 -0400)]
6940: Use nginx as WWW_OWNER on Red Hat distributions.
We recommend deploying with Nginx, and this script interacts with
Nginx. The default Nginx configuration that Phusion ships runs as the
nginx user on these distros. Use that as the WWW_OWNER.
Ideally we want to move to a more sophisticated place where we
introspect the Web server configuration and use the same server it
does. But for now, this is an improvement.
No issue #, but discovered when working on #6940. I cleared this with
Ward.
Tom Clegg [Fri, 14 Aug 2015 21:32:58 +0000 (17:32 -0400)]
Merge branch '6638-backport-deps' refs #6638
Ward Vandewege [Fri, 14 Aug 2015 19:22:07 +0000 (15:22 -0400)]
Make sure to also deploy switchyard.
No issue #
Tom Clegg [Fri, 14 Aug 2015 18:37:12 +0000 (14:37 -0400)]
6638: Use (git clone; git checkout) -- git clone --branch doesn't work with commit hashes.
Ward Vandewege [Fri, 14 Aug 2015 00:38:53 +0000 (20:38 -0400)]
Performance optimization for the run_upload_packages.py script: only
rebuild the freight cache for the distribution that is being updated.
No issue #
Tom Clegg [Thu, 13 Aug 2015 21:06:29 +0000 (17:06 -0400)]
6934: Add sdk/pam to run_upload_packages.py refs #6934
Tom Clegg [Wed, 12 Aug 2015 20:10:17 +0000 (16:10 -0400)]
6638: Fix rvm usage error. Straighten out "need new gem" and "need to build new package from gem" logic.
Tom Clegg [Wed, 12 Aug 2015 20:09:08 +0000 (16:09 -0400)]
6638: Add postgres libraries to build containers, so bundler can build the pg gem.
Tom Clegg [Wed, 12 Aug 2015 16:01:29 +0000 (12:01 -0400)]
6638: Build current version of arvados-src, not github
Tom Clegg [Wed, 12 Aug 2015 16:05:59 +0000 (12:05 -0400)]
6638: Read package dependencies from fpm-info.sh in source tree.
Tom Clegg [Wed, 12 Aug 2015 13:31:32 +0000 (09:31 -0400)]
6638: Dry up debug printfs.
Tom Clegg [Thu, 13 Aug 2015 03:06:15 +0000 (23:06 -0400)]
Merge branch '6934-pam' refs #6934
Tom Clegg [Thu, 13 Aug 2015 02:36:13 +0000 (22:36 -0400)]
6934: Build libpam-arvados only for targets that have libpam-python.
Tom Clegg [Wed, 12 Aug 2015 20:19:06 +0000 (16:19 -0400)]
6934: Add libpam-python dependency on all targets.
Tom Clegg [Wed, 12 Aug 2015 17:57:06 +0000 (13:57 -0400)]
6934: Merge branch 'master' into 6934-pam
Tom Clegg [Mon, 10 Aug 2015 14:41:00 +0000 (10:41 -0400)]
6934: Package libpam-arvados using Python recipe. Add sdk/pam to run-tests.
Ward Vandewege [Fri, 7 Aug 2015 14:56:45 +0000 (10:56 -0400)]
Make sure that we build arvados-src package with the correct version of
the git tree, not latest.
No issue #
Ward Vandewege [Wed, 5 Aug 2015 02:00:51 +0000 (22:00 -0400)]
Fix logic bug in run-deploy: deploy api server first before gathering
the git revision from it to make sure the correct arvados/jobs docker
image is installed.
No issue #
Tom Clegg [Thu, 30 Jul 2015 19:38:03 +0000 (15:38 -0400)]
Merge branch '6592-test-perl' refs #6592
Tom Clegg [Thu, 30 Jul 2015 15:44:44 +0000 (11:44 -0400)]
6592: Build and install the Perl SDK so sdk/cli tests can use it.
Brett Smith [Mon, 27 Jul 2015 14:50:32 +0000 (10:50 -0400)]
Rebuild Python packages before building their distro packages.
Without this, built scripts lay around between builds, with shebang
lines for the first distro they were built for. This can lead to
problems like Debian packages getting shebang lines for Red Hat
systems. See, e.g.,
<https://ci.curoverse.com/job/deploy-to-staging/1250/console>.
No issue #.
Peter Amstutz [Mon, 27 Jul 2015 13:19:46 +0000 (09:19 -0400)]
Fix parameter ordering for minor change in cwltool command line semantics. no issue #
Brett Smith [Mon, 27 Jul 2015 12:55:43 +0000 (08:55 -0400)]
Merge branch '6691-trusty-packages-wip'
Refs #6691. Closes #6742.
Brett Smith [Thu, 23 Jul 2015 21:10:54 +0000 (17:10 -0400)]
6691: Build and ship packages for Ubuntu 14.04.
Brett Smith [Fri, 24 Jul 2015 16:08:09 +0000 (12:08 -0400)]
Merge branch '6682-use-docker-build-cmd-wip'
Refs #6682. Closes #6741.
Brett Smith [Thu, 23 Jul 2015 02:21:59 +0000 (22:21 -0400)]
6682: Add script to orchestrate builds for multiple targets.
Now that we can build packages for multiple targets, it's nice to have
a one-shot script to build them all. This does that.
Brett Smith [Thu, 23 Jul 2015 20:59:11 +0000 (16:59 -0400)]
6497: Bugfix distro names for DebianPackageSuites.
freight is expecting a codename like wheezy or jessie, but we've been
using our internal distro+version names there. This fixes that.
Nico gave a thumbs up to this patch. Refs #6497.
Nico Cesar [Thu, 23 Jul 2015 16:55:13 +0000 (12:55 -0400)]
one of the scopes wasnt working
closes #6700
Nico Cesar [Thu, 23 Jul 2015 14:53:41 +0000 (10:53 -0400)]
scopes in usage() are wrong.
refs #6700
Brett Smith [Thu, 23 Jul 2015 14:07:54 +0000 (10:07 -0400)]
6750: arvados-api-server-upgrade only installs omniauth.rb if it exists.
New installs configure the SSO server via application.yml. Refs #6750.
Brett Smith [Wed, 22 Jul 2015 20:38:33 +0000 (16:38 -0400)]
Merge branch '6497-separate-package-upload-wip'
Closes #6497, #6533, #6498.
Brett Smith [Wed, 22 Jul 2015 20:38:23 +0000 (16:38 -0400)]
6497: Separate package build and uploader scripts.
This lets us avoid redundant uploads as we build for multiple targets,
and lets us better customize the upload process per target.
run-build-packages had code to avoid doing an upload when there was no
new package to build. run_upload_packages uses a timestamp file to
achieve a similar result.
Other changes to run-build-packages to support this:
* Better detection of when Python and Ruby packages have already been
built.
* Refactor Gem building into a function to fix various bugs (wrong
version checked for existing arvados-cli; distro packages not
installed in the proper directory).
* Refactor git log querying to reduce code duplication.
Nico Cesar [Tue, 21 Jul 2015 14:10:24 +0000 (10:10 -0400)]
added exit if git pull fails
no issue #
Brett Smith [Mon, 20 Jul 2015 18:36:46 +0000 (14:36 -0400)]
6656: python-arvados-python-client Recommends: git.
Because arv-copy uses it. Refs #6656.
projects / arvados-dev.git / log