radhika [Tue, 8 Sep 2015 16:34:54 +0000 (12:34 -0400)]
Merge branch 'master' into 6260-test-datamanager
Peter Amstutz [Tue, 8 Sep 2015 13:44:21 +0000 (09:44 -0400)]
Merge branch '7031-build-forked-libcloud' refs #7031
Peter Amstutz [Tue, 8 Sep 2015 13:43:20 +0000 (09:43 -0400)]
Merge branch 'master' into 7031-build-forked-libcloud
Tom Clegg [Mon, 7 Sep 2015 18:13:18 +0000 (14:13 -0400)]
Merge branch '6263-go-coverage' refs #6263
Tom Clegg [Mon, 7 Sep 2015 18:10:53 +0000 (14:10 -0400)]
Merge branch '6263-gitolite-test' refs #6263
Ward Vandewege [Fri, 4 Sep 2015 15:04:17 +0000 (11:04 -0400)]
Make sure the library can always be found.
No issue #
Peter Amstutz [Fri, 4 Sep 2015 14:00:23 +0000 (10:00 -0400)]
Bump version to dev2
Peter Amstutz [Mon, 31 Aug 2015 18:10:53 +0000 (14:10 -0400)]
7031: Fix python prefix
Peter Amstutz [Mon, 31 Aug 2015 17:38:37 +0000 (13:38 -0400)]
7031: Check out our forked libcloud from git and make packages from that.
radhika [Thu, 3 Sep 2015 19:55:31 +0000 (15:55 -0400)]
Merge branch 'master' into 6260-test-datamanager
Ward Vandewege [Thu, 3 Sep 2015 14:06:16 +0000 (10:06 -0400)]
Another tweak to the scopes required for run-deploy.sh to do its thing.
No issue #
Ward Vandewege [Wed, 2 Sep 2015 21:05:35 +0000 (17:05 -0400)]
Fix documented scope needed for run-deploy.sh
No issue #
Tom Clegg [Wed, 2 Sep 2015 06:48:18 +0000 (02:48 -0400)]
6263: Skip re-installing virtualenv if it is already set up. Upgrade pip along with setuptools.
Tom Clegg [Wed, 2 Sep 2015 04:49:33 +0000 (00:49 -0400)]
6263: Check for gitolite in sanity checks.
Ward Vandewege [Tue, 1 Sep 2015 20:57:06 +0000 (16:57 -0400)]
Merge branch '6939-package-sso-server'
closes #6939
Ward Vandewege [Tue, 1 Sep 2015 20:56:17 +0000 (16:56 -0400)]
Merge branch 'master' into 6939-package-sso-server
Conflicts:
jenkins/run-build-packages-all-targets.sh
refs #6939
Ward Vandewege [Tue, 1 Sep 2015 18:52:01 +0000 (14:52 -0400)]
Split out shared functions into a new run-library.sh.
refs #6939
Ward Vandewege [Tue, 1 Sep 2015 18:33:18 +0000 (14:33 -0400)]
First batch of improvements based on review feedback.
refs #6939
Ward Vandewege [Mon, 31 Aug 2015 21:04:23 +0000 (17:04 -0400)]
Add build script for the new Arvados SSO server package.
Also make the run-build-packages-all-targets.sh script a bit smarter so
that we can use it to build the SSO server package.
Add a run-build-packages-one-target.sh script to build any of our
Arvados packages for just one target.
refs #6939
Ward Vandewege [Fri, 28 Aug 2015 15:37:35 +0000 (11:37 -0400)]
Optimization: do not run bundle install for the API server package when
we aren't including the vendor directory in the package we're building.
No issue #
Ward Vandewege [Fri, 28 Aug 2015 15:37:35 +0000 (11:37 -0400)]
Optimization: do not run bundle install for the API server package when
we aren't including the vendor directory in the package we're building.
No issue #
Peter Amstutz [Fri, 28 Aug 2015 14:40:06 +0000 (10:40 -0400)]
Preinstall forked version of libcloud, because nodemanager "pip install" won't
pick it up by default. refs #6507
Ward Vandewege [Fri, 28 Aug 2015 13:51:43 +0000 (09:51 -0400)]
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
No issue #
Ward Vandewege [Fri, 28 Aug 2015 13:51:43 +0000 (09:51 -0400)]
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
No issue #
Tom Clegg [Thu, 27 Aug 2015 19:47:45 +0000 (15:47 -0400)]
6263: Generate Go coverage reports in $WORKSPACE/tmp/coverage-*
Ward Vandewege [Thu, 27 Aug 2015 18:51:29 +0000 (14:51 -0400)]
Update/clarify package assumptions for api server and workbench.
No issue #
Tom Clegg [Wed, 26 Aug 2015 15:29:32 +0000 (11:29 -0400)]
Merge branch '6781-arvados-login-sync' refs #6781
Ward Vandewege [Wed, 19 Aug 2015 18:49:00 +0000 (14:49 -0400)]
Stop building broken debs for our gems. We don't use them and they have
never worked. We can reintroduce a working version later.
refs #6781
Tom Clegg [Thu, 13 Aug 2015 15:02:51 +0000 (11:02 -0400)]
6781: Add arvados-login-sync.
radhika [Tue, 25 Aug 2015 16:38:35 +0000 (12:38 -0400)]
6260: add datamanager to run-tests.sh
Nico Cesar [Mon, 24 Aug 2015 21:35:20 +0000 (17:35 -0400)]
7041: pyvcf was missing from backports in trusty
refs #7041
Brett Smith [Thu, 20 Aug 2015 15:20:54 +0000 (11:20 -0400)]
6940: Use nginx as WWW_OWNER on Red Hat distributions.
We recommend deploying with Nginx, and this script interacts with
Nginx. The default Nginx configuration that Phusion ships runs as the
nginx user on these distros. Use that as the WWW_OWNER.
Ideally we want to move to a more sophisticated place where we
introspect the Web server configuration and use the same server it
does. But for now, this is an improvement.
No issue #, but discovered when working on #6940. I cleared this with
Ward.
Tom Clegg [Fri, 14 Aug 2015 21:32:58 +0000 (17:32 -0400)]
Merge branch '6638-backport-deps' refs #6638
Ward Vandewege [Fri, 14 Aug 2015 19:22:07 +0000 (15:22 -0400)]
Make sure to also deploy switchyard.
No issue #
Tom Clegg [Fri, 14 Aug 2015 18:37:12 +0000 (14:37 -0400)]
6638: Use (git clone; git checkout) -- git clone --branch doesn't work with commit hashes.
Ward Vandewege [Fri, 14 Aug 2015 00:38:53 +0000 (20:38 -0400)]
Performance optimization for the run_upload_packages.py script: only
rebuild the freight cache for the distribution that is being updated.
No issue #
Tom Clegg [Thu, 13 Aug 2015 21:06:29 +0000 (17:06 -0400)]
6934: Add sdk/pam to run_upload_packages.py refs #6934
Tom Clegg [Wed, 12 Aug 2015 20:10:17 +0000 (16:10 -0400)]
6638: Fix rvm usage error. Straighten out "need new gem" and "need to build new package from gem" logic.
Tom Clegg [Wed, 12 Aug 2015 20:09:08 +0000 (16:09 -0400)]
6638: Add postgres libraries to build containers, so bundler can build the pg gem.
Tom Clegg [Wed, 12 Aug 2015 16:01:29 +0000 (12:01 -0400)]
6638: Build current version of arvados-src, not github
Tom Clegg [Wed, 12 Aug 2015 16:05:59 +0000 (12:05 -0400)]
6638: Read package dependencies from fpm-info.sh in source tree.
Tom Clegg [Wed, 12 Aug 2015 13:31:32 +0000 (09:31 -0400)]
6638: Dry up debug printfs.
Tom Clegg [Thu, 13 Aug 2015 03:06:15 +0000 (23:06 -0400)]
Merge branch '6934-pam' refs #6934
Tom Clegg [Thu, 13 Aug 2015 02:36:13 +0000 (22:36 -0400)]
6934: Build libpam-arvados only for targets that have libpam-python.
Tom Clegg [Wed, 12 Aug 2015 20:19:06 +0000 (16:19 -0400)]
6934: Add libpam-python dependency on all targets.
Tom Clegg [Wed, 12 Aug 2015 17:57:06 +0000 (13:57 -0400)]
6934: Merge branch 'master' into 6934-pam
Tom Clegg [Mon, 10 Aug 2015 14:41:00 +0000 (10:41 -0400)]
6934: Package libpam-arvados using Python recipe. Add sdk/pam to run-tests.
Ward Vandewege [Fri, 7 Aug 2015 14:56:45 +0000 (10:56 -0400)]
Make sure that we build arvados-src package with the correct version of
the git tree, not latest.
No issue #
Ward Vandewege [Wed, 5 Aug 2015 02:00:51 +0000 (22:00 -0400)]
Fix logic bug in run-deploy: deploy api server first before gathering
the git revision from it to make sure the correct arvados/jobs docker
image is installed.
No issue #
Tom Clegg [Thu, 30 Jul 2015 19:38:03 +0000 (15:38 -0400)]
Merge branch '6592-test-perl' refs #6592
Tom Clegg [Thu, 30 Jul 2015 15:44:44 +0000 (11:44 -0400)]
6592: Build and install the Perl SDK so sdk/cli tests can use it.
Brett Smith [Mon, 27 Jul 2015 14:50:32 +0000 (10:50 -0400)]
Rebuild Python packages before building their distro packages.
Without this, built scripts lay around between builds, with shebang
lines for the first distro they were built for. This can lead to
problems like Debian packages getting shebang lines for Red Hat
systems. See, e.g.,
<https://ci.curoverse.com/job/deploy-to-staging/1250/console>.
No issue #.
Peter Amstutz [Mon, 27 Jul 2015 13:19:46 +0000 (09:19 -0400)]
Fix parameter ordering for minor change in cwltool command line semantics. no issue #
Brett Smith [Mon, 27 Jul 2015 12:55:43 +0000 (08:55 -0400)]
Merge branch '6691-trusty-packages-wip'
Refs #6691. Closes #6742.
Brett Smith [Thu, 23 Jul 2015 21:10:54 +0000 (17:10 -0400)]
6691: Build and ship packages for Ubuntu 14.04.
Brett Smith [Fri, 24 Jul 2015 16:08:09 +0000 (12:08 -0400)]
Merge branch '6682-use-docker-build-cmd-wip'
Refs #6682. Closes #6741.
Brett Smith [Thu, 23 Jul 2015 02:21:59 +0000 (22:21 -0400)]
6682: Add script to orchestrate builds for multiple targets.
Now that we can build packages for multiple targets, it's nice to have
a one-shot script to build them all. This does that.
Brett Smith [Thu, 23 Jul 2015 20:59:11 +0000 (16:59 -0400)]
6497: Bugfix distro names for DebianPackageSuites.
freight is expecting a codename like wheezy or jessie, but we've been
using our internal distro+version names there. This fixes that.
Nico gave a thumbs up to this patch. Refs #6497.
Nico Cesar [Thu, 23 Jul 2015 16:55:13 +0000 (12:55 -0400)]
one of the scopes wasnt working
closes #6700
Nico Cesar [Thu, 23 Jul 2015 14:53:41 +0000 (10:53 -0400)]
scopes in usage() are wrong.
refs #6700
Brett Smith [Thu, 23 Jul 2015 14:07:54 +0000 (10:07 -0400)]
6750: arvados-api-server-upgrade only installs omniauth.rb if it exists.
New installs configure the SSO server via application.yml. Refs #6750.
Brett Smith [Wed, 22 Jul 2015 20:38:33 +0000 (16:38 -0400)]
Merge branch '6497-separate-package-upload-wip'
Closes #6497, #6533, #6498.
Brett Smith [Wed, 22 Jul 2015 20:38:23 +0000 (16:38 -0400)]
6497: Separate package build and uploader scripts.
This lets us avoid redundant uploads as we build for multiple targets,
and lets us better customize the upload process per target.
run-build-packages had code to avoid doing an upload when there was no
new package to build. run_upload_packages uses a timestamp file to
achieve a similar result.
Other changes to run-build-packages to support this:
* Better detection of when Python and Ruby packages have already been
built.
* Refactor Gem building into a function to fix various bugs (wrong
version checked for existing arvados-cli; distro packages not
installed in the proper directory).
* Refactor git log querying to reduce code duplication.
Nico Cesar [Tue, 21 Jul 2015 14:10:24 +0000 (10:10 -0400)]
added exit if git pull fails
no issue #
Brett Smith [Mon, 20 Jul 2015 18:36:46 +0000 (14:36 -0400)]
6656: python-arvados-python-client Recommends: git.
Because arv-copy uses it. Refs #6656.
Ward Vandewege [Thu, 16 Jul 2015 22:02:30 +0000 (18:02 -0400)]
run-deploy.sh improvements: remove the need for a .ssh/config entry for
the shortname of each API server. Thanks for the suggestion Nico.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 03:14:23 +0000 (23:14 -0400)]
Make sure jessie packages end up in the jessie repository.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 02:59:28 +0000 (22:59 -0400)]
We need to force-tag the arvados/jobs docker image.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 02:53:50 +0000 (22:53 -0400)]
freight is already smart enough to do atomic updates.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 01:55:21 +0000 (21:55 -0400)]
Add dockerfiles/build-all-build-containers.sh script
refs #6504
Peter Amstutz [Wed, 15 Jul 2015 17:29:00 +0000 (13:29 -0400)]
Add datamanager tests. refs #3408
Ward Vandewege [Wed, 15 Jul 2015 17:14:34 +0000 (13:14 -0400)]
Build python3-websocket-client packages, which is now a dependency for
python3-docker-py, which is a dependency for arvados-docker-cleaner.
No issue #
Ward Vandewege [Tue, 14 Jul 2015 01:02:07 +0000 (21:02 -0400)]
Make sure we have a conference/environments/production.rb file so that
we generate the workbench assets correctly before packaging.
refs #6362
Ward Vandewege [Mon, 13 Jul 2015 15:39:25 +0000 (11:39 -0400)]
run_deploy.sh improvements:
* Make sure that group_class is included when creating the public docker
images project.
* Document the required permissions for the arvados token needed to run
run_deploy.sh
closes #6586
Ward Vandewege [Fri, 10 Jul 2015 18:07:39 +0000 (14:07 -0400)]
Add debian8 as a build target.
No issue #
Ward Vandewege [Fri, 10 Jul 2015 02:01:36 +0000 (22:01 -0400)]
Build arvados/jobs docker image separately from the others.
refs #6528
Ward Vandewege [Wed, 8 Jul 2015 20:31:35 +0000 (16:31 -0400)]
Add dependency on libpam-python to the libarvados-pam package.
refs #2256
Ward Vandewege [Wed, 8 Jul 2015 18:56:07 +0000 (14:56 -0400)]
Add libpam-arvados package build to run-build-packages.
refs #6384
Brett Smith [Wed, 8 Jul 2015 14:09:25 +0000 (10:09 -0400)]
Merge branch '6389-add-dockerfiles'
Closes #6389, #6456.
Brett Smith [Mon, 29 Jun 2015 20:59:24 +0000 (16:59 -0400)]
6388: Add Dockerfile to build packages on CentOS 6.
Brett Smith [Mon, 29 Jun 2015 20:56:57 +0000 (16:56 -0400)]
6389: Add Makefile to support Docker builds.
Brett Smith [Mon, 29 Jun 2015 20:56:16 +0000 (16:56 -0400)]
6389: Dockerfiles include the CMD to build packages.
Ward Vandewege [Fri, 26 Jun 2015 21:07:32 +0000 (17:07 -0400)]
Add dockerfiles for Debian7 and Ubuntu1204
refs #6389
Ward Vandewege [Wed, 8 Jul 2015 13:59:54 +0000 (09:59 -0400)]
Fix up git log statements:
* remove duplication between -n1 and --max-count=1
* make sure every git log statement has --first-parent
refs #6096
Ward Vandewege [Wed, 8 Jul 2015 02:05:01 +0000 (22:05 -0400)]
Small fixes for run-deploy.sh
refs #6096
Ward Vandewege [Thu, 11 Jun 2015 01:30:45 +0000 (21:30 -0400)]
Make run-deploy.rb install the correct version of the arvados/jobs
docker image on deploy.
refs #6096
Brett Smith [Mon, 6 Jul 2015 17:29:44 +0000 (13:29 -0400)]
Merge branch '6388-package-scripts-redhat-support-wip'
Refs #6388. Closes #6480.
Brett Smith [Mon, 6 Jul 2015 14:26:44 +0000 (10:26 -0400)]
6388: Extend API server upgrade script to do initial database setup.
This will let administrators duplicating a lot of logic that already
exists in the script.
Brett Smith [Sun, 5 Jul 2015 22:11:23 +0000 (18:11 -0400)]
6388: Rails package upgrade scripts support Red Hat systems.
* Red Hat doesn't use the www-data account name.
* When nginx is installed from a Software Collection, it may have a
name like `nginx16`. Detect and use this.
Brett Smith [Sun, 5 Jul 2015 15:55:25 +0000 (11:55 -0400)]
Bugfix config:check error detection in Rails upgrade scripts.
The previous code was capturing the exit status of echo, which isn't
very interesting. No issue #.
Tom Clegg [Tue, 30 Jun 2015 18:53:47 +0000 (14:53 -0400)]
6377: Don't try to uninstall arvados* gems if they are not installed.
Fixes run-tests failure when no Ruby SDK is installed on the test host
and tmpdirs are new/empty. refs #6377
Tom Clegg [Tue, 30 Jun 2015 18:50:50 +0000 (14:50 -0400)]
6377: Create user-specified tmpdirs if necessary. refs #6377
Ward Vandewege [Mon, 29 Jun 2015 18:18:08 +0000 (14:18 -0400)]
Add rsync to the repo update command in run-build-packages.
No issue #
Ward Vandewege [Fri, 26 Jun 2015 21:09:03 +0000 (17:09 -0400)]
Make sure to also build a backported package for python-llfuse. We need
it for writeable fuse.
refs #3198
Ward Vandewege [Fri, 26 Jun 2015 19:41:47 +0000 (15:41 -0400)]
Update comment references from 'debs' to 'packages'.
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 19:40:26 +0000 (15:40 -0400)]
Drop built packages into packages/$TARGET instead of the old 'debs' directory.
Also, fix the 'find' test so that built packages are actually added to
the repository after upload.
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 19:04:35 +0000 (15:04 -0400)]
Add ubuntu1204 target to run-build-packages.sh
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 18:55:22 +0000 (14:55 -0400)]
Regex match fixes in the character class match for $FPM_RESULTS:
a) we want to match a literal dot, not any character
b) bash 4.2 does not support escaping a dash to include it in the
character class, move the dash to the last character listed in the class
refs #6389
Brett Smith [Fri, 26 Jun 2015 13:12:22 +0000 (09:12 -0400)]
6388: CentOS needs backports for python-daemon and its dependencies.
Refs #6388.
Peter Amstutz [Fri, 26 Jun 2015 03:39:43 +0000 (23:39 -0400)]
Additionally build and upload cwl-runner package. no issue #
projects / arvados-dev.git / log