Tom Clegg [Wed, 10 Dec 2014 22:45:30 +0000 (17:45 -0500)]
4156: Do not use --user-install -- just install to the GEM_HOME we selected.
Tom Clegg [Wed, 10 Dec 2014 21:59:54 +0000 (16:59 -0500)]
4156: Fix install destination for gem install --user-install. Make tmp-gem-home discovery work under rvm.
Radhika Chippada [Wed, 10 Dec 2014 01:33:05 +0000 (20:33 -0500)]
4754: add test_workbench_performance
Tom Clegg [Fri, 5 Dec 2014 17:59:44 +0000 (12:59 -0500)]
4156: Merge branch 'master' into 4156-bundle-install-user
Ward Vandewege [Wed, 3 Dec 2014 15:40:10 +0000 (10:40 -0500)]
Add timing information to the docker tests.
No issue #
Peter Amstutz [Mon, 1 Dec 2014 20:46:35 +0000 (15:46 -0500)]
Install PyYAML into VENVDIR for non-Python tests that use run_test_server.py. no issue #
Tom Clegg [Thu, 27 Nov 2014 21:32:14 +0000 (16:32 -0500)]
4156: Install dependencies to user gempath, not system gempath, even
if not using rvm. Without this, run-tests asks for a sudo password
whenever "bundle install" has any work to do, which is especially
disruptive when two tested components have differing versions of the
arvados gem listed in their Gemfile.lock.
Ward Vandewege [Wed, 26 Nov 2014 19:23:03 +0000 (14:23 -0500)]
Pull the common-workflow-language code from the new
common-workflow-language github organization.
No issue #
Tom Clegg [Tue, 25 Nov 2014 02:14:00 +0000 (21:14 -0500)]
Ignore failure at "easy_install *.egg". refs #4400
Ward Vandewege [Mon, 24 Nov 2014 18:09:01 +0000 (13:09 -0500)]
Make sure that our go packages get generated with a version number that
maps the last change to their code (i.e. the git hash for the directory
their source lives in), just like all our other packages.
No issue #
Tom Clegg [Mon, 24 Nov 2014 14:55:22 +0000 (09:55 -0500)]
Merge branch '4400-dry-python-tests' refs #4400
Ward Vandewege [Fri, 21 Nov 2014 22:37:43 +0000 (17:37 -0500)]
run-tests: make the error message displayed when WORKSPACE isn't set a
bit prettier.
No issue #
radhika [Fri, 21 Nov 2014 20:04:32 +0000 (15:04 -0500)]
closes #4156
Merge branch '4156-enable-cli-tests'
Tom Clegg [Fri, 21 Nov 2014 16:56:14 +0000 (11:56 -0500)]
4400: Fix syntax error.
radhika [Fri, 21 Nov 2014 13:53:58 +0000 (08:53 -0500)]
4156: Since all the cli tests that require apiserver are skipped, all cli tests pass now. Hence, do not skip running cli tests anymore.
Tom Clegg [Thu, 20 Nov 2014 21:55:41 +0000 (16:55 -0500)]
4400: Dry up test recipes for python bits. Tweak "extra test args for suite X" feature.
Ward Vandewege [Wed, 19 Nov 2014 13:57:18 +0000 (08:57 -0500)]
Refactor deploy script a bit; make it less sensitive to transient errors
we don't care about.
No issue #
Ward Vandewege [Tue, 18 Nov 2014 21:31:54 +0000 (16:31 -0500)]
Do not blow up when another puppet process is already running during deploy.
No issue #
Ward Vandewege [Tue, 18 Nov 2014 21:22:21 +0000 (16:22 -0500)]
Build a python-jsonschema package, because cwltool requires version 2.4
which is newer than what's available in wheezy(-backports).
refs #4572
Ward Vandewege [Tue, 18 Nov 2014 18:26:23 +0000 (13:26 -0500)]
Also build a deb package for python-pyexecjs, which is a dependency for
cwltool
refs #4572
Ward Vandewege [Tue, 18 Nov 2014 17:01:10 +0000 (12:01 -0500)]
Package cwltool.
Refs #4572
Brett Smith [Mon, 17 Nov 2014 21:46:41 +0000 (16:46 -0500)]
Merge branch '4027-sdk-debian-packages-wip'
Refs #4027. Closes #4557.
Brett Smith [Mon, 17 Nov 2014 19:02:06 +0000 (14:02 -0500)]
4027: Build packages for Ruby and Perl SDKs.
Ward Vandewege [Mon, 17 Nov 2014 20:55:01 +0000 (15:55 -0500)]
Retry docker push a few times if it fails.
No issue #
Ward Vandewege [Mon, 17 Nov 2014 15:12:11 +0000 (10:12 -0500)]
Also build a package for python-six.
No issue #
Tom Clegg [Wed, 12 Nov 2014 19:58:05 +0000 (14:58 -0500)]
Export vars provided on command line, e.g., ARVADOS_DEBUG=1. No issue #
Brett Smith [Wed, 12 Nov 2014 17:00:33 +0000 (12:00 -0500)]
Refs #3857: Build modern requests package for PySDK.
Ward Vandewege [Tue, 4 Nov 2014 16:14:18 +0000 (11:14 -0500)]
Small tweaks: fix example for running one API test in run-tests.sh;
capitalize Arvados in easy-docker-install.sh
no issue #
Ward Vandewege [Sun, 2 Nov 2014 00:48:00 +0000 (20:48 -0400)]
Rename easy installation script. Update comments at the top of the script.
refs #4383
Ward Vandewege [Sun, 2 Nov 2014 00:41:48 +0000 (20:41 -0400)]
Add first version of the easy-evaluation script.
refs #4383
Ward Vandewege [Sat, 1 Nov 2014 19:08:28 +0000 (15:08 -0400)]
Make run-docker-tests.sh a bit smarter about detecting common failures.
No issue #
Ward Vandewege [Sat, 1 Nov 2014 00:40:58 +0000 (20:40 -0400)]
Make sure all executables we package are executable.
No issue #
Ward Vandewege [Sat, 1 Nov 2014 00:35:30 +0000 (20:35 -0400)]
Upload our docker images to the docker.io registry.
refs #4383
Ward Vandewege [Fri, 24 Oct 2014 20:06:47 +0000 (16:06 -0400)]
Clean up *.pyc files between tests.
closes #4313
Tom Clegg [Wed, 22 Oct 2014 15:03:40 +0000 (11:03 -0400)]
Add services/crunchstat to test suite. refs #3826
Ward Vandewege [Wed, 22 Oct 2014 13:35:49 +0000 (09:35 -0400)]
Make run-deploy.sh a bit smarter about which compute nodes are actually
up.
Note: there are still race conditions to be addressed here. Or maybe
better, we probably just shouldn't care about compute node puppet runs
not completing succesfully. Either the node will get shut down by node
manager, or a regular hourly puppet run will DTRT.
No issue #
Ward Vandewege [Tue, 21 Oct 2014 13:00:59 +0000 (09:00 -0400)]
* exclude /usr/local/lib/python2.7/dist-packages/tests from our fpm packages
* exclude /usr/local/arvados/src/.git from the arvados-src package (bugfix)
No issue #
Ward Vandewege [Tue, 21 Oct 2014 12:39:06 +0000 (08:39 -0400)]
run-build-packages: Test that fpm is callable before trying to do
anything.
No issue #
Ward Vandewege [Tue, 21 Oct 2014 12:17:38 +0000 (08:17 -0400)]
Build a few more dependency packages for node manager.
refs #3244
Brett Smith [Mon, 20 Oct 2014 20:50:44 +0000 (16:50 -0400)]
4239: Build more dependencies of new google-api-python-client.
Refs #4239.
Ward Vandewege [Fri, 17 Oct 2014 15:18:06 +0000 (11:18 -0400)]
A few improvements for run-build-packages.sh:
* fix suggestion of env variable setting
* add a few new python packages that were split off from
google-api-python-client in version 1.3
* print package name when fpm build goes wrong
No issue #
Peter Amstutz [Wed, 15 Oct 2014 18:25:34 +0000 (14:25 -0400)]
Add blacklist to commit hook. no issue #
Ward Vandewege [Wed, 15 Oct 2014 14:55:32 +0000 (10:55 -0400)]
Make sure to catch the merge commit when doing git log in the directory
that is to be packaged.
No issue #
Ward Vandewege [Tue, 14 Oct 2014 17:17:18 +0000 (13:17 -0400)]
Upload our arvados/jobs image to the Docker Hub.
No issue #
Brett Smith [Tue, 14 Oct 2014 14:15:53 +0000 (10:15 -0400)]
2881: Re-enable Node Manager tests.
The work done in refs #4139 should make these more reliable.
Brett Smith [Fri, 10 Oct 2014 15:38:34 +0000 (11:38 -0400)]
2881: Skip Node Manager tests for now.
See comment for detailed rationale. Refs #2881, #4139.
Brett Smith [Thu, 9 Oct 2014 14:33:45 +0000 (10:33 -0400)]
2881: run-tests tests the Node Manager.
Refs #2881.
Ward Vandewege [Thu, 9 Oct 2014 14:16:46 +0000 (10:16 -0400)]
Switch to human readable timestamps (YYYYMMDDhhmmss in UTC) for all our
debian packages.
No issue #
Ward Vandewege [Thu, 9 Oct 2014 14:12:39 +0000 (10:12 -0400)]
Improve output when fpm call fails.
No issue #
Ward Vandewege [Wed, 8 Oct 2014 21:41:08 +0000 (17:41 -0400)]
More updates for run-build-packages.sh:
* more code cleanups
* create node manager package
No issue #
Ward Vandewege [Wed, 8 Oct 2014 21:22:06 +0000 (17:22 -0400)]
Make run-build-packages.sh much more quiet by default. Also make
uploading optional. Change command line argument syntax to bring it in
line with run-tests.sh.
No issue #
Ward Vandewege [Wed, 8 Oct 2014 14:56:24 +0000 (10:56 -0400)]
Allow for run-diagnostics-suite.sh to pull the git revision to test from
a configuration file.
No issue #
Ward Vandewege [Wed, 8 Oct 2014 03:04:52 +0000 (23:04 -0400)]
Allow specification of revision to test with diagnostics suite.
No issue #
Ward Vandewege [Wed, 8 Oct 2014 01:27:04 +0000 (21:27 -0400)]
Fix stupid bug in run-diagnostics-suite.sh - destination config file
needs to be application.yml
No issue #
Ward Vandewege [Wed, 8 Oct 2014 00:34:07 +0000 (20:34 -0400)]
Parameterize the instance to run the diagnostics suite against.
No issue #
Brett Smith [Sun, 5 Oct 2014 20:21:39 +0000 (16:21 -0400)]
Merge branch '3848-build-packages-pyversion'
Refs #3848. Closes #4110.
Brett Smith [Sun, 5 Oct 2014 18:34:10 +0000 (14:34 -0400)]
3848: build-packages respects egg-info versions.
Now that our setup.py files know how to introspect git and generate
build tags, build-packages can use that version number rather than
overriding it.
Ward Vandewege [Sun, 5 Oct 2014 14:18:26 +0000 (10:18 -0400)]
Remove spurious newline
No issue #
Ward Vandewege [Sun, 5 Oct 2014 14:17:01 +0000 (10:17 -0400)]
Add README, COPYING and agpl-3.0.txt.
No issue #
Ward Vandewege [Sat, 4 Oct 2014 01:29:43 +0000 (21:29 -0400)]
Make run-deploy.sh do a bit more work to update the rest of the staging
cluster.
No issue #
Ward Vandewege [Fri, 3 Oct 2014 00:15:35 +0000 (20:15 -0400)]
Deal with Jenkins' new, tight umask in a more thorough way.
No issue #
Ward Vandewege [Wed, 1 Oct 2014 21:22:51 +0000 (17:22 -0400)]
Make sure our generated packages are world-readable, otherwise they
won't be useful in the package repo.
No issue #
Ward Vandewege [Wed, 1 Oct 2014 20:28:42 +0000 (16:28 -0400)]
Add new run-diagnostics-suite.sh jenkins script.
No issue #
Tim Pierce [Tue, 30 Sep 2014 17:27:41 +0000 (13:27 -0400)]
Merge branch '4038-commit-hooks-accept-fixes'
Closes #4038.
Tim Pierce [Tue, 30 Sep 2014 17:10:32 +0000 (13:10 -0400)]
4038: add "fixes #" to acceptable merge commit messages
Brett Smith [Mon, 29 Sep 2014 16:25:53 +0000 (12:25 -0400)]
3848: Stop manually tagging Python builds.
This is now done in setup.py. Refs #3848.
Ward Vandewege [Sat, 27 Sep 2014 01:40:08 +0000 (21:40 -0400)]
Improvements to our package building script: fill in more of the
metadata fields on the deb packages.
No issue #
Brett Smith [Tue, 23 Sep 2014 14:56:57 +0000 (10:56 -0400)]
run-tests.sh uses virtualenv's activate script.
The activate script takes care of additional isolation steps, like
clearing $PYTHONHOME and rehashing $PATH. Prefer that over simply
setting $PATH ourselves. No issue #.
Tom Clegg [Tue, 23 Sep 2014 14:21:30 +0000 (10:21 -0400)]
Fix another -z that should have been -n. refs #3894
Tom Clegg [Mon, 22 Sep 2014 20:34:22 +0000 (16:34 -0400)]
Merge branch '3894-improve-run-tests' closes #3894
Tom Clegg [Mon, 22 Sep 2014 19:40:00 +0000 (15:40 -0400)]
3894: Add comments about CONFIGSRC in --help text.
Tom Clegg [Mon, 22 Sep 2014 19:37:03 +0000 (15:37 -0400)]
3894: Fix -n that should have been -z. Flatten condition.
Tom Clegg [Mon, 22 Sep 2014 19:32:50 +0000 (15:32 -0400)]
3894: Use --first-parent and readable timestamp in fuse package as
well. Use "tr -dc | cut" to eliminate the horrible list of byte
positions. Fix git command to use the desired human-readable timestamp
format.
Tom Clegg [Mon, 22 Sep 2014 15:07:13 +0000 (11:07 -0400)]
3894: Use readable timestamp (yyyymmddhhmmss) for python package
version, and use first-parent to ensure each merge to master results
in a higher version number.
Tom Clegg [Mon, 22 Sep 2014 15:05:39 +0000 (11:05 -0400)]
3894: Remove bundle install stage from sdk/cli.
Tom Clegg [Mon, 22 Sep 2014 15:04:08 +0000 (11:04 -0400)]
3894: Move report_outcomes to a function.
Tom Clegg [Sat, 20 Sep 2014 03:39:11 +0000 (23:39 -0400)]
3894: Use $GEMHOME tmpdir instead of user's $HOME for userspace gem installs.
Tom Clegg [Sat, 20 Sep 2014 03:35:30 +0000 (23:35 -0400)]
3894: Add --help message.
Ward Vandewege [Fri, 19 Sep 2014 20:46:10 +0000 (16:46 -0400)]
Make regexp to match fpm's output slightly more robust.
no issue #
Ward Vandewege [Fri, 19 Sep 2014 20:23:16 +0000 (16:23 -0400)]
Switch from PRM to freight for publishing our packages. It's way, way
faster.
no issue #
Ward Vandewege [Fri, 19 Sep 2014 16:05:41 +0000 (12:05 -0400)]
Copy only the debs that were generated to the remote server.
No issue #
Tom Clegg [Fri, 19 Sep 2014 14:57:20 +0000 (10:57 -0400)]
3894: Fix sequence: do all installs, then all tests.
Tom Clegg [Fri, 19 Sep 2014 14:56:27 +0000 (10:56 -0400)]
3894: Use gem install --user-install to avoid sudo/rvm requirement
Tom Clegg [Fri, 19 Sep 2014 14:54:11 +0000 (10:54 -0400)]
3894: Scrub PYTHONPATH
Tom Clegg [Fri, 19 Sep 2014 14:26:53 +0000 (10:26 -0400)]
3894: Use mktemp instead of $WORKSPACE/tmpgit for test repo
Tom Clegg [Thu, 18 Sep 2014 21:00:58 +0000 (17:00 -0400)]
3894: Fix backward empty/nonempty logic.
Tom Clegg [Thu, 18 Sep 2014 17:35:06 +0000 (13:35 -0400)]
3894: Do not create and abandon new temp dirs if caller has already
provided existing ones. Never delete temp dirs supplied by caller even
if the caller (like the help text) forgets to specify --leave-temp.
Tom Clegg [Thu, 18 Sep 2014 17:14:50 +0000 (13:14 -0400)]
3894: Accept WORKSPACE=foo on command line. Copy config from CONFIGSRC
instead of hard-coded location. Skip this if they don't exist in the
previously hard-coded location and CONFIGSRC is not given.
Ward Vandewege [Wed, 17 Sep 2014 03:14:56 +0000 (23:14 -0400)]
Undo two hunks that I accidentally committed in
565c31196f3ccd4346c8a1ac1e19b1deda22eea6.
no issue #
Ward Vandewege [Wed, 17 Sep 2014 01:42:47 +0000 (21:42 -0400)]
Add some robustness in the postgresql department.
no issue #
Ward Vandewege [Tue, 16 Sep 2014 21:01:29 +0000 (17:01 -0400)]
Only install the latest gem (i.e. the one we just built).
No issue #
Ward Vandewege [Tue, 16 Sep 2014 20:52:01 +0000 (16:52 -0400)]
Also build our java-bwa-samtools docker image.
no issue #
Ward Vandewege [Fri, 12 Sep 2014 20:33:14 +0000 (16:33 -0400)]
Fixes for the deploy stage of the build pipeline: actually upgrade our
debian packages and gems.
no issue #
Ward Vandewege [Sat, 6 Sep 2014 04:06:15 +0000 (00:06 -0400)]
Save a bit of time: do not install RI and RDoc for the gems we install
during testing.
no issue #
Brett Smith [Thu, 4 Sep 2014 20:13:31 +0000 (16:13 -0400)]
Install & test Ruby SDK, then use that version for other tests.
This commit gives our Ruby SDK the same treatment as other parts of
our build process: we build and install it early, then test it. We
use that built version to run other tests, to make sure all the
components are synced up throughout the run.
This required turning off `bundle install --deployment`, because
Bundler will not use locally installed Gems in that mode. This does
make our build process a little less like what we use in production,
but the benefits of consistent testing and tightening the build
loop (you can update a Gem and dependent Gemfiles in one push) seem
worth that cost.
Refs #3720.
Peter Amstutz [Thu, 4 Sep 2014 21:11:09 +0000 (17:11 -0400)]
Fix environment variable setting on command line. no issue #
Peter Amstutz [Thu, 28 Aug 2014 19:21:31 +0000 (15:21 -0400)]
Fix $leave_temp in clear_temp() no issue #
Peter Amstutz [Tue, 26 Aug 2014 14:43:06 +0000 (10:43 -0400)]
Enhanced run-tests.sh command line supporting choosing specific Python and Ruby
tests, and re-using temp directories. no issue #
Peter Amstutz [Thu, 21 Aug 2014 16:00:09 +0000 (12:00 -0400)]
Declare empty variables up top to avoid bash parsing errors. no issue #
Peter Amstutz [Thu, 21 Aug 2014 14:21:39 +0000 (10:21 -0400)]
Permit setting environment variables on command line. Added workbench_test,
apiserver_test and cli_test variables to provide parameters to the respective
rake tests. no issue #