Peter Amstutz [Mon, 27 Jul 2015 13:19:46 +0000 (09:19 -0400)]
Fix parameter ordering for minor change in cwltool command line semantics. no issue #
Brett Smith [Mon, 27 Jul 2015 12:55:43 +0000 (08:55 -0400)]
Merge branch '6691-trusty-packages-wip'
Refs #6691. Closes #6742.
Brett Smith [Thu, 23 Jul 2015 21:10:54 +0000 (17:10 -0400)]
6691: Build and ship packages for Ubuntu 14.04.
Brett Smith [Fri, 24 Jul 2015 16:08:09 +0000 (12:08 -0400)]
Merge branch '6682-use-docker-build-cmd-wip'
Refs #6682. Closes #6741.
Brett Smith [Thu, 23 Jul 2015 02:21:59 +0000 (22:21 -0400)]
6682: Add script to orchestrate builds for multiple targets.
Now that we can build packages for multiple targets, it's nice to have
a one-shot script to build them all. This does that.
Brett Smith [Thu, 23 Jul 2015 20:59:11 +0000 (16:59 -0400)]
6497: Bugfix distro names for DebianPackageSuites.
freight is expecting a codename like wheezy or jessie, but we've been
using our internal distro+version names there. This fixes that.
Nico gave a thumbs up to this patch. Refs #6497.
Nico Cesar [Thu, 23 Jul 2015 16:55:13 +0000 (12:55 -0400)]
one of the scopes wasnt working
closes #6700
Nico Cesar [Thu, 23 Jul 2015 14:53:41 +0000 (10:53 -0400)]
scopes in usage() are wrong.
refs #6700
Brett Smith [Thu, 23 Jul 2015 14:07:54 +0000 (10:07 -0400)]
6750: arvados-api-server-upgrade only installs omniauth.rb if it exists.
New installs configure the SSO server via application.yml. Refs #6750.
Brett Smith [Wed, 22 Jul 2015 20:38:33 +0000 (16:38 -0400)]
Merge branch '6497-separate-package-upload-wip'
Closes #6497, #6533, #6498.
Brett Smith [Wed, 22 Jul 2015 20:38:23 +0000 (16:38 -0400)]
6497: Separate package build and uploader scripts.
This lets us avoid redundant uploads as we build for multiple targets,
and lets us better customize the upload process per target.
run-build-packages had code to avoid doing an upload when there was no
new package to build. run_upload_packages uses a timestamp file to
achieve a similar result.
Other changes to run-build-packages to support this:
* Better detection of when Python and Ruby packages have already been
built.
* Refactor Gem building into a function to fix various bugs (wrong
version checked for existing arvados-cli; distro packages not
installed in the proper directory).
* Refactor git log querying to reduce code duplication.
Nico Cesar [Tue, 21 Jul 2015 14:10:24 +0000 (10:10 -0400)]
added exit if git pull fails
no issue #
Brett Smith [Mon, 20 Jul 2015 18:36:46 +0000 (14:36 -0400)]
6656: python-arvados-python-client Recommends: git.
Because arv-copy uses it. Refs #6656.
Ward Vandewege [Thu, 16 Jul 2015 22:02:30 +0000 (18:02 -0400)]
run-deploy.sh improvements: remove the need for a .ssh/config entry for
the shortname of each API server. Thanks for the suggestion Nico.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 03:14:23 +0000 (23:14 -0400)]
Make sure jessie packages end up in the jessie repository.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 02:59:28 +0000 (22:59 -0400)]
We need to force-tag the arvados/jobs docker image.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 02:53:50 +0000 (22:53 -0400)]
freight is already smart enough to do atomic updates.
No issue #
Ward Vandewege [Thu, 16 Jul 2015 01:55:21 +0000 (21:55 -0400)]
Add dockerfiles/build-all-build-containers.sh script
refs #6504
Peter Amstutz [Wed, 15 Jul 2015 17:29:00 +0000 (13:29 -0400)]
Add datamanager tests. refs #3408
Ward Vandewege [Wed, 15 Jul 2015 17:14:34 +0000 (13:14 -0400)]
Build python3-websocket-client packages, which is now a dependency for
python3-docker-py, which is a dependency for arvados-docker-cleaner.
No issue #
Ward Vandewege [Tue, 14 Jul 2015 01:02:07 +0000 (21:02 -0400)]
Make sure we have a conference/environments/production.rb file so that
we generate the workbench assets correctly before packaging.
refs #6362
Ward Vandewege [Mon, 13 Jul 2015 15:39:25 +0000 (11:39 -0400)]
run_deploy.sh improvements:
* Make sure that group_class is included when creating the public docker
images project.
* Document the required permissions for the arvados token needed to run
run_deploy.sh
closes #6586
Ward Vandewege [Fri, 10 Jul 2015 18:07:39 +0000 (14:07 -0400)]
Add debian8 as a build target.
No issue #
Ward Vandewege [Fri, 10 Jul 2015 02:01:36 +0000 (22:01 -0400)]
Build arvados/jobs docker image separately from the others.
refs #6528
Ward Vandewege [Wed, 8 Jul 2015 20:31:35 +0000 (16:31 -0400)]
Add dependency on libpam-python to the libarvados-pam package.
refs #2256
Ward Vandewege [Wed, 8 Jul 2015 18:56:07 +0000 (14:56 -0400)]
Add libpam-arvados package build to run-build-packages.
refs #6384
Brett Smith [Wed, 8 Jul 2015 14:09:25 +0000 (10:09 -0400)]
Merge branch '6389-add-dockerfiles'
Closes #6389, #6456.
Brett Smith [Mon, 29 Jun 2015 20:59:24 +0000 (16:59 -0400)]
6388: Add Dockerfile to build packages on CentOS 6.
Brett Smith [Mon, 29 Jun 2015 20:56:57 +0000 (16:56 -0400)]
6389: Add Makefile to support Docker builds.
Brett Smith [Mon, 29 Jun 2015 20:56:16 +0000 (16:56 -0400)]
6389: Dockerfiles include the CMD to build packages.
Ward Vandewege [Fri, 26 Jun 2015 21:07:32 +0000 (17:07 -0400)]
Add dockerfiles for Debian7 and Ubuntu1204
refs #6389
Ward Vandewege [Wed, 8 Jul 2015 13:59:54 +0000 (09:59 -0400)]
Fix up git log statements:
* remove duplication between -n1 and --max-count=1
* make sure every git log statement has --first-parent
refs #6096
Ward Vandewege [Wed, 8 Jul 2015 02:05:01 +0000 (22:05 -0400)]
Small fixes for run-deploy.sh
refs #6096
Ward Vandewege [Thu, 11 Jun 2015 01:30:45 +0000 (21:30 -0400)]
Make run-deploy.rb install the correct version of the arvados/jobs
docker image on deploy.
refs #6096
Brett Smith [Mon, 6 Jul 2015 17:29:44 +0000 (13:29 -0400)]
Merge branch '6388-package-scripts-redhat-support-wip'
Refs #6388. Closes #6480.
Brett Smith [Mon, 6 Jul 2015 14:26:44 +0000 (10:26 -0400)]
6388: Extend API server upgrade script to do initial database setup.
This will let administrators duplicating a lot of logic that already
exists in the script.
Brett Smith [Sun, 5 Jul 2015 22:11:23 +0000 (18:11 -0400)]
6388: Rails package upgrade scripts support Red Hat systems.
* Red Hat doesn't use the www-data account name.
* When nginx is installed from a Software Collection, it may have a
name like `nginx16`. Detect and use this.
Brett Smith [Sun, 5 Jul 2015 15:55:25 +0000 (11:55 -0400)]
Bugfix config:check error detection in Rails upgrade scripts.
The previous code was capturing the exit status of echo, which isn't
very interesting. No issue #.
Tom Clegg [Tue, 30 Jun 2015 18:53:47 +0000 (14:53 -0400)]
6377: Don't try to uninstall arvados* gems if they are not installed.
Fixes run-tests failure when no Ruby SDK is installed on the test host
and tmpdirs are new/empty. refs #6377
Tom Clegg [Tue, 30 Jun 2015 18:50:50 +0000 (14:50 -0400)]
6377: Create user-specified tmpdirs if necessary. refs #6377
Ward Vandewege [Mon, 29 Jun 2015 18:18:08 +0000 (14:18 -0400)]
Add rsync to the repo update command in run-build-packages.
No issue #
Ward Vandewege [Fri, 26 Jun 2015 21:09:03 +0000 (17:09 -0400)]
Make sure to also build a backported package for python-llfuse. We need
it for writeable fuse.
refs #3198
Ward Vandewege [Fri, 26 Jun 2015 19:41:47 +0000 (15:41 -0400)]
Update comment references from 'debs' to 'packages'.
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 19:40:26 +0000 (15:40 -0400)]
Drop built packages into packages/$TARGET instead of the old 'debs' directory.
Also, fix the 'find' test so that built packages are actually added to
the repository after upload.
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 19:04:35 +0000 (15:04 -0400)]
Add ubuntu1204 target to run-build-packages.sh
refs #6389
Ward Vandewege [Fri, 26 Jun 2015 18:55:22 +0000 (14:55 -0400)]
Regex match fixes in the character class match for $FPM_RESULTS:
a) we want to match a literal dot, not any character
b) bash 4.2 does not support escaping a dash to include it in the
character class, move the dash to the last character listed in the class
refs #6389
Brett Smith [Fri, 26 Jun 2015 13:12:22 +0000 (09:12 -0400)]
6388: CentOS needs backports for python-daemon and its dependencies.
Refs #6388.
Peter Amstutz [Fri, 26 Jun 2015 03:39:43 +0000 (23:39 -0400)]
Additionally build and upload cwl-runner package. no issue #
Nico Cesar [Thu, 25 Jun 2015 19:53:46 +0000 (15:53 -0400)]
login patch because DockerHub is not friendly
no issue #
Brett Smith [Thu, 25 Jun 2015 19:33:01 +0000 (15:33 -0400)]
Merge branch '6388-python-backports-wip'
Refs #6388. Closes #6425.
Brett Smith [Thu, 25 Jun 2015 19:22:14 +0000 (15:22 -0400)]
6388: Improve portability of Python backports in run-build-packages.
* Prefix package names with the same prefix used by the interpreter
package, to avoid conflicts with system packages when we're using
something like Software Collections.
* Always specify a dependency on that package.
* Exclude more test paths from packages to avoid conflicts.
* Do a little name munging to make sure that output package names
match dependencies generated by fpm.
Brett Smith [Thu, 25 Jun 2015 19:19:22 +0000 (15:19 -0400)]
6388: CentOS 6 needs Python daemon and llfuse backported.
Brett Smith [Thu, 25 Jun 2015 19:18:58 +0000 (15:18 -0400)]
6388: Build specific backport versions for Python OAuth dependencies.
Brett Smith [Thu, 25 Jun 2015 19:16:57 +0000 (15:16 -0400)]
6388: Build PyCURL with NSS on CentOS 6.
Brett Smith [Thu, 25 Jun 2015 15:54:35 +0000 (11:54 -0400)]
6388: Remove old --apt-server switch from run-build-packages.
Refs #6388.
Brett Smith [Thu, 25 Jun 2015 15:41:01 +0000 (11:41 -0400)]
Merge branch '6388-centos-packages-wip'
Refs #6388. Closes #6411.
Brett Smith [Wed, 24 Jun 2015 21:47:30 +0000 (17:47 -0400)]
6388: run-build-packages introspects more Python environment.
* Look for easy_install under a few different names.
* On CentOS, query the package that provides each Python, and list
that as a dependency.
Brett Smith [Wed, 24 Jun 2015 21:06:03 +0000 (17:06 -0400)]
6388: Fix Arvados Perl SDK package build.
Brett Smith [Wed, 24 Jun 2015 20:44:52 +0000 (16:44 -0400)]
6388: Backport pycurl package everywhere.
We need the very latest version per #6073.
Brett Smith [Wed, 24 Jun 2015 17:54:02 +0000 (13:54 -0400)]
6388: Remove version munging from Python package builds.
This code was added to accommodate version numbers that did *not*
comply with PEP 440. We've since fixed those, and they only include
alphanumerics and dots, so we don't need to munge them.
Brett Smith [Wed, 24 Jun 2015 17:51:09 +0000 (13:51 -0400)]
6388: run-build-packages records python2.7 dependency where needed.
Brett Smith [Wed, 24 Jun 2015 17:24:20 +0000 (13:24 -0400)]
6388: Add distro targets to run-build-packages.
This will provide us a single mechanism to make sure that packages are
built to run on a particular distribution: not just in terms of FPM's
output format, but also repository location, backports required, etc.
Brett Smith [Tue, 23 Jun 2015 16:10:54 +0000 (12:10 -0400)]
6388: Remove Debianisms from run-build-packages.
Make sure function and variable names don't refer to Debian when they
can also be used for Red Hat.
Ward Vandewege [Sat, 13 Jun 2015 00:45:38 +0000 (20:45 -0400)]
Fix asset generation in the workbench package. Make sure the build step will error out if asset generation fails for whatever reason.
No issue #
Ward Vandewege [Fri, 12 Jun 2015 18:49:30 +0000 (14:49 -0400)]
Do not build and upload gems that already exist on rubygems.
No issue #
Ward Vandewege [Fri, 12 Jun 2015 15:07:44 +0000 (11:07 -0400)]
Add very rudimentary support for pushing the rpm packages into a yum
repository.
No issue #
Ward Vandewege [Fri, 12 Jun 2015 14:10:11 +0000 (10:10 -0400)]
Add rpm generation support to run-build-packages.sh
No issue #
Ward Vandewege [Thu, 11 Jun 2015 01:11:35 +0000 (21:11 -0400)]
Store the full git hash of the checkout in the arvados-src package, in a file called git-commit.version
refs #6096
Peter Amstutz [Wed, 10 Jun 2015 18:43:59 +0000 (14:43 -0400)]
CWL script won't fail when there are no documentation changes to commit. no issue #
Peter Amstutz [Wed, 10 Jun 2015 14:49:18 +0000 (10:49 -0400)]
Set default identity for cwl repo pushes. no issue #
Peter Amstutz [Wed, 10 Jun 2015 14:28:01 +0000 (10:28 -0400)]
Since jenkins reuses working directories, handle updating existing git repo of
common-workflow-language site that will be automatically pushed. no issue #
Peter Amstutz [Tue, 9 Jun 2015 20:40:47 +0000 (16:40 -0400)]
Automatically build and upload documentation site. no issue #
Ward Vandewege [Tue, 9 Jun 2015 20:35:59 +0000 (16:35 -0400)]
Make the run-deploy.sh script much more general. It can now be used to
deploy any of our clusters after an upgrade of the versions in hiera.
refs #6096
Ward Vandewege [Tue, 9 Jun 2015 14:01:12 +0000 (10:01 -0400)]
Fix links to workbench on run-tests.sh. Also shut up the PyYAML build warnings.
No issue #
Brett Smith [Mon, 8 Jun 2015 17:38:06 +0000 (13:38 -0400)]
Fix directory when dropping API server database connections.
The Python one-liner is written with the expectation that it's in this
directory, and usually fails to find <config/database.yml> otherwise.
No issue #.
Ward Vandewege [Fri, 5 Jun 2015 20:29:33 +0000 (16:29 -0400)]
Merge branch '6096-package-rails-apps'
refs #6096
Ward Vandewege [Fri, 5 Jun 2015 20:29:18 +0000 (16:29 -0400)]
Merge branch 'master' into 6096-package-rails-apps
Peter Amstutz [Fri, 5 Jun 2015 13:51:30 +0000 (09:51 -0400)]
Fix "docker-node-engine.sh" renamed to "build-node-docker.sh", add additional Docker image to upload. no issue #
Ward Vandewege [Thu, 4 Jun 2015 16:40:56 +0000 (12:40 -0400)]
Be sure to include the pre-generated assets (tmp/cache/assets) in the
workbench packages.
refs #6096
Ward Vandewege [Thu, 4 Jun 2015 14:40:29 +0000 (10:40 -0400)]
Name change for the arvados-api-server and arvados-workbench packages - part 2.
refs #6096
Ward Vandewege [Thu, 4 Jun 2015 14:16:38 +0000 (10:16 -0400)]
Name change for the arvados-api-server and arvados-workbench packages. The slim versions are now just
arvados-api-server
arvados-workbench
and the bigger version of the package that includes vendor/bundle is called
arvados-api-server-with-bundle
arvados-workbench-with-bundle
refs #6096
Ward Vandewege [Wed, 3 Jun 2015 21:10:20 +0000 (17:10 -0400)]
Simplified run-deploy.sh now that we have moved away from Capistrano.
refs #6096
Ward Vandewege [Wed, 3 Jun 2015 20:45:12 +0000 (16:45 -0400)]
Default BUILD_BUNDLE_PACKAGES to false.
refs #6096
Ward Vandewege [Wed, 3 Jun 2015 20:40:56 +0000 (16:40 -0400)]
Fix license names in API and workbench packages.
refs #6096
Ward Vandewege [Wed, 3 Jun 2015 20:27:48 +0000 (16:27 -0400)]
Build API server and Workbench debian packages.
Refs #6096
Peter Amstutz [Wed, 27 May 2015 18:40:11 +0000 (14:40 -0400)]
Merge branch '5548-test-cwltool' refs #5548
Peter Amstutz [Wed, 27 May 2015 18:18:23 +0000 (14:18 -0400)]
5548: Remove git commands from run-cwl-tests. Remove cwltool from build-packages.
Peter Amstutz [Wed, 27 May 2015 16:30:51 +0000 (12:30 -0400)]
5548: Script to test, build and upload common workflow language packages and
docker images.
Nico Cesar [Wed, 27 May 2015 14:16:55 +0000 (10:16 -0400)]
Merge branch '6135-docker-git-tag'
closes #6135
Conflicts:
jenkins/run-docker-tests.sh
Nico Cesar [Thu, 21 May 2015 18:35:54 +0000 (14:35 -0400)]
6035: add support for --upload as an option
refs #6035
Nico Cesar [Fri, 22 May 2015 19:32:41 +0000 (15:32 -0400)]
Merge branch '6035-docker-upload-option'
closes #6035
Nico Cesar [Fri, 22 May 2015 19:18:14 +0000 (15:18 -0400)]
style and fashion for classy 'if' statements
refs #6035
Nico Cesar [Thu, 21 May 2015 18:35:54 +0000 (14:35 -0400)]
6035: add support for --upload as an option
refs #6035
Nico Cesar [Thu, 21 May 2015 15:33:27 +0000 (11:33 -0400)]
Merge branch '6061-add-performance-plots' refs #6061
Brett Smith [Thu, 21 May 2015 14:21:45 +0000 (10:21 -0400)]
3793: Fix how python3 fpm arguments are added.
The previous code would add arguments such that they could be
interpreted as the package version. Avoid this by adding arguments
after we're done consuming positional arguments.
Refs #3793.
no more trailing spaces
adding an empty csv in case of error
Brett Smith [Thu, 21 May 2015 14:03:36 +0000 (10:03 -0400)]
Improve packaging of third-party Python 3 packages.
* Fix the bug where we're passing in "python3" as the vendor argument
to build_and_scp_deb.
* Build more dependencies that docker-py needs.
* Automatically add a dependency on the python3 package.
Refs #3793.
Tom Clegg [Thu, 21 May 2015 12:57:50 +0000 (08:57 -0400)]
5766: Remove "remove dependencies" step. Detect existing PyYAML faster.
Tom Clegg [Thu, 21 May 2015 12:51:21 +0000 (08:51 -0400)]
Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
Suggest how to fix missing dependencies.
No issue #
Nico Cesar [Thu, 21 May 2015 14:27:14 +0000 (10:27 -0400)]
6061: comment cleanup and final version tested. it works
closes #6061
Nico Cesar [Wed, 20 May 2015 15:43:31 +0000 (11:43 -0400)]
6061: plot data
added create plot data script to exit_cleanly()
3 initial test to use
Matching more generic test names
refs #6061
projects / arvados-dev.git / log