arvados.git
7 years agoMerge branch '10722-cwl-subworkflow' closes #10722
Peter Amstutz [Fri, 30 Dec 2016 19:12:16 +0000 (14:12 -0500)]
Merge branch '10722-cwl-subworkflow' closes #10722

8 years ago8345: Build python27-python-contextlib2 for centos7.
Tom Clegg [Fri, 30 Dec 2016 15:28:38 +0000 (10:28 -0500)]
8345: Build python27-python-contextlib2 for centos7.

refs #8345

8 years agoFix version# of centos7 llfuse backport.
Tom Clegg [Fri, 30 Dec 2016 00:52:10 +0000 (19:52 -0500)]
Fix version# of centos7 llfuse backport.

No issue #

8 years agoBring back ca-certificates to package-testing images.
Tom Clegg [Thu, 29 Dec 2016 19:57:45 +0000 (14:57 -0500)]
Bring back ca-certificates to package-testing images.

No issue #

8 years agoMerge branch '8345-llfuse-1.0' closes #8345
Tom Clegg [Thu, 29 Dec 2016 16:03:59 +0000 (11:03 -0500)]
Merge branch '8345-llfuse-1.0' closes #8345

8 years agoMerge branch 'master' into 8345-llfuse-1.0
Tom Clegg [Thu, 29 Dec 2016 16:02:34 +0000 (11:02 -0500)]
Merge branch 'master' into 8345-llfuse-1.0

Conflicts:
build/package-test-dockerfiles/debian7/Dockerfile
build/run-build-packages.sh

8 years agoObey --only-build arg in "test packages" mode.
Tom Clegg [Thu, 29 Dec 2016 15:45:55 +0000 (10:45 -0500)]
Obey --only-build arg in "test packages" mode.

8 years agoUpdate package-test images to ruby 2.3
Tom Clegg [Thu, 29 Dec 2016 15:37:27 +0000 (10:37 -0500)]
Update package-test images to ruby 2.3

8 years agoMerge branch '10785-remove-debian7'
Ward Vandewege [Thu, 29 Dec 2016 13:36:37 +0000 (08:36 -0500)]
Merge branch '10785-remove-debian7'

closes #10785

8 years ago8345: Build python27-python-contextlib2 for centos6.
Tom Clegg [Thu, 29 Dec 2016 07:47:40 +0000 (02:47 -0500)]
8345: Build python27-python-contextlib2 for centos6.

8 years ago8345: Upgrade to llfuse 1.0
Tom Clegg [Mon, 18 Apr 2016 16:40:21 +0000 (12:40 -0400)]
8345: Upgrade to llfuse 1.0

8 years ago8345: Revert change to getattr() @catch_exceptions setup.
Tom Clegg [Thu, 10 Mar 2016 20:04:10 +0000 (15:04 -0500)]
8345: Revert change to getattr() @catch_exceptions setup.

8 years ago8345: Fix dirent timestamps.
Tom Clegg [Wed, 9 Mar 2016 18:50:41 +0000 (13:50 -0500)]
8345: Fix dirent timestamps.

8 years ago8345: Pin llfuse to <1.0
Tom Clegg [Tue, 8 Mar 2016 21:40:22 +0000 (16:40 -0500)]
8345: Pin llfuse to <1.0

8 years ago8345: fix whitespace
Tom Clegg [Mon, 7 Mar 2016 20:26:14 +0000 (15:26 -0500)]
8345: fix whitespace

8 years agoupdates test_mount to use st_mtime_ns instead of st_mtime
Joshua Randall [Wed, 10 Feb 2016 12:04:25 +0000 (12:04 +0000)]
updates test_mount to use st_mtime_ns instead of st_mtime

8 years agoupdates setattr for llfuse-python 0.42
Joshua Randall [Tue, 9 Feb 2016 21:41:40 +0000 (21:41 +0000)]
updates setattr for llfuse-python 0.42

8 years agoadds backports specific to centos6
Joshua Randall [Fri, 5 Feb 2016 18:37:17 +0000 (18:37 +0000)]
adds backports specific to centos6

8 years agoadds missing fix for llfuse 0.42+
Joshua Randall [Fri, 5 Feb 2016 11:45:10 +0000 (11:45 +0000)]
adds missing fix for llfuse 0.42+

llfuse.capi._notify_queue => llfuse._notify_queue

8 years agoupdates llfuse requirement and API to >=0.42
Joshua Randall [Wed, 3 Feb 2016 11:48:55 +0000 (11:48 +0000)]
updates llfuse requirement and API to >=0.42

8 years agoRemove debian7 from the build directory.
Ward Vandewege [Wed, 28 Dec 2016 23:14:56 +0000 (18:14 -0500)]
Remove debian7 from the build directory.

refs #10785

8 years ago10722: Issue better error when 'id' is missing.
Peter Amstutz [Wed, 28 Dec 2016 20:50:20 +0000 (15:50 -0500)]
10722: Issue better error when 'id' is missing.

8 years agoMerge branch '10782-container-retry' closes #10782
Peter Amstutz [Wed, 28 Dec 2016 17:39:59 +0000 (12:39 -0500)]
Merge branch '10782-container-retry' closes #10782

8 years ago10782: Bugfix, on retry only update container_uuid on container requests
Peter Amstutz [Wed, 28 Dec 2016 17:05:07 +0000 (12:05 -0500)]
10782: Bugfix, on retry only update container_uuid on container requests
associated with the container in the first place.  Update container retry test.

8 years agoFix crunch script to construct toolpath before it is turned into a File object. refs...
Peter Amstutz [Wed, 28 Dec 2016 15:54:37 +0000 (10:54 -0500)]
Fix crunch script to construct toolpath before it is turned into a File object. refs #10497

8 years agoMerge branch '10497-cwl-better-errors' refs #10497
Peter Amstutz [Wed, 28 Dec 2016 15:30:24 +0000 (10:30 -0500)]
Merge branch '10497-cwl-better-errors' refs #10497

8 years agoArvbox common.sh script won't fail if bundler gem not cached on disk.
Peter Amstutz [Wed, 28 Dec 2016 15:29:30 +0000 (10:29 -0500)]
Arvbox common.sh script won't fail if bundler gem not cached on disk.
Don't push docker images if build fails.  no issue #

8 years ago10497: Add pathlib2 to backports
Peter Amstutz [Wed, 28 Dec 2016 15:05:17 +0000 (10:05 -0500)]
10497: Add pathlib2 to backports

8 years ago10497: Pop cwl:tool parameter from job order
Peter Amstutz [Tue, 27 Dec 2016 22:06:17 +0000 (17:06 -0500)]
10497: Pop cwl:tool parameter from job order

8 years ago10497: Update ruamel.yaml, schema salad, and cwltool
Peter Amstutz [Tue, 27 Dec 2016 20:05:48 +0000 (15:05 -0500)]
10497: Update ruamel.yaml, schema salad, and cwltool

8 years ago10497: Use SourceLine in context manager in PathMapper. Remove unnecessary import.
Peter Amstutz [Wed, 21 Dec 2016 15:34:53 +0000 (10:34 -0500)]
10497: Use SourceLine in context manager in PathMapper.  Remove unnecessary import.

8 years ago10497: Improve up front feature checking and error reporting.
Peter Amstutz [Mon, 19 Dec 2016 20:33:40 +0000 (15:33 -0500)]
10497: Improve up front feature checking and error reporting.

8 years ago10497: Add source line reporting to errors and fix tests to work with CommentedMap...
Peter Amstutz [Mon, 19 Dec 2016 18:39:46 +0000 (13:39 -0500)]
10497: Add source line reporting to errors and fix tests to work with CommentedMap/Seq behavior.

8 years ago10497: Update cwltool & schema salad versions.
Peter Amstutz [Fri, 16 Dec 2016 21:34:59 +0000 (16:34 -0500)]
10497: Update cwltool & schema salad versions.

8 years agoLoad keep-balance config from default location if -config arg not given.
Tom Clegg [Wed, 28 Dec 2016 15:21:25 +0000 (10:21 -0500)]
Load keep-balance config from default location if -config arg not given.

No issue #

8 years agoArvbox use --local to install bundler if gem is available, also recognize .tar.bz2 in
Peter Amstutz [Tue, 13 Dec 2016 14:36:47 +0000 (09:36 -0500)]
Arvbox use --local to install bundler if gem is available, also recognize .tar.bz2 in
pip cache.  no issue #

8 years agoMerge branch '10273-check-superuser-token-scope' refs #10273
Tom Clegg [Tue, 27 Dec 2016 16:16:18 +0000 (11:16 -0500)]
Merge branch '10273-check-superuser-token-scope' refs #10273

8 years ago10273: Check scope=[all] when looking for an existing superuser token.
Tom Clegg [Thu, 20 Oct 2016 15:26:23 +0000 (11:26 -0400)]
10273: Check scope=[all] when looking for an existing superuser token.

8 years agoAdd a mechanism to the arvados-api Gemfile to load 'plugin' gems.
Ward Vandewege [Fri, 23 Dec 2016 02:37:42 +0000 (21:37 -0500)]
Add a mechanism to the arvados-api Gemfile to load 'plugin' gems.

refs #9982

8 years agoMerge branch '10778-expand-job-bug' - Fixes #10778
Tom Morris [Thu, 22 Dec 2016 20:34:24 +0000 (15:34 -0500)]
Merge branch '10778-expand-job-bug' - Fixes #10778

8 years agoEnsure column information is up-to-date before creating full text indexes. refs ...
Tom Clegg [Thu, 22 Dec 2016 19:43:28 +0000 (14:43 -0500)]
Ensure column information is up-to-date before creating full text indexes. refs #10028

8 years agoIgnore editor backup files & .DS_Store
Tom Morris [Thu, 22 Dec 2016 19:33:29 +0000 (14:33 -0500)]
Ignore editor backup files & .DS_Store

no issue #

8 years agoFix pipeline instance expand - fixes #10778
Tom Morris [Thu, 22 Dec 2016 18:55:15 +0000 (13:55 -0500)]
Fix pipeline instance expand - fixes #10778

8 years agoSwitch to 2 space indent - no functional changes
Tom Morris [Thu, 22 Dec 2016 18:48:00 +0000 (13:48 -0500)]
Switch to 2 space indent - no functional changes

8 years agoMerge branch '10585-crunchstat-lost-parent' refs #10585
Tom Clegg [Thu, 22 Dec 2016 15:14:10 +0000 (10:14 -0500)]
Merge branch '10585-crunchstat-lost-parent' refs #10585

8 years ago10585: Clean up defaults and error checks; release ticker when finished.
Tom Clegg [Thu, 22 Dec 2016 06:32:20 +0000 (01:32 -0500)]
10585: Clean up defaults and error checks; release ticker when finished.

8 years agoMerge branch '10671-pipeline-instance-finish-time'
Lucas Di Pentima [Wed, 21 Dec 2016 18:22:23 +0000 (15:22 -0300)]
Merge branch '10671-pipeline-instance-finish-time'
Closes #10671

8 years ago10671: Merge branch 'master' into 10671-pipeline-instance-finish-time
Lucas Di Pentima [Wed, 21 Dec 2016 17:28:41 +0000 (14:28 -0300)]
10671: Merge branch 'master' into 10671-pipeline-instance-finish-time

8 years ago10671: Changed default assigments to be before validation. Also make sure started_at...
Lucas Di Pentima [Wed, 21 Dec 2016 17:26:25 +0000 (14:26 -0300)]
10671: Changed default assigments to be before validation. Also make sure started_at is not nil when pipeline instance is finished.

8 years agoMerge branch '10467-client-disconnect' refs #10467
Tom Clegg [Wed, 21 Dec 2016 15:38:51 +0000 (10:38 -0500)]
Merge branch '10467-client-disconnect' refs #10467

8 years agoMerge branch '10497-crunchrun-logs' refs #10497
Peter Amstutz [Wed, 21 Dec 2016 15:11:56 +0000 (10:11 -0500)]
Merge branch '10497-crunchrun-logs' refs #10497

8 years ago10467: Merge branch 'master' into 10467-client-disconnect
Tom Clegg [Wed, 21 Dec 2016 15:08:44 +0000 (10:08 -0500)]
10467: Merge branch 'master' into 10467-client-disconnect

8 years ago10497: crunch-run records logs even when container is cancelled.
Peter Amstutz [Tue, 20 Dec 2016 16:10:56 +0000 (11:10 -0500)]
10497: crunch-run records logs even when container is cancelled.

8 years ago10671: Merge branch 'master' into 10671-pipeline-instance-finish-time
Lucas Di Pentima [Mon, 19 Dec 2016 20:58:13 +0000 (17:58 -0300)]
10671: Merge branch 'master' into 10671-pipeline-instance-finish-time

8 years ago10671: Set up values for pipeline instance started_at and finished_at attributes...
Lucas Di Pentima [Mon, 19 Dec 2016 20:26:38 +0000 (17:26 -0300)]
10671: Set up values for pipeline instance started_at and finished_at attributes on state changes.
Updated tests accordingly.

8 years agoFix CWL tests affected by minimum_script_version tweaks. no issue #
Peter Amstutz [Mon, 19 Dec 2016 15:11:45 +0000 (10:11 -0500)]
Fix CWL tests affected by minimum_script_version tweaks.  no issue #

8 years agoAdjust arvados repo minimum script versions for job dispatch. no issue #
Peter Amstutz [Mon, 19 Dec 2016 14:42:31 +0000 (09:42 -0500)]
Adjust arvados repo minimum script versions for job dispatch. no issue #

8 years agoFix dev build of arvados/jobs image to avoid installing versions of
Peter Amstutz [Fri, 16 Dec 2016 21:58:19 +0000 (16:58 -0500)]
Fix dev build of arvados/jobs image to avoid installing versions of
pyasn1-modules incompatible with pyasn1.  no issue #

8 years agoFix crunch_script.py to pass name arg, refs #10681
Peter Amstutz [Mon, 19 Dec 2016 14:25:34 +0000 (09:25 -0500)]
Fix crunch_script.py to pass name arg, refs #10681

8 years agoMerge branch '10383-arv-put-incremental-upload'
Lucas Di Pentima [Fri, 16 Dec 2016 22:45:59 +0000 (19:45 -0300)]
Merge branch '10383-arv-put-incremental-upload'
Refs #10383

8 years ago10383: Merge branch 'master' into 10383-arv-put-incremental-upload
Lucas Di Pentima [Fri, 16 Dec 2016 22:15:37 +0000 (19:15 -0300)]
10383: Merge branch 'master' into 10383-arv-put-incremental-upload

8 years ago10383: Fix for making arv-keep-get tests pass.
Lucas Di Pentima [Fri, 16 Dec 2016 21:54:18 +0000 (18:54 -0300)]
10383: Fix for making arv-keep-get tests pass.

8 years ago10585: Add crunchstat -signal-on-dead-ppid option.
Tom Clegg [Fri, 16 Dec 2016 21:02:24 +0000 (16:02 -0500)]
10585: Add crunchstat -signal-on-dead-ppid option.

8 years agoMerge branch '10684-crunch-run-ca-certs' closes #10684
Peter Amstutz [Fri, 16 Dec 2016 19:55:54 +0000 (14:55 -0500)]
Merge branch '10684-crunch-run-ca-certs' closes #10684

8 years agoFix CWL tests refs #10576
Peter Amstutz [Fri, 16 Dec 2016 19:39:50 +0000 (14:39 -0500)]
Fix CWL tests refs #10576

8 years ago10684: Don't overwrite default certs search path unless -ca-certs provided.
Peter Amstutz [Fri, 16 Dec 2016 18:38:02 +0000 (13:38 -0500)]
10684: Don't overwrite default certs search path unless -ca-certs provided.

8 years ago10684: Add -ca-certs option
Peter Amstutz [Fri, 16 Dec 2016 17:21:06 +0000 (12:21 -0500)]
10684: Add -ca-certs option

8 years ago10383: Error out when using --name and --update-collection together.
Lucas Di Pentima [Fri, 16 Dec 2016 16:19:37 +0000 (13:19 -0300)]
10383: Error out when using --name and --update-collection together.

8 years ago10684: Refactor into MakeTLSConfig(). Only look for
Peter Amstutz [Fri, 16 Dec 2016 16:09:41 +0000 (11:09 -0500)]
10684: Refactor into MakeTLSConfig().  Only look for
/etc/arvados/ca-certificates.crt and then fall back onto system certs.  Skip
looking for arvados-specific certs if insecure is true.

8 years ago10383: Merge branch 'master' into 10383-arv-put-incremental-upload
Lucas Di Pentima [Thu, 15 Dec 2016 22:04:07 +0000 (19:04 -0300)]
10383: Merge branch 'master' into 10383-arv-put-incremental-upload

8 years ago10684: Only bind mount certificate when API access is indicated.
Peter Amstutz [Thu, 15 Dec 2016 21:39:23 +0000 (16:39 -0500)]
10684: Only bind mount certificate when API access is indicated.

8 years agoMerge branch '10690-config-dump' refs #10690
Tom Clegg [Thu, 15 Dec 2016 21:26:05 +0000 (16:26 -0500)]
Merge branch '10690-config-dump' refs #10690

8 years agoMerge branch '8460-websocket-go' refs #8460
Tom Clegg [Thu, 15 Dec 2016 21:25:50 +0000 (16:25 -0500)]
Merge branch '8460-websocket-go' refs #8460

8 years ago10467: Abandon Azure requests if client hangs up.
Tom Clegg [Thu, 15 Dec 2016 21:20:10 +0000 (16:20 -0500)]
10467: Abandon Azure requests if client hangs up.

8 years ago10690: Add "rake config:dump" task to Workbench and API server.
Tom Clegg [Thu, 15 Dec 2016 20:00:40 +0000 (15:00 -0500)]
10690: Add "rake config:dump" task to Workbench and API server.

8 years agoWe should not hardcode dependencies in our packages for secondary
Ward Vandewege [Thu, 15 Dec 2016 19:52:05 +0000 (14:52 -0500)]
We should not hardcode dependencies in our packages for secondary
dependencies of our dependencies. If you catch my drift.

pyasn1-modules is a dependency of oauth2client which is a dependency of
google-api-python-client. Which is a dependency of
arvados-python-client.

Pinning a dependency on pyasn1-modules=0.0.5 in arvados-python-client
does not make sense. It also does not appear to be necessary at all. And
it currently breaks the CentOS7 package build.

refs #10713

Revert "8654: Pin pyasn1_modules to version that is compatible with pyasn1==0.1.7."

This reverts commit 436105d3e249ad8dc55fb618d0d2c0d8ed29b395.

8 years ago8460: Merge branch 'master' into 8460-websocket-go
Tom Clegg [Thu, 15 Dec 2016 19:47:52 +0000 (14:47 -0500)]
8460: Merge branch 'master' into 8460-websocket-go

8 years ago10684: crunch-run adds /etc/arvados/ca-certificates.crt to binds, updated tests.
Peter Amstutz [Thu, 15 Dec 2016 19:40:58 +0000 (14:40 -0500)]
10684: crunch-run adds /etc/arvados/ca-certificates.crt to binds, updated tests.

8 years agoTidy up queue usage. No issue #
Tom Clegg [Thu, 15 Dec 2016 19:36:56 +0000 (14:36 -0500)]
Tidy up queue usage. No issue #

8 years ago10684: Add Arvados-specific search path to Go SDK TLSClientConfig.
Peter Amstutz [Thu, 15 Dec 2016 18:51:56 +0000 (13:51 -0500)]
10684: Add Arvados-specific search path to Go SDK TLSClientConfig.

8 years ago10383: Remove files from local_collection which don't exist locally, so that the...
Lucas Di Pentima [Thu, 15 Dec 2016 17:57:48 +0000 (14:57 -0300)]
10383: Remove files from local_collection which don't exist locally, so that the bytes_written count is accurate.

8 years ago10684: Add Arvados-specific search path to Python SDK arvados.util.ca_certs_path.
Peter Amstutz [Thu, 15 Dec 2016 17:33:30 +0000 (12:33 -0500)]
10684: Add Arvados-specific search path to Python SDK arvados.util.ca_certs_path.

8 years ago8460: Tolerate None where dicts are expected in events.
Tom Clegg [Thu, 15 Dec 2016 17:16:22 +0000 (12:16 -0500)]
8460: Tolerate None where dicts are expected in events.

8 years ago8460: Rearrange erb so it is also syntactically valid YAML.
Tom Clegg [Thu, 15 Dec 2016 17:12:47 +0000 (12:12 -0500)]
8460: Rearrange erb so it is also syntactically valid YAML.

8 years agoBump schema-salad version to fix rdflib-jsonld dependency. refs #10576
Peter Amstutz [Thu, 15 Dec 2016 16:36:23 +0000 (11:36 -0500)]
Bump schema-salad version to fix rdflib-jsonld dependency. refs #10576

8 years ago8460: Use new ws server for integration tests only if ARVADOS_TEST_EXPERIMENTAL_WS...
Tom Clegg [Thu, 15 Dec 2016 16:29:50 +0000 (11:29 -0500)]
8460: Use new ws server for integration tests only if ARVADOS_TEST_EXPERIMENTAL_WS is set.

8 years ago8460: Move context-aware-logger code to ctxlog pkg.
Tom Clegg [Thu, 15 Dec 2016 16:08:37 +0000 (11:08 -0500)]
8460: Move context-aware-logger code to ctxlog pkg.

8 years ago8460: Add some docs/comments.
Tom Clegg [Thu, 15 Dec 2016 15:40:43 +0000 (10:40 -0500)]
8460: Add some docs/comments.

8 years ago10383: args.dry_run wasn't being passed to ArvPutUploadJob class. Also avoid output...
Lucas Di Pentima [Wed, 14 Dec 2016 17:25:15 +0000 (14:25 -0300)]
10383: args.dry_run wasn't being passed to ArvPutUploadJob class. Also avoid output messages when in dry-run mode.

8 years agoBump libcloud for ex_get_license() bugfix. refs #10686
Peter Amstutz [Wed, 14 Dec 2016 17:01:57 +0000 (12:01 -0500)]
Bump libcloud for ex_get_license() bugfix. refs #10686

8 years agoMerge branch '10576-cwl-keep-fetcher' closes #10576
Peter Amstutz [Wed, 14 Dec 2016 15:01:50 +0000 (10:01 -0500)]
Merge branch '10576-cwl-keep-fetcher' closes #10576

8 years ago10576: Tweak Docker docs and add pipeline deprecation notice.
Peter Amstutz [Tue, 13 Dec 2016 21:40:48 +0000 (16:40 -0500)]
10576: Tweak Docker docs and add pipeline deprecation notice.

8 years ago10576: Fix formatting in cwl-run-options. Fix dangling refs tutorial-pipeline-workbe...
Peter Amstutz [Tue, 13 Dec 2016 21:31:58 +0000 (16:31 -0500)]
10576: Fix formatting in cwl-run-options.  Fix dangling refs tutorial-pipeline-workbench.html

8 years agoMerge branch 'master' into 10576-cwl-keep-fetcher
Peter Amstutz [Tue, 13 Dec 2016 20:51:24 +0000 (15:51 -0500)]
Merge branch 'master' into 10576-cwl-keep-fetcher

Conflicts:
sdk/cwl/setup.py

8 years ago10576: Add ability to run pipeline templates containing CWL from command line.
Peter Amstutz [Tue, 13 Dec 2016 20:46:40 +0000 (15:46 -0500)]
10576: Add ability to run pipeline templates containing CWL from command line.
Split out command line documentation from running CWL tutorial.

8 years agorefs #10028
radhika [Tue, 13 Dec 2016 20:00:14 +0000 (15:00 -0500)]
refs #10028
Merge branch '10028-functional-full-text-search-indexes'

8 years agorefs #10241
radhika [Tue, 13 Dec 2016 19:46:54 +0000 (14:46 -0500)]
refs #10241
Merge branch '10241-container-diagnostics-test'

8 years agoMerge branch '10383-arv-put-incremental-upload'
Lucas Di Pentima [Fri, 16 Dec 2016 17:46:49 +0000 (14:46 -0300)]
Merge branch '10383-arv-put-incremental-upload'
Closes #10383

8 years ago10241: add a diagnostics test to run a process using a workflow.
radhika [Tue, 6 Dec 2016 18:57:04 +0000 (13:57 -0500)]
10241: add a diagnostics test to run a process using a workflow.

8 years ago10028: upgrade full text search indexes to not include leading space.
radhika [Tue, 13 Dec 2016 19:05:15 +0000 (14:05 -0500)]
10028: upgrade full text search indexes to not include leading space.