Nico Cesar [Sun, 17 Jul 2016 13:37:09 +0000 (09:37 -0400)]
Merge branch '9614-cwl-no-websockets'
closes #9614
Peter Amstutz [Sat, 16 Jul 2016 20:21:58 +0000 (16:21 -0400)]
9614: Don't use websockets for event notification until it is more reliable.
Ward Vandewege [Fri, 15 Jul 2016 21:09:38 +0000 (17:09 -0400)]
Pin the html5lib package we build for centos6 and centos7 to version
0.
9999999, because newer versions have a dependency on a
python-setuptools 0.18.5 which we don't have in the build containers
(yet).
refs #9459
Brett Smith [Fri, 15 Jul 2016 20:14:56 +0000 (16:14 -0400)]
Merge branch '9459-arvbox-demo-version-arg-wip'
Refs #9459. Closes #9531.
Brett Smith [Thu, 30 Jun 2016 21:45:12 +0000 (17:45 -0400)]
9459: arvbox-demo accepts version arguments for Git checkouts.
Peter Amstutz [Fri, 15 Jul 2016 19:16:17 +0000 (15:16 -0400)]
Bump version of setuptools used by run_tests.sh for cwl-runner dependency refs #9570
Peter Amstutz [Fri, 15 Jul 2016 18:30:50 +0000 (14:30 -0400)]
Merge branch '9369-arv-cwl-docs' closes #9369
Brett Smith [Fri, 15 Jul 2016 18:26:45 +0000 (14:26 -0400)]
9369: Copyedit CLI SDK install guide.
Peter Amstutz [Fri, 15 Jul 2016 18:11:37 +0000 (14:11 -0400)]
9369: Remove cwl install from Python SDK page. Remove "sudo -i" from
pip/python install instructions. Fixup use of ~$ prompt in example
instructions.
Peter Amstutz [Fri, 15 Jul 2016 17:11:49 +0000 (13:11 -0400)]
Merge branch 'master' into 9369-arv-cwl-docs
Peter Amstutz [Fri, 15 Jul 2016 17:11:07 +0000 (13:11 -0400)]
Merge branch '9570-cwl-runner-fixes' closes #9570 (again)
Brett Smith [Fri, 15 Jul 2016 15:03:56 +0000 (11:03 -0400)]
9369: Copyedits throughout the CWL documentation.
Peter Amstutz [Fri, 15 Jul 2016 14:00:22 +0000 (10:00 -0400)]
9369: Add note about Javascript eval timeout.
Peter Amstutz [Fri, 15 Jul 2016 13:52:02 +0000 (09:52 -0400)]
9570: Raise exception on workflow failure.
Peter Amstutz [Fri, 15 Jul 2016 13:41:29 +0000 (09:41 -0400)]
9369: bwa-mem-input-local hashbang uses cwltool.
Peter Amstutz [Thu, 14 Jul 2016 20:57:27 +0000 (16:57 -0400)]
9570: Use special path mapper for staging files to output dir.
Peter Amstutz [Thu, 14 Jul 2016 20:40:24 +0000 (16:40 -0400)]
9570: Call normalizeFilesDirs to make sure basename is there.
Peter Amstutz [Thu, 14 Jul 2016 20:34:19 +0000 (16:34 -0400)]
9570: Just add keep: to input keep locators in cwl-runner.
Peter Amstutz [Thu, 14 Jul 2016 19:40:05 +0000 (15:40 -0400)]
9570: Adjust Directory objects, too.
Peter Amstutz [Thu, 14 Jul 2016 19:01:39 +0000 (15:01 -0400)]
9570: Remove spurious print statement.
Peter Amstutz [Thu, 14 Jul 2016 18:54:18 +0000 (14:54 -0400)]
9570: Bump cwltool dependency
Peter Amstutz [Thu, 14 Jul 2016 17:57:09 +0000 (13:57 -0400)]
9570: Update cwl-runner for latest cwltool. Ensure use_container,
tmpdir_prefix and on_error are set.
Tom Clegg [Thu, 14 Jul 2016 20:47:38 +0000 (16:47 -0400)]
Merge branch 'fix-python-sdk-mac' of https://github.com/adambom/arvados
No issue #
Adam Savitzky [Thu, 14 Jul 2016 20:38:38 +0000 (13:38 -0700)]
Monkey patch socket constants on apple only
Peter Amstutz [Thu, 14 Jul 2016 19:21:30 +0000 (15:21 -0400)]
Remove spurious print introduced in
6884eed, refs #9570
Peter Amstutz [Thu, 14 Jul 2016 17:34:25 +0000 (13:34 -0400)]
9369: Add section about Docker. Change cluster references to qr1hi. Fix path to use /home/example.
Peter Amstutz [Thu, 14 Jul 2016 17:32:02 +0000 (13:32 -0400)]
9369: Update CLI install documentation to include arvados-cwl-runner (& arvados-python-client)
Adam Savitzky [Wed, 13 Jul 2016 22:54:54 +0000 (15:54 -0700)]
Patch socket configuration for Apple systems
Peter Amstutz [Wed, 13 Jul 2016 21:32:32 +0000 (17:32 -0400)]
9369: Update tutorial to CWL v1.0
Nico Cesar [Wed, 13 Jul 2016 21:18:52 +0000 (17:18 -0400)]
Merge branch '9459-test_with_arvbox_tweaks'
refs #9459
Nico Cesar [Tue, 12 Jul 2016 19:32:25 +0000 (15:32 -0400)]
9459: added test_in_arvbox.sh and arvbox the docker tag parameter.
refs #9459
Tom Clegg [Wed, 13 Jul 2016 19:17:18 +0000 (15:17 -0400)]
Merge branch '9567-keep-web-session-test'
refs #9567
Tom Clegg [Wed, 13 Jul 2016 19:14:14 +0000 (15:14 -0400)]
9567: Use longer timeouts for keep-web download tests.
Tom Clegg [Wed, 13 Jul 2016 19:05:38 +0000 (15:05 -0400)]
9567: Print log messages from keep et al. as they appear, even in jenkins where stdout is not a tty.
Peter Amstutz [Wed, 13 Jul 2016 18:45:39 +0000 (14:45 -0400)]
Fix tests. refs #9570
Peter Amstutz [Wed, 13 Jul 2016 16:51:21 +0000 (12:51 -0400)]
Merge branch '9486-retry-instance-limit-exceeded' closes #9486
Peter Amstutz [Wed, 13 Jul 2016 16:49:57 +0000 (12:49 -0400)]
Merge branch '9570-cwl-v1.0' closes #9570
Peter Amstutz [Wed, 13 Jul 2016 15:36:03 +0000 (11:36 -0400)]
9570: Set docker_outdir consistently.
Peter Amstutz [Wed, 13 Jul 2016 15:05:34 +0000 (11:05 -0400)]
9570: Update version pin and add reminder to setup.py to do it in the future.
Peter Amstutz [Wed, 13 Jul 2016 13:53:48 +0000 (09:53 -0400)]
9570: Fix secondaryFile staging to ensure that all files are in the same
collection.
Peter Amstutz [Tue, 12 Jul 2016 22:14:53 +0000 (18:14 -0400)]
9570: Fix for Pathmapper API change. Most tests pass or exit unsupported
feature.
Peter Amstutz [Tue, 12 Jul 2016 19:55:34 +0000 (15:55 -0400)]
9570: Support Directory and file literal features. Support
success/temporary/permanent exit codes.
Brett Smith [Tue, 12 Jul 2016 14:19:37 +0000 (10:19 -0400)]
9369: Copyedits throughout the CWL docs.
Peter Amstutz [Mon, 11 Jul 2016 20:02:49 +0000 (16:02 -0400)]
9570: Pass tests except related to directory features
Peter Amstutz [Mon, 11 Jul 2016 19:31:45 +0000 (15:31 -0400)]
9570: CWL v1.0 support wip
Peter Amstutz [Mon, 11 Jul 2016 19:30:47 +0000 (15:30 -0400)]
9570: Add support to crunchrunner for redirecting stderr to support CWL 1.0.
Tom Clegg [Thu, 7 Jul 2016 18:32:03 +0000 (14:32 -0400)]
9567: Avoid ruamel.yaml >0.11.11. refs #9567
Tom Clegg [Thu, 7 Jul 2016 15:43:35 +0000 (11:43 -0400)]
Merge branch '9456-less-clear-trash'
closes #9456
Tom Clegg [Thu, 7 Jul 2016 15:43:17 +0000 (11:43 -0400)]
9456: Do not clear trash lists between runs when the list of keep services has not changed.
Tom Clegg [Thu, 7 Jul 2016 15:23:59 +0000 (11:23 -0400)]
Merge branch '9528-slurm-parsable'
closes #9528
Tom Clegg [Thu, 7 Jul 2016 02:11:56 +0000 (22:11 -0400)]
9528: Drop sbatch --parsable flag. It fails in older versions of slurm,
and we don't need to parse the output anyway.
Tom Clegg [Thu, 7 Jul 2016 14:37:33 +0000 (10:37 -0400)]
Revert "Bump cwltool dependency. Previously pinned version was broken."
This reverts commit
d8900c1b1410b97ad24aba69d9f8fa1f042afc90.
No issue #
Tom Clegg [Thu, 7 Jul 2016 14:16:34 +0000 (10:16 -0400)]
Bump cwltool dependency. Previously pinned version was broken.
cwltool==1.0.
20160630171631 depends on
schema-salad==1.12.
20160610104117, which contains an errant .py file
with syntax errors. Installing with pip emits an error message and
succeeds. Installing as an fpm->deb package emits the same error
message and fails.
Linking and byte-compiling packages for runtime python2.7...
File "/usr/lib/python2.7/dist-packages/schema_salad/ref_resolver_BACKUP_28804.py", line 222
<<<<<<<
973dceb3aeef33920968fa257c7b04885a3a716d
^
SyntaxError: invalid syntax
No issue #
Tom Clegg [Wed, 6 Jul 2016 16:05:11 +0000 (12:05 -0400)]
Merge branch '9542-websocket-queries'
refs #9542
Tom Clegg [Wed, 6 Jul 2016 15:47:45 +0000 (11:47 -0400)]
9542: Avoid retrieving huge result sets from postgres.
Tom Clegg [Tue, 5 Jul 2016 20:50:39 +0000 (16:50 -0400)]
Respect running_job_log_records_to_fetch in JobsController#logs.
Fixes flaky test
WebsocketTest#test_test_running_job_with_too_many_previous_log_records
(see https://ci.curoverse.com/job/arvados-api-server/3092/consoleText)
No issue #
Tom Clegg [Tue, 5 Jul 2016 18:47:05 +0000 (14:47 -0400)]
8016: Rename Poll to PollPeriod. Amends
c63c699aa9948f6a672536ba08e664471fb0d654.
refs #8016
Tom Clegg [Tue, 5 Jul 2016 17:28:51 +0000 (13:28 -0400)]
Merge branch '8016-crunchrun-crunchstat'
closes #8016
radhika [Mon, 4 Jul 2016 21:23:10 +0000 (17:23 -0400)]
closes #9498
Merge branch '9498-containers-in-project-tab'
radhika [Tue, 28 Jun 2016 19:58:27 +0000 (15:58 -0400)]
9498: show top-level container requests in project#Pipelines_and_processes tab.
Tom Clegg [Mon, 4 Jul 2016 21:08:37 +0000 (17:08 -0400)]
8016: Clarify comments/docs and cpuSample initialization.
Tom Clegg [Mon, 4 Jul 2016 21:07:54 +0000 (17:07 -0400)]
8016: Rename Poll to PollPeriod.
Tom Clegg [Mon, 4 Jul 2016 21:06:48 +0000 (17:06 -0400)]
8016: Fix race in test case.
Brett Smith [Fri, 1 Jul 2016 20:55:43 +0000 (16:55 -0400)]
Add libgnutls28-dev to arvados/jobs.
This is necessary to build pycurl, and with it the Python Arvados SDK,
and support arvados_sdk_version in Crunch. I'm guessing we lost it
when we switched the base from arvados/debian:7 to debian:jessie.
No issue #.
Peter Amstutz [Fri, 24 Jun 2016 14:18:22 +0000 (10:18 -0400)]
9486: Add InstanceLimitExceeded to is_cloud_exception
Peter Amstutz [Fri, 1 Jul 2016 19:38:59 +0000 (15:38 -0400)]
9369: tweaks
Peter Amstutz [Thu, 30 Jun 2016 17:31:46 +0000 (13:31 -0400)]
9369: arvados-cwl-runner documentation and examples, add deprecation notices to pipeline documentation
Tom Clegg [Fri, 1 Jul 2016 17:53:21 +0000 (13:53 -0400)]
Merge branch '9437-keep-index-ns'
refs #9437
See https://dev.arvados.org/issues/9437#note-3 and #note-4
Tom Clegg [Fri, 1 Jul 2016 14:10:28 +0000 (10:10 -0400)]
9437: Clarify units for -blob-signature-ttl. Update keepstore -h message in install docs.
Tom Clegg [Fri, 24 Jun 2016 14:08:45 +0000 (10:08 -0400)]
9437: Accept 1-second timestamps from old keepstore servers.
Running a new keep-balance/datamanager with old keepstore servers will
interpret timestamps correctly.
Running an old keep-balance/datamanager with new keepstore servers
will not perform any garbage collection: interpreting the new
nanosecond timestamps as 1-second timestamps will result in all blocks
appearing to come from the future, making them ineligible for
trash/delete.
Tom Clegg [Thu, 23 Jun 2016 20:22:56 +0000 (16:22 -0400)]
9437: gofmt
Tom Clegg [Thu, 23 Jun 2016 20:19:30 +0000 (16:19 -0400)]
9437: Report timestamps as nanoseconds since epoch in keepstore index.
Tom Clegg [Fri, 1 Jul 2016 13:59:49 +0000 (09:59 -0400)]
Merge branch '4753-websocket-test-port'
refs #4753
Tom Clegg [Thu, 30 Jun 2016 21:58:14 +0000 (17:58 -0400)]
4753: Select an available port for the passenger server when testing websockets.
This makes it possible to run API tests concurrently (in different
working dirs on the same host) without fighting over port 3002.
Tom Clegg [Thu, 30 Jun 2016 21:42:03 +0000 (17:42 -0400)]
8016: Report container resource usage in "crunchstat" log.
Tom Clegg [Thu, 30 Jun 2016 15:05:37 +0000 (11:05 -0400)]
8016: Fix some error checking in ThrottledLogger.
Tom Clegg [Thu, 30 Jun 2016 14:41:07 +0000 (10:41 -0400)]
8016: Fix comment grammar and weird variable name.
Tom Clegg [Thu, 30 Jun 2016 14:26:07 +0000 (10:26 -0400)]
8016: Fix timestamp format (add missing nanoseconds), and use it in tests.
Tom Clegg [Thu, 30 Jun 2016 13:48:44 +0000 (09:48 -0400)]
8016: Fix up docs and trim API.
Tom Clegg [Thu, 30 Jun 2016 13:11:49 +0000 (09:11 -0400)]
8016: Reduce logging noise by waiting for cgroup files to appear before polling.
Tom Clegg [Thu, 30 Jun 2016 00:13:09 +0000 (20:13 -0400)]
8016: Split crunchstat into a module and a commmand line tool.
Peter Amstutz [Thu, 30 Jun 2016 17:24:12 +0000 (13:24 -0400)]
Update arvados-cwl-runner cwltool pin to get ruamel.yaml fix (
c8f584840591f3a4a0fad3bce6b39f8866cd9dad) refs #9458
Tom Clegg [Wed, 29 Jun 2016 20:11:41 +0000 (16:11 -0400)]
Merge branch '9513-keepproxy-content-length'
closes #9513
Tom Clegg [Wed, 29 Jun 2016 17:41:57 +0000 (13:41 -0400)]
9513: Do not set response Content-Length to -1 when returning 411 Length Required.
Add tests for requests with bad/missing Content-Length headers.
Tom Clegg [Wed, 29 Jun 2016 19:55:11 +0000 (15:55 -0400)]
Merge branch '9518-username-conflict'
closes #9518
Tom Clegg [Wed, 29 Jun 2016 19:53:27 +0000 (15:53 -0400)]
9518: Fix username conflict resolution.
radhika [Wed, 29 Jun 2016 18:03:55 +0000 (14:03 -0400)]
closes #9407
Merge branch '9407-container-log-viewer'
radhika [Fri, 24 Jun 2016 21:16:12 +0000 (17:16 -0400)]
9407: container log view
Peter Amstutz [Wed, 29 Jun 2016 17:13:02 +0000 (13:13 -0400)]
Merge branch '9511-cwl-type-checking' closes #9511
Peter Amstutz [Wed, 29 Jun 2016 17:04:47 +0000 (13:04 -0400)]
9511: Bump cwltool dependency to get bugfix (
81ff56f239de3ff0d17c8dec98d1a40f1e41cb7d) affecting bcbio run.
Peter Amstutz [Tue, 28 Jun 2016 19:50:02 +0000 (15:50 -0400)]
Merge branch '9442-cwl-runtime-outdir' closes #9442
Peter Amstutz [Tue, 28 Jun 2016 17:25:13 +0000 (13:25 -0400)]
9442: Incorporate check for correct $(runtime.outdir) into tests.
Peter Amstutz [Tue, 28 Jun 2016 17:15:20 +0000 (13:15 -0400)]
9442: Make cwl-runner crunch script backwards compatible (note-6)
Peter Amstutz [Tue, 28 Jun 2016 17:15:02 +0000 (13:15 -0400)]
9442: Fix runtime.outdir for containers.
Peter Amstutz [Tue, 28 Jun 2016 14:03:30 +0000 (10:03 -0400)]
9442: Override kwargs["outdir"] so that $(runtime.outdir) is set correctly.
Lucas Di Pentima [Mon, 27 Jun 2016 19:56:43 +0000 (16:56 -0300)]
Merge branch '9446-refactor-keep-parallel-write-strategy'
9446: PySDK Keep concurrent write refactoring. Simplified thread management using a thread pool and a queue.
ThreadLimiter not needed anymore, so related tests were rewritten for the new code.
Closes #9446
Tom Clegg [Mon, 27 Jun 2016 19:13:32 +0000 (15:13 -0400)]
Merge branch '9491-desired-replicas'
closes #9491
Tom Clegg [Sat, 25 Jun 2016 01:57:30 +0000 (21:57 -0400)]
9491: Fix keepproxy support for X-Keep-Desired-Replicas header.
Tom Clegg [Sat, 25 Jun 2016 01:55:16 +0000 (21:55 -0400)]
9491: Put*() return number of replicas actually stored, even if we ended up storing more than requested.
Tom Clegg [Mon, 27 Jun 2016 19:10:31 +0000 (15:10 -0400)]
Merge branch '8470-resolve-container-req'
closes #8470
projects / arvados.git / log