Tom Clegg [Tue, 5 Jul 2016 17:28:51 +0000 (13:28 -0400)]
Merge branch '8016-crunchrun-crunchstat'
closes #8016
radhika [Mon, 4 Jul 2016 21:23:10 +0000 (17:23 -0400)]
closes #9498
Merge branch '9498-containers-in-project-tab'
radhika [Tue, 28 Jun 2016 19:58:27 +0000 (15:58 -0400)]
9498: show top-level container requests in project#Pipelines_and_processes tab.
Tom Clegg [Mon, 4 Jul 2016 21:08:37 +0000 (17:08 -0400)]
8016: Clarify comments/docs and cpuSample initialization.
Tom Clegg [Mon, 4 Jul 2016 21:07:54 +0000 (17:07 -0400)]
8016: Rename Poll to PollPeriod.
Tom Clegg [Mon, 4 Jul 2016 21:06:48 +0000 (17:06 -0400)]
8016: Fix race in test case.
Brett Smith [Fri, 1 Jul 2016 20:55:43 +0000 (16:55 -0400)]
Add libgnutls28-dev to arvados/jobs.
This is necessary to build pycurl, and with it the Python Arvados SDK,
and support arvados_sdk_version in Crunch. I'm guessing we lost it
when we switched the base from arvados/debian:7 to debian:jessie.
No issue #.
Tom Clegg [Fri, 1 Jul 2016 17:53:21 +0000 (13:53 -0400)]
Merge branch '9437-keep-index-ns'
refs #9437
See https://dev.arvados.org/issues/9437#note-3 and #note-4
Tom Clegg [Fri, 1 Jul 2016 14:10:28 +0000 (10:10 -0400)]
9437: Clarify units for -blob-signature-ttl. Update keepstore -h message in install docs.
Tom Clegg [Fri, 24 Jun 2016 14:08:45 +0000 (10:08 -0400)]
9437: Accept 1-second timestamps from old keepstore servers.
Running a new keep-balance/datamanager with old keepstore servers will
interpret timestamps correctly.
Running an old keep-balance/datamanager with new keepstore servers
will not perform any garbage collection: interpreting the new
nanosecond timestamps as 1-second timestamps will result in all blocks
appearing to come from the future, making them ineligible for
trash/delete.
Tom Clegg [Thu, 23 Jun 2016 20:22:56 +0000 (16:22 -0400)]
9437: gofmt
Tom Clegg [Thu, 23 Jun 2016 20:19:30 +0000 (16:19 -0400)]
9437: Report timestamps as nanoseconds since epoch in keepstore index.
Tom Clegg [Fri, 1 Jul 2016 13:59:49 +0000 (09:59 -0400)]
Merge branch '4753-websocket-test-port'
refs #4753
Tom Clegg [Thu, 30 Jun 2016 21:58:14 +0000 (17:58 -0400)]
4753: Select an available port for the passenger server when testing websockets.
This makes it possible to run API tests concurrently (in different
working dirs on the same host) without fighting over port 3002.
Tom Clegg [Thu, 30 Jun 2016 21:42:03 +0000 (17:42 -0400)]
8016: Report container resource usage in "crunchstat" log.
Tom Clegg [Thu, 30 Jun 2016 15:05:37 +0000 (11:05 -0400)]
8016: Fix some error checking in ThrottledLogger.
Tom Clegg [Thu, 30 Jun 2016 14:41:07 +0000 (10:41 -0400)]
8016: Fix comment grammar and weird variable name.
Tom Clegg [Thu, 30 Jun 2016 14:26:07 +0000 (10:26 -0400)]
8016: Fix timestamp format (add missing nanoseconds), and use it in tests.
Tom Clegg [Thu, 30 Jun 2016 13:48:44 +0000 (09:48 -0400)]
8016: Fix up docs and trim API.
Tom Clegg [Thu, 30 Jun 2016 13:11:49 +0000 (09:11 -0400)]
8016: Reduce logging noise by waiting for cgroup files to appear before polling.
Tom Clegg [Thu, 30 Jun 2016 00:13:09 +0000 (20:13 -0400)]
8016: Split crunchstat into a module and a commmand line tool.
Peter Amstutz [Thu, 30 Jun 2016 17:24:12 +0000 (13:24 -0400)]
Update arvados-cwl-runner cwltool pin to get ruamel.yaml fix (
c8f584840591f3a4a0fad3bce6b39f8866cd9dad) refs #9458
Tom Clegg [Wed, 29 Jun 2016 20:11:41 +0000 (16:11 -0400)]
Merge branch '9513-keepproxy-content-length'
closes #9513
Tom Clegg [Wed, 29 Jun 2016 17:41:57 +0000 (13:41 -0400)]
9513: Do not set response Content-Length to -1 when returning 411 Length Required.
Add tests for requests with bad/missing Content-Length headers.
Tom Clegg [Wed, 29 Jun 2016 19:55:11 +0000 (15:55 -0400)]
Merge branch '9518-username-conflict'
closes #9518
Tom Clegg [Wed, 29 Jun 2016 19:53:27 +0000 (15:53 -0400)]
9518: Fix username conflict resolution.
radhika [Wed, 29 Jun 2016 18:03:55 +0000 (14:03 -0400)]
closes #9407
Merge branch '9407-container-log-viewer'
radhika [Fri, 24 Jun 2016 21:16:12 +0000 (17:16 -0400)]
9407: container log view
Peter Amstutz [Wed, 29 Jun 2016 17:13:02 +0000 (13:13 -0400)]
Merge branch '9511-cwl-type-checking' closes #9511
Peter Amstutz [Wed, 29 Jun 2016 17:04:47 +0000 (13:04 -0400)]
9511: Bump cwltool dependency to get bugfix (
81ff56f239de3ff0d17c8dec98d1a40f1e41cb7d) affecting bcbio run.
Peter Amstutz [Tue, 28 Jun 2016 19:50:02 +0000 (15:50 -0400)]
Merge branch '9442-cwl-runtime-outdir' closes #9442
Peter Amstutz [Tue, 28 Jun 2016 17:25:13 +0000 (13:25 -0400)]
9442: Incorporate check for correct $(runtime.outdir) into tests.
Peter Amstutz [Tue, 28 Jun 2016 17:15:20 +0000 (13:15 -0400)]
9442: Make cwl-runner crunch script backwards compatible (note-6)
Peter Amstutz [Tue, 28 Jun 2016 17:15:02 +0000 (13:15 -0400)]
9442: Fix runtime.outdir for containers.
Peter Amstutz [Tue, 28 Jun 2016 14:03:30 +0000 (10:03 -0400)]
9442: Override kwargs["outdir"] so that $(runtime.outdir) is set correctly.
Lucas Di Pentima [Mon, 27 Jun 2016 19:56:43 +0000 (16:56 -0300)]
Merge branch '9446-refactor-keep-parallel-write-strategy'
9446: PySDK Keep concurrent write refactoring. Simplified thread management using a thread pool and a queue.
ThreadLimiter not needed anymore, so related tests were rewritten for the new code.
Closes #9446
Tom Clegg [Mon, 27 Jun 2016 19:13:32 +0000 (15:13 -0400)]
Merge branch '9491-desired-replicas'
closes #9491
Tom Clegg [Sat, 25 Jun 2016 01:57:30 +0000 (21:57 -0400)]
9491: Fix keepproxy support for X-Keep-Desired-Replicas header.
Tom Clegg [Sat, 25 Jun 2016 01:55:16 +0000 (21:55 -0400)]
9491: Put*() return number of replicas actually stored, even if we ended up storing more than requested.
Tom Clegg [Mon, 27 Jun 2016 19:10:31 +0000 (15:10 -0400)]
Merge branch '8470-resolve-container-req'
closes #8470
Tom Clegg [Fri, 24 Jun 2016 17:55:08 +0000 (13:55 -0400)]
8470: Resolve docker image hash or tag to collection PDH when creating a Container.
Tom Clegg [Fri, 24 Jun 2016 02:28:50 +0000 (22:28 -0400)]
8470: Clean up ContainerRequest tests.
Tom Clegg [Fri, 24 Jun 2016 01:42:51 +0000 (21:42 -0400)]
8470: Resolve mounts to PDH.
Tom Clegg [Thu, 23 Jun 2016 18:16:45 +0000 (14:16 -0400)]
8470: Resolve runtime_constraints ranges to numbers when satisfying a Container Request.
Lucas Di Pentima [Mon, 27 Jun 2016 18:36:17 +0000 (15:36 -0300)]
9446: Applying Peter's review suggestions. refs #9446
Lucas Di Pentima [Fri, 24 Jun 2016 22:10:22 +0000 (19:10 -0300)]
PySDK put() refactoring ready, all local tests worked OK
Lucas Di Pentima [Fri, 24 Jun 2016 15:34:55 +0000 (12:34 -0300)]
Some tests done on new code
Peter Amstutz [Fri, 24 Jun 2016 14:01:31 +0000 (10:01 -0400)]
9488: Add log message to notify users on sucessful websocket reconnect. refs #9488
Tom Clegg [Fri, 24 Jun 2016 14:59:33 +0000 (10:59 -0400)]
Merge branch '9486-filter-nodes'
refs #9486
Lucas Di Pentima [Thu, 23 Jun 2016 22:47:19 +0000 (19:47 -0300)]
One last method before calling it a day
Lucas Di Pentima [Thu, 23 Jun 2016 22:38:23 +0000 (19:38 -0300)]
Ready to start testing code
Tom Clegg [Thu, 23 Jun 2016 16:47:49 +0000 (12:47 -0400)]
9486: Use API to filter nodes by last_ping_at.
Peter Amstutz [Wed, 22 Jun 2016 16:18:49 +0000 (12:18 -0400)]
Update run-build-packages versions for schema_salad and cwltool, refs #8442
Peter Amstutz [Wed, 22 Jun 2016 15:44:03 +0000 (11:44 -0400)]
Merge branch 'origin-8442-cwl-crunch2' closes #8442
Peter Amstutz [Wed, 22 Jun 2016 13:57:58 +0000 (09:57 -0400)]
Merge branch 'master' into origin-8442-cwl-crunch2
Peter Amstutz [Wed, 22 Jun 2016 13:55:12 +0000 (09:55 -0400)]
8442: Code cleanup. Remove dead code. Update comments.
Brett Smith [Wed, 22 Jun 2016 13:46:40 +0000 (09:46 -0400)]
PySDK: Fix spelling of X-Keep-Desired-Replicas header.
This matches what Keep services expect, and other clients use.
No issue #.
Peter Amstutz [Tue, 21 Jun 2016 20:10:46 +0000 (16:10 -0400)]
8442: Fix error reporting on missing Docker image
Peter Amstutz [Tue, 21 Jun 2016 19:57:23 +0000 (15:57 -0400)]
Increase timeout on failing websocket test. refs #9427
Peter Amstutz [Tue, 21 Jun 2016 16:20:48 +0000 (12:20 -0400)]
Merge branch '9427-threaded-websockets' closes #9427
Peter Amstutz [Tue, 21 Jun 2016 16:16:23 +0000 (12:16 -0400)]
8442: Make work_api parameter to ArvCwlRunner constructor optional.
Peter Amstutz [Tue, 21 Jun 2016 16:01:57 +0000 (12:01 -0400)]
8442: cwl-runner crunch script must provide work_api parameter to ArvCwlRunner
constructor.
Peter Amstutz [Tue, 21 Jun 2016 15:55:43 +0000 (11:55 -0400)]
8442: Bugfixes from end-to-end testing.
Peter Amstutz [Tue, 21 Jun 2016 14:56:57 +0000 (10:56 -0400)]
9427: Back out redundant connection pool configuration option and add comment
to database.yml.example about connection pool and websockets.
Peter Amstutz [Tue, 21 Jun 2016 12:22:54 +0000 (08:22 -0400)]
8442: Fix tests. Update comments. Rename jobs -> processes since it doesn't hold just "jobs" any more.
Peter Amstutz [Mon, 20 Jun 2016 21:34:07 +0000 (17:34 -0400)]
8442: Rename --crunch1 and --crunch2 options to --api=jobs and --api=containers
Lucas Di Pentima [Mon, 20 Jun 2016 19:59:38 +0000 (16:59 -0300)]
Merge branch '9180-avoid-overreplication-keepclient'
Closes #9180
Peter Amstutz [Mon, 20 Jun 2016 18:46:54 +0000 (14:46 -0400)]
Merge branch '9353-retry-http-error' closes #9353
Peter Amstutz [Mon, 20 Jun 2016 18:29:21 +0000 (14:29 -0400)]
9427: Add limits for connections, subscriptions, queued notifications, and
database connections to configuration. Check notify queue size on push instead
of on pop and close connection immediately.
Peter Amstutz [Mon, 20 Jun 2016 13:35:21 +0000 (09:35 -0400)]
Add missing arvbox package "virtualenv" refs #9408
Peter Amstutz [Fri, 17 Jun 2016 20:41:03 +0000 (16:41 -0400)]
9427: Add connection limit.
Peter Amstutz [Fri, 17 Jun 2016 20:27:18 +0000 (16:27 -0400)]
Bugfix "arv-ws --id" option, refs #9388.
Peter Amstutz [Fri, 17 Jun 2016 20:12:15 +0000 (16:12 -0400)]
9427: Websockets now uses a thread per connection for database query and
sending outgoing messages. This minimizes interference by slow clients or
clients that are "replaying the world" with a last_log_id far in the past.
radhika [Fri, 17 Jun 2016 15:13:28 +0000 (11:13 -0400)]
closes #9372
Merge branch '9372-container-display'
radhika [Fri, 17 Jun 2016 15:13:12 +0000 (11:13 -0400)]
Merge branch 'master' into 9372-container-display
radhika [Fri, 17 Jun 2016 15:11:48 +0000 (11:11 -0400)]
9372: improve container_work_unit -> uri method to check if @proxied will respond_to table_name instead of using a rescue.
Lucas Di Pentima [Thu, 16 Jun 2016 23:06:21 +0000 (20:06 -0300)]
9180: Changed some of the logic on ThreadLimiter and made unit tests to validate the new behaviour
refs #9180
radhika [Thu, 16 Jun 2016 14:32:21 +0000 (10:32 -0400)]
9372: The children of a Container will be ContainerRequests to which this is the requesting_container_uuid.
radhika [Thu, 16 Jun 2016 12:17:46 +0000 (08:17 -0400)]
9372: variable names and comments.
Brett Smith [Thu, 16 Jun 2016 02:47:05 +0000 (22:47 -0400)]
9408: Restore build-essential and ruby-dev to arvbox.
These were previously being pulled in as Recommends, and they're
necessary to build gems for the demo image.
Refs #9408.
Lucas Di Pentima [Thu, 16 Jun 2016 01:51:07 +0000 (22:51 -0300)]
Merge branch '9417-asserttrue-for-value-comparison'
Closes #9417
Lucas Di Pentima [Wed, 15 Jun 2016 19:27:23 +0000 (16:27 -0300)]
9417: Replaced assertTrue() with assertEqual() on RetryNeedsMultipleServices's tests
Lucas Di Pentima [Wed, 15 Jun 2016 21:45:10 +0000 (18:45 -0300)]
First try, don't think it'll work
Peter Amstutz [Wed, 15 Jun 2016 20:09:22 +0000 (16:09 -0400)]
8442: Generatefiles encode utf-8
Peter Amstutz [Wed, 15 Jun 2016 18:51:14 +0000 (14:51 -0400)]
8442: Submit CommandLineTool containers directly without an intermediate
workflow runner. Handle unsupported feature gracefully.
Peter Amstutz [Wed, 15 Jun 2016 15:23:47 +0000 (11:23 -0400)]
8442: Tweak internal handling of keep: paths, examine exit codes to determine
success/fail.
Peter Amstutz [Mon, 13 Jun 2016 20:42:37 +0000 (16:42 -0400)]
8442: more import fixups, import workflow file correctly.
Peter Amstutz [Mon, 13 Jun 2016 20:28:45 +0000 (16:28 -0400)]
8442: import fixups
Peter Amstutz [Mon, 13 Jun 2016 20:17:45 +0000 (16:17 -0400)]
8442: Debugging container --submit with crunch2
Peter Amstutz [Mon, 13 Jun 2016 19:45:29 +0000 (15:45 -0400)]
8442: Adding --submit support with --crunch2. General refactoring into more/smaller files.
Peter Amstutz [Mon, 13 Jun 2016 14:36:05 +0000 (10:36 -0400)]
8442: Fix message
Peter Amstutz [Thu, 9 Jun 2016 19:16:09 +0000 (15:16 -0400)]
8442: raise UnsupportedRequirement for unsupported features in the conformance
tests. Bump cwltool dependency.
Peter Amstutz [Thu, 9 Jun 2016 12:21:34 +0000 (08:21 -0400)]
8442: Setting up mount points works. Capturing output works.
Peter Amstutz [Wed, 8 Jun 2016 15:45:00 +0000 (11:45 -0400)]
8442: Return PDH for Docker container. Working on setting up mount points.
Peter Amstutz [Wed, 8 Jun 2016 15:15:58 +0000 (11:15 -0400)]
8442: Submit containers Work in progess.
Peter Amstutz [Mon, 6 Jun 2016 19:54:35 +0000 (15:54 -0400)]
8442: Add --crunch1/--crunch2 switch
Peter Amstutz [Mon, 6 Jun 2016 19:24:50 +0000 (15:24 -0400)]
8442: CWL create crunch2 containers WIP
Tom Clegg [Wed, 15 Jun 2016 19:40:58 +0000 (15:40 -0400)]
Merge branch '9395-keep-balance-page-size'
closes #9395
Tom Clegg [Wed, 15 Jun 2016 19:40:28 +0000 (15:40 -0400)]
9395: Fix encoding of integers in query params.
Tom Clegg [Wed, 15 Jun 2016 14:53:01 +0000 (10:53 -0400)]
9395: Explain CollectionBatchSize and CollectionBuffers in -help message.
projects / arvados.git / log