arvados.git
7 years agoMerge branch '10346-rearrange-api-docs' closes #10346
Peter Amstutz [Tue, 15 Nov 2016 17:29:14 +0000 (12:29 -0500)]
Merge branch '10346-rearrange-api-docs' closes #10346

7 years ago10346: Fix formatting on container_request page.
Peter Amstutz [Tue, 15 Nov 2016 17:27:27 +0000 (12:27 -0500)]
10346: Fix formatting on container_request page.

7 years agoMerge branch '10534-go-systemd-sdnotify-v14' of https://github.com/wtsi-hgi/arvados
Tom Clegg [Tue, 15 Nov 2016 16:35:15 +0000 (11:35 -0500)]
Merge branch '10534-go-systemd-sdnotify-v14' of https://github.com/wtsi-hgi/arvados

refs #10534

7 years agoupdate signature of SdNotify for go-systemd v14
Joshua C. Randall [Tue, 15 Nov 2016 15:31:56 +0000 (15:31 +0000)]
update signature of SdNotify for go-systemd v14

7 years agocloses #10290
radhika [Tue, 15 Nov 2016 15:16:46 +0000 (10:16 -0500)]
closes #10290
Merge branch '10290-container-partitions'

7 years ago10290: Add scheduling_parameters map to containers and container_requests, and move...
radhika [Fri, 11 Nov 2016 20:16:25 +0000 (15:16 -0500)]
10290: Add scheduling_parameters map to containers and container_requests, and move partitions array out of runtime_constraints into this.

7 years ago10346: Add session establishment diagram. Clarify role of block signatures.
Peter Amstutz [Tue, 15 Nov 2016 03:43:36 +0000 (22:43 -0500)]
10346: Add session establishment diagram.  Clarify role of block signatures.
Fix typo on resource pages.

7 years agoMerge branch '10517-consistent-select' closes #10517
Tom Clegg [Sat, 12 Nov 2016 23:55:59 +0000 (18:55 -0500)]
Merge branch '10517-consistent-select' closes #10517

7 years agoFixes API server to return default columns for collections
Joshua C. Randall [Sat, 12 Nov 2016 03:57:50 +0000 (03:57 +0000)]
Fixes API server to return default columns for collections

Prior to this fix, if a request has a query param `select=`
(as opposed to just omitting `select` entirely), the API
server would return a full set of columns rather than
enforcing its own defaults.

Fixes #10517, assuming this was a bug and not a feature.

7 years ago10517: Test edge cases in "deselect manifest_text by default" feature.
Tom Clegg [Sat, 12 Nov 2016 22:42:50 +0000 (17:42 -0500)]
10517: Test edge cases in "deselect manifest_text by default" feature.

7 years ago3486: Fix ignored "required: false" in pipeline template. refs #3486 refs #10470
Tom Clegg [Fri, 11 Nov 2016 21:19:58 +0000 (16:19 -0500)]
3486: Fix ignored "required: false" in pipeline template. refs #3486 refs #10470

7 years ago10346: Add slides about Keep and Crunch.
Peter Amstutz [Fri, 11 Nov 2016 19:33:37 +0000 (14:33 -0500)]
10346: Add slides about Keep and Crunch.

7 years ago10346: Fix cross references
Peter Amstutz [Fri, 11 Nov 2016 03:44:55 +0000 (22:44 -0500)]
10346: Fix cross references

7 years ago10346: Updates from review comments.
Peter Amstutz [Thu, 10 Nov 2016 21:45:48 +0000 (16:45 -0500)]
10346: Updates from review comments.

7 years ago10346: Document manifest format in more detail. Document token signatures. Add...
Peter Amstutz [Thu, 10 Nov 2016 17:34:54 +0000 (12:34 -0500)]
10346: Document manifest format in more detail.  Document token signatures.  Add Arvados_permissions.svg

7 years agocloses #10310
radhika [Thu, 10 Nov 2016 15:34:42 +0000 (10:34 -0500)]
closes #10310
Merge branch '10310-crunch2-workbench'

7 years ago10310: invoke index and show actions only when they are not blacklisted by the api...
radhika [Wed, 2 Nov 2016 00:46:31 +0000 (20:46 -0400)]
10310: invoke index and show actions only when they are not blacklisted by the api server.

7 years ago10310: Check discovery doc when deciding ArvadosBase.creatable?.
Tom Clegg [Thu, 20 Oct 2016 09:37:25 +0000 (05:37 -0400)]
10310: Check discovery doc when deciding ArvadosBase.creatable?.

7 years agoMerge branch '10460-cwl-staging-fix' refs #10460
Peter Amstutz [Thu, 10 Nov 2016 14:35:47 +0000 (09:35 -0500)]
Merge branch '10460-cwl-staging-fix' refs #10460

7 years ago10346: Edits from review WIP.
Peter Amstutz [Wed, 9 Nov 2016 17:56:12 +0000 (12:56 -0500)]
10346: Edits from review WIP.

7 years ago10460: Add check for 'dockerOutputDirectory' and raise UnsupportedError for now.
Peter Amstutz [Tue, 8 Nov 2016 22:15:19 +0000 (17:15 -0500)]
10460: Add check for 'dockerOutputDirectory' and raise UnsupportedError for now.

7 years agoMerge branch '10467-client-disconnect' refs #10467
Tom Clegg [Tue, 8 Nov 2016 22:07:47 +0000 (17:07 -0500)]
Merge branch '10467-client-disconnect' refs #10467

7 years ago10460: Want to propagate size fields to output.
Peter Amstutz [Tue, 8 Nov 2016 21:03:55 +0000 (16:03 -0500)]
10460: Want to propagate size fields to output.

7 years ago10467: Merge branch 'master' into 10467-client-disconnect
Tom Clegg [Tue, 8 Nov 2016 20:37:26 +0000 (15:37 -0500)]
10467: Merge branch 'master' into 10467-client-disconnect

7 years ago10460: Don't use adjustFileObjs to set secondaryFiles because it traverses secondaryF...
Peter Amstutz [Tue, 8 Nov 2016 20:26:44 +0000 (15:26 -0500)]
10460: Don't use adjustFileObjs to set secondaryFiles because it traverses secondaryFiles.

7 years ago10467: Use context.Background() for tests and background processes.
Tom Clegg [Tue, 8 Nov 2016 19:07:46 +0000 (14:07 -0500)]
10467: Use context.Background() for tests and background processes.

7 years ago10460: Add implied secondaryFiles based on input parameter spec.
Peter Amstutz [Tue, 8 Nov 2016 19:06:08 +0000 (14:06 -0500)]
10460: Add implied secondaryFiles based on input parameter spec.

7 years ago10466: Report missing 'location' in File object as workflow error.
Peter Amstutz [Tue, 8 Nov 2016 18:42:16 +0000 (13:42 -0500)]
10466: Report missing 'location' in File object as workflow error.

7 years ago10460: Fix initialWorkDir file staging broken by change in visit() behavior for direc...
Peter Amstutz [Tue, 8 Nov 2016 18:00:36 +0000 (13:00 -0500)]
10460: Fix initialWorkDir file staging broken by change in visit() behavior for directories.

7 years agoMerge branch '10468-blob-storage-timeouts' closes #10468
Tom Clegg [Mon, 7 Nov 2016 22:43:17 +0000 (17:43 -0500)]
Merge branch '10468-blob-storage-timeouts' closes #10468

7 years agoMerge branch '10460-cwl-directory-literal' closes #10460
Peter Amstutz [Mon, 7 Nov 2016 22:03:00 +0000 (17:03 -0500)]
Merge branch '10460-cwl-directory-literal' closes #10460

7 years ago10460: Style tweak make_output_collection
Peter Amstutz [Mon, 7 Nov 2016 22:02:18 +0000 (17:02 -0500)]
10460: Style tweak make_output_collection

7 years agoRun keep-exercise tests. No issue #
Tom Clegg [Mon, 7 Nov 2016 22:00:03 +0000 (17:00 -0500)]
Run keep-exercise tests. No issue #

7 years agokeep-exercise: limit PRNG usage to 512KiB per write. No issue #
Tom Clegg [Mon, 7 Nov 2016 21:54:11 +0000 (16:54 -0500)]
keep-exercise: limit PRNG usage to 512KiB per write. No issue #

7 years ago10468: Use consts for defaults.
Tom Clegg [Mon, 7 Nov 2016 20:44:20 +0000 (15:44 -0500)]
10468: Use consts for defaults.

7 years agoPackage keep-exercise.
Ward Vandewege [Mon, 7 Nov 2016 19:10:47 +0000 (14:10 -0500)]
Package keep-exercise.

No issue #

7 years agoUpdate keep-exercise to new SDK. No issue #
Tom Clegg [Mon, 7 Nov 2016 18:41:01 +0000 (13:41 -0500)]
Update keep-exercise to new SDK. No issue #

7 years ago10467: Check for Go 1.7.
Tom Clegg [Mon, 7 Nov 2016 05:51:35 +0000 (00:51 -0500)]
10467: Check for Go 1.7.

7 years ago10467: Update keep-exercise to new SDK.
Tom Clegg [Mon, 7 Nov 2016 05:40:16 +0000 (00:40 -0500)]
10467: Update keep-exercise to new SDK.

7 years ago10467: Interrupt Compare operation if caller disconnects.
Tom Clegg [Mon, 7 Nov 2016 04:41:00 +0000 (23:41 -0500)]
10467: Interrupt Compare operation if caller disconnects.

7 years ago10467: Use ErrClientDisconnect. Convert type assertion panic to 500 error.
Tom Clegg [Mon, 7 Nov 2016 03:11:52 +0000 (22:11 -0500)]
10467: Use ErrClientDisconnect. Convert type assertion panic to 500 error.

7 years ago10467: Fix context error not propagated.
Tom Clegg [Mon, 7 Nov 2016 00:44:22 +0000 (19:44 -0500)]
10467: Fix context error not propagated.

7 years ago10467: Fix ctx not propagated.
Tom Clegg [Mon, 7 Nov 2016 00:41:02 +0000 (19:41 -0500)]
10467: Fix ctx not propagated.

7 years ago10467: Fix context usage: ensure cancel always gets called.
Tom Clegg [Mon, 7 Nov 2016 00:35:05 +0000 (19:35 -0500)]
10467: Fix context usage: ensure cancel always gets called.

7 years ago10467: Tidy up s3 early-cancel.
Tom Clegg [Mon, 7 Nov 2016 00:23:08 +0000 (19:23 -0500)]
10467: Tidy up s3 early-cancel.

7 years ago10467: Use context instead of http.CloseNotifier to interrupt buffer waits.
Tom Clegg [Mon, 7 Nov 2016 00:04:00 +0000 (19:04 -0500)]
10467: Use context instead of http.CloseNotifier to interrupt buffer waits.

7 years ago10467: Fix panic: cannot call CloseNotify() after ServeHTTP finishes.
Tom Clegg [Sun, 6 Nov 2016 23:47:53 +0000 (18:47 -0500)]
10467: Fix panic: cannot call CloseNotify() after ServeHTTP finishes.

7 years ago10467: Abort S3 and release buffer if caller disconnects during S3 PUT request.
Tom Clegg [Sat, 5 Nov 2016 21:17:28 +0000 (17:17 -0400)]
10467: Abort S3 and release buffer if caller disconnects during S3 PUT request.

7 years ago10467: Move http request context setup out to func.
Tom Clegg [Sat, 5 Nov 2016 20:10:34 +0000 (16:10 -0400)]
10467: Move http request context setup out to func.

7 years ago10467: Return "context done" error instead of 404 if client hangs up during GET.
Tom Clegg [Sat, 5 Nov 2016 19:56:45 +0000 (15:56 -0400)]
10467: Return "context done" error instead of 404 if client hangs up during GET.

7 years ago10467: Add Debug flag to config.
Tom Clegg [Sat, 5 Nov 2016 18:53:59 +0000 (14:53 -0400)]
10467: Add Debug flag to config.

7 years ago10467: Abort S3 request and release buffer if caller disconnects while server is...
Tom Clegg [Sat, 5 Nov 2016 01:25:59 +0000 (21:25 -0400)]
10467: Abort S3 request and release buffer if caller disconnects while server is waiting for S3.

7 years ago10468: Add RequestTimeout to Azure volume config.
Tom Clegg [Mon, 7 Nov 2016 15:32:01 +0000 (10:32 -0500)]
10468: Add RequestTimeout to Azure volume config.

7 years ago10468: Use long S3 timeouts by default, instead of waiting forever.
Tom Clegg [Mon, 7 Nov 2016 15:15:55 +0000 (10:15 -0500)]
10468: Use long S3 timeouts by default, instead of waiting forever.

7 years ago10468: Add ConnectTimeout and ReadTimeout configs for S3 volumes.
Tom Clegg [Mon, 7 Nov 2016 15:09:48 +0000 (10:09 -0500)]
10468: Add ConnectTimeout and ReadTimeout configs for S3 volumes.

7 years ago10460: Update cwltool dependency.
Peter Amstutz [Mon, 7 Nov 2016 15:07:32 +0000 (10:07 -0500)]
10460: Update cwltool dependency.

7 years agoMerge branch '10459-cleanup-node-dns' refs #10459
Tom Clegg [Fri, 4 Nov 2016 21:55:17 +0000 (17:55 -0400)]
Merge branch '10459-cleanup-node-dns' refs #10459

7 years ago10459: Tidy up some verbose code.
Tom Clegg [Fri, 4 Nov 2016 21:53:33 +0000 (17:53 -0400)]
10459: Tidy up some verbose code.

7 years ago10459: Remove old IP from DNS when a node changes its IP or hostname.
Tom Clegg [Fri, 4 Nov 2016 21:24:36 +0000 (17:24 -0400)]
10459: Remove old IP from DNS when a node changes its IP or hostname.

7 years ago10460: Support for file and directory literals when constructing output
Peter Amstutz [Fri, 4 Nov 2016 20:54:43 +0000 (16:54 -0400)]
10460: Support for file and directory literals when constructing output
collection.  File locations in final collection reference final output
collection and not intermediate collections.

7 years agoMerge branch '10461-cwl-keepdocker' refs #10461
Peter Amstutz [Fri, 4 Nov 2016 13:41:55 +0000 (09:41 -0400)]
Merge branch '10461-cwl-keepdocker' refs #10461

7 years agoMerge branch '9849-cwl-efficient-log-scan' refs #9849
Peter Amstutz [Fri, 4 Nov 2016 13:41:22 +0000 (09:41 -0400)]
Merge branch '9849-cwl-efficient-log-scan' refs #9849

7 years agoMerge branch '9849-cwl-docker-lookup' refs #9849
Peter Amstutz [Fri, 4 Nov 2016 13:41:08 +0000 (09:41 -0400)]
Merge branch '9849-cwl-docker-lookup' refs #9849

7 years ago10461: Remove arv-keepdocker pull from test_with_arvbox.sh. Only install
Peter Amstutz [Fri, 4 Nov 2016 13:15:18 +0000 (09:15 -0400)]
10461: Remove arv-keepdocker pull from test_with_arvbox.sh.  Only install
arvados-cwl-runner in 'dev' configuration.

7 years ago10461: Add test for fetching arvados_jobs image.
Peter Amstutz [Fri, 4 Nov 2016 13:12:51 +0000 (09:12 -0400)]
10461: Add test for fetching arvados_jobs image.

7 years agoCheck exit code before rethrowing error from catching SystemExit from
Peter Amstutz [Thu, 3 Nov 2016 22:13:29 +0000 (18:13 -0400)]
Check exit code before rethrowing error from catching SystemExit from
keepdocker.main.  no issue #

7 years ago10346: Add storage to sidebar. Tweak headings.
Peter Amstutz [Thu, 3 Nov 2016 20:00:46 +0000 (16:00 -0400)]
10346: Add storage to sidebar.  Tweak headings.

7 years ago10346: Add a brief description about each resource.
Peter Amstutz [Thu, 3 Nov 2016 19:53:17 +0000 (15:53 -0400)]
10346: Add a brief description about each resource.

7 years agoMerge branch '10224-efficient-event-poll-startup' refs #10224
Tom Clegg [Thu, 3 Nov 2016 19:45:13 +0000 (15:45 -0400)]
Merge branch '10224-efficient-event-poll-startup' refs #10224

7 years ago10224: Workaround Firefox failure to parse localized dates in metric locales.
Tom Clegg [Thu, 3 Nov 2016 18:20:44 +0000 (14:20 -0400)]
10224: Workaround Firefox failure to parse localized dates in metric locales.

7 years ago10224: Update tests to expect event_type="delete".
Tom Clegg [Thu, 3 Nov 2016 14:10:14 +0000 (10:10 -0400)]
10224: Update tests to expect event_type="delete".

7 years ago10224: Choose a recent-event threshold without querying the entire event history.
Tom Clegg [Tue, 1 Nov 2016 15:28:49 +0000 (11:28 -0400)]
10224: Choose a recent-event threshold without querying the entire event history.

7 years ago10224: Change event_type to "delete" to match HTTP verb and Python SDK code.
Tom Clegg [Tue, 1 Nov 2016 15:29:42 +0000 (11:29 -0400)]
10224: Change event_type to "delete" to match HTTP verb and Python SDK code.

7 years ago9849: Provide stub log to be parsed.
Peter Amstutz [Thu, 3 Nov 2016 18:22:04 +0000 (14:22 -0400)]
9849: Provide stub log to be parsed.

7 years agoUse --net=host when running jobs inside arvbox to work around Docker-in-Docker
Peter Amstutz [Tue, 1 Nov 2016 17:09:46 +0000 (13:09 -0400)]
Use --net=host when running jobs inside arvbox to work around Docker-in-Docker
networking issues. no issue #

7 years ago9849: Use one regex instead of 3 when scanning log.
Peter Amstutz [Fri, 28 Oct 2016 02:48:47 +0000 (22:48 -0400)]
9849: Use one regex instead of 3 when scanning log.

7 years ago9849: Cache docker lookup.
Peter Amstutz [Fri, 28 Oct 2016 01:54:14 +0000 (21:54 -0400)]
9849: Cache docker lookup.

7 years agoMerge branch '10448-cwl-runner-disable-reuse' closes #10448
Peter Amstutz [Thu, 3 Nov 2016 17:28:04 +0000 (13:28 -0400)]
Merge branch '10448-cwl-runner-disable-reuse' closes #10448

7 years ago10448: Tests include enable_reuse flag and that --disable-reuse is passed to
Peter Amstutz [Thu, 3 Nov 2016 14:29:07 +0000 (10:29 -0400)]
10448: Tests include enable_reuse flag and that --disable-reuse is passed to
inner cwl-runner job or container.

7 years ago10448: Pass through enable_reuse flag to runner.
Peter Amstutz [Wed, 2 Nov 2016 21:21:56 +0000 (17:21 -0400)]
10448: Pass through enable_reuse flag to runner.

7 years agoFix git log to get last log for sdk/cwl or sdk/python only, not repository head refs...
Peter Amstutz [Thu, 3 Nov 2016 16:37:35 +0000 (12:37 -0400)]
Fix git log to get last log for sdk/cwl or sdk/python only, not repository head refs #10438

7 years agorefs #10231
radhika [Wed, 2 Nov 2016 23:46:08 +0000 (19:46 -0400)]
refs #10231
Merge branch '10231-keep-cache-runtime-constraints'

7 years ago10231: validate_runtime_constraints should occur before set_container filter.
radhika [Wed, 2 Nov 2016 22:50:46 +0000 (18:50 -0400)]
10231: validate_runtime_constraints should occur before set_container filter.

7 years agocloses #10231
radhika [Wed, 2 Nov 2016 21:19:12 +0000 (17:19 -0400)]
closes #10231
Merge branch '10231-keep-cache-runtime-constraints'

7 years ago10346: Add in a bunch of technical detail about Keep
Peter Amstutz [Wed, 2 Nov 2016 21:15:18 +0000 (17:15 -0400)]
10346: Add in a bunch of technical detail about Keep

7 years agorefs #10028
radhika [Wed, 2 Nov 2016 21:15:02 +0000 (17:15 -0400)]
refs #10028
Merge branch '10028-api-contents-perf'

7 years ago10028: when an "is_a" filter is used while fetching contents, do not search in all...
radhika [Wed, 2 Nov 2016 18:42:00 +0000 (14:42 -0400)]
10028: when an "is_a" filter is used while fetching contents, do not search in all "other" types.

7 years ago10346: Document api_clients and trusted client.
Peter Amstutz [Tue, 1 Nov 2016 22:05:21 +0000 (18:05 -0400)]
10346: Document api_clients and trusted client.

7 years ago10346: Start documenting how to access Keep.
Peter Amstutz [Tue, 1 Nov 2016 20:04:23 +0000 (16:04 -0400)]
10346: Start documenting how to access Keep.

7 years ago10346: Added UUID type to each resource. Some formatting cleanups.
Peter Amstutz [Tue, 1 Nov 2016 17:37:41 +0000 (13:37 -0400)]
10346: Added UUID type to each resource.  Some formatting cleanups.

7 years ago10346: Document SDK examples uniformly
Peter Amstutz [Tue, 1 Nov 2016 14:20:14 +0000 (10:20 -0400)]
10346: Document SDK examples uniformly

7 years agoMerge branch '10343-dockercleaner-config' refs #10343
Tom Clegg [Mon, 31 Oct 2016 21:38:50 +0000 (17:38 -0400)]
Merge branch '10343-dockercleaner-config' refs #10343

7 years ago10346: Discuss execution. Start writing code examples.
Peter Amstutz [Mon, 31 Oct 2016 21:02:34 +0000 (17:02 -0400)]
10346: Discuss execution.  Start writing code examples.

7 years ago10343: Merge branch 'master' into 10343-dockercleaner-config
Tom Clegg [Mon, 31 Oct 2016 20:08:01 +0000 (16:08 -0400)]
10343: Merge branch 'master' into 10343-dockercleaner-config

7 years ago10343: Add comment about unwieldy ExecStart command.
Tom Clegg [Mon, 31 Oct 2016 19:23:53 +0000 (15:23 -0400)]
10343: Add comment about unwieldy ExecStart command.

7 years agoCompute correct arvados-cwl-runner version, refs #10194
Peter Amstutz [Mon, 31 Oct 2016 18:06:12 +0000 (14:06 -0400)]
Compute correct arvados-cwl-runner version, refs #10194

7 years ago10346: Checkpoint
Peter Amstutz [Mon, 31 Oct 2016 18:04:08 +0000 (14:04 -0400)]
10346: Checkpoint

7 years ago10231: add keep_cache_ram to doc
radhika [Mon, 31 Oct 2016 17:47:20 +0000 (13:47 -0400)]
10231: add keep_cache_ram to doc

7 years agoMerge branch 'master' into 10231-keep-cache-runtime-constraints
radhika [Mon, 31 Oct 2016 16:04:51 +0000 (12:04 -0400)]
Merge branch 'master' into 10231-keep-cache-runtime-constraints

7 years agotag arvados/arvbox-base with latest so that FROM arvados/arvbox-base:latest works...
Peter Amstutz [Mon, 31 Oct 2016 16:00:21 +0000 (12:00 -0400)]
tag arvados/arvbox-base with latest so that FROM arvados/arvbox-base:latest works. refs #10025