arvados.git
7 years agocloses #10677
radhika [Tue, 13 Dec 2016 17:12:58 +0000 (12:12 -0500)]
closes #10677
Merge branch '10677-container-request-rerun'

7 years agoMerge branch '10690-config-dump' refs #10690
Tom Clegg [Thu, 15 Dec 2016 21:26:05 +0000 (16:26 -0500)]
Merge branch '10690-config-dump' refs #10690

7 years agoMerge branch '8460-websocket-go' refs #8460
Tom Clegg [Thu, 15 Dec 2016 21:25:50 +0000 (16:25 -0500)]
Merge branch '8460-websocket-go' refs #8460

7 years ago10690: Add "rake config:dump" task to Workbench and API server.
Tom Clegg [Thu, 15 Dec 2016 20:00:40 +0000 (15:00 -0500)]
10690: Add "rake config:dump" task to Workbench and API server.

7 years agoWe should not hardcode dependencies in our packages for secondary
Ward Vandewege [Thu, 15 Dec 2016 19:52:05 +0000 (14:52 -0500)]
We should not hardcode dependencies in our packages for secondary
dependencies of our dependencies. If you catch my drift.

pyasn1-modules is a dependency of oauth2client which is a dependency of
google-api-python-client. Which is a dependency of
arvados-python-client.

Pinning a dependency on pyasn1-modules=0.0.5 in arvados-python-client
does not make sense. It also does not appear to be necessary at all. And
it currently breaks the CentOS7 package build.

refs #10713

Revert "8654: Pin pyasn1_modules to version that is compatible with pyasn1==0.1.7."

This reverts commit 436105d3e249ad8dc55fb618d0d2c0d8ed29b395.

7 years ago8460: Merge branch 'master' into 8460-websocket-go
Tom Clegg [Thu, 15 Dec 2016 19:47:52 +0000 (14:47 -0500)]
8460: Merge branch 'master' into 8460-websocket-go

7 years agoTidy up queue usage. No issue #
Tom Clegg [Thu, 15 Dec 2016 19:36:56 +0000 (14:36 -0500)]
Tidy up queue usage. No issue #

7 years ago8460: Tolerate None where dicts are expected in events.
Tom Clegg [Thu, 15 Dec 2016 17:16:22 +0000 (12:16 -0500)]
8460: Tolerate None where dicts are expected in events.

7 years ago8460: Rearrange erb so it is also syntactically valid YAML.
Tom Clegg [Thu, 15 Dec 2016 17:12:47 +0000 (12:12 -0500)]
8460: Rearrange erb so it is also syntactically valid YAML.

7 years agoBump schema-salad version to fix rdflib-jsonld dependency. refs #10576
Peter Amstutz [Thu, 15 Dec 2016 16:36:23 +0000 (11:36 -0500)]
Bump schema-salad version to fix rdflib-jsonld dependency. refs #10576

7 years ago8460: Use new ws server for integration tests only if ARVADOS_TEST_EXPERIMENTAL_WS...
Tom Clegg [Thu, 15 Dec 2016 16:29:50 +0000 (11:29 -0500)]
8460: Use new ws server for integration tests only if ARVADOS_TEST_EXPERIMENTAL_WS is set.

7 years ago8460: Move context-aware-logger code to ctxlog pkg.
Tom Clegg [Thu, 15 Dec 2016 16:08:37 +0000 (11:08 -0500)]
8460: Move context-aware-logger code to ctxlog pkg.

7 years ago8460: Add some docs/comments.
Tom Clegg [Thu, 15 Dec 2016 15:40:43 +0000 (10:40 -0500)]
8460: Add some docs/comments.

7 years agoBump libcloud for ex_get_license() bugfix. refs #10686
Peter Amstutz [Wed, 14 Dec 2016 17:01:57 +0000 (12:01 -0500)]
Bump libcloud for ex_get_license() bugfix. refs #10686

7 years agoMerge branch '10576-cwl-keep-fetcher' closes #10576
Peter Amstutz [Wed, 14 Dec 2016 15:01:50 +0000 (10:01 -0500)]
Merge branch '10576-cwl-keep-fetcher' closes #10576

7 years ago10576: Tweak Docker docs and add pipeline deprecation notice.
Peter Amstutz [Tue, 13 Dec 2016 21:40:48 +0000 (16:40 -0500)]
10576: Tweak Docker docs and add pipeline deprecation notice.

7 years ago10576: Fix formatting in cwl-run-options. Fix dangling refs tutorial-pipeline-workbe...
Peter Amstutz [Tue, 13 Dec 2016 21:31:58 +0000 (16:31 -0500)]
10576: Fix formatting in cwl-run-options.  Fix dangling refs tutorial-pipeline-workbench.html

7 years agoMerge branch 'master' into 10576-cwl-keep-fetcher
Peter Amstutz [Tue, 13 Dec 2016 20:51:24 +0000 (15:51 -0500)]
Merge branch 'master' into 10576-cwl-keep-fetcher

Conflicts:
sdk/cwl/setup.py

7 years ago10576: Add ability to run pipeline templates containing CWL from command line.
Peter Amstutz [Tue, 13 Dec 2016 20:46:40 +0000 (15:46 -0500)]
10576: Add ability to run pipeline templates containing CWL from command line.
Split out command line documentation from running CWL tutorial.

7 years ago10576: Add retries to arvcontainer. Add command line options to documentation.
Peter Amstutz [Tue, 13 Dec 2016 19:04:14 +0000 (14:04 -0500)]
10576: Add retries to arvcontainer.  Add command line options to documentation.

7 years ago8460: Fix deadlock at shutdown by closing event stream before unmounting.
Tom Clegg [Tue, 13 Dec 2016 17:46:02 +0000 (12:46 -0500)]
8460: Fix deadlock at shutdown by closing event stream before unmounting.

If llfuse shuts down while a thread is in a "with
llfuse.lock_released" block, the llfuse lock cannot be reacquired, so
the "with" block waits forever instead of exiting. The event listener
thread lands in this state easily because handling a "collection
updated" event makes network requests in a lock_released block.

This deadlock bug started occurring frequently in
tests.test_token_expiry.TokenExpiryTest when using arvados-ws.

7 years ago10677: Add a Re-run button to container_request#show page.
radhika [Tue, 13 Dec 2016 15:44:16 +0000 (10:44 -0500)]
10677: Add a Re-run button to container_request#show page.

7 years ago10576: Integrate CWL extensions page, reorganize running and developing docs.
Peter Amstutz [Tue, 13 Dec 2016 15:33:18 +0000 (10:33 -0500)]
10576: Integrate CWL extensions page, reorganize running and developing docs.

7 years ago10576: Add CWL extensions page.
Peter Amstutz [Mon, 12 Dec 2016 22:02:32 +0000 (17:02 -0500)]
10576: Add CWL extensions page.

7 years agoNodemanager also logs libcloud package version, refs #10587
Peter Amstutz [Mon, 12 Dec 2016 17:46:45 +0000 (12:46 -0500)]
Nodemanager also logs libcloud package version, refs #10587

7 years agoMerge branch '10587-node-manager-report-version' refs #10587
Peter Amstutz [Mon, 12 Dec 2016 16:29:30 +0000 (11:29 -0500)]
Merge branch '10587-node-manager-report-version' refs #10587

7 years ago10587: Print arvados-node-manager package version on startup.
Peter Amstutz [Mon, 12 Dec 2016 16:09:49 +0000 (11:09 -0500)]
10587: Print arvados-node-manager package version on startup.

7 years ago8460: Add event_at and props.{new,old}.{pdh,owner} to websocket v0. Fix fuse crash...
Tom Clegg [Sun, 11 Dec 2016 08:15:54 +0000 (03:15 -0500)]
8460: Add event_at and props.{new,old}.{pdh,owner} to websocket v0. Fix fuse crash when event_at or props.*.* is missing.

7 years ago8460: Add object_kind field.
Tom Clegg [Sun, 11 Dec 2016 06:41:28 +0000 (01:41 -0500)]
8460: Add object_kind field.

7 years ago8460: Merge branch 'master' into 8460-websocket-go
Tom Clegg [Sun, 11 Dec 2016 05:59:55 +0000 (00:59 -0500)]
8460: Merge branch 'master' into 8460-websocket-go

7 years ago8460: Fix copy-paste errors; use html entity instead of literal utf8 bullet.
Tom Clegg [Sun, 11 Dec 2016 05:59:24 +0000 (00:59 -0500)]
8460: Fix copy-paste errors; use html entity instead of literal utf8 bullet.

7 years agoMerge branch '10276-remove-data-manager' closes #10276
Tom Clegg [Fri, 9 Dec 2016 22:11:16 +0000 (17:11 -0500)]
Merge branch '10276-remove-data-manager' closes #10276

7 years ago10576: CWL tutorial documentation refactor WIP.
Peter Amstutz [Fri, 9 Dec 2016 21:37:16 +0000 (16:37 -0500)]
10576: CWL tutorial documentation refactor WIP.

7 years ago10276: Remove data manager (superseded by keep-balance).
Tom Clegg [Fri, 14 Oct 2016 15:04:23 +0000 (11:04 -0400)]
10276: Remove data manager (superseded by keep-balance).

7 years ago10576: Set name default to name of workflow when running single command line
Peter Amstutz [Fri, 9 Dec 2016 20:55:35 +0000 (15:55 -0500)]
10576: Set name default to name of workflow when running single command line
tools from workflow records.

7 years agoMerge branch 'master' into 10576-cwl-keep-fetcher
Peter Amstutz [Fri, 9 Dec 2016 19:20:28 +0000 (14:20 -0500)]
Merge branch 'master' into 10576-cwl-keep-fetcher

Conflicts:
sdk/cwl/arvados_cwl/runner.py
sdk/cwl/tests/test_submit.py

7 years ago10576: Bugfix, set self.uuid on ArvadosContainer.
Peter Amstutz [Fri, 9 Dec 2016 19:18:18 +0000 (14:18 -0500)]
10576: Bugfix, set self.uuid on ArvadosContainer.

7 years agoMerge branch '10587-python-cli-version'
Lucas Di Pentima [Fri, 9 Dec 2016 17:10:32 +0000 (14:10 -0300)]
Merge branch '10587-python-cli-version'
Closes #10587

7 years ago10576: Set template_uuid when creating container request from workflow. Set
Peter Amstutz [Fri, 9 Dec 2016 17:01:52 +0000 (12:01 -0500)]
10576: Set template_uuid when creating container request from workflow.  Set
default container request name from workflow name.

7 years agoMerge branch '10674-wb-workflow-defaults' closes #10674
Peter Amstutz [Fri, 9 Dec 2016 15:59:17 +0000 (10:59 -0500)]
Merge branch '10674-wb-workflow-defaults' closes #10674

7 years ago10674: Add test checking that "Run" button is disabled because required inputs
Peter Amstutz [Fri, 9 Dec 2016 15:58:54 +0000 (10:58 -0500)]
10674: Add test checking that "Run" button is disabled because required inputs
have not been filled in.

7 years ago10585: Merge branch 'master' into 10587-python-cli-version
Lucas Di Pentima [Fri, 9 Dec 2016 15:58:41 +0000 (12:58 -0300)]
10585: Merge branch 'master' into 10587-python-cli-version

7 years ago10674: Extend pipeline/workflow creation test to check that parameter with default...
Peter Amstutz [Fri, 9 Dec 2016 15:16:27 +0000 (10:16 -0500)]
10674: Extend pipeline/workflow creation test to check that parameter with default value is filled in.

7 years agoMerge branch '10681-cwl-wf-name-flag' closes #10681
Peter Amstutz [Thu, 8 Dec 2016 19:09:17 +0000 (14:09 -0500)]
Merge branch '10681-cwl-wf-name-flag' closes #10681

7 years ago10681: Add tests for workflow and template update. Refactor slightly to reduce repet...
Peter Amstutz [Thu, 8 Dec 2016 19:08:42 +0000 (14:08 -0500)]
10681: Add tests for workflow and template update.  Refactor slightly to reduce repetition.

7 years agoMerge branch '10467-client-disconnect' refs #10467
Tom Clegg [Thu, 8 Dec 2016 16:43:42 +0000 (11:43 -0500)]
Merge branch '10467-client-disconnect' refs #10467

7 years ago10681: Apply --name flag to creating and updating workflows and pipeline templates.
Peter Amstutz [Thu, 8 Dec 2016 16:15:02 +0000 (11:15 -0500)]
10681: Apply --name flag to creating and updating workflows and pipeline templates.

7 years agoMerge branch '10686-gce-list-image-project' refs #10686
Peter Amstutz [Thu, 8 Dec 2016 15:33:10 +0000 (10:33 -0500)]
Merge branch '10686-gce-list-image-project' refs #10686

7 years ago10686: Specify explicit project when listing images to fix problem with default
Peter Amstutz [Thu, 8 Dec 2016 15:24:19 +0000 (10:24 -0500)]
10686: Specify explicit project when listing images to fix problem with default
behavior listing debian-cloud images.

7 years agoMerge branch '10681-cwl-name-flag' closes #10681
Peter Amstutz [Wed, 7 Dec 2016 21:28:14 +0000 (16:28 -0500)]
Merge branch '10681-cwl-name-flag' closes #10681

7 years agoMerge branch '10684-ssl-ro' refs #10684
Tom Clegg [Wed, 7 Dec 2016 20:58:35 +0000 (15:58 -0500)]
Merge branch '10684-ssl-ro' refs #10684

7 years ago10684: Use read-only bind mounts for certs and crunchrunner.
Tom Clegg [Wed, 7 Dec 2016 20:57:09 +0000 (15:57 -0500)]
10684: Use read-only bind mounts for certs and crunchrunner.

7 years ago10587: Added setuptools dependency to install_requires lists on arvados-cwl-runner...
Lucas Di Pentima [Wed, 7 Dec 2016 20:56:28 +0000 (17:56 -0300)]
10587: Added setuptools dependency to install_requires lists on arvados-cwl-runner, arvados-python-client, arvados-fuse and arvados-node-manager.

7 years ago10576: Fix tests for cwl.input.json part of container request record and not
Peter Amstutz [Wed, 7 Dec 2016 20:51:06 +0000 (15:51 -0500)]
10576: Fix tests for cwl.input.json part of container request record and not
placed in keep.

7 years ago10587: Added setuptools dependency to fpm package building on arvados-python-client...
Lucas Di Pentima [Wed, 7 Dec 2016 20:36:43 +0000 (17:36 -0300)]
10587: Added setuptools dependency to fpm package building on arvados-python-client, arvados-fuse & arvados-node-manager.

7 years ago10681: Add --name flag to set name on pipeline instances and container
Peter Amstutz [Wed, 7 Dec 2016 20:09:20 +0000 (15:09 -0500)]
10681: Add --name flag to set name on pipeline instances and container
requests.

7 years ago10587: Removed use of multiprocessing module on --version tests. Added a common stder...
Lucas Di Pentima [Wed, 7 Dec 2016 18:29:23 +0000 (15:29 -0300)]
10587: Removed use of multiprocessing module on --version tests. Added a common stderr/stdout redirector context manager for code reuse.

7 years agocloses #10300
radhika [Wed, 7 Dec 2016 16:54:30 +0000 (11:54 -0500)]
closes #10300
Merge branch '10300-pipeline-display-error'

7 years ago8460: Tweak logging keys, more similar to keepstore.
Tom Clegg [Wed, 7 Dec 2016 16:43:27 +0000 (11:43 -0500)]
8460: Tweak logging keys, more similar to keepstore.

7 years ago10576: Fill in cwl.input.json as a "json" mount instead of creating a new collection.
Peter Amstutz [Wed, 7 Dec 2016 16:09:52 +0000 (11:09 -0500)]
10576: Fill in cwl.input.json as a "json" mount instead of creating a new collection.

7 years agoMerge branch 'master' into 10576-cwl-keep-fetcher
Peter Amstutz [Wed, 7 Dec 2016 15:42:11 +0000 (10:42 -0500)]
Merge branch 'master' into 10576-cwl-keep-fetcher

7 years ago10674: Initialize defaults when creating container request from workflow.
Peter Amstutz [Tue, 6 Dec 2016 22:17:49 +0000 (17:17 -0500)]
10674: Initialize defaults when creating container request from workflow.

7 years ago10587: Added tests to Python commands to check for the --version argument.
Lucas Di Pentima [Tue, 6 Dec 2016 22:54:04 +0000 (19:54 -0300)]
10587: Added tests to Python commands to check for the --version argument.

7 years ago10576: Bump cwltool, schema-salad and rdflib-jsonld package versions.
Peter Amstutz [Tue, 6 Dec 2016 20:59:14 +0000 (15:59 -0500)]
10576: Bump cwltool, schema-salad and rdflib-jsonld package versions.

7 years ago10300: old pipeline display is failing due to min_nodes in runtime_constraints being...
radhika [Tue, 6 Dec 2016 20:40:49 +0000 (15:40 -0500)]
10300: old pipeline display is failing due to min_nodes in runtime_constraints being a string; convert to integer to be able to view it.

7 years ago10576: Bump cwltool and schema-salad versions.
Peter Amstutz [Tue, 6 Dec 2016 20:05:15 +0000 (15:05 -0500)]
10576: Bump cwltool and schema-salad versions.

7 years agoREADME.md: add link to the Go Report Card for Arvados.
Ward Vandewege [Tue, 6 Dec 2016 19:56:10 +0000 (14:56 -0500)]
README.md: add link to the Go Report Card for Arvados.

No issue #

7 years agocloses #10215
radhika [Tue, 6 Dec 2016 18:58:18 +0000 (13:58 -0500)]
closes #10215
Merge branch '10215-arvados-cwl-runner-version'

7 years agoMerge branch 'master' into 10215-arvados-cwl-runner-version
radhika [Tue, 6 Dec 2016 18:57:55 +0000 (13:57 -0500)]
Merge branch 'master' into 10215-arvados-cwl-runner-version

7 years ago8460: Add install doc page.
Tom Clegg [Tue, 6 Dec 2016 18:29:26 +0000 (13:29 -0500)]
8460: Add install doc page.

7 years ago8460: Add json mime type.
Tom Clegg [Tue, 6 Dec 2016 16:07:00 +0000 (11:07 -0500)]
8460: Add json mime type.

7 years ago8460: Fix binary path.
Tom Clegg [Tue, 6 Dec 2016 16:06:44 +0000 (11:06 -0500)]
8460: Fix binary path.

7 years agocloses #10604
radhika [Tue, 6 Dec 2016 15:46:45 +0000 (10:46 -0500)]
closes #10604
Merge branch '10604-crunch2-user-guide'

7 years ago10604: rename tutorial-pipeline-workbench.html as tutorial-workbench-workbench.html
radhika [Tue, 6 Dec 2016 15:44:41 +0000 (10:44 -0500)]
10604: rename tutorial-pipeline-workbench.html as tutorial-workbench-workbench.html

7 years agoMerge branch 'master' into 10604-crunch2-user-guide
radhika [Tue, 6 Dec 2016 15:31:50 +0000 (10:31 -0500)]
Merge branch 'master' into 10604-crunch2-user-guide

7 years agoMerge branch '10519-cr-fiddlesticks' closes #10519
Peter Amstutz [Tue, 6 Dec 2016 14:46:11 +0000 (09:46 -0500)]
Merge branch '10519-cr-fiddlesticks' closes #10519

7 years agoMerge branch '10658-crunch-run-missing-files' closes #10658
Peter Amstutz [Tue, 6 Dec 2016 14:40:12 +0000 (09:40 -0500)]
Merge branch '10658-crunch-run-missing-files' closes #10658

7 years agoMerge branch '10649-squeue-stderr' closes #10649
Peter Amstutz [Tue, 6 Dec 2016 14:39:09 +0000 (09:39 -0500)]
Merge branch '10649-squeue-stderr' closes #10649

7 years agoFix typo in comment.
Ward Vandewege [Mon, 5 Dec 2016 22:13:22 +0000 (17:13 -0500)]
Fix typo in comment.

No issue #

7 years ago10576: Running jobs from keep: and arv: prefixes WIP. Tests passing, needs some...
Peter Amstutz [Mon, 5 Dec 2016 22:06:37 +0000 (17:06 -0500)]
10576: Running jobs from keep: and arv: prefixes WIP.  Tests passing, needs some integration testing.

7 years ago8460: golint fixes.
Tom Clegg [Mon, 5 Dec 2016 20:40:25 +0000 (15:40 -0500)]
8460: golint fixes.

7 years ago8460: Log entry at startup.
Tom Clegg [Mon, 5 Dec 2016 20:20:16 +0000 (15:20 -0500)]
8460: Log entry at startup.

7 years ago8460: Cancel context before returning from handler.
Tom Clegg [Mon, 5 Dec 2016 20:20:03 +0000 (15:20 -0500)]
8460: Cancel context before returning from handler.

7 years ago8460: Merge branch 'master' into 8460-websocket-go
Tom Clegg [Mon, 5 Dec 2016 19:36:15 +0000 (14:36 -0500)]
8460: Merge branch 'master' into 8460-websocket-go

7 years ago8460: Add simple /status.json
Tom Clegg [Mon, 5 Dec 2016 19:35:25 +0000 (14:35 -0500)]
8460: Add simple /status.json

7 years ago10215: if version arg is used, just display version info and exit.
radhika [Mon, 5 Dec 2016 18:23:42 +0000 (13:23 -0500)]
10215: if version arg is used, just display version info and exit.

7 years ago10576: Add resolver to execute from keep references and arvados workflow
Peter Amstutz [Mon, 5 Dec 2016 15:40:42 +0000 (10:40 -0500)]
10576: Add resolver to execute from keep references and arvados workflow
records.  Add test for keep references in workflow default input.

7 years ago8460: status.json -> debug.json
Tom Clegg [Sat, 3 Dec 2016 08:52:43 +0000 (03:52 -0500)]
8460: status.json -> debug.json

7 years ago8460: Rename pg -> event_source.go
Tom Clegg [Sat, 3 Dec 2016 08:44:45 +0000 (03:44 -0500)]
8460: Rename pg -> event_source.go

7 years ago10576: Tests & fixes for keep ref url join.
Peter Amstutz [Fri, 2 Dec 2016 22:17:25 +0000 (17:17 -0500)]
10576: Tests & fixes for keep ref url join.

7 years ago10576: Working on fetch & url join for keep references.
Peter Amstutz [Fri, 2 Dec 2016 21:50:06 +0000 (16:50 -0500)]
10576: Working on fetch & url join for keep references.

7 years ago10519: added a test to check Run button when an empty workflow is used with no requir...
radhika [Fri, 2 Dec 2016 20:09:14 +0000 (15:09 -0500)]
10519: added a test to check Run button when an empty workflow is used with no required inputs.

7 years agoBump package iteration on arvados-docker-cleaner for dependency fix. no issue #
Peter Amstutz [Fri, 2 Dec 2016 20:05:08 +0000 (15:05 -0500)]
Bump package iteration on arvados-docker-cleaner for dependency fix.  no issue #

7 years agoPin websocket-client backport dependency to 0.37.0 to avoid tertiary dependency
Peter Amstutz [Fri, 2 Dec 2016 19:47:01 +0000 (14:47 -0500)]
Pin websocket-client backport dependency to 0.37.0 to avoid tertiary dependency
problems on debian7 and ubuntu1204.  no issue #

7 years agoMerge branch 'master' into 10519-cr-fiddlesticks
radhika [Fri, 2 Dec 2016 18:52:52 +0000 (13:52 -0500)]
Merge branch 'master' into 10519-cr-fiddlesticks

7 years agorefs #10293
radhika [Fri, 2 Dec 2016 18:52:13 +0000 (13:52 -0500)]
refs #10293
Merge branch '10293-cr-log-and-output-in-wb'

7 years agoMerge branch '5737-fix-broken-test-with-231'
Lucas Di Pentima [Fri, 2 Dec 2016 18:48:40 +0000 (15:48 -0300)]
Merge branch '5737-fix-broken-test-with-231'
refs #5737

7 years ago10293: update test assertion to expect output collection name, instead of pdh.
radhika [Fri, 2 Dec 2016 18:48:12 +0000 (13:48 -0500)]
10293: update test assertion to expect output collection name, instead of pdh.

7 years ago5737: Updated test so it pass on both 2.1 and 2.3 ruby versions.
Lucas Di Pentima [Fri, 2 Dec 2016 18:45:19 +0000 (15:45 -0300)]
5737: Updated test so it pass on both 2.1 and 2.3 ruby versions.

7 years agoMerge branch 'master' into 10293-cr-log-and-output-in-wb
radhika [Fri, 2 Dec 2016 18:33:20 +0000 (13:33 -0500)]
Merge branch 'master' into 10293-cr-log-and-output-in-wb