arvados.git
5 years ago9945: first set of changes.
Ward Vandewege [Thu, 20 Dec 2018 16:51:39 +0000 (11:51 -0500)]
9945: first set of changes.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '14551-wb-login-as-token'
Lucas Di Pentima [Tue, 4 Dec 2018 20:51:13 +0000 (17:51 -0300)]
Merge branch '14551-wb-login-as-token'
Closes #14551

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14574-thread-count-1' refs #14574
Peter Amstutz [Tue, 4 Dec 2018 16:32:59 +0000 (11:32 -0500)]
Merge branch '14574-thread-count-1' refs #14574

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14576-query-perf' refs #14576
Peter Amstutz [Tue, 4 Dec 2018 16:32:49 +0000 (11:32 -0500)]
Merge branch '14576-query-perf' refs #14576

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14576: Use the 'in' test for user queries
Peter Amstutz [Tue, 4 Dec 2018 16:17:12 +0000 (11:17 -0500)]
14576: Use the 'in' test for user queries

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14576: Adjust query for admins that filters out trashed items
Peter Amstutz [Tue, 4 Dec 2018 15:43:01 +0000 (10:43 -0500)]
14576: Adjust query for admins that filters out trashed items

refs #14576

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14576: Adjust query for admins that filters out trashed items
Peter Amstutz [Tue, 4 Dec 2018 15:43:01 +0000 (10:43 -0500)]
14576: Adjust query for admins that filters out trashed items

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFix docker push command in run-build-docker-jobs-image.sh
Ward Vandewege [Tue, 4 Dec 2018 13:58:36 +0000 (08:58 -0500)]
Fix docker push command in run-build-docker-jobs-image.sh

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoIf a version tag is passed to run-build-docker-jobs-image.sh, actually use that
Ward Vandewege [Mon, 3 Dec 2018 21:29:24 +0000 (16:29 -0500)]
If a version tag is passed to run-build-docker-jobs-image.sh, actually use that
version of our codebase for the packages installed in the image.

run-build-docker-jobs-image.sh now also supports our ARVADOS_BUILDING_VERSION
and ARVADOS_BUILDING_ITERATION mechanism, so versions can be overridden that
way.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14574: Default --thread-count=1 until threading is more fully baked.
Peter Amstutz [Mon, 3 Dec 2018 21:23:28 +0000 (16:23 -0500)]
14574: Default --thread-count=1 until threading is more fully baked.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14560: Limit to_tsvector input to 8K.
Tom Clegg [Mon, 3 Dec 2018 17:00:39 +0000 (12:00 -0500)]
14560: Limit to_tsvector input to 8K.

refs #14560
refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14560-tsvector-too-long' refs #14560
Peter Amstutz [Sun, 2 Dec 2018 02:09:40 +0000 (21:09 -0500)]
Merge branch '14560-tsvector-too-long' refs #14560

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14560: Only index the first 1M bytes of ft-searchable data.
Tom Clegg [Sat, 1 Dec 2018 21:11:28 +0000 (16:11 -0500)]
14560: Only index the first 1M bytes of ft-searchable data.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14558-cwl-pkg-deps' refs #14558
Peter Amstutz [Fri, 30 Nov 2018 21:04:06 +0000 (16:04 -0500)]
Merge branch '14558-cwl-pkg-deps' refs #14558

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14558: Version dependency fixes.
Peter Amstutz [Fri, 30 Nov 2018 20:32:27 +0000 (15:32 -0500)]
14558: Version dependency fixes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14510-cwl-perf' refs #14510
Peter Amstutz [Fri, 30 Nov 2018 16:58:20 +0000 (11:58 -0500)]
Merge branch '14510-cwl-perf' refs #14510

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14510: Bugfix order of precedence for setting collection cache size
Peter Amstutz [Fri, 30 Nov 2018 15:50:20 +0000 (10:50 -0500)]
14510: Bugfix order of precedence for setting collection cache size

Fix a couple other buggy tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14510: Don't initialize workflow redundantly.
Peter Amstutz [Fri, 30 Nov 2018 14:25:47 +0000 (09:25 -0500)]
14510: Don't initialize workflow redundantly.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMake sure the dev version prefix is based on the latest tag in the repo, not
Ward Vandewege [Fri, 30 Nov 2018 02:08:05 +0000 (21:08 -0500)]
Make sure the dev version prefix is based on the latest tag in the repo, not
the latest tag on the master branch (part 3). Make sure the change also works
in Python 3.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMake sure the dev version prefix is based on the latest tag in the repo,
Ward Vandewege [Thu, 29 Nov 2018 19:49:31 +0000 (14:49 -0500)]
Make sure the dev version prefix is based on the latest tag in the repo,
not the latest tag on the master branch (part 2).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14551: Uses v2 token to log in as a user.
Lucas Di Pentima [Thu, 29 Nov 2018 14:06:54 +0000 (11:06 -0300)]
14551: Uses v2 token to log in as a user.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14551: Test exposing the bug.
Lucas Di Pentima [Thu, 29 Nov 2018 16:27:16 +0000 (13:27 -0300)]
14551: Test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMake sure the dev version prefix is based on the latest tag in the repo,
Ward Vandewege [Thu, 29 Nov 2018 14:27:59 +0000 (09:27 -0500)]
Make sure the dev version prefix is based on the latest tag in the repo,
not the latest tag on the master branch.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoWork around ruamel.yaml packaging issue.
Ward Vandewege [Thu, 29 Nov 2018 03:23:34 +0000 (22:23 -0500)]
Work around ruamel.yaml packaging issue.

closes #14552

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoStop relying on the keyservers during arvbox docker image build, they are too
Ward Vandewege [Wed, 28 Nov 2018 20:12:19 +0000 (15:12 -0500)]
Stop relying on the keyservers during arvbox docker image build, they are too
unreliable.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14510: Set collection metadata cache size as 150% of sum of inputs
Peter Amstutz [Wed, 28 Nov 2018 19:42:06 +0000 (14:42 -0500)]
14510: Set collection metadata cache size as 150% of sum of inputs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoRelease 1.2.1 does not require special attention when upgrading.
Ward Vandewege [Wed, 28 Nov 2018 19:19:23 +0000 (14:19 -0500)]
Release 1.2.1 does not require special attention when upgrading.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '14345-add-webdav-user-doc'
Ward Vandewege [Wed, 28 Nov 2018 18:32:11 +0000 (13:32 -0500)]
Merge branch '14345-add-webdav-user-doc'

refs #14345

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch 'master' into 14345-add-webdav-user-doc
Ward Vandewege [Wed, 28 Nov 2018 18:29:43 +0000 (13:29 -0500)]
Merge branch 'master' into 14345-add-webdav-user-doc

refs #14345

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14345: Some more language tweaks.
Ward Vandewege [Wed, 28 Nov 2018 18:29:14 +0000 (13:29 -0500)]
14345: Some more language tweaks.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoAdd OS X description & tweak Windows
Tom Morris [Wed, 28 Nov 2018 18:19:00 +0000 (13:19 -0500)]
Add OS X description & tweak Windows

refs #14345

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years ago14535: fix typos.
Ward Vandewege [Wed, 28 Nov 2018 18:12:34 +0000 (13:12 -0500)]
14535: fix typos.

refs #14535

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoAdd user documentation for accessing Keep from Windows.
Ward Vandewege [Wed, 28 Nov 2018 16:40:53 +0000 (11:40 -0500)]
Add user documentation for accessing Keep from Windows.

refs #14345

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14510: Estimate collection cache size wip
Peter Amstutz [Tue, 27 Nov 2018 23:13:30 +0000 (18:13 -0500)]
14510: Estimate collection cache size wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14510: Update tests/docs for --collection-cache-size
Peter Amstutz [Tue, 27 Nov 2018 22:26:07 +0000 (17:26 -0500)]
14510: Update tests/docs for --collection-cache-size

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14510: Setting collection cache wip
Peter Amstutz [Tue, 27 Nov 2018 20:53:07 +0000 (15:53 -0500)]
14510: Setting collection cache wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14510: Perfomance fixes
Peter Amstutz [Tue, 20 Nov 2018 19:33:53 +0000 (14:33 -0500)]
14510: Perfomance fixes

* Add --collection-cache to enable users to workaround cache thrashing

* Limit task queue size.  Release workflow lock when attempting to
  enqueue a task (which now may block).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoRestore default value of --on-error to 'continue' refs #14440
Peter Amstutz [Tue, 27 Nov 2018 20:54:37 +0000 (15:54 -0500)]
Restore default value of --on-error to 'continue' refs #14440

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14440-fed-user-doc' refs #14440
Peter Amstutz [Tue, 27 Nov 2018 19:18:31 +0000 (14:18 -0500)]
Merge branch '14440-fed-user-doc' refs #14440

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Fix formatting of cwl-run-options table.
Peter Amstutz [Tue, 27 Nov 2018 19:14:54 +0000 (14:14 -0500)]
14440: Fix formatting of cwl-run-options table.

Also expand help text of --on-error option a bit.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14535-ruamel-dependencies'
Nico Cesar [Tue, 27 Nov 2018 18:15:18 +0000 (13:15 -0500)]
Merge branch '14535-ruamel-dependencies'

refs #14535

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <ncesar@veritasgenetics.com>

5 years ago14535: bumping package revision up for python-ruamel.yaml
Nico Cesar [Tue, 27 Nov 2018 18:06:36 +0000 (13:06 -0500)]
14535: bumping package revision up for python-ruamel.yaml

refs #14535

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <ncesar@veritasgenetics.com>

5 years ago14440: Refresh cwl-run-options, add note about dockerCollectionPDH
Peter Amstutz [Tue, 27 Nov 2018 17:47:11 +0000 (12:47 -0500)]
14440: Refresh cwl-run-options, add note about dockerCollectionPDH

Also add note about workbench not displaying remote steps.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14535: add dependencies for ruamel.yaml
Nico Cesar [Tue, 27 Nov 2018 16:27:44 +0000 (11:27 -0500)]
14535: add dependencies for ruamel.yaml

refs #14535

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <ncesar@veritasgenetics.com>

5 years agoMerge branch '14519-wb-textile-issues'
Lucas Di Pentima [Mon, 26 Nov 2018 19:43:29 +0000 (16:43 -0300)]
Merge branch '14519-wb-textile-issues'
Closes #14519

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14519: Adds more table related tags to the whitelist
Lucas Di Pentima [Mon, 26 Nov 2018 19:42:11 +0000 (16:42 -0300)]
14519: Adds more table related tags to the whitelist

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14519: Fixes bug by adding table tags to the sanitizer white list
Lucas Di Pentima [Wed, 21 Nov 2018 22:32:23 +0000 (19:32 -0300)]
14519: Fixes bug by adding table tags to the sanitizer white list

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14519: Test exposing the regression
Lucas Di Pentima [Wed, 21 Nov 2018 22:31:51 +0000 (19:31 -0300)]
14519: Test exposing the regression

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14440: Add discussion of federation features.
Peter Amstutz [Mon, 26 Nov 2018 17:37:07 +0000 (12:37 -0500)]
14440: Add discussion of federation features.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Federated workflow diagram.
Peter Amstutz [Wed, 21 Nov 2018 21:24:31 +0000 (16:24 -0500)]
14440: Federated workflow diagram.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Comments on how federated.cwl works
Peter Amstutz [Wed, 21 Nov 2018 21:23:59 +0000 (16:23 -0500)]
14440: Comments on how federated.cwl works

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Add project
Peter Amstutz [Wed, 21 Nov 2018 19:46:06 +0000 (14:46 -0500)]
14440: Add project

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Add project_uuid to ClusterTarget example
Peter Amstutz [Wed, 21 Nov 2018 18:42:38 +0000 (13:42 -0500)]
14440: Add project_uuid to ClusterTarget example

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Federated workflow docs WIP
Peter Amstutz [Wed, 21 Nov 2018 16:33:40 +0000 (11:33 -0500)]
14440: Federated workflow docs WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14440: Working on docs for federated workflows.
Peter Amstutz [Tue, 20 Nov 2018 22:20:52 +0000 (17:20 -0500)]
14440: Working on docs for federated workflows.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14198-make-builder' refs #14198
Peter Amstutz [Mon, 26 Nov 2018 15:42:02 +0000 (10:42 -0500)]
Merge branch '14198-make-builder' refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Restore behavior of creating an empty Builder object
Peter Amstutz [Mon, 26 Nov 2018 15:13:39 +0000 (10:13 -0500)]
14198: Restore behavior of creating an empty Builder object

Used for do_eval() on set_cluster_target().  Calling _init_job() turns
out to have some unintended side effects as well as doing redundant
work.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFinish fixing setting project_uuid from runtimeContext, fix tests.
Peter Amstutz [Wed, 21 Nov 2018 21:39:59 +0000 (16:39 -0500)]
Finish fixing setting project_uuid from runtimeContext, fix tests.

refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFix ClusterTarget.project_uuid to correctly set the project uuid
Peter Amstutz [Wed, 21 Nov 2018 19:46:38 +0000 (14:46 -0500)]
Fix ClusterTarget.project_uuid to correctly set the project uuid

refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14345-nginx-proxy-request-buffering'
Tom Clegg [Wed, 21 Nov 2018 19:44:23 +0000 (14:44 -0500)]
Merge branch '14345-nginx-proxy-request-buffering'

refs #14345

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14345: Disable proxy request buffering in Nginx keepproxy example.
Tom Clegg [Wed, 21 Nov 2018 19:34:17 +0000 (14:34 -0500)]
14345: Disable proxy request buffering in Nginx keepproxy example.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14345: Disable Nginx proxy request buffering for keep-web requests.
Tom Clegg [Wed, 21 Nov 2018 19:14:04 +0000 (14:14 -0500)]
14345: Disable Nginx proxy request buffering for keep-web requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoFix --submit-runner-cluster tests refs #14198
Peter Amstutz [Wed, 21 Nov 2018 19:00:04 +0000 (14:00 -0500)]
Fix --submit-runner-cluster tests refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14345-proppatch'
Tom Clegg [Wed, 21 Nov 2018 18:28:00 +0000 (13:28 -0500)]
Merge branch '14345-proppatch'

refs #14345

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14198-fed-testing' refs #14198
Peter Amstutz [Wed, 21 Nov 2018 18:06:00 +0000 (13:06 -0500)]
Merge branch '14198-fed-testing' refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Call validate_cluster_target on --submit-runner-cluster
Peter Amstutz [Wed, 21 Nov 2018 17:41:11 +0000 (12:41 -0500)]
14198: Call validate_cluster_target on --submit-runner-cluster

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add check for uuidPrefix if local cluster isn't in the remoteHosts.
Peter Amstutz [Tue, 20 Nov 2018 22:26:17 +0000 (17:26 -0500)]
14198: Add check for uuidPrefix if local cluster isn't in the remoteHosts.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Update build.list for latest cwltool/schema-salad
Peter Amstutz [Tue, 20 Nov 2018 18:30:40 +0000 (13:30 -0500)]
14198: Update build.list for latest cwltool/schema-salad

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Fix secondaryFile tests
Peter Amstutz [Mon, 19 Nov 2018 20:05:01 +0000 (15:05 -0500)]
14198: Fix secondaryFile tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Tests for hints in different places
Peter Amstutz [Sat, 17 Nov 2018 03:01:45 +0000 (22:01 -0500)]
14198: Tests for hints in different places

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Tests prefer setting ClusterTarget hint on the workflow step.
Peter Amstutz [Fri, 16 Nov 2018 21:39:09 +0000 (16:39 -0500)]
14198: Tests prefer setting ClusterTarget hint on the workflow step.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Test cases for places that ClusterTarget can appear
Peter Amstutz [Fri, 16 Nov 2018 21:18:19 +0000 (16:18 -0500)]
14198: Test cases for places that ClusterTarget can appear

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Set ClusterTarget hint on steps instead of tools
Peter Amstutz [Fri, 16 Nov 2018 15:59:52 +0000 (10:59 -0500)]
14198: Set ClusterTarget hint on steps instead of tools

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Rewrite README
Peter Amstutz [Wed, 14 Nov 2018 19:44:19 +0000 (14:44 -0500)]
14198: Rewrite README

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Connect test cases to outputs
Peter Amstutz [Wed, 14 Nov 2018 04:10:07 +0000 (23:10 -0500)]
14198: Connect test cases to outputs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add threestep-remote
Peter Amstutz [Wed, 14 Nov 2018 03:56:56 +0000 (22:56 -0500)]
14198: Add threestep-remote

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add scatter-gather
Peter Amstutz [Wed, 14 Nov 2018 03:06:36 +0000 (22:06 -0500)]
14198: Add scatter-gather

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add test twostep-remote-copy-to-home
Peter Amstutz [Tue, 13 Nov 2018 22:28:59 +0000 (17:28 -0500)]
14198: Add test twostep-remote-copy-to-home

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Fix collection PDHs for remote-case
Peter Amstutz [Tue, 13 Nov 2018 21:37:34 +0000 (16:37 -0500)]
14198: Fix collection PDHs for remote-case

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add some basic doc comments.
Peter Amstutz [Fri, 9 Nov 2018 20:48:14 +0000 (15:48 -0500)]
14198: Add some basic doc comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add remote-case and twostep-both-remote
Peter Amstutz [Fri, 9 Nov 2018 20:33:05 +0000 (15:33 -0500)]
14198: Add remote-case and twostep-both-remote

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add more twostep tests
Peter Amstutz [Fri, 9 Nov 2018 20:02:50 +0000 (15:02 -0500)]
14198: Add more twostep tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Working on two step test cases
Peter Amstutz [Fri, 9 Nov 2018 15:47:23 +0000 (10:47 -0500)]
14198: Working on two step test cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Refactor arvbox federation setup
Peter Amstutz [Fri, 9 Nov 2018 03:19:15 +0000 (22:19 -0500)]
14198: Refactor arvbox federation setup

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Federation feature and testing
Peter Amstutz [Fri, 16 Nov 2018 21:15:11 +0000 (16:15 -0500)]
14198: Federation feature and testing

* Evaluate ClusterTarget in the job step that declares it

* Don't use get_requirement to inherit ClusterTarget.

* Workflow to start and configure a federation of N arvboxes, initialize test user

* Test cases.

* Prepare/scrub before running test

* Use --always-submit-runner

* Tests use isolated inputs.  Return success boolean

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Set arvrunner before calling super constructor
Peter Amstutz [Fri, 16 Nov 2018 19:21:07 +0000 (14:21 -0500)]
14198: Set arvrunner before calling super constructor

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Set arvrunner on ArvadosWorkflowStep
Peter Amstutz [Fri, 16 Nov 2018 19:12:51 +0000 (14:12 -0500)]
14198: Set arvrunner on ArvadosWorkflowStep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Set fake 'class' so _init_job is happy
Peter Amstutz [Fri, 16 Nov 2018 18:55:49 +0000 (13:55 -0500)]
14198: Set fake 'class' so _init_job is happy

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Fix jobstep builder init
Peter Amstutz [Fri, 16 Nov 2018 16:44:55 +0000 (11:44 -0500)]
14198: Fix jobstep builder init

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Fix ruamel.yaml versioning
Peter Amstutz [Fri, 16 Nov 2018 16:28:27 +0000 (11:28 -0500)]
14198: Fix ruamel.yaml versioning

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Construct ArvadosWorkflowStep
Peter Amstutz [Fri, 16 Nov 2018 16:26:51 +0000 (11:26 -0500)]
14198: Construct ArvadosWorkflowStep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Update version deps for schema-salad, cwltool, ruamel.yaml, requests
Peter Amstutz [Fri, 16 Nov 2018 15:28:24 +0000 (10:28 -0500)]
14198: Update version deps for schema-salad, cwltool, ruamel.yaml, requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Update cwltool and schema-salad
Peter Amstutz [Wed, 14 Nov 2018 18:22:16 +0000 (13:22 -0500)]
14198: Update cwltool and schema-salad

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add --always-submit-runner
Peter Amstutz [Wed, 7 Nov 2018 21:24:02 +0000 (16:24 -0500)]
14198: Add --always-submit-runner

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add --submit-runner-cluster test
Peter Amstutz [Mon, 5 Nov 2018 22:44:23 +0000 (17:44 -0500)]
14198: Add --submit-runner-cluster test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Unit tests pass again
Peter Amstutz [Mon, 5 Nov 2018 15:16:48 +0000 (10:16 -0500)]
14198: Unit tests pass again

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: test_submit passes
Peter Amstutz [Fri, 2 Nov 2018 21:05:20 +0000 (17:05 -0400)]
14198: test_submit passes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: More test updates
Peter Amstutz [Fri, 2 Nov 2018 20:41:01 +0000 (16:41 -0400)]
14198: More test updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Bringing unit tests up to date WIP
Peter Amstutz [Fri, 2 Nov 2018 20:05:29 +0000 (16:05 -0400)]
14198: Bringing unit tests up to date WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14198: Add paging on container list, check for valid cluster id
Peter Amstutz [Thu, 1 Nov 2018 18:42:12 +0000 (14:42 -0400)]
14198: Add paging on container list, check for valid cluster id

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>