arvados.git
6 years ago14199: Reduce logging noise from azure test stubs.
Tom Clegg [Tue, 9 Oct 2018 20:07:17 +0000 (16:07 -0400)]
14199: Reduce logging noise from azure test stubs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Merge branch 'master' into 14199-copy-from-remote
Tom Clegg [Tue, 9 Oct 2018 19:56:26 +0000 (15:56 -0400)]
14199: Merge branch 'master' into 14199-copy-from-remote

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14196-api-v2-tokens' refs #14196
Peter Amstutz [Tue, 9 Oct 2018 19:20:38 +0000 (15:20 -0400)]
Merge branch '14196-api-v2-tokens' refs #14196

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Revert debug change to Access-Control-Max-Age
Peter Amstutz [Tue, 9 Oct 2018 19:19:31 +0000 (15:19 -0400)]
14196: Revert debug change to Access-Control-Max-Age

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14274-factory-bot' closes #14274
Peter Amstutz [Tue, 9 Oct 2018 19:16:09 +0000 (15:16 -0400)]
Merge branch '14274-factory-bot' closes #14274

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14199: Abandon processing sooner if client disconnects.
Tom Clegg [Tue, 9 Oct 2018 15:36:52 +0000 (11:36 -0400)]
14199: Abandon processing sooner if client disconnects.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Rename (*remoteResponseCacher)Flush -> Close.
Tom Clegg [Tue, 9 Oct 2018 15:24:44 +0000 (11:24 -0400)]
14199: Rename (*remoteResponseCacher)Flush -> Close.

Finalizing the caching writer is more like Close: it can only happen
once, and only after all Writes are done.  Flush means something
different, especially in http.ResponseWriter context.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Require "X-Keep-Signature: local[, ...]" to invoke signing.
Tom Clegg [Tue, 9 Oct 2018 15:19:16 +0000 (11:19 -0400)]
14199: Require "X-Keep-Signature: local[, ...]" to invoke signing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoarvbox fix "can't write to /dev/stderr" message no issue #
Peter Amstutz [Fri, 5 Oct 2018 18:09:30 +0000 (14:09 -0400)]
arvbox fix "can't write to /dev/stderr" message no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12690-12748-crunchstat-summary'
Peter Amstutz [Fri, 5 Oct 2018 01:36:05 +0000 (21:36 -0400)]
Merge branch '12690-12748-crunchstat-summary'

refs #12690
refs #12748
refs #13759

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Multi-site search works with v2 tokens.
Peter Amstutz [Fri, 5 Oct 2018 01:27:33 +0000 (21:27 -0400)]
14196: Multi-site search works with v2 tokens.

Also update workbench tests use v2 tokens.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14199: Fix deadlock in test suite.
Tom Clegg [Thu, 4 Oct 2018 18:01:00 +0000 (14:01 -0400)]
14199: Fix deadlock in test suite.

If a KeepClient has been created using an API server address that is
no longer reachable, calling keepclient.RefreshServiceDiscovery() puts
the poll() goroutine into an endless retry loop, and a second call
never returns because ent.clear is never ready to receive.

Work around this in the pull worker tests by only refreshing services
from the API server actually being used, not on additional ones
referenced by previous test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Drop Vary response header for non-proxyable requests.
Tom Clegg [Thu, 4 Oct 2018 14:15:22 +0000 (10:15 -0400)]
14199: Drop Vary response header for non-proxyable requests.

The X-Keep-Signature header is ignored for such requests anyway.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Store data if X-Keep-Signature given in proxied GET/HEAD req.
Tom Clegg [Thu, 4 Oct 2018 05:27:19 +0000 (01:27 -0400)]
14199: Store data if X-Keep-Signature given in proxied GET/HEAD req.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12690 & 12748: Read stats from arv-mount.txt
Peter Amstutz [Wed, 3 Oct 2018 20:38:30 +0000 (16:38 -0400)]
12690 & 12748: Read stats from arv-mount.txt

* Compute elapsed time for containers correctly
* Update tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14199: Refactor test case.
Tom Clegg [Wed, 3 Oct 2018 19:54:45 +0000 (15:54 -0400)]
14199: Refactor test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoDocumentation tweaks. Point at Biostars as the forum to discuss Arvados
Ward Vandewege [Wed, 3 Oct 2018 18:01:27 +0000 (14:01 -0400)]
Documentation tweaks. Point at Biostars as the forum to discuss Arvados
questions.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14274: Fix FactoryGirl deprication warning (renamed to FactoryBot)
Peter Amstutz [Tue, 2 Oct 2018 17:56:16 +0000 (13:56 -0400)]
14274: Fix FactoryGirl deprication warning (renamed to FactoryBot)

Also fix "mocha/mini_test" -> "mocha/minitest" warning.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13619-fed-object-list' closes #13619
Peter Amstutz [Wed, 3 Oct 2018 13:23:30 +0000 (09:23 -0400)]
Merge branch '13619-fed-object-list' closes #13619

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Don't crash if element of params[:reader_tokens] is not a String
Peter Amstutz [Tue, 2 Oct 2018 21:14:59 +0000 (17:14 -0400)]
14196: Don't crash if element of params[:reader_tokens] is not a String

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Move configuration options to arvados.Cluster.RequestLimits
Peter Amstutz [Tue, 2 Oct 2018 20:50:07 +0000 (16:50 -0400)]
13619: Move configuration options to arvados.Cluster.RequestLimits

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14201-exclude-home-project' closes #14201
Peter Amstutz [Tue, 2 Oct 2018 18:05:51 +0000 (14:05 -0400)]
Merge branch '14201-exclude-home-project' closes #14201

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Login process returns v2 tokens
Peter Amstutz [Tue, 2 Oct 2018 17:18:33 +0000 (13:18 -0400)]
14196: Login process returns v2 tokens

Unit/functional/integration tests now use v2 tokens by default.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14201: Make exclude_home protected, deduplicate owner uuids
Peter Amstutz [Mon, 1 Oct 2018 20:13:24 +0000 (16:13 -0400)]
14201: Make exclude_home protected, deduplicate owner uuids

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Code cleanups
Peter Amstutz [Mon, 1 Oct 2018 19:48:51 +0000 (15:48 -0400)]
13619: Code cleanups

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '10181-incremental-log'
Tom Clegg [Mon, 1 Oct 2018 19:43:50 +0000 (15:43 -0400)]
Merge branch '10181-incremental-log'

refs #10181

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14201: add exclude_home_project to 'contents' endpoint
Peter Amstutz [Mon, 1 Oct 2018 17:41:44 +0000 (13:41 -0400)]
14201: add exclude_home_project to 'contents' endpoint

Adds tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Merge branch 'master' into 10181-incremental-log
Tom Clegg [Mon, 1 Oct 2018 17:28:57 +0000 (13:28 -0400)]
10181: Merge branch 'master' into 10181-incremental-log

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoa-c-r recognizes that http and https needs to be uploaded to keep
Peter Amstutz [Sat, 29 Sep 2018 15:17:29 +0000 (11:17 -0400)]
a-c-r recognizes that http and https needs to be uploaded to keep

refs #13573

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Restore container's ability to update its own runtime_status.
Tom Clegg [Fri, 28 Sep 2018 20:21:42 +0000 (16:21 -0400)]
10181: Restore container's ability to update its own runtime_status.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13619: Document availability of federation features
Peter Amstutz [Fri, 28 Sep 2018 20:02:19 +0000 (16:02 -0400)]
13619: Document availability of federation features

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Test error reporting when one of the federates fails
Peter Amstutz [Fri, 28 Sep 2018 18:52:40 +0000 (14:52 -0400)]
13619: Test error reporting when one of the federates fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Don't set log collection is_trashed until final save.
Tom Clegg [Fri, 28 Sep 2018 18:40:05 +0000 (14:40 -0400)]
10181: Don't set log collection is_trashed until final save.

Setting is_trashed prevents subsequent updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13619: More tests for paging, error conditions
Peter Amstutz [Fri, 28 Sep 2018 18:32:35 +0000 (14:32 -0400)]
13619: More tests for paging, error conditions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Test that "select" is passed through multi-object query
Peter Amstutz [Thu, 27 Sep 2018 21:23:45 +0000 (17:23 -0400)]
13619: Test that "select" is passed through multi-object query

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Polish error handling
Peter Amstutz [Thu, 27 Sep 2018 21:00:29 +0000 (17:00 -0400)]
13619: Polish error handling

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Fix up redundant error message.
Tom Clegg [Thu, 27 Sep 2018 20:56:41 +0000 (16:56 -0400)]
10181: Fix up redundant error message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13619: Support [uuid, =, ...], cleanups
Peter Amstutz [Thu, 27 Sep 2018 20:44:53 +0000 (16:44 -0400)]
13619: Support [uuid, =, ...], cleanups

Remove call to code that looks for cluster_id from json payload of
POST, the SDKs don't generate this (cluster_id goes in the URL query
string or form encoded).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Clean up, require count=none
Peter Amstutz [Thu, 27 Sep 2018 18:52:22 +0000 (14:52 -0400)]
13619: Clean up, require count=none

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: MultiClusterQuery passes test
Peter Amstutz [Thu, 27 Sep 2018 16:29:35 +0000 (12:29 -0400)]
13619: MultiClusterQuery passes test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13619: Federated multi-object list wip
Peter Amstutz [Thu, 27 Sep 2018 14:53:22 +0000 (10:53 -0400)]
13619: Federated multi-object list wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14236-delete-last-file'
Tom Clegg [Thu, 27 Sep 2018 14:03:40 +0000 (10:03 -0400)]
Merge branch '14236-delete-last-file'

fixes #14236

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Rename configs, add to API server config, support SIGUSR1.
Tom Clegg [Wed, 26 Sep 2018 20:19:16 +0000 (16:19 -0400)]
10181: Rename configs, add to API server config, support SIGUSR1.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoAdd missing copyright headers.
Tom Clegg [Wed, 26 Sep 2018 16:13:06 +0000 (12:13 -0400)]
Add missing copyright headers.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Fix overloaded Size method.
Tom Clegg [Tue, 25 Sep 2018 20:57:55 +0000 (16:57 -0400)]
10181: Fix overloaded Size method.

The method recently added to *dirnode is now TreeSize, rather than
overloading the non-recursive Size method of the inode interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Merge branch 'master' into 10181-incremental-log
Tom Clegg [Tue, 25 Sep 2018 18:51:32 +0000 (14:51 -0400)]
10181: Merge branch 'master' into 10181-incremental-log

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Keep CR logs synchronized with container logs while running.
Tom Clegg [Tue, 25 Sep 2018 18:50:44 +0000 (14:50 -0400)]
10181: Keep CR logs synchronized with container logs while running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14197-federated-containers' refs #14197
Peter Amstutz [Tue, 25 Sep 2018 15:22:05 +0000 (11:22 -0400)]
Merge branch '14197-federated-containers' refs #14197

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14197: Another test checking uuid from federated container request.
Peter Amstutz [Mon, 24 Sep 2018 20:56:59 +0000 (16:56 -0400)]
14197: Another test checking uuid from federated container request.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14197: Federated container request test tweaks
Peter Amstutz [Mon, 24 Sep 2018 20:53:40 +0000 (16:53 -0400)]
14197: Federated container request test tweaks

* Check uuid on created container request.

* Add test for bogus cluster id

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Fix race in MarshalManifest.
Tom Clegg [Sat, 22 Sep 2018 06:07:22 +0000 (02:07 -0400)]
10181: Fix race in MarshalManifest.

Ensure child nodes stay locked between sync and marshal. Otherwise
concurrent file writes can add new memSegments that can't be
marshaled.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Fix timing sensitivity in test case.
Tom Clegg [Fri, 21 Sep 2018 20:48:01 +0000 (16:48 -0400)]
10181: Fix timing sensitivity in test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Load checkpoint configs from discovery doc if available.
Tom Clegg [Fri, 21 Sep 2018 20:34:51 +0000 (16:34 -0400)]
10181: Load checkpoint configs from discovery doc if available.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Cleanup identifiers.
Tom Clegg [Fri, 21 Sep 2018 20:23:58 +0000 (16:23 -0400)]
10181: Cleanup identifiers.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Update container log PDH after updating log collection.
Tom Clegg [Fri, 21 Sep 2018 19:30:05 +0000 (15:30 -0400)]
10181: Update container log PDH after updating log collection.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14197: Add cluster_id to discovery document for create method.
Peter Amstutz [Fri, 21 Sep 2018 19:23:18 +0000 (15:23 -0400)]
14197: Add cluster_id to discovery document for create method.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Permit dispatcher to update log while container is running.
Tom Clegg [Thu, 20 Sep 2018 17:52:47 +0000 (13:52 -0400)]
10181: Permit dispatcher to update log while container is running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14197: Supports routing object create requests, with tests
Peter Amstutz [Fri, 21 Sep 2018 18:25:34 +0000 (14:25 -0400)]
14197: Supports routing object create requests, with tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14236: Fix regexp in test.
Tom Clegg [Fri, 21 Sep 2018 18:07:34 +0000 (14:07 -0400)]
14236: Fix regexp in test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '10865-legacy-jobs-api' closes #10865
Peter Amstutz [Thu, 20 Sep 2018 18:57:30 +0000 (14:57 -0400)]
Merge branch '10865-legacy-jobs-api' closes #10865

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14197: Generalizing federated routing
Peter Amstutz [Thu, 20 Sep 2018 18:55:31 +0000 (14:55 -0400)]
14197: Generalizing federated routing

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13493-document-federation' refs #13493
Peter Amstutz [Thu, 20 Sep 2018 18:13:28 +0000 (14:13 -0400)]
Merge branch '13493-document-federation' refs #13493

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14242-verify-manifest-pdh' closes #14242
Peter Amstutz [Thu, 20 Sep 2018 18:12:33 +0000 (14:12 -0400)]
Merge branch '14242-verify-manifest-pdh' closes #14242

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10865: Update documentation, replace disable_api_methods with enable_legacy_jobs_api
Peter Amstutz [Thu, 20 Sep 2018 17:55:19 +0000 (13:55 -0400)]
10865: Update documentation, replace disable_api_methods with enable_legacy_jobs_api

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoSanitize how we get the rvm GPG key, and remove a dependency on the GPG
Ward Vandewege [Thu, 20 Sep 2018 17:53:51 +0000 (13:53 -0400)]
Sanitize how we get the rvm GPG key, and remove a dependency on the GPG
keyservers which are flakey.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago10865: Add test for check_enable_legacy_jobs_api
Peter Amstutz [Thu, 20 Sep 2018 16:57:40 +0000 (12:57 -0400)]
10865: Add test for check_enable_legacy_jobs_api

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14242: Add a test for manifest_text PDH checking
Peter Amstutz [Thu, 20 Sep 2018 14:46:11 +0000 (10:46 -0400)]
14242: Add a test for manifest_text PDH checking

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10181: Save log collection snapshots periodically during run.
Tom Clegg [Thu, 20 Sep 2018 05:04:52 +0000 (01:04 -0400)]
10181: Save log collection snapshots periodically during run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Add Size method to arvados.CollectionFileSystem.
Tom Clegg [Thu, 20 Sep 2018 04:59:35 +0000 (00:59 -0400)]
10181: Add Size method to arvados.CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14236: Don't omit empty manifest_text in update requests.
Tom Clegg [Wed, 19 Sep 2018 20:43:47 +0000 (16:43 -0400)]
14236: Don't omit empty manifest_text in update requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14236: Test deleting the last file in a collection.
Tom Clegg [Wed, 19 Sep 2018 20:22:58 +0000 (16:22 -0400)]
14236: Test deleting the last file in a collection.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14197: Verify manifest text matches portable data hash that was requested
Peter Amstutz [Wed, 19 Sep 2018 20:15:47 +0000 (16:15 -0400)]
14197: Verify manifest text matches portable data hash that was requested

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix package building - support modern versions of pip.
Ward Vandewege [Wed, 19 Sep 2018 20:06:50 +0000 (16:06 -0400)]
Fix package building - support modern versions of pip.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago10865: Fix query for empty jobs table
Peter Amstutz [Wed, 19 Sep 2018 19:15:11 +0000 (15:15 -0400)]
10865: Fix query for empty jobs table

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago10865: Add enable_legacy_jobs_api configuration option
Peter Amstutz [Wed, 19 Sep 2018 18:52:54 +0000 (14:52 -0400)]
10865: Add enable_legacy_jobs_api configuration option

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13493: Notes about cluster id, work in progress
Peter Amstutz [Wed, 19 Sep 2018 13:35:08 +0000 (09:35 -0400)]
13493: Notes about cluster id, work in progress

Also add arvados federation diagram, copyright notice

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13493: Federation documentation covers record retrieval, keep
Peter Amstutz [Tue, 18 Sep 2018 20:26:59 +0000 (16:26 -0400)]
13493: Federation documentation covers record retrieval, keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13752-migrate-index-data'
Tom Clegg [Tue, 18 Sep 2018 19:47:43 +0000 (15:47 -0400)]
Merge branch '13752-migrate-index-data'

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: Merge branch 'master' into 13752-migrate-index-data
Tom Clegg [Tue, 18 Sep 2018 19:43:28 +0000 (15:43 -0400)]
13752: Merge branch 'master' into 13752-migrate-index-data

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13994-proxy-remote'
Tom Clegg [Tue, 18 Sep 2018 19:42:06 +0000 (15:42 -0400)]
Merge branch '13994-proxy-remote'

refs #13994

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Merge branch 'master' into 13994-proxy-remote
Tom Clegg [Tue, 18 Sep 2018 19:40:26 +0000 (15:40 -0400)]
13994: Merge branch 'master' into 13994-proxy-remote

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: Migrate file_names column using multiple transactions.
Tom Clegg [Mon, 17 Sep 2018 21:50:13 +0000 (17:50 -0400)]
13752: Migrate file_names column using multiple transactions.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: De-duplicate file and stream names in collections.file_names.
Tom Clegg [Mon, 17 Sep 2018 21:49:21 +0000 (17:49 -0400)]
13752: De-duplicate file and stream names in collections.file_names.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Fix missing error message.
Tom Clegg [Mon, 17 Sep 2018 18:34:05 +0000 (14:34 -0400)]
13994: Fix missing error message.

When the request failed too early to produce an HTTP status code, the
error returned to the client was:

"Could not write sufficient replicas: [0] ; [0] "

Now the error message is included. It gets truncated to 100 chars
later on, but it's something:

"Could not write sufficient replicas: [0] Put
http://localhost:52367/831f95ffae4233b52059fbb8a0c62fbf: dial tcp [::1
]:52367: connect: conn; [0] Put
http://localhost:53807/831f95ffae4233b52059fbb8a0c62fbf: dial tcp
[::1]:53807: connect: conn"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Fix service discovery race.
Tom Clegg [Mon, 17 Sep 2018 18:33:59 +0000 (14:33 -0400)]
13994: Fix service discovery race.

Making keepclient.RefreshServiceDiscovery asynchronous didn't work
out: tests rely on it to ensure old services won't be used.

Instead, speed things up a bit by refreshing all known clients'
service lists concurrently instead of serially.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14087-federated-collection-by-pdh' refs #14087
Peter Amstutz [Mon, 17 Sep 2018 15:58:37 +0000 (11:58 -0400)]
Merge branch '14087-federated-collection-by-pdh' refs #14087

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14203-crunch-run-mkdir' closes #14203
Peter Amstutz [Mon, 17 Sep 2018 15:58:22 +0000 (11:58 -0400)]
Merge branch '14203-crunch-run-mkdir' closes #14203

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Add Accept-Encoding to list of headers to delete when proxying
Peter Amstutz [Mon, 17 Sep 2018 15:54:35 +0000 (11:54 -0400)]
14087: Add Accept-Encoding to list of headers to delete when proxying

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13493: Federation docs wip
Peter Amstutz [Mon, 17 Sep 2018 14:56:05 +0000 (10:56 -0400)]
13493: Federation docs wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13994: Fix token handling in collection performance test.
Tom Clegg [Mon, 17 Sep 2018 14:34:43 +0000 (10:34 -0400)]
13994: Fix token handling in collection performance test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Use entire token for blob signatures.
Tom Clegg [Mon, 17 Sep 2018 14:07:35 +0000 (10:07 -0400)]
13994: Use entire token for blob signatures.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14203: Tweak error messages.
Peter Amstutz [Fri, 14 Sep 2018 15:39:26 +0000 (11:39 -0400)]
14203: Tweak error messages.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Refactor federation search for collections by PDH
Peter Amstutz [Fri, 14 Sep 2018 15:27:11 +0000 (11:27 -0400)]
14087: Refactor federation search for collections by PDH

The shorter, non-PDH search case now comes first.  Also adjusts the
logic that decides when to skip federation and makes it wrap the
federation endpoint routing.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13773-will-fail-container-status'
Lucas Di Pentima [Thu, 13 Sep 2018 20:46:02 +0000 (17:46 -0300)]
Merge branch '13773-will-fail-container-status'
Closes #13773

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'master' into 13773-will-fail-container-status
Lucas Di Pentima [Thu, 13 Sep 2018 20:05:48 +0000 (17:05 -0300)]
Merge branch 'master' into 13773-will-fail-container-status

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Update logging level checking for runtime status updates.
Lucas Di Pentima [Thu, 13 Sep 2018 19:30:35 +0000 (16:30 -0300)]
13773: Update logging level checking for runtime status updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13994: Fix import cycle.
Tom Clegg [Thu, 13 Sep 2018 18:43:28 +0000 (14:43 -0400)]
13994: Fix import cycle.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13773: Fix test
Lucas Di Pentima [Wed, 12 Sep 2018 20:58:00 +0000 (17:58 -0300)]
13773: Fix test

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Update test that confirms runtime_status is updated when a child fails
Lucas Di Pentima [Wed, 12 Sep 2018 20:14:41 +0000 (17:14 -0300)]
13773: Update test that confirms runtime_status is updated when a child fails

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13994: Fix deadlock in keepstore tests.
Tom Clegg [Tue, 11 Sep 2018 16:55:05 +0000 (12:55 -0400)]
13994: Fix deadlock in keepstore tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>