Peter Amstutz [Fri, 5 Oct 2018 01:36:05 +0000 (21:36 -0400)]
Merge branch '12690-12748-crunchstat-summary'
refs #12690
refs #12748
refs #13759
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 3 Oct 2018 20:38:30 +0000 (16:38 -0400)]
12690 & 12748: Read stats from arv-mount.txt
* Compute elapsed time for containers correctly
* Update tests
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Ward Vandewege [Wed, 3 Oct 2018 18:01:27 +0000 (14:01 -0400)]
Documentation tweaks. Point at Biostars as the forum to discuss Arvados
questions.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Peter Amstutz [Wed, 3 Oct 2018 13:23:30 +0000 (09:23 -0400)]
Merge branch '13619-fed-object-list' closes #13619
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 2 Oct 2018 20:50:07 +0000 (16:50 -0400)]
13619: Move configuration options to arvados.Cluster.RequestLimits
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 2 Oct 2018 18:05:51 +0000 (14:05 -0400)]
Merge branch '14201-exclude-home-project' closes #14201
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 1 Oct 2018 20:13:24 +0000 (16:13 -0400)]
14201: Make exclude_home protected, deduplicate owner uuids
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 1 Oct 2018 19:48:51 +0000 (15:48 -0400)]
13619: Code cleanups
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 1 Oct 2018 19:43:50 +0000 (15:43 -0400)]
Merge branch '10181-incremental-log'
refs #10181
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 1 Oct 2018 17:41:44 +0000 (13:41 -0400)]
14201: add exclude_home_project to 'contents' endpoint
Adds tests and documentation.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 1 Oct 2018 17:28:57 +0000 (13:28 -0400)]
10181: Merge branch 'master' into 10181-incremental-log
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Sat, 29 Sep 2018 15:17:29 +0000 (11:17 -0400)]
a-c-r recognizes that http and https needs to be uploaded to keep
refs #13573
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 28 Sep 2018 20:21:42 +0000 (16:21 -0400)]
10181: Restore container's ability to update its own runtime_status.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 28 Sep 2018 20:02:19 +0000 (16:02 -0400)]
13619: Document availability of federation features
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 28 Sep 2018 18:52:40 +0000 (14:52 -0400)]
13619: Test error reporting when one of the federates fails
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 28 Sep 2018 18:40:05 +0000 (14:40 -0400)]
10181: Don't set log collection is_trashed until final save.
Setting is_trashed prevents subsequent updates.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 28 Sep 2018 18:32:35 +0000 (14:32 -0400)]
13619: More tests for paging, error conditions
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 21:23:45 +0000 (17:23 -0400)]
13619: Test that "select" is passed through multi-object query
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 21:00:29 +0000 (17:00 -0400)]
13619: Polish error handling
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 27 Sep 2018 20:56:41 +0000 (16:56 -0400)]
10181: Fix up redundant error message.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 20:44:53 +0000 (16:44 -0400)]
13619: Support [uuid, =, ...], cleanups
Remove call to code that looks for cluster_id from json payload of
POST, the SDKs don't generate this (cluster_id goes in the URL query
string or form encoded).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 18:52:22 +0000 (14:52 -0400)]
13619: Clean up, require count=none
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 16:29:35 +0000 (12:29 -0400)]
13619: MultiClusterQuery passes test
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 27 Sep 2018 14:53:22 +0000 (10:53 -0400)]
13619: Federated multi-object list wip
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 27 Sep 2018 14:03:40 +0000 (10:03 -0400)]
Merge branch '14236-delete-last-file'
fixes #14236
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 26 Sep 2018 20:19:16 +0000 (16:19 -0400)]
10181: Rename configs, add to API server config, support SIGUSR1.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 26 Sep 2018 16:13:06 +0000 (12:13 -0400)]
Add missing copyright headers.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 25 Sep 2018 20:57:55 +0000 (16:57 -0400)]
10181: Fix overloaded Size method.
The method recently added to *dirnode is now TreeSize, rather than
overloading the non-recursive Size method of the inode interface.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 25 Sep 2018 18:51:32 +0000 (14:51 -0400)]
10181: Merge branch 'master' into 10181-incremental-log
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 25 Sep 2018 18:50:44 +0000 (14:50 -0400)]
10181: Keep CR logs synchronized with container logs while running.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 25 Sep 2018 15:22:05 +0000 (11:22 -0400)]
Merge branch '14197-federated-containers' refs #14197
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 24 Sep 2018 20:56:59 +0000 (16:56 -0400)]
14197: Another test checking uuid from federated container request.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 24 Sep 2018 20:53:40 +0000 (16:53 -0400)]
14197: Federated container request test tweaks
* Check uuid on created container request.
* Add test for bogus cluster id
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Sat, 22 Sep 2018 06:07:22 +0000 (02:07 -0400)]
10181: Fix race in MarshalManifest.
Ensure child nodes stay locked between sync and marshal. Otherwise
concurrent file writes can add new memSegments that can't be
marshaled.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 21 Sep 2018 20:48:01 +0000 (16:48 -0400)]
10181: Fix timing sensitivity in test case.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 21 Sep 2018 20:34:51 +0000 (16:34 -0400)]
10181: Load checkpoint configs from discovery doc if available.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 21 Sep 2018 20:23:58 +0000 (16:23 -0400)]
10181: Cleanup identifiers.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 21 Sep 2018 19:30:05 +0000 (15:30 -0400)]
10181: Update container log PDH after updating log collection.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 21 Sep 2018 19:23:18 +0000 (15:23 -0400)]
14197: Add cluster_id to discovery document for create method.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 20 Sep 2018 17:52:47 +0000 (13:52 -0400)]
10181: Permit dispatcher to update log while container is running.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 21 Sep 2018 18:25:34 +0000 (14:25 -0400)]
14197: Supports routing object create requests, with tests
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 21 Sep 2018 18:07:34 +0000 (14:07 -0400)]
14236: Fix regexp in test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 18:57:30 +0000 (14:57 -0400)]
Merge branch '10865-legacy-jobs-api' closes #10865
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 18:55:31 +0000 (14:55 -0400)]
14197: Generalizing federated routing
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 18:13:28 +0000 (14:13 -0400)]
Merge branch '13493-document-federation' refs #13493
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 18:12:33 +0000 (14:12 -0400)]
Merge branch '14242-verify-manifest-pdh' closes #14242
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 17:55:19 +0000 (13:55 -0400)]
10865: Update documentation, replace disable_api_methods with enable_legacy_jobs_api
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Ward Vandewege [Thu, 20 Sep 2018 17:53:51 +0000 (13:53 -0400)]
Sanitize how we get the rvm GPG key, and remove a dependency on the GPG
keyservers which are flakey.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 16:57:40 +0000 (12:57 -0400)]
10865: Add test for check_enable_legacy_jobs_api
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 20 Sep 2018 14:46:11 +0000 (10:46 -0400)]
14242: Add a test for manifest_text PDH checking
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 20 Sep 2018 05:04:52 +0000 (01:04 -0400)]
10181: Save log collection snapshots periodically during run.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 20 Sep 2018 04:59:35 +0000 (00:59 -0400)]
10181: Add Size method to arvados.CollectionFileSystem.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 19 Sep 2018 20:43:47 +0000 (16:43 -0400)]
14236: Don't omit empty manifest_text in update requests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 19 Sep 2018 20:22:58 +0000 (16:22 -0400)]
14236: Test deleting the last file in a collection.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Wed, 19 Sep 2018 20:15:47 +0000 (16:15 -0400)]
14197: Verify manifest text matches portable data hash that was requested
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Ward Vandewege [Wed, 19 Sep 2018 20:06:50 +0000 (16:06 -0400)]
Fix package building - support modern versions of pip.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Peter Amstutz [Wed, 19 Sep 2018 19:15:11 +0000 (15:15 -0400)]
10865: Fix query for empty jobs table
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 19 Sep 2018 18:52:54 +0000 (14:52 -0400)]
10865: Add enable_legacy_jobs_api configuration option
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 19 Sep 2018 13:35:08 +0000 (09:35 -0400)]
13493: Notes about cluster id, work in progress
Also add arvados federation diagram, copyright notice
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 18 Sep 2018 20:26:59 +0000 (16:26 -0400)]
13493: Federation documentation covers record retrieval, keep
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 18 Sep 2018 19:47:43 +0000 (15:47 -0400)]
Merge branch '13752-migrate-index-data'
refs #13752
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 18 Sep 2018 19:43:28 +0000 (15:43 -0400)]
13752: Merge branch 'master' into 13752-migrate-index-data
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 18 Sep 2018 19:42:06 +0000 (15:42 -0400)]
Merge branch '13994-proxy-remote'
refs #13994
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 18 Sep 2018 19:40:26 +0000 (15:40 -0400)]
13994: Merge branch 'master' into 13994-proxy-remote
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 21:50:13 +0000 (17:50 -0400)]
13752: Migrate file_names column using multiple transactions.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 21:49:21 +0000 (17:49 -0400)]
13752: De-duplicate file and stream names in collections.file_names.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 18:34:05 +0000 (14:34 -0400)]
13994: Fix missing error message.
When the request failed too early to produce an HTTP status code, the
error returned to the client was:
"Could not write sufficient replicas: [0] ; [0] "
Now the error message is included. It gets truncated to 100 chars
later on, but it's something:
"Could not write sufficient replicas: [0] Put
http://localhost:52367/
831f95ffae4233b52059fbb8a0c62fbf: dial tcp [::1
]:52367: connect: conn; [0] Put
http://localhost:53807/
831f95ffae4233b52059fbb8a0c62fbf: dial tcp
[::1]:53807: connect: conn"
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 18:33:59 +0000 (14:33 -0400)]
13994: Fix service discovery race.
Making keepclient.RefreshServiceDiscovery asynchronous didn't work
out: tests rely on it to ensure old services won't be used.
Instead, speed things up a bit by refreshing all known clients'
service lists concurrently instead of serially.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 17 Sep 2018 15:58:37 +0000 (11:58 -0400)]
Merge branch '14087-federated-collection-by-pdh' refs #14087
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 17 Sep 2018 15:58:22 +0000 (11:58 -0400)]
Merge branch '14203-crunch-run-mkdir' closes #14203
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 17 Sep 2018 15:54:35 +0000 (11:54 -0400)]
14087: Add Accept-Encoding to list of headers to delete when proxying
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 17 Sep 2018 14:56:05 +0000 (10:56 -0400)]
13493: Federation docs wip
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 14:34:43 +0000 (10:34 -0400)]
13994: Fix token handling in collection performance test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 17 Sep 2018 14:07:35 +0000 (10:07 -0400)]
13994: Use entire token for blob signatures.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 14 Sep 2018 15:39:26 +0000 (11:39 -0400)]
14203: Tweak error messages.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 14 Sep 2018 15:27:11 +0000 (11:27 -0400)]
14087: Refactor federation search for collections by PDH
The shorter, non-PDH search case now comes first. Also adjusts the
logic that decides when to skip federation and makes it wrap the
federation endpoint routing.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Thu, 13 Sep 2018 20:46:02 +0000 (17:46 -0300)]
Merge branch '13773-will-fail-container-status'
Closes #13773
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 13 Sep 2018 20:05:48 +0000 (17:05 -0300)]
Merge branch 'master' into 13773-will-fail-container-status
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 13 Sep 2018 19:30:35 +0000 (16:30 -0300)]
13773: Update logging level checking for runtime status updates.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 13 Sep 2018 18:43:28 +0000 (14:43 -0400)]
13994: Fix import cycle.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Wed, 12 Sep 2018 20:58:00 +0000 (17:58 -0300)]
13773: Fix test
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 12 Sep 2018 20:14:41 +0000 (17:14 -0300)]
13773: Update test that confirms runtime_status is updated when a child fails
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Tue, 11 Sep 2018 16:55:05 +0000 (12:55 -0400)]
13994: Fix deadlock in keepstore tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 11 Sep 2018 16:54:41 +0000 (12:54 -0400)]
13994: Proxy to remote cluster if +R hint given.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Wed, 12 Sep 2018 19:37:33 +0000 (15:37 -0400)]
14203: Mkdir failing with os.ErrExist is not a fatal error
Don't fail CaptureOutput and cancel the container if a Mkdir returns
os.ErrExist because a directory already exists. Fixes bcbio failing
postprocess_variants step.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Wed, 12 Sep 2018 18:41:42 +0000 (15:41 -0300)]
13773: Removed superfluous documentation
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 12 Sep 2018 12:13:35 +0000 (09:13 -0300)]
13773: Add well known runtime_status' keys data type validation.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Tue, 11 Sep 2018 15:40:10 +0000 (11:40 -0400)]
Merge branch '13993-federated-collection' refs #13993
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 11 Sep 2018 13:52:27 +0000 (09:52 -0400)]
14087: Rate limit the number of parallel requests
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 21:03:19 +0000 (17:03 -0400)]
14087: Remove debug print. Comments.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 20:49:18 +0000 (16:49 -0400)]
14087: Check for remote clusters before searching
Added some more comments about the implementation as well.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 19:28:35 +0000 (15:28 -0400)]
14087: Fetch federated collection by PDH
Implement parallel query of remotes. First response is returned to
client, other responses in flight are cancelled. Adds tests.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 7 Sep 2018 22:14:26 +0000 (18:14 -0400)]
14087: Federated fetch by PDH WIP
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 11 Sep 2018 15:05:34 +0000 (11:05 -0400)]
13993: Add TestGetLocalCollection
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Tue, 11 Sep 2018 14:44:48 +0000 (11:44 -0300)]
13773: Enhance error/warning runtime status display on workbench.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 11 Sep 2018 14:42:37 +0000 (11:42 -0300)]
13773: Don't intercept logger.info() calls for runtime status.
* On multi line log messages, use the first as a status and the rest as detail.
* Revert done.logtail() usage as it's no longer needed for failed childs
reporting.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 21:04:37 +0000 (17:04 -0400)]
13993: Need to explicitly handle paths that don't end in /
Otherwise Go will generate a 301 Moved Permanently from collection to collections/
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 20:20:32 +0000 (16:20 -0400)]
13993: Don't export internal types
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 10 Sep 2018 20:18:01 +0000 (16:18 -0400)]
13993: Consolidate SignedLocatorPattern with keepclient.SignedLocatorRe
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Mon, 10 Sep 2018 17:48:38 +0000 (14:48 -0300)]
13773: Updates documentation
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
projects / arvados.git / log