Tom Clegg [Sat, 17 Oct 2015 08:23:27 +0000 (04:23 -0400)]
5824: Simplify keepproxy test setup.
For testing, configure KeepClient to use the test proxy server,
instead of telling the API about the test proxy server and using env
vars to trick the client into discovering it.
Let the OS choose an available port for each test server instead of
pre-assigning an arbitrary port number to each test case.
Use a properly formatted fake UUID.
Tom Clegg [Sat, 17 Oct 2015 06:56:08 +0000 (02:56 -0400)]
5824: Refresh keepproxy services list on SIGHUP. Update Workbench upload test to expect success.
Tom Clegg [Wed, 14 Oct 2015 08:07:37 +0000 (04:07 -0400)]
5824: Update bundle
Tom Clegg [Tue, 13 Oct 2015 14:52:06 +0000 (10:52 -0400)]
5824: Use keep-web in Workbench integration tests
Tom Clegg [Mon, 12 Oct 2015 23:15:06 +0000 (19:15 -0400)]
5824: Add option to redirect Workbench downloads to a keep-web service
Tom Clegg [Fri, 2 Oct 2015 02:16:51 +0000 (22:16 -0400)]
5824: gofmt
Tom Clegg [Mon, 7 Sep 2015 07:43:59 +0000 (03:43 -0400)]
5824: Add -attachment-only-host feature.
Tom Clegg [Mon, 7 Sep 2015 06:39:10 +0000 (02:39 -0400)]
5824: Implement "trust all content" mode.
Tom Clegg [Sun, 30 Aug 2015 06:45:58 +0000 (02:45 -0400)]
5824: Add read-error and lots-of-blocks tests.
Tom Clegg [Fri, 28 Aug 2015 15:08:16 +0000 (11:08 -0400)]
5824: Fix up DNS docs.
Tom Clegg [Fri, 28 Aug 2015 06:30:45 +0000 (02:30 -0400)]
5824: Log X-Forwarded-For header value if provided.
Tom Clegg [Fri, 28 Aug 2015 05:45:35 +0000 (01:45 -0400)]
5824: Fail at startup if ARVADOS_API_HOST is not set.
Tom Clegg [Fri, 28 Aug 2015 05:28:43 +0000 (01:28 -0400)]
5824: Accept anonymous tokens on command line.
Tom Clegg [Fri, 28 Aug 2015 05:09:46 +0000 (01:09 -0400)]
5824: Handle various combinations of c= and t= more consistently. Use vhosts in integration tests.
Tom Clegg [Fri, 28 Aug 2015 02:34:21 +0000 (22:34 -0400)]
5824: Comment to explain "authorizer" test helpers.
Tom Clegg [Fri, 28 Aug 2015 02:33:44 +0000 (22:33 -0400)]
5824: Fix up support for PDH in vhostname.
Tom Clegg [Fri, 28 Aug 2015 02:31:19 +0000 (22:31 -0400)]
5824: Modernize install page, cf. other services.
Tom Clegg [Fri, 28 Aug 2015 02:17:54 +0000 (22:17 -0400)]
5824: Clarify difference between keepproxy and keepstore (bandwidth and convenience -- not security).
Tom Clegg [Fri, 28 Aug 2015 02:16:26 +0000 (22:16 -0400)]
5824: Update keepproxy usage.
Tom Clegg [Sun, 16 Aug 2015 04:16:27 +0000 (00:16 -0400)]
5824: Fix up error checking and early-close behavior inCollectionFileReader.
Tom Clegg [Wed, 29 Jul 2015 07:33:19 +0000 (03:33 -0400)]
5824: Add Content-Length header.
Tom Clegg [Wed, 29 Jul 2015 05:56:55 +0000 (01:56 -0400)]
5824: Use vhosts in curl integration tests. Add large file test.
Tom Clegg [Thu, 23 Jul 2015 05:20:28 +0000 (01:20 -0400)]
5824: Support vhost-based collection lookups.
Tom Clegg [Thu, 25 Jun 2015 03:33:08 +0000 (23:33 -0400)]
5824: add (*KeepClient)CollectionFileReader()
Tom Clegg [Thu, 23 Jul 2015 05:09:02 +0000 (01:09 -0400)]
5824: Add doc.go
Tom Clegg [Tue, 23 Jun 2015 23:12:58 +0000 (19:12 -0400)]
5824: Add install doc
Tom Clegg [Wed, 17 Jun 2015 06:47:49 +0000 (02:47 -0400)]
5824: Assign MIME type by file extension. closes #6327
Tom Clegg [Thu, 23 Jul 2015 04:02:11 +0000 (00:02 -0400)]
5824: Add keepdl.
Tom Clegg [Thu, 15 Oct 2015 20:27:58 +0000 (16:27 -0400)]
Merge branch '7159-keepexercise' refs #7159
Tom Clegg [Thu, 15 Oct 2015 20:25:14 +0000 (16:25 -0400)]
7159: Add keep-exercise
Tom Clegg [Thu, 15 Oct 2015 19:47:11 +0000 (15:47 -0400)]
Merge branch 'wtsi-hgi-feature/arv-view'
refs #7204
Tom Clegg [Thu, 15 Oct 2015 19:42:37 +0000 (15:42 -0400)]
7204: Tweak usage message, add a .shellescape, fix error-reporting error
radhika [Thu, 15 Oct 2015 16:35:32 +0000 (12:35 -0400)]
closes #7167
Merge branch '7167-keep-rsync'
radhika [Thu, 15 Oct 2015 16:35:15 +0000 (12:35 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Thu, 15 Oct 2015 16:32:07 +0000 (12:32 -0400)]
Merge branch 'master' into 7167-keep-rsync
Conflicts:
sdk/go/keepclient/keepclient.go
Tom Clegg [Thu, 15 Oct 2015 16:21:07 +0000 (12:21 -0400)]
Merge branch '7173-jessie' refs #7173
Tom Clegg [Mon, 31 Aug 2015 19:00:13 +0000 (15:00 -0400)]
7173: Add jessie to supported distro list.
radhika [Thu, 15 Oct 2015 16:18:20 +0000 (12:18 -0400)]
7167: update the progress message to say "est. time remaining" instead of "ETA" which is not accurate in this case.
radhika [Thu, 15 Oct 2015 15:56:36 +0000 (11:56 -0400)]
7167: rename the newly added StopKeepServers as StopKeepWithParams; it now sounds similar to StartKeepWithParams and is self explanatory.
radhika [Thu, 15 Oct 2015 15:08:10 +0000 (11:08 -0400)]
Merge branch '7167-keep-rsync' of git.curoverse.com:arvados into 7167-keep-rsync
Tom Clegg [Thu, 15 Oct 2015 15:02:30 +0000 (11:02 -0400)]
7167: Always pass --num-keep-servers to python wrapper. Update comments.
Tom Clegg [Thu, 15 Oct 2015 14:50:13 +0000 (10:50 -0400)]
7167: Rename conf flags to -src and -dst to match arv-copy. Always add .conf for short form.
Tom Clegg [Thu, 15 Oct 2015 14:48:33 +0000 (10:48 -0400)]
7167: Fix up progress/ETA logging.
Peter Amstutz [Thu, 15 Oct 2015 14:32:26 +0000 (10:32 -0400)]
Merge branch '7491-go-keepclient-retry' closes #7491
Peter Amstutz [Thu, 15 Oct 2015 14:16:27 +0000 (10:16 -0400)]
7491: Remove unnecessary writable service roots from GET tests.
Tom Clegg [Thu, 15 Oct 2015 13:18:43 +0000 (09:18 -0400)]
Merge branch 'master' into wtsi-hgi-feature/arv-view
Conflicts:
sdk/cli/bin/arv
radhika [Thu, 15 Oct 2015 03:01:13 +0000 (23:01 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Thu, 15 Oct 2015 03:00:05 +0000 (23:00 -0400)]
7167: log progress during keep-rsync and several test improvements.
Brett Smith [Wed, 14 Oct 2015 20:59:40 +0000 (16:59 -0400)]
API server needs an arvados-cli with crunch-job --docker-bin.
crunch-dispatch was extended to use `crunch-job --docker-bin` in
#6838. This commit simply updates the Gemfile to ensure this
dependency is satisfied. No issue #.
radhika [Wed, 14 Oct 2015 19:15:15 +0000 (15:15 -0400)]
Merge branch 'master' into 7167-keep-rsync
Tom Clegg [Wed, 14 Oct 2015 18:51:34 +0000 (14:51 -0400)]
Merge branch '7167-propagate-error' refs #7167
Tom Clegg [Wed, 14 Oct 2015 18:51:01 +0000 (14:51 -0400)]
Merge branch '7159-clean-index' refs #7159 refs #7168
radhika [Wed, 14 Oct 2015 17:43:54 +0000 (13:43 -0400)]
7167: loadConfig setupKeepclient do only one set at a time.
radhika [Wed, 14 Oct 2015 02:27:41 +0000 (22:27 -0400)]
7167: when the config file does not contain '/', use $HOME/.config/arvados/<filename>.
radhika [Wed, 14 Oct 2015 01:47:06 +0000 (21:47 -0400)]
7167: replace keep_existing with num_keep_servers and use it create all required keep servers at once.
radhika [Wed, 14 Oct 2015 01:16:35 +0000 (21:16 -0400)]
7167: replace the keep_existing logic and create all 3 keep servers at once and use the first two as src keepservers and the last one as the dst keep server.
radhika [Tue, 13 Oct 2015 21:01:46 +0000 (17:01 -0400)]
7167: Convert blobSigningKey also into local variable and make necessary changes to accommodate this change.
Remove the New method added in arvadosclient.go and revert MakeArvadosClient to what it was before.
radhika [Tue, 13 Oct 2015 19:46:26 +0000 (15:46 -0400)]
7167: update run_test_servers.py to use action="store_true" instead of converting string to boolean.
Tom Clegg [Tue, 13 Oct 2015 19:33:33 +0000 (15:33 -0400)]
7159: Address golint complaints
Tom Clegg [Tue, 13 Oct 2015 19:33:02 +0000 (15:33 -0400)]
7159: Omit non-Keep blobs from index
radhika [Tue, 13 Oct 2015 17:38:52 +0000 (13:38 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Tue, 13 Oct 2015 17:37:33 +0000 (13:37 -0400)]
7167: Convert most of the globals in keep-sync into locals and update all the code and tests as needed.
Tom Clegg [Tue, 13 Oct 2015 16:11:23 +0000 (12:11 -0400)]
7159: Return benign os.ErrNotExist error from Compare to avoid excessive logs. refs #7159
Tom Clegg [Tue, 13 Oct 2015 15:17:49 +0000 (11:17 -0400)]
7159: Fix error handling when reading full size block. refs #7159
Tom Clegg [Mon, 12 Oct 2015 17:41:04 +0000 (13:41 -0400)]
7167: Propagate read errors to caller. Fixes failing TestTransferShortBuffer.
Tom Clegg [Mon, 12 Oct 2015 16:40:38 +0000 (12:40 -0400)]
Warn about unhandled case if broken node has no ping time. refs #7286
Tom Clegg [Mon, 12 Oct 2015 16:06:47 +0000 (12:06 -0400)]
Merge branch '7159-empty-blob-race' refs #7159
Tom Clegg [Mon, 12 Oct 2015 13:33:24 +0000 (09:33 -0400)]
7159: Shorten race waits during generic tests
Tom Clegg [Fri, 9 Oct 2015 21:09:42 +0000 (17:09 -0400)]
7159: Log when waiting for get/put races
Tom Clegg [Thu, 8 Oct 2015 18:00:00 +0000 (14:00 -0400)]
7159: Exclude new empty blocks from index.
Tom Clegg [Thu, 8 Oct 2015 17:30:18 +0000 (13:30 -0400)]
7159: Test race deadline
Tom Clegg [Thu, 8 Oct 2015 16:52:17 +0000 (12:52 -0400)]
7159: Work around CreateBlob race by polling for updates when a brand new blob is found empty.
radhika [Mon, 12 Oct 2015 14:40:38 +0000 (10:40 -0400)]
7167: break load config logic out of main into loadConfig func and add several tests.
radhika [Mon, 12 Oct 2015 13:13:45 +0000 (09:13 -0400)]
7167: some more error tests such as error getting block from src and error putting block to dst.
radhika [Sat, 10 Oct 2015 00:09:17 +0000 (20:09 -0400)]
7167: stop rsync operation on any errors during Get or Put operations; add additional tests.
radhika [Fri, 9 Oct 2015 22:01:14 +0000 (18:01 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Fri, 9 Oct 2015 22:00:56 +0000 (18:00 -0400)]
Merge branch 'master' into 7167-keep-rsync-test-setup
Conflicts:
sdk/go/keepclient/perms.go
sdk/go/keepclient/perms_test.go
services/keepstore/perms.go
services/keepstore/perms_test.go
Peter Amstutz [Fri, 9 Oct 2015 20:41:52 +0000 (16:41 -0400)]
7491: Fix response body close, error logging.
Peter Amstutz [Fri, 9 Oct 2015 20:06:14 +0000 (16:06 -0400)]
Merge branch 'master' into 7491-go-keepclient-retry
Conflicts:
sdk/go/keepclient/keepclient.go
Peter Amstutz [Fri, 9 Oct 2015 19:52:36 +0000 (15:52 -0400)]
7491: Remove KeepServerError, all errors are BlockNotFound errors for now.
Refactor Get() and Ask() to use common function.
Tom Clegg [Fri, 9 Oct 2015 19:48:28 +0000 (15:48 -0400)]
Merge branch '7491-keepclient-bugs' refs #7491
Peter Amstutz [Fri, 9 Oct 2015 19:05:10 +0000 (15:05 -0400)]
7491: Go through entire server list instead of immediately retrying on error.
Only retry servers that resulted in an error (network or HTTP 5xx error).
Tom Clegg [Fri, 9 Oct 2015 18:51:13 +0000 (14:51 -0400)]
Merge branch '7167-blob-sign-sdk' refs #7167
Peter Amstutz [Fri, 9 Oct 2015 18:50:37 +0000 (14:50 -0400)]
7491: Manual merge with 7491-keepclient-bugs
Peter Amstutz [Fri, 9 Oct 2015 18:36:15 +0000 (14:36 -0400)]
7491: Add test that default replication is read from discovery document.
Tom Clegg [Fri, 9 Oct 2015 18:28:29 +0000 (14:28 -0400)]
7167: Deobfuscate variable names
Tom Clegg [Fri, 9 Oct 2015 18:20:15 +0000 (14:20 -0400)]
7167: Update tests and comments to new error vars.
Tom Clegg [Thu, 8 Oct 2015 21:50:22 +0000 (17:50 -0400)]
7167: Fix up comments
Tom Clegg [Thu, 8 Oct 2015 21:33:55 +0000 (17:33 -0400)]
7167: Replace duplicate tests with PermissionSecret tests
Peter Amstutz [Fri, 9 Oct 2015 16:18:57 +0000 (12:18 -0400)]
7491: Go keepclient retries GET on either network error, or 500 error from
server. Retry count is KeepClient.Retries. Also updated KeepClient
constructor to use "defaultCollectionReplication" from discovery document if
available.
Brett Smith [Fri, 9 Oct 2015 15:24:50 +0000 (11:24 -0400)]
Version the CWL runner's dependency on cwltool.
cwltool development is continuing on with API-incompatible changes.
No issue #.
radhika [Fri, 9 Oct 2015 12:44:46 +0000 (08:44 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Fri, 9 Oct 2015 12:43:37 +0000 (08:43 -0400)]
7167: default replications count from discovery doc test updates.
radhika [Thu, 8 Oct 2015 21:39:20 +0000 (17:39 -0400)]
7167: add tests with prefix during rsync
Tom Clegg [Thu, 8 Oct 2015 21:17:46 +0000 (17:17 -0400)]
7167: Tidy up errors. Remove extra comment copy.
radhika [Wed, 7 Oct 2015 20:47:56 +0000 (16:47 -0400)]
7167: move perms code from keepstore into keepclient go SDK.
radhika [Thu, 8 Oct 2015 20:22:45 +0000 (16:22 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
Conflicts:
tools/keep-rsync/keep-rsync_test.go
Tom Clegg [Thu, 8 Oct 2015 20:20:46 +0000 (16:20 -0400)]
6967: Update test to match improved code.
refs #6967
radhika [Thu, 8 Oct 2015 20:19:37 +0000 (16:19 -0400)]
7167: add tests to replications count
radhika [Thu, 8 Oct 2015 20:00:14 +0000 (16:00 -0400)]
7167: get replications count from destination api discovery doc and use it as default.
projects / arvados.git / log