Peter Amstutz [Fri, 16 Oct 2015 15:42:01 +0000 (11:42 -0400)]
Merge branch '6321-slurm-oserror' closes #6321
Peter Amstutz [Fri, 16 Oct 2015 15:40:36 +0000 (11:40 -0400)]
6321: Add note about rationale for retrying on OSError.
Peter Amstutz [Fri, 16 Oct 2015 15:16:49 +0000 (11:16 -0400)]
6321: Add test that OSError is caught from slurm subprocess invocations.
Tom Clegg [Thu, 15 Oct 2015 20:27:58 +0000 (16:27 -0400)]
Merge branch '7159-keepexercise' refs #7159
Tom Clegg [Thu, 15 Oct 2015 20:25:14 +0000 (16:25 -0400)]
7159: Add keep-exercise
Tom Clegg [Thu, 15 Oct 2015 19:47:11 +0000 (15:47 -0400)]
Merge branch 'wtsi-hgi-feature/arv-view'
refs #7204
Tom Clegg [Thu, 15 Oct 2015 19:42:37 +0000 (15:42 -0400)]
7204: Tweak usage message, add a .shellescape, fix error-reporting error
radhika [Thu, 15 Oct 2015 16:35:32 +0000 (12:35 -0400)]
closes #7167
Merge branch '7167-keep-rsync'
radhika [Thu, 15 Oct 2015 16:35:15 +0000 (12:35 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Thu, 15 Oct 2015 16:32:07 +0000 (12:32 -0400)]
Merge branch 'master' into 7167-keep-rsync
Conflicts:
sdk/go/keepclient/keepclient.go
Tom Clegg [Thu, 15 Oct 2015 16:21:07 +0000 (12:21 -0400)]
Merge branch '7173-jessie' refs #7173
Tom Clegg [Mon, 31 Aug 2015 19:00:13 +0000 (15:00 -0400)]
7173: Add jessie to supported distro list.
radhika [Thu, 15 Oct 2015 16:18:20 +0000 (12:18 -0400)]
7167: update the progress message to say "est. time remaining" instead of "ETA" which is not accurate in this case.
radhika [Thu, 15 Oct 2015 15:56:36 +0000 (11:56 -0400)]
7167: rename the newly added StopKeepServers as StopKeepWithParams; it now sounds similar to StartKeepWithParams and is self explanatory.
radhika [Thu, 15 Oct 2015 15:08:10 +0000 (11:08 -0400)]
Merge branch '7167-keep-rsync' of git.curoverse.com:arvados into 7167-keep-rsync
Tom Clegg [Thu, 15 Oct 2015 15:02:30 +0000 (11:02 -0400)]
7167: Always pass --num-keep-servers to python wrapper. Update comments.
Tom Clegg [Thu, 15 Oct 2015 14:50:13 +0000 (10:50 -0400)]
7167: Rename conf flags to -src and -dst to match arv-copy. Always add .conf for short form.
Tom Clegg [Thu, 15 Oct 2015 14:48:33 +0000 (10:48 -0400)]
7167: Fix up progress/ETA logging.
Peter Amstutz [Thu, 15 Oct 2015 14:32:26 +0000 (10:32 -0400)]
Merge branch '7491-go-keepclient-retry' closes #7491
Peter Amstutz [Thu, 15 Oct 2015 14:16:27 +0000 (10:16 -0400)]
7491: Remove unnecessary writable service roots from GET tests.
Tom Clegg [Thu, 15 Oct 2015 13:18:43 +0000 (09:18 -0400)]
Merge branch 'master' into wtsi-hgi-feature/arv-view
Conflicts:
sdk/cli/bin/arv
radhika [Thu, 15 Oct 2015 03:01:13 +0000 (23:01 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Thu, 15 Oct 2015 03:00:05 +0000 (23:00 -0400)]
7167: log progress during keep-rsync and several test improvements.
Brett Smith [Wed, 14 Oct 2015 20:59:40 +0000 (16:59 -0400)]
API server needs an arvados-cli with crunch-job --docker-bin.
crunch-dispatch was extended to use `crunch-job --docker-bin` in
#6838. This commit simply updates the Gemfile to ensure this
dependency is satisfied. No issue #.
radhika [Wed, 14 Oct 2015 19:15:15 +0000 (15:15 -0400)]
Merge branch 'master' into 7167-keep-rsync
Tom Clegg [Wed, 14 Oct 2015 18:51:34 +0000 (14:51 -0400)]
Merge branch '7167-propagate-error' refs #7167
Tom Clegg [Wed, 14 Oct 2015 18:51:01 +0000 (14:51 -0400)]
Merge branch '7159-clean-index' refs #7159 refs #7168
radhika [Wed, 14 Oct 2015 17:43:54 +0000 (13:43 -0400)]
7167: loadConfig setupKeepclient do only one set at a time.
Peter Amstutz [Wed, 14 Oct 2015 14:49:49 +0000 (10:49 -0400)]
6321: Add OSError to slurm shutdown actor retry conditions, to catch and retry
subprocess errors from the operating system, which is preferrable to crashing
the shutdown actor.
radhika [Wed, 14 Oct 2015 02:27:41 +0000 (22:27 -0400)]
7167: when the config file does not contain '/', use $HOME/.config/arvados/<filename>.
radhika [Wed, 14 Oct 2015 01:47:06 +0000 (21:47 -0400)]
7167: replace keep_existing with num_keep_servers and use it create all required keep servers at once.
radhika [Wed, 14 Oct 2015 01:16:35 +0000 (21:16 -0400)]
7167: replace the keep_existing logic and create all 3 keep servers at once and use the first two as src keepservers and the last one as the dst keep server.
radhika [Tue, 13 Oct 2015 21:01:46 +0000 (17:01 -0400)]
7167: Convert blobSigningKey also into local variable and make necessary changes to accommodate this change.
Remove the New method added in arvadosclient.go and revert MakeArvadosClient to what it was before.
radhika [Tue, 13 Oct 2015 19:46:26 +0000 (15:46 -0400)]
7167: update run_test_servers.py to use action="store_true" instead of converting string to boolean.
Tom Clegg [Tue, 13 Oct 2015 19:33:33 +0000 (15:33 -0400)]
7159: Address golint complaints
Tom Clegg [Tue, 13 Oct 2015 19:33:02 +0000 (15:33 -0400)]
7159: Omit non-Keep blobs from index
radhika [Tue, 13 Oct 2015 17:38:52 +0000 (13:38 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Tue, 13 Oct 2015 17:37:33 +0000 (13:37 -0400)]
7167: Convert most of the globals in keep-sync into locals and update all the code and tests as needed.
Tom Clegg [Tue, 13 Oct 2015 16:11:23 +0000 (12:11 -0400)]
7159: Return benign os.ErrNotExist error from Compare to avoid excessive logs. refs #7159
Tom Clegg [Tue, 13 Oct 2015 15:17:49 +0000 (11:17 -0400)]
7159: Fix error handling when reading full size block. refs #7159
Tom Clegg [Mon, 12 Oct 2015 17:41:04 +0000 (13:41 -0400)]
7167: Propagate read errors to caller. Fixes failing TestTransferShortBuffer.
Tom Clegg [Mon, 12 Oct 2015 16:40:38 +0000 (12:40 -0400)]
Warn about unhandled case if broken node has no ping time. refs #7286
Tom Clegg [Mon, 12 Oct 2015 16:06:47 +0000 (12:06 -0400)]
Merge branch '7159-empty-blob-race' refs #7159
Tom Clegg [Mon, 12 Oct 2015 13:33:24 +0000 (09:33 -0400)]
7159: Shorten race waits during generic tests
Tom Clegg [Fri, 9 Oct 2015 21:09:42 +0000 (17:09 -0400)]
7159: Log when waiting for get/put races
Tom Clegg [Thu, 8 Oct 2015 18:00:00 +0000 (14:00 -0400)]
7159: Exclude new empty blocks from index.
Tom Clegg [Thu, 8 Oct 2015 17:30:18 +0000 (13:30 -0400)]
7159: Test race deadline
Tom Clegg [Thu, 8 Oct 2015 16:52:17 +0000 (12:52 -0400)]
7159: Work around CreateBlob race by polling for updates when a brand new blob is found empty.
radhika [Mon, 12 Oct 2015 14:40:38 +0000 (10:40 -0400)]
7167: break load config logic out of main into loadConfig func and add several tests.
radhika [Mon, 12 Oct 2015 13:13:45 +0000 (09:13 -0400)]
7167: some more error tests such as error getting block from src and error putting block to dst.
radhika [Sat, 10 Oct 2015 00:09:17 +0000 (20:09 -0400)]
7167: stop rsync operation on any errors during Get or Put operations; add additional tests.
radhika [Fri, 9 Oct 2015 22:01:14 +0000 (18:01 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Fri, 9 Oct 2015 22:00:56 +0000 (18:00 -0400)]
Merge branch 'master' into 7167-keep-rsync-test-setup
Conflicts:
sdk/go/keepclient/perms.go
sdk/go/keepclient/perms_test.go
services/keepstore/perms.go
services/keepstore/perms_test.go
Peter Amstutz [Fri, 9 Oct 2015 20:41:52 +0000 (16:41 -0400)]
7491: Fix response body close, error logging.
Peter Amstutz [Fri, 9 Oct 2015 20:06:14 +0000 (16:06 -0400)]
Merge branch 'master' into 7491-go-keepclient-retry
Conflicts:
sdk/go/keepclient/keepclient.go
Peter Amstutz [Fri, 9 Oct 2015 19:52:36 +0000 (15:52 -0400)]
7491: Remove KeepServerError, all errors are BlockNotFound errors for now.
Refactor Get() and Ask() to use common function.
Tom Clegg [Fri, 9 Oct 2015 19:48:28 +0000 (15:48 -0400)]
Merge branch '7491-keepclient-bugs' refs #7491
Peter Amstutz [Fri, 9 Oct 2015 19:05:10 +0000 (15:05 -0400)]
7491: Go through entire server list instead of immediately retrying on error.
Only retry servers that resulted in an error (network or HTTP 5xx error).
Tom Clegg [Fri, 9 Oct 2015 18:51:13 +0000 (14:51 -0400)]
Merge branch '7167-blob-sign-sdk' refs #7167
Peter Amstutz [Fri, 9 Oct 2015 18:50:37 +0000 (14:50 -0400)]
7491: Manual merge with 7491-keepclient-bugs
Peter Amstutz [Fri, 9 Oct 2015 18:36:15 +0000 (14:36 -0400)]
7491: Add test that default replication is read from discovery document.
Tom Clegg [Fri, 9 Oct 2015 18:28:29 +0000 (14:28 -0400)]
7167: Deobfuscate variable names
Tom Clegg [Fri, 9 Oct 2015 18:20:15 +0000 (14:20 -0400)]
7167: Update tests and comments to new error vars.
Tom Clegg [Thu, 8 Oct 2015 21:50:22 +0000 (17:50 -0400)]
7167: Fix up comments
Tom Clegg [Thu, 8 Oct 2015 21:33:55 +0000 (17:33 -0400)]
7167: Replace duplicate tests with PermissionSecret tests
Peter Amstutz [Fri, 9 Oct 2015 16:18:57 +0000 (12:18 -0400)]
7491: Go keepclient retries GET on either network error, or 500 error from
server. Retry count is KeepClient.Retries. Also updated KeepClient
constructor to use "defaultCollectionReplication" from discovery document if
available.
Brett Smith [Fri, 9 Oct 2015 15:24:50 +0000 (11:24 -0400)]
Version the CWL runner's dependency on cwltool.
cwltool development is continuing on with API-incompatible changes.
No issue #.
radhika [Fri, 9 Oct 2015 12:44:46 +0000 (08:44 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Fri, 9 Oct 2015 12:43:37 +0000 (08:43 -0400)]
7167: default replications count from discovery doc test updates.
radhika [Thu, 8 Oct 2015 21:39:20 +0000 (17:39 -0400)]
7167: add tests with prefix during rsync
Tom Clegg [Thu, 8 Oct 2015 21:17:46 +0000 (17:17 -0400)]
7167: Tidy up errors. Remove extra comment copy.
radhika [Wed, 7 Oct 2015 20:47:56 +0000 (16:47 -0400)]
7167: move perms code from keepstore into keepclient go SDK.
radhika [Thu, 8 Oct 2015 20:22:45 +0000 (16:22 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
Conflicts:
tools/keep-rsync/keep-rsync_test.go
Tom Clegg [Thu, 8 Oct 2015 20:20:46 +0000 (16:20 -0400)]
6967: Update test to match improved code.
refs #6967
radhika [Thu, 8 Oct 2015 20:19:37 +0000 (16:19 -0400)]
7167: add tests to replications count
radhika [Thu, 8 Oct 2015 20:00:14 +0000 (16:00 -0400)]
7167: get replications count from destination api discovery doc and use it as default.
Tom Clegg [Tue, 6 Oct 2015 17:48:05 +0000 (13:48 -0400)]
7491: Ensure status channel stays open until all upload workers finish.
Tom Clegg [Tue, 6 Oct 2015 17:39:12 +0000 (13:39 -0400)]
7491: Fix error handling/reporting in keepclient/GET
Tom Clegg [Thu, 8 Oct 2015 19:24:19 +0000 (15:24 -0400)]
Merge branch '6967-yaml-format' closes #6967
Tom Clegg [Thu, 8 Oct 2015 18:46:22 +0000 (14:46 -0400)]
6967: More helpful comment & assertion failure message
Tom Clegg [Thu, 8 Oct 2015 18:45:33 +0000 (14:45 -0400)]
6967: Use git status --porcelain to isolate from user config
Tom Clegg [Wed, 7 Oct 2015 15:11:14 +0000 (11:11 -0400)]
6967: Move source_version detection code from config yaml to lib/app_version.rb.
Tom Clegg [Wed, 7 Oct 2015 14:00:16 +0000 (10:00 -0400)]
6967: Move source_version detection code from config yaml to lib/app_version.rb.
Tom Clegg [Wed, 7 Oct 2015 14:01:22 +0000 (10:01 -0400)]
6967: Treat blob_signing_key like a secret in `rake config:check`.
radhika [Thu, 8 Oct 2015 17:12:37 +0000 (13:12 -0400)]
7167: honor blob signing key while getting blocks.
radhika [Thu, 8 Oct 2015 15:13:41 +0000 (11:13 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
Conflicts:
tools/keep-rsync/keep-rsync_test.go
radhika [Thu, 8 Oct 2015 15:06:57 +0000 (11:06 -0400)]
7167: set enforce_permissions to true if blob signing key argument is provided.
radhika [Wed, 7 Oct 2015 22:24:51 +0000 (18:24 -0400)]
7167: add --keep-enforce-permissions to run_test_servers.py
radhika [Wed, 7 Oct 2015 20:50:04 +0000 (16:50 -0400)]
Merge branch 'master' into 7167-keep-rsync-test-setup
radhika [Wed, 7 Oct 2015 20:49:15 +0000 (16:49 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Wed, 7 Oct 2015 20:47:56 +0000 (16:47 -0400)]
7167: move perms code from keepstore into keepclient go SDK.
Peter Amstutz [Wed, 7 Oct 2015 18:09:40 +0000 (14:09 -0400)]
Merge branch '6142-cancel-slurm' closes #6142
radhika [Wed, 7 Oct 2015 18:03:12 +0000 (14:03 -0400)]
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
radhika [Wed, 7 Oct 2015 18:01:18 +0000 (14:01 -0400)]
7167: merge test setup branch
radhika [Wed, 7 Oct 2015 17:52:43 +0000 (13:52 -0400)]
Merge branch 'master' into 7167-keep-rsync
radhika [Wed, 7 Oct 2015 17:52:33 +0000 (13:52 -0400)]
Merge branch 'master' into 7167-keep-rsync-test-setup
radhika [Wed, 7 Oct 2015 17:51:16 +0000 (13:51 -0400)]
7167: rename MakeArvadosClientWithConfig as New
Tom Clegg [Wed, 7 Oct 2015 15:44:25 +0000 (11:44 -0400)]
Merge branch '7254-dont-lose-replication-arg' closes #7254
Brett Smith [Wed, 7 Oct 2015 14:54:08 +0000 (10:54 -0400)]
Merge branch '7435-node-manager-shutdown-cleanup-wip'
Closes #7435, #7445.
Brett Smith [Wed, 7 Oct 2015 14:47:23 +0000 (10:47 -0400)]
7254: Test arv-put preserves replication when cache load fails.
projects / arvados.git / log