radhika [Fri, 25 Sep 2015 14:13:49 +0000 (10:13 -0400)]
7161: working
radhika [Fri, 25 Sep 2015 12:32:46 +0000 (08:32 -0400)]
Merge branch 'master' into 7161-support-all-service-types
radhika [Fri, 25 Sep 2015 12:29:59 +0000 (08:29 -0400)]
Merge branch 'master' into 7162-support-service-types
radhika [Fri, 25 Sep 2015 12:28:04 +0000 (08:28 -0400)]
7162: updated comment
Peter Amstutz [Thu, 24 Sep 2015 20:36:59 +0000 (16:36 -0400)]
Merge branch '7181-crunch-missing-git' closes #7181
Peter Amstutz [Thu, 24 Sep 2015 20:36:35 +0000 (16:36 -0400)]
Merge branch '7228-crunch-dispatch-umask' closes #7228
radhika [Thu, 24 Sep 2015 20:25:37 +0000 (16:25 -0400)]
7162: move replicasPerThread logic outside the for loop.
Peter Amstutz [Thu, 24 Sep 2015 19:53:16 +0000 (15:53 -0400)]
7181: Check for [Errno 2] in the output and assume that means git is missing.
Remove --quite to get more output if egg_info fails. Use "warn" instead of
"print" because "print" doesn't seem to actually make it to the logs.
radhika [Thu, 24 Sep 2015 19:22:15 +0000 (15:22 -0400)]
7162: SetServiceRoots keepproxy hints only (revert my update).
radhika [Thu, 24 Sep 2015 18:57:40 +0000 (14:57 -0400)]
7161: white space
radhika [Thu, 24 Sep 2015 18:52:39 +0000 (14:52 -0400)]
7161: clarify max_replicas_per_service
radhika [Thu, 24 Sep 2015 15:42:58 +0000 (11:42 -0400)]
Merge branch 'master' into 7162-support-service-types
Tom Clegg [Thu, 24 Sep 2015 05:55:54 +0000 (01:55 -0400)]
7241: Add Delete, IndexTo, Touch, and Mtime.
Tom Clegg [Wed, 23 Sep 2015 21:46:21 +0000 (17:46 -0400)]
7241: Add Put() and Check()
radhika [Wed, 23 Sep 2015 21:03:33 +0000 (17:03 -0400)]
closes #7329
Merge branch '7329-empty-block-issue'
Brett Smith [Wed, 23 Sep 2015 20:56:35 +0000 (16:56 -0400)]
7321: arvados/jobs Dockerfile installs python-pip.
This is also for arvados_sdk_version support. I thought pip was a
dependency of virtualenv, but it's not. Refs #7321.
Tom Clegg [Wed, 23 Sep 2015 20:44:26 +0000 (16:44 -0400)]
7241: Accept command line flags for Azure blob volumes.
radhika [Wed, 23 Sep 2015 20:31:37 +0000 (16:31 -0400)]
Merge branch 'master' into 7329-empty-block-issue
radhika [Wed, 23 Sep 2015 20:29:02 +0000 (16:29 -0400)]
7329: improved TestableVolumeManagerFactory method signature, teardown logic, comments and test file name.
Brett Smith [Wed, 23 Sep 2015 20:27:28 +0000 (16:27 -0400)]
7321: arvados/jobs Dockerfile installs development headers.
These are necessary to build all of the Python SDK's dependencies when
running a job that defines arvados_sdk_version.
Closes #7321.
radhika [Wed, 23 Sep 2015 19:44:09 +0000 (15:44 -0400)]
Merge branch 'master' into 7329-empty-block-issue
radhika [Wed, 23 Sep 2015 19:32:44 +0000 (15:32 -0400)]
7161: Look for all service hints, not just keep proxy. Also, use thread count of 1 when it not a disk.
Peter Amstutz [Wed, 23 Sep 2015 17:00:08 +0000 (13:00 -0400)]
7228: Set umask to 002 so cache files are created writable other processes with same GID.
Peter Amstutz [Wed, 23 Sep 2015 14:42:38 +0000 (10:42 -0400)]
Merge branch '7225-collection-hang' closes #7225
Tom Clegg [Tue, 22 Sep 2015 23:54:12 +0000 (19:54 -0400)]
7241: Add AzureBlobVolume
radhika [Tue, 22 Sep 2015 22:25:53 +0000 (18:25 -0400)]
7329: since moved functional testing into generic testing, removing now redundant empty block tests from volume_unix_test.go and keepstore_test.go.
radhika [Tue, 22 Sep 2015 22:16:15 +0000 (18:16 -0400)]
7329: add generic volume based keepstore tests to have better functional test coverage.
Peter Amstutz [Tue, 22 Sep 2015 20:58:21 +0000 (16:58 -0400)]
7225: Better exception handling for unexpected state changes.
Peter Amstutz [Tue, 22 Sep 2015 19:23:31 +0000 (15:23 -0400)]
7225: Add test to validate fix that 2nd commit call blocks until block is committed
as intended.
Peter Amstutz [Tue, 22 Sep 2015 15:47:25 +0000 (11:47 -0400)]
7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
block commit" code getting skipped. Set PUT thread to daemon thread; users
must call save() or flush() or use `with` to guarantee that blocks get written.
radhika [Tue, 22 Sep 2015 16:47:26 +0000 (12:47 -0400)]
Merge branch 'master' into 7329-empty-block-issue
Tom Clegg [Tue, 22 Sep 2015 16:01:25 +0000 (12:01 -0400)]
7329: Fix infinite loop in Compare when reading an empty file.
refs #7329
radhika [Tue, 22 Sep 2015 15:40:44 +0000 (11:40 -0400)]
7162: Update code to address the bug "current code forgets to reset Using_proxy to false if the API server stops giving out proxies between one call and the next"
radhika [Tue, 22 Sep 2015 15:37:43 +0000 (11:37 -0400)]
Merge branch 'master' into 7162-support-service-types
radhika [Tue, 22 Sep 2015 14:38:26 +0000 (10:38 -0400)]
7162: Update datamanager code to ask for all non-proxy typed keep services and fail if any non-disk types are found in the returned list.
This needs to be enhanced when we are ready to support keep servers with other service types sub as blob storage.
Tom Clegg [Tue, 22 Sep 2015 14:14:20 +0000 (10:14 -0400)]
Merge branch '7309-git-push-default' closes #7309
Tom Clegg [Tue, 22 Sep 2015 14:14:06 +0000 (10:14 -0400)]
7309: Push branches explicitly in tests instead of relying on `git config push.default`
Tom Clegg [Tue, 22 Sep 2015 03:22:00 +0000 (03:22 +0000)]
7329: Add generic tests for empty block handling.
Tom Clegg [Mon, 21 Sep 2015 22:39:49 +0000 (22:39 +0000)]
7329: Fix infinite loop in Compare when reading an empty file (whether
or not the buf we're comparing is also empty).
radhika [Mon, 21 Sep 2015 21:39:48 +0000 (17:39 -0400)]
Merge branch 'master' into 7162-support-service-types
radhika [Mon, 21 Sep 2015 21:36:03 +0000 (17:36 -0400)]
7162: Use replicasPerService to avoid sending redundant replication requests.
Brett Smith [Mon, 21 Sep 2015 18:56:43 +0000 (14:56 -0400)]
API server limits KeepService changes to admins.
Taking the same security policy that applies to disks and applying
them to services as well. No issue #.
radhika [Mon, 21 Sep 2015 15:08:07 +0000 (11:08 -0400)]
7162: enhance SetServiceRoots to look for all service hints, not just K@ hints.
radhika [Thu, 17 Sep 2015 19:03:59 +0000 (15:03 -0400)]
7329: It appears that UnixVolume -> Compare method is falling in infinite loop due to the fact that EOF is not returned when reading an empty file.
Due to this, PutBlock for an EmptyHash is resulting in infinite loop when an EmptyBlock already exists in Keep.
Added a quick fix in the Compare method for unix volume implementation. We can try to see why EOF is not returned and put in a better fix,
but for the time being this helps with unblocking tests.
Added several tests for empty block. Most importantly, added an keep integration test with UnixVolume so that we have integration tests covering some
of these code paths. More tests are definitely called for in this intergration test for better code coverage.
Also did gofmt and golint updates.
Brett Smith [Wed, 16 Sep 2015 21:30:37 +0000 (17:30 -0400)]
Merge branch 'pr/24'
Closes #7262.
radhika [Tue, 15 Sep 2015 20:34:44 +0000 (16:34 -0400)]
refs #7015
Merge branch '7015-remove-pipeline-instance-from-recursive-arv-copy-doc'
radhika [Tue, 15 Sep 2015 20:32:44 +0000 (16:32 -0400)]
7015: correct typo in copied template uuid
radhika [Tue, 15 Sep 2015 19:53:16 +0000 (15:53 -0400)]
7015: Update arv-copy user guide to use a pipeline template instead of instance in the examples.
In addition, until 7001 is resolved, do not include pipeline instance as one of the object types that can be copied over.
radhika [Tue, 15 Sep 2015 17:41:42 +0000 (13:41 -0400)]
closes #6260
Merge branch '6260-test-datamanager'
radhika [Tue, 15 Sep 2015 17:16:38 +0000 (13:16 -0400)]
6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead of locator+size only.
radhika [Tue, 15 Sep 2015 16:24:30 +0000 (12:24 -0400)]
Merge branch 'master' into 6260-test-datamanager
Conflicts:
services/keepstore/keepstore.go
radhika [Tue, 15 Sep 2015 16:18:42 +0000 (12:18 -0400)]
refs #7179
Merge branch '7179-generic-volume-tests'
radhika [Tue, 15 Sep 2015 16:16:31 +0000 (12:16 -0400)]
7179: more updates to address golint suggestions around the config parameters such as never-delete, blob-signature-ttl etc.
Nico Cesar [Tue, 15 Sep 2015 14:53:46 +0000 (10:53 -0400)]
docker builds are failing for compute nodes.
no issue #
Joshua C. Randall [Tue, 15 Sep 2015 10:58:15 +0000 (10:58 +0000)]
7262: blacklists pbr >= 1.7.0
radhika [Tue, 15 Sep 2015 00:16:32 +0000 (20:16 -0400)]
6260: most golint suggestions are addressed.
radhika [Mon, 14 Sep 2015 22:51:45 +0000 (18:51 -0400)]
6260: so many golint complaints!!
radhika [Mon, 14 Sep 2015 19:23:56 +0000 (15:23 -0400)]
6260: golint suggested updates
radhika [Mon, 14 Sep 2015 19:23:21 +0000 (15:23 -0400)]
Merge branch 'master' into 6260-test-datamanager
Conflicts:
services/keepstore/keepstore.go
radhika [Mon, 14 Sep 2015 19:11:47 +0000 (15:11 -0400)]
closes #7179
Merge branch '7179-generic-volume-tests'
radhika [Mon, 14 Sep 2015 15:06:44 +0000 (11:06 -0400)]
6260: golint suggestions incorporated. Also, the earlier commit log message "no longer need the keep.data-manager-token-file" is incorrect. Keepstore still needs this and this file is still generated during testing.
radhika [Mon, 14 Sep 2015 14:31:47 +0000 (10:31 -0400)]
7179: Most of golint suggested updates are made. Some names such as "never_delete" are left alone.
Also suggestions such as "replace errors.New(fmt.Sprintf(...)) with fmt.Errorf(...)" are ignored.
radhika [Mon, 14 Sep 2015 13:33:13 +0000 (09:33 -0400)]
7179: some more golint suggested updates
radhika [Sat, 12 Sep 2015 15:02:22 +0000 (11:02 -0400)]
7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use CamelCase".
Now keepstore.go and handlers.go are almost lint free; a few such as blob_signature_ttl are not renamed at this time.
radhika [Sat, 12 Sep 2015 14:06:23 +0000 (10:06 -0400)]
7179: handler_test.go is now lint free according to golint.
radhika [Sat, 12 Sep 2015 13:32:29 +0000 (09:32 -0400)]
7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes, update each test to either support
both read-only and writable modes, or skip a test if it is indended for the other mode. This helps ensure as many
tests as possible are executed in either case.
radhika [Sat, 12 Sep 2015 00:33:50 +0000 (20:33 -0400)]
7179: A few golint suggested updates. There are still a lot of golint complaints.
radhika [Fri, 11 Sep 2015 23:18:27 +0000 (19:18 -0400)]
7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.
radhika [Fri, 11 Sep 2015 20:46:07 +0000 (16:46 -0400)]
Merge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-generic-volume-tests
Tom Clegg [Fri, 11 Sep 2015 20:13:53 +0000 (16:13 -0400)]
7179: Tighten Put requirements when overwriting existing data.
radhika [Fri, 11 Sep 2015 19:19:21 +0000 (15:19 -0400)]
Merge branch 'master' into 7179-generic-volume-tests
Tom Clegg [Fri, 11 Sep 2015 18:41:28 +0000 (14:41 -0400)]
Merge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310
radhika [Fri, 11 Sep 2015 15:59:05 +0000 (11:59 -0400)]
7179: add generic volume test; as part of this exercise, moved some "generic" tests from volume_unix_test.go
into volume_generic_test.go so that we do not repeat these in all storage device related testing.
Joshua C. Randall [Fri, 11 Sep 2015 15:08:44 +0000 (15:08 +0000)]
7310: adds git v2 style regex to match http 500 error
Tom Clegg [Thu, 10 Sep 2015 15:17:36 +0000 (11:17 -0400)]
6260: Reduced-replication test requires replication>=desired, not replication==len(keepServers).
radhika [Thu, 10 Sep 2015 14:59:48 +0000 (10:59 -0400)]
7179: list out the potential tests
Tom Clegg [Thu, 10 Sep 2015 14:54:45 +0000 (10:54 -0400)]
6260: Remove unnecessary Sleep()
Joshua C. Randall [Thu, 10 Sep 2015 13:31:16 +0000 (14:31 +0100)]
Puts pbr 1.7.0 on blacklist for services/dockercleaner
Joshua C. Randall [Thu, 10 Sep 2015 11:37:25 +0000 (11:37 +0000)]
Blacklists pbr version 1.7.0 as it is not working with mock 1.3.0
radhika [Thu, 10 Sep 2015 01:14:37 +0000 (21:14 -0400)]
6260: no longer need the keep.data-manager-token-file
Tom Clegg [Thu, 10 Sep 2015 00:49:30 +0000 (20:49 -0400)]
Merge branch '7179-test-mocks' refs #7179
Tom Clegg [Thu, 10 Sep 2015 00:48:53 +0000 (20:48 -0400)]
Merge branch 'master' into 7179-test-mocks
Conflicts:
services/keepstore/volume.go
services/keepstore/volume_unix_test.go
Tom Clegg [Wed, 9 Sep 2015 21:41:57 +0000 (17:41 -0400)]
7179: Improve comments.
Peter Amstutz [Wed, 9 Sep 2015 20:39:37 +0000 (16:39 -0400)]
Brown bag typo bugfix. refs #7031
Peter Amstutz [Wed, 9 Sep 2015 20:35:36 +0000 (16:35 -0400)]
Catch errors thrown by sync_node() in Azure driver refs #7031.
Tom Clegg [Wed, 9 Sep 2015 20:12:28 +0000 (16:12 -0400)]
6260: Fix up /status.json checks.
Tom Clegg [Wed, 9 Sep 2015 19:09:46 +0000 (15:09 -0400)]
6260: Simplify verifyBlocks logic.
Tom Clegg [Wed, 9 Sep 2015 19:09:15 +0000 (15:09 -0400)]
6260: Remove test for impossible condition.
Tom Clegg [Wed, 9 Sep 2015 19:09:05 +0000 (15:09 -0400)]
6260: Fix up error messages.
Tom Clegg [Wed, 9 Sep 2015 19:08:31 +0000 (15:08 -0400)]
6260: Remove extra hyphen from command line args.
Tom Clegg [Wed, 9 Sep 2015 19:08:09 +0000 (15:08 -0400)]
6260: Just use the token loaded by the SDK (ARVADOS_API_TOKEN),
instead of requiring a token separately on the command line.
Tom Clegg [Wed, 9 Sep 2015 16:32:30 +0000 (12:32 -0400)]
7179: Improve description of how races must be handled.
Peter Amstutz [Wed, 9 Sep 2015 15:43:54 +0000 (11:43 -0400)]
Bump node manager libcloud pin to dev3 refs #7031
Tom Clegg [Wed, 9 Sep 2015 02:40:22 +0000 (22:40 -0400)]
Merge branch '7121-fix-deadlock' closes #7121
radhika [Wed, 9 Sep 2015 00:48:22 +0000 (20:48 -0400)]
6260: update datamanager.go to return error than Fatalf in some cases; add additional tests to test those errors raised.
radhika [Tue, 8 Sep 2015 22:18:19 +0000 (18:18 -0400)]
6260: Continue to force never_delete to be true until #6221 is resolved; in the meantime, let it be false during testing.
radhika [Tue, 8 Sep 2015 21:30:48 +0000 (17:30 -0400)]
Merge branch 'master' into 6260-test-datamanager
radhika [Tue, 8 Sep 2015 21:30:14 +0000 (17:30 -0400)]
6260: look for two replicas
Tom Clegg [Tue, 8 Sep 2015 21:12:17 +0000 (17:12 -0400)]
7179: Add Volume interface specs.
Peter Amstutz [Tue, 8 Sep 2015 19:35:02 +0000 (15:35 -0400)]
Merge branch '7193-provision-azure-compute' closes #7193
projects / arvados.git / log