arvados.git
9 years agoMerge branch '7225-collection-hang' closes #7225
Peter Amstutz [Wed, 23 Sep 2015 14:42:38 +0000 (10:42 -0400)]
Merge branch '7225-collection-hang' closes #7225

9 years ago7225: Better exception handling for unexpected state changes.
Peter Amstutz [Tue, 22 Sep 2015 20:58:21 +0000 (16:58 -0400)]
7225: Better exception handling for unexpected state changes.

9 years ago7225: Add test to validate fix that 2nd commit call blocks until block is committed
Peter Amstutz [Tue, 22 Sep 2015 19:23:31 +0000 (15:23 -0400)]
7225: Add test to validate fix that 2nd commit call blocks until block is committed
as intended.

9 years ago7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
Peter Amstutz [Tue, 22 Sep 2015 15:47:25 +0000 (11:47 -0400)]
7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
block commit" code getting skipped.  Set PUT thread to daemon thread; users
must call save() or flush() or use `with` to guarantee that blocks get written.

9 years ago7329: Fix infinite loop in Compare when reading an empty file.
Tom Clegg [Tue, 22 Sep 2015 16:01:25 +0000 (12:01 -0400)]
7329: Fix infinite loop in Compare when reading an empty file.

refs #7329

9 years agoMerge branch '7309-git-push-default' closes #7309
Tom Clegg [Tue, 22 Sep 2015 14:14:20 +0000 (10:14 -0400)]
Merge branch '7309-git-push-default' closes #7309

9 years ago7309: Push branches explicitly in tests instead of relying on `git config push.default`
Tom Clegg [Tue, 22 Sep 2015 14:14:06 +0000 (10:14 -0400)]
7309: Push branches explicitly in tests instead of relying on `git config push.default`

9 years agoAPI server limits KeepService changes to admins.
Brett Smith [Mon, 21 Sep 2015 18:56:43 +0000 (14:56 -0400)]
API server limits KeepService changes to admins.

Taking the same security policy that applies to disks and applying
them to services as well.  No issue #.

9 years agoMerge branch 'pr/24'
Brett Smith [Wed, 16 Sep 2015 21:30:37 +0000 (17:30 -0400)]
Merge branch 'pr/24'

Closes #7262.

9 years agorefs #7015
radhika [Tue, 15 Sep 2015 20:34:44 +0000 (16:34 -0400)]
refs #7015
Merge branch '7015-remove-pipeline-instance-from-recursive-arv-copy-doc'

9 years ago7015: correct typo in copied template uuid
radhika [Tue, 15 Sep 2015 20:32:44 +0000 (16:32 -0400)]
7015: correct typo in copied template uuid

9 years ago7015: Update arv-copy user guide to use a pipeline template instead of instance in...
radhika [Tue, 15 Sep 2015 19:53:16 +0000 (15:53 -0400)]
7015: Update arv-copy user guide to use a pipeline template instead of instance in the examples.
In addition, until 7001 is resolved, do not include pipeline instance as one of the object types that can be copied over.

9 years agocloses #6260
radhika [Tue, 15 Sep 2015 17:41:42 +0000 (13:41 -0400)]
closes #6260
Merge branch '6260-test-datamanager'

9 years ago6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead...
radhika [Tue, 15 Sep 2015 17:16:38 +0000 (13:16 -0400)]
6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead of locator+size only.

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Tue, 15 Sep 2015 16:24:30 +0000 (12:24 -0400)]
Merge branch 'master' into 6260-test-datamanager

Conflicts:
services/keepstore/keepstore.go

9 years agorefs #7179
radhika [Tue, 15 Sep 2015 16:18:42 +0000 (12:18 -0400)]
refs #7179
Merge branch '7179-generic-volume-tests'

9 years ago7179: more updates to address golint suggestions around the config parameters such...
radhika [Tue, 15 Sep 2015 16:16:31 +0000 (12:16 -0400)]
7179: more updates to address golint suggestions around the config parameters such as never-delete, blob-signature-ttl etc.

9 years agodocker builds are failing for compute nodes.
Nico Cesar [Tue, 15 Sep 2015 14:53:46 +0000 (10:53 -0400)]
docker builds are failing for compute nodes.

no issue #

9 years ago7262: blacklists pbr >= 1.7.0
Joshua C. Randall [Tue, 15 Sep 2015 10:58:15 +0000 (10:58 +0000)]
7262: blacklists pbr >= 1.7.0

9 years ago6260: most golint suggestions are addressed.
radhika [Tue, 15 Sep 2015 00:16:32 +0000 (20:16 -0400)]
6260: most golint suggestions are addressed.

9 years ago6260: so many golint complaints!!
radhika [Mon, 14 Sep 2015 22:51:45 +0000 (18:51 -0400)]
6260: so many golint complaints!!

9 years ago6260: golint suggested updates
radhika [Mon, 14 Sep 2015 19:23:56 +0000 (15:23 -0400)]
6260: golint suggested updates

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Mon, 14 Sep 2015 19:23:21 +0000 (15:23 -0400)]
Merge branch 'master' into 6260-test-datamanager

Conflicts:
services/keepstore/keepstore.go

9 years agocloses #7179
radhika [Mon, 14 Sep 2015 19:11:47 +0000 (15:11 -0400)]
closes #7179
Merge branch '7179-generic-volume-tests'

9 years ago6260: golint suggestions incorporated. Also, the earlier commit log message "no longe...
radhika [Mon, 14 Sep 2015 15:06:44 +0000 (11:06 -0400)]
6260: golint suggestions incorporated. Also, the earlier commit log message "no longer need the keep.data-manager-token-file" is incorrect. Keepstore still needs this and this file is still generated during testing.

9 years ago7179: Most of golint suggested updates are made. Some names such as "never_delete...
radhika [Mon, 14 Sep 2015 14:31:47 +0000 (10:31 -0400)]
7179: Most of golint suggested updates are made. Some names such as "never_delete" are left alone.
Also suggestions such as "replace errors.New(fmt.Sprintf(...)) with fmt.Errorf(...)" are ignored.

9 years ago7179: some more golint suggested updates
radhika [Mon, 14 Sep 2015 13:33:13 +0000 (09:33 -0400)]
7179: some more golint suggested updates

9 years ago7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use...
radhika [Sat, 12 Sep 2015 15:02:22 +0000 (11:02 -0400)]
7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use CamelCase".
Now keepstore.go and handlers.go are almost lint free; a few such as blob_signature_ttl are not renamed at this time.

9 years ago7179: handler_test.go is now lint free according to golint.
radhika [Sat, 12 Sep 2015 14:06:23 +0000 (10:06 -0400)]
7179: handler_test.go is now lint free according to golint.

9 years ago7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes...
radhika [Sat, 12 Sep 2015 13:32:29 +0000 (09:32 -0400)]
7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes, update each test to either support
both read-only and writable modes, or skip a test if it is indended for the other mode. This helps ensure as many
tests as possible are executed in either case.

9 years ago7179: A few golint suggested updates. There are still a lot of golint complaints.
radhika [Sat, 12 Sep 2015 00:33:50 +0000 (20:33 -0400)]
7179: A few golint suggested updates. There are still a lot of golint complaints.

9 years ago7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.
radhika [Fri, 11 Sep 2015 23:18:27 +0000 (19:18 -0400)]
7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.

9 years agoMerge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-gener...
radhika [Fri, 11 Sep 2015 20:46:07 +0000 (16:46 -0400)]
Merge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-generic-volume-tests

9 years ago7179: Tighten Put requirements when overwriting existing data.
Tom Clegg [Fri, 11 Sep 2015 20:13:53 +0000 (16:13 -0400)]
7179: Tighten Put requirements when overwriting existing data.

9 years agoMerge branch 'master' into 7179-generic-volume-tests
radhika [Fri, 11 Sep 2015 19:19:21 +0000 (15:19 -0400)]
Merge branch 'master' into 7179-generic-volume-tests

9 years agoMerge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310
Tom Clegg [Fri, 11 Sep 2015 18:41:28 +0000 (14:41 -0400)]
Merge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310

9 years ago7179: add generic volume test; as part of this exercise, moved some "generic" tests...
radhika [Fri, 11 Sep 2015 15:59:05 +0000 (11:59 -0400)]
7179: add generic volume test; as part of this exercise, moved some "generic" tests from volume_unix_test.go
into volume_generic_test.go so that we do not repeat these in all storage device related testing.

9 years ago7310: adds git v2 style regex to match http 500 error
Joshua C. Randall [Fri, 11 Sep 2015 15:08:44 +0000 (15:08 +0000)]
7310: adds git v2 style regex to match http 500 error

9 years ago6260: Reduced-replication test requires replication>=desired, not replication==len...
Tom Clegg [Thu, 10 Sep 2015 15:17:36 +0000 (11:17 -0400)]
6260: Reduced-replication test requires replication>=desired, not replication==len(keepServers).

9 years ago7179: list out the potential tests
radhika [Thu, 10 Sep 2015 14:59:48 +0000 (10:59 -0400)]
7179: list out the potential tests

9 years ago6260: Remove unnecessary Sleep()
Tom Clegg [Thu, 10 Sep 2015 14:54:45 +0000 (10:54 -0400)]
6260: Remove unnecessary Sleep()

9 years agoPuts pbr 1.7.0 on blacklist for services/dockercleaner
Joshua C. Randall [Thu, 10 Sep 2015 13:31:16 +0000 (14:31 +0100)]
Puts pbr 1.7.0 on blacklist for services/dockercleaner

9 years agoBlacklists pbr version 1.7.0 as it is not working with mock 1.3.0
Joshua C. Randall [Thu, 10 Sep 2015 11:37:25 +0000 (11:37 +0000)]
Blacklists pbr version 1.7.0 as it is not working with mock 1.3.0

9 years ago6260: no longer need the keep.data-manager-token-file
radhika [Thu, 10 Sep 2015 01:14:37 +0000 (21:14 -0400)]
6260: no longer need the keep.data-manager-token-file

9 years agoMerge branch '7179-test-mocks' refs #7179
Tom Clegg [Thu, 10 Sep 2015 00:49:30 +0000 (20:49 -0400)]
Merge branch '7179-test-mocks' refs #7179

9 years agoMerge branch 'master' into 7179-test-mocks
Tom Clegg [Thu, 10 Sep 2015 00:48:53 +0000 (20:48 -0400)]
Merge branch 'master' into 7179-test-mocks

Conflicts:
services/keepstore/volume.go
services/keepstore/volume_unix_test.go

9 years ago7179: Improve comments.
Tom Clegg [Wed, 9 Sep 2015 21:41:57 +0000 (17:41 -0400)]
7179: Improve comments.

9 years agoBrown bag typo bugfix. refs #7031
Peter Amstutz [Wed, 9 Sep 2015 20:39:37 +0000 (16:39 -0400)]
Brown bag typo bugfix.  refs #7031

9 years agoCatch errors thrown by sync_node() in Azure driver refs #7031.
Peter Amstutz [Wed, 9 Sep 2015 20:35:36 +0000 (16:35 -0400)]
Catch errors thrown by sync_node() in Azure driver refs #7031.

9 years ago6260: Fix up /status.json checks.
Tom Clegg [Wed, 9 Sep 2015 20:12:28 +0000 (16:12 -0400)]
6260: Fix up /status.json checks.

9 years ago6260: Simplify verifyBlocks logic.
Tom Clegg [Wed, 9 Sep 2015 19:09:46 +0000 (15:09 -0400)]
6260: Simplify verifyBlocks logic.

9 years ago6260: Remove test for impossible condition.
Tom Clegg [Wed, 9 Sep 2015 19:09:15 +0000 (15:09 -0400)]
6260: Remove test for impossible condition.

9 years ago6260: Fix up error messages.
Tom Clegg [Wed, 9 Sep 2015 19:09:05 +0000 (15:09 -0400)]
6260: Fix up error messages.

9 years ago6260: Remove extra hyphen from command line args.
Tom Clegg [Wed, 9 Sep 2015 19:08:31 +0000 (15:08 -0400)]
6260: Remove extra hyphen from command line args.

9 years ago6260: Just use the token loaded by the SDK (ARVADOS_API_TOKEN),
Tom Clegg [Wed, 9 Sep 2015 19:08:09 +0000 (15:08 -0400)]
6260: Just use the token loaded by the SDK (ARVADOS_API_TOKEN),
instead of requiring a token separately on the command line.

9 years ago7179: Improve description of how races must be handled.
Tom Clegg [Wed, 9 Sep 2015 16:32:30 +0000 (12:32 -0400)]
7179: Improve description of how races must be handled.

9 years agoBump node manager libcloud pin to dev3 refs #7031
Peter Amstutz [Wed, 9 Sep 2015 15:43:54 +0000 (11:43 -0400)]
Bump node manager libcloud pin to dev3 refs #7031

9 years agoMerge branch '7121-fix-deadlock' closes #7121
Tom Clegg [Wed, 9 Sep 2015 02:40:22 +0000 (22:40 -0400)]
Merge branch '7121-fix-deadlock' closes #7121

9 years ago6260: update datamanager.go to return error than Fatalf in some cases; add additional...
radhika [Wed, 9 Sep 2015 00:48:22 +0000 (20:48 -0400)]
6260: update datamanager.go to return error than Fatalf in some cases; add additional tests to test those errors raised.

9 years ago6260: Continue to force never_delete to be true until #6221 is resolved; in the meant...
radhika [Tue, 8 Sep 2015 22:18:19 +0000 (18:18 -0400)]
6260: Continue to force never_delete to be true until #6221 is resolved; in the meantime, let it be false during testing.

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Tue, 8 Sep 2015 21:30:48 +0000 (17:30 -0400)]
Merge branch 'master' into 6260-test-datamanager

9 years ago6260: look for two replicas
radhika [Tue, 8 Sep 2015 21:30:14 +0000 (17:30 -0400)]
6260: look for two replicas

9 years ago7179: Add Volume interface specs.
Tom Clegg [Tue, 8 Sep 2015 21:12:17 +0000 (17:12 -0400)]
7179: Add Volume interface specs.

9 years agoMerge branch '7193-provision-azure-compute' closes #7193
Peter Amstutz [Tue, 8 Sep 2015 19:35:02 +0000 (15:35 -0400)]
Merge branch '7193-provision-azure-compute' closes #7193

9 years ago7193: Tweak example config.
Peter Amstutz [Tue, 8 Sep 2015 19:23:06 +0000 (15:23 -0400)]
7193: Tweak example config.

9 years ago7225: ArvadosFile.flush() commits all underlying blocks.
Brett Smith [Tue, 8 Sep 2015 19:13:50 +0000 (15:13 -0400)]
7225: ArvadosFile.flush() commits all underlying blocks.

Fix a typo that caused only the last block to be flushed.
Refs #7225.

9 years ago7121: Add collisionOrCorrupt->DiskHashError test cases where the mismatched data...
Tom Clegg [Tue, 8 Sep 2015 18:04:14 +0000 (14:04 -0400)]
7121: Add collisionOrCorrupt->DiskHashError test cases where the mismatched data is in buf1.

9 years ago7170: Fix Ruby configure switch typo.
Brett Smith [Tue, 8 Sep 2015 17:56:19 +0000 (13:56 -0400)]
7170: Fix Ruby configure switch typo.

--no-install-rdoc should be spelled --disable-install-rdoc.
Refs #7170.

9 years agoMove arvados-api-server-upgrade.sh down in install guide.
Brett Smith [Tue, 8 Sep 2015 17:36:17 +0000 (13:36 -0400)]
Move arvados-api-server-upgrade.sh down in install guide.

Same rationale as 2a6a3720271a12bc8a47928ade5b2c641514ca44.
No issue #.

9 years ago7121: Add comment about why buffers must be returned.
Tom Clegg [Tue, 8 Sep 2015 17:31:04 +0000 (13:31 -0400)]
7121: Add comment about why buffers must be returned.

9 years ago6260: break old blocks into used and unused sets to make it easy to understand code...
radhika [Tue, 8 Sep 2015 17:23:00 +0000 (13:23 -0400)]
6260: break old blocks into used and unused sets to make it easy to understand code and some more updates.

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Tue, 8 Sep 2015 16:33:23 +0000 (12:33 -0400)]
Merge branch 'master' into 6260-test-datamanager

9 years ago7121: Fixup log messages (remove excess \n, show which volume had a Get() error).
Tom Clegg [Tue, 8 Sep 2015 15:32:47 +0000 (11:32 -0400)]
7121: Fixup log messages (remove excess \n, show which volume had a Get() error).

9 years ago7121: Log all errors (except the everyday "not found") encountered during CompareAndT...
Tom Clegg [Tue, 8 Sep 2015 15:23:52 +0000 (11:23 -0400)]
7121: Log all errors (except the everyday "not found") encountered during CompareAndTouch().

9 years ago7121: Clarify expected behavior of Compare() in Volume interface.
Tom Clegg [Tue, 8 Sep 2015 15:22:50 +0000 (11:22 -0400)]
7121: Clarify expected behavior of Compare() in Volume interface.

9 years ago7121: Rename mutex_test.go -> mock_mutex_for_test.go
Tom Clegg [Tue, 8 Sep 2015 15:16:44 +0000 (11:16 -0400)]
7121: Rename mutex_test.go -> mock_mutex_for_test.go

9 years ago7121: Update comment to match new argument name.
Tom Clegg [Tue, 8 Sep 2015 15:06:37 +0000 (11:06 -0400)]
7121: Update comment to match new argument name.

9 years ago7121: 7218: Get the serialize lock before opening a file.
Tom Clegg [Tue, 8 Sep 2015 15:05:45 +0000 (11:05 -0400)]
7121: 7218: Get the serialize lock before opening a file.

In certain error cases (Stat() succeeds but Open() fails) this can
cause a failure to wait behind the serialize lock, but in normal
operation ("success" and "not found") it conserves open file handles
when there's a lot of contention for the serialize lock.

9 years ago7121: Test mutex usage with a mock instead of time.Sleep.
Tom Clegg [Thu, 3 Sep 2015 19:15:32 +0000 (15:15 -0400)]
7121: Test mutex usage with a mock instead of time.Sleep.

9 years ago7179: Start a set of generic volume tests.
Tom Clegg [Tue, 8 Sep 2015 04:38:41 +0000 (00:38 -0400)]
7179: Start a set of generic volume tests.

9 years ago7179: Add TestableVolume interface, make UnixVolume tests a bit more reusable.
Tom Clegg [Tue, 8 Sep 2015 04:08:14 +0000 (00:08 -0400)]
7179: Add TestableVolume interface, make UnixVolume tests a bit more reusable.

9 years ago7121: Return DiskHashError instead of CollisionError from Compare() where appropriate.
Tom Clegg [Thu, 3 Sep 2015 17:42:32 +0000 (13:42 -0400)]
7121: Return DiskHashError instead of CollisionError from Compare() where appropriate.

9 years ago7121: Replace Get(loc,true) with CompareAndTouch(). Add Compare method to Volume...
Tom Clegg [Thu, 3 Sep 2015 04:54:52 +0000 (00:54 -0400)]
7121: Replace Get(loc,true) with CompareAndTouch(). Add Compare method to Volume, UnixVolume, MockVolume.

9 years ago7121: Add test case to demonstrate deadlock.
Tom Clegg [Thu, 3 Sep 2015 03:19:23 +0000 (23:19 -0400)]
7121: Add test case to demonstrate deadlock.

9 years agoMerge branch '6263-gitolite-test' refs #6263
Tom Clegg [Mon, 7 Sep 2015 18:10:30 +0000 (14:10 -0400)]
Merge branch '6263-gitolite-test' refs #6263

9 years agoMerge branch 'wtsi-hgi-fix/7205-arv-edit-tty-check'
Tom Clegg [Mon, 7 Sep 2015 17:56:48 +0000 (13:56 -0400)]
Merge branch 'wtsi-hgi-fix/7205-arv-edit-tty-check'

refs #7205

9 years agoUpdate docs to accomodate URL change for the Arvados redmine
Ward Vandewege [Sat, 5 Sep 2015 02:37:05 +0000 (22:37 -0400)]
Update docs to accomodate URL change for the Arvados redmine
installation. It now lives at dev.arvados.org.

refs #7187

9 years ago6260: add check in datamanager to ensure that the configured datamanager token belong...
radhika [Fri, 4 Sep 2015 17:31:45 +0000 (13:31 -0400)]
6260: add check in datamanager to ensure that the configured datamanager token belongs to an admin user.

9 years ago7193: Use tags instead of _make_ping_url
Peter Amstutz [Fri, 4 Sep 2015 15:06:17 +0000 (11:06 -0400)]
7193: Use tags instead of _make_ping_url

9 years agoFix switch typo in arv-mount tutorial docs.
Brett Smith [Fri, 4 Sep 2015 14:53:58 +0000 (10:53 -0400)]
Fix switch typo in arv-mount tutorial docs.

No issue #.

9 years ago7193: Move ex_run_command to post_create_node()
Peter Amstutz [Fri, 4 Sep 2015 14:45:34 +0000 (10:45 -0400)]
7193: Move ex_run_command to post_create_node()

9 years ago7193: Bump dependency to dev2
Peter Amstutz [Fri, 4 Sep 2015 13:58:36 +0000 (09:58 -0400)]
7193: Bump dependency to dev2

9 years ago7193: Set timestamp on ex_run_command to ensure that the provisioning script runs.
Peter Amstutz [Fri, 4 Sep 2015 13:32:55 +0000 (09:32 -0400)]
7193: Set timestamp on ex_run_command to ensure that the provisioning script runs.

9 years agoMoves printing of `arv edit` messages out-of-band
Joshua Randall [Fri, 4 Sep 2015 01:33:20 +0000 (02:33 +0100)]
Moves printing of `arv edit` messages out-of-band

Messages regarding update of object after edit are now
printed out-of-band to STDERR rather than to STDOUT
where they might mix with non-interactive "editors"

9 years agoRemoves unnecessary check that stdout is a TTY
Joshua Randall [Fri, 4 Sep 2015 01:31:38 +0000 (02:31 +0100)]
Removes unnecessary check that stdout is a TTY

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Thu, 3 Sep 2015 19:53:07 +0000 (15:53 -0400)]
Merge branch 'master' into 6260-test-datamanager

9 years ago6260: TestPutAndGetBlocks is working.
radhika [Thu, 3 Sep 2015 19:39:33 +0000 (15:39 -0400)]
6260: TestPutAndGetBlocks is working.

9 years ago6260: do not force never_delete in keepstore
radhika [Thu, 3 Sep 2015 15:26:12 +0000 (11:26 -0400)]
6260: do not force never_delete in keepstore

9 years ago7193: Run provision script after creating node, not in sync_node, because
Peter Amstutz [Thu, 3 Sep 2015 14:40:34 +0000 (10:40 -0400)]
7193: Run provision script after creating node, not in sync_node, because
sync_node doesn't get called until the node has pinged!

9 years agoarv-copy: Make the help text for the --src and --dst options more easily
Ward Vandewege [Thu, 3 Sep 2015 00:17:29 +0000 (20:17 -0400)]
arv-copy: Make the help text for the --src and --dst options more easily
intelligible.

No issue #