arvados.git
9 years agorefs #6859
radhika [Tue, 29 Sep 2015 16:13:04 +0000 (12:13 -0400)]
refs #6859
Added include statement to the script.

9 years ago6034: added scafolding of the first initscript
Nico Cesar [Mon, 28 Sep 2015 18:54:33 +0000 (14:54 -0400)]
6034: added scafolding of the first initscript

refs #6034

9 years agoRuby 2.1.7 is out, update documentation accordingly.
Ward Vandewege [Mon, 28 Sep 2015 14:29:56 +0000 (10:29 -0400)]
Ruby 2.1.7 is out, update documentation accordingly.

No issue #

9 years agoMerge branch 'pr/29'
Brett Smith [Mon, 28 Sep 2015 13:53:20 +0000 (09:53 -0400)]
Merge branch 'pr/29'

Closes #7388.

9 years agoChange ssh public key to 700 after key-sync
LunarShaddow [Mon, 28 Sep 2015 02:48:03 +0000 (10:48 +0800)]
Change ssh public key to 700 after key-sync

9 years agocloses #7161
radhika [Fri, 25 Sep 2015 17:27:24 +0000 (13:27 -0400)]
closes #7161
Merge branch '7161-support-all-service-types'

9 years agoMerge branch '7161-support-all-service-types' of git.curoverse.com:arvados into 7161...
radhika [Fri, 25 Sep 2015 16:44:15 +0000 (12:44 -0400)]
Merge branch '7161-support-all-service-types' of git.curoverse.com:arvados into 7161-support-all-service-types

9 years ago7161: Use default value of 1 for max_replicas_per_service and set it to None when...
radhika [Fri, 25 Sep 2015 14:13:49 +0000 (10:13 -0400)]
7161: Use default value of 1 for max_replicas_per_service and set it to None when writable non-disk type services are found.

9 years ago7161: white space
radhika [Thu, 24 Sep 2015 18:57:40 +0000 (14:57 -0400)]
7161: white space

9 years ago7161: clarify max_replicas_per_service
radhika [Thu, 24 Sep 2015 18:52:39 +0000 (14:52 -0400)]
7161: clarify max_replicas_per_service

9 years ago7161: Look for all service hints, not just keep proxy. Also, use thread count of...
radhika [Wed, 23 Sep 2015 19:32:44 +0000 (15:32 -0400)]
7161: Look for all service hints, not just keep proxy. Also, use thread count of 1 when it not a disk.

9 years agocloses #7162
radhika [Fri, 25 Sep 2015 16:15:36 +0000 (12:15 -0400)]
closes #7162
Merge branch '7162-support-service-types'

9 years ago7162: Removed the call to get keep_disks when there is an error getting keep_services,
radhika [Fri, 25 Sep 2015 16:13:25 +0000 (12:13 -0400)]
7162: Removed the call to get keep_disks when there is an error getting keep_services,
since keep_services has been available since May 2014, #2776 according to Tom.

9 years agoMerge branch 'master' into 7162-support-service-types
radhika [Fri, 25 Sep 2015 16:05:44 +0000 (12:05 -0400)]
Merge branch 'master' into 7162-support-service-types

9 years agoMerge branch '7227-crunch-job-stricter-unmount-wip'
Brett Smith [Fri, 25 Sep 2015 14:46:00 +0000 (10:46 -0400)]
Merge branch '7227-crunch-job-stricter-unmount-wip'

Closes #7227, #7376.

9 years ago7227: crunch-job only unmounts Keep mounts under $CRUNCH_TMP.
Brett Smith [Wed, 23 Sep 2015 17:16:41 +0000 (13:16 -0400)]
7227: crunch-job only unmounts Keep mounts under $CRUNCH_TMP.

This prevents it from trying to unmount other Keep mounts elsewhere on
the system, and considering the cleanup step failed because of that.

9 years agoMerge branch '6967-application-yml-without-git-wip'
Brett Smith [Fri, 25 Sep 2015 14:38:37 +0000 (10:38 -0400)]
Merge branch '6967-application-yml-without-git-wip'

Closes #6967, #7375.

9 years ago6967: Extend application.default.yml source_version detection.
Brett Smith [Wed, 23 Sep 2015 15:38:06 +0000 (11:38 -0400)]
6967: Extend application.default.yml source_version detection.

* In production, get source_version from our package's
  git-commit.version, assuming it's available.
* When we get source information from Git, suppress errors from the
  user, and only use the information if all queries succeed.

9 years ago6967: Remove trailing whitespace in Workbench's application.default.yml.
Brett Smith [Wed, 23 Sep 2015 15:34:53 +0000 (11:34 -0400)]
6967: Remove trailing whitespace in Workbench's application.default.yml.

9 years ago7161: working
radhika [Fri, 25 Sep 2015 14:13:49 +0000 (10:13 -0400)]
7161: working

9 years agoMerge branch 'master' into 7161-support-all-service-types
radhika [Fri, 25 Sep 2015 12:32:46 +0000 (08:32 -0400)]
Merge branch 'master' into 7161-support-all-service-types

9 years agoMerge branch 'master' into 7162-support-service-types
radhika [Fri, 25 Sep 2015 12:29:59 +0000 (08:29 -0400)]
Merge branch 'master' into 7162-support-service-types

9 years ago7162: updated comment
radhika [Fri, 25 Sep 2015 12:28:04 +0000 (08:28 -0400)]
7162: updated comment

9 years agoMerge branch '7181-crunch-missing-git' closes #7181
Peter Amstutz [Thu, 24 Sep 2015 20:36:59 +0000 (16:36 -0400)]
Merge branch '7181-crunch-missing-git' closes #7181

9 years agoMerge branch '7228-crunch-dispatch-umask' closes #7228
Peter Amstutz [Thu, 24 Sep 2015 20:36:35 +0000 (16:36 -0400)]
Merge branch '7228-crunch-dispatch-umask' closes #7228

9 years ago7162: move replicasPerThread logic outside the for loop.
radhika [Thu, 24 Sep 2015 20:25:37 +0000 (16:25 -0400)]
7162: move replicasPerThread logic outside the for loop.

9 years ago7181: Check for [Errno 2] in the output and assume that means git is missing.
Peter Amstutz [Thu, 24 Sep 2015 19:53:16 +0000 (15:53 -0400)]
7181: Check for [Errno 2] in the output and assume that means git is missing.
Remove --quite to get more output if egg_info fails.  Use "warn" instead of
"print" because "print" doesn't seem to actually make it to the logs.

9 years ago7162: SetServiceRoots keepproxy hints only (revert my update).
radhika [Thu, 24 Sep 2015 19:22:15 +0000 (15:22 -0400)]
7162: SetServiceRoots keepproxy hints only (revert my update).

9 years ago7161: white space
radhika [Thu, 24 Sep 2015 18:57:40 +0000 (14:57 -0400)]
7161: white space

9 years ago7161: clarify max_replicas_per_service
radhika [Thu, 24 Sep 2015 18:52:39 +0000 (14:52 -0400)]
7161: clarify max_replicas_per_service

9 years agoMerge branch 'master' into 7162-support-service-types
radhika [Thu, 24 Sep 2015 15:42:58 +0000 (11:42 -0400)]
Merge branch 'master' into 7162-support-service-types

9 years agocloses #7329
radhika [Wed, 23 Sep 2015 21:03:33 +0000 (17:03 -0400)]
closes #7329
Merge branch '7329-empty-block-issue'

9 years ago7321: arvados/jobs Dockerfile installs python-pip.
Brett Smith [Wed, 23 Sep 2015 20:56:35 +0000 (16:56 -0400)]
7321: arvados/jobs Dockerfile installs python-pip.

This is also for arvados_sdk_version support.  I thought pip was a
dependency of virtualenv, but it's not.  Refs #7321.

9 years agoMerge branch 'master' into 7329-empty-block-issue
radhika [Wed, 23 Sep 2015 20:31:37 +0000 (16:31 -0400)]
Merge branch 'master' into 7329-empty-block-issue

9 years ago7329: improved TestableVolumeManagerFactory method signature, teardown logic, comment...
radhika [Wed, 23 Sep 2015 20:29:02 +0000 (16:29 -0400)]
7329: improved TestableVolumeManagerFactory method signature, teardown logic, comments and test file name.

9 years ago7321: arvados/jobs Dockerfile installs development headers.
Brett Smith [Wed, 23 Sep 2015 20:27:28 +0000 (16:27 -0400)]
7321: arvados/jobs Dockerfile installs development headers.

These are necessary to build all of the Python SDK's dependencies when
running a job that defines arvados_sdk_version.

Closes #7321.

9 years agoMerge branch 'master' into 7329-empty-block-issue
radhika [Wed, 23 Sep 2015 19:44:09 +0000 (15:44 -0400)]
Merge branch 'master' into 7329-empty-block-issue

9 years ago7161: Look for all service hints, not just keep proxy. Also, use thread count of...
radhika [Wed, 23 Sep 2015 19:32:44 +0000 (15:32 -0400)]
7161: Look for all service hints, not just keep proxy. Also, use thread count of 1 when it not a disk.

9 years ago7228: Set umask to 002 so cache files are created writable other processes with same...
Peter Amstutz [Wed, 23 Sep 2015 17:00:08 +0000 (13:00 -0400)]
7228: Set umask to 002 so cache files are created writable other processes with same GID.

9 years agoMerge branch '7225-collection-hang' closes #7225
Peter Amstutz [Wed, 23 Sep 2015 14:42:38 +0000 (10:42 -0400)]
Merge branch '7225-collection-hang' closes #7225

9 years ago7329: since moved functional testing into generic testing, removing now redundant...
radhika [Tue, 22 Sep 2015 22:25:53 +0000 (18:25 -0400)]
7329: since moved functional testing into generic testing, removing now redundant empty block tests from volume_unix_test.go and keepstore_test.go.

9 years ago7329: add generic volume based keepstore tests to have better functional test coverage.
radhika [Tue, 22 Sep 2015 22:16:15 +0000 (18:16 -0400)]
7329: add generic volume based keepstore tests to have better functional test coverage.

9 years ago7225: Better exception handling for unexpected state changes.
Peter Amstutz [Tue, 22 Sep 2015 20:58:21 +0000 (16:58 -0400)]
7225: Better exception handling for unexpected state changes.

9 years ago7225: Add test to validate fix that 2nd commit call blocks until block is committed
Peter Amstutz [Tue, 22 Sep 2015 19:23:31 +0000 (15:23 -0400)]
7225: Add test to validate fix that 2nd commit call blocks until block is committed
as intended.

9 years ago7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
Peter Amstutz [Tue, 22 Sep 2015 15:47:25 +0000 (11:47 -0400)]
7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
block commit" code getting skipped.  Set PUT thread to daemon thread; users
must call save() or flush() or use `with` to guarantee that blocks get written.

9 years agoMerge branch 'master' into 7329-empty-block-issue
radhika [Tue, 22 Sep 2015 16:47:26 +0000 (12:47 -0400)]
Merge branch 'master' into 7329-empty-block-issue

9 years ago7329: Fix infinite loop in Compare when reading an empty file.
Tom Clegg [Tue, 22 Sep 2015 16:01:25 +0000 (12:01 -0400)]
7329: Fix infinite loop in Compare when reading an empty file.

refs #7329

9 years ago7162: Update code to address the bug "current code forgets to reset Using_proxy to...
radhika [Tue, 22 Sep 2015 15:40:44 +0000 (11:40 -0400)]
7162: Update code to address the bug "current code forgets to reset Using_proxy to false if the API server stops giving out proxies between one call and the next"

9 years agoMerge branch 'master' into 7162-support-service-types
radhika [Tue, 22 Sep 2015 15:37:43 +0000 (11:37 -0400)]
Merge branch 'master' into 7162-support-service-types

9 years ago7162: Update datamanager code to ask for all non-proxy typed keep services and fail...
radhika [Tue, 22 Sep 2015 14:38:26 +0000 (10:38 -0400)]
7162: Update datamanager code to ask for all non-proxy typed keep services and fail if any non-disk types are found in the returned list.
This needs to be enhanced when we are ready to support keep servers with other service types sub as blob storage.

9 years agoMerge branch '7309-git-push-default' closes #7309
Tom Clegg [Tue, 22 Sep 2015 14:14:20 +0000 (10:14 -0400)]
Merge branch '7309-git-push-default' closes #7309

9 years ago7309: Push branches explicitly in tests instead of relying on `git config push.default`
Tom Clegg [Tue, 22 Sep 2015 14:14:06 +0000 (10:14 -0400)]
7309: Push branches explicitly in tests instead of relying on `git config push.default`

9 years ago7329: Add generic tests for empty block handling.
Tom Clegg [Tue, 22 Sep 2015 03:22:00 +0000 (03:22 +0000)]
7329: Add generic tests for empty block handling.

9 years ago7329: Fix infinite loop in Compare when reading an empty file (whether
Tom Clegg [Mon, 21 Sep 2015 22:39:49 +0000 (22:39 +0000)]
7329: Fix infinite loop in Compare when reading an empty file (whether
or not the buf we're comparing is also empty).

9 years agoMerge branch 'master' into 7162-support-service-types
radhika [Mon, 21 Sep 2015 21:39:48 +0000 (17:39 -0400)]
Merge branch 'master' into 7162-support-service-types

9 years ago7162: Use replicasPerService to avoid sending redundant replication requests.
radhika [Mon, 21 Sep 2015 21:36:03 +0000 (17:36 -0400)]
7162: Use replicasPerService to avoid sending redundant replication requests.

9 years agoAPI server limits KeepService changes to admins.
Brett Smith [Mon, 21 Sep 2015 18:56:43 +0000 (14:56 -0400)]
API server limits KeepService changes to admins.

Taking the same security policy that applies to disks and applying
them to services as well.  No issue #.

9 years ago7162: enhance SetServiceRoots to look for all service hints, not just K@ hints.
radhika [Mon, 21 Sep 2015 15:08:07 +0000 (11:08 -0400)]
7162: enhance SetServiceRoots to look for all service hints, not just K@ hints.

9 years ago7329: It appears that UnixVolume -> Compare method is falling in infinite loop due...
radhika [Thu, 17 Sep 2015 19:03:59 +0000 (15:03 -0400)]
7329: It appears that UnixVolume -> Compare method is falling in infinite loop due to the fact that EOF is not returned when reading an empty file.
Due to this, PutBlock for an EmptyHash is resulting in infinite loop when an EmptyBlock already exists in Keep.
Added a quick fix in the Compare method for unix volume implementation. We can try to see why EOF is not returned and put in a better fix,
but for the time being this helps with unblocking tests.

Added several tests for empty block. Most importantly, added an keep integration test with UnixVolume so that we have integration tests covering some
of these code paths. More tests are definitely called for in this intergration test for better code coverage.

Also did gofmt and golint updates.

9 years agoMerge branch 'pr/24'
Brett Smith [Wed, 16 Sep 2015 21:30:37 +0000 (17:30 -0400)]
Merge branch 'pr/24'

Closes #7262.

9 years agorefs #7015
radhika [Tue, 15 Sep 2015 20:34:44 +0000 (16:34 -0400)]
refs #7015
Merge branch '7015-remove-pipeline-instance-from-recursive-arv-copy-doc'

9 years ago7015: correct typo in copied template uuid
radhika [Tue, 15 Sep 2015 20:32:44 +0000 (16:32 -0400)]
7015: correct typo in copied template uuid

9 years ago7015: Update arv-copy user guide to use a pipeline template instead of instance in...
radhika [Tue, 15 Sep 2015 19:53:16 +0000 (15:53 -0400)]
7015: Update arv-copy user guide to use a pipeline template instead of instance in the examples.
In addition, until 7001 is resolved, do not include pipeline instance as one of the object types that can be copied over.

9 years agocloses #6260
radhika [Tue, 15 Sep 2015 17:41:42 +0000 (13:41 -0400)]
closes #6260
Merge branch '6260-test-datamanager'

9 years ago6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead...
radhika [Tue, 15 Sep 2015 17:16:38 +0000 (13:16 -0400)]
6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead of locator+size only.

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Tue, 15 Sep 2015 16:24:30 +0000 (12:24 -0400)]
Merge branch 'master' into 6260-test-datamanager

Conflicts:
services/keepstore/keepstore.go

9 years agorefs #7179
radhika [Tue, 15 Sep 2015 16:18:42 +0000 (12:18 -0400)]
refs #7179
Merge branch '7179-generic-volume-tests'

9 years ago7179: more updates to address golint suggestions around the config parameters such...
radhika [Tue, 15 Sep 2015 16:16:31 +0000 (12:16 -0400)]
7179: more updates to address golint suggestions around the config parameters such as never-delete, blob-signature-ttl etc.

9 years agodocker builds are failing for compute nodes.
Nico Cesar [Tue, 15 Sep 2015 14:53:46 +0000 (10:53 -0400)]
docker builds are failing for compute nodes.

no issue #

9 years ago7262: blacklists pbr >= 1.7.0
Joshua C. Randall [Tue, 15 Sep 2015 10:58:15 +0000 (10:58 +0000)]
7262: blacklists pbr >= 1.7.0

9 years ago6260: most golint suggestions are addressed.
radhika [Tue, 15 Sep 2015 00:16:32 +0000 (20:16 -0400)]
6260: most golint suggestions are addressed.

9 years ago6260: so many golint complaints!!
radhika [Mon, 14 Sep 2015 22:51:45 +0000 (18:51 -0400)]
6260: so many golint complaints!!

9 years ago6260: golint suggested updates
radhika [Mon, 14 Sep 2015 19:23:56 +0000 (15:23 -0400)]
6260: golint suggested updates

9 years agoMerge branch 'master' into 6260-test-datamanager
radhika [Mon, 14 Sep 2015 19:23:21 +0000 (15:23 -0400)]
Merge branch 'master' into 6260-test-datamanager

Conflicts:
services/keepstore/keepstore.go

9 years agocloses #7179
radhika [Mon, 14 Sep 2015 19:11:47 +0000 (15:11 -0400)]
closes #7179
Merge branch '7179-generic-volume-tests'

9 years ago6260: golint suggestions incorporated. Also, the earlier commit log message "no longe...
radhika [Mon, 14 Sep 2015 15:06:44 +0000 (11:06 -0400)]
6260: golint suggestions incorporated. Also, the earlier commit log message "no longer need the keep.data-manager-token-file" is incorrect. Keepstore still needs this and this file is still generated during testing.

9 years ago7179: Most of golint suggested updates are made. Some names such as "never_delete...
radhika [Mon, 14 Sep 2015 14:31:47 +0000 (10:31 -0400)]
7179: Most of golint suggested updates are made. Some names such as "never_delete" are left alone.
Also suggestions such as "replace errors.New(fmt.Sprintf(...)) with fmt.Errorf(...)" are ignored.

9 years ago7179: some more golint suggested updates
radhika [Mon, 14 Sep 2015 13:33:13 +0000 (09:33 -0400)]
7179: some more golint suggested updates

9 years ago7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use...
radhika [Sat, 12 Sep 2015 15:02:22 +0000 (11:02 -0400)]
7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use CamelCase".
Now keepstore.go and handlers.go are almost lint free; a few such as blob_signature_ttl are not renamed at this time.

9 years ago7179: handler_test.go is now lint free according to golint.
radhika [Sat, 12 Sep 2015 14:06:23 +0000 (10:06 -0400)]
7179: handler_test.go is now lint free according to golint.

9 years ago7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes...
radhika [Sat, 12 Sep 2015 13:32:29 +0000 (09:32 -0400)]
7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes, update each test to either support
both read-only and writable modes, or skip a test if it is indended for the other mode. This helps ensure as many
tests as possible are executed in either case.

9 years ago7179: A few golint suggested updates. There are still a lot of golint complaints.
radhika [Sat, 12 Sep 2015 00:33:50 +0000 (20:33 -0400)]
7179: A few golint suggested updates. There are still a lot of golint complaints.

9 years ago7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.
radhika [Fri, 11 Sep 2015 23:18:27 +0000 (19:18 -0400)]
7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.

9 years agoMerge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-gener...
radhika [Fri, 11 Sep 2015 20:46:07 +0000 (16:46 -0400)]
Merge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-generic-volume-tests

9 years ago7179: Tighten Put requirements when overwriting existing data.
Tom Clegg [Fri, 11 Sep 2015 20:13:53 +0000 (16:13 -0400)]
7179: Tighten Put requirements when overwriting existing data.

9 years agoMerge branch 'master' into 7179-generic-volume-tests
radhika [Fri, 11 Sep 2015 19:19:21 +0000 (15:19 -0400)]
Merge branch 'master' into 7179-generic-volume-tests

9 years agoMerge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310
Tom Clegg [Fri, 11 Sep 2015 18:41:28 +0000 (14:41 -0400)]
Merge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310

9 years ago7179: add generic volume test; as part of this exercise, moved some "generic" tests...
radhika [Fri, 11 Sep 2015 15:59:05 +0000 (11:59 -0400)]
7179: add generic volume test; as part of this exercise, moved some "generic" tests from volume_unix_test.go
into volume_generic_test.go so that we do not repeat these in all storage device related testing.

9 years ago7310: adds git v2 style regex to match http 500 error
Joshua C. Randall [Fri, 11 Sep 2015 15:08:44 +0000 (15:08 +0000)]
7310: adds git v2 style regex to match http 500 error

9 years ago6260: Reduced-replication test requires replication>=desired, not replication==len...
Tom Clegg [Thu, 10 Sep 2015 15:17:36 +0000 (11:17 -0400)]
6260: Reduced-replication test requires replication>=desired, not replication==len(keepServers).

9 years ago7179: list out the potential tests
radhika [Thu, 10 Sep 2015 14:59:48 +0000 (10:59 -0400)]
7179: list out the potential tests

9 years ago6260: Remove unnecessary Sleep()
Tom Clegg [Thu, 10 Sep 2015 14:54:45 +0000 (10:54 -0400)]
6260: Remove unnecessary Sleep()

9 years agoPuts pbr 1.7.0 on blacklist for services/dockercleaner
Joshua C. Randall [Thu, 10 Sep 2015 13:31:16 +0000 (14:31 +0100)]
Puts pbr 1.7.0 on blacklist for services/dockercleaner

9 years agoBlacklists pbr version 1.7.0 as it is not working with mock 1.3.0
Joshua C. Randall [Thu, 10 Sep 2015 11:37:25 +0000 (11:37 +0000)]
Blacklists pbr version 1.7.0 as it is not working with mock 1.3.0

9 years ago6260: no longer need the keep.data-manager-token-file
radhika [Thu, 10 Sep 2015 01:14:37 +0000 (21:14 -0400)]
6260: no longer need the keep.data-manager-token-file

9 years agoMerge branch '7179-test-mocks' refs #7179
Tom Clegg [Thu, 10 Sep 2015 00:49:30 +0000 (20:49 -0400)]
Merge branch '7179-test-mocks' refs #7179

9 years agoMerge branch 'master' into 7179-test-mocks
Tom Clegg [Thu, 10 Sep 2015 00:48:53 +0000 (20:48 -0400)]
Merge branch 'master' into 7179-test-mocks

Conflicts:
services/keepstore/volume.go
services/keepstore/volume_unix_test.go

9 years ago7179: Improve comments.
Tom Clegg [Wed, 9 Sep 2015 21:41:57 +0000 (17:41 -0400)]
7179: Improve comments.

9 years agoBrown bag typo bugfix. refs #7031
Peter Amstutz [Wed, 9 Sep 2015 20:39:37 +0000 (16:39 -0400)]
Brown bag typo bugfix.  refs #7031

9 years agoCatch errors thrown by sync_node() in Azure driver refs #7031.
Peter Amstutz [Wed, 9 Sep 2015 20:35:36 +0000 (16:35 -0400)]
Catch errors thrown by sync_node() in Azure driver refs #7031.