Tom Clegg [Sat, 5 Dec 2015 00:18:49 +0000 (19:18 -0500)]
Fix manifest stats leaking between runs via globals. No issue #
Brett Smith [Fri, 4 Dec 2015 21:46:38 +0000 (16:46 -0500)]
Merge branch '7835-node-manager-price-type-wip'
Refs #7835. Closes #7950.
Brett Smith [Fri, 4 Dec 2015 20:13:49 +0000 (15:13 -0500)]
7835: Node Manager converts node.size into a size object on GCE.
Brett Smith [Fri, 4 Dec 2015 19:37:06 +0000 (14:37 -0500)]
Remove redundant exception from Node Manager NETWORK_ERRORS.
Brett Smith [Fri, 4 Dec 2015 19:35:30 +0000 (14:35 -0500)]
7835: Node Manager coerces node size prices to float.
radhika [Fri, 4 Dec 2015 21:00:55 +0000 (16:00 -0500)]
closes #7753
Merge branch '7753-job-log-accessibility'
radhika [Fri, 4 Dec 2015 18:48:48 +0000 (13:48 -0500)]
7753: show the job Log link in the component panel header only when it is a pipeline_instance display; hide it for job#show
radhika [Fri, 4 Dec 2015 15:56:19 +0000 (10:56 -0500)]
7753: when a job is in "Running" state, the Log link still should be shown in the Component display panel header.
radhika [Fri, 4 Dec 2015 14:49:22 +0000 (09:49 -0500)]
7753: Now that there are 1+ "Log" links in the pipeline instance page, update tests to address capybara ambigous match error.
radhika [Fri, 4 Dec 2015 14:19:33 +0000 (09:19 -0500)]
7753: The name "Job log" has a little problem; on screen small screen sizes, "Job log" is displayed in two lines even when the rest of the items are displayed in single line due to the limited real estate in the panel header.
radhika [Thu, 3 Dec 2015 21:00:52 +0000 (16:00 -0500)]
7753: Add a Log link to pipeline_instance component panel
Nico Cesar [Fri, 4 Dec 2015 15:32:23 +0000 (10:32 -0500)]
7942: changed capybara timeout from 5 sec to 10 sec so iceweasel in ci.curoverse.com doesnt break as easily
Merge branch '7942-ci-iceweasel-problems'
refs #7942
Nico Cesar [Thu, 3 Dec 2015 22:13:31 +0000 (17:13 -0500)]
added more time to ajax timeout
refs #7942
Tom Clegg [Thu, 3 Dec 2015 19:26:19 +0000 (14:26 -0500)]
Merge branch '7888-log-req-phase-times' refs #7888
Tom Clegg [Thu, 3 Dec 2015 17:29:09 +0000 (12:29 -0500)]
7888: Log how much time was spent before/after sending response headers.
Tom Clegg [Thu, 3 Dec 2015 17:03:16 +0000 (12:03 -0500)]
Merge branch '7751-crunch-fuse-output' closes #7751
Tom Clegg [Fri, 27 Nov 2015 02:11:16 +0000 (21:11 -0500)]
7751: Add convenience class for staging task output in $TASK_KEEPMOUNT_TMP.
Tom Clegg [Fri, 27 Nov 2015 01:52:10 +0000 (20:52 -0500)]
7751: Set up an arv-mount scratch directory for each task, and put its path in TASK_KEEPMOUNT_TMP.
radhika [Wed, 2 Dec 2015 22:44:47 +0000 (17:44 -0500)]
closes #6027
Merge branch '6027-git-repo-doc'
radhika [Wed, 2 Dec 2015 22:43:37 +0000 (17:43 -0500)]
6027: fix typo
radhika [Wed, 2 Dec 2015 22:15:12 +0000 (17:15 -0500)]
Merge branch 'master' into 6027-git-repo-doc
Tom Clegg [Wed, 2 Dec 2015 21:56:40 +0000 (16:56 -0500)]
Merge branch '7868-docker-run-args' closes #7868
Tom Clegg [Wed, 25 Nov 2015 22:53:14 +0000 (17:53 -0500)]
7868: Allow admin to add arguments to "docker run" commands.
radhika [Wed, 2 Dec 2015 19:57:52 +0000 (14:57 -0500)]
6027: update git-arvados-guide to instruct the users to create scripts in crunch_scripts subdirectory.
Brett Smith [Wed, 2 Dec 2015 17:54:56 +0000 (12:54 -0500)]
7313: Fix sprintf formatter in crunch-job warning message.
Refs #7313.
radhika [Wed, 2 Dec 2015 17:37:15 +0000 (12:37 -0500)]
closes #7748
Merge branch '7748-datamanager-dry-run'
radhika [Wed, 2 Dec 2015 17:36:12 +0000 (12:36 -0500)]
7748: correct the local variable declaration
radhika [Wed, 2 Dec 2015 17:30:10 +0000 (12:30 -0500)]
Merge branch 'master' into 7748-datamanager-dry-run
Conflicts:
services/datamanager/datamanager_test.go
radhika [Wed, 2 Dec 2015 17:22:00 +0000 (12:22 -0500)]
closes #7710
Merge branch '7710-keepclient-all-service-types'
radhika [Wed, 2 Dec 2015 16:58:39 +0000 (11:58 -0500)]
7710: text updates around the -service-type argument.
radhika [Tue, 1 Dec 2015 18:33:55 +0000 (13:33 -0500)]
7710: update crunchrunner.upload.go to compile; this was broken after the keep-web updates; also, add this package to gostuff in run-tests.sh
radhika [Tue, 1 Dec 2015 14:26:13 +0000 (09:26 -0500)]
7710: update serviceType argument handling to ignore any keepservices that are of a different type. Add a datamanager test with one extra unsupported-typed keepserver, which should hence be ignored by datamanager.
radhika [Mon, 30 Nov 2015 23:09:57 +0000 (18:09 -0500)]
7710: Remove KeepClient.Using_proxy and all it's references since we are no longer using it.
Update datamanager to use a configured supported service type instead of hardcoding "disk" type.
Tom Clegg [Mon, 30 Nov 2015 20:28:19 +0000 (15:28 -0500)]
7710: Add keep_services#accessible test.
radhika [Fri, 27 Nov 2015 22:13:17 +0000 (17:13 -0500)]
7710: improve keepclient to use non-disk timeouts when any non-disk typed keepservices are found; previously, this was done for proxy typed keepservices only.
radhika [Wed, 2 Dec 2015 16:18:51 +0000 (11:18 -0500)]
7748: use local variables for arvLogger.Update
radhika [Wed, 2 Dec 2015 14:52:27 +0000 (09:52 -0500)]
Merge branch 'master' into 7748-datamanager-dry-run
Tom Clegg [Tue, 1 Dec 2015 21:46:03 +0000 (16:46 -0500)]
Merge branch '7697-socket-retry' closes #7697
Tom Clegg [Tue, 1 Dec 2015 20:48:22 +0000 (15:48 -0500)]
7697: Avoid reusing long-idle HTTP connections. Avoid retrying non-idempotent operations.
Brett Smith [Tue, 1 Dec 2015 21:21:43 +0000 (16:21 -0500)]
Merge branch '7870-crunch-dispatch-retry-fail-lock-wip'
Closes #7870, #7877.
Brett Smith [Tue, 1 Dec 2015 15:38:09 +0000 (10:38 -0500)]
7870: Teach crunch-dispatch to fail jobs it already locked.
The fail_job method has been written with the assumption that the job
should be unlocked. The retry support added in #4410 breaks this
assumption. Teach fail_job to acquire the lock only when we don't
know we already have it.
Peter Amstutz [Tue, 1 Dec 2015 17:03:26 +0000 (12:03 -0500)]
Merge branch '6309-file-cache-runtime-constraint' closes #6309
radhika [Tue, 1 Dec 2015 15:41:38 +0000 (10:41 -0500)]
7748: when dryRun continue and leave the regular flow untouched.
Peter Amstutz [Tue, 1 Dec 2015 14:26:59 +0000 (09:26 -0500)]
Merge branch 'master' into 6309-file-cache-runtime-constraint
Peter Amstutz [Tue, 1 Dec 2015 14:26:02 +0000 (09:26 -0500)]
6309: Use arv-mount default cache size. Fix documentation typo.
radhika [Mon, 30 Nov 2015 21:46:36 +0000 (16:46 -0500)]
7748: address dryRun in summary.WritePullLists and keep.SendTrashLists
radhika [Mon, 30 Nov 2015 15:40:29 +0000 (10:40 -0500)]
Merge branch 'master' into 7748-datamanager-dry-run
radhika [Mon, 30 Nov 2015 15:27:47 +0000 (10:27 -0500)]
closes #7252
Merge branch '7252-go-sdk-errors'
radhika [Fri, 27 Nov 2015 16:03:10 +0000 (11:03 -0500)]
Merge branch 'master' into 7748-datamanager-dry-run
radhika [Fri, 27 Nov 2015 16:02:38 +0000 (11:02 -0500)]
Merge branch 'master' into 7252-go-sdk-errors
Tom Clegg [Fri, 27 Nov 2015 09:28:16 +0000 (04:28 -0500)]
Mock dispatch instead of relying on rollback in child proc.
No issue #. amends
d88316705b23d1022ee2167c7ed0e5cf5e460cbe
Tom Clegg [Fri, 27 Nov 2015 07:01:42 +0000 (02:01 -0500)]
Clear job queue before testing crunch-dispatch start/stop.
Otherwise, it might get as far as starting jobs before we send TERM.
No issue # but see build 2249.
Tom Clegg [Fri, 27 Nov 2015 02:16:57 +0000 (21:16 -0500)]
Merge branch '7851-available-port' refs #7851
Tom Clegg [Fri, 27 Nov 2015 02:16:16 +0000 (21:16 -0500)]
Fix crash on removing nonexistent directory.
No issue #
Tom Clegg [Thu, 26 Nov 2015 22:44:18 +0000 (17:44 -0500)]
7851: Use a random display (:100-:499 = port 6100-6499) for Xvfb, instead of always :99.
radhika [Thu, 26 Nov 2015 22:24:41 +0000 (17:24 -0500)]
7748: add dry-run command line argument.
Tom Clegg [Thu, 26 Nov 2015 21:41:49 +0000 (16:41 -0500)]
7851: Choose available ports for selenium and poltergeist.
Also, display the chosen port numbers at startup.
Without this the Selenium driver defaults to 7055, which causes
interference between concurrent test runs.
Tom Clegg [Thu, 26 Nov 2015 19:23:03 +0000 (14:23 -0500)]
Warn, but do not error out, if a collection has conflicting block sizes.
If this is treated as a fatal error, any user can stop all garbage
collection entirely by hand-editing a collection (accidentally or
maliciously) to include the same block with different size hints.
No issue #
radhika [Thu, 26 Nov 2015 18:10:48 +0000 (13:10 -0500)]
refs #7253
Merge branch '7253-datamanager-test-errors'
radhika [Thu, 26 Nov 2015 18:08:07 +0000 (13:08 -0500)]
7253: update the manifest_text for the test fixture multilevel_collection_2 to be valid.
Tom Clegg [Thu, 26 Nov 2015 16:14:29 +0000 (11:14 -0500)]
Update capybara in workbench bundle.
No issue #
radhika [Thu, 26 Nov 2015 16:01:03 +0000 (11:01 -0500)]
Merge branch 'master' into 7252-go-sdk-errors
Conflicts:
sdk/go/manifest/manifest_test.go
radhika [Thu, 26 Nov 2015 15:46:51 +0000 (10:46 -0500)]
closes #7253
Merge branch '7253-datamanager-test-errors'
radhika [Thu, 26 Nov 2015 15:33:31 +0000 (10:33 -0500)]
7253: update manigest.parseManifestStream to raise error when the given manifest does not have any block locators or file tokens.
radhika [Thu, 26 Nov 2015 05:22:12 +0000 (00:22 -0500)]
7253: update BlockIterWithDuplicates to return any errors through Manifest, rather than BlockLocator.
radhika [Thu, 26 Nov 2015 03:52:29 +0000 (22:52 -0500)]
Merge branch 'master' into 7253-datamanager-test-errors
radhika [Thu, 26 Nov 2015 03:50:19 +0000 (22:50 -0500)]
7253: consume all blocks from BlockIterWithDuplicates, not just the first in test.
Peter Amstutz [Thu, 26 Nov 2015 02:14:02 +0000 (21:14 -0500)]
Merge branch '7832-pysdk-use-slots' refs #7832
radhika [Wed, 25 Nov 2015 23:07:30 +0000 (18:07 -0500)]
7253: update arvadostest.stub to offer ServerStub and remove the redundant KeepServerStub; update tests accordingly.
radhika [Wed, 25 Nov 2015 22:27:21 +0000 (17:27 -0500)]
7253: rename arvadostest/testing.go as stub.go
radhika [Wed, 25 Nov 2015 22:21:22 +0000 (17:21 -0500)]
Merge branch 'master' into 7253-datamanager-test-errors
radhika [Wed, 25 Nov 2015 22:16:27 +0000 (17:16 -0500)]
7252: convert fatalf's into errors in logger sdk code; remove AssertFromString from blockdigest and instead use FromString in all places where it was being used.
Peter Amstutz [Wed, 25 Nov 2015 21:41:45 +0000 (16:41 -0500)]
7832: Add __slots__ and remove __del__ which interfers with garbage collection. Memory usage comes down by ~40%
Tom Clegg [Wed, 25 Nov 2015 18:37:24 +0000 (13:37 -0500)]
Move remaining arv-run-pipeline-instance usage info from comments to --help message.
No issue #
Tom Clegg [Wed, 25 Nov 2015 18:06:07 +0000 (13:06 -0500)]
7751: Clean up KEEP_LOCAL_STORE env var after tests.
refs #7751
radhika [Wed, 25 Nov 2015 17:06:05 +0000 (12:06 -0500)]
7253: instead of ignoring errors during BlockIterWithDuplicates, send them to the caller.
Tom Clegg [Wed, 25 Nov 2015 15:49:57 +0000 (10:49 -0500)]
5824: Move run_test_server diag messages from stdout to stderr.
refs #5824
Tom Clegg [Wed, 25 Nov 2015 15:45:24 +0000 (10:45 -0500)]
Merge branch '7751-mount-tmp' refs #7751
Tom Clegg [Tue, 24 Nov 2015 18:45:02 +0000 (13:45 -0500)]
7751: Fix race by telling fuse not to cache the .arvados#collection dirent.
Tom Clegg [Tue, 24 Nov 2015 18:43:42 +0000 (13:43 -0500)]
7751: Refactor TmpCollectionDirectory: generate .arvados#collection less often.
Tom Clegg [Tue, 24 Nov 2015 15:41:15 +0000 (10:41 -0500)]
7751: Fix stack trace on shutdown caused by race between destroy() and llfuse unmount.
Tom Clegg [Mon, 23 Nov 2015 22:42:40 +0000 (17:42 -0500)]
7751: Fix shell wrapper code path.
Tom Clegg [Mon, 23 Nov 2015 21:31:37 +0000 (16:31 -0500)]
7751: Lowercase method name for PEP-0008.
Tom Clegg [Sun, 22 Nov 2015 10:26:13 +0000 (05:26 -0500)]
7751: Suppress help messages when testing usage errors.
Tom Clegg [Fri, 20 Nov 2015 23:37:32 +0000 (18:37 -0500)]
7751: Update explanation of special .arvados#collection file. Fix wayward use of "Keep locator".
Tom Clegg [Fri, 20 Nov 2015 23:09:55 +0000 (18:09 -0500)]
7751: Return an empty array from InodeCache.find() instead of None.
Tom Clegg [Fri, 20 Nov 2015 08:38:10 +0000 (03:38 -0500)]
7751: Test mount arguments.
Tom Clegg [Fri, 20 Nov 2015 01:38:34 +0000 (20:38 -0500)]
7751: Add tests for --mount-tmp option.
Tom Clegg [Thu, 19 Nov 2015 22:37:25 +0000 (17:37 -0500)]
7751: Move code from arv-mount executable to module.
Tom Clegg [Wed, 18 Nov 2015 19:32:18 +0000 (14:32 -0500)]
7751: Add --mount-tmp option.
Tom Clegg [Wed, 25 Nov 2015 15:43:53 +0000 (10:43 -0500)]
Merge branch '7676-dispatch-cheaper-nodes' closes #7676
Tom Clegg [Tue, 24 Nov 2015 21:16:13 +0000 (16:16 -0500)]
7676: Improve comments.
Tom Clegg [Mon, 23 Nov 2015 18:29:52 +0000 (13:29 -0500)]
7676: Test choosing cheaper nodes, and basic crunch-dispatch startup/locking.
Tom Clegg [Mon, 23 Nov 2015 09:03:40 +0000 (04:03 -0500)]
7676: Move crunch dispatch code into lib.
Tom Clegg [Mon, 23 Nov 2015 08:07:40 +0000 (03:07 -0500)]
7676: Prefer cheaper nodes when allocating nodes to jobs.
radhika [Wed, 25 Nov 2015 15:21:11 +0000 (10:21 -0500)]
7253: Update test so that it compiles and passes.
Peter Amstutz [Tue, 24 Nov 2015 20:31:34 +0000 (15:31 -0500)]
6309: Add runtime constraint keep_cache_mb_per_task
radhika [Tue, 24 Nov 2015 20:02:20 +0000 (15:02 -0500)]
7253: Write a test that "does not" fail when an invalid manifest text is retrieved during collection.GetCollections(). The expected behavior: GetCollections() raises error in this case and hence datamanager aborts without deleting any blocks.
radhika [Tue, 24 Nov 2015 17:51:13 +0000 (12:51 -0500)]
7253: Refactor APIStub and KeepServerStub into a testing.go so that the code can be reused from various packages
Tom Clegg [Sun, 22 Nov 2015 10:03:36 +0000 (05:03 -0500)]
7751: 7654: Rename bool to _closed to avoid conflicting with superclass closed() method.
refs #7654