Ward Vandewege [Fri, 31 Oct 2014 02:07:11 +0000 (22:07 -0400)]
Fix typo.
refs #4186
Ward Vandewege [Wed, 29 Oct 2014 16:27:45 +0000 (12:27 -0400)]
Keepproxy just needs an anonymous token.
refs #4186
Ward Vandewege [Wed, 29 Oct 2014 15:40:12 +0000 (11:40 -0400)]
Batch of improvements for the manual installation documentation:
* add prerequisites section
* add shell server section
* add keepproxy section
* many other updates throughout
refs #4186
Ward Vandewege [Wed, 29 Oct 2014 15:43:32 +0000 (11:43 -0400)]
Merge branch 'master' into 4186-install-doc-improvements
Ward Vandewege [Wed, 29 Oct 2014 03:10:36 +0000 (23:10 -0400)]
Workbench: Change from yui to uglifier to compress javascript in
config/environments/production.rb.example
No issue #
Ward Vandewege [Wed, 29 Oct 2014 02:18:37 +0000 (22:18 -0400)]
Improve config/application.yml.example for the API server and workbench:
* API server: define prefix_uuid in development mode
* workbench: add a production section
No issue #
Brett Smith [Tue, 28 Oct 2014 17:32:05 +0000 (13:32 -0400)]
Fix off-by-one error in Node Manager server calculations.
This bug prevented Node Manager from starting nodes for a job that
requested a number of nodes equal to the daemon's max_nodes setting.
No issue #.
Tim Pierce [Mon, 27 Oct 2014 20:13:12 +0000 (16:13 -0400)]
Merge branch '4015-collection-chooser-portable-data-hash'
Fixes #4015.
Tim Pierce [Mon, 27 Oct 2014 19:49:00 +0000 (15:49 -0400)]
4015: update FUSE mount tests (again)
Tim Pierce [Mon, 27 Oct 2014 18:06:12 +0000 (14:06 -0400)]
4015: simplify integration tests
Responding to code review in #4015-8:
We can (and should) test that the collection portable_data_hash is
recorded in individual tests without adding explicit new tests or
fixtures.
Updated test 'Create and run a pipeline' and the helper
create_and_run_pipeline_in_aproject to check that the collection
portable_data_hash, uuid and name have been recorded correctly in the
Advanced tab.
Tom Clegg [Mon, 27 Oct 2014 17:35:34 +0000 (13:35 -0400)]
Merge branch '4297-dispatch-load' refs #4297
Tom Clegg [Mon, 27 Oct 2014 16:54:16 +0000 (12:54 -0400)]
4297: Merge branch 'master' into 4297-dispatch-load
Tom Clegg [Mon, 27 Oct 2014 16:53:28 +0000 (12:53 -0400)]
4297: Rename @have_tags -> @job_tags, @have_commits -> @fetched_commits. Add comments.
Tom Clegg [Mon, 27 Oct 2014 16:52:52 +0000 (12:52 -0400)]
4297: Use save! instead of save to ensure failures get logged.
radhika [Mon, 27 Oct 2014 15:38:01 +0000 (11:38 -0400)]
closes #4285
Merge branch '4285-view-download-collection-subdir-files'
radhika [Mon, 27 Oct 2014 15:28:19 +0000 (11:28 -0400)]
4285: use regexp to prepent ./ to file path
radhika [Sat, 25 Oct 2014 14:01:40 +0000 (10:01 -0400)]
4285: prepend './' to file path being retrieved when it is in a subdir in a collection.
Tim Pierce [Fri, 24 Oct 2014 17:58:15 +0000 (13:58 -0400)]
Merge branch '3699-arv-copy'
Refs #3699.
Tim Pierce [Fri, 24 Oct 2014 17:57:23 +0000 (13:57 -0400)]
3699: fix namespace clash: rename module to arv_copy
Prevents confusion with the Python 'copy' module.
No issue # (bug fix)
Tim Pierce [Fri, 24 Oct 2014 17:20:48 +0000 (13:20 -0400)]
Merge branch '3699-arv-copy'
Closes #3699.
Peter Amstutz [Fri, 24 Oct 2014 14:13:56 +0000 (10:13 -0400)]
Increase the timeout on "lots of events" websocket test. refs #4295
Peter Amstutz [Fri, 24 Oct 2014 13:03:07 +0000 (09:03 -0400)]
Merge branch '4295-efficient-websockets' closes #4295
Peter Amstutz [Fri, 24 Oct 2014 13:02:40 +0000 (09:02 -0400)]
4295: Use EventMachine::schedule when there is potentially more events to send instead of pushing an event to the channel (which wakes up all listeners).
Peter Amstutz [Fri, 24 Oct 2014 12:47:45 +0000 (08:47 -0400)]
4295: Fix typo in comment
Tim Pierce [Thu, 23 Oct 2014 22:35:19 +0000 (18:35 -0400)]
Merge branch 'master' into 4015-collection-chooser-portable-data-hash
Tim Pierce [Thu, 23 Oct 2014 22:34:03 +0000 (18:34 -0400)]
4015: fix pipeline instance tests
radhika [Thu, 23 Oct 2014 22:01:02 +0000 (18:01 -0400)]
closes #4195
Merge branch '4195-pipeline-tutorial'
radhika [Thu, 23 Oct 2014 22:00:12 +0000 (18:00 -0400)]
Merge branch 'master' into 4195-pipeline-tutorial
radhika [Thu, 23 Oct 2014 21:59:04 +0000 (17:59 -0400)]
4195: update workbench dashboard pic
Ward Vandewege [Thu, 23 Oct 2014 21:32:00 +0000 (17:32 -0400)]
Merge branch '4186-install-doc-improvements'
refs #4186
Ward Vandewege [Thu, 23 Oct 2014 21:31:38 +0000 (17:31 -0400)]
Merge branch 'master' into 4186-install-doc-improvements
radhika [Thu, 23 Oct 2014 21:24:29 +0000 (17:24 -0400)]
Merge branch 'master' into 4195-pipeline-tutorial
radhika [Thu, 23 Oct 2014 21:23:00 +0000 (17:23 -0400)]
closes #4228
Merge branch '4228-collection-subdir-files'
radhika [Thu, 23 Oct 2014 21:03:44 +0000 (17:03 -0400)]
Merge branch 'master' into 4195-pipeline-tutorial
Tim Pierce [Thu, 23 Oct 2014 21:00:54 +0000 (17:00 -0400)]
4015: test recording collection hashes for pipeline inputs
Tim Pierce [Wed, 22 Oct 2014 16:44:53 +0000 (12:44 -0400)]
4015: use portable_data_hash to populate chooser
Ward Vandewege [Thu, 23 Oct 2014 20:58:53 +0000 (16:58 -0400)]
Add install dependencies for the SSO server.
refs #4186
radhika [Thu, 23 Oct 2014 20:52:06 +0000 (16:52 -0400)]
Merge branch 'master' into 4228-collection-subdir-files
Conflicts:
apps/workbench/test/integration/collections_test.rb
radhika [Thu, 23 Oct 2014 20:36:32 +0000 (16:36 -0400)]
refs #4227
Merge branch '4227-test-fix'
Ward Vandewege [Thu, 23 Oct 2014 20:29:31 +0000 (16:29 -0400)]
Explain the Keepstore's -permission-key-file argument and API server's
blob_signing_key configuration value.
refs #4186
radhika [Thu, 23 Oct 2014 20:27:02 +0000 (16:27 -0400)]
no issue #
radhika [Thu, 23 Oct 2014 20:24:00 +0000 (16:24 -0400)]
no issue #
Merge branch 'master' of git.curoverse.com:arvados
radhika [Thu, 23 Oct 2014 20:22:50 +0000 (16:22 -0400)]
refs #4227 - update strptime format for parsing pipeline start and finish time.
Peter Amstutz [Thu, 23 Oct 2014 20:22:48 +0000 (16:22 -0400)]
4295: Abort on NoMemoryError
Ward Vandewege [Thu, 23 Oct 2014 19:59:23 +0000 (15:59 -0400)]
First set of improvements:
* Suggest skipping SSO server installation for now, until we upgrade that codebase
* Be more explicit about the desired values for arvados_login_base and
arvados_v1_base in the workbench installation instructions.
refs #4186
Peter Amstutz [Thu, 23 Oct 2014 19:52:04 +0000 (15:52 -0400)]
4295: Add test that large numbers of events are handled efficiently and correctly.
Brett Smith [Thu, 23 Oct 2014 19:44:35 +0000 (15:44 -0400)]
Merge branch '4139-clean-api-node-cruft-wip'
Refs #4139, #4274. Closes #4276.
Brett Smith [Thu, 23 Oct 2014 19:36:52 +0000 (15:36 -0400)]
4139: Node create API requires a node parameter.
This makes it more consistent with other API create methods.
Brett Smith [Tue, 21 Oct 2014 14:58:59 +0000 (10:58 -0400)]
4139: Remove cloud node setup code from API server.
This code has never been used in production, and its functionality has
officially been assumed by Node Manager.
Tom Clegg [Thu, 23 Oct 2014 19:39:40 +0000 (15:39 -0400)]
4297: Cache results of git-fetch-pack, git-tag, and API token
creation. Fail jobs on git repo/version failures instead of retrying
forever.
radhika [Thu, 23 Oct 2014 19:37:52 +0000 (15:37 -0400)]
refs #4227 - rescue from any error in parsing pipeline start and finish time so that jenkins runs do not choke on these failures.
Peter Amstutz [Thu, 23 Oct 2014 19:29:09 +0000 (15:29 -0400)]
4295: Fixed tests
Peter Amstutz [Thu, 23 Oct 2014 19:22:21 +0000 (15:22 -0400)]
4295: First pass at more efficient database utilization. A few tests fail.
Tom Clegg [Thu, 23 Oct 2014 18:53:24 +0000 (14:53 -0400)]
4297: Do not update_node_status more than once per second.
Peter Amstutz [Thu, 23 Oct 2014 17:53:31 +0000 (13:53 -0400)]
Increase limit for collections controller querying same portable_data_hash as
workaround to fix tests, refs #4058, but really needs a proper fix, refs #3400
Tim Pierce [Thu, 23 Oct 2014 17:36:10 +0000 (13:36 -0400)]
3699: figure out correct docker image to fetch
Use arvados.commands.keepdocker.list_images_in_arv to figure out which
Docker image is represented by a given image name+tag.
Peter Amstutz [Thu, 23 Oct 2014 15:18:54 +0000 (11:18 -0400)]
Merge branch '4058-workbench-collection-redirect' closes #4058
Peter Amstutz [Thu, 23 Oct 2014 15:18:29 +0000 (11:18 -0400)]
4058: Typo fix "collection" -> "collections"
Peter Amstutz [Thu, 23 Oct 2014 13:38:15 +0000 (09:38 -0400)]
4058: Tweaked presentation per review feedback
radhika [Thu, 23 Oct 2014 02:25:46 +0000 (22:25 -0400)]
closes #4026
Merge branch '4026-project-move-admin'
radhika [Thu, 23 Oct 2014 02:24:37 +0000 (22:24 -0400)]
4026: simplify writable check
radhika [Thu, 23 Oct 2014 00:10:17 +0000 (20:10 -0400)]
Merge branch 'master' into 4026-project-move-admin
radhika [Wed, 22 Oct 2014 23:38:11 +0000 (19:38 -0400)]
Merge branch 'master' into 4228-collection-subdir-files
radhika [Wed, 22 Oct 2014 23:12:55 +0000 (19:12 -0400)]
4228: add test for combining files from a collection subdir.
radhika [Wed, 22 Oct 2014 21:18:25 +0000 (17:18 -0400)]
4228: fix arv_normalize input when a file in a subdir of a collection is being combined.
Tim Pierce [Wed, 22 Oct 2014 20:28:33 +0000 (16:28 -0400)]
3699: copy docker image links
copy_docker_images and copy_docker_image copy any docker image
collections and add 'docker_image_repo+tag' and 'docker_image_hash'
links as necessary at the destination.
radhika [Wed, 22 Oct 2014 20:17:50 +0000 (16:17 -0400)]
4228: fix arv_normalize input when a file in a subdir of a collection is being combined.
Brett Smith [Wed, 22 Oct 2014 19:13:09 +0000 (15:13 -0400)]
4137: Fix bad state reference in Node Manager daemon.
Refs #4137.
radhika [Wed, 22 Oct 2014 18:57:27 +0000 (14:57 -0400)]
closes #4227
Merge branch '4227-date-display'
radhika [Wed, 22 Oct 2014 18:55:55 +0000 (14:55 -0400)]
Merge branch 'master' into 4227-date-display
Conflicts:
apps/workbench/test/integration/projects_test.rb
radhika [Wed, 22 Oct 2014 18:46:34 +0000 (14:46 -0400)]
refs #4091 - comment out the test that is failing intermittently in jenkins env.
Brett Smith [Wed, 22 Oct 2014 18:28:45 +0000 (14:28 -0400)]
4139: Fix race condition in Node Manager tests.
Get a reference to the monitor mock earlier, before the daemon starts
working with it. Otherwise, we might get inconsistent state results.
Refs #4139.
Tom Clegg [Wed, 22 Oct 2014 18:01:10 +0000 (14:01 -0400)]
Report "swap" memory statistic if kernel provides it. refs #4185
radhika [Wed, 22 Oct 2014 17:12:10 +0000 (13:12 -0400)]
4227: much simplified date display test
radhika [Wed, 22 Oct 2014 15:55:02 +0000 (11:55 -0400)]
Merge branch 'master' into 4227-date-display
Tom Clegg [Wed, 22 Oct 2014 15:01:55 +0000 (11:01 -0400)]
Merge branch '3826-crunchstat-netstats' closes #3826
Tom Clegg [Wed, 22 Oct 2014 14:57:03 +0000 (10:57 -0400)]
3826: Merge branch 'master' into 3826-crunchstat-netstats
Brett Smith [Wed, 22 Oct 2014 14:35:21 +0000 (10:35 -0400)]
Merge branch '4139-node-manager-race-fix-wip'
Refs #4139, #4275.
Brett Smith [Wed, 22 Oct 2014 14:34:23 +0000 (10:34 -0400)]
4139: Node Manager more closely tracks nodes it boots.
When using Node Manager on EC2, we saw a race condition where a node
would finish setting up, but would not be returned in node listings
right away. This adds a "booted" node state to the daemon, where a
node booted by Node Manager is assumed to be up and running unless it
disappears from the listing *after* being listed. It's escalated to
the normal workflow after it appears in a listing.
Brett Smith [Wed, 22 Oct 2014 14:02:33 +0000 (10:02 -0400)]
4139: Node Manager supports subscribing to shutdown events.
This is necessary to fix a race condition. The daemon needs to be
able to track nodes that it has booted even if they don't appear in
listings.
radhika [Tue, 21 Oct 2014 22:12:44 +0000 (18:12 -0400)]
Merge branch 'master' into 4195-pipeline-tutorial
radhika [Tue, 21 Oct 2014 22:11:45 +0000 (18:11 -0400)]
4195: fix the tutorial to match the latest dashboard ui.
Tom Clegg [Tue, 21 Oct 2014 21:28:53 +0000 (17:28 -0400)]
Merge branch '4266-log-performance' refs #4266
Tom Clegg [Tue, 21 Oct 2014 21:12:07 +0000 (17:12 -0400)]
4266: Merge branch 'master' into 4266-log-performance
Tom Clegg [Tue, 21 Oct 2014 20:36:13 +0000 (16:36 -0400)]
4266: Add word missing from comment.
Tom Clegg [Tue, 21 Oct 2014 20:35:41 +0000 (16:35 -0400)]
4266: Use each_line as enumerator instead of building an array with lines.each.
Tom Clegg [Tue, 21 Oct 2014 20:34:53 +0000 (16:34 -0400)]
4266: Add missing "next" statement, oops.
radhika [Tue, 21 Oct 2014 19:57:47 +0000 (15:57 -0400)]
4227: update fixtures to use bigger finish times
radhika [Tue, 21 Oct 2014 19:44:46 +0000 (15:44 -0400)]
Merge branch 'master' of git.curoverse.com:arvados into 4227-date-display
Brett Smith [Tue, 21 Oct 2014 18:35:49 +0000 (14:35 -0400)]
Merge branch '4139-node-manager-polling-logging-wip'
Refs #4275, #4139.
Brett Smith [Tue, 21 Oct 2014 18:31:44 +0000 (14:31 -0400)]
4139: Improve scheduling of Node Manager polls.
* Catch all exceptions, so that we keep polling no matter what
happens. Use CLIENT_ERRORS as a hint about how much logging we
need.
* Make the next poll time calculation a little less stateful and
easier to follow.
* Add tests for poll scheduling.
Brett Smith [Tue, 21 Oct 2014 18:13:37 +0000 (14:13 -0400)]
4139: Improve logging in Node Manager poll actors.
Brett Smith [Tue, 21 Oct 2014 15:19:05 +0000 (11:19 -0400)]
4139: Node Manager callback actor avoids redundant delivery calls.
This helps make it more responsive when it receives a shutdown
request.
radhika [Tue, 21 Oct 2014 17:48:35 +0000 (13:48 -0400)]
4227: some more test and fixture updates.
Tom Clegg [Tue, 21 Oct 2014 17:46:00 +0000 (13:46 -0400)]
4266: Tweak long-line handling.
Tom Clegg [Tue, 21 Oct 2014 17:29:05 +0000 (13:29 -0400)]
4266: Fix OOM when stderr has a lot of data with no newlines.
radhika [Tue, 21 Oct 2014 17:28:57 +0000 (13:28 -0400)]
4227: minor test upadtes
Tom Clegg [Tue, 21 Oct 2014 17:05:36 +0000 (13:05 -0400)]
4266: Clean up streambuf.split(newline) loop.
radhika [Tue, 21 Oct 2014 17:00:01 +0000 (13:00 -0400)]
Merge branch 'master' into 4227-date-display
radhika [Tue, 21 Oct 2014 16:56:33 +0000 (12:56 -0400)]
closes #4054
Merge branch '4054-sharing-email'
projects / arvados.git / log