radhika [Wed, 8 Oct 2014 21:30:59 +0000 (17:30 -0400)]
Merge branch 'master' into 4126-preserve-parameter-hash
radhika [Wed, 8 Oct 2014 21:28:38 +0000 (17:28 -0400)]
4126: when parameter is a hash, use it as value if nothing else matches.
Brett Smith [Wed, 8 Oct 2014 20:52:44 +0000 (16:52 -0400)]
Merge branch '2881-node-manager'
Closes #2881, #4106.
Brett Smith [Fri, 3 Oct 2014 21:53:57 +0000 (17:53 -0400)]
2881: Add Node Manager service.
Tom Clegg [Wed, 8 Oct 2014 14:58:31 +0000 (10:58 -0400)]
Merge branch '4044-crunchstat-wait' refs #4044
Tom Clegg [Wed, 8 Oct 2014 14:58:23 +0000 (10:58 -0400)]
4044: Merge branch 'master' into 4044-crunchstat-wait
Tom Clegg [Wed, 8 Oct 2014 14:58:11 +0000 (10:58 -0400)]
4044: Add comments to "continue" statements.
Tom Clegg [Wed, 8 Oct 2014 14:05:27 +0000 (10:05 -0400)]
4126: Preserve hash form when populating component parameters.
radhika [Wed, 8 Oct 2014 13:58:44 +0000 (09:58 -0400)]
closes #3990
Merge branch '3990-owner-when-rerunning-pipeline'
radhika [Wed, 8 Oct 2014 13:58:10 +0000 (09:58 -0400)]
3990: minor test update
Tom Clegg [Tue, 7 Oct 2014 20:24:15 +0000 (16:24 -0400)]
4044: Clean up channel and pipe usage.
* Remove code for using one goroutine to copy stdout and stderr from
channels to pipes. Instead, copy stderr from channel to pipe in a
simple goroutine, and use give the child our own Stdout to use (we
don't use it ourselves anyway).
* Rename functions (OutputChannel -> CopyChanToPipe, ReadLineByLine ->
CopyPipeToChan).
* Add "stop" channel to shut down polling loop when child process
exits.
* Check for errors when opening cgroup stats files. Report errors
instead of displaying bogus stats.
* Split main() into run() and a short main() with os.Exit logic, so
deferred functions run regardless of exit path.
* Close dangling filehandle when cgroup_cidfile succeeds.
* Fix slight divide-by-zero opportunity when elapsed==0.
* Fix condition triggering the "could not read cid file" error
message.
radhika [Tue, 7 Oct 2014 18:04:26 +0000 (14:04 -0400)]
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
radhika [Tue, 7 Oct 2014 18:02:35 +0000 (14:02 -0400)]
closes #3882
Merge branch '3882-cancel-already-cancelled-job'
radhika [Tue, 7 Oct 2014 17:55:32 +0000 (13:55 -0400)]
3882: update "after_validation :update_timestamps_when_state_changes" to "before_save :update_timestamps_when_state_changes",
and also add comple more test combinations around cancelled state.
radhika [Tue, 7 Oct 2014 17:29:01 +0000 (13:29 -0400)]
Merge branch 'master' into 3882-cancel-already-cancelled-job
radhika [Tue, 7 Oct 2014 17:26:18 +0000 (13:26 -0400)]
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
radhika [Tue, 7 Oct 2014 17:17:33 +0000 (13:17 -0400)]
3990: rerunning pipeline from within a project with no write permission
Tom Clegg [Tue, 7 Oct 2014 16:04:34 +0000 (12:04 -0400)]
Merge branch '3775-fetch-git-repo' closes #3775
Conflicts:
sdk/cli/bin/crunch-job
radhika [Tue, 7 Oct 2014 14:42:57 +0000 (10:42 -0400)]
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
radhika [Tue, 7 Oct 2014 14:41:30 +0000 (10:41 -0400)]
3990: add integration tests to rerun pipeline.
Brett Smith [Tue, 7 Oct 2014 13:44:15 +0000 (09:44 -0400)]
Merge branch '4012-crunch-job-api-retries-wip'
Closes #4012.
Brett Smith [Tue, 7 Oct 2014 13:35:11 +0000 (09:35 -0400)]
4012: crunch-job retries all API operations.
This will make jobs more robust against transient errors when talking
to the API server.
Tom Clegg [Tue, 7 Oct 2014 03:24:53 +0000 (23:24 -0400)]
3775: Fix SDK usage.
Tom Clegg [Tue, 7 Oct 2014 03:07:35 +0000 (23:07 -0400)]
3775: Update comment
Tom Clegg [Tue, 7 Oct 2014 02:16:56 +0000 (22:16 -0400)]
3775: Update comment
Tom Clegg [Tue, 7 Oct 2014 02:06:50 +0000 (22:06 -0400)]
3775: Update perlpod. Use items_available. Be more conservative when
using already-cached commit sha1s, expand related comment. Move "skip
if not possible now" checks into subroutines.
radhika [Mon, 6 Oct 2014 23:54:15 +0000 (19:54 -0400)]
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
radhika [Mon, 6 Oct 2014 23:50:19 +0000 (19:50 -0400)]
3990: refactor pipeline instances integration test to reuse logic that creates and runs a pipeline.
Tom Clegg [Mon, 6 Oct 2014 21:55:57 +0000 (17:55 -0400)]
Merge branch '3828-keepproxy-race' closes #3828
Brett Smith [Mon, 6 Oct 2014 21:36:31 +0000 (17:36 -0400)]
3634: Update user setup tests for preserved tab state.
These tests assume that the page reloads after submitting Ajax
dialogs. They started failing when we started preserving tab state.
Update the tests to expressly refresh the page. Future improvements
might find a solution with lower overhead.
Refs #3634.
Tom Clegg [Mon, 6 Oct 2014 21:25:02 +0000 (17:25 -0400)]
Merge branch '3775-fetch-git-repo' closes #3775
Tom Clegg [Mon, 6 Oct 2014 21:21:25 +0000 (17:21 -0400)]
3775: Set state=Running when creating a Job in local mode.
Tom Clegg [Mon, 6 Oct 2014 21:13:17 +0000 (17:13 -0400)]
3775: Merge branch 'master' into 3775-fetch-git-repo
Conflicts:
sdk/cli/bin/crunch-job
radhika [Mon, 6 Oct 2014 20:27:25 +0000 (16:27 -0400)]
3990: set owner_uuid of a copied pipeline instance to that of the source, provided it is a project and writable by the current user.
Tom Clegg [Mon, 6 Oct 2014 20:05:54 +0000 (16:05 -0400)]
3828: Wait for listener to start before connecting to it. Fix test
panic in listener.Close() when listener does not exist.
Tom Clegg [Mon, 6 Oct 2014 20:04:33 +0000 (16:04 -0400)]
3828: Use defer to close pidfile. Avoids leftover pidfile if Listen fails.
Tom Clegg [Mon, 6 Oct 2014 19:56:17 +0000 (15:56 -0400)]
3828: Remove duplicate "write pidfile" block.
Brett Smith [Mon, 6 Oct 2014 19:16:23 +0000 (15:16 -0400)]
arv-put parses arguments before instantiating an API client.
This helps ensure that `--help` responds quickly even if the server is
slow or down. No issue #.
Phil Hodgson [Mon, 6 Oct 2014 19:11:19 +0000 (15:11 -0400)]
Merge branch '3634-tab-state' refs #3634
radhika [Mon, 6 Oct 2014 18:55:10 +0000 (14:55 -0400)]
closes #4046
Merge branch '4046-job-queue-position'
Phil Hodgson [Mon, 6 Oct 2014 18:53:08 +0000 (14:53 -0400)]
Merge branch 'master' into 3634-tab-state
Conflicts (resolved):
apps/workbench/app/views/application/_title_and_buttons.html.erb
radhika [Mon, 6 Oct 2014 18:51:20 +0000 (14:51 -0400)]
Merge branch 'master' into 4046-job-queue-position
Phil Hodgson [Mon, 6 Oct 2014 18:49:45 +0000 (14:49 -0400)]
3634: add anchor to URL when switching project tabs
radhika [Mon, 6 Oct 2014 18:47:43 +0000 (14:47 -0400)]
4046: update assert error message
Tom Clegg [Mon, 6 Oct 2014 18:37:06 +0000 (14:37 -0400)]
Merge branch '3782-stub-io-pipe' refs #3782
radhika [Mon, 6 Oct 2014 18:30:52 +0000 (14:30 -0400)]
Merge branch 'master' into 4046-job-queue-position
Tim Pierce [Mon, 6 Oct 2014 18:22:00 +0000 (14:22 -0400)]
Merge branch '3825-crunch-pipe-to-arv-put-final'
Closes #3825.
Tim Pierce [Mon, 6 Oct 2014 17:24:47 +0000 (13:24 -0400)]
3825: code review
* avoid overloading "output" (usually used for the output from a task or job than for diagnostic output from crunch)
** renamed:
*** start_output_log -> log_writer_start
*** write_output_log -> log_writer_send
*** finish_output_log -> log_writer_finish
*** output_log_is_active -> log_writer_is_active
* fixed missing semicolon
Tom Clegg [Mon, 6 Oct 2014 17:56:30 +0000 (13:56 -0400)]
3782: Merge branch 'master' into 3782-stub-io-pipe
radhika [Mon, 6 Oct 2014 17:50:11 +0000 (13:50 -0400)]
closes #3583
Merge branch '3583-provenance-graph-issue'
radhika [Mon, 6 Oct 2014 17:49:33 +0000 (13:49 -0400)]
Merge branch 'master' into 3583-provenance-graph-issue
radhika [Mon, 6 Oct 2014 17:47:22 +0000 (13:47 -0400)]
Merge branch 'master' into 4046-job-queue-position
radhika [Mon, 6 Oct 2014 17:45:58 +0000 (13:45 -0400)]
4046: update queue_position method to increment index and add unit test.
Tom Clegg [Mon, 6 Oct 2014 17:40:18 +0000 (13:40 -0400)]
3775: Recognize ssh short form without @ symbol. Expand comments.
Peter Amstutz [Mon, 6 Oct 2014 17:13:15 +0000 (13:13 -0400)]
Merge branch '3859-crunch-job-use-lock' closes #3859
Tom Clegg [Mon, 6 Oct 2014 17:10:39 +0000 (13:10 -0400)]
3782: Avoid using a huge read buffer when client requests a huge range. Add test.
Tom Clegg [Mon, 6 Oct 2014 16:40:12 +0000 (12:40 -0400)]
3782: Stub IO.pipe() with StringIO instead of stubbing content with KEEP_LOCAL_STORE.
radhika [Mon, 6 Oct 2014 16:30:35 +0000 (12:30 -0400)]
Merge branch 'master' into 3882-cancel-already-cancelled-job
radhika [Mon, 6 Oct 2014 16:24:29 +0000 (12:24 -0400)]
3882: Change "update_timestamps_when_state_changes" from before_validation to after_validation,
so that update of success and running flags is done AFTER ensuring the state change is valid.
Otherwise, the update to success and running flags is resulting in overriding the actual state update.
Tom Clegg [Mon, 6 Oct 2014 16:00:50 +0000 (12:00 -0400)]
3782: Restore config settings changed during tests. (copied from API server)
Tom Clegg [Mon, 6 Oct 2014 15:51:49 +0000 (11:51 -0400)]
Merge branch '3687-grant-permission-on-repo-TC' closes #3687
Tom Clegg [Mon, 6 Oct 2014 15:15:05 +0000 (11:15 -0400)]
3687: Merge branch 'master' into 3687-grant-permission-on-repo-TC
radhika [Mon, 6 Oct 2014 15:04:10 +0000 (11:04 -0400)]
Merge branch '3882-cancel-already-cancelled-job-TC' into 3882-cancel-already-cancelled-job
radhika [Mon, 6 Oct 2014 14:57:35 +0000 (10:57 -0400)]
Merge branch 'master' into 3882-cancel-already-cancelled-job
radhika [Mon, 6 Oct 2014 14:39:41 +0000 (10:39 -0400)]
closes #4025
Merge branch '4025-move-project-button-label'
radhika [Mon, 6 Oct 2014 14:23:49 +0000 (10:23 -0400)]
4025: Change the button label 'Move...' to 'Move <object_class>...' and update test assertion accordingly.
radhika [Mon, 6 Oct 2014 14:18:12 +0000 (10:18 -0400)]
Merge branch 'master' into 4025-move-project-button-label
Conflicts:
apps/workbench/app/views/application/_title_and_buttons.html.erb
Tom Clegg [Mon, 6 Oct 2014 13:59:36 +0000 (09:59 -0400)]
3882: refactor update-or-cancel code.
Tom Clegg [Mon, 6 Oct 2014 13:41:25 +0000 (09:41 -0400)]
3882: Test response status and #cancel method.
radhika [Sat, 4 Oct 2014 16:23:34 +0000 (12:23 -0400)]
3882: refactor tests
radhika [Sat, 4 Oct 2014 01:51:39 +0000 (21:51 -0400)]
3882: added a few more tests for cancelled_at state. Most importantly, uncomment the assertion
to raise error when a functional test performs multiple actions in a single test (refs 3889#).
Tom Clegg [Fri, 3 Oct 2014 23:59:00 +0000 (19:59 -0400)]
3687: Add comment about redundant lookup.
Tom Clegg [Fri, 3 Oct 2014 23:44:56 +0000 (19:44 -0400)]
3687: OK here just means this loop is OK, not everything is OK!
Tom Clegg [Fri, 3 Oct 2014 21:41:54 +0000 (17:41 -0400)]
3687: in User#can?, consider permission links with head_uuid pointing directly at the target.
radhika [Fri, 3 Oct 2014 21:22:53 +0000 (17:22 -0400)]
Merge branch 'master' into 3882-cancel-already-cancelled-job
radhika [Fri, 3 Oct 2014 21:22:09 +0000 (17:22 -0400)]
closes #4036
Merge branch '4036-combine-collections-in-project-issue'
radhika [Fri, 3 Oct 2014 21:18:12 +0000 (17:18 -0400)]
3882: add test to verify a cancelled job stays cancelled on futher cancel actions.
Tim Pierce [Fri, 3 Oct 2014 18:48:34 +0000 (14:48 -0400)]
3687: add functional test for sharing within a group
Test that two unprivileged users in the same group can share collections.
Phil Hodgson [Fri, 3 Oct 2014 16:42:27 +0000 (12:42 -0400)]
3634: If a link has the class "force-cache-reload", then on browser-back the page will be reloaded
(a modal will be shown to the user while the reload is occurring)
Phil Hodgson [Fri, 3 Oct 2014 16:41:08 +0000 (12:41 -0400)]
3634: set the response to ask the browser nicely to never cache
(even though it will use its "bfcache" anyway for many cases)
Tom Clegg [Fri, 3 Oct 2014 14:47:13 +0000 (10:47 -0400)]
3775: Merge branch 'master' into 3775-fetch-git-repo
Tom Clegg [Fri, 3 Oct 2014 14:42:23 +0000 (10:42 -0400)]
3775: Remove redundant log message.
Tom Clegg [Fri, 3 Oct 2014 14:41:32 +0000 (10:41 -0400)]
3775: Clean up reporting of exit codes: say "0 with signal 2", not "512".
Tom Clegg [Fri, 3 Oct 2014 14:17:07 +0000 (10:17 -0400)]
3775: Add TODO comments in suspicious-looking collate_output corners.
Tom Clegg [Fri, 3 Oct 2014 14:16:16 +0000 (10:16 -0400)]
3775: Run local/dev jobs on code from arbitrary remote git repositories.
* repository is a local path X -> use local repository X or X/.git.
* repository is a git url (git://, https://, git@host:repo.git, etc)
fetch branches, tags, and objects from the remote and use the result
to resolve script_version to a commit sha1 and run the job.
* else -> look up the named repository in Arvados and use its
fetch_url as a remote git url as above.
* --git-dir is given on the command line -> use that instead of
$Job->{repository} in the above logic. (This isn't new. It ensures
we don't go off fetching arbitrary remotes when crunch-dispatch.rb
has already pulled the code into its own internal git repo.)
Incidental changes:
* Lose support for looking up subversion revision numbers using
git-svn tags.
* Lose support for CRUNCH_DEFAULT_GIT_DIR environment variable. Pass
--git-dir instead.
* Improve log messages during checkout/install phase.
radhika [Fri, 3 Oct 2014 14:13:10 +0000 (10:13 -0400)]
4036: add comment for flash alert div
radhika [Fri, 3 Oct 2014 14:05:59 +0000 (10:05 -0400)]
Merge branch 'master' into 4036-combine-collections-in-project-issue
Tom Clegg [Fri, 3 Oct 2014 13:42:33 +0000 (09:42 -0400)]
3775: Remove stagnant $job_has_uuid flag. Every job has a uuid. The
only time to skip updating the job record on the server side is when
croak() happens before $Job is created/fetched.
Tom Clegg [Fri, 3 Oct 2014 13:33:46 +0000 (09:33 -0400)]
3775: Fix syntax error 2 commits ago.
Phil Hodgson [Fri, 3 Oct 2014 12:35:06 +0000 (08:35 -0400)]
Merge branch 'master' into 3634-tab-state
radhika [Fri, 3 Oct 2014 11:34:33 +0000 (07:34 -0400)]
4025: add fa-fw to the newly added i tag class like all other such usages.
radhika [Fri, 3 Oct 2014 11:15:23 +0000 (07:15 -0400)]
4205: add tooltips to pipeline instance tab line buttons
radhika [Fri, 3 Oct 2014 00:08:54 +0000 (20:08 -0400)]
Merge branch 'master' into 4025-move-project-button-label
radhika [Fri, 3 Oct 2014 00:07:11 +0000 (20:07 -0400)]
4025: add tooltips to some of the tab line buttons and also adjust the title of the "Move" button.
Tim Pierce [Thu, 2 Oct 2014 18:40:25 +0000 (14:40 -0400)]
3825: write log output directly to a pipe.
Added functions:
* start_output_log($logfilename)
* write_output_log($txt)
* finish_output_log()
* output_log_is_active()
Rewrote all code that referenced $local_logfile in terms of these
functions.
Peter Amstutz [Thu, 2 Oct 2014 20:33:16 +0000 (16:33 -0400)]
Merge branch '3769-throttle-logs' closes #3769
Peter Amstutz [Thu, 2 Oct 2014 20:32:02 +0000 (16:32 -0400)]
3769: Spelling fix and remove debugging puts.
radhika [Thu, 2 Oct 2014 19:13:26 +0000 (15:13 -0400)]
4036: correct div class
radhika [Thu, 2 Oct 2014 18:54:59 +0000 (14:54 -0400)]
3583: extend encode quotes logic for script parameters as well, just in case.
radhika [Thu, 2 Oct 2014 16:19:54 +0000 (12:19 -0400)]
Merge branch 'master' into 3583-provenance-graph-issue