arvados.git
10 years agoRuns crunch script using "stdbuf -o0 -e0" so stdout and stderr are unbuffered,
Peter Amstutz [Thu, 12 Jun 2014 17:58:35 +0000 (13:58 -0400)]
Runs crunch script using "stdbuf -o0 -e0" so stdout and stderr are unbuffered,
solves problem of stdout showing up out of order and with incorrect time stamps
in logs.  no issue #

10 years agoMade a note about user how permissions are applied when attempting to create a
Peter Amstutz [Thu, 12 Jun 2014 15:14:03 +0000 (11:14 -0400)]
Made a note about user how permissions are applied when attempting to create a
token for another user.  Test for this case already exists.  closes #2350.

10 years agoMerge branch 'origin-2882-job-process-stats' refs #2882
Peter Amstutz [Thu, 12 Jun 2014 14:23:42 +0000 (10:23 -0400)]
Merge branch 'origin-2882-job-process-stats' refs #2882

10 years ago2882: Search subdirectories of cgroup for stats, because compute0 has a
Peter Amstutz [Thu, 12 Jun 2014 14:19:18 +0000 (10:19 -0400)]
2882: Search subdirectories of cgroup for stats, because compute0 has a
slightly different cgroup layout from what's on my workstation (wtf?).

10 years agorefs #2871
radhika [Wed, 11 Jun 2014 16:30:24 +0000 (12:30 -0400)]
refs #2871
Merge branch '2871-preload-objects'

10 years ago2871: fix the typo caught by integration test
radhika [Wed, 11 Jun 2014 16:22:32 +0000 (12:22 -0400)]
2871: fix the typo caught by integration test

10 years agocloses #2871
radhika [Wed, 11 Jun 2014 15:02:21 +0000 (11:02 -0400)]
closes #2871
Merge branch '2871-preload-objects'

10 years agoBugfix: the discovery doc should not clobber defaults values with the
Ward Vandewege [Tue, 10 Jun 2014 20:31:59 +0000 (16:31 -0400)]
Bugfix: the discovery doc should not clobber defaults values with the
word 'string'. Instead, make sure the default values are always exposed
as a string, which is something the java google apiclient is sensitive
to.

Refs no issue #

10 years agoAdded error-handler for x-editable that extracts Arvados API error field for
Peter Amstutz [Tue, 10 Jun 2014 20:04:56 +0000 (16:04 -0400)]
Added error-handler for x-editable that extracts Arvados API error field for
display, closes #2819

10 years ago2871: add andand to prevent error
radhika [Tue, 10 Jun 2014 20:02:02 +0000 (16:02 -0400)]
2871: add andand to prevent error

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Tue, 10 Jun 2014 19:56:07 +0000 (15:56 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years agoMerge branch '2961-load-tab-partials' closes #2961
Peter Amstutz [Tue, 10 Jun 2014 19:46:42 +0000 (15:46 -0400)]
Merge branch '2961-load-tab-partials' closes #2961

10 years agoarv-run-pipeline-instance should not keep trying to create jobs when the
Ward Vandewege [Tue, 10 Jun 2014 18:59:21 +0000 (14:59 -0400)]
arv-run-pipeline-instance should not keep trying to create jobs when the
API server returns an error on job creation.

This closes #3005

10 years agoMerge branch '2826-simple-go-sdk' closes #2826
Peter Amstutz [Tue, 10 Jun 2014 18:56:01 +0000 (14:56 -0400)]
Merge branch '2826-simple-go-sdk' closes #2826

10 years agoFix inconsistent JSON error return behavior on job create.
Ward Vandewege [Tue, 10 Jun 2014 18:32:09 +0000 (14:32 -0400)]
Fix inconsistent JSON error return behavior on job create.

refs #3004

10 years agoMerge branch '2880-crunch-dispatch-node-constraints'
Brett Smith [Tue, 10 Jun 2014 18:12:31 +0000 (14:12 -0400)]
Merge branch '2880-crunch-dispatch-node-constraints'

Closes #2880, #2976, #2993.

10 years ago2880: Don't dispatch Jobs until runtime constraints are met.
Brett Smith [Fri, 6 Jun 2014 17:18:38 +0000 (13:18 -0400)]
2880: Don't dispatch Jobs until runtime constraints are met.

This retains the same FIFO approach to the Job queue that
crunch-dispatch currently uses, but now when it encounters a Job whose
constraints are not met:

* it may wait for a while to see if the Node Manager makes Nodes
  available, if it hasn't done that this hour; and

* it leaves that Job in the queue and tries to process the next one.

See #2880 for further background.  The exact parameters of "waiting
for Nodes" will probably need tuning, but that will be easier to do
after it's been in production for a while.

10 years ago2880: Improve error reporting when crunch-dispatch updates node status.
Brett Smith [Fri, 6 Jun 2014 16:03:54 +0000 (12:03 -0400)]
2880: Improve error reporting when crunch-dispatch updates node status.

10 years ago2880: crunch-dispatch must stop putting symbols in the database.
Brett Smith [Fri, 6 Jun 2014 16:03:08 +0000 (12:03 -0400)]
2880: crunch-dispatch must stop putting symbols in the database.

10 years ago2961: Removed calls to wait_for_ajax, fixes user tests.
Peter Amstutz [Tue, 10 Jun 2014 17:31:46 +0000 (13:31 -0400)]
2961: Removed calls to wait_for_ajax, fixes user tests.

10 years ago2961: user-setup-modal-window div is no longer in footer_html because it is
Peter Amstutz [Tue, 10 Jun 2014 17:28:13 +0000 (13:28 -0400)]
2961: user-setup-modal-window div is no longer in footer_html because it is
loaded using AJAX.

10 years ago2961: Don't call #render_index from #index on folders_controller because #index
Peter Amstutz [Tue, 10 Jun 2014 17:26:20 +0000 (13:26 -0400)]
2961: Don't call #render_index from #index on folders_controller because #index
is also used by #choose and the index does not have any tabs.

10 years ago2961: Do not include meta tag for websockets if there is no websockets in the
Peter Amstutz [Tue, 10 Jun 2014 17:23:24 +0000 (13:23 -0400)]
2961: Do not include meta tag for websockets if there is no websockets in the
discovery document.

10 years ago2961: Fixed name of collection on collection show page to fix test.
Peter Amstutz [Tue, 10 Jun 2014 17:22:48 +0000 (13:22 -0400)]
2961: Fixed name of collection on collection show page to fix test.

10 years ago2961: Track valid/up to date state for each panel, only reload panels when they
Peter Amstutz [Tue, 10 Jun 2014 14:20:29 +0000 (10:20 -0400)]
2961: Track valid/up to date state for each panel, only reload panels when they
are out of date (based on event log) and visible to user.

10 years ago2961: Added use count so ajax loading icon is accurate when there are multiple
Peter Amstutz [Tue, 10 Jun 2014 14:18:22 +0000 (10:18 -0400)]
2961: Added use count so ajax loading icon is accurate when there are multiple
outstanding ajax requests.

10 years ago2961: Removed dangling close div.
Peter Amstutz [Tue, 10 Jun 2014 14:16:04 +0000 (10:16 -0400)]
2961: Removed dangling close div.

10 years ago2961: Added missing user uuid to session.
Peter Amstutz [Tue, 10 Jun 2014 14:15:34 +0000 (10:15 -0400)]
2961: Added missing user uuid to session.

10 years agoMerge branch 'master' into 2961-load-tab-partials
Peter Amstutz [Tue, 10 Jun 2014 13:21:02 +0000 (09:21 -0400)]
Merge branch 'master' into 2961-load-tab-partials

10 years agoFix for running workbench against server that doesn't have 'Job.supplied_script_versi...
Peter Amstutz [Tue, 10 Jun 2014 13:19:39 +0000 (09:19 -0400)]
Fix for running workbench against server that doesn't have 'Job.supplied_script_version' yet.  refs #2939

10 years ago2961: Job index needs to call super sometimes, render_index other times.
Peter Amstutz [Tue, 10 Jun 2014 13:18:02 +0000 (09:18 -0400)]
2961: Job index needs to call super sometimes, render_index other times.

10 years ago2961: Changed index methods on some controllers to use render_index instead of
Peter Amstutz [Tue, 10 Jun 2014 13:08:38 +0000 (09:08 -0400)]
2961: Changed index methods on some controllers to use render_index instead of
super, to skip the default index query logic but still apply the tab partial
logic.

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Tue, 10 Jun 2014 12:58:22 +0000 (08:58 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2871: trailing whitespace
radhika [Tue, 10 Jun 2014 12:49:39 +0000 (08:49 -0400)]
2871: trailing whitespace

10 years agoSpecify server-side keep-alive ping interval for websockets. no issue #
Peter Amstutz [Mon, 9 Jun 2014 21:05:43 +0000 (17:05 -0400)]
Specify server-side keep-alive ping interval for websockets.  no issue #

10 years ago2961: Escaped erb used embedded in javascript. Updated comments in
Peter Amstutz [Mon, 9 Jun 2014 20:57:50 +0000 (16:57 -0400)]
2961: Escaped erb used embedded in javascript.  Updated comments in
event_log.js.  Removed debug put from ApplicationController.

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Mon, 9 Jun 2014 20:40:39 +0000 (16:40 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2871: test preload_objects_for_dataclass helper method
radhika [Mon, 9 Jun 2014 20:39:31 +0000 (16:39 -0400)]
2871: test preload_objects_for_dataclass helper method

10 years agoAdd puma to the Gemfile for the API server; we use it for the websockets
Ward Vandewege [Mon, 9 Jun 2014 20:33:01 +0000 (16:33 -0400)]
Add puma to the Gemfile for the API server; we use it for the websockets
server.

no issue #

10 years ago2871: preload_log_collections tests
radhika [Mon, 9 Jun 2014 19:37:39 +0000 (15:37 -0400)]
2871: preload_log_collections tests

10 years ago2871: more testing
radhika [Mon, 9 Jun 2014 18:00:53 +0000 (14:00 -0400)]
2871: more testing

10 years ago2871: more tests
radhika [Mon, 9 Jun 2014 17:24:09 +0000 (13:24 -0400)]
2871: more tests

10 years ago2871: test invoking methods a second time to retrieve preloaded data
radhika [Mon, 9 Jun 2014 16:09:56 +0000 (12:09 -0400)]
2871: test invoking methods a second time to retrieve preloaded data

10 years ago2871: add tests for get_n_objects_of_class helper method
radhika [Mon, 9 Jun 2014 15:43:40 +0000 (11:43 -0400)]
2871: add tests for get_n_objects_of_class helper method

10 years ago2961: Collections show controller now delegates to super to handle tabs
Peter Amstutz [Mon, 9 Jun 2014 15:41:19 +0000 (11:41 -0400)]
2961: Collections show controller now delegates to super to handle tabs
properly.

10 years ago2871: add tests for preload_collections_for_objects helper method
radhika [Mon, 9 Jun 2014 14:13:16 +0000 (10:13 -0400)]
2871: add tests for preload_collections_for_objects helper method

10 years ago2871: add input checks for helper methods. start adding functional tests for the...
radhika [Mon, 9 Jun 2014 13:42:04 +0000 (09:42 -0400)]
2871: add input checks for helper methods. start adding functional tests for the helper methods.

10 years agoMerge remote-tracking branch 'origin/master' into 2961-load-tab-partials
Peter Amstutz [Mon, 9 Jun 2014 13:09:20 +0000 (09:09 -0400)]
Merge remote-tracking branch 'origin/master' into 2961-load-tab-partials

10 years ago2961: Save user information as part of the session cookie, eliminates looking
Peter Amstutz [Mon, 9 Jun 2014 13:09:09 +0000 (09:09 -0400)]
2961: Save user information as part of the session cookie, eliminates looking
up current user when rendering every page and panel, this means 2x reduction in
latency when the panel only requires 1 API lookup otherwise.

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Sat, 7 Jun 2014 13:03:19 +0000 (09:03 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years agoMerge branch '2857-collection-tests'
Tim Pierce [Fri, 6 Jun 2014 21:37:18 +0000 (17:37 -0400)]
Merge branch '2857-collection-tests'

Closes #2857.

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Fri, 6 Jun 2014 19:57:19 +0000 (15:57 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2857: strip signatures from manifests before testing equality.
Tim Pierce [Fri, 6 Jun 2014 19:48:43 +0000 (15:48 -0400)]
2857: strip signatures from manifests before testing equality.

As Brett observed, the "post" method in these tests mutates its
argument. For a valid test, we must ensure that the source data has not
been transformed into the result data.

For tests which do not care about validating the signatures, we just
strip out the signatures from the result manifest before comparing it to
the source.

Refs #2857.

10 years ago2961: Added checks if we are rendering a tab partial and skips expensive calls
Peter Amstutz [Fri, 6 Jun 2014 17:45:18 +0000 (13:45 -0400)]
2961: Added checks if we are rendering a tab partial and skips expensive calls
to the API server if not needed to render the requested tab pane.

10 years agoMerge branch 'master' into 2857-collection-tests
Tim Pierce [Fri, 6 Jun 2014 15:31:59 +0000 (11:31 -0400)]
Merge branch 'master' into 2857-collection-tests

10 years agoMerge branch '2955-fail-orphan-jobs' closes #2955
Peter Amstutz [Fri, 6 Jun 2014 14:40:59 +0000 (10:40 -0400)]
Merge branch '2955-fail-orphan-jobs' closes #2955

10 years agoReverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch...
Peter Amstutz [Fri, 6 Jun 2014 14:39:39 +0000 (10:39 -0400)]
Reverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch-job"

10 years agoMerge branch 'master' into 2955-fail-orphan-jobs
Peter Amstutz [Fri, 6 Jun 2014 14:31:18 +0000 (10:31 -0400)]
Merge branch 'master' into 2955-fail-orphan-jobs

10 years agoReverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch...
Peter Amstutz [Fri, 6 Jun 2014 14:30:06 +0000 (10:30 -0400)]
Reverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch-job" refs #2882.

10 years agoMerge branch '2880-compute-ping-stats'
Brett Smith [Fri, 6 Jun 2014 13:47:11 +0000 (09:47 -0400)]
Merge branch '2880-compute-ping-stats'

Closes #2967, #2975.

10 years ago2880: API server saves node statistics from pings.
Brett Smith [Thu, 5 Jun 2014 15:38:46 +0000 (11:38 -0400)]
2880: API server saves node statistics from pings.

Crunch will use this information when scheduling Jobs, to satisfy
specified runtime constraints.  Getting this data into the pings is
the responsibility of the node's ping script.

10 years ago2882: Workbench generic create controller adds JSON href attribute.
Brett Smith [Fri, 6 Jun 2014 13:38:25 +0000 (09:38 -0400)]
2882: Workbench generic create controller adds JSON href attribute.

This is a bugfix commit.  This detail got lost when create stopped
calling show to render the object.  Refs #2882.

10 years ago2961: Now loads tab panes (past the first one) in the background.
Peter Amstutz [Fri, 6 Jun 2014 02:02:30 +0000 (22:02 -0400)]
2961: Now loads tab panes (past the first one) in the background.
Automatically triggers reload of tab panes when receiving a changed event from
websocket.

10 years ago2961: Removed spurious elementId from subscribeToEventLog(). Added arv-log-event...
Peter Amstutz [Thu, 5 Jun 2014 20:31:18 +0000 (16:31 -0400)]
2961: Removed spurious elementId from subscribeToEventLog(). Added arv-log-event-listener[data-object-type="xxxxx"] to
match on uuid types.

10 years agoFixes PipelineInstancesControllerTest. application_controller#create now does
Peter Amstutz [Thu, 5 Jun 2014 19:50:26 +0000 (15:50 -0400)]
Fixes PipelineInstancesControllerTest.  application_controller#create now does
the right thing based on requested response format.  refs #2882

10 years ago2857: fix flaky time-sensitive tests
Tim Pierce [Thu, 5 Jun 2014 19:42:06 +0000 (15:42 -0400)]
2857: fix flaky time-sensitive tests

Allow tests to inject a mock Time object into Blob, in order to generate
consistent behavior for tests that rely on permission signatures.

Closes #2857.

10 years agoMerge branch '2882-job-process-stats' refs #2882
Peter Amstutz [Thu, 5 Jun 2014 18:49:35 +0000 (14:49 -0400)]
Merge branch '2882-job-process-stats' refs #2882

10 years ago2955: Renamed script to "cancel_stale_jobs". Fixed spelling canceled_at -> cancelled_at.
Peter Amstutz [Thu, 5 Jun 2014 18:44:51 +0000 (14:44 -0400)]
2955: Renamed script to "cancel_stale_jobs".  Fixed spelling canceled_at -> cancelled_at.

10 years ago2961: Added tab_pane query parameter which returns just the desired tab pane partial.
Peter Amstutz [Thu, 5 Jun 2014 17:20:39 +0000 (13:20 -0400)]
2961: Added tab_pane query parameter which returns just the desired tab pane partial.

10 years ago2955: Moved logic to clean jobs table from crunch-dispatch into a standalone
Peter Amstutz [Thu, 5 Jun 2014 16:43:09 +0000 (12:43 -0400)]
2955: Moved logic to clean jobs table from crunch-dispatch into a standalone
script.

10 years ago2871: handle if there is no friendly_link_name in link
radhika [Thu, 5 Jun 2014 15:03:11 +0000 (11:03 -0400)]
2871: handle if there is no friendly_link_name in link

10 years ago2871: remove trailing white space
radhika [Thu, 5 Jun 2014 14:25:19 +0000 (10:25 -0400)]
2871: remove trailing white space

10 years ago2871: some more comments
radhika [Thu, 5 Jun 2014 14:04:32 +0000 (10:04 -0400)]
2871: some more comments

10 years ago2871: preload objects for dashboard
radhika [Thu, 5 Jun 2014 13:45:43 +0000 (09:45 -0400)]
2871: preload objects for dashboard

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Thu, 5 Jun 2014 12:52:01 +0000 (08:52 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2871: preload_objects_for_dataclass for pipeline instances in dashboard
radhika [Thu, 5 Jun 2014 12:47:32 +0000 (08:47 -0400)]
2871: preload_objects_for_dataclass for pipeline instances in dashboard

10 years ago2871: add preload_objects_for_dataclass method to application_controller
radhika [Thu, 5 Jun 2014 12:35:11 +0000 (08:35 -0400)]
2871: add preload_objects_for_dataclass method to application_controller

10 years ago2871: improved link_name retrieval
radhika [Wed, 4 Jun 2014 22:24:26 +0000 (18:24 -0400)]
2871: improved link_name retrieval

10 years ago2865: add traffic_test.py for testing Keep performance.
Tim Pierce [Wed, 4 Jun 2014 21:24:49 +0000 (17:24 -0400)]
2865: add traffic_test.py for testing Keep performance.

Refs #2865.

10 years ago2871: metadata tab using preloaded objects. wow, i can't believe how many api calls...
radhika [Wed, 4 Jun 2014 21:13:15 +0000 (17:13 -0400)]
2871: metadata tab using preloaded objects. wow, i can't believe how many api calls are cut down with this :)

10 years ago2871: add link head_uuids to the list of to be preloaded uuids
radhika [Wed, 4 Jun 2014 21:01:03 +0000 (17:01 -0400)]
2871: add link head_uuids to the list of to be preloaded uuids

10 years ago2882: Removed unnecessary no_reuse options
Peter Amstutz [Wed, 4 Jun 2014 20:24:07 +0000 (16:24 -0400)]
2882: Removed unnecessary no_reuse options

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Wed, 4 Jun 2014 20:17:06 +0000 (16:17 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2955: Sets jobrecord.canceled_at when canceling/failing an orphan job.
Peter Amstutz [Wed, 4 Jun 2014 20:00:03 +0000 (16:00 -0400)]
2955: Sets jobrecord.canceled_at when canceling/failing an orphan job.
arv-run-pipeline-instance recognizes canceled jobs when counting the number of ended jobs.

10 years agoMerge branch 'master' into 2919-provenance-graph-cutoff
radhika [Wed, 4 Jun 2014 19:53:46 +0000 (15:53 -0400)]
Merge branch 'master' into 2919-provenance-graph-cutoff

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Wed, 4 Jun 2014 19:53:17 +0000 (15:53 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2871: metadata page updated to use some preloaded collections and links; not fully...
radhika [Wed, 4 Jun 2014 19:50:50 +0000 (15:50 -0400)]
2871: metadata page updated to use some preloaded collections and links; not fully done yet.

10 years agoMerge remote-tracking branch 'origin/master' into 2882-job-process-stats
Peter Amstutz [Wed, 4 Jun 2014 19:23:31 +0000 (15:23 -0400)]
Merge remote-tracking branch 'origin/master' into 2882-job-process-stats

10 years ago2882: Improved re-run job buttons on job status page
Peter Amstutz [Wed, 4 Jun 2014 19:23:11 +0000 (15:23 -0400)]
2882: Improved re-run job buttons on job status page

10 years agoMerge branch 'master' into 2882-job-process-stats refs #2882
Peter Amstutz [Wed, 4 Jun 2014 19:19:10 +0000 (15:19 -0400)]
Merge branch 'master' into 2882-job-process-stats refs #2882

10 years agoMerge branch 'origin-2939-re-run-job-button' into 2882-job-process-stats
Peter Amstutz [Wed, 4 Jun 2014 19:03:37 +0000 (15:03 -0400)]
Merge branch 'origin-2939-re-run-job-button' into 2882-job-process-stats

10 years ago2882: arv-mount restores signal handlers after sp.wait(). Fixed spelling ellapsed...
Peter Amstutz [Wed, 4 Jun 2014 18:52:07 +0000 (14:52 -0400)]
2882: arv-mount restores signal handlers after sp.wait().  Fixed spelling ellapsed -> elapsed.

10 years ago2871: rename get collections and log collections method names to match the format...
radhika [Wed, 4 Jun 2014 17:58:36 +0000 (13:58 -0400)]
2871: rename get collections and log collections method names to match the format of links method.

10 years ago2933: Add Keep signing key configuration to Docker.
Brett Smith [Wed, 4 Jun 2014 14:30:16 +0000 (10:30 -0400)]
2933: Add Keep signing key configuration to Docker.

This is a little hypothetical until we Dockerize the new Keep server,
but it's necessary to get the API server running.

Refs #2933.

10 years ago2933: Update Docker to use new Python package build process.
Brett Smith [Wed, 4 Jun 2014 14:29:05 +0000 (10:29 -0400)]
2933: Update Docker to use new Python package build process.

Refs #2933.

10 years ago2871: get collections methods return all collections for the given uuid.
radhika [Wed, 4 Jun 2014 17:37:00 +0000 (13:37 -0400)]
2871: get collections methods return all collections for the given uuid.

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Wed, 4 Jun 2014 15:29:04 +0000 (11:29 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years agoMerge branch 'master' into 2895-no-more-redis
radhika [Wed, 4 Jun 2014 15:17:31 +0000 (11:17 -0400)]
Merge branch 'master' into 2895-no-more-redis

10 years agoMerge branch 'master' into 2871-preload-objects
radhika [Wed, 4 Jun 2014 15:16:30 +0000 (11:16 -0400)]
Merge branch 'master' into 2871-preload-objects

10 years ago2871: add preload methods for collections and logs in application_controller. Use...
radhika [Wed, 4 Jun 2014 15:15:24 +0000 (11:15 -0400)]
2871: add preload methods for collections and logs in application_controller. Use the preload methods in dashboard.