radhika [Tue, 9 Jun 2015 01:42:30 +0000 (21:42 -0400)]
6203: Use LOCATION_REGEXP from sdk; also add back updated each_manifest_locator method
with the same updates as munge method since this is simpler than the munge method. This
has seems further improved performance.
radhika [Mon, 8 Jun 2015 19:39:37 +0000 (15:39 -0400)]
6203: Use manifest.each_line and line.rstrip! instead of manifest.split("\n").
Performance was comparable in both cases; though each_line itself is twice as fast (we need to do strip, which eats away the gain).
radhika [Mon, 8 Jun 2015 18:32:42 +0000 (14:32 -0400)]
6203: further optimization of munge method; also, match[0].sub(/\+A[^+]*/, '') instead of split+append
resulted in another 2 seconds saving in each of create and update operations!
Tom Clegg [Thu, 4 Jun 2015 04:52:48 +0000 (00:52 -0400)]
6074: Use each instead of find_each, so our order() and limit() constraints are respected.
According to http://apidock.com/rails/ActiveRecord/Batches/find_each,
both order and limit are ignored.
The existing test "max_index_database_read does not interfere with
order" had two fatal bugs that prevented it from catching the
find_each problem:
1. It didn't select the 'name' column, so the 'name' order was
ignored. But the test passed because the name wasn't returned,
item['name'] was nil, and... nil !~ /pattern/ is true. Both
problems are fixed here.
(This explains why it seemed to find 15 names starting with
Collection_9, rather than just the 11 that exist (_9 and _90..99).)
2. It tested only that the returned results followed the requested
order, not that the order was followed when deciding what the limit
should be. All of the items_available were the same size, so _any_
order would have set the limit at 15 and passed the test.
The second problem is fixed by adding a separate test.
radhika [Thu, 4 Jun 2015 03:57:58 +0000 (23:57 -0400)]
6203: add trailing newline character in munge_manifest_locators method only when the original manifest ended with one; one of the unit tests did not like it otherwise.
Tom Clegg [Wed, 3 Jun 2015 20:28:01 +0000 (16:28 -0400)]
6074: Clear any existing ActiveRecord select() before adding our own,
otherwise we'll read all the big values when we're really just trying
to predict the result size.
Tom Clegg [Mon, 25 May 2015 14:35:02 +0000 (10:35 -0400)]
6087: Use HTTPClient's compression feature (instead of adding the
Content-Encoding header ourselves). Rename config knob to describe
purpose instead of implementation.
Tom Clegg [Sun, 31 May 2015 06:02:19 +0000 (02:02 -0400)]
6146: Improvements to "kill srun process if slurm task disappears" feature:
* Log when we notice a process is orphaned.
* Log when we decide to kill an orphaned process.
* Use `squeue --jobs $SLURM_JOBID` so slurm doesn't have to tell us
about other jobs' tasks.
* Do not kill a process that is still reporting stderr.
* Do not check `squeue` at all if every process has reported stderr
since the last squeue check. (In such cases, it seems safe to assume
no children are hung/dead.)
* Use the same timer/interval (15 seconds) for both noticing and
killing orphaned processes.
Tom Clegg [Thu, 28 May 2015 21:13:44 +0000 (17:13 -0400)]
6146: Exit TEMPFAIL early (without failing the job) if worker nodes cannot run a trivial command.
This is meant to improve the way we handle a couple of edge cases.
1. A worker node doesn't get bootstrapped properly. It works well
enough to persuade nodemanager and the API server that it's alive and
ready to run jobs, but it can't actually run jobs. This means there's
a bug in the bootstrapping process -- its startup script shouldn't
tell slurm State=RESUME without checking itself -- but even so this
doesn't deserve to fail a job: it's definitely a system problem,
there's zero chance a different job would have gone any differently.
2. A worker node has a hardware problem, or it has fallen off the
network, or something like that, but slurm hasn't yet noticed and set
its state to DOWN, so slurm still uses it to satisfy crunch-dispatch's
"salloc" commands. As above, there's zero chance this could have gone
differently for any other job, so it doesn't make sense to fail the
job.
Tom Clegg [Mon, 25 May 2015 14:35:02 +0000 (10:35 -0400)]
6087: Use HTTPClient's compression feature (instead of adding the
Content-Encoding header ourselves). Rename config knob to describe
purpose instead of implementation.
projects / arvados.git / log