projects / arvados.git / commit
summary | shortlog | log | commit | commitdiff | tree
(parent: 91c8ec8) | patch
12447: Reduce memory footprint of crunch-run.
authorPeter Amstutz <peter.amstutz@curoverse.com>
Fri, 13 Oct 2017 01:13:17 +0000 (21:13 -0400)
committerPeter Amstutz <peter.amstutz@curoverse.com>
Fri, 13 Oct 2017 01:13:17 +0000 (21:13 -0400)
commit6ee6e654bc873db10037c735a63697d295ec40cb
treea36b3b2ea66e75699de7afb01d2c6ad70530dd90tree | snapshot
parent91c8ec8048ac1c99d47c32c20693b06b8d307446commit | diff
12447: Reduce memory footprint of crunch-run.

* BlockCache reads into a fixed-capacity buffer, to avoid reallocations (this
  cuts resident memory usage by 1/2 and makes memory footprint extremely
  stable)
* Limit the crunch-run BlockCache to 2 blocks instead of 4 (cuts footprint by
  1/2 again)
* Clear the crunch-run BlockCache after loading the Docker image (probably cuts
  post-image-load footprint by 1/2 again).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
sdk/go/keepclient/block_cache.go diff | blob | history
services/crunch-run/crunchrun.go diff | blob | history