Peter Amstutz [Tue, 17 Oct 2017 14:50:23 +0000 (10:50 -0400)]
12452: Fix test
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 16 Oct 2017 19:50:36 +0000 (15:50 -0400)]
12452: Consider unpaired node older than boot grace period 'down'.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 16 Oct 2017 18:01:50 +0000 (14:01 -0400)]
Merge branch '12447-crunch-run-leak' closes #12447
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 16 Oct 2017 17:13:44 +0000 (13:13 -0400)]
12447: Simplify BlockCache locking.
Close response body when content-length is missing.
Tweak messages related to crunch-run -memprofile.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 16 Oct 2017 17:00:55 +0000 (13:00 -0400)]
12447: Require Content-Length in KeepClient.Get
Fix test stub to report Content-Length.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 16 Oct 2017 14:41:18 +0000 (10:41 -0400)]
12447: Fix typo shadowing 'data' in BlockCache.Get
Move locking to BlockCache.setup()
Tweak tests to bind explicitly to loopback address.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Sat, 14 Oct 2017 16:47:42 +0000 (12:47 -0400)]
12447: Remove .Bytes()
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Sat, 14 Oct 2017 15:58:14 +0000 (11:58 -0400)]
12447: Use ReadFull(). Use goimport for import formatting.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 13 Oct 2017 13:50:30 +0000 (09:50 -0400)]
12447: Add ClearBlockCache method to KeepClient
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 13 Oct 2017 01:13:17 +0000 (21:13 -0400)]
12447: Reduce memory footprint of crunch-run.
* BlockCache reads into a fixed-capacity buffer, to avoid reallocations (this
cuts resident memory usage by 1/2 and makes memory footprint extremely
stable)
* Limit the crunch-run BlockCache to 2 blocks instead of 4 (cuts footprint by
1/2 again)
* Clear the crunch-run BlockCache after loading the Docker image (probably cuts
post-image-load footprint by 1/2 again).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 12 Oct 2017 16:59:18 +0000 (12:59 -0400)]
Merge branch '12431-nm-delete-nic' refs #12431
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 12 Oct 2017 16:58:35 +0000 (12:58 -0400)]
12431: Fix test
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 12 Oct 2017 13:23:42 +0000 (09:23 -0400)]
Merge branch '12416-s3-empty-object'
refs #12416
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 11 Oct 2017 20:28:43 +0000 (16:28 -0400)]
12416: Merge Put() into PutReader().
Put() was only used for empty blocks, and passing a nil reader to
PutReader works just as well.
This ensures the "put empty block" fix applies to all writes.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 12 Oct 2017 13:14:26 +0000 (09:14 -0400)]
12416: Merge branch 'fix/keepstore-s3-radosgw-empty-object' of https://github.com/wtsi-hgi/arvados into 12416-s3-empty-object
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Joshua C. Randall [Sun, 8 Oct 2017 12:10:33 +0000 (12:10 +0000)]
fix s3_volume so that it can create empty recent/* objects on Ceph RadosGW
Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>
Peter Amstutz [Wed, 11 Oct 2017 17:03:20 +0000 (13:03 -0400)]
12431: Use libcloud fork 2.2.2.dev1
Should address issue deleting NICs. Also add ex_fetch_power_state=False to
reduce Azure API calls.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 10 Oct 2017 18:04:33 +0000 (14:04 -0400)]
Merge branch '12418-glob-empty-collection' closes #12418
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 10 Oct 2017 18:03:55 +0000 (14:03 -0400)]
12418: Add regression test for glob output on empty collection.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 10 Oct 2017 15:19:47 +0000 (11:19 -0400)]
12418: Explicitly check if collection is not None
The empty collection evaluates to False, so it is necessary to explicitly check
for "not None".
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 10 Oct 2017 17:09:38 +0000 (13:09 -0400)]
pin ciso8601 >=1.0.0, <=1.0.4 closes #12422
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 6 Oct 2017 14:15:39 +0000 (10:15 -0400)]
Update BytesAllocated -> BytesAllocatedCumulative in status report.
The number reported is the total of all buffer allocations since
startup. It doesn't take into account how many of those buffers have
since been freed.
refs #12402
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Ward Vandewege [Thu, 5 Oct 2017 15:17:35 +0000 (11:17 -0400)]
Merge branch '12355-make-cwl-conformance-tests-faster'
closes #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Thu, 5 Oct 2017 15:17:08 +0000 (11:17 -0400)]
Merge branch 'master' into 12355-make-cwl-conformance-tests-faster
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 18:49:26 +0000 (14:49 -0400)]
arvbox: use overlay2 inside arvbox.
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 15:45:09 +0000 (11:45 -0400)]
arvbox: run the rails code in production mode when running in localdemo or
publicdemo.
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Peter Amstutz [Wed, 4 Oct 2017 01:06:58 +0000 (21:06 -0400)]
Add missing test files refs #12213
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 00:44:43 +0000 (20:44 -0400)]
arvbox: containers api: make crunch-dispatch-local poll more agressively
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 00:43:13 +0000 (20:43 -0400)]
arvbox: jobs api: crunch-dispatch: stop excessive stat of /etc/localtime
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 00:42:00 +0000 (20:42 -0400)]
arvbox: bump ruby version up to 2.3.5
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 4 Oct 2017 00:39:07 +0000 (20:39 -0400)]
Run cwl conformance tests with the containers api.
refs #12355
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Tom Clegg [Tue, 3 Oct 2017 14:56:39 +0000 (10:56 -0400)]
Merge branch '8333-docker-repo-tag'
closes #8333
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 3 Oct 2017 13:25:10 +0000 (09:25 -0400)]
Merge branch '12213-expression-keepref' closes #12213
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 3 Oct 2017 13:15:24 +0000 (09:15 -0400)]
8333: Support "myrepo.example:8888/repo/image"
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 3 Oct 2017 13:03:33 +0000 (09:03 -0400)]
8333: Support "myrepo.example:8888/repo/image:tag"
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 2 Oct 2017 18:25:42 +0000 (14:25 -0400)]
12213: Check for correct test output.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Mon, 2 Oct 2017 14:24:59 +0000 (11:24 -0300)]
Merge branch '12073-nodemanager-stale-nodes-recs-bis'
Closes #12073
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Sat, 30 Sep 2017 03:02:19 +0000 (00:02 -0300)]
12073: Prioritize stale node records that have a slot_number when
trying to assign one to a cloud node.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Thu, 28 Sep 2017 19:15:10 +0000 (15:15 -0400)]
12213: Remapping from source to destination collection needs to apply to files as well.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 28 Sep 2017 18:44:48 +0000 (14:44 -0400)]
12213: Add test case.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 29 Sep 2017 20:33:12 +0000 (16:33 -0400)]
Merge branch '12278-cwl-debug-flag' closes #12278
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 29 Sep 2017 18:24:15 +0000 (14:24 -0400)]
Merge branch '12273-skip-special-files'
closes #12273
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 29 Sep 2017 18:20:54 +0000 (14:20 -0400)]
8333: Support "arv keep docker repo:tag" syntax.
"arv keep docker repo tag" still works, too.
"arv keep docker repo:tag tag" is an error.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Fri, 29 Sep 2017 18:16:08 +0000 (15:16 -0300)]
Merge branch '12341-slow-node-destroy'
Closes #12341
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Fri, 29 Sep 2017 14:37:17 +0000 (11:37 -0300)]
12341: Ensure shutdown_actor.stop() is called on node_finished_shutdown.
Updated test related to shutdown race condition.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Fri, 29 Sep 2017 12:56:35 +0000 (08:56 -0400)]
Merge branch '12347-obey-wf-reuse-hint'
closes #12347
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 03:09:16 +0000 (23:09 -0400)]
12347: Don't reuse runner job/container if arv:enableReuse is false.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Thu, 28 Sep 2017 21:09:09 +0000 (18:09 -0300)]
12341: If a node_destroy call succesfully destroys the VM but takes too long
to finish with additional tasks, the cloud_node_update may be called
in the meantime, removing the node from the cloud_node list. If this happens,
node_finished_shutdown() won't be able to retrieve the node, and shouldn't
raise an exception.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 20:54:28 +0000 (16:54 -0400)]
Merge branch '12347-disable-reuse'
refs #12347
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 18:52:01 +0000 (14:52 -0400)]
11583: Fix hung goroutines when stubbed container never runs.
Test suite memory usage goes down from >8GB to 1.5GB.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 18:19:55 +0000 (14:19 -0400)]
11583: Stop signal handlers on shutdown.
Reduces noise in goroutine dumps during test suite.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 17:57:21 +0000 (13:57 -0400)]
12273: Skip non-regular files when uploading outputs to Keep.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 28 Sep 2017 16:52:44 +0000 (12:52 -0400)]
12278: Need to set debug logging in crunch_script.py
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 28 Sep 2017 16:12:33 +0000 (12:12 -0400)]
12278: Bump schema-salad and cwltool for SourceLine fix.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 26 Sep 2017 19:49:43 +0000 (15:49 -0400)]
12278: Filter nameext and nameroot from final cwl.output.json
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 26 Sep 2017 13:44:57 +0000 (09:44 -0400)]
12278: SourceLine yields stack trace if debugging enabled
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 25 Sep 2017 20:10:39 +0000 (16:10 -0400)]
12278: Can provide custom schema salad to dev jobs build.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 25 Sep 2017 15:32:20 +0000 (11:32 -0400)]
12278: Propagate --debug flag to jobs.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 02:24:06 +0000 (22:24 -0400)]
12347: Fix --disable-reuse for jobs API.
The --disable-reuse flag was creating a pipeline instance containing a
job that ran cwl with reuse disabled, but it didn't set
properties.run_options.enable_job_reuse on the pipeline instance
itself. If you had run before with --disable-reuse, that previous run
would be eligible for reuse.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 02:16:32 +0000 (22:16 -0400)]
12347: Fix formatting.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 28 Sep 2017 02:11:11 +0000 (22:11 -0400)]
12347: Fix --disable-reuse for containers API.
The --disable-reuse flag was requesting a container to run
"arvados-cwl-runner --disable-reuse", but it didn't set
use_existing=False on the container request itself. If you had run
before with --disable-reuse, that previous run would be eligible for
reuse.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Ward Vandewege [Wed, 27 Sep 2017 21:57:16 +0000 (17:57 -0400)]
Update documentation to reflect that we are packaging Debian 9
('stretch').
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Peter Amstutz [Wed, 27 Sep 2017 18:21:32 +0000 (14:21 -0400)]
12246: Make "possible causes" message easier to find and read. refs #12246
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Wed, 27 Sep 2017 17:01:36 +0000 (13:01 -0400)]
Merge branch '12246-command-not-found'
closes #12246
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 27 Sep 2017 15:26:22 +0000 (11:26 -0400)]
12246: Add debugging clues when command is not found in container.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Wed, 27 Sep 2017 14:05:14 +0000 (11:05 -0300)]
Merge branch '12195-nodemanager-quota-error'
Closes #12195
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 27 Sep 2017 13:39:36 +0000 (10:39 -0300)]
12195: Fixed warning message when the Retry-After header value cannot
be converted to integer.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Tue, 26 Sep 2017 20:29:42 +0000 (16:29 -0400)]
Suppress superfluous error message.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 26 Sep 2017 19:39:00 +0000 (15:39 -0400)]
12032: Fix compatibility with PostgreSQL 9.4.
9.4 doesn't have idle_in_transaction_session_timeout or row_security
configs, so couldn't execute structure.sql.
9.4 doesn't write "Dumped from/by version X", so 9.4 and 9.6 dev boxes
want an edit war.
refs #12032
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 26 Sep 2017 19:28:32 +0000 (15:28 -0400)]
Merge branch '12287-preserve-json-numbers' closes #12287
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 26 Sep 2017 17:56:04 +0000 (13:56 -0400)]
12287: Add number round-trip test.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 26 Sep 2017 02:43:10 +0000 (22:43 -0400)]
12287: Preserve JSON number types/formats in log file.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Ward Vandewege [Tue, 26 Sep 2017 02:12:55 +0000 (22:12 -0400)]
Merge branch '12316-fix-provenance-graph'
closes #12316
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Mon, 25 Sep 2017 20:53:21 +0000 (16:53 -0400)]
Do not blow up the provenance graph if a PDH used in the workflow no
longer exists.
refs #12316
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Tom Clegg [Fri, 22 Sep 2017 21:00:07 +0000 (17:00 -0400)]
12276: Ensure dirent cache is disabled.
Setting this to zero fixed flakiness in #7751, so we might still rely
on it being zero for reliability.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 22 Sep 2017 18:43:26 +0000 (14:43 -0400)]
12276: Remove debug logging message.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 22 Sep 2017 15:21:07 +0000 (11:21 -0400)]
12276: Reduce number of spurious invalidations sent to kernel.
* New policy to only send invalidations on objects that have a nonzero
kernel reference count.
* When clearing the contents of a CollectionDirectory, only invalidate the
topmost directory entry, this should take care of invalidating all paths
beneath it as well.
* Don't send invalidate_inode() when deleting an inode (by definition, the
inode is unreferenced by the kernel).
* Remove allow_dirent_cache for now (this only relates to caching negative
lookups, which we don't support).
* Set attribute attr_timeout based on the polling period of the underlying
object.
* FuncToJSONFile sets allow_attr_cache = False instead of allow_dirent_cache =
False (which is useless in this case).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Wed, 20 Sep 2017 21:44:22 +0000 (17:44 -0400)]
12298: Allow non-null log when cancelling an unrunnable container.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 22 Sep 2017 18:58:34 +0000 (14:58 -0400)]
12287: Preserve JSON number types/formats in container records.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 25 Sep 2017 19:39:50 +0000 (15:39 -0400)]
Merge branch '12032-project-trash' refs #12032
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Javier Bértoli [Mon, 25 Sep 2017 19:00:02 +0000 (16:00 -0300)]
Merge branch '12271-fpm-cleanup-on-rpm-upgrade' closes #12271
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@veritasgenetics.com>
Lucas Di Pentima [Mon, 25 Sep 2017 13:44:26 +0000 (10:44 -0300)]
12195: Log warning message when a retry-after header is not an integer.
Make sure min_retry_wait & max_retry_wait have sensible values.
Check that the effective retry wait is within bounds and doesn't get reset
when retry-after is zero, so it continues the exponential backoff plan.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Mon, 25 Sep 2017 13:19:50 +0000 (09:19 -0400)]
Merge branch '11068-cwl-missing-docker' closes #11068
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Sat, 23 Sep 2017 00:38:00 +0000 (20:38 -0400)]
Merge branch '12276-fuse-deadlock' refs #12276
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 22 Sep 2017 21:00:07 +0000 (17:00 -0400)]
12276: Ensure dirent cache is disabled.
Setting this to zero fixed flakiness in #7751, so we might still rely
on it being zero for reliability.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 22 Sep 2017 18:43:26 +0000 (14:43 -0400)]
12276: Remove debug logging message.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 22 Sep 2017 15:21:07 +0000 (11:21 -0400)]
12276: Reduce number of spurious invalidations sent to kernel.
* New policy to only send invalidations on objects that have a nonzero
kernel reference count.
* When clearing the contents of a CollectionDirectory, only invalidate the
topmost directory entry, this should take care of invalidating all paths
beneath it as well.
* Don't send invalidate_inode() when deleting an inode (by definition, the
inode is unreferenced by the kernel).
* Remove allow_dirent_cache for now (this only relates to caching negative
lookups, which we don't support).
* Set attribute attr_timeout based on the polling period of the underlying
object.
* FuncToJSONFile sets allow_attr_cache = False instead of allow_dirent_cache =
False (which is useless in this case).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Fri, 22 Sep 2017 13:27:01 +0000 (09:27 -0400)]
Merge branch '12298-cancel-fail'
refs #12298
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 21 Sep 2017 18:04:27 +0000 (14:04 -0400)]
11068: Catch OSError and give better error message when unable to execute Docker.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 21 Sep 2017 15:38:49 +0000 (11:38 -0400)]
Merge branch '12292-cwl-tmpdir-default' refs #12292
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 18 Sep 2017 21:40:05 +0000 (17:40 -0400)]
Install upstream pip to work around Debian bug, no issue #
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 20 Sep 2017 20:05:18 +0000 (16:05 -0400)]
12292: Find crunchrunner vars even when mixed into a partial line
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 20 Sep 2017 19:24:34 +0000 (15:24 -0400)]
12292: Set defaults for tmpdir, outdir, keep
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Javier Bértoli [Wed, 20 Sep 2017 22:51:54 +0000 (19:51 -0300)]
Add parameter for FPM so it removes old directories on upgrade
refs #12271
Tom Clegg [Wed, 20 Sep 2017 21:44:22 +0000 (17:44 -0400)]
12298: Allow non-null log when cancelling an unrunnable container.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 20 Sep 2017 20:03:53 +0000 (16:03 -0400)]
Fix dashboard crash on uncommitted container request.
refs #12246
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Wed, 20 Sep 2017 20:01:08 +0000 (17:01 -0300)]
12195: Catch the special case when a RateLimitReachedError exception is raised,
as it doesn't have a header attribute like its parent class.
Updated integration test to add a retry with this kind of error.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Wed, 20 Sep 2017 15:33:31 +0000 (11:33 -0400)]
Merge branch '12247-no-failed-checksum'
closes #12247
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 20 Sep 2017 13:57:31 +0000 (09:57 -0400)]
Merge branch '12107-faster-git'
closes #12107
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 20 Sep 2017 13:54:47 +0000 (09:54 -0400)]
12107: Add protective comment.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
projects / arvados.git / log