Tom Clegg [Fri, 24 Aug 2018 16:06:56 +0000 (12:06 -0400)]
Merge branch '13959-timeouts-and-logging'
fixes #13959
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 24 Aug 2018 16:06:18 +0000 (12:06 -0400)]
13959: Merge branch 'master' into 13959-timeouts-and-logging
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Ward Vandewege [Fri, 24 Aug 2018 05:50:41 +0000 (01:50 -0400)]
We need to build ws 0.4.2 for ubuntu1404 and ubuntu1604
refs #13823
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Fri, 24 Aug 2018 05:49:40 +0000 (01:49 -0400)]
Switch to use ha.pool.sks-keyservers.net everywhere, which is more reliable
than pool.sks-keyservers.net
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Thu, 23 Aug 2018 21:39:56 +0000 (17:39 -0400)]
Improvement for run-tests.sh: support multiple occurrences of the
--only argument - syntax fix.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Thu, 23 Aug 2018 21:27:18 +0000 (17:27 -0400)]
Bugfix for run-tests.sh: run sdk/R install if docs are being built,
they rely on it.
Improvement for run-tests.sh: support multiple occurrences of the
--only argument.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Thu, 23 Aug 2018 16:16:08 +0000 (12:16 -0400)]
Fix for run-tests.sh: do not run sdk/R install if the --skip sdk/R option is
set.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Thu, 23 Aug 2018 13:01:27 +0000 (09:01 -0400)]
Merge branch '13823-bionic'
refs #13823
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 22 Aug 2018 12:40:41 +0000 (08:40 -0400)]
Add support for package testing to our ubuntu 18.04 packages.
refs #13823
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 22 Aug 2018 12:38:34 +0000 (08:38 -0400)]
Remove spurious error output in package testing script.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 22 Aug 2018 12:38:34 +0000 (08:38 -0400)]
Remove spurious error output in package testing script.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 22 Aug 2018 10:29:33 +0000 (06:29 -0400)]
Merge branch 'master' into 13823-bionic
refs #13823
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Ward Vandewege [Wed, 22 Aug 2018 10:18:53 +0000 (06:18 -0400)]
We no longer build for Ubuntu 12.04
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>
Tom Clegg [Tue, 21 Aug 2018 17:18:26 +0000 (13:18 -0400)]
13959: Fix test regexps.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 21 Aug 2018 16:51:22 +0000 (12:51 -0400)]
13959: Merge branch 'master' into 13959-timeouts-and-logging
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 21 Aug 2018 15:55:51 +0000 (11:55 -0400)]
Merge branch '13924-collections-index'
closes #13924
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 21 Aug 2018 15:30:05 +0000 (11:30 -0400)]
Merge branch '14082-numbertype' refs #14082
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 21 Aug 2018 14:20:33 +0000 (10:20 -0400)]
Merge branch '14020-webdav-cors'
closes #14020
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 21 Aug 2018 01:51:04 +0000 (21:51 -0400)]
14082: Fix test
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Tue, 21 Aug 2018 01:20:21 +0000 (21:20 -0400)]
14082: Datatype validation for command, environment, mounts
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 20 Aug 2018 14:03:00 +0000 (10:03 -0400)]
13924: Drop redundant index.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 20 Aug 2018 13:17:26 +0000 (09:17 -0400)]
13924: Add index for PDH lookups.
Commit
2ee4ee2bb666835c41019b475dd2e286604bdd35 changed PDH lookup
queries to sort on trash_at, which can be slow without an index.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 20 Aug 2018 13:42:59 +0000 (09:42 -0400)]
14020: Allow WebDAV headers in CORS requests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 17 Aug 2018 17:13:47 +0000 (13:13 -0400)]
Merge branch '13146-shared-rails' refs #13146
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 17 Aug 2018 15:23:38 +0000 (11:23 -0400)]
13146: Clarify documentation for shared endpoint
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Fri, 17 Aug 2018 15:01:58 +0000 (12:01 -0300)]
Merge branch '13340-wb-keep-links'
Closes #13340
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Fri, 17 Aug 2018 14:38:39 +0000 (10:38 -0400)]
13959: Merge branch 'master' into 13959-timeouts-and-logging
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 17 Aug 2018 14:32:11 +0000 (10:32 -0400)]
13933: Update error message expectation.
refs #13933
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 17 Aug 2018 14:19:29 +0000 (10:19 -0400)]
13959: Use logrus for crunch-dispatch-local logging.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Thu, 16 Aug 2018 21:40:14 +0000 (18:40 -0300)]
13340: Enhance regex to match non quoted locators
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 21:25:11 +0000 (17:25 -0400)]
Merge branch '13933-dispatch-batch-size'
closes #13933
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 21:23:59 +0000 (17:23 -0400)]
13933: Update error text check.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 21:23:50 +0000 (17:23 -0400)]
13933: Fix formatting.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 19:36:30 +0000 (15:36 -0400)]
13146: Don't return "included" field unless non-nil
Update documention/comments.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 19:38:42 +0000 (15:38 -0400)]
13959: Use logrus for crunch-dispatch-slurm logging.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 19:25:39 +0000 (15:25 -0400)]
13146: "included" is returned as_api_response.
Fix applying column select to "included" records.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 19:08:13 +0000 (15:08 -0400)]
13146: Apply select clause to included objects
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 18:53:21 +0000 (14:53 -0400)]
13959: Default 5-minute timeout for API calls.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 18:48:36 +0000 (14:48 -0400)]
13146: Add "include" parameter and rename return field to "included"
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 18:11:45 +0000 (14:11 -0400)]
Merge branch '13752-index-all-filenames'
refs #13752
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 17:52:07 +0000 (13:52 -0400)]
13752: Don't rewrite entire file_names column during the migration.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Wed, 15 Aug 2018 20:49:05 +0000 (17:49 -0300)]
13340: Correctly show all locator links on a text.
This solve 2 issues:
* Show all locator links on text lines with multiple locators
* Fix the locator regex so that it doesn't include incorrect text parts
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 16 Aug 2018 15:23:57 +0000 (12:23 -0300)]
13340: Add test to expose bug
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 15:31:36 +0000 (11:31 -0400)]
13146: FUSE tests pass
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 15:04:25 +0000 (11:04 -0400)]
13146: FUSE uses the "groups.shared" API for shared/ directory.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 16 Aug 2018 14:38:55 +0000 (10:38 -0400)]
13146: Add tests for shared endpoint.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 14:14:28 +0000 (10:14 -0400)]
Merge branch '13198-metrics-docs'
closes #13198
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 16 Aug 2018 14:14:10 +0000 (10:14 -0400)]
13198: Note keep-web metrics in admin docs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Wed, 15 Aug 2018 18:44:27 +0000 (14:44 -0400)]
13146: Update comment
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 15 Aug 2018 18:28:13 +0000 (14:28 -0400)]
13146: Support for query filters, "includes"
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Tue, 14 Aug 2018 19:38:54 +0000 (16:38 -0300)]
Merge branch '14028-wb-textile-html-sanitization'
Closes #14028
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 14 Aug 2018 18:01:28 +0000 (15:01 -0300)]
14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Tue, 14 Aug 2018 17:49:17 +0000 (13:49 -0400)]
Merge branch 'jrandall/patch-4' refs #14017
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Tue, 14 Aug 2018 17:01:36 +0000 (14:01 -0300)]
14028: Sanitize RedCloth's output.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 14 Aug 2018 17:32:47 +0000 (14:32 -0300)]
14028: Add test exposing the bug
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Joshua C. Randall [Mon, 13 Aug 2018 12:01:10 +0000 (12:01 +0000)]
Make dispatcher Run loop use count=none and BatchSize limit
Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>
Peter Amstutz [Mon, 13 Aug 2018 20:14:59 +0000 (16:14 -0400)]
Merge branch '13931-uri-unquote' refs #13931
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 13 Aug 2018 18:50:53 +0000 (14:50 -0400)]
13931: un-uriquote keep: URIs when converting to (collection, path) tuple
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Joshua C. Randall [Mon, 13 Aug 2018 14:47:03 +0000 (15:47 +0100)]
fix "submitted container [uuid]" log from a-c-r
fixes 14017
Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>
Tom Clegg [Mon, 13 Aug 2018 14:10:36 +0000 (10:10 -0400)]
Note slow migration in upgrade notes.
refs #13752
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 13 Aug 2018 13:37:28 +0000 (09:37 -0400)]
Merge branch '13399-nb-squeue-check'
refs #13399
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 10 Aug 2018 18:28:17 +0000 (14:28 -0400)]
Fix version dependency on 'typing' refs #13931
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 10 Aug 2018 15:17:19 +0000 (11:17 -0400)]
Merge branch '13976-acr-keep-references' refs #13976
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 10 Aug 2018 14:30:55 +0000 (10:30 -0400)]
13976: Add code comment about only_real in upload_dependencies
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 10 Aug 2018 14:14:31 +0000 (10:14 -0400)]
13976: Filter out keep references with secondaryFiles from list of files to upload
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 9 Aug 2018 20:53:06 +0000 (16:53 -0400)]
13399: Don't block job submissions during release/renice batches.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 9 Aug 2018 19:27:57 +0000 (15:27 -0400)]
Merge branch 'wtsi-hgi-13558-debug-log-tag-req-id'
closes #13558
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 9 Aug 2018 19:06:28 +0000 (15:06 -0400)]
13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 9 Aug 2018 16:53:45 +0000 (12:53 -0400)]
Fix version of 'typing' dep refs #13931
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 9 Aug 2018 16:44:11 +0000 (12:44 -0400)]
13146: Shared endpoint wip
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Wed, 8 Aug 2018 20:13:30 +0000 (17:13 -0300)]
13976: Fix keep reference error
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 9 Aug 2018 14:47:28 +0000 (11:47 -0300)]
13976: Add CWL test exposing the bug
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 9 Aug 2018 14:18:05 +0000 (10:18 -0400)]
Merge branch '13939-unqualified-contents-orders'
fixes #13939
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 9 Aug 2018 13:38:32 +0000 (09:38 -0400)]
Merge branch '13752-index-all-filenames'
refs #13752
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 9 Aug 2018 13:30:09 +0000 (09:30 -0400)]
Merge branch '13931-cwltool-deps' refs #13931
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 9 Aug 2018 13:29:13 +0000 (09:29 -0400)]
13931: Reorganize for consistency.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 9 Aug 2018 13:28:22 +0000 (09:28 -0400)]
13931: package deps for centos7
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 8 Aug 2018 21:10:03 +0000 (17:10 -0400)]
13931: Fix "don't rebuild packages that already exist" logic
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Wed, 8 Aug 2018 20:50:36 +0000 (16:50 -0400)]
13931: Add dependencies required by latest cwltool
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Wed, 8 Aug 2018 20:46:29 +0000 (16:46 -0400)]
Merge branch '13198-keep-web-metrics'
refs #13198
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 22:09:25 +0000 (18:09 -0400)]
13198: Remove old counters from status.json.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 20:19:05 +0000 (16:19 -0400)]
Merge branch '13399-lock-starvation'
refs #13399
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 7 Aug 2018 20:17:34 +0000 (16:17 -0400)]
Merge branch '13931-cwl-size' refs #13931
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 18:24:29 +0000 (14:24 -0400)]
13823: Accept ruamel.yaml 0.15.x, build 0.15.34.
ubuntu bionic ships 0.15.34, making it awkward to depend on an older
version.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 7 Aug 2018 17:32:08 +0000 (13:32 -0400)]
13931: Update tests for sizes being added
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 17:24:14 +0000 (13:24 -0400)]
13823: Use pip instead of easy_install if easy_install3 is missing.
easy_install3 no longer exists in ubuntu:bionic.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 17:07:28 +0000 (13:07 -0400)]
13823: Switch to stock fpm.
Upstream has --python-disable-dependency now, so we don't need to use
a fork.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 7 Aug 2018 17:06:50 +0000 (13:06 -0400)]
13823: Upgrade ws4py to version that ships in ubuntu 1804.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 6 Aug 2018 23:33:03 +0000 (19:33 -0400)]
13823: Build packages for ubuntu1804.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 6 Aug 2018 21:22:48 +0000 (17:22 -0400)]
13752: Remove column size limit.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 6 Aug 2018 20:59:10 +0000 (16:59 -0400)]
13931: Add keep size test to test list and fix Dockerfile for dev jobs image
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 6 Aug 2018 20:07:06 +0000 (16:07 -0400)]
13399: Don't spam logs about low priority jobs that are running.
Once a job has started, its priority relative to other jobs doesn't
matter to us anyway, and we already skip low-priority jobs in the
"renice" procedure, so there's no reason to warn about this.
Also change "mysterious low priority" reporting threshold from 1Mi to
20K to match the other "nice/hold race" code.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 6 Aug 2018 19:54:35 +0000 (15:54 -0400)]
13931: Update cwltool for add_sizes fix
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 6 Aug 2018 18:14:04 +0000 (14:14 -0400)]
13931: Support for getting file sizes of files in Keep
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 6 Aug 2018 18:13:16 +0000 (14:13 -0400)]
13931: Add test that file sizes are available.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Mon, 6 Aug 2018 16:59:30 +0000 (12:59 -0400)]
13399: Adjust locking strategy to avoid starvation.
Don't acquire the slurm lock to call scancel or sbatch.
Don't acquire the slurm lock again immediately after updating the
queue and broadcasting. Instead, give readers a chance to reacquire
the lock after Wait().
Acquire a shared lock instead of an exclusive lock in HasUUID() or
SetPriority() when the protected operation is a read.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Mon, 6 Aug 2018 14:26:27 +0000 (10:26 -0400)]
13752: Dump structure with postgresql 9.6.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 6 Aug 2018 12:56:58 +0000 (08:56 -0400)]
Merge branch '13399-renice-too-low-priority-jobs' refs #13399
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Sat, 4 Aug 2018 15:05:45 +0000 (11:05 -0400)]
13399: Don't try to bury jobs below very-low-priority jobs.
Non-zero priorities below 20K are probably the result of renicing jobs
that have priority=0 because they are in admin-hold state.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 3 Aug 2018 18:31:13 +0000 (14:31 -0400)]
Merge branch '13961-separate-polling' refs #13961
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
projects / arvados.git / log