arvados.git
5 years agoMerge branch 'master' into 13823-bionic
Ward Vandewege [Wed, 22 Aug 2018 10:29:33 +0000 (06:29 -0400)]
Merge branch 'master' into 13823-bionic

refs #13823

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoWe no longer build for Ubuntu 12.04
Ward Vandewege [Wed, 22 Aug 2018 10:18:53 +0000 (06:18 -0400)]
We no longer build for Ubuntu 12.04

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '13924-collections-index'
Tom Clegg [Tue, 21 Aug 2018 15:55:51 +0000 (11:55 -0400)]
Merge branch '13924-collections-index'

closes #13924

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14082-numbertype' refs #14082
Peter Amstutz [Tue, 21 Aug 2018 15:30:05 +0000 (11:30 -0400)]
Merge branch '14082-numbertype' refs #14082

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14020-webdav-cors'
Tom Clegg [Tue, 21 Aug 2018 14:20:33 +0000 (10:20 -0400)]
Merge branch '14020-webdav-cors'

closes #14020

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14082: Fix test
Peter Amstutz [Tue, 21 Aug 2018 01:51:04 +0000 (21:51 -0400)]
14082: Fix test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14082: Datatype validation for command, environment, mounts
Peter Amstutz [Tue, 21 Aug 2018 01:20:21 +0000 (21:20 -0400)]
14082: Datatype validation for command, environment, mounts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13924: Drop redundant index.
Tom Clegg [Mon, 20 Aug 2018 14:03:00 +0000 (10:03 -0400)]
13924: Drop redundant index.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13924: Add index for PDH lookups.
Tom Clegg [Mon, 20 Aug 2018 13:17:26 +0000 (09:17 -0400)]
13924: Add index for PDH lookups.

Commit 2ee4ee2bb666835c41019b475dd2e286604bdd35 changed PDH lookup
queries to sort on trash_at, which can be slow without an index.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14020: Allow WebDAV headers in CORS requests.
Tom Clegg [Mon, 20 Aug 2018 13:42:59 +0000 (09:42 -0400)]
14020: Allow WebDAV headers in CORS requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13146-shared-rails' refs #13146
Peter Amstutz [Fri, 17 Aug 2018 17:13:47 +0000 (13:13 -0400)]
Merge branch '13146-shared-rails' refs #13146

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Clarify documentation for shared endpoint
Peter Amstutz [Fri, 17 Aug 2018 15:23:38 +0000 (11:23 -0400)]
13146: Clarify documentation for shared endpoint

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13340-wb-keep-links'
Lucas Di Pentima [Fri, 17 Aug 2018 15:01:58 +0000 (12:01 -0300)]
Merge branch '13340-wb-keep-links'
Closes #13340

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13933: Update error message expectation.
Tom Clegg [Fri, 17 Aug 2018 14:32:11 +0000 (10:32 -0400)]
13933: Update error message expectation.

refs #13933

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13340: Enhance regex to match non quoted locators
Lucas Di Pentima [Thu, 16 Aug 2018 21:40:14 +0000 (18:40 -0300)]
13340: Enhance regex to match non quoted locators

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '13933-dispatch-batch-size'
Tom Clegg [Thu, 16 Aug 2018 21:25:11 +0000 (17:25 -0400)]
Merge branch '13933-dispatch-batch-size'

closes #13933

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13933: Update error text check.
Tom Clegg [Thu, 16 Aug 2018 21:23:59 +0000 (17:23 -0400)]
13933: Update error text check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13933: Fix formatting.
Tom Clegg [Thu, 16 Aug 2018 21:23:50 +0000 (17:23 -0400)]
13933: Fix formatting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13146: Don't return "included" field unless non-nil
Peter Amstutz [Thu, 16 Aug 2018 19:36:30 +0000 (15:36 -0400)]
13146: Don't return "included" field unless non-nil

Update documention/comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: "included" is returned as_api_response.
Peter Amstutz [Thu, 16 Aug 2018 19:25:39 +0000 (15:25 -0400)]
13146: "included" is returned as_api_response.

Fix applying column select to "included" records.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Apply select clause to included objects
Peter Amstutz [Thu, 16 Aug 2018 19:08:13 +0000 (15:08 -0400)]
13146: Apply select clause to included objects

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Add "include" parameter and rename return field to "included"
Peter Amstutz [Thu, 16 Aug 2018 18:48:36 +0000 (14:48 -0400)]
13146: Add "include" parameter and rename return field to "included"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13752-index-all-filenames'
Tom Clegg [Thu, 16 Aug 2018 18:11:45 +0000 (14:11 -0400)]
Merge branch '13752-index-all-filenames'

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13752: Don't rewrite entire file_names column during the migration.
Tom Clegg [Thu, 16 Aug 2018 17:52:07 +0000 (13:52 -0400)]
13752: Don't rewrite entire file_names column during the migration.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13340: Correctly show all locator links on a text.
Lucas Di Pentima [Wed, 15 Aug 2018 20:49:05 +0000 (17:49 -0300)]
13340: Correctly show all locator links on a text.

This solve 2 issues:
* Show all locator links on text lines with multiple locators
* Fix the locator regex so that it doesn't include incorrect text parts

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13340: Add test to expose bug
Lucas Di Pentima [Thu, 16 Aug 2018 15:23:57 +0000 (12:23 -0300)]
13340: Add test to expose bug

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13146: FUSE tests pass
Peter Amstutz [Thu, 16 Aug 2018 15:31:36 +0000 (11:31 -0400)]
13146: FUSE tests pass

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: FUSE uses the "groups.shared" API for shared/ directory.
Peter Amstutz [Thu, 16 Aug 2018 15:04:25 +0000 (11:04 -0400)]
13146: FUSE uses the "groups.shared" API for shared/ directory.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Add tests for shared endpoint.
Peter Amstutz [Thu, 16 Aug 2018 14:38:55 +0000 (10:38 -0400)]
13146: Add tests for shared endpoint.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13198-metrics-docs'
Tom Clegg [Thu, 16 Aug 2018 14:14:28 +0000 (10:14 -0400)]
Merge branch '13198-metrics-docs'

closes #13198

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13198: Note keep-web metrics in admin docs.
Tom Clegg [Thu, 16 Aug 2018 14:14:10 +0000 (10:14 -0400)]
13198: Note keep-web metrics in admin docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13146: Update comment
Peter Amstutz [Wed, 15 Aug 2018 18:44:27 +0000 (14:44 -0400)]
13146: Update comment

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Support for query filters, "includes"
Peter Amstutz [Wed, 15 Aug 2018 18:28:13 +0000 (14:28 -0400)]
13146: Support for query filters, "includes"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14028-wb-textile-html-sanitization'
Lucas Di Pentima [Tue, 14 Aug 2018 19:38:54 +0000 (16:38 -0300)]
Merge branch '14028-wb-textile-html-sanitization'
Closes #14028

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()
Lucas Di Pentima [Tue, 14 Aug 2018 18:01:28 +0000 (15:01 -0300)]
14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch 'jrandall/patch-4' refs #14017
Peter Amstutz [Tue, 14 Aug 2018 17:49:17 +0000 (13:49 -0400)]
Merge branch 'jrandall/patch-4' refs #14017

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14028: Sanitize RedCloth's output.
Lucas Di Pentima [Tue, 14 Aug 2018 17:01:36 +0000 (14:01 -0300)]
14028: Sanitize RedCloth's output.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14028: Add test exposing the bug
Lucas Di Pentima [Tue, 14 Aug 2018 17:32:47 +0000 (14:32 -0300)]
14028: Add test exposing the bug

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMake dispatcher Run loop use count=none and BatchSize limit
Joshua C. Randall [Mon, 13 Aug 2018 12:01:10 +0000 (12:01 +0000)]
Make dispatcher Run loop use count=none and BatchSize limit

Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>

5 years agoMerge branch '13931-uri-unquote' refs #13931
Peter Amstutz [Mon, 13 Aug 2018 20:14:59 +0000 (16:14 -0400)]
Merge branch '13931-uri-unquote' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: un-uriquote keep: URIs when converting to (collection, path) tuple
Peter Amstutz [Mon, 13 Aug 2018 18:50:53 +0000 (14:50 -0400)]
13931: un-uriquote keep: URIs when converting to (collection, path) tuple

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agofix "submitted container [uuid]" log from a-c-r
Joshua C. Randall [Mon, 13 Aug 2018 14:47:03 +0000 (15:47 +0100)]
fix "submitted container [uuid]" log from a-c-r

fixes 14017

Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>

5 years agoNote slow migration in upgrade notes.
Tom Clegg [Mon, 13 Aug 2018 14:10:36 +0000 (10:10 -0400)]
Note slow migration in upgrade notes.

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13399-nb-squeue-check'
Tom Clegg [Mon, 13 Aug 2018 13:37:28 +0000 (09:37 -0400)]
Merge branch '13399-nb-squeue-check'

refs #13399

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoFix version dependency on 'typing' refs #13931
Peter Amstutz [Fri, 10 Aug 2018 18:28:17 +0000 (14:28 -0400)]
Fix version dependency on 'typing' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13976-acr-keep-references' refs #13976
Peter Amstutz [Fri, 10 Aug 2018 15:17:19 +0000 (11:17 -0400)]
Merge branch '13976-acr-keep-references' refs #13976

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13976: Add code comment about only_real in upload_dependencies
Peter Amstutz [Fri, 10 Aug 2018 14:30:55 +0000 (10:30 -0400)]
13976: Add code comment about only_real in upload_dependencies

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13976: Filter out keep references with secondaryFiles from list of files to upload
Peter Amstutz [Fri, 10 Aug 2018 14:14:31 +0000 (10:14 -0400)]
13976: Filter out keep references with secondaryFiles from list of files to upload

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13399: Don't block job submissions during release/renice batches.
Tom Clegg [Thu, 9 Aug 2018 20:53:06 +0000 (16:53 -0400)]
13399: Don't block job submissions during release/renice batches.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch 'wtsi-hgi-13558-debug-log-tag-req-id'
Tom Clegg [Thu, 9 Aug 2018 19:27:57 +0000 (15:27 -0400)]
Merge branch 'wtsi-hgi-13558-debug-log-tag-req-id'

closes #13558

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id
Tom Clegg [Thu, 9 Aug 2018 19:06:28 +0000 (15:06 -0400)]
13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoFix version of 'typing' dep refs #13931
Peter Amstutz [Thu, 9 Aug 2018 16:53:45 +0000 (12:53 -0400)]
Fix version of 'typing' dep refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13146: Shared endpoint wip
Peter Amstutz [Thu, 9 Aug 2018 16:44:11 +0000 (12:44 -0400)]
13146: Shared endpoint wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13976: Fix keep reference error
Lucas Di Pentima [Wed, 8 Aug 2018 20:13:30 +0000 (17:13 -0300)]
13976: Fix keep reference error

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13976: Add CWL test exposing the bug
Lucas Di Pentima [Thu, 9 Aug 2018 14:47:28 +0000 (11:47 -0300)]
13976: Add CWL test exposing the bug

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '13939-unqualified-contents-orders'
Tom Clegg [Thu, 9 Aug 2018 14:18:05 +0000 (10:18 -0400)]
Merge branch '13939-unqualified-contents-orders'

fixes #13939

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13752-index-all-filenames'
Tom Clegg [Thu, 9 Aug 2018 13:38:32 +0000 (09:38 -0400)]
Merge branch '13752-index-all-filenames'

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13931-cwltool-deps' refs #13931
Peter Amstutz [Thu, 9 Aug 2018 13:30:09 +0000 (09:30 -0400)]
Merge branch '13931-cwltool-deps' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: Reorganize for consistency.
Peter Amstutz [Thu, 9 Aug 2018 13:29:13 +0000 (09:29 -0400)]
13931: Reorganize for consistency.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: package deps for centos7
Peter Amstutz [Thu, 9 Aug 2018 13:28:22 +0000 (09:28 -0400)]
13931: package deps for centos7

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: Fix "don't rebuild packages that already exist" logic
Peter Amstutz [Wed, 8 Aug 2018 21:10:03 +0000 (17:10 -0400)]
13931: Fix "don't rebuild packages that already exist" logic

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: Add dependencies required by latest cwltool
Peter Amstutz [Wed, 8 Aug 2018 20:50:36 +0000 (16:50 -0400)]
13931: Add dependencies required by latest cwltool

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13198-keep-web-metrics'
Tom Clegg [Wed, 8 Aug 2018 20:46:29 +0000 (16:46 -0400)]
Merge branch '13198-keep-web-metrics'

refs #13198

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13198: Remove old counters from status.json.
Tom Clegg [Tue, 7 Aug 2018 22:09:25 +0000 (18:09 -0400)]
13198: Remove old counters from status.json.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13399-lock-starvation'
Tom Clegg [Tue, 7 Aug 2018 20:19:05 +0000 (16:19 -0400)]
Merge branch '13399-lock-starvation'

refs #13399

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13931-cwl-size' refs #13931
Peter Amstutz [Tue, 7 Aug 2018 20:17:34 +0000 (16:17 -0400)]
Merge branch '13931-cwl-size' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13823: Accept ruamel.yaml 0.15.x, build 0.15.34.
Tom Clegg [Tue, 7 Aug 2018 18:24:29 +0000 (14:24 -0400)]
13823: Accept ruamel.yaml 0.15.x, build 0.15.34.

ubuntu bionic ships 0.15.34, making it awkward to depend on an older
version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13931: Update tests for sizes being added
Peter Amstutz [Tue, 7 Aug 2018 17:32:08 +0000 (13:32 -0400)]
13931: Update tests for sizes being added

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13823: Use pip instead of easy_install if easy_install3 is missing.
Tom Clegg [Tue, 7 Aug 2018 17:24:14 +0000 (13:24 -0400)]
13823: Use pip instead of easy_install if easy_install3 is missing.

easy_install3 no longer exists in ubuntu:bionic.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13823: Switch to stock fpm.
Tom Clegg [Tue, 7 Aug 2018 17:07:28 +0000 (13:07 -0400)]
13823: Switch to stock fpm.

Upstream has --python-disable-dependency now, so we don't need to use
a fork.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13823: Upgrade ws4py to version that ships in ubuntu 1804.
Tom Clegg [Tue, 7 Aug 2018 17:06:50 +0000 (13:06 -0400)]
13823: Upgrade ws4py to version that ships in ubuntu 1804.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13823: Build packages for ubuntu1804.
Tom Clegg [Mon, 6 Aug 2018 23:33:03 +0000 (19:33 -0400)]
13823: Build packages for ubuntu1804.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13752: Remove column size limit.
Tom Clegg [Mon, 6 Aug 2018 21:22:48 +0000 (17:22 -0400)]
13752: Remove column size limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13931: Add keep size test to test list and fix Dockerfile for dev jobs image
Peter Amstutz [Mon, 6 Aug 2018 20:59:10 +0000 (16:59 -0400)]
13931: Add keep size test to test list and fix Dockerfile for dev jobs image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13399: Don't spam logs about low priority jobs that are running.
Tom Clegg [Mon, 6 Aug 2018 20:07:06 +0000 (16:07 -0400)]
13399: Don't spam logs about low priority jobs that are running.

Once a job has started, its priority relative to other jobs doesn't
matter to us anyway, and we already skip low-priority jobs in the
"renice" procedure, so there's no reason to warn about this.

Also change "mysterious low priority" reporting threshold from 1Mi to
20K to match the other "nice/hold race" code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13931: Update cwltool for add_sizes fix
Peter Amstutz [Mon, 6 Aug 2018 19:54:35 +0000 (15:54 -0400)]
13931: Update cwltool for add_sizes fix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: Support for getting file sizes of files in Keep
Peter Amstutz [Mon, 6 Aug 2018 18:14:04 +0000 (14:14 -0400)]
13931: Support for getting file sizes of files in Keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13931: Add test that file sizes are available.
Peter Amstutz [Mon, 6 Aug 2018 18:13:16 +0000 (14:13 -0400)]
13931: Add test that file sizes are available.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13399: Adjust locking strategy to avoid starvation.
Tom Clegg [Mon, 6 Aug 2018 16:59:30 +0000 (12:59 -0400)]
13399: Adjust locking strategy to avoid starvation.

Don't acquire the slurm lock to call scancel or sbatch.

Don't acquire the slurm lock again immediately after updating the
queue and broadcasting. Instead, give readers a chance to reacquire
the lock after Wait().

Acquire a shared lock instead of an exclusive lock in HasUUID() or
SetPriority() when the protected operation is a read.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13752: Dump structure with postgresql 9.6.
Tom Clegg [Mon, 6 Aug 2018 14:26:27 +0000 (10:26 -0400)]
13752: Dump structure with postgresql 9.6.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13399-renice-too-low-priority-jobs' refs #13399
Peter Amstutz [Mon, 6 Aug 2018 12:56:58 +0000 (08:56 -0400)]
Merge branch '13399-renice-too-low-priority-jobs' refs #13399

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13399: Don't try to bury jobs below very-low-priority jobs.
Tom Clegg [Sat, 4 Aug 2018 15:05:45 +0000 (11:05 -0400)]
13399: Don't try to bury jobs below very-low-priority jobs.

Non-zero priorities below 20K are probably the result of renicing jobs
that have priority=0 because they are in admin-hold state.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13961-separate-polling' refs #13961
Peter Amstutz [Fri, 3 Aug 2018 18:31:13 +0000 (14:31 -0400)]
Merge branch '13961-separate-polling' refs #13961

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13961: Use poll_time as fallback
Peter Amstutz [Fri, 3 Aug 2018 18:29:05 +0000 (14:29 -0400)]
13961: Use poll_time as fallback

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time
Peter Amstutz [Fri, 3 Aug 2018 17:01:49 +0000 (13:01 -0400)]
13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '12762-safeyaml-warning'
Lucas Di Pentima [Fri, 3 Aug 2018 14:54:19 +0000 (11:54 -0300)]
Merge branch '12762-safeyaml-warning'
Closes #12762

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago12762: Suppress SafeYAML warnings by setting its default mode.
Lucas Di Pentima [Thu, 2 Aug 2018 18:59:36 +0000 (15:59 -0300)]
12762: Suppress SafeYAML warnings by setting its default mode.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '13609-r-sdk-copy' refs #13609
Peter Amstutz [Thu, 2 Aug 2018 18:00:37 +0000 (14:00 -0400)]
Merge branch '13609-r-sdk-copy' refs #13609

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13609: Note that move/copy between collections isn't supported
Peter Amstutz [Thu, 2 Aug 2018 17:59:37 +0000 (13:59 -0400)]
13609: Note that move/copy between collections isn't supported

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13609: Spelling fix
Peter Amstutz [Thu, 2 Aug 2018 17:53:47 +0000 (13:53 -0400)]
13609: Spelling fix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13609: Improve README fomatting a bit.
Peter Amstutz [Thu, 2 Aug 2018 17:52:35 +0000 (13:52 -0400)]
13609: Improve README fomatting a bit.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13609: Update version to 0.0.5
Peter Amstutz [Thu, 2 Aug 2018 17:40:28 +0000 (13:40 -0400)]
13609: Update version to 0.0.5

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '13934-guard-comment'
Tom Clegg [Thu, 2 Aug 2018 04:13:15 +0000 (00:13 -0400)]
Merge branch '13934-guard-comment'

closes #13934

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13939: In groups#contents, apply unprefixed orders to all tables.
Tom Clegg [Wed, 1 Aug 2018 17:40:05 +0000 (13:40 -0400)]
13939: In groups#contents, apply unprefixed orders to all tables.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13939: Test orders with no table name qualifier.
Tom Clegg [Wed, 1 Aug 2018 17:13:50 +0000 (13:13 -0400)]
13939: Test orders with no table name qualifier.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13934: Add comment to guard dispatch bugfix.
Tom Clegg [Wed, 1 Aug 2018 16:56:31 +0000 (12:56 -0400)]
13934: Add comment to guard dispatch bugfix.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13497-controller'
Tom Clegg [Wed, 1 Aug 2018 13:14:14 +0000 (09:14 -0400)]
Merge branch '13497-controller'

refs #13497

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13198-keep-web-metrics'
Tom Clegg [Tue, 31 Jul 2018 18:04:39 +0000 (14:04 -0400)]
Merge branch '13198-keep-web-metrics'

refs #13198

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '13892-deprecate-jobs-api' closes #13892
Peter Amstutz [Tue, 31 Jul 2018 17:51:05 +0000 (13:51 -0400)]
Merge branch '13892-deprecate-jobs-api' closes #13892

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13892: Make warning more obvious.
Peter Amstutz [Tue, 31 Jul 2018 17:26:14 +0000 (13:26 -0400)]
13892: Make warning more obvious.

Link to information about migrating for both users and admins.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>