arvados.git
6 years agoMerge branch '14028-wb-textile-html-sanitization'
Lucas Di Pentima [Tue, 14 Aug 2018 19:38:54 +0000 (16:38 -0300)]
Merge branch '14028-wb-textile-html-sanitization'
Closes #14028

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()
Lucas Di Pentima [Tue, 14 Aug 2018 18:01:28 +0000 (15:01 -0300)]
14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'jrandall/patch-4' refs #14017
Peter Amstutz [Tue, 14 Aug 2018 17:49:17 +0000 (13:49 -0400)]
Merge branch 'jrandall/patch-4' refs #14017

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14028: Sanitize RedCloth's output.
Lucas Di Pentima [Tue, 14 Aug 2018 17:01:36 +0000 (14:01 -0300)]
14028: Sanitize RedCloth's output.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14028: Add test exposing the bug
Lucas Di Pentima [Tue, 14 Aug 2018 17:32:47 +0000 (14:32 -0300)]
14028: Add test exposing the bug

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13931-uri-unquote' refs #13931
Peter Amstutz [Mon, 13 Aug 2018 20:14:59 +0000 (16:14 -0400)]
Merge branch '13931-uri-unquote' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: un-uriquote keep: URIs when converting to (collection, path) tuple
Peter Amstutz [Mon, 13 Aug 2018 18:50:53 +0000 (14:50 -0400)]
13931: un-uriquote keep: URIs when converting to (collection, path) tuple

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agofix "submitted container [uuid]" log from a-c-r
Joshua C. Randall [Mon, 13 Aug 2018 14:47:03 +0000 (15:47 +0100)]
fix "submitted container [uuid]" log from a-c-r

fixes 14017

Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>

6 years agoNote slow migration in upgrade notes.
Tom Clegg [Mon, 13 Aug 2018 14:10:36 +0000 (10:10 -0400)]
Note slow migration in upgrade notes.

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13399-nb-squeue-check'
Tom Clegg [Mon, 13 Aug 2018 13:37:28 +0000 (09:37 -0400)]
Merge branch '13399-nb-squeue-check'

refs #13399

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix version dependency on 'typing' refs #13931
Peter Amstutz [Fri, 10 Aug 2018 18:28:17 +0000 (14:28 -0400)]
Fix version dependency on 'typing' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13976-acr-keep-references' refs #13976
Peter Amstutz [Fri, 10 Aug 2018 15:17:19 +0000 (11:17 -0400)]
Merge branch '13976-acr-keep-references' refs #13976

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13976: Add code comment about only_real in upload_dependencies
Peter Amstutz [Fri, 10 Aug 2018 14:30:55 +0000 (10:30 -0400)]
13976: Add code comment about only_real in upload_dependencies

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13976: Filter out keep references with secondaryFiles from list of files to upload
Peter Amstutz [Fri, 10 Aug 2018 14:14:31 +0000 (10:14 -0400)]
13976: Filter out keep references with secondaryFiles from list of files to upload

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13399: Don't block job submissions during release/renice batches.
Tom Clegg [Thu, 9 Aug 2018 20:53:06 +0000 (16:53 -0400)]
13399: Don't block job submissions during release/renice batches.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch 'wtsi-hgi-13558-debug-log-tag-req-id'
Tom Clegg [Thu, 9 Aug 2018 19:27:57 +0000 (15:27 -0400)]
Merge branch 'wtsi-hgi-13558-debug-log-tag-req-id'

closes #13558

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id
Tom Clegg [Thu, 9 Aug 2018 19:06:28 +0000 (15:06 -0400)]
13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix version of 'typing' dep refs #13931
Peter Amstutz [Thu, 9 Aug 2018 16:53:45 +0000 (12:53 -0400)]
Fix version of 'typing' dep refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13976: Fix keep reference error
Lucas Di Pentima [Wed, 8 Aug 2018 20:13:30 +0000 (17:13 -0300)]
13976: Fix keep reference error

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13976: Add CWL test exposing the bug
Lucas Di Pentima [Thu, 9 Aug 2018 14:47:28 +0000 (11:47 -0300)]
13976: Add CWL test exposing the bug

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13939-unqualified-contents-orders'
Tom Clegg [Thu, 9 Aug 2018 14:18:05 +0000 (10:18 -0400)]
Merge branch '13939-unqualified-contents-orders'

fixes #13939

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13752-index-all-filenames'
Tom Clegg [Thu, 9 Aug 2018 13:38:32 +0000 (09:38 -0400)]
Merge branch '13752-index-all-filenames'

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13931-cwltool-deps' refs #13931
Peter Amstutz [Thu, 9 Aug 2018 13:30:09 +0000 (09:30 -0400)]
Merge branch '13931-cwltool-deps' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Reorganize for consistency.
Peter Amstutz [Thu, 9 Aug 2018 13:29:13 +0000 (09:29 -0400)]
13931: Reorganize for consistency.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: package deps for centos7
Peter Amstutz [Thu, 9 Aug 2018 13:28:22 +0000 (09:28 -0400)]
13931: package deps for centos7

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Fix "don't rebuild packages that already exist" logic
Peter Amstutz [Wed, 8 Aug 2018 21:10:03 +0000 (17:10 -0400)]
13931: Fix "don't rebuild packages that already exist" logic

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Add dependencies required by latest cwltool
Peter Amstutz [Wed, 8 Aug 2018 20:50:36 +0000 (16:50 -0400)]
13931: Add dependencies required by latest cwltool

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13198-keep-web-metrics'
Tom Clegg [Wed, 8 Aug 2018 20:46:29 +0000 (16:46 -0400)]
Merge branch '13198-keep-web-metrics'

refs #13198

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13198: Remove old counters from status.json.
Tom Clegg [Tue, 7 Aug 2018 22:09:25 +0000 (18:09 -0400)]
13198: Remove old counters from status.json.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13399-lock-starvation'
Tom Clegg [Tue, 7 Aug 2018 20:19:05 +0000 (16:19 -0400)]
Merge branch '13399-lock-starvation'

refs #13399

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13931-cwl-size' refs #13931
Peter Amstutz [Tue, 7 Aug 2018 20:17:34 +0000 (16:17 -0400)]
Merge branch '13931-cwl-size' refs #13931

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Update tests for sizes being added
Peter Amstutz [Tue, 7 Aug 2018 17:32:08 +0000 (13:32 -0400)]
13931: Update tests for sizes being added

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13752: Remove column size limit.
Tom Clegg [Mon, 6 Aug 2018 21:22:48 +0000 (17:22 -0400)]
13752: Remove column size limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13931: Add keep size test to test list and fix Dockerfile for dev jobs image
Peter Amstutz [Mon, 6 Aug 2018 20:59:10 +0000 (16:59 -0400)]
13931: Add keep size test to test list and fix Dockerfile for dev jobs image

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13399: Don't spam logs about low priority jobs that are running.
Tom Clegg [Mon, 6 Aug 2018 20:07:06 +0000 (16:07 -0400)]
13399: Don't spam logs about low priority jobs that are running.

Once a job has started, its priority relative to other jobs doesn't
matter to us anyway, and we already skip low-priority jobs in the
"renice" procedure, so there's no reason to warn about this.

Also change "mysterious low priority" reporting threshold from 1Mi to
20K to match the other "nice/hold race" code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13931: Update cwltool for add_sizes fix
Peter Amstutz [Mon, 6 Aug 2018 19:54:35 +0000 (15:54 -0400)]
13931: Update cwltool for add_sizes fix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Support for getting file sizes of files in Keep
Peter Amstutz [Mon, 6 Aug 2018 18:14:04 +0000 (14:14 -0400)]
13931: Support for getting file sizes of files in Keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13931: Add test that file sizes are available.
Peter Amstutz [Mon, 6 Aug 2018 18:13:16 +0000 (14:13 -0400)]
13931: Add test that file sizes are available.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13399: Adjust locking strategy to avoid starvation.
Tom Clegg [Mon, 6 Aug 2018 16:59:30 +0000 (12:59 -0400)]
13399: Adjust locking strategy to avoid starvation.

Don't acquire the slurm lock to call scancel or sbatch.

Don't acquire the slurm lock again immediately after updating the
queue and broadcasting. Instead, give readers a chance to reacquire
the lock after Wait().

Acquire a shared lock instead of an exclusive lock in HasUUID() or
SetPriority() when the protected operation is a read.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: Dump structure with postgresql 9.6.
Tom Clegg [Mon, 6 Aug 2018 14:26:27 +0000 (10:26 -0400)]
13752: Dump structure with postgresql 9.6.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13399-renice-too-low-priority-jobs' refs #13399
Peter Amstutz [Mon, 6 Aug 2018 12:56:58 +0000 (08:56 -0400)]
Merge branch '13399-renice-too-low-priority-jobs' refs #13399

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13399: Don't try to bury jobs below very-low-priority jobs.
Tom Clegg [Sat, 4 Aug 2018 15:05:45 +0000 (11:05 -0400)]
13399: Don't try to bury jobs below very-low-priority jobs.

Non-zero priorities below 20K are probably the result of renicing jobs
that have priority=0 because they are in admin-hold state.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13961-separate-polling' refs #13961
Peter Amstutz [Fri, 3 Aug 2018 18:31:13 +0000 (14:31 -0400)]
Merge branch '13961-separate-polling' refs #13961

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13961: Use poll_time as fallback
Peter Amstutz [Fri, 3 Aug 2018 18:29:05 +0000 (14:29 -0400)]
13961: Use poll_time as fallback

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time
Peter Amstutz [Fri, 3 Aug 2018 17:01:49 +0000 (13:01 -0400)]
13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12762-safeyaml-warning'
Lucas Di Pentima [Fri, 3 Aug 2018 14:54:19 +0000 (11:54 -0300)]
Merge branch '12762-safeyaml-warning'
Closes #12762

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12762: Suppress SafeYAML warnings by setting its default mode.
Lucas Di Pentima [Thu, 2 Aug 2018 18:59:36 +0000 (15:59 -0300)]
12762: Suppress SafeYAML warnings by setting its default mode.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13609-r-sdk-copy' refs #13609
Peter Amstutz [Thu, 2 Aug 2018 18:00:37 +0000 (14:00 -0400)]
Merge branch '13609-r-sdk-copy' refs #13609

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13609: Note that move/copy between collections isn't supported
Peter Amstutz [Thu, 2 Aug 2018 17:59:37 +0000 (13:59 -0400)]
13609: Note that move/copy between collections isn't supported

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13609: Spelling fix
Peter Amstutz [Thu, 2 Aug 2018 17:53:47 +0000 (13:53 -0400)]
13609: Spelling fix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13609: Improve README fomatting a bit.
Peter Amstutz [Thu, 2 Aug 2018 17:52:35 +0000 (13:52 -0400)]
13609: Improve README fomatting a bit.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13609: Update version to 0.0.5
Peter Amstutz [Thu, 2 Aug 2018 17:40:28 +0000 (13:40 -0400)]
13609: Update version to 0.0.5

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13934-guard-comment'
Tom Clegg [Thu, 2 Aug 2018 04:13:15 +0000 (00:13 -0400)]
Merge branch '13934-guard-comment'

closes #13934

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13939: In groups#contents, apply unprefixed orders to all tables.
Tom Clegg [Wed, 1 Aug 2018 17:40:05 +0000 (13:40 -0400)]
13939: In groups#contents, apply unprefixed orders to all tables.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13939: Test orders with no table name qualifier.
Tom Clegg [Wed, 1 Aug 2018 17:13:50 +0000 (13:13 -0400)]
13939: Test orders with no table name qualifier.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13934: Add comment to guard dispatch bugfix.
Tom Clegg [Wed, 1 Aug 2018 16:56:31 +0000 (12:56 -0400)]
13934: Add comment to guard dispatch bugfix.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13497-controller'
Tom Clegg [Wed, 1 Aug 2018 13:14:14 +0000 (09:14 -0400)]
Merge branch '13497-controller'

refs #13497

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13198-keep-web-metrics'
Tom Clegg [Tue, 31 Jul 2018 18:04:39 +0000 (14:04 -0400)]
Merge branch '13198-keep-web-metrics'

refs #13198

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13892-deprecate-jobs-api' closes #13892
Peter Amstutz [Tue, 31 Jul 2018 17:51:05 +0000 (13:51 -0400)]
Merge branch '13892-deprecate-jobs-api' closes #13892

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13892: Make warning more obvious.
Peter Amstutz [Tue, 31 Jul 2018 17:26:14 +0000 (13:26 -0400)]
13892: Make warning more obvious.

Link to information about migrating for both users and admins.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13892: Update disable_api_methods with the full list
Peter Amstutz [Tue, 31 Jul 2018 15:40:28 +0000 (11:40 -0400)]
13892: Update disable_api_methods with the full list

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13497: Link issues and commit hashes to redmine.
Tom Clegg [Tue, 31 Jul 2018 15:31:55 +0000 (11:31 -0400)]
13497: Link issues and commit hashes to redmine.

"#1234" and "commit:foo" markup doesn't work here like it did on the
redmine wiki where the content came from.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13497: Add controller to upgrade notes.
Tom Clegg [Tue, 31 Jul 2018 15:26:36 +0000 (11:26 -0400)]
13497: Add controller to upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13497: Add controller to install docs.
Tom Clegg [Mon, 30 Jul 2018 19:53:45 +0000 (15:53 -0400)]
13497: Add controller to install docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13198: Add cache stats to metrics.
Tom Clegg [Mon, 30 Jul 2018 16:03:07 +0000 (12:03 -0400)]
13198: Add cache stats to metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13690-composer-user-guide' closes #13690
Peter Amstutz [Mon, 30 Jul 2018 16:53:54 +0000 (12:53 -0400)]
Merge branch '13690-composer-user-guide' closes #13690

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13690: Add/fix links
Peter Amstutz [Mon, 30 Jul 2018 14:35:37 +0000 (10:35 -0400)]
13690: Add/fix links

* Fix CWL user guide link
* Link from "Developing Workflows" to Composer tutorial
* More detail about what goes in "Docker Repository" field

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13690: Fill in some missing steps.
Peter Amstutz [Fri, 27 Jul 2018 17:57:11 +0000 (13:57 -0400)]
13690: Fill in some missing steps.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13920-arvados-gem-with-newer-rubies'
Lucas Di Pentima [Thu, 26 Jul 2018 20:41:33 +0000 (17:41 -0300)]
Merge branch '13920-arvados-gem-with-newer-rubies'
Closes #13920

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13920: Fixes google-api-client gem issue with ruby 2.3.7+
Lucas Di Pentima [Thu, 26 Jul 2018 19:45:23 +0000 (16:45 -0300)]
13920: Fixes google-api-client gem issue with ruby 2.3.7+

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13690: Walkthrough of Composer on Arvados
Peter Amstutz [Thu, 26 Jul 2018 18:31:53 +0000 (14:31 -0400)]
13690: Walkthrough of Composer on Arvados

* Creating a new git repository through Arvados to store the workflow
* Creating CommandLineTools for “sort” and “uniq”
* Creating a Workflow which uses “sort” and “uniq” to remove duplicate lines from a text file
* Submitting the Workflow to run on Arvados

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13910-arvados-gem-retrieval'
Lucas Di Pentima [Thu, 26 Jul 2018 18:28:18 +0000 (15:28 -0300)]
Merge branch '13910-arvados-gem-retrieval'
Closes #13910

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoFix a few typos.
Ward Vandewege [Thu, 26 Jul 2018 16:22:12 +0000 (12:22 -0400)]
Fix a few typos.

refs #13791

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch 'master' of git.curoverse.com:arvados into 13609-r-sdk-copy
Fuad Muhic [Thu, 26 Jul 2018 15:18:24 +0000 (17:18 +0200)]
Merge branch 'master' of git.curoverse.com:arvados into 13609-r-sdk-copy

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoImprove Collections create and move methods and update documentation
Fuad Muhic [Thu, 26 Jul 2018 14:09:14 +0000 (16:09 +0200)]
Improve Collections create and move methods and update documentation

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '13791-monitoring-docs' closes #13791
Peter Amstutz [Thu, 26 Jul 2018 13:54:10 +0000 (09:54 -0400)]
Merge branch '13791-monitoring-docs' closes #13791

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13892: Add jobs API deprecation notice to arvados-cwl-runner
Peter Amstutz [Thu, 26 Jul 2018 13:42:49 +0000 (09:42 -0400)]
13892: Add jobs API deprecation notice to arvados-cwl-runner

Also document how to disable the jobs API so that a-c-r selects the
containers API automatically.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12444-tmpdir-monitoring'
Lucas Di Pentima [Wed, 25 Jul 2018 20:38:09 +0000 (17:38 -0300)]
Merge branch '12444-tmpdir-monitoring'
Closes #12444

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13910: Fixes arvados gem dependency on arvados-cli & arvados-login-sync
Lucas Di Pentima [Wed, 25 Jul 2018 20:27:27 +0000 (17:27 -0300)]
13910: Fixes arvados gem dependency on arvados-cli & arvados-login-sync

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoarvbox uses keep config file instead of cli switches. no issue #
Peter Amstutz [Wed, 25 Jul 2018 19:18:48 +0000 (15:18 -0400)]
arvbox uses keep config file instead of cli switches.  no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13570-crunch-ram-doc' closes #13570
Peter Amstutz [Wed, 25 Jul 2018 19:10:53 +0000 (15:10 -0400)]
Merge branch '13570-crunch-ram-doc' closes #13570

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13570: Doc tweaks
Peter Amstutz [Wed, 25 Jul 2018 19:09:38 +0000 (15:09 -0400)]
13570: Doc tweaks

* Capitalize "Keep"
* Clarify usage of cgroups
* Clarify "Available compute node types"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoImplement copy method, update move method and remove trailing
Fuad Muhic [Wed, 25 Jul 2018 09:54:08 +0000 (11:54 +0200)]
Implement copy method, update move method and remove trailing
whitespace in Arvados.R.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13570: Add formulas. Add link from cwl-style to crunch discussion.
Peter Amstutz [Tue, 24 Jul 2018 20:19:40 +0000 (16:19 -0400)]
13570: Add formulas.  Add link from cwl-style to crunch discussion.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12444: Allow crunch-run to provide tmpdir to crunchstat
Lucas Di Pentima [Tue, 24 Jul 2018 20:15:21 +0000 (17:15 -0300)]
12444: Allow crunch-run to provide tmpdir to crunchstat

Also, get the temp dir from $TMPDIR if not provided, fix the 'used'
space calculation, and report which dir is being monitored.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13791: Tweak doc ordering.
Peter Amstutz [Tue, 24 Jul 2018 19:09:18 +0000 (15:09 -0400)]
13791: Tweak doc ordering.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13791: More detail about monitoring
Peter Amstutz [Tue, 24 Jul 2018 19:02:52 +0000 (15:02 -0400)]
13791: More detail about monitoring

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12444: Adds tmpdir stats to crunchstat reporting
Lucas Di Pentima [Tue, 24 Jul 2018 14:31:29 +0000 (11:31 -0300)]
12444: Adds tmpdir stats to crunchstat reporting

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13888-cli-dependency-on-api'
Lucas Di Pentima [Tue, 24 Jul 2018 17:07:31 +0000 (14:07 -0300)]
Merge branch '13888-cli-dependency-on-api'
Closes #13888

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13570: Add section describing RAM allocation behavior.
Peter Amstutz [Tue, 24 Jul 2018 15:53:51 +0000 (11:53 -0400)]
13570: Add section describing RAM allocation behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13888: Adds arvados-cli dependency back on API server
Lucas Di Pentima [Tue, 24 Jul 2018 15:43:22 +0000 (12:43 -0300)]
13888: Adds arvados-cli dependency back on API server

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoArvbox fix correct permissions to stop one-shot services. no issue #
Peter Amstutz [Mon, 23 Jul 2018 18:33:02 +0000 (14:33 -0400)]
Arvbox fix correct permissions to stop one-shot services. no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13791: Admin documentation for monitoring wip
Peter Amstutz [Mon, 23 Jul 2018 15:29:28 +0000 (11:29 -0400)]
13791: Admin documentation for monitoring wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13791: Health check endpoint docs wip
Peter Amstutz [Mon, 23 Jul 2018 13:31:51 +0000 (09:31 -0400)]
13791: Health check endpoint docs wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13546-crunch1-timeout' refs #13546
Peter Amstutz [Mon, 23 Jul 2018 13:33:44 +0000 (09:33 -0400)]
Merge branch '13546-crunch1-timeout' refs #13546

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13879-arvados-server' closes #13879
Peter Amstutz [Fri, 20 Jul 2018 20:52:52 +0000 (16:52 -0400)]
Merge branch '13879-arvados-server' closes #13879

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13879: Fix cut and paste typo.
Peter Amstutz [Fri, 20 Jul 2018 20:51:03 +0000 (16:51 -0400)]
13879: Fix cut and paste typo.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13546: crunch-job has timeout on srun_sync
Peter Amstutz [Fri, 20 Jul 2018 20:42:56 +0000 (16:42 -0400)]
13546: crunch-job has timeout on srun_sync

* Add global timeout to srun_sync, default 15 minutes, terminates
  job if a call to srun_sync exceeds the timeout.

* Default can be adjusted by setting CRUNCH_SRUN_SYNC_TIMEOUT in the
  environment of crunch_dispatch.rb (value is in seconds)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13851-wb-multisite-search'
Lucas Di Pentima [Fri, 20 Jul 2018 17:00:47 +0000 (14:00 -0300)]
Merge branch '13851-wb-multisite-search'
Closes #13851

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13493-federation-proxy'
Tom Clegg [Fri, 20 Jul 2018 15:11:36 +0000 (11:11 -0400)]
Merge branch '13493-federation-proxy'

refs #13493

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>