arvados.git
6 years ago13335: Implement EmptyTrashWorkers in azure and filesystem backends. 13335-trash-workers
Tom Clegg [Fri, 4 May 2018 20:48:50 +0000 (16:48 -0400)]
13335: Implement EmptyTrashWorkers in azure and filesystem backends.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13335: Fix warning about expected 404 error.
Tom Clegg [Tue, 24 Apr 2018 21:03:50 +0000 (17:03 -0400)]
13335: Fix warning about expected 404 error.

The expected error (object not found immediately after deleting it)
was being logged as a warning, and the warning erroneously stated the
error was from "HEAD recent/X" rather than "HEAD X".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13335: Add PullWorkers, TrashWorkers, EmptyTrashWorkers configs.
Tom Clegg [Tue, 24 Apr 2018 20:32:24 +0000 (16:32 -0400)]
13335: Add PullWorkers, TrashWorkers, EmptyTrashWorkers configs.

EmptyTrashWorkers only applies to S3 volumes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoUpdate mock to match new arvados.File interface.
Tom Clegg [Tue, 24 Apr 2018 15:07:15 +0000 (11:07 -0400)]
Update mock to match new arvados.File interface.

refs #13111

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 21:25:13 +0000 (18:25 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
Fernando Monserrat [Mon, 23 Apr 2018 21:24:12 +0000 (18:24 -0300)]
13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agoMerge branch 'master' into 13369-update-ubuntu1404-dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 21:16:44 +0000 (18:16 -0300)]
Merge branch 'master' into 13369-update-ubuntu1404-dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 18:30:54 +0000 (15:30 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 test packages Dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 18:06:21 +0000 (15:06 -0300)]
13369: update ubuntu1404 test packages Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agoFix nodemanager tests. no issue #
Peter Amstutz [Mon, 23 Apr 2018 15:29:55 +0000 (11:29 -0400)]
Fix nodemanager tests.  no issue #

Stub out subprocess.check_output to avoid calling the real "scontrol".
Fixes test_failed_arvados_calls_retried, test_subscribe and
test_creation_with_arvados_node in SLURMComputeNodeSetupActorTestCase.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 14:38:41 +0000 (11:38 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 test Dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 14:37:31 +0000 (11:37 -0300)]
13369: update ubuntu1404 test Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agoMerge branch 'thehyve/master' LocalJumpError in arvados-login-sync
Peter Amstutz [Fri, 20 Apr 2018 20:18:45 +0000 (16:18 -0400)]
Merge branch 'thehyve/master' LocalJumpError in arvados-login-sync

no issue #, github PR 65

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13111-webdav-projects'
Tom Clegg [Fri, 20 Apr 2018 19:59:21 +0000 (15:59 -0400)]
Merge branch '13111-webdav-projects'

refs #13111

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Merge branch 'master' into 13111-webdav-projects
Tom Clegg [Fri, 20 Apr 2018 19:56:12 +0000 (15:56 -0400)]
13111: Merge branch 'master' into 13111-webdav-projects

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Fri, 20 Apr 2018 18:11:48 +0000 (15:11 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 Dockerfile
Fernando Monserrat [Fri, 20 Apr 2018 18:10:43 +0000 (15:10 -0300)]
13369: update ubuntu1404 Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Fix bogus form-encoding in example URL.
Tom Clegg [Fri, 20 Apr 2018 18:08:39 +0000 (14:08 -0400)]
13111: Fix bogus form-encoding in example URL.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoSimplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhai...
strangepleasures [Fri, 20 Apr 2018 14:47:46 +0000 (16:47 +0200)]
Simplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel@thehyve.nl>

6 years agoMerge remote-tracking branch 'upstream/master'
strangepleasures [Fri, 20 Apr 2018 14:29:27 +0000 (16:29 +0200)]
Merge remote-tracking branch 'upstream/master'

6 years agoFix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhai...
strangepleasures [Fri, 20 Apr 2018 14:06:58 +0000 (16:06 +0200)]
Fix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel@thehyve.nl>

6 years agoFix R SDK online doc generation refs #13076
Peter Amstutz [Fri, 20 Apr 2018 13:44:59 +0000 (09:44 -0400)]
Fix R SDK online doc generation refs #13076

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13076-r-autogen-api'
Fuad Muhic [Fri, 20 Apr 2018 08:20:12 +0000 (10:20 +0200)]
Merge branch '13076-r-autogen-api'

refs #13076

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoarvbox pin pip to 9.0.3. Add LINES/COLUMNS to "arvbox shell", no issue #
Peter Amstutz [Thu, 19 Apr 2018 17:52:37 +0000 (13:52 -0400)]
arvbox pin pip to 9.0.3.  Add LINES/COLUMNS to "arvbox shell", no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMake the runin-reqs less stingy about RAM refs #13301
Peter Amstutz [Thu, 19 Apr 2018 20:05:28 +0000 (16:05 -0400)]
Make the runin-reqs less stingy about RAM refs #13301

Tests were failing because it turns out 4 MB isn't enough to run Python!

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13076: R SDK online docs include both README and method docs.
Peter Amstutz [Thu, 19 Apr 2018 19:47:23 +0000 (15:47 -0400)]
13076: R SDK online docs include both README and method docs.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13076: Get rid of pkgdown, just call knitr
Peter Amstutz [Thu, 19 Apr 2018 18:24:11 +0000 (14:24 -0400)]
13076: Get rid of pkgdown, just call knitr

Converts README.Rmd to README.html for documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoBump arvados-python-client version dependency.
Peter Amstutz [Wed, 18 Apr 2018 20:26:19 +0000 (16:26 -0400)]
Bump arvados-python-client version dependency.

Set threadsafe api object to use calling command.keepdocker and
command.put.

refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoarv-put: restore_signal_handlers conditional on install_sig_handlers
Peter Amstutz [Wed, 18 Apr 2018 20:23:29 +0000 (16:23 -0400)]
arv-put: restore_signal_handlers conditional on install_sig_handlers

arv-keepdocker: accept api object to use

refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoAdded script to generate html doc for .Rmd files.
Fuad Muhic [Tue, 17 Apr 2018 15:10:04 +0000 (17:10 +0200)]
Added script to generate html doc for .Rmd files.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agofix cwltest junit-xml dependency refs #13108
Peter Amstutz [Tue, 17 Apr 2018 01:26:41 +0000 (21:26 -0400)]
fix cwltest junit-xml dependency refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoUpdated README and small fix for Subcollection docs.
Fuad Muhic [Mon, 16 Apr 2018 15:03:38 +0000 (17:03 +0200)]
Updated README and small fix for Subcollection docs.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMethod documentation now shows headline properly.
Fuad Muhic [Mon, 16 Apr 2018 13:51:47 +0000 (15:51 +0200)]
Method documentation now shows headline properly.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoAdded documentation for hardcoded Arvados class project methods.
Fuad Muhic [Mon, 16 Apr 2018 13:16:47 +0000 (15:16 +0200)]
Added documentation for hardcoded Arvados class project methods.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoAdded auto-generated documentation for Arvados class.
Fuad Muhic [Mon, 16 Apr 2018 12:26:40 +0000 (14:26 +0200)]
Added auto-generated documentation for Arvados class.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13111: Merge branch 'master' into 13111-webdav-projects
Tom Clegg [Fri, 13 Apr 2018 20:49:27 +0000 (16:49 -0400)]
13111: Merge branch 'master' into 13111-webdav-projects

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Accept chunked responses to GET requests.
Tom Clegg [Fri, 13 Apr 2018 18:35:48 +0000 (14:35 -0400)]
13111: Accept chunked responses to GET requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Avoid multi-page API reqs when looking up entries by name.
Tom Clegg [Thu, 12 Apr 2018 20:30:17 +0000 (16:30 -0400)]
13111: Avoid multi-page API reqs when looking up entries by name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13108-acr-threading-fixes' refs #13108
Peter Amstutz [Fri, 13 Apr 2018 17:23:40 +0000 (13:23 -0400)]
Merge branch '13108-acr-threading-fixes' refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix import
Peter Amstutz [Fri, 13 Apr 2018 15:54:17 +0000 (11:54 -0400)]
13108: Fix import

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Refactor signal handling
Peter Amstutz [Fri, 13 Apr 2018 15:41:54 +0000 (11:41 -0400)]
13108: Refactor signal handling

a-c-r handles SystemExit the same way as KeyboardInterrupt.

Bump version of cwltest (to get the terminate-on-timeout bugfix).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12320-access-projects-from-by_id'
Fuad Muhic [Fri, 13 Apr 2018 07:59:02 +0000 (09:59 +0200)]
Merge branch '12320-access-projects-from-by_id'

closes #12320

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13108: crunch_script uses safeapi
Peter Amstutz [Fri, 13 Apr 2018 00:54:57 +0000 (20:54 -0400)]
13108: crunch_script uses safeapi

Also take workflow lock on update_pipeline_component to prevent update races.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Don't try to install signal handler in background thread
Peter Amstutz [Thu, 12 Apr 2018 20:22:52 +0000 (16:22 -0400)]
13108: Don't try to install signal handler in background thread

Fix Docker image upload failure due to arv-put trying to install a
signal handler somewhere other than the main thread.

Also install toplevel signal handler to convert SIGTERM to
KeyboardInterrupt.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agorefs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
Fernando Monserrat [Thu, 12 Apr 2018 18:59:59 +0000 (15:59 -0300)]
refs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_hash variable
Fernando Monserrat [Thu, 12 Apr 2018 18:37:46 +0000 (15:37 -0300)]
13339: fix git_hash variable
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_latest_tag formatting
Fernando Monserrat [Thu, 12 Apr 2018 18:19:28 +0000 (15:19 -0300)]
13339: fix git_latest_tag formatting
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_latest_tag formatting
Fernando Monserrat [Thu, 12 Apr 2018 18:11:38 +0000 (15:11 -0300)]
13339: fix git_latest_tag formatting
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Update docs.
Tom Clegg [Thu, 12 Apr 2018 17:29:20 +0000 (13:29 -0400)]
13111: Update docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Follow attachment-only policy at /by_id/ and /users/ paths.
Tom Clegg [Thu, 12 Apr 2018 17:28:31 +0000 (13:28 -0400)]
13111: Follow attachment-only policy at /by_id/ and /users/ paths.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13339: fix typo assigning git_latest_tag
Fernando Monserrat [Thu, 12 Apr 2018 17:20:40 +0000 (14:20 -0300)]
13339: fix typo assigning git_latest_tag
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Serve /by_id/ dir.
Tom Clegg [Thu, 12 Apr 2018 14:14:31 +0000 (10:14 -0400)]
13111: Serve /by_id/ dir.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Fix skipped error check when Write called before WriteHeader.
Tom Clegg [Thu, 12 Apr 2018 13:14:52 +0000 (09:14 -0400)]
13111: Fix skipped error check when Write called before WriteHeader.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoOptimized API call to not return non project groups
Fuad Muhic [Thu, 12 Apr 2018 09:49:32 +0000 (11:49 +0200)]
Optimized API call to not return non project groups

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoUpdate instructions to get arvados node manager Azure credentials to v2
Ward Vandewege [Thu, 12 Apr 2018 01:17:38 +0000 (21:17 -0400)]
Update instructions to get arvados node manager Azure credentials to v2
of the Azure cli.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago13111: Avoid double WriteHeader after implicit 200 on first Write.
Tom Clegg [Wed, 11 Apr 2018 23:59:18 +0000 (19:59 -0400)]
13111: Avoid double WriteHeader after implicit 200 on first Write.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Ignore projects and collections with "/" in name.
Tom Clegg [Wed, 11 Apr 2018 13:07:09 +0000 (09:07 -0400)]
13111: Ignore projects and collections with "/" in name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13348-recursive-submit'
Tom Clegg [Wed, 11 Apr 2018 20:21:21 +0000 (16:21 -0400)]
Merge branch '13348-recursive-submit'

refs #13348
refs #13108

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13348: Don't submit child jobs/containers with submit=True.
Tom Clegg [Wed, 11 Apr 2018 19:54:15 +0000 (15:54 -0400)]
13348: Don't submit child jobs/containers with submit=True.

Fixes infinite recursion.

see a82b39e1849a67854cb3399a4660b77548edd580

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13108-cwl-parallel-submit' closes #13108
Peter Amstutz [Tue, 10 Apr 2018 19:38:18 +0000 (15:38 -0400)]
Merge branch '13108-cwl-parallel-submit' closes #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Add test for taskqueue
Peter Amstutz [Tue, 10 Apr 2018 18:19:05 +0000 (14:19 -0400)]
13108: Add test for taskqueue

Also tighten up code in a few places.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoUsers can only access groups with group_class set to project from by_id
Fuad Muhic [Tue, 10 Apr 2018 16:29:16 +0000 (18:29 +0200)]
Users can only access groups with group_class set to project from by_id
directory

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '13142-dispatch-nfiles'
Tom Clegg [Tue, 10 Apr 2018 15:48:48 +0000 (11:48 -0400)]
Merge branch '13142-dispatch-nfiles'

refs #13142

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13142: Raise open files limit.
Tom Clegg [Wed, 28 Mar 2018 20:24:04 +0000 (16:24 -0400)]
13142: Raise open files limit.

See https://dev.arvados.org/issues/13142

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13105-wb-ignore-missing-remotes'
Lucas Di Pentima [Tue, 10 Apr 2018 15:34:55 +0000 (10:34 -0500)]
Merge branch '13105-wb-ignore-missing-remotes'
Closes #13105

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13105: discoveryDoc() call simplification.
Lucas Di Pentima [Tue, 10 Apr 2018 15:26:34 +0000 (10:26 -0500)]
13105: discoveryDoc() call simplification.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoArvados projects can be accessed from by_id directory in keep mount
Fuad Muhic [Tue, 10 Apr 2018 13:40:13 +0000 (15:40 +0200)]
Arvados projects can be accessed from by_id directory in keep mount

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '13100-crunch-run-output'
Tom Clegg [Tue, 10 Apr 2018 13:21:26 +0000 (09:21 -0400)]
Merge branch '13100-crunch-run-output'

fixes #13100
fixes #11583
fixes #12606
refs #13048

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13105: Set remoteHosts and remoteHostsViaDNS default values when missing.
Lucas Di Pentima [Thu, 5 Apr 2018 19:04:28 +0000 (16:04 -0300)]
13105: Set remoteHosts and remoteHostsViaDNS default values when missing.

Also fix some discovery doc requests to use the SessionDB cache.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13100: Merge branch 'master' into 13100-crunch-run-output
Tom Clegg [Mon, 9 Apr 2018 20:22:33 +0000 (16:22 -0400)]
13100: Merge branch 'master' into 13100-crunch-run-output

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: 12606: Test chain of relative symlinks.
Tom Clegg [Mon, 9 Apr 2018 15:26:26 +0000 (11:26 -0400)]
13100: 12606: Test chain of relative symlinks.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Handle output symlinks to writable collections.
Tom Clegg [Mon, 9 Apr 2018 15:13:30 +0000 (11:13 -0400)]
13100: Handle output symlinks to writable collections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Handle writable collections mounted below output dir.
Tom Clegg [Mon, 9 Apr 2018 14:19:59 +0000 (10:19 -0400)]
13100: Handle writable collections mounted below output dir.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Update comment.
Tom Clegg [Mon, 9 Apr 2018 13:56:03 +0000 (09:56 -0400)]
13100: Update comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Note safe use of append().
Tom Clegg [Fri, 6 Apr 2018 20:41:11 +0000 (16:41 -0400)]
13111: Note safe use of append().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Update comment.
Tom Clegg [Fri, 6 Apr 2018 20:41:00 +0000 (16:41 -0400)]
13111: Update comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13339: update gemspecs for new versioning
Fernando Monserrat [Fri, 6 Apr 2018 20:13:46 +0000 (17:13 -0300)]
13339: update gemspecs for new versioning
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Redirect /dir to /dir/ at siteFS paths.
Tom Clegg [Fri, 6 Apr 2018 20:08:35 +0000 (16:08 -0400)]
13111: Redirect /dir to /dir/ at siteFS paths.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13108: Add copyright notice to task_queue.py
Peter Amstutz [Fri, 6 Apr 2018 17:49:11 +0000 (13:49 -0400)]
13108: Add copyright notice to task_queue.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix tests.
Peter Amstutz [Fri, 6 Apr 2018 17:37:08 +0000 (13:37 -0400)]
13108: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Refactor task queue into its own class.
Peter Amstutz [Fri, 6 Apr 2018 15:32:03 +0000 (11:32 -0400)]
13108: Refactor task queue into its own class.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix tracking tasks in flight
Peter Amstutz [Fri, 6 Apr 2018 15:21:56 +0000 (11:21 -0400)]
13108: Fix tracking tasks in flight

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Rename parallel_submit_count to thread_count
Peter Amstutz [Fri, 6 Apr 2018 14:29:30 +0000 (10:29 -0400)]
13108: Rename parallel_submit_count to thread_count

Make sure it gets passed through to RunnerContainer.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Rename runnable_queue -> task_queue
Peter Amstutz [Fri, 6 Apr 2018 12:44:23 +0000 (08:44 -0400)]
13108: Rename runnable_queue -> task_queue

Execute synchronously when parallel_submit_count == 1

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Collect outputs in separate threads as well
Peter Amstutz [Thu, 5 Apr 2018 22:20:24 +0000 (18:20 -0400)]
13108: Collect outputs in separate threads as well

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13111: Serve "site filesystem" at keep-web root URL.
Tom Clegg [Thu, 5 Apr 2018 21:54:40 +0000 (17:54 -0400)]
13111: Serve "site filesystem" at keep-web root URL.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Skip collections named "." or "..".
Tom Clegg [Thu, 5 Apr 2018 21:22:08 +0000 (17:22 -0400)]
13111: Skip collections named "." or "..".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Clarify Child() comment.
Tom Clegg [Thu, 5 Apr 2018 21:21:53 +0000 (17:21 -0400)]
13111: Clarify Child() comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Test /users/name and /users paths with cadaver.
Tom Clegg [Thu, 5 Apr 2018 21:20:49 +0000 (17:20 -0400)]
13111: Test /users/name and /users paths with cadaver.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13108: Work queue submit wip
Peter Amstutz [Thu, 5 Apr 2018 21:10:25 +0000 (17:10 -0400)]
13108: Work queue submit wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Simplify locking, add methods for recording process status
Peter Amstutz [Thu, 5 Apr 2018 20:32:19 +0000 (16:32 -0400)]
13108: Simplify locking, add methods for recording process status

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13111: Merge branch 'master' into 13111-webdav-projects
Tom Clegg [Thu, 5 Apr 2018 20:14:35 +0000 (16:14 -0400)]
13111: Merge branch 'master' into 13111-webdav-projects

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix comment. 1.1.4
Ward Vandewege [Thu, 5 Apr 2018 20:00:31 +0000 (16:00 -0400)]
Fix comment.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago13100: Use collection FS for output and logs.
Tom Clegg [Thu, 5 Apr 2018 06:31:10 +0000 (02:31 -0400)]
13100: Use collection FS for output and logs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13108: Use ThreadSafeApiCache
Peter Amstutz [Thu, 5 Apr 2018 19:45:58 +0000 (15:45 -0400)]
13108: Use ThreadSafeApiCache

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Handle multiple instances of arv-keepdocker uploading same docker image
Peter Amstutz [Thu, 5 Apr 2018 19:28:09 +0000 (15:28 -0400)]
13108: Handle multiple instances of arv-keepdocker uploading same docker image

Use cooperative locking with flock to minimize interference.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13201-less-validate' closes #13201
Peter Amstutz [Thu, 5 Apr 2018 17:25:44 +0000 (13:25 -0400)]
Merge branch '13201-less-validate' closes #13201

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12085-anm-metrics'
Lucas Di Pentima [Thu, 5 Apr 2018 15:15:43 +0000 (12:15 -0300)]
Merge branch '12085-anm-metrics'
Closes #12085

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13201: Fix tests.
Peter Amstutz [Thu, 5 Apr 2018 14:47:39 +0000 (10:47 -0400)]
13201: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12085: Unconditionally remove the idle time counter of paired missing nodes.
Lucas Di Pentima [Thu, 5 Apr 2018 14:37:49 +0000 (11:37 -0300)]
12085: Unconditionally remove the idle time counter of paired missing nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>