arvados.git
6 years ago12708: Merge branch 'master' into 12708-balance-storage-classes
Tom Clegg [Thu, 26 Apr 2018 17:14:38 +0000 (13:14 -0400)]
12708: Merge branch 'master' into 12708-balance-storage-classes

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12708: TODO notes about memory use and class combinations.
Tom Clegg [Thu, 26 Apr 2018 17:10:04 +0000 (13:10 -0400)]
12708: TODO notes about memory use and class combinations.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12708: Precompute len(slots) to reduce mallocs. Rename setupCaches.
Tom Clegg [Thu, 26 Apr 2018 17:09:29 +0000 (13:09 -0400)]
12708: Precompute len(slots) to reduce mallocs. Rename setupCaches.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoAdd tag to secrets test so it is recognized by new cwltest as "optional".
Peter Amstutz [Thu, 26 Apr 2018 15:59:25 +0000 (11:59 -0400)]
Add tag to secrets test so it is recognized by new cwltest as "optional".

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13111-projects-by-id'
Tom Clegg [Thu, 26 Apr 2018 15:42:28 +0000 (11:42 -0400)]
Merge branch '13111-projects-by-id'

refs #13111

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Update to Go 1.10.1.
Tom Clegg [Thu, 26 Apr 2018 15:34:38 +0000 (11:34 -0400)]
13111: Update to Go 1.10.1.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Merge branch 'master'
Tom Clegg [Wed, 25 Apr 2018 15:01:36 +0000 (11:01 -0400)]
13111: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agolibcloud 2.3.0 now requires python-requests 2.4.3 or higher, otherwise
Ward Vandewege [Wed, 25 Apr 2018 03:23:08 +0000 (23:23 -0400)]
libcloud 2.3.0 now requires python-requests 2.4.3 or higher, otherwise
it throws

  ImportError: No module named packages.urllib3.poolmanager

when loaded. We only see this problem on ubuntu1404, because that is our
only supported distribution that ships with a python-requests older than
2.4.3. This issue was caught by our automated package testing, yay!

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoRemove remaining build script references to ubuntu1204, which we do not
Ward Vandewege [Wed, 25 Apr 2018 03:21:16 +0000 (23:21 -0400)]
Remove remaining build script references to ubuntu1204, which we do not
build packages for anymore.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agorefs #13369 rollback packages install on ubuntu1404 Dockerfile for builds
Fernando Monserrat [Wed, 25 Apr 2018 01:09:31 +0000 (22:09 -0300)]
refs #13369 rollback packages install on ubuntu1404 Dockerfile for builds
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agorefs #13369 remove added packages install. rollback
Fernando Monserrat [Tue, 24 Apr 2018 22:31:25 +0000 (19:31 -0300)]
refs #13369 remove added packages install. rollback
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agologin-sync test fix also make docker group optional refs #13385
Peter Amstutz [Tue, 24 Apr 2018 21:44:44 +0000 (17:44 -0400)]
login-sync test fix also make docker group optional refs #13385

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13385-login-sync-test' refs #13385
Peter Amstutz [Tue, 24 Apr 2018 20:15:53 +0000 (16:15 -0400)]
Merge branch '13385-login-sync-test' refs #13385

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13385: login-sync test works even if there is no fuse group
Peter Amstutz [Tue, 24 Apr 2018 20:11:18 +0000 (16:11 -0400)]
13385: login-sync test works even if there is no fuse group

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoUpdate mock to match new arvados.File interface.
Tom Clegg [Tue, 24 Apr 2018 15:07:15 +0000 (11:07 -0400)]
Update mock to match new arvados.File interface.

refs #13111

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Update arvbox and package-build images to Go 1.9.5.
Tom Clegg [Tue, 24 Apr 2018 03:23:00 +0000 (23:23 -0400)]
13111: Update arvbox and package-build images to Go 1.9.5.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 21:25:13 +0000 (18:25 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
Fernando Monserrat [Mon, 23 Apr 2018 21:24:12 +0000 (18:24 -0300)]
13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agoMerge branch 'master' into 13369-update-ubuntu1404-dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 21:16:44 +0000 (18:16 -0300)]
Merge branch 'master' into 13369-update-ubuntu1404-dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 18:30:54 +0000 (15:30 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 test packages Dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 18:06:21 +0000 (15:06 -0300)]
13369: update ubuntu1404 test packages Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years agoFix nodemanager tests. no issue #
Peter Amstutz [Mon, 23 Apr 2018 15:29:55 +0000 (11:29 -0400)]
Fix nodemanager tests.  no issue #

Stub out subprocess.check_output to avoid calling the real "scontrol".
Fixes test_failed_arvados_calls_retried, test_subscribe and
test_creation_with_arvados_node in SLURMComputeNodeSetupActorTestCase.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Mon, 23 Apr 2018 14:38:41 +0000 (11:38 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 test Dockerfile
Fernando Monserrat [Mon, 23 Apr 2018 14:37:31 +0000 (11:37 -0300)]
13369: update ubuntu1404 test Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Access projects via /by_id/$uuid/.
Tom Clegg [Mon, 23 Apr 2018 13:05:39 +0000 (09:05 -0400)]
13111: Access projects via /by_id/$uuid/.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch 'thehyve/master' LocalJumpError in arvados-login-sync
Peter Amstutz [Fri, 20 Apr 2018 20:18:45 +0000 (16:18 -0400)]
Merge branch 'thehyve/master' LocalJumpError in arvados-login-sync

no issue #, github PR 65

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13111-webdav-projects'
Tom Clegg [Fri, 20 Apr 2018 19:59:21 +0000 (15:59 -0400)]
Merge branch '13111-webdav-projects'

refs #13111

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Merge branch 'master' into 13111-webdav-projects
Tom Clegg [Fri, 20 Apr 2018 19:56:12 +0000 (15:56 -0400)]
13111: Merge branch 'master' into 13111-webdav-projects

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agorefs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Fernando Monserrat [Fri, 20 Apr 2018 18:11:48 +0000 (15:11 -0300)]
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13369: update ubuntu1404 Dockerfile
Fernando Monserrat [Fri, 20 Apr 2018 18:10:43 +0000 (15:10 -0300)]
13369: update ubuntu1404 Dockerfile
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Fix bogus form-encoding in example URL.
Tom Clegg [Fri, 20 Apr 2018 18:08:39 +0000 (14:08 -0400)]
13111: Fix bogus form-encoding in example URL.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoSimplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhai...
strangepleasures [Fri, 20 Apr 2018 14:47:46 +0000 (16:47 +0200)]
Simplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel@thehyve.nl>

6 years agoMerge remote-tracking branch 'upstream/master'
strangepleasures [Fri, 20 Apr 2018 14:29:27 +0000 (16:29 +0200)]
Merge remote-tracking branch 'upstream/master'

6 years agoFix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhai...
strangepleasures [Fri, 20 Apr 2018 14:06:58 +0000 (16:06 +0200)]
Fix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel@thehyve.nl>

6 years agoFix R SDK online doc generation refs #13076
Peter Amstutz [Fri, 20 Apr 2018 13:44:59 +0000 (09:44 -0400)]
Fix R SDK online doc generation refs #13076

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13076-r-autogen-api'
Fuad Muhic [Fri, 20 Apr 2018 08:20:12 +0000 (10:20 +0200)]
Merge branch '13076-r-autogen-api'

refs #13076

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoarvbox pin pip to 9.0.3. Add LINES/COLUMNS to "arvbox shell", no issue #
Peter Amstutz [Thu, 19 Apr 2018 17:52:37 +0000 (13:52 -0400)]
arvbox pin pip to 9.0.3.  Add LINES/COLUMNS to "arvbox shell", no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMake the runin-reqs less stingy about RAM refs #13301
Peter Amstutz [Thu, 19 Apr 2018 20:05:28 +0000 (16:05 -0400)]
Make the runin-reqs less stingy about RAM refs #13301

Tests were failing because it turns out 4 MB isn't enough to run Python!

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13076: R SDK online docs include both README and method docs.
Peter Amstutz [Thu, 19 Apr 2018 19:47:23 +0000 (15:47 -0400)]
13076: R SDK online docs include both README and method docs.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13076: Get rid of pkgdown, just call knitr
Peter Amstutz [Thu, 19 Apr 2018 18:24:11 +0000 (14:24 -0400)]
13076: Get rid of pkgdown, just call knitr

Converts README.Rmd to README.html for documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12708: Pull blocks to desired storage classes.
Tom Clegg [Thu, 19 Apr 2018 14:59:42 +0000 (10:59 -0400)]
12708: Pull blocks to desired storage classes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoBump arvados-python-client version dependency.
Peter Amstutz [Wed, 18 Apr 2018 20:26:19 +0000 (16:26 -0400)]
Bump arvados-python-client version dependency.

Set threadsafe api object to use calling command.keepdocker and
command.put.

refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoarv-put: restore_signal_handlers conditional on install_sig_handlers
Peter Amstutz [Wed, 18 Apr 2018 20:23:29 +0000 (16:23 -0400)]
arv-put: restore_signal_handlers conditional on install_sig_handlers

arv-keepdocker: accept api object to use

refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoAdded script to generate html doc for .Rmd files.
Fuad Muhic [Tue, 17 Apr 2018 15:10:04 +0000 (17:10 +0200)]
Added script to generate html doc for .Rmd files.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agofix cwltest junit-xml dependency refs #13108
Peter Amstutz [Tue, 17 Apr 2018 01:26:41 +0000 (21:26 -0400)]
fix cwltest junit-xml dependency refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoUpdated README and small fix for Subcollection docs.
Fuad Muhic [Mon, 16 Apr 2018 15:03:38 +0000 (17:03 +0200)]
Updated README and small fix for Subcollection docs.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMethod documentation now shows headline properly.
Fuad Muhic [Mon, 16 Apr 2018 13:51:47 +0000 (15:51 +0200)]
Method documentation now shows headline properly.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoAdded documentation for hardcoded Arvados class project methods.
Fuad Muhic [Mon, 16 Apr 2018 13:16:47 +0000 (15:16 +0200)]
Added documentation for hardcoded Arvados class project methods.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoAdded auto-generated documentation for Arvados class.
Fuad Muhic [Mon, 16 Apr 2018 12:26:40 +0000 (14:26 +0200)]
Added auto-generated documentation for Arvados class.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13111: Merge branch 'master' into 13111-webdav-projects
Tom Clegg [Fri, 13 Apr 2018 20:49:27 +0000 (16:49 -0400)]
13111: Merge branch 'master' into 13111-webdav-projects

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Accept chunked responses to GET requests.
Tom Clegg [Fri, 13 Apr 2018 18:35:48 +0000 (14:35 -0400)]
13111: Accept chunked responses to GET requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Avoid multi-page API reqs when looking up entries by name.
Tom Clegg [Thu, 12 Apr 2018 20:30:17 +0000 (16:30 -0400)]
13111: Avoid multi-page API reqs when looking up entries by name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13108-acr-threading-fixes' refs #13108
Peter Amstutz [Fri, 13 Apr 2018 17:23:40 +0000 (13:23 -0400)]
Merge branch '13108-acr-threading-fixes' refs #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix import
Peter Amstutz [Fri, 13 Apr 2018 15:54:17 +0000 (11:54 -0400)]
13108: Fix import

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Refactor signal handling
Peter Amstutz [Fri, 13 Apr 2018 15:41:54 +0000 (11:41 -0400)]
13108: Refactor signal handling

a-c-r handles SystemExit the same way as KeyboardInterrupt.

Bump version of cwltest (to get the terminate-on-timeout bugfix).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12320-access-projects-from-by_id'
Fuad Muhic [Fri, 13 Apr 2018 07:59:02 +0000 (09:59 +0200)]
Merge branch '12320-access-projects-from-by_id'

closes #12320

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13108: crunch_script uses safeapi
Peter Amstutz [Fri, 13 Apr 2018 00:54:57 +0000 (20:54 -0400)]
13108: crunch_script uses safeapi

Also take workflow lock on update_pipeline_component to prevent update races.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Don't try to install signal handler in background thread
Peter Amstutz [Thu, 12 Apr 2018 20:22:52 +0000 (16:22 -0400)]
13108: Don't try to install signal handler in background thread

Fix Docker image upload failure due to arv-put trying to install a
signal handler somewhere other than the main thread.

Also install toplevel signal handler to convert SIGTERM to
KeyboardInterrupt.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agorefs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
Fernando Monserrat [Thu, 12 Apr 2018 18:59:59 +0000 (15:59 -0300)]
refs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_hash variable
Fernando Monserrat [Thu, 12 Apr 2018 18:37:46 +0000 (15:37 -0300)]
13339: fix git_hash variable
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_latest_tag formatting
Fernando Monserrat [Thu, 12 Apr 2018 18:19:28 +0000 (15:19 -0300)]
13339: fix git_latest_tag formatting
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13339: fix git_latest_tag formatting
Fernando Monserrat [Thu, 12 Apr 2018 18:11:38 +0000 (15:11 -0300)]
13339: fix git_latest_tag formatting
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Update docs.
Tom Clegg [Thu, 12 Apr 2018 17:29:20 +0000 (13:29 -0400)]
13111: Update docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Follow attachment-only policy at /by_id/ and /users/ paths.
Tom Clegg [Thu, 12 Apr 2018 17:28:31 +0000 (13:28 -0400)]
13111: Follow attachment-only policy at /by_id/ and /users/ paths.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13339: fix typo assigning git_latest_tag
Fernando Monserrat [Thu, 12 Apr 2018 17:20:40 +0000 (14:20 -0300)]
13339: fix typo assigning git_latest_tag
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Serve /by_id/ dir.
Tom Clegg [Thu, 12 Apr 2018 14:14:31 +0000 (10:14 -0400)]
13111: Serve /by_id/ dir.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Fix skipped error check when Write called before WriteHeader.
Tom Clegg [Thu, 12 Apr 2018 13:14:52 +0000 (09:14 -0400)]
13111: Fix skipped error check when Write called before WriteHeader.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoOptimized API call to not return non project groups
Fuad Muhic [Thu, 12 Apr 2018 09:49:32 +0000 (11:49 +0200)]
Optimized API call to not return non project groups

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoUpdate instructions to get arvados node manager Azure credentials to v2
Ward Vandewege [Thu, 12 Apr 2018 01:17:38 +0000 (21:17 -0400)]
Update instructions to get arvados node manager Azure credentials to v2
of the Azure cli.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago13111: Avoid double WriteHeader after implicit 200 on first Write.
Tom Clegg [Wed, 11 Apr 2018 23:59:18 +0000 (19:59 -0400)]
13111: Avoid double WriteHeader after implicit 200 on first Write.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Ignore projects and collections with "/" in name.
Tom Clegg [Wed, 11 Apr 2018 13:07:09 +0000 (09:07 -0400)]
13111: Ignore projects and collections with "/" in name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13348-recursive-submit'
Tom Clegg [Wed, 11 Apr 2018 20:21:21 +0000 (16:21 -0400)]
Merge branch '13348-recursive-submit'

refs #13348
refs #13108

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13348: Don't submit child jobs/containers with submit=True.
Tom Clegg [Wed, 11 Apr 2018 19:54:15 +0000 (15:54 -0400)]
13348: Don't submit child jobs/containers with submit=True.

Fixes infinite recursion.

see a82b39e1849a67854cb3399a4660b77548edd580

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13108-cwl-parallel-submit' closes #13108
Peter Amstutz [Tue, 10 Apr 2018 19:38:18 +0000 (15:38 -0400)]
Merge branch '13108-cwl-parallel-submit' closes #13108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Add test for taskqueue
Peter Amstutz [Tue, 10 Apr 2018 18:19:05 +0000 (14:19 -0400)]
13108: Add test for taskqueue

Also tighten up code in a few places.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoUsers can only access groups with group_class set to project from by_id
Fuad Muhic [Tue, 10 Apr 2018 16:29:16 +0000 (18:29 +0200)]
Users can only access groups with group_class set to project from by_id
directory

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '13142-dispatch-nfiles'
Tom Clegg [Tue, 10 Apr 2018 15:48:48 +0000 (11:48 -0400)]
Merge branch '13142-dispatch-nfiles'

refs #13142

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13142: Raise open files limit.
Tom Clegg [Wed, 28 Mar 2018 20:24:04 +0000 (16:24 -0400)]
13142: Raise open files limit.

See https://dev.arvados.org/issues/13142

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13105-wb-ignore-missing-remotes'
Lucas Di Pentima [Tue, 10 Apr 2018 15:34:55 +0000 (10:34 -0500)]
Merge branch '13105-wb-ignore-missing-remotes'
Closes #13105

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13105: discoveryDoc() call simplification.
Lucas Di Pentima [Tue, 10 Apr 2018 15:26:34 +0000 (10:26 -0500)]
13105: discoveryDoc() call simplification.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoArvados projects can be accessed from by_id directory in keep mount
Fuad Muhic [Tue, 10 Apr 2018 13:40:13 +0000 (15:40 +0200)]
Arvados projects can be accessed from by_id directory in keep mount

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '13100-crunch-run-output'
Tom Clegg [Tue, 10 Apr 2018 13:21:26 +0000 (09:21 -0400)]
Merge branch '13100-crunch-run-output'

fixes #13100
fixes #11583
fixes #12606
refs #13048

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13105: Set remoteHosts and remoteHostsViaDNS default values when missing.
Lucas Di Pentima [Thu, 5 Apr 2018 19:04:28 +0000 (16:04 -0300)]
13105: Set remoteHosts and remoteHostsViaDNS default values when missing.

Also fix some discovery doc requests to use the SessionDB cache.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13100: Merge branch 'master' into 13100-crunch-run-output
Tom Clegg [Mon, 9 Apr 2018 20:22:33 +0000 (16:22 -0400)]
13100: Merge branch 'master' into 13100-crunch-run-output

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: 12606: Test chain of relative symlinks.
Tom Clegg [Mon, 9 Apr 2018 15:26:26 +0000 (11:26 -0400)]
13100: 12606: Test chain of relative symlinks.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Handle output symlinks to writable collections.
Tom Clegg [Mon, 9 Apr 2018 15:13:30 +0000 (11:13 -0400)]
13100: Handle output symlinks to writable collections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Handle writable collections mounted below output dir.
Tom Clegg [Mon, 9 Apr 2018 14:19:59 +0000 (10:19 -0400)]
13100: Handle writable collections mounted below output dir.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13100: Update comment.
Tom Clegg [Mon, 9 Apr 2018 13:56:03 +0000 (09:56 -0400)]
13100: Update comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Note safe use of append().
Tom Clegg [Fri, 6 Apr 2018 20:41:11 +0000 (16:41 -0400)]
13111: Note safe use of append().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13111: Update comment.
Tom Clegg [Fri, 6 Apr 2018 20:41:00 +0000 (16:41 -0400)]
13111: Update comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13339: update gemspecs for new versioning
Fernando Monserrat [Fri, 6 Apr 2018 20:13:46 +0000 (17:13 -0300)]
13339: update gemspecs for new versioning
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

6 years ago13111: Redirect /dir to /dir/ at siteFS paths.
Tom Clegg [Fri, 6 Apr 2018 20:08:35 +0000 (16:08 -0400)]
13111: Redirect /dir to /dir/ at siteFS paths.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13108: Add copyright notice to task_queue.py
Peter Amstutz [Fri, 6 Apr 2018 17:49:11 +0000 (13:49 -0400)]
13108: Add copyright notice to task_queue.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix tests.
Peter Amstutz [Fri, 6 Apr 2018 17:37:08 +0000 (13:37 -0400)]
13108: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Refactor task queue into its own class.
Peter Amstutz [Fri, 6 Apr 2018 15:32:03 +0000 (11:32 -0400)]
13108: Refactor task queue into its own class.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Fix tracking tasks in flight
Peter Amstutz [Fri, 6 Apr 2018 15:21:56 +0000 (11:21 -0400)]
13108: Fix tracking tasks in flight

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Rename parallel_submit_count to thread_count
Peter Amstutz [Fri, 6 Apr 2018 14:29:30 +0000 (10:29 -0400)]
13108: Rename parallel_submit_count to thread_count

Make sure it gets passed through to RunnerContainer.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Rename runnable_queue -> task_queue
Peter Amstutz [Fri, 6 Apr 2018 12:44:23 +0000 (08:44 -0400)]
13108: Rename runnable_queue -> task_queue

Execute synchronously when parallel_submit_count == 1

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13108: Collect outputs in separate threads as well
Peter Amstutz [Thu, 5 Apr 2018 22:20:24 +0000 (18:20 -0400)]
13108: Collect outputs in separate threads as well

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13111: Serve "site filesystem" at keep-web root URL.
Tom Clegg [Thu, 5 Apr 2018 21:54:40 +0000 (17:54 -0400)]
13111: Serve "site filesystem" at keep-web root URL.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>