Merge branch '13851-wb-multisite-search'
authorLucas Di Pentima <ldipentima@veritasgenetics.com>
Fri, 20 Jul 2018 17:00:47 +0000 (14:00 -0300)
committerLucas Di Pentima <ldipentima@veritasgenetics.com>
Fri, 20 Jul 2018 17:01:07 +0000 (14:01 -0300)
Closes #13851

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

65 files changed:
README.md
build/run-library.sh
doc/_includes/_skip_sso_server_install.liquid [deleted file]
doc/install/copy_pipeline_from_curoverse.html.textile.liquid
doc/install/index.html.textile.liquid
doc/start/getting_started/firstpipeline.html.textile.liquid
doc/user/cwl/cwl-runner.html.textile.liquid
doc/user/topics/arv-copy.html.textile.liquid
doc/user/tutorials/running-external-program.html.textile.liquid
lib/controller/federation.go [new file with mode: 0644]
lib/controller/federation_test.go [new file with mode: 0644]
lib/controller/handler.go
lib/controller/handler_test.go
lib/controller/proxy.go [new file with mode: 0644]
lib/controller/server_test.go [new file with mode: 0644]
sdk/R/R/Arvados.R
sdk/R/R/ArvadosFile.R
sdk/R/R/Collection.R
sdk/R/R/CollectionTree.R
sdk/R/R/HttpParser.R
sdk/R/R/HttpRequest.R
sdk/R/R/RESTService.R
sdk/R/R/Subcollection.R
sdk/R/R/autoGenAPI.R
sdk/R/tests/testthat/fakes/FakeHttpParser.R
sdk/R/tests/testthat/fakes/FakeHttpRequest.R
sdk/R/tests/testthat/fakes/FakeRESTService.R
sdk/R/tests/testthat/test-ArvadosFile.R
sdk/R/tests/testthat/test-Collection.R
sdk/R/tests/testthat/test-CollectionTree.R
sdk/R/tests/testthat/test-HttpParser.R
sdk/R/tests/testthat/test-HttpRequest.R
sdk/R/tests/testthat/test-RESTService.R
sdk/R/tests/testthat/test-Subcollection.R
sdk/R/tests/testthat/test-util.R
sdk/go/arvados/api_client_authorization.go
sdk/go/arvados/config.go
sdk/go/arvados/postgresql.go [new file with mode: 0644]
sdk/go/arvadostest/fixtures.go
sdk/go/auth/auth.go
sdk/go/auth/salt.go [new file with mode: 0644]
sdk/go/dispatch/dispatch.go
sdk/go/httpserver/metrics.go [new file with mode: 0644]
sdk/python/tests/run_test_server.py
services/api/app/controllers/static_controller.rb
services/api/test/integration/cross_origin_test.rb
services/crunch-dispatch-slurm/crunch-dispatch-slurm_test.go
services/crunch-dispatch-slurm/squeue.go
services/crunch-dispatch-slurm/squeue_test.go
services/keep-web/doc.go
services/keep-web/handler.go
services/keep-web/handler_test.go
services/keep-web/server.go
services/keep-web/server_test.go
services/keepstore/config.go
services/keepstore/handlers.go
services/nodemanager/arvnodeman/computenode/driver/azure.py
services/nodemanager/arvnodeman/computenode/driver/ec2.py
services/nodemanager/arvnodeman/computenode/driver/gce.py
services/nodemanager/arvnodeman/jobqueue.py
services/ws/config.go
services/ws/event_source.go
services/ws/event_source_test.go
services/ws/server.go
vendor/vendor.json

index c480ffda4cca9c5141e954c75bbc41c35fd67b67..12fdd219fc698226033e0283baec5a7ad087e920 100644 (file)
--- a/README.md
+++ b/README.md
@@ -19,9 +19,7 @@ Arvados consists of:
 
 ## Quick start
 
-Curoverse maintains an Arvados public cloud demo at
-[https://cloud.curoverse.com](https://cloud.curoverse.com).  A Google account
-is required to log in.
+Veritas Genetics maintains a public installation of Arvados for evaluation and trial use, the [Arvados Playground](https://playground.arvados.org). A Google account is required to log in.
 
 To try out Arvados on your local workstation, you can use Arvbox, which
 provides Arvados components pre-installed in a Docker container (requires
index 4b18d037b6b30655714ed174fad84b665ebe7f9f..c5a73cbe35a6116fdbed0b8f364f2af4f0e83df5 100755 (executable)
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/bash -xe
 # Copyright (C) The Arvados Authors. All rights reserved.
 #
 # SPDX-License-Identifier: AGPL-3.0
@@ -310,6 +310,7 @@ handle_rails_package() {
     cd "$srcdir"
     local license_path="$1"; shift
     local version="$(version_from_git)"
+    echo "$version" >package-build.version
     local scripts_dir="$(mktemp --tmpdir -d "$pkgname-XXXXXXXX.scripts")" && \
     (
         set -e
diff --git a/doc/_includes/_skip_sso_server_install.liquid b/doc/_includes/_skip_sso_server_install.liquid
deleted file mode 100644 (file)
index eafa4cc..0000000
+++ /dev/null
@@ -1,12 +0,0 @@
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-<div class="alert alert-block alert-info">
-  <button type="button" class="close" data-dismiss="alert">&times;</button>
-  <h4>Note!</h4>
-  <p>The SSO server codebase currently uses OpenID 2.0 to talk to Google's authentication service. Google <a href="https://developers.google.com/accounts/docs/OpenID2">has deprecated that protocol</a>. This means that new clients will not be allowed to talk to Google's authentication services anymore over OpenID 2.0, and they will phase out the use of OpenID 2.0 completely in the coming monts. We are working on upgrading the SSO server codebase to a newer protocol. That work should be complete by the end of November 2014. In the mean time, anyone is free to use the existing Curoverse SSO server for any local Arvados installation. Instructions to do so are provided on the "API server":install-api-server.html page.</p>
-  <p><strong>Recommendation: skip this step</strong></p>
-</div>
index 4961a05b56f025fc18d58f3b8ce95ee977285d58..fa497c93de484e1a5ab094d2914cf802ffb6c6bd 100644 (file)
@@ -1,7 +1,7 @@
 ---
 layout: default
 navsection: installguide
-title: Copy pipeline from Curoverse cloud
+title: Copy pipeline from the Arvados Playground
 ...
 {% comment %}
 Copyright (C) The Arvados Authors. All rights reserved.
@@ -9,27 +9,27 @@ Copyright (C) The Arvados Authors. All rights reserved.
 SPDX-License-Identifier: CC-BY-SA-3.0
 {% endcomment %}
 
-This tutorial describes how to find and copy a publicly shared pipeline from Curoverse cloud. Please note that you can use similar steps to copy any template you can access from Curoverse cloud to your cluster.
+This tutorial describes how to find and copy a publicly shared pipeline from the Arvados Playground. Please note that you can use similar steps to copy any template you can access from the Arvados Playground to your cluster.
 
-h3. Access a public pipeline in Curoverse cloud using Workbench
+h3. Access a public pipeline in the Arvados Playground using Workbench
 
-Curoverse cloud provides access to some public data, which can be used to experience Arvados in action. Let's access a public pipeline and copy it to your cluster, so that you can run it in your environment.
+the Arvados Playground provides access to some public data, which can be used to experience Arvados in action. Let's access a public pipeline and copy it to your cluster, so that you can run it in your environment.
 
-Start by visiting the "*Curoverse public projects page*":https://cloud.curoverse.com/projects/public. This page lists all the publicly accessible projects in this arvados installation. Click on one of these projects to open it. We will use "*lobSTR v.3 (Public)*":https://cloud.curoverse.com/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq as the example in this tutorial.
+Start by visiting the "*Arvados Playground public projects page*":https://playground.arvados.org/projects/public. This page lists all the publicly accessible projects in this arvados installation. Click on one of these projects to open it. We will use "*lobSTR v.3 (Public)*":https://playground.arvados.org/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq as the example in this tutorial.
 
-Once in the "*lobSTR v.3 (Public)*":https://cloud.curoverse.com/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq project, click on the *Pipeline templates* tab. In the pipeline templates tab, you will see a template named *lobSTR v.3*. Click on the <span class="fa fa-lg fa-gears"></span> *Show* button to the left of this name. This will take to you to the "*lobSTR v.3*":https://cloud.curoverse.com/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu template page.
+Once in the "*lobSTR v.3 (Public)*":https://playground.arvados.org/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq project, click on the *Pipeline templates* tab. In the pipeline templates tab, you will see a template named *lobSTR v.3*. Click on the <span class="fa fa-lg fa-gears"></span> *Show* button to the left of this name. This will take to you to the "*lobSTR v.3*":https://playground.arvados.org/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu template page.
 
 Once in this page, you can take the *uuid* of this template from the address bar, which is *qr1hi-p5p6p-9pkaxt6qjnkxhhu*. Next, we will copy this template to your Arvados instance.
 
-h3. Copying a pipeline template from Curoverse cloud to your cluster
+h3. Copying a pipeline template from the Arvados Playground to your cluster
 
-As described above, navigate to the publicly shared pipeline template "*lobSTR v.3*":https://cloud.curoverse.com/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu using Curoverse Workbench.  We will now copy this template with uuid *qr1hi-p5p6p-9pkaxt6qjnkxhhu* to your cluster.
+As described above, navigate to the publicly shared pipeline template "*lobSTR v.3*":https://playground.arvados.org/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu on the Arvados Playground.  We will now copy this template with uuid *qr1hi-p5p6p-9pkaxt6qjnkxhhu* to your cluster.
 
 {% include 'tutorial_expectations' %}
 
 We will use the Arvados *arv-copy* command to copy this template to your cluster. In order to use arv-copy, first you need to setup the source and destination cluster configuration files. Here, *qr1hi* would be the source cluster and your Arvados instance would be the *dst_cluster*.
 
-During this setup, if you have an account in Curoverse cloud, you can use "your access token":#using-your-token to create the source configuration file. If you do not have an account in Curoverse cloud, you can use the "anonymous access token":#using-anonymous-token for the source cluster configuration.
+During this setup, if you have an account in the Arvados Playground, you can use "your access token":#using-your-token to create the source configuration file. If you do not have an account in the Arvados Playground, you can use the "anonymous access token":#using-anonymous-token for the source cluster configuration.
 
 h4(#using-anonymous-token). *Configuring source and destination setup files using anonymous access token*
 
@@ -53,7 +53,7 @@ You can now copy the pipeline template from *qr1hi* to *your cluster*. Replace *
 
 h4(#using-your-token). *Configuring source and destination setup files using personal access token*
 
-If you already have an account in Curoverse cloud, you can follow the instructions in the "*Using arv-copy*":http://doc.arvados.org/user/topics/arv-copy.html user guide to get your *Current token* for source and destination clusters, and use them to create the source *qr1hi.conf* and dst_cluster.conf configuration files.
+If you already have an account in the Arvados Playground, you can follow the instructions in the "*Using arv-copy*":http://doc.arvados.org/user/topics/arv-copy.html user guide to get your *Current token* for source and destination clusters, and use them to create the source *qr1hi.conf* and dst_cluster.conf configuration files.
 
 You can now copy the pipeline template from *qr1hi* to *your cluster* with or without recursion. Replace *dst_cluster* with the *uuid_prefix* of your cluster.
 
index 216810de47174a32d5e913c5faebda29ee90a8b8..c31b2ed43c89b92b5ef8c5c15c8abdd4dd185cbe 100644 (file)
@@ -20,8 +20,8 @@ table(table table-bordered table-condensed).
 |"Arvados-in-a-box":arvbox.html (arvbox)|Easy|no|yes|no|yes|yes|
 |"Arvados on Kubernetes":arvados-on-kubernetes.html|Easy ^1^|yes|yes ^2^|no ^2^|no|yes|
 |"Manual installation":install-manual-prerequisites.html|Complicated|yes|yes|yes|no|no|
-|"Cloud demo":https://cloud.curoverse.com by Veritas Genetics|N/A ^3^|yes|yes|no|no|no|
-|"Cluster Operation Subscription":https://curoverse.com/products by Veritas Genetics|N/A ^3^|yes|yes|yes|yes|yes|
+|"Arvados Playground":https://playground.arvados.org hosted by Veritas Genetics|N/A ^3^|yes|yes|no|no|no|
+|"Cluster Operation Subscription":https://curoverse.com/products supported by Veritas Genetics|N/A ^3^|yes|yes|yes|yes|yes|
 </div>
 
 * ^1^ Assumes a Kubernetes cluster is available
index 53adcd5c1c727ac56ccf771755ef5feea9d6bcef..43369a3bbfc230f3bf95a55923211dc24566c606 100644 (file)
@@ -11,7 +11,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0
 
 h2. LobSTR v3 
 
-In this quickstart guide, we'll run an existing pipeline with pre-existing data. Step-by-step instructions are shown below. You can follow along using your own local install or by using Curoverse's <a href="http://lp.curoverse.com/beta-signup/">hosted version of Arvados</a> (in public beta, any Google account can be used to login).
+In this quickstart guide, we'll run an existing pipeline with pre-existing data. Step-by-step instructions are shown below. You can follow along using your own local install or by using the <a href="https://playground.arvados.org/">Arvados Playground</a> (any Google account can be used to log in).
 
 (For more information about this pipeline, see our <a href="https://dev.arvados.org/projects/arvados/wiki/LobSTR_tutorial">detailed lobSTR guide</a>).
 
index 2319b3cb81f8a85046be6d103fc92efe8ac0b1d8..ad5d3bd83643e6d9134dbfddddfdf2209be66140 100644 (file)
@@ -33,7 +33,7 @@ The tutorial files are located in the "documentation section of the Arvados sour
 </code></pre>
 </notextile>
 
-The tutorial data is hosted on "https://cloud.curoverse.com":https://cloud.curoverse.com (also referred to by the identifier *qr1hi*).  If you are using a different Arvados instance, you may need to copy the data to your own instance.  The easiest way to do this is with "arv-copy":{{site.baseurl}}/user/topics/arv-copy.html (this requires signing up for a free cloud.curoverse.com account).
+The tutorial data is hosted on "https://playground.arvados.org":https://playground.arvados.org (also referred to by the identifier *qr1hi*).  If you are using a different Arvados instance, you may need to copy the data to your own instance.  The easiest way to do this is with "arv-copy":{{site.baseurl}}/user/topics/arv-copy.html (this requires signing up for a free playground.arvados.org account).
 
 <notextile>
 <pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst settings 2463fa9efeb75e099685528b3b9071e0+438</span>
@@ -42,13 +42,13 @@ The tutorial data is hosted on "https://cloud.curoverse.com":https://cloud.curov
 </code></pre>
 </notextile>
 
-If you do not wish to create an account on "https://cloud.curoverse.com":https://cloud.curoverse.com, you may download the files anonymously and upload them to your local Arvados instance:
+If you do not wish to create an account on "https://playground.arvados.org":https://playground.arvados.org, you may download the files anonymously and upload them to your local Arvados instance:
 
-"https://cloud.curoverse.com/collections/2463fa9efeb75e099685528b3b9071e0+438":https://cloud.curoverse.com/collections/2463fa9efeb75e099685528b3b9071e0+438
+"https://playground.arvados.org/collections/2463fa9efeb75e099685528b3b9071e0+438":https://playground.arvados.org/collections/2463fa9efeb75e099685528b3b9071e0+438
 
-"https://cloud.curoverse.com/collections/ae480c5099b81e17267b7445e35b4bc7+180":https://cloud.curoverse.com/collections/ae480c5099b81e17267b7445e35b4bc7+180
+"https://playground.arvados.org/collections/ae480c5099b81e17267b7445e35b4bc7+180":https://playground.arvados.org/collections/ae480c5099b81e17267b7445e35b4bc7+180
 
-"https://cloud.curoverse.com/collections/655c6cd07550151b210961ed1d3852cf+57":https://cloud.curoverse.com/collections/655c6cd07550151b210961ed1d3852cf+57
+"https://playground.arvados.org/collections/655c6cd07550151b210961ed1d3852cf+57":https://playground.arvados.org/collections/655c6cd07550151b210961ed1d3852cf+57
 
 h2. Submitting a workflow to an Arvados cluster
 
index 74868bcab4a0b76d44cbd43eee0c2c89d73b0311..f1adfe28545fe235ddd64e0eed882c7a84966e88 100644 (file)
@@ -21,7 +21,7 @@ h2. arv-copy
 
 @arv-copy@ allows users to copy collections and pipeline templates from one cluster to another. By default, @arv-copy@ will recursively go through a template and copy all dependencies associated with the object.
 
-For example, let's copy from the <a href="https://cloud.curoverse.com/">cloud instance *qr1hi*</a> to *dst_cluster*. The names *qr1hi* and *dst_cluster* are interchangable with any cluster name. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *qr1hi*-4zz18-tci4vn4fa95w0zx, the cluster name is qr1hi.
+For example, let's copy from the <a href="https://playground.arvados.org/">Arvados playground</a>, also known as *qr1hi*, to *dst_cluster*. The names *qr1hi* and *dst_cluster* are interchangable with any cluster name. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *qr1hi*-4zz18-tci4vn4fa95w0zx, the cluster name is qr1hi.
 
 In order to communicate with both clusters, you must create custom configuration files for each cluster. In the Arvados Workbench, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu, and click on the menu item *Current token*. Copy the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ in both of your clusters. Then, create two configuration files, one for each cluster. The names of the files must have the format of *uuid_prefix.conf*. In our example, let's make two files, one for *qr1hi* and one for *dst_cluster*. From your *Current token* page in *qr1hi* and *dst_cluster*, copy the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@.
 
@@ -44,7 +44,7 @@ h3. How to copy a collection
 
 First, select the uuid of the collection you want to copy from the source cluster. The uuid can be found in the collection display page in the collection summary area (top left box), or from the URL bar (the part after @collections/...@)
 
-Now copy the collection from *qr1hi* to *dst_cluster*. We will use the uuid @qr1hi-4zz18-tci4vn4fa95w0zx@ as an example. You can find this collection in the <a href="https://cloud.curoverse.com/collections/qr1hi-4zz18-tci4vn4fa95w0zx">lobSTR v.3 project on cloud.curoverse.com</a>.
+Now copy the collection from *qr1hi* to *dst_cluster*. We will use the uuid @qr1hi-4zz18-tci4vn4fa95w0zx@ as an example. You can find this collection in the <a href="https://playground.arvados.org/collections/qr1hi-4zz18-tci4vn4fa95w0zx">lobSTR v.3 project on playground.arvados.org</a>.
 <notextile>
 <pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster qr1hi-4zz18-tci4vn4fa95w0zx</span>
 qr1hi-4zz18-tci4vn4fa95w0zx: 6.1M / 6.1M 100.0%
index bcbf148e5d66014ff0e519b810d9433981f97d92..a4e58b84be12b29f8f2950a1098d7490910696f6 100644 (file)
@@ -56,7 +56,7 @@ See the "run-command reference":{{site.baseurl}}/user/topics/run-command.html fo
 <pre><code>~$ <span class="userinput">git rev-parse HEAD</span></code></pre>
 </notextile>
 
-* @"docker_image"@ : The docker image hash used is found on the "Collection page":https://cloud.curoverse.com/collections/qr1hi-4zz18-dov6im679g3jr1n as the *Content address*.
+* @"docker_image"@ : The docker image hash used is found on the "Collection page":https://playground.arvados.org/collections/qr1hi-4zz18-dov6im679g3jr1n as the *Content address*.
 
 h2. Running your pipeline
 
@@ -82,4 +82,4 @@ Note: Job reuse can only happen if all input collections do not change.
 <pre><code>~$ <span class="userinput">git rev-parse HEAD</span></code></pre>
 </notextile>
 
-* @"docker_image"@ : This specifies the "Docker":https://www.docker.com/ runtime environment where jobs run their scripts. Docker version control is similar to git, and you can commit and push changes to your images. You must re-use the docker image hash from the previous run to use the same image. It can be found on the "Collection page":https://cloud.curoverse.com/collections/qr1hi-4zz18-dov6im679g3jr1n as the *Content address* or the *docker_image_locator* in a job's metadata.
+* @"docker_image"@ : This specifies the "Docker":https://www.docker.com/ runtime environment where jobs run their scripts. Docker version control is similar to git, and you can commit and push changes to your images. You must re-use the docker image hash from the previous run to use the same image. It can be found on the "Collection page":https://playground.arvados.org/collections/qr1hi-4zz18-dov6im679g3jr1n as the *Content address* or the *docker_image_locator* in a job's metadata.
diff --git a/lib/controller/federation.go b/lib/controller/federation.go
new file mode 100644 (file)
index 0000000..24b9250
--- /dev/null
@@ -0,0 +1,117 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: AGPL-3.0
+
+package controller
+
+import (
+       "bytes"
+       "database/sql"
+       "io/ioutil"
+       "net/http"
+       "net/url"
+       "regexp"
+
+       "git.curoverse.com/arvados.git/sdk/go/arvados"
+       "git.curoverse.com/arvados.git/sdk/go/auth"
+       "git.curoverse.com/arvados.git/sdk/go/httpserver"
+)
+
+var wfRe = regexp.MustCompile(`^/arvados/v1/workflows/([0-9a-z]{5})-[^/]+$`)
+
+func (h *Handler) proxyRemoteCluster(w http.ResponseWriter, req *http.Request, next http.Handler) {
+       m := wfRe.FindStringSubmatch(req.URL.Path)
+       if len(m) < 2 || m[1] == h.Cluster.ClusterID {
+               next.ServeHTTP(w, req)
+               return
+       }
+       remoteID := m[1]
+       remote, ok := h.Cluster.RemoteClusters[remoteID]
+       if !ok {
+               httpserver.Error(w, "no proxy available for cluster "+remoteID, http.StatusNotFound)
+               return
+       }
+       scheme := remote.Scheme
+       if scheme == "" {
+               scheme = "https"
+       }
+       err := h.saltAuthToken(req, remoteID)
+       if err != nil {
+               httpserver.Error(w, err.Error(), http.StatusBadRequest)
+               return
+       }
+       urlOut := &url.URL{
+               Scheme:   scheme,
+               Host:     remote.Host,
+               Path:     req.URL.Path,
+               RawPath:  req.URL.RawPath,
+               RawQuery: req.URL.RawQuery,
+       }
+       client := h.secureClient
+       if remote.Insecure {
+               client = h.insecureClient
+       }
+       h.proxy.Do(w, req, urlOut, client)
+}
+
+// Extract the auth token supplied in req, and replace it with a
+// salted token for the remote cluster.
+func (h *Handler) saltAuthToken(req *http.Request, remote string) error {
+       creds := auth.NewCredentials()
+       creds.LoadTokensFromHTTPRequest(req)
+       if len(creds.Tokens) == 0 && req.Header.Get("Content-Type") == "application/x-www-form-encoded" {
+               // Override ParseForm's 10MiB limit by ensuring
+               // req.Body is a *http.maxBytesReader.
+               req.Body = http.MaxBytesReader(nil, req.Body, 1<<28) // 256MiB. TODO: use MaxRequestSize from discovery doc or config.
+               if err := creds.LoadTokensFromHTTPRequestBody(req); err != nil {
+                       return err
+               }
+               // Replace req.Body with a buffer that re-encodes the
+               // form without api_token, in case we end up
+               // forwarding the request.
+               if req.PostForm != nil {
+                       req.PostForm.Del("api_token")
+               }
+               req.Body = ioutil.NopCloser(bytes.NewBufferString(req.PostForm.Encode()))
+       }
+       if len(creds.Tokens) == 0 {
+               return nil
+       }
+       token, err := auth.SaltToken(creds.Tokens[0], remote)
+       if err == auth.ErrObsoleteToken {
+               // If the token exists in our own database, salt it
+               // for the remote. Otherwise, assume it was issued by
+               // the remote, and pass it through unmodified.
+               db, err := h.db(req)
+               if err != nil {
+                       return err
+               }
+               aca := arvados.APIClientAuthorization{APIToken: creds.Tokens[0]}
+               err = db.QueryRowContext(req.Context(), `SELECT uuid FROM api_client_authorizations WHERE api_token=$1 AND (expires_at IS NULL OR expires_at > current_timestamp) LIMIT 1`, aca.APIToken).Scan(&aca.UUID)
+               if err == sql.ErrNoRows {
+                       // Not ours; pass through unmodified.
+                       token = aca.APIToken
+               } else if err != nil {
+                       return err
+               } else {
+                       // Found; make V2 version and salt it.
+                       token, err = auth.SaltToken(aca.TokenV2(), remote)
+                       if err != nil {
+                               return err
+                       }
+               }
+       } else if err != nil {
+               return err
+       }
+       req.Header.Set("Authorization", "Bearer "+token)
+
+       // Remove api_token=... from the the query string, in case we
+       // end up forwarding the request.
+       if values, err := url.ParseQuery(req.URL.RawQuery); err != nil {
+               return err
+       } else if _, ok := values["api_token"]; ok {
+               delete(values, "api_token")
+               req.URL.RawQuery = values.Encode()
+       }
+       return nil
+}
diff --git a/lib/controller/federation_test.go b/lib/controller/federation_test.go
new file mode 100644 (file)
index 0000000..2682092
--- /dev/null
@@ -0,0 +1,301 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: AGPL-3.0
+
+package controller
+
+import (
+       "encoding/json"
+       "io/ioutil"
+       "net/http"
+       "net/http/httptest"
+       "net/url"
+       "strings"
+       "time"
+
+       "git.curoverse.com/arvados.git/sdk/go/arvados"
+       "git.curoverse.com/arvados.git/sdk/go/arvadostest"
+       "git.curoverse.com/arvados.git/sdk/go/httpserver"
+       "github.com/Sirupsen/logrus"
+       check "gopkg.in/check.v1"
+)
+
+// Gocheck boilerplate
+var _ = check.Suite(&FederationSuite{})
+
+type FederationSuite struct {
+       log *logrus.Logger
+       // testServer and testHandler are the controller being tested,
+       // "zhome".
+       testServer  *httpserver.Server
+       testHandler *Handler
+       // remoteServer ("zzzzz") forwards requests to the Rails API
+       // provided by the integration test environment.
+       remoteServer *httpserver.Server
+       // remoteMock ("zmock") appends each incoming request to
+       // remoteMockRequests, and returns an empty 200 response.
+       remoteMock         *httpserver.Server
+       remoteMockRequests []http.Request
+}
+
+func (s *FederationSuite) SetUpTest(c *check.C) {
+       s.log = logrus.New()
+       s.log.Formatter = &logrus.JSONFormatter{}
+       s.log.Out = &logWriter{c.Log}
+
+       s.remoteServer = newServerFromIntegrationTestEnv(c)
+       c.Assert(s.remoteServer.Start(), check.IsNil)
+
+       s.remoteMock = newServerFromIntegrationTestEnv(c)
+       s.remoteMock.Server.Handler = http.HandlerFunc(s.remoteMockHandler)
+       c.Assert(s.remoteMock.Start(), check.IsNil)
+
+       nodeProfile := arvados.NodeProfile{
+               Controller: arvados.SystemServiceInstance{Listen: ":"},
+               RailsAPI:   arvados.SystemServiceInstance{Listen: ":1"}, // local reqs will error "connection refused"
+       }
+       s.testHandler = &Handler{Cluster: &arvados.Cluster{
+               ClusterID:  "zhome",
+               PostgreSQL: integrationTestCluster().PostgreSQL,
+               NodeProfiles: map[string]arvados.NodeProfile{
+                       "*": nodeProfile,
+               },
+       }, NodeProfile: &nodeProfile}
+       s.testServer = newServerFromIntegrationTestEnv(c)
+       s.testServer.Server.Handler = httpserver.AddRequestIDs(httpserver.LogRequests(s.log, s.testHandler))
+
+       s.testHandler.Cluster.RemoteClusters = map[string]arvados.RemoteCluster{
+               "zzzzz": {
+                       Host:   s.remoteServer.Addr,
+                       Proxy:  true,
+                       Scheme: "http",
+               },
+               "zmock": {
+                       Host:   s.remoteMock.Addr,
+                       Proxy:  true,
+                       Scheme: "http",
+               },
+       }
+
+       c.Assert(s.testServer.Start(), check.IsNil)
+
+       s.remoteMockRequests = nil
+}
+
+func (s *FederationSuite) remoteMockHandler(w http.ResponseWriter, req *http.Request) {
+       s.remoteMockRequests = append(s.remoteMockRequests, *req)
+}
+
+func (s *FederationSuite) TearDownTest(c *check.C) {
+       if s.remoteServer != nil {
+               s.remoteServer.Close()
+       }
+       if s.testServer != nil {
+               s.testServer.Close()
+       }
+}
+
+func (s *FederationSuite) testRequest(req *http.Request) *http.Response {
+       resp := httptest.NewRecorder()
+       s.testServer.Server.Handler.ServeHTTP(resp, req)
+       return resp.Result()
+}
+
+func (s *FederationSuite) TestLocalRequest(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+strings.Replace(arvadostest.WorkflowWithDefinitionYAMLUUID, "zzzzz-", "zhome-", 1), nil)
+       resp := s.testRequest(req)
+       s.checkHandledLocally(c, resp)
+}
+
+func (s *FederationSuite) checkHandledLocally(c *check.C, resp *http.Response) {
+       // Our "home" controller can't handle local requests because
+       // it doesn't have its own stub/test Rails API, so we rely on
+       // "connection refused" to indicate the controller tried to
+       // proxy the request to its local Rails API.
+       c.Check(resp.StatusCode, check.Equals, http.StatusBadGateway)
+       s.checkJSONErrorMatches(c, resp, `.*connection refused`)
+}
+
+func (s *FederationSuite) TestNoAuth(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusUnauthorized)
+       s.checkJSONErrorMatches(c, resp, `Not logged in`)
+}
+
+func (s *FederationSuite) TestBadAuth(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       req.Header.Set("Authorization", "Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusUnauthorized)
+       s.checkJSONErrorMatches(c, resp, `Not logged in`)
+}
+
+func (s *FederationSuite) TestNoAccess(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.SpectatorToken)
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusNotFound)
+       s.checkJSONErrorMatches(c, resp, `.*not found`)
+}
+
+func (s *FederationSuite) TestGetUnknownRemote(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+strings.Replace(arvadostest.WorkflowWithDefinitionYAMLUUID, "zzzzz-", "zz404-", 1), nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusNotFound)
+       s.checkJSONErrorMatches(c, resp, `.*no proxy available for cluster zz404`)
+}
+
+func (s *FederationSuite) TestRemoteError(c *check.C) {
+       rc := s.testHandler.Cluster.RemoteClusters["zzzzz"]
+       rc.Scheme = "https"
+       s.testHandler.Cluster.RemoteClusters["zzzzz"] = rc
+
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusBadGateway)
+       s.checkJSONErrorMatches(c, resp, `.*HTTP response to HTTPS client`)
+}
+
+func (s *FederationSuite) TestGetRemoteWorkflow(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       var wf arvados.Workflow
+       c.Check(json.NewDecoder(resp.Body).Decode(&wf), check.IsNil)
+       c.Check(wf.UUID, check.Equals, arvadostest.WorkflowWithDefinitionYAMLUUID)
+       c.Check(wf.OwnerUUID, check.Equals, arvadostest.ActiveUserUUID)
+}
+
+func (s *FederationSuite) TestOptionsMethod(c *check.C) {
+       req := httptest.NewRequest("OPTIONS", "/arvados/v1/workflows/"+arvadostest.WorkflowWithDefinitionYAMLUUID, nil)
+       req.Header.Set("Origin", "https://example.com")
+       resp := s.testRequest(req)
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       body, err := ioutil.ReadAll(resp.Body)
+       c.Check(err, check.IsNil)
+       c.Check(string(body), check.Equals, "")
+       c.Check(resp.Header.Get("Access-Control-Allow-Origin"), check.Equals, "*")
+       for _, hdr := range []string{"Authorization", "Content-Type"} {
+               c.Check(resp.Header.Get("Access-Control-Allow-Headers"), check.Matches, ".*"+hdr+".*")
+       }
+       for _, method := range []string{"GET", "HEAD", "PUT", "POST", "DELETE"} {
+               c.Check(resp.Header.Get("Access-Control-Allow-Methods"), check.Matches, ".*"+method+".*")
+       }
+}
+
+func (s *FederationSuite) TestRemoteWithTokenInQuery(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+strings.Replace(arvadostest.WorkflowWithDefinitionYAMLUUID, "zzzzz-", "zmock-", 1)+"?api_token="+arvadostest.ActiveToken, nil)
+       s.testRequest(req)
+       c.Assert(len(s.remoteMockRequests), check.Equals, 1)
+       pr := s.remoteMockRequests[0]
+       // Token is salted and moved from query to Authorization header.
+       c.Check(pr.URL.String(), check.Not(check.Matches), `.*api_token=.*`)
+       c.Check(pr.Header.Get("Authorization"), check.Equals, "Bearer v2/zzzzz-gj3su-077z32aux8dg2s1/7fd31b61f39c0e82a4155592163218272cedacdc")
+}
+
+func (s *FederationSuite) TestLocalTokenSalted(c *check.C) {
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+strings.Replace(arvadostest.WorkflowWithDefinitionYAMLUUID, "zzzzz-", "zmock-", 1), nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       s.testRequest(req)
+       c.Assert(len(s.remoteMockRequests), check.Equals, 1)
+       pr := s.remoteMockRequests[0]
+       // The salted token here has a "zzzzz-" UUID instead of a
+       // "ztest-" UUID because ztest's local database has the
+       // "zzzzz-" test fixtures. The "secret" part is HMAC(sha1,
+       // arvadostest.ActiveToken, "zmock") = "7fd3...".
+       c.Check(pr.Header.Get("Authorization"), check.Equals, "Bearer v2/zzzzz-gj3su-077z32aux8dg2s1/7fd31b61f39c0e82a4155592163218272cedacdc")
+}
+
+func (s *FederationSuite) TestRemoteTokenNotSalted(c *check.C) {
+       // remoteToken can be any v1 token that doesn't appear in
+       // ztest's local db.
+       remoteToken := "abcdef00000000000000000000000000000000000000000000"
+       req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+strings.Replace(arvadostest.WorkflowWithDefinitionYAMLUUID, "zzzzz-", "zmock-", 1), nil)
+       req.Header.Set("Authorization", "Bearer "+remoteToken)
+       s.testRequest(req)
+       c.Assert(len(s.remoteMockRequests), check.Equals, 1)
+       pr := s.remoteMockRequests[0]
+       c.Check(pr.Header.Get("Authorization"), check.Equals, "Bearer "+remoteToken)
+}
+
+func (s *FederationSuite) TestWorkflowCRUD(c *check.C) {
+       wf := arvados.Workflow{
+               Description: "TestCRUD",
+       }
+       {
+               body := &strings.Builder{}
+               json.NewEncoder(body).Encode(&wf)
+               req := httptest.NewRequest("POST", "/arvados/v1/workflows", strings.NewReader(url.Values{
+                       "workflow": {body.String()},
+               }.Encode()))
+               req.Header.Set("Content-type", "application/x-www-form-urlencoded")
+               req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+               rec := httptest.NewRecorder()
+               s.remoteServer.Server.Handler.ServeHTTP(rec, req) // direct to remote -- can't proxy a create req because no uuid
+               resp := rec.Result()
+               s.checkResponseOK(c, resp)
+               json.NewDecoder(resp.Body).Decode(&wf)
+
+               defer func() {
+                       req := httptest.NewRequest("DELETE", "/arvados/v1/workflows/"+wf.UUID, nil)
+                       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+                       s.remoteServer.Server.Handler.ServeHTTP(httptest.NewRecorder(), req)
+               }()
+               c.Check(wf.UUID, check.Not(check.Equals), "")
+
+               c.Assert(wf.ModifiedAt, check.NotNil)
+               c.Logf("wf.ModifiedAt: %v", wf.ModifiedAt)
+               c.Check(time.Since(*wf.ModifiedAt) < time.Minute, check.Equals, true)
+       }
+       for _, method := range []string{"PATCH", "PUT", "POST"} {
+               form := url.Values{
+                       "workflow": {`{"description": "Updated with ` + method + `"}`},
+               }
+               if method == "POST" {
+                       form["_method"] = []string{"PATCH"}
+               }
+               req := httptest.NewRequest(method, "/arvados/v1/workflows/"+wf.UUID, strings.NewReader(form.Encode()))
+               req.Header.Set("Content-type", "application/x-www-form-urlencoded")
+               req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+               resp := s.testRequest(req)
+               s.checkResponseOK(c, resp)
+               err := json.NewDecoder(resp.Body).Decode(&wf)
+               c.Check(err, check.IsNil)
+
+               c.Check(wf.Description, check.Equals, "Updated with "+method)
+       }
+       {
+               req := httptest.NewRequest("DELETE", "/arvados/v1/workflows/"+wf.UUID, nil)
+               req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+               resp := s.testRequest(req)
+               s.checkResponseOK(c, resp)
+               err := json.NewDecoder(resp.Body).Decode(&wf)
+               c.Check(err, check.IsNil)
+       }
+       {
+               req := httptest.NewRequest("GET", "/arvados/v1/workflows/"+wf.UUID, nil)
+               req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+               resp := s.testRequest(req)
+               c.Check(resp.StatusCode, check.Equals, http.StatusNotFound)
+       }
+}
+
+func (s *FederationSuite) checkResponseOK(c *check.C, resp *http.Response) {
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       if resp.StatusCode != http.StatusOK {
+               body, err := ioutil.ReadAll(resp.Body)
+               c.Logf("... response body = %q, %v\n", body, err)
+       }
+}
+
+func (s *FederationSuite) checkJSONErrorMatches(c *check.C, resp *http.Response, re string) {
+       var jresp httpserver.ErrorResponse
+       err := json.NewDecoder(resp.Body).Decode(&jresp)
+       c.Check(err, check.IsNil)
+       c.Assert(len(jresp.Errors), check.Equals, 1)
+       c.Check(jresp.Errors[0], check.Matches, re)
+}
index a1a69a88e4ccd1bbb6d4882620581e91b3a03523..25799aae9ea30d3116469e205c77f927c313172a 100644 (file)
@@ -5,8 +5,8 @@
 package controller
 
 import (
-       "context"
-       "io"
+       "database/sql"
+       "errors"
        "net"
        "net/http"
        "net/url"
@@ -17,15 +17,20 @@ import (
        "git.curoverse.com/arvados.git/sdk/go/arvados"
        "git.curoverse.com/arvados.git/sdk/go/health"
        "git.curoverse.com/arvados.git/sdk/go/httpserver"
+       _ "github.com/lib/pq"
 )
 
 type Handler struct {
        Cluster     *arvados.Cluster
        NodeProfile *arvados.NodeProfile
 
-       setupOnce    sync.Once
-       handlerStack http.Handler
-       proxyClient  *arvados.Client
+       setupOnce      sync.Once
+       handlerStack   http.Handler
+       proxy          *proxy
+       secureClient   *http.Client
+       insecureClient *http.Client
+       pgdb           *sql.DB
+       pgdbMtx        sync.Mutex
 }
 
 func (h *Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
@@ -49,104 +54,96 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 
 func (h *Handler) CheckHealth() error {
        h.setupOnce.Do(h.setup)
-       _, err := findRailsAPI(h.Cluster, h.NodeProfile)
+       _, _, err := findRailsAPI(h.Cluster, h.NodeProfile)
        return err
 }
 
+func neverRedirect(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }
+
 func (h *Handler) setup() {
        mux := http.NewServeMux()
        mux.Handle("/_health/", &health.Handler{
                Token:  h.Cluster.ManagementToken,
                Prefix: "/_health/",
        })
-       mux.Handle("/", http.HandlerFunc(h.proxyRailsAPI))
+       hs := http.NotFoundHandler()
+       hs = prepend(hs, h.proxyRailsAPI)
+       hs = prepend(hs, h.proxyRemoteCluster)
+       mux.Handle("/", hs)
        h.handlerStack = mux
 
-       // Changing the global isn't the right way to do this, but a
-       // proper solution would conflict with an impending 13493
-       // merge anyway, so this will do for now.
-       arvados.InsecureHTTPClient.CheckRedirect = func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }
-}
+       sc := *arvados.DefaultSecureClient
+       sc.Timeout = time.Duration(h.Cluster.HTTPRequestTimeout)
+       sc.CheckRedirect = neverRedirect
+       h.secureClient = &sc
 
-// headers that shouldn't be forwarded when proxying. See
-// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
-var dropHeaders = map[string]bool{
-       "Connection":          true,
-       "Keep-Alive":          true,
-       "Proxy-Authenticate":  true,
-       "Proxy-Authorization": true,
-       "TE":                true,
-       "Trailer":           true,
-       "Transfer-Encoding": true,
-       "Upgrade":           true,
-}
+       ic := *arvados.InsecureHTTPClient
+       ic.Timeout = time.Duration(h.Cluster.HTTPRequestTimeout)
+       ic.CheckRedirect = neverRedirect
+       h.insecureClient = &ic
 
-func (h *Handler) proxyRailsAPI(w http.ResponseWriter, reqIn *http.Request) {
-       urlOut, err := findRailsAPI(h.Cluster, h.NodeProfile)
-       if err != nil {
-               httpserver.Error(w, err.Error(), http.StatusInternalServerError)
-               return
+       h.proxy = &proxy{
+               Name:           "arvados-controller",
+               RequestTimeout: time.Duration(h.Cluster.HTTPRequestTimeout),
        }
-       urlOut = &url.URL{
-               Scheme:   urlOut.Scheme,
-               Host:     urlOut.Host,
-               Path:     reqIn.URL.Path,
-               RawPath:  reqIn.URL.RawPath,
-               RawQuery: reqIn.URL.RawQuery,
+}
+
+var errDBConnection = errors.New("database connection error")
+
+func (h *Handler) db(req *http.Request) (*sql.DB, error) {
+       h.pgdbMtx.Lock()
+       defer h.pgdbMtx.Unlock()
+       if h.pgdb != nil {
+               return h.pgdb, nil
        }
 
-       // Copy headers from incoming request, then add/replace proxy
-       // headers like Via and X-Forwarded-For.
-       hdrOut := http.Header{}
-       for k, v := range reqIn.Header {
-               if !dropHeaders[k] {
-                       hdrOut[k] = v
-               }
+       db, err := sql.Open("postgres", h.Cluster.PostgreSQL.Connection.String())
+       if err != nil {
+               httpserver.Logger(req).WithError(err).Error("postgresql connect failed")
+               return nil, errDBConnection
        }
-       xff := reqIn.RemoteAddr
-       if xffIn := reqIn.Header.Get("X-Forwarded-For"); xffIn != "" {
-               xff = xffIn + "," + xff
+       if p := h.Cluster.PostgreSQL.ConnectionPool; p > 0 {
+               db.SetMaxOpenConns(p)
        }
-       hdrOut.Set("X-Forwarded-For", xff)
-       if hdrOut.Get("X-Forwarded-Proto") == "" {
-               hdrOut.Set("X-Forwarded-Proto", reqIn.URL.Scheme)
+       if err := db.Ping(); err != nil {
+               httpserver.Logger(req).WithError(err).Error("postgresql connect succeeded but ping failed")
+               return nil, errDBConnection
        }
-       hdrOut.Add("Via", reqIn.Proto+" arvados-controller")
+       h.pgdb = db
+       return db, nil
+}
 
-       ctx := reqIn.Context()
-       if timeout := h.Cluster.HTTPRequestTimeout; timeout > 0 {
-               var cancel context.CancelFunc
-               ctx, cancel = context.WithDeadline(ctx, time.Now().Add(time.Duration(timeout)))
-               defer cancel()
-       }
+type middlewareFunc func(http.ResponseWriter, *http.Request, http.Handler)
+
+func prepend(next http.Handler, middleware middlewareFunc) http.Handler {
+       return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
+               middleware(w, req, next)
+       })
+}
 
-       reqOut := (&http.Request{
-               Method: reqIn.Method,
-               URL:    urlOut,
-               Host:   reqIn.Host,
-               Header: hdrOut,
-               Body:   reqIn.Body,
-       }).WithContext(ctx)
-       resp, err := arvados.InsecureHTTPClient.Do(reqOut)
+func (h *Handler) proxyRailsAPI(w http.ResponseWriter, req *http.Request, next http.Handler) {
+       urlOut, insecure, err := findRailsAPI(h.Cluster, h.NodeProfile)
        if err != nil {
                httpserver.Error(w, err.Error(), http.StatusInternalServerError)
                return
        }
-       for k, v := range resp.Header {
-               for _, v := range v {
-                       w.Header().Add(k, v)
-               }
+       urlOut = &url.URL{
+               Scheme:   urlOut.Scheme,
+               Host:     urlOut.Host,
+               Path:     req.URL.Path,
+               RawPath:  req.URL.RawPath,
+               RawQuery: req.URL.RawQuery,
        }
-       w.WriteHeader(resp.StatusCode)
-       n, err := io.Copy(w, resp.Body)
-       if err != nil {
-               httpserver.Logger(reqIn).WithError(err).WithField("bytesCopied", n).Error("error copying response body")
+       client := h.secureClient
+       if insecure {
+               client = h.insecureClient
        }
+       h.proxy.Do(w, req, urlOut, client)
 }
 
 // For now, findRailsAPI always uses the rails API running on this
 // node.
-func findRailsAPI(cluster *arvados.Cluster, np *arvados.NodeProfile) (*url.URL, error) {
+func findRailsAPI(cluster *arvados.Cluster, np *arvados.NodeProfile) (*url.URL, bool, error) {
        hostport := np.RailsAPI.Listen
        if len(hostport) > 1 && hostport[0] == ':' && strings.TrimRight(hostport[1:], "0123456789") == "" {
                // ":12345" => connect to indicated port on localhost
@@ -154,11 +151,12 @@ func findRailsAPI(cluster *arvados.Cluster, np *arvados.NodeProfile) (*url.URL,
        } else if _, _, err := net.SplitHostPort(hostport); err == nil {
                // "[::1]:12345" => connect to indicated address & port
        } else {
-               return nil, err
+               return nil, false, err
        }
        proto := "http"
        if np.RailsAPI.TLS {
                proto = "https"
        }
-       return url.Parse(proto + "://" + hostport)
+       url, err := url.Parse(proto + "://" + hostport)
+       return url, np.RailsAPI.Insecure, err
 }
index eb947ea363705293679da1edd3430e2d8d5c0657..963fd1159415e16d93e676401021e974c287ad69 100644 (file)
@@ -34,11 +34,12 @@ type HandlerSuite struct {
 
 func (s *HandlerSuite) SetUpTest(c *check.C) {
        s.cluster = &arvados.Cluster{
-               ClusterID: "zzzzz",
+               ClusterID:  "zzzzz",
+               PostgreSQL: integrationTestCluster().PostgreSQL,
                NodeProfiles: map[string]arvados.NodeProfile{
                        "*": {
                                Controller: arvados.SystemServiceInstance{Listen: ":"},
-                               RailsAPI:   arvados.SystemServiceInstance{Listen: os.Getenv("ARVADOS_TEST_API_HOST"), TLS: true},
+                               RailsAPI:   arvados.SystemServiceInstance{Listen: os.Getenv("ARVADOS_TEST_API_HOST"), TLS: true, Insecure: true},
                        },
                },
        }
@@ -65,12 +66,12 @@ func (s *HandlerSuite) TestRequestTimeout(c *check.C) {
        req := httptest.NewRequest("GET", "/discovery/v1/apis/arvados/v1/rest", nil)
        resp := httptest.NewRecorder()
        s.handler.ServeHTTP(resp, req)
-       c.Check(resp.Code, check.Equals, http.StatusInternalServerError)
+       c.Check(resp.Code, check.Equals, http.StatusBadGateway)
        var jresp httpserver.ErrorResponse
        err := json.Unmarshal(resp.Body.Bytes(), &jresp)
        c.Check(err, check.IsNil)
        c.Assert(len(jresp.Errors), check.Equals, 1)
-       c.Check(jresp.Errors[0], check.Matches, `.*context deadline exceeded`)
+       c.Check(jresp.Errors[0], check.Matches, `.*context deadline exceeded.*`)
 }
 
 func (s *HandlerSuite) TestProxyWithoutToken(c *check.C) {
@@ -101,6 +102,7 @@ func (s *HandlerSuite) TestProxyWithTokenInRequestBody(c *check.C) {
                "_method":   {"GET"},
                "api_token": {arvadostest.ActiveToken},
        }.Encode()))
+       req.Header.Set("Content-type", "application/x-www-form-urlencoded")
        resp := httptest.NewRecorder()
        s.handler.ServeHTTP(resp, req)
        c.Check(resp.Code, check.Equals, http.StatusOK)
@@ -122,9 +124,9 @@ func (s *HandlerSuite) TestProxyNotFound(c *check.C) {
 }
 
 func (s *HandlerSuite) TestProxyRedirect(c *check.C) {
-       req := httptest.NewRequest("GET", "https://example.org:1234/login?return_to=foo", nil)
+       req := httptest.NewRequest("GET", "https://0.0.0.0:1/login?return_to=foo", nil)
        resp := httptest.NewRecorder()
        s.handler.ServeHTTP(resp, req)
        c.Check(resp.Code, check.Equals, http.StatusFound)
-       c.Check(resp.Header().Get("Location"), check.Matches, `https://example\.org:1234/auth/joshid\?return_to=foo&?`)
+       c.Check(resp.Header().Get("Location"), check.Matches, `https://0.0.0.0:1/auth/joshid\?return_to=foo&?`)
 }
diff --git a/lib/controller/proxy.go b/lib/controller/proxy.go
new file mode 100644 (file)
index 0000000..712071b
--- /dev/null
@@ -0,0 +1,83 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: AGPL-3.0
+
+package controller
+
+import (
+       "context"
+       "io"
+       "net/http"
+       "net/url"
+       "time"
+
+       "git.curoverse.com/arvados.git/sdk/go/httpserver"
+)
+
+type proxy struct {
+       Name           string // to use in Via header
+       RequestTimeout time.Duration
+}
+
+// headers that shouldn't be forwarded when proxying. See
+// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
+var dropHeaders = map[string]bool{
+       "Connection":          true,
+       "Keep-Alive":          true,
+       "Proxy-Authenticate":  true,
+       "Proxy-Authorization": true,
+       "TE":                true,
+       "Trailer":           true,
+       "Transfer-Encoding": true,
+       "Upgrade":           true,
+}
+
+func (p *proxy) Do(w http.ResponseWriter, reqIn *http.Request, urlOut *url.URL, client *http.Client) {
+       // Copy headers from incoming request, then add/replace proxy
+       // headers like Via and X-Forwarded-For.
+       hdrOut := http.Header{}
+       for k, v := range reqIn.Header {
+               if !dropHeaders[k] {
+                       hdrOut[k] = v
+               }
+       }
+       xff := reqIn.RemoteAddr
+       if xffIn := reqIn.Header.Get("X-Forwarded-For"); xffIn != "" {
+               xff = xffIn + "," + xff
+       }
+       hdrOut.Set("X-Forwarded-For", xff)
+       if hdrOut.Get("X-Forwarded-Proto") == "" {
+               hdrOut.Set("X-Forwarded-Proto", reqIn.URL.Scheme)
+       }
+       hdrOut.Add("Via", reqIn.Proto+" arvados-controller")
+
+       ctx := reqIn.Context()
+       if p.RequestTimeout > 0 {
+               var cancel context.CancelFunc
+               ctx, cancel = context.WithDeadline(ctx, time.Now().Add(time.Duration(p.RequestTimeout)))
+               defer cancel()
+       }
+
+       reqOut := (&http.Request{
+               Method: reqIn.Method,
+               URL:    urlOut,
+               Host:   reqIn.Host,
+               Header: hdrOut,
+               Body:   reqIn.Body,
+       }).WithContext(ctx)
+       resp, err := client.Do(reqOut)
+       if err != nil {
+               httpserver.Error(w, err.Error(), http.StatusBadGateway)
+               return
+       }
+       for k, v := range resp.Header {
+               for _, v := range v {
+                       w.Header().Add(k, v)
+               }
+       }
+       w.WriteHeader(resp.StatusCode)
+       n, err := io.Copy(w, resp.Body)
+       if err != nil {
+               httpserver.Logger(reqIn).WithError(err).WithField("bytesCopied", n).Error("error copying response body")
+       }
+}
diff --git a/lib/controller/server_test.go b/lib/controller/server_test.go
new file mode 100644 (file)
index 0000000..7742cf4
--- /dev/null
@@ -0,0 +1,68 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: AGPL-3.0
+
+package controller
+
+import (
+       "bytes"
+       "net/http"
+       "os"
+       "path/filepath"
+
+       "git.curoverse.com/arvados.git/sdk/go/arvados"
+       "git.curoverse.com/arvados.git/sdk/go/httpserver"
+       "github.com/Sirupsen/logrus"
+       check "gopkg.in/check.v1"
+)
+
+// logWriter is an io.Writer that writes by calling a "write log"
+// function, typically (*check.C)Log().
+type logWriter struct {
+       logfunc func(...interface{})
+}
+
+func (tl *logWriter) Write(buf []byte) (int, error) {
+       tl.logfunc(string(bytes.TrimRight(buf, "\n")))
+       return len(buf), nil
+}
+
+func integrationTestCluster() *arvados.Cluster {
+       cfg, err := arvados.GetConfig(filepath.Join(os.Getenv("WORKSPACE"), "tmp", "arvados.yml"))
+       if err != nil {
+               panic(err)
+       }
+       cc, err := cfg.GetCluster("zzzzz")
+       if err != nil {
+               panic(err)
+       }
+       return cc
+}
+
+// Return a new unstarted controller server, using the Rails API
+// provided by the integration-testing environment.
+func newServerFromIntegrationTestEnv(c *check.C) *httpserver.Server {
+       log := logrus.New()
+       log.Formatter = &logrus.JSONFormatter{}
+       log.Out = &logWriter{c.Log}
+
+       nodeProfile := arvados.NodeProfile{
+               Controller: arvados.SystemServiceInstance{Listen: ":"},
+               RailsAPI:   arvados.SystemServiceInstance{Listen: os.Getenv("ARVADOS_TEST_API_HOST"), TLS: true, Insecure: true},
+       }
+       handler := &Handler{Cluster: &arvados.Cluster{
+               ClusterID:  "zzzzz",
+               PostgreSQL: integrationTestCluster().PostgreSQL,
+               NodeProfiles: map[string]arvados.NodeProfile{
+                       "*": nodeProfile,
+               },
+       }, NodeProfile: &nodeProfile}
+
+       srv := &httpserver.Server{
+               Server: http.Server{
+                       Handler: httpserver.AddRequestIDs(httpserver.LogRequests(log, handler)),
+               },
+               Addr: nodeProfile.Controller.Listen,
+       }
+       return srv
+}
index 0ec2d115295749067ceb4ee105245aad73df149f..4b65546912393b52b4e05a4d1ecd21b83f44bd28 100644 (file)
@@ -3,9 +3,9 @@
 # SPDX-License-Identifier: Apache-2.0
 
 #' users.get
-#' 
+#'
 #' users.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.get(uuid)
 #' @param uuid The UUID of the User in question.
 #' @return User object.
@@ -13,9 +13,9 @@
 NULL
 
 #' users.create
-#' 
+#'
 #' users.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.create(user, ensure_unique_name = "false")
 #' @param user User object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -24,9 +24,9 @@ NULL
 NULL
 
 #' users.update
-#' 
+#'
 #' users.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.update(user, uuid)
 #' @param user User object.
 #' @param uuid The UUID of the User in question.
@@ -35,9 +35,9 @@ NULL
 NULL
 
 #' users.delete
-#' 
+#'
 #' users.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.delete(uuid)
 #' @param uuid The UUID of the User in question.
 #' @return User object.
@@ -45,93 +45,93 @@ NULL
 NULL
 
 #' users.current
-#' 
+#'
 #' users.current is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.current(NULL)
 #' @return User object.
 #' @name users.current
 NULL
 
 #' users.system
-#' 
+#'
 #' users.system is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.system(NULL)
 #' @return User object.
 #' @name users.system
 NULL
 
 #' users.activate
-#' 
+#'
 #' users.activate is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.activate(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return User object.
 #' @name users.activate
 NULL
 
 #' users.setup
-#' 
+#'
 #' users.setup is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.setup(user = NULL, openid_prefix = NULL,
 #'     repo_name = NULL, vm_uuid = NULL, send_notification_email = "false")
-#' @param user 
-#' @param openid_prefix 
-#' @param repo_name 
-#' @param vm_uuid 
-#' @param send_notification_email 
+#' @param user
+#' @param openid_prefix
+#' @param repo_name
+#' @param vm_uuid
+#' @param send_notification_email
 #' @return User object.
 #' @name users.setup
 NULL
 
 #' users.unsetup
-#' 
+#'
 #' users.unsetup is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.unsetup(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return User object.
 #' @name users.unsetup
 NULL
 
 #' users.update_uuid
-#' 
+#'
 #' users.update_uuid is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.update_uuid(uuid, new_uuid)
-#' @param uuid 
-#' @param new_uuid 
+#' @param uuid
+#' @param new_uuid
 #' @return User object.
 #' @name users.update_uuid
 NULL
 
 #' users.list
-#' 
+#'
 #' users.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$users.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return UserList object.
 #' @name users.list
 NULL
 
 #' api_client_authorizations.get
-#' 
+#'
 #' api_client_authorizations.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.get(uuid)
 #' @param uuid The UUID of the ApiClientAuthorization in question.
 #' @return ApiClientAuthorization object.
@@ -139,9 +139,9 @@ NULL
 NULL
 
 #' api_client_authorizations.create
-#' 
+#'
 #' api_client_authorizations.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.create(apiclientauthorization,
 #'     ensure_unique_name = "false")
 #' @param apiClientAuthorization ApiClientAuthorization object.
@@ -151,9 +151,9 @@ NULL
 NULL
 
 #' api_client_authorizations.update
-#' 
+#'
 #' api_client_authorizations.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.update(apiclientauthorization,
 #'     uuid)
 #' @param apiClientAuthorization ApiClientAuthorization object.
@@ -163,9 +163,9 @@ NULL
 NULL
 
 #' api_client_authorizations.delete
-#' 
+#'
 #' api_client_authorizations.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.delete(uuid)
 #' @param uuid The UUID of the ApiClientAuthorization in question.
 #' @return ApiClientAuthorization object.
@@ -173,50 +173,50 @@ NULL
 NULL
 
 #' api_client_authorizations.create_system_auth
-#' 
+#'
 #' api_client_authorizations.create_system_auth is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.create_system_auth(api_client_id = NULL,
 #'     scopes = NULL)
-#' @param api_client_id 
-#' @param scopes 
+#' @param api_client_id
+#' @param scopes
 #' @return ApiClientAuthorization object.
 #' @name api_client_authorizations.create_system_auth
 NULL
 
 #' api_client_authorizations.current
-#' 
+#'
 #' api_client_authorizations.current is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.current(NULL)
 #' @return ApiClientAuthorization object.
 #' @name api_client_authorizations.current
 NULL
 
 #' api_client_authorizations.list
-#' 
+#'
 #' api_client_authorizations.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_client_authorizations.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return ApiClientAuthorizationList object.
 #' @name api_client_authorizations.list
 NULL
 
 #' containers.get
-#' 
+#'
 #' containers.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.get(uuid)
 #' @param uuid The UUID of the Container in question.
 #' @return Container object.
@@ -224,9 +224,9 @@ NULL
 NULL
 
 #' containers.create
-#' 
+#'
 #' containers.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.create(container,
 #'     ensure_unique_name = "false")
 #' @param container Container object.
@@ -236,9 +236,9 @@ NULL
 NULL
 
 #' containers.update
-#' 
+#'
 #' containers.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.update(container,
 #'     uuid)
 #' @param container Container object.
@@ -248,9 +248,9 @@ NULL
 NULL
 
 #' containers.delete
-#' 
+#'
 #' containers.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.delete(uuid)
 #' @param uuid The UUID of the Container in question.
 #' @return Container object.
@@ -258,78 +258,78 @@ NULL
 NULL
 
 #' containers.auth
-#' 
+#'
 #' containers.auth is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.auth(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Container object.
 #' @name containers.auth
 NULL
 
 #' containers.lock
-#' 
+#'
 #' containers.lock is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.lock(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Container object.
 #' @name containers.lock
 NULL
 
 #' containers.unlock
-#' 
+#'
 #' containers.unlock is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.unlock(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Container object.
 #' @name containers.unlock
 NULL
 
 #' containers.secret_mounts
-#' 
+#'
 #' containers.secret_mounts is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.secret_mounts(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Container object.
 #' @name containers.secret_mounts
 NULL
 
 #' containers.current
-#' 
+#'
 #' containers.current is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.current(NULL)
 #' @return Container object.
 #' @name containers.current
 NULL
 
 #' containers.list
-#' 
+#'
 #' containers.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$containers.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return ContainerList object.
 #' @name containers.list
 NULL
 
 #' api_clients.get
-#' 
+#'
 #' api_clients.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_clients.get(uuid)
 #' @param uuid The UUID of the ApiClient in question.
 #' @return ApiClient object.
@@ -337,9 +337,9 @@ NULL
 NULL
 
 #' api_clients.create
-#' 
+#'
 #' api_clients.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_clients.create(apiclient,
 #'     ensure_unique_name = "false")
 #' @param apiClient ApiClient object.
@@ -349,9 +349,9 @@ NULL
 NULL
 
 #' api_clients.update
-#' 
+#'
 #' api_clients.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_clients.update(apiclient,
 #'     uuid)
 #' @param apiClient ApiClient object.
@@ -361,9 +361,9 @@ NULL
 NULL
 
 #' api_clients.delete
-#' 
+#'
 #' api_clients.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_clients.delete(uuid)
 #' @param uuid The UUID of the ApiClient in question.
 #' @return ApiClient object.
@@ -371,29 +371,29 @@ NULL
 NULL
 
 #' api_clients.list
-#' 
+#'
 #' api_clients.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$api_clients.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return ApiClientList object.
 #' @name api_clients.list
 NULL
 
 #' authorized_keys.get
-#' 
+#'
 #' authorized_keys.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$authorized_keys.get(uuid)
 #' @param uuid The UUID of the AuthorizedKey in question.
 #' @return AuthorizedKey object.
@@ -401,9 +401,9 @@ NULL
 NULL
 
 #' authorized_keys.create
-#' 
+#'
 #' authorized_keys.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$authorized_keys.create(authorizedkey,
 #'     ensure_unique_name = "false")
 #' @param authorizedKey AuthorizedKey object.
@@ -413,9 +413,9 @@ NULL
 NULL
 
 #' authorized_keys.update
-#' 
+#'
 #' authorized_keys.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$authorized_keys.update(authorizedkey,
 #'     uuid)
 #' @param authorizedKey AuthorizedKey object.
@@ -425,9 +425,9 @@ NULL
 NULL
 
 #' authorized_keys.delete
-#' 
+#'
 #' authorized_keys.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$authorized_keys.delete(uuid)
 #' @param uuid The UUID of the AuthorizedKey in question.
 #' @return AuthorizedKey object.
@@ -435,29 +435,29 @@ NULL
 NULL
 
 #' authorized_keys.list
-#' 
+#'
 #' authorized_keys.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$authorized_keys.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return AuthorizedKeyList object.
 #' @name authorized_keys.list
 NULL
 
 #' container_requests.get
-#' 
+#'
 #' container_requests.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$container_requests.get(uuid)
 #' @param uuid The UUID of the ContainerRequest in question.
 #' @return ContainerRequest object.
@@ -465,9 +465,9 @@ NULL
 NULL
 
 #' container_requests.create
-#' 
+#'
 #' container_requests.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$container_requests.create(containerrequest,
 #'     ensure_unique_name = "false")
 #' @param containerRequest ContainerRequest object.
@@ -477,9 +477,9 @@ NULL
 NULL
 
 #' container_requests.update
-#' 
+#'
 #' container_requests.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$container_requests.update(containerrequest,
 #'     uuid)
 #' @param containerRequest ContainerRequest object.
@@ -489,9 +489,9 @@ NULL
 NULL
 
 #' container_requests.delete
-#' 
+#'
 #' container_requests.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$container_requests.delete(uuid)
 #' @param uuid The UUID of the ContainerRequest in question.
 #' @return ContainerRequest object.
@@ -499,29 +499,29 @@ NULL
 NULL
 
 #' container_requests.list
-#' 
+#'
 #' container_requests.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$container_requests.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return ContainerRequestList object.
 #' @name container_requests.list
 NULL
 
 #' collections.get
-#' 
+#'
 #' collections.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.get(uuid)
 #' @param uuid The UUID of the Collection in question.
 #' @return Collection object.
@@ -529,9 +529,9 @@ NULL
 NULL
 
 #' collections.create
-#' 
+#'
 #' collections.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.create(collection,
 #'     ensure_unique_name = "false")
 #' @param collection Collection object.
@@ -541,9 +541,9 @@ NULL
 NULL
 
 #' collections.update
-#' 
+#'
 #' collections.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.update(collection,
 #'     uuid)
 #' @param collection Collection object.
@@ -553,9 +553,9 @@ NULL
 NULL
 
 #' collections.delete
-#' 
+#'
 #' collections.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.delete(uuid)
 #' @param uuid The UUID of the Collection in question.
 #' @return Collection object.
@@ -563,70 +563,70 @@ NULL
 NULL
 
 #' collections.provenance
-#' 
+#'
 #' collections.provenance is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.provenance(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Collection object.
 #' @name collections.provenance
 NULL
 
 #' collections.used_by
-#' 
+#'
 #' collections.used_by is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.used_by(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Collection object.
 #' @name collections.used_by
 NULL
 
 #' collections.trash
-#' 
+#'
 #' collections.trash is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.trash(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Collection object.
 #' @name collections.trash
 NULL
 
 #' collections.untrash
-#' 
+#'
 #' collections.untrash is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.untrash(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Collection object.
 #' @name collections.untrash
 NULL
 
 #' collections.list
-#' 
+#'
 #' collections.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$collections.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact", include_trash = NULL)
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @param include_trash Include collections whose is_trashed attribute is true.
 #' @return CollectionList object.
 #' @name collections.list
 NULL
 
 #' humans.get
-#' 
+#'
 #' humans.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$humans.get(uuid)
 #' @param uuid The UUID of the Human in question.
 #' @return Human object.
@@ -634,9 +634,9 @@ NULL
 NULL
 
 #' humans.create
-#' 
+#'
 #' humans.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$humans.create(human, ensure_unique_name = "false")
 #' @param human Human object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -645,9 +645,9 @@ NULL
 NULL
 
 #' humans.update
-#' 
+#'
 #' humans.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$humans.update(human, uuid)
 #' @param human Human object.
 #' @param uuid The UUID of the Human in question.
@@ -656,9 +656,9 @@ NULL
 NULL
 
 #' humans.delete
-#' 
+#'
 #' humans.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$humans.delete(uuid)
 #' @param uuid The UUID of the Human in question.
 #' @return Human object.
@@ -666,29 +666,29 @@ NULL
 NULL
 
 #' humans.list
-#' 
+#'
 #' humans.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$humans.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return HumanList object.
 #' @name humans.list
 NULL
 
 #' job_tasks.get
-#' 
+#'
 #' job_tasks.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$job_tasks.get(uuid)
 #' @param uuid The UUID of the JobTask in question.
 #' @return JobTask object.
@@ -696,9 +696,9 @@ NULL
 NULL
 
 #' job_tasks.create
-#' 
+#'
 #' job_tasks.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$job_tasks.create(jobtask, ensure_unique_name = "false")
 #' @param jobTask JobTask object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -707,9 +707,9 @@ NULL
 NULL
 
 #' job_tasks.update
-#' 
+#'
 #' job_tasks.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$job_tasks.update(jobtask, uuid)
 #' @param jobTask JobTask object.
 #' @param uuid The UUID of the JobTask in question.
@@ -718,9 +718,9 @@ NULL
 NULL
 
 #' job_tasks.delete
-#' 
+#'
 #' job_tasks.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$job_tasks.delete(uuid)
 #' @param uuid The UUID of the JobTask in question.
 #' @return JobTask object.
@@ -728,29 +728,29 @@ NULL
 NULL
 
 #' job_tasks.list
-#' 
+#'
 #' job_tasks.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$job_tasks.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return JobTaskList object.
 #' @name job_tasks.list
 NULL
 
 #' links.get
-#' 
+#'
 #' links.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.get(uuid)
 #' @param uuid The UUID of the Link in question.
 #' @return Link object.
@@ -758,9 +758,9 @@ NULL
 NULL
 
 #' links.create
-#' 
+#'
 #' links.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.create(link, ensure_unique_name = "false")
 #' @param link Link object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -769,9 +769,9 @@ NULL
 NULL
 
 #' links.update
-#' 
+#'
 #' links.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.update(link, uuid)
 #' @param link Link object.
 #' @param uuid The UUID of the Link in question.
@@ -780,9 +780,9 @@ NULL
 NULL
 
 #' links.delete
-#' 
+#'
 #' links.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.delete(uuid)
 #' @param uuid The UUID of the Link in question.
 #' @return Link object.
@@ -790,39 +790,39 @@ NULL
 NULL
 
 #' links.list
-#' 
+#'
 #' links.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return LinkList object.
 #' @name links.list
 NULL
 
 #' links.get_permissions
-#' 
+#'
 #' links.get_permissions is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$links.get_permissions(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Link object.
 #' @name links.get_permissions
 NULL
 
 #' jobs.get
-#' 
+#'
 #' jobs.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.get(uuid)
 #' @param uuid The UUID of the Job in question.
 #' @return Job object.
@@ -830,26 +830,26 @@ NULL
 NULL
 
 #' jobs.create
-#' 
+#'
 #' jobs.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.create(job, ensure_unique_name = "false",
 #'     find_or_create = "false", filters = NULL,
 #'     minimum_script_version = NULL, exclude_script_versions = NULL)
 #' @param job Job object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
-#' @param find_or_create 
-#' @param filters 
-#' @param minimum_script_version 
-#' @param exclude_script_versions 
+#' @param find_or_create
+#' @param filters
+#' @param minimum_script_version
+#' @param exclude_script_versions
 #' @return Job object.
 #' @name jobs.create
 NULL
 
 #' jobs.update
-#' 
+#'
 #' jobs.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.update(job, uuid)
 #' @param job Job object.
 #' @param uuid The UUID of the Job in question.
@@ -858,9 +858,9 @@ NULL
 NULL
 
 #' jobs.delete
-#' 
+#'
 #' jobs.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.delete(uuid)
 #' @param uuid The UUID of the Job in question.
 #' @return Job object.
@@ -868,77 +868,77 @@ NULL
 NULL
 
 #' jobs.queue
-#' 
+#'
 #' jobs.queue is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.queue(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return Job object.
 #' @name jobs.queue
 NULL
 
 #' jobs.queue_size
-#' 
+#'
 #' jobs.queue_size is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.queue_size(NULL)
 #' @return Job object.
 #' @name jobs.queue_size
 NULL
 
 #' jobs.cancel
-#' 
+#'
 #' jobs.cancel is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.cancel(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Job object.
 #' @name jobs.cancel
 NULL
 
 #' jobs.lock
-#' 
+#'
 #' jobs.lock is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.lock(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Job object.
 #' @name jobs.lock
 NULL
 
 #' jobs.list
-#' 
+#'
 #' jobs.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$jobs.list(filters = NULL, where = NULL,
 #'     order = NULL, select = NULL, distinct = NULL,
 #'     limit = "100", offset = "0", count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return JobList object.
 #' @name jobs.list
 NULL
 
 #' keep_disks.get
-#' 
+#'
 #' keep_disks.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.get(uuid)
 #' @param uuid The UUID of the KeepDisk in question.
 #' @return KeepDisk object.
@@ -946,9 +946,9 @@ NULL
 NULL
 
 #' keep_disks.create
-#' 
+#'
 #' keep_disks.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.create(keepdisk,
 #'     ensure_unique_name = "false")
 #' @param keepDisk KeepDisk object.
@@ -958,9 +958,9 @@ NULL
 NULL
 
 #' keep_disks.update
-#' 
+#'
 #' keep_disks.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.update(keepdisk,
 #'     uuid)
 #' @param keepDisk KeepDisk object.
@@ -970,9 +970,9 @@ NULL
 NULL
 
 #' keep_disks.delete
-#' 
+#'
 #' keep_disks.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.delete(uuid)
 #' @param uuid The UUID of the KeepDisk in question.
 #' @return KeepDisk object.
@@ -980,47 +980,47 @@ NULL
 NULL
 
 #' keep_disks.ping
-#' 
+#'
 #' keep_disks.ping is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.ping(uuid = NULL,
 #'     ping_secret, node_uuid = NULL, filesystem_uuid = NULL,
 #'     service_host = NULL, service_port, service_ssl_flag)
-#' @param uuid 
-#' @param ping_secret 
-#' @param node_uuid 
-#' @param filesystem_uuid 
-#' @param service_host 
-#' @param service_port 
-#' @param service_ssl_flag 
+#' @param uuid
+#' @param ping_secret
+#' @param node_uuid
+#' @param filesystem_uuid
+#' @param service_host
+#' @param service_port
+#' @param service_ssl_flag
 #' @return KeepDisk object.
 #' @name keep_disks.ping
 NULL
 
 #' keep_disks.list
-#' 
+#'
 #' keep_disks.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_disks.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return KeepDiskList object.
 #' @name keep_disks.list
 NULL
 
 #' keep_services.get
-#' 
+#'
 #' keep_services.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.get(uuid)
 #' @param uuid The UUID of the KeepService in question.
 #' @return KeepService object.
@@ -1028,9 +1028,9 @@ NULL
 NULL
 
 #' keep_services.create
-#' 
+#'
 #' keep_services.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.create(keepservice,
 #'     ensure_unique_name = "false")
 #' @param keepService KeepService object.
@@ -1040,9 +1040,9 @@ NULL
 NULL
 
 #' keep_services.update
-#' 
+#'
 #' keep_services.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.update(keepservice,
 #'     uuid)
 #' @param keepService KeepService object.
@@ -1052,9 +1052,9 @@ NULL
 NULL
 
 #' keep_services.delete
-#' 
+#'
 #' keep_services.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.delete(uuid)
 #' @param uuid The UUID of the KeepService in question.
 #' @return KeepService object.
@@ -1062,38 +1062,38 @@ NULL
 NULL
 
 #' keep_services.accessible
-#' 
+#'
 #' keep_services.accessible is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.accessible(NULL)
 #' @return KeepService object.
 #' @name keep_services.accessible
 NULL
 
 #' keep_services.list
-#' 
+#'
 #' keep_services.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$keep_services.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return KeepServiceList object.
 #' @name keep_services.list
 NULL
 
 #' pipeline_templates.get
-#' 
+#'
 #' pipeline_templates.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_templates.get(uuid)
 #' @param uuid The UUID of the PipelineTemplate in question.
 #' @return PipelineTemplate object.
@@ -1101,9 +1101,9 @@ NULL
 NULL
 
 #' pipeline_templates.create
-#' 
+#'
 #' pipeline_templates.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_templates.create(pipelinetemplate,
 #'     ensure_unique_name = "false")
 #' @param pipelineTemplate PipelineTemplate object.
@@ -1113,9 +1113,9 @@ NULL
 NULL
 
 #' pipeline_templates.update
-#' 
+#'
 #' pipeline_templates.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_templates.update(pipelinetemplate,
 #'     uuid)
 #' @param pipelineTemplate PipelineTemplate object.
@@ -1125,9 +1125,9 @@ NULL
 NULL
 
 #' pipeline_templates.delete
-#' 
+#'
 #' pipeline_templates.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_templates.delete(uuid)
 #' @param uuid The UUID of the PipelineTemplate in question.
 #' @return PipelineTemplate object.
@@ -1135,29 +1135,29 @@ NULL
 NULL
 
 #' pipeline_templates.list
-#' 
+#'
 #' pipeline_templates.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_templates.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return PipelineTemplateList object.
 #' @name pipeline_templates.list
 NULL
 
 #' pipeline_instances.get
-#' 
+#'
 #' pipeline_instances.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.get(uuid)
 #' @param uuid The UUID of the PipelineInstance in question.
 #' @return PipelineInstance object.
@@ -1165,9 +1165,9 @@ NULL
 NULL
 
 #' pipeline_instances.create
-#' 
+#'
 #' pipeline_instances.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.create(pipelineinstance,
 #'     ensure_unique_name = "false")
 #' @param pipelineInstance PipelineInstance object.
@@ -1177,9 +1177,9 @@ NULL
 NULL
 
 #' pipeline_instances.update
-#' 
+#'
 #' pipeline_instances.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.update(pipelineinstance,
 #'     uuid)
 #' @param pipelineInstance PipelineInstance object.
@@ -1189,9 +1189,9 @@ NULL
 NULL
 
 #' pipeline_instances.delete
-#' 
+#'
 #' pipeline_instances.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.delete(uuid)
 #' @param uuid The UUID of the PipelineInstance in question.
 #' @return PipelineInstance object.
@@ -1199,39 +1199,39 @@ NULL
 NULL
 
 #' pipeline_instances.cancel
-#' 
+#'
 #' pipeline_instances.cancel is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.cancel(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return PipelineInstance object.
 #' @name pipeline_instances.cancel
 NULL
 
 #' pipeline_instances.list
-#' 
+#'
 #' pipeline_instances.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$pipeline_instances.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return PipelineInstanceList object.
 #' @name pipeline_instances.list
 NULL
 
 #' nodes.get
-#' 
+#'
 #' nodes.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.get(uuid)
 #' @param uuid The UUID of the Node in question.
 #' @return Node object.
@@ -1239,9 +1239,9 @@ NULL
 NULL
 
 #' nodes.create
-#' 
+#'
 #' nodes.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.create(node, ensure_unique_name = "false",
 #'     assign_slot = NULL)
 #' @param node Node object.
@@ -1252,9 +1252,9 @@ NULL
 NULL
 
 #' nodes.update
-#' 
+#'
 #' nodes.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.update(node, uuid, assign_slot = NULL)
 #' @param node Node object.
 #' @param uuid The UUID of the Node in question.
@@ -1264,9 +1264,9 @@ NULL
 NULL
 
 #' nodes.delete
-#' 
+#'
 #' nodes.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.delete(uuid)
 #' @param uuid The UUID of the Node in question.
 #' @return Node object.
@@ -1274,40 +1274,40 @@ NULL
 NULL
 
 #' nodes.ping
-#' 
+#'
 #' nodes.ping is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.ping(uuid, ping_secret)
-#' @param uuid 
-#' @param ping_secret 
+#' @param uuid
+#' @param ping_secret
 #' @return Node object.
 #' @name nodes.ping
 NULL
 
 #' nodes.list
-#' 
+#'
 #' nodes.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$nodes.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return NodeList object.
 #' @name nodes.list
 NULL
 
 #' repositories.get
-#' 
+#'
 #' repositories.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.get(uuid)
 #' @param uuid The UUID of the Repository in question.
 #' @return Repository object.
@@ -1315,9 +1315,9 @@ NULL
 NULL
 
 #' repositories.create
-#' 
+#'
 #' repositories.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.create(repository,
 #'     ensure_unique_name = "false")
 #' @param repository Repository object.
@@ -1327,9 +1327,9 @@ NULL
 NULL
 
 #' repositories.update
-#' 
+#'
 #' repositories.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.update(repository,
 #'     uuid)
 #' @param repository Repository object.
@@ -1339,9 +1339,9 @@ NULL
 NULL
 
 #' repositories.delete
-#' 
+#'
 #' repositories.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.delete(uuid)
 #' @param uuid The UUID of the Repository in question.
 #' @return Repository object.
@@ -1349,38 +1349,38 @@ NULL
 NULL
 
 #' repositories.get_all_permissions
-#' 
+#'
 #' repositories.get_all_permissions is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.get_all_permissions(NULL)
 #' @return Repository object.
 #' @name repositories.get_all_permissions
 NULL
 
 #' repositories.list
-#' 
+#'
 #' repositories.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$repositories.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return RepositoryList object.
 #' @name repositories.list
 NULL
 
 #' specimens.get
-#' 
+#'
 #' specimens.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$specimens.get(uuid)
 #' @param uuid The UUID of the Specimen in question.
 #' @return Specimen object.
@@ -1388,9 +1388,9 @@ NULL
 NULL
 
 #' specimens.create
-#' 
+#'
 #' specimens.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$specimens.create(specimen,
 #'     ensure_unique_name = "false")
 #' @param specimen Specimen object.
@@ -1400,9 +1400,9 @@ NULL
 NULL
 
 #' specimens.update
-#' 
+#'
 #' specimens.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$specimens.update(specimen,
 #'     uuid)
 #' @param specimen Specimen object.
@@ -1412,9 +1412,9 @@ NULL
 NULL
 
 #' specimens.delete
-#' 
+#'
 #' specimens.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$specimens.delete(uuid)
 #' @param uuid The UUID of the Specimen in question.
 #' @return Specimen object.
@@ -1422,29 +1422,29 @@ NULL
 NULL
 
 #' specimens.list
-#' 
+#'
 #' specimens.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$specimens.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return SpecimenList object.
 #' @name specimens.list
 NULL
 
 #' logs.get
-#' 
+#'
 #' logs.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$logs.get(uuid)
 #' @param uuid The UUID of the Log in question.
 #' @return Log object.
@@ -1452,9 +1452,9 @@ NULL
 NULL
 
 #' logs.create
-#' 
+#'
 #' logs.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$logs.create(log, ensure_unique_name = "false")
 #' @param log Log object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -1463,9 +1463,9 @@ NULL
 NULL
 
 #' logs.update
-#' 
+#'
 #' logs.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$logs.update(log, uuid)
 #' @param log Log object.
 #' @param uuid The UUID of the Log in question.
@@ -1474,9 +1474,9 @@ NULL
 NULL
 
 #' logs.delete
-#' 
+#'
 #' logs.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$logs.delete(uuid)
 #' @param uuid The UUID of the Log in question.
 #' @return Log object.
@@ -1484,28 +1484,28 @@ NULL
 NULL
 
 #' logs.list
-#' 
+#'
 #' logs.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$logs.list(filters = NULL, where = NULL,
 #'     order = NULL, select = NULL, distinct = NULL,
 #'     limit = "100", offset = "0", count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return LogList object.
 #' @name logs.list
 NULL
 
 #' traits.get
-#' 
+#'
 #' traits.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$traits.get(uuid)
 #' @param uuid The UUID of the Trait in question.
 #' @return Trait object.
@@ -1513,9 +1513,9 @@ NULL
 NULL
 
 #' traits.create
-#' 
+#'
 #' traits.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$traits.create(trait, ensure_unique_name = "false")
 #' @param trait Trait object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -1524,9 +1524,9 @@ NULL
 NULL
 
 #' traits.update
-#' 
+#'
 #' traits.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$traits.update(trait, uuid)
 #' @param trait Trait object.
 #' @param uuid The UUID of the Trait in question.
@@ -1535,9 +1535,9 @@ NULL
 NULL
 
 #' traits.delete
-#' 
+#'
 #' traits.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$traits.delete(uuid)
 #' @param uuid The UUID of the Trait in question.
 #' @return Trait object.
@@ -1545,29 +1545,29 @@ NULL
 NULL
 
 #' traits.list
-#' 
+#'
 #' traits.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$traits.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return TraitList object.
 #' @name traits.list
 NULL
 
 #' virtual_machines.get
-#' 
+#'
 #' virtual_machines.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.get(uuid)
 #' @param uuid The UUID of the VirtualMachine in question.
 #' @return VirtualMachine object.
@@ -1575,9 +1575,9 @@ NULL
 NULL
 
 #' virtual_machines.create
-#' 
+#'
 #' virtual_machines.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.create(virtualmachine,
 #'     ensure_unique_name = "false")
 #' @param virtualMachine VirtualMachine object.
@@ -1587,9 +1587,9 @@ NULL
 NULL
 
 #' virtual_machines.update
-#' 
+#'
 #' virtual_machines.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.update(virtualmachine,
 #'     uuid)
 #' @param virtualMachine VirtualMachine object.
@@ -1599,9 +1599,9 @@ NULL
 NULL
 
 #' virtual_machines.delete
-#' 
+#'
 #' virtual_machines.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.delete(uuid)
 #' @param uuid The UUID of the VirtualMachine in question.
 #' @return VirtualMachine object.
@@ -1609,48 +1609,48 @@ NULL
 NULL
 
 #' virtual_machines.logins
-#' 
+#'
 #' virtual_machines.logins is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.logins(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return VirtualMachine object.
 #' @name virtual_machines.logins
 NULL
 
 #' virtual_machines.get_all_logins
-#' 
+#'
 #' virtual_machines.get_all_logins is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.get_all_logins(NULL)
 #' @return VirtualMachine object.
 #' @name virtual_machines.get_all_logins
 NULL
 
 #' virtual_machines.list
-#' 
+#'
 #' virtual_machines.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$virtual_machines.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return VirtualMachineList object.
 #' @name virtual_machines.list
 NULL
 
 #' workflows.get
-#' 
+#'
 #' workflows.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$workflows.get(uuid)
 #' @param uuid The UUID of the Workflow in question.
 #' @return Workflow object.
@@ -1658,9 +1658,9 @@ NULL
 NULL
 
 #' workflows.create
-#' 
+#'
 #' workflows.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$workflows.create(workflow,
 #'     ensure_unique_name = "false")
 #' @param workflow Workflow object.
@@ -1670,9 +1670,9 @@ NULL
 NULL
 
 #' workflows.update
-#' 
+#'
 #' workflows.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$workflows.update(workflow,
 #'     uuid)
 #' @param workflow Workflow object.
@@ -1682,9 +1682,9 @@ NULL
 NULL
 
 #' workflows.delete
-#' 
+#'
 #' workflows.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$workflows.delete(uuid)
 #' @param uuid The UUID of the Workflow in question.
 #' @return Workflow object.
@@ -1692,29 +1692,29 @@ NULL
 NULL
 
 #' workflows.list
-#' 
+#'
 #' workflows.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$workflows.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return WorkflowList object.
 #' @name workflows.list
 NULL
 
 #' groups.get
-#' 
+#'
 #' groups.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.get(uuid)
 #' @param uuid The UUID of the Group in question.
 #' @return Group object.
@@ -1722,9 +1722,9 @@ NULL
 NULL
 
 #' groups.create
-#' 
+#'
 #' groups.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.create(group, ensure_unique_name = "false")
 #' @param group Group object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -1733,9 +1733,9 @@ NULL
 NULL
 
 #' groups.update
-#' 
+#'
 #' groups.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.update(group, uuid)
 #' @param group Group object.
 #' @param uuid The UUID of the Group in question.
@@ -1744,9 +1744,9 @@ NULL
 NULL
 
 #' groups.delete
-#' 
+#'
 #' groups.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.delete(uuid)
 #' @param uuid The UUID of the Group in question.
 #' @return Group object.
@@ -1754,72 +1754,72 @@ NULL
 NULL
 
 #' groups.contents
-#' 
+#'
 #' groups.contents is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.contents(filters = NULL,
 #'     where = NULL, order = NULL, distinct = NULL,
 #'     limit = "100", offset = "0", count = "exact",
 #'     include_trash = NULL, uuid = NULL, recursive = NULL)
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @param include_trash Include items whose is_trashed attribute is true.
-#' @param uuid 
+#' @param uuid
 #' @param recursive Include contents from child groups recursively.
 #' @return Group object.
 #' @name groups.contents
 NULL
 
 #' groups.trash
-#' 
+#'
 #' groups.trash is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.trash(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Group object.
 #' @name groups.trash
 NULL
 
 #' groups.untrash
-#' 
+#'
 #' groups.untrash is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.untrash(uuid)
-#' @param uuid 
+#' @param uuid
 #' @return Group object.
 #' @name groups.untrash
 NULL
 
 #' groups.list
-#' 
+#'
 #' groups.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$groups.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact", include_trash = NULL)
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @param include_trash Include items whose is_trashed attribute is true.
 #' @return GroupList object.
 #' @name groups.list
 NULL
 
 #' user_agreements.get
-#' 
+#'
 #' user_agreements.get is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.get(uuid)
 #' @param uuid The UUID of the UserAgreement in question.
 #' @return UserAgreement object.
@@ -1827,9 +1827,9 @@ NULL
 NULL
 
 #' user_agreements.create
-#' 
+#'
 #' user_agreements.create is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.create(useragreement,
 #'     ensure_unique_name = "false")
 #' @param userAgreement UserAgreement object.
@@ -1839,9 +1839,9 @@ NULL
 NULL
 
 #' user_agreements.update
-#' 
+#'
 #' user_agreements.update is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.update(useragreement,
 #'     uuid)
 #' @param userAgreement UserAgreement object.
@@ -1851,9 +1851,9 @@ NULL
 NULL
 
 #' user_agreements.delete
-#' 
+#'
 #' user_agreements.delete is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.delete(uuid)
 #' @param uuid The UUID of the UserAgreement in question.
 #' @return UserAgreement object.
@@ -1861,56 +1861,56 @@ NULL
 NULL
 
 #' user_agreements.signatures
-#' 
+#'
 #' user_agreements.signatures is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.signatures(NULL)
 #' @return UserAgreement object.
 #' @name user_agreements.signatures
 NULL
 
 #' user_agreements.sign
-#' 
+#'
 #' user_agreements.sign is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.sign(NULL)
 #' @return UserAgreement object.
 #' @name user_agreements.sign
 NULL
 
 #' user_agreements.list
-#' 
+#'
 #' user_agreements.list is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.list(filters = NULL,
 #'     where = NULL, order = NULL, select = NULL,
 #'     distinct = NULL, limit = "100", offset = "0",
 #'     count = "exact")
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param select 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param select
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @return UserAgreementList object.
 #' @name user_agreements.list
 NULL
 
 #' user_agreements.new
-#' 
+#'
 #' user_agreements.new is a method defined in Arvados class.
-#' 
+#'
 #' @usage arv$user_agreements.new(NULL)
 #' @return UserAgreement object.
 #' @name user_agreements.new
 NULL
 
 #' project.get
-#' 
+#'
 #' projects.get is equivalent to groups.get method.
-#' 
+#'
 #' @usage arv$projects.get(uuid)
 #' @param uuid The UUID of the Group in question.
 #' @return Group object.
@@ -1918,9 +1918,9 @@ NULL
 NULL
 
 #' project.create
-#' 
+#'
 #' projects.create wrapps groups.create method by setting group_class attribute to "project".
-#' 
+#'
 #' @usage arv$projects.create(group, ensure_unique_name = "false")
 #' @param group Group object.
 #' @param ensure_unique_name Adjust name to ensure uniqueness instead of returning an error on (owner_uuid, name) collision.
@@ -1929,9 +1929,9 @@ NULL
 NULL
 
 #' project.update
-#' 
+#'
 #' projects.update wrapps groups.update method by setting group_class attribute to "project".
-#' 
+#'
 #' @usage arv$projects.update(group, uuid)
 #' @param group Group object.
 #' @param uuid The UUID of the Group in question.
@@ -1940,9 +1940,9 @@ NULL
 NULL
 
 #' project.delete
-#' 
+#'
 #' projects.delete is equivalent to groups.delete method.
-#' 
+#'
 #' @usage arv$project.delete(uuid)
 #' @param uuid The UUID of the Group in question.
 #' @return Group object.
@@ -1950,22 +1950,22 @@ NULL
 NULL
 
 #' project.list
-#' 
+#'
 #' projects.list wrapps groups.list method by setting group_class attribute to "project".
-#' 
+#'
 #' @usage arv$projects.list(filters = NULL,
 #'     where = NULL, order = NULL, distinct = NULL,
 #'     limit = "100", offset = "0", count = "exact",
 #'     include_trash = NULL, uuid = NULL, recursive = NULL)
-#' @param filters 
-#' @param where 
-#' @param order 
-#' @param distinct 
-#' @param limit 
-#' @param offset 
-#' @param count 
+#' @param filters
+#' @param where
+#' @param order
+#' @param distinct
+#' @param limit
+#' @param offset
+#' @param count
 #' @param include_trash Include items whose is_trashed attribute is true.
-#' @param uuid 
+#' @param uuid
 #' @param recursive Include contents from child groups recursively.
 #' @return Group object.
 #' @name projects.list
@@ -2237,19 +2237,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2257,23 +2257,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(user) > 0)
-                               body <- jsonlite::toJSON(list(user = user), 
+                               body <- jsonlite::toJSON(list(user = user),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2281,23 +2281,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(user) > 0)
-                               body <- jsonlite::toJSON(list(user = user), 
+                               body <- jsonlite::toJSON(list(user = user),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2305,19 +2305,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2325,19 +2325,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/current")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2345,19 +2345,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/system")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2365,19 +2365,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}/activate")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2386,21 +2386,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/setup")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(user = user, openid_prefix = openid_prefix,
                                                          repo_name = repo_name, vm_uuid = vm_uuid,
                                                          send_notification_email = send_notification_email)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2408,19 +2408,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}/unsetup")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2428,19 +2428,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users/${uuid}/update_uuid")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(new_uuid = new_uuid)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2450,21 +2450,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("users")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2472,19 +2472,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2493,23 +2493,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(apiclientauthorization) > 0)
-                               body <- jsonlite::toJSON(list(apiclientauthorization = apiclientauthorization), 
+                               body <- jsonlite::toJSON(list(apiclientauthorization = apiclientauthorization),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2517,23 +2517,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(apiclientauthorization) > 0)
-                               body <- jsonlite::toJSON(list(apiclientauthorization = apiclientauthorization), 
+                               body <- jsonlite::toJSON(list(apiclientauthorization = apiclientauthorization),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2541,19 +2541,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2561,20 +2561,20 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations/create_system_auth")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(api_client_id = api_client_id,
                                                          scopes = scopes)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2582,19 +2582,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations/current")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2605,21 +2605,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_client_authorizations")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2627,19 +2627,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2647,23 +2647,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(container) > 0)
-                               body <- jsonlite::toJSON(list(container = container), 
+                               body <- jsonlite::toJSON(list(container = container),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2671,23 +2671,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(container) > 0)
-                               body <- jsonlite::toJSON(list(container = container), 
+                               body <- jsonlite::toJSON(list(container = container),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2695,19 +2695,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2715,19 +2715,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}/auth")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2735,19 +2735,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}/lock")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2755,19 +2755,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}/unlock")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2775,19 +2775,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/${uuid}/secret_mounts")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2795,19 +2795,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers/current")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2818,21 +2818,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("containers")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2840,19 +2840,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_clients/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2860,23 +2860,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_clients")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(apiclient) > 0)
-                               body <- jsonlite::toJSON(list(apiclient = apiclient), 
+                               body <- jsonlite::toJSON(list(apiclient = apiclient),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2884,23 +2884,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_clients/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(apiclient) > 0)
-                               body <- jsonlite::toJSON(list(apiclient = apiclient), 
+                               body <- jsonlite::toJSON(list(apiclient = apiclient),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2908,19 +2908,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_clients/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2931,21 +2931,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("api_clients")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2953,19 +2953,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("authorized_keys/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2974,23 +2974,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("authorized_keys")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(authorizedkey) > 0)
-                               body <- jsonlite::toJSON(list(authorizedkey = authorizedkey), 
+                               body <- jsonlite::toJSON(list(authorizedkey = authorizedkey),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -2998,23 +2998,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("authorized_keys/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(authorizedkey) > 0)
-                               body <- jsonlite::toJSON(list(authorizedkey = authorizedkey), 
+                               body <- jsonlite::toJSON(list(authorizedkey = authorizedkey),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3022,19 +3022,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("authorized_keys/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3045,21 +3045,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("authorized_keys")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3067,19 +3067,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("container_requests/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3088,23 +3088,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("container_requests")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(containerrequest) > 0)
-                               body <- jsonlite::toJSON(list(containerrequest = containerrequest), 
+                               body <- jsonlite::toJSON(list(containerrequest = containerrequest),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3112,23 +3112,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("container_requests/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(containerrequest) > 0)
-                               body <- jsonlite::toJSON(list(containerrequest = containerrequest), 
+                               body <- jsonlite::toJSON(list(containerrequest = containerrequest),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3136,19 +3136,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("container_requests/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3159,21 +3159,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("container_requests")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3181,19 +3181,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3201,23 +3201,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(collection) > 0)
-                               body <- jsonlite::toJSON(list(collection = collection), 
+                               body <- jsonlite::toJSON(list(collection = collection),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3225,23 +3225,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(collection) > 0)
-                               body <- jsonlite::toJSON(list(collection = collection), 
+                               body <- jsonlite::toJSON(list(collection = collection),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3249,19 +3249,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3269,19 +3269,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}/provenance")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3289,19 +3289,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}/used_by")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3309,19 +3309,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}/trash")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3329,19 +3329,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections/${uuid}/untrash")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3352,22 +3352,22 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("collections")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count,
                                                          include_trash = include_trash)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3375,19 +3375,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("humans/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3395,23 +3395,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("humans")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(human) > 0)
-                               body <- jsonlite::toJSON(list(human = human), 
+                               body <- jsonlite::toJSON(list(human = human),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3419,23 +3419,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("humans/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(human) > 0)
-                               body <- jsonlite::toJSON(list(human = human), 
+                               body <- jsonlite::toJSON(list(human = human),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3443,19 +3443,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("humans/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3465,21 +3465,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("humans")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3487,19 +3487,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("job_tasks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3507,23 +3507,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("job_tasks")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(jobtask) > 0)
-                               body <- jsonlite::toJSON(list(jobtask = jobtask), 
+                               body <- jsonlite::toJSON(list(jobtask = jobtask),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3531,23 +3531,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("job_tasks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(jobtask) > 0)
-                               body <- jsonlite::toJSON(list(jobtask = jobtask), 
+                               body <- jsonlite::toJSON(list(jobtask = jobtask),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3555,19 +3555,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("job_tasks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3578,21 +3578,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("job_tasks")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3600,19 +3600,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("links/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3620,23 +3620,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("links")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(link) > 0)
-                               body <- jsonlite::toJSON(list(link = link), 
+                               body <- jsonlite::toJSON(list(link = link),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3644,23 +3644,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("links/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(link) > 0)
-                               body <- jsonlite::toJSON(list(link = link), 
+                               body <- jsonlite::toJSON(list(link = link),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3668,19 +3668,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("links/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3690,21 +3690,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("links")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3712,19 +3712,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("permissions/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3732,19 +3732,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3754,26 +3754,26 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name,
                                                          find_or_create = find_or_create, filters = filters,
                                                          minimum_script_version = minimum_script_version,
                                                          exclude_script_versions = exclude_script_versions)
-                       
+
                        if(length(job) > 0)
-                               body <- jsonlite::toJSON(list(job = job), 
+                               body <- jsonlite::toJSON(list(job = job),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3781,23 +3781,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(job) > 0)
-                               body <- jsonlite::toJSON(list(job = job), 
+                               body <- jsonlite::toJSON(list(job = job),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3805,19 +3805,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3827,21 +3827,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/queue")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3849,19 +3849,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/queue_size")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3869,19 +3869,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/${uuid}/cancel")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3889,19 +3889,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs/${uuid}/lock")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3911,21 +3911,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("jobs")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3933,19 +3933,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3953,23 +3953,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(keepdisk) > 0)
-                               body <- jsonlite::toJSON(list(keepdisk = keepdisk), 
+                               body <- jsonlite::toJSON(list(keepdisk = keepdisk),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -3977,23 +3977,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(keepdisk) > 0)
-                               body <- jsonlite::toJSON(list(keepdisk = keepdisk), 
+                               body <- jsonlite::toJSON(list(keepdisk = keepdisk),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4001,19 +4001,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4023,22 +4023,22 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks/ping")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(uuid = uuid, ping_secret = ping_secret,
                                                          node_uuid = node_uuid, filesystem_uuid = filesystem_uuid,
                                                          service_host = service_host, service_port = service_port,
                                                          service_ssl_flag = service_ssl_flag)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4049,21 +4049,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_disks")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4071,19 +4071,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4092,23 +4092,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(keepservice) > 0)
-                               body <- jsonlite::toJSON(list(keepservice = keepservice), 
+                               body <- jsonlite::toJSON(list(keepservice = keepservice),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4116,23 +4116,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(keepservice) > 0)
-                               body <- jsonlite::toJSON(list(keepservice = keepservice), 
+                               body <- jsonlite::toJSON(list(keepservice = keepservice),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4140,19 +4140,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4160,19 +4160,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services/accessible")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4183,21 +4183,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("keep_services")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4205,19 +4205,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_templates/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4226,23 +4226,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_templates")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(pipelinetemplate) > 0)
-                               body <- jsonlite::toJSON(list(pipelinetemplate = pipelinetemplate), 
+                               body <- jsonlite::toJSON(list(pipelinetemplate = pipelinetemplate),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4250,23 +4250,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_templates/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(pipelinetemplate) > 0)
-                               body <- jsonlite::toJSON(list(pipelinetemplate = pipelinetemplate), 
+                               body <- jsonlite::toJSON(list(pipelinetemplate = pipelinetemplate),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4274,19 +4274,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_templates/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4297,21 +4297,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_templates")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4319,19 +4319,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4340,23 +4340,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(pipelineinstance) > 0)
-                               body <- jsonlite::toJSON(list(pipelineinstance = pipelineinstance), 
+                               body <- jsonlite::toJSON(list(pipelineinstance = pipelineinstance),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4364,23 +4364,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(pipelineinstance) > 0)
-                               body <- jsonlite::toJSON(list(pipelineinstance = pipelineinstance), 
+                               body <- jsonlite::toJSON(list(pipelineinstance = pipelineinstance),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4388,19 +4388,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4408,19 +4408,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances/${uuid}/cancel")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4431,21 +4431,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("pipeline_instances")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4453,19 +4453,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4474,24 +4474,24 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name,
                                                          assign_slot = assign_slot)
-                       
+
                        if(length(node) > 0)
-                               body <- jsonlite::toJSON(list(node = node), 
+                               body <- jsonlite::toJSON(list(node = node),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4499,23 +4499,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(assign_slot = assign_slot)
-                       
+
                        if(length(node) > 0)
-                               body <- jsonlite::toJSON(list(node = node), 
+                               body <- jsonlite::toJSON(list(node = node),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4523,19 +4523,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4543,19 +4543,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes/${uuid}/ping")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ping_secret = ping_secret)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4565,21 +4565,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("nodes")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4587,19 +4587,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4607,23 +4607,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(repository) > 0)
-                               body <- jsonlite::toJSON(list(repository = repository), 
+                               body <- jsonlite::toJSON(list(repository = repository),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4631,23 +4631,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(repository) > 0)
-                               body <- jsonlite::toJSON(list(repository = repository), 
+                               body <- jsonlite::toJSON(list(repository = repository),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4655,19 +4655,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4675,19 +4675,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories/get_all_permissions")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4698,21 +4698,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("repositories")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4720,19 +4720,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("specimens/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4740,23 +4740,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("specimens")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(specimen) > 0)
-                               body <- jsonlite::toJSON(list(specimen = specimen), 
+                               body <- jsonlite::toJSON(list(specimen = specimen),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4764,23 +4764,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("specimens/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(specimen) > 0)
-                               body <- jsonlite::toJSON(list(specimen = specimen), 
+                               body <- jsonlite::toJSON(list(specimen = specimen),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4788,19 +4788,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("specimens/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4811,21 +4811,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("specimens")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4833,19 +4833,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("logs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4853,23 +4853,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("logs")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(log) > 0)
-                               body <- jsonlite::toJSON(list(log = log), 
+                               body <- jsonlite::toJSON(list(log = log),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4877,23 +4877,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("logs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(log) > 0)
-                               body <- jsonlite::toJSON(list(log = log), 
+                               body <- jsonlite::toJSON(list(log = log),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4901,19 +4901,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("logs/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4923,21 +4923,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("logs")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4945,19 +4945,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("traits/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4965,23 +4965,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("traits")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(trait) > 0)
-                               body <- jsonlite::toJSON(list(trait = trait), 
+                               body <- jsonlite::toJSON(list(trait = trait),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -4989,23 +4989,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("traits/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(trait) > 0)
-                               body <- jsonlite::toJSON(list(trait = trait), 
+                               body <- jsonlite::toJSON(list(trait = trait),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5013,19 +5013,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("traits/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5035,21 +5035,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("traits")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5057,19 +5057,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5078,23 +5078,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(virtualmachine) > 0)
-                               body <- jsonlite::toJSON(list(virtualmachine = virtualmachine), 
+                               body <- jsonlite::toJSON(list(virtualmachine = virtualmachine),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5102,23 +5102,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(virtualmachine) > 0)
-                               body <- jsonlite::toJSON(list(virtualmachine = virtualmachine), 
+                               body <- jsonlite::toJSON(list(virtualmachine = virtualmachine),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5126,19 +5126,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5146,19 +5146,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines/${uuid}/logins")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5166,19 +5166,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines/get_all_logins")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5189,21 +5189,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("virtual_machines")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5211,19 +5211,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("workflows/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5231,23 +5231,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("workflows")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(workflow) > 0)
-                               body <- jsonlite::toJSON(list(workflow = workflow), 
+                               body <- jsonlite::toJSON(list(workflow = workflow),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5255,23 +5255,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("workflows/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(workflow) > 0)
-                               body <- jsonlite::toJSON(list(workflow = workflow), 
+                               body <- jsonlite::toJSON(list(workflow = workflow),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5279,19 +5279,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("workflows/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5302,21 +5302,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("workflows")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5324,19 +5324,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5344,23 +5344,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(group) > 0)
-                               body <- jsonlite::toJSON(list(group = group), 
+                               body <- jsonlite::toJSON(list(group = group),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5368,23 +5368,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(group) > 0)
-                               body <- jsonlite::toJSON(list(group = group), 
+                               body <- jsonlite::toJSON(list(group = group),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5392,19 +5392,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5415,22 +5415,22 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/contents")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, distinct = distinct, limit = limit,
                                                          offset = offset, count = count, include_trash = include_trash,
                                                          uuid = uuid, recursive = recursive)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5438,19 +5438,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/${uuid}/trash")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5458,19 +5458,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups/${uuid}/untrash")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5481,22 +5481,22 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("groups")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count,
                                                          include_trash = include_trash)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5504,19 +5504,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5525,23 +5525,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(ensure_unique_name = ensure_unique_name)
-                       
+
                        if(length(useragreement) > 0)
-                               body <- jsonlite::toJSON(list(useragreement = useragreement), 
+                               body <- jsonlite::toJSON(list(useragreement = useragreement),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5549,23 +5549,23 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        if(length(useragreement) > 0)
-                               body <- jsonlite::toJSON(list(useragreement = useragreement), 
+                               body <- jsonlite::toJSON(list(useragreement = useragreement),
                                                         auto_unbox = TRUE)
                        else
                                body <- NULL
-                       
+
                        response <- private$REST$http$exec("PUT", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5573,19 +5573,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/${uuid}")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("DELETE", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5593,19 +5593,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/signatures")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5613,19 +5613,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/sign")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("POST", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5636,21 +5636,21 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- list(filters = filters, where = where,
                                                          order = order, select = select, distinct = distinct,
                                                          limit = limit, offset = offset, count = count)
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
@@ -5658,19 +5658,19 @@ Arvados <- R6::R6Class(
                {
                        endPoint <- stringr::str_interp("user_agreements/new")
                        url <- paste0(private$host, endPoint)
-                       headers <- list(Authorization = paste("OAuth2", private$token), 
+                       headers <- list(Authorization = paste("OAuth2", private$token),
                                        "Content-Type" = "application/json")
                        queryArgs <- NULL
-                       
+
                        body <- NULL
-                       
+
                        response <- private$REST$http$exec("GET", url, headers, body,
                                                           queryArgs, private$numRetries)
                        resource <- private$REST$httpParser$parseJSONResponse(response)
-                       
+
                        if(!is.null(resource$errors))
                                stop(resource$errors)
-                       
+
                        resource
                },
 
index 8f737831c4634cc09a3121a86e04dcbf0361946b..1071302892690a659370d41afaa4336316678362 100644 (file)
@@ -5,9 +5,9 @@
 source("./R/util.R")
 
 #' ArvadosFile
-#' 
+#'
 #' ArvadosFile class represents a file inside Arvados collection.
-#' 
+#'
 #' @section Usage:
 #' \preformatted{file = ArvadosFile$new(name)}
 #'
@@ -15,7 +15,7 @@ source("./R/util.R")
 #' \describe{
 #'   \item{name}{Name of the file.}
 #' }
-#' 
+#'
 #' @section Methods:
 #' \describe{
 #'   \item{getName()}{Returns name of the file.}
@@ -37,7 +37,7 @@ source("./R/util.R")
 #' myFile$write("This is new file content")
 #' fileContent <- myFile$read()
 #' fileContent <- myFile$read("text")
-#' fileContent <- myFile$read("raw", offset = 8, length = 4) 
+#' fileContent <- myFile$read("raw", offset = 8, length = 4)
 #'
 #' #Write a table:
 #' arvConnection <- myFile$connection("w")
@@ -141,14 +141,14 @@ ArvadosFile <- R6::R6Class(
 
         connection = function(rw)
         {
-            if (rw == "r" || rw == "rb") 
+            if (rw == "r" || rw == "rb")
             {
                 REST <- private$collection$getRESTService()
-                return(REST$getConnection(private$collection$uuid,
-                                          self$getRelativePath(),
+                return(REST$getConnection(self$getRelativePath(),
+                                          private$collection$uuid,
                                           rw))
             }
-            else if (rw == "w") 
+            else if (rw == "w")
             {
                 private$buffer <- textConnection(NULL, "w")
 
@@ -156,7 +156,7 @@ ArvadosFile <- R6::R6Class(
             }
         },
 
-        flush = function() 
+        flush = function()
         {
             v <- textConnectionValue(private$buffer)
             close(private$buffer)
@@ -255,7 +255,7 @@ ArvadosFile <- R6::R6Class(
 #'
 #' @param x Instance of ArvadosFile class
 #' @param ... Optional arguments.
-#' @export 
+#' @export
 print.ArvadosFile = function(x, ...)
 {
     collection   <- NULL
index e23da138329786cba49e3a8001479461dd30be77..833b833c80f367ef59d4ca82ef78784e278cf73d 100644 (file)
@@ -8,9 +8,9 @@ source("./R/RESTService.R")
 source("./R/util.R")
 
 #' Collection
-#' 
+#'
 #' Collection class provides interface for working with Arvados collections.
-#' 
+#'
 #' @section Usage:
 #' \preformatted{collection = Collection$new(arv, uuid)}
 #'
@@ -19,7 +19,7 @@ source("./R/util.R")
 #'   \item{arv}{Arvados object.}
 #'   \item{uuid}{UUID of a collection.}
 #' }
-#' 
+#'
 #' @section Methods:
 #' \describe{
 #'   \item{add(content)}{Adds ArvadosFile or Subcollection specified by content to the collection.}
@@ -60,7 +60,7 @@ Collection <- R6::R6Class(
                uuid = NULL,
         # api  = NULL,
 
-               initialize = function(api, uuid) 
+               initialize = function(api, uuid)
         {
             # self$api <- api
             private$REST <- api$getRESTService()
@@ -121,7 +121,7 @@ Collection <- R6::R6Class(
             }
             else
             {
-                relativePath  <- trimFromEnd(relativePath, "/") 
+                relativePath  <- trimFromEnd(relativePath, "/")
                 subcollection <- self$get(relativePath)
             }
 
@@ -148,7 +148,7 @@ Collection <- R6::R6Class(
                 else
                     return(arvadosFiles)
             }
-            else 
+            else
             {
                 stop(paste0("Expected character vector, got ",
                             paste0("(", paste0(class(fileNames), collapse = ", "), ")"),
@@ -181,7 +181,7 @@ Collection <- R6::R6Class(
 
                 "Content removed"
             }
-            else 
+            else
             {
                 stop(paste0("Expected character vector, got ",
                             paste0("(", paste0(class(paths), collapse = ", "), ")"),
@@ -221,14 +221,14 @@ Collection <- R6::R6Class(
             private$tree$getElement(relativePath)
         },
 
-               toJSON = function() 
+               toJSON = function()
         {
                        fields <- sapply(private$classFields, function(field)
                        {
                                self[[field]]
                        }, USE.NAMES = TRUE)
-                       
-                       jsonlite::toJSON(list("collection" = 
+
+                       jsonlite::toJSON(list("collection" =
                      Filter(Negate(is.null), fields)), auto_unbox = TRUE)
                },
 
@@ -275,7 +275,7 @@ Collection <- R6::R6Class(
 #'
 #' @param x Instance of Collection class
 #' @param ... Optional arguments.
-#' @export 
+#' @export
 print.Collection = function(x, ...)
 {
     cat(paste0("Type: ", "\"", "Arvados Collection", "\""), sep = "\n")
index 8686f88c1a8a3c55b695351b9993df55939d0f1a..8b2f8876b0f91a0d86e148f81f1d29d87ab2f479 100644 (file)
@@ -19,7 +19,7 @@ CollectionTree <- R6::R6Class(
             treeBranches <- sapply(fileContent, function(filePath)
             {
                 splitPath <- unlist(strsplit(filePath, "/", fixed = TRUE))
-                branch <- private$createBranch(splitPath)      
+                branch <- private$createBranch(splitPath)
             })
 
             root <- Subcollection$new("")
@@ -80,7 +80,7 @@ CollectionTree <- R6::R6Class(
                     branch <- newFolder
                 }
             }
-            
+
             branch
         },
 
@@ -94,10 +94,10 @@ CollectionTree <- R6::R6Class(
             }
             else
             {
-                # Note: REST always returns folder name alone before other folder 
+                # Note: REST always returns folder name alone before other folder
                 # content, so in first iteration we don't know if it's a file
-                # or folder since its just a name, so we assume it's a file. 
-                # If we encounter that same name again we know 
+                # or folder since its just a name, so we assume it's a file.
+                # If we encounter that same name again we know
                 # it's a folder so we need to replace ArvadosFile with Subcollection.
                 if("ArvadosFile" %in% class(child))
                 {
index 8ce68f3837f158486534c6adc55e4ff23e9386e1..cd492166a139bf56dccebf732f2533c443440cf7 100644 (file)
@@ -10,12 +10,12 @@ HttpParser <- R6::R6Class(
 
         validContentTypes = NULL,
 
-        initialize = function() 
+        initialize = function()
         {
             self$validContentTypes <- c("text", "raw")
         },
 
-        parseJSONResponse = function(serverResponse) 
+        parseJSONResponse = function(serverResponse)
         {
             parsed_response <- httr::content(serverResponse,
                                              as = "parsed",
@@ -41,7 +41,7 @@ HttpParser <- R6::R6Class(
             result[-1]
         },
 
-        getFileSizesFromResponse = function(response, uri)    
+        getFileSizesFromResponse = function(response, uri)
         {
             text <- rawToChar(response$content)
             doc <- XML::xmlParse(text, asText=TRUE)
index 95dd375debe5ce076638c55de49a57db1f2d8f0d..4595fef662c1c9422af13dc0222a10bd9a9df48a 100644 (file)
@@ -13,7 +13,7 @@ HttpRequest <- R6::R6Class(
         validContentTypes = NULL,
         validVerbs = NULL,
 
-        initialize = function() 
+        initialize = function()
         {
             self$validContentTypes <- c("text", "raw")
             self$validVerbs <- c("GET", "POST", "PUT", "DELETE", "PROPFIND", "MOVE")
@@ -30,7 +30,7 @@ HttpRequest <- R6::R6Class(
 
             config <- httr::add_headers(unlist(headers))
             if(toString(Sys.getenv("ARVADOS_API_HOST_INSECURE") == "TRUE"))
-               config$options = list(ssl_verifypeer = FALSE)
+               config$options = list(ssl_verifypeer = 0L)
 
             # times = 1 regular call + numberOfRetries
             response <- httr::RETRY(verb, url = url, body = body,
@@ -58,6 +58,17 @@ HttpRequest <- R6::R6Class(
             }
 
             return("")
+        },
+
+        getConnection = function(url, headers, openMode)
+        {
+            h <- curl::new_handle()
+            curl::handle_setheaders(h, .list = headers)
+
+            if(toString(Sys.getenv("ARVADOS_API_HOST_INSECURE") == "TRUE"))
+               curl::handle_setopt(h, ssl_verifypeer = 0L)
+
+            conn <- curl::curl(url = url, open = openMode, handle = h)
         }
     ),
 
index ac65d0df3f37b6baa6031bc8cbab71b163e27a76..867665299f49899909167c4fd838bc9ae1371dd8 100644 (file)
@@ -66,7 +66,7 @@ RESTService <- R6::R6Class(
         {
             fileURL <- paste0(self$getWebDavHostName(), "c=",
                               uuid, "/", relativePath);
-            headers <- list(Authorization = paste("OAuth2", self$token)) 
+            headers <- list(Authorization = paste("OAuth2", self$token))
 
             serverResponse <- self$http$exec("DELETE", fileURL, headers,
                                              retryTimes = self$numRetries)
@@ -186,18 +186,13 @@ RESTService <- R6::R6Class(
             self$httpParser$parseResponse(serverResponse, "text")
         },
 
-        getConnection = function(uuid, relativePath, openMode)
+        getConnection = function(relativePath, uuid, openMode)
         {
-            fileURL <- paste0(self$getWebDavHostName(), 
+            fileURL <- paste0(self$getWebDavHostName(),
                               "c=", uuid, "/", relativePath);
             headers <- list(Authorization = paste("OAuth2", self$token))
 
-            h <- curl::new_handle()
-            curl::handle_setheaders(h, .list = headers)
-
-            conn <- curl::curl(url = fileURL, open = openMode, handle = h)
-
-            conn
+            conn <- self$http$getConnection(fileURL, headers, openMode)
         }
     ),
 
@@ -210,7 +205,7 @@ RESTService <- R6::R6Class(
         {
             fileURL <- paste0(self$getWebDavHostName(), "c=",
                               uuid, "/", relativePath)
-            headers <- list(Authorization = paste("OAuth2", self$token), 
+            headers <- list(Authorization = paste("OAuth2", self$token),
                             "Content-Type" = contentType)
             body <- NULL
 
index 60714a4ad835b9bc201fb780bb38b5fb8a81461c..45fe34347903192bed1d8618e14aceedf73a18f2 100644 (file)
@@ -5,10 +5,10 @@
 source("./R/util.R")
 
 #' Subcollection
-#' 
+#'
 #' Subcollection class represents a folder inside Arvados collection.
 #' It is essentially a composite of arvadosFiles and other subcollections.
-#' 
+#'
 #' @section Usage:
 #' \preformatted{subcollection = Subcollection$new(name)}
 #'
@@ -16,7 +16,7 @@ source("./R/util.R")
 #' \describe{
 #'   \item{name}{Name of the subcollection.}
 #' }
-#' 
+#'
 #' @section Methods:
 #' \describe{
 #'   \item{getName()}{Returns name of the subcollection.}
@@ -56,7 +56,7 @@ Subcollection <- R6::R6Class(
         },
 
         getName = function() private$name,
-        
+
         getRelativePath = function()
         {
             relativePath <- c(private$name)
@@ -87,7 +87,7 @@ Subcollection <- R6::R6Class(
                                "or Subcollection with same name."))
 
                 if(!is.null(private$collection))
-                {       
+                {
                     if(self$getRelativePath() != "")
                         contentPath <- paste0(self$getRelativePath(),
                                               "/", content$getFileListing())
@@ -302,7 +302,7 @@ Subcollection <- R6::R6Class(
             content
         }
     ),
-    
+
     cloneable = FALSE
 )
 
@@ -312,7 +312,7 @@ Subcollection <- R6::R6Class(
 #'
 #' @param x Instance of Subcollection class
 #' @param ... Optional arguments.
-#' @export 
+#' @export
 print.Subcollection = function(x, ...)
 {
     collection   <- NULL
index 3e8c2fa0cf2b1494c33a7246a9f97a8669a3b514..1aef20b6cb90fe11d7440219bbe24d464af988c2 100644 (file)
@@ -343,7 +343,7 @@ genMethodsDoc <- function(methodResources, resourceNames)
     }, methodResources, resourceNames)))
 
     projectDoc <- genProjectMethodsDoc()
-    
+
     c(methodsDoc, projectDoc)
 }
 
@@ -401,10 +401,10 @@ getAPIClassMethodList <- function(methodResources, resourceNames)
                methodNames[!(methodNames %in% c("index", "show", "destroy"))])
 
     }, methodResources, resourceNames)))
-    
+
     hardcodedMethods <- c("projects.create", "projects.get",
                           "projects.list", "projects.update", "projects.delete")
-    paste0("#' \t\\item{}{\\code{\\link{", sort(c(methodList, hardcodedMethods)), "}}}") 
+    paste0("#' \t\\item{}{\\code{\\link{", sort(c(methodList, hardcodedMethods)), "}}}")
 }
 
 getMethodDoc <- function(methodName, methodMetaData)
@@ -447,7 +447,7 @@ getMethodDescription <- function(methodMetaData)
                                  className <- sapply(prop, function(ref) ref)
                                  objectName <- paste0(tolower(substr(className, 1, 1)),
                                                       substr(className, 2, nchar(className)))
-                                 paste("#' @param", objectName, className, "object.") 
+                                 paste("#' @param", objectName, className, "object.")
                              })))
     }
 
@@ -457,7 +457,7 @@ getMethodDescription <- function(methodMetaData)
     {
         arg <- methodMetaData$parameters[[argName]]
         argDescription <- arg$description
-        paste("#' @param", argName, argDescription) 
+        paste("#' @param", argName, argDescription)
     })))
 
     c(requestDoc, argsDoc)
@@ -541,7 +541,7 @@ formatArgs <- function(prependAtStart, prependToEachSplit,
 {
     if(length(args) > 1)
     {
-        args[1:(length(args) - 1)] <- paste0(args[1:(length(args) - 1)], ",") 
+        args[1:(length(args) - 1)] <- paste0(args[1:(length(args) - 1)], ",")
     }
 
     args[1] <- paste0(prependAtStart, args[1])
@@ -564,12 +564,12 @@ formatArgs <- function(prependAtStart, prependToEachSplit,
 
         argLines <- c(argLines, line)
     }
-    
+
     argLines <- unlist(argLines)
     argLinesLen <- length(argLines)
 
     if(argLinesLen > 1)
-        argLines[2:argLinesLen] <- paste0(prependToEachSplit, argLines[2:argLinesLen]) 
+        argLines[2:argLinesLen] <- paste0(prependToEachSplit, argLines[2:argLinesLen])
 
     argLines
 }
index c97572c193f1eadbd315928fb09d56aff5e2d7a2..c23283989a9c982146168a4ec883a670bd2e7510 100644 (file)
@@ -11,13 +11,13 @@ FakeHttpParser <- R6::R6Class(
         validContentTypes = NULL,
         parserCallCount = NULL,
 
-        initialize = function() 
+        initialize = function()
         {
             self$parserCallCount <- 0
             self$validContentTypes <- c("text", "raw")
         },
 
-        parseJSONResponse = function(serverResponse) 
+        parseJSONResponse = function(serverResponse)
         {
             self$parserCallCount <- self$parserCallCount + 1
 
@@ -47,7 +47,7 @@ FakeHttpParser <- R6::R6Class(
             serverResponse
         },
 
-        getFileSizesFromResponse = function(serverResponse, uri)    
+        getFileSizesFromResponse = function(serverResponse, uri)
         {
             self$parserCallCount <- self$parserCallCount + 1
 
index 2633abdf2c745bf0e4c9afcee1b73b7c5751fbeb..2ebcda2e4d671dc91cef9ca6ca0cdcb6a2707e18 100644 (file)
@@ -22,11 +22,12 @@ FakeHttpRequest <- R6::R6Class(
         JSONEncodedBodyIsProvided               = NULL,
         requestBodyIsProvided                   = NULL,
 
-        numberOfGETRequests    = NULL,
-        numberOfDELETERequests = NULL,
-        numberOfPUTRequests    = NULL,
-        numberOfPOSTRequests   = NULL,
-        numberOfMOVERequests   = NULL,
+        numberOfGETRequests        = NULL,
+        numberOfDELETERequests     = NULL,
+        numberOfPUTRequests        = NULL,
+        numberOfPOSTRequests       = NULL,
+        numberOfMOVERequests       = NULL,
+        numberOfgetConnectionCalls = NULL,
 
         initialize = function(expectedURL      = NULL,
                               serverResponse   = NULL,
@@ -57,6 +58,8 @@ FakeHttpRequest <- R6::R6Class(
             self$numberOfPOSTRequests   <- 0
             self$numberOfMOVERequests   <- 0
 
+            self$numberOfgetConnectionCalls <- 0
+
             self$serverMaxElementsPerRequest <- 5
         },
 
@@ -87,18 +90,24 @@ FakeHttpRequest <- R6::R6Class(
                 return(private$getElements(offset, limit))
             else
                 return(self$content)
+        },
+
+        getConnection = function(url, headers, openMode)
+        {
+            self$numberOfgetConnectionCalls <- self$numberOfgetConnectionCalls + 1
+            c(url, headers, openMode)
         }
     ),
 
     private = list(
 
-        validateURL = function(url) 
+        validateURL = function(url)
         {
             if(!is.null(self$expectedURL) && url == self$expectedURL)
                 self$URLIsProperlyConfigured <- TRUE
         },
 
-        validateHeaders = function(headers) 
+        validateHeaders = function(headers)
         {
             if(!is.null(headers$Authorization))
                 self$requestHeaderContainsAuthorizationField <- TRUE
@@ -115,11 +124,11 @@ FakeHttpRequest <- R6::R6Class(
 
         validateBody = function(body)
         {
-            if(!is.null(body))           
+            if(!is.null(body))
             {
                 self$requestBodyIsProvided <- TRUE
 
-                if(class(body) == "json")           
+                if(class(body) == "json")
                     self$JSONEncodedBodyIsProvided <- TRUE
             }
         },
@@ -143,7 +152,7 @@ FakeHttpRequest <- R6::R6Class(
             {
                 if(offset > self$content$items_available)
                     stop("Invalid offset")
-                
+
                 start <- offset + 1
             }
 
index 08e8717de5e4b97b5776c2c6cc8893c523f4c133..048013f566a3c737188dc04f876bee6d9ae8dd90 100644 (file)
@@ -153,14 +153,14 @@ FakeRESTService <- R6::R6Class(
             self$returnContent
         },
 
-        write = function(uuid, relativePath, content, contentType)
+        write = function(relativePath, uuid, content, contentType)
         {
             self$writeBuffer <- content
             self$writeCallCount <- self$writeCallCount + 1
             self$returnContent
         },
 
-        getConnection = function(relativePath, uuid, openMode)
+        getConnection = function(uuid, relativePath, openMode)
         {
             self$getConnectionCallCount <- self$getConnectionCallCount + 1
             self$returnContent
index fb14888aab91b982d88dbdddca0be9589f757fb8..1f559e999c52be207317b1f9f6f17332ab287ac0 100644 (file)
@@ -9,30 +9,30 @@ context("ArvadosFile")
 test_that("constructor raises error if  file name is empty string", {
 
     expect_that(ArvadosFile$new(""), throws_error("Invalid name."))
-}) 
+})
 
 test_that("getFileListing always returns file name", {
 
     dog <- ArvadosFile$new("dog")
 
     expect_that(dog$getFileListing(), equals("dog"))
-}) 
+})
 
 test_that("get always returns NULL", {
 
     dog <- ArvadosFile$new("dog")
-    
+
     responseIsNull <- is.null(dog$get("something"))
     expect_that(responseIsNull, is_true())
-}) 
+})
 
 test_that("getFirst always returns NULL", {
 
     dog <- ArvadosFile$new("dog")
-    
+
     responseIsNull <- is.null(dog$getFirst())
     expect_that(responseIsNull, is_true())
-}) 
+})
 
 test_that(paste("getSizeInBytes returns zero if arvadosFile",
                 "is not part of a collection"), {
@@ -40,7 +40,7 @@ test_that(paste("getSizeInBytes returns zero if arvadosFile",
     dog <- ArvadosFile$new("dog")
 
     expect_that(dog$getSizeInBytes(), equals(0))
-}) 
+})
 
 test_that(paste("getSizeInBytes delegates size calculation",
                 "to REST service class"), {
@@ -57,7 +57,7 @@ test_that(paste("getSizeInBytes delegates size calculation",
     resourceSize <- fish$getSizeInBytes()
 
     expect_that(resourceSize, equals(100))
-}) 
+})
 
 test_that("getRelativePath returns path relative to the tree root", {
 
@@ -69,7 +69,7 @@ test_that("getRelativePath returns path relative to the tree root", {
     fish$add(shark)
 
     expect_that(shark$getRelativePath(), equals("animal/fish/shark"))
-}) 
+})
 
 test_that("read raises exception if file doesn't belong to a collection", {
 
@@ -77,7 +77,7 @@ test_that("read raises exception if file doesn't belong to a collection", {
 
     expect_that(dog$read(),
                 throws_error("ArvadosFile doesn't belong to any collection."))
-}) 
+})
 
 test_that("read raises exception offset or length is negative number", {
 
@@ -96,7 +96,7 @@ test_that("read raises exception offset or length is negative number", {
                 throws_error("Offset and length must be positive values."))
     expect_that(fish$read(contentType = "text", offset = -1, length = -1),
                 throws_error("Offset and length must be positive values."))
-}) 
+})
 
 test_that("read delegates reading operation to REST service class", {
 
@@ -108,15 +108,15 @@ test_that("read delegates reading operation to REST service class", {
     api$setRESTService(fakeREST)
     collection <- Collection$new(api, "myUUID")
     fish <- collection$get("animal/fish")
-    
+
     fileContent <- fish$read("text")
 
     expect_that(fileContent, equals("my file"))
     expect_that(fakeREST$readCallCount, equals(1))
-}) 
+})
 
 test_that(paste("connection delegates connection creation ro RESTService class",
-                "which returns curl connection opened in read mode when", 
+                "which returns curl connection opened in read mode when",
                 "'r' of 'rb' is passed as argument"), {
 
     collectionContent <- c("animal", "animal/fish")
@@ -130,7 +130,7 @@ test_that(paste("connection delegates connection creation ro RESTService class",
     connection <- fish$connection("r")
 
     expect_that(fakeREST$getConnectionCallCount, equals(1))
-}) 
+})
 
 test_that(paste("connection returns textConnection opened",
                 "in write mode when 'w' is passed as argument"), {
@@ -152,7 +152,7 @@ test_that(paste("connection returns textConnection opened",
 
     expect_that(writeResult[1], equals("file"))
     expect_that(writeResult[2], equals("content"))
-}) 
+})
 
 test_that("flush sends data stored in a connection to a REST server", {
 
@@ -172,7 +172,7 @@ test_that("flush sends data stored in a connection to a REST server", {
     fish$flush()
 
     expect_that(fakeREST$writeBuffer, equals("file content"))
-}) 
+})
 
 test_that("write raises exception if file doesn't belong to a collection", {
 
@@ -180,7 +180,7 @@ test_that("write raises exception if file doesn't belong to a collection", {
 
     expect_that(dog$write(),
                 throws_error("ArvadosFile doesn't belong to any collection."))
-}) 
+})
 
 test_that("write delegates writing operation to REST service class", {
 
@@ -192,11 +192,11 @@ test_that("write delegates writing operation to REST service class", {
     api$setRESTService(fakeREST)
     collection <- Collection$new(api, "myUUID")
     fish <- collection$get("animal/fish")
-    
+
     fileContent <- fish$write("new file content")
 
     expect_that(fakeREST$writeBuffer, equals("new file content"))
-}) 
+})
 
 test_that(paste("move raises exception if arvados file",
                 "doesn't belong to any collection"), {
@@ -205,7 +205,7 @@ test_that(paste("move raises exception if arvados file",
 
     expect_that(animal$move("new/location"),
                 throws_error("ArvadosFile doesn't belong to any collection"))
-}) 
+})
 
 test_that(paste("move raises exception if newLocationInCollection",
                 "parameter is invalid"), {
@@ -227,7 +227,7 @@ test_that(paste("move raises exception if newLocationInCollection",
 
     expect_that(dog$move("objects/dog"),
                 throws_error("Unable to get destination subcollection"))
-}) 
+})
 
 test_that("move raises exception if new location contains content with the same name", {
 
@@ -248,7 +248,7 @@ test_that("move raises exception if new location contains content with the same
     expect_that(dog$move("dog"),
                 throws_error("Destination already contains content with same name."))
 
-}) 
+})
 
 test_that("move moves arvados file inside collection tree", {
 
index c3c70910e4c63acea6d86f5df71cc9bab9f3e72f..af5fabeb73c82c99cfee67e935a1ebfa8e61fd2c 100644 (file)
@@ -100,7 +100,7 @@ test_that("create raises exception if passed argumet is not character vector", {
     collection <- Collection$new(api, "myUUID")
 
     expect_that(collection$create(10),
-                throws_error("Expected character vector, got (numeric).", 
+                throws_error("Expected character vector, got (numeric).",
                              fixed = TRUE))
 })
 
@@ -156,7 +156,7 @@ test_that("remove raises exception if passed argumet is not character vector", {
     collection <- Collection$new(api, "myUUID")
 
     expect_that(collection$remove(10),
-                throws_error("Expected character vector, got (numeric).", 
+                throws_error("Expected character vector, got (numeric).",
                              fixed = TRUE))
 })
 
@@ -238,7 +238,7 @@ test_that("getFileListing returns sorted collection content received from REST s
     api$setRESTService(fakeREST)
     collection <- Collection$new(api, "myUUID")
 
-    contentMatchExpected <- all(collection$getFileListing() == 
+    contentMatchExpected <- all(collection$getFileListing() ==
                                 c("animal", "animal/fish", "ball"))
 
     expect_that(contentMatchExpected, is_true())
index 5c8a40526988bb562c45b5702fd921a743f0a77c..1a3aefecd012325658ad408ee2a699682907dbaf 100644 (file)
@@ -7,7 +7,7 @@ context("CollectionTree")
 test_that("constructor creates file tree from character array properly", {
 
     collection <- "myCollection"
-    characterArray <- c("animal", 
+    characterArray <- c("animal",
                         "animal/dog",
                         "boat")
 
@@ -44,12 +44,12 @@ test_that("constructor creates file tree from character array properly", {
     expect_that(boatIsOfTypeArvadosFile, is_true())
     expect_that(boatsParentIsRoot, is_true())
     expect_that(allElementsBelongToSameCollection, is_true())
-}) 
+})
 
 test_that("getElement returns element from tree if element exists on specified path", {
 
     collection <- "myCollection"
-    characterArray <- c("animal", 
+    characterArray <- c("animal",
                         "animal/dog",
                         "boat")
 
@@ -58,12 +58,12 @@ test_that("getElement returns element from tree if element exists on specified p
     dog <- collectionTree$getElement("animal/dog")
 
     expect_that(dog$getName(), equals("dog"))
-}) 
+})
 
 test_that("getElement returns NULL from tree if element doesn't exists on specified path", {
 
     collection <- "myCollection"
-    characterArray <- c("animal", 
+    characterArray <- c("animal",
                         "animal/dog",
                         "boat")
 
@@ -73,12 +73,12 @@ test_that("getElement returns NULL from tree if element doesn't exists on specif
     fishIsNULL <- is.null(fish)
 
     expect_that(fishIsNULL, is_true())
-}) 
+})
 
 test_that("getElement trims ./ from start of relativePath", {
 
     collection <- "myCollection"
-    characterArray <- c("animal", 
+    characterArray <- c("animal",
                         "animal/dog",
                         "boat")
 
@@ -88,12 +88,12 @@ test_that("getElement trims ./ from start of relativePath", {
     dogWithDotSlash <- collectionTree$getElement("./animal/dog")
 
     expect_that(dogWithDotSlash$getName(), equals(dog$getName()))
-}) 
+})
 
 test_that("getElement trims / from end of relativePath", {
 
     collection <- "myCollection"
-    characterArray <- c("animal", 
+    characterArray <- c("animal",
                         "animal/dog",
                         "boat")
 
@@ -103,4 +103,4 @@ test_that("getElement trims / from end of relativePath", {
     animalWithSlash <- collectionTree$getElement("animal/")
 
     expect_that(animalWithSlash$getName(), equals(animal$getName()))
-}) 
+})
index a119d88bf82fa226e26d5127f3ae001d1b515a2e..82c0fb0dd2fed88598e8fd14a8dd88a11d065b71 100644 (file)
@@ -20,7 +20,7 @@ test_that("parseJSONResponse generates and returns JSON object from server respo
 
     expect_that(barExists, is_true())
     expect_that(unlist(result$bar$foo), equals(10))
-}) 
+})
 
 test_that(paste("parseResponse generates and returns character vector",
                 "from server response if outputType is text"), {
@@ -35,10 +35,10 @@ test_that(paste("parseResponse generates and returns character vector",
     parsedResponse <- parser$parseResponse(serverResponse, "text")
 
     expect_that(parsedResponse, equals("random text"))
-}) 
+})
 
 
-webDAVResponseSample = 
+webDAVResponseSample =
     paste0("<?xml version=\"1.0\" encoding=\"UTF-8\"?><D:multistatus xmlns:",
            "D=\"DAV:\"><D:response><D:href>/c=aaaaa-bbbbb-ccccccccccccccc</D",
            ":href><D:propstat><D:prop><D:resourcetype><D:collection xmlns:D=",
@@ -76,7 +76,7 @@ test_that(paste("getFileNamesFromResponse returns file names belonging to specif
     resultMatchExpected <- all.equal(result, expectedResult)
 
     expect_that(resultMatchExpected, is_true())
-}) 
+})
 
 test_that(paste("getFileSizesFromResponse returns file sizes",
                 "parsed from webDAV server response"), {
@@ -93,4 +93,4 @@ test_that(paste("getFileSizesFromResponse returns file sizes",
     resultMatchExpected <- result == expectedResult
 
     expect_that(resultMatchExpected, is_true())
-}) 
+})
index 5ad8aa03115207035ee7f369ded5fbcd597e0ba7..f12463c805dda10e67325adb2a892d5223600932 100644 (file)
@@ -5,12 +5,12 @@
 context("Http Request")
 
 
-test_that("execyte raises exception if http verb is not valid", {
+test_that("execute raises exception if http verb is not valid", {
 
     http <- HttpRequest$new()
     expect_that(http$exec("FAKE VERB", "url"),
                throws_error("Http verb is not valid."))
-}) 
+})
 
 test_that("createQuery generates and encodes query portion of http", {
 
@@ -22,10 +22,87 @@ test_that("createQuery generates and encodes query portion of http", {
     expect_that(http$createQuery(queryParams),
                 equals(paste0("/?filters=%5B%5B%22color%22%2C%22%3D%22%2C%22red",
                               "%22%5D%5D&limit=20&offset=50")))
-}) 
+})
 
 test_that("createQuery generates and empty string when queryParams is an empty list", {
 
     http <- HttpRequest$new()
     expect_that(http$createQuery(list()), equals(""))
-}) 
+})
+
+test_that("exec calls httr functions correctly", {
+    httrNamespace <- getNamespace("httr")
+
+    # Monkeypatch httr functions and assert that they are called later
+    add_headersCalled <- FALSE
+    unlockBinding("add_headers", httrNamespace)
+    newAddHeaders <- function(h)
+    {
+        add_headersCalled <<- TRUE
+        list()
+    }
+    httrNamespace$add_headers <- newAddHeaders
+    lockBinding("add_headers", httrNamespace)
+
+    expectedConfig <- list()
+    retryCalled <- FALSE
+    unlockBinding("RETRY", httrNamespace)
+    newRETRY <- function(verb, url, body, config, times)
+    {
+        retryCalled <<- TRUE
+        expectedConfig <<- config
+    }
+    httrNamespace$RETRY <- newRETRY
+    lockBinding("RETRY", httrNamespace)
+
+    Sys.setenv("ARVADOS_API_HOST_INSECURE" = TRUE)
+    http <- HttpRequest$new()
+    http$exec("GET", "url")
+
+    expect_that(add_headersCalled, is_true())
+    expect_that(retryCalled, is_true())
+    expect_that(expectedConfig$options, equals(list(ssl_verifypeer = 0L)))
+})
+
+test_that("getConnection calls curl functions correctly", {
+    curlNamespace <- getNamespace("curl")
+
+    # Monkeypatch curl functions and assert that they are called later
+    curlCalled <- FALSE
+    unlockBinding("curl", curlNamespace)
+    newCurl <- function(url, open, handle) curlCalled <<- TRUE
+    curlNamespace$curl <- newCurl
+    lockBinding("curl", curlNamespace)
+
+    new_handleCalled <- FALSE
+    unlockBinding("new_handle", curlNamespace)
+    newHandleFun <- function()
+    {
+        new_handleCalled <<- TRUE
+        list()
+    }
+    curlNamespace$new_handle <- newHandleFun
+    lockBinding("new_handle", curlNamespace)
+
+    handle_setheadersCalled <- FALSE
+    unlockBinding("handle_setheaders", curlNamespace)
+    newHandleSetHeaders <- function(h, .list) handle_setheadersCalled <<- TRUE
+    curlNamespace$handle_setheaders <- newHandleSetHeaders
+    lockBinding("handle_setheaders", curlNamespace)
+
+    handle_setoptCalled <- FALSE
+    unlockBinding("handle_setopt", curlNamespace)
+    newHandleSetOpt <- function(h, ssl_verifypeer) handle_setoptCalled <<- TRUE
+    curlNamespace$handle_setopt <- newHandleSetOpt
+    lockBinding("handle_setopt", curlNamespace)
+
+
+    Sys.setenv("ARVADOS_API_HOST_INSECURE" = TRUE)
+    http <- HttpRequest$new()
+    http$getConnection("location", list(), "r")
+
+    expect_that(new_handleCalled, is_true())
+    expect_that(handle_setheadersCalled, is_true())
+    expect_that(handle_setoptCalled, is_true())
+    expect_that(curlCalled, is_true())
+})
index 859b6180f3380c2d834b99e126aa0c7761155368..26f459b17355903640676b7d4a717bd2c9052890 100644 (file)
@@ -22,7 +22,7 @@ test_that("getWebDavHostName calls REST service properly", {
     expect_that(httpRequest$URLIsProperlyConfigured, is_true())
     expect_that(httpRequest$requestHeaderContainsAuthorizationField, is_true())
     expect_that(httpRequest$numberOfGETRequests, equals(1))
-}) 
+})
 
 test_that("getWebDavHostName returns webDAV host name properly", {
 
@@ -32,8 +32,8 @@ test_that("getWebDavHostName returns webDAV host name properly", {
     REST <- RESTService$new("token", "host",
                             httpRequest, FakeHttpParser$new())
 
-    expect_that("https://myWebDavServer.com", equals(REST$getWebDavHostName())) 
-}) 
+    expect_that("https://myWebDavServer.com", equals(REST$getWebDavHostName()))
+})
 
 test_that("create calls REST service properly", {
 
@@ -51,7 +51,7 @@ test_that("create calls REST service properly", {
     expect_that(fakeHttp$URLIsProperlyConfigured, is_true())
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(fakeHttp$numberOfPUTRequests, equals(1))
-}) 
+})
 
 test_that("create raises exception if server response code is not between 200 and 300", {
 
@@ -60,13 +60,13 @@ test_that("create raises exception if server response code is not between 200 an
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$create("file", uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 test_that("delete calls REST service properly", {
 
@@ -75,7 +75,7 @@ test_that("delete calls REST service properly", {
     fakeHttp <- FakeHttpRequest$new(expectedURL)
     fakeHttpParser <- FakeHttpParser$new()
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, fakeHttpParser,
                             0, "https://webDavHost/")
 
@@ -84,7 +84,7 @@ test_that("delete calls REST service properly", {
     expect_that(fakeHttp$URLIsProperlyConfigured, is_true())
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(fakeHttp$numberOfDELETERequests, equals(1))
-}) 
+})
 
 test_that("delete raises exception if server response code is not between 200 and 300", {
 
@@ -99,7 +99,7 @@ test_that("delete raises exception if server response code is not between 200 an
 
     expect_that(REST$delete("file", uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 test_that("move calls REST service properly", {
 
@@ -108,7 +108,7 @@ test_that("move calls REST service properly", {
     fakeHttp <- FakeHttpRequest$new(expectedURL)
     fakeHttpParser <- FakeHttpParser$new()
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, fakeHttpParser,
                             0, "https://webDavHost/")
 
@@ -118,7 +118,7 @@ test_that("move calls REST service properly", {
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(fakeHttp$requestHeaderContainsDestinationField, is_true())
     expect_that(fakeHttp$numberOfMOVERequests, equals(1))
-}) 
+})
 
 test_that("move raises exception if server response code is not between 200 and 300", {
 
@@ -127,13 +127,13 @@ test_that("move raises exception if server response code is not between 200 and
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$move("file", "newDestination/file", uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 test_that("getCollectionContent retreives correct content from WebDAV server", {
 
@@ -145,7 +145,7 @@ test_that("getCollectionContent retreives correct content from WebDAV server", {
 
     fakeHttp <- FakeHttpRequest$new(expectedURL, returnContent)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
@@ -155,7 +155,7 @@ test_that("getCollectionContent retreives correct content from WebDAV server", {
 
     expect_that(returnedContentMatchExpected, is_true())
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
-}) 
+})
 
 test_that("getCollectionContent raises exception if server returns empty response", {
 
@@ -163,26 +163,26 @@ test_that("getCollectionContent raises exception if server returns empty respons
     response <- ""
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$getCollectionContent(uuid),
                 throws_error("Response is empty, request may be misconfigured"))
-}) 
+})
 
 test_that("getCollectionContent parses server response", {
 
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
     fakeHttpParser <- FakeHttpParser$new()
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             FakeHttpRequest$new(), fakeHttpParser,
                             0, "https://webDavHost/")
 
     REST$getCollectionContent(uuid)
 
     expect_that(fakeHttpParser$parserCallCount, equals(1))
-}) 
+})
 
 test_that("getCollectionContent raises exception if server returns empty response", {
 
@@ -190,13 +190,13 @@ test_that("getCollectionContent raises exception if server returns empty respons
     response <- ""
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$getCollectionContent(uuid),
                 throws_error("Response is empty, request may be misconfigured"))
-}) 
+})
 
 test_that(paste("getCollectionContent raises exception if server",
                 "response code is not between 200 and 300"), {
@@ -206,13 +206,13 @@ test_that(paste("getCollectionContent raises exception if server",
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$getCollectionContent(uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 
 test_that("getResourceSize calls REST service properly", {
@@ -235,7 +235,7 @@ test_that("getResourceSize calls REST service properly", {
     expect_that(fakeHttp$URLIsProperlyConfigured, is_true())
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(returnedContentMatchExpected, is_true())
-}) 
+})
 
 test_that("getResourceSize raises exception if server returns empty response", {
 
@@ -243,13 +243,13 @@ test_that("getResourceSize raises exception if server returns empty response", {
     response <- ""
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$getResourceSize("file", uuid),
                 throws_error("Response is empty, request may be misconfigured"))
-}) 
+})
 
 test_that(paste("getResourceSize raises exception if server",
                 "response code is not between 200 and 300"), {
@@ -259,26 +259,26 @@ test_that(paste("getResourceSize raises exception if server",
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$getResourceSize("file", uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 test_that("getResourceSize parses server response", {
 
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
     fakeHttpParser <- FakeHttpParser$new()
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             FakeHttpRequest$new(), fakeHttpParser,
                             0, "https://webDavHost/")
 
     REST$getResourceSize("file", uuid)
 
     expect_that(fakeHttpParser$parserCallCount, equals(1))
-}) 
+})
 
 test_that("read calls REST service properly", {
 
@@ -290,7 +290,7 @@ test_that("read calls REST service properly", {
 
     fakeHttp <- FakeHttpRequest$new(expectedURL, serverResponse)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
@@ -300,7 +300,7 @@ test_that("read calls REST service properly", {
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(fakeHttp$requestHeaderContainsRangeField, is_true())
     expect_that(returnResult, equals("file content"))
-}) 
+})
 
 test_that("read raises exception if server response code is not between 200 and 300", {
 
@@ -309,48 +309,48 @@ test_that("read raises exception if server response code is not between 200 and
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$read("file", uuid),
                 throws_error("Server code: 404"))
-}) 
+})
 
 test_that("read raises exception if contentType is not valid", {
 
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
     fakeHttp <- FakeHttpRequest$new()
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, HttpParser$new(),
                             0, "https://webDavHost/")
 
     expect_that(REST$read("file", uuid, "some invalid content type"),
                 throws_error("Invalid contentType. Please use text or raw."))
-}) 
+})
 
 test_that("read parses server response", {
 
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
     fakeHttpParser <- FakeHttpParser$new()
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             FakeHttpRequest$new(), fakeHttpParser,
                             0, "https://webDavHost/")
 
     REST$read("file", uuid, "text", 1024, 512)
 
     expect_that(fakeHttpParser$parserCallCount, equals(1))
-}) 
+})
 
 test_that("write calls REST service properly", {
 
-    fileContent <- "new file content" 
+    fileContent <- "new file content"
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
     expectedURL <- "https://webDavHost/c=aaaaa-j7d0g-ccccccccccccccc/file"
     fakeHttp <- FakeHttpRequest$new(expectedURL)
 
-    REST <- RESTService$new("token", "https://host/", 
+    REST <- RESTService$new("token", "https://host/",
                             fakeHttp, FakeHttpParser$new(),
                             0, "https://webDavHost/")
 
@@ -360,12 +360,12 @@ test_that("write calls REST service properly", {
     expect_that(fakeHttp$requestBodyIsProvided, is_true())
     expect_that(fakeHttp$requestHeaderContainsAuthorizationField, is_true())
     expect_that(fakeHttp$requestHeaderContainsContentTypeField, is_true())
-}) 
+})
 
 test_that("write raises exception if server response code is not between 200 and 300", {
 
     uuid <- "aaaaa-j7d0g-ccccccccccccccc"
-    fileContent <- "new file content" 
+    fileContent <- "new file content"
     response <- list()
     response$status_code <- 404
     fakeHttp <- FakeHttpRequest$new(serverResponse = response)
@@ -376,4 +376,17 @@ test_that("write raises exception if server response code is not between 200 and
 
     expect_that(REST$write("file", uuid, fileContent, "text/html"),
                 throws_error("Server code: 404"))
-}) 
+})
+
+test_that("getConnection calls REST service properly", {
+    uuid <- "aaaaa-j7d0g-ccccccccccccccc"
+    fakeHttp <- FakeHttpRequest$new()
+
+    REST <- RESTService$new("token", "https://host/",
+                            fakeHttp, FakeHttpParser$new(),
+                            0, "https://webDavHost/")
+
+    REST$getConnection("file", uuid, "r")
+
+    expect_that(fakeHttp$numberOfgetConnectionCalls, equals(1))
+})
index e025586c58a968f6c0d61a47512087a69d601635..6845801e59abb8155736fddcf4d92595ef67f018 100644 (file)
@@ -15,7 +15,7 @@ test_that("getRelativePath returns path relative to the tree root", {
 
     expect_that(animal$getRelativePath(), equals("animal"))
     expect_that(fish$getRelativePath(), equals("animal/fish"))
-}) 
+})
 
 test_that(paste("getFileListing by default returns sorted path of all files",
                 "relative to the current subcollection"), {
@@ -38,7 +38,7 @@ test_that(paste("getFileListing by default returns sorted path of all files",
                     all(expectedResult == result)
 
     expect_that(resultsMatch, is_true())
-}) 
+})
 
 test_that(paste("getFileListing returns sorted names of all direct children",
                 "if fullPath is set to FALSE"), {
@@ -59,7 +59,7 @@ test_that(paste("getFileListing returns sorted names of all direct children",
                     all(expectedResult == result)
 
     expect_that(resultsMatch, is_true())
-}) 
+})
 
 test_that("add adds content to inside collection tree", {
 
@@ -75,7 +75,7 @@ test_that("add adds content to inside collection tree", {
 
     expect_that(animalContainsFish, is_true())
     expect_that(animalContainsDog, is_true())
-}) 
+})
 
 test_that("add raises exception if content name is empty string", {
 
@@ -86,7 +86,7 @@ test_that("add raises exception if content name is empty string", {
                 throws_error("Content has invalid name.", fixed = TRUE))
 })
 
-test_that(paste("add raises exception if ArvadosFile/Subcollection", 
+test_that(paste("add raises exception if ArvadosFile/Subcollection",
                 "with same name already exists in the subcollection"), {
 
     animal     <- Subcollection$new("animal")
@@ -102,9 +102,9 @@ test_that(paste("add raises exception if ArvadosFile/Subcollection",
     expect_that(animal$add(thirdFish),
                 throws_error(paste("Subcollection already contains ArvadosFile or",
                                    "Subcollection with same name."), fixed = TRUE))
-}) 
+})
 
-test_that(paste("add raises exception if passed argument is", 
+test_that(paste("add raises exception if passed argument is",
                 "not ArvadosFile or Subcollection"), {
 
     animal <- Subcollection$new("animal")
@@ -113,11 +113,11 @@ test_that(paste("add raises exception if passed argument is",
     expect_that(animal$add(number),
                 throws_error(paste("Expected AravodsFile or Subcollection object,",
                                    "got (numeric)."), fixed = TRUE))
-}) 
+})
 
-test_that(paste("add post content to a REST service", 
+test_that(paste("add post content to a REST service",
                 "if subcollection belongs to a collection"), {
-    
+
     collectionContent <- c("animal", "animal/fish")
     fakeREST <- FakeRESTService$new(collectionContent)
 
@@ -131,7 +131,7 @@ test_that(paste("add post content to a REST service",
     animal$add(dog)
 
     expect_that(fakeREST$createCallCount, equals(1))
-}) 
+})
 
 test_that("remove removes content from subcollection", {
 
@@ -144,9 +144,9 @@ test_that("remove removes content from subcollection", {
     returnValueAfterRemovalIsNull <- is.null(animal$get("fish"))
 
     expect_that(returnValueAfterRemovalIsNull, is_true())
-}) 
+})
 
-test_that(paste("remove raises exception", 
+test_that(paste("remove raises exception",
                 "if content to remove doesn't exist in the subcollection"), {
 
     animal <- Subcollection$new("animal")
@@ -154,7 +154,7 @@ test_that(paste("remove raises exception",
     expect_that(animal$remove("fish"),
                 throws_error(paste("Subcollection doesn't contains ArvadosFile",
                                    "or Subcollection with specified name.")))
-}) 
+})
 
 test_that("remove raises exception if passed argument is not character vector", {
 
@@ -164,11 +164,11 @@ test_that("remove raises exception if passed argument is not character vector",
     expect_that(animal$remove(number),
                 throws_error(paste("Expected character,",
                                    "got (numeric)."), fixed = TRUE))
-}) 
+})
 
-test_that(paste("remove removes content from REST service", 
+test_that(paste("remove removes content from REST service",
                 "if subcollection belongs to a collection"), {
-    
+
     collectionContent <- c("animal", "animal/fish", "animal/dog")
     fakeREST <- FakeRESTService$new(collectionContent)
 
@@ -180,9 +180,9 @@ test_that(paste("remove removes content from REST service",
     animal$remove("fish")
 
     expect_that(fakeREST$deleteCallCount, equals(1))
-}) 
+})
 
-test_that(paste("get returns ArvadosFile or Subcollection", 
+test_that(paste("get returns ArvadosFile or Subcollection",
                 "if file or folder with given name exists"), {
 
     animal <- Subcollection$new("animal")
@@ -203,9 +203,9 @@ test_that(paste("get returns ArvadosFile or Subcollection",
 
     expect_that(returnedDogIsArvadosFile, is_true())
     expect_that(returnedDog$getName(), equals("dog"))
-}) 
+})
 
-test_that(paste("get returns NULL if file or folder", 
+test_that(paste("get returns NULL if file or folder",
                 "with given name doesn't exists"), {
 
     animal <- Subcollection$new("animal")
@@ -216,7 +216,7 @@ test_that(paste("get returns NULL if file or folder",
     returnedDogIsNull <- is.null(animal$get("dog"))
 
     expect_that(returnedDogIsNull, is_true())
-}) 
+})
 
 test_that("getFirst returns first child in the subcollection", {
 
@@ -226,7 +226,7 @@ test_that("getFirst returns first child in the subcollection", {
     animal$add(fish)
 
     expect_that(animal$getFirst()$getName(), equals("fish"))
-}) 
+})
 
 test_that("getFirst returns NULL if subcollection contains no children", {
 
@@ -235,7 +235,7 @@ test_that("getFirst returns NULL if subcollection contains no children", {
     returnedElementIsNull <- is.null(animal$getFirst())
 
     expect_that(returnedElementIsNull, is_true())
-}) 
+})
 
 test_that(paste("setCollection by default sets collection",
                 "filed of subcollection and all its children"), {
@@ -248,7 +248,7 @@ test_that(paste("setCollection by default sets collection",
 
     expect_that(animal$getCollection(), equals("myCollection"))
     expect_that(fish$getCollection(), equals("myCollection"))
-}) 
+})
 
 test_that(paste("setCollection sets collection filed of subcollection only",
                 "if parameter setRecursively is set to FALSE"), {
@@ -262,7 +262,7 @@ test_that(paste("setCollection sets collection filed of subcollection only",
 
     expect_that(animal$getCollection(), equals("myCollection"))
     expect_that(fishCollectionIsNull, is_true())
-}) 
+})
 
 test_that(paste("move raises exception if subcollection",
                 "doesn't belong to any collection"), {
@@ -271,7 +271,7 @@ test_that(paste("move raises exception if subcollection",
 
     expect_that(animal$move("new/location"),
                 throws_error("Subcollection doesn't belong to any collection"))
-}) 
+})
 
 test_that("move raises exception if new location contains content with the same name", {
 
@@ -290,7 +290,7 @@ test_that("move raises exception if new location contains content with the same
     expect_that(fish$move("fish"),
                 throws_error("Destination already contains content with same name."))
 
-}) 
+})
 
 test_that(paste("move raises exception if newLocationInCollection",
                 "parameter is invalid"), {
@@ -310,7 +310,7 @@ test_that(paste("move raises exception if newLocationInCollection",
 
     expect_that(fish$move("objects/dog"),
                 throws_error("Unable to get destination subcollection"))
-}) 
+})
 
 test_that("move moves subcollection inside collection tree", {
 
@@ -332,7 +332,7 @@ test_that("move moves subcollection inside collection tree", {
 
     expect_that(fishIsNullOnOldLocation, is_true())
     expect_that(fishExistsOnNewLocation, is_true())
-}) 
+})
 
 test_that(paste("getSizeInBytes returns zero if subcollection",
                 "is not part of a collection"), {
@@ -340,7 +340,7 @@ test_that(paste("getSizeInBytes returns zero if subcollection",
     animal <- Subcollection$new("animal")
 
     expect_that(animal$getSizeInBytes(), equals(0))
-}) 
+})
 
 test_that(paste("getSizeInBytes delegates size calculation",
                 "to REST service class"), {
index 9f5e07c1767af6c089274a308dc3dc270fb25c2f..419e8785fdf53d0b7c0ef538e6dc9eeb8f7a8ee0 100644 (file)
@@ -26,7 +26,7 @@ test_that("listAll always returns all resource items from server", {
     result <- listAll(testFunction)
 
     expect_that(length(result), equals(8))
-}) 
+})
 
 test_that("trimFromStart trims string correctly if string starts with trimCharacters", {
 
@@ -36,7 +36,7 @@ test_that("trimFromStart trims string correctly if string starts with trimCharac
     result <- trimFromStart(sample, trimCharacters)
 
     expect_that(result, equals("random"))
-}) 
+})
 
 test_that("trimFromStart returns original string if string doesn't starts with trimCharacters", {
 
@@ -46,7 +46,7 @@ test_that("trimFromStart returns original string if string doesn't starts with t
     result <- trimFromStart(sample, trimCharacters)
 
     expect_that(result, equals("./something/random"))
-}) 
+})
 
 test_that("trimFromEnd trims string correctly if string ends with trimCharacters", {
 
@@ -56,7 +56,7 @@ test_that("trimFromEnd trims string correctly if string ends with trimCharacters
     result <- trimFromEnd(sample, trimCharacters)
 
     expect_that(result, equals("./something"))
-}) 
+})
 
 test_that("trimFromEnd returns original string if string doesn't end with trimCharacters", {
 
@@ -66,11 +66,11 @@ test_that("trimFromEnd returns original string if string doesn't end with trimCh
     result <- trimFromStart(sample, trimCharacters)
 
     expect_that(result, equals("./something/random"))
-}) 
+})
 
 test_that("RListToPythonList converts nested R list to char representation of Python list", {
 
-    sample <- list("insert", list("random", list("text")), list("here")) 
+    sample <- list("insert", list("random", list("text")), list("here"))
 
     result              <- RListToPythonList(sample)
     resultWithSeparator <- RListToPythonList(sample, separator = ",+")
@@ -78,7 +78,7 @@ test_that("RListToPythonList converts nested R list to char representation of Py
     expect_that(result, equals("[\"insert\", [\"random\", \"text\"], \"here\"]"))
     expect_that(resultWithSeparator,
                 equals("[\"insert\",+[\"random\",+\"text\"],+\"here\"]"))
-}) 
+})
 
 test_that("appendToStartIfNotExist appends characters to beginning of a string", {
 
@@ -88,7 +88,7 @@ test_that("appendToStartIfNotExist appends characters to beginning of a string",
     result <- appendToStartIfNotExist(sample, charactersToAppend)
 
     expect_that(result, equals("Happy New Year"))
-}) 
+})
 
 test_that(paste("appendToStartIfNotExist returns original string if string",
                 "doesn't start with specified characters"), {
@@ -99,7 +99,7 @@ test_that(paste("appendToStartIfNotExist returns original string if string",
     result <- appendToStartIfNotExist(sample, charactersToAppend)
 
     expect_that(result, equals("Happy New Year"))
-}) 
+})
 
 test_that(paste("splitToPathAndName splits relative path to file/folder",
                 "name and rest of the path"), {
@@ -110,4 +110,4 @@ test_that(paste("splitToPathAndName splits relative path to file/folder",
 
     expect_that(result$name, equals("file.exe"))
     expect_that(result$path, equals("path/to/my"))
-}) 
+})
index 3343bdb9aa0e4fcec1d478489dbbf387cc0ff1dd..ec0239eb37bf0a45bb715b35eab757c6c94850d5 100644 (file)
@@ -14,3 +14,7 @@ type APIClientAuthorization struct {
 type APIClientAuthorizationList struct {
        Items []APIClientAuthorization `json:"items"`
 }
+
+func (aca APIClientAuthorization) TokenV2() string {
+       return "v2/" + aca.UUID + "/" + aca.APIToken
+}
index 353901855683f296811a42e64b008568071dbdad..6edd18418bb8015087f8b486acf6ee21d2d26db4 100644 (file)
@@ -56,6 +56,27 @@ type Cluster struct {
        NodeProfiles       map[string]NodeProfile
        InstanceTypes      InstanceTypeMap
        HTTPRequestTimeout Duration
+       RemoteClusters     map[string]RemoteCluster
+       PostgreSQL         PostgreSQL
+}
+
+type PostgreSQL struct {
+       Connection     PostgreSQLConnection
+       ConnectionPool int
+}
+
+type PostgreSQLConnection map[string]string
+
+type RemoteCluster struct {
+       // API endpoint host or host:port; default is {id}.arvadosapi.com
+       Host string
+       // Perform a proxy request when a local client requests an
+       // object belonging to this remote.
+       Proxy bool
+       // Scheme, default "https". Can be set to "http" for testing.
+       Scheme string
+       // Disable TLS verify. Can be set to true for testing.
+       Insecure bool
 }
 
 type InstanceType struct {
@@ -172,6 +193,7 @@ func (np *NodeProfile) ServicePorts() map[ServiceName]string {
 }
 
 type SystemServiceInstance struct {
-       Listen string
-       TLS    bool
+       Listen   string
+       TLS      bool
+       Insecure bool
 }
diff --git a/sdk/go/arvados/postgresql.go b/sdk/go/arvados/postgresql.go
new file mode 100644 (file)
index 0000000..47953ce
--- /dev/null
@@ -0,0 +1,20 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+package arvados
+
+import "strings"
+
+func (c PostgreSQLConnection) String() string {
+       s := ""
+       for k, v := range c {
+               s += strings.ToLower(k)
+               s += "='"
+               s += strings.Replace(
+                       strings.Replace(v, `\`, `\\`, -1),
+                       `'`, `\'`, -1)
+               s += "' "
+       }
+       return s
+}
index a434690775089c38a092499ae79f7fa0fcdec0e0..6a4b6232aceb82754dbee606504a8608ba96d054 100644 (file)
@@ -46,6 +46,8 @@ const (
 
        FooCollectionSharingTokenUUID = "zzzzz-gj3su-gf02tdm4g1z3e3u"
        FooCollectionSharingToken     = "iknqgmunrhgsyfok8uzjlwun9iscwm3xacmzmg65fa1j1lpdss"
+
+       WorkflowWithDefinitionYAMLUUID = "zzzzz-7fd4e-validworkfloyml"
 )
 
 // PathologicalManifest : A valid manifest designed to test
index ea492430e41297ddb8465c73b62c477e20af2357..ad1d398c763d7eaacefefcde8993e39044582f2a 100644 (file)
@@ -34,7 +34,7 @@ var EncodeTokenCookie func([]byte) string = base64.URLEncoding.EncodeToString
 // token.
 var DecodeTokenCookie func(string) ([]byte, error) = base64.URLEncoding.DecodeString
 
-// LoadTokensFromHttpRequest loads all tokens it can find in the
+// LoadTokensFromHTTPRequest loads all tokens it can find in the
 // headers and query string of an http query.
 func (a *Credentials) LoadTokensFromHTTPRequest(r *http.Request) {
        // Load plain token from "Authorization: OAuth2 ..." header
@@ -83,7 +83,21 @@ func (a *Credentials) loadTokenFromCookie(r *http.Request) {
        a.Tokens = append(a.Tokens, string(token))
 }
 
-// TODO: LoadTokensFromHttpRequestBody(). We can't assume in
-// LoadTokensFromHttpRequest() that [or how] we should read and parse
-// the request body. This has to be requested explicitly by the
-// application.
+// LoadTokensFromHTTPRequestBody() loads credentials from the request
+// body.
+//
+// This is separate from LoadTokensFromHTTPRequest() because it's not
+// always desirable to read the request body. This has to be requested
+// explicitly by the application.
+func (a *Credentials) LoadTokensFromHTTPRequestBody(r *http.Request) error {
+       if r.Header.Get("Content-Type") != "application/x-www-form-urlencoded" {
+               return nil
+       }
+       if err := r.ParseForm(); err != nil {
+               return err
+       }
+       if t := r.PostFormValue("api_token"); t != "" {
+               a.Tokens = append(a.Tokens, t)
+       }
+       return nil
+}
diff --git a/sdk/go/auth/salt.go b/sdk/go/auth/salt.go
new file mode 100644 (file)
index 0000000..667a30f
--- /dev/null
@@ -0,0 +1,48 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+package auth
+
+import (
+       "crypto/hmac"
+       "crypto/sha1"
+       "errors"
+       "fmt"
+       "io"
+       "regexp"
+       "strings"
+)
+
+var (
+       reObsoleteToken  = regexp.MustCompile(`^[0-9a-z]{41,}$`)
+       ErrObsoleteToken = errors.New("obsolete token format")
+       ErrTokenFormat   = errors.New("badly formatted token")
+       ErrSalted        = errors.New("token already salted")
+)
+
+func SaltToken(token, remote string) (string, error) {
+       parts := strings.Split(token, "/")
+       if len(parts) < 3 || parts[0] != "v2" {
+               if reObsoleteToken.MatchString(token) {
+                       return "", ErrObsoleteToken
+               } else {
+                       return "", ErrTokenFormat
+               }
+       }
+       uuid := parts[1]
+       secret := parts[2]
+       if len(secret) != 40 {
+               // not already salted
+               hmac := hmac.New(sha1.New, []byte(secret))
+               io.WriteString(hmac, remote)
+               secret = fmt.Sprintf("%x", hmac.Sum(nil))
+               return "v2/" + uuid + "/" + secret, nil
+       } else if strings.HasPrefix(uuid, remote) {
+               // already salted for the desired remote
+               return token, nil
+       } else {
+               // salted for a different remote, can't be used
+               return "", ErrSalted
+       }
+}
index 3289c67b013f37a67ae8ddeaa52d3fd74abe34e5..c3d60309992c9368ba7a2d75586db58829c8f2c0 100644 (file)
@@ -162,19 +162,21 @@ func (d *Dispatcher) checkForUpdates(filters [][]interface{}, todo map[string]*r
        params := arvadosclient.Dict{
                "filters": filters,
                "order":   []string{"priority desc"}}
-
-       var list arvados.ContainerList
-       for offset, more := 0, true; more; offset += len(list.Items) {
+       offset := 0
+       for {
                params["offset"] = offset
+               var list arvados.ContainerList
                err := d.Arv.List("containers", params, &list)
                if err != nil {
                        log.Printf("Error getting list of containers: %q", err)
                        return false
                }
-               more = len(list.Items) > 0 && list.ItemsAvailable > len(list.Items)+offset
                d.checkListForUpdates(list.Items, todo)
+               offset += len(list.Items)
+               if len(list.Items) == 0 || list.ItemsAvailable <= offset {
+                       return true
+               }
        }
-       return true
 }
 
 func (d *Dispatcher) checkListForUpdates(containers []arvados.Container, todo map[string]*runTracker) {
diff --git a/sdk/go/httpserver/metrics.go b/sdk/go/httpserver/metrics.go
new file mode 100644 (file)
index 0000000..77525a8
--- /dev/null
@@ -0,0 +1,129 @@
+// Copyright (C) The Arvados Authors. All rights reserved.
+//
+// SPDX-License-Identifier: Apache-2.0
+
+package httpserver
+
+import (
+       "net/http"
+       "strconv"
+       "strings"
+       "time"
+
+       "git.curoverse.com/arvados.git/sdk/go/stats"
+       "github.com/Sirupsen/logrus"
+       "github.com/gogo/protobuf/jsonpb"
+       "github.com/prometheus/client_golang/prometheus"
+       "github.com/prometheus/client_golang/prometheus/promhttp"
+)
+
+type Handler interface {
+       http.Handler
+
+       // Returns an http.Handler that serves the Handler's metrics
+       // data at /metrics and /metrics.json, and passes other
+       // requests through to next.
+       ServeAPI(next http.Handler) http.Handler
+}
+
+type metrics struct {
+       next         http.Handler
+       logger       *logrus.Logger
+       registry     *prometheus.Registry
+       reqDuration  *prometheus.SummaryVec
+       timeToStatus *prometheus.SummaryVec
+       exportProm   http.Handler
+}
+
+func (*metrics) Levels() []logrus.Level {
+       return logrus.AllLevels
+}
+
+// Fire implements logrus.Hook in order to collect data points from
+// request logs.
+func (m *metrics) Fire(ent *logrus.Entry) error {
+       if tts, ok := ent.Data["timeToStatus"].(stats.Duration); !ok {
+       } else if method, ok := ent.Data["reqMethod"].(string); !ok {
+       } else if code, ok := ent.Data["respStatusCode"].(int); !ok {
+       } else {
+               m.timeToStatus.WithLabelValues(strconv.Itoa(code), strings.ToLower(method)).Observe(time.Duration(tts).Seconds())
+       }
+       return nil
+}
+
+func (m *metrics) exportJSON(w http.ResponseWriter, req *http.Request) {
+       jm := jsonpb.Marshaler{Indent: "  "}
+       mfs, _ := m.registry.Gather()
+       w.Write([]byte{'['})
+       for i, mf := range mfs {
+               if i > 0 {
+                       w.Write([]byte{','})
+               }
+               jm.Marshal(w, mf)
+       }
+       w.Write([]byte{']'})
+}
+
+// ServeHTTP implements http.Handler.
+func (m *metrics) ServeHTTP(w http.ResponseWriter, req *http.Request) {
+       m.next.ServeHTTP(w, req)
+}
+
+// ServeAPI returns a new http.Handler that serves current data at
+// metrics API endpoints (currently "GET /metrics(.json)?") and passes
+// other requests through to next.
+//
+// Typical example:
+//
+//     m := Instrument(...)
+//     srv := http.Server{Handler: m.ServeAPI(m)}
+func (m *metrics) ServeAPI(next http.Handler) http.Handler {
+       return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
+               switch {
+               case req.Method != "GET" && req.Method != "HEAD":
+                       next.ServeHTTP(w, req)
+               case req.URL.Path == "/metrics.json":
+                       m.exportJSON(w, req)
+               case req.URL.Path == "/metrics":
+                       m.exportProm.ServeHTTP(w, req)
+               default:
+                       next.ServeHTTP(w, req)
+               }
+       })
+}
+
+// Instrument returns a new Handler that passes requests through to
+// the next handler in the stack, and tracks metrics of those
+// requests.
+//
+// For the metrics to be accurate, the caller must ensure every
+// request passed to the Handler also passes through
+// LogRequests(logger, ...), and vice versa.
+func Instrument(logger *logrus.Logger, next http.Handler) Handler {
+       if logger == nil {
+               logger = logrus.StandardLogger()
+       }
+       reqDuration := prometheus.NewSummaryVec(prometheus.SummaryOpts{
+               Name: "request_duration_seconds",
+               Help: "Summary of request duration.",
+       }, []string{"code", "method"})
+       timeToStatus := prometheus.NewSummaryVec(prometheus.SummaryOpts{
+               Name: "time_to_status_seconds",
+               Help: "Summary of request TTFB.",
+       }, []string{"code", "method"})
+       registry := prometheus.NewRegistry()
+       registry.MustRegister(timeToStatus)
+       registry.MustRegister(reqDuration)
+       m := &metrics{
+               next:         promhttp.InstrumentHandlerDuration(reqDuration, next),
+               logger:       logger,
+               registry:     registry,
+               reqDuration:  reqDuration,
+               timeToStatus: timeToStatus,
+               exportProm: promhttp.HandlerFor(registry, promhttp.HandlerOpts{
+                       ErrorLog: logger,
+               }),
+       }
+       m.logger.AddHook(m)
+       return m
+}
index 102433cd4186fbf392d8f2fc56af804bdec4d890..8df95553d49e825db63286d9125077b6c53682a6 100644 (file)
@@ -413,6 +413,13 @@ def run_controller():
         f.write("""
 Clusters:
   zzzzz:
+    PostgreSQL:
+      ConnectionPool: 32
+      Connection:
+        host: {}
+        dbname: {}
+        user: {}
+        password: {}
     NodeProfiles:
       "*":
         "arvados-controller":
@@ -420,7 +427,15 @@ Clusters:
         "arvados-api-server":
           Listen: ":{}"
           TLS: true
-        """.format(port, rails_api_port))
+          Insecure: true
+        """.format(
+            _dbconfig('host'),
+            _dbconfig('database'),
+            _dbconfig('username'),
+            _dbconfig('password'),
+            port,
+            rails_api_port,
+        ))
     logf = open(_logfilename('controller'), 'a')
     controller = subprocess.Popen(
         ["arvados-server", "controller", "-config", conf],
index 594dc436297224463f860fe3d74cb9d718e02bdf..f0992c18314ac22bc60034855a204ab3aedce796 100644 (file)
@@ -25,7 +25,7 @@ class StaticController < ApplicationController
   end
 
   def empty
-    render text: "-"
+    render text: ""
   end
 
 end
index ee2f699339f8a66fbc2efc6bd457b33b8c41411f..5109ea46a642a0853528d40986c2314610f050a1 100644 (file)
@@ -36,11 +36,11 @@ class CrossOriginTest < ActionDispatch::IntegrationTest
   ['/arvados/v1/collections',
    '/arvados/v1/users',
    '/arvados/v1/api_client_authorizations'].each do |path|
-    test "CORS headers are set and body is stub at OPTIONS #{path}" do
+    test "CORS headers are set and body is empty at OPTIONS #{path}" do
       options path, {}, {}
       assert_response :success
       assert_cors_headers
-      assert_equal '-', response.body
+      assert_equal '', response.body
     end
 
     test "CORS headers are set at authenticated GET #{path}" do
index 23a8a0ca01124df89575c5724a4b6cb6650527fb..719ec98d27aa19d65eceb7d3db3a46f506aed2f0 100644 (file)
@@ -55,11 +55,12 @@ func (s *IntegrationSuite) TearDownTest(c *C) {
 }
 
 type slurmFake struct {
-       didBatch   [][]string
-       didCancel  []string
-       didRelease []string
-       didRenice  [][]string
-       queue      string
+       didBatch      [][]string
+       didCancel     []string
+       didRelease    []string
+       didRenice     [][]string
+       queue         string
+       rejectNice10K bool
        // If non-nil, run this func during the 2nd+ call to Cancel()
        onCancel func()
        // Error returned by Batch()
@@ -82,6 +83,9 @@ func (sf *slurmFake) Release(name string) error {
 
 func (sf *slurmFake) Renice(name string, nice int64) error {
        sf.didRenice = append(sf.didRenice, []string{name, fmt.Sprintf("%d", nice)})
+       if sf.rejectNice10K && nice > 10000 {
+               return errors.New("scontrol: error: Invalid nice value, must be between -10000 and 10000")
+       }
        return nil
 }
 
index 742943f197580e186e7fd1f7b8084a1357f3661d..fd4851eb0a8a92b48fcacef0e4552ce99d0a7f48 100644 (file)
@@ -14,11 +14,14 @@ import (
        "time"
 )
 
+const slurm15NiceLimit int64 = 10000
+
 type slurmJob struct {
        uuid         string
        wantPriority int64
        priority     int64 // current slurm priority (incorporates nice value)
        nice         int64 // current slurm nice value
+       hitNiceLimit bool
 }
 
 // Squeue implements asynchronous polling monitor of the SLURM queue using the
@@ -103,10 +106,18 @@ func (sqc *SqueueChecker) reniceAll() {
        })
        renice := wantNice(jobs, sqc.PrioritySpread)
        for i, job := range jobs {
-               if renice[i] == job.nice {
+               niceNew := renice[i]
+               if job.hitNiceLimit && niceNew > slurm15NiceLimit {
+                       niceNew = slurm15NiceLimit
+               }
+               if niceNew == job.nice {
                        continue
                }
-               sqc.Slurm.Renice(job.uuid, renice[i])
+               err := sqc.Slurm.Renice(job.uuid, niceNew)
+               if err != nil && niceNew > slurm15NiceLimit && strings.Contains(err.Error(), "Invalid nice value") {
+                       log.Printf("container %q clamping nice values at %d, priority order will not be correct -- see https://dev.arvados.org/projects/arvados/wiki/SLURM_integration#Limited-nice-values-SLURM-15", job.uuid, slurm15NiceLimit)
+                       job.hitNiceLimit = true
+               }
        }
 }
 
@@ -157,14 +168,17 @@ func (sqc *SqueueChecker) check() {
                replacing.nice = n
                newq[uuid] = replacing
 
-               if state == "PENDING" && ((reason == "BadConstraints" && p == 0) || reason == "launch failed requeued held") && replacing.wantPriority > 0 {
+               if state == "PENDING" && ((reason == "BadConstraints" && p <= 2*slurm15NiceLimit) || reason == "launch failed requeued held") && replacing.wantPriority > 0 {
                        // When using SLURM 14.x or 15.x, our queued
                        // jobs land in this state when "scontrol
                        // reconfigure" invalidates their feature
                        // constraints by clearing all node features.
                        // They stay in this state even after the
                        // features reappear, until we run "scontrol
-                       // release {jobid}".
+                       // release {jobid}". Priority is usually 0 in
+                       // this state, but sometimes (due to a race
+                       // with nice adjustments?) it's a small
+                       // positive value.
                        //
                        // "scontrol release" is silent and successful
                        // regardless of whether the features have
@@ -175,7 +189,7 @@ func (sqc *SqueueChecker) check() {
                        // "launch failed requeued held" seems to be
                        // another manifestation of this problem,
                        // resolved the same way.
-                       log.Printf("releasing held job %q", uuid)
+                       log.Printf("releasing held job %q (priority=%d, state=%q, reason=%q)", uuid, p, state, reason)
                        sqc.Slurm.Release(uuid)
                } else if p < 1<<20 && replacing.wantPriority > 0 {
                        log.Printf("warning: job %q has low priority %d, nice %d, state %q, reason %q", uuid, p, n, state, reason)
index c9329fdf95bf87028346fb727b8521dc8edfa1cd..ef036dabd781edd425b29fc28f847ae18370d700 100644 (file)
@@ -103,6 +103,50 @@ func (s *SqueueSuite) TestReniceAll(c *C) {
        }
 }
 
+// If a limited nice range prevents desired priority adjustments, give
+// up and clamp nice to 10K.
+func (s *SqueueSuite) TestReniceInvalidNiceValue(c *C) {
+       uuids := []string{"zzzzz-dz642-fake0fake0fake0", "zzzzz-dz642-fake1fake1fake1", "zzzzz-dz642-fake2fake2fake2"}
+       slurm := &slurmFake{
+               queue:         uuids[0] + " 0 4294000222 PENDING Resources\n" + uuids[1] + " 0 4294555222 PENDING Resources\n",
+               rejectNice10K: true,
+       }
+       sqc := &SqueueChecker{
+               Slurm:          slurm,
+               PrioritySpread: 1,
+               Period:         time.Hour,
+       }
+       sqc.startOnce.Do(sqc.start)
+       sqc.check()
+       sqc.SetPriority(uuids[0], 2)
+       sqc.SetPriority(uuids[1], 1)
+
+       // First attempt should renice to 555001, which will fail
+       sqc.reniceAll()
+       c.Check(slurm.didRenice, DeepEquals, [][]string{{uuids[1], "555001"}})
+
+       // Next attempt should renice to 10K, which will succeed
+       sqc.reniceAll()
+       c.Check(slurm.didRenice, DeepEquals, [][]string{{uuids[1], "555001"}, {uuids[1], "10000"}})
+       // ...so we'll change the squeue response to reflect the
+       // updated priority+nice, and make sure sqc sees that...
+       slurm.queue = uuids[0] + " 0 4294000222 PENDING Resources\n" + uuids[1] + " 10000 4294545222 PENDING Resources\n"
+       sqc.check()
+
+       // Next attempt should leave nice alone because it's already
+       // at the 10K limit
+       sqc.reniceAll()
+       c.Check(slurm.didRenice, DeepEquals, [][]string{{uuids[1], "555001"}, {uuids[1], "10000"}})
+
+       // Back to normal if desired nice value falls below 10K
+       slurm.queue = uuids[0] + " 0 4294000222 PENDING Resources\n" + uuids[1] + " 10000 4294000111 PENDING Resources\n"
+       sqc.check()
+       sqc.reniceAll()
+       c.Check(slurm.didRenice, DeepEquals, [][]string{{uuids[1], "555001"}, {uuids[1], "10000"}, {uuids[1], "9890"}})
+
+       sqc.Stop()
+}
+
 // If the given UUID isn't in the slurm queue yet, SetPriority()
 // should wait for it to appear on the very next poll, then give up.
 func (s *SqueueSuite) TestSetPriorityBeforeQueued(c *C) {
index 89cd26ac49a8b76fcf0053633ca26917477c9478..d65156f98781f99cd3fbc4a20b2f0ba144ea8f97 100644 (file)
 // avoids redirecting requests to keep-web if they depend on
 // TrustAllContent being enabled.
 //
+// Metrics
+//
+// Keep-web exposes request metrics in Prometheus text-based format at
+// /metrics. The same information is also available as JSON at
+// /metrics.json.
+//
 package main
index 7d17be6e7cfe8c59305b452c8d788bca5748acdc..d0ba431aa6312d64f44e518d5ca19d8826ad1c5c 100644 (file)
@@ -31,6 +31,7 @@ import (
 
 type handler struct {
        Config        *Config
+       MetricsAPI    http.Handler
        clientPool    *arvadosclient.ClientPool
        setupOnce     sync.Once
        healthHandler http.Handler
@@ -259,6 +260,9 @@ func (h *handler) ServeHTTP(wOrig http.ResponseWriter, r *http.Request) {
        } else if r.URL.Path == "/status.json" {
                h.serveStatus(w, r)
                return
+       } else if strings.HasPrefix(r.URL.Path, "/metrics") {
+               h.MetricsAPI.ServeHTTP(w, r)
+               return
        } else if siteFSDir[pathParts[0]] {
                useSiteFS = true
        } else if len(pathParts) >= 1 && strings.HasPrefix(pathParts[0], "c=") {
index 206bf6f4381fd98d4e7c4244e787c040de558aad..68ed062160401e59bb79479c71fbfde21a2495e1 100644 (file)
@@ -29,7 +29,7 @@ type UnitSuite struct{}
 
 func (s *UnitSuite) TestCORSPreflight(c *check.C) {
        h := handler{Config: DefaultConfig()}
-       u, _ := url.Parse("http://keep-web.example/c=" + arvadostest.FooCollection + "/foo")
+       u := mustParseURL("http://keep-web.example/c=" + arvadostest.FooCollection + "/foo")
        req := &http.Request{
                Method:     "OPTIONS",
                Host:       u.Host,
@@ -70,8 +70,7 @@ func (s *UnitSuite) TestInvalidUUID(c *check.C) {
                "http://" + bogusID + ".keep-web/t=" + token + "/" + bogusID + "/foo",
        } {
                c.Log(trial)
-               u, err := url.Parse(trial)
-               c.Assert(err, check.IsNil)
+               u := mustParseURL(trial)
                req := &http.Request{
                        Method:     "GET",
                        Host:       u.Host,
index e51376c3bc35cc10a92bf5a6f7c646a18bea3476..58ec348c882b88e6c92116dca947bd24e62c5877 100644 (file)
@@ -5,6 +5,8 @@
 package main
 
 import (
+       "net/http"
+
        "git.curoverse.com/arvados.git/sdk/go/httpserver"
 )
 
@@ -14,7 +16,10 @@ type server struct {
 }
 
 func (srv *server) Start() error {
-       srv.Handler = httpserver.AddRequestIDs(httpserver.LogRequests(nil, &handler{Config: srv.Config}))
+       h := &handler{Config: srv.Config}
+       mh := httpserver.Instrument(nil, httpserver.AddRequestIDs(httpserver.LogRequests(nil, h)))
+       h.MetricsAPI = mh.ServeAPI(http.NotFoundHandler())
+       srv.Handler = mh
        srv.Addr = srv.Config.Listen
        return srv.Server.Start()
 }
index ee585ad5b212af1f12f2bad3f162f8c1c11f3a2f..6688cc2ee743ec53bf4f2ce15fdfcb4621f09253 100644 (file)
@@ -6,10 +6,12 @@ package main
 
 import (
        "crypto/md5"
+       "encoding/json"
        "fmt"
        "io"
        "io/ioutil"
        "net"
+       "net/http"
        "os"
        "os/exec"
        "strings"
@@ -294,6 +296,76 @@ func (s *IntegrationSuite) runCurl(c *check.C, token, host, uri string, args ...
        return
 }
 
+func (s *IntegrationSuite) TestMetrics(c *check.C) {
+       origin := "http://" + s.testServer.Addr
+       req, _ := http.NewRequest("GET", origin+"/notfound", nil)
+       _, err := http.DefaultClient.Do(req)
+       c.Assert(err, check.IsNil)
+       req, _ = http.NewRequest("GET", origin+"/by_id/", nil)
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp, err := http.DefaultClient.Do(req)
+       c.Assert(err, check.IsNil)
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       req, _ = http.NewRequest("GET", origin+"/foo", nil)
+       req.Host = arvadostest.FooCollection + ".example.com"
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp, err = http.DefaultClient.Do(req)
+       c.Assert(err, check.IsNil)
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       buf, _ := ioutil.ReadAll(resp.Body)
+       c.Check(buf, check.DeepEquals, []byte("foo"))
+
+       req, _ = http.NewRequest("GET", origin+"/metrics.json", nil)
+       resp, err = http.DefaultClient.Do(req)
+       c.Assert(err, check.IsNil)
+       c.Check(resp.StatusCode, check.Equals, http.StatusOK)
+       type summary struct {
+               SampleCount string  `json:"sample_count"`
+               SampleSum   float64 `json:"sample_sum"`
+               Quantile    []struct {
+                       Quantile float64
+                       Value    float64
+               }
+       }
+       var ents []struct {
+               Name   string
+               Help   string
+               Type   string
+               Metric []struct {
+                       Label []struct {
+                               Name  string
+                               Value string
+                       }
+                       Summary summary
+               }
+       }
+       json.NewDecoder(resp.Body).Decode(&ents)
+       flat := map[string]summary{}
+       for _, e := range ents {
+               for _, m := range e.Metric {
+                       labels := map[string]string{}
+                       for _, lbl := range m.Label {
+                               labels[lbl.Name] = lbl.Value
+                       }
+                       flat[e.Name+"/"+labels["method"]+"/"+labels["code"]] = m.Summary
+               }
+       }
+       c.Check(flat["request_duration_seconds/get/200"].SampleSum, check.Not(check.Equals), 0)
+       c.Check(flat["request_duration_seconds/get/200"].SampleCount, check.Equals, "2")
+       c.Check(flat["request_duration_seconds/get/404"].SampleCount, check.Equals, "1")
+       c.Check(flat["time_to_status_seconds/get/404"].SampleCount, check.Equals, "1")
+
+       // If the Host header indicates a collection, /metrics.json
+       // refers to a file in the collection -- the metrics handler
+       // must not intercept that route.
+       req, _ = http.NewRequest("GET", origin+"/metrics.json", nil)
+       req.Host = strings.Replace(arvadostest.FooCollectionPDH, "+", "-", -1) + ".example.com"
+       req.Header.Set("Authorization", "Bearer "+arvadostest.ActiveToken)
+       resp, err = http.DefaultClient.Do(req)
+       c.Assert(err, check.IsNil)
+       c.Check(resp.StatusCode, check.Equals, http.StatusNotFound)
+}
+
 func (s *IntegrationSuite) SetUpSuite(c *check.C) {
        arvadostest.StartAPI()
        arvadostest.StartKeep(2, true)
index 3db20e29ce64eaca4d8c5ddd58389595d2f9ce16..1f8c7e31a2997ac2884ae2936ea174a0d859e017 100644 (file)
@@ -9,17 +9,11 @@ import (
        "encoding/json"
        "fmt"
        "io/ioutil"
-       "net/http"
-       "strconv"
        "strings"
        "time"
 
        "git.curoverse.com/arvados.git/sdk/go/arvados"
-       "git.curoverse.com/arvados.git/sdk/go/stats"
        "github.com/Sirupsen/logrus"
-       "github.com/golang/protobuf/jsonpb"
-       "github.com/prometheus/client_golang/prometheus"
-       "github.com/prometheus/client_golang/prometheus/promhttp"
 )
 
 type Config struct {
@@ -54,8 +48,6 @@ type Config struct {
 
        ManagementToken string `doc: The secret key that must be provided by monitoring services
 wishing to access the health check endpoint (/_health).`
-
-       metrics
 }
 
 var (
@@ -161,62 +153,6 @@ func (cfg *Config) Start() error {
        return nil
 }
 
-type metrics struct {
-       registry     *prometheus.Registry
-       reqDuration  *prometheus.SummaryVec
-       timeToStatus *prometheus.SummaryVec
-       exportProm   http.Handler
-}
-
-func (*metrics) Levels() []logrus.Level {
-       return logrus.AllLevels
-}
-
-func (m *metrics) Fire(ent *logrus.Entry) error {
-       if tts, ok := ent.Data["timeToStatus"].(stats.Duration); !ok {
-       } else if method, ok := ent.Data["reqMethod"].(string); !ok {
-       } else if code, ok := ent.Data["respStatusCode"].(int); !ok {
-       } else {
-               m.timeToStatus.WithLabelValues(strconv.Itoa(code), strings.ToLower(method)).Observe(time.Duration(tts).Seconds())
-       }
-       return nil
-}
-
-func (m *metrics) setup() {
-       m.registry = prometheus.NewRegistry()
-       m.timeToStatus = prometheus.NewSummaryVec(prometheus.SummaryOpts{
-               Name: "time_to_status_seconds",
-               Help: "Summary of request TTFB.",
-       }, []string{"code", "method"})
-       m.reqDuration = prometheus.NewSummaryVec(prometheus.SummaryOpts{
-               Name: "request_duration_seconds",
-               Help: "Summary of request duration.",
-       }, []string{"code", "method"})
-       m.registry.MustRegister(m.timeToStatus)
-       m.registry.MustRegister(m.reqDuration)
-       m.exportProm = promhttp.HandlerFor(m.registry, promhttp.HandlerOpts{
-               ErrorLog: log,
-       })
-       log.AddHook(m)
-}
-
-func (m *metrics) exportJSON(w http.ResponseWriter, req *http.Request) {
-       jm := jsonpb.Marshaler{Indent: "  "}
-       mfs, _ := m.registry.Gather()
-       w.Write([]byte{'['})
-       for i, mf := range mfs {
-               if i > 0 {
-                       w.Write([]byte{','})
-               }
-               jm.Marshal(w, mf)
-       }
-       w.Write([]byte{']'})
-}
-
-func (m *metrics) Instrument(next http.Handler) http.Handler {
-       return promhttp.InstrumentHandlerDuration(m.reqDuration, next)
-}
-
 // VolumeTypes is built up by init() funcs in the source files that
 // define the volume types.
 var VolumeTypes = []func() VolumeWithExamples{}
index fb327a386b0f33fdae30f1e0d3e4f880c8d0bfa1..d19be61e9ade77958b376450169f5b2abdafc66b 100644 (file)
@@ -86,17 +86,11 @@ func MakeRESTRouter() http.Handler {
        // 400 Bad Request.
        rtr.NotFoundHandler = http.HandlerFunc(BadRequestHandler)
 
-       theConfig.metrics.setup()
-
        rtr.limiter = httpserver.NewRequestLimiter(theConfig.MaxRequests, rtr)
 
-       mux := http.NewServeMux()
-       mux.Handle("/", theConfig.metrics.Instrument(
-               httpserver.AddRequestIDs(httpserver.LogRequests(nil, rtr.limiter))))
-       mux.HandleFunc("/metrics.json", theConfig.metrics.exportJSON)
-       mux.Handle("/metrics", theConfig.metrics.exportProm)
-
-       return mux
+       stack := httpserver.Instrument(nil,
+               httpserver.AddRequestIDs(httpserver.LogRequests(nil, rtr.limiter)))
+       return stack.ServeAPI(stack)
 }
 
 // BadRequestHandler is a HandleFunc to address bad requests.
index 719124d4000f724a271077d9f1614c50c6788f8d..35c8b5a8c97db40b4d15a7ad20e20acaefdc605a 100644 (file)
@@ -90,7 +90,7 @@ echo %s > /var/tmp/arv-node-data/meta-data/instance-type
             # Need to populate Node.size
             if not n.size:
                 n.size = self.sizes()[n.extra["properties"]["hardwareProfile"]["vmSize"]]
-            n.extra['arvados_node_size'] = n.extra.get('tags', {}).get('arvados_node_size')
+            n.extra['arvados_node_size'] = n.extra.get('tags', {}).get('arvados_node_size') or n.size.id
         return nodes
 
     def broken(self, cloud_node):
index 56812d258a92212b02a53d9775534d8b23b50b69..418a9f9d85499b64e592d614cfa793af70e694ca 100644 (file)
@@ -111,7 +111,7 @@ class ComputeNodeDriver(BaseComputeNodeDriver):
         for n in nodes:
             if not n.size:
                 n.size = self.sizes()[n.extra["instance_type"]]
-            n.extra['arvados_node_size'] = n.extra.get('tags', {}).get('arvados_node_size')
+            n.extra['arvados_node_size'] = n.extra.get('tags', {}).get('arvados_node_size') or n.size.id
         return nodes
 
     @classmethod
index 11025f7840bc00fe6c188ad6b0f9e9bea1795cba..23a1017316656cfe4323646ac9bba5e793f915cb 100644 (file)
@@ -121,7 +121,7 @@ class ComputeNodeDriver(BaseComputeNodeDriver):
             if not hasattr(node.size, 'id'):
                 node.size = self.sizes()[node.size]
             # Get arvados-assigned cloud size id
-            node.extra['arvados_node_size'] = node.extra.get('metadata', {}).get('arvados_node_size')
+            node.extra['arvados_node_size'] = node.extra.get('metadata', {}).get('arvados_node_size') or node.size.id
         return nodelist
 
     @classmethod
index 1020b4a80ced597911b886c40789dea39f1d5598..7ca9c9553721f0fa1291273bfeff5f5f9f7d0e78 100644 (file)
@@ -37,7 +37,10 @@ class ServerCalculator(object):
             self.scratch = 0
             self.cores = 0
             self.bandwidth = 0
-            self.price = 9999999
+            # price is multiplied by 1000 to get the node weight
+            # the maximum node weight is                  4294967280
+            # so use invalid node weight 4294967 * 1000 = 4294967000
+            self.price = 4294967
             self.preemptible = False
             self.extra = {}
 
index d4cf5114399576387eddd8f169005d5a142a8d0f..ead1ec20c6a1de471e82f2e82ad0c24b5e3a4b93 100644 (file)
@@ -12,7 +12,7 @@ import (
 
 type wsConfig struct {
        Client       arvados.Client
-       Postgres     pgConfig
+       Postgres     arvados.PostgreSQLConnection
        PostgresPool int
        Listen       string
        LogLevel     string
@@ -30,7 +30,7 @@ func defaultConfig() wsConfig {
                Client: arvados.Client{
                        APIHost: "localhost:443",
                },
-               Postgres: pgConfig{
+               Postgres: arvados.PostgreSQLConnection{
                        "dbname":                    "arvados_production",
                        "user":                      "arvados",
                        "password":                  "xyzzy",
index 9acfca50e4db639c04dda22a7040d2e91a1c1c4c..309dab7a403e54cc5cb24daaf312dc1b5baa72f2 100644 (file)
@@ -8,7 +8,6 @@ import (
        "context"
        "database/sql"
        "strconv"
-       "strings"
        "sync"
        "sync/atomic"
        "time"
@@ -17,21 +16,6 @@ import (
        "github.com/lib/pq"
 )
 
-type pgConfig map[string]string
-
-func (c pgConfig) ConnectionString() string {
-       s := ""
-       for k, v := range c {
-               s += k
-               s += "='"
-               s += strings.Replace(
-                       strings.Replace(v, `\`, `\\`, -1),
-                       `'`, `\'`, -1)
-               s += "' "
-       }
-       return s
-}
-
 type pgEventSource struct {
        DataSource   string
        MaxOpenConns int
index ea6063a0c3a718dde7baa52a7a9aa5504b0e5f16..ac5d130d61bdd85dfc568bf91c37b983994ae40c 100644 (file)
@@ -7,10 +7,12 @@ package main
 import (
        "database/sql"
        "fmt"
+       "os"
+       "path/filepath"
        "sync"
        "time"
 
-       "git.curoverse.com/arvados.git/sdk/go/config"
+       "git.curoverse.com/arvados.git/sdk/go/arvados"
        check "gopkg.in/check.v1"
 )
 
@@ -18,30 +20,20 @@ var _ = check.Suite(&eventSourceSuite{})
 
 type eventSourceSuite struct{}
 
-func testDBConfig() pgConfig {
-       var railsDB struct {
-               Test struct {
-                       Database string
-                       Username string
-                       Password string
-                       Host     string
-               }
-       }
-       err := config.LoadFile(&railsDB, "../api/config/database.yml")
+func testDBConfig() arvados.PostgreSQLConnection {
+       cfg, err := arvados.GetConfig(filepath.Join(os.Getenv("WORKSPACE"), "tmp", "arvados.yml"))
        if err != nil {
                panic(err)
        }
-       cfg := pgConfig{
-               "dbname":   railsDB.Test.Database,
-               "host":     railsDB.Test.Host,
-               "password": railsDB.Test.Password,
-               "user":     railsDB.Test.Username,
+       cc, err := cfg.GetCluster("zzzzz")
+       if err != nil {
+               panic(err)
        }
-       return cfg
+       return cc.PostgreSQL.Connection
 }
 
 func testDB() *sql.DB {
-       db, err := sql.Open("postgres", testDBConfig().ConnectionString())
+       db, err := sql.Open("postgres", testDBConfig().String())
        if err != nil {
                panic(err)
        }
@@ -52,7 +44,7 @@ func (*eventSourceSuite) TestEventSource(c *check.C) {
        cfg := testDBConfig()
        db := testDB()
        pges := &pgEventSource{
-               DataSource: cfg.ConnectionString(),
+               DataSource: cfg.String(),
                QueueSize:  4,
        }
        go pges.Run()
index 36ce7ae59f15cf9ec2a6fd1609aad1fdd2acd23d..eda7ff2a486a0f9ae59ddc12bf696e3e7a8059c5 100644 (file)
@@ -48,7 +48,7 @@ func (srv *server) setup() {
 
        srv.listener = ln
        srv.eventSource = &pgEventSource{
-               DataSource:   srv.wsConfig.Postgres.ConnectionString(),
+               DataSource:   srv.wsConfig.Postgres.String(),
                MaxOpenConns: srv.wsConfig.PostgresPool,
                QueueSize:    srv.wsConfig.ServerEventQueue,
        }
index f18d4e464cdf34ed86c0be1a4631aadc598179df..aa6b2d773dfb1e47969794dd816a81b179539163 100644 (file)
                        "revision": "7a0fa49edf48165190530c675167e2f319a05268",
                        "revisionTime": "2018-06-25T08:58:08Z"
                },
+               {
+                       "checksumSHA1": "8UEp6v0Dczw/SlasE0DivB0mAHA=",
+                       "path": "github.com/gogo/protobuf/jsonpb",
+                       "revision": "30cf7ac33676b5786e78c746683f0d4cd64fa75b",
+                       "revisionTime": "2018-05-09T16:24:41Z"
+               },
                {
                        "checksumSHA1": "wn2shNJMwRZpvuvkf1s7h0wvqHI=",
                        "path": "github.com/gogo/protobuf/proto",
                        "revisionTime": "2018-01-04T10:21:28Z"
                },
                {
-                       "checksumSHA1": "iVfdaLxIDjfk2KLP8dCMIbsxZZM=",
-                       "path": "github.com/golang/protobuf/jsonpb",
-                       "revision": "1e59b77b52bf8e4b449a57e6f79f21226d571845",
-                       "revisionTime": "2017-11-13T18:07:20Z"
+                       "checksumSHA1": "HPVQZu059/Rfw2bAWM538bVTcUc=",
+                       "path": "github.com/gogo/protobuf/sortkeys",
+                       "revision": "30cf7ac33676b5786e78c746683f0d4cd64fa75b",
+                       "revisionTime": "2018-05-09T16:24:41Z"
                },
                {
-                       "checksumSHA1": "yqF125xVSkmfLpIVGrLlfE05IUk=",
-                       "path": "github.com/golang/protobuf/proto",
-                       "revision": "1e59b77b52bf8e4b449a57e6f79f21226d571845",
-                       "revisionTime": "2017-11-13T18:07:20Z"
+                       "checksumSHA1": "SkxU1+wPGUJyLyQENrZtr2/OUBs=",
+                       "path": "github.com/gogo/protobuf/types",
+                       "revision": "30cf7ac33676b5786e78c746683f0d4cd64fa75b",
+                       "revisionTime": "2018-05-09T16:24:41Z"
                },
                {
-                       "checksumSHA1": "Ylq6kq3KWBy6mu68oyEwenhNMdg=",
-                       "path": "github.com/golang/protobuf/ptypes/struct",
+                       "checksumSHA1": "yqF125xVSkmfLpIVGrLlfE05IUk=",
+                       "path": "github.com/golang/protobuf/proto",
                        "revision": "1e59b77b52bf8e4b449a57e6f79f21226d571845",
                        "revisionTime": "2017-11-13T18:07:20Z"
                },