X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/8acbf6d96287dbdf3ac69a4b0ef98452e4c203ed..e1491e3edb997dc6683acd1ebf1c1a95d09d4298:/sdk/cwl/tests/federation/README

diff --git a/sdk/cwl/tests/federation/README b/sdk/cwl/tests/federation/README
index e537939006..e5eb04c600 100644
--- a/sdk/cwl/tests/federation/README
+++ b/sdk/cwl/tests/federation/README
@@ -2,54 +2,43 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-Things to test.
+Arvados federated workflow testing
 
-Single step --submit --no-wait workflow, matrix of:
+Requires cwltool 1.0.20181109150732 or later
 
-Runner on home/remote cluster
-Docker image on home/remote cluster
-Step on home/remote cluster
+Create main-test.json:
 
-Two step workflow, matrix of:
+{
+    "acr": "/path/to/arvados-cwl-runner",
+    "arvado_api_host_insecure": false,
+    "arvados_api_hosts": [
+        "c97qk.arvadosapi.com",
+        "4xphq.arvadosapi.com",
+        "9tee4.arvadosapi.com"
+    ],
+    "arvados_api_token": "...",
+    "arvados_cluster_ids": [
+        "c97qk",
+        "4xphq",
+        "9tee4"
+    ]
+}
 
-Step1 on home cluster -> Step2 on remote cluster
-Step1 on remote cluster -> Step2 on home cluster
+Or create an arvbox test cluster:
 
-Three step workflow:
+$ cwltool --enable-ext arvbox-make-federation.cwl --arvbox_base ~/.arvbox/ --in_acr /path/to/arvados-cwl-runner > main-test.json
 
-(Step1 on remoteA cluster, Step2 on remoteB cluster) -> Step3 on home cluster
-Step1 on remoteA cluster -> Step2 on remoteB cluster -> Step3 on home cluster
 
-Workflow which has a remote collection in InitialWorkDir, which is captured in output.
+Run tests:
 
-Workflow with file input that has a secondary file in a separate
-collection, which is remote.
+$ cwltool main.cwl main-test.json
 
 
-Need to pull Docker image
-Can't run in container because it doesn't have access to Docker from inside -> how can we workaround?
-Turns out we also can't run in container because cwltool doesn't pathmap inside "Any" (this is a bug -> fix this)
-Try setting up a virtualenv -> annoying because you need C dependencies to build
-Virtualenv sets up shop in /tmp instead of the real directory
---relocatable ???
+List test cases:
 
+$ cwltool --print-targets main.cwl
 
-arvados/jobs doesn't have the docker client
-arvbox has docker client
 
-- Still need venv or something to run cwltool to manage all this.
-- Unless we run cwltool in docker as well.
+Run a specific test case:
 
-Can use cwl-docker.sh to run cwltool in docker
-
-- arvbox needs Docker.  This will *probably* work launched from container if base dir is under PWD
-- want to run arvados-cwl-runner.  this needs Docker to pull images.
-
-For a-c-r solutions are one of:
-
-1) Don't run in Docker, use virtualenv instead.
-2) Access to Docker from inside Docker to pull images.
-2a) Docker-in-Docker
-2b) Bind mount Docker socket
-2c) Communicate to Docker daemon over TCP
-3) Arvados feature to pull images
+$ cwltool -t twostep-remote-copy-to-home main.cwl main-test.json