From 367aa69a24ee56f77417f4b43b72987179eda809 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Fri, 18 Feb 2022 14:24:14 -0500 Subject: [PATCH] 18773: Add --match-submitter-images to docs Also update cwltool dependency for a bugfix. Arvados-DCO-1.1-Signed-off-by: Peter Amstutz --- doc/user/cwl/cwl-run-options.html.textile.liquid | 1 + doc/user/cwl/cwl-runner.html.textile.liquid | 8 ++++++++ sdk/cwl/setup.py | 2 +- 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/doc/user/cwl/cwl-run-options.html.textile.liquid b/doc/user/cwl/cwl-run-options.html.textile.liquid index a1c102593a..d331dad871 100644 --- a/doc/user/cwl/cwl-run-options.html.textile.liquid +++ b/doc/user/cwl/cwl-run-options.html.textile.liquid @@ -51,6 +51,7 @@ table(table table-bordered table-condensed). |==--submit-runner-ram== SUBMIT_RUNNER_RAM|RAM (in MiB) required for the workflow runner (default 1024)| |==--submit-runner-image== SUBMIT_RUNNER_IMAGE|Docker image for workflow runner| |==--always-submit-runner==|When invoked with --submit --wait, always submit a runner to manage the workflow, even when only running a single CommandLineTool| +|==--match-submitter-images==|Where Arvados has more than one Docker image of the same name, use image from the Docker instance on the submitting node.| |==--submit-request-uuid== UUID|Update and commit to supplied container request instead of creating a new one.| |==--submit-runner-cluster== CLUSTER_ID|Submit workflow runner to a remote cluster| |==--name NAME==|Name to use for workflow execution instance.| diff --git a/doc/user/cwl/cwl-runner.html.textile.liquid b/doc/user/cwl/cwl-runner.html.textile.liquid index b108de551a..07663849ad 100644 --- a/doc/user/cwl/cwl-runner.html.textile.liquid +++ b/doc/user/cwl/cwl-runner.html.textile.liquid @@ -113,6 +113,14 @@ h3. Work reuse Workflows submitted with @arvados-cwl-runner@ will take advantage of Arvados job reuse. If you submit a workflow which is identical to one that has run before, it will short cut the execution and return the result of the previous run. This also applies to individual workflow steps. For example, a two step workflow where the first step has run before will reuse results for first step and only execute the new second step. You can disable this behavior with @--disable-reuse@. +h3(#docker). Docker images + +Docker images referenced by the workflow must be uploaded to Arvados. This requires @docker@ to be installed and usable by the user running @arvados-cwl-runner@. If the image is not present in the local Docker instance, @arvados-cwl-runner@ will first attempt to pull the image using @docker pull@, then upload it. + +If there is already a Docker image in Arvados with the same name, it will use the existing image. In this case, the submitter will not use Docker. + +The @--match-submitter-images@ option will check the id of the image in the local Docker instance and compare it to the id of the image already in Arvados with the same name and tag. If they are different, it will choose the image matching the local image id, which will be uploaded it if necessary. This helpful for development, if you locally rebuild the image with the 'latest' tag, the @--match-submitter-images@ will ensure that the newer version is used. + h3. Command line options See "arvados-cwl-runner options":{{site.baseurl}}/user/cwl/cwl-run-options.html diff --git a/sdk/cwl/setup.py b/sdk/cwl/setup.py index e1883d7370..e126d170b7 100644 --- a/sdk/cwl/setup.py +++ b/sdk/cwl/setup.py @@ -36,7 +36,7 @@ setup(name='arvados-cwl-runner', # file to determine what version of cwltool and schema-salad to # build. install_requires=[ - 'cwltool==3.1.20220217190813', + 'cwltool==3.1.20220217222804', 'schema-salad==8.2.20211116214159', 'arvados-python-client{}'.format(pysdk_dep), 'setuptools', -- 2.30.2