X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/efb3422f119be36e133ffbd14519c5d6c74addc8..0eb72b526bf8bbb011551ecf019f604e17a534f1:/doc/user/cwl/cwl-extensions.html.textile.liquid?ds=sidebyside

diff --git a/doc/user/cwl/cwl-extensions.html.textile.liquid b/doc/user/cwl/cwl-extensions.html.textile.liquid
index 8a7e64baa8..95422b6bd3 100644
--- a/doc/user/cwl/cwl-extensions.html.textile.liquid
+++ b/doc/user/cwl/cwl-extensions.html.textile.liquid
@@ -3,6 +3,11 @@ layout: default
 navsection: userguide
 title: Arvados CWL Extensions
 ...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
 
 Arvados provides several extensions to CWL for workflow optimization, site-specific configuration, and to enable access the Arvados API.
 
@@ -11,9 +16,10 @@ To use Arvados CWL extensions, add the following @$namespaces@ section at the to
 <pre>
 $namespaces:
   arv: "http://arvados.org/cwl#"
+  cwltool: "http://commonwl.org/cwltool#"
 </pre>
 
-Arvados extensions must go into the @hints@ section, for example:
+For portability, Arvados extensions should go into the @hints@ section of your CWL file, for example:
 
 <pre>
 hints:
@@ -24,8 +30,16 @@ hints:
   arv:PartitionRequirement:
     partition: dev_partition
   arv:APIRequirement: {}
+  cwltool:LoadListingRequirement:
+    loadListing: shallow_listing
+  arv:IntermediateOutput:
+    outputTTL: 3600
+  arv:ReuseRequirement:
+    enableReuse: false
 </pre>
 
+The one exception to this is @arv:APIRequirement@, see note below.
+
 h2. arv:RunInSingleContainer
 
 Indicates that a subworkflow should run in a single container and not be scheduled as separate steps.
@@ -53,4 +67,37 @@ table(table table-bordered table-condensed).
 
 h2. arv:APIRequirement
 
-Indicates that process wants to access to the Arvados API.  Will be granted limited network access and have @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ set in the environment.
+Indicates that process wants to access to the Arvados API.  Will be granted network access and have @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ set in the environment.  Tools which rely on the Arvados API being present should put @arv:APIRequirement@ in the @requirements@ section of the tool (rather than @hints@) to indicate that that it is not portable to non-Arvados CWL runners.
+
+Use @arv:APIRequirement@ in @hints@ to enable general (non-Arvados-specific) network access for a tool.
+
+h2. cwltool:LoadListingRequirement
+
+In CWL v1.0 documents, the default behavior for Directory objects is to recursively expand the @listing@ for access by parameter references an expressions.  For directory trees containing many files, this can be expensive in both time and memory usage.  Use @cwltool:LoadListingRequirement@ to change the behavior for expansion of directory listings in the workflow runner.
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|loadListing|string|One of @no_listing@, @shallow_listing@, or @deep_listing@|
+
+*no_listing*: Do not expand directory listing at all.  The @listing@ field on the Directory object will be undefined.
+
+*shallow_listing*: Only expand the first level of directory listing.  The @listing@ field on the toplevel Directory object will contain the directory contents, however @listing@ will not be defined on subdirectories.
+
+*deep_listing*: Recursively expand all levels of directory listing.  The @listing@ field will be provided on the toplevel object and all subdirectories.
+
+h2. arv:IntermediateOutput
+
+Specify desired handling of intermediate output collections.
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|outputTTL|int|If the value is greater than zero, consider intermediate output collections to be temporary and should be automatically trashed. Temporary collections will be trashed @outputTTL@ seconds after creation.  A value of zero means intermediate output should be retained indefinitely (this is the default behavior).
+Note: arvados-cwl-runner currently does not take workflow dependencies into account when setting the TTL on an intermediate output collection. If the TTL is too short, it is possible for a collection to be trashed before downstream steps that consume it are started.  The recommended minimum value for TTL is the expected duration of the entire the workflow.|
+
+h2. arv:ReuseRequirement
+
+Enable/disable work reuse for current process.  Default true (work reuse enabled).
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|enableReuse|boolean|Enable/disable work reuse for current process.  Default true (work reuse enabled).|