17390: Document arv:OutputStorageClass extension

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

diff --git a/doc/user/cwl/cwl-extensions.html.textile.liquid b/doc/user/cwl/cwl-extensions.html.textile.liquid
index 0987218e3c0f7a84aff9316614e2c964bc70a561..206b75d58f20695c34f322236ee249ee5f3fefab 100644 (file)
--- a/doc/user/cwl/cwl-extensions.html.textile.liquid
+++ b/doc/user/cwl/cwl-extensions.html.textile.liquid
@@ -24,29 +24,34 @@ For portability, most Arvados extensions should go into the @hints@ section of y
 {% codeblock as yaml %}
 hints:
   arv:RunInSingleContainer: {}
+
   arv:RuntimeConstraints:
     keep_cache: 123456
     outputDirType: keep_output_dir
+
   arv:PartitionRequirement:
     partition: dev_partition
+
   arv:APIRequirement: {}
-  cwltool:LoadListingRequirement:
-    loadListing: shallow_listing
+
   arv:IntermediateOutput:
     outputTTL: 3600
-  arv:ReuseRequirement:
-    enableReuse: false
+
   cwltool:Secrets:
     secrets: [input1, input2]
-  cwltool:TimeLimit:
-    timelimit: 14400
+
   arv:WorkflowRunnerResources:
     ramMin: 2048
     coresMin: 2
     keep_cache: 512
+
   arv:ClusterTarget:
     cluster_id: clsr1
     project_uuid: clsr1-j7d0g-qxc4jcji7n4lafx
+
+  arv:OutputStorageClass:
+    intermediateStorageClass: fast_storage
+    finalStorageClass: robust_storage
 {% endcodeblock %}
 
 h2(#RunInSingleContainer). arv:RunInSingleContainer
@@ -120,6 +125,15 @@ table(table table-bordered table-condensed).
 |cluster_id|string|The five-character alphanumeric cluster id (uuid prefix) where a container or subworkflow will execute.  May be an expression.|
 |project_uuid|string|The uuid of the project which will own container request and output of the container.  May be an expression.|
 
+h2(#OutputStorageClass). arv:OutputStorageClass
+
+Specify the "storage class":{{site.baseurl}}/user/topics/storage-classes.html to use for intermediate and final outputs.
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|intermediateStorageClass|string or array of strings|The storage class for output of intermediate steps.  For example, faster "hot" storage.|
+|finalStorageClass_uuid|string or array of strings|The storage class for the final output.  |
+
 h2. arv:dockerCollectionPDH
 
 This is an optional extension field appearing on the standard @DockerRequirement@.  It specifies the portable data hash of the Arvados collection containing the Docker image.  If present, it takes precedence over @dockerPull@ or @dockerImageId@.
@@ -135,6 +149,16 @@ h1. Deprecated extensions
 
 The following extensions are deprecated because equivalent features are part of the CWL v1.1 standard.
 
+{% codeblock as yaml %}
+hints:
+  cwltool:LoadListingRequirement:
+    loadListing: shallow_listing
+  arv:ReuseRequirement:
+    enableReuse: false
+  cwltool:TimeLimit:
+    timelimit: 14400
+{% endcodeblock %}
+
 h2. cwltool:LoadListingRequirement
 
 For CWL v1.1 scripts, this is deprecated in favor of "loadListing":https://www.commonwl.org/v1.1/CommandLineTool.html#CommandInputParameter or "LoadListingRequirement":https://www.commonwl.org/v1.1/CommandLineTool.html#LoadListingRequirement

diff --git a/doc/user/topics/storage-classes.html.textile.liquid b/doc/user/topics/storage-classes.html.textile.liquid
index 650c3709559546aea019c2348607a30d79246cb2..06fd4d811c2b0e927f13e4fbb1af2c3f8395b386 100644 (file)
--- a/doc/user/topics/storage-classes.html.textile.liquid
+++ b/doc/user/topics/storage-classes.html.textile.liquid
@@ -10,9 +10,11 @@ Copyright (C) The Arvados Authors. All rights reserved.
 SPDX-License-Identifier: CC-BY-SA-3.0
 {% endcomment %}
 
-Storage classes (alternately known as "storage tiers") allow you to control which volumes should be used to store particular collection data blocks.  This can be used to implement data storage policies such as moving data to archival storage.
+Storage classes (sometimes called as "storage tiers") allow you to control which back-end storage volumes should be used to store the data blocks of a particular collection.  This can be used to implement data storage policies such as assigning data collections to "fast", "robust" or "archival" storage.
 
-Names of storage classes are internal to the cluster and decided by the administrator.  Aside from "default", Arvados currently does not define any standard storage class names.
+Names of storage classes are internal to the cluster and decided by the administrator.  Aside from "default", Arvados currently does not define any standard storage class names.  Consult your cluster administrator for guidance on what storage classes are available to use on your specific Arvados instance.
+
+Note that when changing the storage class of an existing collection, it does not take effect immediately, the blocks are asynchronously copied to the new storage class and removed from the old one.  The collection field "storage_classes_confirmed" is updated to reflect when data blocks have been successfully copied.
 
 h3. arv-put
 
@@ -32,14 +34,12 @@ $ arv-mount --storage-classes=transient --mount-tmp=scratch keep
 
 h3. arvados-cwl-runner
 
-You may also specify the desired storage class for the final output collection produced by @arvados-cwl-runner@:
+You may specify the desired storage class for the intermediate and final output collections produced by @arvados-cwl-runner@ on the command line or using the "arv:OutputStorageClass hint":{{site.baseurl}}/user/cwl/cwl-extensions.html#OutputStorageClass .
 
 <pre>
-$ arvados-cwl-runner --storage-classes=hot myworkflow.cwl myinput.yml
+$ arvados-cwl-runner --intermediate-storage-classes=hot_storage --storage-classes=robust_storage myworkflow.cwl myinput.yml
 </pre>
 
-(Note: intermediate collections produced by a workflow run will use the cluster's default storage class(es).)
-
 h3. arv command line
 
 You may set the storage class on an existing collection by setting the "storage_classes_desired" field of a Collection.  For example, at the command line: