From: Peter Amstutz Date: Thu, 16 Oct 2014 15:28:40 +0000 (-0400) Subject: Making arv-run documentation more readable. X-Git-Tag: 1.1.0~2032^2~14 X-Git-Url: https://git.arvados.org/arvados.git/commitdiff_plain/556fb8540d73e3af07a5bea07e3c831fe477b958 Making arv-run documentation more readable. --- diff --git a/doc/user/topics/arv-run.html.textile.liquid b/doc/user/topics/arv-run.html.textile.liquid index b406e6b159..186c7325e0 100644 --- a/doc/user/topics/arv-run.html.textile.liquid +++ b/doc/user/topics/arv-run.html.textile.liquid @@ -8,27 +8,31 @@ The @arv-run@ command enables you create Arvados pipelines at the command line t {% include 'tutorial_expectations' %} -h1. Quick introduction +h1. Usage + +@arv-run@ takes a command or command pipeline, along with stdin and stdout redirection, and creates an Arvados pipeline to run the command. The syntax is designed to mimic standard shell syntax, so it is usually necessary to quote the metacharacters < > and | as either \< \> and \| or '<' '>' and '|'. -Run one @grep@ task per file, and redirect the output to output.txt +@arv-run@ introspects the command line to determine which arguments are file inputs. If you specify a file that is only available on the local filesystem, it will be first uploaded to Arvados, and then the command line will be rewritten to refer to the newly uploaded file. @arv-run@ also works together with @arv-mount@ to identify if a file specified on the command line is part of an Arvados collection. If so, the command line will be rewritten to refer to the file within the collection without any upload necessary. + +@arv-run@ will parallelize on the files listed on the command line after @--@. You may specify @--batch-size N@ after the @--@ but before listing any files to specify how many files to provide put on the command line for each task (see below for example). 
 $ cd ~/keep/by_id/3229739b505d2b878b62aed09895a55a+142
+$ ls
+HWI-ST1027_129_D0THKACXX.1_1.fastq  HWI-ST1027_129_D0THKACXX.1_2.fastq
 $ arv-run grep -H -n ATTGGAGGAAAGATGAGTGAC -- *.fastq \> output.txt
 Running pipeline qr1hi-d1hrv-mg3bju0u7r6w241
-h1. Usage - -@arv-run@ takes a command or command pipeline, along with stdin and stdout redirection, and creates an Arvados pipeline to run the command. The syntax is designed to mimic standard shell syntax, so it is usually necessary to quote the metacharacters < > and | as either \< \> and \| or '<' '>' and '|'. +You may use stdin @<@ redirection on multiple files. This will create a separate task for each input file: -@arv-run@ introspects the command line to determine which arguments are file inputs. If you specify a file that is only available on the local filesystem, it will be first uploaded to Arvados, and then the command line will be rewritten to refer to the newly uploaded file. @arv-run@ also works together with @arv-mount@ to identify if a file specified on the command line is part of an Arvados collection. If so, the command line will be rewritten to refer to the file within the collection without any upload necessary. - -@arv-run@ will parallelize on the files listed on the command line after @--@. You may specify @--batch-size N@ after the @--@ but before listing any files to specify how many files to provide put on the command line for each task (see below for example). - -You may use stdin @<@ redirection on multiple files. This will create a separate task for each input file. + +
+$ arv-run grep -H -n ATTGGAGGAAAGATGAGTGAC \< *.fastq \> output.txt
+
+ You are only permitted to supply a single file name for stdout @>@ redirection. If there are multiple tasks, their output will be collated at the end of the pipeline. Alternately, you may use "run-command":run-command.html parameter substitution in the file name to generate different filenames for each task. @@ -44,11 +48,6 @@ h1. Examples Run one @grep@ task per file, with each input files piped from stdin. Redirect the output to output.txt. - -
-$ arv-run grep -H -n ATTGGAGGAAAGATGAGTGAC \< *.fastq \> output.txt
-
- Run @cat | grep@ once per file. Redirect the output to output.txt.