X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/a6010696b246cc43c721a35d56179c629d54e798..f5764a05d616e8d3c55b94503d5a1f789ac66ea7:/doc/user/topics/tutorial-parallel.html.textile.liquid diff --git a/doc/user/topics/tutorial-parallel.html.textile.liquid b/doc/user/topics/tutorial-parallel.html.textile.liquid index 9be610358b..7d308dcff9 100644 --- a/doc/user/topics/tutorial-parallel.html.textile.liquid +++ b/doc/user/topics/tutorial-parallel.html.textile.liquid @@ -3,8 +3,15 @@ layout: default navsection: userguide title: "Concurrent Crunch tasks" ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. -In the previous tutorials, we used @arvados.job_setup.one_task_per_input_file()@ to automatically create concurrent jobs by creating a separate task per file. For some types of jobs, you may need to split the work up differently, for example creating tasks to process different segments of a single large file. In this this tutorial will demonstrate how to create Crunch tasks directly. +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} + +{% include 'pipeline_deprecation_notice' %} + +In the previous tutorials, we used @arvados.job_setup.one_task_per_input_file()@ to automatically create concurrent jobs by creating a separate task per file. For some types of jobs, you may need to split the work up differently, for example creating tasks to process different segments of a single large file. This tutorial will demonstrate how to create Crunch tasks directly. Start by entering the @crunch_scripts@ directory of your Git repository: @@ -40,7 +47,7 @@ You should now be able to run your new script using Crunch, with "script" referr
~/$USER/crunch_scripts$ cat >~/the_job <<EOF
 {
  "script": "concurrent-hash.py",
- "repository": "$USER",
+ "repository": "$USER/$USER",
  "script_version": "master",
  "script_parameters":
  {
@@ -70,7 +77,7 @@ Because the job ran in concurrent, each instance of concurrent-hash creates a se
 
 
~/$USER/crunch_scripts$ arv keep ls e2ccd204bca37c77c0ba59fc470cd0f7+162
 ./md5sum.txt
-~/$USER/crunch_scripts$ arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162/md5sum.txt
+~/$USER/crunch_scripts$ arv-get e2ccd204bca37c77c0ba59fc470cd0f7+162/md5sum.txt
 0f1d6bcf55c34bed7f92a805d2d89bbf alice.txt
 504938460ef369cd275e4ef58994cffe bob.txt
 8f3b36aff310e06f3c5b9e95678ff77a carol.txt