X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/4f552c0187f8c31d94ff74485c57ef7f9888597e..360b5d19eb0cb595fd34d11aa6426d95b3c974d4:/doc/user/topics/tutorial-parallel.html.textile.liquid diff --git a/doc/user/topics/tutorial-parallel.html.textile.liquid b/doc/user/topics/tutorial-parallel.html.textile.liquid index 021d736385..d7a093a245 100644 --- a/doc/user/topics/tutorial-parallel.html.textile.liquid +++ b/doc/user/topics/tutorial-parallel.html.textile.liquid @@ -6,7 +6,7 @@ title: "Parallel Crunch tasks" In the previous tutorials, we used @arvados.job_setup.one_task_per_input_file()@ to automatically parallelize our jobs by creating a separate task per file. For some types of jobs, you may need to split the work up differently, for example creating tasks to process different segments of a single large file. In this this tutorial will demonstrate how to create Crunch tasks directly. -Start by entering the @crunch_scripts@ directory of your git repository: +Start by entering the @crunch_scripts@ directory of your Git repository:
~$ cd you/crunch_scripts
@@ -17,7 +17,7 @@ Next, using @nano@ or your favorite Unix text editor, create a new file called @
 
 notextile. 
~/you/crunch_scripts$ nano parallel-hash.py
-Add the following code to compute the md5 hash of each file in a collection: +Add the following code to compute the MD5 hash of each file in a collection: {% code 'parallel_hash_script_py' as python %} @@ -25,7 +25,7 @@ Make the file executable: notextile.
~/you/crunch_scripts$ chmod +x parallel-hash.py
-Next, add the file to @git@ staging, commit and push: +Add the file to the Git staging area, commit, and push:
~/you/crunch_scripts$ git add parallel-hash.py
@@ -63,7 +63,7 @@ EOF
 
-(Your shell should automatically fill in @$USER@ with your login name. The job JSON that gets saved should have @"repository"@ pointed at your personal git repository.) +(Your shell should automatically fill in @$USER@ with your login name. The job JSON that gets saved should have @"repository"@ pointed at your personal Git repository.) Because the job ran in parallel, each instance of parallel-hash creates a separate @md5sum.txt@ as output. Arvados automatically collates theses files into a single collection, which is the output of the job: