X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/26446197bb00d7f899673b7f9434e6bd622b8904..448f667a574b50da096051a0d062b9059ab3609f:/doc/user/topics/tutorial-parallel.html.textile.liquid

diff --git a/doc/user/topics/tutorial-parallel.html.textile.liquid b/doc/user/topics/tutorial-parallel.html.textile.liquid
deleted file mode 100644
index 6dbdb8af23..0000000000
--- a/doc/user/topics/tutorial-parallel.html.textile.liquid
+++ /dev/null
@@ -1,80 +0,0 @@
----
-layout: default
-navsection: userguide
-title: "Parallel Crunch tasks"
-...
-
-In the previous tutorials, we used @arvados.job_setup.one_task_per_input_file()@ to automatically parallelize our jobs by creating a separate task per file.  For some types of jobs, you may need to split the work up differently, for example creating tasks to process different segments of a single large file.  In this this tutorial will demonstrate how to create Crunch tasks directly.
-
-Start by entering the @crunch_scripts@ directory of your git repository:
-
-<notextile>
-<pre><code>~$ <span class="userinput">cd <b>you</b>/crunch_scripts</span>
-</code></pre>
-</notextile>
-
-Next, using @nano@ or your favorite Unix text editor, create a new file called @parallel-hash.py@ in the @crunch_scripts@ directory.
-
-notextile. <pre>~/<b>you</b>/crunch_scripts$ <code class="userinput">nano parallel-hash.py</code></pre>
-
-Add the following code to compute the md5 hash of each file in a 
-
-<notextile> {% code 'parallel_hash_script_py' as python %} </notextile>
-
-Make the file executable:
-
-notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x parallel-hash.py</span></code></pre>
-
-Next, add the file to @git@ staging, commit and push:
-
-<notextile>
-<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git add parallel-hash.py</span>
-~/<b>you</b>/crunch_scripts$ <span class="userinput">git commit -m"parallel hash"</span>
-~/<b>you</b>/crunch_scripts$ <span class="userinput">git push origin master</span>
-</code></pre>
-</notextile>
-
-You should now be able to run your new script using Crunch, with "script" referring to our new "parallel-hash.py" script.  We will use a different input from our previous examples.  We will use @887cd41e9c613463eab2f0d885c6dd96+83@ which consists of three files, "alice.txt", "bob.txt" and "carol.txt" (the example collection used previously in "fetching data from Arvados using Keep":{{site.baseurl}}/user/tutorials/tutorial-keep.html#dir).
-
-<notextile>
-<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat &gt;~/the_job &lt;&lt;EOF
-{
- "script": "parallel-hash.py",
- "repository": "<b>you</b>",
- "script_version": "master",
- "script_parameters":
- {
-  "input": "887cd41e9c613463eab2f0d885c6dd96+83"
- }
-}
-EOF</span>
-~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job create --job "$(cat ~/the_job)"</span>
-{
- ...
- "uuid":"qr1hi-xxxxx-xxxxxxxxxxxxxxx"
- ...
-}
-~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
-{
- ...
- "output":"e2ccd204bca37c77c0ba59fc470cd0f7+162",
- ...
-}
-</code></pre>
-</notextile>
-
-Because the job ran in parallel, each instance of parallel-hash creates a separate @md5sum.txt@ as output.  Arvados automatically collates theses files into a single collection, which is the output of the job:
-
-<notextile>
-<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep ls e2ccd204bca37c77c0ba59fc470cd0f7+162</span>
-md5sum.txt
-md5sum.txt
-md5sum.txt
-~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162/md5sum.txt</span>
-0f1d6bcf55c34bed7f92a805d2d89bbf alice.txt
-504938460ef369cd275e4ef58994cffe bob.txt
-8f3b36aff310e06f3c5b9e95678ff77a carol.txt
-</code></pre>
-</notextile>
-
-