doc/user/tutorials/running-external-program.html.textile.liquid

   1 ---
   2 layout: default
   3 navsection: userguide
   4 navmenu: Tutorials
   5 title: "Running external programs"
   6
   7 ...
   8
   9 h1. Running external programs
  10
  11 This tutorial demonstrates how to use Crunch to run an external program by writting a wrapper using the Python SDK.
  12
  13 *This tutorial assumes that you are "logged into an Arvados VM instance":{{site.baseurl}}/user/getting_started/ssh-access.html#login, and have a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html*
  14
  15 In this tutorial, you will use the external program @md5sum@ to compute hashes instead of the built-in Python library used in earlier tutorials.
  16
  17 Start by entering the @crunch_scripts@ directory of your git repository:
  18
  19 <notextile>
  20 <pre><code>~$ <span class="userinput">cd <b>you</b>/crunch_scripts</span>
  21 </code></pre>
  22 </notextile>
  23
  24 Next, using @nano@ or your favorite Unix text editor, create a new file called @run-md5sum.py@ in the @crunch_scripts@ directory.
  25
  26 notextile. <pre>~/<b>you</b>/crunch_scripts$ <code class="userinput">nano run-md5sum.py</code></pre>
  27
  28 Add the following code to use the @md5sum@ program to compute the hash of each file in a collection:
  29
  30 <pre><code class="userinput">{% include 'run_md5sum_py' %}</code></pre>
  31
  32 Make the file executable:
  33
  34 notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x run-md5sum.py</span></code></pre>
  35
  36 Next, add the file to @git@ staging, commit and push:
  37
  38 <notextile>
  39 <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git add run-md5sum.py</span>
  40 ~/<b>you</b>/crunch_scripts$ <span class="userinput">git commit -m"run external md5sum program"</span>
  41 ~/<b>you</b>/crunch_scripts$ <span class="userinput">git push origin master</span>
  42 </code></pre>
  43 </notextile>
  44
  45 You should now be able to run your new script using Crunch, with "script" referring to our new "run-md5sum.py" script.
  46
  47 <notextile>
  48 <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat &gt;~/the_job &lt;&lt;EOF
  49 {
  50  "script": "run-md5sum.py",
  51  "script_version": "you:master",
  52  "script_parameters":
  53  {
  54   "input": "c1bad4b39ca5a924e481008009d94e32+210"
  55  }
  56 }
  57 EOF</span>
  58 ~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job create --job "$(cat the_job)"</span>
  59 {
  60  ...
  61  "uuid":"qr1hi-xxxxx-xxxxxxxxxxxxxxx"
  62  ...
  63 }
  64 ~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
  65 {
  66  ...
  67  "output":"4d164b1658c261b9afc6b479130016a3+54",
  68  ...
  69 }
  70 </code></pre>
  71 </notextile>