doc/install/install-crunch-dispatch.textile

   1 ---
   2 layout: default
   3 navsection: installguide
   4 title: Install the Crunch dispatcher
   5 navorder: 3
   6 ---
   7
   8 {% include alert-stub.html %}
   9
  10 h1. Crunch setup
  11
  12 h4. Perl SDK dependencies
  13
  14 * @apt-get install libjson-perl libwww-perl libio-socket-ssl-perl libipc-system-simple-perl@
  15
  16 h4. Repositories
  17
  18 Crunch scripts must be in git repositories in @/var/cache/git/*/.git@ (or whatever is configured in @services/api/config/environments/production.rb@).
  19
  20 h4. Importing commits
  21
  22 @services/api/script/import_commits.rb production@ must run periodically. Example @/var/service/arvados_import_commits/run@ script for daemontools or runit:
  23
  24 <pre>
  25 #!/bin/sh
  26 set -e
  27 while sleep 60
  28 do
  29   cd /path/to/arvados/services/api
  30   setuidgid www-data env RAILS_ENV=production bundle exec ./script/import_commits.rb
  31 done
  32 </pre>
  33
  34 Once you have imported some commits, you should be able to create a new job:
  35
  36 <pre>
  37 read newjob <<EOF; arv job create --job "$newjob"
  38 {"script_parameters":{"input":"f815ec01d5d2f11cb12874ab2ed50daa"},
  39  "script_version":"master",
  40  "script":"hash"}
  41 EOF
  42 </pre>
  43
  44 Without getting this error:
  45
  46 <pre>
  47 ArgumentError: Specified script_version does not resolve to a commit
  48 </pre>
  49
  50 h4. Running jobs
  51
  52 * @services/api/script/crunch-dispatch.rb@ must be running.
  53 * @crunch-dispatch.rb@ needs @services/crunch/crunch-job@ in its @PATH@.
  54 * @crunch-job@ needs @sdk/perl/lib@ and @warehouse-apps/libwarehouse-perl/lib@ in its @PERLLIB@
  55
  56 Example @/var/service/arvados_crunch_dispatch/run@ script:
  57
  58 <pre>
  59 #!/bin/sh
  60 set -e
  61 PATH=$PATH:/path/to/arvados/services/crunch
  62 export PERLLIB=/path/to/arvados/sdk/perl/lib:/path/to/warehouse-apps/libwarehouse-perl/lib
  63 cd /path/to/arvados/services/api
  64 RAILS_ENV=production bundle exec ./script/crunch-dispatch.rb
  65 </pre>