3 navsection: installguide
4 title: Install the Crunch dispatcher
10 The dispatcher normally runs on the same host/VM as the API server.
12 h4. Perl SDK dependencies
14 * @apt-get install libjson-perl libwww-perl libio-socket-ssl-perl libipc-system-simple-perl@
16 Add this to @/etc/apt/sources.list@
18 @deb http://git.oxf.freelogy.org/apt wheezy main contrib@
22 @apt-get install libwarehouse-perl@
24 h4. Python SDK dependencies
26 On controller and all compute nodes:
28 * @apt-get install python-pip@
29 * @pip install --upgrade virtualenv arvados-python-client@
31 h4. Likely crunch job dependencies
35 * @pip install --upgrade pyvcf@
41 * @apt-get install redis-server@
43 h4. Crunch user account
45 On compute nodes and controller:
49 The crunch user should have the same UID, GID, and home directory on all compute nodes and on the controller.
53 Crunch scripts must be in Git repositories in @/var/lib/arvados/git/*.git@ (or whatever is configured in @services/api/config/environments/production.rb@).
55 Once you have a repository with commits -- and you have read access to the repository -- you should be able to create a new job:
58 read -rd $'\000' newjob <<EOF; arv job create --job "$newjob"
59 {"script_parameters":{"input":"f815ec01d5d2f11cb12874ab2ed50daa"},
60 "script_version":"master",
62 "repository":"arvados"}
66 Without getting this error:
69 ArgumentError: Specified script_version does not resolve to a commit
74 * @services/api/script/crunch-dispatch.rb@ must be running.
75 * @crunch-dispatch.rb@ needs @services/crunch/crunch-job@ in its @PATH@.
76 * @crunch-job@ needs @sdk/perl/lib@ and @warehouse-apps/libwarehouse-perl/lib@ in its @PERLLIB@
77 * @crunch-job@ needs @ARVADOS_API_HOST@ (and, if necessary in a development environment, @ARVADOS_API_HOST_INSECURE@)
79 Example @/var/service/arvados_crunch_dispatch/run@ script:
86 ## uncomment this line if you use rvm:
87 #rvmexec="/usr/local/rvm/bin/rvm-exec 2.1.1"
89 export PATH="$PATH":/path/to/arvados/services/crunch
90 export ARVADOS_API_HOST={{ site.arvados_api_host }}
91 export CRUNCH_DISPATCH_LOCKFILE=/var/lock/crunch-dispatch
93 fuser -TERM -k $CRUNCH_DISPATCH_LOCKFILE || true
95 ## Only if your SSL cert is unverifiable:
96 # export ARVADOS_API_HOST_INSECURE=yes
98 cd /path/to/arvados/services/api
99 export RAILS_ENV=production
100 exec $rvmexec bundle exec ./script/crunch-dispatch.rb 2>&1