--- layout: default navsection: installguide title: Install the Crunch dispatcher navorder: 3 --- h1. Crunch setup The dispatcher normally runs on the same host/VM as the API server. h4. Perl SDK dependencies * @apt-get install libjson-perl libwww-perl libio-socket-ssl-perl libipc-system-simple-perl@ h4. Python SDK dependencies On controller and all compute nodes: * @apt-get install python-pip@ * @pip install --upgrade google-api-python-client@ h4. Likely crunch job dependencies On compute nodes: * @pip install --upgrade pyvcf@ h4. Redis On controller: * @apt-get install redis-server@ h4. Crunch user account On compute nodes and controller: * @adduser crunch@ The crunch user should have the same UID, GID, and home directory on all compute nodes and on the controller. h4. Repositories Crunch scripts must be in git repositories in @/var/cache/git/*/.git@ (or whatever is configured in @services/api/config/environments/production.rb@). h4. Importing commits @services/api/script/import_commits.rb production@ must run periodically. Example @/var/service/arvados_import_commits/run@ script for daemontools or runit:
#!/bin/sh
set -e
while sleep 60
do
  cd /path/to/arvados/services/api
  setuidgid www-data env RAILS_ENV=production /usr/local/rvm/bin/rvm 2.0.0 do bundle exec ./script/import_commits.rb 2>&1
done
Once you have imported some commits, you should be able to create a new job:
read -rd $'\000' newjob <

Without getting this error:

ArgumentError: Specified script_version does not resolve to a commit
h4. Running jobs * @services/api/script/crunch-dispatch.rb@ must be running. * @crunch-dispatch.rb@ needs @services/crunch/crunch-job@ in its @PATH@. * @crunch-job@ needs @sdk/perl/lib@ and @warehouse-apps/libwarehouse-perl/lib@ in its @PERLLIB@ * @crunch-job@ needs @ARVADOS_API_HOST@ (and, if necessary in a development environment, @ARVADOS_API_HOST_INSECURE@) Example @/var/service/arvados_crunch_dispatch/run@ script:
#!/bin/sh
set -e
export PATH="$PATH":/path/to/arvados/services/crunch
export PERLLIB=/path/to/arvados/sdk/perl/lib:/path/to/warehouse-apps/libwarehouse-perl/lib
export ARVADOS_API_HOST={{ site.arvados_api_host }}
export PYTHONPATH=/path/to/arvados/src/sdk/python
export CRUNCH_DISPATCH_LOCKFILE=/var/lock/crunch-dispatch

fuser -TERM -k $CRUNCH_DISPATCH_LOCKFILE || true

## Only if your SSL cert is unverifiable:
# export ARVADOS_API_HOST_INSECURE=yes

cd /path/to/arvados/services/api
RAILS_ENV=production /usr/local/rvm/bin/rvm 2.0.0 do bundle exec ./script/crunch-dispatch.rb 2>&1