X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/7a669da106bd775adffb69ffbfda639509e62e3f..f2cf5a532e7dee1ab68948db89272a0e2579e781:/doc/install/install-crunch-dispatch.html.textile.liquid diff --git a/doc/install/install-crunch-dispatch.html.textile.liquid b/doc/install/install-crunch-dispatch.html.textile.liquid index 4905b45dce..231d1f45e8 100644 --- a/doc/install/install-crunch-dispatch.html.textile.liquid +++ b/doc/install/install-crunch-dispatch.html.textile.liquid @@ -5,28 +5,21 @@ title: Install the Crunch dispatcher ... -h1. Crunch setup + The dispatcher normally runs on the same host/VM as the API server. h4. Perl SDK dependencies -* @apt-get install libjson-perl libwww-perl libio-socket-ssl-perl libipc-system-simple-perl@ - -Add this to @/etc/apt/sources.list@ - -@deb http://git.oxf.freelogy.org/apt wheezy main contrib@ - -Then +Install the Perl SDK on the controller. -@apt-get install libwarehouse-perl@ +* See "Perl SDK":{{site.baseurl}}/sdk/perl/index.html page for details. h4. Python SDK dependencies -On controller and all compute nodes: +Install the Python SDK and CLI tools on controller and all compute nodes. -* @apt-get install python-pip@ -* @pip install --upgrade virtualenv arvados-python-client@ +* See "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html page for details. h4. Likely crunch job dependencies @@ -34,12 +27,6 @@ On compute nodes: * @pip install --upgrade pyvcf@ -h4. Redis - -On controller: - -* @apt-get install redis-server@ - h4. Crunch user account On compute nodes and controller: @@ -50,29 +37,16 @@ The crunch user should have the same UID, GID, and home directory on all compute h4. Repositories -Crunch scripts must be in git repositories in @/var/cache/git/*/.git@ (or whatever is configured in @services/api/config/environments/production.rb@). - -h4. Importing commits +Crunch scripts must be in Git repositories in the directory configured as @git_repositories_dir@/*.git (see the "API server installation":install-api-server.html#git_repositories_dir). -@services/api/script/import_commits.rb production@ must run periodically. Example @/var/service/arvados_import_commits/run@ script for daemontools or runit: - -
-#!/bin/sh -set -e -while sleep 60 -do - cd /path/to/arvados/services/api - setuidgid www-data env RAILS_ENV=production /usr/local/rvm/bin/rvm-exec 2.0.0 bundle exec ./script/import_commits.rb 2>&1 -done -- -Once you have imported some commits, you should be able to create a new job: +Once you have a repository with commits -- and you have read access to the repository -- you should be able to create a new job:
read -rd $'\000' newjob <@@ -94,8 +68,12 @@ Example @/var/service/arvados_crunch_dispatch/run@ script: #!/bin/sh set -e + +rvmexec="" +## uncomment this line if you use rvm: +#rvmexec="/usr/local/rvm/bin/rvm-exec 2.1.1" + export PATH="$PATH":/path/to/arvados/services/crunch -export PERLLIB=/path/to/arvados/sdk/perl/lib:/path/to/warehouse-apps/libwarehouse-perl/lib export ARVADOS_API_HOST={{ site.arvados_api_host }} export CRUNCH_DISPATCH_LOCKFILE=/var/lock/crunch-dispatch @@ -106,5 +84,5 @@ fuser -TERM -k $CRUNCH_DISPATCH_LOCKFILE || true cd /path/to/arvados/services/api export RAILS_ENV=production -exec /usr/local/rvm/bin/rvm-exec 2.0.0 bundle exec ./script/crunch-dispatch.rb 2>&1 +exec $rvmexec bundle exec ./script/crunch-dispatch.rb 2>&1