X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/dd126c77ccfc583e982441352bbcde07212c1178..0f83b9e96a52e1ed192b97edbd1a949b2214c147:/doc/user/tutorial-job1.textile?ds=sidebyside diff --git a/doc/user/tutorial-job1.textile b/doc/user/tutorial-job1.textile index e4b1a45aff..ff5f6a1cce 100644 --- a/doc/user/tutorial-job1.textile +++ b/doc/user/tutorial-job1.textile @@ -17,26 +17,32 @@ h3. Prerequisites If everything is set up correctly, the command @arv -h user current@ will display your account information. +Arv depends on a few gems. It will tell you which ones to install, if they are not present yet. If you need to install the dependencies and are doing so as a non-root user, make sure you set GEM_HOME before you run gem install: + +
+ export GEM_HOME=~/.gem ++ h3. Submit a job We will run the "hash" program, which computes the MD5 hash of each file in a collection. -Pick a data collection. We'll use @f815ec01d5d2f11cb12874ab2ed50daa@ here. +Pick a data collection. We'll use @33a9f3842b01ea3fdf27cc582f5ea2af@ here.
-the_collection=f815ec01d5d2f11cb12874ab2ed50daa +the_collection=33a9f3842b01ea3fdf27cc582f5ea2af-Pick a code version. We'll use @cdde7f246fec59bc99da86145fd4cf4efcf37a68@ here. +Pick a code version. We'll use @5565778cf15ae9af22ad392053430213e9016631@ here.
-the_version=cdde7f246fec59bc99da86145fd4cf4efcf37a68 +the_version=5565778cf15ae9af22ad392053430213e9016631Make a JSON object describing the job.
-read -rd "\000" the_job <-(The @read -rd "\000"@ stuff just helps us get a multi-line string with lots of double quotation marks into a shell variable.) +(The @read -rd $'\000'@ part uses a bash feature to help us get a multi-line string with lots of double quotation marks into a shell variable.) Submit the job. -arv -h job new --job "$newjob" +arv -h job create --job "$the_job" ++ +↓ + ++{ + "kind":"arvados#job", + "etag":"dwbrasqcozpjsqtfshzdjfiii", + "uuid":"qr1hi-8i9sb-3i0yi357k0mauwz", +... + "script":"hash", + "script_parameters":{ + "input":"33a9f3842b01ea3fdf27cc582f5ea2af" + }, + "script_version":"5565778cf15ae9af22ad392053430213e9016631", +... +}-Go to Workbench, drop down the Compute menu, and click Jobs. +h3. Monitor job progress -The job you submitted should appear at the top of the list. +Go to Workbench, drop down the Compute menu, and click Jobs. The job you submitted should appear at the top of the list. -Hit Refresh until it finishes. +Hit "Refresh" until it finishes. You can also watch the log messages while the job runs:curl -s -H "Authorization: OAuth2 $ARVADOS_API_TOKEN" \ - https://{{ site.arvados_api_host }}/arvados/v1/jobs/JOB_UUID_HERE/log_tail_follow + "https://$ARVADOS_API_HOST/arvados/v1/jobs/JOB_UUID_HERE/log_tail_follow"-Great. You ran a job! +This will run until the job finishes or you hit control-C. + +If you're running more than one job today, you can watch log messages from all of them in one stream: + ++my_user_uuid=`arv user current` +curl -s -H "Authorization: OAuth2 $ARVADOS_API_TOKEN" \ + "https://$ARVADOS_API_HOST/arvados/v1/users/$my_user_uuid/event_stream" ++ +This will run until you hit control-C. + +h3. Inspect the job output + +Find the output of the job by looking at the Jobs page (in the Compute menu) in Workbench, or by using the API: + ++arv -h job get --uuid JOB_UUID_HERE ++ +The output locator will look like5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi
. + +List the files in the collection: + ++arv keep ls 5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi ++ +↓ + ++md5sum.txt ++ +Show the contents of the md5sum.txt file: + ++arv keep less 5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi/md5sum.txt ++ +h3. Inspect the code + +The @script@ and @script_version@ attributes of a Job allow you to confirm the code that was used to run the job. Specifically, @script@ refers to a file in the @/crunch_scripts@ directory in the tree indicated by the commit hash @script_version@. + +Example: + ++cd +git clone git://github.com/clinicalfuture/arvados.git +cd arvados +git checkout $the_version +less crunch_scripts/hash +