X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/ec8c1f40e94c74927700efffa7357db2f457c9a5..0f83b9e96a52e1ed192b97edbd1a949b2214c147:/doc/user/tutorial-job1.textile diff --git a/doc/user/tutorial-job1.textile b/doc/user/tutorial-job1.textile index 8860b8e1af..ff5f6a1cce 100644 --- a/doc/user/tutorial-job1.textile +++ b/doc/user/tutorial-job1.textile @@ -17,26 +17,32 @@ h3. Prerequisites If everything is set up correctly, the command @arv -h user current@ will display your account information. +Arv depends on a few gems. It will tell you which ones to install, if they are not present yet. If you need to install the dependencies and are doing so as a non-root user, make sure you set GEM_HOME before you run gem install: + +
+ export GEM_HOME=~/.gem ++ h3. Submit a job We will run the "hash" program, which computes the MD5 hash of each file in a collection. -Pick a data collection. We'll use @f815ec01d5d2f11cb12874ab2ed50daa@ here. +Pick a data collection. We'll use @33a9f3842b01ea3fdf27cc582f5ea2af@ here.
-the_collection=f815ec01d5d2f11cb12874ab2ed50daa +the_collection=33a9f3842b01ea3fdf27cc582f5ea2af-Pick a code version. We'll use @cdde7f246fec59bc99da86145fd4cf4efcf37a68@ here. +Pick a code version. We'll use @5565778cf15ae9af22ad392053430213e9016631@ here.
-the_version=cdde7f246fec59bc99da86145fd4cf4efcf37a68 +the_version=5565778cf15ae9af22ad392053430213e9016631Make a JSON object describing the job.
-read -rd "\000" the_job <+↓ + +-(The @read -rd "\000"@ stuff just helps us get a multi-line string with lots of double quotation marks into a shell variable.) +(The @read -rd $'\000'@ part uses a bash feature to help us get a multi-line string with lots of double quotation marks into a shell variable.) Submit the job. @@ -56,6 +62,25 @@ Submit the job. arv -h job create --job "$the_job"
+{ + "kind":"arvados#job", + "etag":"dwbrasqcozpjsqtfshzdjfiii", + "uuid":"qr1hi-8i9sb-3i0yi357k0mauwz", +... + "script":"hash", + "script_parameters":{ + "input":"33a9f3842b01ea3fdf27cc582f5ea2af" + }, + "script_version":"5565778cf15ae9af22ad392053430213e9016631", +... +} ++ +h3. Monitor job progress + Go to Workbench, drop down the Compute menu, and click Jobs. The job you submitted should appear at the top of the list. Hit "Refresh" until it finishes. @@ -64,7 +89,59 @@ You can also watch the log messages while the job runs:
curl -s -H "Authorization: OAuth2 $ARVADOS_API_TOKEN" \ - https://{{ site.arvados_api_host }}/arvados/v1/jobs/JOB_UUID_HERE/log_tail_follow + "https://$ARVADOS_API_HOST/arvados/v1/jobs/JOB_UUID_HERE/log_tail_follow" ++ +This will run until the job finishes or you hit control-C. + +If you're running more than one job today, you can watch log messages from all of them in one stream: + +
+my_user_uuid=`arv user current` +curl -s -H "Authorization: OAuth2 $ARVADOS_API_TOKEN" \ + "https://$ARVADOS_API_HOST/arvados/v1/users/$my_user_uuid/event_stream" ++ +This will run until you hit control-C. + +h3. Inspect the job output + +Find the output of the job by looking at the Jobs page (in the Compute menu) in Workbench, or by using the API: + +
+arv -h job get --uuid JOB_UUID_HERE-Great. You ran a job! +The output locator will look like
5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi
.
+
+List the files in the collection:
+
++arv keep ls 5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi ++ +↓ + +
+md5sum.txt ++ +Show the contents of the md5sum.txt file: + +
+arv keep less 5894dfae5d6d8edf135f0ea3dba849c2+62+K@qr1hi/md5sum.txt ++ +h3. Inspect the code + +The @script@ and @script_version@ attributes of a Job allow you to confirm the code that was used to run the job. Specifically, @script@ refers to a file in the @/crunch_scripts@ directory in the tree indicated by the commit hash @script_version@. + +Example: + +
+cd +git clone git://github.com/clinicalfuture/arvados.git +cd arvados +git checkout $the_version +less crunch_scripts/hash +