X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/e12b10d886b1739bef1587b39ca5753fb6163771..d5ba0e97f8522ba3ce6ad36edf099c661a43f6b7:/doc/user/tutorials/tutorial-keep.textile diff --git a/doc/user/tutorials/tutorial-keep.textile b/doc/user/tutorials/tutorial-keep.textile index d156e0ae53..6683498e86 100644 --- a/doc/user/tutorials/tutorial-keep.textile +++ b/doc/user/tutorials/tutorial-keep.textile @@ -1,11 +1,12 @@ --- layout: default navsection: userguide +navmenu: Tutorials title: "Storing and Retrieving data using Arvados Keep" -navorder: 111 +navorder: 11 --- -h1. Tutorial: Storing and Retrieving data using Arvados Keep +h1. Storing and Retrieving data using Arvados Keep This tutorial introduces you to the Arvados file storage system. @@ -20,29 +21,38 @@ The Arvados distributed file system is called *Keep*. Keep is a content-address h1. Putting Data into Keep -We will start with downloading a freely available VCF exome from the "Personal Genome Project (PGP)":http://www.personalgenomes.org subject "hu599905":https://my.personalgenomes.org/profile/hu599905 and add it to Keep. From an Arvados VM instance: +We will start with downloading a freely available VCF file from the "Personal Genome Project (PGP)":http://www.personalgenomes.org subject "hu599905":https://my.personalgenomes.org/profile/hu599905 to a staging directory on the VM, and then add it to Keep. + +First, log into the Arvados VM instance and set up the staging area: + +notextile.
$ mkdir /scratch/you
+ +Next, download the file: -
$ curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse.personalgenomes.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2'
+
$ mkdir /scratch/you
+$ cd /scratch/you
+$ curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse.personalgenomes.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2'
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
 100  216M  100  216M    0     0  10.0M      0  0:00:21  0:00:21 --:--:-- 9361k
 
-Alternately, if you have your own data, for example @MyExome.vcf@, you can use @rsync@ on your local computer to copy it to the shell VM: +Alternately, if you have your own data, for example @MyData.vcf@, you can use @scp@ or @rsync@ to copy from your local workstation to the shell VM (run this on your local workstation): -notextile.
$ rsync MyExome.vcf shell.qr1hi:MyExome.vcf
+notextile.
$ scp MyData.vcf you@shell.arvados:/scratch/you/MyData.vcf
Now use @arv keep put@ to add your VCF data to Keep: -
$ arv keep put var-GS000016015-ASM.tsv.bz2
-33a9f3842b01ea3fdf27cc582f5ea2af
+
$ cd /scratch/you
+$ arv keep put var-GS000016015-ASM.tsv.bz2
+c1bad4b39ca5a924e481008009d94e32+210
 
-The output value @33a9f3842b01ea3fdf27cc582f5ea2af@ is the Keep locator. This enables you to access the file you just uploaded, and is explained in the next section. +The output value @c1bad4b39ca5a924e481008009d94e32+210@ is the Keep locator. This enables you to access the file you just uploaded, and is explained in the next section. h2. Putting a directory