X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/6403b8728d3120d6cb22583b3963d2a030e88aa3..a1c4ece152e1e992c18f0cb368528cd2ac843b07:/doc/user/tutorials/tutorial-keep.html.textile.liquid diff --git a/doc/user/tutorials/tutorial-keep.html.textile.liquid b/doc/user/tutorials/tutorial-keep.html.textile.liquid index 74319fda7d..ec7086db96 100644 --- a/doc/user/tutorials/tutorial-keep.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep.html.textile.liquid @@ -3,44 +3,38 @@ layout: default navsection: userguide title: "Uploading data" ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. -This tutorial describes how to to upload new Arvados data collections using the command line tool @arv-put@. This example uses a freely available TSV file containing variant annotations from "Personal Genome Project (PGP)":http://www.pgp-hms.org participant "hu599905.":https://my.pgp-hms.org/profile/hu599905 +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} + +Arvados Data collections can be uploaded using either the @arv-put@ command line tool or using Workbench. + +# "*Upload using command line tool*":#upload-using-command +# "*Upload using Workbench*":#upload-using-workbench notextile.
-# Begin by installing the "Arvados Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html on the system from which you will upload the data (such as your workstation, or a server containing data from your sequencer). This will install the Arvados file upload tool, @arv-put@. Alternately, you can log into an Arvados VM (instructions for "Unix":{{site.baseurl}}/user/getting_started/ssh-access-unix.html#login or "Windows":{{site.baseurl}}/user/getting_started/ssh-access-windows.html#login). -# On system from which you will upload data, configure the environment with the Arvados instance host name and authentication token as decribed in "Getting an API token.":{{site.baseurl}}/user/reference/api-tokens.html (If you are logged into an Arvados VM, you can skip this step.) -# Download the following example file. (If you are uploading your own data, you can skip this step.) - -
~$ curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse.pgp-hms.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2'
-  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
-                                 Dload  Upload   Total   Spent    Left  Speed
-100  216M  100  216M    0     0  10.0M      0  0:00:21  0:00:21 --:--:-- 9361k
-
- -# Now upload the file to Keep using @arv-put@: +h2(#upload-using-command). Upload using command line tool + +{% include 'tutorial_expectations' %} + +To upload a file to Keep using @arv-put@: 
~$ arv-put var-GS000016015-ASM.tsv.bz2
 216M / 216M 100.0%
-c1bad4b39ca5a924e481008009d94e32+210
+Collection saved as ...
+qr1hi-4zz18-xxxxxxxxxxxxxxx
-* The output value @c1bad4b39ca5a924e481008009d94e32+210@ is the Arvados collection locator that uniquely describes this file. -Now visit the Workbench *Dashboard*. In the *My projects* pane, select your *Home* project. Your newly uploaded collection should appear near the top of the *Data collections* tab. The collection locator printed by @arv-put@ will appear under the *name* column. To move the collection to a different project, check the box at the left of the collection row. Pull down the *Selection...* menu near the top of the page tab, and select *Move selected*. +The output value @qr1hi-4zz18-xxxxxxxxxxxxxxx@ is the uuid of the Arvados collection created. -!{{ site.baseurl }}/images/workbench-move-selected.png! +Note: The file used in this example is a freely available TSV file containing variant annotations from the "Personal Genome Project (PGP)":http://www.pgp-hms.org participant "hu599905":https://my.pgp-hms.org/profile/hu599905), downloadable "here":https://warehouse.pgp-hms.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2. Alternatively, you can replace @var-GS000016015-ASM.tsv.bz2@ with the name of any file you have locally, or you could get the TSV file by "downloading it from Keep.":{{site.baseurl}}/user/tutorials/tutorial-keep-get.html -This will open a dialog box where you can select a destination project for the collection. Click a project, then finally the Move button. - -Click on the * Show* button next to the collection's listing on a project page to go to the Workbench page for your collection. On this page, you can see the collection's contents, download individual files, and set sharing options. - -notextile.
- -h2(#dir). Putting a directory - -If you give @arv-put@ a directory, it will recursively upload the entire directory: +It is also possible to upload an entire directory with @arv-put@: 
~$ mkdir tmp
@@ -49,6 +43,43 @@ If you give @arv-put@ a directory, it will recursively upload the entire directo
 ~$ echo "hello carol" > tmp/carol.txt
 ~$ arv-put tmp
 0M / 0M 100.0%
-887cd41e9c613463eab2f0d885c6dd96+83
+Collection saved as ...
+qr1hi-4zz18-yyyyyyyyyyyyyyy
+ +In both examples, the @arv-put@ command created a collection. The first collection contains the single uploaded file. The second collection contains the entire uploaded directory. + +@arv-put@ accepts quite a few optional command line arguments, which are described on the "arv subcommands":{{site.baseurl}}/sdk/cli/subcommands.html#arv-keep-put page. + +h3. Locate your collection in Workbench + +Visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu, select your *Home* project. Your newly uploaded collection should appear near the top of the *Data collections* tab. The collection name printed by @arv-put@ will appear under the *name* column. + +To move the collection to a different project, check the box at the left of the collection row. Pull down the *Selection...* menu near the top of the page tab, and select *Move selected...* button. This will open a dialog box where you can select a destination project for the collection. Click a project, then finally the Move button. + +!{display: block;margin-left: 25px;margin-right: auto;}{{ site.baseurl }}/images/workbench-move-selected.png! + +Click on the * Show* button next to the collection's listing on a project page to go to the Workbench page for your collection. On this page, you can see the collection's contents, download individual files, and set sharing options. + +notextile. + +h2(#upload-using-workbench). Upload using Workbench + +To upload using Workbench, visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu and select your *Home* project or any other project of your choosing. You will see the *Data collections* tab for this project, which lists the collections in this project. + +To upload files into a new collection, click on *Add data* dropdown menu and select *Upload files from my computer*. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-using-workbench.png! + +
This will create a new empty collection in your chosen project and will take you to the *Upload* tab for that collection. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-tab-in-new-collection.png! + +Click on the *Browse...* button and select the files you would like to upload. Selected files will be added to a list of files to be uploaded. After you are done selecting files to upload, click on the * Start* button to start upload. This will start uploading files to Arvados and Workbench will show you the progress bar. When upload is completed, you will see an indication to that effect. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/files-uploaded.png! + +*Note:* If you leave the collection page during the upload, the upload process will be aborted and you will need to upload the files again. + +*Note:* You can also use the Upload tab to add additional files to an existing collection.