From 33b27555b25e70a0d37c5a8267450fddf3bd81b1 Mon Sep 17 00:00:00 2001 From: Peter Amstutz Date: Wed, 19 Aug 2020 17:29:19 -0400 Subject: [PATCH] 15964: Update and reorganize keep & CWL user guide docs Arvados-DCO-1.1-Signed-off-by: Peter Amstutz --- doc/user/cwl/bwa-mem/bwa-mem.cwl | 4 +- .../workbench.html.textile.liquid | 2 +- doc/user/index.html.textile.liquid | 2 +- doc/user/topics/arv-copy.html.textile.liquid | 64 ++++++++++--------- ...p-collection-lifecycle.html.textile.liquid | 40 ++++++------ .../tutorial-keep-get.html.textile.liquid | 60 +++++++++-------- ...l-keep-mount-gnu-linux.html.textile.liquid | 4 +- ...torial-keep-mount-os-x.html.textile.liquid | 4 +- ...ial-keep-mount-windows.html.textile.liquid | 2 +- .../tutorial-keep.html.textile.liquid | 55 +++++++++------- 10 files changed, 128 insertions(+), 109 deletions(-) diff --git a/doc/user/cwl/bwa-mem/bwa-mem.cwl b/doc/user/cwl/bwa-mem/bwa-mem.cwl index a73329288c..018867c83e 100755 --- a/doc/user/cwl/bwa-mem/bwa-mem.cwl +++ b/doc/user/cwl/bwa-mem/bwa-mem.cwl @@ -10,11 +10,11 @@ hints: DockerRequirement: dockerPull: quay.io/biocontainers/bwa:0.7.17--ha92aebf_3 -baseCommand: [mem] +baseCommand: [bwa, mem] arguments: - {prefix: "-t", valueFrom: $(runtime.cores)} - - {prefix: "-R", valueFrom: "@RG\tID:$(inputs.group_id)\tPL:$(inputs.PL)\tSM:$(inputs.sample_id)"} + - {prefix: "-R", valueFrom: '@RG\\\tID:$(inputs.group_id)\\\tPL:$(inputs.PL)\\\tSM:$(inputs.sample_id)'} inputs: reference: diff --git a/doc/user/getting_started/workbench.html.textile.liquid b/doc/user/getting_started/workbench.html.textile.liquid index 52f1931c96..12b981148d 100644 --- a/doc/user/getting_started/workbench.html.textile.liquid +++ b/doc/user/getting_started/workbench.html.textile.liquid @@ -23,6 +23,6 @@ If you are using the "playground" Arvados instance for this guide, you can Acces You will be asked to log in. Arvados uses only your name and email address for identification, and will never access any personal information. If you are accessing Arvados for the first time, the Workbench may indicate your account status is *New / inactive*. If this is the case, contact the administrator of the Arvados instance to request activation of your account. -Once your account is active, logging in to the Workbench will present you with the Dashboard. This gives a summary of your projects and recent activity in the Arvados instance. "You are now ready to run your first pipeline.":{{ site.baseurl }}/user/tutorials/tutorial-workflow-workbench.html +Once your account is active, logging in to the Workbench will present you with the Dashboard. This gives a summary of your projects and recent activity in the Arvados instance. You are now ready to "upload data":{{ site.baseurl }}/user/tutorials/tutorial-keep.html or "run your first pipeline.":{{ site.baseurl }}/user/tutorials/tutorial-workflow-workbench.html !{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/workbench-dashboard.png! diff --git a/doc/user/index.html.textile.liquid b/doc/user/index.html.textile.liquid index 909394ef47..9749d1f284 100644 --- a/doc/user/index.html.textile.liquid +++ b/doc/user/index.html.textile.liquid @@ -9,7 +9,7 @@ Copyright (C) The Arvados Authors. All rights reserved. SPDX-License-Identifier: CC-BY-SA-3.0 {% endcomment %} -This guide provides a reference for using Arvados to solve scientific big data problems, including: +Arvados is an open source platform for managing, processing, and sharing genomic and other large scientific and biomedical data. This guide provides a reference for using Arvados to solve scientific big data problems, including: * Robust storage of very large files, such as whole genome sequences, using the "Arvados Keep":{{site.baseurl}}/user/tutorials/tutorial-keep.html content-addressable cluster file system. * Running compute-intensive scientific analysis pipelines, such as genomic alignment and variant calls using the "Arvados Crunch":{{site.baseurl}}/user/tutorials/intro-crunch.html cluster compute engine. diff --git a/doc/user/topics/arv-copy.html.textile.liquid b/doc/user/topics/arv-copy.html.textile.liquid index e5e631d4e2..d35df4fcec 100644 --- a/doc/user/topics/arv-copy.html.textile.liquid +++ b/doc/user/topics/arv-copy.html.textile.liquid @@ -17,64 +17,66 @@ h2. arv-copy @arv-copy@ allows users to copy collections and workflows from one cluster to another. By default, @arv-copy@ will recursively go through the workflow and copy all dependencies associated with the object. -For example, let's copy from the Arvados playground, also known as *pirca*, to *dstcl*. The names *pirca* and *dstcl* are interchangable with any cluster id. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *zzzzz*-4zz18-tci4vn4fa95w0zx, the cluster name is zzzzz. +For example, let's copy from the Arvados playground, also known as *pirca*, to *dstcl*. The names *pirca* and *dstcl* are interchangable with any cluster id. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *zzzzz*-4zz18-tci4vn4fa95w0zx, the cluster name is *zzzzz* . -In order to communicate with both clusters, you must create custom configuration files for each cluster. In the Arvados Workbench, click on the dropdown menu icon in the upper right corner of the top navigation menu to access the user settings menu, and click on the menu item *Current token*. Copy the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ in both of your clusters. Then, create two configuration files, one for each cluster. The names of the files must have the format of *ClusterID.conf*. In our example, let's make two files, one for *pirca* and one for *dstcl*. From your *Current token* page in *pirca* and *dstcl*, copy the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@. +In order to communicate with both clusters, you must create custom configuration files for each cluster. In the Arvados Workbench, click on the dropdown menu icon in the upper right corner of the top navigation menu to access the user settings menu, and click on the menu item *Current token*. Copy the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ in both of your clusters. Then, create two configuration files in @~/.config/arvados@, one for each cluster. The names of the files must have the format of *ClusterID.conf*. Navigate to the *Current token* page on each of *pirca* and *dstcl* to get the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@. !{display: block;margin-left: 25px;margin-right: auto;}{{ site.baseurl }}/images/api-token-host.png! -Copy your @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ into the config files as shown below in the shell account from which you are executing the commands. For example, the default shell you may have access to is shell.pirca. You can add these files in ~/.config/arvados/ in the pirca shell terminal. +The config file consists of two lines, one for ARVADOS_API_HOST and one for ARVADOS_API_TOKEN: - -
~$ cd ~/.config/arvados
-~$ echo "ARVADOS_API_HOST=pirca.arvadosapi.com" >> pirca.conf
-~$ echo "ARVADOS_API_TOKEN=123456789abcdefghijkl" >> pirca.conf
-~$ echo "ARVADOS_API_HOST=dstcl.arvadosapi.com" >> dstcl.conf
-~$ echo "ARVADOS_API_TOKEN=987654321lkjihgfedcba" >> dstcl.conf
-
-
+
+ARVADOS_API_HOST=zzzzz.arvadosapi.com
+ARVADOS_API_TOKEN=v2/zzzzz-gj3su-xxxxxxxxxxxxxxx/123456789abcdefghijkl
+
+ +Copy your @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ into the config files as shown below in the shell account from which you are executing the commands. In our example, you need two files, @~/.config/arvados/pirca.conf@ and @~/.config/arvados/dstcl.conf@. Now you're ready to copy between *pirca* and *dstcl*! h3. How to copy a collection -First, select the uuid of the collection you want to copy from the source cluster. The uuid can be found in the collection display page in the collection summary area (top left box), or from the URL bar (the part after @collections/...@) +First, determine the uuid or portable data hash of the collection you want to copy from the source cluster. The uuid can be found in the collection display page in the collection summary area (top left box), or from the URL bar (the part after @collections/...@) + +Now copy the collection from *pirca* to *dstcl*. We will use the uuid @jutro-4zz18-tv416l321i4r01e@ as an example. You can find this collection on playground.arvados.org. + +
~$ arv-copy --src pirca --dst dstcl jutro-4zz18-tv416l321i4r01e
+jutro-4zz18-tv416l321i4r01e: 6.1M / 6.1M 100.0%
+arvados.arv-copy[1234] INFO: Success: created copy with uuid dstcl-4zz18-xxxxxxxxxxxxxxx
+
+
+ +You can also copy by content address: -Now copy the collection from *pirca* to *dstcl*. We will use the uuid @pirca-4zz18-jyjuzyqq14qspl7@ as an example. You can find this collection in the User guide resources project on playground.arvados.org. -
~$ arv-copy --src pirca --dst dstcl pirca-4zz18-jyjuzyqq14qspl7
-pirca-4zz18-tci4vn4fa95w0zx: 6.1M / 6.1M 100.0%
-arvados.arv-copy[1234] INFO: Success: created copy with uuid dstcl-4zz18-8765943210cdbae
+
~$ arv-copy --src pirca --dst dstcl 2463fa9efeb75e099685528b3b9071e0+438
+2463fa9efeb75e099685528b3b9071e0+438: 6.1M / 6.1M 100.0%
+arvados.arv-copy[1234] INFO: Success: created copy with uuid dstcl-4zz18-xxxxxxxxxxxxxxx
 
-The output of arv-copy displays the uuid of the collection generated in the destination cluster. By default, the output is placed in your home project in the destination cluster. If you want to place your collection in a pre-created project, you can specify the project you want it to be in using the tag @--project-uuid@ followed by the project uuid. +The output of arv-copy displays the uuid of the collection generated in the destination cluster. By default, the output is placed in your home project in the destination cluster. If you want to place your collection in an existing project, you can specify the project you want it to be in using the tag @--project-uuid@ followed by the project uuid. For example, this will copy the collection to project dstcl-j7d0g-a894213ukjhal12 in the destination cluster. -
~$ arv-copy --src pirca --dst dstcl --project-uuid dstcl-j7d0g-a894213ukjhal12 pirca-4zz18-tci4vn4fa95w0zx
+ 
~$ arv-copy --src pirca --dst dstcl --project-uuid dstcl-j7d0g-a894213ukjhal12 jutro-4zz18-tv416l321i4r01e
 
h3. How to copy a workflow -We will use the uuid @pirca-7fd4e-sampleworkflow1@ as an example workflow. +We will use the uuid @jutro-7fd4e-mkmmq53m1ze6apx@ as an example workflow. -
~$ arv-copy --src pirca --dst dstcl --dst-git-repo $USER/tutorial pirca-7fd4e-sampleworkflow1
-pirca-4zz18-jidprdejysravcr: 1143M / 1143M 100.0%
-2017-01-04 04:11:58 arvados.arv-copy[5906] INFO:
-2017-01-04 04:11:58 arvados.arv-copy[5906] INFO: Success: created copy with uuid dstcl-7fd4e-ojtgpne594ubkt7
+
~$ arv-copy --src jutro --dst pirca --project-uuid pirca-j7d0g-ecak8knpefz8ere jutro-7fd4e-mkmmq53m1ze6apx
+ae480c5099b81e17267b7445e35b4bc7+180: 23M / 23M 100.0%
+2463fa9efeb75e099685528b3b9071e0+438: 156M / 156M 100.0%
+jutro-4zz18-vvvqlops0a0kpdl: 94M / 94M 100.0%
+2020-08-19 17:04:13 arvados.arv-copy[4789] INFO:
+2020-08-19 17:04:13 arvados.arv-copy[4789] INFO: Success: created copy with uuid pirca-7fd4e-s0tw9rfbkpo2fmx
 
-The name, description, and workflow definition from the original workflow will be used for the destination copy. In addition, any *locations* and *docker images* found in the src workflow definition will also be copied to the destination recursively. +The name, description, and workflow definition from the original workflow will be used for the destination copy. In addition, any *collections* and *docker images* referenced in the source workflow definition will also be copied to the destination. If you would like to copy the object without dependencies, you can use the @--no-recursive@ flag. - -For example, we can copy the same object non-recursively using the following: - - -
~$ arv-copy --src pirca --dst dstcl --dst-git-repo $USER/tutorial --no-recursive pirca-7fd4e-sampleworkflow1
-
-
diff --git a/doc/user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid b/doc/user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid index 180e087824..c4b9e31ece 100644 --- a/doc/user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid @@ -11,45 +11,42 @@ SPDX-License-Identifier: CC-BY-SA-3.0 During it's lifetime, a keep collection can be in various states. These states are *persisted*, *expiring*, *trashed* and *permanently deleted*. +The nominal state is *persisted* which means the data can be can be accessed normally and will be retained indefinitely. + A collection is *expiring* when it has a *trash_at* time in the future. An expiring collection can be accessed as normal, but is scheduled to be trashed automatically at the *trash_at* time. A collection is *trashed* when it has a *trash_at* time in the past. The *is_trashed* attribute will also be "true". The delete operation immediately puts the collection in the trash by setting the *trash_at* time to "now". Once trashed, the collection is no longer readable through normal data access APIs. The collection will have *delete_at* set to some time in the future. The trashed collection is recoverable until the delete_at time passes, at which point the collection is permanently deleted. -# "*Collection lifecycle attributes*":#collection_attributes # "*Deleting / trashing collections*":#delete-collection # "*Recovering trashed collections*":#trash-recovery +# "*Collection lifecycle attributes*":#collection_attributes {% include 'tutorial_expectations' %} -h2(#collection_attributes). Collection lifecycle attributes - -As listed above the attributes that are used to manage a collection lifecycle are it's *is_trashed*, *trash_at*, and *delete_at*. The table below lists the values of these attributes and how they influence the state of a collection and it's accessibility. - -table(table table-bordered table-condensed). -|_. collection state|_. is_trashed|_. trash_at|_. delete_at|_. get|_. list|_. list?include_trash=true|_. can be modified| -|persisted collection|false |null |null |yes |yes |yes |yes | -|expiring collection|false |future |future |yes |yes |yes |yes | -|trashed collection|true |past |future |no |no |yes |only is_trashed, trash_at and delete_at attribtues| -|deleted collection|true|past |past |no |no |no |no | - h2(#delete-collection). Deleting / trashing collections A collection can be deleted using either the arv command line tool or the workbench. +h3. Trashing a collection using workbench + +To trash a collection using workbench, go to the Data collections tab in the project, and use the trash icon for this collection row. + h3. Trashing a collection using arv command line tool
 arv collection delete --uuid=zzzzz-4zz18-xxxxxxxxxxxxxxx
 
-h3. Trashing a collection using workbench - -To trash a collection using workbench, go to the Data collections tab in the project, and use the trash icon for this collection row. - h2(#trash-recovery). Recovering trashed collections A collection can be un-trashed / recovered using either the arv command line tool or the workbench. +h3. Un-trashing a collection using workbench + +To untrash a collection using workbench, go to trash page on workbench by clicking on the "Trash" icon in the top navigation in workbench and use the recycle icon or selection dropdown option. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/trash-button-topnav.png! + h3. Un-trashing a collection using arv command line tool You can list the trashed collections using the list command. @@ -64,8 +61,13 @@ You can then untrash a particular collection using arv using it's uuid. arv collection untrash --uuid=zzzzz-4zz18-xxxxxxxxxxxxxxx
-h3. Un-trashing a collection using workbench +h2(#collection_attributes). Collection lifecycle attributes -To untrash a collection using workbench, go to trash page on workbench by clicking on the "Trash" icon in the top navigation in workbench and use the recycle icon or selection dropdown option. +As listed above the attributes that are used to manage a collection lifecycle are it's *is_trashed*, *trash_at*, and *delete_at*. The table below lists the values of these attributes and how they influence the state of a collection and it's accessibility. -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/trash-button-topnav.png! +table(table table-bordered table-condensed). +|_. collection state|_. is_trashed|_. trash_at|_. delete_at|_. get|_. list|_. list?include_trash=true|_. can be modified| +|persisted collection|false |null |null |yes |yes |yes |yes | +|expiring collection|false |future |future |yes |yes |yes |yes | +|trashed collection|true |past |future |no |no |yes |only is_trashed, trash_at and delete_at attribtues| +|deleted collection|true|past |past |no |no |no |no | diff --git a/doc/user/tutorials/tutorial-keep-get.html.textile.liquid b/doc/user/tutorials/tutorial-keep-get.html.textile.liquid index c73675bf11..acf4b33c17 100644 --- a/doc/user/tutorials/tutorial-keep-get.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-get.html.textile.liquid @@ -11,11 +11,39 @@ SPDX-License-Identifier: CC-BY-SA-3.0 Arvados Data collections can be downloaded using either the arv commands or using Workbench. -# "*Downloading using arv commands*":#download-using-arv -# "*Downloading using Workbench*":#download-using-workbench -# "*Downloading a shared collection using Workbench*":#download-shared-collection +# "*Download using Workbench*":#download-using-workbench +# "*Sharing collections*":#download-shared-collection +# "*Download using command line tools*":#download-using-arv -h2(#download-using-arv). Downloading using arv commands +h2(#download-using-workbench). Download using Workbench + +You can also download Arvados data collections using the Workbench. + +Visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu, select your *Home* project. You will see the *Data collections* tab, which lists the collections in this project. + +You can access the contents of a collection by clicking on the * Show* button next to the collection. This will take you to the collection's page. Using this page you can see the collection's contents, and download individual files. + +You can now download the collection files by clicking on the button(s). + +h2(#download-shared-collection). Sharing collections + +h3. Sharing with other Arvados users + +Collections can be shared with other users on the Arvados cluster by sharing the parent project. Navigate to the parent project using the "breadcrumbs" bar, then click on the *Sharing* tab. From the sharing tab, you can choose which users or groups to share with, and their level of access. + +h3. Creating a special download URL + +To share a collection with users that do not have an account on your Arvados cluster, visit the collection page using Workbench as described in the above section. Once on this page, click on the Create sharing link button. + +This will create a sharing link for the collection as shown below. You can copy the sharing link in this page and share it with other users. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/shared-collection.png! + +A user with this url can download this collection by simply accessing this url using browser. It will present a downloadable version of the collection as shown below. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/download-shared-collection.png! + +h2(#download-using-arv). Download using command line tools {% include 'tutorial_expectations' %} @@ -71,27 +99,3 @@ You may also request a collection by UUID. In this case, it will contact the cl
~$ arv-get zzzzz-4zz18-fw6dnjxtkvzdewt/ .
 
- -h2(#download-using-workbench). Downloading using Workbench - -You can also download Arvados data collections using the Workbench. - -Visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu, select your *Home* project. You will see the *Data collections* tab, which lists the collections in this project. - -You can access the contents of a collection by clicking on the * Show* button next to the collection. This will take you to the collection's page. Using this page you can see the collection's contents, download individual files, and set sharing options. - -You can now download the collection files by clicking on the button(s). - -h2(#download-shared-collection). Downloading a shared collection using Workbench - -Collections can be shared to allow downloads by anonymous users. - -To share a collection with anonymous users, visit the collection page using Workbench as described in the above section. Once on this page, click on the Create sharing link button. - -This will create a sharing link for the collection as shown below. You can copy the sharing link in this page and share it with other users. - -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/shared-collection.png! - -A user with this url can download this collection by simply accessing this url using browser. It will present a downloadable version of the collection as shown below. - -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/download-shared-collection.png! diff --git a/doc/user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid b/doc/user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid index e176021992..eb2a91b757 100644 --- a/doc/user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid @@ -1,7 +1,7 @@ --- layout: default navsection: userguide -title: "Accessing Keep from GNU/Linux" +title: "Keep network mount on GNU/Linux" ... {% comment %} Copyright (C) The Arvados Authors. All rights reserved. @@ -9,7 +9,7 @@ Copyright (C) The Arvados Authors. All rights reserved. SPDX-License-Identifier: CC-BY-SA-3.0 {% endcomment %} -This tutoral describes how to access Arvados collections on GNU/Linux using traditional filesystem tools by mounting Keep as a file system using @arv-mount@. +GNU/Linux users can use @arv-mount@ to mount Keep as a file system in order to access Arvados collections using traditional filesystem tools. {% include 'tutorial_expectations' %} diff --git a/doc/user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid b/doc/user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid index 9397d61e05..736952b5cb 100644 --- a/doc/user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid @@ -1,7 +1,7 @@ --- layout: default navsection: userguide -title: "Accessing Keep from OS X" +title: "Keep network mount on OS X" ... {% comment %} Copyright (C) The Arvados Authors. All rights reserved. @@ -19,6 +19,6 @@ This mount is read-only. Write support for the @/users/@ directory is planned fo h3. Accessing a specific collection in Keep (read-write) -In Finder, use "Connect to Server..." under the "Go" menu and enter @https://collections.ClusterID.example.com/@ in popup dialog. When prompted for credentials, put a valid Arvados token in the @Password@ field and anything in the Name field (it will be ignored by Arvados). +In Finder, use "Connect to Server..." under the "Go" menu and enter @https://collections.ClusterID.example.com/c=your-collection-uuid@ in popup dialog. When prompted for credentials, put a valid Arvados token in the @Password@ field and anything in the Name field (it will be ignored by Arvados). This collection is now accessible read/write. diff --git a/doc/user/tutorials/tutorial-keep-mount-windows.html.textile.liquid b/doc/user/tutorials/tutorial-keep-mount-windows.html.textile.liquid index 29b28fff9c..0818f632f8 100644 --- a/doc/user/tutorials/tutorial-keep-mount-windows.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-mount-windows.html.textile.liquid @@ -1,7 +1,7 @@ --- layout: default navsection: userguide -title: "Accessing Keep from Windows" +title: "Keep network mount on Windows" ... {% comment %} Copyright (C) The Arvados Authors. All rights reserved. diff --git a/doc/user/tutorials/tutorial-keep.html.textile.liquid b/doc/user/tutorials/tutorial-keep.html.textile.liquid index 5dd87c2633..21efc475c5 100644 --- a/doc/user/tutorials/tutorial-keep.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep.html.textile.liquid @@ -9,13 +9,44 @@ Copyright (C) The Arvados Authors. All rights reserved. SPDX-License-Identifier: CC-BY-SA-3.0 {% endcomment %} -Arvados Data collections can be uploaded using either the @arv-put@ command line tool or using Workbench. +Arvados Data collections can be uploaded using either Workbench or the @arv-put@ command line tool. -# "*Upload using command line tool*":#upload-using-command # "*Upload using Workbench*":#upload-using-workbench +# "*Creating projects*":#creating-projects +# "*Upload using command line tool*":#upload-using-command + +h2(#upload-using-workbench). Upload using Workbench + +To upload using Workbench, visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu and select your *Home* project or any other project of your choosing. You will see the *Data collections* tab for this project, which lists the collections in this project. + +To upload files into a new collection, click on *Add data* dropdown menu and select *Upload files from my computer*. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-using-workbench.png! + +
This will create a new empty collection in your chosen project and will take you to the *Upload* tab for that collection. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-tab-in-new-collection.png! + +Click on the *Browse...* button and select the files you would like to upload. Selected files will be added to a list of files to be uploaded. After you are done selecting files to upload, click on the * Start* button to start upload. This will start uploading files to Arvados and Workbench will show you the progress bar. When upload is completed, you will see an indication to that effect. + +!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/files-uploaded.png! + +*Note:* If you leave the collection page during the upload, the upload process will be aborted and you will need to upload the files again. + +*Note:* You can also use the Upload tab to add additional files to an existing collection. notextile.
+h2(#creating-projects). Creating projects + +Files are organized into Collections, and Collections are organized by Projects. + +Click on *Projects* *Add a new project* to add a top level project. + +To create a subproject, navigate to the parent project, and click on *Add a subproject*. + +See "Sharing collections":tutorial-keep-get.html#download-shared-collection for information about sharing projects and collections with other users. + h2(#upload-using-command). Upload using command line tool {% include 'tutorial_expectations' %} @@ -63,23 +94,3 @@ To move the collection to a different project, check the box at the left of the Click on the * Show* button next to the collection's listing on a project page to go to the Workbench page for your collection. On this page, you can see the collection's contents, download individual files, and set sharing options. notextile.
- -h2(#upload-using-workbench). Upload using Workbench - -To upload using Workbench, visit the Workbench *Dashboard*. Click on *Projects* dropdown menu in the top navigation menu and select your *Home* project or any other project of your choosing. You will see the *Data collections* tab for this project, which lists the collections in this project. - -To upload files into a new collection, click on *Add data* dropdown menu and select *Upload files from my computer*. - -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-using-workbench.png! - -
This will create a new empty collection in your chosen project and will take you to the *Upload* tab for that collection. - -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/upload-tab-in-new-collection.png! - -Click on the *Browse...* button and select the files you would like to upload. Selected files will be added to a list of files to be uploaded. After you are done selecting files to upload, click on the * Start* button to start upload. This will start uploading files to Arvados and Workbench will show you the progress bar. When upload is completed, you will see an indication to that effect. - -!{display: block;margin-left: 25px;margin-right: auto;border:1px solid lightgray;}{{ site.baseurl }}/images/files-uploaded.png! - -*Note:* If you leave the collection page during the upload, the upload process will be aborted and you will need to upload the files again. - -*Note:* You can also use the Upload tab to add additional files to an existing collection. -- 2.30.2