X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/d6d7788c4e6b1d3da88833329b326fd7a3891503..5ff4dac7a9ee28eb35a796b80421a6d3d1e5ecd0:/doc/user/topics/keep.html.textile.liquid diff --git a/doc/user/topics/keep.html.textile.liquid b/doc/user/topics/keep.html.textile.liquid index 9d5cae5afe..68b6a87d09 100644 --- a/doc/user/topics/keep.html.textile.liquid +++ b/doc/user/topics/keep.html.textile.liquid @@ -3,6 +3,11 @@ layout: default navsection: userguide title: "How Keep works" ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. + +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} The Arvados distributed file system is called *Keep*. Keep is a content-addressable file system. This means that files are managed using special unique identifiers derived from the _contents_ of the file (specifically, the MD5 hash), rather than human-assigned file names. This has a number of advantages: * Files can be stored and replicated across a cluster of servers without requiring a central name server. @@ -14,20 +19,20 @@ In Keep, information is stored in *data blocks*. Data blocks are normally betwe In order to reassemble the file, Keep stores a *collection* data block which lists in sequence the data blocks that make up the original file. A collection data block may store the information for multiple files, including a directory structure. -In this example we will use @c1bad4b39ca5a924e481008009d94e32+210@, which we added to Keep in "how to upload data":{{ site.baseurl }}/user/tutorials/tutorial-keep.html. First let us examine the contents of this collection using @arv keep get@: +In this example we will use @c1bad4b39ca5a924e481008009d94e32+210@, which we added to Keep in "how to upload data":{{ site.baseurl }}/user/tutorials/tutorial-keep.html. First let us examine the contents of this collection using @arv-get@: -
~$ arv keep get c1bad4b39ca5a924e481008009d94e32+210
+
~$ arv-get c1bad4b39ca5a924e481008009d94e32+210
 . 204e43b8a1185621ca55a94839582e6f+67108864 b9677abbac956bd3e86b1deb28dfac03+67108864 fc15aff2a762b13f521baf042140acec+67108864 323d2a3ce20370c4ca1d3462a344f8fd+25885655 0:227212247:var-GS000016015-ASM.tsv.bz2
 
-The command @arv keep get@ fetches the contents of the collection @c1bad4b39ca5a924e481008009d94e32+210@. In this example, this collection includes a single file @var-GS000016015-ASM.tsv.bz2@ which is 227212247 bytes long, and is stored using four sequential data blocks, @204e43b8a1185621ca55a94839582e6f+67108864@, @b9677abbac956bd3e86b1deb28dfac03+67108864@, @fc15aff2a762b13f521baf042140acec+67108864@, and @323d2a3ce20370c4ca1d3462a344f8fd+25885655@. +The command @arv-get@ fetches the contents of the collection @c1bad4b39ca5a924e481008009d94e32+210@. In this example, this collection includes a single file @var-GS000016015-ASM.tsv.bz2@ which is 227212247 bytes long, and is stored using four sequential data blocks, @204e43b8a1185621ca55a94839582e6f+67108864@, @b9677abbac956bd3e86b1deb28dfac03+67108864@, @fc15aff2a762b13f521baf042140acec+67108864@, and @323d2a3ce20370c4ca1d3462a344f8fd+25885655@. -Let's use @arv keep get@ to download the first data block: +Let's use @arv-get@ to download the first data block: notextile.
~$ cd /scratch/you
-/scratch/you$ arv keep get 204e43b8a1185621ca55a94839582e6f+67108864 > block1
+/scratch/you$ arv-get 204e43b8a1185621ca55a94839582e6f+67108864 > block1
{% include 'notebox_begin' %} @@ -35,7 +40,7 @@ When you run this command, you may get this API warning: notextile.
WARNING:root:API lookup failed for collection 204e43b8a1185621ca55a94839582e6f+67108864 (<class 'apiclient.errors.HttpError'>: <HttpError 404 when requesting https://qr1hi.arvadosapi.com/arvados/v1/collections/204e43b8a1185621ca55a94839582e6f%2B67108864?alt=json returned "Not Found">)
-This happens because @arv keep get@ tries to find a collection with this identifier. When that fails, it emits this warning, then looks for a datablock instead, which succeeds. +This happens because @arv-get@ tries to find a collection with this identifier. When that fails, it emits this warning, then looks for a datablock instead, which succeeds. {% include 'notebox_end' %}