X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/0bd1c28bed9a0756c61037947d5a9dccd5066f00..5b3187552676947ee74e4b652e7a04d3d9b9a3a4:/doc/user/tutorials/tutorial-keep-mount.html.textile.liquid diff --git a/doc/user/tutorials/tutorial-keep-mount.html.textile.liquid b/doc/user/tutorials/tutorial-keep-mount.html.textile.liquid index e07be5d907..43af024df2 100644 --- a/doc/user/tutorials/tutorial-keep-mount.html.textile.liquid +++ b/doc/user/tutorials/tutorial-keep-mount.html.textile.liquid @@ -14,19 +14,43 @@ h2. Arv-mount * You can browse, open and read Keep entries as if they are regular files. * It is easy for existing tools to access files in Keep. -* Data is downloaded on demand. It is not necessary to download an entire file or collection to start processing. +* Data is streamed on demand. It is not necessary to download an entire file or collection to start processing. -The default mode permits browsing any collection in Arvados as a subdirectory under the mount directory. To avoid having to fetch a potentially large list of all collections, collection directories only come into existence when explicitly accessed by their keep locator. +The default mode permits browsing any collection in Arvados as a subdirectory under the mount directory. To avoid having to fetch a potentially large list of all collections, collection directories only come into existence when explicitly accessed by their Keep locator. For instance, a collection may be found by its content hash in the @keep/by_id@ directory.
~$ mkdir -p keep
 ~$ arv-mount keep
-~$ cd keep/c1bad4b39ca5a924e481008009d94e32+210
-~/keep/c1bad4b39ca5a924e481008009d94e32+210$ ls
+~$ cd keep/by_id/c1bad4b39ca5a924e481008009d94e32+210
+~/keep/by_id/c1bad4b39ca5a924e481008009d94e32+210$ ls
 var-GS000016015-ASM.tsv.bz2
-~/keep/c1bad4b39ca5a924e481008009d94e32+210$ md5sum var-GS000016015-ASM.tsv.bz2
+~/keep/by_id/c1bad4b39ca5a924e481008009d94e32+210$ md5sum var-GS000016015-ASM.tsv.bz2
 44b8ae3fde7a8a88d2f7ebd237625b4f  var-GS000016015-ASM.tsv.bz2
-~/keep/c1bad4b39ca5a924e481008009d94e32+210$ cd ../..
+~/keep/by_id/c1bad4b39ca5a924e481008009d94e32+210$ cd ../..
 ~$ fusermount -u keep
 
+ +The last line unmounts Keep. Subdirectories will no longer be accessible. + +Within each directory on Keep, there is a @.arvados#collection@ file that does not show up with @ls@. Its contents include, for instance, the @portable_data_hash@, which is the same as the Keep locator. + +h3. Modifying files and directories in Keep + +By default, all files in the Keep mount are read only. However, @arv-mount --enable-write@ enables you to perform the following operations using normal Unix command line tools (@touch@, @mv@, @rm@, @mkdir@, @rmdir@) and your own programs using standard POSIX file system APIs: + +* Create, update, rename and delete individual files within collections +* Create and delete subdirectories inside collections +* Move files and directories within and between collections +* Create and delete collections within a project (using @mkdir@ and @rmdir@ in a project directory) + +Not supported: + +* Symlinks, hard links +* Changing permissions +* Extended attributes +* Moving a subdirectory of a collection into a project, or moving a collection from a project into another collection + +If multiple clients (separate instances of arv-mount or other arvados applications) modify the same file in the same collection within a short time interval, this may result in a conflict. In this case, the most recent commit wins, and the "loser" will be renamed to a conflict file in the form @name~YYYYMMDD-HHMMSS~conflict~@. + +Please note this feature is in beta testing. In particular, the conflict mechanism is itself currently subject to race condiditions with potential for data loss when a collection is being modified simultaneously by multiple clients. This issue will be resolved in future development.