21700: Install Bundler system-wide in Rails postinst

[arvados.git] / doc / sdk / fuse / options.html.textile.liquid

---
layout: default
navsection: sdk
navmenu: FUSE Driver
title: arv-mount options
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.

SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}

This page documents all available @arv-mount@ options with some usage examples.

# "Mount contents":#contents
# "Mount custom layout and filtering":#layout
## "@--filters@ usage and limitations":#filters
# "Mount access and permissions":#access
# "Mount lifecycle management":#lifecycle
# "Mount logging and statistics":#logging
# "Mount local cache setup":#cache
# "Mount interactions with Arvados and Linux":#plumbing
# "Examples":#examples
## "Using @--exec@":#exec
## "Running arv-mount as a systemd service":#systemd

h2(#contents). Mount contents

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--all@|Mount a subdirectory for each mode: @home@, @shared@, @by_id@, and @by_tag@ (default if no @--mount-*@ options are given)|
|@--custom@|Mount a subdirectory for each mode specified by a @--mount-*@ option (default if any @--mount-*@ options are given; see "Mount custom layout and filtering":#layout section)|
|@--collection UUID_OR_PDH@|Mount the specified collection|
|@--home@|Mount your home project|
|@--project UUID@|Mount the specified project|
|@--shared@|Mount a subdirectory for each project shared with you|
|@--by-id@|Mount a magic directory where collections and projects are accessible through subdirectories named after their UUID or portable data hash|
|@--by-pdh@|Mount a magic directory where collections are accessible through subdirectories named after their portable data hash|
|@--by-tag@|Mount a subdirectory for each tag attached to a collection or project|

h2(#layout). Mount custom layout and filtering

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--filters FILTERS@|Filters to apply to all project, shared, and tag directory contents. Pass filters as either a JSON string or a path to a JSON file. The JSON object should be a list of filters in "Arvados API list filter syntax":{{ site.baseurl }}/api/methods.html#filters. See the "example filters":#filters.|
|@--mount-home PATH@|Make your home project available under the mount at @PATH@|
|@--mount-shared PATH@|Make projects shared with you available under the mount at @PATH@|
|@--mount-tmp PATH@|Make a new temporary writable collection available under the mount at @PATH@. This collection is deleted when the mount is unmounted.|
|@--mount-by-id PATH@|Make a magic directory available under the mount at @PATH@ where collections and projects are accessible through subdirectories named after their UUID or portable data hash|
|@--mount-by-pdh PATH@|Make a magic directory available under the mount at @PATH@ where collections are accessible through subdirectories named after portable data hash|
|@--mount-by-tag PATH@|Make a subdirectory for each tag attached to a collection or project available under the mount at @PATH@|

h3(#filters). @--filters@ usage and limitations

Your argument to @--filters@ should be a JSON list of filters in "Arvados API list filter syntax":{{ site.baseurl }}/api/methods.html#filters. If your filter checks any field besides @uuid@, you should prefix it with the @<resource type>.@ Taken together, here's an example that mounts your home directory excluding filter groups, workflow intermediate output collections, and workflow log collections:

<notextile>
<pre><code>$ arv-mount --home <span class="userinput">--filters '[["groups.group_class", "!=", "filter"], ["collections.properties.type", "not in", ["intermediate", "log"]]]'</span> ...
</code></pre>
</notextile>

Because filters can be awkward to write on the command line, you can also write them in a file, and pass that file path to the @--filters@ option. This example does the same filtering:

<notextile>
<pre><code>$ <span class="userinput">cat &gt;~/arv-mount-filters.json &lt;&lt;EOF
[
  [
    "groups.group_class",
    "!=",
    "filter"
  ],
  [
    "collections.properties.type",
    "not in",
    [
      "intermediate",
      "log"
    ]
  ]
]
EOF</span>
$ arv-mount --home <span class="userinput">--filters ~/arv-mount-filters.json</span> ...
</code></pre>
</notextile>

The current implementation of @--filters@ has a few limitations. These may be lifted in a future release:

* You can always access any project or collection by UUID or portable data hash under a magic directory. If you access a project this way, your filters _will_ apply to the project contents.
* Tag directory listings are generated by querying tags alone. Only filters that apply to @links@ will affect these listings.

h2(#access). Mount access and permissions

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--allow-other@|Let other users on this system read mounted data (default false)|
|@--read-only@|Mounted data cannot be modified from the mount (default)|
|@--read-write@|Mounted data can be modified from the mount|

h2(#lifecycle). Mount lifecycle management

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--exec ...@|Mount data, run the specified command, then unmount and exit. @--exec@ reads all remaining options as the command to run, so it must be the last option you specify. Either end your command arguments (and other options) with a @--@ argument, or specify @--exec@ after your mount point.|
|@--foreground@|Run mount process in the foreground instead of daemonizing (default false)|
|@--subtype SUBTYPE@|Set mounted filesystem type to @fuse.SUBTYPE@ (default is just @fuse@)|
|@--replace@|If a FUSE mount is already mounted at the given directory, unmount it before mounting the requested data. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
|@--unmount@|If a FUSE mount is already mounted at the given directory, unmount it and exit. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
|@--unmount-all@|Unmount all FUSE mounts at or below the given directory, then exit. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
|@--unmount-timeout SECONDS@|The number of seconds to wait for a clean unmount after an @--exec@ command has exited (default 2.0). After this time, the mount will be forcefully unmounted.|

h2(#logging). Mount logging and statistics

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--crunchstat-interval SECONDS@|Write stats to stderr every N seconds (default disabled)|
|@--debug@|Log debug information|
|@--logfile LOGFILE@|Write debug logs and errors to the specified file (default stderr)|

h2(#cache). Mount local cache setup

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--disk-cache@|Cache data on the local filesystem (default)|
|@--ram-cache@|Cache data in memory|
|@--disk-cache-dir DIRECTORY@|Filesystem cache location (default @~/.cache/arvados/keep@)|
|@--directory-cache BYTES@|Size of directory data cache in bytes (default 128 MiB)|
|@--file-cache BYTES@|Size of file data cache in bytes (default 8 GiB for filesystem cache, 256 MiB for memory cache)|

h2(#plumbing). Mount interactions with Arvados and Linux

table(table table-bordered table-condensed).
|_. Option(s)|_. Description|
|@--disable-event-listening@|Don't subscribe to events on the API server to update mount contents|
|@--encoding ENCODING@|Filesystem character encoding (default 'utf-8'; specify a name from the "Python codec registry":https://docs.python.org/3/library/codecs.html#standard-encodings)|
|@--retries RETRIES@|Maximum number of times to retry server requests that encounter temporary failures (e.g., server down). Default 10.|
|@--storage-classes CLASSES@|Comma-separated list of storage classes to request for new collections|

h2(#examples). Examples

h3(#exec). Using @--exec@

There are a couple of details that are important to understand when you use @--exec@:

* @--exec@ reads all remaining options as the command to run, so it must be the last option you specify. Either end your command arguments (and other options) with a @--@ argument, or specify @--exec@ after your mount point.
* The command you specify runs from the same directory that you started @arv-mount@ from. To access data inside the mount, you will generally need to pass the path to the mount as an argument.

For example, this generates a recursive listing of all the projects and collections under your home project:

<notextile>
<pre><code>$ <span class="userinput">arv-mount --home --exec find -type d ArvadosHome -- ArvadosHome</span>
</code></pre>
</notextile>

The first @ArvadosHome@ is a path argument to @find@. The second is the mount point argument to @arv-mount@.

h3(#systemd). Running arv-mount as a systemd service

If you want to run @arv-mount@ as a long-running service, it's easy to write a systemd service definition for it. We do not publish one because the entire definition tends to be site-specific, but you can start from this template. You must change the @ExecStart@ path. Comments detail other changes you might want to make.

<notextile>
<pre><code>[Unit]
Description=Arvados FUSE mount
Documentation={{ site.baseurl }}/sdk/fuse/options.html

[Service]
Type=simple
CacheDirectory=arvados/keep
CacheDirectoryMode=0700

# This unit makes the mount available as `Arvados` under the runtime directory root.
# If this is a system service installed under /etc/systemd/system,
# the mount will be at /run/Arvados.
# If this is a user service installed under ~/.config/systemd/user,
# the mount will be at $XDG_RUNTIME_DIR/Arvados.
# If you want to mount at another location on the filesystem, remove RuntimeDirectory
# and replace both instances of %t/Arvados with your desired path.
RuntimeDirectory=Arvados
# The arv-mount path must be the absolute path where you installed the command.
# If you installed from a distribution package, make this /usr/bin/arv-mount.
# If you installed from pip, replace ... with the path to your virtualenv.
# You can add options to select what gets mounted, access permissions,
# cache size, log level, etc.
ExecStart=<span class="userinput">...</span>/bin/arv-mount --foreground --disk-cache-dir %C/arvados/keep %t/Arvados
ExecStop=/usr/bin/fusermount -u %t/Arvados

# This unit assumes the running user has a ~/.config/arvados/settings.conf
# with ARVADOS_API_HOST and ARVADOS_API_TOKEN defined.
# If not, you can write those in a separate file
# and set its path as EnvironmentFile.
# Make sure that file is owned and only readable by the running user (mode 0600).
#EnvironmentFile=...
</code></pre>
</notextile>