5 title: "arv subcommands"
9 Copyright (C) The Arvados Authors. All rights reserved.
11 SPDX-License-Identifier: CC-BY-SA-3.0
14 _In order to use the @arv@ command, make sure that you have a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html_
16 h3(#arv-create). arv create
18 @arv create@ can be used to create Arvados objects from the command line. Arv create opens up the editor of your choice (set the EDITOR environment variable) and allows you to type or paste a json or yaml description. When saved the object will be created on the API server, if it passes validation.
22 $ <code class="userinput">arv create --help</code>
24 --project-uuid, -p <s>: Project uuid in which to create the object
25 --help, -h: Show this message
31 @arv get@ can be used to get a textual representation of Arvados objects from the command line. The output can be limited to a subset of the object's fields. This command can be used with only the knowledge of an object's UUID.
35 $ <code class="userinput">arv get --help</code>
36 Usage: arv [--format json|yaml] get [uuid] [fields...]
38 Fetch the specified Arvados object, select the specified fields,
39 and print a text representation.
43 h3(#arv-edit). arv edit
45 @arv edit@ can be used to edit Arvados objects from the command line. Arv edit opens up the editor of your choice (set the EDITOR environment variable) with the json or yaml description of the object. Saving the file will update the Arvados object on the API server, if it passes validation.
49 $ <code class="userinput">arv edit --help</code>
50 Arvados command line client
51 Usage: arv edit [uuid] [fields...]
53 Fetch the specified Arvados object, select the specified fields,
54 open an interactive text editor on a text representation (json or
55 yaml, use --format) and then update the object. Will use 'nano'
56 by default, customize with the EDITOR or VISUAL environment variable.
60 h3(#arv-copy). arv copy
62 @arv copy@ can be used to copy a pipeline instance, template or collection from one Arvados instance to another. It takes care of copying the object and all its dependencies.
66 $ <code class="userinput">arv copy --help</code>
67 usage: arv_copy.py [-h] [-v] [--progress] [--no-progress] [-f] --src
68 SOURCE_ARVADOS --dst DESTINATION_ARVADOS [--recursive]
69 [--no-recursive] [--dst-git-repo DST_GIT_REPO]
70 [--project-uuid PROJECT_UUID] [--retries RETRIES]
73 Copy a pipeline instance, template or collection from one Arvados instance to
77 object_uuid The UUID of the object to be copied.
80 -h, --help show this help message and exit
81 -v, --verbose Verbose output.
82 --progress Report progress on copying collections. (default)
83 --no-progress Do not report progress on copying collections.
84 -f, --force Perform copy even if the object appears to exist at
85 the remote destination.
86 --src SOURCE_ARVADOS The name of the source Arvados instance (required) -
87 points at an Arvados config file. May be either a
88 pathname to a config file, or (for example) "foo" as
89 shorthand for $HOME/.config/arvados/foo.conf.
90 --dst DESTINATION_ARVADOS
91 The name of the destination Arvados instance
92 (required) - points at an Arvados config file. May be
93 either a pathname to a config file, or (for example)
94 "foo" as shorthand for $HOME/.config/arvados/foo.conf.
95 --recursive Recursively copy any dependencies for this object.
97 --no-recursive Do not copy any dependencies. NOTE: if this option is
98 given, the copied object will need to be updated
99 manually in order to be functional.
100 --dst-git-repo DST_GIT_REPO
101 The name of the destination git repository. Required
102 when copying a pipeline recursively.
103 --project-uuid PROJECT_UUID
104 The UUID of the project at the destination to which
105 the pipeline should be copied.
106 --retries RETRIES Maximum number of times to retry server requests that
107 encounter temporary failures (e.g., server down).
112 h3(#arv-tag). arv tag
114 @arv tag@ is used to tag Arvados objects.
118 $ <code class="userinput">arv tag --help</code>
121 arv tag add tag1 [tag2 ...] --object object_uuid1 [object_uuid2...]
122 arv tag remove tag1 [tag2 ...] --object object_uuid1 [object_uuid2...]
125 --dry-run, -n: Don't actually do anything
126 --verbose, -v: Print some things on stderr
127 --uuid, -u: Return the UUIDs of the objects in the response, one per
129 --json, -j: Return the entire response received from the API server, as
131 --human, -h: Return the response received from the API server, as a JSON
132 object with whitespace added for human consumption
133 --pretty, -p: Synonym of --human
134 --yaml, -y: Return the response received from the API server, in YAML
136 --help, -e: Show this message
143 This is a frontend to @arv-ws@.
145 @arv ws@ provides access to the websockets event stream.
149 $ <code class="userinput">arv ws --help</code>
150 usage: arv-ws [-h] [-u UUID] [-f FILTERS]
151 [--poll-interval POLL_INTERVAL | --no-poll]
152 [-p PIPELINE | -j JOB]
155 -h, --help show this help message and exit
156 -u UUID, --uuid UUID Filter events on object_uuid
157 -f FILTERS, --filters FILTERS
158 Arvados query filter to apply to log events (JSON
160 --poll-interval POLL_INTERVAL
161 If websockets is not available, specify the polling
162 interval, default is every 15 seconds
163 --no-poll Do not poll if websockets are not available, just fail
164 -p PIPELINE, --pipeline PIPELINE
165 Supply pipeline uuid, print log output from pipeline
167 -j JOB, --job JOB Supply job uuid, print log output from jobs
171 h3(#arv-keep). arv keep
173 @arv keep@ commands for accessing the Keep storage service.
177 $ <code class="userinput">arv keep --help</code>
178 Usage: arv keep [method] [--parameters]
179 Use 'arv keep [method] --help' to get more information about specific methods.
181 Available methods: ls, get, put, docker
185 h3(#arv-keep-ls). arv keep ls
187 This is a frontend to @arv-ls@.
191 $ <code class="userinput">arv keep ls --help</code>
192 usage: arv-ls [-h] [--retries RETRIES] [-s] locator
194 List contents of a manifest
196 positional arguments:
197 locator Collection UUID or locator
200 -h, --help show this help message and exit
201 --retries RETRIES Maximum number of times to retry server requests that
202 encounter temporary failures (e.g., server down). Default
204 -s List file sizes, in KiB.
208 h3(#arv-keep-get). arv keep get
210 This is a frontend to @arv-get@.
214 $ <code class="userinput">arv keep get --help</code>
215 usage: arv-get [-h] [--retries RETRIES]
216 [--progress | --no-progress | --batch-progress]
217 [--hash HASH | --md5sum] [-n] [-r] [-f | --skip-existing]
218 locator [destination]
220 Copy data from Keep to a local file or pipe.
222 positional arguments:
223 locator Collection locator, optionally with a file path or
225 destination Local file or directory where the data is to be written.
226 Default: /dev/stdout.
229 -h, --help show this help message and exit
230 --retries RETRIES Maximum number of times to retry server requests that
231 encounter temporary failures (e.g., server down). Default
233 --progress Display human-readable progress on stderr (bytes and, if
234 possible, percentage of total data size). This is the
235 default behavior when it is not expected to interfere
236 with the output: specifically, stderr is a tty _and_
237 either stdout is not a tty, or output is being written to
238 named files rather than stdout.
239 --no-progress Do not display human-readable progress on stderr.
240 --batch-progress Display machine-readable progress on stderr (bytes and,
241 if known, total data size).
242 --hash HASH Display the hash of each file as it is read from Keep,
243 using the given hash algorithm. Supported algorithms
244 include md5, sha1, sha224, sha256, sha384, and sha512.
245 --md5sum Display the MD5 hash of each file as it is read from
247 -n Do not write any data -- just read from Keep, and report
248 md5sums if requested.
249 -r Retrieve all files in the specified collection/prefix.
250 This is the default behavior if the "locator" argument
251 ends with a forward slash.
252 -f Overwrite existing files while writing. The default
253 behavior is to refuse to write *anything* if any of the
254 output files already exist. As a special case, -f is not
255 needed to write to /dev/stdout.
256 --skip-existing Skip files that already exist. The default behavior is to
257 refuse to write *anything* if any files exist that would
258 have to be overwritten. This option causes even devices,
259 sockets, and fifos to be skipped.
263 h3(#arv-keep-put). arv keep put
265 This is a frontend to @arv-put@.
269 $ <code class="userinput">arv keep put --help</code>
270 usage: arv-put [-h] [--max-manifest-depth N | --normalize]
271 [--as-stream | --stream | --as-manifest | --in-manifest | --manifest | --as-raw | --raw]
272 [--use-filename FILENAME] [--filename FILENAME]
273 [--portable-data-hash] [--replication N]
274 [--project-uuid UUID] [--name NAME]
275 [--progress | --no-progress | --batch-progress]
276 [--resume | --no-resume] [--retries RETRIES]
279 Copy data from the local filesystem to Keep.
281 positional arguments:
282 path Local file or directory. Default: read from standard
286 -h, --help show this help message and exit
287 --max-manifest-depth N
288 Maximum depth of directory tree to represent in the
289 manifest structure. A directory structure deeper than
290 this will be represented as a single stream in the
291 manifest. If N=0, the manifest will contain a single
292 stream. Default: -1 (unlimited), i.e., exactly one
293 manifest stream per filesystem directory that contains
295 --normalize Normalize the manifest by re-ordering files and
296 streams after writing data.
297 --as-stream Synonym for --stream.
298 --stream Store the file content and display the resulting
299 manifest on stdout. Do not write the manifest to Keep
300 or save a Collection object in Arvados.
301 --as-manifest Synonym for --manifest.
302 --in-manifest Synonym for --manifest.
303 --manifest Store the file data and resulting manifest in Keep,
304 save a Collection object in Arvados, and display the
305 manifest locator (Collection uuid) on stdout. This is
306 the default behavior.
307 --as-raw Synonym for --raw.
308 --raw Store the file content and display the data block
309 locators on stdout, separated by commas, with a
310 trailing newline. Do not store a manifest.
311 --use-filename FILENAME
312 Synonym for --filename.
313 --filename FILENAME Use the given filename in the manifest, instead of the
314 name of the local file. This is useful when "-" or
315 "/dev/stdin" is given as an input file. It can be used
316 only if there is exactly one path given and it is not
317 a directory. Implies --manifest.
318 --portable-data-hash Print the portable data hash instead of the Arvados
319 UUID for the collection created by the upload.
320 --replication N Set the replication level for the new collection: how
321 many different physical storage devices (e.g., disks)
322 should have a copy of each data block. Default is to
323 use the server-provided default (if any) or 2.
324 --project-uuid UUID Store the collection in the specified project, instead
325 of your Home project.
326 --name NAME Save the collection with the specified name.
327 --progress Display human-readable progress on stderr (bytes and,
328 if possible, percentage of total data size). This is
329 the default behavior when stderr is a tty.
330 --no-progress Do not display human-readable progress on stderr, even
332 --batch-progress Display machine-readable progress on stderr (bytes
333 and, if known, total data size).
334 --resume Continue interrupted uploads from cached state
336 --no-resume Do not continue interrupted uploads from cached state.
337 --retries RETRIES Maximum number of times to retry server requests that
338 encounter temporary failures (e.g., server down).