4 title: Analyzing workflow cost (cloud only)
7 Copyright (C) The Arvados Authors. All rights reserved.
9 SPDX-License-Identifier: CC-BY-SA-3.0
12 {% include 'notebox_begin' %}
14 This is only applicable when Arvados runs in a cloud environment and @arvados-dispatch-cloud@ is used to dispatch @crunch@ jobs. The per node-hour price for each defined InstanceType most be supplied in "config.yml":{{site.baseurl}}/admin/config.html.
16 {% include 'notebox_end' %}
18 The @arvados-client@ program can be used to analyze the cost of a workflow. It can be installed from packages (@apt install arvados-client@ or @yum install arvados-client@). The @arvados-client costanalyzer@ command analyzes the cost accounting information associated with Arvados container requests.
22 The @arvados-client costanalyzer@ tool has a number of command line arguments:
25 <pre><code>~$ <span class="userinput">arvados-client costanalyzer -h</span>
27 arvados-client costanalyzer [options ...] uuid [uuid ...]
29 This program analyzes the cost of Arvados container requests. For each uuid
30 supplied, it creates a CSV report that lists all the containers used to
31 fulfill the container request, together with the machine type and cost of
32 each container. At least one uuid must be specified.
34 When supplied with the uuid of a container request, it will calculate the
35 cost of that container request and all its children.
37 When supplied with the uuid of a collection, it will see if there is a
38 container_request uuid in the properties of the collection, and if so, it
39 will calculate the cost of that container request and all its children.
41 When supplied with a project uuid or when supplied with multiple container
42 request or collection uuids, it will create a CSV report for each supplied
43 uuid, as well as a CSV file with aggregate cost accounting for all supplied
44 uuids. The aggregate cost report takes container reuse into account: if a
45 container was reused between several container requests, its cost will only
50 - This program uses the cost data from config.yml at the time of the
51 execution of the container, stored in the 'node.json' file in its log
52 collection. If the cost data was not correctly configured at the time the
53 container was executed, the output from this program will be incorrect.
55 - If a container was run on a preemptible ("spot") instance, the cost data
56 reported by this program may be wildly inaccurate, because it does not have
57 access to the spot pricing in effect for the node then the container ran. The
58 UUID report file that is generated when the '-output' option is specified has
59 a column that indicates the preemptible state of the instance that ran the
62 - This program does not take into account overhead costs like the time spent
63 starting and stopping compute nodes that run containers, the cost of the
64 permanent cloud nodes that provide the Arvados services, the cost of data
65 stored in Arvados, etc.
67 - When provided with a project uuid, subprojects will not be considered.
69 In order to get the data for the uuids supplied, the ARVADOS_API_HOST and
70 ARVADOS_API_TOKEN environment variables must be set.
72 This program prints the total dollar amount from the aggregate cost
73 accounting across all provided uuids on stdout.
75 When the '-output' option is specified, a set of CSV files with cost details
76 will be written to the provided directory.
80 create and use a local disk cache of Arvados objects (default true)
82 logging level (debug, info, ...) (default "info")
84 output directory for the CSV reports