4 title: Federated Multi-Cluster Workflows
7 Copyright (C) The Arvados Authors. All rights reserved.
9 SPDX-License-Identifier: CC-BY-SA-3.0
12 To support running analysis on geographically dispersed data (avoiding expensive data transfers by sending the computation to the data), and "hybrid cloud" configurations where an on-premise cluster can expand its capabilities by delegating work to a cloud-hosted cluster, Arvados supports federated workflows. In a federated workflow, different steps of a workflow may execute on different clusters. Arvados manages data transfer and delegation of credentials, so that all that is required is adding "arv:ClusterTarget":cwl-extensions.html#clustertarget hints to your existing workflow.
14 !(full-width)federated-workflow.svg!
16 For more information, visit the "architecture":{{site.baseurl}}/architecture/federation.html and "admin":{{site.baseurl}}/admin/federation.html sections about Arvados federation.
18 h2. Get the example files
20 The tutorial files are located in the "documentation section of the Arvados source repository:":https://github.com/curoverse/arvados/tree/master/doc/user/cwl/federated
23 <pre><code>~$ <span class="userinput">git clone https://github.com/curoverse/arvados</span>
24 ~$ <span class="userinput">cd arvados/doc/user/cwl/federated</span>
28 h2. Federated scatter/gather example
30 In this following example, an analysis task is executed on three different clusters with different data, then the results are combined to produce the final output.
32 {% codeblock as yaml %}
33 {% include 'federated_cwl' %}
36 Example input document:
38 {% codeblock as yaml %}
39 {% include 'shards_yml' %}