- admin/index.html.textile.liquid
- Configuration:
- admin/config.html.textile.liquid
+ - admin/federation.html.textile.liquid
+ - admin/collection-managed-properties.html.textile.liquid
- Upgrading and migrations:
- admin/upgrading.html.textile.liquid
- admin/config-migration.html.textile.liquid
- - install/migrate-docker19.html.textile.liquid
- - admin/upgrade-crunch2.html.textile.liquid
- Users and Groups:
- install/cheat_sheet.html.textile.liquid
- admin/activation.html.textile.liquid
- admin/cloudtest.html.textile.liquid
- Data Management:
- admin/collection-versioning.html.textile.liquid
- - admin/collection-managed-properties.html.textile.liquid
- - Other:
- - admin/federation.html.textile.liquid
- admin/controlling-container-reuse.html.textile.liquid
- admin/logs-table-management.html.textile.liquid
+ - Other:
- admin/troubleshooting.html.textile.liquid
+ - install/migrate-docker19.html.textile.liquid
+ - admin/upgrade-crunch2.html.textile.liquid
installguide:
- Overview:
- install/index.html.textile.liquid
An overview on how this feature works is discussed in the "architecture section":{{site.baseurl}}/architecture/federation.html
-h3. API Server configuration
+h2. Configuration
-To accept users from remote clusters, some settings need to be added to the @application.yml@ file. There are two ways in which a remote cluster can be identified: either explictly by listing its prefix-to-hostname mapping, or implicitly by assuming the given remote cluster is public and belongs to the @.arvadosapi.com@ subdomain.
+To enable a cluster to communicate with other clusters, some settings need to be added to the @config.yml@ file. Federated clusters are identified by listing the cluster-to-hostname mapping in the @RemoteClusters@ section.
-For example, if you want to set up a private cluster federation, the following configuration will only allow access to users from @clsr2@ & @clsr3@:
-
-<pre>
-production:
- remote_hosts:
- clsr2: api.cluster2.com
- clsr3: api.cluster3.com
- remote_hosts_via_dns: false
- auto_activate_users_from: []
-</pre>
-
-The additional @auto_activate_users_from@ setting can be used to allow users from the clusters in the federation to not only read but also create & update objects on the local cluster. This feature is covered in more detail in the "user activation section":{{site.baseurl}}/admin/activation.html. In the current example, only manually activated remote users would have full access to the local cluster.
-
-h3. Arvados controller & keepstores configuration
-
-Both @arvados-controller@ and @keepstore@ services also need to be configured, as they proxy requests to remote clusters when needed.
-
-Continuing the previous example, the necessary settings should be added to the @/etc/arvados/config.yml@ file as follows:
+Here is an example of the settings that should be added to the @/etc/arvados/config.yml@ file:
<pre>
Clusters:
clsr2:
Host: api.cluster2.com
Proxy: true
+ ActivateUsers: true
clsr3:
Host: api.cluster3.com
Proxy: true
+ ActivateUsers: false
</pre>
Similar settings should be added to @clsr2@ & @clsr3@ hosts, so that all clusters in the federation can talk to each other.
-h3. Testing
+The @ActivateUsers@ setting indicates whether users from a given cluster are automatically activated or they require manual activation. User activation is covered in more detail in the "user activation section":{{site.baseurl}}/admin/activation.html. In the current example, users from @clsr2@ would be automatically, activated, but users from @clsr3@ would require an admin to activate the account.
+
+h2(#LoginCluster). Federation user management
+
+A federation of clusters can be configured to use a separate user database per cluster, or delegate a central cluster to manage the database.
+
+If clusters belong to separate organizations, each cluster will have its own user database for the members of that organization. Through federation, a user from one organization can be granted access to the cluster of another organization. The admin of the seond cluster controls access on a individual basis by choosing to activate or deactivate accounts from other organizations (with the default policy the value of @ActivateUsers@).
+
+On the other hand, if all clusters belong to the same organization, and users in that organization should have access to all the clusters, user management can be simplified by setting the @LoginCluster@ which manages the user database used by all other clusters in the federation. The @LoginCluster@ configuration should be set for all clusters in the federation to the id of one cluster in the federation which will be the 'master cluster'. This directs all logins to the LoginCluster, and the LoginCluster will issue API tokens which are valid on any cluster in the federation. Users are activated or deactivated across the entire federation based on their status on the master cluster.
+
+<pre>
+Clusters:
+ clsr2:
+ Login:
+ LoginCluster: clsr1
+</pre>
+
+To migrate users of an existing federation with separate user databases to use a single LoginCluster, use "arv-federation-migrate":merge-remote-account.html .
+
+h2. Testing
Following the above example, let's suppose @clsr1@ is our "home cluster", that is to say, we use our @clsr1@ user account as our federated identity and both @clsr2@ and @clsr3@ remote clusters are set up to allow users from @clsr1@ and to auto-activate them. The first thing to do would be to log into a remote workbench using the local user token. This can be done following these steps:
h2. Get user report
+h3. With a LoginCluster
+
+Set the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ environment variables to be an admin user on the @LoginCluster@. It will automatically determine the other clusters that are listed in the federation.
+
+Next, run @arv-federation-migrate@ with the @--report@ flag:
+
+<pre>
+$ arv-federation-migrate --report users.csv
+Getting user list from x6b1s
+Getting user list from x3982
+Wrote users.csv
+</pre>
+
+h3. Without a LoginCluster
+
The first step is to create @tokens.csv@ and list each cluster and API token to access the cluster. API tokens must be trusted tokens with administrator access. This is a simple comma separated value file and can be created in a text editor. Example:
_tokens.csv_
Wrote users.csv
</pre>
+h2. Update the user report
+
This will produce a report of users across all clusters listed in @tokens.csv@, sorted by email address. This file can be loaded into a text editor or spreadsheet program for ease of viewing and editing.
_users.csv_
<pre>
-email,user uuid,primary cluster/user
-person_a@example.com,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
-person_b@example.com,x3982-tpzed-1vl3k7knf7qihbe,
-person_b@example.com,x6b1s-tpzed-w4nhkx2rmrhlr54,
+email,username,user uuid,primary cluster/user
+person_a@example.com,person_a,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
+person_b@example.com,person_b,x3982-tpzed-1vl3k7knf7qihbe,
+person_b@example.com,person_b,x6b1s-tpzed-w4nhkx2rmrhlr54,
</pre>
-The third column describes that user's home cluster. If a user only has one account (identified by email address), the column will be filled in and there is nothing to do. If the column is blank, that means there is more than one Arvados account associated with the user. Edit the file and provide the desired home cluster for each user. In this example, <code>person_b@example.com</code> is assigned the home cluster @x3982@.
+The fourth column describes that user's home cluster. If a user only has one account (identified by email address), the column will be filled in and there is nothing to do. If the column is blank, that means there is more than one Arvados account associated with the user. Edit the file and provide the desired home cluster for each user as necessary (note: if there is a LoginCluster, all users will be migrated to the LoginCluster). It is also possible to change the desired username for a user. In this example, <code>person_b@example.com</code> is assigned the home cluster @x3982@.
_users.csv_
<pre>
-email,user uuid,primary cluster/user
-person_a@example.com,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
-person_b@example.com,x3982-tpzed-1vl3k7knf7qihbe,x3982
-person_b@example.com,x6b1s-tpzed-w4nhkx2rmrhlr54,x3982
+email,username,user uuid,primary cluster/user
+person_a@example.com,person_a,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
+person_b@example.com,person_b,x3982-tpzed-1vl3k7knf7qihbe,x3982
+person_b@example.com,person_b,x6b1s-tpzed-w4nhkx2rmrhlr54,x3982
</pre>
h2. Migrate users
To avoid disruption, advise users to log out and avoid running workflows while performing the migration.
-After updating @users.csv@, use the @--migrate@ option:
+After updating @users.csv@, use the @--migrate@ option (add @--tokens tokens.csv@ if not using LoginCluster):
<pre>
-$ arv-federation-migrate --tokens tokens.csv --migrate users.csv
+$ arv-federation-migrate --migrate users.csv
(person_b@example.com) Migrating x6b1s-tpzed-w4nhkx2rmrhlr54 to x3982-tpzed-1vl3k7knf7qihbe
</pre>
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-The "containers" API is the recommended way to submit compute work to Arvados. It supersedes the "jobs" API, which is deprecated.
+The "containers" API is the recommended way to submit compute work to Arvados. It supersedes the "jobs" API, which is end-of-life as of Arvados 1.5.
h2. Benefits over the "jobs" API
Proxy: true
</pre>
-The @remote_hosts@ section of @application.yml@ (for the API server)
-
-<pre>
-production:
- remote_hosts:
- clsr2: api.cluster2.com
- clsr3: api.cluster3.com
-</pre>
-
In this example, the cluster @clsr1@ is configured to contact @api.cluster2.com@ for requests involving @clsr2@ and @api.cluster3.com@ for requests involving @clsr3@.
h2(#identity). Identity
projects / arvados.git / commitdiff