---
layout: default
navsection: admin
title: "Synchronizing from external sources"
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.

SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}

The @arvados-sync-users@ and @arvados-sync-groups@ tools allow to manage Arvados users & groups from external sources.

These tools are designed to be run periodically reading a file created by a remote auth system (ie: LDAP) dump script, applying what's included on the file as the source of truth.

bq. NOTE: Both tools need to perform several administrative tasks on Arvados, so must be run using a superuser token via @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ environment variables or @~/.config/arvados/settings.conf@ file.

h1. Using arvados-sync-users

This tool reads a CSV (comma-separated values) file having information about user accounts and their expected state on Arvados.

Every line on the file should have 5 fields:

# A user identifier: it could be an email address (default) or a username.
# The user's first name.
# The user's last name.
# The intended user's active state.
# The intended user's admin state: will always be read as @false@ when @active=false@.

The last 2 fields should be represented as @true@/@false@, @yes@/@no@, or @1@/@0@ values.

h2. Options

The following command line options are supported:

table(table table-bordered table-condensed).
|_. Option |_. Description |
|==--help==|This list of options|
|==--case-insensitive==|Uses case-insensitive username matching|
|==--deactivate-unlisted==|Deactivate users that aren't listed on the input file. (Current & system users won't be affected)|
|==--user-id==|Identifier to use in looking up user. One of 'email' or 'username' (Default: 'email')|
|==--verbose==|Log informational messages|
|==--version==|Print version and exit|

The tool will create users when needed, and update those existing records to match the desired state described by the fields on the CSV file.
System users like the root and anonymous are unaffected by this tool.
In the case of a @LoginCluster@ federation, this tool should be run on the cluster that manages the user accounts, and will fail otherwise.

h2. Example

To sync users using the username to identify every account, reading from some @external_users.csv@ file and deactivating existing users that aren't included in it, the command should be called as follows:

<notextile>
<pre><code>~$ <span class="userinput">arvados-sync-users --deactivate-unlisted --user-id username /path/to/external_users.csv </span>
</code></pre>
</notextile>

h1. Using arvados-sync-groups

This tool reads a CSV (comma-separated values) file having information about external groups and their members. When running it for the first time, it'll create a special group named 'Externally synchronized groups' meant to be the parent of all the remote groups.

Every line on the file should have 3 values: a group name, a local user identifier and a permission level, meaning that the named user is a member of the group with the provided permission. The tool will create the group if it doesn't exist, and add the user to it. If any group member is not present on the input file, it will be removed from the group.

Users can be identified by their email address or username: the tool will check if every user exist on the system, and report back when not found. Groups on the other hand, are identified by their name.

Permission level can be one of the following: @can_read@, @can_write@ or @can_manage@, giving the group member read, read/write or managing privileges on the group. For backwards compatibility purposes, if any record omits the third (permission) field, it will default to @can_write@ permission. You can read more about permissions on the "group management admin guide":{{ site.baseurl }}/admin/group-management.html.

When using @arvados-sync-groups@, consider setting @Users.CanCreateRoleGroups: false@ in your "cluster configuration":{{site.baseurl}}/admin/config.html to prevent users from creating additional groups.

h2. Options

The following command line options are supported:

table(table table-bordered table-condensed).
|_. Option |_. Description |
|==--help==|This list of options|
|==--case-insensitive==|Uses case-insensitive username matching|
|==--parent-group-uuid==|UUID of group to own all the externally synchronized groups|
|==--user-id==|Identifier to use in looking up user. One of 'email' or 'username' (Default: 'email')|
|==--verbose==|Log informational messages (Default: False)|
|==--version==|Print version and exit|

h2. Examples

To sync groups using the username to identify every account, reading from some @external_groups.csv@ file, the command should be called as follows:

<notextile>
<pre><code>~$ <span class="userinput">arvados-sync-groups --user-id username /path/to/external_groups.csv </span>
</code></pre>
</notextile>

If you want to use a specific preexisting group as the parent of all the remote groups, you can do it this way:

<notextile>
<pre><code>~$ <span class="userinput">arvados-sync-groups --parent-group-uuid &lt;preexisting group UUID&gt; --user-id username /path/to/external_groups.csv </span>
</code></pre>
</notextile>