----
-layout: default
-navsection: installguide
-title: Install the Git server
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-# "Introduction":#introduction
-# "Install dependencies":#dependencies
-# "Create "git" user and storage directory":#create
-# "Install gitolite":#gitolite
-# "Configure gitolite":#config-gitolite
-# "Configure git synchronization":#sync
-# "Update config.yml":#update-config
-# "Update nginx configuration":#update-nginx
-# "Install arvados-git-httpd package":#install-packages
-# "Restart the API server and controller":#restart-api
-# "Confirm working installation":#confirm-working
-
-h2(#introduction). Introduction
-
-Arvados support for git repository management enables using Arvados permissions to control access to git repositories. Users can create their own private and public git repositories and share them with others.
-
-The git hosting setup involves three components.
-* The "arvados-git-sync.rb" script polls the API server for the current list of repositories, creates bare repositories, and updates the local permission cache used by gitolite.
-* Gitolite provides SSH access. Users authenticate by SSH keys.
-* arvados-git-http provides HTTPS access. Users authenticate by Arvados tokens.
-
-Git services must be installed on the same host as the Arvados Rails API server.
-
-h2(#dependencies). Install dependencies
-
-h3. Centos 7
-
-<notextile>
-<pre><code># <span class="userinput">yum install git perl-Data-Dumper openssh-server</span>
-</code></pre>
-</notextile>
-
-h3. Debian and Ubuntu
-
-<notextile>
-<pre><code># <span class="userinput">apt-get --no-install-recommends install git openssh-server</span>
-</code></pre>
-</notextile>
-
-h2(#create). Create "git" user and storage directory
-
-Gitolite and some additional scripts will be installed in @/var/lib/arvados/git@, which means hosted repository data will be stored in @/var/lib/arvados/git/repositories@. If you choose to install gitolite in a different location, make sure to update the @git_repositories_dir@ entry in your API server's @application.yml@ file accordingly: for example, if you install gitolite at @/data/gitolite@ then your @git_repositories_dir@ will be @/data/gitolite/repositories@.
-
-A new UNIX account called "git" will own the files. This makes git URLs look familiar to users (<code>git@[...]:username/reponame.git</code>).
-
-On Debian- or Red Hat-based systems:
-
-<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git</span>
-gitserver:~$ <span class="userinput">sudo useradd --comment git --home-dir /var/lib/arvados/git git</span>
-gitserver:~$ <span class="userinput">sudo chown -R git:git ~git</span>
-</code></pre>
-</notextile>
-
-The git user needs its own SSH key. (It must be able to run <code>ssh git@localhost</code> from scripts.)
-
-<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo -u git -i bash</span>
-git@gitserver:~$ <span class="userinput">ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa</span>
-git@gitserver:~$ <span class="userinput">cp .ssh/id_rsa.pub .ssh/authorized_keys</span>
-git@gitserver:~$ <span class="userinput">ssh -o stricthostkeychecking=no localhost cat .ssh/id_rsa.pub</span>
-Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
-ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7aBIDAAgMQN16Pg6eHmvc+D+6TljwCGr4YGUBphSdVb25UyBCeAEgzqRiqy0IjQR2BLtSirXr+1SJAcQfBgI/jwR7FG+YIzJ4ND9JFEfcpq20FvWnMMQ6XD3y3xrZ1/h/RdBNwy4QCqjiXuxDpDB7VNP9/oeAzoATPZGhqjPfNS+RRVEQpC6BzZdsR+S838E53URguBOf9yrPwdHvosZn7VC0akeWQerHqaBIpSfDMtaM4+9s1Gdsz0iP85rtj/6U/K/XOuv2CZsuVZZ52nu3soHnEX2nx2IaXMS3L8Z+lfOXB2T6EaJgXF7Z9ME5K1tx9TSNTRcYCiKztXLNLSbp git@gitserver
-git@gitserver:~$ <span class="userinput">rm .ssh/authorized_keys</span>
-</code></pre>
-</notextile>
-
-h2(#gitolite). Install gitolite
-
-Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version. This guide was tested with @v3.6.11@. _Versions below 3.0 are missing some features needed by Arvados, and should not be used._
-
-Download and install the version you selected.
-
-<notextile>
-<pre><code>$ <span class="userinput">sudo -u git -i bash</span>
-git@gitserver:~$ <span class="userinput">echo 'PATH=$HOME/bin:$PATH' >.profile</span>
-git@gitserver:~$ <span class="userinput">. .profile</span>
-git@gitserver:~$ <span class="userinput">git clone --branch <b>v3.6.11</b> https://github.com/sitaramc/gitolite</span>
-...
-Note: checking out '5d24ae666bfd2fa9093d67c840eb8d686992083f'.
-...
-git@gitserver:~$ <span class="userinput">mkdir bin</span>
-git@gitserver:~$ <span class="userinput">gitolite/install -ln ~git/bin</span>
-git@gitserver:~$ <span class="userinput">bin/gitolite setup -pk .ssh/id_rsa.pub</span>
-Initialized empty Git repository in /var/lib/arvados/git/repositories/gitolite-admin.git/
-Initialized empty Git repository in /var/lib/arvados/git/repositories/testing.git/
-WARNING: /var/lib/arvados/git/.ssh/authorized_keys missing; creating a new one
- (this is normal on a brand new install)
-</code></pre>
-</notextile>
-
-_If this didn't go well, more detail about installing gitolite, and information about how it works, can be found on the "gitolite home page":http://gitolite.com/._
-
-Clone the gitolite-admin repository. The arvados-git-sync.rb script works by editing the files in this working directory and pushing them to gitolite. Here we make sure "git push" won't produce any errors or warnings.
-
-<notextile>
-<pre><code>git@gitserver:~$ <span class="userinput">git clone git@localhost:gitolite-admin</span>
-Cloning into 'gitolite-admin'...
-remote: Counting objects: 6, done.
-remote: Compressing objects: 100% (4/4), done.
-remote: Total 6 (delta 0), reused 0 (delta 0)
-Receiving objects: 100% (6/6), done.
-Checking connectivity... done.
-git@gitserver:~$ <span class="userinput">cd gitolite-admin</span>
-git@gitserver:~/gitolite-admin$ <span class="userinput">git config user.email arvados</span>
-git@gitserver:~/gitolite-admin$ <span class="userinput">git config user.name arvados</span>
-git@gitserver:~/gitolite-admin$ <span class="userinput">git config push.default simple</span>
-git@gitserver:~/gitolite-admin$ <span class="userinput">git push</span>
-Everything up-to-date
-</code></pre>
-</notextile>
-
-h2(#config-gitolite). Configure gitolite
-
-Configure gitolite to look up a repository name like @username/reponame.git@ and find the appropriate bare repository storage directory.
-
-Add the following lines to the top of @~git/.gitolite.rc@:
-
-<notextile>
-<pre><code><span class="userinput">my $repo_aliases;
-my $aliases_src = "$ENV{HOME}/.gitolite/arvadosaliases.pl";
-if ($ENV{HOME} && (-e $aliases_src)) {
- $repo_aliases = do $aliases_src;
-}
-$repo_aliases ||= {};
-</span></code></pre>
-</notextile>
-
-Add the following lines inside the section that begins @%RC = (@:
-
-<notextile>
-<pre><code><span class="userinput"> REPO_ALIASES => $repo_aliases,
-</span></code></pre>
-</notextile>
-
-Inside that section, adjust the 'UMASK' setting to @022@, to ensure the API server has permission to read repositories:
-
-<notextile>
-<pre><code> UMASK => <span class="userinput">022</span>,
-</code></pre>
-</notextile>
-
-Uncomment the 'Alias' line in the section that begins @ENABLE => [@:
-
-<notextile>
-<pre><code><span class="userinput"> # access a repo by another (possibly legacy) name
- 'Alias',
-</span></code></pre>
-</notextile>
-
-h2(#sync). Configure git synchronization
-
-Create a configuration file @/var/www/arvados-api/current/config/arvados-clients.yml@ using the following template, filling in the appropriate values for your system.
-* For @arvados_api_token@, use @SystemRootToken@
-* For @gitolite_arvados_git_user_key@, provide the public key you generated above, i.e., the contents of @~git/.ssh/id_rsa.pub@.
-
-<notextile>
-<pre><code>production:
- gitolite_url: /var/lib/arvados/git/repositories/gitolite-admin.git
- gitolite_tmp: /var/lib/arvados/git
- arvados_api_host: <span class="userinput">ClusterID.example.com</span>
- arvados_api_token: "<span class="userinput">zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz</span>"
- arvados_api_host_insecure: <span class="userinput">false</span>
- gitolite_arvados_git_user_key: "<span class="userinput">ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7aBIDAAgMQN16Pg6eHmvc+D+6TljwCGr4YGUBphSdVb25UyBCeAEgzqRiqy0IjQR2BLtSirXr+1SJAcQfBgI/jwR7FG+YIzJ4ND9JFEfcpq20FvWnMMQ6XD3y3xrZ1/h/RdBNwy4QCqjiXuxDpDB7VNP9/oeAzoATPZGhqjPfNS+RRVEQpC6BzZdsR+S838E53URguBOf9yrPwdHvosZn7VC0akeWQerHqaBIpSfDMtaM4+9s1Gdsz0iP85rtj/6U/K/XOuv2CZsuVZZ52nu3soHnEX2nx2IaXMS3L8Z+lfOXB2T6EaJgXF7Z9ME5K1tx9TSNTRcYCiKztXLNLSbp git@gitserver</span>"
-</code></pre>
-</notextile>
-
-<pre>
-$ sudo chown git:git /var/www/arvados-api/current/config/arvados-clients.yml
-$ sudo chmod og-rwx /var/www/arvados-api/current/config/arvados-clients.yml
-</pre>
-
-h3. Test configuration
-
-notextile. <pre><code>$ <span class="userinput">sudo -u git -i bash -c 'cd /var/www/arvados-api/current && bin/bundle exec script/arvados-git-sync.rb production'</span></code></pre>
-
-h3. Enable the synchronization script
-
-The API server package includes a script that retrieves the current set of repository names and permissions from the API, writes them to @arvadosaliases.pl@ in a format usable by gitolite, and triggers gitolite hooks which create new empty repositories if needed. This script should run every 2 to 5 minutes.
-
-Create @/etc/cron.d/arvados-git-sync@ with the following content:
-
-<notextile>
-<pre><code><span class="userinput">*/5 * * * * git cd /var/www/arvados-api/current && bin/bundle exec script/arvados-git-sync.rb production</span>
-</code></pre>
-</notextile>
-
-h2(#update-config). Update config.yml
-
-Edit the cluster config at @config.yml@ .
-
-<notextile>
-<pre><code> Services:
- GitSSH:
- ExternalURL: "<span class="userinput">ssh://git@git.ClusterID.example.com</span>"
- GitHTTP:
- ExternalURL: <span class="userinput">https://git.ClusterID.example.com/</span>
- InternalURLs:
- "http://localhost:9001": {}
- Git:
- GitCommand: <span class="userinput">/var/lib/arvados/git/gitolite/src/gitolite-shell</span>
- GitoliteHome: <span class="userinput">/var/lib/arvados/git</span>
- Repositories: <span class="userinput">/var/lib/arvados/git/repositories</span>
-</code></pre>
-</notextile>
-
-h2(#update-nginx). Update nginx configuration
-
-Use a text editor to create a new file @/etc/nginx/conf.d/arvados-git.conf@ with the following configuration. Options that need attention are marked in <span class="userinput">red</span>.
-
-<notextile>
-<pre><code>upstream arvados-git-httpd {
- server 127.0.0.1:<span class="userinput">9001</span>;
-}
-server {
- listen 443 ssl;
- server_name git.<span class="userinput">ClusterID.example.com</span>;
- proxy_connect_timeout 90s;
- proxy_read_timeout 300s;
-
- ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
- ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
-
- # The server needs to accept potentially large refpacks from push clients.
- client_max_body_size 128m;
-
- location / {
- proxy_pass http://arvados-git-httpd;
- }
-}
-</code></pre>
-</notextile>
-
-h2(#install-packages). Install the arvados-git-httpd package
-
-The arvados-git-httpd package provides HTTP access, using Arvados authentication tokens instead of passwords. It must be installed on the system where your git repositories are stored.
-
-h3. Centos 7
-
-<notextile>
-<pre><code># <span class="userinput">yum install arvados-git-httpd</span>
-</code></pre>
-</notextile>
-
-h3. Debian and Ubuntu
-
-<notextile>
-<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-git-httpd</span>
-</code></pre>
-</notextile>
-
-h2(#restart-api). Restart the API server and controller
-
-After adding Workbench to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
-
-<notextile>
-<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
-</code></pre>
-</notextile>
-
-h2(#confirm-working). Confirm working installation
-
-Create 'testrepo' in the Arvados database.
-
-<notextile>
-<pre><code>~$ <span class="userinput">arv --format=uuid repository create --repository '{"name":"myusername/testrepo"}'</span>
-</code></pre></notextile>
-
-The arvados-git-sync cron job will notice the new repository record and create a repository on disk. Because it is on a timer (default 5 minutes) you may have to wait a minute or two for it to show up.
-
-h3. SSH
-
-Before you do this, go to Workbench and choose *SSH Keys* from the menu, and upload your public key. Arvados uses the public key to identify you when you access the git repo.
-
-<notextile>
-<pre><code>~$ <span class="userinput">git clone git@git.ClusterID.example.com:username/testrepo.git</span>
-</code></pre>
-</notextile>
-
-h3. HTTP
-
-Set up git credential helpers as described in "install shell server":install-shell-server.html#config-git for the git command to use your API token instead of prompting you for a username and password.
-
-<notextile>
-<pre><code>~$ <span class="userinput">git clone https://git.ClusterID.example.com/username/testrepo.git</span>
-</code></pre>
-</notextile>
projects / arvados.git / blobdiff