X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/dc061e4b3c5be2d8687316c6d192fd053b4041e0..231a86fd3f7e30e9f66d71d92ad7c26578637e37:/doc/install/install-arv-git-httpd.html.textile.liquid diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid index 7eec2d645a..7fc332177d 100644 --- a/doc/install/install-arv-git-httpd.html.textile.liquid +++ b/doc/install/install-arv-git-httpd.html.textile.liquid @@ -3,6 +3,11 @@ layout: default navsection: installguide title: Install the Git server ... +{% comment %} +Copyright (C) The Arvados Authors. All rights reserved. + +SPDX-License-Identifier: CC-BY-SA-3.0 +{% endcomment %} Arvados allows users to create their own private and public git repositories, and clone/push them using SSH and HTTPS. @@ -14,6 +19,7 @@ The git hosting setup involves three components. It is not strictly necessary to deploy _both_ SSH and HTTPS access, but we recommend deploying both: * SSH is a more appropriate way to authenticate from a user's workstation because it does not require managing tokens on the client side; * HTTPS is a more appropriate way to authenticate from a shell VM because it does not depend on SSH agent forwarding (SSH clients' agent forwarding features tend to behave as if the remote machine is fully trusted). +* HTTPS is also used by Arvados Composer to access git repositories from the browser. The HTTPS instructions given below will not work if you skip the SSH setup steps. @@ -34,25 +40,10 @@ DNS and network configuration should be set up so port 443 reaches your HTTPS pr h2. Generate an API token -Use the following command to generate an API token, changing *@webserver-user@* to the user of the web server process. This is typically *@www-data@* on Debian systems by default, other systems may use different defaults such the name of the web server software (for example, *@nginx@*). - -Using RVM: - - -
gitserver:~$ cd /var/www/arvados-api/current
-gitserver:/var/www/arvados-api/current$ sudo -u webserver-user RAILS_ENV=production `which rvm-exec` default bundle exec ./script/create_superuser_token.rb
-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-
-
- -Not using RVM: - - -
gitserver:~$ cd /var/www/arvados-api/current
-gitserver:/var/www/arvados-api/current$ sudo -u webserver-user RAILS_ENV=production bundle exec ./script/create_superuser_token.rb
-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
-
-
+{% assign railshost = "gitserver" %} +{% assign railscmd = "bundle exec ./script/create_superuser_token.rb" %} +{% assign railsout = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" %} +Use the following command to generate an API token. {% include 'install_rails_command' %} Copy that token; you'll need it in a minute. @@ -104,14 +95,14 @@ git@gitserver:~$ rm .ssh/authorized_keys h2. Install gitolite -Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version. This guide was tested with @v3.6.3@. _Versions below 3.0 are missing some features needed by Arvados, and should not be used._ +Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version. This guide was tested with @v3.6.4@. _Versions below 3.0 are missing some features needed by Arvados, and should not be used._ Download and install the version you selected.
git@gitserver:~$ echo 'PATH=$HOME/bin:$PATH' >.profile
 git@gitserver:~$ source .profile
-git@gitserver:~$ git clone --branch v3.6.3 git://github.com/sitaramc/gitolite
+git@gitserver:~$ git clone --branch v3.6.4 https://github.com/sitaramc/gitolite
 ...
 Note: checking out '5d24ae666bfd2fa9093d67c840eb8d686992083f'.
 ...
@@ -247,6 +238,7 @@ On Red Hat-based systems:
 
 
 
~$ sudo yum install git arvados-git-httpd
+~$ sudo systemctl enable arvados-git-httpd
 
@@ -254,10 +246,9 @@ Verify that @arvados-git-httpd@ and @git-http-backend@ can be run:
~$ arvados-git-httpd -h
-Usage of arvados-git-httpd:
-  -address="0.0.0.0:80": Address to listen on, "host:port".
-  -git-command="/usr/bin/git": Path to git executable. Each authenticated request will execute this program with a single argument, "http-backend".
-  -repo-root="/path/to/cwd": Path to git repositories.
+[...]
+Usage: arvados-git-httpd [-config path/to/arvados/git-httpd.yml]
+[...]
 ~$ git http-backend
 Status: 500 Internal Server Error
 Expires: Fri, 01 Jan 1980 00:00:00 GMT
@@ -270,48 +261,29 @@ fatal: No REQUEST_METHOD from server
 
 h3. Enable arvados-git-httpd
 
-On Debian-based systems, install runit:
+{% include 'notebox_begin' %}
 
-
-
~$ sudo apt-get install runit
-
-
+The arvados-git-httpd package includes configuration files for systemd. If you're using a different init system, you'll need to configure a service to start and stop an @arvados-git-httpd@ process as desired. -On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor. +{% include 'notebox_end' %} -Configure runit to run arvados-git-httpd, making sure to update the API host to match your site: +Create the configuration file @/etc/arvados/git-httpd/git-httpd.yml@. Run @arvados-git-httpd -h@ to learn more about configuration entries. -
~$ cd /etc/sv
-/etc/sv$ sudo mkdir arvados-git-httpd; cd arvados-git-httpd
-/etc/sv/arvados-git-httpd$ sudo mkdir log
-/etc/sv/arvados-git-httpd$ sudo sh -c 'cat >log/run' <<'EOF'
-#!/bin/sh
-mkdir -p main
-chown git:git main
-exec chpst -u git:git svlogd -tt main
-EOF
-/etc/sv/arvados-git-httpd$ sudo sh -c 'cat >run' <<'EOF'
-#!/bin/sh
-export ARVADOS_API_HOST=uuid_prefix.your.domain
-export GITOLITE_HTTP_HOME=/var/lib/arvados/git
-export GL_BYPASS_ACCESS_CHECKS=1
-export PATH="$PATH:/var/lib/arvados/git/bin"
-exec chpst -u git:git arvados-git-httpd -address=:9001 -git-command=/var/lib/arvados/git/gitolite/src/gitolite-shell -repo-root=/var/lib/arvados/git/repositories 2>&1
-EOF
-/etc/sv/arvados-git-httpd$ sudo chmod +x run log/run
+
Client:
+  APIHost: uuid_prefix.your.domain
+  Insecure: false
+GitCommand: /var/lib/arvados/git/gitolite/src/gitolite-shell
+GitoliteHome: /var/lib/arvados/git
+Listen: :9001
+RepoRoot: /var/lib/arvados/git/repositories
 
-If you are using a different daemon supervisor, or if you want to test the daemon in a terminal window, an equivalent shell command to run arvados-git-httpd is: +Restart the systemd service to ensure the new configuration is used. -
sudo -u git \
-  ARVADOS_API_HOST=uuid_prefix.your.domain \
-  GITOLITE_HTTP_HOME=/var/lib/arvados/git \
-  GL_BYPASS_ACCESS_CHECKS=1 \
-  PATH="$PATH:/var/lib/arvados/git/bin" \
-  arvados-git-httpd -address=:9001 -git-command=/var/lib/arvados/git/gitolite/src/gitolite-shell -repo-root=/var/lib/arvados/git/repositories 2>&1
+
~$ sudo systemctl restart arvados-git-httpd
 
@@ -353,7 +325,7 @@ h3. Configure the API server to advertise the correct HTTPS URLs In your API server's @application.yml@ file, add the following entry: -
git_repo_http_base: https://git.uuid_prefix.your.domain/
+
git_repo_https_base: https://git.uuid_prefix.your.domain/
 
@@ -367,3 +339,47 @@ Restart Nginx to make the Nginx and API server configuration changes take effect
gitserver:~$ sudo nginx -s reload
 
+ +h2. Clone Arvados repository + +Here we create a repository object which will be used to set up a hosted clone of the arvados repository on this cluster. + + +
~$ uuid_prefix=`arv --format=uuid user current | cut -d- -f1`
+~$ echo "Site prefix is '$uuid_prefix'"
+~$ all_users_group_uuid="$uuid_prefix-j7d0g-fffffffffffffff"
+~$ repo_uuid=`arv --format=uuid repository create --repository "{\"owner_uuid\":\"$uuid_prefix-tpzed-000000000000000\", \"name\":\"arvados\"}"`
+~$ echo "Arvados repository uuid is '$repo_uuid'"
+
+ +Create a link object to make the repository object readable by the "All users" group, and therefore by every active user. This makes it possible for users to run the bundled Crunch scripts by specifying @"script_version":"master","repository":"arvados"@ rather than pulling the Arvados source tree into their own repositories. + + +
~$ read -rd $'\000' newlink <<EOF; arv link create --link "$newlink"
+{
+ "tail_uuid":"$all_users_group_uuid",
+ "head_uuid":"$repo_uuid",
+ "link_class":"permission",
+ "name":"can_read"
+}
+EOF
+
+ +In a couple of minutes, your arvados-git-sync cron job will create an empty repository on your git server. Seed it with the real arvados repository. If your git credential helpers were configured correctly when you "set up your shell server":install-shell-server.html, the "git push" command will use your API token instead of prompting you for a username and password. + + +
~$ cd /tmp
+/tmp$ git clone --bare https://github.com/curoverse/arvados.git
+/tmp git --git-dir arvados.git push https://git.uuid_prefix.your.domain/arvados.git '*:*'
+
+
+ +If you did not set up a HTTPS service, you can push to git@git.uuid_prefix.your.domain:arvados.git using your SSH key, or by logging in to your git server and using sudo. + + +
gitserver:~$ sudo -u git -i bash
+git@gitserver:~$ git clone --bare https://github.com/curoverse/arvados.git /tmp/arvados.git
+git@gitserver:~$ cd /tmp/arvados.git
+git@gitserver:/tmp/arvados.git$ gitolite push /var/lib/arvados/git/repositories/your_arvados_repo_uuid.git '*:*'
+
+