19215: Add install page mentioning diagnostics tool.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

diff --git a/doc/_config.yml b/doc/_config.yml
index 56ae1bf191cfa332ff9e6c1d462b04fc0f708fad..ba17560c57d4800f4560c307a4cf734c604d2908 100644 (file)
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -228,6 +228,7 @@ navbar:
       - admin/maintenance-and-upgrading.html.textile.liquid
     - Core:
       - install/install-api-server.html.textile.liquid
+      - install/diagnostics.html.textile.liquid
     - Keep:
       - install/install-keepstore.html.textile.liquid
       - install/configure-fs-storage.html.textile.liquid

diff --git a/doc/install/diagnostics.html.textile.liquid b/doc/install/diagnostics.html.textile.liquid
new file mode 100644 (file)
index 0000000..f139ff6
--- /dev/null
+++ b/doc/install/diagnostics.html.textile.liquid
@@ -0,0 +1,53 @@
+---
+layout: default
+navsection: installguide
+title: Cluster diagnostics tool
+
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+The @diagnostics@ subcommand of @arvados-client@ performs a variety of checks to help confirm that your Arvados installation has been properly configured.  It is extremely helpful to validate that your install is successful.
+
+Here is an example of it in action:
+
+<pre>
+root@api:~$ apt-get install arvados-client
+root@api:~$ export ARVADOS_API_HOST=ClusterID.example.com
+root@api:~$ export ARVADOS_API_TOKEN=YourSytemRootTokenHere
+root@api:~$ arvados-client diagnostics -external-client
+INFO      10: getting discovery document from https://ClusterID.example.com/discovery/v1/apis/arvados/v1/rest
+INFO      20: getting exported config from https://ClusterID.example.com/arvados/v1/config
+INFO      30: getting current user record
+INFO      40: connecting to service endpoint https://keep.ClusterID.example.com/
+INFO      41: connecting to service endpoint https://*.collections.ClusterID.example.com/
+INFO      42: connecting to service endpoint https://download.ClusterID.example.com/
+INFO      43: connecting to service endpoint wss://ws.ClusterID.example.com/websocket
+INFO      44: connecting to service endpoint https://workbench.ClusterID.example.com/
+INFO      45: connecting to service endpoint https://workbench2.ClusterID.example.com/
+INFO      50: checking CORS headers at https://ClusterID.example.com/
+INFO      51: checking CORS headers at https://keep.ClusterID.example.com/d41d8cd98f00b204e9800998ecf8427e+0
+INFO      52: checking CORS headers at https://download.ClusterID.example.com/
+INFO      60: checking internal/external client detection
+INFO      61: reading+writing via keep service at https://keep.ClusterID.example.com:443/
+INFO      80: finding/creating "scratch area for diagnostics" project
+INFO      90: creating temporary collection
+INFO     100: uploading file via webdav
+INFO     110: checking WebDAV ExternalURL wildcard (https://*.collections.ClusterID.example.com/)
+INFO     120: downloading from webdav (https://d41d8cd98f00b204e9800998ecf8427e-0.collections.ClusterID.example.com/foo)
+INFO     121: downloading from webdav (https://d41d8cd98f00b204e9800998ecf8427e-0.collections.ClusterID.example.com/sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412.tar)
+INFO     122: downloading from webdav (https://download.ClusterID.example.com/c=d41d8cd98f00b204e9800998ecf8427e+0/_/foo)
+INFO     123: downloading from webdav (https://download.ClusterID.example.com/c=d41d8cd98f00b204e9800998ecf8427e+0/_/sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412.tar)
+INFO     124: downloading from webdav (https://a15a27cbc1c7d2d4a0d9e02529aaec7e-128.collections.ClusterID.example.com/sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412.tar)
+INFO     125: downloading from webdav (https://download.ClusterID.example.com/c=ce8i5-4zz18-bkfvq2skqqf78xd/_/sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412.tar)
+INFO     130: getting list of virtual machines
+INFO     140: getting workbench1 webshell page
+INFO     150: connecting to webshell service
+INFO     160: running a container
+INFO      ... container request submitted, waiting up to 10m for container to run
+INFO    9990: deleting temporary collection
+INFO    --- no errors ---
+</pre>

diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index 4c9f168e825946bc0b63a86932f844218ab3819b..06f94a8a5f11f26329151fbe64f63a2d6d4c0589 100644 (file)
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -203,17 +203,21 @@ server {
 
 h2(#confirm-working). Confirm working installation
 
-Confirm working controller:
+We recommend using the "Cluster diagnostics tool.":diagnostics.html  The first few tests (10, 20, 30) will succeed if you have a working API server and controller.  Of course, tests for services that you have not yet installed and configured will fail.
+
+Here are some other checks you can perform manually.
+
+h3. Confirm working controller
 
 <notextile><pre><code>$ curl https://<span class="userinput">ClusterID.example.com</span>/arvados/v1/config
 </code></pre></notextile>
 
-Confirm working Rails API server:
+h3. Confirm working Rails API server
 
 <notextile><pre><code>$ curl https://<span class="userinput">ClusterID.example.com</span>/discovery/v1/apis/arvados/v1/rest
 </code></pre></notextile>
 
-Confirm that you can use the system root token to act as the system root user:
+h3. Confirm that you can use the system root token to act as the system root user
 
 <notextile><pre><code>$ curl -H "Authorization: Bearer $system_root_token" https://<span class="userinput">ClusterID.example.com</span>/arvados/v1/users/current
 </code></pre></notextile>

diff --git a/doc/install/install-manual-prerequisites.html.textile.liquid b/doc/install/install-manual-prerequisites.html.textile.liquid
index 784d712f1e420088facc4ee4327c5580a79cdd36..67b9f0de334758addbbb63837da58ea7248eb409 100644 (file)
--- a/doc/install/install-manual-prerequisites.html.textile.liquid
+++ b/doc/install/install-manual-prerequisites.html.textile.liquid
@@ -44,15 +44,14 @@ table(table table-bordered table-condensed).
 |"Keep-balance":install-keep-balance.html |Storage cluster maintenance daemon responsible for moving blocks to their optimal server location, adjusting block replication levels, and trashing unreferenced blocks.|Required to free deleted data from underlying storage, and to ensure proper replication and block distribution (including support for storage classes).|
 |\3=. *User interface*|
 |"Workbench":install-workbench-app.html, "Workbench2":install-workbench2-app.html |Primary graphical user interface for working with file collections and running containers.|Optional.  Depends on API server, keep-web, websockets server.|
-|"Workflow Composer":install-composer.html |Graphical user interface for editing Common Workflow Language workflows.|Optional.  Depends on git server (arvados-git-httpd).|
 |\3=. *Additional services*|
 |"Websockets server":install-ws.html |Event distribution server.|Required to view streaming container logs in Workbench.|
-|"Shell server":install-shell-server.html |Synchronize (create/delete/configure) Unix shell accounts with Arvados users.|Optional.|
-|"Git server":install-arv-git-httpd.html |Arvados-hosted git repositories, with Arvados-token based authentication.|Optional, but required by Workflow Composer.|
+|"Shell server":install-shell-server.html |Grant Arvados users access to Unix shell accounts on dedicated shell nodes.|Optional.|
+|"Git server":install-arv-git-httpd.html |Arvados-hosted git repositories, with Arvados-token based authentication.|Optional|
 |\3=. *Crunch (running containers)*|
-|"arvados-dispatch-cloud":crunch2-cloud/install-dispatch-cloud.html |Allocate and free cloud VM instances on demand based on workload.|Optional, not needed for a static Slurm cluster such as on-premises HPC.|
-|"crunch-dispatch-slurm":crunch2-slurm/install-dispatch.html |Run analysis workflows using Docker or Singularity containers distributed across a Slurm cluster.|Optional, not needed for a Cloud installation, or if you wish to use Arvados for data management only.|
-|"crunch-dispatch-lsf":crunch2-lsf/install-dispatch.html |Run analysis workflows using Docker or Singularity containers distributed across an LSF cluster.|Optional, not needed for a Cloud installation, or if you wish to use Arvados for data management only.|
+|"arvados-dispatch-cloud":crunch2-cloud/install-dispatch-cloud.html |Run analysis workflows on cloud by allocating and freeing cloud VM instances on demand.|Optional|
+|"crunch-dispatch-slurm":crunch2-slurm/install-dispatch.html |Run analysis workflows distributed across a Slurm cluster.|Optional|
+|"crunch-dispatch-lsf":crunch2-lsf/install-dispatch.html |Run analysis workflows distributed across an LSF cluster.|Optional|
 
 h2(#identity). Identity provider
 
@@ -159,8 +158,8 @@ The table below lists the required TLS certificates and DNS names in each scenar
 <div class="offset1">
 table(table table-bordered table-condensed).
 ||_. Wildcard TLS and DNS available|_. Wildcard TLS available|_. Other|
-|TLS|*.@ClusterID.example.com@
-@ClusterID.example.com@
+|TLS|@ClusterID.example.com@
+*.@ClusterID.example.com@
 *.collections.@ClusterID.example.com@|*.@ClusterID.example.com@
 @ClusterID.example.com@|@ClusterID.example.com@
 git.@ClusterID.example.com@