Merge branch '18259-doc-architecture-singularity'

closes #18259

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

diff --git a/doc/_config.yml b/doc/_config.yml
index ff924e2f378381f4c8f4e586f3c2c3570d8202e2..6f1c90d9b0771bfa70ae5681ec142dbfbf9af21a 100644 (file)
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -162,6 +162,7 @@ navbar:
     - Computation with Crunch:
       - api/execution.html.textile.liquid
       - architecture/dispatchcloud.html.textile.liquid
+      - architecture/singularity.html.textile.liquid
     - Other:
       - api/permission-model.html.textile.liquid
       - architecture/federation.html.textile.liquid

diff --git a/doc/architecture/singularity.html.textile.liquid b/doc/architecture/singularity.html.textile.liquid
new file mode 100644 (file)
index 0000000..dc10d28
--- /dev/null
+++ b/doc/architecture/singularity.html.textile.liquid
@@ -0,0 +1,35 @@
+---
+layout: default
+navsection: architecture
+title: Singularity
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+Arvados can be configured to use "Singularity":https://sylabs.io/singularity/ instead of Docker to execute containers on cloud nodes or a SLURM/LSF cluster. Singularity may be preferable due to its simpler installation and lack of long-running daemon process and special system users/groups. See the "Singularity page in the installation guide":{{ site.baseurl }}/install/singularity.html for configuration details.
+
+h2. Design overview
+
+When Arvados is configured to use Singularity as the runtime engine for Crunch, containers are executed by Singularity. The images specified in workflows and tool definitions must be Docker images uploaded via @arv-keepdocker@ or @arvados-cwl-runner@. When Singularity is the runtime engine, these images are converted to Singularity format (@.sif@) at runtime, as needed.
+
+To avoid repeating this conversion work unnecessarily, the @.sif@ files are cached in @Keep@. This is done on a per-user basis. If it does not exist yet, a new Arvados project named @.cache@ is automatically created in the user's home project. Similarly, a subproject named @auto-generated singularity images@ will be created in the @.cache@ project. The automatically generated @.sif@ files are stored in collections in that project, with an expiration date two weeks in the future. If the cached image exists when Crunch runs a new container, the expiration date will be pushed out, so that it is always 2 weeks in the future from the most recent start of a container using the image.
+
+It is safe to empty out or even remove the .cache project or any of its contents; if necessary the cache projects and the @.sif@ files will automatically be regenerated.
+
+h2. Notes
+
+* Programs running in Singularity containers may behave differently than when run in Docker, due to differences between Singularity and Docker. For example, the root (image) filesystem is read-only in a Singularity container. Programs that attempt to write outside a designated output or temporary directory are likely to fail.
+
+* When using Singularity as the runtime engine, the compute node needs to have a compatible Singularity executable installed, as well as the @mksquashfs@ program used to convert Docker images to Singularity's @.sif@ format. The Arvados "compute node image build script":{{ site.baseurl }}/install/crunch2-cloud/install-compute-node.html includes these executables since Arvados 2.3.0.
+
+h2. Limitations
+
+Arvados @Singularity@ support is a work in progress. These are the current limitations of the implementation:
+
+* Even when using the Singularity runtime, users' container images are expected to be saved in Docker format. Specifying a @.sif@ file as an image when submitting a container request is not yet supported.
+* Arvados' Singularity implementation does not yet limit the amount of memory available in a container. Each container will have access to all memory on the host where it runs, unless memory use is restricted by SLURM/LSF.
+* The Docker ENTRYPOINT instruction is ignored.
+* Arvados is tested with Singularity version 3.7.4. Other versions may not work.

diff --git a/doc/install/singularity.html.textile.liquid b/doc/install/singularity.html.textile.liquid
index d843912e1ba61b082403b9555458c0c31a23af9c..b2a39790d7c7bb1e0b373a6fc394faebd18b385d 100644 (file)
--- a/doc/install/singularity.html.textile.liquid
+++ b/doc/install/singularity.html.textile.liquid
@@ -11,19 +11,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0
 
 h2(#overview). Overview
 
-Arvados can be configured to use "Singularity":https://sylabs.io/singularity/ instead of Docker to execute containers on cloud nodes or a SLURM/LSF cluster. Singularity may be preferable due to its simpler installation and lack of long-running daemon process and special system users/groups.
-
-*Current limitations*:
-* Even when using the singularity runtime, users' container images are expected to be saved in Docker format using @arv keep docker@. Arvados converts the Docker image to Singularity format (@.sif@) at runtime as needed. Specifying a @.sif@ file as an image when submitting a container request is not yet supported.
-* Singularity does not limit the amount of memory available in a container. Each container will have access to all memory on the host where it runs, unless memory use is restricted by SLURM/LSF.
-* Programs running in containers may behave differently due to differences between Singularity and Docker.
-** The root (image) filesystem is read-only in a Singularity container. Programs that attempt to write outside a designated output or temporary directory are likely to fail.
-** The Docker ENTRYPOINT instruction is ignored.
-* Arvados is tested with Singularity version 3.7.4. Other versions may not work.
-
-*Notes*:
-
-* Docker images are converted on the fly by @mksquashfs@, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in @/etc/singularity/singularity.conf@ with a line like @mksquashfs mem = 512M@. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory.
+Please refer to the "Singularity":{{site.baseurl}}/architecture/singularity.html documentation in the Architecture section.
 
 h2(#configuration). Configuration
 
@@ -47,3 +35,7 @@ Then update @Containers.RuntimeEngine@ in your cluster configuration:
 </notextile>
 
 Restart your dispatcher (@crunch-dispatch-slurm@, @arvados-dispatch-cloud@, or @arvados-dispatch-lsf@) after updating your configuration file.
+
+h2(#singularity_configuration). Singularity configuration
+
+Docker images are converted on the fly by @mksquashfs@, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in @/etc/singularity/singularity.conf@ with a line like @mksquashfs mem = 512M@. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory.