From: Tom Clegg Date: Tue, 10 May 2022 14:04:34 +0000 (-0400) Subject: Merge branch '19095-singularity-bugfix' X-Git-Tag: 2.5.0~183 X-Git-Url: https://git.arvados.org/arvados.git/commitdiff_plain/16b445f3d52952d284dbaac603e70a7196d9a3e9?hp=9cdec66d5b0238e9d99c1e8e1d5c8571d798689d Merge branch '19095-singularity-bugfix' fixes #19095 Arvados-DCO-1.1-Signed-off-by: Tom Clegg --- diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid index 226c8667d6..227a8cf07b 100644 --- a/doc/admin/upgrading.html.textile.liquid +++ b/doc/admin/upgrading.html.textile.liquid @@ -32,6 +32,10 @@ h2(#main). development main (as of 2022-04-08) "previous: Upgrading to 2.4.0":#v2_4_0 +h3. Now recommending Singularity 3.9.9 + +The compute image "build script":{{site.baseurl}}/install/crunch2-cloud/install-compute-node.html now installs Singularity 3.9.9 instead of 3.7.4. The newer version includes a bugfix that should resolve "intermittent loopback device errors":https://dev.arvados.org/issues/18489 when running containers. + h2(#v2_4_0). v2.4.0 (2022-04-08) "previous: Upgrading to 2.3.1":#v2_3_1 diff --git a/doc/architecture/singularity.html.textile.liquid b/doc/architecture/singularity.html.textile.liquid index 9a82cd93d6..8ebc6f73df 100644 --- a/doc/architecture/singularity.html.textile.liquid +++ b/doc/architecture/singularity.html.textile.liquid @@ -32,4 +32,4 @@ Arvados @Singularity@ support is a work in progress. These are the current limit * Even when using the Singularity runtime, users' container images are expected to be saved in Docker format. Specifying a @.sif@ file as an image when submitting a container request is not yet supported. * Arvados' Singularity implementation does not yet limit the amount of memory available in a container. Each container will have access to all memory on the host where it runs, unless memory use is restricted by Slurm/LSF. * The Docker ENTRYPOINT instruction is ignored. -* Arvados is tested with Singularity version 3.7.4. Other versions may not work. +* Arvados is tested with Singularity version 3.9.9. Other versions may not work. diff --git a/doc/install/crunch2/install-compute-node-singularity.html.textile.liquid b/doc/install/crunch2/install-compute-node-singularity.html.textile.liquid index e61b6cbe37..2afdf8a919 100644 --- a/doc/install/crunch2/install-compute-node-singularity.html.textile.liquid +++ b/doc/install/crunch2/install-compute-node-singularity.html.textile.liquid @@ -36,13 +36,13 @@ This page describes how to configure a compute node so that it can be used to ru h2(#singularity). Set up Singularity -Follow the "Singularity installation instructions":https://sylabs.io/guides/3.7/user-guide/quick_start.html. Make sure @singularity@ and @mksquashfs@ are working: +Follow the "Singularity installation instructions":https://sylabs.io/guides/3.9/user-guide/quick_start.html. Make sure @singularity@ and @mksquashfs@ are working: 
$ singularity version
-3.7.4
+3.9.9
 $ mksquashfs -version
-mksquashfs version 4.3-git (2014/06/09)
+mksquashfs version 4.4 (2019/08/29)
 [...]
@@ -56,20 +56,3 @@ Then update @Containers.RuntimeEngine@ in your cluster configuration: {% include 'singularity_mksquashfs_configuration' %} - -h2(#singularity_loop_device_errors). Singularity loop device errors - -With singularity v3.9.1 and earlier, containers may fail intermittently at startup with an error message similar to the following in the container log's @stderr.txt@ (line breaks added): - - -
FATAL:   container creation failed:
- mount /proc/self/fd/3->/usr/local/var/singularity/mnt/session/rootfs error:
- while mounting image /proc/self/fd/3:
- failed to find loop device:
- could not attach image file to loop device:
- failed to set loop flags on loop device:
- resource temporarily unavailable
-
- - -This problem is addressed in singularity v3.9.2. For details, please see "Arvados issue #18489":https://dev.arvados.org/issues/18489 and "singularity PR #458":https://github.com/sylabs/singularity/pull/458. diff --git a/lib/install/deps.go b/lib/install/deps.go index 62b295a1f1..cdf28e09c6 100644 --- a/lib/install/deps.go +++ b/lib/install/deps.go @@ -34,7 +34,7 @@ const goversion = "1.17.7" const ( rubyversion = "2.7.5" bundlerversion = "2.2.19" - singularityversion = "3.7.4" + singularityversion = "3.9.9" pjsversion = "1.9.8" geckoversion = "0.24.0" gradleversion = "5.3.1" @@ -138,7 +138,6 @@ func (inst *installCommand) RunCommand(prog string, args []string, stdin io.Read pkgs = append(pkgs, "dpkg-dev", "eatmydata", // install it for later steps, even if we're not using it now - "rsync", ) } @@ -185,6 +184,7 @@ func (inst *installCommand) RunCommand(prog string, args []string, stdin io.Read "r-cran-markdown", "r-cran-roxygen2", "r-cran-xml", + "rsync", "sudo", "uuid-dev", "wget", diff --git a/tools/compute-images/scripts/base.sh b/tools/compute-images/scripts/base.sh index 2bc41e3473..c9fd0945d2 100644 --- a/tools/compute-images/scripts/base.sh +++ b/tools/compute-images/scripts/base.sh @@ -120,7 +120,7 @@ rm -rf /var/lib/arvados/go/ curl -s https://storage.googleapis.com/golang/go${GOVERSION}.linux-amd64.tar.gz | tar -C /var/lib/arvados -xzf - ln -sf /var/lib/arvados/go/bin/* /usr/local/bin/ -singularityversion=3.7.4 +singularityversion=3.9.9 curl -Ls https://github.com/sylabs/singularity/archive/refs/tags/v${singularityversion}.tar.gz | tar -C /var/lib/arvados -xzf - cd /var/lib/arvados/singularity-${singularityversion}