X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/f3e3a6cc4b72120f856e14f3039b1c0c1f0326bd..3d76d16c84d60978f9c9006b1e0a5b165f783170:/doc/_includes/_install_compute_docker.liquid diff --git a/doc/_includes/_install_compute_docker.liquid b/doc/_includes/_install_compute_docker.liquid index fd5d88a9c3..a6f2515abb 100644 --- a/doc/_includes/_install_compute_docker.liquid +++ b/doc/_includes/_install_compute_docker.liquid @@ -10,18 +10,21 @@ Linux can report what compute resources are used by processes in a specific cgro To enable cgroups accounting, you must boot Linux with the command line parameters @cgroup_enable=memory swapaccount=1@. +Currently Arvados is not compatible with the new cgroups accounting, also known as cgroups v2. Currently, all supported GNU/Linux distributions don't use cgroups v2 as default +If you are using a distribution in the compute nodes that ships with cgroups v2 enabled, make sure to disable it by booting Linux with the command line parameters @systemd.unified_cgroup_hierarchy=0@. + After making changes, reboot the system to make these changes effective. h3. Red Hat and CentOS -
~$ sudo grubby --update-kernel=ALL --args='cgroup_enable=memory swapaccount=1'
+
~$ sudo grubby --update-kernel=ALL --args='cgroup_enable=memory swapaccount=1 systemd.unified_cgroup_hierarchy=0'
 
h3. Debian and Ubuntu -Open the file @/etc/default/grub@ in an editor. Find where the string @GRUB_CMDLINE_LINUX@ is set. Add @cgroup_enable=memory swapaccount=1@ to that string. Save the file and exit the editor. Then run: +Open the file @/etc/default/grub@ in an editor. Find where the string @GRUB_CMDLINE_LINUX@ is set. Add @cgroup_enable=memory swapaccount=1 systemd.unified_cgroup_hierarchy=0@ to that string. Save the file and exit the editor. Then run:
~$ sudo update-grub
@@ -48,3 +51,11 @@ For information about how to set configuration options for the Docker daemon, se
 h3. Changing ulimits
 
 Docker containers inherit ulimits from the Docker daemon.  However, the ulimits for a single Unix daemon may not accommodate a long-running Crunch job.  You may want to increase default limits for compute containers by passing @--default-ulimit@ options to the Docker daemon.  For example, to allow containers to open 10,000 files, set @--default-ulimit nofile=10000:10000@.
+
+h2. Troubleshooting
+
+h3. Workflows fail with @ValidationException: Not found: '/var/lib/cwl/workflow.json#main'@
+
+A possible configuration error is having Docker installed as a @snap@ package rather than a @deb@ package.  This is a problem because @snap@ packages are partially containerized and may have a different view of the filesystem than @crunch-run@.  This will produce confusing problems, for example, directory bind mounts sent to Docker that are empty (instead of containing the intended files) and resulting in unexpected "file not found" errors.
+
+To check for this situation, run @snap list@ and look for @docker@.  If found, run @snap remove docker@ and follow the instructions to above to "install Docker Engine":#install_docker .