From: Ward Vandewege Date: Mon, 21 Feb 2022 14:32:08 +0000 (-0500) Subject: 18772: document the AWS EBS autoscaler support. Refactor the "Build a X-Git-Url: https://git.arvados.org/arvados.git/commitdiff_plain/3fac6017ba16634a38d80a49e1ae1a47585e95b9 18772: document the AWS EBS autoscaler support. Refactor the "Build a cloud compute node image" documentation page to improve the flow. Arvados-DCO-1.1-Signed-off-by: Ward Vandewege --- diff --git a/doc/install/crunch2-cloud/install-compute-node.html.textile.liquid b/doc/install/crunch2-cloud/install-compute-node.html.textile.liquid index 89771514e9..979bbad25b 100644 --- a/doc/install/crunch2-cloud/install-compute-node.html.textile.liquid +++ b/doc/install/crunch2-cloud/install-compute-node.html.textile.liquid @@ -17,8 +17,11 @@ SPDX-License-Identifier: CC-BY-SA-3.0 # "Create an SSH keypair":#sshkeypair # "Compute image requirements":#requirements # "The build script":#building +# "DNS resolution":#dns-resolution +# "NVIDIA GPU support":#nvidia # "Singularity mksquashfs configuration":#singularity_mksquashfs_configuration # "Build an AWS image":#aws +## "Autoscaling compute node scratch space":#aws-ebs-autoscaler # "Build an Azure image":#azure h2(#introduction). Introduction @@ -56,12 +59,6 @@ foktmqOY8MyctzFgXBpGTxPliGjqo8OkrOyQP2g+FL7v+Km31Xs61P8= -{% assign show_docker_warning = true %} - -{% include 'singularity_mksquashfs_configuration' %} - -The desired amount of memory to make available for @mksquashfs@ can be configured in an argument to "the build script":#building. It defaults to @256M@. - h2(#requirements). Compute image requirements Arvados comes with a build script to automate the creation of a suitable compute node image (see "The build script":#building below). It is provided as a convenience. It is also possible to create a compute node image via other means. These are the requirements: @@ -101,6 +98,8 @@ Options: VPC id for AWS, otherwise packer will pick the default one --aws-subnet-id Subnet id for AWS otherwise packer will pick the default one for the VPC + --aws-ebs-autoscale (default: false) + Install the AWS EBS autoscaler daemon. --gcp-project-id (default: false, required if building for GCP) GCP project id --gcp-account-file (default: false, required if building for GCP) @@ -131,10 +130,29 @@ Options: Output debug information -h2(#building). NVIDIA GPU support +h2(#dns-resolution). DNS resolution + +Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. You can do this by running an internal DNS resolver. The IP address of the resolver should be passed as the value for the @--resolver@ argument to "the build script":#building. + +Alternatively, the services could be hardcoded into an @/etc/hosts@ file. For example: + +
10.20.30.40     ClusterID.example.com
+10.20.30.41     keep1.ClusterID.example.com
+10.20.30.42     keep2.ClusterID.example.com
+
 + +Adding these lines to the @/etc/hosts@ file in the compute node image could be done with a small change to the Packer template and the @scripts/base.sh@ script, which will be left as an exercise for the reader. + +h2(#nvidia). NVIDIA GPU support If you plan on using instance types with NVIDIA GPUs, add @--nvidia-gpu-support@ to the build command line. Arvados uses the same compute image for both GPU and non-GPU instance types. The GPU tooling is ignored when using the image with a non-GPU instance type. +{% assign show_docker_warning = true %} + +{% include 'singularity_mksquashfs_configuration' %} + +The desired amount of memory to make available for @mksquashfs@ can be configured in an argument to "the build script":#building. It defaults to @256M@. + h2(#aws). Build an AWS image 
~$ ./build.sh --json-file arvados-images-aws.json \
@@ -155,17 +173,26 @@ For @ClusterID@, fill in your cluster ID. The @VPC@ and @Subnet@ should be confi
 
 @ArvadosDispatchCloudPublicKeyPath@ should be replaced with the path to the ssh *public* key file generated in "Create an SSH keypair":#sshkeypair, above.
 
-Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. You can do this by running an internal DNS resolver. The IP address of the resolver should replace the string @ResolverIP@ in the command above.
-
-Alternatively, the services could be hardcoded into an @/etc/hosts@ file. For example:
-
-
10.20.30.40     ClusterID.example.com
-10.20.30.41     keep1.ClusterID.example.com
-10.20.30.42     keep2.ClusterID.example.com
+h3(#aws-ebs-autoscaler). Autoscaling compute node scratch space
+
+If you want to add the AWS EBS autoscaler daemon in your images, add the @--aws-ebs-autoscale@ flag to the "the build script":#building. Doing so will make the compute image scratch space scale automatically as needed. The @Containers/InstanceTypes@ list should be modified so that all @AddedScratch@ lines are removed, and the @IncludedScratch@ value should be set to a (fictional) high number. This way, the scratch space requirements will be met by all the defined instance type. For example:
+
+
    InstanceTypes:
+      c5large:
+        ProviderType: c5.large
+        VCPUs: 2
+        RAM: 4GiB
+        IncludedScratch: 16TB
+        Price: 0.085
+      m5large:
+        ProviderType: m5.large
+        VCPUs: 2
+        RAM: 8GiB
+        IncludedScratch: 16TB
+        Price: 0.096
+...
 

 
-Adding these lines to the @/etc/hosts@ file in the compute node image could be done with a small change to the Packer template and the @scripts/base.sh@ script, which will be left as an exercise for the reader.
-
 h2(#azure). Build an Azure image
 
 
~$ ./build.sh --json-file arvados-images-azure.json \
@@ -195,14 +222,3 @@ These secrets can be generated from the Azure portal, or with the cli using a co
 

 
 @ArvadosDispatchCloudPublicKeyPath@ should be replaced with the path to the ssh *public* key file generated in "Create an SSH keypair":#sshkeypair, above.
-
-Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. You can do this by running an internal DNS resolver. The IP address of the resolver should replace the string @ResolverIP@ in the command above.
-
-Alternatively, the services could be hardcoded into an @/etc/hosts@ file. For example:
-
-
10.20.30.40     ClusterID.example.com
-10.20.30.41     keep1.ClusterID.example.com
-10.20.30.42     keep2.ClusterID.example.com
-

-
-Adding these lines to the @/etc/hosts@ file in the compute node image could be done with a small change to the Packer template and the @scripts/base.sh@ script, which will be left as an exercise for the reader.