{% include 'notebox_end' %}
# "Introduction":#introduction
+# "Install Packer":#install-packer
# "Create an SSH keypair":#sshkeypair
# "Compute image requirements":#requirements
# "The build script":#building
* credentials for your cloud account
* configuration details for your cloud account
+h2(#install-packer). Install Packer
+
+"Download Packer here":https://developer.hashicorp.com/packer/downloads
h2(#sshkeypair). Create a SSH keypair
Options:
- --json-file (required)
- Path to the packer json file
- --arvados-cluster-id (required)
- The ID of the Arvados cluster, e.g. zzzzz
- --aws-profile (default: false)
- AWS profile to use (valid profile from ~/.aws/config
- --aws-secrets-file (default: false, required if building for AWS)
- AWS secrets file which will be sourced from this script
- --aws-source-ami (default: false, required if building for AWS)
- The AMI to use as base for building the images
- --aws-region (default: us-east-1)
+ --json-file <path>
+ Path to the packer json file (required)
+ --arvados-cluster-id <xxxxx>
+ The ID of the Arvados cluster, e.g. zzzzz(required)
+ --aws-profile <profile>
+ AWS profile to use (valid profile from ~/.aws/config (optional)
+ --aws-secrets-file <path>
+ AWS secrets file which will be sourced from this script (optional)
+ When building for AWS, either an AWS profile or an AWS secrets file
+ must be provided.
+ --aws-source-ami <ami-xxxxxxxxxxxxxxxxx>
+ The AMI to use as base for building the images (required if building for AWS)
+ --aws-region <region> (default: us-east-1)
The AWS region to use for building the images
- --aws-vpc-id (optional)
- VPC id for AWS, otherwise packer will pick the default one
- --aws-subnet-id
- Subnet id for AWS otherwise packer will pick the default one for the VPC
- --aws-ebs-autoscale (default: false)
- Install the AWS EBS autoscaler daemon.
- --gcp-project-id (default: false, required if building for GCP)
- GCP project id
- --gcp-account-file (default: false, required if building for GCP)
- GCP account file
- --gcp-zone (default: us-central1-f)
+ --aws-vpc-id <vpc-id>
+ VPC id for AWS, if not specified packer will derive from the subnet id or pick the default one.
+ --aws-subnet-id <subnet-xxxxxxxxxxxxxxxxx>
+ Subnet id for AWS, if not specified packer will pick the default one for the VPC.
+ --aws-ebs-autoscale
+ Install the AWS EBS autoscaler daemon (default: do not install the AWS EBS autoscaler).
+ --aws-associate-public-ip <true|false>
+ Associate a public IP address with the node used for building the compute image.
+ Required when the machine running packer can not reach the node used for building
+ the compute image via its private IP. (default: true if building for AWS)
+ Note: if the subnet has "Auto-assign public IPv4 address" enabled, disabling this
+ flag will have no effect.
+ --aws-ena-support <true|false>
+ Enable enhanced networking (default: true if building for AWS)
+ --gcp-project-id <project-id>
+ GCP project id (required if building for GCP)
+ --gcp-account-file <path>
+ GCP account file (required if building for GCP)
+ --gcp-zone <zone> (default: us-central1-f)
GCP zone
- --azure-secrets-file (default: false, required if building for Azure)
- Azure secrets file which will be sourced from this script
- --azure-resource-group (default: false, required if building for Azure)
- Azure resource group
- --azure-location (default: false, required if building for Azure)
- Azure location, e.g. centralus, eastus, westeurope
- --azure-sku (default: unset, required if building for Azure, e.g. 16.04-LTS)
+ --azure-secrets-file <patch>
+ Azure secrets file which will be sourced from this script (required if building for Azure)
+ --azure-resource-group <resouce-group>
+ Azure resource group (required if building for Azure)
+ --azure-location <location>
+ Azure location, e.g. centralus, eastus, westeurope (required if building for Azure)
+ --azure-sku <sku> (required if building for Azure, e.g. 16.04-LTS)
Azure SKU image to use
- --ssh_user (default: packer)
+ --ssh_user <user> (default: packer)
The user packer will use to log into the image
- --resolver (default: host's network provided)
- The dns resolver for the machine
- --reposuffix (default: unset)
+ --resolver <resolver_IP>
+ The dns resolver for the machine (default: host's network provided)
+ --reposuffix <suffix>
Set this to "-dev" to track the unstable/dev Arvados repositories
- --public-key-file (required)
- Path to the public key file that a-d-c will use to log into the compute node
+ --public-key-file <path>
+ Path to the public key file that a-d-c will use to log into the compute node (required)
--mksquashfs-mem (default: 256M)
Only relevant when using Singularity. This is the amount of memory mksquashfs is allowed to use.
- --nvidia-gpu-support (default: false)
- Install all the necessary tooling for Nvidia GPU support
- --debug (default: false)
- Output debug information
+ --nvidia-gpu-support
+ Install all the necessary tooling for Nvidia GPU support (default: do not install Nvidia GPU support)
+ --debug
+ Output debug information (default: no debug output is printed)
</code></pre></notextile>
h2(#dns-resolution). DNS resolution
-Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. You can do this by running an internal DNS resolver. The IP address of the resolver should be passed as the value for the @--resolver@ argument to "the build script":#building.
+Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. If you are on AWS and using Route 53 for your DNS, the default resolver configuration can be used with no extra options.
+
+You can also run your own internal DNS resolver. In that case, the IP address of the resolver should be passed as the value for the @--resolver@ argument to "the build script":#building.
-Alternatively, the services could be hardcoded into an @/etc/hosts@ file. For example:
+As a third option, the services could be hardcoded into an @/etc/hosts@ file. For example:
<notextile><pre><code>10.20.30.40 <span class="userinput">ClusterID.example.com</span>
10.20.30.41 <span class="userinput">keep1.ClusterID.example.com</span>
h2(#aws). Build an AWS image
+For @ClusterID@, fill in your cluster ID.
+
+@AWSProfile@ is the name of an AWS profile in your "credentials file":https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#shared-credentials-file (@~/.aws/credentials@) listing the @aws_access_key_id@ and @aws_secret_access_key@ to use.
+
+The @AMI@ is the identifier for the base image to be used. Current AMIs are maintained by "Debian":https://wiki.debian.org/Cloud/AmazonEC2Image/Buster and "Ubuntu":https://cloud-images.ubuntu.com/locator/ec2/.
+
+The @VPC@ and @Subnet@ should be configured for where you want the compute image to be generated and stored.
+
+@ArvadosDispatchCloudPublicKeyPath@ should be replaced with the path to the ssh *public* key file generated in "Create an SSH keypair":#sshkeypair, above.
+
<notextile><pre><code>~$ <span class="userinput">./build.sh --json-file arvados-images-aws.json \
--arvados-cluster-id ClusterID \
--aws-profile AWSProfile \
--aws-vpc-id VPC \
--aws-subnet-id Subnet \
--ssh_user admin \
- --resolver ResolverIP \
--public-key-file ArvadosDispatchCloudPublicKeyPath
</span>
</code></pre></notextile>
-For @ClusterID@, fill in your cluster ID. The @VPC@ and @Subnet@ should be configured for where you want the compute image to be generated and stored. The @AMI@ is the identifier for the base image to be used. Current AMIs are maintained by "Debian":https://wiki.debian.org/Cloud/AmazonEC2Image/Buster and "Ubuntu":https://cloud-images.ubuntu.com/locator/ec2/.
-
-@AWSProfile@ should be replaced with the name of an AWS profile with sufficient permissions to create the image.
-
-@ArvadosDispatchCloudPublicKeyPath@ should be replaced with the path to the ssh *public* key file generated in "Create an SSH keypair":#sshkeypair, above.
h3(#aws-ebs-autoscaler). Autoscaling compute node scratch space
-If you want to add the "AWS EBS autoscaler":https://github.com/awslabs/amazon-ebs-autoscale daemon in your images, add the @--aws-ebs-autoscale@ flag to the "the build script":#building. Doing so will make the compute image scratch space scale automatically as needed.
+Arvados supports "AWS EBS autoscaler":https://github.com/awslabs/amazon-ebs-autoscale. This feature automatically expands the scratch space on the compute node on demand by 200 GB at a time, up to 5 TB.
+
+If you want to add the daemon in your images, add the @--aws-ebs-autoscale@ flag to the "the build script":#building.
The AWS EBS autoscaler daemon will be installed with this configuration:
}
</code></pre></notextile>
-Changing the configuration is left as an exercise for the reader.
-
-Using this feature also requires a few Arvados configuration changes in @config.yml@:
-
-* The @Containers/InstanceTypes@ list should be modified so that all @AddedScratch@ lines are removed, and the @IncludedScratch@ value should be set to a (fictional) high number. This way, the scratch space requirements will be met by all the defined instance type. For example:
-
-<notextile><pre><code> InstanceTypes:
- c5large:
- ProviderType: c5.large
- VCPUs: 2
- RAM: 4GiB
- IncludedScratch: 16TB
- Price: 0.085
- m5large:
- ProviderType: m5.large
- VCPUs: 2
- RAM: 8GiB
- IncludedScratch: 16TB
- Price: 0.096
-...
-</code></pre></notextile>
-
-* You will also need to create an IAM role in AWS with these permissions:
-
-<notextile><pre><code>{
- "Version": "2012-10-17",
- "Statement": [
- {
- "Effect": "Allow",
- "Action": [
- "ec2:AttachVolume",
- "ec2:DescribeVolumeStatus",
- "ec2:DescribeVolumes",
- "ec2:DescribeTags",
- "ec2:ModifyInstanceAttribute",
- "ec2:DescribeVolumeAttribute",
- "ec2:CreateVolume",
- "ec2:DeleteVolume",
- "ec2:CreateTags"
- ],
- "Resource": "*"
- }
- ]
-}
-</code></pre></notextile>
+Changing the ebs-autoscale configuration is left as an exercise for the reader.
-Then, in @config.yml@ set @Containers/CloudVMs/DriverParameters/IAMInstanceProfile@ to the name of the IAM role. This will make @arvados-dispatch-cloud@ pass an IAMInstanceProfile to the compute nodes as they start up, giving them sufficient permissions to attach and grow EBS volumes.
+This feature also requires a few Arvados configuration changes, described in "EBS Autoscale configuration":install-dispatch-cloud.html#aws-ebs-autoscaler.
h2(#azure). Build an Azure image
projects / arvados.git / blobdiff