X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/19332745d7d2d56b7583a930289e0559adac93da..3b321249456939079404973d40ae7e999872c963:/doc/install/install-dispatch-cloud.html.textile.liquid diff --git a/doc/install/install-dispatch-cloud.html.textile.liquid b/doc/install/install-dispatch-cloud.html.textile.liquid index d622576509..7bff6a4a24 100644 --- a/doc/install/install-dispatch-cloud.html.textile.liquid +++ b/doc/install/install-dispatch-cloud.html.textile.liquid @@ -9,6 +9,10 @@ Copyright (C) The Arvados Authors. All rights reserved. SPDX-License-Identifier: CC-BY-SA-3.0 {% endcomment %} +{% include 'notebox_begin_warning' %} +arvados-dispatch-cloud is only relevant for cloud installations. Skip this section if you are installing a on premise cluster that will spool jobs to Slurm. +{% include 'notebox_end' %} + # "Introduction":#introduction # "Create compute node VM image":#create-image # "Update config.yml":#update-config @@ -23,17 +27,18 @@ The cloud dispatch service is for running containers on cloud VMs. It works with The cloud dispatch service can run on any node that can connect to the Arvados API service, the cloud provider's API, and the SSH service on cloud VMs. It is not resource-intensive, so you can run it on the API server node. -*Only one dispatch process should be running at a time.* If you are migrating a system that currently runs @crunch-dispatch-slurm@, it is safest to remove the @crunch-dispatch-slurm@ service entirely before installing @arvados-dispatch-cloud@. +h2(#create-image). Create compute node VM image and configure resolver - -
~$ sudo systemctl --now disable crunch-dispatch-slurm
-~$ sudo apt-get remove crunch-dispatch-slurm
-
- +Set up a VM following the steps "to set up a compute node":crunch2-slurm/install-compute-node.html -h2(#create-image). Create compute node VM image +Compute nodes must be able to resolve the hostnames of the API server and any keepstore servers to your internal IP addresses. You can do this by running an internal DNS resolver and configuring the compute VMs to use that resolver, or by hardcoding the services in the @/etc/hosts@ file. For example: -Create a VM image following the steps "to set up a compute node":crunch2-slurm/install-compute-node.html +
10.20.30.40     ClusterID.example.com
+10.20.30.41     keep1.ClusterID.example.com
+10.20.30.42     keep2.ClusterID.example.com
+
 + +Once the VM is fully configured, create a reusable VM image from it and make note of the image id. h2(#update-config). Update config.yml @@ -68,7 +73,7 @@ You can delete the key files after you have copied the private key to your confi h3. Configure CloudVMs -Add or update the following portions of your cluster configuration file, @/etc/arvados/config.yml@. Refer to "config.defaults.yml":{{site.baseurl}}/admin/config.html for information about additional configuration options. +Add or update the following portions of your cluster configuration file, @config.yml@. Refer to "config.defaults.yml":{{site.baseurl}}/admin/config.html for information about additional configuration options. 
    Services:
@@ -109,7 +114,7 @@ Add or update the following portions of your cluster configuration file, @/etc/a
-Minimal configuration example for Amazon EC2: +h4. Minimal configuration example for Amazon EC2 
    Containers:
@@ -117,18 +122,18 @@ Minimal configuration example for Amazon EC2:
         ImageID: ami-01234567890abcdef
         Driver: ec2
         DriverParameters:
-          AccessKeyID: EALMF21BJC7MKNF9FVVR
-          SecretAccessKey: yKJAPmoCQOMtYWzEUQ1tKTyrocTcbH60CRvGP3pM
+          AccessKeyID: XXXXXXXXXXXXXXXXXXXX
+          SecretAccessKey: YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
           SecurityGroupIDs:
           - sg-0123abcd
           SubnetID: subnet-0123abcd
           Region: us-east-1
           EBSVolumeType: gp2
-          AdminUsername: debian
+          AdminUsername: arvados
-Minimal configuration example for Azure: +h4. Minimal configuration example for Azure 
    Containers:
@@ -138,7 +143,7 @@ Minimal configuration example for Azure:
         DriverParameters:
           SubscriptionID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
           ClientID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
-          ClientSecret: 2WyXt0XFbEtutnf2hp528t6Wk9S5bOHWkRaaWwavKQo=
+          ClientSecret: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
           TenantID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
           CloudEnvironment: AzurePublicCloud
           ResourceGroup: zzzzz
@@ -152,6 +157,41 @@ Minimal configuration example for Azure:
+Get the @SubscriptionID@ and @TenantID@: + +
+$ az account list
+[
+  {
+    "cloudName": "AzureCloud",
+    "id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX",
+    "isDefault": true,
+    "name": "Your Subscription",
+    "state": "Enabled",
+    "tenantId": "YYYYYYYY-YYYY-YYYY-YYYYYYYY",
+    "user": {
+      "name": "you@example.com",
+      "type": "user"
+    }
+  }
+]
+
+ +You will need to create a "service principal" to use as a delegated authority for API access. + +
$ az ad app create --display-name "Arvados Dispatch Cloud (ClusterID)" --homepage "https://arvados.org" --identifier-uris "https://ClusterID.example.com" --end-date 2299-12-31 --password Your_Password
+$ az ad sp create "appId"
+(appId is part of the response of the previous command)
+$ az role assignment create --assignee "objectId" --role Owner --scope /subscriptions/{subscriptionId}/
+(objectId is part of the response of the previous command)
+
 + +Now update your @config.yml@ file: + +@ClientID@ is the 'appId' value. + +@ClientSecret@ is what was provided as Your_Password. + h3. Test your configuration Run the @cloudtest@ tool to verify that your configuration works. This creates a new cloud VM, confirms that it boots correctly and accepts your configured SSH private key, and shuts it down.