- user/getting_started/workbench.html.textile.liquid
- user/tutorials/tutorial-workflow-workbench.html.textile.liquid
- user/composer/composer.html.textile.liquid
+ - Working at the Command Line:
+ - user/getting_started/setup-cli.html.textile.liquid
+ - user/reference/api-tokens.html.textile.liquid
+ - user/getting_started/check-environment.html.textile.liquid
- Access an Arvados virtual machine:
- user/getting_started/vm-login-with-webshell.html.textile.liquid
- user/getting_started/ssh-access-unix.html.textile.liquid
- user/getting_started/ssh-access-windows.html.textile.liquid
- - user/getting_started/check-environment.html.textile.liquid
- - user/reference/api-tokens.html.textile.liquid
- Working with data sets:
- user/tutorials/tutorial-keep.html.textile.liquid
- user/tutorials/tutorial-keep-get.html.textile.liquid
- user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid
- user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid
- user/tutorials/tutorial-keep-mount-windows.html.textile.liquid
- - user/topics/keep.html.textile.liquid
- user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid
- user/topics/arv-copy.html.textile.liquid
- - user/topics/storage-classes.html.textile.liquid
- user/topics/collection-versioning.html.textile.liquid
- - Working with git repositories:
- - user/tutorials/add-new-repository.html.textile.liquid
- - user/tutorials/git-arvados-guide.html.textile.liquid
+ - user/topics/storage-classes.html.textile.liquid
+ - user/topics/keep.html.textile.liquid
- Running workflows at the command line:
- user/cwl/cwl-runner.html.textile.liquid
- user/cwl/cwl-run-options.html.textile.liquid
- user/cwl/cwl-extensions.html.textile.liquid
- user/cwl/cwl-versions.html.textile.liquid
- user/topics/arv-docker.html.textile.liquid
+ - Working with git repositories:
+ - user/tutorials/add-new-repository.html.textile.liquid
+ - user/tutorials/git-arvados-guide.html.textile.liquid
- Reference:
- user/topics/link-accounts.html.textile.liquid
- user/reference/cookbook.html.textile.liquid
h1(#login). Using SSH to log into an Arvados VM
-To see a list of virtual machines that you have access to and determine the name and login information, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu and click on the menu item *Virtual machines* to go to the Virtual machines page. This page lists the virtual machines you can access. The *Host name* column lists the name of each available VM. The *Login name* column will have a list of comma separated values of the form @you@. In this guide the hostname will be *_shell_* and the login will be *_you_*. Replace these with your hostname and login name as appropriate.
+To see a list of virtual machines that you have access to, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu, then click on the menu item *Virtual machines* to go to the Virtual machines page.
+This page lists the virtual machines you can access. The *Host name* column lists the name of each available VM. The *Login name* column lists your login name on that VM. The *Command line* column provides a sample @ssh@ command line.
+At the bottom of the page there may be additional instructions for connecting your specific Arvados instance. If so, follow your site-specific instructions. If there are no site-specific instructions, you can probably connect directly with @ssh@.
+
+The following are generic instructions. In the examples the login will be *_you_* and the hostname will be *_shell.ClusterID.example.com_* and . Replace these with your login name and hostname as appropriate.
{% endcomment %}
{% include 'notebox_begin' %}
-This tutorial assumes you are using the playground Arvados instance, @qr1hi@. If you are using a different instance, replace @qr1hi@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
+This tutorial assumes you are using the playground Arvados instance, @pirca@. If you are using a different instance, replace @pirca@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
{% include 'notebox_end' %}
{% endcomment %}
{% include 'notebox_begin' %}
-This tutorial assumes that you are logged into an Arvados VM instance (instructions for "Webshell":{{site.baseurl}}/user/getting_started/vm-login-with-webshell.html or "Unix":{{site.baseurl}}/user/getting_started/ssh-access-unix.html#login or "Windows":{{site.baseurl}}/user/getting_started/ssh-access-windows.html#login) or you have installed the Arvados "FUSE Driver":{{site.baseurl}}/sdk/python/arvados-fuse.html and "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html on your workstation and have a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html
+This tutorial assumes that you have access to the "Arvados command line tools":/user/getting_started/setup-cli.html and have set the "API token":{{site.baseurl}}/user/reference/api-tokens.html and confirmed a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html .
{% include 'notebox_end' %}
--- /dev/null
+#!/usr/bin/env cwl-runner
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+cwlVersion: v1.0
+class: CommandLineTool
+inputs: []
+outputs: []
+arguments: ["echo", "hello world!"]
Cluster identifiers are mapped API server hosts one of two ways:
-* Through DNS resolution, under the @arvadosapi.com@ domain. For example, the API server for the cluster @qr1hi@ can be found at @qr1hi.arvadosapi.com@. To register a cluster id for free under @arvadosapi.com@, contact "info@curii.com":mailto:info@curii.com
+* Through DNS resolution, under the @arvadosapi.com@ domain. For example, the API server for the cluster @pirca@ can be found at @pirca.arvadosapi.com@. To register a cluster id for free under @arvadosapi.com@, contact "info@curii.com":mailto:info@curii.com
* Through explicit configuration:
The @RemoteClusters@ section of @/etc/arvados/config.yml@ (for arvados-controller)
+++ /dev/null
-{
- "name":"GATK / exome PE fastq to snp",
- "components":{
- "extract-reference":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"file-select",
- "script_parameters":{
- "names":[
- "human_g1k_v37.fasta.gz",
- "human_g1k_v37.fasta.fai.gz",
- "human_g1k_v37.dict.gz"
- ],
- "input":"d237a90bae3870b3b033aea1e99de4a9+10820+K@qr1hi"
- },
- "output_name":false
- },
- "bwa-index":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"bwa-index",
- "script_parameters":{
- "input":{
- "output_of":"extract-reference"
- },
- "bwa_tbz":{
- "value":"8b6e2c4916133e1d859c9e812861ce13+70",
- "required":true
- }
- },
- "output_name":false
- },
- "bwa-aln":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"bwa-aln",
- "script_parameters":{
- "input":{
- "dataclass":"Collection",
- "required":"true"
- },
- "reference_index":{
- "output_of":"bwa-index"
- },
- "samtools_tgz":{
- "value":"c777e23cf13e5d5906abfdc08d84bfdb+74",
- "required":true
- },
- "bwa_tbz":{
- "value":"8b6e2c4916133e1d859c9e812861ce13+70",
- "required":true
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- },
- "output_name":false
- },
- "picard-gatk2-prep":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"picard-gatk2-prep",
- "script_parameters":{
- "input":{
- "output_of":"bwa-aln"
- },
- "reference":{
- "output_of":"extract-reference"
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- },
- "output_name":false
- },
- "GATK2-realign":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-realign",
- "script_parameters":{
- "input":{
- "output_of":"picard-gatk2-prep"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K@qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- },
- "regions":{
- "value":"13b53dbe1ec032dfc495fd974aa5dd4a+87/S02972011_Covered_sort_merged.bed"
- },
- "region_padding":{
- "value":10
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":2
- },
- "output_name":false
- },
- "GATK2-bqsr":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-bqsr",
- "script_parameters":{
- "input":{
- "output_of":"GATK2-realign"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K@qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- }
- },
- "output_name":false
- },
- "GATK2-merge-call":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-merge-call",
- "script_parameters":{
- "input":{
- "output_of":"GATK2-bqsr"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K@qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- },
- "regions":{
- "value":"13b53dbe1ec032dfc495fd974aa5dd4a+87/S02972011_Covered_sort_merged.bed"
- },
- "region_padding":{
- "value":10
- },
- "GATK2_UnifiedGenotyper_args":{
- "default":[
- "-stand_call_conf",
- "30.0",
- "-stand_emit_conf",
- "30.0",
- "-dcov",
- "200"
- ]
- }
- },
- "output_name":"Variant calls from UnifiedGenotyper"
- }
- }
-}
+++ /dev/null
-{
- "name":"Real Time Genomics / PE fastq to snp",
- "components":{
- "extract_reference":{
- "script":"file-select",
- "script_parameters":{
- "names":[
- "human_g1k_v37.fasta.gz"
- ],
- "input":"d237a90bae3870b3b033aea1e99de4a9+10820+K@qr1hi"
- },
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2"
- },
- "reformat_reference":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-fasta2sdf",
- "script_parameters":{
- "input":{
- "output_of":"extract_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K@qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- },
- "reformat_reads":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-fastq2sdf",
- "script_parameters":{
- "input":{
- "optional":false
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K@qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- },
- "map_reads":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-map",
- "script_parameters":{
- "input":{
- "output_of":"reformat_reads"
- },
- "reference":{
- "output_of":"reformat_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K@qr1hi",
- "rtg_license":{
- "optional":false
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- }
- },
- "report_snp":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-snp",
- "script_parameters":{
- "input":{
- "output_of":"map_reads"
- },
- "reference":{
- "output_of":"reformat_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K@qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- }
- }
-}
+++ /dev/null
-#!/usr/bin/env ruby
-# Copyright (C) The Arvados Authors. All rights reserved.
-#
-# SPDX-License-Identifier: CC-BY-SA-3.0
-
-abort 'Error: Ruby >= 1.9.3 required.' if RUBY_VERSION < '1.9.3'
-
-require 'arvados'
-
-arv = Arvados.new(api_version: 'v1')
-arv.node.list[:items].each do |node|
- if node[:crunch_worker_state] != 'down'
- ping_age = (Time.now - Time.parse(node[:last_ping_at])).to_i rescue -1
- puts "#{node[:uuid]} #{node[:crunch_worker_state]} #{ping_age}"
- end
-end
public static void main(String[] argv) {
ConfigProvider conf = ExternalConfigProvider.builder().
apiProtocol("https").
- apiHost("qr1hi.arvadosapi.com").
+ apiHost("zzzzz.arvadosapi.com").
apiPort(443).
apiToken("...").
build();
{% codeblock as python %}
import arvados
api = arvados.api()
-container_request_uuid="qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid="zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
print(container_request["mounts"]["/var/lib/cwl/cwl.input.json"])
{% endcodeblock %}
import arvados
import arvados.collection
api = arvados.api()
-container_request_uuid="qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid="zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
collection = arvados.collection.CollectionReader(container_request["output_uuid"])
print(collection.open("cwl.output.json").read())
elif c['runtime_status'].get('warning', None):
return 'Warning'
return c['state']
-container_request_uuid = 'qr1hi-xvhdp-zzzzzzzzzzzzzzz'
+container_request_uuid = 'zzzzz-xvhdp-zzzzzzzzzzzzzzz'
print(get_cr_state(container_request_uuid))
{% endcodeblock %}
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
namefilter = "bwa%" # the "like" filter uses SQL pattern match syntax
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
namefilter = "bwa%" # the "like" filter uses SQL pattern match syntax
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
child_requests = api.container_requests().list(filters=[
import arvados
import arvados.collection
api = arvados.api()
-container_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
collection = arvados.collection.CollectionReader(container_request["log_uuid"])
for c in collection:
import arvados
api = arvados.api()
download="https://your.download.server"
-collection_uuid="qr1hi-4zz18-zzzzzzzzzzzzzzz"
+collection_uuid="zzzzz-4zz18-zzzzzzzzzzzzzzz"
token = api.api_client_authorizations().create(body={"api_client_authorization":{"scopes": [
"GET /arvados/v1/collections/%s" % collection_uuid,
"GET /arvados/v1/collections/%s/" % collection_uuid,
import arvados
import arvados.collection
api = arvados.api()
-project_uuid = "qr1hi-tpzed-zzzzzzzzzzzzzzz"
-collection_uuids = ["qr1hi-4zz18-aaaaaaaaaaaaaaa", "qr1hi-4zz18-bbbbbbbbbbbbbbb"]
+project_uuid = "zzzzz-tpzed-zzzzzzzzzzzzzzz"
+collection_uuids = ["zzzzz-4zz18-aaaaaaaaaaaaaaa", "zzzzz-4zz18-bbbbbbbbbbbbbbb"]
combined_manifest = ""
for u in collection_uuids:
c = api.collections().get(uuid=u).execute()
import arvados
import arvados.collection
-project_uuid = "qr1hi-j7d0g-zzzzzzzzzzzzzzz"
+project_uuid = "zzzzz-j7d0g-zzzzzzzzzzzzzzz"
collection_name = "My collection"
filename = "file1.txt"
import arvados
import arvados.collection
-collection_uuid = "qr1hi-4zz18-zzzzzzzzzzzzzzz"
+collection_uuid = "zzzzz-4zz18-zzzzzzzzzzzzzzz"
filename = "file1.txt"
api = arvados.api()
puts "UUID of first repo returned is #{first_repo[:uuid]}"</code>
{% endcodeblock %}
-UUID of first repo returned is qr1hi-s0uqq-b1bnybpx3u5temz
+UUID of first repo returned is zzzzz-s0uqq-b1bnybpx3u5temz
h2. update
{% include 'tutorial_expectations' %}
-{% include 'notebox_begin' %}
-
-By default, the @arvados-cwl-runner@ is installed on Arvados shell nodes. If you want to submit jobs from somewhere else, such as your workstation, you may install "arvados-cwl-runner.":#setup
-
-{% include 'notebox_end' %}
-
This tutorial will demonstrate how to submit a workflow at the command line using @arvados-cwl-runner@.
h2. Running arvados-cwl-runner
Now you can set up @ssh-agent@ (next) or proceed with "adding your key to the Arvados Workbench.":#workbench
-h3. Set up ssh-agent (recommended)
+h3. Set up ssh-agent (optional)
If you find you are entering your passphrase frequently, you can use @ssh-agent@ to manage your credentials. Use @ssh-add -l@ to test if you already have ssh-agent running:
{% include 'ssh_addkey' %}
-h3. Connecting to the virtual machine
+h3. Connecting directly
-Use the following command to connect to the _shell_ VM instance as _you_. Replace *<code>you@shell</code>* at the end of the following command with your *login* and *hostname* from Workbench:
+If the VM is available on the public Internet (or you are on the same private network as the VM) you can connect directly with @ssh@. You can probably copy-and-paste the text from *Command line* column directly into a terminal.
-notextile. <pre><code>$ <span class="userinput">ssh -o "ProxyCommand ssh -p2222 turnout@switchyard.{{ site.arvados_api_host }} -x -a <b>shell</b>" -x <b>you@shell</b></span></code></pre>
+Use the following example command to connect as _you_ to the _shell.ClusterID.example.com_ VM instance. Replace *<code>you@shell.ClusterID.example.com</code>* at the end of the following command with your *login* and *hostname* from Workbench.
+
+notextile. <pre><code>$ <span class="userinput">ssh <b>you@shell.ClusterID.example.com</b></span></code></pre>
+
+h3. Connecting through switchyard
+
+Some Arvados installations use "switchyard" to isolate shell VMs from the public Internet.
+
+Use the following example command to connect to the _shell_ VM instance as _you_. Replace *<code>you@shell</code>* at the end of the following command with your *login* and *hostname* from Workbench:
+
+notextile. <pre><code>$ <span class="userinput">ssh -o "ProxyCommand ssh -p2222 turnout@switchyard.ClusterID.example.com -x -a <b>shell</b>" -x <b>you@shell</b></span></code></pre>
This command does several things at once. You usually cannot log in directly to virtual machines over the public Internet. Instead, you log into a "switchyard" server and then tell the switchyard which virtual machine you want to connect to.
You should now be able to log into the Arvados VM and "check your environment.":check-environment.html
-h3. Configuration (recommended)
+h4. Configuration (recommended)
The command line above is cumbersome, but you can configure SSH to remember many of these settings. Add this text to the file @.ssh/config@ in your home directory (create a new file if @.ssh/config@ doesn't exist):
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This document is for accessing an Arvados VM using SSH keys in Windows environments. If you would like to use to access VM through your browser, please visit the "Accessing an Arvados VM with Webshell":vm-login-with-webshell.html page. If you are using a Unix environment (Linux, OS X, Cygwin), please visit the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.
+This document is for accessing an Arvados VM using SSH keys in Windows environments using PuTTY. If you would like to use to access VM through your browser, please visit the "Accessing an Arvados VM with Webshell":vm-login-with-webshell.html page. If you are using a Unix environment (Linux, OS X, Cygwin, or Windows Subsystem for Linux), please visit the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.
{% include 'ssh_intro' %}
h1(#gettingkey). Getting your SSH key
-(Note: if you are using the SSH client that comes with "Cygwin":http://cygwin.com, please use instructions found in the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.)
+(Note: If you are using the SSH client that comes with "Cygwin":http://cygwin.com or Windows Subsystem for Linux (WSL) please use instructions found in the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.)
We will be using PuTTY to connect to Arvados. "PuTTY":http://www.chiark.greenend.org.uk/~sgtatham/putty/ is a free (MIT-licensed) Win32 Telnet and SSH client. PuTTY includes all the tools a Windows user needs to create private keys and make SSH connections to your virtual machines in the Arvados Cloud.
h3. Initial configuration
+h4. Connecting directly
+
+# Open PuTTY from the Start Menu.
+# On the Session screen set the Host Name (or IP address) to “shell.ClusterID.example.com”, which is the hostname listed in the _Virtual Machines_ page.
+# On the Session screen set the Port to “22”.
+# On the Connection %(rarr)→% Data screen set the Auto-login username to the username listed in the *Login name* column on the Arvados Workbench Virtual machines_ page.
+# Return to the Session screen. In the Saved Sessions box, enter a name for this configuration and click Save.
+
+h4. Connecting through switchyard
+
# Open PuTTY from the Start Menu.
# On the Session screen set the Host Name (or IP address) to “shell”, which is the hostname listed in the _Virtual Machines_ page.
# On the Session screen set the Port to “22”.
Webshell gives you access to an arvados virtual machine from your browser with no additional setup.
+{% include 'notebox_begin' %}
+Some Arvados clusters may not have webshell set up. If you do not see a "Log in" button or "web shell" column, you will have to follow the "Unix":ssh-access-unix.html or "Windows":ssh-access-windows.html @ssh@ instructions.
+{% include 'notebox_end' %}
+
In the Arvados Workbench, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu, and click on the menu item *Virtual machines* to see the list of virtual machines you can access. If you do not have access to any virtual machines, please click on <span class="btn btn-sm btn-primary">Send request for shell access</span> or send an email to "support@curoverse.com":mailto:support@curoverse.com.
Each row in the Virtual Machines panel lists the hostname of the VM, along with a <code>Log in as *you*</code> button under the column "Web shell". Clicking on this button will open up a webshell terminal for you in a new browser tab and log you in.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-If you are using the default Arvados instance for this guide, you can Access Arvados Workbench using this link:
+{% include 'notebox_begin' %}
+This guide covers the classic Arvados Workbench web application, sometimes referred to as "Workbench 1". There is also a new Workbench web application under development called "Workbench 2". Sites which have both Workbench applications installed will have a dropdown menu option "Switch to Workbench 2" to switch between versions.
+
+This guide will be updated to cover "Workbench 2" in the future.
+{% include 'notebox_end' %}
+
+If you are using the "playground" Arvados instance for this guide, you can Access Arvados Workbench using this link:
<a href="{{site.arvados_workbench_host}}/" target="_blank">{{site.arvados_workbench_host}}/</a>
(If you are using a different Arvados instance than the default for this guide, replace *{{ site.arvados_workbench_host }}* with your private instance in all of the examples in this guide.)
-You may be asked to log in using a Google account. Arvados uses only your name and email address from Google services for identification, and will never access any personal information. If you are accessing Arvados for the first time, the Workbench may indicate your account status is *New / inactive*. If this is the case, contact the administrator of the Arvados instance to request activation of your account.
+You will be asked to log in. Arvados uses only your name and email address for identification, and will never access any personal information. If you are accessing Arvados for the first time, the Workbench may indicate your account status is *New / inactive*. If this is the case, contact the administrator of the Arvados instance to request activation of your account.
Once your account is active, logging in to the Workbench will present you with the Dashboard. This gives a summary of your projects and recent activity in the Arvados instance. "You are now ready to run your first pipeline.":{{ site.baseurl }}/user/tutorials/tutorial-workflow-workbench.html
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-{% include 'crunch1only_begin' %}
-On those sites, the "copy a pipeline template" feature described below is not available. However, "copy a workflow" feature is not yet implemented.
-{% include 'crunch1only_end' %}
-
This tutorial describes how to copy Arvados objects from one cluster to another by using @arv-copy@.
{% include 'tutorial_expectations' %}
h2. arv-copy
-@arv-copy@ allows users to copy collections and pipeline templates from one cluster to another. By default, @arv-copy@ will recursively go through a template and copy all dependencies associated with the object.
+@arv-copy@ allows users to copy collections and workflows from one cluster to another. By default, @arv-copy@ will recursively go through the workflow and copy all dependencies associated with the object.
For example, let's copy from the <a href="https://playground.arvados.org/">Arvados playground</a>, also known as *qr1hi*, to *dst_cluster*. The names *qr1hi* and *dst_cluster* are interchangable with any cluster name. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *qr1hi*-4zz18-tci4vn4fa95w0zx, the cluster name is qr1hi.
</code></pre>
</notextile>
-h3. How to copy a pipeline template
-
-{% include 'arv_copy_expectations' %}
-
-We will use the uuid @qr1hi-p5p6p-9pkaxt6qjnkxhhu@ as an example pipeline template.
-
-<notextile>
-<pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster --dst-git-repo $USER/tutorial qr1hi-p5p6p-9pkaxt6qjnkxhhu</span>
-To git@git.dst_cluster.arvadosapi.com:$USER/tutorial.git
- * [new branch] git_git_qr1hi_arvadosapi_com_arvados_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d -> git_git_qr1hi_arvadosapi_com_arvados_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d
-arvados.arv-copy[19694] INFO: Success: created copy with uuid dst_cluster-p5p6p-rym2h5ub9m8ofwj
-</code></pre>
-</notextile>
-
-New branches in the destination git repo will be created for each branch used in the pipeline template. For example, if your source branch was named ac21f0d45a76294aaca0c0c0fdf06eb72d03368d, your new branch will be named @git_git_qr1hi_arvadosapi_com_reponame_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d@.
-
-By default, if you copy a pipeline template recursively, you will find that the template as well as all the dependencies are in your home project.
-
-If you would like to copy the object without dependencies, you can use the @--no-recursive@ tag.
-
-For example, we can copy the same object using this tag.
-
-<notextile>
-<pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster --dst-git-repo $USER/tutorial --no-recursive qr1hi-p5p6p-9pkaxt6qjnkxhhu</span>
-</code></pre>
-</notextile>
-
h3. How to copy a workflow
We will use the uuid @zzzzz-7fd4e-sampleworkflow1@ as an example workflow.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-Arvados repositories are managed through the Git revision control system. You can use these repositories to store your crunch scripts and run them in the arvados cluster.
+Arvados supports managing git repositories. You can acess these repositories using your Arvados credentials and share them with other Arvados users.
{% include 'tutorial_expectations' %}
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This tutorial describes how to work with a new Arvados git repository. Working with an Arvados git repository is analogous to working with other public git repositories. It will show you how to upload custom scripts to a remote Arvados repository, so you can use it in Arvados pipelines.
+This tutorial describes how to work with an Arvados-managed git repository. Working with an Arvados git repository is very similar to working with other public git repositories.
{% include 'tutorial_expectations' %}
{% include 'tutorial_git_repo_expectations' %}
-{% include 'notebox_begin' %}
-For more information about using Git, try
-<notextile>
-<pre><code>$ <span class="userinput">man gittutorial</span></code></pre>
-</notextile> or *"search Google for Git tutorials":http://google.com/#q=git+tutorial*.
-{% include 'notebox_end' %}
-
-h2. Cloning an Arvados repository
+h2. Cloning a git repository
Before you start using Git, you should do some basic configuration (you only need to do this the first time):
h2. Adding scripts to an Arvados repository
-Arvados crunch scripts need to be added in a *crunch_scripts* subdirectory in the repository. If this subdirectory does not exist, first create it in the local repository and change to that directory:
-
-<notextile>
-<pre><code>~/tutorial$ <span class="userinput">mkdir crunch_scripts</span>
-~/tutorial$ <span class="userinput">cd crunch_scripts</span></code></pre>
-</notextile>
-
-Next, using @nano@ or your favorite Unix text editor, create a new file called @hash.py@ in the @crunch_scripts@ directory.
-
-notextile. <pre>~/tutorial/crunch_scripts$ <code class="userinput">nano hash.py</code></pre>
-
-Add the following code to compute the MD5 hash of each file in a collection
+A git repository is a good place to store the CWL workflows that you run on Arvados.
-<notextile> {% code 'tutorial_hash_script_py' as python %} </notextile>
+First, create a simple CWL CommandLineTool:
-Make the file executable:
+notextile. <pre>~/tutorials$ <code class="userinput">nano hello.cwl</code></pre>
-notextile. <pre><code>~/tutorial/crunch_scripts$ <span class="userinput">chmod +x hash.py</span></code></pre>
+<notextile> {% code 'tutorial_hello_cwl' as yaml %} </notextile>
Next, add the file to the git repository. This tells @git@ that the file should be included on the next commit.
-notextile. <pre><code>~/tutorial/crunch_scripts$ <span class="userinput">git add hash.py</span></code></pre>
+notextile. <pre><code>~/tutorial$ <span class="userinput">git add hello.cwl</span></code></pre>
Next, commit your changes. All staged changes are recorded into the local git repository:
<notextile>
-<pre><code>~/tutorial/crunch_scripts$ <span class="userinput">git commit -m "my first script"</span>
+<pre><code>~/tutorial$ <span class="userinput">git commit -m "my first script"</span>
</code></pre>
</notextile>
</code></pre>
</notextile>
-Although this tutorial shows how to add a python script to Arvados, the same steps can be used to add any of your custom bash, R, or python scripts to an Arvados repository.
+The same steps can be used to add any of your custom bash, R, or python scripts to an Arvados repository.
projects / arvados.git / commitdiff