arvados.git
3 years agoMerge branch '16601-new-tutorial' refs #16601
Peter Amstutz [Thu, 17 Sep 2020 13:57:15 +0000 (09:57 -0400)]
Merge branch '16601-new-tutorial' refs #16601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16601: Formatting touch-ups and typo fixes.
Peter Amstutz [Wed, 16 Sep 2020 22:08:15 +0000 (18:08 -0400)]
16601: Formatting touch-ups and typo fixes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16833-replace-epydoc' into master
Ward Vandewege [Wed, 16 Sep 2020 20:06:41 +0000 (16:06 -0400)]
Merge branch '16833-replace-epydoc' into master

closes #16833

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16833: Merge branch 'master' into 16833-replace-epydoc
Ward Vandewege [Wed, 16 Sep 2020 20:00:07 +0000 (16:00 -0400)]
16833: Merge branch 'master' into 16833-replace-epydoc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16833: replace epydoc with pdoc3
Ward Vandewege [Tue, 15 Sep 2020 20:43:12 +0000 (16:43 -0400)]
16833: replace epydoc with pdoc3

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16636-more-metrics' into master
Ward Vandewege [Tue, 15 Sep 2020 19:32:47 +0000 (15:32 -0400)]
Merge branch '16636-more-metrics' into master

closes #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: Merge branch 'master' into 16636-more-metrics
Ward Vandewege [Tue, 15 Sep 2020 19:32:15 +0000 (15:32 -0400)]
16636: Merge branch 'master' into 16636-more-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: fix logic bug in (*Scheduler) updateMetrics()
Ward Vandewege [Tue, 15 Sep 2020 19:30:55 +0000 (15:30 -0400)]
16636: fix logic bug in (*Scheduler) updateMetrics()

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: implement review feedback.
Ward Vandewege [Tue, 15 Sep 2020 17:59:33 +0000 (13:59 -0400)]
16636: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16826-unlogged-attrs-fix'
Lucas Di Pentima [Tue, 15 Sep 2020 14:56:33 +0000 (11:56 -0300)]
Merge branch '16826-unlogged-attrs-fix'
Closes #16826

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16834-stale-run-lock'
Tom Clegg [Tue, 15 Sep 2020 14:49:44 +0000 (10:49 -0400)]
Merge branch '16834-stale-run-lock'

fixes #16834

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16834: Comments about parsing "crunch-run --list" output.
Tom Clegg [Tue, 15 Sep 2020 14:49:13 +0000 (10:49 -0400)]
16834: Comments about parsing "crunch-run --list" output.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16826: Moves configuration set-up to TestCase's setup() method.
Lucas Di Pentima [Mon, 14 Sep 2020 20:36:43 +0000 (17:36 -0300)]
16826: Moves configuration set-up to TestCase's setup() method.

Having a failing assertion on the teardown() method before the config
restoration code has the side effect of failing all the tests following
the one that made an illegal config change. This updates fixes that.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Fixes config maps assignments, mostly on tests.
Lucas Di Pentima [Mon, 14 Sep 2020 20:23:09 +0000 (17:23 -0300)]
16826: Fixes config maps assignments, mostly on tests.

The new config check brought to the spotlight new tests that are setting
configurations incorrectly.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Enhances teardown config check.
Lucas Di Pentima [Mon, 14 Sep 2020 19:19:59 +0000 (16:19 -0300)]
16826: Enhances teardown config check.

It's not enough to check that .keys() returns a list of Symbols, because
ActiveRecord::OrderedOptions behaves somewhat like HashWithIndifferentAccess
so values can need to be accessed by symbol or string. So, we now check that
every hash-like config section is an OrderedOptions object and also returns
its keys as symbols.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
Ward Vandewege [Mon, 14 Sep 2020 19:14:32 +0000 (15:14 -0400)]
16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
       indicates the current longest wait time between seeing a
       container in the queue or requeueing, and starting its crunch-run
       process on a worker.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16826: Adds catch-all check for any config potentially changed during tests.
Lucas Di Pentima [Mon, 14 Sep 2020 18:42:15 +0000 (15:42 -0300)]
16826: Adds catch-all check for any config potentially changed during tests.

On test suite teardown, check if all config keys are still Symbols, so that
no mistake is made by writing tests that rely on config hashes with string
keys.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16813-avoid-noop-user-updates'
Tom Clegg [Mon, 14 Sep 2020 18:39:48 +0000 (14:39 -0400)]
Merge branch '16813-avoid-noop-user-updates'

fixes #16813

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agorun_test_server.py: work around Python bug, appending to a FIFO in
Ward Vandewege [Mon, 14 Sep 2020 17:17:18 +0000 (13:17 -0400)]
run_test_server.py: work around Python bug, appending to a FIFO in
python >=3 <3.8 is broken (https://bugs.python.org/issue27805), more
thorougly this time.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: Merge branch 'master' into 16636-more-metrics
Ward Vandewege [Mon, 14 Sep 2020 17:03:18 +0000 (13:03 -0400)]
16636: Merge branch 'master' into 16636-more-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWork around Python bug, appending to a FIFO in python >=3 <3.8 is broken
Ward Vandewege [Mon, 14 Sep 2020 17:00:45 +0000 (13:00 -0400)]
Work around Python bug, appending to a FIFO in python >=3 <3.8 is broken
(https://bugs.python.org/issue27805).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16578-copy-tutorial' refs #16578
Peter Amstutz [Mon, 14 Sep 2020 15:47:07 +0000 (11:47 -0400)]
Merge branch '16578-copy-tutorial' refs #16578

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
Ward Vandewege [Mon, 14 Sep 2020 13:53:38 +0000 (09:53 -0400)]
16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
       seeing a container in the queue or requeueing, and starting its
       crunch-run process on a worker) across previous starts.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16814-remove-python2-arvbox' into master
Nico Cesar [Mon, 14 Sep 2020 13:37:44 +0000 (09:37 -0400)]
Merge branch '16814-remove-python2-arvbox' into master

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoimplementing https://dev.arvados.org/issues/16814#note-7
Nico Cesar [Mon, 14 Sep 2020 13:35:50 +0000 (09:35 -0400)]
implementing https://dev.arvados.org/issues/16814#note-7

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago16834: Drain instance if crunch-run exits but arv-mount doesn't.
Tom Clegg [Sat, 12 Sep 2020 19:51:25 +0000 (15:51 -0400)]
16834: Drain instance if crunch-run exits but arv-mount doesn't.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16826: Fixes config setting on tests about config maps knobs.
Lucas Di Pentima [Fri, 11 Sep 2020 21:59:54 +0000 (18:59 -0300)]
16826: Fixes config setting on tests about config maps knobs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Fixes the bug.
Lucas Di Pentima [Fri, 11 Sep 2020 21:44:20 +0000 (18:44 -0300)]
16826: Fixes the bug.

The issue is that the config loading code on Rails uses
ActiveSupport::OrderedOptions on config hashes, and this class always
returns keys as symbols. We were incorrectly testing the feature by
assigning a normal Hash to the config.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16826: Adds test exposing the bug.
Lucas Di Pentima [Fri, 11 Sep 2020 21:42:04 +0000 (18:42 -0300)]
16826: Adds test exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago16636: add instances_time_from_shutdown_request_to_disappearance_seconds
Ward Vandewege [Fri, 11 Sep 2020 16:30:07 +0000 (12:30 -0400)]
16636: add instances_time_from_shutdown_request_to_disappearance_seconds
       metric

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump gem versions in doc/Gemfile.lock, cf dependabot.
Ward Vandewege [Thu, 10 Sep 2020 20:51:35 +0000 (16:51 -0400)]
Bump gem versions in doc/Gemfile.lock, cf dependabot.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16813: Avoid writing db when nothing is changing in a batch update.
Tom Clegg [Thu, 10 Sep 2020 20:33:31 +0000 (16:33 -0400)]
16813: Avoid writing db when nothing is changing in a batch update.

Avoids superfluous log entries and potentially costly permission graph
updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years agoMerge branch '15888-remove-more-python2-remnants'
Ward Vandewege [Thu, 10 Sep 2020 20:26:53 +0000 (16:26 -0400)]
Merge branch '15888-remove-more-python2-remnants'

refs #15888

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago15888: Merge branch 'master' into 15888-remove-more-python2-remnants
Ward Vandewege [Thu, 10 Sep 2020 20:26:21 +0000 (16:26 -0400)]
15888: Merge branch 'master' into 15888-remove-more-python2-remnants

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBugfix: make sure that any fpm_args defined in fpm-info.sh get
Ward Vandewege [Thu, 10 Sep 2020 20:18:12 +0000 (16:18 -0400)]
Bugfix: make sure that any fpm_args defined in fpm-info.sh get
introduced at the correct position in the fpm argument list. This only
affects non-Python packages (we already did the right thing for Python
packages).

refs #15348

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoremove python2 packages from Dockerfile.base in arvbox
Nico Cesar [Thu, 10 Sep 2020 19:33:55 +0000 (15:33 -0400)]
remove python2 packages from Dockerfile.base in arvbox

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoMerge branch '16808-add-strict-mode'
Ward Vandewege [Thu, 10 Sep 2020 19:06:34 +0000 (15:06 -0400)]
Merge branch '16808-add-strict-mode'

closes #16808

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16808: Merge branch 'master' into 16808-add-strict-mode
Ward Vandewege [Thu, 10 Sep 2020 19:06:00 +0000 (15:06 -0400)]
16808: Merge branch 'master' into 16808-add-strict-mode

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago15888: update `arvados-server install -type test` for the new
Ward Vandewege [Thu, 10 Sep 2020 18:58:26 +0000 (14:58 -0400)]
15888: update `arvados-server install -type test` for the new
       python3-only world in run-tests.sh

We're still pulling in all of python2 because of the dependency on the
linkchecker and python-epydoc, though.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16636-container-allocation-metrics'
Ward Vandewege [Thu, 10 Sep 2020 18:52:49 +0000 (14:52 -0400)]
Merge branch '16636-container-allocation-metrics'

refs #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16808: move strict flag to the config-check command only
Ward Vandewege [Thu, 10 Sep 2020 18:08:48 +0000 (14:08 -0400)]
16808: move strict flag to the config-check command only

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16578: Add an arv-copy integration test
Peter Amstutz [Thu, 10 Sep 2020 18:00:59 +0000 (14:00 -0400)]
16578: Add an arv-copy integration test

To avoid complexity of starting two Arvados instances, copies a
subproject container a collection from one parent to another on the
same cluster.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16808: add strict mode to arvados-server config-check
Ward Vandewege [Thu, 10 Sep 2020 16:55:13 +0000 (12:55 -0400)]
16808: add strict mode to arvados-server config-check

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16601: Import WGS tutorial from google doc & add textile markup
Peter Amstutz [Thu, 10 Sep 2020 02:31:39 +0000 (22:31 -0400)]
16601: Import WGS tutorial from google doc & add textile markup

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16636: a-d-c: add two more metrics:
Ward Vandewege [Wed, 9 Sep 2020 21:06:31 +0000 (17:06 -0400)]
16636: a-d-c: add two more metrics:

  (gauge) number of containers allocated to VMs but not started yet (because VMs are pending/booting)
  (gauge) number of containers not allocated to VMs (because provider quota is reached)

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoThe new libpam-arvados-go package should conflict with the old
Ward Vandewege [Wed, 9 Sep 2020 17:50:15 +0000 (13:50 -0400)]
The new libpam-arvados-go package should conflict with the old
libpam-arvados package.

refs #15348

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '15888-pycurl' into master
Nico Cesar [Wed, 9 Sep 2020 14:47:40 +0000 (10:47 -0400)]
Merge branch '15888-pycurl' into master

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoAddresses https://dev.arvados.org/issues/15888#note-27
Nico Cesar [Wed, 9 Sep 2020 13:59:22 +0000 (09:59 -0400)]
Addresses https://dev.arvados.org/issues/15888#note-27

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoMerge branch '15888-remove-py2-from-test' into master
Nico Cesar [Wed, 9 Sep 2020 13:22:20 +0000 (09:22 -0400)]
Merge branch '15888-remove-py2-from-test' into master

closes #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago16578: Few more copy script tweaks
Peter Amstutz [Wed, 9 Sep 2020 03:45:21 +0000 (23:45 -0400)]
16578: Few more copy script tweaks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16578: Fix issue with copying large manifests
Peter Amstutz [Wed, 9 Sep 2020 03:43:59 +0000 (23:43 -0400)]
16578: Fix issue with copying large manifests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16578: Simplify copy-tutorial based on arv-copy updates
Peter Amstutz [Wed, 9 Sep 2020 03:08:02 +0000 (23:08 -0400)]
16578: Simplify copy-tutorial based on arv-copy updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16578: Add project copy to arv-copy
Peter Amstutz [Wed, 9 Sep 2020 02:57:22 +0000 (22:57 -0400)]
16578: Add project copy to arv-copy

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoChanges related to https://dev.arvados.org/issues/15888#note-25
Nico Cesar [Wed, 9 Sep 2020 00:29:30 +0000 (20:29 -0400)]
Changes related to https://dev.arvados.org/issues/15888#note-25

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago16679: Unexports Login.TokenLifetime config. Refs #16679.
Lucas Di Pentima [Tue, 8 Sep 2020 21:27:35 +0000 (18:27 -0300)]
16679: Unexports Login.TokenLifetime config. Refs #16679.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agopy3 argparse differs from py2 for action='version'
Nico Cesar [Tue, 8 Sep 2020 19:49:42 +0000 (15:49 -0400)]
py3 argparse differs from py2 for action='version'

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoMerge branch '16421-document-collection-deletion-lifecycle'
Ward Vandewege [Tue, 8 Sep 2020 15:41:42 +0000 (11:41 -0400)]
Merge branch '16421-document-collection-deletion-lifecycle'

closes #16421

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16421: Merge branch 'master' into 16421-document-collection-deletion-lifecycle
Ward Vandewege [Tue, 8 Sep 2020 15:41:04 +0000 (11:41 -0400)]
16421: Merge branch 'master' into 16421-document-collection-deletion-lifecycle

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16578: Copy tutorials script WIP
Peter Amstutz [Tue, 8 Sep 2020 14:10:03 +0000 (10:10 -0400)]
16578: Copy tutorials script WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix some more golint warnings.
Ward Vandewege [Tue, 8 Sep 2020 00:25:25 +0000 (20:25 -0400)]
Fix some more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16578: Copy bwa tutorial from jutro to local cluster
Peter Amstutz [Fri, 4 Sep 2020 20:25:57 +0000 (16:25 -0400)]
16578: Copy bwa tutorial from jutro to local cluster

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix test failure, refs #16689
Peter Amstutz [Fri, 4 Sep 2020 19:19:36 +0000 (15:19 -0400)]
Fix test failure, refs #16689

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16421: improvements based on review comments.
Ward Vandewege [Fri, 4 Sep 2020 15:26:49 +0000 (11:26 -0400)]
16421: improvements based on review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16778-setup-fed-user' refs #16778
Peter Amstutz [Fri, 4 Sep 2020 15:25:42 +0000 (11:25 -0400)]
Merge branch '16778-setup-fed-user' refs #16778

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16778: Make a copy of options
Peter Amstutz [Fri, 4 Sep 2020 15:25:15 +0000 (11:25 -0400)]
16778: Make a copy of options

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16778: Simplify logic
Peter Amstutz [Fri, 4 Sep 2020 14:52:38 +0000 (10:52 -0400)]
16778: Simplify logic

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16796-arvbox-webshell' refs #16796
Peter Amstutz [Fri, 4 Sep 2020 14:15:40 +0000 (10:15 -0400)]
Merge branch '16796-arvbox-webshell' refs #16796

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16636-add-time-to-ssh-metric'
Ward Vandewege [Fri, 4 Sep 2020 14:12:53 +0000 (10:12 -0400)]
Merge branch '16636-add-time-to-ssh-metric'

refs #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: Merge branch 'master' into 16636-add-time-to-ssh-metric
Ward Vandewege [Fri, 4 Sep 2020 14:12:22 +0000 (10:12 -0400)]
16636: Merge branch 'master' into 16636-add-time-to-ssh-metric

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16796: Publish webshell port
Peter Amstutz [Fri, 4 Sep 2020 14:10:26 +0000 (10:10 -0400)]
16796: Publish webshell port

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16778: Setup federated users with VM and repo with LoginCluster set
Peter Amstutz [Thu, 3 Sep 2020 22:08:29 +0000 (18:08 -0400)]
16778: Setup federated users with VM and repo with LoginCluster set

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16636: Address review comments
Ward Vandewege [Thu, 3 Sep 2020 21:19:29 +0000 (17:19 -0400)]
16636: Address review comments

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16636: a-d-c: add a time-to-ssh and time-to-ready-for-container metrics
Ward Vandewege [Thu, 3 Sep 2020 17:10:42 +0000 (13:10 -0400)]
16636: a-d-c: add a time-to-ssh and time-to-ready-for-container metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16796: Remove reference to support@curoverse.com
Peter Amstutz [Thu, 3 Sep 2020 18:59:58 +0000 (14:59 -0400)]
16796: Remove reference to support@curoverse.com

Tweak a few references to playground.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16689-shell-sync-groups' refs #16689
Peter Amstutz [Thu, 3 Sep 2020 18:41:59 +0000 (14:41 -0400)]
Merge branch '16689-shell-sync-groups' refs #16689

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16796: Fix webshell logging.
Peter Amstutz [Thu, 3 Sep 2020 18:39:30 +0000 (14:39 -0400)]
16796: Fix webshell logging.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16796: Fix webshell --service
Peter Amstutz [Thu, 3 Sep 2020 18:34:16 +0000 (14:34 -0400)]
16796: Fix webshell --service

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16803-shell-sync-tokens' refs #16803
Peter Amstutz [Thu, 3 Sep 2020 18:30:48 +0000 (14:30 -0400)]
Merge branch '16803-shell-sync-tokens' refs #16803

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16803: Tweak permission setting on created files/dirs
Peter Amstutz [Thu, 3 Sep 2020 18:29:31 +0000 (14:29 -0400)]
16803: Tweak permission setting on created files/dirs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16803: Update shell node install instructions
Peter Amstutz [Thu, 3 Sep 2020 16:57:36 +0000 (12:57 -0400)]
16803: Update shell node install instructions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agomaybe all this should be re-done
Nico Cesar [Thu, 3 Sep 2020 15:05:20 +0000 (11:05 -0400)]
maybe all this should be re-done

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agocleanup: remove explicit installation of nodejs in our arvados/jobs
Ward Vandewege [Thu, 3 Sep 2020 14:20:26 +0000 (10:20 -0400)]
cleanup: remove explicit installation of nodejs in our arvados/jobs
Dockerfile. It's a dependency of the package now.

refs #16788

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16421: documentation: check in all new files.
Ward Vandewege [Thu, 3 Sep 2020 14:13:37 +0000 (10:13 -0400)]
16421: documentation: check in all new files.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agobugfix: remove unnecessarily dependency on python2 sdk package in our
Ward Vandewege [Thu, 3 Sep 2020 14:09:52 +0000 (10:09 -0400)]
bugfix: remove unnecessarily dependency on python2 sdk package in our
arvados/jobs Docker image.

refs #16580

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16726-anon-user-token' refs #16726
Peter Amstutz [Thu, 3 Sep 2020 13:41:02 +0000 (09:41 -0400)]
Merge branch '16726-anon-user-token' refs #16726

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16689: add/remove users from groups to sync with Arvados
Peter Amstutz [Wed, 2 Sep 2020 22:08:43 +0000 (18:08 -0400)]
16689: add/remove users from groups to sync with Arvados

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoactivate was deleted by mistake
Nico Cesar [Wed, 2 Sep 2020 21:56:05 +0000 (17:56 -0400)]
activate was deleted by mistake

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago16421: documentation: reorganize and update Keep documentation. Add
Ward Vandewege [Wed, 2 Sep 2020 21:34:52 +0000 (17:34 -0400)]
16421: documentation: reorganize and update Keep documentation. Add
       collection and data block lifecycle documentation.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16803: Create settings.conf for the user.
Peter Amstutz [Wed, 2 Sep 2020 21:24:25 +0000 (17:24 -0400)]
16803: Create settings.conf for the user.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16679-wb2-idle-timeout-config'. Refs #16679
Lucas Di Pentima [Wed, 2 Sep 2020 19:12:20 +0000 (16:12 -0300)]
Merge branch '16679-wb2-idle-timeout-config'. Refs #16679

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '16795-boot-outcome-aborted'
Tom Clegg [Wed, 2 Sep 2020 17:39:20 +0000 (13:39 -0400)]
Merge branch '16795-boot-outcome-aborted'

fixes #16795

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16795: Update tests.
Tom Clegg [Wed, 2 Sep 2020 17:38:40 +0000 (13:38 -0400)]
16795: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16795: Pin pycurl version.
Tom Clegg [Wed, 2 Sep 2020 14:06:34 +0000 (10:06 -0400)]
16795: Pin pycurl version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16796: Remove the 'beta' tag from webshell
Peter Amstutz [Wed, 2 Sep 2020 13:44:42 +0000 (09:44 -0400)]
16796: Remove the 'beta' tag from webshell

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16795: Don't shutdown idle nodes just because Create failed.
Tom Clegg [Wed, 2 Sep 2020 12:56:38 +0000 (08:56 -0400)]
16795: Don't shutdown idle nodes just because Create failed.

Previous behavior would prematurely shutdown booting instances when
Create calls were being rate-limited.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16796: Expect password prompt
Peter Amstutz [Wed, 2 Sep 2020 02:40:57 +0000 (22:40 -0400)]
16796: Expect password prompt

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '16773-webshell-js' into 16796-arvbox-webshell
Peter Amstutz [Wed, 2 Sep 2020 02:30:17 +0000 (22:30 -0400)]
Merge branch '16773-webshell-js' into 16796-arvbox-webshell

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16796: Add webshell to arvbox
Peter Amstutz [Wed, 2 Sep 2020 02:07:04 +0000 (22:07 -0400)]
16796: Add webshell to arvbox

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16795: Simplify "kill" behavior in stub driver.
Tom Clegg [Tue, 1 Sep 2020 21:10:09 +0000 (17:10 -0400)]
16795: Simplify "kill" behavior in stub driver.

Now, "kill" just sets a flag, and the "run" stub exits when it notices
the flag.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

3 years ago16726: Integration test using anonymous token to fetch a collection
Peter Amstutz [Tue, 1 Sep 2020 21:04:44 +0000 (17:04 -0400)]
16726: Integration test using anonymous token to fetch a collection

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>