arvados.git
4 years ago16636: fix logic bug in (*Scheduler) updateMetrics()
Ward Vandewege [Tue, 15 Sep 2020 19:30:55 +0000 (15:30 -0400)]
16636: fix logic bug in (*Scheduler) updateMetrics()

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: implement review feedback.
Ward Vandewege [Tue, 15 Sep 2020 17:59:33 +0000 (13:59 -0400)]
16636: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
Ward Vandewege [Mon, 14 Sep 2020 19:14:32 +0000 (15:14 -0400)]
16636: add 'containers_longest_wait_time_seconds' metric: a gauge that
       indicates the current longest wait time between seeing a
       container in the queue or requeueing, and starting its crunch-run
       process on a worker.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: Merge branch 'master' into 16636-more-metrics
Ward Vandewege [Mon, 14 Sep 2020 17:03:18 +0000 (13:03 -0400)]
16636: Merge branch 'master' into 16636-more-metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoWork around Python bug, appending to a FIFO in python >=3 <3.8 is broken
Ward Vandewege [Mon, 14 Sep 2020 17:00:45 +0000 (13:00 -0400)]
Work around Python bug, appending to a FIFO in python >=3 <3.8 is broken
(https://bugs.python.org/issue27805).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16578-copy-tutorial' refs #16578
Peter Amstutz [Mon, 14 Sep 2020 15:47:07 +0000 (11:47 -0400)]
Merge branch '16578-copy-tutorial' refs #16578

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
Ward Vandewege [Mon, 14 Sep 2020 13:53:38 +0000 (09:53 -0400)]
16636: add 'time_from_queue_to_crunch_run' metric: wait times (between
       seeing a container in the queue or requeueing, and starting its
       crunch-run process on a worker) across previous starts.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16814-remove-python2-arvbox' into master
Nico Cesar [Mon, 14 Sep 2020 13:37:44 +0000 (09:37 -0400)]
Merge branch '16814-remove-python2-arvbox' into master

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoimplementing https://dev.arvados.org/issues/16814#note-7
Nico Cesar [Mon, 14 Sep 2020 13:35:50 +0000 (09:35 -0400)]
implementing https://dev.arvados.org/issues/16814#note-7

closes #16814

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years ago16636: add instances_time_from_shutdown_request_to_disappearance_seconds
Ward Vandewege [Fri, 11 Sep 2020 16:30:07 +0000 (12:30 -0400)]
16636: add instances_time_from_shutdown_request_to_disappearance_seconds
       metric

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBump gem versions in doc/Gemfile.lock, cf dependabot.
Ward Vandewege [Thu, 10 Sep 2020 20:51:35 +0000 (16:51 -0400)]
Bump gem versions in doc/Gemfile.lock, cf dependabot.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '15888-remove-more-python2-remnants'
Ward Vandewege [Thu, 10 Sep 2020 20:26:53 +0000 (16:26 -0400)]
Merge branch '15888-remove-more-python2-remnants'

refs #15888

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago15888: Merge branch 'master' into 15888-remove-more-python2-remnants
Ward Vandewege [Thu, 10 Sep 2020 20:26:21 +0000 (16:26 -0400)]
15888: Merge branch 'master' into 15888-remove-more-python2-remnants

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoBugfix: make sure that any fpm_args defined in fpm-info.sh get
Ward Vandewege [Thu, 10 Sep 2020 20:18:12 +0000 (16:18 -0400)]
Bugfix: make sure that any fpm_args defined in fpm-info.sh get
introduced at the correct position in the fpm argument list. This only
affects non-Python packages (we already did the right thing for Python
packages).

refs #15348

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoremove python2 packages from Dockerfile.base in arvbox
Nico Cesar [Thu, 10 Sep 2020 19:33:55 +0000 (15:33 -0400)]
remove python2 packages from Dockerfile.base in arvbox

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoMerge branch '16808-add-strict-mode'
Ward Vandewege [Thu, 10 Sep 2020 19:06:34 +0000 (15:06 -0400)]
Merge branch '16808-add-strict-mode'

closes #16808

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16808: Merge branch 'master' into 16808-add-strict-mode
Ward Vandewege [Thu, 10 Sep 2020 19:06:00 +0000 (15:06 -0400)]
16808: Merge branch 'master' into 16808-add-strict-mode

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago15888: update `arvados-server install -type test` for the new
Ward Vandewege [Thu, 10 Sep 2020 18:58:26 +0000 (14:58 -0400)]
15888: update `arvados-server install -type test` for the new
       python3-only world in run-tests.sh

We're still pulling in all of python2 because of the dependency on the
linkchecker and python-epydoc, though.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16636-container-allocation-metrics'
Ward Vandewege [Thu, 10 Sep 2020 18:52:49 +0000 (14:52 -0400)]
Merge branch '16636-container-allocation-metrics'

refs #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16808: move strict flag to the config-check command only
Ward Vandewege [Thu, 10 Sep 2020 18:08:48 +0000 (14:08 -0400)]
16808: move strict flag to the config-check command only

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16578: Add an arv-copy integration test
Peter Amstutz [Thu, 10 Sep 2020 18:00:59 +0000 (14:00 -0400)]
16578: Add an arv-copy integration test

To avoid complexity of starting two Arvados instances, copies a
subproject container a collection from one parent to another on the
same cluster.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16808: add strict mode to arvados-server config-check
Ward Vandewege [Thu, 10 Sep 2020 16:55:13 +0000 (12:55 -0400)]
16808: add strict mode to arvados-server config-check

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: a-d-c: add two more metrics:
Ward Vandewege [Wed, 9 Sep 2020 21:06:31 +0000 (17:06 -0400)]
16636: a-d-c: add two more metrics:

  (gauge) number of containers allocated to VMs but not started yet (because VMs are pending/booting)
  (gauge) number of containers not allocated to VMs (because provider quota is reached)

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoThe new libpam-arvados-go package should conflict with the old
Ward Vandewege [Wed, 9 Sep 2020 17:50:15 +0000 (13:50 -0400)]
The new libpam-arvados-go package should conflict with the old
libpam-arvados package.

refs #15348

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '15888-pycurl' into master
Nico Cesar [Wed, 9 Sep 2020 14:47:40 +0000 (10:47 -0400)]
Merge branch '15888-pycurl' into master

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoAddresses https://dev.arvados.org/issues/15888#note-27
Nico Cesar [Wed, 9 Sep 2020 13:59:22 +0000 (09:59 -0400)]
Addresses https://dev.arvados.org/issues/15888#note-27

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoMerge branch '15888-remove-py2-from-test' into master
Nico Cesar [Wed, 9 Sep 2020 13:22:20 +0000 (09:22 -0400)]
Merge branch '15888-remove-py2-from-test' into master

closes #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years ago16578: Few more copy script tweaks
Peter Amstutz [Wed, 9 Sep 2020 03:45:21 +0000 (23:45 -0400)]
16578: Few more copy script tweaks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16578: Fix issue with copying large manifests
Peter Amstutz [Wed, 9 Sep 2020 03:43:59 +0000 (23:43 -0400)]
16578: Fix issue with copying large manifests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16578: Simplify copy-tutorial based on arv-copy updates
Peter Amstutz [Wed, 9 Sep 2020 03:08:02 +0000 (23:08 -0400)]
16578: Simplify copy-tutorial based on arv-copy updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16578: Add project copy to arv-copy
Peter Amstutz [Wed, 9 Sep 2020 02:57:22 +0000 (22:57 -0400)]
16578: Add project copy to arv-copy

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoChanges related to https://dev.arvados.org/issues/15888#note-25
Nico Cesar [Wed, 9 Sep 2020 00:29:30 +0000 (20:29 -0400)]
Changes related to https://dev.arvados.org/issues/15888#note-25

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years ago16679: Unexports Login.TokenLifetime config. Refs #16679.
Lucas Di Pentima [Tue, 8 Sep 2020 21:27:35 +0000 (18:27 -0300)]
16679: Unexports Login.TokenLifetime config. Refs #16679.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agopy3 argparse differs from py2 for action='version'
Nico Cesar [Tue, 8 Sep 2020 19:49:42 +0000 (15:49 -0400)]
py3 argparse differs from py2 for action='version'

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoMerge branch '16421-document-collection-deletion-lifecycle'
Ward Vandewege [Tue, 8 Sep 2020 15:41:42 +0000 (11:41 -0400)]
Merge branch '16421-document-collection-deletion-lifecycle'

closes #16421

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16421: Merge branch 'master' into 16421-document-collection-deletion-lifecycle
Ward Vandewege [Tue, 8 Sep 2020 15:41:04 +0000 (11:41 -0400)]
16421: Merge branch 'master' into 16421-document-collection-deletion-lifecycle

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16578: Copy tutorials script WIP
Peter Amstutz [Tue, 8 Sep 2020 14:10:03 +0000 (10:10 -0400)]
16578: Copy tutorials script WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix some more golint warnings.
Ward Vandewege [Tue, 8 Sep 2020 00:25:25 +0000 (20:25 -0400)]
Fix some more golint warnings.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16578: Copy bwa tutorial from jutro to local cluster
Peter Amstutz [Fri, 4 Sep 2020 20:25:57 +0000 (16:25 -0400)]
16578: Copy bwa tutorial from jutro to local cluster

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix test failure, refs #16689
Peter Amstutz [Fri, 4 Sep 2020 19:19:36 +0000 (15:19 -0400)]
Fix test failure, refs #16689

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16421: improvements based on review comments.
Ward Vandewege [Fri, 4 Sep 2020 15:26:49 +0000 (11:26 -0400)]
16421: improvements based on review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16778-setup-fed-user' refs #16778
Peter Amstutz [Fri, 4 Sep 2020 15:25:42 +0000 (11:25 -0400)]
Merge branch '16778-setup-fed-user' refs #16778

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16778: Make a copy of options
Peter Amstutz [Fri, 4 Sep 2020 15:25:15 +0000 (11:25 -0400)]
16778: Make a copy of options

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16778: Simplify logic
Peter Amstutz [Fri, 4 Sep 2020 14:52:38 +0000 (10:52 -0400)]
16778: Simplify logic

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16796-arvbox-webshell' refs #16796
Peter Amstutz [Fri, 4 Sep 2020 14:15:40 +0000 (10:15 -0400)]
Merge branch '16796-arvbox-webshell' refs #16796

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16636-add-time-to-ssh-metric'
Ward Vandewege [Fri, 4 Sep 2020 14:12:53 +0000 (10:12 -0400)]
Merge branch '16636-add-time-to-ssh-metric'

refs #16636

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: Merge branch 'master' into 16636-add-time-to-ssh-metric
Ward Vandewege [Fri, 4 Sep 2020 14:12:22 +0000 (10:12 -0400)]
16636: Merge branch 'master' into 16636-add-time-to-ssh-metric

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16796: Publish webshell port
Peter Amstutz [Fri, 4 Sep 2020 14:10:26 +0000 (10:10 -0400)]
16796: Publish webshell port

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16778: Setup federated users with VM and repo with LoginCluster set
Peter Amstutz [Thu, 3 Sep 2020 22:08:29 +0000 (18:08 -0400)]
16778: Setup federated users with VM and repo with LoginCluster set

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16636: Address review comments
Ward Vandewege [Thu, 3 Sep 2020 21:19:29 +0000 (17:19 -0400)]
16636: Address review comments

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16636: a-d-c: add a time-to-ssh and time-to-ready-for-container metrics
Ward Vandewege [Thu, 3 Sep 2020 17:10:42 +0000 (13:10 -0400)]
16636: a-d-c: add a time-to-ssh and time-to-ready-for-container metrics

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16796: Remove reference to support@curoverse.com
Peter Amstutz [Thu, 3 Sep 2020 18:59:58 +0000 (14:59 -0400)]
16796: Remove reference to support@curoverse.com

Tweak a few references to playground.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16689-shell-sync-groups' refs #16689
Peter Amstutz [Thu, 3 Sep 2020 18:41:59 +0000 (14:41 -0400)]
Merge branch '16689-shell-sync-groups' refs #16689

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16796: Fix webshell logging.
Peter Amstutz [Thu, 3 Sep 2020 18:39:30 +0000 (14:39 -0400)]
16796: Fix webshell logging.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16796: Fix webshell --service
Peter Amstutz [Thu, 3 Sep 2020 18:34:16 +0000 (14:34 -0400)]
16796: Fix webshell --service

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16803-shell-sync-tokens' refs #16803
Peter Amstutz [Thu, 3 Sep 2020 18:30:48 +0000 (14:30 -0400)]
Merge branch '16803-shell-sync-tokens' refs #16803

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16803: Tweak permission setting on created files/dirs
Peter Amstutz [Thu, 3 Sep 2020 18:29:31 +0000 (14:29 -0400)]
16803: Tweak permission setting on created files/dirs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16803: Update shell node install instructions
Peter Amstutz [Thu, 3 Sep 2020 16:57:36 +0000 (12:57 -0400)]
16803: Update shell node install instructions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agomaybe all this should be re-done
Nico Cesar [Thu, 3 Sep 2020 15:05:20 +0000 (11:05 -0400)]
maybe all this should be re-done

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agocleanup: remove explicit installation of nodejs in our arvados/jobs
Ward Vandewege [Thu, 3 Sep 2020 14:20:26 +0000 (10:20 -0400)]
cleanup: remove explicit installation of nodejs in our arvados/jobs
Dockerfile. It's a dependency of the package now.

refs #16788

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16421: documentation: check in all new files.
Ward Vandewege [Thu, 3 Sep 2020 14:13:37 +0000 (10:13 -0400)]
16421: documentation: check in all new files.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agobugfix: remove unnecessarily dependency on python2 sdk package in our
Ward Vandewege [Thu, 3 Sep 2020 14:09:52 +0000 (10:09 -0400)]
bugfix: remove unnecessarily dependency on python2 sdk package in our
arvados/jobs Docker image.

refs #16580

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16726-anon-user-token' refs #16726
Peter Amstutz [Thu, 3 Sep 2020 13:41:02 +0000 (09:41 -0400)]
Merge branch '16726-anon-user-token' refs #16726

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16689: add/remove users from groups to sync with Arvados
Peter Amstutz [Wed, 2 Sep 2020 22:08:43 +0000 (18:08 -0400)]
16689: add/remove users from groups to sync with Arvados

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoactivate was deleted by mistake
Nico Cesar [Wed, 2 Sep 2020 21:56:05 +0000 (17:56 -0400)]
activate was deleted by mistake

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years ago16421: documentation: reorganize and update Keep documentation. Add
Ward Vandewege [Wed, 2 Sep 2020 21:34:52 +0000 (17:34 -0400)]
16421: documentation: reorganize and update Keep documentation. Add
       collection and data block lifecycle documentation.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16803: Create settings.conf for the user.
Peter Amstutz [Wed, 2 Sep 2020 21:24:25 +0000 (17:24 -0400)]
16803: Create settings.conf for the user.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16679-wb2-idle-timeout-config'. Refs #16679
Lucas Di Pentima [Wed, 2 Sep 2020 19:12:20 +0000 (16:12 -0300)]
Merge branch '16679-wb2-idle-timeout-config'. Refs #16679

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '16795-boot-outcome-aborted'
Tom Clegg [Wed, 2 Sep 2020 17:39:20 +0000 (13:39 -0400)]
Merge branch '16795-boot-outcome-aborted'

fixes #16795

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16795: Update tests.
Tom Clegg [Wed, 2 Sep 2020 17:38:40 +0000 (13:38 -0400)]
16795: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16795: Pin pycurl version.
Tom Clegg [Wed, 2 Sep 2020 14:06:34 +0000 (10:06 -0400)]
16795: Pin pycurl version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16796: Remove the 'beta' tag from webshell
Peter Amstutz [Wed, 2 Sep 2020 13:44:42 +0000 (09:44 -0400)]
16796: Remove the 'beta' tag from webshell

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16795: Don't shutdown idle nodes just because Create failed.
Tom Clegg [Wed, 2 Sep 2020 12:56:38 +0000 (08:56 -0400)]
16795: Don't shutdown idle nodes just because Create failed.

Previous behavior would prematurely shutdown booting instances when
Create calls were being rate-limited.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16796: Expect password prompt
Peter Amstutz [Wed, 2 Sep 2020 02:40:57 +0000 (22:40 -0400)]
16796: Expect password prompt

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16773-webshell-js' into 16796-arvbox-webshell
Peter Amstutz [Wed, 2 Sep 2020 02:30:17 +0000 (22:30 -0400)]
Merge branch '16773-webshell-js' into 16796-arvbox-webshell

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16796: Add webshell to arvbox
Peter Amstutz [Wed, 2 Sep 2020 02:07:04 +0000 (22:07 -0400)]
16796: Add webshell to arvbox

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16795: Simplify "kill" behavior in stub driver.
Tom Clegg [Tue, 1 Sep 2020 21:10:09 +0000 (17:10 -0400)]
16795: Simplify "kill" behavior in stub driver.

Now, "kill" just sets a flag, and the "run" stub exits when it notices
the flag.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16726: Integration test using anonymous token to fetch a collection
Peter Amstutz [Tue, 1 Sep 2020 21:04:44 +0000 (17:04 -0400)]
16726: Integration test using anonymous token to fetch a collection

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16795: Fix false-positive bug detection.
Tom Clegg [Tue, 1 Sep 2020 20:22:13 +0000 (16:22 -0400)]
16795: Fix false-positive bug detection.

The dispatcher simulation test occasionally fails with this error when
a crunch-run stub is killed, and the same container is rescheduled on
the same node before the first attempt's startup-phase sleep expires:

    bugf("[test] StubDriver bug or caller bug: pid %d exiting, running[%s]==%d", pid, uuid, svm.running[uuid])
... Error: [test] StubDriver bug or caller bug: pid 9 exiting, running[zzzzz-dz642-000000000000184]==0

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16726-anon-fed' into 16726-anon-user-token
Peter Amstutz [Tue, 1 Sep 2020 19:23:02 +0000 (15:23 -0400)]
Merge branch '16726-anon-fed' into 16726-anon-user-token

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16726: Initialize anonymous user token in arvados-boot
Peter Amstutz [Tue, 1 Sep 2020 19:21:54 +0000 (15:21 -0400)]
16726: Initialize anonymous user token in arvados-boot

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16790-next-marker-omitempty'
Tom Clegg [Tue, 1 Sep 2020 15:33:25 +0000 (11:33 -0400)]
Merge branch '16790-next-marker-omitempty'

fixes #16790

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16679: Exports Login.TokenLifetime to be used by workbench2.
Lucas Di Pentima [Tue, 1 Sep 2020 15:06:53 +0000 (12:06 -0300)]
16679: Exports Login.TokenLifetime to be used by workbench2.

When not zero, workbench2 will store the token on session storage instead
of local storage for enhanced security.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoCheck Content-Length header in HeadObject response.
Tom Clegg [Tue, 1 Sep 2020 14:47:17 +0000 (10:47 -0400)]
Check Content-Length header in HeadObject response.

refs #16596

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16679: Adds Workbench.IdleTimeout config setting for WB2.
Lucas Di Pentima [Tue, 1 Sep 2020 14:41:04 +0000 (11:41 -0300)]
16679: Adds Workbench.IdleTimeout config setting for WB2.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '16739-concurrent-node-create-throttle'
Ward Vandewege [Tue, 1 Sep 2020 14:34:10 +0000 (10:34 -0400)]
Merge branch '16739-concurrent-node-create-throttle'

closes #16739

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16739: Merge branch 'master' into 16739-concurrent-node-create-throttle
Ward Vandewege [Tue, 1 Sep 2020 14:33:27 +0000 (10:33 -0400)]
16739: Merge branch 'master' into 16739-concurrent-node-create-throttle

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16739: doc update: implement review feedback.
Ward Vandewege [Tue, 1 Sep 2020 14:32:47 +0000 (10:32 -0400)]
16739: doc update: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16739: remove throttle.ResetError, update the err field directly from
Ward Vandewege [Tue, 1 Sep 2020 14:23:33 +0000 (10:23 -0400)]
16739: remove throttle.ResetError, update the err field directly from
       the test.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16790: Avoid returning empty NextMarker tag in list responses.
Tom Clegg [Tue, 1 Sep 2020 14:16:47 +0000 (10:16 -0400)]
16790: Avoid returning empty NextMarker tag in list responses.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16739: implement review feedback.
Ward Vandewege [Tue, 1 Sep 2020 13:43:57 +0000 (09:43 -0400)]
16739: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16779: Create user accounts even if they don't have an ssh key
Peter Amstutz [Mon, 31 Aug 2020 18:22:27 +0000 (14:22 -0400)]
16779: Create user accounts even if they don't have an ssh key

refs #16779

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix redundant entry in cluster config, arvbox tells you to use adduser
Peter Amstutz [Mon, 31 Aug 2020 21:37:56 +0000 (17:37 -0400)]
Fix redundant entry in cluster config, arvbox tells you to use adduser

refs #16749

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16788-add-nodejs-dependency-to-a-c-r-package'
Ward Vandewege [Mon, 31 Aug 2020 19:49:12 +0000 (15:49 -0400)]
Merge branch '16788-add-nodejs-dependency-to-a-c-r-package'

closes #16788

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16788: add package dependency on nodejs to arvados-cwl-runner.
Ward Vandewege [Mon, 31 Aug 2020 19:47:51 +0000 (15:47 -0400)]
16788: add package dependency on nodejs to arvados-cwl-runner.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years ago16739: Fix using two different throttles.
Tom Clegg [Mon, 31 Aug 2020 19:34:22 +0000 (15:34 -0400)]
16739: Fix using two different throttles.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16739: a-d-c: add a concurrent node creation throttle option.
Ward Vandewege [Mon, 31 Aug 2020 18:42:14 +0000 (14:42 -0400)]
16739: a-d-c: add a concurrent node creation throttle option.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoInstalling extra python3 libraries:dist_wheel has to go before PyYAML
Nico Cesar [Mon, 31 Aug 2020 18:28:28 +0000 (14:28 -0400)]
Installing extra python3 libraries:dist_wheel has to go before PyYAML

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

4 years agoMerge branch '16602-wb-acr-version' refs #16602
Peter Amstutz [Mon, 31 Aug 2020 15:29:01 +0000 (11:29 -0400)]
Merge branch '16602-wb-acr-version' refs #16602

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoRevert "Installing extra python3 libraries"
Nico Cesar [Mon, 31 Aug 2020 15:23:58 +0000 (11:23 -0400)]
Revert "Installing extra python3 libraries"

This reverts commit ddae1b57f077d8a7c091b3cdf9ecaa562703fb3c.

setuptools should be done outside pip

refs #15888

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>