arvados.git
10 months agoMerge branch '20613-googleapiclient-init-logs' 2.6.3
Brett Smith [Wed, 7 Jun 2023 20:28:53 +0000 (16:28 -0400)]
Merge branch '20613-googleapiclient-init-logs'

Closes #20613.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months agoMerge branch '20548-installer-deploy' refs #20548
Peter Amstutz [Wed, 7 Jun 2023 17:51:21 +0000 (13:51 -0400)]
Merge branch '20548-installer-deploy' refs #20548

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoInitialize rcode variable, refs #20611
Peter Amstutz [Mon, 5 Jun 2023 22:10:07 +0000 (18:10 -0400)]
Initialize rcode variable, refs #20611

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoPut date on 2.6.3 upgrading notes
Peter Amstutz [Mon, 5 Jun 2023 19:44:55 +0000 (15:44 -0400)]
Put date on 2.6.3 upgrading notes

refs #20564

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoMerge branch '20522-load-dispatch-priv-key'
Tom Clegg [Mon, 5 Jun 2023 19:16:11 +0000 (15:16 -0400)]
Merge branch '20522-load-dispatch-priv-key'

closes #20522

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

10 months agoMerge branch '20531-cwl-log-tail' refs #20531
Peter Amstutz [Fri, 2 Jun 2023 17:59:52 +0000 (13:59 -0400)]
Merge branch '20531-cwl-log-tail' refs #20531

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoMerge branch '20541-less-fetching-mounts'
Tom Clegg [Fri, 2 Jun 2023 16:11:02 +0000 (12:11 -0400)]
Merge branch '20541-less-fetching-mounts'

fixes #20541

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

10 months agoMerge branch '20540-crunch-run-retry'
Tom Clegg [Thu, 1 Jun 2023 19:13:57 +0000 (15:13 -0400)]
Merge branch '20540-crunch-run-retry'

fixes #20540

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

10 months agoMerge branch '20545-drop-if-none-match'
Brett Smith [Thu, 1 Jun 2023 15:53:27 +0000 (11:53 -0400)]
Merge branch '20545-drop-if-none-match'

Closes #20545.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months agoMerge branch '20433-crunch-log-zero-bytes-job'
Brett Smith [Thu, 1 Jun 2023 15:53:23 +0000 (11:53 -0400)]
Merge branch '20433-crunch-log-zero-bytes-job'

Closes #20433.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

10 months agoMerge branch '20561-file-copy-logging' refs #20561
Peter Amstutz [Thu, 1 Jun 2023 14:23:41 +0000 (10:23 -0400)]
Merge branch '20561-file-copy-logging' refs #20561

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

10 months agoMerge branch '20485-optional-deploy-ssh-key'
Tom Clegg [Thu, 1 Jun 2023 13:33:48 +0000 (09:33 -0400)]
Merge branch '20485-optional-deploy-ssh-key'

closes #20485

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

10 months agoMerge branch '20229-doc-inspect-requests'
Tom Clegg [Mon, 29 May 2023 19:43:17 +0000 (15:43 -0400)]
Merge branch '20229-doc-inspect-requests'

closes #20229

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

10 months agoMerge branch '20520-instance-init-command'
Tom Clegg [Fri, 26 May 2023 19:30:25 +0000 (15:30 -0400)]
Merge branch '20520-instance-init-command'

refs #20520

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

11 months agoIncrease WatchdogInterval and WatchdogThreshold
Peter Amstutz [Fri, 26 May 2023 19:26:12 +0000 (15:26 -0400)]
Increase WatchdogInterval and WatchdogThreshold

Quick workaround for customer, we should add these to the config file.

refs #20595

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20511-aborted-boot'
Tom Clegg [Fri, 26 May 2023 14:29:16 +0000 (10:29 -0400)]
Merge branch '20511-aborted-boot'

refs #20511

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

11 months agoMerge branch '12684-pysdk-auto-retry'
Brett Smith [Thu, 18 May 2023 15:40:55 +0000 (11:40 -0400)]
Merge branch '12684-pysdk-auto-retry'

Closes #12684.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

11 months agoFix typo 2_6_1 -> 2_6_2
Peter Amstutz [Mon, 22 May 2023 21:20:22 +0000 (17:20 -0400)]
Fix typo 2_6_1 -> 2_6_2

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoAdd 2.6.2 to the upgrading notes
Peter Amstutz [Mon, 22 May 2023 21:16:12 +0000 (17:16 -0400)]
Add 2.6.2 to the upgrading notes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20529-container-deadlocks' refs #20529 2.6.2
Peter Amstutz [Mon, 22 May 2023 15:33:46 +0000 (11:33 -0400)]
Merge branch '20529-container-deadlocks' refs #20529

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20527-group-contents-select-doc'
Brett Smith [Mon, 22 May 2023 14:47:29 +0000 (10:47 -0400)]
Merge branch '20527-group-contents-select-doc'

Closes #20527.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

11 months agoMerge branch '20325-go-docker-distribution-upgrade2'. Refs #20325
Lucas Di Pentima [Wed, 17 May 2023 13:20:54 +0000 (10:20 -0300)]
Merge branch '20325-go-docker-distribution-upgrade2'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

11 months agoRefine PySDK collection walk recipe
Brett Smith [Thu, 11 May 2023 15:53:41 +0000 (11:53 -0400)]
Refine PySDK collection walk recipe

Use PurePosixPath to clarify that we're strictly doing path manipulation.
(It will also behave better on Windows, although I'm not sure if the SDK
itself is Windows-ready yet.)

Keep Path objects in the queue to reduce local state.

No issue #

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

11 months agoMerge branch '20325-jquery-rails-upgrade'. Refs #20325
Lucas Di Pentima [Tue, 9 May 2023 19:49:09 +0000 (16:49 -0300)]
Merge branch '20325-jquery-rails-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

11 months agoMerge branch '20457-queue-churn'
Tom Clegg [Wed, 17 May 2023 15:19:35 +0000 (11:19 -0400)]
Merge branch '20457-queue-churn'

refs #20457
refs #20511

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

11 months agoMerge branch '20457-max-supervisors-overquota'
Tom Clegg [Fri, 5 May 2023 17:57:11 +0000 (13:57 -0400)]
Merge branch '20457-max-supervisors-overquota'

refs #20457

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

11 months agoMerge branch '20470-contents-select' refs #20470
Peter Amstutz [Fri, 5 May 2023 17:33:57 +0000 (13:33 -0400)]
Merge branch '20470-contents-select' refs #20470

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20457-max-supervisors-overquota' refs #20457
Peter Amstutz [Fri, 5 May 2023 17:12:25 +0000 (13:12 -0400)]
Merge branch '20457-max-supervisors-overquota' refs #20457

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20472-priority-update' refs #20472
Peter Amstutz [Thu, 4 May 2023 23:13:51 +0000 (19:13 -0400)]
Merge branch '20472-priority-update' refs #20472

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

11 months agoMerge branch '20475-dump-busy-queue'
Tom Clegg [Thu, 4 May 2023 20:17:08 +0000 (16:17 -0400)]
Merge branch '20475-dump-busy-queue'

closes #20475

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20457-logs-and-mem-usage'
Tom Clegg [Mon, 1 May 2023 20:11:07 +0000 (16:11 -0400)]
Merge branch '20457-logs-and-mem-usage'

refs #20457

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20432-getting-containers' refs #20432
Peter Amstutz [Mon, 1 May 2023 19:17:37 +0000 (15:17 -0400)]
Merge branch '20432-getting-containers' refs #20432

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoMerge branch '20447-less-table-locking'
Tom Clegg [Mon, 1 May 2023 16:07:45 +0000 (12:07 -0400)]
Merge branch '20447-less-table-locking'

fixes #20447

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20462-workflow-prefix' refs #20462
Peter Amstutz [Mon, 1 May 2023 15:44:16 +0000 (11:44 -0400)]
Merge branch '20462-workflow-prefix' refs #20462

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoMerge branch '20422-cache-slot' refs #20422
Peter Amstutz [Fri, 28 Apr 2023 15:31:49 +0000 (11:31 -0400)]
Merge branch '20422-cache-slot' refs #20422

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months ago18799: Version PySDK's typing_extensions dependency
Brett Smith [Wed, 26 Apr 2023 18:29:16 +0000 (14:29 -0400)]
18799: Version PySDK's typing_extensions dependency

3.7.4 added TypedDict, which is exactly what we use.
Refs #18799, #20446.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

12 months agoImport dataclasses for py 3.6, refs #20257
Peter Amstutz [Fri, 21 Apr 2023 17:44:53 +0000 (13:44 -0400)]
Import dataclasses for py 3.6, refs #20257

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoMerge branch '20257-http-import' refs #20257
Peter Amstutz [Fri, 21 Apr 2023 16:23:27 +0000 (12:23 -0400)]
Merge branch '20257-http-import' refs #20257

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoAdd 2.6.1 to upgrade notes. 2.6.1
Peter Amstutz [Mon, 17 Apr 2023 19:30:54 +0000 (15:30 -0400)]
Add 2.6.1 to upgrade notes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoMerge branch '20123-access-token-doc' refs #20123
Peter Amstutz [Mon, 17 Apr 2023 18:43:01 +0000 (14:43 -0400)]
Merge branch '20123-access-token-doc' refs #20123

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoMerge branch '20203-rails-cache'
Tom Clegg [Mon, 17 Apr 2023 15:06:54 +0000 (11:06 -0400)]
Merge branch '20203-rails-cache'

fixes #20203

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20325-go-docker-distribution-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 18:00:56 +0000 (15:00 -0300)]
Merge branch '20325-go-docker-distribution-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20325-globalid-gem-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 16:15:17 +0000 (13:15 -0300)]
Merge branch '20325-globalid-gem-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20334-slow-dedup-migration'
Tom Clegg [Fri, 14 Apr 2023 16:02:25 +0000 (12:02 -0400)]
Merge branch '20334-slow-dedup-migration'

fixes #20334

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20336-filter-bigint'
Tom Clegg [Fri, 14 Apr 2023 16:00:58 +0000 (12:00 -0400)]
Merge branch '20336-filter-bigint'

fixes #20336

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20325-go-containerd-upgrade'. Refs #20325
Lucas Di Pentima [Fri, 14 Apr 2023 15:18:07 +0000 (12:18 -0300)]
Merge branch '20325-go-containerd-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20325-nokogiri-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 17:54:26 +0000 (14:54 -0300)]
Merge branch '20325-nokogiri-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20325-go-x-net-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 17:25:27 +0000 (14:25 -0300)]
Merge branch '20325-go-x-net-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20325-rack-upgrade'. Refs #20325
Lucas Di Pentima [Thu, 13 Apr 2023 16:41:33 +0000 (13:41 -0300)]
Merge branch '20325-rack-upgrade'. Refs #20325

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoArvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 6 Apr 2023 19:06:32 +0000 (15:06 -0400)]
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

refs #20145

12 months agoSet DEFAULT_TAG for this arvbox script
Peter Amstutz [Thu, 6 Apr 2023 18:47:42 +0000 (14:47 -0400)]
Set DEFAULT_TAG for this arvbox script

refs #20145

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoUpdate R package version to 2.6.0
Peter Amstutz [Thu, 6 Apr 2023 18:46:00 +0000 (14:46 -0400)]
Update R package version to 2.6.0

Fortunately the actual release build script replaces the version
number automatically so this is just bringing it in alignment with the
released version.

refs #20145

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months agoAdd 2.6.0 release to upgrade notes 2.6.0
Peter Amstutz [Thu, 6 Apr 2023 16:08:37 +0000 (12:08 -0400)]
Add 2.6.0 release to upgrade notes

refs #20145

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

12 months ago19889: Add an id to the container log method documentation
Brett Smith [Wed, 5 Apr 2023 16:19:23 +0000 (12:19 -0400)]
19889: Add an id to the container log method documentation

To support the release notes.
Refs #19889

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

12 months agoMerge branch '19889-live-log-webdav'
Tom Clegg [Tue, 4 Apr 2023 18:29:13 +0000 (14:29 -0400)]
Merge branch '19889-live-log-webdav'

closes #19889

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '20240-container-update-deadlock'
Tom Clegg [Tue, 4 Apr 2023 18:27:22 +0000 (14:27 -0400)]
Merge branch '20240-container-update-deadlock'

fixes #20240

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Change locking mode to EXCLUSIVE. 20240-container-update-deadlock
Tom Clegg [Tue, 4 Apr 2023 15:18:08 +0000 (11:18 -0400)]
20240: Change locking mode to EXCLUSIVE.

...in order to conflict with SELECT FOR UPDATE.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Don't accept/retry after deadlock in test case.
Tom Clegg [Fri, 31 Mar 2023 15:16:10 +0000 (11:16 -0400)]
20240: Don't accept/retry after deadlock in test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Lock containers table before cascading priority updates.
Tom Clegg [Fri, 31 Mar 2023 15:14:19 +0000 (11:14 -0400)]
20240: Lock containers table before cascading priority updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Expose deadlock in multi-level workflow test case.
Tom Clegg [Thu, 30 Mar 2023 21:19:13 +0000 (17:19 -0400)]
20240: Expose deadlock in multi-level workflow test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Fix test.
Tom Clegg [Thu, 30 Mar 2023 18:11:32 +0000 (14:11 -0400)]
20240: Fix test.

Automatic cancellation of child containers doesn't work as expected if
the parent container is in Queued state, which never happens in real
life, but did happen in the previous version of this test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago20240: Update test to reveal bug.
Tom Clegg [Thu, 23 Mar 2023 21:40:30 +0000 (17:40 -0400)]
20240: Update test to reveal bug.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months ago19889: Fix crash on gateway connect error. 19889-live-log-webdav
Tom Clegg [Tue, 4 Apr 2023 15:00:22 +0000 (11:00 -0400)]
19889: Fix crash on gateway connect error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

12 months agoMerge branch '16379-installer-prometheus-grafana'. Closes #16379
Lucas Di Pentima [Tue, 4 Apr 2023 13:38:50 +0000 (10:38 -0300)]
Merge branch '16379-installer-prometheus-grafana'. Closes #16379

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

12 months agoMerge branch '20295-fix-collection-tree-caching-bug' refs #20295
Peter Amstutz [Tue, 4 Apr 2023 13:26:09 +0000 (09:26 -0400)]
Merge branch '20295-fix-collection-tree-caching-bug' refs #20295

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago18790: Fix logging noise from ENOENT.
Tom Clegg [Mon, 3 Apr 2023 15:23:32 +0000 (11:23 -0400)]
18790: Fix logging noise from ENOENT.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago18790: Fix .../containers/.../log routing.
Tom Clegg [Mon, 3 Apr 2023 15:23:03 +0000 (11:23 -0400)]
18790: Fix .../containers/.../log routing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months agoMerge branch '20264-trusted-clients-portnumber'
Tom Clegg [Fri, 31 Mar 2023 14:12:02 +0000 (10:12 -0400)]
Merge branch '20264-trusted-clients-portnumber'

fixes #20264
fixes #20140

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months agoMerge branch '20235-probe-after-upgrade'
Tom Clegg [Fri, 31 Mar 2023 14:10:26 +0000 (10:10 -0400)]
Merge branch '20235-probe-after-upgrade'

fixes #20235

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months agoAdd unit test 20295-fix-collection-tree-caching-bug
Konrad Rudolph [Thu, 30 Mar 2023 17:25:25 +0000 (19:25 +0200)]
Add unit test

Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>

13 months agoFix collection tree caching bug
Konrad Rudolph [Thu, 30 Mar 2023 16:33:15 +0000 (18:33 +0200)]
Fix collection tree caching bug

Before this fix, accessing two files inside the same collection in
succession via `collection$get` will fail on the second file because the
private tree inside the `collection` is only populated with the relative
path of the first file.

Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>

13 months ago16379: Tweak doc on starring the dashboards on grafana 16379-installer-prometheus-grafana
Peter Amstutz [Thu, 30 Mar 2023 17:18:38 +0000 (13:18 -0400)]
16379: Tweak doc on starring the dashboards on grafana

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago16379: Reorganize local.params examples
Peter Amstutz [Thu, 30 Mar 2023 14:56:44 +0000 (10:56 -0400)]
16379: Reorganize local.params examples

Make sure the things that always need to be changed are at the top.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago16379: Small doc updates about prometheus and grafana
Peter Amstutz [Thu, 30 Mar 2023 14:30:41 +0000 (10:30 -0400)]
16379: Small doc updates about prometheus and grafana

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago16379: Updates for grafana install
Peter Amstutz [Thu, 30 Mar 2023 13:27:35 +0000 (09:27 -0400)]
16379: Updates for grafana install

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago16379: Add prometheus and grafana to workbench aliases
Peter Amstutz [Tue, 28 Mar 2023 22:07:43 +0000 (18:07 -0400)]
16379: Add prometheus and grafana to workbench aliases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago20264: Reject redirect target with userinfo. 20264-trusted-clients-portnumber
Tom Clegg [Tue, 28 Mar 2023 21:05:26 +0000 (17:05 -0400)]
20264: Reject redirect target with userinfo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago16379: doc updates
Peter Amstutz [Tue, 28 Mar 2023 20:38:25 +0000 (16:38 -0400)]
16379: doc updates

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

13 months ago19889: Preserve WebDAV path when proxying to keep-web.
Tom Clegg [Tue, 28 Mar 2023 19:51:59 +0000 (15:51 -0400)]
19889: Preserve WebDAV path when proxying to keep-web.

WebDAV clients expect the path in the server response to match the
request.

Previously when proxying to keep-web we were rewriting the request
from /arvados/v1/containers/{uuid}/log/stderr.txt to
/by_id/{pdh}/stderr.txt, so the response referred to
/by_id/{pdh}/stderr.txt.

With this change, we leave the request path alone and use a new
X-Webdav-Prefix request header (/arvados/v1/containers/{uuid}/log in
this case) to tell keep-web to strip that part when accessing the
virtual filesystem.

New test uses cadaver, which fails on the previous version with

Could not access /arvados/v1/containers/zzzzz-dz642-queuedcontainer/log/ (not WebDAV-enabled?):
Did not find a collection resource.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months agoMerge branch '20270-salt-installer-less-instances'. Closes #20270
Lucas Di Pentima [Tue, 28 Mar 2023 19:40:01 +0000 (16:40 -0300)]
Merge branch '20270-salt-installer-less-instances'. Closes #20270

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Fixes missing 'monitoring' role on the workbench node.
Lucas Di Pentima [Tue, 28 Mar 2023 18:22:16 +0000 (15:22 -0300)]
16379: Fixes missing 'monitoring' role on the workbench node.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Removes prometheus config for keep1 & keepproxy nodes.
Lucas Di Pentima [Tue, 28 Mar 2023 18:08:15 +0000 (15:08 -0300)]
16379: Removes prometheus config for keep1 & keepproxy nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Sets grafana's admin user password through the CLI.
Lucas Di Pentima [Tue, 21 Mar 2023 22:07:18 +0000 (19:07 -0300)]
16379: Sets grafana's admin user password through the CLI.

Although it's being correctly set on the config file, the admin credentials
are not bein honored at first server start, and so the default password
is set in the database.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Makes sure the nginx formula is included in the keepweb's top file.
Lucas Di Pentima [Tue, 21 Mar 2023 20:30:58 +0000 (17:30 -0300)]
16379: Makes sure the nginx formula is included in the keepweb's top file.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Adds nginx dependency to the htpasswd setup.
Lucas Di Pentima [Fri, 17 Mar 2023 20:39:11 +0000 (17:39 -0300)]
16379: Adds nginx dependency to the htpasswd setup.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Adds grafana with default dashboards, moves prometheus hostname.
Lucas Di Pentima [Thu, 9 Mar 2023 20:46:53 +0000 (17:46 -0300)]
16379: Adds grafana with default dashboards, moves prometheus hostname.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Adds basic auth to the prometheus UI.
Lucas Di Pentima [Fri, 3 Mar 2023 19:24:02 +0000 (16:24 -0300)]
16379: Adds basic auth to the prometheus UI.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Moves prometheus to its own subdomain instead of 'mon.prefix.domain.tld'
Lucas Di Pentima [Tue, 28 Feb 2023 18:54:55 +0000 (15:54 -0300)]
16379: Moves prometheus to its own subdomain instead of 'mon.prefix.domain.tld'

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Adds prometheus' node_exporter to all nodes.
Lucas Di Pentima [Mon, 27 Feb 2023 16:29:06 +0000 (13:29 -0300)]
16379: Adds prometheus' node_exporter to all nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago16379: Adds prometheus service for monitoring arvados & postgresql.
Lucas Di Pentima [Tue, 21 Feb 2023 21:21:58 +0000 (18:21 -0300)]
16379: Adds prometheus service for monitoring arvados & postgresql.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago20270: Removes keep1 & keepproxy nodes. Uses SSH jumphost to deploy nodes. 20270-salt-installer-less-instances
Lucas Di Pentima [Tue, 28 Mar 2023 17:19:57 +0000 (14:19 -0300)]
20270: Removes keep1 & keepproxy nodes. Uses SSH jumphost to deploy nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago20270: Refactors the VPC code so that private nodes can access the Internet.
Lucas Di Pentima [Tue, 28 Mar 2023 17:17:30 +0000 (14:17 -0300)]
20270: Refactors the VPC code so that private nodes can access the Internet.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago19889: Add www-authenticate header with 401 Unauthorized response.
Tom Clegg [Mon, 27 Mar 2023 23:24:02 +0000 (19:24 -0400)]
19889: Add www-authenticate header with 401 Unauthorized response.

Test webdav with cadaver.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago19889: Remove unused code.
Tom Clegg [Mon, 27 Mar 2023 23:23:42 +0000 (19:23 -0400)]
19889: Remove unused code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago19889: Fix comment.
Tom Clegg [Mon, 27 Mar 2023 23:23:32 +0000 (19:23 -0400)]
19889: Fix comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months agoMerge branch '20242-dup-trash-lists'
Tom Clegg [Mon, 27 Mar 2023 18:31:40 +0000 (14:31 -0400)]
Merge branch '20242-dup-trash-lists'

fixes #20242

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago20140: Accept wildcards in TrustedClients.
Tom Clegg [Mon, 27 Mar 2023 18:29:38 +0000 (14:29 -0400)]
20140: Accept wildcards in TrustedClients.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

13 months ago20270: Copies .gitignore file to avoid adding binaries to the repository.
Lucas Di Pentima [Mon, 27 Mar 2023 17:43:31 +0000 (14:43 -0300)]
20270: Copies .gitignore file to avoid adding binaries to the repository.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago20270: Splits hosts into public & private, saving on public IPs requirements.
Lucas Di Pentima [Mon, 27 Mar 2023 17:15:02 +0000 (14:15 -0300)]
20270: Splits hosts into public & private, saving on public IPs requirements.

Also, asks for less number of instances: there's no need to multiple keepstore
nodes, and keep-web can be run on the same node as workbench.
This makes the basic default cluster to go from 6 to 4 nodes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

13 months ago20264: Ignore superfluous :443 and :80 in trusted origin check.
Tom Clegg [Mon, 27 Mar 2023 15:38:11 +0000 (11:38 -0400)]
20264: Ignore superfluous :443 and :80 in trusted origin check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>