arvados.git
2 years agoMerge branch '18699-boot-multi-cluster'
Tom Clegg [Thu, 14 Apr 2022 19:38:54 +0000 (15:38 -0400)]
Merge branch '18699-boot-multi-cluster'

closes #18699

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agodoc: update google analytics tag, remove old shim for IE6-9.
Ward Vandewege [Thu, 14 Apr 2022 19:01:17 +0000 (15:01 -0400)]
doc: update google analytics tag, remove old shim for IE6-9.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19011-nokogiri-bump'
Ward Vandewege [Thu, 14 Apr 2022 15:20:22 +0000 (11:20 -0400)]
Merge branch '19011-nokogiri-bump'

closes #19011

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19011: Bump nokogiri to 1.13.4
Ward Vandewege [Thu, 14 Apr 2022 14:43:22 +0000 (10:43 -0400)]
19011: Bump nokogiri to 1.13.4

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoThe compute image building script autodetects the correct Go version to
Ward Vandewege [Wed, 13 Apr 2022 19:04:01 +0000 (15:04 -0400)]
The compute image building script autodetects the correct Go version to
install.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18947-dispatch-local'
Tom Clegg [Wed, 13 Apr 2022 19:02:50 +0000 (15:02 -0400)]
Merge branch '18947-dispatch-local'

refs #18947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoThe package building scripts autodetect the correct Go version to
Ward Vandewege [Wed, 13 Apr 2022 18:41:55 +0000 (14:41 -0400)]
The package building scripts autodetect the correct Go version to
install.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18947: Remove separate keepproxy binary from lib/install steps.
Tom Clegg [Wed, 13 Apr 2022 18:34:10 +0000 (14:34 -0400)]
18947: Remove separate keepproxy binary from lib/install steps.

refs #18947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18947: Fix panic in tests when TestIntegration doesn't run first.
Tom Clegg [Wed, 13 Apr 2022 18:27:52 +0000 (14:27 -0400)]
18947: Fix panic in tests when TestIntegration doesn't run first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18947: Improve logging in test cases.
Tom Clegg [Wed, 13 Apr 2022 18:15:45 +0000 (14:15 -0400)]
18947: Improve logging in test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18947: Close trackers before existing dispatch goroutine.
Tom Clegg [Wed, 13 Apr 2022 18:13:26 +0000 (14:13 -0400)]
18947: Close trackers before existing dispatch goroutine.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18699: Print booted clusterID and controller host:port on stdout.
Tom Clegg [Mon, 4 Apr 2022 14:57:24 +0000 (10:57 -0400)]
18699: Print booted clusterID and controller host:port on stdout.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18699: Remove unneeded OIDC config.
Tom Clegg [Mon, 4 Apr 2022 14:34:46 +0000 (10:34 -0400)]
18699: Remove unneeded OIDC config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18699: boot.Supervisor support for multi-cluster config file.
Tom Clegg [Mon, 4 Apr 2022 14:18:07 +0000 (10:18 -0400)]
18699: boot.Supervisor support for multi-cluster config file.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18976-c-d-l-dep'
Ward Vandewege [Wed, 13 Apr 2022 15:12:48 +0000 (11:12 -0400)]
Merge branch '18976-c-d-l-dep'

closes #18976

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18976: Merge branch 'main' into 18976-c-d-l-dep
Ward Vandewege [Wed, 13 Apr 2022 15:11:59 +0000 (11:11 -0400)]
18976: Merge branch 'main' into 18976-c-d-l-dep

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18700-boot-wb2'
Tom Clegg [Wed, 13 Apr 2022 15:07:21 +0000 (11:07 -0400)]
Merge branch '18700-boot-wb2'

closes #18700

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18947: Bring back keepproxy systemd unit file.
Tom Clegg [Wed, 13 Apr 2022 14:38:46 +0000 (10:38 -0400)]
18947: Bring back keepproxy systemd unit file.

refs #18947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix package building, work around git issue caused by CVE-2022-24765.
Ward Vandewege [Wed, 13 Apr 2022 11:50:06 +0000 (07:50 -0400)]
Fix package building, work around git issue caused by CVE-2022-24765.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoInstaller bugfix (part 2).
Ward Vandewege [Tue, 12 Apr 2022 20:10:10 +0000 (16:10 -0400)]
Installer bugfix (part 2).

refs #16375

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoInstaller bugfix.
Ward Vandewege [Tue, 12 Apr 2022 18:20:18 +0000 (14:20 -0400)]
Installer bugfix.

refs #16375

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18700: Default to domain:443 for workbench2 ExternalURL.
Tom Clegg [Tue, 12 Apr 2022 16:02:38 +0000 (12:02 -0400)]
18700: Default to domain:443 for workbench2 ExternalURL.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Merge branch 'main'
Tom Clegg [Tue, 12 Apr 2022 15:54:26 +0000 (11:54 -0400)]
18700: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Fix debian:10 install.
Tom Clegg [Tue, 12 Apr 2022 15:28:11 +0000 (11:28 -0400)]
18700: Fix debian:10 install.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Renumber default/example ports to 9000-9020 and 4440-4460.
Tom Clegg [Tue, 12 Apr 2022 15:01:10 +0000 (11:01 -0400)]
18700: Renumber default/example ports to 9000-9020 and 4440-4460.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '16375-arvados-multinode-testing'
Javier Bértoli [Tue, 12 Apr 2022 14:05:11 +0000 (11:05 -0300)]
Merge branch '16375-arvados-multinode-testing'

closes #16375
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago16375: address review comments
Javier Bértoli [Tue, 12 Apr 2022 14:04:17 +0000 (11:04 -0300)]
16375: address review comments

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18700: Fix max keepproxy request body size.
Tom Clegg [Tue, 12 Apr 2022 07:42:03 +0000 (03:42 -0400)]
18700: Fix max keepproxy request body size.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Run npm:install to prep workbench1 for package.
Tom Clegg [Tue, 12 Apr 2022 07:02:41 +0000 (03:02 -0400)]
18700: Run npm:install to prep workbench1 for package.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Proxy workbench2 to arvados-server-boot's server.
Tom Clegg [Tue, 12 Apr 2022 04:17:22 +0000 (00:17 -0400)]
18700: Proxy workbench2 to arvados-server-boot's server.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Export TLS.Insecure config.
Tom Clegg [Tue, 12 Apr 2022 04:15:07 +0000 (00:15 -0400)]
18700: Export TLS.Insecure config.

so workbench2 can show an appropriate
ARVADOS_API_HOST_INSECURE=... line in the shell env script.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18700: Mount the whole acme dir.
Tom Clegg [Tue, 12 Apr 2022 04:09:03 +0000 (00:09 -0400)]
18700: Mount the whole acme dir.

acmetool uses symlinks like /var/lib/acme/live/$dom/privkey ->
../../keys/$id/privkey

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago16375: revert insecure flag in multi-host template file
Javier Bértoli [Mon, 11 Apr 2022 20:59:28 +0000 (17:59 -0300)]
16375: revert insecure flag in multi-host template file

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '18947-keepproxy'
Tom Clegg [Mon, 11 Apr 2022 15:19:42 +0000 (11:19 -0400)]
Merge branch '18947-keepproxy'

refs #18947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago16375: Add multinode testing code
Javier Bértoli [Fri, 8 Apr 2022 21:30:51 +0000 (18:30 -0300)]
16375: Add multinode testing code

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoFix doc for correct CUDARequirement fields
Peter Amstutz [Fri, 8 Apr 2022 19:29:50 +0000 (15:29 -0400)]
Fix doc for correct CUDARequirement fields

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoSync Arvados R SDK version with full release
Peter Amstutz [Fri, 8 Apr 2022 14:44:39 +0000 (10:44 -0400)]
Sync Arvados R SDK version with full release

refs #18701

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFix the libarvados-perl package build. Fix the version number for that
Ward Vandewege [Fri, 8 Apr 2022 14:07:11 +0000 (10:07 -0400)]
Fix the libarvados-perl package build. Fix the version number for that
package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18976: crunch-dispatch-local needs arv-mount to be present.
Ward Vandewege [Thu, 7 Apr 2022 20:59:45 +0000 (16:59 -0400)]
18976: crunch-dispatch-local needs arv-mount to be present.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoSet release date in admin/upgrading.html
Peter Amstutz [Thu, 7 Apr 2022 20:50:27 +0000 (16:50 -0400)]
Set release date in admin/upgrading.html

refs #18701

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18631-shell-login-sync'
Javier Bértoli [Wed, 6 Apr 2022 13:50:51 +0000 (10:50 -0300)]
Merge branch '18631-shell-login-sync'

closes #18631
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18631: fix path in shell login-sync cron
Javier Bértoli [Wed, 6 Apr 2022 13:30:07 +0000 (10:30 -0300)]
18631: fix path in shell login-sync cron

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18631: Add shell's vm uuid from the shell node state
Javier Bértoli [Tue, 5 Apr 2022 20:54:38 +0000 (17:54 -0300)]
18631: Add shell's vm uuid from the shell node state

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18947: Update test to new log fields. Add back "locator" log field.
Tom Clegg [Tue, 5 Apr 2022 19:21:21 +0000 (15:21 -0400)]
18947: Update test to new log fields. Add back "locator" log field.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoAdd upgrade note about change in default result order. refs #18943
Peter Amstutz [Tue, 5 Apr 2022 18:53:57 +0000 (14:53 -0400)]
Add upgrade note about change in default result order. refs #18943

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18947: Log version at startup.
Tom Clegg [Tue, 5 Apr 2022 18:20:07 +0000 (14:20 -0400)]
18947: Log version at startup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18947: Refactor keepproxy as an arvados-server subcommand.
Tom Clegg [Tue, 5 Apr 2022 18:13:45 +0000 (14:13 -0400)]
18947: Refactor keepproxy as an arvados-server subcommand.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18865-collection-permission-links'
Tom Clegg [Tue, 5 Apr 2022 15:38:09 +0000 (11:38 -0400)]
Merge branch '18865-collection-permission-links'

fixes #18865

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18887-wb1-sends-v2-anonymous-token'
Ward Vandewege [Mon, 4 Apr 2022 19:38:05 +0000 (15:38 -0400)]
Merge branch '18887-wb1-sends-v2-anonymous-token'

closes #18887

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: Merge branch 'main' into 18887-wb1-sends-v2-anonymous-token
Ward Vandewege [Mon, 4 Apr 2022 19:33:28 +0000 (15:33 -0400)]
18887: Merge branch 'main' into 18887-wb1-sends-v2-anonymous-token

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18943-created-at-index' refs #18943
Peter Amstutz [Mon, 4 Apr 2022 19:01:37 +0000 (15:01 -0400)]
Merge branch '18943-created-at-index' refs #18943

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18887: Fix salted_secret check. Add test.
Ward Vandewege [Mon, 4 Apr 2022 19:01:18 +0000 (15:01 -0400)]
18887: Fix salted_secret check. Add test.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18865: Merge branch 'main'
Tom Clegg [Mon, 4 Apr 2022 18:49:11 +0000 (14:49 -0400)]
18865: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18865: Rearrange code & update comments for clarity.
Tom Clegg [Mon, 4 Apr 2022 18:48:19 +0000 (14:48 -0400)]
18865: Rearrange code & update comments for clarity.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18943: Fix workbench bug
Peter Amstutz [Mon, 4 Apr 2022 18:02:40 +0000 (14:02 -0400)]
18943: Fix workbench bug

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: fix fixtures
Peter Amstutz [Mon, 4 Apr 2022 15:21:38 +0000 (11:21 -0400)]
18943: fix fixtures

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: Fixing fixture
Peter Amstutz [Mon, 4 Apr 2022 15:16:49 +0000 (11:16 -0400)]
18943: Fixing fixture

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18887: address review comments.
Ward Vandewege [Mon, 4 Apr 2022 15:00:25 +0000 (11:00 -0400)]
18887: address review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18943: Tweaking fixtures
Peter Amstutz [Mon, 4 Apr 2022 14:50:06 +0000 (10:50 -0400)]
18943: Tweaking fixtures

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: Fix tests
Peter Amstutz [Mon, 4 Apr 2022 14:18:31 +0000 (10:18 -0400)]
18943: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoAdd csv output mode to the user activity script.
Ward Vandewege [Sat, 2 Apr 2022 15:40:04 +0000 (11:40 -0400)]
Add csv output mode to the user activity script.

refs #18903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18943: Drop redundant indexes
Peter Amstutz [Fri, 1 Apr 2022 21:07:43 +0000 (17:07 -0400)]
18943: Drop redundant indexes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: modified_by index doesn't need to be (desc, desc)
Peter Amstutz [Fri, 1 Apr 2022 21:03:44 +0000 (17:03 -0400)]
18943: modified_by index doesn't need to be (desc, desc)

Can use an (asc, asc) index for a (desc, desc) "order by".

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: Adjust default ordering of results to account for index order
Peter Amstutz [Fri, 1 Apr 2022 20:50:46 +0000 (16:50 -0400)]
18943: Adjust default ordering of results to account for index order

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: consistency between modified_at/created_at indexes
Peter Amstutz [Fri, 1 Apr 2022 20:46:49 +0000 (16:46 -0400)]
18943: consistency between modified_at/created_at indexes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: add modified_at descending index
Peter Amstutz [Fri, 1 Apr 2022 20:36:39 +0000 (16:36 -0400)]
18943: add modified_at descending index

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18943: Adjust keyset_list_all for efficient decending order
Peter Amstutz [Fri, 1 Apr 2022 18:47:39 +0000 (14:47 -0400)]
18943: Adjust keyset_list_all for efficient decending order

Default is ascending (created_at asc, uuid asc) but if acending=False
to use the index properly the query needs to be
(created_at desc, uuid desc)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18940-go-settings-conf'
Tom Clegg [Fri, 1 Apr 2022 18:26:30 +0000 (14:26 -0400)]
Merge branch '18940-go-settings-conf'

refs #18940

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18887: self.check_anonymous_user_token can now handle a full V2 token in
Ward Vandewege [Fri, 1 Apr 2022 18:09:36 +0000 (14:09 -0400)]
18887: self.check_anonymous_user_token can now handle a full V2 token in
       the config file. It can also verify a salted anonymous token.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18943: Migrate created_at indexes to (created_at, uuid)
Peter Amstutz [Fri, 1 Apr 2022 17:24:31 +0000 (13:24 -0400)]
18943: Migrate created_at indexes to (created_at, uuid)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18631: remove unneeded dependency installing api into shell nodes
Javier Bértoli [Fri, 1 Apr 2022 16:46:33 +0000 (13:46 -0300)]
18631: remove unneeded dependency installing api into shell nodes

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '18903-fix-activity-script'
Ward Vandewege [Fri, 1 Apr 2022 13:15:06 +0000 (09:15 -0400)]
Merge branch '18903-fix-activity-script'

closes #18903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: Merge branch 'main' into 18903-fix-activity-script
Ward Vandewege [Fri, 1 Apr 2022 13:14:31 +0000 (09:14 -0400)]
18903: Merge branch 'main' into 18903-fix-activity-script

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: make getCollectionName smarter: look up the name by uuid if it is
Ward Vandewege [Fri, 1 Apr 2022 13:05:23 +0000 (09:05 -0400)]
18903: make getCollectionName smarter: look up the name by uuid if it is
       available and fall back to look up by pdh if not. If the latter,
       always return the name of the oldest collection with that pdh.

       Clarify that start/end parameters are in UTC since all timestamps
       in the database are stored in UTC.

       Fix error when printing help text.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: add an integration test.
Ward Vandewege [Thu, 31 Mar 2022 19:50:07 +0000 (15:50 -0400)]
18887: add an integration test.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18631: don't force api resources on shell nodes
Javier Bértoli [Thu, 31 Mar 2022 18:42:23 +0000 (15:42 -0300)]
18631: don't force api resources on shell nodes

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18631: use SystemRootToken for login-sync
Javier Bértoli [Thu, 31 Mar 2022 00:15:29 +0000 (21:15 -0300)]
18631: use SystemRootToken for login-sync

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoRevert "18631: use SystemRootToken for login-sync"
Javier Bértoli [Thu, 31 Mar 2022 00:14:33 +0000 (21:14 -0300)]
Revert "18631: use SystemRootToken for login-sync"

This reverts commit 809668479c6ab182d879ed2aa0cfcae051a5eda7.
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18631: use SystemRootToken for login-sync
Javier Bértoli [Thu, 31 Mar 2022 00:06:13 +0000 (21:06 -0300)]
18631: use SystemRootToken for login-sync

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '18941-arv-prefetch' refs #18941
Peter Amstutz [Wed, 30 Mar 2022 19:30:24 +0000 (15:30 -0400)]
Merge branch '18941-arv-prefetch' refs #18941

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Update help text in arv-get subcommand
Peter Amstutz [Wed, 30 Mar 2022 19:29:21 +0000 (15:29 -0400)]
18941: Update help text in arv-get subcommand

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18940: Go clients load $HOME/.config/arvados/settings.conf
Tom Clegg [Wed, 30 Mar 2022 18:40:25 +0000 (14:40 -0400)]
18940: Go clients load $HOME/.config/arvados/settings.conf

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18941: Tweak the prefetch thread max to 7
Peter Amstutz [Wed, 30 Mar 2022 17:27:55 +0000 (13:27 -0400)]
18941: Tweak the prefetch thread max to 7

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18631: Address review comments, fix VM UUID in cron
Javier Bértoli [Wed, 30 Mar 2022 15:33:49 +0000 (12:33 -0300)]
18631: Address review comments, fix VM UUID in cron

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18631: Address review comments
Javier Bértoli [Wed, 30 Mar 2022 13:00:34 +0000 (10:00 -0300)]
18631: Address review comments

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18941: Rename cache_slot_get option to 'prefetch' for clarity
Peter Amstutz [Wed, 30 Mar 2022 03:09:46 +0000 (23:09 -0400)]
18941: Rename cache_slot_get option to 'prefetch' for clarity

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix tests.
Peter Amstutz [Wed, 30 Mar 2022 02:55:35 +0000 (22:55 -0400)]
18941: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: clamp prefetch to 6
Peter Amstutz [Wed, 30 Mar 2022 02:13:45 +0000 (22:13 -0400)]
18941: clamp prefetch to 6

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Clamp to 4 block prefetch
Peter Amstutz [Wed, 30 Mar 2022 01:55:26 +0000 (21:55 -0400)]
18941: Clamp to 4 block prefetch

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Clamp between 1 and 16 prefetch threads
Peter Amstutz [Wed, 30 Mar 2022 01:41:20 +0000 (21:41 -0400)]
18941: Clamp between 1 and 16 prefetch threads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoc18941: fix cache_slot_get behavior
Peter Amstutz [Tue, 29 Mar 2022 22:01:54 +0000 (18:01 -0400)]
c18941: fix cache_slot_get behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Separate get() behavior for prefetch
Peter Amstutz [Tue, 29 Mar 2022 20:52:47 +0000 (16:52 -0400)]
18941: Separate get() behavior for prefetch

Add flag to get() which causes it to return None immediately if a cache
slot already exists for a block.

In the standard behavior, if multiple readers try to get() the same
block, the first one will start downloading the block, and all the
others will wait and return the block content from the cache slot
when complete.

With the new optional behavior, if multiple readers try to get() the same
block, the first one will start downloading the block, and all the
others will immediately return None.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix another typo
Peter Amstutz [Tue, 29 Mar 2022 18:54:03 +0000 (14:54 -0400)]
18941: Fix another typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix typo
Peter Amstutz [Tue, 29 Mar 2022 18:53:20 +0000 (14:53 -0400)]
18941: Fix typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Need to leave some space for current block
Peter Amstutz [Tue, 29 Mar 2022 18:51:14 +0000 (14:51 -0400)]
18941: Need to leave some space for current block

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Order of operations
Peter Amstutz [Tue, 29 Mar 2022 17:02:10 +0000 (13:02 -0400)]
18941: Order of operations

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: FUSE set number of get threads based on cache size
Peter Amstutz [Tue, 29 Mar 2022 16:58:05 +0000 (12:58 -0400)]
18941: FUSE set number of get threads based on cache size

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Add --threads option to arv-get
Peter Amstutz [Tue, 29 Mar 2022 16:24:53 +0000 (12:24 -0400)]
18941: Add --threads option to arv-get

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: bugfixing prefetch
Peter Amstutz [Tue, 29 Mar 2022 15:42:11 +0000 (15:42 +0000)]
18941: bugfixing prefetch

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18700: Discard local yarn.lock modifications when checking out wb2.
Tom Clegg [Tue, 29 Mar 2022 13:58:17 +0000 (09:58 -0400)]
18700: Discard local yarn.lock modifications when checking out wb2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>