arvados.git
2 years agoMerge branch '19081-singularity-no-eval'
Tom Clegg [Tue, 17 May 2022 18:13:34 +0000 (14:13 -0400)]
Merge branch '19081-singularity-no-eval'

refs #19081

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19094-log-versions'
Tom Clegg [Tue, 17 May 2022 17:25:54 +0000 (13:25 -0400)]
Merge branch '19094-log-versions'

closes #19094

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoBash syntax in arvados-tests.sh requires bash, fixed.
Peter Amstutz [Mon, 16 May 2022 20:11:28 +0000 (16:11 -0400)]
Bash syntax in arvados-tests.sh requires bash, fixed.

refs #19109

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFix set_secondary recursion error
Peter Amstutz [Mon, 16 May 2022 18:51:20 +0000 (14:51 -0400)]
Fix set_secondary recursion error

skip some problematic conformance tests in test_with_arvbox

refs #19109

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFix arvados-tests.sh to use python from a-c-r package if available.
Peter Amstutz [Mon, 16 May 2022 17:20:29 +0000 (13:20 -0400)]
Fix arvados-tests.sh to use python from a-c-r package if available.

Add note about the purpose of arvados-tests.sh

refs #19070

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19126-nginx-proxy-settings-change'
Ward Vandewege [Fri, 13 May 2022 18:46:48 +0000 (14:46 -0400)]
Merge branch '19126-nginx-proxy-settings-change'

closes #19126

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19109-upload-secondary' refs #19109
Peter Amstutz [Fri, 13 May 2022 15:08:31 +0000 (11:08 -0400)]
Merge branch '19109-upload-secondary' refs #19109

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19070-update-workflow-deps' refs #19070
Peter Amstutz [Fri, 13 May 2022 15:08:08 +0000 (11:08 -0400)]
Merge branch '19070-update-workflow-deps' refs #19070

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '19095-singularity-bugfix'
Tom Clegg [Tue, 10 May 2022 14:04:34 +0000 (10:04 -0400)]
Merge branch '19095-singularity-bugfix'

fixes #19095

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '19127-remove-warnings'
Ward Vandewege [Fri, 13 May 2022 14:13:29 +0000 (10:13 -0400)]
Merge branch '19127-remove-warnings'

closes #19127

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19103-aca-list-default-limit'. Closes #19103
Lucas Di Pentima [Wed, 11 May 2022 14:02:50 +0000 (11:02 -0300)]
Merge branch '19103-aca-list-default-limit'. Closes #19103

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

2 years agoMerge branch '18995-code-cleanup-5'
Ward Vandewege [Fri, 6 May 2022 12:47:39 +0000 (08:47 -0400)]
Merge branch '18995-code-cleanup-5'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19015-rvm-gemset-passenger-fix'
Ward Vandewege [Fri, 6 May 2022 12:20:49 +0000 (08:20 -0400)]
Merge branch '19015-rvm-gemset-passenger-fix'

closes #19015

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '19077-arvbox-fixes' refs #19077
Peter Amstutz [Mon, 2 May 2022 21:32:13 +0000 (17:32 -0400)]
Merge branch '19077-arvbox-fixes' refs #19077

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18696-rnaseq-training' refs #18696
Peter Amstutz [Wed, 27 Apr 2022 13:50:44 +0000 (09:50 -0400)]
Merge branch '18696-rnaseq-training' refs #18696

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18835-uploadfiles-test' refs #18835
Peter Amstutz [Tue, 26 Apr 2022 13:28:43 +0000 (09:28 -0400)]
Merge branch '18835-uploadfiles-test' refs #18835

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17301-cwl-oom' refs #17301
Peter Amstutz [Mon, 25 Apr 2022 20:06:06 +0000 (16:06 -0400)]
Merge branch '17301-cwl-oom' refs #17301

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18888-cwl-dir-arg' refs #18888
Peter Amstutz [Mon, 25 Apr 2022 13:52:48 +0000 (09:52 -0400)]
Merge branch '18888-cwl-dir-arg' refs #18888

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18894-manifest-size-doc' refs #18894
Peter Amstutz [Wed, 20 Apr 2022 14:43:27 +0000 (10:43 -0400)]
Merge branch '18894-manifest-size-doc' refs #18894

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18995-code-cleanup-2'
Ward Vandewege [Fri, 15 Apr 2022 22:10:36 +0000 (18:10 -0400)]
Merge branch '18995-code-cleanup-2'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18995-code-cleanup-4'
Ward Vandewege [Fri, 15 Apr 2022 20:22:24 +0000 (16:22 -0400)]
Merge branch '18995-code-cleanup-4'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18995-code-cleanup-3'
Ward Vandewege [Fri, 15 Apr 2022 18:23:34 +0000 (14:23 -0400)]
Merge branch '18995-code-cleanup-3'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18995-code-cleanup-1'
Ward Vandewege [Fri, 15 Apr 2022 15:45:59 +0000 (11:45 -0400)]
Merge branch '18995-code-cleanup-1'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18995-addrislocal'
Tom Clegg [Thu, 14 Apr 2022 19:41:16 +0000 (15:41 -0400)]
Merge branch '18995-addrislocal'

refs #18995

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18976-c-d-l-dep'
Ward Vandewege [Wed, 13 Apr 2022 15:12:48 +0000 (11:12 -0400)]
Merge branch '18976-c-d-l-dep'

closes #18976

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoInstaller bugfix (part 2).
Ward Vandewege [Tue, 12 Apr 2022 20:10:10 +0000 (16:10 -0400)]
Installer bugfix (part 2).

refs #16375

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoInstaller bugfix.
Ward Vandewege [Tue, 12 Apr 2022 18:20:18 +0000 (14:20 -0400)]
Installer bugfix.

refs #16375

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMake compute node image script compatible with Ubuntu.
Ward Vandewege [Thu, 26 May 2022 16:34:20 +0000 (12:34 -0400)]
Make compute node image script compatible with Ubuntu.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19164: update doc accordingly.
Ward Vandewege [Thu, 26 May 2022 15:56:20 +0000 (11:56 -0400)]
19164: update doc accordingly.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19164: rework the help text for all the options of the compute image
Ward Vandewege [Thu, 26 May 2022 12:27:57 +0000 (08:27 -0400)]
19164: rework the help text for all the options of the compute image
       build script.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19164: add a few flags to the compute image builder script.
Ward Vandewege [Wed, 25 May 2022 19:11:39 +0000 (15:11 -0400)]
19164: add a few flags to the compute image builder script.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoProvision scripts: do not hardcode a ruby version when using RVM, use
Ward Vandewege [Wed, 25 May 2022 20:03:14 +0000 (16:03 -0400)]
Provision scripts: do not hardcode a ruby version when using RVM, use
the generic wrapper instead.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19075: When constructing a workflow request, set use_existing=false
Peter Amstutz [Wed, 27 Apr 2022 20:18:57 +0000 (16:18 -0400)]
19075: When constructing a workflow request, set use_existing=false

refs #19075

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoSalt installer multinode AWS example: modernize Arvados config a bit,
Ward Vandewege [Tue, 26 Apr 2022 12:52:53 +0000 (08:52 -0400)]
Salt installer multinode AWS example: modernize Arvados config a bit,
switch to using one S3 bucket for simplicity, make sure that the default
setting for CloudVMs/DriverParameters/IAMInstanceProfile matches
Volumes/DriverParameters/IAMRole.

refs #19054

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19054: more doc changes.
Ward Vandewege [Mon, 25 Apr 2022 20:32:52 +0000 (16:32 -0400)]
19054: more doc changes.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19054: doc update
Ward Vandewege [Mon, 25 Apr 2022 20:13:38 +0000 (16:13 -0400)]
19054: doc update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '18994-cwl-basename' refs #18994
Peter Amstutz [Tue, 19 Apr 2022 14:13:32 +0000 (10:13 -0400)]
Merge branch '18994-cwl-basename' refs #18994

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18992-hpc-local-keepstore'
Tom Clegg [Fri, 15 Apr 2022 19:28:21 +0000 (15:28 -0400)]
Merge branch '18992-hpc-local-keepstore'

closes #18992

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agodoc: update google analytics tag, remove old shim for IE6-9.
Ward Vandewege [Thu, 14 Apr 2022 19:01:17 +0000 (15:01 -0400)]
doc: update google analytics tag, remove old shim for IE6-9.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago19011: Bump nokogiri to 1.13.4
Ward Vandewege [Thu, 14 Apr 2022 14:43:22 +0000 (10:43 -0400)]
19011: Bump nokogiri to 1.13.4

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoFix package building, work around git issue caused by CVE-2022-24765.
Ward Vandewege [Wed, 13 Apr 2022 11:50:06 +0000 (07:50 -0400)]
Fix package building, work around git issue caused by CVE-2022-24765.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoFix doc for correct CUDARequirement fields
Peter Amstutz [Fri, 8 Apr 2022 19:29:50 +0000 (15:29 -0400)]
Fix doc for correct CUDARequirement fields

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoSync Arvados R SDK version with full release
Peter Amstutz [Fri, 8 Apr 2022 14:44:39 +0000 (10:44 -0400)]
Sync Arvados R SDK version with full release

refs #18701

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoFix the libarvados-perl package build. Fix the version number for that
Ward Vandewege [Fri, 8 Apr 2022 14:07:11 +0000 (10:07 -0400)]
Fix the libarvados-perl package build. Fix the version number for that
package.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoSet release date in admin/upgrading.html
Peter Amstutz [Thu, 7 Apr 2022 20:50:27 +0000 (16:50 -0400)]
Set release date in admin/upgrading.html

refs #18701

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18631-shell-login-sync'
Javier Bértoli [Wed, 6 Apr 2022 13:50:51 +0000 (10:50 -0300)]
Merge branch '18631-shell-login-sync'

closes #18631
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoAdd upgrade note about change in default result order. refs #18943 2.4.0
Peter Amstutz [Tue, 5 Apr 2022 18:53:57 +0000 (14:53 -0400)]
Add upgrade note about change in default result order. refs #18943

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18865: Cherry-pick '18865-collection-permission-links' merge
Tom Clegg [Tue, 5 Apr 2022 15:44:30 +0000 (11:44 -0400)]
18865: Cherry-pick '18865-collection-permission-links' merge

3c73e2337ed73cd44e9bcc2d38a4dd742637ad19 onto 2.4-release

refs #18865

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18943-created-at-index' refs #18943
Peter Amstutz [Mon, 4 Apr 2022 19:01:37 +0000 (15:01 -0400)]
Merge branch '18943-created-at-index' refs #18943

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18887: Fix salted_secret check. Add test.
Ward Vandewege [Mon, 4 Apr 2022 19:01:18 +0000 (15:01 -0400)]
18887: Fix salted_secret check. Add test.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: address review comments.
Ward Vandewege [Mon, 4 Apr 2022 15:00:25 +0000 (11:00 -0400)]
18887: address review comments.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: self.check_anonymous_user_token can now handle a full V2 token in
Ward Vandewege [Fri, 1 Apr 2022 18:09:36 +0000 (14:09 -0400)]
18887: self.check_anonymous_user_token can now handle a full V2 token in
       the config file. It can also verify a salted anonymous token.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: add an integration test.
Ward Vandewege [Thu, 31 Mar 2022 19:50:07 +0000 (15:50 -0400)]
18887: add an integration test.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: undo the warning introduced in 18676 if a v2 anonymous token is
Ward Vandewege [Mon, 28 Mar 2022 20:18:16 +0000 (16:18 -0400)]
18887: undo the warning introduced in 18676 if a v2 anonymous token is
       supplied in the config file, using a v2 anonymous token is now
       fine. Also add a config loader check for the minimum secret
       length if a v2 token is acceptable.

refs #18887, #18676

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: it is OK for controller to forward the local anymous token,
Ward Vandewege [Fri, 25 Mar 2022 13:11:53 +0000 (09:11 -0400)]
18887: it is OK for controller to forward the local anymous token,
       because all anonymous tokens get mapped to the local anonymous
       token on every cluster.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18887: when wb1 sends the anonymous user token, it makes sure to always
Ward Vandewege [Fri, 25 Mar 2022 12:19:10 +0000 (08:19 -0400)]
18887: when wb1 sends the anonymous user token, it makes sure to always
       send a v2 token.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd csv output mode to the user activity script.
Ward Vandewege [Sat, 2 Apr 2022 15:40:04 +0000 (11:40 -0400)]
Add csv output mode to the user activity script.

refs #18903

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18940: Go clients load $HOME/.config/arvados/settings.conf
Tom Clegg [Wed, 30 Mar 2022 18:40:25 +0000 (14:40 -0400)]
18940: Go clients load $HOME/.config/arvados/settings.conf

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18903: make getCollectionName smarter: look up the name by uuid if it is
Ward Vandewege [Fri, 1 Apr 2022 13:05:23 +0000 (09:05 -0400)]
18903: make getCollectionName smarter: look up the name by uuid if it is
       available and fall back to look up by pdh if not. If the latter,
       always return the name of the oldest collection with that pdh.

       Clarify that start/end parameters are in UTC since all timestamps
       in the database are stored in UTC.

       Fix error when printing help text.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: add a date range option to the user activity report.
Ward Vandewege [Sat, 26 Mar 2022 00:30:09 +0000 (20:30 -0400)]
18903: add a date range option to the user activity report.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: handle deleted users properly.
Ward Vandewege [Fri, 25 Mar 2022 21:33:43 +0000 (17:33 -0400)]
18903: handle deleted users properly.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: keep-web: when logging a file upload or download, if the
Ward Vandewege [Wed, 23 Mar 2022 18:08:09 +0000 (14:08 -0400)]
18903: keep-web: when logging a file upload or download, if the
       collection UUID is not known, blank the field rather than
       populating it with the PDH (which is also present in a separate
       field).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: formatting fix for output: always prepend the event timestamp
Ward Vandewege [Wed, 23 Mar 2022 17:04:48 +0000 (13:04 -0400)]
18903: formatting fix for output: always prepend the event timestamp

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: the user activity script needs to look up collections by PDH, not
Ward Vandewege [Wed, 23 Mar 2022 16:07:51 +0000 (12:07 -0400)]
18903: the user activity script needs to look up collections by PDH, not
       UUID (that field is not always populated with a UUID).

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18903: fix uninitialized user object in the user activity script.
Ward Vandewege [Wed, 23 Mar 2022 15:00:08 +0000 (11:00 -0400)]
18903: fix uninitialized user object in the user activity script.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoUpdate hardcoded versions for 2.4 release
Peter Amstutz [Thu, 24 Mar 2022 17:59:08 +0000 (13:59 -0400)]
Update hardcoded versions for 2.4 release

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18941-arv-prefetch' refs #18941
Peter Amstutz [Wed, 30 Mar 2022 19:30:24 +0000 (15:30 -0400)]
Merge branch '18941-arv-prefetch' refs #18941

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Update help text in arv-get subcommand
Peter Amstutz [Wed, 30 Mar 2022 19:29:21 +0000 (15:29 -0400)]
18941: Update help text in arv-get subcommand

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Tweak the prefetch thread max to 7
Peter Amstutz [Wed, 30 Mar 2022 17:27:55 +0000 (13:27 -0400)]
18941: Tweak the prefetch thread max to 7

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Rename cache_slot_get option to 'prefetch' for clarity
Peter Amstutz [Wed, 30 Mar 2022 03:09:46 +0000 (23:09 -0400)]
18941: Rename cache_slot_get option to 'prefetch' for clarity

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix tests.
Peter Amstutz [Wed, 30 Mar 2022 02:55:35 +0000 (22:55 -0400)]
18941: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: clamp prefetch to 6
Peter Amstutz [Wed, 30 Mar 2022 02:13:45 +0000 (22:13 -0400)]
18941: clamp prefetch to 6

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Clamp to 4 block prefetch
Peter Amstutz [Wed, 30 Mar 2022 01:55:26 +0000 (21:55 -0400)]
18941: Clamp to 4 block prefetch

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Clamp between 1 and 16 prefetch threads
Peter Amstutz [Wed, 30 Mar 2022 01:41:20 +0000 (21:41 -0400)]
18941: Clamp between 1 and 16 prefetch threads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoc18941: fix cache_slot_get behavior
Peter Amstutz [Tue, 29 Mar 2022 22:01:54 +0000 (18:01 -0400)]
c18941: fix cache_slot_get behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Separate get() behavior for prefetch
Peter Amstutz [Tue, 29 Mar 2022 20:52:47 +0000 (16:52 -0400)]
18941: Separate get() behavior for prefetch

Add flag to get() which causes it to return None immediately if a cache
slot already exists for a block.

In the standard behavior, if multiple readers try to get() the same
block, the first one will start downloading the block, and all the
others will wait and return the block content from the cache slot
when complete.

With the new optional behavior, if multiple readers try to get() the same
block, the first one will start downloading the block, and all the
others will immediately return None.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix another typo
Peter Amstutz [Tue, 29 Mar 2022 18:54:03 +0000 (14:54 -0400)]
18941: Fix another typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Fix typo
Peter Amstutz [Tue, 29 Mar 2022 18:53:20 +0000 (14:53 -0400)]
18941: Fix typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Need to leave some space for current block
Peter Amstutz [Tue, 29 Mar 2022 18:51:14 +0000 (14:51 -0400)]
18941: Need to leave some space for current block

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Order of operations
Peter Amstutz [Tue, 29 Mar 2022 17:02:10 +0000 (13:02 -0400)]
18941: Order of operations

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: FUSE set number of get threads based on cache size
Peter Amstutz [Tue, 29 Mar 2022 16:58:05 +0000 (12:58 -0400)]
18941: FUSE set number of get threads based on cache size

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: Add --threads option to arv-get
Peter Amstutz [Tue, 29 Mar 2022 16:24:53 +0000 (12:24 -0400)]
18941: Add --threads option to arv-get

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18941: bugfixing prefetch
Peter Amstutz [Tue, 29 Mar 2022 15:42:11 +0000 (15:42 +0000)]
18941: bugfixing prefetch

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18896-controller-logs-tokenUUIDs'
Ward Vandewege [Mon, 28 Mar 2022 19:28:19 +0000 (15:28 -0400)]
Merge branch '18896-controller-logs-tokenUUIDs'

refs #18896

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18896: Test token UUID logging.
Tom Clegg [Mon, 28 Mar 2022 15:23:09 +0000 (11:23 -0400)]
18896: Test token UUID logging.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18896: review feedback, un-pyramid the logrus.Fields update.
Ward Vandewege [Mon, 28 Mar 2022 14:14:09 +0000 (10:14 -0400)]
18896: review feedback, un-pyramid the logrus.Fields update.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18896: add locking. Also handle v1 tokens shorter than 5 characters.
Ward Vandewege [Fri, 25 Mar 2022 20:37:28 +0000 (16:37 -0400)]
18896: add locking. Also handle v1 tokens shorter than 5 characters.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago18896: controller logs the UUIDs of the tokens used in each request.
Ward Vandewege [Fri, 25 Mar 2022 11:58:48 +0000 (07:58 -0400)]
18896: controller logs the UUIDs of the tokens used in each request.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoCorrect company name.
Ward Vandewege [Thu, 24 Mar 2022 21:00:55 +0000 (17:00 -0400)]
Correct company name.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoAdd a citation file.
Ward Vandewege [Thu, 24 Mar 2022 21:00:15 +0000 (17:00 -0400)]
Add a citation file.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoUpdate upgrading notes for pending 2.4.0 release refs #18701
Peter Amstutz [Thu, 24 Mar 2022 18:57:21 +0000 (14:57 -0400)]
Update upgrading notes for pending 2.4.0 release refs #18701

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '18638-noisy-errors-regarding-tests'
Javier Bértoli [Thu, 24 Mar 2022 18:44:43 +0000 (15:44 -0300)]
Merge branch '18638-noisy-errors-regarding-tests'

closes #18638
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18638: Address review comments
Javier Bértoli [Thu, 24 Mar 2022 17:56:33 +0000 (14:56 -0300)]
18638: Address review comments

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years agoMerge branch '18596-preemptible-price-factor'
Tom Clegg [Thu, 24 Mar 2022 15:26:44 +0000 (11:26 -0400)]
Merge branch '18596-preemptible-price-factor'

closes #18596

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18633-install-sudo-passwordless-in-shell'
Javier Bértoli [Thu, 24 Mar 2022 13:27:05 +0000 (10:27 -0300)]
Merge branch '18633-install-sudo-passwordless-in-shell'

closes #18633
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

2 years ago18596: Fix test case to catch up with new default config.
Tom Clegg [Wed, 23 Mar 2022 14:36:30 +0000 (10:36 -0400)]
18596: Fix test case to catch up with new default config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '18180-spot-default' refs #18180
Peter Amstutz [Tue, 22 Mar 2022 21:06:41 +0000 (17:06 -0400)]
Merge branch '18180-spot-default' refs #18180

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18180: Change AlwaysUsePreemptibleInstances: default to false
Peter Amstutz [Tue, 22 Mar 2022 21:04:23 +0000 (17:04 -0400)]
18180: Change AlwaysUsePreemptibleInstances: default to false

Add a little more detail to API docs about preemptible scheduling parameter.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago18596: Warn about PreemptiblePriceFactor / InstanceTypes collision.
Tom Clegg [Tue, 22 Mar 2022 19:24:10 +0000 (15:24 -0400)]
18596: Warn about PreemptiblePriceFactor / InstanceTypes collision.

In case of a collision, use the explicit type, not the automatic one.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago18596: Mention that 1.0 is a reasonable PreemptiblePriceFactor.
Tom Clegg [Tue, 22 Mar 2022 19:23:52 +0000 (15:23 -0400)]
18596: Mention that 1.0 is a reasonable PreemptiblePriceFactor.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>