Stephen Smith [Tue, 2 Apr 2024 13:56:19 +0000 (09:56 -0400)]
21508: Rename class for clarity
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Peter Amstutz [Tue, 2 Apr 2024 13:19:34 +0000 (09:19 -0400)]
21639: Tell the kernel to map in whole blocks with madvise
This increases cold cache performance from 150 MB/s to 400 MB/s
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Stephen Smith [Tue, 2 Apr 2024 13:15:04 +0000 (09:15 -0400)]
21508: Remove remaining io panel image preview code
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Tue, 2 Apr 2024 02:31:14 +0000 (22:31 -0400)]
21508: Condense io panel css and remove redundant wrappers/rules
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Tue, 2 Apr 2024 01:08:07 +0000 (21:08 -0400)]
21508: Fix io panel json content width
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Tue, 2 Apr 2024 01:07:35 +0000 (21:07 -0400)]
21508: Add helpful comments to styles
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Tue, 2 Apr 2024 01:06:33 +0000 (21:06 -0400)]
21508: Add virtual list to io panel and styles to make sizing behave
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Peter Amstutz [Tue, 2 Apr 2024 00:32:49 +0000 (20:32 -0400)]
21639: Note about 0 prefetch threads
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 2 Apr 2024 00:17:22 +0000 (20:17 -0400)]
21639: Keep a running total instead of recomputing total
From profiling, it turns out just summing the block sizes to get the
cache usage was a significant drag on the read() fast path.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 1 Apr 2024 19:58:06 +0000 (15:58 -0400)]
21639: Improve critical path of read() from cache
* Don't use tobytes(), it makes a copy, and it should be be zero-copy.
* Prefetching adds a lot of overhead. Don't do it.
* Don't use a list comprehension to calculate cache size
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Stephen Smith [Sun, 7 Apr 2024 19:48:00 +0000 (15:48 -0400)]
21508: Split mounts and param table styles
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Mon, 1 Apr 2024 14:41:35 +0000 (10:41 -0400)]
21508: Remove image preview from io panel
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Fri, 29 Mar 2024 23:34:05 +0000 (19:34 -0400)]
21508: Expand secondary param values into separate parameters
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Tom Clegg [Mon, 1 Apr 2024 21:51:43 +0000 (17:51 -0400)]
21613: Fix handling of expired token re-validating with new UUID.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Mon, 1 Apr 2024 17:36:16 +0000 (13:36 -0400)]
21448: fixed user-profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 1 Apr 2024 16:59:47 +0000 (12:59 -0400)]
21448: removed move to from processes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 1 Apr 2024 15:50:54 +0000 (11:50 -0400)]
21448: fixed user profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Peter Amstutz [Mon, 1 Apr 2024 15:10:52 +0000 (11:10 -0400)]
21639: Don't return broken cache slots
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Lisa Knox [Mon, 1 Apr 2024 15:07:01 +0000 (11:07 -0400)]
21448: changed int tests to new menu orders Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 1 Apr 2024 13:55:50 +0000 (09:55 -0400)]
21448: fixed toolbar divider so it renders consistently Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Peter Amstutz [Mon, 1 Apr 2024 13:52:53 +0000 (09:52 -0400)]
21639: Use a more efficient data structure for the keep block cache
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Mon, 1 Apr 2024 13:20:02 +0000 (09:20 -0400)]
21636: Test http status returned for unknown backend errors.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 29 Mar 2024 20:58:23 +0000 (16:58 -0400)]
21636: Set ExpiryWindow to avoid credential expiry races.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 29 Mar 2024 20:39:09 +0000 (16:39 -0400)]
Merge branch '21617-fed-content'
fixes #21617
fixes #21533
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 29 Mar 2024 20:37:43 +0000 (16:37 -0400)]
21617: Split test cases.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Fri, 29 Mar 2024 18:44:24 +0000 (14:44 -0400)]
21448: divider in place for toolbar menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 17:54:48 +0000 (13:54 -0400)]
21448: divider in place for context menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 15:20:41 +0000 (11:20 -0400)]
21448: expanded copy to oclipboard change Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 15:15:03 +0000 (11:15 -0400)]
21448: channged wording Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 15:08:11 +0000 (11:08 -0400)]
21448: ms toolbar sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 14:26:14 +0000 (10:26 -0400)]
21448: removed share from process menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 14:07:02 +0000 (10:07 -0400)]
21448: running process also sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 29 Mar 2024 13:48:10 +0000 (09:48 -0400)]
21448: applied sort to all context menus in ticket Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Brett Smith [Thu, 14 Mar 2024 15:17:24 +0000 (11:17 -0400)]
21583: Override base64 dependency so it doesn't appear in Gemfile.lock
See comments and linked bug for rationale.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lisa Knox [Thu, 28 Mar 2024 18:51:28 +0000 (14:51 -0400)]
21448: sorter sorts Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Thu, 28 Mar 2024 17:37:54 +0000 (13:37 -0400)]
21448: initial sorting algo in place Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Thu, 28 Mar 2024 15:01:26 +0000 (11:01 -0400)]
21448: adjusted specs to fit capitalization standardization Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Tom Clegg [Thu, 28 Mar 2024 14:57:44 +0000 (10:57 -0400)]
21598: Don't run trash workers in container-local keepstore.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Thu, 28 Mar 2024 14:36:21 +0000 (10:36 -0400)]
21448: combined contextmenunames and mstoolbaractionnames since they were mostly duplicated and serve the same purpose Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Thu, 28 Mar 2024 13:21:21 +0000 (09:21 -0400)]
21448: replaced all context menu names with string constants Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Peter Amstutz [Thu, 28 Mar 2024 12:17:59 +0000 (08:17 -0400)]
21541: Fix typo
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 27 Mar 2024 21:14:28 +0000 (17:14 -0400)]
21541: Revert change to safeapi.py
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 25 Mar 2024 18:19:32 +0000 (14:19 -0400)]
21541: Code cleanup and additional memory usage improvements
* Add slots to major Directory classes
* Disconnect FuseArvadosFile from ArvadosFile to reduce cyclic
references.
* Clean up _remove_inode loop and use dataclasses for the inode
operations.
* Now calls del_entry on collection_record_file and project_object_file.
It looks like collection_record_file was holding a reference to the
Collection object (and was remaining in the inodes table) even when
CollectionDirectory was cleared. I believe this is the memory leak I
have been looking for.
* Remove the "dead" flag and set parent_inode to None instead. This
clarifies the behavior that directory entries keep their (numeric)
inodes until they are detached from the directory which may have
contributed to infrequent "file not found" errors.
* Adjust cache behavior to only hold objects that are cache-eligible
and have non-zero cache_size. This avoids filling the cache with
entries that are just going to be skipped over.
Overall: Memory usage is mostly stable but does tend to creep up over
time. My best guess is that this is forced because we need to keep
inodes in RAM as long as the kernel maintains a reference to them, so
with multiple processes accessing different filesystem locations, this
is simply RAM required for the working set.
I'm also cautiously optimistic that issues I observed with performance
slowing down with long-lived processes are improved (e.g. fixing
memory leaks means no more unbounded growth of cache_entries, which
means no more time wasted iterating over huge lists).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Lisa Knox [Wed, 27 Mar 2024 15:47:59 +0000 (11:47 -0400)]
21448: expanded string constants Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Wed, 27 Mar 2024 14:01:51 +0000 (10:01 -0400)]
21448: created string constants for menu items Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Tom Clegg [Tue, 26 Mar 2024 14:01:28 +0000 (10:01 -0400)]
21617: Test unauthenticated endpoint + LoginCluster + bad token.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 26 Mar 2024 02:30:10 +0000 (22:30 -0400)]
21617: Interpret any 4xx as token rejection, not an error.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 25 Mar 2024 21:25:10 +0000 (17:25 -0400)]
21617: Accept bogus tokens at endpoints that do not require auth.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Stephen Smith [Mon, 25 Mar 2024 15:47:19 +0000 (11:47 -0400)]
21508: Fix cross origin io parameter image previews
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Stephen Smith [Mon, 25 Mar 2024 15:46:52 +0000 (11:46 -0400)]
21508: Normalize io panel row height
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Brett Smith [Sun, 24 Mar 2024 03:26:58 +0000 (23:26 -0400)]
21601: Ensure pip is new enough to support package building
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 23 Mar 2024 18:16:35 +0000 (14:16 -0400)]
21601: Build Python distro packages from wheels
The immediate problem this solves is that, by building and installing
from a repository of wheels, Python packages can find their
interdependencies without any special logic in the build process.
Other benefits:
* Eliminates some redundant work. We don't have to build the Python SDK
from source multiple times. We can use the published cwltest wheel
instead of building our own.
* Prepares the code for PEP 517 compliance. We only invoke setup.py to
build packages that have not been updated yet. We introspect packages
from their wheels, so we no longer have to introspect the source to
build distro packages.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 23 Mar 2024 18:17:47 +0000 (14:17 -0400)]
21601: Remove PySDK from `install deps`
It's already installed by `install env`. 🤷
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Tue, 19 Mar 2024 03:04:59 +0000 (23:04 -0400)]
21601: Install wheel in run-tests.sh VENV3DIR
We do this basically everywhere else in our build infrastructure, so
doing it here helps avoid discrepancies between those environments.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 16 Mar 2024 23:23:44 +0000 (19:23 -0400)]
21601: Specify Python interdependencies with ~=
This has the same rationale as using <= before, but it's stricter. It
should prevent pip from using release versions to satisfy development
dependencies in the future, and help root out bugs in our build
processes.
DRY up this logic in arvados_version.py.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 16 Mar 2024 23:10:15 +0000 (19:10 -0400)]
21601: Move --short-tests-only check to arvados_version
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 16 Mar 2024 23:02:59 +0000 (19:02 -0400)]
21601: Make arvados_version.py more declarative
The main goal of this change is to introduce the metadata maps near the
top of the file, which we will use to build additional
functionality. The rest of the changes are just modernization or
clean-up based on that.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Thu, 21 Mar 2024 19:11:41 +0000 (15:11 -0400)]
Fix open-ended bundler version in ruby install recipe.
refs #21463
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Thu, 21 Mar 2024 17:17:45 +0000 (13:17 -0400)]
Merge branch '21600-banner-tests'
closes #21600
Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Tom Clegg [Thu, 21 Mar 2024 15:39:15 +0000 (11:39 -0400)]
Merge branch '21578-mount-debug'
closes #21578
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 21 Mar 2024 15:37:16 +0000 (11:37 -0400)]
Merge branch '21603-ec2-subnet-error'
fixes #21603
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 21 Mar 2024 15:21:22 +0000 (11:21 -0400)]
Update test failing on debian 12 if s3cmd is installed.
Also, update `arvados-server install -type test` to install s3cmd if
available.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Thu, 21 Mar 2024 13:10:23 +0000 (09:10 -0400)]
Merge branch '21357-favorites-names'
closes #21357
Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Wed, 20 Mar 2024 13:53:28 +0000 (09:53 -0400)]
21357: changed dispatch in loadFavoritesTRee to a string instead of an object Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 18:41:12 +0000 (14:41 -0400)]
Merge branch 'main' into 21357-favorites-names
refs #21357
Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 17:57:04 +0000 (13:57 -0400)]
21357: added loadsidepaneltree to updatecollection Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 17:26:20 +0000 (13:26 -0400)]
21357: moved resource fetches into a single await Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 14:52:22 +0000 (10:52 -0400)]
21600: combined changes by Stephen and Lucas Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 14:16:09 +0000 (10:16 -0400)]
Merge branch '21412-user-profile-bugs'
closes #21412
Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Tue, 19 Mar 2024 13:21:01 +0000 (09:21 -0400)]
21600: incorporated Stephens changes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 18 Mar 2024 19:59:49 +0000 (15:59 -0400)]
21600: added waits everywhere it could possibly matter Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Tom Clegg [Mon, 18 Mar 2024 19:01:16 +0000 (15:01 -0400)]
21603: Recognize subnet error despite generic error code.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Mon, 18 Mar 2024 18:42:54 +0000 (14:42 -0400)]
21600: tweaked test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Peter Amstutz [Mon, 18 Mar 2024 18:35:21 +0000 (14:35 -0400)]
Merge branch 'main' into 21541-arv-mount-keyerror-rebase refs #21541
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 18 Mar 2024 18:22:05 +0000 (14:22 -0400)]
Merge branch '20455-noopener' refs #20455
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Lisa Knox [Mon, 18 Mar 2024 18:13:45 +0000 (14:13 -0400)]
21600: removed tests to test spec itself Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Stephen Smith [Mon, 18 Mar 2024 17:39:51 +0000 (13:39 -0400)]
Merge branch '21597-cypress-reply-fix' into main. Closes #21597
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Tom Clegg [Mon, 18 Mar 2024 15:39:11 +0000 (11:39 -0400)]
2960: Remove unused code.
refs #2960
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Mon, 18 Mar 2024 15:02:11 +0000 (11:02 -0400)]
21600: changed tooltip expectation Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Tom Clegg [Mon, 18 Mar 2024 14:48:05 +0000 (10:48 -0400)]
Merge branch '21449-lsf-maxruntime'
closes #21449
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 18 Mar 2024 14:44:58 +0000 (10:44 -0400)]
21578: Add -debug alias for -log-level=debug.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 18 Mar 2024 14:31:16 +0000 (10:31 -0400)]
21578: Merge branch 'main' into 21578-mount-debug
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Mon, 18 Mar 2024 14:20:55 +0000 (10:20 -0400)]
21600: used the force Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 18 Mar 2024 13:55:44 +0000 (09:55 -0400)]
21600: set cypress data tag Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 18 Mar 2024 13:16:18 +0000 (09:16 -0400)]
21600: testing wait removal Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 15 Mar 2024 20:33:39 +0000 (16:33 -0400)]
21600: removed only and changed wait to 1 second Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 15 Mar 2024 20:06:45 +0000 (16:06 -0400)]
21600: added 3 second wait Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 15 Mar 2024 19:37:54 +0000 (15:37 -0400)]
21600: added only for testing Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 15 Mar 2024 19:27:54 +0000 (15:27 -0400)]
21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Fri, 15 Mar 2024 19:07:35 +0000 (15:07 -0400)]
21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lucas Di Pentima [Fri, 15 Mar 2024 18:13:11 +0000 (15:13 -0300)]
Merge branch '21165-installer-uninstalls-wb1'. Closes #21165
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Stephen Smith [Fri, 15 Mar 2024 17:15:08 +0000 (13:15 -0400)]
21597: Use req.on('reply' instead of req.reply to workaround cypress bug
https://github.com/cypress-io/cypress/issues/26248#issuecomment-
1623005400
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Peter Amstutz [Fri, 15 Mar 2024 13:56:09 +0000 (09:56 -0400)]
21541: Adjust dependency to arvados-llfuse fork
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 6 Mar 2024 20:03:38 +0000 (15:03 -0500)]
21541: Fix KeyError, segfaults, and memory use issues
* Fixes a segfault on startup due to multiple threads fetching the
cluster config using the same http2 object, which is not threadafe.
Now fetches the relevant configuration
item once (ForwardSlashNameSubstitution), and stores it where all the
threads can access it. (bug #21568)
* Fixes KeyError thrown where a parent inode is removed from the
inodes table before its children.
* In the process of testing, re-discovered a bug where, if the llfuse
_notify_queue fills up, the entire mount process deadlocks.
The previous fix worked by monkey-patching llfuse to replace a
limited-length queue with an unlimited length queue, however changes
in subsequent llfuse versions caused that variable to be hidden from
Python (so the monkey patch didn't fail but it no longer had any
effect either). The solution is to introduce an additional
unlimited-size queue in between the operation handlers and the
limited-size kernel notification queue.
* Because cache management and inode cleanup interact with kernel
notifications (which were moved into a separate thread), I decided
they should also be asynchronous from the operation handlers, so they
are now part of the same thread that processes kernel notifications.
* Attempting to remove an inode that is in use will now at minimum
send a kernel invalidation, which will sometimes nudge the kernel to
forget the inode, enabling us to remove it.
* Filter groups now check if the filter group contains itself so it
doesn't create an infinite path loop that breaks filesystem traversal
tools.
* In the process of testing, found that llfuse didn't wait for the
_notify_queue to drain before closing the FUSE channel, resulting in a
segfault if the _notify_loop thread tried to process any events after
shutdown started. This bug cannot be worked around on the Arvados
side, so I have prepared an arvados-llfuse fork with a bug fix.
* Testing with arv-mount-stress-test (which creates 8 subprocesses that all
traverse the filesystem at the same time) now passes with no
filesystem errors, no deadlocks, no segfaults, and modest memory
usage.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 15 Mar 2024 14:45:03 +0000 (10:45 -0400)]
sdk services installs crunchstat-summary refs #19744
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 15 Mar 2024 16:01:22 +0000 (12:01 -0400)]
Reorder python install steps to address dependency issue.
refs #19744
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 15 Mar 2024 13:56:03 +0000 (09:56 -0400)]
19744: Reorder python install steps to address dependency issue.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lisa Knox [Fri, 15 Mar 2024 13:05:56 +0000 (09:05 -0400)]
Merge branch '21507-long-project-title-wrap'
closes #21507
Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lucas Di Pentima [Thu, 14 Mar 2024 20:54:02 +0000 (17:54 -0300)]
21165: Uninstalls WB1 from single host installations too.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 14 Mar 2024 19:33:33 +0000 (16:33 -0300)]
21165: Adds extra state to uninstall wb1's package from workbench node.
I opted for specifically uninstalling the package instead of adding the
arvados.workbench.package.clean state, because that state also removes
packages that might be needed for other services.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
projects / arvados.git / log