arvados.git
7 months ago21639: Wrap around read counter, have slightly less indirection
Peter Amstutz [Sat, 6 Apr 2024 00:20:03 +0000 (20:20 -0400)]
21639: Wrap around read counter, have slightly less indirection

Also add a comment about the "every 128 read ops" logic.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Reenable prefetch, but not on every read()
Peter Amstutz [Fri, 5 Apr 2024 22:53:37 +0000 (18:53 -0400)]
21639: Reenable prefetch, but not on every read()

Only do prefetch every 128 invocations of read().

This should dramatically reduce the overhead of computing prefetch
while still getting some or moste of the benefits of prefetching.

Indeed, benchmarking suggests that this prefetching strategy, by
advising the kernel to map blocks into RAM, may actually improve
throughput on the high end.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months agoMerge branch '21448-menu-reorder'
Lisa Knox [Fri, 5 Apr 2024 13:11:35 +0000 (09:11 -0400)]
Merge branch '21448-menu-reorder'

closes #21448

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21508-io-panel-performance' into main. Closes #21508
Stephen Smith [Thu, 4 Apr 2024 19:06:32 +0000 (15:06 -0400)]
Merge branch '21508-io-panel-performance' into main. Closes #21508

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months agoMerge branch '21639-keep-cache-dict' refs #21639
Peter Amstutz [Thu, 4 Apr 2024 16:36:15 +0000 (12:36 -0400)]
Merge branch '21639-keep-cache-dict' refs #21639

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Add type hint to cachelist
Peter Amstutz [Thu, 4 Apr 2024 15:57:54 +0000 (11:57 -0400)]
21639: Add type hint to cachelist

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months agoMerge branch '21583-railsapi-base64-gem'
Brett Smith [Thu, 4 Apr 2024 14:11:47 +0000 (10:11 -0400)]
Merge branch '21583-railsapi-base64-gem'

Refs #21583.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21583: Add test for base64 gem lock 21583-railsapi-base64-gem
Brett Smith [Thu, 4 Apr 2024 13:39:35 +0000 (09:39 -0400)]
21583: Add test for base64 gem lock

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21639: Have to actually re-add storage classes test 21639-keep-cache-dict
Peter Amstutz [Wed, 3 Apr 2024 21:32:45 +0000 (17:32 -0400)]
21639: Have to actually re-add storage classes test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: PEP8 cleanup
Peter Amstutz [Wed, 3 Apr 2024 21:28:37 +0000 (17:28 -0400)]
21639: PEP8 cleanup

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Code cleanup
Peter Amstutz [Wed, 3 Apr 2024 21:17:50 +0000 (17:17 -0400)]
21639: Code cleanup

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Move KeepStorageClassesTestCase to another file
Peter Amstutz [Wed, 3 Apr 2024 20:44:42 +0000 (16:44 -0400)]
21639: Move KeepStorageClassesTestCase to another file

It was failing due to some crosstalk with another test in test_keep_client.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21448: adjusted test specs Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii... 21448-menu-reorder
Lisa Knox [Wed, 3 Apr 2024 15:52:11 +0000 (11:52 -0400)]
21448: adjusted test specs Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: removed toolbar width transition Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Wed, 3 Apr 2024 15:26:50 +0000 (11:26 -0400)]
21448: removed toolbar width transition Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21508: Make io panel cypress test more robust 21508-io-panel-performance
Stephen Smith [Wed, 3 Apr 2024 15:23:59 +0000 (11:23 -0400)]
21508: Make io panel cypress test more robust

Scrolls each checked input into view to advance virtual list and enumerates all
expected params

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21448: added default multiselect order Arvados-DCO-1.1-Signed-off-by: Lisa Knox ...
Lisa Knox [Wed, 3 Apr 2024 15:04:14 +0000 (11:04 -0400)]
21448: added default multiselect order Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21508: Fix unit test for io panel by mocking autosizer for virtuallist
Stephen Smith [Wed, 3 Apr 2024 14:10:29 +0000 (10:10 -0400)]
21508: Fix unit test for io panel by mocking autosizer for virtuallist

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21639: Adjust test mocking
Peter Amstutz [Wed, 3 Apr 2024 13:57:00 +0000 (09:57 -0400)]
21639: Adjust test mocking

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21448: moved share button to 1st in 2nd group menuArvados-DCO-1.1-Signed-off-by:...
Lisa Knox [Wed, 3 Apr 2024 13:55:39 +0000 (09:55 -0400)]
21448: moved share button to 1st in 2nd group menuArvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: fixed dividers in toolbar menuArvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Wed, 3 Apr 2024 13:46:24 +0000 (09:46 -0400)]
21448: fixed dividers in toolbar menuArvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21601-setuptools-git-deps'
Brett Smith [Tue, 2 Apr 2024 21:18:58 +0000 (17:18 -0400)]
Merge branch '21601-setuptools-git-deps'

Refs #21601.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Add test for arvados_version.py synchronization 21601-setuptools-git-deps
Brett Smith [Tue, 2 Apr 2024 20:50:41 +0000 (16:50 -0400)]
21601: Add test for arvados_version.py synchronization

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21508: Fix process io cypress tests, remove image preview test
Stephen Smith [Tue, 2 Apr 2024 20:32:47 +0000 (16:32 -0400)]
21508: Fix process io cypress tests, remove image preview test

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Tweak io panel name column width
Stephen Smith [Tue, 2 Apr 2024 19:51:42 +0000 (15:51 -0400)]
21508: Tweak io panel name column width

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Add tooltip to external http/s files
Stephen Smith [Tue, 2 Apr 2024 19:28:34 +0000 (15:28 -0400)]
21508: Add tooltip to external http/s files

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Add tooltip helpers for primitive & primitive array display
Stephen Smith [Tue, 2 Apr 2024 17:54:44 +0000 (13:54 -0400)]
21508: Add tooltip helpers for primitive & primitive array display

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

21508: add guard

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21448: removed dividers from overflow menuArvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Tue, 2 Apr 2024 18:35:15 +0000 (14:35 -0400)]
21448: removed dividers from overflow menuArvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21613-fed-token-exp'
Tom Clegg [Tue, 2 Apr 2024 17:01:44 +0000 (13:01 -0400)]
Merge branch '21613-fed-token-exp'

fixes #21613

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21508: Fix panel container sizing and json tab scrollbars
Stephen Smith [Tue, 2 Apr 2024 15:53:03 +0000 (11:53 -0400)]
21508: Fix panel container sizing and json tab scrollbars

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months agoMerge branch '21541-arv-mount-keyerror-rebase' refs #21541
Peter Amstutz [Tue, 2 Apr 2024 15:26:33 +0000 (11:26 -0400)]
Merge branch '21541-arv-mount-keyerror-rebase' refs #21541

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Make sure it is a mmap object before calling madvise
Peter Amstutz [Tue, 2 Apr 2024 15:23:20 +0000 (11:23 -0400)]
21639: Make sure it is a mmap object before calling madvise

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21508: Add wrappers to chips and move ellipsis styles to fix tooltip alignment
Stephen Smith [Tue, 2 Apr 2024 14:52:49 +0000 (10:52 -0400)]
21508: Add wrappers to chips and move ellipsis styles to fix tooltip alignment

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21541: Code cleanups 21541-arv-mount-keyerror-rebase
Peter Amstutz [Tue, 2 Apr 2024 14:52:09 +0000 (10:52 -0400)]
21541: Code cleanups

* Remove the gc threshold change
* Add/improve comments
* Use "truthy" tests instead of len(foo) > 0

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months agoMerge branch '21636-s3-token-expiry'
Tom Clegg [Tue, 2 Apr 2024 14:44:44 +0000 (10:44 -0400)]
Merge branch '21636-s3-token-expiry'

fixes #21636

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21636: Increase ExpiryWindow and add doc reference. 21636-s3-token-expiry
Tom Clegg [Tue, 2 Apr 2024 14:43:59 +0000 (10:43 -0400)]
21636: Increase ExpiryWindow and add doc reference.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21598-local-keepstore-emptytrash'
Tom Clegg [Tue, 2 Apr 2024 14:28:26 +0000 (10:28 -0400)]
Merge branch '21598-local-keepstore-emptytrash'

fixes #21598

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21639: Don't try to madvise zero length blocks
Peter Amstutz [Tue, 2 Apr 2024 14:05:41 +0000 (10:05 -0400)]
21639: Don't try to madvise zero length blocks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21508: Rename class for clarity
Stephen Smith [Tue, 2 Apr 2024 13:56:19 +0000 (09:56 -0400)]
21508: Rename class for clarity

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21639: Tell the kernel to map in whole blocks with madvise
Peter Amstutz [Tue, 2 Apr 2024 13:19:34 +0000 (09:19 -0400)]
21639: Tell the kernel to map in whole blocks with madvise

This increases cold cache performance from 150 MB/s to 400 MB/s

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21508: Remove remaining io panel image preview code
Stephen Smith [Tue, 2 Apr 2024 13:15:04 +0000 (09:15 -0400)]
21508: Remove remaining io panel image preview code

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Condense io panel css and remove redundant wrappers/rules
Stephen Smith [Tue, 2 Apr 2024 02:31:14 +0000 (22:31 -0400)]
21508: Condense io panel css and remove redundant wrappers/rules

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Fix io panel json content width
Stephen Smith [Tue, 2 Apr 2024 01:08:07 +0000 (21:08 -0400)]
21508: Fix io panel json content width

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Add helpful comments to styles
Stephen Smith [Tue, 2 Apr 2024 01:07:35 +0000 (21:07 -0400)]
21508: Add helpful comments to styles

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Add virtual list to io panel and styles to make sizing behave
Stephen Smith [Tue, 2 Apr 2024 01:06:33 +0000 (21:06 -0400)]
21508: Add virtual list to io panel and styles to make sizing behave

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21639: Note about 0 prefetch threads
Peter Amstutz [Tue, 2 Apr 2024 00:32:49 +0000 (20:32 -0400)]
21639: Note about 0 prefetch threads

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Keep a running total instead of recomputing total
Peter Amstutz [Tue, 2 Apr 2024 00:17:22 +0000 (20:17 -0400)]
21639: Keep a running total instead of recomputing total

From profiling, it turns out just summing the block sizes to get the
cache usage was a significant drag on the read() fast path.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21639: Improve critical path of read() from cache
Peter Amstutz [Mon, 1 Apr 2024 19:58:06 +0000 (15:58 -0400)]
21639: Improve critical path of read() from cache

* Don't use tobytes(), it makes a copy, and it should be be zero-copy.
* Prefetching adds a lot of overhead.  Don't do it.
* Don't use a list comprehension to calculate cache size

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21508: Split mounts and param table styles
Stephen Smith [Sun, 7 Apr 2024 19:48:00 +0000 (15:48 -0400)]
21508: Split mounts and param table styles

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Remove image preview from io panel
Stephen Smith [Mon, 1 Apr 2024 14:41:35 +0000 (10:41 -0400)]
21508: Remove image preview from io panel

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21508: Expand secondary param values into separate parameters
Stephen Smith [Fri, 29 Mar 2024 23:34:05 +0000 (19:34 -0400)]
21508: Expand secondary param values into separate parameters

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21613: Fix handling of expired token re-validating with new UUID. 21613-fed-token-exp
Tom Clegg [Mon, 1 Apr 2024 21:51:43 +0000 (17:51 -0400)]
21613: Fix handling of expired token re-validating with new UUID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21448: fixed user-profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Mon, 1 Apr 2024 17:36:16 +0000 (13:36 -0400)]
21448: fixed user-profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: removed move to from processes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Mon, 1 Apr 2024 16:59:47 +0000 (12:59 -0400)]
21448: removed move to from processes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: fixed user profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Mon, 1 Apr 2024 15:50:54 +0000 (11:50 -0400)]
21448: fixed user profile test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21639: Don't return broken cache slots
Peter Amstutz [Mon, 1 Apr 2024 15:10:52 +0000 (11:10 -0400)]
21639: Don't return broken cache slots

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21448: changed int tests to new menu orders Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Mon, 1 Apr 2024 15:07:01 +0000 (11:07 -0400)]
21448: changed int tests to new menu orders Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: fixed toolbar divider so it renders consistently Arvados-DCO-1.1-Signed-off...
Lisa Knox [Mon, 1 Apr 2024 13:55:50 +0000 (09:55 -0400)]
21448: fixed toolbar divider so it renders consistently Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21639: Use a more efficient data structure for the keep block cache
Peter Amstutz [Mon, 1 Apr 2024 13:52:53 +0000 (09:52 -0400)]
21639: Use a more efficient data structure for the keep block cache

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21636: Test http status returned for unknown backend errors.
Tom Clegg [Mon, 1 Apr 2024 13:20:02 +0000 (09:20 -0400)]
21636: Test http status returned for unknown backend errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21636: Set ExpiryWindow to avoid credential expiry races.
Tom Clegg [Fri, 29 Mar 2024 20:58:23 +0000 (16:58 -0400)]
21636: Set ExpiryWindow to avoid credential expiry races.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21617-fed-content'
Tom Clegg [Fri, 29 Mar 2024 20:39:09 +0000 (16:39 -0400)]
Merge branch '21617-fed-content'

fixes #21617
fixes #21533

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21617: Split test cases.
Tom Clegg [Fri, 29 Mar 2024 20:37:43 +0000 (16:37 -0400)]
21617: Split test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21448: divider in place for toolbar menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Fri, 29 Mar 2024 18:44:24 +0000 (14:44 -0400)]
21448: divider in place for toolbar menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: divider in place for context menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Fri, 29 Mar 2024 17:54:48 +0000 (13:54 -0400)]
21448: divider in place for context menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: expanded copy to oclipboard change Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Fri, 29 Mar 2024 15:20:41 +0000 (11:20 -0400)]
21448: expanded copy to oclipboard change Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: channged wording Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Fri, 29 Mar 2024 15:15:03 +0000 (11:15 -0400)]
21448: channged wording Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: ms toolbar sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Fri, 29 Mar 2024 15:08:11 +0000 (11:08 -0400)]
21448: ms toolbar sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: removed share from process menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox ...
Lisa Knox [Fri, 29 Mar 2024 14:26:14 +0000 (10:26 -0400)]
21448: removed share from process menu Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: running process also sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Fri, 29 Mar 2024 14:07:02 +0000 (10:07 -0400)]
21448: running process also sorted Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: applied sort to all context menus in ticket Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Fri, 29 Mar 2024 13:48:10 +0000 (09:48 -0400)]
21448: applied sort to all context menus in ticket Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21583: Override base64 dependency so it doesn't appear in Gemfile.lock
Brett Smith [Thu, 14 Mar 2024 15:17:24 +0000 (11:17 -0400)]
21583: Override base64 dependency so it doesn't appear in Gemfile.lock

See comments and linked bug for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21448: sorter sorts Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Thu, 28 Mar 2024 18:51:28 +0000 (14:51 -0400)]
21448: sorter sorts Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: initial sorting algo in place Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Thu, 28 Mar 2024 17:37:54 +0000 (13:37 -0400)]
21448: initial sorting algo in place Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: adjusted specs to fit capitalization standardization Arvados-DCO-1.1-Signed...
Lisa Knox [Thu, 28 Mar 2024 15:01:26 +0000 (11:01 -0400)]
21448: adjusted specs to fit capitalization standardization Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21598: Don't run trash workers in container-local keepstore. 21598-local-keepstore-emptytrash
Tom Clegg [Thu, 28 Mar 2024 14:57:44 +0000 (10:57 -0400)]
21598: Don't run trash workers in container-local keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21448: combined contextmenunames and mstoolbaractionnames since they were mostly...
Lisa Knox [Thu, 28 Mar 2024 14:36:21 +0000 (10:36 -0400)]
21448: combined contextmenunames and mstoolbaractionnames since they were mostly duplicated and serve the same purpose Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: replaced all context menu names with string constants Arvados-DCO-1.1-Signed...
Lisa Knox [Thu, 28 Mar 2024 13:21:21 +0000 (09:21 -0400)]
21448: replaced all context menu names with string constants Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21541: Fix typo
Peter Amstutz [Thu, 28 Mar 2024 12:17:59 +0000 (08:17 -0400)]
21541: Fix typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21541: Revert change to safeapi.py
Peter Amstutz [Wed, 27 Mar 2024 21:14:28 +0000 (17:14 -0400)]
21541: Revert change to safeapi.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21541: Code cleanup and additional memory usage improvements
Peter Amstutz [Mon, 25 Mar 2024 18:19:32 +0000 (14:19 -0400)]
21541: Code cleanup and additional memory usage improvements

* Add slots to major Directory classes

* Disconnect FuseArvadosFile from ArvadosFile to reduce cyclic
references.

* Clean up _remove_inode loop and use dataclasses for the inode
operations.

* Now calls del_entry on collection_record_file and project_object_file.

It looks like collection_record_file was holding a reference to the
Collection object (and was remaining in the inodes table) even when
CollectionDirectory was cleared.  I believe this is the memory leak I
have been looking for.

* Remove the "dead" flag and set parent_inode to None instead.  This
clarifies the behavior that directory entries keep their (numeric)
inodes until they are detached from the directory which may have
contributed to infrequent "file not found" errors.

* Adjust cache behavior to only hold objects that are cache-eligible
and have non-zero cache_size.  This avoids filling the cache with
entries that are just going to be skipped over.

Overall: Memory usage is mostly stable but does tend to creep up over
time.  My best guess is that this is forced because we need to keep
inodes in RAM as long as the kernel maintains a reference to them, so
with multiple processes accessing different filesystem locations, this
is simply RAM required for the working set.

I'm also cautiously optimistic that issues I observed with performance
slowing down with long-lived processes are improved (e.g. fixing
memory leaks means no more unbounded growth of cache_entries, which
means no more time wasted iterating over huge lists).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21448: expanded string constants Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Wed, 27 Mar 2024 15:47:59 +0000 (11:47 -0400)]
21448: expanded string constants Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21448: created string constants for menu items Arvados-DCO-1.1-Signed-off-by: Lisa...
Lisa Knox [Wed, 27 Mar 2024 14:01:51 +0000 (10:01 -0400)]
21448: created string constants for menu items Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21617: Test unauthenticated endpoint + LoginCluster + bad token. 21617-fed-content
Tom Clegg [Tue, 26 Mar 2024 14:01:28 +0000 (10:01 -0400)]
21617: Test unauthenticated endpoint + LoginCluster + bad token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21617: Interpret any 4xx as token rejection, not an error.
Tom Clegg [Tue, 26 Mar 2024 02:30:10 +0000 (22:30 -0400)]
21617: Interpret any 4xx as token rejection, not an error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21617: Accept bogus tokens at endpoints that do not require auth.
Tom Clegg [Mon, 25 Mar 2024 21:25:10 +0000 (17:25 -0400)]
21617: Accept bogus tokens at endpoints that do not require auth.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21508: Fix cross origin io parameter image previews
Stephen Smith [Mon, 25 Mar 2024 15:47:19 +0000 (11:47 -0400)]
21508: Fix cross origin io parameter image previews

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

8 months ago21508: Normalize io panel row height
Stephen Smith [Mon, 25 Mar 2024 15:46:52 +0000 (11:46 -0400)]
21508: Normalize io panel row height

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

8 months ago21601: Ensure pip is new enough to support package building
Brett Smith [Sun, 24 Mar 2024 03:26:58 +0000 (23:26 -0400)]
21601: Ensure pip is new enough to support package building

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Build Python distro packages from wheels
Brett Smith [Sat, 23 Mar 2024 18:16:35 +0000 (14:16 -0400)]
21601: Build Python distro packages from wheels

The immediate problem this solves is that, by building and installing
from a repository of wheels, Python packages can find their
interdependencies without any special logic in the build process.

Other benefits:

* Eliminates some redundant work. We don't have to build the Python SDK
  from source multiple times. We can use the published cwltest wheel
  instead of building our own.

* Prepares the code for PEP 517 compliance. We only invoke setup.py to
  build packages that have not been updated yet. We introspect packages
  from their wheels, so we no longer have to introspect the source to
  build distro packages.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Remove PySDK from `install deps`
Brett Smith [Sat, 23 Mar 2024 18:17:47 +0000 (14:17 -0400)]
21601: Remove PySDK from `install deps`

It's already installed by `install env`. ðŸ¤·

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Install wheel in run-tests.sh VENV3DIR
Brett Smith [Tue, 19 Mar 2024 03:04:59 +0000 (23:04 -0400)]
21601: Install wheel in run-tests.sh VENV3DIR

We do this basically everywhere else in our build infrastructure, so
doing it here helps avoid discrepancies between those environments.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Specify Python interdependencies with ~=
Brett Smith [Sat, 16 Mar 2024 23:23:44 +0000 (19:23 -0400)]
21601: Specify Python interdependencies with ~=

This has the same rationale as using <= before, but it's stricter. It
should prevent pip from using release versions to satisfy development
dependencies in the future, and help root out bugs in our build
processes.

DRY up this logic in arvados_version.py.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Move --short-tests-only check to arvados_version
Brett Smith [Sat, 16 Mar 2024 23:10:15 +0000 (19:10 -0400)]
21601: Move --short-tests-only check to arvados_version

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months ago21601: Make arvados_version.py more declarative
Brett Smith [Sat, 16 Mar 2024 23:02:59 +0000 (19:02 -0400)]
21601: Make arvados_version.py more declarative

The main goal of this change is to introduce the metadata maps near the
top of the file, which we will use to build additional
functionality. The rest of the changes are just modernization or
clean-up based on that.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

8 months agoFix open-ended bundler version in ruby install recipe.
Tom Clegg [Thu, 21 Mar 2024 19:11:41 +0000 (15:11 -0400)]
Fix open-ended bundler version in ruby install recipe.

refs #21463

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '21600-banner-tests'
Lisa Knox [Thu, 21 Mar 2024 17:17:45 +0000 (13:17 -0400)]
Merge branch '21600-banner-tests'

closes #21600

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21578-mount-debug'
Tom Clegg [Thu, 21 Mar 2024 15:39:15 +0000 (11:39 -0400)]
Merge branch '21578-mount-debug'

closes #21578

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '21603-ec2-subnet-error'
Tom Clegg [Thu, 21 Mar 2024 15:37:16 +0000 (11:37 -0400)]
Merge branch '21603-ec2-subnet-error'

fixes #21603

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoUpdate test failing on debian 12 if s3cmd is installed.
Tom Clegg [Thu, 21 Mar 2024 15:21:22 +0000 (11:21 -0400)]
Update test failing on debian 12 if s3cmd is installed.

Also, update `arvados-server install -type test` to install s3cmd if
available.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '21357-favorites-names'
Lisa Knox [Thu, 21 Mar 2024 13:10:23 +0000 (09:10 -0400)]
Merge branch '21357-favorites-names'

closes #21357

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>