arvados.git
6 months ago21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 14 May 2024 17:12:01 +0000 (13:12 -0400)]
21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: replaced deleteWorkflow with removeWorkflowPermanently for consistency Arvados...
Lisa Knox [Tue, 14 May 2024 15:58:45 +0000 (11:58 -0400)]
21535: replaced deleteWorkflow with removeWorkflowPermanently for consistency Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: added multi wf delete test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 14 May 2024 15:07:02 +0000 (11:07 -0400)]
21535: added multi wf delete test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: added wf delete dialog Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Mon, 13 May 2024 19:25:12 +0000 (15:25 -0400)]
21535: added wf delete dialog Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: changed wf and process to use new delete icon Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Mon, 13 May 2024 17:56:49 +0000 (13:56 -0400)]
21535: changed wf and process to use new delete icon Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: made ownerUuid param optional and removed it for multiselect Arvados-DCO-1...
Lisa Knox [Mon, 13 May 2024 16:27:32 +0000 (12:27 -0400)]
21535: made ownerUuid param optional and removed it for multiselect Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21535: changed delete method to handle multiple workflows Arvados-DCO-1.1-Signed...
Lisa Knox [Mon, 6 May 2024 18:20:59 +0000 (14:20 -0400)]
21535: changed delete method to handle multiple workflows Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: added missing filter for multiole workflows Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Thu, 4 Apr 2024 17:31:27 +0000 (13:31 -0400)]
21535: added missing filter for multiole workflows Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21601-setuptools-git-deps'
Brett Smith [Tue, 2 Apr 2024 21:18:58 +0000 (17:18 -0400)]
Merge branch '21601-setuptools-git-deps'

Refs #21601.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Add test for arvados_version.py synchronization 21601-setuptools-git-deps
Brett Smith [Tue, 2 Apr 2024 20:50:41 +0000 (16:50 -0400)]
21601: Add test for arvados_version.py synchronization

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21613-fed-token-exp'
Tom Clegg [Tue, 2 Apr 2024 17:01:44 +0000 (13:01 -0400)]
Merge branch '21613-fed-token-exp'

fixes #21613

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21541-arv-mount-keyerror-rebase' refs #21541
Peter Amstutz [Tue, 2 Apr 2024 15:26:33 +0000 (11:26 -0400)]
Merge branch '21541-arv-mount-keyerror-rebase' refs #21541

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21541: Code cleanups 21541-arv-mount-keyerror-rebase
Peter Amstutz [Tue, 2 Apr 2024 14:52:09 +0000 (10:52 -0400)]
21541: Code cleanups

* Remove the gc threshold change
* Add/improve comments
* Use "truthy" tests instead of len(foo) > 0

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months agoMerge branch '21636-s3-token-expiry'
Tom Clegg [Tue, 2 Apr 2024 14:44:44 +0000 (10:44 -0400)]
Merge branch '21636-s3-token-expiry'

fixes #21636

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21636: Increase ExpiryWindow and add doc reference. 21636-s3-token-expiry
Tom Clegg [Tue, 2 Apr 2024 14:43:59 +0000 (10:43 -0400)]
21636: Increase ExpiryWindow and add doc reference.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21598-local-keepstore-emptytrash'
Tom Clegg [Tue, 2 Apr 2024 14:28:26 +0000 (10:28 -0400)]
Merge branch '21598-local-keepstore-emptytrash'

fixes #21598

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21613: Fix handling of expired token re-validating with new UUID. 21613-fed-token-exp
Tom Clegg [Mon, 1 Apr 2024 21:51:43 +0000 (17:51 -0400)]
21613: Fix handling of expired token re-validating with new UUID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21636: Test http status returned for unknown backend errors.
Tom Clegg [Mon, 1 Apr 2024 13:20:02 +0000 (09:20 -0400)]
21636: Test http status returned for unknown backend errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21636: Set ExpiryWindow to avoid credential expiry races.
Tom Clegg [Fri, 29 Mar 2024 20:58:23 +0000 (16:58 -0400)]
21636: Set ExpiryWindow to avoid credential expiry races.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21617-fed-content'
Tom Clegg [Fri, 29 Mar 2024 20:39:09 +0000 (16:39 -0400)]
Merge branch '21617-fed-content'

fixes #21617
fixes #21533

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21617: Split test cases.
Tom Clegg [Fri, 29 Mar 2024 20:37:43 +0000 (16:37 -0400)]
21617: Split test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21598: Don't run trash workers in container-local keepstore. 21598-local-keepstore-emptytrash
Tom Clegg [Thu, 28 Mar 2024 14:57:44 +0000 (10:57 -0400)]
21598: Don't run trash workers in container-local keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21541: Fix typo
Peter Amstutz [Thu, 28 Mar 2024 12:17:59 +0000 (08:17 -0400)]
21541: Fix typo

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21541: Revert change to safeapi.py
Peter Amstutz [Wed, 27 Mar 2024 21:14:28 +0000 (17:14 -0400)]
21541: Revert change to safeapi.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21541: Code cleanup and additional memory usage improvements
Peter Amstutz [Mon, 25 Mar 2024 18:19:32 +0000 (14:19 -0400)]
21541: Code cleanup and additional memory usage improvements

* Add slots to major Directory classes

* Disconnect FuseArvadosFile from ArvadosFile to reduce cyclic
references.

* Clean up _remove_inode loop and use dataclasses for the inode
operations.

* Now calls del_entry on collection_record_file and project_object_file.

It looks like collection_record_file was holding a reference to the
Collection object (and was remaining in the inodes table) even when
CollectionDirectory was cleared.  I believe this is the memory leak I
have been looking for.

* Remove the "dead" flag and set parent_inode to None instead.  This
clarifies the behavior that directory entries keep their (numeric)
inodes until they are detached from the directory which may have
contributed to infrequent "file not found" errors.

* Adjust cache behavior to only hold objects that are cache-eligible
and have non-zero cache_size.  This avoids filling the cache with
entries that are just going to be skipped over.

Overall: Memory usage is mostly stable but does tend to creep up over
time.  My best guess is that this is forced because we need to keep
inodes in RAM as long as the kernel maintains a reference to them, so
with multiple processes accessing different filesystem locations, this
is simply RAM required for the working set.

I'm also cautiously optimistic that issues I observed with performance
slowing down with long-lived processes are improved (e.g. fixing
memory leaks means no more unbounded growth of cache_entries, which
means no more time wasted iterating over huge lists).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21617: Test unauthenticated endpoint + LoginCluster + bad token. 21617-fed-content
Tom Clegg [Tue, 26 Mar 2024 14:01:28 +0000 (10:01 -0400)]
21617: Test unauthenticated endpoint + LoginCluster + bad token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21617: Interpret any 4xx as token rejection, not an error.
Tom Clegg [Tue, 26 Mar 2024 02:30:10 +0000 (22:30 -0400)]
21617: Interpret any 4xx as token rejection, not an error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21617: Accept bogus tokens at endpoints that do not require auth.
Tom Clegg [Mon, 25 Mar 2024 21:25:10 +0000 (17:25 -0400)]
21617: Accept bogus tokens at endpoints that do not require auth.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21601: Ensure pip is new enough to support package building
Brett Smith [Sun, 24 Mar 2024 03:26:58 +0000 (23:26 -0400)]
21601: Ensure pip is new enough to support package building

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Build Python distro packages from wheels
Brett Smith [Sat, 23 Mar 2024 18:16:35 +0000 (14:16 -0400)]
21601: Build Python distro packages from wheels

The immediate problem this solves is that, by building and installing
from a repository of wheels, Python packages can find their
interdependencies without any special logic in the build process.

Other benefits:

* Eliminates some redundant work. We don't have to build the Python SDK
  from source multiple times. We can use the published cwltest wheel
  instead of building our own.

* Prepares the code for PEP 517 compliance. We only invoke setup.py to
  build packages that have not been updated yet. We introspect packages
  from their wheels, so we no longer have to introspect the source to
  build distro packages.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Remove PySDK from `install deps`
Brett Smith [Sat, 23 Mar 2024 18:17:47 +0000 (14:17 -0400)]
21601: Remove PySDK from `install deps`

It's already installed by `install env`. ðŸ¤·

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Install wheel in run-tests.sh VENV3DIR
Brett Smith [Tue, 19 Mar 2024 03:04:59 +0000 (23:04 -0400)]
21601: Install wheel in run-tests.sh VENV3DIR

We do this basically everywhere else in our build infrastructure, so
doing it here helps avoid discrepancies between those environments.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Specify Python interdependencies with ~=
Brett Smith [Sat, 16 Mar 2024 23:23:44 +0000 (19:23 -0400)]
21601: Specify Python interdependencies with ~=

This has the same rationale as using <= before, but it's stricter. It
should prevent pip from using release versions to satisfy development
dependencies in the future, and help root out bugs in our build
processes.

DRY up this logic in arvados_version.py.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Move --short-tests-only check to arvados_version
Brett Smith [Sat, 16 Mar 2024 23:10:15 +0000 (19:10 -0400)]
21601: Move --short-tests-only check to arvados_version

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21601: Make arvados_version.py more declarative
Brett Smith [Sat, 16 Mar 2024 23:02:59 +0000 (19:02 -0400)]
21601: Make arvados_version.py more declarative

The main goal of this change is to introduce the metadata maps near the
top of the file, which we will use to build additional
functionality. The rest of the changes are just modernization or
clean-up based on that.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoFix open-ended bundler version in ruby install recipe.
Tom Clegg [Thu, 21 Mar 2024 19:11:41 +0000 (15:11 -0400)]
Fix open-ended bundler version in ruby install recipe.

refs #21463

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21600-banner-tests'
Lisa Knox [Thu, 21 Mar 2024 17:17:45 +0000 (13:17 -0400)]
Merge branch '21600-banner-tests'

closes #21600

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21578-mount-debug'
Tom Clegg [Thu, 21 Mar 2024 15:39:15 +0000 (11:39 -0400)]
Merge branch '21578-mount-debug'

closes #21578

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21603-ec2-subnet-error'
Tom Clegg [Thu, 21 Mar 2024 15:37:16 +0000 (11:37 -0400)]
Merge branch '21603-ec2-subnet-error'

fixes #21603

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoUpdate test failing on debian 12 if s3cmd is installed.
Tom Clegg [Thu, 21 Mar 2024 15:21:22 +0000 (11:21 -0400)]
Update test failing on debian 12 if s3cmd is installed.

Also, update `arvados-server install -type test` to install s3cmd if
available.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21357-favorites-names'
Lisa Knox [Thu, 21 Mar 2024 13:10:23 +0000 (09:10 -0400)]
Merge branch '21357-favorites-names'

closes #21357

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21357: changed dispatch in loadFavoritesTRee to a string instead of an object Arvados... 21357-favorites-names
Lisa Knox [Wed, 20 Mar 2024 13:53:28 +0000 (09:53 -0400)]
21357: changed dispatch in loadFavoritesTRee to a string instead of an object Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch 'main' into 21357-favorites-names
Lisa Knox [Tue, 19 Mar 2024 18:41:12 +0000 (14:41 -0400)]
Merge branch 'main' into 21357-favorites-names

refs #21357

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21357: added loadsidepaneltree to updatecollection Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Tue, 19 Mar 2024 17:57:04 +0000 (13:57 -0400)]
21357: added loadsidepaneltree to updatecollection Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21357: moved resource fetches into a single await Arvados-DCO-1.1-Signed-off-by:...
Lisa Knox [Tue, 19 Mar 2024 17:26:20 +0000 (13:26 -0400)]
21357: moved resource fetches into a single await Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: combined changes by Stephen and Lucas Arvados-DCO-1.1-Signed-off-by: Lisa... 21600-banner-tests
Lisa Knox [Tue, 19 Mar 2024 14:52:22 +0000 (10:52 -0400)]
21600: combined changes by Stephen and Lucas Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21412-user-profile-bugs'
Lisa Knox [Tue, 19 Mar 2024 14:16:09 +0000 (10:16 -0400)]
Merge branch '21412-user-profile-bugs'

closes #21412

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: incorporated Stephens changes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 19 Mar 2024 13:21:01 +0000 (09:21 -0400)]
21600: incorporated Stephens changes Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: added waits everywhere it could possibly matter Arvados-DCO-1.1-Signed-off...
Lisa Knox [Mon, 18 Mar 2024 19:59:49 +0000 (15:59 -0400)]
21600: added waits everywhere it could possibly matter Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21603: Recognize subnet error despite generic error code. 21603-ec2-subnet-error
Tom Clegg [Mon, 18 Mar 2024 19:01:16 +0000 (15:01 -0400)]
21603: Recognize subnet error despite generic error code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21600: tweaked test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 18 Mar 2024 18:42:54 +0000 (14:42 -0400)]
21600: tweaked test  Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch 'main' into 21541-arv-mount-keyerror-rebase refs #21541
Peter Amstutz [Mon, 18 Mar 2024 18:35:21 +0000 (14:35 -0400)]
Merge branch 'main' into 21541-arv-mount-keyerror-rebase refs #21541

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoMerge branch '20455-noopener' refs #20455
Peter Amstutz [Mon, 18 Mar 2024 18:22:05 +0000 (14:22 -0400)]
Merge branch '20455-noopener' refs #20455

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago21600: removed tests to test spec itself Arvados-DCO-1.1-Signed-off-by: Lisa Knox...
Lisa Knox [Mon, 18 Mar 2024 18:13:45 +0000 (14:13 -0400)]
21600: removed tests to test spec itself Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21597-cypress-reply-fix' into main. Closes #21597
Stephen Smith [Mon, 18 Mar 2024 17:39:51 +0000 (13:39 -0400)]
Merge branch '21597-cypress-reply-fix' into main. Closes #21597

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

8 months ago2960: Remove unused code.
Tom Clegg [Mon, 18 Mar 2024 15:39:11 +0000 (11:39 -0400)]
2960: Remove unused code.

refs #2960

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21600: changed tooltip expectation Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Mon, 18 Mar 2024 15:02:11 +0000 (11:02 -0400)]
21600: changed tooltip expectation Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21449-lsf-maxruntime'
Tom Clegg [Mon, 18 Mar 2024 14:48:05 +0000 (10:48 -0400)]
Merge branch '21449-lsf-maxruntime'

closes #21449

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21578: Add -debug alias for -log-level=debug. 21578-mount-debug
Tom Clegg [Mon, 18 Mar 2024 14:44:58 +0000 (10:44 -0400)]
21578: Add -debug alias for -log-level=debug.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21578: Merge branch 'main' into 21578-mount-debug
Tom Clegg [Mon, 18 Mar 2024 14:31:16 +0000 (10:31 -0400)]
21578: Merge branch 'main' into 21578-mount-debug

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21600: used the force Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Mon, 18 Mar 2024 14:20:55 +0000 (10:20 -0400)]
21600: used the force Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: set cypress data tag Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Mon, 18 Mar 2024 13:55:44 +0000 (09:55 -0400)]
21600: set cypress data tag Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: testing wait removal Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Mon, 18 Mar 2024 13:16:18 +0000 (09:16 -0400)]
21600: testing wait removal Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: removed only and changed wait to 1 second Arvados-DCO-1.1-Signed-off-by: Lisa...
Lisa Knox [Fri, 15 Mar 2024 20:33:39 +0000 (16:33 -0400)]
21600: removed only and changed wait to 1 second Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: added 3 second wait Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Fri, 15 Mar 2024 20:06:45 +0000 (16:06 -0400)]
21600: added 3 second wait Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: added only for testing Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Fri, 15 Mar 2024 19:37:54 +0000 (15:37 -0400)]
21600: added only for testing Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Fri, 15 Mar 2024 19:27:54 +0000 (15:27 -0400)]
21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Fri, 15 Mar 2024 19:07:35 +0000 (15:07 -0400)]
21600: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21165-installer-uninstalls-wb1'. Closes #21165
Lucas Di Pentima [Fri, 15 Mar 2024 18:13:11 +0000 (15:13 -0300)]
Merge branch '21165-installer-uninstalls-wb1'. Closes #21165

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago21597: Use req.on('reply' instead of req.reply to workaround cypress bug 21597-cypress-reply-fix
Stephen Smith [Fri, 15 Mar 2024 17:15:08 +0000 (13:15 -0400)]
21597: Use req.on('reply' instead of req.reply to workaround cypress bug

https://github.com/cypress-io/cypress/issues/26248#issuecomment-1623005400

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

8 months ago21541: Adjust dependency to arvados-llfuse fork
Peter Amstutz [Fri, 15 Mar 2024 13:56:09 +0000 (09:56 -0400)]
21541: Adjust dependency to arvados-llfuse fork

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months ago21541: Fix KeyError, segfaults, and memory use issues
Peter Amstutz [Wed, 6 Mar 2024 20:03:38 +0000 (15:03 -0500)]
21541: Fix KeyError, segfaults, and memory use issues

* Fixes a segfault on startup due to multiple threads fetching the
cluster config using the same http2 object, which is not threadafe.
Now fetches the relevant configuration
item once (ForwardSlashNameSubstitution), and stores it where all the
threads can access it.  (bug #21568)

* Fixes KeyError thrown where a parent inode is removed from the
inodes table before its children.

* In the process of testing, re-discovered a bug where, if the llfuse
_notify_queue fills up, the entire mount process deadlocks.

The previous fix worked by monkey-patching llfuse to replace a
limited-length queue with an unlimited length queue, however changes
in subsequent llfuse versions caused that variable to be hidden from
Python (so the monkey patch didn't fail but it no longer had any
effect either).  The solution is to introduce an additional
unlimited-size queue in between the operation handlers and the
limited-size kernel notification queue.

* Because cache management and inode cleanup interact with kernel
notifications (which were moved into a separate thread), I decided
they should also be asynchronous from the operation handlers, so they
are now part of the same thread that processes kernel notifications.

* Attempting to remove an inode that is in use will now at minimum
send a kernel invalidation, which will sometimes nudge the kernel to
forget the inode, enabling us to remove it.

* Filter groups now check if the filter group contains itself so it
doesn't create an infinite path loop that breaks filesystem traversal
tools.

* In the process of testing, found that llfuse didn't wait for the
_notify_queue to drain before closing the FUSE channel, resulting in a
segfault if the _notify_loop thread tried to process any events after
shutdown started.  This bug cannot be worked around on the Arvados
side, so I have prepared an arvados-llfuse fork with a bug fix.

* Testing with arv-mount-stress-test (which creates 8 subprocesses that all
traverse the filesystem at the same time) now passes with no
filesystem errors, no deadlocks, no segfaults, and modest memory
usage.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agosdk services installs crunchstat-summary refs #19744
Peter Amstutz [Fri, 15 Mar 2024 14:45:03 +0000 (10:45 -0400)]
sdk services installs crunchstat-summary refs #19744

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

8 months agoReorder python install steps to address dependency issue.
Tom Clegg [Fri, 15 Mar 2024 16:01:22 +0000 (12:01 -0400)]
Reorder python install steps to address dependency issue.

refs #19744

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago19744: Reorder python install steps to address dependency issue. 21449-lsf-maxruntime
Tom Clegg [Fri, 15 Mar 2024 13:56:03 +0000 (09:56 -0400)]
19744: Reorder python install steps to address dependency issue.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '21507-long-project-title-wrap'
Lisa Knox [Fri, 15 Mar 2024 13:05:56 +0000 (09:05 -0400)]
Merge branch '21507-long-project-title-wrap'

closes #21507

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21165: Uninstalls WB1 from single host installations too. 21165-installer-uninstalls-wb1
Lucas Di Pentima [Thu, 14 Mar 2024 20:54:02 +0000 (17:54 -0300)]
21165: Uninstalls WB1 from single host installations too.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago21165: Adds extra state to uninstall wb1's package from workbench node.
Lucas Di Pentima [Thu, 14 Mar 2024 19:33:33 +0000 (16:33 -0300)]
21165: Adds extra state to uninstall wb1's package from workbench node.

I opted for specifically uninstalling the package instead of adding the
arvados.workbench.package.clean state, because that state also removes
packages that might be needed for other services.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago21449: Fix ordering in "install deps".
Tom Clegg [Thu, 14 Mar 2024 19:25:46 +0000 (15:25 -0400)]
21449: Fix ordering in "install deps".

"install sdk/cli" is meant to use the current version of the arvados
gem from this checkout, but that can only happen if "install sdk/ruby"
runs first.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21507: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii... 21507-long-project-title-wrap
Lisa Knox [Thu, 14 Mar 2024 17:55:27 +0000 (13:55 -0400)]
21507: tweaked banner test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21449: Propagate max_run_time to LSF scheduler.
Tom Clegg [Wed, 13 Mar 2024 21:22:39 +0000 (17:22 -0400)]
21449: Propagate max_run_time to LSF scheduler.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months agoMerge branch '21123-scheduling-status'
Tom Clegg [Thu, 14 Mar 2024 14:59:37 +0000 (10:59 -0400)]
Merge branch '21123-scheduling-status'

closes #21123

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21123: Add comment re arv-mount dependency.
Tom Clegg [Thu, 14 Mar 2024 14:58:47 +0000 (10:58 -0400)]
21123: Add comment re arv-mount dependency.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21507: cleanup Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>
Lisa Knox [Thu, 14 Mar 2024 14:36:48 +0000 (10:36 -0400)]
21507: cleanup Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21507: expanded nowrap to all of the places it should apply Arvados-DCO-1.1-Signed...
Lisa Knox [Thu, 14 Mar 2024 14:35:30 +0000 (10:35 -0400)]
21507: expanded nowrap to all of the places it should apply Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21507: removed word wrap on collection owner field Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Thu, 14 Mar 2024 14:22:22 +0000 (10:22 -0400)]
21507: removed word wrap on collection owner field Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '21585-installer-imdsv2'. Closes #21585
Lucas Di Pentima [Wed, 13 Mar 2024 20:13:42 +0000 (17:13 -0300)]
Merge branch '21585-installer-imdsv2'. Closes #21585

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago21123: Add arv-mount to "install deps". 21123-scheduling-status
Tom Clegg [Wed, 13 Mar 2024 18:20:47 +0000 (14:20 -0400)]
21123: Add arv-mount to "install deps".

lib/controller integration tests need it to run containers.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21585: Updates installer's Terraform code to require IMDSv2 on service nodes. 21585-installer-imdsv2
Lucas Di Pentima [Wed, 13 Mar 2024 17:45:30 +0000 (14:45 -0300)]
21585: Updates installer's Terraform code to require IMDSv2 on service nodes.

Applying this change to an already working cluster won't recreate any resource,
just change the settings in place.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

8 months ago21123: the empty string -> an empty string
Tom Clegg [Wed, 13 Mar 2024 13:36:35 +0000 (09:36 -0400)]
21123: the empty string -> an empty string

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21357: removed select field Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Wed, 13 Mar 2024 13:20:58 +0000 (09:20 -0400)]
21357: removed select field Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21357: refined resource fetches Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Wed, 13 Mar 2024 13:14:12 +0000 (09:14 -0400)]
21357: refined resource fetches Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21123: Add container_status to discovery doc.
Tom Clegg [Wed, 13 Mar 2024 03:27:36 +0000 (23:27 -0400)]
21123: Add container_status to discovery doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21123: Define scheduling status values as consts.
Tom Clegg [Tue, 12 Mar 2024 21:49:32 +0000 (17:49 -0400)]
21123: Define scheduling status values as consts.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21123: Rename sQueue* to schedQueue*.
Tom Clegg [Tue, 12 Mar 2024 21:49:12 +0000 (17:49 -0400)]
21123: Rename sQueue* to schedQueue*.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21123: Clarify empty means empty string.
Tom Clegg [Tue, 12 Mar 2024 21:48:18 +0000 (17:48 -0400)]
21123: Clarify empty means empty string.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

8 months ago21357: set faves to bulk fetch Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Tue, 12 Mar 2024 18:49:48 +0000 (14:49 -0400)]
21357: set faves to bulk fetch Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21357: publicfavorites now optimistically renders Arvados-DCO-1.1-Signed-off-by:...
Lisa Knox [Tue, 12 Mar 2024 16:07:09 +0000 (12:07 -0400)]
21357: publicfavorites now optimistically renders Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months ago21357: myfavorites now optimistically renders Arvados-DCO-1.1-Signed-off-by: Lisa...
Lisa Knox [Tue, 12 Mar 2024 15:32:36 +0000 (11:32 -0400)]
21357: myfavorites now optimistically renders Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

8 months agoMerge branch '15317-metrics'
Tom Clegg [Tue, 12 Mar 2024 15:12:37 +0000 (11:12 -0400)]
Merge branch '15317-metrics'

closes #15317

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>