arvados.git
6 months agoMerge branch '21705-go121'
Tom Clegg [Tue, 28 May 2024 18:48:08 +0000 (14:48 -0400)]
Merge branch '21705-go121'

refs #21705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21840-jenkins-builds-wheels'
Brett Smith [Tue, 28 May 2024 18:13:28 +0000 (14:13 -0400)]
Merge branch '21840-jenkins-builds-wheels'

Refs #21840.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21840: Fix glob to find wheels
Brett Smith [Tue, 28 May 2024 18:08:45 +0000 (14:08 -0400)]
21840: Fix glob to find wheels

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21840: Add wheels to the Jenkins Python package build 21840-jenkins-builds-wheels
Brett Smith [Tue, 28 May 2024 16:17:59 +0000 (12:17 -0400)]
21840: Add wheels to the Jenkins Python package build

We were previously doing this in run_upload_packages. Now that that tool
is doing pure uploads with twine, we need to build the wheels in Jenkins
instead.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21705: Downgrade to Go 1.21 to avoid glibc compatibility issue. 21705-go121
Tom Clegg [Tue, 28 May 2024 16:10:39 +0000 (12:10 -0400)]
21705: Downgrade to Go 1.21 to avoid glibc compatibility issue.

See
* https://github.com/sylabs/singularity/releases/tag/v4.1.3
* https://github.com/sylabs/singularity/issues/2677
* https://github.com/golang/go/issues/65625
* https://dev.arvados.org/issues/21705#note-13

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21705-go-deps'
Tom Clegg [Tue, 28 May 2024 16:01:36 +0000 (12:01 -0400)]
Merge branch '21705-go-deps'

refs #21705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21705: Spell out entire config key for IAMRole.
Tom Clegg [Tue, 28 May 2024 15:59:19 +0000 (11:59 -0400)]
21705: Spell out entire config key for IAMRole.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoFix test for debian 12.
Tom Clegg [Mon, 27 May 2024 13:44:49 +0000 (09:44 -0400)]
Fix test for debian 12.

A debian 11 container cannot run a binary built on a debian 12 host.
But a debian 12 container can run binaries built on either release.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21705: Remove IAMRole config and update docs. 21705-go-deps
Tom Clegg [Thu, 23 May 2024 20:09:41 +0000 (16:09 -0400)]
21705: Remove IAMRole config and update docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21705: Use context.Background() for aws sdk calls.
Tom Clegg [Wed, 15 May 2024 16:08:18 +0000 (12:08 -0400)]
21705: Use context.Background() for aws sdk calls.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21705: Fixup error wrapping and logging.
Tom Clegg [Wed, 15 May 2024 16:06:31 +0000 (12:06 -0400)]
21705: Fixup error wrapping and logging.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21640-max-nofile'
Tom Clegg [Fri, 24 May 2024 19:03:55 +0000 (15:03 -0400)]
Merge branch '21640-max-nofile'

fixes #21640

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '12917-group-contents-include-container'
Tom Clegg [Fri, 24 May 2024 19:01:53 +0000 (15:01 -0400)]
Merge branch '12917-group-contents-include-container'

closes #12917

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21705-go-deps'
Tom Clegg [Thu, 23 May 2024 20:13:04 +0000 (16:13 -0400)]
Merge branch '21705-go-deps'

refs #21705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoArvbox want openssh-server, no issue #
Peter Amstutz [Thu, 23 May 2024 15:21:05 +0000 (11:21 -0400)]
Arvbox want openssh-server, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

6 months ago21640: Stop accidentally reducing NOFILE limit. 21640-max-nofile
Tom Clegg [Thu, 23 May 2024 14:34:49 +0000 (10:34 -0400)]
21640: Stop accidentally reducing NOFILE limit.

LimitNOFILE=65536 used to increase softlimit from 1024 to 65536.  But
Go >= 1.19 automatically raises softlimit to hardlimit, so now
LimitNOFILE=65536 only serves to reduce hardlimit from 524288 to
65536, which we don't want.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago12917: Update documentation for "include" parameter.
Tom Clegg [Wed, 22 May 2024 20:22:55 +0000 (16:22 -0400)]
12917: Update documentation for "include" parameter.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago12917: Support include=container_uuid at groups#contents endpoint. 12917-group-contents-include-container
Tom Clegg [Wed, 22 May 2024 19:42:56 +0000 (15:42 -0400)]
12917: Support include=container_uuid at groups#contents endpoint.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21718-memoryview-decode' refs #21718
Peter Amstutz [Wed, 22 May 2024 18:16:00 +0000 (14:16 -0400)]
Merge branch '21718-memoryview-decode' refs #21718

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

6 months agoMerge branch '21718-memoryview-readfrom-v2' refs #21718
Peter Amstutz [Wed, 22 May 2024 18:15:46 +0000 (14:15 -0400)]
Merge branch '21718-memoryview-readfrom-v2'  refs #21718

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

6 months agoMerge branch '21712-favorites-overwrite-bug'
Lisa Knox [Wed, 22 May 2024 13:42:41 +0000 (09:42 -0400)]
Merge branch '21712-favorites-overwrite-bug'

closes #21712

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

6 months ago21718: Review comments 21718-memoryview-readfrom-v2
Peter Amstutz [Tue, 21 May 2024 17:56:41 +0000 (13:56 -0400)]
21718: Review comments

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

6 months ago21718: Add "return_memoryview" flag to read() and readfrom()
Peter Amstutz [Mon, 20 May 2024 20:24:41 +0000 (16:24 -0400)]
21718: Add "return_memoryview" flag to read() and readfrom()

If return_memoryview is true, always return memoryview.  This makes
for more consistent read() and readfrom() behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21718: Update cwltool dependency 21718-memoryview-decode
Peter Amstutz [Mon, 13 May 2024 15:13:43 +0000 (11:13 -0400)]
21718: Update cwltool dependency

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21718: Remove unused import
Peter Amstutz [Tue, 7 May 2024 13:11:14 +0000 (09:11 -0400)]
21718: Remove unused import

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21718: Replace .decode method with str(bytes, "utf-8")
Peter Amstutz [Fri, 3 May 2024 14:38:19 +0000 (10:38 -0400)]
21718: Replace .decode method with str(bytes, "utf-8")

This is because some bytes-like objects (such as memoryview) don't
have a "decode()" method. It appears that the preferred way to get a
decoded string is to simply call the constructor with the bytes-like
object and the desired encoding.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

7 months ago21712: added category param in link verification function 21712-favorites-overwrite-bug
Lisa Knox [Fri, 17 May 2024 16:07:02 +0000 (12:07 -0400)]
21712: added category param in link verification function

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21535-multi-wf-delete'
Lisa Knox [Thu, 16 May 2024 13:41:37 +0000 (09:41 -0400)]
Merge branch '21535-multi-wf-delete'

closes #21535

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21666-provision-test-improvement'
Lucas Di Pentima [Wed, 15 May 2024 18:47:15 +0000 (15:47 -0300)]
Merge branch '21666-provision-test-improvement'

closes #21666

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21666: Update --test help message 21666-provision-test-improvement
Brett Smith [Wed, 15 May 2024 18:42:04 +0000 (14:42 -0400)]
21666: Update --test help message

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21535: fixed deletion verify assertion Arvados-DCO-1.1-Signed-off-by: Lisa Knox ... 21535-multi-wf-delete
Lisa Knox [Wed, 15 May 2024 18:21:02 +0000 (14:21 -0400)]
21535: fixed deletion verify assertion Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: added test for individual wf not existinf in dom Arvados-DCO-1.1-Signed-off...
Lisa Knox [Wed, 15 May 2024 18:15:11 +0000 (14:15 -0400)]
21535: added test for individual wf not existinf in dom Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: changed test spec from select all to select each individually Arvados-DCO...
Lisa Knox [Wed, 15 May 2024 17:48:28 +0000 (13:48 -0400)]
21535: changed test spec from select all to select each individually Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21774-package-dependencies'
Brett Smith [Wed, 15 May 2024 16:52:06 +0000 (12:52 -0400)]
Merge branch '21774-package-dependencies'

Closes #21774.
Refs #21663, #21664.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21774: Add libfuse2 dependency to FUSE package 21774-package-dependencies
Brett Smith [Wed, 15 May 2024 14:53:27 +0000 (10:53 -0400)]
21774: Add libfuse2 dependency to FUSE package

Otherwise the `fuse` dependency can be satisfied entirely by
libfuse3. arv-mount doesn't work if you happen to install this way.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21535: removed hover assertion Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Wed, 15 May 2024 14:13:51 +0000 (10:13 -0400)]
21535: removed hover assertion Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21611-log-chunk-delay'
Tom Clegg [Wed, 15 May 2024 14:07:44 +0000 (10:07 -0400)]
Merge branch '21611-log-chunk-delay'

fixes #21611

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Fix formatting of config keys.
Tom Clegg [Wed, 15 May 2024 14:07:06 +0000 (10:07 -0400)]
21611: Fix formatting of config keys.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoLog more politely via test logger.
Tom Clegg [Wed, 15 May 2024 14:03:48 +0000 (10:03 -0400)]
Log more politely via test logger.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21535: test spec tweaks Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii...
Lisa Knox [Wed, 15 May 2024 13:37:10 +0000 (09:37 -0400)]
21535: test spec tweaks Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: removed .only and added more assertions Arvados-DCO-1.1-Signed-off-by: Lisa...
Lisa Knox [Wed, 15 May 2024 13:25:16 +0000 (09:25 -0400)]
21535: removed .only and added more assertions Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: added .only for testing Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Wed, 15 May 2024 13:08:31 +0000 (09:08 -0400)]
21535: added .only for testing Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months agoMerge branch '21766-disk-cache-size'
Tom Clegg [Tue, 14 May 2024 20:30:17 +0000 (16:30 -0400)]
Merge branch '21766-disk-cache-size'

fixes #21766

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21535: moved MultiSelectMenuActionNames to its own file Arvados-DCO-1.1-Signed-off...
Lisa Knox [Tue, 14 May 2024 18:47:20 +0000 (14:47 -0400)]
21535: moved MultiSelectMenuActionNames to its own file Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 14 May 2024 17:38:36 +0000 (13:38 -0400)]
21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 14 May 2024 17:12:01 +0000 (13:12 -0400)]
21535: added timeouts to test spec Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: replaced deleteWorkflow with removeWorkflowPermanently for consistency Arvados...
Lisa Knox [Tue, 14 May 2024 15:58:45 +0000 (11:58 -0400)]
21535: replaced deleteWorkflow with removeWorkflowPermanently for consistency Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21705: Pass static credentials explicitly instead of using env vars.
Tom Clegg [Tue, 14 May 2024 15:15:30 +0000 (11:15 -0400)]
21705: Pass static credentials explicitly instead of using env vars.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21535: added multi wf delete test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa...
Lisa Knox [Tue, 14 May 2024 15:07:02 +0000 (11:07 -0400)]
21535: added multi wf delete test Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21705: Go mod tidy.
Tom Clegg [Tue, 14 May 2024 15:01:30 +0000 (11:01 -0400)]
21705: Go mod tidy.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Update adal to unretracted version.
Tom Clegg [Tue, 14 May 2024 14:11:44 +0000 (10:11 -0400)]
21705: Update adal to unretracted version.

go: warning: github.com/Azure/go-autorest/autorest/adal@v0.9.17: retracted by module author: retracted due to token refresh errors

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Migrate keep-web s3 tests to aws-sdk-go-v2.
Tom Clegg [Mon, 13 May 2024 22:29:07 +0000 (18:29 -0400)]
21705: Migrate keep-web s3 tests to aws-sdk-go-v2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21766: Enable keepclient debug logging in keep-web. 21766-disk-cache-size
Tom Clegg [Tue, 14 May 2024 13:36:33 +0000 (09:36 -0400)]
21766: Enable keepclient debug logging in keep-web.

Clients can now specify where to send logs by setting the Logger field
on an arvados.Client or arvadosclient.ArvadosClient.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Migrate keepstore s3 driver to latest aws-sdk-go-v2.
Tom Clegg [Mon, 13 May 2024 21:27:20 +0000 (17:27 -0400)]
21705: Migrate keepstore s3 driver to latest aws-sdk-go-v2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Upgrade go-retryablehttp and remove workaround.
Tom Clegg [Fri, 10 May 2024 19:25:36 +0000 (15:25 -0400)]
21705: Upgrade go-retryablehttp and remove workaround.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Migrate lib/cloud/ec2 from aws-sdk-go to aws-sdk-go-v2.
Tom Clegg [Thu, 2 May 2024 20:45:28 +0000 (16:45 -0400)]
21705: Migrate lib/cloud/ec2 from aws-sdk-go to aws-sdk-go-v2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21705: Update go runtime to 1.22.2.
Tom Clegg [Thu, 2 May 2024 18:25:29 +0000 (14:25 -0400)]
21705: Update go runtime to 1.22.2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21678-installer-diagnostics-internal'. Closes #21678
Lucas Di Pentima [Mon, 13 May 2024 19:48:05 +0000 (16:48 -0300)]
Merge branch '21678-installer-diagnostics-internal'. Closes #21678

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21678: Removes unused variables and trailing whitespaces. 21678-installer-diagnostics-internal
Lucas Di Pentima [Mon, 13 May 2024 19:40:48 +0000 (16:40 -0300)]
21678: Removes unused variables and trailing whitespaces.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21535: added wf delete dialog Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox...
Lisa Knox [Mon, 13 May 2024 19:25:12 +0000 (15:25 -0400)]
21535: added wf delete dialog Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21678: Improves credentials passing even further.
Lucas Di Pentima [Mon, 13 May 2024 19:12:43 +0000 (16:12 -0300)]
21678: Improves credentials passing even further.

By using a here-doc, the token doesn't get leaked on any of the process lists.
This also has the advantage of not needing any config file management.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21678: Passes credentials through conf file instead of env vars.
Lucas Di Pentima [Mon, 13 May 2024 18:00:46 +0000 (15:00 -0300)]
21678: Passes credentials through conf file instead of env vars.

To avoid leaking the token to the remote host process list, and also
minimize the exposure on the local host, instead of passing the credentials
through environment variables, we build a conf file that get piped to ssh
via stdout/stdin.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21535: changed wf and process to use new delete icon Arvados-DCO-1.1-Signed-off-by...
Lisa Knox [Mon, 13 May 2024 17:56:49 +0000 (13:56 -0400)]
21535: changed wf and process to use new delete icon Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21535: made ownerUuid param optional and removed it for multiselect Arvados-DCO-1...
Lisa Knox [Mon, 13 May 2024 16:27:32 +0000 (12:27 -0400)]
21535: made ownerUuid param optional and removed it for multiselect Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21678: Copyedit installer diagnostics documentation
Brett Smith [Sun, 12 May 2024 21:15:35 +0000 (17:15 -0400)]
21678: Copyedit installer diagnostics documentation

Small wording and punctuation changes that I hope make things a little
clearer.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21766: Fix WebDAVCache.DiskCacheSize config being ignored.
Tom Clegg [Fri, 10 May 2024 18:34:57 +0000 (14:34 -0400)]
21766: Fix WebDAVCache.DiskCacheSize config being ignored.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21207-pytest'
Brett Smith [Fri, 10 May 2024 17:15:28 +0000 (13:15 -0400)]
Merge branch '21207-pytest'

Closes #21207, #21733.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21635-banner-test-fix' into main. Closes #21635
Stephen Smith [Fri, 10 May 2024 13:23:43 +0000 (09:23 -0400)]
Merge branch '21635-banner-test-fix' into main. Closes #21635

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21766: Fix disk cache size percentage calculation.
Tom Clegg [Thu, 9 May 2024 21:29:39 +0000 (17:29 -0400)]
21766: Fix disk cache size percentage calculation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21207: Move FUSE test abort logs into pytest exit message 21207-pytest
Brett Smith [Thu, 9 May 2024 18:29:15 +0000 (14:29 -0400)]
21207: Move FUSE test abort logs into pytest exit message

Since pytest doesn't display live logs, they're no longer helpful. We
could configure pytest to display live logs, but overall just moving the
information to the exit message seems like a cleaner solution.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Abort FUSE tests with exit status 2
Brett Smith [Thu, 9 May 2024 18:25:41 +0000 (14:25 -0400)]
21207: Abort FUSE tests with exit status 2

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21635: Add checks to wait for banner to close in tooltip test 21635-banner-test-fix
Stephen Smith [Wed, 8 May 2024 16:01:51 +0000 (12:01 -0400)]
21635: Add checks to wait for banner to close in tooltip test

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21678: Updates documentation
Lucas Di Pentima [Mon, 6 May 2024 19:16:08 +0000 (16:16 -0300)]
21678: Updates documentation

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21678: Adds 'diagnostics-internal' subcommand to installer.sh
Lucas Di Pentima [Mon, 6 May 2024 18:37:43 +0000 (15:37 -0300)]
21678: Adds 'diagnostics-internal' subcommand to installer.sh

It checks if there's a node with the 'shell' role, and runs diagnostics
on it as root (to be able to use docker)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21207: Abort test runs with pytest.exit
Brett Smith [Mon, 6 May 2024 22:19:48 +0000 (18:19 -0400)]
21207: Abort test runs with pytest.exit

This lets pytest better report the problem versus an un-handle-able
SIGKILL.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Update example test selection for pytest
Brett Smith [Mon, 6 May 2024 22:02:24 +0000 (18:02 -0400)]
21207: Update example test selection for pytest

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21535: changed delete method to handle multiple workflows Arvados-DCO-1.1-Signed...
Lisa Knox [Mon, 6 May 2024 18:20:59 +0000 (14:20 -0400)]
21535: changed delete method to handle multiple workflows Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21611: Fix test race by waiting for dblocks to be released. 21611-log-chunk-delay
Tom Clegg [Mon, 6 May 2024 15:07:47 +0000 (11:07 -0400)]
21611: Fix test race by waiting for dblocks to be released.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Log "disabled by config" log message only once.
Tom Clegg [Mon, 6 May 2024 14:45:39 +0000 (10:45 -0400)]
21611: Log "disabled by config" log message only once.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Remove configs for periodic container log sweep.
Tom Clegg [Mon, 6 May 2024 14:02:14 +0000 (10:02 -0400)]
21611: Remove configs for periodic container log sweep.

Now that there are no new container logs being written to the logs
table, we can just delete the remaining entries and stop checking.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21666: Adds logging and status check to exit prematurely when failing.
Lucas Di Pentima [Tue, 16 Apr 2024 17:37:15 +0000 (14:37 -0300)]
21666: Adds logging and status check to exit prematurely when failing.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21666: Adds 'arvados-client diagnostics' to run-tests.sh
Lucas Di Pentima [Mon, 15 Apr 2024 19:02:58 +0000 (16:02 -0300)]
21666: Adds 'arvados-client diagnostics' to run-tests.sh

In addition and prior to running the hasher wf test, run the arvados-client
diagnostics test.
This assumes that docker and arvados-client are installed on the local
system, making the shell node a good candidate on multi-node (production)
installations.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months agoMerge branch '21383-misc-fixes'. Refs #21383
Lucas Di Pentima [Mon, 6 May 2024 13:22:16 +0000 (10:22 -0300)]
Merge branch '21383-misc-fixes'. Refs #21383

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21383: Adds http_endpoint="enabled" to Terraform's ec2 instances code.
Lucas Di Pentima [Fri, 26 Apr 2024 19:45:42 +0000 (16:45 -0300)]
21383: Adds http_endpoint="enabled" to Terraform's ec2 instances code.

Although the documentation states that it's enabled by default, I was getting
an error like this:

Error: creating EC2 Instance: InvalidParameterValue: A value of '' is not valid for HttpEndpoint. Specify either 'enabled' or 'disabled' and try again.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21611: Add upgrade note about removed configs.
Tom Clegg [Mon, 6 May 2024 13:19:07 +0000 (09:19 -0400)]
21611: Add upgrade note about removed configs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21207: Run Python tests with pytest
Brett Smith [Fri, 3 May 2024 20:27:32 +0000 (16:27 -0400)]
21207: Run Python tests with pytest

This gets us away from a deprecated setup.py invocation, and provides
richer test authoring and result reporting facilities.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Remove Python slow/short tests support
Brett Smith [Fri, 3 May 2024 20:34:38 +0000 (16:34 -0400)]
21207: Remove Python slow/short tests support

After the previous commit there are no more users of this, and this
interface will no longer be supported after our migration to pytest. If
we want this functionality in the future we should set up a custom
"slow" mark in pytest and use pytest's built-in mark filtering.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Remove FUSE performance tests
Brett Smith [Fri, 3 May 2024 20:13:36 +0000 (16:13 -0400)]
21207: Remove FUSE performance tests

None of these tests have been running as intended. The `slow_test`
decorator is written incorrectly such that the return value of
`unittest.skipIf` *replaces* the test method rather than *wrapping* it.

We just did a round of serious performance improvement on FUSE and
literally nobody looked at this code or tried to track down the
profiling results it produces.

I'm not opposed to performance tests on principle. But these tests
aren't that, they just generate some profiling results and then spend a
bunch of time running various checks on the manifest that arv-mount
generated.

Remove them to save runtime and make it easier to run tests under
pytest. We can always write better performance tests later if we want.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Rename CWL standalone test functions
Brett Smith [Fri, 3 May 2024 16:02:56 +0000 (12:02 -0400)]
21207: Rename CWL standalone test functions

These functions are not picked up by the unittest runner (since they're
not TestCase methods) but are picked up by pytest (since they start with
test_*). Rename them to retain existing behavior across test runners.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Set up MockedPut._state with a deepcopy
Brett Smith [Fri, 3 May 2024 13:55:15 +0000 (09:55 -0400)]
21207: Set up MockedPut._state with a deepcopy

Otherwise its internal state could get into EMPTY_STATE and contaminate
other tests.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21611: Remove obsolete configs.
Tom Clegg [Fri, 3 May 2024 18:25:01 +0000 (14:25 -0400)]
21611: Remove obsolete configs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Test that log is saved after logging preemption notice.
Tom Clegg [Fri, 3 May 2024 17:31:59 +0000 (13:31 -0400)]
21611: Test that log is saved after logging preemption notice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Fix unreliable test.
Tom Clegg [Thu, 2 May 2024 14:18:51 +0000 (10:18 -0400)]
21611: Fix unreliable test.

https://ci.arvados.org/job/developer-run-tests-remainder/4413/consoleFull

FAIL: integration_test.go:1158: IntegrationSuite.TestRunTrivialContainer

integration_test.go:1313:
    c.Check(allStatus, check.Matches, `Queued, waiting for dispatch\n`+
        `(Queued, waiting.*\n)*`+
        `(Locked, waiting for dispatch\n)?`+
        `(Locked, waiting for new instance to be ready\n)?`+
        `(Locked, preparing runtime environment\n)?`+
        `(Running, \n)?`+
        `Complete, \n`)
... value string = "" +
...     "Queued, waiting for dispatch\n" +
...     "Locked, waiting for new instance to be ready\n" +
...     "Locked, preparing runtime environment\n" +
...     "Queued, preparing runtime environment\n" +
...     "Locked, waiting for new instance to be ready\n" +
...     "Locked, preparing runtime environment\n" +
...     "Running, \n" +
...     "Complete, \n"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Remove log-to-logs-table code.
Tom Clegg [Wed, 1 May 2024 13:39:29 +0000 (09:39 -0400)]
21611: Remove log-to-logs-table code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '15397-remove-obsolete-apis'
Tom Clegg [Fri, 3 May 2024 17:33:28 +0000 (13:33 -0400)]
Merge branch '15397-remove-obsolete-apis'

refs #15397

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Merge branch 'main'
Tom Clegg [Fri, 3 May 2024 14:53:32 +0000 (10:53 -0400)]
15397: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '12430-output-glob'
Tom Clegg [Thu, 2 May 2024 20:47:51 +0000 (16:47 -0400)]
Merge branch '12430-output-glob'

closes #12430

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21356-clean-imports'
Brett Smith [Thu, 2 May 2024 15:47:42 +0000 (11:47 -0400)]
Merge branch '21356-clean-imports'

Closes #21356.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21717-keepstore-cors'
Tom Clegg [Thu, 2 May 2024 15:27:33 +0000 (11:27 -0400)]
Merge branch '21717-keepstore-cors'

fixes #21717

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21642-io-panel-collection-tab-bug' into main. Closes #21642
Stephen Smith [Wed, 1 May 2024 15:37:07 +0000 (11:37 -0400)]
Merge branch '21642-io-panel-collection-tab-bug' into main. Closes #21642

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>