arvados.git
6 months agoMerge branch '21666-provision-test-improvement'
Lucas Di Pentima [Wed, 15 May 2024 18:47:15 +0000 (15:47 -0300)]
Merge branch '21666-provision-test-improvement'

closes #21666

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21666: Update --test help message 21666-provision-test-improvement
Brett Smith [Wed, 15 May 2024 18:42:04 +0000 (14:42 -0400)]
21666: Update --test help message

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months agoMerge branch '21774-package-dependencies'
Brett Smith [Wed, 15 May 2024 16:52:06 +0000 (12:52 -0400)]
Merge branch '21774-package-dependencies'

Closes #21774.
Refs #21663, #21664.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21774: Add libfuse2 dependency to FUSE package 21774-package-dependencies
Brett Smith [Wed, 15 May 2024 14:53:27 +0000 (10:53 -0400)]
21774: Add libfuse2 dependency to FUSE package

Otherwise the `fuse` dependency can be satisfied entirely by
libfuse3. arv-mount doesn't work if you happen to install this way.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months agoMerge branch '21611-log-chunk-delay'
Tom Clegg [Wed, 15 May 2024 14:07:44 +0000 (10:07 -0400)]
Merge branch '21611-log-chunk-delay'

fixes #21611

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21611: Fix formatting of config keys.
Tom Clegg [Wed, 15 May 2024 14:07:06 +0000 (10:07 -0400)]
21611: Fix formatting of config keys.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoLog more politely via test logger.
Tom Clegg [Wed, 15 May 2024 14:03:48 +0000 (10:03 -0400)]
Log more politely via test logger.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21766-disk-cache-size'
Tom Clegg [Tue, 14 May 2024 20:30:17 +0000 (16:30 -0400)]
Merge branch '21766-disk-cache-size'

fixes #21766

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21766: Enable keepclient debug logging in keep-web. 21766-disk-cache-size
Tom Clegg [Tue, 14 May 2024 13:36:33 +0000 (09:36 -0400)]
21766: Enable keepclient debug logging in keep-web.

Clients can now specify where to send logs by setting the Logger field
on an arvados.Client or arvadosclient.ArvadosClient.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21678-installer-diagnostics-internal'. Closes #21678
Lucas Di Pentima [Mon, 13 May 2024 19:48:05 +0000 (16:48 -0300)]
Merge branch '21678-installer-diagnostics-internal'. Closes #21678

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21678: Removes unused variables and trailing whitespaces. 21678-installer-diagnostics-internal
Lucas Di Pentima [Mon, 13 May 2024 19:40:48 +0000 (16:40 -0300)]
21678: Removes unused variables and trailing whitespaces.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21678: Improves credentials passing even further.
Lucas Di Pentima [Mon, 13 May 2024 19:12:43 +0000 (16:12 -0300)]
21678: Improves credentials passing even further.

By using a here-doc, the token doesn't get leaked on any of the process lists.
This also has the advantage of not needing any config file management.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21678: Passes credentials through conf file instead of env vars.
Lucas Di Pentima [Mon, 13 May 2024 18:00:46 +0000 (15:00 -0300)]
21678: Passes credentials through conf file instead of env vars.

To avoid leaking the token to the remote host process list, and also
minimize the exposure on the local host, instead of passing the credentials
through environment variables, we build a conf file that get piped to ssh
via stdout/stdin.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21678: Copyedit installer diagnostics documentation
Brett Smith [Sun, 12 May 2024 21:15:35 +0000 (17:15 -0400)]
21678: Copyedit installer diagnostics documentation

Small wording and punctuation changes that I hope make things a little
clearer.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21766: Fix WebDAVCache.DiskCacheSize config being ignored.
Tom Clegg [Fri, 10 May 2024 18:34:57 +0000 (14:34 -0400)]
21766: Fix WebDAVCache.DiskCacheSize config being ignored.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months agoMerge branch '21207-pytest'
Brett Smith [Fri, 10 May 2024 17:15:28 +0000 (13:15 -0400)]
Merge branch '21207-pytest'

Closes #21207, #21733.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months agoMerge branch '21635-banner-test-fix' into main. Closes #21635
Stephen Smith [Fri, 10 May 2024 13:23:43 +0000 (09:23 -0400)]
Merge branch '21635-banner-test-fix' into main. Closes #21635

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

6 months ago21766: Fix disk cache size percentage calculation.
Tom Clegg [Thu, 9 May 2024 21:29:39 +0000 (17:29 -0400)]
21766: Fix disk cache size percentage calculation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21207: Move FUSE test abort logs into pytest exit message 21207-pytest
Brett Smith [Thu, 9 May 2024 18:29:15 +0000 (14:29 -0400)]
21207: Move FUSE test abort logs into pytest exit message

Since pytest doesn't display live logs, they're no longer helpful. We
could configure pytest to display live logs, but overall just moving the
information to the exit message seems like a cleaner solution.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21207: Abort FUSE tests with exit status 2
Brett Smith [Thu, 9 May 2024 18:25:41 +0000 (14:25 -0400)]
21207: Abort FUSE tests with exit status 2

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21635: Add checks to wait for banner to close in tooltip test 21635-banner-test-fix
Stephen Smith [Wed, 8 May 2024 16:01:51 +0000 (12:01 -0400)]
21635: Add checks to wait for banner to close in tooltip test

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

6 months ago21678: Updates documentation
Lucas Di Pentima [Mon, 6 May 2024 19:16:08 +0000 (16:16 -0300)]
21678: Updates documentation

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21678: Adds 'diagnostics-internal' subcommand to installer.sh
Lucas Di Pentima [Mon, 6 May 2024 18:37:43 +0000 (15:37 -0300)]
21678: Adds 'diagnostics-internal' subcommand to installer.sh

It checks if there's a node with the 'shell' role, and runs diagnostics
on it as root (to be able to use docker)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21207: Abort test runs with pytest.exit
Brett Smith [Mon, 6 May 2024 22:19:48 +0000 (18:19 -0400)]
21207: Abort test runs with pytest.exit

This lets pytest better report the problem versus an un-handle-able
SIGKILL.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21207: Update example test selection for pytest
Brett Smith [Mon, 6 May 2024 22:02:24 +0000 (18:02 -0400)]
21207: Update example test selection for pytest

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

6 months ago21611: Fix test race by waiting for dblocks to be released. 21611-log-chunk-delay
Tom Clegg [Mon, 6 May 2024 15:07:47 +0000 (11:07 -0400)]
21611: Fix test race by waiting for dblocks to be released.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21611: Log "disabled by config" log message only once.
Tom Clegg [Mon, 6 May 2024 14:45:39 +0000 (10:45 -0400)]
21611: Log "disabled by config" log message only once.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21611: Remove configs for periodic container log sweep.
Tom Clegg [Mon, 6 May 2024 14:02:14 +0000 (10:02 -0400)]
21611: Remove configs for periodic container log sweep.

Now that there are no new container logs being written to the logs
table, we can just delete the remaining entries and stop checking.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

6 months ago21666: Adds logging and status check to exit prematurely when failing.
Lucas Di Pentima [Tue, 16 Apr 2024 17:37:15 +0000 (14:37 -0300)]
21666: Adds logging and status check to exit prematurely when failing.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21666: Adds 'arvados-client diagnostics' to run-tests.sh
Lucas Di Pentima [Mon, 15 Apr 2024 19:02:58 +0000 (16:02 -0300)]
21666: Adds 'arvados-client diagnostics' to run-tests.sh

In addition and prior to running the hasher wf test, run the arvados-client
diagnostics test.
This assumes that docker and arvados-client are installed on the local
system, making the shell node a good candidate on multi-node (production)
installations.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months agoMerge branch '21383-misc-fixes'. Refs #21383
Lucas Di Pentima [Mon, 6 May 2024 13:22:16 +0000 (10:22 -0300)]
Merge branch '21383-misc-fixes'. Refs #21383

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21383: Adds http_endpoint="enabled" to Terraform's ec2 instances code.
Lucas Di Pentima [Fri, 26 Apr 2024 19:45:42 +0000 (16:45 -0300)]
21383: Adds http_endpoint="enabled" to Terraform's ec2 instances code.

Although the documentation states that it's enabled by default, I was getting
an error like this:

Error: creating EC2 Instance: InvalidParameterValue: A value of '' is not valid for HttpEndpoint. Specify either 'enabled' or 'disabled' and try again.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

6 months ago21611: Add upgrade note about removed configs.
Tom Clegg [Mon, 6 May 2024 13:19:07 +0000 (09:19 -0400)]
21611: Add upgrade note about removed configs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21207: Run Python tests with pytest
Brett Smith [Fri, 3 May 2024 20:27:32 +0000 (16:27 -0400)]
21207: Run Python tests with pytest

This gets us away from a deprecated setup.py invocation, and provides
richer test authoring and result reporting facilities.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Remove Python slow/short tests support
Brett Smith [Fri, 3 May 2024 20:34:38 +0000 (16:34 -0400)]
21207: Remove Python slow/short tests support

After the previous commit there are no more users of this, and this
interface will no longer be supported after our migration to pytest. If
we want this functionality in the future we should set up a custom
"slow" mark in pytest and use pytest's built-in mark filtering.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Remove FUSE performance tests
Brett Smith [Fri, 3 May 2024 20:13:36 +0000 (16:13 -0400)]
21207: Remove FUSE performance tests

None of these tests have been running as intended. The `slow_test`
decorator is written incorrectly such that the return value of
`unittest.skipIf` *replaces* the test method rather than *wrapping* it.

We just did a round of serious performance improvement on FUSE and
literally nobody looked at this code or tried to track down the
profiling results it produces.

I'm not opposed to performance tests on principle. But these tests
aren't that, they just generate some profiling results and then spend a
bunch of time running various checks on the manifest that arv-mount
generated.

Remove them to save runtime and make it easier to run tests under
pytest. We can always write better performance tests later if we want.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Rename CWL standalone test functions
Brett Smith [Fri, 3 May 2024 16:02:56 +0000 (12:02 -0400)]
21207: Rename CWL standalone test functions

These functions are not picked up by the unittest runner (since they're
not TestCase methods) but are picked up by pytest (since they start with
test_*). Rename them to retain existing behavior across test runners.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21207: Set up MockedPut._state with a deepcopy
Brett Smith [Fri, 3 May 2024 13:55:15 +0000 (09:55 -0400)]
21207: Set up MockedPut._state with a deepcopy

Otherwise its internal state could get into EMPTY_STATE and contaminate
other tests.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21611: Remove obsolete configs.
Tom Clegg [Fri, 3 May 2024 18:25:01 +0000 (14:25 -0400)]
21611: Remove obsolete configs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Test that log is saved after logging preemption notice.
Tom Clegg [Fri, 3 May 2024 17:31:59 +0000 (13:31 -0400)]
21611: Test that log is saved after logging preemption notice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Fix unreliable test.
Tom Clegg [Thu, 2 May 2024 14:18:51 +0000 (10:18 -0400)]
21611: Fix unreliable test.

https://ci.arvados.org/job/developer-run-tests-remainder/4413/consoleFull

FAIL: integration_test.go:1158: IntegrationSuite.TestRunTrivialContainer

integration_test.go:1313:
    c.Check(allStatus, check.Matches, `Queued, waiting for dispatch\n`+
        `(Queued, waiting.*\n)*`+
        `(Locked, waiting for dispatch\n)?`+
        `(Locked, waiting for new instance to be ready\n)?`+
        `(Locked, preparing runtime environment\n)?`+
        `(Running, \n)?`+
        `Complete, \n`)
... value string = "" +
...     "Queued, waiting for dispatch\n" +
...     "Locked, waiting for new instance to be ready\n" +
...     "Locked, preparing runtime environment\n" +
...     "Queued, preparing runtime environment\n" +
...     "Locked, waiting for new instance to be ready\n" +
...     "Locked, preparing runtime environment\n" +
...     "Running, \n" +
...     "Complete, \n"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21611: Remove log-to-logs-table code.
Tom Clegg [Wed, 1 May 2024 13:39:29 +0000 (09:39 -0400)]
21611: Remove log-to-logs-table code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '15397-remove-obsolete-apis'
Tom Clegg [Fri, 3 May 2024 17:33:28 +0000 (13:33 -0400)]
Merge branch '15397-remove-obsolete-apis'

refs #15397

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Merge branch 'main'
Tom Clegg [Fri, 3 May 2024 14:53:32 +0000 (10:53 -0400)]
15397: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '12430-output-glob'
Tom Clegg [Thu, 2 May 2024 20:47:51 +0000 (16:47 -0400)]
Merge branch '12430-output-glob'

closes #12430

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21356-clean-imports'
Brett Smith [Thu, 2 May 2024 15:47:42 +0000 (11:47 -0400)]
Merge branch '21356-clean-imports'

Closes #21356.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21717-keepstore-cors'
Tom Clegg [Thu, 2 May 2024 15:27:33 +0000 (11:27 -0400)]
Merge branch '21717-keepstore-cors'

fixes #21717

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21642-io-panel-collection-tab-bug' into main. Closes #21642
Stephen Smith [Wed, 1 May 2024 15:37:07 +0000 (11:37 -0400)]
Merge branch '21642-io-panel-collection-tab-bug' into main. Closes #21642

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21700: Remove package-test-dockerfiles/Makefile instructions
Brett Smith [Wed, 1 May 2024 14:38:17 +0000 (10:38 -0400)]
21700: Remove package-test-dockerfiles/Makefile instructions

This Makefile was removed in 792a7d47b560d4aa80e2cb8a933a11e02e3109b3
because it only supported RVM, so there's no more need to update it.
Refs #21700.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago15397: Remove checks for nodes and repositories endpoints.
Tom Clegg [Tue, 30 Apr 2024 18:43:13 +0000 (14:43 -0400)]
15397: Remove checks for nodes and repositories endpoints.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Fix python tests relying on removed APIs/fixtures.
Tom Clegg [Thu, 25 Apr 2024 19:24:02 +0000 (15:24 -0400)]
15397: Fix python tests relying on removed APIs/fixtures.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Move deprecated API links to old version of doc site.
Tom Clegg [Thu, 25 Apr 2024 18:39:55 +0000 (14:39 -0400)]
15397: Move deprecated API links to old version of doc site.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Remove code, configs, and docs for hosted git repo feature.
Tom Clegg [Mon, 22 Apr 2024 14:38:33 +0000 (10:38 -0400)]
15397: Remove code, configs, and docs for hosted git repo feature.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Remove obsolete APIs.
Tom Clegg [Wed, 17 Apr 2024 13:58:09 +0000 (09:58 -0400)]
15397: Remove obsolete APIs.

jobs, job_tasks, pipeline_instances, pipeline_templates, nodes,
repositories, humans, specimens, traits.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago15397: Silence nginx log noise like "closed keepalive connection".
Tom Clegg [Mon, 22 Apr 2024 18:20:14 +0000 (14:20 -0400)]
15397: Silence nginx log noise like "closed keepalive connection".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21717: Fix incorrect header. 21717-keepstore-cors
Tom Clegg [Tue, 30 Apr 2024 18:02:23 +0000 (14:02 -0400)]
21717: Fix incorrect header.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago12430: Add more glob unit test cases. 12430-output-glob
Tom Clegg [Tue, 30 Apr 2024 17:32:02 +0000 (13:32 -0400)]
12430: Add more glob unit test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago12430: Add comments about the two stages of glob-filtering.
Tom Clegg [Tue, 30 Apr 2024 15:37:59 +0000 (11:37 -0400)]
12430: Add comments about the two stages of glob-filtering.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago12430: Avoid loading collections excluded by globs.
Tom Clegg [Tue, 30 Apr 2024 15:18:31 +0000 (11:18 -0400)]
12430: Avoid loading collections excluded by globs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21356: Remove all Python 2/3 compatibility imports 21356-clean-imports
Brett Smith [Mon, 29 Apr 2024 16:11:05 +0000 (12:11 -0400)]
21356: Remove all Python 2/3 compatibility imports

This includes noop imports from __future__ and builtins, as well as
imports from the future and six compatibility libraries.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21642: Fix unit tests to account for hidden instead of absent tab content 21642-io-panel-collection-tab-bug
Stephen Smith [Mon, 29 Apr 2024 16:37:47 +0000 (12:37 -0400)]
21642: Fix unit tests to account for hidden instead of absent tab content

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21642: Change conditionaltabs to render tabs as hidden to avoid re-renders
Stephen Smith [Mon, 29 Apr 2024 13:35:02 +0000 (09:35 -0400)]
21642: Change conditionaltabs to render tabs as hidden to avoid re-renders

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21642: Update caniuse
Stephen Smith [Mon, 29 Apr 2024 13:32:27 +0000 (09:32 -0400)]
21642: Update caniuse

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago12430: Add test for avoiding loading collections excluded by globs.
Tom Clegg [Fri, 26 Apr 2024 20:48:57 +0000 (16:48 -0400)]
12430: Add test for avoiding loading collections excluded by globs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago12430: Expand output_glob documentation.
Tom Clegg [Fri, 26 Apr 2024 14:39:36 +0000 (10:39 -0400)]
12430: Expand output_glob documentation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21700: Install Bundler system-wide in Rails postinst
Brett Smith [Fri, 26 Apr 2024 14:03:37 +0000 (10:03 -0400)]
21700: Install Bundler system-wide in Rails postinst

This effectively reverts 7e2165a5b7561f0c84ede6ab4ae58003a1551a48 and
follow-ups. Other parts of our install process like test-provision
assume that the correct version of Bundler is installed system-wide
after installing arvados-api-server. So just do that and find the
`bundle` command after the fact. Refs #21700.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago14230: Comment about test coverage, add trailing-** case.
Tom Clegg [Fri, 26 Apr 2024 13:36:18 +0000 (09:36 -0400)]
14230: Comment about test coverage, add trailing-** case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21721-refresh-python-deps'
Brett Smith [Fri, 26 Apr 2024 10:55:12 +0000 (06:55 -0400)]
Merge branch '21721-refresh-python-deps'

Refs #21721.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21700-remove-rvm'
Brett Smith [Fri, 26 Apr 2024 10:54:59 +0000 (06:54 -0400)]
Merge branch '21700-remove-rvm'

Closes #21700.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21700: Tighten Gemfile.lock parsing
Brett Smith [Fri, 26 Apr 2024 10:53:29 +0000 (06:53 -0400)]
21700: Tighten Gemfile.lock parsing

Avoid a bug if some subsection comes after `specs` under `GEM`.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21700: Remove arvbox special case from run-tests.sh
Brett Smith [Fri, 26 Apr 2024 09:39:36 +0000 (05:39 -0400)]
21700: Remove arvbox special case from run-tests.sh

This was already a noop after ed268e53d2014ce5c49301559a46e67660f1404c,
and I suspect it's no longer necessary now that we do a better job of
standardizing the Bundler version anywhere, thanks in part to the
version locking added in Bundler 2.3.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21700: Modernize arvbox bundler install recipe
Brett Smith [Fri, 26 Apr 2024 09:39:21 +0000 (05:39 -0400)]
21700: Modernize arvbox bundler install recipe

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21722-remove-chokidar' into main. Closes #21722
Stephen Smith [Thu, 25 Apr 2024 14:36:07 +0000 (10:36 -0400)]
Merge branch '21722-remove-chokidar' into main. Closes #21722

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21717: Allow cross-origin keepstore requests.
Tom Clegg [Thu, 25 Apr 2024 13:59:54 +0000 (09:59 -0400)]
21717: Allow cross-origin keepstore requests.

Also add some missing Allow-Header and Expose-Header entries to
keepproxy CORS headers.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21721: Remove sdk/cwl version pins 21721-refresh-python-deps
Brett Smith [Wed, 24 Apr 2024 20:15:12 +0000 (16:15 -0400)]
21721: Remove sdk/cwl version pins

Both of these were set to work around limitations in the old pip
resolver. We have since standardized on the new one in our build
process, so they are no longer needed.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove msgpack version pin
Brett Smith [Wed, 24 Apr 2024 20:13:47 +0000 (16:13 -0400)]
21721: Remove msgpack version pin

This was pinned to avoid an install bug in version 1.0.4. Newer releases
seem fine. See e2267bd9920.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove pycurl version pin
Brett Smith [Wed, 24 Apr 2024 20:03:47 +0000 (16:03 -0400)]
21721: Remove pycurl version pin

This was originally pinned because of library conflicts on Debian 10,
which we no longer support. See commit
d655e2addaba4168f76fb7c4e2e9f6d649917bcc.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove ruamel.yaml dependency
Brett Smith [Wed, 24 Apr 2024 19:55:06 +0000 (15:55 -0400)]
21721: Remove ruamel.yaml dependency

We import it a few times but never actually use it.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove httplib2 and associated version pins
Brett Smith [Wed, 24 Apr 2024 19:29:54 +0000 (15:29 -0400)]
21721: Remove httplib2 and associated version pins

This whole stack was pinned to stay on an older version of
google-python-api-client, which has since been relaxed, and then keep
things working under the old pip resolver, which we've standardized off
of. Refs #18336.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove google-auth version pin
Brett Smith [Wed, 24 Apr 2024 19:08:22 +0000 (15:08 -0400)]
21721: Remove google-auth version pin

Refs #18001. The original motivation for this pin is not
well-documented. My best guess is it was to retain Python 2.7 support,
which is no longer a concern.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove tests_require=mock
Brett Smith [Wed, 24 Apr 2024 18:46:44 +0000 (14:46 -0400)]
21721: Remove tests_require=mock

unittest.mock in the stdlib is good enough for us.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21719-deps-security-updates'. Closes #21719
Lucas Di Pentima [Wed, 24 Apr 2024 18:46:43 +0000 (15:46 -0300)]
Merge branch '21719-deps-security-updates'. Closes #21719

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21721: Clean various test imports for Python 3
Brett Smith [Wed, 24 Apr 2024 18:15:55 +0000 (14:15 -0400)]
21721: Clean various test imports for Python 3

This is not everything but cleans up files where I happened to notice
explicit version checks.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21721: Remove tests_require=pbr
Brett Smith [Wed, 24 Apr 2024 18:07:09 +0000 (14:07 -0400)]
21721: Remove tests_require=pbr

This module is a setuptools plugin that requires activation to
use. We're currently not doing that, so it's completely unused and safe
to remove.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21719: Upgrades golang.org/x/net to 0.24.0 21719-deps-security-updates
Lucas Di Pentima [Wed, 24 Apr 2024 17:38:06 +0000 (14:38 -0300)]
21719: Upgrades golang.org/x/net to 0.24.0

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

7 months ago21722: Remove node-sass-chokidar to fix security issues 21722-remove-chokidar
Stephen Smith [Wed, 24 Apr 2024 15:22:55 +0000 (11:22 -0400)]
21722: Remove node-sass-chokidar to fix security issues

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months agoMerge branch '21697-slow-client-blocking'
Tom Clegg [Tue, 23 Apr 2024 19:53:32 +0000 (15:53 -0400)]
Merge branch '21697-slow-client-blocking'

fixes #21697

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21697: Fix typo. 21697-slow-client-blocking
Tom Clegg [Tue, 23 Apr 2024 18:10:30 +0000 (14:10 -0400)]
21697: Fix typo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21697: Comment why releaseSession func exists.
Tom Clegg [Tue, 23 Apr 2024 18:09:34 +0000 (14:09 -0400)]
21697: Comment why releaseSession func exists.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months agoMerge branch '21696-slow-propfind'
Tom Clegg [Tue, 23 Apr 2024 15:36:26 +0000 (11:36 -0400)]
Merge branch '21696-slow-propfind'

fixes #21696

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21642: Add unit tests for conditional tabs, add test case for main process output...
Stephen Smith [Tue, 23 Apr 2024 14:35:49 +0000 (10:35 -0400)]
21642: Add unit tests for conditional tabs, add test case for main process output collection

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>

7 months ago21700: Fix option typo 21700-remove-rvm
Brett Smith [Tue, 23 Apr 2024 14:22:42 +0000 (10:22 -0400)]
21700: Fix option typo

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21697: Don't hold session lock while processing read requests.
Tom Clegg [Tue, 23 Apr 2024 13:51:38 +0000 (09:51 -0400)]
21697: Don't hold session lock while processing read requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

7 months ago21700: Set GEM_PATH in Rails postinst
Brett Smith [Tue, 23 Apr 2024 13:25:59 +0000 (09:25 -0400)]
21700: Set GEM_PATH in Rails postinst

This is necessary to get the desired behavior in Ruby 2.7. Otherwise
Bundler gets installed somewhere else.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21700: Adjust `gem fetch` syntax to work on Ruby 2.7
Brett Smith [Mon, 22 Apr 2024 20:07:18 +0000 (16:07 -0400)]
21700: Adjust `gem fetch` syntax to work on Ruby 2.7

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months agoMerge branch '21037-upgrade-react-scripts'
Lisa Knox [Mon, 22 Apr 2024 18:36:10 +0000 (14:36 -0400)]
Merge branch '21037-upgrade-react-scripts'

closes #21307

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21700: Narrow `bundle install` workaround options
Brett Smith [Mon, 22 Apr 2024 18:32:40 +0000 (14:32 -0400)]
21700: Narrow `bundle install` workaround options

This prevents `gem install` from resolving dependencies differently from
Bundler and creating its own set of problems.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21037: added eslintrc to gitignore Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa... 21037-upgrade-react-scripts
Lisa Knox [Mon, 22 Apr 2024 18:09:27 +0000 (14:09 -0400)]
21037: added eslintrc to gitignore Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

7 months ago21700: Install Bundler alongside bundled gems in Rails postinst
Brett Smith [Mon, 22 Apr 2024 17:51:36 +0000 (13:51 -0400)]
21700: Install Bundler alongside bundled gems in Rails postinst

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

7 months ago21700: Work around incomplete `bundle cache` bug
Brett Smith [Sat, 20 Apr 2024 13:43:01 +0000 (09:43 -0400)]
21700: Work around incomplete `bundle cache` bug

It seems like the problem I was working around for zlib 3.1.0 on Ubuntu
20.04 earlier can affect basically any gem at any step, if you happen to
already have the exact same version installed on the system. This commit
adds workarounds for all gems at all steps of the bundle setup.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>