Lucas Di Pentima [Mon, 13 May 2024 18:00:46 +0000 (15:00 -0300)]
21678: Passes credentials through conf file instead of env vars.
To avoid leaking the token to the remote host process list, and also
minimize the exposure on the local host, instead of passing the credentials
through environment variables, we build a conf file that get piped to ssh
via stdout/stdin.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Thu, 9 May 2024 18:29:15 +0000 (14:29 -0400)]
21207: Move FUSE test abort logs into pytest exit message
Since pytest doesn't display live logs, they're no longer helpful. We
could configure pytest to display live logs, but overall just moving the
information to the exit message seems like a cleaner solution.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Fri, 26 Apr 2024 19:45:42 +0000 (16:45 -0300)]
21383: Adds http_endpoint="enabled" to Terraform's ec2 instances code.
Although the documentation states that it's enabled by default, I was getting
an error like this:
Error: creating EC2 Instance: InvalidParameterValue: A value of '' is not valid for HttpEndpoint. Specify either 'enabled' or 'disabled' and try again.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Fri, 3 May 2024 20:34:38 +0000 (16:34 -0400)]
21207: Remove Python slow/short tests support
After the previous commit there are no more users of this, and this
interface will no longer be supported after our migration to pytest. If
we want this functionality in the future we should set up a custom
"slow" mark in pytest and use pytest's built-in mark filtering.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 3 May 2024 20:13:36 +0000 (16:13 -0400)]
21207: Remove FUSE performance tests
None of these tests have been running as intended. The `slow_test`
decorator is written incorrectly such that the return value of
`unittest.skipIf` *replaces* the test method rather than *wrapping* it.
We just did a round of serious performance improvement on FUSE and
literally nobody looked at this code or tried to track down the
profiling results it produces.
I'm not opposed to performance tests on principle. But these tests
aren't that, they just generate some profiling results and then spend a
bunch of time running various checks on the manifest that arv-mount
generated.
Remove them to save runtime and make it easier to run tests under
pytest. We can always write better performance tests later if we want.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 3 May 2024 16:02:56 +0000 (12:02 -0400)]
21207: Rename CWL standalone test functions
These functions are not picked up by the unittest runner (since they're
not TestCase methods) but are picked up by pytest (since they start with
test_*). Rename them to retain existing behavior across test runners.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 26 Apr 2024 14:03:37 +0000 (10:03 -0400)]
21700: Install Bundler system-wide in Rails postinst
This effectively reverts 7e2165a5b7561f0c84ede6ab4ae58003a1551a48 and
follow-ups. Other parts of our install process like test-provision
assume that the correct version of Bundler is installed system-wide
after installing arvados-api-server. So just do that and find the
`bundle` command after the fact. Refs #21700.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 26 Apr 2024 09:39:36 +0000 (05:39 -0400)]
21700: Remove arvbox special case from run-tests.sh
This was already a noop after ed268e53d2014ce5c49301559a46e67660f1404c,
and I suspect it's no longer necessary now that we do a better job of
standardizing the Bundler version anywhere, thanks in part to the
version locking added in Bundler 2.3.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 24 Apr 2024 20:15:12 +0000 (16:15 -0400)]
21721: Remove sdk/cwl version pins
Both of these were set to work around limitations in the old pip
resolver. We have since standardized on the new one in our build
process, so they are no longer needed.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 24 Apr 2024 19:29:54 +0000 (15:29 -0400)]
21721: Remove httplib2 and associated version pins
This whole stack was pinned to stay on an older version of
google-python-api-client, which has since been relaxed, and then keep
things working under the old pip resolver, which we've standardized off
of. Refs #18336.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 24 Apr 2024 19:08:22 +0000 (15:08 -0400)]
21721: Remove google-auth version pin
Refs #18001. The original motivation for this pin is not
well-documented. My best guess is it was to retain Python 2.7 support,
which is no longer a concern.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sat, 20 Apr 2024 13:43:01 +0000 (09:43 -0400)]
21700: Work around incomplete `bundle cache` bug
It seems like the problem I was working around for zlib 3.1.0 on Ubuntu
20.04 earlier can affect basically any gem at any step, if you happen to
already have the exact same version installed on the system. This commit
adds workarounds for all gems at all steps of the bundle setup.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 19 Apr 2024 19:13:22 +0000 (15:13 -0400)]
21700: Select more desired modules in rocky8 test Dockerfile
This prevents needless thrashing when the Dockerfile installs one set of
packages and then packages we test enable and reinstall a different
module stream.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 19 Apr 2024 16:11:36 +0000 (12:11 -0400)]
21700: Remove `bundle install` from package build Dockerfiles
This work is better done in the package build script. I suspect the
problem that it's trying to solve is better solved by isolating the
RailsAPI `vendor` directory as done in 0333f33a7413aadb7159563e0a7aa2a0a50acaec.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
As long as we have something at least as recent as 2.4.0, that should be
enough for function. This recipe meets our functional needs while
striking a balance between being easy to implement and avoiding
redundant installs.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 19 Apr 2024 15:24:36 +0000 (11:24 -0400)]
21700: Remove RVM support
We haven't reliably tested this in a long time, and it's a lot of code
for no benefit. In general we want to support the versions of Ruby that
come with the distributions we support. Anybody in a very weird case can
install a supported version of Ruby from source.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 19 Apr 2024 14:05:45 +0000 (10:05 -0400)]
21700: Better isolate Bundler from the build source
This helps ensure that when we build the RailsAPI package with Bundler,
we don't accidentally pick up settings or files from the developer's
machine. This shouldn't matter when we build packages on Jenkins but
this makes it easier for developers to test package builds on their own
systems.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
projects / arvados.git / log