arvados.git
4 months ago21910: Comment system root token fixture.
Tom Clegg [Wed, 14 Aug 2024 18:23:41 +0000 (14:23 -0400)]
21910: Comment system root token fixture.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21910: Merge branch 'main' into 21910-remove-api_client_id
Tom Clegg [Wed, 14 Aug 2024 13:16:06 +0000 (09:16 -0400)]
21910: Merge branch 'main' into 21910-remove-api_client_id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21910: Trust system root token even if IssueTrustedTokens == false.
Tom Clegg [Wed, 14 Aug 2024 13:15:07 +0000 (09:15 -0400)]
21910: Trust system root token even if IssueTrustedTokens == false.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months agoMerge branch '21811-side-favorites-test'
Lisa Knox [Tue, 13 Aug 2024 15:44:33 +0000 (11:44 -0400)]
Merge branch '21811-side-favorites-test'

closes #21811

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

4 months ago21811: Merge branch 'main' into 21811-side-favorites-test 21811-side-favorites-test
Lisa Knox [Tue, 13 Aug 2024 15:11:04 +0000 (11:11 -0400)]
21811: Merge branch 'main' into 21811-side-favorites-test

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

4 months ago21811: removed .only from favorites spec
Lisa Knox [Tue, 13 Aug 2024 15:04:24 +0000 (11:04 -0400)]
21811: removed .only from favorites spec

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

4 months agoMerge branch '21814-contents-paging' refs #21814
Peter Amstutz [Tue, 13 Aug 2024 14:37:10 +0000 (10:37 -0400)]
Merge branch '21814-contents-paging'  refs #21814

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago21814: Add test assertion that there are two different kinds of objects
Peter Amstutz [Tue, 13 Aug 2024 14:36:21 +0000 (10:36 -0400)]
21814: Add test assertion that there are two different kinds of objects

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago21814: Simplify getting the count and expand comment 21814-contents-paging
Peter Amstutz [Mon, 12 Aug 2024 19:25:54 +0000 (15:25 -0400)]
21814: Simplify getting the count and expand comment

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago21811:
Lisa Knox [Fri, 9 Aug 2024 14:27:16 +0000 (10:27 -0400)]
21811:

* added assertions suggested by reviewer

Arvados-DCO-1.1-Signed-off-by: Lisa Knox <lisa.knox@curii.com>

4 months agoMerge branch '22003-keep-web-and-wb2-redirect-parameter' refs #22003
Peter Amstutz [Wed, 7 Aug 2024 20:58:38 +0000 (16:58 -0400)]
Merge branch '22003-keep-web-and-wb2-redirect-parameter' refs #22003

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago21814: Remove legacy undocumented 'last_object_class' parameter
Peter Amstutz [Mon, 5 Aug 2024 21:17:52 +0000 (17:17 -0400)]
21814: Remove legacy undocumented 'last_object_class' parameter

This was only ever used by workbench 1, was never documented, and has
no tests.  The option never was included in the Go SDK
'GroupContentsOptions' struct so I believe this means it has already
been the case for years that Controller would just ignore it.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months ago21814: more efficiently handle count=none and offset > 0
Peter Amstutz [Mon, 5 Aug 2024 21:05:36 +0000 (17:05 -0400)]
21814: more efficiently handle count=none and offset > 0

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 months agoMerge branch '21927-discovery-doc-test'
Tom Clegg [Wed, 31 Jul 2024 13:59:09 +0000 (09:59 -0400)]
Merge branch '21927-discovery-doc-test'

fixes #21927

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21927: Fix race condition in test. 21927-discovery-doc-test
Tom Clegg [Tue, 30 Jul 2024 16:52:04 +0000 (12:52 -0400)]
21927: Fix race condition in test.

Test was occasionally failing because the "wait for pending reqs, then
count reqs" step ran before the background refresh had progressed far
enough to be counted as a pending req.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21927: Fix test stub not waiting for holdReqs.
Tom Clegg [Tue, 30 Jul 2024 16:43:04 +0000 (12:43 -0400)]
21927: Fix test stub not waiting for holdReqs.

"Ensure requests are handled concurrently" feature was not working for
some of the test stanzas.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months agoMerge branch '21998-zoe-translates-keep-web-directory-listing-encode-url'
Tom Clegg [Tue, 30 Jul 2024 14:30:26 +0000 (10:30 -0400)]
Merge branch '21998-zoe-translates-keep-web-directory-listing-encode-url'

fixes #21998

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months agoMerge branch '21904-no-unqueueable-reqs'
Tom Clegg [Tue, 30 Jul 2024 12:59:57 +0000 (08:59 -0400)]
Merge branch '21904-no-unqueueable-reqs'

fixes #21904

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21998: gofmt. 21998-zoe-translates-keep-web-directory-listing-encode-url
Tom Clegg [Mon, 29 Jul 2024 19:33:45 +0000 (15:33 -0400)]
21998: gofmt.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21998: Fixup comments.
Tom Clegg [Mon, 29 Jul 2024 19:17:04 +0000 (15:17 -0400)]
21998: Fixup comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21998: Use http scheme if indicated by load balancer.
Tom Clegg [Mon, 29 Jul 2024 19:16:05 +0000 (15:16 -0400)]
21998: Use http scheme if indicated by load balancer.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21910: Update documentation to new config behavior.
Tom Clegg [Mon, 29 Jul 2024 15:48:56 +0000 (11:48 -0400)]
21910: Update documentation to new config behavior.

Now, the trust/no-trust decision is made when the token is presented,
rather than implicitly via api_client_id when a token is issued.

Also fix docs to use the correct config key: it's IssueTrustedTokens,
not TrustLoginTokens.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

4 months ago21910: Merge branch 'main' into 21910-remove-api_client_id
Tom Clegg [Mon, 29 Jul 2024 15:40:39 +0000 (11:40 -0400)]
21910: Merge branch 'main' into 21910-remove-api_client_id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '21943-dir-output' refs #21943
Peter Amstutz [Fri, 26 Jul 2024 13:37:54 +0000 (09:37 -0400)]
Merge branch '21943-dir-output' refs #21943

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months agoMerge branch '21993-wf-step-cancel' refs #21993
Peter Amstutz [Thu, 25 Jul 2024 21:47:55 +0000 (17:47 -0400)]
Merge branch '21993-wf-step-cancel' refs #21993

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months ago21943: Fix bug and add integration test 21943-dir-output
Peter Amstutz [Thu, 25 Jul 2024 21:25:49 +0000 (17:25 -0400)]
21943: Fix bug and add integration test

The bug was caused by the fact that the path mapper de-duplicates file
references before doing the mapping, but also did not traverse
directory listings.  If the list of files was de-duplicated by
selecting the entries from the directory listing (given a choice of 2+
places a given file reference appears in output), but then the
directory listings were skipped later on, those files wouldn't be
included in the path map at all.

Now trims the directory listings while constructing the list of files
to path map, so if the file is referenced in directory listings in the
output, it isn't lost to the dedup behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months ago21943: Add test for files appearing multiple times in output
Peter Amstutz [Thu, 25 Jul 2024 18:23:14 +0000 (14:23 -0400)]
21943: Add test for files appearing multiple times in output

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months ago21993: Committed request with priority == 0 is treated as cancel 21993-wf-step-cancel
Peter Amstutz [Thu, 25 Jul 2024 14:44:13 +0000 (10:44 -0400)]
21993: Committed request with priority == 0 is treated as cancel

Now handles the case where a user cancels a container request before
it has a chance to run, which currently remains in the "Committed"
state rather than "Final".  This fixes the bug where the workflow
runner would be stuck forever.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months ago22003: prevent redirect loop in Wb2 client-side redirect handler 22003-keep-web-and-wb2-redirect-parameter
Zoë Ma [Mon, 22 Jul 2024 13:10:43 +0000 (21:10 +0800)]
22003: prevent redirect loop in Wb2 client-side redirect handler

Check that the target URL is not empty before setting
window.location.href (browsing to new URL), because setting it to empty
effectively reloads the page and triggers the redirect handler again,
causing an endless loop.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months ago22003: Workbench2 and keep-web: better interoperability with redirect
Zoë Ma [Fri, 12 Jul 2024 13:27:09 +0000 (21:27 +0800)]
22003: Workbench2 and keep-web: better interoperability with redirect

keep-web: When sending an unauthenticated browser client to a redirect
to Wb2, encode the target URL path in the query part of the redirection
URL in the `Location` header. This avoids possible corrupted header and
confused client.

Workbench2:

- In redirection handler, more robust handling of the input target-path
  passed by the URL query part.
- In the "copy link to clipboard" action in the files panel of a
  collection view, when creating the URL for clipboard, better emulate
  the server-generated redirect URL (see above).

Overall, when working with redirects (either generating redirect URLs or
handling them on the client side), we're better prepared for paths that
may contain special characters.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months ago21998: Try to find original request's scheme in URL generated for wget.
Zoë Ma [Tue, 23 Jul 2024 10:34:04 +0000 (18:34 +0800)]
21998: Try to find original request's scheme in URL generated for wget.

In the "wget" example generated on the directory listing page, try to
make the scheme part in the URL argument closer to the original
request's, by using the value of "X-Forwarded-Proto" header if it's
valid.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months ago21998: Simplify percent-encoding of relative URLs in directory listing
Zoë Ma [Wed, 17 Jul 2024 16:45:07 +0000 (00:45 +0800)]
21998: Simplify percent-encoding of relative URLs in directory listing

In keep-web, generate less-verbose URLs in the href values on the
directory listing page by using the percent-encoding function
appropriate for paths.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months ago21998: properly percent-encode paths in keep-web directory listing page
Zoë Ma [Thu, 11 Jul 2024 17:05:06 +0000 (01:05 +0800)]
21998: properly percent-encode paths in keep-web directory listing page

Use percent-encoded form of the relative URLs as the value of "href"
attribute in directory listing page generated by keep-web

The "wget" command example shown on the page now has single-quotes
around the URL argument.

For testing involving complex URL or path patterns, use the HTML parser
provided by golang.org/x/net/html to scrape the directory listing page,
instead of using regular expressions that may get unwieldy.

Minor edits to the HTML template for compliance and ease of testing.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months agoMerge branch '21935-clean-pysdk-api'
Brett Smith [Wed, 24 Jul 2024 21:18:35 +0000 (17:18 -0400)]
Merge branch '21935-clean-pysdk-api'

Refs #21935.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Mark internal arvados.keep classes as such 21935-clean-pysdk-api
Brett Smith [Wed, 24 Jul 2024 18:42:40 +0000 (14:42 -0400)]
21935: Mark internal arvados.keep classes as such

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Rename SafeHTTPCache to ThreadSafeHTTPCache
Brett Smith [Wed, 24 Jul 2024 18:49:41 +0000 (14:49 -0400)]
21935: Rename SafeHTTPCache to ThreadSafeHTTPCache

This mirrors the name of ThreadSafeAPIClient and clarifies what it is
"safe" for.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935/15397: Remove StreamFileReader
Brett Smith [Wed, 24 Jul 2024 18:12:36 +0000 (14:12 -0400)]
21935/15397: Remove StreamFileReader

Unused after earlier changes.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Add upgrade note about PySDK reorganization
Brett Smith [Wed, 24 Jul 2024 18:04:06 +0000 (14:04 -0400)]
21935: Add upgrade note about PySDK reorganization

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21910: Fix markup.
Tom Clegg [Wed, 24 Jul 2024 13:22:34 +0000 (09:22 -0400)]
21910: Fix markup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Fix typo.
Tom Clegg [Wed, 24 Jul 2024 13:20:54 +0000 (09:20 -0400)]
21910: Fix typo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Rescue displaced sentence.
Tom Clegg [Wed, 24 Jul 2024 13:19:00 +0000 (09:19 -0400)]
21910: Rescue displaced sentence.

Looks like the TrustLoginTokens section was accidentally inserted in
the middle of a section instead of between sections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '22005-s3-etag'
Tom Clegg [Tue, 23 Jul 2024 15:14:25 +0000 (11:14 -0400)]
Merge branch '22005-s3-etag'

fixes #22005

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago22005: Add Etag to s3 response header. 22005-s3-etag
Tom Clegg [Tue, 23 Jul 2024 12:06:43 +0000 (08:06 -0400)]
22005: Add Etag to s3 response header.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '21978-replace-files-docs'
Tom Clegg [Mon, 22 Jul 2024 18:51:44 +0000 (14:51 -0400)]
Merge branch '21978-replace-files-docs'

closes #21978

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoProvide more guidance for using arvbox the first time.
Peter Amstutz [Fri, 12 Jul 2024 15:41:57 +0000 (11:41 -0400)]
Provide more guidance for using arvbox the first time.

When arvbox is ready, now provides more information about things that
need to be done before you log in to arvbox workbench.

Also add a workbench banner with text and links to the Arvados docs,
based on the playground banner.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months ago21904: Remove LogCreateRequestFraction config. 21904-no-unqueueable-reqs
Tom Clegg [Mon, 22 Jul 2024 15:35:34 +0000 (11:35 -0400)]
21904: Remove LogCreateRequestFraction config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21978: Streamline replace_files cross-reference markup 21978-replace-files-docs
Brett Smith [Mon, 22 Jul 2024 15:29:03 +0000 (11:29 -0400)]
21978: Streamline replace_files cross-reference markup

If we're linking to the section anyway, we can use less words and not
worry about whether it's "above" or "below."

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21904: Allow queueing "create log" requests.
Tom Clegg [Mon, 22 Jul 2024 15:08:42 +0000 (11:08 -0400)]
21904: Allow queueing "create log" requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21904: Remove obsolete test stub functionality (POST .../logs).
Tom Clegg [Mon, 22 Jul 2024 15:05:51 +0000 (11:05 -0400)]
21904: Remove obsolete test stub functionality (POST .../logs).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21978: Broaden replace_files argument summary to match new features.
Tom Clegg [Mon, 22 Jul 2024 14:03:41 +0000 (10:03 -0400)]
21978: Broaden replace_files argument summary to match new features.

Also update "below" to "above" in text, to match the current location
of the replace_files section on the page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21978: Note replace_files usage restrictions re federated clusters.
Tom Clegg [Mon, 22 Jul 2024 14:00:19 +0000 (10:00 -0400)]
21978: Note replace_files usage restrictions re federated clusters.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '21907-cache-s3-token'
Brett Smith [Fri, 19 Jul 2024 19:39:31 +0000 (15:39 -0400)]
Merge branch '21907-cache-s3-token'

Closes #21907.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21907: Rename s3SecretCacheTidyInterval 21907-cache-s3-token
Brett Smith [Fri, 19 Jul 2024 17:23:41 +0000 (13:23 -0400)]
21907: Rename s3SecretCacheTidyInterval

For improved clarity.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21907: Make new functions and methods private
Brett Smith [Fri, 19 Jul 2024 17:23:22 +0000 (13:23 -0400)]
21907: Make new functions and methods private

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21907: Use an APIClientAuthorization pointer directly
Brett Smith [Fri, 19 Jul 2024 17:18:59 +0000 (13:18 -0400)]
21907: Use an APIClientAuthorization pointer directly

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21907: Lock mutex when looking up cached S3 secrets
Brett Smith [Fri, 19 Jul 2024 17:17:35 +0000 (13:17 -0400)]
21907: Lock mutex when looking up cached S3 secrets

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21907: Cache valid S3 secrets in keep-web
Brett Smith [Thu, 18 Jul 2024 20:32:47 +0000 (16:32 -0400)]
21907: Cache valid S3 secrets in keep-web

This improves performance by avoiding a call to
/arvados/v1/api_client_authorizations on each request.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months agoMerge branch '22001-run-tests-py3'
Brett Smith [Wed, 17 Jul 2024 14:05:49 +0000 (10:05 -0400)]
Merge branch '22001-run-tests-py3'

Closes #22001.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago22001: Install PySDK earlier in run-tests.sh 22001-run-tests-py3
Brett Smith [Wed, 17 Jul 2024 13:00:50 +0000 (09:00 -0400)]
22001: Install PySDK earlier in run-tests.sh

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago22001: Run arvados_version.py before `pip install`
Brett Smith [Mon, 15 Jul 2024 20:16:41 +0000 (16:16 -0400)]
22001: Run arvados_version.py before `pip install`

Otherwise `pip install` may package and try to install source that
doesn't include _version.py.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago22001: Simplify virtualenv management in run-tests
Brett Smith [Mon, 15 Jul 2024 19:12:18 +0000 (15:12 -0400)]
22001: Simplify virtualenv management in run-tests

A lot of the current virtualenv management code is parameterized,
probably to able to support virtualenvs for Python 2 and 3 in
parallel. Since we don't need that anymore, remove all this code to
simplify the script.

* initialize() calls setup_virtualenv(), which ensures the virtualenv
  exists and then activates it. Everything else in the script can assume
  this is done.
* install_env() installs the Python build tools and support libraries
  we need.
* install_deps() installs our Python modules that are used by other
  components in Arvados.

With all this, there should be no need for regular virtualenv
de/reactivation.

Various old incantations meant to work around old setuptools/pip bugs
got cleaned up as part of this too.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago22001: Remove :py3 suffix from tests
Brett Smith [Mon, 15 Jul 2024 14:39:00 +0000 (10:39 -0400)]
22001: Remove :py3 suffix from tests

We haven't supported Python 2 for a while, so requiring the user to
specify just demands extra typing for no reason.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21910: Mention removed APIs and attributes in upgrade notes.
Tom Clegg [Mon, 15 Jul 2024 13:59:30 +0000 (09:59 -0400)]
21910: Mention removed APIs and attributes in upgrade notes.

Note also covers removals from the previous API cleanup
branch (#15397).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Merge branch 'main' into 21910-remove-api_client_id
Tom Clegg [Sun, 14 Jul 2024 18:44:54 +0000 (14:44 -0400)]
21910: Merge branch 'main' into 21910-remove-api_client_id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Ignore modified_by_client_uuid in select param.
Tom Clegg [Sun, 14 Jul 2024 17:37:21 +0000 (13:37 -0400)]
21910: Ignore modified_by_client_uuid in select param.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21935: Fix docstring markup typo
Brett Smith [Fri, 12 Jul 2024 21:57:47 +0000 (17:57 -0400)]
21935: Fix docstring markup typo

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Add a docstring to arvados.commands
Brett Smith [Fri, 12 Jul 2024 20:01:47 +0000 (16:01 -0400)]
21935: Add a docstring to arvados.commands

Mostly to annotate it @private so pdoc skips rendering it.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Extract BaseDirectory classes to arvados._internal.basedirs
Brett Smith [Fri, 12 Jul 2024 19:57:44 +0000 (15:57 -0400)]
21935: Extract BaseDirectory classes to arvados._internal.basedirs

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Move arvados._pycurlhelper to arvados._internal.pycurl
Brett Smith [Fri, 12 Jul 2024 19:14:04 +0000 (15:14 -0400)]
21935: Move arvados._pycurlhelper to arvados._internal.pycurl

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Merge _normalize_stream and _ranges into arvados._internal.streams
Brett Smith [Fri, 12 Jul 2024 19:02:02 +0000 (15:02 -0400)]
21935: Merge _normalize_stream and _ranges into arvados._internal.streams

All these classes and functions are related to parsing and manipulating
streams, so it makes sense to collect them in one module with a name
reflecting that.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Move arvados.http_to_keep under arvados._internal
Brett Smith [Fri, 12 Jul 2024 18:37:12 +0000 (14:37 -0400)]
21935: Move arvados.http_to_keep under arvados._internal

This functionality could be useful to provide to people writing their
own tools. However, because of the way it was extracted out of
arvados-cwl-runner, the API wasn't really designed to be public, and has
already had one breaking change since it was introduced.

Make the current module internal so the API doesn't ossify. We can
always develop a public API for it in the future.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Move arvados.diskcache under arvados._internal
Brett Smith [Fri, 12 Jul 2024 18:23:35 +0000 (14:23 -0400)]
21935: Move arvados.diskcache under arvados._internal

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Merge arvados.timer into arvados._internal
Brett Smith [Fri, 12 Jul 2024 17:21:49 +0000 (13:21 -0400)]
21935: Merge arvados.timer into arvados._internal

It's one small class. It doesn't need a dedicated module.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Move arvados.util._deprecated to arvados._internal
Brett Smith [Fri, 12 Jul 2024 15:35:29 +0000 (11:35 -0400)]
21935: Move arvados.util._deprecated to arvados._internal

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago15397: Remove documentation about upgrading to Crunch v2
Brett Smith [Fri, 12 Jul 2024 15:20:17 +0000 (11:20 -0400)]
15397: Remove documentation about upgrading to Crunch v2

Now that the Crunch v1 jobs API has been removed, there is no need to
document how to transition away from it.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935/15397: Remove arv-migrate-docker19
Brett Smith [Fri, 12 Jul 2024 15:17:13 +0000 (11:17 -0400)]
21935/15397: Remove arv-migrate-docker19

This tool was written to aid a migration that became necessary with the
release of Docker 1.10 in February 2016. Its time has passed.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Rename safeapi.ThreadSafeApiCache to api.ThreadSafeAPIClient
Brett Smith [Fri, 12 Jul 2024 14:57:45 +0000 (10:57 -0400)]
21935: Rename safeapi.ThreadSafeApiCache to api.ThreadSafeAPIClient

Moving the class lets us delist the safeapi module from reference
documentation and streamline it.

Changing the class name better describes what it does (it doesn't
cache anything!) and follows PEP 8 case rules.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Move arvados.cache.SafeHTTPCache to arvados.api
Brett Smith [Fri, 12 Jul 2024 14:32:19 +0000 (10:32 -0400)]
21935: Move arvados.cache.SafeHTTPCache to arvados.api

This removes a module from the reference documentation while maintaining
backwards compatibility.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21935: Correct return type of arvados.api.http_cache
Brett Smith [Fri, 12 Jul 2024 14:21:06 +0000 (10:21 -0400)]
21935: Correct return type of arvados.api.http_cache

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21974: Remove CentOS discussion from Arvados 3.0 release notes
Brett Smith [Fri, 12 Jul 2024 14:03:49 +0000 (10:03 -0400)]
21974: Remove CentOS discussion from Arvados 3.0 release notes

Redundant since this support was removed in Arvados 2.7.4.
Refs #21974.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21910: Remove ModifiedByClientUUID from SDK & controller responses.
Tom Clegg [Tue, 9 Jul 2024 17:04:24 +0000 (13:04 -0400)]
21910: Remove ModifiedByClientUUID from SDK & controller responses.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Remove modified_by_client_uuid references from workbench.
Tom Clegg [Tue, 9 Jul 2024 12:09:23 +0000 (08:09 -0400)]
21910: Remove modified_by_client_uuid references from workbench.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoFix a couple of upgrade note typos.
Peter Amstutz [Mon, 8 Jul 2024 21:50:13 +0000 (17:50 -0400)]
Fix a couple of upgrade note typos.

refs #21947

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months agoSynchronize upgrade notes between 2.7 branch and main
Peter Amstutz [Mon, 8 Jul 2024 21:46:06 +0000 (17:46 -0400)]
Synchronize upgrade notes between 2.7 branch and main

refs #21947

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

5 months agoMerge branch '21989-headers'
Tom Clegg [Mon, 8 Jul 2024 15:52:49 +0000 (11:52 -0400)]
Merge branch '21989-headers'

fixes #21989

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21989: Fix order-sensitive test. 21989-headers
Tom Clegg [Mon, 8 Jul 2024 15:21:59 +0000 (11:21 -0400)]
21989: Fix order-sensitive test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21989: Keepstore: Use comma separator for X-Keep-Storage-Classes-Confirmed header
Zoë Ma [Mon, 8 Jul 2024 07:15:26 +0000 (15:15 +0800)]
21989: Keepstore: Use comma separator for X-Keep-Storage-Classes-Confirmed header

When generating the X-Keep-Storage-Classes-Confirmed header, use comma
(",") instead of semicolon (";") as the separator for multi-entry value.

This is to be consistent with keep-proxy's behavior and the arv-put
client's expectation. Plus, it follows RFC 9110 (see sec. 5.2).

Without the fix, arv-put wrongly believes that storage classes are not
supported on the cluster it is putting to. The spurious warning
"X-Keep-Storage-Classes header not supported by the cluster" can be seen
in arv-put messages and in Workbench2.

Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>

5 months ago21910: Report s3cmd version.
Tom Clegg [Thu, 4 Jul 2024 13:10:46 +0000 (09:10 -0400)]
21910: Report s3cmd version.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Revert removing dispatch2 token fixture.
Tom Clegg [Wed, 3 Jul 2024 21:01:45 +0000 (17:01 -0400)]
21910: Revert removing dispatch2 token fixture.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Fix test fixtures.
Tom Clegg [Wed, 3 Jul 2024 20:10:47 +0000 (16:10 -0400)]
21910: Fix test fixtures.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '21931-public-suffix-pin'
Brett Smith [Wed, 3 Jul 2024 18:43:23 +0000 (14:43 -0400)]
Merge branch '21931-public-suffix-pin'

Refs #21931.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21910: Remove uses of removed dispatch1 token fixture.
Tom Clegg [Wed, 3 Jul 2024 18:39:25 +0000 (14:39 -0400)]
21910: Remove uses of removed dispatch1 token fixture.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Remove more uses of api_clients API.
Tom Clegg [Wed, 3 Jul 2024 18:38:59 +0000 (14:38 -0400)]
21910: Remove more uses of api_clients API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Remove more uses of api_clients API.
Tom Clegg [Wed, 3 Jul 2024 16:33:34 +0000 (12:33 -0400)]
21910: Remove more uses of api_clients API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months agoMerge branch '21701-replace-files-current-version'
Tom Clegg [Wed, 3 Jul 2024 13:38:08 +0000 (09:38 -0400)]
Merge branch '21701-replace-files-current-version'

refs #21701

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Remove create_superuser_token script.
Tom Clegg [Wed, 3 Jul 2024 11:46:59 +0000 (07:46 -0400)]
21910: Remove create_superuser_token script.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21910: Remove ApiClient model.
Tom Clegg [Fri, 28 Jun 2024 19:47:34 +0000 (15:47 -0400)]
21910: Remove ApiClient model.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

5 months ago21931: Revert net-imap dependency 21931-public-suffix-pin
Brett Smith [Tue, 2 Jul 2024 17:45:08 +0000 (13:45 -0400)]
21931: Revert net-imap dependency

This was updated as part of #21933 in
9167b2053ef2a39a4e8a7c13e4127e12470b14ed.
However, net-imap 0.4 requires Ruby > 2.7.3, while Ubuntu 20.04 only
ships with Ruby 2.7.0. Upgrading this gem doesn't seem to be required
for the new Rails release, so don't.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months ago21931: Narrow public_suffix dependency for arvados-google-api-client gem
Brett Smith [Tue, 2 Jul 2024 16:09:40 +0000 (12:09 -0400)]
21931: Narrow public_suffix dependency for arvados-google-api-client gem

For the usual reason: to force `gem install` to resolve dependencies in
a way that works on Ruby 2.7.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

5 months agoMerge branch 'jszlenk/create_new_subproject' refs #21937
Peter Amstutz [Tue, 2 Jul 2024 17:18:59 +0000 (13:18 -0400)]
Merge branch 'jszlenk/create_new_subproject' refs #21937

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>