Brett Smith [Thu, 2 Feb 2023 15:33:32 +0000 (10:33 -0500)]
19926: Flatten structure of PySDK install instructions
The way there were "two tracks" of install and test instructions on the
page was confusing to readers. (Especially since we did not number those
options consistently.) Incorporate the test instructions into the
install instructions, so the reader only has to follow and care about
one specific method, and then they're done.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 1 Feb 2023 14:10:33 +0000 (09:10 -0500)]
19792: Use binary mode in cookbook download+upload examples
This incorporates a suggestion from review to use a binary mode in a way
that makes the recipes more robust. It's very likely that readers will
want to work with binary files at least as much as text files, and these
recipes still work for text files too for verbatim copies.
Refs #19792.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 30 Jan 2023 23:38:02 +0000 (18:38 -0500)]
19792: Add 'wb' as an example Collection.open mode argument
Give readers a hint that we fully support the mode argument, not just
the basic r/w/a options. Ideally they can look up the reference for full
details.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 30 Jan 2023 23:29:47 +0000 (18:29 -0500)]
19897: Add tests for writing immutable mount objects
I originally wrote these expecting to get them to return ENOTSUP as
well. However, that doesn't happen now because of how the Operations
class calls Directory.writable() and turns False into EPERM. I'm
committing the tests since they're useful as-is, with a note that I'd
like to convert them to ENOTSUP in the future.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 30 Jan 2023 23:24:12 +0000 (18:24 -0500)]
19897: Return ENOTSUP when the user tries to make a file in a project
Before this commit, the NotImplementedError is propagating up to
catch_exceptions, which turns it into EIO. "I/O error" sounds like a
problem with the underlying systems and stresses out users. "Operation
not supported" should better communicate to the user that they're doing
something unexpected and should double-check their work.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sun, 29 Jan 2023 18:28:51 +0000 (13:28 -0500)]
19792: Add PySDK cookbook recipe to walk a collection
After working on the container request recipes, I realized this was a
gap in the cookbook: how do you work on a collection if you don't know
its structure? This starts to document that.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 2 Jan 2023 20:27:07 +0000 (15:27 -0500)]
19792: Revamp the Python SDK cookbook
* Organize the recipes into related sections based on what they work
on.
* Add a preface to each recipe to explain when you might use it, give
API background, etc.
* Write the recipes in a style that emphasizes readability with long
variable names, lots of line breaks, and comments highlighting
key mechanics and variations.
* Modernize SDK usage in the recipes.
* Add links to supporting documentation where available.
As much as practical, I have maintained existing anchor names, even when
I updated header names for more consistent style.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 20 Jan 2023 16:21:28 +0000 (11:21 -0500)]
19886: Describe what component updates container logs more generically
We want to make clear that an Arvados system component does this,
without being overly committal about which component it is. "Dispatcher"
might be taken to mean the Crunch dispatcher specifically. Right now
it's crunch-run, but we don't need to commit to that level of
detail. Settle on "Crunch system" per discussion with Tom in chat.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 20 Jan 2023 14:11:59 +0000 (09:11 -0500)]
19886: Slim down API documentation about container logs
While the API server will let you set this field to a UUID, that
behavior is already poorly supported, and on reflection we're not sure
we want to support it. Document this the way crunch-run uses it and
avoid presenting too much detail for most readers.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 19 Jan 2023 20:39:29 +0000 (15:39 -0500)]
19917: Synthesize scheduling parameters when retrying a container
When we retry a cancelled container, there may be any number of
container requests that want it, each with their own scheduling
parameters. Create a new set of scheduling parameters for the new
container where each parameter has the most lax setting from all the
outstanding container requests.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 18 Jan 2023 19:51:05 +0000 (14:51 -0500)]
19886: Update API documentation notes about container request logs
The existing note was already out-of-date: crunch-run would record logs
about 30 minutes after starting the container. With recent changes,
crunch-run now records logs shortly after starting the container. Update
the note accordingly.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 18 Jan 2023 19:44:55 +0000 (14:44 -0500)]
19886: Correct API documentation notes about container logs
The documented limitation does not exist. In fact, there's a limitation
in the API server that if you want a container's logs propagated to its
associated container request(s), you *must* pass in a portable data
hash. crunch-run consistently updates container records with a portable
data hash in the log field for this reason. Update the note to reflect
this.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 18 Jan 2023 19:35:58 +0000 (14:35 -0500)]
19886: crunch-run records initial log with PDH
The API server will only propagate a container's log collection to
container requests when it is specified with a portable data hash. See
the top of ContainerRequest#update_collections.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
projects / arvados.git / log