X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/9e674938db7e1a92497ffb0cc97cfb92ea37a885..e20590d485505f58f7745d74a311ca539c9be940:/doc/sdk/python/cookbook.html.textile.liquid diff --git a/doc/sdk/python/cookbook.html.textile.liquid b/doc/sdk/python/cookbook.html.textile.liquid index 22aa711db6..f2d087625e 100644 --- a/doc/sdk/python/cookbook.html.textile.liquid +++ b/doc/sdk/python/cookbook.html.textile.liquid @@ -50,6 +50,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0 # "Working with the container request queue":#working-with-container-request-queue ## "List completed container requests":#list-completed-container-requests ## "Cancel a container request":#cancel-a-container-request +## "Cancel multiple pending container requests":#cancel-all-container-requests h2(#introduction). Introduction @@ -84,7 +85,8 @@ The API provides a "dedicated groups method named @shared@":{{ site.baseurl }}/a {% codeblock as python %} for item in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.groups().shared, # Pass filters to limit what objects are returned. # This example returns only subprojects. @@ -129,7 +131,8 @@ The API provides a "dedicated groups method named @contents@":{{ site.baseurl }} {% codeblock as python %} current_user = arv_client.users().current().execute() for item in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.groups().contents, # The UUID of the project whose contents we're listing. # Pass a user UUID to list their home project. @@ -159,7 +162,7 @@ In brief, a permission is represented in Arvados as a link object with the follo * @tail_uuid@ identifies the user or role group that receives the permission. * @head_uuid@ identifies the Arvados object this permission grants access to. -For details, refer to the "Permissions model documentation":{{ site.baseurl }}/api/permission-model.html. Managing permissions is just a matter of ensuring the desired links exist with the standard @create@, @update@, and @delete@ methods. +For details, refer to the "Permissions model documentation":{{ site.baseurl }}/api/permission-model.html. Managing permissions is just a matter of ensuring the desired links exist using the standard @create@, @update@, and @delete@ methods. h3(#grant-permission). Grant permission to an object @@ -188,7 +191,8 @@ To modify an existing permission—for example, to change its access level—fin {% codeblock as python %} import arvados.util for permission in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.links().list, filters=[ # You should use this filter for all permission searches, @@ -217,7 +221,8 @@ To revoke an existing permission, find the existing link object for the permissi {% codeblock as python %} import arvados.util for permission in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.links().list, filters=[ # You should use this filter for all permission searches, @@ -391,44 +396,44 @@ with collection.open('ExampleFile') as my_file: h3(#download-a-file-from-a-collection). Download a file from a collection -Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a source to Python's standard "@shutil.copyfileobj@ function":https://docs.python.org/3/library/shutil.html#shutil.copyfileobj to download it. This code downloads @ExampleFile@ from your collection and saves it to the current working directory as @ExampleDownload@: +Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. You pass a second mode argument like @'rb'@ to open the file in binary mode. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a source to Python's standard "@shutil.copyfileobj@ function":https://docs.python.org/3/library/shutil.html#shutil.copyfileobj to download it. This code downloads @ExampleFile@ from your collection and saves it to the current working directory as @ExampleDownload@: {% codeblock as python %} import arvados.collection import shutil collection = arvados.collection.Collection(...) with ( - collection.open('ExampleFile') as src_file, - open('ExampleDownload', 'w') as dst_file, + collection.open('ExampleFile', 'rb') as src_file, + open('ExampleDownload', 'wb') as dst_file, ): shutil.copyfileobj(src_file, dst_file) {% endcodeblock %} h3(#write-a-file-into-a-new-collection). Write a file to a collection -Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. Pass a second mode argument like @'w'@ or @'a'@ to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. This example writes @Hello, Arvados!@ to a file named @ExampleHello@ in your collection: +Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. Pass a second mode argument like @'w'@, @'a'@, or @'wb'@ to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. This example writes @Hello, Arvados!@ to a file named @ExampleHello@ in your collection: {% codeblock as python %} import arvados.collection collection = arvados.collection.Collection(...) with collection.open('ExampleFile', 'w') as my_file: # Write to my_file as desired. - # This example writes "Hello, world!" to the file. + # This example writes "Hello, Arvados!" to the file. print("Hello, Arvados!", file=my_file) collection.save_new(...) # or collection.save() to update an existing collection {% endcodeblock %} h3(#upload-a-file-into-a-new-collection). Upload a file to a collection -Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. Pass a second mode argument like @'w'@ or @'a'@ to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a destination to Python's standard "@shutil.copyfileobj@ function":https://docs.python.org/3/library/shutil.html#shutil.copyfileobj to upload data from a source file. This example reads @ExampleFile@ from the current working directory and uploads it into your collection as @ExampleUpload@: +Once you have a @Collection@ object, the "@Collection.open@ method":{{ site.baseurl }}/sdk/python/arvados/collection.html#arvados.collection.RichCollectionBase.open lets you open files from a collection the same way you would open files from disk using Python's built-in @open@ function. Pass a second mode argument like @'w'@, @'a'@, or @'wb'@ to write a file in the collection. It returns a file-like object that you can use in many of the same ways you would use any other file object. You can pass it as a destination to Python's standard "@shutil.copyfileobj@ function":https://docs.python.org/3/library/shutil.html#shutil.copyfileobj to upload data from a source file. This example reads @ExampleFile@ from the current working directory and uploads it into your collection as @ExampleUpload@: {% codeblock as python %} import arvados.collection import shutil collection = arvados.collection.Collection(...) with ( - open('ExampleFile') as src_file, - collection.open('ExampleUpload', 'w') as dst_file, + open('ExampleFile', 'rb') as src_file, + collection.open('ExampleUpload', 'wb') as dst_file, ): shutil.copyfileobj(src_file, dst_file) collection.save_new(...) # or collection.save() to update an existing collection @@ -611,7 +616,7 @@ for mount_name, mount_source in container_request['mounts'].items(): pprint.pprint(mount_source.get('content')) {% endcodeblock %} -h3(#get-input-of-a-cwl-workflow). Get input of a container or CWL workflow run +h3(#get-input-of-a-cwl-workflow). Get input of a CWL workflow run When you run a CWL workflow, the CWL inputs are stored in the container request's @mounts@ field as a JSON mount named @/var/lib/cwl/cwl.input.json@. @@ -714,7 +719,8 @@ When a running container creates a container request to do additional work, the {% codeblock as python %} import arvados.util for child_container_requests in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.container_requests().list, filters=[ # Note this is a container UUID, *not* a container request UUID @@ -743,7 +749,8 @@ if parent_container_uuid is None: child_container_requests = () else: child_container_requests = arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.container_requests().list, filters=[ ['requesting_container_uuid', '=', parent_container_uuid], @@ -772,7 +779,8 @@ time_filter = datetime.datetime.utcnow() time_filter -= datetime.timedelta(days=7) for container_request in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.container_requests().list, filters=[ # This is the filter you need to find completed container requests. @@ -810,12 +818,15 @@ cancelled_container_request = arv_client.container_requests().update( ).execute() {% endcodeblock %} -p(#cancel-all-container-requests). If you want to cancel many container requests, you can list container requests with the @state@ field set to @"Committed"@, a @priority@ greater than zero, and any other filters you like. Then update each container request in turn. +h3(#cancel-all-container-requests). Cancel multiple pending container requests + +If you want to cancel multiple pending container requests, you can list container requests with the @state@ field set to @"Committed"@, a @priority@ greater than zero, and any other filters you like. Then update each container request to set its @priority@ field to 0. See the "containers API reference":{{ site.baseurl }}/api/methods/containers.html for details. {% codeblock as python %} import arvados.util for container_request in arvados.util.keyset_list_all( - # Do *not* call the method here, just pass it. + # Pass the method keyset_list_all will call to retrieve items. + # Do not call it yourself. arv_client.container_requests().list, filters=[ # These are the filters you need to find cancellable container requests.