--- /dev/null
--- /dev/null
++/.gradle/
++/bin/
++/build/
++.project
++.classpath
++/.settings/
++.DS_Store
++/.idea/
++/out/
--- /dev/null
--- /dev/null
++.licenseignore
++agpl-3.0.txt
++apache-2.0.txt
++COPYING
--- /dev/null
--- /dev/null
++Unless indicated otherwise in the header of the file, the files in this
++repository are dual-licensed AGPL-3.0 and Apache-2.0
++
++Individual files contain an SPDX tag that indicates the license for the file.
++dual-licensed files use the following tag:
++
++ SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
++
++This enables machine processing of license information based on the SPDX
++License Identifiers that are available here: http://spdx.org/licenses/
++
++The full license text for each license is available in this directory:
++
++ AGPL-3.0: agpl-3.0.txt
++ Apache-2.0: apache-2.0.txt
--- /dev/null
--- /dev/null
++```
++Copyright (C) The Arvados Authors. All rights reserved.
++
++SPDX-License-Identifier: CC-BY-SA-3.0
++```
++
++# Arvados Java SDK
++
++##### About
++Arvados Java Client allows to access Arvados servers and uses two APIs:
++* lower level [Keep Server API](https://doc.arvados.org/api/index.html)
++* higher level [Keep-Web API](https://godoc.org/github.com/curoverse/arvados/services/keep-web) (when needed)
++
++##### Required Java version
++This SDK requires Java 8+
++
++##### Logging
++
++SLF4J is used for logging. Concrete logging framework and configuration must be provided by a client.
++
++##### Configuration
++
++[TypeSafe Configuration](https://github.com/lightbend/config) is used for configuring this library.
++
++Please, have a look at java/resources/reference.conf for default values provided with this library.
++
++* **keepweb-host** - change to host of your Keep-Web installation
++* **keepweb-port** - change to port of your Keep-Web installation
++* **host** - change to host of your Arvados installation
++* **port** - change to port of your Arvados installation
++* **token** - authenticates registered user, one must provide
++ [token obtained from Arvados Workbench](https://doc.arvados.org/user/reference/api-tokens.html)
++* **protocol** - don't change to unless really needed
++* **host-insecure** - insecure communication with Arvados (ignores SSL certificate verification),
++ don't change to *true* unless really needed
++* **split-size** - size of chunk files in megabytes
++* **temp-dir** - temporary chunk files storage
++* **copies** - amount of chunk files duplicates per Keep server
++* **retries** - in case of chunk files send failure this should allow to repeat send
++ (*NOTE*: this parameter is not used at the moment but was left for future improvements)
++
++In order to override default settings one can create application.conf file in an application.
++Example: src/test/resources/application.conf.
++
++Alternatively ExternalConfigProvider class can be used to pass configuration via code.
++ExternalConfigProvider comes with a builder and all of the above values must be provided in order for it to work properly.
++
++ArvadosFacade has two constructors, one without arguments that uses values from reference.conf and second one
++taking ExternalConfigProvider as an argument.
++
++##### API clients
++
++All API clients inherit from BaseStandardApiClient. This class contains implementation of all
++common methods as described in http://doc.arvados.org/api/methods.html.
++
++Parameters provided to common or specific methods are String UUID or fields wrapped in Java objects. For example:
++
++```java
++String uuid = "ardev-4zz18-rxcql7qwyakg1r1";
++
++Collection actual = client.get(uuid);
++```
++
++```java
++ListArgument listArgument = ListArgument.builder()
++ .filters(Arrays.asList(
++ Filter.of("owner_uuid", Operator.LIKE, "ardev%"),
++ Filter.of("name", Operator.LIKE, "Super%"),
++ Filter.of("portable_data_hash", Operator.IN, Lists.newArrayList("54f6d9f59065d3c009d4306660989379+65")
++ )))
++ .build();
++
++CollectionList actual = client.list(listArgument);
++```
++
++Non-standard API clients must inherit from BaseApiClient.
++For example: KeepServerApiClient communicates directly with Keep servers using exclusively non-common methods.
++
++##### Business logic
++
++More advanced API data handling could be implemented as *Facade* classes.
++In current version functionalities provided by SDK are handled by *ArvadosFacade*.
++They include:
++* **downloading single file from collection** - using Keep-Web
++* **downloading whole collection** - using Keep-Web or Keep Server API
++* **listing file info from certain collection** - information is returned as list of *FileTokens* providing file details
++* **uploading single file** - to either new or existing collection
++* **uploading list of files** - to either new or existing collection
++* **creating an empty collection**
++* **getting current user info**
++* **listing current user's collections**
++* **creating new project**
++* **deleting certain collection**
++
++##### Note regarding Keep-Web
++
++Current version requires both Keep Web and standard Keep Server API configured in order to use Keep-Web functionalities.
++
++##### Integration tests
++
++In order to run integration tests all fields within following configuration file must be provided:
++```java
++src/test/resources/integration-test-appliation.conf
++```
++Parameter **integration-tests.project-uuid** should contain UUID of one project available to user,
++whose token was provided within configuration file.
++
++Integration tests require connection to real Arvados server.
++
++##### Note regarding file naming
++
++While uploading via this SDK all uploaded files within single collection must have different names.
++This applies also to uploading files to already existing collection.
++Renaming files with duplicate names is not implemented in current version.
++
--- /dev/null
--- /dev/null
++ GNU AFFERO GENERAL PUBLIC LICENSE
++ Version 3, 19 November 2007
++
++ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
++ Everyone is permitted to copy and distribute verbatim copies
++ of this license document, but changing it is not allowed.
++
++ Preamble
++
++ The GNU Affero General Public License is a free, copyleft license for
++software and other kinds of works, specifically designed to ensure
++cooperation with the community in the case of network server software.
++
++ The licenses for most software and other practical works are designed
++to take away your freedom to share and change the works. By contrast,
++our General Public Licenses are intended to guarantee your freedom to
++share and change all versions of a program--to make sure it remains free
++software for all its users.
++
++ When we speak of free software, we are referring to freedom, not
++price. Our General Public Licenses are designed to make sure that you
++have the freedom to distribute copies of free software (and charge for
++them if you wish), that you receive source code or can get it if you
++want it, that you can change the software or use pieces of it in new
++free programs, and that you know you can do these things.
++
++ Developers that use our General Public Licenses protect your rights
++with two steps: (1) assert copyright on the software, and (2) offer
++you this License which gives you legal permission to copy, distribute
++and/or modify the software.
++
++ A secondary benefit of defending all users' freedom is that
++improvements made in alternate versions of the program, if they
++receive widespread use, become available for other developers to
++incorporate. Many developers of free software are heartened and
++encouraged by the resulting cooperation. However, in the case of
++software used on network servers, this result may fail to come about.
++The GNU General Public License permits making a modified version and
++letting the public access it on a server without ever releasing its
++source code to the public.
++
++ The GNU Affero General Public License is designed specifically to
++ensure that, in such cases, the modified source code becomes available
++to the community. It requires the operator of a network server to
++provide the source code of the modified version running there to the
++users of that server. Therefore, public use of a modified version, on
++a publicly accessible server, gives the public access to the source
++code of the modified version.
++
++ An older license, called the Affero General Public License and
++published by Affero, was designed to accomplish similar goals. This is
++a different license, not a version of the Affero GPL, but Affero has
++released a new version of the Affero GPL which permits relicensing under
++this license.
++
++ The precise terms and conditions for copying, distribution and
++modification follow.
++
++ TERMS AND CONDITIONS
++
++ 0. Definitions.
++
++ "This License" refers to version 3 of the GNU Affero General Public License.
++
++ "Copyright" also means copyright-like laws that apply to other kinds of
++works, such as semiconductor masks.
++
++ "The Program" refers to any copyrightable work licensed under this
++License. Each licensee is addressed as "you". "Licensees" and
++"recipients" may be individuals or organizations.
++
++ To "modify" a work means to copy from or adapt all or part of the work
++in a fashion requiring copyright permission, other than the making of an
++exact copy. The resulting work is called a "modified version" of the
++earlier work or a work "based on" the earlier work.
++
++ A "covered work" means either the unmodified Program or a work based
++on the Program.
++
++ To "propagate" a work means to do anything with it that, without
++permission, would make you directly or secondarily liable for
++infringement under applicable copyright law, except executing it on a
++computer or modifying a private copy. Propagation includes copying,
++distribution (with or without modification), making available to the
++public, and in some countries other activities as well.
++
++ To "convey" a work means any kind of propagation that enables other
++parties to make or receive copies. Mere interaction with a user through
++a computer network, with no transfer of a copy, is not conveying.
++
++ An interactive user interface displays "Appropriate Legal Notices"
++to the extent that it includes a convenient and prominently visible
++feature that (1) displays an appropriate copyright notice, and (2)
++tells the user that there is no warranty for the work (except to the
++extent that warranties are provided), that licensees may convey the
++work under this License, and how to view a copy of this License. If
++the interface presents a list of user commands or options, such as a
++menu, a prominent item in the list meets this criterion.
++
++ 1. Source Code.
++
++ The "source code" for a work means the preferred form of the work
++for making modifications to it. "Object code" means any non-source
++form of a work.
++
++ A "Standard Interface" means an interface that either is an official
++standard defined by a recognized standards body, or, in the case of
++interfaces specified for a particular programming language, one that
++is widely used among developers working in that language.
++
++ The "System Libraries" of an executable work include anything, other
++than the work as a whole, that (a) is included in the normal form of
++packaging a Major Component, but which is not part of that Major
++Component, and (b) serves only to enable use of the work with that
++Major Component, or to implement a Standard Interface for which an
++implementation is available to the public in source code form. A
++"Major Component", in this context, means a major essential component
++(kernel, window system, and so on) of the specific operating system
++(if any) on which the executable work runs, or a compiler used to
++produce the work, or an object code interpreter used to run it.
++
++ The "Corresponding Source" for a work in object code form means all
++the source code needed to generate, install, and (for an executable
++work) run the object code and to modify the work, including scripts to
++control those activities. However, it does not include the work's
++System Libraries, or general-purpose tools or generally available free
++programs which are used unmodified in performing those activities but
++which are not part of the work. For example, Corresponding Source
++includes interface definition files associated with source files for
++the work, and the source code for shared libraries and dynamically
++linked subprograms that the work is specifically designed to require,
++such as by intimate data communication or control flow between those
++subprograms and other parts of the work.
++
++ The Corresponding Source need not include anything that users
++can regenerate automatically from other parts of the Corresponding
++Source.
++
++ The Corresponding Source for a work in source code form is that
++same work.
++
++ 2. Basic Permissions.
++
++ All rights granted under this License are granted for the term of
++copyright on the Program, and are irrevocable provided the stated
++conditions are met. This License explicitly affirms your unlimited
++permission to run the unmodified Program. The output from running a
++covered work is covered by this License only if the output, given its
++content, constitutes a covered work. This License acknowledges your
++rights of fair use or other equivalent, as provided by copyright law.
++
++ You may make, run and propagate covered works that you do not
++convey, without conditions so long as your license otherwise remains
++in force. You may convey covered works to others for the sole purpose
++of having them make modifications exclusively for you, or provide you
++with facilities for running those works, provided that you comply with
++the terms of this License in conveying all material for which you do
++not control copyright. Those thus making or running the covered works
++for you must do so exclusively on your behalf, under your direction
++and control, on terms that prohibit them from making any copies of
++your copyrighted material outside their relationship with you.
++
++ Conveying under any other circumstances is permitted solely under
++the conditions stated below. Sublicensing is not allowed; section 10
++makes it unnecessary.
++
++ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
++
++ No covered work shall be deemed part of an effective technological
++measure under any applicable law fulfilling obligations under article
++11 of the WIPO copyright treaty adopted on 20 December 1996, or
++similar laws prohibiting or restricting circumvention of such
++measures.
++
++ When you convey a covered work, you waive any legal power to forbid
++circumvention of technological measures to the extent such circumvention
++is effected by exercising rights under this License with respect to
++the covered work, and you disclaim any intention to limit operation or
++modification of the work as a means of enforcing, against the work's
++users, your or third parties' legal rights to forbid circumvention of
++technological measures.
++
++ 4. Conveying Verbatim Copies.
++
++ You may convey verbatim copies of the Program's source code as you
++receive it, in any medium, provided that you conspicuously and
++appropriately publish on each copy an appropriate copyright notice;
++keep intact all notices stating that this License and any
++non-permissive terms added in accord with section 7 apply to the code;
++keep intact all notices of the absence of any warranty; and give all
++recipients a copy of this License along with the Program.
++
++ You may charge any price or no price for each copy that you convey,
++and you may offer support or warranty protection for a fee.
++
++ 5. Conveying Modified Source Versions.
++
++ You may convey a work based on the Program, or the modifications to
++produce it from the Program, in the form of source code under the
++terms of section 4, provided that you also meet all of these conditions:
++
++ a) The work must carry prominent notices stating that you modified
++ it, and giving a relevant date.
++
++ b) The work must carry prominent notices stating that it is
++ released under this License and any conditions added under section
++ 7. This requirement modifies the requirement in section 4 to
++ "keep intact all notices".
++
++ c) You must license the entire work, as a whole, under this
++ License to anyone who comes into possession of a copy. This
++ License will therefore apply, along with any applicable section 7
++ additional terms, to the whole of the work, and all its parts,
++ regardless of how they are packaged. This License gives no
++ permission to license the work in any other way, but it does not
++ invalidate such permission if you have separately received it.
++
++ d) If the work has interactive user interfaces, each must display
++ Appropriate Legal Notices; however, if the Program has interactive
++ interfaces that do not display Appropriate Legal Notices, your
++ work need not make them do so.
++
++ A compilation of a covered work with other separate and independent
++works, which are not by their nature extensions of the covered work,
++and which are not combined with it such as to form a larger program,
++in or on a volume of a storage or distribution medium, is called an
++"aggregate" if the compilation and its resulting copyright are not
++used to limit the access or legal rights of the compilation's users
++beyond what the individual works permit. Inclusion of a covered work
++in an aggregate does not cause this License to apply to the other
++parts of the aggregate.
++
++ 6. Conveying Non-Source Forms.
++
++ You may convey a covered work in object code form under the terms
++of sections 4 and 5, provided that you also convey the
++machine-readable Corresponding Source under the terms of this License,
++in one of these ways:
++
++ a) Convey the object code in, or embodied in, a physical product
++ (including a physical distribution medium), accompanied by the
++ Corresponding Source fixed on a durable physical medium
++ customarily used for software interchange.
++
++ b) Convey the object code in, or embodied in, a physical product
++ (including a physical distribution medium), accompanied by a
++ written offer, valid for at least three years and valid for as
++ long as you offer spare parts or customer support for that product
++ model, to give anyone who possesses the object code either (1) a
++ copy of the Corresponding Source for all the software in the
++ product that is covered by this License, on a durable physical
++ medium customarily used for software interchange, for a price no
++ more than your reasonable cost of physically performing this
++ conveying of source, or (2) access to copy the
++ Corresponding Source from a network server at no charge.
++
++ c) Convey individual copies of the object code with a copy of the
++ written offer to provide the Corresponding Source. This
++ alternative is allowed only occasionally and noncommercially, and
++ only if you received the object code with such an offer, in accord
++ with subsection 6b.
++
++ d) Convey the object code by offering access from a designated
++ place (gratis or for a charge), and offer equivalent access to the
++ Corresponding Source in the same way through the same place at no
++ further charge. You need not require recipients to copy the
++ Corresponding Source along with the object code. If the place to
++ copy the object code is a network server, the Corresponding Source
++ may be on a different server (operated by you or a third party)
++ that supports equivalent copying facilities, provided you maintain
++ clear directions next to the object code saying where to find the
++ Corresponding Source. Regardless of what server hosts the
++ Corresponding Source, you remain obligated to ensure that it is
++ available for as long as needed to satisfy these requirements.
++
++ e) Convey the object code using peer-to-peer transmission, provided
++ you inform other peers where the object code and Corresponding
++ Source of the work are being offered to the general public at no
++ charge under subsection 6d.
++
++ A separable portion of the object code, whose source code is excluded
++from the Corresponding Source as a System Library, need not be
++included in conveying the object code work.
++
++ A "User Product" is either (1) a "consumer product", which means any
++tangible personal property which is normally used for personal, family,
++or household purposes, or (2) anything designed or sold for incorporation
++into a dwelling. In determining whether a product is a consumer product,
++doubtful cases shall be resolved in favor of coverage. For a particular
++product received by a particular user, "normally used" refers to a
++typical or common use of that class of product, regardless of the status
++of the particular user or of the way in which the particular user
++actually uses, or expects or is expected to use, the product. A product
++is a consumer product regardless of whether the product has substantial
++commercial, industrial or non-consumer uses, unless such uses represent
++the only significant mode of use of the product.
++
++ "Installation Information" for a User Product means any methods,
++procedures, authorization keys, or other information required to install
++and execute modified versions of a covered work in that User Product from
++a modified version of its Corresponding Source. The information must
++suffice to ensure that the continued functioning of the modified object
++code is in no case prevented or interfered with solely because
++modification has been made.
++
++ If you convey an object code work under this section in, or with, or
++specifically for use in, a User Product, and the conveying occurs as
++part of a transaction in which the right of possession and use of the
++User Product is transferred to the recipient in perpetuity or for a
++fixed term (regardless of how the transaction is characterized), the
++Corresponding Source conveyed under this section must be accompanied
++by the Installation Information. But this requirement does not apply
++if neither you nor any third party retains the ability to install
++modified object code on the User Product (for example, the work has
++been installed in ROM).
++
++ The requirement to provide Installation Information does not include a
++requirement to continue to provide support service, warranty, or updates
++for a work that has been modified or installed by the recipient, or for
++the User Product in which it has been modified or installed. Access to a
++network may be denied when the modification itself materially and
++adversely affects the operation of the network or violates the rules and
++protocols for communication across the network.
++
++ Corresponding Source conveyed, and Installation Information provided,
++in accord with this section must be in a format that is publicly
++documented (and with an implementation available to the public in
++source code form), and must require no special password or key for
++unpacking, reading or copying.
++
++ 7. Additional Terms.
++
++ "Additional permissions" are terms that supplement the terms of this
++License by making exceptions from one or more of its conditions.
++Additional permissions that are applicable to the entire Program shall
++be treated as though they were included in this License, to the extent
++that they are valid under applicable law. If additional permissions
++apply only to part of the Program, that part may be used separately
++under those permissions, but the entire Program remains governed by
++this License without regard to the additional permissions.
++
++ When you convey a copy of a covered work, you may at your option
++remove any additional permissions from that copy, or from any part of
++it. (Additional permissions may be written to require their own
++removal in certain cases when you modify the work.) You may place
++additional permissions on material, added by you to a covered work,
++for which you have or can give appropriate copyright permission.
++
++ Notwithstanding any other provision of this License, for material you
++add to a covered work, you may (if authorized by the copyright holders of
++that material) supplement the terms of this License with terms:
++
++ a) Disclaiming warranty or limiting liability differently from the
++ terms of sections 15 and 16 of this License; or
++
++ b) Requiring preservation of specified reasonable legal notices or
++ author attributions in that material or in the Appropriate Legal
++ Notices displayed by works containing it; or
++
++ c) Prohibiting misrepresentation of the origin of that material, or
++ requiring that modified versions of such material be marked in
++ reasonable ways as different from the original version; or
++
++ d) Limiting the use for publicity purposes of names of licensors or
++ authors of the material; or
++
++ e) Declining to grant rights under trademark law for use of some
++ trade names, trademarks, or service marks; or
++
++ f) Requiring indemnification of licensors and authors of that
++ material by anyone who conveys the material (or modified versions of
++ it) with contractual assumptions of liability to the recipient, for
++ any liability that these contractual assumptions directly impose on
++ those licensors and authors.
++
++ All other non-permissive additional terms are considered "further
++restrictions" within the meaning of section 10. If the Program as you
++received it, or any part of it, contains a notice stating that it is
++governed by this License along with a term that is a further
++restriction, you may remove that term. If a license document contains
++a further restriction but permits relicensing or conveying under this
++License, you may add to a covered work material governed by the terms
++of that license document, provided that the further restriction does
++not survive such relicensing or conveying.
++
++ If you add terms to a covered work in accord with this section, you
++must place, in the relevant source files, a statement of the
++additional terms that apply to those files, or a notice indicating
++where to find the applicable terms.
++
++ Additional terms, permissive or non-permissive, may be stated in the
++form of a separately written license, or stated as exceptions;
++the above requirements apply either way.
++
++ 8. Termination.
++
++ You may not propagate or modify a covered work except as expressly
++provided under this License. Any attempt otherwise to propagate or
++modify it is void, and will automatically terminate your rights under
++this License (including any patent licenses granted under the third
++paragraph of section 11).
++
++ However, if you cease all violation of this License, then your
++license from a particular copyright holder is reinstated (a)
++provisionally, unless and until the copyright holder explicitly and
++finally terminates your license, and (b) permanently, if the copyright
++holder fails to notify you of the violation by some reasonable means
++prior to 60 days after the cessation.
++
++ Moreover, your license from a particular copyright holder is
++reinstated permanently if the copyright holder notifies you of the
++violation by some reasonable means, this is the first time you have
++received notice of violation of this License (for any work) from that
++copyright holder, and you cure the violation prior to 30 days after
++your receipt of the notice.
++
++ Termination of your rights under this section does not terminate the
++licenses of parties who have received copies or rights from you under
++this License. If your rights have been terminated and not permanently
++reinstated, you do not qualify to receive new licenses for the same
++material under section 10.
++
++ 9. Acceptance Not Required for Having Copies.
++
++ You are not required to accept this License in order to receive or
++run a copy of the Program. Ancillary propagation of a covered work
++occurring solely as a consequence of using peer-to-peer transmission
++to receive a copy likewise does not require acceptance. However,
++nothing other than this License grants you permission to propagate or
++modify any covered work. These actions infringe copyright if you do
++not accept this License. Therefore, by modifying or propagating a
++covered work, you indicate your acceptance of this License to do so.
++
++ 10. Automatic Licensing of Downstream Recipients.
++
++ Each time you convey a covered work, the recipient automatically
++receives a license from the original licensors, to run, modify and
++propagate that work, subject to this License. You are not responsible
++for enforcing compliance by third parties with this License.
++
++ An "entity transaction" is a transaction transferring control of an
++organization, or substantially all assets of one, or subdividing an
++organization, or merging organizations. If propagation of a covered
++work results from an entity transaction, each party to that
++transaction who receives a copy of the work also receives whatever
++licenses to the work the party's predecessor in interest had or could
++give under the previous paragraph, plus a right to possession of the
++Corresponding Source of the work from the predecessor in interest, if
++the predecessor has it or can get it with reasonable efforts.
++
++ You may not impose any further restrictions on the exercise of the
++rights granted or affirmed under this License. For example, you may
++not impose a license fee, royalty, or other charge for exercise of
++rights granted under this License, and you may not initiate litigation
++(including a cross-claim or counterclaim in a lawsuit) alleging that
++any patent claim is infringed by making, using, selling, offering for
++sale, or importing the Program or any portion of it.
++
++ 11. Patents.
++
++ A "contributor" is a copyright holder who authorizes use under this
++License of the Program or a work on which the Program is based. The
++work thus licensed is called the contributor's "contributor version".
++
++ A contributor's "essential patent claims" are all patent claims
++owned or controlled by the contributor, whether already acquired or
++hereafter acquired, that would be infringed by some manner, permitted
++by this License, of making, using, or selling its contributor version,
++but do not include claims that would be infringed only as a
++consequence of further modification of the contributor version. For
++purposes of this definition, "control" includes the right to grant
++patent sublicenses in a manner consistent with the requirements of
++this License.
++
++ Each contributor grants you a non-exclusive, worldwide, royalty-free
++patent license under the contributor's essential patent claims, to
++make, use, sell, offer for sale, import and otherwise run, modify and
++propagate the contents of its contributor version.
++
++ In the following three paragraphs, a "patent license" is any express
++agreement or commitment, however denominated, not to enforce a patent
++(such as an express permission to practice a patent or covenant not to
++sue for patent infringement). To "grant" such a patent license to a
++party means to make such an agreement or commitment not to enforce a
++patent against the party.
++
++ If you convey a covered work, knowingly relying on a patent license,
++and the Corresponding Source of the work is not available for anyone
++to copy, free of charge and under the terms of this License, through a
++publicly available network server or other readily accessible means,
++then you must either (1) cause the Corresponding Source to be so
++available, or (2) arrange to deprive yourself of the benefit of the
++patent license for this particular work, or (3) arrange, in a manner
++consistent with the requirements of this License, to extend the patent
++license to downstream recipients. "Knowingly relying" means you have
++actual knowledge that, but for the patent license, your conveying the
++covered work in a country, or your recipient's use of the covered work
++in a country, would infringe one or more identifiable patents in that
++country that you have reason to believe are valid.
++
++ If, pursuant to or in connection with a single transaction or
++arrangement, you convey, or propagate by procuring conveyance of, a
++covered work, and grant a patent license to some of the parties
++receiving the covered work authorizing them to use, propagate, modify
++or convey a specific copy of the covered work, then the patent license
++you grant is automatically extended to all recipients of the covered
++work and works based on it.
++
++ A patent license is "discriminatory" if it does not include within
++the scope of its coverage, prohibits the exercise of, or is
++conditioned on the non-exercise of one or more of the rights that are
++specifically granted under this License. You may not convey a covered
++work if you are a party to an arrangement with a third party that is
++in the business of distributing software, under which you make payment
++to the third party based on the extent of your activity of conveying
++the work, and under which the third party grants, to any of the
++parties who would receive the covered work from you, a discriminatory
++patent license (a) in connection with copies of the covered work
++conveyed by you (or copies made from those copies), or (b) primarily
++for and in connection with specific products or compilations that
++contain the covered work, unless you entered into that arrangement,
++or that patent license was granted, prior to 28 March 2007.
++
++ Nothing in this License shall be construed as excluding or limiting
++any implied license or other defenses to infringement that may
++otherwise be available to you under applicable patent law.
++
++ 12. No Surrender of Others' Freedom.
++
++ If conditions are imposed on you (whether by court order, agreement or
++otherwise) that contradict the conditions of this License, they do not
++excuse you from the conditions of this License. If you cannot convey a
++covered work so as to satisfy simultaneously your obligations under this
++License and any other pertinent obligations, then as a consequence you may
++not convey it at all. For example, if you agree to terms that obligate you
++to collect a royalty for further conveying from those to whom you convey
++the Program, the only way you could satisfy both those terms and this
++License would be to refrain entirely from conveying the Program.
++
++ 13. Remote Network Interaction; Use with the GNU General Public License.
++
++ Notwithstanding any other provision of this License, if you modify the
++Program, your modified version must prominently offer all users
++interacting with it remotely through a computer network (if your version
++supports such interaction) an opportunity to receive the Corresponding
++Source of your version by providing access to the Corresponding Source
++from a network server at no charge, through some standard or customary
++means of facilitating copying of software. This Corresponding Source
++shall include the Corresponding Source for any work covered by version 3
++of the GNU General Public License that is incorporated pursuant to the
++following paragraph.
++
++ Notwithstanding any other provision of this License, you have
++permission to link or combine any covered work with a work licensed
++under version 3 of the GNU General Public License into a single
++combined work, and to convey the resulting work. The terms of this
++License will continue to apply to the part which is the covered work,
++but the work with which it is combined will remain governed by version
++3 of the GNU General Public License.
++
++ 14. Revised Versions of this License.
++
++ The Free Software Foundation may publish revised and/or new versions of
++the GNU Affero General Public License from time to time. Such new versions
++will be similar in spirit to the present version, but may differ in detail to
++address new problems or concerns.
++
++ Each version is given a distinguishing version number. If the
++Program specifies that a certain numbered version of the GNU Affero General
++Public License "or any later version" applies to it, you have the
++option of following the terms and conditions either of that numbered
++version or of any later version published by the Free Software
++Foundation. If the Program does not specify a version number of the
++GNU Affero General Public License, you may choose any version ever published
++by the Free Software Foundation.
++
++ If the Program specifies that a proxy can decide which future
++versions of the GNU Affero General Public License can be used, that proxy's
++public statement of acceptance of a version permanently authorizes you
++to choose that version for the Program.
++
++ Later license versions may give you additional or different
++permissions. However, no additional obligations are imposed on any
++author or copyright holder as a result of your choosing to follow a
++later version.
++
++ 15. Disclaimer of Warranty.
++
++ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
++APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
++HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
++OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
++THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
++PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
++IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
++ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
++
++ 16. Limitation of Liability.
++
++ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
++WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
++THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
++GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
++USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
++DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
++PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
++EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
++SUCH DAMAGES.
++
++ 17. Interpretation of Sections 15 and 16.
++
++ If the disclaimer of warranty and limitation of liability provided
++above cannot be given local legal effect according to their terms,
++reviewing courts shall apply local law that most closely approximates
++an absolute waiver of all civil liability in connection with the
++Program, unless a warranty or assumption of liability accompanies a
++copy of the Program in return for a fee.
++
++ END OF TERMS AND CONDITIONS
++
++ How to Apply These Terms to Your New Programs
++
++ If you develop a new program, and you want it to be of the greatest
++possible use to the public, the best way to achieve this is to make it
++free software which everyone can redistribute and change under these terms.
++
++ To do so, attach the following notices to the program. It is safest
++to attach them to the start of each source file to most effectively
++state the exclusion of warranty; and each file should have at least
++the "copyright" line and a pointer to where the full notice is found.
++
++ <one line to give the program's name and a brief idea of what it does.>
++ Copyright (C) <year> <name of author>
++
++ This program is free software: you can redistribute it and/or modify
++ it under the terms of the GNU Affero General Public License as published by
++ the Free Software Foundation, either version 3 of the License, or
++ (at your option) any later version.
++
++ This program is distributed in the hope that it will be useful,
++ but WITHOUT ANY WARRANTY; without even the implied warranty of
++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
++ GNU Affero General Public License for more details.
++
++ You should have received a copy of the GNU Affero General Public License
++ along with this program. If not, see <http://www.gnu.org/licenses/>.
++
++Also add information on how to contact you by electronic and paper mail.
++
++ If your software can interact with users remotely through a computer
++network, you should also make sure that it provides a way for users to
++get its source. For example, if your program is a web application, its
++interface could display a "Source" link that leads users to an archive
++of the code. There are many ways you could offer source, and different
++solutions will be better for different programs; see section 13 for the
++specific requirements.
++
++ You should also get your employer (if you work as a programmer) or school,
++if any, to sign a "copyright disclaimer" for the program, if necessary.
++For more information on this, and how to apply and follow the GNU AGPL, see
++<http://www.gnu.org/licenses/>.
--- /dev/null
--- /dev/null
++
++ Apache License
++ Version 2.0, January 2004
++ http://www.apache.org/licenses/
++
++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
++
++ 1. Definitions.
++
++ "License" shall mean the terms and conditions for use, reproduction,
++ and distribution as defined by Sections 1 through 9 of this document.
++
++ "Licensor" shall mean the copyright owner or entity authorized by
++ the copyright owner that is granting the License.
++
++ "Legal Entity" shall mean the union of the acting entity and all
++ other entities that control, are controlled by, or are under common
++ control with that entity. For the purposes of this definition,
++ "control" means (i) the power, direct or indirect, to cause the
++ direction or management of such entity, whether by contract or
++ otherwise, or (ii) ownership of fifty percent (50%) or more of the
++ outstanding shares, or (iii) beneficial ownership of such entity.
++
++ "You" (or "Your") shall mean an individual or Legal Entity
++ exercising permissions granted by this License.
++
++ "Source" form shall mean the preferred form for making modifications,
++ including but not limited to software source code, documentation
++ source, and configuration files.
++
++ "Object" form shall mean any form resulting from mechanical
++ transformation or translation of a Source form, including but
++ not limited to compiled object code, generated documentation,
++ and conversions to other media types.
++
++ "Work" shall mean the work of authorship, whether in Source or
++ Object form, made available under the License, as indicated by a
++ copyright notice that is included in or attached to the work
++ (an example is provided in the Appendix below).
++
++ "Derivative Works" shall mean any work, whether in Source or Object
++ form, that is based on (or derived from) the Work and for which the
++ editorial revisions, annotations, elaborations, or other modifications
++ represent, as a whole, an original work of authorship. For the purposes
++ of this License, Derivative Works shall not include works that remain
++ separable from, or merely link (or bind by name) to the interfaces of,
++ the Work and Derivative Works thereof.
++
++ "Contribution" shall mean any work of authorship, including
++ the original version of the Work and any modifications or additions
++ to that Work or Derivative Works thereof, that is intentionally
++ submitted to Licensor for inclusion in the Work by the copyright owner
++ or by an individual or Legal Entity authorized to submit on behalf of
++ the copyright owner. For the purposes of this definition, "submitted"
++ means any form of electronic, verbal, or written communication sent
++ to the Licensor or its representatives, including but not limited to
++ communication on electronic mailing lists, source code control systems,
++ and issue tracking systems that are managed by, or on behalf of, the
++ Licensor for the purpose of discussing and improving the Work, but
++ excluding communication that is conspicuously marked or otherwise
++ designated in writing by the copyright owner as "Not a Contribution."
++
++ "Contributor" shall mean Licensor and any individual or Legal Entity
++ on behalf of whom a Contribution has been received by Licensor and
++ subsequently incorporated within the Work.
++
++ 2. Grant of Copyright License. Subject to the terms and conditions of
++ this License, each Contributor hereby grants to You a perpetual,
++ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
++ copyright license to reproduce, prepare Derivative Works of,
++ publicly display, publicly perform, sublicense, and distribute the
++ Work and such Derivative Works in Source or Object form.
++
++ 3. Grant of Patent License. Subject to the terms and conditions of
++ this License, each Contributor hereby grants to You a perpetual,
++ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
++ (except as stated in this section) patent license to make, have made,
++ use, offer to sell, sell, import, and otherwise transfer the Work,
++ where such license applies only to those patent claims licensable
++ by such Contributor that are necessarily infringed by their
++ Contribution(s) alone or by combination of their Contribution(s)
++ with the Work to which such Contribution(s) was submitted. If You
++ institute patent litigation against any entity (including a
++ cross-claim or counterclaim in a lawsuit) alleging that the Work
++ or a Contribution incorporated within the Work constitutes direct
++ or contributory patent infringement, then any patent licenses
++ granted to You under this License for that Work shall terminate
++ as of the date such litigation is filed.
++
++ 4. Redistribution. You may reproduce and distribute copies of the
++ Work or Derivative Works thereof in any medium, with or without
++ modifications, and in Source or Object form, provided that You
++ meet the following conditions:
++
++ (a) You must give any other recipients of the Work or
++ Derivative Works a copy of this License; and
++
++ (b) You must cause any modified files to carry prominent notices
++ stating that You changed the files; and
++
++ (c) You must retain, in the Source form of any Derivative Works
++ that You distribute, all copyright, patent, trademark, and
++ attribution notices from the Source form of the Work,
++ excluding those notices that do not pertain to any part of
++ the Derivative Works; and
++
++ (d) If the Work includes a "NOTICE" text file as part of its
++ distribution, then any Derivative Works that You distribute must
++ include a readable copy of the attribution notices contained
++ within such NOTICE file, excluding those notices that do not
++ pertain to any part of the Derivative Works, in at least one
++ of the following places: within a NOTICE text file distributed
++ as part of the Derivative Works; within the Source form or
++ documentation, if provided along with the Derivative Works; or,
++ within a display generated by the Derivative Works, if and
++ wherever such third-party notices normally appear. The contents
++ of the NOTICE file are for informational purposes only and
++ do not modify the License. You may add Your own attribution
++ notices within Derivative Works that You distribute, alongside
++ or as an addendum to the NOTICE text from the Work, provided
++ that such additional attribution notices cannot be construed
++ as modifying the License.
++
++ You may add Your own copyright statement to Your modifications and
++ may provide additional or different license terms and conditions
++ for use, reproduction, or distribution of Your modifications, or
++ for any such Derivative Works as a whole, provided Your use,
++ reproduction, and distribution of the Work otherwise complies with
++ the conditions stated in this License.
++
++ 5. Submission of Contributions. Unless You explicitly state otherwise,
++ any Contribution intentionally submitted for inclusion in the Work
++ by You to the Licensor shall be under the terms and conditions of
++ this License, without any additional terms or conditions.
++ Notwithstanding the above, nothing herein shall supersede or modify
++ the terms of any separate license agreement you may have executed
++ with Licensor regarding such Contributions.
++
++ 6. Trademarks. This License does not grant permission to use the trade
++ names, trademarks, service marks, or product names of the Licensor,
++ except as required for reasonable and customary use in describing the
++ origin of the Work and reproducing the content of the NOTICE file.
++
++ 7. Disclaimer of Warranty. Unless required by applicable law or
++ agreed to in writing, Licensor provides the Work (and each
++ Contributor provides its Contributions) on an "AS IS" BASIS,
++ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
++ implied, including, without limitation, any warranties or conditions
++ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
++ PARTICULAR PURPOSE. You are solely responsible for determining the
++ appropriateness of using or redistributing the Work and assume any
++ risks associated with Your exercise of permissions under this License.
++
++ 8. Limitation of Liability. In no event and under no legal theory,
++ whether in tort (including negligence), contract, or otherwise,
++ unless required by applicable law (such as deliberate and grossly
++ negligent acts) or agreed to in writing, shall any Contributor be
++ liable to You for damages, including any direct, indirect, special,
++ incidental, or consequential damages of any character arising as a
++ result of this License or out of the use or inability to use the
++ Work (including but not limited to damages for loss of goodwill,
++ work stoppage, computer failure or malfunction, or any and all
++ other commercial damages or losses), even if such Contributor
++ has been advised of the possibility of such damages.
++
++ 9. Accepting Warranty or Additional Liability. While redistributing
++ the Work or Derivative Works thereof, You may choose to offer,
++ and charge a fee for, acceptance of support, warranty, indemnity,
++ or other liability obligations and/or rights consistent with this
++ License. However, in accepting such obligations, You may act only
++ on Your own behalf and on Your sole responsibility, not on behalf
++ of any other Contributor, and only if You agree to indemnify,
++ defend, and hold each Contributor harmless for any liability
++ incurred by, or claims asserted against, such Contributor by reason
++ of your accepting any such warranty or additional liability.
++
++ END OF TERMS AND CONDITIONS
++
++ APPENDIX: How to apply the Apache License to your work.
++
++ To apply the Apache License to your work, attach the following
++ boilerplate notice, with the fields enclosed by brackets "[]"
++ replaced with your own identifying information. (Don't include
++ the brackets!) The text should be enclosed in the appropriate
++ comment syntax for the file format. We also recommend that a
++ file or class name and description of purpose be included on the
++ same "printed page" as the copyright notice for easier
++ identification within third-party archives.
++
++ Copyright [yyyy] [name of copyright owner]
++
++ Licensed under the Apache License, Version 2.0 (the "License");
++ you may not use this file except in compliance with the License.
++ You may obtain a copy of the License at
++
++ http://www.apache.org/licenses/LICENSE-2.0
++
++ Unless required by applicable law or agreed to in writing, software
++ distributed under the License is distributed on an "AS IS" BASIS,
++ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
++ See the License for the specific language governing permissions and
++ limitations under the License.
--- /dev/null
+ apply plugin: 'java-library'
+ apply plugin: 'eclipse'
+ apply plugin: 'idea'
+ apply plugin: 'maven'
+
+ version = '2.0.0'
+
+ repositories {
+ mavenCentral()
+ }
+
+ dependencies {
+ api 'com.squareup.okhttp3:okhttp:3.9.1'
+ api 'com.fasterxml.jackson.core:jackson-databind:2.9.2'
+ api 'com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.9.2'
+ api 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.9.2'
+ api 'commons-codec:commons-codec:1.11'
+ api 'commons-io:commons-io:2.6'
+ api 'com.google.guava:guava:23.4-jre'
+ api 'org.slf4j:slf4j-api:1.7.25'
+ api 'com.typesafe:config:1.3.2'
+
+ testImplementation 'junit:junit:4.12'
+ testImplementation 'org.mockito:mockito-core:2.12.0'
+ testImplementation 'org.assertj:assertj-core:3.8.0'
+ testImplementation 'com.squareup.okhttp3:mockwebserver:3.9.1'
+ }
+
+ test {
+ useJUnit {
+ excludeCategories 'org.arvados.client.junit.categories.IntegrationTests'
+ }
+
+ testLogging {
+ events "passed", "skipped", "failed"
+ afterSuite { desc, result ->
+ if (!desc.parent) { // will match the outermost suite
+ println "\n---- Test results ----"
+ println "${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
+ println ""
+ }
+ }
+ }
+ }
+
+ task integrationTest(type: Test) {
+ useJUnit {
+ includeCategories 'org.arvados.client.junit.categories.IntegrationTests'
+ }
+ }
--- /dev/null
+ distributionBase=GRADLE_USER_HOME
+ distributionPath=wrapper/dists
+ zipStoreBase=GRADLE_USER_HOME
+ zipStorePath=wrapper/dists
+ distributionUrl=https\://services.gradle.org/distributions/gradle-4.8-bin.zip
--- /dev/null
+ #!/usr/bin/env sh
+
+ ##############################################################################
+ ##
+ ## Gradle start up script for UN*X
+ ##
+ ##############################################################################
+
+ # Attempt to set APP_HOME
+ # Resolve links: $0 may be a link
+ PRG="$0"
+ # Need this for relative symlinks.
+ while [ -h "$PRG" ] ; do
+ ls=`ls -ld "$PRG"`
+ link=`expr "$ls" : '.*-> \(.*\)$'`
+ if expr "$link" : '/.*' > /dev/null; then
+ PRG="$link"
+ else
+ PRG=`dirname "$PRG"`"/$link"
+ fi
+ done
+ SAVED="`pwd`"
+ cd "`dirname \"$PRG\"`/" >/dev/null
+ APP_HOME="`pwd -P`"
+ cd "$SAVED" >/dev/null
+
+ APP_NAME="Gradle"
+ APP_BASE_NAME=`basename "$0"`
+
+ # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+ DEFAULT_JVM_OPTS=""
+
+ # Use the maximum available, or set MAX_FD != -1 to use that value.
+ MAX_FD="maximum"
+
+ warn () {
+ echo "$*"
+ }
+
+ die () {
+ echo
+ echo "$*"
+ echo
+ exit 1
+ }
+
+ # OS specific support (must be 'true' or 'false').
+ cygwin=false
+ msys=false
+ darwin=false
+ nonstop=false
+ case "`uname`" in
+ CYGWIN* )
+ cygwin=true
+ ;;
+ Darwin* )
+ darwin=true
+ ;;
+ MINGW* )
+ msys=true
+ ;;
+ NONSTOP* )
+ nonstop=true
+ ;;
+ esac
+
+ CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
+
+ # Determine the Java command to use to start the JVM.
+ if [ -n "$JAVA_HOME" ] ; then
+ if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
+ # IBM's JDK on AIX uses strange locations for the executables
+ JAVACMD="$JAVA_HOME/jre/sh/java"
+ else
+ JAVACMD="$JAVA_HOME/bin/java"
+ fi
+ if [ ! -x "$JAVACMD" ] ; then
+ die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
+
+ Please set the JAVA_HOME variable in your environment to match the
+ location of your Java installation."
+ fi
+ else
+ JAVACMD="java"
+ which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+
+ Please set the JAVA_HOME variable in your environment to match the
+ location of your Java installation."
+ fi
+
+ # Increase the maximum file descriptors if we can.
+ if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
+ MAX_FD_LIMIT=`ulimit -H -n`
+ if [ $? -eq 0 ] ; then
+ if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
+ MAX_FD="$MAX_FD_LIMIT"
+ fi
+ ulimit -n $MAX_FD
+ if [ $? -ne 0 ] ; then
+ warn "Could not set maximum file descriptor limit: $MAX_FD"
+ fi
+ else
+ warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
+ fi
+ fi
+
+ # For Darwin, add options to specify how the application appears in the dock
+ if $darwin; then
+ GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
+ fi
+
+ # For Cygwin, switch paths to Windows format before running java
+ if $cygwin ; then
+ APP_HOME=`cygpath --path --mixed "$APP_HOME"`
+ CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
+ JAVACMD=`cygpath --unix "$JAVACMD"`
+
+ # We build the pattern for arguments to be converted via cygpath
+ ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
+ SEP=""
+ for dir in $ROOTDIRSRAW ; do
+ ROOTDIRS="$ROOTDIRS$SEP$dir"
+ SEP="|"
+ done
+ OURCYGPATTERN="(^($ROOTDIRS))"
+ # Add a user-defined pattern to the cygpath arguments
+ if [ "$GRADLE_CYGPATTERN" != "" ] ; then
+ OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
+ fi
+ # Now convert the arguments - kludge to limit ourselves to /bin/sh
+ i=0
+ for arg in "$@" ; do
+ CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
+ CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
+
+ if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
+ eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
+ else
+ eval `echo args$i`="\"$arg\""
+ fi
+ i=$((i+1))
+ done
+ case $i in
+ (0) set -- ;;
+ (1) set -- "$args0" ;;
+ (2) set -- "$args0" "$args1" ;;
+ (3) set -- "$args0" "$args1" "$args2" ;;
+ (4) set -- "$args0" "$args1" "$args2" "$args3" ;;
+ (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
+ (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
+ (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
+ (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
+ (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
+ esac
+ fi
+
+ # Escape application args
+ save () {
+ for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
+ echo " "
+ }
+ APP_ARGS=$(save "$@")
+
+ # Collect all arguments for the java command, following the shell quoting and substitution rules
+ eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
+
+ # by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong
+ if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then
+ cd "$(dirname "$0")"
+ fi
+
+ exec "$JAVACMD" "$@"
--- /dev/null
+ @if "%DEBUG%" == "" @echo off
+ @rem ##########################################################################
+ @rem
+ @rem Gradle startup script for Windows
+ @rem
+ @rem ##########################################################################
+
+ @rem Set local scope for the variables with windows NT shell
+ if "%OS%"=="Windows_NT" setlocal
+
+ set DIRNAME=%~dp0
+ if "%DIRNAME%" == "" set DIRNAME=.
+ set APP_BASE_NAME=%~n0
+ set APP_HOME=%DIRNAME%
+
+ @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+ set DEFAULT_JVM_OPTS=
+
+ @rem Find java.exe
+ if defined JAVA_HOME goto findJavaFromJavaHome
+
+ set JAVA_EXE=java.exe
+ %JAVA_EXE% -version >NUL 2>&1
+ if "%ERRORLEVEL%" == "0" goto init
+
+ echo.
+ echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+ echo.
+ echo Please set the JAVA_HOME variable in your environment to match the
+ echo location of your Java installation.
+
+ goto fail
+
+ :findJavaFromJavaHome
+ set JAVA_HOME=%JAVA_HOME:"=%
+ set JAVA_EXE=%JAVA_HOME%/bin/java.exe
+
+ if exist "%JAVA_EXE%" goto init
+
+ echo.
+ echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
+ echo.
+ echo Please set the JAVA_HOME variable in your environment to match the
+ echo location of your Java installation.
+
+ goto fail
+
+ :init
+ @rem Get command-line arguments, handling Windows variants
+
+ if not "%OS%" == "Windows_NT" goto win9xME_args
+
+ :win9xME_args
+ @rem Slurp the command line arguments.
+ set CMD_LINE_ARGS=
+ set _SKIP=2
+
+ :win9xME_args_slurp
+ if "x%~1" == "x" goto execute
+
+ set CMD_LINE_ARGS=%*
+
+ :execute
+ @rem Setup the command line
+
+ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
+
+ @rem Execute Gradle
+ "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS%
+
+ :end
+ @rem End local scope for the variables with windows NT shell
+ if "%ERRORLEVEL%"=="0" goto mainEnd
+
+ :fail
+ rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
+ rem the _cmd.exe /c_ return code!
+ if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
+ exit /b 1
+
+ :mainEnd
+ if "%OS%"=="Windows_NT" endlocal
+
+ :omega
--- /dev/null
+ rootProject.name = 'arvados-java'
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import com.fasterxml.jackson.databind.ObjectMapper;
+ import org.arvados.client.exception.ArvadosApiException;
+ import org.arvados.client.api.client.factory.OkHttpClientFactory;
+ import org.arvados.client.api.model.ApiError;
+ import org.arvados.client.config.ConfigProvider;
+ import okhttp3.OkHttpClient;
+ import okhttp3.Request;
+ import okhttp3.Response;
+ import okhttp3.ResponseBody;
+ import org.slf4j.Logger;
+
+ import java.io.IOException;
+ import java.io.UnsupportedEncodingException;
+ import java.net.URLDecoder;
+ import java.nio.charset.StandardCharsets;
+ import java.util.Objects;
+
+ abstract class BaseApiClient {
+
+ static final ObjectMapper MAPPER = new ObjectMapper().findAndRegisterModules();
+
+ final OkHttpClient client;
+ final ConfigProvider config;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(BaseApiClient.class);
+
+ BaseApiClient(ConfigProvider config) {
+ this.config = config;
+ client = OkHttpClientFactory.builder()
+ .build()
+ .create(config.isApiHostInsecure());
+ }
+
+ Request.Builder getRequestBuilder() {
+ return new Request.Builder()
+ .addHeader("authorization", String.format("OAuth2 %s", config.getApiToken()))
+ .addHeader("cache-control", "no-cache");
+ }
+
+ String newCall(Request request) {
+ return (String) getResponseBody(request, body -> body.string().trim());
+ }
+
+ byte[] newFileCall(Request request) {
+ return (byte[]) getResponseBody(request, ResponseBody::bytes);
+ }
+
+ private Object getResponseBody(Request request, Command command) {
+ try {
+ log.debug(URLDecoder.decode(request.toString(), StandardCharsets.UTF_8.name()));
+ } catch (UnsupportedEncodingException e) {
+ throw new ArvadosApiException(e);
+ }
+
+ try (Response response = client.newCall(request).execute()) {
+ ResponseBody responseBody = response.body();
+
+ if (!response.isSuccessful()) {
+ String errorBody = Objects.requireNonNull(responseBody).string();
+ if (errorBody == null || errorBody.length() == 0) {
+ throw new ArvadosApiException(String.format("Error code %s with message: %s", response.code(), response.message()));
+ }
+ ApiError apiError = MAPPER.readValue(errorBody, ApiError.class);
+ throw new ArvadosApiException(String.format("Error code %s with messages: %s", response.code(), apiError.getErrors()));
+ }
+ return command.readResponseBody(responseBody);
+ } catch (IOException e) {
+ throw new ArvadosApiException(e);
+ }
+ }
+
+ private interface Command {
+ Object readResponseBody(ResponseBody body) throws IOException;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import com.fasterxml.jackson.core.JsonProcessingException;
+ import com.fasterxml.jackson.core.type.TypeReference;
+ import com.fasterxml.jackson.databind.ObjectWriter;
+ import okhttp3.MediaType;
+ import okhttp3.HttpUrl;
+ import okhttp3.HttpUrl.Builder;
+ import okhttp3.Request;
+ import okhttp3.RequestBody;
+ import org.arvados.client.exception.ArvadosApiException;
+ import org.arvados.client.api.model.Item;
+ import org.arvados.client.api.model.ItemList;
+ import org.arvados.client.api.model.argument.ListArgument;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ import java.io.IOException;
+ import java.util.Map;
+
+ public abstract class BaseStandardApiClient<T extends Item, L extends ItemList> extends BaseApiClient {
+
+ private static final MediaType JSON = MediaType.parse(com.google.common.net.MediaType.JSON_UTF_8.toString());
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(BaseStandardApiClient.class);
+
+ BaseStandardApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public L list(ListArgument listArguments) {
+ log.debug("Get list of {}", getType().getSimpleName());
+ Builder urlBuilder = getUrlBuilder();
+ addQueryParameters(urlBuilder, listArguments);
+ HttpUrl url = urlBuilder.build();
+ Request request = getRequestBuilder().url(url).build();
+ return callForList(request);
+ }
+
+ public L list() {
+ return list(ListArgument.builder().build());
+ }
+
+ public T get(String uuid) {
+ log.debug("Get {} by UUID {}", getType().getSimpleName(), uuid);
+ HttpUrl url = getUrlBuilder().addPathSegment(uuid).build();
+ Request request = getRequestBuilder().get().url(url).build();
+ return callForType(request);
+ }
+
+ public T create(T type) {
+ log.debug("Create {}", getType().getSimpleName());
+ String json = mapToJson(type);
+ RequestBody body = RequestBody.create(JSON, json);
+ Request request = getRequestBuilder().post(body).build();
+ return callForType(request);
+ }
+
+ public T delete(String uuid) {
+ log.debug("Delete {} by UUID {}", getType().getSimpleName(), uuid);
+ HttpUrl url = getUrlBuilder().addPathSegment(uuid).build();
+ Request request = getRequestBuilder().delete().url(url).build();
+ return callForType(request);
+ }
+
+ public T update(T type) {
+ String uuid = type.getUuid();
+ log.debug("Update {} by UUID {}", getType().getSimpleName(), uuid);
+ String json = mapToJson(type);
+ RequestBody body = RequestBody.create(JSON, json);
+ HttpUrl url = getUrlBuilder().addPathSegment(uuid).build();
+ Request request = getRequestBuilder().put(body).url(url).build();
+ return callForType(request);
+ }
+
+ @Override
+ Request.Builder getRequestBuilder() {
+ return super.getRequestBuilder().url(getUrlBuilder().build());
+ }
+
+ HttpUrl.Builder getUrlBuilder() {
+ return new HttpUrl.Builder()
+ .scheme(config.getApiProtocol())
+ .host(config.getApiHost())
+ .port(config.getApiPort())
+ .addPathSegment("arvados")
+ .addPathSegment("v1")
+ .addPathSegment(getResource());
+ }
+
+ <TL> TL call(Request request, Class<TL> cls) {
+ String bodyAsString = newCall(request);
+ try {
+ return mapToObject(bodyAsString, cls);
+ } catch (IOException e) {
+ throw new ArvadosApiException("A problem occurred while parsing JSON data", e);
+ }
+ }
+
+ private <TL> TL mapToObject(String content, Class<TL> cls) throws IOException {
+ return MAPPER.readValue(content, cls);
+ }
+
+ private <TL> String mapToJson(TL type) {
+ ObjectWriter writer = MAPPER.writer().withDefaultPrettyPrinter();
+ try {
+ return writer.writeValueAsString(type);
+ } catch (JsonProcessingException e) {
+ log.error(e.getMessage());
+ return null;
+ }
+ }
+
+ T callForType(Request request) {
+ return call(request, getType());
+ }
+
+ L callForList(Request request) {
+ return call(request, getListType());
+ }
+
+ abstract String getResource();
+
+ abstract Class<T> getType();
+
+ abstract Class<L> getListType();
+
+ Request getNoArgumentMethodRequest(String method) {
+ HttpUrl url = getUrlBuilder().addPathSegment(method).build();
+ return getRequestBuilder().get().url(url).build();
+ }
+
+ RequestBody getJsonRequestBody(Object object) {
+ return RequestBody.create(JSON, mapToJson(object));
+ }
+
+ void addQueryParameters(Builder urlBuilder, Object object) {
+ Map<String, Object> queryMap = MAPPER.convertValue(object, new TypeReference<Map<String, Object>>() {});
+ queryMap.keySet().forEach(key -> {
+ Object type = queryMap.get(key);
+ if (!(type instanceof String)) {
+ type = mapToJson(type);
+ }
+ urlBuilder.addQueryParameter(key, (String) type);
+ });
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.api.model.CollectionList;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ public class CollectionsApiClient extends BaseStandardApiClient<Collection, CollectionList> {
+
+ private static final String RESOURCE = "collections";
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(CollectionsApiClient.class);
+
+ public CollectionsApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ @Override
+ public Collection create(Collection type) {
+ Collection newCollection = super.create(type);
+ log.debug(String.format("New collection '%s' with UUID %s has been created", newCollection.getName(), newCollection.getUuid()));
+ return newCollection;
+ }
+
+ @Override
+ String getResource() {
+ return RESOURCE;
+ }
+
+ @Override
+ Class<Collection> getType() {
+ return Collection.class;
+ }
+
+ @Override
+ Class<CollectionList> getListType() {
+ return CollectionList.class;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.MediaType;
+ import okhttp3.RequestBody;
+ import okio.BufferedSink;
+ import okio.Okio;
+ import okio.Source;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+
+ /**
+ * Based on:
+ * {@link} https://gist.github.com/eduardb/dd2dc530afd37108e1ac
+ */
+ public class CountingFileRequestBody extends RequestBody {
+
+ private static final int SEGMENT_SIZE = 2048; // okio.Segment.SIZE
+ private static final MediaType CONTENT_BINARY = MediaType.parse(com.google.common.net.MediaType.OCTET_STREAM.toString());
+
+ private final File file;
+ private final ProgressListener listener;
+
+ CountingFileRequestBody(final File file, final ProgressListener listener) {
+ this.file = file;
+ this.listener = listener;
+ }
+
+ @Override
+ public long contentLength() {
+ return file.length();
+ }
+
+ @Override
+ public MediaType contentType() {
+ return CONTENT_BINARY;
+ }
+
+ @Override
+ public void writeTo(BufferedSink sink) {
+ try (Source source = Okio.source(file)) {
+ long total = 0;
+ long read;
+
+ while ((read = source.read(sink.buffer(), SEGMENT_SIZE)) != -1) {
+ total += read;
+ sink.flush();
+ listener.updateProgress(total);
+
+ }
+ } catch (RuntimeException rethrown) {
+ throw rethrown;
+ } catch (Exception ignored) {
+ //ignore
+ }
+ }
+
+ static class TransferData {
+
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(TransferData.class);
+ private int progressValue;
+ private long totalSize;
+
+ TransferData(long totalSize) {
+ this.progressValue = 0;
+ this.totalSize = totalSize;
+ }
+
+ void updateTransferProgress(long transferred) {
+ float progress = (transferred / (float) totalSize) * 100;
+ if (progressValue != (int) progress) {
+ progressValue = (int) progress;
+ log.debug("{} / {} / {}%", transferred, totalSize, progressValue);
+ }
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.HttpUrl;
+ import okhttp3.HttpUrl.Builder;
+ import okhttp3.Request;
+ import okhttp3.RequestBody;
+ import org.arvados.client.api.model.Group;
+ import org.arvados.client.api.model.GroupList;
+ import org.arvados.client.api.model.argument.ContentsGroup;
+ import org.arvados.client.api.model.argument.UntrashGroup;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ public class GroupsApiClient extends BaseStandardApiClient<Group, GroupList> {
+
+ private static final String RESOURCE = "groups";
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(GroupsApiClient.class);
+
+ public GroupsApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public GroupList contents(ContentsGroup contentsGroup) {
+ log.debug("Get {} contents", getType().getSimpleName());
+ Builder urlBuilder = getUrlBuilder().addPathSegment("contents");
+ addQueryParameters(urlBuilder, contentsGroup);
+ HttpUrl url = urlBuilder.build();
+ Request request = getRequestBuilder().url(url).build();
+ return callForList(request);
+ }
+
+ public Group untrash(UntrashGroup untrashGroup) {
+ log.debug("Untrash {} by UUID {}", getType().getSimpleName(), untrashGroup.getUuid());
+ HttpUrl url = getUrlBuilder().addPathSegment(untrashGroup.getUuid()).addPathSegment("untrash").build();
+ RequestBody requestBody = getJsonRequestBody(untrashGroup);
+ Request request = getRequestBuilder().post(requestBody).url(url).build();
+ return callForType(request);
+ }
+
+ @Override
+ String getResource() {
+ return RESOURCE;
+ }
+
+ @Override
+ Class<Group> getType() {
+ return Group.class;
+ }
+
+ @Override
+ Class<GroupList> getListType() {
+ return GroupList.class;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.Request;
+ import okhttp3.RequestBody;
+ import org.arvados.client.api.client.CountingFileRequestBody.TransferData;
+ import org.arvados.client.common.Headers;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.util.Map;
+
+ public class KeepServerApiClient extends BaseApiClient {
+
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(KeepServerApiClient.class);
+
+ public KeepServerApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public String upload(String url, Map<String, String> headers, File body) {
+
+ log.debug("Upload file {} to server location {}", body, url);
+
+ final TransferData transferData = new TransferData(body.length());
+
+ RequestBody requestBody = new CountingFileRequestBody(body, transferData::updateTransferProgress);
+
+ Request request = getRequestBuilder()
+ .url(url)
+ .addHeader(Headers.X_KEEP_DESIRED_REPLICAS, headers.get(Headers.X_KEEP_DESIRED_REPLICAS))
+ .put(requestBody)
+ .build();
+
+ return newCall(request);
+ }
+
+ public byte[] download(String url) {
+
+ Request request = getRequestBuilder()
+ .url(url)
+ .get()
+ .build();
+
+ return newFileCall(request);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import org.arvados.client.api.model.KeepService;
+ import org.arvados.client.api.model.KeepServiceList;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ public class KeepServicesApiClient extends BaseStandardApiClient<KeepService, KeepServiceList> {
+
+ private static final String RESOURCE = "keep_services";
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(KeepServicesApiClient.class);
+
+ public KeepServicesApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public KeepServiceList accessible() {
+ log.debug("Get list of accessible {}", getType().getSimpleName());
+ return callForList(getNoArgumentMethodRequest("accessible"));
+ }
+
+ @Override
+ String getResource() {
+ return RESOURCE;
+ }
+
+ @Override
+ Class<KeepService> getType() {
+ return KeepService.class;
+ }
+
+ @Override
+ Class<KeepServiceList> getListType() {
+ return KeepServiceList.class;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.HttpUrl;
+ import okhttp3.Request;
+ import org.arvados.client.config.ConfigProvider;
+
+ public class KeepWebApiClient extends BaseApiClient {
+
+ public KeepWebApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public byte[] download(String collectionUuid, String filePathName) {
+ Request request = getRequestBuilder()
+ .url(getUrlBuilder(collectionUuid,filePathName).build())
+ .get()
+ .build();
+
+ return newFileCall(request);
+ }
+
+ private HttpUrl.Builder getUrlBuilder(String collectionUuid, String filePathName) {
+ return new HttpUrl.Builder()
+ .scheme(config.getApiProtocol())
+ .host(config.getKeepWebHost())
+ .port(config.getKeepWebPort())
+ .addPathSegment("c=" + collectionUuid)
+ .addPathSegment(filePathName);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ @FunctionalInterface
+ public interface ProgressListener {
+
+ void updateProgress(long num);
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.Request;
+ import org.arvados.client.api.model.User;
+ import org.arvados.client.api.model.UserList;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ public class UsersApiClient extends BaseStandardApiClient<User, UserList> {
+
+ private static final String RESOURCE = "users";
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(UsersApiClient.class);
+
+ public UsersApiClient(ConfigProvider config) {
+ super(config);
+ }
+
+ public User current() {
+ log.debug("Get current {}", getType().getSimpleName());
+ Request request = getNoArgumentMethodRequest("current");
+ return callForType(request);
+ }
+
+ public User system() {
+ log.debug("Get system {}", getType().getSimpleName());
+ Request request = getNoArgumentMethodRequest("system");
+ return callForType(request);
+ }
+
+ @Override
+ String getResource() {
+ return RESOURCE;
+ }
+
+ @Override
+ Class<User> getType() {
+ return User.class;
+ }
+
+ @Override
+ Class<UserList> getListType() {
+ return UserList.class;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client.factory;
+
+ import okhttp3.OkHttpClient;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.slf4j.Logger;
+
+ import javax.net.ssl.SSLContext;
+ import javax.net.ssl.SSLSocketFactory;
+ import javax.net.ssl.TrustManager;
+ import javax.net.ssl.X509TrustManager;
+ import java.security.KeyManagementException;
+ import java.security.NoSuchAlgorithmException;
+ import java.security.SecureRandom;
+ import java.security.cert.X509Certificate;
+
+ public class OkHttpClientFactory {
+
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(OkHttpClientFactory.class);
+
+ OkHttpClientFactory() {
+ }
+
+ public static OkHttpClientFactoryBuilder builder() {
+ return new OkHttpClientFactoryBuilder();
+ }
+
+ public OkHttpClient create(boolean apiHostInsecure) {
+ OkHttpClient.Builder builder = new OkHttpClient.Builder();
+ if (apiHostInsecure) {
+ trustAllCertificates(builder);
+ }
+ return builder.build();
+ }
+
+ private void trustAllCertificates(OkHttpClient.Builder builder) {
+ log.warn("Creating unsafe OkHttpClient. All SSL certificates will be accepted.");
+ try {
+ // Create a trust manager that does not validate certificate chains
+ final TrustManager[] trustAllCerts = new TrustManager[] { createX509TrustManager() };
+
+ // Install the all-trusting trust manager
+ SSLContext sslContext = SSLContext.getInstance("SSL");
+ sslContext.init(null, trustAllCerts, new SecureRandom());
+ // Create an ssl socket factory with our all-trusting manager
+ final SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
+
+ builder.sslSocketFactory(sslSocketFactory, (X509TrustManager) trustAllCerts[0]);
+ builder.hostnameVerifier((hostname, session) -> true);
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ throw new ArvadosClientException("Error establishing SSL context", e);
+ }
+ }
+
+ private static X509TrustManager createX509TrustManager() {
+ return new X509TrustManager() {
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) {}
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) {}
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[] {};
+ }
+ };
+ }
+
+ public static class OkHttpClientFactoryBuilder {
+ OkHttpClientFactoryBuilder() {
+ }
+
+ public OkHttpClientFactory build() {
+ return new OkHttpClientFactory();
+ }
+
+ public String toString() {
+ return "OkHttpClientFactory.OkHttpClientFactoryBuilder()";
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "errors", "error_token" })
+ public class ApiError {
+
+ @JsonProperty("errors")
+ private List<String> errors;
+ @JsonProperty("error_token")
+ private String errorToken;
+
+ public List<String> getErrors() {
+ return this.errors;
+ }
+
+ public String getErrorToken() {
+ return this.errorToken;
+ }
+
+ public void setErrors(List<String> errors) {
+ this.errors = errors;
+ }
+
+ public void setErrorToken(String errorToken) {
+ this.errorToken = errorToken;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.time.LocalDateTime;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "portable_data_hash", "replication_desired", "replication_confirmed_at", "replication_confirmed", "manifest_text",
+ "name", "description", "properties", "delete_at", "trash_at", "is_trashed" })
+ public class Collection extends Item {
+
+ @JsonProperty("portable_data_hash")
+ private String portableDataHash;
+ @JsonProperty("replication_desired")
+ private Integer replicationDesired;
+ @JsonProperty("replication_confirmed_at")
+ private LocalDateTime replicationConfirmedAt;
+ @JsonProperty("replication_confirmed")
+ private Integer replicationConfirmed;
+ @JsonProperty("manifest_text")
+ private String manifestText;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("description")
+ private String description;
+ @JsonProperty("properties")
+ private Object properties;
+ @JsonProperty("delete_at")
+ private LocalDateTime deleteAt;
+ @JsonProperty("trash_at")
+ private LocalDateTime trashAt;
+ @JsonProperty("is_trashed")
+ private Boolean trashed;
+
+ public String getPortableDataHash() {
+ return this.portableDataHash;
+ }
+
+ public Integer getReplicationDesired() {
+ return this.replicationDesired;
+ }
+
+ public LocalDateTime getReplicationConfirmedAt() {
+ return this.replicationConfirmedAt;
+ }
+
+ public Integer getReplicationConfirmed() {
+ return this.replicationConfirmed;
+ }
+
+ public String getManifestText() {
+ return this.manifestText;
+ }
+
+ public String getName() {
+ return this.name;
+ }
+
+ public String getDescription() {
+ return this.description;
+ }
+
+ public Object getProperties() {
+ return this.properties;
+ }
+
+ public LocalDateTime getDeleteAt() {
+ return this.deleteAt;
+ }
+
+ public LocalDateTime getTrashAt() {
+ return this.trashAt;
+ }
+
+ public Boolean getTrashed() {
+ return this.trashed;
+ }
+
+ public void setPortableDataHash(String portableDataHash) {
+ this.portableDataHash = portableDataHash;
+ }
+
+ public void setReplicationDesired(Integer replicationDesired) {
+ this.replicationDesired = replicationDesired;
+ }
+
+ public void setReplicationConfirmedAt(LocalDateTime replicationConfirmedAt) {
+ this.replicationConfirmedAt = replicationConfirmedAt;
+ }
+
+ public void setReplicationConfirmed(Integer replicationConfirmed) {
+ this.replicationConfirmed = replicationConfirmed;
+ }
+
+ public void setManifestText(String manifestText) {
+ this.manifestText = manifestText;
+ }
+
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ public void setDescription(String description) {
+ this.description = description;
+ }
+
+ public void setProperties(Object properties) {
+ this.properties = properties;
+ }
+
+ public void setDeleteAt(LocalDateTime deleteAt) {
+ this.deleteAt = deleteAt;
+ }
+
+ public void setTrashAt(LocalDateTime trashAt) {
+ this.trashAt = trashAt;
+ }
+
+ public void setTrashed(Boolean trashed) {
+ this.trashed = trashed;
+ }
+
+ public String toString() {
+ return "Collection(portableDataHash=" + this.getPortableDataHash() + ", replicationDesired=" + this.getReplicationDesired() + ", replicationConfirmedAt=" + this.getReplicationConfirmedAt() + ", replicationConfirmed=" + this.getReplicationConfirmed() + ", manifestText=" + this.getManifestText() + ", name=" + this.getName() + ", description=" + this.getDescription() + ", properties=" + this.getProperties() + ", deleteAt=" + this.getDeleteAt() + ", trashAt=" + this.getTrashAt() + ", trashed=" + this.getTrashed() + ")";
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "items" })
+ public class CollectionList extends ItemList {
+
+ @JsonProperty("items")
+ private List<Collection> items;
+
+ public List<Collection> getItems() {
+ return this.items;
+ }
+
+ public void setItems(List<Collection> items) {
+ this.items = items;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.time.LocalDateTime;
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "command", "container_count", "container_count_max", "container_image", "container_uuid", "cwd", "environment", "expires_at",
+ "filters", "log_uuid", "mounts", "output_name", "output_path", "output_uuid", "output_ttl", "priority", "properties", "requesting_container_uuid",
+ "runtime_constraints", "scheduling_parameters", "state", "use_existing" })
+ public class Group extends Item {
+
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("group_class")
+ private String groupClass;
+ @JsonProperty("description")
+ private String description;
+ @JsonProperty("writable_by")
+ private List<String> writableBy;
+ @JsonProperty("delete_at")
+ private LocalDateTime deleteAt;
+ @JsonProperty("trash_at")
+ private LocalDateTime trashAt;
+ @JsonProperty("is_trashed")
+ private Boolean isTrashed;
+ @JsonProperty("command")
+ private List<String> command;
+ @JsonProperty("container_count")
+ private Integer containerCount;
+ @JsonProperty("container_count_max")
+ private Integer containerCountMax;
+ @JsonProperty("container_image")
+ private String containerImage;
+ @JsonProperty("container_uuid")
+ private String containerUuid;
+ @JsonProperty("cwd")
+ private String cwd;
+ @JsonProperty("environment")
+ private Object environment;
+ @JsonProperty("expires_at")
+ private LocalDateTime expiresAt;
+ @JsonProperty("filters")
+ private List<String> filters;
+ @JsonProperty("log_uuid")
+ private String logUuid;
+ @JsonProperty("mounts")
+ private Object mounts;
+ @JsonProperty("output_name")
+ private String outputName;
+ @JsonProperty("output_path")
+ private String outputPath;
+ @JsonProperty("output_uuid")
+ private String outputUuid;
+ @JsonProperty("output_ttl")
+ private Integer outputTtl;
+ @JsonProperty("priority")
+ private Integer priority;
+ @JsonProperty("properties")
+ private Object properties;
+ @JsonProperty("requesting_container_uuid")
+ private String requestingContainerUuid;
+ @JsonProperty("runtime_constraints")
+ private RuntimeConstraints runtimeConstraints;
+ @JsonProperty("scheduling_parameters")
+ private Object schedulingParameters;
+ @JsonProperty("state")
+ private String state;
+ @JsonProperty("use_existing")
+ private Boolean useExisting;
+
+ public String getName() {
+ return this.name;
+ }
+
+ public String getGroupClass() {
+ return this.groupClass;
+ }
+
+ public String getDescription() {
+ return this.description;
+ }
+
+ public List<String> getWritableBy() {
+ return this.writableBy;
+ }
+
+ public LocalDateTime getDeleteAt() {
+ return this.deleteAt;
+ }
+
+ public LocalDateTime getTrashAt() {
+ return this.trashAt;
+ }
+
+ public Boolean getIsTrashed() {
+ return this.isTrashed;
+ }
+
+ public List<String> getCommand() {
+ return this.command;
+ }
+
+ public Integer getContainerCount() {
+ return this.containerCount;
+ }
+
+ public Integer getContainerCountMax() {
+ return this.containerCountMax;
+ }
+
+ public String getContainerImage() {
+ return this.containerImage;
+ }
+
+ public String getContainerUuid() {
+ return this.containerUuid;
+ }
+
+ public String getCwd() {
+ return this.cwd;
+ }
+
+ public Object getEnvironment() {
+ return this.environment;
+ }
+
+ public LocalDateTime getExpiresAt() {
+ return this.expiresAt;
+ }
+
+ public List<String> getFilters() {
+ return this.filters;
+ }
+
+ public String getLogUuid() {
+ return this.logUuid;
+ }
+
+ public Object getMounts() {
+ return this.mounts;
+ }
+
+ public String getOutputName() {
+ return this.outputName;
+ }
+
+ public String getOutputPath() {
+ return this.outputPath;
+ }
+
+ public String getOutputUuid() {
+ return this.outputUuid;
+ }
+
+ public Integer getOutputTtl() {
+ return this.outputTtl;
+ }
+
+ public Integer getPriority() {
+ return this.priority;
+ }
+
+ public Object getProperties() {
+ return this.properties;
+ }
+
+ public String getRequestingContainerUuid() {
+ return this.requestingContainerUuid;
+ }
+
+ public RuntimeConstraints getRuntimeConstraints() {
+ return this.runtimeConstraints;
+ }
+
+ public Object getSchedulingParameters() {
+ return this.schedulingParameters;
+ }
+
+ public String getState() {
+ return this.state;
+ }
+
+ public Boolean getUseExisting() {
+ return this.useExisting;
+ }
+
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ public void setGroupClass(String groupClass) {
+ this.groupClass = groupClass;
+ }
+
+ public void setDescription(String description) {
+ this.description = description;
+ }
+
+ public void setWritableBy(List<String> writableBy) {
+ this.writableBy = writableBy;
+ }
+
+ public void setDeleteAt(LocalDateTime deleteAt) {
+ this.deleteAt = deleteAt;
+ }
+
+ public void setTrashAt(LocalDateTime trashAt) {
+ this.trashAt = trashAt;
+ }
+
+ public void setIsTrashed(Boolean isTrashed) {
+ this.isTrashed = isTrashed;
+ }
+
+ public void setCommand(List<String> command) {
+ this.command = command;
+ }
+
+ public void setContainerCount(Integer containerCount) {
+ this.containerCount = containerCount;
+ }
+
+ public void setContainerCountMax(Integer containerCountMax) {
+ this.containerCountMax = containerCountMax;
+ }
+
+ public void setContainerImage(String containerImage) {
+ this.containerImage = containerImage;
+ }
+
+ public void setContainerUuid(String containerUuid) {
+ this.containerUuid = containerUuid;
+ }
+
+ public void setCwd(String cwd) {
+ this.cwd = cwd;
+ }
+
+ public void setEnvironment(Object environment) {
+ this.environment = environment;
+ }
+
+ public void setExpiresAt(LocalDateTime expiresAt) {
+ this.expiresAt = expiresAt;
+ }
+
+ public void setFilters(List<String> filters) {
+ this.filters = filters;
+ }
+
+ public void setLogUuid(String logUuid) {
+ this.logUuid = logUuid;
+ }
+
+ public void setMounts(Object mounts) {
+ this.mounts = mounts;
+ }
+
+ public void setOutputName(String outputName) {
+ this.outputName = outputName;
+ }
+
+ public void setOutputPath(String outputPath) {
+ this.outputPath = outputPath;
+ }
+
+ public void setOutputUuid(String outputUuid) {
+ this.outputUuid = outputUuid;
+ }
+
+ public void setOutputTtl(Integer outputTtl) {
+ this.outputTtl = outputTtl;
+ }
+
+ public void setPriority(Integer priority) {
+ this.priority = priority;
+ }
+
+ public void setProperties(Object properties) {
+ this.properties = properties;
+ }
+
+ public void setRequestingContainerUuid(String requestingContainerUuid) {
+ this.requestingContainerUuid = requestingContainerUuid;
+ }
+
+ public void setRuntimeConstraints(RuntimeConstraints runtimeConstraints) {
+ this.runtimeConstraints = runtimeConstraints;
+ }
+
+ public void setSchedulingParameters(Object schedulingParameters) {
+ this.schedulingParameters = schedulingParameters;
+ }
+
+ public void setState(String state) {
+ this.state = state;
+ }
+
+ public void setUseExisting(Boolean useExisting) {
+ this.useExisting = useExisting;
+ }
+
+ public String toString() {
+ return "Group(name=" + this.getName() + ", groupClass=" + this.getGroupClass() + ", description=" + this.getDescription() + ", writableBy=" + this.getWritableBy() + ", deleteAt=" + this.getDeleteAt() + ", trashAt=" + this.getTrashAt() + ", isTrashed=" + this.getIsTrashed() + ", command=" + this.getCommand() + ", containerCount=" + this.getContainerCount() + ", containerCountMax=" + this.getContainerCountMax() + ", containerImage=" + this.getContainerImage() + ", containerUuid=" + this.getContainerUuid() + ", cwd=" + this.getCwd() + ", environment=" + this.getEnvironment() + ", expiresAt=" + this.getExpiresAt() + ", filters=" + this.getFilters() + ", logUuid=" + this.getLogUuid() + ", mounts=" + this.getMounts() + ", outputName=" + this.getOutputName() + ", outputPath=" + this.getOutputPath() + ", outputUuid=" + this.getOutputUuid() + ", outputTtl=" + this.getOutputTtl() + ", priority=" + this.getPriority() + ", properties=" + this.getProperties() + ", requestingContainerUuid=" + this.getRequestingContainerUuid() + ", runtimeConstraints=" + this.getRuntimeConstraints() + ", schedulingParameters=" + this.getSchedulingParameters() + ", state=" + this.getState() + ", useExisting=" + this.getUseExisting() + ")";
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "items" })
+ public class GroupList extends ItemList {
+
+ @JsonProperty("items")
+ private List<Group> items;
+
+ public List<Group> getItems() {
+ return this.items;
+ }
+
+ public void setItems(List<Group> items) {
+ this.items = items;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.time.LocalDateTime;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "href", "kind", "etag", "uuid", "owner_uuid", "created_at", "modified_by_client_uuid",
+ "modified_by_user_uuid", "modified_at", "updated_at" })
+ public abstract class Item {
+
+ @JsonProperty("href")
+ private String href;
+ @JsonProperty("kind")
+ private String kind;
+ @JsonProperty("etag")
+ private String etag;
+ @JsonProperty("uuid")
+ private String uuid;
+ @JsonProperty("owner_uuid")
+ private String ownerUuid;
+ @JsonProperty("created_at")
+ private LocalDateTime createdAt;
+ @JsonProperty("modified_by_client_uuid")
+ private String modifiedByClientUuid;
+ @JsonProperty("modified_by_user_uuid")
+ private String modifiedByUserUuid;
+ @JsonProperty("modified_at")
+ private LocalDateTime modifiedAt;
+ @JsonProperty("updated_at")
+ private LocalDateTime updatedAt;
+
+ public String getHref() {
+ return this.href;
+ }
+
+ public String getKind() {
+ return this.kind;
+ }
+
+ public String getEtag() {
+ return this.etag;
+ }
+
+ public String getUuid() {
+ return this.uuid;
+ }
+
+ public String getOwnerUuid() {
+ return this.ownerUuid;
+ }
+
+ public LocalDateTime getCreatedAt() {
+ return this.createdAt;
+ }
+
+ public String getModifiedByClientUuid() {
+ return this.modifiedByClientUuid;
+ }
+
+ public String getModifiedByUserUuid() {
+ return this.modifiedByUserUuid;
+ }
+
+ public LocalDateTime getModifiedAt() {
+ return this.modifiedAt;
+ }
+
+ public LocalDateTime getUpdatedAt() {
+ return this.updatedAt;
+ }
+
+ public void setHref(String href) {
+ this.href = href;
+ }
+
+ public void setKind(String kind) {
+ this.kind = kind;
+ }
+
+ public void setEtag(String etag) {
+ this.etag = etag;
+ }
+
+ public void setUuid(String uuid) {
+ this.uuid = uuid;
+ }
+
+ public void setOwnerUuid(String ownerUuid) {
+ this.ownerUuid = ownerUuid;
+ }
+
+ public void setCreatedAt(LocalDateTime createdAt) {
+ this.createdAt = createdAt;
+ }
+
+ public void setModifiedByClientUuid(String modifiedByClientUuid) {
+ this.modifiedByClientUuid = modifiedByClientUuid;
+ }
+
+ public void setModifiedByUserUuid(String modifiedByUserUuid) {
+ this.modifiedByUserUuid = modifiedByUserUuid;
+ }
+
+ public void setModifiedAt(LocalDateTime modifiedAt) {
+ this.modifiedAt = modifiedAt;
+ }
+
+ public void setUpdatedAt(LocalDateTime updatedAt) {
+ this.updatedAt = updatedAt;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "kind", "etag", "self_link", "offset", "limit", "items_available" })
+ public class ItemList {
+
+ @JsonProperty("kind")
+ private String kind;
+ @JsonProperty("etag")
+ private String etag;
+ @JsonProperty("self_link")
+ private String selfLink;
+ @JsonProperty("offset")
+ private Object offset;
+ @JsonProperty("limit")
+ private Object limit;
+ @JsonProperty("items_available")
+ private Integer itemsAvailable;
+
+ public String getKind() {
+ return this.kind;
+ }
+
+ public String getEtag() {
+ return this.etag;
+ }
+
+ public String getSelfLink() {
+ return this.selfLink;
+ }
+
+ public Object getOffset() {
+ return this.offset;
+ }
+
+ public Object getLimit() {
+ return this.limit;
+ }
+
+ public Integer getItemsAvailable() {
+ return this.itemsAvailable;
+ }
+
+ public void setKind(String kind) {
+ this.kind = kind;
+ }
+
+ public void setEtag(String etag) {
+ this.etag = etag;
+ }
+
+ public void setSelfLink(String selfLink) {
+ this.selfLink = selfLink;
+ }
+
+ public void setOffset(Object offset) {
+ this.offset = offset;
+ }
+
+ public void setLimit(Object limit) {
+ this.limit = limit;
+ }
+
+ public void setItemsAvailable(Integer itemsAvailable) {
+ this.itemsAvailable = itemsAvailable;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.*;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "service_host", "service_port", "service_ssl_flag", "service_type", "read_only" })
+ public class KeepService extends Item {
+
+ @JsonProperty("service_host")
+ private String serviceHost;
+ @JsonProperty("service_port")
+ private Integer servicePort;
+ @JsonProperty("service_ssl_flag")
+ private Boolean serviceSslFlag;
+ @JsonProperty("service_type")
+ private String serviceType;
+ @JsonProperty("read_only")
+ private Boolean readOnly;
+ @JsonIgnore
+ private String serviceRoot;
+
+ public String getServiceHost() {
+ return this.serviceHost;
+ }
+
+ public Integer getServicePort() {
+ return this.servicePort;
+ }
+
+ public Boolean getServiceSslFlag() {
+ return this.serviceSslFlag;
+ }
+
+ public String getServiceType() {
+ return this.serviceType;
+ }
+
+ public Boolean getReadOnly() {
+ return this.readOnly;
+ }
+
+ public String getServiceRoot() {
+ return this.serviceRoot;
+ }
+
+ public void setServiceHost(String serviceHost) {
+ this.serviceHost = serviceHost;
+ }
+
+ public void setServicePort(Integer servicePort) {
+ this.servicePort = servicePort;
+ }
+
+ public void setServiceSslFlag(Boolean serviceSslFlag) {
+ this.serviceSslFlag = serviceSslFlag;
+ }
+
+ public void setServiceType(String serviceType) {
+ this.serviceType = serviceType;
+ }
+
+ public void setReadOnly(Boolean readOnly) {
+ this.readOnly = readOnly;
+ }
+
+ public void setServiceRoot(String serviceRoot) {
+ this.serviceRoot = serviceRoot;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "items" })
+ public class KeepServiceList extends ItemList {
+
+ @JsonProperty("items")
+ private List<KeepService> items;
+
+ public List<KeepService> getItems() {
+ return this.items;
+ }
+
+ public void setItems(List<KeepService> items) {
+ this.items = items;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "API", "vcpus", "ram", "keep_cache_ram" })
+ public class RuntimeConstraints {
+
+ @JsonProperty("API")
+ private Boolean api;
+ @JsonProperty("vcpus")
+ private Integer vcpus;
+ @JsonProperty("ram")
+ private Long ram;
+ @JsonProperty("keep_cache_ram")
+ private Long keepCacheRam;
+
+ public Boolean getApi() {
+ return this.api;
+ }
+
+ public Integer getVcpus() {
+ return this.vcpus;
+ }
+
+ public Long getRam() {
+ return this.ram;
+ }
+
+ public Long getKeepCacheRam() {
+ return this.keepCacheRam;
+ }
+
+ public void setApi(Boolean api) {
+ this.api = api;
+ }
+
+ public void setVcpus(Integer vcpus) {
+ this.vcpus = vcpus;
+ }
+
+ public void setRam(Long ram) {
+ this.ram = ram;
+ }
+
+ public void setKeepCacheRam(Long keepCacheRam) {
+ this.keepCacheRam = keepCacheRam;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "email", "username", "full_name", "first_name", "last_name", "identity_url", "is_active", "is_admin", "is_invited",
+ "prefs", "writable_by", "default_owner_uuid" })
+ public class User extends Item {
+
+ @JsonProperty("email")
+ private String email;
+ @JsonProperty("username")
+ private String username;
+ @JsonProperty("full_name")
+ private String fullName;
+ @JsonProperty("first_name")
+ private String firstName;
+ @JsonProperty("last_name")
+ private String lastName;
+ @JsonProperty("identity_url")
+ private String identityUrl;
+ @JsonProperty("is_active")
+ private Boolean isActive;
+ @JsonProperty("is_admin")
+ private Boolean isAdmin;
+ @JsonProperty("is_invited")
+ private Boolean isInvited;
+ @JsonProperty("prefs")
+ private Object prefs;
+ @JsonProperty("writable_by")
+ private List<String> writableBy;
+ @JsonProperty("default_owner_uuid")
+ private Boolean defaultOwnerUuid;
+
+ public String getEmail() {
+ return this.email;
+ }
+
+ public String getUsername() {
+ return this.username;
+ }
+
+ public String getFullName() {
+ return this.fullName;
+ }
+
+ public String getFirstName() {
+ return this.firstName;
+ }
+
+ public String getLastName() {
+ return this.lastName;
+ }
+
+ public String getIdentityUrl() {
+ return this.identityUrl;
+ }
+
+ public Boolean getIsActive() {
+ return this.isActive;
+ }
+
+ public Boolean getIsAdmin() {
+ return this.isAdmin;
+ }
+
+ public Boolean getIsInvited() {
+ return this.isInvited;
+ }
+
+ public Object getPrefs() {
+ return this.prefs;
+ }
+
+ public List<String> getWritableBy() {
+ return this.writableBy;
+ }
+
+ public Boolean getDefaultOwnerUuid() {
+ return this.defaultOwnerUuid;
+ }
+
+ public void setEmail(String email) {
+ this.email = email;
+ }
+
+ public void setUsername(String username) {
+ this.username = username;
+ }
+
+ public void setFullName(String fullName) {
+ this.fullName = fullName;
+ }
+
+ public void setFirstName(String firstName) {
+ this.firstName = firstName;
+ }
+
+ public void setLastName(String lastName) {
+ this.lastName = lastName;
+ }
+
+ public void setIdentityUrl(String identityUrl) {
+ this.identityUrl = identityUrl;
+ }
+
+ public void setIsActive(Boolean isActive) {
+ this.isActive = isActive;
+ }
+
+ public void setIsAdmin(Boolean isAdmin) {
+ this.isAdmin = isAdmin;
+ }
+
+ public void setIsInvited(Boolean isInvited) {
+ this.isInvited = isInvited;
+ }
+
+ public void setPrefs(Object prefs) {
+ this.prefs = prefs;
+ }
+
+ public void setWritableBy(List<String> writableBy) {
+ this.writableBy = writableBy;
+ }
+
+ public void setDefaultOwnerUuid(Boolean defaultOwnerUuid) {
+ this.defaultOwnerUuid = defaultOwnerUuid;
+ }
+
+ public String toString() {
+ return "User(email=" + this.getEmail() + ", username=" + this.getUsername() + ", fullName=" + this.getFullName() + ", firstName=" + this.getFirstName() + ", lastName=" + this.getLastName() + ", identityUrl=" + this.getIdentityUrl() + ", isActive=" + this.getIsActive() + ", isAdmin=" + this.getIsAdmin() + ", isInvited=" + this.getIsInvited() + ", prefs=" + this.getPrefs() + ", writableBy=" + this.getWritableBy() + ", defaultOwnerUuid=" + this.getDefaultOwnerUuid() + ")";
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model;
+
+ import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ @JsonPropertyOrder({ "items" })
+ public class UserList extends ItemList {
+
+ @JsonProperty("items")
+ private List<User> items;
+
+ public List<User> getItems() {
+ return this.items;
+ }
+
+ public void setItems(List<User> items) {
+ this.items = items;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model.argument;
+
+ import com.fasterxml.jackson.annotation.JsonIgnore;
+
+ public abstract class Argument {
+
+ @JsonIgnore
+ private String uuid;
+
+ public String getUuid() {
+ return this.uuid;
+ }
+
+ public void setUuid(String uuid) {
+ this.uuid = uuid;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model.argument;
+
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonPropertyOrder({ "limit", "order", "filters", "recursive" })
+ public class ContentsGroup extends Argument {
+
+ @JsonProperty("limit")
+ private Integer limit;
+
+ @JsonProperty("order")
+ private String order;
+
+ @JsonProperty("filters")
+ private List<String> filters;
+
+ @JsonProperty("recursive")
+ private Boolean recursive;
+
+ public Integer getLimit() {
+ return this.limit;
+ }
+
+ public String getOrder() {
+ return this.order;
+ }
+
+ public List<String> getFilters() {
+ return this.filters;
+ }
+
+ public Boolean getRecursive() {
+ return this.recursive;
+ }
+
+ public void setLimit(Integer limit) {
+ this.limit = limit;
+ }
+
+ public void setOrder(String order) {
+ this.order = order;
+ }
+
+ public void setFilters(List<String> filters) {
+ this.filters = filters;
+ }
+
+ public void setRecursive(Boolean recursive) {
+ this.recursive = recursive;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model.argument;
+
+ import com.fasterxml.jackson.annotation.JsonFormat;
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ @JsonFormat(shape = JsonFormat.Shape.ARRAY)
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonPropertyOrder({ "attribute", "operator", "operand" })
+ public class Filter {
+
+ @JsonProperty("attribute")
+ private String attribute;
+
+ @JsonProperty("operator")
+ private Operator operator;
+
+ @JsonProperty("operand")
+ private Object operand;
+
+ private Filter(String attribute, Operator operator, Object operand) {
+ this.attribute = attribute;
+ this.operator = operator;
+ this.operand = operand;
+ }
+
+ public static Filter of(String attribute, Operator operator, Object operand) {
+ return new Filter(attribute, operator, operand);
+ }
+
+ public String getAttribute() {
+ return this.attribute;
+ }
+
+ public Operator getOperator() {
+ return this.operator;
+ }
+
+ public Object getOperand() {
+ return this.operand;
+ }
+
+ public boolean equals(Object o) {
+ if (o == this) return true;
+ if (!(o instanceof Filter)) return false;
+ final Filter other = (Filter) o;
+ final Object this$attribute = this.getAttribute();
+ final Object other$attribute = other.getAttribute();
+ if (this$attribute == null ? other$attribute != null : !this$attribute.equals(other$attribute)) return false;
+ final Object this$operator = this.getOperator();
+ final Object other$operator = other.getOperator();
+ if (this$operator == null ? other$operator != null : !this$operator.equals(other$operator)) return false;
+ final Object this$operand = this.getOperand();
+ final Object other$operand = other.getOperand();
+ if (this$operand == null ? other$operand != null : !this$operand.equals(other$operand)) return false;
+ return true;
+ }
+
+ public int hashCode() {
+ final int PRIME = 59;
+ int result = 1;
+ final Object $attribute = this.getAttribute();
+ result = result * PRIME + ($attribute == null ? 43 : $attribute.hashCode());
+ final Object $operator = this.getOperator();
+ result = result * PRIME + ($operator == null ? 43 : $operator.hashCode());
+ final Object $operand = this.getOperand();
+ result = result * PRIME + ($operand == null ? 43 : $operand.hashCode());
+ return result;
+ }
+
+ public String toString() {
+ return "Filter(attribute=" + this.getAttribute() + ", operator=" + this.getOperator() + ", operand=" + this.getOperand() + ")";
+ }
+
+ public enum Operator {
+
+ @JsonProperty("<")
+ LESS,
+
+ @JsonProperty("<=")
+ LESS_EQUALS,
+
+ @JsonProperty(">=")
+ MORE_EQUALS,
+
+ @JsonProperty(">")
+ MORE,
+
+ @JsonProperty("like")
+ LIKE,
+
+ @JsonProperty("ilike")
+ ILIKE,
+
+ @JsonProperty("=")
+ EQUALS,
+
+ @JsonProperty("!=")
+ NOT_EQUALS,
+
+ @JsonProperty("in")
+ IN,
+
+ @JsonProperty("not in")
+ NOT_IN,
+
+ @JsonProperty("is_a")
+ IS_A
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model.argument;
+
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ import java.util.List;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonPropertyOrder({ "limit", "offset", "filters", "order", "select", "distinct", "count" })
+ public class ListArgument extends Argument {
+
+ @JsonProperty("limit")
+ private Integer limit;
+
+ @JsonProperty("offset")
+ private Integer offset;
+
+ @JsonProperty("filters")
+ private List<Filter> filters;
+
+ @JsonProperty("order")
+ private List<String> order;
+
+ @JsonProperty("select")
+ private List<String> select;
+
+ @JsonProperty("distinct")
+ private Boolean distinct;
+
+ @JsonProperty("count")
+ private Count count;
+
+
+ ListArgument(Integer limit, Integer offset, List<Filter> filters, List<String> order, List<String> select, Boolean distinct, Count count) {
+ this.limit = limit;
+ this.offset = offset;
+ this.filters = filters;
+ this.order = order;
+ this.select = select;
+ this.distinct = distinct;
+ this.count = count;
+ }
+
+ public static ListArgumentBuilder builder() {
+ return new ListArgumentBuilder();
+ }
+
+ public enum Count {
+
+ @JsonProperty("exact")
+ EXACT,
+
+ @JsonProperty("none")
+ NONE
+ }
+
+ public static class ListArgumentBuilder {
+ private Integer limit;
+ private Integer offset;
+ private List<Filter> filters;
+ private List<String> order;
+ private List<String> select;
+ private Boolean distinct;
+ private Count count;
+
+ ListArgumentBuilder() {
+ }
+
+ public ListArgumentBuilder limit(Integer limit) {
+ this.limit = limit;
+ return this;
+ }
+
+ public ListArgumentBuilder offset(Integer offset) {
+ this.offset = offset;
+ return this;
+ }
+
+ public ListArgumentBuilder filters(List<Filter> filters) {
+ this.filters = filters;
+ return this;
+ }
+
+ public ListArgumentBuilder order(List<String> order) {
+ this.order = order;
+ return this;
+ }
+
+ public ListArgumentBuilder select(List<String> select) {
+ this.select = select;
+ return this;
+ }
+
+ public ListArgumentBuilder distinct(Boolean distinct) {
+ this.distinct = distinct;
+ return this;
+ }
+
+ public ListArgumentBuilder count(Count count) {
+ this.count = count;
+ return this;
+ }
+
+ public ListArgument build() {
+ return new ListArgument(limit, offset, filters, order, select, distinct, count);
+ }
+
+ public String toString() {
+ return "ListArgument.ListArgumentBuilder(limit=" + this.limit +
+ ", offset=" + this.offset + ", filters=" + this.filters +
+ ", order=" + this.order + ", select=" + this.select +
+ ", distinct=" + this.distinct + ", count=" + this.count + ")";
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.model.argument;
+
+ import com.fasterxml.jackson.annotation.JsonInclude;
+ import com.fasterxml.jackson.annotation.JsonProperty;
+ import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+ @JsonInclude(JsonInclude.Include.NON_NULL)
+ @JsonPropertyOrder({ "ensure_unique_name" })
+ public class UntrashGroup extends Argument {
+
+ @JsonProperty("ensure_unique_name")
+ private Boolean ensureUniqueName;
+
+ public Boolean getEnsureUniqueName() {
+ return this.ensureUniqueName;
+ }
+
+ public void setEnsureUniqueName(Boolean ensureUniqueName) {
+ this.ensureUniqueName = ensureUniqueName;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.common;
+
+ public final class Characters {
+
+ private Characters() {}
+
+ public static final String SPACE = "\\040";
+ public static final String NEW_LINE = "\n";
+ public static final String SLASH = "/";
+ public static final String DOT = ".";
+ public static final String COLON = ":";
+ public static final String PERCENT = "%";
+ public static final String QUOTE = "\"";
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.common;
+
+ public final class Headers {
+
+ private Headers() {}
+
+ public static final String X_KEEP_DESIRED_REPLICAS = "X-Keep-Desired-Replicas";
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.common;
+
+ public final class Patterns {
+
+ public static final String HINT_PATTERN = "^[A-Z][A-Za-z0-9@_-]+$";
+ public static final String FILE_TOKEN_PATTERN = "(\\d+:\\d+:\\S+)";
+ public static final String LOCATOR_PATTERN = "([0-9a-f]{32})\\+([0-9]+)(\\+[A-Z][-A-Za-z0-9@_]*)*";
+ public static final String GROUP_UUID_PATTERN = "[a-z0-9]{5}-j7d0g-[a-z0-9]{15}";
+ public static final String USER_UUID_PATTERN = "[a-z0-9]{5}-tpzed-[a-z0-9]{15}";
+
+ private Patterns() {}
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.config;
+
+ import java.io.File;
+
+ public interface ConfigProvider {
+
+ //API
+ boolean isApiHostInsecure();
+
+ String getKeepWebHost();
+
+ int getKeepWebPort();
+
+ String getApiHost();
+
+ int getApiPort();
+
+ String getApiToken();
+
+ String getApiProtocol();
+
+
+ //FILE UPLOAD
+ int getFileSplitSize();
+
+ File getFileSplitDirectory();
+
+ int getNumberOfCopies();
+
+ int getNumberOfRetries();
+
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.config;
+
+ import java.io.File;
+
+ public class ExternalConfigProvider implements ConfigProvider {
+
+ private boolean apiHostInsecure;
+ private String keepWebHost;
+ private int keepWebPort;
+ private String apiHost;
+ private int apiPort;
+ private String apiToken;
+ private String apiProtocol;
+ private int fileSplitSize;
+ private File fileSplitDirectory;
+ private int numberOfCopies;
+ private int numberOfRetries;
+
+ ExternalConfigProvider(boolean apiHostInsecure, String keepWebHost, int keepWebPort, String apiHost, int apiPort, String apiToken, String apiProtocol, int fileSplitSize, File fileSplitDirectory, int numberOfCopies, int numberOfRetries) {
+ this.apiHostInsecure = apiHostInsecure;
+ this.keepWebHost = keepWebHost;
+ this.keepWebPort = keepWebPort;
+ this.apiHost = apiHost;
+ this.apiPort = apiPort;
+ this.apiToken = apiToken;
+ this.apiProtocol = apiProtocol;
+ this.fileSplitSize = fileSplitSize;
+ this.fileSplitDirectory = fileSplitDirectory;
+ this.numberOfCopies = numberOfCopies;
+ this.numberOfRetries = numberOfRetries;
+ }
+
+ public static ExternalConfigProviderBuilder builder() {
+ return new ExternalConfigProviderBuilder();
+ }
+
+ @Override
+ public String toString() {
+ return "ExternalConfigProvider{" +
+ "apiHostInsecure=" + apiHostInsecure +
+ ", keepWebHost='" + keepWebHost + '\'' +
+ ", keepWebPort=" + keepWebPort +
+ ", apiHost='" + apiHost + '\'' +
+ ", apiPort=" + apiPort +
+ ", apiToken='" + apiToken + '\'' +
+ ", apiProtocol='" + apiProtocol + '\'' +
+ ", fileSplitSize=" + fileSplitSize +
+ ", fileSplitDirectory=" + fileSplitDirectory +
+ ", numberOfCopies=" + numberOfCopies +
+ ", numberOfRetries=" + numberOfRetries +
+ '}';
+ }
+
+ public boolean isApiHostInsecure() {
+ return this.apiHostInsecure;
+ }
+
+ public String getKeepWebHost() {
+ return this.keepWebHost;
+ }
+
+ public int getKeepWebPort() {
+ return this.keepWebPort;
+ }
+
+ public String getApiHost() {
+ return this.apiHost;
+ }
+
+ public int getApiPort() {
+ return this.apiPort;
+ }
+
+ public String getApiToken() {
+ return this.apiToken;
+ }
+
+ public String getApiProtocol() {
+ return this.apiProtocol;
+ }
+
+ public int getFileSplitSize() {
+ return this.fileSplitSize;
+ }
+
+ public File getFileSplitDirectory() {
+ return this.fileSplitDirectory;
+ }
+
+ public int getNumberOfCopies() {
+ return this.numberOfCopies;
+ }
+
+ public int getNumberOfRetries() {
+ return this.numberOfRetries;
+ }
+
+ public static class ExternalConfigProviderBuilder {
+ private boolean apiHostInsecure;
+ private String keepWebHost;
+ private int keepWebPort;
+ private String apiHost;
+ private int apiPort;
+ private String apiToken;
+ private String apiProtocol;
+ private int fileSplitSize;
+ private File fileSplitDirectory;
+ private int numberOfCopies;
+ private int numberOfRetries;
+
+ ExternalConfigProviderBuilder() {
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder apiHostInsecure(boolean apiHostInsecure) {
+ this.apiHostInsecure = apiHostInsecure;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder keepWebHost(String keepWebHost) {
+ this.keepWebHost = keepWebHost;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder keepWebPort(int keepWebPort) {
+ this.keepWebPort = keepWebPort;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder apiHost(String apiHost) {
+ this.apiHost = apiHost;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder apiPort(int apiPort) {
+ this.apiPort = apiPort;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder apiToken(String apiToken) {
+ this.apiToken = apiToken;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder apiProtocol(String apiProtocol) {
+ this.apiProtocol = apiProtocol;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder fileSplitSize(int fileSplitSize) {
+ this.fileSplitSize = fileSplitSize;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder fileSplitDirectory(File fileSplitDirectory) {
+ this.fileSplitDirectory = fileSplitDirectory;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder numberOfCopies(int numberOfCopies) {
+ this.numberOfCopies = numberOfCopies;
+ return this;
+ }
+
+ public ExternalConfigProvider.ExternalConfigProviderBuilder numberOfRetries(int numberOfRetries) {
+ this.numberOfRetries = numberOfRetries;
+ return this;
+ }
+
+ public ExternalConfigProvider build() {
+ return new ExternalConfigProvider(apiHostInsecure, keepWebHost, keepWebPort, apiHost, apiPort, apiToken, apiProtocol, fileSplitSize, fileSplitDirectory, numberOfCopies, numberOfRetries);
+ }
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.config;
+
+ import com.typesafe.config.Config;
+ import com.typesafe.config.ConfigFactory;
+
+ import java.io.File;
+
+ public class FileConfigProvider implements ConfigProvider {
+
+ private static final String DEFAULT_PATH = "arvados";
+ private final Config config;
+
+ public FileConfigProvider() {
+ config = ConfigFactory.load().getConfig(DEFAULT_PATH);
+ }
+
+ public FileConfigProvider(final String configFile) {
+ config = (configFile != null) ?
+ ConfigFactory.load(configFile).getConfig(DEFAULT_PATH) : ConfigFactory.load().getConfig(DEFAULT_PATH);
+ }
+
+ public Config getConfig() {
+ return config;
+ }
+
+ private File getFile(String path) {
+ return new File(config.getString(path));
+ }
+
+ private int getInt(String path) {
+ return config.getInt(path);
+ }
+
+ private boolean getBoolean(String path) {
+ return config.getBoolean(path);
+ }
+
+ private String getString(String path) {
+ return config.getString(path);
+ }
+
+ @Override
+ public boolean isApiHostInsecure() {
+ return this.getBoolean("api.host-insecure");
+ }
+
+ @Override
+ public String getKeepWebHost() {
+ return this.getString("api.keepweb-host");
+ }
+
+ @Override
+ public int getKeepWebPort() {
+ return this.getInt("api.keepweb-port");
+ }
+
+ @Override
+ public String getApiHost() {
+ return this.getString("api.host");
+ }
+
+ @Override
+ public int getApiPort() {
+ return this.getInt("api.port");
+ }
+
+ @Override
+ public String getApiToken() {
+ return this.getString("api.token");
+ }
+
+ @Override
+ public String getApiProtocol() {
+ return this.getString("api.protocol");
+ }
+
+ @Override
+ public int getFileSplitSize() {
+ return this.getInt("split-size");
+ }
+
+ @Override
+ public File getFileSplitDirectory() {
+ return this.getFile("temp-dir");
+ }
+
+ @Override
+ public int getNumberOfCopies() {
+ return this.getInt("copies");
+ }
+
+ @Override
+ public int getNumberOfRetries() {
+ return this.getInt("retries");
+ }
+
+ public String getIntegrationTestProjectUuid() {
+ return this.getString("integration-tests.project-uuid");
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.exception;
+
+ public class ArvadosApiException extends ArvadosClientException {
+
+ private static final long serialVersionUID = 1L;
+
+ public ArvadosApiException(String message) {
+ super(message);
+ }
+
+ public ArvadosApiException(String message, Throwable cause) {
+ super(message, cause);
+ }
+
+ public ArvadosApiException(Throwable cause) {
+ super(cause);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.exception;
+
+ /**
+ * Parent exception for all exceptions in library.
+ * More specific exceptions like ArvadosApiException extend this class.
+ */
+ public class ArvadosClientException extends RuntimeException {
+
+ public ArvadosClientException(String message) {
+ super(message);
+ }
+
+ public ArvadosClientException(String message, Throwable cause) {
+ super(message, cause);
+ }
+
+ public ArvadosClientException(Throwable cause) {
+ super(cause);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.facade;
+
+ import com.google.common.collect.Lists;
+ import org.arvados.client.api.client.CollectionsApiClient;
+ import org.arvados.client.api.client.GroupsApiClient;
+ import org.arvados.client.api.client.KeepWebApiClient;
+ import org.arvados.client.api.client.UsersApiClient;
+ import org.arvados.client.api.model.*;
+ import org.arvados.client.api.model.argument.Filter;
+ import org.arvados.client.api.model.argument.ListArgument;
+ import org.arvados.client.config.FileConfigProvider;
+ import org.arvados.client.config.ConfigProvider;
+ import org.arvados.client.logic.collection.FileToken;
+ import org.arvados.client.logic.collection.ManifestDecoder;
+ import org.arvados.client.logic.keep.FileDownloader;
+ import org.arvados.client.logic.keep.FileUploader;
+ import org.arvados.client.logic.keep.KeepClient;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.util.Arrays;
+ import java.util.Collections;
+ import java.util.List;
+
+ public class ArvadosFacade {
+
+ private final ConfigProvider config;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(ArvadosFacade.class);
+ private CollectionsApiClient collectionsApiClient;
+ private GroupsApiClient groupsApiClient;
+ private UsersApiClient usersApiClient;
+ private FileDownloader fileDownloader;
+ private FileUploader fileUploader;
+ private static final String PROJECT = "project";
+ private static final String SUBPROJECT = "sub-project";
+
+ public ArvadosFacade(ConfigProvider config) {
+ this.config = config;
+ setFacadeFields();
+ }
+
+ public ArvadosFacade() {
+ this.config = new FileConfigProvider();
+ setFacadeFields();
+ }
+
+ private void setFacadeFields() {
+ collectionsApiClient = new CollectionsApiClient(config);
+ groupsApiClient = new GroupsApiClient(config);
+ usersApiClient = new UsersApiClient(config);
+ KeepClient keepClient = new KeepClient(config);
+ ManifestDecoder manifestDecoder = new ManifestDecoder();
+ KeepWebApiClient keepWebApiClient = new KeepWebApiClient(config);
+ fileDownloader = new FileDownloader(keepClient, manifestDecoder, collectionsApiClient, keepWebApiClient);
+ fileUploader = new FileUploader(keepClient, collectionsApiClient, config);
+ }
+
+ /**
+ * This method downloads single file from collection using Arvados Keep-Web.
+ * File is saved on a drive in specified location and returned.
+ *
+ * @param filePathName path to the file in collection. If requested file is stored
+ * directly in collection (not within its subdirectory) this
+ * would be just the name of file (ex. 'file.txt').
+ * Otherwise full file path must be passed (ex. 'folder/file.txt')
+ * @param collectionUuid uuid of collection containing requested file
+ * @param pathToDownloadFolder path to location in which file should be saved.
+ * Passed location must be a directory in which file of
+ * that name does not already exist.
+ * @return downloaded file
+ */
+ public File downloadFile(String filePathName, String collectionUuid, String pathToDownloadFolder) {
+ return fileDownloader.downloadSingleFileUsingKeepWeb(filePathName, collectionUuid, pathToDownloadFolder);
+ }
+
+ /**
+ * This method downloads all files from collection.
+ * Directory named by collection uuid is created in specified location,
+ * files are saved on a drive in this directory and list with downloaded
+ * files is returned.
+ *
+ * @param collectionUuid uuid of collection from which files are downloaded
+ * @param pathToDownloadFolder path to location in which files should be saved.
+ * New folder named by collection uuid, containing
+ * downloaded files, is created in this location.
+ * Passed location must be a directory in which folder
+ * of that name does not already exist.
+ * @param usingKeepWeb if set to true files will be downloaded using Keep Web.
+ * If set to false files will be downloaded using Keep Server API.
+ * @return list containing downloaded files
+ */
+ public List<File> downloadCollectionFiles(String collectionUuid, String pathToDownloadFolder, boolean usingKeepWeb) {
+ if (usingKeepWeb)
+ return fileDownloader.downloadFilesFromCollectionUsingKeepWeb(collectionUuid, pathToDownloadFolder);
+ return fileDownloader.downloadFilesFromCollection(collectionUuid, pathToDownloadFolder);
+ }
+
+ /**
+ * Lists all FileTokens (objects containing information about files) for
+ * specified collection.
+ * Information in each FileToken includes file path, name, size and position
+ * in data stream
+ *
+ * @param collectionUuid uuid of collection for which FileTokens are listed
+ * @return list containing FileTokens for each file in specified collection
+ */
+ public List<FileToken> listFileInfoFromCollection(String collectionUuid) {
+ return fileDownloader.listFileInfoFromCollection(collectionUuid);
+ }
+
+ /**
+ * Creates and uploads new collection containing passed files.
+ * Created collection has a default name and is uploaded to user's 'Home' project.
+ *
+ * @see ArvadosFacade#upload(List, String, String)
+ */
+ public Collection upload(List<File> files) {
+ return upload(files, null, null);
+ }
+
+ /**
+ * Creates and uploads new collection containing a single file.
+ * Created collection has a default name and is uploaded to user's 'Home' project.
+ *
+ * @see ArvadosFacade#upload(List, String, String)
+ */
+ public Collection upload(File file) {
+ return upload(Collections.singletonList(file), null, null);
+ }
+
+ /**
+ * Uploads new collection with specified name and containing selected files
+ * to an existing project.
+ *
+ * @param sourceFiles list of files to be uploaded within new collection
+ * @param collectionName name for the newly created collection.
+ * Collection with that name cannot be already created
+ * in specified project. If null is passed
+ * then collection name is set to default, containing
+ * phrase 'New Collection' and a timestamp.
+ * @param projectUuid uuid of the project in which created collection is to be included.
+ * If null is passed then collection is uploaded to user's 'Home' project.
+ * @return collection object mapped from JSON that is returned from server after successful upload
+ */
+ public Collection upload(List<File> sourceFiles, String collectionName, String projectUuid) {
+ return fileUploader.upload(sourceFiles, collectionName, projectUuid);
+ }
+
+ /**
+ * Uploads a file to a specified collection.
+ *
+ * @see ArvadosFacade#uploadToExistingCollection(List, String)
+ */
+ public Collection uploadToExistingCollection(File file, String collectionUUID) {
+ return fileUploader.uploadToExistingCollection(Collections.singletonList(file), collectionUUID);
+ }
+
+ /**
+ * Uploads multiple files to an existing collection.
+ *
+ * @param files list of files to be uploaded to existing collection.
+ * File names must be unique - both within passed list and
+ * in comparison with files already existing within collection.
+ * @param collectionUUID UUID of collection to which files should be uploaded
+ * @return collection object mapped from JSON that is returned from server after successful upload
+ */
+ public Collection uploadToExistingCollection(List<File> files, String collectionUUID) {
+ return fileUploader.uploadToExistingCollection(files, collectionUUID);
+ }
+
+ /**
+ * Creates and uploads new empty collection to specified project.
+ *
+ * @param collectionName name for the newly created collection.
+ * Collection with that name cannot be already created
+ * in specified project.
+ * @param projectUuid uuid of project that will contain uploaded empty collection.
+ * To select home project pass current user's uuid from getCurrentUser()
+ * @return collection object mapped from JSON that is returned from server after successful upload
+ * @see ArvadosFacade#getCurrentUser()
+ */
+ public Collection createEmptyCollection(String collectionName, String projectUuid) {
+ Collection collection = new Collection();
+ collection.setOwnerUuid(projectUuid);
+ collection.setName(collectionName);
+ return collectionsApiClient.create(collection);
+ }
+
+ /**
+ * Returns current user information based on Api Token provided via configuration
+ *
+ * @return user object mapped from JSON that is returned from server based on provided Api Token.
+ * It contains information about user who has this token assigned.
+ */
+ public User getCurrentUser() {
+ return usersApiClient.current();
+ }
+
+ /**
+ * Gets uuid of current user based on api Token provided in configuration and uses it to list all
+ * projects that this user owns in Arvados.
+ *
+ * @return GroupList containing all groups that current user is owner of.
+ * @see ArvadosFacade#getCurrentUser()
+ */
+ public GroupList showGroupsOwnedByCurrentUser() {
+ ListArgument listArgument = ListArgument.builder()
+ .filters(Arrays.asList(
+ Filter.of("owner_uuid", Filter.Operator.LIKE, getCurrentUser().getUuid()),
+ Filter.of("group_class", Filter.Operator.IN, Lists.newArrayList(PROJECT, SUBPROJECT)
+ )))
+ .build();
+ GroupList groupList = groupsApiClient.list(listArgument);
+ log.debug("Groups owned by user:");
+ groupList.getItems().forEach(m -> log.debug(m.getUuid() + " -- " + m.getName()));
+
+ return groupList;
+ }
+
+ /**
+ * Gets uuid of current user based on api Token provided in configuration and uses it to list all
+ * projects that this user has read access to in Arvados.
+ *
+ * @return GroupList containing all groups that current user has read access to.
+ */
+ public GroupList showGroupsAccessibleByCurrentUser() {
+ ListArgument listArgument = ListArgument.builder()
+ .filters(Collections.singletonList(
+ Filter.of("group_class", Filter.Operator.IN, Lists.newArrayList(PROJECT, SUBPROJECT)
+ )))
+ .build();
+ GroupList groupList = groupsApiClient.list(listArgument);
+ log.debug("Groups accessible by user:");
+ groupList.getItems().forEach(m -> log.debug(m.getUuid() + " -- " + m.getName()));
+
+ return groupList;
+ }
+
+ /**
+ * Filters all collections from selected project and returns list of those that contain passed String in their name.
+ * Operator "LIKE" is used so in order to obtain certain collection it is sufficient to pass just part of its name.
+ * Returned collections in collectionList are ordered by date of creation (starting from oldest one).
+ *
+ * @param collectionName collections containing this param in their name will be returned.
+ * Passing a wildcard is possible - for example passing "a%" searches for
+ * all collections starting with "a".
+ * @param projectUuid uuid of project in which will be searched for collections with given name. To search home
+ * project provide user uuid (from getCurrentUser())
+ * @return object CollectionList containing all collections matching specified name criteria
+ * @see ArvadosFacade#getCurrentUser()
+ */
+ public CollectionList getCollectionsFromProjectByName(String collectionName, String projectUuid) {
+ ListArgument listArgument = ListArgument.builder()
+ .filters(Arrays.asList(
+ Filter.of("owner_uuid", Filter.Operator.LIKE, projectUuid),
+ Filter.of("name", Filter.Operator.LIKE, collectionName)
+ ))
+ .order(Collections.singletonList("created_at"))
+ .build();
+
+ return collectionsApiClient.list(listArgument);
+ }
+
+ /**
+ * Creates new project that will be a subproject of "home" for current user.
+ *
+ * @param projectName name for the newly created project
+ * @return Group object containing information about created project
+ * (mapped from JSON returned from server after creating the project)
+ */
+ public Group createNewProject(String projectName) {
+ Group project = new Group();
+ project.setName(projectName);
+ project.setGroupClass(PROJECT);
+ Group createdProject = groupsApiClient.create(project);
+ log.debug("Project " + createdProject.getName() + " created with UUID: " + createdProject.getUuid());
+ return createdProject;
+ }
+
+ /**
+ * Deletes collection with specified uuid.
+ *
+ * @param collectionUuid uuid of collection to be deleted. User whose token is provided in configuration
+ * must be authorized to delete such collection.
+ * @return collection object with deleted collection (mapped from JSON returned from server after deleting the collection)
+ */
+ public Collection deleteCollection(String collectionUuid) {
+ Collection deletedCollection = collectionsApiClient.delete(collectionUuid);
+ log.debug("Collection: " + collectionUuid + " deleted.");
+ return deletedCollection;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.api.client.GroupsApiClient;
+ import org.arvados.client.api.client.UsersApiClient;
+ import org.arvados.client.exception.ArvadosApiException;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.common.Patterns;
+ import org.arvados.client.config.FileConfigProvider;
+ import org.arvados.client.config.ConfigProvider;
+ import org.arvados.client.exception.ArvadosClientException;
+
+ import java.io.File;
+ import java.time.LocalDateTime;
+ import java.time.format.DateTimeFormatter;
+ import java.util.List;
+ import java.util.Optional;
+
+ public class CollectionFactory {
+
+ private ConfigProvider config;
+ private UsersApiClient usersApiClient;
+ private GroupsApiClient groupsApiClient;
+
+ private final String name;
+ private final String projectUuid;
+ private final List<File> manifestFiles;
+ private final List<String> manifestLocators;
+
+ private CollectionFactory(ConfigProvider config, String name, String projectUuid, List<File> manifestFiles, List<String> manifestLocators) {
+ this.name = name;
+ this.projectUuid = projectUuid;
+ this.manifestFiles = manifestFiles;
+ this.manifestLocators = manifestLocators;
+ this.config = config;
+ setApiClients();
+ }
+
+ public static CollectionFactoryBuilder builder() {
+ return new CollectionFactoryBuilder();
+ }
+
+ private void setApiClients() {
+ if(this.config == null) this.config = new FileConfigProvider();
+
+ this.usersApiClient = new UsersApiClient(config);
+ this.groupsApiClient = new GroupsApiClient(config);
+ }
+
+ public Collection create() {
+ ManifestFactory manifestFactory = ManifestFactory.builder()
+ .files(manifestFiles)
+ .locators(manifestLocators)
+ .build();
+ String manifest = manifestFactory.create();
+
+ Collection newCollection = new Collection();
+ newCollection.setName(getNameOrDefault(name));
+ newCollection.setManifestText(manifest);
+ newCollection.setOwnerUuid(getDesiredProjectUuid(projectUuid));
+
+ return newCollection;
+ }
+
+ private String getNameOrDefault(String name) {
+ return Optional.ofNullable(name).orElseGet(() -> {
+ LocalDateTime dateTime = LocalDateTime.now();
+ DateTimeFormatter formatter = DateTimeFormatter.ofPattern("Y-MM-dd HH:mm:ss.SSS");
+ return String.format("New Collection (%s)", dateTime.format(formatter));
+ });
+ }
+
+ public String getDesiredProjectUuid(String projectUuid) {
+ try {
+ if (projectUuid == null || projectUuid.length() == 0){
+ return usersApiClient.current().getUuid();
+ } else if (projectUuid.matches(Patterns.USER_UUID_PATTERN)) {
+ return usersApiClient.get(projectUuid).getUuid();
+ } else if (projectUuid.matches(Patterns.GROUP_UUID_PATTERN)) {
+ return groupsApiClient.get(projectUuid).getUuid();
+ }
+ } catch (ArvadosApiException e) {
+ throw new ArvadosClientException(String.format("An error occurred while getting project by UUID %s", projectUuid));
+ }
+ throw new ArvadosClientException(String.format("No project with %s UUID found", projectUuid));
+ }
+
+ public static class CollectionFactoryBuilder {
+ private ConfigProvider config;
+ private String name;
+ private String projectUuid;
+ private List<File> manifestFiles;
+ private List<String> manifestLocators;
+
+ CollectionFactoryBuilder() {
+ }
+
+ public CollectionFactoryBuilder config(ConfigProvider config) {
+ this.config = config;
+ return this;
+ }
+
+ public CollectionFactoryBuilder name(String name) {
+ this.name = name;
+ return this;
+ }
+
+ public CollectionFactoryBuilder projectUuid(String projectUuid) {
+ this.projectUuid = projectUuid;
+ return this;
+ }
+
+ public CollectionFactoryBuilder manifestFiles(List<File> manifestFiles) {
+ this.manifestFiles = manifestFiles;
+ return this;
+ }
+
+ public CollectionFactoryBuilder manifestLocators(List<String> manifestLocators) {
+ this.manifestLocators = manifestLocators;
+ return this;
+ }
+
+ public CollectionFactory build() {
+ return new CollectionFactory(config, name, projectUuid, manifestFiles, manifestLocators);
+ }
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import com.google.common.base.Strings;
+ import org.arvados.client.common.Characters;
+
+ public class FileToken {
+
+ private int filePosition;
+ private int fileSize;
+ private String fileName;
+ private String path;
+
+ public FileToken(String fileTokenInfo) {
+ splitFileTokenInfo(fileTokenInfo);
+ }
+
+ public FileToken(String fileTokenInfo, String path) {
+ splitFileTokenInfo(fileTokenInfo);
+ this.path = path;
+ }
+
+ private void splitFileTokenInfo(String fileTokenInfo) {
+ String[] tokenPieces = fileTokenInfo.split(":");
+ this.filePosition = Integer.parseInt(tokenPieces[0]);
+ this.fileSize = Integer.parseInt(tokenPieces[1]);
+ this.fileName = tokenPieces[2].replace(Characters.SPACE, " ");
+ }
+
+ @Override
+ public String toString() {
+ return filePosition + ":" + fileSize + ":" + fileName;
+ }
+
+ public String getFullPath() {
+ return Strings.isNullOrEmpty(path) ? fileName : path + fileName;
+ }
+
+ public int getFilePosition() {
+ return this.filePosition;
+ }
+
+ public int getFileSize() {
+ return this.fileSize;
+ }
+
+ public String getFileName() {
+ return this.fileName;
+ }
+
+ public String getPath() {
+ return this.path;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.arvados.client.logic.keep.KeepLocator;
+
+ import java.util.ArrayList;
+ import java.util.Arrays;
+ import java.util.LinkedList;
+ import java.util.List;
+ import java.util.Objects;
+
+ import static java.util.stream.Collectors.toList;
+ import static org.arvados.client.common.Patterns.FILE_TOKEN_PATTERN;
+ import static org.arvados.client.common.Patterns.LOCATOR_PATTERN;
+
+ public class ManifestDecoder {
+
+ public List<ManifestStream> decode(String manifestText) {
+
+ if (manifestText == null || manifestText.isEmpty()) {
+ throw new ArvadosClientException("Manifest text cannot be empty.");
+ }
+
+ List<String> manifestStreams = new ArrayList<>(Arrays.asList(manifestText.split("\\n")));
+ if (!manifestStreams.get(0).startsWith(". ")) {
+ throw new ArvadosClientException("Invalid first path component (expecting \".\")");
+ }
+
+ return manifestStreams.stream()
+ .map(this::decodeSingleManifestStream)
+ .collect(toList());
+ }
+
+ private ManifestStream decodeSingleManifestStream(String manifestStream) {
+ Objects.requireNonNull(manifestStream, "Manifest stream cannot be empty.");
+
+ LinkedList<String> manifestPieces = new LinkedList<>(Arrays.asList(manifestStream.split("\\s+")));
+ String streamName = manifestPieces.poll();
+ String path = ".".equals(streamName) ? "" : streamName.substring(2).concat(Characters.SLASH);
+
+ List<KeepLocator> keepLocators = manifestPieces
+ .stream()
+ .filter(p -> p.matches(LOCATOR_PATTERN))
+ .map(this::getKeepLocator)
+ .collect(toList());
+
+
+ List<FileToken> fileTokens = manifestPieces.stream()
+ .skip(keepLocators.size())
+ .filter(p -> p.matches(FILE_TOKEN_PATTERN))
+ .map(p -> new FileToken(p, path))
+ .collect(toList());
+
+ return new ManifestStream(streamName, keepLocators, fileTokens);
+
+ }
+
+ private KeepLocator getKeepLocator(String locatorString ) {
+ try {
+ return new KeepLocator(locatorString);
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import com.google.common.collect.ImmutableList;
+ import org.arvados.client.common.Characters;
+
+ import java.io.File;
+ import java.util.Collection;
+ import java.util.List;
+ import java.util.stream.Collectors;
+
+ public class ManifestFactory {
+
+ private Collection<File> files;
+ private List<String> locators;
+
+ ManifestFactory(Collection<File> files, List<String> locators) {
+ this.files = files;
+ this.locators = locators;
+ }
+
+ public static ManifestFactoryBuilder builder() {
+ return new ManifestFactoryBuilder();
+ }
+
+ public String create() {
+ ImmutableList.Builder<String> builder = new ImmutableList.Builder<String>()
+ .add(Characters.DOT)
+ .addAll(locators);
+ long filePosition = 0;
+ for (File file : files) {
+ builder.add(String.format("%d:%d:%s", filePosition, file.length(), file.getName().replace(" ", Characters.SPACE)));
+ filePosition += file.length();
+ }
+ String manifest = builder.build().stream().collect(Collectors.joining(" ")).concat(Characters.NEW_LINE);
+ return manifest;
+ }
+
+ public static class ManifestFactoryBuilder {
+ private Collection<File> files;
+ private List<String> locators;
+
+ ManifestFactoryBuilder() {
+ }
+
+ public ManifestFactory.ManifestFactoryBuilder files(Collection<File> files) {
+ this.files = files;
+ return this;
+ }
+
+ public ManifestFactory.ManifestFactoryBuilder locators(List<String> locators) {
+ this.locators = locators;
+ return this;
+ }
+
+ public ManifestFactory build() {
+ return new ManifestFactory(files, locators);
+ }
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.logic.keep.KeepLocator;
+
+ import java.util.List;
+ import java.util.stream.Collectors;
+ import java.util.stream.Stream;
+
+ public class ManifestStream {
+
+ private String streamName;
+ private List<KeepLocator> keepLocators;
+ private List<FileToken> fileTokens;
+
+ public ManifestStream(String streamName, List<KeepLocator> keepLocators, List<FileToken> fileTokens) {
+ this.streamName = streamName;
+ this.keepLocators = keepLocators;
+ this.fileTokens = fileTokens;
+ }
+
+ @Override
+ public String toString() {
+ return streamName + " " + Stream.concat(keepLocators.stream().map(KeepLocator::toString), fileTokens.stream().map(FileToken::toString))
+ .collect(Collectors.joining(" "));
+ }
+
+ public String getStreamName() {
+ return this.streamName;
+ }
+
+ public List<KeepLocator> getKeepLocators() {
+ return this.keepLocators;
+ }
+
+ public List<FileToken> getFileTokens() {
+ return this.fileTokens;
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import com.google.common.collect.Lists;
+ import org.arvados.client.api.client.CollectionsApiClient;
+ import org.arvados.client.api.client.KeepWebApiClient;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.arvados.client.logic.collection.FileToken;
+ import org.arvados.client.logic.collection.ManifestDecoder;
+ import org.arvados.client.logic.collection.ManifestStream;
+ import org.arvados.client.logic.keep.exception.DownloadFolderAlreadyExistsException;
+ import org.arvados.client.logic.keep.exception.FileAlreadyExistsException;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.util.ArrayList;
+ import java.util.List;
+ import java.util.concurrent.CompletableFuture;
+ import java.util.stream.Collectors;
+ import java.util.stream.Stream;
+
+ public class FileDownloader {
+
+ private final KeepClient keepClient;
+ private final ManifestDecoder manifestDecoder;
+ private final CollectionsApiClient collectionsApiClient;
+ private final KeepWebApiClient keepWebApiClient;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(FileDownloader.class);
+
+ public FileDownloader(KeepClient keepClient, ManifestDecoder manifestDecoder, CollectionsApiClient collectionsApiClient, KeepWebApiClient keepWebApiClient) {
+ this.keepClient = keepClient;
+ this.manifestDecoder = manifestDecoder;
+ this.collectionsApiClient = collectionsApiClient;
+ this.keepWebApiClient = keepWebApiClient;
+ }
+
+ public List<FileToken> listFileInfoFromCollection(String collectionUuid) {
+ Collection requestedCollection = collectionsApiClient.get(collectionUuid);
+ String manifestText = requestedCollection.getManifestText();
+
+ // decode manifest text and get list of all FileTokens for this collection
+ return manifestDecoder.decode(manifestText)
+ .stream()
+ .flatMap(p -> p.getFileTokens().stream())
+ .collect(Collectors.toList());
+ }
+
+ public File downloadSingleFileUsingKeepWeb(String filePathName, String collectionUuid, String pathToDownloadFolder) {
+ FileToken fileToken = getFileTokenFromCollection(filePathName, collectionUuid);
+ if (fileToken == null) {
+ throw new ArvadosClientException(String.format("%s not found in Collection with UUID %s", filePathName, collectionUuid));
+ }
+
+ File downloadedFile = checkIfFileExistsInTargetLocation(fileToken, pathToDownloadFolder);
+ try (FileOutputStream fos = new FileOutputStream(downloadedFile)) {
+ fos.write(keepWebApiClient.download(collectionUuid, filePathName));
+ } catch (IOException e) {
+ throw new ArvadosClientException(String.format("Unable to write down file %s", fileToken.getFileName()), e);
+ }
+ return downloadedFile;
+ }
+
+ public List<File> downloadFilesFromCollectionUsingKeepWeb(String collectionUuid, String pathToDownloadFolder) {
+ String collectionTargetDir = setTargetDirectory(collectionUuid, pathToDownloadFolder).getAbsolutePath();
+ List<FileToken> fileTokens = listFileInfoFromCollection(collectionUuid);
+
+ List<CompletableFuture<File>> futures = Lists.newArrayList();
+ for (FileToken fileToken : fileTokens) {
+ futures.add(CompletableFuture.supplyAsync(() -> this.downloadOneFileFromCollectionUsingKeepWeb(fileToken, collectionUuid, collectionTargetDir)));
+ }
+
+ @SuppressWarnings("unchecked")
+ CompletableFuture<File>[] array = futures.toArray(new CompletableFuture[0]);
+ return Stream.of(array)
+ .map(CompletableFuture::join).collect(Collectors.toList());
+ }
+
+ private FileToken getFileTokenFromCollection(String filePathName, String collectionUuid) {
+ return listFileInfoFromCollection(collectionUuid)
+ .stream()
+ .filter(p -> (p.getFullPath()).equals(filePathName))
+ .findFirst()
+ .orElse(null);
+ }
+
+ private File checkIfFileExistsInTargetLocation(FileToken fileToken, String pathToDownloadFolder) {
+ String fileName = fileToken.getFileName();
+
+ File downloadFile = new File(pathToDownloadFolder + Characters.SLASH + fileName);
+ if (downloadFile.exists()) {
+ throw new FileAlreadyExistsException(String.format("File %s exists in location %s", fileName, pathToDownloadFolder));
+ } else {
+ return downloadFile;
+ }
+ }
+
+ private File downloadOneFileFromCollectionUsingKeepWeb(FileToken fileToken, String collectionUuid, String pathToDownloadFolder) {
+ String filePathName = fileToken.getPath() + fileToken.getFileName();
+ File downloadedFile = new File(pathToDownloadFolder + Characters.SLASH + filePathName);
+ downloadedFile.getParentFile().mkdirs();
+
+ try (FileOutputStream fos = new FileOutputStream(downloadedFile)) {
+ fos.write(keepWebApiClient.download(collectionUuid, filePathName));
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ return downloadedFile;
+ }
+
+ public List<File> downloadFilesFromCollection(String collectionUuid, String pathToDownloadFolder) {
+
+ // download requested collection and extract manifest text
+ Collection requestedCollection = collectionsApiClient.get(collectionUuid);
+ String manifestText = requestedCollection.getManifestText();
+
+ // if directory with this collectionUUID does not exist - create one
+ // if exists - abort (throw exception)
+ File collectionTargetDir = setTargetDirectory(collectionUuid, pathToDownloadFolder);
+
+ // decode manifest text and create list of ManifestStream objects containing KeepLocators and FileTokens
+ List<ManifestStream> manifestStreams = manifestDecoder.decode(manifestText);
+
+ //list of all downloaded files that will be returned by this method
+ List<File> downloadedFilesFromCollection = new ArrayList<>();
+
+ // download files for each manifest stream
+ for (ManifestStream manifestStream : manifestStreams)
+ downloadedFilesFromCollection.addAll(downloadFilesFromSingleManifestStream(manifestStream, collectionTargetDir));
+
+ log.debug(String.format("Total of: %d files downloaded", downloadedFilesFromCollection.size()));
+ return downloadedFilesFromCollection;
+ }
+
+ private File setTargetDirectory(String collectionUUID, String pathToDownloadFolder) {
+ //local directory to save downloaded files
+ File collectionTargetDir = new File(pathToDownloadFolder + Characters.SLASH + collectionUUID);
+ if (collectionTargetDir.exists()) {
+ throw new DownloadFolderAlreadyExistsException(String.format("Directory for collection UUID %s already exists", collectionUUID));
+ } else {
+ collectionTargetDir.mkdirs();
+ }
+ return collectionTargetDir;
+ }
+
+ private List<File> downloadFilesFromSingleManifestStream(ManifestStream manifestStream, File collectionTargetDir){
+ List<File> downloadedFiles = new ArrayList<>();
+ List<KeepLocator> keepLocators = manifestStream.getKeepLocators();
+ DownloadHelper downloadHelper = new DownloadHelper(keepLocators);
+
+ for (FileToken fileToken : manifestStream.getFileTokens()) {
+ File downloadedFile = new File(collectionTargetDir.getAbsolutePath() + Characters.SLASH + fileToken.getFullPath()); //create file
+ downloadedFile.getParentFile().mkdirs();
+
+ try (FileOutputStream fos = new FileOutputStream(downloadedFile, true)) {
+ downloadHelper.setBytesToDownload(fileToken.getFileSize()); //update file size info
+
+ //this part needs to be repeated for each file until whole file is downloaded
+ do {
+ downloadHelper.requestNewDataChunk(); //check if new data chunk needs to be downloaded
+ downloadHelper.writeDownFile(fos); // download data from chunk
+ } while (downloadHelper.getBytesToDownload() != 0);
+
+ } catch (IOException | ArvadosClientException e) {
+ throw new ArvadosClientException(String.format("Unable to write down file %s", fileToken.getFileName()), e);
+ }
+
+ downloadedFiles.add(downloadedFile);
+ log.debug(String.format("File %d / %d downloaded from manifest stream",
+ manifestStream.getFileTokens().indexOf(fileToken) + 1,
+ manifestStream.getFileTokens().size()));
+ }
+ return downloadedFiles;
+ }
+
+ private class DownloadHelper {
+
+ // values for tracking file output streams and matching data chunks with initial files
+ int currentDataChunkNumber;
+ int bytesDownloadedFromChunk;
+ int bytesToDownload;
+ byte[] currentDataChunk;
+ boolean remainingDataInChunk;
+ final List<KeepLocator> keepLocators;
+
+ private DownloadHelper(List<KeepLocator> keepLocators) {
+ currentDataChunkNumber = -1;
+ bytesDownloadedFromChunk = 0;
+ remainingDataInChunk = false;
+ this.keepLocators = keepLocators;
+ }
+
+ private int getBytesToDownload() {
+ return bytesToDownload;
+ }
+
+ private void setBytesToDownload(int bytesToDownload) {
+ this.bytesToDownload = bytesToDownload;
+ }
+
+ private void requestNewDataChunk() {
+ if (!remainingDataInChunk) {
+ currentDataChunkNumber++;
+ if (currentDataChunkNumber < keepLocators.size()) {
+ //swap data chunk for next one
+ currentDataChunk = keepClient.getDataChunk(keepLocators.get(currentDataChunkNumber));
+ log.debug(String.format("%d of %d data chunks from manifest stream downloaded", currentDataChunkNumber + 1, keepLocators.size()));
+ } else {
+ throw new ArvadosClientException("Data chunk required for download is missing.");
+ }
+ }
+ }
+
+ private void writeDownFile(FileOutputStream fos) throws IOException {
+ //case 1: more bytes needed than available in current chunk (or whole current chunk needed) to download file
+ if (bytesToDownload >= currentDataChunk.length - bytesDownloadedFromChunk) {
+ writeDownWholeDataChunk(fos);
+ }
+ //case 2: current data chunk contains more bytes than is needed for this file
+ else {
+ writeDownDataChunkPartially(fos);
+ }
+ }
+
+ private void writeDownWholeDataChunk(FileOutputStream fos) throws IOException {
+ // write all remaining bytes from current chunk
+ fos.write(currentDataChunk, bytesDownloadedFromChunk, currentDataChunk.length - bytesDownloadedFromChunk);
+ //update bytesToDownload
+ bytesToDownload -= (currentDataChunk.length - bytesDownloadedFromChunk);
+ // set remaining data in chunk to false
+ remainingDataInChunk = false;
+ //reset bytesDownloadedFromChunk so that its set to 0 for the next chunk
+ bytesDownloadedFromChunk = 0;
+ }
+
+ private void writeDownDataChunkPartially(FileOutputStream fos) throws IOException {
+ //write all remaining bytes for this file from current chunk
+ fos.write(currentDataChunk, bytesDownloadedFromChunk, bytesToDownload);
+ // update number of bytes downloaded from this chunk
+ bytesDownloadedFromChunk += bytesToDownload;
+ // set remaining data in chunk to true
+ remainingDataInChunk = true;
+ // reset bytesToDownload to exit while loop and move to the next file
+ bytesToDownload = 0;
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import org.arvados.client.api.client.KeepServerApiClient;
+ import org.arvados.client.exception.ArvadosApiException;
+ import org.arvados.client.config.ConfigProvider;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.util.Map;
+
+ public class FileTransferHandler {
+
+ private final String host;
+ private final KeepServerApiClient keepServerApiClient;
+ private final Map<String, String> headers;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(FileTransferHandler.class);
+
+ public FileTransferHandler(String host, Map<String, String> headers, ConfigProvider config) {
+ this.host = host;
+ this.headers = headers;
+ this.keepServerApiClient = new KeepServerApiClient(config);
+ }
+
+ public String put(String hashString, File body) {
+ String url = host + hashString;
+ String locator = null;
+ try {
+ locator = keepServerApiClient.upload(url, headers, body);
+ } catch (ArvadosApiException e) {
+ log.error("Cannot upload file to Keep server.", e);
+ }
+ return locator;
+ }
+
+ public byte[] get(KeepLocator locator) {
+ return get(locator.stripped(), locator.permissionHint());
+ }
+
+ public byte[] get(String blockLocator, String authToken) {
+ String url = host + blockLocator + "+" + authToken;
+ try {
+ return keepServerApiClient.download(url);
+ } catch (ArvadosApiException e) {
+ log.error("Cannot download file from Keep server.", e);
+ return null;
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import com.google.common.collect.Lists;
+ import org.arvados.client.api.client.CollectionsApiClient;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.config.ConfigProvider;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.arvados.client.logic.collection.CollectionFactory;
+ import org.arvados.client.utils.FileMerge;
+ import org.arvados.client.utils.FileSplit;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.io.IOException;
+ import java.util.List;
+ import java.util.Objects;
+ import java.util.UUID;
+
+ import static java.util.stream.Collectors.toList;
+
+ public class FileUploader {
+
+ private final KeepClient keepClient;
+ private final CollectionsApiClient collectionsApiClient;
+ private final ConfigProvider config;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(FileUploader.class);
+
+ public FileUploader(KeepClient keepClient, CollectionsApiClient collectionsApiClient, ConfigProvider config) {
+ this.keepClient = keepClient;
+ this.collectionsApiClient = collectionsApiClient;
+ this.config = config;
+ }
+
+ public Collection upload(List<File> sourceFiles, String collectionName, String projectUuid) {
+ List<String> locators = uploadToKeep(sourceFiles);
+ CollectionFactory collectionFactory = CollectionFactory.builder()
+ .config(config)
+ .name(collectionName)
+ .projectUuid(projectUuid)
+ .manifestFiles(sourceFiles)
+ .manifestLocators(locators)
+ .build();
+
+ Collection newCollection = collectionFactory.create();
+ return collectionsApiClient.create(newCollection);
+ }
+
+ public Collection uploadToExistingCollection(List<File> files, String collectionUuid) {
+ List<String> locators = uploadToKeep(files);
+ Collection collectionBeforeUpload = collectionsApiClient.get(collectionUuid);
+ String oldManifest = collectionBeforeUpload.getManifestText();
+
+ CollectionFactory collectionFactory = CollectionFactory.builder()
+ .config(config)
+ .manifestFiles(files)
+ .manifestLocators(locators).build();
+
+ String newPartOfManifestText = collectionFactory.create().getManifestText();
+ String newManifest = oldManifest + newPartOfManifestText;
+
+ collectionBeforeUpload.setManifestText(newManifest);
+ return collectionsApiClient.update(collectionBeforeUpload);
+ }
+
+ private List<String> uploadToKeep(List<File> files) {
+ File targetDir = config.getFileSplitDirectory();
+ File combinedFile = new File(targetDir.getAbsolutePath() + Characters.SLASH + UUID.randomUUID());
+ List<File> chunks;
+ try {
+ FileMerge.merge(files, combinedFile);
+ chunks = FileSplit.split(combinedFile, targetDir, config.getFileSplitSize());
+ } catch (IOException e) {
+ throw new ArvadosClientException("Cannot create file chunks for upload", e);
+ }
+ combinedFile.delete();
+
+ int copies = config.getNumberOfCopies();
+ int numRetries = config.getNumberOfRetries();
+
+ List<String> locators = Lists.newArrayList();
+ for (File chunk : chunks) {
+ try {
+ locators.add(keepClient.put(chunk, copies, numRetries));
+ } catch (ArvadosClientException e) {
+ log.error("Problem occurred while uploading chunk file {}", chunk.getName(), e);
+ throw e;
+ }
+ }
+ return locators.stream()
+ .filter(Objects::nonNull)
+ .collect(toList());
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import com.google.common.collect.Lists;
+ import org.apache.commons.codec.digest.DigestUtils;
+ import org.apache.commons.io.FileUtils;
+ import org.arvados.client.api.client.KeepServicesApiClient;
+ import org.arvados.client.api.model.KeepService;
+ import org.arvados.client.api.model.KeepServiceList;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.common.Headers;
+ import org.arvados.client.config.ConfigProvider;
+ import org.arvados.client.exception.ArvadosApiException;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.slf4j.Logger;
+
+ import java.io.File;
+ import java.io.IOException;
+ import java.util.ArrayList;
+ import java.util.HashMap;
+ import java.util.List;
+ import java.util.Map;
+ import java.util.Objects;
+ import java.util.concurrent.CompletableFuture;
+ import java.util.function.Function;
+ import java.util.stream.Collectors;
+ import java.util.stream.Stream;
+
+ public class KeepClient {
+
+ private final KeepServicesApiClient keepServicesApiClient;
+ private final Logger log = org.slf4j.LoggerFactory.getLogger(KeepClient.class);
+ private List<KeepService> keepServices;
+ private List<KeepService> writableServices;
+ private Map<String, KeepService> gatewayServices;
+ private final String apiToken;
+ private Integer maxReplicasPerService;
+ private final ConfigProvider config;
+
+ public KeepClient(ConfigProvider config) {
+ this.config = config;
+ keepServicesApiClient = new KeepServicesApiClient(config);
+ apiToken = config.getApiToken();
+ }
+
+ public byte[] getDataChunk(KeepLocator keepLocator) {
+
+ Map<String, String> headers = new HashMap<>();
+ Map<String, FileTransferHandler> rootsMap = new HashMap<>();
+
+ List<String> sortedRoots = mapNewServices(rootsMap, keepLocator, false, false, headers);
+
+ byte[] dataChunk = sortedRoots
+ .stream()
+ .map(rootsMap::get)
+ .map(r -> r.get(keepLocator))
+ .filter(Objects::nonNull)
+ .findFirst()
+ .orElse(null);
+
+ if (dataChunk == null) {
+ throw new ArvadosClientException("No server responding. Unable to download data chunk.");
+ }
+
+ return dataChunk;
+ }
+
+ public String put(File data, int copies, int numRetries) {
+
+ byte[] fileBytes;
+ try {
+ fileBytes = FileUtils.readFileToByteArray(data);
+ } catch (IOException e) {
+ throw new ArvadosClientException("An error occurred while reading data chunk", e);
+ }
+
+ String dataHash = DigestUtils.md5Hex(fileBytes);
+ String locatorString = String.format("%s+%d", dataHash, data.length());
+
+ if (copies < 1) {
+ return locatorString;
+ }
+ KeepLocator locator = new KeepLocator(locatorString);
+
+ // Tell the proxy how many copies we want it to store
+ Map<String, String> headers = new HashMap<>();
+ headers.put(Headers.X_KEEP_DESIRED_REPLICAS, String.valueOf(copies));
+
+ Map<String, FileTransferHandler> rootsMap = new HashMap<>();
+ List<String> sortedRoots = mapNewServices(rootsMap, locator, false, true, headers);
+
+ int numThreads = 0;
+ if (maxReplicasPerService == null || maxReplicasPerService >= copies) {
+ numThreads = 1;
+ } else {
+ numThreads = ((Double) Math.ceil(1.0 * copies / maxReplicasPerService)).intValue();
+ }
+ log.debug("Pool max threads is {}", numThreads);
+
+ List<CompletableFuture<String>> futures = Lists.newArrayList();
+ for (int i = 0; i < numThreads; i++) {
+ String root = sortedRoots.get(i);
+ FileTransferHandler keepServiceLocal = rootsMap.get(root);
+ futures.add(CompletableFuture.supplyAsync(() -> keepServiceLocal.put(dataHash, data)));
+ }
+
+ @SuppressWarnings("unchecked")
+ CompletableFuture<String>[] array = futures.toArray(new CompletableFuture[0]);
+
+ return Stream.of(array)
+ .map(CompletableFuture::join)
+ .reduce((a, b) -> b)
+ .orElse(null);
+ }
+
+ private List<String> mapNewServices(Map<String, FileTransferHandler> rootsMap, KeepLocator locator,
+ boolean forceRebuild, boolean needWritable, Map<String, String> headers) {
+
+ headers.putIfAbsent("Authorization", String.format("OAuth2 %s", apiToken));
+ List<String> localRoots = weightedServiceRoots(locator, forceRebuild, needWritable);
+ for (String root : localRoots) {
+ FileTransferHandler keepServiceLocal = new FileTransferHandler(root, headers, config);
+ rootsMap.putIfAbsent(root, keepServiceLocal);
+ }
+ return localRoots;
+ }
+
+ /**
+ * Return an array of Keep service endpoints, in the order in which they should be probed when reading or writing
+ * data with the given hash+hints.
+ */
+ private List<String> weightedServiceRoots(KeepLocator locator, boolean forceRebuild, boolean needWritable) {
+
+ buildServicesList(forceRebuild);
+
+ List<String> sortedRoots = new ArrayList<>();
+
+ // Use the services indicated by the given +K@... remote
+ // service hints, if any are present and can be resolved to a
+ // URI.
+ //
+ for (String hint : locator.getHints()) {
+ if (hint.startsWith("K@")) {
+ if (hint.length() == 7) {
+ sortedRoots.add(String.format("https://keep.%s.arvadosapi.com/", hint.substring(2)));
+ } else if (hint.length() == 29) {
+ KeepService svc = gatewayServices.get(hint.substring(2));
+ if (svc != null) {
+ sortedRoots.add(svc.getServiceRoot());
+ }
+ }
+ }
+ }
+
+ // Sort the available local services by weight (heaviest first)
+ // for this locator, and return their service_roots (base URIs)
+ // in that order.
+ List<KeepService> useServices = keepServices;
+ if (needWritable) {
+ useServices = writableServices;
+ }
+ anyNonDiskServices(useServices);
+
+ sortedRoots.addAll(useServices
+ .stream()
+ .sorted((ks1, ks2) -> serviceWeight(locator.getMd5sum(), ks2.getUuid())
+ .compareTo(serviceWeight(locator.getMd5sum(), ks1.getUuid())))
+ .map(KeepService::getServiceRoot)
+ .collect(Collectors.toList()));
+
+ return sortedRoots;
+ }
+
+ private void buildServicesList(boolean forceRebuild) {
+ if (keepServices != null && !forceRebuild) {
+ return;
+ }
+ KeepServiceList keepServiceList;
+ try {
+ keepServiceList = keepServicesApiClient.accessible();
+ } catch (ArvadosApiException e) {
+ throw new ArvadosClientException("Cannot obtain list of accessible keep services");
+ }
+ // Gateway services are only used when specified by UUID,
+ // so there's nothing to gain by filtering them by
+ // service_type.
+ gatewayServices = keepServiceList.getItems().stream().collect(Collectors.toMap(KeepService::getUuid, Function.identity()));
+
+ if (gatewayServices.isEmpty()) {
+ throw new ArvadosClientException("No gateway services available!");
+ }
+
+ // Precompute the base URI for each service.
+ for (KeepService keepService : gatewayServices.values()) {
+ String serviceHost = keepService.getServiceHost();
+ if (!serviceHost.startsWith("[") && serviceHost.contains(Characters.COLON)) {
+ // IPv6 URIs must be formatted like http://[::1]:80/...
+ serviceHost = String.format("[%s]", serviceHost);
+ }
+
+ String protocol = keepService.getServiceSslFlag() ? "https" : "http";
+ String serviceRoot = String.format("%s://%s:%d/", protocol, serviceHost, keepService.getServicePort());
+ keepService.setServiceRoot(serviceRoot);
+ }
+
+ keepServices = gatewayServices.values().stream().filter(ks -> !ks.getServiceType().startsWith("gateway:")).collect(Collectors.toList());
+ writableServices = keepServices.stream().filter(ks -> !ks.getReadOnly()).collect(Collectors.toList());
+
+ // For disk type services, max_replicas_per_service is 1
+ // It is unknown (unlimited) for other service types.
+ if (anyNonDiskServices(writableServices)) {
+ maxReplicasPerService = null;
+ } else {
+ maxReplicasPerService = 1;
+ }
+ }
+
+ private Boolean anyNonDiskServices(List<KeepService> useServices) {
+ return useServices.stream().anyMatch(ks -> !ks.getServiceType().equals("disk"));
+ }
+
+ /**
+ * Compute the weight of a Keep service endpoint for a data block with a known hash.
+ * <p>
+ * The weight is md5(h + u) where u is the last 15 characters of the service endpoint's UUID.
+ */
+ private static String serviceWeight(String dataHash, String serviceUuid) {
+ String shortenedUuid;
+ if (serviceUuid != null && serviceUuid.length() >= 15) {
+ int substringIndex = serviceUuid.length() - 15;
+ shortenedUuid = serviceUuid.substring(substringIndex);
+ } else {
+ shortenedUuid = (serviceUuid == null) ? "" : serviceUuid;
+ }
+ return DigestUtils.md5Hex(dataHash + shortenedUuid);
+ }
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import org.arvados.client.exception.ArvadosClientException;
+
+ import java.time.Instant;
+ import java.time.LocalDateTime;
+ import java.time.ZoneOffset;
+ import java.util.ArrayList;
+ import java.util.Arrays;
+ import java.util.LinkedList;
+ import java.util.List;
+ import java.util.Objects;
+ import java.util.stream.Collectors;
+ import java.util.stream.Stream;
+
+ import static org.arvados.client.common.Patterns.HINT_PATTERN;
+
+ public class KeepLocator {
+
+ private final List<String> hints = new ArrayList<>();
+ private String permSig;
+ private LocalDateTime permExpiry;
+ private final String md5sum;
+ private final Integer size;
+
+ public KeepLocator(String locatorString) {
+ LinkedList<String> pieces = new LinkedList<>(Arrays.asList(locatorString.split("\\+")));
+
+ md5sum = pieces.poll();
+ size = Integer.valueOf(Objects.requireNonNull(pieces.poll()));
+
+ for (String hint : pieces) {
+ if (!hint.matches(HINT_PATTERN)) {
+ throw new ArvadosClientException(String.format("invalid hint format: %s", hint));
+ } else if (hint.startsWith("A")) {
+ parsePermissionHint(hint);
+ } else {
+ hints.add(hint);
+ }
+ }
+ }
+
+ public List<String> getHints() {
+ return hints;
+ }
+
+ public String getMd5sum() {
+ return md5sum;
+ }
+
+ @Override
+ public String toString() {
+ return Stream.concat(Stream.of(md5sum, size.toString(), permissionHint()), hints.stream())
+ .filter(Objects::nonNull)
+ .collect(Collectors.joining("+"));
+ }
+
+ public String stripped() {
+ return size != null ? String.format("%s+%d", md5sum, size) : md5sum;
+ }
+
+ public String permissionHint() {
+ if (permSig == null || permExpiry == null) {
+ return null;
+ }
+
+ long timestamp = permExpiry.toEpochSecond(ZoneOffset.UTC);
+ String signTimestamp = Long.toHexString(timestamp);
+ return String.format("A%s@%s", permSig, signTimestamp);
+ }
+
+ private void parsePermissionHint(String hint) {
+ String[] hintSplit = hint.substring(1).split("@", 2);
+ permSig = hintSplit[0];
+
+ int permExpiryDecimal = Integer.parseInt(hintSplit[1], 16);
+ permExpiry = LocalDateTime.ofInstant(Instant.ofEpochSecond(permExpiryDecimal), ZoneOffset.UTC);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep.exception;
+
+ import org.arvados.client.exception.ArvadosClientException;
+
+ /**
+ * Exception indicating that directory with given name was already created in specified location.
+ *
+ * <p> This exception will be thrown during an attempt to download all files from certain
+ * collection to a location that already contains folder named by this collection's UUID.</p>
+ */
+ public class DownloadFolderAlreadyExistsException extends ArvadosClientException {
+
+ public DownloadFolderAlreadyExistsException(String message) {
+ super(message);
+ }
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep.exception;
+
+ import org.arvados.client.exception.ArvadosClientException;
+
+ /**
+ * Signals that an attempt to download a file with given name has failed for a specified
+ * download location.
+ *
+ * <p> This exception will be thrown during an attempt to download single file to a location
+ * that already contains file with given name</p>
+ */
+ public class FileAlreadyExistsException extends ArvadosClientException {
+
+ public FileAlreadyExistsException(String message) { super(message); }
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.utils;
+
+ import java.io.BufferedOutputStream;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.file.Files;
+ import java.util.Collection;
+
+ public class FileMerge {
+
+ public static void merge(Collection<File> files, File targetFile) throws IOException {
+ try (FileOutputStream fos = new FileOutputStream(targetFile); BufferedOutputStream mergingStream = new BufferedOutputStream(fos)) {
+ for (File file : files) {
+ Files.copy(file.toPath(), mergingStream);
+ }
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.utils;
+
+ import org.apache.commons.io.FileUtils;
+
+ import java.io.*;
+ import java.util.ArrayList;
+ import java.util.List;
+
+ /**
+ * Based on:
+ * {@link} https://stackoverflow.com/questions/10864317/how-to-break-a-file-into-pieces-using-java
+ */
+ public class FileSplit {
+
+ public static List<File> split(File f, File dir, int splitSize) throws IOException {
+ int partCounter = 1;
+
+ long sizeOfFiles = splitSize * FileUtils.ONE_MB;
+ byte[] buffer = new byte[(int) sizeOfFiles];
+
+ List<File> files = new ArrayList<>();
+ String fileName = f.getName();
+
+ try (FileInputStream fis = new FileInputStream(f); BufferedInputStream bis = new BufferedInputStream(fis)) {
+ int bytesAmount = 0;
+ while ((bytesAmount = bis.read(buffer)) > 0) {
+ String filePartName = String.format("%s.%03d", fileName, partCounter++);
+ File newFile = new File(dir, filePartName);
+ try (FileOutputStream out = new FileOutputStream(newFile)) {
+ out.write(buffer, 0, bytesAmount);
+ }
+ files.add(newFile);
+ }
+ }
+ return files;
+ }
+ }
--- /dev/null
+ # Arvados client default configuration
+ #
+ # Remarks:
+ # * While providing data remove apostrophes ("") from each line
+ # * See Arvados documentation for information how to obtain a token:
+ # https://doc.arvados.org/user/reference/api-tokens.html
+ #
+
+ arvados {
+ api {
+ keepweb-host = localhost
+ keepweb-port = 8000
+ host = localhost
+ port = 8000
+ token = ""
+ protocol = https
+ host-insecure = false
+ }
+ split-size = 64
+ temp-dir = /tmp/file-split
+ copies = 2
+ retries = 0
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.HttpUrl;
+ import org.arvados.client.api.model.Item;
+ import org.arvados.client.api.model.ItemList;
+ import org.arvados.client.test.utils.ArvadosClientUnitTest;
+ import org.junit.Test;
+ import org.junit.runner.RunWith;
+ import org.mockito.Spy;
+ import org.mockito.junit.MockitoJUnitRunner;
+
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ @RunWith(MockitoJUnitRunner.class)
+ public class BaseStandardApiClientTest extends ArvadosClientUnitTest {
+
+ @Spy
+ private BaseStandardApiClient<?, ?> client = new BaseStandardApiClient<Item, ItemList>(CONFIG) {
+ @Override
+ String getResource() {
+ return "resource";
+ }
+
+ @Override
+ Class<Item> getType() {
+ return null;
+ }
+
+ @Override
+ Class<ItemList> getListType() {
+ return null;
+ }
+ };
+
+ @Test
+ public void urlBuilderBuildsExpectedUrlFormat() {
+ // when
+ HttpUrl.Builder actual = client.getUrlBuilder();
+
+ // then
+ assertThat(actual.build().toString()).isEqualTo("http://localhost:9000/arvados/v1/resource");
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.mockwebserver.RecordedRequest;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.api.model.CollectionList;
+ import org.arvados.client.test.utils.RequestMethod;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Test;
+
+ import static org.arvados.client.test.utils.ApiClientTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class CollectionsApiClientTest extends ArvadosClientMockedWebServerTest {
+
+ private static final String RESOURCE = "collections";
+
+ private CollectionsApiClient client = new CollectionsApiClient(CONFIG);
+
+ @Test
+ public void listCollections() throws Exception {
+
+ // given
+ server.enqueue(getResponse("collections-list"));
+
+ // when
+ CollectionList actual = client.list();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getItemsAvailable()).isEqualTo(41);
+ }
+
+ @Test
+ public void getCollection() throws Exception {
+
+ // given
+ server.enqueue(getResponse("collections-get"));
+
+ String uuid = "112ci-4zz18-p51w7z3fpopo6sm";
+
+ // when
+ Collection actual = client.get(uuid);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + "/" + uuid);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getUuid()).isEqualTo(uuid);
+ assertThat(actual.getPortableDataHash()).isEqualTo("6c4106229b08fe25f48b3a7a8289dd46+143");
+ }
+
+ @Test
+ public void createCollection() throws Exception {
+
+ // given
+ server.enqueue(getResponse("collections-create-simple"));
+
+ String name = "Super Collection";
+
+ Collection collection = new Collection();
+ collection.setName(name);
+
+ // when
+ Collection actual = client.create(collection);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.POST);
+ assertThat(actual.getName()).isEqualTo(name);
+ assertThat(actual.getPortableDataHash()).isEqualTo("d41d8cd98f00b204e9800998ecf8427e+0");
+ assertThat(actual.getManifestText()).isEmpty();
+ }
+
+ @Test
+ public void createCollectionWithManifest() throws Exception {
+
+ // given
+ server.enqueue(getResponse("collections-create-manifest"));
+
+ String name = "Super Collection";
+ String manifestText = ". 7df44272090cee6c0732382bba415ee9+70+Aa5ece4560e3329315165b36c239b8ab79c888f8a@5a1d5708 0:70:README.md\n";
+
+ Collection collection = new Collection();
+ collection.setName(name);
+ collection.setManifestText(manifestText);
+
+ // when
+ Collection actual = client.create(collection);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.POST);
+ assertThat(actual.getName()).isEqualTo(name);
+ assertThat(actual.getPortableDataHash()).isEqualTo("d41d8cd98f00b204e9800998ecf8427e+0");
+ assertThat(actual.getManifestText()).isEqualTo(manifestText);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import com.google.common.collect.Lists;
+ import okhttp3.mockwebserver.RecordedRequest;
+ import org.arvados.client.api.model.Group;
+ import org.arvados.client.api.model.GroupList;
+ import org.arvados.client.api.model.argument.Filter;
+ import org.arvados.client.api.model.argument.ListArgument;
+ import org.arvados.client.test.utils.RequestMethod;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Test;
+
+ import java.util.Arrays;
+
+ import static org.arvados.client.test.utils.ApiClientTestUtils.*;
+ import static org.junit.Assert.assertEquals;
+
+ public class GroupsApiClientTest extends ArvadosClientMockedWebServerTest {
+ private static final String RESOURCE = "groups";
+ private GroupsApiClient client = new GroupsApiClient(CONFIG);
+
+ @Test
+ public void listGroups() throws Exception {
+
+ // given
+ server.enqueue(getResponse("groups-list"));
+
+ // when
+ GroupList actual = client.list();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertEquals(20, actual.getItems().size());
+ }
+
+ @Test
+ public void listProjectsByOwner() throws Exception {
+
+ // given
+ server.enqueue(getResponse("groups-list"));
+ String ownerUuid = "ardev-tpzed-n3kzq4fvoks3uw4";
+ String filterSubPath = "?filters=[%20[%20%22owner_uuid%22,%20%22like%22,%20%22ardev-tpzed-n3kzq4fvoks3uw4%22%20],%20" +
+ "[%20%22group_class%22,%20%22in%22,%20[%20%22project%22,%20%22sub-project%22%20]%20]%20]";
+
+ // when
+ ListArgument listArgument = ListArgument.builder()
+ .filters(Arrays.asList(
+ Filter.of("owner_uuid", Filter.Operator.LIKE, ownerUuid),
+ Filter.of("group_class", Filter.Operator.IN, Lists.newArrayList("project", "sub-project")
+ )))
+ .build();
+ GroupList actual = client.list(listArgument);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + filterSubPath);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertEquals(20, actual.getItems().size());
+ }
+
+ @Test
+ public void getGroup() throws Exception {
+
+ // given
+ server.enqueue(getResponse("groups-get"));
+
+ String uuid = "ardev-j7d0g-bmg3pfqtx3ivczp";
+
+ // when
+ Group actual = client.get(uuid);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + "/" + uuid);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertEquals(uuid, actual.getUuid());
+ assertEquals("3hw0vk4mbl0ofvia5k6x4dwrx", actual.getEtag());
+ assertEquals("ardev-tpzed-n3kzq4fvoks3uw4", actual.getOwnerUuid());
+ assertEquals("TestGroup1", actual.getName());
+ assertEquals("project", actual.getGroupClass());
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import com.google.common.collect.Maps;
+ import okhttp3.mockwebserver.MockResponse;
+ import okhttp3.mockwebserver.RecordedRequest;
+ import okio.Buffer;
+ import org.apache.commons.io.FileUtils;
+ import org.arvados.client.common.Headers;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Test;
+
+ import java.io.File;
+ import java.util.Map;
+
+ import static org.arvados.client.test.utils.ApiClientTestUtils.assertAuthorizationHeader;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class KeepServerApiClientTest extends ArvadosClientMockedWebServerTest {
+
+ private KeepServerApiClient client = new KeepServerApiClient(CONFIG);
+
+ @Test
+ public void uploadFileToServer() throws Exception {
+
+ // given
+ String blockLocator = "7df44272090cee6c0732382bba415ee9";
+ String signedBlockLocator = blockLocator + "+70+A189a93acda6e1fba18a9dffd42b6591cbd36d55d@5a1c17b6";
+ server.enqueue(new MockResponse().setBody(signedBlockLocator));
+
+ String url = server.url(blockLocator).toString();
+ File body = new File("README.md");
+ Map<String, String> headers = Maps.newHashMap();
+ headers.put(Headers.X_KEEP_DESIRED_REPLICAS, "2");
+
+ // when
+ String actual = client.upload(url, headers, body);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertThat(request.getPath()).isEqualTo("/" + blockLocator);
+
+ assertThat(actual).isEqualTo(signedBlockLocator);
+ }
+
+ @Test
+ public void downloadFileFromServer() throws Exception {
+ File data = new File("README.md");
+ byte[] fileBytes = FileUtils.readFileToByteArray(data);
+ server.enqueue(new MockResponse().setBody(new Buffer().write(fileBytes)));
+
+ String blockLocator = "7df44272090cee6c0732382bba415ee9";
+ String signedBlockLocator = blockLocator + "+70+A189a93acda6e1fba18a9dffd42b6591cbd36d55d@5a1c17b6";
+
+ String url = server.url(signedBlockLocator).toString();
+
+ byte[] actual = client.download(url);
+ RecordedRequest request = server.takeRequest();
+ assertThat(request.getPath()).isEqualTo("/" + signedBlockLocator);
+ assertThat(actual).isEqualTo(fileBytes);
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.mockwebserver.RecordedRequest;
+ import org.arvados.client.api.model.KeepService;
+ import org.arvados.client.api.model.KeepServiceList;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Test;
+
+ import static org.arvados.client.test.utils.ApiClientTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class KeepServicesApiClientTest extends ArvadosClientMockedWebServerTest {
+
+ private static final String RESOURCE = "keep_services";
+
+ private KeepServicesApiClient client = new KeepServicesApiClient(CONFIG);
+
+ @Test
+ public void listKeepServices() throws Exception {
+
+ // given
+ server.enqueue(getResponse("keep-services-list"));
+
+ // when
+ KeepServiceList actual = client.list();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+
+ assertThat(actual.getItemsAvailable()).isEqualTo(3);
+
+ }
+
+ @Test
+ public void listAccessibleKeepServices() throws Exception {
+
+ // given
+ server.enqueue(getResponse("keep-services-accessible"));
+
+ // when
+ KeepServiceList actual = client.accessible();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + "/accessible");
+ assertThat(actual.getItemsAvailable()).isEqualTo(2);
+ }
+
+ @Test
+ public void getKeepService() throws Exception {
+
+ // given
+ server.enqueue(getResponse("keep-services-get"));
+
+ String uuid = "112ci-bi6l4-hv02fg8sbti8ykk";
+
+ // whenFs
+ KeepService actual = client.get(uuid);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + "/" + uuid);
+ assertThat(actual.getUuid()).isEqualTo(uuid);
+ assertThat(actual.getServiceType()).isEqualTo("disk");
+ }
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client;
+
+ import okhttp3.mockwebserver.RecordedRequest;
+ import org.arvados.client.api.model.User;
+ import org.arvados.client.api.model.UserList;
+ import org.arvados.client.test.utils.RequestMethod;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Test;
+
+ import static org.arvados.client.common.Characters.SLASH;
+ import static org.arvados.client.test.utils.ApiClientTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class UsersApiClientTest extends ArvadosClientMockedWebServerTest {
+
+ private static final String RESOURCE = "users";
+ private static final String USER_UUID = "ardev-tpzed-q6dvn7sby55up1b";
+
+ private UsersApiClient client = new UsersApiClient(CONFIG);
+
+ @Test
+ public void listUsers() throws Exception {
+
+ // given
+ server.enqueue(getResponse("users-list"));
+
+ // when
+ UserList actual = client.list();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getItemsAvailable()).isEqualTo(13);
+ }
+
+ @Test
+ public void getUser() throws Exception {
+
+ // given
+ server.enqueue(getResponse("users-get"));
+
+ // when
+ User actual = client.get(USER_UUID);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + SLASH + USER_UUID);
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getUuid()).isEqualTo(USER_UUID);
+ }
+
+ @Test
+ public void getCurrentUser() throws Exception {
+
+ // given
+ server.enqueue(getResponse("users-get"));
+
+ // when
+ User actual = client.current();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + SLASH + "current");
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getUuid()).isEqualTo(USER_UUID);
+ }
+
+ @Test
+ public void getSystemUser() throws Exception {
+
+ // given
+ server.enqueue(getResponse("users-system"));
+
+ // when
+ User actual = client.system();
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE + SLASH + "system");
+ assertRequestMethod(request, RequestMethod.GET);
+ assertThat(actual.getUuid()).isEqualTo("ardev-tpzed-000000000000000");
+ }
+
+ @Test
+ public void createUser() throws Exception {
+
+ // given
+ server.enqueue(getResponse("users-create"));
+
+ String firstName = "John";
+ String lastName = "Wayne";
+ String fullName = String.format("%s %s", firstName, lastName);
+ String username = String.format("%s%s", firstName, lastName).toLowerCase();
+
+ User user = new User();
+ user.setFirstName(firstName);
+ user.setLastName(lastName);
+ user.setFullName(fullName);
+ user.setUsername(username);
+
+ // when
+ User actual = client.create(user);
+
+ // then
+ RecordedRequest request = server.takeRequest();
+ assertAuthorizationHeader(request);
+ assertRequestPath(request, RESOURCE);
+ assertRequestMethod(request, RequestMethod.POST);
+ assertThat(actual.getFirstName()).isEqualTo(firstName);
+ assertThat(actual.getLastName()).isEqualTo(lastName);
+ assertThat(actual.getFullName()).isEqualTo(fullName);
+ assertThat(actual.getUsername()).isEqualTo(username);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.api.client.factory;
+
+ import okhttp3.OkHttpClient;
+ import okhttp3.Request;
+ import okhttp3.Response;
+ import okhttp3.mockwebserver.MockResponse;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Assert;
+ import org.junit.Test;
+ import org.junit.runner.RunWith;
+ import org.mockito.junit.MockitoJUnitRunner;
+
+ import javax.net.ssl.KeyManagerFactory;
+ import javax.net.ssl.SSLContext;
+ import javax.net.ssl.SSLSocketFactory;
+ import javax.net.ssl.TrustManagerFactory;
+ import java.io.FileInputStream;
+ import java.security.KeyStore;
+
+
+ @RunWith(MockitoJUnitRunner.class)
+ public class OkHttpClientFactoryTest extends ArvadosClientMockedWebServerTest {
+
+ @Test(expected = javax.net.ssl.SSLHandshakeException.class)
+ public void secureOkHttpClientIsCreated() throws Exception {
+
+ // given
+ OkHttpClientFactory factory = OkHttpClientFactory.builder().build();
+ // * configure HTTPS server
+ SSLSocketFactory sf = getSSLSocketFactoryWithSelfSignedCertificate();
+ server.useHttps(sf, false);
+ server.enqueue(new MockResponse().setBody("OK"));
+ // * prepare client HTTP request
+ Request request = new Request.Builder()
+ .url("https://localhost:9000/")
+ .build();
+
+ // when - then (SSL certificate is verified)
+ OkHttpClient actual = factory.create(false);
+ Response response = actual.newCall(request).execute();
+ }
+
+ @Test
+ public void insecureOkHttpClientIsCreated() throws Exception {
+ // given
+ OkHttpClientFactory factory = OkHttpClientFactory.builder().build();
+ // * configure HTTPS server
+ SSLSocketFactory sf = getSSLSocketFactoryWithSelfSignedCertificate();
+ server.useHttps(sf, false);
+ server.enqueue(new MockResponse().setBody("OK"));
+ // * prepare client HTTP request
+ Request request = new Request.Builder()
+ .url("https://localhost:9000/")
+ .build();
+
+ // when (SSL certificate is not verified)
+ OkHttpClient actual = factory.create(true);
+ Response response = actual.newCall(request).execute();
+
+ // then
+ Assert.assertEquals(response.body().string(),"OK");
+ }
+
+
+ /*
+ This ugly boilerplate is needed to enable self signed certificate.
+
+ It requires selfsigned.keystore.jks file. It was generated with:
+ keytool -genkey -v -keystore mystore.keystore.jks -alias alias_name -keyalg RSA -keysize 2048 -validity 10000
+ */
+ public SSLSocketFactory getSSLSocketFactoryWithSelfSignedCertificate() throws Exception {
+
+ FileInputStream stream = new FileInputStream("src/test/resources/selfsigned.keystore.jks");
+ char[] serverKeyStorePassword = "123456".toCharArray();
+ KeyStore serverKeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
+ serverKeyStore.load(stream, serverKeyStorePassword);
+
+ String kmfAlgorithm = KeyManagerFactory.getDefaultAlgorithm();
+ KeyManagerFactory kmf = KeyManagerFactory.getInstance(kmfAlgorithm);
+ kmf.init(serverKeyStore, serverKeyStorePassword);
+
+ TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(kmfAlgorithm);
+ trustManagerFactory.init(serverKeyStore);
+
+ SSLContext sslContext = SSLContext.getInstance("SSL");
+ sslContext.init(kmf.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);
+ return sslContext.getSocketFactory();
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.facade;
+
+ import org.apache.commons.io.FileUtils;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.config.ExternalConfigProvider;
+ import org.arvados.client.junit.categories.IntegrationTests;
+ import org.arvados.client.logic.collection.FileToken;
+ import org.arvados.client.test.utils.ArvadosClientIntegrationTest;
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.junit.After;
+ import org.junit.Assert;
+ import org.junit.Before;
+ import org.junit.Test;
+ import org.junit.experimental.categories.Category;
+
+ import java.io.File;
+ import java.util.Collections;
+ import java.util.List;
+ import java.util.UUID;
+
+ import static org.arvados.client.test.utils.FileTestUtils.FILE_DOWNLOAD_TEST_DIR;
+ import static org.arvados.client.test.utils.FileTestUtils.FILE_SPLIT_TEST_DIR;
+ import static org.arvados.client.test.utils.FileTestUtils.TEST_FILE;
+ import static org.assertj.core.api.Assertions.assertThat;
+ import static org.junit.Assert.assertEquals;
+ import static org.junit.Assert.assertTrue;
+
+ @Category(IntegrationTests.class)
+ public class ArvadosFacadeIntegrationTest extends ArvadosClientIntegrationTest {
+
+
+ private static final String COLLECTION_NAME = "Test collection " + UUID.randomUUID().toString();
+ private String collectionUuid;
+
+ @Before
+ public void setUp() throws Exception {
+ FileTestUtils.createDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.createDirectory(FILE_DOWNLOAD_TEST_DIR);
+ }
+
+ @Test
+ public void uploadOfFileIsPerformedSuccessfully() throws Exception {
+ // given
+ File file = FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_FOURTH_GB / 200);
+
+ // when
+ Collection actual = FACADE.upload(Collections.singletonList(file), COLLECTION_NAME, PROJECT_UUID);
+ collectionUuid = actual.getUuid();
+
+ // then
+ assertThat(actual.getName()).contains("Test collection");
+ assertThat(actual.getManifestText()).contains(file.length() + Characters.COLON + file.getName());
+ }
+
+ @Test
+ public void uploadOfFilesIsPerformedSuccessfully() throws Exception {
+ // given
+ List<File> files = FileTestUtils.generatePredefinedFiles();
+ files.addAll(FileTestUtils.generatePredefinedFiles());
+
+ // when
+ Collection actual = FACADE.upload(files, COLLECTION_NAME, PROJECT_UUID);
+ collectionUuid = actual.getUuid();
+
+ // then
+ assertThat(actual.getName()).contains("Test collection");
+ files.forEach(f -> assertThat(actual.getManifestText()).contains(f.length() + Characters.COLON + f.getName().replace(" ", Characters.SPACE)));
+ }
+
+ @Test
+ public void uploadToExistingCollectionIsPerformedSuccessfully() throws Exception {
+ // given
+ File file = FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_EIGTH_GB / 500);
+ Collection existing = createTestCollection();
+
+ // when
+ Collection actual = FACADE.uploadToExistingCollection(Collections.singletonList(file), collectionUuid);
+
+ // then
+ assertEquals(collectionUuid, actual.getUuid());
+ assertThat(actual.getManifestText()).contains(file.length() + Characters.COLON + file.getName());
+ }
+
+ @Test
+ public void uploadWithExternalConfigProviderWorksProperly() throws Exception {
+ //given
+ ArvadosFacade facade = new ArvadosFacade(buildExternalConfig());
+ File file = FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_FOURTH_GB / 200);
+
+ //when
+ Collection actual = facade.upload(Collections.singletonList(file), COLLECTION_NAME, PROJECT_UUID);
+ collectionUuid = actual.getUuid();
+
+ //then
+ assertThat(actual.getName()).contains("Test collection");
+ assertThat(actual.getManifestText()).contains(file.length() + Characters.COLON + file.getName());
+ }
+
+ @Test
+ public void creationOfEmptyCollectionPerformedSuccesfully() {
+ // given
+ String collectionName = "Empty collection " + UUID.randomUUID().toString();
+
+ // when
+ Collection actual = FACADE.createEmptyCollection(collectionName, PROJECT_UUID);
+ collectionUuid = actual.getUuid();
+
+ // then
+ assertEquals(collectionName, actual.getName());
+ assertEquals(PROJECT_UUID, actual.getOwnerUuid());
+ }
+
+ @Test
+ public void fileTokensAreListedFromCollection() throws Exception {
+ //given
+ List<File> files = uploadTestFiles();
+
+ //when
+ List<FileToken> actual = FACADE.listFileInfoFromCollection(collectionUuid);
+
+ //then
+ assertEquals(files.size(), actual.size());
+ for (int i = 0; i < files.size(); i++) {
+ assertEquals(files.get(i).length(), actual.get(i).getFileSize());
+ }
+ }
+
+ @Test
+ public void downloadOfFilesPerformedSuccessfully() throws Exception {
+ //given
+ List<File> files = uploadTestFiles();
+ File destination = new File(FILE_DOWNLOAD_TEST_DIR + Characters.SLASH + collectionUuid);
+
+ //when
+ List<File> actual = FACADE.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, false);
+
+ //then
+ assertEquals(files.size(), actual.size());
+ assertTrue(destination.exists());
+ assertThat(actual).allMatch(File::exists);
+ for (int i = 0; i < files.size(); i++) {
+ assertEquals(files.get(i).length(), actual.get(i).length());
+ }
+ }
+
+ @Test
+ public void downloadOfFilesPerformedSuccessfullyUsingKeepWeb() throws Exception {
+ //given
+ List<File> files = uploadTestFiles();
+ File destination = new File(FILE_DOWNLOAD_TEST_DIR + Characters.SLASH + collectionUuid);
+
+ //when
+ List<File> actual = FACADE.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, true);
+
+ //then
+ assertEquals(files.size(), actual.size());
+ assertTrue(destination.exists());
+ assertThat(actual).allMatch(File::exists);
+ for (int i = 0; i < files.size(); i++) {
+ assertEquals(files.get(i).length(), actual.get(i).length());
+ }
+ }
+
+ @Test
+ public void singleFileIsDownloadedSuccessfullyUsingKeepWeb() throws Exception {
+ //given
+ File file = uploadSingleTestFile(false);
+
+ //when
+ File actual = FACADE.downloadFile(file.getName(), collectionUuid, FILE_DOWNLOAD_TEST_DIR);
+
+ //then
+ assertThat(actual).exists();
+ assertThat(actual.length()).isEqualTo(file.length());
+ }
+
+ @Test
+ public void downloadOfOneFileSplittedToMultipleLocatorsPerformedSuccesfully() throws Exception {
+ //given
+ File file = uploadSingleTestFile(true);
+
+ List<File> actual = FACADE.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, false);
+
+ Assert.assertEquals(1, actual.size());
+ assertThat(actual.get(0).length()).isEqualTo(file.length());
+ }
+
+ @Test
+ public void downloadWithExternalConfigProviderWorksProperly() throws Exception {
+ //given
+ ArvadosFacade facade = new ArvadosFacade(buildExternalConfig());
+ List<File> files = uploadTestFiles();
+ //when
+ List<File> actual = facade.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, false);
+
+ //then
+ assertEquals(files.size(), actual.size());
+ assertThat(actual).allMatch(File::exists);
+ for (int i = 0; i < files.size(); i++) {
+ assertEquals(files.get(i).length(), actual.get(i).length());
+ }
+ }
+
+ private ExternalConfigProvider buildExternalConfig() {
+ return ExternalConfigProvider
+ .builder()
+ .apiHostInsecure(CONFIG.isApiHostInsecure())
+ .keepWebHost(CONFIG.getKeepWebHost())
+ .keepWebPort(CONFIG.getKeepWebPort())
+ .apiHost(CONFIG.getApiHost())
+ .apiPort(CONFIG.getApiPort())
+ .apiToken(CONFIG.getApiToken())
+ .apiProtocol(CONFIG.getApiProtocol())
+ .fileSplitSize(CONFIG.getFileSplitSize())
+ .fileSplitDirectory(CONFIG.getFileSplitDirectory())
+ .numberOfCopies(CONFIG.getNumberOfCopies())
+ .numberOfRetries(CONFIG.getNumberOfRetries())
+ .build();
+ }
+
+ private Collection createTestCollection() {
+ Collection collection = FACADE.createEmptyCollection(COLLECTION_NAME, PROJECT_UUID);
+ collectionUuid = collection.getUuid();
+ return collection;
+ }
+
+ private List<File> uploadTestFiles() throws Exception{
+ createTestCollection();
+ List<File> files = FileTestUtils.generatePredefinedFiles();
+ FACADE.uploadToExistingCollection(files, collectionUuid);
+ return files;
+ }
+
+ private File uploadSingleTestFile(boolean bigFile) throws Exception{
+ createTestCollection();
+ Long fileSize = bigFile ? FileUtils.ONE_MB * 70 : FileTestUtils.ONE_EIGTH_GB / 100;
+ File file = FileTestUtils.generateFile(TEST_FILE, fileSize);
+ FACADE.uploadToExistingCollection(Collections.singletonList(file), collectionUuid);
+ return file;
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ FileTestUtils.cleanDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.cleanDirectory(FILE_DOWNLOAD_TEST_DIR);
+
+ if(collectionUuid != null)
+ FACADE.deleteCollection(collectionUuid);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.facade;
+
+ import com.fasterxml.jackson.databind.ObjectMapper;
+ import com.fasterxml.jackson.databind.ObjectWriter;
+ import okhttp3.mockwebserver.MockResponse;
+ import okio.Buffer;
+ import org.apache.commons.io.FileUtils;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.api.model.KeepService;
+ import org.arvados.client.api.model.KeepServiceList;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.junit.After;
+ import org.junit.Before;
+ import org.junit.Test;
+
+ import java.io.File;
+ import java.nio.charset.Charset;
+ import java.nio.file.Files;
+ import java.util.ArrayList;
+ import java.util.Arrays;
+ import java.util.List;
+ import java.util.stream.Collectors;
+
+ import static org.arvados.client.test.utils.ApiClientTestUtils.getResponse;
+ import static org.arvados.client.test.utils.FileTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+ import static org.junit.Assert.assertEquals;
+ import static org.junit.Assert.assertTrue;
+
+ public class ArvadosFacadeTest extends ArvadosClientMockedWebServerTest {
+
+ ArvadosFacade facade = new ArvadosFacade(CONFIG);
+
+ @Before
+ public void setUp() throws Exception {
+ FileTestUtils.createDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.createDirectory(FILE_DOWNLOAD_TEST_DIR);
+ }
+
+ @Test
+ public void uploadIsPerformedSuccessfullyUsingDiskOnlyKeepServices() throws Exception {
+
+ // given
+ String keepServicesAccessible = setMockedServerPortToKeepServices("keep-services-accessible-disk-only");
+ server.enqueue(new MockResponse().setBody(keepServicesAccessible));
+
+ String blockLocator = "7df44272090cee6c0732382bba415ee9";
+ String signedBlockLocator = blockLocator + "+70+A189a93acda6e1fba18a9dffd42b6591cbd36d55d@5a1c17b6";
+ for (int i = 0; i < 8; i++) {
+ server.enqueue(new MockResponse().setBody(signedBlockLocator));
+ }
+ server.enqueue(getResponse("users-get"));
+ server.enqueue(getResponse("collections-create-manifest"));
+
+ FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_FOURTH_GB);
+
+ // when
+ Collection actual = facade.upload(Arrays.asList(new File(TEST_FILE)), "Super Collection", null);
+
+ // then
+ assertThat(actual.getName()).contains("Super Collection");
+ }
+
+ @Test
+ public void uploadIsPerformedSuccessfully() throws Exception {
+
+ // given
+ String keepServicesAccessible = setMockedServerPortToKeepServices("keep-services-accessible");
+ server.enqueue(new MockResponse().setBody(keepServicesAccessible));
+
+ String blockLocator = "7df44272090cee6c0732382bba415ee9";
+ String signedBlockLocator = blockLocator + "+70+A189a93acda6e1fba18a9dffd42b6591cbd36d55d@5a1c17b6";
+ for (int i = 0; i < 4; i++) {
+ server.enqueue(new MockResponse().setBody(signedBlockLocator));
+ }
+ server.enqueue(getResponse("users-get"));
+ server.enqueue(getResponse("collections-create-manifest"));
+
+ FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_FOURTH_GB);
+
+ // when
+ Collection actual = facade.upload(Arrays.asList(new File(TEST_FILE)), "Super Collection", null);
+
+ // then
+ assertThat(actual.getName()).contains("Super Collection");
+ }
+
+ @Test
+ public void downloadOfWholeCollectionIsPerformedSuccessfully() throws Exception {
+
+ //given
+ String collectionUuid = "ardev-4zz18-jk5vo4uo9u5vj52";
+ server.enqueue(getResponse("collections-download-file"));
+
+ String keepServicesAccessible = setMockedServerPortToKeepServices("keep-services-accessible");
+ server.enqueue(new MockResponse().setBody(keepServicesAccessible));
+ File collectionDestination = new File(FILE_DOWNLOAD_TEST_DIR + Characters.SLASH + collectionUuid);
+
+ List<File> files = generatePredefinedFiles();
+ List<byte[]> fileData = new ArrayList<>();
+ for (File f : files) {
+ fileData.add(Files.readAllBytes(f.toPath()));
+ }
+ byte[] filesDataChunk = fileData.stream().reduce(new byte[0], this::addAll);
+
+ server.enqueue(new MockResponse().setBody(new Buffer().write(filesDataChunk)));
+
+ //when
+ List<File> downloadedFiles = facade.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, false);
+
+ //then
+ assertEquals(3, downloadedFiles.size());
+ assertTrue(collectionDestination.exists());
+ assertThat(downloadedFiles).allMatch(File::exists);
+ assertEquals(files.stream().map(File::getName).collect(Collectors.toList()), downloadedFiles.stream().map(File::getName).collect(Collectors.toList()));
+ assertEquals(files.stream().map(File::length).collect(Collectors.toList()), downloadedFiles.stream().map(File::length).collect(Collectors.toList()));
+ }
+
+ @Test
+ public void downloadOfWholeCollectionUsingKeepWebPerformedSuccessfully() throws Exception {
+
+ //given
+ String collectionUuid = "ardev-4zz18-jk5vo4uo9u5vj52";
+ server.enqueue(getResponse("collections-download-file"));
+
+ List<File> files = generatePredefinedFiles();
+ for (File f : files) {
+ server.enqueue(new MockResponse().setBody(new Buffer().write(FileUtils.readFileToByteArray(f))));
+ }
+
+ //when
+ List<File> downloadedFiles = facade.downloadCollectionFiles(collectionUuid, FILE_DOWNLOAD_TEST_DIR, true);
+
+ //then
+ assertEquals(3, downloadedFiles.size());
+ assertThat(downloadedFiles).allMatch(File::exists);
+ assertEquals(files.stream().map(File::getName).collect(Collectors.toList()), downloadedFiles.stream().map(File::getName).collect(Collectors.toList()));
+ assertTrue(downloadedFiles.stream().map(File::length).collect(Collectors.toList()).containsAll(files.stream().map(File::length).collect(Collectors.toList())));
+ }
+
+ @Test
+ public void downloadOfSingleFilePerformedSuccessfully() throws Exception {
+
+ //given
+ String collectionUuid = "ardev-4zz18-jk5vo4uo9u5vj52";
+ server.enqueue(getResponse("collections-download-file"));
+
+ File file = generatePredefinedFiles().get(0);
+ byte[] fileData = FileUtils.readFileToByteArray(file);
+ server.enqueue(new MockResponse().setBody(new Buffer().write(fileData)));
+
+ //when
+ File downloadedFile = facade.downloadFile(file.getName(), collectionUuid, FILE_DOWNLOAD_TEST_DIR);
+
+ //then
+ assertTrue(downloadedFile.exists());
+ assertEquals(file.getName(), downloadedFile.getName());
+ assertEquals(file.length(), downloadedFile.length());
+ }
+
+ private String setMockedServerPortToKeepServices(String jsonPath) throws Exception {
+
+ ObjectMapper mapper = new ObjectMapper().findAndRegisterModules();
+ String filePath = String.format("src/test/resources/org/arvados/client/api/client/%s.json", jsonPath);
+ File jsonFile = new File(filePath);
+ String json = FileUtils.readFileToString(jsonFile, Charset.defaultCharset());
+ KeepServiceList keepServiceList = mapper.readValue(json, KeepServiceList.class);
+ List<KeepService> items = keepServiceList.getItems();
+ for (KeepService keepService : items) {
+ keepService.setServicePort(server.getPort());
+ }
+ ObjectWriter writer = mapper.writer().withDefaultPrettyPrinter();
+ return writer.writeValueAsString(keepServiceList);
+ }
+
+ //Method to copy multiple byte[] arrays into one byte[] array
+ private byte[] addAll(byte[] array1, byte[] array2) {
+ byte[] joinedArray = new byte[array1.length + array2.length];
+ System.arraycopy(array1, 0, joinedArray, 0, array1.length);
+ System.arraycopy(array2, 0, joinedArray, array1.length, array2.length);
+ return joinedArray;
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ FileTestUtils.cleanDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.cleanDirectory(FILE_DOWNLOAD_TEST_DIR);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.junit.categories;
+
+ public interface IntegrationTests {}
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.common.Characters;
+ import org.junit.Assert;
+ import org.junit.Test;
+
+ public class FileTokenTest {
+
+ public static final String FILE_TOKEN_INFO = "0:1024:test-file1";
+ public static final int FILE_POSITION = 0;
+ public static final int FILE_LENGTH = 1024;
+ public static final String FILE_NAME = "test-file1";
+ public static final String FILE_PATH = "c" + Characters.SLASH;
+
+ private static FileToken fileToken = new FileToken(FILE_TOKEN_INFO);
+ private static FileToken fileTokenWithPath = new FileToken(FILE_TOKEN_INFO, FILE_PATH);
+
+ @Test
+ public void tokenInfoIsDividedCorrectly(){
+ Assert.assertEquals(FILE_NAME, fileToken.getFileName());
+ Assert.assertEquals(FILE_POSITION, fileToken.getFilePosition());
+ Assert.assertEquals(FILE_LENGTH, fileToken.getFileSize());
+ }
+
+ @Test
+ public void toStringReturnsOriginalFileTokenInfo(){
+ Assert.assertEquals(FILE_TOKEN_INFO, fileToken.toString());
+ }
+
+ @Test
+ public void fullPathIsReturnedProperly(){
+ Assert.assertEquals(FILE_NAME, fileToken.getFullPath());
+ Assert.assertEquals(FILE_PATH + FILE_NAME, fileTokenWithPath.getFullPath());
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.junit.Assert;
+ import org.junit.Test;
+
+ import java.util.List;
+
+ import static junit.framework.TestCase.fail;
+
+ public class ManifestDecoderTest {
+
+ private ManifestDecoder manifestDecoder = new ManifestDecoder();
+
+ private static final String ONE_LINE_MANIFEST_TEXT = ". " +
+ "eff999f3b5158331eb44a9a93e3b36e1+67108864+Aad3839bea88bce22cbfe71cf4943de7dab3ea52a@5826180f " +
+ "db141bfd11f7da60dce9e5ee85a988b8+34038725+Ae8f48913fed782cbe463e0499ab37697ee06a2f8@5826180f " +
+ "0:101147589:rna.SRR948778.bam" +
+ "\\n";
+
+ private static final String MULTIPLE_LINES_MANIFEST_TEXT = ". " +
+ "930625b054ce894ac40596c3f5a0d947+33 " +
+ "0:0:a 0:0:b 0:33:output.txt\n" +
+ "./c d41d8cd98f00b204e9800998ecf8427e+0 0:0:d";
+
+ private static final String MANIFEST_TEXT_WITH_INVALID_FIRST_PATH_COMPONENT = "a" + ONE_LINE_MANIFEST_TEXT;
+
+
+ @Test
+ public void allLocatorsAndFileTokensAreExtractedFromSimpleManifest() {
+
+ List<ManifestStream> actual = manifestDecoder.decode(ONE_LINE_MANIFEST_TEXT);
+
+ // one manifest stream
+ Assert.assertEquals(1, actual.size());
+
+ ManifestStream manifest = actual.get(0);
+ // two locators
+ Assert.assertEquals(2, manifest.getKeepLocators().size());
+ // one file token
+ Assert.assertEquals(1, manifest.getFileTokens().size());
+
+ }
+
+ @Test
+ public void allLocatorsAndFileTokensAreExtractedFromComplexManifest() {
+
+ List<ManifestStream> actual = manifestDecoder.decode(MULTIPLE_LINES_MANIFEST_TEXT);
+
+ // two manifest streams
+ Assert.assertEquals(2, actual.size());
+
+ // first stream - 1 locator and 3 file tokens
+ ManifestStream firstManifestStream = actual.get(0);
+ Assert.assertEquals(1, firstManifestStream.getKeepLocators().size());
+ Assert.assertEquals(3, firstManifestStream.getFileTokens().size());
+
+ // second stream - 1 locator and 1 file token
+ ManifestStream secondManifestStream = actual.get(1);
+ Assert.assertEquals(1, secondManifestStream.getKeepLocators().size());
+ Assert.assertEquals(1, secondManifestStream.getFileTokens().size());
+ }
+
+ @Test
+ public void manifestTextWithInvalidStreamNameThrowsException() {
+
+ try {
+ List<ManifestStream> actual = manifestDecoder.decode(MANIFEST_TEXT_WITH_INVALID_FIRST_PATH_COMPONENT);
+ fail();
+ } catch (ArvadosClientException e) {
+ Assert.assertEquals("Invalid first path component (expecting \".\")", e.getMessage());
+ }
+
+ }
+
+ @Test
+ public void emptyManifestTextThrowsException() {
+ String emptyManifestText = null;
+
+ try {
+ List<ManifestStream> actual = manifestDecoder.decode(emptyManifestText);
+ fail();
+ } catch (ArvadosClientException e) {
+ Assert.assertEquals("Manifest text cannot be empty.", e.getMessage());
+ }
+
+ emptyManifestText = "";
+ try {
+ List<ManifestStream> actual = manifestDecoder.decode(emptyManifestText);
+ fail();
+ } catch (ArvadosClientException e) {
+ Assert.assertEquals("Manifest text cannot be empty.", e.getMessage());
+ }
+
+ }
+
+
+
+
+
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.assertj.core.util.Lists;
+ import org.junit.Test;
+
+ import java.io.File;
+ import java.util.List;
+
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class ManifestFactoryTest {
+
+ @Test
+ public void manifestIsCreatedAsExpected() throws Exception {
+
+ // given
+ List<File> files = FileTestUtils.generatePredefinedFiles();
+ List<String> locators = Lists.newArrayList("a", "b", "c");
+ ManifestFactory factory = ManifestFactory.builder()
+ .files(files)
+ .locators(locators)
+ .build();
+
+ // when
+ String actual = factory.create();
+
+ // then
+ assertThat(actual).isEqualTo(". a b c 0:1024:test-file1 1024:20480:test-file2 21504:1048576:test-file\\0403\n");
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.collection;
+
+
+ import org.junit.Assert;
+ import org.junit.Test;
+
+ import java.util.List;
+
+ public class ManifestStreamTest {
+
+ private ManifestDecoder manifestDecoder = new ManifestDecoder();
+
+ @Test
+ public void toStringReturnsProperlyConnectedManifestStream() throws Exception{
+ String encodedManifest = ". eff999f3b5158331eb44a9a93e3b36e1+67108864 db141bfd11f7da60dce9e5ee85a988b8+34038725 0:101147589:rna.SRR948778.bam\\n\"";
+ List<ManifestStream> manifestStreams = manifestDecoder.decode(encodedManifest);
+ Assert.assertEquals(encodedManifest, manifestStreams.get(0).toString());
+
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import com.fasterxml.jackson.databind.ObjectMapper;
+ import org.arvados.client.api.client.CollectionsApiClient;
+ import org.arvados.client.api.client.KeepWebApiClient;
+ import org.arvados.client.api.model.Collection;
+ import org.arvados.client.common.Characters;
+ import org.arvados.client.logic.collection.FileToken;
+ import org.arvados.client.logic.collection.ManifestDecoder;
+ import org.arvados.client.logic.collection.ManifestStream;
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.arvados.client.utils.FileMerge;
+ import org.apache.commons.io.FileUtils;
+ import org.junit.After;
+ import org.junit.Assert;
+ import org.junit.Before;
+ import org.junit.Test;
+ import org.junit.runner.RunWith;
+ import org.mockito.InjectMocks;
+ import org.mockito.Mock;
+ import org.mockito.junit.MockitoJUnitRunner;
+
+ import java.io.File;
+ import java.io.IOException;
+ import java.util.ArrayList;
+ import java.util.Arrays;
+ import java.util.List;
+ import java.util.UUID;
+
+ import static org.arvados.client.test.utils.FileTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+ import static org.mockito.Mockito.when;
+
+ @RunWith(MockitoJUnitRunner.class)
+ public class FileDownloaderTest {
+
+ static final ObjectMapper MAPPER = new ObjectMapper().findAndRegisterModules();
+ private Collection collectionToDownload;
+ private ManifestStream manifestStream;
+
+ @Mock
+ private CollectionsApiClient collectionsApiClient;
+ @Mock
+ private KeepClient keepClient;
+ @Mock
+ private KeepWebApiClient keepWebApiClient;
+ @Mock
+ private ManifestDecoder manifestDecoder;
+ @InjectMocks
+ private FileDownloader fileDownloader;
+
+ @Before
+ public void setUp() throws Exception {
+ FileTestUtils.createDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.createDirectory(FILE_DOWNLOAD_TEST_DIR);
+
+ collectionToDownload = prepareCollection();
+ manifestStream = prepareManifestStream();
+ }
+
+ @Test
+ public void downloadingAllFilesFromCollectionWorksProperly() throws Exception {
+ // given
+ List<File> files = generatePredefinedFiles();
+ byte[] dataChunk = prepareDataChunk(files);
+
+ //having
+ when(collectionsApiClient.get(collectionToDownload.getUuid())).thenReturn(collectionToDownload);
+ when(manifestDecoder.decode(collectionToDownload.getManifestText())).thenReturn(Arrays.asList(manifestStream));
+ when(keepClient.getDataChunk(manifestStream.getKeepLocators().get(0))).thenReturn(dataChunk);
+
+ //when
+ List<File> downloadedFiles = fileDownloader.downloadFilesFromCollection(collectionToDownload.getUuid(), FILE_DOWNLOAD_TEST_DIR);
+
+ //then
+ Assert.assertEquals(3, downloadedFiles.size()); // 3 files downloaded
+
+ File collectionDir = new File(FILE_DOWNLOAD_TEST_DIR + Characters.SLASH + collectionToDownload.getUuid());
+ Assert.assertTrue(collectionDir.exists()); // collection directory created
+
+ // 3 files correctly saved
+ assertThat(downloadedFiles).allMatch(File::exists);
+
+ for(int i = 0; i < downloadedFiles.size(); i ++) {
+ File downloaded = new File(collectionDir + Characters.SLASH + files.get(i).getName());
+ Assert.assertArrayEquals(FileUtils.readFileToByteArray(downloaded), FileUtils.readFileToByteArray(files.get(i)));
+ }
+ }
+
+ @Test
+ public void downloadingSingleFileFromKeepWebWorksCorrectly() throws Exception{
+ //given
+ File file = generatePredefinedFiles().get(0);
+
+ //having
+ when(collectionsApiClient.get(collectionToDownload.getUuid())).thenReturn(collectionToDownload);
+ when(manifestDecoder.decode(collectionToDownload.getManifestText())).thenReturn(Arrays.asList(manifestStream));
+ when(keepWebApiClient.download(collectionToDownload.getUuid(), file.getName())).thenReturn(FileUtils.readFileToByteArray(file));
+
+ //when
+ File downloadedFile = fileDownloader.downloadSingleFileUsingKeepWeb(file.getName(), collectionToDownload.getUuid(), FILE_DOWNLOAD_TEST_DIR);
+
+ //then
+ Assert.assertTrue(downloadedFile.exists());
+ Assert.assertEquals(file.getName(), downloadedFile.getName());
+ Assert.assertArrayEquals(FileUtils.readFileToByteArray(downloadedFile), FileUtils.readFileToByteArray(file));
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ FileTestUtils.cleanDirectory(FILE_SPLIT_TEST_DIR);
+ FileTestUtils.cleanDirectory(FILE_DOWNLOAD_TEST_DIR);
+ }
+
+ private Collection prepareCollection() throws IOException {
+ // collection that will be returned by mocked collectionsApiClient
+ String filePath = "src/test/resources/org/arvados/client/api/client/collections-download-file.json";
+ File jsonFile = new File(filePath);
+ return MAPPER.readValue(jsonFile, Collection.class);
+ }
+
+ private ManifestStream prepareManifestStream() throws Exception {
+ // manifestStream that will be returned by mocked manifestDecoder
+ List<FileToken> fileTokens = new ArrayList<>();
+ fileTokens.add(new FileToken("0:1024:test-file1"));
+ fileTokens.add(new FileToken("1024:20480:test-file2"));
+ fileTokens.add(new FileToken("21504:1048576:test-file\\0403"));
+
+ KeepLocator keepLocator = new KeepLocator("163679d58edaadc28db769011728a72c+1070080+A3acf8c1fe582c265d2077702e4a7d74fcc03aba8@5aa4fdeb");
+ return new ManifestStream(".", Arrays.asList(keepLocator), fileTokens);
+ }
+
+ private byte[] prepareDataChunk(List<File> files) throws IOException {
+ File combinedFile = new File(FILE_SPLIT_TEST_DIR + Characters.SLASH + UUID.randomUUID());
+ FileMerge.merge(files, combinedFile);
+ return FileUtils.readFileToByteArray(combinedFile);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import okhttp3.mockwebserver.MockResponse;
+ import okio.Buffer;
+ import org.apache.commons.io.FileUtils;
+ import org.arvados.client.config.FileConfigProvider;
+ import org.arvados.client.config.ConfigProvider;
+ import org.arvados.client.exception.ArvadosClientException;
+ import org.arvados.client.test.utils.ArvadosClientMockedWebServerTest;
+ import org.junit.Assert;
+ import org.junit.Test;
+ import org.junit.runner.RunWith;
+ import org.mockito.InjectMocks;
+ import org.mockito.Mock;
+ import org.mockito.junit.MockitoJUnitRunner;
+
+ import java.io.File;
+
+ import static junit.framework.TestCase.fail;
+ import static org.arvados.client.test.utils.ApiClientTestUtils.getResponse;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ @RunWith(MockitoJUnitRunner.class)
+ public class KeepClientTest extends ArvadosClientMockedWebServerTest {
+
+ private ConfigProvider configProvider = new FileConfigProvider();
+ private static final String TEST_FILE_PATH ="src/test/resources/org/arvados/client/api/client/keep-client-test-file.txt";
+
+ @InjectMocks
+ private KeepClient keepClient = new KeepClient(configProvider);
+
+ @Mock
+ private KeepLocator keepLocator;
+
+ @Test
+ public void uploadedFile() throws Exception {
+ // given
+ server.enqueue(getResponse("keep-services-accessible"));
+ server.enqueue(new MockResponse().setBody("0887c78c7d6c1a60ac0b3709a4302ee4"));
+
+ // when
+ String actual = keepClient.put(new File(TEST_FILE_PATH), 1, 0);
+
+ // then
+ assertThat(actual).isEqualTo("0887c78c7d6c1a60ac0b3709a4302ee4");
+ }
+
+ @Test
+ public void fileIsDownloaded() throws Exception {
+ //given
+ File data = new File(TEST_FILE_PATH);
+ byte[] fileBytes = FileUtils.readFileToByteArray(data);
+
+ // when
+ server.enqueue(getResponse("keep-services-accessible"));
+ server.enqueue(new MockResponse().setBody(new Buffer().write(fileBytes)));
+
+ byte[] actual = keepClient.getDataChunk(keepLocator);
+
+ Assert.assertArrayEquals(fileBytes, actual);
+ }
+
+ @Test
+ public void fileIsDownloadedWhenFirstServerDoesNotRespond() throws Exception {
+ // given
+ File data = new File(TEST_FILE_PATH);
+ byte[] fileBytes = FileUtils.readFileToByteArray(data);
+ server.enqueue(getResponse("keep-services-accessible")); // two servers accessible
+ server.enqueue(new MockResponse().setResponseCode(404)); // first one not responding
+ server.enqueue(new MockResponse().setBody(new Buffer().write(fileBytes))); // second one responding
+
+ //when
+ byte[] actual = keepClient.getDataChunk(keepLocator);
+
+ //then
+ Assert.assertArrayEquals(fileBytes, actual);
+ }
+
+ @Test
+ public void exceptionIsThrownWhenNoServerResponds() throws Exception {
+ //given
+ File data = new File(TEST_FILE_PATH);
+ server.enqueue(getResponse("keep-services-accessible")); // two servers accessible
+ server.enqueue(new MockResponse().setResponseCode(404)); // first one not responding
+ server.enqueue(new MockResponse().setResponseCode(404)); // second one not responding
+
+ try {
+ //when
+ keepClient.getDataChunk(keepLocator);
+ fail();
+ } catch (ArvadosClientException e) {
+ //then
+ Assert.assertEquals("No server responding. Unable to download data chunk.", e.getMessage());
+ }
+ }
+
+ @Test
+ public void exceptionIsThrownWhenThereAreNoServersAccessible() throws Exception {
+ //given
+ server.enqueue(getResponse("keep-services-not-accessible")); // no servers accessible
+
+ try {
+ //when
+ keepClient.getDataChunk(keepLocator);
+ fail();
+ } catch (ArvadosClientException e) {
+ //then
+ Assert.assertEquals("No gateway services available!", e.getMessage());
+ }
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.logic.keep;
+
+ import org.junit.Test;
+
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class KeepLocatorTest {
+
+ private KeepLocator locator;
+
+ @Test
+ public void md5sumIsExtracted() throws Exception {
+
+ // given
+ locator = new KeepLocator("7df44272090cee6c0732382bba415ee9+70");
+
+ // when
+ String actual = locator.getMd5sum();
+
+ // then
+ assertThat(actual).isEqualTo("7df44272090cee6c0732382bba415ee9");
+ }
+
+ @Test
+ public void locatorIsStrippedWithMd5sumAndSize() throws Exception {
+
+ // given
+ locator = new KeepLocator("7df44272090cee6c0732382bba415ee9+70");
+
+ // when
+ String actual = locator.stripped();
+
+ // then
+ assertThat(actual).isEqualTo("7df44272090cee6c0732382bba415ee9+70");
+ }
+
+
+ @Test
+ public void locatorToStringProperlyShowing() throws Exception {
+
+ // given
+ locator = new KeepLocator("7df44272090cee6c0732382bba415ee9+70+Ae8f48913fed782cbe463e0499ab37697ee06a2f8@5826180f");
+
+ // when
+ String actual = locator.toString();
+
+ // then
+ assertThat(actual).isEqualTo("7df44272090cee6c0732382bba415ee9+70+Ae8f48913fed782cbe463e0499ab37697ee06a2f8@5826180f");
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ import org.arvados.client.config.FileConfigProvider;
+ import okhttp3.mockwebserver.MockResponse;
+ import okhttp3.mockwebserver.RecordedRequest;
+ import org.apache.commons.io.FileUtils;
+
+ import java.io.File;
+ import java.io.IOException;
+ import java.nio.charset.Charset;
+
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public final class ApiClientTestUtils {
+
+ static final String BASE_URL = "/arvados/v1/";
+
+ private ApiClientTestUtils() {}
+
+ public static MockResponse getResponse(String filename) throws IOException {
+ String filePath = String.format("src/test/resources/org/arvados/client/api/client/%s.json", filename);
+ File jsonFile = new File(filePath);
+ String json = FileUtils.readFileToString(jsonFile, Charset.defaultCharset());
+ return new MockResponse().setBody(json);
+ }
+
+ public static void assertAuthorizationHeader(RecordedRequest request) {
+ assertThat(request.getHeader("authorization")).isEqualTo("OAuth2 " + new FileConfigProvider().getApiToken());
+ }
+
+ public static void assertRequestPath(RecordedRequest request, String subPath) {
+ assertThat(request.getPath()).isEqualTo(BASE_URL + subPath);
+ }
+
+ public static void assertRequestMethod(RecordedRequest request, RequestMethod requestMethod) {
+ assertThat(request.getMethod()).isEqualTo(requestMethod.name());
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ import org.arvados.client.config.FileConfigProvider;
+ import org.arvados.client.facade.ArvadosFacade;
+ import org.junit.BeforeClass;
+
+ import static org.junit.Assert.assertTrue;
+
+ public class ArvadosClientIntegrationTest {
+
+ protected static final FileConfigProvider CONFIG = new FileConfigProvider("integration-tests-application.conf");
+ protected static final ArvadosFacade FACADE = new ArvadosFacade(CONFIG);
+ protected static final String PROJECT_UUID = CONFIG.getIntegrationTestProjectUuid();
+
+ @BeforeClass
+ public static void validateConfiguration(){
+ String msg = " info must be provided in configuration";
+ CONFIG.getConfig().entrySet()
+ .forEach(e -> assertTrue("Parameter " + e.getKey() + msg, !e.getValue().render().equals("\"\"")));
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ import okhttp3.mockwebserver.MockWebServer;
+ import org.junit.After;
+ import org.junit.Before;
+
+ public class ArvadosClientMockedWebServerTest extends ArvadosClientUnitTest {
+ private static final int PORT = CONFIG.getApiPort();
+ protected MockWebServer server = new MockWebServer();
+
+ @Before
+ public void setUpServer() throws Exception {
+ server.start(PORT);
+ }
+
+ @After
+ public void tearDownServer() throws Exception {
+ server.shutdown();
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ import org.arvados.client.config.FileConfigProvider;
+ import org.junit.BeforeClass;
+
+ import static org.junit.Assert.assertTrue;
+
+ public class ArvadosClientUnitTest {
+
+ protected static final FileConfigProvider CONFIG = new FileConfigProvider("application.conf");
+
+ @BeforeClass
+ public static void validateConfiguration(){
+ String msg = " info must be provided in configuration";
+ CONFIG.getConfig().entrySet().forEach(e -> assertTrue("Parameter " + e.getKey() + msg, !e.getValue().render().equals("\"\"")));
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ import org.apache.commons.io.FileUtils;
+ import org.assertj.core.util.Lists;
+
+ import java.io.File;
+ import java.io.IOException;
+ import java.io.RandomAccessFile;
+ import java.util.List;
+
+ public class FileTestUtils {
+
+ public static final String FILE_SPLIT_TEST_DIR = "/tmp/file-split";
+ public static final String FILE_DOWNLOAD_TEST_DIR = "/tmp/arvados-downloaded";
+ public static final String TEST_FILE = FILE_SPLIT_TEST_DIR + "/test-file";
+ public static long ONE_FOURTH_GB = FileUtils.ONE_GB / 4;
+ public static long ONE_EIGTH_GB = FileUtils.ONE_GB / 8;
+ public static long HALF_GB = FileUtils.ONE_GB / 2;
+ public static int FILE_SPLIT_SIZE = 64;
+
+ public static void createDirectory(String path) throws Exception {
+ new File(path).mkdirs();
+ }
+
+ public static void cleanDirectory(String directory) throws Exception {
+ FileUtils.cleanDirectory(new File(directory));
+ }
+
+ public static File generateFile(String path, long length) throws IOException {
+ RandomAccessFile testFile = new RandomAccessFile(path, "rwd");
+ testFile.setLength(length);
+ testFile.close();
+ return new File(path);
+ }
+
+ public static List<File> generatePredefinedFiles() throws IOException {
+ return Lists.newArrayList(
+ generateFile(TEST_FILE + 1, FileUtils.ONE_KB),
+ generateFile(TEST_FILE + 2, FileUtils.ONE_KB * 20),
+ generateFile(TEST_FILE + " " + 3, FileUtils.ONE_MB)
+ );
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.test.utils;
+
+ public enum RequestMethod {
+
+ GET, POST, PUT, DELETE
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.utils;
+
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.junit.After;
+ import org.junit.Before;
+ import org.junit.Test;
+
+ import java.io.File;
+ import java.util.List;
+
+ import static org.arvados.client.test.utils.FileTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class FileMergeTest {
+
+ @Before
+ public void setUp() throws Exception {
+ FileTestUtils.createDirectory(FILE_SPLIT_TEST_DIR);
+ }
+
+ @Test
+ public void fileChunksAreMergedIntoOneFile() throws Exception {
+
+ // given
+ FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_EIGTH_GB);
+
+ List<File> files = FileSplit.split(new File(TEST_FILE), new File(FILE_SPLIT_TEST_DIR), FILE_SPLIT_SIZE);
+ File targetFile = new File(TEST_FILE);
+
+ // when
+ FileMerge.merge(files, targetFile);
+
+ // then
+ assertThat(targetFile.length()).isEqualTo(FileTestUtils.ONE_EIGTH_GB);
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ FileTestUtils.cleanDirectory(FILE_SPLIT_TEST_DIR);
+ }
+ }
--- /dev/null
+ /*
+ * Copyright (C) The Arvados Authors. All rights reserved.
+ *
+ * SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ *
+ */
+
+ package org.arvados.client.utils;
+
+ import org.arvados.client.test.utils.FileTestUtils;
+ import org.junit.After;
+ import org.junit.Before;
+ import org.junit.Test;
+
+ import java.io.File;
+ import java.util.List;
+
+ import static org.arvados.client.test.utils.FileTestUtils.*;
+ import static org.assertj.core.api.Assertions.assertThat;
+
+ public class FileSplitTest {
+
+ @Before
+ public void setUp() throws Exception {
+ FileTestUtils.createDirectory(FILE_SPLIT_TEST_DIR);
+ }
+
+ @Test
+ public void fileIsDividedIntoSmallerChunks() throws Exception {
+
+ // given
+ int expectedSize = 2;
+ int expectedFileSizeInBytes = 67108864;
+ FileTestUtils.generateFile(TEST_FILE, FileTestUtils.ONE_EIGTH_GB);
+
+ // when
+ List<File> actual = FileSplit.split(new File(TEST_FILE), new File(FILE_SPLIT_TEST_DIR), FILE_SPLIT_SIZE);
+
+ // then
+ assertThat(actual).hasSize(expectedSize);
+ assertThat(actual).allMatch(a -> a.length() == expectedFileSizeInBytes);
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ FileTestUtils.cleanDirectory(FILE_SPLIT_TEST_DIR);
+ }
+ }
--- /dev/null
+ # configuration for unit tests
+
+ arvados {
+ api {
+ port = 9000
+ keepweb-port = 9000
+ token = 1m69yw9m2wanubzyfkb1e9icplqhtr2r969bu9rnzqbqhb7cnb
+ protocol = "http"
+ }
+ }
--- /dev/null
+ # Configuration for integration tests
+ #
+ # Remarks:
+ # * For example see integration-tests-application.conf.example
+ # * While providing data remove apostrophes ("") from each line
+ # * See Arvados documentation for information how to obtain a token:
+ # https://doc.arvados.org/user/reference/api-tokens.html
+ #
+
+ arvados {
+ api {
+ keepweb-host = ""
+ keepweb-port = 443
+ host = ""
+ port = 443
+ token = ""
+ protocol = https
+ host-insecure = false
+ }
+ integration-tests {
+ project-uuid = ""
+ }
+ }
--- /dev/null
+ # example configuration for integration tests
+
+ arvados {
+ api {
+ keepweb-host = collections.ardev.mycompany.com
+ keepweb-port = 443
+ host = api.ardev.mycompany.com
+ port = 443
+ token = mytoken
+ protocol = https
+ host-insecure = false
+ }
+ integration-tests {
+ project-uuid = ardev-j7d0g-aa123f81q6y7skk
+ }
+ }
--- /dev/null
+ mock-maker-inline
--- /dev/null
+ {
+ "href": "/collections/112ci-4zz18-12tncxzptzbec1p",
+ "kind": "arvados#collection",
+ "etag": "bqoujj7oybdx0jybwvtsebj7y",
+ "uuid": "112ci-4zz18-12tncxzptzbec1p",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-21T13:38:56.521853000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-21T13:38:56.521853000Z",
+ "name": "Super Collection",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "manifest_text": ". 7df44272090cee6c0732382bba415ee9+70+Aa5ece4560e3329315165b36c239b8ab79c888f8a@5a1d5708 0:70:README.md\n",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
--- /dev/null
+ {
+ "href": "/collections/112ci-4zz18-12tncxzptzbec1p",
+ "kind": "arvados#collection",
+ "etag": "bqoujj7oybdx0jybwvtsebj7y",
+ "uuid": "112ci-4zz18-12tncxzptzbec1p",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-21T13:38:56.521853000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-21T13:38:56.521853000Z",
+ "name": "Super Collection",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "manifest_text": "",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
--- /dev/null
+ {
+ "href": "/collections/ardev-4zz18-jk5vo4uo9u5vj52",
+ "kind": "arvados#collection",
+ "etag": "2vm76dxmzr23u9774iguuxsrg",
+ "uuid": "ardev-4zz18-jk5vo4uo9u5vj52",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-02-19T11:00:00.852389000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-02-19T11:00:00.852389000Z",
+ "name": "New Collection (2018-02-19 12:00:00.273)",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "49581091dfad651945c12b08d4735d88+112",
+ "manifest_text": ". 163679d58edaadc28db769011728a72c+1070080+A3acf8c1fe582c265d2077702e4a7d74fcc03aba8@5aa4fdeb 0:1024:test-file1 1024:20480:test-file2 21504:1048576:test-file\\0403\n",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
--- /dev/null
+ {
+ "href": "/collections/112ci-4zz18-p51w7z3fpopo6sm",
+ "kind": "arvados#collection",
+ "etag": "52tk5yg024cwhkkcidu3zcmj2",
+ "uuid": "112ci-4zz18-p51w7z3fpopo6sm",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-15T10:36:03.554356000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-15T10:36:03.554356000Z",
+ "name": "Collection With Manifest #2",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "6c4106229b08fe25f48b3a7a8289dd46+143",
+ "manifest_text": ". 66c9daa69630e092e9ce554b7aae8a20+524288+A4a15ffea58f259e09f68d3f7eea29942750a79d0@5a269ff6 435f38dd384b06c248feabee0cabca52+524288+A8a99e8148bd368c49901526098901bb7d7890c3b@5a269ff6 dc5b6c104aab35fff6d70a4dadc28d37+391727+Ab0662d549c422c983fccaad02b4ade7b48a8255b@5a269ff6 0:1440303:lombok.jar\n",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
--- /dev/null
+ {
+ "kind": "arvados#collectionList",
+ "etag": "",
+ "self_link": "",
+ "offset": 0,
+ "limit": 100,
+ "items": [
+ {
+ "href": "/collections/112ci-4zz18-x6xfmvz0chnkzgv",
+ "kind": "arvados#collection",
+ "etag": "8xyiwnih5b5vzmj5sa33348a7",
+ "uuid": "112ci-4zz18-x6xfmvz0chnkzgv",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-15T13:06:36.934337000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-15T13:06:36.934337000Z",
+ "name": "Collection With Manifest #3",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "6c4106229b08fe25f48b3a7a8289dd46+143",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-p51w7z3fpopo6sm",
+ "kind": "arvados#collection",
+ "etag": "8cmhep8aixe4p42pxjoct5502",
+ "uuid": "112ci-4zz18-p51w7z3fpopo6sm",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-15T10:36:03.554356000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-15T10:36:03.554356000Z",
+ "name": "Collection With Manifest #2",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "6c4106229b08fe25f48b3a7a8289dd46+143",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-xb6gf2yraln7cwa",
+ "kind": "arvados#collection",
+ "etag": "de2ol2dyvsba3mn46al760cyg",
+ "uuid": "112ci-4zz18-xb6gf2yraln7cwa",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-15T09:32:44.146172000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-15T09:32:44.146172000Z",
+ "name": "New collection",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-r5jfktpn3a9o0ap",
+ "kind": "arvados#collection",
+ "etag": "dby68gd0vatvi090cu0axvtq3",
+ "uuid": "112ci-4zz18-r5jfktpn3a9o0ap",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-14T13:00:35.431046000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-14T13:00:35.431046000Z",
+ "name": "Collection With Manifest #1",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "3c59518bf8e1100d420488d822682b4a+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-nqxk8xjn6mtskzt",
+ "kind": "arvados#collection",
+ "etag": "2b34uzau862w862a2rv36agv6",
+ "uuid": "112ci-4zz18-nqxk8xjn6mtskzt",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-14T12:59:34.767068000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-14T12:59:34.767068000Z",
+ "name": "Empty Collection #2",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-rs9bcf5qnyfjrkm",
+ "kind": "arvados#collection",
+ "etag": "60aywazztwfspnasltufcjxpa",
+ "uuid": "112ci-4zz18-rs9bcf5qnyfjrkm",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-14T12:52:33.124452000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-14T12:52:33.124452000Z",
+ "name": "Empty Collection #1",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-af656lee4kv7q2m",
+ "kind": "arvados#collection",
+ "etag": "1jward6snif3tsjzftxh8hvwh",
+ "uuid": "112ci-4zz18-af656lee4kv7q2m",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-14T12:09:05.319319000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-14T12:09:05.319319000Z",
+ "name": "create example",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-y2zqix7k9an7nro",
+ "kind": "arvados#collection",
+ "etag": "zs2n4zliu6nb5yk3rw6h5ugw",
+ "uuid": "112ci-4zz18-y2zqix7k9an7nro",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-13T16:59:02.299257000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-13T16:59:02.299257000Z",
+ "name": "Saved at 2017-11-13 16:59:01 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-wq77jfi62u5i4rv",
+ "kind": "arvados#collection",
+ "etag": "eijhemzgy44ofmu0dtrowl604",
+ "uuid": "112ci-4zz18-wq77jfi62u5i4rv",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-13T16:58:10.637548000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-13T16:58:10.637548000Z",
+ "name": "Saved at 2017-11-13 16:58:07 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-unaeckkjgeg7ui0",
+ "kind": "arvados#collection",
+ "etag": "1oq7ye0gfbf3ih6y864w3n683",
+ "uuid": "112ci-4zz18-unaeckkjgeg7ui0",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-10T09:43:07.583862000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-10T09:43:07.583862000Z",
+ "name": "Saved at 2017-11-10 09:43:03 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-5y6atonkxq55lms",
+ "kind": "arvados#collection",
+ "etag": "4qmqlro878yx8q7ikhilo8qwn",
+ "uuid": "112ci-4zz18-5y6atonkxq55lms",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T12:46:15.245770000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T12:46:15.245770000Z",
+ "name": "Saved at 2017-11-09 12:46:13 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-b3fjqd01pxjvseo",
+ "kind": "arvados#collection",
+ "etag": "91v698hngoz241c38bbmh0ogc",
+ "uuid": "112ci-4zz18-b3fjqd01pxjvseo",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:54:07.259998000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:54:07.259998000Z",
+ "name": "Saved at 2017-11-09 11:54:04 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-cwfxl8h41q18n65",
+ "kind": "arvados#collection",
+ "etag": "215t842ckrrgjpxrxr4j0gsui",
+ "uuid": "112ci-4zz18-cwfxl8h41q18n65",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:49:38.276888000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:49:38.276888000Z",
+ "name": "Saved at 2017-11-09 11:49:35 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-uv4xu08739tn1vy",
+ "kind": "arvados#collection",
+ "etag": "90z6i3oqv197osng3wvjjir3t",
+ "uuid": "112ci-4zz18-uv4xu08739tn1vy",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:43:05.917513000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:43:05.917513000Z",
+ "name": "Saved at 2017-11-09 11:43:05 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-pzisn8c5mefzczv",
+ "kind": "arvados#collection",
+ "etag": "5lcf6wvc3wypwobswdz22wen",
+ "uuid": "112ci-4zz18-pzisn8c5mefzczv",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:40:38.804718000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:40:38.804718000Z",
+ "name": "Saved at 2017-11-09 11:40:36 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-mj24uwtnqqrno27",
+ "kind": "arvados#collection",
+ "etag": "98s08xew49avui1gy3mzit8je",
+ "uuid": "112ci-4zz18-mj24uwtnqqrno27",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:40:25.189869000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:40:25.189869000Z",
+ "name": "Saved at 2017-11-09 11:40:24 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-oco162516upgqng",
+ "kind": "arvados#collection",
+ "etag": "a09wnvl4i51xqx7u9yf4qbi94",
+ "uuid": "112ci-4zz18-oco162516upgqng",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:39:04.148785000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:39:04.148785000Z",
+ "name": "Saved at 2017-11-09 11:39:03 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-tlze7dgczsdwkep",
+ "kind": "arvados#collection",
+ "etag": "4ee2xudbc5rkr597drgu9tg10",
+ "uuid": "112ci-4zz18-tlze7dgczsdwkep",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:37:59.478975000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:37:59.478975000Z",
+ "name": "Saved at 2017-11-09 11:37:58 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-nq0kxi9d7w64la1",
+ "kind": "arvados#collection",
+ "etag": "5aa3evnbceo3brnps2e1sq8ts",
+ "uuid": "112ci-4zz18-nq0kxi9d7w64la1",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:32:23.329259000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:32:23.329259000Z",
+ "name": "Saved at 2017-11-09 11:32:22 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-fks9mewtw155pvx",
+ "kind": "arvados#collection",
+ "etag": "97vicgogv8bovmk4s2jymsdq",
+ "uuid": "112ci-4zz18-fks9mewtw155pvx",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:30:17.589462000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:30:17.589462000Z",
+ "name": "Saved at 2017-11-09 11:30:17 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-kp356e0q2wdl2df",
+ "kind": "arvados#collection",
+ "etag": "btktwjclv063s1rd6duvk51v3",
+ "uuid": "112ci-4zz18-kp356e0q2wdl2df",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:29:26.820481000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:29:26.820481000Z",
+ "name": "Saved at 2017-11-09 11:29:25 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-0ey8ob38xf7surq",
+ "kind": "arvados#collection",
+ "etag": "bob83na42pufqli1a5buxryvm",
+ "uuid": "112ci-4zz18-0ey8ob38xf7surq",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:08:53.781498000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:08:53.781498000Z",
+ "name": "Saved at 2017-11-09 11:08:52 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-wu2n0fv3cewna1n",
+ "kind": "arvados#collection",
+ "etag": "7pl1x327eeutqtsjppdj284g8",
+ "uuid": "112ci-4zz18-wu2n0fv3cewna1n",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T11:08:33.423284000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T11:08:33.423284000Z",
+ "name": "Saved at 2017-11-09 11:08:33 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-hyybo6yuvkx4hrm",
+ "kind": "arvados#collection",
+ "etag": "2wg1wn2o18ubrgbhbqwwsslhf",
+ "uuid": "112ci-4zz18-hyybo6yuvkx4hrm",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:44:53.096798000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:44:53.096798000Z",
+ "name": "Saved at 2017-11-09 10:44:51 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-h3gjq7gzd4syanw",
+ "kind": "arvados#collection",
+ "etag": "8jk0at4e69cwjyjamvm4wz2oj",
+ "uuid": "112ci-4zz18-h3gjq7gzd4syanw",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:41:31.278281000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:41:31.278281000Z",
+ "name": "Saved at 2017-11-09 10:41:30 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-jinwyyaeigjs1yg",
+ "kind": "arvados#collection",
+ "etag": "be57zhzufz2hp1tbdwidoro5j",
+ "uuid": "112ci-4zz18-jinwyyaeigjs1yg",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:41:07.083017000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:41:07.083017000Z",
+ "name": "Saved at 2017-11-09 10:41:06 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-etf8aghyxlfxvo1",
+ "kind": "arvados#collection",
+ "etag": "29lj2roie4cygo5ffgrduflly",
+ "uuid": "112ci-4zz18-etf8aghyxlfxvo1",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:40:31.710865000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:40:31.710865000Z",
+ "name": "Saved at 2017-11-09 10:40:31 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-jtbn4edpkkhbm9b",
+ "kind": "arvados#collection",
+ "etag": "6div78e1nhusii4x1xkp3rg2v",
+ "uuid": "112ci-4zz18-jtbn4edpkkhbm9b",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:39:36.999602000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:39:36.999602000Z",
+ "name": "Saved at 2017-11-09 10:39:36 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-whdleimp34hiqp6",
+ "kind": "arvados#collection",
+ "etag": "12wlbsxlmy3sze4v2m0ua7ake",
+ "uuid": "112ci-4zz18-whdleimp34hiqp6",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:19:52.879907000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:19:52.879907000Z",
+ "name": "Saved at 2017-11-09 10:19:52 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-kj8dz72zpo5kbtm",
+ "kind": "arvados#collection",
+ "etag": "9bv1bw9afb3w84gu55uzcgd6h",
+ "uuid": "112ci-4zz18-kj8dz72zpo5kbtm",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T10:16:31.558621000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T10:16:31.558621000Z",
+ "name": "Saved at 2017-11-09 10:16:30 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "5ba3fc508718fabfa20d24390fe31856+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-tr306nau9hrr437",
+ "kind": "arvados#collection",
+ "etag": "683d77tvlhe97etk9bk2bx8ds",
+ "uuid": "112ci-4zz18-tr306nau9hrr437",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:59:44.978811000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:59:44.978811000Z",
+ "name": "Saved at 2017-11-09 09:59:44 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-oxuk69569mxztp0",
+ "kind": "arvados#collection",
+ "etag": "1m34v9jbna2v7gv7auio54i8w",
+ "uuid": "112ci-4zz18-oxuk69569mxztp0",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:59:30.774888000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:59:30.774888000Z",
+ "name": "Saved at 2017-11-09 09:59:30 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-wf8sl6xbyfwjyer",
+ "kind": "arvados#collection",
+ "etag": "7l2a9fhqmxg7ghn7osx0s19v4",
+ "uuid": "112ci-4zz18-wf8sl6xbyfwjyer",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:58:21.496088000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:58:21.496088000Z",
+ "name": "Saved at 2017-11-09 09:58:20 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-drpia2es1hp9ydi",
+ "kind": "arvados#collection",
+ "etag": "33dw426fhs2vlb50b6301ukn0",
+ "uuid": "112ci-4zz18-drpia2es1hp9ydi",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:56:08.506505000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:56:08.506505000Z",
+ "name": "Saved at 2017-11-09 09:56:08 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-5b4px2i2dwyidfi",
+ "kind": "arvados#collection",
+ "etag": "2437tnhn2gmti52lpm8nfq9ct",
+ "uuid": "112ci-4zz18-5b4px2i2dwyidfi",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:54:06.651026000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:54:06.651026000Z",
+ "name": "Saved at 2017-11-09 09:54:06 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-94oslnwnxe1f9wp",
+ "kind": "arvados#collection",
+ "etag": "7e0k48zu93o57zudxjp1yrgjq",
+ "uuid": "112ci-4zz18-94oslnwnxe1f9wp",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:40:04.240297000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:40:04.240297000Z",
+ "name": "Saved at 2017-11-09 09:39:58 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-2fk0d5d4jjc1fmq",
+ "kind": "arvados#collection",
+ "etag": "cuirr803f54e89reakuq50oaq",
+ "uuid": "112ci-4zz18-2fk0d5d4jjc1fmq",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:36:14.952671000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:36:14.952671000Z",
+ "name": "Saved at 2017-11-09 09:36:08 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-xp9pu81xyc5h422",
+ "kind": "arvados#collection",
+ "etag": "3bi5xd8ezxrazk5266cwzn4s4",
+ "uuid": "112ci-4zz18-xp9pu81xyc5h422",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:35:29.552746000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:35:29.552746000Z",
+ "name": "Saved at 2017-11-09 09:35:29 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-znb4lo0if2as58c",
+ "kind": "arvados#collection",
+ "etag": "59uaoxy6uh82i6lrvr3ht8gz1",
+ "uuid": "112ci-4zz18-znb4lo0if2as58c",
+ "owner_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "created_at": "2017-11-09T09:31:08.109971000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-09T09:31:08.109971000Z",
+ "name": "Saved at 2017-11-09 09:31:06 UTC by VirtualBox",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "67cbebb9f739b6b06ca056d21115cf43+53",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-6pvl5ea5u932qzi",
+ "kind": "arvados#collection",
+ "etag": "dksrh8jznxoaidl29i1vv5904",
+ "uuid": "112ci-4zz18-6pvl5ea5u932qzi",
+ "owner_uuid": "112ci-j7d0g-tw71k7mxii6fqgx",
+ "created_at": "2017-11-08T12:48:32.238698000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-f4633qdjs6w8zcy",
+ "modified_by_user_uuid": "112ci-tpzed-nd84czdo4iea1mz",
+ "modified_at": "2017-11-08T12:50:23.946608000Z",
+ "name": "New collection",
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "18c037c51c3f74be53ea2b115afd0c5f+69",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/collections/112ci-4zz18-wq5pyrxfv1t9isu",
+ "kind": "arvados#collection",
+ "etag": "1w1rhhd6oql4ceb7h9t16sf0q",
+ "uuid": "112ci-4zz18-wq5pyrxfv1t9isu",
+ "owner_uuid": "112ci-j7d0g-anonymouspublic",
+ "created_at": "2017-11-03T10:03:20.364737000Z",
+ "modified_by_client_uuid": null,
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:03:20.364737000Z",
+ "name": null,
+ "description": null,
+ "properties": {},
+ "portable_data_hash": "d41d8cd98f00b204e9800998ecf8427e+0",
+ "replication_desired": null,
+ "replication_confirmed": null,
+ "replication_confirmed_at": null,
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
+ ],
+ "items_available": 41
+ }
--- /dev/null
+ {
+ "href": "/groups/ardev-j7d0g-bmg3pfqtx3ivczp",
+ "kind": "arvados#group",
+ "etag": "3hw0vk4mbl0ofvia5k6x4dwrx",
+ "uuid": "ardev-j7d0g-bmg3pfqtx3ivczp",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:09:05.984597000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T11:09:05.984597000Z",
+ "name": "TestGroup1",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
--- /dev/null
+ {
+ "kind": "arvados#groupList",
+ "etag": "",
+ "self_link": "",
+ "offset": 0,
+ "limit": 100,
+ "items": [
+ {
+ "href": "/groups/ardev-j7d0g-ylx7wnu1moge2di",
+ "kind": "arvados#group",
+ "etag": "68vubv3iw7663763bozxebmyf",
+ "uuid": "ardev-j7d0g-ylx7wnu1moge2di",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-18T09:09:21.126649000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-18T09:09:21.126649000Z",
+ "name": "TestProject1",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-mnzhga726itrbrq",
+ "kind": "arvados#group",
+ "etag": "68q7r8r37u9hckr2zsynvton3",
+ "uuid": "ardev-j7d0g-mnzhga726itrbrq",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T12:11:24.389594000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T12:11:24.389594000Z",
+ "name": "TestProject2",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-0w9m1sz46ljtdnm",
+ "kind": "arvados#group",
+ "etag": "ef4vzx5gyudkrg9zml0zdv6qu",
+ "uuid": "ardev-j7d0g-0w9m1sz46ljtdnm",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T12:08:39.066802000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T12:08:39.066802000Z",
+ "name": "TestProject3",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-r20iem5ou6h5wao",
+ "kind": "arvados#group",
+ "etag": "6h6h4ta6yyf9058delxk8fnqs",
+ "uuid": "ardev-j7d0g-r20iem5ou6h5wao",
+ "owner_uuid": "ardev-j7d0g-j7drd8yikkp6evd",
+ "created_at": "2018-04-17T12:03:39.647244000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T12:03:39.647244000Z",
+ "name": "TestProject4",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-j7d0g-j7drd8yikkp6evd",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-j7drd8yikkp6evd",
+ "kind": "arvados#group",
+ "etag": "6se2y8f9o7uu06pbopgq56xds",
+ "uuid": "ardev-j7d0g-j7drd8yikkp6evd",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T11:58:31.339515000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T11:58:31.339515000Z",
+ "name": "TestProject5",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-kh1g7i5va870xt0",
+ "kind": "arvados#group",
+ "etag": "2si26vaig3vig9266pqkqh2gy",
+ "uuid": "ardev-j7d0g-kh1g7i5va870xt0",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:56:54.391676000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:56:54.391676000Z",
+ "name": "TestProject6",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-sclkdyuwm4h2m78",
+ "kind": "arvados#group",
+ "etag": "edgnz6q0vt2u3o13ujtfohb75",
+ "uuid": "ardev-j7d0g-sclkdyuwm4h2m78",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:27:15.914517000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:27:15.914517000Z",
+ "name": "TestProject7",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-593khc577zuyyhe",
+ "kind": "arvados#group",
+ "etag": "39ig9ttgec6lbe096uetn2cb9",
+ "uuid": "ardev-j7d0g-593khc577zuyyhe",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:27:03.858203000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:27:03.858203000Z",
+ "name": "TestProject8",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-iotds0tm559dbz7",
+ "kind": "arvados#group",
+ "etag": "1dpr8v6tx6pta0fozq93eyeou",
+ "uuid": "ardev-j7d0g-iotds0tm559dbz7",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:26:25.180623000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:26:25.180623000Z",
+ "name": "TestProject9",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-gbqay74778tonb8",
+ "kind": "arvados#group",
+ "etag": "dizbavs2opfe1wpx6thocfki0",
+ "uuid": "ardev-j7d0g-gbqay74778tonb8",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:26:06.435961000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:26:06.435961000Z",
+ "name": "TestProject10",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-fmq1t0jlznehbdm",
+ "kind": "arvados#group",
+ "etag": "6xue8m3lx9qpptfvdf13val5t",
+ "uuid": "ardev-j7d0g-fmq1t0jlznehbdm",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-17T10:25:55.546399000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-17T10:25:55.546399000Z",
+ "name": "TestProject11",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-vxju56ch64u51gq",
+ "kind": "arvados#group",
+ "etag": "2gqix9e4m023usi9exhrsjx6z",
+ "uuid": "ardev-j7d0g-vxju56ch64u51gq",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-16T14:09:49.700566000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-16T14:09:49.700566000Z",
+ "name": "TestProject12",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-g8m4w0d22gv6fbj",
+ "kind": "arvados#group",
+ "etag": "73n8x82814o6ihld0kltf468d",
+ "uuid": "ardev-j7d0g-g8m4w0d22gv6fbj",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-11T15:02:35.016850000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-11T15:02:35.016850000Z",
+ "name": "TestProject13",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-lstqed4y78khaqm",
+ "kind": "arvados#group",
+ "etag": "91f7uwq7pj3d3ez1u4smjg3ch",
+ "uuid": "ardev-j7d0g-lstqed4y78khaqm",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-06T15:29:27.754408000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-06T15:29:27.754408000Z",
+ "name": "TestProject14",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-0jbezvnq8i07l7p",
+ "kind": "arvados#group",
+ "etag": "7dbxhvbcfaogwnvo8k4mtqthk",
+ "uuid": "ardev-j7d0g-0jbezvnq8i07l7p",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-04-05T09:32:46.946417000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-04-05T09:32:46.946417000Z",
+ "name": "TestProject15",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-72dxer22g6iltqz",
+ "kind": "arvados#group",
+ "etag": "dhfu203rckzdzvx832wm7jv59",
+ "uuid": "ardev-j7d0g-72dxer22g6iltqz",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:27:02.482218000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T13:17:00.045606000Z",
+ "name": "TestProject16",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-nebzwquxtq1v3o5",
+ "kind": "arvados#group",
+ "etag": "7l9oxbdf4e1m9ddnujokf7czz",
+ "uuid": "ardev-j7d0g-nebzwquxtq1v3o5",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:11:26.235411000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T11:11:26.235411000Z",
+ "name": "TestProject17",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-5589c8dmxevecqh",
+ "kind": "arvados#group",
+ "etag": "83862x2o4453mja2rvypjl5gv",
+ "uuid": "ardev-j7d0g-5589c8dmxevecqh",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:10:58.496482000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T11:10:58.496482000Z",
+ "name": "TestProject18",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-bmg3pfqtx3ivczp",
+ "kind": "arvados#group",
+ "etag": "3hw0vk4mbl0ofvia5k6x4dwrx",
+ "uuid": "ardev-j7d0g-bmg3pfqtx3ivczp",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:09:05.984597000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T11:09:05.984597000Z",
+ "name": "TestProject19",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ },
+ {
+ "href": "/groups/ardev-j7d0g-mfitz2oa4rpycou",
+ "kind": "arvados#group",
+ "etag": "6p9xbxpttj782mpqs537gfvc6",
+ "uuid": "ardev-j7d0g-mfitz2oa4rpycou",
+ "owner_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "created_at": "2018-03-29T11:00:19.809612000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-n3kzq4fvoks3uw4",
+ "modified_at": "2018-03-29T11:00:19.809612000Z",
+ "name": "TestProject20",
+ "group_class": "project",
+ "description": null,
+ "writable_by": [
+ "ardev-tpzed-n3kzq4fvoks3uw4",
+ "ardev-tpzed-n3kzq4fvoks3uw4"
+ ],
+ "delete_at": null,
+ "trash_at": null,
+ "is_trashed": false
+ }
+ ],
+ "items_available": 20
+ }
--- /dev/null
+ Sample text file to test keep client.
--- /dev/null
+ {
+ "kind": "arvados#keepServiceList",
+ "etag": "",
+ "self_link": "",
+ "offset": null,
+ "limit": null,
+ "items": [
+ {
+ "href": "/keep_services/112ci-bi6l4-hv02fg8sbti8ykk",
+ "kind": "arvados#keepService",
+ "etag": "bjzh7og2d9z949lbd38vnnslt",
+ "uuid": "112ci-bi6l4-hv02fg8sbti8ykk",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.314229000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.314229000Z",
+ "service_host": "localhost",
+ "service_port": 9000,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ },
+ {
+ "href": "/keep_services/112ci-bi6l4-f0r03wrqymotwql",
+ "kind": "arvados#keepService",
+ "etag": "7m64l69kko4bytpsykf8cay7t",
+ "uuid": "112ci-bi6l4-f0r03wrqymotwql",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.351577000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.351577000Z",
+ "service_host": "localhost",
+ "service_port": 9001,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ }
+ ],
+ "items_available": 2
+ }
--- /dev/null
+ {
+ "kind": "arvados#keepServiceList",
+ "etag": "",
+ "self_link": "",
+ "offset": null,
+ "limit": null,
+ "items": [
+ {
+ "href": "/keep_services/112ci-bi6l4-hv02fg8sbti8ykk",
+ "kind": "arvados#keepService",
+ "etag": "bjzh7og2d9z949lbd38vnnslt",
+ "uuid": "112ci-bi6l4-hv02fg8sbti8ykk",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.314229000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.314229000Z",
+ "service_host": "localhost",
+ "service_port": 9000,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ },
+ {
+ "href": "/keep_services/112ci-bi6l4-f0r03wrqymotwql",
+ "kind": "arvados#keepService",
+ "etag": "7m64l69kko4bytpsykf8cay7t",
+ "uuid": "112ci-bi6l4-f0r03wrqymotwql",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.351577000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.351577000Z",
+ "service_host": "localhost",
+ "service_port": 9000,
+ "service_ssl_flag": false,
+ "service_type": "gpfs",
+ "read_only": false
+ }
+ ],
+ "items_available": 2
+ }
--- /dev/null
+ {
+ "href": "/keep_services/112ci-bi6l4-hv02fg8sbti8ykk",
+ "kind": "arvados#keepService",
+ "etag": "bjzh7og2d9z949lbd38vnnslt",
+ "uuid": "112ci-bi6l4-hv02fg8sbti8ykk",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.314229000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.314229000Z",
+ "service_host": "10.0.2.15",
+ "service_port": 9000,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ }
--- /dev/null
+ {
+ "kind": "arvados#keepServiceList",
+ "etag": "",
+ "self_link": "",
+ "offset": 0,
+ "limit": 100,
+ "items": [
+ {
+ "href": "/keep_services/112ci-bi6l4-f0r03wrqymotwql",
+ "kind": "arvados#keepService",
+ "etag": "7m64l69kko4bytpsykf8cay7t",
+ "uuid": "112ci-bi6l4-f0r03wrqymotwql",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.351577000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.351577000Z",
+ "service_host": "10.0.2.15",
+ "service_port": 9000,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ },
+ {
+ "href": "/keep_services/112ci-bi6l4-hv02fg8sbti8ykk",
+ "kind": "arvados#keepService",
+ "etag": "bjzh7og2d9z949lbd38vnnslt",
+ "uuid": "112ci-bi6l4-hv02fg8sbti8ykk",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:48.314229000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:48.314229000Z",
+ "service_host": "10.0.2.15",
+ "service_port": 9001,
+ "service_ssl_flag": false,
+ "service_type": "disk",
+ "read_only": false
+ },
+ {
+ "href": "/keep_services/112ci-bi6l4-ko27cfbsf2ssx2m",
+ "kind": "arvados#keepService",
+ "etag": "4be61qkpt6nzdfff4vj9nkpmj",
+ "uuid": "112ci-bi6l4-ko27cfbsf2ssx2m",
+ "owner_uuid": "112ci-tpzed-000000000000000",
+ "created_at": "2017-11-03T10:04:36.355045000Z",
+ "modified_by_client_uuid": "112ci-ozdt8-xxy0ipzwti8gnmt",
+ "modified_by_user_uuid": "112ci-tpzed-000000000000000",
+ "modified_at": "2017-11-03T10:04:36.355045000Z",
+ "service_host": "10.0.2.15",
+ "service_port": 9002,
+ "service_ssl_flag": false,
+ "service_type": "proxy",
+ "read_only": false
+ }
+ ],
+ "items_available": 3
+ }
--- /dev/null
+ {
+ "kind": "arvados#keepServiceList",
+ "etag": "",
+ "self_link": "",
+ "offset": null,
+ "limit": null,
+ "items": [],
+ "items_available": 0
+ }
--- /dev/null
+ {
+ "href": "/users/ardev-tpzed-q6dvn7sby55up1b",
+ "kind": "arvados#user",
+ "etag": "b21emst9eu9u1wdpqcz6la583",
+ "uuid": "ardev-tpzed-q6dvn7sby55up1b",
+ "owner_uuid": "ardev-tpzed-000000000000000",
+ "created_at": "2017-10-30T19:42:43.324740000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-o3km4ug9jhs189j",
+ "modified_at": "2017-10-31T09:01:03.985749000Z",
+ "email": "example@email.com",
+ "username": "johnwayne",
+ "full_name": "John Wayne",
+ "first_name": "John",
+ "last_name": "Wayne",
+ "identity_url": "ardev-tpzed-r09t5ztf5qd3rlj",
+ "is_active": true,
+ "is_admin": null,
+ "is_invited": true,
+ "prefs": {},
+ "writable_by": [
+ "ardev-tpzed-000000000000000",
+ "ardev-tpzed-q6dvn7sby55up1b",
+ "ardev-j7d0g-000000000000000"
+ ]
+ }
--- /dev/null
+ {
+ "href": "/users/ardev-tpzed-q6dvn7sby55up1b",
+ "kind": "arvados#user",
+ "etag": "b21emst9eu9u1wdpqcz6la583",
+ "uuid": "ardev-tpzed-q6dvn7sby55up1b",
+ "owner_uuid": "ardev-tpzed-000000000000000",
+ "created_at": "2017-10-30T19:42:43.324740000Z",
+ "modified_by_client_uuid": "ardev-ozdt8-97tzh5x96spqkay",
+ "modified_by_user_uuid": "ardev-tpzed-o3km4ug9jhs189j",
+ "modified_at": "2017-10-31T09:01:03.985749000Z",
+ "email": "example@email.com",
+ "username": "johnwayne",
+ "full_name": "John Wayne",
+ "first_name": "John",
+ "last_name": "Wayne",
+ "identity_url": "ardev-tpzed-r09t5ztf5qd3rlj",
+ "is_active": true,
+ "is_admin": null,
+ "is_invited": true,
+ "prefs": {},
+ "writable_by": [
+ "ardev-tpzed-000000000000000",
+ "ardev-tpzed-q6dvn7sby55up1b",
+ "ardev-j7d0g-000000000000000"
+ ]
+ }
--- /dev/null
+ {
+ "kind": "arvados#userList",
+ "etag": "",
+ "self_link": "",
+ "offset": 0,
+ "limit": 100,
+ "items": [
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-12389ux30402est",
+ "email": "test.user@email.com",
+ "first_name": "Test",
+ "last_name": "User",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123vn7sby55up1b",
+ "email": "test.user1@email.com",
+ "first_name": "Test1",
+ "last_name": "User1",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123g70lq1m3c6fz",
+ "email": "test.user2@email.com",
+ "first_name": "Test2",
+ "last_name": "User2",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-1233zsoudkgq92e",
+ "email": "test.user3@email.com",
+ "first_name": "Test3",
+ "last_name": "User3",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-1234xjvs0clppd3",
+ "email": "test.user4@email.com",
+ "first_name": "Test4",
+ "last_name": "User4",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123bpggscmn6z8m",
+ "email": "test.user5@email.com",
+ "first_name": "Test5",
+ "last_name": "User5",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-1231uysivaz6ipi",
+ "email": "test.user6@email.com",
+ "first_name": "Test6",
+ "last_name": "User6",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123b0a1wu0q6cm4",
+ "email": "test.user7@email.com",
+ "first_name": "Test7",
+ "last_name": "User7",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123bz6n6si24t6v",
+ "email": "test.user8@email.com",
+ "first_name": "Test8",
+ "last_name": "User8",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123lxhzifligheu",
+ "email": "test.user9@email.com",
+ "first_name": "Test9",
+ "last_name": "User9",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123gaz31qbopewh",
+ "email": "test.user10@email.com",
+ "first_name": "Test10",
+ "last_name": "User10",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-123dmcf65z973uo",
+ "email": "test.user11@email.com",
+ "first_name": "Test11",
+ "last_name": "User11",
+ "is_active": true
+ },
+ {
+ "kind": "arvados#user",
+ "uuid": "ardev-tpzed-1239y3lj7ybpyg8",
+ "email": "test.user12@email.com",
+ "first_name": "Test12",
+ "last_name": "User12",
+ "is_active": true
+ }
+
+ ],
+ "items_available": 13
+ }
--- /dev/null
+ {
+ "href": "/users/ardev-tpzed-000000000000000",
+ "kind": "arvados#user",
+ "etag": "2ehmra38iwfuexvz1cjno5xua",
+ "uuid": "ardev-tpzed-000000000000000",
+ "owner_uuid": "ardev-tpzed-000000000000000",
+ "created_at": "2016-10-19T07:48:04.838534000Z",
+ "modified_by_client_uuid": null,
+ "modified_by_user_uuid": "ardev-tpzed-000000000000000",
+ "modified_at": "2016-10-19T07:48:04.833164000Z",
+ "email": "root",
+ "username": null,
+ "full_name": "root",
+ "first_name": "root",
+ "last_name": "",
+ "identity_url": null,
+ "is_active": true,
+ "is_admin": true,
+ "is_invited": true,
+ "prefs": {},
+ "writable_by": [
+ "ardev-tpzed-000000000000000"
+ ]
+ }
--- /dev/null
+ #!/bin/sh
+ #
+ # Copyright (C) The Arvados Authors. All rights reserved.
+ #
+ # SPDX-License-Identifier: AGPL-3.0 OR Apache-2.0
+ #
+
+ set -e
+ UID=$(id -u)
+ exec docker run --rm --user $UID -v $PWD:$PWD -w $PWD java:8 /bin/sh -c '(./gradlew clean && ./gradlew test); ./gradlew --stop'
projects / arvados.git / commitdiff