Brett Smith [Mon, 26 May 2014 15:36:11 +0000 (11:36 -0400)]
2863: Remove debugging print from Python KeepClient.
Other tools like crunch-job parse the output of tools like arv-get and
arv-put. Right now crunch-job is saving this debug output in the log
attribute of running jobs, which confuses several other tools like
Workbench.
Tim Pierce [Sat, 24 May 2014 04:44:06 +0000 (00:44 -0400)]
2755: incorporate code review.
* Unit tests cover all permutations of signature/authorization when
--enforce-permissions=false
* Keep is more forgiving about the structure of locators, permits
locator hints of unknown type (as long as they begin with an uppercase
letter)
* Keep delivers 400 Bad Request for requests that do not match any
route, or are lexically invalid. 404 Not Found only for requests with
a syntactically valid hash not found on disk.
Tim Pierce [Wed, 21 May 2014 14:58:18 +0000 (10:58 -0400)]
2755: add support for signed locators in the Python SDK.
* arvados.Keep.put() saves the response body (which may contain a
signed locator) and returns it to the caller.
* arvados.Keep.get() passes the full signed locator to the remote Keep
server. The bare MD5 hash is still used for caching and for
shuffled_service_roots
* run_test_server.run_keep() takes arguments 'blob_signing_key' and
'enforce_permissions', for testing permission signatures in unit
tests.
* test_keep_client: new unit tests for permissions:
- with --enforce-permissions=true:
- GET with a signed locator works
- GET with an unsigned locator fails
- unauthenticated GET fails
- with --enforce-permissions=false:
- GET with a signed locator works
- GET with an unsigned locator works
- unauthenticated GET works
Bug fixes to permission handling in the Keep server:
* Locator hints may appear in any order; be flexible. Parse them in
GetBlockHandler rather than in the REST router.
* Returned locators are terminated with newline (consistent with
Warehouse, and more friendly for human debugging).
* The locator returned from a PUT request always has a size hint.
* The correct Authorization header keyword is "OAuth2", not
"OAuth". D'oh.
* Updated unit tests to accommodate newlines, size hints and OAuth2.
Peter Amstutz [Wed, 21 May 2014 20:00:05 +0000 (16:00 -0400)]
2798: Adds client side support for Keep proxy X-Keep-Desired-Replicas and
X-Keep-Replicas-Stored headers, and ARVADOS_KEEP_PROXY environment variable.
Added tests.
Tim Pierce [Wed, 21 May 2014 17:34:22 +0000 (13:34 -0400)]
2755: incorporate code review
* lib/locator.rb renamed => app/models/locator.rb
* Relaxed Locator.parse! handling of hint content.
* Locator.parse() rescues only from ArgumentError.
* Removed blob_signing_ttl from application.yml.example.
* Collections.show only matches locators that are preceded by a space,
when parsing manifest_text.
Peter Amstutz [Tue, 20 May 2014 19:00:42 +0000 (15:00 -0400)]
2764: Fixed scopes_allow_request to use request.request_method (the effective
HTTP method) instead of request.method (the actual HTTP method) because
workbench uses POST even for GET.
Brett Smith [Thu, 15 May 2014 20:38:12 +0000 (16:38 -0400)]
2764: Add wget-friendly Collections file page.
This new route will become the way you share authless Collection links
with others. They can pass it to `wget -r` to download the whole
collection, nicely organized, with nothing extraneous. Since it
doesn't try to load user information or look up related Arvados items,
it can be rendered using an API token with a very narrow scope.
Because wget respects robots.txt, this branch stops using that in
favor of the corresponding <meta> tag. The new view only limits
indexing, so wget can follow the links on the page.
Peter Amstutz [Wed, 21 May 2014 18:29:40 +0000 (14:29 -0400)]
2776: Fixed misspelled accessable->accessible. Thanks Brett! Tweaked
workbench icons for keep services and compute nodes. Removed spurious print
statements. Tighted up definitions of service_host, service_port and
service_ssl_flag in keep_disk model.
Brett Smith [Wed, 21 May 2014 14:07:28 +0000 (10:07 -0400)]
2753: Replace Workbench file tree backgrounds with hover effect.
Based on feedback from Tom in refs #2753. I'm still concerned about
users being able to line up buttons with filenames, so the highlight
on :hover is intended to help with that.
Peter Amstutz [Wed, 21 May 2014 15:00:21 +0000 (11:00 -0400)]
2798: Renamed internal messaging structs in an attempt to use the word "reader"
slightly less. Refactored tests to reduce redundancy slightly. Added test with large number of concurrent readers. Rewrote "how to use" package comments and wrote a small novel about the "theory of operation".
Peter Amstutz [Wed, 21 May 2014 02:23:18 +0000 (22:23 -0400)]
2798: Tracks opening and closing of readers, will block closing AsyncStream
until all readers are closed. Additionally, will block if too many readers are
created.
I do not see any code that would directly render these partials;
they're not indirectly referenced in the pane list; and I can remove
the controller code and run all our tests without tripping on a
NoMethodError exception from a nil.
Tim Pierce [Tue, 13 May 2014 15:06:00 +0000 (11:06 -0400)]
2755: Verify permission signatures on create.
Phase 1 of #2755: when creating a new collection, verify any permission
signatures found in the manifest. Unsigned locators in the manifest are
implicitly permitted (to be disabled in Phase 4)
* New "Locator" class to parse, examine and manipulate Keep locators.
* Collections.create checks permission signatures in a manifest.
* Collections.show signs locators in a manifest.
* collections_controller_test.rb: new unit tests to exercise signed
manifests and related features:
- "create collection with signed manifest"
- "create collection with signed manifest and explicit TTL"
- "create fails with invalid signature"
- "create fails with uuid of signed manifest"
- "multiple locators per line"
- "multiple signed locators per line"
* application.yml.example: new configuration variables
- Rails.configuration.blob_signing_key
- Rails.configuration.blob_signing_ttl
projects / arvados.git / log