Brett Smith [Thu, 22 May 2014 21:10:40 +0000 (17:10 -0400)]
2752: Add basic test for arv-put.
Brett Smith [Thu, 22 May 2014 20:46:15 +0000 (16:46 -0400)]
2752: Extract common Python test methods.
I'm going to reuse these to test arv-put.
Brett Smith [Thu, 22 May 2014 20:11:10 +0000 (16:11 -0400)]
2752: Move arv-put functionality to arvados.commands.put.
This will make it easier to unit test arv-put, and make it easier for
developers to extend or customize its functionality.
Brett Smith [Thu, 22 May 2014 19:49:31 +0000 (15:49 -0400)]
2752: Move Python tests to dedicated subdirectory.
This better follows Python convention, and will let us build out our
test frameworks without cluttering the root source directories.
This change brings a small change to the way we run tests. Pass the
directory name to the discover command:
$ python -m unittest discover tests
Brett Smith [Fri, 30 May 2014 14:40:07 +0000 (10:40 -0400)]
2752: Add arvados.collections.ResumableCollectionWriter.
This is a subclass of CollectionWriter that only accepts data from the
filesystem. In exchange, it can record its own state, and resume
writing from one of those states. arv-put will use this to make the
user experience nicer if a long upload is interrupted.
Brett Smith [Tue, 20 May 2014 16:14:07 +0000 (12:14 -0400)]
2752: Implement CollectionWriter with a work queue.
This will make it easier to capture and restore state.
Brett Smith [Tue, 20 May 2014 15:29:44 +0000 (11:29 -0400)]
2752: Reuse CollectionWriter.write_file() where possible.
Brett Smith [Tue, 20 May 2014 15:15:00 +0000 (11:15 -0400)]
2752: Add method CollectionWriter.write_file().
Brett Smith [Tue, 20 May 2014 15:12:56 +0000 (11:12 -0400)]
2752: Remove trailing whitespace in arv-put.
Brett Smith [Tue, 20 May 2014 14:25:18 +0000 (10:25 -0400)]
2752: Add tests for CollectionWriter.write_directory_tree().
Brett Smith [Tue, 20 May 2014 13:47:27 +0000 (09:47 -0400)]
2752: Clean up after Python SDK Collections tests.
Brett Smith [Tue, 20 May 2014 13:40:53 +0000 (09:40 -0400)]
2752: Refactor Python SDK test_collections.
This eliminates duplication in setup and teardown code, and clarifies
dependencies between tests a bit.
Brett Smith [Mon, 19 May 2014 18:10:41 +0000 (14:10 -0400)]
2752: Clean up arv-put imports.
Tim Pierce [Fri, 30 May 2014 13:49:26 +0000 (09:49 -0400)]
2755: comment doc fix.
radhika [Fri, 30 May 2014 01:59:14 +0000 (21:59 -0400)]
2929: provenance graph is cutoff on the left. Removed the absolute positioning on the div to resolve the issue.
We now have functioning provenance graph display for pipeline instances, jobs, and collections.
Tim Pierce [Thu, 29 May 2014 21:08:22 +0000 (17:08 -0400)]
2755: code review.
* arv-put returns an unsigned manifest UUID.
* The ArvPutSignedManifest test first confirms that the collection is
not present in the API server. Also asserts that the arv-put command
completed successfully.
* Additional tests for KeepPermissionTestCase for each potential
combination of wrong authorization and signature.
Tom Clegg [Thu, 29 May 2014 20:48:30 +0000 (16:48 -0400)]
2762: Do not try to auto-destroy referring links when destroying a model with no uuid. refs #2762
radhika [Thu, 29 May 2014 20:34:53 +0000 (16:34 -0400)]
Merge branch 'master' into 2903-remove-pi-active-and-success
Ward Vandewege [Thu, 29 May 2014 18:17:23 +0000 (14:17 -0400)]
Switch to using auth helpers instead of messing with Thread.current in the user tests.
Tom Clegg [Thu, 29 May 2014 17:58:02 +0000 (13:58 -0400)]
2754: Change "create pipeline" label to "run", to match others in same workflow.
Peter Amstutz [Thu, 29 May 2014 17:41:23 +0000 (13:41 -0400)]
Another fix to sharing scope to use keep_services/accessible. refs #2766
Ward Vandewege [Thu, 29 May 2014 17:32:59 +0000 (13:32 -0400)]
Provide link to users page in notification e-mails.
Peter Amstutz [Thu, 29 May 2014 17:05:43 +0000 (13:05 -0400)]
Added proper sharing scope to permit actually downloading shared files. refs #2766
Tim Pierce [Thu, 29 May 2014 16:56:01 +0000 (12:56 -0400)]
2755: code review.
* CollectionWriter does not cache manifest_text, and recomputes it each
time it is asked.
* Manifest uuid is computed from a manifest that has been stripped of
permission hints, but not any other kind of locator hint.
Ward Vandewege [Thu, 29 May 2014 15:16:21 +0000 (11:16 -0400)]
Change default for
new_user_notification_recipients
new_inactive_user_notification_recipients
in config/application.default.yml to an empty array, so that it is
obvious how to specify multiple recipients.
Brett Smith [Thu, 29 May 2014 14:52:04 +0000 (10:52 -0400)]
2753: Fix Collection inline file view.
The height limit needs to be applied to the images themselves, rather
than the containing div.
Peter Amstutz [Thu, 29 May 2014 14:18:53 +0000 (10:18 -0400)]
Added rescue AccessForbiddenException to collections controller and view to
disable the "sharing link" feature when the client is untrusted. refs #2766
radhika [Thu, 29 May 2014 14:08:02 +0000 (10:08 -0400)]
2903: during validation methods, set state instead of success
radhika [Thu, 29 May 2014 13:37:02 +0000 (09:37 -0400)]
Merge branch 'master' into 2903-remove-pi-active-and-success
Peter Amstutz [Thu, 29 May 2014 13:32:08 +0000 (09:32 -0400)]
Merge branch '2044-share-button' refs #2766
Peter Amstutz [Thu, 29 May 2014 13:22:43 +0000 (09:22 -0400)]
Go Keep client correctly closes response body on client requests, should fix
lingering connections problem. Also added graceful shutdown on SIGINT to
keepproxy. refs #1885
Ward Vandewege [Thu, 29 May 2014 00:29:12 +0000 (20:29 -0400)]
Merge branch 'master' into 2681-new-inactive-user-notification
radhika [Wed, 28 May 2014 23:12:11 +0000 (19:12 -0400)]
2903: active to state usage
Tim Pierce [Wed, 28 May 2014 21:58:09 +0000 (17:58 -0400)]
2755: add support to arv-put for signed manifests.
When arv-put finishes a stream, the manifest it stores in Keep now has
been stripped of signatures and other variable hints.
test_cmdline.py tests arv-put's handling of the manifest to make sure
that, when permissions are enabled, the manifest in Keep lacks
signatures, and the same manifest returned from the API server includes
signatures.
Refs #2755.
radhika [Wed, 28 May 2014 21:30:21 +0000 (17:30 -0400)]
2903: Remove active and success flags from pipeline_instance. Update all other files as necessary.
radhika [Wed, 28 May 2014 16:58:11 +0000 (12:58 -0400)]
Merge branch 'master' of git.curoverse.com:arvados
radhika [Wed, 28 May 2014 16:56:36 +0000 (12:56 -0400)]
An equivalent of this script is now at services/api/script. This was done as part of Feature #2272.
Tom Clegg [Wed, 28 May 2014 16:44:04 +0000 (12:44 -0400)]
2754: Merge branch '2754-pipeline-template-description' refs #2754
Tom Clegg [Wed, 28 May 2014 16:43:20 +0000 (12:43 -0400)]
2893: Add more tests for bogus serialized hash values.
Tom Clegg [Wed, 28 May 2014 16:34:59 +0000 (12:34 -0400)]
2893: Add test for un-symbolizing existing db content.
Tom Clegg [Wed, 28 May 2014 16:22:18 +0000 (12:22 -0400)]
2893: Prevent symbol keys/values in serialized attributes.
Tom Clegg [Wed, 28 May 2014 16:00:33 +0000 (12:00 -0400)]
2754: Add migration missing from commit:
fe84c84827af6092784408629e4cde0663fe6af8
Peter Amstutz [Wed, 28 May 2014 15:19:37 +0000 (11:19 -0400)]
2044: Fixed CollectionsController#search_scopes to use 'filter' correctly
instead of using 'where' incorrectly. Reverted most changes to
ArvadosModel#ensure_owner_uuid_is_permitted. Added rescue to
collections/show.html.erb when log.event_at is nil.
Ward Vandewege [Wed, 28 May 2014 15:00:07 +0000 (11:00 -0400)]
Do not use symbol keys in the properties of permission links.
Peter Amstutz [Wed, 28 May 2014 14:53:23 +0000 (10:53 -0400)]
Merge branch 'master' into 2044-share-button
Tom Clegg [Wed, 28 May 2014 14:04:55 +0000 (10:04 -0400)]
2892: Fix user agreement routing. closes #2892
Tom Clegg [Wed, 28 May 2014 00:01:22 +0000 (20:01 -0400)]
2862: Use ?format=js so browser does not think js and html views are
interchangeable. closes #2862
Tom Clegg [Tue, 27 May 2014 22:16:39 +0000 (18:16 -0400)]
2760: Merge branch '2760-not-all-objects-in-folders' refs #2760
Peter Amstutz [Tue, 27 May 2014 21:07:40 +0000 (17:07 -0400)]
Merge branch 'master' of git.curoverse.com:arvados
Peter Amstutz [Tue, 27 May 2014 21:07:11 +0000 (17:07 -0400)]
Adjusted timeout on websocket_test from 3 seconds to 4 in an attempt to reduce
the number of spurious timeouts on jenkins.
radhika [Tue, 27 May 2014 20:50:26 +0000 (16:50 -0400)]
Merge branch 'master' of git.curoverse.com:arvados
Peter Amstutz [Tue, 27 May 2014 20:50:09 +0000 (16:50 -0400)]
Merge branch 'master' of git.curoverse.com:arvados #2751
radhika [Tue, 27 May 2014 20:20:57 +0000 (16:20 -0400)]
Merge branch '2756-eventbus-in-workbench' into 2678-owned_by
Conflicts:
services/api/script/crunch-dispatch.rb
radhika [Tue, 27 May 2014 19:54:36 +0000 (15:54 -0400)]
2756: fix error where partial lines were potentially logged
Peter Amstutz [Tue, 27 May 2014 19:50:04 +0000 (15:50 -0400)]
Merge branch 'master' of git.curoverse.com:arvados refs #2751
Peter Amstutz [Tue, 27 May 2014 19:49:37 +0000 (15:49 -0400)]
Merge branch '2751-python-sdk-keep-proxy-support' closes #2751
radhika [Tue, 27 May 2014 19:16:05 +0000 (15:16 -0400)]
Merge branch 'master' into 2756-eventbus-in-workbench
radhika [Tue, 27 May 2014 19:15:26 +0000 (15:15 -0400)]
2756: log span title
radhika [Tue, 27 May 2014 19:11:07 +0000 (15:11 -0400)]
2756: fix typo in helper method signature
radhika [Tue, 27 May 2014 18:59:55 +0000 (14:59 -0400)]
2756: update the ui to show job progress, and links in one column
radhika [Tue, 27 May 2014 17:19:53 +0000 (13:19 -0400)]
2525: add some javadoc comments
radhika [Tue, 27 May 2014 17:02:47 +0000 (13:02 -0400)]
2678: upon Tom's feedback, setting owner_uuid of the pi whether or not the pi belongs to a group.
Tom Clegg [Tue, 27 May 2014 15:43:07 +0000 (11:43 -0400)]
2754: Add pipeline template description attribute.
radhika [Tue, 27 May 2014 15:30:44 +0000 (11:30 -0400)]
Merge branch 'master' into 2525-java-sdk
radhika [Tue, 27 May 2014 15:22:00 +0000 (11:22 -0400)]
Merge branch 'master' into 2678-owned_by
radhika [Tue, 27 May 2014 15:16:22 +0000 (11:16 -0400)]
Merge branch 'master' into 2756-eventbus-in-workbench
radhika [Tue, 27 May 2014 15:15:03 +0000 (11:15 -0400)]
Merge branch 'master' into 2525-java-sdk
radhika [Tue, 27 May 2014 15:12:25 +0000 (11:12 -0400)]
Merge branch 'master' into 2767-doc-updates
Tom Clegg [Tue, 27 May 2014 15:08:25 +0000 (11:08 -0400)]
2760: Exclude system objects like VMs and repos from folder behavior.
Tim Pierce [Tue, 27 May 2014 15:06:42 +0000 (11:06 -0400)]
Merge branch '2755-python-sdk-permissions'
Refs #2755.
radhika [Tue, 27 May 2014 14:47:17 +0000 (10:47 -0400)]
Merge branch 'master' into 2525-java-sdk
Tom Clegg [Tue, 27 May 2014 14:44:56 +0000 (10:44 -0400)]
2760: Merge branch '2760-folder-hierarchy' refs #2760
Peter Amstutz [Tue, 27 May 2014 14:43:04 +0000 (10:43 -0400)]
2751: Test tweak to clear settings after changing environment variables.
Tim Pierce [Tue, 27 May 2014 14:38:29 +0000 (10:38 -0400)]
2865: collect garbage after each GET and PUT.
Quick fix for Keep OOM errors: reclaim memory aggressively.
Fixes #2865.
radhika [Tue, 27 May 2014 14:38:32 +0000 (10:38 -0400)]
Merge branch '2756-eventbus-in-workbench' into 2678-owned_by
radhika [Tue, 27 May 2014 14:37:57 +0000 (10:37 -0400)]
Merge branch 'master' into 2756-eventbus-in-workbench
radhika [Tue, 27 May 2014 14:37:12 +0000 (10:37 -0400)]
Merge branch 'master' into 2678-owned_by
Peter Amstutz [Tue, 27 May 2014 14:36:56 +0000 (10:36 -0400)]
2751: Changed to "except:" to "except Exception:". Refactored environment setup in keep_client_Test to be a bit more robust.
radhika [Tue, 27 May 2014 14:35:03 +0000 (10:35 -0400)]
2678: update a-r-p-i to set the owner_uuid on the newly created job when the pi belongs to a group. also, update crunch-dspatch to create the log lines with the group's owner_uuid.
Peter Amstutz [Tue, 27 May 2014 14:23:12 +0000 (10:23 -0400)]
Merge branch 'master' of git.curoverse.com:arvados refs #1885
Peter Amstutz [Tue, 27 May 2014 14:22:38 +0000 (10:22 -0400)]
1885: Tests and bugfixes for -no-get and -no-put options.
Tom Clegg [Tue, 27 May 2014 14:17:33 +0000 (10:17 -0400)]
2760: Merge branch 'master' into 2760-folder-hierarchy
radhika [Mon, 26 May 2014 17:33:45 +0000 (13:33 -0400)]
Merge branch 'master' into 2756-eventbus-in-workbench
Brett Smith [Mon, 26 May 2014 15:36:11 +0000 (11:36 -0400)]
2863: Remove debugging print from Python KeepClient.
Other tools like crunch-job parse the output of tools like arv-get and
arv-put. Right now crunch-job is saving this debug output in the log
attribute of running jobs, which confuses several other tools like
Workbench.
Closes #2863.
radhika [Sun, 25 May 2014 04:08:15 +0000 (00:08 -0400)]
2756: added a link to job when a job is chosen
radhika [Sat, 24 May 2014 22:23:44 +0000 (18:23 -0400)]
2756: log write
radhika [Sat, 24 May 2014 21:08:06 +0000 (17:08 -0400)]
2756: fix issue from merge conflict
radhika [Sat, 24 May 2014 19:49:06 +0000 (15:49 -0400)]
Merge branch 'master' into 2756-eventbus-in-workbench
Conflicts:
apps/workbench/app/assets/javascripts/pipeline_instances.js
Tim Pierce [Sat, 24 May 2014 18:34:45 +0000 (14:34 -0400)]
2829: add X-Block-Size response header
Responses to successful GET or HEAD requests include an X-Block-Size
header.
radhika [Sat, 24 May 2014 17:38:16 +0000 (13:38 -0400)]
Merge branch 'master' into 2767-doc-updates
Tim Pierce [Sat, 24 May 2014 04:44:06 +0000 (00:44 -0400)]
2755: incorporate code review.
* Unit tests cover all permutations of signature/authorization when
--enforce-permissions=false
* Keep is more forgiving about the structure of locators, permits
locator hints of unknown type (as long as they begin with an uppercase
letter)
* Keep delivers 400 Bad Request for requests that do not match any
route, or are lexically invalid. 404 Not Found only for requests with
a syntactically valid hash not found on disk.
Refs #2755.
Tom Clegg [Fri, 23 May 2014 22:44:35 +0000 (18:44 -0400)]
2760: Merge branch 'master' into 2760-folder-hierarchy
Tom Clegg [Fri, 23 May 2014 21:32:21 +0000 (17:32 -0400)]
2760: Disambiguate "hover" from "selected" style. Do not highlight
non-selectable rows on hover.
Tim Pierce [Wed, 21 May 2014 14:58:18 +0000 (10:58 -0400)]
2755: add support for signed locators in the Python SDK.
* arvados.Keep.put() saves the response body (which may contain a
signed locator) and returns it to the caller.
* arvados.Keep.get() passes the full signed locator to the remote Keep
server. The bare MD5 hash is still used for caching and for
shuffled_service_roots
* run_test_server.run_keep() takes arguments 'blob_signing_key' and
'enforce_permissions', for testing permission signatures in unit
tests.
* test_keep_client: new unit tests for permissions:
- with --enforce-permissions=true:
- GET with a signed locator works
- GET with an unsigned locator fails
- unauthenticated GET fails
- with --enforce-permissions=false:
- GET with a signed locator works
- GET with an unsigned locator works
- unauthenticated GET works
Bug fixes to permission handling in the Keep server:
* Locator hints may appear in any order; be flexible. Parse them in
GetBlockHandler rather than in the REST router.
* Returned locators are terminated with newline (consistent with
Warehouse, and more friendly for human debugging).
* The locator returned from a PUT request always has a size hint.
* The correct Authorization header keyword is "OAuth2", not
"OAuth". D'oh.
* Updated unit tests to accommodate newlines, size hints and OAuth2.
Refs #2755.
Tom Clegg [Fri, 23 May 2014 21:12:33 +0000 (17:12 -0400)]
2760: Merge branch 'master' into 2760-folder-hierarchy
Peter Amstutz [Fri, 23 May 2014 21:04:10 +0000 (17:04 -0400)]
Merge branch 'master' into 2044-share-button refs #2044
Tom Clegg [Fri, 23 May 2014 21:01:08 +0000 (17:01 -0400)]
2760: Use "My Folders" to indicate the root of my folder hierarchy.
Peter Amstutz [Fri, 23 May 2014 20:47:45 +0000 (16:47 -0400)]
Merge branch 'master' into 2751-python-sdk-keep-proxy-support refs #2751
Peter Amstutz [Fri, 23 May 2014 19:37:57 +0000 (15:37 -0400)]
Merge branch '1885-keep-proxy' closes #1885
Peter Amstutz [Fri, 23 May 2014 18:08:53 +0000 (14:08 -0400)]
1885: Added logging of invalid requests. Added logging when the server list is
update. Improved KeepClient error reporting.
projects / arvados.git / log