Peter Amstutz [Fri, 5 Sep 2014 20:23:38 +0000 (16:23 -0400)]
3822: Added 'ensure_unique_name' option to #create method for API server to
choose a unique name when there is a name collision in the database.
arv-run-pipeline-instance checks to see if there is an output file with the
same name and contents, and uses 'ensure_unique_name' when creating collection.
Brett Smith [Wed, 3 Sep 2014 21:52:00 +0000 (17:52 -0400)]
3720: Limit Workbench file rendering for large Collections.
Rendering too many files can cause rendering to take too long, and
there's not much point because it can really strain browsers too.
Arbitrarily cap rendering at 10,000 files.
Brett Smith [Fri, 29 Aug 2014 21:37:13 +0000 (17:37 -0400)]
3720: Refactor manifest parsing from API server to Ruby SDK.
To date, the API server has been parsing manifests and returning
parsed information in the files attribute. That's convenient, but
causes performance problems in large Collections. Profiling indicates
that most of the API server's response time is spent in the
as_api_response method; rendering the same information twice really
hurts.
This commits removes the API server's need to always parse manifests,
and moves the parsing code to the Ruby SDK. Both the API server and
Workbench use this Gem to parse manifests on an as-needed basis.
This, combined with improvements to the new Keep::Manifest class, lays
the groundwork for future performance gains.
Brett Smith [Wed, 3 Sep 2014 18:57:25 +0000 (14:57 -0400)]
3720: Clean up+skip Ruby SDK tests.
First I changed these to get them passing again. With that done, I
added the skip. We're not currently running these tests, and getting
them going the right way again (e.g., with run_test_server.py or
mocks) is out of scope for my current task.
Brett Smith [Thu, 4 Sep 2014 20:05:57 +0000 (16:05 -0400)]
3704: Treat project filters consistently in Workbench chooser.
Before this, a chooser that was loaded on a specific project would
return empty results if the user selected a different project. This
is because it would search for items with two different owner_uuids,
both the project selected at load time, and the project the user
selected later. This patch fixes that by separating the project
filter from other filters, and preseeding it in the same place where
it's updated by the project selection AJAX. Closes #3704, #3778.
3710: update the combine_selected_files_into_collection method to also handle the scenarios where a collection uuid or "collection uuid/filename" is passed.
Peter Amstutz [Tue, 2 Sep 2014 19:59:51 +0000 (15:59 -0400)]
Delete names and description columns from jobs that shouldn't be there. Delete
jobs_owner_uuid_name_unique and pipeline_instance_owner_uuid_name_unique
indexes added by mistake. refs #3036.
refs #3637 - merge Tom's updates to chooser implementation, where selected items are passed as parameters.
Merge branch '3637-selection-through-chooser'
Peter Amstutz [Thu, 28 Aug 2014 15:20:35 +0000 (11:20 -0400)]
3709: arv-put now prints collection uuid by default. Add --portable-data-hash
to arv-put to get the old behavior (print PDH). Crunch-job uses arv-put
--portable-data-hash to get the PDH to put in the log field. Crunch-job logs
both PDH and collection UUID.
Peter Amstutz [Wed, 27 Aug 2014 19:02:20 +0000 (15:02 -0400)]
3342: Handle case in PipelineInstance#friendly_link_name where
pipeline_template_uuid doesn't return a PipelineTemplate. Removed unnecessary
code from crunch-job to calculate md5 hash of manifest_text (API server doesn't
require md5 hash to be provided with collection creation any more.)
Tom Clegg [Tue, 26 Aug 2014 18:15:15 +0000 (14:15 -0400)]
Run the task setup script inside the container (if any), instead of
running setup on the host, using bind-mount for CRUNCH_SRC, and
assuming other directories have already been set up in the docker
image.
Tim Pierce [Fri, 22 Aug 2014 15:12:53 +0000 (11:12 -0400)]
3414: adding PullHandler and Replicator
Added PullHandler and a "replicator" package to handle "PUT /pull" requests.
PUT /pull requests are routed to PullHandler, which authenticates the
request and validates the JSON in the request body. Valid requests are
sent to the replicator.
The Keepstore replicator runs a goroutine which repeatedly listens on
its input channel for a new pull list.
TestPullHandler tests each combination of: request from superuser;
request from ordinary user; properly formatted pull request; improperly
formatted pull request. It checks the state of the replicator when done
to make sure that it has the expected number of pull requests.
projects / arvados.git / log