Nico Cesar [Mon, 29 Jun 2015 14:53:18 +0000 (10:53 -0400)]
we need to manually install python-pyasn1=0.1.7 python-pyasn1-modules=0.0.5 because apt-get wont resolve correctly.
apt-get install are less quiet now too.
no issue #
Tom Clegg [Fri, 26 Jun 2015 22:28:57 +0000 (18:28 -0400)]
6440: Display AJAX errors as text, not HTML. closes #6440
Brett Smith [Fri, 26 Jun 2015 19:52:46 +0000 (15:52 -0400)]
Fix bad CollectionReader reference in arv-put.
No issue #.
Brett Smith [Fri, 26 Jun 2015 15:42:10 +0000 (11:42 -0400)]
Merge branch '6432-pysdk-more-ca-certs-wip'
Closes #6432, #6434.
Brett Smith [Thu, 25 Jun 2015 23:36:52 +0000 (19:36 -0400)]
6432: Python SDK can find and use CA certs on Red Hat.
radhika [Fri, 26 Jun 2015 14:36:08 +0000 (10:36 -0400)]
closes #6234
Merge branch '6234-user-edit-self'
radhika [Fri, 26 Jun 2015 14:34:33 +0000 (10:34 -0400)]
Merge branch 'master' into 6234-user-edit-self
Peter Amstutz [Thu, 25 Jun 2015 17:55:07 +0000 (13:55 -0400)]
Update arvados_fuse version dependency on arvados-python-client. refs #3198
Peter Amstutz [Thu, 25 Jun 2015 17:52:18 +0000 (13:52 -0400)]
Merge branch '3198-writable-fuse' closes #3198
Peter Amstutz [Thu, 25 Jun 2015 14:30:28 +0000 (10:30 -0400)]
3198: Rename --enable-write to --read-write. Tweak docs.
radhika [Wed, 24 Jun 2015 14:24:14 +0000 (10:24 -0400)]
refs #6218
Merge branch '6218-add-performance-profiling-to-python-sdk'
radhika [Wed, 24 Jun 2015 14:22:46 +0000 (10:22 -0400)]
6218: update the exception handling.
radhika [Wed, 24 Jun 2015 13:58:39 +0000 (09:58 -0400)]
6218: correct exception handling.
Brett Smith [Tue, 23 Jun 2015 14:21:46 +0000 (10:21 -0400)]
6320: Update Workbench test for active user VM setup.
Now that the test fixtures give the active user access to testvm2,
this test needs to be updated to account for it, just like the API
server tests were. Refs #6320.
Brett Smith [Tue, 23 Jun 2015 14:13:46 +0000 (10:13 -0400)]
Remove unused modules from PySDK setup.py.
No issue #.
radhika [Tue, 23 Jun 2015 13:25:08 +0000 (09:25 -0400)]
6218: update the symlink and comment
Brett Smith [Mon, 22 Jun 2015 20:55:07 +0000 (16:55 -0400)]
Merge branch '4410-slurm-fails-are-tempfails-wip'
Closes #4410, #6283.
Tom Clegg [Mon, 22 Jun 2015 20:55:06 +0000 (16:55 -0400)]
6218: Refactor profiler as a decorator
Brett Smith [Mon, 15 Jun 2015 17:54:36 +0000 (13:54 -0400)]
4410: Crunch retries jobs when all SLURM nodes fail.
See the ticket for detailed background discussion and implementation
rationale, especially notes 13 and 14.
This required a couple of ancillary changes:
* crunch-job now makes a distinction between "task failed because a
node failed," and "task failed for other temporary reason." It uses
this additional information to decide when it should retry tasks
itself, and when it needs to give up and kick the problem up to
crunch-dispatch.
* crunch-job now handles creating log collections itself from
manifests generated by arv-put. This enables it to append to logs
generated during previous attempts to run the job.
Brett Smith [Mon, 15 Jun 2015 21:04:25 +0000 (17:04 -0400)]
4410: crunch-dispatch logs crunch-job exit later.
This makes it easier to log the exit code, and makes the logs look
nicer because the exit log doesn't interrupt crunch-job's stderr.
radhika [Mon, 22 Jun 2015 15:31:40 +0000 (11:31 -0400)]
6218: fix typo in sample test
radhika [Fri, 19 Jun 2015 21:01:32 +0000 (17:01 -0400)]
6218: add performance profiling and a sample test in python sdk.
Peter Amstutz [Thu, 18 Jun 2015 20:55:22 +0000 (16:55 -0400)]
3198: Fix test
Peter Amstutz [Thu, 18 Jun 2015 20:32:21 +0000 (16:32 -0400)]
3198: concurrency test and associated fixes
Peter Amstutz [Wed, 17 Jun 2015 21:06:57 +0000 (17:06 -0400)]
WIP
Brett Smith [Thu, 18 Jun 2015 15:35:40 +0000 (11:35 -0400)]
Merge branch '6320-api-logins-include-groups-wip'
Refs #6320. Closes #6325.
Brett Smith [Tue, 16 Jun 2015 13:40:58 +0000 (09:40 -0400)]
6320: API virtual machines login method include groups information.
This is necessary for the virtual machine user setup script to see the
groups information added in #6254.
This required updating a few tests that assumed the active user had no
access to testvm2.
Brett Smith [Tue, 16 Jun 2015 13:18:32 +0000 (09:18 -0400)]
6320: Add tests for API virtual machines login method.
radhika [Thu, 18 Jun 2015 14:49:23 +0000 (10:49 -0400)]
closes #6156
Merge branch '6156-hostnames-in-nodes'
radhika [Thu, 18 Jun 2015 14:47:54 +0000 (10:47 -0400)]
6156: convert the ping-should-fail test when hostname config is malformed into a controller test.
radhika [Thu, 18 Jun 2015 14:22:55 +0000 (10:22 -0400)]
Merge branch 'master' into 6156-hostnames-in-nodes
radhika [Thu, 18 Jun 2015 02:06:46 +0000 (22:06 -0400)]
refs #6277
Merge branch '6277-manifest-validation-api'
radhika [Thu, 18 Jun 2015 02:04:10 +0000 (22:04 -0400)]
6277: since locators are added to multilevel_collection_1 fixture, the collection retrieved after an update will have
a signed manifest; hence, the test needs to compare the old manifest with the new one after stripping signatures.
radhika [Wed, 17 Jun 2015 23:26:41 +0000 (19:26 -0400)]
refs #6277
Merge branch '6277-manifest-validation-api'
radhika [Wed, 17 Jun 2015 23:06:00 +0000 (19:06 -0400)]
6277: rename the before_validation filter check_manifest_validity as log_invalid_manifest_format until we are ready to actually validate manifest formats.
radhika [Wed, 17 Jun 2015 22:54:36 +0000 (18:54 -0400)]
Merge branch 'master' into 6277-manifest-validation-api
radhika [Wed, 17 Jun 2015 22:34:17 +0000 (18:34 -0400)]
6156: use only sprintf formatting for node slot_number config.
radhika [Wed, 17 Jun 2015 21:36:58 +0000 (17:36 -0400)]
Merge branch 'master' into 6156-hostnames-in-nodes
radhika [Wed, 17 Jun 2015 21:33:46 +0000 (17:33 -0400)]
6234: admin user can see other users' home projects; also add link to users' home projects in /users page rows.
Tom Clegg [Wed, 17 Jun 2015 20:40:29 +0000 (16:40 -0400)]
6277: Simplify manifest-building loop, fix up truncation tests.
radhika [Wed, 17 Jun 2015 18:11:27 +0000 (14:11 -0400)]
6234: In /users page, display the "Show" button to admin users as well; however, ensure redirection to
projects/{user-uuid} takes user to user's home project while /users/{user-uuid} takes to user's page.
Also, move should correctly display user's home project.
Tom Clegg [Wed, 17 Jun 2015 02:30:03 +0000 (22:30 -0400)]
Merge branch '6272-index-eof' closes #6272
Tom Clegg [Tue, 16 Jun 2015 20:14:58 +0000 (16:14 -0400)]
6272: Add blank line to indicate index response EOF. Error out of data manager if not received.
Tom Clegg [Tue, 16 Jun 2015 22:17:03 +0000 (18:17 -0400)]
Do not blow up if HOME is not set. refs #2256
Peter Amstutz [Tue, 16 Jun 2015 19:35:31 +0000 (15:35 -0400)]
3198: Add enable_write flag to FUSE and --enable-write and --read-only to
arv-mount. Update documentation. Code cleanup in locators_and_ranges() and
set_state().
Tom Clegg [Tue, 16 Jun 2015 20:09:04 +0000 (16:09 -0400)]
Merge branch '6222-precompile-regexps' refs #6222
Tom Clegg [Sat, 6 Jun 2015 20:29:48 +0000 (16:29 -0400)]
6222: Precompile all regexps. Remove wasted effort in GetBlockHandler.
radhika [Tue, 16 Jun 2015 15:32:55 +0000 (11:32 -0400)]
6156: support config format for setting a node's hostname
radhika [Mon, 15 Jun 2015 19:02:23 +0000 (15:02 -0400)]
6277: Add check_manifest_validity before_filter in collection model; however, at the moment, this method always true after logging the validation error.
radhika [Mon, 15 Jun 2015 15:34:58 +0000 (11:34 -0400)]
refs #6277
Merge branch '6277-manifest-validation'
radhika [Mon, 15 Jun 2015 15:34:37 +0000 (11:34 -0400)]
Merge branch 'master' into 6277-manifest-validation
radhika [Mon, 15 Jun 2015 15:33:45 +0000 (11:33 -0400)]
6277: all that work and missed the basic nil and empty string check!!!
Peter Amstutz [Mon, 15 Jun 2015 14:49:00 +0000 (10:49 -0400)]
3198: Performance tuning based on benchmarking. Limit number of segments to
look ahead for prefetch, don't prefetch blocks over and over again.
radhika [Mon, 15 Jun 2015 14:45:21 +0000 (10:45 -0400)]
closes #6254
Merge branch '6254-groups-during-setup'
radhika [Sun, 14 Jun 2015 01:31:29 +0000 (21:31 -0400)]
refs #6277 : ruby sdk with manifest validation method
Merge branch '6277-manifest-validation'
radhika [Sun, 14 Jun 2015 01:21:39 +0000 (21:21 -0400)]
6277: extra white space
radhika [Sun, 14 Jun 2015 00:47:04 +0000 (20:47 -0400)]
Merge branch 'master' into 6277-manifest-validation
radhika [Sat, 13 Jun 2015 02:24:49 +0000 (22:24 -0400)]
6254: minor update to Groups text field label to avoid conflict with 'Virtual Machine' lebel.
radhika [Sat, 13 Jun 2015 01:19:35 +0000 (21:19 -0400)]
6254: better groups text field label
radhika [Sat, 13 Jun 2015 01:06:23 +0000 (21:06 -0400)]
6254: instead of tooltip, use a self explanatory label for groups text field.
radhika [Sat, 13 Jun 2015 00:26:28 +0000 (20:26 -0400)]
7254: add groups to vm link.
Peter Amstutz [Fri, 12 Jun 2015 19:54:51 +0000 (15:54 -0400)]
3198: Fix frontrunning (subsequent updates after a commit causing the
collection to conflict with itself). Also fix filename character encoding.
radhika [Fri, 12 Jun 2015 18:30:20 +0000 (14:30 -0400)]
6254: remove redundant statement
radhika [Fri, 12 Jun 2015 18:26:52 +0000 (14:26 -0400)]
6254: add groups and verify in test.
Brett Smith [Fri, 12 Jun 2015 18:05:19 +0000 (14:05 -0400)]
Merge branch '6149-quiet-egg-info-stderr-wip'
Closes #6149, #6284.
Brett Smith [Thu, 11 Jun 2015 21:16:03 +0000 (17:16 -0400)]
6149: crunch-job installer handles egg_info errors better.
Capture the command's stderr and confirm that the error refers to
git. If it does, ignore the stderr and set a build tag. Otherwise,
propagate stderr and abort.
radhika [Fri, 12 Jun 2015 15:02:13 +0000 (11:02 -0400)]
6254: slight performance improvement where the can_login link is not retrieved and checked if the groups passed in are the same as those already saved.
radhika [Fri, 12 Jun 2015 14:51:26 +0000 (10:51 -0400)]
6254: add "groups" to user setup process; these comma separated groups entered in the popup will be saved as an array of groups property for the user's can_login link.
Peter Amstutz [Fri, 12 Jun 2015 14:33:39 +0000 (10:33 -0400)]
3198: Fix tests for change in ~conflict~ format. Tweak comments.
Peter Amstutz [Thu, 11 Jun 2015 19:22:32 +0000 (15:22 -0400)]
3198: Additional fixes based on review feedback.
* Adjust how ~conflict~ files are renamed
* Add finalize() on fuse inodes() to terminate put and get threads.
* Improve naming of some test case helpers
* Relax regex match on authorization token
Peter Amstutz [Thu, 11 Jun 2015 17:12:21 +0000 (13:12 -0400)]
3198: arvfile.py and collection.py updates based on feedback.
* Add retryable ERROR state to buffer blocks.
* Fix race conditions in testing a bufferblock state before trying to change state.
* Change "modified" to "committed" (which has the opposite meaning, but is more
accurate the way the flag is used)
* Refactor block manager background upload/download threads
* Fix Collection.mkdirs() to match behavior of os.makedirs()
Tom Clegg [Thu, 11 Jun 2015 05:58:49 +0000 (01:58 -0400)]
6277: Catch whitespace errors, and "." and ".." in paths. Rename valid? to validate!.
radhika [Fri, 12 Jun 2015 00:13:45 +0000 (20:13 -0400)]
6277: Add more restrictions to manifest format such as cannot start with '/' and end with '/' for file names,
as well as stream names and files names should not contain '//'.
Added the tests provided by Tom during review.
radhika [Thu, 11 Jun 2015 20:19:05 +0000 (16:19 -0400)]
6277: valid manifest must end with new line.
radhika [Thu, 11 Jun 2015 20:11:28 +0000 (16:11 -0400)]
6277: improve error message for missing file tokens.
radhika [Thu, 11 Jun 2015 18:33:48 +0000 (14:33 -0400)]
6277: add Manifest::valid? method in ruby sdk.
Peter Amstutz [Wed, 10 Jun 2015 13:30:07 +0000 (09:30 -0400)]
3198: Adjust language of errors to say what the error is instead of saying what the
input should be.
Peter Amstutz [Wed, 10 Jun 2015 13:10:41 +0000 (09:10 -0400)]
Merge branch 'master' into 3198-writable-fuse
Tom Clegg [Tue, 9 Jun 2015 19:16:28 +0000 (15:16 -0400)]
Update arvados in bundle. refs #6203 refs #6277
Tom Clegg [Tue, 9 Jun 2015 19:12:46 +0000 (15:12 -0400)]
Use default word wrap (no mid-word line breaks). refs #6057
Nico Cesar [Tue, 9 Jun 2015 18:48:29 +0000 (14:48 -0400)]
qr1hi-automated-performance-suite is failing because the test doesnt give enough time for the page to render (now 50s makes the test past.). no issue #
Nico Cesar [Tue, 9 Jun 2015 18:38:18 +0000 (14:38 -0400)]
qr1hi-automated-performance-suite is failing because the test doesnt give enough time for the page to render
no issue #
Nico Cesar [Tue, 9 Jun 2015 15:32:25 +0000 (11:32 -0400)]
openssl self cert creation failed because missing of "tmp" directory.
api server failed to start because missing "tmp/logs" and "tmp/api" ditectories
no issue #
Peter Amstutz [Tue, 9 Jun 2015 15:22:52 +0000 (11:22 -0400)]
Merge branch '6235-go-sdk-discovery' closes #6235
Brett Smith [Tue, 9 Jun 2015 14:48:36 +0000 (10:48 -0400)]
Merge branch '5790-copy-most-recent-docker-image-wip'
Closes #5790, #6103.
Brett Smith [Wed, 3 Jun 2015 20:59:24 +0000 (16:59 -0400)]
5790: Improve Docker image listing in Python SDK.
* Always fetch all relevant Docker links.
* Support finding images by image hash.
* Show image hashes when listing images by name.
* Like Docker itself, when an image has multiple names and we're not
filtering by name, list each one.
* Better match the API server's priority logic:
* Ignore links to collections that aren't found.
* Links with an image_timestamp always have priority over those that
don't, regardless of their respective created_at timestamps.
The main motivation for this change is to make sure arv-copy gets the
right Docker image when copying a pipeline template recursively.
This implementation goes through some trouble to parse timestamps out
of each Docker link only once.
Brett Smith [Tue, 9 Jun 2015 14:26:44 +0000 (10:26 -0400)]
Merge branch '6152-compute-node-no-arv-wip'
Closes #6152, #6256.
Brett Smith [Fri, 5 Jun 2015 19:46:08 +0000 (15:46 -0400)]
6152: Use Python SDK for compute node installation.
This eliminates an otherwise-needless dependency on `arv` and the
entire Ruby stack.
radhika [Tue, 9 Jun 2015 14:25:59 +0000 (10:25 -0400)]
closes #6093
Merge branch '6093-refresh-docs'
radhika [Tue, 9 Jun 2015 14:21:38 +0000 (10:21 -0400)]
6093: remove "Alternate way to add SSH keys" and add the "Manage account" link blurb to "Adding your keys" section itself.
Tom Clegg [Tue, 9 Jun 2015 14:11:35 +0000 (10:11 -0400)]
Merge branch '6203-locator-regexp' refs #6203 refs #6277
Nico Cesar [Wed, 10 Jun 2015 16:26:15 +0000 (12:26 -0400)]
added shell-image back again to the build. I checked on my local machine and works great
no issue #
Brett Smith [Wed, 10 Jun 2015 15:24:38 +0000 (11:24 -0400)]
5790: Fix PySDK Docker image listing comparing ints and datetimes.
Closes #5790.
Ward Vandewege [Wed, 10 Jun 2015 14:32:40 +0000 (10:32 -0400)]
Add instructions to the 'Create standard objects' page to create a cluster-wide readable project for standard Arvados Docker images.
refs #6096
radhika [Wed, 10 Jun 2015 14:15:47 +0000 (10:15 -0400)]
closes #6203
Merge branch '6203-collection-perf-api'
radhika [Wed, 10 Jun 2015 14:06:35 +0000 (10:06 -0400)]
Merge branch '6203-collection-perf-api-TC' of git.curoverse.com:arvados into 6203-collection-perf-api-TC
Conflicts:
services/api/app/models/collection.rb
Tom Clegg [Mon, 8 Jun 2015 19:34:48 +0000 (15:34 -0400)]
6203: Fix loophole allowing locators in bogus manifests to be accepted
without verification and then signed in responses. refs #6277
Tom Clegg [Mon, 8 Jun 2015 16:03:42 +0000 (12:03 -0400)]
6203: Add tests for LOCATOR_REGEXP. Fix regexp to reject "++" and trailing "\n".
radhika [Tue, 9 Jun 2015 14:09:07 +0000 (10:09 -0400)]
Merge branch 'master' into 6093-refresh-docs
Tom Clegg [Mon, 8 Jun 2015 14:10:26 +0000 (10:10 -0400)]
6203: Accept (and discard) hints in client-provided portable_data_hash.