arvados.git
9 years agoFix typo. No issue #
Tom Clegg [Wed, 8 Jul 2015 14:13:04 +0000 (10:13 -0400)]
Fix typo. No issue #

9 years agoMerge branch '5989-api-all-repos-permissions-wip'
Brett Smith [Tue, 7 Jul 2015 21:02:06 +0000 (17:02 -0400)]
Merge branch '5989-api-all-repos-permissions-wip'

Closes #5989, #6479.

9 years ago5989: API repo perms method always returns all repos.
Brett Smith [Sun, 5 Jul 2015 11:39:09 +0000 (07:39 -0400)]
5989: API repo perms method always returns all repos.

Previously the method would not return repositories that were not
accessible to any authorized key.  This change helps deployment tools
like update-gitolite do repository setup correctly with a single API
call.

9 years agoMerge branch '5677-plpgsql-warning' closes #5677
Tom Clegg [Tue, 7 Jul 2015 15:42:12 +0000 (11:42 -0400)]
Merge branch '5677-plpgsql-warning' closes #5677

9 years ago5677: Edit structure.sql after db:structure:dump: avoid warnings when loading, and...
Tom Clegg [Fri, 26 Jun 2015 06:54:31 +0000 (02:54 -0400)]
5677: Edit structure.sql after db:structure:dump: avoid warnings when loading, and editwars when committing.

9 years agocloses #6466
Manoj [Tue, 7 Jul 2015 13:08:21 +0000 (09:08 -0400)]
closes #6466
Merge branch '6466-disable-public-projects'

9 years agoMerge branch 'master' into 6466-disable-public-projects
Manoj [Tue, 7 Jul 2015 13:07:33 +0000 (09:07 -0400)]
Merge branch 'master' into 6466-disable-public-projects

9 years agoMerge branch '6438-multiplatform-install-guide-wip'
Brett Smith [Mon, 6 Jul 2015 21:37:21 +0000 (17:37 -0400)]
Merge branch '6438-multiplatform-install-guide-wip'

Closes #6438, #6455.  Refs #6388.

9 years ago6455: Include more steps in Crunch install docs.
Brett Smith [Mon, 6 Jul 2015 21:29:03 +0000 (17:29 -0400)]
6455: Include more steps in Crunch install docs.

* Document the necessary API server configuration change for
  crunch-dispatch.
* Document Docker installation.
* Note compute node crunch users need to be in the fuse group.

9 years ago6438: Modernize Rails server installation guides.
Brett Smith [Mon, 6 Jul 2015 21:27:08 +0000 (17:27 -0400)]
6438: Modernize Rails server installation guides.

Document the package-based installation procedure, and associated
Nginx configuration.

9 years ago6388: Add CentOS 6.6 instructions to install docs.
Brett Smith [Mon, 6 Jul 2015 21:26:10 +0000 (17:26 -0400)]
6388: Add CentOS 6.6 instructions to install docs.

9 years ago6466 - Update body html for enable public page and added tests to project controller...
Manoj [Mon, 6 Jul 2015 18:00:24 +0000 (14:00 -0400)]
6466 - Update body html for enable public page and added tests to project controller test.

9 years agocloses #6013
radhika [Mon, 6 Jul 2015 16:08:29 +0000 (12:08 -0400)]
closes #6013
Merge branch '6013-cleanup-setup-impl'

9 years ago6013: revert back the deactivate user button leading text in workbench user admin...
radhika [Fri, 3 Jul 2015 21:03:32 +0000 (17:03 -0400)]
6013: revert back the deactivate user button leading text in workbench user admin page.

9 years ago6013: update user notification email upon setup.
radhika [Fri, 3 Jul 2015 20:56:53 +0000 (16:56 -0400)]
6013: update user notification email upon setup.

9 years ago6013: remove "repository" from user setup popup in workbench
radhika [Fri, 3 Jul 2015 19:44:26 +0000 (15:44 -0400)]
6013: remove "repository" from user setup popup in workbench

9 years ago6466:Added config parameter to disable public projects page and added a test to proje...
Manoj [Fri, 3 Jul 2015 16:06:22 +0000 (12:06 -0400)]
6466:Added config parameter to disable public projects page and added a test to projects controller.

9 years agoMerge branch '5989-bug_1' merge with puppet repo "copy and paste" to sync up duplicat...
Nico Cesar [Thu, 2 Jul 2015 14:37:38 +0000 (10:37 -0400)]
Merge branch '5989-bug_1' merge with puppet repo "copy and paste" to sync up duplicated code.

refs #5989

9 years ago5989: bug_1 added puppet repository patch
Nico Cesar [Thu, 2 Jul 2015 14:35:03 +0000 (10:35 -0400)]
5989: bug_1 added puppet repository patch

this is duplicated code from:

puppet/modules/arvados-git-server/templates/usr-local-arvados-update-gitolite.rb.erb

I just copy the code that was there

9 years agoMerge branch '6264-run-command-task-env' refs #6264
Peter Amstutz [Thu, 2 Jul 2015 14:25:20 +0000 (10:25 -0400)]
Merge branch '6264-run-command-task-env' refs #6264

9 years ago6264: Add task.env to documentation.
Peter Amstutz [Thu, 2 Jul 2015 13:47:52 +0000 (09:47 -0400)]
6264: Add task.env to documentation.

9 years ago6264: Support setting environment variables.
Peter Amstutz [Wed, 1 Jul 2015 21:43:28 +0000 (17:43 -0400)]
6264: Support setting environment variables.

9 years ago5989: create directory structure for temp files
Nico Cesar [Wed, 1 Jul 2015 20:23:33 +0000 (16:23 -0400)]
5989: create directory structure for temp files

refs #5989 bug_1

9 years agoInvoke crunch-job with `/usr/bin/env perl`.
Brett Smith [Wed, 1 Jul 2015 12:25:15 +0000 (08:25 -0400)]
Invoke crunch-job with `/usr/bin/env perl`.

No issue #.

9 years agoDockerfile had an error. fixed
Nico Cesar [Mon, 29 Jun 2015 15:00:01 +0000 (11:00 -0400)]
Dockerfile had an error. fixed

no issue #

9 years agowe need to manually install python-pyasn1=0.1.7 python-pyasn1-modules=0.0.5 because...
Nico Cesar [Mon, 29 Jun 2015 14:53:18 +0000 (10:53 -0400)]
we need to manually install python-pyasn1=0.1.7 python-pyasn1-modules=0.0.5 because apt-get wont resolve correctly.
apt-get install are less quiet now too.

no issue #

9 years ago6440: Display AJAX errors as text, not HTML. closes #6440
Tom Clegg [Fri, 26 Jun 2015 22:28:57 +0000 (18:28 -0400)]
6440: Display AJAX errors as text, not HTML. closes #6440

9 years agoFix bad CollectionReader reference in arv-put.
Brett Smith [Fri, 26 Jun 2015 19:52:46 +0000 (15:52 -0400)]
Fix bad CollectionReader reference in arv-put.

No issue #.

9 years agoMerge branch '6432-pysdk-more-ca-certs-wip'
Brett Smith [Fri, 26 Jun 2015 15:42:10 +0000 (11:42 -0400)]
Merge branch '6432-pysdk-more-ca-certs-wip'

Closes #6432, #6434.

9 years ago6432: Python SDK can find and use CA certs on Red Hat.
Brett Smith [Thu, 25 Jun 2015 23:36:52 +0000 (19:36 -0400)]
6432: Python SDK can find and use CA certs on Red Hat.

9 years agocloses #6234
radhika [Fri, 26 Jun 2015 14:36:08 +0000 (10:36 -0400)]
closes #6234
Merge branch '6234-user-edit-self'

9 years agoMerge branch 'master' into 6234-user-edit-self
radhika [Fri, 26 Jun 2015 14:34:33 +0000 (10:34 -0400)]
Merge branch 'master' into 6234-user-edit-self

9 years agoUpdate arvados_fuse version dependency on arvados-python-client. refs #3198
Peter Amstutz [Thu, 25 Jun 2015 17:55:07 +0000 (13:55 -0400)]
Update arvados_fuse version dependency on arvados-python-client.  refs #3198

9 years agoMerge branch '3198-writable-fuse' closes #3198
Peter Amstutz [Thu, 25 Jun 2015 17:52:18 +0000 (13:52 -0400)]
Merge branch '3198-writable-fuse' closes #3198

9 years ago3198: Rename --enable-write to --read-write. Tweak docs.
Peter Amstutz [Thu, 25 Jun 2015 14:30:28 +0000 (10:30 -0400)]
3198: Rename --enable-write to --read-write.  Tweak docs.

9 years agorefs #6218
radhika [Wed, 24 Jun 2015 14:24:14 +0000 (10:24 -0400)]
refs #6218
Merge branch '6218-add-performance-profiling-to-python-sdk'

9 years ago6218: update the exception handling.
radhika [Wed, 24 Jun 2015 14:22:46 +0000 (10:22 -0400)]
6218: update the exception handling.

9 years ago6218: correct exception handling.
radhika [Wed, 24 Jun 2015 13:58:39 +0000 (09:58 -0400)]
6218: correct exception handling.

9 years ago6320: Update Workbench test for active user VM setup.
Brett Smith [Tue, 23 Jun 2015 14:21:46 +0000 (10:21 -0400)]
6320: Update Workbench test for active user VM setup.

Now that the test fixtures give the active user access to testvm2,
this test needs to be updated to account for it, just like the API
server tests were.  Refs #6320.

9 years agoRemove unused modules from PySDK setup.py.
Brett Smith [Tue, 23 Jun 2015 14:13:46 +0000 (10:13 -0400)]
Remove unused modules from PySDK setup.py.

No issue #.

9 years ago6218: update the symlink and comment
radhika [Tue, 23 Jun 2015 13:25:08 +0000 (09:25 -0400)]
6218: update the symlink and comment

9 years agoMerge branch '4410-slurm-fails-are-tempfails-wip'
Brett Smith [Mon, 22 Jun 2015 20:55:07 +0000 (16:55 -0400)]
Merge branch '4410-slurm-fails-are-tempfails-wip'

Closes #4410, #6283.

9 years ago6218: Refactor profiler as a decorator
Tom Clegg [Mon, 22 Jun 2015 20:55:06 +0000 (16:55 -0400)]
6218: Refactor profiler as a decorator

9 years ago4410: Crunch retries jobs when all SLURM nodes fail.
Brett Smith [Mon, 15 Jun 2015 17:54:36 +0000 (13:54 -0400)]
4410: Crunch retries jobs when all SLURM nodes fail.

See the ticket for detailed background discussion and implementation
rationale, especially notes 13 and 14.

This required a couple of ancillary changes:

* crunch-job now makes a distinction between "task failed because a
  node failed," and "task failed for other temporary reason."  It uses
  this additional information to decide when it should retry tasks
  itself, and when it needs to give up and kick the problem up to
  crunch-dispatch.

* crunch-job now handles creating log collections itself from
  manifests generated by arv-put.  This enables it to append to logs
  generated during previous attempts to run the job.

9 years ago4410: crunch-dispatch logs crunch-job exit later.
Brett Smith [Mon, 15 Jun 2015 21:04:25 +0000 (17:04 -0400)]
4410: crunch-dispatch logs crunch-job exit later.

This makes it easier to log the exit code, and makes the logs look
nicer because the exit log doesn't interrupt crunch-job's stderr.

9 years ago6218: fix typo in sample test
radhika [Mon, 22 Jun 2015 15:31:40 +0000 (11:31 -0400)]
6218: fix typo in sample test

9 years ago6218: add performance profiling and a sample test in python sdk.
radhika [Fri, 19 Jun 2015 21:01:32 +0000 (17:01 -0400)]
6218: add performance profiling and a sample test in python sdk.

9 years ago3198: Fix test
Peter Amstutz [Thu, 18 Jun 2015 20:55:22 +0000 (16:55 -0400)]
3198: Fix test

9 years ago3198: concurrency test and associated fixes
Peter Amstutz [Thu, 18 Jun 2015 20:32:21 +0000 (16:32 -0400)]
3198: concurrency test and associated fixes

9 years agoWIP
Peter Amstutz [Wed, 17 Jun 2015 21:06:57 +0000 (17:06 -0400)]
WIP

9 years agoMerge branch '6320-api-logins-include-groups-wip'
Brett Smith [Thu, 18 Jun 2015 15:35:40 +0000 (11:35 -0400)]
Merge branch '6320-api-logins-include-groups-wip'

Refs #6320.  Closes #6325.

9 years ago6320: API virtual machines login method include groups information.
Brett Smith [Tue, 16 Jun 2015 13:40:58 +0000 (09:40 -0400)]
6320: API virtual machines login method include groups information.

This is necessary for the virtual machine user setup script to see the
groups information added in #6254.

This required updating a few tests that assumed the active user had no
access to testvm2.

9 years ago6320: Add tests for API virtual machines login method.
Brett Smith [Tue, 16 Jun 2015 13:18:32 +0000 (09:18 -0400)]
6320: Add tests for API virtual machines login method.

9 years agocloses #6156
radhika [Thu, 18 Jun 2015 14:49:23 +0000 (10:49 -0400)]
closes #6156
Merge branch '6156-hostnames-in-nodes'

9 years ago6156: convert the ping-should-fail test when hostname config is malformed into a...
radhika [Thu, 18 Jun 2015 14:47:54 +0000 (10:47 -0400)]
6156: convert the ping-should-fail test when hostname config is malformed into a controller test.

9 years agoMerge branch 'master' into 6156-hostnames-in-nodes
radhika [Thu, 18 Jun 2015 14:22:55 +0000 (10:22 -0400)]
Merge branch 'master' into 6156-hostnames-in-nodes

9 years agorefs #6277
radhika [Thu, 18 Jun 2015 02:06:46 +0000 (22:06 -0400)]
refs #6277
Merge branch '6277-manifest-validation-api'

9 years ago6277: since locators are added to multilevel_collection_1 fixture, the collection...
radhika [Thu, 18 Jun 2015 02:04:10 +0000 (22:04 -0400)]
6277: since locators are added to multilevel_collection_1 fixture, the collection retrieved after an update will have
a signed manifest; hence, the test needs to compare the old manifest with the new one after stripping signatures.

9 years agorefs #6277
radhika [Wed, 17 Jun 2015 23:26:41 +0000 (19:26 -0400)]
refs #6277
Merge branch '6277-manifest-validation-api'

9 years ago6277: rename the before_validation filter check_manifest_validity as log_invalid_mani...
radhika [Wed, 17 Jun 2015 23:06:00 +0000 (19:06 -0400)]
6277: rename the before_validation filter check_manifest_validity as log_invalid_manifest_format until we are ready to actually validate manifest formats.

9 years agoMerge branch 'master' into 6277-manifest-validation-api
radhika [Wed, 17 Jun 2015 22:54:36 +0000 (18:54 -0400)]
Merge branch 'master' into 6277-manifest-validation-api

9 years ago6156: use only sprintf formatting for node slot_number config.
radhika [Wed, 17 Jun 2015 22:34:17 +0000 (18:34 -0400)]
6156: use only sprintf formatting for node slot_number config.

9 years agoMerge branch 'master' into 6156-hostnames-in-nodes
radhika [Wed, 17 Jun 2015 21:36:58 +0000 (17:36 -0400)]
Merge branch 'master' into 6156-hostnames-in-nodes

9 years ago6234: admin user can see other users' home projects; also add link to users' home...
radhika [Wed, 17 Jun 2015 21:33:46 +0000 (17:33 -0400)]
6234: admin user can see other users' home projects; also add link to users' home projects in /users page rows.

9 years ago6277: Simplify manifest-building loop, fix up truncation tests.
Tom Clegg [Wed, 17 Jun 2015 20:40:29 +0000 (16:40 -0400)]
6277: Simplify manifest-building loop, fix up truncation tests.

9 years ago6234: In /users page, display the "Show" button to admin users as well; however,...
radhika [Wed, 17 Jun 2015 18:11:27 +0000 (14:11 -0400)]
6234: In /users page, display the "Show" button to admin users as well; however, ensure redirection to
projects/{user-uuid} takes user to user's home project while /users/{user-uuid} takes to user's page.
Also, move should correctly display user's home project.

9 years agoMerge branch '6272-index-eof' closes #6272
Tom Clegg [Wed, 17 Jun 2015 02:30:03 +0000 (22:30 -0400)]
Merge branch '6272-index-eof' closes #6272

9 years ago6272: Add blank line to indicate index response EOF. Error out of data manager if...
Tom Clegg [Tue, 16 Jun 2015 20:14:58 +0000 (16:14 -0400)]
6272: Add blank line to indicate index response EOF. Error out of data manager if not received.

9 years agoDo not blow up if HOME is not set. refs #2256
Tom Clegg [Tue, 16 Jun 2015 22:17:03 +0000 (18:17 -0400)]
Do not blow up if HOME is not set. refs #2256

9 years ago3198: Add enable_write flag to FUSE and --enable-write and --read-only to
Peter Amstutz [Tue, 16 Jun 2015 19:35:31 +0000 (15:35 -0400)]
3198: Add enable_write flag to FUSE and --enable-write and --read-only to
arv-mount.  Update documentation.  Code cleanup in locators_and_ranges() and
set_state().

9 years agoMerge branch '6222-precompile-regexps' refs #6222
Tom Clegg [Tue, 16 Jun 2015 20:09:04 +0000 (16:09 -0400)]
Merge branch '6222-precompile-regexps' refs #6222

9 years ago6222: Precompile all regexps. Remove wasted effort in GetBlockHandler.
Tom Clegg [Sat, 6 Jun 2015 20:29:48 +0000 (16:29 -0400)]
6222: Precompile all regexps. Remove wasted effort in GetBlockHandler.

9 years ago6156: support config format for setting a node's hostname
radhika [Tue, 16 Jun 2015 15:32:55 +0000 (11:32 -0400)]
6156: support config format for setting a node's hostname

9 years ago6277: Add check_manifest_validity before_filter in collection model; however, at...
radhika [Mon, 15 Jun 2015 19:02:23 +0000 (15:02 -0400)]
6277: Add check_manifest_validity before_filter in collection model; however, at the moment, this method always true after logging the validation error.

9 years agorefs #6277
radhika [Mon, 15 Jun 2015 15:34:58 +0000 (11:34 -0400)]
refs #6277
Merge branch '6277-manifest-validation'

9 years agoMerge branch 'master' into 6277-manifest-validation
radhika [Mon, 15 Jun 2015 15:34:37 +0000 (11:34 -0400)]
Merge branch 'master' into 6277-manifest-validation

9 years ago6277: all that work and missed the basic nil and empty string check!!!
radhika [Mon, 15 Jun 2015 15:33:45 +0000 (11:33 -0400)]
6277: all that work and missed the basic nil and empty string check!!!

9 years ago3198: Performance tuning based on benchmarking. Limit number of segments to
Peter Amstutz [Mon, 15 Jun 2015 14:49:00 +0000 (10:49 -0400)]
3198: Performance tuning based on benchmarking.  Limit number of segments to
look ahead for prefetch, don't prefetch blocks over and over again.

9 years agocloses #6254
radhika [Mon, 15 Jun 2015 14:45:21 +0000 (10:45 -0400)]
closes #6254
Merge branch '6254-groups-during-setup'

9 years agorefs #6277 : ruby sdk with manifest validation method
radhika [Sun, 14 Jun 2015 01:31:29 +0000 (21:31 -0400)]
refs #6277 : ruby sdk with manifest validation method
Merge branch '6277-manifest-validation'

9 years ago6277: extra white space
radhika [Sun, 14 Jun 2015 01:21:39 +0000 (21:21 -0400)]
6277: extra white space

9 years agoMerge branch 'master' into 6277-manifest-validation
radhika [Sun, 14 Jun 2015 00:47:04 +0000 (20:47 -0400)]
Merge branch 'master' into 6277-manifest-validation

9 years ago6254: minor update to Groups text field label to avoid conflict with 'Virtual Machine...
radhika [Sat, 13 Jun 2015 02:24:49 +0000 (22:24 -0400)]
6254: minor update to Groups text field label to avoid conflict with 'Virtual Machine' lebel.

9 years ago6254: better groups text field label
radhika [Sat, 13 Jun 2015 01:19:35 +0000 (21:19 -0400)]
6254: better groups text field label

9 years ago6254: instead of tooltip, use a self explanatory label for groups text field.
radhika [Sat, 13 Jun 2015 01:06:23 +0000 (21:06 -0400)]
6254: instead of tooltip, use a self explanatory label for groups text field.

9 years ago7254: add groups to vm link.
radhika [Sat, 13 Jun 2015 00:26:28 +0000 (20:26 -0400)]
7254: add groups to vm link.

9 years ago3198: Fix frontrunning (subsequent updates after a commit causing the
Peter Amstutz [Fri, 12 Jun 2015 19:54:51 +0000 (15:54 -0400)]
3198: Fix frontrunning (subsequent updates after a commit causing the
collection to conflict with itself).  Also fix filename character encoding.

9 years ago6254: remove redundant statement
radhika [Fri, 12 Jun 2015 18:30:20 +0000 (14:30 -0400)]
6254: remove redundant statement

9 years ago6254: add groups and verify in test.
radhika [Fri, 12 Jun 2015 18:26:52 +0000 (14:26 -0400)]
6254: add groups and verify in test.

9 years agoMerge branch '6149-quiet-egg-info-stderr-wip'
Brett Smith [Fri, 12 Jun 2015 18:05:19 +0000 (14:05 -0400)]
Merge branch '6149-quiet-egg-info-stderr-wip'

Closes #6149, #6284.

9 years ago6149: crunch-job installer handles egg_info errors better.
Brett Smith [Thu, 11 Jun 2015 21:16:03 +0000 (17:16 -0400)]
6149: crunch-job installer handles egg_info errors better.

Capture the command's stderr and confirm that the error refers to
git.  If it does, ignore the stderr and set a build tag.  Otherwise,
propagate stderr and abort.

9 years ago6254: slight performance improvement where the can_login link is not retrieved and...
radhika [Fri, 12 Jun 2015 15:02:13 +0000 (11:02 -0400)]
6254: slight performance improvement where the can_login link is not retrieved and checked if the groups passed in are the same as those already saved.

9 years ago6254: add "groups" to user setup process; these comma separated groups entered in...
radhika [Fri, 12 Jun 2015 14:51:26 +0000 (10:51 -0400)]
6254: add "groups" to user setup process; these comma separated groups entered in the popup will be saved as an array of groups property for the user's can_login link.

9 years ago3198: Fix tests for change in ~conflict~ format. Tweak comments.
Peter Amstutz [Fri, 12 Jun 2015 14:33:39 +0000 (10:33 -0400)]
3198: Fix tests for change in ~conflict~ format.  Tweak comments.

9 years ago3198: Additional fixes based on review feedback.
Peter Amstutz [Thu, 11 Jun 2015 19:22:32 +0000 (15:22 -0400)]
3198: Additional fixes based on review feedback.

* Adjust how ~conflict~ files are renamed
* Add finalize() on fuse inodes() to terminate put and get threads.
* Improve naming of some test case helpers
* Relax regex match on authorization token

9 years ago3198: arvfile.py and collection.py updates based on feedback.
Peter Amstutz [Thu, 11 Jun 2015 17:12:21 +0000 (13:12 -0400)]
3198: arvfile.py and collection.py updates based on feedback.

* Add retryable ERROR state to buffer blocks.
* Fix race conditions in testing a bufferblock state before trying to change state.
* Change "modified" to "committed" (which has the opposite meaning, but is more
accurate the way the flag is used)
* Refactor block manager background upload/download threads
* Fix Collection.mkdirs() to match behavior of os.makedirs()

9 years ago6277: Catch whitespace errors, and "." and ".." in paths. Rename valid? to validate!.
Tom Clegg [Thu, 11 Jun 2015 05:58:49 +0000 (01:58 -0400)]
6277: Catch whitespace errors, and "." and ".." in paths. Rename valid? to validate!.

9 years ago6277: Add more restrictions to manifest format such as cannot start with '/' and...
radhika [Fri, 12 Jun 2015 00:13:45 +0000 (20:13 -0400)]
6277: Add more restrictions to manifest format such as cannot start with '/' and end with '/' for file names,
as well as stream names and files names should not contain '//'.
Added the tests provided by Tom during review.

9 years ago6277: valid manifest must end with new line.
radhika [Thu, 11 Jun 2015 20:19:05 +0000 (16:19 -0400)]
6277: valid manifest must end with new line.

9 years ago6277: improve error message for missing file tokens.
radhika [Thu, 11 Jun 2015 20:11:28 +0000 (16:11 -0400)]
6277: improve error message for missing file tokens.