arvados.git
8 years agoMerge branch '8231-publish-arvbox' closes #8231
Peter Amstutz [Wed, 2 Mar 2016 15:20:56 +0000 (10:20 -0500)]
Merge branch '8231-publish-arvbox' closes #8231

8 years ago8231: Convert arvbox documentation to textile. Update root Arvados README to
Peter Amstutz [Wed, 2 Mar 2016 15:10:03 +0000 (10:10 -0500)]
8231: Convert arvbox documentation to textile.  Update root Arvados README to
mention Arvbox.

8 years ago8231: Fix nginx logging issue
Peter Amstutz [Wed, 2 Mar 2016 14:38:56 +0000 (09:38 -0500)]
8231: Fix nginx logging issue

8 years ago8373: Expand crunch-job regexp to detect more Keep temporary failures.
Brett Smith [Tue, 1 Mar 2016 22:08:01 +0000 (17:08 -0500)]
8373: Expand crunch-job regexp to detect more Keep temporary failures.

Closes #8373.

8 years ago8231: Move arvbox README into install guide.
Peter Amstutz [Tue, 1 Mar 2016 21:18:03 +0000 (16:18 -0500)]
8231: Move arvbox README into install guide.

8 years agoMerge branch '8015-crunch2-mount' closes #8015
Peter Amstutz [Tue, 1 Mar 2016 20:56:37 +0000 (15:56 -0500)]
Merge branch '8015-crunch2-mount' closes #8015

8 years agoAdd 'tools/arvbox/' from commit 'd3d368758db1f4a9fa5b89f77b5ee61d68ef5b72'
Peter Amstutz [Tue, 1 Mar 2016 20:48:14 +0000 (15:48 -0500)]
Add 'tools/arvbox/' from commit 'd3d368758db1f4a9fa5b89f77b5ee61d68ef5b72'

git-subtree-dir: tools/arvbox
git-subtree-mainline: d77bf0c67422a259afacc17660698729328a1ed3
git-subtree-split: d3d368758db1f4a9fa5b89f77b5ee61d68ef5b72

8 years ago8015: Fix typos in error reporting. Fix tests.
Peter Amstutz [Tue, 1 Mar 2016 20:34:15 +0000 (15:34 -0500)]
8015: Fix typos in error reporting.  Fix tests.

8 years agoMerge branch '6605-arv-copy-http' closes #6605
Peter Amstutz [Tue, 1 Mar 2016 18:15:11 +0000 (13:15 -0500)]
Merge branch '6605-arv-copy-http' closes #6605

8 years ago6605: Add --allow-git-http-src and --allow-git-http-dst instead of looking at ARVADOS...
Peter Amstutz [Mon, 29 Feb 2016 21:42:12 +0000 (16:42 -0500)]
6605: Add --allow-git-http-src and --allow-git-http-dst instead of looking at ARVADOS_API_HOST_INSECURE.

8 years agoWorkaround for https://bugs.python.org/issue7980. No issue #
Tom Clegg [Mon, 29 Feb 2016 20:40:14 +0000 (15:40 -0500)]
Workaround for https://bugs.python.org/issue7980. No issue #

8 years agoDocs: Fix typo in run-command foreach example.
Brett Smith [Mon, 29 Feb 2016 18:21:35 +0000 (13:21 -0500)]
Docs: Fix typo in run-command foreach example.

Reported in IRC.  No issue #.

8 years agoMerge branch '8437-nodemanager-on-failure' closes #8437
Peter Amstutz [Mon, 29 Feb 2016 15:33:48 +0000 (10:33 -0500)]
Merge branch '8437-nodemanager-on-failure' closes #8437

8 years ago8437: Test all exceptions that should be caught.
Peter Amstutz [Mon, 29 Feb 2016 15:33:27 +0000 (10:33 -0500)]
8437: Test all exceptions that should be caught.

8 years agoMerge branch '8416-node-manager-find-node' closes #8416
Peter Amstutz [Mon, 29 Feb 2016 14:40:50 +0000 (09:40 -0500)]
Merge branch '8416-node-manager-find-node' closes #8416

8 years ago8416: Don't catch exception from search_for, potentially hides exceptions we
Peter Amstutz [Mon, 29 Feb 2016 14:40:10 +0000 (09:40 -0500)]
8416: Don't catch exception from search_for, potentially hides exceptions we
wanted to see and the original intended behavior of re-raising the original
exception doesn't work anyway, because sys.exc_info is overwritten when the 2nd
exception is caught.

8 years ago8437: Add missing test.
Peter Amstutz [Fri, 26 Feb 2016 16:38:50 +0000 (11:38 -0500)]
8437: Add missing test.

8 years agoTweak slurm configuration to run as non-root user.
Peter Amstutz [Thu, 25 Feb 2016 16:15:47 +0000 (11:15 -0500)]
Tweak slurm configuration to run as non-root user.

8 years agoFix arvados-login-sync for vm account creation
Peter Amstutz [Thu, 25 Feb 2016 16:14:21 +0000 (11:14 -0500)]
Fix arvados-login-sync for vm account creation

8 years agoGeneralize arvbox 'svrestart' to 'sv' no issue #
Peter Amstutz [Thu, 25 Feb 2016 16:12:00 +0000 (11:12 -0500)]
Generalize arvbox 'svrestart' to 'sv' no issue #

8 years ago8437: Add FullStopActor which uses os.killpg() to terminate node manager on_failure...
Peter Amstutz [Thu, 25 Feb 2016 14:08:01 +0000 (09:08 -0500)]
8437: Add FullStopActor which uses os.killpg() to terminate node manager on_failure.  Added test.

8 years agocloses #5694
radhika [Thu, 25 Feb 2016 02:53:14 +0000 (21:53 -0500)]
closes #5694
Merge branch '5694-running-job-log'

8 years ago5694: remove the extra white space in log tab display.
radhika [Thu, 25 Feb 2016 02:52:23 +0000 (21:52 -0500)]
5694: remove the extra white space in log tab display.

8 years ago8416: Remove find_node and use existing search_for feature.
Peter Amstutz [Tue, 9 Feb 2016 16:36:37 +0000 (11:36 -0500)]
8416: Remove find_node and use existing search_for feature.

8 years agoMerge branch 'master' into 5694-running-job-log
radhika [Wed, 24 Feb 2016 14:34:39 +0000 (09:34 -0500)]
Merge branch 'master' into 5694-running-job-log

8 years ago6605: Use git -c instead of "git config" to set credentials. Refuse to use
Peter Amstutz [Tue, 23 Feb 2016 20:18:13 +0000 (15:18 -0500)]
6605: Use git -c instead of "git config" to set credentials.  Refuse to use
insecure http unless ARVADOS_API_HOST_INSECURE is also true.

8 years ago5694: display 2000 historic log lines in running job's Log tab.
radhika [Mon, 22 Feb 2016 14:30:00 +0000 (09:30 -0500)]
5694: display 2000 historic log lines in running job's Log tab.

8 years agoMerge branch '8484-sanity-check-collection-count' closes #8484
Tom Clegg [Fri, 19 Feb 2016 22:44:52 +0000 (17:44 -0500)]
Merge branch '8484-sanity-check-collection-count' closes #8484

8 years ago8484: Fix fixtures with modified_at=NULL
Tom Clegg [Fri, 19 Feb 2016 22:43:21 +0000 (17:43 -0500)]
8484: Fix fixtures with modified_at=NULL

8 years agoadds missing var collections SdkCollectionList
Joshua C. Randall [Fri, 19 Feb 2016 22:09:35 +0000 (22:09 +0000)]
adds missing var collections SdkCollectionList

8 years agoMerge branch '8485-datamanager-identical-modifiedat' of https://github.com/wtsi-hgi...
Tom Clegg [Fri, 19 Feb 2016 21:45:18 +0000 (16:45 -0500)]
Merge branch '8485-datamanager-identical-modifiedat' of https://github.com/wtsi-hgi/arvados closes #8485

8 years agopackage building is failing because of bad version pinning
Nico Cesar [Fri, 19 Feb 2016 21:38:01 +0000 (16:38 -0500)]
package building is failing because of bad version pinning

see https://ci.curoverse.com/job/build-packages/1733

no issue #

8 years agoPin google-api-python-client and oauth2client versions to deal with
Peter Amstutz [Fri, 19 Feb 2016 15:49:18 +0000 (10:49 -0500)]
Pin google-api-python-client and oauth2client versions to deal with
incompatability between google api client and oauth2 client 2.0.  no issue #

8 years agoModifies sanity check to query for the total expected number of collections
Joshua C. Randall [Fri, 19 Feb 2016 13:42:25 +0000 (13:42 +0000)]
Modifies sanity check to query for the total expected number of collections

8 years agoMakes changes suggested by tomclegg in review of PR #39
Joshua C. Randall [Fri, 19 Feb 2016 12:27:08 +0000 (12:27 +0000)]
Makes changes suggested by tomclegg in review of PR #39

8 years agoRun slurmctld and slurmd inside arvbox. refs #6518
Peter Amstutz [Thu, 18 Feb 2016 21:28:29 +0000 (16:28 -0500)]
Run slurmctld and slurmd inside arvbox. refs #6518

8 years agocloses #8441
radhika [Thu, 18 Feb 2016 19:24:08 +0000 (14:24 -0500)]
closes #8441
Merge branch '8441-project-chooser'

8 years agoMerge branch 'master' into 8441-project-chooser
radhika [Thu, 18 Feb 2016 19:23:09 +0000 (14:23 -0500)]
Merge branch 'master' into 8441-project-chooser

8 years agoMerge branch '8400-additional-gitignore' of https://github.com/wtsi-hgi/arvados close...
Tom Clegg [Thu, 18 Feb 2016 18:57:59 +0000 (13:57 -0500)]
Merge branch '8400-additional-gitignore' of https://github.com/wtsi-hgi/arvados closes #8400

8 years ago6605: Check to see if each git URL works by using git ls-remote.
Peter Amstutz [Thu, 18 Feb 2016 18:42:40 +0000 (13:42 -0500)]
6605: Check to see if each git URL works by using git ls-remote.
Preferentially use https.  Set GIT_ASKPASS=/bin/false to suppress asking for
credentials.

8 years ago8441: Update project chooser modal to display favorites and top-level-my-projects...
radhika [Thu, 18 Feb 2016 17:55:25 +0000 (12:55 -0500)]
8441: Update project chooser modal to display favorites and top-level-my-projects and remove the now unused build_project_trees and shared_project_tree code.

8 years agoChanges GetCollection loop to more reliably fetch all collections
Joshua C. Randall [Thu, 18 Feb 2016 14:36:53 +0000 (14:36 +0000)]
Changes GetCollection loop to more reliably fetch all collections

Modifies the loop termination condition in GetCollection so that
it continues until there are no more items available according to
the API server (returned in collections.ItemsAvailable).

Modifies the query code so that it uses an offset to page through
results in case an entire batch has equal modified_at timestamps.

8 years agoMerge branch '8319-bcbio-cwl' closes #8319
Peter Amstutz [Thu, 18 Feb 2016 14:36:45 +0000 (09:36 -0500)]
Merge branch '8319-bcbio-cwl' closes #8319

8 years agoAdds sanity check on number of collections retrieved
Joshua C. Randall [Thu, 18 Feb 2016 00:02:06 +0000 (00:02 +0000)]
Adds sanity check on number of collections retrieved

Changes GetCollections() to return an error if the number
of collections retrieved is less than the initial number
of collections.

8 years ago8319: Add log message about uploading docker image
Peter Amstutz [Wed, 17 Feb 2016 17:59:57 +0000 (12:59 -0500)]
8319: Add log message about uploading docker image

8 years agocloses #8286
radhika [Wed, 17 Feb 2016 14:32:09 +0000 (09:32 -0500)]
closes #8286
Merge branch '8286-fav-projects'

8 years ago8286: bigger star icon with stronger color contrast.
radhika [Wed, 17 Feb 2016 14:30:58 +0000 (09:30 -0500)]
8286: bigger star icon with stronger color contrast.

8 years ago8319: Add environment variable to disable websockets.
Peter Amstutz [Wed, 17 Feb 2016 14:21:09 +0000 (09:21 -0500)]
8319: Add environment variable to disable websockets.

8 years ago8015: Fix tests again for endianness of Docker logs, added --read-write to arv-mount...
Peter Amstutz [Tue, 16 Feb 2016 18:28:20 +0000 (13:28 -0500)]
8015: Fix tests again for endianness of Docker logs, added --read-write to arv-mount command line.

8 years ago8286: also include link to Home in favorites section of the Projects dropdown, so...
radhika [Tue, 16 Feb 2016 17:00:09 +0000 (12:00 -0500)]
8286: also include link to Home in favorites section of the Projects dropdown, so that the user does not have to scroll through all of his favorites to seek in MyProjects section.

8 years ago8015: Don't use Docker logging driver. Attach stdout/stderr before starting container.
Peter Amstutz [Tue, 16 Feb 2016 15:29:12 +0000 (10:29 -0500)]
8015: Don't use Docker logging driver.  Attach stdout/stderr before starting container.

8 years agoMove services to /var/lib/arvbox/services because they include logs and logs
Peter Amstutz [Tue, 16 Feb 2016 15:16:55 +0000 (10:16 -0500)]
Move services to /var/lib/arvbox/services because they include logs and logs
don't go in /usr. no issue #

8 years agoMerge branch 'master' into 8286-fav-projects
radhika [Tue, 16 Feb 2016 02:33:01 +0000 (21:33 -0500)]
Merge branch 'master' into 8286-fav-projects

8 years agocloses #8079
radhika [Tue, 16 Feb 2016 02:32:00 +0000 (21:32 -0500)]
closes #8079
Merge branch '8079-api-client-auth-uuid'

8 years agoMerge branch 'master' into 8079-api-client-auth-uuid
radhika [Tue, 16 Feb 2016 01:46:05 +0000 (20:46 -0500)]
Merge branch 'master' into 8079-api-client-auth-uuid

8 years ago8286: check if project is starred only when current_user is not null (anonymous user...
radhika [Tue, 16 Feb 2016 01:44:59 +0000 (20:44 -0500)]
8286: check if project is starred only when current_user is not null (anonymous user case).

8 years ago8286: to facilitate in-place star icon refresh without the whole page refresh, it...
radhika [Mon, 15 Feb 2016 22:28:55 +0000 (17:28 -0500)]
8286: to facilitate in-place star icon refresh without the whole page refresh, it became necessary
to refresh the Projects menu after star/unstar action. Hence, moved the breadcrumbs code from
body.html into a partial.

8 years ago8015: check error when deleting temporary directory.
Peter Amstutz [Mon, 15 Feb 2016 21:58:34 +0000 (16:58 -0500)]
8015: check error when deleting temporary directory.

8 years agoExplicitly install Docker 1.9.1 instead of using get.dockerproject.org (which gets...
Peter Amstutz [Mon, 15 Feb 2016 21:37:00 +0000 (16:37 -0500)]
Explicitly install Docker 1.9.1 instead of using get.dockerproject.org (which gets the latest). no issue #

8 years agoAdd crunch-dispatch-local to arvbox. no issue #
Peter Amstutz [Sun, 14 Feb 2016 04:18:27 +0000 (23:18 -0500)]
Add crunch-dispatch-local to arvbox.  no issue #

8 years ago8286: convert star method into action controller action and refresh the star icon...
radhika [Mon, 15 Feb 2016 20:26:05 +0000 (15:26 -0500)]
8286: convert star method into action controller action and refresh the star icon in place rather than a full page refresh.

8 years agoReport when container runner finishes.
Peter Amstutz [Mon, 15 Feb 2016 20:20:20 +0000 (15:20 -0500)]
Report when container runner finishes.

8 years ago8015: Handle Docker logging correctly, interpret frame headers and
Peter Amstutz [Mon, 15 Feb 2016 20:20:04 +0000 (15:20 -0500)]
8015: Handle Docker logging correctly, interpret frame headers and
demultiplex stdout/stderr.

8 years agoMerge branch 'master' into 8286-fav-projects
radhika [Mon, 15 Feb 2016 18:50:48 +0000 (13:50 -0500)]
Merge branch 'master' into 8286-fav-projects

8 years agocloses #8183
radhika [Mon, 15 Feb 2016 18:43:08 +0000 (13:43 -0500)]
closes #8183
Merge branch '8183-projects-dropdown'

8 years agoMerge branch 'master' into 8183-projects-dropdown
radhika [Mon, 15 Feb 2016 18:42:44 +0000 (13:42 -0500)]
Merge branch 'master' into 8183-projects-dropdown

8 years ago8286: include favorites and top-level my projects in projects drowdown.
radhika [Mon, 15 Feb 2016 18:33:43 +0000 (13:33 -0500)]
8286: include favorites and top-level my projects in projects drowdown.

8 years ago8409: Use 80% utilization as keep_cache_mb_per_task reporting threshold. refs #8409
Tom Clegg [Mon, 15 Feb 2016 18:22:03 +0000 (13:22 -0500)]
8409: Use 80% utilization as keep_cache_mb_per_task reporting threshold. refs #8409

8 years ago8079: add rescue to drop index
radhika [Mon, 15 Feb 2016 17:47:22 +0000 (12:47 -0500)]
8079: add rescue to drop index

8 years agoMerge branch '8183-projects-dropdown' into 8286-fav-projects
radhika [Mon, 15 Feb 2016 17:35:02 +0000 (12:35 -0500)]
Merge branch '8183-projects-dropdown' into 8286-fav-projects

Conflicts:
apps/workbench/app/controllers/application_controller.rb
apps/workbench/app/views/application/_projects_tree_menu.html.erb
apps/workbench/test/controllers/projects_controller_test.rb

8 years agoMerge branch 'master' into 8183-projects-dropdown
radhika [Mon, 15 Feb 2016 16:29:04 +0000 (11:29 -0500)]
Merge branch 'master' into 8183-projects-dropdown

8 years agoMerge branch '8178-trash-interface-generic-volume-test' closes #8178
Tom Clegg [Mon, 15 Feb 2016 15:49:39 +0000 (10:49 -0500)]
Merge branch '8178-trash-interface-generic-volume-test' closes #8178

8 years ago8178: Stop accepting zeroed data, now that the s3test bug is fixed.
Tom Clegg [Thu, 4 Feb 2016 23:28:43 +0000 (18:28 -0500)]
8178: Stop accepting zeroed data, now that the s3test bug is fixed.

8 years ago8178: add generic volume tests for trash / untrash interface.
radhika [Wed, 27 Jan 2016 04:31:27 +0000 (23:31 -0500)]
8178: add generic volume tests for trash / untrash interface.

8 years agoMerge branch 'master' into 8079-api-client-auth-uuid
radhika [Mon, 15 Feb 2016 13:40:54 +0000 (08:40 -0500)]
Merge branch 'master' into 8079-api-client-auth-uuid

8 years ago8183: display top three levels of projects in menu, improve message when there are...
radhika [Mon, 15 Feb 2016 13:36:47 +0000 (08:36 -0500)]
8183: display top three levels of projects in menu, improve message when there are too many projects, improve the test.

8 years ago8015: Report container UUID in error message. Uploader fix for empty & zero-length...
Peter Amstutz [Mon, 15 Feb 2016 04:17:58 +0000 (23:17 -0500)]
8015: Report container UUID in error message.  Uploader fix for empty & zero-length streams.

8 years ago8015: Expand arv-mount command line tests
Peter Amstutz [Mon, 15 Feb 2016 03:58:00 +0000 (22:58 -0500)]
8015: Expand arv-mount command line tests

8 years ago8015: Add arv-mount command line test
Peter Amstutz [Mon, 15 Feb 2016 03:27:24 +0000 (22:27 -0500)]
8015: Add arv-mount command line test

8 years ago8015: Improve handling for arv-mount start, arv-mount exit.
Peter Amstutz [Mon, 15 Feb 2016 02:37:14 +0000 (21:37 -0500)]
8015: Improve handling for arv-mount start, arv-mount exit.

8 years ago8015: Fix tests
Peter Amstutz [Mon, 15 Feb 2016 00:59:04 +0000 (19:59 -0500)]
8015: Fix tests

8 years agoProcess live logs for unfinished jobs in pipeline mode, too.
Tom Clegg [Thu, 11 Feb 2016 21:25:54 +0000 (16:25 -0500)]
Process live logs for unfinished jobs in pipeline mode, too.

No issue #

8 years ago8015: Fix compile, need to fix tests.
Peter Amstutz [Sun, 14 Feb 2016 05:32:34 +0000 (00:32 -0500)]
8015: Fix compile, need to fix tests.

8 years ago8015: Fix create and update API calls to use correct nested dict.
Peter Amstutz [Sun, 14 Feb 2016 05:26:05 +0000 (00:26 -0500)]
8015: Fix create and update API calls to use correct nested dict.

8 years ago8015: Test WriteTree
Peter Amstutz [Sun, 14 Feb 2016 03:51:05 +0000 (22:51 -0500)]
8015: Test WriteTree

8 years ago8015: Fix existing tests, still need tests for new functionality
Peter Amstutz [Sun, 14 Feb 2016 02:49:55 +0000 (21:49 -0500)]
8015: Fix existing tests, still need tests for new functionality

8 years ago8183: retrieve only 3 levels of projects while building projects dropdown.
radhika [Sat, 13 Feb 2016 01:57:25 +0000 (20:57 -0500)]
8183: retrieve only 3 levels of projects while building projects dropdown.

8 years ago8015: Fix compilation
Peter Amstutz [Fri, 12 Feb 2016 22:49:28 +0000 (17:49 -0500)]
8015: Fix compilation

8 years ago8203: crunch-job tempfails after failing to install a Docker image.
Brett Smith [Fri, 12 Feb 2016 23:38:36 +0000 (18:38 -0500)]
8203: crunch-job tempfails after failing to install a Docker image.

* Use `bash -o pipefail` to run the image installer shell script, to
  reliably detect more failures.
* Exit EX_RETRY_UNLOCKED after a failure to install the image.

Refs #8203, because I'm cautiously optimistic that this will reduce
the incidence of the "can't find UID" problem.

8 years ago8015: Finished initial implemention of input and output mounts. Needs tests.
Peter Amstutz [Fri, 12 Feb 2016 22:38:58 +0000 (17:38 -0500)]
8015: Finished initial implemention of input and output mounts.  Needs tests.

8 years ago8183: while displaying "my projects" tree, just show only the user's projects and...
radhika [Fri, 12 Feb 2016 21:50:34 +0000 (16:50 -0500)]
8183: while displaying "my projects" tree, just show only the user's projects and omit shared projects
even when there are no more than 300 projects to avoid confusion.

8 years agoMerge branch 'master' into 8183-projects-dropdown
radhika [Fri, 12 Feb 2016 21:24:41 +0000 (16:24 -0500)]
Merge branch 'master' into 8183-projects-dropdown

8 years agoAdd arvbox support for overriding settings in application.yml
Peter Amstutz [Fri, 12 Feb 2016 20:14:01 +0000 (15:14 -0500)]
Add arvbox support for overriding settings in application.yml
no issue #

8 years ago8286: add an integration test to star / unstar project by clicking on the icon.
radhika [Fri, 12 Feb 2016 19:03:57 +0000 (14:03 -0500)]
8286: add an integration test to star / unstar project by clicking on the icon.

8 years ago8015: Working on WriteTree() and setting up mounts.
Peter Amstutz [Fri, 12 Feb 2016 15:46:33 +0000 (10:46 -0500)]
8015: Working on WriteTree() and setting up mounts.

8 years ago8015: Crunch run mounting and output upload WIP
Peter Amstutz [Fri, 12 Feb 2016 05:04:20 +0000 (00:04 -0500)]
8015: Crunch run mounting and output upload WIP

8 years ago8286: added test "unshare project and verify that it is no longer included in shared...
radhika [Thu, 11 Feb 2016 19:53:31 +0000 (14:53 -0500)]
8286: added test "unshare project and verify that it is no longer included in shared user's starred projects"

8 years agoMerge branch '8409-report-keep-cache' closes #8409
Peter Amstutz [Thu, 11 Feb 2016 14:41:17 +0000 (09:41 -0500)]
Merge branch '8409-report-keep-cache' closes #8409

8 years ago8409: Adjust recommended miss rate to below 0.2%
Peter Amstutz [Thu, 11 Feb 2016 14:27:21 +0000 (09:27 -0500)]
8409: Adjust recommended miss rate to below 0.2%

8 years ago8409: Add recommendation if cache miss rate is above 0.5%. Fix tests.
Peter Amstutz [Wed, 10 Feb 2016 21:42:02 +0000 (16:42 -0500)]
8409: Add recommendation if cache miss rate is above 0.5%.  Fix tests.