arvados.git
8 years ago8095: Make install guide bundle commands more consistent. 8095-install-guide-rails-sudo-wip
Brett Smith [Thu, 31 Dec 2015 16:23:34 +0000 (11:23 -0500)]
8095: Make install guide bundle commands more consistent.

* Always instruct the user to use `sudo -u`.  This prevents the
  commands from generating cache files that the web server can't write
  later.
* Always execute the command in the server's main directory.  This
  helps people run on automatic: if they leave a shell session open at
  the API server directory, they can paste in a combination of Rails
  console and script commands without `cd`ing around.
* Always give both RVM and non-RVM instructions.  There might be a
  good way to streamline this, e.g., by telling them to use rvmsudo
  when available, but that can be a later optimization.

8 years ago8082: Ensure consistent column widths in Workbench job banner.
Brett Smith [Wed, 30 Dec 2015 23:53:12 +0000 (18:53 -0500)]
8082: Ensure consistent column widths in Workbench job banner.

4dcf4e849f242a929ce03b5d529e0e1a63fbaeb0 is the commit that introduced
the column inconsistencies.  This is the smallest patch I could find
to address the issue.  It may be less than ideal because it always
allocates a column to something that may not be displayed, but the
logic to keep track of column widths is already hairy and there are so
many conditions here that it stood to get a lot worse.  Closes #8082.

8 years ago8077: Tutorial suggests using the arvados/jobs Docker image.
Brett Smith [Wed, 30 Dec 2015 23:10:38 +0000 (18:10 -0500)]
8077: Tutorial suggests using the arvados/jobs Docker image.

This is all the hash script needs, and clusters are more likely to
have it than arvados/jobs-java-bwa-samtools, which is more of an
internal thing.  Closes #8077.

8 years agoIncrease timeout for bringing up test server.
Tom Clegg [Wed, 23 Dec 2015 20:30:41 +0000 (15:30 -0500)]
Increase timeout for bringing up test server.

No issue #

8 years agoMerge branch '7965-fail-abandoned-jobs' closes #7965
Tom Clegg [Wed, 23 Dec 2015 19:28:41 +0000 (14:28 -0500)]
Merge branch '7965-fail-abandoned-jobs' closes #7965

8 years ago7965: Fail orphaned jobs still marked "running" after reboot.
Tom Clegg [Fri, 18 Dec 2015 19:33:28 +0000 (14:33 -0500)]
7965: Fail orphaned jobs still marked "running" after reboot.

8 years agoFix documentation typo git_repo_https_base not git_repo_http_base, no issue #
Peter Amstutz [Sun, 20 Dec 2015 02:59:26 +0000 (21:59 -0500)]
Fix documentation typo git_repo_https_base not git_repo_http_base, no issue #

8 years agoMerge branch '7884-ajax-log-redirect' closes #7884
Tom Clegg [Fri, 18 Dec 2015 20:40:40 +0000 (15:40 -0500)]
Merge branch '7884-ajax-log-redirect' closes #7884

8 years ago7884: Clarify "credentials" comment.
Tom Clegg [Fri, 18 Dec 2015 20:40:11 +0000 (15:40 -0500)]
7884: Clarify "credentials" comment.

8 years ago7884: Detect when a "download log" response has a JSON-encoded redirect
Tom Clegg [Thu, 17 Dec 2015 06:07:32 +0000 (01:07 -0500)]
7884: Detect when a "download log" response has a JSON-encoded redirect
to keep-web, and convert it to POST to avoid a second redirect that
would be forbidden by SOP.

8 years ago7884: Serve simple cross-origin AJAX POST requests without redirecting.
Tom Clegg [Thu, 17 Dec 2015 06:03:37 +0000 (01:03 -0500)]
7884: Serve simple cross-origin AJAX POST requests without redirecting.

8 years agoMerge branch '7927-docker-cleaner-api-version-wip'
Brett Smith [Fri, 18 Dec 2015 18:30:10 +0000 (13:30 -0500)]
Merge branch '7927-docker-cleaner-api-version-wip'

Closes #7927, #8025.

8 years ago7927: Docker cleaner service uses Docker API v1.14.
Brett Smith [Wed, 16 Dec 2015 16:07:24 +0000 (11:07 -0500)]
7927: Docker cleaner service uses Docker API v1.14.

This helps it work consistently across a variety of Docker versions.
See the comment in the tests for more background.

8 years agoLog request body size. No issue #
Tom Clegg [Thu, 17 Dec 2015 18:50:01 +0000 (13:50 -0500)]
Log request body size. No issue #

8 years agoMerge branch '7901-human-summary' closes #7901
Tom Clegg [Thu, 17 Dec 2015 15:06:50 +0000 (10:06 -0500)]
Merge branch '7901-human-summary' closes #7901

8 years ago7901: Add job stats, elapsed time, summed user+sys and tx+rx, and some human-readable...
Tom Clegg [Thu, 17 Dec 2015 15:05:30 +0000 (10:05 -0500)]
7901: Add job stats, elapsed time, summed user+sys and tx+rx, and some human-readable highlights.

8 years ago7942: Skip unreliable test. refs #7942
Tom Clegg [Wed, 16 Dec 2015 20:34:43 +0000 (15:34 -0500)]
7942: Skip unreliable test. refs #7942

8 years agoMerge branch '7888-azure-read-mux' refs #7888
Tom Clegg [Wed, 16 Dec 2015 19:51:44 +0000 (14:51 -0500)]
Merge branch '7888-azure-read-mux' refs #7888

8 years ago7888: Fix whitespace / gofmt
Tom Clegg [Wed, 16 Dec 2015 19:51:25 +0000 (14:51 -0500)]
7888: Fix whitespace / gofmt

8 years ago7888: Option to use multiple concurrent range requests when fetching from Azure.
Tom Clegg [Wed, 16 Dec 2015 17:43:35 +0000 (12:43 -0500)]
7888: Option to use multiple concurrent range requests when fetching from Azure.

8 years agoMerge branch '7454-azure-custom-data' closes #7454
Peter Amstutz [Wed, 16 Dec 2015 15:29:51 +0000 (10:29 -0500)]
Merge branch '7454-azure-custom-data' closes #7454

8 years ago7454: switch order of arvados_create_kwargs() to be consistent with
Peter Amstutz [Wed, 16 Dec 2015 14:49:52 +0000 (09:49 -0500)]
7454: switch order of arvados_create_kwargs() to be consistent with
create_node().  Use pipes.quote() to shellquote values.  Add ex_customdata
test.

8 years agoMerge branch '6429-fix-container-indexes' refs #6429
Peter Amstutz [Tue, 15 Dec 2015 15:00:46 +0000 (10:00 -0500)]
Merge branch '6429-fix-container-indexes' refs #6429

8 years agoAdd required indexes to containers and container_requests tables. refs #6429
Peter Amstutz [Tue, 15 Dec 2015 14:56:38 +0000 (09:56 -0500)]
Add required indexes to containers and container_requests tables. refs #6429

8 years agoIncrease default Python SDK timeout to 256s.
Tom Clegg [Tue, 15 Dec 2015 14:54:37 +0000 (09:54 -0500)]
Increase default Python SDK timeout to 256s.

No issue #

8 years ago7901: Fix variable name typo in crunchstat-summary.
Brett Smith [Mon, 14 Dec 2015 22:23:22 +0000 (17:23 -0500)]
7901: Fix variable name typo in crunchstat-summary.

Refs #7901.

8 years agoMerge branch '7901-crunchstat-summary' refs #7901
Tom Clegg [Mon, 14 Dec 2015 21:56:53 +0000 (16:56 -0500)]
Merge branch '7901-crunchstat-summary' refs #7901

8 years ago7901: Add crunchstat-summary program.
Tom Clegg [Mon, 14 Dec 2015 21:07:32 +0000 (16:07 -0500)]
7901: Add crunchstat-summary program.

8 years agoCleanup of test cleanup. refs #6429
Peter Amstutz [Mon, 14 Dec 2015 20:38:58 +0000 (15:38 -0500)]
Cleanup of test cleanup.  refs #6429

8 years agoSmall test cleanup. refs #6429
Peter Amstutz [Mon, 14 Dec 2015 20:31:07 +0000 (15:31 -0500)]
Small test cleanup.  refs #6429

8 years agoMerge branch '6429-crunch2-api' closes #6429
Peter Amstutz [Mon, 14 Dec 2015 20:22:39 +0000 (15:22 -0500)]
Merge branch '6429-crunch2-api' closes #6429

8 years agoMerge branch '7939-arv-mount-errors' closes #7939
Peter Amstutz [Mon, 14 Dec 2015 20:21:59 +0000 (15:21 -0500)]
Merge branch '7939-arv-mount-errors' closes #7939

8 years ago7939: Use 100:: for bogus host check and add comment about API check.
Peter Amstutz [Mon, 14 Dec 2015 20:20:23 +0000 (15:20 -0500)]
7939: Use 100:: for bogus host check and add comment about API check.

8 years ago6429: Add for test child container requests are set priority=0 when parent
Peter Amstutz [Mon, 14 Dec 2015 19:22:43 +0000 (14:22 -0500)]
6429: Add for test child container requests are set priority=0 when parent
container is in final state.

8 years ago7939: Additional check for bad token.
Peter Amstutz [Mon, 14 Dec 2015 19:11:33 +0000 (14:11 -0500)]
7939: Additional check for bad token.

8 years ago7939: Check exit code in tests.
Peter Amstutz [Mon, 14 Dec 2015 19:09:20 +0000 (14:09 -0500)]
7939: Check exit code in tests.

8 years ago7939: Move daemonize to _run_standalone() after llfuse.init(), preserving open
Peter Amstutz [Fri, 11 Dec 2015 19:49:01 +0000 (14:49 -0500)]
7939: Move daemonize to _run_standalone() after llfuse.init(), preserving open
file descriptors.  Add tests for various error conditions that can prevent
successful mount.

8 years ago7454: Fix tests
Peter Amstutz [Fri, 11 Dec 2015 20:43:44 +0000 (15:43 -0500)]
7454: Fix tests

8 years ago7454: Add 'size' to arvados_create_kwargs so that it can be included in CustomData.
Peter Amstutz [Fri, 11 Dec 2015 20:37:55 +0000 (15:37 -0500)]
7454: Add 'size' to arvados_create_kwargs so that it can be included in CustomData.

8 years ago7454: use small CustomData script to set /var/tmp/arv-node-data/arv-ping-url
Peter Amstutz [Fri, 11 Dec 2015 20:13:56 +0000 (15:13 -0500)]
7454: use small CustomData script to set /var/tmp/arv-node-data/arv-ping-url

8 years agoMerge branch 'master' into 7454-azure-custom-data
Peter Amstutz [Fri, 11 Dec 2015 19:59:02 +0000 (14:59 -0500)]
Merge branch 'master' into 7454-azure-custom-data

8 years ago6429: Callback calls container_completed! instead of just setting state to final.
Peter Amstutz [Fri, 11 Dec 2015 15:24:06 +0000 (10:24 -0500)]
6429: Callback calls container_completed! instead of just setting state to final.

8 years ago6429: Rename request_finalize to handle_completed
Peter Amstutz [Thu, 10 Dec 2015 21:45:41 +0000 (16:45 -0500)]
6429: Rename request_finalize to handle_completed

8 years ago6429: Improve variable names in tests a bit. Default priority of container
Peter Amstutz [Thu, 10 Dec 2015 21:37:29 +0000 (16:37 -0500)]
6429: Improve variable names in tests a bit.  Default priority of container
request to nil and check for nil priority.  Complete/cancelled container
finalizes requests that created it and sets priority 0 on requests that it
created itself.

8 years agoMerge branch '7393-s3-volume' closes #7393
Tom Clegg [Thu, 10 Dec 2015 21:12:20 +0000 (16:12 -0500)]
Merge branch '7393-s3-volume' closes #7393

8 years ago7393: Add -uuid and -url options, fix memory sharing in -vary-request.
Tom Clegg [Wed, 9 Dec 2015 21:00:34 +0000 (16:00 -0500)]
7393: Add -uuid and -url options, fix memory sharing in -vary-request.

8 years ago7393: Quiet excessive debug printfs.
Tom Clegg [Wed, 9 Dec 2015 20:59:54 +0000 (15:59 -0500)]
7393: Quiet excessive debug printfs.

8 years ago7393: Add S3 volume type.
Tom Clegg [Wed, 9 Dec 2015 17:43:50 +0000 (12:43 -0500)]
7393: Add S3 volume type.

8 years ago7960: Fix typo in arv-run option reference.
Brett Smith [Thu, 10 Dec 2015 20:46:24 +0000 (15:46 -0500)]
7960: Fix typo in arv-run option reference.

Closes #7960.

8 years ago6429: Don't update priority if the container is already in Complete or Cancelled...
Peter Amstutz [Wed, 9 Dec 2015 21:02:53 +0000 (16:02 -0500)]
6429: Don't update priority if the container is already in Complete or Cancelled state.

8 years ago6429: Complete/Cancelled containers finalize associated container requests;
Peter Amstutz [Wed, 9 Dec 2015 20:48:37 +0000 (15:48 -0500)]
6429: Complete/Cancelled containers finalize associated container requests;
container requests update priority for associated containers.

8 years ago6429: Use text instead of string for longer API fields.
Peter Amstutz [Wed, 9 Dec 2015 20:06:46 +0000 (15:06 -0500)]
6429: Use text instead of string for longer API fields.
Move "resolve" from Container to ContainerRequest.
Simplify state transition checking.
Improve formatting of error messages.
Rename workbench models to be singular.
Remove bogus factory files.

8 years ago6429: Tests for priority update propagation to process tree, max priority from
Peter Amstutz [Tue, 8 Dec 2015 16:57:52 +0000 (11:57 -0500)]
6429: Tests for priority update propagation to process tree, max priority from
multiple requests, finalize request when container completes or cancels.

8 years ago7937: Update test case.
Tom Clegg [Tue, 8 Dec 2015 01:50:57 +0000 (20:50 -0500)]
7937: Update test case.

refs #7937

8 years agoMerge branch '7937-ignored-error' refs #7937
Tom Clegg [Mon, 7 Dec 2015 22:31:06 +0000 (17:31 -0500)]
Merge branch '7937-ignored-error' refs #7937

8 years ago7937: Fix ignored error from GetKeepServersAndSummarize.
Tom Clegg [Sat, 5 Dec 2015 08:47:04 +0000 (03:47 -0500)]
7937: Fix ignored error from GetKeepServersAndSummarize.

8 years ago6429: Committing container request creates container
Peter Amstutz [Mon, 7 Dec 2015 21:27:42 +0000 (16:27 -0500)]
6429: Committing container request creates container

8 years ago7829: Add MANIFEST.in for arvados-docker-cleaner.
Brett Smith [Mon, 7 Dec 2015 18:58:17 +0000 (13:58 -0500)]
7829: Add MANIFEST.in for arvados-docker-cleaner.

Refs #7829.

8 years ago7829: Restore MANIFEST.in files; add licenses to them.
Brett Smith [Mon, 7 Dec 2015 18:05:44 +0000 (13:05 -0500)]
7829: Restore MANIFEST.in files; add licenses to them.

This reverts cd78cc7c9d06ec84f1e272c4b5a15d377830edba.  The package
building process works without these files on Debian 8, but not Debian
7, which is what Jenkins is still running.

Refs #7829.

8 years ago6429: Put whitelist_update in module.
Peter Amstutz [Mon, 7 Dec 2015 17:57:57 +0000 (12:57 -0500)]
6429: Put whitelist_update in module.

8 years agoMerge branch '7829-package-licenses-wip'
Brett Smith [Mon, 7 Dec 2015 15:01:26 +0000 (10:01 -0500)]
Merge branch '7829-package-licenses-wip'

Refs #7829.  Closes #7914.

8 years ago7829: Gems include licenses and documentation.
Brett Smith [Thu, 3 Dec 2015 22:50:25 +0000 (17:50 -0500)]
7829: Gems include licenses and documentation.

8 years ago7829: Python bdists include licenses and documentation.
Brett Smith [Thu, 3 Dec 2015 22:35:29 +0000 (17:35 -0500)]
7829: Python bdists include licenses and documentation.

8 years ago7829: Remove unnecessary MANIFEST.in files.
Brett Smith [Thu, 3 Dec 2015 22:24:04 +0000 (17:24 -0500)]
7829: Remove unnecessary MANIFEST.in files.

setuptools includes every file tracked by your VCS.  See
<http://pythonhosted.org/setuptools/setuptools.html#generating-source-distributions>.

8 years agoFix manifest stats leaking between runs via globals. No issue #
Tom Clegg [Sat, 5 Dec 2015 00:18:49 +0000 (19:18 -0500)]
Fix manifest stats leaking between runs via globals. No issue #

8 years agoMerge branch '7835-node-manager-price-type-wip'
Brett Smith [Fri, 4 Dec 2015 21:46:38 +0000 (16:46 -0500)]
Merge branch '7835-node-manager-price-type-wip'

Refs #7835.  Closes #7950.

8 years ago7835: Node Manager converts node.size into a size object on GCE.
Brett Smith [Fri, 4 Dec 2015 20:13:49 +0000 (15:13 -0500)]
7835: Node Manager converts node.size into a size object on GCE.

8 years agoRemove redundant exception from Node Manager NETWORK_ERRORS.
Brett Smith [Fri, 4 Dec 2015 19:37:06 +0000 (14:37 -0500)]
Remove redundant exception from Node Manager NETWORK_ERRORS.

8 years ago7835: Node Manager coerces node size prices to float.
Brett Smith [Fri, 4 Dec 2015 19:35:30 +0000 (14:35 -0500)]
7835: Node Manager coerces node size prices to float.

8 years agocloses #7753
radhika [Fri, 4 Dec 2015 21:00:55 +0000 (16:00 -0500)]
closes #7753
Merge branch '7753-job-log-accessibility'

8 years ago7753: show the job Log link in the component panel header only when it is a pipeline_...
radhika [Fri, 4 Dec 2015 18:48:48 +0000 (13:48 -0500)]
7753: show the job Log link in the component panel header only when it is a pipeline_instance display; hide it for job#show

8 years ago7753: when a job is in "Running" state, the Log link still should be shown in the...
radhika [Fri, 4 Dec 2015 15:56:19 +0000 (10:56 -0500)]
7753: when a job is in "Running" state, the Log link still should be shown in the Component display panel header.

8 years ago7753: Now that there are 1+ "Log" links in the pipeline instance page, update tests...
radhika [Fri, 4 Dec 2015 14:49:22 +0000 (09:49 -0500)]
7753: Now that there are 1+ "Log" links in the pipeline instance page, update tests to address capybara ambigous match error.

8 years ago7753: The name "Job log" has a little problem; on screen small screen sizes, "Job...
radhika [Fri, 4 Dec 2015 14:19:33 +0000 (09:19 -0500)]
7753: The name "Job log" has a little problem; on screen small screen sizes, "Job log" is displayed in two lines even when the rest of the items are displayed in single line due to the limited real estate in the panel header.

8 years ago7753: Add a Log link to pipeline_instance component panel
radhika [Thu, 3 Dec 2015 21:00:52 +0000 (16:00 -0500)]
7753: Add a Log link to pipeline_instance component panel

8 years ago7942: changed capybara timeout from 5 sec to 10 sec so iceweasel in ci.curoverse...
Nico Cesar [Fri, 4 Dec 2015 15:32:23 +0000 (10:32 -0500)]
7942: changed capybara timeout from 5 sec to 10 sec so iceweasel in ci.curoverse.com doesnt break as easily

Merge branch '7942-ci-iceweasel-problems'

refs #7942

8 years agoFinish basic state change & record protection unit tests.
Peter Amstutz [Thu, 3 Dec 2015 23:49:14 +0000 (18:49 -0500)]
Finish basic state change & record protection unit tests.

8 years agoWorking on state change assertions and testing for container record.
Peter Amstutz [Thu, 3 Dec 2015 22:22:54 +0000 (17:22 -0500)]
Working on state change assertions and testing for container record.

8 years agoadded more time to ajax timeout
Nico Cesar [Thu, 3 Dec 2015 22:13:31 +0000 (17:13 -0500)]
added more time to ajax timeout

refs #7942

8 years agoMerge branch '7888-log-req-phase-times' refs #7888
Tom Clegg [Thu, 3 Dec 2015 19:26:19 +0000 (14:26 -0500)]
Merge branch '7888-log-req-phase-times' refs #7888

8 years agoDeclare serialize / accept json fields, start working on states
Peter Amstutz [Thu, 3 Dec 2015 18:07:52 +0000 (13:07 -0500)]
Declare serialize / accept json fields, start working on states

8 years ago7888: Log how much time was spent before/after sending response headers.
Tom Clegg [Thu, 3 Dec 2015 17:29:09 +0000 (12:29 -0500)]
7888: Log how much time was spent before/after sending response headers.

8 years agoMerge branch '7751-crunch-fuse-output' closes #7751
Tom Clegg [Thu, 3 Dec 2015 17:03:16 +0000 (12:03 -0500)]
Merge branch '7751-crunch-fuse-output' closes #7751

8 years ago7751: Add convenience class for staging task output in $TASK_KEEPMOUNT_TMP.
Tom Clegg [Fri, 27 Nov 2015 02:11:16 +0000 (21:11 -0500)]
7751: Add convenience class for staging task output in $TASK_KEEPMOUNT_TMP.

8 years ago7751: Set up an arv-mount scratch directory for each task, and put its path in TASK_K...
Tom Clegg [Fri, 27 Nov 2015 01:52:10 +0000 (20:52 -0500)]
7751: Set up an arv-mount scratch directory for each task, and put its path in TASK_KEEPMOUNT_TMP.

8 years agoStub out controllers and routes. Add basic workbench support.
Peter Amstutz [Thu, 3 Dec 2015 15:18:39 +0000 (10:18 -0500)]
Stub out controllers and routes.  Add basic workbench support.

8 years agocloses #6027
radhika [Wed, 2 Dec 2015 22:44:47 +0000 (17:44 -0500)]
closes #6027
Merge branch '6027-git-repo-doc'

8 years ago6027: fix typo
radhika [Wed, 2 Dec 2015 22:43:37 +0000 (17:43 -0500)]
6027: fix typo

8 years agoMerge branch 'master' into 6027-git-repo-doc
radhika [Wed, 2 Dec 2015 22:15:12 +0000 (17:15 -0500)]
Merge branch 'master' into 6027-git-repo-doc

8 years agoMerge branch '7868-docker-run-args' closes #7868
Tom Clegg [Wed, 2 Dec 2015 21:56:40 +0000 (16:56 -0500)]
Merge branch '7868-docker-run-args' closes #7868

8 years ago7868: Allow admin to add arguments to "docker run" commands.
Tom Clegg [Wed, 25 Nov 2015 22:53:14 +0000 (17:53 -0500)]
7868: Allow admin to add arguments to "docker run" commands.

8 years ago6027: update git-arvados-guide to instruct the users to create scripts in crunch_scri...
radhika [Wed, 2 Dec 2015 19:57:52 +0000 (14:57 -0500)]
6027: update git-arvados-guide to instruct the users to create scripts in crunch_scripts subdirectory.

8 years ago7313: Fix sprintf formatter in crunch-job warning message.
Brett Smith [Wed, 2 Dec 2015 17:54:56 +0000 (12:54 -0500)]
7313: Fix sprintf formatter in crunch-job warning message.

Refs #7313.

8 years agocloses #7748
radhika [Wed, 2 Dec 2015 17:37:15 +0000 (12:37 -0500)]
closes #7748
Merge branch '7748-datamanager-dry-run'

8 years ago7748: correct the local variable declaration
radhika [Wed, 2 Dec 2015 17:36:12 +0000 (12:36 -0500)]
7748: correct the local variable declaration

8 years agoMerge branch 'master' into 7748-datamanager-dry-run
radhika [Wed, 2 Dec 2015 17:30:10 +0000 (12:30 -0500)]
Merge branch 'master' into 7748-datamanager-dry-run

Conflicts:
services/datamanager/datamanager_test.go

8 years ago6429: Initial commit of models creating by rails generate.
Peter Amstutz [Wed, 2 Dec 2015 17:23:36 +0000 (12:23 -0500)]
6429: Initial commit of models creating by rails generate.

8 years agocloses #7710
radhika [Wed, 2 Dec 2015 17:22:00 +0000 (12:22 -0500)]
closes #7710
Merge branch '7710-keepclient-all-service-types'

8 years ago7710: text updates around the -service-type argument.
radhika [Wed, 2 Dec 2015 16:58:39 +0000 (11:58 -0500)]
7710: text updates around the -service-type argument.

8 years ago7710: update crunchrunner.upload.go to compile; this was broken after the keep-web...
radhika [Tue, 1 Dec 2015 18:33:55 +0000 (13:33 -0500)]
7710: update crunchrunner.upload.go to compile; this was broken after the keep-web updates; also, add this package to gostuff in run-tests.sh