Brett Smith [Wed, 3 Jun 2015 20:59:24 +0000 (16:59 -0400)]
5790: Improve Docker image listing in Python SDK.
* Always fetch all relevant Docker links.
* Support finding images by image hash.
* Show image hashes when listing images by name.
* Like Docker itself, when an image has multiple names and we're not
filtering by name, list each one.
* Better match the API server's priority logic:
* Ignore links to collections that aren't found.
* Links with an image_timestamp always have priority over those that
don't, regardless of their respective created_at timestamps.
The main motivation for this change is to make sure arv-copy gets the
right Docker image when copying a pipeline template recursively.
This implementation goes through some trouble to parse timestamps out
of each Docker link only once.
Tom Clegg [Wed, 3 Jun 2015 13:50:55 +0000 (09:50 -0400)]
Merge branch '6146-log-squeue-lost-tasks' refs #6146
Tom Clegg [Sun, 31 May 2015 09:48:22 +0000 (05:48 -0400)]
6146: Better log message.
Tom Clegg [Sun, 31 May 2015 09:32:55 +0000 (05:32 -0400)]
6146: Use new SLURM_JOB_ID env var instead of old SLURM_JOBID
Tom Clegg [Sun, 31 May 2015 06:02:19 +0000 (02:02 -0400)]
6146: Improvements to "kill srun process if slurm task disappears" feature:
* Log when we notice a process is orphaned.
* Log when we decide to kill an orphaned process.
* Use `squeue --jobs $SLURM_JOBID` so slurm doesn't have to tell us
about other jobs' tasks.
* Do not kill a process that is still reporting stderr.
* Do not check `squeue` at all if every process has reported stderr
since the last squeue check. (In such cases, it seems safe to assume
no children are hung/dead.)
* Use the same timer/interval (15 seconds) for both noticing and
killing orphaned processes.
radhika [Wed, 3 Jun 2015 02:19:54 +0000 (22:19 -0400)]
refs #6093
Merge branch '6093-refresh-docs'
radhika [Wed, 3 Jun 2015 02:17:30 +0000 (22:17 -0400)]
6093: delete the redundant details in "alternate way to add ssh keys" section.
radhika [Wed, 3 Jun 2015 02:08:42 +0000 (22:08 -0400)]
Merge branch 'master' into 6093-refresh-docs
Nancy Ouyang [Wed, 3 Jun 2015 00:09:20 +0000 (20:09 -0400)]
closes #5930. Merge branch '5930-smalldocfix'
Nancy Ouyang [Wed, 3 Jun 2015 00:08:16 +0000 (20:08 -0400)]
5930: fixed as per code review
Peter Amstutz [Tue, 2 Jun 2015 20:23:22 +0000 (16:23 -0400)]
Merge branch '6194-python-arvfile-large-write' closes #6194
Peter Amstutz [Tue, 2 Jun 2015 20:17:31 +0000 (16:17 -0400)]
6194: Simplify test_large_write a little bit.
radhika [Tue, 2 Jun 2015 19:39:41 +0000 (15:39 -0400)]
Merge branch 'master' into 6093-refresh-docs
Peter Amstutz [Tue, 2 Jun 2015 17:32:19 +0000 (13:32 -0400)]
6194: Make splitting loop simpler since [n:n+KEEP_BLOCK_SIZE] returns a short
slice when there isn't KEEP_BLOCK_SIZE data. Update test.
Peter Amstutz [Mon, 1 Jun 2015 20:57:54 +0000 (16:57 -0400)]
Merge branch 'master' into 6194-python-arvfile-large-write
Peter Amstutz [Mon, 1 Jun 2015 20:56:15 +0000 (16:56 -0400)]
6194: Fix test. Lots of small writes break across blocks differently than one huge
one.
Peter Amstutz [Mon, 1 Jun 2015 12:51:35 +0000 (08:51 -0400)]
6194: Fix typo in invocation of writeto() and use memoryview to avoid copying slices.
Tom Clegg [Sun, 31 May 2015 12:39:20 +0000 (08:39 -0400)]
Remove non-existent migration from structure.sql. refs #3036
Tom Clegg [Sun, 31 May 2015 12:38:52 +0000 (08:38 -0400)]
Update example dns_server_update_command. refs #6146
Tom Clegg [Sun, 31 May 2015 12:29:51 +0000 (08:29 -0400)]
Merge branch '6146-dns-update-command' refs #6146
Tom Clegg [Sun, 31 May 2015 12:23:44 +0000 (08:23 -0400)]
6146: Add dns_server_update_command. Update docs & tests for DNS update hooks.
Tom Clegg [Sat, 30 May 2015 02:01:06 +0000 (22:01 -0400)]
Tell tar to read to EOF (even if it detects trailing NULs).
Avoids SIGPIPE when feeding a tarball made with tar -A.
refs #6146 refs #6094
Peter Amstutz [Fri, 29 May 2015 20:28:37 +0000 (16:28 -0400)]
6194: Chunk large ArvadosFile writes automatically instead of raising an error.
Tom Clegg [Fri, 29 May 2015 18:04:01 +0000 (14:04 -0400)]
Merge branch '6146-ignore-tar-sigpipe' refs #6146 refs #6094
Tom Clegg [Fri, 29 May 2015 17:32:40 +0000 (13:32 -0400)]
6146: Ignore SIGPIPE while feeding data to tar. Rely on close() retval instead.
Tom Clegg [Fri, 29 May 2015 16:02:03 +0000 (12:02 -0400)]
In install script, log archive hash before running tar. refs #6146
Tom Clegg [Fri, 29 May 2015 15:31:43 +0000 (11:31 -0400)]
Merge branch '6146-job-runtime-sanity' refs #6146
Tom Clegg [Thu, 28 May 2015 21:13:44 +0000 (17:13 -0400)]
6146: Exit TEMPFAIL early (without failing the job) if worker nodes cannot run a trivial command.
This is meant to improve the way we handle a couple of edge cases.
1. A worker node doesn't get bootstrapped properly. It works well
enough to persuade nodemanager and the API server that it's alive and
ready to run jobs, but it can't actually run jobs. This means there's
a bug in the bootstrapping process -- its startup script shouldn't
tell slurm State=RESUME without checking itself -- but even so this
doesn't deserve to fail a job: it's definitely a system problem,
there's zero chance a different job would have gone any differently.
2. A worker node has a hardware problem, or it has fallen off the
network, or something like that, but slurm hasn't yet noticed and set
its state to DOWN, so slurm still uses it to satisfy crunch-dispatch's
"salloc" commands. As above, there's zero chance this could have gone
differently for any other job, so it doesn't make sense to fail the
job.
Tom Clegg [Wed, 27 May 2015 20:10:24 +0000 (16:10 -0400)]
Merge branch '6146-retry-install' refs #6146
Tom Clegg [Wed, 27 May 2015 19:48:54 +0000 (15:48 -0400)]
6146: Retry install (max 3 attempts) if install script fails with no error messages.
Also: if install fails, croak() instead of exit(1) so we still get a log file.
radhika [Wed, 27 May 2015 19:38:59 +0000 (15:38 -0400)]
Merge branch 'master' into 6093-refresh-docs
Conflicts:
doc/user/tutorials/tutorial-submit-job.html.textile.liquid
radhika [Wed, 27 May 2015 19:27:36 +0000 (15:27 -0400)]
closes #6057
Merge branch '6057-public-projects-page'
radhika [Wed, 27 May 2015 19:26:20 +0000 (15:26 -0400)]
6057: few more minor tweaks
Peter Amstutz [Wed, 27 May 2015 19:11:58 +0000 (15:11 -0400)]
Merge branch '6141-doc-workbench-links' refs #6141
radhika [Wed, 27 May 2015 19:03:54 +0000 (15:03 -0400)]
Merge branch 'master' into 6057-public-projects-page
Peter Amstutz [Wed, 27 May 2015 18:23:03 +0000 (14:23 -0400)]
Merge branch '6090-docker-use-local-sso' closes #6138
Ward Vandewege [Wed, 27 May 2015 16:42:44 +0000 (12:42 -0400)]
Add GPG key for RVM installation in the doc.
No issue #
Tom Clegg [Wed, 27 May 2015 13:06:10 +0000 (09:06 -0400)]
Merge branch '6098-full-text-index' refs #6098
Tom Clegg [Wed, 27 May 2015 13:05:40 +0000 (09:05 -0400)]
6098: Recreate full text indexes with leading spaces, to persuade Postgres to actually use them.
radhika [Tue, 26 May 2015 22:58:18 +0000 (18:58 -0400)]
Merge branch 'master' into 6057-public-projects-page
radhika [Tue, 26 May 2015 22:54:13 +0000 (18:54 -0400)]
6057: if /projects/public is accessed when anonymous config is not enabled, show 404.
radhika [Tue, 26 May 2015 20:04:41 +0000 (16:04 -0400)]
Merge branch 'master' into 6093-refresh-docs
radhika [Tue, 26 May 2015 19:57:11 +0000 (15:57 -0400)]
6093: one more
radhika [Tue, 26 May 2015 19:51:44 +0000 (15:51 -0400)]
6093: a few more updates
radhika [Tue, 26 May 2015 15:56:29 +0000 (11:56 -0400)]
6093: some more doc updates.
Tom Clegg [Tue, 26 May 2015 14:31:56 +0000 (10:31 -0400)]
Merge branch '6094-install-script-sigpipe' refs #6094
Tom Clegg [Tue, 26 May 2015 14:18:38 +0000 (10:18 -0400)]
6094: Propagate install script stderr+stdout to job log.
Brett Smith [Tue, 26 May 2015 00:32:19 +0000 (20:32 -0400)]
Merge branch '6095-arv-copy-preserve-object-order-wip'
Closes #6095, #6117.
Brett Smith [Fri, 22 May 2015 21:10:37 +0000 (17:10 -0400)]
6095: arv-copy preserves order of copied JSON.
This means arv-copy no longer loses the order of pipeline template
components, which makes for a nicer presentation in Workbench.
Other Python clients that would like to preserve the order of JSON
responses can use OrderedJsonModel the same way.
radhika [Mon, 25 May 2015 22:36:49 +0000 (18:36 -0400)]
6093: doc updates
Tom Clegg [Fri, 22 May 2015 21:42:17 +0000 (17:42 -0400)]
Merge branch '6094-install-script-sigpipe' (early part) refs #6094
radhika [Fri, 22 May 2015 20:40:59 +0000 (16:40 -0400)]
Merge branch 'master' into 6057-public-projects-page
radhika [Fri, 22 May 2015 20:39:57 +0000 (16:39 -0400)]
6057: add projects/public page, which lists publicly accessible projects.
Peter Amstutz [Fri, 22 May 2015 20:13:22 +0000 (16:13 -0400)]
6141: Remove hard-coded "https://" from "https://{{site.arvados_workbench_host}}" and require that arvados_workbench_host include the url scheme instead.
Tom Clegg [Fri, 22 May 2015 19:40:29 +0000 (15:40 -0400)]
6094: Consider arvados_sdk_version (not just script_version) when
deciding there's no need to extract or install anything.
Tom Clegg [Fri, 22 May 2015 19:39:00 +0000 (15:39 -0400)]
6094: Avoid SIGPIPE by consuming DATA section even when it's not needed.
Peter Amstutz [Fri, 22 May 2015 19:32:47 +0000 (15:32 -0400)]
6138: Added --domain to set ARVADOS_DOMAIN. Removed useless comments in apache2_vhost that mentioned qr1hi.
Ward Vandewege [Fri, 22 May 2015 14:03:01 +0000 (10:03 -0400)]
Add installation instructions for compute nodes; update the installation
instructions for crunch dispatcher.
No issue #
Tom Clegg [Thu, 21 May 2015 21:15:06 +0000 (17:15 -0400)]
Merge branch '6087-collection-timing' (early part) refs #6087 refs #6092
Tom Clegg [Thu, 21 May 2015 20:51:52 +0000 (16:51 -0400)]
6087: Get database time only once per manifest-signing/verifying event, rather than once per locator.
Brett Smith [Thu, 21 May 2015 20:06:16 +0000 (16:06 -0400)]
Update tutorial pipeline page to match new definition.
No issue #.
Peter Amstutz [Thu, 21 May 2015 19:00:10 +0000 (15:00 -0400)]
6090: Docker install uses local SSO server instead of auth.curoverse.com. Also
clean up references to dev.arvados to use @@ARVADOS_DOMAIN@@.
Brett Smith [Thu, 21 May 2015 18:23:46 +0000 (14:23 -0400)]
Update tutorial pipeline template definition.
This helps it run out of the box again.
No issue #.
radhika [Thu, 21 May 2015 16:09:07 +0000 (12:09 -0400)]
refs #6061
6061: set profiling_enabled to true in workbench application.default.yml for test env.
Ward Vandewege [Wed, 20 May 2015 15:29:29 +0000 (11:29 -0400)]
Documentation fix (quoting) for the 'Add an Arvados repository' page.
No issue #
Tom Clegg [Wed, 20 May 2015 14:54:00 +0000 (10:54 -0400)]
5748: Remove unused import. refs #5748
Tom Clegg [Wed, 20 May 2015 14:15:30 +0000 (10:15 -0400)]
5748: Remove runtime.GC() obsoleted by bufferpool. refs #5748
radhika [Wed, 20 May 2015 13:40:35 +0000 (09:40 -0400)]
closes #6064
Merge branch '6064-collection-output-of-perf-issue'
radhika [Wed, 20 May 2015 12:13:14 +0000 (08:13 -0400)]
integration_performance tests: add comment describing how to run the tests; also remove the ENV variable which is not needed
since the tests in this directory will not be part of regular test run because it is not a standard rails test directory.
radhika [Wed, 20 May 2015 03:00:21 +0000 (23:00 -0400)]
6064: rollback cache update so that it can be done in a separate branch.
radhika [Wed, 20 May 2015 02:32:25 +0000 (22:32 -0400)]
Merge branch 'master' into 6064-collection-output-of-perf-issue
Peter Amstutz [Tue, 19 May 2015 20:31:39 +0000 (16:31 -0400)]
Merge branch '6071-java-sdk-timeouts' closes #6071
Peter Amstutz [Tue, 19 May 2015 20:29:36 +0000 (16:29 -0400)]
6071: Bump up timeout to 120 seconds.
Radhika Chippada [Tue, 19 May 2015 19:26:07 +0000 (15:26 -0400)]
refs #6061
Merge branch '6061-intg-perf-tests'
Peter Amstutz [Tue, 19 May 2015 19:31:35 +0000 (15:31 -0400)]
Merge branch '6073-pycurl-version' refs #6073
Radhika Chippada [Tue, 19 May 2015 19:23:34 +0000 (15:23 -0400)]
6061: add workbench integration_performance test folder.
Peter Amstutz [Tue, 19 May 2015 19:20:16 +0000 (15:20 -0400)]
6071: Fix URL
Peter Amstutz [Tue, 19 May 2015 17:55:57 +0000 (13:55 -0400)]
6071: Refactor project organization a bit.
Peter Amstutz [Tue, 19 May 2015 17:52:51 +0000 (13:52 -0400)]
6071: Increase default timeout and enable retry support.
Peter Amstutz [Tue, 19 May 2015 16:54:37 +0000 (12:54 -0400)]
6073: Pycurl depends on latest in Arvados SDK setup.py
Radhika Chippada [Tue, 19 May 2015 03:24:46 +0000 (23:24 -0400)]
closes #4717
Merge branch '4717-read-only-keep-services-flag'
Radhika Chippada [Tue, 19 May 2015 03:11:59 +0000 (23:11 -0400)]
4717: rename writableRoots as writableLocalRoots.
Radhika Chippada [Tue, 19 May 2015 02:22:18 +0000 (22:22 -0400)]
Merge branch 'master' into 4717-read-only-keep-services-flag
Radhika Chippada [Tue, 19 May 2015 02:21:33 +0000 (22:21 -0400)]
closes #5720
Merge branch '5720-ajax-loading-error'
Radhika Chippada [Tue, 19 May 2015 02:09:21 +0000 (22:09 -0400)]
6064: display only 5 objects for which this collection is the output in source_summary.
Tom Clegg [Mon, 18 May 2015 17:36:27 +0000 (13:36 -0400)]
5720: Leave existing content in place (and schedule another reload) if a tab-refresh gets cancelled.
Nico Cesar [Fri, 15 May 2015 21:04:06 +0000 (17:04 -0400)]
Merge branch '5998-arvdock-bridge-option'
closes #5998
Nico Cesar [Fri, 15 May 2015 19:00:21 +0000 (15:00 -0400)]
$start_bridge -> $bridge
options modified
ip command refactored
refs #5998
Ward Vandewege [Fri, 15 May 2015 13:46:07 +0000 (09:46 -0400)]
Check in docker/sso/database.yml.in, forgotten in commit
ea7da293b0a10baef18cfc6e107c87a9c3b09546.
No issue #
Radhika Chippada [Fri, 15 May 2015 02:56:12 +0000 (22:56 -0400)]
Merge branch 'master' into 4717-read-only-keep-services-flag
Radhika Chippada [Fri, 15 May 2015 02:55:25 +0000 (22:55 -0400)]
4717: use keep_services -> read_only flag in python sdk.
Tom Clegg [Fri, 15 May 2015 02:11:49 +0000 (22:11 -0400)]
Merge branch '5748-max-buffers-leak-TC' refs #5748
Tom Clegg [Fri, 15 May 2015 02:10:02 +0000 (22:10 -0400)]
5748: gofmt fixes.
Tom Clegg [Fri, 15 May 2015 02:06:46 +0000 (22:06 -0400)]
5748: Time out leak tests after 20s.
Tom Clegg [Fri, 15 May 2015 01:52:12 +0000 (21:52 -0400)]
5748: Put() always takes a non-nil buf.
Brett Smith [Thu, 14 May 2015 22:32:02 +0000 (18:32 -0400)]
5915: Workbench tries the anonymous reader token first for collection wget listing.
This is necessary to make sure we provide a usable token to arv-get.
If we don't check the anonymous reader token first, we might decide
that another token is usable when in actuality, the reader token is
the one that worked. Closes #5915.
Peter Amstutz [Thu, 14 May 2015 21:01:13 +0000 (17:01 -0400)]
5748: Basic tests to check for bufferpool resource leaks
Peter Amstutz [Thu, 14 May 2015 20:02:43 +0000 (16:02 -0400)]
5748: Fix leaks in GetBlock error handling and PutBlock use of GetBlock where
buffers were not being returned to the buffer pool.
Radhika Chippada [Thu, 14 May 2015 16:39:42 +0000 (12:39 -0400)]
4717: just a couple minor tweaks
Radhika Chippada [Thu, 14 May 2015 16:32:12 +0000 (12:32 -0400)]
4717: writable roots in go sdk
projects / arvados.git / log