arvados.git
7 years agoMerge branch '11369-dispatch-mem' refs #11369
Peter Amstutz [Wed, 3 May 2017 16:48:43 +0000 (12:48 -0400)]
Merge branch '11369-dispatch-mem' refs #11369

7 years agoMerge branch '11369-cwl-crunch2-capacity' refs #11369
Peter Amstutz [Wed, 3 May 2017 16:48:03 +0000 (12:48 -0400)]
Merge branch '11369-cwl-crunch2-capacity' refs #11369

7 years agoMerge branch '11606-rails4-no-munge'
Tom Clegg [Wed, 3 May 2017 14:20:52 +0000 (10:20 -0400)]
Merge branch '11606-rails4-no-munge'

closes #11606

7 years ago11369: Fix tests. Tweak test fixture to include a mounts entry with capacity.
Peter Amstutz [Wed, 3 May 2017 02:53:28 +0000 (22:53 -0400)]
11369: Fix tests.  Tweak test fixture to include a mounts entry with capacity.

7 years ago11369: Containers API: Add capacity to tmp and output mounts. Also support {outputDi...
Peter Amstutz [Wed, 3 May 2017 03:06:28 +0000 (23:06 -0400)]
11369: Containers API:  Add capacity to tmp and output mounts. Also support {outputDirType: keep_output_dir} of RuntimeConstraints.

7 years ago11369: RAM request includes keep cache RAM. Add request for temp disk.
Peter Amstutz [Wed, 3 May 2017 02:21:36 +0000 (22:21 -0400)]
11369: RAM request includes keep cache RAM.  Add request for temp disk.

7 years agoMerge branch '11185-wb-disable-reuse'
Lucas Di Pentima [Tue, 2 May 2017 15:44:26 +0000 (12:44 -0300)]
Merge branch '11185-wb-disable-reuse'
Closes #11185

7 years ago11606: Disable deep_munge behavior. Rename test so it runs in test suite.
Tom Clegg [Tue, 2 May 2017 15:39:52 +0000 (11:39 -0400)]
11606: Disable deep_munge behavior. Rename test so it runs in test suite.

7 years agoMerge branch '11579-arvput-follow-symlinks'
Lucas Di Pentima [Tue, 2 May 2017 15:24:14 +0000 (12:24 -0300)]
Merge branch '11579-arvput-follow-symlinks'
Closes #11579

7 years ago11185: Merge branch 'master' into 11185-wb-disable-reuse
Lucas Di Pentima [Tue, 2 May 2017 14:56:56 +0000 (11:56 -0300)]
11185: Merge branch 'master' into 11185-wb-disable-reuse

7 years ago11579: Merge branch 'master' into 11579-arvput-follow-symlinks
Lucas Di Pentima [Tue, 2 May 2017 14:55:02 +0000 (11:55 -0300)]
11579: Merge branch 'master' into 11579-arvput-follow-symlinks

7 years ago11579: Added assertion on test to check for file symlinks to be copied by default.
Lucas Di Pentima [Tue, 2 May 2017 14:52:44 +0000 (11:52 -0300)]
11579: Added assertion on test to check for file symlinks to be copied by default.
Don't treat a PathDoesNotExist exception as an abnormal termination, just log the proper error message, and quit.

7 years ago11579: Removed feature about not uploading a symlinked dir twice.
Lucas Di Pentima [Mon, 1 May 2017 21:05:02 +0000 (18:05 -0300)]
11579: Removed feature about not uploading a symlinked dir twice.
When using --no-follow-links parameter, symlinked files will be also ignored.
Updated tests.

7 years ago11185: Removed newly added completed-acr fixture and modified the already existing...
Lucas Di Pentima [Mon, 1 May 2017 19:58:48 +0000 (16:58 -0300)]
11185: Removed newly added completed-acr fixture and modified the already existing completed-older CR
to be able to use it on the tests.
Change the reset_form() JS function to something more specific to this usage, to avoid future
conflicts.

7 years agoMerge branch '10988-document-ubuntu1604-support'
Tom Clegg [Mon, 1 May 2017 18:03:24 +0000 (14:03 -0400)]
Merge branch '10988-document-ubuntu1604-support'

closes #10988

7 years ago10988: Remove apt.arvados.org repo priority: we no longer publish/require the older...
Tom Clegg [Mon, 1 May 2017 17:57:43 +0000 (13:57 -0400)]
10988: Remove apt.arvados.org repo priority: we no longer publish/require the older packages.

7 years ago10988: Document our Ubuntu 16.04 support.
Ward Vandewege [Thu, 26 Jan 2017 22:33:05 +0000 (17:33 -0500)]
10988: Document our Ubuntu 16.04 support.

7 years agoMerge branch '11469-volume-tmp' refs #11469
Peter Amstutz [Mon, 1 May 2017 16:44:54 +0000 (12:44 -0400)]
Merge branch '11469-volume-tmp' refs #11469

7 years agoCrunch script doesn't fail in file/directory literals that don't have a 'location...
Peter Amstutz [Mon, 1 May 2017 14:36:44 +0000 (10:36 -0400)]
Crunch script doesn't fail in file/directory literals that don't have a 'location'. refs #11549

7 years ago11469: Docker-managed volumes go in "Volumes" not "Binds".
Peter Amstutz [Mon, 1 May 2017 14:02:56 +0000 (10:02 -0400)]
11469: Docker-managed volumes go in "Volumes" not "Binds".

7 years ago11185: Addressed several requests:
Lucas Di Pentima [Fri, 28 Apr 2017 19:18:56 +0000 (16:18 -0300)]
11185: Addressed several requests:
* Added '...' to 'Re-run' button's title to indicate it will show a dialog.
* Corrected comment.
* Added javascript code so that the 'Cancel' and 'x' buttons on the dialog reset the checkbox.
* Expanded test cases to cover CRs that both do and don't use arvados-cwl-runner command, with
and without container reuse.

7 years agoMerge branch '11537-keepproxy-loop'
Tom Clegg [Fri, 28 Apr 2017 16:59:08 +0000 (12:59 -0400)]
Merge branch '11537-keepproxy-loop'

closes #11537

7 years ago11579: Check if paths exist, raise meaningful exception if not.
Lucas Di Pentima [Fri, 28 Apr 2017 15:25:01 +0000 (12:25 -0300)]
11579: Check if paths exist, raise meaningful exception if not.
Added test for this case.

7 years agoMerge branch '11549-cwl-smarter-mounts' closes #11549
Peter Amstutz [Fri, 28 Apr 2017 14:40:06 +0000 (10:40 -0400)]
Merge branch '11549-cwl-smarter-mounts' closes #11549

7 years ago11537: Add Via header to get/head/post/put responses.
Tom Clegg [Thu, 27 Apr 2017 19:24:41 +0000 (15:24 -0400)]
11537: Add Via header to get/head/post/put responses.

7 years ago11537: Add Via header to proxied keepstore requests.
Tom Clegg [Thu, 27 Apr 2017 14:51:49 +0000 (10:51 -0400)]
11537: Add Via header to proxied keepstore requests.

Propagate keepclient PUT error messages to caller.

7 years agoMerge branch '7709-rails-observers'
Tom Clegg [Thu, 27 Apr 2017 20:55:45 +0000 (16:55 -0400)]
Merge branch '7709-rails-observers'

refs #7709

7 years ago7709: Add rails-observers gem so plugins can use it.
Tom Clegg [Thu, 27 Apr 2017 20:51:46 +0000 (16:51 -0400)]
7709: Add rails-observers gem so plugins can use it.

7 years ago11579: Explicitly following links on test.
Lucas Di Pentima [Thu, 27 Apr 2017 20:15:53 +0000 (17:15 -0300)]
11579: Explicitly following links on test.

7 years ago11579: Tests added.
Lucas Di Pentima [Thu, 27 Apr 2017 20:12:56 +0000 (17:12 -0300)]
11579: Tests added.

7 years ago11579: Added symlinked dir traversal as the default behavior.
Lucas Di Pentima [Thu, 27 Apr 2017 19:51:31 +0000 (16:51 -0300)]
11579: Added symlinked dir traversal as the default behavior.
Added flag to disable this and ignore symlinked dirs: --no-follow-links.

7 years ago11549: Fix container requests so it doesn't mount each file individually,
Peter Amstutz [Thu, 27 Apr 2017 18:24:43 +0000 (14:24 -0400)]
11549: Fix container requests so it doesn't mount each file individually,
instead mount common collections/subdirectories.

7 years ago11549: Test that containers don't mount each file individually, instead mount
Peter Amstutz [Thu, 27 Apr 2017 18:05:00 +0000 (14:05 -0400)]
11549: Test that containers don't mount each file individually, instead mount
common collections/subdirectories.

7 years agoMerge branch '7709-api-rails4'
Tom Clegg [Thu, 27 Apr 2017 16:59:06 +0000 (12:59 -0400)]
Merge branch '7709-api-rails4'

closes #7709

7 years ago11579: Test proving the bug: arv-put does not follow symlinks.
Lucas Di Pentima [Thu, 27 Apr 2017 16:54:32 +0000 (13:54 -0300)]
11579: Test proving the bug: arv-put does not follow symlinks.

7 years ago11185: When re-run, the default behaviour is to copy reuse disabled. Changed modal...
Lucas Di Pentima [Thu, 27 Apr 2017 15:42:49 +0000 (12:42 -0300)]
11185: When re-run, the default behaviour is to copy reuse disabled. Changed modal dialog to allow
the user to enable reuse.
Updated tests.

7 years ago11185: Now we only have one "Re-run" button that will give the user the option to...
Lucas Di Pentima [Thu, 27 Apr 2017 13:55:26 +0000 (10:55 -0300)]
11185: Now we only have one "Re-run" button that will give the user the option to disable container reuse on a modal dialog.
Updated test.

7 years agoMerge branch '10111-cr-provenance-graph'
Lucas Di Pentima [Wed, 26 Apr 2017 18:56:00 +0000 (15:56 -0300)]
Merge branch '10111-cr-provenance-graph'
Closes #10111

7 years ago10111: Merge branch 'master' into 10111-cr-provenance-graph
Lucas Di Pentima [Wed, 26 Apr 2017 18:55:03 +0000 (15:55 -0300)]
10111: Merge branch 'master' into 10111-cr-provenance-graph

7 years ago10111: Replaced graph edges names: cr->child & mounts->input.
Lucas Di Pentima [Wed, 26 Apr 2017 18:29:20 +0000 (15:29 -0300)]
10111: Replaced graph edges names: cr->child & mounts->input.

7 years agoMerge branch '11305-migrate-docker19-doc'
Nico Cesar [Wed, 26 Apr 2017 18:25:22 +0000 (14:25 -0400)]
Merge branch '11305-migrate-docker19-doc'

Documentation for the migration process from Docker 1.9 or less  to Docker 1.10+

Docker changed the format of the images, and there is an arvados tool arvados-migrate-docker19,
this commit explains how to migrate

refs #11305

7 years ago10111: If there is no cwl.input.json object as part of a CR mount list, search for...
Lucas Di Pentima [Wed, 26 Apr 2017 16:28:06 +0000 (13:28 -0300)]
10111: If there is no cwl.input.json object as part of a CR mount list, search for input PDHs on the entire mount list

7 years ago7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg [Wed, 26 Apr 2017 14:33:13 +0000 (10:33 -0400)]
7709: Merge branch 'master' into 7709-api-rails4

7 years ago11185: On re-run without reuse, assign 'false' to CR.use_existing. Added related...
Lucas Di Pentima [Wed, 26 Apr 2017 14:03:35 +0000 (11:03 -0300)]
11185: On re-run without reuse, assign 'false' to CR.use_existing. Added related test.

7 years ago10111: Only include those mounts that are declared on the cwl input definition.
Lucas Di Pentima [Wed, 26 Apr 2017 11:56:01 +0000 (08:56 -0300)]
10111: Only include those mounts that are declared on the cwl input definition.
Removed trailing spaces.

7 years ago11185: Added "Re-run without reuse" button to the completed CR page, so that it
Lucas Di Pentima [Wed, 26 Apr 2017 02:04:49 +0000 (23:04 -0300)]
11185: Added "Re-run without reuse" button to the completed CR page, so that it
copies the CR modifying the arvados-cwl-runner arguments to avoid container
reuse.
Expanded related test.

7 years ago10111: Added "child" CRs to the graph, and removed container image and requesting...
Lucas Di Pentima [Tue, 25 Apr 2017 19:26:38 +0000 (16:26 -0300)]
10111: Added "child" CRs to the graph, and removed container image and requesting container elements.

7 years agoMerge branch '11544-bad-collection-id-500'
Tom Clegg [Tue, 25 Apr 2017 17:22:44 +0000 (13:22 -0400)]
Merge branch '11544-bad-collection-id-500'

closes #11544

7 years ago11544: Return 404 instead of 500 for malformed collection IDs.
Tom Clegg [Tue, 25 Apr 2017 16:57:11 +0000 (12:57 -0400)]
11544: Return 404 instead of 500 for malformed collection IDs.

7 years agoMerge branch '11507-deleted-bufferblocks' refs #11507
Peter Amstutz [Tue, 25 Apr 2017 15:58:34 +0000 (11:58 -0400)]
Merge branch '11507-deleted-bufferblocks' refs #11507

7 years ago11507: Move repack_writes() method from ArvadosFile to BufferBlock.
Peter Amstutz [Fri, 21 Apr 2017 20:36:21 +0000 (16:36 -0400)]
11507: Move repack_writes() method from ArvadosFile to BufferBlock.

7 years ago7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg [Tue, 25 Apr 2017 05:14:23 +0000 (01:14 -0400)]
7709: Merge branch 'master' into 7709-api-rails4

Conflicts:
services/api/db/structure.sql

7 years ago10111: Tests added
Lucas Di Pentima [Tue, 25 Apr 2017 03:50:31 +0000 (00:50 -0300)]
10111: Tests added

7 years ago10111: Merge branch 'master' into 10111-cr-provenance-graph
Lucas Di Pentima [Mon, 24 Apr 2017 20:07:19 +0000 (17:07 -0300)]
10111: Merge branch 'master' into 10111-cr-provenance-graph

7 years ago10111: Added provenance graph for container requests.
Lucas Di Pentima [Mon, 24 Apr 2017 19:30:00 +0000 (16:30 -0300)]
10111: Added provenance graph for container requests.

7 years agoFix panic in test case (overwriting a locked sync.Mutex with an unlocked one).
Tom Clegg [Mon, 24 Apr 2017 18:57:41 +0000 (14:57 -0400)]
Fix panic in test case (overwriting a locked sync.Mutex with an unlocked one).

No issue #

7 years agoMerge branch '11517-no-count-logs'
Tom Clegg [Mon, 24 Apr 2017 18:05:20 +0000 (14:05 -0400)]
Merge branch '11517-no-count-logs'

refs #11517

7 years ago11517: Add test for count=none.
Tom Clegg [Mon, 24 Apr 2017 18:02:54 +0000 (14:02 -0400)]
11517: Add test for count=none.

7 years ago11517: Skip database reset in some workbench unit tests.
Tom Clegg [Mon, 24 Apr 2017 18:00:45 +0000 (14:00 -0400)]
11517: Skip database reset in some workbench unit tests.

Reduces unit test suite time by half.

7 years ago11517: Comment with_count() method
Tom Clegg [Mon, 24 Apr 2017 17:42:31 +0000 (13:42 -0400)]
11517: Comment with_count() method

7 years ago11305: added kernel and module dependency for docker migration in doc
Nico Cesar [Mon, 24 Apr 2017 17:02:15 +0000 (13:02 -0400)]
11305: added kernel and module dependency  for docker migration in doc

refs #11305

7 years ago11517: Use count=none when retrieving logs.
Tom Clegg [Fri, 21 Apr 2017 19:17:46 +0000 (15:17 -0400)]
11517: Use count=none when retrieving logs.

7 years agowe need the tagged image to do this
Nico Cesar [Mon, 24 Apr 2017 12:43:55 +0000 (08:43 -0400)]
we need the tagged image to do this

7 years ago11507: Put self._pending_write_size update back under the lock.
Peter Amstutz [Fri, 21 Apr 2017 18:48:15 +0000 (14:48 -0400)]
11507: Put self._pending_write_size update back under the lock.

7 years ago11507: More small block packing fixes:
Peter Amstutz [Fri, 21 Apr 2017 18:31:49 +0000 (14:31 -0400)]
11507: More small block packing fixes:

* Repack writes before small block merging.
* Update segments of file with new locator/segment offset instead of replacing
  entire file contents.

7 years ago11507: Cleanup
Peter Amstutz [Fri, 21 Apr 2017 13:55:00 +0000 (09:55 -0400)]
11507: Cleanup

7 years ago11507: Put bufferblocks into DELETED state when deleted so they don't get
Peter Amstutz [Thu, 20 Apr 2017 21:45:37 +0000 (17:45 -0400)]
11507: Put bufferblocks into DELETED state when deleted so they don't get
reused accidentally if there are dangling references.  When packing small
blocks, update segments with the block hash instead of the bufferblock id to
avoid dangling references.

7 years ago11507: Test cases that reproduce bug
Peter Amstutz [Thu, 20 Apr 2017 21:10:30 +0000 (17:10 -0400)]
11507: Test cases that reproduce bug

7 years agoMerge branch '11510-sdk-extend-files' refs #11510
Peter Amstutz [Thu, 20 Apr 2017 18:18:44 +0000 (14:18 -0400)]
Merge branch '11510-sdk-extend-files' refs #11510

7 years ago11510: Tweak interface to better match Pythons "io.IOBase" class.
Peter Amstutz [Thu, 20 Apr 2017 18:15:49 +0000 (14:15 -0400)]
11510: Tweak interface to better match Pythons "io.IOBase" class.

7 years agoMerge branch '11470-update-task-fields'
Tom Clegg [Thu, 20 Apr 2017 18:10:50 +0000 (14:10 -0400)]
Merge branch '11470-update-task-fields'

refs #11470

7 years ago11510: Fix truncate() to not change file pointer. Additional tests.
Peter Amstutz [Thu, 20 Apr 2017 17:25:56 +0000 (13:25 -0400)]
11510: Fix truncate() to not change file pointer.  Additional tests.

7 years ago11510: Repack writes any time there's more than one segment referencing the same...
Peter Amstutz [Thu, 20 Apr 2017 14:41:05 +0000 (10:41 -0400)]
11510: Repack writes any time there's more than one segment referencing the same bufferblock.

7 years ago11510: Fix longstanding bug in replace_range() when appending data from a buffer...
Peter Amstutz [Tue, 18 Apr 2017 21:54:57 +0000 (17:54 -0400)]
11510: Fix longstanding bug in replace_range() when appending data from a buffer that was written out of order.

7 years ago11470: Fix identifiers to conventional Go.
Tom Clegg [Thu, 20 Apr 2017 02:12:42 +0000 (22:12 -0400)]
11470: Fix identifiers to conventional Go.

7 years ago11470: Fix accidentally cleared fields during job_task update.
Tom Clegg [Thu, 20 Apr 2017 02:09:45 +0000 (22:09 -0400)]
11470: Fix accidentally cleared fields during job_task update.

Updating with a Task struct had the effect of clearing job_uuid,
created_by_job_task_uuid, and parameters fields, and resetting
sequence to 0.

7 years agoMerge branch '11502-unstripped-manifest-fix'
Lucas Di Pentima [Wed, 19 Apr 2017 20:42:26 +0000 (17:42 -0300)]
Merge branch '11502-unstripped-manifest-fix'
Closes #11502, #11511

7 years ago11502: Merge branch 'master' into 11502-unstripped-manifest-fix
Lucas Di Pentima [Wed, 19 Apr 2017 19:38:30 +0000 (16:38 -0300)]
11502: Merge branch 'master' into 11502-unstripped-manifest-fix

7 years ago11502: Instead of asking for a stripped manifest, replace the tokens with some consta...
Lucas Di Pentima [Wed, 19 Apr 2017 19:28:26 +0000 (16:28 -0300)]
11502: Instead of asking for a stripped manifest, replace the tokens with some constant string to do the comparison.

7 years agoMerge branch '11517-indexes'
Tom Clegg [Wed, 19 Apr 2017 19:04:20 +0000 (15:04 -0400)]
Merge branch '11517-indexes'

refs #11517

7 years ago11517: Add indexes to help slow queries.
Tom Clegg [Wed, 19 Apr 2017 18:00:01 +0000 (14:00 -0400)]
11517: Add indexes to help slow queries.

7 years agocloses #11450
radhika [Wed, 19 Apr 2017 15:27:33 +0000 (11:27 -0400)]
closes #11450
Merge branch '11450-workflows-page'

7 years ago11450: workflows index page
radhika [Fri, 14 Apr 2017 17:29:54 +0000 (13:29 -0400)]
11450: workflows index page

7 years ago11510: Add support for sparse writes to ArvadosFile.
Peter Amstutz [Tue, 18 Apr 2017 20:40:51 +0000 (16:40 -0400)]
11510: Add support for sparse writes to ArvadosFile.

Allow seeking past end of file.  Allow using truncate() to extend file size.  Add zero padding to fill in gaps.

7 years ago11459: Move RAILS_PACKAGE_ITERATION to run-library.
Tom Clegg [Tue, 18 Apr 2017 18:01:47 +0000 (14:01 -0400)]
11459: Move RAILS_PACKAGE_ITERATION to run-library.

Otherwise, the SSO build script cannot see it.

refs #11459

7 years agoMerge branch '11509-keep-web-cors-range'
Tom Clegg [Tue, 18 Apr 2017 17:01:01 +0000 (13:01 -0400)]
Merge branch '11509-keep-web-cors-range'

closes #11509

7 years ago11509: Add comments.
Tom Clegg [Tue, 18 Apr 2017 17:00:27 +0000 (13:00 -0400)]
11509: Add comments.

7 years ago11502: Use os.path.join instead of formatting path-like strings on tests.
Lucas Di Pentima [Tue, 18 Apr 2017 16:45:39 +0000 (13:45 -0300)]
11502: Use os.path.join instead of formatting path-like strings on tests.

7 years ago11502: Avoid races on test by getting the unstripped manifest version before the...
Lucas Di Pentima [Tue, 18 Apr 2017 16:36:44 +0000 (13:36 -0300)]
11502: Avoid races on test by getting the unstripped manifest version before the comparison.

7 years ago11502: Fix regression by writing a collection manifest with its access tokens in...
Lucas Di Pentima [Tue, 18 Apr 2017 15:31:48 +0000 (12:31 -0300)]
11502: Fix regression by writing a collection manifest with its access tokens in place when asked by PDH & UUID.
Added --strip-manifest argument to allow writing manifests without tokens.
Updated tests accordingly.

7 years agoFix ignored symlink failure.
Tom Clegg [Tue, 18 Apr 2017 15:25:15 +0000 (11:25 -0400)]
Fix ignored symlink failure.

No issue #

7 years agorefs #11465
radhika [Tue, 18 Apr 2017 13:57:55 +0000 (09:57 -0400)]
refs #11465
Merge branch '11465-disabled-collection-file-tooltips'

7 years ago11465: tooltip on disabled pencil icon
radhika [Tue, 18 Apr 2017 13:56:48 +0000 (09:56 -0400)]
11465: tooltip on disabled pencil icon

7 years ago11509: Allow cross-origin requests with Range headers.
Tom Clegg [Mon, 17 Apr 2017 23:19:10 +0000 (19:19 -0400)]
11509: Allow cross-origin requests with Range headers.

7 years agocloses #11465
radhika [Mon, 17 Apr 2017 19:17:13 +0000 (15:17 -0400)]
closes #11465
Merge branch '11465-disabled-collection-file-tooltips'

7 years ago11465: correct disabled tooltip behavior on image types
radhika [Mon, 17 Apr 2017 17:34:44 +0000 (13:34 -0400)]
11465: correct disabled tooltip behavior on image types

7 years agoFix crunch-run tests.
Tom Clegg [Mon, 17 Apr 2017 14:01:06 +0000 (10:01 -0400)]
Fix crunch-run tests.

refs #11209

7 years ago11465: add tooltips to disabled collection file controls
radhika [Mon, 17 Apr 2017 01:20:11 +0000 (21:20 -0400)]
11465: add tooltips to disabled collection file controls

7 years agoMerge branch '11209-crunch-unmount-all'
Tom Clegg [Fri, 14 Apr 2017 20:42:12 +0000 (16:42 -0400)]
Merge branch '11209-crunch-unmount-all'

refs #11209

7 years agoMerge branch '11502-arvget-flaky-test'
Lucas Di Pentima [Fri, 14 Apr 2017 20:26:29 +0000 (17:26 -0300)]
Merge branch '11502-arvget-flaky-test'
Closes #11502