arvados.git
7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Mon, 8 May 2017 16:42:10 +0000 (12:42 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years agoMerge branch '11308-python3'
Tom Clegg [Mon, 8 May 2017 16:06:00 +0000 (12:06 -0400)]
Merge branch '11308-python3'

refs #11308

7 years ago11308: Merge branch 'master' into 11308-python3
Tom Clegg [Mon, 8 May 2017 16:05:31 +0000 (12:05 -0400)]
11308: Merge branch 'master' into 11308-python3

7 years ago10645: improve location regex usage
radhika [Mon, 8 May 2017 13:18:19 +0000 (09:18 -0400)]
10645: improve location regex usage

7 years ago10645: table width 100%
radhika [Fri, 5 May 2017 19:35:32 +0000 (15:35 -0400)]
10645: table width 100%

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 17:30:38 +0000 (13:30 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago10645: corrections to test fixture
radhika [Fri, 5 May 2017 17:29:59 +0000 (13:29 -0400)]
10645: corrections to test fixture

7 years agocloses #8019
radhika [Fri, 5 May 2017 16:05:11 +0000 (12:05 -0400)]
closes #8019
Merge branch '8019-crunchrun-log-throttle'

7 years agoMerge branch 'master' into 8019-crunchrun-log-throttle
radhika [Fri, 5 May 2017 16:02:36 +0000 (12:02 -0400)]
Merge branch 'master' into 8019-crunchrun-log-throttle

7 years ago8019: Make flush channel buffered again (because we want to be able to do back-to...
Peter Amstutz [Fri, 5 May 2017 15:54:13 +0000 (11:54 -0400)]
8019: Make flush channel buffered again (because we want to be able to do back-to-back flushes).
Fix "> crunchLogBytesPerEvent" to ">= crunchLogBytesPerEvent" for consistency.

7 years ago10645: small style updates
radhika [Fri, 5 May 2017 15:12:48 +0000 (11:12 -0400)]
10645: small style updates

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 13:44:53 +0000 (09:44 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 01:32:37 +0000 (21:32 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago10645: mount inputs display
radhika [Fri, 5 May 2017 01:31:18 +0000 (21:31 -0400)]
10645: mount inputs display

7 years agoMerge branch '10111-cr-prov-regression-fixes'
Lucas Di Pentima [Thu, 4 May 2017 18:30:52 +0000 (15:30 -0300)]
Merge branch '10111-cr-prov-regression-fixes'
Refs #10111

7 years ago10111: Flipped arrow between the cr and its childs cr.
Lucas Di Pentima [Thu, 4 May 2017 18:26:48 +0000 (15:26 -0300)]
10111: Flipped arrow between the cr and its childs cr.

7 years ago8019: uncomment the test statement
radhika [Thu, 4 May 2017 17:32:34 +0000 (13:32 -0400)]
8019: uncomment the test statement

7 years ago10111: Only add cr input collections by PDH.
Lucas Di Pentima [Thu, 4 May 2017 17:17:54 +0000 (14:17 -0300)]
10111: Only add cr input collections by PDH.

7 years ago8019: Use tl.stopping channel instead of tl.flush to signal Close. Use unbuffered...
Tom Clegg [Thu, 4 May 2017 17:02:40 +0000 (13:02 -0400)]
8019: Use tl.stopping channel instead of tl.flush to signal Close. Use unbuffered channel to signal flush.

7 years ago10111: Skip log outputs from the provenance graph.
Lucas Di Pentima [Thu, 4 May 2017 15:42:19 +0000 (12:42 -0300)]
10111: Skip log outputs from the provenance graph.
Container request's outputs are now added to the graph by their PDHs so that they connect with the CR childs inputs.

7 years ago11308: Merge branch 'master' into 11308-python3
Tom Clegg [Thu, 4 May 2017 15:16:42 +0000 (11:16 -0400)]
11308: Merge branch 'master' into 11308-python3

7 years ago11308: Fix bytes vs. strings from recent merge.
Tom Clegg [Thu, 4 May 2017 15:09:41 +0000 (11:09 -0400)]
11308: Fix bytes vs. strings from recent merge.

7 years agoWe stopped building packages for Ubuntu 12.04, it is now officially EOL.
Ward Vandewege [Thu, 4 May 2017 01:01:05 +0000 (21:01 -0400)]
We stopped building packages for Ubuntu 12.04, it is now officially EOL.

No issue #

7 years agocloses #11522
radhika [Wed, 3 May 2017 21:56:38 +0000 (17:56 -0400)]
closes #11522
Merge branch '11522-remove-user-storage-report-page'

7 years agocloses #11250
radhika [Wed, 3 May 2017 20:55:40 +0000 (16:55 -0400)]
closes #11250
Merge branch '11250-container-state-display'

7 years ago11522: remove link to users storage page in users activity page.
radhika [Wed, 3 May 2017 20:37:58 +0000 (16:37 -0400)]
11522: remove link to users storage page in users activity page.

7 years ago11250: when a container's priority is zero and in Queued state, show the state as...
radhika [Wed, 3 May 2017 20:20:08 +0000 (16:20 -0400)]
11250: when a container's priority is zero and in Queued state, show the state as "Cancelled"

7 years ago10645: rename file
radhika [Wed, 3 May 2017 18:37:38 +0000 (14:37 -0400)]
10645: rename file

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Wed, 3 May 2017 18:24:28 +0000 (14:24 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years agoMerge branch '11369-log-sizes' refs #11369
Peter Amstutz [Wed, 3 May 2017 18:20:08 +0000 (14:20 -0400)]
Merge branch '11369-log-sizes' refs #11369

7 years agoMerge branch '11469-volume-tmp' refs #11469
Peter Amstutz [Wed, 3 May 2017 16:59:34 +0000 (12:59 -0400)]
Merge branch '11469-volume-tmp' refs #11469

7 years agoMerge branch '11369-dispatch-mem' refs #11369
Peter Amstutz [Wed, 3 May 2017 16:48:43 +0000 (12:48 -0400)]
Merge branch '11369-dispatch-mem' refs #11369

7 years agoMerge branch '11369-cwl-crunch2-capacity' refs #11369
Peter Amstutz [Wed, 3 May 2017 16:48:03 +0000 (12:48 -0400)]
Merge branch '11369-cwl-crunch2-capacity' refs #11369

7 years ago11469: Add test for changed tmp mounting behavior.
Peter Amstutz [Wed, 3 May 2017 14:49:04 +0000 (10:49 -0400)]
11469: Add test for changed tmp mounting behavior.

7 years agoMerge branch '11606-rails4-no-munge'
Tom Clegg [Wed, 3 May 2017 14:20:52 +0000 (10:20 -0400)]
Merge branch '11606-rails4-no-munge'

closes #11606

7 years ago11308: Stop test suite if api etc. cannot be started.
Tom Clegg [Wed, 3 May 2017 14:20:35 +0000 (10:20 -0400)]
11308: Stop test suite if api etc. cannot be started.

7 years ago11469: All {kind: tmp} mounts set up under host tempdir, don't try to use
Peter Amstutz [Wed, 3 May 2017 13:52:41 +0000 (09:52 -0400)]
11469: All {kind: tmp} mounts set up under host tempdir, don't try to use
Docker volumes.

7 years ago11369: Fix tests. Tweak test fixture to include a mounts entry with capacity.
Peter Amstutz [Wed, 3 May 2017 02:53:28 +0000 (22:53 -0400)]
11369: Fix tests.  Tweak test fixture to include a mounts entry with capacity.

7 years ago11369: Log node sizes at startup. Fix setting of scratch disk size.
Peter Amstutz [Wed, 3 May 2017 01:26:55 +0000 (21:26 -0400)]
11369: Log node sizes at startup.  Fix setting of scratch disk size.

7 years ago10645: add test for container_request display with input mounts
radhika [Wed, 3 May 2017 04:24:41 +0000 (00:24 -0400)]
10645: add test for container_request display with input mounts

7 years ago11369: Containers API: Add capacity to tmp and output mounts. Also support {outputDi...
Peter Amstutz [Wed, 3 May 2017 03:06:28 +0000 (23:06 -0400)]
11369: Containers API:  Add capacity to tmp and output mounts. Also support {outputDirType: keep_output_dir} of RuntimeConstraints.

7 years ago11369: RAM request includes keep cache RAM. Add request for temp disk.
Peter Amstutz [Wed, 3 May 2017 02:21:36 +0000 (22:21 -0400)]
11369: RAM request includes keep cache RAM.  Add request for temp disk.

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Wed, 3 May 2017 01:05:25 +0000 (21:05 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago8019: need to check and write log if things are closing down
radhika [Tue, 2 May 2017 20:03:28 +0000 (16:03 -0400)]
8019: need to check and write log if things are closing down

7 years ago11308: Merge branch 'master' into 11308-python3
Tom Clegg [Tue, 2 May 2017 17:02:32 +0000 (13:02 -0400)]
11308: Merge branch 'master' into 11308-python3

Conflicts:
sdk/python/arvados/arvfile.py
sdk/python/arvados/commands/put.py
sdk/python/tests/test_collections.py

7 years agoMerge branch '11185-wb-disable-reuse'
Lucas Di Pentima [Tue, 2 May 2017 15:44:26 +0000 (12:44 -0300)]
Merge branch '11185-wb-disable-reuse'
Closes #11185

7 years ago11606: Disable deep_munge behavior. Rename test so it runs in test suite.
Tom Clegg [Tue, 2 May 2017 15:39:52 +0000 (11:39 -0400)]
11606: Disable deep_munge behavior. Rename test so it runs in test suite.

7 years agoMerge branch '11579-arvput-follow-symlinks'
Lucas Di Pentima [Tue, 2 May 2017 15:24:14 +0000 (12:24 -0300)]
Merge branch '11579-arvput-follow-symlinks'
Closes #11579

7 years ago11185: Merge branch 'master' into 11185-wb-disable-reuse
Lucas Di Pentima [Tue, 2 May 2017 14:56:56 +0000 (11:56 -0300)]
11185: Merge branch 'master' into 11185-wb-disable-reuse

7 years ago11579: Merge branch 'master' into 11579-arvput-follow-symlinks
Lucas Di Pentima [Tue, 2 May 2017 14:55:02 +0000 (11:55 -0300)]
11579: Merge branch 'master' into 11579-arvput-follow-symlinks

7 years ago11579: Added assertion on test to check for file symlinks to be copied by default.
Lucas Di Pentima [Tue, 2 May 2017 14:52:44 +0000 (11:52 -0300)]
11579: Added assertion on test to check for file symlinks to be copied by default.
Don't treat a PathDoesNotExist exception as an abnormal termination, just log the proper error message, and quit.

7 years ago10645: container request mounts display
radhika [Tue, 2 May 2017 01:43:37 +0000 (21:43 -0400)]
10645: container request mounts display

7 years ago11579: Removed feature about not uploading a symlinked dir twice.
Lucas Di Pentima [Mon, 1 May 2017 21:05:02 +0000 (18:05 -0300)]
11579: Removed feature about not uploading a symlinked dir twice.
When using --no-follow-links parameter, symlinked files will be also ignored.
Updated tests.

7 years ago11185: Removed newly added completed-acr fixture and modified the already existing...
Lucas Di Pentima [Mon, 1 May 2017 19:58:48 +0000 (16:58 -0300)]
11185: Removed newly added completed-acr fixture and modified the already existing completed-older CR
to be able to use it on the tests.
Change the reset_form() JS function to something more specific to this usage, to avoid future
conflicts.

7 years agoMerge branch '10988-document-ubuntu1604-support'
Tom Clegg [Mon, 1 May 2017 18:03:24 +0000 (14:03 -0400)]
Merge branch '10988-document-ubuntu1604-support'

closes #10988

7 years ago10988: Remove apt.arvados.org repo priority: we no longer publish/require the older...
Tom Clegg [Mon, 1 May 2017 17:57:43 +0000 (13:57 -0400)]
10988: Remove apt.arvados.org repo priority: we no longer publish/require the older packages.

7 years ago10988: Document our Ubuntu 16.04 support.
Ward Vandewege [Thu, 26 Jan 2017 22:33:05 +0000 (17:33 -0500)]
10988: Document our Ubuntu 16.04 support.

7 years agoMerge branch 'master' into origin-8019-crunchrun-log-throttle
Peter Amstutz [Mon, 1 May 2017 17:29:49 +0000 (13:29 -0400)]
Merge branch 'master' into origin-8019-crunchrun-log-throttle

7 years agoMerge branch '11469-volume-tmp' refs #11469
Peter Amstutz [Mon, 1 May 2017 16:44:54 +0000 (12:44 -0400)]
Merge branch '11469-volume-tmp' refs #11469

7 years agoCrunch script doesn't fail in file/directory literals that don't have a 'location...
Peter Amstutz [Mon, 1 May 2017 14:36:44 +0000 (10:36 -0400)]
Crunch script doesn't fail in file/directory literals that don't have a 'location'. refs #11549

7 years ago11469: Docker-managed volumes go in "Volumes" not "Binds".
Peter Amstutz [Mon, 1 May 2017 14:02:56 +0000 (10:02 -0400)]
11469: Docker-managed volumes go in "Volumes" not "Binds".

7 years ago11185: Addressed several requests:
Lucas Di Pentima [Fri, 28 Apr 2017 19:18:56 +0000 (16:18 -0300)]
11185: Addressed several requests:
* Added '...' to 'Re-run' button's title to indicate it will show a dialog.
* Corrected comment.
* Added javascript code so that the 'Cancel' and 'x' buttons on the dialog reset the checkbox.
* Expanded test cases to cover CRs that both do and don't use arvados-cwl-runner command, with
and without container reuse.

7 years ago8019: Rework partial line throttling. Fix sending flush when buffer is ready
Peter Amstutz [Fri, 28 Apr 2017 18:50:38 +0000 (14:50 -0400)]
8019: Rework partial line throttling.  Fix sending flush when buffer is ready
so it does not block.

7 years agoMerge branch '11537-keepproxy-loop'
Tom Clegg [Fri, 28 Apr 2017 16:59:08 +0000 (12:59 -0400)]
Merge branch '11537-keepproxy-loop'

closes #11537

7 years ago11579: Check if paths exist, raise meaningful exception if not.
Lucas Di Pentima [Fri, 28 Apr 2017 15:25:01 +0000 (12:25 -0300)]
11579: Check if paths exist, raise meaningful exception if not.
Added test for this case.

7 years agoMerge branch '11549-cwl-smarter-mounts' closes #11549
Peter Amstutz [Fri, 28 Apr 2017 14:40:06 +0000 (10:40 -0400)]
Merge branch '11549-cwl-smarter-mounts' closes #11549

7 years ago11537: Add Via header to get/head/post/put responses.
Tom Clegg [Thu, 27 Apr 2017 19:24:41 +0000 (15:24 -0400)]
11537: Add Via header to get/head/post/put responses.

7 years ago11537: Add Via header to proxied keepstore requests.
Tom Clegg [Thu, 27 Apr 2017 14:51:49 +0000 (10:51 -0400)]
11537: Add Via header to proxied keepstore requests.

Propagate keepclient PUT error messages to caller.

7 years agoMerge branch '7709-rails-observers'
Tom Clegg [Thu, 27 Apr 2017 20:55:45 +0000 (16:55 -0400)]
Merge branch '7709-rails-observers'

refs #7709

7 years ago7709: Add rails-observers gem so plugins can use it.
Tom Clegg [Thu, 27 Apr 2017 20:51:46 +0000 (16:51 -0400)]
7709: Add rails-observers gem so plugins can use it.

7 years ago11579: Explicitly following links on test.
Lucas Di Pentima [Thu, 27 Apr 2017 20:15:53 +0000 (17:15 -0300)]
11579: Explicitly following links on test.

7 years ago11579: Tests added.
Lucas Di Pentima [Thu, 27 Apr 2017 20:12:56 +0000 (17:12 -0300)]
11579: Tests added.

7 years ago11579: Added symlinked dir traversal as the default behavior.
Lucas Di Pentima [Thu, 27 Apr 2017 19:51:31 +0000 (16:51 -0300)]
11579: Added symlinked dir traversal as the default behavior.
Added flag to disable this and ignore symlinked dirs: --no-follow-links.

7 years ago8019: partial line throttling etc
radhika [Thu, 27 Apr 2017 19:44:37 +0000 (15:44 -0400)]
8019: partial line throttling etc

7 years ago11549: Fix container requests so it doesn't mount each file individually,
Peter Amstutz [Thu, 27 Apr 2017 18:24:43 +0000 (14:24 -0400)]
11549: Fix container requests so it doesn't mount each file individually,
instead mount common collections/subdirectories.

7 years ago11549: Test that containers don't mount each file individually, instead mount
Peter Amstutz [Thu, 27 Apr 2017 18:05:00 +0000 (14:05 -0400)]
11549: Test that containers don't mount each file individually, instead mount
common collections/subdirectories.

7 years agoMerge branch '7709-api-rails4'
Tom Clegg [Thu, 27 Apr 2017 16:59:06 +0000 (12:59 -0400)]
Merge branch '7709-api-rails4'

closes #7709

7 years ago11579: Test proving the bug: arv-put does not follow symlinks.
Lucas Di Pentima [Thu, 27 Apr 2017 16:54:32 +0000 (13:54 -0300)]
11579: Test proving the bug: arv-put does not follow symlinks.

7 years ago11185: When re-run, the default behaviour is to copy reuse disabled. Changed modal...
Lucas Di Pentima [Thu, 27 Apr 2017 15:42:49 +0000 (12:42 -0300)]
11185: When re-run, the default behaviour is to copy reuse disabled. Changed modal dialog to allow
the user to enable reuse.
Updated tests.

7 years ago11185: Now we only have one "Re-run" button that will give the user the option to...
Lucas Di Pentima [Thu, 27 Apr 2017 13:55:26 +0000 (10:55 -0300)]
11185: Now we only have one "Re-run" button that will give the user the option to disable container reuse on a modal dialog.
Updated test.

7 years agoMerge branch '10111-cr-provenance-graph'
Lucas Di Pentima [Wed, 26 Apr 2017 18:56:00 +0000 (15:56 -0300)]
Merge branch '10111-cr-provenance-graph'
Closes #10111

7 years ago10111: Merge branch 'master' into 10111-cr-provenance-graph
Lucas Di Pentima [Wed, 26 Apr 2017 18:55:03 +0000 (15:55 -0300)]
10111: Merge branch 'master' into 10111-cr-provenance-graph

7 years ago10111: Replaced graph edges names: cr->child & mounts->input.
Lucas Di Pentima [Wed, 26 Apr 2017 18:29:20 +0000 (15:29 -0300)]
10111: Replaced graph edges names: cr->child & mounts->input.

7 years agoMerge branch '11305-migrate-docker19-doc'
Nico Cesar [Wed, 26 Apr 2017 18:25:22 +0000 (14:25 -0400)]
Merge branch '11305-migrate-docker19-doc'

Documentation for the migration process from Docker 1.9 or less  to Docker 1.10+

Docker changed the format of the images, and there is an arvados tool arvados-migrate-docker19,
this commit explains how to migrate

refs #11305

7 years ago10111: If there is no cwl.input.json object as part of a CR mount list, search for...
Lucas Di Pentima [Wed, 26 Apr 2017 16:28:06 +0000 (13:28 -0300)]
10111: If there is no cwl.input.json object as part of a CR mount list, search for input PDHs on the entire mount list

7 years ago7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg [Wed, 26 Apr 2017 14:33:13 +0000 (10:33 -0400)]
7709: Merge branch 'master' into 7709-api-rails4

7 years ago11185: On re-run without reuse, assign 'false' to CR.use_existing. Added related...
Lucas Di Pentima [Wed, 26 Apr 2017 14:03:35 +0000 (11:03 -0300)]
11185: On re-run without reuse, assign 'false' to CR.use_existing. Added related test.

7 years ago10111: Only include those mounts that are declared on the cwl input definition.
Lucas Di Pentima [Wed, 26 Apr 2017 11:56:01 +0000 (08:56 -0300)]
10111: Only include those mounts that are declared on the cwl input definition.
Removed trailing spaces.

7 years ago11185: Added "Re-run without reuse" button to the completed CR page, so that it
Lucas Di Pentima [Wed, 26 Apr 2017 02:04:49 +0000 (23:04 -0300)]
11185: Added "Re-run without reuse" button to the completed CR page, so that it
copies the CR modifying the arvados-cwl-runner arguments to avoid container
reuse.
Expanded related test.

7 years ago8019: load log throttling config params during NewContainerRunner
radhika [Tue, 25 Apr 2017 20:43:47 +0000 (16:43 -0400)]
8019: load log throttling config params during NewContainerRunner

7 years ago10111: Added "child" CRs to the graph, and removed container image and requesting...
Lucas Di Pentima [Tue, 25 Apr 2017 19:26:38 +0000 (16:26 -0300)]
10111: Added "child" CRs to the graph, and removed container image and requesting container elements.

7 years ago8019: rateLimit method signature
radhika [Tue, 25 Apr 2017 19:05:31 +0000 (15:05 -0400)]
8019: rateLimit method signature

7 years agoMerge branch 'master' into 8019-crunchrun-log-throttle
radhika [Tue, 25 Apr 2017 18:08:14 +0000 (14:08 -0400)]
Merge branch 'master' into 8019-crunchrun-log-throttle

7 years ago8019: load rate limiting config parameters from discovery document once per log write...
radhika [Tue, 25 Apr 2017 18:06:55 +0000 (14:06 -0400)]
8019: load rate limiting config parameters from discovery document once per log writer creation.

7 years agoMerge branch '11544-bad-collection-id-500'
Tom Clegg [Tue, 25 Apr 2017 17:22:44 +0000 (13:22 -0400)]
Merge branch '11544-bad-collection-id-500'

closes #11544

7 years ago11544: Return 404 instead of 500 for malformed collection IDs.
Tom Clegg [Tue, 25 Apr 2017 16:57:11 +0000 (12:57 -0400)]
11544: Return 404 instead of 500 for malformed collection IDs.

7 years agoMerge branch '11507-deleted-bufferblocks' refs #11507
Peter Amstutz [Tue, 25 Apr 2017 15:58:34 +0000 (11:58 -0400)]
Merge branch '11507-deleted-bufferblocks' refs #11507

7 years ago11507: Move repack_writes() method from ArvadosFile to BufferBlock.
Peter Amstutz [Fri, 21 Apr 2017 20:36:21 +0000 (16:36 -0400)]
11507: Move repack_writes() method from ArvadosFile to BufferBlock.

7 years ago7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg [Tue, 25 Apr 2017 05:14:23 +0000 (01:14 -0400)]
7709: Merge branch 'master' into 7709-api-rails4

Conflicts:
services/api/db/structure.sql

7 years ago10111: Tests added
Lucas Di Pentima [Tue, 25 Apr 2017 03:50:31 +0000 (00:50 -0300)]
10111: Tests added