arvados.git
7 years ago11714: Set CgroupParent under Resources because setting Cgroup in HostConfig 11714-crunch-run-cgroup-parent
Peter Amstutz [Wed, 17 May 2017 15:07:46 +0000 (11:07 -0400)]
11714: Set CgroupParent under Resources because setting Cgroup in HostConfig
doesn't work.

7 years ago11718: Update crunch-run for docker client API change in ContainerWait().
Peter Amstutz [Wed, 17 May 2017 15:04:06 +0000 (11:04 -0400)]
11718: Update crunch-run for docker client API change in ContainerWait().

7 years agoMerge branch '11693-crunch2-output-symlinks' closes #11693
Peter Amstutz [Tue, 16 May 2017 14:52:04 +0000 (10:52 -0400)]
Merge branch '11693-crunch2-output-symlinks' closes #11693

7 years ago11693: Only check collection mounts when determining whether to call
Peter Amstutz [Tue, 16 May 2017 14:38:55 +0000 (10:38 -0400)]
11693: Only check collection mounts when determining whether to call
getCollectionManifestForPath to merge files into output collection.  Remove
redundant assignment of outputSuffix.

7 years agoMerge branch '10111-collection-labels'
Lucas Di Pentima [Tue, 16 May 2017 14:32:48 +0000 (11:32 -0300)]
Merge branch '10111-collection-labels'
Closes #10111

7 years ago10111: Merge branch 'master' into 10111-collection-labels
Lucas Di Pentima [Tue, 16 May 2017 14:32:11 +0000 (11:32 -0300)]
10111: Merge branch 'master' into 10111-collection-labels

7 years ago11693: Detect and error on symlinks pointing to locations outside the output directory.
Peter Amstutz [Tue, 16 May 2017 13:12:58 +0000 (09:12 -0400)]
11693: Detect and error on symlinks pointing to locations outside the output directory.

7 years ago11693: Add test that error capturing output results in cancelled container.
Peter Amstutz [Mon, 15 May 2017 20:58:05 +0000 (16:58 -0400)]
11693: Add test that error capturing output results in cancelled container.

7 years ago11693: Add test cases for symlinks to directories and symlinks inside subdirectories.
Peter Amstutz [Mon, 15 May 2017 20:25:39 +0000 (16:25 -0400)]
11693: Add test cases for symlinks to directories and symlinks inside subdirectories.

7 years ago10111: Moved the api request related code to the controller.
Lucas Di Pentima [Mon, 15 May 2017 20:08:11 +0000 (17:08 -0300)]
10111: Moved the api request related code to the controller.

7 years ago11693: Add tests.
Peter Amstutz [Mon, 15 May 2017 19:52:00 +0000 (15:52 -0400)]
11693: Add tests.

7 years ago11693: Dereference symlinks to input files and add incorporate them into output colle...
Peter Amstutz [Mon, 15 May 2017 18:06:33 +0000 (14:06 -0400)]
11693: Dereference symlinks to input files and add incorporate them into output collection manifest.

7 years ago11693: Mark container as cancelled if there is an error during finalization.
Peter Amstutz [Mon, 15 May 2017 14:19:20 +0000 (10:19 -0400)]
11693: Mark container as cancelled if there is an error during finalization.

7 years agoMerge branch '11590-log-reuse'
Tom Clegg [Fri, 12 May 2017 14:45:20 +0000 (10:45 -0400)]
Merge branch '11590-log-reuse'

refs #11590

7 years ago11590: Reword log messages.
Tom Clegg [Fri, 12 May 2017 14:34:29 +0000 (10:34 -0400)]
11590: Reword log messages.

7 years agoMerge branch '11647-no-perm-tokens'
Lucas Di Pentima [Fri, 12 May 2017 14:29:01 +0000 (11:29 -0300)]
Merge branch '11647-no-perm-tokens'
Refs #11647

7 years ago11647: Merge branch 'master' into 11647-no-perm-tokens
Lucas Di Pentima [Fri, 12 May 2017 14:28:00 +0000 (11:28 -0300)]
11647: Merge branch 'master' into 11647-no-perm-tokens

7 years ago11647: When repacking small blocks into big ones, commit them synchronously.
Lucas Di Pentima [Fri, 12 May 2017 14:00:55 +0000 (11:00 -0300)]
11647: When repacking small blocks into big ones, commit them synchronously.

7 years ago11647: Test proving that when small blocks get repacked and the resulting big block is
Lucas Di Pentima [Fri, 12 May 2017 13:57:58 +0000 (10:57 -0300)]
11647: Test proving that when small blocks get repacked and the resulting big block is
committed asynchronously, it doesn't get the permission token on the manifest.

7 years agoMerge branch '11681-str-newstr' refs #11681
Peter Amstutz [Thu, 11 May 2017 21:27:27 +0000 (17:27 -0400)]
Merge branch '11681-str-newstr' refs #11681

7 years ago11681: Return native str type from Collection.portable_data_hash() (previous commit...
Peter Amstutz [Thu, 11 May 2017 21:25:34 +0000 (17:25 -0400)]
11681: Return native str type from Collection.portable_data_hash()  (previous commit was actually for CollectionWriter)

7 years agoMerge branch '11681-str-newstr'
Tom Clegg [Thu, 11 May 2017 20:35:24 +0000 (16:35 -0400)]
Merge branch '11681-str-newstr'

refs #11681

7 years ago11681: Return native str type from KeepLocator.__str__() and Collection.portable_data...
Tom Clegg [Thu, 11 May 2017 19:37:32 +0000 (15:37 -0400)]
11681: Return native str type from KeepLocator.__str__() and Collection.portable_data_hash()

7 years agocloses #10645
radhika [Thu, 11 May 2017 16:54:53 +0000 (12:54 -0400)]
closes #10645
Merge branch '10645-cr-mounts-display'

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Thu, 11 May 2017 16:54:19 +0000 (12:54 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago11590: Add "reuse jobs even if conflicting outputs exist" and "log reuse decisions...
Tom Clegg [Thu, 11 May 2017 15:38:37 +0000 (11:38 -0400)]
11590: Add "reuse jobs even if conflicting outputs exist" and "log reuse decisions" config options.

7 years agoMerge branch '11369-cwl-keepcache-ram' refs #11369
Peter Amstutz [Thu, 11 May 2017 15:00:38 +0000 (11:00 -0400)]
Merge branch '11369-cwl-keepcache-ram' refs #11369

7 years agoMerge branch '11626-crunch2-log-slurm-errors' closes #11626
Peter Amstutz [Thu, 11 May 2017 14:19:32 +0000 (10:19 -0400)]
Merge branch '11626-crunch2-log-slurm-errors' closes #11626

7 years ago11626: Fix log.Printf -> log.Print
Peter Amstutz [Thu, 11 May 2017 14:19:05 +0000 (10:19 -0400)]
11626: Fix log.Printf -> log.Print

7 years ago10645: display collection link with disposition. preload mount and command collections.
radhika [Thu, 11 May 2017 12:54:59 +0000 (08:54 -0400)]
10645: display collection link with disposition. preload mount and command collections.

7 years agoBe smarter about how we handle packages built in previous runs.
Ward Vandewege [Thu, 11 May 2017 01:10:02 +0000 (21:10 -0400)]
Be smarter about how we handle packages built in previous runs.

This commit fixes a bug: packages that were built but failed testing
will no longer be uploaded later, when a subsequent set of packages
passes the tests.

No issue #

7 years agoThe Workbench package has a build-time dependency on tzdata for
Ward Vandewege [Wed, 10 May 2017 21:09:06 +0000 (17:09 -0400)]
The Workbench package has a build-time dependency on tzdata for
Ubuntu1604, which is not included in the default upstream docker image.
Add it to our list of dependencies.

Also tweak Workbench package dependencies, no need to expliclitly list
g++ for CentOS7.

No issue #

7 years agoFix Workbench package dependency.
Ward Vandewege [Wed, 10 May 2017 20:21:45 +0000 (16:21 -0400)]
Fix Workbench package dependency.

No issue #

7 years agoOur API server and Workbench packages depend on 2 gems that compile
Ward Vandewege [Wed, 10 May 2017 19:11:26 +0000 (15:11 -0400)]
Our API server and Workbench packages depend on 2 gems that compile
native objects, which require g++ to build them: event-machine and
therubyracer. For our debian-based distros, that package is not
necessarily installed, so we're adding it to the dependency list.

No issue #

7 years ago11369: Add keep_cache to base RAM request.
Peter Amstutz [Wed, 10 May 2017 18:57:23 +0000 (14:57 -0400)]
11369: Add keep_cache to base RAM request.

7 years agoMerge branch '10827-checker-cwltool-update' refs #10827
Peter Amstutz [Wed, 10 May 2017 18:50:15 +0000 (14:50 -0400)]
Merge branch '10827-checker-cwltool-update' refs #10827

7 years ago10827: Bump cwltool & schema salad dependency.
Peter Amstutz [Wed, 10 May 2017 15:16:18 +0000 (11:16 -0400)]
10827: Bump cwltool & schema salad dependency.

7 years agocloses #10112
radhika [Wed, 10 May 2017 16:56:30 +0000 (12:56 -0400)]
closes #10112
Merge branch '10112-workflow-show'

7 years agoMerge branch 'master' into 10112-workflow-show
radhika [Wed, 10 May 2017 16:55:21 +0000 (12:55 -0400)]
Merge branch 'master' into 10112-workflow-show

7 years ago10112: preload collections
radhika [Wed, 10 May 2017 15:45:23 +0000 (11:45 -0400)]
10112: preload collections

7 years ago10827: Add --validate, --print-dot and --enable-dev options (pass through to
Peter Amstutz [Wed, 10 May 2017 15:08:36 +0000 (11:08 -0400)]
10827: Add --validate, --print-dot and --enable-dev options (pass through to
existing cwltool functionality).

7 years agoMerge branch '11629-groups-contents-memory'
Tom Clegg [Wed, 10 May 2017 14:23:20 +0000 (10:23 -0400)]
Merge branch '11629-groups-contents-memory'

refs #11629
refs #11458

7 years ago11629: Fix ambiguous column reference.
Tom Clegg [Wed, 10 May 2017 14:22:42 +0000 (10:22 -0400)]
11629: Fix ambiguous column reference.

7 years agoMerge branch '11634-bundler-cache'
Tom Clegg [Tue, 9 May 2017 20:37:47 +0000 (16:37 -0400)]
Merge branch '11634-bundler-cache'

closes #11634

7 years agoMerge branch '11629-groups-contents-memory'
Tom Clegg [Tue, 9 May 2017 20:33:31 +0000 (16:33 -0400)]
Merge branch '11629-groups-contents-memory'

closes #11629

7 years ago11626: Log sbatch errors where the user can see them. Fix tests so there is no linge...
Peter Amstutz [Fri, 5 May 2017 21:11:39 +0000 (17:11 -0400)]
11626: Log sbatch errors where the user can see them.  Fix tests so there is no lingering goroutines.

7 years agoAdd libgnutls28-dev and python3-dev to install package list to satisfy
Peter Amstutz [Tue, 9 May 2017 20:00:39 +0000 (16:00 -0400)]
Add libgnutls28-dev and python3-dev to install package list to satisfy
run-tests.sh dependencies.  refs #11308

7 years agoMerge branch '11308-collection-keys'
Tom Clegg [Tue, 9 May 2017 19:49:41 +0000 (15:49 -0400)]
Merge branch '11308-collection-keys'

refs #11308

7 years ago11308: Fix modes not tested in test case.
Tom Clegg [Tue, 9 May 2017 19:31:44 +0000 (15:31 -0400)]
11308: Fix modes not tested in test case.

7 years ago11308: collection.keys() supports indexing for python2 callers.
Tom Clegg [Tue, 9 May 2017 19:30:19 +0000 (15:30 -0400)]
11308: collection.keys() supports indexing for python2 callers.

7 years ago10112: use show_text_with_locators partial to display the various attributes of the...
radhika [Tue, 9 May 2017 15:29:02 +0000 (11:29 -0400)]
10112: use show_text_with_locators partial to display the various attributes of the definition

7 years agoMerge branch 'master' into 10112-workflow-show
radhika [Tue, 9 May 2017 14:35:38 +0000 (10:35 -0400)]
Merge branch 'master' into 10112-workflow-show

7 years agorefs #10645
radhika [Tue, 9 May 2017 14:31:06 +0000 (10:31 -0400)]
refs #10645
Merge branch '10645-cr-mounts-display'

7 years ago10645: added comment
radhika [Tue, 9 May 2017 14:28:23 +0000 (10:28 -0400)]
10645: added comment

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Tue, 9 May 2017 11:35:16 +0000 (07:35 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago10645: refactor the display of text with locators into a partial
radhika [Tue, 9 May 2017 11:33:51 +0000 (07:33 -0400)]
10645: refactor the display of text with locators into a partial

7 years ago10645: collection links in command
radhika [Tue, 9 May 2017 01:58:07 +0000 (21:58 -0400)]
10645: collection links in command

7 years ago10645: regexp for keep_locator_in_json format
radhika [Tue, 9 May 2017 01:21:02 +0000 (21:21 -0400)]
10645: regexp for keep_locator_in_json format

7 years ago10645: test update
radhika [Mon, 8 May 2017 22:56:53 +0000 (18:56 -0400)]
10645: test update

7 years ago10645: display PDHs in command as clickable collections.
radhika [Mon, 8 May 2017 22:43:54 +0000 (18:43 -0400)]
10645: display PDHs in command as clickable collections.

7 years ago10645: display keep locator in mounts as pdh/file_name
radhika [Mon, 8 May 2017 21:34:01 +0000 (17:34 -0400)]
10645: display keep locator in mounts as pdh/file_name

7 years agoMerge branch '11638-limit-conns'
Tom Clegg [Mon, 8 May 2017 20:04:59 +0000 (16:04 -0400)]
Merge branch '11638-limit-conns'

closes #11638

7 years ago11638: Log a warning if Postgres pool size is unlimited.
Tom Clegg [Mon, 8 May 2017 20:04:01 +0000 (16:04 -0400)]
11638: Log a warning if Postgres pool size is unlimited.

7 years ago10645: Tweak so that left column is col-md-6 (instead of col-md-5). Also tweak
Peter Amstutz [Mon, 8 May 2017 19:14:58 +0000 (15:14 -0400)]
10645: Tweak so that left column is col-md-6 (instead of col-md-5).  Also tweak
full width part to be col-md-12.  Move "priority" back to right hand column.

7 years ago11638: Add PostgresPool config option, and default application_name
Tom Clegg [Mon, 8 May 2017 19:02:54 +0000 (15:02 -0400)]
11638: Add PostgresPool config option, and default application_name

7 years ago11629: Apply memory limits to job/pipeline "components" fields.
Tom Clegg [Mon, 8 May 2017 18:17:46 +0000 (14:17 -0400)]
11629: Apply memory limits to job/pipeline "components" fields.

7 years ago11629: Fix items_available in groups#contents response.
Tom Clegg [Mon, 8 May 2017 18:17:16 +0000 (14:17 -0400)]
11629: Fix items_available in groups#contents response.

7 years ago10645: L&F improvements
radhika [Mon, 8 May 2017 17:59:35 +0000 (13:59 -0400)]
10645: L&F improvements

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Mon, 8 May 2017 16:42:10 +0000 (12:42 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years agoMerge branch '11308-python3'
Tom Clegg [Mon, 8 May 2017 16:06:00 +0000 (12:06 -0400)]
Merge branch '11308-python3'

refs #11308

7 years ago11308: Merge branch 'master' into 11308-python3
Tom Clegg [Mon, 8 May 2017 16:05:31 +0000 (12:05 -0400)]
11308: Merge branch 'master' into 11308-python3

7 years ago10111: When dealing with output collection nodes, render the specific collection...
Lucas Di Pentima [Mon, 8 May 2017 14:52:55 +0000 (11:52 -0300)]
10111: When dealing with output collection nodes, render the specific collection url instead of use its pdh

7 years ago10111: Refactored the graph creation code to minimize the amount of API calls neeeded.
Lucas Di Pentima [Mon, 8 May 2017 14:31:06 +0000 (11:31 -0300)]
10111: Refactored the graph creation code to minimize the amount of API calls neeeded.

7 years ago11634: Prevent bundler from trying to use/update system gem cache.
Tom Clegg [Mon, 8 May 2017 14:07:21 +0000 (10:07 -0400)]
11634: Prevent bundler from trying to use/update system gem cache.

7 years ago11634: Fix whitespace.
Tom Clegg [Mon, 8 May 2017 13:59:35 +0000 (09:59 -0400)]
11634: Fix whitespace.

7 years ago10645: improve location regex usage
radhika [Mon, 8 May 2017 13:18:19 +0000 (09:18 -0400)]
10645: improve location regex usage

7 years ago10645: table width 100%
radhika [Fri, 5 May 2017 19:35:32 +0000 (15:35 -0400)]
10645: table width 100%

7 years ago11629: Limit database reads for all list responses, not just index.
Tom Clegg [Fri, 5 May 2017 18:09:54 +0000 (14:09 -0400)]
11629: Limit database reads for all list responses, not just index.

7 years ago10112: add Definition tab to workflow page
radhika [Fri, 5 May 2017 19:30:46 +0000 (15:30 -0400)]
10112: add Definition tab to workflow page

7 years agoMerge branch 'master' into 10112-workflow-show
radhika [Fri, 5 May 2017 17:38:21 +0000 (13:38 -0400)]
Merge branch 'master' into 10112-workflow-show

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 17:30:38 +0000 (13:30 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago10645: corrections to test fixture
radhika [Fri, 5 May 2017 17:29:59 +0000 (13:29 -0400)]
10645: corrections to test fixture

7 years agocloses #8019
radhika [Fri, 5 May 2017 16:05:11 +0000 (12:05 -0400)]
closes #8019
Merge branch '8019-crunchrun-log-throttle'

7 years agoMerge branch 'master' into 8019-crunchrun-log-throttle
radhika [Fri, 5 May 2017 16:02:36 +0000 (12:02 -0400)]
Merge branch 'master' into 8019-crunchrun-log-throttle

7 years ago8019: Make flush channel buffered again (because we want to be able to do back-to...
Peter Amstutz [Fri, 5 May 2017 15:54:13 +0000 (11:54 -0400)]
8019: Make flush channel buffered again (because we want to be able to do back-to-back flushes).
Fix "> crunchLogBytesPerEvent" to ">= crunchLogBytesPerEvent" for consistency.

7 years ago10645: small style updates
radhika [Fri, 5 May 2017 15:12:48 +0000 (11:12 -0400)]
10645: small style updates

7 years ago10111: On output collections, name the collection node after the output_uuid reference.
Lucas Di Pentima [Fri, 5 May 2017 14:43:47 +0000 (11:43 -0300)]
10111: On output collections, name the collection node after the output_uuid reference.

7 years ago10111: Fix integration test
Lucas Di Pentima [Fri, 5 May 2017 13:49:49 +0000 (10:49 -0300)]
10111: Fix integration test

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 13:44:53 +0000 (09:44 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Fri, 5 May 2017 01:32:37 +0000 (21:32 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago10645: mount inputs display
radhika [Fri, 5 May 2017 01:31:18 +0000 (21:31 -0400)]
10645: mount inputs display

7 years ago10111: Better labels on collection on the provenance graph.
Lucas Di Pentima [Thu, 4 May 2017 21:37:09 +0000 (18:37 -0300)]
10111: Better labels on collection on the provenance graph.

7 years agoMerge branch '10111-cr-prov-regression-fixes'
Lucas Di Pentima [Thu, 4 May 2017 18:30:52 +0000 (15:30 -0300)]
Merge branch '10111-cr-prov-regression-fixes'
Refs #10111

7 years ago10111: Flipped arrow between the cr and its childs cr.
Lucas Di Pentima [Thu, 4 May 2017 18:26:48 +0000 (15:26 -0300)]
10111: Flipped arrow between the cr and its childs cr.

7 years ago8019: uncomment the test statement
radhika [Thu, 4 May 2017 17:32:34 +0000 (13:32 -0400)]
8019: uncomment the test statement

7 years ago10111: Only add cr input collections by PDH.
Lucas Di Pentima [Thu, 4 May 2017 17:17:54 +0000 (14:17 -0300)]
10111: Only add cr input collections by PDH.

7 years ago8019: Use tl.stopping channel instead of tl.flush to signal Close. Use unbuffered...
Tom Clegg [Thu, 4 May 2017 17:02:40 +0000 (13:02 -0400)]
8019: Use tl.stopping channel instead of tl.flush to signal Close. Use unbuffered channel to signal flush.

7 years ago10111: Skip log outputs from the provenance graph.
Lucas Di Pentima [Thu, 4 May 2017 15:42:19 +0000 (12:42 -0300)]
10111: Skip log outputs from the provenance graph.
Container request's outputs are now added to the graph by their PDHs so that they connect with the CR childs inputs.

7 years ago11308: Merge branch 'master' into 11308-python3
Tom Clegg [Thu, 4 May 2017 15:16:42 +0000 (11:16 -0400)]
11308: Merge branch 'master' into 11308-python3

7 years ago11308: Fix bytes vs. strings from recent merge.
Tom Clegg [Thu, 4 May 2017 15:09:41 +0000 (11:09 -0400)]
11308: Fix bytes vs. strings from recent merge.