arvados.git
7 years agoMerge branch '11720-govendor'
Tom Clegg [Sun, 21 May 2017 18:33:08 +0000 (14:33 -0400)]
Merge branch '11720-govendor'

closes #11720

7 years agoMerge branch '11590-log-reuse'
Tom Clegg [Fri, 19 May 2017 23:36:56 +0000 (19:36 -0400)]
Merge branch '11590-log-reuse'

closes #11590

7 years agoMerge branch '9005-disable-keepalive'
Tom Clegg [Fri, 19 May 2017 23:17:22 +0000 (19:17 -0400)]
Merge branch '9005-disable-keepalive'

refs #9005
refs #11726
refs #11729

7 years ago9005: 11726: 11729: Disable http keepalive.
Tom Clegg [Fri, 19 May 2017 23:11:05 +0000 (19:11 -0400)]
9005: 11726: 11729: Disable http keepalive.

The previous workaround for #9005 did not account for the prefetch
feature: if a goroutine is using the HTTP client to prefetch data at
the moment the handler exits, CloseIdleConnections() does not close
that connection, so it stays open indefinitely.

7 years agoMerge branch '9005-conn-leak'
Tom Clegg [Fri, 19 May 2017 20:23:39 +0000 (16:23 -0400)]
Merge branch '9005-conn-leak'

refs #9005

7 years ago9005: Fix missing error checks.
Tom Clegg [Fri, 19 May 2017 19:55:40 +0000 (15:55 -0400)]
9005: Fix missing error checks.

7 years ago9005: Fix missing Close().
Tom Clegg [Fri, 19 May 2017 19:55:22 +0000 (15:55 -0400)]
9005: Fix missing Close().

7 years agocloses #11710
radhika [Fri, 19 May 2017 19:34:32 +0000 (15:34 -0400)]
closes #11710
Merge branch '11710-container-request-show-perf'

7 years ago11710: fix typo in finding children
radhika [Fri, 19 May 2017 18:30:58 +0000 (14:30 -0400)]
11710: fix typo in finding children

7 years ago11710: fetching requesting containers
radhika [Thu, 18 May 2017 19:38:52 +0000 (15:38 -0400)]
11710: fetching requesting containers

7 years ago11710: preload / batch retrieval of children of a container_work_unit
radhika [Thu, 18 May 2017 15:42:25 +0000 (11:42 -0400)]
11710: preload / batch retrieval of children of a container_work_unit

7 years ago9005: Remove debug printf.
Tom Clegg [Fri, 19 May 2017 17:47:52 +0000 (13:47 -0400)]
9005: Remove debug printf.

7 years ago11590: Dry up log_reuse_info() calls.
Tom Clegg [Thu, 18 May 2017 21:25:20 +0000 (17:25 -0400)]
11590: Dry up log_reuse_info() calls.

7 years agorefs #9587
radhika [Thu, 18 May 2017 19:46:15 +0000 (15:46 -0400)]
refs #9587
Merge branch '9587-include-trash-in-group-contents'

7 years ago9587: add support for include_trash in groups_controller -> contents method
radhika [Thu, 18 May 2017 01:47:52 +0000 (21:47 -0400)]
9587: add support for include_trash in groups_controller -> contents method

7 years ago11590: Add container logging tests.
Tom Clegg [Thu, 18 May 2017 18:53:38 +0000 (14:53 -0400)]
11590: Add container logging tests.

7 years agoMerge branch '11644-mounts-api'
Tom Clegg [Thu, 18 May 2017 18:45:00 +0000 (14:45 -0400)]
Merge branch '11644-mounts-api'

closes #11644

7 years ago11644: Add DeviceID() to Volume interface.
Tom Clegg [Thu, 18 May 2017 17:26:36 +0000 (13:26 -0400)]
11644: Add DeviceID() to Volume interface.

7 years ago11644: Unify block-index handlers. Move prefix arg to query string.
Tom Clegg [Thu, 18 May 2017 17:22:54 +0000 (13:22 -0400)]
11644: Unify block-index handlers. Move prefix arg to query string.

7 years ago11590: Log container reuse decisions.
Tom Clegg [Thu, 18 May 2017 16:19:13 +0000 (12:19 -0400)]
11590: Log container reuse decisions.

7 years agoMerge branch '11590-log-reuse'
Tom Clegg [Thu, 18 May 2017 14:26:53 +0000 (10:26 -0400)]
Merge branch '11590-log-reuse'

refs #11590

7 years ago11590: Clarify "job state" condition in log message.
Tom Clegg [Thu, 18 May 2017 14:26:35 +0000 (10:26 -0400)]
11590: Clarify "job state" condition in log message.

7 years ago11720: Add vendor/.gitignore.
Tom Clegg [Thu, 18 May 2017 14:21:56 +0000 (10:21 -0400)]
11720: Add vendor/.gitignore.

7 years ago11720: Update Go dependencies.
Tom Clegg [Wed, 17 May 2017 21:07:41 +0000 (17:07 -0400)]
11720: Update Go dependencies.

7 years ago11720: Merge branch 'master' into 11720-govendor
Tom Clegg [Wed, 17 May 2017 20:48:09 +0000 (16:48 -0400)]
11720: Merge branch 'master' into 11720-govendor

7 years agoMerge branch '11546-fast-lock'
Tom Clegg [Wed, 17 May 2017 20:47:20 +0000 (16:47 -0400)]
Merge branch '11546-fast-lock'

closes #11546

7 years ago11720: Control dependencies with govendor.
Tom Clegg [Wed, 17 May 2017 19:26:26 +0000 (15:26 -0400)]
11720: Control dependencies with govendor.

7 years agocloses #11580
radhika [Wed, 17 May 2017 18:28:43 +0000 (14:28 -0400)]
closes #11580
Merge branch '11580-container-requests'

7 years ago11546: Wrap lock/unlock in transactions.
Tom Clegg [Wed, 17 May 2017 18:28:33 +0000 (14:28 -0400)]
11546: Wrap lock/unlock in transactions.

7 years ago11580: preload containers
radhika [Tue, 16 May 2017 19:33:31 +0000 (15:33 -0400)]
11580: preload containers

7 years ago11580: add container_requests index page
radhika [Tue, 16 May 2017 00:09:25 +0000 (20:09 -0400)]
11580: add container_requests index page

7 years ago11546: Avoid loading/saving non-essential fields in /arvados/v1/containers/lock.
Tom Clegg [Wed, 17 May 2017 17:51:57 +0000 (13:51 -0400)]
11546: Avoid loading/saving non-essential fields in /arvados/v1/containers/lock.

7 years agoMerge branch '11714-crunch-run-cgroup-parent' closes #11714
Peter Amstutz [Wed, 17 May 2017 17:50:03 +0000 (13:50 -0400)]
Merge branch '11714-crunch-run-cgroup-parent' closes #11714

7 years ago11714: Set CgroupParent under Resources because setting Cgroup in HostConfig
Peter Amstutz [Wed, 17 May 2017 15:07:46 +0000 (11:07 -0400)]
11714: Set CgroupParent under Resources because setting Cgroup in HostConfig
doesn't work.

7 years agoMerge branch '11718-crunch-run-docker-wait' closes #11718
Peter Amstutz [Wed, 17 May 2017 17:08:35 +0000 (13:08 -0400)]
Merge branch '11718-crunch-run-docker-wait' closes #11718

7 years ago11718: Update crunch-run for docker client API change in ContainerWait().
Peter Amstutz [Wed, 17 May 2017 15:04:06 +0000 (11:04 -0400)]
11718: Update crunch-run for docker client API change in ContainerWait().

7 years ago11644: Test fields in /mounts response.
Tom Clegg [Tue, 16 May 2017 17:31:27 +0000 (13:31 -0400)]
11644: Test fields in /mounts response.

7 years ago11644: Ensure generated UUIDs are always 27 chars.
Tom Clegg [Tue, 16 May 2017 17:23:02 +0000 (13:23 -0400)]
11644: Ensure generated UUIDs are always 27 chars.

7 years ago11644: Add volume replication level to /mounts response.
Tom Clegg [Tue, 16 May 2017 17:22:42 +0000 (13:22 -0400)]
11644: Add volume replication level to /mounts response.

7 years ago11644: Replace linear search with map for looking up mounts by UUID.
Tom Clegg [Tue, 16 May 2017 17:11:45 +0000 (13:11 -0400)]
11644: Replace linear search with map for looking up mounts by UUID.

7 years ago11644: Test non-empty MountUUID in trash list.
Tom Clegg [Tue, 16 May 2017 17:03:14 +0000 (13:03 -0400)]
11644: Test non-empty MountUUID in trash list.

7 years ago11644: Add pull-to-mount-UUID test. Tidy up pull worker and tests.
Tom Clegg [Tue, 16 May 2017 16:43:39 +0000 (12:43 -0400)]
11644: Add pull-to-mount-UUID test. Tidy up pull worker and tests.

7 years agoMerge branch '11693-crunch2-output-symlinks' closes #11693
Peter Amstutz [Tue, 16 May 2017 14:52:04 +0000 (10:52 -0400)]
Merge branch '11693-crunch2-output-symlinks' closes #11693

7 years ago11693: Only check collection mounts when determining whether to call
Peter Amstutz [Tue, 16 May 2017 14:38:55 +0000 (10:38 -0400)]
11693: Only check collection mounts when determining whether to call
getCollectionManifestForPath to merge files into output collection.  Remove
redundant assignment of outputSuffix.

7 years agoMerge branch '10111-collection-labels'
Lucas Di Pentima [Tue, 16 May 2017 14:32:48 +0000 (11:32 -0300)]
Merge branch '10111-collection-labels'
Closes #10111

7 years ago10111: Merge branch 'master' into 10111-collection-labels
Lucas Di Pentima [Tue, 16 May 2017 14:32:11 +0000 (11:32 -0300)]
10111: Merge branch 'master' into 10111-collection-labels

7 years ago11693: Detect and error on symlinks pointing to locations outside the output directory.
Peter Amstutz [Tue, 16 May 2017 13:12:58 +0000 (09:12 -0400)]
11693: Detect and error on symlinks pointing to locations outside the output directory.

7 years ago11644: Accept index/pull/trash requests for a specific mount.
Tom Clegg [Mon, 15 May 2017 19:36:02 +0000 (15:36 -0400)]
11644: Accept index/pull/trash requests for a specific mount.

7 years ago11693: Add test that error capturing output results in cancelled container.
Peter Amstutz [Mon, 15 May 2017 20:58:05 +0000 (16:58 -0400)]
11693: Add test that error capturing output results in cancelled container.

7 years ago11693: Add test cases for symlinks to directories and symlinks inside subdirectories.
Peter Amstutz [Mon, 15 May 2017 20:25:39 +0000 (16:25 -0400)]
11693: Add test cases for symlinks to directories and symlinks inside subdirectories.

7 years ago10111: Moved the api request related code to the controller.
Lucas Di Pentima [Mon, 15 May 2017 20:08:11 +0000 (17:08 -0300)]
10111: Moved the api request related code to the controller.

7 years ago11693: Add tests.
Peter Amstutz [Mon, 15 May 2017 19:52:00 +0000 (15:52 -0400)]
11693: Add tests.

7 years ago11693: Dereference symlinks to input files and add incorporate them into output colle...
Peter Amstutz [Mon, 15 May 2017 18:06:33 +0000 (14:06 -0400)]
11693: Dereference symlinks to input files and add incorporate them into output collection manifest.

7 years ago11693: Mark container as cancelled if there is an error during finalization.
Peter Amstutz [Mon, 15 May 2017 14:19:20 +0000 (10:19 -0400)]
11693: Mark container as cancelled if there is an error during finalization.

7 years ago11644: Use storage account and bucket name as DeviceID for Azure volumes.
Tom Clegg [Mon, 15 May 2017 14:11:20 +0000 (10:11 -0400)]
11644: Use storage account and bucket name as DeviceID for Azure volumes.

7 years ago11644: Use endpoint and bucket name as DeviceID for S3 volumes.
Tom Clegg [Mon, 15 May 2017 14:10:55 +0000 (10:10 -0400)]
11644: Use endpoint and bucket name as DeviceID for S3 volumes.

7 years ago11644: Use filesystem UUID and path as DeviceID for local disk volumes.
Tom Clegg [Mon, 15 May 2017 05:59:39 +0000 (01:59 -0400)]
11644: Use filesystem UUID and path as DeviceID for local disk volumes.

7 years ago11644: Add /mounts endpoint using random mount UUIDs assigned at runtime.
Tom Clegg [Fri, 12 May 2017 21:21:00 +0000 (17:21 -0400)]
11644: Add /mounts endpoint using random mount UUIDs assigned at runtime.

7 years agoMerge branch '11590-log-reuse'
Tom Clegg [Fri, 12 May 2017 14:45:20 +0000 (10:45 -0400)]
Merge branch '11590-log-reuse'

refs #11590

7 years ago11590: Reword log messages.
Tom Clegg [Fri, 12 May 2017 14:34:29 +0000 (10:34 -0400)]
11590: Reword log messages.

7 years agoMerge branch '11647-no-perm-tokens'
Lucas Di Pentima [Fri, 12 May 2017 14:29:01 +0000 (11:29 -0300)]
Merge branch '11647-no-perm-tokens'
Refs #11647

7 years ago11647: Merge branch 'master' into 11647-no-perm-tokens
Lucas Di Pentima [Fri, 12 May 2017 14:28:00 +0000 (11:28 -0300)]
11647: Merge branch 'master' into 11647-no-perm-tokens

7 years ago11647: When repacking small blocks into big ones, commit them synchronously.
Lucas Di Pentima [Fri, 12 May 2017 14:00:55 +0000 (11:00 -0300)]
11647: When repacking small blocks into big ones, commit them synchronously.

7 years ago11647: Test proving that when small blocks get repacked and the resulting big block is
Lucas Di Pentima [Fri, 12 May 2017 13:57:58 +0000 (10:57 -0300)]
11647: Test proving that when small blocks get repacked and the resulting big block is
committed asynchronously, it doesn't get the permission token on the manifest.

7 years agoMerge branch '11681-str-newstr' refs #11681
Peter Amstutz [Thu, 11 May 2017 21:27:27 +0000 (17:27 -0400)]
Merge branch '11681-str-newstr' refs #11681

7 years ago11681: Return native str type from Collection.portable_data_hash() (previous commit...
Peter Amstutz [Thu, 11 May 2017 21:25:34 +0000 (17:25 -0400)]
11681: Return native str type from Collection.portable_data_hash()  (previous commit was actually for CollectionWriter)

7 years agoMerge branch '11681-str-newstr'
Tom Clegg [Thu, 11 May 2017 20:35:24 +0000 (16:35 -0400)]
Merge branch '11681-str-newstr'

refs #11681

7 years ago11681: Return native str type from KeepLocator.__str__() and Collection.portable_data...
Tom Clegg [Thu, 11 May 2017 19:37:32 +0000 (15:37 -0400)]
11681: Return native str type from KeepLocator.__str__() and Collection.portable_data_hash()

7 years agocloses #10645
radhika [Thu, 11 May 2017 16:54:53 +0000 (12:54 -0400)]
closes #10645
Merge branch '10645-cr-mounts-display'

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Thu, 11 May 2017 16:54:19 +0000 (12:54 -0400)]
Merge branch 'master' into 10645-cr-mounts-display

7 years ago11590: Add "reuse jobs even if conflicting outputs exist" and "log reuse decisions...
Tom Clegg [Thu, 11 May 2017 15:38:37 +0000 (11:38 -0400)]
11590: Add "reuse jobs even if conflicting outputs exist" and "log reuse decisions" config options.

7 years agoMerge branch '11369-cwl-keepcache-ram' refs #11369
Peter Amstutz [Thu, 11 May 2017 15:00:38 +0000 (11:00 -0400)]
Merge branch '11369-cwl-keepcache-ram' refs #11369

7 years agoMerge branch '11626-crunch2-log-slurm-errors' closes #11626
Peter Amstutz [Thu, 11 May 2017 14:19:32 +0000 (10:19 -0400)]
Merge branch '11626-crunch2-log-slurm-errors' closes #11626

7 years ago11626: Fix log.Printf -> log.Print
Peter Amstutz [Thu, 11 May 2017 14:19:05 +0000 (10:19 -0400)]
11626: Fix log.Printf -> log.Print

7 years ago10645: display collection link with disposition. preload mount and command collections.
radhika [Thu, 11 May 2017 12:54:59 +0000 (08:54 -0400)]
10645: display collection link with disposition. preload mount and command collections.

7 years agoBe smarter about how we handle packages built in previous runs.
Ward Vandewege [Thu, 11 May 2017 01:10:02 +0000 (21:10 -0400)]
Be smarter about how we handle packages built in previous runs.

This commit fixes a bug: packages that were built but failed testing
will no longer be uploaded later, when a subsequent set of packages
passes the tests.

No issue #

7 years agoThe Workbench package has a build-time dependency on tzdata for
Ward Vandewege [Wed, 10 May 2017 21:09:06 +0000 (17:09 -0400)]
The Workbench package has a build-time dependency on tzdata for
Ubuntu1604, which is not included in the default upstream docker image.
Add it to our list of dependencies.

Also tweak Workbench package dependencies, no need to expliclitly list
g++ for CentOS7.

No issue #

7 years agoFix Workbench package dependency.
Ward Vandewege [Wed, 10 May 2017 20:21:45 +0000 (16:21 -0400)]
Fix Workbench package dependency.

No issue #

7 years agoOur API server and Workbench packages depend on 2 gems that compile
Ward Vandewege [Wed, 10 May 2017 19:11:26 +0000 (15:11 -0400)]
Our API server and Workbench packages depend on 2 gems that compile
native objects, which require g++ to build them: event-machine and
therubyracer. For our debian-based distros, that package is not
necessarily installed, so we're adding it to the dependency list.

No issue #

7 years ago11369: Add keep_cache to base RAM request.
Peter Amstutz [Wed, 10 May 2017 18:57:23 +0000 (14:57 -0400)]
11369: Add keep_cache to base RAM request.

7 years agoMerge branch '10827-checker-cwltool-update' refs #10827
Peter Amstutz [Wed, 10 May 2017 18:50:15 +0000 (14:50 -0400)]
Merge branch '10827-checker-cwltool-update' refs #10827

7 years ago10827: Bump cwltool & schema salad dependency.
Peter Amstutz [Wed, 10 May 2017 15:16:18 +0000 (11:16 -0400)]
10827: Bump cwltool & schema salad dependency.

7 years agocloses #10112
radhika [Wed, 10 May 2017 16:56:30 +0000 (12:56 -0400)]
closes #10112
Merge branch '10112-workflow-show'

7 years agoMerge branch 'master' into 10112-workflow-show
radhika [Wed, 10 May 2017 16:55:21 +0000 (12:55 -0400)]
Merge branch 'master' into 10112-workflow-show

7 years ago10112: preload collections
radhika [Wed, 10 May 2017 15:45:23 +0000 (11:45 -0400)]
10112: preload collections

7 years ago10827: Add --validate, --print-dot and --enable-dev options (pass through to
Peter Amstutz [Wed, 10 May 2017 15:08:36 +0000 (11:08 -0400)]
10827: Add --validate, --print-dot and --enable-dev options (pass through to
existing cwltool functionality).

7 years agoMerge branch '11629-groups-contents-memory'
Tom Clegg [Wed, 10 May 2017 14:23:20 +0000 (10:23 -0400)]
Merge branch '11629-groups-contents-memory'

refs #11629
refs #11458

7 years ago11629: Fix ambiguous column reference.
Tom Clegg [Wed, 10 May 2017 14:22:42 +0000 (10:22 -0400)]
11629: Fix ambiguous column reference.

7 years agoMerge branch '11634-bundler-cache'
Tom Clegg [Tue, 9 May 2017 20:37:47 +0000 (16:37 -0400)]
Merge branch '11634-bundler-cache'

closes #11634

7 years agoMerge branch '11629-groups-contents-memory'
Tom Clegg [Tue, 9 May 2017 20:33:31 +0000 (16:33 -0400)]
Merge branch '11629-groups-contents-memory'

closes #11629

7 years ago11626: Log sbatch errors where the user can see them. Fix tests so there is no linge...
Peter Amstutz [Fri, 5 May 2017 21:11:39 +0000 (17:11 -0400)]
11626: Log sbatch errors where the user can see them.  Fix tests so there is no lingering goroutines.

7 years agoAdd libgnutls28-dev and python3-dev to install package list to satisfy
Peter Amstutz [Tue, 9 May 2017 20:00:39 +0000 (16:00 -0400)]
Add libgnutls28-dev and python3-dev to install package list to satisfy
run-tests.sh dependencies.  refs #11308

7 years agoMerge branch '11308-collection-keys'
Tom Clegg [Tue, 9 May 2017 19:49:41 +0000 (15:49 -0400)]
Merge branch '11308-collection-keys'

refs #11308

7 years ago11308: Fix modes not tested in test case.
Tom Clegg [Tue, 9 May 2017 19:31:44 +0000 (15:31 -0400)]
11308: Fix modes not tested in test case.

7 years ago11308: collection.keys() supports indexing for python2 callers.
Tom Clegg [Tue, 9 May 2017 19:30:19 +0000 (15:30 -0400)]
11308: collection.keys() supports indexing for python2 callers.

7 years ago10112: use show_text_with_locators partial to display the various attributes of the...
radhika [Tue, 9 May 2017 15:29:02 +0000 (11:29 -0400)]
10112: use show_text_with_locators partial to display the various attributes of the definition

7 years agoMerge branch 'master' into 10112-workflow-show
radhika [Tue, 9 May 2017 14:35:38 +0000 (10:35 -0400)]
Merge branch 'master' into 10112-workflow-show

7 years agorefs #10645
radhika [Tue, 9 May 2017 14:31:06 +0000 (10:31 -0400)]
refs #10645
Merge branch '10645-cr-mounts-display'

7 years ago10645: added comment
radhika [Tue, 9 May 2017 14:28:23 +0000 (10:28 -0400)]
10645: added comment

7 years agoMerge branch 'master' into 10645-cr-mounts-display
radhika [Tue, 9 May 2017 11:35:16 +0000 (07:35 -0400)]
Merge branch 'master' into 10645-cr-mounts-display