Peter Amstutz [Thu, 22 Sep 2016 15:38:21 +0000 (11:38 -0400)]
10117: Tweak behavior of has_ref(). Fix naming of CacheTest.
Peter Amstutz [Wed, 21 Sep 2016 22:10:54 +0000 (18:10 -0400)]
10117: The check needs to be between clear() and the deletion from entries.
Peter Amstutz [Wed, 21 Sep 2016 21:58:29 +0000 (17:58 -0400)]
10117: Check that inode is in entries before trying to remove it, along with
explanation why it is necessary.
Peter Amstutz [Wed, 21 Sep 2016 19:51:34 +0000 (15:51 -0400)]
10117: OrderedDict preserves insertion order, so use inode as key instead of cache_priority.
Peter Amstutz [Wed, 21 Sep 2016 18:55:53 +0000 (14:55 -0400)]
10117: Restore original semantics of invalidate() and add code comment, add
kernel_invalidate().
Peter Amstutz [Wed, 21 Sep 2016 18:17:51 +0000 (14:17 -0400)]
10117: Additional refinement: a directory not in use but still referenced needs
to be invalidated before it can be cleared so that the kernel releases its
references.
Peter Amstutz [Wed, 21 Sep 2016 15:13:32 +0000 (11:13 -0400)]
10117: Consider both use_count and ref_count and check subdirectories to determine if it is safe to evict directory contents from cache.
Peter Amstutz [Tue, 20 Sep 2016 19:48:30 +0000 (15:48 -0400)]
Update cwltool version pin, refs #10088
Peter Amstutz [Tue, 20 Sep 2016 12:22:25 +0000 (08:22 -0400)]
Merge branch '10088-raw-files' refs #10088
Tom Clegg [Mon, 19 Sep 2016 17:49:13 +0000 (13:49 -0400)]
Merge branch '9954-keep-balance-systemd' closes #9954
Tom Clegg [Mon, 19 Sep 2016 17:46:32 +0000 (13:46 -0400)]
Merge branch '10014-collection-error-detail' closes #10014
Tom Clegg [Fri, 16 Sep 2016 15:05:48 +0000 (11:05 -0400)]
10014: Add error details. Use filesystem language ("directory") instead of "subcollection".
Tom Clegg [Tue, 13 Sep 2016 20:25:36 +0000 (16:25 -0400)]
10014: Add details to various error messages.
Peter Amstutz [Mon, 19 Sep 2016 15:35:17 +0000 (11:35 -0400)]
10088: Bump cwltool dependency for another bugfix to scandeps.
Peter Amstutz [Sun, 18 Sep 2016 01:26:36 +0000 (21:26 -0400)]
10088: Bump cwltool dependency for bugfix to scandeps.
Peter Amstutz [Fri, 16 Sep 2016 21:37:13 +0000 (17:37 -0400)]
10088: upload_dependencies reads raw files to find actual dependencies
radhika [Thu, 15 Sep 2016 15:33:52 +0000 (11:33 -0400)]
refs #9835
Merge branch '9835-container-reuse-doc'
radhika [Wed, 14 Sep 2016 13:14:28 +0000 (09:14 -0400)]
9836: Add container reuse to ContainerRequest documentation.
Peter Amstutz [Thu, 15 Sep 2016 15:29:09 +0000 (11:29 -0400)]
Fix arvados-cwl-runner tests, refs #9766
Peter Amstutz [Thu, 15 Sep 2016 14:27:12 +0000 (10:27 -0400)]
Fix misake in arvados-cwl-runner upload_dependencies refactor. refs #9766
Tom Clegg [Thu, 15 Sep 2016 14:27:32 +0000 (10:27 -0400)]
9954: Add systemd unit keep-balance.service.
Tom Clegg [Thu, 15 Sep 2016 13:50:15 +0000 (09:50 -0400)]
Merge branch '9986-fuse-retry-if-killed' closes #9986
Tom Clegg [Wed, 14 Sep 2016 20:34:05 +0000 (16:34 -0400)]
9986: Retry (up to 2 times) any python test suite killed by SIGKILL.
Peter Amstutz [Wed, 14 Sep 2016 21:14:07 +0000 (17:14 -0400)]
Merge branch '100032-cwl-metrics' refs #100032
Peter Amstutz [Wed, 14 Sep 2016 21:13:35 +0000 (17:13 -0400)]
100032: Add metrics to job submission in --debug mode.
Peter Amstutz [Wed, 14 Sep 2016 16:30:56 +0000 (12:30 -0400)]
Merge branch '9766-register-workflow' closes #9766
Lucas Di Pentima [Wed, 14 Sep 2016 16:22:57 +0000 (13:22 -0300)]
Merge branch '9623-reuse-containers'
Closes #9623
Lucas Di Pentima [Wed, 14 Sep 2016 16:20:49 +0000 (13:20 -0300)]
Merge branch 'master' into 9623-reuse-containers
Tom Clegg [Wed, 14 Sep 2016 16:04:00 +0000 (12:04 -0400)]
Merge branch '9986-fuse-test-deadlock' closes #9986
Tom Clegg [Wed, 14 Sep 2016 15:04:35 +0000 (11:04 -0400)]
9986: Move llfuse thread-shutdown check into a "finally" block so it actually runs.
Tom Clegg [Wed, 14 Sep 2016 15:03:43 +0000 (11:03 -0400)]
9986: Report actual thread shutdown wait if >0.1s (instead of just saying it took >1s)
Tom Clegg [Thu, 8 Sep 2016 18:26:58 +0000 (14:26 -0400)]
9986: Kill test suite when danger of deadlock is detected.
Tom Clegg [Thu, 8 Sep 2016 13:34:31 +0000 (09:34 -0400)]
9986: Share one multiprocessing pool with all IntegrationTests.
Tom Clegg [Thu, 8 Sep 2016 12:56:18 +0000 (08:56 -0400)]
9986: Send SIGKILL if llfuse thread does not join.
Peter Amstutz [Wed, 14 Sep 2016 15:44:37 +0000 (11:44 -0400)]
9766: Fix test to expect yaml text.
Lucas Di Pentima [Wed, 14 Sep 2016 15:43:55 +0000 (12:43 -0300)]
9623: Several fixes addressing review comments:
* Fixed ordering for completed containers selection
* Search for Locked or Queued containers in just one query
* Fixed several test issues
Peter Amstutz [Wed, 14 Sep 2016 15:38:13 +0000 (11:38 -0400)]
9766: Rename metavar. Post packed yaml text instead of json. Adjust fixture
to have similar form as posted by --create-workflow. Fix fixture to be a
string instead of structured text.
radhika [Wed, 14 Sep 2016 15:19:53 +0000 (11:19 -0400)]
closes #9970
Merge branch '9970-requesting-container-at-create'
radhika [Wed, 14 Sep 2016 14:03:26 +0000 (10:03 -0400)]
9970: Invalidate record if a requesting_container_uuid is provided at create.
Lucas Di Pentima [Wed, 14 Sep 2016 13:41:11 +0000 (10:41 -0300)]
9623: Added test cases on container request to check for container reuse flow. Also added more container reuse tests to cover missing cases.
Lucas Di Pentima [Wed, 14 Sep 2016 12:39:43 +0000 (09:39 -0300)]
9623: Changed query strategy for better performance. Also fixed the case where state = complete and several containers were found with different outputs.
Lucas Di Pentima [Wed, 14 Sep 2016 12:38:57 +0000 (09:38 -0300)]
9623: Fixed progress values to be between 0.0 and 1.0
Peter Amstutz [Wed, 14 Sep 2016 12:35:15 +0000 (08:35 -0400)]
Merge branch 'master' into 9766-register-workflow
Lucas Di Pentima [Wed, 14 Sep 2016 10:54:16 +0000 (07:54 -0300)]
9623: Container fixtures corrections.
Lucas Di Pentima [Wed, 14 Sep 2016 10:50:18 +0000 (07:50 -0300)]
9623: Moved deep_sort_hash method to ArvadosModel as it's used by both Job and Container classes.
radhika [Tue, 13 Sep 2016 23:52:40 +0000 (19:52 -0400)]
closes #9836
Merge branch '9836-workflows-doc'
radhika [Tue, 13 Sep 2016 23:51:45 +0000 (19:51 -0400)]
Merge branch 'master' into 9836-workflows-doc
radhika [Tue, 13 Sep 2016 23:50:47 +0000 (19:50 -0400)]
closes #9835
Merge branch '9835-container-api-doc'
radhika [Tue, 13 Sep 2016 23:50:27 +0000 (19:50 -0400)]
Merge branch 'master' into 9835-container-api-doc
Lucas Di Pentima [Tue, 13 Sep 2016 21:30:43 +0000 (18:30 -0300)]
Merge branch 'master' into 9623-reuse-containers
Peter Amstutz [Tue, 13 Sep 2016 18:40:16 +0000 (14:40 -0400)]
Fix typo, updated version on wrong package, refs #10013
Peter Amstutz [Tue, 13 Sep 2016 17:14:24 +0000 (13:14 -0400)]
Update cwltool version, refs #10013
radhika [Tue, 13 Sep 2016 16:32:20 +0000 (12:32 -0400)]
9836: Remove the workflow definition related discussion and provide a link to CWL instead.
radhika [Tue, 13 Sep 2016 16:17:34 +0000 (12:17 -0400)]
9835: mount_types improvements.
Peter Amstutz [Tue, 13 Sep 2016 14:05:09 +0000 (10:05 -0400)]
Merge branch '9924-revoked-working-slots' closes #9924
Peter Amstutz [Tue, 13 Sep 2016 14:04:43 +0000 (10:04 -0400)]
9924: Tweak comment and remove superfluous variable assignment.
Peter Amstutz [Tue, 13 Sep 2016 02:13:11 +0000 (22:13 -0400)]
9924: Mark all slots as "failed" on "Unable to confirm allocation" error.
Peter Amstutz [Tue, 13 Sep 2016 01:53:59 +0000 (21:53 -0400)]
Merge branch '10001-cwl-scatter' closes #10001
Peter Amstutz [Tue, 13 Sep 2016 01:53:35 +0000 (21:53 -0400)]
10001: Update cwltool dependency to get parallel scatter fixes.
Peter Amstutz [Tue, 13 Sep 2016 01:42:53 +0000 (21:42 -0400)]
Merge branch '10004-check-sinfo' closes #10004
Peter Amstutz [Tue, 13 Sep 2016 01:42:26 +0000 (21:42 -0400)]
10004: Add comment documenting reason why check_sinfo is needed.
Lucas Di Pentima [Mon, 12 Sep 2016 21:09:51 +0000 (18:09 -0300)]
9623: Restored changes to fixture file
Lucas Di Pentima [Mon, 12 Sep 2016 20:57:59 +0000 (17:57 -0300)]
9623: Rewrote tests to avoid using fixture records.
Lucas Di Pentima [Mon, 12 Sep 2016 20:55:07 +0000 (17:55 -0300)]
9623: Removed fixtures additions meant for Container reuse tests because some of them made other integration tests to fail.
Tom Clegg [Mon, 12 Sep 2016 20:53:55 +0000 (16:53 -0400)]
Fix crash when pipeline template is not readable. refs #10003
Peter Amstutz [Mon, 12 Sep 2016 19:17:23 +0000 (15:17 -0400)]
10004: Add check_sinfo() to end srun_sync() if any of the allocated nodes are
no longer "alloc" according to sinfo.
radhika [Mon, 12 Sep 2016 17:49:35 +0000 (13:49 -0400)]
9836: Add workflow to api documentation.
radhika [Fri, 9 Sep 2016 21:01:31 +0000 (17:01 -0400)]
closes #10005
Merge branch '10005-rename-workflow-in-workflow-as-definition'
radhika [Fri, 9 Sep 2016 19:58:44 +0000 (15:58 -0400)]
10005: update workflow fixture name and description as well and update workbench invocations of workflow.workflow.
radhika [Fri, 9 Sep 2016 19:06:29 +0000 (15:06 -0400)]
10005: Rename workflow attribute in workflow object as definition.
Lucas Di Pentima [Fri, 9 Sep 2016 15:35:45 +0000 (12:35 -0300)]
9623: Added CR fixtures to make services/crunch-dispatch-local tests run OK again.
radhika [Fri, 9 Sep 2016 00:48:30 +0000 (20:48 -0400)]
9835: more details such as mount_types added.
Tom Clegg [Thu, 8 Sep 2016 23:19:42 +0000 (19:19 -0400)]
Merge branch '9898-container-lock-api'
closes #9898
Lucas Di Pentima [Thu, 8 Sep 2016 21:54:41 +0000 (18:54 -0300)]
9623: Fixed test by avoiding Container reusability when calling create_minimal_req! multiple times with the same attributes.
Lucas Di Pentima [Thu, 8 Sep 2016 21:52:50 +0000 (18:52 -0300)]
9623: Cleaned up references to local constants about Container's states. Added optimizations on the serialized attributes sorting.
Tom Clegg [Thu, 8 Sep 2016 21:09:26 +0000 (17:09 -0400)]
9898: Update test for new response code.
Tom Clegg [Thu, 8 Sep 2016 21:06:58 +0000 (17:06 -0400)]
Merge branch '9996-stop-on-error'
closes #9996
Tom Clegg [Thu, 8 Sep 2016 21:06:03 +0000 (17:06 -0400)]
Merge branch '9888-unit-tests'
closes #9888
Tom Clegg [Thu, 8 Sep 2016 21:02:22 +0000 (17:02 -0400)]
9996: Stop retrieving collections if a fatal error makes any further work futile.
Tom Clegg [Thu, 8 Sep 2016 20:56:42 +0000 (16:56 -0400)]
9888: Add find_reusable unit test.
Tom Clegg [Thu, 8 Sep 2016 20:53:24 +0000 (16:53 -0400)]
9898: Fix broken test.
Peter Amstutz [Thu, 8 Sep 2016 20:30:16 +0000 (16:30 -0400)]
Restore use of --build-arg COMMIT in arvados/jobs Dockerfile to select "cut"
above which layers must be rebuilt. no issue #
Tom Clegg [Thu, 8 Sep 2016 20:15:11 +0000 (16:15 -0400)]
9898: Code cleanup.
Tom Clegg [Thu, 8 Sep 2016 20:14:22 +0000 (16:14 -0400)]
9898: Check previous state after obtaining row lock.
Tom Clegg [Thu, 8 Sep 2016 20:13:29 +0000 (16:13 -0400)]
9898: Change state transition error responses from 403 to 422.
Tom Clegg [Thu, 8 Sep 2016 20:11:59 +0000 (16:11 -0400)]
9898: Remove unused code
radhika [Thu, 8 Sep 2016 19:33:05 +0000 (15:33 -0400)]
9898: add unlock method also on the container model.
Peter Amstutz [Thu, 8 Sep 2016 17:58:56 +0000 (13:58 -0400)]
Merge branch 'master' into 9766-register-workflow
radhika [Thu, 8 Sep 2016 16:53:54 +0000 (12:53 -0400)]
9898: add Lock and Unlock methods to dispatch go sdk.
Peter Amstutz [Thu, 8 Sep 2016 16:07:28 +0000 (12:07 -0400)]
Don't use cached layers and always delete intermediate containers when building
arvados/jobs Docker image. no issue #
radhika [Thu, 8 Sep 2016 15:02:39 +0000 (11:02 -0400)]
Merge branch 'master' into 9898-container-lock-api
Conflicts:
services/api/app/models/container.rb
Peter Amstutz [Thu, 8 Sep 2016 02:02:20 +0000 (22:02 -0400)]
Add -f argument to curl to exit nonzero and try mirror if primary
rpmforge-release url fails, refs #9944
radhika [Wed, 7 Sep 2016 23:18:40 +0000 (19:18 -0400)]
9835: Container and ContainerRequest documentation.
Peter Amstutz [Wed, 7 Sep 2016 20:53:10 +0000 (16:53 -0400)]
Don't pass work_api twice in ArvadosCommandTool constructor. Bump arvados-python-client minimum dependency because tests depend on it. refs #9944
Peter Amstutz [Wed, 7 Sep 2016 20:19:50 +0000 (16:19 -0400)]
Merge branch '9944-cwl-lockfile-packaging' closes #9944
Peter Amstutz [Wed, 7 Sep 2016 20:12:47 +0000 (16:12 -0400)]
Merge branch 'master' into 9944-cwl-lockfile-packaging
Peter Amstutz [Wed, 7 Sep 2016 19:58:50 +0000 (15:58 -0400)]
9944: Tweak centos6 Dockerfiles to fall back to mirror if repoforge main is
unavailable.
Peter Amstutz [Wed, 7 Sep 2016 19:58:13 +0000 (15:58 -0400)]
9944: Fix schema-salad 'lockfile' dependency to account for different package
names between centos & debian/ubuntu.
Lucas Di Pentima [Wed, 7 Sep 2016 19:09:40 +0000 (16:09 -0300)]
9623: Check for reusable containers in Completed state that has existing output and log data. Added additional tests to check for correct container reuse preferences.
Peter Amstutz [Tue, 6 Sep 2016 20:56:53 +0000 (16:56 -0400)]
9944: Add 'lockfile' to backports and update cwltool dependency
projects / arvados.git / log