Peter Amstutz [Mon, 22 Aug 2016 02:13:55 +0000 (22:13 -0400)]
9827: Use "arvbox rebuild" to ensure up to date arvados/arvbox-demo image.
Peter Amstutz [Fri, 19 Aug 2016 15:13:51 +0000 (11:13 -0400)]
Merge branch '9820-cwl-poll-jobs' closes #9820
Peter Amstutz [Fri, 19 Aug 2016 14:50:25 +0000 (10:50 -0400)]
9820: Directly poll job or container records that we are interested in. Benefit: puts less load on database than log table polling, and doesn't miss events.
Tom Clegg [Fri, 19 Aug 2016 00:27:37 +0000 (20:27 -0400)]
Merge branch '9684-workflows'
refs #9684
Tom Clegg [Fri, 19 Aug 2016 00:12:25 +0000 (20:12 -0400)]
9684: Style nitpicks.
radhika [Thu, 18 Aug 2016 18:22:59 +0000 (14:22 -0400)]
9684: update arvados_model -> recursive_stringify to convert ":foo" to "foo"
Lucas Di Pentima [Thu, 18 Aug 2016 17:56:43 +0000 (14:56 -0300)]
Merge branch '9333-api-server-invalid-sql'
9333: Fixes to solve workbench integration tests failures. Refs #9333
Lucas Di Pentima [Thu, 18 Aug 2016 17:34:38 +0000 (14:34 -0300)]
Merge branch 'master' into 9333-api-server-invalid-sql
radhika [Thu, 18 Aug 2016 16:47:50 +0000 (12:47 -0400)]
9684: update arvados_model -> recursive_stringify to convert ":foo" to "foo"
Lucas Di Pentima [Thu, 18 Aug 2016 16:06:01 +0000 (13:06 -0300)]
9333: Adding collection model specific sttributes to be ignored on the validation mechanism to fix workbench integration test failures
radhika [Thu, 18 Aug 2016 13:43:23 +0000 (09:43 -0400)]
9684: Add safe_yaml initializer
radhika [Thu, 18 Aug 2016 11:51:21 +0000 (07:51 -0400)]
closes #9684
Merge branch '9684-workflows'
Lucas Di Pentima [Wed, 17 Aug 2016 16:44:41 +0000 (13:44 -0300)]
Merge branch '9333-api-server-invalid-sql'
9333: Selected attributes list validation on the API Server, avoiding invalid SQL queries to the database. Closes #9333
Lucas Di Pentima [Wed, 17 Aug 2016 16:39:24 +0000 (13:39 -0300)]
9333: Removed superfluous quote marks on exception message
Lucas Di Pentima [Tue, 16 Aug 2016 21:22:04 +0000 (18:22 -0300)]
9333: Treating the "attribute is not a string" error case the same as the other invalid attributes. Moved the checks to ArvadosModel. Updated tests.
radhika [Tue, 16 Aug 2016 21:09:33 +0000 (17:09 -0400)]
9684: workflow yaml error logging
radhika [Mon, 15 Aug 2016 22:19:00 +0000 (18:19 -0400)]
9678: use safe_yaml and other updates.
radhika [Mon, 15 Aug 2016 14:26:44 +0000 (10:26 -0400)]
closes #9709
Merge branch '9709-dont-log-manifest-texts'
Lucas Di Pentima [Fri, 12 Aug 2016 00:38:46 +0000 (21:38 -0300)]
9333: Attribute validation on "select" to avoid invalid SQL statements
Peter Amstutz [Thu, 11 Aug 2016 22:18:13 +0000 (18:18 -0400)]
Fix failing tests. refs #9776
Peter Amstutz [Thu, 11 Aug 2016 21:50:49 +0000 (17:50 -0400)]
Fix schema salad package build version. no issue #
Peter Amstutz [Thu, 11 Aug 2016 21:02:15 +0000 (17:02 -0400)]
Merge branch '9776-cwl-HOME-TMPDIR' closes #9776
Peter Amstutz [Thu, 11 Aug 2016 20:55:10 +0000 (16:55 -0400)]
9776: Ensure that $HOME and $TMPDIR are set correctly.
Peter Amstutz [Thu, 11 Aug 2016 20:33:06 +0000 (16:33 -0400)]
Merge branch '9615-cwl-writable-unsupported' refs #9615
Peter Amstutz [Thu, 4 Aug 2016 15:36:56 +0000 (11:36 -0400)]
9615: Add check_writable to check for "writable" field and raise UnsupportedRequirement.
Peter Amstutz [Thu, 11 Aug 2016 20:30:34 +0000 (16:30 -0400)]
Merge branch '9751-cwl-keep-dir' closes #9751
Peter Amstutz [Thu, 11 Aug 2016 18:54:36 +0000 (14:54 -0400)]
9552: import compute_checksums from cwltool.
Peter Amstutz [Thu, 11 Aug 2016 19:35:15 +0000 (15:35 -0400)]
9751: Fix handling local Directories
Peter Amstutz [Thu, 11 Aug 2016 18:46:24 +0000 (14:46 -0400)]
9751: Bump cwltool dependency.
Peter Amstutz [Thu, 11 Aug 2016 18:19:37 +0000 (14:19 -0400)]
9751: Fix unit tests. Rename dir6.cwl to keep-dir-test-input.cwl
Peter Amstutz [Thu, 11 Aug 2016 17:01:41 +0000 (13:01 -0400)]
9751: Override realpath in CollectionFsAccess
Peter Amstutz [Wed, 10 Aug 2016 17:45:56 +0000 (13:45 -0400)]
9751: Fix direct to keep directory references.
Add arvados-specific integration test for arvados-cwl-runner.
Peter Amstutz [Wed, 10 Aug 2016 15:51:44 +0000 (11:51 -0400)]
9751: Provide make_fs_access=CollectionFsAccess and bump cwltool pin.
radhika [Thu, 11 Aug 2016 18:13:36 +0000 (14:13 -0400)]
closes #9678
Merge branch '9678-container-request-log-tab'
radhika [Thu, 11 Aug 2016 18:10:00 +0000 (14:10 -0400)]
9709: do not include manifest_text in collection logs.
radhika [Thu, 11 Aug 2016 13:50:36 +0000 (09:50 -0400)]
closes #9658
Merge branch '9658-cwl-runner-container'
radhika [Wed, 10 Aug 2016 18:41:29 +0000 (14:41 -0400)]
9678: enhance test to verify that the combined logs are displayed.
radhika [Wed, 10 Aug 2016 18:05:02 +0000 (14:05 -0400)]
9678: enhance container log tab display to consider the "combined" object logs.
radhika [Wed, 10 Aug 2016 16:29:31 +0000 (12:29 -0400)]
9658: when arvados-cwl-runner is invoked with --api containers, display the container_request uuid to console.
radhika [Wed, 10 Aug 2016 15:58:22 +0000 (11:58 -0400)]
closes #9745
Merge branch '9745-cds-api-config'
radhika [Tue, 9 Aug 2016 19:51:50 +0000 (15:51 -0400)]
9684: add workflow resource to api server
Tom Clegg [Fri, 5 Aug 2016 15:45:55 +0000 (11:45 -0400)]
9745: Update example config.
Tom Clegg [Fri, 5 Aug 2016 15:44:29 +0000 (11:44 -0400)]
9745: Load API host and token from crunch-dispatch-slurm config file.
Tom Clegg [Fri, 5 Aug 2016 14:54:37 +0000 (10:54 -0400)]
9706: Add example config file.
No issue #
Tom Clegg [Thu, 4 Aug 2016 22:16:24 +0000 (18:16 -0400)]
gofmt.
No issue #
Tom Clegg [Thu, 4 Aug 2016 21:45:10 +0000 (17:45 -0400)]
Fix race in crunchstat.
To fulfill the "nothing more will be logged after Stop() returns"
guarantee, Stop() needs to wait for run() to finish.
Should fix https://ci.curoverse.com/job/run-tests-remainder/38/consoleText
No issue #
Tom Clegg [Thu, 4 Aug 2016 21:44:34 +0000 (17:44 -0400)]
Simplify write flusher using a time.Ticker.
No issue #
Tom Clegg [Thu, 4 Aug 2016 20:53:32 +0000 (16:53 -0400)]
Fix potential race in ThrottledLogger flusher.
No issue #
Peter Amstutz [Thu, 4 Aug 2016 16:01:37 +0000 (12:01 -0400)]
Merge branch '9552-compute-checksum-final-output' refs #9552
Tom Clegg [Thu, 4 Aug 2016 15:31:44 +0000 (11:31 -0400)]
Merge branch '9406-cgroup-parent'
closes #9406
Tom Clegg [Thu, 4 Aug 2016 15:31:34 +0000 (11:31 -0400)]
9406: Explain why there are two separate CgroupParent fields.
Peter Amstutz [Thu, 4 Aug 2016 14:23:12 +0000 (10:23 -0400)]
9552: If compute_checksum is true, check if checksum needs to be computed on final output, and add it if missing.
Tom Clegg [Thu, 4 Aug 2016 14:10:12 +0000 (10:10 -0400)]
Merge branch '9595-mount-json'
closes #9595
Tom Clegg [Thu, 4 Aug 2016 06:26:54 +0000 (02:26 -0400)]
9595: Fix hardcoded temp dir in test. Add checks for CleanupDirs().
Tom Clegg [Thu, 4 Aug 2016 05:48:33 +0000 (01:48 -0400)]
9595: Support "json" mount type.
Ward Vandewege [Thu, 4 Aug 2016 00:48:32 +0000 (20:48 -0400)]
Merge branch '9677-update-install-docs'
closes #9677
Ward Vandewege [Wed, 3 Aug 2016 19:54:34 +0000 (15:54 -0400)]
Document how to download and tag the arvados/jobs Docker image.
refs #9677
Peter Amstutz [Wed, 3 Aug 2016 20:47:05 +0000 (16:47 -0400)]
Merge branch '9674-cwl-initialworkdir' closes #9674
Lucas Di Pentima [Tue, 2 Aug 2016 23:29:04 +0000 (20:29 -0300)]
Merge branch '9617-reject-container-reqs-without-vcpus'
9617, 9618: Added runtime constraints to fixtures.
refs #9617, #9618
Lucas Di Pentima [Tue, 2 Aug 2016 20:27:14 +0000 (17:27 -0300)]
9617, 9618: Added runtime constraints to container requests fixtures. Wrote a new test to load a CR from a fixture and update some attribute to test for the new validations
Lucas Di Pentima [Tue, 2 Aug 2016 15:28:18 +0000 (12:28 -0300)]
Merge branch '9617-reject-container-reqs-without-vcpus'
9617, 9618: Require positive integers on both 'vcpus' and 'ram' runtime constraints when a container request is committed. Closes #9617, #9618
Lucas Di Pentima [Tue, 2 Aug 2016 15:26:03 +0000 (12:26 -0300)]
9617: Corrected variable & test naming
Peter Amstutz [Tue, 2 Aug 2016 15:17:18 +0000 (11:17 -0400)]
9674: InitialWorkDirPathMapper bug fix to set correct source path for files in
virtual working directory.
Lucas Di Pentima [Tue, 2 Aug 2016 14:50:05 +0000 (11:50 -0300)]
9617: Reorganized code reusing the invalid_constraint list on both create & update tests. Separated the successful test case to be on its own.
Tom Clegg [Tue, 2 Aug 2016 13:42:07 +0000 (09:42 -0400)]
9406: Add -cgroup-parent-subsystem argument.
This accommodates setups where the scheduling system (e.g., SLURM)
uses cgroups to impose resource limits on crunch-run itself, and those
limits are meant to include resources used by the container itself.
Normally, when docker creates a container, it creates new cgroups
inside the "docker" cgroup, which circumvents any resource limits set
on crunch-run's own cgroup.
If a subsystem is specified (e.g., -cgroup-parent-subsystem=memory),
crunch-run will look in /proc/self/cgroup to find the current cgroup
for the given subsystem, and instruct docker to use that cgroup as the
parent of the new container's cgroup instead.
Lucas Di Pentima [Tue, 2 Aug 2016 13:28:57 +0000 (10:28 -0300)]
9617: Added one more invalid constraint case to the tests, also check for container_uuid on successful test
Tom Clegg [Tue, 2 Aug 2016 01:42:37 +0000 (21:42 -0400)]
9406: Support passing additional arguments from crunch-dispatch-slurm to crunch-run.
Lucas Di Pentima [Mon, 1 Aug 2016 21:30:16 +0000 (18:30 -0300)]
9617: Corrected style following suggestions and also compacted the code to avoid repetition.
Lucas Di Pentima [Mon, 1 Aug 2016 19:44:16 +0000 (16:44 -0300)]
9617: Added validations on updates & tests on sucessful cases.
Lucas Di Pentima [Fri, 29 Jul 2016 21:07:15 +0000 (18:07 -0300)]
9617, 9618: Added validations and wrote the corresponding test. Also, corrected some other tests to conform this new requirement.
radhika [Thu, 28 Jul 2016 18:27:27 +0000 (14:27 -0400)]
refs #9552
Merge branch '9552-compute-checksum-flag'
radhika [Thu, 28 Jul 2016 17:36:26 +0000 (13:36 -0400)]
9552: add compute-checksum flag to arvados-cwl-runner
Ward Vandewege [Thu, 28 Jul 2016 00:23:06 +0000 (20:23 -0400)]
We have a new build job at ci.curoverse.com. Update the build status
icon in README.md accordingly.
No issue #
radhika [Wed, 27 Jul 2016 21:15:00 +0000 (17:15 -0400)]
closes #9581
Merge branch '9581-slurm-dispatcher-config'
radhika [Tue, 26 Jul 2016 16:28:27 +0000 (12:28 -0400)]
9581: add json config file handling to slurm dispatcher.
Tom Clegg [Wed, 27 Jul 2016 19:22:26 +0000 (15:22 -0400)]
Merge branch '8555-s3-trash'
closes #8555
Tom Clegg [Wed, 27 Jul 2016 15:29:17 +0000 (11:29 -0400)]
8555: Fail Go tests if not gofmted.
Tom Clegg [Tue, 26 Jul 2016 20:29:57 +0000 (16:29 -0400)]
8555: golint
Tom Clegg [Tue, 26 Jul 2016 20:26:15 +0000 (16:26 -0400)]
8555: Fix up comments.
Tom Clegg [Tue, 26 Jul 2016 20:20:30 +0000 (16:20 -0400)]
8555: gofmt
Tom Clegg [Tue, 26 Jul 2016 16:25:35 +0000 (12:25 -0400)]
8555: Test Get() after successful Untrash. Test Put+Mtime in all scenarios.
Tom Clegg [Tue, 26 Jul 2016 15:07:46 +0000 (11:07 -0400)]
8555: Untrash to repair inconsistent state (trash/X without recentX).
Tom Clegg [Tue, 26 Jul 2016 13:44:56 +0000 (09:44 -0400)]
8555: Log statistics in EmptyTrash.
Tom Clegg [Tue, 26 Jul 2016 13:38:07 +0000 (09:38 -0400)]
8555: Improve variable names and comments.
Tom Clegg [Tue, 26 Jul 2016 13:19:30 +0000 (09:19 -0400)]
8555: Reformat test cases.
Tom Clegg [Fri, 22 Jul 2016 18:04:34 +0000 (14:04 -0400)]
8555: De-obfuscate test case.
Tom Clegg [Thu, 21 Jul 2016 15:44:17 +0000 (11:44 -0400)]
8555: Fix EmptyTrash so it can clean up old races.
Tom Clegg [Wed, 20 Jul 2016 15:56:59 +0000 (11:56 -0400)]
8555: golint
Tom Clegg [Wed, 20 Jul 2016 15:42:14 +0000 (11:42 -0400)]
8555: gofmt
Tom Clegg [Wed, 20 Jul 2016 15:40:41 +0000 (11:40 -0400)]
8555: Test various backend states. Update recent/X timestamp during Untrash.
Tom Clegg [Mon, 18 Jul 2016 19:47:23 +0000 (15:47 -0400)]
8555: Move checkRaceWindow out to a func, tweak comments
Tom Clegg [Thu, 14 Jul 2016 20:08:16 +0000 (16:08 -0400)]
8555: Add log messages for time-parsing errors.
Tom Clegg [Thu, 14 Jul 2016 19:53:20 +0000 (15:53 -0400)]
8555: Implement trash with race-recovery for S3 volumes.
See https://dev.arvados.org/projects/arvados/wiki/S3_bucket_volume_implementation
radhika [Wed, 27 Jul 2016 17:26:39 +0000 (13:26 -0400)]
closes #8568
Merge branch '8568-docker-version-support'
radhika [Wed, 27 Jul 2016 17:26:06 +0000 (13:26 -0400)]
Merge branch 'master' into 8568-docker-version-support
radhika [Wed, 27 Jul 2016 16:13:15 +0000 (12:13 -0400)]
closes #9552
Merge branch '9552-compute-checksum-flag'
radhika [Wed, 27 Jul 2016 16:07:35 +0000 (12:07 -0400)]
Merge branch 'master' into 9552-compute-checksum-flag
Conflicts:
sdk/cwl/setup.py
Brett Smith [Tue, 26 Jul 2016 20:54:39 +0000 (16:54 -0400)]
9659: Bump CWL SDK's versioned dependency on PySDK.
The CWL SDK depends on the change made to arv-run in
27816b602e9da83a2565e6fe8f87f250555b1ba5. Update the version
dependency in setup.py to reflect this. Refs #9570, #9659.
radhika [Tue, 26 Jul 2016 20:23:14 +0000 (16:23 -0400)]
8568: Enhance image_hash parsing when using Docker 1.10+
radhika [Tue, 26 Jul 2016 18:47:32 +0000 (14:47 -0400)]
9552: update cwltool version, which offer compute_checksum flag.
projects / arvados.git / log