Tom Clegg [Thu, 4 Aug 2016 14:10:12 +0000 (10:10 -0400)]
Merge branch '9595-mount-json'
closes #9595
Tom Clegg [Thu, 4 Aug 2016 06:26:54 +0000 (02:26 -0400)]
9595: Fix hardcoded temp dir in test. Add checks for CleanupDirs().
Tom Clegg [Thu, 4 Aug 2016 05:48:33 +0000 (01:48 -0400)]
9595: Support "json" mount type.
Ward Vandewege [Thu, 4 Aug 2016 00:48:32 +0000 (20:48 -0400)]
Merge branch '9677-update-install-docs'
closes #9677
Ward Vandewege [Wed, 3 Aug 2016 19:54:34 +0000 (15:54 -0400)]
Document how to download and tag the arvados/jobs Docker image.
refs #9677
Peter Amstutz [Wed, 3 Aug 2016 20:47:05 +0000 (16:47 -0400)]
Merge branch '9674-cwl-initialworkdir' closes #9674
Lucas Di Pentima [Tue, 2 Aug 2016 23:29:04 +0000 (20:29 -0300)]
Merge branch '9617-reject-container-reqs-without-vcpus'
9617, 9618: Added runtime constraints to fixtures.
refs #9617, #9618
Lucas Di Pentima [Tue, 2 Aug 2016 20:27:14 +0000 (17:27 -0300)]
9617, 9618: Added runtime constraints to container requests fixtures. Wrote a new test to load a CR from a fixture and update some attribute to test for the new validations
Lucas Di Pentima [Tue, 2 Aug 2016 15:28:18 +0000 (12:28 -0300)]
Merge branch '9617-reject-container-reqs-without-vcpus'
9617, 9618: Require positive integers on both 'vcpus' and 'ram' runtime constraints when a container request is committed. Closes #9617, #9618
Lucas Di Pentima [Tue, 2 Aug 2016 15:26:03 +0000 (12:26 -0300)]
9617: Corrected variable & test naming
Peter Amstutz [Tue, 2 Aug 2016 15:17:18 +0000 (11:17 -0400)]
9674: InitialWorkDirPathMapper bug fix to set correct source path for files in
virtual working directory.
Lucas Di Pentima [Tue, 2 Aug 2016 14:50:05 +0000 (11:50 -0300)]
9617: Reorganized code reusing the invalid_constraint list on both create & update tests. Separated the successful test case to be on its own.
Tom Clegg [Tue, 2 Aug 2016 13:42:07 +0000 (09:42 -0400)]
9406: Add -cgroup-parent-subsystem argument.
This accommodates setups where the scheduling system (e.g., SLURM)
uses cgroups to impose resource limits on crunch-run itself, and those
limits are meant to include resources used by the container itself.
Normally, when docker creates a container, it creates new cgroups
inside the "docker" cgroup, which circumvents any resource limits set
on crunch-run's own cgroup.
If a subsystem is specified (e.g., -cgroup-parent-subsystem=memory),
crunch-run will look in /proc/self/cgroup to find the current cgroup
for the given subsystem, and instruct docker to use that cgroup as the
parent of the new container's cgroup instead.
Lucas Di Pentima [Tue, 2 Aug 2016 13:28:57 +0000 (10:28 -0300)]
9617: Added one more invalid constraint case to the tests, also check for container_uuid on successful test
Tom Clegg [Tue, 2 Aug 2016 01:42:37 +0000 (21:42 -0400)]
9406: Support passing additional arguments from crunch-dispatch-slurm to crunch-run.
Lucas Di Pentima [Mon, 1 Aug 2016 21:30:16 +0000 (18:30 -0300)]
9617: Corrected style following suggestions and also compacted the code to avoid repetition.
Lucas Di Pentima [Mon, 1 Aug 2016 19:44:16 +0000 (16:44 -0300)]
9617: Added validations on updates & tests on sucessful cases.
Lucas Di Pentima [Fri, 29 Jul 2016 21:07:15 +0000 (18:07 -0300)]
9617, 9618: Added validations and wrote the corresponding test. Also, corrected some other tests to conform this new requirement.
Lucas Di Pentima [Fri, 29 Jul 2016 13:08:36 +0000 (10:08 -0300)]
9463: Unify replication_desired & write_copies parameters to only one, passing it at Collection's constructor and let the class decide which value is best from default when None is passed.
radhika [Thu, 28 Jul 2016 18:27:27 +0000 (14:27 -0400)]
refs #9552
Merge branch '9552-compute-checksum-flag'
radhika [Thu, 28 Jul 2016 17:36:26 +0000 (13:36 -0400)]
9552: add compute-checksum flag to arvados-cwl-runner
Lucas Di Pentima [Thu, 28 Jul 2016 17:58:12 +0000 (14:58 -0300)]
9463: Unified use of 'replication_desired' param on Collection class at instantiation.
Removed the need to pass the number of copies to be written to keep on save_new() method,
it will be inferred from replication_desired setting or looked up from defaults.
Added functionality to Collection class to keep replication_desired configuration when
loading an already existing collection from API server, with tests validating this new
behaviour.
Corrected some already existing tests to work with this changes.
Ward Vandewege [Thu, 28 Jul 2016 00:23:06 +0000 (20:23 -0400)]
We have a new build job at ci.curoverse.com. Update the build status
icon in README.md accordingly.
No issue #
radhika [Wed, 27 Jul 2016 21:15:00 +0000 (17:15 -0400)]
closes #9581
Merge branch '9581-slurm-dispatcher-config'
radhika [Tue, 26 Jul 2016 16:28:27 +0000 (12:28 -0400)]
9581: add json config file handling to slurm dispatcher.
Tom Clegg [Wed, 27 Jul 2016 19:22:26 +0000 (15:22 -0400)]
Merge branch '8555-s3-trash'
closes #8555
Tom Clegg [Wed, 27 Jul 2016 15:29:17 +0000 (11:29 -0400)]
8555: Fail Go tests if not gofmted.
Tom Clegg [Tue, 26 Jul 2016 20:29:57 +0000 (16:29 -0400)]
8555: golint
Tom Clegg [Tue, 26 Jul 2016 20:26:15 +0000 (16:26 -0400)]
8555: Fix up comments.
Tom Clegg [Tue, 26 Jul 2016 20:20:30 +0000 (16:20 -0400)]
8555: gofmt
Tom Clegg [Tue, 26 Jul 2016 16:25:35 +0000 (12:25 -0400)]
8555: Test Get() after successful Untrash. Test Put+Mtime in all scenarios.
Tom Clegg [Tue, 26 Jul 2016 15:07:46 +0000 (11:07 -0400)]
8555: Untrash to repair inconsistent state (trash/X without recentX).
Tom Clegg [Tue, 26 Jul 2016 13:44:56 +0000 (09:44 -0400)]
8555: Log statistics in EmptyTrash.
Tom Clegg [Tue, 26 Jul 2016 13:38:07 +0000 (09:38 -0400)]
8555: Improve variable names and comments.
Tom Clegg [Tue, 26 Jul 2016 13:19:30 +0000 (09:19 -0400)]
8555: Reformat test cases.
Tom Clegg [Fri, 22 Jul 2016 18:04:34 +0000 (14:04 -0400)]
8555: De-obfuscate test case.
Tom Clegg [Thu, 21 Jul 2016 15:44:17 +0000 (11:44 -0400)]
8555: Fix EmptyTrash so it can clean up old races.
Tom Clegg [Wed, 20 Jul 2016 15:56:59 +0000 (11:56 -0400)]
8555: golint
Tom Clegg [Wed, 20 Jul 2016 15:42:14 +0000 (11:42 -0400)]
8555: gofmt
Tom Clegg [Wed, 20 Jul 2016 15:40:41 +0000 (11:40 -0400)]
8555: Test various backend states. Update recent/X timestamp during Untrash.
Tom Clegg [Mon, 18 Jul 2016 19:47:23 +0000 (15:47 -0400)]
8555: Move checkRaceWindow out to a func, tweak comments
Tom Clegg [Thu, 14 Jul 2016 20:08:16 +0000 (16:08 -0400)]
8555: Add log messages for time-parsing errors.
Tom Clegg [Thu, 14 Jul 2016 19:53:20 +0000 (15:53 -0400)]
8555: Implement trash with race-recovery for S3 volumes.
See https://dev.arvados.org/projects/arvados/wiki/S3_bucket_volume_implementation
radhika [Wed, 27 Jul 2016 17:26:39 +0000 (13:26 -0400)]
closes #8568
Merge branch '8568-docker-version-support'
radhika [Wed, 27 Jul 2016 17:26:06 +0000 (13:26 -0400)]
Merge branch 'master' into 8568-docker-version-support
radhika [Wed, 27 Jul 2016 16:13:15 +0000 (12:13 -0400)]
closes #9552
Merge branch '9552-compute-checksum-flag'
radhika [Wed, 27 Jul 2016 16:07:35 +0000 (12:07 -0400)]
Merge branch 'master' into 9552-compute-checksum-flag
Conflicts:
sdk/cwl/setup.py
Lucas Di Pentima [Wed, 27 Jul 2016 12:44:18 +0000 (09:44 -0300)]
9463: Parameter naming unification on _BlockManager. Corrected mocked discovery doc on test.
Lucas Di Pentima [Tue, 26 Jul 2016 22:39:27 +0000 (19:39 -0300)]
9463: Temp files & dirs creation/cleanup moved to setUp and tearDown stages. Replaced usage of reporter function with a second wrapper function to make the test cleaner
Lucas Di Pentima [Tue, 26 Jul 2016 20:55:29 +0000 (17:55 -0300)]
9463: Added logging messages to warn the user about anomalous cases
Brett Smith [Tue, 26 Jul 2016 20:54:39 +0000 (16:54 -0400)]
9659: Bump CWL SDK's versioned dependency on PySDK.
The CWL SDK depends on the change made to arv-run in
27816b602e9da83a2565e6fe8f87f250555b1ba5. Update the version
dependency in setup.py to reflect this. Refs #9570, #9659.
radhika [Tue, 26 Jul 2016 20:23:14 +0000 (16:23 -0400)]
8568: Enhance image_hash parsing when using Docker 1.10+
Lucas Di Pentima [Tue, 26 Jul 2016 19:45:29 +0000 (16:45 -0300)]
9463: Optimizations on _write_ffile()
radhika [Tue, 26 Jul 2016 18:47:32 +0000 (14:47 -0400)]
9552: update cwltool version, which offer compute_checksum flag.
Lucas Di Pentima [Tue, 26 Jul 2016 17:46:09 +0000 (14:46 -0300)]
9463: Use a constant as a template for initializing the empty cache
Lucas Di Pentima [Tue, 26 Jul 2016 14:28:38 +0000 (11:28 -0300)]
Merge branch '9551-python-keep-services-env'
9551: Python SDK support for ARVADOS_KEEP_SERVICES environment variable. Closes #9551
Lucas Di Pentima [Tue, 26 Jul 2016 14:03:57 +0000 (11:03 -0300)]
9551: Replaced ARVADOS_KEEP_PROXY with ARVADOS_KEEP_SERVICES new var on tests. Added proxy URI validation test.
Lucas Di Pentima [Tue, 26 Jul 2016 13:34:12 +0000 (10:34 -0300)]
9551: Proxy URLs validation. Iteration optimization builind _keep_services list
Tom Clegg [Tue, 26 Jul 2016 12:59:48 +0000 (08:59 -0400)]
Merge branch '9550-keep-services-env'
closes #9550
Tom Clegg [Mon, 25 Jul 2016 20:41:25 +0000 (16:41 -0400)]
9550: Ignore extra spaces in ARVADOS_KEEP_SERVICES, and ensure each entry is an absolute URI.
Tom Clegg [Mon, 25 Jul 2016 19:54:41 +0000 (15:54 -0400)]
9550: Update comments.
Tom Clegg [Fri, 22 Jul 2016 13:59:32 +0000 (09:59 -0400)]
9550: Add service discovery override field to new SDK.
Tom Clegg [Fri, 22 Jul 2016 13:41:47 +0000 (09:41 -0400)]
9550: Allow overriding keep services discovery with ARVADOS_KEEP_SERVICES env var.
Lucas Di Pentima [Tue, 26 Jul 2016 12:39:17 +0000 (09:39 -0300)]
9551: Change proxy parameter priority over environment variables. Updated docstring to mention ARVADOS_KEEP_SERVICES.
Lucas Di Pentima [Mon, 25 Jul 2016 20:32:46 +0000 (17:32 -0300)]
9551: Added support for env var ARVADOS_KEEP_SERVICES as a special case of a proxy.
Lucas Di Pentima [Fri, 22 Jul 2016 17:51:52 +0000 (14:51 -0300)]
9463: Finished writing tests for ArvPutUploadJob
Tom Clegg [Thu, 21 Jul 2016 23:35:56 +0000 (19:35 -0400)]
Merge branch '9613-test-profile-save'
closes #9613
Tom Clegg [Thu, 21 Jul 2016 23:34:43 +0000 (19:34 -0400)]
9613: Touch up "find required field" loop.
Tom Clegg [Thu, 21 Jul 2016 19:50:00 +0000 (15:50 -0400)]
9613: Ensure submitted values are saved, and displayed on the edit form after it reloads.
Tom Clegg [Thu, 21 Jul 2016 19:25:28 +0000 (15:25 -0400)]
9613: Dry up user profile tests.
Ward Vandewege [Thu, 21 Jul 2016 19:43:07 +0000 (15:43 -0400)]
Merge branch '9639-split-workbench-tests'
refs #9639
Ward Vandewege [Thu, 21 Jul 2016 19:02:21 +0000 (15:02 -0400)]
9639: Make it possible to run unit, functional and integration tests for
Workbench separately.
refs #9639
Tom Clegg [Thu, 21 Jul 2016 02:35:01 +0000 (22:35 -0400)]
9639: Relax unnecessarily strict time limit.
refs #9639
Tom Clegg [Wed, 20 Jul 2016 21:14:47 +0000 (17:14 -0400)]
9639: Delete all cached discovery docs, but do not delete the cache directory.
refs #9639
Tom Clegg [Wed, 20 Jul 2016 20:03:18 +0000 (16:03 -0400)]
9639: Avoid sharing rvm gemsets between concurrent builds.
refs #9639
Lucas Di Pentima [Tue, 19 Jul 2016 20:09:38 +0000 (17:09 -0300)]
9463: Updated integration test to mock current implementation
radhika [Tue, 19 Jul 2016 20:03:23 +0000 (16:03 -0400)]
closes #9319
Merge branch '9319-all-processes-index'
radhika [Tue, 5 Jul 2016 23:13:13 +0000 (19:13 -0400)]
9319: /all_processes index page.
Tom Clegg [Tue, 19 Jul 2016 19:11:43 +0000 (15:11 -0400)]
9624: Fix format string.
closes #9624
Lucas Di Pentima [Tue, 19 Jul 2016 15:07:22 +0000 (12:07 -0300)]
9463: Using finally: on except: clause to stop the updater thread on error or finished work. When counting the collection size, stopped ignoring items that are not Collection or ArvadosFile, in case some other class is used in the future. When getting block locators (for example when called with --raw), don\'t make sure the blocks are flushed to Keep before asking for segments
Tom Clegg [Tue, 19 Jul 2016 13:30:27 +0000 (09:30 -0400)]
Merge branch '9613-user-profile-string-keys'
refs #9613
Brett Smith [Tue, 19 Jul 2016 13:07:02 +0000 (09:07 -0400)]
9262: Stop setting sbatch --priority in crunch-dispatch-slurm.
There are a few reasons we want to back this out:
* `--priority` is a relatively new feature of SLURM, added in 14.03,
and it's not clear how many users are running that.
* Setting `--priority` requires extra privileges, per the sbatch man
page, and it's not clear crunch-dispatch-slurm should run with
those.
* There are multiple ways to set the "priority" of a SLURM job.
There's also a niceness value. Which is correct to use is probably
ultimately going to depend on site rules or conventions. We need to
get more feedback on what people are doing before we implement
something. We might need configurations options along the lines of
"translate container priority into this option with this scaling
factor," or something like that.
Per discussion with Tom Clegg. Closes #9626.
Lucas Di Pentima [Mon, 18 Jul 2016 21:25:42 +0000 (18:25 -0300)]
Merge branch '9438-http-default-timeout'
9438: Added a new var DefaultSecureClient and assigned a 5 minute timeout to both InsecureHTTPClient and DefaultSecureClient.
refs #9438
Lucas Di Pentima [Mon, 18 Jul 2016 20:47:38 +0000 (17:47 -0300)]
9438: Eliminated redundant casting
Tom Clegg [Mon, 18 Jul 2016 03:24:11 +0000 (23:24 -0400)]
9613: Stop prepending ":" to profile keys.
Nico Cesar [Sun, 17 Jul 2016 13:37:09 +0000 (09:37 -0400)]
Merge branch '9614-cwl-no-websockets'
closes #9614
Peter Amstutz [Sat, 16 Jul 2016 20:21:58 +0000 (16:21 -0400)]
9614: Don't use websockets for event notification until it is more reliable.
Ward Vandewege [Fri, 15 Jul 2016 21:09:38 +0000 (17:09 -0400)]
Pin the html5lib package we build for centos6 and centos7 to version
0.
9999999, because newer versions have a dependency on a
python-setuptools 0.18.5 which we don't have in the build containers
(yet).
refs #9459
Brett Smith [Fri, 15 Jul 2016 20:14:56 +0000 (16:14 -0400)]
Merge branch '9459-arvbox-demo-version-arg-wip'
Refs #9459. Closes #9531.
Brett Smith [Thu, 30 Jun 2016 21:45:12 +0000 (17:45 -0400)]
9459: arvbox-demo accepts version arguments for Git checkouts.
Lucas Di Pentima [Fri, 15 Jul 2016 19:46:09 +0000 (16:46 -0300)]
9438: Added a new var DefaultSecureClient and assigned a 5 minute timeout to both InsecureHTTPClient and DefaultSecureClient
Peter Amstutz [Fri, 15 Jul 2016 19:16:17 +0000 (15:16 -0400)]
Bump version of setuptools used by run_tests.sh for cwl-runner dependency refs #9570
Peter Amstutz [Fri, 15 Jul 2016 18:30:50 +0000 (14:30 -0400)]
Merge branch '9369-arv-cwl-docs' closes #9369
Brett Smith [Fri, 15 Jul 2016 18:26:45 +0000 (14:26 -0400)]
9369: Copyedit CLI SDK install guide.
Peter Amstutz [Fri, 15 Jul 2016 18:11:37 +0000 (14:11 -0400)]
9369: Remove cwl install from Python SDK page. Remove "sudo -i" from
pip/python install instructions. Fixup use of ~$ prompt in example
instructions.
Peter Amstutz [Fri, 15 Jul 2016 17:11:49 +0000 (13:11 -0400)]
Merge branch 'master' into 9369-arv-cwl-docs
Peter Amstutz [Fri, 15 Jul 2016 17:11:07 +0000 (13:11 -0400)]
Merge branch '9570-cwl-runner-fixes' closes #9570 (again)
Brett Smith [Fri, 15 Jul 2016 15:03:56 +0000 (11:03 -0400)]
9369: Copyedits throughout the CWL documentation.
Peter Amstutz [Fri, 15 Jul 2016 14:00:22 +0000 (10:00 -0400)]
9369: Add note about Javascript eval timeout.
Peter Amstutz [Fri, 15 Jul 2016 13:52:02 +0000 (09:52 -0400)]
9570: Raise exception on workflow failure.