arvados.git
8 years ago10383: Tests have been adapted to the new behaviour.
Lucas Di Pentima [Sat, 12 Nov 2016 00:37:30 +0000 (21:37 -0300)]
10383: Tests have been adapted to the new behaviour.

8 years ago10383: Enhancements/fixes to the collection update feature:
Lucas Di Pentima [Fri, 11 Nov 2016 23:38:02 +0000 (20:38 -0300)]
10383: Enhancements/fixes to the collection update feature:
* The cache file saves the collection UUID when uploading for the first time.
* On initial upload success, the cache file is not deleted, so it can be used for updating the collection.
* Once the initial collection is done, the manifest saved in the cache file is used to detect any remote file change, so the local file is then re-uploaded completely.
* If there weren't remote changes from the last upload/update, new files are uploaded and partial uploads resumed.
* If a new upload (to create a new collection) is attempted without deleting the previous cache file, a message will alert the user and the operation will be cancelled.

8 years ago10383: Fixed Collection.find() so that it returns None also when some partial part...
Lucas Di Pentima [Fri, 11 Nov 2016 23:03:16 +0000 (20:03 -0300)]
10383: Fixed Collection.find() so that it returns None also when some partial part of the path does not exist. Updated test to cover more cases.

8 years ago10383: Added --update-collection parameter to arv-put so that it uploads new files...
Lucas Di Pentima [Fri, 4 Nov 2016 21:10:53 +0000 (18:10 -0300)]
10383: Added --update-collection parameter to arv-put so that it uploads new files into an existing collection.

8 years agoMerge branch '10343-dockercleaner-config' refs #10343
Tom Clegg [Mon, 31 Oct 2016 21:38:50 +0000 (17:38 -0400)]
Merge branch '10343-dockercleaner-config' refs #10343

8 years ago10343: Merge branch 'master' into 10343-dockercleaner-config
Tom Clegg [Mon, 31 Oct 2016 20:08:01 +0000 (16:08 -0400)]
10343: Merge branch 'master' into 10343-dockercleaner-config

8 years ago10343: Add comment about unwieldy ExecStart command.
Tom Clegg [Mon, 31 Oct 2016 19:23:53 +0000 (15:23 -0400)]
10343: Add comment about unwieldy ExecStart command.

8 years agoCompute correct arvados-cwl-runner version, refs #10194
Peter Amstutz [Mon, 31 Oct 2016 18:06:12 +0000 (14:06 -0400)]
Compute correct arvados-cwl-runner version, refs #10194

8 years agotag arvados/arvbox-base with latest so that FROM arvados/arvbox-base:latest works...
Peter Amstutz [Mon, 31 Oct 2016 16:00:21 +0000 (12:00 -0400)]
tag arvados/arvbox-base with latest so that FROM arvados/arvbox-base:latest works. refs #10025

8 years agoMerge branch '10315-new-arv-put-performance'
Lucas Di Pentima [Mon, 31 Oct 2016 13:58:38 +0000 (10:58 -0300)]
Merge branch '10315-new-arv-put-performance'
Closes #10315

8 years ago10343: Update postinst script to run on Red Hat systems, and accommodate Python33...
Tom Clegg [Fri, 28 Oct 2016 22:22:24 +0000 (18:22 -0400)]
10343: Update postinst script to run on Red Hat systems, and accommodate Python33 Software Collections.

8 years ago10315: Merge branch 'master' into 10315-new-arv-put-performance
Lucas Di Pentima [Fri, 28 Oct 2016 18:35:19 +0000 (15:35 -0300)]
10315: Merge branch 'master' into 10315-new-arv-put-performance

8 years ago10315: Moved conditional on pending write size checking
Lucas Di Pentima [Fri, 28 Oct 2016 18:33:03 +0000 (15:33 -0300)]
10315: Moved conditional on pending write size checking

8 years ago10343: Improve log message.
Tom Clegg [Fri, 28 Oct 2016 18:06:34 +0000 (14:06 -0400)]
10343: Improve log message.

8 years agoExplicitly list all root-level directories in this repository in the
Ward Vandewege [Fri, 28 Oct 2016 17:26:48 +0000 (13:26 -0400)]
Explicitly list all root-level directories in this repository in the
COPYING file.

No issue #

8 years ago10315: Update pending write size count just before packing small blocks to see if...
Lucas Di Pentima [Fri, 28 Oct 2016 14:56:20 +0000 (11:56 -0300)]
10315: Update pending write size count just before packing small blocks to see if there really are enough to fill a full one.

8 years ago10315: Added back the BlockManager's put threads lazy start, but with a specific...
Lucas Di Pentima [Fri, 28 Oct 2016 14:27:22 +0000 (11:27 -0300)]
10315: Added back the BlockManager's put threads lazy start, but with a specific lock, so it can be called from a @synchronized method.

8 years ago10343: Update arvados-docker-cleaner install docs to use packaged systemd unit file.
Tom Clegg [Fri, 28 Oct 2016 06:57:47 +0000 (02:57 -0400)]
10343: Update arvados-docker-cleaner install docs to use packaged systemd unit file.

8 years ago10343: Allow running without config file, if -config arg is not given.
Tom Clegg [Thu, 27 Oct 2016 16:55:35 +0000 (12:55 -0400)]
10343: Allow running without config file, if -config arg is not given.

8 years agoPin the python-daemon backport for centos7, newer versions have
Ward Vandewege [Wed, 26 Oct 2016 18:58:11 +0000 (14:58 -0400)]
Pin the python-daemon backport for centos7, newer versions have
additional dependencies.

No issue #

8 years agoMerge branch '10008-check-token-exp-on-open' refs #10008
Tom Clegg [Wed, 26 Oct 2016 15:40:42 +0000 (11:40 -0400)]
Merge branch '10008-check-token-exp-on-open' refs #10008

8 years ago10008: Comment to explain known deficiencies.
Tom Clegg [Wed, 26 Oct 2016 15:13:10 +0000 (11:13 -0400)]
10008: Comment to explain known deficiencies.

8 years ago10315: Moved the BlockManager's put threads lazy loading to __init__() to avoid locki...
Lucas Di Pentima [Wed, 26 Oct 2016 15:01:55 +0000 (12:01 -0300)]
10315: Moved the BlockManager's put threads lazy loading to __init__() to avoid locking issues when calling commit_bufferblock() fropm within an already @synchronized method.

8 years ago10315: Brought back the new arv-put tests from 9701 branch.
Lucas Di Pentima [Tue, 25 Oct 2016 23:04:23 +0000 (20:04 -0300)]
10315: Brought back the new arv-put tests from 9701 branch.

8 years ago10315: Enhanced performance when writing lots of small files without flushing by...
Lucas Di Pentima [Tue, 25 Oct 2016 22:52:23 +0000 (19:52 -0300)]
10315: Enhanced performance when writing lots of small files without flushing by adding every closed file's size to a counter instead of scanning the pending bufferblock list every time.

8 years agocloses #10079
radhika [Tue, 25 Oct 2016 22:25:58 +0000 (18:25 -0400)]
closes #10079
Merge branch '10079-pipeline-display-perf'

8 years ago10079: show spinner when a child component details are being displayed using ajax.
radhika [Tue, 25 Oct 2016 16:35:55 +0000 (12:35 -0400)]
10079: show spinner when a child component details are being displayed using ajax.

8 years ago10079: add "parent" to work_unit to aid the child display
radhika [Sun, 23 Oct 2016 01:47:51 +0000 (21:47 -0400)]
10079: add "parent" to work_unit to aid the child display

8 years ago10079: fixed route so that @object is available
radhika [Fri, 21 Oct 2016 21:19:53 +0000 (17:19 -0400)]
10079: fixed route so that @object is available

8 years ago10079: checking log_link on children needs to be an integration test.
radhika [Wed, 19 Oct 2016 21:17:56 +0000 (17:17 -0400)]
10079: checking log_link on children needs to be an integration test.

8 years ago10079: show_child_component method needs anonymous access
radhika [Wed, 19 Oct 2016 20:19:23 +0000 (16:19 -0400)]
10079: show_child_component method needs anonymous access

8 years ago10079: ajax to display child component details; not working yet.
radhika [Tue, 18 Oct 2016 04:24:13 +0000 (00:24 -0400)]
10079: ajax to display child component details; not working yet.

8 years ago10079: Move Log and Output links into child details. This will make it possible
radhika [Thu, 13 Oct 2016 21:21:04 +0000 (17:21 -0400)]
10079: Move Log and Output links into child details. This will make it possible
to not need tt preload the jobs and outputs during the pipeline display. Instead,
they can be fetched when the child panel is expanded by the user.

8 years ago10291: Update --help message. refs #10291
Tom Clegg [Tue, 25 Oct 2016 19:05:10 +0000 (15:05 -0400)]
10291: Update --help message. refs #10291

8 years agoMerge branch '10172-crunch2-container-output' closes #10172
Peter Amstutz [Tue, 25 Oct 2016 17:00:23 +0000 (13:00 -0400)]
Merge branch '10172-crunch2-container-output' closes #10172

8 years ago10008: Comment to explain ancestor update.
Tom Clegg [Tue, 25 Oct 2016 15:39:17 +0000 (11:39 -0400)]
10008: Comment to explain ancestor update.

8 years ago10172: Add container.output check to TestFullRunWithAPI.
Peter Amstutz [Tue, 25 Oct 2016 15:08:43 +0000 (11:08 -0400)]
10172: Add container.output check to TestFullRunWithAPI.

8 years ago10172: Add skip_before_filter :find_object_by_uuid for current. Typo fixes.
Peter Amstutz [Tue, 25 Oct 2016 14:51:14 +0000 (10:51 -0400)]
10172: Add skip_before_filter :find_object_by_uuid for current.  Typo fixes.

8 years agoMerge branch '10194-jobs-image-versioning' refs #10194
Peter Amstutz [Tue, 25 Oct 2016 01:33:40 +0000 (21:33 -0400)]
Merge branch '10194-jobs-image-versioning' refs #10194

8 years ago10194: Improve versioning of arvados/jobs image.
Peter Amstutz [Fri, 21 Oct 2016 20:35:09 +0000 (16:35 -0400)]
10194: Improve versioning of arvados/jobs image.

arvados-cwl-runner sets version to max(cwl-runner, python-sdk).

build/run-build-docker-jobs-image.sh determines package versions corresponding
to current commit and supplies explicit package versions as Docker build arg.

8 years ago10315: Reduced the time between updates.
Lucas Di Pentima [Mon, 24 Oct 2016 14:34:37 +0000 (11:34 -0300)]
10315: Reduced the time between updates.

8 years ago10315: Allow small block packing passing flush=False.
Lucas Di Pentima [Mon, 24 Oct 2016 11:47:32 +0000 (08:47 -0300)]
10315: Allow small block packing passing flush=False.

8 years agoMerge branch '10232-non-exclusive-keys' refs #10232
Tom Clegg [Fri, 21 Oct 2016 20:08:45 +0000 (16:08 -0400)]
Merge branch '10232-non-exclusive-keys' refs #10232

8 years ago10008: Move test setup from __init__() to setUp()
Tom Clegg [Fri, 21 Oct 2016 19:27:34 +0000 (15:27 -0400)]
10008: Move test setup from __init__() to setUp()

8 years ago10008: Merge branch 'master' into 10008-flaky-token-test
Tom Clegg [Fri, 21 Oct 2016 19:19:51 +0000 (15:19 -0400)]
10008: Merge branch 'master' into 10008-flaky-token-test

8 years ago10008: Call checkupdate() on ancestors when opening a file.
Tom Clegg [Fri, 21 Oct 2016 19:16:24 +0000 (15:16 -0400)]
10008: Call checkupdate() on ancestors when opening a file.

8 years ago10232: Call getpwnam() and getgrnam() for every name instead of relying on Etc.to_enum().
Tom Clegg [Fri, 21 Oct 2016 15:18:32 +0000 (11:18 -0400)]
10232: Call getpwnam() and getgrnam() for every name instead of relying on Etc.to_enum().

8 years agoAdd requests to ubuntu1204 backport lists, refs #10194
Peter Amstutz [Fri, 21 Oct 2016 15:06:51 +0000 (11:06 -0400)]
Add requests to ubuntu1204 backport lists, refs #10194

8 years agoMerge branch '10194-cwl-version-skew' closes #10194
Peter Amstutz [Fri, 21 Oct 2016 14:15:56 +0000 (10:15 -0400)]
Merge branch '10194-cwl-version-skew' closes #10194

8 years agoMerge branch 'master' into 10194-cwl-version-skew
Peter Amstutz [Fri, 21 Oct 2016 14:15:10 +0000 (10:15 -0400)]
Merge branch 'master' into 10194-cwl-version-skew

Conflicts:
sdk/cwl/tests/test_job.py

8 years ago10194: Use SETUP_DIR for checking git version and writing _version.py. Add
Peter Amstutz [Fri, 21 Oct 2016 13:52:36 +0000 (09:52 -0400)]
10194: Use SETUP_DIR for checking git version and writing _version.py.  Add
comment about calling cwltool.docker.get_image(dockerRequirement, pull_image).

8 years ago10172: Fix container request test. Replace
Peter Amstutz [Fri, 21 Oct 2016 12:47:05 +0000 (08:47 -0400)]
10172: Fix container request test.  Replace
Thread.current[:api_client_authorization] with
current_api_client_authorization.

8 years ago10232: dont create user logic was wrong
Nico Cesar [Fri, 21 Oct 2016 11:38:37 +0000 (07:38 -0400)]
10232: dont create user logic was wrong

refs #10232

8 years ago10232: --dont-create-user flag and sanity check for non-existing users
Nico Cesar [Fri, 21 Oct 2016 11:10:22 +0000 (07:10 -0400)]
10232: --dont-create-user flag and sanity check for non-existing users

refs #10232

8 years agoMerge branch 'crunch-job_finds_newer_docker_hashes' of https://github.com/tmooney...
Tom Clegg [Fri, 21 Oct 2016 05:18:54 +0000 (01:18 -0400)]
Merge branch 'crunch-job_finds_newer_docker_hashes' of https://github.com/tmooney/arvados

No issue #

8 years agoCheck docker version to decide whether to use -f with docker tag, refs #10025
Peter Amstutz [Fri, 21 Oct 2016 02:46:54 +0000 (22:46 -0400)]
Check docker version to decide whether to use -f with docker tag, refs #10025

8 years ago10172: Add test for API: true and container setting output.
Peter Amstutz [Fri, 21 Oct 2016 01:38:28 +0000 (21:38 -0400)]
10172: Add test for API: true and container setting output.

8 years agoMerge branch '10025-arvbox-layers' closes #10025
Peter Amstutz [Fri, 21 Oct 2016 01:25:15 +0000 (21:25 -0400)]
Merge branch '10025-arvbox-layers' closes #10025

8 years ago10315: Bring back arv-put command from 9701's branch.
Lucas Di Pentima [Thu, 20 Oct 2016 22:11:09 +0000 (19:11 -0300)]
10315: Bring back arv-put command from 9701's branch.

8 years ago10172: If RuntimeConstraints.API is set, refresh container record to check if
Peter Amstutz [Thu, 20 Oct 2016 21:06:48 +0000 (17:06 -0400)]
10172: If RuntimeConstraints.API is set, refresh container record to check if
output field has been already set by the container.

8 years ago10172: Bugfix getting container output when container is reused. Tweak output
Peter Amstutz [Thu, 20 Oct 2016 21:05:28 +0000 (17:05 -0400)]
10172: Bugfix getting container output when container is reused.  Tweak output
to report "Job" or "Container" depending on which API is in use.

8 years ago10172: crunch_script only updates task record on error, will normally be
Peter Amstutz [Thu, 20 Oct 2016 20:08:52 +0000 (16:08 -0400)]
10172: crunch_script only updates task record on error, will normally be
updated in set_crunch_output().

8 years agoMerge branch 'master' into 10172-crunch2-container-output
Peter Amstutz [Thu, 20 Oct 2016 19:47:59 +0000 (15:47 -0400)]
Merge branch 'master' into 10172-crunch2-container-output

8 years ago10172: Detect when running as crunch container or job and set output.
Peter Amstutz [Thu, 20 Oct 2016 19:42:28 +0000 (15:42 -0400)]
10172: Detect when running as crunch container or job and set output.

8 years ago10172: Tests and related fixes for auth_uuid setting output on container.
Peter Amstutz [Thu, 20 Oct 2016 19:16:00 +0000 (15:16 -0400)]
10172: Tests and related fixes for auth_uuid setting output on container.

8 years agoMerge branch '10291-really-disable-apis' closes #10291
Tom Clegg [Thu, 20 Oct 2016 15:53:49 +0000 (11:53 -0400)]
Merge branch '10291-really-disable-apis' closes #10291

8 years ago10291: Test multiple disabled APIs.
Tom Clegg [Thu, 20 Oct 2016 13:11:08 +0000 (09:11 -0400)]
10291: Test multiple disabled APIs.

8 years ago10291: Respond 404 to requests for disabled APIs.
Tom Clegg [Thu, 20 Oct 2016 13:06:10 +0000 (09:06 -0400)]
10291: Respond 404 to requests for disabled APIs.

8 years agoMerge branch '10287-sso-username' refs #10287
Tom Clegg [Thu, 20 Oct 2016 10:01:43 +0000 (06:01 -0400)]
Merge branch '10287-sso-username' refs #10287

8 years agoMerge branch '10291-discovery-blacklist' refs #10291
Tom Clegg [Thu, 20 Oct 2016 09:15:49 +0000 (05:15 -0400)]
Merge branch '10291-discovery-blacklist' refs #10291

8 years ago10287: Perform blacklist and duplicate checks on usernames received from SSO.
Tom Clegg [Thu, 20 Oct 2016 08:58:27 +0000 (04:58 -0400)]
10287: Perform blacklist and duplicate checks on usernames received from SSO.

8 years agoSupport Docker v1.10+ hashes that include sha256: prefix.
Thomas Mooney [Wed, 19 Oct 2016 22:59:52 +0000 (17:59 -0500)]
Support Docker v1.10+ hashes that include sha256: prefix.

8 years ago10287: When creating a new account, if provided use 'username' from SSO server.
Peter Amstutz [Wed, 19 Oct 2016 21:14:51 +0000 (17:14 -0400)]
10287: When creating a new account, if provided use 'username' from SSO server.

8 years agoMerge branch '10216-allow-cr-bypass-reuse'
Lucas Di Pentima [Wed, 19 Oct 2016 20:11:04 +0000 (17:11 -0300)]
Merge branch '10216-allow-cr-bypass-reuse'
Closes #10216

8 years ago10216: Merge branch 'master' into 10216-allow-cr-bypass-reuse
Lucas Di Pentima [Wed, 19 Oct 2016 20:10:16 +0000 (17:10 -0300)]
10216: Merge branch 'master' into 10216-allow-cr-bypass-reuse

8 years ago10216: Modified test case, adding it to an already existing test.
Lucas Di Pentima [Wed, 19 Oct 2016 20:08:24 +0000 (17:08 -0300)]
10216: Modified test case, adding it to an already existing test.

8 years ago10216: Added use_existing field to Container Request. When false and CR is committed...
Lucas Di Pentima [Wed, 19 Oct 2016 18:06:44 +0000 (15:06 -0300)]
10216: Added use_existing field to Container Request. When false and CR is committed, a new container will always be created.
Added test case for this new behaviour.

8 years ago10291: Check discovery document before trying to choose/use a work API.
Tom Clegg [Wed, 19 Oct 2016 15:44:12 +0000 (11:44 -0400)]
10291: Check discovery document before trying to choose/use a work API.

8 years ago10291: Fix catching all exceptions in test case.
Tom Clegg [Wed, 19 Oct 2016 15:30:31 +0000 (11:30 -0400)]
10291: Fix catching all exceptions in test case.

8 years ago10291: Add config to disable API methods.
Tom Clegg [Wed, 19 Oct 2016 15:32:44 +0000 (11:32 -0400)]
10291: Add config to disable API methods.

8 years agoMerge branch '10297-container-image-file-sha256' refs #10297
Tom Clegg [Wed, 19 Oct 2016 09:29:26 +0000 (05:29 -0400)]
Merge branch '10297-container-image-file-sha256' refs #10297

8 years ago10297: Accept docker images named sha256:{...}.tar when specified by PDH.
Tom Clegg [Wed, 19 Oct 2016 09:05:09 +0000 (05:05 -0400)]
10297: Accept docker images named sha256:{...}.tar when specified by PDH.

8 years ago10194: Package testing passes for debian7
Peter Amstutz [Tue, 18 Oct 2016 20:39:55 +0000 (16:39 -0400)]
10194: Package testing passes for debian7

8 years ago10194: Add package test for arvados-cwl-runner. Symlink package tests so
Peter Amstutz [Tue, 18 Oct 2016 17:00:44 +0000 (13:00 -0400)]
10194: Add package test for arvados-cwl-runner.  Symlink package tests so
they run for .deb distributions.

8 years ago10194: Update test
Peter Amstutz [Tue, 18 Oct 2016 15:13:19 +0000 (11:13 -0400)]
10194: Update test

8 years agoFix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE...
Tom Clegg [Tue, 18 Oct 2016 14:49:38 +0000 (10:49 -0400)]
Fix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE. No issue #

8 years agoMerge branch '10232-non-exclusive-keys' refs #10232
Tom Clegg [Tue, 18 Oct 2016 14:37:26 +0000 (10:37 -0400)]
Merge branch '10232-non-exclusive-keys' refs #10232

8 years agoAlways log sbatch command, not just on failure. No issue #
Tom Clegg [Tue, 18 Oct 2016 14:30:01 +0000 (10:30 -0400)]
Always log sbatch command, not just on failure. No issue #

8 years ago10232: Increase ascii art.
Tom Clegg [Tue, 18 Oct 2016 13:01:54 +0000 (09:01 -0400)]
10232: Increase ascii art.

8 years ago10232: Own just a section of authorized_keys instead of clobbering the whole thing...
Tom Clegg [Tue, 18 Oct 2016 12:35:13 +0000 (08:35 -0400)]
10232: Own just a section of authorized_keys instead of clobbering the whole thing, unless --xclusive flag given

8 years ago10194: Create job with pinned cwl version.
Peter Amstutz [Mon, 17 Oct 2016 21:06:17 +0000 (17:06 -0400)]
10194: Create job with pinned cwl version.

8 years ago10194: arvados/jobs a-c-r pinning WIP
Peter Amstutz [Mon, 17 Oct 2016 19:02:31 +0000 (15:02 -0400)]
10194: arvados/jobs a-c-r pinning WIP

8 years agoFix API server version pin on arvados-cli. refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:47:32 +0000 (15:47 -0400)]
Fix API server version pin on arvados-cli. refs #10259

8 years agoMerge branch '10259-cwl-pipeline-null' refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:35:26 +0000 (15:35 -0400)]
Merge branch '10259-cwl-pipeline-null' refs #10259

8 years ago10259: Move trim_listing out of upload_instance
Peter Amstutz [Mon, 17 Oct 2016 19:27:25 +0000 (15:27 -0400)]
10259: Move trim_listing out of upload_instance

8 years agoMerge branch 'master' into 10259-cwl-pipeline-null
Peter Amstutz [Mon, 17 Oct 2016 19:07:09 +0000 (15:07 -0400)]
Merge branch 'master' into 10259-cwl-pipeline-null

8 years agoMerge branch '10200-cwl-crunch-script' closes #10200
Peter Amstutz [Mon, 17 Oct 2016 19:05:53 +0000 (15:05 -0400)]
Merge branch '10200-cwl-crunch-script' closes #10200

8 years ago10259: Update arvados-cli version pin
Peter Amstutz [Mon, 17 Oct 2016 18:18:02 +0000 (14:18 -0400)]
10259: Update arvados-cli version pin

8 years ago3486: Accept false and nil/null as parameter values, and avoid translating nil to...
Tom Clegg [Fri, 14 Oct 2016 21:12:51 +0000 (17:12 -0400)]
3486: Accept false and nil/null as parameter values, and avoid translating nil to false.

8 years ago10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
Peter Amstutz [Mon, 17 Oct 2016 14:47:43 +0000 (10:47 -0400)]
10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
false and null as well.