arvados.git
7 years agoMerge branch '8567-moar-docker' refs #8567
Peter Amstutz [Wed, 12 Apr 2017 20:10:13 +0000 (16:10 -0400)]
Merge branch '8567-moar-docker' refs #8567

7 years agoMerge branch '10401-limit-dir-expansion' closes #10401
Peter Amstutz [Wed, 12 Apr 2017 18:45:24 +0000 (14:45 -0400)]
Merge branch '10401-limit-dir-expansion' closes #10401

7 years ago8567: Migrate unnamed images
Peter Amstutz [Wed, 12 Apr 2017 18:38:33 +0000 (18:38 +0000)]
8567: Migrate unnamed images

7 years agoMerge branch '11341-arvput-resume-error'
Lucas Di Pentima [Wed, 12 Apr 2017 18:37:03 +0000 (15:37 -0300)]
Merge branch '11341-arvput-resume-error'
Closes #11341

7 years agoMerge branch '8465-cwl-containers-stdin-stderr' closes #8465
Peter Amstutz [Wed, 12 Apr 2017 18:27:15 +0000 (14:27 -0400)]
Merge branch '8465-cwl-containers-stdin-stderr' closes #8465

7 years agocloses #11365
radhika [Wed, 12 Apr 2017 16:42:16 +0000 (12:42 -0400)]
closes #11365
Merge branch '11365-collection-lock-button'

7 years ago11341: Fixed log message
Lucas Di Pentima [Wed, 12 Apr 2017 16:34:49 +0000 (13:34 -0300)]
11341: Fixed log message

7 years ago11341: Added additional information log for cases when the user ask to not use any...
Lucas Di Pentima [Wed, 12 Apr 2017 16:32:49 +0000 (13:32 -0300)]
11341: Added additional information log for cases when the user ask to not use any cache.

7 years ago10401: Document LoadListingRequirement in Arvados extensions.
Peter Amstutz [Wed, 12 Apr 2017 16:22:25 +0000 (12:22 -0400)]
10401: Document LoadListingRequirement in Arvados extensions.

7 years ago11365: "lock" collection to prevent user from deleting / renaming files until user...
radhika [Fri, 7 Apr 2017 21:08:14 +0000 (17:08 -0400)]
11365: "lock" collection to prevent user from deleting / renaming files until user explicitly unlocks by clicking the button.

7 years ago10401: Add integration tests for directory listing behavior.
Peter Amstutz [Wed, 12 Apr 2017 16:08:52 +0000 (12:08 -0400)]
10401: Add integration tests for directory listing behavior.

7 years ago11341: Use NamedTemporaryFile instead of mkstemp for cache saving.
Lucas Di Pentima [Wed, 12 Apr 2017 16:07:05 +0000 (13:07 -0300)]
11341: Use NamedTemporaryFile instead of mkstemp for cache saving.
Added logging messages to inform the user what's happening with the cache file.

7 years ago11349: Fix section name in example configs.
Tom Clegg [Wed, 12 Apr 2017 15:37:54 +0000 (11:37 -0400)]
11349: Fix section name in example configs.

refs #11349

7 years ago8465: Add unit test for generating container request with std* redirection.
Peter Amstutz [Wed, 12 Apr 2017 14:33:25 +0000 (10:33 -0400)]
8465: Add unit test for generating container request with std* redirection.

7 years agoMerge branch 'master' into 10401-limit-dir-expansion
Peter Amstutz [Wed, 12 Apr 2017 14:12:06 +0000 (10:12 -0400)]
Merge branch 'master' into 10401-limit-dir-expansion

7 years agocwl/test_with_arvbox.sh pulls arvados/jobs:latest refs #11457
Peter Amstutz [Tue, 11 Apr 2017 23:03:52 +0000 (19:03 -0400)]
cwl/test_with_arvbox.sh pulls arvados/jobs:latest refs #11457

7 years agoBuild SSO packages for Ubuntu 1604. refs #11457
Peter Amstutz [Tue, 11 Apr 2017 22:00:25 +0000 (18:00 -0400)]
Build SSO packages for Ubuntu 1604. refs #11457

7 years ago10401: Get structure.sql from master.
Peter Amstutz [Tue, 11 Apr 2017 19:56:56 +0000 (15:56 -0400)]
10401: Get structure.sql from master.

7 years agoMerge branch 'master' into 10401-limit-dir-expansion
Peter Amstutz [Tue, 11 Apr 2017 19:55:30 +0000 (15:55 -0400)]
Merge branch 'master' into 10401-limit-dir-expansion

7 years agoBuild packages for python "future" module.
Tom Clegg [Tue, 11 Apr 2017 18:31:42 +0000 (14:31 -0400)]
Build packages for python "future" module.

refs #11349
refs #11308

7 years agorefs #8465
radhika [Tue, 11 Apr 2017 18:34:46 +0000 (14:34 -0400)]
refs #8465
Merge branch '8465-stderr-redirection'

7 years agoMerge branch 'master' into 8465-stderr-redirection
radhika [Tue, 11 Apr 2017 18:08:21 +0000 (14:08 -0400)]
Merge branch 'master' into 8465-stderr-redirection

7 years ago8465: arvados-cwl-runner support for stdin and stderr redirection with containers.
Peter Amstutz [Tue, 11 Apr 2017 17:40:30 +0000 (13:40 -0400)]
8465: arvados-cwl-runner support for stdin and stderr redirection with containers.

7 years agoMerge branch '11349-nodemanager-status-api'
Tom Clegg [Tue, 11 Apr 2017 15:33:17 +0000 (11:33 -0400)]
Merge branch '11349-nodemanager-status-api'

refs #11349

7 years ago11349: Move assertion outside loop for clarity.
Tom Clegg [Tue, 11 Apr 2017 15:17:05 +0000 (11:17 -0400)]
11349: Move assertion outside loop for clarity.

7 years ago11349: Warn if management/status server is not enabled.
Tom Clegg [Tue, 11 Apr 2017 15:16:50 +0000 (11:16 -0400)]
11349: Warn if management/status server is not enabled.

7 years agoMerge branch '6304-dns-update-fix'
Lucas Di Pentima [Tue, 11 Apr 2017 13:33:24 +0000 (10:33 -0300)]
Merge branch '6304-dns-update-fix'
Closes #6304

7 years ago6304: Change assertion type for better failure message.
Lucas Di Pentima [Tue, 11 Apr 2017 13:32:10 +0000 (10:32 -0300)]
6304: Change assertion type for better failure message.

7 years agoSecure link to google search.
Tom Clegg [Mon, 10 Apr 2017 19:45:40 +0000 (15:45 -0400)]
Secure link to google search.

No issue #

7 years ago6304: Merge branch 'master' into 6304-dns-update-fix
Lucas Di Pentima [Mon, 10 Apr 2017 17:58:08 +0000 (14:58 -0300)]
6304: Merge branch 'master' into 6304-dns-update-fix

7 years ago6304: Corrected test setup to make it pass.
Lucas Di Pentima [Mon, 10 Apr 2017 17:56:04 +0000 (14:56 -0300)]
6304: Corrected test setup to make it pass.
Added a separator to the temp file so that its name is more readable.

7 years agoMerge branch '8465-stdin-redirection' into 8465-stderr-redirection
radhika [Mon, 10 Apr 2017 17:33:11 +0000 (13:33 -0400)]
Merge branch '8465-stdin-redirection' into 8465-stderr-redirection

7 years ago8465: test cleanup
radhika [Mon, 10 Apr 2017 17:32:22 +0000 (13:32 -0400)]
8465: test cleanup

7 years ago6304: Clean up temp files that could exist in case of writing errors.
Lucas Di Pentima [Mon, 10 Apr 2017 16:51:28 +0000 (13:51 -0300)]
6304: Clean up temp files that could exist in case of writing errors.
Also added a test for this case, not working properly atm.

7 years agoMerge branch '11332-cwl-url-quoting' refs #11332
Peter Amstutz [Mon, 10 Apr 2017 14:59:05 +0000 (10:59 -0400)]
Merge branch '11332-cwl-url-quoting' refs #11332

7 years ago11332: Add @ to quote safelist.
Peter Amstutz [Mon, 10 Apr 2017 14:58:33 +0000 (10:58 -0400)]
11332: Add @ to quote safelist.

7 years ago11349: Report wishlist size in status["nodes_wish"].
Tom Clegg [Mon, 10 Apr 2017 14:20:31 +0000 (10:20 -0400)]
11349: Report wishlist size in status["nodes_wish"].

7 years ago11332: Use urllib quote/unquote to handle special chars (e.g. '#')
Peter Amstutz [Mon, 10 Apr 2017 14:07:23 +0000 (10:07 -0400)]
11332: Use urllib quote/unquote to handle special chars (e.g. '#')

7 years ago11349: Add management server with /status.json
Tom Clegg [Fri, 7 Apr 2017 19:20:04 +0000 (15:20 -0400)]
11349: Add management server with /status.json

7 years ago8465: Set proper container flags to provide data on stdin.
Peter Amstutz [Fri, 7 Apr 2017 21:06:01 +0000 (17:06 -0400)]
8465: Set proper container flags to provide data on stdin.

7 years agoMerge branch '11237-google-api-python-client' refs #11237
Peter Amstutz [Fri, 7 Apr 2017 19:26:54 +0000 (15:26 -0400)]
Merge branch '11237-google-api-python-client' refs #11237

7 years ago11237: Tweak version dependencies.
Peter Amstutz [Fri, 7 Apr 2017 18:31:21 +0000 (14:31 -0400)]
11237: Tweak version dependencies.

7 years ago11237: Don't build oauth2client on ubuntu1604
Peter Amstutz [Fri, 7 Apr 2017 18:21:35 +0000 (14:21 -0400)]
11237: Don't build oauth2client on ubuntu1604

7 years ago11237: Bump google-api-python-client to 1.6.2 for better compatability across
Peter Amstutz [Fri, 7 Apr 2017 18:18:57 +0000 (14:18 -0400)]
11237: Bump google-api-python-client to 1.6.2 for better compatability across
oauth2client and Python versions.

7 years agoMerge branch '11209-unmount-subtype'
Tom Clegg [Fri, 7 Apr 2017 17:23:57 +0000 (13:23 -0400)]
Merge branch '11209-unmount-subtype'

refs #11209

7 years agoIncrease ping delay in WatchdogActorTest to try and reduce spurious test failures...
Peter Amstutz [Fri, 7 Apr 2017 15:43:56 +0000 (11:43 -0400)]
Increase ping delay in WatchdogActorTest to try and reduce spurious test failures.  no issue #

7 years agoAdd check that gnutls.h is available so pycurl can compile. no issue #
Peter Amstutz [Fri, 7 Apr 2017 15:36:30 +0000 (11:36 -0400)]
Add check that gnutls.h is available so pycurl can compile.  no issue #

7 years agoEnsure chown does not fail on disappearing files and dirs
Javier BĂ©rtoli [Fri, 7 Apr 2017 11:29:21 +0000 (08:29 -0300)]
Ensure chown does not fail on disappearing files and dirs

refs #11363

7 years ago11209: Restrict --unmount* operations to given --subtype.
Tom Clegg [Fri, 7 Apr 2017 07:41:53 +0000 (03:41 -0400)]
11209: Restrict --unmount* operations to given --subtype.

Add warnings about affecting other fuse mounts with --unmount flags.

7 years ago8465: added stderr redirection and tests
radhika [Fri, 7 Apr 2017 00:12:59 +0000 (20:12 -0400)]
8465: added stderr redirection and tests

7 years agoAdd missing test file refs #11423
Peter Amstutz [Thu, 6 Apr 2017 21:23:10 +0000 (17:23 -0400)]
Add missing test file refs #11423

7 years agoMerge branch '8104-pycurl-721'
Tom Clegg [Thu, 6 Apr 2017 20:47:18 +0000 (16:47 -0400)]
Merge branch '8104-pycurl-721'

closes #8104

7 years ago8104: Do not close sockets after giving them to pycurl. Reverts 7b31dd40b31bdbf7473b5...
Tom Clegg [Thu, 6 Apr 2017 20:43:36 +0000 (16:43 -0400)]
8104: Do not close sockets after giving them to pycurl. Reverts 7b31dd40b31bdbf7473b50fd407cd6f636657c69.

7 years ago8104: Drop unnecessary lambda.
Tom Clegg [Thu, 6 Apr 2017 20:20:56 +0000 (16:20 -0400)]
8104: Drop unnecessary lambda.

7 years ago8465: do not wait in the go routine while doing io.Copy of stdin and also invoke...
radhika [Thu, 6 Apr 2017 20:13:38 +0000 (16:13 -0400)]
8465: do not wait in the go routine while doing io.Copy of stdin and also invoke runner.stop on error.

7 years ago8104: Stop building pycurl package for ubuntu1604.
Tom Clegg [Thu, 6 Apr 2017 19:42:36 +0000 (15:42 -0400)]
8104: Stop building pycurl package for ubuntu1604.

7 years agoMerge branch '11423-cwl-collection-dir-ref' closes #11423
Peter Amstutz [Thu, 6 Apr 2017 19:46:18 +0000 (15:46 -0400)]
Merge branch '11423-cwl-collection-dir-ref' closes #11423

7 years ago11423: Restore check to use pdh_path because it is only supposed to match Files
Peter Amstutz [Thu, 6 Apr 2017 19:43:38 +0000 (15:43 -0400)]
11423: Restore check to use pdh_path because it is only supposed to match Files
references.

7 years agoMerge branch '7824-arvls-arvput-collection-api-usage'
Lucas Di Pentima [Thu, 6 Apr 2017 19:17:43 +0000 (16:17 -0300)]
Merge branch '7824-arvls-arvput-collection-api-usage'
Closes #7824

7 years ago7824: Merge branch 'master' into 7824-arvls-arvput-collection-api-usage
Lucas Di Pentima [Thu, 6 Apr 2017 19:15:25 +0000 (16:15 -0300)]
7824: Merge branch 'master' into 7824-arvls-arvput-collection-api-usage

7 years ago8104: Remove pycurl<7.21.5 dependency.
Tom Clegg [Thu, 6 Apr 2017 18:49:29 +0000 (14:49 -0400)]
8104: Remove pycurl<7.21.5 dependency.

7 years ago8104: OPENSOCKETFUNCTION accepts calls from pycurl 7.21.
Tom Clegg [Thu, 6 Apr 2017 18:38:44 +0000 (14:38 -0400)]
8104: OPENSOCKETFUNCTION accepts calls from pycurl 7.21.

7 years ago11308: Close socket after pycurl.
Tom Clegg [Mon, 3 Apr 2017 05:06:08 +0000 (01:06 -0400)]
11308: Close socket after pycurl.

7 years agoMerge branch 'master' into 8465-stdin-redirection
radhika [Thu, 6 Apr 2017 18:28:57 +0000 (14:28 -0400)]
Merge branch 'master' into 8465-stdin-redirection

7 years ago8465: test clean up
radhika [Thu, 6 Apr 2017 18:28:00 +0000 (14:28 -0400)]
8465: test clean up

7 years ago11423: Fix bug checking existence of Directories which refer to root of keep collections.
Peter Amstutz [Thu, 6 Apr 2017 15:58:38 +0000 (11:58 -0400)]
11423: Fix bug checking existence of Directories which refer to root of keep collections.

7 years agoMerge branch '11413-nodemanager-gce-fixes' refs #11413
Peter Amstutz [Thu, 6 Apr 2017 13:05:41 +0000 (09:05 -0400)]
Merge branch '11413-nodemanager-gce-fixes' refs #11413

7 years ago8465: added stdin redirection for json mount.
radhika [Wed, 5 Apr 2017 23:53:28 +0000 (19:53 -0400)]
8465: added stdin redirection for json mount.

7 years ago6304: Avoid possible dns update race condition by writing the temp file with unique...
Lucas Di Pentima [Wed, 5 Apr 2017 17:23:49 +0000 (14:23 -0300)]
6304: Avoid possible dns update race condition by writing the temp file with unique name to avoid collisions.
Catch IO specific exceptions when trying to write the config files.
Added test to assert that no config file is written when the update command is not set up but the config dir is.

7 years ago7824: Added test cases.
Lucas Di Pentima [Tue, 4 Apr 2017 21:08:39 +0000 (18:08 -0300)]
7824: Added test cases.

7 years ago8465: added stdin redirection for collection
radhika [Tue, 4 Apr 2017 20:48:12 +0000 (16:48 -0400)]
8465: added stdin redirection for collection

7 years ago11413: Use getattr() in exception handler.
Peter Amstutz [Tue, 4 Apr 2017 18:03:08 +0000 (14:03 -0400)]
11413: Use getattr() in exception handler.

7 years ago11413: Wrap destroy_node with similar logic to create_node: on exception check
Peter Amstutz [Tue, 4 Apr 2017 15:40:58 +0000 (11:40 -0400)]
11413: Wrap destroy_node with similar logic to create_node: on exception check
the node list to determine if the node was actually destroyed successfully.

7 years agoMerge branch '11100-cr-output-ttl'
Tom Clegg [Tue, 4 Apr 2017 15:39:30 +0000 (11:39 -0400)]
Merge branch '11100-cr-output-ttl'

refs #11100

7 years ago11413: Fix issues with node manager on GCE:
Peter Amstutz [Tue, 4 Apr 2017 15:21:57 +0000 (11:21 -0400)]
11413: Fix issues with node manager on GCE:

* Always override Node.size with CloudSizeWrapper

* Get updated node record before setting metadata to minimize 'Supplied
  fingerprint does not match current metadata fingerprint.' error.

* Use ex_set_node_metadata() instead of issuing request directly.

7 years ago7824: Now a subdirectory can be asked to be listed. Also, enhanced error message...
Lucas Di Pentima [Mon, 3 Apr 2017 21:16:08 +0000 (18:16 -0300)]
7824: Now a subdirectory can be asked to be listed. Also, enhanced error message when asking for a non existant collection.

7 years ago7824: Several improvements:
Lucas Di Pentima [Mon, 3 Apr 2017 20:45:34 +0000 (17:45 -0300)]
7824: Several improvements:
* Added error reporting on cases where the user ask for downloading a nonexisting file/subdir inside a collection.
* Enhanced error message when the user ask to download a non existing collection.
* Improved the collection traversal when downloading a subcollection.
* Exit with an error message when asking for a subcollection name without the trailing '/'.
* Error out if asking to get a subcollection with '-' as destination.
* Test updated.

7 years agoMerge branch '11407-go-sdk-debug' closes #11407
Peter Amstutz [Mon, 3 Apr 2017 20:21:28 +0000 (16:21 -0400)]
Merge branch '11407-go-sdk-debug' closes #11407

7 years ago11407: keepclient checks for ARVADOS_DEBUG and enables debugging.
Peter Amstutz [Mon, 3 Apr 2017 19:26:50 +0000 (15:26 -0400)]
11407: keepclient checks for ARVADOS_DEBUG and enables debugging.

7 years ago7824: Log errors when trying to read the collection specified.
Lucas Di Pentima [Mon, 3 Apr 2017 18:07:48 +0000 (15:07 -0300)]
7824: Log errors when trying to read the collection specified.

7 years ago7824: Closing file when finished writing
Lucas Di Pentima [Mon, 3 Apr 2017 17:52:36 +0000 (14:52 -0300)]
7824: Closing file when finished writing

7 years ago8567: Add workaround for "trap EXIT" in pid 1 bash process. Add comments.
Peter Amstutz [Mon, 3 Apr 2017 13:52:13 +0000 (09:52 -0400)]
8567: Add workaround for "trap EXIT" in pid 1 bash process.  Add comments.

7 years ago11100: Adjust delete_at in before_validation hook instead of validation. Permit chang...
Tom Clegg [Sat, 1 Apr 2017 02:07:19 +0000 (22:07 -0400)]
11100: Adjust delete_at in before_validation hook instead of validation. Permit changing trash_at without extending delete_at.

7 years ago11100: Specify units in output_ttl docs.
Tom Clegg [Fri, 31 Mar 2017 21:26:43 +0000 (17:26 -0400)]
11100: Specify units in output_ttl docs.

7 years ago11402: Disable caching if $HOME/ is not writable, instead of returning a broken cache.
Tom Clegg [Fri, 31 Mar 2017 20:17:19 +0000 (16:17 -0400)]
11402: Disable caching if $HOME/ is not writable, instead of returning a broken cache.

closes #11402

7 years agocloses #11368
radhika [Fri, 31 Mar 2017 18:04:02 +0000 (14:04 -0400)]
closes #11368
Merge branch '11368-collection-file-rename-warning'

7 years ago11368: show error if user tries to rename a collection file with a duplicate name.
radhika [Fri, 31 Mar 2017 15:39:13 +0000 (11:39 -0400)]
11368: show error if user tries to rename a collection file with a duplicate name.

7 years agoMerge branch '11398-nodes-items-available'
Tom Clegg [Fri, 31 Mar 2017 17:54:51 +0000 (13:54 -0400)]
Merge branch '11398-nodes-items-available'

refs #11398

7 years ago11398: Fix missing items_available in nodes#index response.
Tom Clegg [Fri, 31 Mar 2017 17:52:36 +0000 (13:52 -0400)]
11398: Fix missing items_available in nodes#index response.

7 years ago7824: Use logging facility to show error messages.
Lucas Di Pentima [Fri, 31 Mar 2017 16:40:23 +0000 (13:40 -0300)]
7824: Use logging facility to show error messages.

7 years agoMerge branch '9132-dockerclient' closes #9132
Peter Amstutz [Fri, 31 Mar 2017 14:41:55 +0000 (10:41 -0400)]
Merge branch '9132-dockerclient' closes #9132

7 years ago7824: Updated code to use newer Collection APIs, leaving all_streams() and all_files...
Lucas Di Pentima [Fri, 31 Mar 2017 14:24:50 +0000 (11:24 -0300)]
7824: Updated code to use newer Collection APIs, leaving all_streams() and all_files() behind.

7 years ago9132: Fix test errors due to rebase.
Peter Amstutz [Fri, 31 Mar 2017 14:17:23 +0000 (10:17 -0400)]
9132: Fix test errors due to rebase.

7 years agoFix package test for ubuntu1204.
Ward Vandewege [Fri, 31 Mar 2017 13:57:56 +0000 (09:57 -0400)]
Fix package test for ubuntu1204.

refs #10797

7 years ago11100: If caller sets collection.delete_at too early, set it to the earliest allowed...
Tom Clegg [Thu, 30 Mar 2017 22:47:12 +0000 (18:47 -0400)]
11100: If caller sets collection.delete_at too early, set it to the earliest allowed time instead of failing.

7 years ago9132: Fix ContainerCreate to include HostConfig, container UUID as name.
Peter Amstutz [Thu, 30 Mar 2017 22:02:11 +0000 (18:02 -0400)]
9132: Fix ContainerCreate to include HostConfig, container UUID as name.

7 years ago11100: Add comment about delete_at validation race.
Tom Clegg [Thu, 30 Mar 2017 21:11:05 +0000 (17:11 -0400)]
11100: Add comment about delete_at validation race.

7 years ago11100: Clean up permission checks.
Tom Clegg [Thu, 30 Mar 2017 20:56:59 +0000 (16:56 -0400)]
11100: Clean up permission checks.

7 years ago11100: Add container_requests.output_ttl field. Fix validation of output/log changes...
Tom Clegg [Thu, 30 Mar 2017 16:34:12 +0000 (12:34 -0400)]
11100: Add container_requests.output_ttl field. Fix validation of output/log changes on finalized CRs.

7 years ago8567: Log total bytes to migrate
Peter Amstutz [Thu, 30 Mar 2017 19:40:28 +0000 (19:40 +0000)]
8567: Log total bytes to migrate