arvados.git
7 years ago7824: Merge branch 'master' into 7824-arvls-arvput-collection-api-usage
Lucas Di Pentima [Thu, 6 Apr 2017 19:15:25 +0000 (16:15 -0300)]
7824: Merge branch 'master' into 7824-arvls-arvput-collection-api-usage

7 years agoMerge branch '11413-nodemanager-gce-fixes' refs #11413
Peter Amstutz [Thu, 6 Apr 2017 13:05:41 +0000 (09:05 -0400)]
Merge branch '11413-nodemanager-gce-fixes' refs #11413

7 years ago7824: Added test cases.
Lucas Di Pentima [Tue, 4 Apr 2017 21:08:39 +0000 (18:08 -0300)]
7824: Added test cases.

7 years ago11413: Use getattr() in exception handler.
Peter Amstutz [Tue, 4 Apr 2017 18:03:08 +0000 (14:03 -0400)]
11413: Use getattr() in exception handler.

7 years ago11413: Wrap destroy_node with similar logic to create_node: on exception check
Peter Amstutz [Tue, 4 Apr 2017 15:40:58 +0000 (11:40 -0400)]
11413: Wrap destroy_node with similar logic to create_node: on exception check
the node list to determine if the node was actually destroyed successfully.

7 years agoMerge branch '11100-cr-output-ttl'
Tom Clegg [Tue, 4 Apr 2017 15:39:30 +0000 (11:39 -0400)]
Merge branch '11100-cr-output-ttl'

refs #11100

7 years ago11413: Fix issues with node manager on GCE:
Peter Amstutz [Tue, 4 Apr 2017 15:21:57 +0000 (11:21 -0400)]
11413: Fix issues with node manager on GCE:

* Always override Node.size with CloudSizeWrapper

* Get updated node record before setting metadata to minimize 'Supplied
  fingerprint does not match current metadata fingerprint.' error.

* Use ex_set_node_metadata() instead of issuing request directly.

7 years ago7824: Now a subdirectory can be asked to be listed. Also, enhanced error message...
Lucas Di Pentima [Mon, 3 Apr 2017 21:16:08 +0000 (18:16 -0300)]
7824: Now a subdirectory can be asked to be listed. Also, enhanced error message when asking for a non existant collection.

7 years ago7824: Several improvements:
Lucas Di Pentima [Mon, 3 Apr 2017 20:45:34 +0000 (17:45 -0300)]
7824: Several improvements:
* Added error reporting on cases where the user ask for downloading a nonexisting file/subdir inside a collection.
* Enhanced error message when the user ask to download a non existing collection.
* Improved the collection traversal when downloading a subcollection.
* Exit with an error message when asking for a subcollection name without the trailing '/'.
* Error out if asking to get a subcollection with '-' as destination.
* Test updated.

7 years agoMerge branch '11407-go-sdk-debug' closes #11407
Peter Amstutz [Mon, 3 Apr 2017 20:21:28 +0000 (16:21 -0400)]
Merge branch '11407-go-sdk-debug' closes #11407

7 years ago11407: keepclient checks for ARVADOS_DEBUG and enables debugging.
Peter Amstutz [Mon, 3 Apr 2017 19:26:50 +0000 (15:26 -0400)]
11407: keepclient checks for ARVADOS_DEBUG and enables debugging.

7 years ago7824: Log errors when trying to read the collection specified.
Lucas Di Pentima [Mon, 3 Apr 2017 18:07:48 +0000 (15:07 -0300)]
7824: Log errors when trying to read the collection specified.

7 years ago7824: Closing file when finished writing
Lucas Di Pentima [Mon, 3 Apr 2017 17:52:36 +0000 (14:52 -0300)]
7824: Closing file when finished writing

7 years ago11100: Adjust delete_at in before_validation hook instead of validation. Permit chang...
Tom Clegg [Sat, 1 Apr 2017 02:07:19 +0000 (22:07 -0400)]
11100: Adjust delete_at in before_validation hook instead of validation. Permit changing trash_at without extending delete_at.

7 years ago11100: Specify units in output_ttl docs.
Tom Clegg [Fri, 31 Mar 2017 21:26:43 +0000 (17:26 -0400)]
11100: Specify units in output_ttl docs.

7 years ago11402: Disable caching if $HOME/ is not writable, instead of returning a broken cache.
Tom Clegg [Fri, 31 Mar 2017 20:17:19 +0000 (16:17 -0400)]
11402: Disable caching if $HOME/ is not writable, instead of returning a broken cache.

closes #11402

7 years agocloses #11368
radhika [Fri, 31 Mar 2017 18:04:02 +0000 (14:04 -0400)]
closes #11368
Merge branch '11368-collection-file-rename-warning'

7 years ago11368: show error if user tries to rename a collection file with a duplicate name.
radhika [Fri, 31 Mar 2017 15:39:13 +0000 (11:39 -0400)]
11368: show error if user tries to rename a collection file with a duplicate name.

7 years agoMerge branch '11398-nodes-items-available'
Tom Clegg [Fri, 31 Mar 2017 17:54:51 +0000 (13:54 -0400)]
Merge branch '11398-nodes-items-available'

refs #11398

7 years ago11398: Fix missing items_available in nodes#index response.
Tom Clegg [Fri, 31 Mar 2017 17:52:36 +0000 (13:52 -0400)]
11398: Fix missing items_available in nodes#index response.

7 years ago7824: Use logging facility to show error messages.
Lucas Di Pentima [Fri, 31 Mar 2017 16:40:23 +0000 (13:40 -0300)]
7824: Use logging facility to show error messages.

7 years agoMerge branch '9132-dockerclient' closes #9132
Peter Amstutz [Fri, 31 Mar 2017 14:41:55 +0000 (10:41 -0400)]
Merge branch '9132-dockerclient' closes #9132

7 years ago7824: Updated code to use newer Collection APIs, leaving all_streams() and all_files...
Lucas Di Pentima [Fri, 31 Mar 2017 14:24:50 +0000 (11:24 -0300)]
7824: Updated code to use newer Collection APIs, leaving all_streams() and all_files() behind.

7 years ago9132: Fix test errors due to rebase.
Peter Amstutz [Fri, 31 Mar 2017 14:17:23 +0000 (10:17 -0400)]
9132: Fix test errors due to rebase.

7 years agoFix package test for ubuntu1204.
Ward Vandewege [Fri, 31 Mar 2017 13:57:56 +0000 (09:57 -0400)]
Fix package test for ubuntu1204.

refs #10797

7 years ago11100: If caller sets collection.delete_at too early, set it to the earliest allowed...
Tom Clegg [Thu, 30 Mar 2017 22:47:12 +0000 (18:47 -0400)]
11100: If caller sets collection.delete_at too early, set it to the earliest allowed time instead of failing.

7 years ago9132: Fix ContainerCreate to include HostConfig, container UUID as name.
Peter Amstutz [Thu, 30 Mar 2017 22:02:11 +0000 (18:02 -0400)]
9132: Fix ContainerCreate to include HostConfig, container UUID as name.

7 years ago11100: Add comment about delete_at validation race.
Tom Clegg [Thu, 30 Mar 2017 21:11:05 +0000 (17:11 -0400)]
11100: Add comment about delete_at validation race.

7 years ago11100: Clean up permission checks.
Tom Clegg [Thu, 30 Mar 2017 20:56:59 +0000 (16:56 -0400)]
11100: Clean up permission checks.

7 years ago11100: Add container_requests.output_ttl field. Fix validation of output/log changes...
Tom Clegg [Thu, 30 Mar 2017 16:34:12 +0000 (12:34 -0400)]
11100: Add container_requests.output_ttl field. Fix validation of output/log changes on finalized CRs.

7 years ago7824: Moved arv-get code to arvados.commands and replaced bin/arv-get with a stub...
Lucas Di Pentima [Thu, 30 Mar 2017 18:40:41 +0000 (15:40 -0300)]
7824: Moved arv-get code to arvados.commands and replaced bin/arv-get with a stub caller.

7 years ago9132: Fix after rebase
Peter Amstutz [Thu, 30 Mar 2017 18:19:42 +0000 (14:19 -0400)]
9132: Fix after rebase

7 years ago9132: Use ThinDockerClientProxy to overcome the import issue around "github.com/docke...
radhika [Wed, 22 Mar 2017 20:47:44 +0000 (16:47 -0400)]
9132: Use ThinDockerClientProxy to overcome the import issue around "github.com/docker/docker/vendor/golang.org/x/net/context".
Perform most of the test updates needed to use the new API.

7 years ago9132: few more updates to use the new API
Peter Amstutz [Thu, 30 Mar 2017 18:09:28 +0000 (14:09 -0400)]
9132: few more updates to use the new API

7 years ago9132: WIP switching to using official Docker Go client.
Peter Amstutz [Thu, 30 Mar 2017 18:07:01 +0000 (14:07 -0400)]
9132: WIP switching to using official Docker Go client.

7 years agoMerge branch '11332-fix-crunchscript' refs #11332
Peter Amstutz [Thu, 30 Mar 2017 18:32:06 +0000 (14:32 -0400)]
Merge branch '11332-fix-crunchscript' refs #11332

7 years ago7824: Further ordering fixes. Now all arv-ls tests passes.
Lucas Di Pentima [Thu, 30 Mar 2017 18:15:20 +0000 (15:15 -0300)]
7824: Further ordering fixes. Now all arv-ls tests passes.

7 years agoMerge branch '7709-api-rails4' (partial)
Tom Clegg [Thu, 30 Mar 2017 17:41:22 +0000 (13:41 -0400)]
Merge branch '7709-api-rails4' (partial)

refs #7709
refs #11100

7 years ago7824: Fix upper/lower case ordering on arv-ls.
Lucas Di Pentima [Thu, 30 Mar 2017 15:06:46 +0000 (12:06 -0300)]
7824: Fix upper/lower case ordering on arv-ls.

7 years ago7709: De-duplicate "ensure unique name" implementations.
Tom Clegg [Tue, 21 Mar 2017 15:59:40 +0000 (11:59 -0400)]
7709: De-duplicate "ensure unique name" implementations.

7 years ago7709: Do not set job_readable when it won't be used.
Tom Clegg [Tue, 21 Mar 2017 16:49:06 +0000 (12:49 -0400)]
7709: Do not set job_readable when it won't be used.

7 years ago7709: Leave container_request alone when applying default keep_cache_ram value.
Tom Clegg [Tue, 21 Mar 2017 15:56:17 +0000 (11:56 -0400)]
7709: Leave container_request alone when applying default keep_cache_ram value.

7 years ago7709: Fix tests (count existing links before adding new ones).
Tom Clegg [Tue, 21 Mar 2017 16:01:33 +0000 (12:01 -0400)]
7709: Fix tests (count existing links before adding new ones).

7 years ago11209: Restore missing import.
Tom Clegg [Thu, 30 Mar 2017 14:29:47 +0000 (10:29 -0400)]
11209: Restore missing import.

refs #11209

7 years ago11332-fix-crunchscript: Set ACCEPTLIST_RE because crunch_script alternate entry
Peter Amstutz [Thu, 30 Mar 2017 14:09:33 +0000 (10:09 -0400)]
11332-fix-crunchscript: Set ACCEPTLIST_RE because crunch_script alternate entry
point doesn't go through main.

7 years ago11355: Arvados gems accept json 2.x.
Tom Clegg [Thu, 30 Mar 2017 01:23:24 +0000 (21:23 -0400)]
11355: Arvados gems accept json 2.x.

refs #11355

7 years agoMerge branch '11235-log-retryable'
Tom Clegg [Wed, 29 Mar 2017 15:51:55 +0000 (11:51 -0400)]
Merge branch '11235-log-retryable'

refs #11235

7 years ago11235: Log a message when a job is interrupted by node failure.
Tom Clegg [Wed, 29 Mar 2017 15:49:49 +0000 (11:49 -0400)]
11235: Log a message when a job is interrupted by node failure.

...and say what's going to happen as a result, even if that is not
"giving up" yet.

7 years agocloses #3821
radhika [Wed, 29 Mar 2017 15:44:42 +0000 (11:44 -0400)]
closes #3821
Merge branch '3821-collection-file-manage'

7 years agoFix invalid records in API collections test fixture and improve error message
Peter Amstutz [Wed, 29 Mar 2017 15:43:20 +0000 (11:43 -0400)]
Fix invalid records in API collections test fixture and improve error message
to include both values.  Incidentally revealed by adding PDH index to
collections table.  refs #11361

7 years ago3821: improve the tooltip on rename file option.
radhika [Wed, 29 Mar 2017 15:28:28 +0000 (11:28 -0400)]
3821: improve the tooltip on rename file option.

7 years agoFix packaging for python-arvados-cwl-runner refs #11333
Peter Amstutz [Wed, 29 Mar 2017 14:54:20 +0000 (10:54 -0400)]
Fix packaging for python-arvados-cwl-runner refs #11333

7 years agoMerge branch '11361-pdh-index'
Tom Clegg [Wed, 29 Mar 2017 04:26:24 +0000 (00:26 -0400)]
Merge branch '11361-pdh-index'

refs #11361

7 years ago11361: Add pdh index to collections table.
Tom Clegg [Tue, 28 Mar 2017 22:01:25 +0000 (18:01 -0400)]
11361: Add pdh index to collections table.

7 years agoMerge branch '11209-unmount-replace'
Tom Clegg [Tue, 28 Mar 2017 20:13:18 +0000 (16:13 -0400)]
Merge branch '11209-unmount-replace'

refs #11209

7 years agoMerge branch '11333-cwl-phone-home' closes #11333
Peter Amstutz [Tue, 28 Mar 2017 20:01:50 +0000 (16:01 -0400)]
Merge branch '11333-cwl-phone-home' closes #11333

7 years ago3821: update test assertion that randomly fails; in fact, I don't see how it ever...
radhika [Tue, 28 Mar 2017 20:01:06 +0000 (16:01 -0400)]
3821: update test assertion that randomly fails; in fact, I don't see how it ever worked with that syntax.

7 years ago11333: Add explicit version pin on 'typing'
Peter Amstutz [Tue, 28 Mar 2017 16:57:25 +0000 (12:57 -0400)]
11333: Add explicit version pin on 'typing'

7 years ago11332: Suppress phone-home behavior in check_exists by recognizing
Peter Amstutz [Tue, 28 Mar 2017 16:44:20 +0000 (12:44 -0400)]
11332: Suppress phone-home behavior in check_exists by recognizing
http://arvados.org/cwl and returning True.

7 years agoMerge branch '11332-cwl-filenames' closes #11332
Peter Amstutz [Tue, 28 Mar 2017 19:50:02 +0000 (15:50 -0400)]
Merge branch '11332-cwl-filenames' closes #11332

7 years agoMerge branch 'master' into 3821-collection-file-manage
radhika [Tue, 28 Mar 2017 19:26:39 +0000 (15:26 -0400)]
Merge branch 'master' into 3821-collection-file-manage

7 years ago3821: collection viewed from another object
radhika [Tue, 28 Mar 2017 18:07:42 +0000 (14:07 -0400)]
3821: collection viewed from another object

7 years ago3821: The "Remove selected files" option should only be presented if the user can...
radhika [Tue, 28 Mar 2017 16:54:56 +0000 (12:54 -0400)]
3821: The "Remove selected files" option should only be presented if the user can update the collection.

7 years ago11332: Override relaxed path checks to be even more relaxed, and bump cwltool &
Peter Amstutz [Mon, 27 Mar 2017 21:29:58 +0000 (17:29 -0400)]
11332: Override relaxed path checks to be even more relaxed, and bump cwltool &
schema salad pin for hashes-in-filenames fix.

7 years ago3821: attr_accessor not needed on collection "filename"
radhika [Tue, 28 Mar 2017 15:33:19 +0000 (11:33 -0400)]
3821: attr_accessor not needed on collection "filename"

7 years ago3821: add an integration test to rename a file in a collection
radhika [Tue, 28 Mar 2017 15:25:35 +0000 (11:25 -0400)]
3821: add an integration test to rename a file in a collection

7 years ago11209: Test using ./bin/arv-mount from source dir.
Tom Clegg [Tue, 28 Mar 2017 05:52:56 +0000 (01:52 -0400)]
11209: Test using ./bin/arv-mount from source dir.

7 years ago3821: integration test to remove files from a collection
radhika [Mon, 27 Mar 2017 23:37:45 +0000 (19:37 -0400)]
3821: integration test to remove files from a collection

7 years ago11209: Replace "--unmount /path/..." with "--unmount-all /path".
Tom Clegg [Mon, 27 Mar 2017 22:43:56 +0000 (18:43 -0400)]
11209: Replace "--unmount /path/..." with "--unmount-all /path".

7 years ago11209: Fix daemon mode.
Tom Clegg [Mon, 27 Mar 2017 22:36:48 +0000 (18:36 -0400)]
11209: Fix daemon mode.

DaemonContext cannot happen after listen_for_events().

7 years ago3821: collection controller tests for remove and rename files.
radhika [Mon, 27 Mar 2017 22:35:56 +0000 (18:35 -0400)]
3821: collection controller tests for remove and rename files.

7 years agoUpdate cwl test_submit refs #10401
Peter Amstutz [Mon, 27 Mar 2017 20:22:46 +0000 (16:22 -0400)]
Update cwl test_submit refs #10401

7 years agoMerge branch '10401-run-upload-dir' refs #10401
Peter Amstutz [Mon, 27 Mar 2017 19:54:41 +0000 (15:54 -0400)]
Merge branch '10401-run-upload-dir' refs #10401

7 years ago10401: Support in arv-run for uploading Directories.
Peter Amstutz [Fri, 24 Mar 2017 18:55:02 +0000 (14:55 -0400)]
10401: Support in arv-run for uploading Directories.

7 years agoPin ws4py <0.4 (our websocket tests fail on 0.4.1)
Tom Clegg [Mon, 27 Mar 2017 16:26:32 +0000 (12:26 -0400)]
Pin ws4py <0.4 (our websocket tests fail on 0.4.1)

No issue #

7 years ago3821: Rename collection file path
radhika [Fri, 24 Mar 2017 20:58:20 +0000 (16:58 -0400)]
3821: Rename collection file path

7 years ago11209: Remove unused imports.
Tom Clegg [Fri, 24 Mar 2017 19:59:56 +0000 (15:59 -0400)]
11209: Remove unused imports.

7 years ago11209: Add tests for --unmount and --replace flags.
Tom Clegg [Fri, 24 Mar 2017 19:59:26 +0000 (15:59 -0400)]
11209: Add tests for --unmount and --replace flags.

7 years ago11209: Fix missing import. Use context handler for standalone mode.
Tom Clegg [Fri, 24 Mar 2017 19:55:24 +0000 (15:55 -0400)]
11209: Fix missing import. Use context handler for standalone mode.

7 years agoRemove debug printf.
Tom Clegg [Fri, 24 Mar 2017 18:42:30 +0000 (14:42 -0400)]
Remove debug printf.

No issue #

7 years agoRemove -i flag from Rails3 pg_dump invocation.
Tom Clegg [Fri, 24 Mar 2017 18:37:22 +0000 (14:37 -0400)]
Remove -i flag from Rails3 pg_dump invocation.

No issue #

7 years ago3821: remove selection file in collection file row
radhika [Fri, 24 Mar 2017 15:37:00 +0000 (11:37 -0400)]
3821: remove selection file in collection file row

7 years ago3821: added 'Remove selected files' to collection#show page.
radhika [Fri, 24 Mar 2017 14:26:56 +0000 (10:26 -0400)]
3821: added 'Remove selected files' to collection#show page.

7 years agoMerge branch '8567-docker-migrator' refs #8567
Peter Amstutz [Fri, 24 Mar 2017 13:31:36 +0000 (09:31 -0400)]
Merge branch '8567-docker-migrator' refs #8567

7 years ago11209: Try unmounting at least once even if timeout=0.
Tom Clegg [Thu, 23 Mar 2017 22:06:01 +0000 (18:06 -0400)]
11209: Try unmounting at least once even if timeout=0.

7 years ago11209: "--unmount /path/..." unmounts /path and all fuse mounts below it.
Tom Clegg [Fri, 17 Mar 2017 20:32:10 +0000 (16:32 -0400)]
11209: "--unmount /path/..." unmounts /path and all fuse mounts below it.

7 years ago11209: Add arv-mount --unmount and --replace flags
Tom Clegg [Thu, 16 Mar 2017 20:49:49 +0000 (16:49 -0400)]
11209: Add arv-mount --unmount and --replace flags

7 years ago7824: Updated arv-ls to use CollectionReader's keys() API instead of the old all_file...
Lucas Di Pentima [Thu, 23 Mar 2017 21:38:41 +0000 (18:38 -0300)]
7824: Updated arv-ls to use CollectionReader's keys() API instead of the old all_files() / all_streams() methods.

7 years agoMerge branch '11325-no-broken-nodes' refs #11323, refs #11324, refs #11325
Peter Amstutz [Thu, 23 Mar 2017 20:16:33 +0000 (16:16 -0400)]
Merge branch '11325-no-broken-nodes' refs #11323, refs #11324, refs #11325

7 years ago11323: Don't try to offer_arvados_pair on unpaired nodes which are being shut down.
Peter Amstutz [Thu, 23 Mar 2017 17:30:35 +0000 (13:30 -0400)]
11323: Don't try to offer_arvados_pair on unpaired nodes which are being shut down.

7 years ago11324: Fix crash in NodeManagerDaemonActor when receiving a node_can_shutdown
Peter Amstutz [Thu, 23 Mar 2017 18:05:51 +0000 (14:05 -0400)]
11324: Fix crash in NodeManagerDaemonActor when receiving a node_can_shutdown
message for a node that has already been shut down.

7 years agoMerge branch '10766-install-ws'
Tom Clegg [Thu, 23 Mar 2017 20:10:48 +0000 (16:10 -0400)]
Merge branch '10766-install-ws'

closes #10766

7 years agoMerge branch '11065-rotate-logs'
Tom Clegg [Thu, 23 Mar 2017 19:56:44 +0000 (15:56 -0400)]
Merge branch '11065-rotate-logs'

closes #11065

7 years ago11065: Avoid scientific notation in PostgreSQL literal.
Tom Clegg [Thu, 23 Mar 2017 19:56:16 +0000 (15:56 -0400)]
11065: Avoid scientific notation in PostgreSQL literal.

7 years ago11065: Update comments.
Tom Clegg [Thu, 23 Mar 2017 19:18:51 +0000 (15:18 -0400)]
11065: Update comments.

7 years agoMerge branch '11221-always-restart-services'
Tom Clegg [Thu, 23 Mar 2017 18:51:21 +0000 (14:51 -0400)]
Merge branch '11221-always-restart-services'

closes #11221

7 years ago11325: Remove "broken node" check. Assume if the node really isn't
Peter Amstutz [Thu, 23 Mar 2017 17:23:54 +0000 (13:23 -0400)]
11325: Remove "broken node" check.  Assume if the node really isn't
functioning, it should be "down" in SLURM anyway.  Remove test_broken_node_not_counted because broken node check is removed.

7 years ago10766: Fix copy-paste error in godoc link.
Tom Clegg [Thu, 23 Mar 2017 18:05:06 +0000 (14:05 -0400)]
10766: Fix copy-paste error in godoc link.

7 years ago10766: Migrate install guide from puma to arvados-ws.
Tom Clegg [Thu, 23 Mar 2017 17:46:37 +0000 (13:46 -0400)]
10766: Migrate install guide from puma to arvados-ws.

7 years agoMerge branch '10669-safe-http-cache'
Tom Clegg [Thu, 23 Mar 2017 14:56:34 +0000 (10:56 -0400)]
Merge branch '10669-safe-http-cache'

closes #10669