arvados.git
8 years agoFix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE...
Tom Clegg [Tue, 18 Oct 2016 14:49:38 +0000 (10:49 -0400)]
Fix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE. No issue #

8 years agoMerge branch '10232-non-exclusive-keys' refs #10232
Tom Clegg [Tue, 18 Oct 2016 14:37:26 +0000 (10:37 -0400)]
Merge branch '10232-non-exclusive-keys' refs #10232

8 years agoAlways log sbatch command, not just on failure. No issue #
Tom Clegg [Tue, 18 Oct 2016 14:30:01 +0000 (10:30 -0400)]
Always log sbatch command, not just on failure. No issue #

8 years ago10232: Increase ascii art.
Tom Clegg [Tue, 18 Oct 2016 13:01:54 +0000 (09:01 -0400)]
10232: Increase ascii art.

8 years ago10232: Own just a section of authorized_keys instead of clobbering the whole thing...
Tom Clegg [Tue, 18 Oct 2016 12:35:13 +0000 (08:35 -0400)]
10232: Own just a section of authorized_keys instead of clobbering the whole thing, unless --xclusive flag given

8 years ago10194: Create job with pinned cwl version.
Peter Amstutz [Mon, 17 Oct 2016 21:06:17 +0000 (17:06 -0400)]
10194: Create job with pinned cwl version.

8 years ago10194: arvados/jobs a-c-r pinning WIP
Peter Amstutz [Mon, 17 Oct 2016 19:02:31 +0000 (15:02 -0400)]
10194: arvados/jobs a-c-r pinning WIP

8 years agoFix API server version pin on arvados-cli. refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:47:32 +0000 (15:47 -0400)]
Fix API server version pin on arvados-cli. refs #10259

8 years agoMerge branch '10259-cwl-pipeline-null' refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:35:26 +0000 (15:35 -0400)]
Merge branch '10259-cwl-pipeline-null' refs #10259

8 years ago10259: Move trim_listing out of upload_instance
Peter Amstutz [Mon, 17 Oct 2016 19:27:25 +0000 (15:27 -0400)]
10259: Move trim_listing out of upload_instance

8 years agoMerge branch 'master' into 10259-cwl-pipeline-null
Peter Amstutz [Mon, 17 Oct 2016 19:07:09 +0000 (15:07 -0400)]
Merge branch 'master' into 10259-cwl-pipeline-null

8 years agoMerge branch '10200-cwl-crunch-script' closes #10200
Peter Amstutz [Mon, 17 Oct 2016 19:05:53 +0000 (15:05 -0400)]
Merge branch '10200-cwl-crunch-script' closes #10200

8 years ago10259: Update arvados-cli version pin
Peter Amstutz [Mon, 17 Oct 2016 18:18:02 +0000 (14:18 -0400)]
10259: Update arvados-cli version pin

8 years ago3486: Accept false and nil/null as parameter values, and avoid translating nil to...
Tom Clegg [Fri, 14 Oct 2016 21:12:51 +0000 (17:12 -0400)]
3486: Accept false and nil/null as parameter values, and avoid translating nil to false.

8 years ago10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
Peter Amstutz [Mon, 17 Oct 2016 14:47:43 +0000 (10:47 -0400)]
10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
false and null as well.

8 years ago10194: Record git version in package
Peter Amstutz [Mon, 17 Oct 2016 17:48:39 +0000 (13:48 -0400)]
10194: Record git version in package

8 years ago10200: Copy cwl-runner crunch script into arvados_cwl package to avoid version skew...
Peter Amstutz [Wed, 12 Oct 2016 16:00:11 +0000 (12:00 -0400)]
10200: Copy cwl-runner crunch script into arvados_cwl package to avoid version skew.  Keep old cwl-runner code as a fallback to avoid breaking installations that haven't been updated.

8 years ago10025: run-build-docker-images.sh can use "arvbox build" instead of "rebuild"
Peter Amstutz [Mon, 17 Oct 2016 15:57:43 +0000 (11:57 -0400)]
10025: run-build-docker-images.sh can use "arvbox build" instead of "rebuild"
now because caching should be better behaved.

8 years ago10025: If $TAG starts with '-' it is not a tag.
Peter Amstutz [Mon, 17 Oct 2016 15:26:44 +0000 (11:26 -0400)]
10025: If $TAG starts with '-' it is not a tag.

8 years ago10025: Add ARG arvados_version to .dev and .demo
Peter Amstutz [Mon, 17 Oct 2016 13:53:37 +0000 (09:53 -0400)]
10025: Add ARG arvados_version to .dev and .demo

8 years ago10025: Use docker tag -f to change "latest".
Peter Amstutz [Mon, 17 Oct 2016 13:48:38 +0000 (09:48 -0400)]
10025: Use docker tag -f to change "latest".

8 years agoRemove obsolete comment. No issue #
Tom Clegg [Mon, 17 Oct 2016 13:24:03 +0000 (09:24 -0400)]
Remove obsolete comment. No issue #

8 years ago9956: Fix startup log message. refs #9956
Tom Clegg [Mon, 17 Oct 2016 12:58:01 +0000 (08:58 -0400)]
9956: Fix startup log message. refs #9956

8 years agoMerge branch '10211-double-close-crash' closes #10211
Tom Clegg [Mon, 17 Oct 2016 12:50:57 +0000 (08:50 -0400)]
Merge branch '10211-double-close-crash' closes #10211

8 years agoMerge branch '10281-directory-replication' closes #10281
Tom Clegg [Mon, 17 Oct 2016 12:39:10 +0000 (08:39 -0400)]
Merge branch '10281-directory-replication' closes #10281

8 years ago10281: Test default replication=1 for directory volumes.
Tom Clegg [Mon, 17 Oct 2016 12:26:26 +0000 (08:26 -0400)]
10281: Test default replication=1 for directory volumes.

8 years ago10281: Update comment.
Tom Clegg [Mon, 17 Oct 2016 12:12:27 +0000 (08:12 -0400)]
10281: Update comment.

8 years ago10281: Add DirectoryReplication config.
Tom Clegg [Mon, 17 Oct 2016 12:06:05 +0000 (08:06 -0400)]
10281: Add DirectoryReplication config.

8 years agoMerge branch '10280-squeue-error' closes #10280
Tom Clegg [Mon, 17 Oct 2016 10:39:41 +0000 (06:39 -0400)]
Merge branch '10280-squeue-error' closes #10280

8 years agoMerge branch '9956-keepstore-config' refs #9956
Tom Clegg [Mon, 17 Oct 2016 10:16:04 +0000 (06:16 -0400)]
Merge branch '9956-keepstore-config' refs #9956

8 years ago10280: Fix ignored error from cmd.Start().
Tom Clegg [Mon, 17 Oct 2016 10:15:29 +0000 (06:15 -0400)]
10280: Fix ignored error from cmd.Start().

8 years ago9956: gofmt
Tom Clegg [Mon, 17 Oct 2016 09:41:02 +0000 (05:41 -0400)]
9956: gofmt

8 years ago10211: Clarify expected result of double-close.
Tom Clegg [Mon, 17 Oct 2016 09:11:25 +0000 (05:11 -0400)]
10211: Clarify expected result of double-close.

8 years ago9956: Fix run_test_server: default is now -enforce-permissions=true.
Tom Clegg [Mon, 17 Oct 2016 08:57:18 +0000 (04:57 -0400)]
9956: Fix run_test_server: default is now -enforce-permissions=true.

8 years agoMerge branch 'master' into 9956-keepstore-config
Tom Clegg [Mon, 17 Oct 2016 08:32:37 +0000 (04:32 -0400)]
Merge branch 'master' into 9956-keepstore-config

Conflicts:
services/keepstore/keepstore.go

8 years ago9956: Show list of supported volume types.
Tom Clegg [Mon, 17 Oct 2016 08:27:55 +0000 (04:27 -0400)]
9956: Show list of supported volume types.

8 years ago9956: Remove obsolete TODO comment.
Tom Clegg [Mon, 17 Oct 2016 08:12:14 +0000 (04:12 -0400)]
9956: Remove obsolete TODO comment.

8 years ago9956: Move deprecated options out to a separate file.
Tom Clegg [Mon, 17 Oct 2016 08:09:41 +0000 (04:09 -0400)]
9956: Move deprecated options out to a separate file.

8 years ago9956: Remove "listen on address with no port" in help message.
Tom Clegg [Mon, 17 Oct 2016 08:08:43 +0000 (04:08 -0400)]
9956: Remove "listen on address with no port" in help message.

8 years agorefs #10227
radhika [Fri, 14 Oct 2016 18:18:46 +0000 (14:18 -0400)]
refs #10227
Merge branch '10227-make-diagnostics-tests-reliable'

8 years ago10211: Return an error instead of crashing if stream is closed twice.
Tom Clegg [Fri, 14 Oct 2016 15:49:21 +0000 (11:49 -0400)]
10211: Return an error instead of crashing if stream is closed twice.

8 years ago10025: Rearrange arvbox layers for better caching. Embed build version to
Peter Amstutz [Fri, 14 Oct 2016 15:29:20 +0000 (11:29 -0400)]
10025: Rearrange arvbox layers for better caching.  Embed build version to
ensure that upper layers rebuild but lower layers do not.

8 years agoMerge branch '10259-cwl-pipeline-value' refs #10259
Peter Amstutz [Fri, 14 Oct 2016 14:57:37 +0000 (10:57 -0400)]
Merge branch '10259-cwl-pipeline-value' refs #10259

8 years ago10272: Fix nil pointer dereference in help message. refs #10272
Tom Clegg [Fri, 14 Oct 2016 14:15:01 +0000 (10:15 -0400)]
10272: Fix nil pointer dereference in help message. refs #10272

8 years ago10172: More work giving auth_uuid limited permission to set progress and output
Peter Amstutz [Fri, 14 Oct 2016 13:39:48 +0000 (09:39 -0400)]
10172: More work giving auth_uuid limited permission to set progress and output
on container.

8 years ago10259: Add one level of indirection on dict parameters so that
Peter Amstutz [Fri, 14 Oct 2016 01:13:58 +0000 (21:13 -0400)]
10259: Add one level of indirection on dict parameters so that
arv-run-pipeline-instance does not try to interpret the value.  Check pipeline
state so that polling loop for cwl-runner job does not get stuck.

8 years ago10172: Allow auth_uuid to set container output. Work in progress.
Peter Amstutz [Thu, 13 Oct 2016 21:22:46 +0000 (17:22 -0400)]
10172: Allow auth_uuid to set container output.  Work in progress.

8 years ago10172: Implement /containers/current endpoint to get current container for API token.
Peter Amstutz [Thu, 13 Oct 2016 21:01:21 +0000 (17:01 -0400)]
10172: Implement /containers/current endpoint to get current container for API token.

8 years agoMerge branch '10234-git-httpd-config' refs #10234 refs #10263
Tom Clegg [Thu, 13 Oct 2016 20:43:55 +0000 (16:43 -0400)]
Merge branch '10234-git-httpd-config' refs #10234 refs #10263

8 years ago10234: Remove doc.go, already superseded by usage.go.
Tom Clegg [Thu, 13 Oct 2016 20:40:05 +0000 (16:40 -0400)]
10234: Remove doc.go, already superseded by usage.go.

8 years agoMerge branch '10221-cwl-pathmapping' closes #10221
Peter Amstutz [Thu, 13 Oct 2016 19:02:32 +0000 (15:02 -0400)]
Merge branch '10221-cwl-pathmapping' closes #10221

8 years agoMerge branch '10259-cwl-pipeline' closes #10259
Peter Amstutz [Thu, 13 Oct 2016 18:48:35 +0000 (14:48 -0400)]
Merge branch '10259-cwl-pipeline' closes #10259

8 years ago10234: Pass through gitolite env vars so old env-var-configured sites continue working.
Tom Clegg [Thu, 13 Oct 2016 18:47:10 +0000 (14:47 -0400)]
10234: Pass through gitolite env vars so old env-var-configured sites continue working.

8 years ago10234: 10263: Update arvados-git-httpd docs to use systemd + config file.
Tom Clegg [Thu, 13 Oct 2016 16:31:02 +0000 (12:31 -0400)]
10234: 10263: Update arvados-git-httpd docs to use systemd + config file.

8 years ago10234: 10263: Use a GitoliteHome configuration entry instead of passing through gitol...
Tom Clegg [Thu, 13 Oct 2016 15:02:18 +0000 (11:02 -0400)]
10234: 10263: Use a GitoliteHome configuration entry instead of passing through gitolite env vars from the caller.

8 years ago10227: The diagnostics tests have been consistently failing since Sep 18th due to...
radhika [Thu, 13 Oct 2016 17:22:51 +0000 (13:22 -0400)]
10227: The diagnostics tests have been consistently failing since Sep 18th due to web sockets based refresh is not happening in the page.
I updated the test to refresh the page in a loop until timeout, rather than rely on the unreliable web sockets.
Also, updated the text_to_look_for to "completed" from "Complete" so that test doesn't misinterpret a job completion as the pipeline completion.

8 years agoMerge branch '10236-disable-event-listening'
Lucas Di Pentima [Thu, 13 Oct 2016 16:59:35 +0000 (13:59 -0300)]
Merge branch '10236-disable-event-listening'
Closes #10236

8 years ago10236: Merge branch 'master' into 10236-disable-event-listening
Lucas Di Pentima [Thu, 13 Oct 2016 16:58:11 +0000 (13:58 -0300)]
10236: Merge branch 'master' into 10236-disable-event-listening

8 years ago10234: Avoid trying to read real config file in test suite.
Tom Clegg [Thu, 13 Oct 2016 05:52:43 +0000 (01:52 -0400)]
10234: Avoid trying to read real config file in test suite.

8 years ago10234: Use YAML for example config file.
Tom Clegg [Thu, 13 Oct 2016 05:52:09 +0000 (01:52 -0400)]
10234: Use YAML for example config file.

8 years ago10234: Clean up program name (arvados-git-httpd) and config paths.
Tom Clegg [Wed, 12 Oct 2016 23:02:48 +0000 (19:02 -0400)]
10234: Clean up program name (arvados-git-httpd) and config paths.

8 years ago10234: Change default repo root from CWD to recommended path.
Tom Clegg [Wed, 12 Oct 2016 23:00:51 +0000 (19:00 -0400)]
10234: Change default repo root from CWD to recommended path.

8 years ago10008: Disable flaky test. refs #10008
Tom Clegg [Thu, 13 Oct 2016 16:35:40 +0000 (12:35 -0400)]
10008: Disable flaky test. refs #10008

8 years agoMerge branch '10243-make-packing-optional'
Lucas Di Pentima [Thu, 13 Oct 2016 15:56:58 +0000 (12:56 -0300)]
Merge branch '10243-make-packing-optional'
Closes #10243

8 years ago10243: Merge branch 'master' into 10243-make-packing-optional
Lucas Di Pentima [Thu, 13 Oct 2016 15:56:11 +0000 (12:56 -0300)]
10243: Merge branch 'master' into 10243-make-packing-optional

8 years ago10236: Added --disable-event-listening to arv-mount command so that it doesn't subscr...
Lucas Di Pentima [Thu, 13 Oct 2016 15:14:55 +0000 (12:14 -0300)]
10236: Added --disable-event-listening to arv-mount command so that it doesn't subscribe to any API Server event.
Also, added a test for this new feature.

8 years agoMerge branch '9848-finalize-on-reuse' closes #9848
Tom Clegg [Thu, 13 Oct 2016 15:14:40 +0000 (11:14 -0400)]
Merge branch '9848-finalize-on-reuse' closes #9848

8 years ago9848: Finalize container request immediately if resolving to an already-finished...
Tom Clegg [Tue, 11 Oct 2016 21:21:06 +0000 (17:21 -0400)]
9848: Finalize container request immediately if resolving to an already-finished container.

8 years ago10259: Workaround for spurious "job_order" on command line. --no-wait returns
Peter Amstutz [Thu, 13 Oct 2016 14:36:30 +0000 (10:36 -0400)]
10259: Workaround for spurious "job_order" on command line.  --no-wait returns
pipeline uuid.

8 years ago10259: When using --submit with jobs API, create RunningOnServer pipeline
Peter Amstutz [Wed, 12 Oct 2016 20:58:25 +0000 (16:58 -0400)]
10259: When using --submit with jobs API, create RunningOnServer pipeline
instance.  This is more similar to the way it works when you run a workflow
from workbench, and ensures that the pipeline will be marked completed even if
the arvados-cwl-runner client goes away.

8 years ago10221: Don't depend on st.keepref from arvados.commands.run.statfile and uploadfile...
Peter Amstutz [Wed, 12 Oct 2016 15:51:03 +0000 (11:51 -0400)]
10221: Don't depend on st.keepref from arvados.commands.run.statfile and uploadfile, use file_pattern to construct predictable keep reference.

8 years ago10221: Add path mapper tests direct keep references, uploaded files, and keep mounted...
Peter Amstutz [Wed, 12 Oct 2016 15:27:37 +0000 (11:27 -0400)]
10221: Add path mapper tests direct keep references, uploaded files, and keep mounted files.

8 years ago10221: Always upload all dependencies up front. Ensures consistent reuse
Peter Amstutz [Wed, 12 Oct 2016 14:59:05 +0000 (10:59 -0400)]
10221: Always upload all dependencies up front.  Ensures consistent reuse
behavior between --local and --submit.  Fix pathmapping bugs where
previously-uploaded files were added to path map but not actually mapped to
target paths.

8 years ago10243: Explicitly calling close(flush=False) on tests so that they work with this...
Lucas Di Pentima [Wed, 12 Oct 2016 21:58:05 +0000 (18:58 -0300)]
10243: Explicitly calling close(flush=False) on tests so that they work with this new feature.

8 years agoMerge branch 'master' into 10243-make-packing-optional
Lucas Di Pentima [Wed, 12 Oct 2016 21:42:17 +0000 (18:42 -0300)]
Merge branch 'master' into 10243-make-packing-optional

8 years agoMerge branch '9463-revert-arv-put-commit'
Lucas Di Pentima [Wed, 12 Oct 2016 21:37:50 +0000 (18:37 -0300)]
Merge branch '9463-revert-arv-put-commit'
Refs #9463 #9701

8 years ago9463: Reverted tests related to old arv-put version.
Lucas Di Pentima [Wed, 12 Oct 2016 21:36:24 +0000 (18:36 -0300)]
9463: Reverted tests related to old arv-put version.

8 years ago10243: Added a flush=True parameter to ArvadosFileWriter.close() so that the repackin...
Lucas Di Pentima [Wed, 12 Oct 2016 21:20:11 +0000 (18:20 -0300)]
10243: Added a flush=True parameter to ArvadosFileWriter.close() so that the repacking feature is disabled by default.

8 years agoMerge branch '9463-revert-arv-put-commit'
Lucas Di Pentima [Wed, 12 Oct 2016 20:59:17 +0000 (17:59 -0300)]
Merge branch '9463-revert-arv-put-commit'
Refs #9463 #9701

8 years ago9463: Commit old arv-put command back, as the new one still have some performance...
Lucas Di Pentima [Wed, 12 Oct 2016 20:56:42 +0000 (17:56 -0300)]
9463: Commit old arv-put command back, as the new one still have some performance issues when dealing with a lot of files.

8 years agoDo not try to chown_R the gid of .ssh, because we're making an
Ward Vandewege [Wed, 12 Oct 2016 15:50:37 +0000 (11:50 -0400)]
Do not try to chown_R the gid of .ssh, because we're making an
assumption about the existance of a group with the same name as the
user, and it's irrelevant anyway because the permissions on .ssh are
0700.

No issue #

8 years agorefs #10144
radhika [Wed, 12 Oct 2016 15:23:37 +0000 (11:23 -0400)]
refs #10144
Merge branch '10144-pipeline-display-perf'

8 years ago10144: call object_for_dataclass only when attr value is not-nil.
radhika [Wed, 12 Oct 2016 00:27:28 +0000 (20:27 -0400)]
10144: call object_for_dataclass only when attr value is not-nil.

8 years ago10144: limit find_object_by_uuid update to the case where find is used; this will...
radhika [Tue, 11 Oct 2016 23:07:38 +0000 (19:07 -0400)]
10144: limit find_object_by_uuid update to the case where find is used; this will ensure that the object is not fetch again when show is used.

8 years ago10144: Conver the link_to_log tests into controller tests so that the #show is executed.
radhika [Tue, 11 Oct 2016 20:56:05 +0000 (16:56 -0400)]
10144: Conver the link_to_log tests into controller tests so that the #show is executed.

8 years ago10144: Preload children during (main) component display. The Job.where in pipeline_in...
radhika [Tue, 11 Oct 2016 18:22:30 +0000 (14:22 -0400)]
10144: Preload children during (main) component display. The Job.where in pipeline_instance_work_unit is fetching all jobs even if they are not readable.

8 years ago10144: Rename :by_name as "by_attr" for preload_objects_for_dataclass
radhika [Tue, 11 Oct 2016 14:46:45 +0000 (10:46 -0400)]
10144: Rename :by_name as "by_attr" for preload_objects_for_dataclass

8 years ago10144: Preloading all the collection using one single large query is much slower...
radhika [Fri, 7 Oct 2016 19:46:30 +0000 (15:46 -0400)]
10144: Preloading all the collection using one single large query is much slower than making several requests
with a smaller subset of  uuids/pdhs when displaying a pipeline instance with many jobs. Hence, make several
requests for 200 collections at a time.

8 years ago10144: preload work_unit children
radhika [Thu, 6 Oct 2016 21:45:22 +0000 (17:45 -0400)]
10144: preload work_unit children

8 years ago10144: preload repositories during work unit component display.
radhika [Thu, 6 Oct 2016 17:30:36 +0000 (13:30 -0400)]
10144: preload repositories during work unit component display.

8 years ago10144: Update link_to_if_arvados_object to use collection_for_pdh when resource_class...
radhika [Thu, 6 Oct 2016 17:12:05 +0000 (13:12 -0400)]
10144: Update link_to_if_arvados_object to use collection_for_pdh when resource_class is a Collection and attrvalue is a pdh.

8 years ago10144: Replace work_unit -> link_to_log with partial so that preloaded objects are...
radhika [Thu, 6 Oct 2016 17:09:17 +0000 (13:09 -0400)]
10144: Replace work_unit -> link_to_log with partial so that preloaded objects are used.

8 years agoMerge branch '10228-arvados-git-httpd-service'
Nico Cesar [Wed, 12 Oct 2016 14:06:34 +0000 (10:06 -0400)]
Merge branch '10228-arvados-git-httpd-service'

closes #10228

8 years agopackage has a .service missing
Nico Cesar [Wed, 12 Oct 2016 10:57:41 +0000 (06:57 -0400)]
package has a .service missing

refs #10228

8 years agoMerge branch '9701-collection-pack-small-files-alt'
Lucas Di Pentima [Tue, 11 Oct 2016 23:18:16 +0000 (20:18 -0300)]
Merge branch '9701-collection-pack-small-files-alt'
Closes #9701 #9463

8 years ago9701: Replaced deprecated use of File.exists?() with File.exist()
Lucas Di Pentima [Tue, 11 Oct 2016 23:13:06 +0000 (20:13 -0300)]
9701: Replaced deprecated use of File.exists?() with File.exist()

8 years ago9701: Fixed arv-put to pass a couple of pending sdk/cli tests. refs #9463
Lucas Di Pentima [Tue, 11 Oct 2016 23:06:37 +0000 (20:06 -0300)]
9701: Fixed arv-put to pass a couple of pending sdk/cli tests. refs #9463

8 years ago9701: Merge branch 'master' into 9701-collection-pack-small-files-alt
Lucas Di Pentima [Tue, 11 Oct 2016 17:54:14 +0000 (14:54 -0300)]
9701: Merge branch 'master' into 9701-collection-pack-small-files-alt

8 years ago9701: Wrapping actual alloc_bufferblock and delete_bufferblock methods inside synchro...
Lucas Di Pentima [Tue, 11 Oct 2016 17:52:22 +0000 (14:52 -0300)]
9701: Wrapping actual alloc_bufferblock and delete_bufferblock methods inside synchronized calls, to be able to use them from within already synchronized BlockManager methods.

8 years agoMerge branch '10186-crunch2-slurm-partition' closes #10186
Peter Amstutz [Tue, 11 Oct 2016 17:30:12 +0000 (13:30 -0400)]
Merge branch '10186-crunch2-slurm-partition' closes #10186