arvados.git
7 years agoMerge branch '10194-jobs-image-versioning' refs #10194
Peter Amstutz [Tue, 25 Oct 2016 01:33:40 +0000 (21:33 -0400)]
Merge branch '10194-jobs-image-versioning' refs #10194

7 years ago10194: Improve versioning of arvados/jobs image.
Peter Amstutz [Fri, 21 Oct 2016 20:35:09 +0000 (16:35 -0400)]
10194: Improve versioning of arvados/jobs image.

arvados-cwl-runner sets version to max(cwl-runner, python-sdk).

build/run-build-docker-jobs-image.sh determines package versions corresponding
to current commit and supplies explicit package versions as Docker build arg.

7 years agoMerge branch '10232-non-exclusive-keys' refs #10232
Tom Clegg [Fri, 21 Oct 2016 20:08:45 +0000 (16:08 -0400)]
Merge branch '10232-non-exclusive-keys' refs #10232

7 years ago10232: Call getpwnam() and getgrnam() for every name instead of relying on Etc.to_enum().
Tom Clegg [Fri, 21 Oct 2016 15:18:32 +0000 (11:18 -0400)]
10232: Call getpwnam() and getgrnam() for every name instead of relying on Etc.to_enum().

7 years agoAdd requests to ubuntu1204 backport lists, refs #10194
Peter Amstutz [Fri, 21 Oct 2016 15:06:51 +0000 (11:06 -0400)]
Add requests to ubuntu1204 backport lists, refs #10194

7 years agoMerge branch '10194-cwl-version-skew' closes #10194
Peter Amstutz [Fri, 21 Oct 2016 14:15:56 +0000 (10:15 -0400)]
Merge branch '10194-cwl-version-skew' closes #10194

7 years agoMerge branch 'master' into 10194-cwl-version-skew
Peter Amstutz [Fri, 21 Oct 2016 14:15:10 +0000 (10:15 -0400)]
Merge branch 'master' into 10194-cwl-version-skew

Conflicts:
sdk/cwl/tests/test_job.py

7 years ago10194: Use SETUP_DIR for checking git version and writing _version.py. Add
Peter Amstutz [Fri, 21 Oct 2016 13:52:36 +0000 (09:52 -0400)]
10194: Use SETUP_DIR for checking git version and writing _version.py.  Add
comment about calling cwltool.docker.get_image(dockerRequirement, pull_image).

7 years ago10232: dont create user logic was wrong
Nico Cesar [Fri, 21 Oct 2016 11:38:37 +0000 (07:38 -0400)]
10232: dont create user logic was wrong

refs #10232

7 years ago10232: --dont-create-user flag and sanity check for non-existing users
Nico Cesar [Fri, 21 Oct 2016 11:10:22 +0000 (07:10 -0400)]
10232: --dont-create-user flag and sanity check for non-existing users

refs #10232

7 years agoMerge branch 'crunch-job_finds_newer_docker_hashes' of https://github.com/tmooney...
Tom Clegg [Fri, 21 Oct 2016 05:18:54 +0000 (01:18 -0400)]
Merge branch 'crunch-job_finds_newer_docker_hashes' of https://github.com/tmooney/arvados

No issue #

7 years agoCheck docker version to decide whether to use -f with docker tag, refs #10025
Peter Amstutz [Fri, 21 Oct 2016 02:46:54 +0000 (22:46 -0400)]
Check docker version to decide whether to use -f with docker tag, refs #10025

7 years agoMerge branch '10025-arvbox-layers' closes #10025
Peter Amstutz [Fri, 21 Oct 2016 01:25:15 +0000 (21:25 -0400)]
Merge branch '10025-arvbox-layers' closes #10025

7 years agoMerge branch '10291-really-disable-apis' closes #10291
Tom Clegg [Thu, 20 Oct 2016 15:53:49 +0000 (11:53 -0400)]
Merge branch '10291-really-disable-apis' closes #10291

7 years ago10291: Test multiple disabled APIs.
Tom Clegg [Thu, 20 Oct 2016 13:11:08 +0000 (09:11 -0400)]
10291: Test multiple disabled APIs.

7 years ago10291: Respond 404 to requests for disabled APIs.
Tom Clegg [Thu, 20 Oct 2016 13:06:10 +0000 (09:06 -0400)]
10291: Respond 404 to requests for disabled APIs.

7 years agoMerge branch '10287-sso-username' refs #10287
Tom Clegg [Thu, 20 Oct 2016 10:01:43 +0000 (06:01 -0400)]
Merge branch '10287-sso-username' refs #10287

7 years agoMerge branch '10291-discovery-blacklist' refs #10291
Tom Clegg [Thu, 20 Oct 2016 09:15:49 +0000 (05:15 -0400)]
Merge branch '10291-discovery-blacklist' refs #10291

7 years ago10287: Perform blacklist and duplicate checks on usernames received from SSO.
Tom Clegg [Thu, 20 Oct 2016 08:58:27 +0000 (04:58 -0400)]
10287: Perform blacklist and duplicate checks on usernames received from SSO.

7 years agoSupport Docker v1.10+ hashes that include sha256: prefix.
Thomas Mooney [Wed, 19 Oct 2016 22:59:52 +0000 (17:59 -0500)]
Support Docker v1.10+ hashes that include sha256: prefix.

7 years ago10287: When creating a new account, if provided use 'username' from SSO server.
Peter Amstutz [Wed, 19 Oct 2016 21:14:51 +0000 (17:14 -0400)]
10287: When creating a new account, if provided use 'username' from SSO server.

7 years agoMerge branch '10216-allow-cr-bypass-reuse'
Lucas Di Pentima [Wed, 19 Oct 2016 20:11:04 +0000 (17:11 -0300)]
Merge branch '10216-allow-cr-bypass-reuse'
Closes #10216

7 years ago10216: Merge branch 'master' into 10216-allow-cr-bypass-reuse
Lucas Di Pentima [Wed, 19 Oct 2016 20:10:16 +0000 (17:10 -0300)]
10216: Merge branch 'master' into 10216-allow-cr-bypass-reuse

7 years ago10216: Modified test case, adding it to an already existing test.
Lucas Di Pentima [Wed, 19 Oct 2016 20:08:24 +0000 (17:08 -0300)]
10216: Modified test case, adding it to an already existing test.

7 years ago10216: Added use_existing field to Container Request. When false and CR is committed...
Lucas Di Pentima [Wed, 19 Oct 2016 18:06:44 +0000 (15:06 -0300)]
10216: Added use_existing field to Container Request. When false and CR is committed, a new container will always be created.
Added test case for this new behaviour.

7 years ago10291: Check discovery document before trying to choose/use a work API.
Tom Clegg [Wed, 19 Oct 2016 15:44:12 +0000 (11:44 -0400)]
10291: Check discovery document before trying to choose/use a work API.

7 years ago10291: Fix catching all exceptions in test case.
Tom Clegg [Wed, 19 Oct 2016 15:30:31 +0000 (11:30 -0400)]
10291: Fix catching all exceptions in test case.

7 years ago10291: Add config to disable API methods.
Tom Clegg [Wed, 19 Oct 2016 15:32:44 +0000 (11:32 -0400)]
10291: Add config to disable API methods.

7 years agoMerge branch '10297-container-image-file-sha256' refs #10297
Tom Clegg [Wed, 19 Oct 2016 09:29:26 +0000 (05:29 -0400)]
Merge branch '10297-container-image-file-sha256' refs #10297

7 years ago10297: Accept docker images named sha256:{...}.tar when specified by PDH.
Tom Clegg [Wed, 19 Oct 2016 09:05:09 +0000 (05:05 -0400)]
10297: Accept docker images named sha256:{...}.tar when specified by PDH.

7 years ago10194: Package testing passes for debian7
Peter Amstutz [Tue, 18 Oct 2016 20:39:55 +0000 (16:39 -0400)]
10194: Package testing passes for debian7

7 years ago10194: Add package test for arvados-cwl-runner. Symlink package tests so
Peter Amstutz [Tue, 18 Oct 2016 17:00:44 +0000 (13:00 -0400)]
10194: Add package test for arvados-cwl-runner.  Symlink package tests so
they run for .deb distributions.

7 years ago10194: Update test
Peter Amstutz [Tue, 18 Oct 2016 15:13:19 +0000 (11:13 -0400)]
10194: Update test

7 years agoFix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE...
Tom Clegg [Tue, 18 Oct 2016 14:49:38 +0000 (10:49 -0400)]
Fix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE. No issue #

7 years agoMerge branch '10232-non-exclusive-keys' refs #10232
Tom Clegg [Tue, 18 Oct 2016 14:37:26 +0000 (10:37 -0400)]
Merge branch '10232-non-exclusive-keys' refs #10232

7 years agoAlways log sbatch command, not just on failure. No issue #
Tom Clegg [Tue, 18 Oct 2016 14:30:01 +0000 (10:30 -0400)]
Always log sbatch command, not just on failure. No issue #

7 years ago10232: Increase ascii art.
Tom Clegg [Tue, 18 Oct 2016 13:01:54 +0000 (09:01 -0400)]
10232: Increase ascii art.

7 years ago10232: Own just a section of authorized_keys instead of clobbering the whole thing...
Tom Clegg [Tue, 18 Oct 2016 12:35:13 +0000 (08:35 -0400)]
10232: Own just a section of authorized_keys instead of clobbering the whole thing, unless --xclusive flag given

7 years ago10194: Create job with pinned cwl version.
Peter Amstutz [Mon, 17 Oct 2016 21:06:17 +0000 (17:06 -0400)]
10194: Create job with pinned cwl version.

7 years ago10194: arvados/jobs a-c-r pinning WIP
Peter Amstutz [Mon, 17 Oct 2016 19:02:31 +0000 (15:02 -0400)]
10194: arvados/jobs a-c-r pinning WIP

7 years agoFix API server version pin on arvados-cli. refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:47:32 +0000 (15:47 -0400)]
Fix API server version pin on arvados-cli. refs #10259

7 years agoMerge branch '10259-cwl-pipeline-null' refs #10259
Peter Amstutz [Mon, 17 Oct 2016 19:35:26 +0000 (15:35 -0400)]
Merge branch '10259-cwl-pipeline-null' refs #10259

7 years ago10259: Move trim_listing out of upload_instance
Peter Amstutz [Mon, 17 Oct 2016 19:27:25 +0000 (15:27 -0400)]
10259: Move trim_listing out of upload_instance

7 years agoMerge branch 'master' into 10259-cwl-pipeline-null
Peter Amstutz [Mon, 17 Oct 2016 19:07:09 +0000 (15:07 -0400)]
Merge branch 'master' into 10259-cwl-pipeline-null

7 years agoMerge branch '10200-cwl-crunch-script' closes #10200
Peter Amstutz [Mon, 17 Oct 2016 19:05:53 +0000 (15:05 -0400)]
Merge branch '10200-cwl-crunch-script' closes #10200

7 years ago10259: Update arvados-cli version pin
Peter Amstutz [Mon, 17 Oct 2016 18:18:02 +0000 (14:18 -0400)]
10259: Update arvados-cli version pin

7 years ago3486: Accept false and nil/null as parameter values, and avoid translating nil to...
Tom Clegg [Fri, 14 Oct 2016 21:12:51 +0000 (17:12 -0400)]
3486: Accept false and nil/null as parameter values, and avoid translating nil to false.

7 years ago10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
Peter Amstutz [Mon, 17 Oct 2016 14:47:43 +0000 (10:47 -0400)]
10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
false and null as well.

7 years ago10194: Record git version in package
Peter Amstutz [Mon, 17 Oct 2016 17:48:39 +0000 (13:48 -0400)]
10194: Record git version in package

7 years ago10200: Copy cwl-runner crunch script into arvados_cwl package to avoid version skew...
Peter Amstutz [Wed, 12 Oct 2016 16:00:11 +0000 (12:00 -0400)]
10200: Copy cwl-runner crunch script into arvados_cwl package to avoid version skew.  Keep old cwl-runner code as a fallback to avoid breaking installations that haven't been updated.

7 years ago10025: run-build-docker-images.sh can use "arvbox build" instead of "rebuild"
Peter Amstutz [Mon, 17 Oct 2016 15:57:43 +0000 (11:57 -0400)]
10025: run-build-docker-images.sh can use "arvbox build" instead of "rebuild"
now because caching should be better behaved.

7 years ago10025: If $TAG starts with '-' it is not a tag.
Peter Amstutz [Mon, 17 Oct 2016 15:26:44 +0000 (11:26 -0400)]
10025: If $TAG starts with '-' it is not a tag.

7 years ago10025: Add ARG arvados_version to .dev and .demo
Peter Amstutz [Mon, 17 Oct 2016 13:53:37 +0000 (09:53 -0400)]
10025: Add ARG arvados_version to .dev and .demo

7 years ago10025: Use docker tag -f to change "latest".
Peter Amstutz [Mon, 17 Oct 2016 13:48:38 +0000 (09:48 -0400)]
10025: Use docker tag -f to change "latest".

7 years agoRemove obsolete comment. No issue #
Tom Clegg [Mon, 17 Oct 2016 13:24:03 +0000 (09:24 -0400)]
Remove obsolete comment. No issue #

7 years ago9956: Fix startup log message. refs #9956
Tom Clegg [Mon, 17 Oct 2016 12:58:01 +0000 (08:58 -0400)]
9956: Fix startup log message. refs #9956

7 years agoMerge branch '10211-double-close-crash' closes #10211
Tom Clegg [Mon, 17 Oct 2016 12:50:57 +0000 (08:50 -0400)]
Merge branch '10211-double-close-crash' closes #10211

7 years agoMerge branch '10281-directory-replication' closes #10281
Tom Clegg [Mon, 17 Oct 2016 12:39:10 +0000 (08:39 -0400)]
Merge branch '10281-directory-replication' closes #10281

7 years ago10281: Test default replication=1 for directory volumes.
Tom Clegg [Mon, 17 Oct 2016 12:26:26 +0000 (08:26 -0400)]
10281: Test default replication=1 for directory volumes.

7 years ago10281: Update comment.
Tom Clegg [Mon, 17 Oct 2016 12:12:27 +0000 (08:12 -0400)]
10281: Update comment.

7 years ago10281: Add DirectoryReplication config.
Tom Clegg [Mon, 17 Oct 2016 12:06:05 +0000 (08:06 -0400)]
10281: Add DirectoryReplication config.

7 years agoMerge branch '10280-squeue-error' closes #10280
Tom Clegg [Mon, 17 Oct 2016 10:39:41 +0000 (06:39 -0400)]
Merge branch '10280-squeue-error' closes #10280

7 years agoMerge branch '9956-keepstore-config' refs #9956
Tom Clegg [Mon, 17 Oct 2016 10:16:04 +0000 (06:16 -0400)]
Merge branch '9956-keepstore-config' refs #9956

7 years ago10280: Fix ignored error from cmd.Start().
Tom Clegg [Mon, 17 Oct 2016 10:15:29 +0000 (06:15 -0400)]
10280: Fix ignored error from cmd.Start().

7 years ago9956: gofmt
Tom Clegg [Mon, 17 Oct 2016 09:41:02 +0000 (05:41 -0400)]
9956: gofmt

7 years ago10211: Clarify expected result of double-close.
Tom Clegg [Mon, 17 Oct 2016 09:11:25 +0000 (05:11 -0400)]
10211: Clarify expected result of double-close.

7 years ago9956: Fix run_test_server: default is now -enforce-permissions=true.
Tom Clegg [Mon, 17 Oct 2016 08:57:18 +0000 (04:57 -0400)]
9956: Fix run_test_server: default is now -enforce-permissions=true.

7 years agoMerge branch 'master' into 9956-keepstore-config
Tom Clegg [Mon, 17 Oct 2016 08:32:37 +0000 (04:32 -0400)]
Merge branch 'master' into 9956-keepstore-config

Conflicts:
services/keepstore/keepstore.go

7 years ago9956: Show list of supported volume types.
Tom Clegg [Mon, 17 Oct 2016 08:27:55 +0000 (04:27 -0400)]
9956: Show list of supported volume types.

7 years ago9956: Remove obsolete TODO comment.
Tom Clegg [Mon, 17 Oct 2016 08:12:14 +0000 (04:12 -0400)]
9956: Remove obsolete TODO comment.

7 years ago9956: Move deprecated options out to a separate file.
Tom Clegg [Mon, 17 Oct 2016 08:09:41 +0000 (04:09 -0400)]
9956: Move deprecated options out to a separate file.

7 years ago9956: Remove "listen on address with no port" in help message.
Tom Clegg [Mon, 17 Oct 2016 08:08:43 +0000 (04:08 -0400)]
9956: Remove "listen on address with no port" in help message.

7 years agorefs #10227
radhika [Fri, 14 Oct 2016 18:18:46 +0000 (14:18 -0400)]
refs #10227
Merge branch '10227-make-diagnostics-tests-reliable'

7 years ago10211: Return an error instead of crashing if stream is closed twice.
Tom Clegg [Fri, 14 Oct 2016 15:49:21 +0000 (11:49 -0400)]
10211: Return an error instead of crashing if stream is closed twice.

7 years ago10025: Rearrange arvbox layers for better caching. Embed build version to
Peter Amstutz [Fri, 14 Oct 2016 15:29:20 +0000 (11:29 -0400)]
10025: Rearrange arvbox layers for better caching.  Embed build version to
ensure that upper layers rebuild but lower layers do not.

7 years agoMerge branch '10259-cwl-pipeline-value' refs #10259
Peter Amstutz [Fri, 14 Oct 2016 14:57:37 +0000 (10:57 -0400)]
Merge branch '10259-cwl-pipeline-value' refs #10259

7 years ago10272: Fix nil pointer dereference in help message. refs #10272
Tom Clegg [Fri, 14 Oct 2016 14:15:01 +0000 (10:15 -0400)]
10272: Fix nil pointer dereference in help message. refs #10272

7 years ago10259: Add one level of indirection on dict parameters so that
Peter Amstutz [Fri, 14 Oct 2016 01:13:58 +0000 (21:13 -0400)]
10259: Add one level of indirection on dict parameters so that
arv-run-pipeline-instance does not try to interpret the value.  Check pipeline
state so that polling loop for cwl-runner job does not get stuck.

7 years agoMerge branch '10234-git-httpd-config' refs #10234 refs #10263
Tom Clegg [Thu, 13 Oct 2016 20:43:55 +0000 (16:43 -0400)]
Merge branch '10234-git-httpd-config' refs #10234 refs #10263

7 years ago10234: Remove doc.go, already superseded by usage.go.
Tom Clegg [Thu, 13 Oct 2016 20:40:05 +0000 (16:40 -0400)]
10234: Remove doc.go, already superseded by usage.go.

7 years agoMerge branch '10221-cwl-pathmapping' closes #10221
Peter Amstutz [Thu, 13 Oct 2016 19:02:32 +0000 (15:02 -0400)]
Merge branch '10221-cwl-pathmapping' closes #10221

7 years agoMerge branch '10259-cwl-pipeline' closes #10259
Peter Amstutz [Thu, 13 Oct 2016 18:48:35 +0000 (14:48 -0400)]
Merge branch '10259-cwl-pipeline' closes #10259

7 years ago10234: Pass through gitolite env vars so old env-var-configured sites continue working.
Tom Clegg [Thu, 13 Oct 2016 18:47:10 +0000 (14:47 -0400)]
10234: Pass through gitolite env vars so old env-var-configured sites continue working.

7 years ago10234: 10263: Update arvados-git-httpd docs to use systemd + config file.
Tom Clegg [Thu, 13 Oct 2016 16:31:02 +0000 (12:31 -0400)]
10234: 10263: Update arvados-git-httpd docs to use systemd + config file.

7 years ago10234: 10263: Use a GitoliteHome configuration entry instead of passing through gitol...
Tom Clegg [Thu, 13 Oct 2016 15:02:18 +0000 (11:02 -0400)]
10234: 10263: Use a GitoliteHome configuration entry instead of passing through gitolite env vars from the caller.

7 years ago10227: The diagnostics tests have been consistently failing since Sep 18th due to...
radhika [Thu, 13 Oct 2016 17:22:51 +0000 (13:22 -0400)]
10227: The diagnostics tests have been consistently failing since Sep 18th due to web sockets based refresh is not happening in the page.
I updated the test to refresh the page in a loop until timeout, rather than rely on the unreliable web sockets.
Also, updated the text_to_look_for to "completed" from "Complete" so that test doesn't misinterpret a job completion as the pipeline completion.

7 years agoMerge branch '10236-disable-event-listening'
Lucas Di Pentima [Thu, 13 Oct 2016 16:59:35 +0000 (13:59 -0300)]
Merge branch '10236-disable-event-listening'
Closes #10236

7 years ago10236: Merge branch 'master' into 10236-disable-event-listening
Lucas Di Pentima [Thu, 13 Oct 2016 16:58:11 +0000 (13:58 -0300)]
10236: Merge branch 'master' into 10236-disable-event-listening

7 years ago10234: Avoid trying to read real config file in test suite.
Tom Clegg [Thu, 13 Oct 2016 05:52:43 +0000 (01:52 -0400)]
10234: Avoid trying to read real config file in test suite.

7 years ago10234: Use YAML for example config file.
Tom Clegg [Thu, 13 Oct 2016 05:52:09 +0000 (01:52 -0400)]
10234: Use YAML for example config file.

7 years ago10234: Clean up program name (arvados-git-httpd) and config paths.
Tom Clegg [Wed, 12 Oct 2016 23:02:48 +0000 (19:02 -0400)]
10234: Clean up program name (arvados-git-httpd) and config paths.

7 years ago10234: Change default repo root from CWD to recommended path.
Tom Clegg [Wed, 12 Oct 2016 23:00:51 +0000 (19:00 -0400)]
10234: Change default repo root from CWD to recommended path.

7 years ago10008: Disable flaky test. refs #10008
Tom Clegg [Thu, 13 Oct 2016 16:35:40 +0000 (12:35 -0400)]
10008: Disable flaky test. refs #10008

7 years agoMerge branch '10243-make-packing-optional'
Lucas Di Pentima [Thu, 13 Oct 2016 15:56:58 +0000 (12:56 -0300)]
Merge branch '10243-make-packing-optional'
Closes #10243

7 years ago10243: Merge branch 'master' into 10243-make-packing-optional
Lucas Di Pentima [Thu, 13 Oct 2016 15:56:11 +0000 (12:56 -0300)]
10243: Merge branch 'master' into 10243-make-packing-optional

7 years ago10236: Added --disable-event-listening to arv-mount command so that it doesn't subscr...
Lucas Di Pentima [Thu, 13 Oct 2016 15:14:55 +0000 (12:14 -0300)]
10236: Added --disable-event-listening to arv-mount command so that it doesn't subscribe to any API Server event.
Also, added a test for this new feature.

7 years agoMerge branch '9848-finalize-on-reuse' closes #9848
Tom Clegg [Thu, 13 Oct 2016 15:14:40 +0000 (11:14 -0400)]
Merge branch '9848-finalize-on-reuse' closes #9848

7 years ago9848: Finalize container request immediately if resolving to an already-finished...
Tom Clegg [Tue, 11 Oct 2016 21:21:06 +0000 (17:21 -0400)]
9848: Finalize container request immediately if resolving to an already-finished container.

7 years ago10259: Workaround for spurious "job_order" on command line. --no-wait returns
Peter Amstutz [Thu, 13 Oct 2016 14:36:30 +0000 (10:36 -0400)]
10259: Workaround for spurious "job_order" on command line.  --no-wait returns
pipeline uuid.

7 years ago10259: When using --submit with jobs API, create RunningOnServer pipeline
Peter Amstutz [Wed, 12 Oct 2016 20:58:25 +0000 (16:58 -0400)]
10259: When using --submit with jobs API, create RunningOnServer pipeline
instance.  This is more similar to the way it works when you run a workflow
from workbench, and ensures that the pipeline will be marked completed even if
the arvados-cwl-runner client goes away.