arvados.git
5 years agoMerge branch 'master' into 13306-arvados-cwl-runner-py3-support
Eric Biagiotti [Mon, 28 Jan 2019 15:34:10 +0000 (10:34 -0500)]
Merge branch 'master' into 13306-arvados-cwl-runner-py3-support

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Removing debug echo
Eric Biagiotti [Mon, 28 Jan 2019 15:14:30 +0000 (10:14 -0500)]
13306: Removing debug echo

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Adding support for building the dev docker jobs image with py3
Eric Biagiotti [Fri, 25 Jan 2019 18:03:12 +0000 (13:03 -0500)]
13306: Adding support for building the dev docker jobs image with py3

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Unicode fix for docker image generation in py3 environment
Eric Biagiotti [Thu, 24 Jan 2019 14:38:25 +0000 (09:38 -0500)]
13306: Unicode fix for docker image generation in py3 environment

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Unicode fixes for the conformance tests
Eric Biagiotti [Thu, 24 Jan 2019 14:37:13 +0000 (09:37 -0500)]
13306: Unicode fixes for the conformance tests

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '14482-rubysdk-empty-dir'
Lucas Di Pentima [Tue, 22 Jan 2019 16:40:02 +0000 (13:40 -0300)]
Merge branch '14482-rubysdk-empty-dir'
Closes #14482

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14482: Stricter token regexes.
Lucas Di Pentima [Fri, 18 Jan 2019 23:24:43 +0000 (20:24 -0300)]
14482: Stricter token regexes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Removing old_div to simplify division operations
Eric Biagiotti [Fri, 18 Jan 2019 16:08:13 +0000 (11:08 -0500)]
13306: Removing old_div to simplify division operations

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14482: Manifest validation including 8-bit escaped chars; with tests.
Lucas Di Pentima [Thu, 17 Jan 2019 23:53:58 +0000 (20:53 -0300)]
14482: Manifest validation including 8-bit escaped chars; with tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Removing list wrapping added by futurize to PathMapper objects
Eric Biagiotti [Thu, 17 Jan 2019 21:03:03 +0000 (16:03 -0500)]
13306: Removing list wrapping added by futurize to PathMapper objects

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

Because cwltool.PathMapper objects have a function called items(), futurize incorrectly added the results of calls to items() to a list object (thinking it was a call to dict.items()). In fact, cwltool.PathMapper.items() already returns a list.

5 years ago13306: Updating dict items as lists to improve py2 performance
Eric Biagiotti [Thu, 17 Jan 2019 20:55:24 +0000 (15:55 -0500)]
13306: Updating dict items as lists to improve py2 performance

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Fixing previous commit to use viewvalues instead of listvalues
Eric Biagiotti [Thu, 17 Jan 2019 20:40:08 +0000 (15:40 -0500)]
13306: Fixing previous commit to use viewvalues instead of listvalues

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updating dict values as a list to improve py2 performance
Eric Biagiotti [Thu, 17 Jan 2019 19:00:05 +0000 (14:00 -0500)]
13306: Updating dict values as a list to improve py2 performance

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updating dict keys as a list to ensure py2/3 compatibility
Eric Biagiotti [Thu, 17 Jan 2019 18:49:39 +0000 (13:49 -0500)]
13306: Updating dict keys as a list to ensure py2/3 compatibility

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '14539-pysdk-empty-dir'
Lucas Di Pentima [Thu, 17 Jan 2019 16:43:25 +0000 (13:43 -0300)]
Merge branch '14539-pysdk-empty-dir'
Closes #14539

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Updating dict value iteration to be more efficient with py2
Eric Biagiotti [Thu, 17 Jan 2019 16:21:04 +0000 (11:21 -0500)]
13306: Updating dict value iteration to be more efficient with py2

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Adds test for when get_current_container results in a 404
Eric Biagiotti [Thu, 17 Jan 2019 16:11:35 +0000 (11:11 -0500)]
13306: Adds test for when get_current_container results in a 404

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Removes a log handler added during a test, breaking subsequent tests.
Eric Biagiotti [Thu, 17 Jan 2019 15:04:03 +0000 (10:04 -0500)]
13306: Removes a log handler added during a test, breaking subsequent tests.

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Prevents multiple RunTimeStatusLoggingHandlers from being added
Eric Biagiotti [Thu, 17 Jan 2019 15:02:43 +0000 (10:02 -0500)]
13306: Prevents multiple RunTimeStatusLoggingHandlers from being added

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14539: Simplifies a bit further the regexp replacement on escape() function.
Lucas Di Pentima [Thu, 17 Jan 2019 09:55:40 +0000 (06:55 -0300)]
14539: Simplifies a bit further the regexp replacement on escape() function.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Removes pointless code comment.
Lucas Di Pentima [Thu, 17 Jan 2019 09:48:53 +0000 (06:48 -0300)]
14539: Removes pointless code comment.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Fixes and simplifies the escape() function. Updates test.
Lucas Di Pentima [Wed, 16 Jan 2019 22:34:34 +0000 (19:34 -0300)]
14539: Fixes and simplifies the escape() function. Updates test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Fixes literal backslash escaping.
Lucas Di Pentima [Wed, 16 Jan 2019 21:33:49 +0000 (18:33 -0300)]
14539: Fixes literal backslash escaping.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Removing unneeded try/except statements in the acr submit tests
Eric Biagiotti [Wed, 16 Jan 2019 01:52:36 +0000 (20:52 -0500)]
13306: Removing unneeded try/except statements in the acr submit tests

Removed the try/except statements and moved the asserts checking if the call to arvados_cwl.main exited properly to the bottom of the tests.  In other words, the tests should be using specific call asserts to establish if it ran correctly and checking the exited value should be a last resort to report an unhandled case.

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14482: Allow unescaped " " on stream and file token regexes (WIP)
Lucas Di Pentima [Tue, 15 Jan 2019 21:27:29 +0000 (18:27 -0300)]
14482: Allow unescaped " " on stream and file token regexes (WIP)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14324-cdc-azure' refs #14324
Peter Amstutz [Tue, 15 Jan 2019 18:40:34 +0000 (13:40 -0500)]
Merge branch '14324-cdc-azure' refs #14324

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14482: Adds failing test about partially unescaped file tokens.
Lucas Di Pentima [Tue, 15 Jan 2019 18:33:58 +0000 (15:33 -0300)]
14482: Adds failing test about partially unescaped file tokens.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14324: Embed test public key instead of reading from file
Peter Amstutz [Tue, 15 Jan 2019 18:29:30 +0000 (13:29 -0500)]
14324: Embed test public key instead of reading from file

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch 'master' into 14482-rubysdk-empty-dir
Lucas Di Pentima [Tue, 15 Jan 2019 17:34:38 +0000 (14:34 -0300)]
Merge branch 'master' into 14482-rubysdk-empty-dir

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Encode \t & \n chars on file/dir names on collection's manifest.
Lucas Di Pentima [Tue, 15 Jan 2019 17:25:39 +0000 (14:25 -0300)]
14539: Encode \t & \n chars on file/dir names on collection's manifest.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Updating get_current_container to raise when not a 404
Eric Biagiotti [Tue, 15 Jan 2019 16:57:45 +0000 (11:57 -0500)]
13306: Updating get_current_container to raise when not a 404

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updating depreciated assertEquals
Eric Biagiotti [Tue, 15 Jan 2019 16:09:11 +0000 (11:09 -0500)]
13306: Updating depreciated assertEquals

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updating depreciated warn function
Eric Biagiotti [Tue, 15 Jan 2019 15:52:04 +0000 (10:52 -0500)]
13306: Updating depreciated warn function

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updates ArvCwlExecutor to properly convert objects to JSON in unicode
Eric Biagiotti [Tue, 15 Jan 2019 14:46:50 +0000 (09:46 -0500)]
13306: Updates ArvCwlExecutor to properly convert objects to JSON in unicode

json.dumps produces a str object in python 2 (bytestring) and a str object in python 3 (unicode), resulting in incompatibility when writing to the file (expects unicode). In order to prevent using py2 and py3 specific code, the solution is to call encode, which will force the string to be bytes in both languages, then decode to unicode from there.

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14538: Update comment.
Lucas Di Pentima [Tue, 15 Jan 2019 14:31:43 +0000 (11:31 -0300)]
14538: Update comment.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13306: Adding py2/3 unicode support to the test_job tests
Eric Biagiotti [Tue, 15 Jan 2019 13:30:53 +0000 (08:30 -0500)]
13306: Adding py2/3 unicode support to the test_job tests

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14324: Fix API throttling header fallbacks. Add gocheck_test
Peter Amstutz [Mon, 14 Jan 2019 19:13:20 +0000 (14:13 -0500)]
14324:  Fix API throttling header fallbacks.  Add gocheck_test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: test_submit additional fixes for py2 unicode support
Eric Biagiotti [Mon, 14 Jan 2019 18:04:55 +0000 (13:04 -0500)]
13306: test_submit additional fixes for py2 unicode support

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch 'master' into 14539-pysdk-empty-dir
Lucas Di Pentima [Mon, 14 Jan 2019 17:16:32 +0000 (14:16 -0300)]
Merge branch 'master' into 14539-pysdk-empty-dir

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoFix arvbox demo for workbench2 no issue #
Peter Amstutz [Mon, 14 Jan 2019 16:00:12 +0000 (11:00 -0500)]
Fix arvbox demo for workbench2 no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: test_submit fixes for py2 unicode support
Eric Biagiotti [Fri, 11 Jan 2019 21:38:47 +0000 (16:38 -0500)]
13306: test_submit fixes for py2 unicode support

Makes the stdout passed to cwltool python version dependent for the submit tests. Also adds unicode specifications to calls to the log.

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14324: Fix tests
Peter Amstutz [Fri, 11 Jan 2019 19:04:06 +0000 (14:04 -0500)]
14324: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: Adding py2 and py3 compatible byte objects for literals
Eric Biagiotti [Fri, 11 Jan 2019 18:55:49 +0000 (13:55 -0500)]
13306: Adding py2 and py3 compatible byte objects for literals

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Updated use of built in 'sorted' and a test_util bytestring
Eric Biagiotti [Fri, 11 Jan 2019 16:22:24 +0000 (11:22 -0500)]
13306: Updated use of built in 'sorted' and a test_util bytestring

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Added py2 and 3 compatible 'next' for test_job.TestWorkflow.test_run
Eric Biagiotti [Fri, 11 Jan 2019 15:12:52 +0000 (10:12 -0500)]
13306: Added py2 and 3 compatible 'next' for test_job.TestWorkflow.test_run

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14324: Use logrus in Azure driver. Fix Sirupsen->sirupsen in imports
Peter Amstutz [Fri, 11 Jan 2019 15:08:37 +0000 (10:08 -0500)]
14324: Use logrus in Azure driver.  Fix Sirupsen->sirupsen in imports

"It's in the past been possible to import Logrus as both upper- and
lower-case. Due to the Go package environment, this caused issues in
the community and we needed a standard. Some environments experienced
problems with the upper-case variant, so the lower-case was
decided. Everything using logrus will need to use the lower-case:
github.com/sirupsen/logrus. Any package that isn't, should be
changed."

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: Fixed test_submit.TestCreateTemplate encoding
Eric Biagiotti [Thu, 10 Jan 2019 18:54:41 +0000 (13:54 -0500)]
13306: Fixed test_submit.TestCreateTemplate encoding

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14324: Remove context from interface, use Stop() instead
Peter Amstutz [Thu, 10 Jan 2019 18:33:50 +0000 (13:33 -0500)]
14324: Remove context from interface, use Stop() instead

Remove 'Context' passed into methods of the InstanceSet interface,
instead the AzureInstanceSet creates its own cancellable background
context.  The Stop() method now cancels the context and waits for any
outstanding cloud driver work to complete before returning.

* Fix test
* Document how to run individual test cases against real cloud
* ManageBlobs runs periodically in the background
* Logging cleaned up

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: Fix for test_job.TestJob.test_done and test_done_use_existing_collection
Eric Biagiotti [Thu, 10 Jan 2019 14:57:49 +0000 (09:57 -0500)]
13306: Fix for test_job.TestJob.test_done and test_done_use_existing_collection

Had to mock a key for the CollectionReader due to 'list' being added to arvjob.py line 249 during stage2 of finalize. The 'list' call interrupts the on demand mocking of the CollectionReader object.

Arvados-DCO-1.1-Signed-off-by:  Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Fix for sdk/cwl tests.test_job.TestJob.test_run
Eric Biagiotti [Thu, 10 Jan 2019 14:38:37 +0000 (09:38 -0500)]
13306: Fix for sdk/cwl tests.test_job.TestJob.test_run

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Fixed formatting for newly introduced imports
Eric Biagiotti [Thu, 10 Jan 2019 14:16:37 +0000 (09:16 -0500)]
13306: Fixed formatting for newly introduced imports

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Changes to arvados-cwl-runner code after running futurize --stage2
Eric Biagiotti [Thu, 10 Jan 2019 14:06:42 +0000 (09:06 -0500)]
13306: Changes to arvados-cwl-runner code after running futurize --stage2

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14324: Azure driver for crunch-dispatch-cloud
Peter Amstutz [Wed, 9 Jan 2019 21:13:56 +0000 (16:13 -0500)]
14324: Azure driver for crunch-dispatch-cloud

* Adds context parameter to methods of InstanceSet interface that may block
* Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14324: Azure driver WIP
Peter Amstutz [Wed, 9 Jan 2019 16:35:55 +0000 (11:35 -0500)]
14324: Azure driver WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Fail on missing node-token tag
Peter Amstutz [Fri, 31 Aug 2018 20:41:07 +0000 (16:41 -0400)]
13964: Fail on missing node-token tag

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Forget the node token once the key fingerprint is known
Peter Amstutz [Fri, 31 Aug 2018 20:27:08 +0000 (16:27 -0400)]
13964: Forget the node token once the key fingerprint is known

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Fix string pointer capture
Peter Amstutz [Fri, 31 Aug 2018 20:15:55 +0000 (16:15 -0400)]
13964: Fix string pointer capture

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Trim space
Peter Amstutz [Fri, 31 Aug 2018 20:11:12 +0000 (16:11 -0400)]
13964: Trim space

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: ssh key checking
Peter Amstutz [Fri, 31 Aug 2018 19:55:43 +0000 (15:55 -0400)]
13964: ssh key checking

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: ssh key verification wip
Peter Amstutz [Fri, 31 Aug 2018 19:11:58 +0000 (15:11 -0400)]
13964: ssh key verification wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Set and check node-token WIP
Peter Amstutz [Fri, 31 Aug 2018 18:47:28 +0000 (14:47 -0400)]
13964: Set and check node-token WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Tweak test
Peter Amstutz [Fri, 31 Aug 2018 18:24:28 +0000 (14:24 -0400)]
13964: Tweak test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: SSH access WIP
Peter Amstutz [Fri, 31 Aug 2018 15:42:01 +0000 (11:42 -0400)]
13964: SSH access WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: InstanceTags type
Peter Amstutz [Thu, 23 Aug 2018 20:38:28 +0000 (16:38 -0400)]
13964: InstanceTags type

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: VM names are namespaced. Can set/get tags.
Peter Amstutz [Thu, 23 Aug 2018 20:22:12 +0000 (16:22 -0400)]
13964: VM names are namespaced.  Can set/get tags.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Don't panic running tests on stubs
Peter Amstutz [Thu, 23 Aug 2018 19:21:05 +0000 (15:21 -0400)]
13964: Don't panic running tests on stubs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Detect and report rate limit and quota exceeded errors
Peter Amstutz [Thu, 23 Aug 2018 19:15:42 +0000 (15:15 -0400)]
13964: Detect and report rate limit and quota exceeded errors

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Return Arvados instance type
Peter Amstutz [Thu, 23 Aug 2018 02:42:26 +0000 (22:42 -0400)]
13964: Return Arvados instance type

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Full lifecycle for NICs/VHDs/VMs
Peter Amstutz [Thu, 23 Aug 2018 02:13:23 +0000 (22:13 -0400)]
13964: Full lifecycle for NICs/VHDs/VMs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Can successfully create a VM, list existing VMs
Peter Amstutz [Tue, 21 Aug 2018 15:28:37 +0000 (11:28 -0400)]
13964: Can successfully create a VM, list existing VMs

Cleanup unused NICs.  Blob cleanup in progress.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13964: Azure provider WIP
Peter Amstutz [Mon, 20 Aug 2018 13:41:16 +0000 (09:41 -0400)]
13964: Azure provider WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoarvbox uses 'down: ready' as ready condition no issue #
Peter Amstutz [Wed, 9 Jan 2019 19:39:43 +0000 (14:39 -0500)]
arvbox uses 'down: ready' as ready condition no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoConfigure UTF-8 locale inside arvbox, no issue #
Peter Amstutz [Wed, 9 Jan 2019 15:13:56 +0000 (10:13 -0500)]
Configure UTF-8 locale inside arvbox, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14660-arvbox-workbench2' refs #14660
Peter Amstutz [Tue, 8 Jan 2019 22:00:24 +0000 (17:00 -0500)]
Merge branch '14660-arvbox-workbench2' refs #14660

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14660: make "sv restart" work for workbench2, prevent fail loop
Peter Amstutz [Tue, 8 Jan 2019 21:57:56 +0000 (16:57 -0500)]
14660: make "sv restart" work for workbench2, prevent fail loop

Print out https url for workbench2 on start.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13306: Changes to arvados-cwl-runner code after running futurize --stage1
ebiagiotti [Mon, 7 Jan 2019 20:38:58 +0000 (20:38 +0000)]
13306: Changes to arvados-cwl-runner code after running futurize --stage1

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago13306: Adding a py3 variant of the sdk/cwl tests to run-tests.sh
ebiagiotti [Mon, 7 Jan 2019 16:19:21 +0000 (16:19 +0000)]
13306: Adding a py3 variant of the sdk/cwl tests to run-tests.sh

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14482: Fix Manifest::files_count()
Lucas Di Pentima [Mon, 7 Jan 2019 15:39:12 +0000 (12:39 -0300)]
14482: Fix Manifest::files_count()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14482: Handle escaped chars on manifest validation.
Lucas Di Pentima [Mon, 7 Jan 2019 14:12:01 +0000 (11:12 -0300)]
14482: Handle escaped chars on manifest validation.

* Escaped & unescaped chars are treated equally, except for \040.
* Empty '.' (and \056) named file tokens are accepted.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14691-suggested-ssh-config-update'
Ward Vandewege [Fri, 4 Jan 2019 22:28:56 +0000 (17:28 -0500)]
Merge branch '14691-suggested-ssh-config-update'

closes #14691

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14482: Don't list empty dir placeholder files on collections listings.
Lucas Di Pentima [Fri, 4 Jan 2019 21:37:15 +0000 (18:37 -0300)]
14482: Don't list empty dir placeholder files on collections listings.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14482: Enhance code readability
Lucas Di Pentima [Fri, 4 Jan 2019 21:34:58 +0000 (18:34 -0300)]
14482: Enhance code readability

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14691: workbench update, bring in line with documentation
Ward Vandewege [Fri, 4 Jan 2019 21:02:54 +0000 (16:02 -0500)]
14691: workbench update, bring in line with documentation

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14691: documentation update
Ward Vandewege [Fri, 4 Jan 2019 15:29:43 +0000 (10:29 -0500)]
14691: documentation update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago14660: Enable workbench2 to connect to development server via websockets.
Peter Amstutz [Fri, 4 Jan 2019 19:30:39 +0000 (14:30 -0500)]
14660: Enable workbench2 to connect to development server via websockets.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14660: Proxy keep-web for https. Install root cert inside the container.
Peter Amstutz [Fri, 4 Jan 2019 16:28:12 +0000 (11:28 -0500)]
14660: Proxy keep-web for https.  Install root cert inside the container.

Now services inside the container can talk to each other without the
"insecure" flag.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14660: Add workbench2 to arvbox. Improve SSL support in arvbox.
Peter Amstutz [Thu, 3 Jan 2019 19:53:25 +0000 (14:53 -0500)]
14660: Add workbench2 to arvbox.  Improve SSL support in arvbox.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14539: Escape backslash & space chars on file/dir names, with tests.
Lucas Di Pentima [Thu, 20 Dec 2018 19:32:39 +0000 (16:32 -0300)]
14539: Escape backslash & space chars on file/dir names, with tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Fixes fuse tests.
Lucas Di Pentima [Tue, 18 Dec 2018 00:54:07 +0000 (21:54 -0300)]
14539: Fixes fuse tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Tests that '\056' cannot be used as a file or directory name.
Lucas Di Pentima [Mon, 17 Dec 2018 23:35:31 +0000 (20:35 -0300)]
14539: Tests that '\056' cannot be used as a file or directory name.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14539: Persists empty directories by adding a '\056' empty file.
Lucas Di Pentima [Mon, 17 Dec 2018 21:18:52 +0000 (18:18 -0300)]
14539: Persists empty directories by adding a '\056' empty file.

Also:
* Fixes test
* Errors out when trying to open a file/dir with '\056' on its path.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14360-dispatch-cloud'
Tom Clegg [Thu, 20 Dec 2018 19:02:05 +0000 (14:02 -0500)]
Merge branch '14360-dispatch-cloud'

closes #14360

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Merge branch 'master' into 14360-dispatch-cloud
Tom Clegg [Thu, 20 Dec 2018 18:58:26 +0000 (13:58 -0500)]
14360: Merge branch 'master' into 14360-dispatch-cloud

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Fix error log level on first probe after boot.
Tom Clegg [Thu, 20 Dec 2018 18:57:27 +0000 (13:57 -0500)]
14360: Fix error log level on first probe after boot.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Test for errant Create calls after failure.
Tom Clegg [Thu, 20 Dec 2018 16:00:10 +0000 (11:00 -0500)]
14360: Test for errant Create calls after failure.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Fix test case.
Tom Clegg [Thu, 20 Dec 2018 15:03:19 +0000 (10:03 -0500)]
14360: Fix test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Fix instance creation limiting.
Tom Clegg [Thu, 20 Dec 2018 14:44:45 +0000 (09:44 -0500)]
14360: Fix instance creation limiting.

Scheduler was claiming one of its unalloc instances only if the
container could be scheduled, so the "need new instance" condition was
not triggered for lower-priority containers needing the same instance
type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Fix panic in test when last container is run twice.
Tom Clegg [Thu, 20 Dec 2018 14:42:49 +0000 (09:42 -0500)]
14360: Fix panic in test when last container is run twice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Debug log when acquiring lock.
Tom Clegg [Thu, 20 Dec 2018 14:27:37 +0000 (09:27 -0500)]
14360: Debug log when acquiring lock.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14360: Shutdown pool between tests to eliminate leaking logs.
Tom Clegg [Thu, 20 Dec 2018 14:25:56 +0000 (09:25 -0500)]
14360: Shutdown pool between tests to eliminate leaking logs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>