arvados.git
5 years agoFix resource leak warned by Python 3
Tom Morris [Thu, 14 Mar 2019 22:30:52 +0000 (18:30 -0400)]
Fix resource leak warned by Python 3

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoAdd crunchstat-summary to Python 3 tests. refs #14939
Tom Morris [Wed, 13 Mar 2019 23:33:03 +0000 (19:33 -0400)]
Add crunchstat-summary to Python 3 tests. refs #14939

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoFix test Python 3 integer divide difference.
Tom Morris [Wed, 13 Mar 2019 23:26:00 +0000 (19:26 -0400)]
Fix test Python 3 integer divide difference.

Use float for both Python 2 & 3 like the rest of the metrics.
refs #14393

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoUpdate crunchstat-summary to Python 3. Refs #14939
Tom Morris [Fri, 8 Mar 2019 23:40:24 +0000 (18:40 -0500)]
Update crunchstat-summary to Python 3. Refs #14939

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoMerge branch '14873-google-api-client-update'
Lucas Di Pentima [Wed, 20 Mar 2019 20:17:07 +0000 (17:17 -0300)]
Merge branch '14873-google-api-client-update'
Refs #14873

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14807-escalate-sigterm'
Tom Clegg [Wed, 20 Mar 2019 19:24:02 +0000 (15:24 -0400)]
Merge branch '14807-escalate-sigterm'

refs #14807

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Merge branch 'master'
Tom Clegg [Wed, 20 Mar 2019 19:20:57 +0000 (15:20 -0400)]
14807: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Tweak log message.
Tom Clegg [Wed, 20 Mar 2019 19:20:38 +0000 (15:20 -0400)]
14807: Tweak log message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14947-run-tests-interactive'
Tom Clegg [Wed, 20 Mar 2019 18:07:06 +0000 (14:07 -0400)]
Merge branch '14947-run-tests-interactive'

closes #14947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Ignore --only / --skip args in interactive mode.
Tom Clegg [Wed, 20 Mar 2019 17:41:35 +0000 (13:41 -0400)]
14947: Ignore --only / --skip args in interactive mode.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14873: Updates arvados & arvados-cli gems dependency on google-api-client.
Lucas Di Pentima [Wed, 20 Mar 2019 17:39:03 +0000 (14:39 -0300)]
14873: Updates arvados & arvados-cli gems dependency on google-api-client.

Start using our own fork arvados-google-api-client that ask for activesupport
version <5.1 so that it doesn't block the rails 5 upgrade.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14947: Fix controller install in deps.
Tom Clegg [Wed, 20 Mar 2019 17:37:28 +0000 (13:37 -0400)]
14947: Fix controller install in deps.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Accept test suite options in interactive mode.
Tom Clegg [Wed, 20 Mar 2019 13:28:24 +0000 (09:28 -0400)]
14947: Accept test suite options in interactive mode.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Install controller (cmd/arvados-server) for integration tests.
Tom Clegg [Tue, 19 Mar 2019 20:23:55 +0000 (16:23 -0400)]
14947: Install controller (cmd/arvados-server) for integration tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Don't send SIGKILL to crunch-run processes.
Tom Clegg [Tue, 19 Mar 2019 17:28:39 +0000 (13:28 -0400)]
14807: Don't send SIGKILL to crunch-run processes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Add comments to "error indicates success" code.
Tom Clegg [Tue, 19 Mar 2019 15:57:32 +0000 (11:57 -0400)]
14807: Add comments to "error indicates success" code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Suggest "install deps" as first interactive command.
Tom Clegg [Tue, 19 Mar 2019 14:30:36 +0000 (10:30 -0400)]
14947: Suggest "install deps" as first interactive command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMove back arvbox version of nodejs to fix composer compile issue
Peter Amstutz [Mon, 18 Mar 2019 20:13:35 +0000 (16:13 -0400)]
Move back arvbox version of nodejs to fix composer compile issue

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14807: Update tests to new worker/pool interface.
Tom Clegg [Mon, 18 Mar 2019 19:16:25 +0000 (15:16 -0400)]
14807: Update tests to new worker/pool interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Lengthen kill delays in stub driver to test term-kill-drain.
Tom Clegg [Mon, 18 Mar 2019 19:08:19 +0000 (15:08 -0400)]
14807: Lengthen kill delays in stub driver to test term-kill-drain.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14322-cwl-uuid-input' refs #14322
Peter Amstutz [Mon, 18 Mar 2019 18:43:31 +0000 (14:43 -0400)]
Merge branch '14322-cwl-uuid-input' refs #14322

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak JSON_KEEP_LOCATOR_REGEXP and comment
Peter Amstutz [Mon, 18 Mar 2019 18:04:19 +0000 (14:04 -0400)]
14322: Tweak JSON_KEEP_LOCATOR_REGEXP and comment

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14807: Escalate sigterm->sigkill->drain instance.
Tom Clegg [Mon, 18 Mar 2019 15:09:09 +0000 (11:09 -0400)]
14807: Escalate sigterm->sigkill->drain instance.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Fail test if a container gets run twice.
Tom Clegg [Mon, 18 Mar 2019 15:44:19 +0000 (11:44 -0400)]
14807: Fail test if a container gets run twice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Fix unreliable test.
Tom Clegg [Mon, 18 Mar 2019 15:44:07 +0000 (11:44 -0400)]
14807: Fix unreliable test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14977-kill-if-not-in-queue'
Tom Clegg [Mon, 18 Mar 2019 15:11:56 +0000 (11:11 -0400)]
Merge branch '14977-kill-if-not-in-queue'

fixes #14977

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Re-order statements for clarity.
Tom Clegg [Mon, 18 Mar 2019 13:42:27 +0000 (09:42 -0400)]
14977: Re-order statements for clarity.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Don't try fixStaleLocks until worker pool state is loaded.
Tom Clegg [Fri, 15 Mar 2019 20:19:51 +0000 (16:19 -0400)]
14977: Don't try fixStaleLocks until worker pool state is loaded.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Kill containers that don't exist according to controller.
Tom Clegg [Fri, 15 Mar 2019 20:11:48 +0000 (16:11 -0400)]
14977: Kill containers that don't exist according to controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Drop containers when the controller says they don't exist.
Tom Clegg [Fri, 15 Mar 2019 19:21:39 +0000 (15:21 -0400)]
14977: Drop containers when the controller says they don't exist.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14322: Fix noisy unicode logging error in tests, add comments
Peter Amstutz [Fri, 15 Mar 2019 14:54:59 +0000 (10:54 -0400)]
14322: Fix noisy unicode logging error in tests, add comments

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Add missing test files
Peter Amstutz [Wed, 13 Mar 2019 21:30:57 +0000 (17:30 -0400)]
14322: Add missing test files

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Workbench links collection uuids in workflow inputs
Peter Amstutz [Wed, 13 Mar 2019 21:26:07 +0000 (17:26 -0400)]
14322: Workbench links collection uuids in workflow inputs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Update comment on bwa-mem-input-mixed.yml
Peter Amstutz [Wed, 13 Mar 2019 21:20:22 +0000 (17:20 -0400)]
14322: Update comment on bwa-mem-input-mixed.yml

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Add additional example input files
Peter Amstutz [Wed, 13 Mar 2019 20:40:33 +0000 (16:40 -0400)]
14322: Add additional example input files

* with uuids
* with both portable data hashes and uuids

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak workbench support from manual testing
Peter Amstutz [Wed, 13 Mar 2019 20:19:37 +0000 (16:19 -0400)]
14322: Tweak workbench support from manual testing

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Update workbench to use new collectionUUID
Peter Amstutz [Wed, 13 Mar 2019 19:52:20 +0000 (15:52 -0400)]
14322: Update workbench to use new collectionUUID

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Fix tests
Peter Amstutz [Wed, 13 Mar 2019 19:25:33 +0000 (15:25 -0400)]
14322: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak documentation to mention uuid keep URI form
Peter Amstutz [Wed, 13 Mar 2019 18:57:53 +0000 (14:57 -0400)]
14322: Tweak documentation to mention uuid keep URI form

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: CollectionFsAccess accepts UUIDs
Peter Amstutz [Wed, 13 Mar 2019 18:57:20 +0000 (14:57 -0400)]
14322: CollectionFsAccess accepts UUIDs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tests for edge cases
Peter Amstutz [Tue, 12 Mar 2019 20:39:12 +0000 (16:39 -0400)]
14322: Tests for edge cases

Report unknown uuids and mismatches between current collection
PDH (from API server lookup) and location PDH.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Propagate collection UUID to container mount
Peter Amstutz [Mon, 11 Mar 2019 21:42:09 +0000 (17:42 -0400)]
14322: Propagate collection UUID to container mount

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Convert uuids on input to portable data hashes
Peter Amstutz [Mon, 11 Mar 2019 21:02:14 +0000 (17:02 -0400)]
14322: Convert uuids on input to portable data hashes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFix a-c-r printing logs on error refs #14870
Peter Amstutz [Fri, 15 Mar 2019 14:10:15 +0000 (10:10 -0400)]
Fix a-c-r printing logs on error refs #14870

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14947: run-tests: add --interactive, skip deps on --skip-install.
Tom Clegg [Thu, 14 Mar 2019 20:24:18 +0000 (16:24 -0400)]
14947: run-tests: add --interactive, skip deps on --skip-install.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14966-api-test-hang-fix'
Lucas Di Pentima [Thu, 14 Mar 2019 19:42:18 +0000 (16:42 -0300)]
Merge branch '14966-api-test-hang-fix'
Closes #14966

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoArvbox updates, no issue #
Peter Amstutz [Thu, 7 Feb 2019 22:07:08 +0000 (17:07 -0500)]
Arvbox updates, no issue #

* Bump Go version, add openjdk.

* Bump node version.  Make sure node is in path.

* Fix $HOME in setup scripts.

* Add "arvbox devenv" command which creates and enters a container
  environment suitable for interactively running development tools

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14966: Implements cleanup on test case teardown.
Lucas Di Pentima [Thu, 14 Mar 2019 18:56:00 +0000 (15:56 -0300)]
14966: Implements cleanup on test case teardown.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14966: Fixes test.
Lucas Di Pentima [Wed, 13 Mar 2019 23:52:25 +0000 (20:52 -0300)]
14966: Fixes test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoSwitch to stretch-slim as the base image for arvados/jobs, this results in a
Ward Vandewege [Wed, 13 Mar 2019 15:42:30 +0000 (11:42 -0400)]
Switch to stretch-slim as the base image for arvados/jobs, this results in a
smaller image.

arvados/jobs         1.3.1.20190308210512   2a6c5e4993e7        4 days ago          284MB
arvados/jobs         latest                 0cfff29120dd        20 minutes ago      232MB

refs #14819

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '13593-async-perm-graph-update'
Lucas Di Pentima [Wed, 13 Mar 2019 15:09:14 +0000 (12:09 -0300)]
Merge branch '13593-async-perm-graph-update'
Closes #13593

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '13937-keepstore-prometheus'
Lucas Di Pentima [Tue, 12 Mar 2019 20:32:43 +0000 (17:32 -0300)]
Merge branch '13937-keepstore-prometheus'
Closes #13937

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Moves & enhances group creation with async=true test.
Lucas Di Pentima [Tue, 12 Mar 2019 18:58:57 +0000 (15:58 -0300)]
13593: Moves & enhances group creation with async=true test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13937: Removes the 'any' operation type for a better way of testing driver labels
Lucas Di Pentima [Tue, 12 Mar 2019 18:19:36 +0000 (15:19 -0300)]
13937: Removes the 'any' operation type for a better way of testing driver labels

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Cleans up old experimental code.
Lucas Di Pentima [Tue, 12 Mar 2019 15:04:49 +0000 (12:04 -0300)]
13593: Cleans up old experimental code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Changes default async permission update interval. Clears cache.
Lucas Di Pentima [Tue, 12 Mar 2019 14:38:09 +0000 (11:38 -0300)]
13593: Changes default async permission update interval. Clears cache.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoSome build script housekeeping: fix the detection logic for the existence of
Ward Vandewege [Tue, 12 Mar 2019 14:32:05 +0000 (10:32 -0400)]
Some build script housekeeping: fix the detection logic for the existence of
Debian/Ubuntu packages on our apt repositories.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBuild the cwltest package, which lives out of tree.
Ward Vandewege [Tue, 12 Mar 2019 14:31:13 +0000 (10:31 -0400)]
Build the cwltest package, which lives out of tree.

refs #9945

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago13937: Removes unnecessary data passing on tests.
Lucas Di Pentima [Tue, 12 Mar 2019 14:10:51 +0000 (11:10 -0300)]
13937: Removes unnecessary data passing on tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Updates API documentation.
Lucas Di Pentima [Tue, 12 Mar 2019 12:51:11 +0000 (09:51 -0300)]
13593: Updates API documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Adds 'async' parameter to groups create & update calls for perms updates.
Lucas Di Pentima [Tue, 12 Mar 2019 12:33:30 +0000 (09:33 -0300)]
13593: Adds 'async' parameter to groups create & update calls for perms updates.

When async=true, the permission graph update will be deferred by the configured
amount of seconds at 'async_permissions_update_interval' (default: 60)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoFix error message.
Tom Clegg [Mon, 11 Mar 2019 20:21:26 +0000 (16:21 -0400)]
Fix error message.

No issue #.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoExplicitly print the version numbers of virtualenv, pip, setuptools and wheel
Ward Vandewege [Mon, 11 Mar 2019 15:04:44 +0000 (11:04 -0400)]
Explicitly print the version numbers of virtualenv, pip, setuptools and wheel
when building Python packages.

refs #14936

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '14938-unreliable-test'
Tom Clegg [Mon, 11 Mar 2019 04:34:33 +0000 (00:34 -0400)]
Merge branch '14938-unreliable-test'

fixes #14938

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14938: Fix unreliable test.
Tom Clegg [Mon, 11 Mar 2019 04:32:22 +0000 (00:32 -0400)]
14938: Fix unreliable test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14880-keepdocker-argparse' refs #14880
Peter Amstutz [Fri, 8 Mar 2019 21:05:12 +0000 (16:05 -0500)]
Merge branch '14880-keepdocker-argparse' refs #14880

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14880: Fix help message typo.
Peter Amstutz [Fri, 8 Mar 2019 21:03:03 +0000 (16:03 -0500)]
14880: Fix help message typo.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago13937: Adds tests for operation & I/O counters.
Lucas Di Pentima [Fri, 8 Mar 2019 20:56:17 +0000 (17:56 -0300)]
13937: Adds tests for operation & I/O counters.

Added {"operation": "any"} to opsCounters so that it's easy to check on tests,
because the kind of operation depends on the driver and the tests were
implemented as part of the volume generic tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '14291-cdc-aws' refs #14291
Peter Amstutz [Fri, 8 Mar 2019 20:19:59 +0000 (15:19 -0500)]
Merge branch '14291-cdc-aws' refs #14291

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: EbsVolumeType → EBSVolumeType
Peter Amstutz [Fri, 8 Mar 2019 20:18:44 +0000 (15:18 -0500)]
14291: EbsVolumeType → EBSVolumeType

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14880: Fix argument parsing issues
Peter Amstutz [Fri, 8 Mar 2019 15:14:27 +0000 (10:14 -0500)]
14880: Fix argument parsing issues

arv-keepdocker passes through some flags to arv-put.  The strategy it
uses is to remove known flags and pass through unknown flags. However
it doesn't know if an unknown flag takes a value or not, so it treats
the values as a positional argument rather than the value
corresponding to the unknown flag.

* Move positional arguments from "keepdocker_parser" to "arg_parser"
* Strip positional arguments (image name, tag) from arguments before
  parsing known keepdocker_parser flags.
* Additionally move a couple of flags that were mistakenly added to
  put.run_opts (and thus appearing in the arv-keepdocker help despite
  making no sense) that should have been in put.upload_opts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Exclude both "shutting-down" and "terminated" instances.
Peter Amstutz [Thu, 7 Mar 2019 21:53:14 +0000 (16:53 -0500)]
14291: Exclude both "shutting-down" and "terminated" instances.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Generate key fingerprints that work with AWS. Filter out terminated instances
Peter Amstutz [Thu, 7 Mar 2019 21:27:18 +0000 (16:27 -0500)]
14291: Generate key fingerprints that work with AWS.  Filter out terminated instances

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14920-unknown-booting-race'
Tom Clegg [Thu, 7 Mar 2019 19:35:01 +0000 (14:35 -0500)]
Merge branch '14920-unknown-booting-race'

fixes #14920

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14920: Update metrics when worker state changes idle->running.
Tom Clegg [Thu, 7 Mar 2019 19:27:13 +0000 (14:27 -0500)]
14920: Update metrics when worker state changes idle->running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14291: Add EbsVolumeType, also test fixes
Peter Amstutz [Thu, 7 Mar 2019 19:16:40 +0000 (14:16 -0500)]
14291: Add EbsVolumeType, also test fixes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Tweak Scratch/AddedScratch/IncludedScratch behavior
Peter Amstutz [Thu, 7 Mar 2019 19:16:11 +0000 (14:16 -0500)]
14291: Tweak Scratch/AddedScratch/IncludedScratch behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14870-retry-logs' refs #14870
Peter Amstutz [Thu, 7 Mar 2019 17:58:04 +0000 (12:58 -0500)]
Merge branch '14870-retry-logs' refs #14870

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Report errors from ImportKeyPair
Peter Amstutz [Thu, 7 Mar 2019 16:43:05 +0000 (11:43 -0500)]
14291: Report errors from ImportKeyPair

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Try again with vendor.json
Peter Amstutz [Thu, 7 Mar 2019 16:26:58 +0000 (11:26 -0500)]
14291: Try again with vendor.json

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14920: Improve comment.
Tom Clegg [Thu, 7 Mar 2019 16:20:38 +0000 (11:20 -0500)]
14920: Improve comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14291: Another update to vendor.json
Peter Amstutz [Thu, 7 Mar 2019 14:55:15 +0000 (09:55 -0500)]
14291: Another update to vendor.json

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agorefs #14891 .Merge branch '14891-upload-to-testing-repo'
Fernando Monserrat [Thu, 7 Mar 2019 14:52:25 +0000 (11:52 -0300)]
refs #14891 .Merge branch '14891-upload-to-testing-repo'
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years agoissue #14891 fix spaces and initialize RC
Fernando Monserrat [Thu, 7 Mar 2019 14:48:58 +0000 (11:48 -0300)]
issue #14891 fix spaces and initialize RC
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>

5 years ago13937: Simplifies volume counter stats assignments. Fixes test.
Lucas Di Pentima [Thu, 7 Mar 2019 14:42:43 +0000 (11:42 -0300)]
13937: Simplifies volume counter stats assignments. Fixes test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14920: Fix state==unknown during create/list race.
Tom Clegg [Thu, 7 Mar 2019 01:51:59 +0000 (20:51 -0500)]
14920: Fix state==unknown during create/list race.

Recognize a new/booting node by its "instance secret" tag instead of
relying on Create() to return the new instance's ID quickly.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago13937: Fixes metrics naming
Lucas Di Pentima [Wed, 6 Mar 2019 22:25:10 +0000 (19:25 -0300)]
13937: Fixes metrics naming

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13937: Simplifies arg passing on Start() funcs
Lucas Di Pentima [Wed, 6 Mar 2019 22:12:20 +0000 (19:12 -0300)]
13937: Simplifies arg passing on Start() funcs

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14291: Add more missing packages to vendor
Peter Amstutz [Wed, 6 Mar 2019 21:35:44 +0000 (16:35 -0500)]
14291: Add more missing packages to vendor

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Add more missing packages to vendor
Peter Amstutz [Wed, 6 Mar 2019 21:30:03 +0000 (16:30 -0500)]
14291: Add more missing packages to vendor

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Add missing packages to vendor
Peter Amstutz [Wed, 6 Mar 2019 21:25:15 +0000 (16:25 -0500)]
14291: Add missing packages to vendor

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14870: Oops need to fix edge case assertion, too
Peter Amstutz [Wed, 6 Mar 2019 21:16:59 +0000 (16:16 -0500)]
14870: Oops need to fix edge case assertion, too

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch 'master' into 13937-keepstore-prometheus
Lucas Di Pentima [Wed, 6 Mar 2019 21:09:28 +0000 (18:09 -0300)]
Merge branch 'master' into 13937-keepstore-prometheus

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14870: Adjust edge case some more, a file named '\' isn't legal either
Peter Amstutz [Wed, 6 Mar 2019 20:59:06 +0000 (15:59 -0500)]
14870: Adjust edge case some more, a file named '\' isn't legal either

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14291: Introduce "AddedScratch" and "IncludedScratch" to InstanceType
Peter Amstutz [Wed, 6 Mar 2019 20:39:44 +0000 (15:39 -0500)]
14291: Introduce "AddedScratch" and "IncludedScratch" to InstanceType

EC2 driver imports public keys on the fly.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14870: Remove \x01 from edge cases
Peter Amstutz [Wed, 6 Mar 2019 19:46:06 +0000 (14:46 -0500)]
14870: Remove \x01 from edge cases

It is no longer accepted by the API server, manifests with control
characters like that are invalid.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoMerge branch '14885-ciso-and-conda-packaging-pr'
Eric Biagiotti [Wed, 6 Mar 2019 19:43:29 +0000 (14:43 -0500)]
Merge branch '14885-ciso-and-conda-packaging-pr'

refs #14885

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '14916-rubysdk-upgrade-note'
Lucas Di Pentima [Wed, 6 Mar 2019 19:38:20 +0000 (16:38 -0300)]
Merge branch '14916-rubysdk-upgrade-note'
Closes #14916

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14916: Rephrases upgrade notes. Enhances example script with success message.
Lucas Di Pentima [Wed, 6 Mar 2019 17:44:11 +0000 (14:44 -0300)]
14916: Rephrases upgrade notes. Enhances example script with success message.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoissue #14891 remove -x
Fernando Monserrat [Wed, 6 Mar 2019 15:03:49 +0000 (12:03 -0300)]
issue #14891 remove -x
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat@veritasgenetics.com>