arvados.git
6 years ago10181: Update container log PDH after updating log collection.
Tom Clegg [Fri, 21 Sep 2018 19:30:05 +0000 (15:30 -0400)]
10181: Update container log PDH after updating log collection.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Permit dispatcher to update log while container is running.
Tom Clegg [Thu, 20 Sep 2018 17:52:47 +0000 (13:52 -0400)]
10181: Permit dispatcher to update log while container is running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Save log collection snapshots periodically during run.
Tom Clegg [Thu, 20 Sep 2018 05:04:52 +0000 (01:04 -0400)]
10181: Save log collection snapshots periodically during run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago10181: Add Size method to arvados.CollectionFileSystem.
Tom Clegg [Thu, 20 Sep 2018 04:59:35 +0000 (00:59 -0400)]
10181: Add Size method to arvados.CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13752-migrate-index-data'
Tom Clegg [Tue, 18 Sep 2018 19:47:43 +0000 (15:47 -0400)]
Merge branch '13752-migrate-index-data'

refs #13752

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: Merge branch 'master' into 13752-migrate-index-data
Tom Clegg [Tue, 18 Sep 2018 19:43:28 +0000 (15:43 -0400)]
13752: Merge branch 'master' into 13752-migrate-index-data

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13994-proxy-remote'
Tom Clegg [Tue, 18 Sep 2018 19:42:06 +0000 (15:42 -0400)]
Merge branch '13994-proxy-remote'

refs #13994

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Merge branch 'master' into 13994-proxy-remote
Tom Clegg [Tue, 18 Sep 2018 19:40:26 +0000 (15:40 -0400)]
13994: Merge branch 'master' into 13994-proxy-remote

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: Migrate file_names column using multiple transactions.
Tom Clegg [Mon, 17 Sep 2018 21:50:13 +0000 (17:50 -0400)]
13752: Migrate file_names column using multiple transactions.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13752: De-duplicate file and stream names in collections.file_names.
Tom Clegg [Mon, 17 Sep 2018 21:49:21 +0000 (17:49 -0400)]
13752: De-duplicate file and stream names in collections.file_names.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Fix missing error message.
Tom Clegg [Mon, 17 Sep 2018 18:34:05 +0000 (14:34 -0400)]
13994: Fix missing error message.

When the request failed too early to produce an HTTP status code, the
error returned to the client was:

"Could not write sufficient replicas: [0] ; [0] "

Now the error message is included. It gets truncated to 100 chars
later on, but it's something:

"Could not write sufficient replicas: [0] Put
http://localhost:52367/831f95ffae4233b52059fbb8a0c62fbf: dial tcp [::1
]:52367: connect: conn; [0] Put
http://localhost:53807/831f95ffae4233b52059fbb8a0c62fbf: dial tcp
[::1]:53807: connect: conn"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Fix service discovery race.
Tom Clegg [Mon, 17 Sep 2018 18:33:59 +0000 (14:33 -0400)]
13994: Fix service discovery race.

Making keepclient.RefreshServiceDiscovery asynchronous didn't work
out: tests rely on it to ensure old services won't be used.

Instead, speed things up a bit by refreshing all known clients'
service lists concurrently instead of serially.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14087-federated-collection-by-pdh' refs #14087
Peter Amstutz [Mon, 17 Sep 2018 15:58:37 +0000 (11:58 -0400)]
Merge branch '14087-federated-collection-by-pdh' refs #14087

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14203-crunch-run-mkdir' closes #14203
Peter Amstutz [Mon, 17 Sep 2018 15:58:22 +0000 (11:58 -0400)]
Merge branch '14203-crunch-run-mkdir' closes #14203

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Add Accept-Encoding to list of headers to delete when proxying
Peter Amstutz [Mon, 17 Sep 2018 15:54:35 +0000 (11:54 -0400)]
14087: Add Accept-Encoding to list of headers to delete when proxying

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13994: Fix token handling in collection performance test.
Tom Clegg [Mon, 17 Sep 2018 14:34:43 +0000 (10:34 -0400)]
13994: Fix token handling in collection performance test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Use entire token for blob signatures.
Tom Clegg [Mon, 17 Sep 2018 14:07:35 +0000 (10:07 -0400)]
13994: Use entire token for blob signatures.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14203: Tweak error messages.
Peter Amstutz [Fri, 14 Sep 2018 15:39:26 +0000 (11:39 -0400)]
14203: Tweak error messages.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Refactor federation search for collections by PDH
Peter Amstutz [Fri, 14 Sep 2018 15:27:11 +0000 (11:27 -0400)]
14087: Refactor federation search for collections by PDH

The shorter, non-PDH search case now comes first.  Also adjusts the
logic that decides when to skip federation and makes it wrap the
federation endpoint routing.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13773-will-fail-container-status'
Lucas Di Pentima [Thu, 13 Sep 2018 20:46:02 +0000 (17:46 -0300)]
Merge branch '13773-will-fail-container-status'
Closes #13773

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'master' into 13773-will-fail-container-status
Lucas Di Pentima [Thu, 13 Sep 2018 20:05:48 +0000 (17:05 -0300)]
Merge branch 'master' into 13773-will-fail-container-status

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Update logging level checking for runtime status updates.
Lucas Di Pentima [Thu, 13 Sep 2018 19:30:35 +0000 (16:30 -0300)]
13773: Update logging level checking for runtime status updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13994: Fix import cycle.
Tom Clegg [Thu, 13 Sep 2018 18:43:28 +0000 (14:43 -0400)]
13994: Fix import cycle.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13773: Fix test
Lucas Di Pentima [Wed, 12 Sep 2018 20:58:00 +0000 (17:58 -0300)]
13773: Fix test

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Update test that confirms runtime_status is updated when a child fails
Lucas Di Pentima [Wed, 12 Sep 2018 20:14:41 +0000 (17:14 -0300)]
13773: Update test that confirms runtime_status is updated when a child fails

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13994: Fix deadlock in keepstore tests.
Tom Clegg [Tue, 11 Sep 2018 16:55:05 +0000 (12:55 -0400)]
13994: Fix deadlock in keepstore tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13994: Proxy to remote cluster if +R hint given.
Tom Clegg [Tue, 11 Sep 2018 16:54:41 +0000 (12:54 -0400)]
13994: Proxy to remote cluster if +R hint given.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14203: Mkdir failing with os.ErrExist is not a fatal error
Peter Amstutz [Wed, 12 Sep 2018 19:37:33 +0000 (15:37 -0400)]
14203: Mkdir failing with os.ErrExist is not a fatal error

Don't fail CaptureOutput and cancel the container if a Mkdir returns
os.ErrExist because a directory already exists.  Fixes bcbio failing
postprocess_variants step.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Removed superfluous documentation
Lucas Di Pentima [Wed, 12 Sep 2018 18:41:42 +0000 (15:41 -0300)]
13773: Removed superfluous documentation

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Add well known runtime_status' keys data type validation.
Lucas Di Pentima [Wed, 12 Sep 2018 12:13:35 +0000 (09:13 -0300)]
13773: Add well known runtime_status' keys data type validation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13993-federated-collection' refs #13993
Peter Amstutz [Tue, 11 Sep 2018 15:40:10 +0000 (11:40 -0400)]
Merge branch '13993-federated-collection' refs #13993

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Rate limit the number of parallel requests
Peter Amstutz [Tue, 11 Sep 2018 13:52:27 +0000 (09:52 -0400)]
14087: Rate limit the number of parallel requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Remove debug print. Comments.
Peter Amstutz [Mon, 10 Sep 2018 21:03:19 +0000 (17:03 -0400)]
14087: Remove debug print.  Comments.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Check for remote clusters before searching
Peter Amstutz [Mon, 10 Sep 2018 20:49:18 +0000 (16:49 -0400)]
14087: Check for remote clusters before searching

Added some more comments about the implementation as well.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Fetch federated collection by PDH
Peter Amstutz [Mon, 10 Sep 2018 19:28:35 +0000 (15:28 -0400)]
14087: Fetch federated collection by PDH

Implement parallel query of remotes.  First response is returned to
client, other responses in flight are cancelled.  Adds tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14087: Federated fetch by PDH WIP
Peter Amstutz [Fri, 7 Sep 2018 22:14:26 +0000 (18:14 -0400)]
14087: Federated fetch by PDH WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13993: Add TestGetLocalCollection
Peter Amstutz [Tue, 11 Sep 2018 15:05:34 +0000 (11:05 -0400)]
13993: Add TestGetLocalCollection

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Enhance error/warning runtime status display on workbench.
Lucas Di Pentima [Tue, 11 Sep 2018 14:44:48 +0000 (11:44 -0300)]
13773: Enhance error/warning runtime status display on workbench.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Don't intercept logger.info() calls for runtime status.
Lucas Di Pentima [Tue, 11 Sep 2018 14:42:37 +0000 (11:42 -0300)]
13773: Don't intercept logger.info() calls for runtime status.

* On multi line log messages, use the first as a status and the rest as detail.
* Revert done.logtail() usage as it's no longer needed for failed childs
reporting.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13993: Need to explicitly handle paths that don't end in /
Peter Amstutz [Mon, 10 Sep 2018 21:04:37 +0000 (17:04 -0400)]
13993: Need to explicitly handle paths that don't end in /

Otherwise Go will generate a 301 Moved Permanently from collection to collections/

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13993: Don't export internal types
Peter Amstutz [Mon, 10 Sep 2018 20:20:32 +0000 (16:20 -0400)]
13993: Don't export internal types

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13993: Consolidate SignedLocatorPattern with keepclient.SignedLocatorRe
Peter Amstutz [Mon, 10 Sep 2018 20:18:01 +0000 (16:18 -0400)]
13993: Consolidate SignedLocatorPattern with keepclient.SignedLocatorRe

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Updates documentation
Lucas Di Pentima [Mon, 10 Sep 2018 17:48:38 +0000 (14:48 -0300)]
13773: Updates documentation

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Intercept logging calls to update runtime_status on runner containers.
Lucas Di Pentima [Fri, 7 Sep 2018 20:57:21 +0000 (17:57 -0300)]
13773: Intercept logging calls to update runtime_status on runner containers.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13993: Support for federated collection requests by uuid
Peter Amstutz [Fri, 7 Sep 2018 19:52:52 +0000 (15:52 -0400)]
13993: Support for federated collection requests by uuid

Do remote record retrevial using a salted token, update the block
signatures to "remote" signatures (changing +A to +R and adding the
remote cluster id), and return it to the original client.

Refactor routing and proxying of federated requeststo provide hooks to
support this and future types of requests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Update runtime_status_error() method to report warning & activity status
Lucas Di Pentima [Fri, 7 Sep 2018 17:04:02 +0000 (14:04 -0300)]
13773: Update runtime_status_error() method to report warning & activity status

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13146: Returns real json response
Peter Amstutz [Thu, 2 Aug 2018 20:09:52 +0000 (16:09 -0400)]
13146: Returns real json response

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13146: Projects shared with me WIP
Peter Amstutz [Thu, 2 Aug 2018 19:35:17 +0000 (15:35 -0400)]
13146: Projects shared with me WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Show 'Warning' label on jobs that have this runtime status and aren't failing.
Lucas Di Pentima [Thu, 6 Sep 2018 14:51:40 +0000 (11:51 -0300)]
13773: Show 'Warning' label on jobs that have this runtime status and aren't failing.

* Show warning information panel on job detailed view
* Show tooltip with error/warning information on dashboard
* Fix test

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoA new version of our libcloud fork is available.
Ward Vandewege [Thu, 6 Sep 2018 14:46:10 +0000 (10:46 -0400)]
A new version of our libcloud fork is available.

refs #14150

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '11065-optional-audit-logging'
Tom Clegg [Thu, 6 Sep 2018 14:37:42 +0000 (10:37 -0400)]
Merge branch '11065-optional-audit-logging'

refs #11065

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11065: Merge branch 'master' into 11065-optional-audit-logging
Tom Clegg [Thu, 6 Sep 2018 14:29:25 +0000 (10:29 -0400)]
11065: Merge branch 'master' into 11065-optional-audit-logging

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoAdd http_timeout to runtimeContext refs #13542
Peter Amstutz [Wed, 5 Sep 2018 21:37:57 +0000 (17:37 -0400)]
Add http_timeout to runtimeContext refs #13542

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14082-round-off' refs #14082
Peter Amstutz [Wed, 5 Sep 2018 20:22:31 +0000 (16:22 -0400)]
Merge branch '14082-round-off' refs #14082

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13973-max-cr-priority' refs #13973
Peter Amstutz [Wed, 5 Sep 2018 20:21:39 +0000 (16:21 -0400)]
Merge branch '13973-max-cr-priority' refs #13973

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix gofmt in slurm.go refs #14110
Peter Amstutz [Wed, 5 Sep 2018 19:44:46 +0000 (15:44 -0400)]
Fix gofmt in slurm.go refs #14110

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13773: Containers & CRs show "Failing" when a child job failed.
Lucas Di Pentima [Wed, 5 Sep 2018 19:41:29 +0000 (16:41 -0300)]
13773: Containers & CRs show "Failing" when a child job failed.

Also, at the detailed view an error panel is displayed with the error message
and detail, if any.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'wtsi/13809-root_url-protocol-port-configuration'
Peter Amstutz [Wed, 5 Sep 2018 18:56:46 +0000 (14:56 -0400)]
Merge branch 'wtsi/13809-root_url-protocol-port-configuration'

refs #13809

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'wtsi/python-api-timeout' refs #13542
Peter Amstutz [Wed, 5 Sep 2018 18:53:17 +0000 (14:53 -0400)]
Merge branch 'wtsi/python-api-timeout' refs #13542

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'wtsi/14110-c-d-s-limit-slurm-concurrency' refs #14110
Peter Amstutz [Wed, 5 Sep 2018 18:47:19 +0000 (14:47 -0400)]
Merge branch 'wtsi/14110-c-d-s-limit-slurm-concurrency' refs #14110

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13973: Use a straightforward maximum() query to compute max priority
Peter Amstutz [Wed, 5 Sep 2018 18:16:28 +0000 (14:16 -0400)]
13973: Use a straightforward maximum() query to compute max priority

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14082: Round up numeric values in container request that must be integers
Peter Amstutz [Wed, 5 Sep 2018 17:57:04 +0000 (13:57 -0400)]
14082: Round up numeric values in container request that must be integers

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14152-s3-v4-signature'
Tom Clegg [Tue, 4 Sep 2018 19:45:45 +0000 (15:45 -0400)]
Merge branch '14152-s3-v4-signature'

refs #14152

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14008-containers-index'
Tom Clegg [Tue, 4 Sep 2018 19:37:57 +0000 (15:37 -0400)]
Merge branch '14008-containers-index'

refs #14008

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoAdd test case for filtering on is_trashed in groups#contents.
Tom Clegg [Tue, 4 Sep 2018 15:25:51 +0000 (11:25 -0400)]
Add test case for filtering on is_trashed in groups#contents.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoRemove redundant test case.
Tom Clegg [Tue, 4 Sep 2018 15:25:18 +0000 (11:25 -0400)]
Remove redundant test case.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14008: Merge branch 'master' into 14008-containers-index
Tom Clegg [Tue, 4 Sep 2018 19:03:58 +0000 (15:03 -0400)]
14008: Merge branch 'master' into 14008-containers-index

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14152: Use V4 signer for S3 in regions that use V4 for EC2.
Tom Clegg [Tue, 4 Sep 2018 18:00:04 +0000 (14:00 -0400)]
14152: Use V4 signer for S3 in regions that use V4 for EC2.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14009-container-reuse-index'
Tom Clegg [Tue, 4 Sep 2018 15:36:15 +0000 (11:36 -0400)]
Merge branch '14009-container-reuse-index'

refs #14009

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13773: Updates tests checking that runtime_status_error() is called when needed
Lucas Di Pentima [Thu, 23 Aug 2018 20:28:53 +0000 (17:28 -0300)]
13773: Updates tests checking that runtime_status_error() is called when needed

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Updates runner container's runtime_status on child failures.
Lucas Di Pentima [Thu, 23 Aug 2018 15:04:56 +0000 (12:04 -0300)]
13773: Updates runner container's runtime_status on child failures.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Add runtime_status to be accesible via API
Lucas Di Pentima [Tue, 21 Aug 2018 15:10:58 +0000 (12:10 -0300)]
13773: Add runtime_status to be accesible via API

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Expand test to prove that non-error runtime_status doesn't avoid reuse.
Lucas Di Pentima [Tue, 14 Aug 2018 15:36:39 +0000 (12:36 -0300)]
13773: Expand test to prove that non-error runtime_status doesn't avoid reuse.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Fix reuse query & add tests
Lucas Di Pentima [Tue, 14 Aug 2018 14:25:10 +0000 (11:25 -0300)]
13773: Fix reuse query & add tests

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Add documentation.
Lucas Di Pentima [Mon, 6 Aug 2018 17:45:43 +0000 (14:45 -0300)]
13773: Add documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Filter out running containers that will fail from reuse selection.
Lucas Di Pentima [Thu, 2 Aug 2018 18:51:01 +0000 (15:51 -0300)]
13773: Filter out running containers that will fail from reuse selection.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13773: Adds runtime_status field to containers on API server
Lucas Di Pentima [Tue, 31 Jul 2018 13:38:56 +0000 (10:38 -0300)]
13773: Adds runtime_status field to containers on API server

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14153: Merge branch '14153-support-ruby-2.3.7' 1.2.0
Ward Vandewege [Tue, 4 Sep 2018 02:21:14 +0000 (22:21 -0400)]
14153: Merge branch '14153-support-ruby-2.3.7'

closes #14153

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoChange dependency from google-api-client to cure-google-api-client so
Ward Vandewege [Mon, 3 Sep 2018 19:53:27 +0000 (15:53 -0400)]
Change dependency from google-api-client to cure-google-api-client so
that we can use ruby 2.3.7

refs #14153

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMake sure to bump the dependency on the arvados gems to the 1.2 series.
Ward Vandewege [Tue, 4 Sep 2018 02:15:56 +0000 (22:15 -0400)]
Make sure to bump the dependency on the arvados gems to the 1.2 series.

refs #14032

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoDocumentation fix: make the use of $uuid_prefix consistent.
Ward Vandewege [Mon, 3 Sep 2018 18:37:44 +0000 (14:37 -0400)]
Documentation fix: make the use of $uuid_prefix consistent.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoRevert ruamel.yaml build to 0.14.12 refs #14075
Peter Amstutz [Sat, 25 Aug 2018 14:53:32 +0000 (10:53 -0400)]
Revert ruamel.yaml build to 0.14.12 refs #14075

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix test for changed expectation refs #13973
Peter Amstutz [Sat, 25 Aug 2018 13:19:15 +0000 (09:19 -0400)]
Fix test for changed expectation refs #13973

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago'trollop' gem renamed to 'optimist' fix build no issue #
Peter Amstutz [Sat, 25 Aug 2018 00:37:41 +0000 (20:37 -0400)]
'trollop' gem renamed to 'optimist' fix build no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoBuild ruamel.yaml 0.15.26 for a-c-r
Peter Amstutz [Fri, 24 Aug 2018 21:27:35 +0000 (17:27 -0400)]
Build ruamel.yaml 0.15.26 for a-c-r

This is last version before a serious regression was introduced in
parsing flow mapping

https://bitbucket.org/ruamel/yaml/issues/227/regression-parsing-flow-mapping

This affects 0.15.27 to 0.15.51

0.15.52 haven't been tested because they have a different
compatibility problem:

https://github.com/common-workflow-language/schema_salad/pull/195

refs #14075

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13456-controller-timeout'
Tom Clegg [Fri, 24 Aug 2018 19:45:08 +0000 (15:45 -0400)]
Merge branch '13456-controller-timeout'

refs #13456

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13456: Use HTTPRequestTimeout in test suite.
Tom Clegg [Thu, 23 Aug 2018 18:32:03 +0000 (14:32 -0400)]
13456: Use HTTPRequestTimeout in test suite.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13973-child-priority' refs #13973
Peter Amstutz [Fri, 24 Aug 2018 19:33:56 +0000 (15:33 -0400)]
Merge branch '13973-child-priority' refs #13973

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14075-uploadfiles' refs #14075
Peter Amstutz [Fri, 24 Aug 2018 19:33:05 +0000 (15:33 -0400)]
Merge branch '14075-uploadfiles' refs #14075

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'master' into 14075-uploadfiles
Peter Amstutz [Fri, 24 Aug 2018 19:28:16 +0000 (15:28 -0400)]
Merge branch 'master' into 14075-uploadfiles

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14075: Tweak ruamel.yaml verision range in setup.py
Peter Amstutz [Fri, 24 Aug 2018 17:51:56 +0000 (13:51 -0400)]
14075: Tweak ruamel.yaml verision range in setup.py

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14075: Fix cwl unit tests
Peter Amstutz [Fri, 24 Aug 2018 16:12:47 +0000 (12:12 -0400)]
14075: Fix cwl unit tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agofix slow queries in uploadfiles
Joshua C. Randall [Mon, 20 Aug 2018 16:31:29 +0000 (17:31 +0100)]
fix slow queries in uploadfiles

change uploadfiles to include pdh in name of collection rather then using ensure_unique_names=True
also don't bother listing existing empty collections

Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>

6 years agoMerge branch '13959-timeouts-and-logging'
Tom Clegg [Fri, 24 Aug 2018 16:06:56 +0000 (12:06 -0400)]
Merge branch '13959-timeouts-and-logging'

fixes #13959

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13959: Merge branch 'master' into 13959-timeouts-and-logging
Tom Clegg [Fri, 24 Aug 2018 16:06:18 +0000 (12:06 -0400)]
13959: Merge branch 'master' into 13959-timeouts-and-logging

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14008: Merge branch 'master' into 14008-containers-index
Tom Clegg [Fri, 24 Aug 2018 16:01:50 +0000 (12:01 -0400)]
14008: Merge branch 'master' into 14008-containers-index

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14008: Add container queue index.
Tom Clegg [Fri, 24 Aug 2018 15:57:07 +0000 (11:57 -0400)]
14008: Add container queue index.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14009: Index containers table for reuse searches.
Tom Clegg [Fri, 24 Aug 2018 15:36:10 +0000 (11:36 -0400)]
14009: Index containers table for reuse searches.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13973: Tweak test for inheriting CR priority
Peter Amstutz [Fri, 24 Aug 2018 14:13:44 +0000 (10:13 -0400)]
13973: Tweak test for inheriting CR priority

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoWe need to build ws 0.4.2 for ubuntu1404 and ubuntu1604
Ward Vandewege [Fri, 24 Aug 2018 05:50:41 +0000 (01:50 -0400)]
We need to build ws 0.4.2 for ubuntu1404 and ubuntu1604

refs #13823

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>