arvados.git
6 years ago14397: Use {sha1}-dev as version in dev builds of Go programs.
Tom Clegg [Mon, 12 Nov 2018 15:40:27 +0000 (10:40 -0500)]
14397: Use {sha1}-dev as version in dev builds of Go programs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14397: Fix unchecked errors in S3 block listing.
Tom Clegg [Mon, 12 Nov 2018 14:27:12 +0000 (09:27 -0500)]
14397: Fix unchecked errors in S3 block listing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14397: Fix S3 ListOps stats.
Tom Clegg [Mon, 12 Nov 2018 14:24:20 +0000 (09:24 -0500)]
14397: Fix S3 ListOps stats.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14397: Improve error reporting.
Tom Clegg [Fri, 9 Nov 2018 20:02:54 +0000 (15:02 -0500)]
14397: Improve error reporting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14397: Update goamz to fix v4 signature.
Tom Clegg [Fri, 9 Nov 2018 19:53:08 +0000 (14:53 -0500)]
14397: Update goamz to fix v4 signature.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14198: WIP handling various cases of remote users submitting container requests
Peter Amstutz [Fri, 9 Nov 2018 21:40:27 +0000 (16:40 -0500)]
14198: WIP handling various cases of remote users submitting container requests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14198-fed-collection-list' refs #14198
Peter Amstutz [Fri, 9 Nov 2018 20:50:53 +0000 (15:50 -0500)]
Merge branch '14198-fed-collection-list' refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14198: Expand federation support for collections and links
Peter Amstutz [Fri, 9 Nov 2018 19:24:00 +0000 (14:24 -0500)]
14198: Expand federation support for collections and links

Add collections and links to the list of endpoints where requests
bearing cluster_id are routed through controller to remote clusters.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14328-containerd'
Tom Clegg [Fri, 9 Nov 2018 19:00:54 +0000 (14:00 -0500)]
Merge branch '14328-containerd'

closes #14328

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14328: Merge branch 'master'
Tom Clegg [Fri, 9 Nov 2018 18:59:29 +0000 (13:59 -0500)]
14328: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14328: Fix flaky test.
Tom Clegg [Fri, 9 Nov 2018 18:55:00 +0000 (13:55 -0500)]
14328: Fix flaky test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14323-container-collection-mounts'
Lucas Di Pentima [Fri, 9 Nov 2018 16:44:55 +0000 (13:44 -0300)]
Merge branch '14323-container-collection-mounts'
Closes #14323

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14198-crunch-run-tokens' refs #14198
Peter Amstutz [Fri, 9 Nov 2018 16:37:59 +0000 (11:37 -0500)]
Merge branch '14198-crunch-run-tokens' refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14198: Add comment about different client objects used by runner
Peter Amstutz [Fri, 9 Nov 2018 16:36:01 +0000 (11:36 -0500)]
14198: Add comment about different client objects used by runner

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14259-pysdk-remote-block-copy'
Lucas Di Pentima [Fri, 9 Nov 2018 16:26:13 +0000 (13:26 -0300)]
Merge branch '14259-pysdk-remote-block-copy'
Closes #14259

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14323: Adds test for the empty collection mount resolution case.
Lucas Di Pentima [Fri, 9 Nov 2018 16:10:11 +0000 (13:10 -0300)]
14323: Adds test for the empty collection mount resolution case.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14198: Use container token for reading input/creating output
Peter Amstutz [Fri, 9 Nov 2018 02:32:17 +0000 (21:32 -0500)]
14198: Use container token for reading input/creating output

Refactor a bit to make it obvious which token is being used when
communicating with the API server.  Operations that could involve
federated collections (reading the Docker image, reading inputs,
copying remote blocks to local) must use the container token.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14328: Remove containerd watchdog.
Tom Clegg [Thu, 8 Nov 2018 22:05:22 +0000 (17:05 -0500)]
14328: Remove containerd watchdog.

No longer needed, now that we periodically ask docker-inspect to check
on the specific container we're running.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14345-webdav-lock-and-empty-dir'
Tom Clegg [Thu, 8 Nov 2018 21:57:19 +0000 (16:57 -0500)]
Merge branch '14345-webdav-lock-and-empty-dir'

refs #14345

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14323: Only get collection mount by UUID when PDH is missing and UUID provided.
Lucas Di Pentima [Thu, 8 Nov 2018 20:53:09 +0000 (17:53 -0300)]
14323: Only get collection mount by UUID when PDH is missing and UUID provided.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14259: Improvements on remote blocks copying logic.
Lucas Di Pentima [Thu, 8 Nov 2018 19:57:13 +0000 (16:57 -0300)]
14259: Improvements on remote blocks copying logic.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14458-controller-panic' refs #14458
Peter Amstutz [Thu, 8 Nov 2018 18:25:05 +0000 (13:25 -0500)]
Merge branch '14458-controller-panic' refs #14458

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14458: Fix syntax error
Peter Amstutz [Thu, 8 Nov 2018 18:21:53 +0000 (13:21 -0500)]
14458: Fix syntax error

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14458: Drain errorChan without waiting for close
Peter Amstutz [Thu, 8 Nov 2018 18:18:59 +0000 (13:18 -0500)]
14458: Drain errorChan without waiting for close

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14345: Handle "MOVE foo/ bar/" requests.
Tom Clegg [Tue, 6 Nov 2018 18:57:14 +0000 (13:57 -0500)]
14345: Handle "MOVE foo/ bar/" requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14345: Accept "." placeholder: create parent dir, but no fake file.
Tom Clegg [Tue, 6 Nov 2018 14:54:10 +0000 (09:54 -0500)]
14345: Accept "." placeholder: create parent dir, but no fake file.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14345: Accept lock/unlock requests as no-ops.
Tom Clegg [Thu, 18 Oct 2018 13:41:37 +0000 (09:41 -0400)]
14345: Accept lock/unlock requests as no-ops.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14345: Unescape all \ooo in names in manifests, not just \040.
Tom Clegg [Mon, 5 Nov 2018 21:46:50 +0000 (16:46 -0500)]
14345: Unescape all \ooo in names in manifests, not just \040.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago"arvbox status" includes cluster id refs #14198
Peter Amstutz [Thu, 8 Nov 2018 16:16:53 +0000 (11:16 -0500)]
"arvbox status" includes cluster id refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14458: Drain errors channel on failure case instead of accumulating separately
Peter Amstutz [Thu, 8 Nov 2018 15:55:47 +0000 (10:55 -0500)]
14458: Drain errors channel on failure case instead of accumulating separately

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14458: Avoid panic by removing defer close()
Peter Amstutz [Wed, 7 Nov 2018 20:53:08 +0000 (15:53 -0500)]
14458: Avoid panic by removing defer close()

Channels will be garbage collected when they go unreferenced.

Make the errors channel buffered to avoid lingering goroutines trying
to send on an unbuffered error channel with no reciever.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14198-index-accepts-cluster-id' refs #14198
Peter Amstutz [Thu, 8 Nov 2018 16:03:55 +0000 (11:03 -0500)]
Merge branch '14198-index-accepts-cluster-id' refs #14198

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14439-federation-admin-docs'
Lucas Di Pentima [Wed, 7 Nov 2018 21:11:32 +0000 (18:11 -0300)]
Merge branch '14439-federation-admin-docs'
Refs #14439

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14323: Fixes indentation.
Lucas Di Pentima [Wed, 7 Nov 2018 21:10:19 +0000 (18:10 -0300)]
14323: Fixes indentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14439: Adds federation configuration page to the admin docs.
Lucas Di Pentima [Wed, 7 Nov 2018 20:43:20 +0000 (17:43 -0300)]
14439: Adds federation configuration page to the admin docs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14198: Add cluster_id to index parameters
Peter Amstutz [Wed, 7 Nov 2018 19:38:45 +0000 (14:38 -0500)]
14198: Add cluster_id to index parameters

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14345: Use "." placeholder to persist empty directories.
Tom Clegg [Mon, 5 Nov 2018 21:15:15 +0000 (16:15 -0500)]
14345: Use "." placeholder to persist empty directories.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14323: Lookup collection mount by uuid when pdh is not given. Update tests.
Lucas Di Pentima [Wed, 7 Nov 2018 17:48:31 +0000 (14:48 -0300)]
14323: Lookup collection mount by uuid when pdh is not given. Update tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14323: Prioritize PDH over UUID for mount resolving when both provided.
Lucas Di Pentima [Wed, 7 Nov 2018 14:05:03 +0000 (11:05 -0300)]
14323: Prioritize PDH over UUID for mount resolving when both provided.

* Don't delete uuid after mount resolving.
* Update tests

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14455-loofah-upgrade'
Lucas Di Pentima [Tue, 6 Nov 2018 19:29:48 +0000 (16:29 -0300)]
Merge branch '14455-loofah-upgrade'
Closes #14455

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14406-remote-data-in-output'
Tom Clegg [Tue, 6 Nov 2018 19:26:56 +0000 (14:26 -0500)]
Merge branch '14406-remote-data-in-output'

refs #14406

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14406: Merge branch 'master'
Tom Clegg [Tue, 6 Nov 2018 18:58:53 +0000 (13:58 -0500)]
14406: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14455: Update loofah & nokogiri on workbench & api for security reasons.
Lucas Di Pentima [Tue, 6 Nov 2018 18:41:07 +0000 (15:41 -0300)]
14455: Update loofah & nokogiri on workbench & api for security reasons.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14406: Only call LocalLocator() once per block per stream.
Tom Clegg [Tue, 6 Nov 2018 16:13:05 +0000 (11:13 -0500)]
14406: Only call LocalLocator() once per block per stream.

Avoid excess calls when many files are packed in a single block.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch 'master' into 14259-pysdk-remote-block-copy
Lucas Di Pentima [Tue, 6 Nov 2018 14:41:06 +0000 (11:41 -0300)]
Merge branch 'master' into 14259-pysdk-remote-block-copy

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14259: Don't scan the entire collection for remote blocks when not needed.
Lucas Di Pentima [Tue, 6 Nov 2018 14:39:14 +0000 (11:39 -0300)]
14259: Don't scan the entire collection for remote blocks when not needed.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14406: Copy remote blocks to local storage during MarshalManifest.
Tom Clegg [Mon, 5 Nov 2018 14:58:40 +0000 (09:58 -0500)]
14406: Copy remote blocks to local storage during MarshalManifest.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14262-remote-container' refs #14262
Peter Amstutz [Thu, 1 Nov 2018 19:57:09 +0000 (15:57 -0400)]
Merge branch '14262-remote-container' refs #14262

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14406: Tone down slow stress test.
Tom Clegg [Thu, 1 Nov 2018 19:22:36 +0000 (15:22 -0400)]
14406: Tone down slow stress test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14262: Avoid out-of-bounds panics checking cluster prefixes
Peter Amstutz [Thu, 1 Nov 2018 19:14:14 +0000 (15:14 -0400)]
14262: Avoid out-of-bounds panics checking cluster prefixes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14406: Remove debug printfs.
Tom Clegg [Thu, 1 Nov 2018 18:54:13 +0000 (14:54 -0400)]
14406: Remove debug printfs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14262: Fix remote_user_test.rb so it doesn't mess up remote_hosts
Peter Amstutz [Thu, 1 Nov 2018 18:53:37 +0000 (14:53 -0400)]
14262: Fix remote_user_test.rb so it doesn't mess up remote_hosts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14406: Fix safety warning.
Tom Clegg [Thu, 1 Nov 2018 17:56:59 +0000 (13:56 -0400)]
14406: Fix safety warning.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14406: Pass output through collectionFS if it has remote block refs.
Tom Clegg [Thu, 1 Nov 2018 17:56:13 +0000 (13:56 -0400)]
14406: Pass output through collectionFS if it has remote block refs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14262: Revert changes to client.go
Peter Amstutz [Thu, 1 Nov 2018 15:04:01 +0000 (11:04 -0400)]
14262: Revert changes to client.go

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Move the context deadline to the top of the handler stack
Peter Amstutz [Thu, 1 Nov 2018 14:19:18 +0000 (10:19 -0400)]
14262: Move the context deadline to the top of the handler stack

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Fix permissions so runtime_token can set container progress/output
Peter Amstutz [Wed, 31 Oct 2018 20:40:19 +0000 (16:40 -0400)]
14262: Fix permissions so runtime_token can set container progress/output

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Only create runtime_token on home cluster for the authorization
Peter Amstutz [Wed, 31 Oct 2018 17:45:58 +0000 (13:45 -0400)]
14262: Only create runtime_token on home cluster for the authorization

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14259: Iterate just once when scanning for remote blocks.
Lucas Di Pentima [Wed, 31 Oct 2018 18:43:46 +0000 (15:43 -0300)]
14259: Iterate just once when scanning for remote blocks.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14259: Don't enforce bandwidth timeouts on keep client HEAD requests.
Lucas Di Pentima [Wed, 31 Oct 2018 18:16:22 +0000 (15:16 -0300)]
14259: Don't enforce bandwidth timeouts on keep client HEAD requests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14259: Test updates & additions related to keep client HEAD requests
Lucas Di Pentima [Wed, 31 Oct 2018 18:12:14 +0000 (15:12 -0300)]
14259: Test updates & additions related to keep client HEAD requests

* Test updates to confirm that when doing HEAD requests, no bandwidth timeout
are enforced.
* Test additions to prove that HEAD request responses don't get cached.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14262: Fix spillover of tests changing remote_hosts
Peter Amstutz [Wed, 31 Oct 2018 16:53:25 +0000 (12:53 -0400)]
14262: Fix spillover of tests changing remote_hosts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Rewrite collectionFederatedRequestHandler PDH search to use channels
Peter Amstutz [Mon, 29 Oct 2018 21:09:08 +0000 (17:09 -0400)]
14262: Rewrite collectionFederatedRequestHandler PDH search to use channels

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Make sure cancel() from proxy.Do() gets called
Peter Amstutz [Mon, 29 Oct 2018 19:36:45 +0000 (15:36 -0400)]
14262: Make sure cancel() from proxy.Do() gets called

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Only allow unknown PDH for images when there are remote_hosts
Peter Amstutz [Mon, 29 Oct 2018 18:56:26 +0000 (14:56 -0400)]
14262: Only allow unknown PDH for images when there are remote_hosts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Fix logic for when to allow unknown PDH for containers
Peter Amstutz [Wed, 24 Oct 2018 19:30:34 +0000 (15:30 -0400)]
14262: Fix logic for when to allow unknown PDH for containers

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14198: Fix error responses in container POST
Peter Amstutz [Wed, 24 Oct 2018 18:44:25 +0000 (14:44 -0400)]
14198: Fix error responses in container POST

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Handle container_request posted as a string parameter
Peter Amstutz [Tue, 23 Oct 2018 18:30:39 +0000 (14:30 -0400)]
14262: Handle container_request posted as a string parameter

Needs to be parsed as json second time (this is how the Ruby 'arv'
client submits it, unfortunately.)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Fix crunch-run tests
Peter Amstutz [Tue, 23 Oct 2018 17:29:53 +0000 (13:29 -0400)]
14262: Fix crunch-run tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Missing file
Peter Amstutz [Tue, 23 Oct 2018 13:26:48 +0000 (09:26 -0400)]
14262: Missing file

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Fix tests
Peter Amstutz [Mon, 22 Oct 2018 21:01:52 +0000 (17:01 -0400)]
14262: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Tests for setting and checking container tokens.
Peter Amstutz [Mon, 22 Oct 2018 16:13:38 +0000 (12:13 -0400)]
14262: Tests for setting and checking container tokens.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Add createAPIToken, with test
Peter Amstutz [Mon, 22 Oct 2018 15:02:02 +0000 (11:02 -0400)]
14262: Add createAPIToken, with test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Refactoring, split up federation code into smaller files
Peter Amstutz [Thu, 18 Oct 2018 21:17:33 +0000 (17:17 -0400)]
14262: Refactoring, split up federation code into smaller files

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Refactoring proxy
Peter Amstutz [Thu, 18 Oct 2018 20:08:28 +0000 (16:08 -0400)]
14262: Refactoring proxy

Split proxy.Do() into ForwardRequest() and ForwardResponse().

Inversion of control eliminates need for "filter" callback, since the
caller can now modify the response in between the calls to
ForwardRequest() and ForwardResponse().

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Fix bug moving api_token to header
Peter Amstutz [Thu, 18 Oct 2018 18:34:49 +0000 (14:34 -0400)]
14262: Fix bug moving api_token to header

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: Use container token for access to load Docker image
Peter Amstutz [Tue, 16 Oct 2018 19:43:12 +0000 (15:43 -0400)]
14262: Use container token for access to load Docker image

Previously used Dispatcher token, which created a security race
condition (you couldn't set a container image that you didn't have
access to, but if your access was revoked it the meantime, the
container would still run.)

Also tweaked API server to allow a PDH for the container image spec
with no further checking (so the API server doesn't have to go out and
search the federation.)  This is no longer a security hazard since it
is now using a user token and not the dispatcher token.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14262: saltAuthToken returns copy of request object
Peter Amstutz [Tue, 16 Oct 2018 18:51:12 +0000 (14:51 -0400)]
14262: saltAuthToken returns copy of request object

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14259-pysdk-remote-block-copy'
Lucas Di Pentima [Tue, 30 Oct 2018 18:00:44 +0000 (15:00 -0300)]
Merge branch '14259-pysdk-remote-block-copy'
Refs #14259

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14259: Collection class copies remote blocks when saving.
Lucas Di Pentima [Tue, 30 Oct 2018 12:19:49 +0000 (09:19 -0300)]
14259: Collection class copies remote blocks when saving.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMake sure keep-balance gets started with the lowest possible system priority by
Ward Vandewege [Mon, 29 Oct 2018 19:56:34 +0000 (15:56 -0400)]
Make sure keep-balance gets started with the lowest possible system priority by
default.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoFix typos.
Ward Vandewege [Fri, 26 Oct 2018 20:08:05 +0000 (16:08 -0400)]
Fix typos.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd a new release note for v1.2.0.
Ward Vandewege [Fri, 26 Oct 2018 19:51:44 +0000 (15:51 -0400)]
Add a new release note for v1.2.0.

Reorganize release note page to reflect stable version releases.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '14328-watch-docker-ps'
Tom Clegg [Fri, 26 Oct 2018 17:14:38 +0000 (13:14 -0400)]
Merge branch '14328-watch-docker-ps'

refs #14328

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMake geckodriver sanity check non-fatal.
Tom Clegg [Fri, 26 Oct 2018 00:54:00 +0000 (20:54 -0400)]
Make geckodriver sanity check non-fatal.

Allows non-workbench test runs to pass without geckodriver.

refs #14407

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix crash in test cleanup.
Tom Clegg [Fri, 26 Oct 2018 00:49:48 +0000 (20:49 -0400)]
Fix crash in test cleanup.

refs #14407

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14398-error-deadlock'
Tom Clegg [Thu, 25 Oct 2018 21:46:54 +0000 (17:46 -0400)]
Merge branch '14398-error-deadlock'

fixes #14398

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14328: Check ContainerInspect instead of List, give up on error.
Tom Clegg [Thu, 25 Oct 2018 21:44:14 +0000 (17:44 -0400)]
14328: Check ContainerInspect instead of List, give up on error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14259: Move & improve test.
Lucas Di Pentima [Thu, 25 Oct 2018 21:33:53 +0000 (18:33 -0300)]
14259: Move & improve test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14407-selenium'
Tom Clegg [Thu, 25 Oct 2018 21:12:36 +0000 (17:12 -0400)]
Merge branch '14407-selenium'

refs #14407

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14388-overreplication'
Tom Clegg [Thu, 25 Oct 2018 20:00:09 +0000 (16:00 -0400)]
Merge branch '14388-overreplication'

fixes #14388

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14407: Add geckodriver to arvbox image.
Tom Clegg [Thu, 25 Oct 2018 19:56:43 +0000 (15:56 -0400)]
14407: Add geckodriver to arvbox image.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14407: Fix "too much recursion" error.
Tom Clegg [Thu, 25 Oct 2018 19:52:21 +0000 (15:52 -0400)]
14407: Fix "too much recursion" error.

The recursion error comes from evaluate_script() trying to serialize
the magical jQuery response. We don't need to see that response so
we can use execute_script() instead.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14407: Upgrade selenium-webdriver, add geckodriver.
Tom Clegg [Thu, 25 Oct 2018 19:24:28 +0000 (15:24 -0400)]
14407: Upgrade selenium-webdriver, add geckodriver.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14398: Fix deadlock on error in GetCurrentState.
Tom Clegg [Wed, 24 Oct 2018 19:59:31 +0000 (15:59 -0400)]
14398: Fix deadlock on error in GetCurrentState.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14388: Merge branch 'master' into 14388-overreplication
Tom Clegg [Wed, 24 Oct 2018 17:03:20 +0000 (13:03 -0400)]
14388: Merge branch 'master' into 14388-overreplication

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14299-keep-balance-past-versions'
Lucas Di Pentima [Wed, 24 Oct 2018 16:16:34 +0000 (13:16 -0300)]
Merge branch '14299-keep-balance-past-versions'
Closes #14299

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14299: Use include_old_versions in keep-balance to avoid losing blocks.
Lucas Di Pentima [Tue, 23 Oct 2018 22:19:07 +0000 (19:19 -0300)]
14299: Use include_old_versions in keep-balance to avoid losing blocks.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14388: Don't prefer positions that are only temporarily untrashable.
Tom Clegg [Tue, 23 Oct 2018 18:30:42 +0000 (14:30 -0400)]
14388: Don't prefer positions that are only temporarily untrashable.

If old copies exist in the preferred positions, and a new copy is
written to a worse position, keep the old copies and wait for the new
copy to become eligible for trash.

Previously, an old copy would be deleted. This would bring replication
down to the desired level in the short term, but would also cause
extra pulling/copying in the future to restore the optimal copy, and
introduce extra probes for all clients in the meantime.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14388: Fix typo.
Tom Clegg [Tue, 23 Oct 2018 18:28:39 +0000 (14:28 -0400)]
14388: Fix typo.

Condition (si.want != si.want) should have been (si.want != sj.want).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>