arvados.git
6 years agoMerge branch '12167-keep-request-id'
Tom Clegg [Thu, 30 Nov 2017 14:38:29 +0000 (09:38 -0500)]
Merge branch '12167-keep-request-id'

refs #12167

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix --on-error formatting
Tom Morris [Wed, 29 Nov 2017 22:14:04 +0000 (17:14 -0500)]
Fix --on-error formatting

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

6 years agoMerge branch '12018-tool-docs'
Lucas Di Pentima [Wed, 29 Nov 2017 21:59:44 +0000 (18:59 -0300)]
Merge branch '12018-tool-docs'
Refs #12018

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Documentation review updates
Tom Morris [Wed, 29 Nov 2017 21:49:37 +0000 (16:49 -0500)]
12018: Documentation review updates

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

6 years agoMerge branch '12614-docker-sanity-check' refs #12614
Peter Amstutz [Wed, 29 Nov 2017 20:00:12 +0000 (15:00 -0500)]
Merge branch '12614-docker-sanity-check' refs #12614

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12614: Restore keepclient error check prior to NewContainerRunner()
Peter Amstutz [Wed, 29 Nov 2017 17:12:22 +0000 (12:12 -0500)]
12614: Restore keepclient error check prior to NewContainerRunner()

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12614: Restore order set state "Running" before starting the container.
Peter Amstutz [Wed, 29 Nov 2017 16:56:46 +0000 (11:56 -0500)]
12614: Restore order set state "Running" before starting the container.

Use "Matches" in tests.

Broken node hook tests that output is captured.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12662-multisite-add-broken'
Lucas Di Pentima [Wed, 29 Nov 2017 16:14:44 +0000 (13:14 -0300)]
Merge branch '12662-multisite-add-broken'
Refs #12662

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12167: Test request ID size.
Tom Clegg [Wed, 29 Nov 2017 16:07:14 +0000 (11:07 -0500)]
12167: Test request ID size.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12167: Use pseudo-random IDs instead of timestamps.
Tom Clegg [Wed, 29 Nov 2017 15:56:44 +0000 (10:56 -0500)]
12167: Use pseudo-random IDs instead of timestamps.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12167: Test LogRequests() and AddRequestIDs().
Tom Clegg [Wed, 29 Nov 2017 15:37:05 +0000 (10:37 -0500)]
12167: Test LogRequests() and AddRequestIDs().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12167: Fix tests.
Tom Clegg [Wed, 29 Nov 2017 07:05:23 +0000 (02:05 -0500)]
12167: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12614: Fix tests
Peter Amstutz [Tue, 28 Nov 2017 21:40:20 +0000 (16:40 -0500)]
12614: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12614: Add tests.
Peter Amstutz [Tue, 28 Nov 2017 21:35:15 +0000 (16:35 -0500)]
12614: Add tests.

Also reorder run sequence slightly so that failure to start container due to
broken node is retryable.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12614: Detect and report broken node errors.
Peter Amstutz [Tue, 28 Nov 2017 20:27:50 +0000 (15:27 -0500)]
12614: Detect and report broken node errors.

1. crunch-run has an errorBlacklist of messages that mean "this Node probably
   cannot run any container successfully"

2. Upon seeing a blacklisted error, the broken node hook script is run

3. The hook script is responsible for marking the node as "broken, don't
schedule work on it, such as executing scontrol reconfigure Node=XXX State=DOWN

4. After the hook script terminates, if possible the container is reset from Locked
back to Queued state.

5. Then crunch-run can terminate.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12662: Ignore anonymous user token when getting the current user
Lucas Di Pentima [Tue, 28 Nov 2017 19:42:11 +0000 (16:42 -0300)]
12662: Ignore anonymous user token when getting the current user

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoAdd jessie-dev to apt.arvados.org.list for jobs image, no issue #
Peter Amstutz [Mon, 27 Nov 2017 21:44:22 +0000 (16:44 -0500)]
Add jessie-dev to apt.arvados.org.list for jobs image, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoFix arvbox localdemo problems. refs #12660
Peter Amstutz [Mon, 27 Nov 2017 16:18:11 +0000 (11:18 -0500)]
Fix arvbox localdemo problems. refs #12660

* Checkout composer when building demo image.
* Fix node.js version.
* Precompile assests for SSO and workbench.
* Set correct workbench rails environment.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12616-acr-validate' closes #12616
Peter Amstutz [Mon, 27 Nov 2017 19:37:11 +0000 (14:37 -0500)]
Merge branch '12616-acr-validate' closes #12616

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12167: Restore timeToStatus + timeWriteBody in keepstore/proxy logs.
Tom Clegg [Mon, 27 Nov 2017 15:51:45 +0000 (10:51 -0500)]
12167: Restore timeToStatus + timeWriteBody in keepstore/proxy logs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12167: Merge branch 'master' into 12167-keep-request-id
Tom Clegg [Sat, 25 Nov 2017 07:20:23 +0000 (02:20 -0500)]
12167: Merge branch 'master' into 12167-keep-request-id

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMake build/run-build-packages-all-targets.sh more explicit about which
Ward Vandewege [Sat, 25 Nov 2017 14:57:47 +0000 (09:57 -0500)]
Make build/run-build-packages-all-targets.sh more explicit about which
target fails.

Remove old ubuntu1204 docker files, we do not support that distribution
anymore. This fixes #12658

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMerge branch '12475-asyncbuf'
Tom Clegg [Sat, 25 Nov 2017 07:17:18 +0000 (02:17 -0500)]
Merge branch '12475-asyncbuf'

refs #12475

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12475: Merge branch 'master' into 12475-asyncbuf
Tom Clegg [Sat, 25 Nov 2017 07:16:05 +0000 (02:16 -0500)]
12475: Merge branch 'master' into 12475-asyncbuf

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12475: Return non-nil error at EOF even if len(p)==0.
Tom Clegg [Sat, 25 Nov 2017 07:05:36 +0000 (02:05 -0500)]
12475: Return non-nil error at EOF even if len(p)==0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12167: Clean up test suite logging.
Tom Clegg [Fri, 24 Nov 2017 21:52:36 +0000 (16:52 -0500)]
12167: Clean up test suite logging.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12167: Propagate X-Request-ID through keepproxy and keepstore.
Tom Clegg [Fri, 24 Nov 2017 21:46:58 +0000 (16:46 -0500)]
12167: Propagate X-Request-ID through keepproxy and keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12018: Replaced '<' & '>' with their html entities counterparts
Lucas Di Pentima [Fri, 24 Nov 2017 17:23:54 +0000 (14:23 -0300)]
12018: Replaced '<' & '>' with their html entities counterparts

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12627-token-scope'
Tom Clegg [Fri, 24 Nov 2017 15:29:25 +0000 (10:29 -0500)]
Merge branch '12627-token-scope'

fixes #12627

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12627: Any token can permit scope, if acting as the right user.
Tom Clegg [Fri, 24 Nov 2017 06:10:56 +0000 (01:10 -0500)]
12627: Any token can permit scope, if acting as the right user.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12627: Set current_user from first valid reader_token
Tom Clegg [Thu, 23 Nov 2017 18:52:46 +0000 (13:52 -0500)]
12627: Set current_user from first valid reader_token

...if primary session token is missing or invalid.

Workbench relies on this for anonymous access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12018: Added documentation for arv-sync-groups
Lucas Di Pentima [Thu, 23 Nov 2017 20:53:04 +0000 (17:53 -0300)]
12018: Added documentation for arv-sync-groups

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12519-multisite-project-search'
Lucas Di Pentima [Thu, 23 Nov 2017 19:22:21 +0000 (16:22 -0300)]
Merge branch '12519-multisite-project-search'
Closes #12519

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12519: Code simplification
Lucas Di Pentima [Thu, 23 Nov 2017 19:13:10 +0000 (16:13 -0300)]
12519: Code simplification

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12627-token-scope'
Tom Clegg [Thu, 23 Nov 2017 18:35:39 +0000 (13:35 -0500)]
Merge branch '12627-token-scope'

fixes #12627

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12475: Faster asyncbuf.
Tom Clegg [Thu, 23 Nov 2017 14:33:34 +0000 (09:33 -0500)]
12475: Faster asyncbuf.

Unlock earlier so multiple readers can copy bytes concurrently.

100 readers: improves from 4.5 GB/s to 17 GB/s
1 reader: unchanged 0.75 GB/s

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12475: Add TestManyFailedPuts with a short timeout.
Tom Clegg [Thu, 23 Nov 2017 06:04:38 +0000 (01:04 -0500)]
12475: Add TestManyFailedPuts with a short timeout.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12475: Rewrite streamer -> asyncbuf.
Tom Clegg [Thu, 23 Nov 2017 04:11:43 +0000 (23:11 -0500)]
12475: Rewrite streamer -> asyncbuf.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12627: Ensure current_user relates to the token that permits scope.
Tom Clegg [Wed, 22 Nov 2017 22:49:08 +0000 (17:49 -0500)]
12627: Ensure current_user relates to the token that permits scope.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12519: Remove trailing slash when guessing the workbench url.
Lucas Di Pentima [Wed, 22 Nov 2017 21:34:54 +0000 (18:34 -0300)]
12519: Remove trailing slash when guessing the workbench url.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12616: Fix validate
Peter Amstutz [Tue, 21 Nov 2017 22:42:42 +0000 (17:42 -0500)]
12616: Fix validate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12377-arvbox-composer' refs #12377
Peter Amstutz [Wed, 22 Nov 2017 20:30:56 +0000 (15:30 -0500)]
Merge branch '12377-arvbox-composer'  refs #12377

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoSet composer config. Bump SSO script for updated passenger.
Peter Amstutz [Wed, 22 Nov 2017 19:31:53 +0000 (14:31 -0500)]
Set composer config.  Bump SSO script for updated passenger.

7 years agoMerge branch '12318-update-libcloud-pin'
Lucas Di Pentima [Tue, 21 Nov 2017 23:06:25 +0000 (20:06 -0300)]
Merge branch '12318-update-libcloud-pin'
Closes #12318

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12519: Activate every search results' tooltip on creation.
Lucas Di Pentima [Tue, 21 Nov 2017 21:52:50 +0000 (18:52 -0300)]
12519: Activate every search results' tooltip on creation.
Removed unnecessary code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12513-make-dashboard-fast-again' 1.1.1
Ward Vandewege [Tue, 21 Nov 2017 19:12:11 +0000 (14:12 -0500)]
Merge branch '12513-make-dashboard-fast-again'

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMerge branch 'master' into 12513-make-dashboard-fast-again
Ward Vandewege [Tue, 21 Nov 2017 19:11:33 +0000 (14:11 -0500)]
Merge branch 'master' into 12513-make-dashboard-fast-again

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMatch the actual name of the CR in the
Ward Vandewege [Tue, 21 Nov 2017 19:08:50 +0000 (14:08 -0500)]
Match the actual name of the CR in the
ApplicationLayoutTest#test_visit_dashboard_as_* tests, rather than the
state of the CR. It's confusing that the name is 'running'!

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMake sure to apply the text-overflow-ellipsis class to the name of
Ward Vandewege [Tue, 21 Nov 2017 19:03:47 +0000 (14:03 -0500)]
Make sure to apply the text-overflow-ellipsis class to the name of
running container requests, too.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12519: Make Capybara happy by using a different kind of for loop.
Lucas Di Pentima [Tue, 21 Nov 2017 18:11:30 +0000 (15:11 -0300)]
12519: Make Capybara happy by using a different kind of for loop.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12519: Use icons on 'show' buttons instead of 'C'/'P' labels.
Lucas Di Pentima [Tue, 21 Nov 2017 16:18:39 +0000 (13:18 -0300)]
12519: Use icons on 'show' buttons instead of 'C'/'P' labels.
Added bootstrap-style tooltips.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoAdd another missing dependency to the sanity check in run-tests.sh.
Ward Vandewege [Tue, 21 Nov 2017 14:28:16 +0000 (09:28 -0500)]
Add another missing dependency to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12519: Move multisite search page to /search
Tom Clegg [Tue, 21 Nov 2017 14:15:56 +0000 (09:15 -0500)]
12519: Move multisite search page to /search

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoAdd missing dependencies to the sanity check in run-tests.sh.
Ward Vandewege [Sun, 19 Nov 2017 23:53:32 +0000 (18:53 -0500)]
Add missing dependencies to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12513: Fix 2 more tests.
Ward Vandewege [Sun, 19 Nov 2017 20:13:56 +0000 (15:13 -0500)]
12513: Fix 2 more tests.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12519: Moved multisite search to its own URL
Lucas Di Pentima [Sat, 18 Nov 2017 00:13:23 +0000 (21:13 -0300)]
12519: Moved multisite search to its own URL

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12530-faster-scopes-filter'
Tom Clegg [Fri, 17 Nov 2017 21:51:22 +0000 (16:51 -0500)]
Merge branch '12530-faster-scopes-filter'

closes #12530

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12519: Updated search box placeholder message
Lucas Di Pentima [Fri, 17 Nov 2017 20:58:35 +0000 (17:58 -0300)]
12519: Updated search box placeholder message

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12519: Simplified code by nesting an additional MergingLoader
Lucas Di Pentima [Fri, 17 Nov 2017 20:53:38 +0000 (17:53 -0300)]
12519: Simplified code by nesting an additional MergingLoader

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12318: Merge branch 'master' into 12318-update-libcloud-pin
Lucas Di Pentima [Fri, 17 Nov 2017 17:19:08 +0000 (14:19 -0300)]
12318: Merge branch 'master' into 12318-update-libcloud-pin

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12318: Updated integration test to correctly raise a RateLimitReachedError
Lucas Di Pentima [Fri, 17 Nov 2017 17:16:48 +0000 (14:16 -0300)]
12318: Updated integration test to correctly raise a RateLimitReachedError
exception, matching libcloud updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoBump schema-salad for extension schema loading bugfix. refs #11163
Peter Amstutz [Thu, 16 Nov 2017 19:07:37 +0000 (14:07 -0500)]
Bump schema-salad for extension schema loading bugfix. refs #11163

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12519: Added project search to multisite search
Lucas Di Pentima [Thu, 16 Nov 2017 20:15:17 +0000 (17:15 -0300)]
12519: Added project search to multisite search
First draft, for approach validaton purposes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12530: Speed up query by skipping unneeded ActiveRecord loading.
Tom Clegg [Thu, 16 Nov 2017 18:59:05 +0000 (13:59 -0500)]
12530: Speed up query by skipping unneeded ActiveRecord loading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12601-pathmap-subwf' closes #12601
Peter Amstutz [Thu, 16 Nov 2017 14:54:47 +0000 (09:54 -0500)]
Merge branch '12601-pathmap-subwf'  closes #12601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '10767-remove-puma-gem'
Ward Vandewege [Wed, 15 Nov 2017 21:44:50 +0000 (16:44 -0500)]
Merge branch '10767-remove-puma-gem'

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoRemove puma from the API server Gemfile.
Ward Vandewege [Wed, 15 Nov 2017 20:03:33 +0000 (15:03 -0500)]
Remove puma from the API server Gemfile.

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12601: Add test
Peter Amstutz [Wed, 15 Nov 2017 18:34:32 +0000 (13:34 -0500)]
12601: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12601: Apply pathmap to job order for RunInSingleContainer subwf.
Peter Amstutz [Wed, 15 Nov 2017 15:56:00 +0000 (10:56 -0500)]
12601: Apply pathmap to job order for RunInSingleContainer subwf.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12513: Update tests, remove more unused code.
Tom Clegg [Mon, 13 Nov 2017 19:57:29 +0000 (14:57 -0500)]
12513: Update tests, remove more unused code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12515-use-configured-wb'
Tom Clegg [Mon, 13 Nov 2017 15:38:50 +0000 (10:38 -0500)]
Merge branch '12515-use-configured-wb'

closes #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12515-configured-workbench-address'
Tom Clegg [Mon, 13 Nov 2017 15:38:34 +0000 (10:38 -0500)]
Merge branch '12515-configured-workbench-address'

refs #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12318: Use libcloud fork 2.2.2.dev3
Lucas Di Pentima [Fri, 10 Nov 2017 17:42:03 +0000 (14:42 -0300)]
12318: Use libcloud fork 2.2.2.dev3

Includes a fix for delete_old_vhd, limit number of retries on
destroy_node & a fix on Retry-After header handling on
exceptions.

Refs #12318, #12452, #12324

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoAdd missing module name.
Tom Clegg [Thu, 9 Nov 2017 20:40:45 +0000 (15:40 -0500)]
Add missing module name.

refs #11220
fixes #12590

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Add discovery doc test.
Tom Clegg [Thu, 9 Nov 2017 18:26:09 +0000 (13:26 -0500)]
12515: Add discovery doc test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Use workbenchUrl advertised by API server discovery doc.
Tom Clegg [Wed, 8 Nov 2017 21:53:22 +0000 (16:53 -0500)]
12515: Use workbenchUrl advertised by API server discovery doc.

Fall back to guessing "workbench."+apiHost (as before) if discovery
doc does not offer a workbenchUrl (e.g., API server is too old).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Advertise Workbench address in discovery doc.
Tom Clegg [Tue, 31 Oct 2017 17:56:07 +0000 (13:56 -0400)]
12515: Advertise Workbench address in discovery doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12568-cwl-collection-cache' closes #12568
Peter Amstutz [Wed, 8 Nov 2017 17:08:54 +0000 (12:08 -0500)]
Merge branch '12568-cwl-collection-cache'  closes #12568

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12538-crunchrun-kill-arvmount' closes #12538
Peter Amstutz [Wed, 8 Nov 2017 17:02:02 +0000 (12:02 -0500)]
Merge branch '12538-crunchrun-kill-arvmount' closes #12538

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12306-dont-stat-mounts'
Tom Clegg [Wed, 8 Nov 2017 16:07:58 +0000 (11:07 -0500)]
Merge branch '12306-dont-stat-mounts'

fixes #12306

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12538: Put cleanup back into separate defer
Peter Amstutz [Wed, 8 Nov 2017 15:36:37 +0000 (10:36 -0500)]
12538: Put cleanup back into separate defer

* Always report unmount command

* Send kill signal to arv-mount on timeout (but don't wait for it)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor crunch-run shutdown
Peter Amstutz [Tue, 7 Nov 2017 15:14:40 +0000 (10:14 -0500)]
12538: Refactor crunch-run shutdown

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor and assume arv-mount --unmount does all the hard work.
Peter Amstutz [Mon, 6 Nov 2017 21:41:05 +0000 (16:41 -0500)]
12538: Refactor and assume arv-mount --unmount does all the hard work.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor force-unmount behavior.
Peter Amstutz [Fri, 3 Nov 2017 20:14:42 +0000 (16:14 -0400)]
12538: Refactor force-unmount behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Fix tests.
Peter Amstutz [Fri, 3 Nov 2017 19:04:00 +0000 (15:04 -0400)]
12538: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Make handling of arv-mount process more robust
Peter Amstutz [Fri, 3 Nov 2017 15:34:46 +0000 (11:34 -0400)]
12538: Make handling of arv-mount process more robust

* If initial "fusermount -u -z" of arv-mount dir times out, use "arv-mount --unmount"

* If that doesn't work, send sigkill to arv-mount and then run "fusermount -u
  -z" again

* Refactor "stop container if arv-mount terminates" behavior

* Make arv-mount logging channel work as intended and enable crunchstat for arv-mount.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoForce kill arv-mount wip
Peter Amstutz [Thu, 2 Nov 2017 21:52:58 +0000 (17:52 -0400)]
Force kill arv-mount wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12306: Resolve any readable symlinks on unmount path.
Tom Clegg [Tue, 7 Nov 2017 23:10:13 +0000 (18:10 -0500)]
12306: Resolve any readable symlinks on unmount path.

This avoids getting stuck in realpath/lstat in certain cases: e.g.,
the mount point is $HOME/keep, $HOME is "/home/foo", and "/home" is a
symlink to "/data/home".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12568: Reduce min_entries
Peter Amstutz [Tue, 7 Nov 2017 20:52:08 +0000 (15:52 -0500)]
12568: Reduce min_entries

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Update test to check which collection gets evicted
Peter Amstutz [Tue, 7 Nov 2017 18:58:16 +0000 (13:58 -0500)]
12568: Update test to check which collection gets evicted

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Add test for collection cache management behavior
Peter Amstutz [Tue, 7 Nov 2017 18:52:28 +0000 (13:52 -0500)]
12568: Add test for collection cache management behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Mange collection cache to avoid excessive memory usage.
Peter Amstutz [Tue, 7 Nov 2017 16:43:16 +0000 (11:43 -0500)]
12568: Mange collection cache to avoid excessive memory usage.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12306: Resolve relative paths early, to avoid realpath() later.
Tom Clegg [Mon, 6 Nov 2017 21:44:07 +0000 (16:44 -0500)]
12306: Resolve relative paths early, to avoid realpath() later.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '11994-no-sysfs-fuse'
Tom Clegg [Mon, 6 Nov 2017 21:11:02 +0000 (16:11 -0500)]
Merge branch '11994-no-sysfs-fuse'

fixes #11994

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12533-token-n-ip-logging'
Lucas Di Pentima [Mon, 6 Nov 2017 20:15:49 +0000 (17:15 -0300)]
Merge branch '12533-token-n-ip-logging'
Closes #12533

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12533: Use @remote_ip to take into account the reverse proxy
Lucas Di Pentima [Mon, 6 Nov 2017 20:13:57 +0000 (17:13 -0300)]
12533: Use @remote_ip to take into account the reverse proxy

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago11994: Fix wrong class in "except ENOENT" handler.
Tom Clegg [Mon, 6 Nov 2017 18:16:02 +0000 (13:16 -0500)]
11994: Fix wrong class in "except ENOENT" handler.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12306: Avoid calling realpath/lstat unnecessarily during --unmount.
Tom Clegg [Fri, 3 Nov 2017 18:18:36 +0000 (14:18 -0400)]
12306: Avoid calling realpath/lstat unnecessarily during --unmount.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12533: Add client IP address & authorization token UUID to the logs.
Lucas Di Pentima [Fri, 3 Nov 2017 20:05:16 +0000 (17:05 -0300)]
12533: Add client IP address & authorization token UUID to the logs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago11220: Fix test setup.
Tom Clegg [Fri, 3 Nov 2017 18:46:29 +0000 (14:46 -0400)]
11220: Fix test setup.

refs #11220

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>