arvados.git
6 years agoMake build/run-build-packages-all-targets.sh more explicit about which
Ward Vandewege [Sat, 25 Nov 2017 14:57:47 +0000 (09:57 -0500)]
Make build/run-build-packages-all-targets.sh more explicit about which
target fails.

Remove old ubuntu1204 docker files, we do not support that distribution
anymore. This fixes #12658

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '12475-asyncbuf'
Tom Clegg [Sat, 25 Nov 2017 07:17:18 +0000 (02:17 -0500)]
Merge branch '12475-asyncbuf'

refs #12475

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12475: Merge branch 'master' into 12475-asyncbuf
Tom Clegg [Sat, 25 Nov 2017 07:16:05 +0000 (02:16 -0500)]
12475: Merge branch 'master' into 12475-asyncbuf

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12475: Return non-nil error at EOF even if len(p)==0.
Tom Clegg [Sat, 25 Nov 2017 07:05:36 +0000 (02:05 -0500)]
12475: Return non-nil error at EOF even if len(p)==0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12167: Clean up test suite logging.
Tom Clegg [Fri, 24 Nov 2017 21:52:36 +0000 (16:52 -0500)]
12167: Clean up test suite logging.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12167: Propagate X-Request-ID through keepproxy and keepstore.
Tom Clegg [Fri, 24 Nov 2017 21:46:58 +0000 (16:46 -0500)]
12167: Propagate X-Request-ID through keepproxy and keepstore.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12018: Replaced '<' & '>' with their html entities counterparts
Lucas Di Pentima [Fri, 24 Nov 2017 17:23:54 +0000 (14:23 -0300)]
12018: Replaced '<' & '>' with their html entities counterparts

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11453: Improve config docs. Disable remote auth by default.
Tom Clegg [Fri, 24 Nov 2017 17:19:27 +0000 (12:19 -0500)]
11453: Improve config docs. Disable remote auth by default.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11453: Check HTTP method of token validation request.
Tom Clegg [Fri, 24 Nov 2017 17:18:11 +0000 (12:18 -0500)]
11453: Check HTTP method of token validation request.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12627-token-scope'
Tom Clegg [Fri, 24 Nov 2017 15:29:25 +0000 (10:29 -0500)]
Merge branch '12627-token-scope'

fixes #12627

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12627: Any token can permit scope, if acting as the right user.
Tom Clegg [Fri, 24 Nov 2017 06:10:56 +0000 (01:10 -0500)]
12627: Any token can permit scope, if acting as the right user.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12627: Set current_user from first valid reader_token
Tom Clegg [Thu, 23 Nov 2017 18:52:46 +0000 (13:52 -0500)]
12627: Set current_user from first valid reader_token

...if primary session token is missing or invalid.

Workbench relies on this for anonymous access.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11453: Only update safe fields from remote user record.
Tom Clegg [Fri, 24 Nov 2017 15:21:28 +0000 (10:21 -0500)]
11453: Only update safe fields from remote user record.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11453: Remove unused arvados client.
Tom Clegg [Fri, 24 Nov 2017 14:18:51 +0000 (09:18 -0500)]
11453: Remove unused arvados client.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12018: Added documentation for arv-sync-groups
Lucas Di Pentima [Thu, 23 Nov 2017 20:53:04 +0000 (17:53 -0300)]
12018: Added documentation for arv-sync-groups

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12519-multisite-project-search'
Lucas Di Pentima [Thu, 23 Nov 2017 19:22:21 +0000 (16:22 -0300)]
Merge branch '12519-multisite-project-search'
Closes #12519

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12519: Code simplification
Lucas Di Pentima [Thu, 23 Nov 2017 19:13:10 +0000 (16:13 -0300)]
12519: Code simplification

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12627-token-scope'
Tom Clegg [Thu, 23 Nov 2017 18:35:39 +0000 (13:35 -0500)]
Merge branch '12627-token-scope'

fixes #12627

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12475: Faster asyncbuf.
Tom Clegg [Thu, 23 Nov 2017 14:33:34 +0000 (09:33 -0500)]
12475: Faster asyncbuf.

Unlock earlier so multiple readers can copy bytes concurrently.

100 readers: improves from 4.5 GB/s to 17 GB/s
1 reader: unchanged 0.75 GB/s

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12475: Add TestManyFailedPuts with a short timeout.
Tom Clegg [Thu, 23 Nov 2017 06:04:38 +0000 (01:04 -0500)]
12475: Add TestManyFailedPuts with a short timeout.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12475: Rewrite streamer -> asyncbuf.
Tom Clegg [Thu, 23 Nov 2017 04:11:43 +0000 (23:11 -0500)]
12475: Rewrite streamer -> asyncbuf.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12627: Ensure current_user relates to the token that permits scope.
Tom Clegg [Wed, 22 Nov 2017 22:49:08 +0000 (17:49 -0500)]
12627: Ensure current_user relates to the token that permits scope.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12519: Remove trailing slash when guessing the workbench url.
Lucas Di Pentima [Wed, 22 Nov 2017 21:34:54 +0000 (18:34 -0300)]
12519: Remove trailing slash when guessing the workbench url.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12616: Fix validate
Peter Amstutz [Tue, 21 Nov 2017 22:42:42 +0000 (17:42 -0500)]
12616: Fix validate

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12377-arvbox-composer' refs #12377
Peter Amstutz [Wed, 22 Nov 2017 20:30:56 +0000 (15:30 -0500)]
Merge branch '12377-arvbox-composer'  refs #12377

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoSet composer config. Bump SSO script for updated passenger.
Peter Amstutz [Wed, 22 Nov 2017 19:31:53 +0000 (14:31 -0500)]
Set composer config.  Bump SSO script for updated passenger.

6 years ago12483: Fix cadaver tests.
Tom Clegg [Wed, 22 Nov 2017 17:45:08 +0000 (12:45 -0500)]
12483: Fix cadaver tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix multiple WriteHeader(). Skip updates on readonly methods.
Tom Clegg [Wed, 22 Nov 2017 17:32:38 +0000 (12:32 -0500)]
12483: Fix multiple WriteHeader(). Skip updates on readonly methods.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Update comments.
Tom Clegg [Wed, 22 Nov 2017 15:00:14 +0000 (10:00 -0500)]
12483: Update comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Drop writableSegment interface.
Tom Clegg [Wed, 22 Nov 2017 14:57:21 +0000 (09:57 -0500)]
12483: Drop writableSegment interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Rename extent->segment, file->filehandle.
Tom Clegg [Wed, 22 Nov 2017 14:56:05 +0000 (09:56 -0500)]
12483: Rename extent->segment, file->filehandle.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12318-update-libcloud-pin'
Lucas Di Pentima [Tue, 21 Nov 2017 23:06:25 +0000 (20:06 -0300)]
Merge branch '12318-update-libcloud-pin'
Closes #12318

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12519: Activate every search results' tooltip on creation.
Lucas Di Pentima [Tue, 21 Nov 2017 21:52:50 +0000 (18:52 -0300)]
12519: Activate every search results' tooltip on creation.
Removed unnecessary code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12513-make-dashboard-fast-again' 1.1.1
Ward Vandewege [Tue, 21 Nov 2017 19:12:11 +0000 (14:12 -0500)]
Merge branch '12513-make-dashboard-fast-again'

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch 'master' into 12513-make-dashboard-fast-again
Ward Vandewege [Tue, 21 Nov 2017 19:11:33 +0000 (14:11 -0500)]
Merge branch 'master' into 12513-make-dashboard-fast-again

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMatch the actual name of the CR in the
Ward Vandewege [Tue, 21 Nov 2017 19:08:50 +0000 (14:08 -0500)]
Match the actual name of the CR in the
ApplicationLayoutTest#test_visit_dashboard_as_* tests, rather than the
state of the CR. It's confusing that the name is 'running'!

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMake sure to apply the text-overflow-ellipsis class to the name of
Ward Vandewege [Tue, 21 Nov 2017 19:03:47 +0000 (14:03 -0500)]
Make sure to apply the text-overflow-ellipsis class to the name of
running container requests, too.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago12519: Make Capybara happy by using a different kind of for loop.
Lucas Di Pentima [Tue, 21 Nov 2017 18:11:30 +0000 (15:11 -0300)]
12519: Make Capybara happy by using a different kind of for loop.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12519: Use icons on 'show' buttons instead of 'C'/'P' labels.
Lucas Di Pentima [Tue, 21 Nov 2017 16:18:39 +0000 (13:18 -0300)]
12519: Use icons on 'show' buttons instead of 'C'/'P' labels.
Added bootstrap-style tooltips.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoAdd another missing dependency to the sanity check in run-tests.sh.
Ward Vandewege [Tue, 21 Nov 2017 14:28:16 +0000 (09:28 -0500)]
Add another missing dependency to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago12519: Move multisite search page to /search
Tom Clegg [Tue, 21 Nov 2017 14:15:56 +0000 (09:15 -0500)]
12519: Move multisite search page to /search

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix receiver type.
Tom Clegg [Tue, 21 Nov 2017 08:03:55 +0000 (03:03 -0500)]
12483: Fix receiver type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Delay flushing collection until successful http response.
Tom Clegg [Tue, 21 Nov 2017 06:59:27 +0000 (01:59 -0500)]
12483: Delay flushing collection until successful http response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Make webdav rename/remove work. Tidy up code.
Tom Clegg [Mon, 20 Nov 2017 23:43:46 +0000 (18:43 -0500)]
12483: Make webdav rename/remove work. Tidy up code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: More loading speed.
Tom Clegg [Mon, 20 Nov 2017 17:11:18 +0000 (12:11 -0500)]
12483: More loading speed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Speed up manifest loading.
Tom Clegg [Mon, 20 Nov 2017 15:30:05 +0000 (10:30 -0500)]
12483: Speed up manifest loading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoAdd missing dependencies to the sanity check in run-tests.sh.
Ward Vandewege [Sun, 19 Nov 2017 23:53:32 +0000 (18:53 -0500)]
Add missing dependencies to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago12513: Fix 2 more tests.
Ward Vandewege [Sun, 19 Nov 2017 20:13:56 +0000 (15:13 -0500)]
12513: Fix 2 more tests.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago12483: Allow seek and write beyond EOF.
Tom Clegg [Sat, 18 Nov 2017 06:44:55 +0000 (01:44 -0500)]
12483: Allow seek and write beyond EOF.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Avoid empty segments.
Tom Clegg [Sat, 18 Nov 2017 05:19:56 +0000 (00:19 -0500)]
12483: Avoid empty segments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add RemoveAll() to CollectionFileSystem.
Tom Clegg [Sat, 18 Nov 2017 01:53:38 +0000 (20:53 -0500)]
12483: Add RemoveAll() to CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Enable rename via webdav.
Tom Clegg [Fri, 17 Nov 2017 16:38:13 +0000 (11:38 -0500)]
12483: Enable rename via webdav.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add Rename(old,new) to CollectionFileSystem.
Tom Clegg [Fri, 17 Nov 2017 16:06:24 +0000 (11:06 -0500)]
12483: Add Rename(old,new) to CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add comments.
Tom Clegg [Fri, 17 Nov 2017 08:04:00 +0000 (03:04 -0500)]
12483: Add comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support O_TRUNC flag.
Tom Clegg [Thu, 16 Nov 2017 21:58:18 +0000 (16:58 -0500)]
12483: Support O_TRUNC flag.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Compress adjacent segments when writing filetokens.
Tom Clegg [Thu, 16 Nov 2017 21:19:58 +0000 (16:19 -0500)]
12483: Compress adjacent segments when writing filetokens.

"0:100:foo 100:100:foo 200:100:foo" --> "0:300:foo"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support O_APPEND. Check for invalid/unsupported file modes.
Tom Clegg [Thu, 16 Nov 2017 21:06:20 +0000 (16:06 -0500)]
12483: Support O_APPEND. Check for invalid/unsupported file modes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support file create/write via webdav.
Tom Clegg [Wed, 15 Nov 2017 18:24:16 +0000 (13:24 -0500)]
12483: Support file create/write via webdav.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12519: Moved multisite search to its own URL
Lucas Di Pentima [Sat, 18 Nov 2017 00:13:23 +0000 (21:13 -0300)]
12519: Moved multisite search to its own URL

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12530-faster-scopes-filter'
Tom Clegg [Fri, 17 Nov 2017 21:51:22 +0000 (16:51 -0500)]
Merge branch '12530-faster-scopes-filter'

closes #12530

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12519: Updated search box placeholder message
Lucas Di Pentima [Fri, 17 Nov 2017 20:58:35 +0000 (17:58 -0300)]
12519: Updated search box placeholder message

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12519: Simplified code by nesting an additional MergingLoader
Lucas Di Pentima [Fri, 17 Nov 2017 20:53:38 +0000 (17:53 -0300)]
12519: Simplified code by nesting an additional MergingLoader

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12318: Merge branch 'master' into 12318-update-libcloud-pin
Lucas Di Pentima [Fri, 17 Nov 2017 17:19:08 +0000 (14:19 -0300)]
12318: Merge branch 'master' into 12318-update-libcloud-pin

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12318: Updated integration test to correctly raise a RateLimitReachedError
Lucas Di Pentima [Fri, 17 Nov 2017 17:16:48 +0000 (14:16 -0300)]
12318: Updated integration test to correctly raise a RateLimitReachedError
exception, matching libcloud updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoBump schema-salad for extension schema loading bugfix. refs #11163
Peter Amstutz [Thu, 16 Nov 2017 19:07:37 +0000 (14:07 -0500)]
Bump schema-salad for extension schema loading bugfix. refs #11163

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12519: Added project search to multisite search
Lucas Di Pentima [Thu, 16 Nov 2017 20:15:17 +0000 (17:15 -0300)]
12519: Added project search to multisite search
First draft, for approach validaton purposes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12530: Speed up query by skipping unneeded ActiveRecord loading.
Tom Clegg [Thu, 16 Nov 2017 18:59:05 +0000 (13:59 -0500)]
12530: Speed up query by skipping unneeded ActiveRecord loading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12601-pathmap-subwf' closes #12601
Peter Amstutz [Thu, 16 Nov 2017 14:54:47 +0000 (09:54 -0500)]
Merge branch '12601-pathmap-subwf'  closes #12601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '10767-remove-puma-gem'
Ward Vandewege [Wed, 15 Nov 2017 21:44:50 +0000 (16:44 -0500)]
Merge branch '10767-remove-puma-gem'

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoRemove puma from the API server Gemfile.
Ward Vandewege [Wed, 15 Nov 2017 20:03:33 +0000 (15:03 -0500)]
Remove puma from the API server Gemfile.

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago12601: Add test
Peter Amstutz [Wed, 15 Nov 2017 18:34:32 +0000 (13:34 -0500)]
12601: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12483: Fix makeParentDirs bug when higher levels already exist.
Tom Clegg [Wed, 15 Nov 2017 18:14:13 +0000 (13:14 -0500)]
12483: Fix makeParentDirs bug when higher levels already exist.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Persist empty files.
Tom Clegg [Wed, 15 Nov 2017 18:01:29 +0000 (13:01 -0500)]
12483: Persist empty files.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12601: Apply pathmap to job order for RunInSingleContainer subwf.
Peter Amstutz [Wed, 15 Nov 2017 15:56:00 +0000 (10:56 -0500)]
12601: Apply pathmap to job order for RunInSingleContainer subwf.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12483: Track memory use. Flush filled blocks while writing.
Tom Clegg [Tue, 14 Nov 2017 20:17:12 +0000 (15:17 -0500)]
12483: Track memory use. Flush filled blocks while writing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix missing Rlock() on inode accesses.
Tom Clegg [Tue, 14 Nov 2017 14:34:13 +0000 (09:34 -0500)]
12483: Fix missing Rlock() on inode accesses.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Return error on impossible manifest.
Tom Clegg [Tue, 14 Nov 2017 05:55:58 +0000 (00:55 -0500)]
12483: Return error on impossible manifest.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Avoid write/marshal race. Remove dead code.
Tom Clegg [Tue, 14 Nov 2017 05:04:24 +0000 (00:04 -0500)]
12483: Avoid write/marshal race. Remove dead code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12453: Persist written data to keep.
Tom Clegg [Mon, 13 Nov 2017 21:50:22 +0000 (16:50 -0500)]
12453: Persist written data to keep.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Remove unnecessary OpenFile() from inode interface.
Tom Clegg [Mon, 13 Nov 2017 21:50:07 +0000 (16:50 -0500)]
12483: Remove unnecessary OpenFile() from inode interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12513: Update tests, remove more unused code.
Tom Clegg [Mon, 13 Nov 2017 19:57:29 +0000 (14:57 -0500)]
12513: Update tests, remove more unused code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add Mkdir(), Remove().
Tom Clegg [Mon, 13 Nov 2017 15:36:50 +0000 (10:36 -0500)]
12483: Add Mkdir(), Remove().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12515-use-configured-wb'
Tom Clegg [Mon, 13 Nov 2017 15:38:50 +0000 (10:38 -0500)]
Merge branch '12515-use-configured-wb'

closes #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12515-configured-workbench-address'
Tom Clegg [Mon, 13 Nov 2017 15:38:34 +0000 (10:38 -0500)]
Merge branch '12515-configured-workbench-address'

refs #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add MarshalManifest().
Tom Clegg [Sun, 12 Nov 2017 08:31:16 +0000 (03:31 -0500)]
12483: Add MarshalManifest().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Remove unnecessary repack.
Tom Clegg [Sun, 12 Nov 2017 08:30:18 +0000 (03:30 -0500)]
12483: Remove unnecessary repack.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add tests for concurrent writers and random write sequences.
Tom Clegg [Sat, 11 Nov 2017 21:48:54 +0000 (16:48 -0500)]
12483: Add tests for concurrent writers and random write sequences.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add (File)Truncate().
Tom Clegg [Sat, 11 Nov 2017 20:32:47 +0000 (15:32 -0500)]
12483: Add (File)Truncate().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Simplify extent packing, reduce type casting.
Tom Clegg [Sat, 11 Nov 2017 18:26:37 +0000 (13:26 -0500)]
12483: Simplify extent packing, reduce type casting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Implement some writable cases.
Tom Clegg [Sat, 11 Nov 2017 08:30:24 +0000 (03:30 -0500)]
12483: Implement some writable cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add basic file writing tests.
Tom Clegg [Fri, 10 Nov 2017 21:34:28 +0000 (16:34 -0500)]
12483: Add basic file writing tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Rewrite collection filesystem.
Tom Clegg [Thu, 9 Nov 2017 18:24:34 +0000 (13:24 -0500)]
12483: Rewrite collection filesystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12318: Use libcloud fork 2.2.2.dev3
Lucas Di Pentima [Fri, 10 Nov 2017 17:42:03 +0000 (14:42 -0300)]
12318: Use libcloud fork 2.2.2.dev3

Includes a fix for delete_old_vhd, limit number of retries on
destroy_node & a fix on Retry-After header handling on
exceptions.

Refs #12318, #12452, #12324

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoAdd missing module name.
Tom Clegg [Thu, 9 Nov 2017 20:40:45 +0000 (15:40 -0500)]
Add missing module name.

refs #11220
fixes #12590

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12515: Add discovery doc test.
Tom Clegg [Thu, 9 Nov 2017 18:26:09 +0000 (13:26 -0500)]
12515: Add discovery doc test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12515: Use workbenchUrl advertised by API server discovery doc.
Tom Clegg [Wed, 8 Nov 2017 21:53:22 +0000 (16:53 -0500)]
12515: Use workbenchUrl advertised by API server discovery doc.

Fall back to guessing "workbench."+apiHost (as before) if discovery
doc does not offer a workbenchUrl (e.g., API server is too old).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12515: Advertise Workbench address in discovery doc.
Tom Clegg [Tue, 31 Oct 2017 17:56:07 +0000 (13:56 -0400)]
12515: Advertise Workbench address in discovery doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12568-cwl-collection-cache' closes #12568
Peter Amstutz [Wed, 8 Nov 2017 17:08:54 +0000 (12:08 -0500)]
Merge branch '12568-cwl-collection-cache'  closes #12568

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12538-crunchrun-kill-arvmount' closes #12538
Peter Amstutz [Wed, 8 Nov 2017 17:02:02 +0000 (12:02 -0500)]
Merge branch '12538-crunchrun-kill-arvmount' closes #12538

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12306-dont-stat-mounts'
Tom Clegg [Wed, 8 Nov 2017 16:07:58 +0000 (11:07 -0500)]
Merge branch '12306-dont-stat-mounts'

fixes #12306

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>