arvados.git
7 years ago12627: Ensure current_user relates to the token that permits scope.
Tom Clegg [Wed, 22 Nov 2017 22:49:08 +0000 (17:49 -0500)]
12627: Ensure current_user relates to the token that permits scope.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12377-arvbox-composer' refs #12377
Peter Amstutz [Wed, 22 Nov 2017 20:30:56 +0000 (15:30 -0500)]
Merge branch '12377-arvbox-composer'  refs #12377

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoSet composer config. Bump SSO script for updated passenger.
Peter Amstutz [Wed, 22 Nov 2017 19:31:53 +0000 (14:31 -0500)]
Set composer config.  Bump SSO script for updated passenger.

7 years agoMerge branch '12318-update-libcloud-pin'
Lucas Di Pentima [Tue, 21 Nov 2017 23:06:25 +0000 (20:06 -0300)]
Merge branch '12318-update-libcloud-pin'
Closes #12318

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12513-make-dashboard-fast-again' 1.1.1
Ward Vandewege [Tue, 21 Nov 2017 19:12:11 +0000 (14:12 -0500)]
Merge branch '12513-make-dashboard-fast-again'

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMerge branch 'master' into 12513-make-dashboard-fast-again
Ward Vandewege [Tue, 21 Nov 2017 19:11:33 +0000 (14:11 -0500)]
Merge branch 'master' into 12513-make-dashboard-fast-again

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMatch the actual name of the CR in the
Ward Vandewege [Tue, 21 Nov 2017 19:08:50 +0000 (14:08 -0500)]
Match the actual name of the CR in the
ApplicationLayoutTest#test_visit_dashboard_as_* tests, rather than the
state of the CR. It's confusing that the name is 'running'!

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMake sure to apply the text-overflow-ellipsis class to the name of
Ward Vandewege [Tue, 21 Nov 2017 19:03:47 +0000 (14:03 -0500)]
Make sure to apply the text-overflow-ellipsis class to the name of
running container requests, too.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoAdd another missing dependency to the sanity check in run-tests.sh.
Ward Vandewege [Tue, 21 Nov 2017 14:28:16 +0000 (09:28 -0500)]
Add another missing dependency to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoAdd missing dependencies to the sanity check in run-tests.sh.
Ward Vandewege [Sun, 19 Nov 2017 23:53:32 +0000 (18:53 -0500)]
Add missing dependencies to the sanity check in run-tests.sh.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12513: Fix 2 more tests.
Ward Vandewege [Sun, 19 Nov 2017 20:13:56 +0000 (15:13 -0500)]
12513: Fix 2 more tests.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoMerge branch '12530-faster-scopes-filter'
Tom Clegg [Fri, 17 Nov 2017 21:51:22 +0000 (16:51 -0500)]
Merge branch '12530-faster-scopes-filter'

closes #12530

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12318: Merge branch 'master' into 12318-update-libcloud-pin
Lucas Di Pentima [Fri, 17 Nov 2017 17:19:08 +0000 (14:19 -0300)]
12318: Merge branch 'master' into 12318-update-libcloud-pin

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12318: Updated integration test to correctly raise a RateLimitReachedError
Lucas Di Pentima [Fri, 17 Nov 2017 17:16:48 +0000 (14:16 -0300)]
12318: Updated integration test to correctly raise a RateLimitReachedError
exception, matching libcloud updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoBump schema-salad for extension schema loading bugfix. refs #11163
Peter Amstutz [Thu, 16 Nov 2017 19:07:37 +0000 (14:07 -0500)]
Bump schema-salad for extension schema loading bugfix. refs #11163

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12530: Speed up query by skipping unneeded ActiveRecord loading.
Tom Clegg [Thu, 16 Nov 2017 18:59:05 +0000 (13:59 -0500)]
12530: Speed up query by skipping unneeded ActiveRecord loading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12601-pathmap-subwf' closes #12601
Peter Amstutz [Thu, 16 Nov 2017 14:54:47 +0000 (09:54 -0500)]
Merge branch '12601-pathmap-subwf'  closes #12601

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '10767-remove-puma-gem'
Ward Vandewege [Wed, 15 Nov 2017 21:44:50 +0000 (16:44 -0500)]
Merge branch '10767-remove-puma-gem'

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoRemove puma from the API server Gemfile.
Ward Vandewege [Wed, 15 Nov 2017 20:03:33 +0000 (15:03 -0500)]
Remove puma from the API server Gemfile.

refs #10767

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12601: Add test
Peter Amstutz [Wed, 15 Nov 2017 18:34:32 +0000 (13:34 -0500)]
12601: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12601: Apply pathmap to job order for RunInSingleContainer subwf.
Peter Amstutz [Wed, 15 Nov 2017 15:56:00 +0000 (10:56 -0500)]
12601: Apply pathmap to job order for RunInSingleContainer subwf.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12513: Update tests, remove more unused code.
Tom Clegg [Mon, 13 Nov 2017 19:57:29 +0000 (14:57 -0500)]
12513: Update tests, remove more unused code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12515-use-configured-wb'
Tom Clegg [Mon, 13 Nov 2017 15:38:50 +0000 (10:38 -0500)]
Merge branch '12515-use-configured-wb'

closes #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12515-configured-workbench-address'
Tom Clegg [Mon, 13 Nov 2017 15:38:34 +0000 (10:38 -0500)]
Merge branch '12515-configured-workbench-address'

refs #12515

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12318: Use libcloud fork 2.2.2.dev3
Lucas Di Pentima [Fri, 10 Nov 2017 17:42:03 +0000 (14:42 -0300)]
12318: Use libcloud fork 2.2.2.dev3

Includes a fix for delete_old_vhd, limit number of retries on
destroy_node & a fix on Retry-After header handling on
exceptions.

Refs #12318, #12452, #12324

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoAdd missing module name.
Tom Clegg [Thu, 9 Nov 2017 20:40:45 +0000 (15:40 -0500)]
Add missing module name.

refs #11220
fixes #12590

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Add discovery doc test.
Tom Clegg [Thu, 9 Nov 2017 18:26:09 +0000 (13:26 -0500)]
12515: Add discovery doc test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Use workbenchUrl advertised by API server discovery doc.
Tom Clegg [Wed, 8 Nov 2017 21:53:22 +0000 (16:53 -0500)]
12515: Use workbenchUrl advertised by API server discovery doc.

Fall back to guessing "workbench."+apiHost (as before) if discovery
doc does not offer a workbenchUrl (e.g., API server is too old).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12515: Advertise Workbench address in discovery doc.
Tom Clegg [Tue, 31 Oct 2017 17:56:07 +0000 (13:56 -0400)]
12515: Advertise Workbench address in discovery doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12568-cwl-collection-cache' closes #12568
Peter Amstutz [Wed, 8 Nov 2017 17:08:54 +0000 (12:08 -0500)]
Merge branch '12568-cwl-collection-cache'  closes #12568

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12538-crunchrun-kill-arvmount' closes #12538
Peter Amstutz [Wed, 8 Nov 2017 17:02:02 +0000 (12:02 -0500)]
Merge branch '12538-crunchrun-kill-arvmount' closes #12538

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12306-dont-stat-mounts'
Tom Clegg [Wed, 8 Nov 2017 16:07:58 +0000 (11:07 -0500)]
Merge branch '12306-dont-stat-mounts'

fixes #12306

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12538: Put cleanup back into separate defer
Peter Amstutz [Wed, 8 Nov 2017 15:36:37 +0000 (10:36 -0500)]
12538: Put cleanup back into separate defer

* Always report unmount command

* Send kill signal to arv-mount on timeout (but don't wait for it)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor crunch-run shutdown
Peter Amstutz [Tue, 7 Nov 2017 15:14:40 +0000 (10:14 -0500)]
12538: Refactor crunch-run shutdown

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor and assume arv-mount --unmount does all the hard work.
Peter Amstutz [Mon, 6 Nov 2017 21:41:05 +0000 (16:41 -0500)]
12538: Refactor and assume arv-mount --unmount does all the hard work.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Refactor force-unmount behavior.
Peter Amstutz [Fri, 3 Nov 2017 20:14:42 +0000 (16:14 -0400)]
12538: Refactor force-unmount behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Fix tests.
Peter Amstutz [Fri, 3 Nov 2017 19:04:00 +0000 (15:04 -0400)]
12538: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12538: Make handling of arv-mount process more robust
Peter Amstutz [Fri, 3 Nov 2017 15:34:46 +0000 (11:34 -0400)]
12538: Make handling of arv-mount process more robust

* If initial "fusermount -u -z" of arv-mount dir times out, use "arv-mount --unmount"

* If that doesn't work, send sigkill to arv-mount and then run "fusermount -u
  -z" again

* Refactor "stop container if arv-mount terminates" behavior

* Make arv-mount logging channel work as intended and enable crunchstat for arv-mount.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoForce kill arv-mount wip
Peter Amstutz [Thu, 2 Nov 2017 21:52:58 +0000 (17:52 -0400)]
Force kill arv-mount wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12306: Resolve any readable symlinks on unmount path.
Tom Clegg [Tue, 7 Nov 2017 23:10:13 +0000 (18:10 -0500)]
12306: Resolve any readable symlinks on unmount path.

This avoids getting stuck in realpath/lstat in certain cases: e.g.,
the mount point is $HOME/keep, $HOME is "/home/foo", and "/home" is a
symlink to "/data/home".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12568: Reduce min_entries
Peter Amstutz [Tue, 7 Nov 2017 20:52:08 +0000 (15:52 -0500)]
12568: Reduce min_entries

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Update test to check which collection gets evicted
Peter Amstutz [Tue, 7 Nov 2017 18:58:16 +0000 (13:58 -0500)]
12568: Update test to check which collection gets evicted

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Add test for collection cache management behavior
Peter Amstutz [Tue, 7 Nov 2017 18:52:28 +0000 (13:52 -0500)]
12568: Add test for collection cache management behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12568: Mange collection cache to avoid excessive memory usage.
Peter Amstutz [Tue, 7 Nov 2017 16:43:16 +0000 (11:43 -0500)]
12568: Mange collection cache to avoid excessive memory usage.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12306: Resolve relative paths early, to avoid realpath() later.
Tom Clegg [Mon, 6 Nov 2017 21:44:07 +0000 (16:44 -0500)]
12306: Resolve relative paths early, to avoid realpath() later.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '11994-no-sysfs-fuse'
Tom Clegg [Mon, 6 Nov 2017 21:11:02 +0000 (16:11 -0500)]
Merge branch '11994-no-sysfs-fuse'

fixes #11994

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12533-token-n-ip-logging'
Lucas Di Pentima [Mon, 6 Nov 2017 20:15:49 +0000 (17:15 -0300)]
Merge branch '12533-token-n-ip-logging'
Closes #12533

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12533: Use @remote_ip to take into account the reverse proxy
Lucas Di Pentima [Mon, 6 Nov 2017 20:13:57 +0000 (17:13 -0300)]
12533: Use @remote_ip to take into account the reverse proxy

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago11994: Fix wrong class in "except ENOENT" handler.
Tom Clegg [Mon, 6 Nov 2017 18:16:02 +0000 (13:16 -0500)]
11994: Fix wrong class in "except ENOENT" handler.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12306: Avoid calling realpath/lstat unnecessarily during --unmount.
Tom Clegg [Fri, 3 Nov 2017 18:18:36 +0000 (14:18 -0400)]
12306: Avoid calling realpath/lstat unnecessarily during --unmount.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12533: Add client IP address & authorization token UUID to the logs.
Lucas Di Pentima [Fri, 3 Nov 2017 20:05:16 +0000 (17:05 -0300)]
12533: Add client IP address & authorization token UUID to the logs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago11220: Fix test setup.
Tom Clegg [Fri, 3 Nov 2017 18:46:29 +0000 (14:46 -0400)]
11220: Fix test setup.

refs #11220

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12551-check-open-errors'
Tom Clegg [Fri, 3 Nov 2017 17:18:40 +0000 (13:18 -0400)]
Merge branch '12551-check-open-errors'

fixes #12551

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '11220-manifest-fetch-error'
Tom Clegg [Fri, 3 Nov 2017 15:27:38 +0000 (11:27 -0400)]
Merge branch '11220-manifest-fetch-error'

closes #11220

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12551: Check for errors after calling open().
Tom Clegg [Fri, 3 Nov 2017 14:45:04 +0000 (10:45 -0400)]
12551: Check for errors after calling open().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12546-skip-container-children'
Tom Clegg [Thu, 2 Nov 2017 19:55:26 +0000 (15:55 -0400)]
Merge branch '12546-skip-container-children'

fixes #12546

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12478-duration-fmt'
Tom Clegg [Thu, 2 Nov 2017 19:45:24 +0000 (15:45 -0400)]
Merge branch '12478-duration-fmt'

fixes #12478

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12546: Update help message.
Tom Clegg [Thu, 2 Nov 2017 19:09:28 +0000 (15:09 -0400)]
12546: Update help message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12546: Obey --skip-child-jobs flag for containers, too.
Tom Clegg [Thu, 2 Nov 2017 19:05:34 +0000 (15:05 -0400)]
12546: Obey --skip-child-jobs flag for containers, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12478: Fix time-offset formatting + use more natural tick intervals.
Tom Clegg [Thu, 2 Nov 2017 17:59:42 +0000 (13:59 -0400)]
12478: Fix time-offset formatting + use more natural tick intervals.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12462-search-hyphen'
Tom Clegg [Thu, 2 Nov 2017 16:04:31 +0000 (12:04 -0400)]
Merge branch '12462-search-hyphen'

fixes #12462

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12533-row-lock-contention'
Lucas Di Pentima [Thu, 2 Nov 2017 15:56:01 +0000 (12:56 -0300)]
Merge branch '12533-row-lock-contention'
Refs #12533

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch 'master' into 12533-row-lock-contention
Lucas Di Pentima [Thu, 2 Nov 2017 15:54:56 +0000 (12:54 -0300)]
Merge branch 'master' into 12533-row-lock-contention

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12167-workbench-request-id'
Tom Clegg [Thu, 2 Nov 2017 15:06:25 +0000 (11:06 -0400)]
Merge branch '12167-workbench-request-id'

refs #12167

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMerge branch '12018-sync-groups-tool'
Lucas Di Pentima [Thu, 2 Nov 2017 14:57:45 +0000 (11:57 -0300)]
Merge branch '12018-sync-groups-tool'
Closes #12018

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch 'master' into 12018-sync-groups-tool
Lucas Di Pentima [Thu, 2 Nov 2017 14:56:50 +0000 (11:56 -0300)]
Merge branch 'master' into 12018-sync-groups-tool

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12183-crunch-run-symlinks' closes #12183
Peter Amstutz [Thu, 2 Nov 2017 14:53:17 +0000 (10:53 -0400)]
Merge branch '12183-crunch-run-symlinks'  closes #12183

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Tweak documenation for clarity.
Peter Amstutz [Thu, 2 Nov 2017 14:52:15 +0000 (10:52 -0400)]
12183: Tweak documenation for clarity.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12533: Do not update the API Client Authorization record to avoid
Lucas Di Pentima [Thu, 2 Nov 2017 14:48:03 +0000 (11:48 -0300)]
12533: Do not update the API Client Authorization record to avoid
row contention when lots of jobs are queued with the same token.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12183: Fix section headers.
Peter Amstutz [Thu, 2 Nov 2017 14:02:55 +0000 (10:02 -0400)]
12183: Fix section headers.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Fix renamed symbol
Peter Amstutz [Thu, 2 Nov 2017 13:32:40 +0000 (09:32 -0400)]
12183: Fix renamed symbol

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Update documentation.
Peter Amstutz [Wed, 1 Nov 2017 21:10:20 +0000 (17:10 -0400)]
12183: Update documentation.

Improve error message about follow symlink chains that are too long.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Handle circular directory symlinks
Peter Amstutz [Mon, 16 Oct 2017 20:39:36 +0000 (16:39 -0400)]
12183: Handle circular directory symlinks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Fix tests.
Peter Amstutz [Mon, 16 Oct 2017 19:26:55 +0000 (15:26 -0400)]
12183: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Refactor derefOutputSymlink into its own method.
Peter Amstutz [Mon, 16 Oct 2017 18:22:58 +0000 (14:22 -0400)]
12183:  Refactor derefOutputSymlink into its own method.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Add comment about purpose of "relocated"
Peter Amstutz [Thu, 5 Oct 2017 14:40:04 +0000 (10:40 -0400)]
12183: Add comment about purpose of "relocated"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Refactor file upload so there is one walk to handle both symlinks and
Peter Amstutz [Thu, 5 Oct 2017 12:47:04 +0000 (08:47 -0400)]
12183: Refactor file upload so there is one walk to handle both symlinks and
regular files.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Simplify walking symlinks in upload.
Peter Amstutz [Mon, 2 Oct 2017 20:07:18 +0000 (16:07 -0400)]
12183: Simplify walking symlinks in upload.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Improve error handling if unable to delete symlinks marked for deletion.
Peter Amstutz [Mon, 2 Oct 2017 19:34:38 +0000 (15:34 -0400)]
12183: Improve error handling if unable to delete symlinks marked for deletion.

Add unit test for EvalSymlinks.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Support uploading symlinks to directories.
Peter Amstutz [Fri, 29 Sep 2017 20:06:59 +0000 (16:06 -0400)]
12183: Support uploading symlinks to directories.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago12183: Evaluate symlinks to handle chained links and absolute paths.
Peter Amstutz [Fri, 29 Sep 2017 19:11:46 +0000 (15:11 -0400)]
12183: Evaluate symlinks to handle chained links and absolute paths.

Ensure that all symlinks either resolve within the output directory or to a
keep mount.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years agoMerge branch '12494-arv-git-cors' closes #12494
Peter Amstutz [Wed, 1 Nov 2017 16:54:02 +0000 (12:54 -0400)]
Merge branch '12494-arv-git-cors' closes #12494

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

7 years ago11220: Avoid implying that manifests can be loaded from Keep.
Tom Clegg [Wed, 1 Nov 2017 15:36:35 +0000 (11:36 -0400)]
11220: Avoid implying that manifests can be loaded from Keep.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago11220: Fix test case for new API call sequence.
Tom Clegg [Wed, 1 Nov 2017 15:31:31 +0000 (11:31 -0400)]
11220: Fix test case for new API call sequence.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago11220: Remove test for unsupported case.
Tom Clegg [Wed, 1 Nov 2017 15:31:11 +0000 (11:31 -0400)]
11220: Remove test for unsupported case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12494: Add CORS test.
Tom Clegg [Wed, 1 Nov 2017 07:58:36 +0000 (03:58 -0400)]
12494: Add CORS test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12167: Include a request ID in each API response and log entry.
Tom Clegg [Tue, 31 Oct 2017 21:27:41 +0000 (17:27 -0400)]
12167: Include a request ID in each API response and log entry.

Use the value of the X-Request-Id request header if given, otherwise
generate one.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12018: Added API server start/stop calls to test suite.
Lucas Di Pentima [Tue, 31 Oct 2017 21:07:01 +0000 (18:07 -0300)]
12018: Added API server start/stop calls to test suite.
Removed manual database cleanup in favour of using the database
reset endpoint after every test run.
Moved tool config set up code from the suite set up call to the test
set up, because the database get reset to fixture state after every
test run.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12167: Improve request-id tracking in Workbench.
Tom Clegg [Tue, 31 Oct 2017 03:07:52 +0000 (23:07 -0400)]
12167: Improve request-id tracking in Workbench.

Include current request ID in API requests via X-Request-Id header
instead of a form parameter.  (Also move Arvados API token from form
to Authorization request header.)

Always log request ID as its own field, instead of adding it to params
(where it might get truncated before being logged).

Use new request ID format: "req-" + 20-random-alphanum-chars.

Use an around_filter to set/clear the current request ID in one place,
instead of setting it in controller code and clearing it in logging
code.

Return request ID to browser via X-Request-Id header.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12462: Treat "/" and "-" as word chars, too.
Tom Clegg [Tue, 31 Oct 2017 19:31:33 +0000 (15:31 -0400)]
12462: Treat "/" and "-" as word chars, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years ago12377: Arvbox support for Composer
Peter Amstutz [Tue, 31 Oct 2017 16:40:27 +0000 (12:40 -0400)]
12377: Arvbox support for Composer

7 years ago12018: Set up fields per record, so it won't try to guess it from
Lucas Di Pentima [Tue, 31 Oct 2017 14:00:28 +0000 (11:00 -0300)]
12018: Set up fields per record, so it won't try to guess it from
the first one being read.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12018: Enhanced readability of ProcessFile() function.
Lucas Di Pentima [Tue, 31 Oct 2017 02:17:42 +0000 (23:17 -0300)]
12018: Enhanced readability of ProcessFile() function.
Added test using usernames instead of emails.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoWhile we are at it, also throw out the broken progress bar and replace
Ward Vandewege [Tue, 31 Oct 2017 01:26:06 +0000 (21:26 -0400)]
While we are at it, also throw out the broken progress bar and replace
it with an indication that the workflow is running.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years agoRip out a bunch of very very slow workbench code that doesn't seem to
Ward Vandewege [Tue, 31 Oct 2017 01:11:33 +0000 (21:11 -0400)]
Rip out a bunch of very very slow workbench code that doesn't seem to
serve much purpose.

refs #12513

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

7 years ago12018: Enhanced error message when having a parse error.
Lucas Di Pentima [Mon, 30 Oct 2017 22:42:06 +0000 (19:42 -0300)]
12018: Enhanced error message when having a parse error.
Added test cleanup.
Added test to confirm records with empty fields are ignored.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12018: Moved Link & LinkList types to GoSDK arvados package
Lucas Di Pentima [Mon, 30 Oct 2017 21:17:52 +0000 (18:17 -0300)]
12018: Moved Link & LinkList types to GoSDK arvados package

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years ago12018: Fixed positional argument handling.
Lucas Di Pentima [Mon, 30 Oct 2017 14:35:58 +0000 (11:35 -0300)]
12018: Fixed positional argument handling.
Fixed & added tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

7 years agoMerge branch '12462-search-periods'
Tom Clegg [Mon, 30 Oct 2017 13:19:36 +0000 (09:19 -0400)]
Merge branch '12462-search-periods'

refs #12462

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

7 years agoMigration for indexes on containers and container_requests.
Peter Amstutz [Fri, 27 Oct 2017 18:42:03 +0000 (14:42 -0400)]
Migration for indexes on containers and container_requests.

refs #12500

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>