arvados.git
6 years ago12483: Add webdav COPY method.
Tom Clegg [Tue, 28 Nov 2017 15:17:55 +0000 (10:17 -0500)]
12483: Add webdav COPY method.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Use io.Seek* constants instead of deprecated os.SEEK_*.
Tom Clegg [Tue, 28 Nov 2017 14:20:51 +0000 (09:20 -0500)]
12483: Use io.Seek* constants instead of deprecated os.SEEK_*.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Remove dead code, update tests.
Tom Clegg [Mon, 27 Nov 2017 23:23:55 +0000 (18:23 -0500)]
12483: Remove dead code, update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Error 405 if writing to PDH, 403 if lacking write permission.
Tom Clegg [Mon, 27 Nov 2017 18:12:53 +0000 (13:12 -0500)]
12483: Error 405 if writing to PDH, 403 if lacking write permission.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix loading error details from API error response.
Tom Clegg [Mon, 27 Nov 2017 18:10:51 +0000 (13:10 -0500)]
12483: Fix loading error details from API error response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Recover from errant empty dirs where GOPATH symlink goes.
Tom Clegg [Mon, 27 Nov 2017 18:08:28 +0000 (13:08 -0500)]
12483: Recover from errant empty dirs where GOPATH symlink goes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix cadaver tests.
Tom Clegg [Wed, 22 Nov 2017 17:45:08 +0000 (12:45 -0500)]
12483: Fix cadaver tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix multiple WriteHeader(). Skip updates on readonly methods.
Tom Clegg [Wed, 22 Nov 2017 17:32:38 +0000 (12:32 -0500)]
12483: Fix multiple WriteHeader(). Skip updates on readonly methods.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Update comments.
Tom Clegg [Wed, 22 Nov 2017 15:00:14 +0000 (10:00 -0500)]
12483: Update comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Drop writableSegment interface.
Tom Clegg [Wed, 22 Nov 2017 14:57:21 +0000 (09:57 -0500)]
12483: Drop writableSegment interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Rename extent->segment, file->filehandle.
Tom Clegg [Wed, 22 Nov 2017 14:56:05 +0000 (09:56 -0500)]
12483: Rename extent->segment, file->filehandle.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix receiver type.
Tom Clegg [Tue, 21 Nov 2017 08:03:55 +0000 (03:03 -0500)]
12483: Fix receiver type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Delay flushing collection until successful http response.
Tom Clegg [Tue, 21 Nov 2017 06:59:27 +0000 (01:59 -0500)]
12483: Delay flushing collection until successful http response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Make webdav rename/remove work. Tidy up code.
Tom Clegg [Mon, 20 Nov 2017 23:43:46 +0000 (18:43 -0500)]
12483: Make webdav rename/remove work. Tidy up code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: More loading speed.
Tom Clegg [Mon, 20 Nov 2017 17:11:18 +0000 (12:11 -0500)]
12483: More loading speed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Speed up manifest loading.
Tom Clegg [Mon, 20 Nov 2017 15:30:05 +0000 (10:30 -0500)]
12483: Speed up manifest loading.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Allow seek and write beyond EOF.
Tom Clegg [Sat, 18 Nov 2017 06:44:55 +0000 (01:44 -0500)]
12483: Allow seek and write beyond EOF.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Avoid empty segments.
Tom Clegg [Sat, 18 Nov 2017 05:19:56 +0000 (00:19 -0500)]
12483: Avoid empty segments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add RemoveAll() to CollectionFileSystem.
Tom Clegg [Sat, 18 Nov 2017 01:53:38 +0000 (20:53 -0500)]
12483: Add RemoveAll() to CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Enable rename via webdav.
Tom Clegg [Fri, 17 Nov 2017 16:38:13 +0000 (11:38 -0500)]
12483: Enable rename via webdav.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add Rename(old,new) to CollectionFileSystem.
Tom Clegg [Fri, 17 Nov 2017 16:06:24 +0000 (11:06 -0500)]
12483: Add Rename(old,new) to CollectionFileSystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add comments.
Tom Clegg [Fri, 17 Nov 2017 08:04:00 +0000 (03:04 -0500)]
12483: Add comments.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support O_TRUNC flag.
Tom Clegg [Thu, 16 Nov 2017 21:58:18 +0000 (16:58 -0500)]
12483: Support O_TRUNC flag.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Compress adjacent segments when writing filetokens.
Tom Clegg [Thu, 16 Nov 2017 21:19:58 +0000 (16:19 -0500)]
12483: Compress adjacent segments when writing filetokens.

"0:100:foo 100:100:foo 200:100:foo" --> "0:300:foo"

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support O_APPEND. Check for invalid/unsupported file modes.
Tom Clegg [Thu, 16 Nov 2017 21:06:20 +0000 (16:06 -0500)]
12483: Support O_APPEND. Check for invalid/unsupported file modes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Support file create/write via webdav.
Tom Clegg [Wed, 15 Nov 2017 18:24:16 +0000 (13:24 -0500)]
12483: Support file create/write via webdav.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix makeParentDirs bug when higher levels already exist.
Tom Clegg [Wed, 15 Nov 2017 18:14:13 +0000 (13:14 -0500)]
12483: Fix makeParentDirs bug when higher levels already exist.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Persist empty files.
Tom Clegg [Wed, 15 Nov 2017 18:01:29 +0000 (13:01 -0500)]
12483: Persist empty files.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Track memory use. Flush filled blocks while writing.
Tom Clegg [Tue, 14 Nov 2017 20:17:12 +0000 (15:17 -0500)]
12483: Track memory use. Flush filled blocks while writing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Fix missing Rlock() on inode accesses.
Tom Clegg [Tue, 14 Nov 2017 14:34:13 +0000 (09:34 -0500)]
12483: Fix missing Rlock() on inode accesses.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Return error on impossible manifest.
Tom Clegg [Tue, 14 Nov 2017 05:55:58 +0000 (00:55 -0500)]
12483: Return error on impossible manifest.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Avoid write/marshal race. Remove dead code.
Tom Clegg [Tue, 14 Nov 2017 05:04:24 +0000 (00:04 -0500)]
12483: Avoid write/marshal race. Remove dead code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12453: Persist written data to keep.
Tom Clegg [Mon, 13 Nov 2017 21:50:22 +0000 (16:50 -0500)]
12453: Persist written data to keep.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Remove unnecessary OpenFile() from inode interface.
Tom Clegg [Mon, 13 Nov 2017 21:50:07 +0000 (16:50 -0500)]
12483: Remove unnecessary OpenFile() from inode interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add Mkdir(), Remove().
Tom Clegg [Mon, 13 Nov 2017 15:36:50 +0000 (10:36 -0500)]
12483: Add Mkdir(), Remove().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add MarshalManifest().
Tom Clegg [Sun, 12 Nov 2017 08:31:16 +0000 (03:31 -0500)]
12483: Add MarshalManifest().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Remove unnecessary repack.
Tom Clegg [Sun, 12 Nov 2017 08:30:18 +0000 (03:30 -0500)]
12483: Remove unnecessary repack.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add tests for concurrent writers and random write sequences.
Tom Clegg [Sat, 11 Nov 2017 21:48:54 +0000 (16:48 -0500)]
12483: Add tests for concurrent writers and random write sequences.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add (File)Truncate().
Tom Clegg [Sat, 11 Nov 2017 20:32:47 +0000 (15:32 -0500)]
12483: Add (File)Truncate().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Simplify extent packing, reduce type casting.
Tom Clegg [Sat, 11 Nov 2017 18:26:37 +0000 (13:26 -0500)]
12483: Simplify extent packing, reduce type casting.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Implement some writable cases.
Tom Clegg [Sat, 11 Nov 2017 08:30:24 +0000 (03:30 -0500)]
12483: Implement some writable cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Add basic file writing tests.
Tom Clegg [Fri, 10 Nov 2017 21:34:28 +0000 (16:34 -0500)]
12483: Add basic file writing tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12483: Rewrite collection filesystem.
Tom Clegg [Thu, 9 Nov 2017 18:24:34 +0000 (13:24 -0500)]
12483: Rewrite collection filesystem.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12568-cwl-collection-cache' closes #12568
Peter Amstutz [Wed, 8 Nov 2017 17:08:54 +0000 (12:08 -0500)]
Merge branch '12568-cwl-collection-cache'  closes #12568

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12538-crunchrun-kill-arvmount' closes #12538
Peter Amstutz [Wed, 8 Nov 2017 17:02:02 +0000 (12:02 -0500)]
Merge branch '12538-crunchrun-kill-arvmount' closes #12538

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12306-dont-stat-mounts'
Tom Clegg [Wed, 8 Nov 2017 16:07:58 +0000 (11:07 -0500)]
Merge branch '12306-dont-stat-mounts'

fixes #12306

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12538: Put cleanup back into separate defer
Peter Amstutz [Wed, 8 Nov 2017 15:36:37 +0000 (10:36 -0500)]
12538: Put cleanup back into separate defer

* Always report unmount command

* Send kill signal to arv-mount on timeout (but don't wait for it)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12538: Refactor crunch-run shutdown
Peter Amstutz [Tue, 7 Nov 2017 15:14:40 +0000 (10:14 -0500)]
12538: Refactor crunch-run shutdown

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12538: Refactor and assume arv-mount --unmount does all the hard work.
Peter Amstutz [Mon, 6 Nov 2017 21:41:05 +0000 (16:41 -0500)]
12538: Refactor and assume arv-mount --unmount does all the hard work.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12538: Refactor force-unmount behavior.
Peter Amstutz [Fri, 3 Nov 2017 20:14:42 +0000 (16:14 -0400)]
12538: Refactor force-unmount behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12538: Fix tests.
Peter Amstutz [Fri, 3 Nov 2017 19:04:00 +0000 (15:04 -0400)]
12538: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12538: Make handling of arv-mount process more robust
Peter Amstutz [Fri, 3 Nov 2017 15:34:46 +0000 (11:34 -0400)]
12538: Make handling of arv-mount process more robust

* If initial "fusermount -u -z" of arv-mount dir times out, use "arv-mount --unmount"

* If that doesn't work, send sigkill to arv-mount and then run "fusermount -u
  -z" again

* Refactor "stop container if arv-mount terminates" behavior

* Make arv-mount logging channel work as intended and enable crunchstat for arv-mount.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoForce kill arv-mount wip
Peter Amstutz [Thu, 2 Nov 2017 21:52:58 +0000 (17:52 -0400)]
Force kill arv-mount wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12306: Resolve any readable symlinks on unmount path.
Tom Clegg [Tue, 7 Nov 2017 23:10:13 +0000 (18:10 -0500)]
12306: Resolve any readable symlinks on unmount path.

This avoids getting stuck in realpath/lstat in certain cases: e.g.,
the mount point is $HOME/keep, $HOME is "/home/foo", and "/home" is a
symlink to "/data/home".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12568: Reduce min_entries
Peter Amstutz [Tue, 7 Nov 2017 20:52:08 +0000 (15:52 -0500)]
12568: Reduce min_entries

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12568: Update test to check which collection gets evicted
Peter Amstutz [Tue, 7 Nov 2017 18:58:16 +0000 (13:58 -0500)]
12568: Update test to check which collection gets evicted

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12568: Add test for collection cache management behavior
Peter Amstutz [Tue, 7 Nov 2017 18:52:28 +0000 (13:52 -0500)]
12568: Add test for collection cache management behavior

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12568: Mange collection cache to avoid excessive memory usage.
Peter Amstutz [Tue, 7 Nov 2017 16:43:16 +0000 (11:43 -0500)]
12568: Mange collection cache to avoid excessive memory usage.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12306: Resolve relative paths early, to avoid realpath() later.
Tom Clegg [Mon, 6 Nov 2017 21:44:07 +0000 (16:44 -0500)]
12306: Resolve relative paths early, to avoid realpath() later.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11994-no-sysfs-fuse'
Tom Clegg [Mon, 6 Nov 2017 21:11:02 +0000 (16:11 -0500)]
Merge branch '11994-no-sysfs-fuse'

fixes #11994

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12533-token-n-ip-logging'
Lucas Di Pentima [Mon, 6 Nov 2017 20:15:49 +0000 (17:15 -0300)]
Merge branch '12533-token-n-ip-logging'
Closes #12533

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12533: Use @remote_ip to take into account the reverse proxy
Lucas Di Pentima [Mon, 6 Nov 2017 20:13:57 +0000 (17:13 -0300)]
12533: Use @remote_ip to take into account the reverse proxy

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11994: Fix wrong class in "except ENOENT" handler.
Tom Clegg [Mon, 6 Nov 2017 18:16:02 +0000 (13:16 -0500)]
11994: Fix wrong class in "except ENOENT" handler.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12306: Avoid calling realpath/lstat unnecessarily during --unmount.
Tom Clegg [Fri, 3 Nov 2017 18:18:36 +0000 (14:18 -0400)]
12306: Avoid calling realpath/lstat unnecessarily during --unmount.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12533: Add client IP address & authorization token UUID to the logs.
Lucas Di Pentima [Fri, 3 Nov 2017 20:05:16 +0000 (17:05 -0300)]
12533: Add client IP address & authorization token UUID to the logs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11220: Fix test setup.
Tom Clegg [Fri, 3 Nov 2017 18:46:29 +0000 (14:46 -0400)]
11220: Fix test setup.

refs #11220

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12551-check-open-errors'
Tom Clegg [Fri, 3 Nov 2017 17:18:40 +0000 (13:18 -0400)]
Merge branch '12551-check-open-errors'

fixes #12551

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11220-manifest-fetch-error'
Tom Clegg [Fri, 3 Nov 2017 15:27:38 +0000 (11:27 -0400)]
Merge branch '11220-manifest-fetch-error'

closes #11220

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12551: Check for errors after calling open().
Tom Clegg [Fri, 3 Nov 2017 14:45:04 +0000 (10:45 -0400)]
12551: Check for errors after calling open().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12546-skip-container-children'
Tom Clegg [Thu, 2 Nov 2017 19:55:26 +0000 (15:55 -0400)]
Merge branch '12546-skip-container-children'

fixes #12546

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12478-duration-fmt'
Tom Clegg [Thu, 2 Nov 2017 19:45:24 +0000 (15:45 -0400)]
Merge branch '12478-duration-fmt'

fixes #12478

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12546: Update help message.
Tom Clegg [Thu, 2 Nov 2017 19:09:28 +0000 (15:09 -0400)]
12546: Update help message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12546: Obey --skip-child-jobs flag for containers, too.
Tom Clegg [Thu, 2 Nov 2017 19:05:34 +0000 (15:05 -0400)]
12546: Obey --skip-child-jobs flag for containers, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12478: Fix time-offset formatting + use more natural tick intervals.
Tom Clegg [Thu, 2 Nov 2017 17:59:42 +0000 (13:59 -0400)]
12478: Fix time-offset formatting + use more natural tick intervals.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12462-search-hyphen'
Tom Clegg [Thu, 2 Nov 2017 16:04:31 +0000 (12:04 -0400)]
Merge branch '12462-search-hyphen'

fixes #12462

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12533-row-lock-contention'
Lucas Di Pentima [Thu, 2 Nov 2017 15:56:01 +0000 (12:56 -0300)]
Merge branch '12533-row-lock-contention'
Refs #12533

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'master' into 12533-row-lock-contention
Lucas Di Pentima [Thu, 2 Nov 2017 15:54:56 +0000 (12:54 -0300)]
Merge branch 'master' into 12533-row-lock-contention

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12167-workbench-request-id'
Tom Clegg [Thu, 2 Nov 2017 15:06:25 +0000 (11:06 -0400)]
Merge branch '12167-workbench-request-id'

refs #12167

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12018-sync-groups-tool'
Lucas Di Pentima [Thu, 2 Nov 2017 14:57:45 +0000 (11:57 -0300)]
Merge branch '12018-sync-groups-tool'
Closes #12018

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'master' into 12018-sync-groups-tool
Lucas Di Pentima [Thu, 2 Nov 2017 14:56:50 +0000 (11:56 -0300)]
Merge branch 'master' into 12018-sync-groups-tool

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12183-crunch-run-symlinks' closes #12183
Peter Amstutz [Thu, 2 Nov 2017 14:53:17 +0000 (10:53 -0400)]
Merge branch '12183-crunch-run-symlinks'  closes #12183

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Tweak documenation for clarity.
Peter Amstutz [Thu, 2 Nov 2017 14:52:15 +0000 (10:52 -0400)]
12183: Tweak documenation for clarity.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12533: Do not update the API Client Authorization record to avoid
Lucas Di Pentima [Thu, 2 Nov 2017 14:48:03 +0000 (11:48 -0300)]
12533: Do not update the API Client Authorization record to avoid
row contention when lots of jobs are queued with the same token.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12183: Fix section headers.
Peter Amstutz [Thu, 2 Nov 2017 14:02:55 +0000 (10:02 -0400)]
12183: Fix section headers.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Fix renamed symbol
Peter Amstutz [Thu, 2 Nov 2017 13:32:40 +0000 (09:32 -0400)]
12183: Fix renamed symbol

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Update documentation.
Peter Amstutz [Wed, 1 Nov 2017 21:10:20 +0000 (17:10 -0400)]
12183: Update documentation.

Improve error message about follow symlink chains that are too long.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Handle circular directory symlinks
Peter Amstutz [Mon, 16 Oct 2017 20:39:36 +0000 (16:39 -0400)]
12183: Handle circular directory symlinks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Fix tests.
Peter Amstutz [Mon, 16 Oct 2017 19:26:55 +0000 (15:26 -0400)]
12183: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Refactor derefOutputSymlink into its own method.
Peter Amstutz [Mon, 16 Oct 2017 18:22:58 +0000 (14:22 -0400)]
12183:  Refactor derefOutputSymlink into its own method.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Add comment about purpose of "relocated"
Peter Amstutz [Thu, 5 Oct 2017 14:40:04 +0000 (10:40 -0400)]
12183: Add comment about purpose of "relocated"

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Refactor file upload so there is one walk to handle both symlinks and
Peter Amstutz [Thu, 5 Oct 2017 12:47:04 +0000 (08:47 -0400)]
12183: Refactor file upload so there is one walk to handle both symlinks and
regular files.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Simplify walking symlinks in upload.
Peter Amstutz [Mon, 2 Oct 2017 20:07:18 +0000 (16:07 -0400)]
12183: Simplify walking symlinks in upload.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Improve error handling if unable to delete symlinks marked for deletion.
Peter Amstutz [Mon, 2 Oct 2017 19:34:38 +0000 (15:34 -0400)]
12183: Improve error handling if unable to delete symlinks marked for deletion.

Add unit test for EvalSymlinks.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Support uploading symlinks to directories.
Peter Amstutz [Fri, 29 Sep 2017 20:06:59 +0000 (16:06 -0400)]
12183: Support uploading symlinks to directories.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12183: Evaluate symlinks to handle chained links and absolute paths.
Peter Amstutz [Fri, 29 Sep 2017 19:11:46 +0000 (15:11 -0400)]
12183: Evaluate symlinks to handle chained links and absolute paths.

Ensure that all symlinks either resolve within the output directory or to a
keep mount.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12494-arv-git-cors' closes #12494
Peter Amstutz [Wed, 1 Nov 2017 16:54:02 +0000 (12:54 -0400)]
Merge branch '12494-arv-git-cors' closes #12494

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11220: Avoid implying that manifests can be loaded from Keep.
Tom Clegg [Wed, 1 Nov 2017 15:36:35 +0000 (11:36 -0400)]
11220: Avoid implying that manifests can be loaded from Keep.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11220: Fix test case for new API call sequence.
Tom Clegg [Wed, 1 Nov 2017 15:31:31 +0000 (11:31 -0400)]
11220: Fix test case for new API call sequence.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11220: Remove test for unsupported case.
Tom Clegg [Wed, 1 Nov 2017 15:31:11 +0000 (11:31 -0400)]
11220: Remove test for unsupported case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12494: Add CORS test.
Tom Clegg [Wed, 1 Nov 2017 07:58:36 +0000 (03:58 -0400)]
12494: Add CORS test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>