arvados.git
6 years ago12018: Splitted api client calls to their own funcs.
Lucas Di Pentima [Wed, 25 Oct 2017 14:22:58 +0000 (11:22 -0300)]
12018: Splitted api client calls to their own funcs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Added basic tests
Lucas Di Pentima [Wed, 25 Oct 2017 02:54:30 +0000 (23:54 -0300)]
12018: Added basic tests

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Move remote group retrieving and member removal code to
Lucas Di Pentima [Wed, 25 Oct 2017 01:11:31 +0000 (22:11 -0300)]
12018: Move remote group retrieving and member removal code to
separate functions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Split setup code on its own functions
Lucas Di Pentima [Tue, 24 Oct 2017 21:37:42 +0000 (18:37 -0300)]
12018: Split setup code on its own functions

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: User, UserList, Group & GroupList types go into Go SDK.
Lucas Di Pentima [Tue, 24 Oct 2017 12:07:04 +0000 (09:07 -0300)]
12018: User, UserList, Group & GroupList types go into Go SDK.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Group membership need two-way links between the user and
Lucas Di Pentima [Mon, 23 Oct 2017 23:56:13 +0000 (20:56 -0300)]
12018: Group membership need two-way links between the user and
the group. On group members loading, check that both links exist.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Do not tag remote groups with tag links. Use the parent
Lucas Di Pentima [Mon, 23 Oct 2017 22:30:54 +0000 (19:30 -0300)]
12018: Do not tag remote groups with tag links. Use the parent
group as an indication that the group is synced from a remote
source.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Stop using arvadosclient in favor of arvados package for API
Lucas Di Pentima [Mon, 23 Oct 2017 19:36:54 +0000 (16:36 -0300)]
12018: Stop using arvadosclient in favor of arvados package for API
server access.
Drop "-retries" parameter as it's not supported on the newer SDK.
Use specific types for resources instead of [][]string.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: resourceList interface simplification. Code re-stying.
Lucas Di Pentima [Fri, 20 Oct 2017 14:33:04 +0000 (11:33 -0300)]
12018: resourceList interface simplification. Code re-stying.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Simplify ListAll()
Lucas Di Pentima [Fri, 20 Oct 2017 02:32:59 +0000 (23:32 -0300)]
12018: Simplify ListAll()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Manage two way links for group memberships.
Lucas Di Pentima [Fri, 20 Oct 2017 00:41:25 +0000 (21:41 -0300)]
12018: Manage two way links for group memberships.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Various updates, described below:
Lucas Di Pentima [Thu, 19 Oct 2017 22:06:47 +0000 (19:06 -0300)]
12018: Various updates, described below:

* Code styling.
* Added system_user as permission links owner.
* Improved userId option handling.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Skip CSV register if one of its fields is empty
Lucas Di Pentima [Wed, 18 Oct 2017 22:39:14 +0000 (19:39 -0300)]
12018: Skip CSV register if one of its fields is empty

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Removed superfluous check
Lucas Di Pentima [Wed, 18 Oct 2017 21:36:22 +0000 (18:36 -0300)]
12018: Removed superfluous check

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Added flag to specify a parent group UUID. When not given,
Lucas Di Pentima [Wed, 18 Oct 2017 21:32:28 +0000 (18:32 -0300)]
12018: Added flag to specify a parent group UUID. When not given,
it tries to search for a default & system owned group, or create
one if none found.
Group List & Create operations adjusted to use the correct parent
group as owner.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Several changes, listed below:
Lucas Di Pentima [Wed, 18 Oct 2017 17:17:01 +0000 (14:17 -0300)]
12018: Several changes, listed below:
* Removed initial Python version.
* Removed superfluous input files checks and moved file opening
to be before any API calls, to avoid doing them is there's a problem
with the file.
* Added user, group & link types so they're populated by json decoding.
* Cleaned up ListAll() func so it can work with different resource
types.
* Changed usage of set style types from map[string]struct{} to be
map[string]bool to simplify membership checking.
* Corrected error messages to start with lowercase.
* Added more debug messages for -verbose mode.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Ignore compiled binary
Lucas Di Pentima [Wed, 11 Oct 2017 11:15:56 +0000 (08:15 -0300)]
12018: Ignore compiled binary

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Remote group synchronization tool. First draft.
Lucas Di Pentima [Wed, 11 Oct 2017 11:13:49 +0000 (08:13 -0300)]
12018: Remote group synchronization tool. First draft.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Avoid making duplicate api calls to add an already added user when
Lucas Di Pentima [Wed, 27 Sep 2017 03:44:47 +0000 (00:44 -0300)]
12018: Avoid making duplicate api calls to add an already added user when
the input file has the same line more than once.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12018: Added arv-sync-groups command, to synchronize groups membership
Lucas Di Pentima [Wed, 27 Sep 2017 03:14:20 +0000 (00:14 -0300)]
12018: Added arv-sync-groups command, to synchronize groups membership
from a CSV file.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12032-project-trash' refs #12032
Peter Amstutz [Mon, 25 Sep 2017 19:39:50 +0000 (15:39 -0400)]
Merge branch '12032-project-trash' refs #12032

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12271-fpm-cleanup-on-rpm-upgrade' closes #12271
Javier Bértoli [Mon, 25 Sep 2017 19:00:02 +0000 (16:00 -0300)]
Merge branch '12271-fpm-cleanup-on-rpm-upgrade' closes #12271

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@veritasgenetics.com>

6 years agoMerge branch '11068-cwl-missing-docker' closes #11068
Peter Amstutz [Mon, 25 Sep 2017 13:19:50 +0000 (09:19 -0400)]
Merge branch '11068-cwl-missing-docker' closes #11068

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12276-fuse-deadlock' refs #12276
Peter Amstutz [Sat, 23 Sep 2017 00:38:00 +0000 (20:38 -0400)]
Merge branch '12276-fuse-deadlock' refs #12276

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12276: Ensure dirent cache is disabled.
Tom Clegg [Fri, 22 Sep 2017 21:00:07 +0000 (17:00 -0400)]
12276: Ensure dirent cache is disabled.

Setting this to zero fixed flakiness in #7751, so we might still rely
on it being zero for reliability.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12276: Remove debug logging message.
Peter Amstutz [Fri, 22 Sep 2017 18:43:26 +0000 (14:43 -0400)]
12276: Remove debug logging message.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12276: Reduce number of spurious invalidations sent to kernel.
Peter Amstutz [Fri, 22 Sep 2017 15:21:07 +0000 (11:21 -0400)]
12276: Reduce number of spurious invalidations sent to kernel.

* New policy to only send invalidations on objects that have a nonzero
  kernel reference count.
* When clearing the contents of a CollectionDirectory, only invalidate the
  topmost directory entry, this should take care of invalidating all paths
  beneath it as well.
* Don't send invalidate_inode() when deleting an inode (by definition, the
  inode is unreferenced by the kernel).
* Remove allow_dirent_cache for now (this only relates to caching negative
  lookups, which we don't support).
* Set attribute attr_timeout based on the polling period of the underlying
  object.
* FuncToJSONFile sets allow_attr_cache = False instead of allow_dirent_cache =
  False (which is useless in this case).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12298-cancel-fail'
Tom Clegg [Fri, 22 Sep 2017 13:27:01 +0000 (09:27 -0400)]
Merge branch '12298-cancel-fail'

refs #12298

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11068: Catch OSError and give better error message when unable to execute Docker.
Peter Amstutz [Thu, 21 Sep 2017 18:04:27 +0000 (14:04 -0400)]
11068: Catch OSError and give better error message when unable to execute Docker.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12292-cwl-tmpdir-default' refs #12292
Peter Amstutz [Thu, 21 Sep 2017 15:38:49 +0000 (11:38 -0400)]
Merge branch '12292-cwl-tmpdir-default' refs #12292

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoInstall upstream pip to work around Debian bug, no issue #
Peter Amstutz [Mon, 18 Sep 2017 21:40:05 +0000 (17:40 -0400)]
Install upstream pip to work around Debian bug, no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12292: Find crunchrunner vars even when mixed into a partial line
Peter Amstutz [Wed, 20 Sep 2017 20:05:18 +0000 (16:05 -0400)]
12292: Find crunchrunner vars even when mixed into a partial line

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12292: Set defaults for tmpdir, outdir, keep
Peter Amstutz [Wed, 20 Sep 2017 19:24:34 +0000 (15:24 -0400)]
12292: Set defaults for tmpdir, outdir, keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoAdd parameter for FPM so it removes old directories on upgrade
Javier Bértoli [Wed, 20 Sep 2017 22:51:54 +0000 (19:51 -0300)]
Add parameter for FPM so it removes old directories on upgrade

refs #12271

6 years ago12298: Allow non-null log when cancelling an unrunnable container.
Tom Clegg [Wed, 20 Sep 2017 21:44:22 +0000 (17:44 -0400)]
12298: Allow non-null log when cancelling an unrunnable container.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix dashboard crash on uncommitted container request.
Tom Clegg [Wed, 20 Sep 2017 20:03:53 +0000 (16:03 -0400)]
Fix dashboard crash on uncommitted container request.

refs #12246

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12247-no-failed-checksum'
Tom Clegg [Wed, 20 Sep 2017 15:33:31 +0000 (11:33 -0400)]
Merge branch '12247-no-failed-checksum'

closes #12247

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12107-faster-git'
Tom Clegg [Wed, 20 Sep 2017 13:57:31 +0000 (09:57 -0400)]
Merge branch '12107-faster-git'

closes #12107

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Add protective comment.
Tom Clegg [Wed, 20 Sep 2017 13:54:47 +0000 (09:54 -0400)]
12107: Add protective comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12084-dispatch-stuck'
Tom Clegg [Wed, 20 Sep 2017 13:45:06 +0000 (09:45 -0400)]
Merge branch '12084-dispatch-stuck'

closes #12084

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12084: Fix dispatcher getting bogged down on "too many open files".
Tom Clegg [Wed, 20 Sep 2017 02:14:37 +0000 (22:14 -0400)]
12084: Fix dispatcher getting bogged down on "too many open files".

Previously, handling the "too many open files" error from popen3
consisted of sleeping 1 second and trying the next job in the
queue. When lots of jobs were queued, this meant getting stuck in
start_jobs() for a long time, futilely trying to start different jobs,
which all failed for the same reason.

Worse, being stuck here meant none of the working jobs could finish,
which meant no file descriptors could be freed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12084: Remove dead code.
Tom Clegg [Wed, 20 Sep 2017 01:54:05 +0000 (21:54 -0400)]
12084: Remove dead code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12084: Reap children whose threads exit abnormally (status == nil).
Tom Clegg [Wed, 20 Sep 2017 01:51:38 +0000 (21:51 -0400)]
12084: Reap children whose threads exit abnormally (status == nil).

Ruby's thread status is false if the thread exited normally, nil if it
exited abnormally.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Fix test suite dependence on outer git config.
Tom Clegg [Tue, 19 Sep 2017 14:02:47 +0000 (10:02 -0400)]
12107: Fix test suite dependence on outer git config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12247: Simplify (Hash)Sum() usage.
Tom Clegg [Tue, 19 Sep 2017 14:47:06 +0000 (10:47 -0400)]
12247: Simplify (Hash)Sum() usage.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12247: Propagate write errors, don't hide them with "bad checksum".
Tom Clegg [Tue, 19 Sep 2017 14:37:43 +0000 (10:37 -0400)]
12247: Propagate write errors, don't hide them with "bad checksum".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Fix broken commit-exists-in-internal-repo check.
Tom Clegg [Tue, 19 Sep 2017 13:56:29 +0000 (09:56 -0400)]
12107: Fix broken commit-exists-in-internal-repo check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Fix broken "git branch --contains" check.
Tom Clegg [Tue, 19 Sep 2017 13:52:02 +0000 (09:52 -0400)]
12107: Fix broken "git branch --contains" check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Test fetch-and-tag on unreferenced and non-tip commits.
Tom Clegg [Mon, 18 Sep 2017 20:46:13 +0000 (16:46 -0400)]
12107: Test fetch-and-tag on unreferenced and non-tip commits.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12107: Use a fresh internal.git repo for each test run.
Tom Clegg [Mon, 18 Sep 2017 20:41:31 +0000 (16:41 -0400)]
12107: Use a fresh internal.git repo for each test run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoBump PhantomJS version no issue #
Peter Amstutz [Mon, 18 Sep 2017 19:13:36 +0000 (15:13 -0400)]
Bump PhantomJS version no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoArvbox explicitly initializes postgres with en_US.UTF-8 locale. no issue #
Peter Amstutz [Fri, 15 Sep 2017 15:42:01 +0000 (11:42 -0400)]
Arvbox explicitly initializes postgres with en_US.UTF-8 locale.  no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix quoting in test_with_arvbox no issue #
Peter Amstutz [Mon, 18 Sep 2017 19:09:08 +0000 (15:09 -0400)]
Fix quoting in test_with_arvbox no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'master' into 12107-faster-git
Tom Clegg [Mon, 18 Sep 2017 18:33:36 +0000 (14:33 -0400)]
Merge branch 'master' into 12107-faster-git

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12197-show-pdh-in-trash'
Tom Clegg [Mon, 18 Sep 2017 18:25:25 +0000 (14:25 -0400)]
Merge branch '12197-show-pdh-in-trash'

closes #12197

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch 'master' into 12197-show-pdh-in-trash
Tom Clegg [Mon, 18 Sep 2017 18:24:48 +0000 (14:24 -0400)]
Merge branch 'master' into 12197-show-pdh-in-trash

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12197: Update PDH column label for consistency.
Tom Clegg [Mon, 18 Sep 2017 18:10:40 +0000 (14:10 -0400)]
12197: Update PDH column label for consistency.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoRemove -f from 'docker tag' due to Docker update refs #12234
Peter Amstutz [Mon, 18 Sep 2017 14:13:58 +0000 (10:13 -0400)]
Remove -f from 'docker tag' due to Docker update refs #12234

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12197: Test PDH search and display on trash page.
Tom Clegg [Mon, 18 Sep 2017 03:03:29 +0000 (23:03 -0400)]
12197: Test PDH search and display on trash page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12197: Remove trash/delete labels on content rows.
Tom Clegg [Thu, 14 Sep 2017 15:31:05 +0000 (11:31 -0400)]
12197: Remove trash/delete labels on content rows.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoArvbox support for caching and offline install of npm packages. no issue #
Peter Amstutz [Fri, 15 Sep 2017 00:22:49 +0000 (20:22 -0400)]
Arvbox support for caching and offline install of npm packages.  no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Add permission_refresh_lock
Peter Amstutz [Thu, 14 Sep 2017 19:06:17 +0000 (15:06 -0400)]
12032: Add permission_refresh_lock

This ensures that when refreshing the permission view in a transaction, any
other transactions affecting permissions will either be committed (so that
results are visible to the refresh) or blocked (waiting to perform its own
refresh).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Don't need to refresh permission view in database seeds, should be done already.
Peter Amstutz [Wed, 13 Sep 2017 20:14:10 +0000 (16:14 -0400)]
12032: Don't need to refresh permission view in database seeds, should be done already.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Remove user_uuid special case from all_perms.
Peter Amstutz [Wed, 13 Sep 2017 19:59:57 +0000 (15:59 -0400)]
12032: Remove user_uuid special case from all_perms.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Remove unnecessary checks on user_uuids.
Peter Amstutz [Wed, 13 Sep 2017 19:50:35 +0000 (15:50 -0400)]
12032: Remove unnecessary checks on user_uuids.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Remove hopefully-unnecessary view refreshes between tests.
Peter Amstutz [Wed, 13 Sep 2017 19:43:56 +0000 (15:43 -0400)]
12032: Remove hopefully-unnecessary view refreshes between tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Refresh materialized_permission_view immediately after migration.
Peter Amstutz [Wed, 13 Sep 2017 17:17:46 +0000 (13:17 -0400)]
12032: Refresh materialized_permission_view immediately after migration.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Remove startnode from materialized_permission_view (again), tests pass
Peter Amstutz [Wed, 13 Sep 2017 15:51:28 +0000 (11:51 -0400)]
12032: Remove startnode from materialized_permission_view (again), tests pass
without it.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Update structure.sql from master schema + migration only.
Peter Amstutz [Wed, 13 Sep 2017 15:32:24 +0000 (11:32 -0400)]
12032: Update structure.sql from master schema + migration only.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Container readable check uses subquery instead join to avoid returning
Peter Amstutz [Mon, 11 Sep 2017 20:57:06 +0000 (16:57 -0400)]
12032: Container readable check uses subquery instead join to avoid returning
duplicate rows.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Database reset used by integration tests refreshes permission view.
Peter Amstutz [Mon, 11 Sep 2017 18:18:32 +0000 (14:18 -0400)]
12032: Database reset used by integration tests refreshes permission view.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Fix down migration.
Peter Amstutz [Mon, 11 Sep 2017 16:12:42 +0000 (12:12 -0400)]
12032: Fix down migration.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Restore create_permission_view.sql with note that it is only
Peter Amstutz [Fri, 8 Sep 2017 18:29:47 +0000 (14:29 -0400)]
12032: Restore create_permission_view.sql with note that it is only
present for migration reasons.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Update install guide to specify minimum Postgres version of 9.3
Peter Amstutz [Fri, 8 Sep 2017 17:59:16 +0000 (13:59 -0400)]
12032: Update install guide to specify minimum Postgres version of 9.3

Include instructions for using SCL to get a newer Postgres on CentOS7.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Add missing files.
Peter Amstutz [Fri, 8 Sep 2017 16:36:53 +0000 (12:36 -0400)]
12032: Add missing files.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Rename permission_view and remove Rails permission cache.
Peter Amstutz [Fri, 8 Sep 2017 16:33:28 +0000 (12:33 -0400)]
12032: Rename permission_view and remove Rails permission cache.

Always refresh permission view when permission graph changes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Controller & integration test teardown invalidates permission view.
Peter Amstutz [Thu, 7 Sep 2017 21:44:10 +0000 (17:44 -0400)]
12032: Controller & integration test teardown invalidates permission view.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Add comments to migration. Also special case api_client_authorizations
Peter Amstutz [Thu, 7 Sep 2017 17:39:12 +0000 (13:39 -0400)]
12032: Add comments to migration.  Also special case api_client_authorizations
in readable_by.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Update structure.sql
Peter Amstutz [Thu, 7 Sep 2017 14:59:42 +0000 (10:59 -0400)]
12032: Update structure.sql

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Added migration for materialized view
Peter Amstutz [Thu, 7 Sep 2017 03:41:34 +0000 (23:41 -0400)]
12032: Added migration for materialized view

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Ensure that permission_view is invalidated & refreshed
Peter Amstutz [Thu, 7 Sep 2017 03:12:41 +0000 (23:12 -0400)]
12032: Ensure that permission_view is invalidated & refreshed

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Fix owner_check to include user_uuids and target_owner_uuid
Peter Amstutz [Wed, 6 Sep 2017 20:01:36 +0000 (16:01 -0400)]
12032: Fix owner_check to include user_uuids and target_owner_uuid

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Make queries to groups table a special case, streamlines permission logic.
Peter Amstutz [Wed, 6 Sep 2017 18:54:50 +0000 (14:54 -0400)]
12032: Make queries to groups table a special case, streamlines permission logic.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Add copyright notice
Peter Amstutz [Thu, 31 Aug 2017 18:58:43 +0000 (14:58 -0400)]
12032: Add copyright notice

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Bug fix so include_trash still respects permissions.
Peter Amstutz [Thu, 31 Aug 2017 18:50:29 +0000 (14:50 -0400)]
12032: Bug fix so include_trash still respects permissions.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Test for ensure_unique_name when untrashing groups.
Peter Amstutz [Thu, 31 Aug 2017 16:10:11 +0000 (12:10 -0400)]
12032: Test for ensure_unique_name when untrashing groups.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Refactor readable_by to minimize subqueries.
Peter Amstutz [Thu, 31 Aug 2017 14:37:13 +0000 (10:37 -0400)]
12032: Refactor readable_by to minimize subqueries.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Log.readable_by uses permission_view
Peter Amstutz [Wed, 30 Aug 2017 17:13:29 +0000 (13:13 -0400)]
12032: Log.readable_by uses permission_view

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Controller support for group trash.
Peter Amstutz [Tue, 29 Aug 2017 13:08:19 +0000 (09:08 -0400)]
12032: Controller support for group trash.

Conttroller support and testing for contents, index, show, trash, untrash,
containers.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Adding controller tests
Peter Amstutz [Mon, 28 Aug 2017 13:57:05 +0000 (09:57 -0400)]
12032: Adding controller tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Add test fixtures
Peter Amstutz [Fri, 25 Aug 2017 18:23:30 +0000 (14:23 -0400)]
12032: Add test fixtures

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Fix hiding trashed groups directly owned by user.
Peter Amstutz [Fri, 25 Aug 2017 01:58:38 +0000 (21:58 -0400)]
12032: Fix hiding trashed groups directly owned by user.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Tests for group delete behavior.
Peter Amstutz [Thu, 24 Aug 2017 21:09:49 +0000 (17:09 -0400)]
12032: Tests for group delete behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Additional refactoring of readable_by. Refactor "trashable" into module.
Peter Amstutz [Thu, 24 Aug 2017 20:22:12 +0000 (16:22 -0400)]
12032: Additional refactoring of readable_by.  Refactor "trashable" into module.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Use permission_view in subquery to filter objects readable by user.
Peter Amstutz [Thu, 24 Aug 2017 16:03:09 +0000 (12:03 -0400)]
12032: Use permission_view in subquery to filter objects readable by user.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12032: Benchmark for group permissions.
Peter Amstutz [Wed, 6 Sep 2017 15:35:55 +0000 (11:35 -0400)]
12032: Benchmark for group permissions.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12248-crunch2-reversemap' closes #12248
Peter Amstutz [Thu, 14 Sep 2017 16:18:40 +0000 (12:18 -0400)]
Merge branch '12248-crunch2-reversemap' closes #12248

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '12236-libcloud-blob-storage-regression'
Lucas Di Pentima [Thu, 14 Sep 2017 15:20:15 +0000 (12:20 -0300)]
Merge branch '12236-libcloud-blob-storage-regression'
Refs #12236

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12236: Bumped libcloud version dependency to 2.2.1.dev2
Lucas Di Pentima [Thu, 14 Sep 2017 15:03:28 +0000 (12:03 -0300)]
12236: Bumped libcloud version dependency to 2.2.1.dev2

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12194-search-always-valid'
Tom Clegg [Wed, 13 Sep 2017 21:07:41 +0000 (17:07 -0400)]
Merge branch '12194-search-always-valid'

closes #12194

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>