12377: Update workbench test breaking since 484246ac0.

refs #12377

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12715-slow-propfind'

fixes #12715

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12822-keepproxy-token-scope'

refs #12822

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12377-workbench-composer-link' refs #12377

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12821-sbatch-script-race'

refs #12821

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12821: Fix race in concurrent calls to submit().

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12721-build-test-version'

refs #12721

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Add run-build-test-packages-one-target.sh --build-version arg

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12715: Rename flag !reading => alwaysReadEOF.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Add composer_url to arvbox workbench config refs #12377

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12715: Fix slow PROPFIND by avoiding content probing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12822: Allow reading from keepproxy with collection sharing tokens.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12377: Add link to composer on workbench

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '8311-mount-git'

closes #8311

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Fix gitUrl discovery doc for config git_repo_https_base: true.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Ensure git tree is readable even with umask 077.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Remove support for writable git_tree mount.

Feature is of dubious value, and expected behavior isn't clear --
e.g., should all files be world-writable?

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Ensure git tree tmpdir is readable by container.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Pass in container token to extractTree

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

8311: Ensure gitUrl is always a string in the discovery doc.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Fix API tests broken in 4343e1f.

* Restore arvados repo dropped from tarball.

* Fix git repo fixtures being removed in API test teardown.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Tidy up integration test services.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Update docs to describe current implementation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Reject invalid/unsupported git mounts. Default to path="/".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Try some git_tree mounts in TestSetupMounts.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Add git_tree mount type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Add auth handler tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Access git repos by UUID.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Fix "read" or "write" mistakenly appended to response data.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

8311: Remove debug printf.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Fix Access-Control-Allow-Headers CORS header tests refs #12060

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Add Content-Type to Access-Control-Allow-Headers CORS headers refs #12060

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '4019-query-properties' closes #4019

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4019: More tests for error cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4019: Improve error checking, add tests for some error cases

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12614-node-states' refs #12614

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Tag arvados/jobs image with $cwl_runner_version instead of git hash refs #12811

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12614: Can shutdown failed nodes.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12614: Support daemon actor aware of 'fail' node state

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12614: Make node manager aware of "fail" node state.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12803-unescaped-colon-filename'

fixes #12803

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12765-workbench-404-trashed' refs #12765

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12803: Accept unescaped colons in filenames.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4019: Add index for efficient jsonb query on collection properties

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4019: Add URI quoting option for keys.  Update docs.  Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4019: Update docs.  Tweak syntax of 'exists' and add alternate form.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12550-crunch1-exit-race'

refs #12550

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12511-is-trashed-query' refs #12511

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

4019: Expand testing.  Make behavior consistent for != and "not in"

Rename "?" to "exists" and add "not exists".

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12167-python-request-id'

refs #12167

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Allow --silent to be combined with -*-progress flags.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Test arv-get and arv-put X-Request-Id logging.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12511: readable_by filters on is_trashed directly

Fix performance problem in the negative join NOT EXISTS(...) used to filter out
trashed items for admin users.  The clause
(uuid = target_uuid OR owner_uuid = target_uuid) doesn't use the index
efficiently and results in a very expensive seq scan.

However, the "OR" in the negative clause is not necessary.  We only need
materialized_permission_view to know if "owner_uuid" is trashed (directly or
indirectly), not "uuid".  If the record type has an is_trashed flag (currently
only collections and groups) then it is more efficient to filter on that
directly.

This commit refactors the readable_by query to only check owner_uuid, and
additionally filter on "is_trashed" for collections and groups.

As an additional cleanup, because "readable_by" now explicitly knows to filter
on "is_trashed" this makes the default_scope in Collections redundant.  This
commit also removes default_scope & various uses of unscoped reduces complexity.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12550: Fix race: read buffered data from stderr after child exits.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Add missing _version.py, now part of source tree.

refs #12721

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

4019: Initial support for queries on jsonb subproperties.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '12765-permission-graph-trash' refs #12765

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12765: Add explicit flag whether "trashed" propagates across an edge

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12765: Add test case that "trashed" should not propagate over permission links.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '11908-properties-column-json'

refs #11908

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12721-build-package-version'

refs #12721

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Test X-Request-Id request headers in Keep get/put/head reqs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Use one X-Request-Id per arv-put; display it if not --silent.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Use one X-Request-Id per arv-get process; display it if -v.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Send caller-provided or random X-Request-Id in Keep requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12167: Send caller-provided or random X-Request-Id in API requests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Write composer.yml instead of arvados-configuration.yml refs #12353

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12765: Fix untrash button on 404 page.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

12765: Add test for untrash on 404 page

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

Merge branch '11453-federated-tokens'

refs #11453

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Restore update of email/prefs/name on subsequent remote auth.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Test username is not updated once assigned.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Assign username when creating mirror of remote user.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: "sso_insecure" config applies to remote Arvados auth, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Merge branch 'master' into 11453-federated-tokens

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11908: Add comment about dropping/recreating FT index.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>

11908: Fix crash if index being deleted is already deleted.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>

11908: Migrate collections.properties to jsonb.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>

11807: Migrate data from YAML to JSON.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>

11453: Add IP: alt names to self-signed cert so python accepts it.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Install bundles in builder images.

Avoids downloading/building the entire bundle every time a Rails
package is built.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Omit cache dirs for platform A in package for platform B.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Fix dev-version build.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Don't rebuild libcloud if package already exists.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Fix ignored iteration numbers in build.list.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Merge branch 'master' into 12721-build-package-version

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Tests for invalid / future-proof / reanimated tokens.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Test group membership with regular groups.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

11453: Fix & test remote token caching.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Handle --build-version 1.2.3-4-5 (v="1.2.3-4", iter="5")

...using the debian version string rules (the last hyphen is the
version-iteration separator).

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Fix python package iteration and rails fpm invocation.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Update documentation to the new name of the sync-groups tool.

refs #12018

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

Small build script fixes.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

Merge branch '12018-rename-sync-groups'

refs #12018

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

Do not hardcode the name of the executable in the sync-groups code.

refs #12018

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

11453: Mollify python by giving 127.0.0.1 as the first cert CN.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Rename the group sync tool to follow our standard naming scheme.

refs #12018

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

12721: --version => --build-version. Make iteration mandatory.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

12721: Support package iterations via --version 1.2.3-5

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

Merge branch '12752-discover-keep-web-service' refs #12752

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>