arvados.git
6 years ago14299: Use include_old_versions in keep-balance to avoid losing blocks.
Lucas Di Pentima [Tue, 23 Oct 2018 22:19:07 +0000 (19:19 -0300)]
14299: Use include_old_versions in keep-balance to avoid losing blocks.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13561-collection-versions-doc'
Lucas Di Pentima [Tue, 23 Oct 2018 13:56:18 +0000 (10:56 -0300)]
Merge branch '13561-collection-versions-doc'
Closes #13561

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Rephrased some sentences to avoid confusion.
Lucas Di Pentima [Tue, 23 Oct 2018 13:55:12 +0000 (10:55 -0300)]
13561: Rephrased some sentences to avoid confusion.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13561-trashed-collection-versions-fix'
Lucas Di Pentima [Mon, 22 Oct 2018 16:29:27 +0000 (13:29 -0300)]
Merge branch '13561-trashed-collection-versions-fix'
Refs #13561

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Formatting fixes, examples enhancements & additions.
Lucas Di Pentima [Mon, 22 Oct 2018 15:21:55 +0000 (12:21 -0300)]
13561: Formatting fixes, examples enhancements & additions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14260-expired-token-delete' refs #14260
Peter Amstutz [Fri, 19 Oct 2018 14:11:33 +0000 (10:11 -0400)]
Merge branch '14260-expired-token-delete' refs #14260

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Use straight SQL DELETE instead of destroy_all
Peter Amstutz [Fri, 19 Oct 2018 14:06:50 +0000 (10:06 -0400)]
14260: Use straight SQL DELETE instead of destroy_all

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Code cleanup.
Lucas Di Pentima [Thu, 18 Oct 2018 23:20:19 +0000 (20:20 -0300)]
13561: Code cleanup.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Exclude past versions regardless of user & include_trash setting.
Lucas Di Pentima [Thu, 18 Oct 2018 22:46:18 +0000 (19:46 -0300)]
13561: Exclude past versions regardless of user & include_trash setting.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14260-runtime-token' refs #14260
Peter Amstutz [Thu, 18 Oct 2018 17:44:35 +0000 (13:44 -0400)]
Merge branch '14260-runtime-token' refs #14260

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14366-crunch2-upgrade' refs #14366
Peter Amstutz [Thu, 18 Oct 2018 17:42:13 +0000 (13:42 -0400)]
Merge branch '14366-crunch2-upgrade' refs #14366

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Remove redundant schema_migrations in structure.sql
Peter Amstutz [Thu, 18 Oct 2018 17:18:19 +0000 (13:18 -0400)]
14260: Remove redundant schema_migrations in structure.sql

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Sort runtime_auth_scopes
Peter Amstutz [Thu, 18 Oct 2018 16:20:50 +0000 (12:20 -0400)]
14260: Sort runtime_auth_scopes

Container.for_current_token returns nil or a Container (not an active
record query).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'master' into 14260-runtime-token
Peter Amstutz [Thu, 18 Oct 2018 15:58:57 +0000 (11:58 -0400)]
Merge branch 'master' into 14260-runtime-token

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Re-phrase user guide & add admin guide section.
Lucas Di Pentima [Wed, 10 Oct 2018 20:12:27 +0000 (17:12 -0300)]
13561: Re-phrase user guide & add admin guide section.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Update documentation on collection versioning.
Lucas Di Pentima [Fri, 5 Oct 2018 18:55:41 +0000 (15:55 -0300)]
13561: Update documentation on collection versioning.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13561-collection-versions-fixes'
Lucas Di Pentima [Wed, 17 Oct 2018 20:12:48 +0000 (17:12 -0300)]
Merge branch '13561-collection-versions-fixes'
Refs #13561

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14366: Add "upgrade to containers api" to "upgrading" section
Peter Amstutz [Wed, 17 Oct 2018 19:54:00 +0000 (15:54 -0400)]
14366: Add "upgrade to containers api" to "upgrading" section

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14366: Improve wording of "upgrading to containers API" section
Peter Amstutz [Wed, 17 Oct 2018 19:43:15 +0000 (15:43 -0400)]
14366: Improve wording of "upgrading to containers API" section

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Raise exception instead of ignoring versioning attributes updates.
Lucas Di Pentima [Wed, 17 Oct 2018 18:51:36 +0000 (15:51 -0300)]
13561: Raise exception instead of ignoring versioning attributes updates.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Fixes api to get a collection with its past versions.
Lucas Di Pentima [Wed, 17 Oct 2018 18:05:50 +0000 (15:05 -0300)]
13561: Fixes api to get a collection with its past versions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoFor the package test docker images, sanitize how we get the rvm GPG key,
Ward Vandewege [Wed, 17 Oct 2018 01:38:54 +0000 (21:38 -0400)]
For the package test docker images, sanitize how we get the rvm GPG key,
and remove a dependency on the GPG keyservers which are flakey.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd a systemd service file to arvados-node-manager.
Ward Vandewege [Tue, 16 Oct 2018 20:49:15 +0000 (16:49 -0400)]
Add a systemd service file to arvados-node-manager.

Reset package iteration number for arvados-node-manager to version 1.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoRemove spurious output from run-build-packages.sh
Ward Vandewege [Tue, 16 Oct 2018 20:38:01 +0000 (16:38 -0400)]
Remove spurious output from run-build-packages.sh

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14344: bump package version for arvados-docker-cleaner so that we get a new
Ward Vandewege [Tue, 16 Oct 2018 20:37:00 +0000 (16:37 -0400)]
14344: bump package version for arvados-docker-cleaner so that we get a new
package built with the systemd service file.

refs #14344

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14344: Merge branch '14344-fix-rpm-vs-deb-detection'
Ward Vandewege [Tue, 16 Oct 2018 19:48:44 +0000 (15:48 -0400)]
14344: Merge branch '14344-fix-rpm-vs-deb-detection'

closes #14344

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14344: Move "set -e" back to top.
Tom Clegg [Tue, 16 Oct 2018 13:35:17 +0000 (09:35 -0400)]
14344: Move "set -e" back to top.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix the detection of rpm vs deb systems.
Ward Vandewege [Tue, 16 Oct 2018 01:49:30 +0000 (21:49 -0400)]
Fix the detection of rpm vs deb systems.

refs #14344

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '14285-keep-balance-metrics'
Tom Clegg [Tue, 16 Oct 2018 17:24:04 +0000 (13:24 -0400)]
Merge branch '14285-keep-balance-metrics'

closes #14285

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Update admin/metrics docs.
Tom Clegg [Mon, 15 Oct 2018 21:36:03 +0000 (17:36 -0400)]
14285: Update admin/metrics docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Add deduplication ratio to keep-balance metrics.
Tom Clegg [Mon, 15 Oct 2018 21:02:22 +0000 (17:02 -0400)]
14285: Add deduplication ratio to keep-balance metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoFix missing namespace in metric names.
Tom Clegg [Tue, 16 Oct 2018 15:41:54 +0000 (11:41 -0400)]
Fix missing namespace in metric names.

refs #14285

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoRevert change to setup.py for arvados-docker-cleaner, pip doesn't agree with
Ward Vandewege [Mon, 15 Oct 2018 23:06:17 +0000 (19:06 -0400)]
Revert change to setup.py for arvados-docker-cleaner, pip doesn't agree with
it...

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14260: secret_mounts works with runtime_token (really this time)
Peter Amstutz [Mon, 15 Oct 2018 21:11:11 +0000 (17:11 -0400)]
14260: secret_mounts works with runtime_token (really this time)

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14338-workbench-flag-to-disable-repositories'
Ward Vandewege [Mon, 15 Oct 2018 21:06:04 +0000 (17:06 -0400)]
Merge branch '14338-workbench-flag-to-disable-repositories'

closes #14338

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd a flag to hide repositories entries in the Workbench menus. This is useful
Ward Vandewege [Fri, 12 Oct 2018 15:15:44 +0000 (11:15 -0400)]
Add a flag to hide repositories entries in the Workbench menus. This is useful
in a scenario where the jobs api is disabled and there are no local git
repositories. Local git repositories are optional when using the containers api.

refs #14338

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14260: secret_mounts works with runtime_token
Peter Amstutz [Mon, 15 Oct 2018 20:53:40 +0000 (16:53 -0400)]
14260: secret_mounts works with runtime_token

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: add runtime_user_uuid and runtime_auth_scopes to API response
Peter Amstutz [Mon, 15 Oct 2018 20:47:38 +0000 (16:47 -0400)]
14260: add runtime_user_uuid and runtime_auth_scopes to API response

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMake sure that the arvados-docker-cleaner systemd service file is installed in
Ward Vandewege [Mon, 15 Oct 2018 20:36:07 +0000 (16:36 -0400)]
Make sure that the arvados-docker-cleaner systemd service file is installed in
the correct directory.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14260: Bugfix for runtime_token/secret_mounts and retries
Peter Amstutz [Mon, 15 Oct 2018 20:20:39 +0000 (16:20 -0400)]
14260: Bugfix for runtime_token/secret_mounts and retries

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14285-keep-balance-metrics'
Tom Clegg [Mon, 15 Oct 2018 20:09:36 +0000 (16:09 -0400)]
Merge branch '14285-keep-balance-metrics'

refs #14285

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Merge branch 'master' into 14285-keep-balance-metrics
Tom Clegg [Mon, 15 Oct 2018 20:08:22 +0000 (16:08 -0400)]
14285: Merge branch 'master' into 14285-keep-balance-metrics

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoWebshell requires the short hostname for the shell server object.
Ward Vandewege [Mon, 15 Oct 2018 19:39:14 +0000 (15:39 -0400)]
Webshell requires the short hostname for the shell server object.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoThe `arv group create` command doesn't set group_class by default, and we need
Ward Vandewege [Fri, 12 Oct 2018 20:01:44 +0000 (16:01 -0400)]
The `arv group create` command doesn't set group_class by default, and we need
that field set to 'project' for the 'Arvados Standard Docker Images' project.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoSmall documentation fixes.
Ward Vandewege [Fri, 12 Oct 2018 19:53:37 +0000 (15:53 -0400)]
Small documentation fixes.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '14086-keepweb-serve-past-versions'
Lucas Di Pentima [Mon, 15 Oct 2018 18:10:20 +0000 (15:10 -0300)]
Merge branch '14086-keepweb-serve-past-versions'
Closes #14086

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14285: Add tests for LoadToken and RequireLiteralToken.
Tom Clegg [Mon, 15 Oct 2018 17:50:10 +0000 (13:50 -0400)]
14285: Add tests for LoadToken and RequireLiteralToken.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14260: Fix workbench functionals test
Peter Amstutz [Mon, 15 Oct 2018 17:36:02 +0000 (13:36 -0400)]
14260: Fix workbench functionals test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14086: Add tests to prove that keep-web can serve past collection versions.
Lucas Di Pentima [Mon, 15 Oct 2018 17:02:29 +0000 (14:02 -0300)]
14086: Add tests to prove that keep-web can serve past collection versions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14260: Fix tests.
Peter Amstutz [Mon, 15 Oct 2018 16:09:35 +0000 (12:09 -0400)]
14260: Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14285: Merge branch 'master' into 14285-keep-balance-metrics
Tom Clegg [Mon, 15 Oct 2018 15:23:18 +0000 (11:23 -0400)]
14285: Merge branch 'master' into 14285-keep-balance-metrics

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14260: Add invalid remote token test
Peter Amstutz [Mon, 15 Oct 2018 15:09:58 +0000 (11:09 -0400)]
14260: Add invalid remote token test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Additional validation when container uuid is provided in token
Peter Amstutz [Mon, 15 Oct 2018 15:02:38 +0000 (11:02 -0400)]
14260: Additional validation when container uuid is provided in token

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Update API doc for runtime_* fields (token/user_uuid/auth_scopes)
Peter Amstutz [Fri, 12 Oct 2018 21:34:47 +0000 (17:34 -0400)]
14260: Update API doc for runtime_* fields (token/user_uuid/auth_scopes)

Also add missing documentation for secret_mounts.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: crunch-run constructs a v2 token with added container uuid
Peter Amstutz [Fri, 12 Oct 2018 21:07:33 +0000 (17:07 -0400)]
14260: crunch-run constructs a v2 token with added container uuid

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Container auth tokens must have the container uuid in the 4th position
Peter Amstutz [Fri, 12 Oct 2018 21:01:23 +0000 (17:01 -0400)]
14260: Container auth tokens must have the container uuid in the 4th position

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14285: Add keep-balance to health aggregator test stub.
Tom Clegg [Fri, 12 Oct 2018 18:56:51 +0000 (14:56 -0400)]
14285: Add keep-balance to health aggregator test stub.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14260: Don't set auth_uuid when runtime_token is set.
Peter Amstutz [Fri, 12 Oct 2018 18:46:02 +0000 (14:46 -0400)]
14260: Don't set auth_uuid when runtime_token is set.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14285: Add keep-balance to example cluster config.
Tom Clegg [Fri, 12 Oct 2018 17:56:22 +0000 (13:56 -0400)]
14285: Add keep-balance to example cluster config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Add token middleware. Require management token for metrics.
Tom Clegg [Fri, 12 Oct 2018 17:52:08 +0000 (13:52 -0400)]
14285: Add token middleware. Require management token for metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Update sample config.
Tom Clegg [Fri, 12 Oct 2018 14:04:04 +0000 (10:04 -0400)]
14285: Update sample config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Split Server to server.go.
Tom Clegg [Fri, 12 Oct 2018 14:01:50 +0000 (10:01 -0400)]
14285: Split Server to server.go.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14260: Added runtime_token to container record
Peter Amstutz [Thu, 11 Oct 2018 22:27:51 +0000 (18:27 -0400)]
14260: Added runtime_token to container record

* runtime_token, runtime_user_uuid and runtime_auth_scopes are now
  part of container initialization and reuse decisions

* Determine runtime_user_uuid and runtime_auth_scopes as part of
  Container.resolve

* Use runtime_user_uuid to create container token (when runtime_token
  is not set)

* act_as runtime_user_uuid when resolving container request fields

* tokens used for runtime_token will be left untouched (remove expire_destroy)

* added/updated/fixed tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13561-collection-versions-api'
Lucas Di Pentima [Thu, 11 Oct 2018 18:36:03 +0000 (15:36 -0300)]
Merge branch '13561-collection-versions-api'
Refs #13561

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Move versioning code inside locking block.
Lucas Di Pentima [Thu, 11 Oct 2018 16:17:47 +0000 (13:17 -0300)]
13561: Move versioning code inside locking block.

Also, don't allow current_version_uuid updates on current versions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Fix validation & update related test.
Lucas Di Pentima [Thu, 11 Oct 2018 14:09:37 +0000 (11:09 -0300)]
13561: Fix validation & update related test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14260: runtime_token goes in container auth
Peter Amstutz [Thu, 11 Oct 2018 14:08:08 +0000 (10:08 -0400)]
14260: runtime_token goes in container auth

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Test container runtime_token
Peter Amstutz [Wed, 10 Oct 2018 19:58:21 +0000 (15:58 -0400)]
14260: Test container runtime_token

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Container runtime token wip
Peter Amstutz [Fri, 5 Oct 2018 20:40:33 +0000 (16:40 -0400)]
14260: Container runtime token wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14260: Migration adding runtime_token, runtime_user_uuid, runtime_auth_scopes
Peter Amstutz [Fri, 5 Oct 2018 19:31:08 +0000 (15:31 -0400)]
14260: Migration adding runtime_token, runtime_user_uuid, runtime_auth_scopes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Refactor version creation & sync logic.
Lucas Di Pentima [Wed, 10 Oct 2018 01:16:30 +0000 (22:16 -0300)]
13561: Refactor version creation & sync logic.

* Avoid overriding save! and move code to collection's callbacks.
* Use around_update to make the necessary actions and get a lock before saving.
* Arrange version creation and syncing actions so that they happen after the
  current version's update.
* Update test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '14199-copy-from-remote'
Tom Clegg [Tue, 9 Oct 2018 20:07:44 +0000 (16:07 -0400)]
Merge branch '14199-copy-from-remote'

refs #14199

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Reduce logging noise from azure test stubs.
Tom Clegg [Tue, 9 Oct 2018 20:07:17 +0000 (16:07 -0400)]
14199: Reduce logging noise from azure test stubs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Merge branch 'master' into 14199-copy-from-remote
Tom Clegg [Tue, 9 Oct 2018 19:56:26 +0000 (15:56 -0400)]
14199: Merge branch 'master' into 14199-copy-from-remote

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Update metrics doc page.
Tom Clegg [Tue, 9 Oct 2018 19:51:38 +0000 (15:51 -0400)]
14285: Update metrics doc page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '14196-api-v2-tokens' refs #14196
Peter Amstutz [Tue, 9 Oct 2018 19:20:38 +0000 (15:20 -0400)]
Merge branch '14196-api-v2-tokens' refs #14196

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Revert debug change to Access-Control-Max-Age
Peter Amstutz [Tue, 9 Oct 2018 19:19:31 +0000 (15:19 -0400)]
14196: Revert debug change to Access-Control-Max-Age

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '14274-factory-bot' closes #14274
Peter Amstutz [Tue, 9 Oct 2018 19:16:09 +0000 (15:16 -0400)]
Merge branch '14274-factory-bot' closes #14274

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14199: Abandon processing sooner if client disconnects.
Tom Clegg [Tue, 9 Oct 2018 15:36:52 +0000 (11:36 -0400)]
14199: Abandon processing sooner if client disconnects.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Rename (*remoteResponseCacher)Flush -> Close.
Tom Clegg [Tue, 9 Oct 2018 15:24:44 +0000 (11:24 -0400)]
14199: Rename (*remoteResponseCacher)Flush -> Close.

Finalizing the caching writer is more like Close: it can only happen
once, and only after all Writes are done.  Flush means something
different, especially in http.ResponseWriter context.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Require "X-Keep-Signature: local[, ...]" to invoke signing.
Tom Clegg [Tue, 9 Oct 2018 15:19:16 +0000 (11:19 -0400)]
14199: Require "X-Keep-Signature: local[, ...]" to invoke signing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14285: Export stats as prometheus metrics.
Tom Clegg [Mon, 8 Oct 2018 20:43:40 +0000 (16:43 -0400)]
14285: Export stats as prometheus metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoarvbox fix "can't write to /dev/stderr" message no issue #
Peter Amstutz [Fri, 5 Oct 2018 18:09:30 +0000 (14:09 -0400)]
arvbox fix "can't write to /dev/stderr" message no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch 'master' into 13561-collection-versions-api
Lucas Di Pentima [Fri, 5 Oct 2018 13:09:16 +0000 (10:09 -0300)]
Merge branch 'master' into 13561-collection-versions-api

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12690-12748-crunchstat-summary'
Peter Amstutz [Fri, 5 Oct 2018 01:36:05 +0000 (21:36 -0400)]
Merge branch '12690-12748-crunchstat-summary'

refs #12690
refs #12748
refs #13759

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Multi-site search works with v2 tokens.
Peter Amstutz [Fri, 5 Oct 2018 01:27:33 +0000 (21:27 -0400)]
14196: Multi-site search works with v2 tokens.

Also update workbench tests use v2 tokens.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Allow request old collection version by UUID.
Lucas Di Pentima [Fri, 5 Oct 2018 00:49:25 +0000 (21:49 -0300)]
13561: Allow request old collection version by UUID.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Ignore version & current_version_uuid parama on API create endpoint.
Lucas Di Pentima [Thu, 4 Oct 2018 21:55:40 +0000 (18:55 -0300)]
13561: Ignore version & current_version_uuid parama on API create endpoint.

Also, updated discovery doc to include 'include_old_versions' param on index.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Avoid permission links to be attached to past collection versions.
Lucas Di Pentima [Thu, 4 Oct 2018 18:54:18 +0000 (15:54 -0300)]
13561: Avoid permission links to be attached to past collection versions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13561: Expand index API to include past versions.
Lucas Di Pentima [Thu, 4 Oct 2018 18:22:31 +0000 (15:22 -0300)]
13561: Expand index API to include past versions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14199: Fix deadlock in test suite.
Tom Clegg [Thu, 4 Oct 2018 18:01:00 +0000 (14:01 -0400)]
14199: Fix deadlock in test suite.

If a KeepClient has been created using an API server address that is
no longer reachable, calling keepclient.RefreshServiceDiscovery() puts
the poll() goroutine into an endless retry loop, and a second call
never returns because ent.clear is never ready to receive.

Work around this in the pull worker tests by only refreshing services
from the API server actually being used, not on additional ones
referenced by previous test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Drop Vary response header for non-proxyable requests.
Tom Clegg [Thu, 4 Oct 2018 14:15:22 +0000 (10:15 -0400)]
14199: Drop Vary response header for non-proxyable requests.

The X-Keep-Signature header is ignored for such requests anyway.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago14199: Store data if X-Keep-Signature given in proxied GET/HEAD req.
Tom Clegg [Thu, 4 Oct 2018 05:27:19 +0000 (01:27 -0400)]
14199: Store data if X-Keep-Signature given in proxied GET/HEAD req.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12690 & 12748: Read stats from arv-mount.txt
Peter Amstutz [Wed, 3 Oct 2018 20:38:30 +0000 (16:38 -0400)]
12690 & 12748: Read stats from arv-mount.txt

* Compute elapsed time for containers correctly
* Update tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13561: Avoid collections.index to include old versions
Lucas Di Pentima [Wed, 3 Oct 2018 20:11:38 +0000 (17:11 -0300)]
13561: Avoid collections.index to include old versions

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago14199: Refactor test case.
Tom Clegg [Wed, 3 Oct 2018 19:54:45 +0000 (15:54 -0400)]
14199: Refactor test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoDocumentation tweaks. Point at Biostars as the forum to discuss Arvados
Ward Vandewege [Wed, 3 Oct 2018 18:01:27 +0000 (14:01 -0400)]
Documentation tweaks. Point at Biostars as the forum to discuss Arvados
questions.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago14274: Fix FactoryGirl deprication warning (renamed to FactoryBot)
Peter Amstutz [Tue, 2 Oct 2018 17:56:16 +0000 (13:56 -0400)]
14274: Fix FactoryGirl deprication warning (renamed to FactoryBot)

Also fix "mocha/mini_test" -> "mocha/minitest" warning.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13619-fed-object-list' closes #13619
Peter Amstutz [Wed, 3 Oct 2018 13:23:30 +0000 (09:23 -0400)]
Merge branch '13619-fed-object-list' closes #13619

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago14196: Don't crash if element of params[:reader_tokens] is not a String
Peter Amstutz [Tue, 2 Oct 2018 21:14:59 +0000 (17:14 -0400)]
14196: Don't crash if element of params[:reader_tokens] is not a String

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>