arvados.git
4 years agoWarn if MaxKeepBlobBuffers > MaxConcurrentRequests.
Tom Clegg [Thu, 9 Apr 2020 20:58:36 +0000 (16:58 -0400)]
Warn if MaxKeepBlobBuffers > MaxConcurrentRequests.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16219: Test populated container fields.
Tom Clegg [Thu, 5 Mar 2020 21:42:39 +0000 (16:42 -0500)]
16219: Test populated container fields.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16219: Load all fields needed to compute node size.
Tom Clegg [Thu, 5 Mar 2020 19:44:22 +0000 (14:44 -0500)]
16219: Load all fields needed to compute node size.

Without ContainerImage and Mounts, the scratch size requirement can't
be computed correctly.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16221: Fix test for config endpoint.
Tom Clegg [Thu, 12 Mar 2020 04:15:16 +0000 (00:15 -0400)]
16221: Fix test for config endpoint.

x['configs'] raises KeyError on an old discovery doc -- test
x.get('configs', False) instead.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoFix instance IDs in arvados-dispatch-cloud log messages.
Tom Clegg [Wed, 11 Mar 2020 20:52:04 +0000 (16:52 -0400)]
Fix instance IDs in arvados-dispatch-cloud log messages.

Some logs had {"Instance": {}} instead of the provider's instance ID.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16270: Fill in missing scratch fields on InstanceType entries.
Tom Clegg [Wed, 1 Apr 2020 20:44:02 +0000 (16:44 -0400)]
16270: Fill in missing scratch fields on InstanceType entries.

Previously they were being filled in correctly when written as an
array in the config file, but not when written as a map.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoBump rake from 12.3.2 to 13.0.1 in /services/api
dependabot[bot] [Tue, 3 Mar 2020 21:27:27 +0000 (21:27 +0000)]
Bump rake from 12.3.2 to 13.0.1 in /services/api

Bumps [rake](https://github.com/ruby/rake) from 12.3.2 to 13.0.1.
- [Release notes](https://github.com/ruby/rake/releases)
- [Changelog](https://github.com/ruby/rake/blob/master/History.rdoc)
- [Commits](https://github.com/ruby/rake/compare/v12.3.2...v13.0.1)

Signed-off-by: dependabot[bot] <support@github.com>
4 years agoBump rake from 12.3.2 to 13.0.1 in /apps/workbench
dependabot[bot] [Tue, 3 Mar 2020 21:27:30 +0000 (21:27 +0000)]
Bump rake from 12.3.2 to 13.0.1 in /apps/workbench

Bumps [rake](https://github.com/ruby/rake) from 12.3.2 to 13.0.1.
- [Release notes](https://github.com/ruby/rake/releases)
- [Changelog](https://github.com/ruby/rake/blob/master/History.rdoc)
- [Commits](https://github.com/ruby/rake/compare/v12.3.2...v13.0.1)

Signed-off-by: dependabot[bot] <support@github.com>
4 years agoBump nokogiri from 1.10.4 to 1.10.8 in /apps/workbench
dependabot[bot] [Tue, 25 Feb 2020 08:04:58 +0000 (08:04 +0000)]
Bump nokogiri from 1.10.4 to 1.10.8 in /apps/workbench

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.4 to 1.10.8.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.10.4...v1.10.8)

Signed-off-by: dependabot[bot] <support@github.com>
4 years agoBump loofah from 2.2.3 to 2.3.1 in /apps/workbench
dependabot[bot] [Fri, 8 Nov 2019 21:48:47 +0000 (21:48 +0000)]
Bump loofah from 2.2.3 to 2.3.1 in /apps/workbench

Bumps [loofah](https://github.com/flavorjones/loofah) from 2.2.3 to 2.3.1.
- [Release notes](https://github.com/flavorjones/loofah/releases)
- [Changelog](https://github.com/flavorjones/loofah/blob/master/CHANGELOG.md)
- [Commits](https://github.com/flavorjones/loofah/compare/v2.2.3...v2.3.1)

Signed-off-by: dependabot[bot] <support@github.com>
4 years agoBump nokogiri from 1.10.2 to 1.10.8 in /services/api
dependabot[bot] [Tue, 25 Feb 2020 08:04:57 +0000 (08:04 +0000)]
Bump nokogiri from 1.10.2 to 1.10.8 in /services/api

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.2 to 1.10.8.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.10.2...v1.10.8)

Signed-off-by: dependabot[bot] <support@github.com>
4 years ago16266: Applies monkeypatch to fix CVE-2020-5267 on workbench1.
Lucas Di Pentima [Mon, 23 Mar 2020 21:07:56 +0000 (18:07 -0300)]
16266: Applies monkeypatch to fix CVE-2020-5267 on workbench1.

As adviced on https://github.com/advisories/GHSA-65cv-r6x7-79hv

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16266: Adds tests exposing potential XSS vulnerability on escape_javascript()
Lucas Di Pentima [Thu, 26 Mar 2020 16:29:40 +0000 (13:29 -0300)]
16266: Adds tests exposing potential XSS vulnerability on escape_javascript()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16138: Remove additional/code references to copying pipelines
Peter Amstutz [Tue, 24 Mar 2020 19:35:38 +0000 (15:35 -0400)]
16138: Remove additional/code references to copying pipelines

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16138: Remove additional pipelines related code
Peter Amstutz [Mon, 23 Mar 2020 15:00:07 +0000 (11:00 -0400)]
16138: Remove additional pipelines related code

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16138: whitelist fields for arv-copy
Peter Amstutz [Fri, 20 Mar 2020 19:47:00 +0000 (15:47 -0400)]
16138: whitelist fields for arv-copy

Also removed code for copying
PipelineTemplate/PipelineInstance (obsolete jobs API).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoRemove Gemfile.lock from login-sync 2.0.1
Peter Amstutz [Tue, 3 Mar 2020 21:17:38 +0000 (16:17 -0500)]
Remove Gemfile.lock from login-sync

no issue #

To be consistent with sdk/ruby sdk/cli behavior.

There's a circular dependency where the version number of the gem is
embedded in Gemfile.lock, but the version number is derived from the
git commit timestamp, so when it updates Gemfile.lock it creates a new
version, which requires a updating Gemfile.lock, and so on.

However, unlike the Ruby on Rails deb/rpm packages, when we create
gems (dev or production) it does not include Gemfile.lock, so there's
no benefit to having it checked in.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoRuby gems respect ARVADOS_BUILDING_VERSION
Peter Amstutz [Tue, 3 Mar 2020 20:34:05 +0000 (15:34 -0500)]
Ruby gems respect ARVADOS_BUILDING_VERSION

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoNodemanager and crunchstat-summary sync with pysdk version
Peter Amstutz [Thu, 27 Feb 2020 18:56:49 +0000 (13:56 -0500)]
Nodemanager and crunchstat-summary sync with pysdk version

refs #16151

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16169: Monkey patch load_tool.resolve_and_validate_document to fix bug
Peter Amstutz [Fri, 28 Feb 2020 17:07:35 +0000 (12:07 -0500)]
16169: Monkey patch load_tool.resolve_and_validate_document to fix bug

There is a bug in upstream cwltool where the version updater needs to
replace the document fragments in the loader index with the updated
ones, but actually it only does it for the root document.  Normally we
just fix the bug in upstream but that's challenging because current
cwltool dropped support for Python 2.7 and we're still supporting py2
in Arvados 2.0 (although py2 support will most likely be dropped in
Arvados 2.1).  Making a bugfix fork comes with its own
complications (it would need to be added to PyPi) so monkey patching
is the least disruptive fix (and is relatively safe because our
cwltool dependency is pinned to a specific version).  This
should be removed as soon as a bugfix goes into upstream cwltool and
we upgrade to it.

4 years ago16169: Adds test exposing the requirement propagation bug.
Lucas Di Pentima [Wed, 26 Feb 2020 23:58:59 +0000 (20:58 -0300)]
16169: Adds test exposing the requirement propagation bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16177: Fix tests, test server config now has TrustAllContent: false
Peter Amstutz [Thu, 27 Feb 2020 17:21:34 +0000 (12:21 -0500)]
16177: Fix tests, test server config now has TrustAllContent: false

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16177: Use correct config item Collections.TrustAllContent
Peter Amstutz [Wed, 26 Feb 2020 20:31:58 +0000 (15:31 -0500)]
16177: Use correct config item Collections.TrustAllContent

Also added fallback error handler, if view or download link doesn't
work, render error text (otherwise you get a very confusing Rails
crash page).

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16202: Add Workbench.SSHHelpHostSuffix
Peter Amstutz [Wed, 26 Feb 2020 21:12:22 +0000 (16:12 -0500)]
16202: Add Workbench.SSHHelpHostSuffix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoFix web shell.
Ward Vandewege [Wed, 26 Feb 2020 18:49:28 +0000 (13:49 -0500)]
Fix web shell.

refs #16203

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago15954: Propagate remote param.
Tom Clegg [Tue, 25 Feb 2020 17:14:32 +0000 (12:14 -0500)]
15954: Propagate remote param.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16101: Redirect logout to Workbench if return_to param missing.
Tom Clegg [Thu, 20 Feb 2020 20:43:50 +0000 (15:43 -0500)]
16101: Redirect logout to Workbench if return_to param missing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16101: Handle logout without sso-provider.
Tom Clegg [Mon, 17 Feb 2020 18:55:27 +0000 (13:55 -0500)]
16101: Handle logout without sso-provider.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16100: Move test to integration suite to avoid logging errors.
Tom Clegg [Sat, 15 Feb 2020 21:47:34 +0000 (16:47 -0500)]
16100: Move test to integration suite to avoid logging errors.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16100: Add package dependency for /etc/mime.types.
Tom Clegg [Fri, 14 Feb 2020 18:40:01 +0000 (13:40 -0500)]
16100: Add package dependency for /etc/mime.types.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16100: Log a warning if /etc/mime.types is missing.
Tom Clegg [Fri, 14 Feb 2020 18:39:38 +0000 (13:39 -0500)]
16100: Log a warning if /etc/mime.types is missing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16100: Test content-type detection.
Tom Clegg [Fri, 14 Feb 2020 18:25:30 +0000 (13:25 -0500)]
16100: Test content-type detection.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16100: Fix bogus timing stats for empty responses.
Tom Clegg [Thu, 13 Feb 2020 07:29:22 +0000 (02:29 -0500)]
16100: Fix bogus timing stats for empty responses.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Touch python clients so they run tests with the latest SDK.
Tom Clegg [Thu, 13 Feb 2020 02:50:07 +0000 (21:50 -0500)]
16039: Touch python clients so they run tests with the latest SDK.

Otherwise, "pip install" (during "install services/nodemanager", for
example) downgrades the SDK from the current version to the latest
version published on pip, and all test suites run with that version,
instead of the version we think we're testing.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Install missing python dep.
Tom Clegg [Thu, 13 Feb 2020 02:47:44 +0000 (21:47 -0500)]
16039: Install missing python dep.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Use current python sdk in py3 tests.
Tom Clegg [Wed, 12 Feb 2020 18:49:50 +0000 (13:49 -0500)]
16039: Use current python sdk in py3 tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Fix & add test for sanitized/unsanitized name conflict.
Tom Clegg [Wed, 12 Feb 2020 15:48:05 +0000 (10:48 -0500)]
16039: Fix & add test for sanitized/unsanitized name conflict.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Accommodate API servers with no config export.
Tom Clegg [Wed, 12 Feb 2020 15:37:48 +0000 (10:37 -0500)]
16039: Accommodate API servers with no config export.

Add more checks in test cases.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16039: Obey ForwardSlashNameSubstitution config in arv-mount.
Tom Clegg [Tue, 11 Feb 2020 15:58:28 +0000 (10:58 -0500)]
16039: Obey ForwardSlashNameSubstitution config in arv-mount.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoTweak upgrade notes
Peter Amstutz [Fri, 14 Feb 2020 18:54:08 +0000 (13:54 -0500)]
Tweak upgrade notes

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16152-keep-balance-http-crash' 2.0.0
Tom Clegg [Fri, 14 Feb 2020 00:22:45 +0000 (19:22 -0500)]
Merge branch '16152-keep-balance-http-crash'

fixes #16152

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16152: Fix nil http handler and ignored config args.
Tom Clegg [Thu, 13 Feb 2020 20:58:09 +0000 (15:58 -0500)]
16152: Fix nil http handler and ignored config args.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16144-nil-checks' refs #16144
Peter Amstutz [Wed, 12 Feb 2020 03:37:44 +0000 (22:37 -0500)]
Merge branch '16144-nil-checks' refs #16144

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16144: Adds more checks and simplifies nested conditionals.
Lucas Di Pentima [Tue, 11 Feb 2020 23:14:42 +0000 (20:14 -0300)]
16144: Adds more checks and simplifies nested conditionals.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16144: Fixes the bug.
Lucas Di Pentima [Tue, 11 Feb 2020 22:22:24 +0000 (19:22 -0300)]
16144: Fixes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16144: Adds tests exposing the bug.
Lucas Di Pentima [Tue, 11 Feb 2020 22:21:09 +0000 (19:21 -0300)]
16144: Adds tests exposing the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '16141-gosdk-missing-fields'
Lucas Di Pentima [Tue, 11 Feb 2020 17:57:54 +0000 (14:57 -0300)]
Merge branch '16141-gosdk-missing-fields'

Closes #16141

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Fixes User type, adjusts fixture to make the test pass.
Lucas Di Pentima [Tue, 11 Feb 2020 17:10:48 +0000 (14:10 -0300)]
16141: Fixes User type, adjusts fixture to make the test pass.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Updates the test to use a different, more complete collection fixture.
Lucas Di Pentima [Tue, 11 Feb 2020 16:41:41 +0000 (13:41 -0300)]
16141: Updates the test to use a different, more complete collection fixture.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Fixes GoSDK's Collection type.
Lucas Di Pentima [Tue, 11 Feb 2020 16:41:17 +0000 (13:41 -0300)]
16141: Fixes GoSDK's Collection type.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Fixes test to use the 1st keep_service's uuid the test server returns.
Lucas Di Pentima [Mon, 10 Feb 2020 19:22:07 +0000 (16:22 -0300)]
16141: Fixes test to use the 1st keep_service's uuid the test server returns.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Adds better message on test in failing situations.
Lucas Di Pentima [Mon, 10 Feb 2020 18:40:47 +0000 (15:40 -0300)]
16141: Adds better message on test in failing situations.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '15835-metrics-docs'
Tom Clegg [Mon, 10 Feb 2020 18:03:15 +0000 (13:03 -0500)]
Merge branch '15835-metrics-docs'

closes #15835

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoPin virtualenv<20
Peter Amstutz [Mon, 10 Feb 2020 17:54:49 +0000 (12:54 -0500)]
Pin virtualenv<20

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16141: Fixes GoSDK's User type.
Lucas Di Pentima [Mon, 10 Feb 2020 17:41:44 +0000 (14:41 -0300)]
16141: Fixes GoSDK's User type.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Expands test for other object types.
Lucas Di Pentima [Mon, 10 Feb 2020 17:41:10 +0000 (14:41 -0300)]
16141: Expands test for other object types.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoSet upper version of setuptools for Python 2 in Centos7 container
Peter Amstutz [Mon, 10 Feb 2020 16:11:26 +0000 (11:11 -0500)]
Set upper version of setuptools for Python 2 in Centos7 container

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoSet upper version of setuptools for Python 2 support
Peter Amstutz [Mon, 10 Feb 2020 16:09:36 +0000 (11:09 -0500)]
Set upper version of setuptools for Python 2 support

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16141: Fixes Collection type on Go SDK. Adjusts test.
Lucas Di Pentima [Mon, 10 Feb 2020 15:41:12 +0000 (12:41 -0300)]
16141: Fixes Collection type on Go SDK. Adjusts test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoMerge branch '16139-cwl-secondaryfiles-fix'
Peter Amstutz [Mon, 10 Feb 2020 15:14:05 +0000 (10:14 -0500)]
Merge branch '16139-cwl-secondaryfiles-fix'

fixes #16139

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAnnotate notify uses localStorage instead of sessionStorage
Peter Amstutz [Mon, 10 Feb 2020 15:13:07 +0000 (10:13 -0500)]
Annotate notify uses localStorage instead of sessionStorage

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16139: Add test
Peter Amstutz [Sun, 9 Feb 2020 23:07:15 +0000 (18:07 -0500)]
16139: Add test

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16141: Adds missing collection fields to GoSDK.
Lucas Di Pentima [Fri, 7 Feb 2020 23:40:51 +0000 (20:40 -0300)]
16141: Adds missing collection fields to GoSDK.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16141: Exposes the bug.
Lucas Di Pentima [Fri, 7 Feb 2020 23:40:17 +0000 (20:40 -0300)]
16141: Exposes the bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago16139: Fix secondaryFile errors when running --submit --no-wait
Peter Amstutz [Fri, 7 Feb 2020 20:59:10 +0000 (15:59 -0500)]
16139: Fix secondaryFile errors when running --submit --no-wait

When submitting, we want to preserve the original CWL version of the
document.

However, when it creates a RunnerContainer (a cwltool.Process) it
examines the input interface and expects it to be in the 1.1 data
model.  But if we preserve/reload the original document, it is still
in the 1.0 data model.  This causes

The code was deliberately overriding the CWL version in metadata to
make this work.  This results in the problem reported on this ticket.

The fix is to maintain both the updated and preserved documents, and
use them appropriately where they are expected.

refs #16139

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago15835: Update metrics docs.
Tom Clegg [Fri, 7 Feb 2020 21:07:06 +0000 (16:07 -0500)]
15835: Update metrics docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoAdd note that some services require a minimal config.yml to start.
Peter Amstutz [Fri, 7 Feb 2020 16:36:53 +0000 (11:36 -0500)]
Add note that some services require a minimal config.yml to start.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoImprove readability of upgrade notes
Peter Amstutz [Thu, 6 Feb 2020 22:36:12 +0000 (17:36 -0500)]
Improve readability of upgrade notes

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16133-federation-loop'
Tom Clegg [Thu, 6 Feb 2020 19:59:07 +0000 (14:59 -0500)]
Merge branch '16133-federation-loop'

fixes #16133

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16133: Don't take multiple federation hops.
Tom Clegg [Thu, 6 Feb 2020 19:16:55 +0000 (14:16 -0500)]
16133: Don't take multiple federation hops.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '16136-original-owner' closes #16136
Peter Amstutz [Thu, 6 Feb 2020 17:25:54 +0000 (12:25 -0500)]
Merge branch '16136-original-owner' closes #16136

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16136: Prioritize 'Function' over 'Value' and ignore null
Peter Amstutz [Thu, 6 Feb 2020 16:42:59 +0000 (11:42 -0500)]
16136: Prioritize 'Function' over 'Value' and ignore null

Because arvados-server config-dump will emit "Value: null" even if the
original config file didn't provide it.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16133: Don't take multiple hops when getting collections.
Tom Clegg [Thu, 6 Feb 2020 16:09:32 +0000 (11:09 -0500)]
16133: Don't take multiple hops when getting collections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16133: Avoid cycle between old and new controllers.
Tom Clegg [Thu, 6 Feb 2020 15:45:48 +0000 (10:45 -0500)]
16133: Avoid cycle between old and new controllers.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16133: Avoid cycle when searching federation for PDH.
Tom Clegg [Thu, 6 Feb 2020 15:19:04 +0000 (10:19 -0500)]
16133: Avoid cycle when searching federation for PDH.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoAdd a note to the install section
Peter Amstutz [Wed, 5 Feb 2020 18:33:02 +0000 (13:33 -0500)]
Add a note to the install section

Add note that you should look in the SDKs section for client tools.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16111-ssh-help' closes #16111
Peter Amstutz [Tue, 4 Feb 2020 14:43:36 +0000 (09:43 -0500)]
Merge branch '16111-ssh-help' closes #16111

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '16104-select-uuid'
Tom Clegg [Mon, 3 Feb 2020 22:04:16 +0000 (17:04 -0500)]
Merge branch '16104-select-uuid'

refs #16104

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years ago16111: Add SSHHelpPageHTML so admin can provide custom instructions
Peter Amstutz [Mon, 3 Feb 2020 21:02:36 +0000 (16:02 -0500)]
16111: Add SSHHelpPageHTML so admin can provide custom instructions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16104: Don't impose federated query limitations on local-only query.
Tom Clegg [Mon, 3 Feb 2020 20:28:04 +0000 (15:28 -0500)]
16104: Don't impose federated query limitations on local-only query.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoIn our systemd service files, set a reasonable default for the open file
Ward Vandewege [Mon, 3 Feb 2020 19:39:20 +0000 (14:39 -0500)]
In our systemd service files, set a reasonable default for the open file
limit (65536).

refs #16089

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoMerge branch '16108-favorites' refs #16108
Peter Amstutz [Mon, 3 Feb 2020 19:35:42 +0000 (14:35 -0500)]
Merge branch '16108-favorites' refs #16108

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16108: Use zzzzz- in examples
Peter Amstutz [Mon, 3 Feb 2020 19:34:53 +0000 (14:34 -0500)]
16108: Use zzzzz- in examples

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago16108: Change wb1 'favorites' to filter on user_uuid instead of tail_uuid
Peter Amstutz [Sun, 2 Feb 2020 17:58:16 +0000 (12:58 -0500)]
16108: Change wb1 'favorites' to filter on user_uuid instead of tail_uuid

Fixes potential exploit by which a malicious user could inject
favorites into another user's favorites list.

Public favorites owned by "All Users" will now appear in wb1.

Move documentation about 'star' links from admin section to API
documentation.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate maintainer and vendor for our OS packages.
Ward Vandewege [Sun, 2 Feb 2020 15:55:17 +0000 (10:55 -0500)]
Update maintainer and vendor for our OS packages.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoWorkbench 2 isn't really beta if we're telling people to use it.
Peter Amstutz [Thu, 30 Jan 2020 23:28:41 +0000 (18:28 -0500)]
Workbench 2 isn't really beta if we're telling people to use it.

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate more documentation for new config file, add 'group favorites'
Peter Amstutz [Thu, 30 Jan 2020 23:17:43 +0000 (18:17 -0500)]
Update more documentation for new config file, add 'group favorites'

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoWorkbench1 package depends on graphviz
Peter Amstutz [Thu, 30 Jan 2020 21:57:08 +0000 (16:57 -0500)]
Workbench1 package depends on graphviz

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAdds information about the new vocabulary format on the release notes.
Lucas Di Pentima [Thu, 30 Jan 2020 19:52:17 +0000 (16:52 -0300)]
Adds information about the new vocabulary format on the release notes.

No issue #

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoPin bundler to version compatible with all Rubies we support
Peter Amstutz [Thu, 30 Jan 2020 18:00:31 +0000 (13:00 -0500)]
Pin bundler to version compatible with all Rubies we support

refs #16015

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUpdate text to refer to Arvados 2.0
Peter Amstutz [Thu, 30 Jan 2020 15:50:38 +0000 (10:50 -0500)]
Update text to refer to Arvados 2.0

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoMerge branch '15902-slow-query'
Lucas Di Pentima [Tue, 28 Jan 2020 22:06:50 +0000 (19:06 -0300)]
Merge branch '15902-slow-query'

Closes #15902

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years ago15902: Adjust order to avoid poorly optimized query.
Tom Clegg [Tue, 28 Jan 2020 20:05:02 +0000 (15:05 -0500)]
15902: Adjust order to avoid poorly optimized query.

When many requesting_container_uuid values are passed in this query,
the default order (["modified_at desc", "uuid"]) sometimes causes
PostgreSQL to begin with the timestamp index, which results in a slow
query. The requesting_container_uuid index makes the query much
faster.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>

4 years agoMerge branch '15944-arvbox-publicdev-fix'
Lucas Di Pentima [Tue, 28 Jan 2020 18:51:35 +0000 (15:51 -0300)]
Merge branch '15944-arvbox-publicdev-fix'

Closes #15944

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoarvados/jobs image uses python3 for arvados-cwl-runner
Peter Amstutz [Mon, 27 Jan 2020 23:05:23 +0000 (18:05 -0500)]
arvados/jobs image uses python3 for arvados-cwl-runner

This avoids the deprecation warning from cwltool.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoAdd python3 build of arvados-cwl-runner
Peter Amstutz [Mon, 27 Jan 2020 18:31:49 +0000 (13:31 -0500)]
Add python3 build of arvados-cwl-runner

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago15944: Takes arvbox container's default gw IP as internal client.
Lucas Di Pentima [Fri, 24 Jan 2020 22:22:06 +0000 (19:22 -0300)]
15944: Takes arvbox container's default gw IP as internal client.

This makes Keep work on 'publicdev' mode.
Also, turns off nginx's response caching for easier debugging.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

4 years agoAdd a bit more discussion about DNS on compute node images closes #16085
Peter Amstutz [Fri, 24 Jan 2020 20:22:48 +0000 (15:22 -0500)]
Add a bit more discussion about DNS on compute node images closes #16085

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years agoUse updated run_upload_packages
Peter Amstutz [Fri, 24 Jan 2020 16:36:30 +0000 (11:36 -0500)]
Use updated run_upload_packages

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

4 years ago15944: Merge branch 'master' into 15944-arvbox-publicdev-fix
Lucas Di Pentima [Fri, 24 Jan 2020 14:41:05 +0000 (11:41 -0300)]
15944: Merge branch 'master' into 15944-arvbox-publicdev-fix