arvados.git
3 years ago17708: when crunch-run encounters an unsupported file type while copying
Ward Vandewege [Fri, 21 May 2021 01:30:31 +0000 (21:30 -0400)]
17708: when crunch-run encounters an unsupported file type while copying
       the output of a job, do not error out, just print a warning and
       skip the offending file.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix case statement for selection of wb1 login UI, refs #17499
Peter Amstutz [Thu, 20 May 2021 20:02:37 +0000 (16:02 -0400)]
Fix case statement for selection of wb1 login UI, refs #17499

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17680-oidctoken-doc'
Tom Clegg [Thu, 20 May 2021 14:01:45 +0000 (10:01 -0400)]
Merge branch '17680-oidctoken-doc'

closes #17680

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17680: Add OIDC access token section to "tokens" page.
Tom Clegg [Wed, 19 May 2021 20:22:54 +0000 (16:22 -0400)]
17680: Add OIDC access token section to "tokens" page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17680: Document accepting OIDC access tokens.
Tom Clegg [Wed, 19 May 2021 18:36:37 +0000 (14:36 -0400)]
17680: Document accepting OIDC access tokens.

Also update examples to use Bearer instead of OAuth2 in Authorization
header.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoChange the default for Collections.WebDAVCache.MaxBlockEntries from 4 to
Ward Vandewege [Wed, 19 May 2021 16:15:17 +0000 (12:15 -0400)]
Change the default for Collections.WebDAVCache.MaxBlockEntries from 4 to
20.

refs #17689

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoUpdate "supported distributions" table no issue #
Peter Amstutz [Wed, 19 May 2021 14:06:12 +0000 (10:06 -0400)]
Update "supported distributions" table no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17677-authenticate-endpoint' refs #17677
Peter Amstutz [Tue, 18 May 2021 21:16:32 +0000 (17:16 -0400)]
Merge branch '17677-authenticate-endpoint' refs #17677

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17604-change-collections-urls'
Javier Bértoli [Tue, 18 May 2021 21:00:08 +0000 (18:00 -0300)]
Merge branch '17604-change-collections-urls'

closes #17604
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofeat(provision): use LE wildcard cert for *.collections
Javier Bértoli [Tue, 18 May 2021 13:50:44 +0000 (10:50 -0300)]
feat(provision): use LE wildcard cert for *.collections

refs #17604
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '17606-file-globbing-issues'
Javier Bértoli [Tue, 18 May 2021 20:24:38 +0000 (17:24 -0300)]
Merge branch '17606-file-globbing-issues'

closes #17606
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofix(provision): prevent bash globbing issues with empty dirs
Javier Bértoli [Tue, 18 May 2021 17:50:51 +0000 (14:50 -0300)]
fix(provision): prevent bash globbing issues with empty dirs

refs #17606
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofix(provision): don't install docker-compose
Javier Bértoli [Mon, 17 May 2021 22:48:30 +0000 (19:48 -0300)]
fix(provision): don't install docker-compose

no issue #
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoA few more doc tweaks.
Ward Vandewege [Mon, 17 May 2021 21:22:52 +0000 (17:22 -0400)]
A few more doc tweaks.

refs #17668

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17677: Document the "authenticate" endpoint
Peter Amstutz [Mon, 17 May 2021 20:48:16 +0000 (16:48 -0400)]
17677: Document the "authenticate" endpoint

Clean up a few other details related to login as well.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17678-costanalyzer-doc'
Ward Vandewege [Mon, 17 May 2021 20:24:07 +0000 (16:24 -0400)]
Merge branch '17678-costanalyzer-doc'

closes #17678

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17678: Merge branch 'master' into 17678-costanalyzer-doc
Ward Vandewege [Mon, 17 May 2021 19:52:08 +0000 (15:52 -0400)]
17678: Merge branch 'master' into 17678-costanalyzer-doc

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17678: implement review feedback.
Ward Vandewege [Mon, 17 May 2021 19:20:28 +0000 (15:20 -0400)]
17678: implement review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17668-doc-container-shell-access'
Ward Vandewege [Mon, 17 May 2021 16:18:53 +0000 (12:18 -0400)]
Merge branch '17668-doc-container-shell-access'

closes #17668

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17668: Merge branch 'master' into 17668-doc-container-shell-access
Ward Vandewege [Mon, 17 May 2021 16:18:19 +0000 (12:18 -0400)]
17668: Merge branch 'master' into 17668-doc-container-shell-access

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17668: one more tweak to the firewall rule description for this feature.
Ward Vandewege [Mon, 17 May 2021 16:17:07 +0000 (12:17 -0400)]
17668: one more tweak to the firewall rule description for this feature.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17668: A few more tweaks after review feedback.
Ward Vandewege [Mon, 17 May 2021 15:25:29 +0000 (11:25 -0400)]
17668: A few more tweaks after review feedback.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17499-wb-login-form' refs #17499
Peter Amstutz [Mon, 17 May 2021 14:57:03 +0000 (10:57 -0400)]
Merge branch '17499-wb-login-form' refs #17499

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17499: Fix tests
Peter Amstutz [Mon, 17 May 2021 13:42:53 +0000 (09:42 -0400)]
17499: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17668: basic documentation for the new container shell feature.
Ward Vandewege [Fri, 14 May 2021 22:09:38 +0000 (18:09 -0400)]
17668: basic documentation for the new container shell feature.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17499: Fixing tests WIP
Peter Amstutz [Fri, 14 May 2021 21:34:00 +0000 (17:34 -0400)]
17499: Fixing tests WIP

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17678: add documentation for the `arvados-client costanalyzer` command.
Ward Vandewege [Fri, 14 May 2021 21:06:21 +0000 (17:06 -0400)]
17678: add documentation for the `arvados-client costanalyzer` command.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17499: Rewrite javascript to pass phantomJS tests
Peter Amstutz [Fri, 14 May 2021 20:19:39 +0000 (16:19 -0400)]
17499: Rewrite javascript to pass phantomJS tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17499: Adjust the default welcome page text used by wb1/wb2
Peter Amstutz [Fri, 14 May 2021 19:13:25 +0000 (15:13 -0400)]
17499: Adjust the default welcome page text used by wb1/wb2

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17499: Adjust formatting. improve error reporting.
Peter Amstutz [Fri, 14 May 2021 19:03:46 +0000 (15:03 -0400)]
17499: Adjust formatting. improve error reporting.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMention the 'Rendezvous' parameter on Keepstore's InternalURLs, for
Ward Vandewege [Fri, 14 May 2021 17:40:40 +0000 (13:40 -0400)]
Mention the 'Rendezvous' parameter on Keepstore's InternalURLs, for
migration from old config.

refs #13647

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17499: Add logo so default WelcomePageHTML works
Peter Amstutz [Thu, 13 May 2021 21:33:58 +0000 (17:33 -0400)]
17499: Add logo so default WelcomePageHTML works

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17499: Support username/password login on workbench1
Peter Amstutz [Thu, 13 May 2021 21:27:28 +0000 (17:27 -0400)]
17499: Support username/password login on workbench1

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix integration test closes #17673
Peter Amstutz [Thu, 13 May 2021 20:13:10 +0000 (16:13 -0400)]
Fix integration test closes #17673

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17589-contents-count-none' refs #17589
Peter Amstutz [Thu, 13 May 2021 13:46:12 +0000 (09:46 -0400)]
Merge branch '17589-contents-count-none' refs #17589

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17589: when count=none and offset=0 don't do unnecessary row counts
Peter Amstutz [Wed, 12 May 2021 22:53:11 +0000 (18:53 -0400)]
17589: when count=none and offset=0 don't do unnecessary row counts

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17589: Disallow offset with group contents count=none
Peter Amstutz [Wed, 12 May 2021 22:03:43 +0000 (18:03 -0400)]
17589: Disallow offset with group contents count=none

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix typo in a comment on the configuration reference.
Ward Vandewege [Wed, 12 May 2021 17:20:56 +0000 (13:20 -0400)]
Fix typo in a comment on the configuration reference.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17589: Use more efficient & stable keyset paging.
Peter Amstutz [Tue, 11 May 2021 20:29:40 +0000 (16:29 -0400)]
17589: Use more efficient & stable keyset paging.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17589: Ensure that 'group contents' requests are routed correctly
Peter Amstutz [Tue, 11 May 2021 20:09:19 +0000 (16:09 -0400)]
17589: Ensure that 'group contents' requests are routed correctly

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17657-ssh-port-forward'
Tom Clegg [Mon, 10 May 2021 19:48:12 +0000 (15:48 -0400)]
Merge branch '17657-ssh-port-forward'

refs #17657

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17449-gxp-docs' refs #17449
Peter Amstutz [Mon, 10 May 2021 17:25:53 +0000 (13:25 -0400)]
Merge branch '17449-gxp-docs' refs #17449

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17449: Clarifications and edits.
Peter Amstutz [Mon, 10 May 2021 17:21:51 +0000 (13:21 -0400)]
17449: Clarifications and edits.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17651-sso-deprecated'
Nico Cesar [Mon, 10 May 2021 14:39:10 +0000 (10:39 -0400)]
Merge branch '17651-sso-deprecated'

Documentation updates about sso deprecation

closes #17651

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago17657: Improve test failure messages.
Tom Clegg [Mon, 10 May 2021 14:17:16 +0000 (10:17 -0400)]
17657: Improve test failure messages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17657: Test port forwarding using OpenSSH client.
Tom Clegg [Sun, 9 May 2021 05:08:13 +0000 (01:08 -0400)]
17657: Test port forwarding using OpenSSH client.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17657: Allow ssh port forwarding to container ports.
Tom Clegg [Sun, 9 May 2021 00:04:15 +0000 (20:04 -0400)]
17657: Allow ssh port forwarding to container ports.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17651: Wording tweak on the SSO authentication flow page.
Ward Vandewege [Fri, 7 May 2021 21:09:49 +0000 (17:09 -0400)]
17651: Wording tweak on the SSO authentication flow page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17590-ec2-config-keys'
Tom Clegg [Fri, 7 May 2021 15:33:25 +0000 (11:33 -0400)]
Merge branch '17590-ec2-config-keys'

fixes #17590

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17590: Add upgrade note about changed config keys.
Tom Clegg [Fri, 7 May 2021 14:48:45 +0000 (10:48 -0400)]
17590: Add upgrade note about changed config keys.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17590: Error out if both old+new S3 credential configs are provided.
Tom Clegg [Fri, 7 May 2021 14:33:10 +0000 (10:33 -0400)]
17590: Error out if both old+new S3 credential configs are provided.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17590: Fix spelling of S3 driver name.
Tom Clegg [Fri, 7 May 2021 14:18:59 +0000 (10:18 -0400)]
17590: Fix spelling of S3 driver name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17449: Expand the "token expiration" admin page
Peter Amstutz [Thu, 6 May 2021 21:42:30 +0000 (17:42 -0400)]
17449: Expand the "token expiration" admin page

Adds discussion of Workbench.IdleTimeout and API.MaxTokenLifetime and
attempts to explain the differences between them.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoSSO deprecation in documentation
Nico Cesar [Thu, 6 May 2021 19:27:26 +0000 (15:27 -0400)]
SSO deprecation in documentation

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago17590: Improve log messages when old AWS config keys are used.
Tom Clegg [Thu, 6 May 2021 17:21:36 +0000 (13:21 -0400)]
17590: Improve log messages when old AWS config keys are used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17590: Merge branch 'master'
Tom Clegg [Thu, 6 May 2021 15:29:27 +0000 (11:29 -0400)]
17590: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17650-fix-cwl-tests'
Tom Clegg [Thu, 6 May 2021 14:40:27 +0000 (10:40 -0400)]
Merge branch '17650-fix-cwl-tests'

fixes #17650

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17650: Fix test order dependency.
Tom Clegg [Wed, 5 May 2021 20:39:54 +0000 (16:39 -0400)]
17650: Fix test order dependency.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoThe upgrade to go 1.16.3 requires a `go mod tidy` to make the build
Ward Vandewege [Wed, 5 May 2021 19:36:44 +0000 (15:36 -0400)]
The upgrade to go 1.16.3 requires a `go mod tidy` to make the build
work.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17590: Rename S3 volume config keys to match AWS terms.
Tom Clegg [Tue, 4 May 2021 21:24:03 +0000 (17:24 -0400)]
17590: Rename S3 volume config keys to match AWS terms.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix test regexp.
Tom Clegg [Wed, 5 May 2021 17:32:29 +0000 (13:32 -0400)]
Fix test regexp.

amends commit 51d072fb24a69336c09fe4320102f5797db0d5e0

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUpdate `arvados-server install` to use go 1.16.3
Ward Vandewege [Wed, 5 May 2021 16:42:32 +0000 (12:42 -0400)]
Update `arvados-server install` to use go 1.16.3

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17587-workbench-federated-query'
Tom Clegg [Wed, 5 May 2021 13:40:29 +0000 (09:40 -0400)]
Merge branch '17587-workbench-federated-query'

fixes #17587

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17587: Comment SetUpTest calls.
Tom Clegg [Wed, 5 May 2021 13:39:48 +0000 (09:39 -0400)]
17587: Comment SetUpTest calls.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix "docker port" usage.
Tom Clegg [Wed, 5 May 2021 13:29:35 +0000 (09:29 -0400)]
Fix "docker port" usage.

"docker port foo tcp/9999" can return multiple addresses like
"0.0.0.0:12345\n:::12345\n".

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoOur python3 packages should be marked as 'Replaces:' for the old python
Ward Vandewege [Tue, 4 May 2021 21:40:30 +0000 (17:40 -0400)]
Our python3 packages should be marked as 'Replaces:' for the old python
packages, so that there are no "trying to overwrite" errors, e.g. when
upgrading python-crunchstat-summary from 2.0.4 to the latest 2.2.x
release.

refs #17498

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17587: Don't reject federated query with large limit param.
Tom Clegg [Tue, 4 May 2021 18:20:27 +0000 (14:20 -0400)]
17587: Don't reject federated query with large limit param.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17610-remote-token-scopes'
Tom Clegg [Tue, 4 May 2021 18:46:04 +0000 (14:46 -0400)]
Merge branch '17610-remote-token-scopes'

fixes #17610

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoDon't ask nginx to log to stderr when using setuid.
Tom Clegg [Tue, 4 May 2021 17:52:22 +0000 (13:52 -0400)]
Don't ask nginx to log to stderr when using setuid.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoUpdate tests for go 1.16.
Tom Clegg [Tue, 4 May 2021 17:52:08 +0000 (13:52 -0400)]
Update tests for go 1.16.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17598-keep-web-url-case' refs #17598
Peter Amstutz [Tue, 4 May 2021 15:41:29 +0000 (11:41 -0400)]
Merge branch '17598-keep-web-url-case' refs #17598

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17284-redact-railsapi-host'
Tom Clegg [Tue, 4 May 2021 14:48:07 +0000 (10:48 -0400)]
Merge branch '17284-redact-railsapi-host'

refs #17284

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17284: Fix missing "/" in error message.
Tom Clegg [Tue, 4 May 2021 05:15:20 +0000 (01:15 -0400)]
17284: Fix missing "/" in error message.

Was showing "//railsapi.internalarvados/..." instead of
"//railsapi.internal/arvados/...".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoBump the Go version to 1.16.3
Ward Vandewege [Mon, 3 May 2021 21:13:24 +0000 (17:13 -0400)]
Bump the Go version to 1.16.3

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17598: Also use lowercase for URL host comparison
Peter Amstutz [Mon, 3 May 2021 18:31:21 +0000 (14:31 -0400)]
17598: Also use lowercase for URL host comparison

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17610: Check scopes when using a remote token.
Tom Clegg [Mon, 3 May 2021 18:08:31 +0000 (14:08 -0400)]
17610: Check scopes when using a remote token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17598-keep-web-url' refs #17598
Peter Amstutz [Thu, 29 Apr 2021 21:35:13 +0000 (17:35 -0400)]
Merge branch '17598-keep-web-url' refs #17598

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17598: stripDefaultPort behavior, error message when !credentialsOK
Peter Amstutz [Thu, 29 Apr 2021 20:10:11 +0000 (16:10 -0400)]
17598: stripDefaultPort behavior, error message when !credentialsOK

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17598: Handle comparison URLs with :80 or :443
Peter Amstutz [Thu, 29 Apr 2021 19:07:44 +0000 (15:07 -0400)]
17598: Handle comparison URLs with :80 or :443

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoThe sks-keyservers network is dying. Update our documentation to use
Ward Vandewege [Thu, 29 Apr 2021 01:28:04 +0000 (21:28 -0400)]
The sks-keyservers network is dying. Update our documentation to use
pgp.mit.edu instead.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16997-sort-config-for-diff'
Tom Clegg [Wed, 28 Apr 2021 15:31:06 +0000 (11:31 -0400)]
Merge branch '16997-sort-config-for-diff'

fixes #16997

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17284-redact-railsapi-host'
Tom Clegg [Tue, 27 Apr 2021 17:07:47 +0000 (13:07 -0400)]
Merge branch '17284-redact-railsapi-host'

fixes #17284

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoDo not pin arvados-login-sync on an ancient version of the arvados gem.
Ward Vandewege [Tue, 27 Apr 2021 16:08:01 +0000 (12:08 -0400)]
Do not pin arvados-login-sync on an ancient version of the arvados gem.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16997: Use go-yaml fork to avoid key-sorting bug.
Tom Clegg [Tue, 27 Apr 2021 16:00:45 +0000 (12:00 -0400)]
16997: Use go-yaml fork to avoid key-sorting bug.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago16997: Test map key order in check/dump. Fix duplicate warnings.
Tom Clegg [Mon, 26 Apr 2021 19:11:58 +0000 (15:11 -0400)]
16997: Test map key order in check/dump. Fix duplicate warnings.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17507-listobjectsv2'
Tom Clegg [Mon, 26 Apr 2021 14:14:15 +0000 (10:14 -0400)]
Merge branch '17507-listobjectsv2'

fixes #17507

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17343-workbench1-optional'
Tom Clegg [Mon, 26 Apr 2021 13:47:01 +0000 (09:47 -0400)]
Merge branch '17343-workbench1-optional'

closes #17343

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17284: Redact RailsAPI host:port in error messages.
Tom Clegg [Mon, 26 Apr 2021 13:40:46 +0000 (09:40 -0400)]
17284: Redact RailsAPI host:port in error messages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17521-dot-slash-glob' refs #17521
Peter Amstutz [Fri, 23 Apr 2021 21:02:41 +0000 (17:02 -0400)]
Merge branch '17521-dot-slash-glob' refs #17521

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17521: Use normpath to clean up the path segment.
Peter Amstutz [Fri, 23 Apr 2021 20:07:02 +0000 (16:07 -0400)]
17521: Use normpath to clean up the path segment.

Removes leading ./, trailing slashes, redundant '.' and blank path
segments, and other stuff that potentially confuses the path matching.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17521: Add tests, refs #10380 and refs #17521
Peter Amstutz [Fri, 23 Apr 2021 19:58:42 +0000 (15:58 -0400)]
17521: Add tests, refs #10380 and refs #17521

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17507: Fix calling Child() without lock.
Tom Clegg [Thu, 22 Apr 2021 20:14:38 +0000 (16:14 -0400)]
17507: Fix calling Child() without lock.

Encountered in services/keep-web tests.

START: handler_test.go:654: IntegrationSuite.TestDirectoryListingWithNoAnonymousToken
START: server_test.go:431: IntegrationSuite.SetUpTest

[...]

fatal error: concurrent map iteration and map write

goroutine 2192 [running]:
runtime.throw(0xe545ab, 0x26)
        /var/lib/arvados/go/src/runtime/panic.go:1112 +0x72 fp=0xc002544c78 sp=0xc002544c48 pc=0x4366b2
runtime.mapiternext(0xc002544d18)
        /var/lib/arvados/go/src/runtime/map.go:853 +0x552 fp=0xc002544cf8 sp=0xc002544c78 pc=0x411442
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc00202e480, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:336 +0x106 fp=0xc002544d98 sp=0xc002544cf8 pc=0x8c7de6
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc0003a66c0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:337 +0xe3 fp=0xc002544e38 sp=0xc002544d98 pc=0x8c7dc3
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc0020c3680, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:337 +0xe3 fp=0xc002544ed8 sp=0xc002544e38 pc=0x8c7dc3
git.arvados.org/arvados.git/sdk/go/arvados.(*fileSystem).MemorySize(0xc001af0000, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:631 +0x33 fp=0xc002544ef8 sp=0xc002544ed8 pc=0x8ca1e3
git.arvados.org/arvados.git/services/keep-web.(*cache).collectionBytes(0xc00261da08, 0x3ff0000000000101)
        /home/tom/arvados/services/keep-web/cache.go:448 +0x21c fp=0xc002544f70 sp=0xc002544ef8 pc=0xc2903c
git.arvados.org/arvados.git/services/keep-web.(*cache).updateGauges(0xc00261da08)
        /home/tom/arvados/services/keep-web/cache.go:170 +0x2f fp=0xc002544f90 sp=0xc002544f70 pc=0xc269bf
git.arvados.org/arvados.git/services/keep-web.(*cache).setup.func1(0xc00261da08)
        /home/tom/arvados/services/keep-web/cache.go:164 +0x6d fp=0xc002544fd8 sp=0xc002544f90 pc=0xc65dad
runtime.goexit()
        /var/lib/arvados/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc002544fe0 sp=0xc002544fd8 pc=0x468dd1
created by git.arvados.org/arvados.git/services/keep-web.(*cache).setup
        /home/tom/arvados/services/keep-web/cache.go:162 +0x1db

goroutine 2144 [runnable]:
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).Child(0xc00202e480, 0xc002c4e117, 0x9, 0xc0005c2070, 0x10c9680, 0xc0021d0240, 0x0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:289 +0x171
git.arvados.org/arvados.git/sdk/go/arvados.(*lookupnode).Readdir(0xc00202e480, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_lookup.go:53 +0x27b
git.arvados.org/arvados.git/sdk/go/arvados.(*filehandle).Readdir(0xc00263c140, 0x0, 0xc002066360, 0xc00263c140, 0x10c4f40, 0xc00263c140, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_filehandle.go:81 +0x1d2
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xc00161a574, 0xc, 0x10c17c0, 0xc00227c280, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:772 +0x25e
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xe2f941, 0x5, 0x10c17c0, 0xc002b1a4c0, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:786 +0x564
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xc00216be76, 0x0, 0x10c17c0, 0xc002b1a0c0, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:786 +0x564
golang.org/x/net/webdav.(*Handler).handlePropfind(0xc002b1a080, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900, 0xc0022bc718, 0x415ad3, 0xc002b1a080)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/webdav.go:566 +0x3d7
golang.org/x/net/webdav.(*Handler).ServeHTTP(0xc002b1a080, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/webdav.go:67 +0x556
git.arvados.org/arvados.git/services/keep-web.(*handler).serveSiteFS(0xc002221ae0, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900, 0xc002b22010, 0x1, 0x1, 0x101)
        /home/tom/arvados/services/keep-web/handler.go:593 +0x80b
git.arvados.org/arvados.git/services/keep-web.(*handler).ServeHTTP(0xc002221ae0, 0x10b6ec0, 0xc002b094a0, 0xc002b04900)
        /home/tom/arvados/services/keep-web/handler.go:330 +0x2a0b
git.arvados.org/arvados.git/sdk/go/httpserver.LogRequests.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /home/tom/arvados/sdk/go/httpserver/logger.go:56 +0x8d8
net/http.HandlerFunc.ServeHTTP(0xc002282460, 0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.AddRequestIDs.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /home/tom/arvados/sdk/go/httpserver/id_generator.go:57 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc002282480, 0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.HandlerWithContext.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04700)
        /home/tom/arvados/sdk/go/httpserver/logger.go:30 +0x107
net/http.HandlerFunc.ServeHTTP(0xc002901b30, 0x7f1c58319c58, 0xc002b09470, 0xc002b04700)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func1(0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/github.com/prometheus/client_golang@v1.2.1/prometheus/promhttp/instrument_server.go:68 +0x11c
net/http.HandlerFunc.ServeHTTP(0xc002901e00, 0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.(*metrics).ServeHTTP(0xc002113e00, 0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /home/tom/arvados/sdk/go/httpserver/metrics.go:70 +0x51
git.arvados.org/arvados.git/services/keep-web.(*IntegrationSuite).testDirectoryListing(0xc000010138, 0xc0007d41e0)
        /home/tom/arvados/services/keep-web/handler_test.go:864 +0x1a07
git.arvados.org/arvados.git/services/keep-web.(*IntegrationSuite).TestDirectoryListingWithNoAnonymousToken(0xc000010138, 0xc0007d41e0)
        /home/tom/arvados/services/keep-web/handler_test.go:656 +0x67
reflect.Value.call(0xe2bbc0, 0xc000010138, 0x4613, 0xe2deb6, 0x4, 0xc000282f08, 0x1, 0x1, 0x171d800, 0xc000282e48, ...)
        /var/lib/arvados/go/src/reflect/value.go:460 +0x8ab
reflect.Value.Call(0xe2bbc0, 0xc000010138, 0x4613, 0xc000282f08, 0x1, 0x1, 0xc0007d42d0, 0xc000128060, 0xc00287a720)
        /var/lib/arvados/go/src/reflect/value.go:321 +0xb4
gopkg.in/check%2ev1.(*suiteRunner).forkTest.func1(0xc0007d41e0)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:772 +0x628
gopkg.in/check%2ev1.(*suiteRunner).forkCall.func1(0xc000174c00, 0xc0007d41e0, 0xc00222b760)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:666 +0x98
created by gopkg.in/check%2ev1.(*suiteRunner).forkCall
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:663 +0x1fb

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17507: Support ListObjectsV2 API.
Tom Clegg [Thu, 22 Apr 2021 19:12:17 +0000 (15:12 -0400)]
17507: Support ListObjectsV2 API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17566-max-request-size'
Tom Clegg [Wed, 21 Apr 2021 20:25:59 +0000 (16:25 -0400)]
Merge branch '17566-max-request-size'

fixes #17566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17204-not-modified'
Tom Clegg [Wed, 21 Apr 2021 20:25:37 +0000 (16:25 -0400)]
Merge branch '17204-not-modified'

fixes #17204

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17346-keep-balance-empty-block'
Tom Clegg [Wed, 21 Apr 2021 19:50:27 +0000 (15:50 -0400)]
Merge branch '17346-keep-balance-empty-block'

fixes #17346

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17343: Add option to boot without workbench1 to speed up tests.
Tom Clegg [Wed, 21 Apr 2021 19:03:59 +0000 (15:03 -0400)]
17343: Add option to boot without workbench1 to speed up tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17570-remove-extra-example-agrantfile'
Javier Bértoli [Wed, 21 Apr 2021 15:10:24 +0000 (12:10 -0300)]
Merge branch '17570-remove-extra-example-agrantfile'

closes #17570
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofix(provision): remove unneeded vagantfile
Javier Bértoli [Wed, 21 Apr 2021 15:01:30 +0000 (12:01 -0300)]
fix(provision): remove unneeded vagantfile

refs #17570
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17204: Fix misleading log message on 304 Not Modified response.
Tom Clegg [Tue, 20 Apr 2021 23:43:57 +0000 (19:43 -0400)]
17204: Fix misleading log message on 304 Not Modified response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>