arvados.git
2 years agoMerge branch '17449-gxp-docs' refs #17449
Peter Amstutz [Mon, 10 May 2021 17:25:53 +0000 (13:25 -0400)]
Merge branch '17449-gxp-docs' refs #17449

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17449: Clarifications and edits.
Peter Amstutz [Mon, 10 May 2021 17:21:51 +0000 (13:21 -0400)]
17449: Clarifications and edits.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17651-sso-deprecated'
Nico Cesar [Mon, 10 May 2021 14:39:10 +0000 (10:39 -0400)]
Merge branch '17651-sso-deprecated'

Documentation updates about sso deprecation

closes #17651

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

2 years ago17651: Wording tweak on the SSO authentication flow page.
Ward Vandewege [Fri, 7 May 2021 21:09:49 +0000 (17:09 -0400)]
17651: Wording tweak on the SSO authentication flow page.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17590-ec2-config-keys'
Tom Clegg [Fri, 7 May 2021 15:33:25 +0000 (11:33 -0400)]
Merge branch '17590-ec2-config-keys'

fixes #17590

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17590: Add upgrade note about changed config keys.
Tom Clegg [Fri, 7 May 2021 14:48:45 +0000 (10:48 -0400)]
17590: Add upgrade note about changed config keys.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17590: Error out if both old+new S3 credential configs are provided.
Tom Clegg [Fri, 7 May 2021 14:33:10 +0000 (10:33 -0400)]
17590: Error out if both old+new S3 credential configs are provided.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17590: Fix spelling of S3 driver name.
Tom Clegg [Fri, 7 May 2021 14:18:59 +0000 (10:18 -0400)]
17590: Fix spelling of S3 driver name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17449: Expand the "token expiration" admin page
Peter Amstutz [Thu, 6 May 2021 21:42:30 +0000 (17:42 -0400)]
17449: Expand the "token expiration" admin page

Adds discussion of Workbench.IdleTimeout and API.MaxTokenLifetime and
attempts to explain the differences between them.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoSSO deprecation in documentation
Nico Cesar [Thu, 6 May 2021 19:27:26 +0000 (15:27 -0400)]
SSO deprecation in documentation

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

2 years ago17590: Improve log messages when old AWS config keys are used.
Tom Clegg [Thu, 6 May 2021 17:21:36 +0000 (13:21 -0400)]
17590: Improve log messages when old AWS config keys are used.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17590: Merge branch 'master'
Tom Clegg [Thu, 6 May 2021 15:29:27 +0000 (11:29 -0400)]
17590: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17650-fix-cwl-tests'
Tom Clegg [Thu, 6 May 2021 14:40:27 +0000 (10:40 -0400)]
Merge branch '17650-fix-cwl-tests'

fixes #17650

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17650: Fix test order dependency.
Tom Clegg [Wed, 5 May 2021 20:39:54 +0000 (16:39 -0400)]
17650: Fix test order dependency.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoThe upgrade to go 1.16.3 requires a `go mod tidy` to make the build
Ward Vandewege [Wed, 5 May 2021 19:36:44 +0000 (15:36 -0400)]
The upgrade to go 1.16.3 requires a `go mod tidy` to make the build
work.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17590: Rename S3 volume config keys to match AWS terms.
Tom Clegg [Tue, 4 May 2021 21:24:03 +0000 (17:24 -0400)]
17590: Rename S3 volume config keys to match AWS terms.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix test regexp.
Tom Clegg [Wed, 5 May 2021 17:32:29 +0000 (13:32 -0400)]
Fix test regexp.

amends commit 51d072fb24a69336c09fe4320102f5797db0d5e0

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoUpdate `arvados-server install` to use go 1.16.3
Ward Vandewege [Wed, 5 May 2021 16:42:32 +0000 (12:42 -0400)]
Update `arvados-server install` to use go 1.16.3

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years agoMerge branch '17587-workbench-federated-query'
Tom Clegg [Wed, 5 May 2021 13:40:29 +0000 (09:40 -0400)]
Merge branch '17587-workbench-federated-query'

fixes #17587

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17587: Comment SetUpTest calls.
Tom Clegg [Wed, 5 May 2021 13:39:48 +0000 (09:39 -0400)]
17587: Comment SetUpTest calls.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoFix "docker port" usage.
Tom Clegg [Wed, 5 May 2021 13:29:35 +0000 (09:29 -0400)]
Fix "docker port" usage.

"docker port foo tcp/9999" can return multiple addresses like
"0.0.0.0:12345\n:::12345\n".

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoOur python3 packages should be marked as 'Replaces:' for the old python
Ward Vandewege [Tue, 4 May 2021 21:40:30 +0000 (17:40 -0400)]
Our python3 packages should be marked as 'Replaces:' for the old python
packages, so that there are no "trying to overwrite" errors, e.g. when
upgrading python-crunchstat-summary from 2.0.4 to the latest 2.2.x
release.

refs #17498

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17587: Don't reject federated query with large limit param.
Tom Clegg [Tue, 4 May 2021 18:20:27 +0000 (14:20 -0400)]
17587: Don't reject federated query with large limit param.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17610-remote-token-scopes'
Tom Clegg [Tue, 4 May 2021 18:46:04 +0000 (14:46 -0400)]
Merge branch '17610-remote-token-scopes'

fixes #17610

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoDon't ask nginx to log to stderr when using setuid.
Tom Clegg [Tue, 4 May 2021 17:52:22 +0000 (13:52 -0400)]
Don't ask nginx to log to stderr when using setuid.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoUpdate tests for go 1.16.
Tom Clegg [Tue, 4 May 2021 17:52:08 +0000 (13:52 -0400)]
Update tests for go 1.16.

No issue #

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17598-keep-web-url-case' refs #17598
Peter Amstutz [Tue, 4 May 2021 15:41:29 +0000 (11:41 -0400)]
Merge branch '17598-keep-web-url-case' refs #17598

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years agoMerge branch '17284-redact-railsapi-host'
Tom Clegg [Tue, 4 May 2021 14:48:07 +0000 (10:48 -0400)]
Merge branch '17284-redact-railsapi-host'

refs #17284

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years ago17284: Fix missing "/" in error message.
Tom Clegg [Tue, 4 May 2021 05:15:20 +0000 (01:15 -0400)]
17284: Fix missing "/" in error message.

Was showing "//railsapi.internalarvados/..." instead of
"//railsapi.internal/arvados/...".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoBump the Go version to 1.16.3
Ward Vandewege [Mon, 3 May 2021 21:13:24 +0000 (17:13 -0400)]
Bump the Go version to 1.16.3

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

2 years ago17598: Also use lowercase for URL host comparison
Peter Amstutz [Mon, 3 May 2021 18:31:21 +0000 (14:31 -0400)]
17598: Also use lowercase for URL host comparison

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17610: Check scopes when using a remote token.
Tom Clegg [Mon, 3 May 2021 18:08:31 +0000 (14:08 -0400)]
17610: Check scopes when using a remote token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

2 years agoMerge branch '17598-keep-web-url' refs #17598
Peter Amstutz [Thu, 29 Apr 2021 21:35:13 +0000 (17:35 -0400)]
Merge branch '17598-keep-web-url' refs #17598

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17598: stripDefaultPort behavior, error message when !credentialsOK
Peter Amstutz [Thu, 29 Apr 2021 20:10:11 +0000 (16:10 -0400)]
17598: stripDefaultPort behavior, error message when !credentialsOK

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

2 years ago17598: Handle comparison URLs with :80 or :443
Peter Amstutz [Thu, 29 Apr 2021 19:07:44 +0000 (15:07 -0400)]
17598: Handle comparison URLs with :80 or :443

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoThe sks-keyservers network is dying. Update our documentation to use
Ward Vandewege [Thu, 29 Apr 2021 01:28:04 +0000 (21:28 -0400)]
The sks-keyservers network is dying. Update our documentation to use
pgp.mit.edu instead.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '16997-sort-config-for-diff'
Tom Clegg [Wed, 28 Apr 2021 15:31:06 +0000 (11:31 -0400)]
Merge branch '16997-sort-config-for-diff'

fixes #16997

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17284-redact-railsapi-host'
Tom Clegg [Tue, 27 Apr 2021 17:07:47 +0000 (13:07 -0400)]
Merge branch '17284-redact-railsapi-host'

fixes #17284

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoDo not pin arvados-login-sync on an ancient version of the arvados gem.
Ward Vandewege [Tue, 27 Apr 2021 16:08:01 +0000 (12:08 -0400)]
Do not pin arvados-login-sync on an ancient version of the arvados gem.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago16997: Use go-yaml fork to avoid key-sorting bug.
Tom Clegg [Tue, 27 Apr 2021 16:00:45 +0000 (12:00 -0400)]
16997: Use go-yaml fork to avoid key-sorting bug.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago16997: Test map key order in check/dump. Fix duplicate warnings.
Tom Clegg [Mon, 26 Apr 2021 19:11:58 +0000 (15:11 -0400)]
16997: Test map key order in check/dump. Fix duplicate warnings.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17507-listobjectsv2'
Tom Clegg [Mon, 26 Apr 2021 14:14:15 +0000 (10:14 -0400)]
Merge branch '17507-listobjectsv2'

fixes #17507

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17343-workbench1-optional'
Tom Clegg [Mon, 26 Apr 2021 13:47:01 +0000 (09:47 -0400)]
Merge branch '17343-workbench1-optional'

closes #17343

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17284: Redact RailsAPI host:port in error messages.
Tom Clegg [Mon, 26 Apr 2021 13:40:46 +0000 (09:40 -0400)]
17284: Redact RailsAPI host:port in error messages.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17521-dot-slash-glob' refs #17521
Peter Amstutz [Fri, 23 Apr 2021 21:02:41 +0000 (17:02 -0400)]
Merge branch '17521-dot-slash-glob' refs #17521

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17521: Use normpath to clean up the path segment.
Peter Amstutz [Fri, 23 Apr 2021 20:07:02 +0000 (16:07 -0400)]
17521: Use normpath to clean up the path segment.

Removes leading ./, trailing slashes, redundant '.' and blank path
segments, and other stuff that potentially confuses the path matching.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17521: Add tests, refs #10380 and refs #17521
Peter Amstutz [Fri, 23 Apr 2021 19:58:42 +0000 (15:58 -0400)]
17521: Add tests, refs #10380 and refs #17521

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17507: Fix calling Child() without lock.
Tom Clegg [Thu, 22 Apr 2021 20:14:38 +0000 (16:14 -0400)]
17507: Fix calling Child() without lock.

Encountered in services/keep-web tests.

START: handler_test.go:654: IntegrationSuite.TestDirectoryListingWithNoAnonymousToken
START: server_test.go:431: IntegrationSuite.SetUpTest

[...]

fatal error: concurrent map iteration and map write

goroutine 2192 [running]:
runtime.throw(0xe545ab, 0x26)
        /var/lib/arvados/go/src/runtime/panic.go:1112 +0x72 fp=0xc002544c78 sp=0xc002544c48 pc=0x4366b2
runtime.mapiternext(0xc002544d18)
        /var/lib/arvados/go/src/runtime/map.go:853 +0x552 fp=0xc002544cf8 sp=0xc002544c78 pc=0x411442
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc00202e480, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:336 +0x106 fp=0xc002544d98 sp=0xc002544cf8 pc=0x8c7de6
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc0003a66c0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:337 +0xe3 fp=0xc002544e38 sp=0xc002544d98 pc=0x8c7dc3
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).MemorySize(0xc0020c3680, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:337 +0xe3 fp=0xc002544ed8 sp=0xc002544e38 pc=0x8c7dc3
git.arvados.org/arvados.git/sdk/go/arvados.(*fileSystem).MemorySize(0xc001af0000, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:631 +0x33 fp=0xc002544ef8 sp=0xc002544ed8 pc=0x8ca1e3
git.arvados.org/arvados.git/services/keep-web.(*cache).collectionBytes(0xc00261da08, 0x3ff0000000000101)
        /home/tom/arvados/services/keep-web/cache.go:448 +0x21c fp=0xc002544f70 sp=0xc002544ef8 pc=0xc2903c
git.arvados.org/arvados.git/services/keep-web.(*cache).updateGauges(0xc00261da08)
        /home/tom/arvados/services/keep-web/cache.go:170 +0x2f fp=0xc002544f90 sp=0xc002544f70 pc=0xc269bf
git.arvados.org/arvados.git/services/keep-web.(*cache).setup.func1(0xc00261da08)
        /home/tom/arvados/services/keep-web/cache.go:164 +0x6d fp=0xc002544fd8 sp=0xc002544f90 pc=0xc65dad
runtime.goexit()
        /var/lib/arvados/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc002544fe0 sp=0xc002544fd8 pc=0x468dd1
created by git.arvados.org/arvados.git/services/keep-web.(*cache).setup
        /home/tom/arvados/services/keep-web/cache.go:162 +0x1db

goroutine 2144 [runnable]:
git.arvados.org/arvados.git/sdk/go/arvados.(*treenode).Child(0xc00202e480, 0xc002c4e117, 0x9, 0xc0005c2070, 0x10c9680, 0xc0021d0240, 0x0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_base.go:289 +0x171
git.arvados.org/arvados.git/sdk/go/arvados.(*lookupnode).Readdir(0xc00202e480, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_lookup.go:53 +0x27b
git.arvados.org/arvados.git/sdk/go/arvados.(*filehandle).Readdir(0xc00263c140, 0x0, 0xc002066360, 0xc00263c140, 0x10c4f40, 0xc00263c140, 0x0)
        /home/tom/arvados/sdk/go/arvados/fs_filehandle.go:81 +0x1d2
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xc00161a574, 0xc, 0x10c17c0, 0xc00227c280, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:772 +0x25e
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xe2f941, 0x5, 0x10c17c0, 0xc002b1a4c0, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:786 +0x564
golang.org/x/net/webdav.walkFS(0x10bbe00, 0xc002b09590, 0x10bd900, 0xc002af1e60, 0xffffffffffffffff, 0xc00216be76, 0x0, 0x10c17c0, 0xc002b1a0c0, 0xc0005c2628, ...)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/file.go:786 +0x564
golang.org/x/net/webdav.(*Handler).handlePropfind(0xc002b1a080, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900, 0xc0022bc718, 0x415ad3, 0xc002b1a080)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/webdav.go:566 +0x3d7
golang.org/x/net/webdav.(*Handler).ServeHTTP(0xc002b1a080, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/golang.org/x/net@v0.0.0-20200202094626-16171245cfb2/webdav/webdav.go:67 +0x556
git.arvados.org/arvados.git/services/keep-web.(*handler).serveSiteFS(0xc002221ae0, 0x7f1c582d9120, 0xc0029e5400, 0xc002b04900, 0xc002b22010, 0x1, 0x1, 0x101)
        /home/tom/arvados/services/keep-web/handler.go:593 +0x80b
git.arvados.org/arvados.git/services/keep-web.(*handler).ServeHTTP(0xc002221ae0, 0x10b6ec0, 0xc002b094a0, 0xc002b04900)
        /home/tom/arvados/services/keep-web/handler.go:330 +0x2a0b
git.arvados.org/arvados.git/sdk/go/httpserver.LogRequests.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /home/tom/arvados/sdk/go/httpserver/logger.go:56 +0x8d8
net/http.HandlerFunc.ServeHTTP(0xc002282460, 0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.AddRequestIDs.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /home/tom/arvados/sdk/go/httpserver/id_generator.go:57 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc002282480, 0x7f1c58319c58, 0xc002b09470, 0xc002b04800)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.HandlerWithContext.func1(0x7f1c58319c58, 0xc002b09470, 0xc002b04700)
        /home/tom/arvados/sdk/go/httpserver/logger.go:30 +0x107
net/http.HandlerFunc.ServeHTTP(0xc002901b30, 0x7f1c58319c58, 0xc002b09470, 0xc002b04700)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func1(0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/github.com/prometheus/client_golang@v1.2.1/prometheus/promhttp/instrument_server.go:68 +0x11c
net/http.HandlerFunc.ServeHTTP(0xc002901e00, 0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /var/lib/arvados/go/src/net/http/server.go:2012 +0x44
git.arvados.org/arvados.git/sdk/go/httpserver.(*metrics).ServeHTTP(0xc002113e00, 0x10b8bc0, 0xc002b1a000, 0xc002b04700)
        /home/tom/arvados/sdk/go/httpserver/metrics.go:70 +0x51
git.arvados.org/arvados.git/services/keep-web.(*IntegrationSuite).testDirectoryListing(0xc000010138, 0xc0007d41e0)
        /home/tom/arvados/services/keep-web/handler_test.go:864 +0x1a07
git.arvados.org/arvados.git/services/keep-web.(*IntegrationSuite).TestDirectoryListingWithNoAnonymousToken(0xc000010138, 0xc0007d41e0)
        /home/tom/arvados/services/keep-web/handler_test.go:656 +0x67
reflect.Value.call(0xe2bbc0, 0xc000010138, 0x4613, 0xe2deb6, 0x4, 0xc000282f08, 0x1, 0x1, 0x171d800, 0xc000282e48, ...)
        /var/lib/arvados/go/src/reflect/value.go:460 +0x8ab
reflect.Value.Call(0xe2bbc0, 0xc000010138, 0x4613, 0xc000282f08, 0x1, 0x1, 0xc0007d42d0, 0xc000128060, 0xc00287a720)
        /var/lib/arvados/go/src/reflect/value.go:321 +0xb4
gopkg.in/check%2ev1.(*suiteRunner).forkTest.func1(0xc0007d41e0)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:772 +0x628
gopkg.in/check%2ev1.(*suiteRunner).forkCall.func1(0xc000174c00, 0xc0007d41e0, 0xc00222b760)
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:666 +0x98
created by gopkg.in/check%2ev1.(*suiteRunner).forkCall
        /home/tom/arvados/tmp/GOPATH/pkg/mod/gopkg.in/check.v1@v1.0.0-20161208181325-20d25e280405/check.go:663 +0x1fb

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17507: Support ListObjectsV2 API.
Tom Clegg [Thu, 22 Apr 2021 19:12:17 +0000 (15:12 -0400)]
17507: Support ListObjectsV2 API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17566-max-request-size'
Tom Clegg [Wed, 21 Apr 2021 20:25:59 +0000 (16:25 -0400)]
Merge branch '17566-max-request-size'

fixes #17566

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17204-not-modified'
Tom Clegg [Wed, 21 Apr 2021 20:25:37 +0000 (16:25 -0400)]
Merge branch '17204-not-modified'

fixes #17204

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17346-keep-balance-empty-block'
Tom Clegg [Wed, 21 Apr 2021 19:50:27 +0000 (15:50 -0400)]
Merge branch '17346-keep-balance-empty-block'

fixes #17346

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17343: Add option to boot without workbench1 to speed up tests.
Tom Clegg [Wed, 21 Apr 2021 19:03:59 +0000 (15:03 -0400)]
17343: Add option to boot without workbench1 to speed up tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17570-remove-extra-example-agrantfile'
Javier Bértoli [Wed, 21 Apr 2021 15:10:24 +0000 (12:10 -0300)]
Merge branch '17570-remove-extra-example-agrantfile'

closes #17570
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofix(provision): remove unneeded vagantfile
Javier Bértoli [Wed, 21 Apr 2021 15:01:30 +0000 (12:01 -0300)]
fix(provision): remove unneeded vagantfile

refs #17570
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17204: Fix misleading log message on 304 Not Modified response.
Tom Clegg [Tue, 20 Apr 2021 23:43:57 +0000 (19:43 -0400)]
17204: Fix misleading log message on 304 Not Modified response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17528-install-warnings'
Tom Clegg [Tue, 20 Apr 2021 20:04:22 +0000 (16:04 -0400)]
Merge branch '17528-install-warnings'

fixes #17528

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17437-drop-content-length-header'
Tom Clegg [Tue, 20 Apr 2021 19:04:14 +0000 (15:04 -0400)]
Merge branch '17437-drop-content-length-header'

refs #17437

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17246-salt-install-improvements'
Javier Bértoli [Tue, 20 Apr 2021 18:35:48 +0000 (15:35 -0300)]
Merge branch '17246-salt-install-improvements'

closes #17246

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofeat(provision): pin arvados version to latest stable
Javier Bértoli [Mon, 19 Apr 2021 18:52:17 +0000 (15:52 -0300)]
feat(provision): pin arvados version to latest stable

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17437: Don't copy incoming Content-Length header when proxying.
Tom Clegg [Tue, 20 Apr 2021 16:17:30 +0000 (12:17 -0400)]
17437: Don't copy incoming Content-Length header when proxying.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17566: Don't bypass controller in SDK integration tests.
Tom Clegg [Mon, 19 Apr 2021 22:04:36 +0000 (18:04 -0400)]
17566: Don't bypass controller in SDK integration tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17566: Allow large request bodies in integration tests.
Tom Clegg [Mon, 19 Apr 2021 22:04:14 +0000 (18:04 -0400)]
17566: Allow large request bodies in integration tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17566: Use configured MaxRequestSize.
Tom Clegg [Mon, 19 Apr 2021 22:01:39 +0000 (18:01 -0400)]
17566: Use configured MaxRequestSize.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agofix(provision): default to using the master branch of arvados-formula
Javier Bértoli [Mon, 19 Apr 2021 17:53:10 +0000 (14:53 -0300)]
fix(provision): default to using the master branch of arvados-formula

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agodocs(provision): improve the documentation in the examples
Javier Bértoli [Mon, 19 Apr 2021 13:00:43 +0000 (10:00 -0300)]
docs(provision): improve the documentation in the examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agorefactor(provision): add snakeoil certs in single host multiple hostnames
Javier Bértoli [Fri, 16 Apr 2021 22:05:34 +0000 (19:05 -0300)]
refactor(provision): add snakeoil certs in single host multiple hostnames

We could use the example states in the formula, but added them here
for the sake of completeness

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agodocs(provision): improve the documentation in the examples
Javier Bértoli [Fri, 16 Apr 2021 22:02:27 +0000 (19:02 -0300)]
docs(provision): improve the documentation in the examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agorefactor(provision): add host entries in single host multiple hostnames
Javier Bértoli [Fri, 16 Apr 2021 22:01:12 +0000 (19:01 -0300)]
refactor(provision): add host entries in single host multiple hostnames

We could use the example states in the formula, but added them here
for the sake of completeness

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agochore(provision): enable both examples in vagrant
Javier Bértoli [Fri, 16 Apr 2021 22:00:07 +0000 (19:00 -0300)]
chore(provision): enable both examples in vagrant

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agodocs(provision): explain options in config examples
Javier Bértoli [Fri, 16 Apr 2021 19:09:13 +0000 (16:09 -0300)]
docs(provision): explain options in config examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofeat(provision): add some checks before run
Javier Bértoli [Fri, 16 Apr 2021 19:07:57 +0000 (16:07 -0300)]
feat(provision): add some checks before run

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agorefactor(provision): delete parameters moved to config files
Javier Bértoli [Fri, 16 Apr 2021 19:06:08 +0000 (16:06 -0300)]
refactor(provision): delete parameters moved to config files

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agochore(provision): ignore some files in git
Javier Bértoli [Fri, 16 Apr 2021 19:04:15 +0000 (16:04 -0300)]
chore(provision): ignore some files in git

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agorefactor(provision): use uniform database parameters across examples
Javier Bértoli [Fri, 16 Apr 2021 19:01:52 +0000 (16:01 -0300)]
refactor(provision): use uniform database parameters across examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agoMerge branch '17530-arvados-client-fastfail-take2'
Nico Cesar [Fri, 16 Apr 2021 21:00:42 +0000 (17:00 -0400)]
Merge branch '17530-arvados-client-fastfail-take2'

closes #17530

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agofix(provision): run hosts/snakeoil states before nginx so it does not fail on missing...
Javier Bértoli [Fri, 16 Apr 2021 18:58:34 +0000 (15:58 -0300)]
fix(provision): run hosts/snakeoil states before nginx so it does not fail on missing deps

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agochore(provision): set postgresql cidr in single host examples
Javier Bértoli [Fri, 16 Apr 2021 18:37:07 +0000 (15:37 -0300)]
chore(provision): set postgresql cidr in single host examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years agofix(provision): use uniform database parameters across examples
Javier Bértoli [Fri, 16 Apr 2021 18:36:18 +0000 (15:36 -0300)]
fix(provision): use uniform database parameters across examples

refs #17246
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17530: Added check if API host is missing
Nico Cesar [Thu, 15 Apr 2021 16:26:42 +0000 (12:26 -0400)]
17530: Added check if API host is missing

CallRaw() will have a retry logic so 3s * 20 = 60s to fail,
and it will be explicit why this is happening.

KeepClient.discoverServices() to make it fail faster

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years ago17346: Add test case.
Tom Clegg [Fri, 16 Apr 2021 14:48:01 +0000 (10:48 -0400)]
17346: Add test case.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17529-listinstances-rate-limit'
Tom Clegg [Fri, 16 Apr 2021 14:44:07 +0000 (10:44 -0400)]
Merge branch '17529-listinstances-rate-limit'

refs #17529

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17529: Heed MaxCloudOpsPerSecond in Instances() and returned insts.
Tom Clegg [Fri, 16 Apr 2021 13:59:01 +0000 (09:59 -0400)]
17529: Heed MaxCloudOpsPerSecond in Instances() and returned insts.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17529-ec2-rate-limit'
Tom Clegg [Fri, 16 Apr 2021 13:40:25 +0000 (09:40 -0400)]
Merge branch '17529-ec2-rate-limit'

fixes #17529

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17528: Set ready=0 if config is incomplete/missing during install.
Tom Clegg [Thu, 15 Apr 2021 20:17:03 +0000 (16:17 -0400)]
17528: Set ready=0 if config is incomplete/missing during install.

Reverts accidental change.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17171-pysdk-timeout-fix'
Lucas Di Pentima [Thu, 15 Apr 2021 20:03:45 +0000 (17:03 -0300)]
Merge branch '17171-pysdk-timeout-fix'
Closes #17171

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago17528: Fix assets:precompile @ build time, skip at install time.
Tom Clegg [Thu, 15 Apr 2021 20:02:11 +0000 (16:02 -0400)]
17528: Fix assets:precompile @ build time, skip at install time.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17171: Sets default timeout on arvados.api() to 10 seconds.
Lucas Di Pentima [Thu, 15 Apr 2021 19:53:41 +0000 (16:53 -0300)]
17171: Sets default timeout on arvados.api() to 10 seconds.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago17528: Remove bin/yarn from Rails projects.
Tom Clegg [Thu, 15 Apr 2021 19:45:40 +0000 (15:45 -0400)]
17528: Remove bin/yarn from Rails projects.

"rake assets:precompile" uses the existence of the bin/yarn stub to
decide whether yarn is needed.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17528: Don't load config when doing assets:precompile in postinst.
Tom Clegg [Thu, 15 Apr 2021 18:55:44 +0000 (14:55 -0400)]
17528: Don't load config when doing assets:precompile in postinst.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17529: Update comment.
Tom Clegg [Thu, 15 Apr 2021 18:21:55 +0000 (14:21 -0400)]
17529: Update comment.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17529: Catch rate limit errors from DescribeInstances, too.
Tom Clegg [Thu, 15 Apr 2021 18:20:04 +0000 (14:20 -0400)]
17529: Catch rate limit errors from DescribeInstances, too.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17529: Heed MaxCloudOpsPerSecond when calling SetTags.
Tom Clegg [Thu, 15 Apr 2021 18:03:13 +0000 (14:03 -0400)]
17529: Heed MaxCloudOpsPerSecond when calling SetTags.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17171: Merge branch 'master' into 17171-pysdk-timeout-fix
Lucas Di Pentima [Thu, 15 Apr 2021 17:10:01 +0000 (14:10 -0300)]
17171: Merge branch 'master' into 17171-pysdk-timeout-fix

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago17171: Fixes the bug by removing the default timeout value.
Lucas Di Pentima [Thu, 15 Apr 2021 17:08:07 +0000 (14:08 -0300)]
17171: Fixes the bug by removing the default timeout value.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years ago17171: Exposes bug with a test.
Lucas Di Pentima [Thu, 15 Apr 2021 17:07:36 +0000 (14:07 -0300)]
17171: Exposes bug with a test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@di-pentima.com.ar>

3 years agoMerge branch '17531-remove-forcelegacyapi14-config'
Tom Clegg [Thu, 15 Apr 2021 14:45:42 +0000 (10:45 -0400)]
Merge branch '17531-remove-forcelegacyapi14-config'

closes #17531

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17462-secondaryfiles-regression' refs #17462
Peter Amstutz [Thu, 15 Apr 2021 14:23:00 +0000 (10:23 -0400)]
Merge branch '17462-secondaryfiles-regression' refs #17462

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17462: Add integration test to confirm fix
Peter Amstutz [Thu, 15 Apr 2021 14:18:07 +0000 (10:18 -0400)]
17462: Add integration test to confirm fix

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17346: Don't try to rebalance the zero-length data block.
Tom Clegg [Thu, 15 Apr 2021 13:46:37 +0000 (09:46 -0400)]
17346: Don't try to rebalance the zero-length data block.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>