arvados.git
21 months agoMerge branch '19961-spot-interruption'
Tom Clegg [Tue, 28 Feb 2023 19:19:57 +0000 (14:19 -0500)]
Merge branch '19961-spot-interruption'

closes #19961

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Update preemptionNotice text. 19961-spot-interruption
Tom Clegg [Mon, 27 Feb 2023 19:14:16 +0000 (14:14 -0500)]
19961: Update preemptionNotice text.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Clarify preemptionNotice significance.
Tom Clegg [Mon, 27 Feb 2023 19:10:56 +0000 (14:10 -0500)]
19961: Clarify preemptionNotice significance.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19963-keep-web-auth-flow' into main
Tom Clegg [Thu, 23 Feb 2023 22:03:06 +0000 (17:03 -0500)]
Merge branch '19963-keep-web-auth-flow' into main

fixes #19963

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Save separate preemptionNotice key in runtime_status.
Tom Clegg [Thu, 23 Feb 2023 21:59:03 +0000 (16:59 -0500)]
19961: Save separate preemptionNotice key in runtime_status.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19963: Test workbench2 login flow starting with no token. 19963-keep-web-auth-flow
Tom Clegg [Thu, 23 Feb 2023 16:26:36 +0000 (11:26 -0500)]
19963: Test workbench2 login flow starting with no token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19963: Enable login flow without anonymous token configured.
Tom Clegg [Thu, 23 Feb 2023 15:45:14 +0000 (10:45 -0500)]
19963: Enable login flow without anonymous token configured.

Response is now 401 (was 404) when the requested collection is not
even loaded because the token is either invalid or passed in an
unsupported way (e.g., in an Authorization header with a
public-data-only URL path).

This revealed some testing issues, e.g., Test404 was attempting to
check that authenticated requests for nonexistent
files/dirs/collections returned 404, but it was in fact exercising a
code path that returned 404 because the token was not accepted.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19963: Dedup interface type.
Tom Clegg [Wed, 22 Feb 2023 15:56:52 +0000 (10:56 -0500)]
19963: Dedup interface type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19963: Fix crash on no-cache + invalid token.
Tom Clegg [Wed, 22 Feb 2023 15:52:13 +0000 (10:52 -0500)]
19963: Fix crash on no-cache + invalid token.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19973-dispatch-throttle' into main
Tom Clegg [Tue, 21 Feb 2023 22:04:30 +0000 (17:04 -0500)]
Merge branch '19973-dispatch-throttle' into main

fixes #19973
fixes #19984

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Mention interrupt handling on admin doc page.
Tom Clegg [Tue, 21 Feb 2023 21:58:02 +0000 (16:58 -0500)]
19961: Mention interrupt handling on admin doc page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19980-arv-prefix-properties'
Brett Smith [Tue, 21 Feb 2023 21:27:28 +0000 (16:27 -0500)]
Merge branch '19980-arv-prefix-properties'

Closes #19980.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19980: System properties only require arv: prefix
Brett Smith [Tue, 21 Feb 2023 21:24:17 +0000 (16:24 -0500)]
19980: System properties only require arv: prefix

The decision to require a following alphabetic character was
well-intentioned but misplaced: it doesn't stop these properties from
existing, just changes the layer they would be defined at.

It might make sense to add some name rules to properties, but that would
be better done across the board as a separate story, not as a specific
corner case of system properties.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19961: Reuse metadata token within expected TTL, adjust retry logic.
Tom Clegg [Tue, 21 Feb 2023 21:14:04 +0000 (16:14 -0500)]
19961: Reuse metadata token within expected TTL, adjust retry logic.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Merge branch 'main'
Tom Clegg [Tue, 21 Feb 2023 20:45:12 +0000 (15:45 -0500)]
19961: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Split unit test cases into multiple funcs. 19973-dispatch-throttle
Tom Clegg [Tue, 21 Feb 2023 16:57:27 +0000 (11:57 -0500)]
19973: Split unit test cases into multiple funcs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Merge branch 'main'
Tom Clegg [Tue, 21 Feb 2023 15:24:52 +0000 (10:24 -0500)]
19973: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: un-embed requestLimiter.
Tom Clegg [Tue, 21 Feb 2023 15:22:24 +0000 (10:22 -0500)]
19973: un-embed requestLimiter.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '20138-fix-sync-test-race'
Brett Smith [Tue, 21 Feb 2023 14:40:33 +0000 (09:40 -0500)]
Merge branch '20138-fix-sync-test-race'

Closes #20138.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago20138: Avoid timing failures in TestSyncNonCanonicalManifest 20138-fix-sync-test-race
Brett Smith [Mon, 20 Feb 2023 21:13:03 +0000 (16:13 -0500)]
20138: Avoid timing failures in TestSyncNonCanonicalManifest

See comments for rationale.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago20137: Fix incorrect keyword in Workbench1 config reload loop
Brett Smith [Mon, 20 Feb 2023 20:30:57 +0000 (15:30 -0500)]
20137: Fix incorrect keyword in Workbench1 config reload loop

This has the same rationale as e03ebe77c. I don't know if this code
is being exercised anywhere, but I don't want to find out the hard way
that the answer is yes.
Refs #20137.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19980: Expand documentation of reserved properties
Brett Smith [Mon, 20 Feb 2023 16:55:32 +0000 (11:55 -0500)]
19980: Expand documentation of reserved properties

* Note they will be prefixed with `arv:` from now on.
* Explain why and how these are set.
* Explain these can be set even with a strict vocabulary.
* Add the arvados-cwl-runner Git properties.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19980: Recognize the arv: prefix for all system properties
Brett Smith [Mon, 20 Feb 2023 15:53:06 +0000 (10:53 -0500)]
19980: Recognize the arv: prefix for all system properties

arvados-cwl-runner currently uses this prefix to record Git information,
and we want to use it for all future system properties.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago20137: Fix incorrect keyword in RailsAPI config reload loop
Brett Smith [Mon, 20 Feb 2023 19:46:41 +0000 (14:46 -0500)]
20137: Fix incorrect keyword in RailsAPI config reload loop

The reason rails_restart_test.go has been racey is because the code
under test is actually buggy. If the configuration reload loop ever
encounters a condition where it wants to restart (probably
`hash == hash_lastload`), it tries to call a nonexistent method
`continue`, the thread crashes, the configuration never gets reloaded,
and the test fails.

Use the intended Ruby keyword `next` instead.
Closes #20137.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19961: Fix races in tests.
Tom Clegg [Thu, 16 Feb 2023 22:26:14 +0000 (17:26 -0500)]
19961: Fix races in tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19961: Fix tests sabotaging other tests.
Tom Clegg [Thu, 16 Feb 2023 21:56:14 +0000 (16:56 -0500)]
19961: Fix tests sabotaging other tests.

Goroutines were calling "s.executor.exit <- exitCode" after their own
test was finished, at which point s.executor belonged to a subsequent
test, which would then exit too early or with the wrong exit code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '20083-sync-readonly'
Tom Clegg [Thu, 16 Feb 2023 20:56:06 +0000 (15:56 -0500)]
Merge branch '20083-sync-readonly'

fixes #20083

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '20121-update-prom-client'
Tom Clegg [Thu, 16 Feb 2023 16:37:39 +0000 (11:37 -0500)]
Merge branch '20121-update-prom-client'

fixes #20121

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago20121: Update prometheus client.
Tom Clegg [Thu, 16 Feb 2023 16:37:17 +0000 (11:37 -0500)]
20121: Update prometheus client.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Fix tests.
Tom Clegg [Thu, 16 Feb 2023 05:24:46 +0000 (00:24 -0500)]
19973: Fix tests.

Stub pool AtQuota() was miscounting instances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoAdds 'awscli' package to the salt installer state for encrypted ssl privkeys.
Lucas Di Pentima [Wed, 15 Feb 2023 20:58:24 +0000 (17:58 -0300)]
Adds 'awscli' package to the salt installer state for encrypted ssl privkeys.

refs #20035

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago19973: Test throttling after receiving 503 error.
Tom Clegg [Wed, 15 Feb 2023 17:09:42 +0000 (12:09 -0500)]
19973: Test throttling after receiving 503 error.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoFixes statement_timeout unit handling on int->bigint migration. Refs #20074
Lucas Di Pentima [Wed, 15 Feb 2023 14:42:23 +0000 (11:42 -0300)]
Fixes statement_timeout unit handling on int->bigint migration. Refs #20074

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago19961: Detect and log EC2 spot interruption notices.
Tom Clegg [Tue, 14 Feb 2023 20:53:54 +0000 (15:53 -0500)]
19961: Detect and log EC2 spot interruption notices.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '20074-bigint-migration-fix'. Refs #20074
Lucas Di Pentima [Tue, 14 Feb 2023 18:57:47 +0000 (15:57 -0300)]
Merge branch '20074-bigint-migration-fix'. Refs #20074

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20074: Disables statement timeout on rails migration. 20074-bigint-migration-fix
Lucas Di Pentima [Tue, 14 Feb 2023 18:34:16 +0000 (15:34 -0300)]
20074: Disables statement timeout on rails migration.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months agoMerge branch '20035-salt-installer-privkey'. Closes #20035
Lucas Di Pentima [Tue, 14 Feb 2023 15:43:52 +0000 (12:43 -0300)]
Merge branch '20035-salt-installer-privkey'. Closes #20035

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Fixes systemd unit Exec* value quoting. 20035-salt-installer-privkey
Lucas Di Pentima [Tue, 14 Feb 2023 15:37:03 +0000 (12:37 -0300)]
20035: Fixes systemd unit Exec* value quoting.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Adds a warning on systemd's unit file re: runtime dirs.
Lucas Di Pentima [Tue, 14 Feb 2023 15:22:22 +0000 (12:22 -0300)]
20035: Adds a warning on systemd's unit file re: runtime dirs.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Add missing jinja vars quoting on systemd's unit file template.
Lucas Di Pentima [Tue, 14 Feb 2023 15:19:42 +0000 (12:19 -0300)]
20035: Add missing jinja vars quoting on systemd's unit file template.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months agoMerge branch '20096-cwltest' refs #20096
Peter Amstutz [Mon, 13 Feb 2023 21:02:43 +0000 (16:02 -0500)]
Merge branch '20096-cwltest' refs #20096

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20096: Report cwl runner version too
Peter Amstutz [Mon, 13 Feb 2023 20:53:40 +0000 (15:53 -0500)]
20096: Report cwl runner version too

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20035: Manages named pipe file from systemd instead of salt.
Lucas Di Pentima [Mon, 13 Feb 2023 20:37:05 +0000 (17:37 -0300)]
20035: Manages named pipe file from systemd instead of salt.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20096: Tweak test_with_arvbox.sh 20096-cwltest
Peter Amstutz [Mon, 13 Feb 2023 19:38:38 +0000 (14:38 -0500)]
20096: Tweak test_with_arvbox.sh

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20035: Enhances code redability, quotes jinja vars as yaml values.
Lucas Di Pentima [Fri, 10 Feb 2023 19:23:13 +0000 (16:23 -0300)]
20035: Enhances code redability, quotes jinja vars as yaml values.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months agoMerge branch '19385-cwl-fast-pack' refs #19385
Peter Amstutz [Fri, 10 Feb 2023 19:26:27 +0000 (14:26 -0500)]
Merge branch '19385-cwl-fast-pack' refs #19385

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20035: Improves file/dir permission setting.
Lucas Di Pentima [Fri, 10 Feb 2023 19:10:27 +0000 (16:10 -0300)]
20035: Improves file/dir permission setting.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Improves the connector script in various ways.
Lucas Di Pentima [Fri, 10 Feb 2023 18:16:31 +0000 (15:16 -0300)]
20035: Improves the connector script in various ways.

* Passes /dev/null to AWS CLI as credential file instead of an non-existent
  path.
* Single-quotes every Jinja variable to avoid side shell related effects.
* Passes AWS region name as a pillar.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Updates nginx pillars to read the certificate privkey password.
Lucas Di Pentima [Tue, 7 Feb 2023 19:15:40 +0000 (16:15 -0300)]
20035: Updates nginx pillars to read the certificate privkey password.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Adds pillars & states to retrieve the password from AWS secrets manager.
Lucas Di Pentima [Tue, 7 Feb 2023 19:13:18 +0000 (16:13 -0300)]
20035: Adds pillars & states to retrieve the password from AWS secrets manager.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Adds configuration to enable the use of encryped SSL private keys.
Lucas Di Pentima [Tue, 7 Feb 2023 19:11:17 +0000 (16:11 -0300)]
20035: Adds configuration to enable the use of encryped SSL private keys.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Manages AWS secret and gives read access to service nodes.
Lucas Di Pentima [Mon, 6 Feb 2023 18:00:01 +0000 (15:00 -0300)]
20035: Manages AWS secret and gives read access to service nodes.

The secret's value will have to be set manually by the operator, so that
no traces of it is kept on disk.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago20035: Ensures proper permissions on places where certificate's keys are saved.
Lucas Di Pentima [Thu, 2 Feb 2023 18:16:19 +0000 (15:16 -0300)]
20035: Ensures proper permissions on places where certificate's keys are saved.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago19973: Update tests.
Tom Clegg [Fri, 10 Feb 2023 17:17:34 +0000 (12:17 -0500)]
19973: Update tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Add metrics for automatic container concurrency limit.
Tom Clegg [Fri, 10 Feb 2023 15:54:49 +0000 (10:54 -0500)]
19973: Add metrics for automatic container concurrency limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoRename Workbench.BannerURL to Workbench.BannerUUID
Peter Amstutz [Fri, 10 Feb 2023 15:27:03 +0000 (10:27 -0500)]
Rename Workbench.BannerURL to Workbench.BannerUUID

closes #20078

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago20083: Don't copy remote blocks when computing local PDH. 20083-sync-readonly
Tom Clegg [Fri, 10 Feb 2023 15:11:09 +0000 (10:11 -0500)]
20083: Don't copy remote blocks when computing local PDH.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago20083: Track both loadedPDH and savedPDH for checking local changes.
Tom Clegg [Fri, 10 Feb 2023 07:34:34 +0000 (02:34 -0500)]
20083: Track both loadedPDH and savedPDH for checking local changes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago20083: Test false positive in "save local changes" logic.
Tom Clegg [Thu, 9 Feb 2023 16:57:07 +0000 (11:57 -0500)]
20083: Test false positive in "save local changes" logic.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Limit container concurrency when API is returning 503.
Tom Clegg [Thu, 9 Feb 2023 16:11:22 +0000 (11:11 -0500)]
19973: Limit container concurrency when API is returning 503.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19973: Limit arvados.Client request concurrency.
Tom Clegg [Wed, 8 Feb 2023 22:04:25 +0000 (17:04 -0500)]
19973: Limit arvados.Client request concurrency.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19385: Add comments and minor reorganization 19385-cwl-fast-pack
Peter Amstutz [Tue, 7 Feb 2023 21:28:27 +0000 (16:28 -0500)]
19385: Add comments and minor reorganization

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago19385: Fix tests
Peter Amstutz [Tue, 7 Feb 2023 19:32:58 +0000 (14:32 -0500)]
19385: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months agoMerge branch '19907-oidc-cache-error'
Tom Clegg [Tue, 7 Feb 2023 19:08:40 +0000 (14:08 -0500)]
Merge branch '19907-oidc-cache-error'

refs #19907

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19385: Fix test_copy_deps
Peter Amstutz [Tue, 7 Feb 2023 16:15:37 +0000 (11:15 -0500)]
19385: Fix test_copy_deps

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months agoAdds Arvados legal logo to the doc site. Refs #20079
Lucas Di Pentima [Tue, 7 Feb 2023 16:07:57 +0000 (13:07 -0300)]
Adds Arvados legal logo to the doc site. Refs #20079

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

21 months ago19385: Replace upload_workflow code
Peter Amstutz [Mon, 6 Feb 2023 20:26:38 +0000 (15:26 -0500)]
19385: Replace upload_workflow code

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago19385: Clean up debug prints
Peter Amstutz [Mon, 6 Feb 2023 19:50:11 +0000 (14:50 -0500)]
19385: Clean up debug prints

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months ago19385: Fix tests because wrapper collection now includes original files
Peter Amstutz [Fri, 3 Feb 2023 21:38:50 +0000 (16:38 -0500)]
19385: Fix tests because wrapper collection now includes original files

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months agoMerge branch '20032-tunnel-test'
Tom Clegg [Fri, 3 Feb 2023 19:40:33 +0000 (14:40 -0500)]
Merge branch '20032-tunnel-test'

fixes #20032

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19907: Update go-oidc to v3. 19907-oidc-cache-error
Tom Clegg [Mon, 16 Jan 2023 18:28:12 +0000 (13:28 -0500)]
19907: Update go-oidc to v3.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19907: Don't cache network/5xx errors when checking UserInfo.
Tom Clegg [Mon, 16 Jan 2023 18:15:06 +0000 (13:15 -0500)]
19907: Don't cache network/5xx errors when checking UserInfo.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19907: Log when caching negative result for OIDC token check.
Tom Clegg [Sun, 15 Jan 2023 22:02:42 +0000 (17:02 -0500)]
19907: Log when caching negative result for OIDC token check.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch 'main' into 19385-cwl-fast-pack
Peter Amstutz [Fri, 3 Feb 2023 19:11:09 +0000 (14:11 -0500)]
Merge branch 'main' into 19385-cwl-fast-pack

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

21 months agoMerge branch '19954-permission-dedup-doc'
Tom Clegg [Fri, 3 Feb 2023 14:50:41 +0000 (09:50 -0500)]
Merge branch '19954-permission-dedup-doc'

closes #19954

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19954: Update wording to admit multiple overlapping permission links. 19954-permission-dedup-doc
Tom Clegg [Thu, 2 Feb 2023 21:58:10 +0000 (16:58 -0500)]
19954: Update wording to admit multiple overlapping permission links.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19624-priority-doc'
Tom Clegg [Thu, 2 Feb 2023 21:53:40 +0000 (16:53 -0500)]
Merge branch '19624-priority-doc'

fixes #19624

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago20032: Fix unnecessary race in test. 20032-tunnel-test
Tom Clegg [Thu, 2 Feb 2023 21:28:25 +0000 (16:28 -0500)]
20032: Fix unnecessary race in test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19926-pysdk-install-docs-fix'
Brett Smith [Thu, 2 Feb 2023 21:02:25 +0000 (16:02 -0500)]
Merge branch '19926-pysdk-install-docs-fix'

Closes #19926.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19926: Remove "option 1/2" remnants from the Python SDK install docs
Brett Smith [Thu, 2 Feb 2023 21:01:31 +0000 (16:01 -0500)]
19926: Remove "option 1/2" remnants from the Python SDK install docs

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months agoMerge branch '18075-max-compute-vms'
Tom Clegg [Thu, 2 Feb 2023 20:20:12 +0000 (15:20 -0500)]
Merge branch '18075-max-compute-vms'

fixes #18075

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19954: Move special behavior comments into the method sections.
Tom Clegg [Thu, 2 Feb 2023 20:17:40 +0000 (15:17 -0500)]
19954: Move special behavior comments into the method sections.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19624: Layout container request lifecycle in two columns 19624-priority-doc
Brett Smith [Thu, 2 Feb 2023 16:31:36 +0000 (11:31 -0500)]
19624: Layout container request lifecycle in two columns

Provide a more readable layout. Normal full lifecycle on the left, early
end states on the right.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19926: Modernize pip install instructions
Brett Smith [Thu, 2 Feb 2023 15:39:37 +0000 (10:39 -0500)]
19926: Modernize pip install instructions

Running `python3 -m pip` instead of just plain `pip` helps ensure that
you install packages for the version of Python that you intend.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19926: Use noteboxes consistently in PySDK install instructions
Brett Smith [Thu, 2 Feb 2023 15:38:20 +0000 (10:38 -0500)]
19926: Use noteboxes consistently in PySDK install instructions

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19926: Flatten structure of PySDK install instructions
Brett Smith [Thu, 2 Feb 2023 15:33:32 +0000 (10:33 -0500)]
19926: Flatten structure of PySDK install instructions

The way there were "two tracks" of install and test instructions on the
page was confusing to readers. (Especially since we did not number those
options consistently.) Incorporate the test instructions into the
install instructions, so the reader only has to follow and care about
one specific method, and then they're done.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago18075: Note obsolete/deprecated config. 18075-max-compute-vms
Tom Clegg [Wed, 1 Feb 2023 18:51:35 +0000 (13:51 -0500)]
18075: Note obsolete/deprecated config.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago18075: Update AtQuota comment, check quota in Create, log at limit.
Tom Clegg [Wed, 1 Feb 2023 18:46:40 +0000 (13:46 -0500)]
18075: Update AtQuota comment, check quota in Create, log at limit.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19624: Layout graph L->R, say "update" instead of "assign".
Tom Clegg [Wed, 1 Feb 2023 19:14:14 +0000 (14:14 -0500)]
19624: Layout graph L->R, say "update" instead of "assign".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19954: Update API docs to reflect permission link deduplication.
Tom Clegg [Wed, 1 Feb 2023 18:34:51 +0000 (13:34 -0500)]
19954: Update API docs to reflect permission link deduplication.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19897-fuse-enotsup'
Brett Smith [Wed, 1 Feb 2023 15:49:41 +0000 (10:49 -0500)]
Merge branch '19897-fuse-enotsup'

Closes #19897.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19792: Use binary mode in cookbook download+upload examples
Brett Smith [Wed, 1 Feb 2023 14:10:33 +0000 (09:10 -0500)]
19792: Use binary mode in cookbook download+upload examples

This incorporates a suggestion from review to use a binary mode in a way
that makes the recipes more robust. It's very likely that readers will
want to work with binary files at least as much as text files, and these
recipes still work for text files too for verbatim copies.

Refs #19792.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19792: Fix typos throughout
Brett Smith [Wed, 1 Feb 2023 14:07:08 +0000 (09:07 -0500)]
19792: Fix typos throughout

Refs #19792.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19624: Redo diagram to show CR and container states.
Tom Clegg [Tue, 31 Jan 2023 21:26:06 +0000 (16:26 -0500)]
19624: Redo diagram to show CR and container states.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months ago19624: Redo diagram to show container states.
Tom Clegg [Tue, 31 Jan 2023 15:57:07 +0000 (10:57 -0500)]
19624: Redo diagram to show container states.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

21 months agoMerge branch '19792-pysdk-cookbook'
Brett Smith [Mon, 30 Jan 2023 23:40:43 +0000 (18:40 -0500)]
Merge branch '19792-pysdk-cookbook'

Closes #19792.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19792: Add 'wb' as an example Collection.open mode argument
Brett Smith [Mon, 30 Jan 2023 23:38:02 +0000 (18:38 -0500)]
19792: Add 'wb' as an example Collection.open mode argument

Give readers a hint that we fully support the mode argument, not just
the basic r/w/a options. Ideally they can look up the reference for full
details.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19897: Add tests for writing immutable mount objects
Brett Smith [Mon, 30 Jan 2023 23:29:47 +0000 (18:29 -0500)]
19897: Add tests for writing immutable mount objects

I originally wrote these expecting to get them to return ENOTSUP as
well. However, that doesn't happen now because of how the Operations
class calls Directory.writable() and turns False into EPERM. I'm
committing the tests since they're useful as-is, with a note that I'd
like to convert them to ENOTSUP in the future.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19897: Return ENOTSUP when the user tries to make a file in a project
Brett Smith [Mon, 30 Jan 2023 23:24:12 +0000 (18:24 -0500)]
19897: Return ENOTSUP when the user tries to make a file in a project

Before this commit, the NotImplementedError is propagating up to
catch_exceptions, which turns it into EIO. "I/O error" sounds like a
problem with the underlying systems and stresses out users. "Operation
not supported" should better communicate to the user that they're doing
something unexpected and should double-check their work.

Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>

21 months ago19624: Container request state transition / lifecycle diagram.
Tom Clegg [Mon, 30 Jan 2023 20:43:08 +0000 (15:43 -0500)]
19624: Container request state transition / lifecycle diagram.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>