Tom Clegg [Thu, 23 Mar 2023 21:40:30 +0000 (17:40 -0400)]
20240: Update test to reveal bug.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Tue, 4 Apr 2023 13:38:50 +0000 (10:38 -0300)]
Merge branch '16379-installer-prometheus-grafana'. Closes #16379
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Peter Amstutz [Tue, 4 Apr 2023 13:26:09 +0000 (09:26 -0400)]
Merge branch '20295-fix-collection-tree-caching-bug' refs #20295
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 31 Mar 2023 14:12:02 +0000 (10:12 -0400)]
Merge branch '20264-trusted-clients-portnumber'
fixes #20264
fixes #20140
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 31 Mar 2023 14:10:26 +0000 (10:10 -0400)]
Merge branch '20235-probe-after-upgrade'
fixes #20235
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Konrad Rudolph [Thu, 30 Mar 2023 17:25:25 +0000 (19:25 +0200)]
Add unit test
Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>
Konrad Rudolph [Thu, 30 Mar 2023 16:33:15 +0000 (18:33 +0200)]
Fix collection tree caching bug
Before this fix, accessing two files inside the same collection in
succession via `collection$get` will fail on the second file because the
private tree inside the `collection` is only populated with the relative
path of the first file.
Arvados-DCO-1.1-Signed-off-by: Konrad Rudolph <konrad.rudolph@gmail.com>
Peter Amstutz [Thu, 30 Mar 2023 17:18:38 +0000 (13:18 -0400)]
16379: Tweak doc on starring the dashboards on grafana
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 30 Mar 2023 14:56:44 +0000 (10:56 -0400)]
16379: Reorganize local.params examples
Make sure the things that always need to be changed are at the top.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 30 Mar 2023 14:30:41 +0000 (10:30 -0400)]
16379: Small doc updates about prometheus and grafana
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 30 Mar 2023 13:27:35 +0000 (09:27 -0400)]
16379: Updates for grafana install
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 28 Mar 2023 22:07:43 +0000 (18:07 -0400)]
16379: Add prometheus and grafana to workbench aliases
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Tue, 28 Mar 2023 21:05:26 +0000 (17:05 -0400)]
20264: Reject redirect target with userinfo.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Tue, 28 Mar 2023 20:38:25 +0000 (16:38 -0400)]
16379: doc updates
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Lucas Di Pentima [Tue, 28 Mar 2023 19:40:01 +0000 (16:40 -0300)]
Merge branch '20270-salt-installer-less-instances'. Closes #20270
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 28 Mar 2023 18:22:16 +0000 (15:22 -0300)]
16379: Fixes missing 'monitoring' role on the workbench node.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 28 Mar 2023 18:08:15 +0000 (15:08 -0300)]
16379: Removes prometheus config for keep1 & keepproxy nodes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 21 Mar 2023 22:07:18 +0000 (19:07 -0300)]
16379: Sets grafana's admin user password through the CLI.
Although it's being correctly set on the config file, the admin credentials
are not bein honored at first server start, and so the default password
is set in the database.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 21 Mar 2023 20:30:58 +0000 (17:30 -0300)]
16379: Makes sure the nginx formula is included in the keepweb's top file.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Fri, 17 Mar 2023 20:39:11 +0000 (17:39 -0300)]
16379: Adds nginx dependency to the htpasswd setup.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 9 Mar 2023 20:46:53 +0000 (17:46 -0300)]
16379: Adds grafana with default dashboards, moves prometheus hostname.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Fri, 3 Mar 2023 19:24:02 +0000 (16:24 -0300)]
16379: Adds basic auth to the prometheus UI.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 28 Feb 2023 18:54:55 +0000 (15:54 -0300)]
16379: Moves prometheus to its own subdomain instead of 'mon.prefix.domain.tld'
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Mon, 27 Feb 2023 16:29:06 +0000 (13:29 -0300)]
16379: Adds prometheus' node_exporter to all nodes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 21 Feb 2023 21:21:58 +0000 (18:21 -0300)]
16379: Adds prometheus service for monitoring arvados & postgresql.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 28 Mar 2023 17:19:57 +0000 (14:19 -0300)]
20270: Removes keep1 & keepproxy nodes. Uses SSH jumphost to deploy nodes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 28 Mar 2023 17:17:30 +0000 (14:17 -0300)]
20270: Refactors the VPC code so that private nodes can access the Internet.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Mon, 27 Mar 2023 18:31:40 +0000 (14:31 -0400)]
Merge branch '20242-dup-trash-lists'
fixes #20242
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 27 Mar 2023 18:29:38 +0000 (14:29 -0400)]
20140: Accept wildcards in TrustedClients.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Mon, 27 Mar 2023 17:43:31 +0000 (14:43 -0300)]
20270: Copies .gitignore file to avoid adding binaries to the repository.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Mon, 27 Mar 2023 17:15:02 +0000 (14:15 -0300)]
20270: Splits hosts into public & private, saving on public IPs requirements.
Also, asks for less number of instances: there's no need to multiple keepstore
nodes, and keep-web can be run on the same node as workbench.
This makes the basic default cluster to go from 6 to 4 nodes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Mon, 27 Mar 2023 15:38:11 +0000 (11:38 -0400)]
20264: Ignore superfluous :443 and :80 in trusted origin check.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Fri, 24 Mar 2023 19:46:57 +0000 (15:46 -0400)]
Merge branch '20259-banner-tooltip-doc' refs #20259
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 24 Mar 2023 15:18:03 +0000 (11:18 -0400)]
20235: Install crunch-run before run-probe even if boot-probe fails.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 24 Mar 2023 05:29:34 +0000 (01:29 -0400)]
Merge branch '20187-cache-discovery-doc'
closes #20187
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 23 Mar 2023 18:36:55 +0000 (14:36 -0400)]
20242: Trash only one when identical replicas are eligible to trash.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 23 Mar 2023 15:23:16 +0000 (11:23 -0400)]
20187: Remove references to discovery doc cache.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Wed, 22 Mar 2023 20:57:49 +0000 (16:57 -0400)]
20259: Add documentation for banner and tooltip features
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Tue, 21 Mar 2023 15:14:27 +0000 (11:14 -0400)]
Merge branch '19967-crunch-run-cost-updates'
Closes #19967.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 20 Mar 2023 19:52:56 +0000 (15:52 -0400)]
19967: Update container cost when crunch-run receives SIGUSR2
The Crunch dispatcher sends this signal periodically, so this provides
regular cost updates for running containers.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Thu, 16 Mar 2023 16:25:08 +0000 (12:25 -0400)]
20187: Don't use rails cache for discovery doc.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 16 Mar 2023 16:23:11 +0000 (12:23 -0400)]
20187: Expire cache and start returning errors after 24h.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 16 Mar 2023 15:05:16 +0000 (11:05 -0400)]
20187: Use validate func instead of making cache discoverydoc-aware.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 16 Mar 2023 15:05:04 +0000 (11:05 -0400)]
20187: Tidy up test case.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Thu, 16 Mar 2023 14:54:58 +0000 (11:54 -0300)]
Merge branch '19937-build-packages-fix'. Closes #19937
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Wed, 15 Mar 2023 18:38:53 +0000 (14:38 -0400)]
20187: Fix test race.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 15 Mar 2023 18:28:16 +0000 (14:28 -0400)]
20187: Check that the discovery doc is really a discovery doc.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 15 Mar 2023 14:08:08 +0000 (10:08 -0400)]
Merge branch '20183-update-priority-thread'
closes #20183
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 15 Mar 2023 03:35:10 +0000 (23:35 -0400)]
20187: Preserve CORS and other misc headers.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 9 Mar 2023 20:39:00 +0000 (15:39 -0500)]
20187: Test discovery doc cache.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 9 Mar 2023 14:27:30 +0000 (09:27 -0500)]
20187: 1 minute timeout for getting discovery doc from rails.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 8 Mar 2023 18:35:35 +0000 (13:35 -0500)]
20187: Update test.
Discovery doc can no longer be counted on to proxy through to rails.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 8 Mar 2023 17:51:00 +0000 (12:51 -0500)]
20187: Validate and cache discovery doc in controller.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Tue, 14 Mar 2023 17:45:02 +0000 (14:45 -0300)]
19937: Only output development versions.
Also, fixed a previously existing issue where a release branch commit with
its nearest tag being older than the merge base commit was getting versioned
as the most newest development version of any given time.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Fri, 27 Jan 2023 20:22:01 +0000 (17:22 -0300)]
19937: Avoids processing empty input (e.g: when statically linked).
This prevents red herring error messages like:
...
== Packages dependencies for ./usr/share/python3/dist/python3-arvados-cwl-runner/lib/python3.7/site-packages/schema_salad/ref_resolver.cpython-37m-x86_64-linux-gnu.so ==
dpkg-query: error: --search needs at least one file name pattern argument
Use --help for help about querying packages.
...
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Mon, 13 Mar 2023 14:57:40 +0000 (10:57 -0400)]
Merge branch '19972-go-client-retry'
closes #19972
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 13 Mar 2023 14:56:39 +0000 (10:56 -0400)]
19972: Fix racy test case.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 19:45:23 +0000 (14:45 -0500)]
Merge branch '20223-trash-sweep' refs #20223
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 15:52:47 +0000 (10:52 -0500)]
20227: Change default BalacePeriod to 6 hours.
This matches BalanceTimeout.
The previous time of 10 minutes was much to short, since sweeps are
not incremental, a full sweep on a moderately sized cluster could
easily take 8 minutes out of the 10 minute interval meaning
keep-balance would be running almost constantly.
Add upgrading note about keep-balance.
refs #20227
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 18:52:51 +0000 (13:52 -0500)]
20223: Even more efficient to use pluck instead of batches
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 10 Mar 2023 18:37:51 +0000 (13:37 -0500)]
19972: Lint.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 10 Mar 2023 18:35:09 +0000 (13:35 -0500)]
19972: Remove unused code.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 18:24:25 +0000 (13:24 -0500)]
20223: Need to use each_record instead of each
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 18:19:09 +0000 (13:19 -0500)]
20223: Use in_batches in used_by to tamp down memory usage
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 18:13:48 +0000 (13:13 -0500)]
20223: Make trash sweep use in_batches
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 10 Mar 2023 16:07:17 +0000 (11:07 -0500)]
Merge branch '20188-insufficient-subnet'
fixes #20188
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 10 Mar 2023 16:06:25 +0000 (11:06 -0500)]
20188: Handle InsufficientVolumeCapacity as quota error.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 14:31:57 +0000 (09:31 -0500)]
Merge branch '20223-container-bloat' refs #20223
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 10 Mar 2023 14:29:34 +0000 (09:29 -0500)]
20223: Use pluck to avoid loading full record just to get log
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Thu, 9 Mar 2023 21:50:03 +0000 (16:50 -0500)]
20183: Update container priority in creation order.
When multiple containers in a tree need updates, this is effectively
top-down order, which is more efficient. The topmost update will
update the whole tree anyway, so updating subtrees first would be a
waste.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 9 Mar 2023 21:38:59 +0000 (16:38 -0500)]
20183: Clean up sql queries.
Remove "container_requests.uuid IS NOT NULL" (this is a tautology in
an inner join)
Use same "state=Committed" condition in Rails and controller
query. This is already implied by the "containers.state in (Queued,
Locked, Running)" condition since only a Committed request should
reference such a container. However, if the distinction ever comes
up (race? manual database update?) we would want to skip the
non-Committed requests, so this serves as cheap insurance.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 9 Mar 2023 21:38:24 +0000 (16:38 -0500)]
20183: Explain priority update strategy in comments.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Thu, 9 Mar 2023 21:36:09 +0000 (16:36 -0500)]
20223: Try to make priority update much lighter weight
Uses select and pluck in several places to avoid loading full records.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 8 Mar 2023 18:28:44 +0000 (13:28 -0500)]
Merge branch '20214-r-sdk' refs #20214
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
AnetaSta22 [Thu, 23 Feb 2023 13:19:02 +0000 (14:19 +0100)]
speed up write function
Contributed by Aneta Stanczyk <aneta.stanczyk@contractors.roche.com>
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Wed, 1 Mar 2023 20:16:39 +0000 (15:16 -0500)]
19972: Disable auto-retry in diagnostics and rpc.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 28 Feb 2023 21:04:10 +0000 (16:04 -0500)]
19972: Disable auto-retry in ws and dispatcher.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 28 Feb 2023 16:30:26 +0000 (11:30 -0500)]
19972: Auto retry with exponential backoff.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 2 Mar 2023 19:48:49 +0000 (14:48 -0500)]
20188: Recognize InsufficientFreeAddressesInSubnet as quota error.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 18:42:58 +0000 (13:42 -0500)]
Merge branch '19975-oom-resubmit' refs #19975
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 18:42:05 +0000 (13:42 -0500)]
19975: s/request/specify
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 18:40:48 +0000 (13:40 -0500)]
19975: Add note about runtime.ram
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 18:36:00 +0000 (13:36 -0500)]
19975: Typo fix.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 18:21:03 +0000 (13:21 -0500)]
19975: Add OutOfMemoryRetry to CWL extensions page.
Out of memory retry is now a warning for better visibility.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 15:38:07 +0000 (10:38 -0500)]
Fix keep-web, refs #19899
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 14:54:38 +0000 (09:54 -0500)]
Merge branch '20183-update-priority-thread' refs #20183
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 7 Mar 2023 14:46:43 +0000 (09:46 -0500)]
Merge branch '20200-log-limit' refs #20200
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Stephen Smith [Mon, 6 Mar 2023 22:32:22 +0000 (17:32 -0500)]
Merge branch '19899-webdav-cache-control' into main. Refs #19899
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Brett Smith [Mon, 6 Mar 2023 21:26:59 +0000 (16:26 -0500)]
Merge branch '19981-reuse-flex-keep-cache'
Refs #19981.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Mon, 6 Mar 2023 20:10:18 +0000 (15:10 -0500)]
20200: Set MaxConcurrentRequests for test
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 6 Mar 2023 17:42:15 +0000 (12:42 -0500)]
19975: Add integration test for out-of-memory resubmit
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 6 Mar 2023 18:18:55 +0000 (13:18 -0500)]
20200: Fix test typo
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 6 Mar 2023 17:47:59 +0000 (12:47 -0500)]
20200: Fix defer mistake
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Stephen Smith [Mon, 6 Mar 2023 15:24:03 +0000 (10:24 -0500)]
19899: Add Cache-Control header to keep-web CORS response
Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
Peter Amstutz [Mon, 6 Mar 2023 14:45:50 +0000 (09:45 -0500)]
20200: Cleanups, mention changes in upgrade notes
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Mon, 6 Mar 2023 14:31:28 +0000 (09:31 -0500)]
20183: Don't do a periodic priority-update check at startup.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 6 Mar 2023 04:18:02 +0000 (23:18 -0500)]
20183: Fix test fixture.
lsf test was relying on Rails to automatically reset "lockedcontainer"
fixture priority to 0.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 6 Mar 2023 00:27:22 +0000 (19:27 -0500)]
20183: Move update_priority tests. Fix updater starvation.
Previously (in the Rails implementation) the "find containers that
have priority=0 but need priority>0" query was returning all
containers with active (committed, priority>0) requests. However, it
is possible for all such requests to have parent
containers (requesting_container_uuid) to have priority=0, in which
case Container.update_priority!() leaves the container priority at 0.
With the controller implementation, this was manifesting as lack of
progress, and warnings in logs.
This commit fixes it by not including such containers as needing
priority>0.
With the previous Rails setup, this could cause infinite recursion:
the updater thread called update_priority!(), which left
priority=0 but still triggered an after_commit hook, which
(if the updater thread had already been running for >5s) started a new
updater thread, which called update_priority!() on the same container,
etc.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Sun, 5 Mar 2023 22:06:41 +0000 (17:06 -0500)]
19981: Explain how keep_cache_* runtime constraints affect reuse
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Sun, 5 Mar 2023 21:57:11 +0000 (16:57 -0500)]
19981: Fix verb tense typo
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
projects / arvados.git / log