Brett Smith [Wed, 26 Jul 2023 13:40:39 +0000 (09:40 -0400)]
Merge branch '20543-pip-install-virtualenv'
Closes #20543.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 19:44:03 +0000 (15:44 -0400)]
20543: Recommend users install Python packages in a virtualenv
`pip install --user` doesn't work on distributions that have adopted
PEP 668, including Debian 12+ and Ubuntu 23.04+.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 24 Jul 2023 14:53:13 +0000 (10:53 -0400)]
Merge branch '20343-regexp-prefixes'
Closes #20343.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Fri, 21 Jul 2023 23:05:33 +0000 (19:05 -0400)]
17244: Use crunch-run's cgroup, not init's, for "host" stats.
This gets much closer to "everything on the host" with minimal
effort.
Tracking the furthest ancestor of crunch-run's cgroup might be even
better, but will require additional support in crunchstat.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 21 Jul 2023 13:53:34 +0000 (09:53 -0400)]
17244: Test findCgroup on cgroup files in crunchstat testdata.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 21 Jul 2023 13:21:41 +0000 (09:21 -0400)]
Merge branch '20726-s3-session-release'
refs #20726
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 20 Jul 2023 15:26:51 +0000 (11:26 -0400)]
20726: Fix unreleased session in s3 handler.
Caused subsequent requests with the same token to hang after reaching
expiry time.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 19 Jul 2023 14:40:07 +0000 (10:40 -0400)]
17244: Fix singularity reporting host stats as container stats.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 18 Jul 2023 20:28:30 +0000 (16:28 -0400)]
17244: Make -cgroup-parent-subsystem=X work in cgroups v2.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 11 Jul 2023 14:03:58 +0000 (10:03 -0400)]
17244: Refactor crunchstat to use cgroup unified/hybrid modes.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 19 Jul 2023 04:54:42 +0000 (00:54 -0400)]
17244: Remove crunchstat debian package.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Tue, 18 Jul 2023 14:20:50 +0000 (10:20 -0400)]
Merge branch '20731-user-guide-wb2' refs #20731
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Thu, 13 Jul 2023 20:48:04 +0000 (16:48 -0400)]
Merge branch '20726-s3list-pages'
refs #20726
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 12 Jul 2023 04:19:08 +0000 (00:19 -0400)]
20726: Fix traversing projects/collections that precede page marker.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Tue, 11 Jul 2023 21:06:47 +0000 (17:06 -0400)]
20731: Make some text changes & reorganization based on workbench 2 updates
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Tue, 11 Jul 2023 19:25:22 +0000 (15:25 -0400)]
20726: Fix ListObjects[V2] pages duplicating last item on next page.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 11 Jul 2023 17:38:38 +0000 (13:38 -0400)]
20726: Update s3ListObjects paging test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 11 Jul 2023 17:39:20 +0000 (13:39 -0400)]
Merge branch '20667-maxsuper-atquota'
fixes #20667
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Mon, 10 Jul 2023 15:56:21 +0000 (11:56 -0400)]
Merge branch '20684-user-install-doc-fixes'
Closes #20684.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 10 Jul 2023 15:50:56 +0000 (11:50 -0400)]
20684: Modernize pycurl dependency discussion
These instructions should be good for Debian 10+ and I'm assuming
corresponding Ubuntu too.
* Update libssl package name
* Clarify the command is good for multiple distros
* Prefer apt over apt-get
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Mon, 10 Jul 2023 15:48:21 +0000 (11:48 -0400)]
20684: Simplify curl dependency background
Specifying that the "Python SDK" depends on curl is needless detail
here.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 21:19:13 +0000 (17:19 -0400)]
20684: Note tutorial expectations for all Crunch CLI tools
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 21:18:15 +0000 (17:18 -0400)]
20684: Note crunchstat-summary can be installed in a virtualenv
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 21:03:46 +0000 (17:03 -0400)]
20684: Copyedit tutorial expectations include
* Fix run-on sentence
* Clean up punctuation spacing
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 20:57:38 +0000 (16:57 -0400)]
20684: Expand "Getting started at the command line" documentation
* Add more structure to the page
* Be explicit that we usually recommend using a VM
* Highlight that using our package repository is the simplest way to
install tools
* List all the tools you can install, with more details
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 7 Jul 2023 20:27:19 +0000 (16:27 -0400)]
20684: Clean up FUSE references in a-c-r install instructions
It looks like this page started as a copy of the corresponding FUSE
page, and we didn't update all the references.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Fri, 7 Jul 2023 13:49:24 +0000 (09:49 -0400)]
Merge branch '20559-dav-concurrent-writes'
refs #20559
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 19:34:43 +0000 (16:34 -0300)]
Fixes salt bootstrapping on nodes by the installer. Refs #20665
Salt v3004 is no more regarded as 'stable', so the boostrapping script started
to fail. Asking for 'old-stable' fixes this, at least for some time.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Thu, 6 Jul 2023 18:01:12 +0000 (14:01 -0400)]
Merge branch '20722-docker-upgrade'
fixes #20722
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 6 Jul 2023 15:54:40 +0000 (11:54 -0400)]
20722: Update ContainerCreate usage for new docker client library.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 15:15:01 +0000 (12:15 -0300)]
Merge branch '20707-aws-compute-ami-cgroupsv1'. Closes #20707
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 14:52:53 +0000 (11:52 -0300)]
20722: Cleans up dependencies with 'go mod tidy'
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 14:51:40 +0000 (11:51 -0300)]
20722: Upgrades github.com/docker/docker to latest version.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 14:26:05 +0000 (11:26 -0300)]
Merge branch '20718-grpc-upgrade'. Closes #20718
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 13:41:50 +0000 (10:41 -0300)]
20718: Cleans up by running "go mod tidy -compat=1.17"
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 6 Jul 2023 13:36:31 +0000 (10:36 -0300)]
20718: Upgrades google.golang.org/grpc to 1.56.1
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Wed, 5 Jul 2023 20:58:07 +0000 (16:58 -0400)]
20343: Define Python regexps with raw strings
Prevent the DeprecationWarning added to Python 3.6 for unrecognized
string escape sequences. Found candidates by running:
git grep --line-number -E '\bre\.\w+\([^r]*\\'
Most of the changes just add the raw string prefix `r` to these regexps,
but I did make some other readability improvements while I was at it,
including switching from regexps to plain string test methods where
appropriate.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 5 Jul 2023 19:46:21 +0000 (15:46 -0400)]
Merge branch '20710-shlex-quoting'
Closes #20710.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Wed, 5 Jul 2023 19:45:25 +0000 (15:45 -0400)]
20710: Remove old backwards compatibility import fallback
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Wed, 5 Jul 2023 17:28:54 +0000 (14:28 -0300)]
20707: Don't rely on GRUB_CMDLINE_LINUX be empty in /etc/default/grub
Debian 11 AWS AMIs come with some settings added in that variable.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Brett Smith [Wed, 5 Jul 2023 16:20:57 +0000 (12:20 -0400)]
20710: Switch from pipes.quote to shlex.quote
pipes is deprecated in Python 3.11+ and slated to be removed in Python
3.13. Fortunately we only use pipes.quote, which is literally the same
as shlex.quote:
>>> import pipes
<stdin>:1: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
>>> import shlex
>>> pipes.quote is shlex.quote
True
Switch to that instead for continued support.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Lucas Di Pentima [Tue, 4 Jul 2023 19:25:46 +0000 (16:25 -0300)]
20707: Updates default to Debian 11 AMI. Forces cgroups v1 for crunchstat.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Fri, 30 Jun 2023 14:28:38 +0000 (10:28 -0400)]
20559: Trim session count explicitly instead of using LRU cache.
The LRU cache automatically implemented the session count limit by
dropping the oldest session, which would break our "one session per
token" rule (by evicting a session and creating a new one while the
old session was still in use) when there were more active sessions
than the configured limit.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Zoë Ma [Fri, 30 Jun 2023 07:24:55 +0000 (15:24 +0800)]
Doc: Update the included note about Workbench 2 transition.
Reword the text to center the Workbench 2.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 30 Jun 2023 06:57:31 +0000 (14:57 +0800)]
Doc: Update "Working with an Arvados git repository"
- Updated inline elements for Workbench 2.
TODO: Should double-check the instructions of using Workbench when a git
repo is ready.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 30 Jun 2023 06:54:07 +0000 (14:54 +0800)]
Doc: Update "Adding a new Arvados git repository"
- Updated inline elements and the first 2 figures for Workbench 2.
TODO: The last figure remains to be updated.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Wed, 28 Jun 2023 15:52:03 +0000 (23:52 +0800)]
Doc: Update "Starting a Workflow at the Command Line"
A minor fix for the mention of starting a workflow from Workbench.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Wed, 28 Jun 2023 15:39:51 +0000 (23:39 +0800)]
Doc: Update "Trashing and untrashing data"
Revised the part about untrashing using Workbench for Workbench 2,
including replacing the figure.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Mon, 26 Jun 2023 02:32:00 +0000 (10:32 +0800)]
Doc: Add a figure caption in "Downloading data".
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Sun, 25 Jun 2023 09:38:52 +0000 (17:38 +0800)]
Doc: Update "Downloading data".
- Updated the inline content for Workbench 2.
- Added a new screenshot for the creation of sharing URL.
TODO: The view of the URL from the PoV of the recipient has not been
updated.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Sat, 24 Jun 2023 09:09:30 +0000 (17:09 +0800)]
Doc: Update "Uploading data"
- Updated the text for Workbench 2 and expanded the steps.
- Removed the old figures and replaced them with new ones from Workbench
2.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Sat, 24 Jun 2023 06:17:41 +0000 (14:17 +0800)]
Doc: Remove reference to "dashboard" in template about adding SSH key
- The "dashboard" is no longer a named component of the Workbench UI.
- The <h3> under the <h1> is superfluous.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Sat, 24 Jun 2023 06:12:58 +0000 (14:12 +0800)]
Doc: Update "Accessing an Arvados VM with SSH - Unix Environments"
- Updated the included template about adding SSH keys, to reflect
changes in Workbench 2. A screenshot was updated.
- Added slightly more context to instructions about using ssh keypair
and agent for the benefit of new users.
- Minor textual fixes for clarity.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Sat, 24 Jun 2023 00:33:19 +0000 (08:33 +0800)]
Doc: Update "Accessing an Arvados VM with Webshell"
- Updated inline elements for Workbench 2.
- Replaced screenshot with higher-definition and updated one.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 23 Jun 2023 13:23:47 +0000 (21:23 +0800)]
Doc: Update "Getting an API token".
- Updated the references to Workbench 2 UI elements.
- Expand and reword some instructions about creating file in shell
(because a novice user may accidentally wipe out the file), and how
appending to .bashrc does its job.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 23 Jun 2023 09:48:55 +0000 (17:48 +0800)]
Doc: Update "Running a workflow using Workbench"
- Updated the procedures to match the operations in Workbench2.
- Updated the styles of inline elements to match their roles in
Workbench2.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 23 Jun 2023 09:47:58 +0000 (17:47 +0800)]
Doc: Update "Accessing Arvados Workbench" for Workbench 2
- Remove references to the "Dashboard"
- Replace the screenshot with higher-definition, updated one.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Zoë Ma [Fri, 23 Jun 2023 06:50:50 +0000 (14:50 +0800)]
Doc: Update Ruby dependencies and fix Rakefile
- Updated rubygems with "bundle update" for dependencies.
- Fixed Rakefile by replacing the obsoleted method aliases.
Arvados-DCO-1.1-Signed-off-by: Zoë Ma <zoe.ma@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 19:38:04 +0000 (16:38 -0300)]
20610: Tweaks postgresql access control to allow multiple controller hosts.
Also, explicitly adds the dispatcher's IP to the ACL, in case is located
on a different node.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 19:33:28 +0000 (16:33 -0300)]
20610: Tweaks controller's nginx pillar to support "balanced mode".
When balancer is enabled:
* SSL nginx is not needed
* Each controller should answer to its own hostname instead of the cluster's
domain.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 19:13:06 +0000 (16:13 -0300)]
20610: Updates arvados' pillar to handle the dispatcher's IP separately.
Also, re-arranges the _INT_IP envvars to be more explicit about IP sharing.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 19:05:31 +0000 (16:05 -0300)]
20610: Adds the new pillars for the balancer role.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 18:59:53 +0000 (15:59 -0300)]
20610: Adds 'balancer' role to the installer scripts.
Also, fixes a salt bootstrap issue dur to the fact that as of today, the 3004
version is no longer considered 'stable'.
Also, changes the way we use SSH so that no envvars are forwarded to remote
hosts, to avoid having 'setlocale' warnings mixed with salt cmd.run
outputs when using Debian distros to run the installer. This avoids issues
on the first run, before salt configures the remote node's locale.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Thu, 29 Jun 2023 18:57:40 +0000 (15:57 -0300)]
20610: Removes unused keepstore reference from Terraform.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Thu, 29 Jun 2023 20:47:57 +0000 (16:47 -0400)]
Merge branch '20606-reuse-preemptible'
fixes #20606
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 29 Jun 2023 18:59:16 +0000 (14:59 -0400)]
20559: Explain locking strategy in comments.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 29 Jun 2023 15:57:29 +0000 (11:57 -0400)]
20559: Refactor session sharing.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 14 Jun 2023 21:17:18 +0000 (17:17 -0400)]
20559: Garbage collect unused per-collection mutexes.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 14 Jun 2023 20:45:58 +0000 (16:45 -0400)]
20559: Serialize writes for a given collection ID.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 13 Jun 2023 21:47:14 +0000 (17:47 -0400)]
20559: Test concurrent writes using same token.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 27 Jun 2023 14:36:08 +0000 (10:36 -0400)]
20606: Update retry tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Lucas Di Pentima [Tue, 27 Jun 2023 13:31:38 +0000 (10:31 -0300)]
Merge branch '20665-installer-secrets-handling'. Closes #20665
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Tue, 27 Jun 2023 13:29:27 +0000 (10:29 -0300)]
20665: Fixes logic script error & minor docs improvements.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Mon, 26 Jun 2023 23:00:40 +0000 (19:00 -0400)]
20606: Don't reuse unstarted preemptible ctr for non-preemptible cr.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 26 Jun 2023 23:02:49 +0000 (19:02 -0400)]
20667: Add InitialQuotaEstimate config.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Mon, 26 Jun 2023 17:24:39 +0000 (13:24 -0400)]
Merge branch '20619-arv-client-libfuse' closes #20619
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Lucas Di Pentima [Mon, 26 Jun 2023 15:52:56 +0000 (12:52 -0300)]
20665: Provides examples on how to securely handle the secrets file.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Lucas Di Pentima [Mon, 26 Jun 2023 15:47:54 +0000 (12:47 -0300)]
20665: Code style improvements.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>
Tom Clegg [Mon, 26 Jun 2023 15:17:48 +0000 (11:17 -0400)]
20667: Add at_quota metric.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 26 Jun 2023 15:06:35 +0000 (11:06 -0400)]
20667: Adjust timeout/retry settings to address occasional failures.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 26 Jun 2023 15:00:33 +0000 (11:00 -0400)]
20667: Reduce max supervisors if pool reaches cloud quota.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Peter Amstutz [Fri, 23 Jun 2023 19:46:31 +0000 (15:46 -0400)]
20619: Add test arvados-client executable starts
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Fri, 23 Jun 2023 18:44:17 +0000 (14:44 -0400)]
Merge branch '20663-login-sync-config'
Closes #20663.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Fri, 23 Jun 2023 18:42:04 +0000 (14:42 -0400)]
20619: Add fpm-info for arvados-client to depend on fuse
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Brett Smith [Fri, 23 Jun 2023 18:14:38 +0000 (14:14 -0400)]
20663: Expand the default SyncIgnoredGroups
We discussed at standup that we prioritize the security of the default
configuration over backwards compatibility. This new default does that.
The list of groups is primarily informed by:
<https://wiki.debian.org/SystemGroups#Other_System_Groups>
<https://docs.fedoraproject.org/en-US/packaging-guidelines/UsersAndGroups/#_list_of_statically_allocated_uidgid_and_corresponding_package>
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 23 Jun 2023 18:01:21 +0000 (14:01 -0400)]
20663: Merge 2.6.3 upgrading notes
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 23 Jun 2023 15:24:22 +0000 (11:24 -0400)]
20663: Add IgnoredGroups configuration to arvados-login-sync
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Fri, 23 Jun 2023 13:25:24 +0000 (09:25 -0400)]
20663: Bugfix configuration lookup
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Tom Clegg [Thu, 22 Jun 2023 17:27:24 +0000 (13:27 -0400)]
Merge branch '20425-fed-pdh-retry'
fixes #20425
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 22 Jun 2023 16:55:05 +0000 (12:55 -0400)]
20425: Merge branch 'main' into 20425-fed-pdh-retry
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Brett Smith [Thu, 22 Jun 2023 15:10:29 +0000 (11:10 -0400)]
20663: Add configuration for arvados-login-sync
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 22 Jun 2023 14:52:05 +0000 (10:52 -0400)]
20663: Clarify comment about what "safe" config means
I might've liked to reveal configuration to authenticated clients, but
not unauthenticated ones. Unfortunately the code doesn't currently
support that.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 22 Jun 2023 14:43:36 +0000 (10:43 -0400)]
20663: Make arvados-login-sync actions configurable
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 22 Jun 2023 14:18:02 +0000 (10:18 -0400)]
20663: Use set operations for group management
Make code more consistent for easier readability.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 22 Jun 2023 14:16:40 +0000 (10:16 -0400)]
20663: Make minimum groups configurable
Note this no longer adds the user's own group to the membership
list. The administrator can control that by adjusting USERGROUPS_ENAB
in /etc/login.defs.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Brett Smith [Thu, 22 Jun 2023 13:48:44 +0000 (09:48 -0400)]
20663: Improve permissions and ownership handling
* Set permissions of everything at creation time.
* Only change ownership for things we touch.
* Manage group ownership as well as user
(having things owned by user:root is weird).
* Modernize style.
This is preparation for allowing administrators to configure what
resources arvados-login-sync manages.
Note this means arvados-login-sync no longer changes permissions for a
user's home directory. The administrator can do that by setting UMASK
in /etc/login.defs.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith@curii.com>
Peter Amstutz [Thu, 22 Jun 2023 14:43:12 +0000 (10:43 -0400)]
Merge branch '20374-workflow-main-prop' refs #20374
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 22 Jun 2023 14:41:10 +0000 (10:41 -0400)]
Merge branch '20660-r-sdk-pr-204' refs #20660
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 22 Jun 2023 14:29:53 +0000 (10:29 -0400)]
20374: add arv:workflowMain to metadata properties doc.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 22 Jun 2023 13:54:01 +0000 (09:54 -0400)]
20660: Tweaks to eliminate check warnings
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
projects / arvados.git / log