Javier Bértoli [Fri, 4 Dec 2020 16:22:38 +0000 (13:22 -0300)]
fix(provision): pin formulas' versions
* Pin formulas versions, to prevent changes upstream breaking the installer
* Remove verbose/debug flags
* Remove unused entries in the config pillars
refs #17177
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Javier Bértoli [Fri, 4 Dec 2020 12:17:34 +0000 (09:17 -0300)]
fix(provision): Document CA certificate purpose and installation
refs #17177
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Javier Bértoli [Thu, 3 Dec 2020 23:00:46 +0000 (20:00 -0300)]
fix(provision): Add a CA and sign certificates with it
refs #17177
As discussed [here](https://forum.arvados.org/t/debugging-arvados-deployed-with-salt/58/8)
and [here](https://gitter.im/arvados/community?at=
5fc65683496ca3372e3474a3), Arvados needs
certs signed by a known CA to work correctly.
This PR adds a CA and leaves a copy of the certificate in the installer directory.
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Javier Bértoli [Wed, 2 Dec 2020 13:44:27 +0000 (10:44 -0300)]
fix(docs): broken link
reported on [github](https://github.com/arvados/arvados-formula/issues/3#issuecomment-
733019750)
No issue #
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Peter Amstutz [Tue, 1 Dec 2020 21:16:05 +0000 (16:16 -0500)]
Merge branch '17072-update-cwltool' refs #17072
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 30 Nov 2020 22:03:33 +0000 (17:03 -0500)]
Merge branch '17022-user-activity-report' refs #17022
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 30 Nov 2020 19:16:56 +0000 (14:16 -0500)]
17022: Update install docs & fix Marc's email.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Mon, 30 Nov 2020 16:55:41 +0000 (11:55 -0500)]
Now that our k8s helm charts retain state, update the documentation
accordingly.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Mon, 30 Nov 2020 16:03:53 +0000 (11:03 -0500)]
Merge branch '16774-keep-web-errors' refs #16774
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Mon, 30 Nov 2020 01:51:27 +0000 (20:51 -0500)]
Fix more golint warnings - update crunchrun tests.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sun, 29 Nov 2020 02:26:28 +0000 (21:26 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Fri, 27 Nov 2020 13:28:41 +0000 (08:28 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Thu, 26 Nov 2020 22:17:10 +0000 (17:17 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Javier Bértoli [Thu, 26 Nov 2020 18:49:50 +0000 (15:49 -0300)]
Merge branch 'pvanheus-fix_cluster_test' from github.com/pvanheus:fix_cluster_test
No issue #
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Peter van Heusden [Thu, 26 Nov 2020 18:35:16 +0000 (20:35 +0200)]
Fix hardcoded initial user name and email
Arvados-DCO-1.1-Signed-off-by: Peter Van Heusden <vh@sanbi.ac.za>
Ward Vandewege [Wed, 25 Nov 2020 22:24:35 +0000 (17:24 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Wed, 25 Nov 2020 21:45:53 +0000 (16:45 -0500)]
17072: Fix for added args on cwltool.docker.get_image
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 25 Nov 2020 21:13:58 +0000 (16:13 -0500)]
17072: Increase default submit thread concurrency to 4
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 25 Nov 2020 21:10:57 +0000 (16:10 -0500)]
17072: Fix imports. Use task_queue from cwltool.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 25 Nov 2020 20:42:08 +0000 (15:42 -0500)]
17072: Bump cwltool version for unbound 'result' fix
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Javier Bértoli [Wed, 25 Nov 2020 18:06:50 +0000 (15:06 -0300)]
Merge branch '17150-system-root-token'
closes #17146, #17147, #17150
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Tom Clegg [Wed, 25 Nov 2020 17:14:34 +0000 (12:14 -0500)]
17009: Fix bucket-level ops using virtual host-style requests.
refs #17009
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Javier Bértoli [Wed, 25 Nov 2020 02:03:37 +0000 (23:03 -0300)]
fix(test): wrong variable comparison
refs #17146, #17147, #17150
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Javier Bértoli [Wed, 25 Nov 2020 02:02:31 +0000 (23:02 -0300)]
fix(docker): formula upgraded
refs #17146, #17147, #17150
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Peter Amstutz [Tue, 24 Nov 2020 23:00:12 +0000 (18:00 -0500)]
17022: Add example usage of keyset_list_all to code cookbook.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 24 Nov 2020 22:51:21 +0000 (17:51 -0500)]
17022: Add test cases for keyset_list_all
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 24 Nov 2020 22:03:32 +0000 (17:03 -0500)]
17022: Improve output. Added documentation page.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Javier Bértoli [Tue, 24 Nov 2020 20:17:25 +0000 (17:17 -0300)]
Fix salt-install and test scripts
* add checks to find existing resources
* run some shellcheck
refs #17146, #17147, #17150
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Peter Amstutz [Tue, 24 Nov 2020 18:40:21 +0000 (13:40 -0500)]
16774: text/plain response uses crlf. Tests check error codes.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Tue, 24 Nov 2020 18:17:44 +0000 (13:17 -0500)]
Documentation cleanups.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 24 Nov 2020 18:17:06 +0000 (13:17 -0500)]
Make run-build-packages-python-and-ruby.sh say so when it decides to
skip building Python packages.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Tom Clegg [Tue, 24 Nov 2020 16:55:48 +0000 (11:55 -0500)]
Merge branch '17106-s3-fed-token'
fixes #17106
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 24 Nov 2020 15:26:36 +0000 (10:26 -0500)]
17106: Add examples to S3 auth instructions.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Peter van Heusden [Wed, 18 Nov 2020 14:01:32 +0000 (16:01 +0200)]
Fix salt-install's crunch-dispatch-local config and tests
* added [@pvanheus fix]:https://github.com/arvados/arvados/pull/140
* added [a debug parameter to the provision script](https://gitter.im/arvados/community?at=
5fb54b4bd37a1a13d6b46a05)
* added a test script to verify the cluster is able to run a CWL workflow
* document salt-install's workflow test
refs #17146, #17147, #17150
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>
Tom Clegg [Tue, 24 Nov 2020 04:45:47 +0000 (23:45 -0500)]
17106: Fix key unescape: don't convert + to space.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Peter Amstutz [Mon, 23 Nov 2020 20:04:48 +0000 (15:04 -0500)]
16774: Fix tests. Use encoder for xml error response.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Mon, 23 Nov 2020 19:52:57 +0000 (14:52 -0500)]
17106: Allow use of URL-encoded token as S3 access/secret key.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Ward Vandewege [Mon, 23 Nov 2020 19:48:35 +0000 (14:48 -0500)]
Fix OS package iteration number in run-build-docker-jobs-image.sh
refs #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Mon, 23 Nov 2020 16:51:21 +0000 (11:51 -0500)]
Update error checking in run-build-docker-jobs-image.sh
refs #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Mon, 23 Nov 2020 16:47:53 +0000 (11:47 -0500)]
Remove now-unneeded code from sdk/cwl/test_with_arvbox.sh, add comment
to clarify that calculate_python_sdk_cwl_package_versions now outputs
python-style package suffixes.
refs #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Mon, 23 Nov 2020 16:23:51 +0000 (11:23 -0500)]
Merge branch '17154-wb-profile' refs #17154
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 23 Nov 2020 15:59:06 +0000 (10:59 -0500)]
17154: Add comment. Use strings.HasPrefix
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Mon, 23 Nov 2020 14:20:56 +0000 (09:20 -0500)]
Now that calculate_python_sdk_cwl_package_versions returns python-style
version string suffixes (.dev/rc), make sure the 2 scripts that use that
function are adapted accordingly.
refs #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sun, 22 Nov 2020 13:35:06 +0000 (08:35 -0500)]
Move the handle_python_package() function from run-library.sh to
run-build-packages-python-and-ruby.sh, it is only used in the latter.
Update it to call python3 instead of just 'python'.
refs #15888
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sat, 21 Nov 2020 20:53:01 +0000 (15:53 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Fri, 20 Nov 2020 23:03:30 +0000 (18:03 -0500)]
16774: Keep-web errors include messages
Errors on the regular keep-web side return plain text responses.
Errors on the S3 side return XML error responses that S3 clients
expect.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Fri, 20 Nov 2020 22:31:50 +0000 (17:31 -0500)]
Remove one more python2 remnant. This is a fix for the
python3-arvados-cwl-runner package, it now installs /usr/bin/cwltool
again.
refs #15888
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Fri, 20 Nov 2020 21:01:25 +0000 (16:01 -0500)]
17154: Copy updates on federated users to local cluster
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 20 Nov 2020 20:25:27 +0000 (15:25 -0500)]
17154: Make sure most current user record is loaded.
Convert 'option' items (which are symbols) to strings for value comparison
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 20 Nov 2020 17:59:12 +0000 (12:59 -0500)]
Merge branch '17015-cds-correct-containers' refs #17015
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Fri, 20 Nov 2020 15:40:30 +0000 (10:40 -0500)]
17106: Recommend using full tokens for S3 access.
Accept munged ("/" => "_") tokens in S3 requests with V2 signatures.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Thu, 19 Nov 2020 21:50:26 +0000 (16:50 -0500)]
Merge branch '17009-s3-bucket-vhost'
closes #17009
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Thu, 19 Nov 2020 21:48:15 +0000 (16:48 -0500)]
17009: Mention S3 considerations in keep-web install doc.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Peter Amstutz [Thu, 19 Nov 2020 20:36:29 +0000 (15:36 -0500)]
17015: Ensure that containers belong to this cluster
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Wed, 18 Nov 2020 22:35:29 +0000 (17:35 -0500)]
17009: Support accessing S3 with virtual hosted-style URLs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Ward Vandewege [Wed, 18 Nov 2020 21:24:19 +0000 (16:24 -0500)]
Merge branch '16950-add-costanalyzer' into master
closes #16950
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Wed, 18 Nov 2020 21:23:51 +0000 (16:23 -0500)]
Merge branch 'master' into 16950-add-costanalyzer
refs #16950
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Wed, 18 Nov 2020 21:20:40 +0000 (16:20 -0500)]
16950: switch to os.UserHomeDir(), and send all output to stderr.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Wed, 18 Nov 2020 15:09:51 +0000 (10:09 -0500)]
16950: make specifying of the output directory mandatory (remove
default).
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Wed, 18 Nov 2020 20:32:21 +0000 (15:32 -0500)]
17022: Fix packaging
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 18 Nov 2020 20:23:36 +0000 (15:23 -0500)]
17022: Make this a fully packaged tool.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Wed, 18 Nov 2020 20:02:55 +0000 (15:02 -0500)]
17022: Python packaging for arv-user-activity
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 17 Nov 2020 22:46:42 +0000 (17:46 -0500)]
17022: Specify time period in days on command line.
Also print timestamps in local time zone.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Fri, 13 Nov 2020 23:17:54 +0000 (18:17 -0500)]
17022: Produce user activity report from the audit logs
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 9 Nov 2020 18:42:12 +0000 (13:42 -0500)]
17022: Start work on user activity reporting script
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Tom Clegg [Wed, 18 Nov 2020 15:10:02 +0000 (10:10 -0500)]
17106: Update docs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Wed, 18 Nov 2020 15:02:40 +0000 (10:02 -0500)]
17106: Comment on stored_secret behavior.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 17 Nov 2020 20:42:28 +0000 (15:42 -0500)]
17106: Improve handling of bare tokens issued by remote clusters.
When caching, use the remote cluster's original token UUID.
When returning the current api_client_authorization record, include
the secret supplied by the caller, even when only storing the HMAC in
the database.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 17 Nov 2020 20:42:11 +0000 (15:42 -0500)]
17106: Clean up test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 17 Nov 2020 14:39:50 +0000 (09:39 -0500)]
17106: Skip s3cmd test if s3cmd not installed.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 17 Nov 2020 02:25:12 +0000 (21:25 -0500)]
17106: Test S3 with modified v2 token issued by LoginCluster.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Tue, 17 Nov 2020 01:37:05 +0000 (20:37 -0500)]
17106: Accept v2 token with / replaced by _ as s3 access/secret key.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Ward Vandewege [Tue, 17 Nov 2020 22:44:21 +0000 (17:44 -0500)]
Merge branch '17012-fix-python-version-calculation' into master
closes #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 17 Nov 2020 22:43:46 +0000 (17:43 -0500)]
Merge branch 'master' into 17012-fix-python-version-calculation
refs #17012
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 17 Nov 2020 22:35:31 +0000 (17:35 -0500)]
17012: switch calculate_python_sdk_cwl_package_versions() to use the
python version calculation code. This effectively is bash calling
python calling bash, which I don't like, but the code is simpler.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 17 Nov 2020 15:26:58 +0000 (10:26 -0500)]
17012: update calculation of python version number in run-library.sh
(used by a few docker image scripts).
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 17 Nov 2020 14:52:53 +0000 (09:52 -0500)]
17012: remove old gittaggers.py files.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Mon, 16 Nov 2020 20:08:26 +0000 (15:08 -0500)]
17012: when calculating the version of our Python packages, take the
build directory into account.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 17 Nov 2020 01:40:47 +0000 (20:40 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sun, 15 Nov 2020 19:18:40 +0000 (14:18 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sat, 14 Nov 2020 18:22:50 +0000 (13:22 -0500)]
16950: refactor caching; use ~/.cache/arvados/costanalyzer for that
purpose.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Fri, 13 Nov 2020 21:22:57 +0000 (16:22 -0500)]
16950: more changes after review.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Tom Clegg [Fri, 13 Nov 2020 21:57:25 +0000 (16:57 -0500)]
Merge branch '17111-deselect-unsigned-manifest'
fixes #17111
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Tom Clegg [Fri, 13 Nov 2020 20:44:43 +0000 (15:44 -0500)]
17111: Deselect unsigned_manifest_text by default in groups#contents.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
Ward Vandewege [Fri, 13 Nov 2020 01:02:56 +0000 (20:02 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Thu, 12 Nov 2020 03:07:48 +0000 (22:07 -0500)]
16950: changes after review.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sun, 8 Nov 2020 21:41:21 +0000 (16:41 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sat, 7 Nov 2020 13:45:42 +0000 (08:45 -0500)]
Merge branch '17086-add-focal-fossa' into master
closes #17086
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Sat, 7 Nov 2020 13:45:09 +0000 (08:45 -0500)]
17086: Merge branch 'master' into 17086-add-focal-fossa
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Fri, 6 Nov 2020 13:34:38 +0000 (08:34 -0500)]
Fix require path in the 'script/rails' script for api server.
closes #17094
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Thu, 5 Nov 2020 19:43:37 +0000 (14:43 -0500)]
17086: build packages for Ubuntu 20.04, Focal Fossa.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Thu, 5 Nov 2020 20:15:13 +0000 (15:15 -0500)]
Merge branch '17090-exclusive-lock' refs #17090
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 5 Nov 2020 19:55:35 +0000 (14:55 -0500)]
17090: Migration to refresh_permissions
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Thu, 5 Nov 2020 16:09:30 +0000 (11:09 -0500)]
17090: Use EXCLUSIVE lock for permission updates
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Ward Vandewege [Thu, 5 Nov 2020 00:52:42 +0000 (19:52 -0500)]
Fix more golint warnings.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Ward Vandewege [Tue, 3 Nov 2020 21:34:16 +0000 (16:34 -0500)]
16950: add tests for the costanalyzer.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>
Peter Amstutz [Tue, 3 Nov 2020 18:29:21 +0000 (13:29 -0500)]
Merge branch '17040-slow-query' refs #17040
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 3 Nov 2020 18:01:57 +0000 (13:01 -0500)]
17040: Update structure.sql
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Tue, 3 Nov 2020 17:12:59 +0000 (12:12 -0500)]
17040: RefreshTrashedGroups migration, clear_permissions calls forget_cached_group_perms
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
Peter Amstutz [Mon, 2 Nov 2020 21:20:02 +0000 (16:20 -0500)]
17040: Cache results of User.group_permissions
When requesting a list of groups (either directly, or with the
"shared" endpoint) it calls writable_by for each group. This
indirectly calls User.group_permissions, which makes a database query.
When it does this for every group (with a database query each time),
which gets very expensive.
To address this, this commit caches the result of group_permissions on
the user object.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>
projects / arvados.git / log