arvados.git
3 years agoMerge branch '17388-arvcopy-storage-classes' into main. Closes #17388
Lucas Di Pentima [Mon, 26 Jul 2021 14:32:05 +0000 (11:32 -0300)]
Merge branch '17388-arvcopy-storage-classes' into main. Closes #17388

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17388: Adds storage classes support to arv-copy.
Lucas Di Pentima [Fri, 23 Jul 2021 22:52:00 +0000 (19:52 -0300)]
17388: Adds storage classes support to arv-copy.

Also, updates tests and documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Handle case where the cache collection update fails
Peter Amstutz [Fri, 23 Jul 2021 18:35:15 +0000 (14:35 -0400)]
17813: Handle case where the cache collection update fails

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17755: Merge branch 'main' into 17755-add-singularity-to-compute-image
Ward Vandewege [Fri, 23 Jul 2021 18:04:16 +0000 (14:04 -0400)]
17755: Merge branch 'main' into 17755-add-singularity-to-compute-image

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17535: Modify provision script to run on Centos
Javier Bértoli [Fri, 23 Jul 2021 18:02:45 +0000 (15:02 -0300)]
17535: Modify provision script to run on Centos

Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Thu, 22 Jul 2021 21:32:44 +0000 (17:32 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Fixes tests.
Lucas Di Pentima [Thu, 22 Jul 2021 21:02:53 +0000 (18:02 -0300)]
16665: Fixes tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: continue refactor & fix tests
Peter Amstutz [Thu, 22 Jul 2021 20:44:19 +0000 (16:44 -0400)]
17813: continue refactor & fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago16665: Wraps error type to be able to differentiate it in a case switch.
Lucas Di Pentima [Thu, 22 Jul 2021 20:30:51 +0000 (17:30 -0300)]
16665: Wraps error type to be able to differentiate it in a case switch.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago16665: Exposes bug through updated test.
Lucas Di Pentima [Thu, 22 Jul 2021 20:24:26 +0000 (17:24 -0300)]
16665: Exposes bug through updated test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17813: Refactor singularity image loading / caching / conversion
Peter Amstutz [Thu, 22 Jul 2021 20:22:39 +0000 (16:22 -0400)]
17813: Refactor singularity image loading / caching / conversion

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17830: Copies request's X-Request-Id header to response. Moves tests.
Lucas Di Pentima [Thu, 22 Jul 2021 18:31:11 +0000 (15:31 -0300)]
17830: Copies request's X-Request-Id header to response. Moves tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Sets up X-Request-Id if not provided by client. Adds it to the response.
Lucas Di Pentima [Wed, 21 Jul 2021 23:51:10 +0000 (20:51 -0300)]
17830: Sets up X-Request-Id if not provided by client. Adds it to the response.

Uses the IDGenerator to explicitly set one request id if needed, so that it
can return it to the client without the need to dig it out from the railsAPI
response.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds controller tests exposing the X-Request-Id header propagation bug.
Lucas Di Pentima [Tue, 20 Jul 2021 20:59:58 +0000 (17:59 -0300)]
17830: Adds controller tests exposing the X-Request-Id header propagation bug.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Customizes ActionDispatch::RequestId middleware to our needs.
Lucas Di Pentima [Tue, 20 Jul 2021 19:23:36 +0000 (16:23 -0300)]
17830: Customizes ActionDispatch::RequestId middleware to our needs.

Moves all the request id code to the proper middleware, and updates tests.
As functional tests are "unit tests for controllers", things that happen inside
middlewares need to be tested from integration tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17830: Adds test exposing incorrect X-Request-Id format on railsAPI.
Lucas Di Pentima [Tue, 20 Jul 2021 14:26:21 +0000 (11:26 -0300)]
17830: Adds test exposing incorrect X-Request-Id format on railsAPI.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17756: Fix test.
Tom Clegg [Thu, 22 Jul 2021 15:10:25 +0000 (11:10 -0400)]
17756: Fix test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Note that bsub sudo user must exist on worker nodes.
Tom Clegg [Thu, 22 Jul 2021 15:06:29 +0000 (11:06 -0400)]
17756: Note that bsub sudo user must exist on worker nodes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Move "no suitable instance type" reporting to dispatch lib.
Tom Clegg [Thu, 22 Jul 2021 14:36:14 +0000 (10:36 -0400)]
17756: Move "no suitable instance type" reporting to dispatch lib.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17863-bundler-update'
Ward Vandewege [Thu, 22 Jul 2021 14:27:45 +0000 (10:27 -0400)]
Merge branch '17863-bundler-update'

closes #17863

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17863: Merge branch 'main' into 17863-bundler-update
Ward Vandewege [Thu, 22 Jul 2021 14:25:59 +0000 (10:25 -0400)]
17863: Merge branch 'main' into 17863-bundler-update

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17863: further cleanup in our rails postinst scripts and documentation:
Ward Vandewege [Thu, 22 Jul 2021 13:02:27 +0000 (09:02 -0400)]
17863: further cleanup in our rails postinst scripts and documentation:

* replace `bundle exec` calls with the appropriate binstub
* update deprecated use of `bundle install --path` to avoid warning

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17962-bump-lib-pq'
Ward Vandewege [Wed, 21 Jul 2021 12:46:46 +0000 (08:46 -0400)]
Merge branch '17962-bump-lib-pq'

closes #17962

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17962: Merge branch 'main' into 17962-bump-lib-pq
Ward Vandewege [Wed, 21 Jul 2021 12:46:12 +0000 (08:46 -0400)]
17962: Merge branch 'main' into 17962-bump-lib-pq

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17813: Change default value of runtime_auth_scopes to []
Peter Amstutz [Tue, 20 Jul 2021 21:01:24 +0000 (17:01 -0400)]
17813: Change default value of runtime_auth_scopes to []

It is documented to be an array of strings, so having the default be
an object just seems to be a mistake.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17813: Fix tests
Peter Amstutz [Tue, 20 Jul 2021 21:01:08 +0000 (17:01 -0400)]
17813: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMake singularity suid in arvbox, crunch-dispatch-local runs as user
Peter Amstutz [Tue, 20 Jul 2021 20:28:30 +0000 (16:28 -0400)]
Make singularity suid in arvbox, crunch-dispatch-local runs as user

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17962: Bump the version of lib/pq to 1.10.2
Ward Vandewege [Tue, 20 Jul 2021 19:25:05 +0000 (15:25 -0400)]
17962: Bump the version of lib/pq to 1.10.2

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAnother doc tweak.
Ward Vandewege [Tue, 20 Jul 2021 17:08:46 +0000 (13:08 -0400)]
Another doc tweak.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoAdd some more clarifications to the doc page about restricting uploads
Ward Vandewege [Tue, 20 Jul 2021 15:53:27 +0000 (11:53 -0400)]
Add some more clarifications to the doc page about restricting uploads
and downloads.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17813: Handle setting $HOME as a special case
Peter Amstutz [Tue, 20 Jul 2021 15:15:44 +0000 (11:15 -0400)]
17813: Handle setting $HOME as a special case

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17748-extra-claims-take2'
Nico Cesar [Tue, 20 Jul 2021 14:07:09 +0000 (10:07 -0400)]
Merge branch '17748-extra-claims-take2'

closes #17748

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curii.com>

3 years agoFix typos in the documentation.
Ward Vandewege [Tue, 20 Jul 2021 12:47:52 +0000 (08:47 -0400)]
Fix typos in the documentation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoOIDC given name / family name fields support
Nico Cesar [Mon, 19 Jul 2021 22:03:38 +0000 (18:03 -0400)]
OIDC given name / family name fields support

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@nicocesar.com>

3 years ago17813: Upload .sif file to cache project & read it from keep
Peter Amstutz [Mon, 19 Jul 2021 20:13:33 +0000 (16:13 -0400)]
17813: Upload .sif file to cache project & read it from keep

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17863: update to using bundler 2.2.19 everywhere.
Ward Vandewege [Mon, 19 Jul 2021 15:17:46 +0000 (11:17 -0400)]
17863: update to using bundler 2.2.19 everywhere.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix arvbox demo image build.
Ward Vandewege [Mon, 19 Jul 2021 14:25:56 +0000 (10:25 -0400)]
Fix arvbox demo image build.

refs #17816

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17813: Singularity image caching wip
Peter Amstutz [Fri, 16 Jul 2021 21:26:31 +0000 (17:26 -0400)]
17813: Singularity image caching wip

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17756: Review feedback.
Tom Clegg [Fri, 16 Jul 2021 20:42:46 +0000 (16:42 -0400)]
17756: Review feedback.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17573-export-storage-classes' into main. Refs #17573
Lucas Di Pentima [Fri, 16 Jul 2021 19:03:20 +0000 (16:03 -0300)]
Merge branch '17573-export-storage-classes' into main. Refs #17573

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17816-singularity-cwd' into main refs #17816
Peter Amstutz [Fri, 16 Jul 2021 15:27:06 +0000 (11:27 -0400)]
Merge branch '17816-singularity-cwd' into main refs #17816

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17394-collectionfs-storage-class' into main
Tom Clegg [Fri, 16 Jul 2021 15:23:31 +0000 (11:23 -0400)]
Merge branch '17394-collectionfs-storage-class' into main

closes #17394

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17816: Sort bind mounts on singularity command line
Peter Amstutz [Fri, 16 Jul 2021 15:00:57 +0000 (11:00 -0400)]
17816: Sort bind mounts on singularity command line

Need to mount parent directories before we can mount things inside
them.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: Add test.
Tom Clegg [Fri, 16 Jul 2021 14:44:57 +0000 (10:44 -0400)]
17816: Add test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17816: Set the current working directory in the singularity container
Peter Amstutz [Thu, 15 Jul 2021 17:50:22 +0000 (13:50 -0400)]
17816: Set the current working directory in the singularity container

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: crunch-dispatch-local uses cluster config.
Peter Amstutz [Thu, 15 Jul 2021 19:58:45 +0000 (15:58 -0400)]
17816: crunch-dispatch-local uses cluster config.

Added upgrade note.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: Fix tests
Peter Amstutz [Wed, 14 Jul 2021 20:47:45 +0000 (16:47 -0400)]
17816: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: Make singularity-in-arvbox work
Peter Amstutz [Wed, 14 Jul 2021 20:39:02 +0000 (16:39 -0400)]
17816: Make singularity-in-arvbox work

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: Add --runtime-engine to crunch-dispatch-local and crunch-dispatch-slurm
Peter Amstutz [Wed, 14 Jul 2021 20:37:19 +0000 (16:37 -0400)]
17816: Add --runtime-engine to crunch-dispatch-local and crunch-dispatch-slurm

refs #17816

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoDon't pull arvbox checkouts after clone
Peter Amstutz [Wed, 14 Jul 2021 18:50:07 +0000 (14:50 -0400)]
Don't pull arvbox checkouts after clone

We already have latest and tags/detached HEADS can't be pulled anyway
(and they fail, which is the real problem).

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoarvbox docs include a docker image tag
Peter Amstutz [Wed, 14 Jul 2021 18:12:08 +0000 (14:12 -0400)]
arvbox docs include a docker image tag

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd ARVADOS_BRANCH and WORKBENCH2_BRANCH to arvbox
Peter Amstutz [Wed, 14 Jul 2021 17:57:37 +0000 (13:57 -0400)]
Add ARVADOS_BRANCH and WORKBENCH2_BRANCH to arvbox

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix "wrong bundler" error starting API server in arvbox
Peter Amstutz [Wed, 14 Jul 2021 15:19:58 +0000 (11:19 -0400)]
Fix "wrong bundler" error starting API server in arvbox

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17816: Close stdin/stdout/stderr from main instead of executor.
Tom Clegg [Fri, 16 Jul 2021 14:24:42 +0000 (10:24 -0400)]
17816: Close stdin/stdout/stderr from main instead of executor.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17394: Check by_id dir obeys storage classes.
Tom Clegg [Thu, 15 Jul 2021 14:13:39 +0000 (10:13 -0400)]
17394: Check by_id dir obeys storage classes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17394: Update callback name.
Tom Clegg [Thu, 15 Jul 2021 13:44:17 +0000 (09:44 -0400)]
17394: Update callback name.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17394: Use BlockWrite interface in crunch-run.
Tom Clegg [Thu, 15 Jul 2021 13:42:21 +0000 (09:42 -0400)]
17394: Use BlockWrite interface in crunch-run.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17394: Test BlockWrite storage classes precedence.
Tom Clegg [Thu, 15 Jul 2021 13:27:25 +0000 (09:27 -0400)]
17394: Test BlockWrite storage classes precedence.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17573: Adds test storage classes to arvados-server test mode.
Lucas Di Pentima [Thu, 15 Jul 2021 01:22:37 +0000 (22:22 -0300)]
17573: Adds test storage classes to arvados-server test mode.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17573: Exposes storage classes for every configured volume.
Lucas Di Pentima [Thu, 8 Jul 2021 19:31:27 +0000 (16:31 -0300)]
17573: Exposes storage classes for every configured volume.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17394: Propagate storage classes when writing data via collectionfs.
Tom Clegg [Wed, 14 Jul 2021 20:52:12 +0000 (16:52 -0400)]
17394: Propagate storage classes when writing data via collectionfs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoDon't pull arvbox checkouts after clone
Peter Amstutz [Wed, 14 Jul 2021 18:50:07 +0000 (14:50 -0400)]
Don't pull arvbox checkouts after clone

We already have latest and tags/detached HEADS can't be pulled anyway
(and they fail, which is the real problem).

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoarvbox docs include a docker image tag
Peter Amstutz [Wed, 14 Jul 2021 18:12:08 +0000 (14:12 -0400)]
arvbox docs include a docker image tag

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd ARVADOS_BRANCH and WORKBENCH2_BRANCH to arvbox
Peter Amstutz [Wed, 14 Jul 2021 17:57:37 +0000 (13:57 -0400)]
Add ARVADOS_BRANCH and WORKBENCH2_BRANCH to arvbox

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoFix "wrong bundler" error starting API server in arvbox
Peter Amstutz [Wed, 14 Jul 2021 15:19:58 +0000 (11:19 -0400)]
Fix "wrong bundler" error starting API server in arvbox

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoarvbox uses react-scripts, refs #17782
Peter Amstutz [Tue, 13 Jul 2021 21:11:57 +0000 (17:11 -0400)]
arvbox uses react-scripts, refs #17782

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17913-addressable-upgrade' into main. Closes #17913
Lucas Di Pentima [Tue, 13 Jul 2021 19:55:30 +0000 (16:55 -0300)]
Merge branch '17913-addressable-upgrade' into main. Closes #17913

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17913: Upgrades 'addressable' gem to 2.8.0
Lucas Di Pentima [Tue, 13 Jul 2021 19:14:49 +0000 (16:14 -0300)]
17913: Upgrades 'addressable' gem to 2.8.0

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoMerge branch '17782-nodejs-update' into main
Lucas Di Pentima [Mon, 12 Jul 2021 15:14:28 +0000 (12:14 -0300)]
Merge branch '17782-nodejs-update' into main
Refs #17782

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years agoPin networkx to avoid new version that pulls in scipy/numpy
Peter Amstutz [Fri, 9 Jul 2021 14:54:30 +0000 (10:54 -0400)]
Pin networkx to avoid new version that pulls in scipy/numpy

refs #17879

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoAdd missing link to "restricting-upload-download" to doc refs #17464
Peter Amstutz [Thu, 8 Jul 2021 21:02:36 +0000 (17:02 -0400)]
Add missing link to "restricting-upload-download" to doc refs #17464

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17879-update-schema-salad' into main refs #17879
Peter Amstutz [Thu, 8 Jul 2021 19:24:55 +0000 (15:24 -0400)]
Merge branch '17879-update-schema-salad' into main refs #17879

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17879: Add test that acr doesn't fail on sbg fields
Peter Amstutz [Thu, 8 Jul 2021 19:20:51 +0000 (15:20 -0400)]
17879: Add test that acr doesn't fail on sbg fields

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17879: Default --thread-count=0, upgrade schema-salad
Peter Amstutz [Thu, 8 Jul 2021 18:36:00 +0000 (14:36 -0400)]
17879: Default --thread-count=0, upgrade schema-salad

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoPut the default arvados-cwl-runner --thread-count back down to 1.
Peter Amstutz [Wed, 7 Jul 2021 20:23:27 +0000 (16:23 -0400)]
Put the default arvados-cwl-runner --thread-count back down to 1.

On account of reports of workflows deadlocking.

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17858-cwl-exprtool-defaults' into main refs #17858
Peter Amstutz [Wed, 7 Jul 2021 20:18:58 +0000 (16:18 -0400)]
Merge branch '17858-cwl-exprtool-defaults' into main refs #17858

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Tweak logic to avoid KeyError
Peter Amstutz [Wed, 7 Jul 2021 20:11:54 +0000 (16:11 -0400)]
17858: Tweak logic to avoid KeyError

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17705-adc-arch-doc'
Tom Clegg [Wed, 7 Jul 2021 19:10:13 +0000 (15:10 -0400)]
Merge branch '17705-adc-arch-doc'

closes #17705

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17782: Updates nodejs to v12, as required by workbench2.
Lucas Di Pentima [Wed, 7 Jul 2021 18:50:38 +0000 (15:50 -0300)]
17782: Updates nodejs to v12, as required by workbench2.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas.dipentima@curii.com>

3 years ago17858: Add ExpressionTool to classes checked for ids when visiting
Peter Amstutz [Wed, 7 Jul 2021 14:02:05 +0000 (10:02 -0400)]
17858: Add ExpressionTool to classes checked for ids when visiting

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Integration test passes
Peter Amstutz [Wed, 7 Jul 2021 17:42:13 +0000 (13:42 -0400)]
17858: Integration test passes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years ago17858: Add test to reproduce bug
Peter Amstutz [Wed, 7 Jul 2021 13:51:09 +0000 (09:51 -0400)]
17858: Add test to reproduce bug

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>

3 years agoMerge branch '17841-add-duration'
Ward Vandewege [Tue, 6 Jul 2021 21:37:16 +0000 (17:37 -0400)]
Merge branch '17841-add-duration'

closes #17841

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17841: Merge branch 'main' into 17841-add-duration
Ward Vandewege [Tue, 6 Jul 2021 21:36:43 +0000 (17:36 -0400)]
17841: Merge branch 'main' into 17841-add-duration

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17841: update documentation
Ward Vandewege [Tue, 6 Jul 2021 21:36:13 +0000 (17:36 -0400)]
17841: update documentation

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago17756: Merge branch 'main'
Tom Clegg [Tue, 6 Jul 2021 13:28:47 +0000 (09:28 -0400)]
17756: Merge branch 'main'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoMerge branch '17853-fix-write-with-rlock' into main
Tom Clegg [Tue, 6 Jul 2021 13:11:01 +0000 (09:11 -0400)]
Merge branch '17853-fix-write-with-rlock' into main

fixes #17853

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Add arvados-dispatch-lsf install doc page.
Tom Clegg [Mon, 5 Jul 2021 20:56:45 +0000 (16:56 -0400)]
17756: Add arvados-dispatch-lsf install doc page.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17705: Improve InstanceSetID wording, update link to mgmt API docs.
Tom Clegg [Mon, 5 Jul 2021 18:46:03 +0000 (14:46 -0400)]
17705: Improve InstanceSetID wording, update link to mgmt API docs.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17853: Fix map write when only RLock held.
Tom Clegg [Mon, 5 Jul 2021 18:32:39 +0000 (14:32 -0400)]
17853: Fix map write when only RLock held.

Update DebugLocksPanicMode to check RLock/Lock as appropriate.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Fix tests to accommodate sudo and added bjobs args.
Tom Clegg [Mon, 5 Jul 2021 14:10:11 +0000 (10:10 -0400)]
17756: Fix tests to accommodate sudo and added bjobs args.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Add DispatchLSF to health aggregator test setup.
Tom Clegg [Mon, 5 Jul 2021 13:59:36 +0000 (09:59 -0400)]
17756: Add DispatchLSF to health aggregator test setup.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Build arvados-dispatch-lsf package.
Tom Clegg [Sun, 4 Jul 2021 17:34:16 +0000 (13:34 -0400)]
17756: Build arvados-dispatch-lsf package.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Add checkLsfQueueForOrphans.
Tom Clegg [Fri, 2 Jul 2021 21:08:08 +0000 (17:08 -0400)]
17756: Add checkLsfQueueForOrphans.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Rewrite lsfqueue "wait for next update".
Tom Clegg [Fri, 2 Jul 2021 21:07:27 +0000 (17:07 -0400)]
17756: Rewrite lsfqueue "wait for next update".

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years ago17756: Add lsf dispatcher.
Tom Clegg [Fri, 2 Jul 2021 17:27:46 +0000 (13:27 -0400)]
17756: Add lsf dispatcher.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>

3 years agoFix api server tests.
Ward Vandewege [Fri, 2 Jul 2021 01:24:45 +0000 (21:24 -0400)]
Fix api server tests.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix keepproxy tests.
Ward Vandewege [Thu, 1 Jul 2021 21:36:09 +0000 (17:36 -0400)]
Fix keepproxy tests.

refs #17464

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoFix arv-git-httpd tests.
Ward Vandewege [Thu, 1 Jul 2021 21:17:10 +0000 (17:17 -0400)]
Fix arv-git-httpd tests.

refs #17817

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch '17395-container-output-storage-class' into main
Peter Amstutz [Thu, 1 Jul 2021 20:13:58 +0000 (16:13 -0400)]
Merge branch '17395-container-output-storage-class' into main

refs #17395

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>