arvados-k8s.git
3 years ago18278: Postgresql: fix mount point for persistent volume, bump Postgres to 18278-GKE-fixes
Ward Vandewege [Tue, 19 Oct 2021 16:41:18 +0000 (12:41 -0400)]
18278: Postgresql: fix mount point for persistent volume, bump Postgres to
version 10.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years ago18278: Stop specifying a k8s version on GKE.
Ward Vandewege [Tue, 19 Oct 2021 15:23:31 +0000 (11:23 -0400)]
18278: Stop specifying a k8s version on GKE.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump Arvados to version 2.2.1.
Ward Vandewege [Sat, 17 Jul 2021 13:14:44 +0000 (09:14 -0400)]
Bump Arvados to version 2.2.1.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoBump Arvados to 2.2.0. Bump the rails-runtime image to Ubuntu 20.04
Ward Vandewege [Mon, 12 Jul 2021 16:57:17 +0000 (12:57 -0400)]
Bump Arvados to 2.2.0. Bump the rails-runtime image to Ubuntu 20.04
(Focal Fossa) because the upstream changed. Fix a WebDAV configuration
bug that manifested as workbench2 not being able to show the contents of
a collection. Remove a deprecated config file entry.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoWhen starting a Minikube cluster, the keepstores should be able to reuse
Ward Vandewege [Mon, 12 Jul 2021 13:58:54 +0000 (09:58 -0400)]
When starting a Minikube cluster, the keepstores should be able to reuse
an existing persistent volume (i.e. not error out if the /keepdata
directory already exists).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoPin the shell image to Ubuntu 18.04.
Ward Vandewege [Wed, 23 Dec 2020 14:41:18 +0000 (09:41 -0500)]
Pin the shell image to Ubuntu 18.04.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoThe Arvados apt repositories have been reorganized.
Ward Vandewege [Wed, 23 Dec 2020 14:00:07 +0000 (09:00 -0500)]
The Arvados apt repositories have been reorganized.

refs #17218

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch 'mluds-master'
Ward Vandewege [Mon, 30 Nov 2020 15:40:36 +0000 (10:40 -0500)]
Merge branch 'mluds-master'

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoMerge branch 'master' of https://github.com/mluds/arvados-k8s into mluds-master
Ward Vandewege [Mon, 30 Nov 2020 15:15:44 +0000 (10:15 -0500)]
Merge branch 'master' of https://github.com/mluds/arvados-k8s into mluds-master

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

3 years agoconvert postgres to statefulset for persistence - credit ward@curii.com
Mike Ludwig [Thu, 26 Nov 2020 00:16:58 +0000 (19:16 -0500)]
convert postgres to statefulset for persistence - credit ward@curii.com
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

3 years agosupport persistent volumes for postgres and the keep store
Mike Ludwig [Thu, 26 Nov 2020 00:06:41 +0000 (19:06 -0500)]
support persistent volumes for postgres and the keep store
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agoUpgrade to Arvados 2.1.1
Ward Vandewege [Wed, 18 Nov 2020 18:35:17 +0000 (13:35 -0500)]
Upgrade to Arvados 2.1.1

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agosupport alternate namespaces for slurm dns
Mike Ludwig [Thu, 29 Oct 2020 18:51:54 +0000 (14:51 -0400)]
support alternate namespaces for slurm dns
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agosupport internal IP address range configuration for api-server
Mike Ludwig [Tue, 27 Oct 2020 20:36:51 +0000 (16:36 -0400)]
support internal IP address range configuration for api-server
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agoUpgrade the helm charts to Arvados 2.1.0.
Ward Vandewege [Fri, 6 Nov 2020 18:22:22 +0000 (13:22 -0500)]
Upgrade the helm charts to Arvados 2.1.0.

This removes the (deprecated) SSO server. The minimum Arvados version
that will work with the chart is now 2.1.0.

closes #17095

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoA few fixes:
Ward Vandewege [Thu, 5 Nov 2020 16:23:40 +0000 (11:23 -0500)]
A few fixes:

* our api and workbench packages now depend on arvados-server, handle
  that correctly in the rails Dockerfile, bootstrap.sh and the
  respective helm deployment charts.
* make the apt-get install call in bootstrap.sh a bit more resilient,
  allow downgrading.
* the Makefile commands that push the base docker images now only push
  the latest tag

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch 'mluds-master' into master
Ward Vandewege [Tue, 27 Oct 2020 15:06:14 +0000 (11:06 -0400)]
Merge branch 'mluds-master' into master

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch 'master' of https://github.com/mluds/arvados-k8s into mluds-master
Ward Vandewege [Tue, 27 Oct 2020 15:05:02 +0000 (11:05 -0400)]
Merge branch 'master' of https://github.com/mluds/arvados-k8s into mluds-master

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoadd custom CA bundle to workbench deployment
Mike Ludwig [Thu, 22 Oct 2020 18:52:54 +0000 (14:52 -0400)]
add custom CA bundle to workbench deployment
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agosupport annotations on loadbalancer services for metallb
Mike Ludwig [Tue, 20 Oct 2020 22:54:26 +0000 (18:54 -0400)]
support annotations on loadbalancer services for metallb
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agosupport custom CA bundle for the api-server
Mike Ludwig [Thu, 22 Oct 2020 14:25:57 +0000 (10:25 -0400)]
support custom CA bundle for the api-server
Arvados-DCO-1.1-Signed-off-by: Michael Ludwig <michael.david.ludwig@gmail.com>

4 years agodocumentation: README.md: add a reference to the Coding Standards page,
Ward Vandewege [Wed, 21 Oct 2020 18:11:42 +0000 (14:11 -0400)]
documentation: README.md: add a reference to the Coding Standards page,
specifically the 'Copyright headers' section.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoFix build status link in README.md
Ward Vandewege [Tue, 20 Oct 2020 14:57:41 +0000 (10:57 -0400)]
Fix build status link in README.md

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curii.com>

4 years agoMerge branch '16430-add-jenkins-integration-test-minikube'
Ward Vandewege [Wed, 27 May 2020 14:04:23 +0000 (10:04 -0400)]
Merge branch '16430-add-jenkins-integration-test-minikube'

closes #16429
closes #16430

4 years ago16430: test feedback: add timeout to loop that waits for a healthy 16430-add-jenkins-integration-test-minikube
Ward Vandewege [Wed, 27 May 2020 14:03:43 +0000 (10:03 -0400)]
16430: test feedback: add timeout to loop that waits for a healthy
       Arvados cluster.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16430: Refactor to share code between the GKE and Minikube tests.
Ward Vandewege [Tue, 26 May 2020 21:53:03 +0000 (17:53 -0400)]
16430: Refactor to share code between the GKE and Minikube tests.

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16430: add Minikube integration test
Ward Vandewege [Tue, 26 May 2020 16:10:34 +0000 (12:10 -0400)]
16430: add Minikube integration test

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago16429: add GKE integration test 16429-add-jenkins-integration-test-GKE
Ward Vandewege [Wed, 13 May 2020 20:45:35 +0000 (16:45 -0400)]
16429: add GKE integration test

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoAdd workbench2, and make it the default.
Ward Vandewege [Thu, 30 Apr 2020 19:25:40 +0000 (15:25 -0400)]
Add workbench2, and make it the default.

refs #16384

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoBugfixes: fix arvados-slurm-controller-0 container start on minikube,
Ward Vandewege [Thu, 30 Apr 2020 15:57:49 +0000 (11:57 -0400)]
Bugfixes: fix arvados-slurm-controller-0 container start on minikube,
fix arvados-slurm-controller StatefulSet warning.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoWork around silly limitation of 6 search domains in /etc/resolv.conf.
Ward Vandewege [Tue, 28 Apr 2020 17:36:31 +0000 (13:36 -0400)]
Work around silly limitation of 6 search domains in /etc/resolv.conf.
This limitation was removed from the GNU C library release 2.26 in 2017
but Kubernetes still enforces it. Kubernetes uses 3 search domains. This
is no problem on Minikube, but on GKE the other 3 slots are populated by
GCP.

We can actually edit resolv.conf, but only in place, which is why we
fall back on `ed`...

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years ago* support Minikube
Ward Vandewege [Sun, 26 Apr 2020 19:39:59 +0000 (15:39 -0400)]
* support Minikube
* support K8s 1.18
* upgrade to Helm 3
* upgrade to latest Arvados release (2.0.2)
* streamline Helm charts

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoRevert "Switch to phusion/passenger-ruby25 for the base of the rails-runtime image."
Ward Vandewege [Sat, 11 Apr 2020 15:37:40 +0000 (11:37 -0400)]
Revert "Switch to phusion/passenger-ruby25 for the base of the rails-runtime image."

The SSO server doesn't support Ruby 2.5, so let's go back to 2.3.

This reverts commit 6216ebc8a3e82243ee4b9d22c73169da4f66af0c.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoPass validation.
Ward Vandewege [Fri, 10 Apr 2020 14:05:35 +0000 (10:05 -0400)]
Pass validation.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

4 years agoBring community information up to date (again).
Ward Vandewege [Tue, 7 Apr 2020 20:23:48 +0000 (16:23 -0400)]
Bring community information up to date (again).

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@jhvc.com>

5 years agoAdd the ca-certificates package to the arvados-runtime image.
Ward Vandewege [Wed, 28 Aug 2019 18:14:31 +0000 (14:14 -0400)]
Add the ca-certificates package to the arvados-runtime image.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoSwitch to phusion/passenger-ruby25 for the base of the rails-runtime image.
Ward Vandewege [Wed, 28 Aug 2019 17:52:51 +0000 (13:52 -0400)]
Switch to phusion/passenger-ruby25 for the base of the rails-runtime image.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBump everything up to the latest stable release (1.3.1), and use Ubuntu 18.04
Ward Vandewege [Wed, 27 Feb 2019 22:31:09 +0000 (17:31 -0500)]
Bump everything up to the latest stable release (1.3.1), and use Ubuntu 18.04
as the operating system.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBring community information up to date.
Ward Vandewege [Fri, 25 Jan 2019 21:10:06 +0000 (16:10 -0500)]
Bring community information up to date.

refs #14617

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBump the Arvados package version to 1.3.0, which is the latest stable release.
Ward Vandewege [Fri, 25 Jan 2019 21:06:12 +0000 (16:06 -0500)]
Bump the Arvados package version to 1.3.0, which is the latest stable release.

refs #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoExpose the cluster identifier in values.yaml
Ward Vandewege [Mon, 25 Jun 2018 20:37:15 +0000 (16:37 -0400)]
Expose the cluster identifier in values.yaml

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoExpose the number of compute nodes in values.yaml
Ward Vandewege [Mon, 25 Jun 2018 20:18:06 +0000 (16:18 -0400)]
Expose the number of compute nodes in values.yaml

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAdd README.md
Ward Vandewege [Mon, 25 Jun 2018 18:53:43 +0000 (14:53 -0400)]
Add README.md

refs #13650

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoSome further cleanup.
Ward Vandewege [Mon, 25 Jun 2018 17:56:50 +0000 (13:56 -0400)]
Some further cleanup.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoFix section numbering in README.md
Ward Vandewege [Fri, 15 Jun 2018 14:18:28 +0000 (10:18 -0400)]
Fix section numbering in README.md

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoUpdates to README.md
Ward Vandewege [Tue, 12 Jun 2018 20:35:23 +0000 (16:35 -0400)]
Updates to README.md

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoUpdate Arvados package versions to latest.
Ward Vandewege [Tue, 12 Jun 2018 20:04:32 +0000 (16:04 -0400)]
Update Arvados package versions to latest.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoAlways pull latest version of the docker images.
Ward Vandewege [Tue, 12 Jun 2018 20:02:09 +0000 (16:02 -0400)]
Always pull latest version of the docker images.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMake the default e-mail and password for the initial cluster user a config
Ward Vandewege [Tue, 12 Jun 2018 20:01:31 +0000 (16:01 -0400)]
Make the default e-mail and password for the initial cluster user a config
option in values.yaml

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoNo more need for apps/v1beta2, we can now use apps/v1
Ward Vandewege [Tue, 5 Jun 2018 21:43:38 +0000 (17:43 -0400)]
No more need for apps/v1beta2, we can now use apps/v1

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMake the StatefulSets start and stop faster.
Ward Vandewege [Tue, 5 Jun 2018 21:38:27 +0000 (17:38 -0400)]
Make the StatefulSets start and stop faster.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoUpdate the nginx config for the keep-proxy-https pod.
Ward Vandewege [Tue, 5 Jun 2018 21:37:14 +0000 (17:37 -0400)]
Update the nginx config for the keep-proxy-https pod.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoFix the websockets service.
Ward Vandewege [Tue, 5 Jun 2018 21:33:28 +0000 (17:33 -0400)]
Fix the websockets service.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoDisable the old jobs api.
Ward Vandewege [Tue, 5 Jun 2018 21:32:36 +0000 (17:32 -0400)]
Disable the old jobs api.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years ago* Add a basic Slurm config
Ward Vandewege [Tue, 5 Jun 2018 17:55:31 +0000 (13:55 -0400)]
* Add a basic Slurm config
* Add crunch-dispatch-slurm

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoTweak the ssl.conf generated by cert-gen.sh, so that pycurl doesn't trip itself
Ward Vandewege [Wed, 30 May 2018 20:37:23 +0000 (16:37 -0400)]
Tweak the ssl.conf generated by cert-gen.sh, so that pycurl doesn't trip itself
up with the error message

  gnutls_handshake() failed: Key usage violation in certificate has been detected.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMake sure that the external source IP address of requests is passed trough to
Ward Vandewege [Wed, 30 May 2018 18:58:17 +0000 (14:58 -0400)]
Make sure that the external source IP address of requests is passed trough to
the API server. We rely on this to serve the correct list of keep services to
clients, depending on where they are.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Make sure the shell node has a token to create the keepService objects.
Ward Vandewege [Fri, 25 May 2018 21:30:33 +0000 (17:30 -0400)]
helm: Make sure the shell node has a token to create the keepService objects.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Add the dockerfiles used by the helm charts.
Ward Vandewege [Fri, 25 May 2018 17:32:41 +0000 (13:32 -0400)]
helm: Add the dockerfiles used by the helm charts.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: reorganize repository a bit.
Ward Vandewege [Fri, 25 May 2018 16:06:21 +0000 (12:06 -0400)]
helm: reorganize repository a bit.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Convert the SSO server to be served over SSL.
Ward Vandewege [Tue, 22 May 2018 01:39:48 +0000 (21:39 -0400)]
helm: Convert the SSO server to be served over SSL.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Convert the anonymous and superuser token to values in values.yaml
Ward Vandewege [Mon, 21 May 2018 21:16:59 +0000 (17:16 -0400)]
helm: Convert the anonymous and superuser token to values in values.yaml

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Make the package versions part of the values.yaml file.
Ward Vandewege [Mon, 21 May 2018 19:08:06 +0000 (15:08 -0400)]
helm: Make the package versions part of the values.yaml file.
helm: Make the external IP part of the values.yaml file.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agohelm: Add a Helm chart for installing Arvados
Kevin Lin [Tue, 24 Apr 2018 18:07:13 +0000 (11:07 -0700)]
helm: Add a Helm chart for installing Arvados

This commit adds a Helm chart for installing most of Arvados,
except for the Git server and SLURM.

No issue #

Arvados-DCO-1.1-Signed-off-by: Kevin Lin <kevin@kelda.io>

6 years agoInitial commit: add license files.
Ward Vandewege [Fri, 25 May 2018 15:14:36 +0000 (11:14 -0400)]
Initial commit: add license files.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>