- Manual installation:
- install/install-manual-prerequisites.html.textile.liquid
- install/packages.html.textile.liquid
- - admin/upgrading.html.textile.liquid
- Configuration:
- install/config.html.textile.liquid
- - admin/config-migration.html.textile.liquid
- - admin/config.html.textile.liquid
- admin/config-urls.html.textile.liquid
+ - admin/config.html.textile.liquid
+ - Maintenance and upgrading:
+ - admin/upgrading.html.textile.liquid
+ - admin/maintenance-and-upgrading.html.textile.liquid
- Core:
- install/install-api-server.html.textile.liquid
- Keep:
+++ /dev/null
----
-layout: default
-navsection: installguide
-title: Migrating Configuration from v1.4 to v2.0
-...
-
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-{% include 'notebox_begin_warning' %}
-_New installations of Arvados 2.0+ can skip this section_
-{% include 'notebox_end' %}
-
-Arvados 2.0 migrates to a centralized configuration file for all components. The centralized Arvados configuration is @/etc/arvados/config.yml@. Components that support the new centralized configuration are listed below. During the migration period, legacy configuration files are still loaded and take precedence over the centralized configuration file.
-
-h2. API server
-
-The legacy API server configuration is stored in @config/application.yml@ and @config/database.yml@. After migration to @/etc/arvados/config.yml@, both of these files should be moved out of the way and/or deleted.
-
-Change to the API server directory and use the following commands:
-
-<pre>
-$ RAILS_ENV=production bin/rake config:migrate > config.yml
-$ cp config.yml /etc/arvados/config.yml
-</pre>
-
-This will print the contents of @config.yml@ after merging the legacy @application.yml@ and @database.yml@ into the existing systemwide @config.yml@. It may be redirected to a file and copied to @/etc/arvados/config.yml@ (it is safe to copy over, all configuration items from the existing @/etc/arvados/config.yml@ will be included in the migrated output).
-
-If you wish to update @config.yml@ configuration by hand, or check that everything has been migrated, use @config:diff@ to print configuration items that differ between @application.yml@ and the system @config.yml@.
-
-<pre>
-$ RAILS_ENV=production bin/rake config:diff
-</pre>
-
-This command will also report if no migrations are required.
-
-h2. Workbench
-
-The legacy workbench configuration is stored in @config/application.yml@. After migration to @/etc/arvados/config.yml@, this file should be moved out of the way and/or deleted.
-
-Change to the workbench server directory and use the following commands:
-
-<pre>
-$ RAILS_ENV=production bin/rake config:migrate > config.yml
-$ cp config.yml /etc/arvados/config.yml
-</pre>
-
-This will print the contents of @config.yml@ after merging the legacy @application.yml@ into the existing systemwide @config.yml@. It may be redirected to a file and copied to @/etc/arvados/config.yml@ (it is safe to copy over, all configuration items from the existing @/etc/arvados/config.yml@ will be included in the migrated output).
-
-If you wish to update @config.yml@ configuration by hand, or check that everything has been migrated, use @config:diff@ to print configuration items that differ between @application.yml@ and the system @config.yml@.
-
-<pre>
-$ RAILS_ENV=production bin/rake config:diff
-</pre>
-
-This command will also report if no migrations are required.
-
-h2. keepstore, keep-web, crunch-dispatch-slurm, arvados-ws, keepproxy, arv-git-httpd, keep-balance
-
-The legacy config for each component (loaded from @/etc/arvados/component/component.yml@ or a different location specified via the -legacy-component-config command line argument) takes precedence over the centralized config. After you migrate everything from the legacy config to the centralized config, you should delete @/etc/arvados/component/component.yml@ and/or stop using the corresponding -legacy-component-config argument.
-
-To migrate a component configuration, do this on each node that runs an Arvados service:
-
-# Ensure that the latest @config.yml@ is installed on the current node
-# Install @arvados-server@ using @apt-get@ or @yum@.
-# Run @arvados-server config-check@, review and apply the recommended changes to @/etc/arvados/config.yml@
-# After applying changes, re-run @arvados-server config-check@ again to check for additional warnings and recommendations.
-# When you are satisfied, delete the legacy config file, restart the service, and check its startup logs.
-# Copy the updated @config.yml@ file to your next node, and repeat the process there.
-# When you have a @config.yml@ file that includes all volumes on all keepstores, it is important to add a 'Rendezvous' parameter to the InternalURLs entries to make sure the old volume identifiers line up with the new config. If you don't do this, @keep-balance@ will want to shuffle all the existing data around to match the new volume order. The 'Rendezvous' value should be the last 15 characters of the keepstore's UUID in the old configuration. Here's an example:
-
-<notextile>
-<pre><code>Clusters:
- xxxxx:
- Services:
- Keepstore:
- InternalURLs:
- "http://keep1.xxxxx.arvadosapi.com:25107": {Rendezvous: "eim6eefaibesh3i"}
- "http://keep2.xxxxx.arvadosapi.com:25107": {Rendezvous: "yequoodalai7ahg"}
- "http://keep3.xxxxx.arvadosapi.com:25107": {Rendezvous: "eipheho6re1shou"}
- "http://keep4.xxxxx.arvadosapi.com:25107": {Rendezvous: "ahk7chahthae3oo"}
-</code></pre>
-</notextile>
-
-In this example, the keepstore with the name `keep1` had the uuid `xxxxx-bi6l4-eim6eefaibesh3i` in the old configuration.
-
-After migrating and removing all legacy config files, make sure the @/etc/arvados/config.yml@ file is identical across all system nodes -- API server, keepstore, etc. -- and restart all services to make sure they are using the latest configuration.
-
-h2. Cloud installations only: node manager
-
-Node manager is deprecated and replaced by @arvados-dispatch-cloud@. No automated config migration is available. Follow the instructions to "install the cloud dispatcher":../install/crunch2-cloud/install-dispatch-cloud.html
-
-*Only one dispatch process should be running at a time.* If you are migrating a system that currently runs Node manager and @crunch-dispatch-slurm@, it is safest to remove the @crunch-dispatch-slurm@ service entirely before installing @arvados-dispatch-cloud@.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo systemctl --now disable crunch-dispatch-slurm</span>
-~$ <span class="userinput">sudo apt-get remove crunch-dispatch-slurm</span>
-</code></pre>
-</notextile>
-
-h2. arvados-controller, arvados-dispatch-cloud
-
-Already uses centralized config exclusively. No migration needed.
The Arvados configuration is stored at @/etc/arvados/config.yml@
-See "Migrating Configuration":config-migration.html for information about migrating from legacy component-specific configuration files.
-
{% codeblock as yaml %}
{% include 'config_default_yml' %}
{% endcodeblock %}
--- /dev/null
+---
+layout: default
+navsection: installguide
+title: Maintenance and upgrading
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+# "Commercial support":#commercial_support
+# "Maintaining Arvados":#maintaining
+## "Modification of the config.yml file":#configuration
+## "Distributing the configuration file":#distribution
+## "Restart the services affected by the change":#restart
+# "Upgrading Arvados":#upgrading
+
+h2(#commercial_support). Commercial support
+
+Arvados is "100% open source software":{{site.baseurl}}/user/copying/copying.html. Anyone can download, install, maintain and upgrade it. However, if this is not something you want to spend your time and energy doing, "Curii Corporation":https://curii.com provides managed Arvados installations as well as commercial support for Arvados. Please contact "info@curii.com":mailto:info@curii.com for more information.
+
+If you'd prefer to do things yourself, a few starting points for a maintaining and upgrading Arvados can be found below.
+
+h2(#maintaining). Maintaining Arvados
+
+After Arvados is installed, periodic configuration changes may be required to adapt the software to your needs. Arvados uses a unified configuration file, which is normally found at @/etc/arvados/config.yml@.
+
+Making a configuration change to Arvados typically involves three steps:
+
+* modification of the @config.yml@ file
+* distribution of the modified file to the machines in the cluster
+* restarting of the services affected by the change
+
+h3(#configchange). Modification of the @config.yml@ file
+
+Consult the "configuration reference":{{site.baseurl}}/admin/config.html or another part of the documentation to identify the change to be made.
+
+Preserve a copy of your existing configuration file as a backup, and make the desired modification.
+
+Run @arvados-server config-check@ to make sure the configuration file has no errors and no warnings.
+
+h3(#distribution). Distribute the configuration file
+
+We recommend to keep the @config.yml@ file in sync between all the Arvados system nodes, to avoid issues with services running on different versions of the configuration.
+
+Distribution of the configuration file can be done in many ways, e.g. scp, configuration management software, etc.
+
+h3(#restart). Restart the services affected by the change
+
+If you know which Arvados service uses the specific configuration that was modified, restart those services. When in doubt, restart all Arvados system services.
+
+h2(#upgrading). Upgrading Arvados
+
+Upgrading Arvados typically involves the following steps:
+
+# consult the "upgrade notes":{{site.baseurl}}/admin/upgrading.html and the "release notes":https://arvados.org/releases/ for the release you want to upgrade to
+# Wait for the cluster to be idle and stop Arvados services.
+# Make a backup of your database, as a precaution.
+# update the configuration file for the new release, if necessary (see "Maintaining Arvados":#maintaining above)
+# rebuild and deploy the "compute node image":{{site.baseurl}}/install/crunch2-cloud/install-compute-node.html (cloud only)
+# Install new packages using @apt-get upgrade@ or @yum upgrade@.
+# Wait for package installation scripts as they perform any necessary data migrations.
+# Verify that the Arvados services were restarted as part of the package upgrades.
---
layout: default
navsection: installguide
-title: "Upgrading Arvados and Release notes"
+title: "Arvados upgrade notes"
...
{% comment %}
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-For Arvados administrators, this page will cover what you need to know and do in order to ensure a smooth upgrade of your Arvados installation. For general release notes covering features added and bugs fixed, see "Arvados releases":https://arvados.org/releases .
+For Arvados administrators, this page will cover what you need to know and do in order to ensure a smooth upgrade of your Arvados installation. For general release notes covering features added and bugs fixed, see "Arvados releases":https://arvados.org/releases.
-h2. General process
-
-# Consult upgrade notes below to see if any manual configuration updates are necessary.
-# Wait for the cluster to be idle and stop Arvados services.
-# Make a backup of your database, as a precaution.
-# Install new packages using @apt-get upgrade@ or @yum upgrade@.
-# Wait for package installation scripts as they perform any necessary data migrations.
-# Restart Arvados services.
+Upgrade instructions can be found at "Maintenance and upgrading":{{site.baseurl}}/admin/maintenance-and-upgrading.html#upgrading.
h2. Upgrade notes
-Some versions introduce changes that require special attention when upgrading: e.g., there is a new service to install, or there is a change to the default configuration that you might need to override in order to preserve the old behavior.
+Some versions introduce changes that require special attention when upgrading: e.g., there is a new service to install, or there is a change to the default configuration that you might need to override in order to preserve the old behavior. These notes are listed below, organized by release version. Scroll down to the version number you are upgrading to.
{% comment %}
Note to developers: Add new items at the top. Include the date, issue number, commit, and considerations/instructions for those about to upgrade.
h2(#main). development main (as of 2022-02-10)
-"previous: Upgrading from 2.3.0":#v2_3_0
+"previous: Upgrading to 2.3.0":#v2_3_0
h3. Anonymous token changes
h3. Migrating to centralized config.yml
-See "Migrating Configuration":config-migration.html for notes on migrating legacy per-component configuration files to the new centralized @/etc/arvados/config.yml@.
+See "Migrating Configuration":https://doc.arvados.org/v2.1/admin/config-migration.html for notes on migrating legacy per-component configuration files to the new centralized @/etc/arvados/config.yml@.
To ensure a smooth transition, the per-component config files continue to be read, and take precedence over the centralized configuration. Your cluster should continue to function after upgrade but before doing the full configuration migration. However, several services (keepstore, keep-web, keepproxy) require a minimal `/etc/arvados/config.yml` to start:
You can no longer specify individual keep services to balance via the @config.KeepServiceList@ command line option or @KeepServiceList@ legacy config option. Instead, keep-balance will operate on all keepstore servers with @service_type:disk@ as reported by the @arv keep_service list@ command. If you are still using the legacy config, @KeepServiceList@ should be removed or keep-balance will produce an error.
-Please see the "config migration guide":{{site.baseurl}}/admin/config-migration.html and "keep-balance install guide":{{site.baseurl}}/install/install-keep-balance.html for more details.
+Please see the "config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html and "keep-balance install guide":{{site.baseurl}}/install/install-keep-balance.html for more details.
h3. Arv-git-httpd configuration migration
-(feature "#14712":https://dev.arvados.org/issues/14712 ) The arv-git-httpd package can now be configured using the centralized configuration file at @/etc/arvados/config.yml@. Configuration via individual command line arguments is no longer available. Please see "arv-git-httpd's config migration guide":{{site.baseurl}}/admin/config-migration.html#arv-git-httpd for more details.
+(feature "#14712":https://dev.arvados.org/issues/14712 ) The arv-git-httpd package can now be configured using the centralized configuration file at @/etc/arvados/config.yml@. Configuration via individual command line arguments is no longer available. Please see "arv-git-httpd's config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html#arv-git-httpd for more details.
h3. Keepstore and keep-web configuration migration
keep-web now supports the legacy @keep-web.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check "keep-web's install guide":{{site.baseurl}}/install/install-keep-web.html for more details.
-keepstore now supports the legacy @keepstore.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check the "keepstore config migration notes":{{site.baseurl}}/admin/config-migration.html#keepstore and "keepstore install guide":{{site.baseurl}}/install/install-keepstore.html for more details.
+keepstore now supports the legacy @keepstore.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check the "keepstore config migration notes":https://doc.arvados.org/v2.1/admin/config-migration.html#keepstore and "keepstore install guide":{{site.baseurl}}/install/install-keepstore.html for more details.
h3. Keepproxy configuration migration
-(feature "#14715":https://dev.arvados.org/issues/14715 ) Keepproxy can now be configured using the centralized config at @/etc/arvados/config.yml@. Configuration via individual command line arguments is no longer available and the @DisableGet@, @DisablePut@, and @PIDFile@ configuration options are no longer supported. If you are still using the legacy config and @DisableGet@ or @DisablePut@ are set to true or @PIDFile@ has a value, keepproxy will produce an error and fail to start. Please see "keepproxy's config migration guide":{{site.baseurl}}/admin/config-migration.html#keepproxy for more details.
+(feature "#14715":https://dev.arvados.org/issues/14715 ) Keepproxy can now be configured using the centralized config at @/etc/arvados/config.yml@. Configuration via individual command line arguments is no longer available and the @DisableGet@, @DisablePut@, and @PIDFile@ configuration options are no longer supported. If you are still using the legacy config and @DisableGet@ or @DisablePut@ are set to true or @PIDFile@ has a value, keepproxy will produce an error and fail to start. Please see "keepproxy's config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html#keepproxy for more details.
h3. Delete "keep_services" records
h3. New configuration
-Arvados is migrating to a centralized configuration file for all components. During the migration, legacy configuration files will continue to be loaded. See "Migrating Configuration":config-migration.html for details.
+Arvados is migrating to a centralized configuration file for all components. During the migration, legacy configuration files will continue to be loaded. See "Migrating Configuration":https://doc.arvados.org/v2.1/admin/config-migration.html for details.
h2(#v1_3_3). v1.3.3 (2019-05-14)
# "Run the provision.sh script":#run_provision_script
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
-
-
+# "After the installation":#post_install
h2(#introduction). Introduction
INFO Final process status is success
</code></pre>
</notextile>
+
+h2(#post_install). After the installation
+
+Once the installation is complete, it is recommended to keep a copy of your local configuration files. Committing them to version control is a good idea.
+
+Re-running the Salt-based installer is not recommended for maintaining and upgrading Arvados, please see "Maintenance and upgrading":{{site.baseurl}}/admin/maintenance-and-upgrading.html for more information.
## "DNS configuration (single host / multiple hostnames)":#single_host_multiple_hostnames_dns_configuration
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
+# "After the installation":#post_install
h2(#single_host). Single host install using the provision.sh script
INFO Final process status is success
</code></pre>
</notextile>
+
+h2(#post_install). After the installation
+
+Once the installation is complete, it is recommended to keep a copy of your local configuration files. Committing them to version control is a good idea.
+
+Re-running the Salt-based installer is not recommended for maintaining and upgrading Arvados, please see "Maintenance and upgrading":{{site.baseurl}}/admin/maintenance-and-upgrading.html for more information.
projects / arvados.git / commitdiff